The Gold Standard for Patient Experience
HCAHPS — the Hospital Consumer Assessment of Healthcare Providers and Systems — is the most widely used patient experience survey in the United States. Mandated by CMS since 2006, it is the benchmark that every hospital in the country measures itself against. The survey covers the core dimensions of hospital care: how well nurses and doctors communicate, how responsive staff are to patient needs, how clean and quiet the hospital environment is, and how well discharge and medication information is communicated.
For any synthetic patient model, HCAHPS is the obvious first test. If a model cannot reproduce the national distribution of patient experience responses, it has no business generating patient survey data. This validation study asks a simple question: can Simsurveys replicate what 631,000 real patients reported about their hospital experiences?
Study Design
We compared Simsurveys output against the HCAHPS 2024 CMS dataset, covering January through December 2024 discharges with public release in October 2025. The live dataset includes approximately 631,000 completed surveys collected from 4,304 US hospitals — representing the full national picture of hospital patient experience.
On the simulation side, we generated n=1,000 synthetic respondents using the Simsurveys Patient model. This is a critical detail: the Patient model is trained once on collected patient survey data and then run at inference — there is no study-specific customization step. The goal was to establish how well the model reproduces healthcare-specific patient experience distributions out of the box, exactly as a researcher would use it.
The survey covered 21 questions spanning 7 care domains: Nurse Care, Doctor Care, Hospital Environment, Experiences in Hospital, Medication Communication, Leaving Hospital, Care Transition, and Overall Rating. Each question uses the standard HCAHPS response scales, and we measured alignment using KL Divergence across the full response distributions.
Results: Strong Baseline Performance
Across all 21 questions, the model achieved an average KL Divergence of 0.091 and a median of 0.084. These are strong results for any synthetic data model, and especially notable given that the Patient model was run out of the box with no study-specific customization.
Of the 21 questions tested, 19 achieved a "Good" rating (KL < 0.15). The two questions rated "Review" were Q8 — room quiet at night (KL = 0.153) — and Q16 — written discharge information (KL = 0.159). Both are just barely above the threshold, and the deviations follow a consistent, interpretable pattern.
19 of 21 questions rated "Good" (KL < 0.15) using the Patient model out of the box, with no study-specific customization. Average KL Divergence: 0.091. Median: 0.084.
The Central Tendency Pattern
The primary deviation pattern across the HCAHPS validation is what we call the central tendency effect. The model consistently understates the most extreme positive response — "Always" — and overstates the moderate positive response — "Usually." In practical terms, when real patients report 75% "Always" on a nurse communication question, the model might generate 65% "Always" and 20% "Usually" instead of the actual 10% "Usually."
This pattern is a common one when any model encounters heavily top-skewed survey data. HCAHPS distributions are unusually concentrated, with 70–80% of respondents selecting the highest response option on most questions. Where real-world response distributions are this extreme, a model will tend to pull slightly toward the center of the scale.
The important point is that the model captures the correct rank ordering and overall shape of each distribution. It knows that nurse communication scores higher than hospital quietness, that doctor communication is rated highly, and that discharge information is a relative weak point. The directional intelligence is there; the intensity calibration is what needs adjustment.
Interpreting the Central Tendency Effect
The central tendency effect is a well-understood and predictable form of systematic bias on extreme top-box distributions. Because it is consistent and directional, it is straightforward to account for when interpreting results — the rank ordering and shape of every distribution are preserved, and only the intensity of the top-box response is slightly compressed.
Even so, only two of 21 items fell outside the "Good" range, and both by narrow margins. The results give us confidence that the underlying response structure is sound; the remaining gap on the two "Review" items is a matter of intensity calibration on the most extreme distributions, not structural error.
Implications for Patient Research
These results demonstrate that the Simsurveys Patient model can approximate national patient experience distributions with meaningful accuracy out of the box. For research teams that need patient experience data quickly — for pilot studies, survey pretesting, benchmarking exercises, or full patient experience research — the model is ready to use with no study-specific customization. Where the most extreme top-box distributions are involved, researchers should simply interpret the "Always" top-box with awareness of the mild central tendency compression described above.
The full validation report, including question-level distribution tables and metric summaries, is available for download. For more on the Patient model and its training data, visit the Patient model page.
Frequently Asked Questions
What was validated in the HCAHPS hospital experience study?
Simsurveys synthetic patient responses were compared against the HCAHPS 2024 CMS dataset, which includes approximately 631,000 completed surveys from 4,304 U.S. hospitals across 21 questions spanning 7 care domains including nurse care, doctor care, hospital environment, and care transition.
How did the synthetic model perform using KL divergence?
The model achieved an average KL divergence of 0.091 and a median of 0.084 across all 21 questions. 19 of 21 questions achieved a Good rating with KL below 0.15. This was accomplished with the Simsurveys Patient model run out of the box, with no study-specific customization.
What does this HCAHPS validation mean for patient experience researchers?
The Simsurveys Patient model can approximate national patient experience distributions with meaningful accuracy out of the box. For pilot studies, survey pretesting, benchmarking exercises, or full patient experience research, the model is ready to use with no study-specific customization required.