Evidence

Validation Studies

Side-by-side comparisons of Simsurveys synthetic output against published live panel data. 11 studies across healthcare, consumer, patient, retail, and B2B technology domains, plus data coherence tests of how answers relate within each respondent and a patient digital twin validation.

10

Published studies

5

Research domains

0.04

Median KL divergence (245 questions)

Data Coherence

Data Coherence Test

ACSI Processed Food: Within-Respondent Coherence

Goes inside each respondent rather than comparing totals: do a synthetic respondent's twelve ratings of a food brand relate to each other the way real customers' do? Benchmarked against 1,968 buyers in the publicly released American Customer Satisfaction Index 2015 sample. Synthetic answers reproduce the live pattern of question-to-question links (0.89, against 0.97 between two halves of the live sample), 54 of 66 question pairs fall within 0.10 of the live correlation, and six of nine ACSI model links fall within or at the edge of the live range.

The respondent-level live and synthetic data files are available on request for your own analysis.

Consumer Download PDF
Data Coherence Test

Trust in Named News Brands: Within-Respondent Coherence

With the brand named, do synthetic respondents tell brands apart and link their answers the way real people do? Benchmarked against 12,043 U.S. adults in Pew Research Center's American Trends Panel (Wave 57, 2019), each saying whether they have heard of, trust or distrust 30 named news outlets. Synthetic answers reproduce the live pattern of links between answers (0.89, against 0.99 between two halves of the live sample), trust across the 30 brands correlates with live at 0.91, and 27 of 30 brands split by party in the same direction as live.

The respondent-level live and synthetic data files are available on request for your own analysis.

Consumer Download PDF
Digital Twin Validation

Patient Twins vs. Real Respondents: HINTS 7

Do patient twins answer, and stay consistent, the way the real people behind them do? 200 Simsurveys patient twins, each matched to a real respondent in HINTS 7 (the National Cancer Institute's Health Information National Trends Survey), answered 25 HINTS questions in sequence using the production twin setup: each twin draws on its respondent's real answers to other questions, never the one being asked. Answer totals average a KL divergence of 0.049 (24 of 25 questions at 0.15 or less), the pattern of links between answers matches the real respondents at 0.82, and twins chose exactly the same answer as their real respondent 43% of the time, against about 24% by chance.

Patient / Digital Twins Download PDF

Healthcare

Validation Study

AMA Prior Authorization Survey

Validation of synthetic physician responses against the American Medical Association's prior authorization survey. Compares synthetic and live panel data on physician attitudes, administrative burden, and care delay metrics.

Healthcare Download PDF
Validation Study

Commonwealth Fund / Kaiser 2015

Synthetic replication of the Commonwealth Fund/Kaiser Family Foundation 2015 National Survey of Primary Care Providers. Benchmarks synthetic respondent output against national data on primary-care physicians' practices, satisfaction, payer mix, and views on ACA impact.

Healthcare Download PDF
Validation Study

Physician Sarcopenia Knowledge

Validation of synthetic physician responses on sarcopenia awareness, diagnostic practices, and treatment approaches. Tests the healthcare model's accuracy in replicating specialized clinical knowledge and practice patterns.

Healthcare Download PDF

Consumer & Retail

Validation Study

Consumer Returns in Retail

Validation study comparing synthetic and live consumer data on retail return behaviors, motivations, and satisfaction. Tests the consumer model's ability to replicate purchase and post-purchase decision patterns.

Consumer / Retail Download PDF
Validation Study

IFIC Food & Health Survey

Synthetic replication of the International Food Information Council's annual Food and Health Survey. Validates the consumer model against nationally representative data on food attitudes, nutrition knowledge, and dietary behaviors.

Consumer / Food Download PDF
Validation Study

Walmart Retail Rewired 2025

Synthetic replication of Walmart's Retail Rewired 2025 consumer study. Benchmarks the consumer model against large-scale retail data on shopping behaviors, channel preferences, and emerging retail technology adoption.

Consumer / Retail Download PDF

Patient

Validation Study

KFF GLP-1 Weight Loss Drug Survey

Validation of the patient model against the Kaiser Family Foundation's 2023 survey on GLP-1 weight loss drug awareness, usage, and attitudes. Tests patient model accuracy across demographic segments on a rapidly evolving healthcare topic.

Patient / Healthcare Download PDF
Validation Study

HCAHPS Hospital Patient Experience

Validation against CMS Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey data. Benchmarks the patient model on hospital experience metrics including communication, responsiveness, and discharge information.

Patient / Healthcare Download PDF
Validation Study

US Pain Foundation 2022 Survey

Validation against the US Pain Foundation's 2022 national survey on chronic pain experiences. Tests patient model accuracy on pain management, treatment satisfaction, provider communication, and quality-of-life impact.

Patient / Healthcare Download PDF
Validation Study

Crohn's & Colitis Foundation IBD Healthcare Access Survey

Validation of the patient model against the Crohn's & Colitis Foundation's 2023 Healthcare Access Survey of 2,281 people living with inflammatory bowel disease and their caregivers, published in Inflammatory Bowel Diseases (2024). Covers medication access, insurance approval delays, step therapy and financial burden: 17 of 20 questions at a KL divergence of 0.15 or less, with an average of 0.074.

Patient / Healthcare Download PDF

Business & Technology

Validation Study

UK Cloud Computing Survey

Synthetic replication of Public First's Driving Digital Acceleration research for AWS. Benchmarks synthetic B2B respondents against UK senior business decision-makers on cloud purchase priorities, adoption barriers, and provider relationships.

B2B / Technology Download PDF

How we validate: Every study follows a paired-comparison design using KL-Divergence, Spearman Rank Correlation, Top-K Overlap, and BERTScore. For full methodology details, thresholds, and metric explanations, see our Validation Framework & Benchmarks.

Run your own validation.

Generate synthetic data from your survey and compare against your live benchmarks.

Request the data files

We’ll email you the respondent-level live and synthetic files with a codebook.