Open benchmark · 5 survey datasets (4 in the Index) · 1,138 questions
· updated Oct 18, 2026
How close can synthetic respondents get to real people?
The gap to real people Each row is one system, sorted by its SynthBench Index. At 0, a system matches people no better than random answers. At 100, it matches them as well as a second sample of real people does. The line through each mark is its 95% interval. ! marks a system whose public and private questions disagree by more than the review threshold. Scores below −40 are drawn as an arrow at the left edge. Tap or click a row for details.
Why scores sit near random
LLMs get the order right and the shares wrong
Random answers spread evenly across the options, which happens to suit questions where people
are split. LLMs put too much weight on the most common answer.
Shares vs. order