Leaderboard
Systems are grouped into tiers. Every system in a tier has a 95% interval that overlaps the interval of the tier's top system, so it can't be reliably told apart from it.
How to read this table
Tiers
Reading down the table, the top system not yet placed starts a tier. Every system below it whose 95% interval still overlaps that system's interval joins the tier. The first system whose interval sits entirely below starts the next one. Tiers are numbered 1, 2, 3… with no gaps, and tiers are recomputed for the systems currently shown. A difference of a few points inside a tier is noise.Reference rows
Striped rows are reference points, not systems: random answers (Index 0) and always choosing the most popular answer.Compare
Tick up to three rows to compare them side by side. Click a row for details. Tap a row for its details, then Add to compare. Pick up to three to see them side by side. NTIA (†) is shown for reference and isn't part of the Index: its questions are yes/no facts about internet use, not opinions.Reproduce these numbers
Every number on this page is recomputed from per-question data when the site is built. The scoring code and public data are open, so you can re-run any of it:
pip install "synthbench-eval[subpop] @ git+https://github.com/DataViking-Tech/SynthBench.git"
synthbench run --provider <yours> --dataset subpop --samples 30
synthbench validate results/<your-run>.json
synthbench compare results/<your-run>.json <any-published-run>.json run scores your provider on the same public questions. validate checks the
result file against the integrity rules. compare runs a paired bootstrap test against any
published run. See the API contract
for the JSON format.
Scores by demographic group
Which groups each system matches best and worst: political ideology, party, education, income, age, race, religion and country, with reference points and 95% intervals.