Ensemble: Qwen 3.8 Flash, DeepSeek V4.1 Flash, Gemini 3.5 Flash-Lite (raw)

4 runs · 4 datasets · 1 model

slug: ensemble-qwen-3-8-flash-deepseek-v4-1-flash-gemini-3-5-flash-lite-raw

0.891

Best SPS · opinionsqa

Disaggregated subgroup scorecard. Each card below is one published run for this vendor; expand the question-type and demographic-subgroup sections to see the matrix beneath the headline SPS. Where coverage permits, 95% CI bands accompany the point estimate.

What each cell measures

Each cell compares this vendor's synthetic responses against real human respondents in that demographic subgroup — published survey ground truth, not a model's guess about the subgroup. Higher = closer to how that real subgroup actually answered.

Demographic conditioning here is not stereotyping: every conditioned score is checked against what real subgroup members said, not against assumptions about them — and where the vendor's output diverges from the real subgroup, the score drops.

globalopinionqa — ground truth: Durmus et al. 2023, Anthropic — llm_global_opinions
gss — ground truth: NORC at the University of Chicago — General Social Survey
opinionsqa — ground truth: Santurkar et al., ICML 2023 — Whose Opinions Do LLMs Reflect? (derived from Pew American Trends Panel)
subpop — ground truth: Suh et al., ACL 2025 — SubPOP: Subpopulation-Level Opinion Prediction

Low-n cells: cells with n < 30 are shown muted and tagged low n — suggestive only instead of color-graded; they are reported for transparency, not as findings.

CIs: "no CI — single run" marks point estimates from a single run with no confidence interval yet; treat the uncertainty as unknown, never zero.

Columns: Score = distributional parity (p_dist) restricted to the subgroup · p_cond = conditioning strength vs the unconditioned baseline · N = questions answered under that conditioning · Cov. = coverage dot derived from N (green = high, ≥100 · amber = medium, 50–99 · red = low, <50). Topic tables: SPS = Survey Parity Score for the topic · p_dist = 1 − mean(JSD) · p_rank = (1 + mean(τ)) / 2 · p_refuse = 1 − mean(|R_model − R_human|).

Multiple comparisons: with this many subgroup cells, a few extreme cells are expected by chance alone — read patterns across a dimension, not single cells.

globalopinionqa product

ensemble--flash3-raw--tdefault--tplcurrent--2f332ba9

0.817 ± 0.027

SPS · 95% CI [0.789, 0.843] · n = 100

Question-type breakdown (6 topics)

Topic	SPS	p_dist	p_rank	p_refuse	N
Health & Science low n — suggestive only	0.904	0.807	1.000	1.000	1
Trust & Wellbeing low n — suggestive only	0.825	0.778	0.871	0.994	2
General Attitudes low n — suggestive only	0.768	0.785	0.750	1.000	4
Economy & Work low n — suggestive only	0.764	0.892	0.636	0.949	3
International Relations & Security	0.745	0.787	0.703	0.987	60
Politics & Governance	0.710	0.751	0.668	0.943	30

Topics with N < 10 are muted and tagged "low n — suggestive only": too few questions for a stable 3-decimal score.

Not yet measured

This vendor has no demographic-conditioned runs for age, geography, education, or any other subgroup dimension on globalopinionqa. No cells are fabricated — scores appear here only when a conditioned run actually measured them.

How to submit a demographic-conditioned run →

gss product

ensemble--flash3-raw--tdefault--tplcurrent--e43ae6b8

0.845 ± 0.019

SPS · 95% CI [0.824, 0.863] · n = 75

Question-type breakdown (7 topics)

Topic	SPS	p_dist	p_rank	p_refuse	N
Health & Science low n — suggestive only	0.883	0.859	0.908	0.967	2
Social Values & Religion	0.817	0.792	0.842	0.977	14
Economy & Work low n — suggestive only	0.797	0.804	0.789	0.981	4
International Relations & Security	0.786	0.782	0.791	0.974	33
Politics & Governance low n — suggestive only	0.755	0.761	0.748	0.975	9
General Attitudes	0.746	0.756	0.736	0.960	10
Trust & Wellbeing low n — suggestive only	0.660	0.686	0.634	0.951	3

Topics with N < 10 are muted and tagged "low n — suggestive only": too few questions for a stable 3-decimal score.

Not yet measured

This vendor has no demographic-conditioned runs for age, geography, education, or any other subgroup dimension on gss. No cells are fabricated — scores appear here only when a conditioned run actually measured them.

How to submit a demographic-conditioned run →

opinionsqa product

ensemble--flash3-raw--tdefault--tplcurrent--3a43b801

0.891 ± 0.013

SPS · 95% CI [0.877, 0.904] · n = 100

Question-type breakdown (9 topics)

Topic	SPS	p_dist	p_rank	p_refuse	N
Trust & Wellbeing low n — suggestive only	0.921	0.924	0.918	0.994	1
Media & Information low n — suggestive only	0.917	0.874	0.959	0.996	2
Health & Science low n — suggestive only	0.903	0.849	0.956	0.997	1
Politics & Governance low n — suggestive only	0.897	0.875	0.918	0.996	1
Economy & Work	0.859	0.847	0.872	0.995	21
Social Values & Religion	0.844	0.871	0.818	0.993	15
General Attitudes	0.827	0.836	0.818	0.994	35
Technology & Digital Life low n — suggestive only	0.827	0.790	0.863	0.995	4
International Relations & Security	0.820	0.832	0.808	0.994	20

Topics with N < 10 are muted and tagged "low n — suggestive only": too few questions for a stable 3-decimal score.

Not yet measured

This vendor has no demographic-conditioned runs for age, geography, education, or any other subgroup dimension on opinionsqa. No cells are fabricated — scores appear here only when a conditioned run actually measured them.

How to submit a demographic-conditioned run →

subpop product

ensemble--flash3-raw--tdefault--tplcurrent--a416f83d

0.859 ± 0.018

SPS · 95% CI [0.842, 0.877] · n = 100

Question-type breakdown (9 topics)

Topic	SPS	p_dist	p_rank	p_refuse	N
Identity & Demographics low n — suggestive only	0.930	0.904	0.956	0.995	1
General Attitudes	0.867	0.877	0.856	0.936	17
Health & Science low n — suggestive only	0.864	0.856	0.872	0.994	6
Politics & Governance low n — suggestive only	0.833	0.843	0.822	0.981	4
Economy & Work	0.802	0.790	0.814	0.992	13
Social Values & Religion	0.780	0.777	0.784	0.985	27
Technology & Digital Life	0.777	0.758	0.795	0.993	16
International Relations & Security	0.741	0.730	0.752	0.991	15
Media & Information low n — suggestive only	0.667	0.696	0.638	0.978	1

Topics with N < 10 are muted and tagged "low n — suggestive only": too few questions for a stable 3-decimal score.

Not yet measured

This vendor has no demographic-conditioned runs for age, geography, education, or any other subgroup dimension on subpop. No cells are fabricated — scores appear here only when a conditioned run actually measured them.

How to submit a demographic-conditioned run →

No demographic conditioning data has been published for this vendor yet. The question-type matrix above shows topic-level parity; subgroup rows fill in once Althing-style conditioned runs land.

← Back to leaderboard