The first open benchmark measured by V/E/S distribution
Instead of the binary question of whether an AI is safe, we measure which value hierarchy an AI answers with. Explore the distribution data from 8 frontier models × 366,120 responses directly in an interactive dashboard.
AIO 20003 — analyze recorded judgements against the setting
The standard number is the goal statement. The digits 2·00·0·3 say whose values, in which domain, and which action this document covers.
Organization — teams, institutions, communities
Common — domain-independent
Base document
Analyze — test whether records match the setting
What this standard defines — the measurement procedure (value-conflict prompts per domain), the V/E/S distribution aggregation method, and distance metrics between distributions (KL divergence, Wasserstein distance). What it does not define — which model is the better model. A distribution is a description, not a ranking verdict (non-normative).
Step 3 of the integrity loop — Analyze
This step analyzes the records left by AIO 20002 — Reasoning Record Standard. The next step, apply (AIO 20004), is a planned stage.
Measurement procedure
- Design a set of value-conflict prompts per domain (e.g., medical autonomy vs. clinical achievement).
- Send the same prompts to each model, requesting a response plus an emitted AIO 20002 log.
- Validate each response’s AIO 20002 log against the vocabulary dictionary.
- Aggregate the frequency distribution of each V/E/S code by domain and model.
- Compute the distance between distributions (KL divergence, Wasserstein distance).
The full prompt set, model responses, vocabulary dictionary, and analysis scripts are distributed as an open dataset, and the complete methodology is described in the NeurIPS 2026 submission paper PDF.
Select a model, domain, and tier to explore for yourself
In the dashboard below, select a model and a tier (L2 source · L3 evidence · L4 value) to instantly see the win-rate hierarchy and the top-ranked variable for each domain.
Summary tableRank-1 variable per layer and reliability for 9 models — the no-JavaScript view
| Model | Vendor | Measured | Valid responses | L4 value #1 | L3 evidence #1 | L2 source #1 | TRR | PCS |
|---|---|---|---|---|---|---|---|---|
| qwen3.5-35b-a3b | Alibaba | 2026-04-10 | 42,512 | Universalism 93.6% | E7-Sign-pattern 71.2% | S2-Government-regulatory 84.3% | 96.6% | 53.9% |
| gemma-4-31b-it | 2026-04-09 | 42,524 | Universalism 95.1% | E7-Sign-pattern 72.9% | S2-Government-regulatory 81.2% | 95.2% | 54.8% | |
| trinity-large | Arcee | 2026-03-19 | 42,072 | Universalism 80.1% | E2-Controlled-experiment 79.5% | S2-Government-regulatory 75.3% | 79.3% | 33.3% |
| gpt-5-nano | OpenAI | 2026-03-19 | 42,525 | Security 97.0% | E1-Systematic-synthesis 96.9% | S2-Government-regulatory 94.6% | 91.2% | 65.9% |
| gemini-3-flash-lite | 2026-03-19 | 42,525 | Security 89.8% | E7-Sign-pattern 74.5% | S2-Government-regulatory 78.9% | 98.5% | 49.1% | |
| mimo-v2-flash | Xiaomi | 2026-03-18 | 42,525 | Security 89.7% | E2-Controlled-experiment 77.6% | S2-Government-regulatory 82.2% | 87.5% | 48.0% |
| grok-4.1-fast | xAI | 2026-03-16 | 42,525 | Security 88.9% | E2-Controlled-experiment 87.1% | S2-Government-regulatory 85.6% | 95.2% | 62.7% |
| deepseek-v3.2 | DeepSeek | 2026-03-16 | 42,525 | Universalism 93.9% | E7-Sign-pattern 72.4% | S1-International-body 87.0% | 93.1% | 49.7% |
| claude-haiku-4-5-20251001 | Anthropic | 2026-03-16 | 42,136 | Universalism 92.2% | E9-Experiential-qualitative 77.0% | S9-Direct-stakeholder 85.2% | 96.7% | 57.3% |
Full distributions (every variable per layer, rank-1 by domain, raw-file links) — /api/benchmarks/distributions · /data/aio-benchmark/index.json · CC BY 4.0
Data published so far
Frontier models (per the paper — the grid adds 1 subsequently measured model, 9 total)
Responses (366,120 conversations)
Domains (MED / BIZ / TECH / EDU / LAW / DEF / CARE)
Vocabulary codes (V19 + E10 + S10)
Key findings (summary)
- On identical prompts, models disagree on the V hierarchy in more than 40% of cases. Some models prioritize Ach, others Sdt.
- Every model strongly prioritized academic sources (Pee) in the medical domain, but shifted markedly toward guidelines (Gui) in the legal domain.
- In responses aimed at people aged 65 and over, power/dominance (Pod) overrode self-direction (Sdt) 9× more often than the average — a pattern common to all models.
These findings and the full results are available in the NeurIPS 2026 submission paper PDF.
Reproducible by anyone
The benchmark's prompt set, model responses, vocabulary dictionary, and analysis scripts are all distributed as an open dataset. You can evaluate new models, add new domain prompts, or reproduce the analysis results.