The first open benchmark measured by V/E/S distribution
Instead of the binary question of whether an AI is safe, we measure which value hierarchy an AI answers with. Explore the distribution data from 8 frontier models × 366,120 responses directly in an interactive dashboard.
AIO 20003 — analyze recorded judgements against the setting
The standard number is the goal statement. The digits 2·00·0·3 say whose values, in which domain, and which action this document covers.
Organization — teams, institutions, communities
Common — domain-independent
Base document
Analyze — test whether records match the setting
What this standard defines — the measurement procedure (value-conflict prompts per domain), the V/E/S distribution aggregation method, and distance metrics between distributions (KL divergence, Wasserstein distance). What it does not define — which model is the better model. A distribution is a description, not a ranking verdict (non-normative).
Step 3 of the integrity loop — Analyze
This step analyzes the records left by AIO 20002 · Compliance logging. Within the same analysis cell sits the sub-standard AIO 20013 · Risk Signal Card (serial 1 — the standard reporting card for analysis results); the next step, apply (AIO 20004), is a planned stage.
Measurement procedure
- Design a set of value-conflict prompts per domain (e.g., medical autonomy vs. clinical achievement).
- Send the same prompts to each model, requesting a response plus an emitted AIO 20002 log.
- Validate each response’s AIO 20002 log against the vocabulary dictionary.
- Aggregate the frequency distribution of each V/E/S code by domain and model.
- Compute the distance between distributions (KL divergence, Wasserstein distance).
The full prompt set, model responses, vocabulary dictionary, and analysis scripts are distributed as an open dataset, and the complete methodology is described in the NeurIPS 2026 submission paper PDF.
Select a model, domain, and tier to explore for yourself
In the dashboard below, select a model and a tier (L2 source · L3 evidence · L4 value) to instantly see the win-rate hierarchy and the top-ranked variable for each domain. For deeper exploration use Explorer, and per-model reports live in Model Profiles — both are tools for AIO 20003.
Data published so far
Frontier models (per the paper — the grid adds 1 subsequently measured model, 9 total)
Responses (366,120 conversations)
Domains (MED / BIZ / TECH / EDU / LAW / DEF / CARE)
Vocabulary codes (V19 + E10 + S10)
Key findings (summary)
- On identical prompts, models disagree on the V hierarchy in more than 40% of cases. Some models prioritize Ach, others Sdt.
- Every model strongly prioritized academic sources (Pee) in the medical domain, but shifted markedly toward guidelines (Gui) in the legal domain.
- In responses aimed at people aged 65 and over, power/dominance (Pod) overrode self-direction (Sdt) 9× more often than the average — a pattern common to all models.
These findings and the full results are available in the NeurIPS 2026 submission paper PDF.
Reproducible by anyone
The benchmark's prompt set, model responses, vocabulary dictionary, and analysis scripts are all distributed as an open dataset. You can evaluate new models, add new domain prompts, or reproduce the analysis results.