Capability ladder methodology

Every model page shows one capability number: a ladder score out of 220. This page explains what it is, whose measurements it rests on, and what it deliberately does not do.

Each model's number links to its own breakdown page (e.g. /models/<model>/capability/), which lists the points it earned on each of the 22 rungs and marks which were measured and which were fitted.

Source and attribution. The benchmark scores underneath this number are per-benchmark results published on Artificial Analysis's public model pages. Permission to use this data was requested from Artificial Analysis on 2026-09-05. Artificial Analysis has not endorsed this site or this method — the ladder is this site's own computation over their published numbers, not a number Artificial Analysis itself publishes or has reviewed.

What the number is

The ladder is cumulative and open-ended. Every public benchmark this pipeline tracks is a rung, worth a fixed 10 points times the model's chance-corrected score on it (a model at the multiple-choice guessing floor scores 0 on that rung, not a share of 10). A model's ladder score is the sum of its points across every rung:

ladder score = Σ (10 × chance-corrected score) over all rungs

Rungs are never retired and never re-normalised. A new, harder benchmark becomes a new rung worth its own 10 points; points already earned on older rungs don't move. That's the property this design optimizes for: a 2023 model and a 2026 model sit on the same ruler, and neither model's score changes when the frontier moves. The scale has no fixed ceiling — 22 rungs today, for a maximum of 220, growing by 10 with every new rung.

The rungs

"Models measured" counts every Artificial Analysis row with a published result on that rung as of the 2026-09-29 capture (all effort variants and providers), not only the models on this site. A rung with few measured models still has a fitted difficulty, but that difficulty is less certain.

When rungs are added

A rung is added when Artificial Analysis starts publishing a new accuracy-shaped benchmark at frontier scale. Elo-style scores (GDPval, the AA-Briefcase Elo sub-metrics) and composite indices are not rungs: they have no chance floor and no ceiling, so they cannot be worth "10 points times the score". Adding a rung raises every model's maximum by 10 and re-runs the item-response fit, so every score moves a little; the site records each change here.

• 2026-09-22 — three rungs added: AA-Briefcase (rubric pass-rate), AutomationBench, MLCR (19 → 22 rungs, max 190 → 220). Artificial Analysis retired GPQA Diamond from its own index on 2026-09-04 and stopped publishing Terminal-Bench 2.1 results for new models, while these three had been live since Intelligence Index v4.2 and were being ignored by the pipeline. Without them, every model released from September 2026 onward would be fitted on the discriminating rungs that older models were measured on. The case that exposed it: Grok 4.7 sat below Grok 4.6 (137.8 vs 139.4) on the old rung set despite beating it on every benchmark the two shared and on Artificial Analysis's own index; with the three rungs it sits above (155.7 vs 153.2), by a margin the fit cannot certify. Across the 139 scored models on this site the change moved scores by a median of +3.0 points (range -2.2 to +17.9); 12 models moved five or more rank places, and the top three are unchanged.
• 2026-09-29 — data refresh, no rung change. Eight models gained a score as benchmark results were published for them: Claude Opus 5.5 (163.5), Claude Sonnet 5.5 (156.9), GPT-6 Sol (155.0), MiMo-V2.6-Flash (149.0), GPT-6 Luna (142.2), Qwen3.8-27B (133.6), Mercury 2.5 (110.6) and Mistral NeMo (27.2, an alias this site had missed). The 127 already-scored models that moved did so by a median of 0.6 points. One moved for a different reason: GLM-5.3 went from 154.3 to 142.6 because a low-effort row was published alongside its max-effort row, and with two rows the median-effort rule takes the lower. Its max-effort result did not change.

Why this replaces the old Intelligence Index number

An earlier version of this site displayed Artificial Analysis's own Intelligence Index. That index is re-normalised to 0–100 on every version release, and old benchmarks are dropped once the frontier saturates them — which erases exactly the benchmarks that used to distinguish older, weaker models from each other, the models this corpus spends the most effort characterizing. Comparable numbers only exist within one index version. The ladder is built from the same publisher's underlying per-benchmark results, but sums them cumulatively instead of re-normalising, so a model's score is stable across time and comparable across the whole corpus, from 2023's smallest open-weight releases to this week's frontier.

How missing cells are handled

Not every model has been run on every benchmark. This pipeline treats that as an item-response measurement problem rather than a gap: each benchmark has an estimated difficulty, each model has one latent ability fit from every cell that does exist, and a missing cell is filled in as the score that ability predicts at that difficulty. An old, easy benchmark a model was never run on fills in near-ceiling; a new, hard one fills in near zero. Both are estimates, and the page says so — each model's card shows "measured k of n rungs", so you can see how much of the score is an observed result versus a fitted one.

As an independent check on that fit, the same method was also run over Epoch AI's Capabilities Index data (CC BY 4.0) — a different publisher's benchmark measurements — and the two fits place shared rungs at closely matching difficulties, which is evidence the underlying item-response model is measuring something real rather than overfitting to one publisher's quirks.

Why the median-effort row

Where a lab ships a model at several selectable reasoning-effort levels, the ladder uses the row for its median thinking budget, not its highest. The personality samples this corpus analyzes were collected at each model's default effort, not maximum reasoning spend, so the capability number sits next to the personality data it contextualizes. And "what this model can do at maximum token spend" is a different, narrower question than "what this model is like to use," which is the question the rest of the site asks.

The 3-rung floor

A model with fewer than 3 measured rungs gets no fitted ability estimate at all — there isn't enough signal to tell "no data" apart from "a real, extreme ability" — so it shows as not yet scored rather than a number that would look precise but isn't. This mostly affects small, early open-weight 2023 releases that few benchmark trackers ever ran.

Two sources, for now

Not every model on this site has enough published Artificial Analysis data to clear the 3-rung floor. Where that's the case, the card falls back to the last Artificial Analysis Intelligence Index (AAII) figure this site recorded for it, labelled with its index version (e.g. "AAII 46 · v4.1.1") so it's never mistaken for a ladder score. The two numbers are never combined onto one scale: AAII versions aren't comparable with each other, and no AAII figure is comparable with the ladder, so a card shows one or the other, never both, and sorting always ranks every ladder- scored model above every AAII-fallback model above every unscored one. Expect the fallback to shrink over time as more benchmark history becomes available for these models.

Limitations

• Fitted-heavy for very old models. The 2023 tail (6B–7B chat models, early GPT-3.5-era releases) has only a handful of measured rungs; most of the score is extrapolation. "Measured k of n" is there so that's never hidden.
• Evaluator lag for very new models. A model released within the last few days may show fewer measured rungs than it eventually will, or may not clear the 3-rung floor yet.
• No benchmark, no number. A handful of small open-weight releases with no public Artificial Analysis page show "not yet scored" (or the AAII fallback, above) rather than a hidden zero.

Data and code

The full scoring pipeline (chance-correction, the item-response fit, the rung list, per-model provenance) lives at model-capability.danieltenner.com (may not be live yet; the link stays here regardless). The alias map from this corpus's model slugs to Artificial Analysis's model identifiers, and the design note this scheme was built from, are in this repository at internal/capability-ladder/DESIGN.md.