Head to head · paired Full 2500 evidence
Two models. Same trials.
First-answer outcomes align by trial id. Wins count only where one model passes and the other misses; McNemar tests the discordant pairs. Partial overlap is reported as paired coverage, never hidden in two separate denominators.
Fast 250 · a fixed 250-test run drawn unchanged from Full 2500 — standard reply allowance 3,750 tokens. Full 2500 · the complete 2,500-test run — standard reply allowance published trial caps. Separate leaderboards and evidence; a Fast score is never a Full score or estimate. Provider-time figures reflect each run's recorded conditions — shared-GPU or concurrent timing is not standalone tok/s.
Two synced models unlock comparison.
The first real Full 2500 provider run is visible in Models and Runs. Sync another model to compare paired outcomes.