SLMB 0.1.0 · Full 2500 · gate GREEN · fp b8c02463 · 2,500 trials

Head to head · paired Full 2500 evidence

Two models. Same trials.

First-answer outcomes align by trial id. Wins count only where one model passes and the other misses; McNemar tests the discordant pairs. Partial overlap is reported as paired coverage, never hidden in two separate denominators.

Fast 250 · a fixed 250-test run drawn unchanged from Full 2500 — standard reply allowance 3,750 tokens. Full 2500 · the complete 2,500-test run — standard reply allowance published trial caps. Separate leaderboards and evidence; a Fast score is never a Full score or estimate. Provider-time figures reflect each run's recorded conditions — shared-GPU or concurrent timing is not standalone tok/s.

Two synced models unlock comparison.

The first real Full 2500 provider run is visible in Models and Runs. Sync another model to compare paired outcomes.