SLMB 0.1.0 · Fast 250 · gate GREEN · fp 71f186ff · 250 trials

The measured field · Fast 250 · 01 / 30

The SLMB leaderboard.

Actual local-model strikes. One current, complete Fast 250 run per model. Pass@1 decides the order; grade-weighted pass breaks ties. Every score links back to its run contract and trial evidence.
1 / 30published

Fast 250 · a fixed 250-test run drawn unchanged from Full 2500 — standard reply allowance 3,750 tokens. Full 2500 · the complete 2,500-test run — standard reply allowance published trial caps. Separate leaderboards and evidence; a Fast score is never a Full score or estimate. Provider-time figures reflect each run's recorded conditions — shared-GPU or concurrent timing is not standalone tok/s.

Cohort

Fast 250 · 250 gated trials

Ranking

pass@1 · 250-test run only

Reply budget

standard 3,750 tokens

The podium

1 qualifying run

01 · current leader

pass at first answer

12.0%

lfm2.5-350m-gtap-v3

30/250 first-answer passes · Wilson 9%–17%

More ranks appear when another complete provider strike is synced.

Current · Fast 250

Top 30 results

1 real result qualify today. The board expands as new complete strikes are synced; no scores are simulated to fill the remaining positions.

Rank always follows pass@1 — column headers re-sort the board without changing official position. Tier T1, T2, … groups models whose 95% intervals overlap the tier leader, so positions inside a tier are not statistically separated. Time, tokens, and heap are per-trial means; tok/s is derived. Oracle heap is the checker's memory, not the model's. Wilson uses trial rows; independent answer-shape clusters are shown on each model profile.

Inspect run provenance