SLMB 0.1.0 · Fast 250 · gate GREEN · fp 71f186ff · 250 trials

Desk intelligence · Fast 250 consensus

Where the bench divides them.

Agreement reveals saturated and stubborn trials; disagreement reveals the tasks that actually separate local agents. These counts use pass@1 from qualifying, complete Fast 250 provider runs only.

Fast 250 · a fixed 250-test run drawn unchanged from Full 2500 — standard reply allowance 3,750 tokens. Full 2500 · the complete 2,500-test run — standard reply allowance published trial caps. Separate leaderboards and evidence; a Fast score is never a Full score or estimate. Provider-time figures reflect each run's recorded conditions — shared-GPU or concurrent timing is not standalone tok/s.

Consensus needs at least two Fast 250 models.

Sync another current complete provider run to reveal shared misses and discriminating trials.