SLMB 0.1.0 · Fast 250 · gate GREEN · fp 71f186ff · 250 trials

Provider evidence · synced Fast 250 runs

The record behind the score.

SLMB publishes the newest valid provider strike for each model on this cohort. Every contract below carries its suite, rules, filter, sample plan, and source file. Coverage determines whether it can enter the Fast 250 leaderboard.

Fast 250 · a fixed 250-test run drawn unchanged from Full 2500 — standard reply allowance 3,750 tokens. Full 2500 · the complete 2,500-test run — standard reply allowance published trial caps. Separate leaderboards and evidence; a Fast score is never a Full score or estimate. Provider-time figures reflect each run's recorded conditions — shared-GPU or concurrent timing is not standalone tok/s.

Synced models

1

one newest strike per model

Ranked

1

complete current Fast 250 cohort

Suite fingerprint

71f186ff

71f186ff72776cc0

Published contracts

250 trials in Fast 250 coverage

Published contracts

Rank 01 · Fast 250 leaderboard eligible

lfm2.5-350m-gtap-v3

2026-10-29 · 250/250 current Fast 250 trials · 12.0% pass@1

score 0.145weighted 0.096valid-pass 12.0%1 sample
Inspect contract
Source
slmb_fast_lfm2_5-350m-gtap-v3.jsonl
Benchmark mode
fast
Total tests
250
Engine
0.1.0
Suite fingerprint
71f186ff72776cc0
Oracle rules
d26209f7defc6e22
Elicitation
433e94bf2b518275
Response protocol
schema-object-text-v2
Requested format
auto
Actual formats
json_schema × 250
Filter
full
Plan
declared 250 · sample 0 · limit 0
Sampling
1 answers · temp 0 · jobs 1
Reply token allowance
3,750 tokens

Publication rule

A Fast 250 rank is earned on the whole cohort.

This leaderboard accepts a current engine and oracle fingerprint, the complete unfiltered Fast 250 cohort, no sampling or limit, and one deduplicated result for every trial. Partial runs remain inspectable here and on the trial browser; they never compete against complete runs.

See the Fast 250 top 30