System One Bench

Runnable suite with frozen, hash-checked datasets, identical request payloads, released predictions, and accuracy / calibration / cost / latency per system — 16 datasets, 3,250 cases, 4,850 decisions across English, French, and Arabic, with a public board at systemonebench.dev. The methodology is the reason it's here: local timing kept separate from API comparisons, and latency rows excluded when a control model shows a materially different serving environment — discipline the field's incomparable-milliseconds problem needs. Public datasets can overlap training data; hosted-model runs may change.