TypedBench

Academic benchmark for the class from DFKI, RPTU, Imperial, and Mila: seven policy-labelled generators and nine suites scored as median across paraphrases, calibration against the finite-sample noise floor, selective prediction, ordinal proper scoring rules, and realised cost under asymmetric matrices. Headline finding: the hosted model follows stated policy but is wording-sensitive and systematically underconfident, enough that trusting its probabilities can cost more than just taking the top answer.