▌system-one

~ / evaluation / class-native-suites / decisionrobust

DecisionRobust

Robustness benchmark for the class: 1,058 curated attacks across meaning-preserving text perturbations, prompt injection, image perturbations, in-image instruction overlays, and instruction injection beside images, with ground truth on whether the attack preserves the correct answer. Findings worth the read: resistance to perturbation does not imply resistance to injection, and label-consistency checks substantially shrink measured attack success.

section
Evaluation
group
Class-native suites
source
arxiv.org
added to list
oct 24

also in Class-native suites