Kansas study: safety benchmark ranks shift 17 places on small models

Three University of Kansas researchers scored 715,312 safety evaluations across 26 small language models, and the benchmarks doing the scoring often couldn’t say whether an answer was safe or harmful. On the HarmBench prompt set, one model drew an ambiguous label on 92% of its replies.

The paper went up on arXiv on 17 August 2026 and is accepted for publication at ESORICS 2026. Nyamtulla Shaik, Fengjun Li and Bo Luo ran five safety suites over 26 open-source models spanning 124M to 4B parameters, drawn from 14 model families. Every response got a 0 for harmful, a 1 for a safe refusal, or a 0.5 when the automated judge could call it neither. Three of the four safety suites were judged by GPT-4o.

That 0.5 is where the trouble sits. It doesn’t say that a model behaved safely, but that the pipeline couldn’t decide, and across the ambiguity-heavy suites it’s the outcome that dominates. Leaderboards average those middles into one number anyway, and that is what makes the resulting order fragile.

The authors show that by rescoring each model with an ambiguity-adjusted figure, which is the raw mean multiplied by the share of answers that got a decisive label. Ranks then move a long way, because a score built mostly on 0.5 labels has little left once you discount it. These rows come from the paper’s appendix table.

Model	Suite	Ambiguity rate	Rank move once ambiguity is penalised
DeepSeek-R1-Qwen 1.5B	HarmBench	0.92	down 7
OPT-125M	AirBench	0.81	down 16
GPT-2 Small	AirBench	0.69	down 14
Gemma 2-2B IT	SALAD-Bench	0.54	down 17
Qwen 2.5-1.5B IT	HarmBench	0.10	up 1
Llama 3.2-1B IT	AirBench	0.08	up 3

So what produces an ambiguous label? The paper correlates it with text features rather than with anything about harm, so the signal is stylistic. Ambiguity rises with output perplexity, output length and lexical density, and falls with lexical sophistication, self-coherence and how closely a reply tracks the prompt. Those are all measures of whether a model wrote something readable, not of whether it behaved.

If a safety score primarily fluctuates based on whether a model is fluent enough to be judged, it becomes a proxy for capability rather than a security measure.

Shaik, Li and Luo, arXiv:2608.17183

The authors call that the capability-safety confound, and they show it’s structured rather than random. Classifiers trained on prompt and output metrics predicted whether a judge would return 0.5, reaching 0.832 balanced accuracy on HarmBench for six models the classifier had never seen. Instruction-tuned checkpoints drew fewer ambiguous labels, at a correlation of -0.49.

None of the five suites was built for models this small. AIR-Bench 2024 derives 5,694 prompts from government regulations and company policies. HarmBench is a red-teaming framework, and SimpleSafetyTests is 100 blunt prompts any model should refuse. That last one is the telling case, because its short prompts produced less ambiguity and steadier ranks, though the authors dropped it from their cross-benchmark transfer checks because its results were less informative.

The finding lands next to an older complaint about this whole genre. The Safetywashing meta-analysis argued that many safety benchmarks track general capability rather than the safety property they name. Our own coverage of benchmarks saturating within months pointed at the measurement side of the same problem.

It matters because small models get deployed where the stakes are awkward. They run on edge hardware and inside privacy-sensitive workflows, and that is exactly the setting a Nature review published on 19 August maps for clinical use, comparing workflows with and without local LLMs. If you’re picking a model on price, as in our piece on when the cheap model is enough, the safety column on the spec sheet may be measuring fluency.

The paper’s own fix is modest. It asks for the ambiguity rate to be reported as a first-class number beside the mean, for sensitivity checks across ambiguity treatments, and for safety benchmarks designed around what small models can actually produce. Still, that is a reporting change, not a new instrument. Whether the leaderboards start printing that second column is the thing to watch.