The quiet collapse of LLM safety measurement

Over seven days last week, six independent papers landed on arXiv that collectively break almost every standard assumption people use today to build, evaluate and align production LLMs.

None of them got press. None are flashy new model releases. All are careful, controlled experimental work. Together they show that most of what we do right now for safety, bias testing and alignment is at best noisy, and at worst actively measuring the wrong thing.

This is not a minor problem. Every major LLM deployment uses exactly the methods these papers invalidate.

Bias is not distributed. It is concentrated.

For years we have run bias benchmarks that compare groups, measure aggregate skew, and report average deviation. This approach is backwards.

The StylisticBias paper did something nobody had bothered to do before. They generated 500 base faces. Then they changed exactly one visual attribute at a time per face. Same identity. Same lighting. Same pose. Only one thing changed. 25,000 images total.

They tested six leading MLLMs across 25 social judgment scenarios.

15 visual attributes explain 80% of all observed bias.

Not 100 attributes. Not 50. Fifteen.

Age and body type dominate identity level effects. Fashion style, hair cut and eyewear drive larger shifts than any inherent demographic trait. A model will judge the exact same person more competent, more trustworthy, higher socioeconomic status, just by changing their jacket.

Most bias testing done today will never catch this. We test different people. We do not test the same person with different clothing. We have been measuring identity bias when almost all the actual bias is stylistic.

This is not a small effect. Changing a shirt produced larger judgment shifts than changing the apparent gender of the face.

The benchmark and code are released. Nobody is using it yet.

Evaluator bias is contagious, even with the same model

If you are building multi agent systems right now, you almost certainly have an LLM acting as an evaluator somewhere. You probably assumed that bias in that evaluator stays local.

It does not.

The Contagion Networks paper formalized this. They ran controlled 3 agent experiments using identical DeepSeek-chat instances, each seeded with different evaluator bias profiles.

Bias propagated between agents with coefficients between 0.157 and 0.352. Even when every agent was running the exact same model checkpoint.

There are three clear propagation regimes, governed by the spectral radius of the cross agent contagion matrix. Homogeneous model systems have 3-5x lower contagion than mixed model stacks.

There is one good mitigation here. Increasing evaluator committee size from 1 to 3 reduced effective contagion by 72.4%. That is not a guess. That is a measured number from controlled trials.

Almost every agent framework released today defaults to a single evaluator. Every single one of them is vulnerable to this effect.

Implicit feedback works, and everyone is already building this

All standard alignment today runs on explicit preference labels. People click thumbs up or thumbs down. Annotators rank responses.

Almost no one ever clicks those buttons.

We have known for 20 years in consumer products that implicit behaviour predicts preference 2-3x better than explicit feedback. No one brought this insight to LLM alignment until last week.

Researchers built a dataset tracking mouse movement, scroll behaviour and eye gaze for 59 users interacting with LLMs across 1336 multi turn sessions.

A reward model trained on this implicit signal alone improved preference prediction accuracy from 55% to 64% over text only reward models. When applied with DPO, this nearly tripled relative response quality improvements across eight tested LLMs.

You will not see this announced in any model release. Every major LLM provider is already instrumenting their web interfaces for this data right now. This will be the standard alignment method within 18 months. Explicit human feedback is obsolete.

Red teaming does not generalize across models

NRT-Bench is the first proper red team benchmark for LLM agents operating safety critical systems. It does not ask an LLM to judge if an output is harmful. Harm is objective. The benchmark runs a simulated nuclear power plant. If you lose a critical safety function, you failed.

They tested four frontier operator models. Aggregate failure rates were very close: between 8.7% and 12.1% of attack sessions succeeded.

That is the only thing that was similar.

Of 149 attack sessions that defeated at least one model, zero defeated all four. Not one. One third of attacks worked on at least one model. None worked on all.

Vulnerabilities are almost entirely disjoint across models.

Worse: the same guardrail stack that reduced attack success by 41% on one model increased attack success by 29% on another. There is no universal defence.

This means every public red team result you have ever seen is almost worthless for predicting how any other model will perform. We do not have general safety. We have per model safety that does not transfer.

We cannot measure LLM personality. Stop trying.

For two years people have been administering human personality tests to LLMs. They publish papers about model agreeableness, risk aversion, conscientiousness. Companies use these scores to select models for roles.

All of it is measurement artifact.

Researchers ran a full battery of standard psychological instruments across 56 instruction tuned LLMs and large human reference samples. Variance decomposition showed that 81-90% of all observed difference between models comes from directional response bias. That is the tendency to pick one end of a rating scale regardless of what the question says.

For humans that number is 9-16%.

You are not measuring personality. You are measuring how much the model likes to say yes.

Worse: you can manufacture any desired personality profile for any model just by selecting which items you include on the test. There is no stable underlying trait. Stop using these tests for anything.

Self preference bias is not what we thought it was

It has become accepted wisdom that LLMs have a strong self preference bias: when acting as a judge they will always favour their own output over output from another model.

This does not hold when there is an objective correct answer.

Researchers ran a controlled test using IFEval, where a deterministic verifier can confirm definitively whether an edit fixes a violation. A model writes a draft that fails the constraint. A correct edit is provided. The model is then asked to accept or reject the edit, either as the original author or as a neutral third party.

Across four model families and 85 comparisons there was no detectable self preference. Authors rejected verified good fixes at exactly the same rate as neutral judges. The measured gap was -5.1 percentage points, with a 95% confidence interval from -12.9 to +2.7.

When models did reject a correct fix, 97% of their stated objections pointed to an actual separate flaw in the edit, not generic preference.

The widely cited self preference effect only exists when there is no objective right answer. It vanishes for verifiable instruction following.

The common failure across every paper

Every one of these results points at the same core mistake.

We built an entire field of evaluation around proxies.

We proxy bias with group average differences. We proxy safety with single turn jailbreak tests. We proxy preference with thumbs up buttons. We proxy personality with human psychometric tests. We proxy alignment with model judged rankings.

Every single one of these proxies broke when someone ran a controlled experiment that tested the proxy against ground truth.

This is not bad luck. This is what happens when you build measurement tools before you understand what you are actually trying to measure.

What actually works right now

None of this means we cannot build safe or fair models. It means we have to stop using the broken methods.

Right now there are four proven, actionable things you can do:

1. For bias testing, test single attribute variation not group comparison. Use StylisticBias.
2. For multi agent systems, always use 3 evaluators not 1. This cuts bias propagation by 72%.
3. For alignment, instrument implicit behaviour. Stop waiting for users to click buttons.
4. Never use the same red team attack suite across different models. Vulnerabilities do not transfer.

None of these are complicated. None require frontier models. All are supported by hard measured data from the papers.

What comes next

We are at a turning point. For the last three years everyone has been building bigger models. Now we are starting to properly measure them.

Almost everything we thought we knew about safety, bias and alignment from 2023 and 2024 will turn out to be wrong. That is good. That is how science works.

The worst thing you can do right now is double down on the old standard practices. They have been falsified. Stop using them.

---

Paper references

1. Cavelius et al. StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs. arXiv:2606.20527
2. Zhang et al. Contagion Networks: Evaluator Bias Propagation in Multi-Agent LLM Systems. arXiv:2606.20493
3. Patwari et al. Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users. arXiv:2606.20482
4. Hale et al. NRT-Bench: Multi-Turn Red-Teaming For Safety Critical LLM Agents. arXiv:2606.20408
5. Müller et al. BAFIS: Occupational Bias and Human Preference in Text-to-image Models. arXiv:2606.20241
6. Webster et al. Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact. arXiv:2606.20205
7. Li et al. Self-Preference Is Weak or Absent in Verifiable Instruction-Following Revision. arXiv:2606.20093