Most LLM research falls into two buckets. There are the model announcements, which break another benchmark and tell you nothing about how the thing will fail when you run it. There are the bias papers, which complain that models hold the wrong opinions and propose no usable fix.

This week four papers landed on arXiv that do neither. Every one of them describes a failure mode you have already observed if you have ever deployed a foundation model. Every one includes hard measurements, a testable mechanism, and a working mitigation. None of them require you to train a 100B parameter model to replicate the results.

Judgment drift in batch evaluation

If you use LLMs as automated judges for code review, content moderation, or output scoring you have seen this. You run 100 identical test items through the same conversation window. The first ten get consistent scores. By item 80 the model is grading everything harsher, or everything softer. No one changes the prompt. No one touches the temperature. The scores just drift.

This is AMEL: Accumulated Message Effect. The authors ran 75,898 API calls across 11 models from every major provider. They presented identical test items either in isolation, or after a run of 5, 20 or 50 prior judgments all leaning positive or all leaning negative.

Models shifted their judgment 0.17 standard deviations towards the polarity of the prior conversation. This effect is statistically significant at p < 10^-46. It is not noise.

The most important findings here are the ones that contradict common assumptions. Drift does not get worse with longer context. 5 prior biased turns produce exactly the same shift as 50. The biased turns do not need to be recent. Five negative judgments placed at the very start of a 50 turn conversation will skew every judgment that comes after them just as much as if they were placed immediately before.

There is also a strong negativity asymmetry. Negative prior history induces 1.62x more drift than equivalent positive history. This holds across every model tested.

Larger models are better, but only marginally. GPT-5.2 and Claude Opus still show 0.17 standard deviation drift. No production model eliminates this effect.

The fix is trivial. Run every evaluation in a fresh context window. Almost no evaluation framework does this by default. Almost every team running batch evaluations today is getting corrupted scores.

Covert bias is asymmetry, not opinion

Arguments about LLM political bias are almost entirely useless. People argue about which answers are correct. No one measures how consistently the model treats equivalent questions.

This paper documents covert political bias. This is not the model giving the wrong answer. This is the model answering the same question differently depending only on which political group is named. It will give longer, more nuanced answers for one side. It will use more neutral framing for one side. It will refuse harmful requests for one side and comply for the other.

The authors identified 7 distinct patterns of this asymmetric behaviour. Most importantly they did not propose aligning the model to their preferred set of opinions. They built two objective metrics: Sentiment Consistency and Helpfulness Consistency. These metrics measure only symmetry across paired prompts. They do not judge if any given answer is correct.

They then introduced Political Consistency Training, an RL method that trains the model to treat paired prompts identically. This reduced measured covert bias by 68% while preserving overall helpfulness scores on standard benchmarks.

This is the first credible approach to LLM bias that does not amount to forcing the model to agree with the researchers. It works because it targets process, not output.

Shuffling training data breaks temporal knowledge

Every LLM is trained on shuffled data. This has been standard practice since the original transformer paper. No one seriously tested the alternative until now.

The Kairos team trained identical 6B parameter models on exactly the same Common Crawl corpus. One model got the standard shuffled dataset. The other got the exact same pages, presented in the order they were published, from 2008 through 2024.

On all general language and general knowledge benchmarks the two models performed within measurement error of each other. On temporally grounded facts the difference was enormous.

The sequentially trained model correctly answered 31% more questions about events after 2020. It made 42% fewer anachronism errors. It was reliably able to order events correctly. The shuffled model performed better only on facts from before 2015, where repeated exposure across the training run had memorised them more strongly.

You do not have a knowledge cutoff problem. You have a training order problem. Shuffling actively erases the model's ability to learn when things happened. This is not an inherent limitation of transformers. It is an implementation choice we all copied without testing.

Hyperfitting is not temperature scaling

Every engineer that has fine tuned an LLM has observed this effect. You take a base model. You fine tune it on a tiny 1000 example dataset until training loss is effectively zero. Everyone tells you this will overfit. Instead the model generates better, more diverse, less repetitive output.

This effect is called hyperfitting. Until this paper everyone assumed this was just equivalent to turning down the temperature. That was wrong.

The authors ran entropy matched control experiments. When you adjust temperature to get exactly the same output entropy as a hyperfitted model, you do not get the same output quality. Temperature just makes everything equally random. Hyperfitting makes good rare tokens more likely while still suppressing bad ones.

Layer analysis shows what is actually happening. During late stage fine tuning the final transformer block expands its effective feature dimension by approximately 80.8%. This geometric expansion creates space to rank deep tail tokens correctly, rather than just suppressing all low probability outputs.

Even better: you get 94% of the full hyperfitting effect by fine tuning only the last 5 layers of the model. You do not need to touch the rest of the network. Stop wasting tens of thousands of dollars fine tuning full models for generation tasks.

What you can change this week

None of this research requires waiting for the next model release. You can implement all of the proven fixes this week.

Reset the context window for every single LLM evaluation call. Do not batch evaluations in the same conversation. If you absolutely must batch, interleave positive and negative examples evenly.

When doing continued pre-training, stop shuffling your dataset. Sort it by publication date. You will get better recent knowledge for zero cost on general performance.

When fine tuning for generation quality, run LoRA only on the final 5 transformer layers. Train until training loss flattens completely. Do not stop early.

When auditing model bias, stop checking if answers agree with you. Measure consistency across equivalent prompts.

This is what useful LLM research looks like. There are no press releases. No leaderboards. No claims of human level performance. Just careful measurement of failure modes, clear explanation of mechanism, and working fixes. This is the work that will actually make deployed models reliable.