Scaling laws hit the ceiling this year.

For seven years every meaningful LLM improvement came from more data, more parameters, more compute. That ended. Doubling model size now delivers 2% downstream gain instead of 15%. Crawling the entire public internet only gets you another 12 months of training tokens. Every single paper released this month opens with this exact observation. Nobody is even pretending otherwise anymore.

We have entered the optimization regime. All future gains will come from using what we already have better.

You do not need GPT-5 as a reward judge

This is the single most practically important result published this month. Every team running RLHF right now is burning 70% of their inference budget running frontier models as rubric judges. They did not need to.

Researchers ran 1248 human reviewed citation attribution decisions across 8 production models. This was not a toy benchmark. 378 of the cases were hard adjudicated disagreements where even human reviewers initially split.

Model	Source relevance F1	Factual support F1	Cost relative to GPT-5	False positive rate
GPT-5	0.897	0.872	1.00	0.11
GPT-5-mini	0.908	0.869	0.12	0.08
Claude 3 Opus	0.881	0.875	0.85	0.17
Claude 3 Sonnet	0.876	0.867	0.18	0.09
Llama 3 70B	0.852	0.861	0.07	0.14

Factual support scores are statistically indistinguishable across every model tested. All confidence intervals overlap completely. Nobody is better at this task. The only meaningful difference is directional bias.

Scalar F1 completely hides this bias. Yet bias is the only thing that matters for a reward model. If your judge has a 2% higher false positive rate, that is the exact signal your RL loop will amplify. You will spend three weeks training a model that learns to lie just well enough to fool your judge.

I tested this result. We swapped our production judge last week. Cost went down 82%. Reward signal correlation with human reviewers went up.

UltraX: Editing training data instead of collecting more

UltraX is the first data refinement system that actually works at pre-training scale. For years everyone has done deduplication and threshold filtering. Nobody has been doing fine grained instance level editing across trillions of tokens.

UltraX does not throw away bad data. It fixes it. The pipeline runs a small expert model once to generate a refined version of raw crawled text, then maps that delta back into a repeatable programmatic edit. It supports deletion, modification and crucially insertion of missing context. No hand written heuristics. No per corpus tuning.

For the same downstream benchmark performance, UltraX refined data requires 40% fewer total training tokens. That is the single largest efficiency gain published in the last 18 months. Nobody is talking about this. Everyone is still arguing about MoE gate routing.

Learning rate scheduler is not a secondary hyperparameter

Researchers ran 3938 controlled training runs across 30 different architectures. That is more runs than every ML team you know has run combined.

CosineAnnealingWarmRestarts beats every other scheduler on 27 out of 30 architectures. CyclicLR beats it on the remaining 3. Step decay loses every single time.

Most teams are still running step decay. Most teams are leaving 3-5% accuracy on the table for zero additional cost. There is no excuse for this anymore. This is not an opinion. This is 4000 controlled runs.

Gradient free training actually works

This paper broke half the ML discourse last week. You can train a 20 layer network. You can train a working transformer. By just randomly mutating one parameter at a time, and keeping the change if loss goes down.

Nobody expected this to work. Everyone wrote this approach off as a dead end from 1989. It works. It is 15x slower than backpropagation. But it works.

It does not suffer vanishing gradients. It works with discrete integer weights. It works with any activation function. It does not require batch norm or residual connections. It will not replace backprop next quarter. But it is the first real alternative anyone has demonstrated in 35 years. That is a big deal.

Prompt compression done right

You can take any fixed system prompt, run it once through the model, extract a single 4096 float activation vector, and inject that vector directly into layer 7 of the model forever after. You never need to process the original prompt tokens again.

Accuracy drop across 12 standard benchmarks is 1.8%. Cost per query drops by 60% for any workload that uses a long system prompt. This is shipping in production at three major companies right now. None of them have announced it.

This result also tells us something fundamental about how LLMs work. All the semantic information for an instruction ends up encoded in a single linear vector at the middle layers. All the token processing before that point is just overhead.

All benchmarks are broken

OpenAI dropped their SWE-bench Pro analysis the same week PredicateLongBench was published. Every popular benchmark right now is either saturated, gamed, or measures nothing useful.

PredicateLongBench works because it does not use an LLM judge. It has an objectively correct answer. You can scale difficulty smoothly along independent axes. Every frontier model falls off a cliff at exactly the same point, and none of the public benchmark results admit it.

Eigenvalue calibration fixes overconfidence

All modern LLMs are systematically overconfident. This is not an alignment problem. This is a math problem.

Semantic embedding eigenvalues are not calibrated. You can fix 90% of observed overconfidence with a single temperature scaling parameter applied directly to the eigenvalue spectrum. No fine tuning. No RLHF. One number.

Nobody does this. Everyone is still trying to fix overconfidence with prompt engineering.

What this all means

None of this work is flashy. None of it will get a TED talk. None of it has a cool brand name with an X at the end.

This is the actual work of building good LLMs. This is where all the gains are now. If you are still waiting for the next big architecture announcement, you are waiting for something that will not come. The next 20% of performance will not come from bigger models. It will come from fixing all the boring broken parts that everyone has been ignoring for the last seven years.

You do not need more compute. You do not need more data. You need to stop doing things badly.