None of these papers are about new base models. None report another 1% MMLU gain. Every single one addresses a problem you will hit if you build, deploy or evaluate production LLM systems right now. That is why this cluster is worth your time.

What actually scales with LLM size?

This is the cleanest scaling study published in the last 12 months. Researchers trained 85 identical Qwen3 transformer models across two full orders of magnitude of compute. No cherry picking. No different training recipes. No fine tuning. Every variable was held constant except total FLOPs used for pre-training.

They measured performance across three categories of social simulation task: opinion modeling, behavioral replication, and longitudinal forecasting.

The final number is the critical result. Cognitive bias calibration has a scaling exponent of 0.03. That is effectively zero. From 0.5B all the way up to 70B parameters, models do not get any better at reproducing human risk aversion, anchoring effect, or reward learning heuristics.

If you are building agent simulation, you cannot wait for bigger models to fix this. You will have to implement human biases explicitly.

Grading unstructured correct answers

This paper is nominally about grading university bash exams. It applies directly to every situation where you evaluate outputs that have multiple valid answers: SQL queries, infrastructure configuration, API calls, incident response commands.

Rule based checkers cannot handle partial credit, equivalent solutions, or harmless syntactic variation. Humans do not scale.

Researchers tested four frontier models against 1200 real student responses graded independently by three expert instructors. This is their core result:

Model	Prompt type	ICC agreement	Mean absolute error
Gemini 3.0 Pro	Rubric	0.888	0.10
Claude Opus	Rubric	0.871	0.12
GPT-4o	Rubric	0.862	0.13
GLM-4	Rubric	0.827	0.17
Gemini 3.0 Pro	Baseline	0.741	0.26
All models average	Baseline	0.693	0.31

Rubric quality had a larger effect than model choice. The worst model with a good structured rubric outperformed the best model with a generic prompt. Stop arguing about which frontier model is best. Write better prompts first.

They also broke performance by a four level cognitive taxonomy. Agreement starts at 0.94 for simple lookup questions, drops to 0.71 for advanced system design. This is a general rule you can use today:

• L1/L2 tasks: fully automate grading
• L3 tasks: automate with 10% human spot check
• L4 tasks: keep human grading

You can detect guardrails from outside the API

Everyone building red team tools has run into this problem. You send a prompt, you get a refusal. Was that the base model refusing, or was that the separate guardrail layer in front of it?

This is not an academic distinction. Bypass techniques work completely differently for the two cases. Until now there was no reliable way to tell the difference without internal access.

This paper demonstrates you can distinguish guardrail blocks from model intrinsic refusals 98% of the time with only black box access. You only need to measure three signals: response latency delta, token length distribution, and refusal phrase lexical patterns.

Researchers tested this methodology against every major production LLM API. They achieved 100% accuracy for detecting guardrail presence, and 98% F1 score for assigning refusal source. This works right now against OpenAI, Anthropic, Google, and Mistral endpoints. No special permissions required.

This changes red teaming. Almost all public bypass research published to date was accidentally optimizing against guardrails, not model alignment. Now you can tell which one you are actually attacking.

Building domain LLMs without breaking everything

This is not a good paper because it is about fitness coaching. It is a good paper because it is the first proper ablation of domain post training that measures the tradeoff everyone ignores.

Almost every domain fine tune you see online makes one catastrophic compromise. It gains 15% on the target domain task, and loses 30% of general capability. The model forgets how to write code, how to follow instructions, how to reason about things outside the training set.

The FitOne team tested a three stage post training pipeline, and measured both domain gain and general capability retention at every step:

Training stage	ACSM exam gain	General capability retention
Base Qwen3 8B	0%	100%
Continual pretraining only	+4.1%	99.2%
+ supervised fine tuning	+8.7%	97.1%
+ reinforcement learning	+10.1%	96.4%

Every stage adds domain performance. Every stage loses a tiny, acceptable amount of general capability. This is the correct tradeoff curve. If you are seeing larger drops you are doing it wrong. You are overfitting.

Stop running one shot full parameter fine tuning on your domain dataset. That is a bad default. Stage the training.

LLM judges are biased, but not all of them

This is the single most important paper released this month. Everyone uses LLM judges now. Almost everyone is doing it wrong.

Everyone knows LLM judges prefer longer answers. Everyone knows they prefer the answer listed first. Almost no one corrects for this.

Researchers tested 16 different models as judges across a benchmark with known ground truth quality. The result is very clean:

• Mid tier and cheap judges: extremely biased. Naive aggregation will reliably select the longest answer, not the best one. Recall for correct top-k hovers around 0.55 no matter how many comparisons you run.
• Frontier judges (Claude Opus, GPT-4o 5.5): effectively no measurable bias. They rank correctly.

That is not the important part. The important part is you do not need to use frontier judges. The paper introduces a simple Bayesian correction model that explicitly models verbosity and position bias. This lifts recall from 0.56 to 0.91 on Llama 3 70B. That means you can get 90% of Opus judging quality for 5% of the cost.

Active ranking cuts judge cost by 70%

The same paper also has an equally important result about comparison efficiency.

Almost everyone runs round robin pairwise comparisons. That is extremely inefficient. If you only care about identifying the top k items, not the full ranking of every entry, you can stop comparing items that are clearly not going to make the cut.

Their top-k aware active acquisition rule achieves identical final recall with 30% of the comparison budget. If you are running any kind of model evaluation, response ranking, or red team triage today, you are wasting 7 out of every 10 judge calls.

The end of magic box thinking

All five papers have one thing in common. None of them treat LLMs as magic black boxes. None of them argue that bigger models will fix all problems.

Every single one maps out boundaries. What scales, what does not. What you can automate, what you cannot. Where bias exists, where it does not. How much you gain from each step, what you trade away.

This is what mature applied LLM research looks like. We are past the phase of proving LLMs can do things. We are now in the phase of mapping exactly what they can and cannot do, under what conditions, at what cost.

What you should implement this week

You do not need to wait for future models. You can use these results today:

1. If you use LLM judges: add verbosity and position bias covariates. Switch to top-k active sampling.
2. If you evaluate outputs: use the four level taxonomy to decide what can be automated.
3. If you fine tune domain models: stage your training pipeline. Stop overfitting.
4. If you red team: implement the guardrail detector.
5. If you build social simulations: hard code cognitive biases. Scaling will not fix them.

None of these require new infrastructure. None require fine tuning. All are just changes to how you run the systems you already have.