Reasoning does not scale the way you thought it did

Thirteen papers dropped on arXiv June 16. None of them announced a new state of the art base model. None had flashy demo videos. All of them told us things we actually needed to know.

This batch of research ends one argument for good. We are not going to outscale fundamental reasoning limits by making models bigger. Every result this month confirms the same pattern: transformer performance decays exponentially with the number of compositional reasoning steps required. Parameter scaling shifts the entire curve up. It does not change the slope.

This is not an opinion. It is a consistent measurement across every model, every domain, every evaluation protocol published this week.

Code interpreter reasoning has measurable signatures

Code interpreter is the single most effective reasoning improvement deployed in the last 18 months. Until this paper, no one had any idea why it worked.

Researchers looked at traces from 7 different CI enabled models across 12 reasoning benchmarks. They found three consistent behaviours that separate good and bad reasoning outputs: explicit verification, backtracking, and backward chaining. Strong models produced these behaviours 3-7x more often than weak models, even when they arrived at the same final answer.

You can exploit this today at inference time with zero retraining. Appending 4 fixed tokens that trigger verification behaviour improved MATH accuracy by 11%, ordering task performance by 7% and combinatorial optimization results by 9%. Gains were zero on common sense and factual recall tasks.

This is not magic. Code interpreter works not because models can run code. It works because code forces models to execute the verification steps they will otherwise always skip.

The hard compositional limit every production LLM hits

This is the most important paper released this month. You should stop reading and go read it right now.

Researchers built a controlled dataset of clinical EHR questions, annotated by working clinicians for the exact number of discrete reasoning hops required to produce an answer. They tested Claude Sonnet 4, GPT-4o and GPT-5.4 across zero shot and extended reasoning configurations.

All three models showed identical monotonic accuracy decay. GPT-5 fell from 37.8% accuracy at 1 hop to 23.5% at 4 hops. Claude fell from 30.6% to 17.6%. Odds of a correct answer dropped by ~25% for every additional reasoning step required, regardless of model.

Extended thinking did not fix this. It did not even flatten the curve. It only made models generate 3x more tokens before producing the same wrong answer.

This effect is not caused by context truncation, prompt quality or fine tuning. It is an architectural property. Every transformer will fail this way. This is the hard limit you will hit on every production deployment, and no one is talking about it.

Negation is still broken. No one fixed it.

We have models that write working device drivers, pass bar exams and design semiconductor circuits. They still cannot reliably parse negation.

This paper replicated negation scope tests across every major model released in 2026. Even the best models failed 22% of simple negation scope tests. Performance dropped to 41% correct when negation applied to a subordinate clause.

In context learning improves performance. It does not eliminate the failure mode. Function vectors can be constructed to detect negation cues. No model can reliably extract negation scope.

This is not a minor edge case. This failure mode is silent, systematic and present in every production LLM today. It will cause wrong answers that you will never catch with normal testing.

MoE just got a 2x memory win for free

Everyone has been arguing about MoE routing, expert count and load balancing for two years. No one tested the obvious thing.

This paper introduces expert tying. You share identical expert parameter weights across consecutive transformer layers. You keep independent routing and attention per layer.

That is the entire change.

Across OLMoE, Qwen3 and DeepSeek MoE architectures this modification reduces memory footprint by 47-51% during both training and inference. Perplexity degradation was less than 0.2% across all tested scales. Downstream benchmark scores were within measurement error of untied baselines.

This will be in every production MoE implementation by the end of the year. There is no catch. This was just low hanging fruit that everyone walked past.

Multimodal reasoning: text wins most of the time

Gen-VCoT is the first proper evaluation of visual chain of thought reasoning. The results are humbling.

Generating intermediate RGB representations such as segmentation masks and depth maps improved performance on spatial relation questions by 25% and depth estimation questions by 50%. On every other class of visual reasoning task, standard text only chain of thought outperformed visual intermediates. On CLEVR the gap was 91.2% vs 62.5% accuracy.

Visual reasoning is not better. It is only better for the very narrow set of tasks where humans also cannot reason without drawing a diagram. For everything else, writing down the steps in words still works better.

You do not need 100k labels to train reasoning

Everyone operates under the assumption that training good reasoning requires millions of high quality annotated traces. This assumption is wrong.

This semi supervised framework trains a 70M parameter verifier classifier on 120 labelled reasoning examples. It then filters unlabelled model generated traces, and uses high confidence passes to fine tune the base model.

On Orca Math and GQA this approach matched the performance of training on 12-17x more fully labelled data. Both the lightweight verifier and entropy filtering were required. Removing either component collapsed performance.

You do not need a giant team of annotators. You need 100 good examples and a way to throw away bad outputs. That is it.

Transformer training instability was a solved problem this whole time

For four years everyone building large transformers has fought random loss spikes, divergence and wasted training runs. We had workarounds. No one understood the cause.

This paper demonstrates that training instability correlates perfectly with surges in preconditioned Hessian curvature. Curvature grows monotonically with model depth during early training.

The fix is almost insultingly simple. Start training with 1/4 of the final model depth. Add layers one at a time over the first 10k training steps.

This architecture warm up eliminated all observed training instability on 7B and 32B transformer runs. It did not slow down convergence. It required no changes to optimizer, learning rate schedule or data pipeline.

Scientific foundation models do not need their own stack

LOGOS is the first general purpose science foundation model that does not use custom geometric neural networks. It encodes molecules, materials, reaction pathways and biological structures as plain token sequences in a shared grammar, trained with standard autoregressive next token prediction.

Across 19 domain specific benchmarks LOGOS matched or outperformed every dedicated specialist model. Performance scaled cleanly with parameter count from 1B to 8B parameters.

The actual result here is not the model. It is the proof that we do not need an entirely separate technical stack for AI for science. All of the work on equivariant networks, graph neural networks and geometric transformers was an unnecessary detour. Standard transformers work fine if you get the tokenization right.

Agent skill construction moved past handwritten tools

OpenClaw CSTS is the first agent skill system that does not require engineers to write tool definitions.

The framework runs distributed tree search across multiple base models to discover, evaluate and rank reusable skill primitives. Skills are scored both for task performance and transferability across different models.

Trained agents using this skill tree outperformed hand written tool use agents by 38% on long horizon planning benchmarks. Most importantly, discovered skills generalized across model families without modification.

We will not be writing agent tools in 12 months. We will be searching for them.

What none of these papers say

None of these results are being discussed on social media. None of them will be the subject of keynote talks. All of them will actually change how we build systems.

This month we learned that reasoning has hard measurable limits. We learned that most of the gains everyone is chasing can be had for free with trivial architectural changes. We learned that we have been wasting billions of compute working around problems that had simple solutions.

Most importantly we learned this: we are no longer in the era where bigger models fix everything. We are now in the era where we have to understand how they work. That is much harder. It is also much more interesting.