No one released a 2 trillion parameter model last week. No one announced 10x faster inference. There was no product launch, no demo that went viral on twitter.

Instead four papers landed on ArXiv within 48 hours of each other, all working the same quiet unglamorous corner of the field. All four attack base assumptions that everyone building LLMs has accepted without question for the last six years. This is the most important batch of ML research published in the last two years. Almost no one has noticed yet.

No one is scaling parameters right now

We have passed the peak of blind scaling. For the first time since 2020, all of the highest impact work in LLMs is not making models bigger. It is fixing the parts we built wrong the first time.

Every one of these four papers delivers improvements that would previously have required 10-100x more training compute. None of them use more compute. None of them train a larger model. All of them go back, look at a core design decision made in 2017 or 2019, and say "we got this wrong".

This is not a temporary trend. This is the field maturing. The easy gains from scaling are gone. Everyone is now digging into the foundations.

Tokens are not vectors

For seven years every single transformer implementation has treated tokens as points in a d-dimensional euclidean vector space. Every optimization, every quantization scheme, every inference kernel, every parallelism strategy is built on this assumption. No one even bothered to justify it. It was just how things worked.

The Token Is a Group Element throws this out entirely.

This paper demonstrates that tokens should not be vectors. Tokens are elements of a matrix Lie group. They are transformations, not features. There is no payload. There is no embedding vector.

When you model tokens this way, attention stops being a learned kernel. It becomes a closed form geometric measurement. The attention score between two tokens is just the norm of the logarithm of their relative group pose. No MLP. No learned projections. No Clebsch-Gordan coefficients. No magic temperature parameters you have to tune.

The results are not subtle. On SE(2) and SO(3) sequence completion tasks this construction matches the performance of a fully learned MLP attention kernel using 50 to 80x fewer parameters. The standard vector token baseline breaks equivariance by five to twelve orders of magnitude. That is not a small error. That is the baseline not working at all.

Most importantly this construction works for affine groups with scale and shear. Every prior equivariant attention method could not handle these groups. Everyone had concluded they were fundamentally out of reach for transformer architectures. They were not. We were just using the wrong representation for tokens.

This is the most important transformer architecture paper since the original 2017 announcement. It will be in every open source model released 12 months from now.

In-context learning is Bayesian inference

We have spent three years arguing about what in-context learning actually is. People have proposed it is implicit gradient descent. People have proposed it is a lookup table. People have proposed it is just pattern matching.

All of those explanations are wrong.

Multi-Task Bayesian In-Context Learning demonstrates that in-context learning is amortized hierarchical Bayesian inference. And when you build it explicitly as such, almost all of the known failure modes of in-context learning disappear.

Prior work had one fatal flaw: all in-context models were permanently coupled to the prior distribution they were trained on. If you presented them with data drawn from a prior outside the training distribution, performance collapsed completely. Everyone accepted this as a fundamental limitation.

This paper removes that limitation. They train a transformer that accepts prior information as an ordinary prefix sequence of in-context examples. At test time you can swap out entire prior distributions just by changing the first 16 tokens of the prompt. No fine tuning. No additional training.

Across all test cases this model matches the performance of exact oracle Bayesian predictors while running three orders of magnitude faster. It works even when presented with priors completely outside the meta distribution used for training. It works for high dimensional latent spaces where exact inference is computationally intractable.

This is not an incremental improvement. This means you will never again fine tune a small model for a domain specific prediction task. You will just pass the prior as a prompt prefix.

LLMs lie because they never had to resolve conflict

Every production LLM today operates with one hard coded rule. Either always believe the parametric knowledge in the weights. Or always believe the context in the prompt.

There is no middle ground. There is no mechanism that says "both of these might be wrong. Let's check both."

This is the single largest cause of hallucinations in deployed systems. It is also the problem that every existing RAG and self correction approach completely ignores. All existing systems just pick one source to trust and ignore conflicts entirely.

Explicit Knowledge Conflict Resolution for LLM Inference fixes this.

The proposed MACR framework first uses a modified semantic entropy measure to quantify the model's actual confidence in its internal parametric answer for a query. Not the fake logprob confidence that can be arbitrarily calibrated. A measurement of how much the model actually agrees with itself across multiple rollouts.

If confidence is low, it retrieves external context. If confidence is high, it first externalises its own internal knowledge as written text. Then it runs all available sources through three specialised reasoning agents: one to extract explicit rules from each source, one to map all conflicts between sources, one to resolve inconsistencies without default privileging any source.

Across standard conflict benchmarks MACR outperforms all existing state of the art baselines by between 19% and 27%. Most importantly it produces a human readable audit trail for every resolution. You can see exactly which conflict was identified, and exactly which reasoning step was used to resolve it.

You can implement a working version of this pipeline this week for your own RAG system. There is no good reason to run anything else.

We have been measuring reasoning completely wrong

All existing LLM reasoning benchmarks are broken.

They have contamination. They have fixed answer sets that get memorised during training. They have no ability to adjust difficulty in fine grained increments. Most importantly they cannot distinguish between a model that actually performed reasoning and a model that guessed the correct answer from pattern matching.

QMFOL fixes this.

QMFOL is a benchmark generator that produces provably unique, provably consistent monadic first order logic reasoning tasks. You can dial exact parameters: number of logical steps, number of disjunctions, number of distractor statements, label type. Every generated test case is verified round trip with an external theorem prover. There is no leakage. There is no possibility of memorisation.

The authors ran QMFOL against every top reasoning model available as of June 2026. All of them fall off a performance cliff exactly at 4 logical steps. All of them perform 30-40% worse on false or unknown labels than on true labels. All of them degrade linearly with the number of distractor statements.

None of this shows up on MMLU. None of this shows up on GSM8K. None of this shows up on any of the standard benchmarks that vendors use for marketing. All of those benchmarks are effectively useless for measuring actual reasoning ability.

If you are evaluating an LLM for any high stakes use case, stop running standard benchmarks. Run QMFOL. It will tell you the actual truth about what the model can and cannot do.

The common thread across all four papers

None of these papers use a new trick. None of them introduce a clever new training method. None of them require more data.

All four papers do the exact same thing. They go back to an assumption that everyone stopped questioning in 2020, and test if it was actually correct.

We stopped doing this for half a decade. Everyone was too busy scaling. Everyone assumed that if you made the model big enough, any flaws in the base architecture would just disappear.

That did not happen. The flaws did not disappear. They just scaled too.

Now we are finally going back and fixing them. This is the work that will define the next generation of LLMs. Not bigger models. Correct models.

What this means for production systems

You will not have to wait three years for these results. You will start seeing them in production systems before the end of this year.

Lie algebra attention will land first in robotics and multimodal models, where equivariance failure is already a hard blocking problem. It will then propagate to general language models once people realise it also reduces attention drift in long context windows.

MACR conflict resolution will become a standard component of every RAG pipeline within six months. There is no downside and the performance gains are too large to ignore.

QMFOL will become the standard evaluation benchmark for all reasoning models. Vendor marketing claims will stop including 10 step reasoning performance, because no model can actually do it and this benchmark proves it.

Bayesian in-context learning will kill 80% of all domain fine tuning use cases. Most teams fine tuning small models today are just manually encoding a prior distribution. You will soon be able to do that with 16 tokens in a prompt.

Open questions

None of these papers answer every question. All of them leave large open gaps that no one has started exploring yet.

Can you combine Lie algebra attention and Bayesian in-context learning? No one has tried. There is no obvious reason you could not. The resulting system would have exactly the properties everyone has been trying to get from LLMs for years.

How does MACR scale when there are 15 conflicting sources instead of 2? The paper only tests up to 4 sources. We do not know where it breaks.

What happens if you train a model exclusively on QMFOL generated tasks? Does the 4 step performance cliff remain? Or is it an artifact of the training data that all current models use?

We will get answers to most of these questions over the next three months. That is the speed this field now moves at.

Closing observation

For five years the entire field operated under one unspoken rule: all problems can be solved with more parameters.

That era is over.

The papers released last week are the end of that era. The next generation of LLMs will not be 10x bigger. They will not be 100x bigger. They will be built correctly. They will be built on foundations that were actually tested, not just assumed.

That is a very good thing.