The inference pivot is here

This was the week the LLM industry stopped arguing about who has the smartest model.

Every major release across the last 72 hours is about inference. Not architecture. Not training data. Not MMLU scores. Every single one.

That is the takeaway. If you run LLMs in production, you can stop reading headline benchmark scores. The game has changed.

For three years the entire field competed on training. Whoever could train the largest model won. That race is over. The new race is who can run an existing model the cheapest, the fastest, at the highest concurrency.

This week that race got very serious.

GPT-5.6: Pricing is the real announcement

Everyone is talking about benchmark scores. Ignore them.

The actual news from OpenAI this week is that they cut the effective cost per unit of capability by 50% overnight. They did not just release a better model. They reset the entire market price line.

Model	Input $/M token	Output $/M token	Relative capability
GPT-5.6 Luna	1.00	6.00	Lightweight
GPT-5.6 Terra	2.50	15.00	Parity GPT-5.5
GPT-5.6 Sol	5.00	30.00	Better than GPT-5.5
Claude Fable 5	10.00	50.00	Parity GPT-5.5
Claude Mythos 5	10.00	50.00	Parity GPT-5.6 Sol
GPT-5.5 Pro	30.00	180.00	Legacy flagship

Sol matches Mythos 5 performance for 1/3rd the cost. Terra matches GPT-5.5 for half the price. Luna undercuts every other hosted model on the market.

OpenAI did not win this round with better model capability. They won it with better inference efficiency. Every price point on this table is a direct statement about how much cheaper they can run inference than Anthropic.

This is not a model release. This is a price war.

DeepSeek DSpark: Speculative decoding finally works in production

This is the release that will actually change what you run in production 30 days from now.

Speculative decoding has existed as a research demo for two years. Everyone knew it could make generation faster on benchmarks. No one could run it at production load. Every implementation fell apart under real concurrent traffic, because validating bad draft tokens wasted more batch capacity than you gained.

DeepSeek did not invent speculative decoding. They fixed the part that everyone was ignoring.

DSpark delivers 60-85% faster end user generation speed. At exactly the same total throughput. Zero quality loss. No changes required to your base model.

They ran this on real production traffic. Not a lab benchmark. Real users. Real load. And it works.

And they open sourced the entire stack.

You do not have to switch to DeepSeek V4 to use this. You can train a DSpark draft model for Qwen3, Gemma, or any other model you are already running today. You can drop this into your existing inference stack and make every request 70% faster next week.

That is the most important release of the entire month. No one is talking about it enough.

DSpark architecture breakdown

There are exactly two new ideas in DSpark. Neither is glamorous. Both work.

First, semi-autoregressive draft generation. Parallel draft models generate 16 tokens at once, but the last 10 are garbage and almost always get rejected. DSpark adds a tiny serial head that models dependencies inside the draft block. Acceptance rate per draft goes up 30% overnight.

Second, confidence scheduled validation. This is the trick that makes it work at scale.

Every previous implementation would send every single draft token to the base model for validation. Even the ones that had a 2% chance of being accepted. At high load this destroyed throughput.

DSpark adds a single linear confidence head to the draft model. It predicts for every token the probability it will pass validation. Then it truncates the draft before sending it. It only validates the tokens that are actually worth validating.

No magic. No fancy new architecture. Just stop wasting GPU time validating tokens that are going to be rejected anyway.

That is it. That is the trick that gives 85% faster generation.

DSpark benchmark results

In production testing against their existing MTP-1 baseline, DSpark moved the entire Pareto frontier.

For context, this is a larger generation speed improvement than you would get from upgrading an entire generation of GPU hardware. And it costs you nothing.

SpectralQuant: 4bit quantization just stopped being a tradeoff

For 18 months there was an accepted tradeoff. If you quantized a model to 4bit to run on consumer hardware, you gave up 5-10% of model quality. That was just the tax. Everyone paid it.

That tax no longer exists.

Spectral Labs released a calibration aware quantization method this week that produces standard, unmodified Q4_K_M GGUF files. Same size. Same speed. 100% compatible with existing llama.cpp builds. No runtime changes required.

And it recovers 96.5% of the quality gap between vanilla 4bit and full BF16.

Model	BPW	Size MiB	heldout120 loss
BF16 Reference	16.0	1446	2.9809
SpectralQuant Q4_K_M	4.52	415	2.9961
llama.cpp vanilla Q4_K_M	4.52	415	3.4135
Unsloth IQ4_NL	5.26	483	3.0484

At exactly the same file size and speed, SpectralQuant beats every existing 4bit and 5bit quantization method. It is almost indistinguishable from BF16 on the test suite.

This is not incremental. This removes the single largest practical tradeoff for local and edge LLM deployment.

Mamba hybrid models break the context speed curve

If you have ever run a large context model, you know the rule. Decode speed falls off linearly with context length. At 128K context you get half the speed you get at 4K. At 256K you get a quarter. That was just how transformers worked.

That rule is also dead.

The Nemotron 3 Super hybrid Mamba+MoE model runs perfect needle retrieval at 504K tokens. On 4x RTX 3090. And it only loses 68% of its decode speed going from 1K to 504K context.

At 504K context, Nemotron runs 11x faster than an equivalent standard MoE model. And it retrieves needles perfectly at every depth.

It is not perfect. It has strong recency bias. Instructions at the start of context will be overridden by instructions at the end. That is a known Mamba limitation, and one that is already being addressed.

But for any workload that processes large documents, logs, or codebases, this changes everything.

The new frontier

Six months ago the production LLM tradeoff curve was fixed. You paid a certain amount of money for a certain speed and capability.

This week every single point on that curve got 2x better.

You can run your existing model 70% faster for free with DSpark. You can run your existing model at 1/4th the memory for effectively zero quality loss with SpectralQuant. You can run 500K context at usable speed on consumer hardware.

And OpenAI just cut the price of hosted inference in half.

None of this required training a new 1T parameter model. None of this required new breakthroughs in model capability. All of this was just fixing the boring, unsexy engineering problems that everyone ignored while they were racing to train bigger models.

What you should be testing right now

Stop waiting for GPT-5.6 access. Stop arguing about benchmark scores.

This week you should:

1. Pull the DeepSpec repository. Train a DSpark draft model for the base model you run in production today.
2. Download the SpectralQuant Qwen 0.8B quant. Run it against your existing test suite.
3. Spin up Nemotron 3 Super on any 4 GPU machine you have. Throw your longest context workload at it.

All three of these things will reduce your production costs and improve your user experience more than any new flagship model release this year.

The training race is over. The inference race has just started. And this was the first very loud starting gun.