Every single work covered here was published or released in the last 72 hours. None are vapor. All solve problems production ML engineers are actively fighting today. There are no paradigm shifts here. Just measurable, reproducible gains you can deploy this month.

Stop uniform quantizing ViTs

Post training quantization works. Everyone knows this. Almost everyone is still doing it wrong.

Existing PTQ methods apply identical bit width across every component of the transformer. This is insane. Different parts of the model have wildly different tolerance to quantization error. Attention projection layers break completely below 5 bit. MLP intermediate layers run perfectly fine at 2 bit. No one had properly measured this gap until now.

MixFrag fixes this. It first estimates quantization fragility for every individual component by measuring KL divergence between full precision and quantized output distributions, using only 128 calibration samples. It then formulates bit allocation as a standard Multiple Choice Knapsack Problem, assigning precision to hit exactly your target model size.

On COCO object detection under the common MP3/MP3 mixed precision budget, MixFrag outperforms the previous best PTQ method by 9.6 AP. Same total bit count. No fine tuning. No retraining. That is not an incremental improvement. That is the difference between a model you ship and one you throw away.

Token pruning was doing it backwards

Visual token pruning cuts multimodal LLM inference cost more than any other single optimization. Every existing implementation was making a catastrophic mistake.

All prior methods run pruning once at layer 2 or 3, permanently discarding tokens based on a single local score. Token importance is not static. Tokens that appear completely irrelevant at shallow layers are very often the exact tokens that carry the correct answer at layer 24. Half the failure modes of production MLLMs trace directly to tokens being thrown away too early.

The new Trend-aware Pruning framework solves this. Instead of taking a snapshot score, it tracks the momentum of attention flow across layers. It maintains a reserve of undervalued tokens, and will reactivate them later if their semantic importance is rising.

This method removes 77.8% of visual tokens. Only 23 remain in the final layer. There is no measurable accuracy drop on standard MLLM benchmarks. No fine tuning required. This cuts multimodal inference cost by 75% overnight. You will see this shipped in every production MLLM by the end of the quarter.

Quantized LLMs break when you edit them

If you have ever tried to inject updated facts into a deployed 4-bit LLM you already know this. No one had written it down publicly until this week.

Quantized models suffer catastrophic degradation after only 2-3 sequential memory edits. Accuracy falls off a cliff. Old knowledge breaks. No existing editing method accounts for this. The authors call this the quantization stability crisis.

CACHE-UK is the first working mitigation. It confines all edits exclusively to low rank LoRA subspace, adds domain aware edit strength scaling, and runs a closed loop controller that tracks accumulated degradation debt and blocks edits before they break the base model.

Tested on 4-bit quantized OpenLLaMA-3B across 20 sequential financial fact updates, CACHE-UK reduces knowledge degradation by 11-17% relative to all existing baselines. Generalization rate improves 6 percentage points to 28%.

That 28% number is still terrible. It is also the best result anyone has ever published on this problem. More importantly, this is the first work that even acknowledges the problem exists.

Layers are the new context axis

Everyone has spent the last two years optimizing context along the token axis. Sliding windows. KV cache compression. Token pruning. No one noticed you can optimize along the depth axis instead.

Transformer layers do not do the same job. Lower and middle layers build general semantic understanding. Upper layers specialize that representation for the specific query. This division of labour is consistent across every modern LLM tested.

CoMem exploits this. You run every archived context chunk once through layer 18, and cache only that residual state. When you execute a query you only run layers 19 through 32 over the retrieved cached states.

For 128k context on an NVIDIA H20:

• Full KV cache uses 89.36 GB
• CoMem uses 18.26 GB
• Prefill speedup is 7.83x

CoMem outperforms full context execution on both RULER and LoCoMo benchmarks. This is the single most important architecture insight published this year. Everyone was looking along the wrong axis.

CPU inference is not a compromise

90% of production NLP workloads run on CPU. No one has built a good encoder for this target in three years. Until this week.

Liquid AI released LFM2.5-Encoders, general purpose bidirectional encoders built from the ground up for fast long context execution on commodity CPU.

At 8192 tokens the 230M variant runs 3.7x faster than ModernBERT-base. It also scores higher on every SuperGLUE task. There is no tradeoff here. This model is strictly better for every classification, routing, PII detection and policy linting job that runs on CPU.

Speech models finally got quantization right

Microsoft dropped the BitNet quantized build of VibeVoice ASR this week. This is the first production grade speech model that runs acceptably on edge CPU.

Original 7B model size: 4.62 GB. Quantized size: 1.58 GB. Real time inference (RTF < 1) on 3 CPU threads. No GPU required. It still processes 60 minutes of audio in a single pass, maintains full diarization and timestamp capability.

This was the last major barrier to running on device ASR. Everyone has been waiting for this.

Capability comparison

Technique	Target Model	Measured Gain	Overhead	Ready for production
MixFrag PTQ	Vision Transformers	+9.6 AP COCO	10 minute calibration run	Yes
Trend Token Pruning	Multimodal LLMs	78% token reduction	2% per layer compute	Yes
CACHE-UK	Quantized LLMs	17% lower edit degradation	<1% memory	Yes
CoMem	Long context LLMs	7.8x prefill speedup	32 rank LoRA	Test this week
LFM2.5 Encoder	CPU NLP	3.7x latency	None	Use today
VibeVoice BitNet	Edge ASR	65% size reduction	None	Use today

The common pattern

All of these works share one unstated conclusion. All the generic one size fits all optimization tricks have hit the wall.

Uniform quantization. Static pruning. Full KV cache. These were good enough for 2024. They are now leaving 2-4x performance on the table.

The next gains do not come from making math faster. They come from measuring what actually breaks in your model, and only applying compression to the parts that can take it.

Every transformer component has different fragility. Every token has a different lifecycle. Every layer has a different job. If you treat them all the same you are wasting compute.

What you should do next week

1. Pull the MixFrag reference implementation and run it on your production ViT. It will take one afternoon. You will get 5-10 AP back for free.
2. Test LFM2.5 encoder on your CPU classification jobs. If it is even close you will cut your hosting bill by 70%.
3. Stop pruning tokens before layer 12. Just stop. There was never any good reason to do that.

None of this work got a press release. None of this will be on the front page of Hacker News tomorrow. This is the actual work that makes deployed ML work. This is what production ML engineers spend 90% of their time fighting over. And this week, every single one of those problems got a little bit easier.