Nothing this month came with a blog post, a press tour, or a venture capital announcement. Every major advance posted to /r/LocalLLaMA in the last two weeks was built by one person, in their spare time, on used hardware.

This is not incremental progress. This is the point where the open ecosystem stopped just copying closed model capabilities, and started moving faster than the companies that invented this technology.

The 3 line bug that broke every P100 for 3 years

For three years, every single Tesla P100 running llama.cpp was producing systematically wrong logits. Nobody noticed.

The bug was trivial. llama.cpp had an exemption for sm_61 architecture cards (GTX 10 series, P40) forcing them to run accumulation in fp32, because their fp16 hardware is broken. The P100, sm_60, was left on the fp16 path. It has perfectly good fp16 hardware. It just wasn't being used correctly.

The numbers are brutal:

Metric	Unpatched	Patched	Improvement
Median KL Divergence vs fp32	0.0023	0.000001	2300x
Next token agreement	96.5%	99.9%	1 in 29 tokens wrong → 1 in 1000
Decode speed	22.7 t/s	23.0 t/s	+1.4% faster

This was not a tradeoff. The "fast" path was slower and wrong.

P100s sell for $80 shipped right now. Until last week everyone thought they were garbage for LLM inference. Most of that reputation was this bug.

The patch is three lines. It was merged into every major fork within 48 hours.

Voodoo Quant: per tensor quantization that doesn't overfit

Unsloth Dynamic Quantization was the state of the art for six weeks. It worked great. Everyone used it. Then someone noticed it only worked great in llama.cpp. Run the exact same quantized model in PyTorch and quality fell off a cliff.

Unsloth had overfit their quantization heuristic exactly to the numerical quirks of llama.cpp's execution graph.

Voodoo Quant fixes this. Instead of optimizing at the block level, it scores and sets precision for every single tensor individually. It does not overfit to one runtime.

That 95% reduction in KLD is not a typo. This is the single largest improvement to quantization quality in the last 12 months.

Right now it only exists for small models. The author is working on 27B and 35B variants.

J-space left the lab: what works, what breaks

Two weeks ago Anthropic published their J-space paper. Everyone expected it would stay a lab curiosity for six months.

It didn't.

Within 72 hours there was a working open Jacobian Lens implementation for GGUF. Within 5 days people were wiring it into production agent guard rails.

This works. But it does not work everywhere. Someone ran a full benchmark across 7 datasets and 11,400 examples, and mapped exactly where this signal works and where it fails completely:

Task	J-space hallucination detection performance
Long tail fact retrieval	100% precision at 5% review rate
Trivia QA	92% precision
GSM8K math	Zero transferable threshold
TruthfulQA adversarial myths	Literally worse than random

J-space entropy detects guessing. It does not detect lies the model believes are true. It cannot tell the difference between hard thinking and making something up.

This is not a universal hallucination detector. It is an extremely good detector for one very specific and very dangerous failure mode. That is enough.

You can now perform surgery on fine tuned models

Everyone knew you could add layers to a base model during pre-training. Everyone also knew that if you tried to add layers to an already fine tuned model, it would die horribly.

That was wrong.

The first attempt failed for exactly the reason you would expect. If you initialize new layers as identity pass-through, they never receive any gradient signal. They just sit there as dead weight.

The fix is obvious once someone says it out loud. Initialize each new layer as a weighted average of the two layers immediately adjacent to it. It will do something useful from the very first training step. It will receive gradient. It will learn.

Then run two healing rounds: first train only the new layers, then unfreeze the whole model for a small number of steps.

That is it.

The resulting extended model did not forget anything. It did not collapse. On 9 out of 10 test domains it performed identically to the original parent model. On one hard physics question it outperformed the original model that it was grown from.

This is not a better model. That was never the point. The point is you can now open up a finished, fine tuned model, add capacity, and it will heal. Nobody has ever demonstrated this before.

Running 700B models on a laptop

Someone posted their setup.

Laptop: i7-8750H, 20GB RAM, GTX 1050 Mobile 4GB VRAM. Model: Deepseek V4 Flash 700B MoE, IQ3_XXS quant. Speed: 1.0 - 1.8 tokens per second.

It works.

All experts are offloaded to NVMe. KV cache is quantized to Q4. Mmap is enabled. He uses it for reverse engineering and code auditing. He sends a prompt, goes for a walk, and comes back to a correct answer.

This is not a gimmick. This is usable. There are people running production workloads like this today.

The $200 3 user production LLM server

Two used P102-100 cards. $100 total. 20GB combined VRAM, 448 GB/s bandwidth.

They run Qwen3.6 35B at IQ4_XS. Three concurrent users. 23 tokens per second per user. 32k context each.

This matches the per user performance you will get from an RTX 4090. For 1/20th the price.

Llama.cpp did this. Not Nvidia. Not any cloud provider. A bunch of volunteers writing open source software turned $100 worth of obsolete mining garbage into production grade inference hardware.

What this means

None of this work was funded. None of it was announced. None of it came from any of the companies that everyone agrees own this technology.

The gap between closed and open models is still there. But the gap between closed and open tooling is gone. And it is now closing faster than the big companies can move.

You do not need permission to run a 35B model. You do not need a budget. You do not need to ask anyone. You can do it this weekend, for $100.

You can look inside the model while it thinks. You can modify its behaviour. You can cut it open and make it bigger.

This is not the future. This is right now.

Open questions

We still do not know if other GPU architectures have similar silent precision bugs.

We do not know how far model surgery can be pushed. Can you double the size of a model? Triple it?

We do not know if J-space thresholds scale with model size, or if every model will need its own calibration.

We do not know where the floor is for usable inference hardware.

All of these questions will be answered by random people on the internet, in their spare time, in the next 30 days.