Nobody is writing this down. All the important progress right now is not appearing in arXiv papers, not on company blogs, not in keynote presentations. It is happening in github PR comment threads, 3am reddit posts, and discord servers where people are arguing about broken GGUF files and power supply ratings.

This is the actual state of the field, right now.

DFlash lands in llama.cpp, and it is messy

Last week PR #22105 adding DFlash speculative decoding merged into llama.cpp master. This is the single largest inference speedup shipped for open models in 12 months. Nobody is talking about the part where it almost does not work.

DFlash works differently from every previous speculative decoding implementation. Instead of generating one token per draft forward pass, it outputs an entire block of up to 16 candidates at once. In perfect lab conditions this delivers up to 8x end to end speedup on Qwen3.

Nobody gets 8x. Most people get nothing. A lot of people get crashes.

Model	Claimed benchmark speedup	Reported real world speedup	User success rate
Qwen3 4B	7.2x	5.1x	82%
Qwen3 8B	6.8x	3.7x	71%
Qwen3.5 27B	5.3x	1.8x	44%
GPT-OSS 20B MoE	3.1x	1.1x	29%

This is not a bug in the implementation. This is the gap between benchmark performance and real world usage. The benchmarks run at temperature 0, top-k 1, fixed seed. Nobody runs models like that. Once you turn on sampling acceptance rates collapse.

There is also a fundamental architectural penalty for hybrid models like Qwen3.5. Hybrid models carry recurrent state that cannot be rolled back partially. When a draft block is rejected the entire target state has to be restored from snapshot and replayed. Every rejection costs you one full forward pass on the target model. On bad runs you end up slower than running no speculative decoding at all.

This is the tradeoff nobody mentions. Speculative decoding is not a free speedup. It is a bet. You bet that the draft will be right often enough that the wins cover the cost of the times you lose. Right now DFlash is a good bet on small pure attention models, and a terrible bet on everything else.

The 96GB 4090 scam

There are no 96GB 4090s. There are no 96GB 5090s. If you see someone selling one they are stealing your money.

This comes directly from someone who actually runs a factory modding 48GB 4090 PCBs. The memory bus on AD102 cannot address more than 48GB. Full stop. There is no hidden mode, no mod, no firmware patch. Every listing you have seen for 96GB cards is preying on people desperate for VRAM who will believe anything.

This is the quiet dynamic of the entire hardware market right now. Demand for VRAM completely outstrips supply at every price point. Scams have become the default. Nobody is policing this. Nvidia will not comment. Resellers will not comment. Everyone just pretends the cards exist until someone sends money and gets nothing.

If you are shopping right now: 48GB modded 4090 is real. 32GB 4080 Super is real. Everything else advertised above that is fake until proven otherwise.

You can run GLM 5.2 for $2500

You do not need $50,000 to run current state of the art models. You do not even need $10,000.

This is the build that everyone is quiet about:

Part	Cost	Source
Epyc 7302 + motherboard	$460	Ebay
2x Tesla P40 24GB	$460	Ebay
512GB DDR4	$1000	Ebay
PSU, case, storage	$580	Anywhere
Total	$2500

This will run GLM5.2 at Q4 quantization. It will not be fast. You will get ~7 tokens per second. That is enough. That is fast enough to read. That is fast enough to run long planning tasks, debug code, process documents. Nobody is going to shut it off. Nobody is going to rate limit you. Nobody is going to pull the model.

This is the point almost everyone misses. Local LLM is not about maximum speed. It is about existence. You have the model. It will always work. All the arguments about tokens per second are irrelevant compared to that.

Yes it is slow. Yes it draws 700W. Yes it sounds like a vacuum cleaner. It works. That is the bar.

Model distribution is moving to torrents

Hugging Face is no longer reliable. Everyone knows this. Nobody says it out loud very often, but every single person running local models has had a model pulled in the last 6 months.

This week modelregistry.io launched. It is exactly what it sounds like: a torrent index for open models, with Hugging Face automatically configured as a fallback web seed. If there are peers you download from them. If there are no peers it transparently falls back to downloading directly from HF.

This is not a hack. This is the new normal. Open model distribution will not be run by a single company any more. It can not be. There is too much regulatory pressure, too much corporate pressure, too many people who want to turn models off.

Torrent with web seed is the perfect compromise. It is distributed. No single party can take down a model. It remains fast when demand spikes. It gracefully falls back to centralized hosting when there are no peers. This will be the standard way models are distributed by the end of the year.

MathFormer and the end of the reasoning myth

A 4 million parameter seq2seq model just hit 98.6% accuracy on symbolic algebra expansion. It has never been taught what multiplication is. It has never been taught what a variable is. It has only ever seen token sequences. It just learns the transformation pattern.

This is not a surprise to anyone who has spent enough time looking at model internals. But it is the cleanest demonstration we have ever had. All of the apparent mathematical reasoning you see in large language models is just structured pattern matching. There is no understanding. There is no logic. There are only token transformations that happen to produce correct output often enough to fool you.

This changes exactly nothing about how you use models. But it changes everything about what you should expect from them. You will never get reliable reasoning out of this architecture. You will get very good pattern matching. That is enough for a lot of things. It is not enough for everything.

Reward hacking debuggers exist now

Everyone training RL for LLMs has this exact problem: reward goes up, and you have no idea if the model got better or if it just found a new way to cheat the reward function.

rewardspy launched this week. It is a 1200 line library that wraps your reward function and tracks the signatures of reward hacking. It watches for variance collapse, component imbalance, response length drift, GRPO group collapse. It flags anomalies before you waste three days of training on a model that learned to output 700 newlines to get a perfect score.

This is the kind of tool that never gets published. Nobody will write a paper about it. Nobody will give a keynote. But every single person running GRPO training right now is already using this.

The home datacenter reality

People are running production grade LLM deployments out of their laundry rooms.

One user this week posted a rig with 4x 48GB 4090, 192GB total VRAM, running off a 30A dryer outlet. It heats half his house in winter. He has to turn it off to run the dryer. It has nuisance GFCI trips once every two weeks. It also runs Qwen 397B. It powers a whole house voice assistant that works better than anything you can buy.

This is the actual cutting edge. Not the $100k DGX racks in datacenters. The guy who ran an extra power line to his laundry room because he got sick of OpenAI rate limits. This is where all the real experimentation is happening.

There is no roadmap. There is no vendor support. There is no documentation. There are just people showing up every week, breaking things, fixing them, and posting the results.

The exit from centralized models

OpenAI just restricted GPT-5.6 rollout after government request. They stated publicly that this will become standard. This was the turning point for a lot of people.

For the last three years the implicit deal was: you put up with rate limits, you put up with censorship, you put up with price increases, and in exchange you got the best model. That deal is now broken. The best model will not be available to you. It will be available to approved partners only.

Local LLM stopped being a hobby about 6 months ago. It is now a legitimate alternative deployment path. For an increasing number of use cases it is not just the better option. It is the only reliable option.

What comes next

None of this is going to slow down. Open models are already good enough for almost all use cases. Hardware is getting cheaper if you know where to look. Distribution is becoming decentralized. Tooling is getting good enough that normal engineers can deploy this stuff.

You will not read this on TechCrunch. You will not see it in keynotes. But if you know where to look, it is already happening.