We are now in a second open model golden age

Three months ago people were complaining that open models had stalled. That closed models had pulled permanently ahead. That no one would ever release another good open base model.

That was wrong. In the last 30 days five separate Chinese labs have released fully open weights for models that match or beat GPT-4o Mini on standard benchmarks. Every single one has usable quants available within 48 hours of announcement.

None of this is press releases. None of this is gated access. You can download every one of these models right now, run them on consumer hardware, modify them, and redistribute them. This is not an anomaly. This is the new baseline.

GLM-5.2 breaks coding expectations

Unsloth published GGUF quants for GLM-5.2 36 hours after the model was released. That is no longer notable. That is now standard operating procedure.

What is notable is Guillermo Rauch, Vercel CEO, stating he was "almost shocked" at the model's coding performance. This is not a random internet comment. This is someone who runs one of the largest production LLM deployment platforms on the planet testing models every single day.

Independent testing has already confirmed GLM-5.2 12B outperforms Qwen 3.6 27B on most coding benchmarks. It runs at over 80 tps on a single RTX 4090 at 4 bit. For local coding agents this is currently the best option available by a wide margin.

Qwen exits open source

Qwen will not release open weights for 3.7. This is confirmed.

After Junyang Lin was fired from the team, all large model development moved to closed releases. Rumors out of Weibo indicate the small model team has also been disbanded. Qwen 3.6 was the last open release from the team that defined open local LLMs for the last two years.

This is not a tragedy. As of this month every other major Chinese AI lab is releasing open models faster and better than Qwen ever did. The field does not stop. Teams come and go. Good models will continue to be published.

The only permanent loss is institutional knowledge. Qwen built the standard for what good open model release practice looks like. No one has yet matched their documentation, calibration consistency, or willingness to engage with the community.

AutoRound is the best quantization method no one is using

Almost everyone is still quantizing models with AWQ. Almost everyone is leaving 5-10% accuracy on the table for no reason.

AutoRound, originally published by Intel, delivers materially better perplexity retention at 4 bit than any other public quantization method. On long context and reasoning tasks the gap is large enough that you can visibly see the difference in output quality. It runs on pure PyTorch. It works identically on Nvidia, AMD and Intel hardware. It natively exports standard GGUF files.

Almost no one uses it.

The only barrier is branding. People see Intel's name on the repository and assume it is vendor locked. They will not even test it. That is the entire explanation. There is no hidden downside. There is no performance penalty at inference. The 15 minute calibration time is identical to AWQ for equivalent model sizes.

If you are baking quants for your own use this week, use AutoRound. You will not go back.

KV cache quantization progress on Gemma 4

Gemma 4 launched to widespread disappointment among local users. Standard 8 bit KV cache quantization introduced enough output degradation that most people wrote the model off entirely for local deployment.

That problem is fixed.

New testing shows Quantization Aware Training variants of Gemma 4 hold 99.9% KLD parity with full 16 bit KV cache when run at Q8_0. For reference, 99.9% KLD is the threshold below which human evaluators cannot reliably distinguish quantized output from base model output.

This changes everything. Gemma 4 31B can now run comfortably in 24GB of VRAM with effectively zero quality loss. No one has published formal benchmarks yet. Go test this.

Running modern models on 2018 AMD hardware

Almost no one runs H100s for local workloads. Almost every serious private cluster is built from retired datacenter hardware.

This month a user published full benchmark results running MiniMax M3 on 8 and 16 MI50 cards. These cards were released in 2018. They cost approximately $70 each on eBay right now.

On 8 cards the setup achieved 19.2 tps token generation and 1005 tps prefill with EAGLE3 speculative decoding. Acceptance rate hit 50.28% with an average accepted length of 3.51 tokens per draft. This is usable speed for agent workloads. This is faster than most hosted API endpoints.

All of this runs on a maintained public fork of vllm with full ROCm 7.2 support for gfx906. There is no secret sauce. All commands and configurations are published. You can build this cluster this weekend for under $600.

This is the point of local LLMs. This is what all the optimization work is actually for. Running state of the art models on hardware that corporations threw away.

Decentralized model distribution has arrived

Hugging Face is no longer a reliable single source for open models. Everyone watched Fable get taken down. Everyone watched Chinese models start getting removed last month. Everyone understood this was coming.

Noema Atlas launched this week. It is a peer to peer distribution network for LLM weights built on Iroh. Every file is identified and verified by its BLAKE3 content hash. Downloads automatically fail over between peers. Identical files are stored once on disk using reflinks. You can rescue and reseed any model that has been removed from centralized hubs.

This is not a crypto project. This is not a scam. This is boring, well engineered infrastructure built by people who have been active in this community for years. It is Apache 2 licensed. It has native desktop clients and a full CLI.

Install it. Seed the models you already have. You will be glad you did this before the next round of removals.

What changed this month

This month did not bring one big announcement. It brought ten small ones that add up to a permanent shift.

Open models are now good enough that even people running production closed systems are noticing. Quantization has gotten good enough that the quality gap between local and hosted is effectively invisible for most use cases. Legacy hardware is good enough to run state of the art models at production speed. We finally have an alternative to centralized model hosting.

None of this was true 12 months ago. All of it is true today.

Operational takeaways for teams

These are actions you can take tomorrow:

1. Replace your current local coding model with unsloth GLM-5.2 12B GGUF Q4_K_M. It will outperform anything else you are running.
2. Stop generating AWQ quants. Use AutoRound for all new model quantization.
3. Test Gemma 4 QAT with 8 bit KV cache.
4. If you run AMD hardware, pull and test the gfx906 vllm fork. Even if you run newer cards many of the optimizations will apply.
5. Install Noema Atlas. Start seeding at least the 3 models you use most often.

Open questions

There are still gaps in what we know:

• No one has yet published full accuracy benchmarks for AutoRound across all major model families.
• We do not know how well GLM-5.2 holds up at context lengths above 128k.
• The long term durability of the peer to peer model network is untested.
• No one has yet demonstrated AutoRound quants working reliably with speculative decoding.

Someone will answer all of these questions in the next two weeks. That is how fast this field moves now.

You do not need to wait for official announcements. You do not need to wait for blog posts. All of this work is happening out in the open, every single day, by people just running tests and posting the results. That is the part almost no one outside this community understands. This is not research anymore. This is operations.