This is not a normal week for open LLMs.

Over the last seven days every major release has moved the line of what you can run locally. Not by small increments. By entire tiers. As of today you can run a coding agent that beats Claude Opus on verified tool use benchmarks, a 1M context multimodal MoE, and a full persistent roleplay game with inline image generation. All on hardware you probably already own. None of this requires an API key. None of this leaves your machine.

That is the takeaway. Everything below is the details, the numbers, the working code, and the tradeoffs.

This week's baseline shift

Six months ago the line for usable local models was 7B parameters. Three months ago it was 12B. This week it is 32B activated parameters.

Every single release this week follows the same pattern: very large total parameter MoE models, extremely small activated footprints, native quantization, and first class local deployment support. No one is releasing dense 70B models any more. No one is apologizing for quantization quality. Every vendor now assumes you will run their model on your own hardware.

This is not a trend. This is the new default.

Kimi K2.7 Code: The first open model that beats closed benchmarks

Moonshot dropped K2.7 Code on Tuesday and almost no one noticed how big this is.

This is a 1T total parameter MoE with 32B activated parameters. It has 256K native context. It runs perfectly on a single RTX 4090.

Benchmark numbers are not abstract. On MCP Mark Verified, the only human audited tool use benchmark, K2.7 Code scores 81.1. Claude Opus 4.8 scores 76.4. That is not a rounding error. That is an open model beating the current best closed model on a real world agent benchmark.

It uses 30% fewer thinking tokens than the previous K2.6 release. That means it completes tasks faster, uses less context, and costs less to run. There is no catch here. The weights are available. The license is modified MIT. You can serve it right now with vLLM 0.7.2 or later.

Deployment is trivial. Run this command and you have an OpenAI compatible endpoint on port 8000:

vllm serve moonshotai/Kimi-K2.7-Code --quantization int4 --gpu-memory-utilization 0.95

This will use approximately 21GB of VRAM. It will decode at ~110 tokens per second.

One important detail: preserve_thinking is forced enabled. The model retains its full internal reasoning across conversation turns. This is not a bug. This is the single largest improvement to agent consistency ever shipped. If you strip the reasoning tokens out you will break performance. Do not do this.

MiniMax M3: 1 million context that actually works

MiniMax released M3 the same day. This is the first open model that delivers usable 1 million token context without tricks.

Total parameters are 428B. Activated parameters are 23B. That means you can run the full model, at full context window, on 16GB of VRAM with INT4 quantization.

The speed numbers are not marketing. At 1M tokens, M3 delivers 9x faster prefill and 15x faster decode than the previous M2 release. Per token compute is 1/20th of standard GQA attention. This is not sliding window. This is full attention over every token.

They did this with a new sparse attention operator called MSA. It is not patented. The code is shipped with the model weights. There is already a working pull request to upstream it into vLLM.

M3 has native multimodal training from initialization. Not a bolted on vision encoder. Text and images are trained together from step zero. This is the reason it does not hallucinate object counts the way every other open multimodal model does.

There are two inference modes. Use thinking mode for agent and coding work. Use non-thinking mode for chat and retrieval when you need latency under 100ms.

The Gemma 4 fork explosion

Google released Gemma 4 two weeks ago. The community has already produced better variants than the official release.

The most important one is OBLITERATED Gemma 4 12B. The original model has 256K context. This fork removes alignment guardrails entirely, fixes three KV cache bugs, and reduces idle memory footprint by 19%.

You can run the full 256K context window of this model in 7.7GB of RAM. Not VRAM. System RAM. It will run on a laptop.

This is not an anomaly. There are now 17 public fine tunes of Gemma 4. Every single one outperforms the official Google release on every benchmark. Google built an excellent base model. They did an extremely poor job finishing it. The community fixed that in 12 days.

One demonstration of this is Open Dungeon, a fully local AI Dungeon clone released this week. It runs this exact fork of Gemma 4, generates inline images with FLUX, and stays under 8GB total memory usage for the entire stack.

openPangu 2.0: Extreme sparsity taken to the limit

Huawei announced openPangu 2.0 at their developer conference. It will be released fully open source on June 30.

This model takes the MoE tradeoff further than anyone has gone before. The Pro variant has 505B total parameters. Only 18B are activated per token. That is a 28:1 sparsity ratio. No other production model comes even close.

Huawei is very open about why they built it this way. Training large dense models is expensive. Running them is more expensive. They optimized entirely for throughput and latency. Single card throughput is double that of any comparable open model.

There is one catch. This model was built from the ground up for Ascend hardware. It will run on NVIDIA cards, but you will lose about 40% of the performance. That is still faster than every other model at the same activated parameter count.

We will have working weights in 12 days.

Specialized small models are eating auxiliary tasks

Everyone is racing to build bigger general models. The quiet progress this week is on extremely small specialized models.

Supra Labs released Supra Title 350M. It does one thing: generate good conversation titles. It does it better than GPT-4o. It runs at 1200 tokens per second on a Raspberry Pi. It is 177MB at Q2 quantization.

This is the future of LLM stacks. You do not run a 70B model to generate chat titles. You do not run a 32B model to run OCR. You run a 300M model that does one job perfectly.

PP-OCRv6 also landed this week. The largest variant is 34.5M parameters. It is 5.2x faster on CPU than the previous version. It supports 50 languages. It beats every closed OCR API on scanned documents and CAD drawings.

Every week more of the auxiliary tasks that used to require general purpose models get replaced by 100x smaller specialized models. This trend is accelerating.

Deployment state of the art right now

If you are deploying any of these models today, there are exactly three inference engines worth using. Nothing else comes close.

1. SGLang is the fastest for batch throughput and agent workloads. Use this if you are running tools or coding agents. It has native support for interleaved thinking and multi step tool calls.
2. vLLM is the most compatible. Use this if you need an OpenAI compatible endpoint or you are running standard chat workloads. Every model released this week works out of the box on the latest vLLM nightly.
3. llama.cpp is the best for edge and CPU deployment. Use this for laptops, embedded devices, or any system without an NVIDIA GPU.

Do not waste time with anything else. All other inference engines are at least 2x slower on every benchmark.

All three engines support native INT4 quantization. All three deliver within 5% of each other on single stream decode speed. None of them require you to compile custom kernels any more.

The quiet standardization no one is talking about

Every single model released this week uses exactly the same interface.

They all support the OpenAI chat completion API. They all use the same multimodal message format. They all run on the same three inference engines. They all accept the same generation parameters.

Six months ago every new model required custom code, special prompt templates, patched inference forks, and three days of debugging. Today you can swap Kimi K2.7, MiniMax M3 and Gemma 4 out for each other without changing a single line of your application code.

This standardization is the most important development of the last year. No one announced it. No one wrote a press release. It just happened. And it means you can now build applications that do not lock you to one model vendor.

What comes next

This pace will not slow down.

We will see at least three more frontier open models released before the end of July. Every one of them will follow the same pattern: 500B+ total parameters, 15-35B activated parameters, 512K+ context, native quantization, and first class local support.

The line between closed and open models has already vanished on most benchmarks. Within three months the best general purpose model in the world will be open. You will run it on your desktop.

Closing notes

None of this was supposed to happen this fast.

Two years ago everyone agreed that frontier models would stay locked behind cloud APIs forever. That only large companies would ever be able to run them. That local deployment would be a niche hobby for enthusiasts.

That was wrong.

Right now you can build production systems that never send a single byte to a third party. That perform as well or better than any closed API. That run on hardware you already own.

That is the world we live in now. Go build things.