Frontier models stopped being cloud only. That is the single most important thing that happened this month.

19 months ago you could not run GPT-4 equivalent capability anywhere except closed provider servers. Today you can run a model that competes on most tasks on a 24GB consumer graphics card. You can build a server that runs every released open model for $17k, no recurring bills. This is not a future promise. This is working right now, today, for people posting real verifiable results.

What landed this week

This was the single busiest 7 day period for usable local LLM releases ever. No announcements had press releases. Almost none had official benchmark papers. All dropped directly to huggingface and reddit.

• DeepSeek V4-Flash-0731: 128B MoE, first frontier model that runs on consumer hardware
• Qwen 3.8: 27B base and Max variants, validated at 17GB VRAM
• KAT Coder 2.5 Dev: 35B coding model, outperforms most larger alternatives
• MiniMax H3: Multimodal model with native 2K 15s video generation
• GLM 5.3: Leaked upcoming release, spotted in public SDK commits

DeepSeek V4-Flash: Quantization tradeoffs

This is the model everyone lost their mind over. For good reason. This is the first open model that is unambiguously in the same performance tier as closed frontier models, and it will fit in VRAM that almost every serious enthusiast already owns.

Quantization does not work evenly here. This is the single most important observation from the first weekend of testing. Most models degrade gracefully as you lower quant levels. This one does not. It falls off a cliff.

Quant Level	VRAM Required	Relative Performance	Recommendation
FP16	256GB	100%	Production only
Q8	128GB	97%	Great if you have the hardware
Q4	64GB	90%	Sweet spot for workstations
Q3	48GB	72%	Bare minimum usable
Q2	32GB	48%	Not worth running

At Q3 you can just squeeze this onto a single 5090. It will run. It will also be noticeably worse. It will still beat almost every 30B model ever released, but you are throwing away most of what makes this model special.

At Q2 you are better off just running Qwen 3.6 27B at Q8. It will be faster, more reliable, and produce better output. Nobody has yet found a quant below Q3 that preserves the reasoning capability this model is known for.

This model is built for agents. It is terrible at static knowledge. It will happily make up facts. It will also flawlessly chain 7 tool calls in sequence, correctly handle 30k token system prompts, and debug broken code that every other open model will give up on. If you are building airgapped systems this is not your model. If you are building anything that uses tools, this is currently the best option you can run locally.

Qwen 3.8 27B: The new baseline

Qwen dropped this on July 28 with almost no fanfare. Daniel Han from Unsloth confirmed this model will run fully offloaded at 17GB VRAM. That means this will run on a 2060 Super, a card you can buy used for $120.

We do not have full independent benchmarks yet. Every early report says this is a clean upgrade over Qwen 3.6 27B, which was already the default workhorse for almost every local deployment. If this holds up, this will become the new default model for 90% of use cases. You will not need anything larger for most internal business tools.

KAT Coder 2.5: Nobody is talking about this and they should

This is the sleeper release of the month. Almost nobody posted about it. Everyone that actually tested it walked away impressed.

This is a 35B coding model quantized to A3B. It runs 5x faster than Qwen 3.6 35B, and produces better output on real code modification tasks.

The testing done here is not synthetic benchmark garbage. This was tested against a real research codebase, where the pass condition was not "looks like correct code" but "produces the correct numerical output when run". Almost every popular model failed this test.

Model	Score	Notes
Qwen 3.6 27B	8/10	Only model that produced correct measurement
KAT-Coder-V2.5-Dev 35B A3B	7/10	Only clean run across all notebooks
Gemma 4 31B	5/10	Produced plausible numbers that were wrong
Ornith 1.0 35B	3/10	Plausible output, all values incorrect
Qwen 3.6 35B A3B	3/10	Nothing would run
Gemma 4 26B	2/10	Failed to implement required logic

This is exactly the test that matters. Benchmarks lie. Real code does not. If you are running local models for software development, stop what you are doing and go test this model.

The 256GB VRAM production build

An infrastructure engineer posted an 8 month operational review of a 10 GPU local server built for a small business. This is the best real world deployment report we have ever had. This is not a mining rig. This is not a hobby build. This is a production system that runs every working day, pays for itself, and works.

Total cost: ~$17,000. Total VRAM: 256GB. This machine will run any released open model at any quant. It runs LLM inference and image generation at the same time for 6 users. It has no API bills. It has no rate limits. It never goes down.

Specs:

• 8x RTX 3090 24GB
• 2x RTX 5090 32GB
• 64 core Threadripper 3995WX
• 512GB ECC DDR4
• Ubuntu 25.10
• Backend: llama.cpp / koboldcpp
• Frontend: Open WebUI

This build disproves almost every common assumption about local LLM hardware. You do not need new hardware. You do not need H100s. You do not need a data center. You just need someone that knows how to configure PCIe bifurcation.

Hardware configuration lessons learned

Almost nothing that went wrong with this build was documented anywhere. All of these are things you will hit if you try to build something similar.

First: Bifurcation. You cannot plug 10 GPUs into a motherboard and turn it on. It will not work. You have to go into BIOS first with one GPU installed, manually enable bifurcation on every slot, set PCIe generation explicitly, enable above 4G decoding and Resizable BAR. If you skip this step you will waste three days debugging why half your cards do not show up.

Second: Transient power spikes. This is the #1 cause of unexplained resets. The system will run fine at 30% load, then reset for no apparent reason. All ten GPUs will spike power at exactly the same moment for 1ms. The average power draw is fine. The transient voltage drop will reset the motherboard.

The fix is trivial and nobody tells you this. Lock the GPU clocks. Not overclock. Lock them to a fixed low clock. For 3090s lock to 1200Mhz. For 5090s lock to 2000Mhz. That is it. This build ran stable for 8 months straight after this one change.

Third: Power draw is not what you think. This system has 2900W of PSU capacity. Under sustained LLM load it never draws more than 1600W. MoE inference is bandwidth bound, not compute bound. The GPUs spend most of their time waiting for data over PCIe. They will never run at full TDP. This entire system will run fine on a standard 20A wall outlet.

Fourth: Put it on wheels. This is not a joke. 10 GPUs dump 1600W of heat into whatever room they are in. You cannot beat physics. You can however roll the entire server into an unused room when you are not standing next to it. This is the single most appreciated feature of the entire build.

Current model VRAM requirements

These are working tested numbers as of July 2025 for llama.cpp with 128k context:

Model	Minimum usable VRAM	Good experience VRAM	Full precision
Qwen 3.8 27B	17GB	24GB	54GB
KAT Coder 2.5 35B	20GB	24GB	70GB
DeepSeek V4-Flash Q3	24GB	64GB	256GB
DeepSeek V4-Flash Q4	48GB	64GB	256GB
MiniMax H3	32GB	64GB	128GB

Mixed workload deployment architecture

Most people run one model at a time. Production deployments run multiple workloads at the same time. This is the working configuration used on the 10 GPU build.

This works. You can run a 128B MoE and generate images at the same time, with zero interference. The trick is manual tensor allocation. Do not let the backend automatically assign layers. Tell it exactly how much VRAM it is allowed to use on each card, and explicitly exclude any cards reserved for other workloads.

What is still broken

This is not all perfect. There are very real unsolved problems.

Quantization degradation on new MoE models is not understood. Nobody knows why DeepSeek V4 falls apart so badly below Q4. All existing quantization methods were designed for dense models. They do not work correctly on the new generation of MoEs.

Multi user scheduling is still terrible. There is no good open source scheduler that will fairly allocate VRAM between concurrent users. If you have more than 4 active users you will start hitting hard limits.

Documentation is non existent. Almost every working configuration is passed around as one line snippets on reddit. Nothing is written down. Nothing is systematically tested. You will have to reverse engineer working setups from comment threads.

Power management on multi GPU systems is still tribal knowledge. Nobody publishes the clock lock trick. Everyone learns it after their system crashes 15 times.

Closing observations

We crossed a line this month.

For the first time, you do not need cloud access to use frontier class AI. You do not need a grant. You do not need a six figure budget. You need a mid range gaming PC, or $17k for a server that will support an entire small business.

This is not a demo. This is not a prototype. These systems are running today, doing real work, paying for themselves. The companies that figure this out first will have an enormous advantage over every competitor that is still paying per token.

Nobody is coming to give you this. None of the big cloud vendors will tell you this is possible. All of the information is out, posted by random people on the internet, for free. Go build it.