Six months ago if you told a production ML team you were running agentic workloads on consumer hardware they would have laughed. Today it is the default choice for anyone working with regulated data.

That is the single most important shift nobody wrote a press release about.

This is not a hobby anymore

The UCL RRBench paper did not test toy problems. They tested the exact boring data preparation work that eats 60% of researcher time in longitudinal health studies. Work that you cannot send to OpenAI, full stop. GDPR does not care how good GPT-5 is.

They built a ground truth dataset from real British cohort study data, defined 20 standard data cleaning and harmonization tasks, and ran every production ready open weight model against it. The results ended the argument:

Model Size	Average Task Completion	Minimum VRAM Required
7B	51.2%	16 GB
14B	72.7%	24 GB
31-35B	87.9%	48 GB

87.9% is not "good for a local model". That is good enough. That is the threshold where teams stop arguing and start deploying. That is the line where local AI stops being a hobby and becomes operational infrastructure.

This benchmark was run on consumer grade hardware. No data centre GPUs. No special permission. Just a standard workstation you can build this afternoon.

The 35B sweet spot

This is the inflection point everyone missed. For three years the conversation was about ever larger models. Now every single vendor is converging on exactly this class: 30-35B total parameters, 3B active per token, MoE architecture, FP8 quantized.

Fits perfectly on a single 48GB consumer GPU. Runs at over 200 tokens per second. Good enough for 90% of all production AI tasks.

Hetzner launched their experimental inference endpoint with exactly this model. AMD dropped their first public open model exactly at this size. Every top coding and reasoning model released in the last quarter lands here.

This is not a coincidence. This is the point on the cost/performance curve where local beats cloud on every axis except raw world knowledge.

Hetzner is calling the commodity inference bluff

Hetzner did not launch a product. They ran an experiment. And it is the most important thing that happened in inference all year.

Everyone knew open weight inference was a commodity. Nobody had the guts to act like it. Until Hetzner showed up, dropped an OpenAI compatible API, no billing, no SLA, no marketing, and just said "tell us if you use this".

The numbers they got are not impressive because they are fast. They are impressive because they are boring. 153ms median time to first token. 224 output tokens per second. That is not a demo. That is production performance. And Hetzner did it on hardware they already had sitting idle.

This is the end of 10x margin inference. If Hetzner follows through, every other inference provider will have to cut their prices by 70% inside 12 months.

Local AI does not need to be perfect

The single dumbest argument against local models is that they are not as good as GPT-5.

Nobody is arguing that. Nobody is asking them to be.

90% of all AI usage in production today is not writing research papers or inventing new code. It is summarizing. Classifying. Extracting fields. Normalizing categories. Cleaning data.

For all of these tasks, a good 35B model will beat a human operator 7 times out of 10. And it will never exfiltrate the data. It will never have an outage. It will never raise prices. It will never change its terms of service.

You do not build trust with users by writing a privacy policy. You build trust by not needing one.

Almost every task lands on E. Almost every developer today picks F by default. That is a mistake.

The silent install arms race

Vendors noticed this too. And they are responding exactly the way you would expect.

Google Chrome now silently installs a 4GB Gemini Nano model on every eligible machine. No consent. No notification. If you delete it, it comes back. They bundled it with a security update so you would not notice.

Anthropic did exactly the same thing with Claude Desktop three months earlier.

This is not about making features fast. This is a land grab. Every vendor understands that the default model on the user's device will get 90% of all usage. They are all racing to install theirs first, before anyone gets around to asking users if they want this.

At Chrome's scale this one push emitted between 6,000 and 60,000 tonnes of CO2. Nobody was consulted. Nobody voted. It just happened.

The hardware trap nobody warns you about

If you are building a multi-GPU local rig right now, do not use Intel consumer platforms.

This is not fanboyism. This is a hard hardware failure that will waste weeks of your time.

Arrow Lake consumer CPUs advertise 24 PCIe 5.0 lanes. They support bifurcation. They pass every check. And PCIe P2P between GPUs is completely broken.

Configuration	Measured P2P Bandwidth	Observed Behaviour
Intel Z890, stock Nvidia drivers	11.8 GB/s	P2P disabled entirely
Intel Z890, patched P2P drivers	5.5 GB/s	Model output corrupts silently
AMD AM5, stock Nvidia drivers	59 GB/s	Works correctly

You will not find this on any motherboard spec sheet. You will not find this in any review. You will only find this after you have spent $6000 on hardware, installed everything, and wonder why your tensor parallel models output lines of exclamation marks.

Nvidia knows this. They explicitly block P2P on Intel consumer platforms in their stock drivers. They just did not bother telling anyone.

What people actually use local models for

Nobody runs local models to write poetry.

When you ask the community who exclusively uses local models, this is the list that comes back every single time:

• Contract review for client confidential documents
• Medical research data cleaning
• Source code analysis for proprietary codebases
• Log parsing and incident response
• Email and calendar extraction
• Bulk transcription of internal recordings

Every single one of these is a task that would cost $100k+ a year in cloud API bills. Every single one is a task you cannot legally send to a third party.

None of these people care about SWE-bench scores. They care that the data never leaves the building.

The unspoken tradeoff

There is one thing almost all of the pro-local arguments skip.

Local AI removes trust from the cloud provider. It transfers that trust to the model publisher.

Right now almost everyone running local models is downloading arbitrary weight files from Hugging Face. Almost nobody verifies them. Almost nobody audits them. Almost nobody checks for backdoors, hidden logging, or poisoned weights.

We have traded one set of opaque third parties for another. We have not yet built the infrastructure to make this actually safe.

That is the next problem. It is not going to solve itself.

Where this is going

By the end of 2027, half of all AI inference will run outside of the big three cloud providers.

Most of it will not run on fancy B300 clusters. Most of it will run on consumer GPUs under desks, on spare server capacity, on colocation boxes, on user phones.

The cloud providers will keep the frontier models. They will keep the 1T parameter workloads. Everything else is leaving.

This is not ideological. This is just arithmetic.

When a $1500 consumer GPU can run a model that does 90% of what you need, and will pay for itself in 7 weeks, you do not have an argument. You just place the order.