We are at a fork. For the first time since the transformer was published, we have a realistic path out of the hyperscaler AI monopoly. We have protocols that can train frontier models across untrusted consumer hardware. We have models that run well on phones and laptops. We have good tooling.

And we are blowing it.

The fork in the road

There are two completely incompatible visions for on-device AI that are both being sold under the same name.

One vision: you decide when to run a model. You decide which model runs. You can delete it. You can replace it. No data leaves your device unless you explicitly send it.

The other vision: a vendor pushes a model onto your hardware in the background. They update it silently. They run it whenever they decide. You can delete it, and they will put it back. You will never be told exactly what it does.

Both are called "local AI". Both run inference on your CPU. Only one of them gives you control. Right now the second one is winning by a very large margin.

BlockTrain breaks the training monopoly

Almost every argument for centralized AI starts and ends with training. You need tens of thousands of connected accelerators. You need hundreds of millions of dollars. You need a data centre. This was true until last month.

BlockTrain is the first working decentralized training protocol that delivers near parity with end-to-end trained transformers. It does not use gradient checkpointing. It does not use federated averaging. It splits the model into independent vertical blocks, each trained by a separate worker against a derived local objective. No worker ever sees the full model. No worker ever holds full optimizer state. No trusted coordinator is required.

Configuration	Cross Entropy	Perplexity
End-to-end reference Transformer	1.319	3.74
BlockTrain 6 worker distributed	1.385	3.99
BlockTrain single worker	1.359	3.89

This is not a simulation. The authors ran this across three public internet GPU hosts over raw TCP. They trained a logical 75.8B parameter model while moving only 15.22 GB of checkpoint data total. For inference it outperforms standard autoregressive pipeline parallelism across WAN links, because it emits full sequences per traversal instead of one token at a time.

The structural advantage of hyperscalers just vanished. You do not need a single cluster any more. You just need six random people on the internet with consumer GPUs who agree to run a block each. This is not theoretical. This works today.

What local AI actually does well

Almost every public argument about local AI is about the wrong thing. People spend thousands of words arguing about whether a local model can beat GPT-4o. That is the wrong comparison.

90% of all AI use in production software right now falls into exactly five categories:

1. Summarize this text
2. Extract 3 bullet points of facts
3. Classify this item into one of 7 categories
4. Rewrite this text to be clearer or more polite
5. Pull dates, names and addresses out of unstructured input

None of these need a frontier model. All of these run perfectly reliably on a 3B parameter model that will run on any phone manufactured after 2021.

You do not need a model that can pass the bar exam to summarize the news article the user already loaded onto their screen. You just need a model that will not make up facts. Local models are already very good at this.

The typed output revolution no one is talking about

The single most important improvement to AI tooling in the last 18 months has received almost no attention outside Apple developer circles. It is not bigger context windows. It is not faster inference. It is typed model output.

Instead of asking the model for JSON and praying it follows your schema, you define a normal Swift struct, add natural language guides for each field, and ask the model to generate an instance of that type. No parsing. No fallback logic. No regex hacks. You get a real native type you can pass directly to your UI.

This is the difference between AI as a novelty and AI as a boring, reliable software subsystem. And this works perfectly locally. There is no reason to ever send user data over the internet for this operation.

Google broke the social contract first

Everyone expected the fight for AI control would be about forcing people to use cloud models. No one expected the fight would be about forcing people to run local models.

Starting in April 2026, Google Chrome began silently installing a 4 GB Gemini Nano weights file onto every eligible desktop device. There was no consent prompt. There was no notification. The settings UI that lets you disable the feature is not even enabled until after the download completes. If you delete the file, Chrome will re-download it on the next idle window.

This is not a bug. This is deliberate architecture. Chrome profiles received the full model even when they had never received any human input, never opened any AI feature, and never touched any UI related to this functionality.

The silent deployment math

At Chrome's scale, this is not just a privacy violation. It is an environmental event with measurable planetary impact.

At the lowest estimated adoption rate this single push emitted between 6,000 and 60,000 tonnes of CO2. That is equivalent to flying 13,000 people one way between London and New York. No vote was held. No announcement was made. One company just decided to do this.

We are measuring the wrong thing

Every site about local AI looks like canirun.ai. They will tell you exactly how big a model you can run on your hardware. No one will tell you how small a model you actually need for the job.

We have 0.6B parameter models that summarize text perfectly. We have 1.5B parameter models that extract structured data with 99% reliability. We have 2B parameter models that outperform GPT-3.5 on classification tasks.

Almost no one is building for these models. Almost every developer reaches straight for 70B benchmarks, then concludes that local AI is not good enough. We have this entirely backwards. We should not be trying to run the largest possible model locally. We should be trying to run the smallest possible model that does the job reliably.

The unspoken trust boundary

There is a line almost no one will say out loud. A local model controlled by a vendor is not private.

If Google wrote the model, if Google can update it silently, if Google controls what prompts it executes, then it does not matter that inference runs on your CPU. It is still Google's computer. It just happens to be sitting on your desk.

This was the trick. Everyone was watching for cloud surveillance. No one noticed that exactly the same surveillance can run locally. You do not win by moving inference to the device. You win by controlling the model that runs there.

What comes next

This is not inevitable. We still have choices.

Stop defaulting to OpenAI API calls for every trivial feature. Stop treating 70B parameters as the baseline for every use case. Build for the 3B model first.

Demand that models are opt in. Demand that you can delete them. Demand that you can replace them. Demand that the name of the file on your disk tells you what it actually is.

BlockTrain proved that we do not need hyperscaler clusters to train good models. The hardware already exists, distributed across billions of devices. The only thing missing is the decision to use it for something other than extending the existing monopoly.

We did not build local AI so that Google could silently install 4GB files on your hard drive. We built it so that you own the software you run. We can still have that. But we have to stop arguing about benchmarks and start arguing about control.