Nobody is arguing about who has the best model anymore.

If you work on production ML systems you already know this. The conversation stopped being about benchmark scores in late 2025. Today every team is arguing about the same three things: how to keep their GPUs busy, how to avoid being locked into a single model provider, and how to debug agents that stop working at 3am.

None of these problems are being solved by OpenAI, Anthropic, or any of the big cloud vendors. All of them are being solved by open source tools, most of which did not exist 6 months ago.

The constraint moved

For the first 7 years of the modern AI boom, the binding constraint was model capability. If you had a better model you won. That is no longer true.

Capability is now abundant. There are half a dozen models good enough for 95% of production use cases. What is scarce is well utilized compute.

This is not a new pattern. Every maturing technology follows the same arc. First you fight to build the thing that works. Then you fight to run the thing you built efficiently.

Aviation went through this exact transition. Airlines did not compete on who built the best aircraft. They competed on how many hours per day they could keep the aircraft they already owned in the air. An aircraft costs the same amount every hour, whether it is flying or parked on the tarmac.

A GPU works exactly the same way.

Idle GPUs are the new grounded aircraft

The Hugging Face post on GPU management laid this out clearly, and it has already become the default operating assumption for every serious ML infrastructure team.

Every GPU you own is costing you money every single second. Financing, depreciation, power, cooling, rack space. All of these run on calendar time. None of them pause when the GPU is not running a workload.

Two teams with identical GPU fleets can have 3x difference in actual useful output. That gap is already larger than the difference between any two production models available today.

This is the most important number in AI right now, and almost nobody measures it correctly. Most teams track average GPU occupancy. Almost none track useful occupancy.

You can run a low priority background job and get 99% occupancy on your dashboards. That does not mean you are getting value out of the hardware.

Workload mismatch is the invisible utilization killer

Most wasted capacity does not look like idle GPUs. It looks like busy GPUs running the wrong work.

Modern GPU clusters run at least 6 distinct workload types: real time inference, batch inference, fine tuning, full training, quantization, and embedding generation. Each of these has completely different requirements for latency, memory, duration and preemption tolerance.

A scheduler tuned for low latency inference will leave large gaps of unused memory between jobs. A scheduler tuned for training will let user facing requests time out.

This is not a problem you solve by buying more GPUs. Adding more hardware just gives you more capacity that will sit in the wrong shape at the wrong time.

The new generation of open ML tooling

Over the last 6 months an entire stack of open source infrastructure has appeared to solve exactly these problems. None of this is research code. All of it is running in production right now.

Tool	Purpose	Release date	GitHub stars
Opik	LLM observability & evaluation	March 2026	21,400
Code Graph RAG	Codebase knowledge graphs	May 2026	8,700
Free Claude Code	Coding agent proxy	June 2026	12,200
OlmoEarth Platform	Planetary scale geospatial inference	July 2026	3,100
InvokeAI 4	Local generative media engine	July 2026	49,800

This stack did not exist at the start of the year. It is already more capable for production use cases than every closed commercial alternative.

Coding agent proxies: unbundling model access

The single most widely adopted new tool this quarter is free-claude-code. It solves one very specific, very painful problem: the official Claude Code and Codex agents lock you into Anthropic and OpenAI respectively.

This proxy sits locally on your machine, implements the exact API contract those agents expect, and routes requests to any model you want. 31 providers are supported today, including every local inference runtime.

You can run the unmodified official Claude Code VS Code extension, backed by Llama 3.3 running locally on your machine, or Nemotron 120B on NVIDIA NIM, or any combination. You can route Opus tier requests to one model, Sonnet to another, and Haiku to a third.

This is not a hack. This is how every engineering team runs coding agents today.

Code Graph RAG: stop asking LLMs to guess your codebase

RAG for code has been broken for 3 years. Everyone was doing it wrong.

Chunking source code files and doing semantic search works for toy repositories. It fails completely on monorepos, large codebases, or any code with non trivial structure. LLMs cannot reconstruct call graphs from random chunks of text.

Code Graph RAG fixes this. It parses every file in your repository with Tree-sitter, builds an actual knowledge graph of functions, classes, imports and call edges, stores this in Memgraph, and runs natural language queries by generating Cypher instead of doing vector search.

It can find dead code. It can trace data flow across 12 function calls. It can rewrite code structurally across the entire repository. It works across 14 languages on the same monorepo.

This is the first code understanding tool that actually works for production codebases.

LLM observability that you can actually run

Opik changed the observability space completely when it launched.

Every prior LLM observability platform was either closed, paid per token, or offered a crippled self hosted version. Opik released the entire platform, backend and all, under Apache 2.0. You can run every single feature on your own infrastructure, no license required, no sales calls.

Capability	Opik	LangSmith	Langfuse
Full platform open source	✅ Apache 2.0	❌	✅ MIT core
Self host all features	✅	❌ Enterprise only	✅
Agent trace trees	✅	✅	✅
LLM as judge metrics	✅	✅	✅
Framework agnostic	✅	❌ LangChain first	✅
CI/CD evaluation	✅	✅	✅

Teams are migrating off closed observability platforms at a rate I have never seen before. Nobody wants to send every single LLM request their agents make to a third party vendor. Nobody wants to pay per token for software that runs on their own hardware.

Specialized infrastructure for specialized workloads

The OlmoEarth platform demonstrates the next step for this stack. General purpose ML infrastructure only gets you so far.

For geospatial inference at continental scale, you do not run a generic Kubernetes cluster. You split every job into three separate stages: CPU for data fetching and preprocessing, GPU for pure inference, CPU again for post processing and stitching.

When they ran the North America wildfire risk map, they used 19,600 CPUs and 994 GPUs. They turned 4737 hours of serial compute into 30.5 hours wall clock time. That is a 155x speedup.

You will not get that from any generic cloud ML service. This is the kind of infrastructure you only build when you stop treating models as magic black boxes.

What this all adds up to

We are at an inflection point.

For the last five years every production ML team built their stack top down. You picked the model first, then built everything else around whatever that model required.

Now teams are building bottom up. They build the infrastructure layer first, then run whatever model makes sense for each individual task.

Open source won this fight because closed vendors will never build neutral infrastructure. Anthropic will never make it easy to run their agent on Mistral. OpenAI will never release an observability platform that works with every model. Cloud providers will never build tools that help you use 80% of your GPU capacity instead of 30%.

None of this is ideological. It is just practical.

What comes next

This stack is still incomplete. There are still large missing pieces.

We still do not have a standard open source GPU scheduler that correctly handles mixed workload types. We still do not have good standard tools for capacity planning. We still do not have good cross provider benchmarking for coding agents.

But those things will arrive. They are being built right now, by the same teams that built the tools you are already using.

The model wars are over. The infrastructure wars have just started.