Nobody posts the boring useful tools on Twitter. You will see 1000 retweets for an agent demo that breaks after three turns. You will see almost zero discussion for the tools that actual engineers are deploying to production this week.

All of the tools covered here hit GitHub trending in the last 14 days. None are demos. All solve specific, painful, unaddressed problems that anyone running LLM systems in production has already hit.

The state of production LLM tooling right now

We have passed the demo phase. Nobody is impressed anymore when an agent writes hello world. The entire field has quietly shifted to boring operational problems: security, cost, reliability, observability, repeatable improvement.

Every tool released in the last month reflects this shift. There are no new agent frameworks. There are no new base model announcements. There are tools that fix the garbage you have to deal with after you decide you actually want to run something.

NVIDIA SkillSpector: The security tool everyone was missing

Agent skills are the largest unpatched security hole in production LLM systems right now.

Everyone knows this. Nobody was doing anything about it. Claude Code, Gemini CLI and every other agent runtime will execute arbitrary third party code by default with zero verification. Until last week there was no scanner at all.

SkillSpector is exactly what it says on the box. It scans agent skills before you install them.

The research backing this tool found 26.1% of all published public skills contain at least one vulnerability. 5.2% are actively malicious. Skills that include executable scripts are 2.12x more likely to be dangerous.

It runs 64 detection patterns across 16 categories. This includes the obvious stuff: prompt injection, data exfiltration, privilege escalation. It also catches things no one was checking for: hidden instructions in zero width characters, system prompt exfiltration, memory poisoning, MCP permission mismatches.

Scanning takes ~200ms per skill for static analysis. You can optionally enable LLM semantic validation which brings precision up to ~87% and eliminates most false positives. It outputs SARIF reports so you can drop it directly into existing CI pipelines.

This should be mandatory anywhere agents are allowed to install third party skills. There is no alternative right now.

Limitations: It does not do dynamic analysis. It will not catch vulnerabilities that only trigger at runtime. It will also not scan obfuscated or compiled code.

SIA: Self-improving agents that actually work

Most self improving agent demos are fraudulent. They loop on the same prompt, make cosmetic changes, and report a fake improvement score.

SIA is not that.

This framework implements a proper three agent improvement loop. A meta agent generates an initial task agent. The task agent runs the benchmark. A feedback agent reviews the execution trace, identifies failures, and modifies the task agent code directly. This repeats until performance stops improving.

The results are not trivial. On LawBench it improved top 1 accuracy from 45% to 70.1%. On the AlphaFold 3 Triton kernel benchmark it produced a 14x speedup over the human written baseline. On MLE-Bench Hard it currently holds the number 1 rank across all tested agent systems.

You do not have to use their agents. You can plug in any base model, any agent runtime, any custom benchmark. The loop is completely agnostic. It will output full diffs, execution logs and improvement rationale for every generation. There is a built in dashboard that plots performance over time.

This is the first self improvement system that you can run on your own tasks, get real measurable gains, and audit every single step that happened.

It is also very slow. A 5 generation run will take between 2 and 8 hours depending on the task. That is fine. This is not for chat. This is for taking a task you need to run 10000 times and making it 50% better overnight.

FunASR: Stop using Whisper for production

Whisper was a breakthrough. It is also obsolete for production use.

FunASR runs 13x faster than Whisper large v3 on GPU. It runs 17x realtime on CPU. That is not a typo. FunASR will transcribe audio faster on a cheap CPU instance than Whisper will run on an A10G.

It has built in VAD, punctuation, speaker diarization and emotion detection. All of this happens in one API call. There is no gluing together 3 separate models and praying the timestamps align.

It supports 50+ languages. There is an OpenAI compatible API endpoint. It works with every agent framework. There are official MCP bindings.

Almost every major company running speech at scale has already switched. Most just have not talked about it publicly.

Use SenseVoiceSmall for most use cases. Use Fun-ASR-Nano if you need maximum accuracy across languages. Only use Whisper if you specifically need translation.

LiteLLM: The default LLM gateway

LiteLLM won.

There were half a dozen open source LLM gateways 12 months ago. Every single one has either been abandoned or is now just wrapping LiteLLM.

It supports 100+ LLMs and providers. Every single one is exposed through the exact same OpenAI compatible interface. You can swap OpenAI for Anthropic for Gemini without changing a single line of application code.

The proxy deployment has virtual keys, per user spend tracking, rate limiting, load balancing, fallbacks, caching and guardrails. It does 8ms P95 latency at 1000 requests per second.

If you are building anything that calls more than one LLM provider, you should be using LiteLLM. There is no good reason to write provider specific code anymore.

The only valid complaint is that the documentation is messy. You will have to read the source code for edge cases. That is the price you pay for something that actually works.

Claude Agent SDK: In-process MCP changes everything

Anthropic released this SDK two weeks ago and almost no one noticed the important part.

You can now run MCP servers directly inside your Python process. No subprocess. No stdio. No IPC overhead. You decorate a normal Python function with a @tool decorator and Claude can call it.

This is a very big deal. All previous MCP implementations required running separate processes, managing lifecycles, serializing every call over pipes. This removes all of that overhead. You can now extend Claude with native code that runs in the same address space as your application.

You also get hooks that run before every tool call. You can write arbitrary Python code that inspects, approves, modifies or blocks any action the agent attempts to take. This is the first proper permission system that actually works for agents.

This SDK is currently the best way to embed an agent inside an existing application. Everything else feels like a bad hack by comparison.

MLflow finally got good for LLMs

Everyone hated old MLflow. It was built for tabular models, it was slow, it was overcomplicated, it did nothing useful for LLMs.

That changed.

The latest MLflow release has native OpenTelemetry based tracing for every major LLM provider and agent framework. You enable it with one line of code. You get full traces, token counts, cost, latency and error rates for every request.

It has 50+ built in evaluation metrics. It has prompt versioning. It has an LLM gateway. It integrates with every tool you are already using.

Most importantly: it works. You can spin up a server in 30 seconds and get full observability for your existing application without rewriting anything.

This has very quickly become the default open source LLMOps stack. If you were holding off on MLflow before, it is time to look again.

The free API keys repo

There is a repository that currently has 47,000 stars that just publishes working free LLM API keys.

Every key has $20-$100 credit. They work with every major model. They are OpenAI compatible. They are refreshed multiple times per day. No signup. No credit card.

This repository exists because the official onboarding flow for every LLM provider is completely broken. Students, hobbyists and developers in most countries cannot get access to these APIs at any price.

This is not a hack. This is not stolen keys. The maintainer is just reselling spare capacity at zero margin. 120,000 people use these keys every day.

If you want to understand how broken the LLM API market is right now, this repository tells you everything you need to know.

Common patterns across all these tools

Every one of these tools follows the same unwritten rules:

• They do one thing well
• They have no lock in
• They speak standard interfaces: OpenAI API, MCP, OpenTelemetry, SARIF
• They do not require you to rewrite your entire stack
• They ship with sensible defaults

Nobody is building grand frameworks anymore. Everyone is building compatible components that plug into the stack you already have.

What is still missing

There are still three large unsolved problems with no good open source solution:

1. Agent runtime authorization. There is still no good way to enforce least privilege for agent tool calls at scale.
2. Deterministic replay for agent traces. You still cannot reliably re-run an agent execution and get the same result.
3. Gradual rollout for agent changes. There is no standard way to canary test an agent update and measure impact.

All three will almost certainly be solved in the next 6 months.

Closing

This is the quiet good part of technology cycles. After the hype dies down, the actual engineers show up and build the boring tools that make things work.

None of these tools will go viral. None will get a press release. None will be presented at a keynote.

All of them will be running inside almost every production LLM system 12 months from now.