Every senior ML engineer building agents has had this exact experience. You demo your agent on turn 12. It works perfectly. Everyone cheers. You ship it. On turn 78 it quietly does exactly the thing you told it never to do. No error is logged. No alert fires. It writes a very polite, convincing explanation for why it did the wrong thing.

You did not break the model. You broke the runtime.

For the last 18 months almost all public discussion of agents has focused on prompt templates, model benchmarks and toy demos. Almost none has discussed the runtime layer that determines if an agent will run for 10 turns or 1000. This changed over the last two weeks, with seven independent papers and field reports landing that together define the actual state of production agent engineering.

This is not a survey. This is a synthesis of what works, what breaks, and what we now know for certain.

Scaffolds are not glue code

We used to think agents were LLMs with a tool loop. That was wrong.

AgentSpec makes this unambiguous. Agents are distributed systems. The LLM is just one component, and rarely the one that limits performance.

AgentSpec tested every common combination of memory, reasoning, reflection and action modules across four standard environments: DeliveryBench, ALFRED, MiniGrid and RoboTHOR. The single strongest predictor of success was not the quality of any individual module. It was interface compatibility between modules. A mediocre memory module paired correctly with a mediocre reasoning module will reliably outperform two state of the art modules connected with ad-hoc glue.

This is the most underrated result in agent research right now. Everyone is racing to build better reasoning modules. No one is standardizing the interfaces between them. You can take the best open source memory implementation, the best reflection loop, the best action executor, wire them together and you will get worse performance than a correctly integrated system running Llama 3 70B.

Interaction effects account for 62% of the performance variance measured across all AgentSpec trials. Individual module strength accounts for 18%.

If you are building an agent right now, stop tuning prompts. Go audit the interfaces between your components.

Parallel workflows do not need text

Every agent system built today runs parallel worker branches. You split a problem into subtasks. You run each one. Then you concatenate all the outputs and feed them back into a synthesizer LLM.

This is stupid. It is also universal.

Parallel-Synthesis demonstrates that you never needed to serialize those branches back to text at all. You can merge the KV caches directly.

The implementation is simple. A cache mapper calibrates position and attention offsets across independently generated branch caches. A small fine tuned adapter layer teaches the base model to read from this non sequential cache interface. Training data is generated by distilling the behaviour of standard text concatenation synthesis.

The results are not marginal. Time to first token drops between 2.5x and 11x. Accuracy stays the same or improves on 7/9 test datasets. There is no catch. This is just a better interface for agent workflows.

We have spent three years treating text as the universal intermediate format for agents. It never was. It was just the only interface the LLM exposed. Now we are starting to build native interfaces for the things agents actually do.

This will be standard in every agent runtime within 12 months.

Silent failure is not an edge case

If you have run an agent in production for more than two weeks you have already encountered fail-plausible.

The longitudinal study of production agent failures documents 22 incidents over 8 weeks running a personal assistant agent with 40 scheduled jobs, 8 LLM providers, 4286 unit tests and 827 governance checks. 28 separate times the system did not just fail silently. It transformed the error into a fluent, plausible, completely wrong narrative that was delivered to the end user.

No unit test will catch this. No governance check will catch this. 70% of these failures were first noticed by human users, not by any automated system.

The taxonomy of silent failures is worth memorizing:

1. Environment and platform quirks
2. Design assumption mismatches
3. Error swallowing and dilution
4. Chained hallucination and fabrication
5. Operational omission and forensic blind spots

Class 4 is unique to LLM systems. It is also the only failure mode in computing history that actively covers its own tracks. When this failure occurs, every log, every trace, every output from the system will lie to you.

You cannot test for this. You can only design your runtime so that failures are loud, ugly and obvious. If your agent ever looks like it is working perfectly, it is already lying to you.

Latent failures kill plans

Most agent benchmarks only test for immediate failure. They check if the first step runs. They do not check if step 12 will irreversibly break everything three hours from now.

SIMMER is the first benchmark that measures this class of failure. It implements a symbolic world model for the kitchen domain with 77 actions, 262 objects and 46800 verified possible interactions. It runs plans end to end without execution feedback, and flags failures that will not surface until mid execution.

Across six frontier LLMs, at most 17% of generated plans were completely error free. 56% of plans contained latent failures that would not surface until well after execution had begun. 31% of plans would cause irreversible damage.

Half of all plans that look correct when generated will destroy something later.

This is not a reasoning problem. This is a simulation problem. LLMs do not naturally run counterfactual checks. They will happily tell you to preheat the oven before you put the food inside. They will never stop to check what happens if you leave that oven running for 90 minutes while you go chop vegetables.

Explicit counterfactual simulation reduces latent failures by 72%. There is no other intervention that comes even close. If you are building a planner, you need a world model. There are no shortcuts.

Almost all memory systems do nothing

GitOfThoughts is the most honest paper published about agent memory to date.

The authors tested five different memory substrates: none, markdown, vector, graph, and their own git based implementation. Tests ran across two benchmarks, two model scales, with full pre registered replication.

No memory format reliably improved accuracy on novel problems.

None.

Memory only works above the copyability threshold. When the current problem is more than 80% identical to a previously seen problem, accuracy jumps sharply. Below that threshold, memory does not help. It often hurts.

All the impressive memory demos you have seen are demonstrations of answer retrieval. None are demonstrations of method transfer. No agent running today can extract a general principle from a past example and apply it to a new dissimilar problem. Even a 4.5x larger model cannot do this.

This is not an implementation problem. This is a fundamental capability limit of current models. You can stop building better vector databases.

The only valid reason to build an agent memory system right now is auditability, provenance and debuggability. It will not make your agent smarter. It will let you find out why it was stupid.

You must teach your agent to forget

The single worst mistake you can make building a long running agent is trying to keep everything.

Context window is RAM, not disk. It is a desk. If you keep piling paper on it eventually you will not be able to find the one sheet that matters.

Compaction is not summarization. Compaction is deliberate, selective forgetting.

Good compaction keeps exactly six things:

1. What was decided, and the why behind it
2. Current state of files and external systems
3. Explicit user constraints and preferences
4. Work currently in progress
5. Concrete next steps
6. Critical reference identifiers

Everything else gets deleted.

If you run naive summarization you will get cumulative erosion. Every compaction loses a little bit of signal. After five compactions your agent is working from a summary of a summary, and the constraint you stated at turn 3 is gone forever.

Trigger compaction at 85% context utilization, not 95%. Prune raw tool output before you run the summarizer. Keep the last 10 turns completely verbatim. Never paraphrase user constraints.

And tell the user when you did it. A silent compaction is not an optimization. It is a behavioural discontinuity.

Reflection does not work the way you thought

Everyone adds a reflection loop to their agent. Almost no one measures if it helps.

RePro demonstrates that online reflection during execution consistently hurts performance. Retrospective reflection after completing a trajectory consistently helps.

Agents cannot judge their own progress while they are in the middle of a task. They can judge it perfectly once they know how it ended.

This is a very human result. You cannot accurately evaluate how well you are doing halfway through a problem. You can look back afterwards and see every mistake immediately.

All existing reflection implementations are doing it backwards. They are asking the agent to judge progress while it is acting. You should instead store full trajectories, then run reflection offline and use that signal to train future behaviour.

RePro delivered up to 12% absolute success rate gains across WebShop, ALFWorld and Sokoban. No other training modification came close.

Failures live in the seams

Across every study, every benchmark, every production postmortem, one pattern repeats without exception.

The longest lived failures do not live inside components. They live in the seams between components.

Incident latency, the time between a failure being introduced and it being detected, tracks failure location not code complexity. Failures inside a module are found in on average 13 hours. Failures at the interface between two modules live an average of 11 days. Failures at the intersection of three or more modules live an average of 60 days.

No test runs in the seams. No audit runs in the seams. No logging is enabled in the seams. This is where every catastrophic agent failure will happen.

The unspoken rule of agent engineering

There is one pattern that runs through every one of these papers and every production failure.

None of the hard problems in agent engineering are LLM problems.

All of the hard problems are distributed systems problems. They are interface design problems. They are observability problems. They are state management problems. They are exactly the same problems engineers have been solving for 50 years.

We got distracted by the magic of the model and forgot everything we already knew about building reliable systems.

Agents are not AI research. Agents are software engineering.

The teams that win at this will not be the teams that have the best model. They will be the teams that are best at building boring, reliable, observable runtimes.

That is the secret no one will tell you. The agent revolution will not be won by prompt engineers. It will be won by backend engineers.