The standard ReAct loop is brittle. Agents act before they explore, duplicate effort instead of assembling complementary evidence, game evaluation metrics without fidelity checks, and cannot search their own architecture space effectively. Four papers this week propose structural fixes for each failure mode. Together, they make a clear case: capable autonomous agents need deliberation infrastructure, not just more compute.

The premature exploitation problem

Look Before You Leap: Autonomous Exploration for LLM Agents formalizes something practitioners have observed but never cleanly named: premature exploitation. LLM agents trained with standard task-oriented RL tend to act on prior knowledge before gathering enough environment-specific information. In familiar settings this works. In novel environments, it produces narrow, repetitive behavior that misses critical affordances entirely.

The paper introduces Exploration Checkpoint Coverage (ECC), a verifiable metric that quantifies how broadly an agent discovers key states, objects, and affordances. ECC measures exploration quality independently of task success, which matters because task rewards alone do not produce good explorers. Their systematic evaluation confirms this: agents trained with standard task-oriented RL consistently exhibit narrow behavioral patterns. They find the nearest solution path and hammer it, ignoring large portions of the state space.

The fix is a training strategy that interleaves two rollout types. Task-execution rollouts are optimized with task rewards. Exploration rollouts are optimized with exploration rewards. Each gets its own verifiable signal. This yields the Explore-then-Act framework: agents first spend an interaction budget acquiring grounded environmental knowledge, then leverage that knowledge for task resolution.

Exploration is a trainable skill that requires its own reward signal. You cannot implicitly learn to explore by only rewarding task completion. The decoupling is what makes transfer to unfamiliar environments possible.

Why parallel rollouts hit diminishing returns

Argus: Evidence Assembly for Scalable Deep Research Agents addresses a different scaling problem. Current deep research systems scale inference-time compute by running parallel search trajectories and aggregating results. The trouble is that parallel rollouts tend to duplicate effort rather than finding complementary evidence. Five agents find the same source, not five different sources. Returns diminish while the aggregation context swells toward the model's limit.

Argus reframes deep research as assembling a jigsaw puzzle. The answer is composed of complementary evidence pieces, and the goal is to find the missing pieces, not duplicate the ones already in hand.

The system has two components. A Searcher collects evidence traces for a given sub-query through standard ReAct-style interaction. A Navigator maintains a shared evidence graph, verifies which pieces are still missing, dispatches Searchers to gather them, and reasons over the completed graph to produce a source-traced final answer.

Both components train independently. The Navigator uses reinforcement learning to learn verification, dispatch, and synthesis. The Searcher remains a standard ReAct agent. The Navigator supports rollouts with one Searcher or many in parallel, with no retraining when you change the Searcher count.

Both are built on a 35B-A3B mixture-of-experts backbone. With a single Searcher, Argus gains 5.5 points averaged over eight benchmarks. With 8 parallel Searchers, it gains 12.7 points. With 64 Searchers it reaches 86.2 on BrowseComp, surpassing every proprietary agent benchmarked. The Navigator's reasoning context stays under 21.5K tokens throughout.

That 21.5K figure matters more than it seems. Most approaches to coordinating multiple agents either pass full outputs between agents, which blows up context, or use a fixed summarization pipeline, which loses detail. The evidence graph is a third option: a structured intermediate representation that preserves what matters (which evidence exists, which is missing, which queries are dispatched) while discarding what does not (raw search traces, duplicate findings). Scaling from 1 to 64 Searchers without retraining the Navigator is a direct consequence of this design.

Tree search with guardrails for scientific modeling

Prospective Multi-Pathogen Disease Forecasting using Autonomous LLM-Guided Tree Search applies agentic tree search where the stakes are concrete: infectious disease forecasting. The system iteratively generates, evaluates, and optimizes executable forecasting software using LLM-guided tree search.

In a fully prospective, real-time evaluation during the 2025-2026 US respiratory season, the system autonomously discovered methodologically diverse models for influenza, COVID-19, and RSV. Aggregating these machine-generated models produced an ensemble that consistently matched or outperformed the CDC's gold-standard human-curated hub ensembles out-of-sample. It also handled data-scarce "cold start" scenarios for RSV, where limited historical data makes model initialization difficult.

Two controlled retrospective ablations deserve attention. First, optimizing log-scale distance metrics prevents reward hacking. When the system optimized on raw distance metrics, it found ways to game the evaluation. Log-scale metrics closed that loophole. Second, an automated judge-in-the-loop ensures structural fidelity to complex scientific theories. The LLM generates code, but the judge verifies that the code actually implements the epidemiological theory it claims to implement, not a superficially similar shortcut.

Without the judge, the tree search optimizes for surface-level evaluation metrics rather than genuine scientific validity. This is the same pattern you see in any optimization system: it will find the shortest path to high reward, and if that path involves shortcuts that look correct but violate domain constraints, it will take them. The judge-in-the-loop is the mechanism that keeps the search honest.

The system does not just generate plausible code. It generates code that passes epidemiological fidelity checks and outperforms human-curated ensembles on real-world forecasting tasks. That is a high bar. CDC hub ensembles represent the aggregated effort of multiple expert modeling teams, and matching them out-of-sample is not a benchmark trick.

Agents designing their own architectures

Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design pushes furthest into speculative territory. The question: can LLM agents autonomously design foundation model architectures that match or beat hand-designed baselines?

AIRA-Compose handles high-level architecture search. It deploys 11 agents to explore fundamental computational primitives under a 24-hour budget. Agents evaluate million-parameter candidates and extrapolate top designs to 350M, 1B, and 3B scales. This produces 14 architectures across two families: AIRAformers (Transformer-based) and AIRAhybrids (Transformer-Mamba hybrids).

Pre-trained at 1B scale, these architectures consistently outperform Llama 3.2 and Composer-found baselines. On downstream tasks, AIRAformer-D improves accuracy by 2.4% over Llama 3.2. AIRAhybrid-D improves by 3.8%. Two-point-four percent at 1B scale is a real margin, not a rounding error.

The scaling efficiency results are more striking. AIRAformer-C scales 54% faster than Llama 3.2 and 71% faster than Composer's best Transformer. AIRAhybrid-C outscales Nemotron-2 by 23% and Composer's best hybrid by 37%. An architecture that is slightly more accurate and substantially more compute-efficient is exactly the kind of result that matters in practice, where training compute is the binding constraint.

AIRA-Design handles low-level mechanistic implementation. It tasks 20 agents with writing novel attention mechanisms for long-range dependencies and high-performing training scripts. On the Long Range Arena benchmark, agent-designed architectures reach within 2.3% and 2.6% of human state-of-the-art on document matching and text classification respectively. On the Autoresearch benchmark, Greedy Opus 4.5 achieves 0.968 validation bits-per-byte under a fixed time budget, surpassing the published minimum.

The dual-framework approach is architecturally clean. AIRA-Compose searches the space of high-level computational primitives. AIRA-Design searches the space of low-level mechanisms. Together they cover the full design stack. The multi-agent decomposition is what makes the search space tractable: 11 agents exploring architecture space in parallel under time pressure can cover more ground than a single agent iterating sequentially.

The results are not yet at the point where agent-designed architectures dominate human designs across all metrics. But they are competitive, and in scaling efficiency they are clearly ahead. Whether this gap continues to close as agent capabilities improve is an open question, and an important one.

Deliberation as infrastructure

Read these four papers together and a pattern emerges. Each identifies a specific failure mode in naive agent architectures and proposes a structural intervention.

Look Before You Leap shows that agents need explicit exploration mechanisms trained with their own reward signals. Without exploration, exploitation is blind.

Argus shows that parallel scaling needs a coordination layer that tracks what evidence has been found and what is still missing. Without it, parallelism is expensive duplication. The evidence graph is the key data structure.

The disease forecasting paper shows that tree search for scientific model generation needs guardrails: log-scale metrics to prevent reward hacking, and a judge-in-the-loop to enforce structural fidelity. Without these, the search optimizes for the wrong objective.

AIRA shows that architecture search can be decomposed into high-level composition and low-level design, with agents operating at each level. The decomposition makes the search space tractable.

The common thread is deliberation infrastructure. These systems do not just act. They explore before acting, assemble evidence before synthesizing, verify before committing, and decompose before searching. Each paper adds a specific piece of infrastructure that makes deliberation tractable for LLM agents.

This is a departure from the "just add more compute" school of agent design. More parallel rollouts, longer ReAct chains, bigger context windows. These help at the margin, but they do not fix structural deficits. The next generation of capable agents will need structured mechanisms for exploration, coordination, verification, and decomposition.

There are reasonable concerns. These papers show that structured deliberation works, but they do not show it is easy to set up. Each requires careful reward design, training pipelines, and domain-specific infrastructure. The AIRA results are at 1B scale, and it is unclear whether agent-designed architectures maintain their advantages at 70B+ scales where training runs cost millions. The disease forecasting system matched CDC ensembles, but CDC ensembles themselves are aggregates of human models, so the comparison baseline, while strong, is not a ground truth.

What holds across all four papers is the engineering lesson: naive agent loops fail in predictable ways, and the fixes are structural, not scalar. You do not solve premature exploitation by increasing the action budget. You solve it by adding an exploration phase with its own reward. You do not solve evidence duplication by adding more parallel agents. You solve it by adding a coordination layer that tracks the evidence graph. You do not solve reward hacking by tightening the reward function. You solve it by changing the metric and adding a judge. You do not solve architecture search by giving a single agent more time. You solve it by decomposing the search across multiple agents with different scopes.

The pattern is consistent enough to be actionable. If you are building an agent system and it is failing, the first question to ask is not "can I scale compute?" It is "what structural mechanism am I missing?"