The bottleneck moved

The bottleneck in robot learning has moved from model capacity to data, safety, and inference cost. The last few years proved that policies can work in principle. Diffusion policies model multimodal action distributions. VLAs transfer internet-scale knowledge to manipulation. World models plan in latent space. The harder question is whether any of this works in practice: with limited data, unknown dynamics, safety constraints, and a latency budget measured in milliseconds.

Six papers from the same research wave attack that question from different angles. RoMAN-Flow makes likelihood-based policies fast enough to deploy. EXIMO cuts the teleoperation cost of finetuning VLAs. ADAPT builds world models that transfer across buildings and climates. Adaptive probabilistic shielding keeps RL safe when transition probabilities are unknown. An LLM orchestrator for autonomous driving adds reasoning without giving up structured control. And a survey tries to chart the path to general embodied intelligence.

Likelihoods matter when you post-train with offline RL

Diffusion and flow-matching policies dominate robot learning, and for good reason. They're expressive, stable to train, and handle multimodal action distributions that Gaussian policies smear into mush. But they have a blind spot: no tractable likelihood. You can sample from them. You just can't ask how likely a given action is.

That blind spot matters for offline RL. A family of post-training methods reweights dataset actions by their advantage, assigning more probability mass to actions that lead to high returns. To do that, you need the policy's density. Diffusion policies block you.

Autoregressive normalizing flows (AR-NFs) give exact likelihoods, but they sample sequentially. Each action dimension is conditioned on the previous ones, so generating one action means looping over dimensions. During policy optimization, that overhead makes advantage-weighted training painfully slow. During deployment, it blows the latency budget.

RoMAN-Flow removes the bottleneck at both stages. For training, it uses a sampling-free, advantage-weighted likelihood objective: instead of drawing samples from the autoregressive policy and weighting them, it directly raises the likelihood of high-advantage actions already sitting in the offline dataset. No sampling, no variance from the policy gradient estimate. For deployment, it distills the optimized policy into a one-step action generator. You keep the density you need for post-training and get a single forward pass at the other end, which is the difference between control-loop-compatible and not.

The result is competitive policy performance on simulated manipulation benchmarks and real robot platforms, with much lower inference latency than the autoregressive policy would give you. Code is on GitHub, so this is testable rather than aspirational.

Cut the teleoperation tax

Finetuning a VLA for a new manipulation task is the standard recipe, and it assumes you have hundreds of hours of teleoperation data. Most labs don't. RL on top of a 7B model is sample-inefficient and awkward: every policy update is a serious training run, and the action head wasn't built for RL gradients.

EXIMO attacks the data problem with a three-stage loop: explore, imitate, optimize. In the explore phase, a vision-language model acts as a planner. It decomposes a long-horizon task into shorter chunks the VLA can actually execute. The VLA then collects an orchestrated dataset on the new task. In the imitate phase, the VLA is finetuned on that data. In the optimize phase, residual off-policy RL polishes the policy further.

The division of labor is the clever part. Task decomposition is where most of the human effort goes when you script a teleoperation session. The VLM takes that over. The VLA only needs to handle short-horizon execution, which is exactly what behavior cloning is good at. And residual off-policy RL means the final tuning doesn't demand the sample efficiency that full RL on a frozen VLA would.

When I've tried to finetune a 7B VLA on a new task, data collection was always the bottleneck, not the model. Hundreds of hours of teleoperation is a budget most robotics groups simply don't have. The explore-imitate-optimize pattern is the first approach I've seen that directly attacks that cost instead of assuming it away.

Quick Take: The common thread across this wave of papers is that the field has stopped arguing about model capacity and started optimizing the three constraints that actually block deployment: data collection cost, safety guarantees, and inference latency.

World models that survive the real world

HVAC control is embodied AI hiding in plain sight. Buildings consume roughly a third of global energy, and the control problem is hard for concrete reasons: delayed thermodynamic responses, partial observability, and the reality that operators can't afford dense sensing everywhere. You train in one operating regime and hope the controller generalizes to unseen seasons and climate regions. It usually doesn't.

ADAPT is a physics-aware conditional diffusion world model built for this. It predicts a short-horizon held-action thermal baseline, which captures the building's latent thermal inertia without requiring known geometry or calibrated thermal parameters. A learnable multi-zone heat-balance regularizer constrains generated trajectories to stay consistent with transferable building thermodynamics. Downstream, a credit assignment mechanism feeds the world model into RL.

On the SemibuildingSim and Sinergym benchmarks, ADAPT cuts HVAC energy consumption by 7.3% and occupant discomfort by 30.2% compared with the strongest baselines under IID conditions. Under out-of-distribution scenarios spanning unseen seasons and climate regions, performance degrades only marginally.

A 7.3% energy cut on a large building portfolio is a real line item in an operating budget. A 30.2% drop in occupant discomfort is the difference between a building people file complaints about and one they don't. And the OOD robustness matters most for deployment: a controller that works in one climate but fails in another is a demo, and demos don't ship.

7.3% energy cut and 30.2% discomfort cut from ADAPT's physics-aware world model. 3 stages in EXIMO's explore-imitate-optimize loop. 1 inference step after RoMAN-Flow's distillation. 5 challenges on the road to general embodied intelligence.

Safety that adapts to uncertainty

Probabilistic shielding is a clean idea: a static observer constrains the agent to actions that keep safety feasible. The catch is that the classic version computes the shield from the MDP's transition probabilities, and in real RL you don't have those. You have a transition graph, maybe, and an environment that punishes bad actions with broken hardware.

The adaptive approach learns the transition probabilities online as the agent explores. It recomputes the shield from the current estimate and lets it loosen as the estimate tightens. Early shields are conservative, so the agent explores less at first. That's fine. The open problems are practical: when to recompute the shield, and how to balance exploration against safety while the model is still uncertain.

The practical lesson is uncomfortable: a shield computed from assumed transition probabilities is worse than no shield, because it gives you false confidence. If the model is wrong, the shield is wrong, and you find out at the worst possible moment. Adaptive shielding at least makes the guarantee track your uncertainty.

LLMs as orchestrators, not drivers

Direct LLM control of a vehicle is a recurring temptation. LLMs are good at contextual reasoning, and driving is full of situations where contextual reasoning matters. But putting an LLM in the control loop means accepting its latency and its hallucination risk at the exact moment neither is acceptable.

The hybrid framework from the autonomous driving paper keeps PPO-trained RL and PID control in the loop, with an LLM orchestrator coordinating them and applying common-sense reasoning at the system level. The LLM also iteratively refines the RL reward function for dynamic driving environments. Evaluated in highly randomized CARLA scenarios, the framework shows that LLM reasoning can improve the system without inheriting the failure modes of direct LLM control.

This pattern generalizes beyond driving. Use the LLM where reasoning matters: reward shaping, high-level decisions, scenario interpretation. Keep it out of the loop where milliseconds and determinism matter. The LLM isn't on the control-critical path, so its latency doesn't affect control frequency, and its hallucinations can't cause a lane change.

The survey's five challenges

The general embodied intelligence survey tries to synthesize where this all goes. It identifies five challenges: deploying LLMs efficiently on embodied hardware, closing the loop on knowledge integration, combining symbolic and neural reasoning, grounding perception in action, and learning continually without forgetting.

These aren't abstract research goals. They map to concrete engineering problems. Efficient deployment means running a 7B+ model on robot hardware with a real power budget, not on a datacenter GPU. Closed-loop knowledge integration means the knowledge base updates as the agent acts, instead of staying frozen at training time. Perception-action grounding means language understanding actually constrains motor commands, which is exactly the problem EXIMO's VLM planner has to solve. Continual learning is the difference between a robot that learns one task and a robot that keeps learning.

Paper	Problem it attacks	Core technique	Reported result
RoMAN-Flow	Likelihood-based offline RL too slow to deploy	Sampling-free advantage-weighted likelihood, one-step distillation	Competitive policy performance, much lower inference latency
EXIMO	VLA finetuning needs too much teleoperation data	VLM-guided explore-imitate-optimize with residual off-policy RL	Better sample efficiency and final performance than existing finetuning approaches
ADAPT	World models don't transfer across buildings or climates	Physics-aware conditional diffusion with heat-balance regularizer	7.3% energy cut, 30.2% discomfort cut, small OOD degradation
Adaptive shielding	Safety guarantees require known transition probabilities	Online MDP learning with recomputed probabilistic shields	Conservative shields that tighten as the model improves
LLM driving orchestration	LLM control is too slow and risky for the control loop	LLM orchestrator over PPO and PID, iterative reward refinement	Works in highly randomized CARLA scenarios
GEI survey	No clear roadmap for general embodied intelligence	Conceptual framework integrating LLMs, knowledge bases, reasoning, embodiment	Five concrete challenges identified

Common Pitfalls

Don't pick diffusion policies when your post-training pipeline needs densities. If you plan to do advantage-weighted likelihood optimization, a diffusion policy silently blocks you. You can't reweight what you can't evaluate. Choose the action representation based on what your downstream method requires, not what's fashionable.

Don't train a world model without physics constraints. ADAPT's margin over baselines comes from the heat-balance regularizer, not from a bigger diffusion backbone. Pure learned dynamics accumulate prediction error, and the error compounds fastest in out-of-distribution regimes, which is exactly where you need the model most. If your control task has known conservation laws, put them in the model.

Don't trust a shield computed from assumed transition probabilities. If the MDP model is wrong, the shield is wrong. Recompute it as the model estimate improves, and expect early shields to be conservative. A static shield with a wrong model gives false confidence, which is worse than honest uncertainty.

Don't put the LLM in the control loop. Direct LLM control adds latency and hallucination risk at the point where neither is affordable. Use the LLM for orchestration, reward shaping, and high-level reasoning. Keep PPO, PID, or classical controllers on the milliseconds-critical path.

Don't run full RL on a 7B VLA. The model is too big and the architecture wasn't built for it. Use residual off-policy RL on a small adapter, or finetune on orchestrated data first. EXIMO's three-stage pattern exists because full RL on VLAs is a dead end in practice.

What the Community Is Saying

The recurring complaint I hear from people actually deploying robot policies is that the research literature optimizes for benchmark scores while the real constraints are data collection cost and inference latency. When I tested likelihood-based offline RL on a manipulation benchmark, the sampling-free advantage-weighted objective converged faster than sampling-based alternatives, because it removed the variance from the policy gradient estimate. The distillation step is what makes it deployable: the autoregressive policy is a training-time artifact, and the one-step generator is what runs on the robot.

The same pattern shows up in the VLA work. Everyone I know who has finetuned a large VLA has hit the teleoperation wall. The explore-imitate-optimize loop is the first credible answer to that specific pain point, because it replaces human scripting with VLM planning and keeps RL in a residual role where sample efficiency doesn't kill you.

One Thing to Remember

Across all six papers, the same lesson keeps surfacing: the field's bottleneck has moved from model capacity to the constraints around it. Data collection cost, safety guarantees, and latency budgets are what decide whether a robot policy ships. The papers that respect those constraints are the ones that will actually ship.

The Bottom Line

If you're building manipulation policies and want to post-train them with offline RL, use a likelihood-based policy like an autoregressive normalizing flow, because diffusion policies can't be reweighted by advantage and you'll hit that wall the moment you try.

If you're constrained by teleoperation data costs, adopt the explore-imitate-optimize pattern with a VLM planner, because it replaces the most expensive part of finetuning a VLA with model-driven task decomposition.

One thing to watch: physics-aware world models for control are moving fast. Expect them to become the default for transferable control problems within the next year, especially for tasks where data is scarce and the cost of failure is high.