One Architecture, Everywhere

Four unrelated robot stories landed in the same week. Unitree released a world-action model, UnifoLM-X2-1.0, that lets a G1 humanoid fight without a remote control. JD opened a coffee shop in Beijing where a robot arm hands finished drinks to a pickup locker. A developer wired a desktop Reachy Mini into his home agent and cut its answer latency from 15 seconds to about 3. And two arXiv papers arrived, one on why visuomotor policies grab the wrong object, one on how to let an LLM refine robot policies without wrecking local learning.

They are the same story. Robot control is splitting into two layers. A slow layer decides what to do using goals, language, and world knowledge. A fast layer moves the joints. The split shows up everywhere with different names. Figure's Helix 02 has a top layer that understands the environment, a middle layer that turns judgments into whole-body actions, and a bottom layer for balance and contact. The arXiv navigation paper calls it round-level policy reasoning versus tick-level action selection. The living-room robot calls it an intent allowlist in front of a lean agent. The coffee shop calls it a digital recipe in front of a production line.

The two layers are easy to draw. The seam between them is the hard part. This week produced unusually concrete evidence about where that seam breaks and what actually fixes it.

The Grounding Failure Nobody Puts in the Demo Reel

Visuomotor imitation policies look great in the lab. Train Action Chunking with Transformers (ACT) on a tabletop task and you get high success under in-distribution conditions. Then add a distractor cup that looks like the target cup, and the policy reaches for the wrong one. Add a visually similar receptacle and the placement lands in the wrong slot. The trajectory skill is intact. The policy just selects the wrong visual target.

The paper treats this as a problem of conditional visual grounding. The visual target needed for successful control changes with the manipulation phase: in the picking phase the target is the object, in the placing phase it is the receptacle. In more complex tasks it also changes with the observed task state. The authors introduced distractor objects and receptacles with controlled color and shape similarity, then localized the failures to picking and placement.

Two findings matter. Distractor sensitivity is specific to the type of visual similarity, and it is specific to the manipulation stage. A policy can handle picking fine while fumbling placement when only a similarly shaped receptacle appears. The manipulation skill survives; the selection fails.

That diagnosis points to the fix. If the skill is intact and the selection is broken, supervise target selection directly. The paper evaluates three complementary interventions.

Intervention	What it targets	Where it recovered performance
Distractor augmentation	Adds visually similar objects to training scenes	Picking in simulation and on the physical UR3e
Phase-dependent attention regularization	Stops attention from drifting in the pick phase vs the place phase	Placement errors that augmentation alone missed
Appearance-based visual prompting	Shows a pretrained policy what the target looks like	A state-conditioned instrument-handling task with a VLA policy

The result: all three improved robustness substantially in simulation and on the physical UR3e arm. The same failure pattern appears in a pretrained vision-language-action policy on an instrument-handling task, where the observed state of a medical instrument determines the correct destination.

The practical translation for anyone training these policies: a cluttered home breaks these policies at target selection. More trajectory data will not fix it. If the skill survives but the selection fails, the policy needs to know what counts as the target, and it needs that supervision at the right phase.

LLMs at Round Speed, Not Tick Speed

The second paper attacks the seam from the other side. A team of heterogeneous robots navigates a decentralized system. Each robot combines an LLM policy agent, a UCB bandit, and a Double DQN controller. Over 30 rounds, the full configuration reached the goal in all 90 correlated robot-round records, with a median completion time of 42 ticks in the NetLogo-Python simulation and a P90 of 73.2 ticks. Its median was 25% to 39% lower than every other configuration.

The architecture choice is the finding, not the model choice. LLM inference runs only at round-level policy generation and refinement. It never touches tick-level action selection. The UCB bandit decides when a policy refinement is worth trying. The policy-conditioned Double DQN then picks actions at tick level using navigation variables, active policy parameters, and the LLM's action prior. The robots coordinate through a shared round summary of policies, outcomes, and learning feedback. There is no central LLM generating team actions.

This division holds up because no one wants a language model at control frequency. Tokens take hundreds of milliseconds at best, and a control loop running at tens of hertz cannot wait for the first word. Worse, an LLM that replans mid-round fights the local learner, overwriting what the Double DQN just updated. The schema bound is exactly what keeps refinement from destabilizing local learning. The LLM writes proposals at round speed, the bandit gates them, and the controller executes them at tick speed.

Every team that shipped something this week drew this line somewhere. The ones who defended it got speed and stability. The ones who ignored it got a 15-second weather answer.

Quick Take: the robot stories from this week, a coffee machine, a boxing ring, a living room, are one architecture story: split the thinking layer from the moving layer, then spend your engineering budget on the seam between them.

The Living Room Robot That Forced the Broker Pattern

I put a Reachy Mini in my living room. It is a small desktop robot from Pollen Robotics, now part of Hugging Face: a head on a body, two antennas, a camera, a speaker, and a microphone array. Inside is a Raspberry Pi CM4 running a daemon that exposes motors, audio, and camera over an HTTP API on port 8000. Out of the box it waves, dances, and follows faces.

The obvious integration is to make the robot a client of my agent. Install an app that captures audio, sends the transcript to my OpenClaw gateway, and speaks the answer. I got that working and tore it apart for two reasons.

The robot is not a machine I can trust. I checked rather than assumed: fetch /openapi.json off the robot and you get 100 endpoints and zero security schemes. Anything on the LAN can drive the motors or open the camera. Put the agent on the robot and the robot holds a gateway token, and that token reaches an agent with my calendar, my house, and my shell. Not acceptable.

The second reason showed up in the audit trail. A general question was a full agent turn: a system prompt around 30k tokens carrying tool profiles and a skills index, plus a persistent session that had grown to 51k tokens, plus a reasoning model spending about 500 tokens of thought before its first word. Median 15 seconds to answer "what's the weather today?". At 15 seconds nobody in the room waits. They reach for a phone.

The shape I landed on is a broker. One rule drives the whole design: the robot is untrusted, and everything that could leak stays behind the security I already invested in my agent setup. The robot runs a single app that does wake-word matching, voice activity detection, motion, and expressions, and holds exactly one bearer token scoped to the broker. It never sees a credential, a model, or a calendar entry. Audio goes up, a policy and a reply come down.

The broker is a small Python HTTP server on the Mac. It is the only path between the robot and my data. If the robot were fully compromised tomorrow, an attacker gets the intent allowlist and nothing else.

The intent allowlist is a closed set of things a shared-room robot may ask for. The robot cannot phrase a request. It names an intent and passes typed arguments. Free text reaches exactly one intent, general.ask, which is also the least privileged: no house access, no calendar, no files. Routing uses rules, not a model. An LLM router would add a second model round trip to every turn, and it can be talked into picking a different intent by whatever is said in the room. The doctrine is one line: an imperative actuates, a question never does, and a command that names no known room asks which one.

Redaction also happens at the broker, before a word is spoken. My calendar status is coarse: busy or free, until when, and a whereabouts word from a closed vocabulary. Never event titles, locations, or attendees.

The latency fix was mostly a prompt fix. I gave the robot its own agent with no skills, no coding tools, no workspace injection, thinking and reasoning off, memory search disabled. Every setting removes tokens. The same question that measured 15 seconds now measures about 2.9 seconds for the model leg and 3.4 to 3.9 seconds for the whole turn including speech synthesis, at roughly 5k prompt tokens instead of 30k. About a tenth of the cost. House questions and house commands never leave the Mac. General questions go to the LLM through the gateway and never carry a fact about my family.

What I did not expect is that drawing the trust boundary made the fun parts possible. Once the robot could only name an intent, I stopped worrying about what it might be talked into doing and started adding games and ambient motion that makes the kids laugh. None of it needed a new trust decision, because there is only one, and it is enforced in a single file.

The surrounding community is pushing the same edge stack forward. Around the same week a builder posted two Gemma 4 12B models holding a voice conversation, one on an RTX PRO 4500 and one on a Jetson Orin NX 16GB, with lip sync, expressions, and gestures, all open source. The detail that impressed me: the inference engine is a small C library that beats llama.cpp on the Jetson and shows no degradation after long voice prompts. A 12B model with a face on an 8GB edge board is the kind of thing that makes the living-room setup feel less like a hobby.

From the Coffee Shop to the Boxing Ring

The same week the living room got a broker, commercial deployments were running the same split at industrial scale. The cleanest example is not a flashy demo. It is a 24-hour coffee shop.

Seven Fresh Coffee opened in Beijing's Galaxy SOHO on August 16. There is no barista and no counter. A robot arm picks up the finished cup and places it into one of about 100 locker slots. On opening day it served 202 cups in one hour, a Guinness world record. CBS and Australia's 7NEWS covered the shop. The robots are the visible part. The system behind them is a full loop: AI scans market trends and user reviews to propose flavors, R&D turns a winning concept into a digital recipe, the automated line executes it, the arm delivers it, and sales data feeds the next round of decisions.

The numbers that matter for operations: 30 seconds average per drink, up to 7 or 8 cups in production at once, and a ratio tolerance within 1% to 3% on a near-fully digitized recipe. Fresh milk must be stored at 0 to 5 degrees Celsius and served at about 60, and the whole heat-and-serve cycle fits in those 30 seconds. That speed is the headroom for peak hours, and it is why the shop can run 24 hours with negligible marginal cost for late-night demand.

The real-world noise is where the design earns its keep. Cold cups sweat, so the gripper deals with wet surfaces. Locker occupancy changes every minute, and one order can contain multiple cups. The production line does a weigh check on every drink and automatically remakes a failed one. The operator's framing is blunt: a demo asks whether the system can do the task, a store asks whether it can keep doing the task all day.

Unitree's week was flashier and structurally identical. UnifoLM-X2-1.0 is a world-action model that predicts, plans, and executes actions in real time. The demo is an autonomous G1 boxing bout. The punch library is the same two moves from last year, a hook and a leg sweep. What changed is who decides. Last year an operator triggered moves from a pre-trained action library. Now the robot decides when to throw and where to step, and after a heavy hit it sways, recovers its center, and keeps following the opponent.

The interesting detail is where the decisions run. The demo's running logs show a server receiving observations and replanning actions, then sending them back to the robot. Unitree's setup resembles phone navigation: the phone knows where it is, the server computes the route, the phone executes step by step. The semantic layer lives off-board.

Figure is spending real money on the same seam. Helix 02 splits the brain into a top layer that understands the environment and judges the task, a middle layer that turns judgment into whole-body actions, and a bottom layer for balance, contact, and coordination. Training used more than 1,000 hours of human motion data and over 200,000 parallel simulated environments. Its Index program pays ordinary people for first-person cooking and cleaning videos, and has collected over 16 million clips at a cost of 15 million dollars, about a dollar per video. In the Helix 02 launch demo, a robot ran a full kitchen chain of 61 actions over about 4 minutes with no human takeover or reset. Earlier, a Figure 02 at the BMW plant accumulated more than 1,250 hours and moved over 90,000 parts.

Key numbers.

• 202 cups served in the first hour at Seven Fresh Coffee, a Guinness record.
• 30 seconds average per drink, with 7 to 8 cups in production at once.
• 42 ticks median completion for the full LLM+UCB+DQN stack, 25% to 39% faster than the alternatives.
• 15 seconds to 3 seconds on the living-room voice loop after the prompt dropped from 30k to 5k tokens, about a tenth of the cost.
• 1,250 hours and 90,000 parts moved by a Figure 02 at the BMW plant.

Common Pitfalls

1. Running the language model at control frequency. An LLM replanning every tick stalls the loop and fights the local learner. The navigation paper keeps LLM inference at round speed and gated by a bandit. The living-room setup does the same with a broker. The fast controller owns the ticks.

2. Training visuomotor policies in visually sterile scenes. The ACT policy grabs the similar-looking distractor the moment you deploy in a cluttered space, and it can handle picking while failing placement, or the reverse. Distractor sensitivity is stage-specific, so fix it with phase-dependent supervision, not just more demonstrations.

3. Putting a full agent token on untrusted hardware. The Reachy daemon exposes 100 unauthenticated endpoints. Put the gateway token on the robot and the robot becomes your calendar and your shell. A broker with a scoped token and an intent allowlist contains the blast radius in one file.

4. Optimizing the model when the prompt is the problem. The living-room latency fix was not a model upgrade. Dropping the context from 30k to 5k tokens cut latency by about 4x and cost by about 10x. Measure prompt size before you buy a bigger GPU.

5. Validating on demos instead of sustained operation. A pick-and-place that works once in the lab does not survive a 24-hour shift. The coffee shop's weigh check and auto-remake are the product. If your evaluation never runs for hours under real noise, you are testing a demo, not a system.

One Thing to Remember

Every system that worked this week, in a lab, a living room, a coffee shop, or a boxing ring, drew a hard line between the semantic layer and the reactive layer, then defended that line with a mechanism. ACT defends it with phase-dependent attention. The navigation stack defends it with schema bounds and a bandit. The broker defends it with an intent allowlist. Figure defends it with explicit model tiers. The systems that feel brittle are the ones where the seam is blurry, where the language model can touch the motors directly and target selection is left to chance.

The Bottom Line

1. If you are training or fine-tuning visuomotor policies for real environments, treat visual target selection as its own failure mode. The ACT results show the skill surviving while selection fails, and the recovery comes from stage-specific interventions, distractor augmentation, phase-dependent attention regularization, and appearance-based prompting, not from more trajectory data.