Nine RL papers dropped on arXiv in a 48 hour window last week. None had flashy demo reels. None got tweeted by tech influencers. None claimed AGI. Every single one contains working, reproducible methods that solve real problems ML engineers are actually fighting with right now.

This is not a roundup. This is a breakdown of what actually changed.

Nobody won the dexterous manipulation argument. Everyone lost.

For ten years the field split into warring camps. One side said only end-to-end RL would ever solve dexterous hands. The other said RL was useless and only precise motion planning worked. Both sides spent tens of millions of dollars running experiments.

REGRIND beat all of them. The pipeline is almost insultingly simple. Record one single human demonstrating the task. Retarget the hand and object keypoints to the robot kinematics. Train a tiny residual RL policy in simulation only to track those keypoints. Deploy zero shot to hardware.

That is it. No reward engineering. No domain randomization. No million step training runs. It works on scissors. It works on screwdrivers. It works across two completely different robot hand designs. It hit 79% success rate on hardware where the previous state of the art was 31%.

Progress often looks like humiliation. Everyone was trying to build a system that could invent manipulation. It turned out we just needed one that could copy a human badly.

The offline to online RL problem everyone ignored

Everyone pretends RL deployment works like this: train a policy offline on logged data. Deploy it. Done.

That never happens. What actually happens: you train 7 candidate policies offline. They all look good on offline metrics. You get a budget of 1200 real world rollouts to validate and improve before full deployment.

What do you do with those rollouts? Do you spend them evaluating which policy is actually good? Do you spend them fine tuning? No one had ever formalized this tradeoff before this paper. No one had even acknowledged this was a problem.

Active Offline-to-Online RL solves it. The method allocates rollouts between evaluation and fine tuning using upper confidence bounds on linear performance forecasts. It beats every existing baseline by 18-32% across every tested environment. This is not an incremental improvement. This fixes the single largest failure mode for deploying RL in production.

SKooP: finally a physics prior that works on real robots

Everyone says you should inject physics priors into RL. Everyone tests those priors on CartPole.

SKooP is the first one that works on actual high dimensional robots. It learns a Koopman linear approximation of the system dynamics alongside the policy. Critically: only the critic sees these predictions. The actor never does. That is the entire trick. No one noticed this arrangement before.

It reduces required environment steps by 62% for quadruped locomotion. It improves final achieved reward by 41%. Policies transfer zero shot between three different simulation engines with no retraining.

Distributional RL is lying to you

This is the most important paper published this month. It will be ignored.

For five years we have been deploying distributional RL agents with the explicit claim that they output reliable risk estimates. We use them for safety monitoring. We use them for risk sensitive control. We build entire regulatory arguments around their output.

This paper ran a proper statistical audit of QR-DQN, C51 and IQN. 40-95% of all claimed risk tradeoffs are statistically false at 95% confidence. The distribution heads are not measuring environment stochasticity. They are measuring training seed noise. Acting on CVaR outputs at high stakes states is worse than random chance.

The authors ran full positive controls. Their audit correctly detects real risk differences 96% of the time. When they ran it against every standard agent it found nothing. Nothing at all.

If you are running distributional RL for safety critical control right now, stop. Go back to vanilla DQN. It is not less safe. It is more safe, because you will not be making decisions based on imaginary numbers.

CycleGRPO: LLM grounding does not need labels

You do not need human labels to train an MLLM to ground image regions. You do not need captions. You do not need annotators.

You just do this:

1. Show the model a bounding box. Ask it to describe what is inside.
2. Take that description. Ask the model to draw the bounding box for it.
3. If it draws the same box, the description was good. That is the reward.

That is CycleGRPO. That is the entire thing. It beats every supervised baseline on every grounding benchmark. It improves region captioning, VQA and referring segmentation all at the same time. No human input required.

This is the first RL alignment method that actually scales. No one has realized this yet.

SCOPE-RL fixes the biggest flaw in GRPO

GRPO is the best policy optimizer we have for LLMs. It has one fatal flaw. It only rewards the final answer. It will happily generate 700 tokens of garbage reasoning that accidentally arrives at the correct number. It will prefer that garbage over a correct 80 step proof every single time.

SCOPE-RL adds two tiny reward terms. One gives partial credit for progress towards the answer before success. One penalizes redundant tokens after success. That is it.

It delivers an 11.2 point accuracy gain on math benchmarks. It reduces reasoning tokens used by 27.1%. Everyone spent six months arguing about policy gradient math. The problem was just the reward signal all along.

RL as a post processing layer

Look at the BCI decoding paper. Look at the EEG seizure detection paper. Look at REGRIND. Look at SKooP.

Nobody is training end to end RL any more. The winning pattern everywhere now is:

1. Train a boring, stable supervised baseline first.
2. Run a tiny RL agent only on the residual error of that baseline.
3. Get 30-40% improvement. No extra data. No changes to the base model.

This pattern works for neural decoding. It works for robot control. It works for medical signal processing. It works for LLM reasoning. It is showing up in every single new paper. Nobody has written a blog post about it.

Benchmark performance summary

Method	Domain	Relative improvement over baseline	Compute required	Zero shot transfer
REGRIND	Dexterous manipulation	+155% task success	12 GPU hours	Yes
Active O2O	Offline RL deployment	+27% average reward	0.1 GPU hours	N/A
SKooP	Legged locomotion	-62% sample requirement	8 GPU hours	Yes
CycleGRPO	MLLM grounding	+14.7 mAP	32 GPU hours	Yes
SCOPE-RL	LLM reasoning	+11.2 pp accuracy	16 GPU hours	Yes
Residual RL BCI	Neural decoding	+41% correlation	2 GPU hours	Yes
Transformer NAS	Architecture search	-92% search cost	RTX 3060 / 3 hours	Yes

The architecture pattern that won this month

All of these papers converged on exactly the same pipeline, completely independently.

This is the new standard RL pipeline. Everything you learned from 2020-2025 tutorials is obsolete.

What no one is saying

All of these results are embarrassing for the entire field. Every single advance came from removing complexity, not adding it.

End to end training lost. Residual correction won. Dense reward engineering lost. Sparse verifiable anchors won. Big models lost. Good priors won. Pure simulation lost. One real demonstration won.

This is what real progress looks like. No press releases. No demo reels. Just quiet papers that say "we tried the obvious thing everyone laughed at three years ago, and it works much better".

If you are building RL systems right now, stop reading the 2023 best practice guides. All of the rules changed last week.