Stop reading about RL that only wins Atari games or beats human go players. This month three papers landed that solve actual problems you have right now if you run ML in production. None of them are flashy. All of them work.

Stop tuning compression hyperparameters manually

Everyone who has ever compressed a production model knows the drill. You spend three weeks running grid searches over pruning ratios, bit widths, quantization schemes. You get one good result, then the model architecture changes and you do it all again. This is not engineering. This is janitor work.

Every existing automated compression tool fails at joint pruning and quantization. They optimize one step then the other, they produce suboptimal results, they require per-model hand tuned search bounds. Nobody uses them for real production deployments.

HiReLC changes this. It is the first automated joint pruning and quantization system that reliably beats hand tuned compression on both CNNs and ViTs without per model tuning.

How HiReLC hierarchical RL works

This is not just wrapping an RL loop around compression. The authors fixed the core problem that broke all prior RL compression attempts: search space explosion.

For a standard 12 block ViT there are over 10^41 possible compression configurations. No RL agent can search that space. No heuristic will navigate it well.

Instead the authors split the problem into two completely separate agent layers. Low level agents run per network block, only making decisions for that block's kernel configuration. High level agents allocate the global compression budget across the entire network using Fisher information sensitivity scores.

No agent ever sees the full network. No agent has an action space larger than 12 possible choices. This decomposition is the trick that makes this run fast enough to use.

HiReLC benchmark results

These are not cherry picked numbers. The authors ran every test against the current standard industry baselines used by Meta, Nvidia and Google for production compression. All runs used exactly the same fine tuning budget.

Model	Baseline Top-1 Accuracy	HiReLC Compression Ratio	Accuracy Delta
ResNet50 ImageNet	78.51%	6.72x	+3.83%
ViT-B/16 ImageNet	81.79%	6.14x	-0.55%
EfficientNetV2	83.12%	5.99x	-2.17%
ConvNeXt-B	83.84%	6.31x	-5.62%

Yes that plus sign is correct. On ResNet50 HiReLC produced a 6.7x smaller model that is more accurate than the uncompressed base model. This is not a trick. Good compression removes noise. Most hand tuned compression is bad.

The active learning surrogate trick

The other dirty secret of all prior RL compression work: every policy step required a full fine tuning run. That meant 1000+ GPU hours per model. Nobody will ever run that for production.

HiReLC uses a tiny 2 layer MLP surrogate trained on observed fine tuning results. This surrogate is only used for reward shaping during search. It never replaces actual final evaluation.

This one change cut total runtime from 1200 GPU hours to 72 GPU hours for ResNet50. That is the difference between a paper trick and something you can run on your cluster this week.

Power constrained control is not a reward hack

Next paper. Everyone building mobile robots or autonomous vehicles has done this. You write a reward function for task performance, then you add -0.01 * power at the end. Then you spend two weeks tuning that 0.01 constant. If you make it too low the robot wastes power. If you make it too high the robot stops moving entirely. And when you change vehicle, battery or environment you start over.

This is a terrible way to engineer systems. You do not want a tradeoff. You want to say "use at most 12 watts" and have the controller do the best possible job it can under that hard limit.

Explicit constraints beat weighted rewards

The authors did exactly that. They reformulated underwater vehicle control as a constrained MDP, then ran standard PPO with a Lagrangian dual variable updated online. No magic. No new RL algorithm. Just correctly formulating the problem.

Across 12 test runs, the constrained policy hit the exact requested power budget every single time. It never missed by more than 1.2%. There was zero manual tuning across vehicles or tasks. You set the number. It works.

This is not a robotics specific result. This applies to every control problem where you have a hard resource limit. Power, latency, memory, bandwidth. Stop tuning reward weights. Add a constraint.

Stop approximating value functions

Third paper. This is the quietest one and the most important one long term.

For 30 years every RL and dynamic programming approach has followed the same playbook: approximate the value function, then extract the policy from the value function. Everyone accepts that this is how you do it.

This paper shows this is backwards. Optimal policies are almost always piecewise constant. They have very simple boundaries. Value functions are always smooth and high dimensional. You are approximating the hard thing to get the easy thing.

Policy tessellations

The optimal policy for almost every structured MDP divides the state space into a small number of regions. Inside each region you always take the same action. All errors happen right on the boundary between regions.

Instead of fitting a value function over the entire state space, you just fit the boundaries. That is policy tessellation.

On standard inventory control and queue admission benchmarks this approach achieved 1/7th the policy error of PPO, 12x faster evaluation, and zero training instability. It did this with 1% of the parameters.

Nobody is talking about this result. It breaks half the implicit assumptions of modern RL.

What this means for you

None of these papers introduce a fancy new foundation model. None of them have 100 author lists. None of them got press releases.

They are exactly the kind of work that actually moves the field forward. Each one took an existing, widely accepted bad practice, showed exactly why it was broken, and provided a working replacement.

You can implement HiReLC next month for your production models. You can switch to constrained PPO for your robot controller this week. You can stop fitting value functions for your operational decision systems right now.

Closing observation

Most RL research for the last seven years has been optimized for impressive demo videos. This batch is different. All three papers are written for engineers. All three include exact implementation instructions. All three include failure modes. All three publish every single baseline number.

This is what good applied ML looks like. It does not win awards. It just solves problems.