Skip to content

This Month In Applied ML: No One Is Scaling Parameters Anymore

#applied-machine-learning #research-roundup #domain-ml #arxiv-roundup

This is not the ML research you remember from 2023. There is not a single 100B+ parameter model in this entire batch. No one claims AGI. No one benchmarks against MMLU.

Almost every paper here solves a problem that someone, somewhere, has actually been paid to fix. This is good. This is what the field looks like when it grows up.

We are going to walk through 27 new papers from the third week of July 2026, group them by what they actually do, call out the good ideas, the bad ones, and the things you should steal for your production systems tomorrow.

The most important paper no one will talk about

Staypoint detection is the single most widely deployed ML primitive on the planet. It is the thing that turns raw GPS pings into "you were at the coffee shop for 12 minutes". Every location app, every delivery fleet, every urban analytics system runs this algorithm.

Until this week, there had never been a single systematic benchmark. Every production system used one of 8 unvetted heuristics, everyone knew they broke with noisy phone GPS, and no one had ever bothered to measure how bad it was.

The authors built 16 simulated datasets with realistic noise levels, ran every published algorithm, and found that every single state of the art method falls apart completely at noise levels that every modern phone produces. Their proposed unsupervised method cuts error by 62% under real world conditions.

No flash. No foundation models. Just fixing a 15 year old broken primitive that billions of people use every day. This is the best kind of ML research.

Video reasoning stopped trying to look at every frame

OmniReasoner is the correct architecture for long video. Full stop.

Everyone has been doing this wrong for three years. For long audio/video, you do not run the model at 30fps. You do not downsample uniformly. You do not slide windows.

You build a cheap low fidelity preview of the entire stream first. Then you let the model decide where it needs to zoom in.

That is exactly what OmniReasoner does. It learns to call a zoom tool. It only pays the high fidelity cost for the 2% of the stream that actually matters for the answer. On standard benchmarks this improves accuracy and cuts compute cost by 78%.

This is not a trick. This is how humans watch video. This is how every production system will handle long media 12 months from now.

ML is eating physical engineering, properly this time

Three papers this month do what everyone said ML would do for engineering five years ago, and actually deliver. None of these are black box models. All of them respect existing physics. None of them try to replace first principles models. They only learn the deviation from the known good baseline.

This is the correct pattern for scientific ML. You do not throw away 100 years of thermodynamics. You teach the network to only predict the delta that the equations get wrong.

PaperProblemImprovement over baselineSpeedup vs high fidelity model
TAIR ThermodynamicsSupercritical combustion property calculation7.5x lower RMSE on compressibilityFixed cost inference
RL Reactor ClusteringGas turbine lean blowout prediction19% lower prediction error112x
ATLAS Amorphous SamplerGlassy material energy landscape sampling0.2% free energy error512x

Everyone finally figured out that benchmark splits are lying to you

Two separate papers this month call out the exact same failure mode that makes almost all published ML results useless.

For financial fraud detection, everyone was using random train/test splits. That means the same company appears in both train and test. When you split by company instead, every published state of the art model's AUC drops from 0.92 to 0.61.

The authors did exactly one simple thing: they split the dataset such that no company appears in both train and test. That is how this task works in the real world. That is how you should split every single dataset you ever work with.

This is not a new insight. It is just that no one had the nerve to publish that everyone else's results are fake.

Cross lingual factual bias is worse than you thought

LLMs do not have one set of facts. They have different facts for every language.

Ask the exact same factual question in English then in Bulgarian. You will get different answers 41% of the time. Not translation differences. Different factual claims.

This paper tested four different fixes. The best one was not activation engineering. It was not fine tuning. It was just adding one sentence to the prompt: "Answer this question as you would answer it for someone living in Bulgaria".

That one line fixed 72% of the inconsistency. All the fancy invasive methods performed worse. This is a running theme this month. Simple almost always beats clever.

Theory of mind benchmarks are finally not stupid

MeetingToM is the first good multimodal social reasoning benchmark.

It does not ask "what emotion is this person showing". It asks: "Three people agreed in the meeting. Which one is lying and will vote against this tomorrow?".

Every tested MLLM scored at chance on this task. GPT-4o got 52% accuracy. Humans get 78%.

This is the actual hard limit for meeting assistants right now. All the fancy meeting summarizers you see advertised cannot tell the difference between agreement and politeness.

The end of end to end time series

MASHT is the funniest result this month.

Take your time series. Run it through MultiRocket, a 6 year old method that generates random convolutional features. Feed those features into a generic tabular foundation model. Do zero training. Do zero fine tuning.

This matches or beats every state of the art time series model published in the last three years.

All the fancy end to end transformer time series architectures. All the pretrained time series foundation models. Beaten by a random feature transform and a generic tabular model.

No one saw this coming.

The quiet revolution in offline RL

Three separate offline RL papers this month all converge on the same conclusion: uncertainty estimation is not an optional extra. It is the entire problem.

S3 for hierarchical RL. Conservative Query for preference learning. Utility Augmented Transformers for sequential decision making. All of them get their gains not from better architecture, not from more data, but from properly measuring what the model does not know, and then acting accordingly.

For ten years everyone in RL was trying to maximize expected return. Now everyone is finally trying to minimize surprise. This works much better.

Scientific code generation is terrible

SciCodePile ran 15 leading code models on actual scientific code. The best model got 12.3% Pass@1.

That is not a typo. 8 out of 9 times, the code that GPT-4o writes for computational science will not run correctly.

This is the largest unspoken gap in LLMs right now. Everyone knows they write good CRUD code. Almost no one has measured how bad they are at code that actually does math.

Robot brains are getting small

Athena-Brain-8B is exactly what everyone should be building for embodied agents.

It is an 8B parameter model. It is not smarter than frontier models. It does not try to be. It just does not ramble. It outputs exactly what the robot needs, no extra prose, no hedging, no apologies. It beats 70B models on zero shot embodied tasks.

No one needs a 100B parameter model on a robot. You need a model that will reliably say "move arm 12cm left" instead of writing three paragraphs about how it will help you move the arm.

What all this means

This batch of papers marks a turning point.

We have stopped trying to build one model that does everything. We have stopped competing on parameter count. We have stopped benchmarking on artificial tests that no one cares about.

Instead people are building tools that solve specific problems. They are measuring performance under real world conditions. They are finding that simple approaches usually beat complicated ones.

This is boring. This is incremental. This is how useful technology gets built.

There will be no big announcement. No keynote. No viral demo. ML will just quietly start working, one boring problem at a time.

References

All papers are linked in the article front matter. Open source code is available for every paper mentioned unless explicitly noted.