Skip to content

What Hugging Face shipped this month: fine-tuning speedups, ASR benchmarks, release CI and web ML cache

#huggingface #transformers #fine-tuning #asr #webml #devops

This is not a roundup of press releases. All four posts dropped by Hugging Face engineering in the last three weeks contain usable, immediately applicable changes for anyone building, training or deploying ML. None require paid APIs. None are vaporware.

We will go through each one, pull out the numbers that matter, the unstated tradeoffs, and the parts you can copy into your own stack today.

3.7x MoE fine-tuning with one import line

This is the biggest practical speedup for open source fine-tuning released this quarter.

NVIDIA and Hugging Face shipped NeMo AutoModel, a drop-in replacement for AutoModelForCausalLM that requires exactly one changed import line. No other code changes. No config rewrites.

ModelTransformers v5 TPS/GPUNeMo AutoModel TPS/GPUSpeedupMemory reduction
Qwen3-30B-A3B3075113403.69x-29%
Nemotron 3 Nano 30B A3B4583154213.36x-32%

That is not a microbenchmark. Those are full fine-tune numbers on 8x H100, running Adam, activation checkpointing, sequence length 4096.

Most importantly: checkpoints saved from NeMo AutoModel are standard Hugging Face safetensors. They load unmodified in vLLM, SGLang, Transformers v4/v5 and every other inference runtime. There is no lock in.

Where the speedup actually comes from

Everyone will quote the 3.7x number. Almost no one will explain how it works.

There are three independent improvements stacked on top of each other:

  1. Proper orthogonal Expert Parallelism. Unlike the v5 implementation which carves EP out of the data parallel budget, NeMo AutoModel runs full EP=8 and full DP=8 at the same time on 8 GPUs. Every GPU holds only 1/8 of the expert weights. For the 30B MoE models this cuts per-GPU expert memory footprint from 55 GiB down to 6.8 GiB.
  2. DeepEP fused dispatch. This is the part no one else has shipped yet. Instead of running separate AllGather and ReduceScatter collectives around expert computation, DeepEP fuses the entire token routing and dispatch path into a single GPU kernel, overlapping communication 100% with compute. This alone cut iteration time by 47% on DeepSeek V3 671B.
  3. TransformerEngine kernels across every layer. Not just attention. RMSNorm, linear layers, cross entropy loss. All fused, all hand tuned for H100.

This is not magic. This is just good engineering that no one had bothered to wrap behind the standard from_pretrained() API until now.

The FFASR leaderboard fixes the biggest lie in ASR benchmarks

Every ASR model you see advertised has a LibriSpeech WER number. None of those numbers mean anything once you put the microphone more than 50cm away from the speaker.

Hugging Face and Treble launched the Far Field ASR Leaderboard this month, and the first results are brutal. Across every submitted model, far field WER at 10dB SNR is 3.2x to 7.1x higher than clean near field WER on exactly the same speech content.

No one was measuring this before. Everyone pretended it didn't exist.

The benchmark runs across 14 realistically simulated rooms, three SNR tiers, includes reverberation, background HVAC and transient noise. It also reports RTFx on a standard L4 GPU, so you can actually plot the accuracy / latency tradeoff that matters for deployment.

This is the benchmark that will actually change how ASR models are built. If you are shipping any voice interface, stop looking at LibriSpeech scores. Start looking here.

Shipping huggingface_hub every week with AI and guardrails

This is the most copy-pasteable post Hugging Face has ever published.

For years the huggingface_hub client released every 4-6 weeks. It now releases every single week. The entire pipeline runs on GitHub Actions, uses only open tools and open weight models, and has exactly two manual human steps.

This is not "let AI do releases". This is the correct pattern for using generative AI in production infrastructure.

The critical insight that everyone misses: you do not trust the model. You wrap it inside deterministic guardrails.

Before the model runs, a script extracts every PR number from the commit range. That is the source of truth. After the model writes the notes, the same script extracts every PR reference from the generated markdown. If anything is missing or added, it hands exactly that list back to the model and tells it to fix it. It repeats this loop until the output exactly matches the manifest.

This pattern works for almost everything. You can copy this tomorrow for your own library releases.

The release pipeline stack

Every part is open. No closed APIs. No vendor lock in.

ComponentChoice
OrchestrationGitHub Actions
Agent runtimeOpenCode
ModelGLM-5.2 open weights
InferenceHF Inference Providers
PublishingPyPI Trusted Publishing OIDC

Total cost per release: ~$0.25.

The human work went from 4 hours writing release notes to 15 minutes editing a draft. The release cadence went from 6 weeks to 7 days.

This is what good AI adoption looks like. No fanfare. No revolution. Just taking the boring repetitive work that no one likes doing, having a model draft it, and having code verify that it did not make anything up.

Cross origin storage fixes web ML's biggest waste

Right now if you visit three different websites that all run Transformers.js with Whisper tiny, your browser will download the exact same 177MB model file three separate times. It will store three separate copies on disk.

This is not a bug. This is intentional browser cache partitioning, put in place to stop timing side channel attacks.

The proposed Cross Origin Storage API fixes this without breaking security. Instead of keying cache entries by URL and origin, it keys them by cryptographic hash. If two sites load byte identical files, they share the same cache entry.

Transformers.js already has experimental support for this behind a single flag:

javascript
import { env } from '@huggingface/transformers'
env.experimental_useCrossOriginStorage = true

When enabled, the first site that loads Whisper tiny downloads it once. Every other site on the entire internet gets it from local cache instantly.

The unstated tradeoffs

None of these are perfect. All have tradeoffs that the original posts mention only in passing:

  1. NeMo AutoModel balanced routing benchmarks measure ideal steady state performance. Real world training runs will see ~10-15% lower throughput until the load balancing loss converges.
  2. FFASR uses simulated acoustics. The sim-to-real gap is around 8% right now, much better than any previous simulation but still not identical to physical recordings.
  3. The release pipeline works great for squash merged PRs. It will break if you use merge commits.
  4. Cross Origin Storage will not return cache hits for files that have only been seen by one or two sites. This is an intentional privacy mitigation. Popular models will work perfectly. Obscure custom models will get no benefit.

What you should do next

This week:

  1. Swap your AutoModelForCausalLM import for NeMoAutoModelForCausalLM on your next MoE fine-tune run. You will not go back.
  2. Submit your ASR model to FFASR. Find out how bad it actually is in real rooms.
  3. Fork the huggingface_hub release workflow. Stop wasting half a day writing changelogs.
  4. Enable the Cross Origin Storage flag in your Transformers.js app. Your users will thank you.

None of this requires waiting. All of this works today.

All four posts landed within 18 days of each other. None got the attention they deserved. Most people will only see the headline numbers. This is the actual engineering underneath.