Skip to content

July 2026 Diffusion CV Roundup: The Quiet End Of Task Specific Models

#diffusion-models #computer-vision #video-generation #model-safety #3d-vision

Every single one of these nine papers executes the same trick. They take an off the shelf frozen generative diffusion model, add 0.1-1% extra parameters, and beat every existing state of the art on a classical computer vision task that nobody ever built diffusion for.

This is not incremental progress. This is the end of the 40 year era where we built separate model architectures for each vision task.

The core pattern that won this month

There is a standard recipe now. It works for every problem. Nobody published this pattern. Everyone just independently arrived at it this month.

  1. Take a frozen base diffusion transformer. Do not edit the base weights.
  2. Add a tiny adapter, head or routing layer. Never more than 2% of the base parameter count.
  3. Reframe the original discriminative task as a generation problem.
  4. Fine tune only the adapter. Synthetic training data works fine. Real data is optional.
  5. Run a trivial classical post processor on the generated output.
  6. Publish new state of the art.

This pattern beat pose tracking, video extrapolation, face restoration, concept unlearning, safety defense, image editing, outpainting and 3D captioning all in the same week.

Pose tracking is now video translation

ProxyPose is the most important paper of this batch. 6-DoF pose tracking was the last unbroken stronghold of classical computer vision. For twenty years everyone built custom correspondence matchers, depth heads, keypoint detectors and rigid transform optimizers. All of them failed on textureless, transparent, reflective or deformable surfaces.

ProxyPose does none of that. Given monocular video and one single marked pixel on the first frame, it asks a fine tuned video diffusion model: draw a coloured cube moving exactly like the surface under that pixel.

Because the cube geometry is known, recovering full 6-DoF pose reduces to running OpenCV PnP on the output. That is the entire pipeline.

It beats every existing pose tracker on every public benchmark. It works on glass, polished metal, skin and cloth. The authors never showed it a single real world video during fine tuning. All adapter training was done on 100k synthetic Blender animations.

Long video finally works without full fine tuning

Identity drift killed every long video system before this. After 4 seconds characters turn into other people. Objects vanish. Lighting resets. Everyone assumed this required full model fine tuning and enormous context windows.

PACR-Video solves this with prompt routing. The base video transformer remains completely frozen. The system maintains a recursive bank of compact embeddings for every entity, location, action and style observed in prior shots. For each new frame it routes only the relevant embeddings through adapter gates attached to the DiT attention layers.

It maintains consistent identity across 12+ shots. No context window extension. No full fine tuning.

MethodVBench Long ScoreIdentity ConsistencyHuman Preference Win RateTrainable Parameters
Base Wan 2.261.20.4212%0
Full Model Fine Tune74.10.6847%5B
Memory Augmented Baseline70.80.6131%120M
PACR-Video80.30.8779%18M

5B parameter video diffusion runs on your phone

Everyone accepted that mobile diffusion meant small compromised models. MobileWan proves this assumption was wrong.

Starting from the full server grade Wan2.2 5B video transformer, the authors did three changes:

  1. Reformulated block attention to run as a recurrent process with constant memory usage
  2. Pruned 62% of attention heads that contributed zero output variance
  3. Distilled sampling steps down to 12 with negligible quality loss

The resulting model runs unmodified on commercial Android devices. It generates 5 second 480x832 video with 20 second end to end latency. It achieves a VBench score of 83.79, within 2 points of the original server model running on an A100.

Six months ago the best mobile video model scored 67.

Concept unlearning stopped breaking models

All prior concept unlearning methods operated like sledgehammers. You could erase the target concept, but you would also break general model quality, introduce artifacts, and erase visually similar benign concepts. Production teams never used them.

TILDE reframes unlearning as a distributional alignment problem. It never edits base weights. It learns a tiny score correction that tilts the output distribution just enough to suppress the unwanted concept, while preserving the relative probability of all benign outputs.

On standard benchmarks TILDE achieves 98% target forgetting rate, while retaining 99.7% of the base model FID on benign prompts. Prior best was 92% forgetting and 81% retention. This is the first unlearning method that can actually be deployed.

Safety defenses moved to inference time

Visual synonym jailbreaks broke every existing safety filter last quarter. Adversaries found prompts that looked completely benign, but would reliably generate prohibited content through implicit visual associations learned during pre-training. Prompt filters could not detect them.

AEGIS does not check prompts. It watches generation. The authors found there are exactly 17 sparse attention heads in Stable Diffusion 1.4 that activate whenever any unsafe visual concept is being constructed, regardless of the input prompt.

At inference time AEGIS applies a tiny repulsion signal only to those heads. No fine tuning. No prompt filtering. It drops attack success rate from 92% to 3%, and has no measurable effect on benign output quality. It transfers unchanged to SD 2.1 and FLUX.1 after re-identifying the equivalent critical heads.

Editing works at native camera resolution

All prior diffusion editors broke above 1024px resolution. Editing an 8K photo meant downscaling, editing, upscaling, and getting a blurry mess.

UltraDiffEdit fixes this. It runs denoising across overlapping patches at three different scales, merges latents before decoding, and enforces consistency across patch boundaries. No fine tuning. No custom models. It runs unmodified Stable Diffusion XL and edits native 8K images on a single RTX 3090.

Boundary artifacts are gone. Fine detail is preserved in unedited regions. This is the first editing system that works correctly at the native resolution of modern phone cameras.

3D captioning switched to voxel diffusion

PVCap delivered 11.41% and 13.99% CIDEr improvements over the prior state of the art on ScanRefer and Nr3D 3D captioning benchmarks.

They did not invent a new multimodal transformer. They did not invent a new alignment mechanism. They took an existing voxel diffusion backbone, attached a standard LLM caption head, and augmented training data by shuffling object positions inside scenes. That was enough to beat every purpose built 3D vision language model.

The unspoken tradeoff

None of these papers discuss the downside of this pattern.

Every single system described here inherits every failure mode of the base diffusion model. If the base model has a bias, your pose tracker will have that exact same bias. If the base model hallucinates, your face restorer will hallucinate facial features. If the base model fails on dark skin, every task built on top of it will fail on dark skin.

We are moving from a world where 100 different tasks had 100 different independent failure modes, to a world where every single computer vision system fails in exactly the same way.

This is an enormous operational win. It is also an enormous single point of failure.

What comes next

Nobody is building task specific models anymore. If you start a new computer vision project today and you are not building on top of a frozen general diffusion backbone, you are already behind.

The next fight is not about accuracy. It is about auditability. Nobody knows why the video diffusion model can track pose. Nobody can explain which pre-training data gave it that capability. It just can.

We did not solve computer vision this month. We outsourced it.