Skip to content

June 2026 Diffusion & 3D Generative Vision Breakdown

#diffusion-transformers #3d-generation #gaussian-splatting #text-to-image #computer-vision

This is the most important week for generative vision research in 2026 so far. Every single one of these four papers invalidates common working assumptions that almost every lab has been operating on for the last two years. If you are building production generative AI systems, you can stop reading everything else right now. All of this matters.

The ImageNet evaluation scam

The core result from DiffusionBench is not incremental. It is a bomb. Pearson correlation between ImageNet class conditional ranking and text-to-image ranking is negative. Not zero. Negative. Between -0.377 and -0.580 across three independent metrics. Methods that win on ImageNet lose on text to image. This is not noise. This is active overfitting. Every single DiT improvement published in the last 18 months was optimized for a benchmark that does not correlate at all with the task everyone actually builds and ships. Nobody noticed because nobody bothered to run the actual test. Everyone agreed that training T2I was too expensive. It was not.

NanoGen framework design

The authors built NanoGen, a unified DiT training and evaluation framework. You change 12 lines of configuration and switch between ImageNet and T2I training for identical compute cost. NanoGen supports RAE, VAE, pixel space and MeanFlow diffusion. All run the same training loop. Same batch size. Same optimizer schedule. They did not cheat. All baselines match published SOTA numbers exactly when run on ImageNet. The correlation only breaks when you move to text. You should throw out every DiT paper that only reports ImageNet results starting today. The authors explicitly recommend that. DiffusionBench requires running both tasks. Any method that improves the combined score will actually transfer.

MetricPearson correlation ImageNet vs T2I
FID-0.580
CLIP Score-0.412
Human Preference-0.377

3D generation just stopped being a research toy

For the last 12 months every 3D generation demo ran on 3D Gaussians. Everyone knew they had no well defined surface. You could not put them into a game engine. You could not run physics on them. You could not export them as usable assets. They looked good on twitter and that was it. This week two separate papers fixed this.

FLAT: Triangle splatting works now

FLAT decodes triangle splats directly from video diffusion latents. One forward pass. No optimization. No per asset refinement loop. This was considered impossible 6 months ago. Triangle regression had terrible gradient flow. Everyone gave up and went back to Gaussians. They fixed it with two tiny changes. Ray centered rotation parameterization. Product window function for differentiable rendering. That is it. FLAT produces triangle soup that renders at identical speed to 3DGS, has 42% lower geometric error, and can be converted to a closed watertight mesh in 120ms at test time. You can drop the output directly into Unreal or Unity. It works. No hacks.

Representation tradeoff head to head

The authors ran the first fair comparison ever done between all modern feedforward 3D representations. All trained on exactly the same data, same model size, same compute budget. No cherry picking.

RepresentationFIDChamfer DistanceRuntime msWatertightPhysics Compatible
3D Gaussian Splatting12.10.0872.1NoNo
2D Gaussian Splatting11.70.0722.8NoNo
FLAT Triangle Splat12.30.0502.3YesYes

Nobody won on visual quality. Everyone lost on geometry except FLAT. That is the tradeoff. You give up 1.6% FID and you get an actual usable asset. For production systems this is not even a choice.

FLUX3D fixes the detail bottleneck

FLUX3D attacks the other side of the 3DGS problem. All existing methods blurred fine detail. Nobody could get fine grained texture or edges to transfer correctly from the input image. This was not a diffusion problem. This was a feature selection problem. Everyone was using CLIP or DINO features to build voxel latents. Those features are explicitly optimized to throw away appearance detail. That is their entire design goal. FLUX3D uses diffusion aligned latents instead. They also added modal aware rotary positional embeddings for cross attention between 2D image tokens and sparse 3D voxels.

Result: 28% lower LPIPS on held out test sets. It preserves scratches, logos, text on objects. All the stuff that every prior work turned into blurry mush.

IV-CoT: Chain of thought for images

Everyone has known for 18 months that text to image models cannot count. They cannot place objects correctly. They cannot bind attributes. Everyone also knew that chain of thought works for text. Nobody could make it work for images without showing intermediate sketches at inference time. IV-CoT does it entirely implicitly. They add sketch supervision only at training time. At inference time it is one single forward pass. No intermediate outputs. No extra steps. The model learns to internally generate a latent structural plan before rendering appearance. It does this entirely inside the conditioning stream. No one ever tried this exact setup before. It beats every existing model on GenEval and T2I-CompBench by 17% on structure following. It can reliably draw exactly 7 apples on a table. It can put the red one on the left. It does not mix up attributes.

What this means for production

None of this is incremental. This is not +2% on a benchmark. This is resetting baseline assumptions for the entire field. Stop evaluating DiTs on ImageNet. Stop building production systems on 3D Gaussians. Stop building text to image models without separate structural conditioning. All of these papers have working reference implementations. All will be merged into open source frameworks within 30 days. Most teams will ignore this for another 6 months. The ones that do not will ship production systems that are not even comparable.

Closing observation

There is a pattern across all four papers. Every single one of these improvements came from stopping and asking a very simple question that everyone had stopped asking. Is this benchmark actually measuring what we care about? Is this representation actually usable for anything other than demos? Why are we using features optimized for classification for generation? Why are we forcing all reasoning into one single tensor? None of them required a larger model. None required more compute. None required new hardware. They just stopped repeating what everyone else was doing. That is how progress actually happens.