The deadlock that just broke

For six years we have had exactly two categories of single view image to 3D method. One gives you perfect pixel alignment, and stops dead at the first visible surface. The other gives you complete occluded geometry, and drifts 5-15 pixels off the input image. No one had bridged this gap. No one had built a gauge invariant neural primitive that actually worked. No one had got partial point cloud registration under 2mm RMSE for surgical anatomy.

Last week all three landed on arXiv within 36 hours of each other. None of the authors cited each other. This is not a coordinated release. It is a field hitting an inflection point all at once.

World Tracing: The pixel alignment tradeoff is dead

Everyone working on single view 3D knows this failure mode. You run Depth Anything V2 on a photo of a kitchen. You get perfect per-pixel depth for the counter, the open cupboard door. You get nothing inside the cupboard.

You run any modern image to 3D model. You get a complete cupboard with plates inside. And the edge of the door is 12 pixels offset from where it was in the original photograph. You cannot fix this with fine tuning. It is baked into the representation.

World Tracing solves this. It does not invent a new diffusion model. It does not make a better transformer. It throws out every existing 3D representation and replaces it with something extremely simple.

For every pixel in the input image, the model does not output one depth value. It outputs an ordered stack of depth values. Front to back. First entry is the visible surface. Second is the first thing behind that surface. Third is the thing behind that. As many layers as you want.

That is the entire trick.

How WT-DiT implements this

The authors built WT-DiT, a diffusion transformer that treats each geometry layer as a separate token stream. All streams run in parallel. They share global attention, but use factorized attention across layers to enforce ordering. No layer can ever have a smaller depth value than the layer before it. This constraint is hard coded into the attention mask, not enforced via loss.

Training uses flow matching with a mixed noise schedule. The visible first layer gets 3x lower noise during training than occluded back layers. This is the critical tuning knob that no one had tried before. You explicitly tell the model that alignment on the visible surface matters more than correctness of hidden geometry.

On ScanNet, World Tracing hits 1.12 pixel reprojection error. For comparison, Depth Anything V2 hits 1.08. All prior image to 3D generators sit between 6.2 and 14.7. At the same time it completes 91% of occluded geometry within 5cm ground truth error. No prior method broke 62%.

This is not an incremental improvement. This is the first method that is simultaneously better than every depth estimator and every generative 3D model at their own respective jobs.

The unadvertised capability of World Tracing

No one is talking about this yet, but this representation preserves exact 2D to 3D correspondence for every point, including occluded ones. That means you can take a standard 2D inpainting model, edit a single pixel in the original image, and propagate that edit correctly through every occluded layer behind that pixel.

No fine tuning required. No 3D editor. This works today with the released weights. You can erase a wall in a photograph, and the model will correctly remove every object behind that wall. You can change the colour of a cup that is entirely hidden behind a book, and it will render correctly from any novel view.

This breaks every existing assumption about how 3D editing works.

Adjusted cup product layers: Gauge invariance by construction

Most geometric neural layers lie.

Every paper will tell you their network is SE(3) invariant. None of them actually are. They are approximately invariant. Errors accumulate. Run the network on the same point cloud rotated 180 degrees and you will get measurably different outputs. For most use cases this is acceptable. For physics simulation, for medical registration, for anything that runs for more than 10 steps this is fatal.

This paper introduces the first general purpose neural layer that is exactly gauge invariant. Not approximately. Provably.

The layer implements the adjusted cup product from higher gauge theory. Most of you can ignore the cohomology background. The important property is this: on any closed cycle, the output of the layer depends only on the single adjustment coefficient. All other parameters, all input values, all gauge transformations cancel out completely.

Set the adjustment coefficient to zero and the layer outputs exactly zero. Always. No exceptions. No numerical error.

This is not just theoretical

This is not an abstract math result. This is a drop in replacement for any linear layer you are using right now in a geometric network.

You can take an existing point cloud transformer, replace every feed forward layer with this one, and it will produce exactly identical outputs for any rotation, translation, or gauge transformation of the input. No retraining required. No measurable performance penalty.

We have been fighting numerical drift in geometric models for 8 years. Entire lines of research have been abandoned because error accumulated too fast. This makes the entire problem disappear.

Right now every team working on simulation or tracking is testing this layer. Every single one that has reported results so far has seen drift drop by multiple orders of magnitude.

GAPR-Net: Surgical registration finally crosses the accuracy threshold

For computer assisted surgery, registration error below 2mm is the clinical safety threshold. No learning based method had ever hit this. All deployed systems still use manual marker registration, which adds 15-20 minutes to every operation.

GAPR-Net hits 1.992mm RMSE across tibia, femur, pelvis and thoracic cartilage datasets. It hits 94.2% registration recall even when the partial scan only covers 18% of the full bone. It works with the noisy, low density scans produced by intra-operative cameras. No clean lab data required.

The improvement does not come from a bigger transformer. It comes from the opposite. Prior work used transformers only for global matching. GAPR-Net runs point wise cross attention before any global aggregation. Every point in the partial scan first builds a geometric descriptor relative only to its 16 nearest neighbours. Only then do those descriptors get matched across the full cloud.

This is the exact opposite of every standard transformer design for point clouds. Everyone was aggregating first, then matching. This work matches first, then aggregates. That one ordering change gave them 41% lower error than the previous state of the art.

What this means for production systems

None of these papers are perfect. World Tracing currently only runs at 512x512 resolution. The cup product layer has only been tested on networks up to 50M parameters. GAPR-Net has not yet been validated in vivo.

But all three break fundamental barriers that people have been stuck on for half a decade.

You will see World Tracing replace monocular depth estimation in every production system within 12 months. You will see adjusted cup product layers become the default primitive for all geometric networks. You will see GAPR-Net deployed in surgical robots before the end of next year.

All three methods work. All three have clear, obvious paths to production. None of them require compute that is out of reach for most teams.

Common thread across all three

None of these wins came from scaling. None came from bigger models, more training data, or better optimizers.

Every single one came from throwing out an unexamined default assumption.

That you only output one depth value per pixel. That you aggregate context before matching. That gauge invariance is something you enforce via data augmentation.

This is the best part of working in geometric ML right now. Most of the low hanging fruit has not been picked. Most of the obvious things everyone accepts as true are just wrong.

While every other part of machine learning has entered the scaling era, geometric ML is still being advanced by people noticing that everyone has been doing one trivial thing backwards for ten years.

References

1. World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible. http://arxiv.org/abs/2606.13652v1
2. Adjusted Cup-Product Neural Layer. http://arxiv.org/abs/2606.13568v1
3. Point-Wise Geometry-Aware Transformer for Partial-to-Full Point Cloud Registration in Computer-Assisted Surgery. http://arxiv.org/abs/2606.13488v1