Last Wednesday two papers dropped on arXiv that will end the argument about whether you can audit what an LLM memorized.

For three years, everyone working on model governance agreed training data attribution was the only real solution for copyright claims, contamination detection, and hallucination root cause analysis. Nobody used it. It was too slow, too expensive, and every practical approximation produced results that were effectively random.

That changed this week.

The problem no one could solve

Training Data Attribution answers one very simple question: given an output from a trained model, which individual examples in the training set caused that output to exist?

The gold standard answer has always been causal intervention. Remove one document from the training set, retrain the entire model from scratch, run the same prompt, and measure the difference. Do this for every document. This works perfectly. It also costs approximately $198 per attribution query for a 7B parameter model run on standard p4d instances. For a 70B model the cost crosses $10,000 per query.

No production team will ever do this. So for half a decade everyone settled for gradient based approximations. All of these methods worked on the same principle: estimate how much changing a single training example would shift each weight in the model, then aggregate that across all weights to produce an influence score.

This approach failed. It worked fine on 100M parameter models. Above 1B parameters the signal vanished into parameter noise. The best published method before this week had 0.71 AUROC on standard attribution benchmarks. That is barely better than guessing.

What everyone got wrong until now

Every research group spent five years optimizing the wrong mapping.

Everyone assumed the chain of causality ran: training example → parameter change → model output. So all work focused on measuring the first link. Nobody stopped to verify that this link actually carries usable signal.

It does not. For any individual training example, 99.9% of the parameter delta is uncorrelated noise. The actual signal from a training document never makes it into permanent weight changes in any readable form. It lives transiently in activation space, during forward passes. That was the dead end that trapped the entire field until this month.

Both papers released this week abandon parameters entirely. That is the common insight nobody saw coming.

STRIDE: Stop looking at parameters

STRIDE starts with an admission that we will never be able to read influence out of model weights. Instead it builds a library of tiny proxy operators that simulate the effect of each training document.

The method works in three stages. First, during a single forward pass over the training set, it learns 128 dimensional steering vectors for each document. These vectors do not modify model weights. They represent the shift that would be applied to intermediate activations if that document had been the only thing the model ever saw.

Second, when given a test output, STRIDE applies each steering vector independently and measures how much it shifts the model's log probability for that exact output.

Third, it frames attribution as a sparse recovery problem. Most training documents have zero measurable effect on any given output. Using standard compressive sensing algorithms, STRIDE decomposes the observed output into the minimal linear combination of steering vectors that would produce it.

The authors do not lead with the most important number. Everyone will quote the 13x speedup over prior gradient methods. That is irrelevant. STRIDE is 1170x faster than leave-one-out retraining. You can run a full attribution query on a 7B model on a single A10G in 12 seconds for approximately $0.17.

That is the number that changes everything.

Bidirectional gradients: the other approach

The second paper takes an even more counterintuitive approach. It does not even attempt to simulate what training did. It asks the inverse question.

If I take the finished model, and nudge it very slightly to make this exact output more likely, which training examples will experience the largest change in training loss?

This is not a trick. This is a direct measurement of alignment. If a training example was responsible for the model producing this output, nudging the model towards the output will make that training example fit much better. Nudging the model away from the output will make that training example fit much worse.

The method runs one gradient ascent step and one gradient descent step on the test output. It then measures the delta in loss across the entire training set. Training examples with the largest absolute delta are the ones that contributed to the output.

This method has one enormous advantage that almost nobody has noticed yet. You do not need any training artifacts. You do not need checkpoints. You do not need optimizer state. You do not even need to have trained the model. You can run this attribution on any public LLM, including closed API models, if you can get gradient access through logit probes.

You can run this on GPT-4o outputs right now. OpenAI does not have to cooperate.

Benchmark head to head

Both teams evaluated on exactly the same standard benchmark: 1000 test outputs from a 7B Llama model trained on a known 1M document subset of The Pile, with ground truth attribution established via full leave-one-out retraining.

Prior state of the art (TracIn 2025) scored 0.71 AUROC. Bidirectional gradient method scored 0.87 AUROC. STRIDE scored 0.89 AUROC.

This is not an incremental improvement. This is crossing the threshold from useless to usable. For context, 0.9 AUROC is generally accepted as the bar for a measurement that you can rely on for operational decisions. Both methods land within rounding error of that bar.

They have different strengths. STRIDE is vastly better at attributing exact factual recall and verbatim copying. The bidirectional method is the first approach ever that can reliably attribute stylistic choices, tone, and structural patterns. It will correctly tell you that a poem was influenced by a specific poet's works in the training set, even when no literal text was copied.

The fine print nobody is talking about

Neither method is perfect. All of the hard limitations are buried in appendix B of both papers, and none of them are mentioned in the abstracts.

Neither method works reliably above 70B parameters right now. Steering vector alignment breaks down at very large model scales. The authors are confident this can be fixed with better normalization, but it is not fixed today.

Both methods systematically underattribute examples that were seen more than three times during training. Documents that appear many times in the training set spread their influence across so many activation paths that they become effectively invisible to both approaches. This is the largest remaining open problem.

STRIDE overattributes to long documents. The bidirectional method fails completely on outputs shorter than approximately 8 tokens. Neither method can reliably attribute the first token generated in a sequence.

None of these are fundamental limitations. All are engineering problems that will be fixed over the next six months.

What this actually breaks

We are no longer talking about academic research. This is usable production tooling that will restructure how we build and operate LLMs.

Copyright arguments will stop being arguments. Within 12 months every copyright claim against an LLM will include an attribution report. Courts will accept these reports as evidence. There will be no more debate about whether something was copied. You will have an ordered list of every training document that contributed more than 1% to the output.

Data contamination detection will stop being garbage. Right now all contamination checks work by searching for exact string matches. That misses 90% of actual contamination. You will now be able to test every validation example before you run a benchmark, and get an exact influence score for every training document.

You will be able to debug hallucinations. Not guess. Debug. When your model outputs a fake legal case, you will be able to go directly to the garbage training document that taught it that fact existed.

What changes for production teams this quarter

Both teams have released working reference implementations as of this morning. You can run this on your fine tunes next week.

Right now you should be building this into your model audit pipeline. If you are deploying customer facing LLMs you will be legally required to produce this kind of attribution inside 18 months in both the EU and California. You can get ahead now.

You should also stop wasting time on most other interpretability research. Almost all of the work on activation steering, concept erasure, and weight editing just became obsolete. None of those methods worked reliably because they were all trying to operate blindly. Now we have a map.

The end of black box training

For five years we have operated under an unspoken agreement. We build these models. We do not understand how they work. We pretend that this is acceptable.

That period just ended.

This is not just another interpretability trick. This is the first reliable window we have ever had into what training data actually did inside a trained model. We will look back at this week as the point where LLMs stopped being black boxes.

There will be bad consequences. There will be good consequences. None of that matters right now. What matters is that the genie is out. You can build this today.

If you are running LLMs in production, this is the most important thing that has happened all year.