REVIEW 3 cited by
MAGIC: Near-Optimal Data Attribution for Deep Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The goal of predictive data attribution is to estimate how adding or removing a given set of training datapoints will affect model predictions. In convex settings, this goal is straightforward (i.e., via the infinitesimal jackknife). In large-scale (non-convex) settings, however, existing methods are far less successful -- current methods' estimates often only weakly correlate with ground truth. In this work, we present a new data attribution method (MAGIC) that combines classical methods and recent advances in metadifferentiation to (nearly) optimally estimate the effect of adding or removing training data on model predictions.
Forward citations
Cited by 3 Pith papers
-
Small edits, large models: How Wikipedia advocacy shapes LLM values
Wikipedia edits by animal welfare advocates measurably influence LLM outputs on animal welfare topics, shown via retrieval and gradient attribution plus fine-tuning experiments.
-
LLM generation novelty through the lens of semantic similarity
An output is called novel when no semantically similar pretraining context can be retrieved; using this embedding-based measure, SmolLM models show longer-range reuse and instruction tuning increases novelty.
-
Better Training Data Attribution via Better Inverse Hessian-Vector Products
ASTRA, an EKFAC-preconditioned Neumann series iteration, computes more accurate inverse Hessian-vector products and improves training data attribution scores over EKFAC baselines.
Discussion (0). Continue with ORCID to comment.