Pith. sign in

REVIEW 3 major objections 5 minor 67 references

KAIROS: Scalable Model-Agnostic Data Valuation

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a closed-form MMD influence score, computable from kernel means without any model training, faithfully recovers leave-one-out data valuations within $O(1/N^2)$ error and scales to streaming updates in $O(mN)$ time.

desk verdict KAIROS is a genuinely useful closed-form MMD influence score for data valuation with solid empirical results, but the O(1/N^2) LOO-ranking guarantee is asserted rather than proved and the conditional-score sign flips between equations and algorithm, so it needs careful revision before the headline claims can be trusted. read the letter →

arxiv 2506.23799 v2 pith:7QUEXYLF submitted 2025-06-30 cs.LG

Jiongli Zhu , Parjanya Prajakta Prashant , Alex Cloninger , Babak Salimi This is my paper
classification cs.LG
keywords datavaluationmaximummeandiscrepancyinfluencefunctionleave-one-outrankingmodel-agnosticstreaminglabelnoisedetectionbackdoor
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KAIROS argues that the value of a training example can be read off from a single closed-form quantity: how much the example pulls the training distribution away from a clean reference distribution, measured by Maximum Mean Discrepancy (MMD), a kernel-based distance between distributions. The paper's central claim is that this influence score, the difference between the point's average kernel similarity to clean validation data and its average similarity to the rest of the training data, reproduces the ranking an exact leave-one-out retraining would produce, with error shrinking like $O(1/N^2)$, while requiring no model training at all. If that claim holds, data valuation stops being an expensive retraining loop and becomes a kernel-mean subtraction that works for any downstream model and can be updated as new batches arrive in $O(mN)$ time. The paper demonstrates on image and text benchmarks that this score flags noisy, mislabeled, and poisoned examples earlier than existing model-based, Shapley-based, and Wasserstein-based methods, and that online updates run up to 50 times faster than the strongest prior baseline.

What carries the argument

The load-bearing object is the finite-sample MMD influence score $\widehat{\mathrm{IF}}(x_i)=\frac{1}{n_{\mathrm{val}}}\sum_{j=1}^{n_{\mathrm{val}}}k(x_j^{\mathrm{val}},x_i)-\frac{1}{n_{\mathrm{train}}-1}\sum_{j\ne i}k(x_j^{\mathrm{train}},x_i)$, which is the closed-form directional derivative of the Maximum Mean Discrepancy under infinitesimal up-weighting of point $x_i$. It carries the argument because it converts leave-one-out valuation, normally a retraining problem, into a subtraction of two kernel means. For labels, the paper uses the conditional extension MCMD, whose influence at $(x,y)$ is the negative distance between the clean conditional mean embedding and the point's own label embedding; with categorical labels this becomes a simple residual $\|\hat{P}(y\mid x)-e_y\|_2$. The net score is the weighted sum of these two terms, and the online update maintains the kernel-mean statistics so a new batch of size $m$ costs $O(mN)$.

What would settle it

Take a fixed dataset, compute the exact leave-one-out MMD ranking and compare it with KAIROS's closed-form ranking as $N$ grows and as the kernel bandwidth is moved away from the median heuristic; if the rank overlap of the top-100 sets drops substantially or the empirical error does not shrink like $O(1/N^2)$, the central fidelity claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that the directional derivative of MMD admits a closed form: up to additive and positive multiplicative constants, the influence of a point $x$ is $\mathrm{IF}_{\mathrm{MMD}}(x;P,Q)=\mathbb{E}_{x'\sim P}[k(x',x)]-\mathbb{E}_{x'\sim Q}[k(x',x)]$, the excess of its mean kernel similarity to a clean reference distribution over its mean similarity to the training distribution. The paper shows this score matches the exact leave-one-out MMD influence and approximates leave-one-out valuation within $O(1/N^2)$ error, so rankings are faithful without retraining. The same derivative applied to the expected Maximum Conditional Mean Discrepancy yields a label-aware score that detects mislabels and backdoor triggers, and the two combine into a single net influence. KAIROS also proves the score satisfies symmetry, guaranteeing fair rankings for equally contributing points, and a density-separation property, guaranteeing a threshold that separates low- from high-quality data when the kernel is chosen appropriately.

Load-bearing premise

The load-bearing premise is that the first-order MMD influence, evaluated with a fixed Gaussian kernel whose bandwidth is chosen by the median heuristic, preserves the true leave-one-out ordering; the paper proves a density-separation guarantee for some bandwidth, not for the particular one used in the experiments.

Editorial extensions

If this is right

  • No retraining: a point's value is a closed-form kernel-mean subtraction, so valuations do not depend on any particular fitted model and cannot flip when the model changes.
  • Streaming valuation: when a batch of $m$ points arrives, all scores update in $O(mN)$ time, making web-scale and online data valuation practical.
  • Unified corruption detection: the marginal MMD term catches feature noise and covariate shift, while the conditional MCMD term catches label flips and backdoor triggers.
  • Faithful top-$k$ sets: because the score approximates exact leave-one-out rankings to $O(1/N^2)$ error, the highest- and lowest-valued points identified by KAIROS are nearly the same as those a full retraining audit would find.
  • Interpretable thresholds: the density-separation property implies a single global score threshold separates low- from high-quality data, which can be used for automated pruning and compliance auditing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the ranking guarantee holds across bandwidths, KAIROS could be used as a pre-training filter: points with strongly negative influence could be down-weighted before any model is fit, which the paper does not explicitly test.
  • The $O(mN)$ streaming update suggests a natural active-learning loop: each newly labeled point re-ranks the entire pool cheaply, so acquisition strategies could be driven by MMD influence rather than model uncertainty.
  • Because the score is defined purely by distributions and a kernel, a testable extension is to replace the median-heuristic bandwidth with a learned or cross-validated kernel; Proposition 3 only guarantees some bandwidth separates densities, so learned kernels may either improve or void the threshold interpretation.
  • The paper fixes one balancing weight $\lambda$ between feature and label influence; a task-adaptive $\lambda$ could be calibrated on a small labeled audit set to trade off covariate-shift and label-corruption sensitivity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes KAIROS, a model-agnostic data valuation method that scores each training point by its influence on the Maximum Mean Discrepancy (MMD) between the training distribution and a clean validation distribution. The influence is derived as a closed-form directional derivative, estimated in Eq. (7) by the difference between the average validation-set kernel similarity and the leave-one-out training-set kernel similarity. The scheme is extended with a conditional MMD term for label-error detection, combined in Eq. (12), and implemented with streaming updates in Algorithm 1. The authors claim an O(1/N^2) error relative to exact leave-one-out rankings, symmetry and density-separation guarantees, and demonstrate strong empirical performance on feature-noise, label-noise, and poisoning benchmarks, together with large runtime gains in the online setting.

Significance. The paper addresses a timely and practically important problem: scalable, model-agnostic data valuation. The closed-form MMD influence is elegant, is model-agnostic, admits O(mN) streaming updates, and the empirical evaluation on OpenDataVal across four datasets and several corruption types is broad. The authors provide code, report standard deviations over five seeds, and make falsifiable predictions such as the pruning behavior shown in Figure 5. If the theoretical ranking guarantee and the internal sign conventions are repaired, the framework would be a useful practical contribution. At present, however, the central O(1/N^2) claim is not established, and the equations describing the conditional influence are internally inconsistent.

major comments (3)
  1. [Sections 3.1-3.2, Eq. (6)-(7), Proposition 1] The O(1/N^2) leave-one-out ranking guarantee is not established. In Appendix B.1, the derivation drops an additive constant after noting it is independent of x for a fixed Q. However, the finite-sample estimator in Eq. (7) uses Q_{-i} in the second term, so the omitted constant depends on i. The first-order expansion of the exact leave-one-out difference MMD(P,Q_n) - MMD(P,Q_{-i}) contains extra i-dependent terms such as <mu_P - mu_{Q_{-i}}, mu_{Q_{-i}}>, so cIF_i is not the first-order term of that difference. The citation to [55] does not cover this setting because the median-heuristic bandwidth is data-dependent and changes with each leave-one-out subset. Please either prove the O(1/N^2) ranking bound under explicit conditions (e.g., MMD(P,Q) bounded below, bounded kernel derivatives, score gaps Omega(1/N)) or replace the claim with a weaker O(1/N) statement and adjust the abstract and contribution list accordingly.
  2. [Section 3.4, Eq. (10), (13), (14), Algorithm 1] The sign and weighting of the conditional influence are internally inconsistent. Eq. (10) defines cIFcond as a negative quantity, while Eq. (13) sets R_i = ||y_i - yhat_i||_2 >= 0 and Eq. (14) adds lambda R_i, which would assign the highest scores to the most mislabeled points. Algorithm 1, line 15, further writes V_i = lambda(B_i - A_i) + (1-lambda)R_i, swapping the weights relative to Eq. (14). The experiments do not state which convention was actually implemented. Since the label-noise detection results depend directly on this sign, the authors must specify the intended definition, correct the equations and algorithm consistently, and confirm that the reported numbers correspond to the stated formula.
  3. [Section 3.2, Proposition 3 and Appendix B.3] The density-separation theorem proves only the existence of some Gaussian kernel with a sufficiently small bandwidth; it does not apply to the median-heuristic bandwidth (Appendix D) used in all experiments, and the appendix itself notes that finite samples may violate the separation. Consequently, the claimed 'density-separation for interpretable thresholds' is not a supported guarantee for the deployed method. Please state the result as a motivating property or prove separation for the actual bandwidth estimator under finite-sample conditions.
minor comments (5)
  1. [Section 3.3, Theorem 1] The statement's definition of ||L'||* is not well-formed: 'sup_{y in Y} L'(x,y)' is a scalar, not an element of H_Y, and the first norm inside the infimum is a function of x whose H_X norm should be written explicitly. Please rewrite the statement with standard notation and recheck the proof of the bounds for A and B.
  2. [Section 3.2, Proposition 2] The symmetry assumption is stated for all subsets S, but the proof only uses the maximal subset S = Dtrain \ {i,j}; either strengthen the statement to match the proof or clarify the intended axiom. Also, the notation \MMD is used without a formal definition.
  3. [Section 4 and Appendix D] The main text says the balancing factor lambda is fixed to 0.03, while Appendix D says it is determined by aligning the scale of the two components; please state the exact selection procedure and report sensitivity to lambda, since the relative weight of the conditional term directly affects label-noise detection.
  4. [Figure 1 and Section 4] The '99% overlap' claim for KAIROS's top-100 set should be defined precisely (e.g., Jaccard index between the top-100 sets) and should ideally be accompanied by standard errors or repeated-seed variability.
  5. [Throughout] There are typographical issues, such as 'archives' for 'achieves' in Section 4, and inconsistent formatting of 'KNNShapley' and 'LAVA'; these should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: KAIROS's MMD influence score is derived from first principles with external benchmarks and a textbook asymptotic citation; no fitted input is renamed as a prediction.

full rationale

I walked the derivation chain set out in the paper. Definition 1 defines distributional influence as the Gateaux derivative of a distributional distance; Proposition 1 derives the MMD influence from the closed-form squared-MMD identity by differentiating the kernel mean embedding expression and applying the chain rule, explicitly discarding additive and positive multiplicative constants. The finite-sample estimator in Eq. (7) is exactly the derived x-dependent term, evaluated with validation data in the first part and leave-one-out training data in the second part. No parameter is fitted to the leave-one-out rankings that KAIROS then 'predicts': the Gaussian bandwidth is fixed a priori by the median heuristic (citations [15, 6, 19], external), and the balancing coefficient is fixed at 0.03 with its stated role being scale alignment of the two components, not optimization against the benchmark targets. The empirical evaluations on CIFAR-10, STL-10, IMDB, AG News, and the poisoning/noise benchmarks are external comparisons, not inputs to the score. The O(1/N^2) leave-one-out approximation is attributed to Van der Vaart [55], an external standard textbook; whether its regularity conditions hold for the finite-sample Q_{-i} and the data-dependent median-heuristic bandwidth is a correctness risk, not a circularity risk. Proposition 2 is a conditional symmetry statement proved from the finite-sample estimator, and Proposition 3 is a direct consequence of the defining expression IF(x) = E_P[k(x, x')] - E_Q[k(x, x')]; neither imports a conclusion from the paper's own prior claims. The limitations paragraph explicitly acknowledges fixed kernels and a fixed balancing coefficient, which is a stated scope restriction rather than a circular step. I find no load-bearing reduction of a claimed prediction to a fitted input, no self-citation chain used to forbid alternatives, and no renaming of a known result as new derivation. The central derivation is self-contained given standard MMD theory and classical influence-function asymptotics.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no invented entities. It needs a clean validation distribution, a calibrated label classifier, and a kernel whose bandwidth is chosen by heuristic; the O(1/N^2) LOO fidelity is an imported asymptotic assumption. The hyperparameters are not fitted to the detection benchmarks, which keeps circularity low.

free parameters (2)
  • Gaussian kernel bandwidth sigma = Median of sampled pairwise distances, estimated from 10,000 pairs
    Set by the median heuristic on the input data; the theory in Proposition 3 only promises separation for a specially chosen small bandwidth, not for this heuristic choice.
  • Balancing factor lambda = 0.03
    Chosen by hand to align the scale of the marginal MMD and conditional E-MCMD terms in Eq. (12); its value changes the relative weight of feature versus label influence.
assumptions (4)
  • domain assumption Validation set Dval is a clean sample from the target distribution P.
    The whole valuation is defined as distance from training to this reference; if Dval is noisy or non-representative, scores are not meaningful. Used in Eq. (7) and throughout.
  • ad hoc to paper The first-order influence function approximates leave-one-out utility with O(1/N^2) error.
    Invoked in Section 3.1 with citation [55]; the paper does not verify the smoothness and regularity conditions for the MMD ranking task, and no proof is given for the finite-sample estimator in Eq. (7).
  • domain assumption A classifier trained on Dval yields calibrated conditional probabilities P(y|x) used in Eq. (10).
    The conditional influence depends on the estimated class probabilities; miscalibration would change the label-conditional scores and could break mislabel detection.
  • ad hoc to paper A Gaussian kernel with median-heuristic bandwidth satisfies the conditions needed for density separation and ranking fidelity.
    Experiments use this kernel; the formal density-separation proof in Appendix B.3 only shows existence of some sigma*, not that the chosen heuristic bandwidth satisfies the separation property.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KAIROS: Scalable Model-Agnostic Data Valuation." pith.science (2026). https://pith.science/paper/7QUEXYLF

@misc{pith2026250623799,
  author       = {Pith},
  title        = {Pith review of: KAIROS: Scalable Model-Agnostic Data Valuation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7QUEXYLF}},
  note         = {Machine review of arXiv:2506.23799}
}
abstract

Training data increasingly shapes not only model accuracy but also regulatory compliance and market valuation of AI assets. Yet existing valuation methods remain inadequate: model-based techniques depend on a single fitted model and inherit its biases, while algorithm-based approaches such as Data Shapley require costly retrainings at web scale. Recent Wasserstein-based model-agnostic methods rely on approximations that misrank examples relative to their true leave-one-out (LOO) utility. We introduce KAIROS, a scalable, model-agnostic valuation framework that assigns each example a distributional influence score: its contribution to the Maximum Mean Discrepancy (MMD) between the empirical training distribution and a clean reference set. Unlike Wasserstein surrogates, our MMD-based influence admits a closed-form solution that faithfully approximates the exact LOO ranking within $O(1/N^2)$ error, requires no retraining, and naturally extends to conditional kernels for unified label- and feature-error detection. Moreover, KAIROS supports efficient online updates: when a new batch of size m arrives, all scores can be updated in $O(mN)$ time, delivering up to 50x speedup without compromising ranking quality. Empirical evaluations on noise, mislabeling, and poisoning benchmarks show that KAIROS consistently outperforms state-of-the-art model-, Shapley-, and Wasserstein-based baselines in both accuracy and runtime. We provide rigorous theoretical guarantees, including symmetry for reproducible rankings and density-separation for interpretable thresholds.

Figures

Figures reproduced from arXiv: 2506.23799 by the authors.

Figure 1
Figure 1. Comparison of Wasserstein- and MMD-based influence methods. ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Feature noise detection results. 0.0 0.5 1.0 (a) 0.00 0.25 0.50 0.75 1.00 CIFAR 10 0.0 0.5 1.0 (b) 0.00 0.25 0.50 0.75 1.00 STL 10 0.0 0.5 1.0 (c) 0.00 0.25 0.50 0.75 1.00 IMDB 0.0 0.5 1.0 (d) 0.00 0.25 0.50 0.75 1.00 AG News % Inspected Data % Covered Corrupted Data KNNShapley DataOOB DVRL LAVA KAIROS [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Label noise detection results. Noise and Mislabel Detection. In this experiment, we introduce noise into 20% of the data. Following [25, 26, 64, 63], we inject feature noises by adding white noise to the images and randomly replacing words with other words for texts, and introduce label noises by randomly changing the labels of corrupted samples to other classes. Figures 2 and 3 present the performance of different … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Malicious data detection results. 0.0 0.1 0.2 (a) 0.90 0.92 0.94 0.96 STL-10 (Lowest) 0.0 0.1 0.2 (b) 0.85 0.90 0.95 STL-10 (Highest) 0.0 0.1 0.2 (c) 0.50 0.60 0.70 0.80 IMDB (Lowest) 0.0 0.1 0.2 (d) 0.60 0.70 0.80 IMDB (Highest) % Removed Data Test Accuracy KNNShapley…
Figure 5
Figure 5. Figure 5: Effect of removing the least valuable (a,c) and the most valuable (b,d) data points on test accuracy. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Label noise detection accuracy under varying validation set sizes. Effect of Validation Sample Size. In practice, validation sets are often small due to expensive labeling. To understand how many validation samples are needed to obtain reliable data values, we conduct …
Figure 7
Figure 7. Figure 7: Offline runtime comparison. Offline Runtime. To understand the scalability of different meth￾ods, we measure the runtime of methods on CIFAR-10 data with label noise. We vary the training set size and keep the static vali￾dation set size of 300. As shown in [PITH_FULL…
Figure 8
Figure 8. Figure 8: Online runtime comparison. Online Runtime. To show the adaptability of KAIROS in the online setting, we split 10000 of CIFAR-10 data into 100 batches, each containing 100 samples, and feed them in a streaming way. We mea￾sure the accumulated time taken to conduct data …
Figure 9
Figure 9. Figure 9: Influence distribution obtained from influence based on MMD (K [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 44 canonical work pages

  1. [55]

    Asymptotic statistics, volume 3

    Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000

    2000

  2. [1]

    Official Journal of the European Union L 2024/1689, 12 July 2024, July 2024

    Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union L 2024/1689, 12 July 2024, July 2024. CELEX 32024R1689

    work page 2024

  3. [2]

    A general class of coefficients of divergence of one distribution from another

    Syed Mumtaz Ali and Samuel D Silvey. A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society: Series B (Methodological), 28(1):131–142, 1966

    work page 1966

  4. [3]

    Fundamentals of task-agnostic data valuation

    Mohammad Mohammadi Amiri, Frederic Berdoz, and Ramesh Raskar. Fundamentals of task-agnostic data valuation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9226–9234, 2023

    work page 2023

  5. [4]

    Training data attribution via approximate unrolling

    Juhan Bae, Wu Lin, Jonathan Lorraine, and Roger B Grosse. Training data attribution via approximate unrolling. Advances in Neural Information Processing Systems, 37:66647–66686, 2024

    work page 2024

  6. [5]

    A theory of learning from different domains

    Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79:151–175, 2010

    2010

  7. [6]

    Mmd-fuse: Learning and combining kernels for two-sample testing without data splitting

    Felix Biggs, Antonin Schrab, and Arthur Gretton. Mmd-fuse: Learning and combining kernels for two-sample testing without data splitting. Advances in Neural Information Processing Systems, 36:75151–75188, 2023

    work page 2023

  8. [7]

    Truthful data acquisition via peer prediction

    Yiling Chen, Yiheng Shen, and Shuran Zheng. Truthful data acquisition via peer prediction. Advances in Neural Information Processing Systems, 33:18194–18204, 2020

    work page 2020

Show all 67 references
  1. [8]

    What is your data worth to gpt? llm-scale data valuation with influence functions

    Sang Keun Choe, Hwijeen Ahn, Juhan Bae, Kewen Zhao, Minsoo Kang, Youngseog Chung, Adithya Pratapa, Willie Neiswanger, Emma Strubell, Teruko Mitamura, et al. What is your data worth to gpt? llm-scale data valuation with influence functions. arXiv preprint arXiv:2405.13954, 2024

    2024 arXiv

  2. [9]

    An analysis of single-layer networks in unsupervised feature learning

    Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011

    2011

  3. [10]

    Joint distribution optimal trans- portation for domain adaptation

    Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal trans- portation for domain adaptation. Advances in neural information processing systems, 30, 2017

    2017

  4. [11]

    Stochastic amortization: A unified approach to accelerate feature and data attribution

    Ian Covert, Chanwoo Kim, Su-In Lee, James Y Zou, and Tatsunori B Hashimoto. Stochastic amortization: A unified approach to accelerate feature and data attribution. Advances in Neural Information Processing Systems, 37:4374–4423, 2024

    2024

  5. [12]

    Information-type measures of difference of probability distributions and indirect observation

    Imre Csiszár. Information-type measures of difference of probability distributions and indirect observation. studia scientiarum Mathematicarum Hungarica, 2:229–318, 1967

    1967

  6. [13]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013

    2013

  7. [14]

    EU AI Act’s ban on prohibited practices takes effect: First compliance milestones, 2025

    DLA Piper. EU AI Act’s ban on prohibited practices takes effect: First compliance milestones, 2025. Accessed 10 May 2025

    2025

  8. [15]

    Large sample analysis of the median heuristic

    Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa. Large sample analysis of the median heuristic. arXiv preprint arXiv:1707.07269, 2017

    2017 arXiv

  9. [16]

    Sample complexity of sinkhorn divergences

    Aude Genevay, Lénaic Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample complexity of sinkhorn divergences. In The 22nd international conference on artificial intelligence and statistics , pages 1574–1583. PMLR, 2019

    2019

  10. [17]

    A distributional framework for data valuation

    Amirata Ghorbani, Michael Kim, and James Zou. A distributional framework for data valuation. In International Conference on Machine Learning, pages 3535–3544. PMLR, 2020

    2020

  11. [18]

    Data shapley: Equitable valuation of data for machine learning

    Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. InInternational conference on machine learning, pages 2242–2251. PMLR, 2019

    2019

  12. [19]

    A kernel two-sample test

    Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012

    2012

  13. [20]

    Studying large language model generalization with influence functions

    Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large language model generalization with influence functions. arXiv preprint arXiv:2308.03296, 2023

    2023 arXiv

  14. [21]

    Badnets: Identifying vulnerabilities in the machine learning model supply chain

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017

    2017 arXiv

  15. [22]

    Lisa: Lazy safety alignment for large language models against harmful fine-tuning attack

    Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Tekin, and Ling Liu. Lisa: Lazy safety alignment for large language models against harmful fine-tuning attack. Advances in Neural Information Processing Systems , 37:104521–104555, 2024. 10

    2024

  16. [23]

    Efficient task-specific data valuation for nearest neighbor algorithms

    Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nezihe Merve Gurel, Bo Li, Ce Zhang, Costas J Spanos, and Dawn Song. Efficient task-specific data valuation for nearest neighbor algorithms. arXiv preprint arXiv:1908.08619, 2019

    1908 arXiv

  17. [24]

    Towards efficient data valuation based on the shapley value

    Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics, pa...

    2019

  18. [25]

    Opendataval: a unified benchmark for data valuation

    Kevin Jiang, Weixin Liang, James Y Zou, and Yongchan Kwon. Opendataval: a unified benchmark for data valuation. Advances in Neural Information Processing Systems, 36, 2023

    2023

  19. [26]

    Lava: Data valuation without pre-specified learning algorithms

    Hoang Anh Just, Feiyang Kang, Jiachen T Wang, Yi Zeng, Myeongseob Ko, Ming Jin, and Ruoxi Jia. Lava: Data valuation without pre-specified learning algorithms. arXiv preprint arXiv:2305.00054, 2023

    2023 arXiv

  20. [27]

    Revisiting methods for finding influential examples

    Karthik Karthikeyan and Anders Søgaard. Revisiting methods for finding influential examples. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9877–9885, 2022. arXiv:2111.04683

    2022 arXiv

  21. [29]

    Sava: Scalable learning-agnostic data valuation

    Samuel Kessler, Tam Le, and Vu Nguyen. Sava: Scalable learning-agnostic data valuation. In Proceedings of the 13th International Conference on Learning Representations (ICLR), 2025. arXiv:2406.01130

    2025 arXiv

  22. [30]

    Understanding black-box predictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885–1894. PMLR, 2017

    2017

  23. [31]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  24. [32]

    Beta shapley: a unified and noise-reduced data valuation framework for machine learning

    Yongchan Kwon and James Zou. Beta shapley: a unified and noise-reduced data valuation framework for machine learning. arXiv preprint arXiv:2110.14049, 2021

    2021 arXiv

  25. [33]

    Data-oob: Out-of-bag estimate as a simple and efficient data value

    Yongchan Kwon and James Zou. Data-oob: Out-of-bag estimate as a simple and efficient data value. In International conference on machine learning, pages 18135–18152. PMLR, 2023

    2023

  26. [34]

    Robust statistics—the approach based on influence functions, 1986

    John Law. Robust statistics—the approach based on influence functions, 1986

    1986

  27. [35]

    Learning transferable features with deep adaptation networks

    Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International conference on machine learning, pages 97–105. PMLR, 2015

    2015

  28. [36]

    Learning word vectors for sentiment analysis

    Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142–150, 2011

    2011

  29. [37]

    Integral probability metrics and their generating classes of functions

    Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in applied probability, 29(2):429–443, 1997

    1997

  30. [38]

    A measure-theoretic approach to kernel conditional mean embeddings

    Junhyung Park and Krikamol Muandet. A measure-theoretic approach to kernel conditional mean embeddings. Advances in neural information processing systems, 33:21247–21259, 2020

    2020

  31. [39]

    Trak: Attributing model behavior at scale

    Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. Trak: Attributing model behavior at scale. arXiv preprint arXiv:2303.14186, 2023

    2023 arXiv

  32. [40]

    Estimating training data influence by tracing gradient descent

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. In Proc. NeurIPS, pages 19920–19930, 2020

    2020

  33. [41]

    Mind the style of text! adversarial and backdoor attacks based on text style transfer

    Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, and Maosong Sun. Mind the style of text! adversarial and backdoor attacks based on text style transfer. arXiv preprint arXiv:2110.07139, 2021

    2021 arXiv

  34. [42]

    Adaptivity and computation-statistics tradeoffs for kernel and distance based high dimensional two sample testing

    Aaditya Ramdas, Sashank J Reddi, Barnabas Poczos, Aarti Singh, and Larry Wasserman. Adaptivity and computation-statistics tradeoffs for kernel and distance based high dimensional two sample testing. arXiv preprint arXiv:1508.00655, 2015

    2015 arXiv

  35. [43]

    On the high dimensional power of a linear-time two sample test under mean-shift alternatives

    Sashank Reddi, Aaditya Ramdas, Barnabás Póczos, Aarti Singh, and Larry Wasserman. On the high dimensional power of a linear-time two sample test under mean-shift alternatives. In Artificial Intelligence and Statistics, pages 772–780. PMLR, 2015

    2015

  36. [44]

    Theoretical analysis of domain adaptation with optimal transport

    Ievgen Redko, Amaury Habrard, and Marc Sebban. Theoretical analysis of domain adaptation with optimal transport. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part II 10, pa...

    2017

  37. [45]

    Conditional generative moment-matching networks

    Yong Ren, Jun Zhu, Jialian Li, and Yucen Luo. Conditional generative moment-matching networks. Advances in Neural Information Processing Systems, 29, 2016

    2016

  38. [46]

    Time, openai sign multi-year content deal, June 2024

    Reuters. Time, openai sign multi-year content deal, June 2024. Published 27 Jun 2024. 11

    2024

  39. [47]

    Cs-shapley: class-wise shapley values for data valuation in classification

    Stephanie Schoch, Haifeng Xu, and Yangfeng Ji. Cs-shapley: class-wise shapley values for data valuation in classification. Advances in Neural Information Processing Systems, 35:34574–34585, 2022

    2022

  40. [48]

    Poison frogs! targeted clean-label poisoning attacks on neural networks.Advances in neural information processing systems, 31, 2018

    Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks on neural networks.Advances in neural information processing systems, 31, 2018

    2018

  41. [49]

    A value for n-person games

    Lloyd S Shapley et al. A value for n-person games. 1953

    1953

  42. [50]

    Ai deals to make up a third of shutterstock’s revenue by 2027, June 2024

    Sherwood News. Ai deals to make up a third of shutterstock’s revenue by 2027, June 2024. Accessed 10 May 2025

    2027

  43. [51]

    Hilbert space embeddings and metrics on probability measures

    Bharath K Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, and Gert RG Lanckriet. Hilbert space embeddings and metrics on probability measures. The Journal of Machine Learning Research, 11:1517– 1561, 2010

    2010

  44. [52]

    Density ratio estimation: A comprehensive review (statistical experiment and its related topics)

    Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation: A comprehensive review (statistical experiment and its related topics). RIMS Kokyuroku (Research Institute for Mathematical Sciences, Kyoto University), 1703:10–31, 2010

    2010

  45. [53]

    2d-oob: Attributing data contribution through joint valuation framework

    Yifan Sun, Jingyan Shen, and Yongchan Kwon. 2d-oob: Attributing data contribution through joint valuation framework. arXiv preprint arXiv:2408.03572, 2024

    2024 arXiv

  46. [54]

    Incentivizing collaboration in machine learning via synthetic data rewards

    Sebastian Shenghong Tay, Xinyi Xu, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Incentivizing collaboration in machine learning via synthetic data rewards. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9448–9456, 2022

    2022

  47. [56]

    Topics in optimal transportation, volume 58

    Cédric Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021

    2021

  48. [57]

    Optimal transport: old and new, volume 338

    Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2008

    2008

  49. [58]

    Data banzhaf: A robust data valuation framework for machine learning

    Jiachen T Wang and Ruoxi Jia. Data banzhaf: A robust data valuation framework for machine learning. In International Conference on Artificial Intelligence and Statistics, pages 6388–6421. PMLR, 2023

    2023

  50. [59]

    Data shapley in one training run

    Jiachen T Wang, Prateek Mittal, Dawn Song, and Ruoxi Jia. Data shapley in one training run. arXiv preprint arXiv:2406.11011, 2024

    2024 arXiv

  51. [60]

    Rethinking data shapley for data selection tasks: Misleads and merits

    Jiachen T Wang, Tianji Yang, James Zou, Yongchan Kwon, and Ruoxi Jia. Rethinking data shapley for data selection tasks: Misleads and merits. arXiv preprint arXiv:2405.03875, 2024

    2024 arXiv

  52. [61]

    Lossval: Efficient data valuation for neural networks

    Tim Wibiral, Mohamed Karim Belaid, Maximilian Rabus, and Ansgar Scherp. Lossval: Efficient data valuation for neural networks. arXiv preprint arXiv:2412.04158, 2024

    2024 arXiv

  53. [62]

    Davinz: Data valuation using deep neural networks at initialization

    Zhaoxuan Wu, Yao Shu, and Bryan Kian Hsiang Low. Davinz: Data valuation using deep neural networks at initialization. In International Conference on Machine Learning, pages 24150–24176. PMLR, 2022

    2022

  54. [63]

    Noising and denoising natural language: Diverse backtranslation for grammar correction

    Ziang Xie, Guillaume Genthial, Stanley Xie, Andrew Y Ng, and Dan Jurafsky. Noising and denoising natural language: Diverse backtranslation for grammar correction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisti...

    2018

  55. [64]

    Data noising as smoothing in neural network language models

    Ziang Xie, Sida I Wang, Jiwei Li, Daniel Lévy, Aiming Nie, Dan Jurafsky, and Andrew Y Ng. Data noising as smoothing in neural network language models. arXiv preprint arXiv:1703.02573, 2017

    2017 arXiv

  56. [65]

    Data distribution valuation

    Xinyi Xu, Shuaiqi Wang, Chuan Sheng Foo, Bryan Kian Hsiang Low, and Giulia Fanti. Data distribution valuation. Advances in Neural Information Processing Systems, 37:2407–2448, 2024

    2024

  57. [66]

    Validation free and replication robust volume-based data valuation

    Xinyi Xu, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Validation free and replication robust volume-based data valuation. Advances in Neural Information Processing Systems, 34:10837–10848, 2021

    2021

  58. [67]

    Data valuation using reinforcement learning

    Jinsung Yoon, Sercan Arik, and Tomas Pfister. Data valuation using reinforcement learning. In International Conference on Machine Learning, pages 10842–10851. PMLR, 2020

    2020

  59. [68]

    median heuristic

    Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015. 12 A Related Work Model-based Data Valuation A common approach for data valuation is the leave-one-out (LOO) sc...

    2015

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.