Pith. sign in

REVIEW 5 major objections 5 minor 71 references

Dataset distillation can be solved by matching the influence of data on the final model rather than its training dynamics.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 19:43 UTC pith:C3NJRXP2

load-bearing objection Novel outcome-matching objective with strong results, but the theoretical core is asserted, and the additivity assumption behind Eq. (6) is load-bearing and unverified. the 5 major comments →

arxiv 2607.16859 v1 pith:C3NJRXP2 submitted 2026-07-18 cs.CV

Dataset Distillation by Influence Matching

Haoru Tan , Wang Wang , Sitong Wu , Xiuzhe Wu , Yangtian Sun , Chirui Chang , Shaofeng Zhang , Xiaojuan Qi This is my paper
classification cs.CV
keywords dataset distillationinfluence functionsoutcome matchingparameter alignmentdifferentiable influence estimatorSGD trajectory unrollingvision-language distillation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that dataset distillation should optimize for the final outcome—what the full dataset does to the converged model parameters—instead of heuristic proxies like aligning per-step gradients or training trajectories. It introduces a sample-level influence estimator that runs in linear time by unrolling the optimization and using first-order Taylor approximation, avoiding inverse-Hessian products. The synthetic set is learned by minimizing the mismatch between the influence of removing the whole real dataset and adding the synthetic set, which, by additivity of influences, is equivalent to aligning the final models. The authors report state-of-the-art accuracy on CIFAR-10/100, Tiny-ImageNet, and Flickr30K, with gains up to +4.7% over the previous best method.

Core claim

The central claim is that the objective ||I⁻_D + I⁺_S||, where I⁻_D is the parameter shift from removing the real dataset and I⁺_S the shift from adding the synthetic set, equals (up to additivity) the displacement between θ*_D and θ*_S, the models trained on real and synthetic data. Therefore, minimizing that norm directly performs outcome alignment. The paper derives a differentiable estimator for these influences from an SGD trajectory and proves a polynomial (instead of exponential) worst-case error bound. In practice, Inf-Match achieves the best classification accuracy on all tested benchmarks and sets new results on vision-language distillation.

What carries the argument

The key object is the influence estimator defined by unrolling the optimization: for each checkpoint along the trajectory, it computes a Hessian-gradient product (H^t_D G^t_Z + H^t_Z G^t_D) weighted by the remaining step fraction, and sums over steps. This yields removal/addition influence estimates without inverse Hessians or convexity. The identity ||I⁻_D + I⁺_S|| = ||θ*_S - θ*_D|| is what converts dataset distillation into a parameter-matching problem.

Load-bearing premise

The objective assumes that removing the full dataset and adding the synthetic set have additive influences that exactly sum to the difference between the two final trained models; this holds for linear models or approximate influence functions, not necessarily for retrained deep networks.

What would settle it

Train the same architecture on the real dataset D and on the distilled set S from Inf-Match, using identical initializations and hyperparameters, and measure the actual parameter distance ||θ*_S - θ*_D||. If this distance is not significantly smaller than the distance between θ*_D and a model trained on a randomly selected subset of the same size, the outcome-alignment claim is falsified. A complementary check is to compare the optimized objective value with the realized displacement over a range of IPC values.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If outcome alignment is the right objective, then process-matching methods (gradient/trajectory) are suboptimal by construction, and further gains should come from better influence estimators rather than better proxies.
  • The linear-time, convexity-free estimator makes influence-based distillation tractable on large and multimodal datasets, as demonstrated on Flickr30K.
  • Synthetic images tuned for influence can look noisy yet transfer across architectures, indicating that visual fidelity is not a reliable proxy for distilled utility.
  • The objective's dependence on a reference trajectory suggests that distillation quality is tied to the quality of the base model that produced the checkpoints.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • We infer that the additivity identity in Remark 1 is an equality only for linear models or approximate influence functions; for retrained nonconvex networks it is an approximation, and whether minimizing the estimated mismatch actually yields θ*_S ≈ θ*_D is not empirically verified in the paper.
  • We infer that the error bound in Theorem 2, growing as T^3, implies a trade-off: using more checkpoints improves trajectory coverage but may worsen the worst-case guarantee; a checkpoint-sampling schedule that balances both is an implicit design choice worth testing.
  • We infer that extending the method to adaptive optimizers (Adam, etc.) would require revising the first-order Taylor argument, making that a testable extension.
  • We infer that independently reproducing the reported Tiny-ImageNet gain with a fixed training budget would clarify the practical margin of the method.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a new dataset distillation method, Influence Matching (Inf-Match), which learns a synthetic set so that its estimated influence on converged model parameters matches the estimated influence of the full dataset. The authors define removal and addition influences, introduce a differentiable estimator by unrolling SGD and applying a first-order Taylor approximation, and minimize the norm of the sum of the removal influence of the real data and the addition influence of the synthetic data. Experiments on CIFAR-10/100, Tiny-ImageNet, and Flickr30K report state-of-the-art accuracy and retrieval performance.

Significance. If the outcome-alignment claim were rigorously established, this would be a conceptual advance over process-matching baselines. The empirical results are strong, the method scales to vision-language tasks, and the code is promised. However, the theoretical foundations are currently not established: the core additivity argument is a linearization that is not valid for retrained nonconvex networks, and the two theorems are stated without proof. The reported gains also lack variance information, making some SOTA claims difficult to assess.

major comments (5)
  1. [§4.2, Remark 1 and Eq. (6)] The central claim that minimizing ||I^-_D + I^+_S|| aligns final model parameters is not established. With the definitions in §4.1, I^-_D = θ*_∅ − θ*_D and I^+_S = θ*_{D+S} − θ*_D, so their sum is not θ*_S − θ*_D. Additivity of influences holds only under a linear/quadratic approximation around the same optimum, not for exact retraining of nonconvex networks. The paper provides neither a proof nor an empirical check (e.g., correlating Eq. (6) with ||θ*_S − θ*_D|| on a small problem). As submitted, Eq. (6) is an unvalidated surrogate for outcome alignment.
  2. [§4.1, Theorems 1 and 2] Theorems 1 and 2 are stated without proof or derivation. Theorem 1's estimator (Eqs. 2–3) is the core of the method; without a derivation, the reader cannot assess what approximation is made or whether it actually estimates the defined influences. Theorem 2's error bound is also unproved, and its form (e.g., the T^3 factor, the role of |Z|/|D|, the constant '2') is not justified. No empirical validation of the estimator against exact retraining is reported. These theorems must either be proven (or moved to an appendix with full proofs) or removed and replaced by clearly labeled heuristic claims.
  3. [§5.1 and all result tables/figures] The paper states that each experiment is repeated 10 times, but no standard deviations or confidence intervals are reported anywhere (Figure 2, Figure 4, Table 1, Figure 3). Several claimed improvements are small (e.g., CIFAR-10 IPC=10 72.5% vs NCFM ~71.8%; Flickr30K I2T 7.4% vs DATM 6.1%). Without variance estimates, these differences are not statistically supportable. Report mean ± std over the 10 repeats.
  4. [§5.5] The statement that 'our approach, which directly optimizes the original problem' is not supported, given that Eq. (6) is a surrogate whose connection to the original bilevel problem is exactly the unproven additivity assertion. This claim should be reframed as approximate outcome matching, or supported by a direct comparison of the optimized objective with the true parameter displacement.
  5. [§4.2, Eq. (7) and Algorithm 1] The paper claims that using random mini-batches in Eq. (7) 'preserves unbiased gradient estimates.' However, the loss is the norm of a sum of expectations; the gradient of the norm of an unbiased stochastic estimate is not an unbiased estimator of the gradient of the norm of the expectation. This is a technical but load-bearing issue for the mini-batch optimization, and it should be addressed or the claim removed.
minor comments (5)
  1. [Section 1] Duplicate phrase: 'without any convexity assumptions or inverse-Hessian computation. without convexity assumptions and without inverse–Hessian computations.' Remove the duplicated sentence.
  2. [Section 2] Subsection numbering is inconsistent: after '(2). Process matching' there is another '(2). Others.' It should be '(3). Others.'
  3. [Eq. (5) and Eq. (6)] The notation |·| is used both for the vector norm (Eq. 6) and for set cardinality / absolute value (Eq. 5). Use a distinct notation such as ||·|| for norms and |D| for cardinality to avoid confusion.
  4. [§5.2.2] The text says 'See Table 3 for details,' but the cross-architecture results are displayed as Figure 3. Update the cross-reference.
  5. [Abstract] The phrase 'leading impressive average' is awkward and ambiguous. Clarify what average is being reported (e.g., mean over settings) and how the 2.5% improvement is computed.

Circularity Check

0 steps flagged

No significant circularity: reported gains are external and the objective is not a fitted input renamed as a prediction, though the additivity link to outcome alignment is unproved.

full rationale

I walked the derivation chain. The central objective Eq.(6) minimizes ||I^-_D + I^+_S||, where the influences are defined as parameter differences from θ*_D. The synthetic set is optimized against this surrogate and then evaluated by actual retraining on CIFAR/Tiny-ImageNet/Flickr30K, so the reported accuracy is not forced by construction. The load-bearing link from Eq.(6) to outcome alignment is Remark 1, which invokes "the additivity of influence functions [4,28,62]" and writes I^-_D + I^+_S = (θ*_D + I^-_D + I^+_S) - θ*_D. That displayed identity is a tautology; the substantive claim that this sum equals θ*_S - θ*_D is not proven and is not a consequence of the paper's own definitions, since I^-_D = θ*_∅ - θ*_D and I^+_S = θ*_{D∪S} - θ*_D generally do not combine to θ*_S - θ*_D for retrained nonconvex networks. This is a correctness/assumption gap, not circularity: the equality is not an input to the optimization, and the final evaluation does not use the estimator's own prediction as ground truth. The paper's self-citations [49,50,51] appear only in the related-work survey and are not used to justify the estimator or the additivity step. Theorems 1 and 2 are stated without proof, weakening support but not making the derivation circular. No exhibited reduction of a prediction to its own fitted inputs was found; score 1 reflects the minor non-load-bearing self-citations and the unvalidated surrogate rather than a circular core.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method introduces no new physical entities, but it does introduce a new surrogate objective whose validity depends on two unproved pillars: the influence estimator itself and the additivity of exact influences. Both are load-bearing for the central claim.

free parameters (4)
  • synthetic image learning rate = 50.0
    Hand-chosen in Sec 5.1; the large value is standard in dataset distillation but is a tuned constant.
  • synthetic soft-label learning rate = 7.0
    Hand-chosen in Sec 5.1; controls how fast soft labels are updated.
  • checkpoint count tm and sampling schedule
    Algorithm 1 samples tm checkpoints from the trajectory; exact tm and the 'similar to DATM' schedule are not specified, and they directly control the loss in Eq(7).
  • finite-difference epsilon in Eq(4)
    The Hessian-gradient product is approximated via a finite difference; the epsilon value is implicit in the autodiff implementation and not reported.
axioms (5)
  • ad hoc to paper Eqs(2)-(3): the true parameter shift from adding/removing data equals a weighted sum of Hessian-gradient products along the real-data SGD trajectory.
    Stated as Theorem 1 without proof or derivation in Sec 4.1; the unrolling/Taylor argument is not shown.
  • ad hoc to paper Exact removal/addition influences add linearly: I^-_D + I^+_S = θ*_S - θ*_D.
    Invoked in Remark 1 to justify Eq(6); true only for approximate/linear influence, not for exact retrained parameter differences on nonconvex losses.
  • domain assumption Gradient ℓ-Lipschitz continuity and bounded gradient norm g, used in Theorem 2.
    Standard smoothness assumptions, but no proof of the stated bound is provided.
  • domain assumption A model trained from scratch on the synthetic set behaves like the influence estimated at real-data checkpoints.
    The outcome-matching claim requires this transfer, and it is asserted without verification.
  • standard math Mini-batch sampling gives unbiased gradients of the influence-matching objective.
    Assumed in Algorithm 1; reasonable, but requires independent sampling assumptions.

pith-pipeline@v1.3.0-alltime-deepseek · 13605 in / 13203 out tokens · 135412 ms · 2026-08-01T19:43:35.081030+00:00 · methodology

0 comments
read the original abstract

We revisit dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5\%, a +4.7\% improvement over NCFM. Beyond classification, Inf-Match scales to vision-language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5\%. The code will be released via https://github.com/hrtan/infmatch.

Figures

Figures reproduced from arXiv: 2607.16859 by Chirui Chang, Haoru Tan, Shaofeng Zhang, Sitong Wu, Wang Wang, Xiaojuan Qi, Xiuzhe Wu, Yangtian Sun.

Figure 1
Figure 1. Figure 1: This figure outlines three core paradigms for data comparison or generation: (a) Feature matching, where a feature extractor is [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance comparison of different methods on CIFAR-10, CIFAR-100, and Tiny-ImageNet. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance comparison of different dataset distillation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualization of the learning curve (MTT [ [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: An illustration of why our method performs well for [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

71 extracted references · 15 linked inside Pith

  1. [1]

    Automatic differentiation in py- torch

    Adam Paszke, Sam Gross, Soumith Chintala, G Chanan, E Yang, Zachary Devito, Zeming Lin, Alban Desmaison, L Antiga, A Lerer, and et.al. Automatic differentiation in py- torch. InAdvances in neural information processing systems Workshop, 2017. 4

    2017

  2. [2]

    Learning multiple layers of features from tiny images.Technical report, 2009

    Alex Krizhevsky. Learning multiple layers of features from tiny images.Technical report, 2009. 6

    2009

  3. [3]

    Neural networks as kernel learners: The silent alignment effect.arXiv preprint arXiv:2111.00034, 2021

    Alexander Atanasov, Blake Bordelon, and Cengiz Pehlevan. Neural networks as kernel learners: The silent alignment effect.arXiv preprint arXiv:2111.00034, 2021. 3

    Pith/arXiv arXiv 2021

  4. [4]

    On second- order group influence functions for black-box predictions

    Samyadeep Basu, Xuchen You, and Soheil Feizi. On second- order group influence functions for black-box predictions. In International Conference on Machine Learning, 2020. 1, 3, 5

    2020

  5. [5]

    Flexible dataset distillation: Learn labels instead of images

    Ondrej Bohdal, Yongxin Yang, and Timothy Hospedales. Flexible dataset distillation: Learn labels instead of images. InProceedings of the Advances in Neural Information Pro- cessing Systems (NeurIPS), Workshop, 2020. 5

    2020

  6. [6]

    Smith, and Karen Si- monyan

    Andy Brock, Soham De, Samuel L. Smith, and Karen Si- monyan. High-performance large-scale image recognition without normalization. InProceedings of the 38th Interna- tional Conference on Machine Learning, pages 1059–1071. PMLR, 2021. 7

    2021

  7. [7]

    Dataset distillation by matching training trajectories

    George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4750–4759, 2022. 1, 2, 3, 6, 7, 8

    2022

  8. [8]

    Generalizing dataset distillation via deep generative prior

    George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Generalizing dataset distillation via deep generative prior. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3739–3748, 2023. 3

    2023

  9. [9]

    Rkhs-shap: Shapley values for kernel methods.Ad- vances in neural information processing systems, 35:13050– 13063, 2022

    Siu Lun Chau, Robert Hu, Javier Gonzalez, and Dino Sejdi- novic. Rkhs-shap: Shapley values for kernel methods.Ad- vances in neural information processing systems, 35:13050– 13063, 2022. 3

    2022

  10. [10]

    Aligning effective tokens with video anomaly in large language models

    Yingxian Chen, Jiahui Liu, Ruidi Fan, Yanwei Li, Chirui Chang, Shizhen Zhao, Wilton WT Fok, Xiaojuan Qi, and Yik- Chung Wu. Aligning effective tokens with video anomaly in large language models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 22695– 22706, 2025. 7

    2025

  11. [11]

    The loss surfaces of multi- layer networks.Journal of Machine Learning Research, 38: 192–204, 2015

    Anna Choromanska, Mikael Henaff, Michael Mathieu, Ger- ard Ben Arous, and Yann LeCun. The loss surfaces of multi- layer networks.Journal of Machine Learning Research, 38: 192–204, 2015. 3

    2015

  12. [12]

    Assessment of local influence.Journal of the Royal Statistical Society Series B: Statistical Methodology, 48(2):133–155, 1986

    R Dennis Cook. Assessment of local influence.Journal of the Royal Statistical Society Series B: Statistical Methodology, 48(2):133–155, 1986. 3

    1986

  13. [13]

    Scaling up dataset distillation to imagenet-1k with constant memory

    Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet-1k with constant memory. InInternational Conference on Machine Learning, pages 6565–6590. PMLR, 2023. 2, 3, 6

    2023

  14. [14]

    Iden- tifying and attacking the saddle point problem in high- dimensional non-convex optimization

    Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. Iden- tifying and attacking the saddle point problem in high- dimensional non-convex optimization. InAdvances in Neural Information Processing Systems, pages 2933–2941, 2014. 3

    2014

  15. [15]

    Remember the past: Distilling datasets into addressable memories for neural net- works.Advances in Neural Information Processing Systems, 35:34391–34404, 2022

    Zhiwei Deng and Olga Russakovsky. Remember the past: Distilling datasets into addressable memories for neural net- works.Advances in Neural Information Processing Systems, 35:34391–34404, 2022. 3

    2022

  16. [16]

    Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018. 6, 7

    Pith/arXiv arXiv 2018

  17. [17]

    Privacy for free: How does dataset condensation help privacy? InInternational Conference on Machine Learning, pages 5378–5396

    Tian Dong, Bo Zhao, and Lingjuan Lyu. Privacy for free: How does dataset condensation help privacy? InInternational Conference on Machine Learning, pages 5378–5396. PMLR,

  18. [18]

    An image is worth 16x16 words: Transformers for image recognition at scale.ICLR, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale.ICLR, 2021. 6

    2021

  19. [19]

    Minimizing the accumulated trajectory error to improve dataset distillation

    Jiawei Du, Yidi Jiang, Vincent YF Tan, Joey Tianyi Zhou, and Haizhou Li. Minimizing the accumulated trajectory error to improve dataset distillation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3749–3758, 2023. 2, 3

    2023

  20. [20]

    Sequential subset matching for dataset distillation.Advances in Neural Infor- mation Processing Systems, 36, 2024

    Jiawei Du, Qin Shi, and Joey Tianyi Zhou. Sequential subset matching for dataset distillation.Advances in Neural Infor- mation Processing Systems, 36, 2024. 3

    2024

  21. [21]

    Embarrassingly simple dataset distillation

    Yunzhen Feng, Shanmukha Ramakrishna Vedantam, and Julia Kempe. Embarrassingly simple dataset distillation. InThe Twelfth International Conference on Learning Representa- tions, 2023. 2

    2023

  22. [22]

    Studying large language model generalization with influence functions.arXiv preprint arXiv:2308.03296, 2023

    Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large language model generalization with influence functions.arXiv preprint arXiv:2308.03296, 2023. 1, 3, 4

    Pith/arXiv arXiv 2023

  23. [23]

    Sum- marizing stream data for memory-restricted online continual learning.arXiv preprint arXiv:2305.16645, 2023

    Jianyang Gu, Kai Wang, Wei Jiang, and Yang You. Sum- marizing stream data for memory-restricted online continual learning.arXiv preprint arXiv:2305.16645, 2023. 1

    Pith/arXiv arXiv 2023

  24. [24]

    Towards lossless dataset distillation via difficulty-aligned trajectory matching.arXiv preprint arXiv:2310.05773, 2023

    Ziyao Guo, Kai Wang, George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You. Towards lossless dataset distillation via difficulty-aligned trajectory matching.arXiv preprint arXiv:2310.05773, 2023. 1, 2, 3, 5, 6, 7, 8

    Pith/arXiv arXiv 2023

  25. [25]

    Training data influ- ence analysis and estimation: A survey.arXiv preprint arXiv:2212.04612, 2022

    Zayd Hammoudeh and Daniel Lowd. Training data influ- ence analysis and estimation: A survey.arXiv preprint arXiv:2212.04612, 2022. 3

    Pith/arXiv arXiv 2022

  26. [26]

    Data cleansing for models trained with sgd.Advances in Neural Information Processing Systems, 32, 2019

    Satoshi Hara, Atsushi Nitanda, and Takanori Maehara. Data cleansing for models trained with sgd.Advances in Neural Information Processing Systems, 32, 2019. 4

    2019

  27. [27]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InIEEE / CVF Computer Vision and Pattern Recognition Conference, 2016. 6

    2016

  28. [28]

    Pang Koh, Kai-Siang Ang, Hubert H. K. Teo, and Percy Liang. On the accuracy of influence functions for measuring group effects. InAdvances in neural information processing systems,

  29. [29]

    Understanding black-box pre- dictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black-box pre- dictions via influence functions. InInternational Conference on Machine Learning, 2017. 1, 3, 4

    2017

  30. [30]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Im- agenet classification with deep convolutional neural networks. InProceedings of the International Conference on Neural Information Processing Systems, page 1097–1105, 2012. 6

    2012

  31. [31]

    Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models.arXiv preprint arXiv:2310.00902,

    Yongchan Kwon, Eric Wu, Kevin Wu, and James Zou. Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models.arXiv preprint arXiv:2310.00902,

    Pith/arXiv arXiv

  32. [32]

    Lecun, L

    Y . Lecun, L. Bottou, Y . Bengio, and P. Haffner. Gradient- based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998. 6

    1998

  33. [33]

    Awesome dataset distillation

    Guang Li, Bo Zhao, and Tongzhou Wang. Awesome dataset distillation. https : / / github . com / Guang000 / Awesome-Dataset-Distillation, 2022. 1

    2022

  34. [34]

    Mars3d: A plug-and-play motion- aware model for semantic segmentation on multi-scan 3d point clouds

    Jiahui Liu, Chirui Chang, Jianhui Liu, Xiaoyang Wu, Lan Ma, and Xiaojuan Qi. Mars3d: A plug-and-play motion- aware model for semantic segmentation on multi-scan 3d point clouds. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9372–9381,

  35. [35]

    Very deep convolutional neural network based image classification using small training sample size

    Shuying Liu and Weihong Deng. Very deep convolutional neural network based image classification using small training sample size. In2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pages 730–734, 2015. 6

    2015

  36. [36]

    Efficient dataset distillation using random feature approxima- tion.Advances in Neural Information Processing Systems, 35:13877–13891, 2022

    Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Efficient dataset distillation using random feature approxima- tion.Advances in Neural Information Processing Systems, 35:13877–13891, 2022. 3

    2022

  37. [37]

    Herding dynamical weights to learn

    Max Welling. Herding dynamical weights to learn. InInter- national Conference on Machine Learning, 2009. 6, 7

    2009

  38. [38]

    Dataset meta-learning from kernel ridge-regression

    Timothy Nguyen, Zhourong Chen, and Jaehoon Lee. Dataset meta-learning from kernel ridge-regression. InInternational Conference on Learning Representations, 2020. 3

    2020

  39. [39]

    Pearlmutter

    Barak A. Pearlmutter. Fast exact multiplication by the hessian. Neural Computation, 6(1):147–160, 1994. 4

    1994

  40. [40]

    Flickr30k entities: Collecting region-to-phrase correspon- dences for richer image-to-sentence models

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspon- dences for richer image-to-sentence models. InProceedings of the IEEE international conference on computer vision, pages 2641–2649, 2015. 7

    2015

  41. [41]

    Estimating training data influence by tracing gradient descent.arXiv preprint arXiv:2002.08484, 2020

    Garima Pruthi, Frederick Liu, Sundararajan Mukund, and Satyen Kale. Estimating training data influence by tracing gradient descent.arXiv preprint arXiv:2002.08484, 2020. 4

    Pith/arXiv arXiv 2002

  42. [42]

    Datadam: Efficient dataset distillation with attention matching

    Ahmad Sajedi, Samir Khaki, Ehsan Amjadian, Lucy Z Liu, Yuri A Lawryshyn, and Konstantinos N Plataniotis. Datadam: Efficient dataset distillation with attention matching. InPro- ceedings of the IEEE/CVF International Conference on Com- puter Vision, pages 17097–17107, 2023. 2

    2023

  43. [43]

    Scaling up influence functions

    Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. InProceedings of the AAAI Conference on Artificial Intelligence, 2022. 1, 3, 4

    2022

  44. [44]

    Theoretical and practical perspectives on what influence functions do.Advances in Neural Information Pro- cessing Systems, 36, 2024

    Andrea Schioppa, Katja Filippova, Ivan Titov, and Polina Zablotskaia. Theoretical and practical perspectives on what influence functions do.Advances in Neural Information Pro- cessing Systems, 36, 2024. 4

    2024

  45. [45]

    A value for n-person games

    Lloyd S Shapley et al. A value for n-person games. 1953. 3

    1953

  46. [46]

    Generalized large-scale data condensation via var- ious backbone and statistical matching.arXiv preprint arXiv:2311.17950, 2023

    Xindong Zhang Shitong Shao, Zeyuan Yin and Zhiqiang Shen. Generalized large-scale data condensation via var- ious backbone and statistical matching.arXiv preprint arXiv:2311.17950, 2023. 6

    Pith/arXiv arXiv 2023

  47. [47]

    Soft-label dataset distillation and text dataset distillation

    Ilia Sucholutsky and Matthias Schonlau. Soft-label dataset distillation and text dataset distillation. In2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2021. 3

    2021

  48. [48]

    On the diversity and realism of distilled dataset: An efficient dataset distilla- tion paradigm.arXiv preprint arXiv:2312.03526, 2023

    Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin. On the diversity and realism of distilled dataset: An efficient dataset distilla- tion paradigm.arXiv preprint arXiv:2312.03526, 2023. 3

    Pith/arXiv arXiv 2023

  49. [49]

    Data pruning via moving-one- sample-out

    Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moving-one- sample-out. InAdvances in neural information processing systems, 2023. 3

    2023

  50. [50]

    Understanding data influence with differential approximation, 2025

    Haoru Tan, Sitong Wu, Xiuzhe Wu, Wang Wang, Bo Zhao, Zeke Xie, Gui-Song Xia, and Xiaojuan Qi. Understanding data influence with differential approximation, 2025

    2025

  51. [51]

    Understanding data influence in reinforcement finetuning

    Haoru Tan, Xiuzhe Wu, Sitong Wu, Shaofeng Zhang, Yan- feng Chen, Xingwu Sun, Jeanne Shen, and XIAOJUAN QI. Understanding data influence in reinforcement finetuning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 3

    2025

  52. [52]

    Cafe: Learning to condense dataset by align- ing features

    Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by align- ing features. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12196– 12205, 2022. 1, 2

    2022

  53. [53]

    Dim: Distilling dataset into generative model

    Kai Wang, Jianyang Gu, Daquan Zhou, Zheng Zhu, Wei Jiang, and Yang You. Dim: Distilling dataset into generative model. arXiv preprint arXiv:2303.04707, 2023. 3

    Pith/arXiv arXiv 2023

  54. [54]

    Dataset distil- lation with neural characteristic function: A minmax perspec- tive, 2025

    Shaobo Wang, Yicun Yang, Zhiyuan Liu, Chenghao Sun, Xuming Hu, Conghui He, and Linfeng Zhang. Dataset distil- lation with neural characteristic function: A minmax perspec- tive, 2025. 1, 2, 6, 8

    2025

  55. [55]

    Dataset distillation.arXiv preprint arXiv:1811.10959, 2018

    Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation.arXiv preprint arXiv:1811.10959, 2018. 1, 3, 5, 6

    Pith/arXiv arXiv 2018

  56. [56]

    Saco loss: Sample-wise affinity consistency for vision-language pre-training

    Sitong Wu, Haoru Tan, Zhuotao Tian, Yukang Chen, Xiaojuan Qi, and Jiaya Jia. Saco loss: Sample-wise affinity consistency for vision-language pre-training. In2024 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 27348–27359, 2024. 7

    2024

  57. [57]

    Mixture- of-scores: Robust image-text data valuation via three lines of code

    Sitong Wu, Haoru Tan, Yukang Chen, Shaofeng Zhang, Jingyao Li, Bei Yu, Xiaojuan Qi, and Jiaya Jia. Mixture- of-scores: Robust image-text data valuation via three lines of code. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision (ICCV), pages 24603–24614, 2025. 7

    2025

  58. [58]

    Multimodal dataset distillation for image-text re- trieval.arXiv preprint arXiv:2308.07545, 2023

    Xindi Wu, Byron Zhang, Zhiwei Deng, and Olga Rus- sakovsky. Multimodal dataset distillation for image-text re- trieval.arXiv preprint arXiv:2308.07545, 2023. 7

    Pith/arXiv arXiv 2023

  59. [59]

    Dreamomni2: Multimodal instruction-based editing and generation, 2025

    Bin Xia, Bohao Peng, Yuechen Zhang, Junjia Huang, Jiyang Liu, Jingyao Li, Haoru Tan, Sitong Wu, Chengyao Wang, Yi- tong Wang, Xinglong Wu, Bei Yu, and Jiaya Jia. Dreamomni2: Multimodal instruction-based editing and generation, 2025. 7

    2025

  60. [60]

    Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge.CS 231N, 2015. 6

    2015

  61. [61]

    An efficient dataset condensation plugin and its application to continual learning

    Enneng Yang, Li Shen, Zhenyi Wang, Tongliang Liu, and Guibing Guo. An efficient dataset condensation plugin and its application to continual learning. InThirty-seventh Con- ference on Neural Information Processing Systems, 2023. 1

    2023

  62. [62]

    Dataset pruning: Reducing training data by ex- amining generalization influence

    Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, and Ping Li. Dataset pruning: Reducing training data by ex- amining generalization influence. InInternational Conference on Learning Representations, 2023. 5

    2023

  63. [63]

    Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective.Advances in Neural Information Processing Systems, 36, 2024

    Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective.Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  64. [64]

    Dataset condensation via generative model.arXiv preprint arXiv:2309.07698, 2023

    David Junhao Zhang, Heng Wang, Chuhui Xue, Rui Yan, Wenqing Zhang, Song Bai, and Mike Zheng Shou. Dataset condensation via generative model.arXiv preprint arXiv:2309.07698, 2023. 3

    Pith/arXiv arXiv 2023

  65. [65]

    M3d: Dataset condensation by minimizing maximum mean discrepancy

    Hansong Zhang, Shikun Li, Pengju Wang, Dan Zeng, and Shiming Ge. M3d: Dataset condensation by minimizing maximum mean discrepancy. InProceedings of the AAAI Conference on Artificial Intelligence, pages 9314–9322, 2024. 2, 3

    2024

  66. [66]

    Dataset condensation with distri- bution matching.arXiv preprint arXiv:2110.04181, 2021

    Bo Zhao and Hakan Bilen. Dataset condensation with distri- bution matching.arXiv preprint arXiv:2110.04181, 2021. 1, 2, 3, 6, 8

    Pith/arXiv arXiv 2021

  67. [67]

    Dataset condensation with differ- entiable siamese augmentation

    Bo Zhao and Hakan Bilen. Dataset condensation with differ- entiable siamese augmentation. InInternational Conference on Machine Learning, pages 12674–12685. PMLR, 2021. 1, 2, 3

    2021

  68. [68]

    Dataset condensation with gradient matching

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. InInternational Con- ference on Learning Representations, 2020. 1, 2, 3, 6, 7, 8

    2020

  69. [69]

    Im- proved distribution matching for dataset condensation

    Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Im- proved distribution matching for dataset condensation. 2023. 2, 6

    2023

  70. [70]

    Equipping vision foundation model with mixture of experts for out-of-distribution detection

    Shizhen Zhao, Jiahui Liu, Xin Wen, Haoru Tan, and Xiao- juan Qi. Equipping vision foundation model with mixture of experts for out-of-distribution detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 1751–1761, 2025. 7

    2025

  71. [71]

    Dataset distillation using neural feature regression.Advances in Neu- ral Information Processing Systems, 35:9813–9827, 2022

    Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression.Advances in Neu- ral Information Processing Systems, 35:9813–9827, 2022. 3

    2022