A Discrepancy-Based Perspective on Dataset Condensation

Raghavendra Selvan; Tong Chen

REVIEW 2 major objections 5 minor 80 references

Dataset condensation is best understood not as a bag of heuristics but as the problem of choosing a synthetic distribution that minimizes a statistical discrepancy against the original data, and the paper proves that this discrepancy direct

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 17:52 UTC pith:7H6WFTA4

load-bearing objection Useful taxonomy of dataset condensation as discrepancy minimization, but the central proof is flawed and the theory does not actually cover the practical methods it classifies. the 2 major comments →

arxiv 2509.10367 v1 pith:7H6WFTA4 submitted 2025-09-12 cs.LG

A Discrepancy-Based Perspective on Dataset Condensation

Tong Chen , Raghavendra Selvan This is my paper

classification cs.LG

keywords dataset condensationdistribution discrepancyintegral probability metricgeneralization discrepancydistribution matchinggradient matchingdataset distillationcoreset selection

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the many heuristic methods for dataset condensation—building a small synthetic dataset whose trained models match the accuracy of models trained on the full data—are all instances of one formal problem: choose a synthetic distribution that minimizes a discrepancy against the original distribution. It defines the generalization discrepancy as the drop in test performance when training on synthetic rather than real data, and shows this drop is bounded by twice the distribution discrepancy, making discrepancy minimization a principled surrogate for the original bi-level objective. From this it derives a hierarchy of surrogate measures—value discrepancy and parameter discrepancy—and shows how existing condensation methods correspond to choosing one of these discrepancies together with a space (input or latent) in which to match. The payoff is a common vocabulary for comparing methods, an explicit statement of their shared assumptions, and a route to extending condensation beyond accuracy toward robustness and privacy.

Core claim

The central claim is that dataset condensation should be formalized as a bi-level optimization problem in which the outer loop seeks a synthetic distribution µ2 whose optimally trained model performs well on the original distribution µ1. The paper proves GD(µ1, µ2; H) ≤ 2·D(µ1, µ2; H), where GD is the absolute difference between the generalization losses of the two optimal models and D is a model-agnostic distribution discrepancy such as an integral probability metric. It further derives GD ≤ L·VD ≤ LC·PD, so matching learned outputs or parameters is a legitimate surrogate. The paper then organizes a wide range of existing DC methods—distribution matching, gradient matching, trajectory match

What carries the argument

The load-bearing construction is a family of distribution discrepancies over a hypothesis class H. Generalization discrepancy (GD) compares the test loss of the model trained on synthetic data with that of the model trained on real data, both evaluated on the real distribution. The paper bounds GD by 2·D, where D is an integral probability metric or MMD/Wasserstein distance; it then introduces value discrepancy (VD, the sup-norm difference between trained models) and parameter discrepancy (PD, the difference in trained parameters), giving the chain GD ≤ L·VD ≤ LC·PD. The other essential piece is the change-of-space machinery: matching in a latent space is valid only when the encoder is injec

Load-bearing premise

The generalization bound assumes that the hypothesis class H used to define the discrepancy contains both the model trained on the synthetic distribution and the model trained on the original distribution; practical methods replace H with a batch of random or pretrained networks, which generally does not contain the final trained model.

What would settle it

On a fixed binary classification task, take a batch of random feature networks as H, and construct a synthetic set whose empirical MMD (or gradient-matching loss) to the real data is very small but whose freshly trained model loses more than twice that discrepancy in accuracy on the real test set; this would show the bound's practical surrogate fails exactly where the assumption on H is violated.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Minimizing any distribution discrepancy—IPM, MMD, or Wasserstein—between synthetic and real empirical distributions is a surrogate for minimizing the generalization gap, up to the factor-2 bound.
Feature matching, gradient matching, and trajectory matching are not separate ideas; they are the same condensation objective expressed through value or parameter discrepancy.
Condensation in a latent space incurs a fidelity cost: without an injective encoder or perfect reconstruction, the induced discrepancy may no longer be a true metric, so some information loss is unavoidable.
The same discrepancy-minimization formulation handles objectives beyond generalization, such as adversarial robustness (by changing the loss to an adversarial loss) and differential privacy (by adding calibrated noise during matching).

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The bound's assumption that H contains both trained models is typically unmet in practical distribution matching, which samples random or pretrained networks; this suggests the guarantee should be read as a design principle rather than a certificate for a specific trained model.
The taxonomy implies a testable ordering: methods that minimize a task-coupled discrepancy (GD/VD/PD) should control generalization more reliably than methods that minimize a purely distributional discrepancy such as characteristic discrepancy, which carries no generalization guarantee.
A natural benchmark would compare methods by fixing the discrepancy metric and data space, isolating the effect of auxiliary regularization terms (diversity, contrastive, curvature) rather than comparing whole pipelines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful taxonomy of dataset condensation as discrepancy minimization, but the central proof is flawed and the theory does not actually cover the practical methods it classifies.

read the letter

This paper is best read as a survey with a unifying vocabulary: it sorts dataset condensation methods by the discrepancy they minimize (GD, VD, PD, CD, DD) and by the space they match and optimize in. That organizational work is genuinely useful—someone entering the field gets a clear map, and the change-of-space discussion via push-forwards is a nice conceptual frame. The literature coverage is broad and the taxonomy in Table 4 is a reasonable reading of the cited methods. There are no experiments, which is fine for this kind of paper.

The problems are mathematical and load-bearing. The proof of GD ≤ 2D contains an invalid inequality: inside the absolute value, the paper replaces φ(h*_μ2, μ2) − φ(h*_μ1, μ1) with φ(h*_μ1, μ2) − φ(h*_μ1, μ1). Since h*_μ2 minimizes over μ2, the first quantity is ≤ the second, but replacing a term inside an absolute value with a larger term can decrease the absolute value. The inequality chain is simply not valid. Also, the paper states that MMD and the Wasserstein metric are special cases of IPMs, but only the 1-Wasserstein has the Lipschitz dual; W_p for p > 1 is not an IPM in that sense.

The stress-test critique also lands. The theoretical bound assumes H contains both h*_μ1 and h*_μ2, which is fine in the idealized statement. But the practical distribution matching methods in Section 5.1.4 approximate the supremum over H by a small batch of random or pretrained models, H_batch. That batch generally does not contain the model eventually trained on the synthetic data. Minimizing sup over H_batch is minimizing a lower bound on the true D, and the chain |φ(h*_S, μ_T) − φ(h*_S, μ_S)| ≤ D(μ_T, μ_S; H) cannot be invoked when h*_S ∉ H_batch. The paper notes the containment assumption but never squares it with its own description of the practical algorithms. So the claimed principled foundation for these methods is not supported by the bound; the taxonomy remains descriptive.

There are also minor typos and a section on privacy/robustness that is really a list of related work rather than a consequence of the framework.

Who is this for? Researchers who want a one-stop map of dataset condensation methods and a common terminology. Not for someone seeking a new theorem or a rigorous foundation. The paper deserves a serious referee because the taxonomy is useful and the flaws are fixable, but it needs major revision: correct the proof, fix the Wasserstein claim, and be upfront about the gap between the theory and the practical surrogates. If the authors cannot repair the bound, they should reframe the paper as a survey and stop presenting the bound as the theoretical justification for the methods it classifies.

Referee Report

2 major / 5 minor

Summary. The paper proposes a unified discrepancy-based framework for dataset condensation (DC). It introduces several discrepancy notions—generalization discrepancy (GD), value discrepancy (VD), parameter discrepancy (PD), distribution discrepancy (DD), and characteristic discrepancy (CD)—and derives a hierarchy of upper bounds GD ≤ L·VD ≤ LC·PD and GD ≤ 2·D, where D is an IPM-type discrepancy over a hypothesis class H that contains the trained models. The framework is then used to organize a broad taxonomy of existing DC methods according to the discrepancy metric, the space in which matching/optimization is performed, and additional tricks such as augmentation and regularization. The paper also extends the discussion to privacy- and robustness-aware condensation. The central theoretical bound is sound under its stated containment assumption: the displayed inequality chain in Section 3 is justified because the optimality of h*_μ1 and h*_μ2 makes both the base difference and the added term nonnegative.

Significance. If taken as a high-level organizing framework, the paper provides a useful common language for comparing dataset condensation methods and clarifies that many existing heuristics can be viewed as variants of distributional discrepancy minimization. Its strengths are the breadth of covered methods, the explicit formalization of DC as discrepancy minimization, and the extension to multi-objective settings such as privacy and robustness. The paper does not provide new algorithms or empirical validation, which is acceptable for a framework/taxonomy paper. However, the normative claim that existing practical methods inherit the theoretical generalization bound is not established; the practical algorithms optimize a lower-bound approximation of the discrepancy over a small hypothesis class, so the theoretical guarantee does not transfer as stated.

major comments (2)

[§5.1.4 and §3] The generalization surrogate bound GD(μ1, μ2; H) ≤ 2·D(μ1, μ2; H) in Section 3 requires h*_μ1, h*_μ2 ∈ H. In the practical distribution matching, gradient matching, and higher-order moment matching methods described in Section 5.1.4, the supremum over H is replaced by a batch of randomly initialized or pretrained models H_batch. The model actually trained on the synthetic data, h*_S, is generally not in H_batch. Since D(·, ·; H_batch) ≤ D(·, ·; H), minimizing the batch discrepancy need not drive the true discrepancy small, and the chain |φ(h*_S, μ_T) − φ(h*_S, μ_S)| ≤ D(μ_T, μ_S; H) cannot be invoked. The paper should either state explicit conditions on H_batch under which the bound transfers, or soften the claim that these methods are principled instances of the theoretical framework beyond a descriptive taxonomy.
[§2, canonical metrics and Prop. 2.1(3)] The text states that both MMD and the Wasserstein metric are special cases of IPM. This is true for W_1 via the Kantorovich–Rubinstein duality, but false for p > 1: W_p is not generally representable as sup_{h∈H} |E_{μ1} h − E_{μ2} h| for a fixed function class. Proposition 2.1(3) also asserts injectivity for W_p for all p∈[1,∞], which is true as a metric property but does not make W_p an IPM. Since the paper's discrepancy definition and the taxonomy (e.g., the Wasserstein-distance DC method of Liu et al.) only need p = 1, the characterization should be restricted to W_1 or the IPM claim should be reformulated.

minor comments (5)

[§4, first sentence] The text refers to 'Theorem 2.1' but only Proposition 2.1 exists. The reference should be corrected.
[§2, Proposition 2.1 proof] The proof claims that the indicator 1_U of an arbitrary open set U can be uniformly approximated by bounded continuous functions. This is false in general; a continuous function cannot be uniformly close to a discontinuous indicator unless the set is clopen. The injectivity result itself is standard and can be proven correctly, but the proof as written needs revision.
[§5.1.4, notation] In the gradient matching and higher-order moment matching objectives, expressions such as ∇h(T) are used without defining whether ∇h(D) denotes the gradient of the empirical loss averaged over D or something else. Please define this notation precisely.
[§5.1.1, CIG] The displayed definition of the outer loss, '˜L(h*_S, S) := L(h*_S, T)', seems to use the wrong argument (S rather than T). It should likely be a function of S that evaluates the model's loss on T; clarify or correct.
[Table 3] In the last row, the second column is labeled 'Latent space R^n' and the minimization is over Z⊆R^n; this appears to be a typo for R^m, since the other latent-space entries optimize over latent vectors. Please check the row/column labels.

Circularity Check

0 steps flagged

No significant circularity: the central GD≤2D bound is a direct inequality from the paper's definitions; the taxonomy is descriptive and the single self-citation is not load-bearing.

full rationale

The central bound (Section 3) is not circular: GD(μ1, μ2; H) is defined as |φ(h*_μ2, μ1) − φ(h*_μ1, μ1)| and D(μ1, μ2; H) as sup_{h∈H} |φ(h, μ1) − φ(h, μ2)|. The proof adds and subtracts φ(h*_μ2, μ2) and φ(h*_μ1, μ2); each term is then bounded by the supremum, using only the stated containment assumption "assuming H contains both h*_μ1 and h*_μ2". This is a conditional theorem about defined quantities, not an input smuggled in as a prediction. The VD/PD hierarchy likewise follows from Lipschitz assumptions rather than from fitting. The taxonomy (Sections 5.1–5.3, Table 4) is descriptive: existing methods are labeled by which discrepancy they approximate; no fitted constant is later reported as a prediction. The only self-citation, Chen and Selvan (2025) in Table 4, is one listed method and is not used to justify the framework. The flagged limitation — Section 3's containment assumption versus Section 5.1.4's practical replacement of H by "a batch of randomly initialized or pretrained models" — is a real theory–practice gap affecting the transfer of the bound, but it is explicitly acknowledged and does not make the derivation circular, because the theorem does not define D in terms of GD or vice versa. Hence score 0.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 1 invented entities

The framework introduces no fitted numeric parameters. It relies on standard measure-theoretic assumptions, plus specific domain assumptions about the hypothesis class and Lipschitz/compactness conditions for the generalization bound. The only introduced constructs are named discrepancies, which are recombinations of existing quantities.

axioms (5)

domain assumption Each sample in D is drawn i.i.d. from an unknown ground-truth distribution mu, and the empirical distribution approximates mu.
Section 2, Preliminaries: 'We assume that each sample in D is drawn i.i.d. from an unknown ground-truth distribution mu.' This is the standard statistical modeling assumption underlying all distributional claims.
domain assumption The hypothesis class H used to define D contains both h*_mu1 and h*_mu2.
Section 3, bound on generalization discrepancy: 'assuming H contains both h*_mu1 and h*_mu2.' This is load-bearing for the GD <= 2D result and is not satisfied by the random-network approximations used by the surveyed methods.
domain assumption The loss is L-Lipschitz and models are uniformly bounded on a compact input domain.
Section 3, after the redistributed inequality: 'normalized inputs x in [0,1]^n ... uniformly bounded parameters ... L-Lipschitz continuous.' Without this, GD <= L*VD is not justified.
domain assumption Metric preservation of push-forward matching requires injective encoder or surjective decoder.
Section 4: 'this holds precisely when g_e#mu1 = g_e#mu2 implies mu1 = mu2, which is true if and only if g_e is injective' and the decoder case requires surjectivity. Practical autoencoders and GANs violate this, and the paper acknowledges the resulting approximation error.
standard math Universal kernels make MMD injective, and dense function classes make IPMs injective.
Proposition 2.1 invokes the universal approximation property of RKHS and density of C^infty in C(Omega), adapted from Dudley (2002). These are known results, not proven in the paper.

invented entities (1)

Discrepancy terms GD, VD, PD, CD no independent evidence
purpose: Name and organize the different objectives minimized by existing DC methods.
These are definitions that combine existing empirical losses, model outputs, parameters, and characteristic functions. They are useful labels but carry no falsifiable prediction outside the taxonomy.

pith-pipeline@v1.3.0-alltime-deepseek · 27544 in / 17931 out tokens · 197903 ms · 2026-08-04T17:52:10.459125+00:00 · methodology

0 comments

read the original abstract

Given a dataset of finitely many elements $\mathcal{T} = \{\mathbf{x}_i\}_{i = 1}^N$, the goal of dataset condensation (DC) is to construct a synthetic dataset $\mathcal{S} = \{\tilde{\mathbf{x}}_j\}_{j = 1}^M$ which is significantly smaller ($M \ll N$) such that a model trained from scratch on $\mathcal{S}$ achieves comparable or even superior generalization performance to a model trained on $\mathcal{T}$. Recent advances in DC reveal a close connection to the problem of approximating the data distribution represented by $\mathcal{T}$ with a reduced set of points. In this work, we present a unified framework that encompasses existing DC methods and extend the task-specific notion of DC to a more general and formal definition using notions of discrepancy, which quantify the distance between probability distribution in different regimes. Our framework broadens the objective of DC beyond generalization, accommodating additional objectives such as robustness, privacy, and other desirable properties.

Figures

Figures reproduced from arXiv: 2509.10367 by Raghavendra Selvan, Tong Chen.

discussion (0)

Reference graph

Works this paper leans on

80 extracted references · 21 linked inside Pith

[1]

L. F. W. Anthony, B. Kanding, and R. Selvan. Carbontracker: Tracking and predicting the carbon footprint of training deep learning models. arXiv preprint arXiv:2007.03051, 2020

Pith/arXiv arXiv 2007
[2]

Arjovsky, S

M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214--223. PMLR, 2017

2017
[3]

Betker, G

J. Betker, G. Goh, L. Jing, T. Brooks, J. Wang, L. Li, L. Ouyang, J. Zhuang, J. Lee, Y. Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2 0 (3): 0 8, 2023

2023
[4]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

1901
[5]

M. A. Carreira-Perpin \'a n. Model compression as constrained optimization, with application to neural nets. part i: General framework. arXiv preprint arXiv:1707.01209, 2017

Pith/arXiv arXiv 2017
[6]

M. A. Carreira-Perpin \'a n and Y. Idelbayev. Model compression as constrained optimization, with application to neural nets. part ii: Quantization. arXiv preprint arXiv:1707.04319, 2017

Pith/arXiv arXiv 2017
[7]

Cazenavette, T

G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4750--4759, 2022

2022
[8]

Cazenavette, T

G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu. Generalizing dataset distillation via deep generative prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3739--3748, 2023

2023
[9]

D. Chen, R. Kerkouche, and M. Fritz. Private set generation with discriminative information. Advances in Neural Information Processing Systems, 35: 0 14678--14690, 2022

2022
[10]

Chen and R

T. Chen and R. Selvan. Is adversarial training with compressed datasets effective? In Image Analysis, pages 88--103. Springer Nature Switzerland, 2025. ISBN 978-3-031-95911-0

2025
[11]

Chen, Y.-C

W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, and J.-B. Huang. A closer look at few-shot classification. arXiv preprint arXiv:1904.04232, 2019

Pith/arXiv arXiv 1904
[12]

T. Dong, B. Zhao, and L. Lyu. Privacy for free: How does dataset condensation help privacy? In International Conference on Machine Learning, pages 5378--5396. PMLR, 2022

2022
[13]

R. M. Dudley. Real Analysis and Probability. Cambridge Studies in Advanced Mathematics. Cambridge University Press, 2 edition, 2002

2002
[14]

Y. Feng, S. R. Vedantam, and J. Kempe. Embarrassingly simple dataset distillation. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=PLoWVP7Mjc

2024
[15]

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014

2014
[16]

J. Gu, S. Vahidian, V. Kungurtsev, H. Wang, W. Jiang, Y. You, and Y. Chen. Efficient dataset distillation via minimax diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15793--15803, 2024

2024
[17]

Harder, K

F. Harder, K. Adamczewski, and M. Park. Dp-merf: Differentially private mean embeddings with randomfeatures for practical privacy-preserving data generation. In International conference on artificial intelligence and statistics, pages 1819--1827. PMLR, 2021

2021
[18]

Hestness, S

J. Hestness, S. Narang, N. Ardalani, G. Diamos, H. Jun, H. Kianinejad, M. M. A. Patwary, Y. Yang, and Y. Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409, 2017

Pith/arXiv arXiv 2017
[19]

Higgins, L

I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations, 2017

2017
[20]

Hinton, O

G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

Pith/arXiv arXiv 2015
[21]

J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

2020
[22]

Hubara, M

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. Quantized neural networks: Training neural networks with low precision weights and activations. arxiv. arXiv preprint arXiv:1609.07061, 2016

Pith/arXiv arXiv 2016
[23]

Ilyas, S

A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019

2019
[24]

Jaderberg, A

M. Jaderberg, A. Vedaldi, and A. Zisserman. Speeding up convolutional neural networks with low rank expansions. arXiv preprint arXiv:1405.3866, 2014

Pith/arXiv arXiv 2014
[25]

Kaplan, S

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

Pith/arXiv arXiv 2001
[26]

Karras, S

T. Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401--4410, 2019

2019
[27]

Karras, S

T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110--8119, 2020

2020
[28]

Karras, M

T. Karras, M. Aittala, S. Laine, E. H \"a rk \"o nen, J. Hellsten, J. Lehtinen, and T. Aila. Alias-free generative adversarial networks. Advances in neural information processing systems, 34: 0 852--863, 2021

2021
[29]

J.-H. Kim, J. Kim, S. J. Oh, S. Yun, H. Song, J. Jeong, J.-W. Ha, and H. O. Song. Dataset condensation via efficient synthetic-data parameterization. In International Conference on Machine Learning, pages 11102--11118. PMLR, 2022

2022
[30]

D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

Pith/arXiv arXiv 2013
[31]

LeCun, Y

Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. nature, 521 0 (7553): 0 436--444, 2015

2015
[32]

S. Lee, S. Chun, S. Jung, S. Yun, and S. Yoon. Dataset condensation with contrastive signals. In International Conference on Machine Learning, pages 12352--12364. PMLR, 2022

2022
[33]

Li, W.-C

C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang, and B. P \'o czos. Mmd gan: Towards deeper understanding of moment matching network. Advances in neural information processing systems, 30, 2017

2017
[34]

H. Liu, T. Xing, L. Li, V. Dalal, J. He, and H. Wang. Dataset distillation via the wasserstein metric. arXiv preprint arXiv:2311.18531, 2023 a

Pith/arXiv arXiv 2023
[35]

S. Liu, K. Wang, X. Yang, J. Ye, and X. Wang. Dataset distillation via factorization. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=luGXvawYWJ

2022
[36]

Y. Liu, J. Gu, K. Wang, Z. Zhu, W. Jiang, and Y. You. Dream: Efficient dataset distillation by representative matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17314--17324, 2023 b

2023
[37]

Y. Liu, J. Gu, K. Wang, Z. Zhu, K. Zhang, W. Jiang, and Y. You. Dream+: Efficient dataset distillation by bidirectional representative matching. arXiv preprint arXiv:2310.15052, 2023 c

Pith/arXiv arXiv 2023
[38]

N. Loo, R. Hasani, A. Amini, and D. Rus. Efficient dataset distillation using random feature approximation. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=h8Bd7Gm3muB

2022
[39]

N. Loo, R. Hasani, M. Lechner, and D. Rus. Dataset distillation with convexified implicit gradients. In International Conference on Machine Learning, pages 22649--22674. PMLR, 2023

2023
[40]

Z. Ma, D. Gao, S. Yang, X. Wei, and Y. Gong. Dataset condensation via expert subspace projection. Sensors, 23 0 (19): 0 8148, 2023

2023
[41]

Mirza and S

M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014

Pith/arXiv arXiv 2014
[42]

Molchanov, S

P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440, 2016

Pith/arXiv arXiv 2016
[43]

B. B. Moser, F. Raue, S. Palacio, S. Frolov, and A. Dengel. Latent dataset distillation with diffusion models. arXiv preprint arXiv:2403.03881, 2024

arXiv 2024
[44]

Nguyen, Z

T. Nguyen, Z. Chen, and J. Lee. Dataset meta-learning from kernel ridge-regression. In International Conference on Learning Representations, 2021 a . URL https://openreview.net/forum?id=l-PrrQrK0QR

2021
[45]

Nguyen, R

T. Nguyen, R. Novak, L. Xiao, and J. Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems, 34: 0 5186--5198, 2021 b

2021
[46]

Patterson, J

D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, and J. Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350, 2021

Pith/arXiv arXiv 2021
[47]

Puskorius and L

G. Puskorius and L. Feldkamp. Truncated backpropagation through time and kalman filter training for neurocontrol. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94), volume 4, pages 2488--2493. IEEE, 1994

1994
[48]

Rombach, A

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695, 2022

2022
[49]

Saharia, W

C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35: 0 36479--36494, 2022

2022
[50]

Sajedi, S

A. Sajedi, S. Khaki, E. Amjadian, L. Z. Liu, Y. A. Lawryshyn, and K. N. Plataniotis. Datadam: Efficient dataset distillation with attention matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17097--17107, 2023

2023
[51]

Schmidhuber

J. Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61: 0 85--117, 2015

2015
[52]

T. R. Shaham, T. Dekel, and T. Michaeli. Singan: Learning a generative model from a single natural image. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4570--4580, 2019

2019
[53]

S. Shin, H. Bae, D. Shin, W. Joo, and I.-C. Moon. Loss-curvature matching for dataset selection and condensation. In International Conference on Artificial Intelligence and Statistics, pages 8606--8628. PMLR, 2023

2023
[54]

Shokri, M

R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3--18. IEEE, 2017

2017
[55]

K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015

2015
[56]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15 0 (1): 0 1929--1958, 2014

1929
[57]

Strubell, A

E. Strubell, A. Ganesh, and A. McCallum. Energy and policy considerations for modern deep learning research. Proceedings of the AAAI Conference on Artificial Intelligence, 34 0 (09): 0 13693--13696, Apr. 2020. doi:10.1609/aaai.v34i09.7123. URL https://ojs.aaai.org/index.php/AAAI/article/view/7123

doi:10.1609/aaai.v34i09.7123 2020
[58]

D. Su, J. Hou, W. Gao, Y. Tian, and B. Tang. D 4: Dataset distillation via disentangled diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5809--5818, 2024

2024
[59]

Tsilivis, J

N. Tsilivis, J. Su, and J. Kempe. Can we achieve robustness from data alone? arXiv preprint arXiv:2207.11727, 2022

Pith/arXiv arXiv 2022
[60]

Van Den Oord, O

A. Van Den Oord, O. Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

2017
[61]

K. Wang, B. Zhao, X. Peng, Z. Zhu, S. Yang, S. Wang, G. Huang, H. Bilen, X. Wang, and Y. You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12196--12205, 2022

2022
[62]

K. Wang, J. Gu, H. Zhang, D. Zhou, Z. Zhu, W. Jiang, and Y. You. Dim: Distilling dataset into generative model. In European Conference on Computer Vision, pages 42--59. Springer, 2025

2025
[63]

Wang, J.-Y

T. Wang, J.-Y. Zhu, A. Torralba, and A. A. Efros. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018

Pith/arXiv arXiv 2018
[64]

R. J. Williams and J. Peng. An efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural computation, 2 0 (4): 0 490--501, 1990

1990
[65]

Y. Wu, X. Li, F. Kerschbaum, H. Huang, and H. Zhang. Towards robust dataset learning. arXiv preprint arXiv:2211.10752, 2022

Pith/arXiv arXiv 2022
[66]

E. Xue, Y. Li, H. Liu, P. Wang, Y. Shen, and H. Wang. Towards adversarially robust dataset distillation by curvature regularization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 9041--9049, 2025

2025
[67]

S. Yang, S. Cheng, M. Hong, H. Fan, X. Wei, and S. Liu. Neural spectral decomposition for dataset distillation. In European Conference on Computer Vision, pages 275--290. Springer, 2024

2024
[68]

R. Yu, S. Liu, J. Ye, and X. Wang. Teddy: Efficient large-scale dataset distillation via taylor-approximated matching. In European Conference on Computer Vision, pages 1--17. Springer, 2024

2024
[69]

Zhang, S

C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530, 2016

Pith/arXiv arXiv 2016
[70]

D. J. Zhang, H. Wang, C. Xue, R. Yan, W. Zhang, S. Bai, and M. Z. Shou. Dataset condensation via generative model. arXiv preprint arXiv:2309.07698, 2023

Pith/arXiv arXiv 2023
[71]

Zhang, S

H. Zhang, S. Li, P. Wang, D. Zeng, and S. Ge. M3d: Dataset condensation by minimizing maximum mean discrepancy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 9314--9322, 2024

2024
[72]

Zhao and H

B. Zhao and H. Bilen. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pages 12674--12685. PMLR, 2021

2021
[73]

Zhao and H

B. Zhao and H. Bilen. Synthesizing informative training samples with GAN . In NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research, 2022. URL https://openreview.net/forum?id=frAv0jtUMfS

2022
[74]

Zhao and H

B. Zhao and H. Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6514--6523, 2023

2023
[75]

B. Zhao, K. R. Mopuri, and H. Bilen. Dataset condensation with gradient matching. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=mSAKhLYLSsl

2021
[76]

G. Zhao, G. Li, Y. Qin, and Y. Yu. Improved distribution matching for dataset condensation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7856--7865, 2023

2023
[77]

S. Zhao, J. Song, and S. Ermon. Infovae: Information maximizing variational autoencoders. arXiv preprint arXiv:1706.02262, 2017

Pith/arXiv arXiv 2017
[78]

W. Zhou, G. Zheng, and X. Ding. Dataset distillation with channel efficient process. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5505--5509. IEEE, 2024

2024
[79]

Y. Zhou, E. Nezhadarya, and J. Ba. Dataset distillation using neural feature regression. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=2clwrA2tfik

2022
[80]

J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223--2232, 2017

2017

[1] [1]

L. F. W. Anthony, B. Kanding, and R. Selvan. Carbontracker: Tracking and predicting the carbon footprint of training deep learning models. arXiv preprint arXiv:2007.03051, 2020

Pith/arXiv arXiv 2007

[2] [2]

Arjovsky, S

M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214--223. PMLR, 2017

2017

[3] [3]

Betker, G

J. Betker, G. Goh, L. Jing, T. Brooks, J. Wang, L. Li, L. Ouyang, J. Zhuang, J. Lee, Y. Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2 0 (3): 0 8, 2023

2023

[4] [4]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

1901

[5] [5]

M. A. Carreira-Perpin \'a n. Model compression as constrained optimization, with application to neural nets. part i: General framework. arXiv preprint arXiv:1707.01209, 2017

Pith/arXiv arXiv 2017

[6] [6]

M. A. Carreira-Perpin \'a n and Y. Idelbayev. Model compression as constrained optimization, with application to neural nets. part ii: Quantization. arXiv preprint arXiv:1707.04319, 2017

Pith/arXiv arXiv 2017

[7] [7]

Cazenavette, T

G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4750--4759, 2022

2022

[8] [8]

Cazenavette, T

G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu. Generalizing dataset distillation via deep generative prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3739--3748, 2023

2023

[9] [9]

D. Chen, R. Kerkouche, and M. Fritz. Private set generation with discriminative information. Advances in Neural Information Processing Systems, 35: 0 14678--14690, 2022

2022

[10] [10]

Chen and R

T. Chen and R. Selvan. Is adversarial training with compressed datasets effective? In Image Analysis, pages 88--103. Springer Nature Switzerland, 2025. ISBN 978-3-031-95911-0

2025

[11] [11]

Chen, Y.-C

W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, and J.-B. Huang. A closer look at few-shot classification. arXiv preprint arXiv:1904.04232, 2019

Pith/arXiv arXiv 1904

[12] [12]

T. Dong, B. Zhao, and L. Lyu. Privacy for free: How does dataset condensation help privacy? In International Conference on Machine Learning, pages 5378--5396. PMLR, 2022

2022

[13] [13]

R. M. Dudley. Real Analysis and Probability. Cambridge Studies in Advanced Mathematics. Cambridge University Press, 2 edition, 2002

2002

[14] [14]

Y. Feng, S. R. Vedantam, and J. Kempe. Embarrassingly simple dataset distillation. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=PLoWVP7Mjc

2024

[15] [15]

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014

2014

[16] [16]

J. Gu, S. Vahidian, V. Kungurtsev, H. Wang, W. Jiang, Y. You, and Y. Chen. Efficient dataset distillation via minimax diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15793--15803, 2024

2024

[17] [17]

Harder, K

F. Harder, K. Adamczewski, and M. Park. Dp-merf: Differentially private mean embeddings with randomfeatures for practical privacy-preserving data generation. In International conference on artificial intelligence and statistics, pages 1819--1827. PMLR, 2021

2021

[18] [18]

Hestness, S

J. Hestness, S. Narang, N. Ardalani, G. Diamos, H. Jun, H. Kianinejad, M. M. A. Patwary, Y. Yang, and Y. Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409, 2017

Pith/arXiv arXiv 2017

[19] [19]

Higgins, L

I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations, 2017

2017

[20] [20]

Hinton, O

G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

Pith/arXiv arXiv 2015

[21] [21]

J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

2020

[22] [22]

Hubara, M

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. Quantized neural networks: Training neural networks with low precision weights and activations. arxiv. arXiv preprint arXiv:1609.07061, 2016

Pith/arXiv arXiv 2016

[23] [23]

Ilyas, S

A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019

2019

[24] [24]

Jaderberg, A

M. Jaderberg, A. Vedaldi, and A. Zisserman. Speeding up convolutional neural networks with low rank expansions. arXiv preprint arXiv:1405.3866, 2014

Pith/arXiv arXiv 2014

[25] [25]

Kaplan, S

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

Pith/arXiv arXiv 2001

[26] [26]

Karras, S

T. Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401--4410, 2019

2019

[27] [27]

Karras, S

T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110--8119, 2020

2020

[28] [28]

Karras, M

T. Karras, M. Aittala, S. Laine, E. H \"a rk \"o nen, J. Hellsten, J. Lehtinen, and T. Aila. Alias-free generative adversarial networks. Advances in neural information processing systems, 34: 0 852--863, 2021

2021

[29] [29]

J.-H. Kim, J. Kim, S. J. Oh, S. Yun, H. Song, J. Jeong, J.-W. Ha, and H. O. Song. Dataset condensation via efficient synthetic-data parameterization. In International Conference on Machine Learning, pages 11102--11118. PMLR, 2022

2022

[30] [30]

D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

Pith/arXiv arXiv 2013

[31] [31]

LeCun, Y

Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. nature, 521 0 (7553): 0 436--444, 2015

2015

[32] [32]

S. Lee, S. Chun, S. Jung, S. Yun, and S. Yoon. Dataset condensation with contrastive signals. In International Conference on Machine Learning, pages 12352--12364. PMLR, 2022

2022

[33] [33]

Li, W.-C

C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang, and B. P \'o czos. Mmd gan: Towards deeper understanding of moment matching network. Advances in neural information processing systems, 30, 2017

2017

[34] [34]

H. Liu, T. Xing, L. Li, V. Dalal, J. He, and H. Wang. Dataset distillation via the wasserstein metric. arXiv preprint arXiv:2311.18531, 2023 a

Pith/arXiv arXiv 2023

[35] [35]

S. Liu, K. Wang, X. Yang, J. Ye, and X. Wang. Dataset distillation via factorization. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=luGXvawYWJ

2022

[36] [36]

Y. Liu, J. Gu, K. Wang, Z. Zhu, W. Jiang, and Y. You. Dream: Efficient dataset distillation by representative matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17314--17324, 2023 b

2023

[37] [37]

Y. Liu, J. Gu, K. Wang, Z. Zhu, K. Zhang, W. Jiang, and Y. You. Dream+: Efficient dataset distillation by bidirectional representative matching. arXiv preprint arXiv:2310.15052, 2023 c

Pith/arXiv arXiv 2023

[38] [38]

N. Loo, R. Hasani, A. Amini, and D. Rus. Efficient dataset distillation using random feature approximation. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=h8Bd7Gm3muB

2022

[39] [39]

N. Loo, R. Hasani, M. Lechner, and D. Rus. Dataset distillation with convexified implicit gradients. In International Conference on Machine Learning, pages 22649--22674. PMLR, 2023

2023

[40] [40]

Z. Ma, D. Gao, S. Yang, X. Wei, and Y. Gong. Dataset condensation via expert subspace projection. Sensors, 23 0 (19): 0 8148, 2023

2023

[41] [41]

Mirza and S

M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014

Pith/arXiv arXiv 2014

[42] [42]

Molchanov, S

P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440, 2016

Pith/arXiv arXiv 2016

[43] [43]

B. B. Moser, F. Raue, S. Palacio, S. Frolov, and A. Dengel. Latent dataset distillation with diffusion models. arXiv preprint arXiv:2403.03881, 2024

arXiv 2024

[44] [44]

Nguyen, Z

T. Nguyen, Z. Chen, and J. Lee. Dataset meta-learning from kernel ridge-regression. In International Conference on Learning Representations, 2021 a . URL https://openreview.net/forum?id=l-PrrQrK0QR

2021

[45] [45]

Nguyen, R

T. Nguyen, R. Novak, L. Xiao, and J. Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems, 34: 0 5186--5198, 2021 b

2021

[46] [46]

Patterson, J

D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, and J. Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350, 2021

Pith/arXiv arXiv 2021

[47] [47]

Puskorius and L

G. Puskorius and L. Feldkamp. Truncated backpropagation through time and kalman filter training for neurocontrol. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94), volume 4, pages 2488--2493. IEEE, 1994

1994

[48] [48]

Rombach, A

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695, 2022

2022

[49] [49]

Saharia, W

C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35: 0 36479--36494, 2022

2022

[50] [50]

Sajedi, S

A. Sajedi, S. Khaki, E. Amjadian, L. Z. Liu, Y. A. Lawryshyn, and K. N. Plataniotis. Datadam: Efficient dataset distillation with attention matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17097--17107, 2023

2023

[51] [51]

Schmidhuber

J. Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61: 0 85--117, 2015

2015

[52] [52]

T. R. Shaham, T. Dekel, and T. Michaeli. Singan: Learning a generative model from a single natural image. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4570--4580, 2019

2019

[53] [53]

S. Shin, H. Bae, D. Shin, W. Joo, and I.-C. Moon. Loss-curvature matching for dataset selection and condensation. In International Conference on Artificial Intelligence and Statistics, pages 8606--8628. PMLR, 2023

2023

[54] [54]

Shokri, M

R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3--18. IEEE, 2017

2017

[55] [55]

K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015

2015

[56] [56]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15 0 (1): 0 1929--1958, 2014

1929

[57] [57]

Strubell, A

E. Strubell, A. Ganesh, and A. McCallum. Energy and policy considerations for modern deep learning research. Proceedings of the AAAI Conference on Artificial Intelligence, 34 0 (09): 0 13693--13696, Apr. 2020. doi:10.1609/aaai.v34i09.7123. URL https://ojs.aaai.org/index.php/AAAI/article/view/7123

doi:10.1609/aaai.v34i09.7123 2020

[58] [58]

D. Su, J. Hou, W. Gao, Y. Tian, and B. Tang. D 4: Dataset distillation via disentangled diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5809--5818, 2024

2024

[59] [59]

Tsilivis, J

N. Tsilivis, J. Su, and J. Kempe. Can we achieve robustness from data alone? arXiv preprint arXiv:2207.11727, 2022

Pith/arXiv arXiv 2022

[60] [60]

Van Den Oord, O

A. Van Den Oord, O. Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

2017

[61] [61]

K. Wang, B. Zhao, X. Peng, Z. Zhu, S. Yang, S. Wang, G. Huang, H. Bilen, X. Wang, and Y. You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12196--12205, 2022

2022

[62] [62]

K. Wang, J. Gu, H. Zhang, D. Zhou, Z. Zhu, W. Jiang, and Y. You. Dim: Distilling dataset into generative model. In European Conference on Computer Vision, pages 42--59. Springer, 2025

2025

[63] [63]

Wang, J.-Y

T. Wang, J.-Y. Zhu, A. Torralba, and A. A. Efros. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018

Pith/arXiv arXiv 2018

[64] [64]

R. J. Williams and J. Peng. An efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural computation, 2 0 (4): 0 490--501, 1990

1990

[65] [65]

Y. Wu, X. Li, F. Kerschbaum, H. Huang, and H. Zhang. Towards robust dataset learning. arXiv preprint arXiv:2211.10752, 2022

Pith/arXiv arXiv 2022

[66] [66]

E. Xue, Y. Li, H. Liu, P. Wang, Y. Shen, and H. Wang. Towards adversarially robust dataset distillation by curvature regularization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 9041--9049, 2025

2025

[67] [67]

S. Yang, S. Cheng, M. Hong, H. Fan, X. Wei, and S. Liu. Neural spectral decomposition for dataset distillation. In European Conference on Computer Vision, pages 275--290. Springer, 2024

2024

[68] [68]

R. Yu, S. Liu, J. Ye, and X. Wang. Teddy: Efficient large-scale dataset distillation via taylor-approximated matching. In European Conference on Computer Vision, pages 1--17. Springer, 2024

2024

[69] [69]

Zhang, S

C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530, 2016

Pith/arXiv arXiv 2016

[70] [70]

D. J. Zhang, H. Wang, C. Xue, R. Yan, W. Zhang, S. Bai, and M. Z. Shou. Dataset condensation via generative model. arXiv preprint arXiv:2309.07698, 2023

Pith/arXiv arXiv 2023

[71] [71]

Zhang, S

H. Zhang, S. Li, P. Wang, D. Zeng, and S. Ge. M3d: Dataset condensation by minimizing maximum mean discrepancy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 9314--9322, 2024

2024

[72] [72]

Zhao and H

B. Zhao and H. Bilen. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pages 12674--12685. PMLR, 2021

2021

[73] [73]

Zhao and H

B. Zhao and H. Bilen. Synthesizing informative training samples with GAN . In NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research, 2022. URL https://openreview.net/forum?id=frAv0jtUMfS

2022

[74] [74]

Zhao and H

B. Zhao and H. Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6514--6523, 2023

2023

[75] [75]

B. Zhao, K. R. Mopuri, and H. Bilen. Dataset condensation with gradient matching. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=mSAKhLYLSsl

2021

[76] [76]

G. Zhao, G. Li, Y. Qin, and Y. Yu. Improved distribution matching for dataset condensation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7856--7865, 2023

2023

[77] [77]

S. Zhao, J. Song, and S. Ermon. Infovae: Information maximizing variational autoencoders. arXiv preprint arXiv:1706.02262, 2017

Pith/arXiv arXiv 2017

[78] [78]

W. Zhou, G. Zheng, and X. Ding. Dataset distillation with channel efficient process. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5505--5509. IEEE, 2024

2024

[79] [79]

Y. Zhou, E. Nezhadarya, and J. Ba. Dataset distillation using neural feature regression. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=2clwrA2tfik

2022

[80] [80]

J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223--2232, 2017

2017