Pith. sign in

REVIEW 3 major objections 3 minor 121 references

Self-Supervised Representation-Guided Generative Dataset Distillation

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes SRG, a framework that steers diffusion sampling with class-wise self-supervised prototypes, and claims it outperforms generative dataset-distillation baselines while scaling to ImageNet-1K.

desk verdict Solid empirical paper with real contribution; main caveat is the unguided-DiT control missing from the cross-encoder table, plus no code release. read the letter →

arxiv 2608.03218 v1 pith:KZHVTRNL submitted 2026-08-04 cs.CV cs.AI

Mingzhuo Li , Guang Li , Linfeng Ye , Jiafeng Mao , Takahiro Ogawa , Konstantinos N. Plataniotis , Miki Haseyama This is my paper
classification cs.CVcs.AI
keywords datasetdistillationgenerativediffusionguidanceself-supervisedlearningfrozenpretrainedencodersprototypealignmentlinearprobingImageNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dataset distillation usually assumes a model trained from random weights, but modern practice often freezes a self-supervised encoder and trains only a lightweight head. This paper tries to establish that a distilled set for that setting should preserve the discriminative structure of the encoder's representation space, and that a diffusion generator can be steered to do so. SRG clusters real images into class-wise prototypes in the frozen encoder's space, assigns each denoising trajectory a prototype, and guides early sampling with a real-image latent anchor and later sampling with three representation-space objectives. The reported results show consistent gains over generative baselines across datasets and IPC budgets, transfer across different SSL encoders, and scalable generation for ImageNet-1K. A sympathetic reader would therefore take the paper's contribution to be a recipe for turning any frozen encoder's geometry into diffusion guidance.

What carries the argument

The load-bearing object is the class-wise SSL prototype set $\{p_{c,k}\}$ obtained by spherical K-means in the normalized representation space of a frozen encoder, together with the three SSL-space objectives (prototype alignment, inter-class discrimination, intra-class assignment) whose gradients are backpropagated to the predicted clean latent. The stage-wise schedule $g^i_{SRG} = \mathbb{I}[t_i \ge t_{\mathrm{lat}}] g^i_{\mathrm{lat}} + \mathbb{I}[t_i < t_{\mathrm{SSL}}] g^i_{SSL}$ is the mechanism that combines the two signals: early latent anchoring to a prototype-associated real image preserves visual realism, while later SSL-space guidance refines class-discriminative structure.

What would settle it

Measure, at late denoising steps, the cosine similarity between the proposed SSL-space guidance gradient $\nabla_{\hat z_0^i} L_{\mathrm{SSL}}$ and the actual change in the decoded SSL representation after the guided step; if this correlation is near zero across seeds and classes, the guidance is not steering by representation geometry. Alternatively, replace the assigned prototype with a random point inside the same class cluster: if downstream accuracy does not drop, the prototype is not carrying the claimed information.

Watch

Extended reading notes

Core claim

The central claim is that the discriminative geometry of a frozen SSL encoder can be used as a diffusion guidance signal for generative dataset distillation. SRG computes class-wise prototypes via spherical K-means on $\ell^2$-normalized SSL features, assigns each generation trajectory to one prototype, and at each denoising step maps the predicted clean latent through the VAE decoder and the frozen encoder to obtain $r_i = \mathrm{norm}_2(f_\phi(D_\psi(\hat z_0^i)))$. Three objectives act on this representation: prototype alignment $L_{\mathrm{proto}} = 1 - \mathrm{sim}(r_i, p_{c,k})$, inter-class discrimination $L_{\mathrm{inter}}$ via log-sum-exp over other-class prototypes, and intra-cla

Load-bearing premise

The load-bearing premise is that a gradient of the SSL-space objectives, pushed through the VAE decoder and the frozen SSL encoder, is a reliable direction to add to the diffusion latent; if that mapping becomes uninformative at the relevant noise levels, the central steering mechanism loses its grounding.

Editorial extensions

If this is right

  • If SRG's central claim holds, generative dataset distillation for frozen SSL encoders no longer needs per-dataset, per-IPC optimization: one pretrained diffusion model and one frozen encoder produce distilled sets at any IPC budget.
  • The ablation shows the two guidance signals are complementary: latent anchoring alone gives a small gain, SSL-space guidance gives a large one, and the combination is best.
  • Cross-encoder results imply that a dataset distilled for one SSL representation space remains useful for other encoders, with the best accuracy when distillation and evaluation encoders match.
  • Because SRG outperforms DiT and MGD3 on ImageNet-R, representation-guided sampling preserves semantic structure that survives style shifts, not just source-domain appearance.
  • SRG's moderate sampling overhead and one-time prototype construction make it a practical alternative to optimization-based methods at higher IPC.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the prototype-and-guidance recipe suggests a testable extension: applying the same SSL-space objectives to other generative samplers, such as latent diffusion models with non-ImageNet generators or conditional GANs, and checking whether final-sample SSL coverage improves as claimed.
  • The stage-wise assumption implies a measurable diagnostic that the paper does not report: at late denoising steps, the gradient direction $\nabla_{\hat z_0^i} L_{\mathrm{SSL}}$ should correlate with the actual movement of the decoded SSL representation after the guided step.
  • The cross-encoder transfer results hint that SRG-generated sets could serve as a shared, encoder-agnostic distilled benchmark, though the paper does not claim this.
  • The paper's own limitation of relying on an ImageNet-pretrained generator invites a concrete next step: once domain-matched generators exist, the same representation-guided objectives should transfer to non-ImageNet domains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes SRG, a generative dataset-distillation method for downstream linear probing on frozen self-supervised (SSL) encoders. SRG constructs class-wise prototypes from real-image SSL representations, assigns each diffusion trajectory to a prototype, and guides DiT sampling with two stage-wise signals: early latent-space anchoring to a real image whose representation is nearest the prototype, and later SSL-space guidance using prototype alignment, inter-class discrimination, and intra-class assignment (Eqs. 6-12). Experiments on ImageNet-IDC, ImageNet-100, ImageNet-1K, fine-grained subsets, and ImageNet-R report consistent gains over generative baselines (DiT, MGD3, IGD) at IPC=1,3,5, with ablations and sensitivity analyses in the main text and supplementary material.

Significance. If the claims hold, SRG is a useful contribution to dataset distillation for the increasingly common setting of frozen pretrained encoders: it is generative, scales to ImageNet-1K, and adds only moderate runtime over unguided DiT (Table 5). The paper's strengths are its breadth of experiments, standard-deviation error bars over five linear-probe runs, clean ablations (Table 4), and extensive hyperparameter sensitivity tables in the supplement. The same-encoder circularity concern is partially mitigated by cross-encoder Table 3 and by the Neighbor baseline in Table 1. However, the headline cross-encoder transfer claim is not established because Table 3 omits the unguided DiT / selection baselines for each evaluation encoder.

major comments (3)
  1. [Abstract/Conclusion and §Experiments, Table 3] The claim that 'cross-encoder evaluation further indicates transfer across pretrained representation spaces' lacks a necessary control. From Table 1, unguided DiT on ImageNet-100 at IPC=5 evaluated with DINOv2 obtains 89.0. In Table 3, SRG distilled with CLIP and evaluated with DINOv2 obtains only 85.0, and the average of the CLIP/EVA-02/MoCov3-distilled sets evaluated on DINOv2 is (85.0+90.6+90.8)/3 = 88.8, slightly below unguided DiT. Thus, for at least one encoder pair, SRG's guidance is harmful relative to no guidance, and the off-diagonal utility may reflect the ImageNet-pretrained DiT prior rather than SRG. Add unguided DiT (and, ideally, Random/Neighbor) rows or columns for each evaluation encoder across Tables 3 and 6, or temper the cross-encoder claim accordingly.
  2. [§Method, Eqs. (10)-(12)] The core guidance mechanism is an additive heuristic without a score-based or optimal-transport justification. The gradient of the SSL objective is taken w.r.t. the predicted clean latent z0_hat, but it is added to the next noisy latent v_{i+1}. This is not the same as updating the predicted clean latent, and its validity depends on an unanalyzed Jacobian of the VAE decoder and SSL encoder plus the noise level at timestep t_i. Sensitivity analyses (Fig. 5, Tables 7-10) de-risk hyperparameters but do not validate that this gradient direction is reliable. Please provide a derivation or a diagnostic experiment (e.g., compare adding the gradient to v_{i+1} vs. to z0_hat, or measure whether the gradient decreases the SSL objective of the final decoded sample) to justify the mechanism.
  3. [§Experiments, Table 1 and footnote] The ImageNet-1K comparisons against LGM and CLP-DD rely on results reported in the original papers for settings that may not match the authors' evaluation protocol (e.g., augmentation, linear-probe epochs, feature resolution). Since Table 1 is the main evidence for ImageNet-1K scaling, the text should state explicitly which protocol the starred values come from and whether they are directly comparable to the other rows. This is a comparability issue rather than a fatal flaw, but it affects the strength of the scaling claim.
minor comments (3)
  1. [Algorithm 1, line 19] The pseudocode says 'Compute g^i_lat with target m_{c,k} with Eq. (2)', but the main text (Eq. 11 discussion) replaces m_{c,k} with a_{c,k}, the latent of the real image nearest the prototype. Update the pseudocode to avoid inconsistency.
  2. [Table 1 / Experiment Settings] The acronym 'ImageNet-IDC' is used without definition. Define it at first use (the dataset composition appears only in the supplement).
  3. [§Method, Eq. (9) and §Supplementary, Table 10] The main text says 'we use the same weights for all three objectives,' but the supplement introduces weights w_inter and w_intra. Clarify in the main text that the default is unit weights and that those weights are not tuned in the main results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the same-encoder evaluation is a deliberate design alignment, not a by-construction fit, and the missing DiT control in Table 3 is a validity concern, not circularity.

full rationale

The derivation chain is: extract frozen SSL representations of real training images, compute class-wise prototypes by spherical k-means, map the diffusion-predicted clean latent through the VAE decoder and the same SSL encoder, define three differentiable objectives (Eqs. 6-8), backpropagate their sum to the predicted clean latent (Eq. 10), add the result to the reverse transition (Eqs. 11-12), and finally evaluate the distilled set by linear probing. None of these steps defines the evaluation metric in terms of the guidance objective, and no parameter is fitted to the downstream test labels. The same-encoder evaluation in Table 1 means the guidance space and the probe space coincide, but that is an experimental design choice rather than a reduction: the ablation in Table 4 shows that removing SSL-space guidance lowers accuracy, and the sensitivity analyses in Tables 7-10 show that performance varies with the guidance scales and temperatures, so the reported gains are not forced by construction. Cross-encoder results (Table 3) and the ImageNet-R experiment (Table 11) provide checks using representation spaces or domains not used during guidance. The self-citations in the paper, including CLP-DD (Peng et al. 2026), are baselines or related-work context and are not load-bearing for SRG's mechanism; there is no uniqueness theorem or ansatz smuggled in from the authors' prior work. The skeptic's point that Table 3 omits an unguided DiT row is a legitimate missing-control concern about the cross-encoder transfer claim, but omitting a baseline does not make the result equivalent to its inputs by construction, so it does not constitute circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The ledger's main items are the two load-bearing modeling assumptions: that prototype coverage is sufficient, and that the gradient-as-guidance update is a valid steering mechanism. The SSL-space guidance hyperparameters are chosen on one benchmark and then fixed; there are no invented physical entities.

free parameters (6)
  • lambda_lat = 0.1
    Latent-space guidance strength; selected by joint sensitivity analysis on ImageNet-100 IPC=5 (Table 9) and fixed across all datasets and IPC settings.
  • lambda_SSL = 15
    SSL-space guidance strength; selected by sensitivity analysis on ImageNet-100 IPC=5 (Table 9).
  • t_lat = 20
    Denoising timestep where latent-space guidance stops; set jointly with t_SSL, balancing performance and compute (Table 7).
  • t_SSL = 20
    Denoising timestep where SSL-space guidance starts; sensitivity analysis shows final-steps-only guidance already helps (Table 7).
  • tau_inter = 1.0
    Temperature for the inter-class log-sum-exp objective; default chosen from Table 8.
  • tau_intra = 5.0
    Temperature for the intra-class assignment cross-entropy objective; default chosen from Table 8.
assumptions (5)
  • domain assumption The pretrained DiT/VAE pair provides a class-conditional image manifold of sufficient fidelity for the distilled samples to be useful.
    All generated samples are constrained to DiT's output; if the generator is weak or domain-mismatched, the method inherits that weakness, as the conclusion concedes.
  • domain assumption The frozen SSL encoder's class-wise spherical K-means prototypes faithfully summarize the discriminative regions needed for downstream linear probe accuracy.
    The method builds prototypes and steers all samples toward them; if prototype coverage misses useful regions, the distilled set degrades. Invoked in Eq. (4) and used throughout generation.
  • ad hoc to paper Backpropagating the SSL-space objectives through the VAE decoder and frozen encoder into the predicted clean latent, then adding that gradient to the next latent, is a valid way to guide diffusion sampling.
    This is the method's core update (Eqs. 10-12). It follows MGD3's heuristic but is not derived from a score-based guidance framework.
  • domain assumption Early denoising latents are too noisy for SSL comparison while later latents are semantically discriminative, justifying the stage-wise schedule.
    Eq. 11 and Section 'Overall Generation'; supported by the cited Revelio result and by sensitivity analysis, but it is a modeling choice.
  • domain assumption Linear-probe accuracy on the frozen SSL features is the right downstream utility for the distilled dataset.
    Evaluation protocol; reasonable for the frozen-encoder setting but excludes full fine-tuning or other adaptations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Supervised Representation-Guided Generative Dataset Distillation." pith.science (2026). https://pith.science/paper/KZHVTRNL

@misc{pith2026260803218,
  author       = {Pith},
  title        = {Pith review of: Self-Supervised Representation-Guided Generative Dataset Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KZHVTRNL}},
  note         = {Machine review of arXiv:2608.03218}
}
read the original abstract

Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target randomly initialized networks, whereas modern vision systems often adapt frozen pretrained encoders with lightweight modules. Distilled samples should therefore preserve the discriminative geometry of the pretrained representation space, which existing generative objectives do not explicitly consider. We propose self-supervised representation-guided generative dataset distillation (SRG), a framework that translates the SSL geometry into diffusion guidance. Specifically, SRG constructs class-wise prototypes from real-image SSL representations and performs guidance through three SSL-space objectives for prototype alignment, inter-class discrimination, and intra-class assignment. During diffusion sampling, it adopts a stage-wise guidance strategy: early denoising is anchored to the latent of the real image whose SSL representation is nearest to the assigned prototype, whereas later denoising is guided by the SSL-space objectives. This division preserves the visual realism provided by the generative prior while progressively steering samples toward representative and class-discriminative regions of the SSL representation space. SRG consistently outperforms the evaluated generative baselines across multiple datasets and IPC settings. A cross-encoder evaluation further indicates transfer across pretrained representation spaces. These results demonstrate the effectiveness of representation-guided generation for dataset distillation with pretrained SSL models.

Figures

Figures reproduced from arXiv: 2608.03218 by the authors.

Figure 1
Figure 1. Overview of dataset distillation and downstream [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of SRG. Class-wise prototypes are constructed by clustering real-image representations extracted by a frozen [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. UMAP visualization of real samples, class proto [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Sensitivity analysis of the SSL-space guidance [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Evolution of prototype distances during denoising. The first row shows the complete trajectories. The second and third [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Samples generated by DiT, MGD3, and SRG with different SSL encoders for two representative classes at IPC [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Comparison of prototype neighbors, SRG-generated samples, and sample neighbors in the DINOv2 representation [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Comparison of samples generated with standard SRG guidance, mismatched prototype neighbors, and mismatched [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

121 extracted references · 56 canonical work pages

  1. [1]

    Proceedings of the International Conference on Machine Learning , pages=

    Dataset condensation with Differentiable Siamese Augmentation , author=. Proceedings of the International Conference on Machine Learning , pages=

  2. [2]

    The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization , year=

    Hendrycks, Dan and Basart, Steven and Mu, Norman and Kadavath, Saurav and Wang, Frank and Dorundo, Evan and Desai, Rahul and Zhu, Tyler and Parajuli, Samyak and Guo, Mike and Song, Dawn and Steinhardt, Jacob and Gilmer, Justin , booktitle=. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization , year=

  3. [3]

    Proceedings of the Advances in Neural Information Processing Systems , year=

    Dataset Distillation for Pre-Trained Self-Supervised Vision Models , author=. Proceedings of the Advances in Neural Information Processing Systems , year=

  4. [4]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =

    Jing, Longlong and Tian, Yingli , title =. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. 2021 , doi =

    2021

  5. [5]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =

    Gui, Jie and Chen, Tuo and Zhang, Jing and Cao, Qiong and Sun, Zhenan and Luo, Hao and Tao, Dacheng , title =. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. 2024 , doi =

    2024

  6. [6]

    Technologies , volume =

    Jaiswal, Ashish and Babu, Ashwin Ramesh and Zadeh, Mohammad Zaki and Banerjee, Debapriya and Makedon, Fillia , title =. Technologies , volume =. 2021 , doi =

    2021

  7. [7]

    Proceedings of the 37th International Conference on Machine Learning , series =

    Chen, Ting and Kornblith, Simon and Norouzi, Mohammad and Hinton, Geoffrey , title =. Proceedings of the 37th International Conference on Machine Learning , series =. 2020 , publisher =

    2020

  8. [8]

    Proceedings of the 38th International Conference on Machine Learning , series =

    Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and Krueger, Gretchen and Sutskever, Ilya , title =. Proceedings of the 38th International Conference on Machine Learning , series =. 2021 , publisher =

    2021

Show all 121 references
  1. [9]

    Emerging Properties in Self-Supervised Vision Transformers , booktitle =

    Caron, Mathilde and Touvron, Hugo and Misra, Ishan and J. Emerging Properties in Self-Supervised Vision Transformers , booktitle =

  2. [10]

    Transactions on Machine Learning Research , year =

    Oquab, Maxime and Darcet, Timoth. Transactions on Machine Learning Research , year =

  3. [11]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    He, Kaiming and Fan, Haoqi and Wu, Yuxin and Xie, Saining and Girshick, Ross , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  4. [12]

    Proceedings of the European Conference on Computer Vision , year=

    Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation , author=. Proceedings of the European Conference on Computer Vision , year=

  5. [13]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages =

    Chen, Xinlei and Xie, Saining and He, Kaiming , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages =

  6. [14]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Fang, Yuxin and Wang, Wen and Xie, Binhui and Sun, Quan and Wu, Ledell and Wang, Xinggang and Huang, Tiejun and Wang, Xinlong and Cao, Yue , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  7. [15]

    Image and Vision Computing , volume =

    Fang, Yuxin and Sun, Quan and Wang, Xinggang and Huang, Tiejun and Wang, Xinlong and Cao, Yue , title =. Image and Vision Computing , volume =. 2024 , doi =

    2024

  8. [16]

    Informatics in Medicine Unlocked , volume =

    A histopathological image dataset for grading breast invasive ductal carcinomas , author =. Informatics in Medicine Unlocked , volume =. 2020 , issn =

    2020

  9. [17]

    BMC Research Notes , VOLUME =

    Forootan, Mojgan and Rajabnia, Mohsen and Mafi, Ahmad R and Tehrani, Hamed Azhdari and Ghadirzadeh, Erfan and Setayeshfar, Mahziar and Ghaffari, Zahra and Tashakoripour, Mohammad and Zali, Mohammad Reza and Bolhasani, Hamidreza , TITLE =. BMC Research Notes , VOLUME =. 2024 , NUMBER =

    2024

  10. [18]

    Learning multiple layers of features from tiny images , year =

    Krizhevsky, Alex and Hinton, Geoffrey , institution =. Learning multiple layers of features from tiny images , year =

  11. [19]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

    Liu, Zhuang and Mao, Hanzi and Wu, Chao-Yuan and Feichtenhofer, Christoph and Darrell, Trevor and Xie, Saining , title=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

  12. [20]

    Proceedings of the International Conference on Learning Representations , year=

    Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil , title=. Proceedings of the...

  13. [21]

    Algorithms , VOLUME =

    Chen, Jianzhang and Zhou, Shuo and Qiu, Jie and Xu, Yixin and Zeng, Bozhe and Fang, Wanchuan and Chen, Xiangying and Huang, Yipeng and Xu, Zhengquan and Chen, Youqin , TITLE =. Algorithms , VOLUME =. 2024 , NUMBER =

    2024

  14. [22]

    Proceedings of the IEEE International Symposium on Biomedical Imaging , pages=

    MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis , author=. Proceedings of the IEEE International Symposium on Biomedical Imaging , pages=

  15. [23]

    Neural Networks , volume =

    Domain-guided conditional diffusion model for unsupervised domain adaptation , author =. Neural Networks , volume =. 2025 , issn =

    2025

  16. [24]

    Neural Networks , volume =

    Deep learning in neural networks: An overview , author =. Neural Networks , volume =. 2015 , issn =

    2015

  17. [25]

    Neural Networks , volume =

    Continual Object Detection: A review of definitions, strategies, and challenges , author =. Neural Networks , volume =. 2023 , issn =

    2023

  18. [26]

    Neural Computation & Application , volume=

    Deep learning: systematic review, models, challenges, and research directions , author=. Neural Computation & Application , volume=. 2023 , doi=

    2023

  19. [27]

    arXiv preprint arXiv:2001.08361 , pages=

    Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , pages=

    2001 arXiv

  20. [28]

    , title =

    Strubell, Emma and Ganesh, Ananya and McCallum, Andrew. , title =. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

  21. [29]

    arXiv preprint arXiv:1802.05799 , year=

    Horovod: fast and easy distributed deep learning in TensorFlow , author=. arXiv preprint arXiv:1802.05799 , year=

    arXiv

  22. [30]

    , journal=

    Liu, Ximeng and Xie, Lehui and Wang, Yaopeng and Zou, Jian and Xiong, Jinbo and Ying, Zuobin and Vasilakos, Athanasios V. , journal=. Privacy and Security Issues in Deep Learning: A Survey , year=

  23. [31]

    Proceedings of the International Conference on Learning Representations , pages=

    Once for All: Train One Network And Specialize It for Efficient Deployment , author=. Proceedings of the International Conference on Learning Representations , pages=

  24. [32]

    arXiv preprint arXiv:2301.08727 , year=

    Neural Architecture Search: Insights from 1000 Papers , author=. arXiv preprint arXiv:2301.08727 , year=

    arXiv

  25. [33]

    Proceedings of the International Conference on Machine Learning , pages=

    Privacy for Free: How does Dataset Condensation Help Privacy? , author=. Proceedings of the International Conference on Machine Learning , pages=

  26. [34]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Dataset Distillation: A Comprehensive Review , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2023 , publisher=

    2023

  27. [35]

    Proceedings of the International Joint Conference on Artificial Intelligence , year=

    A Survey on Dataset Distillation: Approaches, Applications and Future Directions , author=. Proceedings of the International Joint Conference on Artificial Intelligence , year=

  28. [36]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    A Comprehensive Survey to Dataset Distillation , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2023 , publisher=

    2023

  29. [37]

    arXiv preprint arXiv:2502.05673 , year=

    The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions , author=. arXiv preprint arXiv:2502.05673 , year=

    arXiv

  30. [38]

    Journal of Informatics Electrical and Electronics Engineering , year=

    Image Classification: A Survey , author=. Journal of Informatics Electrical and Electronics Engineering , year=

  31. [39]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

    Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

  32. [40]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

    Towards Stable and Storage-efficient Dataset Distillation: Matching Convexified Trajectory , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

  33. [41]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Dataset Distillation via the Wasserstein Metric , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  34. [42]

    2025 , pages=

    Zhong, Xinhao and Fang, Hao and Chen, Bin and Gu, Xulin and Dai, Tao and Qiu, Meikang and Xia, Shu-Tao , booktitle=. 2025 , pages=

    2025

  35. [43]

    Proceedings of the IEEE International Conference on Multimedia and Expo , year=

    Enhancing Diffusion-based Dataset Distillation via Adversary-Guided Curriculum Sampling , author=. Proceedings of the IEEE International Conference on Multimedia and Expo , year=

  36. [44]

    Zhang, Jiaqing and Yin, Mingjia and Wang, Hao and Li, Yawen and Ye, Yuyang and Lou, Xingyu and Du, Junping and Chen, Enhong , booktitle=

  37. [45]

    Proceedings of the International Conference on Machine Learning , pages=

    Synthetic Text Generation for Training Large Language Models via Gradient Matching , author=. Proceedings of the International Conference on Machine Learning , pages=

  38. [46]

    Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention , pages=

    High-Order Progressive Trajectory Matching for Medical Image Dataset Distillation , author=. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention , pages=

  39. [47]

    arXiv preprint arXiv:2411.18109 , pages=

    Training Data Synthesis with Difficulty Controlled Diffusion Model , author=. arXiv preprint arXiv:2411.18109 , pages=

  40. [48]

    arXiv preprint arXiv:1611.01578 , pages=

    Generative Adversarial Networks , author=. arXiv preprint arXiv:1611.01578 , pages=

    arXiv

  41. [49]

    Proceedings of Advances in Neural Information Processing Systems , pages =

    Conditional Image Generation with PixelCNN Decoders , author =. Proceedings of Advances in Neural Information Processing Systems , pages =

  42. [50]

    Proceedings of the International Conference on Machine Learning , pages =

    Variational Inference with Normalizing Flows , author =. Proceedings of the International Conference on Machine Learning , pages =

  43. [51]

    ACM Computer Survey , pages =

    Yang, Ling and Zhang, Zhilong and Song, Yang and Hong, Shenda and Xu, Runsheng and Zhao, Yue and Zhang, Wentao and Cui, Bin and Yang, Ming-Hsuan , title =. ACM Computer Survey , pages =. 2023 , doi =

    2023

  44. [52]

    arXiv preprint arXiv:2404.07771 , pages=

    An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization , author=. arXiv preprint arXiv:2404.07771 , pages=

    arXiv

  45. [53]

    Proceedings of the International Conference on Learning Representations , pages=

    Denoising Diffusion Implicit Models , author=. Proceedings of the International Conference on Learning Representations , pages=

  46. [54]

    Proceedings of Advances in Neural Information Processing Systems , pages=

    Ho, Jonathan and Jain, Ajay and Abbeel, Pieter , title =. Proceedings of Advances in Neural Information Processing Systems , pages=

  47. [55]

    Proceedings of the International Conference on Learning Representations , pages=

    Score-Based Generative Modeling through Stochastic Differential Equations , author=. Proceedings of the International Conference on Learning Representations , pages=

  48. [56]

    arXiv preprint arXiv:2207.12598 , year=

    Classifier-Free Diffusion Guidance , author=. arXiv preprint arXiv:2207.12598 , year=

    arXiv

  49. [57]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bjorn , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  50. [58]

    Proceedings of the AAAI Conference on Artificial Intelligence , pages=

    Summarizing Stream Data for Memory-Restricted Online Continual Learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , pages=

  51. [59]

    Proceedings of the European Conference on Computer Vision , pages=

    Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents , author=. Proceedings of the European Conference on Computer Vision , pages=

  52. [60]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Improving Noise Efficiency in Privacy-preserving Dataset Distillation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  53. [61]

    arXiv preprint arXiv:2405.17535 , year=

    Calibrated Dataset Condensation for Faster Hyperparameter Search , author=. arXiv preprint arXiv:2405.17535 , year=

    arXiv

  54. [62]

    Proceedings of the International Conference on Learning Representations , pages=

    Dataset Condensation with Gradient Matching , author=. Proceedings of the International Conference on Learning Representations , pages=

  55. [63]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Dataset Distillation by Matching Training Trajectories , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  56. [64]

    Wang, Kai and Zhao, Bo and Peng, Xiangyu and Zhu, Zheng and Yang, Shuo and Wang, Shuo and Huang, Guan and Bilen, Hakan and Wang, Xinchao and You, Yang , booktitle=

  57. [65]

    Proceedings of the International Conference on Machine Learning , year=

    GeoDM: Geometry-aware Distribution Matching for Dataset Distillation , author=. Proceedings of the International Conference on Machine Learning , year=

  58. [66]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

    Dataset Distillation via the Wasserstein Metric , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

  59. [67]

    2018 , publisher =

    McInnes, Leland and Healy, John and Saul, Nathaniel and Großberger, Lukas , title =. 2018 , publisher =. doi:10.21105/joss.00861 , url =

    2018 doi

  60. [68]

    Proceedings of the Advances in Neural Information Processing Systems, Workshop , year=

    Synthesizing Informative Training Samples with GAN , author=. Proceedings of the Advances in Neural Information Processing Systems, Workshop , year=

  61. [69]

    Proceedings of the International Conference on Machine Learning , pages=

    Dataset Condensation via Efficient Synthetic-Data Parameterization , author=. Proceedings of the International Conference on Machine Learning , pages=

  62. [70]

    Proceedings of the European Conference on Computer Vision , pages=

    Dataset Distillation by Automatic Training Trajectories , author=. Proceedings of the European Conference on Computer Vision , pages=

  63. [71]

    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

    Dataset Condensation with Distribution Matching , author=. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

  64. [72]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

    Diversity-Enhanced Distribution Alignment for Dataset Distillation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

  65. [73]

    Proceedings of the International Conference on Learning Representations , pages=

    Dataset Meta-Learning from Kernel Ridge-Regression , author=. Proceedings of the International Conference on Learning Representations , pages=

  66. [74]

    Proceedings of Advances in Neural Information Processing Systems , pages=

    Dataset Distillation with Infinitely Wide Convolutional Networks , author=. Proceedings of Advances in Neural Information Processing Systems , pages=

  67. [75]

    Proceedings of Advances in Neural Information Processing Systems , pages=

    Efficient Dataset Distillation using Random Feature Approximation , author=. Proceedings of Advances in Neural Information Processing Systems , pages=

  68. [76]

    Proceedings of Advances in Neural Information Processing Systems , pages=

    Provable and Efficient Dataset Distillation for Kernel Ridge Regression , author=. Proceedings of Advances in Neural Information Processing Systems , pages=

  69. [77]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Generalizing Dataset Distillation via Deep Generative Prior , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  70. [78]

    Wang, Haoxuan and Zhao, Zhenghao and Wu, Junyi and Shang, Yuzhang and Liu, Gaowen and Yan, Yan , booktitle=

  71. [79]

    Proceedings of the ACM International Conference on Multimedia , pages=

    Diversified Semantic Distribution Matching for Dataset Distillation , author=. Proceedings of the ACM International Conference on Multimedia , pages=

  72. [80]

    and Lawryshyn, Yuri A

    Sajedi, Ahmad and Khaki, Samir and Amjadian, Ehsan and Liu, Lucy Z. and Lawryshyn, Yuri A. and Plataniotis, Konstantinos N. , booktitle=

  73. [81]

    Proceedings of the International Conference on Machine Learning , pages=

    Point Cloud Dataset Distillation , author=. Proceedings of the International Conference on Machine Learning , pages=

  74. [82]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Towards Universal Dataset Distillation via Task-Driven Diffusion , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  75. [83]

    arXiv preprint arXiv:1811.10959 , pages=

    Dataset Distillation , author=. arXiv preprint arXiv:1811.10959 , pages=

    arXiv

  76. [84]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  77. [85]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Scalable Diffusion Models with Transformers , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  78. [86]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Efficient Dataset Distillation via Minimax Diffusion , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  79. [87]

    and Tirupattur, Praveen and Nayak, Gaurav Kumar and Liu, Gaowen and Shah, Mubarak , booktitle=

    Chan-Santiago, Jeffrey A. and Tirupattur, Praveen and Nayak, Gaurav Kumar and Liu, Gaowen and Shah, Mubarak , booktitle=. 2025 , pages=

    2025

  80. [88]

    arXiv preprint arXiv:1708.00489 , pages=

    Active learning for convolutional neural networks: A core-set approach , author=. arXiv preprint arXiv:1708.00489 , pages=

    arXiv

  81. [89]

    Proceedings of the International Conference on Machine Learning , pages =

    Herding dynamical weights to learn , author =. Proceedings of the International Conference on Machine Learning , pages =

  82. [90]

    Su, Duo and Hou, Junjie and Gao, Weizhi and Tian, Yingjie and Tang, Bowen , booktitle=

  83. [91]

    Proceedings of Advances in Neural Information Processing Systems , year=

    Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective , author=. Proceedings of Advances in Neural Information Processing Systems , year=

  84. [92]

    arXiv preprint arXiv:1312.6114 , pages=

    Auto-Encoding Variational Bayes , author=. arXiv preprint arXiv:1312.6114 , pages=

    arXiv

  85. [93]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  86. [94]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Dynamic few-shot visual learning without forgetting , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  87. [95]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Deep residual learning for image recognition , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  88. [96]

    Berg and Li Fei-Fei , Title=

    Olga Russakovsky and Jia Deng and Hao Su and Jonathan Krause and Sanjeev Satheesh and Sean Ma and Zhiheng Huang and Andrej Karpathy and Aditya Khosla and Michael Bernstein and Alexander C. Berg and Li Fei-Fei , Title=. 2015 , journal=

    2015

  89. [97]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    ImageNet: A large-scale hierarchical image database , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  90. [98]

    2019 , note=

    Fastai , title=. 2019 , note=

    2019

  91. [99]

    Li, Guang and Zhao, Bo and Wang, Tongzhou , title=

  92. [100]

    Proceedings of the IEEE International Conference on Image Processing , pages=

    Soft-Label Anonymous Gastric X-Ray Image Distillation , author=. Proceedings of the IEEE International Conference on Image Processing , pages=

  93. [101]

    Computer Methods and Programs in Biomedicine , year=

    Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharing , author=. Computer Methods and Programs in Biomedicine , year=

  94. [102]

    Proceedings of the IEEE International Conference on Image Processing , pages=

    Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory , author=. Proceedings of the IEEE International Conference on Image Processing , pages=

  95. [103]

    arXiv preprint arXiv:2605.07194 , year=

    Closed-Form Linear-Probe Dataset Distillation for Pre-trained Vision Models , author=. arXiv preprint arXiv:2605.07194 , year=

    arXiv

  96. [104]

    Proceedings of the International Conference on Learning Representations , year=

    Influence-Guided Diffusion for Dataset Distillation , author=. Proceedings of the International Conference on Learning Representations , year=

  97. [105]

    Proceedings of the International Conference on Learning Representations , year=

    Diffusion Models as Dataset Distillation Priors , author=. Proceedings of the International Conference on Learning Representations , year=

  98. [106]

    Wang, Kai and Gu, Jianyang and Zhou, Daquan and Zhu, Zheng and Jiang, Wei and You, Yang , booktitle=

  99. [107]

    Kim, Dahye and Thomas, Xavier and Ghadiyaram, Deepti , booktitle=

  100. [108]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , pages=

    Generative Dataset Distillation: Balancing Global Structure and Local Details , author=. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , pages=

  101. [109]

    IEEE/CVF International Conference on Computer Vision Workshops , year=

    Task-Specific Generative Dataset Distillation with Difficulty-Guided Sampling , author=. IEEE/CVF International Conference on Computer Vision Workshops , year=

  102. [110]

    Advances in Neural Information Processing Systems Workshops , year=

    Information-Guided Diffusion Sampling for Dataset Distillation , author=. Advances in Neural Information Processing Systems Workshops , year=

  103. [111]

    arXiv preprint arXiv:2511.17890 , year=

    Decoupled Audio-Visual Dataset Distillation , author=. arXiv preprint arXiv:2511.17890 , year=

  104. [112]

    Ma, Hongxu and Li, Guang and Wang, Shijie and Zhou, Dongzhan and Sun, Baoli and Ogawa, Takahiro and Haseyama, Miki and Wang, Zhihui , booktitle=

  105. [113]

    European Conference on Computer Vision Workshops , year=

    Generative Dataset Distillation Based on Diffusion Model , author=. European Conference on Computer Vision Workshops , year=

  106. [114]

    Neural Networks , year=

    Importance-Aware Adaptive Dataset Distillation , author=. Neural Networks , year=

  107. [115]

    IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences , year=

    Dataset Distillation using Parameter Pruning , author=. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences , year=

  108. [116]

    Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Workshop , pages=

    Dataset Distillation for Medical Dataset Sharing , author=. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Workshop , pages=

  109. [117]

    IEEE International Conference on Image Processing (ICIP) , year=

    Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory , author=. IEEE International Conference on Image Processing (ICIP) , year=

  110. [118]

    IEEE/CVF International Conference on Computer Vision , year=

    Dataset Distillation via Vision-Language Category Prototype , author=. IEEE/CVF International Conference on Computer Vision , year=

  111. [119]

    Cai, Wenqi and Zou, Yawen and Li, Guang and Gu, Chunzhi and Zhang, Chao , booktitle=

  112. [120]

    Transactions on Machine Learning Research , year=

    Dataset Condensation with Color Compensation , author=. Transactions on Machine Learning Research , year=

  113. [121]

    Advances in Neural Information Processing Systems , year=

    Hyperbolic Dataset Distillation , author=. Advances in Neural Information Processing Systems , year=

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.