Pith. sign in

REVIEW 4 major objections 6 minor 68 references

CONCORD: Concept-Informed Diffusion for Dataset Distillation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM concepts guide diffusion to build better tiny training sets

desk verdict A useful, well-ablated training-free guidance for diffusion-based dataset distillation, but the paper omits the latent-to-CLIP interface that its whole mechanism rests on and overstates SOTA. read the letter →

arxiv 2505.18358 v1 pith:NDU55TCA submitted 2025-05-23 cs.CV

Jianyang Gu , Haonan Wang , Ruoxi Jia , Saeed Vahidian , Vyacheslav Kungurtsev , Wei Jiang , Yiran Chen This is my paper
classification cs.CV
keywords datasetdistillationdiffusionmodelsLLMconceptsCLIPguidanceimagesynthesisNettrainingdatageneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dataset distillation compresses a large dataset into a tiny surrogate set that a model can train on nearly as effectively. CONCORD argues that diffusion-based distillation produces images that imitate the overall data distribution but often omit or distort the specific visual details needed to recognize a category, and that this instance-level “concept incompleteness” is costly when only a few images per class are allowed. The method retrieves fine-grained, class-distinguishing visual descriptions from an LLM, filters them by how well they match real images using CLIP, and then uses a contrastive similarity objective during the denoising steps to push each generated sample toward those concepts and away from concepts of similar categories. The paper reports that this training-free, classifier-free guidance consistently improves the accuracy of distilled datasets over the Minimax and unCLIP baselines on ImageWoof, ImageNet-100, ImageNet-1K, and Food-101, achieving the best numbers among the compared methods. The work matters because it offers a route toward controllable and interpretable generation of high-quality training data, especially for resource-limited users who want custom distilled datasets without retraining a generative model.

What carries the argument

The central mechanism is concept-informed denoising, a gradient guidance term added to the predicted noise in Eq. 9: the corrected noise is $\hat{\epsilon} = \epsilon_\theta(x(t), t) - \lambda \sqrt{1 - \bar{\alpha}_t} \nabla_{x(t)} O(x(t), C, \tilde{C})$, where $O$ is the image-text contrastive objective of Eq. 11, $C$ is the set of LLM-retrieved positive concepts for the class, and $\tilde{C}$ is a set of negative concepts from similar categories weighted by label similarity. The concepts are produced by prompting GPT-4o with a prompt that asks for appearance-based, class-distinguishing phrases, then filtering with CLIP cosine similarity to real images.

What would settle it

A direct test is to compute CLIP similarity between the final generated images and the reference concepts before and after CONCORD, and measure whether the images that gain the most in concept similarity are also the ones that improve downstream training accuracy. If concept similarity rises while accuracy stays flat or drops, the guidance would be optimizing a correlate rather than the training-relevant property.

Watch

Extended reading notes

Core claim

The paper claims that inserting an LLM-derived, CLIP-scored concept-matching objective into the denoising loop of a diffusion model improves the quality of distilled datasets. For each target class, GPT-4o supplies ten short appearance descriptions; the five with the highest average cosine similarity to real images are kept. During each reverse diffusion step, the predicted noise is adjusted by the gradient of a contrastive objective that pulls the noisy sample toward positive concepts from the target class and pushes it away from negative concepts sampled from visually similar classes. The updated noise then drives the DDIM denoising step. The authors report that this procedure, applied at inference time without fine-tuning or classifiers, improves top-1 accuracy of models trained on the distilled data across both latent diffusion baselines, with the largest gains at moderate-to-large images-per-class settings; for example, on ImageNet-1K with 50 images per class, Minimax with CONCORD reaches 59.4% versus 59.1% for Minimax, and on ImageNet-100 with 10 images per class it reaches 33.3% versus 31.6%. They interpret these gains as evidence that instance-level concept completeness, not just distribution matching, is a load-bearing ingredient in diffusion-based dataset distillation.

Load-bearing premise

The central assumption is that the gradient of a CLIP text-image cosine similarity, computed on the noisy latent $x(t)$ and added to the predicted noise, actually steers the denoising trajectory toward images whose object details are more complete and more faithful to the category.

Editorial extensions

If this is right

  • The concept-informed guidance can be added to any existing diffusion-based dataset distillation pipeline without retraining or re-tuning the generative model, so improvements should transfer to other generative DD frameworks.
  • The same guidance should let practitioners distill custom datasets from a pretrained text-to-image or image-to-image model, needing only category names to obtain concepts, which suits resource-limited settings.
  • Scaling the distilled dataset size up to 200 images per class with CONCORD reaches the full-dataset accuracy on ImageWoof under the same validation protocol, suggesting that concept completeness becomes the limiting factor at larger budgets.
  • Since the concepts give a textual reason for each refinement, the resulting distilled datasets come with a human-readable explanation of which visual details were deemed essential.
  • Negative-concept guidance from similar categories should generalize beyond ImageNet to other fine-grained benchmarks where class confusion is the main error source.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be to replace CLIP with a different embedding model or a vision-language model with finer visual granularity and check whether larger gains appear on fine-grained classes; the paper's own ablations only vary the LLM and the prompt, not the scoring model.
  • The fact that gains shrink as class count grows suggests that the mechanism may be targeting intra-class detail rather than inter-class separation; on larger datasets, dataset-level diversity might swamp the benefit of instance-level fidelity.
  • The method's extra cost is one CLIP image encoding per denoising step, plus a gradient through that encoder; applying the guidance only in selected denoising stages might retain most of the gain at a fraction of the reported 1-1.5x overhead.
  • Because the objective is a cosine similarity score, the concept filter and the guidance share the same CLIP space; the choice of the filter threshold and of the number of concepts per class likely interact with the guidance weight $\lambda$, and the paper tunes these only separately.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CONCORD, a training-free guidance scheme for diffusion-based dataset distillation. For each class, an LLM retrieves fine-grained visual concepts, which are filtered by CLIP similarity to real images; during DDIM sampling, a contrastive objective between the noisy sample and positive/negative concepts is differentiated to modify the predicted noise (Eqs. 9–11). The method is applied to two baselines, Minimax (DiT) and Stable Diffusion unCLIP Img2Img, and evaluated on ImageNet-1K, ImageNet-100, ImageWoof, and Food-101. The authors report accuracy gains over both baselines in most settings and claim state-of-the-art performance on ImageNet-1K and its subsets.

Significance. If the mechanism is implemented as described, CONCORD is a conceptually appealing contribution: it injects LLM-derived semantic knowledge into diffusion sampling for dataset distillation without retraining the generative model, and it provides a degree of interpretability through the selected concepts. The experimental matrix is broad, including multiple baselines, architectures, IPC settings, and ablations (classifier guidance, cosine vs. contrastive objectives, negative-sampling strategies, prompt designs). The paper also ships code and reports inference-time overhead. However, the significance is currently undercut by an unspecified but central implementation detail for latent-space diffusion models, by overclaimed state-of-the-art status, and by a hyperparameter inconsistency between Sections 4.1 and 4.4.

major comments (4)
  1. [Sec. 3.3, Eqs. (9)–(11)] The core steering signal ∇_{x(t)} O(x(t), C) in Eq. (9) is not well-defined for either deployment baseline. Minimax uses a DiT operating on a VAE latent, and unCLIP uses Stable Diffusion, also on VAE latents; CLIP expects RGB images. The manuscript never states whether x(t) is decoded to pixels at each denoising step, how the decoded image is differentiated through the decoder, or whether the latent is fed to CLIP directly (which would place the CLIP encoder out of distribution). This is not a mere implementation footnote: if the gradient is computed on a misaligned representation, the claimed semantics of 'concept completeness' in Eq. (11) do not follow, and the observed gains could be an artifact of an unspecified latent-space regularizer. The paper must specify the exact mapping and backpropagation path, or the central mechanism is not verifiable.
  2. [Abstract and Sec. 4.2, Tables 1–2] The abstract claims 'state-of-the-art performance on ImageNet-1K and its subsets,' and Sec. 4.2 claims 'significant performance improvements ... across all IPC settings and architectures.' Both statements are too strong. In Table 2, at IPC=1 RDED outperforms Minimax^C on ImageNet-1K (6.6 vs. 6.4) and on ImageNet-100 (8.1 vs. 7.1). In Table 1, on ImageWoof IPC=1 with ConvNet, unCLIP^C (19.9) is worse than the unCLIP baseline (20.5). The text partially acknowledges the IPC=1 caveat in Sec. 4.2, but the abstract and the 'across all' phrasing should be revised to reflect the actual scope of the improvements, e.g., 'consistent gains over our baselines in most settings, with the largest gains at higher IPC.'
  3. [Sec. 4.1 vs. Sec. 4.4] The value of the informing weight λ is stated inconsistently. Sec. 4.1 says 'The informing weight λ in Eq. 9 is set as 1.' Sec. 4.4, after the parameter analysis, says 'we set the value of λ as 2.0 for balance between sufficient control and stable denoising.' The paper must state which value was used for all main experiments (Tables 1–3 and Fig. 4a), and reconcile the two statements. This is a reproducibility-critical hyperparameter.
  4. [Sec. 3.2 and Sec. 3.3] The concept selection and the guidance objective use the same CLIP model: Sec. 3.2 selects concepts with the highest activation on real images, and Eq. (11) later maximizes the similarity to those selected concepts. This creates a potential circularity: the objective may simply reinforce CLIP alignment to the real data distribution rather than measuring an independent notion of 'concept completeness.' The paper should explicitly discuss this issue and, ideally, provide an external validation of the selected concepts (e.g., human evaluation or an attribute detector) to break the circularity.
minor comments (6)
  1. [General] The LLM model names are inconsistent across the paper: Table 4 uses 'Ours-3.5' and 'Ours-4,' while Fig. 7 and the supplementary use 'Ours-4o.' Please standardize.
  2. [Sec. 4.4] There is a typo: 'contastive objective' should be 'contrastive objective.'
  3. [Sec. 3.1, Eqs. (6) and (9)] The relationship between the classifier-guidance-style update in Eq. (6) and the DDIM update in Eq. (9) is not fully explained; in particular, the factor λ√(1−ᾱ_t) in Eq. (9) is introduced without derivation. A sentence clarifying how this scaling follows from the DDIM formulation would help.
  4. [Supplementary, Tab. 10] The reported inference times show roughly a 2× increase for CONCORD (e.g., unCLIP: 9.7s vs. 20.8s), but the text says 'increases the original inference cost by approximately 1-1.5 times.' Please clarify whether this means a 100–150% increase (i.e., total factor 2–2.5) or a 1–1.5× total factor; the numbers support the former.
  5. [Fig. 1 and Figs. 8–10] The claimed defects (e.g., 'unrealistic back legs of the beagle,' 'missing wing') are difficult to discern in the printed figures. Adding zoomed insets or arrows would make the qualitative claims more convincing.
  6. [References] Reference [14] is cited for ImageWoof but the title indicates ImageNette; please verify the correct source for the ImageWoof dataset.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claim is measured by downstream ResNet validation accuracy on real test sets, not by the CLIP objective that drives generation.

full rationale

The paper's load-bearing claim is that concept-informed denoising guidance improves distilled-dataset quality, and this is verified by an external benchmark: distilled images are trained into ResNet/ConvNet models and evaluated on the original ImageNet validation sets under the RDED protocol (Sec. 4.1 and Appendix B). Concept selection (Sec. 3.2) and the informing objective (Eq. 11) both use CLIP similarity, so the internal notion of 'concept completeness' is CLIP-relative, but no equation or claim equates that similarity with downstream accuracy; the reported numbers are not a fitted parameter renamed as a prediction. The informing weight, number of concepts, and number of negative samples are fixed or swept in ablations, not tuned per benchmark to reproduce a target test accuracy. The two self-citations are not load-bearing: [15] is the Minimax baseline that CONCORD extends and that is independently implemented and evaluated, and [26] appears only in the supplementary as analytical motivation for 'instrumental DD,' not as a derivation of the method's effectiveness. There is no imported uniqueness theorem and no claim that alternatives are forbidden by a self-cited result. The paper's genuine weakness—Eq. 9 takes gradients with respect to x(t) while both deployment models are latent diffusion models and the manuscript does not specify the latent-to-image mapping for CLIP or the backpropagation path—is a reproducibility and correctness concern about whether the guidance signal is meaningful, not circularity: the mechanism is an empirical hypothesis, not an input that is definitionally equal to the output. The appended limitations (extra computational cost and failure cases where the informing cannot restore missing body parts) further confirm that the paper presents an empirical method with acknowledged gaps rather than a derivation whose result is guaranteed by its assumptions. Therefore no circular step is present, and the score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The method rests on standard diffusion sampling equations (DDPM/DDIM), the classifier-guidance approximation, and the assumption that CLIP similarity of noisy samples to LLM-retrieved text concepts is a faithful control signal. The fitted numeric parameters are λ, the number of positive and negative concepts, the denoising steps, and the unreported temperature τ. No new physical or mathematical entities are introduced beyond the explanatory construct "concept completeness."

free parameters (6)
  • informing weight λ = reported as 1 (Sec. 4.1), 2.0 (Sec. 4.4), 1 (supplementary B)
    Controls the strength of the concept gradient added to the predicted noise; tuned per dataset or setting.
  • number of positive concepts |C| = 5
    Top-5 CLIP-selected descriptions per class, chosen by hand.
  • number of negative concepts = 10
    Number of negative descriptions in the contrastive objective, tuned in Fig. 4c.
  • denoising steps = 50
    Total DDIM steps; analyzed in Fig. 5 of the supplementary material.
  • contrastive temperature τ = not reported
    Scales the logits in Eq. 11 and is never given a numeric value in the paper.
  • classifier guidance weight (ablation only) = 0.05
    Used for the classifier-guidance comparison in Table 6; not part of the final method.
assumptions (4)
  • domain assumption The classifier-guidance score approximation: adding λ√(1−ᾱ_t)∇_{x(t)} O to the predicted noise approximates sampling from a distribution biased toward high O(x(t), C).
    Standard assumption inherited from Dhariwal and Nichol (cited as [11]); the paper's Eq. 9 relies on it without re-deriving it for a CLIP-based objective.
  • domain assumption CLIP text-image cosine similarity is a valid proxy for concept completeness and for the visual presence of an attribute in a noisy image.
    Used both in concept validity filtering (Sec. 3.2) and in the guidance objective (Eq. 10-11).
  • domain assumption LLM-retrieved descriptions for a category are accurate, mostly visual, and sufficiently distinguishable to guide generation.
    The method depends on GPT-4o outputs; Table 4 shows that prompt and LLM choice materially affect results.
  • domain assumption DDIM deterministic sampling can be perturbed by an additive gradient without destabilizing the trajectory.
    Eq. 7-9 assume the corrected noise ϵ̂ still lies on a valid denoising path.
invented entities (1)
  • concept completeness
    purpose: Explanatory construct for why informing helps: generated images may miss or misrepresent essential object details, and informing fixes this.
    The paper never measures concept completeness directly; it is inferred from visual inspection (Figs. 1 and 3) and downstream accuracy, so it is a postulated latent quality rather than an independently verified quantity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CONCORD: Concept-Informed Diffusion for Dataset Distillation." pith.science (2026). https://pith.science/paper/NDU55TCA

@misc{pith2026250518358,
  author       = {Pith},
  title        = {Pith review of: CONCORD: Concept-Informed Diffusion for Dataset Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NDU55TCA}},
  note         = {Machine review of arXiv:2505.18358}
}
read the original abstract

Dataset distillation (DD) has witnessed significant progress in creating small datasets that encapsulate rich information from large original ones. Particularly, methods based on generative priors show promising performance, while maintaining computational efficiency and cross-architecture generalization. However, the generation process lacks explicit controllability for each sample. Previous distillation methods primarily match the real distribution from the perspective of the entire dataset, whereas overlooking concept completeness at the instance level. The missing or incorrectly represented object details cannot be efficiently compensated due to the constrained sample amount typical in DD settings. To this end, we propose incorporating the concept understanding of large language models (LLMs) to perform Concept-Informed Diffusion (CONCORD) for dataset distillation. Specifically, distinguishable and fine-grained concepts are retrieved based on category labels to inform the denoising process and refine essential object details. By integrating these concepts, the proposed method significantly enhances both the controllability and interpretability of the distilled image generation, without relying on pre-trained classifiers. We demonstrate the efficacy of CONCORD by achieving state-of-the-art performance on ImageNet-1K and its subsets. The code implementation is released in https://github.com/vimar-gu/CONCORD.

Figures

Figures reproduced from arXiv: 2505.18358 by the authors.

Figure 1
Figure 1. Comparison of generated images with and without our proposed C [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of the proposed CONCORD method. Descriptive concepts are retrieved and utilized to inform the diffusion denoising process. Better instance-level concept completeness helps to construct a surrogate dataset with higher overall quality. 3. Method In this section, we demonstrate the detailed modules of our proposed Concept-Informed Diffusion method (CONCORD). Firstly, we present the preliminary knowl￾edge o… view at source ↗
Figure 3
Figure 3. Comparison between images informed by fine-grained descriptive concepts (the first row) and the class name alone (the second [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: (a) Applying the proposed concept informing brings con [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Parameter analysis on the denoising step number. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Feature distribution visualization of (a) samples generated by unCLIP with and without C [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: The comparison between concepts retrieved by different prompts and LLMs. “Cls” refers to prompts used for zero-shot classifi [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Example generated image comparison on Minimax with and without the proposed C [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Example generated image comparison on unCLIP Img2Img with and without the proposed C [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Example cases where CONCORD fails to supplement or modify incorrect concepts in the images (C indicates the application of CONCORD). 7 [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Example images generated by the proposed C [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Example animal images generated by the proposed C [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Example other images generated by the proposed C [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 54 canonical work pages

  1. [1]

    Sequential modeling enables scal- able learning for large vision models

    Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan Yuille, Trevor Darrell, Jitendra Malik, and Alexei A Efros. Sequential modeling enables scal- able learning for large vision models. arXiv preprint arXiv:2312.00785, 2023. 1

    arXiv 2023

  2. [2]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In ECCV, pages 446–461. Springer, 2014. 5

    work page 2014

  3. [3]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In NeurIPS, pages 1877–1901, 2020. 1

    work page 1901

  4. [4]

    Dataset distillation by matching training trajectories

    George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In CVPR, pages 4750–4759,

    work page

  5. [5]

    Generalizing dataset dis- tillation via deep generative prior

    George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Generalizing dataset dis- tillation via deep generative prior. In CVPR, pages 3739– 3748, 2023. 2

    work page 2023

  6. [6]

    When large language models meet personalization: Perspectives of challenges and opportuni- ties

    Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, et al. When large language models meet personalization: Perspectives of challenges and opportuni- ties. World Wide Web, 27(4):42, 2024. 1

    work page 2024

  7. [7]

    Diffedit: Diffusion-based semantic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion-based semantic image editing with mask guidance. In ICLR, 2023. 2

    work page 2023

  8. [8]

    Scaling up dataset distillation to imagenet-1k with constant memory

    Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet-1k with constant memory. In ICML, pages 6565–6590. PMLR, 2023. 2

    work page 2023

Show all 68 references
  1. [9]

    ImageNet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, pages 248–255, 2009. 1, 5

    2009

  2. [10]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 1

    2018 arXiv

  3. [11]

    Diffusion Models Beat GANs on Image Synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion Models Beat GANs on Image Synthesis. In NeurIPS, pages 8780– 8794, 2021. 2, 3

    2021

  4. [12]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition ...

    2022

  5. [13]

    Human-centered explainable ai (hcxai): Reloading explainability in the era of large language models (llms)

    Upol Ehsan, Elizabeth A Watkins, Philipp Wintersberger, Carina Manger, Sunnie SY Kim, Niels Van Berkel, Andreas Riener, and Mark O Riedl. Human-centered explainable ai (hcxai): Reloading explainability in the era of large language models (llms). In Extended Abstracts of the CH...

    2024

  6. [14]

    Fastai/imagenette: A smaller subset of 10 easily clas- sified classes from imagenet, and a little more french

    Fastai. Fastai/imagenette: A smaller subset of 10 easily clas- sified classes from imagenet, and a little more french. 5

  7. [15]

    Efficient dataset distillation via minimax diffusion

    Jianyang Gu, Saeed Vahidian, Vyacheslav Kungurtsev, Hao- nan Wang, Wei Jiang, Yang You, and Yiran Chen. Efficient dataset distillation via minimax diffusion. In CVPR, pages 15793–15803, 2024. 2, 3, 5, 7, 1

    2024

  8. [16]

    Sum- marizing stream data for memory-restricted online continual learning

    Jianyang Gu, Kai Wang, Wei Jiang, and Yang You. Sum- marizing stream data for memory-restricted online continual learning. In AAAI, pages 12217–12225, 2024. 2

    2024

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016. 1, 2

    2016

  10. [18]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, pages 16000–16009, 2022. 1

    2022

  11. [19]

    Prompt-to-prompt image editing with cross-attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-prompt image editing with cross-attention control. In ICLR, 2023. 2

    2023

  12. [20]

    Denoising Dif- fusion Probabilistic Models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Dif- fusion Probabilistic Models. In NeurIPS, pages 6840–6851,

  13. [21]

    Supervised contrastive learning

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. InNeurIPS, pages 18661–18673, 2020. 5

    2020

  14. [22]

    Dif- fusionclip: Text-guided diffusion models for robust image manipulation

    Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Dif- fusionclip: Text-guided diffusion models for robust image manipulation. In CVPR, pages 2426–2435, 2022. 2

    2022

  15. [23]

    Dataset condensation via efficient synthetic- data parameterization

    Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joonhyun Jeong, Jung-Woo Ha, and Hyun Oh Song. Dataset condensation via efficient synthetic- data parameterization. In ICML, pages 11102–11118, 2022. 2

    2022

  16. [24]

    Variational diffusion models

    Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In NeurIPS, pages 21696– 21707, 2021. 2

    2021

  17. [25]

    Are large lan- guage models post hoc explainers? arXiv preprint arXiv:2310.05797, 2023

    Nicholas Kroeger, Dan Ley, Satyapriya Krishna, Chi- rag Agarwal, and Himabindu Lakkaraju. Are large lan- guage models post hoc explainers? arXiv preprint arXiv:2310.05797, 2023. 1

    2023 arXiv

  18. [26]

    Dataset distillation from first principles: Integrating core in- formation extraction and purposeful learning

    Vyacheslav Kungurtsev, Yuanfang Peng, Jianyang Gu, Saeed Vahidian, Anthony Quinn, Fadwa Idlahcen, and Yiran Chen. Dataset distillation from first principles: Integrating core in- formation extraction and purposeful learning. arXiv preprint arXiv:2409.01410, 2024. 1 9

    2024 arXiv

  19. [27]

    Diffusion-based image translation using disentangled style and content representa- tion

    Gihyun Kwon and Jong Chul Ye. Diffusion-based image translation using disentangled style and content representa- tion. In ICLR, 2023. 2

    2023

  20. [28]

    Dataset distillation via factorization

    Songhua Liu, Kai Wang, Xingyi Yang, Jingwen Ye, and Xin- chao Wang. Dataset distillation via factorization. NeurIPS, 35:1100–1113, 2022. 2

    2022

  21. [29]

    Dream: Efficient dataset distillation by rep- resentative matching

    Yanqing Liu, Jianyang Gu, Kai Wang, Zheng Zhu, Wei Jiang, and Yang You. Dream: Efficient dataset distillation by rep- resentative matching. In ICCV, pages 17314–17324, 2023. 2

    2023

  22. [30]

    Efficient dataset distillation using random feature ap- proximation

    Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Efficient dataset distillation using random feature ap- proximation. In NeurIPS, pages 13877–13891, 2022. 2

    2022

  23. [31]

    Dataset distillation with convexified implicit gradients.arXiv preprint arXiv:2302.06755, 2023

    Noel Loo, Ramin Hasani, Mathias Lechner, and Daniela Rus. Dataset distillation with convexified implicit gradients.arXiv preprint arXiv:2302.06755, 2023. 2

    2023 arXiv

  24. [32]

    Dilm: Distilling dataset into language model for text-level dataset distillation

    Aru Maekawa, Satoshi Kosugi, Kotaro Funakoshi, and Man- abu Okumura. Dilm: Distilling dataset into language model for text-level dataset distillation. arXiv preprint arXiv:2404.00264, 2024. 2

    2024 arXiv

  25. [33]

    Visual classification via description from large language models

    Sachit Menon and Carl V ondrick. Visual classification via description from large language models. In ICLR, 2023. 4, 6

    2023

  26. [34]

    Latent dataset distillation with diffusion models

    Brian B Moser, Federico Raue, Sebastian Palacio, Stanislav Frolov, and Andreas Dengel. Latent dataset distillation with diffusion models. arXiv preprint arXiv:2403.03881, 2024. 2, 3

    2024

  27. [35]

    Dataset meta-learning from kernel ridge-regression

    Timothy Nguyen, Zhourong Chen, and Jaehoon Lee. Dataset meta-learning from kernel ridge-regression. In ICLR, 2021. 2

    2021

  28. [36]

    Dataset distillation with infinitely wide convolutional networks

    Timothy Nguyen, Roman Novak, Lechao Xiao, and Jaehoon Lee. Dataset distillation with infinitely wide convolutional networks. NeurIPS, 34:5186–5198, 2021. 2

    2021

  29. [37]

    Improved Denoising Diffusion Probabilistic Models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models. In ICML, pages 8162–8171, 2021. 2

    2021

  30. [38]

    Eclipse: A resource-efficient text-to-image prior for image generations

    Maitreya Patel, Changhoon Kim, Sheng Cheng, Chitta Baral, and Yezhou Yang. Eclipse: A resource-efficient text-to-image prior for image generations. arXiv preprint arXiv:2312.04655, 2023. 5

    2023 arXiv

  31. [39]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, pages 4195–4205, 2023. 5, 1, 3

    2023

  32. [40]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, pages 8748–8763. PMLR, 2021. 1, 5

    2021

  33. [41]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125,

    arXiv

  34. [42]

    An empirical bayes approach to statis- tics

    Herbert E Robbins. An empirical bayes approach to statis- tics. In Breakthroughs in Statistics: Foundations and basic theory, pages 388–394. Springer, 1992. 4

    1992

  35. [43]

    High-Resolution Image Synthesis With Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-Resolution Image Synthesis With Latent Diffusion Models. In CVPR, pages 10684–10695, 2022. 3

    2022

  36. [44]

    Data distillation: A survey

    Noveen Sachdeva and Julian McAuley. Data distillation: A survey. Transactions on Machine Learning Research, 2023. 1, 2

    2023

  37. [45]

    Laion-5b: An open large-scale dataset for train- ing next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for train- ing next generation image-text models. In NeurIPS, pages 2527...

    2022

  38. [46]

    Active learning for convolu- tional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for convolu- tional neural networks: A core-set approach. InInternational Conference on Learning Representations, 2018. 4

    2018

  39. [47]

    A fast knowledge distillation framework for visual recognition

    Zhiqiang Shen and Eric Xing. A fast knowledge distillation framework for visual recognition. In ECCV, pages 673–690. Springer, 2022. 2

    2022

  40. [48]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 4

    2010 arXiv

  41. [49]

    Dˆ 4: Dataset distillation via disentangled diffusion model

    Duo Su, Junjie Hou, Weizhi Gao, Yingjie Tian, and Bowen Tang. Dˆ 4: Dataset distillation via disentangled diffusion model. In CVPR, pages 5809–5818, 2024. 2, 3

    2024

  42. [50]

    On the diversity and realism of distilled dataset: An efficient dataset distilla- tion paradigm

    Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin. On the diversity and realism of distilled dataset: An efficient dataset distilla- tion paradigm. In CVPR, 2024. 2, 3, 5

    2024

  43. [51]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1

    2023 arXiv

  44. [52]

    Group distribution- ally robust dataset distillation with risk minimization

    Saeed Vahidian, Mingyu Wang, Jianyang Gu, Vyacheslav Kungurtsev, Wei Jiang, and Yiran Chen. Group distribution- ally robust dataset distillation with risk minimization. arXiv preprint arXiv:2402.04676, 2024. 2

    2024 arXiv

  45. [53]

    Dim: Distilling dataset into genera- tive model

    Kai Wang, Jianyang Gu, Daquan Zhou, Zheng Zhu, Wei Jiang, and Yang You. Dim: Distilling dataset into genera- tive model. arXiv preprint arXiv:2303.04707, 2023. 2

    2023 arXiv

  46. [54]

    Dataset distillation

    Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018. 1, 2

    2018 arXiv

  47. [55]

    Dancing with images: Video distillation via static-dynamic disentan- glement

    Ziyu Wang, Yue Xu, Cewu Lu, and Yong-Lu Li. Dancing with images: Video distillation via static-dynamic disentan- glement. arXiv preprint arXiv:2312.00362, 2023. 2

    2023 arXiv

  48. [56]

    Sparse parameterization for epitomic dataset distillation

    Xing Wei, Anjia Cao, Funing Yang, and Zhiheng Ma. Sparse parameterization for epitomic dataset distillation. In NeurIPS, 2024. 2

    2024

  49. [57]

    Herding dynamical weights to learn

    Max Welling. Herding dynamical weights to learn. In ICML, pages 1121–1128, 2009. 4

    2009

  50. [58]

    Understanding data augmentation for classification: when to warp? In DICTA, pages 1–6

    Sebastien C Wong, Adam Gatt, Victor Stamatescu, and Mark D McDonnell. Understanding data augmentation for classification: when to warp? In DICTA, pages 1–6. IEEE,

  51. [59]

    FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning

    Yuanhao Xiong, Ruochen Wang, Minhao Cheng, Felix Yu, and Cho-Jui Hsieh. FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning. In CVPR, pages 16323–16332, 2023. 2

    2023

  52. [60]

    Controlled training data generation with diffusion mod- els

    Teresa Yeo, Andrei Atanov, Harold Benoit, Aleksandr Alek- seev, Ruchira Ray, Pooya Esmaeil Akhoondi, and Amir Za- 10 mir. Controlled training data generation with diffusion mod- els. arXiv preprint arXiv:2403.15309, 2024. 2

    2024 arXiv

  53. [61]

    Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective

    Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective. In NeurIPS, pages 73582–73603, 2023. 5

    2023

  54. [62]

    Dataset dis- tillation: A comprehensive review

    Ruonan Yu, Songhua Liu, and Xinchao Wang. Dataset dis- tillation: A comprehensive review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 1, 2

    2023

  55. [63]

    Cutmix: Regu- larization strategy to train strong classifiers with localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features. In CVPR, pages 6023–6032, 2019. 2

    2019

  56. [64]

    Dataset condensation with distri- bution matching

    Bo Zhao and Hakan Bilen. Dataset condensation with distri- bution matching. In WACV, pages 6514–6523, 2023. 2

    2023

  57. [65]

    Dataset condensation with gradient matching

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. In ICLR, 2021. 1, 2

    2021

  58. [66]

    Im- proved Distribution Matching for Dataset Condensation

    Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Im- proved Distribution Matching for Dataset Condensation. In CVPR, pages 7856–7865, 2023. 2, 4

    2023

  59. [67]

    Dataset quantization

    Daquan Zhou, Kai Wang, Jianyang Gu, Xiangyu Peng, Dongze Lian, Yifan Zhang, Yang You, and Jiashi Feng. Dataset quantization. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 17205– 17216, 2023. 2, 3

    2023

  60. [68]

    a photo of a nice {$class name}

    Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression. In NeurIPS, pages 9813–9827, 2022. 2 11 CONCORD : Concept-Informed Diffusion for Dataset Distillation Supplementary Material The appendix is organized into the following sectio...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.