Data Pruning in Generative Diffusion Models

Jiangtao Wang; Rania Briq; Stefan Kesselheim

REVIEW 4 major objections 5 minor 65 references

Data Pruning in Generative Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Diffusion models tolerate losing 90% of their ImageNet training data, and a simple clustering rule turns the pruned set into a better generator than the full dataset.

desk verdict The paper's broad pruning survey is useful, but its central 90%-tolerance claim is confounded by fixed-step training that gives pruned models ~10x more epochs per sample. read the letter →

arxiv 2411.12523 v3 pith:ZKKQZM2W submitted 2024-11-19 cs.LG cs.CV

Rania Briq , Jiangtao Wang , Stefan Kesselheim This is my paper · ORCID

classification cs.LGcs.CV

keywords datapruningdiffusionmodelsgenerativecoresetselectionembeddingclusteringdatasetbalancingImageNetCelebA-HQ

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether data pruning, discarding part of the training set, can help or hurt generative diffusion models, which must model a whole distribution rather than a decision boundary. It argues that pruning is not only possible but often beneficial: on ImageNet, removing up to 90% of the data leaves FID unchanged, and a simple clustering rule that keeps samples closest to cluster centers in a pretrained model's embedding space beats both the unpruned model and more expensive gradient-based pruning methods. On the smaller CelebA-HQ dataset, random selection is competitive, and the authors show how per-cluster balanced sampling can rebalance a skewed distribution so underrepresented groups are generated reliably. The significance is practical: if correct, large diffusion models can be trained on a fraction of the data, and the same mechanism offers an unsupervised handle on fairness in generation.

What carries the argument

The central device is embedding-space clustering as a data-selection rule. A pretrained visual model extracts a feature vector for each image; k-means clusters those embeddings; and the pruning rule keeps, from each cluster, the samples closest to the cluster center (or, in the ablations, furthest or in-between). The nearest-to-center samples are treated as typical representatives of each mode, so aggregating them covers the distribution's dense regions. The other load-bearing component is the latent-space setup: a VQ-VAE trained on the full unpruned dataset encodes images into a compact latent space, and the diffusion model is trained in that space with a flow-matching objective, allowing the autoencoder to reinterpret the model's output even after most training samples are gone.

What would settle it

Train the VQ-VAE on exactly the pruned subset (or on a disjoint half of the data) and repeat the ImageNet 90% pruning run; if the FID advantage over the unpruned baseline disappears or turns negative, the reported tolerance is an artifact of the full-data autoencoder rather than a property of diffusion training.

Watch

Extended reading notes

Core claim

Diffusion models have a high tolerance threshold for pruning. On ImageNet, pruning 90% of the training data causes no decline in FID, and selecting samples nearest the centroids of clusters formed in CLIP or DINO embedding space yields better FID than training on the full dataset. On CelebA-HQ, no pruning strategy reliably beats random sampling, indicating that for smaller, more homogeneous datasets the main requirement is coverage of the distribution. The authors also show that taking an equal number of samples from each cluster, rather than a number proportional to cluster size, produces generated data with more balanced group representations, at little or no FID cost on ImageNet.

Load-bearing premise

The autoencoder that defines the latent space is trained on the full unpruned dataset, so information from pruned-away samples may still be encoded and available to the diffusion model.

Editorial extensions

If this is right

On ImageNet-scale data, around 90% of training samples can be discarded for latent diffusion training, so compute, storage, and energy budgets could shrink by roughly an order of magnitude without hurting FID.
A frozen pretrained embedding backbone plus k-means is a cheaper and more effective pruning recipe than gradient-norm-, loss-, or forgetting-based scoring on large diverse datasets.
Balanced per-cluster sampling can steer a diffusion model toward underrepresented groups in an unsupervised way, with a small FID penalty on skewed small datasets and almost none on ImageNet.
For small datasets such as CelebA-HQ, random pruning is the practical choice at high pruning ratios, because expensive scoring strategies do not beat it and can even hurt.
Diffusion models' step-by-step denoising makes them more resilient to data removal than single-pass generative models such as GANs and VAEs, which degrade immediately on pruned data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable corollary is that the tolerance should persist in pixel-space diffusion models if the mechanism is iterative denoising; if it only appears with a full-data autoencoder, the latent space, not denoising, is what preserves quality.
The nearest-to-centroid rule can be viewed as an ordering of samples by typicality; this suggests data-scaling laws for generative models might be expressible in terms of effective cluster mass rather than raw sample count.
The success of CLIP/DINO-based clustering likely depends on how well the backbone's embeddings match the target distribution's modes; on out-of-distribution domains the recipe may need a backbone fine-tuned on that domain.
A practical workflow emerges from the paper's ablations: embed with a frozen foundation model, cluster, keep centroid-near samples for dense coverage, and switch to equal-per-cluster sampling when demographic balance matters more than raw FID.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know about this paper. It's the most thorough empirical comparison of data pruning methods for diffusion models I've seen, and the inverse-selection ablations are a genuinely good design choice. But the headline claim—that ImageNet diffusion models tolerate 90% pruning without FID loss—is confounded by the training schedule. They run a fixed 120k iterations for all conditions. At batch 256 on 1.2M images, that's ~26 epochs for the unpruned model. At PR=0.9, the same 120k steps give the pruned model ~256 epochs, ten times more updates per image. So the pruned model isn't just seeing less data; it's seeing the retained data many more times. The unpruned baseline is undertrained, and the comparison doesn't isolate data redundancy from extra per-sample training. This isn't a minor detail; it's the load-bearing result of Sections 1 and 4.2.

What's actually new: applying pruning to generative diffusion models is relatively underexplored, and the systematic comparison of gradient-, loss-, and cluster-based methods at two scales is useful. The inverse-selection experiments are a smart sanity check—e.g., GraNd's inverse degrading as expected gives some confidence in the method. The balanced-cluster experiment for fair sampling is a nice side contribution. The finding that simple clustering in CLIP/DINO space outperforms expensive gradient methods is plausible and worth taking seriously.

The soft spots beyond the training-budget confound: single runs, no error bars or multiple seeds anywhere, which matters for FID differences of ~2–5. The VQ-VAE trained on the full dataset is a real confound, and the authors acknowledge it in Section 4.3 but don't control for it—no experiment retrains the autoencoder on pruned data. The clustering uses k=1000 on ImageNet, equal to the number of classes; that choice may leak label information, and they don't test sensitivity to k.

Who's this for: researchers working on data-efficient generative training, especially those deciding whether to invest in pruning strategies. They'll find a broad map of what works and what doesn't, but they should treat the quantitative tolerance claims as upper bounds until fixed-epoch baselines are added. I'd send it to a serious referee—the topic is timely, the experiments are extensive, and the confounds are addressable. The authors should be asked to rerun or analyze fixed-epoch conditions and to report variance across seeds. If they can show the tolerance survives those controls, it becomes a strong paper.

Referee Report

4 major / 5 minor

Summary. This paper studies data pruning for generative diffusion models. The authors implement several pruning criteria (random, loss monotonicity, GraNd, EL2N, MoSo, and clustering in CLIP/DINO embedding spaces) and evaluate them on CelebA-HQ and ImageNet using a DiT/SiT flow-matching diffusion model operating on a VQ-VAE latent space. They also run inverse-selection ablations and a balanced-sampling variant of clustering. The main claims are that diffusion models tolerate pruning up to 90% on ImageNet without FID degradation, that cluster-center selection in a large visual model's embedding space outperforms the unpruned model, and that balanced clustering can mitigate skewed distributions.

Significance. The paper addresses a timely and practically important question and assembles a broad set of comparisons; the inverse-selection ablations are a thoughtful way to test whether a scoring criterion carries signal, and the code release is a strength. If the headline results were established, the finding that simple CLIP/DINO clustering beats more expensive gradient-based methods on ImageNet would be a useful practical result. However, the central quantitative claims are currently undermined by a training-budget confound and a full-data VQ-VAE confound, and they rest on single-run evaluations. The balanced-sampling application is interesting but also currently supported mainly by qualitative evidence.

major comments (4)

[Section 4, Training] The ImageNet comparison fixes the number of optimizer steps (120k iterations, batch size 256) rather than the number of epochs. The unpruned model is trained for 120k / (1.2M / 256) ≈ 25.6 epochs, while a PR=0.9 model is trained for 120k / (120k / 256) ≈ 256 epochs, so each retained sample is presented to the optimizer roughly ten times more often. This unequal training effort per sample can explain the Section 4.2 result that PR=0.9 matches or improves on the unpruned baseline without implying that the discarded 90% is redundant. Please report results with equal epoch budgets, or with multiple compute budgets, or explicitly reframe the claim as a fixed-step result rather than a data-redundancy result.
[Section 4.3] The VQ-VAE is trained on the full unpruned dataset, and the paper itself identifies the autoencoder as 'another pivotal factor in preserving the performance.' Because the latent encoder/decoder has seen all images, information from pruned-out samples may be encoded in the latent geometry, so the experiment does not cleanly isolate the diffusion model's tolerance to data pruning. Please add an ablation with a VQ-VAE trained only on the pruned subset (or another control that removes the full-data representation confound) and qualify the headline claim accordingly.
[Sections 4.1-4.2] All FID, F-score, Inception, and Vendi results appear to be single runs with no error bars or confidence intervals. The headline comparison (e.g., 'at PR=0.9 the gap between unpruned and ClusterC is approximately 5') and the relative ordering of ClusterC/D versus random rest on differences that could be within run-to-run noise. Please report means and standard deviations over at least three seeds, and state the number of seeds used.
[Section 4.3, Comparison with other generative models] The text asserts that a VAE and a GAN degrade 'instantaneously' when trained on a randomly pruned subset, but no quantitative results, figure, or experimental details are given. This claim is load-bearing for the conclusion that DMs are unique in their pruning tolerance, so it needs to be either reported with evidence or removed.

minor comments (5)

[Throughout] Numerous typos and grammatical errors (e.g., 'compehensive', 'monotoncity', 'otpimization', 'mdeosl', 'instantateous') make the text harder to read; a careful proofread is needed.
[Section 3.2 and Section 4] The pruning ratio is never formally defined; please state explicitly that PR denotes the fraction of data removed (so PR=0.9 leaves 10% of the data).
[Section 3.2] The notation for clustering variants is inconsistent ('Cluster backbone', 'Cluster mid backbone', 'Cluster −1 backbone' vs 'ClusterC/D' vs table entries 'clusterC'/'clusterD'); unify the notation.
[Section 4] The choice of k (24 for CelebA-HQ, 1000 for ImageNet) is reasonable but sensitivity to k is not reported; since clustering performance may depend on this hyperparameter, a small sensitivity analysis would strengthen the claims.
[Table 1] The column layout is crowded and values run together (e.g., '7.74345.5829'), making the table difficult to read; fix the formatting.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the pruning-tolerance claims are empirical comparisons against external benchmarks, with no fitted-to-target parameters, no self-citation chain, and no definitional reduction.

full rationale

The paper's central claims (pruning 90% of ImageNet without FID decline; clustering in CLIP/DINO embedding space outperforming unpruned training) are empirical results evaluated with external metrics (FID, F-score, Vendi, Inception) and external pretrained backbones. There is no mathematical derivation chain whose conclusion is equivalent to an input by construction. No parameter is fitted to the target FID values, and the pruning scores are defined independently of the evaluation metrics. The full-data VQ-VAE is a real confound and is explicitly acknowledged in Section 4.2 ('Having been trained on a more comprehesive dataset, the autoencoder can reinterpret the DM's output in a rich and representative latent space'), but an acknowledged confound is not a circular step: it does not make the empirical outcome equal to its own premise. The fixed 120k-iteration training budget is a methodological fairness concern (pruned models receive more epochs than the unpruned baseline), not a circularity, because the reported FID values are not derived from the training-budget assumption. The paper contains no self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation. Accordingly, the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities or fitted physical constants. Its claims rest on external embedding backbones, standard evaluation metrics, and a full-data autoencoder whose role is a potential confound.

free parameters (1)

Number of clusters k = 24 for CelebA-HQ, 1000 for ImageNet
k is chosen via inertia analysis on CelebA-HQ and set equal to the number of ImageNet classes. Subset composition depends on k, and no sensitivity analysis is provided.

assumptions (3)

domain assumption CLIP and DINO embeddings group semantically similar images, so cluster centers represent dense regions of the true distribution.
Clustering-based pruning depends on this assumption to select representative samples; invoked in Section 3.2.
domain assumption FID, F-score, and Vendi scores reliably reflect generation quality at high pruning ratios.
All conclusions are drawn from these metrics without error bars or a demonstration that they remain unbiased under heavy pruning; used throughout Section 4.
ad hoc to paper A VQ-VAE trained on the full dataset does not invalidate the pruning comparison.
The autoencoder is trained on all data and is credited with preserving performance; this pipeline-specific assumption blends pruning effects with representation learning. Acknowledged in Section 4.2.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Data Pruning in Generative Diffusion Models." pith.science (2026). https://pith.science/paper/ZKKQZM2W

@misc{pith2026241112523,
  author       = {Pith},
  title        = {Pith review of: Data Pruning in Generative Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZKKQZM2W}},
  note         = {Machine review of arXiv:2411.12523}
}

read the original abstract

Data pruning is the problem of identifying a core subset that is most beneficial to training and discarding the remainder. While pruning strategies are well studied for discriminative models like those used in classification, little research has gone into their application to generative models. Generative models aim to estimate the underlying distribution of the data, so presumably they should benefit from larger datasets. In this work we aim to shed light on the accuracy of this statement, specifically answer the question of whether data pruning for generative diffusion models could have a positive impact. Contrary to intuition, we show that eliminating redundant or noisy data in large datasets is beneficial particularly when done strategically. We experiment with several pruning methods including recent-state-of-art methods, and evaluate over CelebA-HQ and ImageNet datasets. We demonstrate that a simple clustering method outperforms other sophisticated and computationally demanding methods. We further exhibit how we can leverage clustering to balance skewed datasets in an unsupervised manner to allow fair sampling for underrepresented populations in the data distribution, which is a crucial problem in generative models.

Figures

Figures reproduced from arXiv: 2411.12523 by the authors.

**Figure 3.** Qualitative results that display samples [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

**Figure 4.** Qualitative results showing the clean samples using clustering-based pruning for PR=0.5. tial noise vector, the model frequently (but not always) generates a similar image. In the case of human faces, we can regognize the identity of the face is the same despite slight differences in appearance. This happens even when the model is trained on two disjoint datasets, which happens when we train a model using a prunin… view at source ↗

Figures from the paper (1 more)

**Figure 6.** Figure 6: Qualitative results showing the clean samples of MoSo and GraNd and their inverse comprehesive dataset, the autoencoder can reinterpret the DM’s output in a rich and representative latent space. Balancing skewed distributions. In table 2 we vary the proportion of samp…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 40 canonical work pages

[1]

Effective pruning of web-scale datasets based on complexity of concept clusters

Amro Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, and Ari S Morcos. Effective pruning of web-scale datasets based on complexity of concept clusters. arXiv preprint arXiv:2401.04578, 2024. 2, 3

arXiv 2024
[2]

Dall- eval: Probing the reasoning skills and social biases of text-to-image generation models

Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall- eval: Probing the reasoning skills and social biases of text-to-image generation models. In Proceedings of the IEEE/CVF International Conference on Com- puter Vision , pages 3043–3054, 2023. 2

work page 2023
[3]

Selection via proxy: Efficient data selection for deep learning

Cody Coleman, Christopher Yeh, Stephen Muss- mann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data selection for deep learning. arXiv preprint arXiv:1906.11829 , 2019. 2

arXiv 1906
[4]

Diffusion models beat gans on image synthesis

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems , 34:8780– 8794, 2021. 1

work page 2021
[5]

Ethi- cal considerations and policy interventions concern- ing the impact of generative ai tools in the economy and in society

Mirko Farina, Xiao Yu, and Andrea Lavazza. Ethi- cal considerations and policy interventions concern- ing the impact of generative ai tools in the economy and in society. AI and Ethics , pages 1–9, 2024. 2

work page 2024
[6]

What neural networks memorize and why: Discovering the long tail via influence estimation

Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural In- formation Processing Systems , 33:2881–2891, 2020. 2, 3

work page 2020
[7]

Sparsegpt: Mas- sive language models can be accurately pruned in one-shot

Elias Frantar and Dan Alistarh. Sparsegpt: Mas- sive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323–10337. PMLR, 2023. 1

work page 2023
[8]

The vendi score: A diversity evaluation metric for machine learning

Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. arXiv preprint arXiv:2210.02410 , 2022. 5

arXiv 2022

Show all 65 references

[9]

Generative adversar- ial nets

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversar- ial nets. Advances in neural information processing systems, 27, 2014. 1

2014
[10]

Data and parameter scaling laws for neural machine translation

Mitchell A Gordon, Kevin Duh, and Jared Kaplan. Data and parameter scaling laws for neural machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 5915–5922, 2021. 2

2021
[11]

Im- proved training of wasserstein gans

Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. Advances in neural information processing systems , 30, 2017. 1

2017
[12]

Safety and fairness for content moderation in genera- tive models

Susan Hao, Piyush Kumar, Sarah Laszlo, Shivani Poddar, Bhaktipriya Radharapu, and Renee Shelby. Safety and fairness for content moderation in genera- tive models. arXiv preprint arXiv:2306.06135 , 2023. 2

2023 arXiv
[13]

Smaller coresets for k-median and k-means clustering

Sariel Har-Peled and Akash Kushal. Smaller coresets for k-median and k-means clustering. In Proceedings of the twenty-first annual symposium on Computa- tional geometry, pages 126–134, 2005. 2

2005
[14]

Large-scale dataset pruning with dynamic un- certainty

Muyang He, Shuo Yang, Tiejun Huang, and Bo Zhao. Large-scale dataset pruning with dynamic un- certainty. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 7713–7722, 2024. 2, 3

2024
[15]

Imagen video: High definition video generation with diffusion models

Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022. 1

arXiv
[16]

Denois- ing diffusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denois- ing diffusion probabilistic models. Advances in neu- ral information processing systems , 33:6840–6851,
[17]

Training compute- optimal large language models

Jordan Hoffmann, Sebastian Borgeaud, Arthur Men- sch, Elena Buchatskaya, Trevor Cai, Eliza Ruther- ford, Diego de Las Casas, Lisa Anne Hendricks, Jo- hannes Welbl, Aidan Clark, et al. Training compute- optimal large language models. arxiv. arXiv preprint arXiv:2203.15556, 2022. 2

2022 arXiv
[18]

Data distribution search to select core-set for machine learning

Myunggwon Hwang, Yuna Jeong, and Wonkyung Sung. Data distribution search to select core-set for machine learning. In The 9th International Confer- ence on Smart Media and Applications , pages 172– 176, 2020. 2

2020
[19]

Scaling laws for neural language models

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. 2

2001 arXiv
[20]

Prism: A unified framework of parameterized submodu- lar information measures for targeted data sub- set selection and summarization

Vishal Kaushal, Suraj Kothawade, Ganesh Ramakr- ishnan, Jeff Bilmes, and Rishabh Iyer. Prism: A unified framework of parameterized submodu- lar information measures for targeted data sub- set selection and summarization. arXiv preprint arXiv:2103.00128, 2021. 2

2021 arXiv
[21]

Denoising diffusion restoration mod- 13 els

Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration mod- 13 els. In Advances in Neural Information Processing Systems, 2022. 1

2022
[22]

Grad-match: Gradient matching based data subset selection for efficient deep model training

Krishnateja Killamsetty, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training. In In- ternational Conference on Machine Learning , pages 5464–5474. PMLR, 2021. 2

2021
[23]

Harmful biases in artificial intelli- gence

Morgan King. Harmful biases in artificial intelli- gence. The Lancet Psychiatry , 9(11):e48, 2022. 2

2022
[24]

Auto-encoding variational bayes

Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 , 2013. 1

2013 arXiv
[25]

Videopoet: A large language model for zero- shot video generation, 2024

Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos´ e Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero- shot video generation, 2024. URL https://arxiv. org/abs/2312.14125. 1

2024 arXiv
[26]

Improved precision and recall metric for assessing generative models.Ad- vances in neural information processing systems , 32,

Tuomas Kynk¨ a¨ anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models.Ad- vances in neural information processing systems , 32,
[27]

Holistic evaluation of text-to-image models

Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yi- fan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, et al. Holistic evaluation of text-to-image models. Advances in Neural Information Processing Systems, 36, 2024. 2

2024
[28]

Diffusion models for image restoration and enhancement–a comprehensive survey

Xin Li, Yulin Ren, Xin Jin, Cuiling Lan, Xin- grui Wang, Wenjun Zeng, Xinchao Wang, and Zhibo Chen. Diffusion models for image restoration and enhancement–a comprehensive survey. arXiv preprint arXiv:2308.09388, 2023. 1

2023
[29]

Flow match- ing for generative modeling

Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow match- ing for generative modeling. arXiv preprint arXiv:2210.02747, 2022. 5

2022 arXiv
[30]

Sit: Exploring flow and diffusion-based gener- ative models with scalable interpolant transformers

Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based gener- ative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740 , 2024. 5

2024 arXiv
[31]

Analyz- ing quality, bias, and performance in text-to-image generative models

Nila Masrourisaadat, Nazanin Sedaghatkish, Fate- meh Sarshartehrani, and Edward A Fox. Analyz- ing quality, bias, and performance in text-to-image generative models. arXiv preprint arXiv:2407.00138,

arXiv
[32]

A non-parametric test to detect data- copying in generative models

Casey Meehan, Kamalika Chaudhuri, and Sanjoy Dasgupta. A non-parametric test to detect data- copying in generative models. In International Con- ference on Artificial Intelligence and Statistics, 2020. 5

2020
[33]

Rdcgan: Un- supervised representation learning with regularized deep convolutional generative adversarial networks

Mehran Mehralian and Babak Karasfi. Rdcgan: Un- supervised representation learning with regularized deep convolutional generative adversarial networks. In 2018 9th conference on artificial intelligence and robotics and 2nd Asia-pacific international sympo- sium, pages 31–38. I...

2018
[34]

Coresets for data-efficient training of ma- chine learning models

Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of ma- chine learning models. In International Confer- ence on Machine Learning, pages 6950–6960. PMLR,
[35]

Geometry- complete diffusion for 3d molecule generation and optimization

Alex Morehead and Jianlin Cheng. Geometry- complete diffusion for 3d molecule generation and optimization. Communications Chemistry, 7(1):150,
[36]

Diffusion models, image super- resolution, and everything: A survey

Brian B Moser, Arundhati S Shanbhag, Federico Raue, Stanislav Frolov, Sebastian Palacio, and An- dreas Dengel. Diffusion models, image super- resolution, and everything: A survey. IEEE Trans- actions on Neural Networks and Learning Systems ,
[37]

Maxime Oquab, Timoth´ ee Darcet, Theo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Fran- cisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicol...
[38]

Deep learning on a data diet: Finding im- portant examples early in training

Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding im- portant examples early in training. Advances in neu- ral information processing systems , 34:20596–20607,
[39]

Scalable dif- fusion models with transformers

William Peebles and Saining Xie. Scalable dif- fusion models with transformers. arXiv preprint arXiv:2212.09748, 2022. 5

2022 arXiv
[40]

Learning transferable visual models 14 from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models 14 from natural language supervision. In International conference on machine learning ...

2021
[41]

Neural synthesis of binaural speech from mono audio

Alexander Richard, Dejan Markovic, Israel D Ge- bru, Steven Krenn, Gladstone Alexander Butler, Fer- nando Torre, and Yaser Sheikh. Neural synthesis of binaural speech from mono audio. In International Conference on Learning Representations, 2021. 1

2021
[42]

High- resolution image synthesis with latent diffusion mod- els, 2021

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨ orn Ommer. High- resolution image synthesis with latent diffusion mod- els, 2021. 1

2021
[43]

Assessing gen- erative models via precision and recall

Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing gen- erative models via precision and recall. Advances in neural information processing systems , 31, 2018. 5

2018
[44]

What matters in the structured prun- ing of generative language models? arXiv preprint arXiv:2302.03773, 2023

Michael Santacroce, Zixin Wen, Yelong Shen, and Yuanzhi Li. What matters in the structured prun- ing of generative language models? arXiv preprint arXiv:2302.03773, 2023. 1

2023 arXiv
[45]

Generative model- ing by estimating gradients of the data distribution

Yang Song and Stefano Ermon. Generative model- ing by estimating gradients of the data distribution. Advances in neural information processing systems , 32, 2019. 1

2019
[46]

Solving inverse problems in medical imaging with score-based generative models

Yang Song, Liyue Shen, Lei Xing, and Stefano Er- mon. Solving inverse problems in medical imaging with score-based generative models. arXiv preprint arXiv:2111.08005, 2021. 1

2021 arXiv
[47]

Score-based generative modeling through stochastic differential equations

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020. 1

2011 arXiv
[48]

Beyond neural scal- ing laws: beating power law scaling via data pruning

Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scal- ing laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35:19523–19536, 2022. 3

2022
[49]

A simple and effective pruning ap- proach for large language models

Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning ap- proach for large language models. arXiv preprint arXiv:2306.11695, 2023. 1

2023 arXiv
[50]

Autore- gressive model beats diffusion: Llama for scalable image generation

Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525 ,

arXiv
[51]

Data pruning via moving-one-sample-out

Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moving-one-sample-out. Advances in Neural In- formation Processing Systems, 36, 2024. 2, 3, 4

2024
[52]

Struc- tured pruning for efficient generative pre-trained lan- guage models

Chaofan Tao, Lu Hou, Haoli Bai, Jiansheng Wei, Xin Jiang, Qun Liu, Ping Luo, and Ngai Wong. Struc- tured pruning for efficient generative pre-trained lan- guage models. In Findings of the Association for Computational Linguistics: ACL 2023 , pages 10880– 10895, 2023. 1

2023
[53]

On measuring fairness in gener- ative models

Christopher Teo, Milad Abdollahzadeh, and Ngai- Man Man Cheung. On measuring fairness in gener- ative models. Advances in Neural Information Pro- cessing Systems, 36, 2024. 2

2024
[54]

Fair generative models via trans- fer learning

Christopher TH Teo, Milad Abdollahzadeh, and Ngai-Man Cheung. Fair generative models via trans- fer learning. In Proceedings of the AAAI conference on artificial intelligence , volume 37, pages 2429– 2437, 2023. 2

2023
[55]

An empirical study of exam- ple forgetting during deep neural network learning

Mariya Toneva, Alessandro Sordoni, Remi Ta- chet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of exam- ple forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159 , 2018. 2, 4

2018 arXiv
[56]

Neural discrete representation learning

Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017. 5

2017
[57]

Diffusion models for medical image reconstruction

George Webber and Andrew J Reader. Diffusion models for medical image reconstruction. BJR— Ar- tificial Intelligence , page ubae013, 2024. 1

2024
[58]

Moderate coreset: A universal method of data selection for real-world data-efficient deep learning

Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning. In The Eleventh International Con- ference on Learning Representations, 2022. 2, 3

2022
[59]

Geodiff: A geometric diffusion model for molecular conformation genera- tion

Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Ste- fano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation genera- tion. arXiv preprint arXiv:2203.02923 , 2022. 1

2022 arXiv
[60]

Mind the boundary: Coreset selection via reconstructing the decision boundary

Shuo Yang, Zhe Cao, Sheng Guo, Ruiheng Zhang, Ping Luo, Shengping Zhang, and Liqiang Nie. Mind the boundary: Coreset selection via reconstructing the decision boundary. In Forty-first International Conference on Machine Learning , 2024. 2, 3

2024
[61]

Dataset pruning: Reducing training data by examining generalization influence

Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Ming- ming Sun, and Ping Li. Dataset pruning: Reducing training data by examining generalization influence. arXiv preprint arXiv:2205.09329 , 2022. 2

2022 arXiv
[62]

A generalized dual-domain gen- erative framework with hierarchical consistency for 15 medical image reconstruction and synthesis

Jiadong Zhang, Kaicong Sun, Junwei Yang, Yan Hu, Yuning Gu, Zhiming Cui, Xiaopeng Zong, Fei Gao, and Dinggang Shen. A generalized dual-domain gen- erative framework with hierarchical consistency for 15 medical image reconstruction and synthesis. Com- munications Engineering, 2...

2023
[63]

The unreasonable ef- fectiveness of deep features as a perceptual metric

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable ef- fectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. 2

2018
[64]

Energy-efficient high-fidelity image reconstruction with memristor arrays for medical di- agnosis

Han Zhao, Zhengwu Liu, Jianshi Tang, Bin Gao, Qi Qin, Jiaming Li, Ying Zhou, Peng Yao, Yue Xi, Yu- deng Lin, et al. Energy-efficient high-fidelity image reconstruction with memristor arrays for medical di- agnosis. Nature Communications, 14(1):2276, 2023. 1

2023
[65]

Coverage-centric coreset selection for high pruning rates

Haizhong Zheng, Rui Liu, Fan Lai, and Atul Prakash. Coverage-centric coreset selection for high pruning rates. arXiv preprint arXiv:2210.15809 ,

arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Effective pruning of web-scale datasets based on complexity of concept clusters

Amro Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, and Ari S Morcos. Effective pruning of web-scale datasets based on complexity of concept clusters. arXiv preprint arXiv:2401.04578, 2024. 2, 3

arXiv 2024

[2] [2]

Dall- eval: Probing the reasoning skills and social biases of text-to-image generation models

Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall- eval: Probing the reasoning skills and social biases of text-to-image generation models. In Proceedings of the IEEE/CVF International Conference on Com- puter Vision , pages 3043–3054, 2023. 2

work page 2023

[3] [3]

Selection via proxy: Efficient data selection for deep learning

Cody Coleman, Christopher Yeh, Stephen Muss- mann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data selection for deep learning. arXiv preprint arXiv:1906.11829 , 2019. 2

arXiv 1906

[4] [4]

Diffusion models beat gans on image synthesis

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems , 34:8780– 8794, 2021. 1

work page 2021

[5] [5]

Ethi- cal considerations and policy interventions concern- ing the impact of generative ai tools in the economy and in society

Mirko Farina, Xiao Yu, and Andrea Lavazza. Ethi- cal considerations and policy interventions concern- ing the impact of generative ai tools in the economy and in society. AI and Ethics , pages 1–9, 2024. 2

work page 2024

[6] [6]

What neural networks memorize and why: Discovering the long tail via influence estimation

Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural In- formation Processing Systems , 33:2881–2891, 2020. 2, 3

work page 2020

[7] [7]

Sparsegpt: Mas- sive language models can be accurately pruned in one-shot

Elias Frantar and Dan Alistarh. Sparsegpt: Mas- sive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323–10337. PMLR, 2023. 1

work page 2023

[8] [8]

The vendi score: A diversity evaluation metric for machine learning

Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. arXiv preprint arXiv:2210.02410 , 2022. 5

arXiv 2022