Pith. sign in

REVIEW 5 major objections 6 minor 67 references

Image Generation Diversity Issues and How to Tame Them

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims diversity in generative models is measurable through image retrieval, shows current diffusion models top out at 77% of training-data diversity, and presents DiADM to close the gap without quality loss.

desk verdict A useful retrieval-based diversity metric with a serious normalization problem; the headline numbers are not yet trustworthy, but the paper deserves a serious referee. read the letter →

arxiv 2411.16171 v2 pith:7R2FHD5S submitted 2024-11-25 cs.CV

Mischa Dombrowski , Weitong Zhang , Sarah Cechnicka , Hadrien Reynaud , Bernhard Kainz This is my paper · ORCID
classification cs.CV
keywords imagediversitydiffusionmodelsRetrievalScoregenerativemodelevaluationcouponcollectorproblemmodecollapsefeatureextractorspseudo-unconditionalgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the diversity of a generative model is a tractable, measurable quantity: how many distinct training images the model's outputs can stand in for. It packages this as the Image Retrieval Score (IRS), which counts the fraction of training images that are the nearest feature-space neighbor of at least one synthetic sample, with confidence intervals derived from the coupon collector problem. Using IRS, the paper argues that current feature extractors used in generative evaluation badly underestimate diversity (the 'measurement gap'), and that state-of-the-art diffusion models reach at most 77% of training-data diversity on ImageNet. It then introduces Diversity-Aware Diffusion Models (DiADM), which feed precomputed image features into an unconditional diffusion model as pseudo-labels, and reports that this increases IRS substantially on several datasets while also improving FID. A sympathetic reader would care because if true, diversity becomes as easy to measure and optimize as quality, rather than being hidden behind visually plausible samples.

What carries the argument

The load-bearing object is the Image Retrieval Score (IRS), defined in Eq. (4) as $N_{\mathrm{learned}}/N_{\mathrm{train}}$, where a training image counts as learned if it is the nearest neighbor, in a pretrained feature space, of at least one synthetic image (Eq. (3)). The score is read through the coupon collector problem: drawing synthetic samples is modeled as drawing with replacement from the $N_{\mathrm{train}}$ training images, so the probability of observing exactly $k$ distinct retrieved images is $P(k,n,s)=\mathrm{Stir}(n,k)\,s!/(s-k)!\,s^{-n}$, evaluated in log space with Temme's asymptotic Stirling estimate to yield $\mathrm{IRS}_\infty$ and confidence bounds for infinite sampling. A normalization step divides synthetic IRS by real-data IRS to cancel the feature extractor's 'measurement gap'. The diversity-improving half of the paper is DiADM, which replaces unconditional placeholder labels with precomputed image features (pseudo-unconditional conditioning) so that each training instance behaves as its own class, disentangling fidelity from coverage.

What would settle it

Take a model trained on a dataset where 50% of training images are exact duplicates or where a held-out human can identify each synthetic image's source; if the adjusted IRS of such a deliberately half-collapsed model does not approach 50%, or if the same model's adjusted IRS changes rank when SwAV is replaced by DINOv2, the central claim that IRS measures diversity rather than extractor behavior is falsified.

Watch

Extended reading notes

Core claim

The central discovery claim is that diversity can be defined operationally as retrievability: a real training image is 'learned' when at least one synthetic image is closer to it, in a chosen feature space, than to any other training image; the model's diversity is the fraction of the training set that is learned, extrapolated to infinite sampling via a Stirling-number occupancy model. The paper asserts that every current feature extractor collapses the real data distribution, so a fair score must normalize synthetic diversity by the diversity measured between real and real reference data. On that basis it reports that no current state-of-the-art diffusion model surpasses 77% of training-data diversity, and that its DiADM module, which uses Inception features as pseudo-unconditional conditioning, raises this ceiling on ImageNet, FFHQ, ChestX-ray14, CelebV-HQ, and Dynamic while simultaneously improving FID.

Load-bearing premise

Each synthetic image is assumed to have exactly one 'main component' training image, that component is the nearest neighbor in a pretrained feature space, and dividing synthetic by real IRS fully cancels the feature space's measurement gap; if any of these fails, IRS measures feature-space uniqueness rather than true distributional diversity.

Editorial extensions

If this is right

  • With IRS, diversity can be monitored with far fewer samples than FID needs, and checkpoints can be rejected early when observed duplicates make low diversity statistically certain.
  • Because common feature extractors collapse real-data diversity, existing precision, recall, and coverage numbers understate the diversity problem; diversity claims should be re-checked on feature spaces that retrieve real images well.
  • If no model surpasses 77% diversity, then conditional guidance and scaling alone have not solved mode coverage, and diversity needs an explicit training signal.
  • DiADM's pseudo-unconditional features give unconditional diffusion models a way to be queried for diverse outputs without label supervision, and in the paper's tables the added diversity comes with lower FID, not higher.
  • Applied to text-to-image models with a balanced reference set, IRS turns fairness questions like gender bias into a measurable diversity deficit of roughly 50%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If IRS is adopted, model rankings by diversity will become extractor- and reference-set-dependent; the paper's 77% ceiling should be read as a property of the chosen SwAV feature space and ImageNet reference, not a universal constant.
  • DiADM's design suggests a general template: any conditional model could be steered by features rather than labels, but when the features encode identity, the method may trade diversity for memorization unless a retrieval-based guardrail is applied.
  • The real-vs-real normalization assumes the measurement gap affects synthetic and real images identically; a testable implication is that a feature extractor that collapses synthetic features more than real ones would inflate adjusted IRS, so practitioners should validate the ratio on datasets with known diversity.
  • Coupling IRS to fairness framing suggests diversity metrics could be used as bias audits, a direction the paper only sketches.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes the Image Retrieval Score (IRS), a diversity metric for generative models that counts how many training images are retrieved by synthetic queries in a pretrained feature space, and derives an estimator from the coupon collector problem with confidence intervals. Using IRS, the authors argue that current feature extractors are inadequate for diversity evaluation, report that state-of-the-art diffusion models on ImageNet do not exceed 77% of training-set diversity, and introduce Diversity-Aware Diffusion Models (DiADM), which conditions unconditional diffusion models on per-image features to improve diversity without sacrificing FID. An open-source Python package is provided.

Significance. If the metric is valid, IRS offers a more interpretable, statistically grounded diversity score than existing metrics, and the claim that current diffusion models cover only about three-quarters of the training distribution is an important empirical finding. The class-removal sanity check in Fig. 5 is a genuinely useful validation experiment, and the provided package and reproducible feature-extraction pipeline are strengths. However, the adjusted IRS can exceed 1, the confidence-interval formulas appear inconsistent, the metric depends on the chosen feature extractor, and the DiADM evaluation is confounded by the use of Inception features both for conditioning and for FID computation. These issues currently prevent the headline claims from being accepted as stated.

major comments (5)
  1. [Eq. (2), Sec. 3.1] Equation (2) is mathematically inconsistent with the summation in Eq. (1). The sum of k terms each equal to 1 - ((Ntrain-1)/Ntrain)^n should be k times that value; as printed, Eq. (2) omits the factor k (presumably Ntrain) and gives a probability-like value rather than the expected number of learned images. If the intended quantity is the expected fraction of learned images, the equation should be normalized by Ntrain. This matters because Fig. 4 and Table 6 use the 'idealized scenario' as a reference; please correct the equation and ensure the plotted reference uses the corrected form.
  2. [Eqs. (8)-(9), Sec. 3.3] The confidence-interval formulas appear to have the wrong optimization direction and the wrong summation limits. For a lower bound, one should find the smallest s such that the probability of observing at least Nlearned unique images is at least alpha_e; the arg max in Eq. (8) is the opposite of that. For the upper bound, the sum should run over k = 1..Nlearned (the probability of observing at most Nlearned unique images), not over k = 1..s. As written, the intervals reported in Sec. 4.2 (e.g., 86% [75%, 100%]) are not valid confidence intervals. Please re-derive these formulas and re-run the reported intervals.
  3. [Sec. 3.4, Table 3] The adjusted IRS can exceed 1, and the paper's own interpretation that IRS represents 'the percentage of samples that the model can generate at its limit' is then broken. Table 3 reports IRS_inf,a = 1.51 for FFHQ and 1.08 for ChestX-ray14 with DiADM, and Sec. 3.4 excuses this only 'for low alpha values,' yet Table 3 uses the paper's standard sampling budget. Because the 77% ceiling in Table 2 is computed with the same adjusted ratio, the ceiling is not established as a bound on training-data diversity. Please report the unadjusted IRS_inf,snth and IRS_inf,real values separately and either demonstrate that the ratio is a valid, bounded diversity measure across alphas and extractors or reinterpret the adjusted score as a relative feature-space diversity index rather than a percentage of the training distribution.
  4. [Tab. 6, Sec. 4.1] The claim that IRS is 'hyperparameter-free' is misleading in light of the strong dependence on the feature extractor and distance metric. Table 6 shows IRS_real ranging from about 57% to 88% across extractors on the same datasets, and the supplementary table shows that the relative ranking of extractors changes between cosine and Euclidean distance. The paper selects SwAV based on consensus agreement (Tab. 1) but does not show whether the adjusted IRS values, the 77% ceiling, or the rankings in Table 2 are stable across different choices of F and P. Please add a sensitivity analysis of adjusted IRS over feature extractors and distance measures, or temper the 'hyperparameter-free' and 77% claims.
  5. [Sec. 3.5, Table 3] The DiADM evaluation is confounded by the fact that DiADM conditions on Inception-v3 features while FID is computed with the Inception-v3 feature space. The large FID improvements in Table 3 (e.g., 40.92 to 6.24 on FFHQ) may reflect the conditioning shortcut rather than genuine quality gains. Likewise, the diversity gain may be inflated because the model is explicitly trained to map training-set features to their source images, and IRS counts unique training-image retrievals under the same retrieval paradigm. Please evaluate DiADM with feature extractors not used for conditioning (for both FID and IRS), and include a baseline conditioned on random or low-information features to isolate the effect. Reporting memorization metrics would also address the concern acknowledged in Sec. 4.5.
minor comments (6)
  1. [Throughout] The paper contains several typos, e.g., 'Eucledian' (Sec. 3.2), 'quantize' for 'quantify' (Sec. 4.2), 'snyhtetic' (Sec. 3.4), 'accroding' (Sec. 4.2), and 'prove' for 'proof' (Fig. 11 caption).
  2. [Eq. (5) vs Eqs. (7)-(9)] The symbol s is overloaded: in Eq. (5) it denotes Ntrain, while in Eqs. (7)-(9) it denotes the unknown true number of distinct images that the model can generate. Please use separate symbols to avoid confusion.
  3. [Sec. 12, Table 6 caption] The phrase 'idealized scenario reaches IRS 7/3 = 96.98%' is unclear: 7/3 is the sampling ratio alpha, not an IRS value, and the numerical conversion to 96.98% does not follow from the stated Ntrain and Nsample values. Please clarify the computation.
  4. [Fig. 4] The legend distinguishes 'Theoretical limit' and 'Idealized'; the text says the idealized scenario follows Eq. (2), so the relationship between these two curves should be stated explicitly.
  5. [Sec. 3.4] The adjustment step in Sec. 3.4 is an assumption that dividing by IRS_real removes the measurement gap, but the paper does not provide a formal justification or a test of this assumption across feature extractors. Please add a short discussion or reference to the supplementary analysis beyond the class-removal experiment.
  6. [Reference list] Several references lack venue or publication details (e.g., [22] 'Classifier-Free Diffusion Guidance'), and the arXiv-only references should be marked as such consistently.

Circularity Check

1 steps flagged · score 6.0 of 10

DiADM's reported diversity gain is wired into the setup: per-training-image features are used as conditioning and IRS scores retrieval of those same training images.

  1. self definitional [Sec. 3.2 Eq. (3)-(4); Sec. 3.5; Sec. 4.3]
    "we use these pre-computed image features and feed them directly into the model. ... The core idea is that, with proper training, the model behaves as if each training instance represents its own class, allowing for direct diversification of training instances during sampling time. ... We directly utilize features extracted from F of the training dataset to generate synthetic data, resulting in a synthetic dataset that should maintain diversity if trained properly. ..."

    Diversity is defined by Eq. (3)-(4) as the fraction of training images retrieved by nearest-neighbor search in a pretrained feature space F. DiADM is fed exactly those pretrained per-training-image features ('we directly utilize features extracted from F of the training dataset'), turning unconditional generation into per-instance conditional decoding. Reported IRS gains (FFHQ 0.23 -> 1.51, Table 3) then measure whether the model emits samples that retrieve the training image whose code was supplied. Even though evaluation uses SwAV and conditioning uses Inception, both are pretrained retrieval features, so the model is handed per-image identity codes and scored on retrieval coverage of those very images.

full rationale

The coupon-collector derivation (Sec. 3.1-3.3) is self-contained: it computes an MLE of the eventual number of distinct retrieved training images from observed retrieval counts, with no fitted parameter and an explicit idealized urn model. The IRS metric is also given an external sanity check by removing ImageNet classes in Sec. 4.1/Fig. 5, so the metric's core is not circular. No load-bearing self-citation was found: references [14] and [40] are same-author citations but only supply a re-identification extractor and a synthetic dataset; they do not justify the central premises. However, the DiADM contribution (Sec. 3.5, 4.3) is partially circular: the method conditions the generator on pretrained per-training-image features, changing the task from unconditional sampling to per-instance conditional decoding, and then evaluates 'diversity' with a retrieval metric that counts distinct training images. Under that criterion, a model that can decode its conditioning codes will score high by construction. Sec. 3.4's adjusted score, which the paper admits can exceed one, is an additional interpretive risk: it makes the 77% ceiling and the DiADM gains depend on a depressed real-data denominator rather than on an independent bound on training-data diversity. I treat that as a validity concern rather than a separate circular step. Overall, the IRS measurement framework is independent, but the headline DiADM diversity improvement reduces in large part to the evaluation criterion being fed in as conditioning, so a partial-circularity score of 6 is warranted.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the composite-sample assumption, the uniform coupon-collector model, the feature-nearest-neighbor identification of the main component, and the normalization step. These are domain assumptions specific to the paper rather than standard mathematical axioms, and the normalization is effectively ad hoc. No new physical entities are postulated; DiADM is a training procedure and is covered under the axioms and free parameter choices.

free parameters (3)
  • Feature extractor F for IRS = SwAV
    IRS is defined with respect to F; Table 6 shows scores vary from 53% to 88% across extractors on real data. The paper selects SwAV by agreement with an ensemble of the same candidate extractors (Tab. 1), not by external diversity accuracy.
  • Distance metric P for image retrieval = Euclidean distance
    Tab. 4 shows cosine and Euclidean distances change which extractor performs best; Euclidean is chosen because it connects to precision/recall, not because it is validated as the correct retrieval geometry.
  • Reference dataset split for adjusted IRS = 50k samples, half-split for small datasets
    IRS∞,a = IRS∞,snth / IRS∞,real depends on the reference set; small or self-similar reference sets can push the ratio above 1.0 (Table 3), so the score is not invariant to this design choice.
assumptions (4)
  • domain assumption Every synthetic image is a composition in which one training image is the dominant 'main component'.
    Stated in Sec. 3.1 before Eq. (1); the whole count of 'learned' images relies on each synthetic sample being attributable to one training image.
  • domain assumption In the ideal model, training images are sampled uniformly with probability 1/Ntrain.
    Used in Eq. (2) for the coupon collector expectation; real training distributions are not uniform, so the idealized benchmark is an upper anchor rather than a ground truth.
  • domain assumption Nearest neighbor in feature space identifies the main component.
    Eq. (3) defines Xlearned by argmin over P(F(xt), F(x')); the paper's own measurement-gap results show feature spaces collapse, so this equivalence is not established.
  • ad hoc to paper Dividing synthetic IRS by real IRS removes the measurement gap.
    Sec. 3.4 assumes the feature collapse affects both numerator and denominator equally and that the ratio is unbiased; no proof is given, and ratios above 1.0 for synthetic data suggest the assumption fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Image Generation Diversity Issues and How to Tame Them." pith.science (2026). https://pith.science/paper/7R2FHD5S

@misc{pith2026241116171,
  author       = {Pith},
  title        = {Pith review of: Image Generation Diversity Issues and How to Tame Them},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7R2FHD5S}},
  note         = {Machine review of arXiv:2411.16171}
}
read the original abstract

Generative methods now produce outputs nearly indistinguishable from real data but often fail to fully capture the data distribution. Unlike quality issues, diversity limitations in generative models are hard to detect visually, requiring specific metrics for assessment. In this paper, we draw attention to the current lack of diversity in generative models and the inability of common metrics to measure this. We achieve this by framing diversity as an image retrieval problem, where we measure how many real images can be retrieved using synthetic data as queries. This yields the Image Retrieval Score (IRS), an interpretable, hyperparameter-free metric that quantifies the diversity of a generative model's output. IRS requires only a subset of synthetic samples and provides a statistical measure of confidence. Our experiments indicate that current feature extractors commonly used in generative model assessment are inadequate for evaluating diversity effectively. Consequently, we perform an extensive search for the best feature extractors to assess diversity. Evaluation reveals that current diffusion models converge to limited subsets of the real distribution, with no current state-of-the-art models superpassing 77% of the diversity of the training data. To address this limitation, we introduce Diversity-Aware Diffusion Models (DiADM), a novel approach that improves diversity of unconditional diffusion models without loss of image quality. We do this by disentangling diversity from image quality by using a diversity aware module that uses pseudo-unconditional features as input. We provide a Python package offering unified feature extraction and metric computation to further facilitate the evaluation of generative models https://github.com/MischaD/beyondfid.

Figures

Figures reproduced from arXiv: 2411.16171 by the authors.

Figure 1
Figure 1. Predicted gender diversity - as one possible example [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We model finding image pairs (image retrieval) as ran [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of our proposed DiADM model. Instead of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Visualization of the measurement gap across a diverse [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Measuring diversity of datasets by removing classes and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Detecting low diversity models. By leveraging the statis [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: IRS∞, indicated by dots, and confidence intervals IRS∞,L and IRS∞,U for three different simulated ImageNet mod￾els. By increasing the number of observations (α), we also in￾crease the confidence of the IRS∞ predictions. The black dashed line indicates the ground truth …
Figure 8
Figure 8. Figure 8: Illustration of the selection process of the threshold for a fixed number of samples with different trainingset sizes. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results of image retrieval on ImageNet (Top) and FFHQ (bottom). [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Comparison of Consensus and Agreement of different feature extractors for the ImageNet Dataset [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Measured diversity according to IRS of common fea [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Measuring diversity of datasets by removing classes [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 41 canonical work pages

  1. [1]

    Understanding hallucinations in diffu- sion models through mode interpolation

    Sumukh K Aithal, Pratyush Maini, Zachary C Lipton, and J Zico Kolter. Understanding hallucinations in diffu- sion models through mode interpolation. arXiv preprint arXiv:2406.09358, 2024. 1

    arXiv 2024

  2. [2]

    Baraniuk

    Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G. Baraniuk. Self-Consuming Generative Models Go MAD, 2023. arXiv:2307.01850 [cs]. 1

    arXiv 2023

  3. [3]

    data2vec: A General Frame- work for Self-supervised Learning in Speech, Vision and Language

    Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. data2vec: A General Frame- work for Self-supervised Learning in Speech, Vision and Language. In Proceedings of the 39th International Confer- ence on Machine Learning, pages 1298–1312. PMLR, 2022. ISSN: 2640-3498. 6

    work page 2022

  4. [4]

    How well can text-to-image generative models un- derstand ethical natural language interventions? arXiv preprint arXiv:2210.15230, 2022

    Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. How well can text-to-image generative models un- derstand ethical natural language interventions? arXiv preprint arXiv:2210.15230, 2022. 1, 3

    arXiv 2022

  5. [5]

    Baum and Patrick Billingsley

    Leonard E. Baum and Patrick Billingsley. Asymptotic Dis- tributions for the Coupon Collector’s Problem.The Annals of Mathematical Statistics, 36(6):1835–1839, 1965. Publisher: Institute of Mathematical Statistics. 3

    work page 1965

  6. [6]

    Extracting Training Data from Diffusion Models, 2023

    Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagiel- ski, Vikash Sehwag, Florian Tram `er, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting Training Data from Diffusion Models, 2023. arXiv:2301.13188 [cs]. 2, 8

    arXiv 2023

  7. [7]

    Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Pi- otr Bojanowski, and Armand Joulin. Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. In Advances in Neural Information Processing Systems , pages 9912–9924. Curran Associates, Inc., 2020. 6

    work page 2020

  8. [8]

    Ultra- sound Image-to-Video Synthesis via Latent Dynamic Dif- fusion Models

    Tingxiu Chen, Yilei Shi, Zixuan Zheng, Bingcong Yan, Jingliang Hu, Xiao Xiang Zhu, and Lichao Mou. Ultra- sound Image-to-Video Synthesis via Latent Dynamic Dif- fusion Models. In Medical Image Computing and Com- puter Assisted Intervention – MICCAI 2024, pages 764–774. Springer Nature Switzerland, Cham, 2024. Series Title: Lec- ture Notes in Computer Science. 2

    work page 2024

Show all 67 references
  1. [9]

    Debiasing vision- language models via biased prompts

    Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Anto- nio Torralba, and Stefanie Jegelka. Debiasing vision- language models via biased prompts. arXiv preprint arXiv:2302.00070, 2023. 3

    2023 arXiv

  2. [10]

    ImageNet: A Large-Scale Hierarchical Image Database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. 2015. 5

    2015

  3. [11]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 1, 2

    2021

  4. [12]

    Diffusion Models Beat GANs on Image Synthesis, 2021

    Prafulla Dhariwal and Alex Nichol. Diffusion Models Beat GANs on Image Synthesis, 2021. arXiv:2105.05233 [cs, stat]. 3, 5, 8

    2021 arXiv

  5. [13]

    Cogview: Mastering text-to-image generation via transformers

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822–19835, 2021. 2, 3

    2021

  6. [14]

    Uncovering Hidden Subspaces in Video Diffusion Models Using Re-Identification, 2024

    Mischa Dombrowski, Hadrien Reynaud, and Bernhard Kainz. Uncovering Hidden Subspaces in Video Diffusion Models Using Re-Identification, 2024. arXiv:2411.04956. 2, 7, 16

    2024 arXiv

  7. [15]

    Diverse Data Augmentation with Dif- fusions for Effective Test-time Prompt Tuning

    Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse Data Augmentation with Dif- fusions for Effective Test-time Prompt Tuning. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2704–2714, Paris, France, 2023. IEEE. 2

    2023

  8. [16]

    The Vendi Score: A Diversity Evaluation Metric for Machine Learning, 2023

    Dan Friedman and Adji Bousso Dieng. The Vendi Score: A Diversity Evaluation Metric for Machine Learning, 2023. arXiv:2210.02410. 3

    2023 arXiv

  9. [17]

    Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness, 2023

    Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, and Kristian Kersting. Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness, 2023. arXiv:2302.10893. 2

    2023 arXiv

  10. [18]

    Richemond, Elena Buchatskaya, Carl Do- ersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mo- hammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, R´emi Munos, and Michal Valko

    Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Do- ersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mo- hammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, R´emi Munos, and Michal Valko. Bootstrap your own ...

    2020 arXiv

  11. [19]

    Masked Autoencoders Are Scalable Vision Learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Pi- otr Dollar, and Ross Girshick. Masked Autoencoders Are Scalable Vision Learners. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979–15988, New Orleans, LA, USA, 2022. IEEE. 6

    2022

  12. [20]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 3, 16

    2017

  13. [21]

    GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2017. 6

    2017

  14. [22]

    Classifier-Free Diffusion Guidance

    Jonathan Ho and Tim Salimans. Classifier-Free Diffusion Guidance. 2022. 5 9

    2022

  15. [23]

    A Style- Based Generator Architecture for Generative Adversarial Networks

    Tero Karras, Samuli Laine, and Timo Aila. A Style- Based Generator Architecture for Generative Adversarial Networks. 2019. 5

    2019

  16. [24]

    Analyzing and Im- proving the Training Dynamics of Diffusion Models, 2024

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hell- sten, Timo Aila, and Samuli Laine. Analyzing and Im- proving the Training Dynamics of Diffusion Models, 2024. arXiv:2312.02696 [cs, stat]. 5, 8

    2024 arXiv

  17. [25]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1931–1941, 2023. 3

    1931

  18. [26]

    Improved Precision and Recall Metric for Assessing Generative Models

    Tuomas Kynk ¨a¨anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved Precision and Recall Metric for Assessing Generative Models. 2019. 3, 15

    2019

  19. [27]

    Autoregressive Image Generation without Vec- tor Quantization, 2024

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive Image Generation without Vec- tor Quantization, 2024. arXiv:2406.11838 [cs]. 3, 5, 8

    2024 arXiv

  20. [28]

    Residual denoising diffu- sion models

    Jiawei Liu, Qiang Wang, Huijie Fan, Yinong Wang, Yan- dong Tang, and Liangqiong Qu. Residual denoising diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2773– 2783, 2024. 2, 3

    2024

  21. [29]

    A ConvNet for the 2020s, 2022

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A ConvNet for the 2020s, 2022. arXiv:2201.03545. 6

    2022 arXiv

  22. [30]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. In International Conference on Learning Representa- tions, 2022. 2, 3

    2022

  23. [31]

    Training diffusion models towards diverse image generation with reinforcement learning

    Zichen Miao, Jiang Wang, Ze Wang, Zhengyuan Yang, Li- juan Wang, Qiang Qiu, and Zicheng Liu. Training diffusion models towards diverse image generation with reinforcement learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 108...

    2024

  24. [32]

    Reliable Fidelity and Diversity Metrics for Generative Models

    Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable Fidelity and Diversity Metrics for Generative Models. 2020. 3

    2020

  25. [33]

    OaReilly and Fawad Asadi

    Jamie A. OaReilly and Fawad Asadi. Pre-trained vs. Ran- dom Weights for Calculating Fr ´echet Inception Distance in Medical Imaging. In 2021 13th Biomedical Engineering In- ternational Conference (BMEiCON), pages 1–4, Ayutthaya, Thailand, 2021. IEEE. 6, 16

    2021

  26. [34]

    DINOv2: Learning Robust Visual Features without Supervision, 2024

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mah- moud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michae...

    2024 arXiv

  27. [35]

    Langlotz, Paul A

    David Ouyang, Bryan He, Amirata Ghorbani, Neal Yuan, Joseph Ebinger, Curtis P. Langlotz, Paul A. Heidenreich, Robert A. Harrington, David H. Liang, Euan A. Ashley, and James Y . Zou. Video-based AI for beat-to-beat assessment of cardiac function. Nature, 580(7802):252–256, 202...

    2020

  28. [36]

    Scalable Diffusion Models with Transformers

    William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. In2023 IEEE/CVF International Confer- ence on Computer Vision (ICCV) , pages 4172–4182, Paris, France, 2023. IEEE. 3, 5, 8

    2023

  29. [37]

    Class-balancing diffusion models

    Yiming Qin, Huangjie Zheng, Jiangchao Yao, Mingyuan Zhou, and Ya Zhang. Class-balancing diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 18434–18443, 2023. 3

    2023

  30. [38]

    Learning Transferable Vi- sual Models From Natural Language Supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Vi- sual Models From Natural Language Supervision, 2021. arXiv:2103.00020. 6

    2021 arXiv

  31. [39]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  32. [40]

    EchoNet-Synthetic: Privacy-preserving Video Generation for Safe Medical Data Sharing, 2024

    Hadrien Reynaud, Qingjie Meng, Mischa Dombrowski, Ari- jit Ghosh, Thomas Day, Alberto Gomez, Paul Leeson, and Bernhard Kainz. EchoNet-Synthetic: Privacy-preserving Video Generation for Safe Medical Data Sharing, 2024. arXiv:2406.00808 [cs]. 6, 7, 16

    2024 arXiv

  33. [41]

    High-Resolution Image Synthesis with Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models. 2021. 3, 5

    2021

  34. [42]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3

    2022

  35. [43]

    High-Resolution Image Synthesis with Latent Diffusion Models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models, 2022. arXiv:2112.10752 [cs]. 8

    2022 arXiv

  36. [44]

    DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. 2022. 2

    2022

  37. [45]

    Improved Techniques for Training GANs

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved Techniques for Training GANs. 2016. 6

    2016

  38. [46]

    StyleGAN- XL: Scaling StyleGAN to Large Diverse Datasets, 2022

    Axel Sauer, Katja Schwarz, and Andreas Geiger. StyleGAN- XL: Scaling StyleGAN to Large Diverse Datasets, 2022. arXiv:2202.00273 [cs]. 3

    2022 arXiv

  39. [47]

    Generating high fidelity data from low-density regions using diffusion models

    Vikash Sehwag, Caner Hazirbas, Albert Gordo, Firat Oz- genel, and Cristian Canton. Generating high fidelity data from low-density regions using diffusion models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11492–11501, 2022. 2

    2022

  40. [48]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 2 10

    2010 arXiv

  41. [49]

    Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L

    George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, J. Eric T. Taylor, and Gabriel Loaiza- Ganem. Exposing flaws of generative model evaluation met- rics and their unfair treatment of diffusion...

    2023 arXiv

  42. [50]

    Rethinking the Inception Ar- chitecture for Computer Vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the Inception Ar- chitecture for Computer Vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818–2826, Las Vegas, NV , USA, 2016. IEEE. 6

    2016

  43. [51]

    N. M. Temme. Asymptotic Estimates of Stirling Numbers. Studies in Applied Mathematics, 89(3):233–243, 1993. 4

    1993

  44. [52]

    ChestX-ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Com- mon Thorax Diseases

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Moham- madhadi Bagheri, and Ronald M Summers. ChestX-ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Com- mon Thorax Diseases. 2017. 5

    2017

  45. [53]

    Diffusion models: A comprehensive survey of methods and applications

    Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Run- sheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming- Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 56(4): 1–39, 2023. 1, 3

    2023

  46. [54]

    Improving diffusion-based image synthesis with context pre- diction

    Ling Yang, Jingwei Liu, Shenda Hong, Zhilong Zhang, Zhilin Huang, Zheming Cai, Wentao Zhang, and Bin Cui. Improving diffusion-based image synthesis with context pre- diction. Advances in Neural Information Processing Sys- tems, 36, 2024. 2

    2024

  47. [55]

    Iti- gen: Inclusive text-to-image generation

    Cheng Zhang, Xuanbai Chen, Siqi Chai, Chen Henry Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre. Iti- gen: Inclusive text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 3969–3980, 2023. 2

    2023

  48. [56]

    Stability and gener- alizability in sde diffusion models with measure-preserving dynamics

    Weitong Zhang, Chengqi Zang, Liu Li, Sarah Cechnicka, Cheng Ouyang, and Bernhard Kainz. Stability and gener- alizability in sde diffusion models with measure-preserving dynamics. In NeurIPS, 2024. 1, 2

    2024

  49. [57]

    A unified conditional framework for diffusion-based image restoration

    Yi Zhang, Xiaoyu Shi, Dasong Li, Xiaogang Wang, Jian Wang, and Hongsheng Li. A unified conditional framework for diffusion-based image restoration. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  50. [58]

    Dora Zhao, Jerone T. A. Andrews, Orestis Papakyriakopou- los, and Alice Xiang. Position: Measure Dataset Diversity, Don’t Just Claim It, 2024. arXiv:2407.08188. 1

    2024 arXiv

  51. [59]

    CelebV- HQ: A Large-Scale Video Facial Attributes Dataset, 2022

    Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. CelebV- HQ: A Large-Scale Video Facial Attributes Dataset, 2022. arXiv:2207.12393 [cs]. 5 11 Image Generation Diversity Issues and How to Tame Them Supplementary Material

    2022 arXiv

  52. [60]

    Model Rejection Based on IRS Building on the methodology introduced in the main paper, we explore how IRS can provide additional insights into model diversity during training. IRS, which requires a min- imal number of samples for computation, proves to be par- ticularly useful...

  53. [61]

    Further Visual Results To illustrate the image retrieval results, we present visual- izations for ImageNet-512 in Fig. 9. For a randomly se- lected subset of images, we compute the image correspon- 12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Different images (k) 0.00 0.05 0.10 0.15 ...

  54. [62]

    4.1 we argued that due to the adjustment step in- troduced in Sec

    Image Retrieval Agreement and Consensus In Sec. 4.1 we argued that due to the adjustment step in- troduced in Sec. 3.4 we can in theory choose any kind of feature extractor F. In order to maximize the interpretabil- ity of IRS we use the extractor that has the best agreement w...

  55. [63]

    The first one is the cosine distance derived from cosine similarity

    Further Distance Metric Sensitivty Analysis We consider two different measurements P to compute the distance between fx = F (xt) of the query image and all reference images fx′ = F (x′ t). The first one is the cosine distance derived from cosine similarity. It is used by many ...

  56. [64]

    We set the desired IRS to 80% and the probability of error to 5% with 50000 ref- erence and synthetic samples each

    Computational Requirement To benchmark the proposed lacking diversity rejection method we use the method for the official ImageNet-512 train set with N train = 1281166. We set the desired IRS to 80% and the probability of error to 5% with 50000 ref- erence and synthetic sample...

    2000

  57. [65]

    Specifically, we examine how the performance associated with the observed mea- surement gap changes when feature extractors are tailored to the dataset

    Results with Domain Specific Feature Ex- tractors In the next step, we analyze the impact of feature extrac- tors on prediction performance. Specifically, we examine how the performance associated with the observed mea- surement gap changes when feature extractors are tailored...

  58. [66]

    IRSreal Results In Sec. 4.1 we explain the measurement gap stemming from feature extractors collapsing to smaller features spaces that ImageNet FFHQ CXR-14 CelebV Dynamic 0 20 40 60 80 100Diversity (%) BYOL CLIP ConvNeXt data2vec DINOv2 Inception MAE SwA V Random Figure 11. Me...

  59. [67]

    Metrics Analysis and Comparison 13.1. IRS over FID in Measuring Diversity Insuf- ficiency and Bias Amplification Here, we analyze the properties of IRS and demonstrate its superiority over FID [20] in detecting diversity insuffi- ciency and bias amplification in generative mod...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.