Pith. sign in

REVIEW 3 major objections 5 minor 142 references

Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoders

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that conceptual blindspots in text-to-image models are systematic and measurable, and demonstrates a sparse-autoencoder method that finds them across four diffusion models.

desk verdict A genuinely useful SAE-based instrument for finding concept-level failures in text-to-image models, but the theoretical framing has a concrete algebra error and the cross-domain faithfulness of the SAE basis is unverified. read the letter →

arxiv 2506.19708 v1 pith:3GHDPZ7O submitted 2025-06-24 cs.GR cs.AIcs.CV

Matyas Bohacek , Thomas Fel , Maneesh Agrawala , Ekdeep Singh Lubana This is my paper · ORCID
classification cs.GRcs.AIcs.CV
keywords conceptualblindspotssparseautoencoderstext-to-imagegenerationdiffusionmodelsinterpretabilityDINOv2energy-basedconceptfidelity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the well-known failures of text-to-image models—hands with six fingers, objects that will not appear in groups of four—are not isolated quirks but instances of a structural, measurable phenomenon. It defines a conceptual blindspot as a concept whose frequency in generated images differs sharply from its frequency in the natural images the model was trained on, and measures that difference with a 32,000-concept sparse autoencoder over DINOv2 features. Applied to Stable Diffusion 1.5/2.1, PixArt, and Kandinsky, the method finds consistent left-skewed distributions: models suppress concepts more often than they exaggerate them. Concrete suppressed blindspots include bird feeders, glossy DVD discs, and whitespaces on documents, while exaggerated ones include wood background texture and palm trees. A sympathetic reader would care because this turns anecdotal complaints into an automated, model-comparable audit of what a generative model can and cannot represent.

What carries the argument

The load-bearing object is the energy-difference score $\delta_{g_\theta\leftrightarrow G}(k)$, defined as the sigmoid of the mean sparse-activation gap between generated and natural images for concept $k$; it is a strictly increasing reparameterization of the odds ratio, so ranking by $\delta$, by energy gap, or by odds gives the same order. The companion machinery is an archetypal sparse autoencoder (RA-SAE) with 32,000 concept atoms trained on DINOv2 features, whose dictionary is constrained to the convex hull of the training data and whose top-$k$ sparse codes provide the concept-energy estimates. Comparing the same prompt-conditioned image pairs in this shared concept basis is what lets the method move from anecdotal failure reports to population-level blindspot rankings.

What would settle it

Run a blinded study in which annotators mark the presence of concepts such as bird feeder, DVD disc, and whitespace on documents in matched real and generated images; if the generated images contain those concepts as often as the natural ones, the claimed blindspots are measurement artifacts.

Watch

Extended reading notes

Core claim

Formally, the paper claims that for each concept $c_k$ one can estimate an energy difference $\delta(k)=\sigma(\mathbb{E}_{x'}[\xi_k(x')]-\mathbb{E}_x[\xi_k(x)])$ between generated and natural images, and that values below $0.1$ or above $0.9$ mark suppressed or exaggerated conceptual blindspots. Using an archetypal sparse autoencoder (RA-SAE) with 32,000 concept dimensions trained on DINOv2 features, the sparse activation for a dimension is treated as the concept's energy, making real and generated images comparable in a shared basis. Across all four tested diffusion models the $\delta(k)$ histograms are heavy-tailed with a denser left tail, which the paper reads as a general tendency toward concept omission; specific concepts such as solid white on documents are suppressed by all models, while others such as pan are suppressed only by Kandinsky. At the datapoint level, near-zero energy differences flag memorized templates rather than faithful generation, and large differences flag prompt ambiguity or genuine inability. The paper also reports that rare concepts in the natural data are the most likely to be suppressed, and that DPO fine-tuning narrows the datapoint-wise energy gap.

Load-bearing premise

The method assumes a concept's absence from the SAE's 32,000 DINOv2-based directions would also be absent from a human's vocabulary; if the encoder silently blends or omits a concept, every blindspot ranking built on it inherits that blindspot.

Editorial extensions

If this is right

  • Blindspot audits become a repeatable per-model report: for any text-to-image model, one can list the concepts it suppresses and exaggerates relative to its training distribution.
  • Because rare concepts are the most suppressed, fixing blindspots may require reweighting or augmenting long-tail concepts in the training data rather than changing only model size or architecture.
  • Post-training interventions such as DPO can be evaluated by their effect on concept-fidelity geometry, not just by output quality scores.
  • Near-zero datapoint energy differences give a scalable signal for detecting memorization artifacts.
  • Shared blindspots across architectures point to dataset-level causes, while model-specific blindspots point to training dynamics, enabling attribution of failures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the framework is right, concept-level audits could become a standard pre-deployment checklist, catching failures that human prompt-writers would never think to test.
  • The same energy-difference recipe should transfer to video and 3D generators once a suitable self-supervised encoder and an archetypal SAE exist for those modalities; the paper does not test this.
  • The method's reliance on paired captions means it could also measure caption-image misalignment in web-scraped datasets, a use the paper mentions only in passing.
  • Because $\delta$ is monotone in the odds ratio, practitioners can substitute raw activation differences for $\delta$ without changing concept rankings; this follows from the paper's Appendix J monotonicity theorem.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces a formal framework, "conceptual blindspots," for comparing concept prevalence between natural images and text-to-image model outputs. The method trains a 32,000-atom archetypal sparse autoencoder (RA-SAE) on DINOv2 features of ImageNet, applies it to 10,000 LAION image-text pairs and matching generations from SD 1.5, SD 2.1, PixArt, and Kandinsky, and defines a per-concept energy-difference score delta(k). Concepts with delta below 0.1 are labeled suppressed and above 0.9 exaggerated; the paper reports distributional asymmetries, cross-model correlations, qualitative examples, datapoint-level memorization artifacts, effects of DPO fine-tuning, and a relationship between concept frequency and misalignment. The authors release the model, code, and an interactive web tool.

Significance. If the central claim holds, this is a useful step beyond anecdotal failure documentation: it offers an unsupervised, scalable, and interpretable procedure for locating concept-level gaps in generative models, with concrete assets (open-source 32,000-concept RA-SAE, code, web tool) and falsifiable predictions such as the bird-feeder blindspot in Kandinsky and the glossy-DVD-disc blindspot in SD 1.5. The stress-test protocol in Appendix G is a genuine attempt at external validation, and the cross-model and post-training analyses demonstrate the breadth of the pipeline. The main reservations are not about the empirical effort but about the correctness of the theoretical wrapper and the faithfulness of the ImageNet-trained encoder on the target distributions.

major comments (3)
  1. [Definition 2, Eq. (1)] The equality delta_{g_theta<->G}(k) = sigma(E_{x'}[xi_k(x')] - E_x[xi_k(x)]) = p_k(D'_X)/(p_k(D_X)+p_k(D'_X)) is incorrect under the stated definition p_k(D) proportional to exp(-sum_{x in D} xi_k(x)). With n = |D_X| = |D'_X|, the right-hand side equals sigma(n(E_{x'}[xi_k(x')] - E_x[xi_k(x)])), not sigma of the unscaled gap. The identity would hold only if p_k(D) were defined with a 1/n exponent, i.e., p_k(D) proportional to exp(-(1/n) sum_{x in D} xi_k(x)). Because this equality is presented as the probabilistic grounding of delta, it needs to be corrected; the monotonicity argument in Appendix J preserves the ranking interpretation only after this scaling inconsistency is resolved.
  2. [Section 4, Eq. (2)-(3); Appendix B] The RA-SAE dictionary is trained exclusively on DINOv2 features of ImageNet-1k tokens, with atoms constrained to the convex hull of ImageNet training tokens, and is then applied to LAION natural images and to images generated by LAION-trained models. The paper does not report reconstruction error, explained variance, or any out-of-distribution fidelity statistic for the SAE on either target distribution. Since every delta(k), every threshold assignment, and every stress-test result is computed from these activations, a domain-shift failure would make the reported blindspots artifacts of the encoder rather than properties of the generative models. The Limitations paragraph concedes that concepts poorly represented by DINOv2/RA-SAE "will escape our analysis," but the central claim requires more than non-escape: the activations must track concept prevalence accurately on DX and D'_X. I request reconstruction-fidelity metrics on the actual target sets, and ideally a validation that identified blindspots are stable under a different dictionary or feature extractor.
  3. [Section 5.5 and Fig. 12] The DPO comparison is computed on the same ImageNet-trained RA-SAE basis as all other analyses, so the claim that DPO "regularizes the model's concept distribution" inherits the domain-shift concern above. In addition, the comparison uses a single DPO checkpoint and a single baseline; without a reconstruction-fidelity check on these generated distributions, the reported reduction in the L2 norm of concept-energy differences could reflect the encoder's response to distribution shift rather than a genuine improvement in concept fidelity. This does not invalidate the observation, but it weakens the causal interpretation as stated.
minor comments (5)
  1. [Appendix I, Theorem 4] The McDiarmid bound has a factor-two error: for 2n variables each with bounded-difference constant L/n, the exponent should be -n*epsilon^2/L^2, not -2n*epsilon^2/L^2. The qualitative conclusion is unaffected, but the stated bound is not what the proof delivers.
  2. [Appendix K, Theorem 6] Under the assumption D^T D = I_d, the map v -> Dv is an exact isometry, so the Wasserstein-2 distance is preserved exactly and the sigma_min/sigma_max bounds are unnecessary and misleading; moreover, for overcomplete D the matrix DD^T has a zero eigenvalue, so the empirical statement sigma_min approx 1 is inconsistent with the theorem's assumptions.
  3. [Section 3 and Appendix A.3] In Appendix A.3, the text says exaggerated blindspots are chosen with delta < 0.9; this should be delta > 0.9. Also, the thresholds lambda_min = 0.1 and lambda_max = 0.9 are introduced without a sensitivity analysis; given the heavy-tailed distributions in Fig. 6, it would strengthen the paper to show that the qualitative findings are robust to reasonable threshold choices.
  4. [Section 5.4] The "memorization artifacts" in Fig. 11a are identified by qualitative inspection of examples with near-zero average delta; no quantitative criterion distinguishes memorization from faithful generation, so the terminology should be softened or supplemented with a reproducibility/nearest-neighbor analysis.
  5. [Notation and typos] Figure 9's caption uses S_theta while the text consistently uses g_theta; Fig. 10's caption spells "exaggeratted"; Section 2.2 writes "DDMPs" where "DDPMs" is meant. These are minor but should be fixed in revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the δ measurements are post-hoc comparisons from a fixed, pre-trained SAE, not fitted predictions, and the self-cited RA-SAE is an open-source published method not tuned to the reported blindspots.

full rationale

The paper's central measurement is an operational comparison rather than a fitted prediction. δ(k) is computed by applying a fixed RA-SAE, trained once on ImageNet DINOv2 features (Appendix B), to LAION and generated images and then taking a sigmoid of the difference of mean activations; no parameter of the SAE, and no threshold λ_min/λ_max, is fitted to the blindspot findings, and the existence of concepts with δ(k)<0.1 or δ(k)>0.9 is an empirical outcome rather than an identity. The statement that 'the activation associated by the SAE to a dimension will serve as our approximation of the ground-truth energy function' is an explicit operationalization, not a hidden equation that forces the reported results. The self-citation of RA-SAE [56] by two of the authors is not load-bearing circularity: it is a published, open-source method, not tuned to produce specific blindspots, so per the review rules it counts as independent evidence. The paper's own Limitations paragraph concedes that 'concepts poorly represented by them will escape our analysis,' which is an honest coverage caveat rather than a disguised input; if anything it weakens the strength of the claims but does not make the derivation circular. The stress tests in Appendix G include manual human review of generated images, and the qualitative examples in Section 5.3 are manually inspected, so the 'blindspot' labels are not merely the thresholds reapplied without external check. The apparent algebraic mismatch in Definition 2 between the exponentiated sums over datasets and the sigmoid of mean differences is a correctness concern, not a circularity. Accordingly, no circular step can be exhibited from the text, and the appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central measurement treats SAE activations as concept energies, so the ledger is dominated by domain assumptions about DINOv2 and RA-SAE fidelity. The main free parameters are the blindspot thresholds and sigmoid temperatures, which are hand-set. No new physical entities are introduced.

free parameters (5)
  • Blindspot thresholds λ_min, λ_max = 0.1, 0.9
    Hand-set in Definition 3 and Section 5.1 to label suppressed and exaggerated concepts; changing them changes which concepts are called blindspots.
  • Sigmoid temperature T = 0.8 (distribution level), 0.4 (Section 5.6)
    Applied to energy differences to compute δ; the temperature rescales values and thus affects threshold crossings, without a principled derivation.
  • RA-SAE dictionary size K' = 32,000
    Chosen hyperparameter; larger dictionaries change concept granularity and the resulting blindspot rankings.
  • RA-SAE top-k sparsity = 5
    Chosen sparsity level; controls how many concepts activate per image and therefore the activation statistics.
  • Number of image-text pairs n = 10,000
    Sampled from LAION; limits statistical resolution for rare concepts as the authors note.
assumptions (5)
  • domain assumption The data-generating process is invertible and latents are independent with a Boltzmann distribution and linearly decomposable energy.
    Definition 1, Section 3; used to justify interpreting concept activations as energies, but image generation does not provably satisfy this.
  • domain assumption DINOv2 features are a linear transform of the true latent concepts (they invert the DGP).
    Section 4 relies on prior results [126,128,133,134] that apply to contrastive learning, not DINOv2 specifically.
  • domain assumption SAE activations approximate the ground-truth energy for each concept.
    Section 4 states this directly; no independent verification that activation magnitude equals concept prevalence.
  • domain assumption Concepts learned from ImageNet-1k DINOv2 features transfer to LAION images and generated images.
    RA-SAE is trained on ImageNet (Appendix B) and applied to LAION and model outputs; the Discussion lists this as a limitation.
  • ad hoc to paper Paired real and generated images are independent for the concentration bound.
    Theorem 4 in Appendix I treats 2n variables as independent, but each pair shares a text prompt, so the images are dependent; the bound's stated form is not justified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoders." pith.science (2026). https://pith.science/paper/3GHDPZ7O

@misc{pith2026250619708,
  author       = {Pith},
  title        = {Pith review of: Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3GHDPZ7O}},
  note         = {Machine review of arXiv:2506.19708}
}
read the original abstract

Despite their impressive performance, generative image models trained on large-scale datasets frequently fail to produce images with seemingly simple concepts -- e.g., human hands or objects appearing in groups of four -- that are reasonably expected to appear in the training data. These failure modes have largely been documented anecdotally, leaving open the question of whether they reflect idiosyncratic anomalies or more structural limitations of these models. To address this, we introduce a systematic approach for identifying and characterizing "conceptual blindspots" -- concepts present in the training data but absent or misrepresented in a model's generations. Our method leverages sparse autoencoders (SAEs) to extract interpretable concept embeddings, enabling a quantitative comparison of concept prevalence between real and generated images. We train an archetypal SAE (RA-SAE) on DINOv2 features with 32,000 concepts -- the largest such SAE to date -- enabling fine-grained analysis of conceptual disparities. Applied to four popular generative models (Stable Diffusion 1.5/2.1, PixArt, and Kandinsky), our approach reveals specific suppressed blindspots (e.g., bird feeders, DVD discs, and whitespaces on documents) and exaggerated blindspots (e.g., wood background texture and palm trees). At the individual datapoint level, we further isolate memorization artifacts -- instances where models reproduce highly specific visual templates seen during training. Overall, we propose a theoretically grounded framework for systematically identifying conceptual blindspots in generative models by assessing their conceptual fidelity with respect to the underlying data-generating process.

Figures

Figures reproduced from arXiv: 2506.19708 by the authors.

Figure 1
Figure 1. δ(k) quantifies a model’s tendency to over- or under-generate a concept ck compared to its natural-data frequency. We deem concepts with δ(k) < 0.1 as suppressed conceptual blindspots and concepts with δ(k) > 0.9 as exaggerated conceptual blindspots. The depicted images, generated by four popular generative image models, show examples of images with conceptual blindspots, as well as aligned concepts. to demonstratio… view at source ↗
Figure 2
Figure 2. Intuitive mapping of our framework. Latent concepts ck ∈ C are mapped to observations (x, t) through the (invertible) DGP. Sθ generates images x ′ based on t. The energy model ξk(x) extracts concept representations from both x and x ′ , enabling comparison of concept distributions to identify blindspots. In essence, the individual dimensions of the latent space reflect the Concepts underlying the data-distribution P… view at source ↗
Figure 3
Figure 3. Concept Extraction Pipeline. For a triplet (x, t, x ′ ), the concepts in x and t are extracted by obtaining each image’s DINOv2 features, which are further processed by a RA-SAE into sparse concept embeddings, yielding energy models ξ(x) and ξ(x ′ ), respectively. In particular, ξk(x) = E(ck) holds the energy in the k th concept. 137–139]. The intuition here is that if the concepts underlying the generative process … view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: Representative image-caption pairs from LAION-5B with matching images generated with the same prompt by SD 1.5, SD 2.1, PixArt, and Kandinsky. suppressed and exaggerated concepts; a • datapoint-level analysis surfaces failures tied to ambiguity, omission, and memorizat…
Figure 5
Figure 5. Figure 5: Overview of the Exploratory Tool. The web interface displays a UMAP projection for each evaluated model, where each dot represents a concept, color-coded by its energy difference. When a concept is selected, a detail panel presents illustrative images, statistics, and …
Figure 6
Figure 6. Figure 6: • Concept Energy Distribution. Log-scale histograms of energy differences δ(k) across 32, 000 concepts, comparing the natural and synthesized distributions for each evaluated model. Values left of zero represent suppressed concepts (under-represented); values right of …
Figure 7
Figure 7. Figure 7: • Structure of Concept Energy Differences. UMAP visualizations of 32, 000 concepts, colored according to their energy difference δ(k) between the natural and synthesized distributions. Clusters reveal patterns of conceptual blindspots, with suppressed concepts on the b…
Figure 8
Figure 8. Figure 8: • Cross-Model Concept Energy Correlation. Pairwise scatter plots of δ(k) across all four evaluated models, with Pearson correlation coefficients reported top left. Strong alignment between SD 1.5 and SD 2.1 contrasts with weaker correlations among other architectures, …
Figure 9
Figure 9. Figure 9: • Examples of Suppressed Conceptual Blindspots. The natural images x, representative ck and t, shown alongside four synthesized images x ′ , generated using Sθ. The universal blindspot is present in all evaluated models; the unique blindspot is only present in Kandinsk…
Figure 10
Figure 10. Figure 10: • Example of an Exaggeratted Con￾ceptual Blindspot. Four synthesized images D′ X , archetypal for the concept shadow under animal, alongside the corresponding natural im￾age DX and caption t used to generate it. Note the accentuated, nearly solid black shadows under￾n…
Figure 11
Figure 11. Figure 11: • Datapoint-level Conceptual Align￾ment. (a) Examples with minimal energy differ￾ences where models appear to memorize train￾ing patterns. (b) Examples with large differ￾ences where significant concept divergences due to prompt ambiguity or model limitations occur. Th…
Figure 12
Figure 12. Figure 12: • Effect of DPO on Concept Fidelity. Histogram of datapoint-wise energy differences between the synthesized and natural distribution of SD 1.5 models with and without DPO. Post-training protocols, e.g., safety fine-tuning, have been argued to reduce the diversity of m…
Figure 13
Figure 13. Figure 13: • Concept Fidelity Across Frequency Spectrum. Scatter plots showing the relationship between concept frequency (x-axis) and the energy difference (y-axis) across four evaluated models. Each point represents a concept with size is proportional to its activation frequen…
Figure 14
Figure 14. Figure 14: Examples of conceptual blindspots in Stable Diffusion 1.5. For each concept, the prototypical natural (for suppressed blindspots) or synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is su…
Figure 15
Figure 15. Figure 15: Examples of conceptual blindspots in Stable Diffusion 2.1. For each concept, the prototypical natural (for suppressed blindspots) or synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is su…
Figure 16
Figure 16. Figure 16: Examples of conceptual blindspots in Kandinsky. For each concept, the prototypical natural (for suppressed blindspots) or synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is superimposed …
Figure 17
Figure 17. Figure 17: Examples of conceptual blindspots in PixArt. For each concept, the prototypical natural (for suppressed blindspots) or synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is superimposed ato…
Figure 18
Figure 18. Figure 18: • Sparsity and Structural Divergence. On the left: L0 norm of the co-occurrence matrix ZZT as a function of ϵ (threshold), indicating how many entries remain active in each model. On the right: Number of unique entries in the synthesized distribution relative to the n…
Figure 19
Figure 19. Figure 19: • Spectral Structure of Co-occurrence. Log-log plot of the eigenvalue spectra from co-occurrence matrices ZZT across models and the natural distribution. All evaluated models match the heavy-tailed decay of the natural distribution. However, as illustrated in [PITH_F…
Figure 20
Figure 20. Figure 20: • Concept Basis Similarity. Cosine similarity heatmaps between the top 100 eigenvectors of the natural and synthesized co-occurrence matrices ZZT . Diagonal structure shows alignment of dominant conceptual directors, with varying degrees of alignment across the four m…
Figure 21
Figure 21. Figure 21: Examples of images generated with various prompts involving the [PITH_FULL_IMAGE:figures/full_fig_p033_21.png]
Figure 22
Figure 22. Figure 22: Examples of images generated with various prompts involving the [PITH_FULL_IMAGE:figures/full_fig_p034_22.png]
Figure 23
Figure 23. Figure 23: Additional image-caption pair examples from LAION-5B with matching images generated [PITH_FULL_IMAGE:figures/full_fig_p035_23.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

142 extracted references · 41 canonical work pages

  1. [1]

    Sora: Creating video from text, 2024

    OpenAI. Sora: Creating video from text, 2024. URL https://openai.com/sora/

    2024

  2. [2]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  3. [3]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pages 8821–8831. PMLR, 2021

    2021

  4. [4]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479–36494, 2022

    2022

  5. [5]

    GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mc- Grew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv:2112.10741, 2021

    arXiv 2021

  6. [6]

    Geco: Generative image- to-3d within a second

    Chen Wang, Jiatao Gu, Xiaoxiao Long, Yuan Liu, and Lingjie Liu. Geco: Generative image- to-3d within a second. arXiv preprint arXiv:2405.20327, 2024

    arXiv 2024

  7. [7]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022

    arXiv 2022

  8. [8]

    Texture: Text-guided texturing of 3d shapes

    Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or. Texture: Text-guided texturing of 3d shapes. In ACM SIGGRAPH 2023 conference proceedings, pages 1–11, 2023

    2023

Show all 142 references
  1. [9]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  2. [10]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  3. [11]

    Inner monologue: Embodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022

    2022 arXiv

  4. [12]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning, pages 9118–9147. PMLR, 2022

    2022

  5. [13]

    Text-guided controllable mesh refinement for interactive 3d modeling

    Yun-Chun Chen, Selena Ling, Zhiqin Chen, Vladimir G Kim, Matheus Gadelha, and Alec Ja- cobson. Text-guided controllable mesh refinement for interactive 3d modeling. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024

    2024

  6. [14]

    Smoodi: Stylized motion diffusion model

    Lei Zhong, Yiming Xie, Varun Jampani, Deqing Sun, and Huaizu Jiang. Smoodi: Stylized motion diffusion model. In European Conference on Computer Vision , pages 405–421. Springer, 2024

    2024

  7. [15]

    Meshgpt: Generating triangle meshes with decoder-only transformers

    Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern...

    2024

  8. [16]

    A very preliminary analysis of dall-e 2.arXiv preprint arXiv:2204.13807, 2022

    Gary Marcus, Ernest Davis, and Scott Aaronson. A very preliminary analysis of dall-e 2.arXiv preprint arXiv:2204.13807, 2022

    2022 arXiv

  9. [17]

    Discovering and validating ai errors with crowdsourced failure reports

    Ángel Alexander Cabrera, Abraham J Druck, Jason I Hong, and Adam Perer. Discovering and validating ai errors with crowdsourced failure reports. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2):1–22, 2021. 13

    2021

  10. [18]

    Generative artificial intelligence in creative contexts: a systematic review and future research agenda

    Rebecca Heigl. Generative artificial intelligence in creative contexts: a systematic review and future research agenda. Management Review Quarterly, pages 1–38, 2025

    2025

  11. [19]

    Discovering failure modes of text-guided diffusion models via adversarial search.arXiv preprint arXiv:2306.00974, 2023

    Qihao Liu, Adam Kortylewski, Yutong Bai, Song Bai, and Alan Yuille. Discovering failure modes of text-guided diffusion models via adversarial search.arXiv preprint arXiv:2306.00974, 2023

    2023 arXiv

  12. [20]

    Relations, negations, and numbers: Looking for logic in generative text-to-image models

    Colin Conwell, Rupert Tawiah-Quashie, and Tomer Ullman. Relations, negations, and numbers: Looking for logic in generative text-to-image models. arXiv preprint arXiv:2411.17066, 2024

    2024 arXiv

  13. [21]

    HandRefiner: Re- fining malformed hands in generated images by diffusion-based conditional inpainting

    Wenquan Lu, Yufei Xu, Jing Zhang, Chaoyue Wang, and Dacheng Tao. HandRefiner: Re- fining malformed hands in generated images by diffusion-based conditional inpainting. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 7085–7093, 2024

    2024

  14. [22]

    HanDiffuser: Text-to-image generation with realistic hand appearances

    Supreeth Narasimhaswamy, Uttaran Bhattacharya, Xiang Chen, Ishita Dasgupta, Saayan Mitra, and Minh Hoai. HanDiffuser: Text-to-image generation with realistic hand appearances. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2468–2479, 2024

    2024

  15. [23]

    Layout-agnostic scene text image synthesis with diffusion models

    Qilong Zhangli, Jindong Jiang, Di Liu, Licheng Yu, Xiaoliang Dai, Ankit Ramchandani, Guan Pang, Dimitris N Metaxas, and Praveen Krishnan. Layout-agnostic scene text image synthesis with diffusion models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (C...

    2024

  16. [24]

    TextInVision: Text and prompt complexity driven visual text generation benchmark

    Forouzan Fallah, Maitreya Patel, Agneet Chatterjee, Vlad I Morariu, Chitta Baral, and Yezhou Yang. TextInVision: Text and prompt complexity driven visual text generation benchmark. arXiv:2503.13730, 2025

    2025 arXiv

  17. [25]

    Text-to-image diffusion models cannot count, and prompt refinement cannot help

    Yuefan Cao, Xuyang Guo, Jiayan Huo, Yingyu Liang, Zhenmei Shi, Zhao Song, Jiahao Zhang, and Zhen Zhuang. Text-to-image diffusion models cannot count, and prompt refinement cannot help. arXiv preprint arXiv:2503.06884, 2025

    2025

  18. [26]

    Testing relational understanding in text-guided image generation

    Colin Conwell and Tomer Ullman. Testing relational understanding in text-guided image generation. arXiv preprint arXiv:2208.00005, 2022

    2022 arXiv

  19. [27]

    Blindspot: Hidden biases of good people

    Mahzarin R Banaji and Anthony G Greenwald. Blindspot: Hidden biases of good people . Bantam, 2016

    2016

  20. [28]

    Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models

    George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances i...

    2023

  21. [29]

    A study of the evaluation metrics for generative images containing combinational creativity

    Boheng Wang, Yunhuai Zhu, Liuqing Chen, Jingcheng Liu, Lingyun Sun, and Peter Childs. A study of the evaluation metrics for generative images containing combinational creativity. AI EDAM, 37:e11, 2023

    2023

  22. [30]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  23. [31]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514–7528, 2021

    2021

  24. [32]

    Image generation diversity issues and how to tame them

    Mischa Dombrowski, Weitong Zhang, Sarah Cechnicka, Hadrien Reynaud, and Bernhard Kainz. Image generation diversity issues and how to tame them. arXiv:2411.16171, 2024

    2024 arXiv

  25. [33]

    Anomaly score: Evaluating generative models and individual generated images based on complexity and vulnerability

    Jaehui Hwang, Junghyuk Lee, and Jong-Seok Lee. Anomaly score: Evaluating generative models and individual generated images based on complexity and vulnerability. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8754–8763, 2024. 14

    2024

  26. [34]

    A note on the evaluation of generative models

    Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. arXiv:1511.01844, 2015

    2015 arXiv

  27. [35]

    Reliable fidelity and diversity metrics for generative models

    Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In International conference on machine learning, pages 7176–7185. PMLR, 2020

    2020

  28. [36]

    Human evaluation of text-to-image models on a multi-task benchmark

    Vitali Petsiuk, Alexander E Siemenn, Saisamrit Surbehera, Zad Chin, Keith Tyser, Gregory Hunter, Arvind Raghavan, Yann Hicke, Bryan A Plummer, Ori Kerret, et al. Human evaluation of text-to-image models on a multi-task benchmark. arXiv:2211.12112, 2022

    2022 arXiv

  29. [37]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903–15935, 2023

    2023

  30. [38]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. 2023

    2023

  31. [39]

    Seeing what a GAN cannot generate

    David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, and Antonio Torralba. Seeing what a GAN cannot generate. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4502–4511, 2019

    2019

  32. [40]

    Deep inside convolutional networks: Visualising image classification models and saliency maps

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. Proceedings of the International Conference on Learning Representations (ICLR), 2013

    2013

  33. [41]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. Proceedings of the International Conference on Machine Learning (ICML), 2017

    2017

  34. [42]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017

    2017

  35. [43]

    Sanity checks for saliency maps

    Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in Neural Information Processing Systems (NIPS), 2018

    2018

  36. [44]

    Interpretation of neural networks is fragile

    Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017

    2017

  37. [45]

    Peter Hase and Mohit Bansal. Evaluating explainable ai: Which algorithmic explanations help users predict model behavior? Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2020

    2020

  38. [46]

    Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). Proceedings of the International Conference on Machine Learning (ICML), 2018

    2018

  39. [47]

    Network dissec- tion: Quantifying interpretability of deep visual representations

    David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissec- tion: Quantifying interpretability of deep visual representations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    2017

  40. [48]

    Craft: Concept recursive activation factorization for explainability

    Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. Craft: Concept recursive activation factorization for explainability. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  41. [49]

    Understanding video transformers via universal concept discovery

    Matthew Kowal, Achal Dave, Rares Ambrus, Adrien Gaidon, Konstantinos G Derpanis, and Pavel Tokmakov. Understanding video transformers via universal concept discovery. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 15

    2024

  42. [50]

    Sparse autoencoders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. ArXiv e-print, 2023

    2023

  43. [51]

    Towards monosemanticity: Decomposing language models with dictionary learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen,...

    2023

  44. [52]

    A holistic approach to unifying automatic concept extraction and concept importance estimation

    Thomas Fel, Victor Boutin, Mazda Moayeri, Remi Cadene, Louis Bethune, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  45. [53]

    A is for absorption: Studying feature splitting and absorption in sparse autoencoders

    David Chanin, James Wilken-Smith, Tomas Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders. ArXiv e-print, 2024

    2024

  46. [54]

    Relational composition in neural networks: A survey and call to action

    Martin Wattenberg and Fernanda B Viegas. Relational composition in neural networks: A survey and call to action. ArXiv e-print, 2024

    2024

  47. [55]

    Towards unifying interpretability and control: Evaluation via intervention

    Usha Bhalla, Suraj Srinivas, Asma Ghandeharioun, and Himabindu Lakkaraju. Towards unifying interpretability and control: Evaluation via intervention. ArXiv e-print, 2024

    2024

  48. [56]

    Archetypal SAE: Adaptive and stable dictionary learning for concept extraction in large vision models

    Thomas Fel, Ekdeep Singh Lubana, Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal SAE: Adaptive and stable dictionary learning for concept extraction in large vision models. 2025

    2025

  49. [57]

    Prompt-CAM: A simpler interpretable transformer for fine-grained analysis

    Arpita Chowdhury, Dipanjyoti Paul, Zheda Mai, Jianyang Gu, Ziheng Zhang, Kazi Sajeed Mehrab, Elizabeth G Campolongo, Daniel Rubenstein, Charles V Stewart, Anuj Karpatne, et al. Prompt-CAM: A simpler interpretable transformer for fine-grained analysis. arXiv:2501.09333, 2025

    2025 arXiv

  50. [58]

    GAN dissection: Visualizing and understanding generative adversarial networks

    David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B Tenenbaum, William T Freeman, and Antonio Torralba. GAN dissection: Visualizing and understanding generative adversarial networks. 2018

    2018

  51. [59]

    What the DAAM: Interpreting stable diffusion using cross attention

    Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. What the DAAM: Interpreting stable diffusion using cross attention. 2022

    2022

  52. [60]

    Qualitative failures of image generation models and their application in detecting deepfakes

    Ali Borji. Qualitative failures of image generation models and their application in detecting deepfakes. Image and Vision Computing, 137:104771, 2023

    2023

  53. [61]

    Hierarchical text-conditional image generation with CLIP latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    2022 arXiv

  54. [62]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  55. [63]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023

    2023

  56. [64]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR, 2021

    2021

  57. [65]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    2022

  58. [66]

    VDT: General-purpose video diffusion transformers via mask modeling

    Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu, Ping Luo, and Mingyu Ding. VDT: General-purpose video diffusion transformers via mask modeling. arXiv preprint arXiv:2305.13311, 2023. 16

    2023 arXiv

  59. [67]

    ModelScope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. ModelScope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023

    2023 arXiv

  60. [68]

    AnimateDiff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. AnimateDiff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023

    2023 arXiv

  61. [69]

    AnimateDiff-Lightning: Cross-model diffusion distillation

    Shanchuan Lin and Xiao Yang. AnimateDiff-Lightning: Cross-model diffusion distillation. arXiv preprint arXiv:2403.12706, 2024

    2024 arXiv

  62. [70]

    CogVideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. CogVideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022

    2022 arXiv

  63. [71]

    VideoCrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. VideoCrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023

    arXiv

  64. [72]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  65. [73]

    Magic3D: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3D: High-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    2023

  66. [74]

    Shap-E: Generating conditional 3d implicit functions

    Heewoo Jun and Alex Nichol. Shap-E: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023

    2023 arXiv

  67. [75]

    Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12619–12629, 2023

    2023

  68. [76]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  69. [77]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  70. [78]

    Stable Diffusion 2.0 Release, 2022

    Stability AI. Stable Diffusion 2.0 Release, 2022. URL https://stability.ai/news/ stable-diffusion-v2-release

    2022

  71. [79]

    Introducing Stable Diffusion 3, 2024

    Stability AI. Introducing Stable Diffusion 3, 2024. URL https://stability.ai/news/ stable-diffusion-3

    2024

  72. [80]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  73. [81]

    StyleDrop: Text-to-image generation in any style

    Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. StyleDrop: Text-to-image generation in any style. arXiv preprint arXiv:2306.00983, 2023

    2023 arXiv

  74. [82]

    Arbitrary style guidance for enhanced diffusion-based text-to-image generation

    Zhihong Pan, Xin Zhou, and Hao Tian. Arbitrary style guidance for enhanced diffusion-based text-to-image generation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 4461–4471, 2023

    2023

  75. [83]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 17

    2023

  76. [84]

    DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aber- man. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 225...

    2023

  77. [85]

    Id-booth: Identity-consistent face generation with diffusion models

    Darian Tomavsevic, Fadi Boutros, Chenhao Lin, Naser Damer, Vitomir Struc, and Peter Peer. Id-booth: Identity-consistent face generation with diffusion models. arXiv preprint arXiv:2504.07392, 2025

    2025

  78. [86]

    Kandinsky: An improved text-to-image synthesis with image prior and latent diffusion

    Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva, Vladimir Arkhipkin, Igor Pavlov, Ilya Ryabov, Angelina Kuts, Alexander Panchenko, Andrey Kuznetsov, and Denis Dimitrov. Kandinsky: An improved text-to-image synthesis with image prior and latent diffusion. arXiv preprin...

    arXiv

  79. [87]

    Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023

    arXiv

  80. [88]

    DeepFloyd IF: A Powerful Open-Source Text-to-Image Model, 2023

    Stability AI. DeepFloyd IF: A Powerful Open-Source Text-to-Image Model, 2023. URL https://stability.ai/news/deepfloyd-if-text-to-image-model

    2023

  81. [89]

    Announcing Black Forest Labs, 2024

    Black Forest Labs. Announcing Black Forest Labs, 2024. URL https://blackforestlabs. ai/announcing-black-forest-labs/

    2024

  82. [90]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. Proceed- ings of the...

    2020

  83. [91]

    Scaling autoregressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022

    2022 arXiv

  84. [92]

    Deep learning scaling is predictable, empirically

    Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409, 2017

    2017 arXiv

  85. [93]

    LAION-5B: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. LAION-5B: An open large-scale dataset for training next generation image-text models. Advances in neural informa...

    2022

  86. [94]

    Common crawl corpus

    Common Crawl. Common crawl corpus. https://commoncrawl.org, 2017

    2017

  87. [95]

    COYO-700M: Image-text pair dataset, 2022

    Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. COYO-700M: Image-text pair dataset, 2022

    2022

  88. [96]

    Conceptual Captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual Captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of ACL, 2018

    2018

  89. [97]

    Into the LAION’s den: Investigating hate in multimodal datasets

    Abeba Birhane, Sanghyun Han, Vishnu Boddeti, Sasha Luccioni, et al. Into the LAION’s den: Investigating hate in multimodal datasets. Advances in neural information processing systems, 36:21268–21284, 2023

    2023

  90. [98]

    The dark side of dataset scaling: Evaluating racial classification in multimodal models

    Abeba Birhane, Sepehr Dehdashtian, Vinay Prabhu, and Vishnu Boddeti. The dark side of dataset scaling: Evaluating racial classification in multimodal models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency , pages 1229–1244, 2024

    2024

  91. [99]

    The bias amplification paradox in text-to- image generation

    Preethi Seshadri, Sameer Singh, and Yanai Elazar. The bias amplification paradox in text-to- image generation. arXiv preprint arXiv:2308.00755, 2023. 18

    2023 arXiv

  92. [100]

    Multimodal datasets: Misog- yny, pornography, and malignant stereotypes

    Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: Misog- yny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963, 2021

    2021 arXiv

  93. [101]

    Identifying and eliminating CSAM in generative ml training data and models

    David Thiel. Identifying and eliminating CSAM in generative ml training data and models. Stanford Internet Observatory, Cyber Policy Center, December, 23:3, 2023

    2023

  94. [102]

    What makes ImageNet look unlike LAION

    Ali Shirali and Moritz Hardt. What makes ImageNet look unlike LAION. arXiv preprint arXiv:2306.15769, 2023

    2023 arXiv

  95. [103]

    Improving multimodal datasets with image captioning

    Thao Nguyen, Samir Yitzhak Gadre, Gabriel Ilharco, Sewoong Oh, and Ludwig Schmidt. Improving multimodal datasets with image captioning. Advances in Neural Information Processing Systems, 36:22047–22069, 2023

    2023

  96. [104]

    Emergence of simple-cell receptive field properties by learning a sparse code for natural images

    Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 1996

    1996

  97. [105]

    Sparse and redundant representations: from theory to applications in signal and image processing

    Michael Elad. Sparse and redundant representations: from theory to applications in signal and image processing. 2010

    2010

  98. [106]

    Sparse modeling for image and vision processing

    Julien Mairal, Francis Bach, and Jean Ponce. Sparse modeling for image and vision processing. Foundations and Trends in Computer Graphics and Vision, 2014

    2014

  99. [107]

    Compressed sensing

    David L Donoho. Compressed sensing. IEEE Transactions on Information Theory, 2006

    2006

  100. [108]

    Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information

    Emmanuel J Candès, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006

    2006

  101. [109]

    Least squares quantization in pcm

    Stuart Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 1982

    1982

  102. [110]

    Learning the parts of objects by non-negative matrix factorization

    Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 1999

    1999

  103. [111]

    Nonnegative matrix factorization

    Nicolas Gillis. Nonnegative matrix factorization. 2020

    2020

  104. [112]

    Sparse principal component analysis

    Hui Zou, Trevor Hastie, and Robert Tibshirani. Sparse principal component analysis. Journal of Computational and Graphical Statistics, 2006

    2006

  105. [113]

    K-svd: An algorithm for designing overcomplete dictionaries for sparse representation

    Michal Aharon, Michael Elad, and Alfred Bruckstein. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006

    2006

  106. [114]

    Online dictionary learning for sparse coding

    Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online dictionary learning for sparse coding. Proceedings of the International Conference on Machine Learning (ICML), 2009

    2009

  107. [115]

    Structured sparse principal component analysis

    Rodolphe Jenatton, Guillaume Obozinski, and Francis Bach. Structured sparse principal component analysis. International Conference on Artificial Intelligence and Statistics, 2010

    2010

  108. [116]

    Exact recovery of sparsely-used dictionaries

    Daniel A Spielman, Huan Wang, and John Wright. Exact recovery of sparsely-used dictionaries. The Journal of Machine Learning Research (JMLR), 2012

    2012

  109. [117]

    Statistical limits of dictionary learning: random matrix theory and the spectral replica method

    Jean Barbier and Nicolas Macris. Statistical limits of dictionary learning: random matrix theory and the spectral replica method. Physical Review E, 2022

    2022

  110. [118]

    Convolutional dictionary learning via local processing

    Vardan Papyan, Yaniv Romano, and Michael Elad. Convolutional dictionary learning via local processing. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017

    2017

  111. [119]

    Codebook features: Sparse and discrete interpretability for neural networks

    Alex Tamkin, Mohammad Taufeeque, and Noah D Goodman. Codebook features: Sparse and discrete interpretability for neural networks. ArXiv e-print, 2023

    2023

  112. [120]

    Emergence and evolution of interpretable concepts in diffusion models

    Berk Tinaz, Zalan Fabian, and Mahdi Soltanolkotabi. Emergence and evolution of interpretable concepts in diffusion models. arXiv:2504.15473, 2025. 19

    2025

  113. [121]

    Interpretable generative models through post-hoc concept bottlenecks

    Akshay Kulkarni, Ge Yan, Chung-En Sun, Tuomas Oikarinen, and Tsui-Wei Weng. Interpretable generative models through post-hoc concept bottlenecks. arXiv preprint arXiv:2503.19377, 2025

    2025 arXiv

  114. [122]

    Post-hoc concept bottleneck models

    Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205.15480, 2022

    2022 arXiv

  115. [123]

    Improved precision and recall metric for assessing generative models

    Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019

    2019

  116. [124]

    Assessing sample quality via the latent space of generative models

    Jingyi Xu, Hieu Le, and Dimitris Samaras. Assessing sample quality via the latent space of generative models. In European Conference on Computer Vision, pages 449–464. Springer, 2024

    2024

  117. [125]

    Measuring the interpretability of unsupervised representations via quantized reverse probing

    Iro Laina, Yuki M Asano, and Andrea Vedaldi. Measuring the interpretability of unsupervised representations via quantized reverse probing. arXiv:2209.03268, 2022

    2022 arXiv

  118. [126]

    Self-supervised learning with data augmentations provably isolates content from style

    Julius V on Kugelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Scholkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  119. [127]

    Challenging common assumptions in the unsupervised learning of disentangled representations

    Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Scholkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. Proceedings of the International Conference on Machine Learning (...

    2019

  120. [128]

    Contrastive learning inverts the data generating process

    Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Bren- del. Contrastive learning inverts the data generating process. Proceedings of the International Conference on Machine Learning (ICML), 2021

    2021

  121. [129]

    The incomplete rosetta stone problem: Identifiability results for multi-view nonlinear ica

    Luigi Gresele, Paul K Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Scholkopf. The incomplete rosetta stone problem: Identifiability results for multi-view nonlinear ica. Uncertainty in Artificial Intelligence, 2020

    2020

  122. [130]

    Independent mechanism analysis, a new concept? Advances in Neural Information Processing Systems (NeurIPS), 2021

    Luigi Gresele, Julius V on Kugelgen, Vincent Stimper, Bernhard Scholkopf, and Michel Besserve. Independent mechanism analysis, a new concept? Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  123. [131]

    Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task

    Maya Okawa, Ekdeep S Lubana, Robert Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task. Advances in Neural Information Processing Systems, 36:50173–50195, 2023

    2023

  124. [132]

    Emergence of hidden capabilities: Exploring learning dynamics in concept space

    Core Francisco Park, Maya Okawa, Andrew Lee, Ekdeep S Lubana, and Hidenori Tanaka. Emergence of hidden capabilities: Exploring learning dynamics in concept space. Advances in Neural Information Processing Systems, 37:84698–84729, 2024

    2024

  125. [133]

    Variational autoencoders and nonlinear ica: A unifying framework

    Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. Proceedings of the International Conference on Machine Learning (ICML), 2020

    2020

  126. [134]

    Nonlinear ICA using auxiliary variables and generalized contrastive learning

    Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ICA using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859–868. PMLR, 2019

    2019

  127. [135]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. ArXiv e-print, 2023

    2023

  128. [136]

    Toy models of superposition

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposi...

    2022

  129. [137]

    Scaling and evaluating sparse autoencoders.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  130. [138]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua ...

    2024

  131. [139]

    Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, Janos Kramar, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. ArXiv e-print, 2024

    2024

  132. [140]

    Understanding the effects of rlhf on llm generalisation and diversity

    Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of rlhf on llm generalisation and diversity. arXiv preprint arXiv:2310.06452, 2023

    arXiv

  133. [141]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  134. [142]

    <blindspot>

    Colin McDiarmid et al. On the method of bounded differences. Surveys in combinatorics, 141 (1):148–188, 1989. 21 Appendix Table of Contents A Experimental Setup 23 A.1 Observation Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 A.2 Synthesized Images...

    1989

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.