Pith. sign in

REVIEW 3 major objections 7 minor 47 references

Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CASO replaces text prompts with classifier-optimized semantic embeddings and claims these embeddings converge to attribute class means, enabling disentangled, cross-domain editing.

desk verdict A useful prompt-free editing method whose theoretical justification overreaches; the empirical core is worth a look, but Proposition 2 should be rewritten or cut. read the letter →

arxiv 2505.14254 v1 pith:YWC4S5WS submitted 2025-05-20 cs.CV

Yuanyuan Chang , Yinghua Yao , Tao Qin , Mengmeng Wang , Ivor Tsang , Guang Dai This is my paper · ORCID
classification cs.CV
keywords text-to-imagediffusionsemanticembeddingoptimizationclassifierguidanceimageeditingneuralcollapsedisentangledcross-domaingeneralizationStable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a trainable continuous embedding, optimized only through gradients of a frozen attribute classifier, can replace hand-written text prompts when steering Stable Diffusion to edit images. At convergence, the paper argues, the embedding is pinned to the globally-centered class mean $\mu_a$ of the target attribute in the classifier's feature space, so it is the optimal semantic direction at the dataset level. This principle is used to edit human faces, animals, and cars with better disentanglement and much stronger cross-domain generalization than text-prompt methods or unsupervised latent-direction methods. A sympathetic reading is that any sufficiently well-trained attribute classifier can be turned into a precise, prompt-free editing controller for a text-to-image diffusion model.

What carries the argument

The central object is a trainable semantic embedding $e_a$ inserted into Stable Diffusion's conditioning path in place of text token embeddings. It is optimized end-to-end through classifier-free guidance so that edited images are classified as the target attribute class, with a reconstruction loss preserving unrelated details. The theoretical machinery is neural collapse: Theorem 1 and Theorem 3, imported from the neural-collapse literature, state that a well-trained classifier's last-layer weights converge to the centered class means forming an equiangular tight frame. Proposition 2 extends this to the distribution of generated images and concludes that the fixed classifier's weights align with $\mu'_a(e_a)$, so the embedding itself is pinned by $\mu_a$. A one-step denoising approximation of the DDIM inversion/editing pipeline, with a Jensen-gap bound borrowed from diffusion posterior sampling, keeps the optimization cheap enough to run on 100–200 images.

What would settle it

Retrain the classifier on the images generated after optimizing $e_a$ and compare its last-layer weights with the original classifier's weights; if they move substantially, Proposition 2's fixed-classifier alignment fails. A cheaper check is to replace the classifier's last layer with random fixed weights and repeat the optimization: if the embeddings still produce the target edit, the claim that the optimal embedding is determined by the class mean is not supported.

Watch

Extended reading notes

Core claim

The central claim is that optimizing $\{e_a\}_{a=1}^K$ with the edit loss $L_{\mathrm{edit}} = \mathbb{E}_{x,a}[\ell_c(F(G(x,e_a)), a)]$, while keeping the classifier $F$ fixed, makes the normalized last-layer weight $w_a/\|w_a\|_2$ align with the normalized globally-centered class mean $\mu'_a(e_a)/\|\mu'_a(e_a)\|_2$ of images generated under $e_a$. Because the same alignment already holds for the classifier on its real training data (neural collapse), the paper concludes that the optimal attribute embedding in Eq. (3) is determined by the attribute class mean $\mu_a$. The authors take this as theoretical justification that the learned embeddings encode exact, dataset-level attribute semantics, and they support it with experiments showing that edited-image features land on the target class mean in T-SNE visualizations.

Load-bearing premise

The proof assumes that the frozen classifier's last-layer weights are the weights that would be learned on the images generated during embedding optimization, even though the classifier is never retrained on those generated images.

Editorial extensions

If this is right

  • Users can edit by selecting an attribute class and tuning the classifier-free guidance scale; a negative scale produces reverse edits such as sparse eyebrows from the bushy-eyebrows embedding.
  • Attribute embeddings trained on different datasets can be concatenated for multi-attribute edits, and the order of concatenation does not affect the result, which the paper reads as evidence of semantic decoupling.
  • Guiding reconstruction with the embedding of the image's own class improves reconstruction quality and lowers FID, so the method is useful beyond editing.
  • With a VGG16 classifier and roughly 100–200 training images, the learned embedding transfers to artistic, anime, and other unseen styles, which the paper attributes to capturing dataset-level semantics rather than dataset-specific texture.
  • When sampled from Gaussian noise, the learned embeddings produce images with the target attribute more reliably than text prompts or NoiseCLR directions, as measured by Inception Score and visual inspection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's experiments, if Proposition 2 holds then the editing direction should be readable directly from the classifier's feature means without any embedding optimization; comparing such read-off directions with CASO's optimized embeddings would be a clean test of the theory.
  • The fixed-classifier assumption in the proof implies a practical prediction the authors do not state: attribute classifiers with strong neural collapse should yield more stable and more transferable edits, while classifiers trained with heavy regularization or imbalanced data should give degraded directions.
  • A natural extension of the method is to treat the per-class embedding as an attribute-strength dial for a whole family of diffusion backbones that accept continuous conditioning, not just Stable Diffusion-v1.5.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes CASO (Classifier-Guided Semantic Optimization), a method for text-to-image editing with Stable Diffusion that replaces text prompts with learnable continuous semantic embeddings per attribute class. A frozen attribute classifier provides an edit loss on generated images, and a reconstruction loss preserves the input structure; the embeddings are trained on a small set of images without updating the diffusion model. The paper claims a theoretical justification (Proposition 2) that these learned embeddings are optimal in the sense of being determined by the attribute class mean of the classifier, enabling disentangled and cross-domain edits. Experiments on faces, animals, and cars compare LPIPS against several baselines and show qualitative generalization to non-photorealistic styles.

Significance. If the theoretical claim were sound, the paper would offer a principled, prompt-free editing mechanism that connects classifier weights to semantic directions in a generative latent space, with practical value due to its lightweight training (100–200 images) and single-step edit capability. The empirical results are promising in their cross-domain generalization and the paper ships code and detailed appendices. However, the central theoretical contribution has a serious proof gap, as described in the major comments, and the quantitative evaluation relies almost exclusively on LPIPS, which does not directly measure editing success. The significance is therefore conditional on either repairing the theory or reframing the contributions as purely empirical.

major comments (3)
  1. [Section 3.2, Eq. (10) vs. Proposition 2] The proof of Proposition 2 applies Theorem 3 to write W = (1/K) M(e_a)^T (Sigma'_B(e_a))^dagger, but Theorem 3 characterizes the terminal-phase optimum of a classifier trained on a given dataset. Here the classifier F is fixed while e_a is optimized; the minimization of Eq. (3) does not train the classifier on the generated features. For a fixed full-row-rank last layer, any feature h satisfying W h = y_a - b for the corresponding class yields zero edit loss, so the optimization does not force neural collapse or alignment between W and the generated class means. Consequently Eq. (5) does not follow, and the claim that 'the optimal attribute embedding e_a is determined by the attribute class mean mu_a' is unsupported. This is load-bearing because the abstract and contribution list explicitly claim a theoretical justification.
  2. [Section 3.2, Eq. (10) vs. Proposition 2] The theoretical result concerns the exact editing process G(x, e_a) = D(hat z_0) from full DDIM inversion plus iterative denoising, whereas the training loss in Eq. (3) is computed with the one-step approximation G(x, e_a) = D((hat z_L - sqrt(1-alpha_L) tilde_epsilon_theta(hat z_L, e_a))/sqrt(alpha_L)) from Eq. (10). The paper does not show that Proposition 2 holds for this approximate G, nor that the Jensen-gap bound in Eq. (11) is small in the training regime; the bound depends on L and on gradients of the classifier, but no quantitative argument is given. Thus, even if Proposition 2 were correct for the exact G, it would not directly justify the embeddings that are actually optimized during training.
  3. [Section 4.2, Tables 1 and 2] The quantitative comparisons report only LPIPS (lower is better), which measures perceptual distance between the input and the edited image. This metric cannot distinguish successful editing from weak editing, since a method that barely changes the image will achieve a low LPIPS. The paper does not report any attribute-level accuracy on the edited images (e.g., using the attribute classifier or an independent one), nor a user study, so the central empirical claims of accurate and disentangled editing are not directly quantified. Adding such metrics would substantially strengthen the evaluation.
minor comments (7)
  1. [Section 3.2, Definition 1] The notation E_i[h_{i,a}] in the definition of mu_a implicitly conditions on the class a, but this is not stated; the expectation should be written explicitly as over samples i in class a to avoid ambiguity.
  2. [Appendix A, Eq. (20)] The phrase 'no-zero' should be 'nonzero' throughout the appendix.
  3. [Section 4.1] The notation for timesteps is inconsistent: the paper says 'L is set to 0.3T for human face and 0.4T for others' and then 't in [0.1T, 0.3T]' but later states 'T = 50'; please define L, t, and T explicitly and use them consistently.
  4. [Section 4.2] The sentence 'No more than 10% of the directions learned by NoiseCLR are semantic meaningful' should be 'semantically meaningful'.
  5. [Appendix A, after Eq. (19)] The statement that 'M(e_a) has exactly K-1 no-zero and equal singular values' is asserted after invoking Theorem 3; it would be clearer to state explicitly that this follows from the equiangular tight frame property in Eq. (19).
  6. [Section 2] The related work section contains several long, run-on sentences and informal phrasings (e.g., 'have become a hot spot of current research'); a copyedit pass would improve readability.
  7. [Tables 1, 2, and Figure 2] The attribute 'Old' is capitalized inconsistently; please unify the treatment of attribute names across tables and figures.

Circularity Check

1 steps flagged · score 6.0 of 10

Proposition 2's proof assumes the fixed classifier is retrained on generated data; the claimed optimality of the learned embedding is circular.

  1. ansatz smuggled in via citation [Appendix A, proof of Proposition 2, Eq. (21)]
    "According to Theorem 3, when Eq. (3)(refer to the main text), we can obtain W = 1 K M (ea)⊤(Σ′ B(ea))†."

    Theorem 3 (Han et al. 2021; Papyan et al. 2020) describes the terminal-phase optimum of a classifier trained on a fixed dataset: its weights W converge to (1/K) M^T Σ_B^†, where M is the matrix of class means of the training features. In Eq. (3) of the main text, however, only the embeddings e_a are optimized; the classifier F, and hence W, is explicitly frozen. Nothing in minimizing the edit loss retrains W on the generated images or forces the generated feature distribution to satisfy the neural-collapse conditions (Σ_W -> 0, ETF geometry) that Theorem 3 presupposes. Writing W = (1/K) M(e_a)^T (Σ'_B(e_a))^† for the generated-data means is therefore exactly the alignment that Proposition 2 is supposed to prove: it assumes w_a is proportional to µ'_a(e_a). The rest of the proof (Eqs.

full rationale

The empirical CASO pipeline is largely self-contained: it optimizes embeddings with a frozen classifier and evaluates them against external baselines, so the method's practical contribution does not rest on a self-citation chain. The circularity is confined to the theoretical justification. The key step is the proof of Proposition 2 in Appendix A, where Eq. (21) applies a neural-collapse theorem for trained classifiers to a classifier whose weights are fixed, treating the generated-data class means as if they determine W. That is the very claim to be established. Consequently, the statement that the learned embeddings are the optimal representation of attribute semantics is unsupported by the proof as written; it reduces to an assumption smuggled in via Theorem 3. Because the central theoretical claim is circular but the empirical work retains independent content, a score of 6 is appropriate rather than 0-2.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claim rests on several hand-chosen hyperparameters and on a domain assumption about the transferability of classifiers. The most fragile item is the ad hoc assumption that the fixed classifier weights align with generated-image class means, which is needed for the theoretical proof but not derived from the optimization.

free parameters (5)
  • Diffusion noise timestep L for training = 0.3T (faces), 0.4T (others)
    Chosen by hand to balance edit strength and classifier accuracy; affects the approximation in Eq. (9).
  • Start edit time for guidance at inference = t in [0.1T,0.3T] for subtle features, [0.8T,0.9T] for coarse changes
    Chosen empirically per attribute type; directly controls which denoising steps receive the embedding.
  • Classifier-free guidance scale lambda = 10 (default), smaller for reconstruction
    Controls edit intensity; used for interpolation and reverse edits; values are chosen per experiment.
  • Reconstruction loss weight gamma = not reported
    The final objective Eq. (13) includes gamma but no value or schedule is given, so the balance between edit and reconstruction is unspecified.
  • Number of training images = 100-200 (or ~20 minimum)
    Empirically chosen; more images improve results (Appendix F.3).
assumptions (4)
  • standard math Neural collapse results (Theorem 1 and Theorem 3 from Han et al. 2021 and Papyan et al. 2020) apply to the classifier F on real training data.
    These are cited external theorems; the paper relies on them for the classical classifier setting.
  • ad hoc to paper The fixed classifier's last-layer weights w_a align with the class means of generated images mu'_a(e_a) (Eq. 21 in Appendix A).
    This is the load-bearing assumption used to prove Proposition 2; it treats the fixed classifier as optimal for the generated-data distribution without retraining, which is not established by the optimization.
  • domain assumption The Jensen-gap approximation in Eq. (9) is accurate for the chosen L, so the classification loss on the one-step prediction is a valid proxy.
    Remark 1 gives an upper bound but does not verify it numerically for the actual L and classifier; this underpins the training efficiency approximation.
  • domain assumption The classifier F is well-trained and transferable to target editing domains (e.g., faces, animals, cars).
    The whole method assumes the attribute classifiers encode the desired semantics and that optimizing embeddings on one domain (e.g., FFHQ) generalizes to other domains.
invented entities (1)
  • Learnable semantic embedding e_a per attribute class independent evidence
    purpose: Acts as a continuous condition replacing text prompts to guide Stable Diffusion toward the desired attribute.
    The embedding is a new learned vector; it generates images that can be evaluated by classifiers or human users, so it has an external falsifiable handle. It is not a physical entity but a new model parameter.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization." pith.science (2026). https://pith.science/paper/YWC4S5WS

@misc{pith2026250514254,
  author       = {Pith},
  title        = {Pith review of: Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YWC4S5WS}},
  note         = {Machine review of arXiv:2505.14254}
}
read the original abstract

Text-to-image diffusion models have emerged as powerful tools for high-quality image generation and editing. Many existing approaches rely on text prompts as editing guidance. However, these methods are constrained by the need for manual prompt crafting, which can be time-consuming, introduce irrelevant details, and significantly limit editing performance. In this work, we propose optimizing semantic embeddings guided by attribute classifiers to steer text-to-image models toward desired edits, without relying on text prompts or requiring any training or fine-tuning of the diffusion model. We utilize classifiers to learn precise semantic embeddings at the dataset level. The learned embeddings are theoretically justified as the optimal representation of attribute semantics, enabling disentangled and accurate edits. Experiments further demonstrate that our method achieves high levels of disentanglement and strong generalization across different domains of data.

Figures

Figures reproduced from arXiv: 2505.14254 by the authors.

Figure 1
Figure 1. ClAssifier-guide Semantic Optimization (CASO). The trainable continuous semantic embedding for the target attribute a, guides Stable Diffusion for desired edits. 2024]. The rapid rise in popularity of text-to-image diffu￾sion models, such as Stable Diffusion (SD) [Rombach et al., 2022], DeepFloyd IF [Saharia et al., 2022], and Latent Con￾sistency Models [Luo et al., 2023], has further inspired re￾searchers to achiev… view at source ↗
Figure 2
Figure 2. CASO Edit Result. Our method generalizes well to data with different styles. using real face data, their applicability often diminishes when extended to other types of images, such as artistic or anime￾style faces. Moreover, they can only obtain a limited number of subtle editing directions, often lack the ability to specify attributes, and rely on the clear structure or inherent variance within the dataset itself. … view at source ↗
Figure 3
Figure 3. Comparison of different methods for attribute “Mustache”. Our method shows the best generalization because it captures the exact semantics at the dataset level. learned by NoiseCLR are semantic meaningful. LOCO is only suitable for local editing, and some major structural changes such as cat→dog are not feasible. 3 Method In this section, we describe our proposed method. First, we briefly discuss the background of l… view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: CASO Interpolation Results. Our method allow users to implement fine-grained editing and bidirectional editing by simply changing the classifier free guidance scale. where Ezˆ0∼p(ˆz0|zˆL) [ˆz0] = zˆL− √ 1−αLϵ˜θ(ˆzL,ea) √αL . Now the decoded image xˆ in Eq. (3) can be: …
Figure 6
Figure 6. Figure 6: Generation Result. We use different direction: learned from NoiseCLR, learned from ours, and two kinds of text prompt (see Appendix) to guide the generation of images from Gaussian noise with corresponding attributes and calculate their IS scores. Some examples are sho…
Figure 5
Figure 5. Figure 5: CASO Muti-attribute Edit. In complex and challenging scenarios, our method can still achieve perfect editing [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 8
Figure 8. Figure 8: Improve image quality during editing and reconstruc￾tion. Our guidance can not only faithfully reconstruct the structure of the original image, but also achieves the editing of the target at￾tribute. mean, consistent with our theoretical result. This demon￾strate that …
Figure 7
Figure 7. Figure 7: T-SNE of cat→dog image edit. Red dot: real cat images; blue dots: real dog images; black dots: real cat images randomly sampled for editing; green dots: the generated images (to be edited as dog from the sampled cat images). To demonstrate that our model can learn the …
Figure 9
Figure 9. Figure 9: Sample results generated with the same random seed but [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: The reconstruction results of Fig. 6b (On the far left is [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Details about how our embedding are injected into the [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Effect of using different numbers of images for training. [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: T-LOCO edit for cat→dog. Because of its poor effect, we do not conduct statistics of the relevant results. Input 20 25 30 35 40 45 Without Reconstruction loss [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: With T = 50, comparison of the effect of starting editing at different timesteps. G Guidance Generation We show more results of sample from Gaussian noise with different guidance for attribute “Bushy Eyebrows” ( [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]
Figure 15
Figure 15. Figure 15: Generation with different guidance for attribute “Bushy Eyebrows”. [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: Our embeddings can help in faster and better reconstruction of images of the same class. [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 17
Figure 17. Figure 17: Our embeddings can help in faster and better reconstruction of images of the same class. [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 25 canonical work pages

  1. [1]

    Sega: Instructing text-to-image models using semantic guidance

    [Brack et al., 2023] Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek, Patrick Schramowski, and Kristian Kersting. Sega: Instructing text-to-image models using semantic guidance. Advances in Neural Information Processing Systems, 36:25365–25389,

    work page 2023

  2. [2]

    , wK]T ∈ RK×p, where K is the number of classes, p is the dimension of the linear classifier

    The linear classifier can be represented by weights W = [ w1, w2, . . . , wK]T ∈ RK×p, where K is the number of classes, p is the dimension of the linear classifier. We define the between-class covariance as: ΣB = Ea µaµ⊤ a , (14) the within-class covariance: ΣW = Ei,a (hi,a − Ei[hi,a])(hi,a − Ei[hi,a])⊤ , (15) the train total covariance: ΣT = Ei,a (hi,a ...

    work page 2020

  3. [6]

    Diffusion models beat gans on image synthe- sis

    [Dhariwal and Nichol, 2021] Prafulla Dhariwal and Alexan- der Nichol. Diffusion models beat gans on image synthe- sis. Advances in neural information processing systems , 34:8780–8794,

    2021

  4. [9]

    Gen- erative adversarial nets

    [Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. Advances in neural information processing systems, 27,

    2014

  5. [10]

    Neural collapse under mse loss: Proximity to and dynamics on the central path

    [Han et al., 2021] XY Han, Vardan Papyan, and David L Donoho. Neural collapse under mse loss: Proximity to and dynamics on the central path. arXiv preprint arXiv:2106.02073,

    arXiv 2021

  6. [12]

    Prompt-to-prompt image editing with cross attention con- trol

    [Hertz et al., 2022b] Amir Hertz, Ron Mokady, Jay Tenen- baum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention con- trol. arXiv preprint arXiv:2208.01626,

    arXiv

  7. [13]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    [Heusel et al., 2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30,

    work page 2017

  8. [15]

    Denoising diffusion probabilistic models

    [Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840–6851,

    2020

Show all 47 references
  1. [16]

    A style-based generator architecture for generative adversarial networks

    [Karras et al., 2019] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4396–4405,

    2019

  2. [18]

    Diffusionclip: Text-guided diffusion mod- els for robust image manipulation

    [Kim et al., 2022] Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Diffusionclip: Text-guided diffusion mod- els for robust image manipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2426–2435,

    2022

  3. [19]

    Auto-encoding varia- tional bayes

    [Kingma, 2013] Diederik P Kingma. Auto-encoding varia- tional bayes. arXiv preprint arXiv:1312.6114,

    2013 arXiv

  4. [22]

    Diffusion-based conditional image editing through optimized inference with guidance.arXiv preprint arXiv:2412.15798,

    [Lee et al., 2024] Hyunsoo Lee, Minsoo Kang, and Bo- hyung Han. Diffusion-based conditional image editing through optimized inference with guidance.arXiv preprint arXiv:2412.15798,

    2024 arXiv

  5. [23]

    Compositional vi- sual generation with composable diffusion models

    [Liu et al., 2022] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional vi- sual generation with composable diffusion models. In Eu- ropean Conference on Computer Vision , pages 423–439. Springer,

    2022

  6. [24]

    Decoupled weight decay regularization

    [Loshchilov, 2017] I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101,

    2017 arXiv

  7. [25]

    Hierar- chical diffusion autoencoders and disentangled image ma- nipulation

    [Lu et al., 2024] Zeyu Lu, Chengyue Wu, Xinyuan Chen, Yaohui Wang, Lei Bai, Yu Qiao, and Xihui Liu. Hierar- chical diffusion autoencoders and disentangled image ma- nipulation. In Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision, pages 5374– 5383,

    2024

  8. [26]

    Latent consistency models: Syn- thesizing high-resolution images with few-step inference

    [Luo et al., 2023] Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Syn- thesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378,

    2023 arXiv

  9. [27]

    Null-text inver- sion for editing real images using guided diffusion models

    [Mokady et al., 2022] Ron Mokady, Amir Hertz, Kfir Aber- man, Yael Pritch, and Daniel Cohen-Or. Null-text inver- sion for editing real images using guided diffusion models. arXiv preprint arXiv:2211.09794,

    2022 arXiv

  10. [28]

    Prevalence of neural collapse during the terminal phase of deep learning training

    [Papyan et al., 2020] Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the Na- tional Academy of Sciences, 117(40):24652–24663,

    2020

  11. [29]

    Diffusion autoencoders: Toward a meaningful and decodable representation

    [Preechakul et al., 2022] Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwa- janakorn. Diffusion autoencoders: Toward a meaningful and decodable representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pa...

    2022

  12. [30]

    Learning transferable visual models from nat- ural language supervision

    [Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International confere...

    2021

  13. [31]

    High-resolution image synthesis with latent diffusion models

    [Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684–10695,

    2022

  14. [32]

    Photorealistic text-to-image diffusion models with deep language un- derstanding

    [Saharia et al., 2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language un- derstanding. Advan...

    2022

  15. [33]

    Improved techniques for training gans.Advances in neural information processing systems, 29,

    [Salimans et al., 2016] Tim Salimans, Ian Goodfellow, Woj- ciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans.Advances in neural information processing systems, 29,

    2016

  16. [35]

    Denoising diffusion implicit models

    [Song et al., 2020] Jiaming Song, Chenlin Meng, and Ste- fano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502,

    2020 arXiv

  17. [36]

    Score-based generative modeling through stochastic differential equations

    [Song et al., 2021] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations,

    2021

  18. [37]

    Plug-and-play diffusion fea- tures for text-driven image-to-image translation

    [Tumanyan et al., 2023] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion fea- tures for text-driven image-to-image translation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1921–1930,

    2023

  19. [38]

    Unitune: Text-driven im- age editing by fine tuning an image generation model on a single image

    [Valevskiet al., 2022] Dani Valevski, Matan Kalman, Yossi Matias, and Yaniv Leviathan. Unitune: Text-driven im- age editing by fine tuning an image generation model on a single image. arXiv preprint arXiv:2210.09477, 2(3):5,

    2022 arXiv

  20. [39]

    Stylediffusion: Controllable disentangled style transfer via diffusion models

    [Wang et al., 2023] Zhizhong Wang, Lei Zhao, and Wei Xing. Stylediffusion: Controllable disentangled style transfer via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7677–7689,

    2023

  21. [40]

    A latent space of stochastic diffusion models for zero-shot image editing and guidance

    [Wu and De la Torre, 2023] Chen Henry Wu and Fernando De la Torre. A latent space of stochastic diffusion models for zero-shot image editing and guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7378–7387,

    2023

  22. [41]

    Uncovering the disentanglement capa- bility in text-to-image diffusion models

    [Wu et al., 2023] Qiucheng Wu, Yujian Liu, Handong Zhao, Ajinkya Kale, Trung Bui, Tong Yu, Zhe Lin, Yang Zhang, and Shiyu Chang. Uncovering the disentanglement capa- bility in text-to-image diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and patt...

    2023

  23. [42]

    [Yang et al., 2022] Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie, Zhouchen Lin, and Dacheng Tao. Induc- ing neural collapse in imbalanced learning: Do we really need a learnable classifier at the end of deep neural net- work? Advances in neural information processing sys- ...

    2022

  24. [43]

    Object- aware inversion and reassembly for image editing

    [Yang et al., 2023] Zhen Yang, Ganggui Ding, Wen Wang, Hao Chen, Bohan Zhuang, and Chunhua Shen. Object- aware inversion and reassembly for image editing. arXiv preprint arXiv:2310.12149,

    2023 arXiv

  25. [44]

    The unreasonable effectiveness of deep features as a perceptual metric

    [Zhang et al., 2018] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595,

    2018

  26. [45]

    A Proof of Proposition 2 First, we introduce some additional theoretical results from [Han et al., 2021; Papyan et al., 2020] to lay the foun- dation for the proof of Proposition

    2021

  27. [47]

    (25) The proof is complete. B Direct text guidance is disentangled ”an old man” ”an elderly man” ”a man of old age” ”an aged man” ”an old man with a hat” Figure 9: Sample results generated with the same random seed but different text prompts. C Problem with blurred images afte...

    2017

  28. [48]

    Bushy Eyebrows

    InputEdit Figure 13: T-LOCO edit for cat→dog. Because of its poor effect, we do not conduct statistics of the relevant results. 30 35 20 25 4540 Input Without Reconstruction loss Figure 14: With T = 50, comparison of the effect of starting editing at different timesteps. G Gui...

    2024

  29. [2013]

    Deepfakes: a new threat to face recognition? assessment and detection

    [Korshunov and Marcel, 2018] Pavel Korshunov and S´ebastien Marcel. Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685,

    2018 arXiv

  30. [2014]

    Prompt tuning inversion for text-driven image editing using diffusion models

    [Dong et al., 2023] Wenkai Dong, Song Xue, Xiaoyue Duan, and Shumin Han. Prompt tuning inversion for text-driven image editing using diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 7430–7440,

    2023

  31. [2016]

    Very deep convolutional networks for large-scale image recognition

    [Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

    2014 arXiv

  32. [2017]

    Classifier-free diffusion guidance

    [Ho and Salimans, 2022] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598,

    2022 arXiv

  33. [2018]

    3d object representations for fine- grained categorization

    [Krause et al., 2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the IEEE inter- national conference on computer vision workshops, pages 554–561,

    2013

  34. [2019]

    Progressive growing of gans for improved quality, stability, and variation

    [Karras, 2017] Tero Karras. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196,

    2017 arXiv

  35. [2020]

    Diffusion posterior sampling for general noisy inverse problems

    [Chung et al., 2022] Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687,

    2022 arXiv

  36. [2021]

    Nice: Non-linear independent compo- nents estimation

    [Dinh et al., 2014] Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent compo- nents estimation. arXiv preprint arXiv:1410.8516,

    2014 arXiv

  37. [2022]

    Noiseclr: A contrastive learning approach for unsupervised discovery of interpretable directions in dif- fusion models

    [Dalva and Yanardag, 2024] Yusuf Dalva and Pinar Ya- nardag. Noiseclr: A contrastive learning approach for unsupervised discovery of interpretable directions in dif- fusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 2...

    2024

  38. [2023]

    Exploring low- dimensional subspaces in diffusion models for controllable image editing

    [Chen et al., 2024] Siyi Chen, Huijie Zhang, Minzhe Guo, Yifu Lu, Peng Wang, and Qing Qu. Exploring low- dimensional subspaces in diffusion models for controllable image editing. arXiv preprint arXiv:2409.02374,

    2024

  39. [2024]

    Stargan v2: Diverse image synthesis for multiple domains

    [Choi et al., 2020] Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition,

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.