Pith. sign in

REVIEW 4 major objections 4 minor 39 references

LLM-guided Instance-level Image Manipulation with Diffusion U-Net Cross-Attention Maps

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a training-free pipeline that repositions a single detected instance in a generated image using LLM parsing, open-vocabulary detection, and cross-attention guidance.

desk verdict Training-free instance repositioning built from known parts, but the headline appearance-preservation claim is contradicted by the paper's own text. read the letter →

arxiv 2501.14046 v1 pith:EAFCO5DO submitted 2025-01-23 cs.CV

Andrey Palaev , Adil Khan , Syed M. Ahsan Kazmi This is my paper · ORCID
classification cs.CV
keywords instance-levelimageeditingdiffusionmodelscross-attentionmapsLLM-guidedopen-vocabularydetectionguidanceStableXL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that instance-level image manipulation—moving one object in a generated image while leaving other objects untouched—can be done without fine-tuning the diffusion model and without user-supplied masks or bounding boxes. It combines a large language model that reads the prompt and lists the objects, an open-vocabulary detector that finds those objects in the already generated image, and a guidance term that acts on cross-attention maps to shift the selected object while a second term preserves other objects' appearances via intermediate U-Net activations. If the method works as described, a user could select a particular monkey among several in a scene and have it move sideways with the rest of the picture intact. The paper reports qualitative examples and acknowledges failure cases for large objects and hyperparameter sensitivity.

What carries the argument

The pipeline's load-bearing components are: (1) an LLM that extracts objects and attributes from the prompt, (2) an open-vocabulary detector that localizes those objects on the generated image with bounding boxes, and (3) guidance terms applied during sampling—a position term $g_{\mathrm{position}}(o)$ that suppresses cross-attention mass at the original box $M_{\mathrm{orig}}$ and encourages it at the shifted box $M_{\mathrm{target}}$, and a preservation term $g_{\mathrm{preserve}}(o)$ that penalizes differences between original and edited intermediate activations $\Psi_{\mathrm{orig}}$ and $\Psi_{\mathrm{target}}$ over the other objects. The total guidance is combined with classifier-free guidance as in Eq. 4.

What would settle it

Take a prompt with several instances of an object (e.g., 'three monkeys') and a prompt with a multi-word object ('green motorcycle'); request a horizontal shift of the rightmost monkey and of the motorcycle, then measure whether the detected bounding box actually moves by the requested fraction and whether the moved instance's appearance is preserved. The paper already reports that the motorcycle did not move, so a systematic evaluation across multi-token objects of varying sizes would decide whether the instance-level control claim holds.

Watch

Extended reading notes

Core claim

The central discovery is that the attention maps inside a pretrained diffusion U-Net, when targeted at a token whose object has been localized by an LLM-plus-detector stage, can be used as an energy function to drag that object to a new position while a separate preservation term on intermediate activations keeps all other detected instances visually consistent. This gives instance-level control that neither object-type editing via cross-attention maps (which moves every occurrence of a word) nor mask-requiring methods provide, and it does so with no training and no extra user input beyond a desired shift.

Load-bearing premise

The method assumes that each detected object can be mapped to one specific text token whose cross-attention map cleanly covers that object; for phrases like 'green motorcycle' this mapping is not explained, and if it fails, the guidance will move attention in the wrong place or leave the object unmoved.

Editorial extensions

If this is right

  • Users can select one instance among several of the same object type (e.g., the rightmost of three monkeys) and move it without moving the others.
  • No fine-tuning or masks means the pipeline runs entirely on pretrained components and can be applied to any prompt the detector can parse.
  • Cross-attention-only preservation is insufficient; intermediate activations are necessary to keep appearance, so the method's design point is the combination of the two terms.
  • The method can be extended to other manipulations beyond position because the same guidance machinery applies to any attribute encoded by attention or features, though the paper implements only position.
  • Large-object displacements may fail as the paper reports, so instance-level control has practical size limits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If token-to-instance alignment is made robust for multi-token phrases (e.g., using per-token aggregation or resolved noun phrases), the same pipeline could generalize to attribute-based edits like color or size without masks.
  • The position guidance only uses the cross-attention map of one token; combining it with cross-attention refinement could fix the failure on large objects that the paper reports.
  • Because the preservation term runs on all detected non-manipulated objects, the method's quality degrades gracefully with detection mistakes: if the detector misses an object or mislabels it, that object is neither preserved nor steered.
  • A quantitative benchmark measuring how far the moved instance's bounding box actually shifts versus the requested shift would let practitioners compare this approach with mask-based methods on equal footing; the paper itself relies on qualitative comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a training-free pipeline for instance-level object repositioning in diffusion-generated images. It combines an LLM and an open-vocabulary detector (Gemma-7b, OWLv2) to locate objects mentioned in the prompt, then applies guidance terms based on SDXL cross-attention maps and U-Net intermediate activations to move a selected instance. The total guidance (Eq. 7) combines a preservation term for non-manipulated objects (Eq. 6) and a position term for the manipulated object (Eq. 5). The paper reports qualitative comparisons with Self-Guidance and DragonDiffusion on four examples involving monkeys and a motorcycle.

Significance. If the claimed capabilities were fully demonstrated, the pipeline would provide a useful zero-shot alternative to mask-based and fine-tuning-based editing, and the integration of LLM-parsed prompts with detector bounding boxes is a sensible direction for instance-level control. The paper also ships a public code link and states that only pretrained models are used, which are strengths. However, the central claim of precise instance-level manipulation with preserved appearances is currently undermined by the method's own admitted limitations, the absence of quantitative evaluation, and an unspecified token-mapping component.

major comments (4)
  1. [§4.1 (with Abstract and §1)] The central claim is contradicted by the paper's own results. The abstract and introduction state that the pipeline enables 'precise manipulations at the instance level' and 'ensuring the preservation of original appearances,' but §4.1 explicitly admits that 'the appearance of the moved object changes completely because the position term in Eq. 5 uses cross-attention maps that contain only general location and shape information, not appearances.' Moreover, Eq. 6 defines the preservation loss only for 'the rest of the objects that are not manipulated,' so the moved object is never feature-preserved. This means the method regenerates a category-level object at the target location rather than editing the specific detected instance, which invalidates the comparison to DragonDiffusion and the headline claim of instance-level manipulation as stated.
  2. [§3.1 and §3.2.1, Eq. (5)] The position guidance requires 'the cross-attention map corresponding to token k' for the manipulated object, but the paper does not specify how an LLM-extracted object (e.g., 'green motorcycle' or 'monkey id=0') is mapped to a specific token in SDXL's two text encoders. For multi-token phrases or repeated nouns, the mapping is nontrivial; without a defined and validated mapping, the position term may steer attention in the wrong spatial region or affect the wrong object. This is load-bearing because instance-level precision is the entire contribution.
  3. [§4.1 and Fig. 2] The paper's claim of 'precise manipulation' rests on four qualitative examples, one of which is explicitly admitted to fail: the motorcycle move 'resulted in no displacement.' The text refers to 'additional examples and quantitative evaluation' in supplementary materials, but no such materials accompany this submission. For a precision-oriented editing method, a quantitative evaluation is necessary, at minimum measuring bounding-box displacement accuracy and appearance preservation (e.g., LPIPS or identity similarity) for the moved object and the background.
  4. [§4.1 and §5] The paper acknowledges that 'the weights require tuning, as a combination that works well for one object may not yield satisfactory results for another.' Since the guidance weights w0 and w1 in Eq. 7, the scales s and v in Eq. 4, and the U-Net layer choices are free parameters selected per manipulation, the method is not parameter-free or fully automatic as the abstract's 'without fine-tuning or auxiliary information' might imply. The authors should either provide a fixed parameter-selection protocol or a sensitivity analysis showing that results are robust over a reasonable range of hyperparameters.
minor comments (4)
  1. [§3.2.1, Eq. (5)] In Eq. (5), the function is written as gposition(o), but the formula uses a cross-attention map Ak; the index k should be defined or the summation over attention heads/maps should be made explicit.
  2. [§3.2.3, Eq. (7)] Eq. (7) refers to gmanipulation(ok) but the earlier definitions only introduce gposition(o); this notation should be clarified.
  3. [Fig. 2 and Fig. 3 captions] The captions state 'Coordinates shift is represented by (x, y)' but do not specify the units or normalization; the reader should be told whether the shift is in pixels, latent-space coordinates, or normalized image coordinates.
  4. [§4.1] The sentence 'Additional examples and quantitative evaluation can be found in the supplementary materials' is not verifiable in the current submission, since no supplementary material is provided; this should either be included or removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the guidance terms are defined directly from cross-attention maps and intermediate activations, with no fitted quantity or self-citation chain serving as the central premise.

full rationale

The paper's pipeline is not circular in the senses enumerated: the position term (Eq. 5) and preservation term (Eq. 6) are constructed directly from cross-attention maps and intermediate U-Net activations of pretrained models; neither term is fitted to the qualitative outcomes it is later used to explain, nor is any 'prediction' obtained by renaming an input. The LLM/detector stage is borrowed from prior work by Wu et al. [37], but that is an external citation, not a self-citation, and it supplies object localization rather than the editing result. No uniqueness theorem or prior-work assumption by the present authors is invoked to forbid alternatives. The paper's acknowledged weaknesses—manual hyperparameter tuning per manipulation, ambiguity in mapping detected instances to text tokens, and complete appearance change of the moved object—are correctness and reproducibility limitations, not circularity: they concern whether the claimed instance-level appearance preservation actually holds, not whether the method's equations reduce to their own inputs. Since every load-bearing component is explicitly defined in terms of observable attention/activation quantities of an independently pretrained diffusion model, the derivation chain is self-contained and no circular step can be exhibited.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method depends on several domain assumptions inherited from prior work: that cross-attention maps encode object position (Self-Guidance), intermediate activations encode appearance (DragonDiffusion), and that LLM parsing plus OWLv2 detection yields reliable instance masks. The pipeline introduces no new free parameters beyond the guidance weights and layer choices from Eq. 4/5/7, all of which require per-manipulation tuning according to Section 4.1.

free parameters (3)
  • Guidance weights w0 and w1 = not reported, tuned per manipulation
    Eq. 7 combines preservation and manipulation terms; Section 4.1 notes careful hyperparameter selection needed for each manipulation.
  • Guidance scale s and v from Eq. 4 = not reported
    Total guidance is applied via Eq. 4 which includes s and v, not specified in the paper.
  • Diffusion U-Net layer choices = first upper block for attention, third upper block for features
    Section 4 states these blocks were used, with no justification or search.
assumptions (5)
  • domain assumption Cross-attention maps in the U-Net encode object position and shape (from Self-Guidance [10])
    Section 3.2 relies on this to justify Eq. 5.
  • domain assumption Intermediate activations encode object appearance and can be used for preservation (from DragonDiffusion [25])
    Section 3.2.2 uses this for Eq. 6.
  • domain assumption OWLv2 open-vocabulary detector reliably detects objects with bounding boxes
    Section 3.1 depends on it to get instance masks; no failure analysis.
  • domain assumption LLM (Gemma-7b) correctly parses objects and attributes from prompt
    Section 3.1 relies on it, following SLD [37].
  • standard math Guidance equation Eq. 4 from prior work (classifier-free guidance with energy function)
    Section 2.2 introduces Eq. 4 as background; the method applies it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-guided Instance-level Image Manipulation with Diffusion U-Net Cross-Attention Maps." pith.science (2026). https://pith.science/paper/EAFCO5DO

@misc{pith2026250114046,
  author       = {Pith},
  title        = {Pith review of: LLM-guided Instance-level Image Manipulation with Diffusion U-Net Cross-Attention Maps},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EAFCO5DO}},
  note         = {Machine review of arXiv:2501.14046}
}
read the original abstract

The advancement of text-to-image synthesis has introduced powerful generative models capable of creating realistic images from textual prompts. However, precise control over image attributes remains challenging, especially at the instance level. While existing methods offer some control through fine-tuning or auxiliary information, they often face limitations in flexibility and accuracy. To address these challenges, we propose a pipeline leveraging Large Language Models (LLMs), open-vocabulary detectors, cross-attention maps and intermediate activations of diffusion U-Net for instance-level image manipulation. Our method detects objects mentioned in the prompt and present in the generated image, enabling precise manipulation without extensive training or input masks. By incorporating cross-attention maps, our approach ensures coherence in manipulated images while controlling object positions. Our method enables precise manipulations at the instance level without fine-tuning or auxiliary information such as masks or bounding boxes. Code is available at https://github.com/Palandr123/DiffusionU-NetLLM

Figures

Figures reproduced from arXiv: 2501.14046 by the authors.

Figure 1
Figure 1. Overview of our pipeline. Firstly, LLM parses the objects from the prompt. Then, [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Examples of the position manipulations. Coordinates shift is represented by (x, y) [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Comparison of different preservation terms in our method. Coordinates shift is [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 27 canonical work pages

  1. [1]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18208–18218, June 2022

    work page 2022

  2. [2]

    Break-a-scene: Extracting multiple concepts from a single image

    Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski. Break-a-scene: Extracting multiple concepts from a single image. In SIGGRAPH Asia 2023 Conference Papers, 2023

    work page 2023

  3. [3]

    Blended latent diffusion

    Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM Transactions on Graphics, July 2023. PALAEV , KHAN, KAZMI: LLM-GUIDED INSTANCE-LEVEL IMAGE MANIPULA TION 11

    work page 2023

  4. [4]

    Universal guidance for diffusion models

    Arpit Bansal et al. Universal guidance for diffusion models. In The Twelfth Interna- tional Conference on Learning Representations , 2024

    work page 2024

  5. [5]

    Improving image generation with better captions, 2023

    James Betker et al. Improving image generation with better captions, 2023. URL https://cdn.openai.com/papers/dall-e-3.pdf

    work page 2023

  6. [6]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18392–18402, June 2023

    work page 2023

  7. [7]

    Language models are few-shot learners

    Tom Brown et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901, 2020

    work page 1901

  8. [8]

    DiffEdit: Diffusion-based semantic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. DiffEdit: Diffusion-based semantic image editing with mask guidance. In The Eleventh Interna- tional Conference on Learning Representations , 2023

    work page 2023

Show all 39 references
  1. [9]

    Diffusion models beat GANs on image syn- thesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image syn- thesis. In Advances in Neural Information Processing Systems, volume 34, pages 8780– 8794, 2021

    2021

  2. [10]

    Diffu- sion Self-Guidance for controllable image generation

    Dave Epstein, Allan Jabri, Ben Poole, Alexei Efros, and Aleksander Holynski. Diffu- sion Self-Guidance for controllable image generation. In Advances in Neural Informa- tion Processing Systems, volume 36, pages 16222–16239, 2023

    2023

  3. [11]

    Scaling rectified flow transformers for high-resolution image syn- thesis

    Patrick Esser et al. Scaling rectified flow transformers for high-resolution image syn- thesis. arXiv preprint arXiv:2403.03206, 2024

    2024 arXiv

  4. [12]

    An image is worth one word: Personalizing text-to-image genera- tion using textual inversion

    Rinon Gal et al. An image is worth one word: Personalizing text-to-image genera- tion using textual inversion. In The Eleventh International Conference on Learning Representations, 2023

    2023

  5. [13]

    Diffusion models as plug-and-play priors

    Alexandros Graikos, Nikolay Malkin, Nebojsa Jojic, and Dimitris Samaras. Diffusion models as plug-and-play priors. In Advances in Neural Information Processing Sys- tems, volume 35, pages 14715–14728, 2022

    2022

  6. [14]

    Prompt-to-prompt image editing with cross-attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-prompt image editing with cross-attention control. In The Eleventh International Conference on Learning Representations , 2023

    2023

  7. [15]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications , 2021

    2021

  8. [16]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems , volume 33, pages 6840–6851, 2020

    2020

  9. [17]

    Jiang et al

    Albert Q. Jiang et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  10. [18]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar et al. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6007–6017, June 2023. 12 PALAEV , KHAN, KAZMI: LLM-GUIDED INSTANCE-LEVEL IMAGE MANIPULA TION

    2023

  11. [19]

    Variational diffusion models

    Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In Advances in Neural Information Processing Systems , volume 34, pages 21696–21707, 2021

    2021

  12. [20]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1931–1941, June 2023

    1931

  13. [21]

    Llm-grounded diffusion: En- hancing prompt understanding of text-to-image diffusion models with large language models

    Long Lian, Boyi Li, Adam Yala, and Trevor Darrell. Llm-grounded diffusion: En- hancing prompt understanding of text-to-image diffusion models with large language models. arXiv preprint arXiv:2305.13655, 2023

    2023 arXiv

  14. [22]

    Compo- sitional visual generation with composable diffusion models

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compo- sitional visual generation with composable diffusion models. In European Conference on Computer Vision, pages 423–439, 2022

    2022

  15. [23]

    Gemma: Open models based on gemini research and technol- ogy

    Thomas Mesnard et al. Gemma: Open models based on gemini research and technol- ogy. arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  16. [24]

    Scaling open-vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection. In Advances in Neural Information Processing Systems , volume 36, pages 72983–73007, 2023

    2023

  17. [25]

    DragonDiffu- sion: Enabling drag-style manipulation on diffusion models

    Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. DragonDiffu- sion: Enabling drag-style manipulation on diffusion models. In The Twelfth Interna- tional Conference on Learning Representations , 2024

    2024

  18. [26]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell et al. SDXL: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024

    2024

  19. [27]

    Learning transferable visual models from natural language super- vision

    Alec Radford et al. Learning transferable visual models from natural language super- vision. In Proceedings of the 38th International Conference on Machine Learning , volume 139, pages 8748–8763, 18–24 July 2021

    2021

  20. [28]

    Zero-shot text-to-image generation

    Aditya Ramesh et al. Zero-shot text-to-image generation. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 8821–8831, 18–24 July 2021

    2021

  21. [29]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Om- mer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10684–10695, June 2022

    2022

  22. [30]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page...

    2023

  23. [31]

    Photorealistic text-to-image diffusion models with deep lan- guage understanding

    Chitwan Saharia et al. Photorealistic text-to-image diffusion models with deep lan- guage understanding. In Advances in Neural Information Processing Systems , vol- ume 35, pages 36479–36494, 2022. PALAEV , KHAN, KAZMI: LLM-GUIDED INSTANCE-LEVEL IMAGE MANIPULA TION 13

    2022

  24. [32]

    Denoising diffusion implicit mod- els

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit mod- els. In International Conference on Learning Representations , 2021

    2021

  25. [33]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Er- mon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations , 2021

    2021

  26. [34]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  27. [35]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2023

    2023

  28. [36]

    Dy- namic prompt learning: Addressing cross-attention leakage for text-based image edit- ing

    Kai Wang, Fei Yang, Shiqi Yang, Muhammad Atif Butt, and Joost van de Weijer. Dy- namic prompt learning: Addressing cross-attention leakage for text-based image edit- ing. In NeurIPS, volume 36, pages 26291–26303, 2023

    2023

  29. [37]

    Self- correcting llm-controlled diffusion models

    Tsung-Han Wu, Long Lian, Joseph E Gonzalez, Boyi Li, and Trevor Darrell. Self- correcting llm-controlled diffusion models. arXiv preprint arXiv:2311.16090, 2023

    2023 arXiv

  30. [38]

    Adding conditional control to text- to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text- to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3836–3847, October 2023

    2023

  31. [39]

    Sur- adapter: Enhancing text-to-image pre-trained diffusion models with large language models

    Shanshan Zhong, Zhongzhan Huang, Weushao Wen, Jinghui Qin, and Liang Lin. Sur- adapter: Enhancing text-to-image pre-trained diffusion models with large language models. In Proceedings of the 31st ACM International Conference on Multimedia , page 567–578, 2023

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.