Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

ComposeAnything: Composite Object Priors for Text-to-Image Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ComposeAnything claims that swapping a diffusion model's random noise for an LLM-planned composite object prior, with depth-aware layering and masked spatial attention, lets text-to-image models follow complex and surreal arrangements…

desk verdict The 2D and count gains are probably real; the big 3D number rests on a self-defined GPT-4.1 rubric that needs a human study before it can be believed. read the letter →

arxiv 2505.24086 v1 pith:SEKCQF4X submitted 2025-05-30 cs.CV

Zeeshan Khan , Shizhe Chen , Cordelia Schmid This is my paper
classification cs.CV
keywords text-to-imagegenerationcompositionalLLMplanning2.5Dlayoutcompositeobjectpriordiffusionmodelstraining-freemethodspatialcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ComposeAnything claims that a text-to-image diffusion model can follow complex and surreal compositions, including correct 2D/3D positions, object counts, and unusual interactions, without any retraining, by replacing its random starting noise with a coarse picture of the intended scene. The paper's pipeline first asks an LLM to reason through the prompt into a 2.5D layout: per-object captions, bounding boxes, and relative depths. Individual object images are cut out and composited in depth order into a rough object prior, which is then noised and injected into the denoising process. On T2I-CompBench and NSR-1K the method outperforms state-of-the-art baselines, with the largest reported gain on 3D spatial composition (77.16 vs 49.43 over the SD3-M base model). The paper also reports human pairwise evaluations that favor the method over two strong baselines.

What carries the argument

The load-bearing object is the 2.5D semantic layout together with the composite object prior built from it. From an input prompt, the LLM produces object captions, bounding boxes $box_i$, depth values $depth_i$, a background caption, and a compositional caption; individual object images are then generated, segmented, resized into the boxes, and layered by depth to form a composite image $o_p$ with mask $m_p$. This prior enters diffusion through two complementary mechanisms: object prior reinforcement, which overwrites the foreground latent with the noisy prior until time $t_p$, and spatial-controlled denoising, which partitions the latent by masks and runs joint self-attention between each image region and its own caption in early steps. Afterwards, standard diffusion refines the result, giving the framework its claimed balance between composition fidelity and image quality.

What would settle it

Take the 300 T2I-CompBench 3D-spatial prompts and have several independent human raters score ComposeAnything and SD3-M images using the paper's own 0-2 rubric, then compare human scores with the GPT-4.1 scores; if the human-measured gap is far below 27.7 points, or raters do not consistently prefer the method, the central 3D claim is a metric artifact.

Watch

Extended reading notes

Core claim

The central claim is that a composite object prior, a coarse RGB composite of the objects placed in their planned 2.5D positions, is a stronger starting point for diffusion-based text-to-image generation than pure Gaussian noise or attention-only layout guidance. Starting from a chain-of-thought LLM plan, the method generates each object separately with the base text-to-image model, extracts its mask with a referring-segmentation model, resizes and layers the objects depth-aware onto a canvas, and reinitializes the background region with pure noise. During denoising, the foreground prior is repeatedly re-injected until an intermediate timestep $t_p$, while a spatially controlled attention path aligns each object region with its own text embedding during the early steps. The paper argues that this preserves global structure where it matters and then releases control to standard diffusion, yielding both faithful composition and high visual fidelity.

Load-bearing premise

The headline 3D-spatial gain rests on a new GPT-4.1-based scoring rule that was not human-validated, and the same model family also writes the layouts being scored; if that rule is lenient toward the method, the 27.7-point improvement could be partly an artifact of grading rather than generation.

Editorial extensions

If this is right

  • If the central claim holds, compositional control becomes an inference-time service for existing closed-weight diffusion models: no adapter training or fine-tuning is required.
  • The paper reports the same framework also improves SDXL, so the mechanism is tied to generic denoising and flow-matching diffusion rather than to one architecture.
  • Because the object prior itself already matches or beats many baselines on 2D-spatial and count categories, the pipeline offers an inexpensive, inspectable layout preview before the final render.
  • Two hyper-parameters, $t_p$ and $N_{sc}$, let users dial between strict prompt faithfulness and generative freedom, which matters for surreal prompts where the two goals conflict.
  • The ablation results show that prior reinforcement and spatial-controlled denoising add complementary gains, so both mechanisms are needed for the full reported effect.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same LLM family that writes the layouts also grades the 3D results with a newly introduced rubric; until independent human or cross-model evaluation is published, the 27.7-point 3D margin should be treated as an upper bound.
  • Editorial inference: a natural testable extension is to build the composite prior with a different text-to-image model than the base denoiser, which would isolate how much of the gain comes from planning and compositing versus the diffusion backbone.
  • Editorial inference: because the prior is a visible RGB image, the pipeline doubles as an interpretable debugger, letting failures be traced to either the LLM layout or the diffusion refinement step.
  • Editorial inference: combining this prior initialization with inference-time candidate search, rather than a single prior render, is a plausible next step that could address the remaining failures where a flawed initial composite is unrecoverable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ComposeAnything, a training-free framework for improving compositional text-to-image generation. It uses GPT-4.1 chain-of-thought planning to produce a 2.5D semantic layout (object captions, bounding boxes, relative depths), then composites individually generated and segmented object images into a coarse prior that replaces the stochastic noise initialization of a diffusion model such as SD3-M. The denoising process is guided by object prior reinforcement and spatial-controlled denoising for the first few steps, followed by standard denoising to refine quality. The method is evaluated on T2I-CompBench and NSR-1K, with reported gains over SD3-M and other baselines on 2D-Spatial, 3D-Spatial, Complexity, and Spatial categories, plus human evaluations against RPG and CreatiLayout.

Significance. If the reported results hold, the paper makes a useful contribution: it shows that structured LLM planning plus a coarse composite image prior can improve compositional fidelity of an off-the-shelf diffusion model without retraining, a practically appealing recipe. The 2D-Spatial and Count results are measured with established detector-based metrics and show clear gains, and the paper includes ablations, hyperparameter analysis, and qualitative failure-mode discussion. The main uncertainty is the headline 3D-Spatial gain, which depends on a bespoke GPT-4.1 evaluation rubric that has not been validated against human judgments and whose definition of 'hidden by' is relaxed relative to the standard interpretation. The paper also slightly overstates its benchmark claims in the abstract, since NSR-1K Count is worse than CreatiLayout.

major comments (3)
  1. [Sec. 4.1 and Appendix G] The 27.7-point 3D-Spatial gain over SD3-M rests on a new GPT-4.1-based metric whose rubric is not validated and is definitionally aligned with the method's design. Appendix G instructs the evaluator to 'Ignore visual occlusion' and to treat 'hidden by' as satisfied whenever the occluded object is 'clearly located behind it in 3D space, even if visible'; under the ordinary meaning of 'hidden by', an object that is visible is not hidden. Meanwhile, Appendix F, rule 6c requires that boxes 'must not fully occlude or obscure one another,' so ComposeAnything explicitly avoids occlusion in its constructed priors. The same GPT-4.1 model family performs the LLM planning and the grading, and no human validation of the new metric is provided. Table 2 compounds the concern: the composite object prior alone scores 86.71 on 3D-Spatial, higher than the final image's 77.16, even though the prior is a direct depth-ordered compositing of object crops. To support the headline claim, the authors should report the original T2I-CompBench depth-and-box metric for all methods, and/or carry out a human study on the 3D-Spatial set that validates the GPT-4.1 rubric and compares ComposeAnything against the baselines.
  2. [Abstract and Table 1] The abstract claims ComposeAnything 'outperforms state-of-the-art methods' on benchmarks for prompts with 'high object counts,' but Table 1 shows NSR-1K Count is 59.36 for ComposeAnything versus 63.40 for CreatiLayout, a result the paper itself acknowledges in Section 4.2. The abstract's blanket claim should be qualified to exclude the Count category, or the discrepancy should be explained, since it contradicts the stated central claim.
  3. [Sec. 4.2, Figure 6] The human evaluation only compares ComposeAnything against RPG and CreatiLayout on 30 prompts per category, and it is used to support overall quality claims but not to validate the automatic 3D metric. The paper reports average inter-annotator agreement around 80% but no per-category agreement, confidence intervals, or significance tests for the pairwise preferences. If the human study is intended to establish the 3D-Spatial advantage, it should cover that category explicitly and report the statistics; if it is not, the 3D metric remains the only evidence for the largest reported gain.
minor comments (5)
  1. [Sec. 4.2] The text reports 'absolute gains of 16.9% on 2D-Spatial, 7.9% on Count, 27.7% on 3D-Spatial, and 0.9% on Complex'; these are percentage-point differences (e.g., 48.24 vs 31.32 is 16.92 points), so the units should be stated as 'points' rather than '%'.
  2. [Eq. (15)] The hyperparameter ratiobase in the latent merge equation is never defined or given a value in Section 4.1 or the appendix; please state its setting and, ideally, its sensitivity.
  3. [Sec. 4.2] The list of compared methods mentions GLIGEN, but Table 1 does not report GLIGEN results; either add the missing entries or remove GLIGEN from the list of compared methods.
  4. [Appendix C] There is an incomplete sentence: 'For sample annotators perform a 4-way classification task...' — this should be rewritten to describe how annotators were sampled and instructed.
  5. [Figure 6] The bar chart would be more informative with error bars or per-category counts, and the number of raters per comparison should be stated clearly in the caption.

Circularity Check

1 steps flagged · score 5.0 of 10

The 27.7-point 3D-Spatial gain is measured by a bespoke GPT-4.1 rubric that redefines 'hidden by' to ignore occlusion, while the LLM planner is explicitly told to keep hidden objects visible; the 2D, Count, Complex, and NSR-1K results remain externally benchmarked.

  1. self definitional [Sec. 4.1 (Evaluation metrics); Appendix F rule 6c; Appendix G]
    "For the 3D-spatial category, the original T2I-CompBench metric relies on depth estimation and bounding box detection, which we found inaccurate and overly punitive. To address this, we introduce an MLLM-based metric using GPT-4.1 [42]... 'Ignore visual occlusion —an object can still be considered "hidden by" another object if it is clearly located behind it in 3D space, even if visible.'"

    The headline 27.7-point 3D-Spatial gain over SD3-M rests on this bespoke GPT-4.1 metric, not on the standard T2I-CompBench depth-and-box protocol. The rubric explicitly instructs the evaluator to ignore occlusion and to count a visible object as 'hidden by' the occluder whenever it is merely behind it. Meanwhile, the LLM planning prompt (Appendix F, rule 6c) requires that 'bounding boxes must be placed such that every object remains visibly distinct... Even when objects differ in depth, they must not fully occlude or obscure one another.' ComposeAnything is therefore constrained by construction to avoid true occlusion, and the new evaluator is defined to reward exactly the visible-behind configuration the planner is forced to produce.

full rationale

There is no equation-level circularity in the core pipeline: the composite prior construction (Eqs. 1-7), object prior reinforcement (Eq. 8), and spatial-controlled attention (Eqs. 9-15) are concrete mechanisms whose outputs are then compared against benchmarks; they are not derived from the benchmark numbers. Moreover, the 2D-Spatial, Count, Complex, and NSR-1K results are measured with external protocols (UniDet, GLIP, CLIP, BLIP-VQA) and therefore provide independent grounding for most of the paper's claims. The circularity is concentrated in the 3D-Spatial evaluation. The authors replaced the standard T2I-CompBench 3D metric with a GPT-4.1 prompt of their own design, and that prompt defines 'hidden by' as mere depth ordering while explicitly ignoring occlusion. The LLM planning prompt used by the method independently instructs that objects 'must not fully occlude or obscure one another,' so the generator is optimized to keep hidden objects visible. The evaluator then accepts visible objects as 'hidden by' the occluder. This is a definitional alignment between the method's optimization target and the scoring rubric, not an independent measurement. The same GPT-4.1 model family is used for both planning and evaluation, and the new metric has no human validation; Table 2 shows the raw prior scoring 86.71 on 3D-Spatial versus 77.16 for the final image, indicating that the metric rewards the direct depth-ordered composite rather than the diffusion model's 3D reasoning. Because the single largest reported advantage is affected but the majority of the benchmark results are externally measured, the overall circularity is partial: score 5.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The pipeline depends on five explicit assumptions: the validity of intermediate-step diffusion initialization, the segmentability of MM-DiT attention, the reliability of LLM planning, the accuracy of segmentation masks, and the validity of the new 3D metric. The only hand-tuned numbers are tp, Nsc, and the unreported ratiobase. No new physical entities are introduced.

free parameters (3)
  • tp (prior noise timestep) = 0.91 (high noise on Flow Matching schedule)
    Controls how strongly the composite object prior is preserved; lower values give stronger priors but less generative flexibility. Set to 0.91 for all main results, chosen from the Appendix B trade-off analysis.
  • Nsc (number of spatial-controlled denoising steps) = 3
    Controls how many initial denoising steps use mask-guided attention over object and background latents. Set to 3 for main results; ablation shows performance varies with this value.
  • ratiobase (latent merge weight in Eq. 15) = not reported
    Weighted sum of base latent and object-background latent; the value is not specified in the paper, yet it determines the balance between global coherence and spatial control.
assumptions (5)
  • standard math The reverse ODE of a diffusion or flow-matching model can be initialized from any intermediate timestep tp using a noisy image prior.
    Eqs. (4)-(6) assume the denoiser remains valid when starting at tp instead of T and when the foreground prior is re-injected. This is an established property of diffusion ODEs, not derived in the paper.
  • domain assumption MM-DiT joint self-attention in SD3 can be faithfully partitioned into object and background latents, processed separately with paired text embeddings, and recomposed without drift.
    Section 3.3, Eqs. (9)-(15): the method assumes attention over masked latent segments is equivalent to global attention for structural control. The paper does not prove this, and quality depends on it.
  • domain assumption GPT-4.1 chain-of-thought planning yields correct object decompositions, depths, and isolated captions for arbitrary prompts.
    Section 3.1 and the Limitations section state that the main failure mode is LLM layout errors; a wrong prior cannot be recovered, so this assumption is load-bearing.
  • domain assumption Hyperseg produces clean, accurate masks of generated objects that faithfully separate foreground from background.
    Section 3.2, Eqs. (1)-(2): masks are used for compositing and later for re-injecting the prior; mask errors propagate directly into the prior and final image.
  • ad hoc to paper The new GPT-4.1 3D evaluation metric is a valid and comparable measure of 3D spatial correctness.
    Section 4.1 and Appendix G: the authors replace the T2I-CompBench 3D metric because they found it 'inaccurate and overly punitive'; no validation against human labels is provided, and the same model family that generates layouts is used to grade them. The 27.7-point 3D gain relies on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ComposeAnything: Composite Object Priors for Text-to-Image Generation." pith.science (2026). https://pith.science/paper/SEKCQF4X

@misc{pith2026250524086,
  author       = {Pith},
  title        = {Pith review of: ComposeAnything: Composite Object Priors for Text-to-Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEKCQF4X}},
  note         = {Machine review of arXiv:2505.24086}
}
read the original abstract

Generating images from text involving complex and novel object arrangements remains a significant challenge for current text-to-image (T2I) models. Although prior layout-based methods improve object arrangements using spatial constraints with 2D layouts, they often struggle to capture 3D positioning and sacrifice quality and coherence. In this work, we introduce ComposeAnything, a novel framework for improving compositional image generation without retraining existing T2I models. Our approach first leverages the chain-of-thought reasoning abilities of LLMs to produce 2.5D semantic layouts from text, consisting of 2D object bounding boxes enriched with depth information and detailed captions. Based on this layout, we generate a spatial and depth aware coarse composite of objects that captures the intended composition, serving as a strong and interpretable prior that replaces stochastic noise initialization in diffusion-based T2I models. This prior guides the denoising process through object prior reinforcement and spatial-controlled denoising, enabling seamless generation of compositional objects and coherent backgrounds, while allowing refinement of inaccurate priors. ComposeAnything outperforms state-of-the-art methods on the T2I-CompBench and NSR-1K benchmarks for prompts with 2D/3D spatial arrangements, high object counts, and surreal compositions. Human evaluations further demonstrate that our model generates high-quality images with compositions that faithfully reflect the text.

Figures

Figures reproduced from arXiv: 2505.24086 by the authors.

Figure 1
Figure 1. The proposed ComposeAnything framework enables text-to-image generation for com￾plex compositions involving surreal spatial relationships and high object counts. It enhances both visual quality and faithfulness to the input text compared to diffusion-based models (e.g., SD3 [11], FLUX [3]) and 2D layout conditioned models (e.g., RPG [56] and CreatiLayout [58]). Abstract Generating images from text involving complex … view at source ↗
Figure 2
Figure 2. The ComposeAnything framework, which enhances text-to-image diffusion models e.g. SD3-M [11] with layouts and composite object priors for complex compositional generation. generative capabilities, they still struggle with compositional generation. To address this, prior work in this area can be broadly categorized into two main directions. Training-based methods fine-tune pretrained diffusion models [45, 11, 3, 5, 6… view at source ↗
Figure 3
Figure 3. Chain-of-thought LLM planning for generating 2.5D semantic layouts from text. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: Overview of prior-guided diffusion. The spatial [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: State-of-the-art comparison against SD3-M [ [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Human evaluations against RPG and CreatiLayout. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: LLM generated object prior and their corresponding final image generation. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Effect of Object Prior Reinforcement and Spatial-Controlled Denoising. Increasing either [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Human evaluation results on the correctness of prior and final image pairs on the three [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Labeling interface for evaluating object prior and the final image. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Object prior and the corresponding generation for 2D-Spatial compositions from T2I [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Object prior and the corresponding generation for Complex compositions from T2I [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Object prior and the corresponding generation for 3D-Spatial compositions from T2I [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Object prior and the corresponding generation for Numeracy compositions from T2I [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: Labeling interface for human evaluations. [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]
Figure 16
Figure 16. Figure 16: Instructions for LLM planning (to be continued). [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: Instructions for LLM planning (to be continued). [PITH_FULL_IMAGE:figures/full_fig_p025_17.png]
Figure 18
Figure 18. Figure 18: Instructions for LLM planning (to be continued). [PITH_FULL_IMAGE:figures/full_fig_p026_18.png]
Figure 19
Figure 19. Figure 19: Instructions for LLM planning (to be continued). [PITH_FULL_IMAGE:figures/full_fig_p027_19.png]
Figure 20
Figure 20. Figure 20: Instructions for LLM planning. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]
Figure 21
Figure 21. Figure 21: LLM planning for object prior generation, with final generated image. [PITH_FULL_IMAGE:figures/full_fig_p029_21.png]
Figure 22
Figure 22. Figure 22: LLM instructions for evaluating 3D-spatial relations. [PITH_FULL_IMAGE:figures/full_fig_p030_22.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ETPDesigner: Multi-Agent Orchestration for Interactive Multimodal Electronic Theater Program

    cs.CV 2026-07 conditional novelty 5.0 of 10

    ETPDesigner automatically generates multi-page electronic theater programs from scripts using a multi-agent LLM pipeline with a global style anchor and interactive character chat.

Reference graph

Works this paper leans on

64 extracted references · 53 canonical work pages · cited by 1 Pith paper

  1. [1]

    A-star: Test-time attention segregation and retention for text-to-image synthesis

    Aishwarya Agarwal, Srikrishna Karanam, K J Joseph, Apoorv Saxena, Koustava Goswami, and Bal- aji Vasan Srinivasan. A-star: Test-time attention segregation and retention for text-to-image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2283–2293, October 2023

    work page 2023

  2. [2]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    work page 2022

  3. [3]

    Black Forest Labs. FLUX. https://blackforestlabs.ai, 2024

    work page 2024

  4. [4]

    Attend-and-excite: Attention- based semantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention- based semantic guidance for text-to-image diffusion models. ACM Trans. Graph., 2023

    work page 2023

  5. [5]

    Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023. URL https://arxiv.org/abs/2310.00426

    arXiv 2023

  6. [6]

    Pixart-δ: Fast and controllable image generation with latent consistency models, 2024

    Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart-δ: Fast and controllable image generation with latent consistency models, 2024. URL https: //arxiv.org/abs/2401.05252

    arXiv 2024

  7. [7]

    Geodiffu- sion: Text-prompted geometric control for object detection data generation

    Kai Chen, Enze Xie, Zhe Chen, Yibo Wang, Lanqing HONG, Zhenguo Li, and Dit-Yan Yeung. Geodiffu- sion: Text-prompted geometric control for object detection data generation. In The Twelfth International Conference on Learning Representations, 2024

    work page 2024

  8. [8]

    Training-free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024

    work page 2024

Show all 64 references
  1. [9]

    Zero- shot spatial layout conditioning for text-to-image diffusion models

    Guillaume Couairon, Marlène Careil, Matthieu Cord, Stéphane Lathuilière, and Jakob Verbeek. Zero- shot spatial layout conditioning for text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  2. [10]

    Be yourself: Bounded attention for multi-subject text-to-image generation

    Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or. Be yourself: Bounded attention for multi-subject text-to-image generation. In European Conference on Computer Vision (ECCV), 2024

    2024

  3. [11]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthes...

    2024

  4. [12]

    Training-free structured diffusion guidance for composi- tional text-to-image synthesis

    Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for composi- tional text-to-image synthesis. In The Eleventh International Conference on Lear...

    2023

  5. [13]

    LayoutGPT: Compositional visual planning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Xuehai He, S Basu, Xin Eric Wang, and William Yang Wang. LayoutGPT: Compositional visual planning and generation with large language models. In Thirty-seventh Conference on Neural Information Processing Sys...

    2023

  6. [14]

    Ranni: Taming text-to-image diffusion for accurate instruction following

    Yutong Feng, Biao Gong, Di Chen, Yujun Shen, Yu Liu, and Jingren Zhou. Ranni: Taming text-to-image diffusion for accurate instruction following. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  7. [15]

    LLM blueprint: Enabling text-to-image generation with complex and detailed prompts

    Hanan Gani, Shariq Farooq Bhat, Muzammal Naseer, Salman Khan, and Peter Wonka. LLM blueprint: Enabling text-to-image generation with complex and detailed prompts. In The Twelfth International Conference on Learning Representations, 2024

    2024

  8. [16]

    Check locate rectify: A training-free layout calibration system for text-to-image generation

    Biao Gong, Siteng Huang, Yutong Feng, Shiwei Zhang, Yuyuan Li, and Yu Liu. Check locate rectify: A training-free layout calibration system for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6624–6634...

    2024

  9. [17]

    Initno: Boosting text-to- image diffusion models via initial noise optimization

    Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang. Initno: Boosting text-to- image diffusion models via initial noise optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  10. [18]

    Prompt-to- prompt image editing with cross-attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to- prompt image editing with cross-attention control. In The Eleventh International Conference on Learning Representations, 2023

    2023

  11. [19]

    Ella: Equip diffusion models with llm for enhanced semantic alignment

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment. CoRR, 2024

    2024

  12. [20]

    Scenecraft: An LLM agent for synthesizing 3D scenes as blender code

    Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. Scenecraft: An LLM agent for synthesizing 3D scenes as blender code. In Forty-first International Conference on Machine Learning, 2024

    2024

  13. [21]

    T2I-compBench: A comprehensive benchmark for open-world compositional text-to-image generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2I-compBench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    2023

  14. [22]

    Composite diffusion: whole >= sparts

    Vikram Jamwal and Ramaneswaran S. Composite diffusion: whole >= sparts. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024

    2024

  15. [23]

    Comat: Aligning text-to-image diffusion model with image-to-text concept matching

    Dongzhi Jiang, Guanglu Song, Xiaoshi Wu, Renrui Zhang, Dazhong Shen, Zhuofan Zong, Yu Liu, and Hongsheng Li. Comat: Aligning text-to-image diffusion model with image-to-text concept matching. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  16. [24]

    Dense text-to-image generation with attention modulation

    Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu. Dense text-to-image generation with attention modulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  17. [25]

    Evaluating and improving compositional text-to-visual generation

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Xide Xia, Pengchuan Zhang, Graham Neubig, and Deva Ramanan. Evaluating and improving compositional text-to-visual generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2024

  18. [26]

    Grounded language- image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language- image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and ...

    2022

  19. [27]

    Con- trolnet ++: Improving conditional controls with efficient consistency feedback

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. Con- trolnet ++: Improving conditional controls with efficient consistency feedback. In European Conference on Computer Vision (ECCV), 2024

    2024

  20. [28]

    Mulan: Multimodal-llm agent for progressive multi-object diffusion

    Sen Li, Ruochen Wang, Cho-Jui Hsieh, Minhao Cheng, and Tianyi Zhou. Mulan: Multimodal-llm agent for progressive multi-object diffusion. arXiv preprint arXiv:2402.12741, 2024

    2024 arXiv

  21. [29]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  22. [30]

    Divide & bind your attention for improved generative semantic nursing

    Yumeng Li, Margret Keuper, Dan Zhang, and Anna Khoreva. Divide & bind your attention for improved generative semantic nursing. In BMVC, 2023

    2023

  23. [31]

    LLM-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models

    Long Lian, Boyi Li, Adam Yala, and Trevor Darrell. LLM-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856

    2024

  24. [32]

    Ctrl-adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model

    Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. Ctrl-adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  25. [33]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  26. [34]

    Tenenbaum

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B. Tenenbaum. Compositional visual generation with composable diffusion models. In European Conference on Computer Vision (ECCV), 2022. 11

    2022

  27. [35]

    Inference-time scaling for diffusion models beyond scaling denoising steps

    Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, and Saining Xie. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09732, 2025

    2025 arXiv

  28. [36]

    Lewis, Thomas Leung, and W

    Wan-Duo Kurt Ma, Avisek Lahiri, J.P. Lewis, Thomas Leung, and W. Bastiaan Kleijn. Directed diffusion: direct control of object placement through attention guidance. In Association for the Advancement of Artificial Intelligence (AAAI), 2024

    2024

  29. [37]

    Guided image synthesis via initial image editing in diffusion model

    Jiafeng Mao, Xueting Wang, and Kiyoharu Aizawa. Guided image synthesis via initial image editing in diffusion model. In Proceedings of the 31st ACM International Conference on Multimedia, 2023

    2023

  30. [38]

    SDEdit: Guided image synthesis and editing with stochastic differential equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations (ICLR), 2022

    2022

  31. [39]

    Conform: Contrast is all you need for high-fidelity text-to-image diffusion models

    Tuna Han Salih Meral, Enis Simsar, Federico Tombari, and Pinar Yanardag. Conform: Contrast is all you need for high-fidelity text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  32. [40]

    T2i- adapter: learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i- adapter: learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

    2024

  33. [41]

    Compositional text-to-image generation with dense blob representations

    Weili Nie, Sifei Liu, Morteza Mardani, Chao Liu, Benjamin Eckart, and Arash Vahdat. Compositional text-to-image generation with dense blob representations. In Forty-first International Conference on Machine Learning, 2024

    2024

  34. [42]

    OpenAI. GPT-4.1. https://openai.com/index/gpt-4-1/ , 2025

    2025

  35. [43]

    Richter, Christopher J

    Pablo Pernias, Dominic Rampas, Mats L. Richter, Christopher J. Pal, and Marc Aubreville. Wuerstchen: An efficient architecture for large-scale text-to-image diffusion models, 2023. URL https://arxiv. org/abs/2306.00637

    2023 arXiv

  36. [44]

    Grounded text-to-image synthesis with attention refocusing

    Quynh Phung, Songwei Ge, and Jia-Bin Huang. Grounded text-to-image synthesis with attention refocusing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  37. [45]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023. URL https://arxiv.org/abs/2307.01952

    2023 arXiv

  38. [46]

    Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment

    Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  39. [47]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  40. [48]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021

    2021

  41. [49]

    Object-attribute binding in text-to-image generation: Evaluation and control

    Maria Mihaela Trusca, Wolf Nuyts, Jonathan Thomm, Robert Honig, Thomas Hofmann, Tinne Tuytelaars, and Marie-Francine Moens. Object-attribute binding in text-to-image generation: Evaluation and control. arXiv preprint arXiv:2404.13766, 2024

    2024 arXiv

  42. [50]

    Plug-and-play diffusion features for text- driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text- driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  43. [51]

    Dynamic prompt learning: Addressing cross-attention leakage for text-based image editing

    Kai Wang, Fei Yang, Shiqi Yang, Muhammad Atif Butt, and Joost van de Weijer. Dynamic prompt learning: Addressing cross-attention leakage for text-based image editing. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  44. [52]

    Instancediffusion: Instance-level control for image generation

    Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra. Instancediffusion: Instance-level control for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  45. [53]

    Tokencompose: Text-to-image diffusion with token-level supervision

    Zirui Wang, Zhizhou Sha, Zheng Ding, Yilin Wang, and Zhuowen Tu. Tokencompose: Text-to-image diffusion with token-level supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 12

    2024

  46. [54]

    Hyperseg: Towards universal visual segmentation with large language model, 2024

    Cong Wei, Yujie Zhong, Haoxian Tan, Yong Liu, Zheng Zhao, Jie Hu, and Yujiu Yang. Hyperseg: Towards universal visual segmentation with large language model, 2024. URL https://arxiv.org/abs/2411. 17606

    2024

  47. [55]

    Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion

    Jinheng Xie, Yuexiang Li, Yawen Huang, Haozhe Liu, Wentian Zhang, Yefeng Zheng, and Mike Zheng Shou. Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  48. [56]

    Mastering text- to-image diffusion: Recaptioning, planning, and generating with multimodal LLMs

    Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Stefano Ermon, and Bin CUI. Mastering text- to-image diffusion: Recaptioning, planning, and generating with multimodal LLMs. In International Conference on Machine Learning (ICML), 2024

    2024

  49. [57]

    Reco: Region-controlled text-to-image generation

    Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Reco: Region-controlled text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  50. [58]

    Creatilayout: Siamese multimodal diffusion transformer for creative layout-to-image generation

    Hui Zhang, Dexiang Hong, Tingwei Gao, Yitong Wang, Jie Shao, Xinglong Wu, Zuxuan Wu, and Yu-Gang Jiang. Creatilayout: Siamese multimodal diffusion transformer for creative layout-to-image generation. arXiv preprint arXiv:2412.03859, 2024

    2024 arXiv

  51. [59]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In International Conference on Computer Vision (ICCV), 2023

    2023

  52. [60]

    Realcompo: Balancing realism and compositionality improves text- to-image diffusion models

    Xinchen Zhang, Ling Yang, YaQi Cai, Zhaochen Yu, Kai-Ni Wang, xie jiake, Ye Tian, Minkai Xu, Yong Tang, Yujiu Yang, and Bin CUI. Realcompo: Balancing realism and compositionality improves text- to-image diffusion models. In The Thirty-eighth Annual Conference on Neural Informa...

    2024

  53. [61]

    Local conditional controlling for text-to-image diffusion models

    Yibo Zhao, Liang Peng, Yang Yang, Zekai Luo, Hengjia Li, Yao Chen, Wei Zhao, Qinglin Lu, Wei Liu, and Boxi Wu. Local conditional controlling for text-to-image diffusion models. CoRR, abs/2312.08768, 2023

    2023 arXiv

  54. [62]

    Layoutdiffusion: Controllable diffusion model for layout-to-image generation

    Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  55. [63]

    = 3 𝑡!= 0.79 𝑁!

    Xingyi Zhou, Vladlen Koltun, and Philipp Krähenbühl. Simple multi-dataset detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 13 Appendix In the appendix, we first discuss the broader impact of our work in Section A. Se...

    2022

  56. [64]

    Objects:

    Foreground object Extraction:a. Extract only foreground objects without severe entanglements as planned.b. Strictly keep the original counts, and enumerate every object one at a time. c. Make sure to extract the accurate counts and enumerate every object one at a time. 4. Obje...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.