LocRef-Diffusion:Tuning-Free Layout and Appearance-Guided Generation

Fan Deng; Guangyu Yan; Jian Yang; Qiang Xu; Xiangjun Huang; Xinyang Yu; Yaguang Wu

REVIEW 4 major objections 4 minor 24 references

LocRef-Diffusion:Tuning-Free Layout and Appearance-Guided Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LocRef-Diffusion claims that a mask-weighted fusion of composite-image and per-instance cross-attention features lets a frozen diffusion model control both the placement and the appearance of multiple objects in one image without…

desk verdict Novel fusion rule with a clean ablation, but the missing SOTA table and a muddled segmentation story keep the paper from being fully convincing as submitted. read the letter →

arxiv 2411.15252 v1 pith:W2ZUH473 submitted 2024-11-22 cs.CV cs.AI

Fan Deng , Yaguang Wu , Xinyang Yu , Xiangjun Huang , Jian Yang , Guangyu Yan , Qiang Xu This is my paper · ORCID

classification cs.CVcs.AI

keywords text-to-imagegenerationdiffusionmodelslayout-guidedpersonalizedimagetuning-freecustomizationcross-attentionmulti-instance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LocRef-Diffusion tries to let a user control two things at once in text-to-image generation: where each object appears and what each object looks like, for several objects in one image. This matters because prior multi-instance generators often misplace objects or let one subject's features bleed into another. The method takes a caption, reference pictures, and bounding boxes, and generates a scene that respects both, without fine-tuning the base model for a new object. The central mechanism is a mask-weighted fusion of cross-attention features: background regions keep the original text-conditioned hidden states, while inside each box the model adds features from a composite image and from each individual reference image. Training only two lightweight modules on COCO and OpenImages, the paper reports that this design beats earlier multi-instance methods on both localization accuracy and appearance similarity.

What carries the argument

The load-bearing object is the hidden-state fusion rule of Eq. (1): $HS=HS*BM+(CIHS+RIHS)*FM$. Here $HS$ is the output of a cross-attention layer in the frozen UNet, $BM$ and $FM$ are binary masks marking background and the union of the target bounding boxes, $CIHS$ is the hidden state obtained by cross-attending over a composite image (each reference pasted into its own box), and $RIHS$ is the hidden state from region-restricted cross-attention over each individual reference image. The rule's function is to let the original text-conditioned state paint the background while the composite and per-instance states control the interior of each box, with the per-instance mask blocking one object's features from leaking into a neighbor's region. The Appearance-Net projection that feeds these states is deliberately small (two linear layers and a normalization layer), which is what keeps the added trainable cost to about 56M parameters.

What would settle it

A test set of reference images where the target object is partially occluded or blends into a visually similar background would settle the foreground-extraction assumption: if the no-mask version's YOLOv8 mAP and CLIP-I scores drop sharply on exactly those images, the projection network does not stand in for segmentation.

Watch

Extended reading notes

Core claim

On its own terms, the paper's claim is that precise multi-instance layout control and high-fidelity appearance transfer are not conflicting goals: they can be combined in a frozen pre-trained diffusion model by attaching a small Appearance-Net and a small Layout-net. Appearance-Net uses a frozen CLIP image encoder followed by a trainable two-layer projection to pull foreground features out of reference images while suppressing background. Layout-net builds a composite image in which each reference is pasted into its target box, encodes it through the same appearance network, and adds separate region-cross-attention layers for each individual reference, masked so that each reference only influences its own box. The two signals are merged by the rule $HS=HS*BM+(CIHS+RIHS)*FM$, which the paper treats as the mechanism that defines generation boundaries and prevents cross-attention leakage. On COCO2017 and OpenImages, the authors report that this setup outperforms GLIGEN, MS-Diffusion, and MIGC on the reported mAP and CLIP-I metrics.

Load-bearing premise

The load-bearing assumption is that a small two-layer projection can separate a reference image's foreground from its background well enough to skip instance segmentation at inference, even though the paper's own ablation shows that adding a segmentation mask improves the localization metric (mAP) from 0.62 to 0.683.

Editorial extensions

If this is right

A user can specify several reference images and boxes in a single prompt and receive one coherent scene, with no test-time fine-tuning for the new objects.
Because the base model weights are frozen, the added modules can be reused on open-world reference images beyond the training categories.
The mask-split fusion rule gives a clean design principle: keep global coherence in the background hidden states and put per-instance identity inside each box.
Training cost stays comparatively low because only the Appearance-Net and Layout-net parameters (about 56M) are updated while the base model stays fixed.
On the reported evaluations, the method's localization and appearance scores exceed those of the compared multi-instance baselines, meaning fewer mislocated objects and less feature leakage in practice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the same mask-weighted fusion rule could be applied to other conditioning modalities: replace the composite-image embedding with depth, sketch, or semantic-map features and keep the mask split between region and background.
Reading the ablation as a design hint, a testable variant is to make the instance-segmentation branch optional and use its predicted mask only when the projection network's foreground confidence is low, potentially capturing part of the reported mAP gain without a full segmentation model.
Because the reported evaluation relies on CLIP-I and a YOLOv8 detector, a natural next test is whether images generated this way improve downstream tasks such as instance segmentation or human judgments of layout and identity, rather than only detector and CLIP scores.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the specific fusion rule in Eq. 1 — composite-image embedding plus masked per-instance reference embeddings in one cross-attention update — is genuinely new, and the ablation shows each module earns its place. But the paper as submitted can't support its headline SOTA claim, and the segmentation-free story has a hole that needs patching.

What's good: the method keeps the base model frozen and adds ~56M trainable parameters, which is lightweight. The ablation is monotonic and the individual gains are sensible: region guidance lifts mAP from 0.232 to 0.548, and instance-aware fusion adds more. The qualitative figures look clean. The writing is mostly clear, and the related work is appropriately cited — no obvious citation inflation.

Where it's soft: Table I, the only quantitative comparison against GLIGEN, MS-Diffusion, and MIGC, is missing from the manuscript. You can read the caption and the text claims 'superior performance,' but the numbers aren't there. That's the load-bearing evidence for the SOTA claim, and it's absent.

Second, the instance segmentation ambiguity is real. Section III-B explicitly argues they avoid a segmentation model, saying the projection network can extract clean foreground features. But the best ablation row (config 4) adds 'Instance Segmentation' and gains 0.62→0.683 mAP and 0.812→0.828 CLIP-I. The ablation text admits the model without the mask filter suffers from appendages spilling outside the bbox. The paper never says which configuration is the final method. If it's config 3, then the method is as described, and config 4 is an optional refinement; if it's config 4, the 'tuning-free without segmentation' claim is wrong at inference. The authors need to state this explicitly.

Third, no error bars or significance tests anywhere, and the training and evaluation sets overlap (COCO/OpenImages both used for training and testing). They call it zero-shot, but zero-shot relative to what? This needs a clearer statement of held-out categories. The CLIP-I circularity is a known issue in this literature — the appearance net conditions on CLIP embeddings and CLIP-I measures the same space — but it's minor and not unique to this paper.

Bottom line: the core idea is plausible and the ablation is honest, but the missing table and the segmentation ambiguity mean I can't verify the claims as written. This is addressable in revision. I'd send it to peer review — the fusion mechanism deserves scrutiny — but I'd ask for the full table, a precise description of the inference-time segmentation usage, and a dataset split that supports the zero-shot language.

Referee Report

4 major / 4 minor

Summary. The paper proposes LocRef-Diffusion, a tuning-free framework for multi-instance layout- and appearance-guided image generation. The method freezes a pre-trained Stable Diffusion model and trains only a Layout-net and an Appearance-Net (about 56M parameters). Appearance features are extracted via a frozen CLIP image encoder followed by a two-layer projection; layout control is achieved by placing reference images into bounding boxes on a black canvas, injecting the composite-image embedding and per-instance reference-image embeddings through additional cross-attention layers, and fusing the hidden states via the rule HS = HS*BM + (CIHS + RIHS)*FM (Eq. 1). Experiments are reported on COCO2017 and OpenImages using YOLOv8 mAP for localization and CLIP-I for appearance fidelity, with ablations over three proposed modules.

Significance. If the reported results are reproducible and the final inference protocol is exactly as described, the paper would make a useful contribution: it offers a lightweight, tuning-free alternative for multi-instance personalization, with an independent localization metric (YOLOv8 mAP) and several ablations showing monotonic improvements from the proposed components. The architectural idea of decoupling instance-level cross-attention and gating the final hidden states by bounding-box masks is plausible and clearly presented. However, the significance is currently conditional because the central state-of-the-art claim is not verifiable (Table I is missing), the inference protocol appears internally inconsistent regarding instance segmentation, and the zero-shot claim is weakened by the lack of explicit train/test separation.

major comments (4)

[IV-C and Table I] Table I, which is the only quantitative support for the central claim of state-of-the-art performance against GLIGEN, MS-Diffusion, and MIGC, is missing from the manuscript. Only the caption and a reference to it in the text are present. Without the actual numbers, the headline claim in the abstract cannot be checked. The authors must supply the full comparison table, including the control conditions (bbox/img), all metrics, and the specific dataset splits used for each entry.
[III-B vs IV-D (Table II)] There is an internal inconsistency about the final inference protocol. Section III-B explicitly states that an instance segmentation model is deliberately avoided, and that the two-layer projection network is expected to output relatively clean foreground features. However, Table II configuration 4 ('Instance Segmentation') adds an explicit foreground segmentation mask and yields the best scores, improving YOLO mAP from 0.62 to 0.683 and CLIP-I from 0.812 to 0.828. If configuration 4 is the reported model, then the method is not segmentation-free at inference and may gain an unfair advantage over baselines that do not use mask preprocessing. If configuration 3 is the reported model, then the best row in the ablation is not the method being claimed. Please clarify which configuration constitutes the final method and describe the exact test-time preprocessing for reference images.
[IV-A and IV-C] The training and evaluation datasets are not separated. Section IV-A states that the training data is built from COCO2017 and OpenImage, and Section IV-C evaluates on COCO2017 and OpenImages. No train/test split, category split, or seen/unseen split is described. As a result, the 'zero-shot open-world' claim cannot be assessed, because the model may have been trained and evaluated on the same distributions or even the same images. The authors should state the exact split protocol, and, ideally, evaluate on held-out categories or datasets to support the zero-shot claim.
[IV-B and Table II] The primary appearance metric, CLIP-I, measures cosine similarity in a CLIP embedding space that is also the feature space used by the frozen CLIP image encoder in Appearance-Net. Part of the reported appearance fidelity gain may therefore be expected by construction, rather than indicating true perceptual fidelity. The authors should report at least one additional appearance metric that is not tied to the conditioning representation (for example, DINO feature similarity, LPIPS, or a user study), and should provide per-instance rather than only averaged scores. In addition, Table II reports no variance or significance tests, so the magnitudes of the improvements, some of which are small, cannot be assessed.

minor comments (4)

[III-C, Eq. 1 and Eq. 2] The definitions of FM and BM should be stated precisely: are they binary 0/1 masks derived only from bounding boxes, or can they be soft masks, and does the final model ever replace FM with instance-segmentation masks? The notation appears to shift between the method description and the ablation discussion.
[IV-D, Table II] The row labels in Table II are hard to read because index 1 contains a bare '1' and the table is not typeset with clear column separations. Please format the table so that each configuration is unambiguous.
[References] Several references are incomplete or informally formatted, e.g., reference [17] is missing the actual paper title, and reference [12] appears to be truncated. The reference list should be cleaned up to journal standards.
[IV-C] The qualitative results in Fig. 2 are described in only one sentence. Please add a more detailed qualitative analysis, including failure cases, to help readers understand the conditions under which the method degrades.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the layout fusion and appearance conditioning are design choices, not predictions reduced to inputs.

full rationale

The paper contains no circular derivation in the sense of this review. The fusion rule HS=HS*BM+(CIHS+RIHS)*FM is a design choice for how layout masks and appearance embeddings are combined; it does not make the evaluated YOLO mAP or CLIP-I equal an input by construction, because the diffusion decoder must still produce detectable, recognizable objects within the masked regions. The appearance branch uses a frozen CLIP image encoder for conditioning, and CLIP-I also uses CLIP embeddings; this shared representation is a metric-alignment concern, but the paper's training loss is a foreground-weighted diffusion loss (Eq. 2), not a direct CLIP-I objective, and the output CLIP-I is not fed back into the model. The only self-citation (DetDiffusion, Ref. [4], co-authored by Q. Xu) appears as a generic conditioning-method citation and is not load-bearing. The internal inconsistency about whether instance segmentation is used at inference (Section III-B vs. Table II row 4) and the missing Table I are substantive correctness/evidence problems, but they are not circularity. Hence no circular step is identifiable from the manuscript text.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on architectural choices and evaluation assumptions rather than new physical postulates. The main burdens are the hand-set loss weights, the untested additive fusion rule, and the assumed sufficiency of CLIP features and YOLO-based metrics.

free parameters (3)

Foreground loss weight W_f = 2.3
Hand-set weighting in Eq. 2 to emphasize foreground fidelity; affects the trade-off between layout and appearance.
Background loss weight W_b = 0.3
Hand-set weight in Eq. 2; the relative balance with W_f is not justified by a search or theory.
Trainable parameter count = 56M
Size of the new modules, fixed by architectural choices rather than fitted to data; listed for transparency.

assumptions (5)

domain assumption CLIP image features capture sufficient appearance information for faithful reference transfer
Appearance-Net relies on a frozen CLIP encoder; no independent validation that CLIP features are sufficient for fine-grained appearance.
domain assumption Adding new cross-attention layers to frozen Stable Diffusion preserves base model capability
Assumed throughout training; potential degradation of text-driven background quality is not measured.
ad hoc to paper Additive fusion of CIHS and RIHS hidden states in Eq. 1 yields valid feature composition
No derivation or comparison to alternative fusion rules; only ablation against removing modules.
domain assumption YOLOv8 mAP on generated images measures layout accuracy faithfully
Detection scores depend on the detector's own box statistics; no human evaluation is reported.
domain assumption COCO2017 and OpenImages provide a distribution that transfers to open-world categories
Training and evaluation on the same datasets with unspecified split details weakens this assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LocRef-Diffusion:Tuning-Free Layout and Appearance-Guided Generation." pith.science (2026). https://pith.science/paper/W2ZUH473

@misc{pith2026241115252,
  author       = {Pith},
  title        = {Pith review of: LocRef-Diffusion:Tuning-Free Layout and Appearance-Guided Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W2ZUH473}},
  note         = {Machine review of arXiv:2411.15252}
}

read the original abstract

Recently, text-to-image models based on diffusion have achieved remarkable success in generating high-quality images. However, the challenge of personalized, controllable generation of instances within these images remains an area in need of further development. In this paper, we present LocRef-Diffusion, a novel, tuning-free model capable of personalized customization of multiple instances' appearance and position within an image. To enhance the precision of instance placement, we introduce a Layout-net, which controls instance generation locations by leveraging both explicit instance layout information and an instance region cross-attention module. To improve the appearance fidelity to reference images, we employ an appearance-net that extracts instance appearance features and integrates them into the diffusion model through cross-attention mechanisms. We conducted extensive experiments on the COCO and OpenImages datasets, and the results demonstrate that our proposed method achieves state-of-the-art performance in layout and appearance guided generation.

Figures

Figures reproduced from arXiv: 2411.15252 by the authors.

**Figure 1.** The overall architecture of our proposed LocRef-Diffusion with Layout-net and Appearance-Net. Only the newly added modules (in red color) are [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Qualitative result for multi-instance generation. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 6 canonical work pages

[1]

LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs,

Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Je- nia Jitsev, and Aran Komatsuzaki, “LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs,” [Online]. Available: https://arxiv.org/abs/2111.02114, 2021

arXiv 2021
[2]

Adding Condi- tional Control to Text-to-Image Diffusion Models,

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala, “Adding Condi- tional Control to Text-to-Image Diffusion Models,” [Online]. Available: https://arxiv.org/abs/2302.05543, 2023

arXiv 2023
[3]

Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Genera- tion,

Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or, “Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Genera- tion,” [Online]. Available: https://arxiv.org/abs/2403.16990, 2024

arXiv 2024
[4]

DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception,

Yibo Wang, Ruiyuan Gao, Kai Chen, Kaiqiang Zhou, Yingjie Cai, Lan- qing Hong, Zhenguo Li, Lihui Jiang, Dit-Yan Yeung, Qiang Xu, and Kai Zhang, “DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception,” [Online]. Available: https://arxiv.org/abs/2403.13304, 2024

arXiv 2024
[5]

Training-Free Lay- out Control with Cross-Attention Guidance,

Minghao Chen, Iro Laina, and Andrea Vedaldi, “Training-Free Lay- out Control with Cross-Attention Guidance,” [Online]. Available: https://arxiv.org/abs/2304.03373, 2023

arXiv 2023
[6]

Multi-Region Text-Driven Manipulation of Diffusion Imagery,

Yiming Li, Peng Zhou, Jun Sun, and Yi Xu, “Multi-Region Text-Driven Manipulation of Diffusion Imagery,” in Proc. AAAI Conf. Artif. Intell., Mar. 2024, vol. 38, no. 4, pp. 3261-3269

work page 2024
[7]

GLIGEN: Open- Set Grounded Text-to-Image Generation,

Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee, “GLIGEN: Open- Set Grounded Text-to-Image Generation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., June 2023, pp. 22511-22521

work page 2023
[8]

Subject-Diffusion: Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning,

Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu, “Subject-Diffusion: Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning,” 2023

work page 2023

Show all 24 references

[9]

IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models,

Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang, “IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models,” [Online]. Available: https://arxiv.org/abs/2308.06721, 2023

2023 arXiv
[10]

FastComposer: Tuning-Free Multi-Subject Image Genera- tion with Localized Attention,

Guangxuan Xiao, Tianwei Yin, William Freeman, Frédo Durand, and Song Han, “FastComposer: Tuning-Free Multi-Subject Image Genera- tion with Localized Attention,” 10.48550/arXiv.2305.10431, 2023

2023 doi arXiv
[11]

InstantStyle: Free Lunch towards Style- Preserving in Text-to-Image Generation,

Haofan Wang, Matteo Spinelli, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen, “InstantStyle: Free Lunch towards Style- Preserving in Text-to-Image Generation,” [Online]. Available: https://arxiv.org/abs/2404.02733, 2024

2024 arXiv
[12]

High- fidelity Person-centric Subject-to-Image Synthesis,

Yibin Wang, Weizhong Zhang, Jianwei Zheng, and Cheng Jin, “High- fidelity Person-centric Subject-to-Image Synthesis,” [Online]. Available: https://arxiv.org/abs/2311.10329, 2024

2024 arXiv
[13]

Denoising Diffusion Prob- abilistic Models,

Jonathan Ho, Ajay Jain, and Pieter Abbeel, “Denoising Diffusion Prob- abilistic Models,” [Online]. Available: https://arxiv.org/abs/2006.11239, 2020

2006 arXiv
[14]

Hierarchical Text-Conditional Image Generation with CLIP Latents,

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen, “Hierarchical Text-Conditional Image Generation with CLIP Latents,” [Online]. Available: https://cdn.openai.com/papers/dall-e-2.pdf, 2022

2022
[15]

Learning Transferable Vi- sual Models From Natural Language Supervision,

Alec Radford, Jong Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever, “Learning Transferable Vi- sual Models From Natural Language Supervision,” [Online]. Available: ...

2021 arXiv
[16]

High-Resolution Image Synthesis with Latent Dif- fusion Models,

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer, “High-Resolution Image Synthesis with Latent Dif- fusion Models,” [Online]. Available: https://arxiv.org/abs/2112.10752, 2022

2022 arXiv
[17]

In Proceedings of the 31st International Conference on International Con- ference on Machine Learning,

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra, “In Proceedings of the 31st International Conference on International Con- ference on Machine Learning,” 2014

2014
[18]

Resolution-robust Large Mask Inpainting with Fourier Convolutions,

Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky, “Resolution-robust Large Mask Inpainting with Fourier Convolutions,” [Online]. Available: https://arxiv...

2021 arXiv
[19]

MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis,

Dewei Zhou, You Li, Fan Ma, Xiaoting Zhang, and Yi Yang, “MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis,” [On- line]. Available: https://arxiv.org/abs/2402.05408, 2024

2024 arXiv
[20]

StyleAlign: Analysis and Applications of Aligned StyleGAN Models,

Zongze Wu, Yotam Nitzan, Eli Shechtman, and Dani Lischinski, “StyleAlign: Analysis and Applications of Aligned StyleGAN Models,” [Online]. Available: https://arxiv.org/abs/2110.11323, 2022

2022 arXiv
[21]

LayoutDiffusion: Controllable Diffu- sion Model for Layout-to-image Generation,

Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li, “LayoutDiffusion: Controllable Diffu- sion Model for Layout-to-image Generation,” [Online]. Available: https://arxiv.org/abs/2303.17189, 2024

2024 arXiv
[22]

GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation,

Kai Chen, Enze Xie, Zhe Chen, Yibo Wang, Lanqing Hong, Zhen- guo Li, and Dit-Yan Yeung, “GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation,” [Online]. Available: https://arxiv.org/abs/2306.04607, 2024

2024 arXiv
[23]

Denoising Diffusion Implicit Models,

Jiaming Song, Chenlin Meng, and Stefano Ermon, “Denoising Diffusion Implicit Models,” [Online]. Available: https://arxiv.org/abs/2010.02502, 2022

2010 arXiv
[24]

MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance,

Xiaowei Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang, “MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance,” [Online]. Available: https://arxiv.org/abs/2406.07209, 2024

2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs,

Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Je- nia Jitsev, and Aran Komatsuzaki, “LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs,” [Online]. Available: https://arxiv.org/abs/2111.02114, 2021

arXiv 2021

[2] [2]

Adding Condi- tional Control to Text-to-Image Diffusion Models,

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala, “Adding Condi- tional Control to Text-to-Image Diffusion Models,” [Online]. Available: https://arxiv.org/abs/2302.05543, 2023

arXiv 2023

[3] [3]

Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Genera- tion,

Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or, “Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Genera- tion,” [Online]. Available: https://arxiv.org/abs/2403.16990, 2024

arXiv 2024

[4] [4]

DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception,

Yibo Wang, Ruiyuan Gao, Kai Chen, Kaiqiang Zhou, Yingjie Cai, Lan- qing Hong, Zhenguo Li, Lihui Jiang, Dit-Yan Yeung, Qiang Xu, and Kai Zhang, “DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception,” [Online]. Available: https://arxiv.org/abs/2403.13304, 2024

arXiv 2024

[5] [5]

Training-Free Lay- out Control with Cross-Attention Guidance,

Minghao Chen, Iro Laina, and Andrea Vedaldi, “Training-Free Lay- out Control with Cross-Attention Guidance,” [Online]. Available: https://arxiv.org/abs/2304.03373, 2023

arXiv 2023

[6] [6]

Multi-Region Text-Driven Manipulation of Diffusion Imagery,

Yiming Li, Peng Zhou, Jun Sun, and Yi Xu, “Multi-Region Text-Driven Manipulation of Diffusion Imagery,” in Proc. AAAI Conf. Artif. Intell., Mar. 2024, vol. 38, no. 4, pp. 3261-3269

work page 2024

[7] [7]

GLIGEN: Open- Set Grounded Text-to-Image Generation,

Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee, “GLIGEN: Open- Set Grounded Text-to-Image Generation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., June 2023, pp. 22511-22521

work page 2023

[8] [8]

Subject-Diffusion: Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning,

Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu, “Subject-Diffusion: Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning,” 2023

work page 2023