Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

SceneBooth: Diffusion-based Framework for Subject-preserved Text-to-Image Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SceneBooth claims that fixing the subject image and generating only a layout-guided background preserves subject fidelity better than methods that regenerate the subject from a learned representation.

desk verdict Solid task reframing with a promising two-stage design, but the headline outperformance claim is undermined by an unfair baseline protocol and must be re-evaluated. read the letter →

arxiv 2501.03490 v1 pith:PSW26WKT submitted 2025-01-07 cs.CV

Shang Chai , Zihang Lin , Min Zhou , Xubin Li , Liansheng Zhuang , Houqiang Li This is my paper · ORCID
classification cs.CV
keywords subject-preservedtext-to-imagegenerationlayoutdiffusionmodelslatentControlNetgatedself-attentionsubjectfidelitybackgroundinpainting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SceneBooth is a framework for subject-preserved text-to-image generation: given one subject image, a scene caption, and phrases describing the other objects, it keeps the subject pixels exactly as provided and generates only the surrounding scene. The paper's central claim is that this "fix the subject, paint the background" strategy sidesteps the fidelity-diversity trade-off that plagues subject-driven methods such as DreamBooth, so logos, textures, and fine details survive unchanged. The framework first predicts a scene layout that positions and scales the subject and other objects, then paints a background that harmonizes with the pasted subject using a latent diffusion model with layout and visual control. If the claim holds, personalized image generation for products, selfies, and illustrations becomes a matter of background synthesis rather than subject resynthesis.

What carries the argument

Two components carry the argument. MCLayoutDM is a diffusion-based layout denoiser built on LayoutDM, extended with a cross-attention layer so that a CLIP text embedding of the caption and a CLIP visual embedding of the randomly scaled and padded subject image guide the denoising of object bounding boxes. PaintNet is a latent diffusion model built on LDM and Stable Diffusion, into which ControlNet injects a conditioning image of the subject pasted on a blank canvas and gated self-attention layers inject grounding tokens derived from object phrases and bounding boxes; its defining design choice is training with instance masks rather than random masks, so the model learns that the known region is a complete subject.

What would settle it

Run SceneBooth on a subject image that includes background clutter or a pose that conflicts with the generated layout, such as a lying person assigned a standing-object box; if the output shows a visible cut-out seam around the subject or an implausible object relation, the claimed harmonization is falsified for that input class. A simpler quantitative check is to feed the same subject image with and without a tight, background-free crop and compare DINO and CLIP-I scores along with boundary artifact rates.

Watch

Extended reading notes

Core claim

On its own terms, the paper argues that precise subject preservation is better achieved by construction than by representation learning: rather than embedding the subject into the prompt and regenerating it, SceneBooth treats the input subject image as fixed foreground and treats text-to-image generation as a conditional background-painting problem. The layout module conditions a diffusion layout denoiser on CLIP text and image features, producing bounding boxes for the subject and every background object; the painting module then feeds a canvas with the subject pasted at its predicted box into a latent diffusion model via ControlNet, while gated self-attention layers inject layout tokens. Training PaintNet with instance masks, where the known region is the whole subject rather than a random hole, teaches the model that the unmasked area is a complete subject whose surroundings must blend with it. The reported outcome is that SceneBooth beats inpainting-based baselines on FID, CLIP-I, and DINO, and wins the majority of human preference judgments on quality, fidelity, object-phrase match, and caption alignment.

Load-bearing premise

The framework assumes the user supplies a single, clean, unoccluded subject image whose aspect ratio fits its predicted bounding box, plus object phrases for every other scene object; if the subject carries background pixels, is partly occluded, or is squeezed into a mismatched box, the pasted foreground cannot blend cleanly and the harmonization claim breaks down.

Editorial extensions

If this is right

  • If the claim is correct, subject fidelity no longer competes with prompt diversity: the subject is literally the input pixels, so logos and fine textures cannot drift.
  • Object-level layout prediction makes counting and positional relationships explicit, which the paper's human evaluations show standard CLIP-T alignment scores underweight.
  • Because the background painter inherits a large pretrained latent diffusion model, SceneBooth can place unseen subjects, such as a Hello Kitty doll, into novel scenes without retraining for that subject.
  • Editing the subject's bounding box acts as a drag control, repositioning the subject while preserving its appearance.
  • The measured gap between random-mask and instance-mask training suggests the same painting module could serve other "complete foreground, synthesize surroundings" tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own limitations section implies that its central claim is conditional on the subject image being clean: adding a segmentation step that strips background pixels before pasting is a direct, testable extension that should remove the reported occlusion and seam failures.
  • Because the method assumes exactly one preserved subject, a natural multi-subject extension is to run layout generation per object and composite several fixed foregrounds one at a time; the paper does not attempt this.
  • The comparison set is inpainting models rather than subject-driven personalization models, so the broadest inference, that fixed pixels are a stronger fidelity prior than learned subject embeddings, needs a direct head-to-head with DreamBooth-style methods under identical prompts to be confirmed.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes SceneBooth, a two-stage framework for subject-preserved text-to-image generation. Given a subject image, object phrases, and a scene caption, MCLayoutDM first generates a scene layout that determines the subject's position and scale along with boxes for other objects; PaintNet then synthesizes a background using a latent diffusion model conditioned on the layout, the caption, the object phrases, and the subject image injected via ControlNet and Gated Self-Attention. The final output composites the pasted subject with the generated background using a binary mask. The method is evaluated on COCO 2017 against inpainting baselines (StableDiffusion-inpaint and ControlNet-inpaint) and through ablations of ControlNet, mask strategy, attention type, and visual embedding. The headline claim is that SceneBooth significantly outperforms baselines in subject preservation, image harmonization, and overall quality.

Significance. If the evaluation were fully convincing, SceneBooth would make a useful contribution by showing that avoiding subject resynthesis altogether is a practical route to high-fidelity customization, complementary to DreamBooth-style finetuning. The paper is clearly written, the two modules are sensible, and the open-world examples suggest the framework generalizes beyond COCO categories. The ablations (Tables II-V) provide useful evidence about the design choices, and the authors honestly disclose limitations in Section V. However, the central comparative claim is weakened by an evaluation protocol that conflates layout quality with background-painting quality and by subject-fidelity metrics that partly reflect the copy-and-paste formulation rather than the learned components. The contribution is still credible and potentially publishable, but the experimental evidence needs reworking before the strong superiority claim can be accepted.

major comments (4)
  1. [Section IV.B, Table I] The random-placement protocol for the inpainting baselines, stated in note (2) of Section IV.B, confounds the layout module's contribution with the background-painting module's contribution. Baselines receive subjects at random positions with 20% scale variation while SceneBooth uses its learned layout, so the FID and human-preference differences cannot be attributed to PaintNet alone. The ground-truth-layout ablation in Table II is fairer, but it compares PaintNet only against GLIGEN variants, not against the same inpainting baselines used in Table I. I recommend rerunning Table I with all methods given identical layouts (either ground-truth or SceneBooth-generated), so that the comparison isolates background-painting quality from layout prediction.
  2. [Equations (2)-(3), Table I] The CLIP-I and DINO advantages in Table I are partly by construction. Equation (3) shows that the output subject pixels are exactly the rescaled and pasted input subject, never regenerated by the diffusion model. Any method that pastes the input subject without any learned background generation would inherit most of this subject-fidelity benefit, so these metrics do not measure the learned model's contribution to subject preservation. I suggest adding a paste-only baseline (subject pasted onto a blank or trivial background) and reporting subject-fidelity metrics restricted to the background region, or otherwise quantifying the added value of PaintNet beyond the copy-and-paste operation.
  3. [Section IV.C, Table II] The ablation intended to isolate the effect of ControlNet does not do so cleanly. GLIGEN-repaint and GLIGEN-inpaint receive no visual information about the subject, whereas PaintNet receives the subject image through ControlNet. The large differences in CLIP-I, DINO, and human preference could therefore be due to the mere presence of subject appearance conditioning rather than to ControlNet specifically. A stronger ablation would give the GLIGEN variants access to the subject's visual features through some other mechanism (for example, concatenating the subject image as an extra conditioning channel), or would compare PaintNet to a variant that replaces ControlNet with a different image-conditioning adapter while keeping the same information available.
  4. [Section V, Limitations] The paper's own limitations section, together with Fig. 8, shows that the framework assumes a clean, unoccluded, single subject whose bounding box is compatible with its aspect ratio (also stated in Section III.A). When these assumptions are violated, the paste-and-blend strategy leads to visible seams or irrational layouts, which undermines the general harmonization claim. This scope restriction should be stated as a central limitation in the abstract or introduction, and the paper should discuss or at least acknowledge potential mitigations (e.g., matting, subject-hole inpainting) rather than leaving the issue as a peripheral failure case.
minor comments (6)
  1. [Section IV.D] The text refers to 'Figure 5.7' when discussing open-world generation, but the actual figure is Fig. 7; please correct the cross-reference.
  2. [Section III.D, Eq. (12)] The token selection operation TS is used in Eq. (12) but never defined before its first use; please define it in the text or in the equation caption.
  3. [Section IV.A] The dataset name is written as 'COCO2017' in several places; use 'COCO 2017' for consistency with the reference [22].
  4. [References] References [6] and [29] both cite the same DisenBooth paper; please consolidate or differentiate them.
  5. [Section IV.A] The extension to Max. IoU @ k is described as detailed in supplementary materials, but no supplementary material is included in the arXiv submission; please include the detail or remove the reference.
  6. [Table I caption] The human-evaluation metrics Pquality, Pfidelity, Pobjf, and Ppromptf are introduced in the text but not defined in the table caption; adding a one-line definition would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: subject preservation is a task definition, not a derived prediction, and the learned components are evaluated against external baselines.

full rationale

SceneBooth's output is defined by construction as a compositing of the input subject with a PaintNet-painted background (Eqs. 2-3), so the 'subject-preserved' property is a task formulation rather than an empirical prediction. This is not a circular derivation because the paper's measurable claims target background generation and layout quality, which are not fixed by the compositing formula: PaintNet must still synthesize a coherent background, and MCLayoutDM must infer subject position and scale from text, phrases, and image features. The headline comparisons in Table I are against inpainting baselines that also receive a pasted subject (Section IV.B note 2), so the CLIP-I and DINO deltas are not forced by the formula alone. The only author-overlap citation is LayoutDM [19], used openly as the MCLayoutDM backbone; Table III evaluates MCLayoutDM against LayoutDM and other layout generators on Max. IoU and FID, so the self-citation is not load-bearing in a way that closes the argument. ControlNet and Gated Self-Attention come from external works [14,21]. No fitted parameter is relabeled as a prediction, and no uniqueness theorem is imported from the authors' prior work. The limitations section (Fig. 8) further documents failure modes, which is inconsistent with a tautological claim. Overall, no step in the derivation chain reduces to its own inputs, so the paper is self-contained against external benchmarks and receives a score of 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical entities or conserved quantities are introduced. The framework's modules (MCLayoutDM, PaintNet) are learned artifacts, not invented entities. The central claim rests on standard diffusion models and pretrained encoders, plus the hand-set parameters and task-specific assumptions listed above; the main empirical free parameters are the data filtering range, the augmentation scale, the GLIGEN gate constant, and the training schedule.

free parameters (4)
  • Object-count filtering range = 3 to 8 objects
    Samples with fewer than 3 or more than 8 objects are removed from COCO (Section IV.A); this threshold shapes layout complexity and directly affects the training distribution.
  • Subject random-scale augmentation range = 20% scale variation
    In MCLayoutDM training, the subject image is randomly scaled within 20% before pasting (Section III.C), a hand-set augmentation that controls position and scale leakage.
  • Gated self-attention scaling beta = beta = 1
    Follows GLIGEN (Section III.D), a hand-set constant controlling gating strength.
  • Learning rates and iterations = MCLayoutDM LR 1e-5, 400k iters; PaintNet LR 5e-5, 102k iters
    Chosen hyperparameters reported in Section IV.A that affect final model quality and are not derived from theory.
assumptions (4)
  • standard math The denoising objective (epsilon-prediction from DDPM/LDM) is a valid training target for the layout and image generation modules.
    Invoked in Equation 13 and Section III.E; the paper relies on the standard diffusion training framework without proving its convergence.
  • domain assumption CLIP text and image encoders provide semantic embeddings sufficient for layout and background generation.
    Used in Section III.C and III.D as frozen encoders; the method assumes CLIP features capture object semantics and layout-relevant visual information.
  • domain assumption COCO ground-truth segmentation, layout, and caption annotations are accurate and representative for training the two-stage framework.
    The dataset split and annotations are used for training and evaluation (Section IV.A); any annotation noise directly transfers to the learned modules.
  • ad hoc to paper A single subject with given object phrases and caption is sufficient input to determine a plausible scene layout.
    Stated as an assumption in Section III.A; this task-specific premise limits applicability to simple scenes and is not a general property of text-to-image generation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SceneBooth: Diffusion-based Framework for Subject-preserved Text-to-Image Generation." pith.science (2026). https://pith.science/paper/PSW26WKT

@misc{pith2026250103490,
  author       = {Pith},
  title        = {Pith review of: SceneBooth: Diffusion-based Framework for Subject-preserved Text-to-Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PSW26WKT}},
  note         = {Machine review of arXiv:2501.03490}
}
read the original abstract

Due to the demand for personalizing image generation, subject-driven text-to-image generation method, which creates novel renditions of an input subject based on text prompts, has received growing research interest. Existing methods often learn subject representation and incorporate it into the prompt embedding to guide image generation, but they struggle with preserving subject fidelity. To solve this issue, this paper approaches a novel framework named SceneBooth for subject-preserved text-to-image generation, which consumes inputs of a subject image, object phrases and text prompts. Instead of learning the subject representation and generating a subject, our SceneBooth fixes the given subject image and generates its background image guided by the text prompts. To this end, our SceneBooth introduces two key components, i.e., a multimodal layout generation module and a background painting module. The former determines the position and scale of the subject by generating appropriate scene layouts that align with text captions, object phrases, and subject visual information. The latter integrates two adapters (ControlNet and Gated Self-Attention) into the latent diffusion model to generate a background that harmonizes with the subject guided by scene layouts and text descriptions. In this manner, our SceneBooth ensures accurate preservation of the subject's appearance in the output. Quantitative and qualitative experimental results demonstrate that SceneBooth significantly outperforms baseline methods in terms of subject preservation, image harmonization and overall quality.

Figures

Figures reproduced from arXiv: 2501.03490 by the authors.

Figure 1
Figure 1. Results generated by subject-driven and subject-perseved text-to-image [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed SceneBooth. It consists of a layout generation module, MCLayoutDM, and a background painting module, PaintNet. We [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. (a) Architecture of the layout denoiser in MCLayoutDM. Fourier, SA, CA, and FFN denote the fourier embedding layer, self-attention layer, cross [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison with existing methods on COCO dataset. The subject in object phrases is highlighted in red. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Ablation study on the effectiveness of ControlNet. We qualitatively compare PaintNet with GLIGEN-repaint and GLIGEN-inpaint on test dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of subject translation in the scene layout. The direction of translation is indicated using dots and arrows. Row 1-3 have captions: [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Generation under open-world setting. Row 1-2 have captions: “A [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Two problematic cases. Row 1-2 have captions: “A man in shorts is [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. T-Stars-Poster: A Framework for Product-Centric Advertising Image Design

    cs.CV 2025-01 conditional novelty 5.0 of 10

    A four-stage product-centric framework automatically creates advertising posters from a product photo, taglines, and target size, with better visual-quality scores and about 3 percent higher click-through rates in Tao...

Reference graph

Works this paper leans on

61 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [1]

    Multi- concept customization of text-to-image diffusion,

    N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J.-Y . Zhu, “Multi- concept customization of text-to-image diffusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 1931–1941

    2023

  2. [2]

    Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,

    N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 22 500–22 510

    2023

  3. [3]

    Paint by example: Exemplar-based image editing with diffusion models,

    B. Yang, S. Gu, B. Zhang, T. Zhang, X. Chen, X. Sun, D. Chen, and F. Wen, “Paint by example: Exemplar-based image editing with diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 18 381–18 391

    work page 2023

  4. [4]

    An image is worth one word: Person- alizing text-to-image generation using textual inversion,

    R. Gal, Y . Alaluf, Y . Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or, “An image is worth one word: Person- alizing text-to-image generation using textual inversion,” arXiv preprint arXiv:2208.01618, 2022

    arXiv 2022

  5. [5]

    Blip-diffusion: Pre-trained subject representa- tion for controllable text-to-image generation and editing,

    D. Li, J. Li, and S. Hoi, “Blip-diffusion: Pre-trained subject representa- tion for controllable text-to-image generation and editing,” Advances in Neural Information Processing Systems , vol. 36, 2024

    work page 2024

  6. [7]

    Anydoor: Zero-shot object-level image customization,

    X. Chen, L. Huang, Y . Liu, Y . Shen, D. Zhao, and H. Zhao, “Anydoor: Zero-shot object-level image customization,” arXiv preprint arXiv:2307.09481, 2023

    arXiv 2023

  7. [8]

    Compositional gan: Learning image-conditional binary composition,

    S. Azadi, D. Pathak, S. Ebrahimi, and T. Darrell, “Compositional gan: Learning image-conditional binary composition,” International Journal of Computer Vision , vol. 128, no. 10, pp. 2570–2585, 2020

    work page 2020

  8. [9]

    Painterly image har- monization using diffusion model,

    L. Lu, J. Li, J. Cao, L. Niu, and L. Zhang, “Painterly image har- monization using diffusion model,” in Proceedings of the 31st ACM International Conference on Multimedia , 2023, pp. 233–241

    work page 2023

Show all 61 references
  1. [10]

    Learning object placement via dual-path graph completion,

    S. Zhou, L. Liu, L. Niu, and L. Zhang, “Learning object placement via dual-path graph completion,” in European Conference on Computer Vision. Springer, 2022, pp. 373–389

    2022

  2. [11]

    Bridging composite and real: towards end-to-end deep image matting,

    J. Li, J. Zhang, S. J. Maybank, and D. Tao, “Bridging composite and real: towards end-to-end deep image matting,” International Journal of Computer Vision, vol. 130, no. 2, pp. 246–266, 2022

    2022

  3. [12]

    Countering language drift via visual grounding,

    J. Lee, K. Cho, and D. Kiela, “Countering language drift via visual grounding,” arXiv preprint arXiv:1909.04499 , 2019

    1909 arXiv

  4. [13]

    Countering language drift with seeded iterated learning,

    Y . Lu, S. Singhal, F. Strub, A. Courville, and O. Pietquin, “Countering language drift with seeded iterated learning,” inInternational Conference on Machine Learning . PMLR, 2020, pp. 6437–6447

    2020

  5. [14]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 3836–3847

    2023

  6. [15]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  7. [16]

    Mat: Mask- aware transformer for large hole image inpainting,

    W. Li, Z. Lin, K. Zhou, L. Qi, Y . Wang, and J. Jia, “Mat: Mask- aware transformer for large hole image inpainting,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 10 758–10 768

    2022

  8. [17]

    Repaint: Inpainting using denoising diffusion probabilistic models,

    A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. Van Gool, “Repaint: Inpainting using denoising diffusion probabilistic models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 11 461–11 471

    2022

  9. [18]

    Imagen editor and editbench: Advancing and evaluating text-guided image inpainting,

    S. Wang, C. Saharia, C. Montgomery, J. Pont-Tuset, S. Noy, S. Pelle- grini, Y . Onoe, S. Laszlo, D. J. Fleet, R. Soricut et al., “Imagen editor and editbench: Advancing and evaluating text-guided image inpainting,” in Proceedings of the IEEE/CVF conference on computer vision a...

    2023

  10. [19]

    Layoutdm: Transformer-based diffusion model for layout generation,

    S. Chai, L. Zhuang, and F. Yan, “Layoutdm: Transformer-based diffusion model for layout generation,” in Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , 2023, pp. 18 349– 18 358

    2023

  11. [20]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

  12. [21]

    Gligen: Open-set grounded text-to-image generation,

    Y . Li, H. Liu, Q. Wu, F. Mu, J. Yang, J. Gao, C. Li, and Y . J. Lee, “Gligen: Open-set grounded text-to-image generation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 511–22 521

    2023

  13. [22]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 . Springer,...

    2014

  14. [23]

    Denoising diffusion probabilistic mod- els,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic mod- els,” in Proceedings of the 34th International Conference on Neural Information Processing Systems , 2020, pp. 6840–6851

    2020

  15. [24]

    Deep unsupervised learning using nonequilibrium thermodynamics,

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning . PMLR, 2015, pp. 2256–2265

    2015

  16. [25]

    Score-based generative modeling through stochastic differ- ential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differ- ential equations,” arXiv preprint arXiv:2011.13456 , 2020

    2011 arXiv

  17. [26]

    Improved denoising diffusion probabilis- tic models,

    A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilis- tic models,” in International Conference on Machine Learning. PMLR, 2021, pp. 8162–8171

    2021

  18. [27]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021

    2021

  19. [28]

    Fastdiff: A fast conditional diffusion model for high-quality speech synthesis,

    R. Huang, M. W. Y . Lam, J. Wang, D. Su, D. Yu, Y . Ren, and Z. Zhao, “Fastdiff: A fast conditional diffusion model for high-quality speech synthesis,” in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23...

    2022

  20. [29]

    Disenbooth: Disentangled parameter-efficient tuning for subject-driven text-to-image generation,

    H. Chen, Y . Zhang, X. Wang, X. Duan, Y . Zhou, and W. Zhu, “Disenbooth: Disentangled parameter-efficient tuning for subject-driven text-to-image generation,” arXiv preprint arXiv:2305.03374 , 2023

    2023 arXiv

  21. [30]

    Instantbooth: Personalized text-to-image generation without test-time finetuning,

    J. Shi, W. Xiong, Z. Lin, and H. J. Jung, “Instantbooth: Personalized text-to-image generation without test-time finetuning,” arXiv preprint arXiv:2304.03411, 2023

    2023 arXiv

  22. [31]

    Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation,

    Y . Wei, Y . Zhang, Z. Ji, J. Bai, L. Zhang, and W. Zuo, “Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation,” arXiv preprint arXiv:2302.13848 , 2023

    2023 arXiv

  23. [32]

    Cones: Concept neurons in diffusion models for customized generation,

    Z. Liu, R. Feng, K. Zhu, Y . Zhang, K. Zheng, Y . Liu, D. Zhao, J. Zhou, and Y . Cao, “Cones: Concept neurons in diffusion models for customized generation,” arXiv preprint arXiv:2303.05125 , 2023

    2023 arXiv

  24. [33]

    p+: Ex- tended textual conditioning in text-to-image generation,

    A. V oynov, Q. Chu, D. Cohen-Or, and K. Aberman, “ p+: Ex- tended textual conditioning in text-to-image generation,” arXiv preprint arXiv:2303.09522, 2023

    2023 arXiv

  25. [34]

    Subject-driven text-to-image generation via apprenticeship learning,

    W. Chen, H. Hu, Y . Li, N. Ruiz, X. Jia, M.-W. Chang, and W. W. Cohen, “Subject-driven text-to-image generation via apprenticeship learning,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  26. [35]

    A neural space- time representation for text-to-image personalization,

    Y . Alaluf, E. Richardson, G. Metzer, and D. Cohen-Or, “A neural space- time representation for text-to-image personalization,” ACM Transac- tions on Graphics (TOG) , vol. 42, no. 6, pp. 1–10, 2023

    2023

  27. [36]

    T2i- adapter: Learning adapters to dig out more controllable ability for text- to-image diffusion models,

    C. Mou, X. Wang, L. Xie, Y . Wu, J. Zhang, Z. Qi, and Y . Shan, “T2i- adapter: Learning adapters to dig out more controllable ability for text- to-image diffusion models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, 2024, pp. 4296–4304

    2024

  28. [37]

    Uni-controlnet: All-in-one control to text-to-image diffusion models,

    S. Zhao, D. Chen, Y .-C. Chen, J. Bao, S. Hao, L. Yuan, and K.-Y . K. Wong, “Uni-controlnet: All-in-one control to text-to-image diffusion models,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  29. [38]

    Layoutvae: Stochastic scene layout generation from a label set,

    A. A. Jyothi, T. Durand, J. He, L. Sigal, and G. Mori, “Layoutvae: Stochastic scene layout generation from a label set,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 9895–9904

    2019

  30. [39]

    Layoutgan: Gen- erating graphic layouts with wireframe discriminators,

    J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “Layoutgan: Gen- erating graphic layouts with wireframe discriminators,” arXiv preprint arXiv:1901.06767, 2019

    1901 arXiv

  31. [40]

    Attribute- conditioned layout gan for automatic graphic design,

    J. Li, J. Yang, J. Zhang, C. Liu, C. Wang, and T. Xu, “Attribute- conditioned layout gan for automatic graphic design,”IEEE Transactions JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 11 on Visualization and Computer Graphics , vol. 27, no. 10, pp. 4039– 4048, 2020

    2021

  32. [41]

    Variational transformer networks for layout generation,

    D. M. Arroyo, J. Postels, and F. Tombari, “Variational transformer networks for layout generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 13 642–13 652

    2021

  33. [42]

    Layouttransformer: Layout generation and completion with self-attention,

    K. Gupta, J. Lazarow, A. Achille, L. S. Davis, V . Mahadevan, and A. Shrivastava, “Layouttransformer: Layout generation and completion with self-attention,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 1004–1014

    2021

  34. [43]

    Layout-bridging text-to-image synthesis,

    J. Liang, W. Pei, and F. Lu, “Layout-bridging text-to-image synthesis,” IEEE Transactions on Circuits and Systems for Video Technology, 2023

    2023

  35. [44]

    Background layout generation and object knowledge transfer for text-to-image generation,

    Z. Chen, Z. Mao, S. Fang, and B. Hu, “Background layout generation and object knowledge transfer for text-to-image generation,” in Proceed- ings of the 30th ACM International Conference on Multimedia , 2022, pp. 4327–4335

    2022

  36. [45]

    Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation,

    L. Qu, S. Wu, H. Fei, L. Nie, and T.-S. Chua, “Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation,” in Proceedings of the 31st ACM International Conference on Multimedia , 2023, pp. 643–654

    2023

  37. [46]

    Llm-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models,

    L. Lian, B. Li, A. Yala, and T. Darrell, “Llm-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models,” arXiv preprint arXiv:2305.13655 , 2023

    2023 arXiv

  38. [47]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  39. [48]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM , vol. 65, no. 1, pp. 99–106, 2021

    2021

  40. [49]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  41. [50]

    On the spectral bias of neural networks,

    N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y . Bengio, and A. Courville, “On the spectral bias of neural networks,” in International Conference on Machine Learning . PMLR, 2019, pp. 5301–5310

    2019

  42. [51]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

  43. [52]

    Clipscore: A reference-free evaluation metric for image captioning,

    J. Hessel, A. Holtzman, M. Forbes, R. Le Bras, and Y . Choi, “Clipscore: A reference-free evaluation metric for image captioning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 7514–7528

    2021

  44. [53]

    Image synthesis from layout with locality-aware mask adaption,

    Z. Li, J. Wu, I. Koh, Y . Tang, and L. Sun, “Image synthesis from layout with locality-aware mask adaption,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 13 819–13 828

    2021

  45. [54]

    Constrained graphic layout generation via latent optimization,

    K. Kikuchi, E. Simo-Serra, M. Otani, and K. Yamaguchi, “Constrained graphic layout generation via latent optimization,” in Proceedings of the 29th ACM International Conference on Multimedia , 2021, pp. 88–96

    2021

  46. [55]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems , vol. 32, 2019

    2019

  47. [56]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  48. [57]

    Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic seg- mentation,

    Q. Nguyen, T. Vu, A. Tran, and K. Nguyen, “Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic seg- mentation,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  49. [58]

    Image synthesis from reconfigurable layout and style,

    W. Sun and T. Wu, “Image synthesis from reconfigurable layout and style,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 10 531–10 540

    2019

  50. [59]

    Learning layout and style reconfigurable gans for controllable image synthesis,

    W. Sun and T. Wu, “Learning layout and style reconfigurable gans for controllable image synthesis,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 9, pp. 5070–5087, 2021

    2021

  51. [60]

    Object- centric image generation from layouts,

    T. Sylvain, P. Zhang, Y . Bengio, R. D. Hjelm, and S. Sharma, “Object- centric image generation from layouts,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 3, 2021, pp. 2647– 2655

    2021

  52. [61]

    Layoutdiffusion: Controllable diffusion model for layout-to-image generation,

    G. Zheng, X. Zhou, X. Li, Z. Qi, Y . Shan, and X. Li, “Layoutdiffusion: Controllable diffusion model for layout-to-image generation,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 490–22 499

    2023

  53. [62]

    Drag your gan: Interactive point-based manipulation on the generative image manifold,

    X. Pan, A. Tewari, T. Leimk ¨uhler, L. Liu, A. Meka, and C. Theobalt, “Drag your gan: Interactive point-based manipulation on the generative image manifold,” in ACM SIGGRAPH 2023 Conference Proceedings , 2023, pp. 1–11

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.