Pith. sign in

REVIEW 3 major objections 8 minor 64 references

Control and Realism: Best of Both Worlds in Layout-to-Image without Training

T0 review · 3 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training-free layout-to-image generation can have both precise control and realistic output; WinWinLay claims to get both by correcting attention bias and latent drift.

desk verdict The paper has a promising recipe, but the non-local prior term as written has a sign inconsistency that undercuts its main mechanism unless the equation is a typo. read the letter →

arxiv 2506.15563 v1 pith:3W53F2CU submitted 2025-06-18 cs.CV

Bonan Li , Yinhan Hu , Songhua Liu , Xinchao Wang This is my paper · ORCID
classification cs.CV
keywords layout-to-imagegenerationtraining-freediffusionmodelscross-attentionguidanceLangevindynamicsnon-localattentionboundingboxeslatent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

WinWinLay is a training-free method for layout-to-image generation that claims to fix two known failure modes of backward guidance in diffusion models: objects bunching into a small part of their bounding box, and images degrading as layout control is strengthened. The paper argues that the standard attention energy function biases optimization toward already-strong attention peaks, and that the usual backpropagation update pushes latents away from the pre-trained distribution. It replaces the energy with a non-local attention prior that spreads attention over the box, and replaces the update with a short Langevin-dynamics step whose gradient strength adapts automatically. On COCO2014 and Flickr30K, the reported numbers improve over prior training-free methods in both detection-based layout accuracy and image-quality metrics.

What carries the argument

The load-bearing object is the pair consisting of the non-local attention energy function and the adaptive Langevin update. The energy (Eq. 13) sums the original attention energy with a KL term $R_{\mathrm{nap}}$ that pulls the normalized attention distribution inside each box toward a centered prior $\tau_u\propto\exp(-\lambda d_u)$, whose strength $\rho$ decays over denoising steps. The update (Eq. 16) runs $O=4$ Langevin steps on the score formed by the pre-trained diffusion model plus $-\nu\nabla E_{\mathrm{naef}}$, with $\nu$ set at each step to $\|\nabla\log p(z_t)\|/\|\nabla E_{\mathrm{naef}}\|$, a ratio that follows from Corollary 4.2's analytic Nash bargaining solution.

What would settle it

Run WinWinLay on COCO2014 with the same hyperparameters but replace the four Langevin steps with one step and with twenty steps, keeping the step size scaled by the same signal-to-noise ratio; if AP and FID do not degrade or improve along the predicted curve, the Langevin claim is not the mechanism. A second check: visualize the cross-attention map of the target token inside its box after the energy update; if the mass still concentrates locally instead of spreading, the non-local prior is not doing the work attributed to it.

Watch

Extended reading notes

Core claim

The central claim is that both halves of the control-quality trade-off can be eliminated at inference time, without retraining the base text-to-image model. The paper proves a model-level statement (Theorem 4.1) that a softmax attention score optimized against a binary mask amplifies the gap between already-large and already-small entries, so energy descent concentrates attention locally; the proposed non-local prior counters this by pushing attention toward a centered distribution inside the box. For the update, it treats layout conditioning as sampling from a Gibbs posterior $p(z_t|m)\propto p(z_t)\exp(-\nu E_{\mathrm{naef}})$, so the correct score adds the pre-trained model's score to the negative energy gradient, and it sets $\nu$ adaptively via the Nash bargaining solution of the two-task gradient combination. With these two pieces, the paper reports, on COCO2014, detection AP of 19.74 versus 17.63 for the strongest baseline and FID of 26.85 versus 27.51, along with higher CLIP-s, PickScore, and ImageReward.

Load-bearing premise

The load-bearing premise is that four short random gradient-guided steps without any accept/reject correction really draw latents from the intended layout-aware distribution, and that the simple softmax analysis in Theorem 4.1 describes what the full pre-trained network's attention actually does.

Editorial extensions

If this is right

  • Pre-trained diffusion models can be steered by bounding boxes without task-specific fine-tuning, outperforming the four prior training-free methods on the evaluated benchmarks.
  • Strengthening layout control no longer has to degrade photorealism, because the update keeps the latent near the pre-trained distribution.
  • Objects generated inside boxes occupy the full region instead of collapsing to a local attention peak, which the non-local prior brings about.
  • No manual search over the guidance strength $\nu$ is needed; the Nash-ratio formula sets it per step, keeping the method efficient.
  • The gains hold across datasets and metrics: COCO2014 AP 19.74 and FID 26.85; Flickr30K AP 17.28 and FID 27.04.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's box-conditioning experiments, the same two corrections should apply to other spatial hints such as scribbles, keypoints, or segmentation masks, because they target generic attention spread and latent drift rather than box-specific code.
  • Because the Langevin step is unadjusted (the Metropolis-Hastings correction is explicitly omitted), the four-step schedule is a heuristic; a strict-MCMC version with acceptance checks would reveal how much of the gain comes from the Gibbs ideal versus the short-cut.
  • The adaptive ratio $\nu=\|\nabla\log p\|/\|\nabla E\|$ can be read as an automatic temperature schedule, so a systematic ablation against fixed values of $\nu$ would isolate how much of the quality gain is due to adaptation alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes WinWinLay, a training-free layout-to-image generation method built on a pre-trained text-to-image diffusion model. Two main contributions are presented: (i) a non-local attention energy function that augments the standard attention energy with a KL-divergence-based prior intended to spread cross-attention over the full bounding box, and (ii) an adaptive Langevin-dynamics update rule designed to balance layout adherence against staying in the pre-trained image manifold. The method is evaluated on COCO2014 and Flickr30K with YOLOv7 AP, CLIP-s, FID, PickScore, ImageReward, and user studies, and it is reported to outperform Layout-Control, AttRe, R&B, and CSG.

Significance. If the claims hold, WinWinLay would be a meaningful advance in training-free layout-to-image generation: it provides a new theoretical perspective on the local concentration bias of attention energy functions (Theorem 4.1), a simple non-local attention prior, and an adaptive update that avoids manual hyperparameter tuning. The experimental setup follows field conventions and covers multiple metrics and a user study. The theoretical derivations of Theorem 4.1 and Corollary 4.2 are correct. However, the internal inconsistency between the sign of the non-local prior in Eq. (13) and the described maximization objective is load-bearing and currently prevents the results from being reproducible or the mechanism from being validated.

major comments (3)
  1. [Section 4.1, Eq. (13) and Fig. 6] The paper defines the non-local attention energy as E_naef = E_aef + ρ R_nap, where R_nap = Σ â log(â/τ), and states in Section 4.2 that gradient updates minimize E_naef. However, Section 4.1 says the method maximizes the KL divergence between the attention distribution and the prior τ, and Fig. 6 shows that increasing ρ aligns objects with box edges. Since τ_u ∝ exp(-λ d_u) is center-peaked, minimizing +ρ R_nap pulls attention toward the box center, which is the opposite of the claimed edge-spreading behavior. Maximizing R_nap (or equivalently using -ρ R_nap in the loss) would push attention to the boundary, not toward the center. This sign inconsistency directly affects the main contribution and the +9.0 AP improvement attributed to the non-local prior in Table 2. The authors must correct either the sign in Eq. (13), the stated optimization direction, or the definition of the prior, and should clarify which objective was actually optimized in the experiments.
  2. [Section 4.2, Eq. (16)] The paper presents the update as Langevin dynamics and states it converges to p(z_t|m) as ξ→0 and O→∞, while acknowledging that the Metropolis-Hastings correction is omitted. The actual implementation uses O=4 steps and a finite step size determined by r=0.06. These finite-O, unadjusted updates are not covered by the stated convergence guarantee, and the paper provides no finite-sample analysis or quantitative evidence that four steps bring the iterate close to the intended guided posterior. Since the adaptive update is the second central contribution, the claim that it "eliminates the trade-off" between control and quality rests on empirical observation alone. Please provide an analysis of the approximation error for finite O, or temper the theoretical claim and add a more thorough sensitivity study of O and r.
  3. [Table 1] No error bars, standard deviations, or significance tests are reported for any of the quantitative metrics. Several improvements over the best baseline are small in magnitude, for example PickScore 21.41 vs. 21.22 and ImageReward 0.7218 vs. 0.7109 on COCO2014, and CLIP-s 0.309 vs. 0.291 on Flickr30K. Without variance estimates it is not possible to assess whether these differences are statistically meaningful. Please report results over multiple seeds or runs, or provide a statistical test, to support the claim that WinWinLay 'consistently outperforms' existing methods across controllability and realism.
minor comments (8)
  1. [Section 4.1, Theorem 4.1] The theorem analyzes gradient ascent on the toy objective f(v)=m·softmax(v), but the actual method backpropagates through the U-Net and updates the latent z. The paper calls the toy objective 'simple yet universal' without an explicit argument for why the ratio-amplification result transfers to the full network's attention energy gradient; please clarify this logical link or soften the claim.
  2. [Section 5.1] The value of λ (the variance-controlling parameter of the prior τ_u) is never specified, and the linear decay schedule for ρ is not precisely defined (e.g., from 5 to 0 over which denoising steps). These details are necessary for reproducibility.
  3. [Section 5.1 (FID reference)] The FID metric is attributed to Kynkäänniemi et al., but the standard FID reference is Heusel et al. (2017); please correct the citation.
  4. [Table 1] There is a typo: 'Flicker30K' should be 'Flickr30K'.
  5. [Section 5.3] There is a typo in the section header: 'Adapative Update' should be 'Adaptive Update'.
  6. [Section 4.2] There is a typo: 'apative update rule' should be 'adaptive update rule'.
  7. [Section 2.2] The sentence 'collecting such extensive labeled images is not non-trivial' uses a double negative; it should read 'is non-trivial' or 'is not trivial'.
  8. [Figure 7] The axes and the meaning of the plotted points are unclear; the caption mentions 'Coefficient of υ' while the text uses ν, and it is not obvious what AP and Cs values correspond to which hyperparameter settings. Please clarify the figure.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the derivation uses standard external sampling results and is evaluated against external benchmarks; the Eq. 13 sign inconsistency is a correctness issue, not a circular reduction.

full rationale

The paper's derivation chain is not circular. The non-local attention prior is an ansatz constructed from bounding-box geometry; the claimed bias-amplification of the attention energy function is a self-contained softmax-gradient theorem (Theorem 4.1) about the toy objective f(v)=m·softmax(v), and the transfer to the full network is an unproven assumption rather than a circular reduction. The adaptive update is obtained from Bayes' rule (Eq. 14), score decomposition (Eq. 15), standard Langevin dynamics (Eq. 16, citing Song & Ermon 2019 and Song et al. 2021b), and a Nash-MTL corollary (Corollary 4.2) whose adaptive weight nu = ||∇log p(zt)|| / ||∇E_naef(zt,m)|| is computed from the current gradients, not fitted to a subset of data and then reported as a prediction. Evaluation is against external benchmarks (COCO2014, Flickr30K, user study) and external baselines, not against the optimized loss itself. The only possibly same-author citation (Liu et al. 2024b, Hico) appears in a background paragraph and is not load-bearing. There is an internal sign inconsistency: Eq. 13 adds +rho·KL to an energy that Section 4.2 says is minimized, while the text says the prior is applied by 'maximizing the KL divergence'; this is a correctness/reproducibility concern about what was actually run, but it is a contradiction between the printed equation and the prose, not an equivalence of a prediction to its input. Therefore no significant circularity is present.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on modeling choices: the prior over object position in the box, the Gibbs form of the layout constraint, the use of the frozen model's score, and the finite-step Langevin approximation. Most are domain assumptions inherited from the backward-guidance literature or ad hoc constructions introduced for this paper; none are externally validated beyond the benchmark results.

free parameters (5)
  • rho (non-local prior weight) = 5 (max), decayed linearly to 0 over denoising steps
    Controls trade-off between box coverage and natural object structure; tuned on the evaluation benchmarks (Section 5.1, 5.3).
  • lambda (prior variance) = not specified
    Shape parameter of the center-weighted prior tau_u proportional to exp(-lambda d_u) in Eq. (12); no value given in the paper.
  • r (signal-to-noise ratio for Langevin step size) = 0.06
    Set via grid search in Figure 7; determines step size xi in the adaptive update (Section 4.2).
  • O (Langevin update steps) = 4
    Number of inner update iterations per denoising step; chosen as a balance of efficiency and quality (Section 4.2, 5.1).
  • layout-constrained denoising steps = 10 of 50 DDIM steps
    Layout constraint applied only in the first 10 denoising steps; a schedule hyperparameter set by the authors (Section 5.1).
assumptions (6)
  • domain assumption Cross-attention maps in the frozen T2I model encode subject location, and steering them toward layout boxes transfers to pixel-space placement.
    Core premise inherited from backward guidance (Chen et al., 2024d); used to justify optimizing E_naef. Sections 3.2 and 4.1.
  • ad hoc to paper The conditional distribution p(m|z_t) proportional to exp(-nu E_naef(z_t,m)) is a valid Gibbs measure for layout adherence.
    Introduced directly in Section 4.2 to enable the Bayes/Langevin derivation; no independent justification.
  • domain assumption The pretrained diffusion model's score, gradient of log p(z_t) with respect to z_t, is well approximated during the DDIM sampling trajectory.
    Standard score-based diffusion assumption; used in Eq. (15) to combine with the layout gradient.
  • ad hoc to paper The simplified softmax objective in Eq. (6) faithfully models the behavior of the full attention energy function under backpropagation through the U-Net.
    Theorem 4.1 is proven for the toy objective; the paper asserts it is 'simple yet universal' (Section 4.1) without a formal bridge to the actual network.
  • ad hoc to paper Finite O=4 unadjusted Langevin steps (no Metropolis-Hastings) still produce useful samples for the guided posterior.
    The convergence guarantee requires xi to go to 0 and O to go to infinity; the paper omits the MH correction 'for convenience in practice' (Section 4.2).
  • ad hoc to paper The center-weighted prior tau_u proportional to exp(-lambda d_u) is a suitable target for natural object placement inside a box.
    Designed to reward central placement with full coverage (Eq. 12); its mismatch with actual object shape is mitigated by a decaying rho, but the prior itself is not derived from data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Control and Realism: Best of Both Worlds in Layout-to-Image without Training." pith.science (2026). https://pith.science/paper/3W53F2CU

@misc{pith2026250615563,
  author       = {Pith},
  title        = {Pith review of: Control and Realism: Best of Both Worlds in Layout-to-Image without Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3W53F2CU}},
  note         = {Machine review of arXiv:2506.15563}
}
read the original abstract

Layout-to-Image generation aims to create complex scenes with precise control over the placement and arrangement of subjects. Existing works have demonstrated that pre-trained Text-to-Image diffusion models can achieve this goal without training on any specific data; however, they often face challenges with imprecise localization and unrealistic artifacts. Focusing on these drawbacks, we propose a novel training-free method, WinWinLay. At its core, WinWinLay presents two key strategies, Non-local Attention Energy Function and Adaptive Update, that collaboratively enhance control precision and realism. On one hand, we theoretically demonstrate that the commonly used attention energy function introduces inherent spatial distribution biases, hindering objects from being uniformly aligned with layout instructions. To overcome this issue, non-local attention prior is explored to redistribute attention scores, facilitating objects to better conform to the specified spatial conditions. On the other hand, we identify that the vanilla backpropagation update rule can cause deviations from the pre-trained domain, leading to out-of-distribution artifacts. We accordingly introduce a Langevin dynamics-based adaptive update scheme as a remedy that promotes in-domain updating while respecting layout constraints. Extensive experiments demonstrate that WinWinLay excels in controlling element placement and achieving photorealistic visual fidelity, outperforming the current state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2506.15563 by the authors.

Figure 1
Figure 1. Given user-provided bounding boxes and prompts of subjects, our [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visualization of cross-attention between text to [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Qualitative comparison of our WinWinLay and state-of-the-art methods. Zoom in for more details. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (2 more)
Figure 6
Figure 6. Figure 6: Ablation on hyperparameter of non-local attention [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Ablation on hyperparameter of Adaptive Update [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 51 canonical work pages

  1. [1]

    Dreamstyler: Paint by style inversion with text-to-image diffusion models

    Ahn, N., Lee, J., Lee, C., Kim, K., Kim, D., Nam, S.-H., and Hong, K. Dreamstyler: Paint by style inversion with text-to-image diffusion models. In AAAI, pp.\ 674--681, 2024

    work page 2024

  2. [2]

    Multidiffusion: Fusing diffusion paths for controlled image generation

    Bar-Tal, O., Yariv, L., Lipman, Y., and Dekel, T. Multidiffusion: Fusing diffusion paths for controlled image generation. In ICML, pp.\ 1737--1752, 2023

    work page 2023

  3. [3]

    Brooks, T., Holynski, A., and Efros, A. A. Instructpix2pix: Learning to follow image editing instructions. In CVPR, pp.\ 18392--18402, 2023

    work page 2023

  4. [4]

    Artadapter: Text-to-image style transfer using multi-level style encoder and explicit adaptation

    Chen, D.-Y., Tennent, H., and Hsu, C.-W. Artadapter: Text-to-image style transfer using multi-level style encoder and explicit adaptation. In CVPR, pp.\ 8619--8628, 2024 a

    work page 2024

  5. [5]

    Boundary attention constrained zero-shot layout-to-image generation

    Chen, H., Li, J., Zhuang, W., Vikalo, H., and Lyu, L. Boundary attention constrained zero-shot layout-to-image generation. arXiv preprint arXiv:2411.10495, 2024 b

    work page arXiv 2024

  6. [6]

    Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Chen, J., Jincheng, Y., Chongjian, G., Yao, L., Xie, E., Wang, Z., Kwok, J., Luo, P., Lu, H., and Li, Z. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, 2024 c

    work page 2024

  7. [7]

    Training-free layout control with cross-attention guidance

    Chen, M., Laina, I., and Vedaldi, A. Training-free layout control with cross-attention guidance. In WACV, pp.\ 5343--5353, 2024 d

    work page 2024

  8. [8]

    Vp3d: Unleashing 2d visual prompt for text-to-3d generation

    Chen, Y., Pan, Y., Yang, H., Yao, T., and Mei, T. Vp3d: Unleashing 2d visual prompt for text-to-3d generation. In CVPR, pp.\ 4896--4905, 2024 e

    work page 2024

Show all 64 references
  1. [9]

    Zero-shot spatial layout conditioning for text-to-image diffusion models

    Couairon, G., Careil, M., Cord, M., Lathuiliere, S., and Verbeek, J. Zero-shot spatial layout conditioning for text-to-image diffusion models. In ICCV, pp.\ 2174--2183, 2023

    2023

  2. [10]

    and Nichol, A

    Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. In NeurIPS, pp.\ 8780--8794, 2021

    2021

  3. [11]

    J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A

    Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. C., and Bengio, Y. Generative adversarial nets. In NeurIPS, 2014

    2014

  4. [12]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. In NeurIPS, pp.\ 6840--6851, 2020

    2020

  5. [13]

    J., Norouzi, M., and Salimans, T

    Ho, J., Saharia, C., Chan, W., Fleet, D. J., Norouzi, M., and Salimans, T. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23 0 (47): 0 1--33, 2022

    2022

  6. [14]

    Ssmg: Spatial-semantic map guided diffusion model for free-form layout-to-image generation

    Jia, C., Luo, M., Dang, Z., Dai, G., Chang, X., Wang, M., and Wang, J. Ssmg: Spatial-semantic map guided diffusion model for free-form layout-to-image generation. In AAAI, pp.\ 2480--2488, 2024

    2024

  7. [15]

    C., and Liu, Z

    Jiang, Y., Wu, T., Yang, S., Si, C., Lin, D., Qiao, Y., Loy, C. C., and Liu, Z. Videobooth: Diffusion-based video generation with image prompts. In CVPR, pp.\ 6689--6700, 2024

    2024

  8. [16]

    Imagic: Text-based real image editing with diffusion models

    Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., and Irani, M. Imagic: Text-based real image editing with diffusion models. In CVPR, pp.\ 6007--6017, 2023

    2023

  9. [17]

    Dense text-to-image generation with attention modulation

    Kim, Y., Lee, J., Kim, J.-H., Ha, J.-W., and Zhu, J.-Y. Dense text-to-image generation with attention modulation. In ICCV, pp.\ 7701--7711, 2023

    2023

  10. [18]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Kirstain, Y., Polyak, A., Singer, U., Matiana, S., Penna, J., and Levy, O. Pick-a-pic: An open dataset of user preferences for text-to-image generation. In NeurIPS, pp.\ 36652--36663, 2023

    2023

  11. [19]

    W., Zhou, Y., Liu, D., Lee, J.-Y., Cai, H., Liu, B., Liu, F., and Uh, Y

    Kwon, M., Oh, S. W., Zhou, Y., Liu, D., Lee, J.-Y., Cai, H., Liu, B., Liu, F., and Uh, Y. Harivo: Harnessing text-to-image models for video generation. In ECCV, pp.\ 19--36, 2025

    2025

  12. [20]

    The role of imagenet classes in fr \'e chet inception distance

    Kynk \"a \"a nniemi, T., Karras, T., Aittala, M., Aila, T., and Lehtinen, J. The role of imagenet classes in fr \'e chet inception distance. In ICLR, 2023

    2023

  13. [21]

    Instant3d: Instant text-to-3d generation

    Li, M., Zhou, P., Liu, J.-W., Keppo, J., Lin, M., Yan, S., and Xu, X. Instant3d: Instant text-to-3d generation. International Journal of Computer Vision, pp.\ 1--17, 2024

    2024

  14. [22]

    Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., Li, C., and Lee, Y. J. Gligen: Open-set grounded text-to-image generation. In CVPR, pp.\ 22511--22521, 2023

    2023

  15. [23]

    Image synthesis from layout with locality-aware mask adaption

    Li, Z., Wu, J., Koh, I., Tang, Y., and Sun, L. Image synthesis from layout with locality-aware mask adaption. In ICCV, pp.\ 13819--13828, 2021

    2021

  16. [24]

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In ECCV, pp.\ 740--755, 2014

    2014

  17. [25]

    Training-free composite scene generation for layout-to-image synthesis

    Liu, J., Huang, T., and Xu, C. Training-free composite scene generation for layout-to-image synthesis. In ECCV, pp.\ 37--53, 2024 a

    2024

  18. [26]

    Hico: Hierarchical controllable diffusion model for layout-to-image generation

    Liu, S., Ma, A., Wu, X., Leng, D., Yin, Y., et al. Hico: Hierarchical controllable diffusion model for layout-to-image generation. In NeurIPS, 2024 b

    2024

  19. [27]

    Null-text inversion for editing real images using guided diffusion models

    Mokady, R., Hertz, A., Aberman, K., Pritch, Y., and Cohen-Or, D. Null-text inversion for editing real images using guided diffusion models. In CVPR, pp.\ 6038--6047, 2023

    2023

  20. [28]

    Multi-task learning as a bargaining game

    Navon, A., Shamsian, A., Achituve, I., Maron, H., Kawaguchi, K., Chechik, G., and Fetaya, E. Multi-task learning as a bargaining game. ArXiv, abs/2202.01017, 2022

    2022 arXiv

  21. [29]

    Zero-shot image-to-image translation

    Parmar, G., Kumar Singh, K., Zhang, R., Li, Y., Lu, J., and Zhu, J.-Y. Zero-shot image-to-image translation. In SIGGRAPH, pp.\ 1--11, 2023

    2023

  22. [30]

    and Xie, S

    Peebles, W. and Xie, S. Scalable diffusion models with transformers. In ICCV, pp.\ 4195--4205, 2023

    2023

  23. [31]

    Grounded text-to-image synthesis with attention refocusing

    Phung, Q., Ge, S., and Huang, J.-B. Grounded text-to-image synthesis with attention refocusing. In CVPR, pp.\ 7932--7942, 2024

    2024

  24. [32]

    A., Wang, L., Cervantes, C

    Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., and Lazebnik, S. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In ICCV, pp.\ 2641--2649, 2015

    2015

  25. [33]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., M \"u ller, J., Penna, J., and Rombach, R. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  26. [34]

    Hierarchical spatio-temporal decoupling for text-to-video generation

    Qing, Z., Zhang, S., Wang, J., Wang, X., Wei, Y., Zhang, Y., Gao, C., and Sang, N. Hierarchical spatio-temporal decoupling for text-to-video generation. In CVPR, pp.\ 6635--6645, 2024

    2024

  27. [35]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In ICML, pp.\ 8748--8763, 2021

    2021

  28. [36]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, pp.\ 10684--10695, 2022

    2022

  29. [37]

    U-net: Convolutional networks for biomedical image segmentation

    Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pp.\ 234--241, 2015

    2015

  30. [38]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pp.\ 22500--22510, 2023

    2023

  31. [39]

    L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al

    Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al. Photorealistic text-to-image diffusion models with deep language understanding. In NeurIPS, pp.\ 36479--36494, 2022

    2022

  32. [40]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, S., Crowson, K., Schmidt, L., Kaczmarczyk, R., and Jitsev, J. Laion-5b: An open large-scale dataset for training next gen...

    arXiv

  33. [41]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models. In NeurIPS, pp.\ 25278--25294, 2022 b

    2022

  34. [42]

    Shi, J., Xiong, W., Lin, Z., and Jung, H. J. Instantbooth: Personalized text-to-image generation without test-time finetuning. In CVPR, pp.\ 8543--8552, 2024

    2024

  35. [43]

    High-fidelity guided image synthesis with latent diffusion models

    Singh, J., Gould, S., and Zheng, L. High-fidelity guided image synthesis with latent diffusion models. In CVPR, pp.\ 5997--6006, 2023

    2023

  36. [44]

    Styledrop: Text-to-image synthesis of any style

    Sohn, K., Jiang, L., Barber, J., Lee, K., Ruiz, N., Krishnan, D., Chang, H., Li, Y., Essa, I., Rubinstein, M., et al. Styledrop: Text-to-image synthesis of any style. In NeurIPS, 2024

    2024

  37. [45]

    Denoising diffusion implicit models

    Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In ICLR, 2021 a

    2021

  38. [46]

    and Ermon, S

    Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In NeurIPS, 2019

    2019

  39. [47]

    P., Kumar, A., Ermon, S., and Poole, B

    Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In ICLR, 2021 b

    2021

  40. [48]

    Z., and Poggi, M

    Tosi, F., Ramirez, P. Z., and Poggi, M. Diffusion models for monocular depth estimation: Overcoming challenging conditions. In ECCV, pp.\ 236--257, 2025

    2025

  41. [49]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Tumanyan, N., Geyer, M., Bagon, S., and Dekel, T. Plug-and-play diffusion features for text-driven image-to-image translation. In CVPR, pp.\ 1921--1930, 2023

    1921

  42. [50]

    Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y. M. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In CVPR, pp.\ 7464--7475, 2023

    2023

  43. [51]

    S., Girdhar, R., and Misra, I

    Wang, X., Darrell, T., Rambhatla, S. S., Girdhar, R., and Misra, I. Instancediffusion: Instance-level control for image generation. In CVPR, pp.\ 6232--6242, 2024 a

    2024

  44. [52]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation

    Wang, Z., Lu, C., Wang, Y., Bao, F., Li, C., Su, H., and Zhu, J. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. In NeurIPS, 2024 b

    2024

  45. [53]

    Z., Ge, Y., Wang, X., Lei, S

    Wu, J. Z., Ge, Y., Wang, X., Lei, S. W., Gu, Y., Shi, Y., Hsu, W., Shan, Y., Qie, X., and Shou, M. Z. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, pp.\ 7623--7633, 2023

    2023

  46. [54]

    Ifadapter: Instance feature control for grounded text-to-image generation

    Wu, Y., Zhou, X., Ma, B., Su, X., Ma, K., and Wang, X. Ifadapter: Instance feature control for grounded text-to-image generation. arXiv preprint arXiv:2409.08240, 2024

    2024 arXiv

  47. [55]

    R&b: Region and boundary aware zero-shot grounded text-to-image generation

    Xiao, J., Lv, H., Li, L., Wang, S., and Huang, Q. R&b: Region and boundary aware zero-shot grounded text-to-image generation. In ICLR, 2024

    2024

  48. [56]

    Xie, J., Li, Y., Huang, Y., Liu, H., Zhang, W., Zheng, Y., and Shou, M. Z. Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion. In ICCV, pp.\ 7452--7461, 2023

    2023

  49. [57]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., and Dong, Y. Imagereward: Learning and evaluating human preferences for text-to-image generation. In NeurIPS, pp.\ 15903--15935, 2023

    2023

  50. [58]

    Inversion-free image editing with language-guided diffusion models

    Xu, S., Huang, Y., Pan, J., Ma, Z., and Chai, J. Inversion-free image editing with language-guided diffusion models. In CVPR, pp.\ 9452--9461, 2024

    2024

  51. [59]

    Freestyle layout-to-image synthesis

    Xue, H., Huang, Z., Sun, Q., Song, L., and Zhang, W. Freestyle layout-to-image synthesis. In CVPR, pp.\ 14256--14266, 2023

    2023

  52. [60]

    Reco: Region-controlled text-to-image generation

    Yang, Z., Wang, J., Gan, Z., Li, L., Lin, K., Wu, C., Duan, N., Liu, Z., Liu, C., Zeng, M., et al. Reco: Region-controlled text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 14246--14255, 2023

    2023

  53. [61]

    Towards consistent video editing with text-to-image diffusion models

    Zhang, Z., Li, B., Nie, X., Han, C., Guo, T., and Liu, L. Towards consistent video editing with text-to-image diffusion models. In NeurIPS, pp.\ 58508--58519, 2023

    2023

  54. [62]

    Layoutdiffusion: Controllable diffusion model for layout-to-image generation

    Zheng, G., Zhou, X., Li, X., Qi, Z., Shan, Y., and Li, X. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. In CVPR, pp.\ 22490--22499, 2023

    2023

  55. [63]

    Migc: Multi-instance generation controller for text-to-image synthesis

    Zhou, D., Li, Y., Ma, F., Zhang, X., and Yang, Y. Migc: Multi-instance generation controller for text-to-image synthesis. In CVPR, pp.\ 6818--6828, 2024

    2024

  56. [64]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.