Pith. sign in

REVIEW 3 major objections 6 minor 69 references

Controllable Coupled Image Generation via Diffusion Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A time-varying attention weight keeps backgrounds identical across generated images while each prompt's subject varies.

desk verdict The method is a plausible starting point for coupled image generation, but the empirical claims don't survive contact with the fact that the evaluation prompts are the same ones used to optimize the method's parameters. read the letter →

arxiv 2506.06826 v1 pith:Q7FEABEU submitted 2025-06-07 cs.CV cs.AI

Chenfei Yuan , Nanshan Jia , Hangqi Li , Peter W. Glynn , Zeyu Zheng This is my paper
classification cs.CVcs.AI
keywords coupledimagegenerationdiffusionmodelscross-attentioncontrolbackgroundsimilaritytext-imagealignmentisotonicoptimizationpromptdisentanglementFlux
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that multiple images can be generated in one diffusion pass so that they share a nearly identical background while each foreground subject follows its own text prompt. The authors call this coupled image generation and argue it matters for video frame interpolation, multi-view 3D reconstruction, and building paired datasets for image editing. Their method splits every prompt into a shared background prompt and a per-image entity prompt using a large language model, then modifies the cross-attention layers of a pre-trained text-to-image model to blend these two text sources with a weight that changes at each denoising step. The weights are chosen by an isotonic optimization—one that enforces an ascending order across denoising steps—maximizing a combined score of background similarity and text alignment. Experiments on a Flux base model report higher background similarity and text-image alignment than random-seed, prompt-to-prompt, and inversion-editing baselines.

What carries the argument

The central mechanism is a parameterized cross-attention control: after an LLM splits each prompt into a shared background prompt and a per-prompt entity prompt, the model's cross-attention modules accept both text embeddings and blend them with a step-dependent weight $\theta_i \in [0,1]$, one weight per denoising step. In QKV-level concatenation the background key is scaled by $(1-\theta_i)$ and the entity key by $\theta_i$; in embedding-level concatenation the image hidden state is interpolated as $\theta_i$ times the entity branch plus $(1-\theta_i)$ times the background branch. The schedule $\theta_1 \le \cdots \le \theta_N$ is learned by an isotonic optimization that maximizes a combined metric of background similarity (Euclidean distance over a Segment Anything foreground mask) and text-image alignment (CLIPScore).

What would settle it

Run the training of $\theta$ on a set of prompt pairs disjoint from the evaluation pairs, then compute background similarity and CLIPScore on held-out prompt pairs against the same baselines; if the held-out scores do not beat random-seed and prompt-to-prompt baselines, the central claim of generalizable improvement collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that the entanglement of background and entity in a text-to-image model's cross-attention is the reason separately generated images drift apart in background, and that this entanglement can be corrected by explicit prompt decomposition plus a time-varying scalar weight inside the attention computation. Concretely, the authors concatenate background keys scaled by $(1-\theta)$ and entity keys scaled by $\theta$ (in QKV-level cross-attention) or interpolate the image hidden states from separate background and entity branches (in embedding-level cross-attention), with $\theta$ increasing from near 0 to near 1 over the 50 denoising steps. They state that this schedule makes early steps commit to shared background structure and later steps refine prompt-specific entities. The reported result is that this attention-level control yields the best combined background-similarity and text-alignment scores among the compared methods while adding no sampling-time cost beyond optimizing the small $\theta$ vector.

Load-bearing premise

The paper tunes the weight schedule by maximizing the same background-similarity-plus-alignment score on the very prompt pairs it later uses for the comparison tables, so the claimed advantage assumes that tuning on the test prompts gives a fair picture of performance on new prompts.

Editorial extensions

If this is right

  • The method adds no extra sampling steps beyond the base model's normal denoising, since only the small weight vector is optimized.
  • Any set of prompts that share a describable background can be coupled, whether the entities are entirely different objects or the same object in different poses.
  • The generated pairs can serve as training data for image-editing models, conditioning frames for video generation, and multi-view inputs for 3D reconstruction.
  • The proposed combined metric gives a quantitative yardstick for background similarity and text-image alignment that other coupled-generation methods can be measured against.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A held-out evaluation would test whether the learned weight schedule generalizes, because the reported tables optimize $\theta$ on the same prompt pairs they later score; the paper does not report such a split.
  • The ascending $\theta$ schedule resembles a coarse-to-fine curriculum, so a fixed schedule trained on one prompt family might transfer across unrelated prompt families, which would remove the need to re-optimize per prompt pair.
  • The background-similarity metric masks out the union of entity regions, so it may not capture global consistency of lighting, shadows, and texture; a perceptual or feature-space distance could rank methods differently.
  • For video generation, the same time-varying weights could be shared across frames to keep backgrounds stable rather than re-optimized per frame, potentially simplifying first-last-frame conditioning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a method for coupled image generation, where multiple images are generated simultaneously with shared backgrounds but prompt-specific foreground entities. The method uses an LLM to decompose each prompt into a shared background prompt and per-prompt entity prompts, then modifies the cross-attention modules of a text-to-image diffusion model (Flux.1-dev) to take both prompt types with a time-varying weight parameter θ_i per denoising step. The θ sequence is fit by solving an isotonic optimization problem (Eq. 12) that maximizes a combined objective fc (Eq. 3) consisting of a background-similarity term (based on SAM segmentation and L2 distance on masked regions) and a text-alignment term (CLIPScore). The paper reports quantitative comparisons on three prompt-pair examples (Figures 3-5, Tables 1-3) against random-seed baseline, Prompt-to-Prompt, and RF-inversion, plus ablations on parameterized θ functions and centers.

Significance. The task of coupled image generation is practically relevant, and the idea of separating background and entity prompts and controlling their relative influence through time-varying cross-attention weights is a plausible and interesting approach. The isotonic constraint to reflect the coarse-to-fine structure of diffusion sampling is also a reasonable inductive bias. However, the paper's central empirical claim—that the method outperforms existing approaches—is not supported by the evaluation as presented. The reported metrics are computed on the same prompt pairs used to fit θ, baselines receive no similar per-prompt fitting, the numbers do not reproduce from Eq. (3) with the stated hyperparameters, and there are only three examples with no variance reported. These issues undermine the validity of the quantitative conclusions.

major comments (3)
  1. [Section 4.3, Eq. (12); Section 5.2, Tables 1-3] The evaluation is in-sample: the θ sequence is optimized by maximizing the combined metric fc (Eq. 12) on training sets {T_{k,1},...,T_{k,n}}, and Figure 6 explicitly shows the trained θ values used for the three examples in Figures 3-5. Tables 1-3 then report fc values on those exact same prompt pairs. Consequently, the reported 'Ours' metric values are fitted values, not independent predictions. Baselines (Random seed, P2P, RF-inversion) do not receive an analogous per-prompt optimization of a tunable parameter against fc. This does not establish that the method generalizes to new prompt pairs; it only shows that the optimizer can achieve a high objective on the training instances. The authors should evaluate on held-out prompt sets, or perform cross-validation, and should give baselines the same optimization budget (where applicable) to make a fair comparison.
  2. [Equation (3) vs. Tables 1-3] The combined metric values reported in Tables 1-3 are not consistent with Eq. (3) under the stated hyperparameters λ_bg=300 and λ_ti=1/30. For Table 1, 'Ours', using the reported background similarity fbg = -2.080×10^-4 and text-image alignment 22.61 as the average per-prompt CLIPScore (n=2), Eq. (3) gives fc = 300×(-2.080×10^-4) + (1/30)×22.61 = -0.0624 + 0.7537 = 0.6913, not the reported 1.558. Similar discrepancies appear for all rows in Tables 1-3 (e.g., Random seed in Table 1 yields -1.3976 instead of -0.919). The authors must either clarify the computation (e.g., different normalization, different interpretation of the reported text-alignment number) or correct the tables and equation; as written, the numerical results are not reproducible.
  3. [Section 5.2] The experimental comparison is based on only three prompt pairs, with no repetition across random seeds or multiple runs and no variance or significance estimates. Each table reports a single image pair per method, and the visual examples in Figures 3-5 are single instances. Given the inherent stochasticity of diffusion generation even with fixed seeds (as the authors themselves note in Figure 1), the claim that the method 'outperforms existing approaches across these criteria' requires a larger evaluation set and at least mean ± std over repeated draws. Without this, the quantitative advantage is not established.
minor comments (6)
  1. [Section 4, first paragraph] The word 'genrated' is a typo; it should be 'generated'.
  2. [Section 5.1.1] The phrase 'isotomic constraint' is a typo; it should be 'isotonic constraint'.
  3. [Figure 1 caption] The caption contains a grammatical error: 'The first row illustrates that by given prompts' should likely be 'The first row illustrates that, given prompts...'.
  4. [Section 4.1 and Figure 2] The paper states that 'a pre-trained LLM' is used for prompt disentanglement but does not specify which LLM, its size, or the exact prompt template. This information is needed for reproducibility.
  5. [Section 5.1.1] The base model is referred to as 'Flux.1dev' here and 'Flux Labs' elsewhere; please use a consistent name and provide the exact version used.
  6. [Appendix B.2, Figure 6] The figure shows three θ sequences, but the x-axis label and legend are not described in the caption; please clarify what the axes represent.

Circularity Check

1 steps flagged · score 8.0 of 10

Reported gains are in-sample: the same prompt pairs used to optimize the time-varying θ (Eq. 12) are the pairs evaluated in Tables 1–3, so the comparison does not establish generalizable outperformance.

  1. fitted input called prediction [Section 4.3 (Eq. 12); Section 5.2; Tables 1–3; Figure 6]
    "we formulate the problem of learning θ as an isotonic optimization problem: max_θ (1/M) Σ_{k=1}^M f_c(T_{k,1},...,T_{k,n},θ), subject to θ_1 ≤ θ_2 ≤ ... ≤ θ_N, θ∈Θ. ... Visualization of the trained time-varying parameters used in the three examples. ... Combined metric (↑) 1.558 -0.919 1.246 -0.214"

    Section 4.3 defines θ as the maximizer of the combined objective f_c over training prompt sets. Figure 6 identifies the resulting θ as the 'trained time-varying parameters used in the three examples,' and Tables 1–3 report f_c for exactly those three prompt pairs (Figures 3–5). Therefore the reported 'Ours' combined metric is the value of the training objective at its optimizer on the evaluation prompts; it is not a held-out prediction. The baselines receive no analogous per-prompt fitted parameter, so the comparison reports an in-sample optimum against unoptimized baselines. No held-out prompt pairs or cross-validation are presented to show that the fitted θ sequence generalizes to new prompt pairs.

full rationale

The paper's attention-control mechanism is a genuine architectural intervention, and the LLM-based background/entity disentanglement could in principle be evaluated visually. However, the quantitative claim of outperformance rests on an in-sample fit: Section 4.3 (Eq. 12) defines θ as the maximizer of the combined objective f_c over training prompt sets, and Figure 6 presents the trained θ used for the three examples whose metrics appear in Tables 1–3. Thus the reported combined scores for 'Ours' are the optimized objective values on the evaluation prompts, not predictions for unseen prompt pairs. This is the pattern of fitting a parameter to the evaluation data and then reporting the fit as a result. The absence of held-out prompt sets makes the comparison to baselines (which receive no such fitted parameter) unable to support the claimed generalizable outperformance. A secondary internal inconsistency compounds the issue: using the stated λ_bg=300, λ_ti=1/30 and Table 1's background similarity (-2.080×10^-4) and text-alignment (22.61) in Eq. (3) yields about 0.69, not the reported 1.558; similar mismatches appear in Tables 2–3, making the reported combined metric difficult to interpret. There are no load-bearing self-citations or imported uniqueness theorems, so the circularity is confined to the evaluation protocol rather than to the method's mathematical derivation.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central method introduces a 50-dimensional θ per prompt set, directly fitted to the reported evaluation metric, plus hand-chosen loss weights. No new physical or architectural entities are invented; the method reuses existing attention modules with a new interpolation scheme.

free parameters (3)
  • Time-varying cross-attention weights θ_i (i=1..N=50) = Per prompt set, e.g., Figure 6 shows sequences starting near 0 and rising to 1
    Optimized by Eq. (12) to maximize the combined metric on the same prompt pairs later used for evaluation; this is the core free parameter of the method.
  • Combined objective weights λ_bg and λ_ti = λ_bg=300, λ_ti=1/30
    Chosen by hand 'for proper normalization' in Section 5.1.2; the reported combined metrics do not match these weights with Eq. (3).
  • Ablation function center c and scale k = e.g., c=6.7, k=0.5 for arctan; c=10, k=0.8 for sin
    In supplemental ablations B.3, the center and scale are set manually per example; these are not part of the central method but illustrate how the θ schedule can be parameterized.
assumptions (5)
  • domain assumption A pretrained LLM reliably decomposes any prompt set into a shared background prompt and per-image entity prompts without losing meaning
    Section 4.1, Figure 2: the entire method depends on this decomposition being accurate enough for the cross-attention control to work.
  • ad hoc to paper Scaling keys by (1-θ) and θ and normalizing by sqrt(d_text + d_img) preserves attention score scale, and at θ=1 the modified attention equals the original entity-only attention
    Section 4.2, Eq. (5) and following paragraph: this assertion is used to claim compatibility with the base model, but it is false as written because Qbg remains in the query matrix.
  • domain assumption The f_bg metric computed with SAM segmentation and masked L2 distance is a valid measure of background coupling
    Section 5.1.2, Eq. (15): the metric can be unstable when the union entity mask is large, and it is a proxy for perceptual background similarity.
  • domain assumption Diffusion models first generate coarse structure then refine details, motivating the ascending θ constraint
    Section 4.3 cites Choi et al. 2022, Park et al. 2023, Yue et al. 2024, Wang et al. 2024a; this background assumption is imported without verification on Flux.
  • domain assumption CLIPScore is an adequate measure of text-image alignment for the optimization objective
    Section 5.1.2 adopts CLIPScore as f_ti; no human evaluation is reported, so the objective may not capture perceptual quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Controllable Coupled Image Generation via Diffusion Models." pith.science (2026). https://pith.science/paper/Q7FEABEU

@misc{pith2026250606826,
  author       = {Pith},
  title        = {Pith review of: Controllable Coupled Image Generation via Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q7FEABEU}},
  note         = {Machine review of arXiv:2506.06826}
}
read the original abstract

We provide an attention-level control method for the task of coupled image generation, where "coupled" means that multiple simultaneously generated images are expected to have the same or very similar backgrounds. While backgrounds coupled, the centered objects in the generated images are still expected to enjoy the flexibility raised from different text prompts. The proposed method disentangles the background and entity components in the model's cross-attention modules, attached with a sequence of time-varying weight control parameters depending on the time step of sampling. We optimize this sequence of weight control parameters with a combined objective that assesses how coupled the backgrounds are as well as text-to-image alignment and overall visual quality. Empirical results demonstrate that our method outperforms existing approaches across these criteria.

Figures

Figures reproduced from arXiv: 2506.06826 by the authors.

Figure 1
Figure 1. The first row illustrates that by given prompts that have almost the same description and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A zero-shot example of prompting a pre-trained LLM to extract the shared background [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Comparison of our method to other baselines. Generated pictures in the first row correspond [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison of our method to other baselines. Generated pictures in the first row correspond [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Comparison of our method to other baselines. Generated pictures in the first row correspond [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Visualization of the trained time-varying parameters used in the three examples. [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Comparison of different parameterized functions. The prompts are [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Comparison of different parameter centers. (sin) (scale=0.8) [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Comparison of different parameter centers. (arctan) (scale=0.5) [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Comparison of different parameter centers. (01) [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: A zero-shot example of prompting a pre-trained LLM to extract the shared background [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 40 canonical work pages

  1. [1]

    Wan 2.1 flf2v: First-last frame video generation

    Alibaba Tongyi Wanxiang Team . Wan 2.1 flf2v: First-last frame video generation. https://www.runcomfy.com/comfyui-workflows/wan-2-1-flf2v-first-last-frame-video-generation, 2025. Accessed: May 14, 2025

    work page 2025

  2. [2]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18208--18218, 2022

    work page 2022

  3. [3]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014

    arXiv 2014

  4. [4]

    Flux.1 redux [dev]: Image variation adapter for flux.1 models

    Black Forest Labs . Flux.1 redux [dev]: Image variation adapter for flux.1 models. https://huggingface.co/black-forest-labs/FLUX.1-Redux-dev, 2025

    work page 2025

  5. [5]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18392--18402, 2023

    work page 2023

  6. [6]

    Multi-view consistency loss for improved single-image 3d reconstruction of clothed people

    Akin Caliskan, Armin Mustafa, Evren Imre, and Adrian Hilton. Multi-view consistency loss for improved single-image 3d reconstruction of clothed people. In Proceedings of the Asian Conference on Computer Vision, 2020

    work page 2020

  7. [7]

    Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22560--22570, 2023

    2023

  8. [8]

    Training-free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 5343--5353, 2024

    work page 2024

Show all 69 references
  1. [9]

    Perception prioritized training of diffusion models

    Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11472--11481, 2022

    2022

  2. [10]

    Flux.1-dev-controlnet-inpainting-beta

    Alimama Creative. Flux.1-dev-controlnet-inpainting-beta. https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta, 2024

    2024

  3. [11]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34: 0 8780--8794, 2021

    2021

  4. [12]

    Internlm-xcomposer2: Mastering free-form text-image composition and comprehension in vision-language large model

    Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. Internlm-xcomposer2: Mastering free-form text-image composition and comprehension in vision-language large model. arXiv preprint arXiv:2401.16420, 2024

    2024 arXiv

  5. [13]

    Efros, and Aleksander Holynski

    Dave Epstein, Allan Jabri, Ben Poole, Alexei A. Efros, and Aleksander Holynski. Diffusion self-guidance for controllable image generation. 2023

    2023

  6. [14]

    Casteer: Steering diffusion models for controllable generation

    Tatiana Gaintseva, Chengcheng Ma, Ziquan Liu, Martin Benning, Gregory Slabaugh, Jiankang Deng, and Ismail Elezi. Casteer: Steering diffusion models for controllable generation. arXiv preprint arXiv:2503.09630, 2025

    2025 arXiv

  7. [15]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63 0 (11): 0 139--144, 2020

    2020

  8. [16]

    Generative adversarial nets

    Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014

    2014

  9. [17]

    Prompt-to-prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022

    2022 arXiv

  10. [18]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718, 2021

    2021 arXiv

  11. [19]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  12. [20]

    3d shape completion with multi-view consistent inference

    Tao Hu, Zhizhong Han, and Matthias Zwicker. 3d shape completion with multi-view consistent inference. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 10997--11004, 2020

    2020

  13. [21]

    Globally and locally consistent image completion

    Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and locally consistent image completion. ACM Transactions on Graphics (ToG), 36 0 (4): 0 1--14, 2017

    2017

  14. [22]

    Video interpolation with diffusion models

    Siddhant Jain, Daniel Watson, Eric Tabellion, Ben Poole, Janne Kontkanen, et al. Video interpolation with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7341--7351, 2024

    2024

  15. [23]

    Artist: Aesthetically controllable text-driven stylization without training

    Ruixiang Jiang and Changwen Chen. Artist: Aesthetically controllable text-driven stylization without training. arXiv preprint arXiv:2407.15842, 2024

    2024 arXiv

  16. [24]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6007--6017, 2023

    2023

  17. [25]

    Dense text-to-image generation with attention modulation

    Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu. Dense text-to-image generation with attention modulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7701--7711, 2023

    2023

  18. [26]

    Auto-encoding variational bayes, 2013

    Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013

    2013

  19. [27]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015--4026, 2023

    2023

  20. [28]

    o hler, Christian Schuler, Bernhard Sch \

    Rolf K \"o hler, Christian Schuler, Bernhard Sch \"o lkopf, and Stefan Harmeling. Mask-specific inpainting with deep neural networks. In Pattern Recognition: 36th German Conference, GCPR 2014, M \"u nster, Germany, September 2-5, 2014, Proceedings 36 , pages 523--534. Springer, 2014

    2014

  21. [29]

    Kuaishou unveils proprietary video generation model 'kling'

    Kuaishou Technology . Kuaishou unveils proprietary video generation model 'kling'. https://ir.kuaishou.com/news-releases/news-release-details/kuaishou-unveils-proprietary-video-generation-model-kling, 2024. Accessed: May 14, 2025

    2024

  22. [30]

    Flux.1 [dev]

    Black Forest Labs. Flux.1 [dev]. https://huggingface.co/black-forest-labs/FLUX.1-dev, 2024 a

    2024

  23. [31]

    Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2024 b

    2024

  24. [32]

    Context-aware synthesis and placement of object instances

    Donghoon Lee, Sifei Liu, Jinwei Gu, Ming-Yu Liu, Ming-Hsuan Yang, and Jan Kautz. Context-aware synthesis and placement of object instances. Advances in neural information processing systems, 31, 2018

    2018

  25. [33]

    Layerdiffusion: Layered controlled image editing with diffusion models

    Pengzhi Li, Qinxuan Huang, Yikang Ding, and Zhiheng Li. Layerdiffusion: Layered controlled image editing with diffusion models. In SIGGRAPH Asia 2023 Technical Communications, pages 1--4. 2023 a

    2023

  26. [34]

    Mt-gan: toward realistic image composition based on spatial features

    Xiang Li, Guowei Teng, Ping An, and Hai-yan Yao. Mt-gan: toward realistic image composition based on spatial features. EURASIP Journal on Advances in Signal Processing, 2023 0 (1): 0 46, 2023 b

    2023

  27. [35]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  28. [36]

    Image inpainting for irregular holes using partial convolutions

    Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European conference on computer vision (ECCV), pages 85--100, 2018

    2018

  29. [37]

    Mardini: Masked autoregressive diffusion for video generation at scale

    Haozhe Liu, Shikun Liu, Zijian Zhou, Mengmeng Xu, Yanping Xie, Xiao Han, Juan C P \'e rez, Ding Liu, Kumara Kahatapitiya, Menglin Jia, et al. Mardini: Masked autoregressive diffusion for video generation at scale. arXiv preprint arXiv:2410.20280, 2024 a

    arXiv

  30. [38]

    Diffpop: Plausibility-guided object placement diffusion for image composition

    Jiacheng Liu, Hang Zhou, Shida Wei, and Rui Ma. Diffpop: Plausibility-guided object placement diffusion for image composition. In Computer Graphics Forum, volume 43, page e15246. Wiley Online Library, 2024 b

    2024

  31. [39]

    Step1x-edit: A practical framework for general image editing

    Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Step1x-edit: A practical framework for general image editing. arXiv preprint arXiv:2504.17761, 2025

    2025 arXiv

  32. [40]

    Directed diffusion: Direct control of object placement through attention guidance

    Wan-Duo Kurt Ma, Avisek Lahiri, John P Lewis, Thomas Leung, and W Bastiaan Kleijn. Directed diffusion: Direct control of object placement through attention guidance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4098--4106, 2024

    2024

  33. [41]

    Null-text inversion for editing real images using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6038--6047, 2023

    2023

  34. [42]

    Intelligent scissors for image composition

    Eric N Mortensen and William A Barrett. Intelligent scissors for image composition. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pages 191--198, 1995

    1995

  35. [43]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 42...

    2024

  36. [44]

    Understanding the latent space of diffusion models through the lens of riemannian geometry

    Yong-Hyun Park, Mingi Kwon, Jaewoong Choi, Junghyo Jo, and Youngjung Uh. Understanding the latent space of diffusion models through the lens of riemannian geometry. Advances in Neural Information Processing Systems, 36: 0 24129--24142, 2023

    2023

  37. [45]

    Context encoders: Feature learning by inpainting

    Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2536--2544, 2016

    2016

  38. [46]

    Grounded text-to-image synthesis with attention refocusing

    Quynh Phung, Songwei Ge, and Jia-Bin Huang. Grounded text-to-image synthesis with attention refocusing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7932--7942, 2024

    2024

  39. [47]

    Deadiff: An efficient stylization diffusion model with disentangled representations

    Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Jiawei Liu, Lang Chen, Qian He, and Yongdong Zhang. Deadiff: An efficient stylization diffusion model with disentangled representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pag...

    2024

  40. [48]

    Unicontrol: A unified diffusion model for controllable visual generation in the wild

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffusion model for controllable visual generation in the wild. arXiv preprint arXiv:2305.11147, 2023

    2023 arXiv

  41. [49]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  42. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695, 2022

    2022

  43. [51]

    Semantic image inversion and editing using rectified stochastic differential equations

    Litu Rout, Yujia Chen, Nataniel Ruiz, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Semantic image inversion and editing using rectified stochastic differential equations. arXiv preprint arXiv:2410.10792, 2024

    2024 arXiv

  44. [52]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  45. [53]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020 a

    2010 arXiv

  46. [54]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020 b

    2011 arXiv

  47. [55]

    Parasol: Parametric style control for diffusion image synthesis

    Gemma Canet Tarr \'e s, Dan Ruta, Tu Bui, and John Collomosse. Parasol: Parametric style control for diffusion image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2432--2442, 2024

    2024

  48. [56]

    Add-it: Training-free object insertion in images with pretrained diffusion models

    Yoad Tewel, Rinon Gal, Dvir Samuel, Yuval Atzmon, Lior Wolf, and Gal Chechik. Add-it: Training-free object insertion in images with pretrained diffusion models. arXiv preprint arXiv:2411.07232, 2024

    2024 arXiv

  49. [57]

    Towards effective usage of human-centric priors in diffusion models for text-based human image generation

    Junyan Wang, Zhenhong Sun, Zhiyu Tan, Xuanbai Chen, Weihua Chen, Hao Li, Cheng Zhang, and Yang Song. Towards effective usage of human-centric priors in diffusion models for text-based human image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  50. [58]

    Stylediffusion: Controllable disentangled style transfer via diffusion models

    Zhizhong Wang, Lei Zhao, and Wei Xing. Stylediffusion: Controllable disentangled style transfer via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7677--7689, 2023

    2023

  51. [59]

    Tokencompose: Text-to-image diffusion with token-level supervision

    Zirui Wang, Zhizhou Sha, Zheng Ding, Yilin Wang, and Zhuowen Tu. Tokencompose: Text-to-image diffusion with token-level supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8553--8564, 2024 b

    2024

  52. [60]

    Uncovering the disentanglement capability in text-to-image diffusion models

    Qiucheng Wu, Yujian Liu, Handong Zhao, Ajinkya Kale, Trung Bui, Tong Yu, Zhe Lin, Yang Zhang, and Shiyu Chang. Uncovering the disentanglement capability in text-to-image diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pag...

    1900

  53. [61]

    Deep convolutional neural network for image deconvolution

    Li Xu, Jimmy S Ren, Ce Liu, and Jiaya Jia. Deep convolutional neural network for image deconvolution. Advances in neural information processing systems, 27, 2014

    2014

  54. [62]

    Attngan: Fine-grained text to image generation with attentional generative adversarial networks

    Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. Attngan: Fine-grained text to image generation with attentional generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages ...

    2018

  55. [63]

    Dynamic prompt learning: Addressing cross-attention leakage for text-based image editing

    Fei Yang, Shiqi Yang, Muhammad Atif Butt, Joost van de Weijer, et al. Dynamic prompt learning: Addressing cross-attention leakage for text-based image editing. Advances in Neural Information Processing Systems, 36: 0 26291--26303, 2023

    2023

  56. [64]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.(2023)

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.(2023). arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  57. [65]

    Semantic image inpainting with deep generative models

    Raymond A Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N Do. Semantic image inpainting with deep generative models. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5485--5493, 2017

    2017

  58. [66]

    Generative image inpainting with contextual attention

    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5505--5514, 2018

    2018

  59. [67]

    Exploring diffusion time-steps for unsupervised representation learning

    Zhongqi Yue, Jiankun Wang, Qianru Sun, Lei Ji, Eric I Chang, Hanwang Zhang, et al. Exploring diffusion time-steps for unsupervised representation learning. arXiv preprint arXiv:2401.11430, 2024

    2024 arXiv

  60. [68]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836--3847, 2023 a

    2023

  61. [69]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077, 2023 b

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.