Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

ColorEdit: Training-free Image-Guided Color editing with diffusion model

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free method recolors objects in diffusion-generated and real images by aligning cross-attention Value matrices with a reference color image during early denoising.

desk verdict A real training-free recolor recipe plus a useful first benchmark, but the paper's own ablations undercut its clean 'Value matrices carry color, not geometry' story. read the letter →

arxiv 2411.10232 v2 pith:MWZJGSJS submitted 2024-11-15 cs.CV cs.AI

Xingxi Yin , Zhi Li , Jingfeng Zhang , Chenglin Li , Yin Zhang This is my paper · ORCID
classification cs.CVcs.AI
keywords training-freeimageeditingcross-attentionValuealignmentAdaINdiffusionmodelcolorCOLORBENCHimage-guidedchangenull-textinversionobject-level
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an object's color can be edited without training or fine-tuning by aligning the Value matrices of a text-to-image diffusion model's cross-attention layers with those of a reference color image during early denoising. This matters because text-guided editing often fails to change color reliably, due to attention leakage and attribute collision between the object and a color word in the prompt. An image reference avoids introducing that color word, so the failure modes disappear. The paper also contributes a benchmark, COLORBENCH, for evaluating color-change methods on generated and real images.

What carries the argument

The load-bearing mechanism is color attribute alignment via AdaIN on cross-attention Value matrices: $V^* = \operatorname{AdaIN}(V^T, V^c)$, which rescales the target Value distribution to match the mean and standard deviation of the reference Value distribution. This alignment runs only in the first, second, and third CrossAttnUpBlocks of the U-Net decoder during the early denoising steps $t \in [0.8T, T]$, when shape and contour are being fixed. The supporting components are self-attention map replacement from the source image, latent blending of the reference image's noise at a ratio near 0.1, and background restoration in the final steps; these guards keep object structure and background intact. The early-window timing is what separates clean color transfer from structural breakage, since the paper's ablation shows that aligning Value matrices across all steps severely degrades structure.

What would settle it

Take a fixed source object and align its cross-attention Value matrices with a reference image of a different object of unrelated shape but the same target color, keeping all other steps identical; if the output object's structure distorts significantly even in the early-window setting, then Value matrices are not color-separable and the central claim fails. A more focused variant would swap only the Value matrices of the third CrossAttnUpBlock while leaving the other up-blocks untouched, to test whether the claimed locus of color information is correct.

Watch

Extended reading notes

Core claim

The central discovery is that stable object-level color editing can be achieved by substituting the cross-attention Value matrices of the target image with Value matrices extracted from a reference color image, rather than by amplifying the Key or Value of a color term in the prompt. The paper argues that shape, contour, and texture are established in the early denoising stage of the U-Net decoder, specifically in the first, second, and third CrossAttnUpBlocks, and that Value alignment applied only during that early window transfers color without destroying structure. Because no color term is added to the text prompt, cross-attention leakage and attribute collision are eliminated. The paper further builds COLORBENCH and reports that the method outperforms text-guided baselines in preserving object structure and background while matching them on color change.

Load-bearing premise

The method assumes that the Value matrices of cross-attention in the U-Net decoder carry color and appearance information that can be transferred without also transferring shape, pose, or texture; if those matrices also encode object identity, the alignment step would alter or break the object.

Editorial extensions

If this is right

  • Because the Value matrices of a reference color image need to be extracted only once, the same reference can recolor many different source images with no additional inversion cost.
  • The method extends to real images by first running null-text inversion, so it is not limited to synthesized inputs.
  • Avoiding a color word in the prompt sidesteps cross-attention leakage and attribute collision, which are the specific failure mechanisms the paper identifies in text-guided editing.
  • The ablation evidence implies that early-stage Value alignment is both sufficient and necessary for color transfer, with full-process alignment harming object identity.
  • The paper itself notes that smaller objects are harder to recolor and that multi-object color changes must be performed in a multi-turn way.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Value alignment truly isolates color, the same principle could plausibly extend to transferring other appearance attributes, such as lighting or material gloss, by aligning Value statistics in the same U-Net decoder blocks; the paper does not test that generalization.
  • An independent test of the core assumption could align Values from a reference image of a different object with unrelated shape; if structure remains intact, color information is genuinely separable, whereas if it distorts, the method depends on coarse geometric similarity between source and reference objects.
  • COLORBENCH may become a standard evaluation set for future color-change methods, even though the paper only evaluates it on the proposed method and text-guided baselines.
  • The method inherits segmentation quality from the mask-generation model used, so thin or small objects are likely to be under-covered; this is consistent with the paper's reported difficulty with small objects.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper targets the task of object color editing in diffusion-based image generation and editing. It analyzes cross-attention layers in Stable Diffusion and proposes ColorEdit, a training-free method that recolors an object by aligning the Value matrices of cross-attention in the U-Net decoder with those extracted from a reference color image via AdaIN during the early denoising stage, while replacing self-attention maps from the source to preserve structure and blending a masked reference latent for stronger color transfer. The authors also introduce COLORBENCH, a real-image color-change benchmark, and compare against six text-guided editing baselines with automatic and human metrics.

Significance. The paper addresses a well-defined, understudied editing task and offers a simple training-free baseline plus a benchmark, both of which are potentially useful to the community. The ablation study is systematic and shows that each component contributes, and the human evaluation provides complementary evidence. However, the central mechanistic claim about Value matrices being color-specific is not established by the paper's own results, and the quantitative evaluation has methodological weaknesses that require revision.

major comments (4)
  1. [Section 4.1 and Section 4.3, Eq. (3)-(4), Tables 2 and 4] The load-bearing mechanistic claim is that Value matrices in cross-attention carry color/appearance information separable from geometry, so that AdaIN alignment transfers color without transferring structure. The paper's own evidence undercuts this claim: Table 2 shows that self-attention map replacement is necessary for structure preservation, and Table 4 shows that extending Value alignment from 20% to 50% of timesteps drops DS from 0.957 to 0.876 and SSIM from 0.830 to 0.715, while all-timestep alignment collapses DS to 0.622. These results indicate that the Value subspace also encodes identity and structure, and that the stability of the method is carried by the early-only alignment window and the auxiliary structural constraints. Please soften the mechanistic interpretation in Section 4.1 or add a positive control that isolates the Value subspace, for example by ablating alignment on a low-rank projection of V, to show what information is actually transferred.
  2. [Section 5, Tables 4 and 5] The default hyperparameters (alignment window tau, latent blending ratio R, and background-preservation count) are tuned on the same generated dataset used for the main results in Table 1: Table 4 tunes the alignment window and Table 5 tunes the blending ratio on that set. There is no validation split and no error bars or significance tests, so the comparison against all baselines uses settings selected to maximize the proposed method's scores. Please report results on a held-out split or with fixed defaults, and include multi-seed variance or significance tests for the main metrics.
  3. [Section 4.4 and Algorithms 1-2] The algorithm specification is ambiguous in two places. First, the loop in Algorithms 1 and 2 iterates 'for i = 1 to n', but n is never defined or set in the paper. Second, the background-preservation condition in line 14 is 'if t < T - N', while the text says preservation is applied only in the final few steps; with T=50 and N=5, this condition is true for almost the entire denoising trajectory, so either the inequality is reversed or N has a different meaning that must be stated. These ambiguities prevent faithful reproduction of the method.
  4. [Section 8.3 and Table 1 (COLORBENCH)] The evaluation metrics for the color-change goal are not fully adequate. L1Hue is computed as a linear distance on the HSV hue channel, but hue is circular and is undefined for black, gray, and white, which are included in the seven target colors. On COLORBENCH, LPIPS on the object area is used as a color-change score, but LPIPS is a perceptual content metric rather than a color metric, so a larger value can reflect unintended structure or texture change rather than successful recoloring. In addition, the real-image benchmark sets all backgrounds to white, so the background-preservation results on this benchmark may not transfer to natural scenes. Please add a proper masked-object color metric and report results on more realistic background distributions.
minor comments (7)
  1. [Section 8.1] There is a typo in 'For symthesised images'; it should be 'synthesized'.
  2. [Figure 10 caption] The phrase 'keeping preview turn editing results' is unclear and should be rephrased.
  3. [Section 4.2 and Eq. (2)] The notation for the projection functions fQ, fK, and fV is missing the argument parentheses in Eq. (2); the intended mathematical expression should be made explicit.
  4. [Section 5.2] The paper states that for the generated dataset the best SAM mask was 'manually selected'; please clarify how this selection was performed and whether the same selected masks are used for all compared methods.
  5. [Section 5.2 and Section 8.2] It is unclear whether the generated dataset is also part of COLORBENCH or only the real-image set; please define the benchmark scope precisely.
  6. [Section 5 and Section 8.5] The text says R=0.15 gives better human-perceived color change while R=0.1 is used in the main quantitative results; please state the exact default used in Table 1 and justify the choice.
  7. [General] No code or implementation details such as the PyTorch hook setup are released; providing the code or a detailed configuration would substantially improve reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Generated-dataset SOTA is partly constructed: hyperparameters are selected from the same test metrics, while the core Value-alignment effect still has independent ablation support.

  1. fitted input called prediction [Section 4.4 (main text), Sections 8.4-8.5 and Tables 4-5 (Supplementary), reported in Table 1]
    "In practice, we observe that setting the ratio to 0.1 yields the best quantitative results, while setting it to 0.15 gives better color changes perceived by humans."

    The main quantitative comparison is reported after selecting hyperparameters on the same generated dataset. Section 4.4 fixes the latent blending ratio to 0.1 because it 'yields the best quantitative results,' and Table 4 labels the chosen alignment window as 'First 20% time step (our)' after showing that longer windows lower DS from 0.957 to 0.876 and SSIM from 0.830 to 0.715. Table 1 then reports the DS, SSIM, L1Hue, L1HSV and LPIPS_bg for 'Ours' on that same generated dataset. The structure-preservation and color-balance advantages in that comparison are therefore not independent predictions: they are values that the hyperparameter choice was selected to maximize.

full rationale

The core editing recipe is not definitionally circular: Eqs. (3)-(4) define an AdaIN operation on cross-attention Value matrices, and the claim that Value alignment transfers color is an empirical observation (Fig. 7) that is tested by ablation. The ablation is informative: without cross-attention alignment, L1Hue rises from 50.800 to 52.677, so the color-change effect is not merely an artifact of self-attention replacement or latent blending. There is also no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in by the authors' own prior work. The concrete circular element is in the quantitative evaluation: the latent blending ratio and the Value-alignment window are chosen by inspecting the same generated-dataset metrics (Tables 4 and 5), and then those same metrics are reported as the method's performance in Table 1. That makes the generated-dataset structural and color-balance superiority partly a fitted input called a prediction. The real-image COLORBENCH results and the human evaluation provide separate, independent support, so the circularity is partial. The near-zero LPIPS_bg is also expected because the algorithm explicitly copies source background latents with a mask, but that is an evaluation-design caveat rather than a derivation-circularity step.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or model entities are introduced; COLORBENCH is a dataset, not an entity. The method relies on one new mechanism (AdaIN on Value matrices) and several standard components. The free parameters are data-tuned hyperparameters rather than fitted physical constants.

free parameters (3)
  • alignment interval tau = 0.8T (first 20% of denoising steps)
    Chosen from the sweep in Table 4; more alignment improves color but breaks structure, so the best quantitative trade-off is selected on the evaluation set.
  • reference latent blending ratio R = 0.1 (0.15 for human-preferred)
    Selected from the sweep in Table 5; the authors state 0.1 yields the best quantitative results and 0.15 gives better human-perceived color changes.
  • background preservation steps N = last 5 timesteps
    Fixed hyperparameter with no ablation across values, used to keep the background unchanged.
assumptions (5)
  • domain assumption In Stable Diffusion's U-Net decoder up-blocks, object shape, contour and texture are established early in denoising, with shape, contour and texture in the 1st, 2nd, and 3rd CrossAttnUpBlocks.
    Established by the authors' cross-attention map visualizations (Figs. 4, 5, 11) but not proven; underlies the choice to align only early decoder up-block layers.
  • ad hoc to paper Value matrices in cross-attention encode color/appearance semantics separable from geometry, and AdaIN transfers color without transferring other content.
    Core mechanism of Eq. (3)-(4); no proof, only empirical evidence from editing examples, so if false the method breaks object structure.
  • domain assumption Null-text Inversion faithfully reconstructs source and reference images.
    Used to obtain z_s, z_c and V_c; if inversion is lossy, color and structure transfer is corrupted. Cited from Mokady et al. [27].
  • domain assumption Self-attention map replacement preserves object structure.
    Borrowed from PnP [42], MasaCtrl [4] and FreePromptEditing [24]; accepted prior result.
  • domain assumption SAM segmentations of the object are accurate.
    Mask quality limits color transfer and background preservation; small objects are acknowledged as a failure mode.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ColorEdit: Training-free Image-Guided Color editing with diffusion model." pith.science (2026). https://pith.science/paper/MWZJGSJS

@misc{pith2026241110232,
  author       = {Pith},
  title        = {Pith review of: ColorEdit: Training-free Image-Guided Color editing with diffusion model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MWZJGSJS}},
  note         = {Machine review of arXiv:2411.10232}
}
read the original abstract

Text-to-image (T2I) diffusion models, with their impressive generative capabilities, have been adopted for image editing tasks, demonstrating remarkable efficacy. However, due to attention leakage and collision between the cross-attention map of the object and the new color attribute from the text prompt, text-guided image editing methods may fail to change the color of an object, resulting in a misalignment between the resulting image and the text prompt. In this paper, we conduct an in-depth analysis on the process of text-guided image synthesizing and what semantic information different cross-attention blocks have learned. We observe that the visual representation of an object is determined in the up-block of the diffusion model in the early stage of the denoising process, and color adjustment can be achieved through value matrices alignment in the cross-attention layer. Based on our findings, we propose a straightforward, yet stable, and effective image-guided method to modify the color of an object without requiring any additional fine-tuning or training. Lastly, we present a benchmark dataset called COLORBENCH, the first benchmark to evaluate the performance of color change methods. Extensive experiments validate the effectiveness of our method in object-level color editing and surpass the performance of popular text-guided image editing approaches in both synthesized and real images.

Figures

Figures reproduced from arXiv: 2411.10232 by the authors.

Figure 1
Figure 1. Multi-Object color editing. Each outcome image is changing the color of the hat first and then changing the color of the bowl and coat, using the associated reference color image [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Example of color change. Text-guided editing methods may fail to change the color of an object while maintaining the structure of it or the background information. , removing [9, 10, 50], or replacing objects [2, 3, 6, 7, 9– 11, 13, 19, 20, 24, 27, 29, 48, 51] , and changing the back￾ground [2, 22]. Structural editing modifies the composition of an image, including repositioning [6, 8], changing the size and shape [… view at source ↗
Figure 3
Figure 3. The Image-guided Color Editing Framework. Our framework including: (a) Color Image Inversion. The reference color image is inverted to initial noise and extracted the Value matrices of the cross-attention layers. (b) Source Image inversion. Extracting the self-attention map and latent Z s T , ..., Zs 0 of the source image. (c) Denoising Process, which including latent blending at the begining, cross￾attention layer … view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Cross-attention map of objects in the decoder of U￾net. We visualize the average cross-attention maps of various ob￾jects across all timesteps. As observed, the shape, contour, and texture of an object are determined in the U-Net decoder [PITH_FULL_IMAGE:figures/full_…
Figure 5
Figure 5. Figure 5: Cross-attention map of object in different denoising steps in the decoder of U-net. We visualize the cross-attention maps of an object at various diffusion steps within the U-Net de￾coder [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Cross-attention maps for object and color attributes using P2P to edit the color. As indicated, the mis-distribution or collision of the attention map between the color attribute and the object are the reasons fail to change the color of an object. to amplify the Key o…
Figure 7
Figure 7. Figure 7: Resulting image and cross-attention after amplifying the Key matrices and Value matrices of color term when using P2P to change the color of the phone to red. where ”*x” means * time of the original value. As illustrated, compared to altering the Key, modifying the Val…
Figure 8
Figure 8. Figure 8: Comparisons to text-guided editing methods. For generated image, the text prompt is ”A photo of pikachu” and the object is ”pikachu”. For real image, the object is ”cauliflower”. For real image and the P2P method, we use Null text inversion method inverse the real imag…
Figure 10
Figure 10. Figure 10: Color editing with different sizes of object on real image. As demonstrated, Our method can edit objects of different sizes in the same image while keeping preview results unchanged. LP IP Sobj to measure the color change of the object and using LP IP Sbg to measure t…
Figure 11
Figure 11. Figure 11: Visualizing cross-attention map in the text-guided diffusion image generation. In this case, the text prompt is ”a photo of *”, where ”*” represents a specific object. through text prompts, and the phenomenon of attention leakage also exists. As demonstrated in [PITH…
Figure 12
Figure 12. Figure 12: Visualization of the Cross-attention maps for object and color attributes using InstructPix2Pix to edit the color. In this example, the source image is generated with the prompt ”an image of a toaster”, while the edit text prompt is ”turn the color of toaster to *,” w…
Figure 15
Figure 15. Figure 15: Visualization of the edited image using P2P with the ground-truth mask. As demonstrated, utilizing cross-attention maps to extract the binary mask of an object does not perfectly seg￾ment the object from the background, resulting in changes along the object’s edges. I…
Figure 16
Figure 16. Figure 16: Comparisons to text-guided editing methods for generated image. In each row, we present the modified image reflecting the desired color for the object using various methods. From top to bottom, the desired colors are ”white,” ”gray,” ”black,” ”red,” ”yellow,” ”blue” a…
Figure 17
Figure 17. Figure 17: Comparisons to text-guided editing methods for real images. In each row, we present the modified image reflecting the desired color for the object using various methods. From top to bottom, the desired colors are ”white,” ”gray,” ”black,” ”red,” ”yellow,” ”blue” and ”…
Figure 18
Figure 18. Figure 18: Color editing for generated images. The first row displays the source image, while the subsequent rows depict the modified image with the desired color. photo of a {}”, 2) ”an image of a {}” , 3) ”a photo of a nice {}”, 4) ”a photo of a large {}”, 5) ”a good photo of …
Figure 19
Figure 19. Figure 19: Color editing for real images. The first row displays the source image, while the subsequent rows depict the modified image with the desired color. we compute the DINO score using the grayscale versions of both the source and target images and call it DS. For structur…
Figure 20
Figure 20. Figure 20: Example data of COLORBENCH. The COLORBENCH dataset including 406 images from 100 subjects, and the color of object in each image are change to 7 different colors, including white, gray, black, red, yellow, blue, and green. formation. For color accuracy measure, the RG…
Figure 21
Figure 21. Figure 21: Screenshot from the Human Evaluation. The second row of the image shows the results obtained by different methods. The user is asked to rank the performance of each method from 0 to 5. 0 means the worst,while 5 means the best. DS ↑ SSIM ↑ CS ↑ L1 Hue obj ↓ L1 HSV obj …
Figure 22
Figure 22. Figure 22: Ablation study. The original image was generated using the prompt “a teddy bear”, with the object is ’teddy bear’. [12] Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vec￾tor quantized diffusion model for text-to-image…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Color Me Correctly: Bridging Perceptual Color Spaces and Text Embeddings for Improved Diffusion Generation

    cs.CV 2025-09 conditional novelty 6.0 of 10

    Diffusion models can render compound color names more accurately when an LLM disambiguates terms and embeddings are interpolated using CIELab distances.

Reference graph

Works this paper leans on

53 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Attention is all you need

    Vaswani Ashish. Attention is all you need. Advances in neural information processing systems, 30:I, 2017. 3

    work page 2017

  2. [2]

    Blended latent diffusion

    Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM Transactions on Graphics (TOG), 42 (4):1–11, 2023. 1, 2

    work page 2023

  3. [3]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 1, 2, 4, 6, 9, 11, 12

    work page 2023

  4. [4]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 22560–22570, 2023. 2, 5, 10

    work page 2023

  5. [5]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 6, 14

    work page 2021

  6. [6]

    Anydoor: Zero-shot object-level im- age customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level im- age customization. arXiv preprint arXiv:2307.09481, 2023. 1, 2

    arXiv 2023

  7. [7]

    Diffedit: Diffusion-based seman- tic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion-based seman- tic image editing with mask guidance. arXiv preprint arXiv:2210.11427, 2022. 2

    arXiv 2022

  8. [8]

    Diffusion self-guidance for control- lable image generation

    Dave Epstein, Allan Jabri, Ben Poole, Alexei Efros, and Aleksander Holynski. Diffusion self-guidance for control- lable image generation. Advances in Neural Information Processing Systems, 36:16222–16239, 2023. 2

    2023

Show all 53 references
  1. [9]

    Guiding instruction-based im- age editing via multimodal large language models

    Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, and Zhe Gan. Guiding instruction-based im- age editing via multimodal large language models. arXiv preprint arXiv:2309.17102, 2023. 1, 2, 4, 6, 9, 11, 12

    2023 arXiv

  2. [10]

    Instructdiffusion: A generalist modeling inter- face for vision tasks

    Zigang Geng, Binxin Yang, Tiankai Hang, Chen Li, Shuyang Gu, Ting Zhang, Jianmin Bao, Zheng Zhang, Houqiang Li, Han Hu, et al. Instructdiffusion: A generalist modeling inter- face for vision tasks. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Reco...

    2024

  3. [11]

    a teddy bear

    Vidit Goel, Elia Peruzzo, Yifan Jiang, Dejia Xu, Nicu Sebe, Trevor Darrell, Zhangyang Wang, and Humphrey Shi. Pair-diffusion: Object-level image editing with structure- and-appearance paired diffusion models. arXiv preprint arXiv:2303.17546, 1, 2023. 1, 2 18 Figure 22. Ablatio...

    2023 arXiv

  4. [12]

    Vec- tor quantized diffusion model for text-to-image synthesis

    Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vec- tor quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 10696–10706, 2022. 1

    2022

  5. [13]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 1, 2, 4, 5, 6, 10, 11, 12

    2022 arXiv

  6. [14]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 1

    2022 arXiv

  7. [15]

    Denoising dif- 19 fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- 19 fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  8. [16]

    Kv inversion: Kv embeddings learning for text-conditioned real image action editing

    Jiancheng Huang, Yifan Liu, Jin Qin, and Shifeng Chen. Kv inversion: Kv embeddings learning for text-conditioned real image action editing. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV) , pages 172–184. Springer, 2023. 2

    2023

  9. [17]

    Arbitrary style transfer in real-time with adaptive instance normalization

    Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceed- ings of the IEEE international conference on computer vi- sion, pages 1501–1510, 2017. 5

    2017

  10. [18]

    Unicolor: A unified framework for multi-modal colorization with trans- former

    Zhitong Huang, Nanxuan Zhao, and Jing Liao. Unicolor: A unified framework for multi-modal colorization with trans- former. ACM Transactions on Graphics (TOG), 41(6):1–16,

  11. [19]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007–6017, 2023. 1, 2, 6

    2023

  12. [20]

    Dif- fusionclip: Text-guided diffusion models for robust image manipulation

    Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Dif- fusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2426– 2435, 2022. 2

    2022

  13. [21]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 202...

    2023

  14. [22]

    Layerdiffusion: Layered controlled image editing with dif- fusion models

    Pengzhi Li, Qinxuan Huang, Yikang Ding, and Zhiheng Li. Layerdiffusion: Layered controlled image editing with dif- fusion models. In SIGGRAPH Asia 2023 Technical Commu- nications, pages 1–4. 2023. 2

    2023

  15. [23]

    Control color: Multi- modal diffusion-based interactive image colorization

    Zhexin Liang, Zhaochen Li, Shangchen Zhou, Chongyi Li, and Chen Change Loy. Control color: Multi- modal diffusion-based interactive image colorization. arXiv preprint arXiv:2402.10855, 2024. 1

    2024 arXiv

  16. [24]

    Towards understanding cross and self-attention in stable diffusion for text-guided image editing

    Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7817–7826, 2024. 1, 2...

    2024

  17. [25]

    Tf-icon: Diffusion-based training-free cross-domain image composi- tion

    Shilin Lu, Yanzhu Liu, and Adams Wai-Kin Kong. Tf-icon: Diffusion-based training-free cross-domain image composi- tion. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 2294–2305, 2023. 1

    2023

  18. [26]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 14297–14306, 2023. 1

    2023

  19. [27]

    Null-text inversion for editing real im- ages using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real im- ages using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038–6047, 2023. 2, 5, 9, 12

    2023

  20. [28]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 1

    2021 arXiv

  21. [29]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–11, 2023. 1, 2

    2023

  22. [30]

    Pytorch: An im- perative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An im- perative style, high-performance deep learning library. Ad- vances in neural information processing systems ...

    2019

  23. [31]

    Localizing object-level shape variations with text-to-image diffusion models

    Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch- Elor, and Daniel Cohen-Or. Localizing object-level shape variations with text-to-image diffusion models. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 23051–23061, 2023. 2

    2023

  24. [32]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1

    2023 arXiv

  25. [33]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  26. [34]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1

    2022 arXiv

  27. [35]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 6

    2022

  28. [36]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  29. [37]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  30. [38]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- 20 ence on machine learning, pages 2256–2265. PMLR, 2015. 2

    2015

  31. [39]

    Styledrop: Text-to-image generation in any style

    Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. Styledrop: Text-to-image generation in any style. arXiv preprint arXiv:2306.00983 ,

    arXiv

  32. [40]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 6

    2010 arXiv

  33. [41]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 2

    2011 arXiv

  34. [42]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1921–1930, 2023. 1, 2, 5, 6, 11, 12

    1921

  35. [43]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 3

    2017

  36. [44]

    Imagen editor and editbench: Advancing and evaluating text-guided im- age inpainting

    Su Wang, Chitwan Saharia, Ceslee Montgomery, Jordi Pont- Tuset, Shai Noy, Stefano Pellegrini, Yasumasa Onoe, Sarah Laszlo, David J Fleet, Radu Soricut, et al. Imagen editor and editbench: Advancing and evaluating text-guided im- age inpainting. In Proceedings of the IEEE/CVF c...

    2023

  37. [45]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 6, 7, 15

    2004

  38. [46]

    Stylediffusion: Controllable disentangled style transfer via diffusion models

    Zhizhong Wang, Lei Zhao, and Wei Xing. Stylediffusion: Controllable disentangled style transfer via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7677–7689, 2023. 2

    2023

  39. [47]

    L-cad: Language-based colorization with any-level descrip- tions using diffusion priors

    Shuchen Weng, Peixuan Zhang, Yu Li, Si Li, Boxin Shi, et al. L-cad: Language-based colorization with any-level descrip- tions using diffusion priors. Advances in Neural Information Processing Systems, 36, 2024. 1

    2024

  40. [48]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22428–22437, 2023. 1, 2

    2023

  41. [49]

    Cyclenet: Rethinking cycle consistency in text- guided diffusion for image manipulation

    Sihan Xu, Ziqiao Ma, Yidong Huang, Honglak Lee, and Joyce Chai. Cyclenet: Rethinking cycle consistency in text- guided diffusion for image manipulation. Advances in Neu- ral Information Processing Systems, 36, 2024. 2

    2024

  42. [50]

    Inst-inpaint: Instructing to remove objects with diffusion models

    Ahmet Burak Yildirim, Vedat Baday, Erkut Erdem, Aykut Erdem, and Aysegul Dundar. Inst-inpaint: Instructing to remove objects with diffusion models. arXiv preprint arXiv:2304.03246, 2023. 2

    2023 arXiv

  43. [51]

    Inpaint anything: Segment anything meets image inpainting

    Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. Inpaint anything: Segment anything meets image inpainting. arXiv preprint arXiv:2304.06790, 2023. 2

    2023 arXiv

  44. [52]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6, 15

    2018

  45. [53]

    Null-text guidance in diffu- sion models is secretly a cartoon-style creator

    Jing Zhao, Heliang Zheng, Chaoyue Wang, Long Lan, Wan- rong Huang, and Wenjing Yang. Null-text guidance in diffu- sion models is secretly a cartoon-style creator. In Proceed- ings of the 31st ACM International Conference on Multime- dia, pages 5143–5152, 2023. 2 21

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.