Pith. sign in

REVIEW 4 major objections 8 minor 47 references

Difference Inversion: Interpolate and Isolate the Difference with Token Consistency for Image Analogy Generation

T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the difference between two images can be distilled into a few text tokens that, added to any query image's prompt, transfer the edit across general Stable Diffusion models.

desk verdict Solid incremental advance in image analogy editing; the core idea is plausible but the quantitative evidence is weaker than it looks because of circular CLIP scoring and small evaluation sets. read the letter →

arxiv 2506.07750 v1 pith:3TSXHBWG submitted 2025-06-09 cs.CV

Hyunsoo Kim , Donghyun Kim , Suhyun Kim This is my paper
classification cs.CV
keywords imageanalogygenerationdifferenceinversiontextualdeltainterpolationtokenconsistencylosszeroinitializationstablediffusionmodel-agnosticediting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Difference Inversion, a method that takes one image pair (A, A') and a query image B, and produces B' such that A:A'::B:B' — the same change that turns A into A' is applied to B. The central move is to compress the difference between A and A' into a handful of continuous text tokens (DIFF tokens) that are appended to B's text prompt, so the method works with any general Stable Diffusion model, not just models trained for editing or inpainting. The difference is defined as an interpolated vector in CLIP space, blending the image-embedding delta with the text-embedding delta from captions, and the tokens are trained with a bidirectional consistency loss plus zero-initialized embeddings so that only the intended modification is encoded. If correct, this makes image-analogy editing plug-and-play, model-agnostic, and free of text instructions.

What carries the argument

The load-bearing object is the DIFF token, a small set of continuous text embeddings (5 in the final configuration) optimized to match the Interpolated $\Delta$. The Interpolated $\Delta$ is computed by spherical linear interpolation between the CLIP image-embedding difference $E_I(A') - E_I(A)$ and the CLIP text-embedding difference $E_T(\text{caption}_{A'}) - E_T(\text{caption}_A)$ at blend ratio $\alpha = 0.8$. Three mechanisms protect the inversion: Token Consistency Loss imposes cycle-consistency in both directions so the tokens encode only the delta, Zero Initialization starts the token embeddings at zero (following the ControlNet design) to avoid injecting unrelated content, and a CLIP cosine loss ties the tokens to the Interpolated $\Delta$.

What would settle it

Take a set of image pairs (A, A') whose CLIP directional similarity is low (the paper's own MNIST colorization example shows such pairs can score only around 0.5), optimize DIFF tokens, and apply them to a variety of B; if the resulting B' fails to reflect the intended edit for a substantial fraction, the Interpolated $\Delta$ claim fails. Alternatively, measure whether the learned DIFF tokens, when applied to a B that shares no content with A, produce artifacts from A or A' that reveal the tokens encode more than the difference.

Watch

Extended reading notes

Core claim

The central claim is that the edit between A and A' can be isolated in CLIP space as an Interpolated $\Delta$ ($D_{\text{inter}} = \text{Slerp}(D_{\text{img}}, D_{\text{txt}}; \alpha)$ with $\alpha = 0.8$), and that this vector can be optimized into a few DIFF tokens that transfer to arbitrary B. Combined with a token-consistency loss that enforces bidirectional fidelity (prompt_A + DIFF reconstructs A' while prompt_A' − DIFF reconstructs A) and zero-initialized token embeddings, the method produces B' that completes the analogy across Stable Diffusion backbones and, per the paper's experiments, outperforms DIA, Analogist, and VISII in directional scores, human evaluation, and VLM evaluation.

Load-bearing premise

The method assumes that a single vector obtained by spherical interpolation between the CLIP image delta and the CLIP text delta (with alpha = 0.8) faithfully represents the intended edit, and that a few text tokens optimized to match that vector in CLIP space will apply the same edit to any query image; if CLIP space misaligns with the perceived change, the tokens encode the wrong transformation.

Editorial extensions

If this is right

  • The method generates B' from only the image triplet, with no text instruction, yet applies to general Stable Diffusion models such as SD2.1 and SDXL.
  • Difference tokens are plug-and-play: once optimized on (A, A'), they can be concatenated with any query image's prompt, including prompts obtained via PEZ or captioning models, without extra training.
  • Reported directional scores with DINO-v2 surpass all baselines, including text-guided SDEdit and InstructPix2Pix, while the CLIP directional score is comparable to VISII despite VISII being evaluated on its own training distribution.
  • Transferability holds across backbones because the method is inference-based and never touches the diffusion model weights, so it works even when the model is provided in API form.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence not explored by the paper: if the Interpolated Delta is genuinely a faithful edit descriptor, the same DIFF tokens should transfer not only to visually similar B but also to B drawn from very different domains; measuring directional score as a function of the distance between B and A would bound the transfer radius.
  • The paper's own failure analysis locates the bound in CLIP space; one could extend the method by replacing the CLIP delta with a stronger embedding space (such as DINO-v2) or a learned mapping, which the paper itself suggests, and check whether token-consistency loss remains sufficient.
  • The bidirectional training objective implies that subtracting DIFF tokens from a prompt should invert the edit, an operation the paper does not test but that follows directly from the cycle-consistency formulation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes Difference Inversion, a method for image analogy generation A:A'::B:B' that learns a set of text-token embeddings (DIFF tokens) encoding the difference between A and A' and applies them to a query image B. The difference is defined as an interpolated CLIP delta between image and text deltas (Eq. 4-5), and the tokens are optimized with a CLIP cosine loss and a bidirectional token consistency loss (Eq. 6-10). The authors report quantitative and qualitative comparisons against DIA, Analogist, and VISII, plus human and VLM evaluations, and demonstrate transferability to SDXL.

Significance. If the empirical claims hold, the method would provide a model-agnostic, instruction-free mechanism for transferring visual edits across images, which is a timely contribution given the popularity of Stable Diffusion. The paper's strengths include a concrete algorithm (Algorithm 1), a plausible mechanism for avoiding residual image information via zero-initialization and bidirectional consistency, and demonstrations on multiple diffusion backbones. However, the evidence is currently not fully convincing: the main quantitative metric is correlated with the training objective, the independent metric shows a negligible margin with no error bars, and some implementation details are missing.

major comments (4)
  1. [Sec. 5.3, Eq. (6)] The CLIP directional score in Table 1 is not an independent evaluation of the claimed improvement. Because the DIFF tokens are optimized to minimize the cosine distance to D_inter (Eq. 6), and D_inter is built from CLIP image and text deltas, high scores on this metric are expected by construction. The DINO-v2 metric is more independent, but the margin over InstructPix2Pix is only 0.0009 (Table 1), and no confidence intervals or significance tests are provided. Please report error bars over multiple runs and include an evaluation metric not aligned with the training target.
  2. [Sec. 4, Algorithm 1] The method for obtaining the prompt for the query image B is never specified. Algorithm 1 only describes learning DIFF tokens from the pair {A, A'}; Fig. 3 shows an 'Optimized prompt (B)' without explaining how it is produced. At inference, the user must supply a text prompt for B, but the paper does not state whether this comes from a captioning model, from PEZ optimization on B, or from manual input. This missing specification is central to the plug-and-play claim and must be provided.
  3. [Sec. 5.1] The hyperparameters (alpha=0.8, lambda_tc=0.01, lambda_clip=6, token count=5) appear to be tuned on the same 300-sample InstructPix2Pix set used for evaluation, with no separate validation split. The reported numbers may therefore reflect overfitting to the test set. Please use a held-out validation split (or nested cross-validation) and report results on the held-out set.
  4. [Sec. 5.3, Tables 2-3] The human and VLM evaluations use only 50 examples each and do not include any statistical significance testing or confidence intervals. Given the small margins in the automated metrics, these results are too sparse to verify the central claim of superiority. Please increase the sample size or provide error bars and significance tests (e.g., a paired permutation test).
minor comments (8)
  1. [Sec. 4.3] The sentence 'where t_before, t_after ~ N(0,1)' is incorrect; the timesteps should be drawn uniformly, as correctly stated in Algorithm 1 (U(0,T)).
  2. [Sec. 4.3] The description of token consistency is misstated: 'the prompt concatenated with prompt_A and D~ should accurately reconstruct image A' contradicts Eq. (7); it should read 'the prompt concatenated with prompt_A' and -D~ should reconstruct A'.
  3. [Fig. 5] The configuration labels (Config. A-E) are not defined in the text or caption; please clarify which component is ablated in each column.
  4. [Sec. 5.3] The claim that the DINO-v2 score 'surpasses even SDEdit and InstructPix2Pix' rests on a margin of 0.0009 (Table 1), which is likely within noise; please add error bars or interpret this margin cautiously.
  5. [Sec. 6] The reference to a supplementary MNIST colorization analysis is not self-contained; please either include the figure or summarize the result in the main text.
  6. [Algorithm 1 and Sec. 4.3] The text states 'where lambda_tc and lambda_tc denotes weight parameter'; the second occurrence should be 'lambda_clip'.
  7. [Fig. 8] The text does not clarify whether the SDXL results use the same DIFF tokens optimized with SD2.1 or re-optimized for SDXL; please specify the protocol.
  8. [Abstract] The claim 'isolates only the difference' is an overstatement given the failure cases described in Sec. 6; please soften the wording.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training objective and evaluation metrics are not the same by construction, and independent DINO-v2, human, and VLM evaluations support the central claim.

full rationale

Difference Inversion optimizes DIFF tokens against an interpolated CLIP-space delta (Eq. 4-6) plus a bidirectional noise-prediction consistency loss (Eq. 7-10), but it does not fit the final B' output to the evaluation metrics. Table 1's CLIP directional score measures cosine similarity between A->A' and B->B' image deltas, which is not the L_clip objective cosine(D, D_inter); the generation of B' passes through a frozen Stable Diffusion sampler, so a high score is not forced by the training loss. DINO-v2 directional score, human evaluation (Table 2), and three VLM evaluations (Table 3) use embeddings and judgments outside the training space. The paper's own Sec. 6 limitation about CLIP-space failures is an honest external check, not a circular step. No load-bearing self-citation is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The central claim rests on 5 hand-tuned hyperparameters and 5 modeling assumptions about CLIP space, hard-prompt anchoring, zero initialization, and cycle consistency. The DIFF tokens and interpolated delta are internal constructs with no external validation other than the reported images and preference scores.

free parameters (5)
  • Alpha (Slerp interpolation ratio) = 0.8
    Blends image delta and text delta in Eq. 4-5; tuned by qualitative ablation in Fig. 7 on the evaluation setting.
  • Lambda_tc (token consistency weight) = 0.01
    Set in Sec. 4.3; balances cycle reconstruction against CLIP alignment; no separate validation described.
  • Lambda_clip (CLIP cosine weight) = 6
    Set in Sec. 4.3; drives DIFF tokens toward interpolated delta; tuned empirically.
  • Number of DIFF tokens = 5
    Chosen from ablation in Fig. 6; fewer tokens underfit, more tokens capture unrelated information.
  • Learning rate gamma = unspecified in main text
    Algorithm 1 uses gamma for token updates but the main text omits its value; affects optimization stability.
assumptions (5)
  • domain assumption CLIP embedding differences represent editable image differences.
    Eq. 2-3 define the Image and Text Deltas in CLIP space; Sec. 4.2 assumes this space can express the A-to-A' edit even though Sec. 6 notes CLIP's limitations.
  • ad hoc to paper Slerp between image and text deltas yields a more precise difference than either alone.
    Eq. 4-5 with alpha=0.8; no principled justification beyond CLIP cosine training, and the ratio is tuned.
  • domain assumption PEZ hard prompts fully anchor A and A' so the DIFF tokens contain only the delta.
    Sec. 4.1 freezes the optimized prompts; if anchoring is imperfect, residual image content leaks into the DIFF tokens.
  • domain assumption Zero initialization prevents unintended content from entering the difference tokens.
    Sec. 4.4, motivated by ControlNet; an empirical heuristic without formal guarantee.
  • domain assumption DDIM inversion and Stable Diffusion noise prediction provide a reliable cycle-consistency signal.
    The token consistency loss (Eq. 7-9) assumes that adding or removing DIFF tokens changes the predicted noise in a way that tracks the edit; this holds only approximately in practice.
invented entities (2)
  • DIFF tokens (D~)
    purpose: Trainable text embeddings that encode the A-to-A' difference and can be concatenated with any query image prompt.
    Central new object; its success is measured only through generated images and preference scores in this paper, with no external falsifiable prediction.
  • Interpolated Delta (D_inter)
    purpose: A Slerp-blended vector of image and text deltas used as the learning target for DIFF tokens.
    A constructed target in CLIP space; no independent handle outside the paper's own evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Difference Inversion: Interpolate and Isolate the Difference with Token Consistency for Image Analogy Generation." pith.science (2026). https://pith.science/paper/3TSXHBWG

@misc{pith2026250607750,
  author       = {Pith},
  title        = {Pith review of: Difference Inversion: Interpolate and Isolate the Difference with Token Consistency for Image Analogy Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3TSXHBWG}},
  note         = {Machine review of arXiv:2506.07750}
}
read the original abstract

How can we generate an image B' that satisfies A:A'::B:B', given the input images A,A' and B? Recent works have tackled this challenge through approaches like visual in-context learning or visual instruction. However, these methods are typically limited to specific models (e.g. InstructPix2Pix. Inpainting models) rather than general diffusion models (e.g. Stable Diffusion, SDXL). This dependency may lead to inherited biases or lower editing capabilities. In this paper, we propose Difference Inversion, a method that isolates only the difference from A and A' and applies it to B to generate a plausible B'. To address model dependency, it is crucial to structure prompts in the form of a "Full Prompt" suitable for input to stable diffusion models, rather than using an "Instruction Prompt". To this end, we accurately extract the Difference between A and A' and combine it with the prompt of B, enabling a plug-and-play application of the difference. To extract a precise difference, we first identify it through 1) Delta Interpolation. Additionally, to ensure accurate training, we propose the 2) Token Consistency Loss and 3) Zero Initialization of Token Embeddings. Our extensive experiments demonstrate that Difference Inversion outperforms existing baselines both quantitatively and qualitatively, indicating its ability to generate more feasible B' in a model-agnostic manner.

Figures

Figures reproduced from arXiv: 2506.07750 by the authors.

Figure 1
Figure 1. We propose Difference Inversion, a method that generates B ′ conditioned on an image triplet {A, A′ , B} that satisfies A : A ′ :: B : B ′ . Our method produces a significantly more plausible B ′ than other baselines. Note that SDEdit and InstructPix2Pix take the text instruction as input, whereas DIA, Analogist, VISII, and our Difference Inversion only use the image triplet A, A′ , B as input. Abstract How can we g… view at source ↗
Figure 2
Figure 2. Architectural comparison of image analogy gener￾ation baselines. We compare our approach with three baseline methods: (a) DIA, (b) Analogist, and (c) VISII. Detailed descrip￾tions can be found in Sec. 2. It is noteworthy that, unlike (b) and (c), which each depend on specific models, our method can be ap￾plied to general stable diffusion models. information emerges when the token is present and disap￾pears when it i… view at source ↗
Figure 3
Figure 3. Overall pipeline of Difference Inversion. Given an image triplet {A, A′ , B}, Difference Inversion extracts the difference between A and A ′ using DIFF tokens. The Difference is estimated as the Interpolated Delta, computed between A and A’ by applying Spherical Linear Interpolation (Slerp) on the Image Delta and Text Delta, effectively capturing both visual and semantic information. The circled numbers in the figur… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison to baseline methods. Given inputs A, A ′ and B, we generate B ′ without utilizing any text instructions. triplets in the form of {before image, after image, text in￾struction}. We utilized the before image and after image as A and A′ , respective…
Figure 5
Figure 5. Figure 5: Qualitative comparison with ablation studies for each proposed method. Each column shows the results when the cor￾responding method is not applied. Model DIA Analogist VISII Ours Percentage (%) 4% 15% 9% 72% [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 9
Figure 9. Figure 9: Failure cases. When the difference is complex or diffi￾cult to define precisely, Difference Inversion may struggle to cap￾ture it effectively. global differences (a) such as weather shifts or transforma￾tions like image-to-sketch cannot be effectively applied to B′ whe…
Figure 8
Figure 8. Figure 8: Transferability to other Stable Diffusion models. Our Difference Inversion can also be applied to other general Stable Diffusion models, such as SDXL. dation in consistency with the original image (B). Without Delta Interpolation and using only Image Delta, the method …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 21 canonical work pages

  1. [1]

    Conditional image genera- tion with score-based diffusion models.arXiv preprint arXiv:2111.13606, 2021

    Georgios Batzolis, Jan Stanczuk, Carola-Bibiane Sch ¨onlieb, and Christian Etmann. Conditional image genera- tion with score-based diffusion models.arXiv preprint arXiv:2111.13606, 2021. 2

    arXiv 2021

  2. [2]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 2, 3, 4

    work page 2023

  3. [3]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198, 2024. 5

    work page 2024

  4. [4]

    General image-to-image translation with one-shot image guidance

    Bin Cheng, Zuhao Liu, Yunbo Peng, and Yue Lin. General image-to-image translation with one-shot image guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22736–22746, 2023. 3

    work page 2023

  5. [5]

    Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 2, 3

    2021

  6. [6]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    arXiv

  7. [7]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. InForty-first International Conference on Machine Learn- ing. 2

    work page

  8. [8]

    An image is worth one word: Personalizing text-to- image generation using textual inversion.arXiv preprint arXiv:2208.01618, 2022

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion.arXiv preprint arXiv:2208.01618, 2022. 3, 4

    arXiv 2022

Show all 47 references
  1. [9]

    Photoswap: Personalized subject swapping in images.Advances in Neural Information Processing Sys- tems, 36, 2024

    Jing Gu, Yilin Wang, Nanxuan Zhao, Tsu-Jui Fu, Wei Xiong, Qing Liu, Zhifei Zhang, He Zhang, Jianming Zhang, Hyun- Joon Jung, et al. Photoswap: Personalized subject swapping in images.Advances in Neural Information Processing Sys- tems, 36, 2024. 3

    2024

  2. [10]

    Analogist: Out-of-the-box visual in-context learning with image diffusion model.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024

    Zheng Gu, Shiyuan Yang, Jing Liao, Jing Huo, and Yang Gao. Analogist: Out-of-the-box visual in-context learning with image diffusion model.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024. 2

    2024

  3. [11]

    Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022. 3

    2022 arXiv

  4. [12]

    Image analogies

    Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, and David H Salesin. Image analogies. InSem- inal Graphics Papers: Pushing the Boundaries, Volume 2, pages 557–570. 2023. 2

    2023

  5. [13]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 2

    2022 arXiv

  6. [14]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  7. [15]

    Reversion: Diffusion-based relation inversion from images.arXiv preprint arXiv:2303.13495, 2023

    Ziqi Huang, Tianxing Wu, Yuming Jiang, Kelvin CK Chan, and Ziwei Liu. Reversion: Diffusion-based relation inversion from images.arXiv preprint arXiv:2303.13495, 2023. 3

    2023 arXiv

  8. [16]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007–6017, 2023. 2

    2023

  9. [17]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1931–1941, 2023. 3

    1931

  10. [18]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InIn- ternational conference on machine learning, pages 19730– 19742. PMLR, 2023. 4

    2023

  11. [19]

    Visual attribute transfer through deep image analogy

    Jing Liao, Yuan Yao, Lu Yuan, Gang Hua, and Sing Bing Kang. Visual attribute transfer through deep image analogy. arXiv preprint arXiv:1705.01088, 2017. 2

    2017 arXiv

  12. [20]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 5

    2024

  13. [21]

    Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36, 2024. 5

    2024

  14. [22]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions.arXiv preprint arXiv:2108.01073, 2021

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions.arXiv preprint arXiv:2108.01073, 2021. 2

    2021 arXiv

  15. [23]

    Instructgie: Towards generalizable image editing

    Zichong Meng, Changdi Yang, Jun Liu, Hao Tang, Pu Zhao, and Yanzhi Wang. Instructgie: Towards generalizable image editing. InEuropean Conference on Computer Vision, pages 18–34. Springer, 2025. 2, 5

    2025

  16. [24]

    Null-text inversion for editing real im- ages using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real im- ages using guided diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038–6047, 2023. 3

    2023

  17. [25]

    Visual instruction inversion: Image editing via image prompting.Advances in Neural Information Processing Sys- tems, 36, 2024

    Thao Nguyen, Yuheng Li, Utkarsh Ojha, and Yong Jae Lee. Visual instruction inversion: Image editing via image prompting.Advances in Neural Information Processing Sys- tems, 36, 2024. 2, 3

    2024

  18. [26]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  19. [27]

    Openai: Introducing chatgpt, 2022

    OpenAI. Openai: Introducing chatgpt, 2022. 6, 7

    2022

  20. [28]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 7

    2023 arXiv

  21. [29]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 2

    2023 arXiv

  22. [30]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  23. [31]

    Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  24. [32]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3

    2022

  25. [33]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22500–...

    2023

  26. [34]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information ...

    2022

  27. [35]

    Animating rotation with quaternion curves

    Ken Shoemake. Animating rotation with quaternion curves. InProceedings of the 12th annual conference on Computer graphics and interactive techniques, pages 245–254, 1985. 5

    1985

  28. [36]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. InInternational confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 3

    2015

  29. [37]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020. 3

    2010 arXiv

  30. [38]

    Score-based generative modeling through stochastic differential equa- tions.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions.arXiv preprint arXiv:2011.13456, 2020. 3

    2011 arXiv

  31. [39]

    Diffusion image analo- gies

    Ad ´ela ˇSubrtov´a, Michal Luk ´aˇc, Jan ˇCech, David Futschik, Eli Shechtman, and Daniel S `ykora. Diffusion image analo- gies. InACM SIGGRAPH 2023 Conference Proceedings, pages 1–10, 2023. 2

    2023

  32. [40]

    p+: Extended textual conditioning in text-to- image generation.arXiv preprint arXiv:2303.09522, 2023

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. p+: Extended textual conditioning in text-to- image generation.arXiv preprint arXiv:2303.09522, 2023. 3

    2023 arXiv

  33. [41]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024. 6, 7

    2024 arXiv

  34. [42]

    Hard prompts made easy: Gradient-based discrete optimization for prompt tun- ing and discovery.Advances in Neural Information Process- ing Systems, 36, 2024

    Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tun- ing and discovery.Advances in Neural Information Process- ing Systems, 36, 2024. 3, 4

    2024

  35. [43]

    Cyclenet: Rethinking cycle consistency in text- guided diffusion for image manipulation.Advances in Neu- ral Information Processing Systems, 36, 2024

    Sihan Xu, Ziqiao Ma, Yidong Huang, Honglak Lee, and Joyce Chai. Cyclenet: Rethinking cycle consistency in text- guided diffusion for image manipulation.Advances in Neu- ral Information Processing Systems, 36, 2024. 5

    2024

  36. [44]

    xgen-mm (blip-3): A family of open large multimodal models.arXiv preprint arXiv:2408.08872, 2024

    Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yu- tong Dai, Michael S Ryoo, et al. xgen-mm (blip-3): A family of open large multimodal models.arXiv preprint arXiv:2408.08872, 2024. 5

    2024

  37. [45]

    Imagebrush: Learning visual in-context instructions for exemplar-based image ma- nipulation.Advances in Neural Information Processing Sys- tems, 36, 2024

    Yifan Yang, Houwen Peng, Yifei Shen, Yuqing Yang, Han Hu, Lili Qiu, Hideki Koike, et al. Imagebrush: Learning visual in-context instructions for exemplar-based image ma- nipulation.Advances in Neural Information Processing Sys- tems, 36, 2024. 2, 3, 5

    2024

  38. [46]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 5

    2023

  39. [47]

    Unpaired image-to-image translation using cycle- consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. InProceedings of the IEEE international conference on computer vision, pages 2223– 2232, 2017. 5

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.