Pith. sign in

REVIEW 3 major objections 5 minor 52 references

DECOR:Decomposition and Projection of Text Embeddings for Text-to-Image Customization

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that overfitting in LoRA-based text-to-image customization is caused by word-token text embeddings becoming entangled with reference images, and that a training-free orthogonal projection that strips out the word-token…

desk verdict A useful training-free trick for LoRA overfitting, but the paper's causal story about word-token subspaces is not yet backed by the right controls. read the letter →

arxiv 2412.09169 v1 pith:7MJDYFKJ submitted 2024-12-12 cs.CV

Geonhui Jang , Jin-Hwa Kim , Yong-Hyun Park , Junho Kim , Gayoung Lee , Yonghyun Jeong This is my paper · ORCID
classification cs.CV
keywords text-to-imagecustomizationLoRAfine-tuningoverfittingtextembeddingprojectionsingularvaluedecompositionpersonalizationstylizationcontent-stylemixing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fine-tuning a text-to-image model with LoRA on a handful of reference images often makes the model ignore the prompt and reproduce or leak the reference content. The paper identifies the cause on the text side: during fine-tuning, the word-token parts of the CLIP text embedding become entangled with the reference image, so the LoRA branch reads an overfitted conditioning signal. DECOR is a training-free remedy: it projects the text embedding onto the subspace spanned by the word-token embeddings, subtracts that component with a strength parameter α, and feeds only the cleaned embedding into the LoRA layers at inference. The paper's experiments report that this single geometric change improves prompt alignment and reduces content leakage across personalization, stylization, and content-style mixing while keeping identity and style, and that it combines with methods such as ControlNet and the DCO loss. If the paper is right, a large class of few-shot customization failures can be patched without retraining.

What carries the argument

The load-bearing object is the projection matrix $P_{X_w} = \tilde{V}\tilde{V}^T$, where $\tilde{V}$ comes from the SVD $\tilde{X} = \tilde{U}\tilde{\Sigma}\tilde{V}^T$ of the word-token embedding matrix $\tilde{X} = X_w$. It defines the 'undesired semantics' to suppress as the subspace spanned by the word-token vectors, and the adjusted embedding $X' = X - \alpha X P_{X_w}$ removes that subspace from the LoRA branch's input, with $\alpha \in [0,1]$ controlling how much is removed. The paper also uses a hierarchical SVD of the full embedding $X = U\Sigma V^T$ to argue that the word-token components, not the padding or noise components, carry the overfitting signal. The mechanism is a feed-time geometric edit, no training or weight change.

What would settle it

Train the same LoRA on a reference image, then generate with DECOR's word-token-subspace projection and with a projection onto a random subspace of the same dimension and norm. If the random projection reduces content leakage just as much, the claimed mechanism is wrong; if it does not, that would confirm the word-token subspace is the specific carrier. A second check is to ablate whether feeding the projected embedding to the base model as well, instead of only to the LoRA branch, destroys subject identity, which would test the paper's assumption that the base model should keep the original embedding.

Watch

Extended reading notes

Core claim

The central claim is that the overfitting seen in LoRA-based T2I customization is a text-conditioning problem, not merely a model-capacity problem. Using SVD to decompose the CLIP text embedding matrix, the paper finds that the first singular direction captures the padding tokens, a band of low-order components carries the word-token embeddings, and residual components behave like noise; when the word-token components are fed to the LoRA layers, they produce strong overfitting and distorted images. DECOR therefore removes from the input embedding its projection onto the word-token subspace, computing $X' = X - \alpha X P_{X_w}$ with $P_{X_w}$ built from the SVD of $X_w$, and feeds this modified embedding to the LoRA branch while the base model receives the original embedding. The paper's experiments report that this reduces prompt misalignment and content leakage in personalization, stylization, and content-style mixing, and that varying $\alpha$ trades off text alignment against visual similarity.

Load-bearing premise

The method works only if the overfitting signal is carried by the word-token part of the text embedding; if it lives in other parts of the embedding, or if stripping that subspace also erases essential identity or style information, the projection cannot deliver the reported gains.

Editorial extensions

If this is right

  • DECOR improves text alignment and reduces content leakage in LoRA-based personalization, stylization, and content-style mixing tasks.
  • The single scalar $\alpha$ gives users a continuous control between text fidelity and visual similarity, and in stylization it also controls how much fine style detail is kept.
  • The method is training-free and costs under a second at inference, so it can be layered onto existing LoRA pipelines without retraining.
  • DECOR can be combined with other components such as ControlNet and the DCO loss, and with different $\alpha$ values for content and style LoRAs when merging them.
  • Attention-map analysis shows that after projection, prompt tokens attend to the correct image regions, which the paper presents as evidence that the overfitting is a text-side attention problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same projection recipe could be applied to any conditioning signal that goes through a tokenizer, not just CLIP text in LoRA customization; if the entanglement mechanism generalizes, similar projections could help multi-concept generation and language-conditioned editing.
  • Editorial inference: the projection norm of a text embedding onto the word-token subspace could serve as a cheap, training-free diagnostic for overfitting; if it correlates with observed content leakage across prompts, it could guide $\alpha$ selection or early stopping.
  • Editorial inference: the paper's separation of $\alpha$ for content versus style LoRA in mixing suggests a general recipe for decomposing a merged adapter's conditioning into semantic axes, which might extend to other PEFT combinations beyond LoRA.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses overfitting in LoRA-based text-to-image customization, manifested as prompt misalignment and content leakage. It decomposes the CLIP text embedding matrix with SVD and claims that word-token embeddings become entangled with reference images, causing overfitting. The proposed method, DECOR, is a training-free inference-time modification that projects the text embedding fed to the LoRA branch onto a space orthogonal to the word-token subspace, controlled by a hyperparameter alpha. Experiments cover personalization, stylization, and content-style mixing, reporting improved text-alignment/identity trade-offs with qualitative comparisons against several baselines.

Significance. If the causal claim is correct, the paper identifies a simple, interpretable mechanism for overfitting in LoRA-based customization and provides a training-free remedy that can be plugged into existing pipelines. The empirical evaluation is broad: three tasks, multiple baselines, quantitative CLIP/DINO metrics, preference-model comparisons, and an ablation over alpha. The method is computationally lightweight, with an overhead of under a second, and the appendix documents integrations with ControlNet and DCO loss. However, the central mechanistic claim is not yet supported by controlled evidence: the SVD analysis and reconstruction experiments are qualitative, and the projection intervention is not compared with norm-matched or subspace-matched controls. The paper's value as an empirical method is plausible, but its advertised novelty rests on a causal analysis that currently remains a hypothesis.

major comments (3)
  1. [3.1, Fig. 3] The causal identification of word tokens as the primary locus of overfitting is not supported by controlled evidence. The analysis in Sec. 3.1 consists of an SVD decomposition and a qualitative reconstruction experiment with a few illustrative prompts; the 'subsequent components' (singular value indices 3–10%) are selected by rank, not by token identity, so the conclusion that word-token embeddings are entangled with reference images is an interpretation rather than a demonstrated mechanism. I recommend replacing or supplementing Fig. 3 with a quantitative attribution study (e.g., measuring text-alignment and identity metrics under ablation of specific component ranges, or comparing the effect of projecting out individual token subspaces) before the causal claim can be accepted.
  2. [3.2, Eq. (2), Fig. 5] The central intervention is not tested against norm-matched or subspace-matched controls, so the empirical gains in Figs. 8 and 11 are also consistent with the weaker hypothesis that reducing the LoRA branch's overall influence alleviates overfitting. For alpha in [0,1], the operation X' = X - alpha X P_{X_w} shrinks the component of every token along the word-token subspace and hence reduces the magnitude of the LoRA-branch input; a simple reduction of the LoRA adapter scale, or an isotropic scaling of the LoRA input, could produce a similar Pareto trade-off. The paper should ablate (i) a random subspace projection of the same rank, (ii) the [PAD]-token subspace, (iii) the special subject token alone, and (iv) a global LoRA scale reduction matched in output norm. Fig. 6 compares only zeroing the word-token rows and SVD truncation, neither of which is a norm-matched or subspace-matched control. Without these ablations, the specific geometry of X_w is not established as the operative factor.
  3. [4.4, Table A-1] The content-style mixing quantitative results are relegated to the appendix and reported only as aggregate mean±std scores; no per-pair breakdown or significance test is provided, and the best alpha combination is selected post hoc from 25 combinations (24 DECOR variants plus DreamBooth). This makes it difficult to assess whether the improvement over ZipLoRA (CLIP 0.305 vs 0.296; DINO 0.404 vs 0.297) is robust across the 96 subject-style pairs, especially given the variance magnitudes. I would like to see either a paired comparison with error bars on the differences or, at minimum, a report of the fraction of pairs where DECOR wins.
minor comments (5)
  1. [3.1, Fig. 2] The definitions of 'primary', 'subsequent', and 'residual' components appear in the caption before they are introduced in the body; the body should define them first.
  2. [3.1] The decision to omit the start-of-text token and to include padding tokens is stated but not justified; please explain the effect of this choice on the SVD analysis.
  3. [4.1] Because StyleDrop uses a different backbone and an unofficial reproduction, the stylization comparison should include a discussion of how this affects the interpretation of the quantitative results.
  4. [5] The sentence 'This process highlights reduces unwanted feature in the text embeddings' contains a grammatical error and should be rewritten.
  5. [References] Reference [7] cites a blog post with a placeholder URL (exampleblog.com); this should be replaced with a citable source or removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DECOR's projection is an empirical intervention, and the causal analysis is not equivalent to the method by construction.

full rationale

The paper's derivation chain is self-contained rather than circular. The causal claim is supported by an independent intervention in Sec. 3.1: the SVD component analysis first identifies a 'subsequent' component group whose token-level cosine similarity concentrates on word tokens (Fig. 2b), and then Fig. 3 shows that feeding this component group into the LoRA branch, scaled to match the original embedding, produces strong overfitting and distortion. That is an experimental intervention, not a definition. The proposed projection in Eq. (2)/(3) uses P_{X_w}, the projection onto the row space of the word-token rows X_w, which is computed directly from the token rows and is not the same object as the SVD 'subsequent' components used in the diagnosis. Success of DECOR is therefore not the sole evidence for the word-token-entanglement claim; the component-amplification experiment provides independent, if imperfectly controlled, evidence. No fitted parameter is relabeled as a prediction: alpha is an explicit hyperparameter whose sweep generates the reported Pareto frontier. The only overlapping-author citation ([15], Visual Style Prompting, by co-authors Junho Kim and Gayoung Lee) is used as a comparison baseline and for a remark about StyleAligned's text alignment; it is not load-bearing for the central projection claim. Missing controls, such as random-subspace projection or equivalent LoRA-scale reduction, would test whether X_w is the unique mechanism, but that is an evidence/completeness limitation, not circularity. The appended limitation section (Fig. A-8) honestly acknowledges style-detail loss, which is a scope statement rather than a circular step. Accordingly, no circular step can be quoted with a by-construction reduction, and the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central method has essentially one free parameter (alpha) and rests on three key assumptions: the SVD component taxonomy, the suitability of X_w as the suppression subspace, and the claim that overfitting is primarily a text-conditioning phenomenon. No new entities are introduced beyond the projection operation itself.

free parameters (1)
  • alpha (projection strength) = 0.8 for personalization and stylization; 0.25 for content LoRA and 1.0 for style LoRA in content-style mixing
    alpha controls the degree of removal of the projected word-token component in Eq. (2). The paper sets alpha per task to values that 'provide the best results' (Sec. 4.1), indicating it is tuned on the evaluation data rather than fixed a priori.
assumptions (3)
  • domain assumption The CLIP text embedding matrix X can be meaningfully decomposed by SVD into primary (PAD), subsequent (word-token), and residual (noise) components, with the subsequent components carrying the word-token information.
    This is established in Sec. 3.1 via Fig. 2, which reports cosine similarity patterns for 20 prompts of similar length. It is an empirical observation about CLIP embeddings, not a proven theorem, and it underlies the choice of the projection target.
  • domain assumption The word-token subspace X_w is the correct subspace to suppress for reducing overfitting, and removing its component from the embedding fed to LoRA does not destroy the identity or style information needed for customization.
    This is the core design choice in Sec. 3.2 (Eq. (2) with tilde X = X_w). The paper supports it with the experiment in Fig. 3, where feeding reconstructed 'subsequent components' into LoRA induces overfitting, but this is not a controlled comparison against other subspaces, and the identity-preservation property is only implicitly evaluated via DINO similarity.
  • domain assumption Overfitting in LoRA-based customization manifests primarily through the text-conditioning branch, so that supplying the original embedding to the base cross-attention and the projected embedding to the LoRA branch is sufficient to reduce prompt misalignment and content leakage.
    This is assumed in the pipeline shown in Fig. 5(b), where the base weight W0 receives the original embedding and only the LoRA branch receives X'. The paper does not ablate alternate placements of the projection, so the claim that the overfitting signal lives in the LoRA branch is an unverified premise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DECOR:Decomposition and Projection of Text Embeddings for Text-to-Image Customization." pith.science (2026). https://pith.science/paper/7MJDYFKJ

@misc{pith2026241209169,
  author       = {Pith},
  title        = {Pith review of: DECOR:Decomposition and Projection of Text Embeddings for Text-to-Image Customization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MJDYFKJ}},
  note         = {Machine review of arXiv:2412.09169}
}
read the original abstract

Text-to-image (T2I) models can effectively capture the content or style of reference images to perform high-quality customization. A representative technique for this is fine-tuning using low-rank adaptations (LoRA), which enables efficient model customization with reference images. However, fine-tuning with a limited number of reference images often leads to overfitting, resulting in issues such as prompt misalignment or content leakage. These issues prevent the model from accurately following the input prompt or generating undesired objects during inference. To address this problem, we examine the text embeddings that guide the diffusion model during inference. This study decomposes the text embedding matrix and conducts a component analysis to understand the embedding space geometry and identify the cause of overfitting. Based on this, we propose DECOR, which projects text embeddings onto a vector space orthogonal to undesired token vectors, thereby reducing the influence of unwanted semantics in the text embeddings. Experimental results demonstrate that DECOR outperforms state-of-the-art customization models and achieves Pareto frontier performance across text and visual alignment evaluation metrics. Furthermore, it generates images more faithful to the input prompts, showcasing its effectiveness in addressing overfitting and enhancing text-to-image customization.

Figures

Figures reproduced from arXiv: 2412.09169 by the authors.

Figure 1
Figure 1. Our DECOR improves generation quality across personalization, stylization, and content-style mixing customization tasks. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Customization results with the original embeddings [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 2
Figure 2. (a) The CLIP text embeddings have a large first singular [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: When the components along the axis of the unwanted [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 6
Figure 6. Figure 6: Simply removing the word embedding Xw from the original embedding or using an embedding reconstructed without the subsequent components cannot solve the overfitting problem. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]
Figure 7
Figure 7. Figure 7: Qualitative personalization comparison. 4.1. Experimental setup Dataset. For the personalization experiments, we used a subset of the DreamBooth dataset [37], consisting of 12 subjects. For each subject, we trained the LoRA layers us￾ing 4–5 reference images. In the st…
Figure 8
Figure 8. Figure 8: Quantitative personalization comparison. Our method [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Personalization results using realistic templates. [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 11
Figure 11. Figure 11: Quantitative stylization comparison. In contrast to [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 10
Figure 10. Figure 10: Qualitative stylization comparison. to the Gaussian noise during training LoRA layers helps capture flat stylistic textures, which is suitable for creating illustrative styles. Therefore, we applied this noise offset technique, setting the offset scale to 0.1 for both…
Figure 13
Figure 13. Figure 13: Ablation study on α. In the stylization task, varying α reveals two key effects: preventing overfitting, such as content leakage (top), and controlling fine style components (bottom). Dream - Booth coffee mug Fau- -vism A painting of a coffee mug in fauvism art style …
Figure 12
Figure 12. Figure 12: Qualitative content-style mixing comparison with [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 19 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 3

    arXiv 2023

  2. [2]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 6

    2021

  3. [3]

    Muse: Text- to-image generation via masked generative transformers.Pro- ceedings of the 40th International Conference on Machine Learning, 202, 2023

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text- to-image generation via masked generative transformers.Pro- ceedings of the 40th International Conference on Machine Learning, 202, 2023. 2

    work page 2023

  4. [4]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 5

    arXiv 2010

  5. [5]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 2, 6

    arXiv 2022

  6. [6]

    Semantic projection: recovering human knowledge of multiple, distinct object features from word embeddings

    Gabriel Grand, Idan Asher Blank, Francisco Pereira, and Evelina Fedorenko. Semantic projection: recovering human knowledge of multiple, distinct object features from word embeddings. arXiv preprint arXiv:1802.01241, 2018. 4

    work page Pith review arXiv 2018

  7. [7]

    Diffusion with offset noise

    Nicholas Guttenberg. Diffusion with offset noise. https: //exampleblog.com/diffusion-offset-noise , 2023. Blog post. 6

    work page 2023

  8. [8]

    Svdiff: Compact parame- ter space for diffusion fine-tuning

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dim- itris Metaxas, and Feng Yang. Svdiff: Compact parame- ter space for diffusion fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7323–7334, 2023. 2

    work page 2023

Show all 52 references
  1. [9]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 1

    2022 arXiv

  2. [10]

    Style aligned image generation via shared at- tention

    Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared at- tention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4775–4785,

  3. [11]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 3

    2022 arXiv

  4. [12]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 6, 7

    2020

  5. [13]

    Parameter-efficient transfer learning for nlp

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. PMLR, 2019. 2

    2019

  6. [14]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 2

    2021 arXiv

  7. [15]

    Visual style prompting with swapping self- attention

    Jaeseok Jeong, Junho Kim, Yunjey Choi, Gayoung Lee, and Youngjung Uh. Visual style prompting with swapping self- attention. arXiv preprint arXiv:2402.12974, 2024. 2, 6, 7

    2024 arXiv

  8. [16]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:36652–36663,

  9. [17]

    The singular value decompo- sition: Its computation and some applications

    Virginia Klema and Alan Laub. The singular value decompo- sition: Its computation and some applications. IEEE Trans- actions on automatic control, 25(2):164–176, 1980. 3

    1980

  10. [18]

    Multi-concept customization of text- to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shecht- man, and Jun-Yan Zhu. Multi-concept customization of text- to-image diffusion. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1931–1941, 2023. 2, 6

    1931

  11. [19]

    Direct consistency optimization for compositional text- to-image personalization

    Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, and Jinwoo Shin. Direct consistency optimization for compositional text- to-image personalization. arXiv preprint arXiv:2402.12004,

    arXiv

  12. [20]

    The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021. 2

    2021 arXiv

  13. [21]

    Get what you want, not what you don’t: Image content suppression for text- to-image diffusion models.arXiv preprint arXiv:2402.05375,

    Senmao Li, Joost van de Weijer, Taihang Hu, Fahad Shahbaz Khan, Qibin Hou, Yaxing Wang, and Jian Yang. Get what you want, not what you don’t: Image content suppression for text- to-image diffusion models.arXiv preprint arXiv:2402.05375,

    arXiv

  14. [22]

    P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2021. 2

    arXiv

  15. [23]

    Gpt understands, too

    Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. AI Open, 2023. 2

    2023

  16. [24]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Be- linkov. Locating and editing factual associations in gpt. Ad- vances in Neural Information Processing Systems, 35:17359– 17372, 2022. 3

    2022

  17. [25]

    Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models

    Daiki Miyake, Akihiro Iohara, Yu Saito, and Toshiyuki Tanaka. Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models. arXiv preprint arXiv:2305.16807, 2023. 7, 1

    2023 arXiv

  18. [26]

    Null-text inversion for editing real im- ages using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real im- ages using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038–6047, 2023. 7, 1

    2023

  19. [27]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 6

    2023 arXiv

  20. [28]

    Moa: Mixture-of-attention for subject-context 9 disentanglement in personalized image generation

    Daniil Ostashev, Yuwei Fang, Sergey Tulyakov, Kfir Aber- man, et al. Moa: Mixture-of-attention for subject-context 9 disentanglement in personalized image generation. arXiv preprint arXiv:2404.11565, 2024. 3

    2024 arXiv

  21. [29]

    amused: An open muse reproduction

    Suraj Patil, William Berman, Robin Rombach, and Patrick von Platen. amused: An open muse reproduction. arXiv preprint arXiv:2401.01808, 2024. 5

    2024 arXiv

  22. [30]

    Sdxl: Improving latent diffusion models for high- resolution image synthesis.arXiv preprint arXiv:2307.01952,

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨ uller, Joe Penna, and Robin Rom- bach. Sdxl: Improving latent diffusion models for high- resolution image synthesis.arXiv preprint arXiv:2307.01952,

    arXiv

  23. [31]

    Feature projection for im- proved text classification

    Qi Qin, Wenpeng Hu, and Bing Liu. Feature projection for im- proved text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 8161–8171, 2020. 4

    2020

  24. [32]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  25. [33]

    Direct prefer- ence optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct prefer- ence optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 4

    2024

  26. [34]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning , pages 8821–8831. Pmlr, 2021. 2

    2021

  27. [35]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  28. [36]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  29. [37]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages ...

  30. [38]

    Photorealistic text-to-image diffusion models with deep lan- guage understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep lan- guage understanding. Advances in neural informati...

    2022

  31. [39]

    Zur theorie der linearen und nichtlinearen in- tegralgleichungen

    Erhard Schmidt. Zur theorie der linearen und nichtlinearen in- tegralgleichungen. Mathematische Annalen, 63(4):433–476,

  32. [40]

    Ziplora: Any subject in any style by effectively merging loras

    Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svetlana Lazebnik, Yuanzhen Li, and Varun Jampani. Ziplora: Any subject in any style by effectively merging loras. InEuropean Conference on Computer Vision , pages 422–438. Springer,

  33. [41]

    Ziplora-pytorch

    Makoto Shing. Ziplora-pytorch. https://github.com/ mkshing/ziplora-pytorch, 2023. GitHub repository. 7

    2023

  34. [42]

    Styledrop: Text-to-image genera- tion in any style

    Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. Styledrop: Text-to-image genera- tion in any style. arXiv preprint arXiv:2306.00983, 2023. 2, 5

    2023 arXiv

  35. [43]

    Measuring style similarity in diffusion models

    Gowthami Somepalli, Anubhav Gupta, Kamal Gupta, Shra- may Palta, Micah Goldblum, Jonas Geiping, Abhinav Shri- vastava, and Tom Goldstein. Measuring style similarity in diffusion models. arXiv preprint arXiv:2404.01292, 2024. 4

    2024 arXiv

  36. [44]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502,

    2010 arXiv

  37. [45]

    Key-locked rank one editing for text-to-image personaliza- tion

    Yoad Tewel, Rinon Gal, Gal Chechik, and Yuval Atzmon. Key-locked rank one editing for text-to-image personaliza- tion. In ACM SIGGRAPH 2023 Conference Proceedings , pages 1–11, 2023. 3

    2023

  38. [46]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  39. [47]

    Fastcomposer: Tuning-free multi- subject image generation with localized attention

    Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. arXiv preprint arXiv:2305.10431, 2023. 2

    2023 arXiv

  40. [48]

    Break-for-make: Modular low-rank adaptations for composable content-style customization

    Yu Xu, Fan Tang, Juan Cao, Yuxin Zhang, Oliver Deussen, Weiming Dong, Jintao Li, and Tong-Yee Lee. Break-for-make: Modular low-rank adaptations for composable content-style customization. arXiv preprint arXiv:2403.19456, 2024. 2

    2024 arXiv

  41. [49]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721 ,

    arXiv

  42. [50]

    Infusion: Preventing customized text-to-image diffusion from overfitting

    Weili Zeng, Yichao Yan, Qi Zhu, Zhuo Chen, Pengzhi Chu, Weiming Zhao, and Xiaokang Yang. Infusion: Preventing customized text-to-image diffusion from overfitting. arXiv preprint arXiv:2404.14007, 2024. 3

    2024 arXiv

  43. [51]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InPro- ceedings of the IEEE/CVF International Conference on Com- puter Vision, pages 3836–3847, 2023. 3

    2023

  44. [52]

    in origami style

    Chenyi Zhuang, Ying Hu, and Pan Gao. Magnet: We never know how text-to-image diffusion models work, until we learn how vision-language models function. arXiv preprint arXiv:2409.19967, 2024. 4 10 DECOR: Decomposition and Projection of Text Embeddings for Text-to-Image Customiz...

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.