Pith. sign in

REVIEW 4 major objections 6 minor 69 references

Identity-Preserving Text-to-Image Generation via Dual-Level Feature Decoupling and Expert-Guided Fusion

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-module training recipe separates subject identity from background and improves identity preservation in subject-driven text-to-image generation, reporting CLIP-I 0.789, DINO 0.546, and CLIP-T 0.260 on the DreamBooth benchmark.

desk verdict Decent incremental recipe for identity-preserving T2I, but Eq. 6 contradicts the text and figure on which features enter the fusion module, so the paper as written cannot be faithfully reproduced. read the letter →

arxiv 2505.22360 v1 pith:4LK6GCD7 submitted 2025-05-28 cs.CV

Kewen Chen , Xiaobin Hu , Wenqi Ren This is my paper
classification cs.CV
keywords subject-driventext-to-imagegenerationidentitypreservationfeaturedecouplingforeground-backgroundseparationmixtureofexpertsdiffusionmodelsimageinpaintingparameter-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that subject-driven text-to-image generation fails when identity-relevant and identity-irrelevant information stay entangled, and that a two-module training recipe can separate them. The Implicit-Explicit Foreground-Background Decoupling Module (IEDM) extracts background features twice, once through a learnable adapter and once through segmentation plus inpainting, and the Feature Fusion Module (FFM) recombines the two streams with a Mixture-of-Experts gate. On the DreamBooth benchmark this recipe reports the best identity preservation among the compared fine-tuning methods (CLIP-I 0.789, DINO 0.546) with competitive text alignment (CLIP-T 0.260) and only 9.8 MB of extra parameters.

What carries the argument

The load-bearing object is the Implicit-Explicit Foreground-Background Decoupling Module (IEDM) paired with a Mixture-of-Experts Feature Fusion Module (FFM). IEDM builds two feature streams: an implicit one from a learnable-mask adapter over CLIP image features, and an explicit one by segmenting the subject, inpainting the background, and re-encoding it. Three cosine losses align the implicit background stream $f_i$ with the explicit background encoding $f'_i$ and separate both from the text-derived identity feature $f_s$. FFM forms $f_{com}=f_s+f'_i$ and computes $f_r = \sum_{k} R(f_{com})_k\,\mathrm{Expert}_k(f_{com})$, so a gating network reweights foreground and background information and can suppress residual leakage when decoupling is imperfect. This converts 'separate identity from context' into measurable feature streams with a learnable tolerance for imperfect separation.

What would settle it

Compare training runs on the same subjects with predicted masks versus manually corrected oracle masks: if CLIP-I or DINO does not drop when masks are corrupted (or does not recover with oracle masks), the explicit decoupling branch is not doing the claimed work; likewise, a subject whose segmentation fails (e.g., a chameleon on foliage) should show visible background artifacts or identity loss in generated images.

Watch

Extended reading notes

Core claim

The paper argues that existing customization methods either overfit to the reference images or lose the subject because they never cleanly separate what the subject is from where the subject is. Its discovery is that a dual-level decoupling signal fixes this: an implicit adapter on CLIP image features learns identity-irrelevant content, while an explicit segmentation-and-inpainting path provides a background-only image whose encoding acts as a clean target; three contrastive losses tie these streams to the text feature. A gated Mixture-of-Experts fusion then recombines foreground text features with background features, so even residual leakage from imperfect masks is down-weighted. With LoRA on cross-attention and text-encoder layers, the method reports CLIP-I 0.789 and DINO 0.546 on DreamBooth, ahead of the compared baselines, while holding text alignment at CLIP-T 0.260.

Load-bearing premise

The segmentation model reliably finds the subject and the inpainting model rebuilds a clean background, so the background-only image really contains no subject pixels; if the mask is wrong or inpainting leaks identity, the contrastive losses train on contaminated signals.

Editorial extensions

If this is right

  • Identity preservation on the DreamBooth benchmark improves over all compared fine-tuning baselines (CLIP-I 0.789, DINO 0.546).
  • Scene-changing prompts no longer drag in training-set backgrounds, because the background is explicitly removed and then re-added through the fusion gate.
  • The MoE gate gives robustness to imperfect decoupling: residual foreground information in the background stream can be down-weighted instead of corrupting the subject.
  • The three complementary losses are all needed; ablations show performance degrades progressively as they are removed, with L2 (separating $f_i$ from $f_s$) the most influential.
  • The whole customization remains parameter-efficient, adding 9.8 MB of tuned parameters rather than a full model copy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The explicit decoupling branch is only as good as the mask and inpainting; a direct test is to train with oracle masks on hard subjects and compare CLIP-I/DINO to the paper's reported numbers.
  • Editorial inference: Since FFM consumes $f'_i$ in $f_{com}$, while the adapter output $f_i$ is trained to match $f'_i$, an ablation swapping these two background features would show whether the implicit or explicit path is the one that matters.
  • Editorial inference: The three losses form a triangle that a triplet loss could collapse into one term; matching performance with a single loss would indicate the fusion module, not the loss design, drives the gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a training framework for subject-driven text-to-image generation built on Stable Diffusion v2.1. It introduces an Implicit-Explicit Foreground-Background Decoupling Module (IEDM) that combines a learnable adapter on CLIP image features with a segmentation/inpainting pipeline to obtain identity-irrelevant background features, and a Mixture-of-Experts-based Feature Fusion Module (FFM) that fuses these background features with text-derived identity features. Three auxiliary contrastive losses are added to the standard diffusion reconstruction loss, and LoRA is applied to the cross-attention layers of the U-Net and text encoder. The paper reports improved CLIP-I, DINO, and CLIP-T scores over Textual Inversion, DreamBooth, AttnDreamBooth, DisenBooth, and TextBoost on the DreamBooth dataset, with a total storage cost of 9.8 MB.

Significance. The paper addresses a meaningful problem in subject-driven customization, and the idea of using inpainting to produce explicit background supervision is a creative and potentially useful contribution. The ablation study is reasonably comprehensive, and the DINO improvements provide some evidence that identity gains are not entirely an artifact of CLIP-space training. However, the internal inconsistency between Eq. (6) and the architecture description in Section 3.1/Figure 2, together with the missing value of the hyperparameter λ4, currently prevent faithful reproduction and undermine attribution of the reported gains to the described method. With these issues corrected, the method could be a solid contribution; as written, the central quantitative claim is not fully supported.

major comments (4)
  1. [Section 3.3, Eq. (6)] The definition fcom = fs + f'_i is inconsistent with Section 3.1 and Figure 2, which state that the adapter output fi is combined with fs and fed into the FFM. If Eq. (6) is literal, then fi never enters the conditioning path of the U-Net; the implicit decoupling branch participates only through the auxiliary losses L2–L4, and the reported gains cannot be attributed to the 'implicit-explicit' fusion described in the abstract and Section 3.1. If Eq. (6) is a typo and fcom should be fs + fi, then the paper as written is not reproducible. Moreover, Section 3.4 states that at inference only text features are used; the authors should explicitly state whether the IEDM and FFM are training-only modules and, if so, what exactly the 9.8 MB storage contains. Please correct the equation, the text, and the figure to describe one consistent architecture and clarify the inference-time use of each module.
  2. [Section 4.1, Implementation details] The total loss in Eq. (8) contains four weights λ1–λ4, but the text specifies values only for λ1, λ2, and λ3 (all set to 0.001). The missing value of λ4 is needed for exact replication and for interpreting the ablation in Table 3, since changing λ4 alone could alter the contribution of the explicit-decoupling losses. Please report λ4.
  3. [Section 4.2, Table 1 and Section 4.3, Tables 2–3] CLIP-I is computed with CLIP embeddings, and the auxiliary training losses L2–L4 in Eqs. (2)–(5) are cosine similarities in the same CLIP feature space. Although these losses do not directly optimize the CLIP-I metric, the alignment between training objective and evaluation metric weakens the evidential value of the CLIP-I improvements. The DINO score is an independent metric and its improvement (e.g., 0.522→0.546 in Table 2) is a genuine strength, but the paper should report confidence intervals or per-subject standard deviations, since the differences in Table 1 (e.g., CLIP-I 0.789 vs 0.778) appear small and may not be statistically significant.
  4. [Section 3.2, Eq. (3)] The explicit decoupling branch assumes that EfficientViT-SAM produces accurate masks and that LaMa reliably inpaints the subject region, so that f'_i encodes only identity-irrelevant background. Erroneous masks or inpainted regions that leak subject pixels will contaminate f'_i and cause L3 and L4 to train the adapter and text embedding with misleading signals, directly affecting the identity-preservation claim. The paper provides no failure-case analysis or mask-quality evaluation. Please add a qualitative and quantitative analysis of mask/inpainting quality, and, if possible, an ablation with ground-truth masks to bound the sensitivity of the method.
minor comments (6)
  1. [Section 4.1, Baseline] The baseline list says 'Disenbooth [47]', but DisenBooth is reference [8]; the current citation points to DreamBooth [47].
  2. [Figure 2 and Section 4.1] Figure 2 illustrates three experts in the FFM, but Section 4.1 states that the number of experts is 2; please make the figure and text consistent.
  3. [Section 4.2] The text says 'storage storage capacity'; this contains a duplicated word and should be corrected.
  4. [Table 3] Labels such as 'w/o L2 + L3 + L4' are ambiguous; a clearer label such as 'without all three auxiliary losses' would make the ablation design easier to interpret.
  5. [Section 3.4] The paper refers to L2–L4 as 'three complementary loss functions' even though L1 is also part of the total objective; consider calling them 'three auxiliary losses' for clarity.
  6. [References] Reference [37] contains a typographical spacing error ('Huy V . V o'), and several other reference entries would benefit from proofreading.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; reported identity gains are not forced by construction and are corroborated by an independent DINO metric.

full rationale

The paper's central empirical claim (CLIP-I 0.789, DINO 0.546 on DreamBooth, Table 1) is not equivalent to its training inputs. The auxiliary losses in Eqs. 2, 4, and 5 operate on CLIP embeddings of the text prompt, the adapter output, and the inpainted background, but CLIP-I is measured on generated images versus real images and is never directly optimized; the independent DINO metric also improves (Table 1 and Tables 2-3), so the identity gain cannot be reduced to an artifact of training in the evaluation space. The only author-overlapping citations are background mentions of AnyMaker [28] and Diffumatting [23] in related work; neither is load-bearing and no self-cited uniqueness theorem or ansatz is invoked. The FFM definition in Eq. 6 using f'_i rather than fi, the inference description using only text features, and the unspecified lambda_4 in Section 4.1 are internal-consistency or reproducibility gaps, not circular reductions. The method is evaluated on fixed DreamBooth prompts without fitting test-set parameters, and ablations show monotonic contributions of the proposed modules. Therefore no circularity step meets the quoted-equation standard.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method rests on standard diffusion fine-tuning plus a set of hand-chosen hyperparameters and the reliability of off-the-shelf segmentation and inpainting models. No new physical entities are introduced.

free parameters (5)
  • loss weights λ1, λ2, λ3, λ4 = λ1=λ2=λ3=0.001, λ4 unspecified
    Set by hand with no sensitivity analysis; the total loss (Eq. 8) balances a diffusion loss with three contrastive terms.
  • number of MoE experts k = 2
    Chosen without ablation; the gating sums to 1 (Eq. 6).
  • LoRA rank = 4
    Applied to UNet cross-attention and text encoder; no sweep reported.
  • learning rates = 5e-4 (main), 1e-3 (embedding)
    Taken from prior work [38]; no tuning reported.
  • training epochs = 250
    Fixed; no early stopping or convergence analysis.
assumptions (4)
  • domain assumption CLIP embedding space supports clean separation of identity-related and identity-irrelevant features via cosine contrastive losses
    L2-L4 (Eqs. 2, 4, 5) assume cosine similarity in CLIP space sufficiently isolates identity from background.
  • domain assumption The segmentation and inpainting models produce reliable foreground/background separation
    Eq. 3 uses EfficientViT-SAM [68] and LaMa [57] to generate x'_i; failures corrupt the supervision signal.
  • domain assumption Fine-tuning only cross-attention LoRA and the text encoder captures the identity
    Section 3.4, based on findings of [30,39], not independently validated here.
  • domain assumption Stable Diffusion V2.1 is an adequate base for the task
    Fixed backbone; no evidence of transfer to other diffusion models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identity-Preserving Text-to-Image Generation via Dual-Level Feature Decoupling and Expert-Guided Fusion." pith.science (2026). https://pith.science/paper/4LK6GCD7

@misc{pith2026250522360,
  author       = {Pith},
  title        = {Pith review of: Identity-Preserving Text-to-Image Generation via Dual-Level Feature Decoupling and Expert-Guided Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4LK6GCD7}},
  note         = {Machine review of arXiv:2505.22360}
}
read the original abstract

Recent advances in large-scale text-to-image generation models have led to a surge in subject-driven text-to-image generation, which aims to produce customized images that align with textual descriptions while preserving the identity of specific subjects. Despite significant progress, current methods struggle to disentangle identity-relevant information from identity-irrelevant details in the input images, resulting in overfitting or failure to maintain subject identity. In this work, we propose a novel framework that improves the separation of identity-related and identity-unrelated features and introduces an innovative feature fusion mechanism to improve the quality and text alignment of generated images. Our framework consists of two key components: an Implicit-Explicit foreground-background Decoupling Module (IEDM) and a Feature Fusion Module (FFM) based on a Mixture of Experts (MoE). IEDM combines learnable adapters for implicit decoupling at the feature level with inpainting techniques for explicit foreground-background separation at the image level. FFM dynamically integrates identity-irrelevant features with identity-related features, enabling refined feature representations even in cases of incomplete decoupling. In addition, we introduce three complementary loss functions to guide the decoupling process. Extensive experiments demonstrate the effectiveness of our proposed method in enhancing image generation quality, improving flexibility in scene adaptation, and increasing the diversity of generated outputs across various textual descriptions.

Figures

Figures reproduced from arXiv: 2505.22360 by the authors.

Figure 1
Figure 1. Example images generated by our proposed method. Our approach produces high-quality images that maintain identity consistency while aligning with the input text prompts. foreground-background Decoupling Module (IEDM) and a Feature Fusion Module (FFM) based on a Mixture of Experts (MoE) model. Specifically, the IEDM employs a learnable adapter at the feature level to extract identity￾irrelevant features, achieving im… view at source ↗
Figure 2
Figure 2. Overview of our proposed method. The framework consists of the Implicit-Explicit foreground-background Decoupling Module (IEDM) for separating identity-related and identity-irrelevant features, and the Mixture of Experts (MoE)-based Feature Fusion Module (FFM) for refining the combined feature representations. The process begins with a text prompt that generates identity-related features, followed by dual-level deco… view at source ↗
Figure 3
Figure 3. Qualitative result. We compared our approach with current state-of-the-art methods, including Textual Inversion, DreamBooth, AttnDreamBooth, DisenBooth, and TextBoost, on the Dreambooth dataset. Our method demonstrates outstanding performance across multiple objects and animals, generating high-quality images with strong identity preservation and text alignment. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of Ablation results. We applied the prompt ”a photo of a V* stuffed animal in the snow” to the specific subject ”bear plushie.”, illustrating the impact of different compo￾nents of our proposed method. Abalation on ours proposed module. To gain deeper in￾…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 36 canonical work pages

  1. [1]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18208–18218, 2022. 2

    work page 2022

  2. [2]

    ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 2

    arXiv 2022

  3. [3]

    Multimodal garment designer: Human-centric latent diffusion models for fashion image editing

    Alberto Baldrati, Davide Morelli, Giuseppe Cartella, Mar- cella Cornia, Marco Bertini, and Rita Cucchiara. Multimodal garment designer: Human-centric latent diffusion models for fashion image editing. In Proceedings of the IEEE/CVF international conference on computer vision , pages 23393– 23402, 2023. 2

    work page 2023

  4. [4]

    Text2live: Text-driven layered image and video editing

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kas- ten, and Tali Dekel. Text2live: Text-driven layered image and video editing. In European conference on computer vi- sion, pages 707–723. Springer, 2022. 2

    work page 2022

  5. [5]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 2

    2023

  6. [6]

    A survey on mixture of experts

    Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. A survey on mixture of experts. CoRR, abs/2407.06204, 2024. 3

    arXiv 2024

  7. [7]

    Decoupled textual embeddings for cus- tomized image generation

    Yufei Cai, Yuxiang Wei, Zhilong Ji, Jinfeng Bai, Hu Han, and Wangmeng Zuo. Decoupled textual embeddings for cus- tomized image generation. In Proceedings of the AAAI Con- ference on Artificial Intelligence , pages 909–917, 2024. 1, 3

    work page 2024

  8. [8]

    Disenbooth: Identity- preserving disentangled tuning for subject-driven text-to- image generation

    Hong Chen, Yipeng Zhang, Simin Wu, Xin Wang, Xuguang Duan, Yuwei Zhou, and Wenwu Zhu. Disenbooth: Identity- preserving disentangled tuning for subject-driven text-to- image generation. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. 1, 3, 7, 8

    work page 2024

Show all 69 references
  1. [9]

    Towards aligned layout generation via diffusion model with aesthetic constraints

    Jian Chen, Ruiyi Zhang, Yufan Zhou, and Changyou Chen. Towards aligned layout generation via diffusion model with aesthetic constraints. In The Twelfth International Confer- ence on Learning Representations, ICLR 2024, Vienna, Aus- tria, May 7-11, 2024. OpenReview.net, 2024. 2

    2024

  2. [10]

    Anydoor: Zero-shot object-level im- age customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level im- age customization. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 6593–6602, 2024. 3

    2024

  3. [11]

    Diffu- sion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Quinn Nichol. Diffu- sion models beat gans on image synthesis. In Advances in Neural Information Processing Systems 34: Annual Con- ference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 8780– 8...

    2021

  4. [12]

    Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fe- dus, Maarten P

    Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fe- dus, Maarten P. Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kath- leen S. ...

    2022

  5. [13]

    Learning factored representations in a deep mixture of ex- perts

    David Eigen, Marc’Aurelio Ranzato, and Ilya Sutskever. Learning factored representations in a deep mixture of ex- perts. In 2nd International Conference on Learning Rep- resentations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Workshop Track Proceedings, 2014. 3

    2014

  6. [14]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first international conference on mach...

  7. [15]

    Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity. J. Mach. Learn. Res. , 23:120:1– 120:39, 2022. 3, 5

    2022

  8. [16]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh Interna- tional Conference on Learning Representations, ICLR 20...

    2023

  9. [17]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 2

    2020

  10. [18]

    Svdiff: Compact param- eter space for diffusion fine-tuning

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdiff: Compact param- eter space for diffusion fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7323–7334, 2023. 1

    2023

  11. [19]

    Fastermoe: modeling and optimizing training of large-scale dynamic pre-trained mod- els

    Jiaao He, Jidong Zhai, Tiago Antunes, Haojie Wang, Fuwen Luo, Shangfeng Shi, and Qin Li. Fastermoe: modeling and optimizing training of large-scale dynamic pre-trained mod- els. In PPoPP ’22: 27th ACM SIGPLAN Symposium on Prin- ciples and Practice of Parallel Programming, Seou...

    2022

  12. [20]

    Anystory: Towards unified single and multiple subject personalization in text-to-image generation

    Junjie He, Yuxiang Tuo, Binghui Chen, Chongyang Zhong, Yifeng Geng, and Liefeng Bo. Anystory: Towards unified single and multiple subject personalization in text-to-image generation. arXiv preprint arXiv:2501.09503, 2025. 3

    2025 arXiv

  13. [21]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  14. [22]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In 9 The Tenth International Conference on Learning Represen- tations, ICLR 2022, Virtual Event, April 25-29, 202...

    2022

  15. [23]

    Diffumatting: Synthe- sizing arbitrary objects with matting-level annotation

    Xiaobin Hu, Xu Peng, Donghao Luo, Xiaozhong Ji, Jin- long Peng, Zhengkai Jiang, Jiangning Zhang, Taisong Jin, Chengjie Wang, and Rongrong Ji. Diffumatting: Synthe- sizing arbitrary objects with matting-level annotation. arXiv preprint arXiv:2403.06168, 2024. 2

    2024 arXiv

  16. [24]

    Consistentid: Portrait genera- tion with multimodal fine-grained identity preserving

    Jiehui Huang, Xiao Dong, Wenhui Song, Zheng Chong, Zhenchao Tang, Jun Zhou, Yuhao Cheng, Long Chen, Han- hui Li, Yiqiang Yan, et al. Consistentid: Portrait genera- tion with multimodal fine-grained identity preserving. arXiv preprint arXiv:2404.16771, 2024. 2, 3

    2024 arXiv

  17. [25]

    Adaptive mixtures of local experts.Neu- ral computation, 3(1):79–87, 1991

    Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts.Neu- ral computation, 3(1):79–87, 1991. 3

    1991

  18. [26]

    Stableviton: Learning semantic correspon- dence with latent diffusion model for virtual try-on

    Jeongho Kim, Guojung Gu, Minho Park, Sunghyun Park, and Jaegul Choo. Stableviton: Learning semantic correspon- dence with latent diffusion model for virtual try-on. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8176–8185, 2024. 2

    2024

  19. [27]

    Stylemc: Multi-channel based fast text-guided im- age generation and manipulation

    Umut Kocasari, Alara Dirik, Mert Tiftikci, and Pinar Ya- nardag. Stylemc: Multi-channel based fast text-guided im- age generation and manipulation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer vision, pages 895–904, 2022. 2

    2022

  20. [28]

    Anymaker: Zero-shot gen- eral object customization via decoupled dual-level ID injec- tion

    Lingjie Kong, Kai Wu, Xiaobin Hu, Wenhui Han, Jin- long Peng, Chengming Xu, Donghao Luo, Jiangning Zhang, Chengjie Wang, and Yanwei Fu. Anymaker: Zero-shot gen- eral object customization via decoupled dual-level ID injec- tion. CoRR, abs/2406.11643, 2024. 3

    2024 arXiv

  21. [29]

    Omg: Occlusion-friendly personalized multi-concept generation in diffusion models

    Zhe Kong, Yong Zhang, Tianyu Yang, Tao Wang, Kaihao Zhang, Bizhu Wu, Guanying Chen, Wei Liu, and Wenhan Luo. Omg: Occlusion-friendly personalized multi-concept generation in diffusion models. In European Conference on Computer Vision, pages 253–270. Springer, 2024. 1

    2024

  22. [30]

    Multi-concept customiza- tion of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customiza- tion of text-to-image diffusion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Van- couver, BC, Canada, June 17-24, 2023 , pages 1931–1941. IE...

    2023

  23. [31]

    Generating multi-image synthetic data for text-to-image customization

    Nupur Kumari, Xi Yin, Jun-Yan Zhu, Ishan Misra, and Samaneh Azadi. Generating multi-image synthetic data for text-to-image customization. arXiv preprint arXiv:2502.01720, 2025. 1

    2025

  24. [32]

    Gshard: Scaling giant models with conditional computation and automatic sharding

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computation and automatic sharding. In 9th International Conference on Learning Representations, I...

    2021

  25. [33]

    Blip-diffusion: Pre- trained subject representation for controllable text-to-image generation and editing

    Dongxu Li, Junnan Li, and Steven Hoi. Blip-diffusion: Pre- trained subject representation for controllable text-to-image generation and editing. Advances in Neural Information Pro- cessing Systems, 36:30146–30166, 2023. 3

    2023

  26. [34]

    Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024. 2, 3

    2024

  27. [35]

    Ladi-vton: Latent diffusion textual-inversion enhanced virtual try-on

    Davide Morelli, Alberto Baldrati, Giuseppe Cartella, Mar- cella Cornia, Marco Bertini, and Rita Cucchiara. Ladi-vton: Latent diffusion textual-inversion enhanced virtual try-on. In Proceedings of the 31st ACM international conference on multimedia, pages 8580–8589, 2023. 2

    2023

  28. [36]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 1, 2

    2021 arXiv

  29. [37]

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael ...

    2024

  30. [38]

    Attndreambooth: To- wards text-aligned personalized text-to-image generation

    Lianyu Pang, Jian Yin, Baoquan Zhao, Feize Wu, Fu Lee Wang, Qing Li, and Xudong Mao. Attndreambooth: To- wards text-aligned personalized text-to-image generation. In Advances in Neural Information Processing Systems 38: An- nual Conference on Neural Information Processing Syst...

    2024

  31. [39]

    Text- boost: Towards one-shot personalization of text-to-image models via fine-tuning text encoder

    NaHyeon Park, Kunhee Kim, and Hyunjung Shim. Text- boost: Towards one-shot personalization of text-to-image models via fine-tuning text encoder. CoRR, abs/2409.08248,

    arXiv

  32. [40]

    Styleclip: Text-driven manipulation of stylegan imagery

    Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. Styleclip: Text-driven manipulation of stylegan imagery. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 2085–2094,

  33. [41]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  34. [42]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1, 2

    2023 arXiv

  35. [43]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  36. [44]

    Dreamblend: Advancing person- 10 alized fine-tuning of text-to-image diffusion models

    Shwetha Ram, Tal Neiman, Qianli Feng, Andrew Stuart, Son Tran, and Trishul Chilimbi. Dreamblend: Advancing person- 10 alized fine-tuning of text-to-image diffusion models. 2025. 3

    2025

  37. [45]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  38. [46]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 , pages 10674–...

    2022

  39. [47]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, CVPR 2023, Vancouver, BC, Ca...

    2023

  40. [48]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  41. [49]

    Visual layout composer: Image-vector dual diffusion model for de- sign layout generation

    Mohammad Amin Shabani, Zhaowen Wang, Difan Liu, Nanxuan Zhao, Jimei Yang, and Yasutaka Furukawa. Visual layout composer: Image-vector dual diffusion model for de- sign layout generation. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, CVPR 2024, Seattle, W...

    2024

  42. [50]

    Ziplora: Any subject in any style by effectively merging loras

    Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svet- lana Lazebnik, Yuanzhen Li, and Varun Jampani. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, pages 422–438. Springer, 2024. 1

    2024

  43. [51]

    Le, Geoffrey E

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V . Le, Geoffrey E. Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, Fran...

    2017

  44. [52]

    Scaling vision-language models with sparse mixture of experts

    Sheng Shen, Zhewei Yao, Chunyuan Li, Trevor Darrell, Kurt Keutzer, and Yuxiong He. Scaling vision-language models with sparse mixture of experts. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2023, Singa- pore, December 6-10, 2023 , pages 11329–11344. A...

    2023

  45. [53]

    Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, and Denny Zhou

    Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Long- pre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Al- bert Webson, Yunxuan Li, Vincent Y . Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, and Denny Zhou. Mixture-of- ex...

    2024

  46. [54]

    In- stantbooth: Personalized text-to-image generation without test-time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. In- stantbooth: Personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 8543–8552, 2024. 3

    2024

  47. [55]

    Rela- tionbooth: Towards relation-aware customized object gen- eration

    Qingyu Shi, Lu Qi, Jianzong Wu, Jinbin Bai, Jingbo Wang, Yunhai Tong, Xiangtai Li, and Ming-Husan Yang. Rela- tionbooth: Towards relation-aware customized object gen- eration. arXiv preprint arXiv:2410.23280, 2024. 3

    2024 arXiv

  48. [56]

    Moma: Multimodal llm adapter for fast personalized image generation

    Kunpeng Song, Yizhe Zhu, Bingchen Liu, Qing Yan, Ahmed Elgammal, and Xiao Yang. Moma: Multimodal llm adapter for fast personalized image generation. In European Con- ference on Computer Vision, pages 117–132. Springer, 2024. 3

    2024

  49. [57]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. InProceedings of the IEEE/CVF winter...

    2022

  50. [58]

    Instantid: Zero-shot identity-preserving gener- ation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and An- thony Chen. Instantid: Zero-shot identity-preserving gener- ation in seconds. CoRR, abs/2401.07519, 2024. 3

    2024 arXiv

  51. [59]

    Ms-diffusion: Multi-subject zero-shot im- age personalization with layout guidance

    Xierui Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. Ms-diffusion: Multi-subject zero-shot im- age personalization with layout guidance. arXiv preprint arXiv:2406.07209, 2024. 3

    2024 arXiv

  52. [60]

    Dolfin: Diffusion layout transformers without autoencoder

    Yilin Wang, Zeyuan Chen, Liangjun Zhong, Zheng Ding, and Zhuowen Tu. Dolfin: Diffusion layout transformers without autoencoder. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29- October 4, 2024, Proceedings, Part LI , pages 326–343. Springe...

    2024

  53. [61]

    Groundingbooth: Grounding text- to-image customization

    Zhexiao Xiong, Wei Xiong, Jing Shi, He Zhang, Yizhi Song, and Nathan Jacobs. Groundingbooth: Grounding text- to-image customization. arXiv preprint arXiv:2409.08520 ,

    arXiv

  54. [62]

    Predict, prevent, and evaluate: Disentangled text-driven im- age manipulation empowered by pre-trained vision-language model

    Zipeng Xu, Tianwei Lin, Hao Tang, Fu Li, Dongliang He, Nicu Sebe, Radu Timofte, Luc Van Gool, and Errui Ding. Predict, prevent, and evaluate: Disentangled text-driven im- age manipulation empowered by pre-trained vision-language model. In Proceedings of the IEEE/CVF Conference...

    2022

  55. [63]

    Dreammix: Decoupling object attributes for enhanced ed- itability in customized image inpainting

    Yicheng Yang, Pengxiang Li, Lu Zhang, Liqian Ma, Ping Hu, Siyu Du, Yunzhi Zhuge, Xu Jia, and Huchuan Lu. Dreammix: Decoupling object attributes for enhanced ed- itability in customized image inpainting. arXiv preprint arXiv:2411.17223, 2024. 3

    2024

  56. [64]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. CoRR, abs/2308.06721, 2023. 3

    2023 arXiv

  57. [65]

    Inpaint anything: 11 Segment anything meets image inpainting

    Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. Inpaint anything: 11 Segment anything meets image inpainting. arXiv preprint arXiv:2304.06790, 2023. 2

    2023 arXiv

  58. [66]

    Cat-dm: Controllable acceler- ated virtual try-on with diffusion model

    Jianhao Zeng, Dan Song, Weizhi Nie, Hongshuo Tian, Tong- tong Wang, and An-An Liu. Cat-dm: Controllable acceler- ated virtual try-on with diffusion model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8372–8382, 2024. 2

    2024

  59. [67]

    Ssr-encoder: Encoding selective subject representation for subject-driven generation

    Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Ssr-encoder: Encoding selective subject representation for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  60. [68]

    Efficientvit-sam: Accelerated segment anything model without performance loss

    Zhuoyang Zhang, Han Cai, and Song Han. Efficientvit-sam: Accelerated segment anything model without performance loss. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024 - Workshops, Seattle, WA, USA, June 17-18, 2024, pages 7859–7863. IEEE, 2024. 5

    2024

  61. [69]

    Multibooth: Towards generating all your concepts in an im- age from text

    Chenyang Zhu, Kai Li, Yue Ma, Chunming He, and Xiu Li. Multibooth: Towards generating all your concepts in an im- age from text. arXiv preprint arXiv:2404.14239, 2024. 1 12

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.