Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FreeLoRA fuses subject LoRAs at inference, no re-tuning.

desk verdict A simple, plausible training-free LoRA fusion recipe for autoregressive image models; the central claim rests on an asserted token–content alignment that the paper never directly checks. read the letter →

arxiv 2507.01792 v1 pith:VEMFI4IC submitted 2025-07-02 cs.CV

Peng Zheng , Ye Wang , Rui Ma , Zuxuan Wu This is my paper
classification cs.CV
keywords multi-subjectpersonalizationLoRAfusionautoregressiveimagegenerationsubject-awareinferencefulltokentuningvisualmodellow-rankadaptationtraining-freecomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that multi-subject personalization does not require joint optimization or re-tuning: each subject gets an independently trained LoRA, and at inference the LoRA is switched on only for the subject's tokens in the prompt. The method, called FreeLoRA, is implemented on the visual autoregressive model Infinity-2B, with Full Token Tuning during adaptation and Subject-Aware Inference at generation time. The reported experiments on DreamBooth and ViCo show that this training-free fusion matches or exceeds tuning-based baselines and is competitive with large-scale training-based systems. The payoff is that adding a new subject is just training one LoRA, and composing subjects is just writing a prompt.

What carries the argument

The mechanism is a token-gated LoRA switch inside cross-attention. For ordinary prompt tokens, keys and values come from the frozen base projection W; for tokens belonging to a subject, the LoRA-augmented projection W + αBA is used instead. Full Token Tuning makes the LoRA learn weak associations between prompt components and image content across all tokens, which is what lets the same activation rule work for subjects it was never jointly trained with. Subject-Aware Inference is what turns those per-token associations into a training-free fusion rule.

What would settle it

Run FreeLoRA on a backbone other than Infinity-2B with a prompt containing two similar subjects, for example 'a A1 dog and a B2 dog', and measure identity fidelity against the no-SAI baseline; if fidelity drops to the no-SAI level, or if attention maps show each subject token attending to the other subject's region, the token-alignment assumption is false.

Watch

Extended reading notes

Core claim

The central discovery is that a token-content alignment emerges from a weakly supervised objective, and this alignment is enough to localize each LoRA's effect at inference. Each LoRA is trained on prompts such as 'a photo of A1 cat' with the LoRA applied to the key and value projections of every token; no explicit supervision ties 'A1' to the subject's pixels. At inference, the LoRA for a subject is applied only to tokens matching that subject, so each independently trained module contributes only its own identity and nothing else. The paper reports that this outperforms other tuning-based methods on single- and multi-subject benchmarks and reaches parity with the strongest training-based approach, UNO, in multi-subject settings.

Load-bearing premise

Everything rests on the assumption that Full Token Tuning makes each LoRA naturally associate with its own subject token, so that activating the LoRA only on those tokens at inference yields correct rendering; the paper offers no direct measurement of this alignment, and the claim collapses if the association fails on general prompts or different backbones.

Editorial extensions

If this is right

  • Adding a new subject to a composition costs only one independent LoRA training run; no re-tuning of existing modules is needed.
  • The base model stays frozen, so subject adaptation does not degrade the pretrained model's general generation ability.
  • Multi-subject quality no longer depends on merging algorithms or layout guidance, since each LoRA acts only on its own tokens.
  • The method is formulated for Transformer-based autoregressive models and is reported to be compatible with any such backbone, not just Infinity-2B.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A straightforward extension is to test the same token-gated KV LoRA in diffusion-transformer backbones; the central assumption would be that the emergent token-content alignment transfers across architectures.
  • The paper does not describe automatic subject-token extraction from free-form prompts; for general prompts, tokenization and identifier matching could limit which compositions are possible.
  • The claimed emergence could be checked directly: attention maps during inference should show each subject token attending to its own generated region, and a strong violation of that pattern would indicate the method is bound to specific prompts rather than general.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces FreeLoRA, a method for multi-subject personalization in autoregressive image generation. It tunes one LoRA per subject, inserted into the K/V projections of cross-attention, using Full Token Tuning (FTT) over all prompt tokens, and at inference activates each LoRA only on its corresponding subject tokens (Subject-Aware Inference, SAI). Experiments on ViCo and DreamBooth with the Infinity-2B backbone report strong single-subject fidelity and prompt consistency, and multi-subject results comparable to training-based systems such as UNO, without joint optimization or layout conditioning. The central claim is that independently tuned LoRAs can be fused training-free because FTT induces token-content alignment that SAI exploits.

Significance. FreeLoRA is a simple and appealing recipe: if the token-content alignment assumption holds, it would be the first training-free fusion scheme for multi-subject personalization in visual autoregressive models, avoiding the joint optimization required by Mix-of-Show, Custom Diffusion, and CLoRA. The reported single-subject ViCo result (Idino 0.735 vs ARBooth 0.705) and multi-subject DreamBooth parity with UNO are encouraging, and the ablation studies give qualitative support for both FTT and SAI. However, the paper's central mechanism, emergent alignment between subject tokens and LoRA behavior, is asserted rather than directly measured, and the subject-token extraction procedure is unspecified, so the generality of the method is not yet established. With additional evidence the contribution would be solid.

major comments (4)
  1. [Section 4.3 (Eq. (6))] The only description of Subject-Aware Inference is 'We extract a subject token set S from the prompt'; no algorithm, regex, tokenization rule, or fallback behavior is specified. All experiments use fixed unique identifiers such as 'A1 cat', so it is unclear how SAI behaves with class nouns, adjectives, paraphrases, or identifiers split across subword tokens. Because SAI is the mechanism that makes training-free fusion possible, please specify the extraction procedure precisely and evaluate at least one setting without artificial unique identifiers, reporting tokenization boundaries.
  2. [Section 4.4] The paper asserts that 'the model naturally learns this correspondence' between subject tokens and visual content, but provides no direct measurement of this alignment. Table 4 only compares all-token vs subject-token gating at inference; it does not show per-token contributions, attention maps, or KV attribution. Please add direct evidence that LoRA_A's modifications are localized to the tokens or image regions of subject A, for example per-token activation maps or attention analyses, and document cases where alignment fails.
  3. [Tables 1-3] Most baseline numbers are taken from ARBooth and UNO rather than re-run under the same backbone, prompt set, and sampling protocol, and no standard deviations or significance tests are reported. Headline differences are small (e.g., Table 3 Idino 0.529 vs 0.542 for UNO; Table 2 Idino 0.720 vs 0.760), so the 'on par / outperforms' claims need error bars and matched evaluation. Please re-run at least the closest baselines under identical conditions or report per-subject distributions.
  4. [Section 4.7, Table 4] The ablation 'w/o SAI' has higher Idino and Iclip but lower Tclip; the paper attributes this to background overfitting, but no metric isolates background vs foreground fidelity. Please quantify overfitting directly (e.g., background masking, per-subject DINO on foreground regions) or soften the overfitting interpretation.
minor comments (4)
  1. [Figure 5] The caption calls CLoRA and Custom Diffusion 'non-training-based methods', though they are tuning-based and require additional optimization; the text in Section 4.6 uses the correct categorization. Please align the caption with the text.
  2. [Section 4.5] Training and inference hyperparameters are missing: LoRA rank r, scaling factor alpha, learning rate, number of steps, batch size, optimizer, and sampling parameters (e.g., temperature, top-k). These are needed for reproducibility.
  3. [Section 4.4 / Conclusion] The method is described as model-agnostic, but experiments are only on Infinity-2B. Please state this as an unverified expectation rather than a general claim, or add a small experiment on a second backbone.
  4. [Section 4.3] The phrase 'a A1 cat and a B2 dog' is ungrammatical; use 'an A1 cat' or rephrase. Also, the name 'Full Token Tuning' is somewhat misleading for applying LoRA to all tokens during tuning; consider 'All-Token Tuning' for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FreeLoRA is an empirical method evaluated against external benchmarks, with no fitted parameter renamed as a prediction and no load-bearing self-citation chain.

full rationale

FreeLoRA is presented as an empirical recipe rather than a derivation: the paper specifies a training strategy (Full Token Tuning, Section 4.2), an inference-time gating rule (Subject-Aware Inference, Section 4.3), and then measures subject fidelity and prompt consistency on the external DreamBooth and ViCo benchmarks (Tables 1-3). The LoRA rank and scaling factor are ordinary hyperparameters; they are not fitted to make the reported metrics come out, so no fitted input is renamed as a prediction. The inference rule in Eq. (6) simply selects which LoRA update is applied to which prompt tokens; it is not defined in terms of the evaluation metrics, and the reported improvements are empirical outcomes rather than consequences of a definition. The paper's claim that FTT plus SAI lets the model 'naturally learn' token-content alignment (Section 4.4) is an asserted emergent property, not a circular justification; it may be undersupported, but undersupport is a correctness risk, not circularity. There are no self-citations by the authors that carry the central argument, and no uniqueness theorem is imported from prior work. The unspecified procedure for extracting the subject token set from arbitrary prompts is a missing implementation detail that limits generality, but it does not make any derived quantity equivalent to its input. The central claim, therefore, is not circular: it is a testable empirical hypothesis evaluated against independent benchmarks.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

FreeLoRA introduces no new physical or architectural entities; it reuses LoRA, which is an established technique. The assumptions above are the load-bearing premises that a verification would need to check.

free parameters (3)
  • LoRA rank r
    The rank of the low-rank adapters is chosen by hand; it controls capacity and is not disclosed in the paper.
  • LoRA scaling factor alpha
    The scaling factor for the LoRA update is a hand-chosen hyperparameter; its value is not reported.
  • Training hyperparameters (learning rate, steps, optimizer)
    Not specified, so a re-implementation must choose them by hand.
assumptions (4)
  • domain assumption The pretrained Infinity-2B model has sufficient compositional capability to render multiple subjects in one scene.
    The paper's Conclusion states the method 'may fail in complex multi-subject scenes where the pretrained model lacks sufficient compositional capability', admitting this reliance.
  • domain assumption Subject tokens such as 'A1 cat' can be automatically identified and matched at inference.
    Section 4.3 says 'We extract a subject token set...' but does not describe the matching mechanism or its failure modes.
  • ad hoc to paper Full Token Tuning yields emergent token-content alignment without explicit supervision.
    Section 4.4 postulates that the network 'naturally learns this correspondence' as a result of FTT+SAI; this is a load-bearing assumption that is asserted but not directly measured.
  • domain assumption Injecting LoRA only into KV projections of cross-attention layers is sufficient to capture subject identity.
    The paper does not ablate alternative injection points (e.g., Q, or MLP) and assumes KV-only adaptation is adequate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization." pith.science (2026). https://pith.science/paper/VEMFI4IC

@misc{pith2026250701792,
  author       = {Pith},
  title        = {Pith review of: FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VEMFI4IC}},
  note         = {Machine review of arXiv:2507.01792}
}
read the original abstract

Subject-driven image generation plays a crucial role in applications such as virtual try-on and poster design. Existing approaches typically fine-tune pretrained generative models or apply LoRA-based adaptations for individual subjects. However, these methods struggle with multi-subject personalization, as combining independently adapted modules often requires complex re-tuning or joint optimization. We present FreeLoRA, a simple and generalizable framework that enables training-free fusion of subject-specific LoRA modules for multi-subject personalization. Each LoRA module is adapted on a few images of a specific subject using a Full Token Tuning strategy, where it is applied across all tokens in the prompt to encourage weakly supervised token-content alignment. At inference, we adopt Subject-Aware Inference, activating each module only on its corresponding subject tokens. This enables training-free fusion of multiple personalized subjects within a single image, while mitigating overfitting and mutual interference between subjects. Extensive experiments show that FreeLoRA achieves strong performance in both subject fidelity and prompt consistency.

Figures

Figures reproduced from arXiv: 2507.01792 by the authors.

Figure 1
Figure 1. Personalized image generation with FreeLoRA. Each LoRA module is independently tuned [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of FreeLoRA. During tuning, we insert subject-specific LoRA modules into the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison of single-subject personalization. Both our method and ARBooth [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of single-subject personalization between autoregressive models. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of multi-subject personalization among non-training-based meth [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of multi-subject personalization with training-based methods. Our [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative ablation study on Subject-Aware Inference (SAI) and Full Token Tuning (FTT). [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Training-Free Multi-Concept LoRA Composition with Prompt-Aware Weighting

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Prompt-aware weighting strategies W-Switch and W-Composite improve multi-concept LoRA composition in diffusion models without training.

Reference graph

Works this paper leans on

57 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021

    2021

  2. [2]

    Re-imagen: Retrieval- augmented text-to-image generator

    Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W Cohen. Re-imagen: Retrieval- augmented text-to-image generator. arXiv preprint arXiv:2209.14491, 2022

    arXiv 2022

  3. [3]

    Fine-tuning visual autoregressive models for subject-driven generation

    Jiwoo Chung, Sangeek Hyun, Hyunjun Kim, Eunseo Koh, MinKyu Lee, and Jae-Pil Heo. Fine-tuning visual autoregressive models for subject-driven generation. arXiv preprint arXiv:2504.02612, 2025

    arXiv 2025

  4. [4]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  5. [5]

    Freecustom: Tuning-free customized image generation for multi-concept composition

    Ganggui Ding, Canyu Zhao, Wen Wang, Zhen Yang, Zide Liu, Hao Chen, and Chunhua Shen. Freecustom: Tuning-free customized image generation for multi-concept composition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9089–9098, 2024

    2024

  6. [6]

    How to continually adapt text-to-image diffusion models for flexible customization? Advances in Neural Information Processing Systems, 37:130057– 130083, 2024

    Jiahua Dong, Wenqi Liang, Hongliu Li, Duzhen Zhang, Meng Cao, Henghui Ding, Salman H Khan, and Fahad Shahbaz Khan. How to continually adapt text-to-image diffusion models for flexible customization? Advances in Neural Information Processing Systems, 37:130057– 130083, 2024

    work page 2024

  7. [7]

    Personalize anything for free with diffusion transformer

    Haoran Feng, Zehuan Huang, Lin Li, Hairong Lv, and Lu Sheng. Personalize anything for free with diffusion transformer. arXiv preprint arXiv:2503.12590, 2025

    arXiv 2025

  8. [8]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

    arXiv 2022

Show all 57 references
  1. [9]

    Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models

    Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yunpeng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, et al. Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models. Advances in Neural Information Processing Sys...

    2023

  2. [10]

    Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. arXiv preprint arXiv:2412.04431, 2024

    2024 arXiv

  3. [11]

    Shaozhe Hao, Kai Han, Shihao Zhao, and Kwan-Yee K. Wong. Vico: Detail-preserving visual condition for personalized text-to-image generation. 2023

    2023

  4. [12]

    Anys- tory: Towards unified single and multiple subject personalization in text-to-image generation

    Junjie He, Yuxiang Tuo, Binghui Chen, Chongyang Zhong, Yifeng Geng, and Liefeng Bo. Anys- tory: Towards unified single and multiple subject personalization in text-to-image generation. arXiv preprint arXiv:2501.09503, 2025

    2025 arXiv

  5. [13]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  6. [14]

    Resolving multi- condition confusion for finetuning-free personalized image generation

    Qihan Huang, Siming Fu, Jinlong Liu, Hao Jiang, Yipeng Yu, and Jie Song. Resolving multi- condition confusion for finetuning-free personalized image generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3707–3714, 2025. 10

    2025

  7. [15]

    Flux already knows-activating subject-driven image generation without training

    Hao Kang, Stathi Fotiadis, Liming Jiang, Qing Yan, Yumin Jia, Zichuan Liu, Min Jin Chong, and Xin Lu. Flux already knows-activating subject-driven image generation without training. arXiv preprint arXiv:2504.11478, 2025

    2025 arXiv

  8. [16]

    Multi- concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi- concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1931–1941, 2023

    1931

  9. [17]

    Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing

    Dongxu Li, Junnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. Advances in Neural Information Processing Systems, 36:30146–30166, 2023

    2023

  10. [18]

    Autoregressive image generation without vector quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37:56424–56445, 2025

    2025

  11. [19]

    Cones: Concept neurons in diffusion models for customized generation

    Zhiheng Liu, Ruili Feng, Kai Zhu, Yifei Zhang, Kecheng Zheng, Yu Liu, Deli Zhao, Jingren Zhou, and Yang Cao. Cones: Concept neurons in diffusion models for customized generation. arXiv preprint arXiv:2303.05125, 2023

    2023 arXiv

  12. [20]

    Subject-diffusion: Open domain per- sonalized text-to-image generation without test-time fine-tuning

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject-diffusion: Open domain per- sonalized text-to-image generation without test-time fine-tuning. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024

    2024

  13. [21]

    Realcustom++: Representing images as real-word for real-time customization

    Zhendong Mao, Mengqi Huang, Fei Ding, Mingcong Liu, Qian He, and Yongdong Zhang. Realcustom++: Representing images as real-word for real-time customization. arXiv preprint arXiv:2408.09744, 2024

    2024

  14. [22]

    Clora: A contrastive approach to compose multiple lora models

    Tuna Han Salih Meral, Enis Simsar, Federico Tombari, and Pinar Yanardag. Clora: A contrastive approach to compose multiple lora models. arXiv preprint arXiv:2403.19776, 2024

    2024 arXiv

  15. [23]

    Dreamo: A unified framework for image customization

    Chong Mou, Yanze Wu, Wenxu Wu, Zinan Guo, Pengze Zhang, Yufeng Cheng, Yiming Luo, Fei Ding, Shiwen Zhang, Xinghui Li, et al. Dreamo: A unified framework for image customization. arXiv preprint arXiv:2504.16915, 2025

    2025

  16. [24]

    Dreammatcher: appearance matching self-attention for semantically-consistent text-to-image personalization

    Jisu Nam, Heesu Kim, DongJae Lee, Siyoon Jin, Seungryong Kim, and Seunggyu Chang. Dreammatcher: appearance matching self-attention for semantically-consistent text-to-image personalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  17. [25]

    K-lora: Unlocking training-free fusion of any subject and style loras

    Ziheng Ouyang, Zhen Li, and Qibin Hou. K-lora: Unlocking training-free fusion of any subject and style loras. arXiv preprint arXiv:2502.18461, 2025

    2025 arXiv

  18. [26]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  19. [27]

    Bootpig: Bootstrapping zero-shot personalized image generation capabilities in pretrained diffusion models

    Senthil Purushwalkam, Akash Gokul, Shafiq Joty, and Nikhil Naik. Bootpig: Bootstrapping zero-shot personalized image generation capabilities in pretrained diffusion models. arXiv preprint arXiv:2401.13974, 2024

    2024 arXiv

  20. [28]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  21. [29]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  22. [30]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22500...

    2023

  23. [31]

    Low-rank adaptation for fast text-to-image diffusion fine-tuning

    Simo Ryu. Low-rank adaptation for fast text-to-image diffusion fine-tuning. https://github. com/cloneofsimo/lora

  24. [32]

    Ziplora: Any subject in any style by effectively merging loras

    Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svetlana Lazebnik, Yuanzhen Li, and Varun Jampani. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, pages 422–438. Springer, 2024

    2024

  25. [33]

    Instantbooth: Personalized text-to- image generation without test-time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. Instantbooth: Personalized text-to- image generation without test-time finetuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8543–8552, 2024

    2024

  26. [34]

    Llamafusion: Adapting pretrained language models for multimodal generation

    Weijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke Zettlemoyer, and Lili Yu. Llamafusion: Adapting pretrained language models for multimodal generation. arXiv preprint arXiv:2412.15188, 2024

    2024 arXiv

  27. [35]

    Large-scale text-to- image model with inpainting is a zero-shot subject-driven image generator

    Chaehun Shin, Jooyoung Choi, Heeseung Kim, and Sungroh Yoon. Large-scale text-to- image model with inpainting is a zero-shot subject-driven image generator. arXiv preprint arXiv:2411.15466, 2024

    2024 arXiv

  28. [36]

    Personalized text-to-image generation with auto-regressive models

    Kaiyue Sun, Xian Liu, Yao Teng, and Xihui Liu. Personalized text-to-image generation with auto-regressive models. arXiv preprint arXiv:2504.13162, 2025

    2025 arXiv

  29. [37]

    Ominicontrol: Minimal and universal control for diffusion transformer

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint arXiv:2411.15098, 2024

    2024 arXiv

  30. [38]

    Chameleon: Mixed-modal early-fusion foundation models

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024

    2024 arXiv

  31. [39]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  32. [40]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839–84865, 2025

    2025

  33. [41]

    Metamorph: Multimodal under- standing and generation via instruction tuning

    Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal under- standing and generation via instruction tuning. arXiv preprint arXiv:2412.14164, 2024

    2024 arXiv

  34. [42]

    Ms-diffusion: Multi- subject zero-shot image personalization with layout guidance

    Xierui Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. Ms-diffusion: Multi- subject zero-shot image personalization with layout guidance. arXiv preprint arXiv:2406.07209, 2024

    2024 arXiv

  35. [43]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024

    2024 arXiv

  36. [44]

    Maskbit: Embedding-free image generation via bit tokens

    Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang- Chieh Chen. Maskbit: Embedding-free image generation via bit tokens. arXiv preprint arXiv:2409.16211, 2024

    2024 arXiv

  37. [45]

    Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15943– 15953, 2023

    2023

  38. [46]

    Less-to- more generalization: Unlocking more controllability by in-context generation

    Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He. Less-to- more generalization: Unlocking more controllability by in-context generation. arXiv preprint arXiv:2504.02160, 2025

    2025 arXiv

  39. [47]

    Vila-u: a unified foundation model integrating visual understanding and generation

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.04429, 2024. 12

    2024 arXiv

  40. [48]

    Proxy-tuning: Tailoring multimodal autoregressive models for subject-driven image generation.arXiv preprint arXiv:2503.10125, 2025

    Yi Wu, Lingting Zhu, Lei Liu, Wandi Qiao, Ziqiang Li, Lequan Yu, and Bin Li. Proxy-tuning: Tailoring multimodal autoregressive models for subject-driven image generation.arXiv preprint arXiv:2503.10125, 2025

    2025

  41. [49]

    Omnigen: Unified image generation

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. arXiv preprint arXiv:2409.11340, 2024

    2024 arXiv

  42. [50]

    Show-o: One single trans- former to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single trans- former to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024

    2024 arXiv

  43. [51]

    Lora-composer: Leveraging low-rank adaptation for multi-concept customization in training-free diffusion models.arXiv preprint arXiv:2403.11627, 2024

    Yang Yang, Wen Wang, Liang Peng, Chaotian Song, Yao Chen, Hengjia Li, Xiaolong Yang, Qinglin Lu, Deng Cai, Boxi Wu, et al. Lora-composer: Leveraging low-rank adaptation for multi-concept customization in training-free diffusion models.arXiv preprint arXiv:2403.11627, 2024

    2024 arXiv

  44. [52]

    Randomized autore- gressive visual generation

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang-Chieh Chen. Randomized autore- gressive visual generation. arXiv preprint arXiv:2411.00776, 2024

    2024 arXiv

  45. [53]

    Bringing characters to new stories: Training-free theme-specific image generation via dynamic visual prompting

    Yuxin Zhang, Minyan Luo, Weiming Dong, Xiao Yang, Haibin Huang, Chongyang Ma, Oliver Deussen, Tong-Yee Lee, and Changsheng Xu. Bringing characters to new stories: Training-free theme-specific image generation via dynamic visual prompting. arXiv preprint arXiv:2501.15641, 2025

    2025 arXiv

  46. [54]

    Ssr-encoder: Encoding selective subject representation for subject- driven generation

    Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Ssr-encoder: Encoding selective subject representation for subject- driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2024

  47. [55]

    Multi-lora composition for image generation

    Ming Zhong, Yelong Shen, Shuohang Wang, Yadong Lu, Yizhu Jiao, Siru Ouyang, Donghan Yu, Jiawei Han, and Weizhu Chen. Multi-lora composition for image generation. arXiv preprint arXiv:2402.16843, 2024

    2024 arXiv

  48. [56]

    Transfusion: Predict the next token and diffuse images with one multi-modal model

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408.11039, 2024

    2024 arXiv

  49. [57]

    Magictailor: Component-controllable personalization in text-to-image diffusion models

    Donghao Zhou, Jiancheng Huang, Jinbin Bai, Jiaze Wang, Hao Chen, Guangyong Chen, Xiaowei Hu, and Pheng-Ann Heng. Magictailor: Component-controllable personalization in text-to-image diffusion models. arXiv preprint arXiv:2410.13370, 2024. 13

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.