Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LoRA.rar trains a 0.49M-parameter hypernetwork to merge unseen content and style LoRAs by predicting column-wise coefficients in a single forward pass, matching or beating ZipLoRA's per-pair optimization at over 4000x speedup.

desk verdict A real efficiency win for LoRA merging, with a novel amortized hypernetwork, but the quality advantage is measured with an author-designed metric on a small test set; still, this deserves a serious referee. read the letter →

arxiv 2412.05148 v2 pith:YLEWLVDB submitted 2024-12-06 cs.CV cs.AIcs.LG

Donald Shenaj , Ondrej Bohdal , Mete Ozay , Pietro Zanuttigh , Umberto Michieli This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords LoRAmerginghypernetworksubject-stylepersonalizationdiffusionmodelsmultimodalLLMevaluationmodelzero-shotgeneralizationimagegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LoRA.rar claims that merging a content LoRA and a style LoRA does not need to be re-solved by optimization for every new pair. Instead, a small pre-trained hypernetwork reads the two LoRA weight matrices and outputs the column-wise coefficients that combine them, doing in one 0.037-second forward pass what ZipLoRA does in about 158 seconds of per-pair gradient updates. To train it, the authors collect a dataset of 360 content-style LoRA pairs and repurpose ZipLoRA's merging loss, originally designed to optimize coefficients for one pair, into a loss for the hypernetwork's weights. If the claim is right, subject-and-style personalization becomes cheap enough for real-time use on resource-constrained devices, and the paper also introduces a new multimodal-LLM evaluation protocol called MARS2 that it reports aligns with human preference far better than CLIP-I or DINO on this task.

What carries the argument

The load-bearing object is the hypernetwork $H$: a two-layer MLP with separate input layers for the two LoRA matrix shapes found in the diffusion model, a shared hidden layer of width 128, and an output layer that emits two coefficients per column. Column independence is the design that makes this tractable: instead of flattening a full LoRA matrix, the network is fed each column pair $[w_i^c, w_i^s]$ and produces $(m_{c,i}, m_{s,i})$, so all columns of a LoRA pair are processed as one minibatch. The coefficients are trained with ZipLoRA's merging loss, which asks the merged model to reproduce the content-only and style-only outputs and adds an orthogonality term $\lambda |m_c \cdot m_s|$. In the paper's configuration the hypernetwork controls query and output projections while key and value projections are averaged directly, a choice validated by ablation.

What would settle it

Run ZipLoRA on the paper's test pairs to obtain its optimized coefficients, then compare them with the hypernetwork's predictions for the same pairs: if the predictions do not approximate the optimized values even on training pairs, or if feeding ZipLoRA's coefficients into LoRA.rar's pipeline raises MARS2 substantially, the column-wise prediction premise is false.

Watch

Extended reading notes

Core claim

The central claim is that a single small hypernetwork can learn the map from a pair of LoRA weight updates to the column-wise coefficients that merge them, and that this map transfers to subject-style combinations not seen during training. The merged update is computed as $\Delta W_m = m_c \otimes \Delta W_c + m_s \otimes \Delta W_s$, where the vectors $m_c$ and $m_s$ come from one forward pass of the hypernetwork over the concatenated columns of the content and style matrices. The paper reports that this transfer is strong enough to beat ZipLoRA's per-pair optimization on the MARS2 metric in the average case (0.71 vs 0.58) while reducing the time to produce coefficients from 158 seconds to 0.037 seconds, and that the same setup also wins on a smaller diffusion backbone.

Load-bearing premise

The method assumes that the right merging coefficient for each column of a LoRA can be predicted from that column alone, without needing to know what the subject or style is or how the columns work together.

Editorial extensions

If this is right

  • A content-style merge for a new pair costs one forward pass (0.037s on an NVIDIA 4090) instead of 100 gradient steps, removing the main obstacle to real-time personalization on smartphones.
  • The hypernetwork generalizes beyond its training distribution: the paper reports success on new subjects and styles such as a toaster, television, sofa, and cyberpunk, and on retrained splits with no semantic overlap between training and test categories, though with somewhat lower scores.
  • The MARS2 protocol gives a scalable automated judge for joint subject-style fidelity, with reported correlation 0.76 against human preference versus 0.08 for CLIP-I and -0.01 for DINO, so future studies can compare methods without costly per-pair optimization.
  • Users need fewer attempts to obtain an acceptable image (2.28 vs 2.55 on average), and the method adds no extra memory at test time beyond the base diffusion model.
  • The same merging recipe transfers to a lightweight diffusion backbone, where it also outperforms ZipLoRA on MARS2 in the average case (0.56 vs 0.51).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the column-independence assumption implies that the method's representational ceiling is local: if future LoRA families encode global or cross-layer semantics in individual columns, the hypernetwork input would need to be richer or the assumption would break.
  • Beyond the paper, the 360-pair LoRA-weight dataset could be reused as a training resource; regressing a predictor directly onto ZipLoRA's optimized coefficients, or distilling the hypernetwork into a smaller quantized model for even more constrained devices, are immediate testable extensions.
  • Beyond the paper, since the hypernetwork maps any pair of LoRA matrices to coefficients, applying it compositionally to more than two LoRAs (e.g., subject + style + lighting) is a natural next step; whether pairwise composition preserves fidelity is an open empirical question.
  • Beyond the paper, the MARS2 judge could be used as a reward signal for sampling or for further fine-tuning of the hypernetwork, not only for evaluation; the paper does not explore this use.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes LoRA.rar, a hypernetwork-based method for merging content and style LoRAs for text-to-image generation. A small (0.49M-parameter) hypernetwork takes the weight-update matrices of a content LoRA and a style LoRA and predicts column-wise merging coefficients in a single forward pass, replacing ZipLoRA's per-pair optimization. The hypernetwork is trained on a dataset of 360 content-style LoRA pairs using ZipLoRA's merge loss, which enforces content fidelity, style fidelity, and orthogonality between the two coefficient vectors. The paper also introduces MARS2, an MLLM-based evaluation protocol that uses LLaVA-Critic to judge jointly whether content and style are preserved. On a test set of 25 subject-style pairs, LoRA.rar reports average MARS2 0.71 versus 0.58 for ZipLoRA, with best-case scores of 1.00 for both, and is over 4000x faster than ZipLoRA at coefficient prediction. The supplement provides ablations, results on a lightweight diffusion model, new-split generalization experiments, and qualitative recontextualization results.

Significance. The paper addresses a real deployment bottleneck: ZipLoRA requires minutes of per-pair optimization, which is impractical for real-time or resource-constrained use. The idea of pre-training a lightweight hypernetwork on LoRA weight updates is sensible, and the authors collect a useful LoRA-pair dataset and provide a detailed algorithm, multiple ablations, and supplementary generalization tests on deliberately challenging train/test splits. If the quality claims hold, this is a meaningful step toward practical personalized generation on-device. The MARS2 metric is also a potentially valuable contribution, with initial evidence that it aligns better with human preference than CLIP-I or DINO for joint subject-style evaluation. However, the central claim of improved image quality rests on a small test set, a single author-proposed metric, and a human study whose 'best-case' images are preselected by that same MLLM judge; the per-column independence assumption that makes the hypernetwork architecture tractable is not directly validated. These issues are proportionate to a major revision rather than grounds for rejection.

major comments (4)
  1. [Sec. 3.3] The per-column independence assumption is load-bearing but unvalidated. The hypernetwork receives only [w_i^c, w_i^s] for each column, with no column index, matrix position, or pair-level semantic information, while the training loss in Eq. (3) couples all columns through the diffusion model. The paper does not directly test whether the optimal merging coefficient for a column can be predicted from that column's entries alone. I recommend adding (i) an ablation in which the hypernetwork also receives global statistics or positional embeddings; (ii) a comparison between the hypernetwork's predicted coefficients and ZipLoRA's optimized coefficients on held-out pairs (e.g., correlation or mean absolute error); and (iii) a sanity check such as shuffling column order or feeding columns from mismatched content/style pairs. Without such evidence, the generalization claim in Sec. 3.1 is not fully established.
  2. [Sec. 5.1 / Table 1] The headline comparison rests on 25 test subject-style pairs and a single author-proposed metric, MARS2, with no confidence intervals or significance tests. The reported best-case scores are identical (1.00 for both ZipLoRA and LoRA.rar), so the claimed improvement is entirely contained in the average-case difference (0.71 vs 0.58), whose variance is unknown. Please report per-pair scores with confidence intervals or a paired significance test, and avoid the word 'significantly' unless it is supported by a statistical test.
  3. [Sec. 4 / Fig. 7] The human evaluation uses MARS2 to pre-select 'best' images, and the results are reported only as aggregate percentages without counts, statistical tests, or inter-rater agreement. Because the image selection for the 'best-case' scenario is based on the same unvalidated MLLM judge used for the main quantitative claim, the human study does not independently validate MARS2. Please report per-participant raw counts and a paired significance test, and ideally include an analysis with images selected without MARS2 pre-filtering.
  4. [Sec. A2.1 / Table A2] The paper's own standard-metric results show that LoRA.rar has lower DINO than ZipLoRA (0.643 vs 0.741) and is only slightly higher on CLIP-I and CLIP-T. The argument that these metrics are inappropriate for joint subject-style generation is plausible, but it places the entire quantitative improvement claim on MARS2. The reported alignment of MARS2 with human preference (correlation 0.76) is computed only on the direct-merge baseline, not on ZipLoRA or LoRA.rar. I recommend computing MARS2-versus-human correlation per image across all compared methods, and reporting the variance of that correlation.
minor comments (4)
  1. [Sec. 3.3] The sentence 'Since different rows are treated as a mini-batch' should read 'columns' rather than 'rows'.
  2. [Sec. 5.1 / Fig. 8] The claim that ZipLoRA 'mostly converges to a binary selection' is descriptive rather than quantitative; consider reporting the distribution of the optimized coefficients to support this statement.
  3. [Table 2] The parameter count for ZipLoRA (1.5M) is reported for a single subject-style pair, while LoRA.rar's 0.49M is a fixed global model; the comparison would benefit from stating the storage cost per additional pair for each method.
  4. [Sec. A1.1] Figure A2's style prompt contains the instruction 'Do not check similarity with the subject'; this is appropriate, but the supplement should clarify how often the MLLM judge disagrees with human judgment on the style criterion alone, since the paper acknowledges the judge 'may overlook small details' (Sec. A3.1).

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity in the hypernetwork prediction; one minor evaluation loop via MARS2-preselected best-case human study.

  1. other [Sec. 4 (Human Evaluation) and Sec. 5.1 (Metrics Alignment with Human Preference); Supp. A1.2 (Human Evaluation Study)]
    "2) take a best sample, i.e. accepted by the MLLM model (if there are multiple samples with correct style and content, randomly choose one of them). ... We computed the correlation between CLIP-I, DINO, and our MARS2 metric against the human evaluation score for the direct merge approach ... and obtained [0.08, −0.01, 0.76] respectively."

    The best-case human evaluation is not an independent check of the proposed MARS2 metric: the images that humans rate are pre-filtered by MARS2 ('accepted by the MLLM model'), so the human 'best' results are conditional on MARS2's own accept/reject decisions. The paper then uses human evaluation to support MARS2's alignment with user preference (correlation 0.76). This creates a closed loop in the best-case branch: MARS2 selects the stimuli, and the same human protocol is cited as validating MARS2. It does not make the hypernetwork's coefficient prediction circular, and the random-case human evaluation plus held-out/new-split experiments remain independent evidence.

full rationale

The central claim is that a 0.49M-parameter hypernetwork, trained on ZipLoRA's merging loss Lmerge (Eq. 3) over training LoRA pairs, predicts column-wise merging coefficients for unseen content/style LoRAs in a single forward pass. The training objective is a diffusion-output reconstruction loss, not the MARS2 evaluation metric, so there is no direct training-evaluation loop. The per-column independence assumption of Sec. 3.3 is a genuine representational assumption and a generalization risk, but it is not circular: it is an input design choice, not a conclusion derived from itself. Self-citations in the related work (e.g., [1], [4], [17], [59]) are contextual and not load-bearing for the method's derivation. Held-out test pairs, new-concept experiments (A2.5), and new-split experiments (A2.6) provide external evidence of generalization. The only notable circularity is the evaluation-loop described in the step above: the best-case human evaluation is conditioned on MARS2's selections, and MARS2's human alignment is cited from the same human-evaluation protocol. This is a partial evaluation circularity, not a circular derivation of the merging coefficients, so the overall score is 2 rather than higher.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim depends on (i) an architectural assumption that columns of LoRA weight matrices are sufficient inputs for predicting merge coefficients, (ii) the transferability of ZipLoRA's denoising loss as a training signal, (iii) the validity of LLaVA-Critic as an evaluator, and (iv) several hyperparameters chosen on validation. MARS2 is tracked as an invented entity because it is a new author-defined instrument, though it has some independent empirical support.

free parameters (4)
  • Orthogonality regularization weight lambda = 0.01
    Set in Sec. A1.3; balances content/style fidelity losses against the mc·ms orthogonality penalty in Lmerge (Eq. 3).
  • Hypernetwork hidden dimension = 128
    Two-layer MLP hidden size, chosen on validation (Sec. A1.3).
  • LoRA rank = 64
    All LoRAs trained with rank 64 following ZipLoRA (Sec. A1.3); the merging operates on these fixed-rank updates.
  • Merged coefficient vectors mc, ms = predicted per column (learned)
    These are the outputs of the hypernetwork optimized on Lmerge; they are the learned quantities that define the merge in Eq. (2) and directly determine image quality.
assumptions (5)
  • ad hoc to paper The merging coefficient for each column of a LoRA weight update can be predicted from that column's entries alone.
    Introduced in Sec. 3.3 to make the input dimensionality tractable; it is the core representational assumption for generalization.
  • domain assumption ZipLoRA's Lmerge is a sufficient training objective for learning a general merging strategy.
    Sec. 3.4 repurposes the per-pair loss from [38] to train the hypernetwork; the paper asserts it promotes generalization but does not prove this.
  • domain assumption Content and style information in LoRA weights can be separated and recombined through per-column linear coefficients.
    The whole merging formulation (Eq. 2) assumes that a weighted sum of the two updates yields coherent content and style.
  • domain assumption LLaVA-Critic binary judgments are a valid proxy for human preference in joint subject-style fidelity.
    Used by MARS2 in Sec. 4; the paper validates this only via a correlation of 0.76 against human scores for direct merge and manual spot checks.
  • domain assumption A hypernetwork trained on 360 subject-style pairs generalizes to unseen subjects and styles.
    Sec. 3.2 and Sec. 5 rely on this empirical generalization claim; supported by held-out test sets and new-split experiments, but not guaranteed.
invented entities (1)
  • MARS2 evaluation metric independent evidence
    purpose: Binary MLLM-based judge of whether a generated image preserves both subject and style, used to rank methods and select best samples.
    Introduced in Sec. 4 and validated by a human-preference correlation of 0.76 (Sec. 5.1) and manual checks; however it is an author-defined instrument and its use in selecting 'best' samples feeds into the human evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image Generation." pith.science (2026). https://pith.science/paper/YLEWLVDB

@misc{pith2026241205148,
  author       = {Pith},
  title        = {Pith review of: LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YLEWLVDB}},
  note         = {Machine review of arXiv:2412.05148}
}
abstract

Recent advancements in image generation models have enabled personalized image creation with both user-defined subjects (content) and styles. Prior works achieved personalization by merging corresponding low-rank adapters (LoRAs) through optimization-based methods, which are computationally demanding and unsuitable for real-time use on resource-constrained devices like smartphones. To address this, we introduce LoRA$.$rar, a method that not only improves image quality but also achieves a remarkable speedup of over $4000\times$ in the merging process. We collect a dataset of style and subject LoRAs and pre-train a hypernetwork on a diverse set of content-style LoRA pairs, learning an efficient merging strategy that generalizes to new, unseen content-style pairs, enabling fast, high-quality personalization. Moreover, we identify limitations in existing evaluation metrics for content-style quality and propose a new protocol using multimodal large language models (MLLMs) for more accurate assessment. Our method significantly outperforms the current state of the art in both content and style fidelity, as validated by MLLM assessments and human evaluations.

Figures

Figures reproduced from arXiv: 2412.05148 by the authors.

Figure 1
Figure 1. We address the problem of joint content-style image [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Method Overview. LoRA.rar pre-trains a hypernetwork that dynamically generates merging coefficients for new, unseen content-style LoRA pairs at deployment. In contrast, existing solutions are limited by either costly test-time training, as with ZipLoRA, or produce lower-quality outputs, as with conventional merging strategies. mization. However, this approach requires several minutes per merge at test time, limiting… view at source ↗
Figure 3
Figure 3. ZipLoRA’s Merging Coefficients mc,ms for randomly selected columns of the LoRA weight update matrices. The coeffi￾cients are visibly different for various combinations of content and style scenarios, showing the need for adaptive solutions. 3.1. Problem Formulation We use a pre-trained image generation diffusion model D with weights W0 and LoRA L with weight update matrix ∆W. For simplicity, we consider one layer at… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Hypernetwork Structure. The hypernetwork has multi￾ple input layers, each matching the dimensionality of corresponding layers in the generative model. Here, an example layer with dimen￾sions matching Input Layer 1 is shown. Content and style LoRAs are concatenated and …
Figure 5
Figure 5. Figure 5: Limitation of Existing Metrics. Top: CLIP-I is maxi￾mized when the style image (shown in the small upper right thumb￾nail) content is replicated. Bottom: DINO is maximized when the generated image has no style transfer. have shown high effectiveness in evaluating text-…
Figure 6
Figure 6. Figure 6: Evaluation via MLLM Judge. Generated images are checked separately for content and style. We mark the image as correct if both are approved. also conduct human evaluations on a subset of generated im￾ages, comparing our results with those from ZipLoRA, the primary comp…
Figure 7
Figure 7. Figure 7: Human Evaluation for generated images sampled ran￾domly or according to MARS2 . More than 75% respondents con￾sider LoRA.rar comparable or better than ZipLoRA. ΔWc columns id −0.2 0.0 0.2 0.4 M ergin g c o e f ficie nts mc ΔWs columns id 0.0 0.1 0.2 0.3 M ergin g c o e…
Figure 8
Figure 8. Figure 8: LoRA.rar’s Merging Coefficients mc,ms for ran￾domly selected columns of the LoRA weight update matrices. LoRA.rar learns a non-trivial strategy with superior performance. result shows the MARS2 metric is well-aligned with hu￾man preference, while CLIP-I and DINO are no…
Figure 9
Figure 9. Figure 9: Qualitative Comparison. LoRA.rar generates better images than other merging strategies, including ZipLoRA. Styles Contents [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: LoRA.rar Evaluation across different subject-style combinations. Our solution consistently produces good results. attempts than ZipLoRA to produce a high-quality image that aligns with both content and style—2.28 attempts for LoRA.rar versus 2.55 for ZipLoRA. This imp…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCA

    cs.LG 2025-06 conditional novelty 6.0 of 10

    FedRPCA decomposes federated LoRA client updates with Robust PCA into common and client-specific components, averaging the common part and scaled-averaging the sparse part, which improves accuracy and convergence over...

  2. Decouple and Orthogonalize: A Data-Free Framework for LoRA Merging

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A data-free LoRA merging framework that decouples weight magnitude from direction and orthogonalizes directions to reduce task interference, outperforming existing merging methods across vision, language and multimoda...

  3. Semantic-guided LoRA Parameters Generation

    cs.LG 2025-09 conditional novelty 5.0 of 10

    SG-LoRA generates LoRA parameters for unseen tasks from text descriptions alone, using semantic expert selection plus a conditional VAE, matching or exceeding oracle fine-tuning on retrieval benchmarks.

Reference graph

Works this paper leans on

61 extracted references · 51 canonical work pages · cited by 3 Pith papers

  1. [1]

    Dreamcache: Finetuning-free lightweight personalized image generation via feature caching

    Emanuele Aiello, Umberto Michieli, Diego Valsesia, Mete Ozay, and Enrico Magli. Dreamcache: Finetuning-free lightweight personalized image generation via feature caching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 2

    work page 2025

  2. [2]

    Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, and Amit H. Bermano. Domain- agnostic tuning-encoder for fast personalization of text-to- image models. In SIGGRAPH Asia 2023 Conference Papers,

    work page 2023

  3. [3]

    Hypernetwork-Driven Model Fusion for Federated Domain Generalization

    Marc Bartholet, Taehyeon Kim, Ami Beuret, Se-Young Yun, and Joachim M Buhmann. Non-linear fusion in federated learning: A hypernetwork approach to federated domain gen- eralization. arXiv preprint arXiv:2402.06974, 2024. 3

    work page Pith review arXiv 2024

  4. [4]

    A study of parameter efficient fine-tuning by learning to efficiently fine-tune

    Taha Ceritli, Savas Ozkan, Jeongwon Min, Eunchung Noh, Cho Min, and Mete Ozay. A study of parameter efficient fine-tuning by learning to efficiently fine-tune. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024. 3

    work page 2024

  5. [5]

    A brief review of hypernetworks in deep learning

    Vinod Kumar Chauhan, Jiandong Zhou, Ping Lu, Soheila Molaei, and David A Clifton. A brief review of hypernetworks in deep learning. Artificial Intelligence Review, 57(9), 2024. 3

    work page 2024

  6. [6]

    Mllm-as-a-judge: Assessing multi- modal llm-as-a-judge with vision-language benchmark

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Pan Zhou, Yao Wan, and Lichao Sun. Mllm-as-a-judge: Assessing multi- modal llm-as-a-judge with vision-language benchmark. In International Conference on Machine Learning, 2024. 5

    work page 2024

  7. [7]

    Subject-driven text-to-image generation via apprenticeship learning

    Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Rui, Xuhui Jia, Ming-Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning. In Ad- vances in Neural Information Processing Systems, 2024. 2

    work page 2024

  8. [8]

    Civitai: The Home of Open-Source Generative AI

    Civitai. Civitai: The Home of Open-Source Generative AI. https://civitai.com/, 2024. Accessed: November

    work page 2024

Show all 61 references
  1. [9]

    Low-rank adaptation for fast text-to- image diffusion finetuning

    Clonesofimo. Low-rank adaptation for fast text-to- image diffusion finetuning. https://github.com/ cloneofsimo/lora, 2022. Accessed: November 2024. 2

    2022

  2. [10]

    Dreamartist: Towards controllable one-shot text-to-image generation via positive- negative prompt-tuning

    Ziyi Dong, Pengxu Wei, and Liang Lin. Dreamartist: Towards controllable one-shot text-to-image generation via positive- negative prompt-tuning. arXiv preprint arXiv:2211.11337,

    arXiv

  3. [11]

    Implicit style-content separation using b-lora

    Yarden Frenkel, Yael Vinker, Ariel Shamir, and Daniel Cohen- Or. Implicit style-content separation using b-lora. In Euro- pean Conference on Computer Vision, 2024. 2

    2024

  4. [12]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In International Confer- ence on Learning Representations, 2022. 2

    2022

  5. [13]

    Encoder-based domain tuning for fast personalization of text-to-image models

    Rinon Gal, Moab Arar, Yuval Atzmon, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Encoder-based domain tuning for fast personalization of text-to-image models. ACM Transactions on Graphics (TOG), 42(4), 2023. 2

    2023

  6. [14]

    Arcee’s mergekit: A toolkit for merging large language models

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee’s mergekit: A toolkit for merging large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Lang...

    2024

  7. [15]

    Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models

    Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yun- peng Chen, Zihan Fan, WUYOU XIAO, Rui Zhao, Shuning Chang, Weijia Wu, Yixiao Ge, Ying Shan, and Mike Zheng Shou. Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models. InAdvan...

    2023

  8. [16]

    Dai, and Quoc V

    David Ha, Andrew M. Dai, and Quoc V . Le. Hypernetworks. In International Conference on Learning Representations ,

  9. [17]

    Model merging and safety alignment: One bad model spoils the bunch

    Hasan Abed Al Kader Hammoud, Umberto Michieli, Fabio Pizzati, Philip Torr, Adel Bibi, Bernard Ghanem, and Mete Ozay. Model merging and safety alignment: One bad model spoils the bunch. In Findings of the Association for Computa- tional Linguistics: EMNLP 2024, 2024. 2

    2024

  10. [18]

    Svdiff: Compact parame- ter space for diffusion fine-tuning

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dim- itris Metaxas, and Feng Yang. Svdiff: Compact parame- ter space for diffusion fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision ,

  11. [19]

    Vico: Plug-and-play visual condition for personalized text-to- image generation

    Shaozhe Hao, Kai Han, Shihao Zhao, and Kwan-Yee K Wong. Vico: Plug-and-play visual condition for personalized text-to- image generation. arXiv preprint arXiv:2306.00971, 2023. 2

    2023 arXiv

  12. [20]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In Advances in Neural Information Processing Systems, 2020. 1

    2020

  13. [21]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021. 1, 2

    2021

  14. [22]

    Hugging Face – The AI community building the future

    HuggingFace. Hugging Face – The AI community building the future. https://huggingface.co/ , 2024. Ac- cessed: November 2024. 1

    2024

  15. [23]

    Edit- ing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Edit- ing models with task arithmetic. In International Conference on Learning Representations, 2023. 2

    2023

  16. [24]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019. 2

    2019

  17. [25]

    Imagenhub: Standardizing 9 the evaluation of conditional image generation models

    Max Ku, Tianle Li, Kai Zhang, Yujie Lu, Xingyu Fu, Wen- wen Zhuang, and Wenhu Chen. Imagenhub: Standardizing 9 the evaluation of conditional image generation models. In International Conference on Learning Representations, 2024. 5

    2024

  18. [26]

    Multi-concept customization of text- to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shecht- man, and Jun-Yan Zhu. Multi-concept customization of text- to-image diffusion. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2023. 2, 6

    2023

  19. [27]

    Koala: Empirical lessons toward memory-efficient and fast diffusion models for text-to-image synthesis

    Youngwan Lee, Kwanyong Park, Yoorhim Cho, Yong-Ju Lee, and Sung Ju Hwang. Koala: Empirical lessons toward memory-efficient and fast diffusion models for text-to-image synthesis. In Advances in Neural Information Processing Systems, 2024. 8, 15

    2024

  20. [28]

    Blip-diffusion: Pre-trained subject representation for controllable text-to- image generation and editing

    Dongxu Li, Junnan Li, and Steven CH Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to- image generation and editing. In Advances in Neural In- formation Processing Systems, 2024. 2

    2024

  21. [29]

    Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning. In ACM SIGGRAPH 2024 Conference Papers, 2024

    2024

  22. [30]

    Kosmos-g: Generating images in context with multimodal large language models

    Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. Kosmos-g: Generating images in context with multimodal large language models. In International Conference on Learning Representations, 2024

    2024

  23. [31]

    λ-ECLIPSE: Multi-concept personalized text-to-image diffu- sion models by leveraging CLIP latent space

    Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. λ-ECLIPSE: Multi-concept personalized text-to-image diffu- sion models by leveraging CLIP latent space. Transactions on Machine Learning Research, 2024. 2

    2024

  24. [32]

    Dreambench++: A human-aligned bench- mark for personalized image generation

    Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. Dreambench++: A human-aligned bench- mark for personalized image generation. In International Conference on Learning Representations, 2025. 5

    2025

  25. [33]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In International Conference on Learning Representations, 2024. 6

    2024

  26. [34]

    Bootpig: Bootstrapping zero-shot personalized image generation capabilities in pretrained diffusion models

    Senthil Purushwalkam, Akash Gokul, Shafiq Joty, and Nikhil Naik. Bootpig: Bootstrapping zero-shot personalized image generation capabilities in pretrained diffusion models. In European Conference on Computer Vision. Springer, 2024. 2

    2024

  27. [35]

    Rb-modulation: Training-free stylization using reference-based modulation

    L Rout, Y Chen, N Ruiz, A Kumar, C Caramanis, S Shakkot- tai, and W Chu. Rb-modulation: Training-free stylization using reference-based modulation. In International Confer- ence on Learning Representations, 2025. 2

    2025

  28. [36]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 1, ...

    2023

  29. [37]

    HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  30. [38]

    Ziplora: Any subject in any style by effectively merging loras

    Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svetlana Lazebnik, Yuanzhen Li, and Varun Jampani. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, 2024. 1, 2, 4, 5, 6, 7, 13, 14, 15

    2024

  31. [39]

    Styledrop: Text-to-image synthesis of any style

    Kihyuk Sohn, Lu Jiang, Jarred Barber, Kimin Lee, Nataniel Ruiz, Dilip Krishnan, Huiwen Chang, Yuanzhen Li, Irfan Essa, Michael Rubinstein, Yuan Hao, Glenn Entis, Irina Blok, and Daniel Castro Chin. Styledrop: Text-to-image synthesis of any style. In Advances in Neural Informat...

    2023

  32. [40]

    Key-locked rank one editing for text-to-image personalization

    Yoad Tewel, Rinon Gal, Gal Chechik, and Yuval Atzmon. Key-locked rank one editing for text-to-image personalization. In ACM SIGGRAPH 2023 Conference Proceedings, 2023. 2

    2023

  33. [41]

    p+: Extended textual conditioning in text-to-image generation

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. p+: Extended textual conditioning in text-to-image generation. arXiv preprint arXiv:2303.09522, 2023. 2

    2023 arXiv

  34. [42]

    Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 2

    2023

  35. [43]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing infer- ence time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Re- becca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Ko- rnblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing ...

  36. [44]

    Difflora: Generat- ing personalized low-rank adaptation weights with diffusion

    Yujia Wu, Yiming Shi, Jiwei Wei, Chengwei Sun, Yuyang Zhou, Yang Yang, and Heng Tao Shen. Difflora: Generat- ing personalized low-rank adaptation weights with diffusion. arXiv preprint arXiv:2408.06740, 2024. 2

    2024 arXiv

  37. [45]

    Llava- critic: Learning to evaluate multimodal models

    Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. Llava- critic: Learning to evaluate multimodal models. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 5, 6

    2025

  38. [46]

    Break-for-make: Modular low-rank adaptations for composable content-style customization

    Yu Xu, Fan Tang, Juan Cao, Yuxin Zhang, Oliver Deussen, Weiming Dong, Jintao Li, and Tong-Yee Lee. Break-for-make: Modular low-rank adaptations for composable content-style customization. arXiv preprint arXiv:2403.19456, 2024. 2

    2024 arXiv

  39. [47]

    Freetuner: Any subject in any style with training-free diffu- sion

    Youcan Xu, Zhen Wang, Jun Xiao, Wei Liu, and Long Chen. Freetuner: Any subject in any style with training-free diffu- sion. arXiv preprint arXiv:2405.14201, 2024. 2

    2024 arXiv

  40. [48]

    Ties-merging: Resolving interference when merging models

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. In Advances in Neural Information Processing Systems, 2024. 2, 6, 15

    2024

  41. [49]

    Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities

    Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xi- aochun Cao, Jie Zhang, and Dacheng Tao. Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities. arXiv preprint arXiv:2408.07666, 2024. 2

    2024 arXiv

  42. [50]

    Controllable textual 10 inversion for personalized text-to-image generation

    Jianan Yang, Haobo Wang, Yanming Zhang, Ruixuan Xiao, Sai Wu, Gang Chen, and Junbo Zhao. Controllable textual 10 inversion for personalized text-to-image generation. arXiv preprint arXiv:2304.05265, 2023. 2

    2023 arXiv

  43. [51]

    Lora-composer: Leveraging low-rank adapta- tion for multi-concept customization in training-free diffusion models

    Yang Yang, Wen Wang, Liang Peng, Chaotian Song, Yao Chen, Hengjia Li, Xiaolong Yang, Qinglin Lu, Deng Cai, Boxi Wu, et al. Lora-composer: Leveraging low-rank adapta- tion for multi-concept customization in training-free diffusion models. arXiv preprint arXiv:2403.11627, 2024. 2

    2024 arXiv

  44. [52]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  45. [53]

    Language models are super mario: Absorbing abilities from homologous models as a free lunch

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In International Conference on Machine Learning, 2024. 2, 6, 15

    2024

  46. [54]

    Can MLLMs perform text-to-image in-context learning? In Conference on Language Modeling,

    Yuchen Zeng, Wonjun Kang, Yicong Chen, Hyung Il Koo, and Kangwook Lee. Can MLLMs perform text-to-image in-context learning? In Conference on Language Modeling,

  47. [55]

    Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation

    Yu Zeng, Vishal M Patel, Haochen Wang, Xun Huang, Ting- Chun Wang, Ming-Yu Liu, and Yogesh Balaji. Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogni...

    2024

  48. [56]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems, 2023. 5

    2023

  49. [57]

    Cus- tomization assistant for text-to-image generation

    Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, and Tong Sun. Cus- tomization assistant for text-to-image generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  50. [58]

    Toffee: Efficient million-scale dataset construction for subject-driven text-to-image generation

    Yufan Zhou, Ruiyi Zhang, Kaizhi Zheng, Nanxuan Zhao, Jiux- iang Gu, Zichao Wang, Xin Eric Wang, and Tong Sun. Toffee: Efficient million-scale dataset construction for subject-driven text-to-image generation. arXiv preprint arXiv:2406.09305,

    arXiv

  51. [59]

    Option 1

    Yongshuo Zong, Ondrej Bohdal, and Timothy Hospedales. VL-ICL bench: The devil in the details of multimodal in- context learning. In International Conference on Learning Representations, 2025. 5 11 LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditione...

    2025

  52. [60]

    Training subjects: objects (no animals included); Test subjects: stuffed animals; Training styles: 3D renderings; Test styles: cartoon

  53. [61]

    dog2” and “wolf plushie

    Training subjects: animals and stuffed animals; Test subjects: objects; Training styles: watercolor paintings; Test styles: abstract rainbow, wooden sculpture, melting golden rendering. The results are shown in Fig. A10. Despite the challeng- ing setups with no overlap between...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.