K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs
read the original abstract
Recent studies have explored combining different LoRAs to jointly generate learned style and content. However, existing methods either fail to effectively preserve both the original subject and style simultaneously or require additional training. In this paper, we argue that the intrinsic properties of LoRA can effectively guide diffusion models in merging learned subject and style. Building on this insight, we propose K-LoRA, a simple yet effective training-free LoRA fusion approach. In each attention layer, K-LoRA compares the Top-K elements in each LoRA to be fused, determining which LoRA to select for optimal fusion. This selection mechanism ensures that the most representative features of both subject and style are retained during the fusion process, effectively balancing their contributions. Experimental results demonstrate that the proposed method effectively integrates the subject and style information learned by the original LoRAs, outperforming state-of-the-art training-based approaches in both qualitative and quantitative results.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
OrthoFuse: Training-free Riemannian Fusion of Orthogonal Style-Concept Adapters for Diffusion Models
Training-free Riemannian fusion merges orthogonal style and concept adapters for diffusion models via geodesic approximation on GS matrices plus spectra restoration.
-
LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
LoRA-Mixer routes modular LoRA experts into attention projection matrices with an adaptive Routing Specialization Loss to improve multi-task performance while using fewer trainable parameters than prior LoRA-MoE methods.
-
Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization
Disco-LoRA proposes disentangling content-style and content-motion via dual-LoRA with statistical regularization to enable multi-concept video customization.
-
AD-Relight: Training-Free Banner Relighting via Illumination Translation with Diffusion Priors
AD-Relight adapts diffusion-based relighting models at test time via a multi-stage framework to relight custom ad banners so they match scene illumination, outperforming warping and prior relighting approaches.
-
FREE-Switch: Frequency-based Dynamic LoRA Switch for Style Transfer
FREE-Switch dynamically switches LoRA adapters using frequency importance per diffusion step and adds semantic alignment to reduce content drift when merging specialized image generators.
-
CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Models
CraftGraffiti applies LoRA-tuned diffusion transformers followed by identity-augmented self-attention and CLIP-guided pose extension to generate graffiti while preserving facial features.
-
Context Recycling for Long-Horizon LLM Inference
ContextForge recycles context in long-horizon LLM tasks via query generation, memory retrieval, and synthesis, yielding reduced token use and improved consistency on a 15-turn healthcare benchmark while preserving accuracy.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.