REVIEW 13 cited by
LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Customization generation techniques have significantly advanced the synthesis of specific concepts across varied contexts. Multi-concept customization emerges as the challenging task within this domain. Existing approaches often rely on training a fusion matrix of multiple Low-Rank Adaptations (LoRAs) to merge various concepts into a single image. However, we identify this straightforward method faces two major challenges: 1) concept confusion, where the model struggles to preserve distinct individual characteristics, and 2) concept vanishing, where the model fails to generate the intended subjects. To address these issues, we introduce LoRA-Composer, a training-free framework designed for seamlessly integrating multiple LoRAs, thereby enhancing the harmony among different concepts within generated images. LoRA-Composer addresses concept vanishing through concept injection constraints, enhancing concept visibility via an expanded cross-attention mechanism. To combat concept confusion, concept isolation constraints are introduced, refining the self-attention computation. Furthermore, latent re-initialization is proposed to effectively stimulate concept-specific latent within designated regions. Our extensive testing showcases a notable enhancement in LoRA-Composer's performance compared to standard baselines, especially when eliminating the image-based conditions like canny edge or pose estimations. Code is released at \url{https://github.com/Young98CN/LoRA_Composer}
Forward citations
Cited by 13 Pith papers
-
AutoLoRA: Automatic LoRA Retrieval and Fine-Grained Gated Fusion for Text-to-Image Generation
AutoLoRA uses a learned encoder to map LoRA weights and text prompts into one embedding space for retrieval, and then fuses the retrieved LoRAs with per-dimension learned gates during generation.
-
FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization
A method for multi-subject image personalization that fuses independently trained LoRA modules at inference time on visual autoregressive models.
-
Decouple and Orthogonalize: A Data-Free Framework for LoRA Merging
A data-free LoRA merging framework that decouples weight magnitude from direction and orthogonalizes directions to reduce task interference, outperforming existing merging methods across vision, language and multimoda...
-
Cached Multi-Lora Composition for Multi-Concept Image Generation
CMLoRA schedules adapter activation by high- and low-frequency content and caches non-dominant adapters, improving multi-LoRA composition scores while not consistently reducing compute versus all baselines.
-
DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder
DreamFit generates human images from a garment reference and text by encoding the reference through LoRA-activated layers of a frozen Stable Diffusion UNet and injecting features with adaptive attention.
-
GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generators
GCA-3D adapts 3D generators to text or one-shot image domains without dataset synthesis, using depth-aware score distillation and hierarchical spatial consistency losses.
-
LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image Generation
A hypernetwork pretrained on pairs of subject and style LoRAs predicts column-wise merging coefficients, enabling real-time, high-quality joint subject-style image personalization.
-
From Wardrobe to Canvas: Wardrobe Polyptych LoRA for Part-level Controllable Human Image Generation
Wardrobe Polyptych LoRA lets a single diffusion model compose a person's face and clothing from multiple reference photos into new full-body images, generalizing to unseen identities without inference-time fine-tuning.
-
LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusion
LiON-LoRA adds a learned scaling token to video-diffusion LoRA adapters, enabling linear and independent control of camera trajectory and object motion strength.
-
Multi LoRA Meets Vision: Merging multiple adapters to create a multi task model
Merging LoRA adapters across six vision tasks produces usable multitask models, with performance depending strongly on dataset similarity.
-
3D Object Manipulation in a Single Image using Generative Models
A single-image object manipulation framework that reconstructs an object in 3D, refines its texture with a custom-tuned diffusion model, corrects background lighting, and renders the edited or animated object back int...
-
Efficient Diffusion Models: A Survey
The paper organizes research on efficient diffusion models into a taxonomy spanning algorithms, systems, and frameworks, and provides a curated reference list.
-
Parameter-Efficient Fine-Tuning for Foundation Models
A survey that categorizes and summarizes parameter-efficient fine-tuning methods across large language, vision, and multimodal models.
Discussion (0). Continue with ORCID to comment.