REVIEW 4 cited by
Non-confusing Generation of Customized Concepts in Diffusion Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We tackle the common challenge of inter-concept visual confusion in compositional concept generation using text-guided diffusion models (TGDMs). It becomes even more pronounced in the generation of customized concepts, due to the scarcity of user-provided concept visual examples. By revisiting the two major stages leading to the success of TGDMs -- 1) contrastive image-language pre-training (CLIP) for text encoder that encodes visual semantics, and 2) training TGDM that decodes the textual embeddings into pixels -- we point that existing customized generation methods only focus on fine-tuning the second stage while overlooking the first one. To this end, we propose a simple yet effective solution called CLIF: contrastive image-language fine-tuning. Specifically, given a few samples of customized concepts, we obtain non-confusing textual embeddings of a concept by fine-tuning CLIP via contrasting a concept and the over-segmented visual regions of other concepts. Experimental results demonstrate the effectiveness of CLIF in preventing the confusion of multi-customized concept generation.
Forward citations
Cited by 4 Pith papers
-
TAP: Parameter-efficient Task-Aware Prompting for Adverse Weather Removal
A two-stage prompt-tuning method with low-rank and contrastive prompt enhancement claims all-in-one adverse weather removal at 2.75M parameters.
-
HOComp: Interaction-Aware Human-Object Composition
A diffusion-transformer method that composes a foreground object into a human image with MLLM-chosen interaction regions, pose keypoint supervision, and appearance/background consistency losses, plus a new paired dataset.
-
Show and Polish: Reference-Guided Identity Preservation in Face Video Restoration
IP-FVR restores degraded face videos with consistent identity by conditioning a video diffusion model on a reference photo of the same person.
-
IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models
A Gaussian-path transition equation lets a pretrained Stable Diffusion model serve as the denoiser inside image restoration bridges, cutting per-task training to a lightweight ControlNet.
Discussion (0). Sign in to comment.