REVIEW 6 cited by
Improving Compositional Attribute Binding in Text-to-Image Generative Models via Enhanced Text Embeddings
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Text-to-image diffusion-based generative models have the stunning ability to generate photo-realistic images and achieve state-of-the-art low FID scores on challenging image generation benchmarks. However, one of the primary failure modes of these text-to-image generative models is in composing attributes, objects, and their associated relationships accurately into an image. In our paper, we investigate compositional attribute binding failures, where the model fails to correctly associate descriptive attributes (such as color, shape, or texture) with the corresponding objects in the generated images, and highlight that imperfect text conditioning with CLIP text-encoder is one of the primary reasons behind the inability of these models to generate high-fidelity compositional scenes. In particular, we show that (i) there exists an optimal text-embedding space that can generate highly coherent compositional scenes showing that the output space of the CLIP text-encoder is sub-optimal, and (ii) the final token embeddings in CLIP are erroneous as they often include attention contributions from unrelated tokens in compositional prompts. Our main finding shows that significant compositional improvements can be achieved (without harming the model's FID score) by fine-tuning only a simple and parameter-efficient linear projection on CLIP's representation space in Stable-Diffusion variants using a small set of compositional image-text pairs.
Forward citations
Cited by 6 Pith papers
-
ActionParty: Multi-Subject Action Binding in Generative Video Games
ActionParty binds discrete actions to individual subjects in a single generated video by jointly modeling subject state tokens and video latents, controlling up to seven players across 46 Melting Pot games.
-
Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds
RTD, a training-free method, improves multi-concept text-to-image generation by applying a single bounded gradient update to the initial latent to reduce overlap between concept attention maps.
-
RADIANCE: Relative Adaptive Denoising with IP-Adapter for Novel Concept Enhancement
A closed-loop, training-free controller uses CLIP similarity feedback and bidirectional IP-Adapter scales to keep rare attributes and base objects balanced throughout the diffusion trajectory.
-
Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
A frozen VLM's dual-query Yes/No log-odds act as a differentiable semantic-and-spatial critic, improving alignment and geometry in both SDS-based and feed-forward text-to-3D pipelines.
-
MaskAttn-SDXL: Controllable Region-Level Text-To-Image Generation
MaskAttn-SDXL adds token-conditioned spatial gating to SDXL cross-attention to sparsify irrelevant token-to-location bindings and improve region-level controllability without retraining or inference edits.
-
MaskAttn-SDXL: Controllable Region-Level Text-To-Image Generation
MaskAttn-SDXL adds learned binary token-location gates before softmax in SDXL cross-attention, improving compositional consistency on multi-object prompts.
Discussion (0). Sign in to comment.