Pith. sign in

REVIEW 9 cited by

Training-Free Layout Control with Cross-Attention Guidance

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.03373 v2 pith:Z6SFWLQJ submitted 2023-04-06 cs.CV

Minghao Chen , Iro Laina , Andrea Vedaldi This is my paper · ORCID
classification cs.CV
keywords layoutguidancetextualapproachattentionbackwardcontrolcross-attention
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Recent diffusion-based generators can produce high-quality images from textual prompts. However, they often disregard textual instructions that specify the spatial layout of the composition. We propose a simple approach that achieves robust layout control without the need for training or fine-tuning of the image generator. Our technique manipulates the cross-attention layers that the model uses to interface textual and visual information and steers the generation in the desired direction given, e.g., a user-specified layout. To determine how to best guide attention, we study the role of attention maps and explore two alternative strategies, forward and backward guidance. We thoroughly evaluate our approach on three benchmarks and provide several qualitative examples and a comparative analysis of the two strategies that demonstrate the superiority of backward guidance compared to forward guidance, as well as prior work. We further demonstrate the versatility of layout guidance by extending it to applications such as editing the layout and context of real images.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features

    cs.CV 2025-02 conditional novelty 7.0 of 10

    ConceptAttention shows that linear projections in the output space of DiT attention layers yield sharper concept-localizing saliency maps than cross-attention maps, reaching state-of-the-art zero-shot segmentation.

  2. Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers

    cs.CV 2025-06 conditional novelty 6.0 of 10

    TACA scales cross-modal attention logits by a timestep-dependent temperature to rebalance text and visual tokens, improving T2I-CompBench alignment on FLUX and SD3.5.

  3. ISAC: Training-Free Instance-to-Semantic Attention Control for Multi-Instance Generation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    ISAC improves multi-instance image generation by carving out instance regions from self-attention first and then assigning semantics to those regions.

  4. Mojito: Motion Trajectory and Intensity Control for Video Generation

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Mojito enables both trajectory and intensity control in text-to-video generation by combining training-free cross-attention guidance with optical-flow-conditioned motion intensity embeddings.

  5. All Seeds Are Not Equal: Enhancing Compositional Text-to-Image Generation with Reliable Random Seeds

    cs.CV 2024-11 conditional novelty 6.0 of 10

    Certain random seeds yield consistently more accurate compositional text-to-image outputs, and mining these seeds plus fine-tuning on the resulting self-generated images improves numerical and spatial composition accuracy.

  6. AnySynth: Harnessing the Power of Image Synthetic Data Generation for Generalized Vision-Language Tasks

    cs.CV 2024-11 conditional novelty 5.0 of 10

    AnySynth is a single synthetic-data pipeline that produces layouts, images, and annotations for multiple vision tasks, and its data improves benchmark scores by small but consistent margins.

  7. QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation

    cs.CV 2025-07 conditional novelty 4.0 of 10

    QR-LoRA freezes the QR-decomposed basis of pretrained weights, trains only a residual matrix, and reports halved trainable parameters with improved content-style disentanglement in diffusion models.

  8. SmartSpatial: Enhancing the 3D Spatial Arrangement Capabilities of Stable Diffusion Models and Introducing a Novel 3D Spatial Evaluation Framework

    cs.CV 2025-01 reject novelty 4.0 of 10

    SmartSpatial combines depth injection and attention guidance in Stable Diffusion with a new VLM-based spatial metric, but reported improvements are not statistically significant per the paper's own p-value statement.

  9. LocRef-Diffusion:Tuning-Free Layout and Appearance-Guided Generation

    cs.CV 2024-11 conditional novelty 4.0 of 10

    LocRef-Diffusion inserts two lightweight cross-attention modules into Stable Diffusion to control both layout and appearance of multiple instances without fine-tuning per object.

Pith tools