REVIEW 32 cited by
Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce Playground v3 (PGv3), our latest text-to-image model that achieves state-of-the-art (SoTA) performance across multiple testing benchmarks, excels in graphic design abilities and introduces new capabilities. Unlike traditional text-to-image generative models that rely on pre-trained language models like T5 or CLIP text encoders, our approach fully integrates Large Language Models (LLMs) with a novel structure that leverages text conditions exclusively from a decoder-only LLM. Additionally, to enhance image captioning quality-we developed an in-house captioner, capable of generating captions with varying levels of detail, enriching the diversity of text structures. We also introduce a new benchmark CapsBench to evaluate detailed image captioning performance. Experimental results demonstrate that PGv3 excels in text prompt adherence, complex reasoning, and accurate text rendering. User preference studies indicate the super-human graphic design ability of our model for common design applications, such as stickers, posters, and logo designs. Furthermore, PGv3 introduces new capabilities, including precise RGB color control and robust multilingual understanding.
Forward citations
Cited by 32 Pith papers
-
WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing
WeEdit trains a glyph-guided, RL-optimized image editor on a 330K-pair synthetic multilingual dataset and reports open-source SOTA on its own bilingual and multilingual text-editing benchmarks.
-
A Comprehensive Study of Decoder-Only LLMs for Text-to-Image Generation
Layer-normalized averaging of all decoder-only LLM hidden states, rather than last-layer embeddings, improves text-to-image compositional alignment and beats T5 on GenAI-Bench.
-
Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images
Stem uses a conditional diffusion model to infer spatially resolved gene expression from H&E histology images, outperforming regression baselines on several datasets.
-
EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM
EasyRef uses a multimodal LLM to encode multiple reference images and a text prompt into learned tokens, enabling zero-shot multi-reference personalization of Stable Diffusion XL without per-group fine-tuning.
-
Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
A lightweight Q-Former proxy trained on VLM hidden states reveals that localization signals peak in input-dependent intermediate layers, not the final layers used by standard editing pipelines.
-
SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices
A compact elastic diffusion transformer with adaptive sparse attention and knowledge-guided distribution-matching distillation achieves 4-step 1K image generation on a phone in roughly 1.8 seconds.
-
Reconstruction Alignment Improves Unified Multimodal Models
RECA, a self-supervised post-training objective that conditions unified multimodal models on their own visual understanding embeddings to reconstruct input images, improves text-to-image and editing benchmarks across ...
-
SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning
An RL framework that trains vision-language models to self-correct captions via a scene-graph-based reward outperforms SFT and DPO on caption quality.
-
PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement
PoemTale Diffusion generates a coherent set of images from a poem by combining emotion-based segmentation, multi-stage LLM prompt refinement, and consistent self-attention, outperforming direct poem-to-image approache...
-
VCapsBench: A Large-scale Fine-grained Benchmark for Video Caption Quality Evaluation
VCapsBench is a video caption quality benchmark with 109,796 QA pairs across 21 fine-grained dimensions on 5,677 videos, evaluating caption accuracy, inconsistency, and coverage.
-
RICO: Improving Accuracy and Completeness in Image Recaptioning via Visual Reconstruction
RICO refines image captions by reconstructing them into images with a text-to-image model and asking GPT-4o to fix discrepancies against the original, iteratively, with a DPO-distilled fast variant.
-
RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning
RePrompt uses RL-trained reasoning traces to enhance text-to-image prompts, boosting spatial composition and counting scores across FLUX, SD3, and PixArt-Σ while keeping image generators fixed.
-
I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models
ThinkDiff aligns vision-language model features to a T5 decoder via captioning, then injects those features into a T5-based diffusion decoder, achieving 46.3% on the CoBSAT benchmark without reasoning-specific training data.
-
MSF: Efficient Diffusion Model Via Multi-Scale Latent Factorize
MSF factorizes the diffusion denoising target into a low-frequency base and a high-frequency residual, generating them sequentially to improve FID and cut sampling cost by roughly 4x over DiT.
-
Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models
Aligning a VAE's latent space with DINOv2 features resolves the reconstruction-generation trade-off in latent diffusion, enabling faster DiT training and a state-of-the-art ImageNet FID of 1.35.
-
1.58-bit FLUX
A post-training method reduces 99.5% of FLUX.1-dev's transformer weights to ternary values and reports roughly comparable text-to-image quality with large storage and memory savings.
-
SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training
SnapGen is a 379M-parameter UNet with cross-architecture distillation and a 1.38M-parameter decoder that generates 1024x1024 images on a phone in about 1.4 seconds, with GenEval 0.66 and ImageNet FID 2.06.
-
Chimera: Improving Generalist Model with Domain-Specific Experts
Chimera fuses frozen domain-expert encoders into a generalist multimodal LLM via routing and a 30% masking of general tokens, lifting InternVL2-8B from 61.6 to 64.9 on MathVista and from 31.3 to 32.4 on MathVerse.
-
CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation
A siamese-branch layout adapter lets multimodal diffusion transformers follow detailed region captions and bounding boxes, beating prior layout-to-image methods on a new 2.7M-pair dataset and benchmark.
-
IQA-Adapter: Exploring Knowledge Transfer from Image Quality Assessment to Diffusion-based Generative Models
An adapter module trained on IQA/IAA scores gives SDXL controllable quality-aware generation, improving perceived quality and enabling reference-based distortion transfer.
-
DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer
DC-AR generates 512x512 images in 12 masked autoregressive steps plus 20 diffusion refinement steps, using a 32x compressed 2D tokenizer, and reports gFID 5.49 on MJHQ-30K.
-
CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation
CycleVAR adapts a pretrained visual autoregressive model to unpaired image translation using softmax-relaxed quantization and source-token prefixes, achieving FID scores competitive with CycleGAN-Turbo.
-
PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Framework
PosterCraft improves text-to-poster generation by cascading four stages of training (text rendering, region-weighted fine-tuning, preference optimization, and vision-language feedback), outperforming open-source basel...
-
Ultra-High-Resolution Image Synthesis: Data, Method and Evaluation
Diffusion-4K combines a scale-consistent VAE and a wavelet-based fine-tuning loss to generate 4K images, though the wavelet loss is equivalent to the standard flow-matching loss.
-
Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models
Normalized Attention Guidance (NAG) stabilizes attention-space extrapolation with L1 normalization and refinement, restoring negative prompting in few-step diffusion models across architectures and modalities.
-
Harnessing Caption Detailness for Data-Efficient Text-to-Image Generation
A detailness score combining object coverage and per-object description depth selects 20% of captions that train a text-to-image model better than the full dataset.
-
SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer
SANA-1.5 combines layer growth, depth pruning, and VLM-judged best-of-N sampling to push GenEval text-to-image alignment from 0.81 to 0.96.
-
AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generation
AnyStory introduces a unified feed-forward approach for single and multi-subject text-to-image personalization using a simplified ReferenceNet and CLIP encoder, plus a decoupled instance-aware router.
-
UNIC-Adapter: Unified Image-instruction Adapter with Multi-modal Transformer for Image Generation
A unified adapter for Stable Diffusion 3 that enables one model to follow conditional images and task instructions across spatial, subject, and style control tasks.
-
X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models
X-Prompt compresses in-context image examples into a few learned tokens and adds text-description tasks, enabling a Chameleon-style autoregressive model to handle multiple image generation tasks in one framework.
-
Evaluating Uncertainty and Quality of Visual Language Action-enabled Robots
Across three VLA models and four simulated manipulation tasks, motion-instability and goal-distance metrics correlate with expert-rated execution quality, showing that binary success rates hide large quality differences.
-
Towards Evaluating Robustness of Prompt Adherence in Text to Image Models
New benchmark results show that Stable Diffusion 3.x and Janus Pro models struggle to place simple geometric shapes in the correct image quadrant, with best F1 scores around 0.41 to 0.5.
Discussion (0). Continue with ORCID to comment.