REVIEW 45 cited by
DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
With recent advancements in diffusion models, users can generate high-quality images by writing text prompts in natural language. However, generating images with desired details requires proper prompts, and it is often unclear how a model reacts to different prompts or what the best prompts are. To help researchers tackle these critical challenges, we introduce DiffusionDB, the first large-scale text-to-image prompt dataset totaling 6.5TB, containing 14 million images generated by Stable Diffusion, 1.8 million unique prompts, and hyperparameters specified by real users. We analyze the syntactic and semantic characteristics of prompts. We pinpoint specific hyperparameter values and prompt styles that can lead to model errors and present evidence of potentially harmful model usage, such as the generation of misinformation. The unprecedented scale and diversity of this human-actuated dataset provide exciting research opportunities in understanding the interplay between prompts and generative models, detecting deepfakes, and designing human-AI interaction tools to help users more easily use these models. DiffusionDB is publicly available at: https://poloclub.github.io/diffusiondb.
Forward citations
Cited by 45 Pith papers
-
DeepFreqMark: End-To-End Learnable Frequency-Domain Watermarking with Spherical Attack Simulation for Latent Diffusion Models
A learnable frequency-domain watermarking scheme for latent diffusion models uses a Slerp-based attack simulation during training and claims lower BER with up to 256-bit payloads.
-
Navigating the Open-Source Model Ecosystem: An Empirical Study of Creator Practices in Artistic Image Generation
A novel 6M-image Pixiv dataset shows open-source image generation has long-tail model usage, slow life cycles with version inertia, and surging multi-LoRA customization linked to higher engagement.
-
Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes
Text-to-3D models lose prompt sensitivity for out-of-distribution shapes due to sink traps but retain geometric diversity via unconditional priors, enabling a decoupled inversion method for robust editing.
-
Semantic Watermarking Reinvented: Enhancing Robustness and Generation Quality with Fourier Integrity
Enforcing Hermitian symmetry when embedding watermarks in the Fourier domain of latent diffusion models improves detection robustness and image fidelity.
-
Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Space
Prefilled autoregression in a shared embedding space lets Nexus-Gen unify image understanding, generation, and editing, achieving competitive benchmark scores with a 7B model.
-
Multimodal LLMs Can Reason about Aesthetics in Zero-Shot
A zero-shot two-stage prompting baseline (ArtCoT) makes multimodal LLMs' aesthetic judgments align substantially better with human expert rankings in pairwise artwork comparisons.
-
KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers
Using learned 32×32 Kronecker block transforms as an online activation smoother improves W4A4 image quality of PixArt-Sigma, SANA, and FLUX.1-schnell over SVDQuant and LoRaQ, with a kernel up to 14% faster than SmoothQuant.
-
Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision
Diffusion image models can be aligned without human labels by supervising every denoising step with score targets from original versus degraded prompts.
-
Elastic ViTs from Pretrained Models without Retraining
A single-shot, label-free, retraining-free structured pruning method generates elastic ViTs at any sparsity by reweighting gradient-based importance scores with block correlations learned by an evolutionary strategy.
-
TetriServe: Efficiently Serving Mixed DiT Workloads
TetriServe's step-level, deadline-aware sequence parallelism improves SLO attainment for mixed-resolution diffusion transformer serving by up to 32% over fixed-SP systems.
-
Maestro: Self-Improving Text-to-Image Generation via Agent Orchestration
A multi-agent prompt-refinement system using pairwise AI judging and targeted edit signals outperforms prior automated methods on complex text-to-image tasks.
-
Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference
Direct-Align and SRPO fine-tune FLUX using ground-truth-noise recovery and text-conditional relative rewards, improving human-evaluated realism and aesthetics roughly 3x.
-
Understanding and evaluating computer vision models through the lens of counterfactuals
Counterfactual-based methods for concept attribution in classifiers and for dynamic bias evaluation and mitigation in text-to-image models.
-
Guiding Noisy Label Conditional Diffusion Models with Score-based Discriminator Correction
A discriminator trained to distinguish clean from corrupt image-label pairs can be used during sampling to correct the score of a noisy-label conditional diffusion model, improving class-wise fidelity without retraining.
-
4KAgent: Agentic Any Image to 4K Super-Resolution
An agentic pipeline that plans and executes image restoration from a toolbox of pretrained models to upscale arbitrary images to 4K, reporting state-of-the-art results on many benchmarks.
-
Instella-T2I: Pushing the Limits of 1D Discrete Latent Space Image Generation
1D binary image latents reduce a 1024x1024 image to 128 discrete tokens and support text-to-image generation with diffusion and autoregressive models.
-
Ambient Diffusion Omni: Training Good Models with Bad Data
Ambient Diffusion Omni trains diffusion models on mixed-quality data by learning when corrupted images can be treated as clean, improving generation quality and diversity.
-
SEED: A Benchmark Dataset for Sequential Facial Attribute Editing with Diffusion Models
SEED is a 91,526-image benchmark of diffusion-generated sequential facial edits with sequence, mask, and prompt annotations, and FAITH adds DWT high-frequency cues to a transformer for edit-sequence detection.
-
Towards Self-Improvement of Diffusion Models via Group Preference Optimization
Group Preference Optimization (GPO) uses standardized rewards over self-generated image groups to improve diffusion models' counting, text rendering, and prompt alignment without human preference annotations.
-
GIFDL: Generated Image Fluctuation Distortion Learning for Enhancing Steganographic Security
GIFDL trains a GAN-based steganographic distortion model with fluctuation images from a text-to-image generator, raising steganalysis detection error by an average of 3.30% over the GMAN baseline.
-
SketchFlex: Facilitating Spatial-Semantic Coherence in Text-to-Image Generation with Region-Based Sketches
SketchFlex combines sketch-aware prompt recommendation with decompose-and-recompose shape refinement to help novices generate multi-object images from rough region sketches.
-
T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation
T2ISafety is a large annotated benchmark plus a fine-tuned MLLM evaluator (ImageGuard) for measuring toxicity, privacy, and fairness in text-to-image models.
-
Data-Free Group-Wise Fully Quantized Winograd Convolution via Learnable Scales
A data-free method that tunes only the diagonal scales of Winograd transform matrices enables accurate 8-bit fully quantized Winograd convolution for diffusion models and ResNets.
-
Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM
Prompt-A-Video refines text prompts for video diffusion models via evolutionary search and DPO alignment, improving generated video quality on Open-Sora and CogVideoX.
-
Finding a Wolf in Sheep's Clothing: Combating Adversarial Text-To-Image Prompts with Text Summarization
Summarizing LLM-obfuscated text-to-image prompts before classification improves content-moderation F1 scores on the new ATTIP dataset.
-
The Art of Deception: Color Visual Illusions and Diffusion Models
DDIM inversion in diffusion models produces brightness and color shifts that track human visual illusions, and a diffusion-based optimizer can generate new illusions in realistic images that fool human observers.
-
FiVA: Fine-grained Visual Attribute Dataset for Text-to-Image Diffusion Models
The paper introduces FiVA, a ~1M-image synthetic dataset with fine-grained visual attribute labels, and FiVA-Adapter, a diffusion adapter that transfers and combines attributes like lighting, color, and motion from re...
-
The Efficacy of Transfer-based No-box Attacks on Image Watermarking: A Pragmatic Analysis
Transfer-based no-box watermark evasion largely fails without aligned surrogate models, and a simple one-surrogate perturbation (OFT) matches or exceeds the expensive optimization-based attack in 11 of 12 tested confi...
-
Omegance: A Single Parameter for Various Granularities in Diffusion-Based Synthesis
Scaling the denoising noise by one knob, omega, controls granularity in diffusion outputs globally, per region, or per timestep.
-
DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling
DiffServe combines a discriminator-based diffusion model cascade with MILP-based resource allocation to serve text-to-image queries with higher quality and fewer SLO violations.
-
Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward
A learned latent-space surrogate reward enables stable fine-tuning of one/two-step diffusion models with arbitrary, non-differentiable reward signals, outperforming policy-gradient baselines.
-
ComfyGI: Automatic Improvement of Image Generation Workflows
A genetic-improvement search over ComfyUI workflow JSONs, guided by ImageReward, raises median reward scores by about 50% and wins about 90% of human preference comparisons.
-
Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models
The paper introduces Image Regeneration, an evaluation benchmark where text-to-image models must reproduce a reference image from MLLM-generated prompts, along with the ImageRepainter framework and two new datasets.
-
Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
A compact unified model that reuses a frozen VLM encoder and hybrid continuous/discrete tokens reaches competitive image understanding and generation with 15.6M training images and about $2,000 in compute.
-
Cost-Aware Routing for Efficient Text-To-Image Generation
A cost-aware router selects per prompt the best among nine pre-trained text-to-image models, beating every single model on the quality-versus-cost frontier.
-
Exploring Language Patterns of Prompts in Text-to-Image Generation and Their Impact on Visual Diversity
User prompts on CivitAI became more lexically repetitive over seven months, and higher prompt similarity was associated with lower visual diversity in generated images.
-
EliGen: Entity-Level Controlled Image Generation with Regional Attention
EliGen adds entity-level control to FLUX by composing attention masks that bind each entity prompt to its spatial region, then fine-tunes with LoRA on 500k synthetic annotated images.
-
VersaGen: Unleashing Versatile Visual Control for Text-to-Image Synthesis
VersaGen enables users to control text-to-image diffusion models with partial sketch inputs at object and scene levels, with automatic localization and adaptive control strength.
-
Attention Dynamics in Diffusion Models: A Visual Analytics Framework for Human-AI Collaboration
A DAAM-based visual analytics workflow links step-resolved token attention trajectories, phase summaries, and spatial competition maps for Stable Diffusion-class models on a 60-prompt benchmark.
-
NTIRE 2025 challenge on Text to Image Generation Model Quality Assessment
The NTIRE 2025 challenge report compares 20 methods for fine-grained text-to-image quality assessment, introduces the EvalMuse-Structure dataset, and finds every participating team outperformed the baselines.
-
DejAIvu: Identifying and Explaining AI Art on the Web in Real-Time with Saliency Maps
DejAIvu is a browser extension that classifies images as AI-generated or human-made in real time and highlights the evidence with saliency heatmaps.
-
Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback
The paper proposes learning from language feedback to synthesize multimodal preference pairs, but the evidence is weakened by an undefined improvement metric and small, unvalidated effect sizes.
-
Detecting Facial Image Manipulations with Multi-Layer CNN Models
Extending MesoNet to six convolutional layers and swapping the deepfake training images for Stable Diffusion images raises three-class accuracy from 43% to 76% on a small in-distribution dataset, without external validation.
-
Artificial Intelligence Policy Framework for Institutions
A proposed AI governance flowchart for institutions, assembled from standard principles, without evidence that it works.
-
Visual question answering based evaluation metrics for text-to-image generation
A text-to-image evaluation metric that combines ChatGPT-generated yes/no questions answered by a VQA model with a no-reference image quality score, with adjustable weights.
Discussion (0). Continue with ORCID to comment.