Pith. sign in

REVIEW 4 major objections 5 minor 11 cited by

This paper introduces a 6-million-image text-to-image dataset whose chain-of-thought annotations are meant to teach generation reasoning, alongside a seven-track benchmark that measures where current models still fail.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-04 18:43 UTC pith:YJGSNUSL

load-bearing objection A substantial open dataset and benchmark, but the paper's main claim—that GCoT supervises reasoning—is asserted, not demonstrated; still worth a serious referee. the 4 major comments →

arxiv 2509.09680 v1 pith:YJGSNUSL submitted 2025-09-11 cs.CV cs.CL

FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark

classification cs.CV cs.CL
keywords text-to-image generationchain-of-thoughtlarge-scale datasetbenchmarkreasoningvision-language modelsevaluationbilingual captions
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

FLUX-Reason-6M is this paper's attempt to give open text-to-image models what they lack: a million-scale training resource built to teach complex reasoning, not just image-text association. Each of its 6 million images is annotated with a generation chain-of-thought, a detailed prose breakdown of how and why the scene is constructed, organized around six overlapping characteristics: imagination, entity, text rendering, style, affection, and composition. The companion benchmark, PRISM-Bench, adds a seventh long-text track built from these chain-of-thought prompts and uses advanced vision-language models to score prompt-image alignment and aesthetics. In evaluations of 19 models, the paper reports that long instructions and text rendering are the hardest tasks for every model, and that the gap between open and closed-source systems is widening. If the claims hold, this would be the first open resource at industrial scale for teaching generation reasoning.

Core claim

The central claim, stated in the paper's own terms, is that explicit generation chain-of-thought—a step-by-step narrative explaining the compositional, stylistic, and semantic logic of an image—provides a training signal that can teach text-to-image models to reason about complex scenes. Unlike ordinary captions, GCoT descriptions reveal how and why an image is put together, and because the six characteristics intentionally overlap, a single image can carry multiple reasoning lessons at once. The paper further claims that PRISM-Bench translates these reasoning dimensions into a measurement standard: seven tracks of 100 prompts each, scored by vision-language models on alignment and aesthetic

What carries the argument

Generation chain-of-thought (GCoT): a dense, multi-sentence description, synthesized by a vision-language model from an image and its category-specific captions, that deconstructs the image layer by layer—spatial relationships, artistic choices, color harmonies, emotional undertones, compositional balance. GCoT is the dataset's cornerstone training signal and the source of PRISM-Bench's long-text track; the benchmark's evaluation protocol is the second critical piece, using track-specific prompts to make a vision-language model judge alignment and aesthetics separately for each of the seven tracks.

Load-bearing premise

The dataset and benchmark rest on the assumption that vision-language model judgments—used to filter image quality and to score generated images—are a reliable proxy for human judgment, with no human-correlation study reported.

What would settle it

Take a random sample of the 700 PRISM-Bench generated images, have human raters score the same alignment and aesthetic questions, and compute the rank correlation with the VLM scores; if the correlation is weak, the benchmark's ability to measure true capability collapses. Alternatively, train two models on identical images—one with GCoT captions and one without—and compare long-text performance; if GCoT gives no measurable gain, the dataset's core premise is disproved.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Open-source text-to-image models gain access to a training resource at a scale previously confined to industrial labs.
  • The benchmark identifies long-instruction following and text rendering as universal weaknesses, pointing model developers to the areas that most need improvement.
  • The bilingual (English and Chinese) release, with English text preserved in the text-rendering captions, may let Chinese-capable models improve typographic control without losing accuracy.
  • If adopted widely, PRISM-Bench offers a common 700-prompt standard that makes model-to-model comparisons more meaningful than ad hoc image-text similarity scores.
  • The reported widening gap between open and closed models provides a concrete target for future open-source development.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Testable extension: fine-tune a base model on a GCoT-annotated subset of the dataset and on an equal-sized set of plain captions; if GCoT is doing the work, the GCoT-trained model should improve disproportionately on long-text prompts.
  • The benchmark's ranking depends entirely on vision-language model judgment, with no reported human correlation; a human-rated subset of the 700 prompts would settle whether these scores track human preference.
  • Because the six categories overlap by design, the dataset can also be used to study how reasoning dimensions co-occur—for instance, how emotional tone and spatial composition reinforce each other in training.
  • The text-preserving translation strategy could be extended beyond Chinese to any language, potentially improving multilingual text rendering in image generators.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces FLUX-Reason-6M, a synthesized dataset of 6 million FLUX.1-dev images with 20 million bilingual (English/Chinese) captions, organized by six characteristics (Imagination, Entity, Text rendering, Style, Affection, Composition) and augmented with Generation Chain-of-Thought (GCoT) annotations. It also presents PRISM-Bench, a seven-track benchmark of 700 prompts evaluated by GPT-4.1 and Qwen2.5-VL-72B on prompt-image alignment and aesthetics. The authors report evaluations of 19 text-to-image models, finding that closed-source models lead overall and that text rendering and long-text instruction following remain the hardest tasks. The dataset, benchmark, and evaluation code are released.

Significance. The scale and public release of a bilingual, multi-label synthesized dataset is a potentially valuable community resource, and PRISM-Bench covers dimensions (Imagination, Affection) that are underrepresented in prior T2I benchmarks. The paper also provides a broad, up-to-date comparison of 19 models. However, the central intellectual claim—that GCoT captions provide effective intermediate supervision for teaching T2I reasoning—is asserted rather than demonstrated, and the benchmark's reliability as a human-aligned evaluation tool is not validated. If the missing training evidence and human correlation studies are supplied, the resources could be significant; as presented, the claims outrun the experiments.

major comments (4)
  1. [§2.1 and §2.4] The paper's core claim is that GCoT captions "provide powerful intermediate supervisory signals for training" (§2.1) and are "the main contribution and key feature" (§2.4). Yet no experiment in §4 or §5 trains or fine-tunes any T2I model on FLUX-Reason-6M, nor ablates GCoT against category-specific or raw captions. The benchmark tables evaluate existing models and say nothing about whether training on this corpus transfers to better reasoning. At minimum, a small-scale fine-tuning study (e.g., on FLUX.1-dev or SDXL) with GCoT versus non-GCoT captions is needed to substantiate the cornerstone mechanism.
  2. [§2.3] The quality filtering and multidimensional classification use Qwen-VL scores with "carefully set calibrated thresholds," but the thresholds are never reported. This makes the 8M→6M filtering step irreproducible and the claim of "6 million high-quality images" unverifiable. In addition, the relevance scores are not validated against human ratings; without such validation, the multi-label category assignments that drive both dataset curation and benchmark sampling rest on an unmeasured accuracy.
  3. [§3.2] PRISM-Bench's evaluation relies on "VLM's advanced cognitive abilities as a proxy for human judgment," but no human correlation study is reported and no confidence intervals or inter-annotator agreement are given for the GPT-4.1/Qwen2.5-VL scores. In Tables 1 and 2, top models are separated by less than one point (e.g., Qwen-Image 79.9 vs. SEEDream 3.0 79.6 in Table 1; GPT-Image-1 80.7 vs. Gemini2.5 80.4 in Table 2). Without uncertainty quantification, the benchmark's claimed discriminative reliability is not established. Please report bootstrapped confidence intervals and a human-agreement study on a subset of prompts.
  4. [§3.1 and Figure 5] The benchmark is not independent of the dataset it is meant to evaluate. 50 of 100 prompts per track are sampled from FLUX-Reason-6M, and the Long Text track is built from GCoT captions in the same dataset. Moreover, evaluation uses Qwen2.5-VL-72B, the same model family that generated the dataset annotations. This creates a circularity risk: models trained on FLUX-Reason-6M could appear advantaged on PRISM-Bench merely because prompts and evaluator are drawn from the same distribution. The paper should quantify the overlap, justify why this does not bias the benchmark, and ideally include held-out prompts not derived from the dataset.
minor comments (5)
  1. [Abstract and Figure 1] Typographical issues: "We introduceFLUX-Reason-6M" and "The image are organized" in the abstract; "SEEDreeam" in Figure 1.
  2. [Figure 4] The prompt-length panel lacks axis labels and a clear caption; the source-percentage panel would benefit from a note on how the three sources are defined.
  3. [Tables 1-2] Scores are reported without error bars. Also, Bagel and Bagel-CoT are identical in Table 2 for all seven tracks; please clarify whether this is intentional or a data/copying error.
  4. [§2.5] The text says "20 million unique captions" while the abstract says "20 million associated captions." Please clarify whether the count is unique captions or total caption-image pairs, and define the deduplication rule.
  5. [References] Inconsistent naming of "Laion-aesthetic" vs. "Laion-Aesthetics"; some URLs are not properly formatted (e.g., [46]).

Circularity Check

0 steps flagged

No significant circularity: the dataset and benchmark are constructed artifacts, and the cited self-references are not load-bearing.

full rationale

FLUX-Reason-6M and PRISM-Bench are constructed datasets and evaluation protocols, not derivations that reduce predictions to fitted inputs. The GCoT captions are produced post-hoc by a VLM from images and category captions (Section 2.4), so the claim that they provide powerful supervisory signals is empirically unvalidated, but it is not circular: the dataset does not fit a parameter and then predict the captions from the target of the claim. PRISM-Bench samples 50 of 100 prompts per track from FLUX-Reason-6M and uses GCoT annotations for the Long Text track; this makes the benchmark distributionally related to the dataset, but the benchmark's target is T2I model evaluation, and no benchmark score is defined as an output of the dataset's construction process. Using Qwen2.5-VL-72B for evaluation while the same model family was used for annotation is a potential bias or validity concern, not a circular reduction, because there is no equation or calibrated parameter that forces evaluation scores to equal annotation outputs. Self-citations to GoT [18] and T2I-ReasonBench [57] are used only to position prior limitations and not as load-bearing evidence for the present claims. The main weakness is missing causal validation (no training experiment showing GCoT improves T2I reasoning), which is a correctness risk, not circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claims rest on hand-chosen thresholds, unvalidated VLM judgments, and an untested assumption that GCoT transcripts provide useful training signal. There are no new physical or conceptual entities beyond the GCoT data format, which is a methodological artifact rather than a postulated entity.

free parameters (5)
  • Characteristic relevance thresholds = not reported
    Qwen-VL assigns 1-10 relevance scores for each of the six characteristics; thresholds for assigning categories are 'carefully set' but never stated (Section 2.3).
  • Quality filtering thresholds = not reported
    Clarity and structural consistency filters discard images based on unspecified criteria and cutoffs (Section 2.3).
  • Original caption retention threshold = not reported
    An alignment score threshold decides which Laion captions are retained alongside FLUX images; the value is not given (Section 2.5).
  • Number of clusters per PRISM track = k=50
    K-Means clustering uses k=50 groups per track for representative prompt sampling, without justification for the choice (Section 3.1).
  • Qwen3 creative expansion temperature = high, unspecified
    Temperature is increased to encourage creativity in imagination prompt generation, but the exact value is not reported (Section 2.2).
axioms (5)
  • domain assumption Qwen-VL provides accurate judgments of image clarity, structural consistency, characteristic relevance, and caption-image alignment.
    Used throughout Section 2.3 filtering and Section 2.5 integration; no validation against human labels is reported.
  • domain assumption GPT-4.1 and Qwen2.5-VL-72B scores align with human perception of prompt-image alignment and aesthetics.
    Section 3.2 states VLM judgment is a 'proxy for human judgment' but presents no human correlation study.
  • ad hoc to paper The six characteristics (Imagination, Entity, Text rendering, Style, Affection, Composition) comprehensively cover T2I reasoning.
    Introduced in Section 2.1 without empirical or theoretical justification.
  • ad hoc to paper GCoT captions provide useful intermediate supervision for teaching reasoning to T2I models.
    Core premise of the dataset (Section 2.1), but never tested by training a model on the dataset.
  • domain assumption FLUX.1-dev generated images are high-quality and suitable as training data.
    Reliance on FLUX.1-dev as the synthesis engine (Section 2.2) without comparison to other generators.

pith-pipeline@v1.3.0-alltime-deepseek · 22041 in / 9583 out tokens · 94405 ms · 2026-08-04T18:43:37.162542+00:00 · methodology

0 comments
read the original abstract

The advancement of open-source text-to-image (T2I) models has been hindered by the absence of large-scale, reasoning-focused datasets and comprehensive evaluation benchmarks, resulting in a performance gap compared to leading closed-source systems. To address this challenge, We introduce FLUX-Reason-6M and PRISM-Bench (Precise and Robust Image Synthesis Measurement Benchmark). FLUX-Reason-6M is a massive dataset consisting of 6 million high-quality FLUX-generated images and 20 million bilingual (English and Chinese) descriptions specifically designed to teach complex reasoning. The image are organized according to six key characteristics: Imagination, Entity, Text rendering, Style, Affection, and Composition, and design explicit Generation Chain-of-Thought (GCoT) to provide detailed breakdowns of image generation steps. The whole data curation takes 15,000 A100 GPU days, providing the community with a resource previously unattainable outside of large industrial labs. PRISM-Bench offers a novel evaluation standard with seven distinct tracks, including a formidable Long Text challenge using GCoT. Through carefully designed prompts, it utilizes advanced vision-language models for nuanced human-aligned assessment of prompt-image alignment and image aesthetics. Our extensive evaluation of 19 leading models on PRISM-Bench reveals critical performance gaps and highlights specific areas requiring improvement. Our dataset, benchmark, and evaluation code are released to catalyze the next wave of reasoning-oriented T2I generation. Project page: https://flux-reason-6m.github.io/ .

Figures

Figures reproduced from arXiv: 2509.09680 by Aldrich Yu, Chengqi Duan, Hongsheng Li, Kun Wang, LinJiang Huang, Rongyao Fang, Shuai Bai, Si Liu, Xihui Liu, Yuxuan Cai.

Figure 1
Figure 1. Figure 1: Evaluation of state-of-the-art text-to-image models with the proposed PRISM-Bench. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Showcase of FLUX-Reason-6M in six different characteristics and generation chain of thought. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An overview of FLUX-Reason-6M data curation pipeline. The entire process was completed [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Left: Three subsets of raw prompt sources. Middle: Image category ratio. Right: Prompt Suite Statistics. tional quality where the rendered text directly corresponds to the refined caption, forming a pristine training corpus for the Text rendering category. This comprehensive synthesis effort, combining a high-quality baseline with targeted augmentation strategies, results in a massive pool of 8 million ima… view at source ↗
Figure 5
Figure 5. Figure 5: An overview of the prompt design and evaluation protocol of PRISM-Bench. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Showcase of Long text track in the PRISM-Bench. GPT4.1 is not only required to score based on image-text alignment and image aesthetics, but also to provide a brief justification. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Showcase of Text rendering track in the PRISM-Bench-ZH. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric

    cs.CV 2026-07 conditional novelty 6.0

    A text-prompted perceptual metric (TPIPS) trained on a new human-judgment dataset matches human aspect-conditioned similarity choices better than existing VLMs and prior metrics.

  2. SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation

    cs.CV 2026-06 unverdicted novelty 6.0

    Introduces SciIR-82k dataset and SciIR-Bench for scientific image reasoning generation organized by Peirce's semiotic triad, with fine-tuning raising model score from 35% to 43%.

  3. Unlocking Complex Visual Generation via Closed-Loop Verified Reasoning

    cs.CV 2026-05 unverdicted novelty 6.0

    CLVR couples verified logical planning with pixel diffusion, uses proxy reinforcement learning on distilled histories, and merges weights to cut inference to 4 NFEs while outperforming open-source T2I models on comple...

  4. Unlocking Complex Visual Generation via Closed-Loop Verified Reasoning

    cs.CV 2026-05 unverdicted novelty 6.0

    CLVR framework adds closed-loop visual verification, proxy prompt reinforcement learning, and delta-space weight merge to improve complex text-to-image generation over single-step or unverified multi-step baselines.

  5. InsightTok: Improving Text and Face Fidelity in Discrete Tokenization for Autoregressive Image Generation

    cs.CV 2026-05 conditional novelty 6.0

    InsightTok improves text and face fidelity in discrete image tokenization via content-aware perceptual losses, with gains transferring to autoregressive generation.

  6. LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model

    cs.CV 2026-04 unverdicted novelty 6.0

    LLaDA2.0-Uni unifies multimodal understanding and generation inside one discrete diffusion large language model with a semantic tokenizer, MoE backbone, and diffusion decoder.

  7. Self-Adversarial One Step Generation via Condition Shifting

    cs.CV 2026-04 unverdicted novelty 6.0

    APEX derives self-adversarial gradients from condition-shifted velocity fields in flow models to achieve high-fidelity one-step generation, outperforming much larger models and multi-step teachers.

  8. Guiding Token-Sparse Diffusion Models

    cs.CV 2026-01 conditional novelty 6.0

    Token-sparsity gaps at inference can replace classifier-free guidance for sparsely trained diffusion models, yielding better FID and lower compute.

  9. Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer

    cs.CV 2025-11 conditional novelty 6.0

    A 6B single-stream diffusion transformer trained with heavily curated data reaches top open-source image-generation quality in 314K H800 GPU hours, releasing Turbo and Edit variants.

  10. CAPEval: A Decoupled Caption Evaluation across Understanding and Generation

    cs.CV 2026-08 conditional novelty 5.0

    Caption quality can be split into Coverage and Precision; in controlled training runs, Coverage predicts VLM understanding while Precision predicts T2I generation.

  11. Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer

    cs.CV 2025-11 unverdicted novelty 5.0

    Z-Image is an efficient 6B-parameter foundation model for image generation that rivals larger commercial systems in photorealism and bilingual text rendering through a new single-stream diffusion transformer and strea...

Reference graph

Works this paper leans on

72 extracted references · 35 linked inside Pith · cited by 9 Pith papers

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

  2. [2]

    Hrs-bench: Holistic, reliable and scalable benchmark for text-to-image models

    Eslam Mohamed Bakr, Pengzhan Sun, Xiaoqian Shen, Faizan Farooq Khan, Li Erran Li, and Mo- hamed Elhoseiny. Hrs-bench: Holistic, reliable and scalable benchmark for text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20041–20053, 2023

  3. [3]

    Flux, 2024

    BlackForest. Flux, 2024. URLhttps://github.com/black-forest-labs/flux

  4. [4]

    Flux.1 krea, 2025

    BlackForest. Flux.1 krea, 2025. URLhttps://www.krea.ai/apps/image/flux-krea

  5. [5]

    Hidream-i1: A high-efficient image generative foundation model with sparse diffusion transformer.arXiv preprint arXiv:2505.22705, 2025

    Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang, Fengbin Gao, Peihan Xu, et al. Hidream-i1: A high-efficient image generative foundation model with sparse diffusion transformer.arXiv preprint arXiv:2505.22705, 2025

  6. [6]

    Conceptual 12m: Pushing web- scale image-text pre-training to recognize long-tail visual concepts

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web- scale image-text pre-training to recognize long-tail visual concepts. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3558–3568, 2021

  7. [7]

    Attend-and-excite: Attention- based semantic guidance for text-to-image diffusion models.ACM transactions on Graphics (TOG), 42 (4):1–10, 2023

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention- based semantic guidance for text-to-image diffusion models.ACM transactions on Graphics (TOG), 42 (4):1–10, 2023

  8. [8]

    Textdiffuser: Diffusion models as text painters.Advances in Neural Information Processing Systems, 36:9353–9387, 2023

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters.Advances in Neural Information Processing Systems, 36:9353–9387, 2023

  9. [9]

    Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset.arXiv preprint arXiv:2505.09568, 2025

    Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset.arXiv preprint arXiv:2505.09568, 2025

  10. [10]

    Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

  11. [11]

    Postercraft: Rethinking high-quality aesthetic poster generation in a unified framework.arXiv preprint arXiv:2506.10741, 2025

    SiXiang Chen, Jianyu Lai, Jialin Gao, Tian Ye, Haoyu Chen, Hengyu Shi, Shitong Shao, Yunlong Lin, Song Fei, Zhaohu Xing, et al. Postercraft: Rethinking high-quality aesthetic poster generation in a unified framework.arXiv preprint arXiv:2506.10741, 2025

  12. [12]

    Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811, 2025

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811, 2025

  13. [13]

    Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation.arXiv preprint arXiv:2310.18235, 2023

    Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation.arXiv preprint arXiv:2310.18235, 2023

  14. [14]

    Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

  15. [15]

    Got-r1: Unleashing reasoning capability of mllm for visual generation with reinforcement learning.arXiv preprint arXiv:2505.17022, 2025

    Chengqi Duan, Rongyao Fang, Yuqing Wang, Kun Wang, Linjiang Huang, Xingyu Zeng, Hongsheng Li, and Xihui Liu. Got-r1: Unleashing reasoning capability of mllm for visual generation with reinforcement learning.arXiv preprint arXiv:2505.17022, 2025

  16. [16]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. InForty-first international conference on machine learning, 2024

  17. [17]

    Puma: Empowering unified mllm with multi-granular visual generation.arXiv preprint arXiv:2410.13861, 2024

    Rongyao Fang, Chengqi Duan, Kun Wang, Hao Li, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Hongsheng Li, and Xihui Liu. Puma: Empowering unified mllm with multi-granular visual generation.arXiv preprint arXiv:2410.13861, 2024

  18. [18]

    Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing.arXiv preprint arXiv:2503.10639, 2025

    Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, et al. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing.arXiv preprint arXiv:2503.10639, 2025. 16

  19. [19]

    Training-free structured diffusion guidance for compositional text-to-image synthesis.arXiv preprint arXiv:2212.05032, 2022

    Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis.arXiv preprint arXiv:2212.05032, 2022

  20. [20]

    Commonsense-t2i challenge: Can text-to-image generation models understand commonsense?arXiv preprint arXiv:2406.07546, 2024

    Xingyu Fu, Muyu He, Yujie Lu, William Yang Wang, and Dan Roth. Commonsense-t2i challenge: Can text-to-image generation models understand commonsense?arXiv preprint arXiv:2406.07546, 2024

  21. [21]

    Datacomp: In search of the next generation of multimodal datasets.Advances in Neural Information Processing Systems, 36:27092–27112, 2023

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Datacomp: In search of the next generation of multimodal datasets.Advances in Neural Information Processing Systems, 36:27092–27112, 2023

  22. [22]

    Seedream 3.0 technical report.arXiv preprint arXiv:2504.11346, 2025

    Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al. Seedream 3.0 technical report.arXiv preprint arXiv:2504.11346, 2025

  23. [23]

    Geneval: An object-focused framework for evaluating text-to-image alignment.Advances in Neural Information Processing Systems, 36:52132– 52152, 2023

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment.Advances in Neural Information Processing Systems, 36:52132– 52152, 2023

  24. [24]

    Seedream 2.0: A native chinese-english bilingual image generation foundation model.arXiv preprint arXiv:2503.07703, 2025

    Lixue Gong, Xiaoxia Hou, Fanshi Li, Liang Li, Xiaochen Lian, Fei Liu, Liyang Liu, Wei Liu, Wei Lu, Yichun Shi, et al. Seedream 2.0: A native chinese-english bilingual image generation foundation model.arXiv preprint arXiv:2503.07703, 2025

  25. [25]

    Gemini2.5-pro, 2025

    Google. Gemini2.5-pro, 2025. URLhttps://deepmind.google/models/gemini/pro/

  26. [26]

    Imagen4, 2025

    Google. Imagen4, 2025. URLhttps://deepmind.google/models/imagen/

  27. [27]

    Gemini2.5-flash-image, 2025

    Google. Gemini2.5-flash-image, 2025. URLhttps://deepmind.google/models/gemini/image/

  28. [28]

    Evalmuse-40k: A reliable and fine-grained benchmark with comprehensive human annotations for text-to-image generation model evaluation.arXiv preprint arXiv:2412.18150, 2024

    Shuhao Han, Haotian Fan, Jiachen Fu, Liang Li, Tao Li, Junhui Cui, Yunqiu Wang, Yang Tai, Jingwei Sun, Chunle Guo, et al. Evalmuse-40k: A reliable and fine-grained benchmark with comprehensive human annotations for text-to-image generation model evaluation.arXiv preprint arXiv:2412.18150, 2024

  29. [29]

    Clipscore: A reference- free evaluation metric for image captioning.arXiv preprint arXiv:2104.08718, 2021

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning.arXiv preprint arXiv:2104.08718, 2021

  30. [30]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

  31. [31]

    Scaling up vision-language pre-training for image captioning

    Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, and Lijuan Wang. Scaling up vision-language pre-training for image captioning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17980–17989, 2022

  32. [32]

    Ella: Equip diffusion models with llm for enhanced semantic alignment.arXiv preprint arXiv:2403.05135, 2024

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment.arXiv preprint arXiv:2403.05135, 2024

  33. [33]

    Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answer- ing

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answer- ing. InProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20406–20417, 2023

  34. [34]

    T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

  35. [35]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. InInternational conference on machine learning, pp. 4904–4916. PMLR, 2021

  36. [36]

    Genetic k-means algorithm.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 29(3):433–439, 1999

    KMMN Krishna and M Narasimha Murty. Genetic k-means algorithm.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 29(3):433–439, 1999

  37. [37]

    Genai-bench: Evaluating and improving compositional text-to-visual generation.arXiv preprint arXiv:2406.13743, 2024

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, et al. Genai-bench: Evaluating and improving compositional text-to-visual generation.arXiv preprint arXiv:2406.13743, 2024. 17

  38. [38]

    Playground v2

    Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation.arXiv preprint arXiv:2402.17245, 2024

  39. [39]

    Oscar: Object-semantics aligned pre-training for vision-language tasks

    Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. InEuropean conference on computer vision, pp. 121–137. Springer, 2020

  40. [40]

    Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding.arXiv preprint arXiv:2405.08748, 2024

    Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, et al. Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding.arXiv preprint arXiv:2405.08748, 2024

  41. [41]

    Nuwa-infinity: Autoregressive over autoregressive generation for infinite visual synthesis.Advances in Neural Information Processing Systems, 35:15420–15432, 2022

    Jian Liang, Chenfei Wu, Xiaowei Hu, Zhe Gan, Jianfeng Wang, Lijuan Wang, Zicheng Liu, Yuejian Fang, and Nan Duan. Nuwa-infinity: Autoregressive over autoregressive generation for infinite visual synthesis.Advances in Neural Information Processing Systems, 35:15420–15432, 2022

  42. [42]

    Evaluating text-to-visual generation with image-to-text generation

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. InEuropean Conference on Computer Vision, pp. 366–384. Springer, 2024

  43. [43]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

  44. [44]

    Summarizing emotions from text using plutchik’s wheel of emotions

    Manshad Abbasi Mohsin and Anatoly Beltiukov. Summarizing emotions from text using plutchik’s wheel of emotions. In7th scientific conference on information technologies for intelligent decision making support (ITIDS 2019), pp. 291–294. Atlantis Press, 2019

  45. [45]

    Gpt-4.1, 2025

    OpenAI. Gpt-4.1, 2025. URLhttps://openai.com/index/gpt-4-1/

  46. [46]

    Gpt-image-1, 2025

    OpenAI. Gpt-image-1, 2025. URL https://openai.com/index/ introducing-4o-image-generation/

  47. [47]

    Dall·e 3, September 2023

    OpenAI. Dall·e 3, September 2023. URLhttps://openai.com/zh-Hans-CN/index/dall-e-3/

  48. [48]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

  49. [49]

    Lumina-image 2.0: A unified and efficient image generative framework.arXiv preprint arXiv:2503.21758, 2025

    Qi Qin, Le Zhuo, Yi Xin, Ruoyi Du, Zhen Li, Bin Fu, Yiting Lu, Jiakang Yuan, Xinyue Li, Dongyang Liu, et al. Lumina-image 2.0: A unified and efficient image generative framework.arXiv preprint arXiv:2503.21758, 2025

  50. [50]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684–10695, 2022

  51. [51]

    Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35:25278–25294, 2022

  52. [52]

    Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2556–2565, 2018

  53. [53]

    From pixels to prose: A large dataset of dense image captions.arXiv preprint arXiv:2406.10328, 2024

    Vasu Singla, Kaiyu Yue, Sukriti Paul, Reza Shirkavand, Mayuka Jayawardhana, Alireza Ganjdanesh, Heng Huang, Abhinav Bhatele, Gowthami Somepalli, and Tom Goldstein. From pixels to prose: A large dataset of dense image captions.arXiv preprint arXiv:2406.10328, 2024

  54. [54]

    Stable diffusion 2.1, 2022

    Stability-AI. Stable diffusion 2.1, 2022. URL https://huggingface.co/stabilityai/ stable-diffusion-2-1

  55. [55]

    Stable diffusion 3, 2024

    Stability-AI. Stable diffusion 3, 2024. URL https://huggingface.co/stabilityai/ stable-diffusion-3-medium

  56. [56]

    Stable diffusion 3.5, 2024

    Stability-AI. Stable diffusion 3.5, 2024. URLhttps://github.com/Stability-AI/sd3.5

  57. [57]

    T2i-reasonbench: Benchmarking reasoning-informed text-to-image generation.arXiv preprint arXiv:2508.17472, 2025

    Kaiyue Sun, Rongyao Fang, Chengqi Duan, Xian Liu, and Xihui Liu. T2i-reasonbench: Benchmarking reasoning-informed text-to-image generation.arXiv preprint arXiv:2508.17472, 2025. 18

  58. [58]

    Journeydb: A benchmark for generative image understanding

    Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, et al. Journeydb: A benchmark for generative image understanding. Advances in neural information processing systems, 36:49659–49678, 2023

  59. [59]

    Anytext: Multilingual visual text generation and editing.arXiv preprint arXiv:2311.03054, 2023

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text generation and editing.arXiv preprint arXiv:2311.03054, 2023

  60. [60]

    Textatlas5m: A large-scale dataset for dense text image generation.arXiv preprint arXiv:2502.07870, 2025

    Alex Jinpeng Wang, Dongxing Mao, Jiawei Zhang, Weiming Han, Zhuobai Dong, Linjie Li, Yiqi Lin, Zhengyuan Yang, Libo Qin, Fuwei Zhang, et al. Textatlas5m: A large-scale dataset for dense text image generation.arXiv preprint arXiv:2502.07870, 2025

  61. [61]

    Nuwa: Visual synthesis pre-training for neural visual world creation

    Chenfei Wu, Jian Liang, Lei Ji, Fan Yang, Yuejian Fang, Daxin Jiang, and Nan Duan. Nuwa: Visual synthesis pre-training for neural visual world creation. InEuropean conference on computer vision, pp. 720–736. Springer, 2022

  62. [62]

    Qwen-image technical report.arXiv preprint arXiv:2508.02324, 2025

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report.arXiv preprint arXiv:2508.02324, 2025

  63. [63]

    Omnigen2: Exploration to advanced multimodal generation.arXiv preprint arXiv:2506.18871, 2025

    Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, et al. Omnigen2: Exploration to advanced multimodal generation.arXiv preprint arXiv:2506.18871, 2025

  64. [64]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv preprint arXiv:2306.09341, 2023

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv preprint arXiv:2306.09341, 2023

  65. [65]

    Conceptmix: A com- positional image generation benchmark with controllable difficulty.Advances in Neural Information Processing Systems, 37:86004–86047, 2024

    Xindi Wu, Dingli Yu, Yangsibo Huang, Olga Russakovsky, and Sanjeev Arora. Conceptmix: A com- positional image generation benchmark with controllable difficulty.Advances in Neural Information Processing Systems, 37:86004–86047, 2024

  66. [66]

    Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer.arXiv preprint arXiv:2501.18427, 2025

    Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, et al. Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer.arXiv preprint arXiv:2501.18427, 2025

  67. [67]

    Show-o2: Improved native unified multimodal models.arXiv preprint arXiv:2506.15564, 2025

    Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal models.arXiv preprint arXiv:2506.15564, 2025

  68. [68]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

  69. [69]

    Scaling autoregressive models for content-rich text-to-image generation.arXiv preprint arXiv:2206.10789, 2(3):5, 2022

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation.arXiv preprint arXiv:2206.10789, 2(3):5, 2022

  70. [70]

    Capsfusion: Rethinking image-text data at scale

    Qiying Yu, Quan Sun, Xiaosong Zhang, Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, and Jingjing Liu. Capsfusion: Rethinking image-text data at scale. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14022–14032, 2024

  71. [71]

    Vinvl: Making visual representations matter in vision-language models.arXiv preprint arXiv:2101.00529, 1(6):8, 2021

    Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. Vinvl: Making visual representations matter in vision-language models.arXiv preprint arXiv:2101.00529, 1(6):8, 2021

  72. [72]

    Lumina-next: Making lumina-t2x stronger and faster with next-dit.Advances in Neural Information Processing Systems, 37:131278–131315, 2024

    Le Zhuo, Ruoyi Du, Han Xiao, Yangguang Li, Dongyang Liu, Rongjie Huang, Wenze Liu, Xiangyang Zhu, Fu-Yun Wang, Zhanyu Ma, et al. Lumina-next: Making lumina-t2x stronger and faster with next-dit.Advances in Neural Information Processing Systems, 37:131278–131315, 2024. 19