Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models

Huixuan Zhang; Xiaojun Wan

REVIEW 3 major objections 4 minor 25 references

Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current image-text alignment metrics do not reliably rank text-to-image models, because their scores shift under random seeds and tiny pixel changes, and small metric gaps can be statistically significant without meaning the winner is…

desk verdict Useful empirical warning about seed and perturbation instability in T2I metrics, but the randomness argument overreaches by treating sampling variation as metric failure. read the letter →

arxiv 2506.08480 v1 pith:ZQOHKRIN submitted 2025-06-10 cs.CL cs.AIcs.CV

Huixuan Zhang , Xiaojun Wan This is my paper

classification cs.CLcs.AIcs.CV

keywords image-textalignmenttext-to-imagegenerationevaluationmetricsrobustnessstatisticalsignificancedominanceratioCLIPScoreVQAScore

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Text-to-image evaluation is usually judged only by how well it agrees with human ratings. This paper argues that a trustworthy evaluation must also be stable under the randomness of diffusion generation and under tiny image changes, and that its reported differences must reflect actual per-prompt superiority. Testing CLIPScore, VQAScore, and DSGScore on four diffusion models with 1,000 MSCOCO prompts, the paper finds that CLIPScore and DSGScore change model rankings across random seeds, and all three metrics show large worst-case score jumps when a single pixel value is increased by one. It also finds that a score gap can be statistically significant even when the 'better' model only wins about 60% of prompts, and sometimes less. The contribution is diagnostic: the paper sets out two properties that future evaluation frameworks should satisfy, rather than proposing a new metric.

What carries the argument

The carrying device is a two-property evaluation checklist applied to any metric $J$. Robustness is tested in two ways: running generation over three random seeds and checking whether model rankings stay fixed, and computing the absolute difference $|J(p_i, I_i) - J(p_i, I'_i)|$ for a perturbation that adds 1 to every non-255 pixel value. Significance is tested with a paired t-test over the per-prompt score sets and a dominance ratio $R = \frac{1}{N} \sum_{i=1}^N \mathbb{I}[s^i_{M_1} > s^i_{M_2}]$, the empirical probability that one model beats another prompt-by-prompt. These tools turn 'is this metric trustworthy' into measurable questions about ranking stability, worst-case score gaps, and per-prompt win rates.

What would settle it

Run the three metrics over the same 1,000 prompts and four models with 100 random seeds instead of 3; if the pairwise model rankings stay fixed in nearly all cases, the claimed seed-robustness failure disappears. Similarly, if perturbing each image by plus-one pixel using many different starting offsets produces negligible maximum gaps, the claimed worst-case sensitivity is an artifact of the specific perturbation.

Watch

Extended reading notes

Core claim

The paper's central claim is that current mainstream image-text alignment metrics are not fully trustworthy evaluators because they fail two basic properties. Robustness requires that evaluation results stay consistent when the input is reasonably perturbed: under different random seeds of the same diffusion model, and under a one-unit pixel-value change to the generated image. Significance requires that an observed score difference reflect a meaningful, per-prompt superiority rather than only a statistical artifact. The empirical demonstration is that CLIPScore and DSGScore produce different model rankings for different seeds, and that CLIPScore, VQAScore, and DSGScore all have maximum absolute score gaps of 7.30, 10.36, and 50.00 points respectively under a visually imperceptible perturbation. Paired t-tests show small score gaps can be statistically significant, but dominance ratios show the 'better' model wins only about 60% of prompts at best, and sometimes less.

Load-bearing premise

The central premise is that images which look the same to a human must receive close evaluation scores, so a one-unit pixel shift is treated as a 'reasonable perturbation' that should not change a metric's assessment.

Editorial extensions

If this is right

Model rankings from CLIPScore and DSGScore should be reported with variance across seeds; a single seed can be misleading.
Any metric used for text-to-image comparison should be stress-tested with the one-pixel perturbation before its rankings are trusted.
A statistically significant metric gap should not be read as a per-prompt win; dominance ratios should accompany p-values.
Evaluation conclusions drawn from one generated image per prompt are especially fragile because diffusion randomness alone can flip rankings.
Future evaluation frameworks should treat robustness and significance as design requirements, not optional diagnostics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper tests three metrics; the same failure modes likely extend to other model-based and component-based metrics, since they share similar embedding or VQA backbones.
A practical repair would be to report worst-case perturbation gaps as error bars, or to average scores over several perturbed copies of each image.
The dominance-ratio result implies that claims like 'model A beats model B by 5 points' should be rephrased as 'model A wins on about 60% of prompts,' which changes how progress in text-to-image generation is discussed.
The one-pixel perturbation is a lower bound on perceptual sensitivity; adversarial perturbations tuned per image would likely produce even larger score swings, so the reported gaps are conservative.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This paper gets at something real: standard text-to-image evaluation practice often reports a single ranking of models, and this paper shows that ranking can flip across three random seeds and that a one-unit pixel perturbation can produce large score swings. Those observations are worth having. The demonstrations in Tables 1 and 2—CLIPScore and DSGScore swapping model ranks across seeds, and all three metrics showing large worst-case gaps under a simple perturbation—are new, and the dominance-ratio analysis is a useful reminder that a statistically significant mean advantage does not imply that the better model wins most prompts. The paper also deserves credit for stating two properties (robustness and significance) clearly and for honestly noting that it does not propose a fix. The soft spots are real, though. The stress-test note is right: the rank flips in Table 1 may be sampling variability rather than metric unreliability. The metric is deterministic given the generated image, and the images differ across seeds. With only three seeds and near-tied means, a perfect metric could produce such flips. The paper's own paired t-tests could adjudicate, but they are never linked to the rank reversals. The perturbation assumption—that visually similar images should receive close scores—is stated but not defended beyond that, and the 'one-unit pixel shift' is a global +1 shift, not a localized or semantically neutral perturbation; the manual check is not a substitute for a controlled calibration. No code or data is provided, the perturbation analysis lacks error bars and uses one generator, and the significance findings are mathematically expected with N=1000 (small effects become significant). The paper would be stronger if it compared the observed flip rate to what a stable metric would do over more seeds and if it reported confidence intervals. The central direction is sound, though. Even if the randomness argument is overstated, the empirical observations suggest that practitioners should use multiple seeds and report variance or confidence intervals. That is a useful, actionable message. The paper deserves serious peer review, but the referee should push for a larger seed count, better linking of significance tests to the robustness results, and more careful language about metric versus protocol reliability.

Referee Report

3 major / 4 minor

Summary. The paper argues that trustworthiness of automatic image-text alignment metrics should be assessed not only by correlation with human judgments, but also by two additional properties: Robustness (stability under random seeds and under small image perturbations) and Significance (whether observed score differences reflect meaningful model superiority). The authors evaluate three metrics (CLIPScore, VQAScore, DSGScore) on four diffusion models (SD3, SD-XL, SD1.5, PixArt) using 1000 MSCOCO prompts and three random seeds. They report that CLIPScore and DSGScore change model rankings across seeds, that all three metrics exhibit large maximum score gaps under a one-pixel perturbation, and that paired t-tests can declare statistical significance even when the dominance ratio is near 50%. The paper concludes with recommendations for future evaluation practices.

Significance. If the empirical claims were fully established, this would be a useful contribution: the field mostly validates metrics against human ratings and largely ignores statistical stability and perturbation sensitivity. The paper also honestly states its main limitation (no improved evaluation framework is proposed). The strengths are the clean formulation of the two properties and the use of standard statistical tools (paired t-test, dominance ratio). However, the evidence base is thin: three seeds, one perturbation type, one generator for the perturbation analysis, and no error bars. The central claim about ranking instability currently conflates metric reliability with sampling variability, so the significance of the contribution is conditional on resolving that issue.

major comments (3)

[Section 2.2 and Table 1] The argument that rank flips demonstrate metric unreliability is not established because the paper does not test whether the observed mean differences are statistically distinguishable. For instance, CLIPScore with seed 3407 gives Pixart 25.71 vs SD1.5 25.58, and seed 5096 gives SD1.5 25.76 vs Pixart 25.75; these could easily be near-ties. The paired t-test machinery introduced in Section 4.2 is never applied to the rank reversals in Table 1, so the flips may reflect generation sampling variability rather than inconsistency of the metric. Takeaway 1 therefore overstates what the data show.
[Section 2.2 and Table 1] The robustness-to-randomness analysis uses only three seeds and reports no error bars or confidence intervals. The claim that 'significant robustness failures' occur even at 1000 prompts is not supported without a statistical model of ranking instability. With only three seeds, the probability of observing a rank flip under a true near-tie is high, so the observed flips do not imply a defect of CLIPScore or DSGScore. The paper should include more seeds (e.g., at least 10), bootstrap intervals on the mean scores, or a formal test of whether the ranking distribution is stable.
[Section 2.2, Eqs. (1)-(2), and Table 2] The perturbation-based robustness conclusion depends entirely on the assumption that visually similar images should receive close scores. The paper states this as a 'fundamental assumption' but does not validate it against other plausible notions of robustness, nor does it compare with other minimal perturbations (e.g., -1 pixel shift, small blur, or JPEG compression). The large maximum gaps in Table 2 (e.g., DSGScore 50.00, VQAScore 10.36) could be artifacts of categorical or thresholded outputs of the metrics rather than a general robustness failure. Without additional perturbation families and a justification for why the '+1' direction is representative, the claim that this reveals 'potential for metric exploitation' is not fully supported.

minor comments (4)

[Section 3] The selection of the 1000 MSCOCO validation prompts is not described; no random seed or sampling procedure is given, which impedes reproducibility.
[Section 4.1 and Table 2] The image perturbation experiment uses only SD-3 and a single fixed seed (42). A second generator or an additional seed would make the conclusion that 'all three metrics fail' more robust.
[Throughout] Naming is inconsistent: 'SD-3' and 'SD3' both appear, and Takeaway 2 says 'worse case' instead of 'worst case'.
[Figure 2] The heatmaps lack axis labels and a clear caption; it is difficult to map rows and columns to the four models without consulting Table 1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper empirically tests externally specified metrics against normative robustness/significance criteria rather than deriving conclusions from fitted inputs or self-citations.

full rationale

The paper's derivation chain is empirical rather than definitional: it defines the two properties Robustness and Significance as normative evaluation criteria, then measures existing metrics (CLIPScore, VQAScore, DSGScore) on standard models and MSCOCO prompts using standard statistical tools (paired t-tests and dominance ratios). No parameter is fitted to a subset of data and then renamed as a prediction, and no conclusion is obtained by substituting an equation into its own definition. The robustness-to-randomness result compares rankings across three fixed seeds, and the perturbation result compares each metric's score on an original image versus a one-unit pixel-shifted image; both are direct observations of metric behavior, not constructions that presuppose the conclusion. The paper explicitly disclaims that any particular ranking, such as VQAScore's, is 'correct' before demonstrating ranking instability, which avoids treating one metric's output as ground truth for judging another. The central assumption that visually similar images should receive close scores is a stated normative premise, and while one could disagree with it, disagreement with an assumption is not circular reasoning. The paper also contains no load-bearing self-citation: all cited metrics and models are external prior work, and the authors' own prior results are not invoked to justify the central claims. Therefore, despite legitimate methodological questions about whether three seeds are sufficient or whether pixel-level stability is the right notion of robustness, the paper's conclusions do not reduce by construction to its inputs, and no significant circularity is present.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities or fitted parameters. Its central claims rest on domain assumptions about what trustworthy evaluation requires and on standard statistical tools applied without full validation.

assumptions (5)

domain assumption Visually similar images should receive close evaluation scores.
Stated in Section 2.2 under Robustness to Image Perturbation as a fundamental assumption; the perturbation analysis and all conclusions about metric flaws depend on it.
domain assumption Consistent model rankings across random seeds are necessary for a trustworthy evaluation.
Used in Section 4.1 to declare CLIPScore and DSGScore less trustworthy when rankings swap across seeds 42, 3407, and 5096.
standard math Paired t-test on prompt-level metric scores is a valid significance test.
Used in Sections 2.2 and 4.2 without checking independence, normality, or multiple-comparison corrections.
domain assumption The +1 pixel perturbation produces visually indistinguishable images.
Section 2.2 states that visual imperceptibility is confirmed by manual inspection of multiple cases rather than by a quantitative perceptual metric.
domain assumption The selected 1000 prompts and 3 seeds are representative enough to draw general conclusions.
Section 3 says only that 1000 prompts were selected from the MSCOCO validation split; the selection procedure is not described.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models." pith.science (2026). https://pith.science/paper/ZQOHKRIN

@misc{pith2026250608480,
  author       = {Pith},
  title        = {Pith review of: Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZQOHKRIN}},
  note         = {Machine review of arXiv:2506.08480}
}

read the original abstract

Text-to-image models often struggle to generate images that precisely match textual prompts. Prior research has extensively studied the evaluation of image-text alignment in text-to-image generation. However, existing evaluations primarily focus on agreement with human assessments, neglecting other critical properties of a trustworthy evaluation framework. In this work, we first identify two key aspects that a reliable evaluation should address. We then empirically demonstrate that current mainstream evaluation frameworks fail to fully satisfy these properties across a diverse range of metrics and models. Finally, we propose recommendations for improving image-text alignment evaluation.

Figures

Figures reproduced from arXiv: 2506.08480 by the authors.

**Figure 2.** The T-test p-value and better ratio R between models. The value at i-th row and j-th column in a matrix represents the result of model i compared against model j. The random seed used is shown in the title of the corresponding heatmap. dards, we consider results with p-value < 0.05 as statistically significant. Our analysis using VQAScore reveals several notable findings: SD-3 demonstrates statistically significan… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 9 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, and 1 others. 2023. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8

work page 2023
[4]

Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. 2024 a . https://arxiv.org/abs/2403.04692 Pixart- : Weak-to-strong training of diffusion transformer for 4k text-to-image generation . arXiv

arXiv 2024
[5]

Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. 2024 b . Unified hallucination detection for multimodal large language models. arXiv preprint arXiv:2402.03190

arXiv 2024
[6]

Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. 2023. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-image generation. arXiv preprint arXiv:2310.18235

arXiv 2023
[7]

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, and 1 others. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning

work page 2024
[8]

Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/a3bf71c7c63f0c3bcb7ff67c67b1e7b1-Paper-Datasets_and_Benchmarks.pdf Geneval: An object-focused framework for evaluating text-to-image alignment . In Advances in Neural Information Processing Systems, volume 36, pages 52132--52152. Curran ...

work page 2023

Show all 25 references

[9]

Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, and Yezhou Yang. 2023. https://arxiv.org/abs/2212.10015 Benchmarking spatial relationships in text-to-image generation . Preprint, arXiv:2212.10015

2023 arXiv
[10]

Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514--7528

2021
[11]

Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. 2023. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision ...

2023
[12]

Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2023. https://arxiv.org/abs/2307.06350 T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation . Preprint, arXiv:2307.06350

2023 arXiv
[13]

Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Xide Xia, Pengchuan Zhang, Graham Neubig, and Deva Ramanan. 2024. Evaluating and improving compositional text-to-visual generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

2024
[14]

Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, pages 12888--12900. PMLR

2022
[15]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, ...

2014
[16]

Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. 2024. Evaluating text-to-visual generation with image-to-text generation. arXiv preprint arXiv:2404.01291

2024 arXiv
[17]

Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkil \"a , and Shin’ichi Satoh. 2023. Toward verifiable and reproducible human evaluation for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pa...

2023
[18]

Maitreya Patel, Tejas Gokhale, Chitta Baral, and Yezhou Yang. 2024. https://arxiv.org/abs/2306.04695 Conceptbed: Evaluating concept learning abilities of text-to-image diffusion models . Preprint, arXiv:2306.04695

2024 arXiv
[19]

Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M \"u ller, Joe Penna, and Robin Rombach. 2023. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952

2023 arXiv
[20]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine...

2021
[21]

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj\"orn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684--10695

2022
[22]

Olivia Wiles, Chuhan Zhang, Isabela Albuquerque, Ivana Kajić, Su Wang, Emanuele Bugliarello, Yasumasa Onoe, Pinelopi Papalampidi, Ira Ktena, Chris Knutsen, Cyrus Rashtchian, Anant Nawalgaria, Jordi Pont-Tuset, and Aida Nematzadeh. 2025. https://arxiv.org/abs/2404.16820 Revisit...

2025 arXiv
[23]

Xindi Wu, Dingli Yu, Yangsibo Huang, Olga Russakovsky, and Sanjeev Arora. 2024. https://arxiv.org/abs/2408.14339 Conceptmix: A compositional image generation benchmark with controllable difficulty . Preprint, arXiv:2408.14339

2024 arXiv
[24]

Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. 2024. What you see is what you read? improving text-image alignment evaluation. Advances in Neural Information Processing Systems, 36

2024
[25]

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, and 1 others. 2023. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8

work page 2023

[4] [4]

Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. 2024 a . https://arxiv.org/abs/2403.04692 Pixart- : Weak-to-strong training of diffusion transformer for 4k text-to-image generation . arXiv

arXiv 2024

[5] [5]

Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. 2024 b . Unified hallucination detection for multimodal large language models. arXiv preprint arXiv:2402.03190

arXiv 2024

[6] [6]

Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. 2023. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-image generation. arXiv preprint arXiv:2310.18235

arXiv 2023

[7] [7]

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, and 1 others. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning

work page 2024

[8] [8]

Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/a3bf71c7c63f0c3bcb7ff67c67b1e7b1-Paper-Datasets_and_Benchmarks.pdf Geneval: An object-focused framework for evaluating text-to-image alignment . In Advances in Neural Information Processing Systems, volume 36, pages 52132--52152. Curran ...

work page 2023