Pith. sign in

REVIEW 3 major objections 9 minor 1 cited by

T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation

T0 review · 3 major / 9 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training an open-source vision-language model with group-relative policy optimization and a continuous reward, using only coarse quality scores, produces a text-to-image evaluator whose scores and written rationales align with human…

desk verdict Plausible RL-for-T2I-evaluation paper whose pairwise ImageReward result may be an in-sample fit; the single-wise continuous-reward contribution is real and worth refereeing. read the letter →

arxiv 2505.17897 v1 pith:2XFUXDPO submitted 2025-05-23 cs.AI cs.CL

Zi-Ao Ma , Tian Lan , Rong-Cheng Tu , Shu-Hang Liu , Heyan Huang , Zhijing Wu , Chen Xu , Xian-Ling Mao This is my paper · ORCID
classification cs.AIcs.CL
keywords text-to-imageevaluationinterpretablereinforcementlearningGRPOmultimodallargelanguagemodelchain-of-thoughtrewardmodelinghumanalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that an open-source multimodal language model can be trained to grade text-to-image outputs as reliably as proprietary models such as GPT-4o, while explaining each grade in chain-of-thought reasoning, using only coarse human quality scores as supervision instead of expensive detailed critique annotations. It does this by applying Group Relative Policy Optimization (GRPO) with a continuous reward that smoothly penalizes distance from the reference score, rather than a binary correct/incorrect reward. The resulting system, T2I-Eval-R1, reports higher Spearman and Kendall correlations with human judgments than strong baselines on the T2I-Eval benchmark, generalizes to the previously unseen faithfulness dimension of TIFA v1.0, and reaches 66.07% pairwise preference accuracy on ImageReward. The paper also finds that its written rationales are preferred by human raters over those from supervised fine-tuning baselines. If these results hold, interpretable automatic evaluation of generated images becomes much cheaper and more scalable.

What carries the argument

The central mechanism is the T2I-Eval-R1 objective: Group Relative Policy Optimization (GRPO) combined with a continuous, distance-based reward. GRPO samples a group of candidate evaluations per input, normalizes the group's rewards into advantages, and updates the policy without a separate critic network. The continuous reward functions, Rsingle and Rpair, replace the usual binary 0/1 correctness reward with a smooth function of the distance between the predicted and reference score, so a prediction of 8 for a reference of 7 is rewarded more than a prediction of 3. All evaluation modes—single-wise scoring and pairwise preference confidence—are expressed through one template-driven evaluator function E that takes a prompt, one or two images, an evaluation dimension set D, and guidelines G, and returns a rationale r and a judgment q.

What would settle it

Check whether the ImageReward test pairs used for Table 3 are disjoint from the roughly 35,000 training pairs resampled from ImageRewardDB; if they overlap, the reported 66.07% preference accuracy is an in-sample measurement rather than a prediction. A cleaner falsification would be to train the same T2I-Eval-R1Pair pipeline on one portion of ImageRewardDB and evaluate on a held-out, strictly disjoint portion (or on a new pairwise preference set), and see whether the preference accuracy remains above the 64-65% range of prior baselines.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that coarse-grained judgment scores are sufficient supervision to train a multimodal LLM to act as an interpretable text-to-image evaluator, provided the training objective is outcome-based reinforcement learning with a continuous reward rather than imitation of reference rationales. Concretely, the T2I-Eval-R1 objective replaces the binary rewards used in prior GRPO reasoning work with distance-based rewards (Rsingle and Rpair) that map prediction errors to [-1,1], giving the policy useful gradients for near-correct scores. The authors train Qwen2.5-VL-7B-Instruct in four variants (Base, Enhance, Pair, General) on resampled T2I-Eval and ImageRewardDB corpora, and report that the best variants surpass GPT-4o-based and open-source baselines on all three benchmarks: overall Spearman ρ of 58.74 on T2I-Eval, 70.43 on TIFA v1.0 faithfulness, and 66.07% pairwise preference accuracy on ImageReward. In addition, the model's chain-of-thought rationales receive higher scores from a GPT-4o judge and are chosen by human annotators more often than rationales from LLMScore, VIEScore, and T2I-Eval baselines.

Load-bearing premise

The claim rests on the assumption that the coarse quality scores used as training rewards faithfully represent human judgment, and that the pairwise evaluation set is disjoint from the training pairs drawn from the same pool.

Editorial extensions

If this is right

  • Open-source MLLMs can reach state-of-the-art T2I evaluation performance without fine-grained critique annotations, needing only coarse scores or pairwise preferences.
  • Because the training signal is just a numeric reference, the same pipeline can be retargeted to new evaluation dimensions by swapping the prompt blocks and collecting coarse labels, without retraining from scratch.
  • The mixed General variant shows that one model can serve both single-wise scoring and pairwise preference evaluation simultaneously, with no loss on the faithfulness benchmark.
  • The ablation shows that the continuous reward is the key ingredient: replacing it with a binary reward drops T2I-Eval overall Spearman ρ from 58.74 to 47.25 and TIFA ρ from 69.03 to 61.03.
  • The model's rationales are not merely plausible text: in the human preference study, annotators preferred T2I-Eval-R1's explanations over VIEScore's in 94% of comparisons when ties are excluded.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If coarse-score supervision suffices, the same GRPO-with-continuous-reward recipe could be applied to other subjective evaluation domains—video quality, document quality, dialogue helpfulness—where fine-grained rationales are expensive but coarse human ratings are abundant.
  • A decisive next test is whether the pairwise accuracy holds on a strictly disjoint split of ImageRewardDB, since the paper does not state that the evaluation pairs are separate from training pairs.
  • One natural extension the paper leaves implicit is to reward rationale quality directly (e.g., by penalizing rationales that contradict the predicted score), which could further improve the interpretability that is currently only indirectly shaped.
  • Since the reward only needs a reference value, it could also be applied to evaluate generated images in a loop during diffusion-model training, using the same open-source evaluator as a cheap reward model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 9 minor

Summary. The paper proposes T2I-Eval-R1, a reinforcement-learning framework that trains open-source multimodal LLMs (Qwen2.5-VL-7B) as interpretable text-to-image evaluators using only coarse-grained quality scores or pairwise preferences. The method uses GRPO with a newly proposed continuous reward that penalizes score deviation smoothly, unlike binary rewards, and produces both scalar/confidence scores and chain-of-thought rationales. The authors train several variants on the T2I-Eval training set and on ImageRewardDB, then evaluate on T2I-Eval, TIFA v1.0, and ImageReward benchmarks, reporting Spearman/Kendall correlations for single-wise settings and pairwise preference accuracy for the pairwise setting. They also report GPT-4o-based and human studies of rationale quality. The main claim is that T2I-Eval-R1 achieves significantly higher alignment with human assessments and more accurate interpretable rationales than strong baselines.

Significance. If the results hold, the work makes a useful contribution by showing that interpretable T2I evaluators can be trained with coarse-grained supervision, avoiding the need for expensive fine-grained rationale annotations. The continuous-reward GRPO formulation is a sensible and clearly presented idea, and the ablation comparing continuous versus binary rewards provides evidence for its effectiveness. The paper also includes human evaluation of rationale quality, which strengthens the interpretability claims. However, the pairwise evaluation result is potentially compromised by training/evaluation overlap, and the headline claim of 'significantly higher' performance is not supported by any statistical testing. These issues are load-bearing for the central claims, so the paper requires substantial revision before the empirical claims can be accepted.

major comments (3)
  1. [§3.3, §4.1, Table 3, Appendix C] The pairwise training corpus is constructed by resampling ImageRewardDB into ~35,000 pairs (§3.3), and the pairwise evaluation is reported on the ImageReward benchmark (Table 3, Appendix C) without stating that the test pairs are disjoint from the training pairs. Since ImageRewardDB is the same human-preference pool from which the ImageReward benchmark is drawn, any overlap makes the reported 66.07% preference accuracy an in-sample fit rather than a held-out prediction. The paper does not report the number of evaluation pairs, the overlap count, or any cross-validation protocol. This is a load-bearing issue: if the pairwise result is invalidated, the claimed support from one of the three benchmarks is removed, and the pairwise aspect of the framework (continuous confidence) loses its empirical grounding. Please specify the exact construction of the evaluation set, verify and state disjointness from the training pairs, and if overlap exists, recompute or re-collect the evaluation.
  2. [§4.2, Tables 1–3, Abstract] The abstract claims 'significantly higher alignment with human assessments', but no confidence intervals, standard errors, or significance tests are reported anywhere in the experimental section. The margins over strong baselines are small in several cases (e.g., TIFA ρ = 70.43 vs 69.50 for VQAScore; ImageReward preference accuracy 66.07 vs 65.14 for ImageReward and 64.24 for UnifiedReward-Think). Without paired statistical tests or bootstrap confidence intervals, the term 'significantly' is not supported. Please add appropriate uncertainty quantification and hypothesis tests, or temper the claim accordingly.
  3. [Table 2, §4.2] The row labeled 'T2I-Eval-R1Hybrid' in Table 2 is not defined in §4.1, where the variants introduced are Base, Enhance, Pair, and General. The text in §4.2 states that the 'General' variant achieves the best TIFA result, so the table appears to contain a naming error. This must be corrected for reproducibility.
minor comments (9)
  1. [Abstract] Typo: 'easy acessible' should be 'easily accessible'.
  2. [Appendix C] The text refers to 'Eval-GRPO' when describing the training method; this appears to be an inconsistent name for 'T2I-Eval-R1'.
  3. [§4.2] Typo: 'sinificantly' should be 'significantly'.
  4. [§4.2] Typo: 'abiltiy' should be 'ability'.
  5. [§4.5.1] Typo: 'out method' should be 'our method'.
  6. [Table 1] The 'Inter-Annotator' row is not explained in the text or appendix; please clarify how this baseline was computed (e.g., split-half agreement between annotators) and why it serves as an upper bound.
  7. [Table 7] The column labels '-T' and '+T' are not defined in the table or text. Please clarify whether 'T' refers to including/excluding ties, and add a caption describing the metric.
  8. [Appendix F] The prompt template text contains formatting issues such as missing spaces (e.g., 'asyourannotationscore') and the two-column layout in Figure 2 is difficult to read; please reformat for clarity.
  9. [§4.5] The human preference study reports win rates but does not state the number of annotators or the inter-annotator agreement; please provide this information to support the reliability of the subjective evaluation.

Circularity Check

0 steps flagged · score 2.0 of 10

No demonstrated circularity: held-out single-wise benchmarks and external TIFA/ImageReward support the central claims; only minor self-citation from the authors' own T2I-Eval benchmark.

full rationale

The paper's claimed derivation chain is: (i) define evaluator E in Eq. (1); (ii) define the GRPO objective in Eq. (7) with continuous rewards in Eqs. (8) and (9); (iii) train Qwen2.5-VL-7B on coarse T2I-Eval scores and ImageRewardDB-derived preferences; (iv) measure Spearman/Kendall correlations on the T2I-Eval test split, TIFA v1.0, and preference accuracy on the ImageReward test set. None of these steps equates a prediction to a training input by construction: the reward functions are optimization objectives, not evaluation metrics, and the evaluation labels come from held-out or external benchmarks. The T2I-Eval benchmark was created by overlapping authors (reference [31] includes four authors of this paper), so there is a minor self-citation; however, the paper trains only on the training split and reports against the held-out test split, and it additionally validates on TIFA v1.0 and ImageReward, so the self-citation is not load-bearing. The pairwise training corpus is resampled from ImageRewardDB, and Table 6 explicitly refers to an 'ImageReward test set'; while the paper does not state the disjointness of pairwise training and test pairs, this is a reporting gap rather than a demonstrated reduction. Absent evidence of overlap or of a fitted parameter renamed as a prediction, I find no significant circularity.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The method's success rests on treating existing human preference scores as gold, on the choice of Qwen2.5-VL as policy, and on the assumption that the ImageReward test set is independent of the ImageRewardDB training pairs. The unstated split assumption is the most fragile: it is not an axiom of the math but a domain assumption the paper never verifies.

free parameters (8)
  • GRPO group size G = 8
    Set to 8 for all runs; no sensitivity analysis reported.
  • KL penalty coefficient beta = not reported
    Controls deviation from the reference policy; value omitted from training settings.
  • GRPO clip coefficient epsilon = not reported
    Standard clipping threshold; value omitted from training settings.
  • LoRA rank and alpha = rank 256, alpha 512
    Chosen by hand for computational efficiency; no tuning study reported.
  • Pairwise rating-gap sampling weights = 1:2:2:1 for Delta-r = 1,2,3,4
    Chosen to balance easy and hard pairs; affects the training distribution.
  • Single-wise per-dimension sample count = 9,000 per dimension, 36,000 total
    Chosen for balanced coverage across the three T2I-Eval dimensions and overall quality.
  • Rejection-sampling enhanced set size = 4,000
    Used for the Enhance variant; size chosen without reported sensitivity analysis.
  • Continuous reward normalization = 1 - 2*|clip(spred)-sref|/(smax-smin)
    Hand-designed mapping from score deviation to [-1,1]; alternative scalings not explored.
assumptions (6)
  • domain assumption Human quality labels in T2I-Eval and ImageRewardDB are valid gold standards for evaluation quality.
    The entire reward signal is derived from these coarse labels; inter-annotator correlations are high but not perfect.
  • domain assumption Spearman and Kendall correlations with human labels measure evaluator quality.
    All headline results are reported as rank correlations against human judgments; this is the standard but unstated evaluation premise.
  • domain assumption The TIFA faithfulness score is a valid proxy for human judgment on an unseen dimension.
    Used as the out-of-domain benchmark; the paper does not independently validate TIFA labels against fresh human ratings.
  • domain assumption ImageReward test pairs are disjoint from ImageRewardDB training pairs.
    Never stated in the paper; if false, the pairwise accuracy result is inflated by training/evaluation overlap.
  • domain assumption GPT-4o can serve as a reliable proxy judge for rationale quality.
    Used as the automatic rationale judge in Section 4.5.1; human validation is limited to small samples of 200 and 50 items.
  • domain assumption Qwen2.5-VL-7B-Instruct can express useful evaluative reasoning through chain-of-thought outputs.
    The method relies on the backbone MLLM generating coherent rationales alongside scores; this is not proven for other backbones.

how reviews work

0 comments
Cite this review

Pith. "Pith review of T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation." pith.science (2026). https://pith.science/paper/2XFUXDPO

@misc{pith2026250517897,
  author       = {Pith},
  title        = {Pith review of: T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2XFUXDPO}},
  note         = {Machine review of arXiv:2505.17897}
}
read the original abstract

The rapid progress in diffusion-based text-to-image (T2I) generation has created an urgent need for interpretable automatic evaluation methods that can assess the quality of generated images, therefore reducing the human annotation burden. To reduce the prohibitive cost of relying on commercial models for large-scale evaluation, and to improve the reasoning capabilities of open-source models, recent research has explored supervised fine-tuning (SFT) of multimodal large language models (MLLMs) as dedicated T2I evaluators. However, SFT approaches typically rely on high-quality critique datasets, which are either generated by proprietary LLMs-with potential issues of bias and inconsistency-or annotated by humans at high cost, limiting their scalability and generalization. To address these limitations, we propose T2I-Eval-R1, a novel reinforcement learning framework that trains open-source MLLMs using only coarse-grained quality scores, thereby avoiding the need for annotating high-quality interpretable evaluation rationale. Our approach integrates Group Relative Policy Optimization (GRPO) into the instruction-tuning process, enabling models to generate both scalar scores and interpretable reasoning chains with only easy accessible annotated judgment scores or preferences. Furthermore, we introduce a continuous reward formulation that encourages score diversity and provides stable optimization signals, leading to more robust and discriminative evaluation behavior. Experimental results on three established T2I meta-evaluation benchmarks demonstrate that T2I-Eval-R1 achieves significantly higher alignment with human assessments and offers more accurate interpretable score rationales compared to strong baseline methods.

Figures

Figures reproduced from arXiv: 2505.17897 by the authors.

Figure 1
Figure 1. Comparison of our method with repre￾sentative baselines in the results of GPT-4o-based meta-evaluation for interpretable evaluation. Quality stability : Among all the compar￾isons in the interpretable evaluation, our method consistently has the lowest standard deviation of the scores, revealing that the quality of inter￾pretable rationales in our method is stabler than previous methods. While GPT-4o provides a scala… view at source ↗
Figure 2
Figure 2. Prompt template for T2I-Eval-R1 evaluators. The evaluation prompt is dynamically [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Prompt template for Appearance Quality evaluation from T2I-Eval dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Prompt template for Intrnsic Attribute Consistency evaluation from T2I-Eval dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Prompt template for Relationship Attribute Consistency evaluation from T2I-Eval dataset. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Prompt template for Overall Quality evaluation from T2I-Eval dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Prompt template for Text-to-Image Faithfulness evaluation from TIFA-v1.0 dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Prompt template for evaluation from ImageReward dataset. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Two evaluation cases for appearance quality rationales from [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Two evaluation cases for intrinsic attribute consistency rationales from [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Two evaluation cases for relationship attribute consistency rationales from [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Two evaluation cases for overall rationales from [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: Two evaluation cases for overall rationales from [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: Two evaluation cases for overall rationales from [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A unified taxonomy and comparative meta-evaluation of automatic evaluation methods across text, vision, and speech generation, concluding that LLM-based evaluators dominate current practice.

Reference graph

Works this paper leans on

39 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    L., Almeida , D., Altenschmidt , J., Altman , S., Anadkat , S., & others (2023) Gpt-4 technical report

    Achiam , J., Adler , S., Agarwal , S., Ahmad , L., Akkaya , I., Aleman , F. L., Almeida , D., Altenschmidt , J., Altman , S., Anadkat , S., & others (2023) Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    M., Hauth , A., Millican , K., Silver , D., Petrov , S., Johnson , M., Antonoglou , I., Schrittwieser , J., Glaese , A., Chen , J., Pitler , E., Lillicrap , T

    Anil , R., Borgeaud , S., Wu , Y ., Alayrac , J., Yu , J., Soricut , R., Schalkwyk , J., Dai , A. M., Hauth , A., Millican , K., Silver , D., Petrov , S., Johnson , M., Antonoglou , I., Schrittwieser , J., Glaese , A., Chen , J., Pitler , E., Lillicrap , T. P., Lazaridou , A., Firat , O., Molloy , J., Isard , M., Barham , P. R., Hennigan , T., Lee , B., V...

    arXiv 2023

  3. [3]

    5-vl technical report

    Bai , S., Chen , K., Liu , X., Wang , J., Ge , W., Song , S., Dang , K., Wang , P., Wang , S., Tang , J., & others (2025) Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

    arXiv 2025

  4. [4]

    Bradley , R. A. & Terry , M. E. (1952) Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika 39(3/4):324–345. 10

    work page 1952

  5. [5]

    (2024) Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation

    Cho , J., Hu , Y ., Baldridge , J., Garg , R., Anderson , P., Krishna , R., Bansal , M., Pont-Tuset , J., & Wang , S. (2024) Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation. In ICLR

    work page 2024

  6. [6]

    (2023) Flashattention-2: Faster attention with better parallelism and work partitioning

    Dao , T. (2023) Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691

    arXiv 2023

  7. [7]

    arXiv preprint arXiv:2312.09244

    Eisenstein , J., Nagpal , C., Agarwal , A., Beirami , A., D’Amour , A., Dvijotham , D., Fisch , A., Heller , K., Pfohl , S., Ramachandran , D., & others (2023) Helping or herding? reward model ensembles mitigate but do not eliminate reward hacking. arXiv preprint arXiv:2312.09244

    arXiv 2023

  8. [8]

    In Forty-first international conference on machine learning

    Esser , P., Kulal , S., Blattmann , A., Entezari , R., Müller , J., Saini , H., Levi , Y ., Lorenz , D., Sauer , A., Boesel , F., & others (2024) Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning

    work page 2024

Show all 39 references
  1. [9]

    (2023) Dreamsim: Learning new dimensions of human visual similarity using synthetic data

    Fu , S., Tamir , N., Sundaram , S., Chai , L., Zhang , R., Dekel , T., & Isola , P. (2023) Dreamsim: Learning new dimensions of human visual similarity using synthetic data. Advances in Neural Information Processing Systems 36:50742–50768

    2023

  2. [10]

    arXiv preprint arXiv:2501.12948

    Guo , D., Yang , D., Zhang , H., Song , J., Zhang , R., Xu , R., Zhu , Q., Ma , S., Wang , P., Bi , X., & others (2025) Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  3. [11]

    D., Oosterhuis , H., Rijke , M., & Shukla , S

    Gupta , S., Ahuja , C., Lin , T.-Y ., Roy , S. D., Oosterhuis , H., Rijke , M., & Shukla , S. N. (2025) A simple and effective reinforcement learning method for text-to-image diffusion fine-tuning. arXiv preprint arXiv:2503.00897

    2025

  4. [12]

    Hara , K., Adams , A., Milland , K., Savage , S., Callison-Burch , C., & Bigham , J. P. (2018) A data-driven analysis of workers’ earnings on amazon mechanical turk. In Proceedings of the 2018 CHI conference on human factors in computing systems pages 1–14

    2018

  5. [13]

    L., & Choi , Y

    Hessel , J., Holtzman , A., Forbes , M., Bras , R. L., & Choi , Y . (2021) Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718

    2021 arXiv

  6. [14]

    (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Heusel , M., Ramsauer , H., Unterthiner , T., Nessler , B., & Hochreiter , S. (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30

    2017

  7. [15]

    J., Shen , Y ., Wallis , P., Allen-Zhu , Z., Li , Y ., Wang , S., Wang , L., Chen , W., & others (2022) Lora: Low-rank adaptation of large language models

    Hu , E. J., Shen , Y ., Wallis , P., Allen-Zhu , Z., Li , Y ., Wang , S., Wang , L., Chen , W., & others (2022) Lora: Low-rank adaptation of large language models. ICLR 1(2):3

    2022

  8. [16]

    Hu , Y ., Liu , B., Kasai , J., Wang , Y ., Ostendorf , M., Krishna , R., & Smith , N. A. (2023) Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. InProceedings of the IEEE/CVF International Conference on Computer Vision pages 20406–20417

    2023

  9. [17]

    (2025) Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Huang , W., Jia , B., Zhai , Z., Cao , S., Ye , Z., Zhao , F., Xu , Z., Hu , Y ., & Lin , S. (2025) Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749

    2025 arXiv

  10. [18]

    (2025) Search-r1: Train- ing llms to reason and leverage search engines with reinforcement learning

    Jin , B., Zeng , H., Yue , Z., Yoon , J., Arik , S., Wang , D., Zamani , H., & Han , J. (2025) Search-r1: Train- ing llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516

    2025 arXiv

  11. [19]

    (2023) Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Kirstain , Y ., Polyak , A., Singer , U., Matiana , S., Penna , J., & Levy , O. (2023) Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems 36: 36652–36663

    2023

  12. [20]

    (2024) Viescore: Towards explainable metrics for conditional image synthesis evaluation

    Ku , M., Jiang , D., Wei , C., Yue , X., & Chen , W. (2024) Viescore: Towards explainable metrics for conditional image synthesis evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) pages 12268–12290

    2024

  13. [21]

    (2023) Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Li , J., Li , D., Savarese , S., & Hoi , S. (2023) Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning pages 19730–19742. PMLR

    2023

  14. [22]

    (2024) Evaluating text-to-visual generation with image-to-text generation

    Lin , Z., Pathak , D., Li , B., Li , J., Xia , X., Neubig , G., Zhang , P., & Ramanan , D. (2024) Evaluating text-to-visual generation with image-to-text generation. In A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, and G. Varol, (eds.), Computer Vision - ECCV 20...

    2024

  15. [23]

    E., & Wang , W

    Lu , Y ., Yang , X., Li , X., Wang , X. E., & Wang , W. Y . (2023) Llmscore: Unveiling the power of large language models in text-to-image synthesis evaluation. Advances in Neural Information Processing Systems 36: 23075–23093

    2023

  16. [24]

    & Xie , S

    Peebles , W. & Xie , S. (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision pages 4195–4205

    2023

  17. [25]

    (2023) Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Podell , D., English , Z., Lacey , K., Blattmann , A., Dockhorn , T., Müller , J., Penna , J., & Rombach , R. (2023) Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations

    2023

  18. [26]

    D., Ermon , S., & Finn , C

    Rafailov , R., Sharma , A., Mitchell , E., Manning , C. D., Ermon , S., & Finn , C. (2023) Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems 36:53728–53741

    2023

  19. [27]

    (2020) Zero: Memory optimizations toward training trillion parameter models

    Rajbhandari , S., Rasley , J., Ruwase , O., & He , Y . (2020) Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis pages 1–16. IEEE

    2020

  20. [28]

    (2022) High-resolution image synthesis with latent diffusion models

    Rombach , R., Blattmann , A., Lorenz , D., Esser , P., & Ommer , B. (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition pages 10684–10695

    2022

  21. [29]

    (2016) Improved techniques for training gans

    Salimans , T., Goodfellow , I., Zaremba , W., Cheung , V ., Radford , A., & Chen , X. (2016) Improved techniques for training gans. Advances in neural information processing systems 29

    2016

  22. [30]

    arXiv preprint arXiv:2307.09288

    Touvron , H., Martin , L., Stone , K., Albert , P., Almahairi , A., Babaei , Y ., Bashlykov , N., Batra , S., Bhargava , P., Bhosale , S., & others (2023) Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  23. [31]

    (2024) Automatic evaluation for text-to-image generation: Task-decomposed framework, distilled training, and meta-evaluation benchmark

    Tu , R.-C., Ma , Z.-A., Lan , T., Zhao , Y ., Huang , H., & Mao , X.-L. (2024) Automatic evaluation for text-to-image generation: Task-decomposed framework, distilled training, and meta-evaluation benchmark. arXiv preprint arXiv:2411.15488

    2024 arXiv

  24. [32]

    Wang , Y ., Li , Z., Zang , Y ., Wang , C., Lu , Q., Jin , C., & Wang , J. (2025. ) Unified multimodal chain-of-thought reward model through reinforcement fine-tuning. arXiv preprint arXiv:2505.03318

    2025

  25. [33]

    Wang , Y ., Zang , Y ., Li , H., Jin , C., & Wang , J. (2025. ) Unified reward model for multimodal understanding and generation. arXiv preprint arXiv:2503.05236

    2025 arXiv

  26. [34]

    (2020) Neural text degeneration with unlikelihood training

    Welleck , S., Kulikov , I., Roller , S., Dinan , E., Cho , K., & Weston , J. (2020) Neural text degeneration with unlikelihood training. In 8th International Conference on Learning Representations, ICLR 2020

    2020

  27. [35]

    IEEE Transactions on Knowledge and Data Engineering

    Xu , C., Lan , T., Ji , Y ., Yu , C., Wang , W., Gao , J., Dong , Q., Qian , K., Li , P., Bi , W., & others (2025) Decider: A dual-system rule-controllable decoding framework for language generation. IEEE Transactions on Knowledge and Data Engineering

    2025

  28. [36]

    (2023) Imagereward: Learning and evaluating human preferences for text-to-image generation

    Xu , J., Liu , X., Wu , Y ., Tong , Y ., Li , Q., Ding , M., Tang , J., & Dong , Y . (2023) Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems 36:15903–15935

    2023

  29. [37]

    (2025) R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization

    Zhang , J., Huang , J., Yao , H., Liu , S., Zhang , X., Lu , S., & Tao , D. (2025) R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503.12937

    2025 arXiv

  30. [38]

    A., Shechtman , E., & Wang , O

    Zhang , R., Isola , P., Efros , A. A., Shechtman , E., & Wang , O. (2018) The unreasonable effectiveness of deep features as a perceptual metric. In CVPR

    2018

  31. [39]

    appearance quality

    Zhao , Y ., Huang , J., Hu , J., Wang , X., Mao , Y ., Zhang , D., Jiang , Z., Wu , Z., Ai , B., Wang , A., & others (2025) Swift: a scalable lightweight infrastructure for fine-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence 39, pp. 29733–29735. 12 A ...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.