Pith. sign in

REVIEW 3 major objections 4 minor 37 references

EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that an image-captioning evaluation metric can be both state-of-the-art at matching human scores and genuinely explainable, if trained on large-scale structured explanations paired with human judgments.

desk verdict EXPERT is a solid, useful reference-free metric that explains its scores, with two soft spots—cross-dataset leakage and a thin explanation-quality study—that need to be addressed in revision. read the letter →

arxiv 2506.24016 v1 pith:YCGZTW3E submitted 2025-06-30 cs.CL cs.AIcs.CV

Hyunjong Kim , Sangyeop Kim , Jongheon Jeong , Yeongjae Cho , Sungzoon Cho This is my paper
classification cs.CLcs.AIcs.CV
keywords imagecaptioningevaluationexplainablemetricreference-freevision-languagemodelstructuredexplanationsfluencyrelevancedescriptivenesshumanscoresmoothing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces EXPERT, a reference-free metric for evaluating image captions that returns not just a score but a structured justification. The justification is organized around three criteria—fluency, relevance, and descriptiveness—the same criteria used in recent human judgment datasets. To train EXPERT, the authors construct two large-scale explanation datasets, Polaris-exp and Nebula-exp, by having GPT-4o generate explanations for caption-image pairs that already carry human scores, then fine-tune a vision-language model on a two-stage prompt: first assign a score, then explain it. On six benchmark datasets, EXPERT achieves the best or second-best correlation with human judgments among reference-free metrics, and human annotators rate its explanations as significantly more consistent, factual, and informative than those of the previous explainable metric FLEUR. The paper argues that standardized criteria alone are not enough; supervision with high-quality paired explanations is what improves both scoring and explanation quality.

What carries the argument

The load-bearing machinery is a two-stage evaluation template that decomposes the metric's output into a score first, then a structured explanation. The scoring stage asks the VLM to assign a score from 0 to 1; scores are pre-processed by score binning (rounding to the nearest 0.10) to reduce token-level complexity. The explanation stage then asks for a brief justification organized by the three criteria—fluency, relevance, and descriptiveness—each with a fixed definition. At inference, the model is run greedily and score smoothing is applied: for each decimal place the model outputs a probability distribution over digits, and the expected value is computed as the final score. This template, together with supervised fine-tuning on the GPT-4o-generated explanation datasets, is what carries the claim that scoring and explanation can be jointly learned and improved.

What would settle it

Take a fresh set of 100 image-caption pairs, run EXPERT to produce a score and explanation for each, and give annotators only the explanation and the model's score, not the image or caption. Ask them to judge, on a 4-point scale, whether the explanation justifies the score. If EXPERT's mean justification rating is not significantly higher than FLEUR's on this protocol, or if the ratings fall below 3.0, the paper's central claim that EXPERT provides significantly higher-quality, score-consistent explanations would be falsified.

Watch

Extended reading notes

Core claim

EXPERT's central claim is that an image-captioning evaluation metric can be both state-of-the-art at predicting human scores and genuinely explainable, provided it is trained on large-scale structured explanations paired with human scores. The paper shows that by fine-tuning LLaVA-1.5 (13B) on Polaris-exp and Nebula-exp—two datasets of over 42,000 image-caption pairs, each with a GPT-4o-generated explanation structured by fluency, relevance, and descriptiveness—the resulting model achieves the best or second-best correlation with human judgments among reference-free metrics on all six tested benchmarks, and outperforms reference-based metrics on four of them. The explanation quality is then verified through human evaluation: on 100 held-out captions, four annotators rate EXPERT's explanations significantly higher than FLEUR's on consistency, factuality, and informativeness, with statistical significance at the 0.01 level. The paper claims this is the first systematic assessment of explanation quality for explainable captioning metrics.

Load-bearing premise

The training explanations generated by GPT-4o are accurate and consistent with the human scores; this is validated on only 100 sampled explanations.

Editorial extensions

If this is right

  • EXPERT achieves the best or second-best correlation with human judgments among reference-free metrics on Flickr8k-EX, Flickr8k-CF, COMPOSITE, Polaris, Nebula, and Pascal-50S.
  • EXPERT outperforms reference-based metrics (which use human-written reference captions) on Flickr8k-CF, COMPOSITE, Polaris, and Nebula.
  • Human raters score EXPERT's explanations significantly higher than FLEUR's on consistency, factuality, and informativeness, at a 0.01 significance level.
  • Score binning improves performance across all benchmark datasets, while score smoothing is essential for competitive results.
  • The combined Polaris-exp and Nebula-exp datasets (42,166 explanations) are the first large-scale structured explanation resources for image captioning evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The finding that standardized criteria alone (the EXPERTw/o SFT baseline) do not improve explanation quality suggests that future metrics should invest in curated explanation-corpus training rather than prompt engineering alone.
  • The structured three-criterion format may transfer to other pairwise or generative evaluation tasks—e.g., text-to-image generation or video captioning—where a single number is insufficient and a justification is required.
  • The error analysis shows a bias toward overpenalizing detail-light captions; a testable extension is to oversample concise captions with high human scores during training, which the authors themselves propose.
  • Since GPT-4o's score-smoothing performance is degraded by its tokenizer and limited log-probability access, open VLMs with full token-probability control may become preferred for score-smoothed evaluation, potentially lowering evaluation cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes EXPERT, a reference-free image captioning evaluation metric built by fine-tuning LLaVA-1.5 with a two-stage template: the model first generates a score for an image-caption pair and then produces a structured explanation along three criteria (fluency, relevance, descriptiveness). The training data are constructed by extending the Polaris and Nebula human-judgment datasets with GPT-4o-generated explanations, validated by a small human study. The authors report correlations on Flickr8k-EX, Flickr8k-CF, COMPOSITE, Polaris, Nebula, and Pascal-50S, claiming state-of-the-art results among reference-free metrics, and they report a human evaluation in which EXPERT's explanations are rated significantly higher than FLEUR's on consistency, factuality, and informativeness.

Significance. If the benchmark results are clean, EXPERT is a practically useful contribution: it offers an open, reference-free alternative to proprietary LLM judges while also producing standardized, human-checkable explanations. The paper's strengths include a large explanation-augmented training set, a clearly specified two-stage evaluation template, deterministic inference with greedy decoding, detailed training and inference-time reporting, and a promised release of code and data. The main risks are empirical rather than conceptual: potential cross-dataset train/test leakage for Polaris and Nebula, and the narrow basis of the explanation-quality claim, which rests on 100 samples from one dataset rated by four annotators. Both issues are checkable and fixable, so the underlying approach remains credible.

major comments (3)
  1. [Sections 2.3 and 3.1] The deduplication described in Section 2.3 is applied to the combined training splits of Polaris-exp and Nebula-exp, but the evaluation in Section 3.1 uses the official Polaris and Nebula test splits. Because Polaris and Nebula intentionally share overlapping image-caption pairs, a pair from the Polaris training split could appear in the Nebula test split (or vice versa), and Section 2.3's cross-dataset merge only averages duplicates inside the training data. In that case, EXPERT would have been trained on the exact test pair and could memorize its human score, inflating the Polaris and Nebula columns of Table 2 (61.1 vs. 58.3 and 54.9 vs. 51.7). Please verify that the union of the Polaris and Nebula test splits is disjoint from the other dataset's training pairs, or rerun the evaluation after deduplicating across the full train/test union before averaging scores. This is load-bearing for the state-of-the-art claim.
  2. [Section 3.3 and Figure 4] The claim that EXPERT provides significantly higher-quality explanations rests on a human evaluation of 100 image-caption pairs from Flickr8k-EX rated by four annotators. No test statistic, unit of analysis, or inter-annotator agreement measure is reported, so the 0.01 significance statement is not verifiable as presented. More importantly, a single dataset does not support the general wording in the abstract. Please report the statistical test details (e.g., a paired test on sample-level averaged ratings), add inter-annotator agreement, and either extend the evaluation to additional datasets or explicitly restrict the claim to the evaluated setting.
  3. [Section 2.1 and Table 1] The synthetic explanations in Polaris-exp and Nebula-exp are validated on only 100 sampled explanations from the combined set, rated by four annotators. If the GPT-4o explanations are not consistently aligned with the human scores with which they are paired, the fine-tuned model learns a score-explanation mapping that is not faithful to human reasoning; this would undermine the explanation-quality result in Section 3.3 even if the correlation numbers are unaffected. The reported mean ratings of 3.72-3.84 are reassuring, but the sample is small and the paper does not report per-score-interval agreement or annotator agreement. Please provide a larger validation sample or a stratified analysis by score interval, and add a corresponding caveat to the explanation-quality claim.
minor comments (4)
  1. [Abstract and Section 3.2] The abstract says 'state-of-the-art results,' but Table 2 shows that HICE-S outperforms EXPERT on Pascal-50S HC, MM, and Avg. Please qualify the summary statement as 'state-of-the-art among reference-free metrics except Pascal-50S' or otherwise align the abstract with the table.
  2. [Appendix D] The score-binning comparison in Table 4 reports only with and without binning at bin size 0.10. Since bin size is a hyperparameter, please add a small sensitivity analysis (e.g., 0.05, 0.10, 0.20) or justify the chosen value with reference to the tokenization behavior described in Appendix D.
  3. [Appendix E.2 and Table 5] The GPT-4o comparison is informative, but the conclusion that EXPERT outperforms GPT-4o depends on score smoothing, and GPT-4o's smoothing is constrained by the top-20 logprob limit. Please state more explicitly that the comparison is conditional on the score-smoothing procedure and that GPT-4o's results would change if full token probabilities were available.
  4. [Throughout] The model name 'LLaV A-1.5' appears with a spurious space in several places; please use 'LLaVA-1.5' consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EXPERT's scoring and explanation claims are supported by held-out human judgments and independent baselines, not by self-referential fits.

full rationale

The paper's derivation chain is empirically supervised rather than circular. The scoring component is trained on human scores from Polaris-exp and Nebula-exp and then evaluated on held-out human judgment splits, including external benchmarks (Flickr8k-EX, Flickr8k-CF, COMPOSITE, Pascal-50S); no score or correlation value is obtained by plugging the training labels back into the metric by construction. The explanation component is trained on GPT-4o-generated structured explanations, and the paper validates those synthetic explanations on a 100-sample human-evaluation subset; this is a data-quality check, not a definitional equivalence. The final explanation-quality claim is measured by human raters on 100 Flickr8k-EX samples, with the scores also generated by the metrics themselves, so consistency with the generated score is an external human judgment rather than an analytic identity. The paper's use of prior work (FLEUR, Polos, DENEB, HICE-S) is independent citation of other research groups; there is no load-bearing self-citation chain or uniqueness argument imported from the authors' own prior work. The only substantive concerns raised by the reviewer—small-sample validation of synthetic explanations and possible cross-dataset train/test leakage between Polaris and Nebula—are correctness and data-hygiene risks, not circular reasoning. In particular, the sentence 'we use the test split to ensure no overlap with the training data' asserts a data-splitting safeguard; if that safeguard fails, the harm would be memorization-based leakage, which is a distinct failure mode from equating inputs with outputs. Because no equation reduces to a fitted parameter and no prediction is defined in terms of its own target, the paper is self-contained against external benchmarks and receives a circularity score of 0.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the quality and consistency of the synthetic training explanations, which are validated on a small sample, and on inherited assumptions about the human judgment datasets. No new physical or conceptual entities are introduced.

free parameters (1)
  • score binning size = 0.10
    Chosen to round scores to nearest multiple of 0.10 during training; ablation in Table 4 shows consistent improvement, so the final metric depends on this choice, though it is a design hyperparameter rather than a fitted constant.
assumptions (5)
  • domain assumption Fluency, relevance, and descriptiveness are the fundamental and sufficient criteria for caption quality.
    Inherited from Polaris and Nebula dataset design (Section 2.1, Appendix I). The entire explanation structure rests on these three axes.
  • domain assumption GPT-4o-generated explanations in Polaris-exp and Nebula-exp are high-quality and consistent with human scores.
    Human evaluation of only 100 sampled explanations gives high average scores (Table 1, Section 2.1), but the full 42k explanations are not individually validated.
  • domain assumption Human judgments in Polaris and Nebula are reliable ground truth.
    The metric is trained on these scores and evaluated against them; the paper does not audit the original annotation quality.
  • domain assumption The test splits of Polaris and Nebula do not overlap with training data.
    The paper states this (Section 3.1) but does not provide a leakage check.
  • domain assumption LLaVA-1.5 (13B) is a sufficient base model for learning both scoring and explanation generation.
    The paper selects this model without comparing base models (Section 2.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations." pith.science (2026). https://pith.science/paper/YCGZTW3E

@misc{pith2026250624016,
  author       = {Pith},
  title        = {Pith review of: EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YCGZTW3E}},
  note         = {Machine review of arXiv:2506.24016}
}
read the original abstract

Recent advances in large language models and vision-language models have led to growing interest in explainable evaluation metrics for image captioning. However, these metrics generate explanations without standardized criteria, and the overall quality of the generated explanations remains unverified. In this paper, we propose EXPERT, a reference-free evaluation metric that provides structured explanations based on three fundamental criteria: fluency, relevance, and descriptiveness. By constructing large-scale datasets of high-quality structured explanations, we develop a two-stage evaluation template to effectively supervise a vision-language model for both scoring and explanation generation. EXPERT achieves state-of-the-art results on benchmark datasets while providing significantly higher-quality explanations than existing metrics, as validated through comprehensive human evaluation. Our code and datasets are available at https://github.com/hjkim811/EXPERT.

Figures

Figures reproduced from arXiv: 2506.24016 by the authors.

Figure 1
Figure 1. Example of a structured explanation. has two limitations. First, the explanations pro￾vided by existing metrics are not generated based on standardized criteria or guidelines, which may lead to inconsistency in content and structure. Sec￾ond, previous studies lack a thorough evaluation of the quality of generated explanations, leaving the overall quality unverified. To address these issues, we propose EXPERT1 , a VL… view at source ↗
Figure 2
Figure 2. Overall framework of EXPERT. EXPERT generates significantly higher-quality ex￾planations than existing explainable metrics. To the best of our knowledge, we are the first to sys￾tematically assess the explanation capabilities of explainable image captioning evaluation metrics. 2 EXPERT 2.1 Dataset Construction Generating Explanations Existing image cap￾tioning datasets consist of image-caption pairs with the corresp… view at source ↗
Figure 3
Figure 3. Examples of explanations generated by EXPERT and FLEUR. Green text indicates valid justifications for [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Human evaluation results on explanation qual [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Evaluation form for human evaluation of Polaris-exp and Nebula-exp datasets. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Evaluation form for human evaluation of explanations of FLEUR, EXPERT [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Additional examples of EXPERT. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 11 canonical work pages

  1. [1]

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S \'e bastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905

    arXiv 2024

  2. [2]

    Somak Aditya, Yezhou Yang, Chitta Baral, Cornelia Fermuller, and Yiannis Aloimonos. 2015. From images to sentences through scene description graphs using commonsense reasoning and knowledge. arXiv preprint arXiv:1511.03292

    arXiv 2015

  3. [3]

    Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. Spice: Semantic propositional image caption evaluation. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, pages 382--398. Springer

    2016

  4. [4]

    David Chan, Suzanne Petryk, Joseph Gonzalez, Trevor Darrell, and John Canny. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.841 CLAIR : Evaluating image captions with large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13638--13646, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.841 2023

  5. [5]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  6. [6]

    Taraneh Ghandi, Hamidreza Pourreza, and Hamidreza Mahyar. 2023. https://doi.org/10.1145/3617592 Deep learning approaches on image captioning: A review . ACM Comput. Surv., 56(3)

    doi:10.1145/3617592 2023

  7. [7]

    Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung, Yizhou Yu, Ping Luo, and Sifei Liu. 2024. Regiongpt: Towards region understanding vision language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13796--13806

    work page 2024

  8. [8]

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.595 CLIPS core: A reference-free evaluation metric for image captioning . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514--7528, Online and Punta Cana, Dominican Republic. Associa...

    doi:10.18653/v1/2021.emnlp-main.595 2021

Show all 37 references
  1. [9]

    Micah Hodosh, Peter Young, and Julia Hockenmaier. 2013. Framing image description as a ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research, 47:853--899

    2013

  2. [10]

    Anwen Hu, Shizhe Chen, Liang Zhang, and Qin Jin. 2023. https://doi.org/10.18653/v1/2023.acl-long.178 I nfo M et IC : An informative metric for reference-free image caption evaluation . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (...

    2023 doi

  3. [11]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  4. [12]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  5. [13]

    Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, and Jianfeng Gao. 2019. https://doi.org/10.18653/v1/D19-1220 TIGE r: Text-to-image grounding for image caption evaluation . In Proceedings of the 2019 Conference on Empirical Methods in Nat...

    2019 doi

  6. [14]

    Alon Lavie and Abhaya Agarwal. 2007. Meteor: an automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, StatMT '07, page 228–231, USA. Association for Computational Linguistics

    2007

  7. [15]

    Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, and Kyomin Jung. 2021. https://doi.org/10.18653/v1/2021.acl-short.29 UMIC : An unreferenced metric for image captioning via contrastive learning . In Proceedings of the 59th Annual Meeting of the Association for Compu...

    2021 doi

  8. [16]

    Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, and Kyomin Jung. 2020. https://doi.org/10.18653/v1/2020.eval4nlp-1.4 V i LBERTS core: Evaluating image caption using vision-and-language BERT . In Proceedings of the First Workshop on Evaluation and Comp...

    2020 doi

  9. [17]

    Yebin Lee, Imseong Park, and Myungjoo Kang. 2024. https://doi.org/10.18653/v1/2024.acl-long.205 FLEUR : An explainable reference-free evaluation metric for image captioning using a large multimodal model . In Proceedings of the 62nd Annual Meeting of the Association for Comput...

    2024 doi

  10. [18]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888--12900. PMLR

    2022

  11. [19]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  12. [20]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024

  13. [21]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892--34916

    2023

  14. [22]

    Kazuki Matsuda, Yuiga Wada, and Komei Sugiura. 2024. Deneb: A hallucination-robust automatic evaluation metric for image captioning. In Proceedings of the Asian Conference on Computer Vision, pages 3570--3586

    2024

  15. [23]

    OpenAI. 2024. https://arxiv.org/abs/2410.21276 Gpt-4o system card

    2024 arXiv

  16. [24]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  17. [25]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  18. [26]

    Cyrus Rashtchian, Peter Young, Micah Hodosh, and Julia Hockenmaier. 2010. https://aclanthology.org/W10-0721/ Collecting image annotations using A mazon`s M echanical T urk . In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with A mazon`s Mecha...

    2010

  19. [27]

    Sara Sarto, Manuele Barraco, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. 2023. Positive-augmented contrastive learning for image and video captioning evaluation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6914--6924

    2023

  20. [28]

    Sara Sarto, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. 2024. Bridge: Bridging gaps in image captioning evaluation with stronger visual cues. In European Conference on Computer Vision, pages 70--87. Springer

    2024

  21. [29]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  22. [30]

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566--4575

    2015

  23. [31]

    Yuiga Wada, Kanta Kaneda, Daichi Saito, and Komei Sugiura. 2024. Polos: Multimodal metric learning from human feedback for image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13559--13568

    2024

  24. [32]

    Bo Wan, Michael Tschannen, Yongqin Xian, Filip Pavetic, Ibrahim M Alabdulmohsin, Xiao Wang, Andr \'e Susano Pinto, Andreas Steiner, Lucas Beyer, and Xiaohua Zhai. 2024. Locca: Visual pretraining with location-aware captioners. Advances in Neural Information Processing Systems,...

    2024

  25. [33]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  26. [34]

    Zequn Zeng, Jianqiao Sun, Hao Zhang, Tiansheng Wen, Yudi Su, Yan Xie, Zhengjue Wang, and Bo Chen. 2024. Hicescore: A hierarchical metric for image captioning evaluation. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 866--875

    2024

  27. [35]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with bert . In International Conference on Learning Representations

    2020

  28. [36]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  29. [37]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.