Pith. sign in

REVIEW 4 major objections 5 minor 26 references

Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MiRAGE evaluates multimodal RAG by scoring every claim a system makes against sources.

desk verdict Claim-level multimodal RAG evaluation framework worth engaging with, but the human-alignment claim rests on a statistically impossible Kendall's tau and an overbroad abstract. read the letter →

arxiv 2510.24870 v2 pith:PET6MDH4 submitted 2025-10-28 cs.CL cs.CVcs.IR

Alexander Martin , William Walden , Reno Kriz , Dengjia Zhang , Kate Sanders , Eugene Yang , Chihsheng Jin , Benjamin Van Durme This is my paper · ORCID
classification cs.CLcs.CVcs.IR
keywords multimodalretrievalaugmentedgenerationclaimdecompositionfactualityevaluationcitationsupportvideo-basedRAGINFOF1CITEF1human
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces MiRAGE, an evaluation framework for retrieval-augmented generation from multimodal sources such as video, audio, and images. Its central claim is that all information in any modality can be decomposed into textual subclaims, and that scoring those subclaims for factual support (INFOF1) and citation support (CITEF1) captures output quality better than text-centric metrics. The paper reports that when humans apply MiRAGE, its scores align strongly with extrinsic judgments of quality on the WikiVideo article-generation task, while three leading text-based RAG metrics show poor or negative agreement. If this holds, MiRAGE gives the field a way to judge whether a system's answer is factually grounded and properly cited, rather than merely lexically similar to a reference.

What carries the argument

The machinery is subclaim decomposition plus a binary support scoring function $s(p,h)$. Sentences in both the prediction and the reference are broken into declarative subclaims, and each subclaim is scored as supported or not by the cited or collection evidence for INFOF1, or by a reference or citation proxy for CITEF1. The metrics are precision-recall pairs—INFOF1-P and INFOF1-R for factuality and coverage, CITEF1-P and CITEF1-R for citation support and attribution—with F1 as the summary. Claim-level granularity is what lets the metric verify each atomic proposition against a video instead of requiring an entire multi-claim sentence to be supported at once.

What would settle it

Take a video where the decisive fact is a continuous, non-linguistic property—for example, that a temperature trend rose steadily over 24 hours with no single numeric claim being independently supported—generate an output that states that fact, and ask humans for extrinsic quality scores while MiRAGE scores the subclaims. If humans rate the output high but MiRAGE rates it low because the fact cannot be decomposed into discrete supported subclaims, the paper's decomposition premise is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that claim-level, source-grounded evaluation transfers to multimodal RAG: decomposing generated and reference sentences into subclaims, then measuring support precision and recall against either the evidence collection or a reference, produces scores that track human judgments of quality, whereas sentence-level TextRAG metrics fail. On WikiVideo, human-annotated MiRAGE agrees with extrinsic quality judgments at Kendall's tau between roughly 0.49 and 0.81, while ROUGE and BERTScore capture what information is present but not whether it is grounded in cited sources. The paper also shows that automatic verification is the bottleneck: zero-shot vision-language models are not calibrated for subclaim-versus-video support judgments, with near-chance performance in one 7-billion-parameter configuration, and supervised next-token training improves calibration.

Load-bearing premise

The load-bearing premise is that all information in a video, image, or audio source can be decomposed into discrete textual subclaims whose support can be judged by a scoring function; if some content (a continuous visual trend, a melody, a non-linguistic relational fact) is not expressible as such a subclaim, INFOF1 and CITEF1 will simply miss it.

Editorial extensions

If this is right

  • On WikiVideo, human-annotated MiRAGE correlates with extrinsic quality judgments at Kendall tau 0.49–0.81, well above ROUGE and BERTScore, so it can rank systems by usefulness rather than surface similarity.
  • ALCE, ARGUE, and RAGAS, when adapted to video, show low or negative agreement with human judgments because they verify at sentence level, assume concatenable sources, or assume exhaustive extraction from video.
  • ROUGE and BERTScore track what information is present, but not whether it is grounded in cited sources; a system can score high on them while citing nothing.
  • Zero-shot vision-language claim verification is near chance for subclaim-video pairs, so automatic MiRAGE is currently unreliable; supervised next-token training raises the reported F1 substantially, suggesting calibration is achievable.
  • For practical evaluation the paper recommends a quick reference-based suite (ROUGE, BERTScore, INFOF1-ref, CITEF1-ref) and a comprehensive collection-based suite that adds RAGAS-Faithfulness for full grounding checks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: if the decomposition premise is right, the same subclaim-scoring pipeline should rank systems on image-only and audio-only RAG without retuning; a cross-modal consistency check would compare MiRAGE ranks to extrinsic quality judgments on each modality.
  • The framework implicitly defines a research program: train a single vision-language or language verifier on subclaim grounding across modalities. The paper's appendix suggests supervised next-token training is a promising baseline, and confidence-calibrated scalar judgments would sharpen the ranking.
  • Because CITEF1-recall takes the max over cited videos, a system could use it as a training signal to suppress citations that do not actually contain the claimed information, making citation generation optimizable end-to-end.
  • The reference-precision variant of INFOF1 conflates factuality with salience, because a true but non-salient claim gets penalized; evaluators who want pure factuality should prefer collection precision, a distinction the paper leaves as a practical choice for the user.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces MiRAGE, a claim-level evaluation framework for multimodal retrieval-augmented generation (RAG). MiRAGE decomposes generated and reference text into subclaims and computes INFOF1 (factuality and information coverage) and CITEF1 (citation support and attribution), with reference-based and reference-free (collection-based) variants. The authors collect three kinds of human judgments on 10 WikiVideo topics from three generation systems (LLM-only, CAG-2 Oracle, CAG-2 RAG), compare MiRAGE and three adapted text-based RAG metrics (ALCE, ARGUE, RAGAS) against these judgments using Kendall's tau, and report experiments on automatic LLM/VLM claim verification. They conclude that human-applied INFOF1 aligns strongly with extrinsic quality judgments, that text-centric metrics fail to capture grounding, and that current automatic verifiers are poorly calibrated. The paper releases open-source implementations and provides detailed appendices with metric formulations, annotation instructions, and additional correlation tables.

Significance. If the central claim is narrowed appropriately, this is a useful contribution. The claim-level decomposition is well motivated, the adaptation of existing text-based RAG metrics to multimodal sources is clearly described, and the release of open-source code plus detailed annotation protocols supports reproducibility. The paper is also honest in reporting that automatic verifiers are not yet calibrated (Appendix H, Table 10), and the circularity concern about metric validation does not materialize: the automatic verifier is trained on 47 held-out topics, and the human metric judgments are collected by independent annotation protocols rather than fitted to the extrinsic judgments. However, the headline claim that 'MIRAGE, when applied by humans, strongly aligns with extrinsic judgments of output quality' is only directly supported for the INFOF1 component, not for CITEF1, and the correlation evidence rests on a very small sample without significance testing. The framework itself is a reasonable scaffold for future work, but the validation section needs to be re-scoped and the recommendations reconciled with the reported automatic-metric performance.

major comments (4)
  1. [Abstract and §5, Table 2] The abstract states that 'MIRAGE, when applied by humans, strongly aligns with extrinsic judgments of output quality,' but the evidence in Table 2 supports only the INFOF1 component. The intrinsic claim judgments (ICJ), which mirror INFOF1-Ref, show Kendall's tau between 49.0 and 81.2 against the extrinsic quality judgments (EQJ), while the grounding judgments (GJ), which mirror CITEP, show tau of 18.2, -5.5, and 23.0, i.e., near zero or negative. Since CITEF1 is part of MiRAGE, the headline claim should be narrowed to INFOF1, or the abstract and Section 5 need to report separately the components that are validated and the components that are not.
  2. [§5 Evaluation Setup and Tables 1-5] The human-validation analysis is computed over only 10 topics and 3 systems, and no confidence intervals or significance tests are reported for any Kendall's tau value. With 3 systems, the rank correlation is heavily quantized and can be dominated by coarse system-level differences rather than by the ability of a metric to discriminate quality at the topic level. Since the central claim of the paper is the strong alignment between human-applied MiRAGE and extrinsic quality judgments, the absence of any uncertainty quantification is load-bearing. I would like to see at least a bootstrap confidence interval, a per-topic breakdown, or a paired significance test.
  3. [§5, Table 1, and §6] Section 5 states that 'the agreement across the board for these automatic metrics is low, outside of ROUGE and BERTScore,' and Appendix H (Table 10) shows that a zero-shot VLM verifier achieves F1=41.7, below chance on a balanced test set. Nevertheless, Section 6 recommends that evaluators report automatic INFOF1 and CITEF1 (both reference and collection variants) as part of both the quick and comprehensive evaluation suites. This recommendation is inconsistent with the paper's own evidence that the automatic implementations are not calibrated for support judgments. The recommendations should be re-scoped to human-annotated or explicitly calibrated variants, or the paper should provide a concrete decision rule for when automatic MiRAGE scores are reliable enough to use.
  4. [§5, Tables 1 and 3] The text in Section 5 claims that 'claim-based precision metrics, RAGAS Faith and INFOP, not only align with EQJs and ICJs, but also with GJs' and that 'CITEF1 has the highest agreement for both EQJs and GJs.' The values in Table 1 show INFOP agreement with GJ of 6.6, 21.5, and 1.8, and RAGAS Faith agreement with GJ of 24.8; these are weak correlations, not clear alignment. In Table 3, the highest EQJ value appears under the ALCE Citation Quality column (54.3), not under CITEF1. The claims need to be re-derived from the actual table values, and the table headers/labels need to be clarified so that the comparison is interpretable.
minor comments (5)
  1. [Throughout] There are numerous typos and misspellings, including 'mulitmodal' in the abstract, 'effectievly' and 'groudns' in §4.2, 'asseses' in the Figure 1 caption, 'alterantive' in §4.2.1, 'wills tart' in Appendix B.2, and 'agrement' in §5. A careful proofreading pass is needed.
  2. [Appendix B, Eqs. (8) and (11)] The weighted precision and recall formulations are written as 1/|C| * sum(s * I_i). If I_i is intended as an importance weight, the denominator should be sum(I_i) rather than the number of claims; otherwise the weighted score depends on the arbitrary scale of I and is not a proper weighted proportion. Please clarify the intended normalization.
  3. [Table 3] Table 3 has four data rows but the caption says 'Row 1: EQJ, Row 2: GJ,' and the column header 'R C R CQ SS CR' is ambiguous. Please give a clear two-row header that identifies each metric variant and enumerates the annotator rows explicitly.
  4. [Appendix F] For RAGAS Context Relevance, the paper says the denominator is simulated by eliciting a detailed VLM summary, but the connection to the original formula 'extracted sentences / total sentences' is not made explicit. Please provide the exact formula used for the multimodal adaptation.
  5. [Appendix H] The balanced test set used to report verifier F1 is described, but it is not stated whether the reported F1 is macro-averaged or micro-averaged, or how the 787-claim evaluation set was balanced relative to the original label distribution. Please specify this for reproducibility.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor construct-overlap in the human validation of INFOF1; the automatic-evaluation and verifier claims are not circular.

  1. self definitional [Section 5 (Tables 1-2), Figure 9 (EQJ instructions), Figure 10 (ICJ instructions), and Section 4.1 (INFOF1 definition)]
    ""Consistency/Factuality: Does the article make only true statements about the topic in question, given what the reference says about that topic? ... Adequacy: Does the article adequately capture all of the information contained in the reference article?" (Fig. 9); "The goal of this task is to mirror how we evaluate multimodal RAG in InfoF1." (Fig. 10)"

    The claimed validation of human-applied INFOF1 is the agreement between intrinsic claim judgments (ICJ) and extrinsic quality judgments (EQJ). The EQJ rubric defines quality in terms of the same two quantities INFOF1 computes: factuality as "true statements ... given what the reference says" and adequacy as capturing "all of the information contained in the reference article." The ICJ instructions then explicitly instruct annotators to "mirror how we evaluate multimodal RAG in InfoF1." Thus the reported Kendall tau values (0.49-0.81) partly measure agreement between two instruments built from the same reference-anchored definition of factuality and coverage. Some of the "alignment" is inherited from the shared rubric rather than discovered empirically.

full rationale

The paper's central derivation chain is otherwise self-contained. INFOF1 and CITEF1 are defined by explicit equations (1)-(5) over subclaims; no parameter is fitted to the evaluation data. The automatic verifier is trained on 47 WikiVideo topics held out from the 10 evaluation topics, so the report of low automatic agreement (and near-chance zero-shot VLM verification, Appendix H) is an honest, non-circular finding. The Citation-support results are presented as weak or negative (Table 2, GJ near zero), which undercuts rather than manufactures the headline claim. The main circularity concern is limited to the human validation of INFOF1, where the EQJ rubric and the ICJ/INFOF1 instructions operationalize the same reference-relative factuality and coverage constructs; this is a construct-overlap rather than a fitted-input or self-citation chain, and the framework's automatic evaluation claims retain independent content.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

MiRAGE introduces no new physical entities or fitted parameters. Its reliance is on the subclaim decomposition premise, the exhaustiveness of references, and the adequacy of model-based support scoring. The trained verifiers in Appendix H are standard models, not parameters of the metric itself.

assumptions (3)
  • domain assumption All information in any modality can be decomposed into unimodal linguistic subclaims.
    Core premise of MIRAGE stated in the Introduction. If visual, audio, or relational content cannot be captured as discrete textual subclaims, INFOF1 and CITEF1 will miss that information.
  • domain assumption The human-written reference is a complete extraction of relevant information from the source collection.
    Underlies reference-based INFOF1 and CITEF1. Appendix C acknowledges references are non-exhaustive, which can penalize factual claims that are not salient in the reference.
  • domain assumption The scoring function s(p,h) can be computed reliably by LLM or VLM support judgments.
    Automatic MiRAGE depends on model-based support judgments. Appendix H shows zero-shot LLM and VLM verifiers are not calibrated for this task, so the automatic variant inherits this fragility.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation." pith.science (2026). https://pith.science/paper/PET6MDH4

@misc{pith2026251024870,
  author       = {Pith},
  title        = {Pith review of: Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PET6MDH4}},
  note         = {Machine review of arXiv:2510.24870}
}
read the original abstract

We introduce MiRAGE, an evaluation framework for retrieval-augmented generation (RAG) from multimodal sources. As audiovisual media becomes a prevalent source of information online, it is essential for RAG systems to integrate information from these sources into generation. However, existing evaluations for RAG are text-centric, limiting their applicability to multimodal settings. MiRAGE is a claim-centric approach to multimodal RAG evaluation, consisting of InfoF1, which assesses factuality and information coverage, and CiteF1, which assesses citation support and completeness. We show that, when applied by humans, MiRAGE strongly aligns with extrinsic judgments of output quality. We additionally introduce an automatic implementation of MiRAGE as well as multimodal variants of three prominent text-based RAG metrics -- ALCE, ARGUE, and RAGAS -- demonstrating the limitations of text-centric work and laying the groundwork for automatic evaluation. We release open-source implementations and outline evaluation methods for multimodal RAG.

Figures

Figures reproduced from arXiv: 2510.24870 by the authors.

Figure 1
Figure 1. MIRAGE asseses predictions at the claim level, evaluating information factuality and coverage, and citation support and attribution, enabling RAG eval￾uation grounded in text, image, audio, and video. Evaluation. Inspired by factuality evaluations (Min et al., 2023), the core of MIRAGE is the idea that all information—whether represented in text, im￾age, audio, or video—can be decomposed into uni￾modal linguistic ob… view at source ↗
Figure 2
Figure 2. MIRAGE decomposes generated text into subclaims to assess two dimensions of quality: INFOF1, which measures factuality and information coverage, and CITEF1, which measures citation precision and completeness. The framework enables consistent, claim-level evaluation across text, image, audio, and video modalities. the proportion of predicted subclaims supported by the evidence—and INFOR—the proportion of reference su… view at source ↗
Figure 3
Figure 3. Zero-Shot Prompt for Claim Verification in Videos [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Zero-Shot Prompt for Claim Verification in Text [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Zero-Shot Prompt for Claim Verification in Citation (Text) [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Zero-Shot Prompt for ALCE Citation Quality [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Zero-Shot prompt for extracting the detailed summary used in the denominator of [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 9
Figure 9. Figure 9: Instructions for the human likert judgments (EQJ) [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Instructions for the human metric judgments (ICJ) [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 19 canonical work pages

  1. [1]

    We find a large discrepancy between support annotated by a zero-shot LLM and zero-shot VLM for INFOF1 precision, with agreement significantly dropping when annotating sup- port with a VLM

    work page

  2. [2]

    We find that LLMs struggle with recall judg- ments. In Appendix H, we explore the performance of the underlying LLM and VLM on verifying claims in videos or documents, but broadly note the cal- ibration of these models as an interesting avenue for future work. Paper ImplementationFor INFOF1, we im- plement the single-source variation in our paper, prompti...

    work page 2024

  3. [3]

    statements

    We find that the only metric citation metric that correlates with human grounding judg- ments is INFOF1 Precision. 15 Paper ImplementationFor CITEF1, we im- plement the single-source variation in our paper, prompting the verifiers with Figure 3 for the VLM and either Figure 4 (precision) or Figure 5 (recall) for the LLM. C Reference vs Collection Metrics ...

    work page 2023

  4. [4]

    A claim isNOT SUPPORTED (False)if:

    If the claim includesquantitative or categorical facts(numbers, names, locations, affiliations), these details must exactly match what is stated in the other article. A claim isNOT SUPPORTED (False)if:

    work page

  5. [9]

    Siddharth Vashishtha, Alexander Martin, William Gantt, Benjamin Van Durme, and Aaron White

    Support evaluation for the trec 2024 rag track: Comparing human versus llm judges.Preprint, arXiv:2504.15205. Siddharth Vashishtha, Alexander Martin, William Gantt, Benjamin Van Durme, and Aaron White. 2024. FA- MuS: Frames across multiple sources. InProceed- ings of the 2024 Conference of the North American Chapter of the Association for Computational Li...

    arXiv 2024

  6. [10]

    doc- ument,

    AlignScore: Evaluating factual consistency with a unified alignment function. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11328–11348, Toronto, Canada. Association for Computational Linguistics. Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. 2021. O...

    arXiv 2021

  7. [13]

    smoke billowed from the tower

    We interestingly find that Collection Precision has higher correlation with extrinsic quality judgments than the reference version. We be- lieve this is due to the descriptive nature of VLMs, which often produce subclaims like “smoke billowed from the tower.” We dis- cuss this more in Appendix C, but this low agreement is broadly due to the reference pre-...

    work page

  8. [14]

    smoke billowed from the tower

    We note that the low agreement between ICJ precision and CITEF1 precision is expected. There’s a difference in a claim being sup- ported in a video “smoke billowed from the tower” and what a human would include in an article. However, high recall is expected be- cause it matters if the information is recalled from the reference to then be matched to the s...

    work page

Show all 26 references
  1. [16]

    Extrinsic Quality Judgments. Human anno- tators are given information about the topic and are asked to consider the following at- tributes (in order of importance): factuality, adequacy, coherence, relevancy, and fluency

  2. [17]

    Intrinsic Claim Judgments. Human anno- tators are given the prediction, predicted sub- claims, reference, and reference subclaims, and are asked to annotate whether or not a claim is supported by the other text.This gives human annotations for INFOF1 (ref)

  3. [18]

    Grounding Judgments. Human annotators are given all the relevant videos for a topic and the predicted subclaims and asked to annotate whether or not the subclaims are supported by (grounded in) the videos. For each grounding judgment, we calculate the ranking between the three...

    2025

  4. [19]

    the [event] happened at [a time] in [a place]

    and Qwen2.5-VL-7B(Bai et al., 2025), re- spectively. These heads output prediction proba- bilities, which are then aggregated to determine whether a given claim is supported by the corre- sponding video or article evidence. For the NT approach, we leverage the genera- tive cap...

    2025

  5. [20]

    •Minor wording differences are acceptable if they don’t change meaning

    Themeaning and intentof the claim arefully consistentwith the information in the article. •Minor wording differences are acceptable if they don’t change meaning

  6. [21]

    Thetemporal or causal contextmatches (e.g., dates, events, outcomes are consistent)

  7. [23]

    The other articlecontradictsany part of the claim

  8. [24]

    The other articleomits or is ambiguousabout key details needed to verify the claim

  9. [25]

    The claim requiresinference beyond what’s stated, such as outside knowledge, assumptions, or general reasoning not grounded in the text

  10. [26]

    The claim ispartially supported, but not fully — i.e., some parts are correct while others are missing or uncertain. General rule: Only mark a claim asTrue(supported) if you can clearly point to a sentence or set of sentences in the other article that fully confirm it, leaving...

    2025

  11. [2015]

    InProceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing, pages 379–389, Lisbon, Portugal

    A neural attention model for abstractive sen- tence summarization. InProceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing, pages 379–389, Lisbon, Portugal. Association for Computational Linguistics. Saron Samuel, Dan DeGenaro, Jimena Guallar...

    2015

  12. [2018]

    InProceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1797–1807, Brussels, Bel- gium

    Don’t give me the details, just the summary! topic-aware convolutional neural networks for ex- treme summarization. InProceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1797–1807, Brussels, Bel- gium. Association for Computational L...

    2018 arXiv

  13. [2019]

    InProceedings of the 23rd Conference on Computa- tional Natural Language Learning (CoNLL), pages 404–418, Hong Kong, China

    Automated pyramid summarization evaluation. InProceedings of the 23rd Conference on Computa- tional Natural Language Learning (CoNLL), pages 404–418, Hong Kong, China. Association for Com- putational Linguistics. Anisha Gunjal and Greg Durrett. 2024. Molecular facts: Desiderat...

    2024 arXiv

  14. [2021]

    InProceed- ings of the 16th Conference of the European Chap- ter of the Association for Computational Linguistics: Main Volume, pages 634–644, Online

    GRIT: Generative role-filler transformers for document-level event entity extraction. InProceed- ings of the 16th Conference of the European Chap- ter of the Association for Computational Linguistics: Main Volume, pages 634–644, Online. Association for Computational Linguistic...

    2024

  15. [2022]

    InProceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing, pages 2648–2674, Abu Dhabi, United Arab Emirates

    Calibrating zero-shot cross-lingual (un- )structured predictions. InProceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing, pages 2648–2674, Abu Dhabi, United Arab Emirates. Association for Computa- tional Linguistics. Zhengping Jiang, Anqi Li...

    2022 arXiv

  16. [2023]

    InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language 9 Processing, pages 6465–6488, Singapore

    Enabling large language models to generate text with citations. InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language 9 Processing, pages 6465–6488, Singapore. Associa- tion for Computational Linguistics. Yanjun Gao, Chen Sun, and Rebecca J. Passonneau

    2023

  17. [2024]

    InFind- ings of the Association for Computational Linguistics: EMNLP 2024, pages 5042–5063, Miami, Florida, USA

    FaithScore: Fine-grained evaluations of hallu- cinations in large vision-language models. InFind- ings of the Association for Computational Linguistics: EMNLP 2024, pages 5042–5063, Miami, Florida, USA. Association for Computational Linguistics. William Jurayj, Jeffrey Cheng, ...

    2024 arXiv

  18. [2025]

    Zhengping Jiang, Anqi Liu, and Benjamin Van Durme

    Conformal linguistic calibration: Trading- off between factuality and specificity.Preprint, arXiv:2502.19110. Zhengping Jiang, Anqi Liu, and Benjamin Van Durme

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.