Pith. sign in

REVIEW 3 major objections 5 minor 37 references

Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Conformal-RAG claims that a retrieval-aware relevance score can certify sub-claim factuality in RAG answers with probability at least $1-\alpha$, while discarding far fewer claims than an LLM-confidence-only filter.

desk verdict A sound and clearly argued conformal RAG extension whose central guarantee rests on an unvalidated LLM annotator; worth refereeing with a mandatory annotator-validation request. read the letter →

arxiv 2506.20978 v1 pith:PT7L4HT3 submitted 2025-06-26 cs.IR

Naihe Feng , Yi Sui , Shiyi Hou , Jesse C. Cresswell , Ga Wu This is my paper · ORCID
classification cs.IR
keywords retrieval-augmentedgenerationconformalpredictionfactualityguaranteesub-claimfilteringgroup-conditionalcoveragehallucinationdetectionquestionansweringLLM-as-a-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Conformal-RAG claims that the factuality of a retrieval-augmented generation (RAG) answer can be guaranteed per sub-claim without needing ground truth at inference time. The method calibrates a threshold on retrieval-informed relevance scores via conformal prediction, so that the retained claims are entailed by the true answer with probability at least $1-\alpha$. On four question-answering benchmarks, this keeps up to 60% more high-quality sub-claims than applying conformal factuality directly to an LLM, at the same factuality target. A conditional extension gives the same guarantee per group (domain, topic, difficulty) using Mondrian conformal prediction.

What carries the argument

The load-bearing object is the nested filtering family $F_q(\hat{y}) = \{c \in \hat{y} : R(c) \ge q\}$ built from the retrieval-weighted relevance score $R(c)$ of Algorithm 1, together with the per-example conformal score $S(x_i,y_i^*)$ of eq. (5), the smallest threshold at which all surviving claims pass the automatic annotator $A$. Calibrating the quantile of $S$ over the calibration set produces $\hat{q}$; because $F_q$ is nested, the event that the test score falls below $\hat{q}$ implies every retained claim is factual, which is exactly the coverage event in eq. (3).

What would settle it

Take the calibrated threshold and evaluate the retained claims on a held-out test set against independent human factuality judgments; if the human-judged fraction of retained claims that are true falls below $1-\alpha$ even though the paper's annotator labels them true, the annotation assumption is violated and the coverage guarantee fails.

Watch

Extended reading notes

Core claim

The core claim is that retrieval context, not just the LLM's confidence, should define the conformity score in conformal factuality. Conformal-RAG scores each sub-claim $c$ by $R(c)=\max_j \cos(x,d_j)\cos(c,d_j)$, the best query-relevance-weighted match against retrieved documents, and filters claims whose score falls below a calibrated threshold $\hat{q}$. The threshold is the conformal quantile of per-example scores $S(x_i,y_i^*)$, each defined as the smallest $q$ at which all claims surviving $F_q$ are judged factual by an LLM annotator. Under the assumption that the annotator is correct on calibration data, the retained set satisfies $P(y^*_{test} \Rightarrow y_{test}(x_{test};\hat{q})) \ge 1-\alpha$, and the Mondrian variant satisfies the same inequality conditionally on each group. The experiment shows the retrieval-aware score removes far fewer claims: at an 85% factuality target on FActScore, it removes 8.9% of claims versus 86.8% for the LLM-confidence baseline.

Load-bearing premise

The method's guarantee is only as good as the automatic judge that decides whether each sub-claim is true: if the judge makes mistakes on the calibration data, the promised factuality level is calibrated to those mistakes rather than to actual factuality.

Editorial extensions

If this is right

  • A RAG system can publish filtered answers with a formal statement: with probability at least $1-\alpha$, the answer's retained sub-claims are entailed by ground truth.
  • Ground-truth answers are only needed for calibration, not at inference, so the method applies to open-domain question answering at scale.
  • Group-conditional calibration yields per-topic or per-domain factuality guarantees, reducing uneven coverage across subgroups.
  • The retriever's relevance signal does double duty: it boosts factuality coverage and removes off-topic but factual claims, making answers both safer and more focused.
  • The framework is agnostic to the underlying generator and retriever, so it can be layered on existing RAG systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the guarantee inherits the annotator's judgments, a direct validation on human-annotated factuality would be needed before deployment; the paper does not provide one.
  • The relevance score could be replaced by a learned cross-encoder or an LLM-based relevance judge; the conformal guarantee would survive as long as the score orders claims consistently, possibly improving retention further.
  • One could apply the same filtering recursively: after removing low-relevance claims, re-run the RAG pipeline on the surviving claims to regenerate longer answers while keeping the coverage guarantee.
  • The coverage statement is about entailment by the ground-truth answer, not about downstream utility; a factual but unhelpful claim can still pass.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents Conformal-RAG, a conformal-prediction framework for filtering sub-claims of RAG-generated answers. A relevance score per sub-claim is computed from cosine similarities between the query, retrieved documents, and claim; a GPT-4o annotator labels calibration sub-claims as factual or non-factual against ground-truth answers; and a conformal threshold is calibrated to guarantee that retained sub-claims are entailed by the ground truth with probability at least 1-alpha. The method is extended to group-conditional (Mondrian) coverage by calibrating separate thresholds per group. Experiments on FActScore, PopQA, HotpotQA, and MedLFQA compare Conformal-RAG with a Conformal-LLM baseline and report retention gains up to 60% at the same target factuality level.

Significance. The central statistical construction is standard and correctly nested; if the annotation assumption holds, Eq. (3) provides a valid marginal guarantee and the Mondrian extension provides per-group coverage. The paper ships a practical implementation, builds on a clearly identified prior method, and evaluates on four benchmarks, giving a useful baseline for CP-based RAG filtering. Its significance is, however, conditional on the factuality judge being reliable: no human validation of the automatic annotator is provided, and the test-time factuality evaluation may use the same annotator, which risks circularity. The guarantee also applies to the filtered claim set, not to the final merged response.

major comments (3)
  1. [3.2, Eq. (5)] Section 3.2, Eq. (5): the conformal score S_i is defined through the annotation function A, and the paper explicitly conditions the guarantee on A being correct on calibration data, but it provides no evidence for this assumption. No human agreement study, no error analysis, and no annotation prompt are reported. If A mislabels non-factual claims as factual on calibration data, S_i underestimates the threshold needed for true factuality, qhat is too low, and Eq. (3) fails for the property users care about. Please add a validation of A (e.g., agreement with human annotations on a subset, or use of existing human labels such as FActScore's), report the annotation prompt, and quantify annotator error rates.
  2. [4.1] Section 4.1, 'empirical factuality': the text says the metric is 'calculated as the average factuality using the ground-truth labels from the test data,' but it does not state who produced those labels. If the labels come from the same GPT-4o annotation function A used in calibration, the empirical factuality results cannot detect annotator bias and the comparison with Conformal-LLM is partially circular. Please clarify the label source and, if automatic, validate against independent human labels or use dataset-provided labels.
  3. [3.1] Section 3.1, final paragraph: after thresholding, the retained claims are fed to an LLM and merged into a coherent response, but Eq. (3) is stated for y_test(x_test; qhat), the set of filtered sub-claims, not for the merged response. The merging step can introduce new claims, drop claims, or paraphrase them, so the final answer shown to the user is outside the statistical guarantee. The paper should either explicitly restrict all claims, including those in the abstract and conclusions, to the sub-claim level, or extend the analysis to the merge step.
minor comments (5)
  1. [Figures 2 and 3] Figures 2 and 3 and the 'up to 60%' claim in Section 5 come from single evaluation runs with no error bars, standard deviations, or repeated-seed analysis; please add uncertainty quantification to these empirical comparisons.
  2. [Abstract] The abstract's phrase 'without the need for ground truth answers' is imprecise because the calibration stage uses ground-truth answers; the intended meaning is 'without requiring ground-truth answers at inference time.' Please rephrase.
  3. [3.2] Section 3.2 invokes Theorem 4.1 of Mohri and Hashimoto [22] without restating the exchangeability and i.i.d. conditions; please state the conditions explicitly so the reader can verify the invocation.
  4. [3.3] Section 3.3 says each group is calibrated independently and hence Eq. (3) holds for each group, but the paper does not report the per-group calibration set sizes. Small groups would make the group-conditional guarantee unstable; please report group sizes and discuss minimum data requirements.
  5. [3.1, Algorithm 1] Algorithm 1 defines r_k = max({s_kj} union {0}), but cosine similarity can be negative; the intent is clear, yet it should be stated explicitly that negative relevance scores are clipped to zero.

Circularity Check

1 steps flagged · score 4.0 of 10

Empirical factuality validation is self-referential: test labels appear to be produced by the same GPT-4o annotation function used to calibrate the conformal threshold, so the reported coverage measures agreement with the judge, not independent entailment.

  1. self definitional [Section 3.1 (Automatic Calibration Set Annotation); Section 3.2, eq. (5); Section 4.1 (Results); Experimental Setup]
    "In addition, we use a GPT-4o model for annotation, sub-claim decomposition, and sub-claim merging as described in section 3.1. ... For empirical factuality, calculated as the average factuality using the ground-truth labels from the test data, we find that both Conformal-RAG and Conformal-LLM maintain a level at or above the target, as expected from the guarantee in eq. (3)."

    The conformal score S_i in eq. (5) is defined through the annotation function A, and the calibrated threshold qhat is chosen so that retained claims are A-factual on calibration data. The paper never describes any independent source of sub-claim factuality labels for the test data; the only annotation pipeline presented is the GPT-4o function A from Section 3.1. Therefore the 'ground-truth labels' used to compute empirical factuality in Section 4.1 are evidently A's outputs. Reporting that empirical factuality stays at or above 1-alpha then follows from applying conformal prediction to the same labeling function used for calibration; it is not evidence about genuine entailment by y*.

full rationale

The conformal derivation itself is not circular: qhat is the standard CP quantile of the scores in eq. (5), and the coverage statement follows from exchangeability and the nesting property of F_q, exactly as in Mohri and Hashimoto's Theorem 4.1. The paper explicitly conditions eq. (3) on correctness of A, so the theorem is conditional rather than self-referential. The retention comparison against Conformal-LLM is an empirical result that does not reduce to a fit, and no load-bearing self-citation is present. The circularity that remains is in the evaluation: the only annotation mechanism described is the GPT-4o function A, and test 'ground-truth labels' are never described as coming from a different judge. If they come from A, the empirical factuality curves simply recycle the calibration labels through CP's coverage guarantee. That makes the headline 'factuality' a statement about agreement with the judge used to fit the threshold, not about independent ground-truth entailment. No human agreement or error analysis is supplied to break the loop. On the 0-10 scale this warrants a 4 rather than a higher score because the central statistical machinery is theorem-driven and the retention gain is independently evaluable; only the factuality validation is self-referential.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No numeric free parameters are fit; the conformal threshold is a calibrated quantile. The main axioms are exchangeability and correctness of the LLM annotator. No new entities are introduced.

assumptions (5)
  • standard math Calibration and test data are exchangeable (i.i.d.) from the same distribution.
    Required for the CP coverage guarantee in eq. (2), eq. (3), and the per-group guarantee eq. (8). Stated in Section 2 and implicitly assumed in Section 3.2.
  • domain assumption The LLM annotation function A correctly labels whether a sub-claim is entailed by the ground-truth answer.
    Section 3.2: 'Under the assumption that the annotation function is correct on the calibration data.' No human agreement or error analysis is provided; this is the main load-bearing premise.
  • domain assumption The ground-truth entailment relation (y* => y) is a well-defined and correctly applied criterion.
    Used to define factuality in eq. (3) and in the annotation prompt. Entailment is assessed by a GPT-4o prompt, which is not formally defined.
  • domain assumption For each calibration example, retrieved documents D and ground-truth answer y* are available.
    Algorithm 1 and the annotation function in Section 3.1 require D and y*. At inference only D is available, so the calibration distribution must include these inputs.
  • domain assumption Group labels g(x) are available at inference and calibration data within each group is exchangeable.
    Mondrian CP in Section 3.3 calibrates a separate threshold per group. The paper notes group labels are available during inference in Section 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality." pith.science (2026). https://pith.science/paper/PT7L4HT3

@misc{pith2026250620978,
  author       = {Pith},
  title        = {Pith review of: Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PT7L4HT3}},
  note         = {Machine review of arXiv:2506.20978}
}
read the original abstract

Existing research on Retrieval-Augmented Generation (RAG) primarily focuses on improving overall question-answering accuracy, often overlooking the quality of sub-claims within generated responses. Recent methods that attempt to improve RAG trustworthiness, such as through auto-evaluation metrics, lack probabilistic guarantees or require ground truth answers. To address these limitations, we propose Conformal-RAG, a novel framework inspired by recent applications of conformal prediction (CP) on large language models (LLMs). Conformal-RAG leverages CP and internal information from the RAG mechanism to offer statistical guarantees on response quality. It ensures group-conditional coverage spanning multiple sub-domains without requiring manual labelling of conformal sets, making it suitable for complex RAG applications. Compared to existing RAG auto-evaluation methods, Conformal-RAG offers statistical guarantees on the quality of refined sub-claims, ensuring response reliability without the need for ground truth answers. Additionally, our experiments demonstrate that by leveraging information from the RAG system, Conformal-RAG retains up to 60\% more high-quality sub-claims from the response compared to direct applications of CP to LLMs, while maintaining the same reliability guarantee.

Figures

Figures reproduced from arXiv: 2506.20978 by the authors.

Figure 1
Figure 1. Conformal-RAG filters RAG’s responses based on a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Sub-claim removal rates (top) and empirical factuality levels (bottom) for target factuality levels [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Empirical factuality by group for Conformal-LLM [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 18 canonical work pages

  1. [1]

    Garima Agrawal, Tharindu Kumarage, Zeyad Alghamdi, and Huan Liu. 2024. Mindful-RAG: A Study of Points of Failure in Retrieval Augmented Generation. In2024 2nd International Conference on Foundation and Large Language Models. 607–611. doi:10.1109/FLLM63129.2024.10852457

    arXiv 2024

  2. [2]

    Anastasios N Angelopoulos and Stephen Bates. 2021. A gentle introduc- tion to conformal prediction and distribution-free uncertainty quantification. arXiv:2107.07511(2021)

    arXiv 2021

  3. [3]

    John Cherian, Isaac Gibbs, and Emmanuel Candes. 2024. Large language model validity via enhanced conformal prediction methods. InAdvances in Neural Information Processing Systems, Vol. 37

    work page 2024

  4. [4]

    Cresswell, Bhargava Kumar, Yi Sui, and Mouloud Belbahri

    Jesse C. Cresswell, Bhargava Kumar, Yi Sui, and Mouloud Belbahri. 2025. Con- formal Prediction Sets Can Cause Disparate Impact. InThe Thirteenth Interna- tional Conference on Learning Representations. https://openreview.net/forum? id=fZK6AQXlUU

    work page 2025

  5. [5]

    Cresswell, Yi Sui, Bhargava Kumar, and Noël Vouitsis

    Jesse C. Cresswell, Yi Sui, Bhargava Kumar, and Noël Vouitsis. 2024. Confor- mal prediction sets improve human decision making. InProceedings of the 41th International Conference on Machine Learning

    work page 2024

  6. [6]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The Faiss library.arXiv:2401.08281(2024)

    arXiv 2024

  7. [7]

    Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. RAGAs: Automated Evaluation of Retrieval Augmented Generation. InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, Nikolaos Aletras and Orphee De Clercq (Eds.). 150–158

    work page 2024

  8. [8]

    Rina Foygel Barber, Emmanuel J Candès, Aaditya Ramdas, and Ryan J Tibshirani

    work page

Show all 37 references
  1. [9]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey.arXiv:2312.10997(2023)

    2023 arXiv

  2. [10]

    Isaac Gibbs, John J Cherian, and Emmanuel J Candès. 2023. Conformal prediction with conditional guarantees.arXiv:2305.12616(2023)

    2023 arXiv

  3. [11]

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al . 2024. A Survey on LLM-as-a-Judge.arXiv:2411.15594(2024)

    2024 arXiv

  4. [12]

    Kuchibhotla, and Aaditya Ramdas

    Chirag Gupta, Arun K. Kuchibhotla, and Aaditya Ramdas. 2022. Nested conformal prediction and quantile out-of-bag ensemble methods.Pattern Recognition127 (July 2022), 108496. doi:10.1016/j.patcog.2021.108496

    2022

  5. [13]

    Jie Huang and Kevin Chen-Chuan Chang. 2023. Towards Reasoning in Large Language Models: A Survey. InFindings of the Association for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toron...

    2023 doi

  6. [14]

    Minbyul Jeong, Hyeon Hwang, Chanwoong Yoon, Taewhoo Lee, and Jaewoo Kang. 2024. OLAPH: Improving Factuality in Biomedical Long-form Question Answering.arXiv:2405.12701(2024)

    2024 arXiv

  7. [15]

    Mintong Kang, Nezihe Merve Gürel, Ning Yu, Dawn Song, and Bo Li. 2024. C- RAG: Certified Generation Risks for Retrieval-Augmented Language Models. In Proceedings of the 41st International Conference on Machine Learning, Vol. 235. 22963–23000

    2024

  8. [16]

    Jing Lei and Larry Wasserman. 2013. Distribution-free Prediction Bands for Non- parametric Regression.Journal of the Royal Statistical Society Series B: Statistical Methodology76, 1 (07 2013), 71–96. doi:10.1111/rssb.12021

    2013 doi

  9. [17]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. InAdvances in N...

    2020

  10. [18]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. InProceedings of the 61st Annual Meeting of the Association for Comput...

    2023 doi

  11. [19]

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On Faithfulness and Factuality in Abstractive Summarization. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1906–1919. d...

    2020 doi

  12. [20]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine- grained Atomic Evaluation of Factual Precision in Long Form Text Generation. InProceedings of the 2023 Conference on Empiric...

    2023 doi

  13. [21]

    Seyed Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2025. GSM-Symbolic: Understanding the Limi- tations of Mathematical Reasoning in Large Language Models. InThe Thirteenth International Conference on Learning Representatio...

    2025

  14. [22]

    Christopher Mohri and Tatsunori Hashimoto. 2024. Language Models with Con- formal Factuality Guarantees. InProceedings of the 41st International Conference on Machine Learning, Vol. 235. 36029–36047. https://proceedings.mlr.press/v235/ mohri24a.html

    2024

  15. [23]

    Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, KaShun Shum, Randy Zhong, Juntong Song, and Tong Zhang. 2024. RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computati...

    2024 doi

  16. [24]

    Jaakkola, and Regina Barzilay

    Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S. Jaakkola, and Regina Barzilay. 2024. Conformal Language Modeling. InThe Twelfth International Conference on Learning Representations. https://openreview. net/forum?id=pzUhfQ74c5

    2024

  17. [25]

    Yaniv Romano, Rina Foygel Barber, Chiara Sabatti, and Emmanuel Candès. 2020. With Malice Toward None: Assessing Uncertainty via Equalized Coverage.Har- vard Data Science Review2, 2 (2020)

    2020

  18. [26]

    Dongyu Ru, Lin Qiu, Xiangkun Hu, Tianhang Zhang, Peng Shi, Shuaichen Chang, Cheng Jiayang, Cunxiang Wang, Shichao Sun, Huanyu Li, Zizhao Zhang, Binjie Wang, Jiarong Jiang, Tong He, Zhiguo Wang, Pengfei Liu, Yue Zhang, and Zheng Zhang. 2024. RAGChecker: A Fine-grained Framework...

    2024

  19. [27]

    Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. 2024. ARES: An Automated Evaluation Framework for Retrieval-Augmented Genera- tion Systems. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics:...

    2024 doi

  20. [28]

    Maojia Song, Shang Hong Sim, Rishabh Bhardwaj, Hai Leong Chieu, Navonil Majumder, and Soujanya Poria. 2025. Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse. In The Thirteenth International Conference on Learning Repre...

    2025

  21. [29]

    2005.Algorithmic Learning in a Random World

    Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. 2005.Algorithmic Learning in a Random World. Springer

    2005

  22. [30]

    Vladimir Vovk, David Lindsay, Ilia Nouretdinov, and Alex Gammerman. 2003. Mondrian confidence machine.Technical Report(2003)

    2003

  23. [31]

    Di Wu, Jia-Chen Gu, Fan Yin, Nanyun Peng, and Kai-Wei Chang. 2024. Synchro- nous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 9390–9406. doi:10.18653/v1/2024.em...

    2024 doi

  24. [32]

    Kevin Wu, Eric Wu, Ally Cassasola, Angela Zhang, Kevin Wei, Teresa Nguyen, Sith Riantawan, Patricia Shi Riantawan, Daniel E Ho, and James Zou. 2024. How well do LLMs cite relevant medical references? An evaluation framework and analyses.arXiv:2402.02008(2024)

    2024 arXiv

  25. [33]

    Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. InThe Twelfth International Conference on Learning Representations. https://openreview.net/for...

    2024

  26. [34]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language ...

    2018 doi

  27. [35]

    Hao Yu, Aoran Gan, Kai Zhang, Shiwei Tong, Qi Liu, and Zhaofeng Liu. 2025. Evaluation of Retrieval-Augmented Generation: A Survey. InBig Data. Springer Nature Singapore, 102–120

    2025

  28. [36]

    Yujia Zhou, Yan Liu, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Zheng Liu, Chaozhuo Li, Zhicheng Dou, Tsung-Yi Ho, and Philip S Yu. 2024. Trustworthiness in retrieval- augmented generation systems: A survey.arXiv preprint arXiv:2409.10102(2024)

    2024 arXiv

  29. [2020]

    doi:10.1093/imaiai/iaaa017

    The limits of distribution-free conditional predictive inference.Information and Inference: A Journal of the IMA10, 2 (08 2020). doi:10.1093/imaiai/iaaa017

    2020 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.