Computation and Language · arXiv:2608.09080 v1 · submitted 10 Aug 2026 · 25 references
When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information
The paper says
“Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks. However, their reliability under uncertainty remains poorly understood which raises critical concerns for deployment in high-stakes clinical settings.”
more of the abstract
In such environments, incorrect predictions are inherently risky, but confident incorrect predictions can be particularly harmful as they may mislead clinical decision-making. In this paper, we conduct a systematic behavioral analysis of LLMs under clinical information uncertainty. We propose an evaluation framework based on the MedMCQA dataset consisting of two complementary uncertainty settings. First, we introduce linguistic uncertainty cues through prompt modifications to simulate ambiguous clinical contexts. Second, we construct an answer removal setting, wherein the correct option is deliberately excluded mandating the model to recognize insufficient information and abstain. We analyze both model accuracy and confidence behavior using multiple calibration metrics including calibration gap, Expected Calibration Error (ECE), and Unsafe Confident Error Rate (UCER) across 500 medical questions. Our results reveal a consistent failure mode, i.e., although accuracy degrades under increasing uncertainty, model confidence remains misaligned with accuracy. This leads to a substantial increase in unsafe confident errors, indicating that model confidence remains largely insensitive to clinically meaningful information loss. Furthermore, we observe significant variation across models in their ability to abstain when the correct answer is unavailable, with some models persistently producing high confidence hallucinated answers. These findings expose critical limitations in the epistemic reliability of current LLMs and highlight the need for uncertainty aware evaluation methods prior to their deployment in clinical workflows.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 11 Aug 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. Worth reading for the setup, but the 'confidence is insensitive' claim isn't supported by the experiments as reported. This line is wrong
5 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 25 references say.
The paper says, at Section III-B, Eq. (3), Table I
The review did not quote this passage.
Pith says: Objection 1 of 5 · major
Setting A keeps the answer options A_i and ground-truth labels y_i unchanged across C0–C3; only the prompt wording is transformed. A calibrated model whose posterior over the same options does not change has no reason to lower its confidence. Flat self-reported confidence under C0–C3 is therefore expected and cannot, by itself, support the claim that confidence is insensitive to information loss.
more of objection 1
The authors should provide a direct measure of whether confidence should move under their framing, or compare calibration metrics such as ECE/UCER against a C0 baseline rather than interpreting stability in raw confidence as miscalibration.
The paper says, at Section III-D3 and Section V-C
The review did not quote this passage.
Pith says: Objection 2 of 5 · major
The methods section promises paired Wilcoxon signed-rank tests for self-reported confidence comparisons, with Holm–Bonferroni correction, but Section V-C reports only McNemar tests for model accuracy. No p-values, effect sizes, or corrected significance statements for confidence are presented anywhere. Consequently, the claim in Section V-A that 'uncertainty framing does not lead to any meaningful reduction in confidence' is not backed by the stated inferential analysis.
more of objection 2
Please report the confidence comparisons (at minimum C0 vs. C3 for each model) with appropriate multiple-testing correction.
The paper says, at Section III-C, Eq. (4), Tables III–IV
The review did not quote this passage.
Pith says: Objection 3 of 5 · major
In Setting B the correct answer is removed, but the paper reports only aggregate abstention, hallucination, and mean confidence in the removal condition. There is no paired per-question comparison with a condition in which the correct answer is present (e.g., C0).
more of objection 3
Thus the claim that confidence fails to decrease when information is removed is asserted rather than measured. The authors should provide per-question paired differences in confidence between the intact condition and the removal condition, and ideally model abstention conditional on the original question, to test sensitivity directly.
This review says the paper's mathematics is wrong. Pith did not check that against the paper's source.
The paper says, at Table IV, LLaMA-3 row
The review did not quote this passage.
Pith says: Objection 4 of 5 · major
Eq. (10) defines Hallucination = 1 − Abstention, so an abstention rate of 0.156 for LLaMA-3 should yield a hallucination rate of 0.844, not 0.798 as reported. Additionally, the Unsafe rate of 0.828 exceeds the hallucination rate of 0.798, which is impossible under the definition in Section V-B2 that unsafe predictions are a subset of hallucinated (incorrect) predictions.
more of objection 4
The MeanConf for LLaMA also differs between Table III (0.970) and Table IV (0.963). These inconsistencies must be corrected and the affected safety conclusions re-verified.
The paper says, at Section V-B3, Table III
The review did not quote this passage.
Pith says: Objection 5 of 5 · major
The paper itself shows that self-reported and logit-based confidence diverge dramatically (e.g., LLaMA-3: 0.970 vs. 0.280; Mistral-7B: 0.998 vs. 0.657) and concludes that self-reported confidence 'cannot be reliably interpreted as a measure of model uncertainty.' Yet the abstract and Section V use self-reported confidence as evidence about 'model confidence.' The central claim should either be explicitly restricted to verbalized confidence, or the paper should provide evidence that the self-reported score is a valid uncertainty measure in this setting.
6 minor objections
The paper says, at Table II
Pith says: minor objection 1 of 6
Several rows contain values that are run together (e.g., '10.8020.904' and '30.8360.798'), making the table difficult to read; add explicit column separators.
The paper says, at Fig. 5 caption
Pith says: minor objection 2 of 6
The caption says 'across Uncertainty Framing Conditions (C0–C3)' but the figure reports the answer-removal experiment; the caption should be corrected.
show 4 more
The paper says, at Section III-D2, Eq. (8)
Pith says: minor objection 3 of 6
The UCER threshold τ is defined in Eq. (8) but its value τ=0.8 is introduced only later in Section V-B2; state and justify the threshold in the metrics section.
The paper says, at Section III-C, Eq. (6)
Pith says: minor objection 4 of 6
The text describes log-probability confidence as 'the probability of each answer option,' but Eq. (6) computes a softmax over single answer tokens; clarify that this is a token-level proxy rather than a full response probability.
The paper says, at Section IV-B
Pith says: minor objection 5 of 6
The abstention option position is said to be randomized for each question, but no random seed is reported for this randomization, which weakens the stated reproducibility.
The paper says, at Section IV-C
Pith says: minor objection 6 of 6
Temperature 0 is reported for all models, but API-based models do not guarantee deterministic outputs; please add a caveat about possible nondeterminism in the proprietary models.
What would settle it
Recompute the two experiments on the same 500 MedMCQA questions and produce a paired sensitivity test: for each question, compare self-reported confidence and token-logit confidence (Eq. 6-7) between the C0 condition (correct answer present) and the Setting B condition (correct answer removed, abstention option added). Also compute token-logit confidence under C0-C3 in Setting A. Report mean paired differences and a Wilcoxon signed-rank test with Holm-Bonferroni correction. Additionally, validate c_i as an uncertainty measure by computing AUROC of confidence against correctness within each condition; if AUROC is near 0.5 or token-logit confidence drops under C0-C3 while c_i does not, the conclusion should be restricted to verbalized confidence. Finally, recompute Table IV with hallucination = 1 - abstention and verify unsafe <= hallucination.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 11 Aug 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 5 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The paper has a clean empirical core: five models, 500 MedMCQA questions, two uncertainty manipulations, and a sensible metric suite (ECE, UCER, abstention). The observation that self-reported confidence and token-logit confidence diverge wildly (LLaMA: 0.970 vs 0.280) is a useful reminder that verbalized confidence is not the same as internal probability. Those measurements are new and could be a solid base for a good paper.
But the central claim — that LLM confidence is largely insensitive to clinically meaningful information loss — is not actually tested by the experimental design. In Setting A, the choice set and clinical content are unchanged; you only add a generic warning phrase. A calibrated model need not lower its confidence when the task is objectively the same, so flat confidence across C0–C3 is neither surprising nor evidence of miscalibration. In Setting B, the correct answer is removed, but the paper never reports a paired comparison of confidence on the same questions with and without the correct option. It only gives absolute confidence in the degraded condition. So "insensitivity" is asserted, not measured. If the goal is to show models fail to reduce confidence when the answer disappears, you need the same questions in both conditions, not an aggregate across different sets.
The missing statistics are a real problem too. Section III-D3 promises Wilcoxon signed-rank tests for confidence comparisons, but the results section reports only McNemar tests on accuracy. No confidence p-values or effect sizes appear anywhere. That is a major omission for a paper whose thesis is about confidence. And Table IV has an internal inconsistency: LLaMA-3's abstention of 0.156 implies a hallucination rate of 0.844, but the table says 0.798; the unsafe rate of 0.828 also needs reconciliation with the hallucination rate. These are checkable numbers, and they should be fixed before anyone trusts the aggregate figures.
I also want to flag the reliance on self-reported confidence as the main measure. The paper's own Table III shows that this can be radically disconnected from logit-based probability. That doesn't invalidate the study, but it means the claim is really about verbalized confidence, not model uncertainty more broadly. The paper should either validate the self-reported scores or restrict the conclusion accordingly.
Who is this for? People building or evaluating medical QA systems, especially those interested in abstention and calibration. The framework is a useful starting point, and the empirical patterns are plausible, but the analysis needs substantial revision. I would send it to peer review rather than desk reject, because the core idea is timely and the measurements are genuinely new. The authors need to add the direct confidence comparisons, report the Wilcoxon results, fix Table IV, and soften or re-justify the central claim. A serious referee can push them to do that.
The strongest objection
Section V claims LLM confidence is largely insensitive to clinically meaningful information loss, but the experimental design never directly measures sensitivity. In Setting A (uncertainty framing), Eq. 3 transforms only the prompt wording; the option set and clinical content are unchanged, so a calibrated model need not change its posterior confidence. Flat self-reported confidence across C0-C3 is therefore not itself evidence of miscalibration. In Setting B, the correct answer is removed (Eq. 4), but the paper reports only absolute confidence in that condition; it does not pair each question with its C0 baseline, so 'insensitivity' is asserted rather than measured. The only confidence signal used in Setting A is the self-reported numeric score c_i from Section III-C, which the paper's own Table III shows can be radically disconnected from token-logit confidence (LLaMA-3: 0.970 vs 0.280; Mistral-7B: 0.998 vs 0.657). Thus the central claim may reduce to 'verbally elicited confidence scores are high and stable,' not 'model confidence fails to track information loss.' Separately, Table IV is internally inconsistent (LLaMA-3 abstention 0.156 implies hallucination 0.844, not 0.798, and unsafe 0.828 cannot exceed hallucination 0.798), so the numeric basis for the conclusion needs verification.
The reader's strongest claim is that confidence signals from current LLMs cannot be relied on as calibrated indicators of correctness in medical QA. The paper's evidence for this is mostly absolute high self-reported confidence; however, a confidence signal that is unvalidated and not compared within-question cannot support an 'insensitivity' claim. The log-prob discrepancy in Table III indicates c_i is at least partly prompt compliance rather than epistemic uncertainty. This concern is not a disagreement with field consensus; it is an internal evidence gap. The proposed test would either strengthen the paper (if log-prob also stays flat and AUROC is meaningfully above chance) or force a more limited claim about verbalized confidence. The reader's weakest assumption identified self-reported confidence validity, which matches my concern, but I extend it: even before validity, no paired comparison of information presence versus absence is reported. The numerical inconsistency in Table IV further underscores the need for corrected, released data and code. Since the concern is addressable by additional analysis, the conditional verdict remains appropriate; I would keep the verdict conditional and unchanged from the reader's CONDITIONAL.
The referee's report
If the central claim holds, the paper would provide a useful warning about the reliability of verbalized confidence in medical decision support. The study has several strengths: it uses a public benchmark, evaluates both proprietary and open-weight models, defines quantitative metrics (ECE, UCER), reports deterministic inference settings, and includes a limitations section. The qualitative pattern of overconfidence in open-weight models is visible in Table II. However, as detailed below, the evidence currently does not directly support the insensitivity claim because the experimental design lacks the necessary paired comparisons, the promised statistical tests for confidence are not reported, and one key table contains internally inconsistent numbers.
Pith's reading of the paper (a machine's summary, not the authors')
This paper tries to establish that large language models, despite strong accuracy on medical multiple-choice questions, do not lower their confidence when the clinical information they receive becomes uncertain or incomplete. Using 500 questions from the MedMCQA benchmark, the authors inject graded linguistic uncertainty cues into prompts and, in a second setting, remove the correct answer and replace it with an abstention option. Accuracy stays roughly flat or drops while reported confidence remains high, and the rate of wrong answers delivered with confidence above 0.8 rises sharply. The authors conclude that current LLM confidence scores cannot be trusted as indicators of answer correctness in clinical settings, and that prompt-level warnings do not repair this. A confidently wrong suggestion from a decision-support system could mislead a clinician, so the finding matters for any plan to deploy these models in medical workflows.
What carries the argument. The central object is a two-setting evaluation framework built on the MedMCQA benchmark. Setting A prepends graded linguistic uncertainty statements — C0 (none), C1, C2, C3 — to otherwise identical questions, leaving the answer set unchanged, so any behavioral shift can be attributed to the framing. Setting B removes the ground-truth option and replaces it with an abstention option, so the only safe behavior is selecting 'Insufficient information to determine the correct answer.' Confidence is collected two ways: a self-reported numeric score elicited from the model and a logit-based probability computed by softmax over the answer-token logits. The metrics — accuracy, calibration gap, ECE, UCER, abstention and hallucination rates — convert the setup into a test of whether confidence tracks information loss.
Core claim. The paper's central discovery is a systematic failure mode in LLM confidence behavior. When uncertainty cues are added to clinical questions, or when the correct answer is removed and an explicit 'insufficient information' option is offered, the models' self-reported confidence remains largely insensitive to the information loss. In the framing setting, mean confidence stays near or above 0.85 for most models even at the strongest cue level, with calibration gaps and UCER staying roughly constant; in the removal setting, the best abstention rate is only 38.2%, and hallucinated answers are frequently produced with confidence above 0.8. The paper also finds that open-weight models show a large split between self-reported confidence and logit-based probability, meaning their stated confidence overestimates what their own output distribution implies. The stated conclusion is that current LLMs lack reliable uncertainty awareness in medical QA, with the failure strongest in open-weight models but present across proprietary ones.
The premise it rests on. The load-bearing premise is that the confidence score a model writes when asked to 'provide a confidence score' reflects its internal uncertainty, and that the inserted linguistic cues are a valid proxy for clinically meaningful information loss; if either fails, the claim that confidence is insensitive does not follow.
If the paper is right (Pith's inferences, not the authors')
If the paper is right, a high confidence score from an LLM in medical QA carries little information about whether the answer is actually correct once information is incomplete.
Prompt-based uncertainty cues like C1–C3 are not a sufficient safeguard: they do not meaningfully reduce confidence or increase abstention.
Open-weight models such as LLaMA-3 and Mistral-7B would be particularly unsafe as clinical decision-support tools under current calibration, with near-maximal confidence on hallucinated answers.
Even the better-calibrated proprietary models abstain less than 40% of the time when the correct answer is absent, so closed-world benchmarks overestimate reliability.
Medical LLM evaluation should include abstention and unsafe-error metrics rather than accuracy alone if it is meant to reflect real clinical conditions.
One consequence the authors leave implicit is that the same accuracy–confidence decoupling should be expected in other high-stakes domains whose benchmarks assume a correct answer always exists, since the proposed cause is the training objective rather than medicine-specific content.
The large self-reported versus logit-based confidence gap suggests a concrete, testable mitigation that the paper does not propose: replace or rank by token-logit probabilities when the API exposes them, rather than trusting elicited scores.
Because the paper's uncertainty cues are inserted prompts, a stronger test would use naturally occurring ambiguous clinical cases; if confidence stays high there too, the insensitivity claim would be more robust to the concern that the cues are simply too weak.
A boundary condition worth probing is whether some cue strength or instruction (for example, explicitly instructing the model to abstain when unsure) eventually breaks the insensitivity; the paper only shows that the tested prompts do not.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks. However, their reliability under uncertainty remains poorly understood which raises critical concerns for deployment in high-stakes clinical settings. In such environments, incorrect predictions are inherently risky, but confident incorrect predictions can be particularly harmful as they may mislead clinical decision-making. In this paper, we conduct a systematic behavioral analysis of LLMs under clinical information uncertainty. We propose an evaluation framework based on the MedMCQA dataset consisting of two complementary uncertainty settings. First, we introduce linguistic uncertainty cues through prompt modifications to simulate ambiguous clinical contexts. Second, we construct an answer removal setting, wherein the correct option is deliberately excluded mandating the model to recognize insufficient information and abstain. We analyze both model accuracy and confidence behavior using multiple calibration metrics including calibration gap, Expected Calibration Error (ECE), and Unsafe Confident Error Rate (UCER) across 500 medical questions. Our results reveal a consistent failure mode, i.e., although accuracy degrades under increasing uncertainty, model confidence remains misaligned with accuracy. This leads to a substantial increase in unsafe confident errors, indicating that model confidence remains largely insensitive to clinically meaningful information loss. Furthermore, we observe significant variation across models in their ability to abstain when the correct answer is unavailable, with some models persistently producing high confidence hallucinated answers. These findings expose critical limitations in the epistemic reliability of current LLMs and highlight the need for uncertainty aware evaluation methods prior to their deployment in clinical workflows.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-23
The authors' main claim, as the classifier picked it out 0.86
“These findings expose critical limitations in the epistemic reliability of current LLMs and highlight the need for uncertainty aware evaluation methods prior to their deployment in clinical workflows.”
How the main claim is stated: asserted 0.99
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.20 · human participants 0.06 · controlled comparison 0.29 · randomized assignment 0.02 · states quantitative result 0.69 · compares with prior results 0.14 · mentions code or data availability 0.02
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks. | background 1.00 | 0.05 | 0.01 | asserted 1.00 |
| However, their reliability under uncertainty remains poorly understood which raises critical concerns for deployment in high-stakes clinical settings. | gap 0.92 | 0.05 | 0.04 | asserted 0.99 |
| In such environments, incorrect predictions are inherently risky, but confident incorrect predictions can be particularly harmful as they may mislead clinical decision-making. | background 0.98 | 0.04 | 0.03 | hedged possibility 0.87 |
| In this paper, we conduct a systematic behavioral analysis of LLMs under clinical information uncertainty. | contribution 0.97 | 0.94 | 0.02 | asserted 0.95 |
| We propose an evaluation framework based on the MedMCQA dataset consisting of two complementary uncertainty settings. | method 0.85 | 0.93 | 0.02 | asserted 1.00 |
| First, we introduce linguistic uncertainty cues through prompt modifications to simulate ambiguous clinical contexts. | method 0.99 | 0.94 | 0.02 | asserted 0.98 |
| Second, we construct an answer removal setting, wherein the correct option is deliberately excluded mandating the model to recognize insufficient information and abstain. | method 0.99 | 0.95 | 0.02 | asserted 0.98 |
| We analyze both model accuracy and confidence behavior using multiple calibration metrics including calibration gap, Expected Calibration Error (ECE), and Unsafe Confident Error Rate (UCER) across 500 medical questions. | method 0.99 | 0.96 | 0.05 | asserted 0.98 |
| Our results reveal a consistent failure mode, i.e., although accuracy degrades under increasing uncertainty, model confidence remains misaligned with accuracy. | result 1.00 | 0.96 | 0.56 | asserted 1.00 |
| This leads to a substantial increase in unsafe confident errors, indicating that model confidence remains largely insensitive to clinically meaningful information loss. | result 0.75 | 0.95 | 0.42 | asserted 0.62 |
| Furthermore, we observe significant variation across models in their ability to abstain when the correct answer is unavailable, with some models persistently producing high confidence hallucinated answers. | result 1.00 | 0.95 | 0.09 | asserted 1.00 |
| These findings expose critical limitations in the epistemic reliability of current LLMs and highlight the need for uncertainty aware evaluation methods prior to their deployment in clinical workflows. | interpretation 0.95 | 0.48 | 0.87 | asserted 0.99 |
What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.92
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 8,771 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (6)
Figures are the authors', reproduced from the paper.
References (25)
- [1] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language Models Are Few-shot Learners,”Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901, 2020 on Pith
- [2] M. M. Perera, A. Mahmood, K. E. Wijethilake, F. Islam, M. Taher- mazandarani, and Q. Z. Sheng, “A survey of the state of the art in con- versational question answering systems,”Knowledge and Information Systems, vol. 68, no. 1, Dec. 2026 on Pith
- [3] T. H. Kung, M. Cheatham, A. Medenilla, C. Sillos, L. De Leon, C. Elepa ˜no, M. Madriaga, R. Aggabao, G. Diaz-Candido, J. Maningo et al., “Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education using Large Language Models,”PLoS Digital Health, vol. 2, no. ... on Pith
- [4] A. Pal, L. K. Umapathi, and M. Sankarasubbu, “MedMCQA: A large- Scale Multi-Subject Multi-Choice Dataset for Medical Domain Question Answering,” inConference on Health, Inference, and Learning. PMLR, 2022, pp. 248–260 on Pith
- [5] K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohlet al., “Large Language Models Encode Clinical Knowledge,”Nature, vol. 620, no. 7972, pp. 172–180, 2023 on Pith
- [6] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y . Xu, E. Ishii, Y . J. Bang, A. Madotto, and P. Fung, “Survey of Hallucination in Natural Language Generation,”ACM Computing Surveys, vol. 55, no. 12, pp. 1–38, 2023 on Pith
- [7] S. Tripathi, M. T. Nafis, I. Hussain, and J. Gao, “The Confidence Paradox: Can LLM Know When It’s Wrong?” inProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computati... on Pith
- [8] M. Steyvers, H. Tejeda, A. Kumar, C. Belem, S. Karny, X. Hu, L. W. Mayer, and P. Smyth, “What Large Language Models Know and What People Think They Know,”Nature Machine Intelligence, vol. 7, no. 2, pp. 221–231, 2025 on Pith
- [9] C. Guo, G. Pleiss, Y . Sun, and K. Q. Weinberger, “On Calibration of Modern Neural Networks,” inProceedings of the 34th Interna- tional Conference on Machine Learning - Volume 70, ser. ICML’17. JMLR.org, 2017, p. 1321–1330 on Pith
- [10] S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. Hatfield-Dodds, N. DasSarma, E. Tran-Johnsonet al., “Language Models (mostly) Know What They Know,”arXiv preprint arXiv:2207.05221, 2022 on PithPith's reviewarXiv
- [11] C. Xu, B. Wen, B. Han, R. Wolfe, L. L. Wang, and B. Howe, “Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs,” inFindings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 25 655– 25 672 on Pith
- [12] R. Bentegeac, B. Le Guellec, G. Kuchcinski, P. Amouyel, and A. Ham- roun, “Token Probabilities to Mitigate Large Language Models Overcon- fidence in Answering Medical Questions: Quantitative Study,”Journal of Medical Internet Research, vol. 27, p. e64348, 2025 on Pith
- [13] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qinet al., “A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions,”ACM Transactions on Information Systems, vol. 43, no. 2, pp. 1–55, 2025 on Pith
- [14] K. Tian, E. Mitchell, A. Zhou, A. Sharma, R. Rafailov, H. Yao, C. Finn, and C. D. Manning, “Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback,” inProceedings of the 2023 Conference on Empirical M... on Pith
- [15] T. Groot and M. Valdenegro-Toro, “Overconfidence is Key: Verbal- ized Uncertainty Evaluation in Large Language and Vision-Language Models,” inProceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024), 2024, pp. 145–171 on Pith
- [16] X. Qiu and R. Miikkulainen, “Semantic Density: Uncertainty Quantifi- cation for Large Language Models Through Confidence Measurement in Semantic Space,”Advances in Neural Information Processing Systems, vol. 37, pp. 134 507–134 533, 2024 on Pith
- [17] G. P. Wightman, A. Delucia, and M. Dredze, “Strength in Numbers: Es- timating Confidence of Large Language Models by Prompt Agreement,” inProceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), 2023, pp. 326–362 on Pith
- [18] N. Madhusudhan, S. T. Madhusudhan, V . Yadav, and M. Hashemi, “Do LLMs Know When to Not Answer? Investigating Abstention Abilities of Large Language Models,” inProceedings of the 31st International Conference on Computational Linguistics, 2025, pp. 9329–9345 on Pith
- [19] Z. Yin, Q. Sun, Q. Guo, J. Wu, X. Qiu, and X.-J. Huang, “Do Large Language Models Know What They Don’t Know?” inFindings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 8653– 8665 on Pith
- [20] J. Geng, F. Cai, Y . Wang, H. Koeppl, P. Nakov, and I. Gurevych, “A Survey of Confidence Estimation and Calibration in Large Language Models,” inProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Te... on Pith
show 5 more
- [21] P. Chhikara, “Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models,”arXiv preprint arXiv:2502.11028, 2025 on PitharXiv
- [22] S. Machcha, S. Yerra, S. Gupta, A. Sahoo, S. Sultana, H. Yu, and Z. Yao, “Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty,”arXiv preprint arXiv:2601.12471, 2026 on PitharXiv
- [23] Q. McNemar, “Note on the Sampling Error of the Difference between Correlated Proportions or Percentages,”Psychometrika, vol. 12, no. 2, pp. 153–157, 1947 on Pith
- [24] F. Wilcoxon, “Individual Comparisons by Ranking Methods,”Biometrics Bulletin, vol. 1, no. 6, pp. 80–83, 1945 on Pith
- [25] S. Holm, “A Simple Sequentially Rejective Multiple Test Procedure,” Scandinavian Journal of Statistics, pp. 65–70, 1979 on Pith
Cited by 1 Pith paper
Reviewed papers on Pith that cite this one. The full citation record
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 2 sentences without a confident label
Cites as background 1 paper
-
Overconfidence refers to the systematic tendency of models to assign high confidence to incorrect predictions, i.e., when predicted confidence exceeds empirical accuracy [ref] .
2502.11028 · Confidence and Overconfidence in LLMs · 0.94
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment