Verbal confidence in LLMs tracks future commit/abstain decisions more than answer correctness, while log-probabilities track correctness.
hub
Manning, and Chelsea Finn
27 Pith papers cite this work. Polarity classification is still indexing.
27
Pith papers citing it
abstract
A trustworthy real-world prediction system should produce well-calibrated confidence scores; that is, its confidence in an answer should be indicative of the likelihood that the answer is correct, enabling deferral to an expert in cases of low-confidence predictions. Recent studies have shown that unsupervised pre-training produces large language models (LMs) whose conditional probabilities are remarkably well-calibrated. However, the most widely-used LMs are fine-tuned with reinforcement learning from human feedback (RLHF-LMs), and some studies have suggested that RLHF-LMs produce conditional probabilities that are very poorly calibrated. In light of this perceived weakness, we conduct a broad evaluation of methods for extracting confidence scores from RLHF-LMs. For RLHF-LMs such as ChatGPT, GPT-4, and Claude, we find that verbalized confidences emitted as output tokens are typically better-calibrated than the model's conditional probabilities on the TriviaQA, SciQ, and TruthfulQA benchmarks, often reducing the expected calibration error by a relative 50%.
hub tools
citation-role summary
method 1
citation-polarity summary
roles
method 1polarities
use method 1representative citing papers
LLM rerankers can internally predict ranking quality via self-consistency of sampled outputs, matching SOTA external QPP while direct confidence is overconfident; supervised token-efficient methods improve calibration.
Inducing artificial uncertainty on trivial tasks allows training probes that achieve higher calibration on hard data than standard approaches while retaining performance on easy data.
UsefulBench is a new benchmark dataset that separates relevance from usefulness in information retrieval, revealing that similarity-based systems and current LLMs fall short on decision-useful content.
UQ4CT integrates functional-level uncertainty calibration into mixture-of-experts LoRA fine-tuning via a dedicated loss, cutting expected calibration error by over 25% on multiple-choice and generative QA tasks.
A large-scale multilingual evaluation of LLM uncertainty estimation methods across 22 languages and 9 models finds that English reasoning closes the UE gap for low-resource languages and that optimal UE method choice depends on model scale.
A composite loss with Brier calibration, anchor regularization, contrastive alignment from 2x2 perturbations, and KL stabilization reduces calibration error by over 60% in medical VQA while preserving accuracy.
ExtractConf fuses Hunter-Mapper disagreement with LLM uncertainty, OCR, image quality and layout into a classifier that reaches 0.928 ROC AUC on DocILE invoices and 0.858 on CORD receipts, cutting selective prediction risk 70% versus logprob baselines.
NBQ is a plug-and-play framework for adaptive question selection in conversations to produce structured user profiles, with QuickMatch enabling scalable reciprocal matching through approximate vector search.
BioConCal raises AUROC for selecting correct biomedical NER candidates from a multi-LLM panel from 0.753 to 0.910 and selects over four times as many candidates at 0.95 precision compared to raw agreement.
Introduces functional equivalence methods and functional entropy to predict functional correctness of LLM-generated code via uncertainty quantification, outperforming NLI-based baselines in most tested settings.
LLMs show inconsistent belief updates from evidence, with learned heuristics sometimes beating exact Bayesian computation due to misspecified world models.
Average token log-probability provides a zero-shot confidence signal for small LLMs that matches supervised baselines in-distribution and outperforms them out-of-distribution, with a new retrieval-conditional variant improving further at lower latency.
A hybrid confidence framework for LLM-based short answer grading combines model signals with aleatoric uncertainty from semantic clustering of responses and improves selective grading reliability over single-source methods.
LLMs implement a second-order confidence architecture where the PANL activation encodes both error likelihood and the ability to correct it, beyond verbal confidence or log-probabilities.
Seven 3-9B instruction-tuned LLMs produce verbal confidence that saturates at high values and fails psychometric validity criteria for Type-2 discrimination under minimal elicitation.
SELFDOUBT introduces the Hedge-to-Verify Ratio from reasoning traces as a single-pass uncertainty signal, with no-hedge traces correct 96% of the time and outperforming semantic entropy at 10x lower cost.
Language models deploy multidimensional internal confidence representations and threshold-based policies to control abstention behavior, with causal support from activation steering experiments.
Mechanistic experiments on Gemma 3 27B, Qwen 2.5 7B and Magistral Small 24B show verbal confidence is cached at post-answer positions from answer tokens and captures richer answer-quality information beyond token log-probabilities.
LLMs systematically misalign with human student difficulty perceptions, converging to a machine consensus rather than simulating human cognitive limits across medical and math domains.
C3RL is a new RL algorithm combining correctness, calibration, and reference accuracy rewards to improve LLM confidence calibration, enabling CAS to outperform majority voting with up to 12.33x lower inference cost.
No benchmark exists for doctrinal legal reasoning in LLMs, leaving the EU AI Act's accuracy mandate for judicial AI without an operational test.
A simple per-agent, per-confidence-band online correction with symmetric exponentially weighted moving averages fixes the calibration gap left by frozen design-time methods and turns raw verbalized confidence from sub-random into a useful multi-agent selection signal.
Learning a margin-based confidence ranker for LLM judges improves agreement-target success in cascaded selective evaluation compared to heuristic confidence scores.
citing papers explorer
-
Reported Confidence in LLMs Tracks Commitment More Than Correctness
Verbal confidence in LLMs tracks future commit/abstain decisions more than answer correctness, while log-probabilities track correctness.
-
Can LLM Rerankers Predict Their Own Ranking Performance?
LLM rerankers can internally predict ranking quality via self-consistency of sampled outputs, matching SOTA external QPP while direct confidence is overconfident; supervised token-efficient methods improve calibration.
-
Inducing Artificial Uncertainty in Language Models
Inducing artificial uncertainty on trivial tasks allows training probes that achieve higher calibration on hard data than standard approaches while retaining performance on easy data.
-
UsefulBench: Towards Decision-Useful Information as a Target for Information Retrieval
UsefulBench is a new benchmark dataset that separates relevance from usefulness in information retrieval, revealing that similarity-based systems and current LLMs fall short on decision-useful content.
-
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs
UQ4CT integrates functional-level uncertainty calibration into mixture-of-experts LoRA fine-tuning via a dedicated loss, cutting expected calibration error by over 25% on multiple-choice and generative QA tasks.
-
Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs
A large-scale multilingual evaluation of LLM uncertainty estimation methods across 22 languages and 9 models finds that English reasoning closes the UE gap for low-resource languages and that optimal UE method choice depends on model scale.
-
Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA
A composite loss with Brier calibration, anchor regularization, contrastive alignment from 2x2 perturbations, and KL stabilization reduces calibration error by over 60% in medical VQA while preserving accuracy.
-
Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extraction
ExtractConf fuses Hunter-Mapper disagreement with LLM uncertainty, OCR, image quality and layout into a classifier that reaches 0.928 ROC AUC on DocILE invoices and 0.858 on CORD receipts, cutting selective prediction risk 70% versus logprob baselines.
-
NBQ: Next-Best-Question for Dynamic Profiling
NBQ is a plug-and-play framework for adaptive question selection in conversations to produce structured user profiles, with QuickMatch enabling scalable reciprocal matching through approximate vector search.
-
Beyond Agreement: Scoring Panel-Surfaced Biomedical Entity Candidates for Curator Triage
BioConCal raises AUROC for selecting correct biomedical NER candidates from a multi-LLM panel from 0.753 to 0.910 and selects over four times as many candidates at 0.95 precision compared to raw agreement.
-
Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification
Introduces functional equivalence methods and functional entropy to predict functional correctness of LLM-generated code via uncertainty quantification, outperforming NLI-based baselines in most tested settings.
-
LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs
LLMs show inconsistent belief updates from evidence, with learned heuristics sometimes beating exact Bayesian computation due to misspecified world models.
-
Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren't Worth Training
Average token log-probability provides a zero-shot confidence signal for small LLMs that matches supervised baselines in-distribution and outperforms them out-of-distribution, with a new retrieval-conditional variant improving further at lower latency.
-
Confidence Estimation in Automatic Short Answer Grading with LLMs
A hybrid confidence framework for LLM-based short answer grading combines model signals with aleatoric uncertainty from semantic clustering of responses and improves selective grading reliability over single-source methods.
-
How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals
LLMs implement a second-order confidence architecture where the PANL activation encodes both error likelihood and the ability to correct it, beyond verbal confidence or log-probabilities.
-
Verbal Confidence Saturation in 3-9B Open-Weight Instruction-Tuned LLMs: A Pre-Registered Psychometric Validity Screen
Seven 3-9B instruction-tuned LLMs produce verbal confidence that saturates at high values and fails psychometric validity criteria for Type-2 discrimination under minimal elicitation.
-
SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio
SELFDOUBT introduces the Hedge-to-Verify Ratio from reasoning traces as a single-pass uncertainty signal, with no-hedge traces correct 96% of the time and outperforming semantic entropy at 10x lower cost.
-
Causal Evidence that Language Models use Confidence to Drive Behavior
Language models deploy multidimensional internal confidence representations and threshold-based policies to control abstention behavior, with causal support from activation steering experiments.
-
How do LLMs Compute Verbal Confidence
Mechanistic experiments on Gemma 3 27B, Qwen 2.5 7B and Magistral Small 24B show verbal confidence is cached at post-answer positions from answer tokens and captures richer answer-quality information beyond token log-probabilities.
-
Can LLMs Estimate Student Struggles? Human-AI Difficulty Alignment with Proficiency Simulation for Item Difficulty Prediction
LLMs systematically misalign with human student difficulty perceptions, converging to a machine consensus rather than simulating human cognitive limits across medical and math domains.
-
Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling
C3RL is a new RL algorithm combining correctness, calibration, and reference accuracy rewards to improve LLM confidence calibration, enabling CAS to outperform majority voting with up to 12.33x lower inference cost.
-
The Measurement Gap in the Automation of EU Law: Benchmarking Doctrinal Legal Reasoning under the EU AI Act
No benchmark exists for doctrinal legal reasoning in LLMs, leaving the EU AI Act's accuracy mandate for judicial AI without an operational test.
-
MARGIN: Runtime Confidence Calibration for Multi-Agent Foundation Model Coordination
A simple per-agent, per-confidence-band online correction with symmetric exponentially weighted moving averages fixes the calibration gap left by frozen design-time methods and turns raw verbalized confidence from sub-random into a useful multi-agent selection signal.
-
Margin-Adaptive Confidence Ranking for Reliable LLM Judgement
Learning a margin-based confidence ranker for LLM judges improves agreement-target success in cascaded selective evaluation compared to heuristic confidence scores.
-
Act or Escalate? Evaluating Escalation Behavior in Automation with Language Models
Language models display model-specific escalation thresholds in uncertain decisions that are not explained by scale or architecture, and supervised fine-tuning on explicit uncertainty reasoning produces robust, generalizable policies.
-
Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA
A multi-strategy interrogation method with auxiliary expert assessment reduces expected calibration error by 40% on average across three medical VQA datasets for MLLMs.
- VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Models