REVIEW 9 cited by
Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
A major barrier towards the practical deployment of large language models (LLMs) is their lack of reliability. Three situations where this is particularly apparent are correctness, hallucinations when given unanswerable questions, and safety. In all three cases, models should ideally abstain from responding, much like humans, whose ability to understand uncertainty makes us refrain from answering questions we don't know. Inspired by analogous approaches in classification, this study explores the feasibility and efficacy of abstaining while uncertain in the context of LLMs within the domain of question-answering. We investigate two kinds of uncertainties, statistical uncertainty metrics and a distinct verbalized measure, termed as In-Dialogue Uncertainty (InDU). Using these uncertainty measures combined with models with and without Reinforcement Learning with Human Feedback (RLHF), we show that in all three situations, abstention based on the right kind of uncertainty measure can boost the reliability of LLMs. By sacrificing only a few highly uncertain samples we can improve correctness by 2% to 8%, avoid 50% hallucinations via correctly identifying unanswerable questions and increase safety by 70% up to 99% with almost no additional computational overhead.
Forward citations
Cited by 9 Pith papers
-
Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety
In open-ended medical conversations with missing information, LLM judges are more lenient than clinicians and a model's same-provider judge can skew apparent safety rankings.
-
Verbalizing LLM's Higher-order Uncertainty via Imprecise Probabilities
Prompting LLMs to report imprecise probability intervals (lower/upper confidence) instead of a single point value yields higher-order uncertainty scores that track prediction error and question ambiguity more coherent...
-
Energy-Based Transformers are Scalable Learners and Thinkers
Energy-Based Transformers learn to predict by gradient-descent minimization of a learned energy function, and the paper reports faster pretraining scaling and inference-time thinking gains over Transformer++ and Diffu...
-
AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
Reasoning fine-tuning makes LLMs more accurate on answerable problems but worse at abstaining on unanswerable ones, across a new 20-dataset benchmark.
-
MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM
MIRAGE is a benchmark that separates reasoning hallucinations from perception errors in multimodal LLMs, and Logos is a curriculum reinforcement fine-tuning method that reduces logical hallucinations.
-
Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selection
ZEUS selects chain-of-thought demonstrations by measuring answer uncertainty under perturbations, outperforming prior zero-shot prompting methods on four reasoning benchmarks.
-
Knowledge Boundary of Large Language Models: A Survey
A survey that formalizes the knowledge boundary of LLMs into a four-type taxonomy and reviews detection and mitigation methods.
-
A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions
A review that organizes LLM uncertainty quantification into token-level, self-verbalized, semantic-similarity, and mechanistic interpretability categories.
-
Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.
Discussion (0). Continue with ORCID to comment.