REVIEW 5 cited by
On Hallucination and Predictive Uncertainty in Conditional Language Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Despite improvements in performances on different natural language generation tasks, deep neural models are prone to hallucinating facts that are incorrect or nonexistent. Different hypotheses are proposed and examined separately for different tasks, but no systematic explanations are available across these tasks. In this study, we draw connections between hallucinations and predictive uncertainty in conditional language generation. We investigate their relationship in both image captioning and data-to-text generation and propose a simple extension to beam search to reduce hallucination. Our analysis shows that higher predictive uncertainty corresponds to a higher chance of hallucination. Epistemic uncertainty is more indicative of hallucination than aleatoric or total uncertainties. It helps to achieve better results of trading performance in standard metric for less hallucination with the proposed beam search variant.
Forward citations
Cited by 5 Pith papers
-
HalluField: Detecting LLM Hallucinations via Field-Theoretic Modeling
HalluField flags LLM hallucinations using a hand-weighted temperature-perturbation of token-level 'free energy' (negative log-likelihood) and Shannon entropy.
-
Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?
Reasoning language models are systematically overconfident, deeper reasoning makes them more overconfident, and a two-stage introspective prompting method improves calibration for some models.
-
Estimating LLM Uncertainty with Evidence
LogTokU splits token uncertainty into aleatoric and epistemic components from the top-K logits, enabling single-pass hallucination detection and uncertainty-guided decoding.
-
keepitsimple at SemEval-2025 Task 3: LLM-Uncertainty based Approach for Multilingual Hallucination Span Detection
Hallucinated spans can be spotted by comparing many stochastic responses from the same model and flagging text where responses disagree, an entropy-based method that worked across 14 languages.
-
Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.
Discussion (0). Continue with ORCID to comment.