REVIEW 23 cited by
How Language Model Hallucinations Can Snowball
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
A major risk of using language models in practical applications is their tendency to hallucinate incorrect statements. Hallucinations are often attributed to knowledge gaps in LMs, but we hypothesize that in some cases, when justifying previously generated hallucinations, LMs output false claims that they can separately recognize as incorrect. We construct three question-answering datasets where ChatGPT and GPT-4 often state an incorrect answer and offer an explanation with at least one incorrect claim. Crucially, we find that ChatGPT and GPT-4 can identify 67% and 87% of their own mistakes, respectively. We refer to this phenomenon as hallucination snowballing: an LM over-commits to early mistakes, leading to more mistakes that it otherwise would not make.
Forward citations
Cited by 23 Pith papers
-
Reliability Scales Inversely: Hallucinations Snowball Faster in Bigger Language Models
Hallucination in LLMs is driven by an oracle-invisible “decoding risk” term that grows with scale and causally compounds errors within a response.
-
Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain
Misleading health documents in RAG context sharply lower LLM accuracy, and heavily helpful-biased retrieval pools restore it.
-
Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models
Social bias is a statistically significant cause of faithfulness hallucinations in LLMs, with anti-stereotypical contexts increasing errors and pro-stereotypical contexts decreasing them.
-
Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Models
Pre-trained multivariate time-series imputation models frequently return values that violate known relations between variables, and a diffusion-based score can detect and filter these errors.
-
Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Framework
A new error-attribution dataset and fine-tuned judge model that outputs score, error category, and feedback for LLM responses.
-
ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM
Directly steering attention during decoding, boosting text-centric heads and suppressing over-attended visual tokens, reduces object hallucination in multimodal LLMs by up to 38.2 percent with no retraining.
-
Generating Grounded Responses to Counter Misinformation via Learning Efficient Fine-Grained Critiques
MisMitiFact trains lightweight T5 critique models on fact-checking data to identify errors in numbers, entities, and topics, and uses their short critiques to refine LLM counter-responses at about 5x lower feedback cost.
-
Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding
Ensemble Decoding reduces object hallucination in large vision-language models by ensembling logits from attention-weighted image sub-images.
-
The Hallucination Tax of Reinforcement Finetuning
Standard RFT sharply reduces LLM refusal on unanswerable questions, and adding 10% synthetic unanswerable math during RFT restores refusal with small accuracy losses.
-
PlanGlow: Personalized Study Planning with an Explainable and Controllable LLM-Driven System
An LLM-driven study planning system that adds hierarchical explanations and real-time user controls outperformed ChatGPT and Khanmigo on user-rated explainability, controllability, and expert-assessed plan quality.
-
Optimization is Better than Generation: Optimizing Commit Message Leveraging Human-written Commit Message
A commit-message optimization method that starts from human-written messages and uses GPT-4 plus automated evaluators outperforms message generation and completion methods on three of four quality metrics.
-
Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models
An automated agent can jailbreak GPT-4o and other vision-language models using only individually safe images and benign-sounding prompts, escalating responses to harmful content.
-
REIN: Bridging the Gap between Reasoning and Reliability via Reflection and Abstention Alignment
REIN trains reasoning models with reflection-veracity and abstention rewards, letting them answer or abstain in a single pass, and reports large drops in a false-endorsement metric across four benchmarks.
-
Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination
A game-framed multi-agent system synthesizes large chemistry CoT/QA corpora and trains OmniChem-7B to near GPT-4o-mini performance with a large reported drop in hallucinations.
-
Towards Reliable Generative AI-Driven Scaffolding: Reducing Hallucinations and Enhancing Quality in Self-Regulated Learning Support
A multi-agent reliability check and an LLM-as-a-judge quality check both reduce hallucinations in AI-generated study scaffolds, with the multi-agent check matching human expert judgments almost perfectly.
-
Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding
FarSight adds upper-triangular negative biases to the causal mask to absorb outlier-token attention, reducing hallucinations in MLLMs without training.
-
EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models
Multimodal LLMs frequently accept hallucinated emotion claims on a new adversarial benchmark, with the worst failures on image, audio, and video perception rather than on textbook emotion knowledge.
-
Characterizing Unintended Consequences in Human-GUI Agent Collaboration for Web Browsing
Unintended consequences of web-browsing GUI agents fall into input, action, output, and feedback failures, with impacts ranging from frustration and financial loss to privacy breaches and eroded trust.
-
Towards Contamination Resistant Benchmarks
The authors define contamination resistance as a benchmark property and show that most tested LLMs score near zero on Caesar-cipher encoding and decoding when the shift is not 3 and the text is random nonsense.
-
A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy
A survey that organizes responsible-LLM research into five risk dimensions and four intervention phases, reviewing privacy, hallucination, value, toxicity, and jailbreak mitigation.
-
How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey
A survey that categorizes pre-trained-model-based vision-language methods into four challenge-driven paradigms, with performance tables and a discussion of risks.
-
Improving the Reliability of LLMs: Combining CoT, RAG, Self-Consistency, and Self-Verification
A small benchmark study claims self-verification is the best hallucination-reduction method, but the evaluation leaks the ground-truth answer into the verification prompt.
-
Language Models for Materials Discovery and Sustainability: Progress, Challenges, and Opportunities
A broad review and perspective on LLM applications in materials science, with proposals for automated discovery and sustainability screening.
Discussion (0). Continue with ORCID to comment.