REVIEW 6 cited by
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In the era of large language models (LLMs), hallucination (i.e., the tendency to generate factually incorrect content) poses great challenge to trustworthy and reliable deployment of LLMs in real-world applications. To tackle the LLM hallucination, three key questions should be well studied: how to detect hallucinations (detection), why do LLMs hallucinate (source), and what can be done to mitigate them (mitigation). To address these challenges, this work presents a systematic empirical study on LLM hallucination, focused on the the three aspects of hallucination detection, source and mitigation. Specially, we construct a new hallucination benchmark HaluEval 2.0, and designs a simple yet effective detection method for LLM hallucination. Furthermore, we zoom into the different training or utilization stages of LLMs and extensively analyze the potential factors that lead to the LLM hallucination. Finally, we implement and examine a series of widely used techniques to mitigate the hallucinations in LLMs. Our work has led to several important findings to understand the hallucination origin and mitigate the hallucinations in LLMs. Our code and data can be accessed at https://github.com/RUCAIBox/HaluEval-2.0.
Forward citations
Cited by 6 Pith papers
-
MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them
A new benchmark with a three-way hallucination taxonomy, snapshot-based test cases, and an LLM judge shows LLM agents hallucinate at over 30% of risky decision points, with open and closed models closer than expected.
-
The Cost of Knowing: A Resource-Aware Protocol for Benchmarking Hallucination Beyond Static Leaderboards
MAS-HQ defines a resource-aware Q-Score and shows that the system with the highest raw factuality is often not the winner once normalized cost is subtracted.
-
Decision Protocols in Multi-Agent Large Language Model Conversations
Consensus decision protocols beat voting/judge on knowledge QA for Llama-3 multi-agent chats, while voting and judge win on logic tasks; independent initial drafts raise accuracy and extra voting-time info barely helps.
-
Enhancing Factual Accuracy and Citation Generation in LLMs via Multi-Stage Self-Verification
The paper proposes a four-stage self-verification prompting method but explicitly labels its experimental results as fabricated, so it cannot support its claimed gains.
-
Building Task Bots with Self-learning for Enhanced Adaptability, Extensibility, and Factuality
A thesis that combines self-learning from dialog logs, schema-guided prompting, and self-aligned factuality to build task bots with minimal human intervention.
-
A comprehensive taxonomy of hallucinations in Large Language Models
A survey that organizes LLM hallucination types, causes, benchmarks, and mitigations, and restates the theorem that hallucination is inevitable for computable LLMs.
Discussion (0). Sign in to comment.