REVIEW 13 cited by
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (large LMs) are susceptible to producing text that contains hallucinated content. An important instance of this problem is self-contradiction, where the LM generates two contradictory sentences within the same context. In this work, we present a comprehensive investigation into self-contradiction for various instruction-tuned LMs, covering evaluation, detection, and mitigation. Our primary evaluation task is open-domain text generation, but we also demonstrate the applicability of our approach to shorter question answering. Our analysis reveals the prevalence of self-contradictions, e.g., in 17.7% of all sentences produced by ChatGPT. We then propose a novel prompting-based framework designed to effectively detect and mitigate self-contradictions. Our detector achieves high accuracy, e.g., around 80% F1 score when prompting ChatGPT. The mitigation algorithm iteratively refines the generated text to remove contradictory information while preserving text fluency and informativeness. Importantly, our entire framework is applicable to black-box LMs and does not require retrieval of external knowledge. Rather, our method complements retrieval-based methods, as a large portion of self-contradictions (e.g., 35.2% for ChatGPT) cannot be verified using online text. Our approach is practically effective and has been released as a push-button tool to benefit the public at https://chatprotect.ai/.
Forward citations
Cited by 13 Pith papers
-
Benchmarking LLM-Assisted Blue Teaming via Standardized Threat Hunting
Standardized modular threat-hunting workflows (CyberTeam) improve LLM performance on blue team tasks compared to open-ended ICL, CoT, and ToT prompting across 30 tasks and 452k samples.
-
Unsupervised Hallucination Detection by Inspecting Reasoning Processes
IRIS detects LLM hallucinations by training a lightweight probe on hidden states elicited during the model's own step-by-step verification, using the model's verbalized confidence as soft pseudolabels.
-
Cross-Layer Attention Probing for Fine-Grained Hallucination Detection
CLAP, a cross-layer attention probe over all LLM layer activations, improves hallucination detection and enables a detect-then-mitigate decoding strategy.
-
Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Models
Pre-trained multivariate time-series imputation models frequently return values that violate known relations between variables, and a diffusion-based score can detect and filter these errors.
-
Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
A benchmark with expert-annotated reasoning steps and an LLM judge that scores reasoning by step coverage, showing strong correlation with expert evaluation.
-
Beyond Facts: Evaluating Intent Hallucination in Large Language Models
The paper proposes a query-centric evaluation of LLM "intent hallucination" via constraint decomposition, but the headline metric comparison is undermined by a self-referential human evaluation design.
-
Shaking to Reveal: Perturbation-Based Detection of LLM Hallucinations
SSP adds a learned, sample-specific noise prompt to an LLM input and scores hallucination by the cosine shift in intermediate representations, outperforming output-confidence baselines on QA benchmarks.
-
Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning
A new benchmark called GEOHALUBENCH measures how often LLMs invent, omit, or confuse real-world places and relations, and a dynamic-beta KTO method reduces these errors on the benchmark.
-
Pretrained LLMs Learn Multiple Types of Uncertainty
LLMs encode multiple dataset-specific linear directions in their hidden states that predict their own answer correctness, and these directions are nearly independent across benchmarks.
-
LLM-Driven Data Generation and a Novel Soft Metric for Evaluating Text-to-SQL in Aviation MRO
Introduces an F1-based soft metric for text-to-SQL evaluation and an LLM-driven data generation pipeline, tested on an aviation MRO database.
-
HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMs
Modeling the full token-by-token trajectory of LLM hidden states with neural ODEs, CDEs, and SDEs improves hallucination detection by over 14% AUC on a constructed true/false benchmark, though gains shrink on QA datasets.
-
A Mathematical Theory of Discursive Networks
A two-state Markov model of error propagation suggests that small amounts of cross-agent peer review can flip a network of fallible language models from a falsehood-dominant to a truth-dominant state.
-
Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.
Discussion (0). Sign in to comment.