SinkProbe detects hallucinations in LLMs by analyzing attention sinks in attention maps, showing they indicate transitions to prior-dominated computation and achieving state-of-the-art results.
Hallucination Detection in LLMs with Topological Divergence on Attention Graphs
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models (LLMs). We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting, which leverages a topological divergence metric to quantify the structural properties of graphs induced by attention matrices. Examining the topological divergence between prompt and response subgraphs reveals consistent patterns: higher divergence values in specific attention heads correlate with hallucinated outputs, independent of the dataset. Extensive experiments - including evaluation on question answering and summarization tasks - show that our approach achieves state-of-the-art or competitive results on several benchmarks while requiring minimal annotated data and computational resources. Our findings suggest that analyzing the topological structure of attention matrices can serve as an efficient and robust indicator of factual reliability in LLMs.
fields
cs.CL 2representative citing papers
This survey compiles 137 papers on Topological Data Analysis in NLP, categorizing them into theoretical explanations of language and practical integrations into ML systems while noting open challenges.
citing papers explorer
-
Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
SinkProbe detects hallucinations in LLMs by analyzing attention sinks in attention maps, showing they indicate transitions to prior-dominated computation and achieving state-of-the-art results.
-
Topological Data Analysis Applications in Natural Language Processing: A Survey
This survey compiles 137 papers on Topological Data Analysis in NLP, categorizing them into theoretical explanations of language and practical integrations into ML systems while noting open challenges.