REVIEW 6 cited by
LettuceDetect: A Hallucination Detection Framework for RAG Applications
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
LettuceDetect: A Hallucination Detection Framework for RAG Applications
read the original abstract
Retrieval Augmented Generation (RAG) systems remain vulnerable to hallucinated answers despite incorporating external knowledge sources. We present LettuceDetect a framework that addresses two critical limitations in existing hallucination detection methods: (1) the context window constraints of traditional encoder-based methods, and (2) the computational inefficiency of LLM based approaches. Building on ModernBERT's extended context capabilities (up to 8k tokens) and trained on the RAGTruth benchmark dataset, our approach outperforms all previous encoder-based models and most prompt-based models, while being approximately 30 times smaller than the best models. LettuceDetect is a token-classification model that processes context-question-answer triples, allowing for the identification of unsupported claims at the token level. Evaluations on the RAGTruth corpus demonstrate an F1 score of 79.22% for example-level detection, which is a 14.8% improvement over Luna, the previous state-of-the-art encoder-based architecture. Additionally, the system can process 30 to 60 examples per second on a single GPU, making it more practical for real-world RAG applications.
Forward citations
Cited by 6 Pith papers
-
RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
RAGognizer adds a detection head to LLMs for joint training on generation and token-level hallucination detection, yielding SOTA detection and fewer hallucinations in RAG while preserving output quality.
-
SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems
SIRIN unifies probing, uncertainty, and judge-style hallucination detectors with query answerability under one pluggable toolkit, evaluation pipeline, and interactive span-highlighting UI.
-
Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)
Perturbing retrieved context and scoring likelihood drops plus JSD yields a training-free span-level RAG hallucination detector competitive with chunk-level NLI and stronger than perplexity.
-
Detecting Hallucinations for Large Language Model-based Knowledge Graph Reasoning
LUCID detects hallucinations in LLM-KG reasoning by extracting node/edge features from attention and semantics then integrating them with KG structure in a GNN, achieving SOTA on nine new benchmark datasets versus 15 ...
-
Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAG
A claim-by-claim hierarchical verifier improves RAG hallucination detection over baselines, and a re-annotated benchmark finds 1.68x more hallucinated cases than the original labels.
-
Do Benchmarks Underestimate LLM Performance? Evaluating Hallucination Detection With LLM-First Human-Adjudicated Assessment
Human adjudication of conflicts between original benchmark labels and LLM predictions on QAGS-C and SummEval increases triple agreement by 6-8% and LLM accuracy by 2-9%, with adjudicators often siding with models that...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.