REVIEW 5 cited by
GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework
read the original abstract
Methods to evaluate Large Language Model (LLM) responses and detect inconsistencies, also known as hallucinations, with respect to the provided knowledge, are becoming increasingly important for LLM applications. Current metrics fall short in their ability to provide explainable decisions, systematically check all pieces of information in the response, and are often too computationally expensive to be used in practice. We present GraphEval: a hallucination evaluation framework based on representing information in Knowledge Graph (KG) structures. Our method identifies the specific triples in the KG that are prone to hallucinations and hence provides more insight into where in the response a hallucination has occurred, if at all, than previous methods. Furthermore, using our approach in conjunction with state-of-the-art natural language inference (NLI) models leads to an improvement in balanced accuracy on various hallucination benchmarks, compared to using the raw NLI models. Lastly, we explore the use of GraphEval for hallucination correction by leveraging the structure of the KG, a method we name GraphCorrect, and demonstrate that the majority of hallucinations can indeed be rectified.
Forward citations
Cited by 5 Pith papers
-
MedHal-Loc: Are "Explainable-by-Architecture" Medical Hallucination Detectors Faithful Localizers? A Localization Benchmark
MedHal-Loc benchmark shows KG-triple hallucination detectors localize errors no better than chance on controlled medical statements due to entity extraction limits, while NLI and consistency methods succeed above chan...
-
Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrieval
A structured survey organizing graph-LLM integration methods by purpose, modality, and strategy across application domains.
-
From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives
MAGNET multi-agent generation with persona grounding and ATLAS graph verification yields 34-50% fewer hallucinations and annotations than single-model or IBSEN baselines at 100-page scale.
-
Cross Paraphrastic Invariance Learning for Hallucination Detection
CPIL is a contrastive two-stage method that enforces paraphrase invariance on limited labeled data to outperform baselines in hallucination detection across 11 tasks.
-
Position: How can Graphs Help Large Language Models?
Graphs can help LLMs reduce hallucinations, boost reasoning via prompting techniques, and better process structured data.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.