REVIEW 5 cited by
The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large Language Models (LLMs) have transformed the Natural Language Processing (NLP) landscape with their remarkable ability to understand and generate human-like text. However, these models are prone to ``hallucinations'' -- outputs that do not align with factual reality or the input context. This paper introduces the Hallucinations Leaderboard, an open initiative to quantitatively measure and compare the tendency of each model to produce hallucinations. The leaderboard uses a comprehensive set of benchmarks focusing on different aspects of hallucinations, such as factuality and faithfulness, across various tasks, including question-answering, summarisation, and reading comprehension. Our analysis provides insights into the performance of different models, guiding researchers and practitioners in choosing the most reliable models for their applications.
Forward citations
Cited by 5 Pith papers
-
HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representations
A training-free, single-generation hallucination detector based on HSIC dependence between hidden states of input and output tokens outperforms single-pass baselines and approaches multi-pass accuracy.
-
Expect the Unexpected: FailSafe Long Context QA for Finance
FailSafeQA, a 220-example financial long-context benchmark, shows no tested LLM can both stay robust to input perturbations and refuse to hallucinate when context is missing or irrelevant.
-
ViBe: A Text-to-Video Benchmark for Evaluating Hallucination in Large Multimodal Models
The paper builds and human-labels a 3,782-video dataset spanning five hallucination categories in text-to-video outputs, and shows that standard classifiers reach only about 35% accuracy on the resulting classification task.
-
Self-Training Large Language Models for Tool-Use Without Demonstrations
Correctness-filtered, self-generated tool-use traces fine-tuned via SFT or DPO improve accuracy on long-tail QA (PopQA +3.7) while giving mixed results elsewhere.
-
The Science of Evaluating Foundation Models
A survey-and-checklist proposal that organizes LLM evaluation into an ABCD framework (Algorithm, Big Data, Computation, Domain Expertise) for context-aware, documented assessment.
Discussion (0). Continue with ORCID to comment.