REVIEW 13 cited by
FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Ensuring faithfulness to context in large language models (LLMs) and retrieval-augmented generation (RAG) systems is crucial for reliable deployment in real-world applications, as incorrect or unsupported information can erode user trust. Despite advancements on standard benchmarks, faithfulness hallucination-where models generate responses misaligned with the provided context-remains a significant challenge. In this work, we introduce FaithEval, a novel and comprehensive benchmark tailored to evaluate the faithfulness of LLMs in contextual scenarios across three diverse tasks: unanswerable, inconsistent, and counterfactual contexts. These tasks simulate real-world challenges where retrieval mechanisms may surface incomplete, contradictory, or fabricated information. FaithEval comprises 4.9K high-quality problems in total, validated through a rigorous four-stage context construction and validation framework, employing both LLM-based auto-evaluation and human validation. Our extensive study across a wide range of open-source and proprietary models reveals that even state-of-the-art models often struggle to remain faithful to the given context, and that larger models do not necessarily exhibit improved faithfulness.Project is available at: https://github.com/SalesforceAIResearch/FaithEval.
Forward citations
Cited by 13 Pith papers
-
ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning
ReSum trains LLMs via RLVR to self-summarize reasoning trajectories, yielding 4% average performance gains and 18.6% shorter rollouts through contrastive rollout branches.
-
Discourse-Role Labels as Presentation-Time Variables for Context Use in Language Models
Discourse-role labels on identical misleading context cause 56-84 percentage point shifts in LLMs adopting the injected wrong answer.
-
Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding
Context-Fidelity Boosting reduces faithfulness hallucinations by applying context-based logit boosts to source-supported tokens during LLM decoding.
-
MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models
MMOOC, a 41K-pair benchmark, shows current multimodal LLMs struggle to both refuse truly out-of-context questions and correctly answer questions that remain answerable despite misleading or shifted context.
-
On Improving Faithfulness of Podcasts from Documents
AI-generated podcasts often add unsupported claims; a turn-level detector plus rewrite pass improves measured faithfulness across five models and in- and out-of-domain documents.
-
ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning
ReSum's contrastive RL branching on self-summarization points improves LLM math reasoning accuracy by about 4% and shortens rollouts by about 18.6% across tested backbones.
-
A Benchmark Construction and Evaluation Framework for Specialist Domains: Case Study on Defense-related Documents
DoRA is a new synthetic benchmark for RAG-based QA on defense documents where fine-tuning Llama3.1-8B-Instruct on it improves task success by up to 26% and cuts hallucination rates by 47%.
-
Shorter, but Still Trustworthy? An Empirical Study of Chain-of-Thought Compression
CoT compression frequently introduces trustworthiness regressions with method-specific degradation profiles; a proposed normalized efficiency score and alignment-aware DPO variant reduce length by 19.3% with smaller t...
-
When Less is More: The LLM Scaling Paradox in Context Compression
Larger LLM compressors in lossy setups often yield less faithful context reconstructions due to knowledge overwriting and semantic drift, with mid-sized models outperforming larger ones across 27 tested configurations.
-
ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking
FAITH masks numbers in real 10-K reports to test when financial LLMs hallucinate, and finds even top models err on 10-20% of multi-step calculations.
-
Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
Hallucinations on structured knowledge arise from attention concentrating on shortcut structural cues and feed-forward layers failing to ground provided knowledge, with these patterns generalizing across single-hop, m...
-
A Benchmark Construction and Evaluation Framework for Specialist Domains: Case Study on Defense-related Documents
DoRA generates synthetic RAG training and evaluation data from 40 defense documents, halving hallucination rates in a LoRA-adapted Llama3.1-8B compared to 8 baselines.
-
Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs
On a new extended needle-in-a-haystack benchmark, explicit anti-hallucination prompts and dispersed fact placement cause some long-context LLMs to over-refuse or collapse in accuracy, while others remain robust.
Discussion (0). Sign in to comment.