REVIEW 23 cited by
Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
read the original abstract
Large Language Models (LLMs) are deployed as powerful tools for several natural language processing (NLP) applications. Recent works show that modern LLMs can generate self-explanations (SEs), which elicit their intermediate reasoning steps for explaining their behavior. Self-explanations have seen widespread adoption owing to their conversational and plausible nature. However, there is little to no understanding of their faithfulness. In this work, we discuss the dichotomy between faithfulness and plausibility in SEs generated by LLMs. We argue that while LLMs are adept at generating plausible explanations -- seemingly logical and coherent to human users -- these explanations do not necessarily align with the reasoning processes of the LLMs, raising concerns about their faithfulness. We highlight that the current trend towards increasing the plausibility of explanations, primarily driven by the demand for user-friendly interfaces, may come at the cost of diminishing their faithfulness. We assert that the faithfulness of explanations is critical in LLMs employed for high-stakes decision-making. Moreover, we emphasize the need for a systematic characterization of faithfulness-plausibility requirements of different real-world applications and ensure explanations meet those needs. While there are several approaches to improving plausibility, improving faithfulness is an open challenge. We call upon the community to develop novel methods to enhance the faithfulness of self explanations thereby enabling transparent deployment of LLMs in diverse high-stakes settings.
Forward citations
Cited by 23 Pith papers
-
Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth
Introduces BonaFide benchmark of 3,066 ground-truth labeled CoTs showing most faithfulness metrics perform near chance with biases and poor scaling to longer chains.
-
Forecasting Future Behavior as a Learning Task
Behavior Forecasters trained on LRM trajectories outperform larger models in predicting repeatability and input sensitivity at low cost.
-
Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Models
Concept-based abductive and contrastive explanations find minimal high-level concepts that causally determine vision model outcomes on individual images or groups sharing a specified behavior.
-
When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't
VLMs violate their own stated introspective rules for attributing colors to objects in nearly 60% of cases on items with strong color priors, unlike humans who largely follow theirs, revealing miscalibrated self-knowledge.
-
What LLMs explain is not what they believe: Evaluating explanation sufficiency under models' own input beliefs
Proposes SCSuff metric for evaluating LLM explanation sufficiency via model-generated alternative inputs, showing explanations are typically insufficient and predictable from hidden states.
-
BetXplain: An Explanation-Annotated Dataset for Detecting Manipulative Betting Advertisements on Social Media
The authors introduce an explanation-annotated dataset of manipulative betting advertisements collected from Instagram and Reddit to support explainable detection models.
-
Answer Engineering: Local Trajectory Editing for Protocol-Constrained Decision Making in Large Language Models
Answer Engineering uses local trajectory editing during autoregressive generation to raise protocol compliance on a clinical SSNHL benchmark from 25.1% to 83.5% and balanced accuracy from 42.0% to 80.7%.
-
A Finetuned SpeechLLM for Joint Multi-Granular L2 Assessment and Natural-Language Rationales
A rubric-guided SpeechLLM jointly predicts multi-granular L2 proficiency labels and generates natural-language rationales using hybrid SFT and Bounded DPO, matching prior performance on SpeechOcean762 with plausible s...
-
Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy
CIE-Scorer detects unfaithful CoT by tracing compact sentence-level circuits, building internal-external reasoning graphs, and scoring their discrepancy with Fused Gromov-Wasserstein distance, reporting SOTA results o...
-
Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization
Optimizing LLMs for parametric CoT faithfulness improves both paradigms consistently while contextual optimization yields more variable gains, and different contextual metrics do not transfer reliably to each other.
-
Interpretability Can Be Actionable
Interpretability research should be judged by actionability—the degree to which its insights support concrete decisions and interventions—rather than explanatory power alone.
-
PageGuide: Browser extension to assist users in navigating a webpage and locating information
PageGuide is a browser extension that grounds LLM responses in webpage DOM elements via visual overlays for Find, Guide, and Hide modes, reporting performance gains over unaided browsing in a 94-user study.
-
PageGuide: Browser extension to assist users in navigating a webpage and locating information
PageGuide grounds LLM answers in webpage DOM elements using visual overlays for find, guide, and hide modes, yielding measurable gains in a 94-user study.
-
AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
AtManRL learns an additive attention mask on CoT traces to produce a saliency reward that, when combined with outcome rewards in GRPO, trains LLMs to generate reasoning that genuinely influences final predictions.
-
When AI Persuades: Adversarial Explanation Attacks on Human Trust in AI-Assisted Decision Making
Adversarial explanation attacks preserve nearly all human trust in wrong AI outputs by using persuasive framing, shown in a study varying reasoning, evidence, style, and format with over 200 participants.
-
The Shape of Reasoning: Topological Analysis of Reasoning Traces in Large Language Models
A TDA-based framework is proposed to assess LLM reasoning trace quality, showing that topological features predict quality better than standard graph metrics.
-
Spatial Reasoning via Modality Switching Between Language and Symbolic Representation
Introduces a modality-switching mechanism for LLMs on spatial reasoning tasks using a trustworthiness and complexity based metric, showing up to 42% performance improvement.
-
Spatial Reasoning via Modality Switching Between Language and Symbolic Representation
Introduces a trustworthiness-and-complexity switching metric that lets LLMs choose between language and grid modalities for spatial reasoning, yielding up to 42% gains in tested settings.
-
BetXplain: An Explanation-Annotated Dataset for Detecting Manipulative Betting Advertisements on Social Media
Introduces BetXplain, an explanation-annotated dataset of social media betting ads collected from Instagram and Reddit for detecting manipulative and deceptive advertising.
-
A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification
Self-explanations from LLMs produce faithful token subsets for correct predictions but align with human rationales only conditionally on text length and task complexity, unlike post-hoc attribution methods that highli...
-
Governing Generative AI Across Financial Institutions: An SR 26-2-Compatible Framework for Generative AI Risk Control
GAICF maps SR 26-2 model-risk principles into approved-use gates, risk tiers, evidence checks, and output monitoring for generative AI outside the formal model boundary.
-
OpenAI o1 System Card
OpenAI reports that chain-of-thought reasoning in o1 models enables deliberative alignment, yielding state-of-the-art results on selected safety benchmarks for illicit advice, stereotypes, and jailbreaks.
-
Scientific Explanations in Health Sciences: Causality, Trust, and Epistemic Adequacy
A review paper arguing that accounts of explanation from philosophy of science supply necessary conditions for adequate XAI design in clinical settings.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.