Pith. sign in

REVIEW 7 cited by

Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.11207 v1 pith:KTHXHADB submitted 2023-10-17 cs.CL cs.LG

Shiyuan Huang , Siddarth Mamidanna , Shreedhar Jangam , Yilun Zhou , Leilani H. Gilpin This is my paper
classification cs.CLcs.LG
keywords self-explanationsmodelssentimentchatgptexplanationlanguageproducethem
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) such as ChatGPT have demonstrated superior performance on a variety of natural language processing (NLP) tasks including sentiment analysis, mathematical reasoning and summarization. Furthermore, since these models are instruction-tuned on human conversations to produce "helpful" responses, they can and often will produce explanations along with the response, which we call self-explanations. For example, when analyzing the sentiment of a movie review, the model may output not only the positivity of the sentiment, but also an explanation (e.g., by listing the sentiment-laden words such as "fantastic" and "memorable" in the review). How good are these automatically generated self-explanations? In this paper, we investigate this question on the task of sentiment analysis and for feature attribution explanation, one of the most commonly studied settings in the interpretability literature (for pre-ChatGPT models). Specifically, we study different ways to elicit the self-explanations, evaluate their faithfulness on a set of evaluation metrics, and compare them to traditional explanation methods such as occlusion or LIME saliency maps. Through an extensive set of experiments, we find that ChatGPT's self-explanations perform on par with traditional ones, but are quite different from them according to various agreement metrics, meanwhile being much cheaper to produce (as they are generated along with the prediction). In addition, we identified several interesting characteristics of them, which prompt us to rethink many current model interpretability practices in the era of ChatGPT(-like) LLMs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 21 citations worldwide. Full citation record

  1. PEDANTIC: A Dataset for the Automatic Examination of Definiteness in Patent Claims

    cs.CL 2025-05 conditional novelty 7.0 of 10

    PEDANTIC provides the first public dataset of 14k patent claims labeled with examiner-cited reasons for indefiniteness, along with baselines showing LLMs still lag logistic regression on binary prediction.

  2. Social Media Clones: Exploring the Impact of Social Delegation with AI Clones through a Design Workbook Study

    cs.HC 2025-09 conditional novelty 6.0 of 10

    Using interviews and a design workbook with 32 social media users, the study finds that AI clones may create convenience and comfort but also threaten authenticity, increase skepticism, and lead users to mimic their clones.

  3. Identifying Implicit Bias in LLM-based Chat AI Toward People with Intellectual Disabilities

    cs.CY 2026-06 conditional novelty 5.0 of 10

    Across 25,000 stories from five LLMs, an LLM judge rated stories mentioning intellectual disabilities as more infantile, paternalistic, dependent, and inspirational than stories without the label.

  4. HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving

    cs.RO 2025-05 conditional novelty 5.0 of 10

    The HCRMP planner feeds LLM semantic hints into state representation and critic weighting instead of letting the LLM decide actions, reporting better CARLA driving metrics.

  5. From Plausible to Actionable: A Position on LLM Self-Explanations

    cs.CL 2026-07 conditional novelty 4.0 of 10

    Self-explanations from LLMs should be evaluated by their actionability for stakeholders rather than by plausibility or faithfulness alone.

  6. Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations

    cs.CL 2025-10 conditional novelty 4.0 of 10

    LLM explanations split into local and mechanistic tracks; the paper argues they are trustworthy only if they pass causal and contrastive stress tests, adapt to the explainee, and satisfy eight trust principles.

  7. Know-MRI: A Knowledge Mechanisms Revealer&Interpreter for Large Language Models

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Know-MRI combines eleven existing LLM interpretation methods into one extensible toolkit with automatic input-to-method matching and dual UI and code interfaces.

Pith tools