Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Self-Critique and Refinement for Faithful Natural Language Explanations

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs can make their own post-hoc explanations more faithful to their actual reasoning by iteratively critiquing and refining them, cutting unfaithfulness from 54.81% to 36.02% without any fine-tuning.

desk verdict A transparent but methodologically under-powered study: the self-refinement framework is new and the experiments are consistent, yet the faithfulness metric and the random-word baseline undermine the central claim. read the letter →

arxiv 2505.22823 v2 pith:TMOKOOPL submitted 2025-05-28 cs.CL

Yingming Wang , Pepa Atanasova This is my paper
classification cs.CL
keywords naturallanguageexplanationsfaithfulnessself-refinementself-critiquefeatureattributionimportantwordfeedbackcounterfactualtestlargemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a large language model can make its own natural language explanations more faithful to its actual reasoning, without external supervision or training. The proposed framework, SR-NLE, runs an iterative loop in which the model critiques its explanation and rewrites it, guided either by free-text self-feedback or by a list of the input words that most influenced the prediction. Across three reasoning datasets and four instruction-tuned models, the strongest variant, attention-based important-word feedback, lowers the average unfaithfulness rate from 54.81% to 36.02% on a counterfactual faithfulness test. The authors conclude that LLMs can substantially improve the faithfulness of their own explanations, given the right kind of feedback.

What carries the argument

The central object is the important-word feedback (IWF) signal: a small set of input words that the model's own prediction depended on, extracted either by prompting the model to rank words or by computing feature attributions (attention weights or integrated gradients). Each round, the model receives the top-5 important words and is asked to integrate them into its explanation. This works by making the explanation mention the lexical evidence that actually changed the prediction, which is exactly what the counterfactual test checks. The iterative loop, running for up to three rounds, converts this guidance into progressively more faithful text, with attention-based scoring yielding the best results.

What would settle it

Compare SR-NLE's refined explanations against a length-matched baseline that asks the model to rewrite without important-word feedback, or against a different faithfulness metric (e.g., simulatability or human agreement); if the unfaithfulness reduction disappears or reverses, the central claim is falsified.

Watch

Extended reading notes

Core claim

The paper introduces SR-NLE and claims that an LLM can refine its own post-hoc explanations so they better reflect the reasoning that produced its prediction. The framework generates an initial explanation, then iterates between feedback and refinement; feedback can be natural-language self-critique or a list of the top-N most important input words identified by prompt-based, attention-based, or gradient-based attribution. Using a counterfactual test that judges an explanation faithful when it contains the intervened word, SR-NLE with attention-based important word feedback reduces the unfaithfulness rate from 54.81% to 36.02% on average across ComVE, ECQA, and e-SNLI with Falcon, Llama, Mistral, and Qwen models. The claim is that the improvement is driven by the iterative self-critique process itself and by feedback that points the model to the words that actually carried its decision.

Load-bearing premise

The paper's central claim rests on the counterfactual test's rule that an explanation is faithful if it contains the intervened word; if that string-match criterion does not capture whether the explanation reflects the model's actual reasoning, the measured reduction in unfaithfulness would not establish the claimed benefit.

Editorial extensions

If this is right

  • If SR-NLE is correct, explanation faithfulness can be improved at inference time with no fine-tuning, no architectural changes, and no extra models, using only the target LLM itself.
  • Prompt-based important-word feedback performs nearly as well as attribution-based feedback (37.23% vs 36.02% unfaithfulness), indicating that explicit access to internal attributions is not required.
  • Most of the improvement comes in the first refinement round, so two to three rounds offer a practical trade-off between faithfulness and compute.
  • Even the best variant leaves an average unfaithfulness rate of 36%, so self-refinement alone does not fully solve the faithfulness problem; further gains would require better feedback signals or training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because prompt-based IWF nearly matches attribution-based IWF, the method likely transfers to API-only LLMs with no gradient or attention access; testing SR-NLE on a closed-weight model would verify this.
  • Editorial inference: the inverse relationship between explanation length and unfaithfulness (Figure 6) leaves open whether part of the gain is a verbosity effect; a length-matched baseline that rewrites without important-word feedback would separate the feedback mechanism from mere elaboration.
  • Editorial inference: the appendix finding that random important words still reduce unfaithfulness, and that the intervened word is only captured 30–46% of the time in top-5 selections, suggests the refinement loop itself carries much of the benefit; an even simpler feedback prompt (e.g., 'mention the word that changed your prediction') might capture a similar effect.
  • Editorial inference: a natural next test is to apply SR-NLE to chain-of-thought rationales or jointly generated explanations, which the authors explicitly leave open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SR-NLE, an iterative self-critique and refinement framework that aims to improve the faithfulness of post-hoc natural language explanations (NLEs) generated by LLMs. The framework generates an initial NLE, then iteratively refines it using either free-form natural language feedback (NLF) or important-word feedback (IWF) derived from prompt-based, attention-based, or gradient-based attribution. Experiments on ComVE, ECQA, and e-SNLI with four instruction-tuned LLMs under 10B parameters evaluate faithfulness using counterfactual tests in which an explanation is considered faithful if it contains the intervened word by string matching. The best variant, IWF-Attn, reduces the average unfaithfulness rate from 54.81% to 36.02%, an absolute reduction of 18.79 percentage points.

Significance. If the claimed effect is genuine, the paper would make a practically useful contribution: an inference-time method for improving explanation faithfulness without fine-tuning or architectural changes, with released code and a broad experimental matrix (3 datasets x 4 models). The paper also includes useful ablations and an honest random-word baseline in Appendix F.6. However, the central claim is not yet established because the evaluation metric is aligned with the IWF feedback mechanism, no significance tests are provided, and the length-faithfulness correlation is uncontrolled. The paper's own Limitations section concedes that the counterfactual test reflects only one type of faithfulness criterion. The value of the paper therefore depends on whether the evaluation can be made more robust and whether the gains can be shown to be specific to informative feedback rather than to prompt-driven word echoing or length increases.

major comments (4)
  1. The counterfactual metric in Eq. (10) counts an explanation as faithful if it contains the intervened word, matched by string. For IWF variants, the refinement prompt (Appendix I, Table 22) instructs the model to "integrate the important words naturally," and the intervened word is part of the supplied feedback whenever it appears in the top-N list. Thus a refined explanation can pass the test by echoing a fed-back word rather than by demonstrating the model's actual reasoning. This concern is supported by the paper's own random-word baseline: Table 12 shows that random word selection achieves an unfaithfulness of 29.28% on e-SNLI, close to IWF-Pmt (30.10%) and IWF-IG (28.39%), while Table 13 shows the intervened word is in the top-5 only 46.25% of the time for IWF-Attn and 34.00% for random. The Limitations section concedes that the counterfactual test "reflects only one type of faithfulness criterion." Without an additional semantically grounded faithfulness metric, the reported absolute reduction of 18.79% does not establish that the explanations better reflect the model's reasoning.
  2. Figure 6 shows a strong inverse relationship between explanation length and unfaithfulness across refinement rounds, but the paper provides no length-controlled baseline. Since longer explanations are more likely to contain the intervened word by chance, the observed improvement could be an artifact of the refinement prompts producing longer text. The claim in §5.3 that IWF methods "optimize explanation content rather than merely increasing length" requires a length-matched comparison or a length-normalized metric; as presented, the causal interpretation is not supported.
  3. The abstract and §5.1 state that SR-NLE "significantly reduces" unfaithfulness, but the main results in Table 1 are reported without significance tests, confidence intervals, or standard deviations. Differences between IWF variants are small (e.g., 36.02% vs. 36.30% vs. 37.23%), and the random-word baseline in Appendix F.6 overlaps with these numbers. Statistical testing across the 12 model-dataset conditions is needed to support the comparative and "significant" claims.
  4. The framework assumes that LLMs can identify and improve their own explanations when guided with appropriate prompts, but this assumption is not tested directly. The random-word baseline in Table 12 achieving near-IWF performance suggests that the model may be following a surface instruction to incorporate supplied words rather than evaluating which words are causally important for its prediction. A control that measures self-assessment ability independently of the feedback content, or a comparison with feedback containing genuinely irrelevant words in a semantically meaningful way, would strengthen this assumption.
minor comments (4)
  1. Several numerical entries are run together without spaces (e.g., "24.3243.12" in Table 1 under ECQA/Llama/IWF-Pmt), making the tables difficult to read.
  2. The abstract claims the method requires "no external supervision," but Appendix C uses GPT-4o to generate counterfactual interventions. Please clarify that this external model is used only for evaluation, not for the refinement process.
  3. The comparison with G-TEX is acknowledged as not fully controlled, but the abstract and conclusion still describe SR-NLE as outperforming prior methods; either remove this comparison or keep the caveat in the summary statements.
  4. Ablation studies are conducted on 100-instance subsets without confidence intervals; the choice of N=5 and other hyperparameters would be more convincing with error bars or a small repeat-seed analysis.

Circularity Check

1 steps flagged · score 6.0 of 10

IWF feedback injects the very token that the counterfactual metric checks, so part of SR-NLE's reported faithfulness gain is by construction.

  1. self definitional [Section 3.2.2 Eq. (6), Section 4.3 Eq. (10), Table 22]
    "Identify unfaithful instances: counter instances whose NLEs (generated by baseline methods or SR-NLE) do not contain the intervened word (determined by string matching). ... Integrate the important words naturally and fluently—do not list or quote them directly."

    For IWF, feedback fiw = FORMAT(I) is a list of top-N input words; refinement uses er = M(pref ⊕ x ⊕ y ⊕ e_{r-1} ⊕ fiw), and Table 22 instructs the model to integrate those words. The evaluation (Eq. 10) counts an explanation as faithful exactly when it contains the intervened word. Thus, whenever the intervened word is among the selected top-N words, the refined explanation passes by construction: the target token was supplied by the feedback and the prompt forced its inclusion. Table 13 shows this happens for 46.25% (Attn) of counter instances, and Table 12 shows a random-word baseline matches IWF, confirming the improvement is not tied to identifying the true reasoning word. The reported 18.79% reduction is therefore partly an artifact of aligning feedback with the metric.

full rationale

The clearest circularity is in the IWF variants: the feedback mechanism feeds back input words and instructs their integration, while the sole evaluation metric rewards explanations that contain the intervened word. For instances where the intervened word is in the top-N feedback (46.25% of counter instances for IWF-Attn per Table 13), the 'faithfulness' improvement is by construction. This is not a complete tautology: the intervened word is not always selected, the random-word baseline (29.28%) still improves over Init-NLE, and the paper's Appendix F.6 transparently shows that word-selection quality is not the driver. Nevertheless, the claimed 18.79% average reduction is partially an artifact of the metric/feedback alignment, and the conclusion that LLMs 'better reflect their actual reasoning process' is not established by the string-matching test alone. The counterfactual test is cited from Atanasova et al. (2023), which includes a co-author of this paper, but that prior work is an external, published benchmark and is not circular by itself. The absence of a length-controlled baseline and the reliance on a single metric are validity concerns rather than circular derivation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new theoretical entities. Its main load-bearing assumptions are about the validity of the counterfactual metric and the model's self-assessment capability. The free parameters (N, K, IG steps) are tuned on the same evaluation data, which is a circularity concern.

free parameters (4)
  • Top-N important words = 5
    Selected based on ablation on the same test subsets (Appendix E.3), where unfaithfulness decreased as N increased from 1 to 9; N=5 was chosen as a trade-off. This is a free parameter tuned to the evaluation data.
  • Number of refinement rounds K = 3
    Chosen based on observed diminishing returns in Figure 5; the paper limits experiments to K=3. This is a hyperparameter selected using the evaluation metric.
  • IG integration steps = 500 (Falcon, Llama, Mistral), 1000 (Qwen)
    Selected based on convergence analysis on 100 instances (Appendix E.1, Figure 7). Choice affects attribution scores and thus feedback.
  • SC-NLE sample count and temperature = 20 candidates, temperature 1.0
    Baseline parameters chosen via ablation (Appendix E.2); not central to SR-NLE but a fitted configuration.
assumptions (4)
  • domain assumption LLMs can identify and improve their own explanations when guided with appropriate prompts.
    Stated in Section 3.1: 'The SR-NLE framework operates on the assumption that LLMs have the capability to identify and improve their own explanations when guided with appropriate prompts.' This is the foundation of the self-critique approach.
  • domain assumption The counterfactual test of Atanasova et al. (2023) is a valid measure of explanation faithfulness.
    Section 4.3 adopts this test as the sole evaluation metric, defining unfaithfulness as the explanation not containing the intervened word. The paper does not validate this against other faithfulness criteria.
  • domain assumption GPT-4o-generated interventions are meaningful and coherent.
    The paper uses GPT-4o to generate 20 interventions per instance and manually checked 50 per dataset (Appendix C). This relies on an external model for a critical evaluation input.
  • domain assumption The word-level aggregation of token attributions is a valid way to identify important words.
    Section 3.3 assumes that summing absolute attribution scores over tokens and then over words yields importance scores that reflect the model's reasoning for the answer span.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Critique and Refinement for Faithful Natural Language Explanations." pith.science (2026). https://pith.science/paper/TMOKOOPL

@misc{pith2026250522823,
  author       = {Pith},
  title        = {Pith review of: Self-Critique and Refinement for Faithful Natural Language Explanations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TMOKOOPL}},
  note         = {Machine review of arXiv:2505.22823}
}
read the original abstract

With the rapid development of Large Language Models (LLMs), Natural Language Explanations (NLEs) have become increasingly important for understanding model predictions. However, these explanations often fail to faithfully represent the model's actual reasoning process. While existing work has demonstrated that LLMs can self-critique and refine their initial outputs for various tasks, this capability remains unexplored for improving explanation faithfulness. To address this gap, we introduce Self-critique and Refinement for Natural Language Explanations (SR-NLE), a framework that enables models to improve the faithfulness of their own explanations -- specifically, post-hoc NLEs -- through an iterative critique and refinement process without external supervision. Our framework leverages different feedback mechanisms to guide the refinement process, including natural language self-feedback and, notably, a novel feedback approach based on feature attribution that highlights important input words. Our experiments across three datasets and four state-of-the-art LLMs demonstrate that SR-NLE significantly reduces unfaithfulness rates, with our best method achieving an average unfaithfulness rate of 36.02%, compared to 54.81% for baseline -- an absolute reduction of 18.79%. These findings reveal that the investigated LLMs can indeed refine their explanations to better reflect their actual reasoning process, requiring only appropriate guidance through feedback without additional training or fine-tuning.

Figures

Figures reproduced from arXiv: 2505.22823 by the authors.

Figure 1
Figure 1. Illustration of our framework SR-NLE im [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The SR-NLE framework. (a) Answer and Explanation Generation Phase: The framework pro￾duces the answer x and initial explanation e 0 . (b) Itera￾tive Critique and Refinement Phase: The framework iteratively improves explanations through feedback￾refinement loops over multiple rounds. 3.2 SR-NLE Framework Our SR-NLE framework consists of two main phases: (a) Answer and Explanation Generation, which produces the answer… view at source ↗
Figure 3
Figure 3. Illustration of attribution-based IWF SCORE. Word-level Aggregation To obtain word-level importance, we map token attributions back to the original words in the task input (excluding prompt tokens). For words split into multiple tokens during tokenization, we combine their attribution scores: score(w) = X i∈indices(w) ai (9) where indices(w) represents the indices of all to￾kens corresponding to word w in the task i… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Faithfulness state transitions from e 0 to e 3 for different feedback methods, averaged across 12 model￾dataset combinations. The left group shows the pro￾portion of initially faithful explanations that become unfaithful, while the right group shows the proportion of i…
Figure 5
Figure 5. Figure 5: Unfaithfulness rates across successive refine [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: Impact of number of integration steps on con [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 9
Figure 9. Figure 9: Analysis of top-N important words selection. (a) Shows how unfaithfulness decreases with increasing [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Radar chart visualization of unfaithfulness [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Training Large Language Models for Self-Explanation Faithfulness

    cs.LG 2026-07 conditional novelty 6.0 of 10

    RL fine-tuning with a counterfactual mention/influence reward raises LLM self-explanation faithfulness (Phi-CCT) from near zero to ~0.66 in-distribution for two 8B models, with partial transfer to held-out tasks.

Reference graph

Works this paper leans on

42 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. 2021. https://doi.org/10.18653/v1/2021.acl-long.238 E xplanations for C ommonsense QA : N ew D ataset and M odels . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen...

    doi:10.18653/v1/2021.acl-long.238 2021

  2. [2]

    Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. https://arxiv.org/abs/2311.16867 The falcon series of open language models . Preprint, arXiv:2...

    arXiv 2023

  3. [3]

    Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasiewicz, Jakob Grue Simonsen, and Isabelle Augenstein. 2023. https://doi.org/10.18653/v1/2023.acl-short.25 Faithfulness tests for natural language explanations . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 283--294...

    doi:10.18653/v1/2023.acl-short.25 2023

  4. [4]

    Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2020 a . https://doi.org/10.18653/v1/2020.emnlp-main.263 A diagnostic study of explainability techniques for text classification . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3256--3274, Online. Association for Comput...

    doi:10.18653/v1/2020.emnlp-main.263 2020

  5. [5]

    Generating Fact Checking Explanations

    Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2020 b . https://doi.org/10.18653/v1/2020.acl-main.656 "Generating Fact Checking Explanations" . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7352--7364, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.acl-main.656 2020

  6. [6]

    Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2022. Diagnostics-guided explanation generation . In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10445--10453

    work page 2022

  7. [7]

    Milan Bhan, Jean-Noel Vittaut, Nicolas Chesneau, and Marie-Jeanne Lesot. 2024. https://arxiv.org/abs/2402.12038 Self-amplify: Improving small language models with self post hoc explanations . Preprint, arXiv:2402.12038

    work page Pith review arXiv 2024

  8. [8]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. https://arxiv.org/abs/2005.14165 Language models are few-shot learners . Preprint, arXiv:2005.14165

    arXiv 2020

Show all 42 references
  1. [9]

    Oana-Maria Camburu, Tim Rockt\" a schel, Thomas Lukasiewicz, and Phil Blunsom. 2018. https://proceedings.neurips.cc/paper_files/paper/2018/file/4c7a167bb329bd92580a99ce422d6fa6-Paper.pdf e-snli: Natural language inference with natural language explanations . In Advances in Neu...

    2018

  2. [10]

    Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. https://doi.org/10.18653/v1/2020.acl-main.408 ERASER : A benchmark to evaluate rationalized NLP models . In Proceedings of the 58th Annual Meeting of the A...

    2020 doi

  3. [11]

    Dai, et al

    Gemini, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, et al. 2025. https://arxiv.org/abs/2312.11805 Gemini: A family of highly capable multimodal models . Preprint, arXiv:2312.11805

    2025 arXiv

  4. [12]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  5. [13]

    Sarthak Jain and Byron C. Wallace. 2019. https://arxiv.org/abs/1902.10186 Attention is not explanation . Preprint, arXiv:1902.10186

    2019 arXiv

  6. [14]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  7. [15]

    Yeo Wei Jie, Ranjan Satapathy, Rick Goh, and Erik Cambria. 2024. How Interpretable are Reasoning Explanations from Prompting Large Language Models? In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2148--2164

    2024

  8. [16]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. https://openreview.net/forum?id=e2TBb5y0yFf Large language models are zero-shot reasoners . In Advances in Neural Information Processing Systems

    2022

  9. [17]

    Satyapriya Krishna, Jiaqi Ma, Dylan Slack, Asma Ghandeharioun, Sameer Singh, and Himabindu Lakkaraju. 2023. https://arxiv.org/abs/2305.11426 Post hoc explanations of language models can improve language models . Preprint, arXiv:2305.11426

    2023 arXiv

  10. [18]

    Nicholas Kroeger, Dan Ley, Satyapriya Krishna, Chirag Agarwal, and Himabindu Lakkaraju. 2023. https://openreview.net/forum?id=mAzhEP9jPv Are large language models post hoc explainers? In XAI in Action: Past, Present, and Future Applications

    2023

  11. [19]

    Bowman, and Ethan Perez

    Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish...

    2023 arXiv

  12. [20]

    Scott Lundberg and Su-In Lee. 2017. https://arxiv.org/abs/1705.07874 A unified approach to interpreting model predictions . Preprint, arXiv:1705.07874

    2017 arXiv

  13. [21]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. https://openrev...

    2023

  14. [22]

    Bodhisattwa Prasad Majumder, Oana Camburu, Thomas Lukasiewicz, and Julian Mcauley. 2022. https://proceedings.mlr.press/v162/majumder22a.html Knowledge-grounded self-rationalization via extractive and natural language explanations . In Proceedings of the 39th International Conf...

    2022

  15. [23]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, et al. 2024. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2024 arXiv

  16. [24]

    Letitia Parcalabescu and Anette Frank. 2024. On measuring faithfulness or self-consistency of natural language explanations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6048--6089

    2024

  17. [25]

    Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le ...

    2025 arXiv

  18. [26]

    Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. https://doi.org/10.18653/v1/P19-1487 Explain yourself! leveraging language models for commonsense reasoning . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistic...

    2019 doi

  19. [27]

    Sagnik Ray Choudhury, Pepa Atanasova, and Isabelle Augenstein. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.783 Explaining interactions between text spans . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12709--12730, Sing...

    2023 doi

  20. [28]

    Nils Reimers and Iryna Gurevych. 2019. https://arxiv.org/abs/1908.10084 Sentence-bert: Sentence embeddings using siamese bert-networks . Preprint, arXiv:1908.10084

    2019 arXiv

  21. [29]

    Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. https://arxiv.org/abs/2303.11366 Reflexion: Language agents with verbal reinforcement learning . Preprint, arXiv:2303.11366

    2023 arXiv

  22. [30]

    Noah Siegel, Oana-Maria Camburu, Nicolas Heess, and Maria Perez-Ortiz. 2024. https://doi.org/10.18653/v1/2024.acl-short.49 The probabilities also matter: A more faithful metric for faithfulness of free-text explanations in large language models . In Proceedings of the 62nd Ann...

    2024 doi

  23. [31]

    Jingyi Sun, Pepa Atanasova, and Isabelle Augenstein. 2025. https://doi.org/10.18653/v1/2025.naacl-long.530 Evaluating input feature explanations through a unified diagnostic evaluation framework . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of ...

    2025 doi

  24. [32]

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. https://arxiv.org/abs/1703.01365 Axiomatic attribution for deep networks . Preprint, arXiv:1703.01365

    2017 arXiv

  25. [33]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. https://openreview.net/forum?id=bzs4uPLXvi Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting . In Thirty-seventh Conference on Neural Information Proces...

    2023

  26. [34]

    Cunxiang Wang, Shuailong Liang, Yili Jin, Yilong Wang, Xiaodan Zhu, and Yue Zhang. 2020. https://doi.org/10.18653/v1/2020.semeval-1.39 S em E val-2020 task 4: Commonsense validation and explanation . In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 307--...

    2020 doi

  27. [35]

    PeiFeng Wang, Aaron Chan, Filip Ilievski, Muhao Chen, and Xiang Ren. 2023 a . https://openreview.net/forum?id=WBXbRs63oVu PINTO : Faithful language reasoning using prompt-generated rationales . In The Eleventh International Conference on Learning Representations

    2023

  28. [36]

    Qianli Wang, Tatiana Anikina, Nils Feldhus, Simon Ostermann, Sebastian Möller, and Vera Schmitt. 2024. https://arxiv.org/abs/2409.07123 Cross-refine: Improving natural language explanation generation by learning in tandem . Preprint, arXiv:2409.07123

    2024 arXiv

  29. [37]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 b . https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Confer...

    2023

  30. [38]

    Chi, Quoc V Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. https://openreview.net/forum?id=_VjQlMeSB_J Chain of thought prompting elicits reasoning in large language models . In Advances in Neural Information Proc...

    2022

  31. [39]

    Sarah Wiegreffe, Ana Marasović, and Noah A. Smith. 2022. https://arxiv.org/abs/2010.12762 Measuring association between labels and free-text rationales . Preprint, arXiv:2010.12762

    2022 arXiv

  32. [40]

    Shuzhou Yuan, Jingyi Sun, Ran Zhang, Michael Färber, Steffen Eger, Pepa Atanasova, and Isabelle Augenstein. 2025. Graph-guided textual explanation generation framework. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), Suzhou, C...

    2025

  33. [41]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.