Pith. sign in

REVIEW 2 major objections 4 minor 2 cited by

LLMs asked for minimal counterfactual edits usually fail to flip their own predictions

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 19:07 UTC pith:P5LKYUFP

load-bearing objection The paper makes a real, measurable contribution—showing a validity-minimality trade-off in self-generated counterfactuals—but the headline numbers lack repeated-sample variance, so the quantitative strength is not fully pinned down. the 2 major comments →

arxiv 2509.09396 v1 pith:P5LKYUFP submitted 2025-09-11 cs.LG cs.AIcs.CL

LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations

Harry Mayne , Ryan Othniel Kearns , Yushi Yang , Andrew M. Bean , Eoin Delaney , Chris Russell , Adam Mahdi This is my paper
classification cs.LG cs.AIcs.CL
keywords self-generated counterfactual explanationsLLM self-explanationvalidityminimalitydecision boundaryself-predictionexplainabilitytabular data
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that large language models cannot reliably explain their own predictions with counterfactual explanations: what would need to change for the model to give a different answer. Across seven models and three tabular datasets, unconstrained counterfactuals almost always flip the prediction but overshoot the decision boundary by a large margin, while explicitly requested minimal counterfactuals usually change too little to flip anything. The only way to get both validity and minimality is essentially by luck, and no model achieves both criteria consistently. The authors argue this makes self-generated counterfactual explanations an unreliable, potentially misleading explainability tool, particularly when deployed in high-stakes settings like clinical decision support.

Core claim

Using datasets that enumerate every combination of a small set of discrete features, the authors compute the exact decision boundary of each LLM and the distance to the true minimal counterfactual for every input. They then ask the model to explain its own prediction. Unconstrained SCEs are valid in roughly 90–100% of cases but land on average 0.06–0.50 of the dataset span beyond the boundary; minimal SCEs are valid only 10–67% of the time, and when valid are often exactly at the boundary. The paper's central claim is that LLMs do not know their own decision boundaries: they cannot both locate the boundary and stop at it, and even when explicitly instructed to self-predict, performance barel

What carries the argument

The evaluation is made possible by complete tabular datasets: every combination of two to four discrete features is enumerated, so the model's predictions over the whole input space can be elicited once and used as a lookup table for validity. Minimality is measured with Gower's distance, a normalized per-feature distance the paper shows LLMs can compute in context (reasoning models score 98–100% on a multiple-choice distance task). The key mechanism is the measured distance between the chosen counterfactual and the true nearest point that flips the prediction: excess distance for unconstrained prompts, and validity failure for minimal prompts.

Load-bearing premise

The results rest on assuming that a model's fresh-context answer to the revised input is the correct ground truth for 'valid', and that synthetic tabular datasets of two to four discrete features with Gower's distance capture how counterfactual explanations are actually used; if either assumption fails, the measured trade-off may not hold in real deployments.

What would settle it

Run the same experiment and find a model that, when explicitly asked for a minimal counterfactual, achieves both high validity (say >80%) and small excess distance (say <0.05) across all three datasets; or show that on a non-tabular version of the same task with the same input space the trade-off disappears.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Users asking an LLM for a counterfactual explanation without constraints will usually get a valid-sounding answer that reveals almost nothing about where the model's decision boundary actually sits.
  • Asking for a minimal counterfactual is worse: in many cases the proposed edit does not change the model's answer at all, so the explanation is actively wrong.
  • High-stakes deployments that rely on self-generated counterfactuals for actionability (e.g., 'lower blood pressure to 110') cannot assume the proposed change is necessary or sufficient.
  • The failure is not explained by inability to compute distances or by inconsistent decision boundaries, and prompting models to self-predict inside their reasoning traces does not fix it.
  • Any claim that LLMs can faithfully self-explain should be tested for minimality as well as validity, since the two trade off.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the trade-off is a general property of LLM self-explanation, then counterfactual-exclamation benchmarks that only check validity overstate model capability; future benchmarks should include distance-to-boundary targets.
  • The finding that explicit self-prediction instructions barely improve behavior suggests that standard training objectives never reward accurate self-modelling, implying that new training objectives—not just prompting—would be needed to make self-explanations trustworthy.
  • A testable extension: fine-tune a model with a reward that requires it to predict its own fresh-context answer before proposing an edit, then re-measure validity and minimality; the paper's self-prediction ablation is prompt-only, so it leaves this open.
  • A natural next experiment would apply the same complete-dataset methodology to non-tabular inputs (e.g., short texts from a template family) to see whether the trade-off persists when edits are lexical rather than numeric.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies self-generated counterfactual explanations (SCEs) in LLMs. Using three synthetic tabular datasets (income, house prices, heart disease) with fully enumerated discrete feature spaces, the authors elicit each model's predictions over all instances to define a decision boundary, then ask the model to produce counterfactual inputs that would flip its own prediction. They compare two prompting settings: unconstrained counterfactual generation and explicit minimality instructions. Validity is measured by whether the SCE crosses the model's own decision boundary (via a lookup table from the first prediction stage), and minimality is measured by excess Gower distance from the exact minimal counterfactual. Across 7 models (Gemma 2, Llama 3.3, two DeepSeek-R1 distills, Claude Sonnet 3.7, GPT-4.1, o3) and 3 datasets, they find that unconstrained SCEs are valid but far from minimal, while minimal prompted SCEs are often invalid but more minimal when valid. This validity–minimality trade-off is robust to temperature, alternative distance functions, and 20 prompt perturbations. The paper additionally examines limiting factors and finds that models can operationalize Gower distance but do not spontaneously self-predict, and forcing self-prediction via a six-step prompt does not substantially improve performance.

Significance. If correct, this is a valuable negative result for LLM explainability. The study's design is a strength: exhaustive discrete input spaces make the minimal counterfactual exactly computable, the evaluation covers a broad array of models and settings, the robustness checks (distance functions, prompt variations, temperature) are extensive, and the code is released. The claim is falsifiable and the protocol is transparent. The paper provides concrete evidence that self-generated counterfactual explanations are not reliable for understanding LLM decision boundaries, which has direct implications for high-stakes deployment. The main weakness is the lack of uncertainty quantification for headline validity/minimality numbers, particularly for the stochastic o3 model, but the qualitative trade-off appears consistent across many independent conditions.

major comments (2)
  1. [§3.5, §A.3, Table 6] Validity and minimal-counterfactual labels for o3 (and for temperature-1 runs) are derived from a single sample of the model's prediction function; no confidence intervals, repeated draws, or bootstrap estimates are reported for the headline validity and excess-distance numbers. The decision-boundary consistency check in §4.5 covers only Llama 3.3 70B on the income dataset. Since the quantitative strength of the trade-off (e.g., validity dropping to 10–40% in the minimal setting) is a core message, please add repeated-seed or bootstrap uncertainty intervals for at least o3 and a subset of other models, and consider extending the boundary-consistency analysis to additional models/datasets. This would not change the qualitative conclusion but would make the reported magnitudes trustworthy.
  2. [§4.5, §E] The claim that 'models do not spontaneously engage in self-prediction' is supported by only 30 reasoning traces from DeepSeek-R1 70B, and the self-prediction prompt ablation is also performed on a single model without reporting variance (validity 21.44%→23.34%, ED 0.025→0.033). Given that chain-of-thought faithfulness is debated (as the paper acknowledges), this evidence is thin for a mechanism-level conclusion. I recommend extending the ablation to at least one additional frontier model (e.g., o3 or GPT-4.1) and reporting the number of traces reviewed, or tempering the mechanistic claims in Section 6.
minor comments (4)
  1. [§3.5] Exact Match (EM) is defined as the proportion of instances where the model exactly identifies a minimal counterfactual. Since the discrete input spaces can contain multiple minimal counterfactuals with equal Gower distance, please clarify how ties are handled. If the model selects any one of several minima, EM may be underestimated.
  2. [§A.3] The lookup-table method for validity is efficient, but for stochastic models (o3 at temperature 1) it represents a single draw from the prediction distribution. Please state explicitly that no repeated fresh-context verification was performed for these models, or add a supplementary verification on a subset.
  3. [§4.1, Figure 3] The caption says three SCEs are 'randomly selected,' but it is unclear whether they are selected from specific starting points or uniformly across the dataset. Please specify the selection procedure. The arrow lines in Figure 3 are also somewhat hard to follow.
  4. [§5] The related-work paragraph on SCEs would benefit from a direct comparison of this work's validity measurement with Dehghanighobadi et al. (2025), which also uses fresh context windows. This would better position the novelty of the minimality evaluation.

Circularity Check

0 steps flagged

No significant circularity: the study is an empirical evaluation, not a derivation, and its ground truth is the model's own decision boundary by design.

full rationale

The paper does not derive a mathematical claim from assumptions that secretly encode the conclusion. It is a measurement study: SCEs are generated by LLMs and then scored for validity and minimality. Validity is defined as the SCE flipping the model's own prediction when re-evaluated in a new context window (Section 3.5). This is the natural target concept for 'self-generated counterfactual explanations', not a hidden circular input. Minimality is defined relative to a minimal counterfactual computed from the same decision boundary, which is again the intended reference for measuring excess distance. The appendix states that first-stage predictions are used as a lookup table (Section A.3), but this is a deterministic shortcut for temperature-0 models and is explicitly supplemented by decision-boundary consistency checks and temperature-1 experiments. There is no fitted parameter that is later renamed as a prediction, no self-citation that carries the argument, and no uniqueness theorem imported from the authors' prior work. The robustness analyses with alternative distance functions, prompt perturbations, and temperature provide independent evidence beyond any single definitional choice. The paper even identifies limitations transparently, such as the choice of Gower's distance and the synthetic discrete datasets. Overall, the central claim is an empirical finding about LLM behaviour, not a result that reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

No free parameters are fitted; the study is observational. The central assumptions are the stability of the decision boundary, the choice of Gower distance, and the representativeness of synthetic tabular tasks for natural-language self-explanation.

axioms (3)
  • domain assumption The model's prediction function is treated as a stable decision boundary; re-evaluating the SCE in a new context window yields the counterfactual outcome.
    Section 3.5 defines validity this way. Section 4.5 tests boundary consistency and finds 91.4% of invalid SCEs remain invalid across 50 prompt perturbations, so it is defended but still an assumption.
  • domain assumption Gower's distance is the distance metric for defining minimality; alternative metrics (L1, L2, cosine) are tested but the primary metric is Gower.
    Sections A.4 and 4.4. If a different metric were used, the exact minimal counterfactual and excess distance would change, though robustness analysis shows the trade-off persists.
  • domain assumption The fully enumerated discrete feature spaces (2 to 4 features) are a representative environment for SCE behavior; models that fail on these simple tasks are expected to fail on harder ones.
    Section 7 acknowledges this is an inference, not proven. This is load-bearing for the real-world implication.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations." pith.science (2026). https://pith.science/paper/P5LKYUFP

@misc{pith2026250909396,
  author       = {Pith},
  title        = {Pith review of: LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P5LKYUFP}},
  note         = {Machine review of arXiv:2509.09396}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

To collaborate effectively with humans, language models must be able to explain their decisions in natural language. We study a specific type of self-explanation: self-generated counterfactual explanations (SCEs), where a model explains its prediction by modifying the input such that it would have predicted a different outcome. We evaluate whether LLMs can produce SCEs that are valid, achieving the intended outcome, and minimal, modifying the input no more than necessary. When asked to generate counterfactuals, we find that LLMs typically produce SCEs that are valid, but far from minimal, offering little insight into their decision-making behaviour. Worryingly, when asked to generate minimal counterfactuals, LLMs typically make excessively small edits that fail to change predictions. The observed validity-minimality trade-off is consistent across several LLMs, datasets, and evaluation settings. Our findings suggest that SCEs are, at best, an ineffective explainability tool and, at worst, can provide misleading insights into model behaviour. Proposals to deploy LLMs in high-stakes settings must consider the impact of unreliable self-explanations on downstream decision-making. Our code is available at https://github.com/HarryMayne/SCEs.

Figures

Figures reproduced from arXiv: 2509.09396 by Adam Mahdi, Andrew M. Bean, Chris Russell, Eoin Delaney, Harry Mayne, Ryan Othniel Kearns, Yushi Yang.

Figure 1
Figure 1. Figure 1: Study design. A. We evaluate models in tabular data, binary classification tasks. The model predictions form a decision boundary across the input space. B. We ask models to provide self-generated counterfactual explanations (SCEs) for their predictions. SCEs are valid when they cross the decision boundary (here, red → blue) and are minimal if they are close to the dashed instance at the decision boundary. … view at source ↗
Figure 2
Figure 2. Figure 2: SCE validity and minimality for income (A), house prices (B), and heart disease (C) datasets. In the unconstrained prompting setting (purple), models typically provide valid SCEs, but they are far from minimal. In the minimal prompting setting (green), validity is notably lower, but, conditional on being valid, minimality is much better. No model can consistently satisfy both criteria across all three data… view at source ↗
Figure 3
Figure 3. Figure 3: Llama 3.3 70B’s behaviour on the income dataset. The three SCEs shown are randomly selected and are representative of the model’s general behaviour. In the unconstrained prompting setting, all three SCEs cross the decision boundary and are therefore valid. However, the model makes an excessively large, non-minimal edit to the original input on the far right of the figure. In the minimal prompting setting, … view at source ↗
Figure 5
Figure 5. Figure 5: shows that the model’s predictions are stable, except in a narrow region around the bound￾ary. Importantly, we find that 91.4% of the original invalid SCEs remain invalid across all 50 versions of the decision boundary. This suggests that model failures cannot be attributed to decision boundary inconsistency. Operationalising distance To select the closest valid SCE to the original input, models need to be… view at source ↗
Figure 6
Figure 6. Figure 6: DeepSeek-R1 70B can operationalise distance, but does not attempt self-prediction. DeepSeek-R1 70B leverages Gower’s Distance perfectly to compare changes, but does not appear to realise it is predicting its own behaviour. The reasoning traces are taken from an SCE generated on the house price dataset under the minimal prompting setting. measuring minimality requires identifying the min￾imal counterfactual… view at source ↗
Figure 7
Figure 7. Figure 7: Initial prompts to elicit model predictions for income (A), house prices (B), and heart disease (C) datasets. These predictions are used as the first stage in the experimental pipeline. These prompts are templates where specific feature values are imputed based on the tabular data instance [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Unconstrained prompts for income (A), house prices (B), and heart disease (C) datasets. In our first experiment, we ask each model to provide a counterfactual input with no constraints on minimality. Below, we provide the prompts for each dataset. In all prompts, the value {possible_values} refers to the complete list of ordinal values as provided in [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Minimal prompts for income (A), house prices (B), and heart disease (C) datasets. In our minimal prompt setting, we again ask each model to provide a counterfactual input, though we now instruct the model to provide the smallest change necessary to flip its decision. We also instruct the model to calculate minimality using Gower’s Distance, and provide a brief explanation of Gower’s Distance in each prompt… view at source ↗
Figure 10
Figure 10. Figure 10: The distribution of SCEs for Llama 3.3 70B (A) and o3 (B). This shows the full distribution of SCEs for every starting point in the income dataset. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Prompt given to o3 to generate the prompt perturbations. The value {original_prompt} shows where the original prompt is inserted into the template. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: SCE validity and minimality in the income (A), house prices (B), and heart disease (C) datasets at temperature 1.0. In the unconstrained prompting setting, models are able to provide valid SCEs, however, they are far from minimal. In the minimal prompting setting, validity drops sharply. No model can satisfy both criteria. Orange regions indicate the direction of increasing validity and minimality. Datase… view at source ↗
Figure 13
Figure 13. Figure 13: Instructions passed to OpenAI’s o3 model and six of the resulting prompt perturbations. The values {in_red} indicate where the numerical values are inserted into the templates. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Prompt for the operationalising distance experiment. The values {in_red} indicate where the sampled numerical values are inserted into the template. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Prompt for self-prediction ablation experiment. We provide the models with a metacognition-inspired prompt with a six-step plan to generate a minimal valid counterfactual. This prompt encourages the LLMs to make explicit self-predictions for every candidate SCE they consider. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: A reasoning trace from DeepSeek-R1 70B following step-by-step self-prediction instructions. We prompt the model using the template from §E. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: A full reasoning trace from DeepSeek-R1 70B. Excerpts from the model response suggest that, despite the question being clearly framed as a self-explanation task (‘revise the original house data such that your answer to the question. . . ’), DeepSeek-R1 70B does not interpret the problem as such. The model instead appeals to general intuitions (e.g., ‘More bathrooms usually mean a higher price’) and never … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Decision Potential Surface: A Theoretical and Practical Approximation of Large Language Model Decision Boundary

    cs.LG 2025-09 unverdicted novelty 7.0

    Defines Decision Potential Surface (DPS) whose zero isohypse equals an LLM decision boundary and supplies a K-sample approximation algorithm with derived upper bounds on absolute, expected, and concentration errors.

  2. Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

    cs.AI 2026-07 accept novelty 5.0

    Passing medical exams does not make LLMs safe for autonomous triage: they fail to seek missing red flags, and current benchmarks do not test that.

Reference graph

Works this paper leans on

47 extracted references · 1 canonical work pages · cited by 2 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Anthropic . 2025. Claude 3.7 sonnet and claude code. https://www.anthropic.com/news/claude-3-7-sonnet

    2025

  4. [4]

    Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. 2025. https://arxiv.org/abs/2505.08775 Healthbench: Evaluating large language models towards improved human health . Preprint, arXiv:2505.08775

    Pith/arXiv arXiv 2025

  5. [5]

    Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi

    Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. 2025. https://arxiv.org/abs/2503.11926 Monitoring reasoning models for misbehavior and the risks of promoting obfuscation . Preprint, arXiv:2503.11926

    Pith/arXiv arXiv 2025

  6. [6]

    Fazl Barez, Tung-Yu Wu, Iván Arcuschin, Michael Lan, Vincent Wang, Noah Siegel, Nicolas Collignon, Clement Neo, Isabelle Lee, Alasdair Paren, Adel Bibi, Robert Trager, Damiano Fornasiere, John Yan, Yanai Elazar, and Yoshua Bengio. 2025. https://arxiv.org/abs/2025.02v1 Chain-of-thought is not explainability . Preprint, alphaXiv:2025.02v1. AlphaXiv preprint...

    2025

  7. [7]

    Bowman, Jan Leike, Jared Kaplan, and Ethan Perez

    Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez. 2025. https://arxiv.org/abs/2505.05410 Reasoning models don't always say what they think . Preprint, arXiv:2505.05410

    Pith/arXiv arXiv 2025

  8. [8]

    Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, and Kathleen Mckeown. 2024. https://proceedings.mlr.press/v235/chen24bl.html Do models explain themselves? C ounterfactual simulatability of natural language explanations . In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings o...

    2024

  9. [9]

    Cruz, Moritz Hardt, and Celestine Mendler-D\" u nner

    Andr\' e F. Cruz, Moritz Hardt, and Celestine Mendler-D\" u nner. 2024. https://openreview.net/forum?id=qrZxL3Bto9 Evaluating language models as risk scores . In The Thirty-eighth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2024

  10. [10]

    DeepSeek-AI . 2025. DeepSeek-R1-Distill-Llama-70B . https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B. MIT license; accessed 2025-05-17

    2025

  11. [11]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. https://arxiv.org/abs/2501.12948 DeepSeek-R1 : Incentivizing reasoning capability in LLM s via reinforcement l...

    Pith/arXiv arXiv 2025

  12. [12]

    Zahra Dehghanighobadi, Asja Fischer, and Muhammad Bilal Zafar. 2025. https://arxiv.org/abs/2502.18156 Can LLMs explain themselves counterfactually? Preprint, arXiv:2502.18156

    arXiv 2025

  13. [13]

    Elson, Rif A

    Scott Emmons, Erik Jenner, David K. Elson, Rif A. Saurous, Senthooran Rajamanoharan, Heng Chen, Irhum Shafkat, and Rohin Shah. 2025. https://arxiv.org/abs/2507.05246 When chain of thought is necessary, language models struggle to evade monitors . Preprint, arXiv:2507.05246

    Pith/arXiv arXiv 2025

  14. [14]

    Gemma Team , Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179 others. 2024. https://arxiv.org/abs/2408.00118 Gemma 2:...

    Pith/arXiv arXiv 2024

  15. [15]

    John C Gower. 1971. A general coefficient of similarity and some of its properties. Biometrics, pages 857--871

    1971

  16. [16]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. https://arxiv.org/abs/2407.21783 The Llama 3...

    Pith/arXiv arXiv 2024

  17. [17]

    Andras Janosi, William Steinbrunn, Matthias Pfisterer, and Robert Detrano. 1989. Heart Disease . UCI Machine Learning Repository. DOI : https://doi.org/10.24432/C52P4X

    doi:10.24432/c52p4x 1989

  18. [18]

    Mark T Keane and Barry Smyth. 2020. Good counterfactuals and where to find them: A case-based technique for generating counterfactuals for explainable AI ( XAI ). In Case-Based Reasoning Research and Development: 28th International Conference, ICCBR 2020, Salamanca, Spain, June 8--12, 2020, Proceedings 28, pages 163--178. Springer

    2020

  19. [19]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with PagedAttention . In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

  20. [20]

    Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, and 11 others. 2023. https://arxiv.org/abs/2307.13702 Measurin...

    Pith/arXiv arXiv 2023

  21. [21]

    David Lewis. 1973. Counterfactuals. Basil Blackwell Ltd

    1973

  22. [22]

    Zachary C Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31--57

    2018

  23. [23]

    Gabrielle Kaili-May Liu, Gal Yona, Avi Caciularu, Idan Szpektor, Tim G. J. Rudner, and Arman Cohan. 2025. https://arxiv.org/abs/2505.24858 Metafaith: Faithful natural language uncertainty expression in LLMs . Preprint, arXiv:2505.24858

    arXiv 2025

  24. [24]

    Andreas Madsen, Sarath Chandar, and Siva Reddy. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.19 Are self-explanations from large language models faithful? In Findings of the Association for Computational Linguistics: ACL 2024, pages 295--337, Bangkok, Thailand. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-acl.19 2024

  25. [25]

    Andreas Madsen, Himabindu Lakkaraju, Siva Reddy, and Sarath Chandar. 2024 b . https://arxiv.org/abs/2405.05386 Interpretability needs a new paradigm . Preprint, arXiv:2405.05386

    Pith/arXiv arXiv 2024

  26. [26]

    Stephen McAleese and Mark Keane. 2024. https://arxiv.org/abs/2411.02643 A comparative analysis of counterfactual explanation methods for text classifiers . Preprint, arXiv:2411.02643

    Pith/arXiv arXiv 2024

  27. [27]

    Tim Miller. 2019. https://doi.org/10.1016/j.artint.2018.07.007 Explanation in artificial intelligence: Insights from the social sciences . Artificial Intelligence, 267:1--38

    doi:10.1016/j.artint.2018.07.007 2019

  28. [28]

    Van Bach Nguyen, Paul Youssef, Christin Seifert, and Jörg Schlötterer. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.870 LLMs for generating and evaluating counterfactuals: A comprehensive study . In Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 14809--14824. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-emnlp.870 2024

  29. [29]

    Open AI . 2024. Learning to reason with LLM s. https://openai.com/index/learning-to-reason-with-llms/. Accessed: 2025-05-20

    2024

  30. [30]

    OpenAI . 2025 a . Introducing GPT-4.1 in the API . https://openai.com/index/gpt-4-1/

    2025

  31. [31]

    OpenAI . 2025 b . Introducing OpenAI o3 and o4-mini. https://openai.com/index/introducing-o3-and-o4-mini/. Accessed: 2025-05-19

    2025

  32. [32]

    Letitia Parcalabescu and Anette Frank. 2024. https://doi.org/10.18653/v1/2024.acl-long.329 On measuring faithfulness or self-consistency of natural language explanations . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6048--6089, Bangkok, Thailand. Association for Computational Li...

    doi:10.18653/v1/2024.acl-long.329 2024

  33. [33]

    János Podani. 1999. https://doi.org/10.2307/1224438 Extending Gower 's general coefficient of similarity to ordinal characters . TAXON, 48(2):331--340

    work page doi:10.2307/1224438 1999

  34. [34]

    Premakumar, Michael Vaiana, Florin Pop, Judd Rosenblatt, Diogo Schwerz de Lucena, Kirsten Ziman, and Michael S

    Vickram N. Premakumar, Michael Vaiana, Florin Pop, Judd Rosenblatt, Diogo Schwerz de Lucena, Kirsten Ziman, and Michael S. A. Graziano. 2024. https://arxiv.org/abs/2407.10188 Unexpected benefits of self-modeling in neural systems . Preprint, arXiv:2407.10188

    Pith/arXiv arXiv 2024

  35. [35]

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf Improving language understanding by generative pre‐training . Technical report, Open AI

    2018

  36. [36]

    Korbinian Randl, John Pavlopoulos, Aron Henriksson, and Tony Lindgren. 2025. https://doi.org/10.1007/s10994-025-06838-6 Mind the gap: from plausible to valid self-explanations in large language models . Machine Learning, 114:220

    doi:10.1007/s10994-025-06838-6 2025

  37. [37]

    Noah Siegel, Oana-Maria Camburu, Nicolas Heess, and Maria Perez-Ortiz. 2024. https://doi.org/10.18653/v1/2024.acl-short.49 The probabilities also matter: A more faithful metric for faithfulness of free-text explanations in large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pa...

    doi:10.18653/v1/2024.acl-short.49 2024

  38. [38]

    Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet : Masked and permuted pre-training for language understanding. Advances in Neural Information Processing Systems, 33:16857--16867

    2020

  39. [39]

    Mark Steyvers and Megan A. K. Peters. 2025. https://arxiv.org/abs/2504.14045 Metacognition and uncertainty communication in humans and large language models . Preprint, arXiv:2504.14045

    arXiv 2025

  40. [40]

    Sara Mahdavi, Christopher Semturs, and 7 others

    Tao Tu, Mike Schaekermann, Anil Palepu, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Yong Cheng, Elahe Vedadi, Nenad Tomasev, Shekoofeh Azizi, Karan Singhal, Le Hou, Albert Webson, Kavita Kulkarni, S. Sara Mahdavi, Christopher Semturs, and 7 others. 2025. https://doi.org/10.1038/s41586-025-08866-7 Towards conversational dia...

    doi:10.1038/s41586-025-08866-7 2025

  41. [41]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. https://openreview.net/forum?id=bzs4uPLXvi Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  42. [42]

    Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR . Harv. JL & Tech., 31:841

    2017

  43. [43]

    Yongjie Wang, Xiaoqi Qiu, Yu Yue, Xu Guo, Zhiwei Zeng, Yuhong Feng, and Zhiqi Shen. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.276 A survey on natural language counterfactual generation . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4798--4818, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-emnlp.276 2024

  44. [44]

    Sarah Wiegreffe and Ana Marasovic. 2021. https://openreview.net/forum?id=ogNcxJn32BZ Teach me to explain: A review of datasets for explainable natural language processing . In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)

    2021

  45. [45]

    Sarah Wiegreffe, Ana Marasovi \'c , and Noah A. Smith. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.804 M easuring association between labels and free-text rationales . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10266--10284, Online and Punta Cana, Dominican Republic. Association for Computational...

    doi:10.18653/v1/2021.emnlp-main.804 2021

  46. [46]

    Tongshuang Sherry Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel S Weld. 2021. Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models. In Proc. Association for Computational Linguistics (ACL)

    2021

  47. [47]

    M Yasser. 2022. https://www.kaggle.com/datasets/yasserh/housing-prices-dataset Housing prices dataset . Accessed: 2025-05-17

    2022

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.