Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Counterfactual text generation becomes more reliable when the edited words come from BERT feature attributions instead of the LLM itself, and automatically verified label-flipping demonstrations improve few-shot prompting.

desk verdict A sensible, clearly described framework for feature-importance-guided counterfactual generation, but the headline result is undermined by the same-LLM evaluation protocol and the lack of uncertainty estimates. read the letter →

arxiv 2501.00777 v3 pith:HULIEMLY submitted 2025-01-01 cs.CL cs.LG

Qianli Wang , Nils Feldhus , Simon Ostermann , Luis Felipe Villa-Arenas , Sebastian Möller , Vera Schmitt This is my paper · ORCID
classification cs.CLcs.LG
keywords counterfactualgenerationfeatureattributionfew-shotpromptinglabelflipverificationfaithfulnesstextclassificationlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that counterfactual examples—minimal edits that flip a classifier's prediction—are generated more reliably when the words an LLM edits are supplied by BERT feature attributions rather than by the LLM's own guesses, and that feeding verified examples back as few-shot demonstrations improves quality further. It introduces ZeroCF for the zero-shot step and FitCF for the verified-demonstration framework. The central payoffs are practical: no human-written counterfactuals are needed, and the faithfulness of the attribution method becomes a usable predictor of generation quality. The paper reports that FitCF consistently beats three state-of-the-art baselines on AG News and SST2, with the number of demonstrations the most influential component.

What carries the argument

The carrying object is the verified demonstration. ZeroCF first extracts the top-attributed words from a BERT classifier (using Gradient, Integrated Gradients, LIME, or SHAP) and prompts an LLM to edit the input in a zero-shot setting. FitCF then uses the same BERT model as a label-flip verifier: a candidate counterfactual is kept as a demonstration only if the BERT prediction on the edited text differs from the prediction on the original. These verified input–counterfactual pairs are selected from clusters of sentence embeddings and inserted into the final few-shot prompt, so the LLM generates the target counterfactual by analogy to examples that are known to flip the label. The verifier is the gate that prevents flawed zero-shot outputs from poisoning the demonstrations.

What would settle it

Recompute Soft Label Flip Rate on the same AG News and SST2 instances using an independent classifier—a different BERT variant or human annotators—rather than the LLM that generated the counterfactuals. If FitCF's flip-rate advantage over FIZLE shrinks or vanishes, the reported gains are at least partly self-consistency rather than genuine label flips.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that feature attribution scores from a fine-tuned BERT model can serve as trustworthy edit cues for counterfactual generation, and that label-flip verification turns those zero-shot outputs into demonstrations that make few-shot prompting stronger. The authors report that ZeroCF often outperforms the zero-shot baseline FIZLE, and that FitCF consistently outperforms POLYJUICE, BAE, FIZLE, and ZeroCF on both datasets across all evaluated LLMs. Ablations show each of the three components—important words, demonstrations, and verification—contributes, with the number of demonstrations dominating. The paper also reports a strong correlation between faithfulness of the attribution scores and quality of the generated counterfactuals, up to Kendall's $\tau = 1$ for sufficiency on AG News, which it frames as a useful finding for future work.

Load-bearing premise

The load-bearing premise is that a label flip counted by the same LLM that wrote the counterfactual is a true label flip; if generator and evaluator share the same errors, high flip rates can just show the model agrees with itself.

Editorial extensions

If this is right

  • If correct, counterfactual datasets for classification can be assembled automatically, since verified LLM edits replace human-crafted demonstrations.
  • If correct, attribution faithfulness becomes a practical selection criterion: choosing a more faithful method such as LIME or SHAP should yield better counterfactuals.
  • If correct, the number of verified demonstrations is the highest-leverage tuning knob in the pipeline, more so than the exact attribution method.
  • If correct, the framework transfers beyond BERT because the paper's design only requires any effective classifier as the attribution and verification backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the faithfulness–quality correlation generalizes, a cheap pre-filter emerges: evaluate attribution faithfulness on a small validation set and pick the most faithful method, saving the compute of generating counterfactuals for every candidate attribution method.
  • Editorial inference: because the Soft Label Flip Rate uses the same LLM for generation and verification, some of the reported advantage may be self-agreement; a held-out classifier or human judgment would separate genuine flips from consistent errors.
  • Editorial inference: the verified-demonstration loop should transfer beyond text classification—question answering and open-ended generation would need a different correctness oracle, but the verify-then-demonstrate pattern is the same.
  • Editorial inference: using a single BERT model for both attribution and verification means the findings are anchored to that explainee; retraining or swapping the target classifier could change which words matter and therefore which counterfactuals pass verification.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ZeroCF, a zero-shot counterfactual generation method that feeds top words from BERT-based feature attribution (Gradient, IG, LIME, SHAP) into an LLM prompt, and FitCF, a few-shot extension that automatically constructs demonstrations by running ZeroCF on cluster-selected examples, filtering them through BERT label-flip verification, and then using these verified pairs as in-context demonstrations. The authors evaluate on AG News and SST2 with three LLMs (Llama3-8B, Qwen2.5-32B, Qwen2.5-72B) against baselines POLYJUICE, BAE, and FIZLE, using SLFR, perplexity, and text similarity. They also ablate the three FitCF components (important words, number of demonstrations, label-flip verification) and report a correlation between faithfulness of attribution scores and counterfactual quality.

Significance. If the empirical claims hold, the paper makes two useful contributions: it provides a practical pipeline that replaces hallucination-prone LLM-generated important words with BERT-derived attribution words, and it shows that automatically verified demonstrations can improve few-shot counterfactual generation. The study is systematic: it covers multiple attribution methods, multiple LLMs, two datasets, and includes ablation studies. The code is publicly available, and the paper is transparent about many experimental details. However, the core quantitative claim—that FitCF 'consistently outperforms' the baselines—rests almost entirely on SLFR as computed by the same LLM that generated the counterfactuals. The manuscript's own appendix acknowledges this design, and the justification provided (similar aggregate accuracy between LLM and BERT) is not sufficient to establish the validity of the metric. The correlation analysis in Section 6.4 also relies on only four attribution methods, making the strong correlation claim fragile. These issues are central to the paper's conclusions and require substantive revision.

major comments (3)
  1. [§5.1] The primary evaluation metric, Soft Label Flip Rate, is computed by the same LLM that generated the counterfactuals: Section 5.1 states 'we use the same LLM for both counterfactual generation and classification,' and Appendix G confirms this. The only justification given, in Appendix G.1, is that the LLM and BERT have similar classification accuracy on the two datasets, so 'the tendency of SLFR will be consistent.' This is not a valid inference: similar aggregate accuracy does not imply per-instance agreement, and per-instance divergence is especially likely for adversarially edited texts near the decision boundary. The risk becomes concrete in Table 1: with Llama3-8B on SST2, FitCF attains 100% SLFR for IG, LIME, and SHAP, even though the same model's zero-shot accuracy on SST2 is only 89.75%. A perfect flip rate in a binary task is implausible as a measure of genuine label change and instead suggests that the generator is exploiting its own decision boundary and the evaluator is confirming self-consistency. Since the headline claim that FitCF 'consistently outperforms' FIZLE and other baselines is based on SLFR, this is a load-bearing issue. The authors should report flip rates using the BERT classifier (the target model used for feature attribution and verification) or another held-out classifier, and discuss any discrepancies.
  2. [Table 1] No error bars, confidence intervals, or significance tests accompany the automatic evaluation results. The SLFR values are reported in increments of 0.5%, suggesting a test set of about 200 instances per dataset; under that assumption, differences of a few percentage points (e.g., 94.5% vs. 95.5% for AG News with Llama3-8B) are well within sampling noise. The claim that FitCF 'consistently outperforms' all baselines across all settings needs a quantitative basis; otherwise, the observed differences may not be reliable. The authors should provide at least bootstrap confidence intervals or multiple runs with different seeds, and preferably a paired significance test for the main FitCF-vs-FIZLE and FitCF-vs-ZeroCF comparisons.
  3. [§6.4] The correlation analysis between faithfulness of attribution scores and counterfactual quality uses only four attribution methods (Gradient, IG, LIME, SHAP) per dataset and model. With n=4, a Kendall's tau of 1.0—reported for AG News—can arise by chance with non-negligible probability, and the manuscript gives no significance value or confidence interval for these correlations. Figure 3 appears to show only the tau values, not the underlying scatterplots, so the reader cannot assess whether the correlation is driven by a single outlier or is stable across the data. The authors should either include more attribution methods, bootstrap over instances to obtain confidence intervals, or at minimum explicitly state that the finding is based on four points and should be interpreted as preliminary.
minor comments (5)
  1. [§3.2] The relationship between the number of clusters k and the number of demonstrations ℓ is not defined in the method section; Section 5.2.2 states that ℓ = 2k, but this choice should be introduced in §3.2 where the demonstration sampling is described.
  2. [Table 1] The caption says 'Bold faced values indicate for each approach, which feature importance method is the best performing'—but the table also includes baselines (POLYJUICE, BAE) that have no feature importance method. The caption should clarify how bolding applies to those rows.
  3. [§6.2] The arrow notation in the ablation tables is confusing: for example, '↑0.63' for TS in Table 2 appears to indicate an increase, but since lower TS is better, an upward arrow is not clearly an improvement. The convention is stated in §6.2, but it is easy to misread; consider using explicit 'better/worse' labels or separate improvement/deterioration columns.
  4. [Appendix F.2] The sentence 'Table 9 shows inference time for ZEROCF and FITCF using Llama3-8B, Qwen2.5-32B and Qwen2.5-32B across AG News and SST2 datasets' repeats 'Qwen2.5-32B'; the second occurrence should be 'Qwen2.5-72B'.
  5. [Appendix G.1] The sentence 'as selected LLMs and BERT perform similarly on the two datasets, we can assume that the tendency of SLFR will be consistent' is a non-sequitur; the authors should explicitly acknowledge that this is an assumption, not a derivation, and discuss its limitations.

Circularity Check

1 steps flagged · score 4.0 of 10

The headline SLFR metric is computed by the same LLM that generates the counterfactuals, so the main flip-rate comparisons partly measure the generator's self-consistency rather than an independent label change; no other circular step is load-bearing.

  1. self definitional [The circular step is located in Section 5.1 (Soft Label Flip Rate definition), Appendix G, and Appendix G.1.]
    "Section 5.1: 'Note that we use the same LLM for both counterfactual generation and classification.' Appendix G: 'We use the same LLM to serve as both the flip label verifier and the counterfactual generator (§5.1).' Appendix G.1: 'as selected LLMs and BERT perform similarly on the two datasets (Table 6), we can assume that the tendency of SLFR will be consistent, and our conclusion should remain unchanged.'"

    SLFR is the central metric used to claim that FITCF 'consistently outperforms' all baselines. It is defined as the rate at which the predicted label changes after perturbation, but the predicted label is supplied by the same LLM that produced the perturbation in Eq. (3). The evaluation prompt in Appendix G.2 asks that generator itself whether the original and edited instances have different classifications. Consequently, a high SLFR can be achieved by the model approving its own edits, i.e., self-consistency, rather than by a verified label flip on an independent classifier or on the BERT model used elsewhere in the pipeline.

full rationale

FitCF is an empirical system, not a formal derivation, and most of its pipeline is not circular: feature attribution scores come from FERRET applied to a fine-tuned BERT, important words are extracted and passed into an LLM prompt, demonstrations are filtered by a separate BERT label-flip check, and final counterfactuals are additionally scored by GPT-2 perplexity and Levenshtein textual similarity. The comparisons against POLYJUICE, BAE, and FIZLE provide external grounding, and the paper's self-citations (Wang et al. 2024a,b) are contextual rather than load-bearing. The one material circularity is evaluative: the primary flip-rate metric, SLFR, is computed by the same LLM that generates the counterfactuals, as stated in Section 5.1 and Appendix G, and the Appendix G.1 justification by aggregate accuracy similarity does not establish per-instance independence. The correlation analysis in Section 6.4 is not itself circular, because faithfulness is measured independently with FERRET metrics while counterfactual quality also includes external PPL and TS; the observed correlation is an empirical finding rather than a definitional identity. Because the central 'consistently outperforming' claim rests heavily on SLFR, the paper is partially circular, but it retains independent content through the external baselines and the non-SLFR metrics.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new theoretical entities or fitted constants; its free parameters are hyperparameters of the pipeline, and its assumptions are empirical claims about the validity of its components. The same-LLM evaluation assumption is the most consequential because it directly supports the central comparison.

free parameters (3)
  • number of clusters k = 5 for AG News, 4 for SST2
    Hand-chosen in Section 3.2 and used to select diverse demonstrations via k-means on SBERT embeddings. The reported performance depends on this clustering granularity.
  • number of demonstrations l = 10 for AG News, 8 for SST2
    Set to twice the number of clusters (l = 2k) in Section 3.2. The ablation in Section 5.2.2 shows this is the most influential component, so the headline results are sensitive to this hand-set value.
  • top-k important words threshold = not specified
    Section 3.1 selects top attributed words but does not state the exact count used in the prompts. The post-processing rules for subwords and special tokens affect which words are passed to the LLM.
assumptions (4)
  • domain assumption A BERT model fine-tuned on the target dataset reliably represents the classifier whose behavior is explained and verified.
    Section 3.1 and Section 3.3 use BERT for both feature attribution and label flip verification. If BERT's predictions diverge from the LLM's behavior, the important words and verified demonstrations may not transfer.
  • domain assumption FERRET-based faithfulness metrics (comprehensiveness, sufficiency, Leave-One-Out) are valid proxies for feature attribution quality.
    Section 5.3 builds the correlation analysis on these metrics. The conclusion that faithfulness is strongly correlated with counterfactual quality inherits the validity of these proxies.
  • domain assumption Soft Label Flip Rate computed by the same LLM used for generation is a valid evaluation metric.
    Section 5.1 and Appendix G state that the same LLM both generates counterfactuals and predicts labels for SLFR. This is the weakest evaluation assumption because it can inflate flip rates through self-consistency.
  • domain assumption SBERT embeddings plus k-means produce diverse and representative demonstrations.
    Section 3.2 assumes that centroid-near examples cover the input space well enough to make few-shot demonstrations useful. The paper provides clustering visualizations but no quantitative diversity check.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation." pith.science (2026). https://pith.science/paper/HULIEMLY

@misc{pith2026250100777,
  author       = {Pith},
  title        = {Pith review of: FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HULIEMLY}},
  note         = {Machine review of arXiv:2501.00777}
}
read the original abstract

Counterfactual examples are widely used in natural language processing (NLP) as valuable data to improve models, and in explainable artificial intelligence (XAI) to understand model behavior. The automated generation of counterfactual examples remains a challenging task even for large language models (LLMs), despite their impressive performance on many tasks. In this paper, we first introduce ZeroCF, a faithful approach for leveraging important words derived from feature attribution methods to generate counterfactual examples in a zero-shot setting. Second, we present a new framework, FitCF, which further verifies aforementioned counterfactuals by label flip verification and then inserts them as demonstrations for few-shot prompting, outperforming two state-of-the-art baselines. Through ablation studies, we identify the importance of each of FitCF's core components in improving the quality of counterfactuals, as assessed through flip rate, perplexity, and similarity measures. Furthermore, we show the effectiveness of LIME and Integrated Gradients as backbone attribution methods for FitCF and find that the number of demonstrations has the largest effect on performance. Finally, we reveal a strong correlation between the faithfulness of feature attribution scores and the quality of generated counterfactuals, which we hope will serve as an important finding for future research in this direction.

Figures

Figures reproduced from arXiv: 2501.00777 by the authors.

Figure 1
Figure 1. Given an instance x from the AG News dataset classified as “sci/tech”, our ZEROCF approach generates few-shot examples, whose important words are determined by LIME for a BERT model. FITCF then generates a counterfactual x˜ on this basis. The edits to original instance x are highlighted in orange, yielding x˜ which is classified as “business”. et al., 2022) are widely used in XAI and NLP. Ap￾plications include creat… view at source ↗
Figure 2
Figure 2. The upper part of the figure illustrates how counterfactuals are generated by [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A Kendall’s tau (τ ) that quantifies the degree of correspondence between the ranking of generated counterfactuals’ quality and the ranking of feature attribution evaluation results is reported. tion methods for counterfactual example genera￾tion in a zero-shot setting. Building on this, we proposed FITCF, a framework that automatically constructs high-quality demonstrations using ZE￾ROCF, eliminating the need for h… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of clustering in AG News and SST2, where stars denote cluster centroids. [original input] {original_input_1} [edit input] {edit_input_1} ... [original input] {input_text} [edit input] B Detailed Information of Deployed BERT [PITH_FULL_IMAGE:figures/full_…
Figure 6
Figure 6. Figure 6: Example instances from AG News and SST2. AG News SST2 ZEROCF FITCF ZEROCF FITCF Llama3-8B 8h 13h 2h 5h Qwen2.5-32B 9h 17h 7h 12h Qwen2.5-72B 38h 47h 8h 16h [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 5
Figure 5. Figure 5: Label distribution of AG News and SST2 [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Truth or Twist? Optimal Model Selection for Reliable Label Flipping Evaluation in LLM-based Counterfactuals

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Independent, non-fine-tuned judge models align most closely with human label-flip judgments, yet all automated judges fall well short of human evaluation.

Reference graph

Works this paper leans on

52 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  2. [2]

    Shengnan An, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Jian-Guang Lou, and Dongmei Zhang. 2023. https://doi.org/10.18653/v1/2023.acl-long.618 How do in-context examples affect compositional generalization? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11027--11052, Toronto, Can...

    doi:10.18653/v1/2023.acl-long.618 2023

  3. [3]

    Giuseppe Attanasio, Eliana Pastor, Chiara Di Bonaventura, and Debora Nozza. 2023. ferret: a framework for benchmarking explainers on transformers. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics

    work page 2023

  4. [4]

    Tanja Baeumel, Soniya Vijayakumar, Josef van Genabith, Guenter Neumann, and Simon Ostermann. 2023. https://doi.org/10.18653/v1/2023.blackboxnlp-1.20 Investigating the encoding of words in BERT ' s neurons using feature textualization . In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 261--270, Singa...

    work page doi:10.18653/v1/2023.blackboxnlp-1.20 2023

  5. [5]

    Milan Bhan, Jean-noel Vittaut, Nicolas Chesneau, and Marie-jeanne Lesot. 2023 a . https://doi.org/10.18653/v1/2023.trustnlp-1.19 Enhancing textual counterfactual explanation intelligibility through counterfactual feature importance . In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 221--231, Toronto, Can...

    doi:10.18653/v1/2023.trustnlp-1.19 2023

  6. [6]

    Milan Bhan, Jean-No \"e l Vittaut, Nicolas Chesneau, and Marie-Jeanne Lesot. 2023 b . Tigtec: Token importance guided text counterfactuals. In Machine Learning and Knowledge Discovery in Databases: Research Track, pages 496--512, Cham. Springer Nature Switzerland

    work page 2023

  7. [7]

    Amrita Bhattacharjee, Raha Moraffah, Joshua Garland, and Huan Liu. 2024 a . https://doi.org/10.1109/BigData62323.2024.10825537 Zero-shot LLM-guided Counterfactual Generation: A Case Study on NLP Model Evaluation . In 2024 IEEE International Conference on Big Data (BigData), pages 1243--1248, Los Alamitos, CA, USA. IEEE Computer Society

    arXiv 2024

  8. [8]

    Amrita Bhattacharjee, Raha Moraffah, Joshua Garland, and Huan Liu. 2024 b . Towards llm-guided causal explainability for black-box text classifiers. In AAAI 2024 Workshop on Responsible Language Models, Vancouver, BC, Canada

    2024

Show all 52 references
  1. [9]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffr...

    2020 arXiv

  2. [10]

    Zeming Chen, Qiyue Gao, Antoine Bosselut, Ashish Sabharwal, and Kyle Richardson. 2023. https://doi.org/10.18653/v1/2023.acl-long.302 DISCO : Distilling counterfactuals with large language models . In Proceedings of the 61st Annual Meeting of the Association for Computational L...

    2023 doi

  3. [11]

    Zahra Dehghanighobadi, Asja Fischer, and Muhammad Bilal Zafar. 2025. https://arxiv.org/abs/2502.18156 Can llms explain themselves counterfactually? Preprint, arXiv:2502.18156

    2025

  4. [12]

    Julien Delaunay, Luis Galárraga, and Christine Largouët. 2024. https://arxiv.org/abs/2404.14943 Does it make sense to explain a black box with another black box? Preprint, arXiv:2404.14943

    2024 arXiv

  5. [13]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  6. [14]

    Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. https://doi.org/10.18653/v1/2020.acl-main.408 ERASER : A benchmark to evaluate rationalized NLP models . In Proceedings of the 58th Annual Meeting of the A...

    2020 doi

  7. [15]

    Marharyta Domnich, Julius Välja, Rasmus Moorits Veski, Giacomo Magnifico, Kadi Tulver, Eduard Barbu, and Raul Vicente. 2025. https://doi.org/10.1609/aaai.v39i15.33791 Towards unifying evaluation of counterfactual explanations: Leveraging large language models for human-centric...

    2025 doi

  8. [16]

    Angela Fan, Mike Lewis, and Yann Dauphin. 2018. https://doi.org/10.18653/v1/P18-1082 Hierarchical neural story generation . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889--898, Melbourne, Australia....

    2018 doi

  9. [17]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. https://openreview.net/forum?id=tcbBPnfwxS OPTQ : Accurate quantization for generative pre-trained transformers . In The Eleventh International Conference on Learning Representations

    2023

  10. [18]

    Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A

    Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qian...

    2020 doi

  11. [19]

    Siddhant Garg and Goutham Ramakrishnan. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.498 BAE : BERT -based adversarial examples for text classification . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6174--6181, O...

    2020 doi

  12. [20]

    Yingqiang Ge, Shuchang Liu, Zelong Li, Shuyuan Xu, Shijie Geng, Yunqi Li, Juntao Tan, Fei Sun, and Yongfeng Zhang. 2021. https://arxiv.org/abs/2109.01962 Counterfactual evaluation for explainable ai . Preprint, arXiv:2109.01962

    2021 arXiv

  13. [21]

    Gilad Gressel, Niranjan Hegde, Archana Sreekumar, Rishikumar Radhakrishnan, Kalyani Harikumar, Anjali S., and Krishnashree Achuthan. 2023. https://arxiv.org/abs/2106.14815 Feature importance guided attack: A model agnostic adversarial attack . Preprint, arXiv:2106.14815

    2023 arXiv

  14. [22]

    Sarthak Jain and Byron C. Wallace. 2019. https://doi.org/10.18653/v1/N19-1357 A ttention is not E xplanation . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and...

    2019 doi

  15. [23]

    Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2020. https://openreview.net/forum?id=Sklgs0NFvr Learning the difference that makes a difference with counterfactually-augmented data . In International Conference on Learning Representations

    2020

  16. [24]

    Satyapriya Krishna, Jiaqi Ma, Dylan Slack, Asma Ghandeharioun, Sameer Singh, and Himabindu Lakkaraju. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/ce65173b994cf7c925c71b482ee14a8d-Paper-Conference.pdf Post hoc explanations of language models can improve lan...

    2023

  17. [25]

    Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, and Min Zhang. 2023. https://arxiv.org/abs/2311.03731 A survey of large language models attribution . Preprint, arXiv:2311.03731

    2023 arXiv

  18. [26]

    Yongqi Li, Mayi Xu, Xin Miao, Shen Zhou, and Tieyun Qian. 2024. https://aclanthology.org/2024.lrec-main.1156 Prompting large language models for counterfactual generation: An empirical study . In Proceedings of the 2024 Joint International Conference on Computational Linguisti...

    2024

  19. [27]

    Scott M Lundberg and Su-In Lee. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf A unified approach to interpreting model predictions . In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc

    2017

  20. [28]

    Nishtha Madaan, Inkit Padhi, Naveen Panwar, and Diptikalyan Saha. 2021. Generate your counterfactuals: Towards controlled counterfactual generation for text. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13516--13524

    2021

  21. [29]

    Andreas Madsen, Siva Reddy, and Sarath Chandar. 2022. https://doi.org/10.1145/3546577 Post-hoc interpretability for neural nlp: A survey . ACM Comput. Surv., 55(8)

    2022 doi

  22. [30]

    Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1--38

    2019

  23. [31]

    o rg Schl \

    Van Bach Nguyen, Paul Youssef, Christin Seifert, and J \"o rg Schl \"o tterer. 2024. https://aclanthology.org/2024.findings-emnlp.870 LLM s for generating and evaluating counterfactuals: A comprehensive study . In Findings of the Association for Computational Linguistics: EMNL...

    2024

  24. [32]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  25. [33]

    Marco Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. https://doi.org/10.18653/v1/N16-3020 `` why should I trust you? '' : Explaining the predictions of any classifier . In Proceedings of the 2016 Conference of the North A merican Chapter of the Association for Computational...

    2016 doi

  26. [34]

    Alexis Ross, Ana Marasovi \'c , and Matthew Peters. 2021. https://doi.org/10.18653/v1/2021.findings-acl.336 Explaining NLP models via minimal contrastive editing ( M i CE ) . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3840--3852, Onlin...

    2021 doi

  27. [35]

    Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. https://doi.org/10.18653/v1/2022.naacl-main.191 Learning to retrieve prompts for in-context learning . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: ...

    2022 doi

  28. [36]

    Rachneet Sachdeva, Martin Tutek, and Iryna Gurevych. 2024. https://aclanthology.org/2024.eacl-long.113 CAT f OOD : Counterfactual augmented training for improving out-of-domain performance and calibration . In Proceedings of the 18th Conference of the European Chapter of the A...

    2024

  29. [37]

    Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, and Arianna Bisazza. 2023. https://doi.org/10.18653/v1/2023.acl-demo.40 Inseq: An interpretability toolkit for sequence generation models . In Proceedings of the 61st Annual Meeting of the Associa...

    2023 doi

  30. [38]

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop at International Conference on Learning Representations

    2014

  31. [39]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. https://www.aclweb.org/anthology/D13-1170 Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 Conferenc...

    2013

  32. [40]

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 3319–3328. JMLR.org

    2017

  33. [41]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  34. [42]

    Guerreiro, and Andr \'e Martins

    Marcos Treviso, Alexis Ross, Nuno M. Guerreiro, and Andr \'e Martins. 2023. https://doi.org/10.18653/v1/2023.acl-long.842 CREST : A joint framework for rationalization and counterfactual text generation . In Proceedings of the 61st Annual Meeting of the Association for Computa...

    2023 doi

  35. [43]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. https://openreview.net/forum?id=bzs4uPLXvi Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting . In Thirty-seventh Conference on Neural Information Proces...

    2023

  36. [44]

    Qianli Wang, Tatiana Anikina, Nils Feldhus, Josef Genabith, Leonhard Hennig, and Sebastian M \"o ller. 2024 a . https://doi.org/10.18653/v1/2024.hcinlp-1.9 LLMC heckup: Conversational examination of large language models via interpretability tools and self-explanations . In Pr...

    2024 doi

  37. [45]

    Qianli Wang, Tatiana Anikina, Nils Feldhus, Simon Ostermann, and Sebastian M \"o ller. 2024 b . https://aclanthology.org/2024.findings-emnlp.76 C o XQL : A dataset for parsing explanation requests in conversational XAI systems . In Findings of the Association for Computational...

    2024

  38. [46]

    Yequan Wang, Jiawen Deng, Aixin Sun, and Xuying Meng. 2023. https://arxiv.org/abs/2210.05892 Perplexity from plm is unreliable for evaluating text quality . Preprint, arXiv:2210.05892

    2023 arXiv

  39. [47]

    Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhan Qin, and Kui Ren. 2021. Feature importance-aware transferable adversarial attacks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7639--7648

    2021

  40. [48]

    Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2021. https://doi.org/10.18653/v1/2021.acl-long.523 Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models . In Proceedings of the 59th Annual Meeting of the Association for Comp...

    2021 doi

  41. [49]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. https://proceedings.neurips.cc/paper_files/paper/2015/file/250cf8b51c773f3f8dc8b4be867a9a02-Paper.pdf Character-level convolutional networks for text classification . In Advances in Neural Information Processing Systems, volume 28...

    2015

  42. [50]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2023. https://openreview.net/forum?id=5NTt8GFjUHkr Automatic chain of thought prompting in large language models . In The Eleventh International Conference on Learning Representations

    2023

  43. [51]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.