Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Information

Antoni B. Chan; Xin Huang

REVIEW 3 major objections 6 minor 48 references

Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Information

T0 review · 3 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that attribution faithfulness comparisons are unfair unless the expected number of retained words is fixed, and that its π-Soft-NC/NS metrics plus the Grad-ELLM attribution method provide that fair comparison and a leading

desk verdict π-Soft-NC/NS is a useful fix to Soft-NC/NS, but as written the fairness guarantee doesn't cover the baselines, and the empirical support for Grad-ELLM is thinner than the conclusions claim. read the letter →

arxiv 2601.03089 v2 pith:BCPIXLYJ submitted 2026-01-06 cs.CL cs.AIcs.LG

Xin Huang , Antoni B. Chan This is my paper

classification cs.CLcs.AIcs.LG

keywords faithfulnessevaluationinputattributiondecoder-onlyLLMssoftperturbationcompletenesssufficiencygradient-basedexplanationattention

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that existing soft-perturbation faithfulness metrics for LLM attributions are internally biased: because attribution methods produce scores on different scales, soft perturbation retains different expected numbers of words, so a method can look more faithful simply by keeping more of the input. To fix this, it proposes π-Soft-NC and π-Soft-NS, which rescale each method's scores so the expected retention probability equals a preset value π, then compare methods at the same retention level. It also introduces Grad-ELLM, an attribution method for decoder-only LLMs that combines gradient-derived channel weights with attention-derived token weights at each layer to produce per-token heatmaps at each generation step. Experiments across classification and open-generation tasks with two open model families indicate Grad-ELLM is competitive and often best under the new metrics. The significance is a fairer standard for deciding whether one explanation method genuinely outperforms another.

What carries the argument

The load-bearing identity is the logit decomposition through the transformer stack, l_t ≈ Σ_k LP(o_t^(k)) + LP(z_t^(N)), which lets the next-token logit be written as a weighted combination of attention value vectors. Channel importance w_c is the gradient of the logit with respect to the attention output, token importance λ_i is a 0-1-normalized attention similarity used in place of softmax, and the per-token attribution is H_i = ReLU(Σ_c w_c λ_i v_ic). The fairness machinery is the α-transformation: for a target retention probability π, solve for α so that (1/m)Σ s_i^α = π; because soft perturbation masks each token with e_i ~ Bernoulli(s_i^α), every method then has the same expected numbe

What would settle it

Take two attribution methods whose scores are identical up to a positive scale factor, so their token orderings agree and only magnitudes differ. If π-Soft-NC/NS is fair, they must receive identical scores at every π after α-matching, because nothing except scale distinguished them; any divergence would show the transformation still leaves retention differences or distorts the metric. Equivalently, feed a method such as Integrated Gradients or DeepLIFT — which outputs values outside [0,1] — into the stated s^α formula; if the Bernoulli means do not equal π, the fairness guarantee fails for tha

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that faithfulness comparisons for LLM attributions conflate two things: how well scores reflect importance, and how much text the scores happen to keep under random masking. The proposed π-Soft-NC/NS removes the second confound by transforming scores s_i to s_i^α with α chosen so the average of s_i^α equals π, making the expected number of retained words identical across methods before Hellinger-distance shifts in output distributions are measured. The second contribution, Grad-ELLM, is claimed to be a faithful decoder-only attribution method: it linearly decomposes the next-token logit through the transformer layers and assigns weight to ea

Load-bearing premise

The load-bearing premise is that every attribution method's scores can be rescaled into valid Bernoulli retention probabilities in [0,1] with a prescribed mean π — via s_i^α — without changing what the faithfulness metric measures; for methods that output negative or larger-than-one scores, no normalization is specified, so the fairness guarantee is currently undefined.

Editorial extensions

If this is right

If π-Soft-NC/NS is adopted, any two attribution methods can be compared on equal information-loss terms, and the area under the π-curve becomes a single faithfulness summary across retention levels.
Methods whose raw scores are systematically large lose the advantage they previously gained by retaining more words, so rankings can change once the confound is removed.
The same π-matching can be inserted into existing Soft-NC/NS implementations with a bisection search for α, requiring no model changes.
For decoder-only LLMs, Grad-ELLM produces per-token, per-generation-step attributions from a small number of forward/backward passes, making faithfulness evaluation feasible for long open-ended generations.
Because π-Soft-NC/NS treats attribution scores as calibrated probabilities, it reframes the goal of attribution from ordering tokens to producing well-calibrated importance distributions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Since s^α is monotone for scores in [0,1], the transformation preserves token rankings while changing gaps between scores; π-Soft-NC/NS therefore rewards distribution shape, not just ordering, so a complete benchmark should report distribution-based and order-based metrics side by side.
A direct testable extension is to apply the same π-matching to insertion/deletion curves, whose stepwise masking currently suffers from the same unequal-retention confound.
Random attribution is a strong baseline under the new metrics on several settings; a useful threshold for any proposed method is beating random at every π, since methods that only edge out random under controlled retention are likely capturing weak signal.
For proprietary models without gradient access, the π-controlled metric could test whether attention-only or activation-only approximations preserve the faithfulness ranking, revealing how much gradient information actually matters.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things worth knowing: the core fix in π-Soft-NC/NS is a genuine improvement over Soft-NC/NS, and Grad-ELLM is a clean adaptation of Grad-ECLIP to decoder-only LLMs. But the fairness guarantee is not actually operationalized for the baselines, and the results don't support “consistently outperforms.”

What's new and what works: matching the expected retention probability across methods directly addresses a real bias in Soft-NC/NS. The alpha-transformation trick is simple and the intuition is right—comparing methods at the same retained-information level is the right thing to do. Grad-ELLM is a reasonable extension of Grad-ECLIP, and the paper gives enough detail to reproduce it. The qualitative examples are helpful, and the comparison to original Soft-NC/NS in Table 2 makes the problem concrete.

Now the soft spots. The load-bearing issue is the [0,1] normalization. The paper says “assuming scores si∈[0,1)” and then uses s_i^α, but it never tells you how a raw Integrated Gradients or DeepLIFT score gets into [0,1). Those methods produce unbounded, often negative values. Without a specified normalization, the π-Soft-NC/NS values in Table 1 are not well-defined and the central fairness claim doesn't actually cover the experiments. The stress-test note is right about this, and it's not a minor omission—the whole method depends on it. There's also the infeasibility problem: if any score is exactly 0 or 1, then 0^α and 1^α can't be moved, so some target π values have no solution. The paper never says how infeasible cases are handled.

Even if you grant the normalization, the empirical evidence is shaky. The random baseline is within 0.02–0.05 of Grad-ELLM on several Llama π-Soft-NS rows, and random beats Grad-ELLM on Mistral π-Soft-NC. That's not “superior faithfulness”; that's a metric that isn't separating signal from noise. There are no error bars or significance tests—three perturbation samples per text isn't enough to claim “significantly outperforms.” The abstract and conclusion also contradict each other: the abstract says no dominant method under π-Soft-NS, but the tables show Grad-ELLM first there on both models, while the conclusion claims consistent superiority across all metrics.

Minor point: Eq. (5) ignores MLP sublayers and LayerNorm in the logit decomposition. That's a common linearization for gradient-based explanations, so I'd treat it as a limitation, not a fatal flaw.

Who this is for: people working on faithfulness evaluation for LLM attributions. The metric idea is worth engaging with even if this version isn't fully baked. It deserves a serious referee, but I'd expect major revision: specify the normalization, report how infeasible π cases are handled, add error bars, and temper the conclusions.

Recommendation: send to peer review, but the referee should push hard on the normalization protocol and the random-baseline problem.

Referee Report

3 major / 6 minor

Summary. The paper makes two contributions. First, it proposes Grad-ELLM, an attribution method for decoder-only LLMs that aggregates gradient-derived channel weights with loosened attention-map token weights across transformer layers. Second, it proposes π-Soft-NC and π-Soft-NS, modifications of the Soft-NC/NS faithfulness metrics that rescale attribution scores with a power transform so that the expected number of retained words equals a target probability π. The authors evaluate on Llama and Mistral over sentiment classification and open-generation datasets, reporting AUC curves and claiming that Grad-ELLM consistently outperforms attention, gradient, and perturbation baselines.

Significance. The motivation is sound: Table 2 shows clearly that raw Soft-NC/NS comparisons are confounded by differences in expected retention rate, and a metric that equalizes retention is a useful contribution if the transformation is well defined. Grad-ELLM is also a genuinely architecture-aware method, and the qualitative examples suggest reasonable behavior. However, the current manuscript leaves a load-bearing precondition of the metric unstated and applies it to methods whose scores are not in the required range. The empirical support for 'consistently outperforms' is also weaker than claimed: the margins over a random baseline are small on π-Soft-NS, and a random baseline is the best method on π-Soft-NC for Mistral. No variance or significance tests are reported. These issues are fixable in revision, so I do not see them as fatal to the general idea.

major comments (3)

[§3.3, Eq. (20)] The α-transformation is defined only for scores s_i ∈ [0,1), but none of the gradient-based baselines in Table 1 (Integrated Gradients, DeepLIFT, Input×Gradient, Saliency) produce scores in that range; their raw outputs are unbounded and usually signed. No normalization step is specified before Eq. (20) is applied. Consequently, the reported π-Soft-NC/NS values for these methods are undefined as written, and the claimed fairness guarantee—'both soft-perturbed text will now have the same expected number of retained words'—does not actually hold for the comparisons that are made. The same issue affects Grad-ELLM itself, since Eq. (13) is a ReLU of a weighted sum of value vectors and is not guaranteed to lie in [0,1). The authors must specify a normalization protocol that maps every method's raw scores into the feasible domain, analyze feasibility for targets π when scores contain exact 0 o
[§3.1–§3.2, Eq. (5)] The derivation decomposes the logit as l_t ≈ Σ_k LP(o_t^(k)) + LP(z_t^(N)), using the relation z^(k−1) = o^(k−1) + z^(k). Standard transformer decoder blocks also contain LayerNorm and MLP sublayers; these are omitted from this residual-stream relation. As written, Eq. (5) does not describe the actual forward computation of Llama or Mistral. The authors should either include the full block decomposition or justify dropping the MLP and normalization terms as part of the linear approximation. Without this, the architectural grounding of Grad-ELLM is incomplete.
[§4.4, Table 1, Conclusion] The statement that Grad-ELLM 'consistently outperforms baselines' is stronger than the data. On Mistral π-Soft-NC, Random has the highest AUC on four of five datasets (IMDb, SST2, TellMeWhy, WikiBio), with Grad-ELLM second. On π-Soft-NS, Grad-ELLM's average AUC advantage over Random is only 0.022 for Llama (0.401 vs 0.379) and 0.050 for Mistral (0.383 vs 0.333). No variance, confidence intervals, or significance tests are reported, despite the use of only three soft-perturbation samples per π. The authors should add error bars or statistical tests and temper the abstract/conclusion claims accordingly.

minor comments (6)

[§3.1] The layer indexing is confusing: z^(N) is called token embeddings and z^(0) final hidden states, while Eq. (5) sums over k = 0,...,N−1. Please clarify the layer-order convention explicitly.
[§3.2, Eqs. (7)–(9)] The 'linear approximation' f̃ with w = ∂f/∂o_t is simply the first-order Taylor expansion; the optimization formulation is tautological. Stating this directly would be clearer.
[Table 1] AUC values for π-Soft-NC can exceed 1 (e.g., the BoolQ rows). The paper should state explicitly whether this is intended because Soft-NC is a ratio that can exceed 1.
[References] The citation 'Anonymous (2024)' is an anonymous under-review submission. Please replace it with a stable reference or remove it.
[Abstract / text] There are several typos: 'faithfulneses' in the abstract; 'Grad-CLIP' in §3.2 should be 'Grad-ECLIP'; 'loss' vs 'loosen' in the Limitations section.
[Table 2] E[R] values are reported without sequence length, so the reader cannot assess the retention rates across datasets. Please include average input length or normalize E[R].

Circularity Check

0 steps flagged · score 2.0 of 10

No definitional circularity: π-Soft-NC/NS is a designed calibration invariant, and Grad-ELLM's superiority is empirically benchmarked against external baselines.

full rationale

The central metric claim—that π-Soft-NC/NS fixes unequal retention—is true by construction: Eq. (20) solves for α so that (1/m)Σ s_i^α = π, making the expected number of retained words mπ identical across methods. This is a designed invariant, not a fitted input renamed as a prediction, and α is not fitted to the outcome metric. The attribution method Grad-ELLM reuses the authors' earlier Grad-ECLIP gradient/attention aggregation (Zhao et al., 2024a; 2025), but the paper's headline results are comparisons against external baselines (IG, DeepLIFT, Saliency, Value Zeroing, etc.) on Llama/Mistral, so the self-citation is not load-bearing. No uniqueness theorem is invoked, and no known result is merely renamed. The notable weakness is technical rather than circular: Eq. (20) states "Assuming scores si ∈ [0,1)" and uses s_i^α, but standard baselines such as Integrated Gradients, DeepLIFT, and Input×Gradient produce negative or unbounded scores, and the paper specifies no normalization before applying the transformation; exact 0/1 values can also make some target π infeasible. This undermines reproducibility of Table 1 for those baselines but does not make any derivation equivalent to its own inputs, so it does not raise the circularity score. Score 2 reflects the minor self-citation of the Grad-ECLIP construction, not load-bearing circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper relies on calibration transforms and architectural approximations rather than new physical entities. The main adjustable knobs are the α exponent, the number of aggregated layers, and the loosened attention normalization; the [0,1] assumption is load-bearing but undocumented.

free parameters (3)

α (power exponent in π-Soft-NC/NS) = Solved per text/method so that (1/m)Σ s_i^α = π
Chosen by bisection to force the expected retention probability; not fitted to a prediction target, but an unexamined calibration knob that shapes all metric values.
Number of aggregated layers in Grad-ELLM = 32 (default)
Set to all layers 'to get the most comprehensive information'; no ablation is given, and this choice directly affects the attribution heatmaps.
Loosened attention normalization Φ = 0–1 normalization of q·k similarities replacing softmax
Heuristic to de-sparsify attention maps; no theoretical justification is provided, and it changes which tokens receive attribution weight.

assumptions (4)

ad hoc to paper The transformer output logit can be approximated as a sum of per-layer attention contributions: LP(z_t^(0)) ≈ Σ_k LP(o_t^(k)) + LP(z_t^(N)) (Eq. 5).
Invoked in §3.2 without accounting for MLP sublayers or LayerNorm; the residual-stream linearity is an approximation that is load-bearing for Grad-ELLM.
ad hoc to paper 0–1 normalized attention similarities are suitable token importance weights.
Section 3.2 replaces softmax(q·k_i/√d) with Φ(q·k_i) to 'reveal unattended tokens', but no evidence is given that this improves faithfulness.
domain assumption Attribution scores s_i can be treated as Bernoulli keep probabilities after the α-transform.
Required by Eq. (14) and Eq. (20), but raw scores from many baselines are not in [0,1]; the normalization protocol is omitted.
domain assumption Hellinger distance and zero-baseline relative effect (Soft-NC/NS) are valid faithfulness measures.
Inherited from Zhao and Shan (2024); the paper modifies the retention probability but does not revisit the underlying distributional assumptions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Information." pith.science (2026). https://pith.science/paper/BCPIXLYJ

@misc{pith2026260103089,
  author       = {Pith},
  title        = {Pith review of: Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BCPIXLYJ}},
  note         = {Machine review of arXiv:2601.03089}
}

abstract

Large Language Models (LLMs) are increasingly evaluated with input attribution methods, yet comparing such explanations remains challenging. Existing soft-perturbation faithfulness metrics, such as Soft-NC and Soft-NS, can conflate attribution quality with the number of words retained during perturbation: attribution methods with larger average scores may keep more words and therefore obtain inflated scores. To address this issue, we propose $\pi$-Soft-NC and $\pi$-Soft-NS, an evaluation framework that compares attribution methods under the same expected retaining probability, thus controlling the number of retained words. We further introduce Grad-ELLM, a gradient-based attribution method tailored to autoregressive decoder-only LLMs, which combines gradient-derived channel importance with attention-derived token importance at each decoding step. Experiments on classification and open-generation tasks with Llama and Mistral show that Grad-ELLM achieves strong comprehensiveness-oriented faithfulness under $\pi$-Soft-NC, while there is no dominant method under $\pi$-Soft-NS. Our evaluation metric serves as a rigorous framework to compare XAI methods for LLMs, which will support progress in the field.

Figures

Figures reproduced from arXiv: 2601.03089 by the authors.

**Figure 2.** Proposed π-Soft-NS/NC vs. Retaining Probability on IMDb for different XAI methods with Llama. (a) Llama: AUC of π-Soft-NS/NC (↑) Methods Dataset Attn DL I×G IG LGA Rnd Sal VZ Ours AUC π-Soft-NS ↑ IMDb 0.526 0.401 0.384 0.412 0.384 0.557 0.420 0.398 0.572 SST2 0.361 0.493 0.493 0.526 0.493 0.549 0.503 0.325 0.563 BoolQ 0.121 0.326 0.317 0.266 0.316 0.334 0.324 0.116 0.370 TellMeWhy 0.225 0.313 0.299 0.286 0.299 0.283… view at source ↗

**Figure 3.** Explanations for negative sample from IMDb. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Explanations for positive sample from SST2. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Explanations for negative sample from IMDb. The importance of words are visualized by the degree [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 6.** Figure 6: Explanations for positive sample from SST2. The importance of words are visualized by the degree [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 2 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Samira Abnar and Willem Zuidema. 2020. https://doi.org/10.18653/v1/2020.acl-main.385 Quantifying attention flow in transformers . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190--4197, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.acl-main.385 2020
[4]

Marco Ancona, Enea Ceolini, Cengiz \"O ztireli, and Markus Gross. 2018. Towards better understanding of gradient-based attribution methods for deep neural networks. In International Conference on Learning Representations

2018
[5]

Anonymous. 2024. https://openreview.net/forum?id=2mUNv2hCRV Evaluating human alignment and model faithfulness of LLM rationale . In Submitted to ACL Rolling Review - June 2024. Under review

2024
[6]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. http://arxiv.org/abs/1409.0473 Neural machine translation by jointly learning to align and translate . In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

arXiv 2015
[7]

Oren Barkan, Edan Hauon, Avi Caciularu, Ori Katz, Itzik Malkiel, Omri Armstrong, and Noam Koenigstein. 2021. Grad-sam: Explaining transformers via gradient self-attention maps. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2882--2887

2021
[8]

Jasmijn Bastings and Katja Filippova. 2020. https://doi.org/10.18653/v1/2020.blackboxnlp-1.14 The elephant in the interpretability room: Why use attention as explanation when we have saliency methods? In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 149--155, Online. Association for Computationa...

doi:10.18653/v1/2020.blackboxnlp-1.14 2020

Show all 48 references

[9]

Reuben Binns. 2018. Fairness in machine learning: Lessons from political philosophy. In Conference on fairness, accountability and transparency, pages 149--159. PMLR

2018
[10]

Alan David Boyle, Furui Cheng, Vil \'e m Zouhar, and Mennatallah El-Assady. 2025. Cafga: Customizing feature attributions to explain language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 461--470

2025
[11]

Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, and Aleksander Madry. 2024. Contextcite: Attributing model generation to context. Advances in Neural Information Processing Systems, 37:95764--95807

2024
[12]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

2019
[13]

Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. https://doi.org/10.18653/v1/2020.acl-main.408 ERASER : A benchmark to evaluate rationalized NLP models . In Proceedings of the 58th Annual Meeting of the A...

2020 doi
[14]

Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. Ragas: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages...

2024
[15]

Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. 2019. A guide to deep learning in healthcare. Nature medicine, 25(1):24--29

2019
[16]

Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. 2019. A benchmark for interpretability methods in deep neural networks. Advances in neural information processing systems, 32

2019
[17]

Sarthak Jain and Byron C. Wallace. 2019. https://api.semanticscholar.org/CorpusID:67855860 Attention is not explanation . In North American Chapter of the Association for Computational Linguistics

2019
[18]

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

2023 arXiv
[19]

Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. 2021. Layercam: Exploring hierarchical class activation maps for localization. IEEE transactions on image processing, 30:5875--5888

2021
[20]

Liqiang Jing, Ruosen Li, Yunmo Chen, and Xinya Du. 2024. Faithscore: Fine-grained evaluations of hallucinations in large vision-language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5042--5063

2024
[21]

Yash Kumar Lal, Nathanael Chambers, Raymond Mooney, and Niranjan Balasubramanian. 2021. https://doi.org/10.18653/v1/2021.findings-acl.53 T ell M e W hy: A dataset for answering why-questions in narratives . In Findings of the Association for Computational Linguistics: ACL-IJCN...

2021 doi
[22]

Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. https://doi.org/10.18653/v1/D16-1011 Rationalizing neural predictions . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107--117, Austin, Texas. Association for Computational L...

2016 doi
[23]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://openreview.net/forum?id=SyxS0T4tvS Ro \ bert \ a: A robustly optimized \ bert \ pretraining approach

2020
[24]

Haoyan Luo and Lucia Specia. 2024. From understanding to utilization: A survey on explainability for large language models. arXiv preprint arXiv:2401.12874

2024 arXiv
[25]

Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142--150

2011
[26]

Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 9004--9017

2023
[27]

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359--17372

2022
[28]

Hosein Mohebbi, Willem Zuidema, Grzegorz Chrupa a, and Afra Alishahi. 2023. https://doi.org/10.18653/v1/2023.eacl-main.245 Quantifying context mixing in transformers . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistic...

2023 doi
[29]

Vitali Petsiuk, Abir Das, and Kate Saenko. 2018. http://bmvc2018.org/contents/papers/1064.pdf RISE: randomized input sampling for explanation of black-box models . In British Machine Vision Conference 2018, BMVC 2018, Newcastle, UK, September 3-6, 2018 , page 151. BMVA Press

2018
[30]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine...

2021
[31]

why should i trust you?

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135--1144

2016
[32]

Wojciech Samek, Alexander Binder, Gr \'e goire Montavon, Sebastian Lapuschkin, and Klaus-Robert M \"u ller. 2016. Evaluating the visualization of what a deep neural network has learned. IEEE transactions on neural networks and learning systems, 28(11):2660--2673

2016
[33]

Sofia Serrano and Noah A. Smith. 2019. https://doi.org/10.18653/v1/P19-1282 Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931--2951, Florence, Italy. Association for Computational Linguistics

2019 doi
[34]

Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In International conference on machine learning, pages 3145--3153. PMlR

2017
[35]

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. http://arxiv.org/abs/1312.6034 Deep inside convolutional networks: Visualising image classification models and saliency maps . In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada,...

2014 arXiv
[36]

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

2013
[37]

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319--3328. PMLR

2017
[38]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

2023 arXiv
[39]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

2017
[40]

Michael Veale and Reuben Binns. 2017. Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society, 4(2):2053951717743530

2017
[41]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, and 1 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical ...

2020
[42]

Fan Yin, Zhouxing Shi, Cho-Jui Hsieh, and Kai-Wei Chang. 2022. On the sensitivity and stability of model interpretations in nlp. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2631--2647

2022
[43]

Sheng Zhang, Jin Wang, Haitao Jiang, and Rui Song. 2022. https://doi.org/10.18653/v1/2022.naacl-main.159 Locally aggregated feature attribution on natural language model understanding . In Proceedings of the 2022 Conference of the North American Chapter of the Association for ...

2022 doi
[44]

Chenyang Zhao, Kun Wang, Janet H Hsiao, and Antoni B Chan. 2025. Grad-eclip: Gradient-based visual and textual explanations for clip. arXiv preprint arXiv:2502.18816

2025 arXiv
[45]

Chenyang Zhao, Kun Wang, Xingyu Zeng, Rui Zhao, and Antoni B Chan. 2024 a . Gradient-based visual explanation for transformer-based clip. In International Conference on Machine Learning, pages 61072--61091. PMLR

2024
[46]

Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024 b . Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1--38

2024
[47]

Zhixue Zhao and Boxuan Shan. 2024. Reagent: A model-agnostic feature attribution method for generative language models. arXiv preprint arXiv:2402.00794

2024 arXiv
[48]

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921--2929

2016

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Samira Abnar and Willem Zuidema. 2020. https://doi.org/10.18653/v1/2020.acl-main.385 Quantifying attention flow in transformers . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190--4197, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.acl-main.385 2020

[4] [4]

Marco Ancona, Enea Ceolini, Cengiz \"O ztireli, and Markus Gross. 2018. Towards better understanding of gradient-based attribution methods for deep neural networks. In International Conference on Learning Representations

2018

[5] [5]

Anonymous. 2024. https://openreview.net/forum?id=2mUNv2hCRV Evaluating human alignment and model faithfulness of LLM rationale . In Submitted to ACL Rolling Review - June 2024. Under review

2024

[6] [6]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. http://arxiv.org/abs/1409.0473 Neural machine translation by jointly learning to align and translate . In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

arXiv 2015

[7] [7]

Oren Barkan, Edan Hauon, Avi Caciularu, Ori Katz, Itzik Malkiel, Omri Armstrong, and Noam Koenigstein. 2021. Grad-sam: Explaining transformers via gradient self-attention maps. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2882--2887

2021

[8] [8]

Jasmijn Bastings and Katja Filippova. 2020. https://doi.org/10.18653/v1/2020.blackboxnlp-1.14 The elephant in the interpretability room: Why use attention as explanation when we have saliency methods? In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 149--155, Online. Association for Computationa...

doi:10.18653/v1/2020.blackboxnlp-1.14 2020