pith. sign in

arxiv: 2607.05355 · v1 · pith:M5KUPD7S · submitted 2026-07-06 · cs.CL · cs.ET· cs.LG

Faithfulness to Refusal: A Causal Audit of Neuron Selectors

Ananth Eswar , Pratinav Seth , Utsav Avaiya , Vinay Kumar Sankarapu This is my paper

pith:M5KUPD7Sreviewed 2026-07-07 15:17 UTCmodel glm-5.2open to challenge →

classification cs.CL cs.ETcs.LG
keywords rowsattributionauditrefusalselectorscausalcausallyidentify
0
0 comments X

The pith

Zeroing attributed neuron rows installs refusal — and rank stability misses it

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks a direct question: when attribution methods score the importance of neuron rows (individual output dimensions of a transformer's weight matrices), do those scores actually track causal importance? The authors test this with a one-shot intervention — zeroing the rows a selector calls dispensable (LeRF) or important (MoRF) — and measuring the damage. They run this audit at two levels. At the language-modeling level, attribution-based selectors (LRP, IG, and their Borda consensus) identify dispensable rows two to four orders of magnitude more accurately than magnitude, activation, or random baselines across five LLMs. At the behavior level, they drive the same selectors with a contrastive harmful-versus-benign signal: zeroing the top-ranked compliance-promoting rows installs refusal on hate and crime prompts (reaching 0.82–0.95 refusal rates) while keeping benign over-refusal below 0.08 and preserving fluency. Layer-matched random controls that zero the same number of rows per layer but with random identities fail to reproduce the effect, confirming the result depends on which specific rows are zeroed, not just which layers are targeted. The sharpest cross-cutting finding is that rank stability — the common proxy that a selector is reliable if its rankings don't change across calibration subsets — is dissociated from causal validity: MeanActivation has near-perfect rank stability (Spearman 0.994) yet catastrophic causal invalidity, while LRP is the least stable yet most causally faithful. The authors also find that refusal lives in a redundant subspace: LRP and IG share only 3–6% of their top rows yet both install refusal, so each method recovers one sufficient set rather than a unique mechanism.

Core claim

The central object is the validity gap: the difference in model damage between zeroing the rows a selector ranks least important (LeRF) versus most important (MoRF). A faithful selector produces a large positive gap because its low-ranked rows are genuinely safe to remove and its high-ranked rows are genuinely damaging. Attribution-based selectors (LRP, IG, CONSENSUS-2) produce gaps orders of magnitude larger than non-attribution baselines across five model architectures. The same intervention, driven by a contrastive harmful-versus-benign margin, identifies rows whose zeroing is sufficient to install refusal behavior and specific in that random row controls at matched layer depths fail. The

What carries the argument

The neuron-row zeroing intervention: setting one row of a transformer's Linear weight matrices to zero, silencing a single output dimension without fine-tuning or inference hooks. The validity gap (MoRF PPL minus LeRF PPL) measures whether a selector's importance rankings track causal importance. The contrastive refusal margin (mean logit of refusal-onset tokens minus mean logit of compliance-onset tokens, computed at the last prompt position across matched harmful/benign prompt pairs) drives the same selectors to identify behavior-specific rows. CONSENSUS-2 is a Borda rank aggregation of LRP and IG scores. VETO variants isolate rows where the two methods disagree.

If this is right

  • Pruning recipes that select neurons by rank stability or calibration accuracy alone may silently use unfaithful selectors — the paper shows a selector can be perfectly reproducible yet rank causally important rows as dispensable.
  • Safety edits that zero attribution-identified rows can install refusal without fine-tuning, but the best method (LRP vs. IG) reverses across architectures, so practitioners must validate on their target model rather than assume a universal winner.
  • Circuit discovery claims that read a selector's top rows as the mechanism are not supported by this audit: refusal lives in a redundant subspace where disjoint row sets suffice, so attribution recovers one sufficient set, not a necessary one.
  • Rows where LRP and IG disagree carry stronger causal signal than where they agree (VETO-LRP outperforms consensus), suggesting disagreement-aware aggregation as a better strategy than intersection or averaging.
  • Signal density (fraction of rows with high contrastive compliance scores) predicts the operating-point sparsity needed per harm category and can be computed from calibration data before any test-set evaluation.

Load-bearing premise

The behavior-level results depend on a single refusal classifier and a contrastive margin over refusal-onset tokens; a model that learned to produce refusal-shaped opening phrases without genuinely refusing would satisfy both the mask construction and the judge. Transfer to 45 unseen SorryBench categories makes pure surface mimicry unlikely but does not fully eliminate the confound.

What would settle it

If zeroing the top-k rows identified by attribution selectors does not raise refusal rates above what layer-matched random controls achieve, or if the validity gap at the LM level is near zero for attribution selectors, the core claims would be falsified. The rank-stability dissociation would be falsified if a highly stable selector also produced a large validity gap.

read the original abstract

Attribution scores increasingly identify which neuron rows of a language model matter for applications such as pruning, interpretability, and editing for safety, yet whether they identify causally important rows is rarely tested directly. We address this with two paired audits built on one-shot neuron-row zeroing. We first audit selectors at the language-modeling level: attribution methods substantially outperform activation and magnitude-based baselines at identifying dispensable rows across five LLMs. We then adapt the same intervention into a behavior test by driving it with a contrastive harmful-versus-benign signal; the attributed rows are sufficient to install refusal on hate and crime while keeping benign over-refusal low and preserving language model fluency, and specific in that layer-matched random controls at the same depths fail. Highly rank-stable selectors can be among the least causally valid. Refusal moreover lives in a redundant subspace, where different attribution methods install it through largely disjoint row sets, so the recovered edit is one realization of a sufficient set rather than a unique mechanism. Together, these findings show that rank-stability proxies miss the kinds of selector failures a direct causal audit can surface.%

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 8 minor

Summary. This paper presents a two-level causal audit of neuron-row selectors in LLMs. At the language-modeling level, it evaluates seven selectors (Random, Magnitude, Wanda, MeanActivation, LRP, IG, CONSENSUS-2) across five models using LeRF/MoRF zeroing sweeps and a validity-gap metric. At the behavior level, it uses contrastive harmful-versus-benign attribution to construct refusal-installing masks, testing sufficiency (refusal installation), specificity (layer-matched random controls), and utility retention across four instruction-tuned models and five CAST harm domains. Key findings include: (1) attribution-based selectors are orders of magnitude more causally faithful than non-attribution baselines at the LM level; (2) rank-stability is neither necessary nor sufficient for causal validity (MeanActivation is most stable yet least valid); (3) contrastive attribution installs refusal on hate/crime while preserving benign accuracy and fluency; (4) the best method reverses across architectures; and (5) refusal lives in a redundant subspace where LRP and IG select largely disjoint row sets yet both suffice. The experimental design is strong: five models, seven selectors, layer-matched controls, Wilson intervals, and a 70-cell behavior grid with held-out OOD benchmarks.

Significance. The paper makes a substantive contribution to the interpretability and safety-editing literature by providing a direct causal validity test for neuron selectors, filling a gap left by rank-stability and calibration-accuracy proxies. The stability-validity dissociation (Table 1, replicated on Qwen3-8B) is a clean, actionable finding for practitioners. The layer-matched control design (Table 2, Table 29) and the veto/intersection controls (§4.3) are methodologically well-constructed and rule out several trivial explanations. The cross-architecture method reversal (Figure 3) and the redundant-subspace finding (§5.5, Table 35) are genuinely novel structural results. The commitment to releasing 70 edited checkpoints as standard HuggingFace artifacts is a strength. The paper is also commendably transparent about limitations, including the surface-form confound discussed below.

major comments (1)
  1. §5.1–5.3, Algorithm 1 and Limitations: The behavior-level sufficiency claim rests on a potential circularity between mask construction and evaluation. The contrastive margin m(x) (Algorithm 1, Step 1) is computed over refusal-onset token sets S (e.g., 'I must decline', 'Unfortunately'), and the evaluation uses ProtectAI/distilroberta-base-rejection-v1, which the authors acknowledge is 'sensitive to refusal surface forms' (Limitations). An intervention that shifts the model toward producing refusal-shaped openings—without genuinely refusing—would satisfy both the margin objective and the classifier. The authors correctly identify this confound and argue that SorryBench transfer across 45 unseen categories 'makes pure surface mimicry unlikely.' However, SorryBench is scored by the same classifier, so a model that learned to produce refusal-shaped openings on any harmful-looking prompt (a泛化
minor comments (8)
  1. §3.2: The term 'faithful' is given an operational definition (sufficiency + specificity), but the phrase 'causally faithful' appears throughout the paper (abstract, §1, §6) without always re-grounding the reader in this specific operational meaning. A brief footnote or parenthetical reminder at first use in the abstract/intro would help readers unfamiliar with this framing.
  2. Table 5: Several MoRF values are reported in scientific notation with varying precision (e.g., 1.04e11 for LRP LLaMA-1B vs. 821049 for IG LLaMA-1B). Standardizing to consistent significant figures or always using scientific notation for values above 1e6 would improve readability.
  3. §5.1: The operating-point selection procedure (argmax over 36 (λ,k) cells with feasibility constraints) could introduce optimistic bias. The authors note that 'neighbouring feasible cells land within ~0.1-0.2 of the reported optimum' (Appendix E), but a formal report of the distribution of feasible-cell values, or a mean-of-feasible-cells comparison, would strengthen the claim that reported rates are not isolated spikes.
  4. Figure 2: The PPL color encoding (green/orange/red by PPL band) is informative but the boundary thresholds (≤15, 16-25, 26-40, >40) are stated only in the caption. Adding these as tick marks or annotations on the figure itself would aid interpretation.
  5. §5.3: The claim that 'refusal is MLP-concentrated' (77.6% of LRP top-1% rows are gate_proj or up_proj) is interesting but the complementary fraction (15% attention) is mentioned without further analysis. A brief discussion of what the attention rows represent—whether they are safety-relevant attention heads or noise—would contextualize the finding.
  6. References: The paper cites several self-authored works ([14] DLBacktrace, [17] Interpretability as alignment, [20] xai_evals, [21] Bridging the gap in XAI, [31] C-delta-theta). These are used appropriately for context or as compared baselines, not as load-bearing premises. However, the density of self-citations in §2 could be read as over-promoting related work. The authors should ensure each self-citation is accompanied by a clear statement of what specific method or finding it contributes to the current paper.
  7. §4.3: The VETO-LRP result (outperforms real CONSENSUS-2 on both axes on both models) is presented as motivation for 'disagreement-aware aggregation as a direction for follow-up work.' Given that this is a strong empirical result within the paper's own audit, the authors should either briefly explain why they do not develop it further here (e.g., scope reasons) or note whether a follow-up is in preparation. The current framing undersells the finding.
  8. Table 2 caption: The note about re-scoring variance ('slight differences from Table 45 reflect re-scoring variance') is important but buried. Given that Table 2 is a key specificity result, the authors should either co-evaluate all results in a single run or more prominently flag the re-scoring and its magnitude.

Simulated Author's Rebuttal

3 responses · 1 unresolved

We thank the referee for the careful reading and the constructive recommendation. The referee raises one major point concerning a potential circularity between the mask construction objective (which uses refusal-onset token sets) and the evaluation classifier (which is sensitive to refusal surface forms). We agree this is a genuine concern, acknowledge that our current SorryBench transfer argument does not fully break the circularity, and will revise the manuscript to address it.

read point-by-point responses

  1. Referee: §5.1–5.3, Algorithm 1 and Limitations: The behavior-level sufficiency claim rests on a potential circularity between mask construction and evaluation. The contrastive margin m(x) (Algorithm 1, Step 1) is computed over refusal-onset token sets S (e.g., 'I must decline', 'Unfortunately'), and the evaluation uses ProtectAI/distilroberta-base-rejection-v1, which the authors acknowledge is 'sensitive to refusal surface forms' (Limitations). An intervention that shifts the model toward producing refusal-shaped openings—without genuinely refusing—would satisfy both the margin objective and the classifier. The authors correctly identify this confound and argue that SorryBench transfer across 45 unseen categories 'makes pure surface mimicry unlikely.' However, SorryBench is scored by the same classifier, so a model that learned to produce refusal-shaped openings on any harmful-looking prompt [cut

    Authors: The referee is correct that the SorryBench transfer argument does not fully break the circularity, because SorryBench is scored by the same ProtectAI classifier that is sensitive to refusal surface forms. We acknowledge this in the Limitations section ('a template-diverse judge is the definitive check') but agree the current text overstates the force of the SorryBench argument. We will revise accordingly. revision: yes

  2. Referee: [Continued from above] The referee's concern that a model producing refusal-shaped openings on any harmful-looking prompt would satisfy both the margin and the classifier.

    Authors: We offer three lines of evidence that partially mitigate the confound, while agreeing that none is definitive without a template-diverse or human judge. First, the layer-matched random control (Table 2) is directly relevant: random row zeroing at the same per-layer counts produces near-baseline refusal (0.003–0.38), while attribution masks reach 0.82–0.95. If the effect were simply 'model damage produces refusal-shaped openings,' we would expect the random control to also trigger the classifier at non-trivial rates, but it does not. The effect requires specific rows, not just damage. Second, benign over-refusal remains low (≤0.08 on hate/crime at the operating point). A model that learned to emit refusal-shaped openings indiscriminately would over-refuse on benign prompts, but the contrastive construction's harmful-minus-benign subtraction specifically penalizes this. Third, the utility retention results (PPL within +2–4, MMLU drops of only 0.06–0.09 on hate/crime) show the model is not merely broken into producing degenerate refusal-shaped text; it retains coherent generation on standard benchmarks. However, we concede that none of these arguments fully rules out the scenario the referee describes: a model that selectively produces refusal-shaped openings on harmful-looking prompts while remaining coherent on benign ones. The definitive check requires either (a) a human evaluation of generated outputs, or (b) a second classifier that does not key on refusal surface forms. We will add both to the revised manuscript: (1) We will include a sample of 50 generated completions per domain (hate, crime, adult) from the LRP-edited LLaMA-3.1-8B model for manual inspection, showing that the refusals are semantically appropriate (e.g., 'I cannot help with planning a hate crime' vs revision: partial

  3. Referee: The referee's concern about the circularity between mask construction and evaluation, continued.

    Authors: degenerate 'I must decline' regardless of input). (2) We will re-score a subset of the CAST hate and crime test sets using a second, independently trained refusal classifier (e.g., LlamaGuard or a prompt-based LLM judge) and report the agreement rate with the ProtectAI classifier. If the two classifiers agree on the attribution-vs-control separation, the effect is unlikely to be an artifact of a single classifier's surface-form sensitivity. (3) We will strengthen the Limitations section to explicitly state that SorryBench transfer alone does not break the circularity because it uses the same judge, and that the added human/second-judge evaluation is the appropriate control. We note that the order-of-magnitude effects (0.024→0.92+ against controls at 0.003) are large enough that they are unlikely to be fully explained by surface-form sensitivity alone, but we agree that this is an argument from effect size, not a proof, and the revised text will reflect that distinction. revision: partial

standing simulated objections not resolved
  • We cannot fully rule out the surface-form confound in the current manuscript without the additional evaluations described above. The referee's point is correct: the mask construction objective and the evaluation classifier both key on refusal surface forms, and SorryBench transfer does not break this circularity because it uses the same classifier. The strongest we can honestly claim is that the layer-matched random control, low benign over-refusal, and utility retention provide converging partial evidence against pure surface mimicry, while acknowledging that a template-diverse judge or human evaluation is needed for a definitive conclusion.

Circularity Check

0 steps flagged

No significant circularity. The paper's central claims are grounded in direct causal interventions (LeRF/MoRF PPL sweeps, layer-matched random controls, SorryBench/OR-Bench transfer) that are independent of the selectors being audited. Self-citations are contextual, not load-bearing.

full rationale

The paper's two-level audit is structurally non-circular. At the LM level, selectors rank neuron rows, rows are zeroed, and WikiText-2 PPL is measured — the evaluation metric (perplexity) is independent of the selector's scoring function. The validity gap (MoRF PPL − LeRF PPL) is a differencing measurement that cancels generic ablation shock, not a self-referential definition. At the behavior level, the skeptic's concern about coupling between the margin construction (Algorithm 1, Step 1: m(x) = mean_{t∈S} z_t − mean_{t∈H} z_t over refusal-onset token logits) and the evaluation classifier (ProtectAI/distilroberta-base-rejection-v1) is a legitimate confound, but it is not circularity by construction: the construction uses logit values of specific tokens at the last prompt position, while the evaluation uses a separate classifier on the full greedy-decoded response (up to 200 tokens). These are related signals (both key on refusal surface forms) but not identical metrics, so the evaluation is not tautologically equal to the construction objective. The paper explicitly acknowledges this limitation ('a model that only learned refusal-shaped openings would satisfy both') and provides mitigations: SorryBench transfer across 45 unseen harm categories, OR-Bench-Hard over-refusal measurement, utility preservation (MMLU/GSM8K/IFEval), and layer-matched random controls showing +0.46 to +0.77 attribution-vs-control gaps. The self-citations ([14] DLBacktrace, [17] Interpretability as alignment, [20] xai_evals, [21] Bridging the gap in XAI, [31] C-δΘ) are used for context or as related-work pointers; none is invoked as a load-bearing premise, uniqueness theorem, or ansatz that would force the conclusion. The selectors actually tested (LRP, IG, CONSENSUS-2, Magnitude, Wanda, MeanActivation, Random) are defined independently and evaluated against external benchmarks (CAST, SorryBench, OR-Bench-Hard, lm-eval-harness, WikiText-2). The signal-density-to-operating-point prediction ('The operating point can be predicted from calibration before any test-set evaluation') is a genuine forward prediction from calibration data, not a fit renamed as prediction. Score 1 reflects the minor residual coupling between construction and evaluation at the behavior level, which is a correctness risk rather than a circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The free parameters (lambda, k) are selected via grid search with stated constraints, which is standard for this type of experiment but introduces post-hoc selection risk. The domain assumptions are clearly stated and partially addressed by controls (e.g., SorryBench transfer for the classifier assumption). No invented entities are pulled from a hat; CONSENSUS-2 and VETO variants are tested against external benchmarks.

free parameters (5)
  • lambda (LM-protect strength) = selected from {0.2, 0.3, 0.5, 0.6, 0.7, 0.8} per cell
    Controls the penalty on LM-critical rows during contrastive mask construction; selected via grid search to maximize malign refusal subject to benign/PPL constraints.
  • k (mask sparsity) = selected from {0.005, 0.01, 0.02, 0.05, 0.075, 0.1} per cell
    Fraction of rows zeroed; selected via grid search per (model, selector, domain) cell.
  • benign cap = 0.10 (relaxed to 0.20 for one cell)
    Threshold for acceptable benign over-refusal; relaxed once in the 70-cell grid.
  • IG interpolation steps = 16
    Number of steps for Integrated Gradients; fixed by design choice.
  • calibration samples = 128 (LM level), 64 pairs (behavior level)
    Number of WikiText-2 samples or CAST pairs used for attribution; fixed by design.
axioms (4)
  • domain assumption Zeroing a neuron row is a minimal structural intervention that silences one output dimension without introducing inference-time artifacts.
    Section 3.1: the entire framework rests on row-zeroing being a clean intervention traceable to the selector.
  • domain assumption The validity gap (MoRF PPL - LeRF PPL) cancels generic ablation shock because both arms apply the same intervention class.
    Section 3.3: the differencing argument assumes LeRF and MoRF experience identical distribution shift, which holds only if the shift is rate-dependent not identity-dependent.
  • domain assumption The ProtectAI/distilroberta-base-rejection-v1 classifier accurately distinguishes genuine refusal from compliance.
    Section 5.2 and Limitations: all behavior-level refusal rates depend on this classifier's accuracy.
  • domain assumption Contrastive margin over refusal-onset tokens isolates refusal-specific subspace from general prompt-processing.
    Section 5.1: the margin construction assumes that averaging logits of refusal/compliance tokens captures refusal-relevant signal rather than surface-form processing.
invented entities (2)
  • CONSENSUS-2 independent evidence
    purpose: Borda-style rank aggregation of LRP and IG scores to reduce false negatives
    Tested against four control variants (layer-matched, rank-randomized, strict intersection, veto) and evaluated on external benchmarks; its performance is falsifiable and not circular.
  • VETO-LRP / VETO-IG independent evidence
    purpose: Veto aggregators that keep rows ranked important by one method but not the other
    Tested as controls in Appendix C.4; their performance is compared against real CONSENSUS-2 on external metrics.

pith-pipeline@v1.1.0-glm · 54048 in / 2795 out tokens · 212043 ms · 2026-07-07T15:17:56.450785+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages · 23 internal anchors

  1. [1]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models. InICLR, 2024

    work page 2024

  2. [2]

    SparseGPT: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. SparseGPT: Massive language models can be accurately pruned in one-shot. In ICML, 2023. 10 Faithfulness to Refusal: A Causal Audit of Neuron Selectors

    work page 2023

  3. [3]

    Fluctuation-based Adaptive Structured Pruning for Large Language Models

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. FLAP: Fluctuation-based adaptive structured pruning for large language models. InAAAI, 2024. arXiv:2312.11983

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  4. [4]

    Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs

    Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Reduan Achtibat, Patrick Kahardipraja, Thomas Wiegand, Wojciech Samek, Alexander Binder, and Sebastian Lapuschkin. Attribution-guided pruning for insight and control: Circuit discovery and targeted correction in small-scale llms.CoRR, abs/2506.13727, 2025. URL https://arxiv.org/abs/2506.13727

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  5. [5]

    Programming Refusal with Conditional Activation Steering

    Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Na- gireddy, and Amit Dhurandhar. Programming refusal with conditional activation steering. InICLR, 2025. arXiv:2409.05907

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  6. [6]

    Representation Engineering: A Top-Down Approach to AI Transparency

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down approach to A...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  7. [7]

    SORRY-Bench: Systematically evaluating large language model safety refusal

    Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, and Prateek Mittal. SORRY-Bench: Systematically evaluating large language model safety refusal. InICLR, 2025

    work page 2025

  8. [8]

    OR-Bench: An Over-Refusal Benchmark for Large Language Models

    Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. OR-Bench: An over-refusal benchmark for large language models. InICML, 2025. arXiv:2405.20947

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. InICLR, 2021

    work page 2021

  10. [10]

    Training Verifiers to Solve Math Word Problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  11. [11]

    Instruction-Following Evaluation for Large Language Models

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  12. [12]

    LLM-Pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. LLM-Pruner: On the structural pruning of large language models. InNeurIPS, 2023

    work page 2023

  13. [13]

    Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers

    Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Reduan Achtibat, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin. Pruning by explaining revisited: Optimizing attribution methods to prune CNNs and transformers.CoRR, abs/2408.12568, 2024. URLhttps://arxiv.org/abs/2408.12568

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  14. [14]

    DLBacktrace: A Model Agnostic Explainability for any Deep Learning Models

    Vinay Kumar Sankarapu, Chintan Chitroda, Yashwardhan Rathore, Neeraj Kumar Singh, and Pratinav Seth. DLBacktrace: A model agnostic explainability for any deep learning models.CoRR, abs/2411.12643, 2024. URL https://arxiv.org/abs/2411.12643

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  15. [15]

    Interpretability in the wild: a circuit for indirect object identification in GPT-2 small

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. InICLR, 2023

    work page 2023

  16. [16]

    Attribution Patching Outperforms Automated Circuit Discovery

    Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. CoRR, abs/2310.10348, 2023. URLhttps://arxiv.org/abs/2310.10348

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  17. [17]

    Interpretability as alignment: Making internal understanding a design principle.CoRR, abs/2509.08592, 2025

    Aadit Sengupta, Pratinav Seth, and Vinay Kumar Sankarapu. Interpretability as alignment: Making internal understanding a design principle.CoRR, abs/2509.08592, 2025. URLhttps://arxiv.org/abs/2509.08592

    work page arXiv 2025

  18. [18]

    Evaluating the visualization of what a deep neural network has learned.IEEE Transactions on Neural Networks and Learning Systems, 28(11):2660–2673, 2017

    Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. Evaluating the visualization of what a deep neural network has learned.IEEE Transactions on Neural Networks and Learning Systems, 28(11):2660–2673, 2017. doi: 10.1109/TNNLS.2016.2599820

    work page doi:10.1109/tnnls.2016.2599820 2017

  19. [19]

    A benchmark for interpretability methods in deep neural networks

    Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretability methods in deep neural networks. InNeurIPS, 2019

    work page 2019

  20. [20]

    xai_evals : A Framework for Evaluating Post-Hoc Local Explanation Methods

    Pratinav Seth, Yashwardhan Rathore, Neeraj Kumar Singh, Chintan Chitroda, and Vinay Kumar Sankarapu. xai_evals: A framework for evaluating post-hoc local explanation methods.CoRR, abs/2502.03014, 2025. URL https://arxiv.org/abs/2502.03014

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  21. [21]

    Bridging the gap in XAI: Why reliable metrics matter for explainability and compliance.CoRR, abs/2502.04695, 2025

    Pratinav Seth and Vinay Kumar Sankarapu. Bridging the gap in XAI: Why reliable metrics matter for explainability and compliance.CoRR, abs/2502.04695, 2025. URLhttps://arxiv.org/abs/2502.04695

    work page arXiv 2025

  22. [22]

    On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.PLoS ONE, 10(7):e0130140, 2015

    Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.PLoS ONE, 10(7):e0130140, 2015. doi: 10.1371/journal.pone.0130140

    work page doi:10.1371/journal.pone.0130140 2015

  23. [23]

    Layer-wise relevance propagation: An overview

    Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller. Layer-wise relevance propagation: An overview. InExplainable AI: Interpreting, Explaining and Visualizing 11 Faithfulness to Refusal: A Causal Audit of Neuron Selectors Deep Learning, volume 11700 ofLecture Notes in Computer Science, pages 193–209. Springer...

    work page doi:10.1007/978-3-030-28954-6_10 2019

  24. [24]

    AttnLRP: Attention-aware layer-wise relevance propagation for transformers

    Reduan Achtibat, Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Aakriti Jain, Thomas Wiegand, Sebastian Lapuschkin, and Wojciech Samek. AttnLRP: Attention-aware layer-wise relevance propagation for transformers. InICML, 2024

    work page 2024

  25. [25]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. InICML, 2017

    work page 2017

  26. [26]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  27. [27]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InNeurIPS, 2023

    work page 2023

  28. [28]

    Steering Llama 2 via contrastive activation addition

    Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition. InACL, 2024

    work page 2024

  29. [29]

    Refusal in Language Models Is Mediated by a Single Direction

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. InNeurIPS, 2024. arXiv:2406.11717

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  30. [30]

    Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications

    Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. Assessing the brittleness of safety alignment via pruning and low-rank modifications. InICML, 2024. arXiv:2402.05162

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  31. [31]

    C- ∆Θ: Circuit-restricted weight arithmetic for selective refusal.CoRR, abs/2602.04521, 2026

    Aditya Kasliwal, Pratinav Seth, and Vinay Kumar Sankarapu. C- ∆Θ: Circuit-restricted weight arithmetic for selective refusal.CoRR, abs/2602.04521, 2026. URLhttps://arxiv.org/abs/2602.04521

    work page arXiv 2026

  32. [32]

    Towards understanding safety alignment: A mechanistic perspective from safety neurons

    Jianhui Chen, Xiaozhi Wang, Zijun Yao, Yushi Bai, Lei Hou, and Juanzi Li. Towards understanding safety alignment: A mechanistic perspective from safety neurons. InNeurIPS, 2025. arXiv:2406.14144

    work page arXiv 2025

  33. [33]

    Understanding Refusal in Language Models with Sparse Autoencoders

    Wei Jie Yeo, Nirmalendu Prakash, Clement Neo, Ranjan Satapathy, Roy Ka-Wei Lee, and Erik Cambria. Under- standing refusal in language models with sparse autoencoders. InFindings of EMNLP, 2025. arXiv:2505.23556

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  34. [34]

    Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal

    Nirmalendu Prakash, Wei Jie Yeo, Amir Abdullah, Ranjan Satapathy, Erik Cambria, and Roy Ka-Wei Lee. Beyond I’m sorry, I can’t: Dissecting large language model refusal.CoRR, abs/2509.09708, 2025. URL https://arxiv.org/abs/2509.09708

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  35. [35]

    The geometry of refusal in large language models: Concept cones and representational independence

    Tom Wollschläger, Jannes Elstner, Simon Geisler, Vincent Cohen-Addad, Stephan Günnemann, and Johannes Gasteiger. The geometry of refusal in large language models: Concept cones and representational independence. CoRR, abs/2502.17420, 2025. URLhttps://arxiv.org/abs/2502.17420

    work page arXiv 2025

  36. [36]

    A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models

    Hamid Kazemi, Atoosa Chegini, and Maria Safi. A single neuron is sufficient to bypass safety alignment in large language models.CoRR, abs/2605.08513, 2026. URLhttps://arxiv.org/abs/2605.08513

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  37. [37]

    Pointer Sentinel Mixture Models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models.CoRR, abs/1609.07843, 2016. URLhttps://arxiv.org/abs/1609.07843

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  38. [38]

    Lessons from the trenches on reproducible evaluation of language models

    Stella Biderman, Hailey Schoelkopf, Lintang Sutawika, Leo Gao, Jonathan Tow, Baber Abbasi, et al. Lessons from the trenches on reproducible evaluation of language models. https://github.com/EleutherAI/ lm-evaluation-harness, 2024

    work page 2024

  39. [39]

    Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retraining

    Andreas Madsen, Nicholas Meade, Vaibhav Adlakha, and Siva Reddy. Evaluating the faithfulness of importance measures in NLP by recursively masking allegedly important tokens and retraining.CoRR, abs/2110.08412, 2021. URLhttps://arxiv.org/abs/2110.08412

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  40. [40]

    The Llama 3 Herd of Models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    Qwen3 Technical Report

    Qwen Team. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  42. [42]

    Gemma 3 Technical Report

    Gemma Team. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  43. [43]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1–67, 2020

    work page 2020

  44. [44]

    ProtectAI/distilroberta-base-rejection-v1: Refusal classifier

    ProtectAI. ProtectAI/distilroberta-base-rejection-v1: Refusal classifier. https://huggingface.co/ProtectAI/ distilroberta-base-rejection-v1, 2024

    work page 2024

  45. [45]

    Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts

    Hongyu Chen and Seraphina Goldfarb-Tarrant. Safer or luckier? LLMs as safety evaluators are not robust to artifacts. InACL, 2025. arXiv:2503.09347

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  46. [46]

    Layers” counts all prunable Linear layers; “Neurons

    Leo Schwinn, Moritz Ladenburger, Tim Beyer, Mehrnaz Mofakhami, Gauthier Gidel, and Stephan Günnemann. A coin flip for safety: LLM judges fail to reliably measure adversarial robustness.CoRR, abs/2603.06594, 2026. URLhttps://arxiv.org/abs/2603.06594. 12 Faithfulness to Refusal: A Causal Audit of Neuron Selectors Appendix A Selector Descriptions and Full 30...

    work page arXiv 2026