Pith.

Artificial Intelligence · arXiv:2611.04782 v1 · submitted 3 Nov 2026 · 59 references

Reducing Logprob Without Causal Verification: Diagnosing SAE-Based PII Suppression in LLMs

Chaerin Lee, Wonho Lee, Sohee Park, Daeseon Choi

The paper says

“SAE-based feature-level interventions to suppress PII leakage in LLMs are primarily evaluated by log-probability reduction. We show that this criterion cannot distinguish feature specific suppression from a magnitude artifact: a direction-agnostic effect of sufficiently large activation perturbations.”

more of the abstract

Across 36 (model, layer, PII type) cases, we compare selected SAE feature interventions with coherencematched random direction controls using logprob and rank-aware metrics. While 17 cases (47%) appear successful under logprob-only evaluation, only 1 case (2.8%) passes our multimetric framework. The rest reveal two failure modes: random directions outperform selected features, and feature selection collapse across PII types. These results indicate that existing evaluations cannot verify whether interventions target causal pathways of PII leakage. A 9,000 run causal mediation feasibility study corroborates this: the verifier separates candidates reliably, but correlation-selected candidates rarely pass it. We therefore propose a new research mission for NLP: causally grounded PII suppression through circuit-level localization and causal mediation analysis.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 4 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. The 36-cell random-control diagnostic is sound and worth publishing; the mediation 'verifier' in §5.1 is an uncalibrated probe and should not be read as causal corroboration. This line is wrong

3 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 59 references say.

The paper says, at §5.1, Table 1, Appendix A.14

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.76):

“We note that this design is itself partially circular, since logprob serves both as the selection criterion and as the evaluated met- ric; Section 5.1 therefore revisits the same question with candidates selected independently of logprob.”

Pith says: Objection 1 of 3 · major

The NIE estimator is an uncalibrated single-feature, single-position noising probe, and the 97.3% 'indistinguishable' rate is largely mechanical. NIE_f = lp(y|x1, f=0) − lp(y|x0); since x0 and x1 share all preceding text and the intervention zero-ablates one feature at the single token immediately before the PII span, any feature whose one-token ablation has a negligible downstream effect yields lp(y|x1, f=0) ≈ lp(y|x1) and hence NIE_f ≈ TE.

more of objection 1

The matched random direction collapses identically (NIE_rand ≈ TE), forcing NIE_f/NIE_rand ≈ 1 into the (0.5, 2.0) window for nearly every candidate. The authors acknowledge the selection-circularity limitation (Limitations, 'Circular selection design'; §3) but do not address this probe-sensitivity confound: no calibration on a feature known — by construction or by an independent method — to mediate PII generation is reported anywhere in Appendix A.14. Without such calibration, '146/150 (97.3%) are indistinguishable' cannot support the abstract's claim that correlation-selected candidates 'rarely pass' causal verification, because a completely insensitive probe produces exactly this pattern. The data as analyzed cannot distinguish 'candidates are non-causal' from 'the probe cannot detect single-feature mediation at one position.'

The paper says, at §5.1 / Table 1

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The direction of the observed separability pattern argues against the claim that 'the verifier separates candidates reliably.' The only cell in which any candidate is separable is Gemma name (11/15 inside the window, i.e., 4/15 distinguishable), which has the smallest positive TE (+0.609).

more of objection 2

The three email cells, which carry the strongest memorization signal (TE = +2.580, +3.560, +2.466 across the three models), are 100% indistinguishable. A verifier that separates candidates by 'whether genuine memorization is present' should show more separability where memorization is stronger, not less. The observed ordering is precisely what one expects if the individual ratios are noisy TE-over-TE quantities: when TE is large the ratio is stably near 1, and when TE is small sampling noise pushes a few ratios outside the window. The Section 5 conclusion that 'the bottleneck is candidate generation' therefore does not follow from the reported data: the alternative that the probe lacks sensitivity to single-feature mediation fits the same data at least as well.

The paper says, at Appendix A.14

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The mediation analysis reports no uncertainty quantification, and the (0.5, 2.0) window is never calibrated. With 50 pairs per cell, the means NIE_f and NIE_rand are subject to nontrivial sampling noise, yet the Indist. percentages in Table 1 are treated as exact.

more of objection 3

The window is also intrinsically conservative: a feature that genuinely mediates 50% of TE yields NIE_f/NIE_rand = 0.5, exactly on the boundary, so any mediation below roughly 50% is 'indistinguishable' by construction, regardless of probe quality. The mediation fraction MF_f is defined in §5.1 but never reported; reporting MF_f with confidence intervals, or a sensitivity analysis on synthetic mediators with known effects, is the minimum needed to make the feasibility study interpretable as evidence about candidate quality.

4 minor objections

The paper says, at §5.1

Pith says: minor objection 1 of 4

The sentence 'The sole exception, Gemma-2-9B/name (11/15), is precisely the one combination exhibiting a positive memorization signal' is contradicted by the paper's own Table 18 (Gemma email, TE = +2.580); the intended wording is the Table 1 note's 'the only non-email cell with a positive memorization signal,' and the main text should match it.

The paper says, at Table 9

Pith says: minor objection 2 of 4

Two cells (Llama L24/phone, Llama L24/email) are labeled 'inactive' with undefined selectivity; a sentence stating explicitly that such cells fail every gate and remain in the 36-cell denominators of the 17/36 and 1/36 counts would remove ambiguity for readers.

show 2 more

The paper says, at Table 9 / Appendix A.12

Pith says: minor objection 3 of 4

Cell-level means are reported without dispersion (e.g., over the 450 evaluation contexts or the three random-control seeds). Given that several conclusions hinge on ratio-based selectivity values, including Qwen L27/id Sellp = 115.1 and Llama L16/name Sellp ≈ 1995 with the random denominator at the measurement floor, reporting per-cell standard errors or the distribution of per-pair values would aid assessment.

The paper says, at Appendix A.12 / Section 4

Pith says: minor objection 4 of 4

The alpha-robustness claim is established on only three representative cells and two templates (T1, T5), and one of the three cells (Qwen L18/name) flips from selective to random-dominant as alpha varies; the main text should state this scope where robustness to intervention strength is claimed.

What would settle it

Calibrate the A.14 verifier on a ground-truth mediator with a positive control: take an arbitrary non-dead SAE feature g in a cell with |TE| ≥ 0.05, add β·W_dec[g] to the hidden state at the probed pre-PII token during x1 runs, with β chosen so the induced change in lp(y|x1) is comparable to the natural TE, so g is a mediator at exactly the probed position. Then run the zero-ablation protocol for g against its matched random direction. If NIE_g/NIE_rand is not below 0.5, the probe is too insensitive for the 146/150 statistic to carry the paper's causal conclusion. As a zero-cost additional check, run the same protocol on the existing positive-control feature f=32619 (Llama L16/name, Sel_lp ≈ 1995).

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 4 Nov 2026

  1. Reader holds up, with conditions confidence high
  2. Skeptic agrees 1 min 31 s · 25,042 tokens read, 13,810 written
  3. Editor wrote the verdict above 31 s · 25,996 tokens read, 3,415 written
  4. Referee major revision: 3 major objections, 4 minor 1 min 29 s · 25,287 tokens read, 14,779 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Bottom line: the Section 4 diagnostic is the real contribution and it holds. The mediation run in §5.1 does not, and the abstract overstates it. I would send this to review, but only with §5.1 either recalibrated or explicitly downgraded to an uncalibrated pilot.

What is new: the 36-cell grid that applies random-direction controls to SAE-based PII suppression. The random-baseline critique itself is not new—Korznikov, Ronge, Heap, and Méloux have made the general point—but the instantiation is. Logprob-only evaluation passes 17/36 cells; a conjunctive criterion with matched controls passes 1/36. The two failure modes, random-dominant and selection collapse, are documented cleanly, with a positive control (Llama L16/name, Sel_lp ≈ 1995, no utility loss) showing the criterion is not impossibly strict. The controls are matched for magnitude, span-level coherence, and dictionary alignment, with sensitivity analyses on thresholds and α. Code, data, and SAEs are released, and I checked the headline arithmetic in Table 9: 17 and 1 are right.

Where it wobbles: §5.1. The noising-style NIE_f is a single-feature, single-position zero-ablation. Since x0 and x1 share the preceding text, standard patching is identically zero, and the substitute probe is never calibrated. For almost any feature, ablating it at one position leaves lp(y|x1,f=0) ≈ lp(y|x1), so NIE_f ≈ TE; the random control collapses the same way. The 97.3% indistinguishability rate is close to the expected null of an insensitive probe, not evidence about candidate quality. Table 1 makes the problem visible: the only separable cell has 11/15 inside the window and the smallest positive TE, while the email cells with TE > 2.4 are 100% indistinguishable. The authors acknowledge circularity and scope their conclusion elsewhere ([b29], [b123]), and they report the numbers honestly, but the abstract's 'verifier separates candidates reliably' is not supported. The mediation section should not be cited as causal verification.

Who gets value: people working on SAE interpretability, privacy defenses, and evaluation methodology. The Section 4 result is reproducible and relevant, and it deserves a serious referee. Recommendation: accept with revision—recalibrate the mediation probe on a known mediator or soften the causal-verification claims.

The strongest objection

The load-bearing weakness is the causal-mediation corroboration in §5.1/Table 1/A.14, not the Section 4 diagnostic. The redefined NIE_f = lp(y|x1, f=0) − lp(y|x0) is a single-feature, single-position noising ablation. Since x0 and x1 share all preceding text, a feature with no real downstream effect leaves lp(y|x1, f=0) ≈ lp(y|x1), so NIE_f ≈ TE; the matched random direction collapses the same way, forcing NIE_f/NIE_rand ≈ 1. The 146/150 (97.3%) rate inside the uncalibrated (0.5,2.0) window is therefore the expected null of an insensitive probe, not evidence about candidate quality. Table 1's only separable cell, Gemma name, has 11/15 inside the window (4 distinguishable) and the smallest positive TE (+0.609), while all email cells with TE > 2.4 are 100% indistinguishable; this contradicts the abstract's 'verifier separates candidates reliably'. The authors acknowledge partial circularity [b29] and attribute the bottleneck to candidate generation [b416], but no calibration on a known mediator is reported, so the data cannot distinguish 'candidates are non-causal' from 'the probe cannot detect single-feature causation'.

The reader's CONDITIONAL verdict is justified. The Section 4 diagnostic—logprob-only evaluation cannot separate feature-specific suppression from magnitude-matched random perturbation—is well supported by held-out splits, isotropic and dictionary-aligned random controls, the Llama L16/name positive control, and threshold/alpha sensitivity; the headline arithmetic reproduces 17/36 and 1/36. The single load-bearing weakness is the secondary causal-mediation corroboration. Because NIE_f is a one-feature, one-position noising ablation and x0/x1 share the prefix, non-mediating features automatically give NIE_f ≈ TE, and the random control gives the same; the (0.5,2.0) threshold was never calibrated against a known mediator. Thus the 97.3% rate cannot be read as support for 'the verifier separates candidates reliably', and Table 1 shows only one cell with any separation. This does not weaken the central diagnostic, but it means the causal-mediation claim should be reported as a feasibility result with a sensitivity caveat, not as independent corroboration. The proposed calibration test would settle the question; until then CONDITIONAL remains the right verdict, so the reader's verdict should be unchanged.

The referee's report

If the Section 4 diagnostic is correct, it is a timely negative result for interpretability-based privacy: the shared evaluation signal of recent SAE-based PII suppression work (PrivacyScalpel, ProPILE, PII-Scope) cannot by itself establish feature-specific suppression. The experimental design is considerably more careful than is typical in this literature: mutually disjoint discovery/validation/synthetic-SAE corpora, two random-control families including same-SAE dictionary directions, a genuinely selective positive control, and sensitivity analyses over thresholds and intervention strength. The promised release of code, corpora, and trained SAEs is a further strength. The causal-mediation feasibility study is the weak link: the proposed verifier is an uncalibrated single-feature, single-position noising probe, and the paper's claim that it 'separates candidates reliably' is contradicted in direction by its own Table 1. The Section 4 claims do not depend on the mediation study, but the abstract's corroboration claim and the conclusion that the bottleneck is candidate generation do.

Pith's reading of the paper (a machine's summary, not the authors')

This paper tries to establish that the standard measure of success for SAE-based PII suppression in LLMs, the reduction of the gold PII token's log-probability, cannot by itself certify that an intervention targeted a PII-specific mechanism: any sufficiently large hidden-state perturbation will lower the probabilities of subsequent tokens regardless of direction. Across 36 (model, layer, PII type) cells, 17 (47%) look successful under a logprob-only gate ($\Delta_{\text{lp}} \leq -0.3$), but only 1 (2.8%, Llama L16/name) survives a conjunctive multi-metric gate that also requires a rank-aware extraction-cost improvement and 5-fold selectivity over coherence-matched random and dictionary-aligned controls. The failures separate into two patterns: random directions frequently outperform the selected feature, and the same top-1 feature can be selected for all four PII types while producing type-asymmetric effects. A 9,000-run causal mediation feasibility study corroborates the diagnosis, finding 146 of 150 activation-rate-selected candidates statistically indistinguishable from a random control. If the paper is right, previously reported SAE-based PII-suppression results are unverified as feature-specific, and logprob-based selection and evaluation should be replaced by causal mediation analysis and circuit-level localization.

What carries the argument. The load-bearing device is the coherence-matched random direction control. The selected feature intervention $h'_t = h_t + \alpha z_t[f] W_{\text{dec}}[f]$ is compared with $h'_t = h_t + \alpha m_t r$, where $r$ is an isotropic random unit vector (or a normalized decoder direction of another feature from the same SAE), $m_t = \|z_t[f]W_{\text{dec}}[f]\|_2$, and the same direction is shared across all intervened positions so that span-level direction coherence is preserved. Because magnitude, position pattern, and coherence are matched, the only remaining difference is the direction, which is exactly the carrier of feature-specificity. Selectivity is the ratio of effect magnitudes, $\text{Sel}_{\text{lp}} = |\Delta^{\text{target}}_{\text{lp}}| / |\Delta^{\text{random}}_{\text{lp}}|$ and analogously $\text{Sel}_{\text{br}}$, and the clean-strong label additionally requires $\Delta_{\text{lp}} \leq -0.3$ and $\Delta_{\text{br}} \geq +1.0$. In the mediation leg, the central identity is $\mathrm{NIE}_f = \mathrm{lp}(y|x_1, f=0) - \mathrm{lp}(y|x_0)$ against $\mathrm{TE} = \mathrm{lp}(y|x_1) - \mathrm{lp}(y|x_0)$, computed with a single-feature zero-ablation because the standard activation-patching NIE is shown to be identically zero when $x_0$ and $x_1$ share all preceding context.

Core claim. On the paper's own terms, the central claim is that using the log-probability of the gold PII span both to select SAE features and to judge whether their ablation or steering suppressed leakage is circular and non-specific: any sufficiently large directional perturbation of the hidden state reproduces the headline effect. The paper operationalizes specificity as beating a coherence-matched control by at least a factor of five on both the logprob metric and a rank-aware extraction-cost proxy, where the control preserves intervention strength, token-wise magnitudes, and span-level direction coherence and differs only in direction. Across three 8B/9B LoRA-fine-tuned models, only one of 36 configurations, Llama L16/name, passes all four conjunctive criteria ($\Delta_{\text{lp}} \leq -0.3$, $\Delta_{\text{br}} \geq +1.0$, $\text{Sel}_{\text{lp}} \geq 5$, $\text{Sel}_{\text{br}} \geq 5$), and this collapse from 47% to 2.8% survives threshold and intervention-$\alpha$ ($\alpha \in \{-1,-2,-3,-4,-5\}$) robustness checks. The failures are non-exclusive: random-dominant cells (e.g., Gemma L20/name, where 20/20 dictionary-aligned controls suppress more than the selected direction) and feature-selection-collapse cells (e.g., Llama L8, where one feature is selected for all PII types yet its effect is 2.3x stronger on name than on the others at matched activation magnitudes). A feasibility study with 9,000 causal patching runs, using a noising-style single-feature zero-ablation $\mathrm{NIE}_f = \mathrm{lp}(y|x_1, f=0) - \mathrm{lp}(y|x_0)$ against $\mathrm{TE} = \mathrm{lp}(y|x_1) - \mathrm{lp}(y|x_0)$, finds 146 of 150 (97.3%) activation-rate-selected candidates within a factor of two of the random control, with the sole exception being Gemma-2-9B/name (11 of 15 distinguishable); the paper concludes that the mediation verifier works and that the bottleneck is candidate generation.

The premise it rests on. The weakest load-bearing assumption is that zero-ablating one SAE feature at the single token immediately before the PII span is a sensitive enough causal probe to detect a genuine PII mediator; the authors adopt this noising-style probe because the standard activation-patching NIE is identically zero at that position, but they do not calibrate its sensitivity, so the 97.3% 'indistinguishable from random' rate could be a mechanical consequence of a weak probe rather than evidence that the features are causally inert.

If the paper is right (Pith's inferences, not the authors')

SAE-based PII suppression studies that report only logprob or leakage-rate reductions cannot be taken as evidence of feature-specific intervention; the 35 non-clean-strong cells are consistent with generic magnitude artifacts.

The conclusion is threshold-independent: loosening the gates still yields at most 3/36 (8.3%) clean-strong cells, so the failure of logprob-only evaluation is not an artifact of strict cutoffs.

Genuine feature-specific effects exist and are detectable: Llama L16/name passes all criteria across all nine templates, including out-of-distribution Korean and JSON formats, and preserves downstream accuracy on PubMedQA, MedQA, and MMLU.

Causal mediation analysis is implementable for PII leakage with 9,000 runs and correctly separates cells by whether memorization is present; the bottleneck it reveals is causally motivated candidate generation, to be addressed by circuit-level localization.

Extraction-attack benchmarks should be used alongside feature-specificity diagnostics, not instead of them, since extraction-rate drops can stem from general generation degradation.

Editorial extension: the coherence-matched random control could become a standard sanity check for other SAE steering applications (refusal, honesty, safety) that evaluate success by likelihood changes, because the non-specificity critique is not unique to PII.

Editorial extension: the paper's own Table 1 suggests the mediation probe may be underpowered: the only distinguishable cell has a modest positive memorization signal, while the strongly memorized email cells (TE > 2.4) yield no distinguishable candidate at the single-token probe; a multi-token ablation is the natural next test.

Editorial extension: strong email memorization coexisting with no single distinguishable feature is consistent with a distributed causal pathway, which would explain why single-feature interventions fail and would motivate circuit-level rather than single-feature editing.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

SAE-based feature-level interventions to suppress PII leakage in LLMs are primarily evaluated by log-probability reduction. We show that this criterion cannot distinguish feature specific suppression from a magnitude artifact: a direction-agnostic effect of sufficiently large activation perturbations. Across 36 (model, layer, PII type) cases, we compare selected SAE feature interventions with coherencematched random direction controls using logprob and rank-aware metrics. While 17 cases (47%) appear successful under logprob-only evaluation, only 1 case (2.8%) passes our multimetric framework. The rest reveal two failure modes: random directions outperform selected features, and feature selection collapse across PII types. These results indicate that existing evaluations cannot verify whether interventions target causal pathways of PII leakage. A 9,000 run causal mediation feasibility study corroborates this: the verifier separates candidates reliably, but correlation-selected candidates rarely pass it. We therefore propose a new research mission for NLP: causally grounded PII suppression through circuit-level localization and causal mediation analysis.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-04

The authors' main claim, as the classifier picked it out 0.89

“These results indicate that existing evaluations cannot verify whether interventions target causal pathways of PII leakage.”

How the main claim is stated: hedged possibility 0.95

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.31 · human participants 0.03 · controlled comparison 0.98 · randomized assignment 0.08 · states quantitative result 0.99 · compares with prior results 0.92 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
SAE-based feature-level interventions to suppress PII leakage in LLMs are primarily evaluated by log-probability reduction. background 1.00 0.07 0.03 asserted 1.00
We show that this criterion cannot distinguish feature specific suppression from a magnitude artifact: a direction-agnostic effect of sufficiently large activation perturbations. result 0.64 0.87 0.30 asserted 1.00
Across 36 (model, layer, PII type) cases, we compare selected SAE feature interventions with coherencematched random direction controls using logprob and rank-aware metrics. method 0.93 0.95 0.04 asserted 0.99
While 17 cases (47%) appear successful under logprob-only evaluation, only 1 case (2.8%) passes our multimetric framework. result 1.00 0.96 0.27 asserted 1.00
The rest reveal two failure modes: random directions outperform selected features, and feature selection collapse across PII types. result 0.97 0.92 0.12 asserted 1.00
These results indicate that existing evaluations cannot verify whether interventions target causal pathways of PII leakage. interpretation 1.00 0.61 0.63 hedged possibility 0.95
A 9,000 run causal mediation feasibility study corroborates this: the verifier separates candidates reliably, but correlation-selected candidates rarely pass it. result 0.87 0.91 0.13 asserted 1.00
We therefore propose a new research mission for NLP: causally grounded PII suppression through circuit-level localization and causal mediation analysis. future work 1.00 0.15 0.36 asserted 0.89

What kind of paper the classifier reports this is: empirical study 0.97 · evidence from observation or experiment 0.84

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-04: 1 request, 6,205 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (3)
Figure 1 of the paper
Figure 1. Main diagnostic results across 36 (model, layer, PII type) cells. (a) Number of cells passing single-metric
Figure 2 of the paper
Figure 2. Selected feature intervention effect (|∆ target lp |, y-axis) versus coherence-matched random direction effect (|∆random lp |, x-axis), per (model, layer, PII type) configuration; both axes log scale. Large markers indicate cell mean over 9 templates × 50 validation pairs (450 evaluation contexts); small dots indicate per-template means within a cell. Gray dashed: Sellp = 1 (target = random). Orange dashed line: Sellp = 5 (clean-strong selectivity threshold). Diamond (♦): cell satisfying all clean-strong criteria; hollow markers: inactive configurations (zt[f] ≈ 0). Of 36…
Figure 3 of the paper
Figure 3. α sensitivity on three representative cells. Yellow band: main-paper setting (α = −2). (a) Llama L16/name (clean-strong): selected-feature dominance is preserved across all tested α. (b) Gemma L20/name (Failure mode A): random-direction dominance is preserved across all α, including the over-perturbation regime at α = −5. (c) Qwen L18/name (clean-weak): sellp exceeds 1 at α = −1, −2 but crosses below 1 from α = −3 onward, indicating that the weak selectivity observed in the main experiment is itself α-dependent. intervention applied exactly as in the main experi￾ments (α = −2,…

Figures are the authors', reproduced from the paper.

References (59)
  1. [1] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Alex Tamkin, Karina Nguyen, Brayden McLe... on Pith
  2. [2] Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tram\` e r, and Chiyuan Zhang. 2023. https://openreview.net/forum?id=TatRHT_1cK Quantifying memorization across neural language models . In The Eleventh International Conference on Learning Representations on Pith
  3. [3] Nicholas Carlini, Florian Tram\` e r, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, \' U lfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extr... on Pith
  4. [5] Zhiyu Zoey Chen, Jing Ma, Xinlu Zhang, Nan Hao, An Yan, Armineh Nourbakhsh, Xianjun Yang, Julian McAuley, Linda Petzold, and William Yang Wang. 2024 b . http://arxiv.org/abs/2405.01769 A survey on large language models for critical societal domains: Finance, healthcare, and la... on PithPith's reviewarXiv
  5. [6] Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. http://arxiv.org/abs/2309.08600 Sparse autoencoders find highly interpretable features in language models . Computing Research Repository, arXiv:2309.08600 on PithPith's reviewarXiv
  6. [8] Leo Gao, Tom Dupr\' e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025. https://openreview.net/forum?id=nZeVKeeFYf9 Scaling and evaluating sparse autoencoders . In The Thirteenth International Conference on Learning... on Pith
  7. [10] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. https://arxiv.org/abs/2407.21783 The Llama 3 herd of models . arXiv preprint arXiv:2407.21783 on PithPith's reviewarXiv
  8. [11] Thomas Heap, Tim Lawson, Lucy Farnik, and Laurence Aitchison. 2025. Sparse autoencoders can interpret randomly initialized transformers. Computing Research Repository, arXiv:2501.17727 on PitharXiv
  9. [12] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 LoRA : Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representations on Pith
  10. [13] Daphne Ippolito, Florian Tram\` e r, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, and Nicholas Carlini. 2022. http://arxiv.org/abs/2210.17546 Preventing verbatim memorization in language models gives a false sense of privacy . Com... on PithPith's reviewarXiv
  11. [15] Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. 2025. https://aclanthology.org/2025.coling-main.37/ Exploring concept depth: How large language models acqu... on Pith
  12. [16] Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2023. https://arxiv.org/abs/2307.01881 ProPILE : Probing privacy leakage in large language models . In Advances in Neural Information Processing Systems 36 (NeurIPS 2023) on PithPith's reviewarXiv
  13. [17] Anton Korznikov, Andrey Galichin, Alexey Dontsov, Oleg Rogov, Ivan Oseledets, and Elena Tutubalina. 2026. http://arxiv.org/abs/2602.14111 Sanity checks for sparse autoencoders: Do SAE s beat random baselines? Computing Research Repository, arXiv:2602.14111 on PithPith's reviewarXiv
  14. [19] Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, and Byron C. Wallace. 2021. https://aclanthology.org/2021.naacl-main.73 Does BERT pretrained on clinical notes reveal sensitive data? In Proceedings of the 2021 Conference of the North American Chapter of the Association... on Pith
  15. [20] Maxime M \'e loux, Giada Dirupo, Fran c ois Portet, and Maxime Peyrard. 2025. The dead salmons of AI interpretability. Computing Research Repository, arXiv:2512.18792 on PitharXiv
  16. [21] Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT . In Advances in Neural Information Processing Systems, volume 35 on Pith
  17. [22] Krishna Kanth Nakka, Ahmed Frikha, Ricardo Mendes, Xue Jiang, and Xuebing Zhou. 2024. https://arxiv.org/abs/2410.06704 PII-Scope : A benchmark for training data PII leakage assessment in LLM s . Computing Research Repository, arXiv:2410.06704 on PithPith's reviewarXiv
  18. [23] Judea Pearl. 2009. Causality: Models, Reasoning, and Inference, 2nd edition. Cambridge University Press, Cambridge, UK on Pith
  19. [24] Judea Pearl and Dana Mackenzie. 2018. The Book of Why: The New Science of Cause and Effect. Basic Books, New York, NY on Pith
  20. [25] Raphael Ronge, Markus Maier, and Frederick Eberhardt. 2026. http://arxiv.org/abs/2601.03047 When the coffee feature activates on coffins: An analysis of feature extraction and steering for mechanistic interpretability . Computing Research Repository, arXiv:2601.03047 on PitharXiv
show 39 more
  1. [27] Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. https://proceedings.neurips.cc/paper/2020/hash/92650b2e92217715fe312e6fa7b90d82-Abstract.html Investigating gender bias in language models using causal mediation ... on Pith
  2. [28] Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2022. http://arxiv.org/abs/2211.00593 Interpretability in the wild: a circuit for indirect object identification in GPT -2 small . Computing Research Repository, arXiv:2211.00593 on PithPith's reviewarXiv
  3. [29] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. http://arxiv.org/abs/2505.09388 Qwen3 technical report . Computing Research Repository, arXiv:2505.09388 on PithPith's reviewarXiv
  4. [30] Alfred V. Aho and Jeffrey D. Ullman , title =. 1972 on Pith
  5. [31] Publications Manual , year = "1983", publisher = on Pith
  6. [32] Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243 on Pithdoi:10.1145/322234.322243
  7. [33] Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of on Pith
  8. [34] Dan Gusfield , title =. 1997 on Pith
  9. [35] Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url = on Pith
  10. [36] Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages = on Pith
  11. [37] Zhiyu Zoey Chen and Jing Ma and Xinlu Zhang and Nan Hao and An Yan and Armineh Nourbakhsh and Xianjun Yang and Julian McAuley and Linda Petzold and William Yang Wang , title =. Computing Research Repository , volume =. 2024 , url = on Pith
  12. [38] Nicholas Carlini and Florian Tram\`. Extracting Training Data from Large Language Models , booktitle =. 2021 , url = on Pith
  13. [39] Nicholas Carlini and Daphne Ippolito and Matthew Jagielski and Katherine Lee and Florian Tram\`. Quantifying Memorization Across Neural Language Models , booktitle =. 2023 , url = on Pith
  14. [40] Daphne Ippolito and Florian Tram\`. Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy , journal =. 2022 , url = on Pith
  15. [41] Ruizhe Chen and Tianxiang Hu and Yang Feng and Zuozhu Liu , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages =. 2024 , address =. doi:10.18653/v1/2024.acl-short.25 , url = on Pithdoi:10.18653/v1/2024.acl-short.25
  16. [42] Trenton Bricken and Adly Templeton and Joshua Batson and Brian Chen and Adam Jermyn and Tom Conerly and Nicholas L. Turner and Cem Anil and Carson Denison and Amanda Askell and Robert Lasenby and Yifan Wu and Shauna Kravec and Nicholas Schiefer and Tim Maxwell and Nicholas Jos... on Pith
  17. [43] Ahmed Frikha and Muhammad Reza Ar Razi and Krishna Kanth Nakka and Ricardo Mendes and Xue Jiang and Xuebing Zhou , title =. Proceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages =. 2025 , address =. doi:10.18653/v1/2025.blackbo... on Pithdoi:10.18653/v1/2025.blackboxnlp-1.13
  18. [44] Anton Korznikov and Andrey Galichin and Alexey Dontsov and Oleg Rogov and Ivan Oseledets and Elena Tutubalina , title =. Computing Research Repository , volume =. 2026 , url = on Pith
  19. [45] Raphael Ronge and Markus Maier and Frederick Eberhardt , title =. Computing Research Repository , volume =. 2026 , url = on Pith
  20. [46] Ilya Lasy and Peter Knees and Stefan Woltran , title =. Proceedings of the First Workshop on Large Language Model Memorization (L2M2) , pages =. 2025 , address =. doi:10.18653/v1/2025.l2m2-1.7 , url = on Pithdoi:10.18653/v1/2025.l2m2-1.7
  21. [47] Jesse Vig and Sebastian Gehrmann and Yonatan Belinkov and Sharon Qian and Daniel Nevo and Yaron Singer and Stuart Shieber , title =. Advances in Neural Information Processing Systems , volume =. 2020 , url = on Pith
  22. [48] Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov , title =. Advances in Neural Information Processing Systems , volume = on Pith
  23. [49] Hoagy Cunningham and Aidan Ewart and Logan Riggs and Robert Huben and Lee Sharkey , title =. Computing Research Repository , volume =. 2023 , url = on Pith
  24. [50] Kevin Wang and Alexandre Variengien and Arthur Conmy and Buck Shlegeris and Jacob Steinhardt , title =. Computing Research Repository , volume =. 2022 , url = on Pith
  25. [51] Judea Pearl , title =. 2009 , edition = on Pith
  26. [52] Judea Pearl and Dana Mackenzie , title =. 2018 , publisher = on Pith
  27. [53] Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , title =. The Tenth International Conference on Learning Representations , year = on Pith
  28. [54] Leo Gao and Tom Dupr\'. Scaling and Evaluating Sparse Autoencoders , booktitle =. 2025 , url = on Pith
  29. [55] Ian Tenney and Dipanjan Das and Ellie Pavlick , title =. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages =. 2019 , address =. doi:10.18653/v1/P19-1452 , url = on Pithdoi:10.18653/v1/p19-1452
  30. [56] Ganesh Jawahar and Beno\^. What Does. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages =. 2019 , address =. doi:10.18653/v1/P19-1356 , url = on Pithdoi:10.18653/v1/p19-1356
  31. [57] Mingyu Jin and Qinkai Yu and Jingyuan Huang and Qingcheng Zeng and Zhenting Wang and Wenyue Hua and Haiyan Zhao and Kai Mei and Yanda Meng and Kaize Ding and Fan Yang and Mengnan Du and Yongfeng Zhang , title =. Proceedings of the 31st International Conference on Computational... on Pith
  32. [58] Grattafiori, Aaron and Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and Letman, Aiesha and Mathur, Akhil and Schelten, Alan and Vaughan, Alex and others , journal =. The. 2024 , url = on Pith
  33. [59] An Yang and Anfeng Li and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Gao and Chengen Huang and Chenxu Lv and others , title =. Computing Research Repository , volume =. 2025 , url = on Pith
  34. [60] arXiv preprint arXiv:2408.00118 , year = on PithPith's reviewarXiv
  35. [61] Eric Lehman and Sarthak Jain and Karl Pichotta and Yoav Goldberg and Byron C. Wallace , title =. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages =. 2021 , address = on Pith
  36. [62] Siwon Kim and Sangdoo Yun and Hwaran Lee and Martin Gubri and Sungroh Yoon and Seong Joon Oh , title =. Advances in Neural Information Processing Systems 36 (NeurIPS 2023) , year = on Pith
  37. [63] Krishna Kanth Nakka and Ahmed Frikha and Ricardo Mendes and Xue Jiang and Xuebing Zhou , title =. Computing Research Repository , volume =. 2024 , url = on Pith
  38. [64] Sparse Autoencoders Can Interpret Randomly Initialized Transformers , author=. Computing Research Repository , volume= on Pith
  39. [65] M. The Dead Salmons of. Computing Research Repository , volume= on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment