Pith. sign in

REVIEW 3 major objections 4 minor 60 references

Measuring Semantic Abstractness of SAE Features via Nonlocality

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read A gradient-based entropy called Feature Nonlocality is claimed to measure the semantic abstractness of a model's internal features without labels or LLM judges.

desk verdict The FNL metric is a useful, label-free tool for ranking SAE features by abstraction; the paper is honest and reproducible, with the main caveats being no causal-attribution baseline and a BOS convention that should be flagged. read the letter →

arxiv 2608.10537 v1 pith:QLLCVPYS submitted 2026-08-11 cs.AI cs.LG

classification cs.AIcs.LG
keywords FeatureNonlocalitysparseautoencodersmechanisticinterpretabilitysemanticabstractiongradientinfluencecontextualreachjailbreakmitigationactivationsteering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Feature Nonlocality (FNL), defined as the entropy of the normalized per-position gradient influence that an SAE feature's activation receives from each input token. Its claim is that FNL is an LLM-independent, label-free, correlational witness of a feature's semantic abstraction level: token-level and positional features score low, while context-dependent reasoning features score high. The paper supports this with correlations against token-injection recovery and paraphrase robustness, with pair classification AUC of 0.73–0.84, and with two downstream case studies: auditing jailbreak-mitigation features and steering high-FNL features for math reasoning. If true, FNL gives interpretability researchers a cheap, quantitative axis for deciding whether a feature genuinely encodes a concept or merely reads a lexical cue.

What carries the argument

The load-bearing object is Feature Nonlocality (FNL). Given an SAE feature $a$ firing at the last token of a prompt $P$ with context window $T$, the paper defines the per-position influence of token $t$ as $J_a(t,T,P) = \|\partial z_a(T,P)/\partial x_t\|^2$, the squared norm of the gradient of the feature's activation with respect to the token embedding. Normalizing these influences to a probability distribution $p_a^{(P)}(t)$ and taking the Shannon entropy $H(a,P) = -\sum_{t=1}^T p_a^{(P)}(t)\log_2 p_a^{(P)}(t)$ yields the per-prompt nonlocality; dataset-level FNL averages this over firing events. The entropic form is what lets a single-token feature score near zero while a feature that reads evidence from everywhere across the window scores several bits; the paper also shows that FNL ranking survives across corpora, rises with layer depth before saturating, and tracks decoder-direction geometry rather than semantic similarity.

What would settle it

Find a feature with high FNL whose activation is nevertheless entirely driven by a single token: patching or ablating all other prefix tokens leaves the activation unchanged, while removing that one token silences it. That would show the gradient entropy can be large without any real contextual dependence, breaking the claimed link between FNL and semantic abstraction.

Watch

Extended reading notes

Core claim

The central discovery is that the contextual reach of an SAE feature can be read off from a single backward pass: for each firing event, normalize the squared gradient norm of the feature's activation with respect to every prefix token to a probability distribution, take its Shannon entropy, and average over firing events. The paper argues that this quantity, Feature Nonlocality, tracks the level of semantic abstraction: single-token detectors such as a "Robert" indicator sit near zero bits, while an epistemic-uncertainty marker spreads influence across a passage and reaches about five bits on a 128-token window. Empirically FNL correlates negatively with token-injection recovery (Spearman about −0.39 to −0.46), positively with paraphrase robustness (0.27), and separates token-driven from context-dependent reasoning features with 73–84% AUC. On the audit side, the paper finds that most effective jailbreak-mitigation features are positional features with zero FNL rather than content-recognizing features, and on the intervention side, steering a high-FNL envelope improves MATH-500 accuracy by 4.6 points over unsteered DeepSeek-R1-Distill-Llama-8B, though the authors present the steering result as a proof of concept with model-specific gains.

Load-bearing premise

The load-bearing premise is that the squared gradient of a feature's activation with respect to each input token faithfully measures that token's causal contribution to the feature firing, so the entropy of the resulting distribution genuinely reflects how much context the feature reads.

Editorial extensions

If this is right

  • Feature selection for interpretability audits can be done with gradients alone, removing the need for curated contrastive datasets and LLM-generated feature descriptions when the goal is to separate token-level from context-level features.
  • A steered feature's behavioral effectiveness no longer implies conceptual content: a feature that defends against jailbreaks can do so by marking a positional artifact, so intervention results should be audited with a contextual-reach measure before drawing mechanistic conclusions.
  • FNL gives a continuous axis of abstraction that is stable across corpora, so a single dictionary-wide ranking could support cross-task and cross-model comparisons of where semantic processing happens in a network.
  • In the paper's proof of concept, a high-FNL envelope beat low-FNL and random envelopes on MATH-500 for the reasoning-distilled model, suggesting that contextual-reach ranking could be a viable label-free selection criterion for reasoning interventions, although the paper notes the gains are model-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural validation would be to compare FNL against activation patching or token-ablation ground truth on a handful of features; if gradients overstate the influence of positions that patching shows are irrelevant, FNL would need reweighting before it is used as an audit tool.
  • The jailbreak-mitigation audit implies a concrete safety lesson not drawn by the paper: defenses that work by low-FNL positional features may fail as soon as the attack moves the harmful request out of the beginning-of-sequence position, so wrapper defenses should be tested at varied positions and formats.
  • Because FNL is defined for any linear subspace of the residual stream, not just SAE features, it could serve as a common abstraction axis across interpretability methods such as probes, steering vectors, and circuit analyses, letting results from different toolkits be compared on the same scale.
  • A dictionary-wide FNL scan plus a small steering experiment could become a standard screening step: compute FNL for all features, audit the tails, and use the high-FNL envelope as a candidate feature set, with the low-FNL tail as a natural control for confounding positional effects.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Feature Nonlocality (FNL), defined as the entropy of the normalized per-position squared-gradient influence on an SAE feature's activation at the final token position. The authors claim that FNL is an LLM-independent, label-free correlational witness of the semantic abstractness of SAE features. They validate FNL against two existing proxy metrics: token-injection activation recovery (Spearman rho between -0.39 and -0.46 across layers/dictionaries) and paraphrase robustness (rho = 0.27), and they report that FNL separates token-driven from context-dependent features with AUC 0.73-0.84. Two downstream applications are presented: an audit of jailbreak-mitigation features, where 21 of 25 effective features are reported to have zero FNL and are characterized as positional/BOS indicators, and a proof-of-concept steering study on DeepSeek-R1-Distill-Llama-8B in which steering high-FNL features improves MATH-500 accuracy by 4.6 points over the unsteered baseline, outperforming low-FNL and random envelopes. The paper includes substantial reproducibility details, including seeds, hyperparameter tables, and bootstrap confidence intervals.

Significance. If the central claim is supported, FNL would be a useful and economical tool for feature selection and for diagnosing whether an SAE feature is a surface-level token detector or a genuinely context-dependent representation, with potential value in safety auditing and interpretability pipelines. The paper is careful in several ways: the cross-dataset stability analysis uses split-half reliability ceilings (Table S1), the steering experiment is explicitly labeled a proof-of-concept, and the code repository includes fixed seeds and detailed run configurations. However, the strength of the contribution depends on whether the gradient-based influence measure faithfully tracks causal influence, and on whether the reported jailbreak-audit result is partially manufactured by the metric's support convention. The correlation magnitudes are modest, and the steering gains are model-specific and tuned on held-out subsets, so the central claims should be treated as promising but not fully established.

major comments (3)
  1. [Method (Definition 1, Eq. (4))] The paper identifies the squared gradient norm ||dz_a(T,P)/dx_t||^2 with the per-position causal influence of input token t on feature activation. This is a local linear sensitivity measure, not an intervention-based causal quantity. The validation against token-injection recovery is at risk of being partly definitional: a feature whose activation is easily moved by local token perturbations will have concentrated gradients (low FNL), and a feature insensitive to single-token changes will have diffuse gradients (high FNL). The paper does not report a causal-patching, activation-patching, or alternative attribution baseline to show that FNL tracks intervention-based influence. To support the central claim that FNL measures 'contextual reach,' please add a comparison with a causal attribution method (e.g., activation patching across prefix positions) and report the correlation between FNL and intervention-based nonlocality, as well as a control that accounts for gradient concentration at the firing token itself.
  2. [Supplementary: Beginning-of-sequence convention; Table 3] The supplementary 'Beginning-of-sequence convention' states that the window-initial position, which hosts the attention sink, is excluded from the influence support. As a result, any feature whose activation is driven by position 0 has FNL identically zero by construction. This is exactly how 21 of the 25 CC-Delta jailbreak-mitigation features in Table 3 are classified. The paper presents this as a striking mechanistic finding ('most effective features are positional features with low FNL'), but the zero-FNL classification is partly an artifact of the support convention. The steering decomposition in Table 3 provides independent evidence for the positional nature of these features, but the FNL-based characterization does not. Please re-compute FNL for these features with the initial position included (or with a BOS token in the window) and report whether the 21 features still have low FNL; otherwise, make explicit that the zero-nonlocality result is a direct consequence of the exclusion convention.
  3. [FNL-Guided Steering; Table 4] The main-text steering claim rests on a comparison of high-FNL, low-FNL, and random envelopes in Table 4. The steering gain gamma is selected per condition on held-out MATH-500 subsets ('we perform pilot sweeps over steering strengths and postselect the best-performing gamma for a held-out subset of MATH-500'). Because the high, low, and random arms may be tuned to different strengths, the reported +4.6 vs +3.8 vs +3.6 differences could reflect steering-strength differences rather than FNL itself. The supplementary bootstrap intervals in Table S4 partially mitigate this, but the main text should either report the gamma values used for each arm in Table 4 or state that the strengths were matched across arms. This point is load-bearing for the steering application, even though the paper labels the experiment as a proof of concept.
minor comments (4)
  1. [Method: Feature Nonlocality] The sentence 'The correlation bewteen FNL and semantic abstractness will be discussed later in Sec. .' contains an undefined cross-reference; please fill in the correct section number.
  2. [Data and Code Availability] The text states that 'All figures and tables in this work are reproducible in CPU minutes,' while the supplementary reproducibility section reports on the order of 1,000 GPU-hours of runtime. Please reconcile these statements, or specify that the plotting/analysis code for figures runs in CPU minutes while the underlying gradient and inference computations require GPU-hours.
  3. [Abstract and Introduction] The abstract and introduction describe FNL as 'LLM-independent,' but the validation pipeline uses LLM-generated paraphrases and LLM judges in several places. The metric itself is LLM-free, so consider using 'LLM-free' or 'computed without LLM calls' to avoid ambiguity.
  4. [Table 3] In Table 3, the all-25-feature direction (held-out safety 0.887) is outperformed by the 21-feature positional subset (0.911) even though the all-25 direction has a larger norm (||v||=3.19 vs 2.04). The text says the positional subset 'accounts for most of the improvement,' but the relative ordering suggests that adding the four content features may actually reduce held-out safety. A brief explanation of this non-monotonicity would help the reader interpret the decomposition.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: FNL is independently defined and externally validated; the BOS support convention is disclosed and does not manufacture the empirical positional finding.

full rationale

FNL (Definition 1, Eqs. 4-6) is defined from squared-gradient influences with no fitted parameters and no dependence on the proxy labels used for validation. The token-injection and paraphrase-robustness pipelines of Ma et al. (2026) are re-implemented and reproduced nearly exactly (class counts 46/12/21/21 vs their 46/12/19/23), and FNL is computed independently by backward passes on the same features; the reported Spearman correlations and AUCs are empirical outcomes, not quantities solved for by the definition. The negative correlation with injection recovery is mechanistically expected from a gradient-entropy measure, but it is not an equation identity, and the positive paraphrase-robustness correlation plus auto-interp, geometry, and steering experiments supply independent evidence. The Supplementary 'Beginning-of-sequence convention' is a disclosed limitation: excluding the window-initial/attention-sink position makes position-0-driven features have zero FNL by definition, and the paper says so explicitly ('giving zero contextual reach by definition'); importantly, the audit also reports empirically that these features activate almost exclusively at position-0, so the 'positional' characterization is not manufactured by the support convention. The only self-citation, the Agentic Publication Protocol (Lu and Qi 2026), concerns publication format and is not load-bearing for any scientific claim. The steering gain gamma is tuned on held-out MATH-500 subsets, a fitting step in the application rather than in the definition of FNL, and the paper labels the steering study a proof-of-concept. No significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the faithfulness of gradient-based influence as a measure of contextual reach, on the validity of external proxies for abstraction, and on the SAE/linear-representation framework. The paper introduces no new physical or ontological entities beyond the FNL metric itself, which is operationalized as an entropy over existing gradient quantities.

free parameters (5)
  • Context window T = 128 (64 in some runs)
    Context length used for FNL computation; chosen based on approximate saturation of H across T in [8,2048] (Supplementary Table S5).
  • Firing events per feature k = 32 (60 in stability tests)
    Number of top-k max-activation events averaged per feature; chosen for estimator stability (Supplementary Table S5).
  • Firing threshold tau = 0, 0.2
    Threshold above which a feature is considered firing; depends on SAE gate (ReLU or JumpReLU).
  • Steering gain gamma = per cell, e.g., 1.10, 1.2, 2, 5
    Multiplicative gain for clamping steering, selected per experiment on held-out tuning problems (largest gain with no new generation failures).
  • Envelope fraction = 20%, 10%, 5%
    Fraction of dictionary steered as high/low FNL envelope; fixed per experiment.
assumptions (4)
  • domain assumption SAE features are meaningful linear directions in the residual stream (Linear Representation Hypothesis).
    The paper's framing of features as causal directions relies on this hypothesis, cited from Park, Choe, and Veitch 2024 and Elhage et al. 2022, without independent proof.
  • ad hoc to paper The squared gradient norm ||dz_a/dx_t||^2 faithfully measures the causal influence of input position t on feature activation z_a.
    This is the core modeling choice of Definition 1, Eq. (4); the paper motivates it as 'contextual reach' but does not verify it against causal patching or attribution baselines.
  • domain assumption The token-injection recovery and paraphrase-robustness scores from Ma et al. (2026) are valid proxies for semantic abstractness.
    FNL correlations are validated against these external proxies; if the proxies are flawed, the validation is weakened.
  • domain assumption Excluding the window-initial position from the influence support does not change feature orderings.
    Disclosed in the Beginning-of-sequence convention paragraph of the supplementary; the ordering claim is plausible but not demonstrated for the jailbreak features that fire at position 0.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Measuring Semantic Abstractness of SAE Features via Nonlocality." pith.science (2026). https://pith.science/paper/QLLCVPYS

@misc{pith2026260810537,
  author       = {Pith},
  title        = {Pith review of: Measuring Semantic Abstractness of SAE Features via Nonlocality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QLLCVPYS}},
  note         = {Machine review of arXiv:2608.10537}
}
abstract

Sparse autoencoders (SAEs) have helped uncover mechanistic explanations for LLM behaviours such as reasoning, jailbreaking etc., via understanding the corresponding task-relevant and causally effective features. To evaluate such mechanistic explanations, downstream studies must distinguish surface lexical features from genuinely high-level ones. However, neither an autointerp-based semantic description nor causal steering utility fully resolves the abstraction level of a feature. To this end, we introduce \emph{Feature Nonlocality} (FNL), defined as the entropy of the normalized per-position influence on an SAE feature's activation. We report that FNL correlates with existing LLM-based proxy metrics of feature semantic abstractness, and successfully distinguishes context-dependent reasoning features from token-driven ones, correctly assigning the higher FNL to the contextual feature in $73$--$84\%$ of randomly drawn pairs that consist of one contextual and one token-level feature. We demonstrate two downstream applications. We audit SAE-based features used for jailbreak mitigation and find surprisingly that most effective features are positional features with low FNL rather than genuinely recognizing harmful intents. We report that steering high-FNL features in DeepSeek-R1-Distill-Llama-8B improves MATH-500 accuracy by $4.6$ points over the unsteered model and outperforms steering low-FNL features, though the gains are model-specific. We conclude that FNL provides an LLM-independent, label-free, correlational witness of the abstraction level of an SAE feature, with applications in evaluating mechanistic explanations as well as selecting features for downstream interventions.

Figures

Figures reproduced from arXiv: 2608.10537 by the authors.

Figure 1
Figure 1. GemmaScope feature nonlocality as a function of [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Decoder-cosine geometry pins H(a) (Gemma-2- 2B layer 12). For each of six anchor features we take its top-10 decoder-cosine neighbours and plot every feature’s H(a). For each individual feature, the computation of FNL samples top-k (k = 32) max-activation firing events across a sample WikiText corpus; the error bar represents the IQR of FNL statistics. Nonlocality and Semantic Abstractness In this section, we presen… view at source ↗
Figure 3
Figure 3. Feature nonlocality traces a concrete-to-abstract continuum (Gemma-2-2B layer 12, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Paraphrase robustness [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 30 canonical work pages

  1. [1]

    Agentic Publication Protocol: An Attempt to Modernize Scientific Publication

    Agentic Publication Protocol: An Attempt to Modernize Scientific Publication , author =. arXiv preprint arXiv:2606.27386 , year =

  2. [2]

    Lieberum, Tom and Rajamanoharan, Senthooran and Conmy, Arthur and Smith, Lewis and Sonnerat, Nicolas and Varma, Vikrant and Kram. Gemma. arXiv preprint arXiv:2408.05147 , year =

  3. [3]

    arXiv preprint arXiv:1609.07843 , year =

    Pointer Sentinel Mixture Models , author =. arXiv preprint arXiv:1609.07843 , year =

  4. [4]

    Transformer Circuits Thread , year =

    A Mathematical Framework for Transformer Circuits , author =. Transformer Circuits Thread , year =

  5. [5]

    and McDougall, Callum and MacDiarmid, Monte and Freeman, C

    Templeton, Adly and Conerly, Tom and Marcus, Jonathan and Lindsey, Jack and Bricken, Trenton and Chen, Brian and Pearce, Adam and Citro, Craig and Ameisen, Emmanuel and Jones, Andy and Cunningham, Hoagy and Turner, Nicholas L. and McDougall, Callum and MacDiarmid, Monte and Freeman, C. Daniel and Sumers, Theodore R. and Rees, Edward and Batson, Joshua and...

  6. [6]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =

  7. [7]

    Guha, Etash and Marten, Ryan and Keh, Sedrick and Raoof, Negin and Smyrnis, Georgios and Bansal, Hritik and others , journal =. Open

  8. [8]

    arXiv preprint arXiv:2403.19647 , year =

    Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models , author =. arXiv preprint arXiv:2403.19647 , year =

Show all 60 references
  1. [9]

    arXiv preprint arXiv:2501.09929 , year =

    Interpretable Steering of Large Language Models with Feature Guided Activation Additions , author =. arXiv preprint arXiv:2501.09929 , year =

  2. [10]

    He, Zirui and Jin, Mingyu and Shen, Bo and Payani, Ali and Zhang, Yongfeng and Du, Mengnan , journal =

  3. [11]

    The Twelfth International Conference on Learning Representations (ICLR) , year =

    Sparse Autoencoders Find Highly Interpretable Features in Language Models , author =. The Twelfth International Conference on Learning Representations (ICLR) , year =

  4. [12]

    Transformer Circuits Thread , year =

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author =. Transformer Circuits Thread , year =

  5. [13]

    OpenAI Blog , year =

    Language Models Can Explain Neurons in Language Models , author =. OpenAI Blog , year =

  6. [14]

    Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (

    Kim, Been and Wattenberg, Martin and Gilmer, Justin and Cai, Carrie and Wexler, James and Vi. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (. Proceedings of the 35th International Conference on Machine Learning (ICML) , pages =

  7. [15]

    International Conference on Learning Representations (ICLR) Workshop Track , year =

    Understanding Intermediate Layers Using Linear Classifier Probes , author =. International Conference on Learning Representations (ICLR) Workshop Track , year =

  8. [16]

    Advances in Neural Information Processing Systems 37 (NeurIPS) , year =

    Refusal in Language Models Is Mediated by a Single Direction , author =. Advances in Neural Information Processing Systems 37 (NeurIPS) , year =

  9. [17]

    Conference on Language Modeling (COLM) , year =

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author =. Conference on Language Modeling (COLM) , year =

  10. [18]

    Proceedings of the 41st International Conference on Machine Learning (ICML) , year =

    The Linear Representation Hypothesis and the Geometry of Large Language Models , author =. Proceedings of the 41st International Conference on Machine Learning (ICML) , year =

  11. [19]

    Transformer Circuits Thread , year =

    Toy Models of Superposition , author =. Transformer Circuits Thread , year =

  12. [20]

    arXiv preprint arXiv:1301.3781 , year =

    Efficient Estimation of Word Representations in Vector Space , author =. arXiv preprint arXiv:1301.3781 , year =

  13. [21]

    arXiv preprint arXiv:2308.10248 , year =

    Steering Language Models With Activation Engineering , author =. arXiv preprint arXiv:2308.10248 , year =

  14. [22]

    Steering

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander Matt , booktitle =. Steering

  15. [23]

    arXiv preprint arXiv:2310.01405 , year =

    Representation Engineering: A Top-Down Approach to AI Transparency , author =. arXiv preprint arXiv:2310.01405 , year =

  16. [24]

    2025 , eprint=

    Automatically Interpreting Millions of Features in Large Language Models , author=. 2025 , eprint=

  17. [25]

    2023 , eprint=

    Rigorously Assessing Natural Language Explanations of Neurons , author=. 2023 , eprint=

  18. [26]

    2025 , eprint=

    Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders , author=. 2025 , eprint=

  19. [27]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Saes are good for steering--if you select the right features , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

  20. [28]

    2026 , eprint=

    Controllable LLM Reasoning via Sparse Autoencoder-Based Steering , author=. 2026 , eprint=

  21. [29]

    2025 , eprint=

    Reasoning-Finetuning Repurposes Latent Representations in Base Models , author=. 2025 , eprint=

  22. [30]

    Sparse Autoencoders are Capable

    Yannick Assogba and Jacopo Cortellazzi and Javier Abad and Pau Rodriguez and Xavier Suau and Arno Blaas , year=. Sparse Autoencoders are Capable. 2602.12418 , archivePrefix=

  23. [31]

    2026 , eprint=

    Do Sparse Autoencoders Identify Reasoning Features in Language Models? , author=. 2026 , eprint=

  24. [32]

    International Conference on Learning Representations (ICLR) , year=

    Efficient Streaming Language Models with Attention Sinks , author=. International Conference on Learning Representations (ICLR) , year=

  25. [33]

    Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

    What does BERT learn about the structure of language? , author=. Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

  26. [34]

    1988 , publisher =

    Cohen, Jacob , title =. 1988 , publisher =

  27. [35]

    2025 , eprint=

    Open Problems in Mechanistic Interpretability , author=. 2025 , eprint=

  28. [36]

    Advances in Neural Information Processing Systems 36 (NeurIPS) , year=

    Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , author=. Advances in Neural Information Processing Systems 36 (NeurIPS) , year=

  29. [37]

    2025 , eprint=

    Persona Vectors: Monitoring and Controlling Character Traits in Language Models , author=. 2025 , eprint=

  30. [38]

    2024 , eprint=

    Scaling and evaluating sparse autoencoders , author=. 2024 , eprint=

  31. [39]

    2023 , eprint=

    Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching , author=. 2023 , eprint=

  32. [40]

    2026 , eprint=

    Remarks on the disproof of the unit distance conjecture , author=. 2026 , eprint=

  33. [41]

    arXiv preprint arXiv:2410.20526 , year=

    Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders , author=. arXiv preprint arXiv:2410.20526 , year=

  34. [42]

    Wang, Yubo and Ma, Xueguang and Zhang, Ge and Ni, Yuansheng and Chandra, Abhranil and Guo, Shiguang and Ren, Weiming and Arulraj, Aaran and He, Xuan and Jiang, Ziyan and Li, Tianle and Ku, Max and Wang, Kai and Zhuang, Alex and Fan, Rongqi and Yue, Xiang and Chen, Wenhu , booktitle=

  35. [43]

    2025 , eprint=

    s1: Simple test-time scaling , author=. 2025 , eprint=

  36. [44]

    2025 , eprint=

    A mathematical perspective on Transformers , author=. 2025 , eprint=

  37. [45]

    arXiv preprint arXiv:2508.12535 , year=

    CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features , author=. arXiv preprint arXiv:2508.12535 , year=

  38. [46]

    International journal of theoretical physics , volume=

    The large-N limit of superconformal field theories and supergravity , author=. International journal of theoretical physics , volume=. 1999 , publisher=

  39. [47]

    arXiv preprint arXiv:2110.14168 , year =

    Training Verifiers to Solve Math Word Problems , author =. arXiv preprint arXiv:2110.14168 , year =

  40. [48]

    Measuring Mathematical Problem Solving With the

    Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob , journal =. Measuring Mathematical Problem Solving With the

  41. [49]

    arXiv preprint arXiv:2305.20050 , year =

    Let's Verify Step by Step , author =. arXiv preprint arXiv:2305.20050 , year =

  42. [50]

    The Stack: 3

    Kocetkov, Denis and Li, Raymond and Ben Allal, Loubna and Li, Jia and Mou, Chenghao and Mu. The Stack: 3. arXiv preprint arXiv:2211.15533 , year =

  43. [51]

    Gao, Leo and Biderman, Stella and Black, Sid and Golding, Laurence and Hoppe, Travis and Foster, Charles and Phang, Jason and He, Horace and Thite, Anish and Nabeshima, Noa and Presser, Shawn and Leahy, Connor , journal =. The

  44. [52]

    arXiv preprint arXiv:2605.11887 , year =

    Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models , author =. arXiv preprint arXiv:2605.11887 , year =

  45. [53]

    Souly, Alexandra and Lu, Qingyuan and Bowen, Dillon and Trinh, Tu and Hsieh, Elvis and Pandey, Sana and Abbeel, Pieter and Svegliato, Justin and Emmons, Scott and Watkins, Olivia and Toyer, Sam , journal =. A

  46. [54]

    Hugging Face , year =

    FineWeb-Edu: The Finest Collection of Educational Content , author =. Hugging Face , year =

  47. [55]

    arXiv preprint arXiv:2506.18167 , year =

    Understanding Reasoning in Thinking Language Models via Steering Vectors , author =. arXiv preprint arXiv:2506.18167 , year =

  48. [56]

    The American Journal of Psychology , volume =

    The Proof and Measurement of Association between Two Things , author =. The American Journal of Psychology , volume =

  49. [57]

    British Journal of Psychology , volume =

    Correlation Calculated from Faulty Data , author =. British Journal of Psychology , volume =

  50. [58]

    British Journal of Psychology , volume =

    Some Experimental Results in the Correlation of Mental Abilities , author =. British Journal of Psychology , volume =

  51. [59]

    arXiv preprint arXiv:2309.07597 , year =

    C-Pack: Packaged Resources To Advance General Chinese Embedding , author =. arXiv preprint arXiv:2309.07597 , year =

  52. [60]

    and Gonzalez, Joseph E

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Li, Tianle and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Li, Zhuohan and Lin, Zi and Xing, Eric P. and Gonzalez, Joseph E. and Stoica, Ion and Zhang, Hao , journal =

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.