Pith. sign in

REVIEW 3 major objections 5 minor 300 references

One learned persistence coefficient per sparse feature turns a token-wise sparse autoencoder into a model that separates local detectors from a persistent semantic state.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 18:57 UTC pith:7RF6GU46

load-bearing objection Useful SAE variant with clean fast/slow separation, but the 'learns for how long' claim outruns the 256-token training window. the 3 major comments →

arxiv 2607.17117 v1 pith:7RF6GU46 submitted 2026-07-19 cs.LG cs.CL

Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models

Haoyan Luo , Mateo Espinosa Zarlenga , Mateja Jamnik This is my paper
classification cs.LG cs.CL
keywords sparse autoencoderspersistence coefficientsfeature timescaleslanguage model interpretabilitysemantic stateprompt injection monitoringtemporal structureunsupervised feature selection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Persistent Sparse Autoencoders aim to show that sparse autoencoders can learn, from reconstruction error alone, which features should carry information forward in a language model and for how long. The method adds one learned persistence coefficient per feature, turning each feature into a decaying state that integrates token-specific evidence. The learned timescales split into many fast, locally interpretable detectors and a small set of slow features that concentrate topic-level context. In a long-context prompt-injection monitoring study, slow features retain a decodable and causally effective signal hundreds of tokens after the injection text has passed. A sympathetic reader would care because this offers a label-free route to persistent semantic state in language models, without requiring predefined temporal partitions.

Core claim

The paper's central claim, stated plainly in its conclusion, is that Persistent SAEs learn, from reconstruction alone, which sparse features should retain information across tokens and for how long. The discovery is that a simple diagonal recurrence over the sparse evidence — h_j,t = λ_j h_j,t−1 + (1−λ_j) a_j,t — is sufficient for the timescale to emerge. Features with large λ carry evidence forward and end up forming a compact semantic state that clusters by topic, while features with small λ stay fast and locally interpretable. The paper demonstrates this through competitive reconstruction quality, subject-probing efficiency (the slowest 1% of features nearly matches a label-supervised ran

What carries the argument

The central object is the per-feature persistence coefficient λ_j, parameterized as a sigmoid over an unconstrained logit and capped below 1, embedded in a diagonal exponential-moving-average recurrence. This recurrence has three properties that carry the argument: the (1−λ_j) factor keeps the persistent state bounded under constant evidence; the diagonal structure preserves feature-level attribution; and because λ_j is learned by gradient descent on reconstruction loss, the model's own optimization discovers which features should persist. The resulting timescale is exponential, with effective memory −1/log(λ_j) tokens, so the paper reads a learned spectrum directly off λ.

Load-bearing premise

The load-bearing assumption is that a 256-token training sequence is long enough for the reconstruction objective to reward persistence at the scales later evaluated; if useful long-range carry depends on correlations beyond that window, the learned coefficients at 350 tokens are extrapolations and the monitoring advantage could be an artifact of finite-window averaging rather than genuine learned persistence.

What would settle it

Retrain Persistent SAEs on the same activations with the same budget but replace the contiguous training sequences with truncated windows or with a shuffled control that breaks global continuity; if the slow-feature advantage in subject probing and in 350-token injection monitoring persists, it is likely an artifact of averaging rather than learned persistence. Alternatively, measure whether the learned λ values predict held-out reconstruction gain at lags beyond the training window — if a feature whose λ implies a 1000-token timescale shows no measurable benefit past 256 tokens, the paper's c

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, standard sparse-autoencoder training can be upgraded to produce a per-feature timescale at no extra supervision, separating token-local evidence from a persistent semantic state.
  • Slow features give an unsupervised criterion for selecting a compact topic-representing subspace; ranking features by λ recovers most of the subject-probing signal from a small fraction of the dictionary.
  • In monitoring settings, slow-feature directions maintain both decodability and causal influence long after local evidence has disappeared, which matters for agents that act only after many tokens.
  • The persistent state does not degrade reconstruction quality: it matches or approaches token-specific sparse-autoencoder baselines while adding temporal structure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's design choice of 256-token training windows means the learned λ values are rewarded for persistence only within that horizon; whether slow features genuinely capture timescales beyond 256 tokens remains an open extension.
  • Because the recurrence is diagonal and fixed per feature, it cannot model abrupt resets or interactions between features; a learned gating term is a natural extension that could handle topic shifts.
  • The slow-state representation could serve as a reusable context readout for other long-horizon tasks, such as tracking instruction adherence or factual consistency, beyond prompt-injection detection.
  • The causal steering result suggests a testable extension: intervene on slow-feature directions in the acting agent itself, not just an external monitor, to see whether long-range behaviour changes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Persistent Sparse Autoencoders, an extension of BatchTopK SAEs in which each feature has a learned persistence coefficient λ_j controlling an exponential moving average of its sparse evidence across tokens. Training only from reconstruction loss, the method separates fast, locally interpretable features from slow features that carry topic-level or injection-relevant information over long contexts. The empirical sections show: (1) standard SAE features already exhibit temporal persistence that a shuffled control does not; (2) Persistent SAEs retain roughly competitive reconstruction quality while producing a spectrum of timescales; (3) slow features form subject-specific clusters and support topic probing almost as well as a supervised feature ranking; (4) in a prompt-injection monitoring case study, a probe trained at the injection point transfers better to later tokens when using slow features, and steering directions from slow features shift a monitor's judgment at distances up to 350 tokens.

Significance. If the findings hold, Persistent SAEs are a useful unsupervised mechanism for exposing persistent semantic state in LLM activations, with plausible applications in interpretability and safety monitoring. The paper includes several well-controlled diagnostics: a shuffled-document control that isolates global continuity, held-out probe evaluation, and a causal intervention in a monitor model. The central claim — that reconstruction alone can induce per-feature timescales — is interesting and partially supported for timescales within the training window. However, the mismatch between the 256-token training horizon and the 350-token evaluation, together with unmatched probe dimensionality and missing error bars, means the strongest claims in the abstract and conclusion are not fully supported as stated.

major comments (3)
  1. [§4, Eq. (5) / Appendix B.1 vs. §6] Training uses eight sequences of context length 256 per step, so the reconstruction gradient for λ_j at position t cannot reward retention beyond the 256-token window. Section 6 nonetheless evaluates slow features at distances up to 350 tokens, and the conclusion claims the model learns "for how long" features persist. For τ > 256, the λ_j values are not identified by the training objective; any λ high enough to survive the window is nearly equivalent within training, especially with the λmax=0.999 cap. The slow-feature advantage at d=350 may reflect the recurrence mechanics rather than learned timescales. This undermines the central claim as stated. Please retrain with longer contexts (e.g., 1024 tokens) or verify the empirical decay, and at minimum qualify the claim to timescales ≤256.
  2. [§6 / Appendix H] The slow-state probe uses the largest 1% of h_t coordinates (≈164 dimensions), while the baseline SAE representations are evaluated at their full 16,384-dimension sparse activation vectors. Logistic regression on high-dimensional inputs with no explicit regularization can overfit at d=0 and fail to transfer, making the slow-state advantage at d=350 partly an artifact of dimensionality. No error bars or cross-validation details are reported for the probe or the intervention. Please match the dimensionality (e.g., use top-1% features for all representations), use regularized probes with cross-validated hyperparameters, and report confidence intervals across trajectory folds or model seeds.
  3. [§5.1 / Table 1] On Gemma-2-2B, the Persistent SAE achieves FVE 0.66 and cosine similarity 0.75 versus 0.72/0.79 for BatchTopK, a material gap that the text calls "competitive." No error bars or multiple seeds are reported, so this claim is not statistically supported. The same applies to the AUC comparisons in Figure 6, where differences at low feature budgets may be sensitive to run-to-run variation. Please report variance or justify the trade-off in reconstruction quality as acceptable for the persistence benefits.
minor comments (5)
  1. [Figure 4 caption] "a broad range of timescales of λ" should read "timescales of λ_j".
  2. [Appendix B.3] "We initialise the λj to 0.5" should be "λ_j to 0.5".
  3. [Discussion] Typo: "from a slow features that retains" should be "from slow features that retain".
  4. [References] The entry "monology. 2023. Pile uncopyrighted. Hugging Face dataset." is not in standard citation format; provide full author/version details. Also, the Pythia reference lists "and 1 others" instead of the complete author list.
  5. [Discussion] The 256-token training horizon is not listed as a limitation. Even if the experiments are kept, this should be acknowledged and its consequences for the long-context claims discussed.

Circularity Check

0 steps flagged

No significant circularity: persistence coefficients are learned from reconstruction and downstream claims rest on held-out evaluations, not on fitted outcomes.

full rationale

The paper's central derivation is not circular. Persistent SAEs parameterize each feature's persistence as λ_j and learn it by minimizing reconstruction loss over contiguous sequences (Eq. 5; Appendix B.1). The claim that 'high-λ features carry information across tokens' is a direct, explicit consequence of the recurrence h_t = λ ⊙ h_{t-1} + (1−λ) ⊙ a_t, not a disguised prediction: the paper states that large λ yields slow features and that past evidence is attenuated by λ_j^k. This is model definition, not a fitted result renamed as a discovery. The empirical contributions—topic clustering, probe transfer, and causal steering—are evaluated on held-out data (MMLU, AgentDojo) using features ranked by λ learned on Pile; the probe in Section 6 is trained only at d=0 and evaluated later without retraining, so it is a genuine transfer test rather than a fit to the evaluated distances. The slow-feature advantage at long distances could be viewed as partly inherited from the recurrence, but the paper never claims otherwise; its claim is that reconstruction alone learns which features should have this property, and that the resulting slow state contains useful semantic/monitoring information. There is no load-bearing self-citation: the only author self-citation (Luo & Specia 2024) is a background survey reference. One could raise a correctness concern that the 256-token training window limits identifiability of timescales beyond 256 tokens, but that is an extrapolation/support issue, not a circularity. Overall, the derivation chain is self-contained and the main results are not equivalent to the training objective by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method's central claim depends on the assumption that per-feature exponential decay over a 256-token training window captures the temporal structure of LLM hidden states, plus a handful of hand-set hyperparameters (λmax, k, λ-init). It introduces no new physical entities; 'persistent state' and 'slow features' are constructs derived from existing activations.

free parameters (4)
  • Per-feature persistence coefficients λ_j = learned; range (0, 0.999) after sigmoid and λmax cap
    The central learned parameters. The paper's claim that reconstruction alone determines timescales is a claim about these values.
  • λmax hard cap = 0.999
    Hand-set upper bound on all persistence coefficients; bounds the maximum timescale and is not learned.
  • BatchTopK budget k = 20
    Sparsity budget matched across all SAE baselines; chosen by the authors, not derived from the problem.
  • Persistence logit initialization = λ = 0.5
    All λ_j initialized to 0.5; the final timescale distribution could depend on this choice.
axioms (5)
  • domain assumption Per-feature exponential moving average (Eq. 5) is a sufficient model of temporal persistence in LLM hidden states.
    The method is built on this recurrence; §7 concedes it cannot capture feature interactions or resets.
  • domain assumption 256-token contiguous training sequences provide enough signal to estimate persistence coefficients claimed to be valid at 350-token distances.
    Appendix B.1 uses context length 256; §6 evaluates up to distance 350. If longer-range correlations are needed, λ estimates are extrapolated.
  • domain assumption The shuffled-control dataset preserves local continuity while breaking only global continuity.
    Used in §3 to attribute persistence to global document structure; if shuffling changes low-level statistics, the control is confounded.
  • domain assumption gpt-4o-mini judge explanations are a valid proxy for feature interpretability.
    Automated interpretability in Appendix E inherits judge-model biases; scores support only relative comparisons.
  • standard math Standard SAE reconstruction objective and BatchTopK sparsity mechanism from prior work are adopted unchanged.
    Eq. 8-10 from Bussmann et al.; the paper extends rather than re-derives them.

pith-pipeline@v1.3.0-alltime-deepseek · 16719 in / 16966 out tokens · 164445 ms · 2026-08-01T18:57:28.957262+00:00 · methodology

0 comments
read the original abstract

Sparse autoencoders (SAEs) decompose language model activations into sparse features, but standard SAEs encode each token independently and do not expose information that persists across a sequence. We introduce Persistent Sparse Autoencoders (Persistent SAEs), which extend standard SAEs by learning a persistence coefficient for each feature, allowing the model to learn which features should persist and for how long. Our experiments show that they retain competitive reconstruction quality while learning a spectrum of feature timescales: fast features behave as locally interpretable detectors, whereas slow features concentrate topic-level information in a persistent state. Moreover, as shown in a prompt-injection monitoring case study, slow features preserve detection signals and remain causally effective over long contexts. These results suggest that Persistent SAEs open up new opportunities for interpreting and monitoring language models through persistent semantic representations.

Figures

Figures reproduced from arXiv: 2607.17117 by Haoyan Luo, Mateja Jamnik, Mateo Espinosa Zarlenga.

Figure 1
Figure 1. Figure 1: From token-specific evidence to persistent semantic state. Persistent SAEs learn a timescale for each feature: fast features remain token-specific and locally interpretable to explain the reconstruction, while slow features integrate sparse evidence over tokens to track the active semantic context. influence model behaviour (Templeton et al., 2024; Marks et al., 2025; Arad et al., 2025). Although causal tr… view at source ↗
Figure 2
Figure 2. Figure 2: Sparse features carry temporal structure that a token-specific SAE leaves unused. (a) Natural documents preserve global continuity; the shuffled control joins sentences from different sources, preserving local continuity but breaking global continuity. (b, c) Decoder-aligned feature persistence Rj (τ ) decays more slowly in natural documents than in the shuffled control. (d, e) Past activations improve hel… view at source ↗
Figure 3
Figure 3. Figure 3: Persistent Sparse Autoencoders. (a) Each feature integrates sparse token-specific evidence at into state ht with its learned persistence coefficient λj . (b) The learned feature timescales separate fast, locally interpretable features from a compact set of slow features that track semantic state. token t. We test this using a restricted context correction experiment. Starting from the standard reconstructi… view at source ↗
Figure 4
Figure 4. Figure 4: Learned timescales of Persistent SAEs and coactivation of slow features. (a) Persistence coefficients λj form a broad range of timescales of λ, with many fast features and fewer slow features. (b) The slow features coactivate more strongly than random features, indicating a structured persistent subspace. larger λj when carrying feature j forward helps re￾construct later hidden states. The model therefore … view at source ↗
Figure 5
Figure 5. Figure 5: Token-level MMLU representations, coloured by subject. The Persistent SAE’s high- [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Subject-probing AUC on spliced MMLU sequences. Ranking features by decreasing [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Persistent monitoring of prompt injection. (a) A probe trained at the end of an untrusted span is evaluated later in the trajectory. (b) Slow states retain more predictive information (top) and the strongest causal effect on the monitor’s judgement (bottom). each SAE representation r, we construct a steer￾ing vector vr by weighting the features of its most predictive features by their probe coefficients. W… view at source ↗
Figure 8
Figure 8. Figure 8: Persistent SAE training dynamics. Training loss (left), the number of non-zero coordinates in the persistent state ht (state L0; centre), and the fraction of state activation mass contributed by the carried term λ⊙ht−1 (right) for Pythia-160M at layer 8 and Gemma-2B at layer 12. After a short initial transient, the loss and active-state size stabilise, while the carried-mass fraction remains well away from… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

300 extracted references · 1 canonical work pages

  1. [1]

    The Fourteenth International Conference on Learning Representations , year=

    Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretability , author=. The Fourteenth International Conference on Learning Representations , year=

  2. [2]

    The Thirteenth International Conference on Learning Representations , year=

    Concept Bottleneck Large Language Models , author=. The Thirteenth International Conference on Learning Representations , year=

  3. [3]

    Jinqi Luo and Tianjiao Ding and Kwan Ho Ryan Chan and Darshan Thaker and Aditya Chattopadhyay and Chris Callison-Burch and Rene Vidal , booktitle=. Pa. 2024 , url=

    2024

  4. [4]

    and Mihalcea, Rada , title =

    Lee, Andrew and Bai, Xiaoyan and Pres, Itamar and Wattenberg, Martin and Kummerfeld, Jonathan K. and Mihalcea, Rada , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  5. [5]

    Model Editing as a Robust and Denoised variant of

    Rheeya Uppaal and Apratim Dey and Yiting He and Yiqiao Zhong and Junjie Hu , booktitle=. Model Editing as a Robust and Denoised variant of. 2025 , url=

    2025

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    Inference-time intervention: Eliciting truthful answers from a language model , author=. Advances in Neural Information Processing Systems , volume=

  7. [7]

    Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space

    Geva, Mor and Caciularu, Avi and Wang, Kevin and Goldberg, Yoav. Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.3

    doi:10.18653/v1/2022.emnlp-main.3 2022

  8. [8]

    Transformer Feed-Forward Layers Are Key-Value Memories

    Geva, Mor and Schuster, Roei and Berant, Jonathan and Levy, Omer. Transformer Feed-Forward Layers Are Key-Value Memories. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021. doi:10.18653/v1/2021.emnlp-main.446

    doi:10.18653/v1/2021.emnlp-main.446 2021

  9. [9]

    2024 , eprint=

    From Understanding to Utilization: A Survey on Explainability for Large Language Models , author=. 2024 , eprint=

    2024

  10. [10]

    Locating and Editing Factual Associations in

    Kevin Meng and David Bau and Alex J Andonian and Yonatan Belinkov , booktitle=. Locating and Editing Factual Associations in. 2022 , url=

    2022

  11. [11]

    Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models , year =

    Gandikota, Rohit and Materzy\'. Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models , year =. Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XL , pages =. doi:10.1007/978-3-031-73661-2_10 , abstract =

    doi:10.1007/978-3-031-73661-2_10 2024

  12. [12]

    The Eleventh International Conference on Learning Representations , year=

    Label-free Concept Bottleneck Models , author=. The Eleventh International Conference on Learning Representations , year=

  13. [13]

    The Twelfth International Conference on Learning Representations , year=

    Concept Bottleneck Generative Models , author=. The Twelfth International Conference on Learning Representations , year=

  14. [14]

    The Eleventh International Conference on Learning Representations , year=

    Post-hoc Concept Bottleneck Models , author=. The Eleventh International Conference on Learning Representations , year=

  15. [15]

    Proceedings of the 37th International Conference on Machine Learning , pages =

    Concept Bottleneck Models , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , editor =

    2020

  16. [16]

    Advances in Neural Information Processing Systems , editor=

    Concept Embedding Models , author=. Advances in Neural Information Processing Systems , editor=. 2022 , url=

    2022

  17. [17]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Learning to Receive Help: Intervention-Aware Concept Embedding Models , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  18. [18]

    Concept whitening for interpretable image recognition , volume=

    Chen, Zhi and Bei, Yijie and Rudin, Cynthia , year=. Concept whitening for interpretable image recognition , volume=. Nature Machine Intelligence , publisher=. doi:10.1038/s42256-020-00265-z , number=

    doi:10.1038/s42256-020-00265-z

  19. [19]

    1998 , publisher =

    The Atomic Components of Thought , author =. 1998 , publisher =

    1998

  20. [20]

    2004 , publisher =

    On Intelligence , author =. 2004 , publisher =

    2004

  21. [21]

    Position: We Can

    John Hewitt and Robert Geirhos and Been Kim , booktitle=. Position: We Can. 2025 , url=

    2025

  22. [22]

    First Conference on Language Modeling , year=

    Guiding Language Model Reasoning with Planning Tokens , author=. First Conference on Language Modeling , year=

  23. [23]

    The Twelfth International Conference on Learning Representations , year=

    Think before you speak: Training Language Models With Pause Tokens , author=. The Twelfth International Conference on Learning Representations , year=

  24. [24]

    2025 , url=

    Tianzhe Chu and Yuexiang Zhai and Jihan Yang and Shengbang Tong and Saining Xie and Dale Schuurmans and Quoc V Le and Sergey Levine and Yi Ma , booktitle=. 2025 , url=

    2025

  25. [25]

    Ouyang, Long and Wu, Jeff and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll L. and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and Schulman, John and Hilton, Jacob and Kelton, Fraser and Miller, Luke and Simens, Maddie and Askell, Amanda and Welinder, Peter and Christiano, Paul and Leike, Jan and Lowe, R...

    2022

  26. [26]

    Transactions on Machine Learning Research , issn=

    Emergent Abilities of Large Language Models , author=. Transactions on Machine Learning Research , issn=. 2022 , url=

    2022

  27. [27]

    doi:10.48550/arXiv.2503.09532 , abstract =

    Karvonen, Adam and Rager, Can and Lin, Johnny and Tigges, Curt and Bloom, Joseph and Chanin, David and Lau, Yeu-Tong and Farrell, Eoin and McDougall, Callum and Ayonrinde, Kola and Wearden, Matthew and Conmy, Arthur and Marks, Samuel and Nanda, Neel , month = mar, year =. doi:10.48550/arXiv.2503.09532 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.09532

  28. [28]

    Makelov, Aleksandar and Lange, George and Nanda, Neel , month = may, year =. Towards. doi:10.48550/arXiv.2405.08366 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.08366

  29. [29]

    The Thirteenth International Conference on Learning Representations , year=

    Scaling and evaluating sparse autoencoders , author=. The Thirteenth International Conference on Learning Representations , year=

  30. [30]

    Hendel, Roee and Geva, Mor and Globerson, Amir , month = oct, year =. In-

  31. [31]

    arXiv:2307.15043 , year=

    Universal and transferable adversarial attacks on aligned language models , author=. arXiv:2307.15043 , year=

    Pith/arXiv arXiv

  32. [32]

    Adversarial Attacks on LLMs

    Weng, Lilian. Adversarial Attacks on LLMs. lilianweng.github.io. 2023

    2023

  33. [33]

    2024 , note=

    Many-shot Jailbreaking , author=. 2024 , note=

    2024

  34. [34]

    2024 , journal=

    Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability , author=. 2024 , journal=

    2024

  35. [35]

    2023 , journal=

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author=. 2023 , journal=

    2023

  36. [36]

    Forty-first International Conference on Machine Learning,

    Tamar Rott Shaham and Sarah Schwettmann and Franklin Wang and Achyuta Rajaram and Evan Hernandez and Jacob Andreas and Antonio Torralba , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  37. [37]

    Zhengxuan Wu and Aryaman Arora and Zheng Wang and Atticus Geiger and Dan Jurafsky and Christopher D Manning and Christopher Potts , booktitle=. Re. 2024 , url=

    2024

  38. [38]

    2009 , edition=

    Causality: Models, Reasoning, and Inference , author=. 2009 , edition=

    2009

  39. [39]

    Investigating Gender Bias in Language Models Using Causal Mediation Analysis , booktitle =

    Vig, Jesse and Gehrmann, Sebastian and Belinkov, Yonatan and Qian, Sharon and Nevo, Daniel and Singer, Yaron and Shieber, Stuart , editor =. Investigating Gender Bias in Language Models Using Causal Mediation Analysis , booktitle =. 2020 , volume =

    2020

  40. [40]

    Causal Abstractions of Neural Networks , url =

    Geiger, Atticus and Lu, Hanson and Icard, Thomas and Potts, Christopher , booktitle =. Causal Abstractions of Neural Networks , url =

  41. [41]

    https://arxiv.org/abs/2303.02536

    Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations , author=. arXiv:2303.02536 , url="https://arxiv.org/abs/2303.02536", year =

    Pith/arXiv arXiv

  42. [42]

    Locating and editing factual associations in

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , editor =. Locating and editing factual associations in. Advances in Neural Information Processing Systems , publisher =

  43. [43]

    Alignment Forum , year=

    Causal scrubbing: A method for rigorously testing interpretability hypotheses , author=. Alignment Forum , year=

  44. [44]

    arXiv:2304.05969 , year=

    Localizing model behavior with path patching , author=. arXiv:2304.05969 , year=

    Pith/arXiv arXiv

  45. [45]

    and Potts, Christopher , editor =

    Geiger, Atticus and Wu, Zhengxuan and Lu, Hanson and Rozner, Josh and Kreiss, Elisa and Icard, Thomas and Goodman, Noah D. and Potts, Christopher , editor =. Inducing Causal Structure for Interpretable Neural Networks , volume =. International Conference on Machine Learning,

  46. [46]

    2024 , journal=

    A Geometric Notion of Causal Probing , author=. 2024 , journal=

    2024

  47. [47]

    Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information

    Giulianelli, Mario and Harding, Jack and Mohnert, Florian and Hupkes, Dieuwke and Zuidema, Willem. Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information. Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyzing and Interpreting Neural Networks for NLP. 2018. doi:10.18653/v1/W18-5426

    doi:10.18653/v1/w18-5426 2018

  48. [48]

    Interpretability in the Wild: a Circuit for Indirect Object Identification in

    Wang, Kevin Ro and Variengien, Alexandre and Conmy, Arthur and Shlegeris, Buck and Steinhardt, Jacob , urldate =. Interpretability in the Wild: a Circuit for Indirect Object Identification in. The Eleventh International Conference on Learning Representations,

  49. [49]

    Extracting Latent Steering Vectors from Pretrained Language Models

    Subramani, Nishant and Suresh, Nivedita and Peters, Matthew. Extracting Latent Steering Vectors from Pretrained Language Models. Findings of the Association for Computational Linguistics: ACL 2022. 2022. doi:10.18653/v1/2022.findings-acl.48

    doi:10.18653/v1/2022.findings-acl.48 2022

  50. [50]

    Autoencoding beyond pixels using a learned similarity metric , booktitle =

    Anders Boesen Lindbo Larsen and S. Autoencoding beyond pixels using a learned similarity metric , booktitle =. 2016 , url =

    2016

  51. [51]

    2016 , journal=

    Sampling Generative Networks , author=. 2016 , journal=

    2016

  52. [52]

    Distributed Representations of Words and Phrases and their Compositionality , booktitle =

    Tom. Distributed Representations of Words and Phrases and their Compositionality , booktitle =. 2013 , url =

    2013

  53. [53]

    and Saligrama, Venkatesh and Kalai, Adam T

    Bolukbasi, Tolga and Chang, Kai-Wei and Zou, James Y. and Saligrama, Venkatesh and Kalai, Adam T. , editor =. Man is to computer programmer as woman is to homemaker?. Advances in Neural Information Processing Systems , publisher =

  54. [54]

    G lo V e: Global Vectors for Word Representation

    Pennington, Jeffrey and Socher, Richard and Manning, Christopher. G lo V e: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ). 2014. doi:10.3115/v1/D14-1162

    doi:10.3115/v1/d14-1162 2014

  55. [55]

    Gardner and Geoff Pleiss and Robert Pless and Noah Snavely and Kavita Bala and Kilian Q

    Paul Upchurch and Jacob R. Gardner and Geoff Pleiss and Robert Pless and Noah Snavely and Kavita Bala and Kilian Q. Weinberger , title =. 2017. 2017 , url =. doi:10.1109/CVPR.2017.645 , timestamp =

    work page doi:10.1109/cvpr.2017.645 2017

  56. [56]

    Implicit Semantic Data Augmentation for Deep Networks , booktitle =

    Yulin Wang and Xuran Pan and Shiji Song and Hong Zhang and Gao Huang and Cheng Wu , editor =. Implicit Semantic Data Augmentation for Deep Networks , booktitle =. 2019 , url =

    2019

  57. [57]

    Decoding The Thought Vector , author=

  58. [58]

    Alignment Forum , year =

    Alexander Matt Turner and Peli Grietzer and Ulisse Mini and Monte M and David Udell , title =. Alignment Forum , year =

  59. [59]

    Alignment Forum , year =

    Alexander Matt Turner and Peli Grietzer and Lisa Thiergart , title =. Alignment Forum , year =

  60. [60]

    2024 , journal=

    Steering Language Models With Activation Engineering , author=. 2024 , journal=

    2024

  61. [61]

    2023 , journal=

    Understanding and Controlling a Maze-Solving Policy Network , author=. 2023 , journal=

    2023

  62. [62]

    Advances in Neural Information Processing Systems , author =

    Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , volume =. Advances in Neural Information Processing Systems , author =. 2024 , langid =

    2024

  63. [63]

    2024 , journal=

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. 2024 , journal=

    2024

  64. [64]

    Byun and Zifan Wang and Alex Mallen and Steven Basart and Sanmi Koyejo and Dawn Song and Matt Fredrikson and J

    Andy Zou and Long Phan and Sarah Chen and James Campbell and Phillip Guo and Richard Ren and Alexander Pan and Xuwang Yin and Mantas Mazeika and Ann-Kathrin Dombrowski and Shashwat Goel and Nathaniel Li and Michael J. Byun and Zifan Wang and Alex Mallen and Steven Basart and Sanmi Koyejo and Dawn Song and Matt Fredrikson and J. Zico Kolter and Dan Hendryc...

    Pith/arXiv arXiv

  65. [65]

    Zou , title =

    Sheng Liu and Haotian Ye and Lei Xing and James Y. Zou , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  66. [66]

    2024 , journal=

    Extending Activation Steering to Broad Skills and Multiple Behaviours , author=. 2024 , journal=

    2024

  67. [67]

    2024 , journal=

    Improving Steering Vectors by Targeting Sparse Autoencoder Features , author=. 2024 , journal=

    2024

  68. [68]

    Steering Llama 2 via Contrastive Activation Addition

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander. Steering Llama 2 via Contrastive Activation Addition. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.828

    doi:10.18653/v1/2024.acl-long.828 2024

  69. [69]

    2024 , journal=

    Can sparse autoencoders be used to decompose and interpret steering vectors? , author=. 2024 , journal=

    2024

  70. [70]

    Sparse autoencoders match supervised features for model steering on the

    Makelov, Aleksandar , booktitle=. Sparse autoencoders match supervised features for model steering on the. 2024 , url=

    2024

  71. [71]

    2024 , journal=

    Steering Language Model Refusal with Sparse Autoencoders , author=. 2024 , journal=

    2024

  72. [72]

    2024 , url =

    Esin Durmus and Alex Tamkin and Jack Clark and Jerry Wei and Jonathan Marcus and Joshua Batson and Kunal Handa and Liane Lovitt and Meg Tong and Miles McCain and Oliver Rausch and Saffron Huang and Sam Bowman and Stuart Ritchie and Tom Henighan and Deep Ganguli , title =. 2024 , url =

    2024

  73. [73]

    2023 , url=

    Experiments in Evaluating Steering Vectors , author=. 2023 , url=

    2023

  74. [74]

    A Structural Probe for Finding Syntax in Word Representations

    Hewitt, John and Manning, Christopher D. A Structural Probe for Finding Syntax in Word Representations. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019. doi:10.18653/v1/N19-1419

    doi:10.18653/v1/n19-1419 2019

  75. [75]

    What you can cram into a single \ & ! \# * vector: Probing sentence embeddings for linguistic properties

    Conneau, Alexis and Kruszewski, German and Lample, Guillaume and Barrault, Lo. What you can cram into a single \ & ! \# * vector: Probing sentence embeddings for linguistic properties. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018. doi:10.18653/v1/P18-1198

    doi:10.18653/v1/p18-1198 2018

  76. [76]

    What do Neural Machine Translation Models Learn about Morphology?

    Belinkov, Yonatan and Durrani, Nadir and Dalvi, Fahim and Sajjad, Hassan and Glass, James. What do Neural Machine Translation Models Learn about Morphology?. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017. doi:10.18653/v1/P17-1080

    doi:10.18653/v1/p17-1080 2017

  77. [77]

    acting: A trade-off between world modeling & agent modeling , author=

    Predicting vs. acting: A trade-off between world modeling & agent modeling , author=. arXiv preprint arXiv:2407.02446 , year=

    Pith/arXiv arXiv

  78. [78]

    International Conference on Learning Representations , year=

    Towards a Unified View of Parameter-Efficient Transfer Learning , author=. International Conference on Learning Representations , year=

  79. [79]

    pyvene: A Library for Understanding and Improving P y T orch Models via Interventions

    Wu, Zhengxuan and Geiger, Atticus and Arora, Aryaman and Huang, Jing and Wang, Zheng and Goodman, Noah and Manning, Christopher and Potts, Christopher. pyvene: A Library for Understanding and Improving P y T orch Models via Interventions. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hum...

    doi:10.18653/v1/2024.naacl-demo.16 2024

  80. [80]

    Proceedings of the 34th International Conference on Machine Learning - Volume 70 , pages =

    Sundararajan, Mukund and Taly, Ankur and Yan, Qiqi , title =. Proceedings of the 34th International Conference on Machine Learning - Volume 70 , pages =. 2017 , publisher =

    2017

Showing first 80 references.