Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Interpreting and Steering Protein Language Models through Sparse Autoencoders

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A sparse autoencoder on ESM-2 8M yields both interpretable and steerable features: amplifying two zinc-finger latents produces recognized zinc finger regions in 24 of 180 sequences, versus zero for baseline and random latents.

desk verdict Useful first steering result for protein LMs via SAE latents, with open code and an external motif predictor, but the headline 24/180 contrast is statistically under-supported and needs better controls before it can carry the causal claim. read the letter →

arxiv 2502.09135 v1 pith:YXMTQON5 submitted 2025-02-13 cs.LG q-bio.BM

Edith Natalia Villegas Garcia , Alessio Ansuini This is my paper · ORCID
classification cs.LGq-bio.BM
keywords sparseautoencodersmechanisticinterpretabilityproteinlanguagemodelsmodelsteeringsequencegenerationzincfingermotifsESM-2intrinsicdimension
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that sparse autoencoders — the dictionary-learning tool used to disentangle polysemantic representations in language models — transfer to protein language models, and that the features they extract are not just descriptive but causally usable. Training an SAE on layer-3 activations of the ESM-2 8M model, the authors find 395 latent–annotation associations above a precision/recall threshold of 0.8, spanning transmembrane regions, binding sites, and zinc finger motifs. They then show that amplifying the two latents most associated with zinc finger regions, while sampling iteratively from the intervened model, yields 24 recognized zinc finger regions out of 180 generated sequences, where the baseline model and random-latent interventions produce none. If this holds, SAE latents in protein models are causal handles, and mechanistic interpretability offers a practical steering mechanism for protein sequence design.

What carries the argument

The load-bearing object is the sparse autoencoder inserted into the residual stream at layer 3 of ESM-2 8M, with encoder $z = \mathrm{ReLU}(W_{\mathrm{enc}}(x - b_{\mathrm{dec}}) + b_{\mathrm{enc}})$, decoder $\hat{x} = W_{\mathrm{dec}} z + b_{\mathrm{dec}}$, an L1 sparsity penalty, and unit-norm decoder columns enforced during training. Its interpretability is obtained by a precision/recall statistic against UniProt residue annotations, with association claimed when either exceeds 0.80. Its actionability comes from the steering step $x^* = f_{\mathrm{dec}}(z^*_k) + x_{\mathrm{err}}$, which treats the decoder plus the saved reconstruction error as an invertible hook for editing one latent at a time. A secondary mechanism is the layer-selection heuristic: an intrinsic-dimension estimator whose plateau identifies layer 3 as the point where abstract features are most cleanly encoded.

What would settle it

A dose–response test would settle the causal question: scale one zinc-finger latent across a fine range of amplitudes and check whether the fraction of predicted zinc-finger residues rises monotonically while the rest of the latent vector and the model loss stay flat. If the hit rate saturates or non-target latents co-move, the error-preserving inverse in Eq. (8) is not faithful and the 24/180 count could be a distortion artifact; equivalently, repeating the intervention with randomly shuffled decoder columns should produce zero zinc-finger matches if the effect is feature-specific.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that specific sparse-autoencoder latents in ESM-2 8M carry disentangled biological features, and that artificially amplifying two of them — the pair with the highest recall for the zinc finger annotation — steers the model's generative output toward that feature. The intervention sets the chosen latent to $z^*_k = a z_k + b$, decodes it, and adds back the saved reconstruction error before continuing the forward pass; repeating inference-and-sampling for 100 iterations and keeping the sequence with the maximal target activation produced sequences that an external motif/Pfam predictor recognized as zinc finger regions in 24 of 180 cases. The same procedure with the single most salient latent produced 3 matches, and with baseline or random-latent interventions produced none, which the paper reads as evidence that the extracted latents are faithful, actionable feature directions rather than incidental correlates.

Load-bearing premise

The steering result depends on the intervention $x^* = f_{\mathrm{dec}}(z^*_k) + x_{\mathrm{err}}$ being a causally faithful inverse, so that boosting one latent alters only the intended feature; with a reported cross-entropy increase of 0.10, an average of 18 active latents per token, and 573 dead latents, the autoencoder is demonstrably lossy, and the paper never checks that amplification is monotone or that unrelated latents stay fixed.

Editorial extensions

If this is right

  • SAE dictionaries on protein models can be annotated against curated biology: 395 latent–feature pairs meet the 0.80 precision/recall threshold, covering transmembrane regions, binding sites, active sites, disulfide bonds, and zinc fingers.
  • Latent amplification is a working steering mechanism for protein sequence generation: the two-latent zinc finger intervention produced 24 recognized motif regions out of 180 sequences, against zero for baseline and random-latent controls.
  • Steered generation explores, not copies: the matched sequences average 31% similarity to known zinc finger motifs (48% maximum), so the intervention biases the model without pinning it to particular sequences.
  • The pipeline is feature-agnostic: the same shortlist-and-amplify recipe should apply to any annotation with a sufficiently selective latent, and the intrinsic-dimension plateau heuristic can pick layers for other ESM-2 sizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper validates its latents by correlation and a single intervention amplitude schedule, not by a dose–response curve, so the causal reading is a hypothesis worth testing rather than a closed case; a monotone amplification check would decide between genuine feature control and output distortion.
  • The 13 percent hit rate (24/180) is a conservative floor on steering efficacy: the scorer requires a match to known motifs or Pfam families, while the paper's own ESMFold examples show zinc-finger-like beta-sheet-and-helix arrangements even in uncounted sequences, so a structure-based evaluator would likely credit more of the 180.
  • A cheap mechanistic ablation follows directly from the method: permuting the decoder columns post-training and repeating the intervention should erase the zinc finger hits if the effect is truly carried by the identified feature directions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper trains sparse autoencoders (SAEs) on the residual-stream activations of ESM-2 8M at layer 3, associates individual latents with protein annotations from UniProt via precision/recall thresholds, and then uses high-recall zinc-finger latents to steer the autoregressive generation of new sequences. The central empirical result is that simultaneous intervention on the two highest-recall zinc-finger latents yields 24/180 generated sequences that an external motif predictor recognizes as zinc-finger regions, versus 3/180 for a single latent and 0/180 for baseline and random-latent interventions. The authors conclude that SAE latents can be leveraged to steer the model toward non-trivial structural features.

Significance. If the steering result is robust, this is a useful advance: it provides one of the first demonstrations that SAE latents in a protein language model are causally actionable, and the paper ships the trained weights, code, and supplementary associations, which supports reproducibility. The use of an external motif predictor to score the generated sequences is a real strength and avoids the circularity of scoring with the same model that was steered. However, the key 24/180 contrast is not yet supported by a valid control or uncertainty quantification, and the interpretability table lacks a null model. These are fixable with additional experiments and reporting, but they are load-bearing for the paper's central claims.

major comments (4)
  1. [§4.2, §A.3.2] The steering claim rests on the contrast between 24/180 matches for the two target latents and 0/180 for the random-latent control, but the control is not specified closely enough to be a fair null. The paper does not state how the random latents were sampled, whether they were alive at the time of intervention, what their pre- and post-intervention activation magnitudes were, or how the norm of the residual-stream perturbation compared with the target intervention. With 573 dead latents in the selected SAE (§A.3.2), a randomly chosen latent or pair may be dead, in which case the control intervention does nothing and the 0/180 outcome is vacuous. Please report the number of random draws, seeds, alive/active status of each control latent, and matched perturbation norms, and provide a confidence interval for the 24/180 rate (e.g., a binomial CI).
  2. [§4.2] The paper does not disclose how many latent candidates or latent pairs were screened before the 'two most prominent' zinc-finger latents were selected. If the authors examined many high-recall latents or many pairs and then reported the best outcome, the 24/180 count is a selection statistic rather than a property of the chosen latents. Please report the full screening protocol, the number of candidates tested, and, if feasible, a multiple-testing correction or a held-out evaluation on latents selected by a pre-registered rule.
  3. [§3.3, Table 1] The 395 putative [k, φ] associations in Table 1 are reported without a null model or multiple-testing correction. Precision and recall are computed over a large number of latent-feature pairs with different base rates, and a threshold of 0.8 can be exceeded by chance, especially for low-base-rate annotations. Please provide a permutation null (for example, permuting annotation labels while preserving token marginals or permuting latent identities) and report how many associations would be expected under that null at the chosen thresholds. This is necessary before asserting that specific latents are associated with specific protein features.
  4. [§4.2, Eq. (8)] The intervention in Eq. (8) is not validated for causal specificity. Algebraically, because f_dec is linear and the saved error x_err cancels, the intervention is equivalent to adding a scaled decoder column to the original activation; the reconstruction-error term is therefore not the weak point. The weak point is that the paper never shows that changing the target latent leaves unrelated latents fixed, that the steering effect is monotone in the gain a, or that the 24/180 matches are not produced by a generic distributional shift. Given that the SAE has L0 = 18, a cross-entropy increase of 0.10, and 573 dead latents (§A.3.2), the authors should add control interventions on equally active latents unrelated to zinc fingers, report the effect of varying a and b on the success rate, and analyze whether the matched regions are full zinc-finger domains rather than short degenerate fragments.
minor comments (5)
  1. [§3.2] The intrinsic-dimension plateau that motivates the choice of layer 3 is identified visually; please state an explicit, reproducible criterion for selecting the layer from the ID curve.
  2. [§3.3] The activation threshold τ_z is introduced in the main text, but the values (0.01, 0.10, 1.00) appear only in Table 1; please state in the main text which thresholds were used to produce the reported 395 associations.
  3. [§4.2] The sweep of 180 combinations of sequence length, gain a, and shift b is described, but it is unclear whether the same (a, b) is applied to both latents in the two-latent intervention; please clarify the parameterization.
  4. [§4.2] The reported average sequence similarity of 31% is not accompanied by a definition of the similarity measure; please specify the alignment and identity metric used.
  5. [Acknowledgements] There is a typo in the funding statement: 'A.A. ws supported' should read 'A.A. was supported.'

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; minor self-citation in layer selection does not determine the externally validated steering result.

full rationale

The paper's central derivation chain is not circular. The latent-feature associations in Sec. 3.3 are defined by precision/recall against UniProt annotations (Eqs. 5-6); these are correlational labels, not predictions of the steering outcome. The steering protocol in Sec. 3.4 (Eqs. 7-8) is a standard linear intervention: modifying a latent and adding back the reconstruction error is algebraically equivalent to adding a scaled decoder column to the residual-stream activation, so it does not presuppose the zinc-finger result. The claimed success is then evaluated externally: generated sequences are scored by the independent zinc-finger motif/HMM predictor of Sathyaseelan et al. (2023), which is not used in latent selection. The baseline and random-latent controls, while arguably underpowered as nulls, are part of the experimental contrast rather than an input-equivalent construction. The only notable self-citation is the layer-selection heuristic in Sec. 3.2, which relies on the intrinsic-dimension plateau result of Valeriani et al. (2024), whose author list overlaps with the present paper; however, choosing layer 3 is not what constructs the 24/180 motif matches, and the steering claim is independently testable. This is a minor, non-load-bearing self-citation, not circularity; hence score 2 rather than 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several hand-chosen thresholds (activation threshold, 0.8 association cutoff, layer 3 selection), on the unvalidated causal inversion assumption for the SAE decoder, and on external annotation and motif-prediction tools used as ground truth. These are the main inputs the reader must accept beyond the released code and data.

free parameters (6)
  • Activation threshold τ_k = 0.01, 0.10, 1.00
    Defines when a latent is counted as active for precision/recall; Table 1 shows the number of associations changes strongly with this threshold.
  • Association threshold for precision/recall = 0.80
    A latent-feature pair is considered an association if either precision or recall exceeds 0.8; the choice is arbitrary and no sensitivity analysis around it is given.
  • L1 sparsity penalty λ = not reported for the selected model
    Selected from the sweep [0.0003, 0.001, 0.005] to balance CE increase and L0; the final value is not stated in the paper.
  • SAE dictionary size multiplier = 10
    Chosen among [5, 10, 32]; the selected SAE has a hidden size 10 times the ESM-2 hidden size.
  • Steering gain a and shift b = grid searched, a in [2,5,10,20,30], b in [0.1,1,10,50,100,200]
    The intervention uses z* = a·z + b; the grids are swept but the specific values that produced the 24 matches are not reported.
  • Layer choice = layer 3
    Picked from the intrinsic dimension plateau in Figure 4; the plateau heuristic is from Valeriani et al., and no comparison across layers is provided.
assumptions (5)
  • domain assumption SAE latents trained with L1 sparsity on a single transformer layer yield interpretable, causally actionable features in a protein language model.
    The whole pipeline depends on this property, which is carried over from NLP work (Templeton et al., 2024; Gao et al., 2024) without validation for ESM-2.
  • domain assumption The intrinsic dimension plateau marks the layer where abstract features are most clearly encoded.
    Used to select layer 3; the paper relies on Valeriani et al. (2024) and does not test other layers for interpretability or steering.
  • domain assumption UniProt annotations are accurate and complete enough at amino-acid level to serve as ground truth for the features of interest.
    Precision and recall are computed against these labels; errors or incompleteness would bias all association scores.
  • ad hoc to paper The intervention x* = f_dec(z*_k) + x_err is a valid causal injection into the residual stream.
    This is the paper's own method (Eq. 8) and no independent evidence establishes that the decoder is a faithful inverse at layer 3.
  • domain assumption The external zinc finger predictor (Sathyaseelan et al.) correctly identifies true zinc finger motifs in the generated sequences.
    Steering success is measured entirely by this predictor, so its false-positive/negative rates directly set the reported success counts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpreting and Steering Protein Language Models through Sparse Autoencoders." pith.science (2026). https://pith.science/paper/YXMTQON5

@misc{pith2026250209135,
  author       = {Pith},
  title        = {Pith review of: Interpreting and Steering Protein Language Models through Sparse Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YXMTQON5}},
  note         = {Machine review of arXiv:2502.09135}
}
read the original abstract

The rapid advancements in transformer-based language models have revolutionized natural language processing, yet understanding the internal mechanisms of these models remains a significant challenge. This paper explores the application of sparse autoencoders (SAE) to interpret the internal representations of protein language models, specifically focusing on the ESM-2 8M parameter model. By performing a statistical analysis on each latent component's relevance to distinct protein annotations, we identify potential interpretations linked to various protein characteristics, including transmembrane regions, binding sites, and specialized motifs. We then leverage these insights to guide sequence generation, shortlisting the relevant latent components that can steer the model towards desired targets such as zinc finger domains. This work contributes to the emerging field of mechanistic interpretability in biological sequence models, offering new perspectives on model steering for sequence design.

Figures

Figures reproduced from arXiv: 2502.09135 by the authors.

Figure 1
Figure 1. Sequence generation procedure. (A) To steer the model outputs, the base Protein Language [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Distribution of the number of latent SAE components that detect a feature with a mini [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of generated sequences subsequently folded with ESMFold (Lin et al., 2023). [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Evolution of the intrinsic dimension estimate through the layers of the ESM-2 8M model. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Cross-entropy increase vs sparsity trade-off for all the vanilla sparse autoencoders trained [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: (A) P(k +|ϕ +) - Percentage of tokens for which each latent component is active when there is a C2H2 zinc finger type label, (B) P(k +|ϕ −) - percentage when there is no C2H2 zinc finger label, and (C) difference between these two values for three different activation …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FoldSAE: Learning to Steer Protein Folding Through Sparse Representations

    q-bio.QM 2025-11 conditional novelty 6.0 of 10

    Sparse-autoencoder features inside RFdiffusion predict—and, when amplified or suppressed, causally steer—helix and strand content in generated protein backbones.

Reference graph

Works this paper leans on

15 extracted references · 6 canonical work pages · cited by 1 Pith paper

  1. [6]

    Simulating 500 million years of evolution with a language model

    Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model. bioRxiv, pp. 2024–07,

    work page 2024

  2. [7]

    Sparse autoencoders match supervised features for model steering on the ioi task

    Aleksandar Makelov. Sparse autoencoders match supervised features for model steering on the ioi task. In ICML 2024 Workshop on Mechanistic Interpretability,

    work page 2024

  3. [8]

    https://transformer-circuits.pub/2022/mech-interp-essay/index. html. Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. arXiv preprint arXiv:2311.03658,

    arXiv 2022

  4. [9]

    A practical review of mecha- nistic interpretability for transformer-based language models

    Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. A practical review of mecha- nistic interpretability for transformer-based language models. arXiv preprint arXiv:2407.02646,

    arXiv

  5. [11]

    Interplm: Discovering interpretable features in protein language mod- els via sparse autoencoders

    Elana Simon and James Zou. Interplm: Discovering interpretable features in protein language mod- els via sparse autoencoders. bioRxiv, pp. 2024–11,

    work page 2024

  6. [12]

    Unpacking sdxl turbo: Interpreting text-to-image models with sparse autoencoders

    Viacheslav Surkov, Chris Wendler, Mikhail Terekhov, Justin Deschenaux, Robert West, and Caglar Gulcehre. Unpacking sdxl turbo: Interpreting text-to-image models with sparse autoencoders. arXiv preprint arXiv:2410.22366,

    arXiv

  7. [13]

    pub/2024/scaling-monosemanticity/ [Accessed: 2024]

    https://transformer-circuits. pub/2024/scaling-monosemanticity/ [Accessed: 2024]. Uniprot. Uniprot: the universal protein knowledgebase in

    work page 2024

  8. [2014]

    Scaling and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093,

    Leo Gao, Tom Dupr ´e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093,

    arXiv

Show all 15 references
  1. [2017]

    Bertology meets biology: Interpreting attention in protein language models

    Jesse Vig, Ali Madani, Lav R Varshney, Caiming Xiong, Richard Socher, and Nazneen Fatema Ra- jani. Bertology meets biology: Interpreting attention in protein language models. arXiv preprint arXiv:2006.15222,

    2006 arXiv

  2. [2019]

    URL https://www.biorxiv.org/content/10.1101/622803v4

    doi: 10.1101/622803. URL https://www.biorxiv.org/content/10.1101/622803v4. Chakkarai Sathyaseelan, L Ponoop Prasad Patro, and Thenmalarchelvi Rathinavelan. Sequence patterns and hmm profiles to predict proteome wide zinc finger motifs. Pattern Recognition, 135: 109134,

    doi

  3. [2020]

    dead” and we re-initialize its weights to “revive

    A A PPENDIX A.1 S PARSE AUTOENCODER TRAINING A.1.1 T RAINING DATASET We train our model using the Astral SCOPe 2.08 dataset, filtered to 40% sequence identity, which includes approximately 15k highly non-redundant protein sequences (Fox et al., 2014). This dataset provides a m...

    2014

  4. [2022]

    Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al

    https://transformer-circuits.pub/ 2022/toy_model/index.html. Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: Toward un- derstanding the language of life ...

    2022

  5. [2023]

    pub/2023/monosemantic-features/index.html [Accessed: 2024]

    https://transformer-circuits. pub/2023/monosemantic-features/index.html [Accessed: 2024]. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600,

    2023 arXiv

  6. [2024]

    The missing curve detectors of inceptionv1: Applying sparse autoencoders to incep- tionv1 early vision

    Liv Gorton. The missing curve detectors of inceptionv1: Applying sparse autoencoders to incep- tionv1 early vision. arXiv preprint arXiv:2406.03662,

    arXiv

  7. [2025]

    URL https: //www.biorxiv.org/content/early/2025/02/08/2025.02.06.636901

    doi: 10.1101/2025.02.06.636901. URL https: //www.biorxiv.org/content/early/2025/02/08/2025.02.06.636901. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: ...

    2025 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.