Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models

Arjun Banerjee; Camille Dang; David Martinez; Ethan Tam

REVIEW

Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models

Arjun Banerjee, David Martinez, Camille Dang, Ethan Tam

Machine Learning · Submitted 8 Jul 2025 · 13 references

Pith's verdict: First neuron-labeling-plus-steering pipeline for PLMs, with public code and data, but the label-validation loop doesn't prove the text matters and the generation results are mostly qualitative.

T0 · AI review

How this review was made: An AI referee (deepseek-v4-flash) read the paper on August 6, 2026 and answered a fixed list of questions about it. T0 is the starting level every paper gets. The levels describe how deeply Pith has checked the paper, never the authors.

T0 AI review An AI read the paper and answered a fixed list of questions.
T1 Audit Automatic checks that the references and files the paper relies on exist and match.
T2 Compute The paper's code or calculations re-run with the same results.
T3 Formal statement The claims written out in a formal math language a computer can read.
T4 Proof Those claims proved, and the proof checked by computer.

How Pith checks papers →

· 4 major objections · 6 minor · Ask about this paper · The compiled record · Read the full review →

Challenge this review Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID2507.06458 (version 1)

Pith IDWVQTHEDE, Pith's permanent record of this paper and its review

CitationArjun Banerjee, Camille Dang, David Martinez, Ethan Tam. "Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models." arXiv:2507.06458 (2025). https://pith.science/paper/WVQTHEDE

Cite this reviewBibTeX for Pith's review of this paper

Claim paper

Why Pith says this

Highlight any text to comment on it.

Editor's note

read the letter

Quick take: this is a genuine first—an automated pipeline that labels individual ESM2 neurons with natural language and then steers generation by modifying those neuron activations. The code and labels are public, which is a real plus. But the paper leans on a label-validation step that doesn't actually prove the label text is doing the work, and the generation evidence is mostly qualitative.

The new bit is the combination: prior work used SAEs or heuristic probing; this goes after individual neurons with an LLM explainer and a Longformer simulator, then applies affine interventions to a handful of selected neurons to push a random sequence toward a target property. The control experiment with random neurons is a good idea, and the negative-steering results are honest about being feature-dependent. The authors also clearly admit the generated proteins don't fold (ΔG 99–172 kcal/mol) and that labels can be hallucinated. That candor earns credit.

The weak spot is the r(h) validation. The simulator is fine-tuned on the same feature set the explainer sees, and the paper never checks whether the hypothesis text contributes anything beyond the raw features. The stress-test note is right: for a fixed neuron, h is constant when r(h) is computed, so a high correlation mainly means the simulator can predict activation from sequence/features. The authors do compare r(h) across candidate hypotheses for the same neuron, so the text *can* influence the ranking, but there is no features-only or permuted-label baseline to show that the selected labels carry information. Without that, the 'accurate description' claim is not supported.

The generation results are also underpowered: single trajectories, no error bars, and the affine coefficients are hand-tuned differently for the 8M and 35M models (A=10 vs A=200). One zinc finger in 100 generations is a near-miss, not a demonstration. The 'scaling law' is a qualitative trend in label frequencies across three models—it's a decent observation about how feature granularity grows with size, but calling it a law is a stretch.

None of this kills the paper. The steering direction seems real—the random-neuron control and the monotonic divergence for molecular weight, GRAVY, and instability suggest the labels are pointing at something. For a workshop paper, this is a reasonable proof of concept, and the released resources make it citable.

My recommendation: engage with it as a serious but major-revision paper. A referee should ask for a features-only ablation, seeded repeats with error bars, a random-hypothesis control, and a quantitative operationalization of the scaling claim. If those land, this becomes a solid contribution to PLM interpretability.

Referee report

Major objections (4)

[Section 2.2, Simulator Model] The simulator score r(h) cannot validate the semantic content of a hypothesis as written. For a fixed neuron, h is constant across the held-out sequences used to compute r(h), so the only inputs that vary are the sequence x_j and the feature vector f_j; these are exactly the same features the explainer used to propose h and on which the simulator was trained. A high r(h) can therefore reflect the simulator's ability to predict activation from sequence and features alone, independent of whether the label text is a true description. No features-only baseline, no hypothesis-permutation control, and no human evaluation are reported, and Section 5.1 explicitly concedes that oversimplified or hallucinated labels are possible and defers human assessment to future work. Because the generator's neuron selection (Section 2.2, Generator) is driven by these labels, the validity of the entire steering and interpretability pipeline rests on this unvalidated premise. I request an ablation showing how r(h) changes when h is removed, permuted, or replaced by a random hypothesis, together with the distribution of r(h) values across neurons.
[Section 3.1, Figures 2 and 3] The characteristic-steering evidence is qualitative. The plots show trajectories with no error bars, no number of independent random restarts, and no quantitative summary such as final means, standard deviations, or effect sizes; the text only asserts that the trajectories 'diverge.' The control condition is described in one sentence ('the control experiments show no random change') but no control trajectory, table, or statistic is provided. Since the affine intervention parameters A and B are free per model (A=10/B=3 for the 35M model and A=200/B=10 for the 8M model), and no sensitivity analysis is reported, the robustness of the divergence to these choices is unknown. Please add repeated-run statistics, control trajectories, and a sensitivity analysis over A and B.
[Section 3.2, Structural Guidance] The structural-guidance claim is supported by one successful canonical C2H2 zinc finger out of 100 generations, two 'near hit' examples, and secondary-structure predictions from PsiPred with no baseline comparison or quantitative measure of enrichment over unsteered or randomly steered sequences. The authors themselves report in Appendix C that all generated structures have ΔG values of 99.64, 113.05, and 172.49 kcal/mol, far above the typical 5–15 kcal/mol range, so the generated proteins are not expected to fold. As stated, the paper demonstrates in-silico motif appearance under steering, not generation of viable structural targets. The claims in the abstract and Section 3.2 should be calibrated accordingly, or additional evidence should be provided, such as comparison to unsteered generation and AlphaFold/ESMFold foldability scores.
[Section 4, Figures 8 and 9] The scaling-law and feature-location conclusions are based on relative occurrence of labels by layer sextile, with no error bars, no neuron counts, and no statistical test; the model list in the first paragraph also repeats 'esm2 t36 8B UR50D' twice, making the comparison unclear. Even setting aside the label-validity issue raised in the first major comment, these plots do not by themselves establish a scaling law. Please report raw counts, confidence intervals, and a statistical test (for example, a trend test across model sizes), and correct the model designations.

Minor objections (6)

[Abstract and Figure 6 caption] The abstract contains 'raveling' where 'revealing' is intended, and the Figure 6 caption contains 'seqeunce' instead of 'sequence.'
[Section 4, first paragraph] The list of labeled models reads 'esm2 t36 8B UR50D, esm2 t12 8B UR50D, and esm2 t36 8B UR50D'; this appears to be a copy-paste error and conflicts with the ESM-8M, ESM-35M, and ESM-3B labels used in Figures 8 and 9.
[Section 2.2, Dataset and Simulator] The normalization of activation values and the choice of 11 activation buckets are described as 'empirically found to be optimal' without reporting the criterion used; please provide the exact procedure and rationale.
[Section 3.1] 'GRA VY' should be written as 'GRAVY' throughout the paper.
[Appendix C] The ΔG decomposition is reported without method details; since these numbers support the non-viability statement, a brief description of the implementation or an explicit citation to the exact FoldX/Guerois procedure would help reproducibility.
[Section 5.1 and Section 4] Minor grammar issues: 'condense variant' should be 'condensed variant' and 'hierarchal' should be 'hierarchical.'

How Pith checks papers → · Full audit of this paper →

The paper

The paper claims

Automated natural-language neuron labels provide a gradient-free interface for steering protein language model outputs toward specified biochemical and structural properties.

The reading

The paper sets out to show that every neuron in a protein language model can be given a concise natural-language description of the biological feature it responds to, and that those descriptions are accurate enough to act as control knobs. It builds an explainer–simulator loop: an LLM proposes candidate hypotheses from the sequences that activate a neuron most strongly, and a simulator scores each hypothesis by how well it predicts the neuron's activation, with the highest-correlation hypothesis becoming the label. The paper then uses the labels to select neurons and apply an affine activation intervention during masked inpainting, reporting that generated sequences converge toward target biochemical properties such as molecular weight, GRAVY score, and instability index, and toward structural motifs such as alpha helices, beta strands, and a zinc-finger pattern. It also claims that the labeled neurons reveal a hierarchical layer organization and a scaling trend across model sizes. If true, this gives a scalable, gradient-free route from opaque model internals to controllable protein generation.

What carries the argument

The load-bearing mechanism is the explainer–simulator loop. For each neuron $\phi$, an explainer LLM receives the $k$ top-activating sequences together with their biological feature vectors and proposes $m$ candidate hypotheses; a simulator $S(h, x, f)$, a fine-tuned long-context transformer, predicts the neuron's activation from the hypothesis, sequence, and features. Hypotheses are ranked by the Pearson correlation $r(h) = \mathrm{Corr}(S(h, x_j, f_j), \phi(x_j))$ on held-out data, and the top hypothesis becomes the neuron's label. The generator then converts labels into interventions: an LLM selects neurons relevant to the target property and masked inpainting applies the affine activation change $z^*_k = a z_k(x) + b$ to push generation toward the property. This makes the labeled neuron basis the control interface, avoiding extra trained autoencoders.

Extended reading notes

Core claim

The central claim is that neuron labels produced by the explainer and simulator are a valid, gradient-free control interface for protein language models. Given a target property $\tau$, the method finds neurons whose labels indicate sensitivity to $\tau$, then applies $z^*_k = a z_k(x) + b$ to their activations during masked inpainting, biasing the model's output toward sequences with enhanced expression of $\tau$. The evidence is that steering trajectories for 'high' and 'low' variants of molecular weight, GRAVY score, and instability index diverge monotonically while a random-neuron control does not change the target metrics; structural steering yields $\alpha$-helix and $\beta$-strand content and, in one of 100 generated sequences, a canonical C2H2 zinc-finger motif. On the interpretability side, the paper claims that labels concentrate by layer type—local biochemical properties early, structural features in the middle, functional roles late—and that larger models encode more fine-grained features and distribute feature attribution more broadly.

Load-bearing premise

The load-bearing premise is that the simulator's Pearson correlation validates a hypothesis as a true description of a neuron, but since the simulator is trained on the same biological feature set used to generate the hypotheses and no features-only baseline is reported, the selection could be reflecting the raw features rather than the label text.

Editorial extensions

If this is right

Neuron-level labels give a way to inspect all neurons in a protein language model without training sparse autoencoders, at scales of hundreds of thousands of neurons.
Label-guided steering produces monotonic divergence between high and low trajectories for molecular weight, GRAVY score, and instability index, while random-neuron controls show no effect.
The same interventions can generate sequences enriched in secondary structure such as alpha helices and beta strands, and occasionally canonical tertiary motifs such as C2H2 zinc fingers.
Label distributions support a hierarchical picture: low-level biochemical features dominate early layers, structural features the middle, and functional roles the later layers.
Larger models appear to capture finer-grained features and spread functional attribution more broadly across layers, consistent with a scaling trend rather than simple parameter addition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the label-validity claim holds, multi-objective control is the direct next step: activating several labeled neuron sets together should push one sequence toward combined properties such as stability and binding, which the paper lists as future work.
The scaling-law results are indirect, because labels are generated by an LLM from a fixed feature vocabulary; a controlled study using identical prompts and features across model sizes would separate model-internal trends from explainer artifacts.
The same explainer–simulator pipeline could be applied to hidden units in structure-prediction models, not just sequence models, if their internal activations are collected on a labeled dataset.
Since the steering intervention is affine and gradient-free, it could be composed with a foldability oracle that rejects high-$\Delta G$ sequences; the paper's own $\Delta G$ analysis suggests viability is the current bottleneck.

Original abstract

Protein language models (PLMs) encode rich biological information, yet their internal neuron representations are poorly understood. We introduce the first automated framework for labeling every neuron in a PLM with biologically grounded natural language descriptions. Unlike prior approaches relying on sparse autoencoders or manual annotation, our method scales to hundreds of thousands of neurons, revealing individual neurons are selectively sensitive to diverse biochemical and structural properties. We then develop a novel neuron activation-guided steering method to generate proteins with desired traits, enabling convergence to target biochemical properties like molecular weight and instability index as well as secondary and tertiary structural motifs, including alpha helices and canonical Zinc Fingers. We finally show that analysis of labeled neurons in different model sizes reveals PLM scaling laws and a structured neuron space distribution.

The compiled record

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 4,272 input tokens.

What the abstract itself states, as probabilities: claims first 0.95 · states limitation 0.08 · human participants 0.03 · controlled comparison 0.13 · randomized assignment 0.01 · states quantitative result 0.26 · compares with prior results 0.89 · mentions code or data availability 0.03.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; show every role.

Protein language models (PLMs) encode rich biological information, yet their internal neuron representations are poorly understood. We introduce the first automated framework for labeling every neuron in a PLM with biologically grounded natural language descriptions. Unlike prior approaches relying on sparse autoencoders or manual annotation, our method scales to hundreds of thousands of neurons, revealing individual neurons are selectively sensitive to diverse biochemical and structural properties. We then develop a novel neuron activation-guided steering method to generate proteins with desired traits, enabling convergence to target biochemical properties like molecular weight and instability index as well as secondary and tertiary structural motifs, including alpha helices and canonical Zinc Fingers. We finally show that analysis of labeled neurons in different model sizes reveals PLM scaling laws and a structured neuron space distribution.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	Protein language models (PLMs) encode rich biological information, yet their internal neuron representations are poorly understood.	background 0.61	0.05	0.03	asserted 1.00
s1	We introduce the first automated framework for labeling every neuron in a PLM with biologically grounded natural language descriptions.	contribution 0.99	0.96	0.15	asserted 1.00
s2	Unlike prior approaches relying on sparse autoencoders or manual annotation, our method scales to hundreds of thousands of neurons, revealing individual neurons are selectively sensitive to diverse biochemical and structural properties.	contribution 0.51	0.91	0.17	asserted 1.00
s3	We then develop a novel neuron activation-guided steering method to generate proteins with desired traits, enabling convergence to target biochemical properties like molecular weight and instability index as well as secondary and tertiary structural motifs, including alpha helices and canonical Zinc Fingers.	contribution 0.64	0.96	0.21	asserted 1.00
s4	We finally show that analysis of labeled neurons in different model sizes reveals PLM scaling laws and a structured neuron space distribution.	result 0.91	0.96	0.21	asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from simulation or computation 0.59.

What was examined, and what was not

sources read

arXiv listing abstract v1

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

Figures

**Figure 1.** Pipeline for Automatic Neuron Labelling 1. Dataset Construction: We create a dataset D = {(xi , fi , ϕ(xi))} n i=1, ϕ(xi) ∈ R+ is the normalized activation of neuron ϕ on a protein sequence xi . . 2. Explainer Model: We create an explainer function E : Dtop ⊂ D → H which maps a subset of top-activating examples to a hypothesis h in natural language. 3. Simulator Model: We create a simulator S : H × 1All code is available at: https://github.com/arjunbanerjee/PLMNeuron X × F → [0, 10] which predicts a discretized activation score ϕˆ(x) given a description h, a sequence x, and i

**Figure 2.** Steering with esm2 t12 35M, A = 10 and B = 3

**Figure 3.** Steering with esm2 t6 8M, A = 200 and B = 10 3

**Figure 4.** Steering with esm2 t12 35M, A = -10 and B = -5 We find that the negative steering relationship is highly feature dependent; for instance, in the case of GRAVY score, negative steering led to the complete inverse result of positive steering, while in the case of instability index it led to no conclusive direction changes. Interestingly, in the case of molecular weight, negative steering of low molecular weight and negative steering of high molecular weight both increased the weight. 3.2. Structural Guidance 3.2.1. SECONDARY STRUCTURE GUIDANCE We start by testing the ability of our…

**Figure 5.** (Left): Visualization of the starting protein; (Mid

**Figure 6.** PsiPred outputs of the neutral sequence (top),

**Figure 7.** Visualization of a Generated Zinc Finger Motif

**Figure 8.** Relative occurance of structural features for each

**Figure 10.** Distribution of features on label dataset

**Figure 11.** Relative Position of the Generated Zinc Finger Protein

**Figure 12.** Relative Position of the Generated Alpha Helix Protein

**Figure 13.** Relative Position of the Generated Beta Strand Protein

Showing 12 of the paper's 13 figures. See all figures in the PDF ↗ Figures reproduced from arXiv:2507.06458 by the authors.

References

Works cited (13)

[1]

Examine the protein sequences and their corresponding biological features

View on Pith
[2]

Pay attention to the activation values for each sequence

View on Pith
[3]

Look for common patterns or characteristics among sequences with high activation values

View on Pith
[4]

Consider your knowledge of biology to interpret the significance of these patterns. Formulate your description:

View on Pith
[6]

Identify the most important 1-2 features that consistently appear in sequences with high activation

View on Pith
[7]

Focus on features that are common across most or all high-activation sequences

View on Pith
[8]

Disregard features that vary significantly among the examples

View on Pith
[9]

Create a concise, one-sentence description that captures the essence of what causes the neuron to strongly activate. Output your final description inside <neuron_description> tags. Ensure your description: - Is limited to one sentence - Uses as few words as possible - Directly states the relevant features without introductory phrases - Describes only cons...

View on Pith
[10]

Carefully read through all the sequences and their associated biological features

View on Pith
[11]

Identify common themes or patterns in the biological features across the sequences

View on Pith
[12]

Focus on the most prominent and frequently occurring features

View on Pith
[13]

Synthesize these common features into a single, concise statement. Your summary should capture the essence of the shared biological features using the fewest words possible while still conveying the key information. Provide your summary in the following format: <summary> [Your one-sentence summary of shared biological features] </summary> Remember, brevit...

View on Pith
[2022]

doi: 10.1109/TPAMI.2021.3095381. Epub 2022 Sep 14. PMID: 34232869. Farrell, E., Lau, Y .-T., and Conmy, A. Applying sparse autoencoders to unlearn knowledge in language mod- els, 2024. URL https://arxiv.org/abs/2410. 19278. arXiv preprint arXiv:2410.19278 (v2, updated Nov 2, 2024); Subjects: Machine Learning (cs.LG); Arti- ficial Intelligence (cs.AI). Gar...

View on Pith arXiv 2021

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Generative Latent Priors for Interpreting and Steering Protein Language Models
cs.AI 2026-11 conditional novelty 6.0 of 10

A generative prior learned on ESM-2 activations yields the strongest single-unit probes of Swiss-Prot concepts at both host scales, an edge the paper shows is mostly concentration, while its correction of steered acti...
Efficient Auto-Interpretability of AI Models in Biology
q-bio.QM 2026-08 conditional novelty 6.0 of 10

Cross-seed stability of sparse autoencoder features in Boltz-1 predicts which hidden features are interpretable, cutting LLM interpretation cost by 4.4 to 5.2 times while recovering over half of the interpretable latents.

How the literature uses it

1 arXiv paper cite this in 1 sentence

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

What it cites · 6 sentences without a confident label

Uses the method or tool of 1 paper

We fine-tune Longformer to serve as the simulator due to its ability to handle long context lengths [ref] .

Longformer: The Long-Document Transformer · Model · 0.99

Cites as background 1 paper

Early studies in PLM interpretability relied largely on heuristic probing, which showed that PLM attention heads encoded structural information about proteins [ref] and are able to identify functional sites like allosteric residues [ref] .

BERTology Meets Biology: Interpreting Attention in Protein Language Models · Introduction · 0.82

Discussion

Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.

[1] [1]

Examine the protein sequences and their corresponding biological features

View on Pith

[2] [2]

Pay attention to the activation values for each sequence

View on Pith

[3] [3]

Look for common patterns or characteristics among sequences with high activation values

View on Pith

[4] [4]

Consider your knowledge of biology to interpret the significance of these patterns. Formulate your description:

View on Pith

[5] [6]

Identify the most important 1-2 features that consistently appear in sequences with high activation

View on Pith

[6] [7]

Focus on features that are common across most or all high-activation sequences

View on Pith

[7] [8]

Disregard features that vary significantly among the examples

View on Pith

[8] [9]

Create a concise, one-sentence description that captures the essence of what causes the neuron to strongly activate. Output your final description inside <neuron_description> tags. Ensure your description: - Is limited to one sentence - Uses as few words as possible - Directly states the relevant features without introductory phrases - Describes only cons...

View on Pith

[9] [10]

Carefully read through all the sequences and their associated biological features

View on Pith

[10] [11]

Identify common themes or patterns in the biological features across the sequences

View on Pith

[11] [12]

Focus on the most prominent and frequently occurring features

View on Pith

[12] [13]

Synthesize these common features into a single, concise statement. Your summary should capture the essence of the shared biological features using the fewest words possible while still conveying the key information. Provide your summary in the following format: <summary> [Your one-sentence summary of shared biological features] </summary> Remember, brevit...

View on Pith

[13] [2022]

doi: 10.1109/TPAMI.2021.3095381. Epub 2022 Sep 14. PMID: 34232869. Farrell, E., Lau, Y .-T., and Conmy, A. Applying sparse autoencoders to unlearn knowledge in language mod- els, 2024. URL https://arxiv.org/abs/2410. 19278. arXiv preprint arXiv:2410.19278 (v2, updated Nov 2, 2024); Subjects: Machine Learning (cs.LG); Arti- ficial Intelligence (cs.AI). Gar...

View on Pith arXiv 2021