Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Protein language models complete repeats by combining induction heads that copy the aligned residue from the other repeat copy with neurons encoding amino-acid similarity; the approximate-repeat circuit contains and generalizes the exact-repeat circuit.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:26 UTC pith:MCECDGOT

load-bearing objection First mechanistic map of repeat completion in PLMs, with a solid induction-head core and an under-supported BLOSUM62 layer. the 4 major comments →

arxiv 2602.23179 v5 pith:MCECDGOT submitted 2026-02-26 cs.LG q-bio.BM

Induction Meets Biology: Mechanisms of Repeat Detection in Protein Language Models

Gal Pomerants , Yaniv Nikankin , Anja Reusch , Tomer Tsaban , Ora Schueler-Furman , Yonatan Belinkov This is my paper
classification cs.LG q-bio.BM
keywords plmsrepeatsproteinapproximateexactsegmentsbiologicalheads
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Proteins are chains of amino acids that often contain repeated segments. These repeats matter for structure and function, and biology has many algorithms for finding them. Language models trained on protein sequences can also complete repeats: when a token is masked inside a repeat, the model looks at the matching position in the other copy and predicts the right amino acid with high accuracy (79–99% depending on the setting, versus ~30% for random guessing).

The authors reverse-engineered how two protein models (ESM-3 and ESM-C) do this. They used attribution patching: corrupt the second copy of the repeat, see which model components' behavior changes, and thereby find the small set of attention heads and MLP neurons that drive the prediction. Three kinds of attention heads appear: heads that attend to fixed positional offsets, 'induction' heads that attend from the masked position to the aligned position in the other copy (the same pattern-matching-and-copying mechanism found in text language models), and heads biased toward particular amino acids. Among MLP neurons, the important ones fall into categories: neurons selective for a single amino acid, neurons selective for groups of biochemically similar amino acids (matching the BLOSUM62 substitution matrix used by classical alignment tools), and neurons that activate throughout repeated regions.

The proposed mechanism is three-stage. Early layers use position heads and biochemical neurons to build representations; middle layers use induction heads to copy the aligned residue, while repeat-region neurons play an inhibitory role; final layers shape the output distribution. The approximate-repeat circuit contains the exact-repeat circuit and works for both, so the authors call it the broader mechanism. ESM-3 additional

Core claim

The central claim: PLMs detect approximate repeats via a two-stage circuit — "PLMs first build feature representations using both general positional attention heads and biologically specialized components, such as neurons that encode amino-acid similarity. Then, induction heads attend to aligned tokens across repeated segments, promoting the correct answer" (Abstract). The approximate-repeat circuit functionally subsumes the exact-repeat circuit (Sec. 3.2, Fig. 3), and this integration of LLM-style induction with BLOSUM62-aligned biochemical neurons is what the paper rises or falls on. If correct, repeat completion in ESM-3/ESM-C is a modular computation over identifiable, interpretable components.

Load-bearing premise

Load-bearing premise: the approximate-repeat task, together with the success filter, isolates 'repeat detection' as a single mechanism. The task masks only positions that are identical to their aligned counterpart and adjacent to a substitution site (Sec. 2, Table 3); indel-containing repeats are excluded because accuracy collapses to 42.9–48.1% (App. C.1); and every circuit and neuron analysis is restricted to sequences where the model already scores 100% (synthetic/identical) or >80% (approximate) (App. G.1). If the model succeeds on this slice via positional copy without genuinely assessing biochemical similarity — or uses different machinery on substituted sites, indels, or hard examples — then the two-stage mechanism is a mechanism for easy masked-copy completion, not for "how PLMs detect both exact and approximate repeats" as claimed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper reverse-engineers repeat detection in two protein language models, ESM-3 and ESM-C. Using attribution patching with integrated gradients (AP-IG), the authors discover sparse circuits for synthetic, natural-identical, and natural-approximate repeat tasks, reporting that 14–17% of components suffice for 85% faithfulness in ESM-3 and about 25% in ESM-C. Structural and functional comparisons lead them to claim that the approximate-repeat circuit subsumes the exact-repeat circuits. Individual component analysis clusters attention heads into relative-position, induction, and amino-acid-biased heads, and identifies MLP neurons selective for amino-acid identity, BLOSUM62 substitution groups, biochemical classes, and repeat regions. Interaction and logit-lens analyses suggest a three-stage mechanism: early layers build aligned representations with position heads and biochemical-similarity neurons, middle-layer induction heads copy the aligned token, and late MLPs refine the prediction. The paper concludes that PLMs combine generic LM-style induction mechanisms with protein-specific biological knowledge.

Significance. If the central claim holds, this is a strong mechanistic interpretability contribution: it extends circuit analysis from text LMs to protein LMs, documents the same core mechanism in two independently trained models, and identifies interpretable individual neurons without sparse autoencoders. The paper also ships code/data and reports faithfulness in a quantitative framework, which is a methodological strength. The convergence with known induction-head findings in LLMs is a useful external anchor. The main significance risk is that the task definition and counterfactual choice may not isolate 'approximate repeat detection requiring biochemical similarity'; several concrete controls are needed before the claimed biological specialization is established.

major comments (4)
  1. [§2, Table 3; App. C.1, G.1] The approximate-repeat task masks only positions that are identical to their aligned counterpart and adjacent to a substitution; indel repeats are excluded (App. C.1). Since substitution-only repeats are gap-free, the aligned source token is at a constant sequence offset from the masked token. A fixed-offset, exact-copy mechanism therefore suffices for every eligible position, and biochemical similarity is not needed to predict the masked residue. The Section 5 claim that Stage (I) uses 'biochemical similarity neurons' to align tokens across repeats is not uniquely supported, especially because circuit analysis is restricted to sequences with >80% accuracy (App. G.1), which may select for offset-copy successes. Please add a control that masks substituted positions (or otherwise requires similarity-based alignment), and/or a fixed-offset exact-copy baseline.
  2. [§3.1 vs. §4.2; App. F] AP-IG counterfactuals are constructed by replacing every residue in the non-masked repeat with its BLOSUM62-best substitution (§3.1). The importance ranking therefore selects components sensitive to BLOSUM62-defined residue changes. The same selected components are then labeled using concepts derived from BLOSUM62 cliques (§4.2, App. I.2). This pipeline risks a selection circularity: any residue-sensitive neuron is likely to be important under a residue-substitution perturbation, and the BLOSUM62 concept vocabulary may match some of them by chance. The biological-specialization conclusion would be much stronger if the neuron-concept analysis were repeated on circuits discovered under a non-BLOSUM counterfactual (e.g., 100% Mask or Permutation in App. F) and shown to still enrich for BLOSUM62 groups.
  3. [§3.2, Fig. 3; §3.1, Fig. 2] The claim that the approximate-repeat circuit 'functionally subsumes' the exact-repeat circuits is confounded by circuit size: approximate circuits use ~17% of components vs ~14% for identical/synthetic circuits (Fig. 2). A larger circuit trivially has higher recall, and after normalization can report >1.0 cross-task faithfulness (Fig. 3c). No random or size-matched circuit baseline is provided. Please report cross-task faithfulness for size-matched random circuits, or use a metric that controls for circuit cardinality, before concluding functional subsumption.
  4. [§4.2, Fig. 9; §5] The causal role of the 'biochemical similarity' neuron group is not established. Figure 9 reports attribution importances and Fig. 7 reports aggregated EAP-IG interactions; both are correlational summaries. The paper never ablates the BLOSUM62/IMGT/secondary-structure neuron group (or AA-biased heads) and compares approximate vs exact repeat faithfulness. Without such a group ablation, the central claim that PLMs use biological specialization for approximate repeats is not directly tested. Please add group-level ablation experiments, e.g., counterfactually ablating the top BLOSUM62-matched neurons while keeping other circuit components intact, on both identical and approximate tasks.
minor comments (4)
  1. [App. G.3] There is a duplicated sentence beginning 'For each circuit, we fix...' in the neuron-level analysis section; please remove the repetition.
  2. [Table 7] In the Induction Score definition, the Copying term uses j* but the arg-max definition appears only in the Pattern Matching term; clarify the notation and the order of definitions.
  3. [App. F.2] The text says 'we proceed with multiple counterfactuals for circuit discovery and compare the resulting circuits,' but the main pipeline uses only 100% BLOSUM; please reconcile this sentence with the actual primary choice, or point readers to F.3 for the comparison.
  4. [Fig. 6] Panel (B) is dense; the caption could state more explicitly that 'Biological (Other)' excludes amino-acid identity, BLOSUM62, and repeat concepts, since the color legend alone is ambiguous.

Circularity Check

1 steps flagged

BLOSUM62 is used both to build circuit-discovery counterfactuals and to label the resulting neurons, partially embedding the biological-similarity conclusion; the core induction-head mechanism remains independently measured.

specific steps
  1. other [Section 3.1 (Circuit Discovery and Evaluation) and Section 4.2 (Quantitative analysis) / Appendix I.2]
    "we construct each counterfactual ŝi by replacing every amino acid in the non-masked repeat occurrence with its highest-matching BLOSUM62 substitution (Henikoff & Henikoff, 1992) ... as well as groupings derived from the BLOSUM62 substitution matrix by identifying amino-acid groups that are more likely to substitute for one another."

    BLOSUM62 is used twice in the same pipeline: first as the intervention that defines which neurons are 'important' for the approximate-repeat circuit (AP-IG counterfactuals), and second as one of the concept families used to label those neurons (BLOSUM62 cliques). A neuron that responds to the BLOSUM62 substitution pattern will be selected as important and then matched to a BLOSUM62-clique concept, so the finding that 'many important neurons ... are associated with BLOSUM62 groups' is partly guaranteed by the shared input. The paper's conclusion that PLM similarity 'aligns with' BLOSUM62 is therefore not fully independent of the experimental construction. Although Appendix F.3 shows that other counterfactuals recover overlapping circuits, the primary neuron analysis is run on the BLOSUM-sel

full rationale

The central claim—that PLMs detect approximate repeats through a two-stage circuit of relative-position/biological feature-building followed by induction-head copying—is not circular as a whole. The induction heads and relative-position heads are measured directly from attention maps and logit-lens contributions, and the induction-head finding converges with external LLM literature (Olsson et al., 2022). Circuit faithfulness is evaluated on separate subsets, and the approximate-repeat circuit's functional generalization to exact and synthetic repeats is an empirical cross-task result, not a fitted prediction. The paper's self-citations (Hanna et al., 2024; Nikankin et al., 2025a) provide methodology and faithfulness thresholds but are not the load-bearing evidence for the biological conclusion. The main circularity concern is local: BLOSUM62 is used both to construct the counterfactuals that rank neurons as important and to define the BLOSUM62-based concepts used to label those same neurons. This double use partially explains the finding of BLOSUM62-associated neurons and weakens the 'internal similarity aligns with BLOSUM62' claim. However, the induction-head mechanism and the general circuit structure do not reduce to this construction, and the paper itself acknowledges the inductive bias of manually defined concepts in the Limitations. I therefore assign 4 rather than a higher score: one substantive partial circularity, with the central mechanism retaining independent empirical content.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The ledger captures the choices the central mechanism claim imports without independent justification: the task rules that define 'approximate repeat', the thresholds that define the circuit and its neuron groups, and the tooling/biological classification systems the interpretation leans on. No new entities (particles, forces, conserved quantities) are introduced; the 'concepts' used to label neurons are analytical categories drawn from prior biochemical classifications. None of the entries is a curve-fit, but several (cluster count, AUROC thresholds, BLOSUM62-as-truth) shape the reported mechanism and should be read as premises.

free parameters (5)
  • Circuit sufficiency threshold = 85% faithfulness
    Convention from prior work (Hanna et al. 2025; Nikankin et al. 2025a). Determines which components enter the circuit and hence all downstream characterization.
  • Attention-head cluster number k = k=3 for both models
    Elbow at k=3 for both; silhouette favors k=3 for ESM-3 but k=2 for ESM-C. Authors chose k=3 for ESM-C to separate AA-biased from relative-position heads (App. H.3); the three-way head taxonomy underlies the mechanism narrative.
  • Neuron retention percentage = 3% (ESM-3), 7% (ESM-C) per layer
    Set to recover ~80% circuit faithfulness (App. G.3); combined with the 4/5-seed consistency filter, defines the 3,394 / 3,510 analyzed neurons.
  • Concept-match AUROC thresholds = 0.75 (most concepts); 0.99 (special/aligned-token concepts)
    Thresholds for labeling neurons as concept-selective; the ESM-3 vs ESM-C biological-neuron comparison (Fig. 6c) counts neurons at AUROC ≥ 0.75.
  • Mask eligibility rule = positions identical to aligned counterpart, adjacent to a substitution; substitution rate ≤ 50%
    Task definition (Sec. 2, Table 3). The model is never asked to predict a substituted site, which bounds what 'approximate repeat detection' means.
axioms (6)
  • domain assumption AP-IG attribution scores approximate the true causal importance of each component
    All circuit discovery rests on attribution patching with integrated gradients (Sec. 3.1); the authors mitigate known variability (Méloux et al. 2025) via 4/5-seed consistency, but correctness depends on this premise.
  • domain assumption Faithfulness (Eq. 1) is a valid measure of mechanism sufficiency
    Used to define 'the mechanism' and to support the subsumption claim; values >1.0 in cross-task normalization (Fig. 3c) show the metric is not a clean probability measure.
  • domain assumption RADAR on SEG-masked sequences correctly identifies the ground-truth repeats
    Dataset curation (App. B) uses RADAR calls to define the task and the repeat-region concepts for neurons; errors would propagate to both.
  • standard math BLOSUM62 is the correct similarity model both for counterfactuals and for neuron concepts
    Used at both ends — to build corruptions that define importance (Sec. 3.1) and as the hypothesis set for labeling neurons (App. I.2) — introducing a mild circular pressure toward finding BLOSUM62-aligned neurons.
  • standard math IMGT physicochemical and Chou-Fasman secondary-structure groupings are meaningful biological categories
    Standard classifications (Pommie et al. 2004; Chou & Fasman 1978) used to label neurons, e.g., 'helix-breaker neurons'; the labels inherit the validity of the groupings.
  • domain assumption The TransformerLens fork computes ESM-3/ESM-C forward passes, patching, and ablations correctly
    All experiments depend on this tooling, which is promised but not yet public (App. G.2).

pith-pipeline@v1.3.0-alltime-deepseek · 4805 in / 4799 out tokens · 193175 ms · 2026-08-02T20:26:27.513768+00:00 · methodology

0 comments
read the original abstract

Protein sequences are abundant in repeating segments, both as exact copies and as approximate segments with mutations. These repeats are important for protein structure and function, motivating decades of algorithmic work on repeat identification. Recent work has shown that protein language models (PLMs) identify repeats, by examining their behavior in masked-token prediction. To elucidate their internal mechanisms, we investigate how PLMs detect both exact and approximate repeats. We find that the mechanism for approximate repeats functionally subsumes that of exact repeats. We then characterize this mechanism, revealing two main stages: PLMs first build feature representations using both general positional attention heads and biologically specialized components, such as neurons that encode amino-acid similarity. Then, induction heads attend to aligned tokens across repeated segments, promoting the correct answer. Our results reveal how PLMs solve this biological task by combining language-based pattern matching with specialized biological knowledge, thereby establishing a basis for studying more complex evolutionary processes in PLMs.

Figures

Figures reproduced from arXiv: 2602.23179 by Anja Reusch, Gal Pomerants, Ora Schueler-Furman, Tomer Tsaban, Yaniv Nikankin, Yonatan Belinkov.

Figure 1
Figure 1. Figure 1: Visualization of the repeat identification mechanism. The model predicts the masked token by integrating repetition-related (left) and biological features (right). (I) First, relative-position attention heads attend to tokens located at fixed offsets (±n) from the masked position, followed by the activation of biologically specialized neurons, such as neurons that selective for biochemically similar amino … view at source ↗
Figure 2
Figure 2. Figure 2: ESM-3 circuit faithfulness scores. Across the three tasks, the discovered circuits achieve high faithfulness (above the 85% threshold) using a small fraction of model components. how changing its activation from its value on si to its value on sˆi affects the log probability Lai of the correct amino acid completion ai in the masked position. Intuitively, if a component is important then changing its activa… view at source ↗
Figure 3
Figure 3. Figure 3: Cross-Task Circuit Comparisons in ESM-3. We compare the IoU, recall and cross-task faithfulness of the circuits found for the three repeat tasks. The IoU (left) shows relatively high overlap between all three tasks. The recall (middle) measures the fraction of the ground-truth circuit (x-axis) recovered by the predicted circuit (y-axis), and shows that the synthetic and identical circuits are largely subsu… view at source ↗
Figure 4
Figure 4. Figure 4: Active attention patterns in ESM-3. (a–c) show exam￾ple attention maps from three attention heads in the approximate￾repeat circuit, for a single representative input: (a) fixed relative￾position attention (diagonal); (b) induction attention between aligned repeat positions (two partial diagonals); and (c) amino￾acid–biased attention (vertical). (d) Circuit attention heads are clustered and visualized usin… view at source ↗
Figure 5
Figure 5. Figure 5: Circuit faithfulness of subsets of neurons in ESM-3. 3% of MLP neurons per layer are sufficient to recover most circuit faithfulness (80/85%), indicating sparsity in the neuron basis. rons activate on subsets of amino acids that share similar biochemical properties or exhibit high evolutionary substi￾tution likelihood according to BLOSUM62.5 Third, some neurons activate more strongly within repeated segmen… view at source ↗
Figure 6
Figure 6. Figure 6: (A) Examples of three task-relevant neurons in ESM-3: an amino-acid–specific neuron, a neuron that responds to substitutable amino acids according to BLOSUM62 groupings, and a repeat-focused neuron. We show token-level activations on an example sequence (left) and activation distributions over the full dataset (right). (B) AUROC scores for each neuron’s best-matching concept across ESM-3 layers. Each point… view at source ↗
Figure 7
Figure 7. Figure 7: Aggregated interactions between component groups in ESM-3. Larger scores indicate stronger influence. Induction heads and MLPs show the strongest influence on the logits, with weaker contributions from AA-biased heads. The input primarily affects non-induction components, and interactions are strongest between attention heads and MLPs, highlighting the central role of MLPs in mediating information flow. ap… view at source ↗
Figure 9
Figure 9. Figure 9: Attribution analysis of neuron-group importance across layers in ESM-3. Biochemical-similarity neurons are most important in early layers, amino-acid neurons contribute stronger in later layers, and repeat-related neurons often show inhibitory effects, reflected by negative contributions across layers. (Ismail et al., 2025) and post-hoc methods that analyze trained models; we take a post-hoc approach. Earl… view at source ↗
Figure 10
Figure 10. Figure 10: ESM-C circuit faithfulness scores. Across all three task settings, the discovered circuits achieve high faithfulness using approximately 25% of model components. E.2 Cross-Task Circuits Comparison We repeat the cross-task circuit comparisons (Section 3.1) for ESM-C [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Cross-task circuit comparisons in ESM-C. All circuits show similar structural overlap, while only the approximate repeats circuit exhibits strong functional generalization to the other tasks. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Attention patterns in ESM-C. Example attention maps from three attention heads in ESM-C approximate repeats circuit. for a single representative input, illustrating the same recurring patterns we observe in ESM-3: Left: fixed relative position attention, producing a diagonal pattern; Middle: induction attention between aligned repeat positions, resulting in two partial diagonals due to bidirectional align… view at source ↗
Figure 13
Figure 13. Figure 13: UMAP projection of clustered attention heads in ESM-C, colored by pattern type. E.4 Neuron Classification We find MLP neurons important to the approximate repeat task in ESM-C and rank them (as done in Section 4.2). Results ( [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Circuit faithfulness with a subset of neurons in ESM-C. Retaining approximately 7% of MLP neurons per layer achieves 80% faithfulness. Manual inspection of important neuron activations in ESM-C (see Appendix I.4 for examples) reveals patterns similar to those observed in ESM-3: (i) neurons that exhibit a bias toward specific amino acids; (ii) neurons that activate on subsets of amino acids with similar bi… view at source ↗
Figure 15
Figure 15. Figure 15: AUROC scores for each neuron’s best-matching concept across ESM-C layers. Each point corresponds to a neuron, plotted by layer and AUROC score, and colored by concept category. “Biological (Other)” denotes groupings from IMGT physicochemical classes and secondary-structure propensities. E.5 Interaction between groups of components We measure interactions between component groups in the ESM-C approximate r… view at source ↗
Figure 16
Figure 16. Figure 16: Aggregated interactions between component groups in ESM-C. Positive interactions between source and destination component groups, reflecting connections that support task performance. E.6 Direct contributions to prediction We measure the direct contributions of each ESM-C component group to the prediction of the correct answer (as described in Section 5). The results ( [PITH_FULL_IMAGE:figures/full_fig_p… view at source ↗
Figure 17
Figure 17. Figure 17: Logit-lens analysis of ESM-C component outputs. Induction heads promote the correct token in the middle layers, while final-layer MLPs continue to increase the prediction confidence. Following our analysis in Section 5, we plot the average importance scores for all circuit neurons with AUROC scores above 0.75, aggregated by neuron group. The results ( [PITH_FULL_IMAGE:figures/full_fig_p019_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Attribution-based analysis of neuron-group importance across layers in ESM-C. Biochemical similarity and amino-acid neurons are most important in mid layers, with amino-acid neurons contributing more strongly in the final layer. Repeat-related neurons exhibit inhibitory effects, indicated by negative contributions across multiple layers. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Effect of counterfactual corruption strategies on ESM-3 prediction performance across repeat tasks. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Component-level circuit evaluation for different counterfactual strategies on ESM-3. Faithfulness is shown as a function of the fraction of included components. Results are averaged across random seeds. Cross-counterfactual recall. We measure circuit overlap by computing cross-counterfactual recall ( [PITH_FULL_IMAGE:figures/full_fig_p022_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Cross-counterfactual recall of discovered circuits. Recall measures the fraction of components in the target circuit that are recovered by the source circuit. Circuit sizes (number of components) achieving 85% faithfulness are shown in parentheses, averaged across seeds. Cross-counterfactual faithfulness. To assess functional equivalence across counterfactuals, we evaluate circuits discov￾ered under one c… view at source ↗
Figure 22
Figure 22. Figure 22: Cross-counterfactual faithfulness of discovered circuits. Rows correspond to counterfactuals used for circuit discovery, columns correspond to counterfactuals used in evaluation. Faithfulness is normalized per column, relative to the circuit discovered under that counterfactual. Circuit sizes (number of components) achieving 85% faithfulness are shown in parentheses. Summary. Counterfactual choice affects… view at source ↗
Figure 23
Figure 23. Figure 23: Cluster selection metrics for attention-head clustering. We report inertia (Elbow method, left) and Silhouette scores (right) for k ∈ [2, 10] for ESM-3 (top) and ESM-C (bottom). 33 [PITH_FULL_IMAGE:figures/full_fig_p033_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Example sequence visualizations for a neuron in ESM-3 (layer 38, neuron 2850) selective for the aligned repeat token to the masked position. The neuron activates on the token aligned with the mask (denoted by “?”), corresponding to the correct retrieval target. Repeat tokens are highlighted in green. UniProt accessions: A0A0P7XBW1 (top), B0DTM3 (bottom). Repeat Tokens. We visualize a neuron in ESM-C locat… view at source ↗
Figure 25
Figure 25. Figure 25: Example sequence visualization for a repeat-selective neuron in ESM-C (layer 24, neuron 2224). The neuron exhibits positive activations (red) at positions corresponding to approximate repeat tokens, highlighted in green. UniProt accession: A0A8X6HTE9. Helix breakers. We visualize a neuron in ESM-3 located at layer 2, neuron index 3282, which achieves an AUROC of 0.997 on the Helix Breakers concept. This n… view at source ↗
Figure 26
Figure 26. Figure 26: Example sequence visualization for a helix-breaker–selective neuron in ESM-3 (layer 2, neuron 3282). The neuron exhibits strong positive activations (red) on proline (P) and glycine (G), amino acids known to disrupt α-helical secondary structure. UniProt accession: B0DTM3. Aromatic Ring. We visualize a neuron in ESM-3 at layer 35, neuron index 3011, which achieves an AUROC of 0.963 on the Aromatic Ring co… view at source ↗
Figure 27
Figure 27. Figure 27: Example sequence visualization for an aromatic-ring–selective neuron in ESM-3 (layer 35, neuron 3011). The neuron exhibits strong positive activations (red) on aromatic amino acids (F, Y, H, W). UniProt accession: A0A0P7XBW1. Hydrogen Donor. We visualize a neuron in ESM-C located at layer 7, neuron index 907, which achieves an AUROC of 0.995 on the Hydrogen Donor (IMGT) concept. This neuron exhibits posit… view at source ↗
Figure 28
Figure 28. Figure 28: Example sequence visualization for a hydrogen-donor–selective neuron in ESM-C (layer 7, neuron 907). The neuron exhibits strong positive activations (red) on amino acids R, K, and W, consistent with hydrogen-donor side-chain chemistry. UniProt accession: A0A2M8A3Y9. Bidirectional Amino-Acid Neurons. We observe many amino-acid neurons in both ESM-3 and ESM-C that exhibit strong activations with opposite si… view at source ↗
Figure 29
Figure 29. Figure 29: Example visualizations of bidirectional amino-acid neurons in ESM-3 (top) and ESM-C (bottom), showing strong but oppositely signed responses to pairs of amino acids. BLOSUM62. We visualize additional neurons in ESM-3 and ESM-C associated with BLOSUM62 substitution cliques, where each clique represents a group of amino acids that are mutually substitutable according to the BLOSUM62 matrix. Specifically, we… view at source ↗
Figure 30
Figure 30. Figure 30: Example sequence visualizations for neurons selective for the BLOSUM62 substitution clique {D, E, N} in ESM-3 (top) and ESM-C (bottom). Special Tokens. We visualize neurons in ESM-3 and ESM-C associated with special tokens. Specifically, we show a neuron in ESM-3 at layer 0, neuron index 125 (AUROC = 1.0), and a neuron in ESM-C at layer 24, neuron index 2127 (AUROC = 0.99), both selective for the BOS toke… view at source ↗
Figure 31
Figure 31. Figure 31: Example sequence visualizations for neurons selective for special tokens in ESM-3 (top) and ESM-C (bottom). Both neurons exhibit strong, selective activations to the BOS token, with weaker activations to the EOS token. In the visualizations, < denotes the BOS token and > denotes the EOS token. 42 [PITH_FULL_IMAGE:figures/full_fig_p042_31.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Design-CP: Context Parallelism for Design of Protein Nanoparticles

    cs.LG 2026-07 conditional novelty 6.0

    Context-parallel inference for RFdiffusion 3 enables end-to-end all-atom design of large symmetric protein nanoparticles on multi-GPU hardware without retraining.

Reference graph

Works this paper leans on

21 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [3]

    2010.06.003

    URL https://doi.org/10.1016/j.tig. 2010.06.003. Reusch, A. and Belinkov, Y . Reverse-engineering the retrieval process in genir models. InProceedings of the 48th International ACM SIGIR Conference on Re- search and Development in Information Retrieval, SIGIR ’25, pp. 668–677, New York, NY , USA, 2025. Associa- tion for Computing Machinery. ISBN 9798400715...

    arXiv 2010

  2. [5]

    ijcnlp-long.153/

    URL https://aclanthology.org/2025. ijcnlp-long.153/. Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y ., Chen, A., Conerly, T., DasSarma, N., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., and ...

    2025

  3. [7]

    ISBN 979-8-89176-190-2

    Association for Computational Linguistics. ISBN 979-8-89176-190-2. doi: 10.18653/v1/2025.naacl-short

    work page doi:10.18653/v1/2025.naacl-short 2025

  4. [13]

    doi: 10.1093/bioinformatics/ btae690

    ISSN 1367-4811. doi: 10.1093/bioinformatics/ btae690. URL https://doi.org/10.1093/ bioinformatics/btae690. Mueller, A., Geiger, A., Wiegreffe, S., Arad, D., Arcuschin, I., Belfki, A., Chan, Y . S., Fiotto-Kaufman, J. F., Haklay, T., Hanna, M., et al. MIB: A mechanistic interpretability benchmark. InForty-second International Conference on Machine Learning...

    Pith/arXiv arXiv 2025

  5. [18]

    URL https://openreview.net/forum? id=YWtLZvLmud7. Wang, K. R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: a circuit for indi- rect object identification in GPT-2 small. InThe Eleventh International Conference on Learning Representations,

  6. [41]

    naacl-short.41/

    URL https://aclanthology.org/2025. naacl-short.41/. Hirsh, L., Piovesan, D., Paladin, L., and Tosatto, S. C. E. Identification of repetitive units in protein structures with ReUPred.Amino Acids, 48(6):1391– 1400, Jun 2016. ISSN 1438-2199. doi: 10.1007/ s00726-016-2187-2. URL https://doi.org/10. 1007/s00726-016-2187-2. Hrabe, T. and Godzik, A. ConSole: usi...

    2025

  7. [446]

    emnlp-main.446/

    URL https://aclanthology.org/2021. emnlp-main.446/. Gujral, O., Bafna, M., Alm, E., and Berger, B. Sparse autoencoders uncover biologically interpretable features in protein language model representations. Proceedings of the National Academy of Sciences, 122(34):e2506316122, 2025. doi: 10.1073/pnas. 2506316122. URL https://www.pnas.org/doi/ abs/10.1073/pn...

    doi:10.1073/pnas 2021

  8. [660]

    acl-long.660/

    URL https://aclanthology.org/2023. acl-long.660/. Barbero, F., Arroyo, A., Gu, X., Perivolaropoulos, C., Veliˇckovi´c, P., Pascanu, R., and Bronstein, M. M. Why do LLMs attend to the first token? InSecond Con- ference on Language Modeling, 2025. URL https: //openreview.net/forum?id=tu4dFUsW5z. Biegert, A. and S ¨oding, J. De novo identifica- tion of highl...

    arXiv 2023

  9. [1987]

    for k∈[2,10] , separately for ESM-3 and ESM-C. As shown in Figure 23, the inertia curves for both models exhibit an elbow at k= 3 , indicating that increasing the number of clusters beyond three yields only minor improvements in within-cluster compactness. The Silhouette analysis yields different behaviors across models. For ESM-3 (Figure 23, top row), th...

    1977

  10. [2005]

    Newman, A

    ISBN 9780716743392. Newman, A. M. and Cooper, J. B. XSTREAM: A prac- tical algorithm for identification and architecture mod- eling of tandem repeats in protein sequences.BMC Bioinformatics, 8(1):382, Oct 2007. ISSN 1471-2105. doi: 10.1186/1471-2105-8-382. URL https://doi. org/10.1186/1471-2105-8-382. Nikankin, Y ., Arad, D., Gandelsman, Y ., and Belinkov...

    work page doi:10.1186/1471-2105-8-382 2007

  11. [2010]

    doi: 10.1016/j.tig.2010.06

    ISSN 0168-9525. doi: 10.1016/j.tig.2010.06

    work page doi:10.1016/j.tig.2010.06 2010

  12. [2020]

    com/posts/AcKRB8wDpdaN6v6ru/ interpreting-gpt-the-logit-lens

    URL https://www.lesswrong. com/posts/AcKRB8wDpdaN6v6ru/ interpreting-gpt-the-logit-lens. Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y ., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brow...

    Pith/arXiv arXiv 2022

  13. [2021]

    pub/2021/framework/index.html

    URL https://transformer-circuits. pub/2021/framework/index.html. ESM Team. ESM Cambrian: Revealing the mys- teries of proteins with unsupervised learning, 2024. URL https://evolutionaryscale.ai/blog/ esm-cambrian. Feucht, S., Todd, E., Wallace, B., and Bau, D. The dual- route model of induction. InSecond Conference on Lan- guage Modeling, 2025. URL https:...

    Pith/arXiv arXiv 2021

  14. [2022]

    org/CorpusID:254096064

    URL https://api.semanticscholar. org/CorpusID:254096064. Ismail, A. A., Oikarinen, T., Wang, A., Adebayo, J., Stan- ton, S. D., Bravo, H. C., Cho, K., and Frey, N. C. Con- cept bottleneck language models for protein design. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview. net/forum?id=Yt9CFhOOFe. Jorda,...

    Pith/arXiv arXiv 2025

  15. [2023]

    Wootton, J

    URL https://openreview.net/forum? id=NpsVSN6o4ul. Wootton, J. C. Non-globular domains in protein sequences: Automated segmentation using com- plexity measures.Computers & Chemistry, 18 (3):269–285, 1994. ISSN 0097-8485. doi: https://doi.org/10.1016/0097-8485(94)85023-2. URL https://www.sciencedirect.com/ science/article/pii/0097848594850232. Zhang, J., ZH...

    arXiv 1994

  16. [2024]

    Liu, X., Lei, H., Liu, Y ., Liu, Y ., and Hu, W

    URL https://openreview.net/forum? id=R5unwb9KPc. Liu, X., Lei, H., Liu, Y ., Liu, Y ., and Hu, W. ProtSAE: Disentangling and interpreting protein language models via semantically-guided sparse autoencoders, 2026. URL https://arxiv.org/abs/2509.05309. Lloyd, S. Least squares quantization in PCM.IEEE Trans- actions on Information Theory, 28(2):129–137, 1982...

    arXiv 2026

  17. [2025]

    Andrade, M

    URL https://openreview.net/forum? id=zdOGBRQEbz. Andrade, M. A., Perez-Iratxeta, C., and Ponting, C. P. Protein repeats: Structures, functions, and evolution. Journal of Structural Biology, 134(2):117–131, 2001. ISSN 1047-8477. doi: https://doi.org/10.1006/jsbi.2001

    work page doi:10.1006/jsbi.2001 2001

  18. [2026]

    Biological (Other)

    URL https://openreview.net/forum? id=X0QxVexIJX. 13 Induction Meets Biology: Mechanisms of Repeat Detection in Protein Language Models A Protein repeats: Evolutionary background Sequence repeats play a significant role in protein evolution. As a result of unequal recombination, a DNA-copying error that occurs when two similar chromosomes align incorrectly...

    2001

  19. [2105]

    URL https: //doi.org/10.1186/1471-2105-15-119

    doi: 10.1186/1471-2105-15-119. URL https: //doi.org/10.1186/1471-2105-15-119. Hu, B., Xia, J.-X., Zheng, J., Tan, C., Huang, Y ., Xu, Y ., and Li, S. Z. Protein language models and structure prediction: Connection and progression.ArXiv, abs/2211.16742,

    Pith/arXiv arXiv

  20. [3136]

    Mozaffari, S., Arr ´ıas, P

    URL https://www.sciencedirect.com/ science/article/pii/S0022283699931364. Mozaffari, S., Arr ´ıas, P. N., Clementel, D., Piovesan, D., Ferrari, C., Tosatto, S. C. E., and Monzon, A. M. STRPsearch: fast detection of structured tandem re- peat proteins.Bioinformatics, 40(12):btae690, 11

  21. [4392]

    Arthur, D

    URL https://www.sciencedirect.com/ science/article/pii/S1047847701943928. Arthur, D. and Vassilvitskii, S. k-means++: the advantages of careful seeding. InProceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’07, pp. 1027–1035, USA, 2007. Society for Indus- trial and Applied Mathematics. ISBN 9780898716245. Banerjee, N., Ch...

    arXiv 2007