Pith. sign in

REVIEW 3 major objections 5 minor 47 references

Steering Protein Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Activation steering—adding a mean-difference vector to a protein language model's internal activations during inference—enables training-free control of sequence generation and, with mutation-site selection, outperforms search-based…

desk verdict Activation steering for PLMs is a neat trick, but the evaluation is circular: the same predictors define the steering target and measure success. read the letter →

arxiv 2509.07983 v2 pith:VFNZV6Q6 submitted 2025-07-01 q-bio.BM cs.LG

Long-Kai Huang , Rongyi Zhu , Bing He , Jianhua Yao This is my paper
classification q-bio.BMcs.LG
keywords activationsteeringproteinlanguagemodelsoptimizationthermostabilitysolubilityGFPbrightnessmutationsiteidentificationtraining-freecontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that protein language models already encode information about properties such as thermostability, solubility, and GFP brightness in their internal representations, and that this knowledge can be tapped at inference time by adding a steering vector to the activations. The method, adapted from activation addition in text LLMs, shifts generation toward sequences with the desired property without any weight updates or fine-tuning. The paper extends this to protein optimization through ASPO, which first identifies the amino-acid positions most opposed to the target property and then regenerates them under steering. If the results hold, a lightweight, training-free control mechanism could replace or complement fine-tuning and search-based methods in protein engineering.

What carries the argument

The central object is the steering vector $v_l$, computed at layer $l$ as the mean activation of the positive set minus the mean activation of the negative set, averaged over tokens for auto-encoding PLMs and using the last token for autoregressive PLMs. During inference the activation is edited as $\tilde h_l = h_l + \alpha v_l$ and renormalized to the original norm, with $\alpha$ controlling steering strength. ASPO adds a relatedness score $s_k = \cos(h_k^l, v_l)$ at the most informative layer, selected by linear classifier validation accuracy, and masks the $T$ tokens with lowest scores for regeneration under steering, repeated for $R$ rounds.

What would settle it

Measure the melting temperature, solubility, or fluorescence of ASPO-generated top-scoring sequences in a wet-lab assay; if the predicted gains over the initial set do not appear under direct experimental measurement, the central claim that steering pushes generation toward genuinely improved proteins collapses.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that activation steering works for protein language models across both auto-encoding architectures (ESM2, ESM3) and an autoregressive architecture (ProLLaMA). Steering vectors computed as the mean activation difference between positive and negative protein sets, added at every layer except the input and followed by norm rescaling, reliably raise predicted thermostability, solubility, and GFP brightness of generated lysozyme-like sequences while maintaining or improving diversity and novelty. For optimization, ASPO selects mutation sites by the cosine relatedness of token representations to the steering vector and then masks and regenerates those positions under steering, achieving the highest fitness in medium- and hard-difficulty optimization tasks for thermostability, solubility, and GFP brightness when compared with AdaLead, PEX, and GWG, while staying close to the initial sequence set.

Load-bearing premise

The load-bearing premise is that the surrogate predictors used to define the positive and negative sets and to score the generated sequences—thermostability Spearman 0.76, solubility accuracy 0.708, and the GFP brightness predictor—are accurate enough that steering toward their labels and measuring success with the same predictors reflects real biological improvement.

Editorial extensions

If this is right

  • Activation steering can be integrated into both auto-encoding and autoregressive protein language models without retraining, preserving the model's general knowledge while biasing outputs toward a target property.
  • Steering outperforms fine-tuning and the unmodified model on lysozyme-like thermostability and solubility generation across ProLLaMA, ESM2, and ESM3, and improves novelty and diversity in several settings.
  • ASPO achieves the highest fitness on thermostability, solubility, and GFP brightness optimization at both medium and hard difficulty, with lower dissimilarity to the initial set than the search-based baselines.
  • The method is sample-efficient, with 100 contrast sequences giving near-peak performance, and it is sensitive to steering strength, with over-steering at $\alpha = 20$ collapsing solubility performance.
  • On ESM2-3B, activation steering yields larger gains than on ESM2-650M, indicating that larger protein language models leverage activation editing even more effectively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same recipe should transfer to other protein families and other measurable properties, such as binding affinity or immunogenicity, whenever a contrast set can be constructed from a predictor or an assay.
  • We infer that the strong dependence on surrogate predictors sets the practical ceiling of the method: if the predictors are smooth or biased in ways the steering vector can exploit, wet-lab validation is the only way to know whether the gains are real biological improvement.
  • The multi-property experiment in the appendix suggests steering vectors for different properties can be combined linearly, pointing toward compositional, multi-objective control of protein generation.
  • Because the method needs no training and only a few hundred labeled examples, it may make targeted protein design accessible to groups that lack the compute or data for fine-tuning large models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper adapts activation steering (ActAdd) to protein language models (PLMs). Steering vectors are computed as the mean activation difference between positive and negative protein sets defined by surrogate predictors of thermostability and solubility. The vectors are added to internal activations during generation for autoregressive (ProLLaMA) and auto-encoding (ESM2, ESM3) PLMs, and the approach is extended to a protein optimization framework (ASPO) that selects mutation sites by projecting token representations onto the steering vector. Experiments report improvements in predicted fitness, diversity, novelty, and optimization metrics over fine-tuning and search-based baselines, including a GFP brightness task. The paper claims training-free, precise control over protein generation and optimization.

Significance. If the reported fitness gains reflect genuine improvements in real protein properties, the method would be a valuable lightweight, architecture-agnostic alternative to fine-tuning and iterative optimization. The paper's strengths include a simple formulation, evaluation across multiple PLM architectures, sensitivity analyses for key hyperparameters, and a multi-property steering extension. However, the central empirical claim rests entirely on surrogate predictors that define both the steering target and the evaluation metric, so the current evidence does not distinguish real biological improvement from predictor exploitation. That concern, plus the modified baseline setup, substantially limits the significance of the results as presented.

major comments (3)
  1. [4.1.1 and A.2] The positive/negative sets for steering vector extraction are constructed using predicted thermostability/solubility values (Section 4.1.1, Data), and the reported fitness metrics in Tables 1-4 are computed with the same predictors (Section A.1, Fitness). This creates a circular evaluation: the steering vector is the mean activation difference between sequences labeled high and low by the predictor, and the success metric is that predictor's output on steered sequences. If the predictor is approximately linear in the representation layer, adding the steering vector moves activations in the direction that increases the predicted score by construction. The observed gains therefore do not establish that the generated or optimized sequences have increased real thermostability, solubility, or GFP brightness. The authors should evaluate on independent experimental data (e.g., the experimental GFP brightness values underlying the [13] benchmark, or held-out Meltome proteins) or at least use a different predictive model for evaluation. Without such validation, the central claim of precise protein engineering is unsupported.
  2. [4.2.1] The comparison with AdaLead, PEX, and GGS is not a fair comparison of the methods as originally designed. Section 4.2.1 states that because the original AdaLead and PEX update their surrogate fitness predictor using ground-truth fitness from wet-lab experiments each round, the authors assume no access to ground-truth fitness and do not update the surrogate for these baselines. This removes the intended mechanism of those algorithms, whereas ASPO does not require ground-truth updates. The result is a comparison against degraded variants of the baselines. To support the claim that ASPO 'significantly outperforms all baselines' (Section 4.2.2), the baselines should either be run with their native feedback mechanism or the paper should explicitly frame the comparison as being against static-surrogate variants only. As presented, the optimization results do not demonstrate superiority in a realistic setting.
  3. [3.1] The premise verification in Section 3.1 uses t-SNE visualizations of PLM activations for proteins partitioned into positive and negative sets defined by the same surrogate predictors used later for steering. The observed cluster separation is therefore partly a consequence of the predictors' signal already being present in the PLM representations, and it does not independently establish that PLMs 'inherently encapsulate intrinsic knowledge' about the properties. The authors should demonstrate cluster separation using experimentally labeled data (e.g., Meltome or the GFP dataset) rather than the surrogate labels. This would provide a stronger and non-circular justification for the approach.
minor comments (5)
  1. [4.3] The text references 'Fig. 4.2.2' in the sensitivity to steering strength discussion; this appears to be an incorrect figure reference, likely intended to be Figure 4(a) and Figure 4(d).
  2. [Tables 2 and 3] The column headers 'Dissiminit' and 'Dissimhigh' should be typeset as 'Dissim_init' and 'Dissim_high' for readability, and for consistency with the metric definitions in Section A.1.
  3. [4.1.1] For AE-PLMs, the generation procedure is described as randomly masking and regenerating 10% of tokens from a reference sequence (Section 4.1.1, Base Models). This is more accurately a mutagenesis or denoising procedure than de novo sequence generation; the terminology should be clarified to avoid overstating the generation capability.
  4. [5] The conclusion states that the method does not require 'explicit fitness predictors' because it does not use them during inference. However, the steering vectors are constructed using predictions from property-specific predictors. This should be acknowledged explicitly to avoid misleading readers about the reliance on proxy labels.
  5. [General] The paper does not state whether code will be released. For reproducibility, please provide code or detailed hyperparameters for the mutation site selection, especially the choice of layer ℓ used for relatedness scores.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported fitness improvements are evaluated with the same surrogate predictors used to label the positive/negative sets from which the steering vector is derived, so the central 'precise control' claim reduces to optimizing the predictor by construction.

  1. fitted input called prediction [Section 4.1.1 (Data), Section A.1 (Fitness), Section A.2 (Fitness Predictor), Tables 1-4]
    "Data: To construct the positive and negative sets for steering vector extraction, we first predict thermostability or solubility for all lysozyme-like proteins in the UniRef50 dataset using property-specific predictors. Fitness quantifies how well a protein exhibits the desired properties. We estimate fitness using predictors described in Section A.2 for thermostability and solubility. For GFP, we use the predictor from [13] to estimate the log fluorescence intensity."

    The positive/negative sets are thresholded from surrogate predictions; the steering vector (Eq. 2) is the mean activation difference between those sets, i.e., the direction from 'predictor says low' to 'predictor says high'. The Fitness metric is the output of the same predictors. Thus any sequence moving along the steering vector is scored as higher-fitness by construction; Tables 1-4 measure alignment with the surrogate, not independent thermostability/solubility/GFP brightness. With surrogate accuracies of only 0.76 Spearman and 0.708 accuracy, the reported 'precise control' can be achieved by exploiting predictor biases.

full rationale

The central empirical claim is that activation steering and ASPO produce proteins with enhanced thermostability, solubility, or GFP brightness. However, the paper defines the target property by thresholding predictions from surrogate models (Section 4.1.1) and evaluates fitness with those same surrogate models (Sections A.1-A.2). The steering vector is the mean activation difference between the high- and low-scoring sets, so it is, by construction, a direction that increases the surrogate score. Consequently the reported fitness improvements are a mathematical consequence of the experimental setup rather than independent evidence of real biological improvement. The GFP experiment is the clearest case: both the task split and the evaluation predictor come from [13], and the surrogate is used as both target and metric. The comparison against AdaLead/PEX/GGS does not break the circularity because those baselines are run with a static surrogate and no ground-truth fitness updates (Section 4.2.1). Diversity and novelty metrics are independent and do show that steering does not simply copy the positive set, but the headline claim of 'precise, training-free control' over target properties rests on the same surrogate. This is a partial, not total, circularity: the method genuinely controls generation in the predictor's direction, but the paper's biological claims are not independently validated.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central method rests on the premise that activation-space differences correlate with real protein properties and that the same predictor can serve as both the source of steering labels and the evaluation oracle. Free parameters include steering strength, set sizes, mutation rounds, and per-task thresholds. No new physical entities are introduced.

free parameters (6)
  • Steering strength alpha = 1.0 (default)
    Controls the magnitude of activation edit; sensitivity explored in Fig. 4, but the default is a hand-set choice.
  • Positive/negative set size = 100 sequences each
    Recommended default; performance stabilizes beyond 100 in Fig. 3.
  • Mutation sites per round T = 4 (thermostability), 2 (solubility/GFP)
    Hand-set per task; no sensitivity analysis reported.
  • Optimization rounds R = 8 (thermostability), 4 (solubility/GFP)
    Hand-set per task; no sensitivity analysis reported.
  • Layer l for relatedness scores = Layer with highest validation accuracy
    Selected by a linear classifier's validation accuracy; not a fixed pre-specified value.
  • Positive/negative thresholds = Thermostability >70C/<50C; solubility >0.8/<0.15
    Hand-chosen cutoffs for defining property sets.
assumptions (5)
  • domain assumption PLMs encode property information in their activation spaces
    Supported only by t-SNE visualizations (Fig. 1), which are qualitative and not a quantitative test.
  • ad hoc to paper The mean activation difference between positive and negative sets is a valid steering direction
    Adopted from ActAdd in LLMs; no theoretical justification for protein sequence spaces.
  • ad hoc to paper Cosine similarity of a token's representation to the steering vector identifies mutation sites that oppose the target property
    This is the core of ASPO; it is a heuristic with no empirical or theoretical support beyond the paper's own results.
  • domain assumption The surrogate fitness predictors are accurate proxies for real thermostability, solubility, and GFP brightness
    Thermostability predictor has Spearman 0.76, solubility accuracy 0.708; GFP predictor from [13]; no wet-lab validation in this paper.
  • domain assumption Renormalizing activations after steering preserves generation quality
    Standard practice in ActAdd, but not independently verified for PLMs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Steering Protein Language Models." pith.science (2026). https://pith.science/paper/VFNZV6Q6

@misc{pith2026250907983,
  author       = {Pith},
  title        = {Pith review of: Steering Protein Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VFNZV6Q6}},
  note         = {Machine review of arXiv:2509.07983}
}
read the original abstract

Protein Language Models (PLMs), pre-trained on extensive evolutionary data from natural proteins, have emerged as indispensable tools for protein design. While powerful, PLMs often struggle to produce proteins with precisely specified functionalities or properties due to inherent challenges in controlling their outputs. In this work, we investigate the potential of Activation Steering, a technique originally developed for controlling text generation in Large Language Models (LLMs), to direct PLMs toward generating protein sequences with targeted properties. We propose a simple yet effective method that employs activation editing to steer PLM outputs, and extend this approach to protein optimization through a novel editing site identification module. Through comprehensive experiments on lysozyme-like sequence generation and optimization, we demonstrate that our methods can be seamlessly integrated into both auto-encoding and autoregressive PLMs without requiring additional training. These results highlight a promising direction for precise protein engineering using foundation models.

Figures

Figures reproduced from arXiv: 2509.07983 by the authors.

Figure 1
Figure 1. t-SNE visualization of PLM activations from ESM2, ESM3, and ProLLaMA for proteins [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Overview of Activation Steering for PLMs and Activation Steering based Protein Optimiza [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Sensitivity to the number of samples used for steering vector extraction. Columns show [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sensitivity of steering strength α. Columns show results for protein generation, medium￾difficulty protein optimization, and hard-difficulty protein optimization. The first row is for ther￾mostability, and the second row is for solubility.. 4.2.2 Results and Analysis T…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 22 canonical work pages

  1. [13]

    Improving protein optimization with smoothed fitness landscapes

    Andrew Kirjner, Jason Yim, Raman Samusevich, Shahar Bracha, Tommi S Jaakkola, Regina Barzilay, and Ila R Fiete. Improving protein optimization with smoothed fitness landscapes. In The Twelfth International Conference on Learning Representations, 2023

    work page 2023

  2. [1]

    Discovering Bias in Latent Space: An Unsupervised Debiasing Approach

    Dyah Adila, Shuai Zhang, Boran Han, and Yuyang Wang. Discovering bias in latent space: An unsupervised debiasing approach.arXiv preprint arXiv:2406.03631, 2024

    work page Pith review arXiv 2024

  3. [2]

    Model-based reinforcement learning for biological sequence design

    Christof Angermueller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Colwell. Model-based reinforcement learning for biological sequence design. In International conference on learning representations, 2019

    2019

  4. [3]

    Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization.arXiv preprint arXiv:2406.00045, 2024

    Yuanpu Cao, Tianrong Zhang, Bochuan Cao, Ziyi Yin, Lu Lin, Fenglong Ma, and Jinghui Chen. Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization.arXiv preprint arXiv:2406.00045, 2024

    arXiv 2024

  5. [4]

    Bioinformatics approaches for improved recombinant protein production in escherichia coli: protein solubility prediction.Briefings in bioinformatics, 15(6):953–962, 2014

    Catherine Ching Han Chang, Jiangning Song, Beng Ti Tey, and Ramakrishnan Nagasundara Ra- manan. Bioinformatics approaches for improved recombinant protein production in escherichia coli: protein solubility prediction.Briefings in bioinformatics, 15(6):953–962, 2014

    work page 2014

  6. [5]

    Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset.PLOS Computational Biology, 20(7):e1012248, 2024

    Simon KS Chu, Kush Narang, and Justin B Siegel. Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset.PLOS Computational Biology, 20(7):e1012248, 2024

    work page 2024

  7. [6]

    Controllable protein design with language models.Nature Machine Intelligence, 4(6):521–532, 2022

    Noelia Ferruz and Birte Höcker. Controllable protein design with language models.Nature Machine Intelligence, 4(6):521–532, 2022

    work page 2022

  8. [7]

    Protein language model fitness is a matter of preference.bioRxiv, pages 2024–10, 2024

    Cade Gordon, Amy X Lu, and Pieter Abbeel. Protein language model fitness is a matter of preference.bioRxiv, pages 2024–10, 2024

    work page 2024

Show all 47 references
  1. [8]

    Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q

    Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J. Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q. Tran, Jonathan Deaton, Marius Wiggert, Rohil Badkundri, Irhum Shafkat, Jun Gong, Alexander Derry, Raul S. Molina, Neil Thomas, Yousuf A. Khan, Chetan Mishra, Caroly...

    2024

  2. [9]

    Efficient evolution of human antibodies from general protein language models.Nature Biotechnology, 42(2):275–283, 2024

    Brian L Hie, Varun R Shanker, Duo Xu, Theodora UJ Bruun, Payton A Weidenbacher, Shaogeng Tang, Wesley Wu, John E Pak, and Peter S Kim. Efficient evolution of human antibodies from general protein language models.Nature Biotechnology, 42(2):275–283, 2024

    2024

  3. [10]

    Evolutionary velocity with protein language models predicts evolutionary dynamics of diverse proteins.Cell Systems, 13(4):274–285, 2022

    Brian L Hie, Kevin K Yang, and Peter S Kim. Evolutionary velocity with protein language models predicts evolutionary dynamics of diverse proteins.Cell Systems, 13(4):274–285, 2022

    2022

  4. [11]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022

    2022

  5. [12]

    Meltome at- las—thermal proteome stability across the tree of life.Nature methods, 17(5):495–503, 2020

    Anna Jarzab, Nils Kurzawa, Thomas Hopf, Matthias Moerch, Jana Zecha, Niels Leijten, Yangyang Bian, Eva Musiol, Melanie Maschberger, Gabriele Stoehr, et al. Meltome at- las—thermal proteome stability across the tree of life.Nature methods, 17(5):495–503, 2020

    2020

  6. [14]

    Programming refusal with conditional activation steering.arXiv preprint arXiv:2409.05907, 2024

    Bruce W Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. Programming refusal with conditional activation steering.arXiv preprint arXiv:2409.05907, 2024

    2024 arXiv

  7. [15]

    Robust optimization in protein fitness landscapes using reinforcement learning in latent space.arXiv preprint arXiv:2405.18986, 2024

    Minji Lee, Luiz Felipe Vecchietti, Hyunkyu Jung, Hyun Joo Ro, Meeyoung Cha, and Ho Min Kim. Robust optimization in protein fitness landscapes using reinforcement learning in latent space.arXiv preprint arXiv:2405.18986, 2024

    2024 arXiv

  8. [16]

    Inference- time intervention: Eliciting truthful answers from a language model.Advances in Neural Information Processing Systems, 36, 2024

    Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model.Advances in Neural Information Processing Systems, 36, 2024. 11

    2024

  9. [17]

    Language models of protein sequences at the scale of evolution enable accurate structure prediction

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv, 2022

    2022

  10. [18]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    2023

  11. [19]

    In-context vectors: Making in context learning more effective and controllable through latent space steering.arXiv preprint arXiv:2311.06668, 2023

    Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering.arXiv preprint arXiv:2311.06668, 2023

    2023 arXiv

  12. [20]

    Prollama: A protein large language model for multi-task protein language processing.arXiv preprint arXiv:2402.16445, 2024

    Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Li Yuan, and Yonghong Tian. Prollama: A protein large language model for multi-task protein language processing.arXiv preprint arXiv:2402.16445, 2024

    2024 arXiv

  13. [21]

    Large language models generate functional protein sequences across diverse families.Nature Biotechnology, 41(8):1099–1106, 2023

    Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse families.Nature Biotechnology, 41(8):1099–...

    2023

  14. [22]

    Progen: Language modeling for protein generation

    Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R Eguchi, Po-Ssu Huang, and Richard Socher. Progen: Language modeling for protein generation. arXiv preprint arXiv:2004.03497, 2020

    2004 arXiv

  15. [23]

    Protein fitness prediction is impacted by the interplay of language models, ensemble learning, and sampling methods.Pharmaceutics, 15(5):1337, 2023

    Mehrsa Mardikoraem and Daniel Woldring. Protein fitness prediction is impacted by the interplay of language models, ensemble learning, and sampling methods.Pharmaceutics, 15(5):1337, 2023

    2023

  16. [24]

    Language models enable zero-shot prediction of the effects of mutations on protein function.Advances in neural information processing systems, 34:29287–29303, 2021

    Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu, and Alex Rives. Language models enable zero-shot prediction of the effects of mutations on protein function.Advances in neural information processing systems, 34:29287–29303, 2021

    2021

  17. [25]

    Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    2023

  18. [26]

    Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval

    Pascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena-Hurtado, Aidan N Gomez, Debora Marks, and Yarin Gal. Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. InInternational Conference on Machine Learning, pages 16990–...

    2022

  19. [27]

    Proteingym: Large- scale benchmarks for protein fitness prediction and design.Advances in Neural Information Processing Systems, 36, 2024

    Pascal Notin, Aaron Kollasch, Daniel Ritter, Lood Van Niekerk, Steffanie Paul, Han Spinner, Nathan Rollins, Ada Shaw, Rose Orenbuch, Ruben Weitzman, et al. Proteingym: Large- scale benchmarks for protein fitness prediction and design.Advances in Neural Information Processing S...

    2024

  20. [28]

    Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681, 2023

    Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681, 2023

    2023 arXiv

  21. [29]

    Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning.arXiv preprint arXiv:2402.17810, 2024

    Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning.arXiv preprint arXiv:2402.17810, 2024

    2024 arXiv

  22. [30]

    Spec- tral editing of activations for large language model alignment.arXiv preprint arXiv:2405.09719, 2024

    Yifu Qiu, Zheng Zhao, Yftah Ziser, Anna Korhonen, Edoardo M Ponti, and Shay B Cohen. Spec- tral editing of activations for large language model alignment.arXiv preprint arXiv:2405.09719, 2024

    2024 arXiv

  23. [31]

    Proximal exploration for model-guided protein sequence design

    Zhizhou Ren, Jiahan Li, Fan Ding, Yuan Zhou, Jianzhu Ma, and Jian Peng. Proximal exploration for model-guided protein sequence design. InInternational Conference on Machine Learning, pages 18520–18536. PMLR, 2022. 12

    2022

  24. [32]

    Exploring protein fitness landscapes by directed evolution.Nature reviews Molecular cell biology, 10(12):866–876, 2009

    Philip A Romero and Frances H Arnold. Exploring protein fitness landscapes by directed evolution.Nature reviews Molecular cell biology, 10(12):866–876, 2009

    2009

  25. [33]

    Adalead: A simple and robust adaptive greedy search algorithm for sequence design

    Sam Sinai, Richard Wang, Alexander Whatley, Stewart Slocum, Elina Locane, and Eric D Kelsic. Adalead: A simple and robust adaptive greedy search algorithm for sequence design. arXiv preprint arXiv:2010.02141, 2020

    2010 arXiv

  26. [34]

    Mimic: Minimally modified counterfactuals in the representation space

    Shashwat Singh, Shauli Ravfogel, Jonathan Herzig, Roee Aharoni, Ryan Cotterell, and Ponnu- rangam Kumaraguru. Mimic: Minimally modified counterfactuals in the representation space. arXiv preprint arXiv:2402.09631, 2024

    2024 arXiv

  27. [35]

    Accelerating bayesian optimization for biological se- quence design with denoising autoencoders

    Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating bayesian optimization for biological se- quence design with denoising autoencoders. InInternational Conference on Machine Learning, pages 204...

    2022

  28. [36]

    Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017

    Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017

    2017

  29. [37]

    Deep generative modeling for protein design.Current opinion in structural biology, 72:226–236, 2022

    Alexey Strokach and Philip M Kim. Deep generative modeling for protein design.Current opinion in structural biology, 72:226–236, 2022

    2022

  30. [38]

    Extracting latent steering vectors from pretrained language models.arXiv preprint arXiv:2205.05124, 2022

    Nishant Subramani, Nivedita Suresh, and Matthew E Peters. Extracting latent steering vectors from pretrained language models.arXiv preprint arXiv:2205.05124, 2022

    2022 arXiv

  31. [39]

    Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31(6):926–932, 2015

    Baris E Suzek, Yuqi Wang, Hongzhan Huang, Peter B McGarvey, Cathy H Wu, and UniProt Consortium. Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31(6):926–932, 2015

    2015

  32. [40]

    Activation addition: Steering language models without optimization

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization. arXiv e-prints, pages arXiv–2308, 2023

    2023

  33. [41]

    Backdoor activation attack: Attack large language models using activation steering for safety-alignment.arXiv preprint arXiv:2311.09433, 2023

    Haoran Wang and Kai Shu. Backdoor activation attack: Attack large language models using activation steering for safety-alignment.arXiv preprint arXiv:2311.09433, 2023

    2023 arXiv

  34. [42]

    Informed training set design enables efficient machine learning-assisted directed protein evolution.Cell systems, 12(11):1026–1045, 2021

    Bruce J Wittmann, Yisong Yue, and Frances H Arnold. Informed training set design enables efficient machine learning-assisted directed protein evolution.Cell systems, 12(11):1026–1045, 2021

    2021

  35. [43]

    Machine learning-assisted directed protein evolution with combinatorial libraries.Proceedings of the National Academy of Sciences, 116(18):8852–8858, 2019

    Zachary Wu, SB Jennifer Kan, Russell D Lewis, Bruce J Wittmann, and Frances H Arnold. Machine learning-assisted directed protein evolution with combinatorial libraries.Proceedings of the National Academy of Sciences, 116(18):8852–8858, 2019

    2019

  36. [44]

    Active learning-assisted directed evolution.Nature Communications, 16(1):714, 2025

    Jason Yang, Ravi G Lal, James C Bowden, Raul Astudillo, Mikhail A Hameedi, Sukhvinder Kaur, Matthew Hill, Yisong Yue, and Frances H Arnold. Active learning-assisted directed evolution.Nature Communications, 16(1):714, 2025

    2025

  37. [45]

    Multi-scale representation learning for protein fitness prediction.arXiv preprint arXiv:2412.01108, 2024

    Zuobai Zhang, Pascal Notin, Yining Huang, Aurélie Lozano, Vijil Chenthamarakshan, Debora Marks, Payel Das, and Jian Tang. Multi-scale representation learning for protein fitness prediction.arXiv preprint arXiv:2412.01108, 2024

    2024 arXiv

  38. [46]

    Structure- informed language models are protein designers

    Zaixiang Zheng, Yifan Deng, Dongyu Xue, Yi Zhou, Fei Ye, and Quanquan Gu. Structure- informed language models are protein designers. InInternational conference on machine learning, pages 42317–42338. PMLR, 2023

    2023

  39. [47]

    Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023. 13 A Supplementary Exp...

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.