Pith. sign in

REVIEW 4 major objections 5 minor 80 references

PFMBench: Protein Foundation Model Benchmark

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PFMBench argues that ProTrek beats ESM2 on 75% of representative protein tasks, that zero-shot ProteinGym scores do not predict supervised performance, and that scaling ESM2 only pays off at 15B parameters.

desk verdict A valuable and reusable benchmark, but its main ProTrek ranking needs a leakage-controlled redo before it can be believed. read the letter →

arxiv 2506.14796 v1 pith:UPVQ3RGT submitted 2025-06-01 q-bio.BM cs.AIcs.LG

Zhangyang Gao , Hao Wang , Cheng Tan , Chenrui Xu , Mengdi Liu , Bozhen Hu , Linlin Chao , Xiaoming Zhang
show 1 more author
Stan Z. Li
This is my paper
classification q-bio.BMcs.AIcs.LG
keywords proteinfoundationmodelsbenchmarkparameter-efficientfine-tuningzero-shotevaluationtaskcorrelationmultimodalmodelscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PFMBench argues that protein foundation models can be compared fairly in a single unified suite, and that two widespread evaluation beliefs are wrong. The benchmark spans 38 tasks in eight areas and 17 models, then filters to 12 core models and 28 low-variance core tasks and clusters them into 11 representative tasks. On those tasks, the multimodal model ProTrek beats sequence-only ESM2 on 75% of tasks, while zero-shot ProteinGym rank and supervised rank do not match. The paper also reports that ESM2 scaling is flat until 15B parameters, so pretraining strategy matters more than raw scale.

What carries the argument

The carrying machinery is a two-stage selection plus task-correlation protocol. ESM2-Adapter is run three times on each task, and only the 28 tasks with under 5% run-to-run bias are kept; models that reach at least 85% of ESM2's EC F1 score become the 12 core models. Spearman correlations between tasks are clustered into 11 groups, one representative task per group, and model strength is summarized by a winning rate (#Win), the share of representative tasks where a model exceeds ESM2. The same protocol is crossed with six PEFT methods and with a mutual-information-difference analysis relative to ESM2-35M to connect the rankings to pretraining behavior.

What would settle it

Run the same adapter-based protocol on a held-out set of protein tasks not used for clustering or core-task selection and compare ProTrek's winning rate against ESM2; if it drops to or below 50%, the representativeness claim fails.

Watch

Extended reading notes

Core claim

The paper's discovery is a scaled, structured evaluation landscape that changes what should be reported about protein foundation models. Under a fixed adapter-tuning protocol, the 28 core tasks correlate into 11 clusters, and model rankings across those clusters show that sequence-only encoders rarely beat ESM2, decoder-only models perform worst, and multimodal models with contrastive alignment lead, with ProTrek winning 75% of representative tasks. A second finding is that ProteinGym zero-shot scores do not correlate with supervised results, so zero-shot fitness benchmarks are not a proxy for general protein understanding. A third finding is that the ESM2 scaling curve only improves at the 15B size and at disproportionate cost, whereas ProTrek-650M beats ESM2-15B on most tasks.

Load-bearing premise

The conclusions depend on the assumption that the 12 core models and 28 core tasks chosen by the filtering rules represent the wider protein foundation model landscape, so that rankings and recommendations transfer to other models and tasks.

Editorial extensions

If this is right

  • A new protein foundation model can be evaluated meaningfully on the 11 representative tasks instead of an exhaustive 38-task sweep, cutting the cost of fair comparison.
  • Zero-shot ProteinGym results should be reported separately from supervised results, because they rank models differently and cannot be used interchangeably for model selection.
  • Multimodal, contrastively aligned models emerge as the strongest current direction, while decoder-only generative models are poor defaults for protein understanding tasks.
  • Scaling parameter count in the ESM2 family is not a reliable route to better downstream performance until the 15B scale, so improving pretraining data and objectives is the cheaper lever.
  • Adapter tuning is sufficient as a default PEFT protocol, with DoRA as a competitive alternative, so future benchmarks can standardize on a single efficient tuning method.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct decontamination test that removes benchmark-overlapping sequences from ProTrek's pretraining data would settle whether its annotation-task advantage is semantic alignment or memory; the paper itself flags label leakage as a concern.
  • The divergence between ProteinGym and supervised rankings suggests fitness prediction and general protein understanding are separate capabilities, so protein engineering pipelines may need two separate model selections.
  • The scaling conclusion is specific to ESM2's pretraining recipe; scaling a contrastive model like ProTrek from 650M to 3B or 15B and comparing cost-adjusted gains would test whether 'scaling is not worth it' generalizes.
  • The 11-task subset's representativeness can be validated externally by deriving task clusters on an independent set of protein datasets and checking whether the ProTrek-versus-ESM2 ranking reproduces.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces PFMBench, a benchmark for protein foundation models spanning 38 tasks (annotation, solubility, localization, mutation, interaction, structure, production, and zero-shot ProteinGym) and 17 models, with standardized data splits, a set of PEFT methods, and a hierarchical analysis pipeline. The authors report that core-task and core-model selection lead to 11 representative tasks and 12 core models, that ProTrek achieves a 75% winning rate against ESM2 on representative tasks, that zero-shot ProteinGym performance does not correlate with supervised performance, and that scaling ESM2 only helps at 15B parameters while pretraining strategy matters more. The paper also provides code and a modular framework.

Significance. If the central claims hold, PFMBench would be a valuable community resource: it offers a broad task/model coverage, standardized 30% sequence-similarity splits, a reproducible Hydra/PyTorch Lightning framework, a comparison of several PEFT methods, and a concrete streamlined evaluation protocol. The analysis of task correlations and the MSA mutual-information diagnostic are useful additions beyond a simple leaderboard. However, the headline model-ranking result depends on a possible label-leakage confound that the manuscript itself flags in Appendix A.2, and the selection of core tasks/models is post hoc relative to the main conclusions. These issues need to be addressed before the benchmark's recommendations can be relied upon.

major comments (4)
  1. [§4.1, Table 3; Appendix A.2, Table 7] The central claim that ProTrek outperforms ESM2 with a 75% winning rate is not established because of a likely label-leakage confound. ProTrek is pretrained on Swiss-Prot/GO functional annotations, and the representative tasks include EC, GO BP, GO MF, GO CC, Metal Ion Binding, and DeepLoc2-Multi, whose labels come from the same or overlapping functional-annotation sources. The manuscript's own Table 7 caption concedes that 'potential label leakage from overlapping functional annotation data remains a concern for function-aware models,' yet no overlap analysis or leakage-controlled experiment is presented. I request a stratified analysis separating annotation-overlapping tasks from non-overlapping tasks, reporting ProTrek's win rate in each stratum, and, if feasible, a leakage-controlled evaluation where training labels overlapping ProTrek's pretraining annotations are removed.
  2. [§3.2 and §3.3] The core-task and core-model selection criteria are post hoc and can bias the conclusions. Core tasks are selected by requiring ESM2-Adapter run-to-run bias below 5%, and core models are selected by requiring EC F1 at least 85% of ESM2's score. Because these filters are applied before computing the task-correlation matrix, the representative tasks, and the model rankings, the reported relationships are conditional on the choice of ESM2 as the reference and on the adapter protocol. Please provide a sensitivity analysis that, for example, includes the 10 excluded tasks or relaxes the EC threshold, to show that the task clusters, the 11 representative tasks, and the ProTrek-versus-ESM2 ranking are robust to these choices.
  3. [Tables 3, 5, 6, and 7] Almost all model-task results are single runs with no error bars or significance tests, while the paper's own task-bias filter (Table 1) shows that 10 of 38 tasks have ESM2-Adapter run-to-run bias above 5%, some as high as 114%. Many pairwise model differences in Table 3 are below 0.01 in F1 or Spearman, so the #Win rates and the 'ProTrek consistently outperforms ESM2' conclusions may reflect noise. Please report results over at least three seeds with standard errors for the representative tasks, or provide a paired significance test for the winning-rate comparisons.
  4. [§3.1] The use of AF2DB or ESMFold predicted structures for structure-aware models, rather than experimentally determined structures, is a protocol choice that can interact with model pretraining and task difficulty, but the paper does not analyze this dependence. For example, SaProt is trained with a structure-aware vocabulary derived from predicted structures, which may give it an advantage or disadvantage on tasks evaluated with predicted structures. Please state how many tasks use predicted versus experimental structures and include a sensitivity check on a subset where experimental structures are available.
minor comments (5)
  1. [Throughout] There are several typos and inconsistencies, including 'SaPort' for SaProt in Table 2 and Figure 1, 'ProtoT5' for ProtT5 in §4.2, 'foucus' in §2, 'adpot' in §2, and 'enumerious' in the Figure 3 caption.
  2. [Figure 4 and Figure 7] The task-correlation matrix and the model-ranking figure are difficult to read at the printed resolution; please increase the figure size and font, and consider providing a zoomable version or a table of the underlying correlations.
  3. [Appendix A.3] The mutual-information difference metric depends on the aligned overlapping regions and on the masking procedure, but the text does not specify how gaps and length differences are handled in the alignment or how many MSA clusters are used for the figure; please clarify these implementation details.
  4. [§3.4] The hyperparameter section states that the optimizer is AdamW with batch size 64 and up to 50 epochs, but it does not specify the learning-rate scheduler, warmup, weight decay, or the random seed policy across tasks; please provide these details for reproducibility.
  5. [Table 1] The task table lists mean performance and bias for ESM2-Adapter but does not report the number of evaluation seeds used to compute the bias; please state that explicitly, since the 5% core-task threshold depends on it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PFMBench is an empirical benchmark whose claims are direct measurements on external datasets and model checkpoints, not derivations that reduce to their own inputs.

full rationale

PFMBench makes no formal derivation whose output is defined in terms of its input. The central claims—ProTrek's 75% winning rate on representative tasks, ProteinGym zero-shot performance not correlating with supervised results, and scaling helping only at 15B—are obtained by measuring pretrained external checkpoints (ESM2, ProtT5, ProtGPT2, etc.) on external datasets (TAPE, PEER, Venus, ProteinGym, and standard function/structure benchmarks), not by fitting parameters that are then renamed as predictions. The core-model selection gate (EC F1 at least 85% of ESM2) does constrain EC variation among the 12 selected models, and EC is later one of the 11 representative tasks, but passing the gate is not equivalent to beating ESM2 on EC, and the headline winning rates aggregate eight tasks, so the ranking is not forced by construction. The core-task selection by ESM2-Adapter run-to-run bias below 5% and the exclusion of poorly performing tasks are representativeness choices, not circular steps. The Table 7 caption's own caveat—'potential label leakage from overlapping functional annotation data remains a concern for function-aware models'—flags a data-contamination/correctness risk that would need an overlap analysis to substantiate; it is not an instance of circular reasoning because the benchmark's conclusions are not defined in terms of ProTrek's pretraining labels. The only apparent self-citation, ProteinInvBench [14], appears in Related Work descriptively and is not load-bearing. No imported uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result as organization were found. The benchmark is self-contained against external tasks and external model checkpoints, so the honest finding is no significant circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

All the 'free' numbers here are experimental design choices, not fitted parameters in a model. They are nonetheless load-bearing for the benchmark's conclusions, since changing them could change which tasks and models are considered core.

free parameters (7)
  • sequence similarity cutoff = 30%
    Used for dataset splitting (Sec 3.1); chosen by the authors, not derived.
  • train/val/test ratio = 8:1:1
    Data split ratio (Sec 3.1).
  • core task bias threshold = 5%
    Tasks with ESM2-Adapter run-to-run bias below 5% are designated as core (Sec 3.2).
  • core model EC threshold = 85% of ESM2
    Models with EC F1 >= 85% of ESM2 are selected as core (Sec 3.3).
  • adapter architecture = 6-layer transformer, hidden 480, 20 heads
    Task-specific adapter used for all evaluations (Sec 3.4).
  • learning rate candidates = 1e-5, 1e-4
    Grid searched for each task (Sec 3.4).
  • training epochs = 50 with early stopping patience 5
    Training budget (Sec 3.4).
assumptions (4)
  • domain assumption Predicted structures from AF2DB or ESMFold are adequate for evaluating structure-aware models.
    Used to generate structural inputs for all datasets (Sec 3.1); if predicted structures are poor, structure-aware model scores are pessimistic.
  • domain assumption Mutation datasets can keep their original splits despite high sequence similarity.
    Exempt from the 30% cutoff (Sec 3.1); this deviates from the standardized protocol and may inflate mutation task scores.
  • domain assumption Adapter tuning with a 6-layer transformer is a fair and representative fine-tuning protocol for all models.
    All models are evaluated with the same adapter (Sec 3.4); models with different pretraining objectives may be better served by other heads.
  • ad hoc to paper The 12 core models selected by EC performance are representative of the broader set of protein foundation models.
    Models scoring below 85% of ESM2 on EC are excluded from most analyses (Sec 3.3); this selection can bias rankings.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PFMBench: Protein Foundation Model Benchmark." pith.science (2026). https://pith.science/paper/UPVQ3RGT

@misc{pith2026250614796,
  author       = {Pith},
  title        = {Pith review of: PFMBench: Protein Foundation Model Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UPVQ3RGT}},
  note         = {Machine review of arXiv:2506.14796}
}
read the original abstract

This study investigates the current landscape and future directions of protein foundation model research. While recent advancements have transformed protein science and engineering, the field lacks a comprehensive benchmark for fair evaluation and in-depth understanding. Since ESM-1B, numerous protein foundation models have emerged, each with unique datasets and methodologies. However, evaluations often focus on limited tasks tailored to specific models, hindering insights into broader generalization and limitations. Specifically, researchers struggle to understand the relationships between tasks, assess how well current models perform across them, and determine the criteria in developing new foundation models. To fill this gap, we present PFMBench, a comprehensive benchmark evaluating protein foundation models across 38 tasks spanning 8 key areas of protein science. Through hundreds of experiments on 17 state-of-the-art models across 38 tasks, PFMBench reveals the inherent correlations between tasks, identifies top-performing models, and provides a streamlined evaluation protocol. Code is available at \href{https://github.com/biomap-research/PFMBench}{\textcolor{blue}{GitHub}}.

Figures

Figures reproduced from arXiv: 2506.14796 by the authors.

Figure 1
Figure 1. Protein foundation model performance on predictive tasks. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. PFMBench: More tasks, multimodal PFMs, a simplified protocol, and hierarchical analysis. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The Overall framework of PFMBench. The framework includes: (1) a user-friendly [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Task relations in supervised tuning. Core Model Performance on Representative Tasks [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: UMAP visualization of ESM2, ProstT5, and ProTrek on Deeploc2_Multi. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The MID distribution of sequence-only models relative to ESM2-35M. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Model rank on tasks. A.3 Mutual Information Mutual Information Difference Metric. For a set of MSA sequences {x (0), x(1), x(2), x(3) , · · · }, we compute the mutual information (MI) [62, 50, 44] between the target sequence x (0) and a query sequence x (i) . When the …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

80 extracted references · 47 canonical work pages

  1. [1]

    Deeploc: prediction of protein subcellular localization using deep learning

    José Juan Almagro Armenteros, Casper Kaae Sønderby, Søren Kaae Sønderby, Henrik Nielsen, and Ole Winther. Deeploc: prediction of protein subcellular localization using deep learning. Bioinformatics, 33(21):3387–3395, 2017

    work page 2017

  2. [2]

    Gene ontology: tool for the unification of biology.Nature genetics, 25(1):25–29, 2000

    Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan P Davis, Kara Dolinski, Selina S Dwight, Janan T Eppig, et al. Gene ontology: tool for the unification of biology.Nature genetics, 25(1):25–29, 2000

    2000

  3. [3]

    The enzyme database in 2000.Nucleic acids research, 28(1):304–305, 2000

    Amos Bairoch. The enzyme database in 2000.Nucleic acids research, 28(1):304–305, 2000

    work page 2000

  4. [4]

    Foundation models of protein sequences: A brief overview.Current Opinion in Structural Biology, 91:103004, 2025

    Andreas Bjerregaard, Peter Mørch Groth, Søren Hauberg, Anders Krogh, and Wouter Boomsma. Foundation models of protein sequences: A brief overview.Current Opinion in Structural Biology, 91:103004, 2025

    work page 2025

  5. [5]

    xtrimopglm: unified 100b-scale pre-trained transformer for deciphering the language of protein.arXiv preprint arXiv:2401.06199, 2024

    Bo Chen, Xingyi Cheng, Pan Li, Yangli-ao Geng, Jing Gong, Shen Li, Zhilei Bei, Xu Tan, Boyan Wang, Xin Zeng, et al. xtrimopglm: unified 100b-scale pre-trained transformer for deciphering the language of protein.arXiv preprint arXiv:2401.06199, 2024

    arXiv 2024

  6. [6]

    Structure-aware protein solubility prediction from sequence through graph convolutional network and predicted contact map.Journal of cheminformatics, 13:1–10, 2021

    Jianwen Chen, Shuangjia Zheng, Huiying Zhao, and Yuedong Yang. Structure-aware protein solubility prediction from sequence through graph convolutional network and predicted contact map.Journal of cheminformatics, 13:1–10, 2021

    work page 2021

  7. [7]

    Flip: Benchmark tasks in fitness landscape inference for proteins

    Christian Dallago, Jody Mou, Kadina E Johnston, Bruce Wittmann, Nick Bhattacharya, Samuel Goldman, Ali Madani, and Kevin K Yang. Flip: Benchmark tasks in fitness landscape inference for proteins. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)

    work page

  8. [8]

    Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems, 36:10088– 10115, 2023

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems, 36:10088– 10115, 2023

    2023

Show all 80 references
  1. [9]

    Loca: Location-aware cosine adaptation for parameter-efficient fine-tuning

    Zhekai Du, Yinjie Min, Jingjing Li, Ke Lu, Changliang Zou, Liuhua Peng, Tingjin Chu, and Mingming Gong. Loca: Location-aware cosine adaptation for parameter-efficient fine-tuning. arXiv preprint arXiv:2502.06820, 2025

    2025 arXiv

  2. [10]

    Ankh: Optimized protein language model unlocks general-purpose modelling.arXiv preprint arXiv:2301.06568, 2023

    Ahmed Elnaggar, Hazem Essam, Wafaa Salah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Charlotte Rochereau, and Burkhard Rost. Ankh: Optimized protein language model unlocks general-purpose modelling.arXiv preprint arXiv:2301.06568, 2023

    2023 arXiv

  3. [11]

    Prottrans: towards cracking the language of life’s code through self-supervised learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44:7112–7127, 2021

    Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: towards cracking the language of life’s code through self-supervised learning.IEEE Transactions on Patter...

    2021

  4. [12]

    Protgpt2 is a deep unsupervised language model for protein design.Nature communications, 13(1):4348, 2022

    Noelia Ferruz, Steffen Schmidt, and Birte Höcker. Protgpt2 is a deep unsupervised language model for protein design.Nature communications, 13(1):4348, 2022

    2022

  5. [13]

    Deep learning prediction of enzyme optimum ph.bioRxiv, pages 2023–06, 2023

    Japheth E Gado, Matthew Knotts, Ada Y Shaw, Debora Marks, Nicholas P Gauthier, Chris Sander, and Gregg T Beckham. Deep learning prediction of enzyme optimum ph.bioRxiv, pages 2023–06, 2023

    2023

  6. [14]

    Proteinin- vbench: Benchmarking protein inverse folding on diverse tasks, models, and metrics.Advances in Neural Information Processing Systems, 36:68207–68220, 2023

    Zhangyang Gao, Cheng Tan, Yijie Zhang, Xingran Chen, Lirong Wu, and Stan Z Li. Proteinin- vbench: Benchmarking protein inverse folding on diverse tasks, models, and metrics.Advances in Neural Information Processing Systems, 36:68207–68220, 2023

    2023

  7. [15]

    Quan- titative missense variant effect prediction using large-scale mutagenesis data.Cell systems, 6(1):116–124, 2018

    Vanessa E Gray, Ronald J Hause, Jens Luebeck, Jay Shendure, and Douglas M Fowler. Quan- titative missense variant effect prediction using large-scale mutagenesis data.Cell systems, 6(1):116–124, 2018

    2018

  8. [16]

    Foundation models in bioinformatics.National Science Review, page nwaf028, 2025

    Fei Guo, Renchu Guan, Yaohang Li, Qi Liu, Xiaowo Wang, Can Yang, and Jianxin Wang. Foundation models in bioinformatics.National Science Review, page nwaf028, 2025. 10

    2025

  9. [17]

    Using support vector machine combined with auto covariance to predict protein–protein interactions from protein sequences

    Yanzhi Guo, Lezheng Yu, Zhining Wen, and Menglong Li. Using support vector machine combined with auto covariance to predict protein–protein interactions from protein sequences. Nucleic acids research, 36(9):3025–3030, 2008

    2008

  10. [18]

    Simulating 500 million years of evolution with a language model.Science, page eads0018, 2025

    Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model.Science, page eads0018, 2025

    2025

  11. [19]

    To- wards a unified view of parameter-efficient transfer learning

    Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. To- wards a unified view of parameter-efficient transfer learning. InInternational Conference on Learning Representations

  12. [20]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  13. [21]

    Bilingual language model for protein sequence and structure.NAR Genomics and Bioinformatics, 6(4):lqae150, 2024

    Michael Heinzinger, Konstantin Weissenow, Joaquin Gomez Sanchez, Adrian Henkel, Milot Mirdita, Martin Steinegger, and Burkhard Rost. Bilingual language model for protein sequence and structure.NAR Genomics and Bioinformatics, 6(4):lqae150, 2024

    2024

  14. [22]

    Long short-term memory.Neural computation, 9(8):1735–1780, 1997

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory.Neural computation, 9(8):1735–1780, 1997

    1997

  15. [23]

    Parameter-efficient transfer learning for nlp

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. InInternational conference on machine learning, pages 2790–2799. PMLR, 2019

    2019

  16. [24]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  17. [25]

    Exploring evolution-aware &-free protein language models as protein function predictors

    Mingyang Hu, Fajie Yuan, Kevin Yang, Fusong Ju, Jin Su, Hui Wang, Fei Yang, and Qiuyang Ding. Exploring evolution-aware &-free protein language models as protein function predictors. Advances in Neural Information Processing Systems, 35:38873–38884, 2022

    2022

  18. [26]

    Meltome at- las—thermal proteome stability across the tree of life.Nature methods, 17(5):495–503, 2020

    Anna Jarzab, Nils Kurzawa, Thomas Hopf, Matthias Moerch, Jana Zecha, Niels Leijten, Yangyang Bian, Eva Musiol, Melanie Maschberger, Gabriele Stoehr, et al. Meltome at- las—thermal proteome stability across the tree of life.Nature methods, 17(5):495–503, 2020

    2020

  19. [27]

    Deepsol: a deep learning framework for sequence-based protein solubility prediction

    Sameer Khurana, Reda Rawi, Khalid Kunji, Gwo-Yu Chuang, Halima Bensmail, and Raghven- dra Mall. Deepsol: a deep learning framework for sequence-based protein solubility prediction. Bioinformatics, 34(15):2605–2613, 2018

    2018

  20. [28]

    Netsurfp-2.0: Improved prediction of protein structural features by integrated deep learning.Proteins: Structure, Function, and Bioinformatics, 87(6):520–527, 2019

    Michael Schantz Klausen, Martin Closter Jespersen, Henrik Nielsen, Kamilla Kjaergaard Jensen, Vanessa Isabell Jurtz, Casper Kaae Soenderby, Morten Otto Alexander Sommer, Ole Winther, Morten Nielsen, Bent Petersen, et al. Netsurfp-2.0: Improved prediction of protein structural ...

    2019

  21. [29]

    Kyohei Koyama, Kosuke Hashimoto, Chioko Nagao, and Kenji Mizuguchi. Attention network for predicting t-cell receptor–peptide binding can associate attention with interpretable protein structural properties.Frontiers in Bioinformatics, 3:1274599, 2023

    2023

  22. [30]

    The power of scale for parameter-efficient prompt tuning

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, 2021

    2021

  23. [31]

    Deep learning-based k cat prediction enables improved enzyme- constrained model reconstruction.Nature Catalysis, 5(8):662–672, 2022

    Feiran Li, Le Yuan, Hongzhong Lu, Gang Li, Yu Chen, Martin KM Engqvist, Eduard J Kerkhoven, and Jens Nielsen. Deep learning-based k cat prediction enables improved enzyme- constrained model reconstruction.Nature Catalysis, 5(8):662–672, 2022

    2022

  24. [32]

    Learning deep representations of enzyme thermal adaptation.Protein Science, 31(12):e4480, 2022

    Gang Li, Filip Buric, Jan Zrimec, Sandra Viknander, Jens Nielsen, Aleksej Zelezniak, and Martin KM Engqvist. Learning deep representations of enzyme thermal adaptation.Protein Science, 31(12):e4480, 2022. 11

    2022

  25. [33]

    Progress and opportunities of foundation models in bioinformatics

    Qing Li, Zhihang Hu, Yixuan Wang, Lei Li, Yimin Fan, Irwin King, Gengjie Jia, Sheng Wang, Le Song, and Yu Li. Progress and opportunities of foundation models in bioinformatics. Briefings in Bioinformatics, 25(6):bbae548, 2024

    2024

  26. [34]

    Prefix-tuning: Optimizing continuous prompts for generation

    Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pape...

    2021

  27. [35]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    2023

  28. [36]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.Advances in Neural Information Processing Systems, 35:1950–1965, 2022

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.Advances in Neural Information Processing Systems, 35:1950–1965, 2022

    1950

  29. [37]

    Bindingdb: a web-accessible database of experimentally determined protein–ligand binding affinities.Nucleic acids research, 35(suppl_1):D198–D201, 2007

    Tiqing Liu, Yuhmei Lin, Xin Wen, Robert N Jorissen, and Michael K Gilson. Bindingdb: a web-accessible database of experimentally determined protein–ligand binding affinities.Nucleic acids research, 35(suppl_1):D198–D201, 2007

    2007

  30. [38]

    Forging the basis for developing protein–ligand interaction scoring functions.Accounts of chemical research, 50(2):302–309, 2017

    Zhihai Liu, Minyi Su, Li Han, Jie Liu, Qifan Yang, Yan Li, and Renxiao Wang. Forging the basis for developing protein–ligand interaction scoring functions.Accounts of chemical research, 50(2):302–309, 2017

    2017

  31. [39]

    Scop: a structural classification of proteins database.Nucleic acids research, 28(1):257–259, 2000

    Loredana Lo Conte, Bart Ailey, Tim JP Hubbard, Steven E Brenner, Alexey G Murzin, and Cyrus Chothia. Scop: a structural classification of proteins database.Nucleic acids research, 28(1):257–259, 2000

    2000

  32. [40]

    Prollama: A protein large language model for multi-task protein language processing.IEEE Transactions on Artificial Intelligence, 2025

    Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Li Yuan, and Yonghong Tian. Prollama: A protein large language model for multi-task protein language processing.IEEE Transactions on Artificial Intelligence, 2025

    2025

  33. [41]

    Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41(8):1099–1106, 2023

    Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos Jr, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41(8):10...

    2023

  34. [42]

    Peft: State-of-the-art parameter-efficient fine-tuning methods

    Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. Peft: State-of-the-art parameter-efficient fine-tuning methods. https: //github.com/huggingface/peft, 2022

    2022

  35. [43]

    Dora: Enhancing parameter-efficient fine-tuning with dynamic rank distribution

    Yulong Mao, Kaiyu Huang, Changhao Guan, Ganglin Bao, Fengran Mo, and Jinan Xu. Dora: Enhancing parameter-efficient fine-tuning with dynamic rank distribution. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag...

    2024

  36. [44]

    Formal limitations on the measurement of mutual infor- mation

    David McAllester and Karl Stratos. Formal limitations on the measurement of mutual infor- mation. InInternational Conference on Artificial Intelligence and Statistics, pages 875–884. PMLR, 2020

    2020

  37. [45]

    Skempi: a structural kinetic and energetic database of mutant protein interactions and its use in empirical models.Bioinformatics, 28(20):2600–2607, 2012

    Iain H Moal and Juan Fernández-Recio. Skempi: a structural kinetic and energetic database of mutant protein interactions and its use in empirical models.Bioinformatics, 28(20):2600–2607, 2012

    2012

  38. [46]

    Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    2023

  39. [47]

    Proteingym: Large- scale benchmarks for protein fitness prediction and design.Advances in Neural Information Processing Systems, 36:64331–64379, 2023

    Pascal Notin, Aaron Kollasch, Daniel Ritter, Lood Van Niekerk, Steffanie Paul, Han Spinner, Nathan Rollins, Ada Shaw, Rose Orenbuch, Ruben Weitzman, et al. Proteingym: Large- scale benchmarks for protein fitness prediction and design.Advances in Neural Information Processing S...

    2023

  40. [48]

    Large-scale prediction of human protein- protein interactions from amino acid sequence based on latent topic features.Journal of proteome research, 9(10):4992–5001, 2010

    Xiao-Yong Pan, Ya-Nan Zhang, and Hong-Bin Shen. Large-scale prediction of human protein- protein interactions from amino acid sequence based on latent topic features.Journal of proteome research, 9(10):4992–5001, 2010

    2010

  41. [49]

    Adapterfusion: Non-destructive task composition for transfer learning.arXiv preprint arXiv:2005.00247, 2020

    Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning.arXiv preprint arXiv:2005.00247, 2020

    2005 arXiv

  42. [50]

    On variational bounds of mutual information

    Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. InInternational conference on machine learning, pages 5171–

  43. [51]

    Procyon: A multimodal foundation model for protein phenotypes.BioRxiv, pages 2024–12, 2024

    Owen Queen, Yepeng Huang, Robert Calef, Valentina Giunchiglia, Tianlong Chen, George Dasoulas, LeAnn Tai, Yasha Ektefaie, Ayush Noori, Joseph Brown, et al. Procyon: A multimodal foundation model for protein phenotypes.BioRxiv, pages 2024–12, 2024

    2024

  44. [52]

    Evaluating protein transfer learning with tape.Advances in neural information processing systems, 32, 2019

    Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Peter Chen, John Canny, Pieter Abbeel, and Yun Song. Evaluating protein transfer learning with tape.Advances in neural information processing systems, 32, 2019

    2019

  45. [53]

    Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the National A...

    2021

  46. [54]

    Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017

    Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017

    2017

  47. [55]

    Saprot: Protein language modeling with structure-aware vocabulary

    Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure-aware vocabulary. InThe Twelfth International Conference on Learning Representations

  48. [56]

    Protrek: Navigating the protein universe through tri-modal contrastive learning.bioRxiv, pages 2024–05, 2024

    Jin Su, Xibin Zhou, Xuting Zhang, and Fajie Yuan. Protrek: Navigating the protein universe through tri-modal contrastive learning.bioRxiv, pages 2024–05, 2024

    2024

  49. [57]

    Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31(6):926–932, 2015

    Baris E Suzek, Yuqi Wang, Hongzhan Huang, Peter B McGarvey, Cathy H Wu, and UniProt Consortium. Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31(6):926–932, 2015

    2015

  50. [58]

    Peta: evaluating the impact of protein transfer learning with sub-word tokenization on downstream applications.Journal of Cheminformatics, 16(1):92, 2024

    Yang Tan, Mingchen Li, Ziyi Zhou, Pan Tan, Huiqun Yu, Guisheng Fan, and Liang Hong. Peta: evaluating the impact of protein transfer learning with sub-word tokenization on downstream applications.Journal of Cheminformatics, 16(1):92, 2024

    2024

  51. [59]

    Venusfactory: A unified platform for protein engineering data retrieval and language model fine-tuning.arXiv preprint arXiv:2503.15438, 2025

    Yang Tan, Chen Liu, Jingyuan Gao, Banghao Wu, Mingchen Li, Ruilin Wang, Lingrong Zhang, Huiqun Yu, Guisheng Fan, Liang Hong, et al. Venusfactory: A unified platform for protein engineering data retrieval and language model fine-tuning.arXiv preprint arXiv:2503.15438, 2025

    2025 arXiv

  52. [60]

    Protsolm: Protein solubility prediction with multi-modal features

    Yang Tan, Jia Zheng, Liang Hong, and Bingxin Zhou. Protsolm: Protein solubility prediction with multi-modal features. In2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 223–232. IEEE, 2024

    2024

  53. [61]

    Deeploc 2.0: multi-label subcellular localization prediction using protein language models.Nucleic acids research, 50(W1):W228–W234, 2022

    Vineet Thumuluri, José Juan Almagro Armenteros, Alexander Rosenberg Johansen, Henrik Nielsen, and Ole Winther. Deeploc 2.0: multi-label subcellular localization prediction using protein language models.Nucleic acids research, 50(W1):W228–W234, 2022

    2022

  54. [62]

    On mutual information maximization for representation learning

    Michael Tschannen, Josip Djolonga, Paul K Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. InInternational Conference on Learning Representations

  55. [63]

    Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models.Nucleic acids research, 50(D1):D439–D444, 2022

    Mihaly Varadi, Stephen Anyango, Mandar Deshpande, Sreenath Nair, Cindy Natassia, Galabina Yordanova, David Yuan, Oana Stroe, Gemma Wood, Agata Laydon, et al. Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-a...

    2022

  56. [64]

    Attention is all you need.NeurIPS, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.NeurIPS, 30, 2017

    2017

  57. [65]

    Prediction of protein solubility based on sequence physicochemical patterns and distributed representation information with deepsolue.BMC biology, 21(1):12, 2023

    Chao Wang and Quan Zou. Prediction of protein solubility based on sequence physicochemical patterns and distributed representation information with deepsolue.BMC biology, 21(1):12, 2023

    2023

  58. [66]

    Huilin Wang, Mingjun Wang, Hao Tan, Yuan Li, Ziding Zhang, and Jiangning Song. Predppcrys: accurate prediction of sequence cloning, protein production, purification and crystallization propensity from protein sequences using multi-step heterogeneous feature fusion and selectio...

    2014

  59. [67]

    A comprehensive computational benchmark for evaluating deep learning-based protein function prediction approaches.Briefings in Bioinformatics, 25(2):bbae050, 2024

    Wenkang Wang, Yunyan Shuai, Qiurong Yang, Fuhao Zhang, Min Zeng, and Min Li. A comprehensive computational benchmark for evaluating deep learning-based protein function prediction approaches.Briefings in Bioinformatics, 25(2):bbae050, 2024

    2024

  60. [68]

    Diffusion language models are versatile protein learners

    Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Diffusion language models are versatile protein learners. InInternational Conference on Machine Learning, pages 52309–52333. PMLR, 2024

    2024

  61. [69]

    Mixture of lora experts.arXiv preprint arXiv:2404.13628, 2024

    Xun Wu, Shaohan Huang, and Furu Wei. Mixture of lora experts.arXiv preprint arXiv:2404.13628, 2024

    2024 arXiv

  62. [70]

    Ccbhla: pan-specific peptide–hla class i binding prediction via convolutional and bilstm features.bioRxiv, pages 2023–04, 2023

    Yejian Wu, Lujing Cao, Zhipeng Wu, Xinyi Wu, Xinqiao Wang, and Hongliang Duan. Ccbhla: pan-specific peptide–hla class i binding prediction via convolutional and bilstm features.bioRxiv, pages 2023–04, 2023

    2023

  63. [71]

    Protst: Multi-modality learning of protein sequences and biomedical texts

    Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang. Protst: Multi-modality learning of protein sequences and biomedical texts. InInternational Conference on Machine Learning, pages 38749–38767. PMLR, 2023

    2023

  64. [72]

    Peer: a comprehensive and multi-task benchmark for protein sequence understanding.Advances in Neural Information Processing Systems, 35:35156–35173, 2022

    Minghao Xu, Zuobai Zhang, Jiarui Lu, Zhaocheng Zhu, Yangtian Zhang, Ma Chang, Runcheng Liu, and Jian Tang. Peer: a comprehensive and multi-task benchmark for protein sequence understanding.Advances in Neural Information Processing Systems, 35:35156–35173, 2022

    2022

  65. [73]

    Care: a benchmark suite for the classification and retrieval of enzymes

    Jason Yang, Ariane Mora, Shengchao Liu, Bruce Wittmann, Animashree Anandkumar, Frances Arnold, and Yisong Yue. Care: a benchmark suite for the classification and retrieval of enzymes. Advances in Neural Information Processing Systems, 37:3094–3121, 2024

    2024

  66. [74]

    Improved protein structure prediction using predicted interresidue orientations

    Jianyi Yang, Ivan Anishchenko, Hahnbeom Park, Zhenling Peng, Sergey Ovchinnikov, and David Baker. Improved protein structure prediction using predicted interresidue orientations. Proceedings of the National Academy of Sciences, 117(3):1496–1503, 2020

    2020

  67. [75]

    Proteinbench: A holistic evaluation of protein foundation models.arXiv preprint arXiv:2409.06744, 2024

    Fei Ye, Zaixiang Zheng, Dongyu Xue, Yuning Shen, Lihao Wang, Yiming Ma, Yan Wang, Xinyou Wang, Xiangxin Zhou, and Quanquan Gu. Proteinbench: A holistic evaluation of protein foundation models.arXiv preprint arXiv:2409.06744, 2024

    2024 arXiv

  68. [76]

    Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models

    Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 1–9, 2022

    2022

  69. [77]

    Ontoprotein: Protein pretraining with gene ontology embedding

    Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Qiang Zhang, Jiazhang Lian, and Huajun Chen. Ontoprotein: Protein pretraining with gene ontology embedding. InInternational Conference on Learning Representations

  70. [78]

    Adalora: Adaptive budget allocation for parameter- efficient fine-tuning.arXiv preprint arXiv:2303.10512, 2023

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adalora: Adaptive budget allocation for parameter- efficient fine-tuning.arXiv preprint arXiv:2303.10512, 2023

    2023 arXiv

  71. [79]

    Protein representation learning by geometric structure pretraining

    Zuobai Zhang, Minghao Xu, Arian Rokkum Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. In The Eleventh International Conference on Learning Representations

  72. [80]

    Protclip: Function-informed protein multi-modal learning

    Hanjing Zhou, Mingze Yin, Wei Wu, Mingyang Li, Kun Fu, Jintai Chen, Jian Wu, and Zheng Wang. Protclip: Function-informed protein multi-modal learning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 22937–22945, 2025. 14 A Appendix A.1 Support...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.