Pith. sign in

REVIEW 4 major objections 6 minor 37 references

Scaling and Data Saturation in Protein Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims protein language models trained on UniRef100 snapshots 2011–2024 have not saturated: zero-shot fitness prediction improves but not monotonically; supervised performance scales mainly with labeled data.

desk verdict Useful temporal data-scaling measurement on AMPLIFY snapshots, but the saturation inference is underdetermined by single-seed, composition-confounded comparisons. read the letter →

arxiv 2507.22210 v1 pith:BPN3KD7B submitted 2025-07-29 q-bio.QM

Aviv Spinner , Erika DeBenedictis , Corey M. Hudson This is my paper · ORCID
classification q-bio.QM
keywords proteinlanguagemodelsdatascalinglawssaturationzero-shotfitnesspredictionsemi-supervisedlearningGymbenchmarkUniRef100snapshotsdeepmutationalscanning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether protein language models have reached data saturation, the point where adding more unlabeled protein sequences stops improving function prediction. By benchmarking a suite of models (AMPLIFY) trained on yearly UniRef100 snapshots from 2011 to 2024 against experimental fitness measurements, the authors find no evidence of saturation. Zero-shot performance improves overall across the decade but fluctuates year to year, with a noticeable drop between 2018 and 2021 despite roughly a billion additional sequences. Supervised ridge regression depends mainly on how much labeled data is supplied, not on the pretraining snapshot year. The authors conclude that for protein function prediction more data still helps, but the gains do not yet follow a simple, monotonic scaling law.

What carries the argument

The empirical instrument is a series of 14 AMPLIFY protein language models, a family of small masked language models sharing the same architecture and pretrained on yearly UniRef100 snapshots from 2011 to 2024. Performance is measured as the Spearman correlation between the log-likelihood a model assigns to a mutant sequence and its experimentally measured fitness in ProteinGym, with supervised experiments adding ridge regression on mean-pooled final-layer embeddings. The engineered train/test splits — random, contiguous, and modulo — are the mechanism that controls for positional data leakage, allowing the paper to separate genuine generalization from memorization of mutation positions.

What would settle it

Retrain the same AMPLIFY architecture on size-matched, composition-controlled subsets taken from different UniRef100 release years, then check whether the zero-shot Spearman curve becomes monotonic; a flat or monotonic curve would show that the non-monotonicity reported here is a data-composition artifact rather than evidence about saturation.

Watch

Extended reading notes

Core claim

The central claim is that biological data scaling for protein fitness prediction has not yet saturated and does not follow the smooth power laws seen in natural language processing. Using yearly UniRef100 checkpoints of the same model architecture, zero-shot Spearman correlation on ProteinGym substitution datasets fluctuates without monotonic improvement: the 2018-to-2021 models perform worse on average even though UniRef100 gained over a billion sequences. Supervised ridge regression on frozen embeddings shows a different pattern, with performance scaling mainly with labeled data volume, and with one-hot encodings beating embeddings under random splits but collapsing under leakage-controlled splits. In the β-Lactamase case study, unsupervised predictions improve from roughly 0.25 to above 0.6 Spearman correlation across pretraining years, yet training on a single experimental dataset suffices to match a decade of pretraining data in the supervised setting. These observations together imply that pretraining data volume is not yet the binding constraint for function prediction.

Load-bearing premise

The load-bearing premise is that the yearly AMPLIFY checkpoints isolate the effect of pretraining data scale, even though UniRef100 snapshots also differ in redundancy and composition over time and each snapshot comes from a single training run.

Editorial extensions

If this is right

  • Unlabeled protein sequence repositories still have room to improve function prediction; the field has not yet passed the point of data saturation.
  • Pretraining data quantity and labeled data volume act differently: zero-shot gains from pretraining are uneven, while supervised gains track the size of the labeled training set.
  • Leakage-free evaluation splits overturn the apparent advantage of one-hot encodings under random splits, so scaling conclusions depend on controlling for positional leakage.
  • Aggregate trends hide stratification: proteins with deep MSAs and organismal-fitness assays improve with later pretraining years, while low-MSA-depth proteins can decline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because yearly UniRef100 snapshots confound data quantity with changing redundancy, the 'no saturation' reading would be strengthened by controlling for sequence diversity per snapshot.
  • Editorial inference: a testable extension is to train on cluster-split subsets with matched diversity across years; if the performance curve flattens, scaling gains come from new sequence diversity rather than raw count.
  • Editorial inference: the β-Lactamase crossover, where one labeled dataset matches a decade of pretraining, suggests that assay collection may currently buy more performance per unit of effort than further unlabeled sequencing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates how protein language model performance scales with the amount of pretraining data by evaluating the AMPLIFY model suite, which contains checkpoints trained on yearly snapshots of UniRef100 from 2011 to 2024. Zero-shot variant effect prediction is measured by Spearman correlation between model log-likelihoods and ProteinGym substitution DMS scores, and supervised performance is measured by ridge regression on AMPLIFY embeddings under several train/test split schemes. The authors report that zero-shot performance fluctuates non-monotonically across pretraining years, that supervised performance depends mainly on labeled data volume rather than pretraining year, and that β-Lactamase cross-dataset evaluation shows improving unsupervised trends. They conclude that biological data scaling for protein function prediction does not follow a simple monotonic trend and that data saturation has not yet been reached.

Significance. If the central claim were established, the paper would be a useful contribution to the under-explored question of data scaling in protein language models, with direct implications for data acquisition and curation. The paper has concrete strengths: it uses an external benchmark (ProteinGym), leverages a unique suite of time-split checkpoints from a single training pipeline, releases all code, and includes leakage-controlled split schemes (contiguous and modulo) that correctly expose the failure of one-hot encodings. The β-Lactamase cross-dataset analysis is a thoughtful prospective evaluation. However, the central inference from non-monotonic year-over-year performance to 'not yet saturated' is confounded, as the year axis conflates data scale with snapshot composition and single-seed training noise. The empirical observations are valuable, but the main conclusion needs substantial reframing or additional controls.

major comments (4)
  1. [Section 3, Figure 2] The central zero-shot comparison treats the year of the UniRef100 snapshot as a proxy for data scale, but the manuscript itself documents that the snapshots differ in composition as well as size: Figure 1C shows declining UniRef90/UniRef100 and UniRef50/UniRef100 ratios, and Section 3 attributes the 2018-2021 performance drop to 'the specific sequences added or removed at each timepoint.' Because year is not a controlled measure of data quantity, the non-monotonicity in Figure 2 does not by itself establish that scaling is non-monotonic or that saturation has not been reached. To make the central claim load-bearing, the authors should either control for snapshot composition (e.g., by matching redundancy or taxonomic composition across years), provide repeated-seed or bootstrap confidence intervals, or reframe the conclusion as 'no consistent scaling trend is observable under uncontrolled composition shifts' rather than 'not yet saturated.'
  2. [Section 3 and Section 4] The inference from 'variability in correlation ... indicates that it has not yet reached data saturation' is logically incomplete. Non-monotonic performance under changing data composition is compatible with a model that has already saturated in terms of data quantity but remains sensitive to distribution shift or optimization noise; conversely, monotonic improvement does not by itself prove absence of saturation. The paper should define an explicit, testable criterion for saturation (for example, a plateau in performance when data scale is increased under fixed composition and repeated training seeds) and evaluate that criterion rather than equating non-monotonicity with lack of saturation.
  3. [Section 2.2.2, Figure 3] The ridge regression experiments are a central part of the supervised scaling analysis, but the regularization strength (alpha) is never reported. Since ridge regression with high-dimensional embeddings can be sensitive to alpha, the manuscript must state whether alpha was fixed across models and splits, how it was chosen, and whether the conclusions are robust to its value. Without this information, the claim that supervised performance shows 'minimal performance differences across AMPLIFY models' is not fully reproducible or verifiable.
  4. [Figure 2, Section 3] The zero-shot Spearman correlations are reported as point estimates from single-seed models, with no confidence intervals or statistical tests. The year-over-year differences emphasized in the text (e.g., the 2018-2021 drop) are on the order of 0.02-0.04 in the figure, which may be within the range of optimization noise or dataset sampling variability. The authors should report bootstrap confidence intervals across the 213 ProteinGym datasets, or per-dataset paired comparisons across models, to support the claim that the observed non-monotonicity is a real effect of pretraining data rather than noise.
minor comments (6)
  1. [Abstract] The abstract states that 'performance improves—but not monotonically—with added data,' but Figure 2 shows an overall increase from 2011 to 2024 with fluctuations; please clarify whether 'improves' refers to the overall trend or to most individual year-over-year steps.
  2. [Section 2.2.2] The sentence 'Results are show in Figure 3B' contains a typo and should read 'Results are shown in Figure 3B.'
  3. [Appendix Table 1] The UniProt count for 2016 (60,268,458) is lower than the 2015 count (89,998,523), which appears implausible for a cumulative database; please verify this entry or add a note explaining any change in counting methodology.
  4. [Figure S5 caption] The caption refers to 'Figure 2B' but should refer to Figure 3B, since the modulo split is compared with the random split in Figure 3A.
  5. [Section 1.3] The phrase 'we would project data saturation on the field' is unclear; consider rephrasing to 'we would predict data saturation and provide concrete guidance for achieving it.'
  6. [General] For reproducibility, please provide a specific version or commit hash of the GitHub repository and the exact AMPLIFY checkpoint identifiers, as the HuggingFace paths use a <YEAR> placeholder.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central trend is read directly from external ProteinGym evaluations of third-party AMPLIFY checkpoints, with no fitted parameter or self-citation chain doing load-bearing work.

full rationale

The paper's derivation chain is straightforward and self-contained with respect to external evidence. It takes AMPLIFY models pretrained by another group (Fournier et al.) on yearly UniRef100 snapshots, computes sequence log-likelihoods and embeddings on ProteinGym variants, and compares them against experimentally measured DMS fitness values via Spearman correlation. The central claim—that performance improves but not monotonically with added pretraining data and has not reached saturation—is an empirical reading of those external evaluations. No equation defines the conclusion in terms of its inputs, and no fitted parameter is renamed as a prediction: the ridge regression results are standard held-out evaluations, and the zero-shot results use no trainable parameters at all. The concern that yearly UniRef100 snapshots differ in composition as well as size (Figure 1C), and that single-seed training leaves fluctuations unquantified, is a validity limitation on the causal interpretation of the year axis, not a circularity, because the reported observations would hold regardless of how the paper frames them. The paper cites prior work such as ProteinGym and AMPLIFY, but these are external resources produced by other research groups, and none of the cited results is a self-citation that supplies the paper's conclusion. No uniqueness theorem, no ansatz-smuggling citation, and no renaming of a known result into new coordinates occurs. Therefore the analysis exhibits no significant circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the ProteinGym metric, the temporal snapshot setup, and an interpretive step linking non-monotonicity to saturation. The only free parameter is the ridge regularization strength, which is unreported; no new entities are introduced.

free parameters (1)
  • Ridge regression regularization strength (alpha)
    Used for all supervised experiments in Section 2.2.2; the value is not reported in the text, so it is effectively a hand-chosen hyperparameter that could affect the supervised scaling conclusions.
assumptions (3)
  • domain assumption ProteinGym substitution DMS Spearman correlation is a valid proxy for protein function prediction performance.
    The entire analysis is anchored to this metric; ProteinGym covers a narrow, curated slice of protein space. Stated in Section 2.1.
  • domain assumption Yearly AMPLIFY checkpoints isolate the effect of pretraining data scale.
    The paper acknowledges redundancy changes across years (Figure 1C) and that AMPLIFY is single-seed; year-to-year differences may reflect composition or seed rather than data size. Invoked throughout Section 3.
  • ad hoc to paper Absence of monotonic scaling implies sensitivity to data and lack of saturation.
    The inference in Section 3 and the Conclusion equates non-monotonic performance with 'has yet to saturate', an interpretive step that is not proven by the data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling and Data Saturation in Protein Language Models." pith.science (2026). https://pith.science/paper/BPN3KD7B

@misc{pith2026250722210,
  author       = {Pith},
  title        = {Pith review of: Scaling and Data Saturation in Protein Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BPN3KD7B}},
  note         = {Machine review of arXiv:2507.22210}
}
read the original abstract

Data in biology is redundant, noisy, and sparse. How does the type and scale of available data impact model performance? In this work, we specifically investigate how protein language models (pLMs) scale with increasing pretraining data. We investigate this relationship by measuring the performance of protein function prediction on a suite of pLMs pretrained on yearly snapshots of UniRef100 from 2011 to 2024. We find no evidence of model saturation on this task: performance improves--but not monotonically--with added data, and this trend differs between unsupervised and supervised experiments. Using a well-characterized Beta-Lactamase protein from E. coli, we find that unsupervised model predictions get better year-over-year, though they do not yet consistently perform better than the supervised baseline. Our results underscore the need for targeted data acquisition and deeper study of data scaling in protein modeling. All training, inference, analysis, and visualization code is available at: https://github.com/Align-to-Innovate/data-saturation-and-scaling.

Figures

Figures reproduced from arXiv: 2507.22210 by the authors.

Figure 1
Figure 1. A) Overview of the benchmarking pipeline: protein sequences from UniRef train a suite of protein language mod￾els. We use these models to compute logits and embeddings for each sequence in ProteinGym. These are used for zero-shot predic￾tion or representations for linear regression, respectively. Model performance is assessed using Spearman correlation with experi￾mental fitness scores. B) Growth of UniRef clusters … view at source ↗
Figure 2
Figure 2. Sequence growth does not guarantee performance gains. The total number of sequences in UniProt increases steadily from 2011 to 2024. Meanwhile, the Spearman correlation of an AMPLIFY model trained on yearly data sources fluctuates and does not show a monotonic improvement in performance. 2.2.2. SUPERVISED Sequence Embeddings: To obtain each protein’s sequence embedding for supervised learning tasks, we tokenize the … view at source ↗
Figure 3
Figure 3. Effect of semi-supervised training data size and split strategy on model performance. A) Random train/test splits with increasing amounts of labeled data show minimal perfor￾mance differences across AMPLIFY models. One-hot encoding outperforms model embeddings until a large volume of labeled data is used. B) Contiguous train/test splits reveal no clear rela￾tionship between model performance and the amount of pretra… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Tradeoff between pretrained model and semi￾supervised learning. Unsupervised performance (grey traces) improves as AMPLIFY models are pretrained on more UniRef100 data. In contrast, semi-supervised learning (blue traces) yields relatively stable performance across mode…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 26 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    C., Hoffnagle, A

    Bhatnagar, A., Jain, S., Beazer, J., Curran, S. C., Hoffnagle, A. M., Ching, K., Martyn, M., Nayfach, S., Ruffolo, J. A., and Madani, A. Scaling unlocks broader generation and deeper functional understanding of proteins. bioRxiv, pp.\ 2025--04, 2025

    work page 2025

  3. [3]

    Training compute-optimal protein language models

    Cheng, X., Chen, B., Li, P., Gong, J., Tang, J., and Song, L. Training compute-optimal protein language models. Advances in Neural Information Processing Systems, 37: 0 69386--69418, 2024

    work page 2024

  4. [4]

    Chorlton, S. D. Ten common issues with reference sequence databases and how to mitigate them. Frontiers in Bioinformatics, 4, 2024. doi:10.3389/fbinf.2024.1278228. URL https://doi.org/10.3389/fbinf.2024.1278228. Link https://doi.org/10.3389/fbinf.2024.1278228

    work page arXiv 2024

  5. [5]

    Consortium, T. U. Uniprot: the universal protein knowledgebase in 2025. Nucleic Acids Research, 53 0 (D1): 0 D609--D617, 11 2024. ISSN 1362-4962. doi:10.1093/nar/gkae1010. URL https://doi.org/10.1093/nar/gkae1010

    doi:10.1093/nar/gkae1010 2025

  6. [6]

    P., Roux, S., Beracochea, M., Mirdita, M., Ovchinnikov, S., and Hwang, Y

    Cornman, A., West-Roberts, J., Camargo, A. P., Roux, S., Beracochea, M., Mirdita, M., Ovchinnikov, S., and Hwang, Y. The omg dataset: An open metagenomic corpus for mixed-modality genomic language modeling. bioRxiv, 2024. doi:10.1101/2024.08.14.607850. URL https://www.biorxiv.org/content/early/2024/08/17/2024.08.14.607850

    doi:10.1101/2024.08.14.607850 2024

  7. [7]

    and Steinhardt, J

    Ding, F. and Steinhardt, J. Protein language models are biased by unequal sequence sampling across the tree of life. BioRxiv, pp.\ 2024--03, 2024

    work page 2024

  8. [8]

    Prottrans: Toward understanding the language of life through self-supervised learning

    Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. Prottrans: Toward understanding the language of life through self-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 44 0 (10): 0 7112--7127, 2021

    work page 2021

Show all 37 references
  1. [9]

    M., van der Sloot, A., Schulz, B., Chandar, S., and Langmead, C

    Fournier, Q., Vernon, R. M., van der Sloot, A., Schulz, B., Chandar, S., and Langmead, C. J. Protein language models: is scaling necessary? bioRxiv, pp.\ 2024--09, 2024

    2024

  2. [10]

    X., and Abbeel, P

    Gordon, C., Lu, A. X., and Abbeel, P. Protein language model fitness is a matter of preference. bioRxiv, pp.\ 2024--10, 2024

    2024

  3. [11]

    M., Kerrn, M., Olsen, L., Salomon, J., and Boomsma, W

    Groth, P. M., Kerrn, M., Olsen, L., Salomon, J., and Boomsma, W. Kermut: Composite kernel regression for protein variant effects. Advances in Neural Information Processing Systems, 37: 0 29514--29565, 2024

    2024

  4. [12]

    Scaling laws for transfer

    Hernandez, D., Kaplan, J., Henighan, T., and McCandlish, S. Scaling laws for transfer. arXiv preprint arXiv:2102.01293, 2021

    2021 arXiv

  5. [13]

    Rita: a study on scaling up generative protein sequence models

    Hesslow, D., Zanichelli, N., Notin, P., Poli, I., and Marks, D. Rita: a study on scaling up generative protein sequence models. arXiv preprint arXiv:2205.05789, 2022

    2022 arXiv

  6. [14]

    Hoerl, A. E. and Kennard, R. W. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12 0 (1): 0 55--67, 1970

    1970

  7. [15]

    Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A., Welbl, J., Clark, A., et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022

    2022 arXiv

  8. [16]

    Learning protein fitness models from evolutionary and assay-labeled data

    Hsu, C., Nisonoff, H., Fannjiang, C., and Listgarten, J. Learning protein fitness models from evolutionary and assay-labeled data. Nature biotechnology, 40 0 (7): 0 1114--1122, 2022

    2022

  9. [17]

    Jeffery, C. J. Current successes and remaining challenges in protein function prediction. Frontiers in Bioinformatics, 3: 0 1222182, 2023. doi:10.3389/fbinf.2023.1222182. Link https://doi.org/10.3389/fbinf.2023.1222182

    2023

  10. [18]

    Highly accurate protein structure prediction with alphafold

    Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Z \' dek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. nature, 596 0 (7873): 0 583--589, 2021

    2021

  11. [19]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  12. [20]

    P., Yue, Y., Yang, K

    Li, F.-Z., Amini, A. P., Yue, Y., Yang, K. K., and Lu, A. X. Feature reuse and scaling: Understanding transfer learning with protein language models. bioRxiv, pp.\ 2024--02, 2024

    2024

  13. [21]

    Evolutionary-scale prediction of atomic-level protein structure with a language model

    Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379 0 (6637): 0 1123--1130, 2023

    2023

  14. [22]

    Louca, S., Mazel, F., Doebeli, M., and Parfrey, L. W. A census-based estimate of earth's bacterial and archaeal diversity. PLoS biology, 17 0 (2): 0 e3000106, 2019

    2019

  15. [23]

    Y., and Cho, K

    Lourie, N., Hu, M. Y., and Cho, K. Scaling laws are unreliable for downstream tasks: A reality check. arXiv preprint arXiv:2507.00885, 2025

    2025

  16. [24]

    R., Subramanian, S., Mohr, B

    Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos Jr, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. Large language models generate functional protein sequences across diverse families. Nature biotechnology, 41 0 (8): 0 1099--1106, 2023

    2023

  17. [25]

    G., Kang, B., Katrekar, D., Li, D

    Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S. H., Brixi, G., et al. Sequence modeling and design from molecular to genome scale with evo. Science, 386 0 (6723): 0 eado9336, 2024

    2024

  18. [26]

    A., Weinstein, E

    Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., and Madani, A. Progen2: exploring the boundaries of protein language models. Cell systems, 14 0 (11): 0 968--978, 2023

    2023

  19. [27]

    N., Marks, D., and Gal, Y

    Notin, P., Dias, M., Frazer, J., Marchena-Hurtado, J., Gomez, A. N., Marks, D., and Gal, Y. Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. In International Conference on Machine Learning, pp.\ 16990--17017. PMLR, 2022

    2022

  20. [28]

    Proteingym: Large-scale benchmarks for protein fitness prediction and design

    Notin, P., Kollasch, A., Ritter, D., Van Niekerk, L., Paul, S., Spinner, H., Rollins, N., Shaw, A., Orenbuch, R., Weitzman, R., et al. Proteingym: Large-scale benchmarks for protein fitness prediction and design. Advances in Neural Information Processing Systems, 36: 0 64331--...

    2023

  21. [29]

    Proteinnpt: Improving protein property prediction and design with non-parametric transformers

    Notin, P., Weitzman, R., Marks, D., and Gal, Y. Proteinnpt: Improving protein property prediction and design with non-parametric transformers. Advances in Neural Information Processing Systems, 36: 0 33529--33563, 2023 b

    2023

  22. [30]

    L., Feuermann, M., Gasteiger, E., Gehant, S., Gerritsen, V., Gos, A., Gruaz-Gumowski, N., Hinz, U., Hulo, C., Jungo, F., Keller, G., Lieberherr, D., Lombardot, T., Martin, M

    Poux, S., Magrane, M., Bateman, A., Belda, E., Boeckmann, B., Boutet, E., Breuza, L., Bridge, A., Coudert, E., Esperet, E., Famiglietti, M. L., Feuermann, M., Gasteiger, E., Gehant, S., Gerritsen, V., Gos, A., Gruaz-Gumowski, N., Hinz, U., Hulo, C., Jungo, F., Keller, G., Lieb...

    2016

  23. [31]

    Evaluating protein transfer learning with tape

    Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, P., Canny, J., Abbeel, P., and Song, Y. Evaluating protein transfer learning with tape. Advances in neural information processing systems, 32, 2019

    2019

  24. [32]

    P., Abbasian, M., and Orengo, C

    Rauer, C., Sen, N., Waman, V. P., Abbasian, M., and Orengo, C. A. Computational approaches to predict protein functional families and functional sites. Current Opinion in Structural Biology, 70: 0 108--122, 2021. ISSN 0959-440X. doi:https://doi.org/10.1016/j.sbi.2021.05.012. U...

    2021 doi

  25. [33]

    Mgnify: the microbiome sequence data analysis resource in 2023

    Richardson, L., Allen, B., Baldi, G., Beracochea, M., Bileschi, M., Burdett, T., Burgin, J., Caballero-Pérez, J., Cochrane, G., Colwell, L., Curtis, T., Escobar-Zepeda, A., Gurbich, T., Kale, V., Korobeynikov, A., Raj, S., Rogers, A., Sakharova, E., Sanchez, S., Wilkinson, D.,...

    2023 doi

  26. [34]

    L., Ma, J., et al

    Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 118 0 (...

    2021

  27. [35]

    F., Stone, J., Bianchi, A

    Rubin, A. F., Stone, J., Bianchi, A. H., Capodanno, B. J., Da, E. Y., Dias, M., Esposito, D., Frazer, J., Fu, Y., Grindstaff, S. B., et al. Mavedb 2024: a curated community database with over seven million variant effects from multiplexed functional assays. Genome Biology, 26 ...

    2024

  28. [36]

    Are protein language models compute optimal? arXiv preprint arXiv:2406.07249, 2024

    Serrano, Y., Ciudad, \'A ., and Molina, A. Are protein language models compute optimal? arXiv preprint arXiv:2406.07249, 2024

    2024 arXiv

  29. [37]

    Enhancing efficiency of protein language models with minimal wet-lab data through few-shot learning

    Zhou, Z., Zhang, L., Yu, Y., Wu, B., Li, M., Hong, L., and Tan, P. Enhancing efficiency of protein language models with minimal wet-lab data through few-shot learning. Nature Communications, 15 0 (1): 0 5566, 2024. ISSN 2041-1723. doi:10.1038/s41467-024-49798-6. URL https://do...

    2024 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.