REVIEW 4 major objections 6 minor 1 cited by
Diffusion Sequence Models for Enhanced Protein Representation and Generation
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that replacing masked-language modeling with masked diffusion in an ESM2 protein model yields one architecture that both matches similarly sized models on representation benchmarks and generates biomimetic sequences and…
desk verdict Retrofitting ESM2 with masked diffusion gives a solid representation-plus-reconstruction result, but the binder-generation headline rests on an unvalidated closed-source scorer and a winner's-curse comparison. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the masked-diffusion training objective and its reverse sampling. During training, a random fraction of tokens is replaced by the mask token and the model must predict every masked token in a single forward pass, with the loss scaled by $1/(t+\epsilon)$ so that heavily corrupted examples dominate; this is the LLaDA recipe applied to an ESM2 backbone with a modified head that caps logits before softmax and ties the final projection to the token-embedding matrix. At generation time, the model starts from all masks and progressively unmasks a chosen number of tokens per step, randomly re-masking the rest, until a full sequence emerges. The same denoising operation lets DSM both reconstruct 90 percent masked inputs and serve as a representation encoder, and the conditional variant DSM(ppi) simply concatenates a target sequence before the masked partner, making the diffusion objective target-aware.
What would settle it
Express and measure, by surface plasmon resonance or biolayer interferometry, the top DSM(ppi) designs for EGFR and PD-L1 alongside the known literature binders that the predictor ranks below them; if the measured affinities do not reproduce the predicted ordering, the claim of superior binders fails.
Extended reading notes
Core claim
The paper's central claim is that masked diffusion is a natural learning objective for protein language models, unifying high-quality representation with generative design in a single architecture. Concretely, the authors take pretrained ESM2 checkpoints and continue training them with a LLaDA-style masked-diffusion loss that masks tokens at a random fraction $t$ and asks the model to fill all masked positions in one pass, weighted by $1/(t+\epsilon)$. The resulting DSM650 reconstructs 90 percent masked validation and test sequences with an Alignment Score around $0.277$, over four standard deviations above randomly paired natural proteins, while matching or beating similarly sized models on frozen-embedding linear probes and outperforming an autoregressive pLM almost twice its size. Generated sequences are biomimetic but distinct: amino-acid k-mer, predicted secondary-structure, and predicted-function distributions overlap natural ones with low Jensen-Shannon divergence even though chi-square tests reject identical distributions. When fine-tuned on high-confidence protein-protein interaction pairs as DSM(ppi), the model reconstructs an interacting partner better when the target is provided, and template-guided screening over 100,000 candidates per BenchBB target yields designs with predicted binding affinities above the best known literature binder for multiple targets. The authors explicitly note that all structural, functional, and affinity evaluations used in-silico proxies rather than wet-lab measurements.
Load-bearing premise
The binder-generation claims rest on the premise that the closed-source predictor's ppKd values faithfully rank real binding affinity; if that predictor is miscalibrated or exploitable, only the reconstruction and representation evidence remains.
Editorial extensions
If this is right
- Any MLM-based pLM can be retrained with a few changes to the masking scheme, loss, and head, turning an encoder into a generator without losing representation quality.
- Diffusion-based pLMs show a different sequence-reconstruction scaling: performance stays high at 90 percent masking where ESM2 collapses, so single-forward-pass generation becomes practical.
- A single 650M-parameter model can serve both annotation tasks and de novo generation, potentially replacing separate representation and design models at that scale.
- Target-conditioned fine-tuning on PPI data improves partner reconstruction and produces candidates predicted to bind more strongly than known BindCraft, AlphaProteo, and Adaptyv binders, suggesting template-guided screening as a viable design loop.
Reading between the lines
- Beyond the paper: because the same closed-source predictor both screened the designs and scored the known binders, the 'superior binder' result is only as strong as that predictor's calibration; a wet-lab affinity panel on a few top candidates would be the decisive check.
- Beyond the paper: the diffusion objective is agnostic to supervision source, so a direct next step is to feed structure-aware or experimentally measured fitness into the same scaffold; a testable prediction is that such variants improve wet-lab hit rates over the current in-silico-only versions.
- Beyond the paper: the random remasking strategy was chosen because logit-based selection underperformed locally, but search-based unmasking schedules are an open knob that could push generation quality and diversity beyond what the paper reports.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the Diffusion Sequence Model (DSM), a masked-diffusion training objective applied to ESM2 checkpoints, and claims that one pLM can simultaneously provide high-quality frozen representations and generative protein design. DSM is evaluated on linear-probe representation benchmarks, on sequence reconstruction with mask rates from 5% to 90%, on unconditional generation statistics (amino acid k-mers, predicted secondary structure, predicted Annotation Vocabulary terms), and on a binder-design case study using the BenchBB targets. A PPI-conditioned variant, DSM(ppi), is fine-tuned with LoRA on StringDB pairs. The central reported results are: DSM650 representations are competitive with or better than similarly sized pLMs; DSM reconstructs heavily masked sequences with alignment scores around 0.27; generated sequences match several distributional statistics of natural proteins; and DSM(ppi) produces binders with higher Synteract2-predicted affinity than literature binders on the BenchBB targets.
Significance. If the claims hold, the paper would be significant: a single architecture that reconciles representation quality with controllable generation would be a useful step beyond MLM-only pLMs and autoregressive models. The paper has real strengths: reconstruction comparisons across models use identical masked positions via shared random seeds; the representation probes include both a random-vector negative control and a randomized-transformer control; the PPI fine-tuning includes a no-context control model to show the effect of target conditioning; and code, selected datasets, and weights are promised on GitHub. The main limitation is that the most practically salient claim, superior binder generation, currently rests on a closed-source scorer used both to screen and to evaluate, and on a comparison of the maximum of 100,000 scored candidates to a single literature value. The in-silico nature of all functional and binding evaluations is acknowledged in Section 5, but the missing null model means the binder case study does not yet support the abstract's claim of biologically coherent generation.
major comments (4)
- [Section 3.7, Table S4, Figure S9] The binder screen has no null baseline, so the 'superior predicted binding affinity' claim is not secured. For each target, 100,000 template-masked candidates are generated and scored, and the maximum Synteract2 ppKd is compared against a single literature binder's pKd. Even for a perfectly calibrated oracle, the maximum over many noisy predictions will exceed a single reference value by chance (winner's curse). A necessary control is a non-DSM baseline, for example randomly mutating or shuffling the template sequences and running the identical 100,000-candidate screen, or reporting the full distribution of ppKd values for template-only random perturbations. The Section 5 caveat that binding affinity is an in-silico proxy is honest, but it does not address this missing null model.
- [Section 4.1, Figure 3, Table S1] The distributional evidence for 'biologically plausible' generation is weakened by the use of Translator-predicted Annotation Vocabulary terms as ground truth for function, without a null generative model. Table S1 reports low F1 for several Translator aspects, e.g., BP F1 = 0.23 in the case study and EC F1 = 0.33 on the test set, so the AV-term comparison in Figure 3 inherits substantial predictor noise. Moreover, no baseline such as length-matched random sequences, a Markov model of order 1-3, or an MLM-only ESM2 is compared. Without such a null, the low JS between natural and generated k-mer distributions does not establish that DSM specifically captures function-related statistics rather than trivial amino acid composition.
- [Section 3.2, Section 4.3, Figure 5] The representation improvement of DSM over its ESM2 base is potentially confounded by continued pretraining on OMG prot50. Since DSM starts from ESM2 checkpoints and then trains for 100,000 steps on 207 million additional sequences, the correct control for isolating the diffusion objective is an ESM2 model trained with standard MLM on the same OMG data for the same number of steps and the same compute budget. Without this control, the gains in Figure 5 could be due to additional data exposure and fine-tuning rather than to masked diffusion. The paper currently compares only against frozen, un-continued ESM2 checkpoints.
- [Section 4.2, Figure 4] The reconstruction claim that ASc ~0.27 is 'over four standard deviations above the mean of randomly paired natural protein sequences' needs more support. The random-pair null distribution should be defined for length-matched sequences and the standard deviation should be reported, ideally with confidence intervals for the ASc at each mask rate. As written, the single-number comparison is hard to interpret, and the reader cannot tell whether the reported DSM/DPLM advantage over ESM2 is statistically meaningful at the individual mask rates.
minor comments (6)
- [Section 3.3.1, Section 4.1] The secondary structure distributions in Figure 2 are produced by a predictor whose base model is ESMC-600 (Supplemental Figure S7), not by DSM itself; this should be stated explicitly in the main text near Figure 2 so readers do not attribute these predictions to DSM.
- [Figure 5, Supplemental Figure S4] The normalized score in Figure 5 can amplify small raw F1 differences; the raw scores in Figure S4 show that DSM650 is not uniformly ahead of DPLM and ESM2 baselines across tasks, so the headline 'match or exceed' should be supported by reporting per-task raw scores with error bars or a significance test.
- [Section 3.2] The 'soft-logit cap' and the modified language modeling head are described only verbally; an equation or pseudocode would be needed for reproducibility.
- [Section 3.7, Table S4] The column labeled 'Template ppKd error' is not defined in the main text or table caption; please state what quantity it measures and how it is computed.
- [Section 2.3] For the ASc formula, the paper should clarify the handling of possibly negative Needleman-Wunsch scores and report the standard deviation of the random-sequence null distribution, not only the mean.
- [Figure 6 and Figure S7 captions] There are typos in the captions: 'Adapytv' should be 'Adaptyv' and 'None-class' should be 'nine-class'.
Circularity Check
The binder-generation claim reduces to a self-scored maximum over 100,000 Synteract2 predictions with no null baseline; representation and reconstruction evidence are independent.
-
fitted input called prediction
[Section 3.7, Section 4.4, Table S4, Supplemental A.1.1]
"Target proteins and their newly designed binders were sent to Synteract2 through the Synthyra API to predict their binding affinity Supplemental section A.1.1(56). ... Reported is the average predicted binding affinity (ppKd), success rate (percentage higher than best known binder ppKd)."
Synteract2 is the authors' own closed-source fitted model, and it is both the source of every ppKd and the definition of 'success' (ppKd greater than the known binder's pKd). With 100,000 random maskings of a known strong binder per target and no shuffled-template or random-sequence null control, the maximum Synteract2 score can exceed a single literature reference value purely by selection over a large noisy sample, even if DSM contributes no generative skill. The reported 'superior predicted binding affinity' is therefore a property of the max-over-100k statistic and Synteract2's calibration, not an independent measurement of DSM's generative ability; the binder claim reduces to Synteract2 ranking its own top outputs highly.
full rationale
The representation and reconstruction evaluations are self-contained and not circular: DSM is benchmarked against ESM2, DPLM, and other pLMs on public datasets with controlled linear probes, and the high-corruption reconstruction uses held-out sequences with identical masked positions. The function-distribution comparison relies on the authors' Translator model for annotations, which is a validity and independence concern but not a by-construction reduction. The binder case study, however, is structurally circular: 100,000 template mutants are generated per target and all are scored by Synteract2, the authors' closed-source PPI model; the 'success rate' is the fraction with Synteract2 ppKd above one literature pKd. Because the maximum of a large sample of noisy predictions exceeds a single reference value by selection even under a null model of random mutation, the reported superiority does not establish that DSM's generation improved on the template. The paper's Section 5 caveat that binding affinity is an in-silico proxy and wet-lab validation is future work is honest but does not supply the missing null control, so the most practically significant generation claim remains unsecured.
Assumptions & free parameters
free parameters (5)
- sampling temperature T =
1.0
- tokens unmasked per step s =
5
- LoRA rank and alpha =
r=8, alpha=32, dropout=0.01
- PPI dataset confidence threshold =
StringDB combined score >= 900
- CD-hit sequence identity cutoff =
90%
assumptions (8)
- domain assumption Pretrained ESM2 weights and architecture provide a sufficiently strong prior that masked diffusion fine-tuning preserves representation quality while adding generative ability.
- standard math The LLaDA masked-diffusion objective with 1/t loss is Fisher-consistent and scales favorably, making it suitable for protein sequences.
- domain assumption Normalized Needleman-Wunsch alignment with BLOSUM62 (ASc) is a valid measure of biological sequence similarity.
- domain assumption Jensen-Shannon divergence and chi-square tests on k-mer and annotation-term frequency distributions capture whether generated proteins are biomimetic.
- ad hoc to paper Synteract2 predicted pKd is a valid proxy for true binding affinity.
- ad hoc to paper Translator-predicted Annotation Vocabulary terms are accurate enough to compare generated and natural proteins.
- domain assumption StringDB entries with combined score at least 900 represent true high-confidence protein-protein interactions.
- domain assumption Random transformer embeddings provide a meaningful baseline for correlation based on basic sequence homology.
Cite this review
Pith. "Pith review of Diffusion Sequence Models for Enhanced Protein Representation and Generation." pith.science (2026). https://pith.science/paper/CWQSZ5GB
@misc{pith2026250608293,
author = {Pith},
title = {Pith review of: Diffusion Sequence Models for Enhanced Protein Representation and Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/CWQSZ5GB}},
note = {Machine review of arXiv:2506.08293}
}
read the original abstract
Proteins are fundamental to biology, executing diverse functions through complex physicochemical interactions, and they hold transformative potential across medicine, materials science, and environmental applications. Protein Language Models (pLMs) aim to unlock insights from the vast space of unlabeled protein sequences by learning rich, semantic representations from primary sequences via masked language modeling. However, these models typically exhibit limited generative capacity. In this work, we introduce the Diffusion Sequence Model (DSM), a novel pLM trained with masked diffusion to enable both high-quality representation learning and generative protein design. DSM builds upon the ESM2 architecture by incorporating a masked forward diffusion process inspired by the LLaDA framework. After training, DSM is capable of generating diverse, biomimetic sequences that align with expected amino acid compositions, secondary structures, and predicted functions, even with 90\% token corruption. Furthermore, DSM's learned representations match or exceed those of similarly sized pLMs on downstream tasks. We also introduce DSM(ppi), a variant fine-tuned to generate protein binders by attending to target sequences. We demonstrate DSM(ppi)'s effectiveness on the challenging Bench-tested Binder Benchmark (BenchBB), where both DSM and DSM(ppi) produce candidates with superior predicted binding affinity compared to known binders. Our results establish masked diffusion as a powerful paradigm for unifying protein representation and generation in a single framework.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
A Survey on Diffusion Language Models
A comprehensive survey of diffusion language models covering taxonomy, training and inference techniques, and comparisons with autoregressive models.
Reference graph
Works this paper leans on
-
[1]
Xukang Shen, Siliang Song, Chuan Li, and Jianzhi Zhang. Synonymous mutations in representative yeast genes are mostly strongly non-neutral.Nature, 606(7915): 725–731, June 2022. ISSN 1476-4687. doi:10.1038/s41586-022-04823-w
- [2]
- [3]
-
[4]
The UniProt Consortium. Uniprot: the universal protein knowledgebase in 2023.Nucleic Acids Research, 51(D1):D523–D531, January 2023. ISSN 0305-1048. doi: 10.1093/nar/gkac1052
-
[5]
Craig E Jones, Alfred L Brown, and Ute Baumann. Estimating the annotation error rate of curated GO database sequence annotations.BMC Bioinformatics, 8:170, 2007. ISSN 1471-2105. doi:10.1186/1471-2105-8-170
-
[6]
Sabrina de Azevedo Silveira, Raquel Cardoso de Melo-Minardi, Carlos Henrique da Silveira, Marcelo Matos Santoro, and Wagner Meira Jr. Enzymap: Exploiting protein annotation for modeling and predicting ec number changes in uniprot/swiss-prot.PLOS ONE, 9(2):e89162, February 2014. ISSN 1932-6203. doi:10.1371/journal.pone. 0089162
-
[7]
Braun Markus, Gruber Christian C, Krassnigg Andreas, Kummer Arkadij, Lutz Stefan, Oberdorfer Gustav, Siirola Elina, and Snajdrova Radka. Accelerating biocatalysis discovery with machine learning: A paradigm shift in enzyme engineering, discovery, and design.ACS Catalysis, 13(21):14454–14469, November 2023. doi: 10.1021/acscatal.3c03417
-
[8]
Catherine S. Millar-Haskell, Allyson M. Dang, and Jason P . Gleghorn. Coupling synthetic biology and programmable materials to construct complex tissue ecosystems. MRS communications, 9(2):421–432, June 2019. ISSN 2159-6859. doi:10.1557/mrc.2019.69
Show all 76 references
-
[9]
Edwards, Elias McComb, Jason P
Vonetta L. Edwards, Elias McComb, Jason P . Gleghorn, Larry Forney, Patrik M. Bavoil, and Jacques Ravel. Three-dimensional models of the cervicovaginal epithelia to study host-microbiome interactions and sexually transmitted infections.Pathogens and Disease, 80(1):ftac026, Aug...
2022 doi
-
[10]
DeLong, Maitham A
John P . DeLong, Maitham A. Al-Sammak, Zeina T. Al-Ameeli, David D. Dunigan, Kyle F . Edwards, Jeffry J. Fuhrmann, Jason P . Gleghorn, Hanqun Li, Kona Haramoto, Amelia O. Harrison, Marcia F . Marston, Ryan M. Moore, Shawn W. Polson, Barbra D. Ferrell, Miranda E. Salsbery, Chri...
2022
-
[11]
Nelson, N’Dea Irvin-Choy, Matthew K
Katherine M. Nelson, N’Dea Irvin-Choy, Matthew K. Hoffman, Jason P . Gleghorn, and Emily S. Day. Diseases and conditions that impact maternal and fetal health and the potential for nanomedicine therapies.Advanced Drug Delivery Reviews, 170:425–438, March 2021. ISSN 1872-8294. ...
2021 doi
-
[12]
Gilbert and Jason P
Rachel M. Gilbert and Jason P . Gleghorn. Connecting clinical, environmental, and genetic factors point to an essential role for vitamin a signaling in the pathogenesis of congenital diaphragmatic hernia.American Journal of Physiology. Lung Cellular and Molecular Physiology, 3...
2023
-
[13]
Gleghorn
Yuanjun Shen and Jason P . Gleghorn. Class iii phosphatidylinositol-3 kinase/vacuolar protein sorting 34 in cardiovascular health and disease.Journal of Cardiovascular Translational Research, 18(2):392–407, April 2025. ISSN 1937-5395. doi:10.1007/s12265-024-10581-z
2025 doi
-
[14]
Roberts, Logan Hallee, and Chi Keung Lam
Richard J. Roberts, Logan Hallee, and Chi Keung Lam. The potential of hsp90 in targeting pathological pathways in cardiac diseases.J Pers Med, 11(12):1373, 2021. ISSN 2075-4426. doi:10.3390/jpm11121373
2021 doi
-
[15]
The potential of gamma secretase as a therapeutic target for cardiac diseases.J Pers Med, 11(12):1294, 2021
Sujoita Sen, Logan Hallee, and Chi Keung Lam. The potential of gamma secretase as a therapeutic target for cardiac diseases.J Pers Med, 11(12):1294, 2021. doi: 10.3390/jpm11121294. Number: 12 Publisher: Multidisciplinary Digital Publishing Institute
2021 doi
-
[16]
Enrique Herrero Acero, Doris Ribitsch, Anita Dellacher, Sabine Zitzenbacher, Annemarie Marold, Georg Steinkellner, Karl Gruber, Helmut Schwab, and Georg M. Guebitz. Surface engineering of a cutinase from thermobifida cellulosilytica for improved polyester hydrolysis.Biotechnol...
2013 doi
-
[17]
The catalytic role of rubisco for in situ co2 recycling in escherichia coli.Frontiers in Bioengineering and Biotechnology, 8, November 2020
Ju-Jiun Pang, Jong-Shik Shin, and Si-Yu Li. The catalytic role of rubisco for in situ co2 recycling in escherichia coli.Frontiers in Bioengineering and Biotechnology, 8, November 2020. ISSN 2296-4185. doi:10.3389/fbioe.2020.543807
2020
-
[18]
Protein-based biological materials: Molecular design and artificial production.Chemical Reviews, 123(5):2049–2111, March 2023
Ali Miserez, Jing Yu, and Pezhman Mohammadi. Protein-based biological materials: Molecular design and artificial production.Chemical Reviews, 123(5):2049–2111, March 2023. ISSN 0009-2665. doi:10.1021/acs.chemrev.2c00621
2023 doi
-
[19]
ProtTrans: Toward understanding the language of life through self-supervised learning.IEEE Trans Pattern Anal Mach Intell, 44 (10):7112–7127, 2022
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, Debsindhu Bhowmik, and Burkhard Rost. ProtTrans: Toward understanding the language of life through self-supervised learning...
2022
-
[20]
Gleghorn
Logan Hallee, Nikolaos Rafailidis, and Jason P . Gleghorn. cdsBERT - extending protein language models with codon awareness.bioRxiv, 2023. doi:10.1101/2023.09.15. 558027. Pages: 2023.09.15.558027 Section: New Results
2023 doi
-
[21]
CodonBERT: Large language models for mRNA design and optimization.bioRxiv, 2023
Sizhen Li, Saeed Moayedpour, Ruijiang Li, Michael Bailey, Saleh Riahi, Milad Miladi, Jacob Miner, Dinghai Zheng, Jun Wang, Akshay Balsubramani, Khang Tran, Minnie Zacharia, Monica Wu, Xiaobo Gu, Ryan Clinton, Carla Asquith, Joseph Skalesk, Lianne Boeglin, Sudha Chivukula, Anus...
2023 doi
-
[22]
CodonBERT: a BERT-based architecture tailored for codon optimization using the cross-attention mechanism.Bioinformatics, page btae330, 2024
Zilin Ren, Lili Jiang, Y axin Di, Dufei Zhang, Jianli Gong, Jianting Gong, Qiwei Jiang, Zhiguo Fu, Pingping Sun, Bo Zhou, and Ming Ni. CodonBERT: a BERT-based architecture tailored for codon optimization using the cross-attention mechanism.Bioinformatics, page btae330, 2024. I...
2024 doi
-
[23]
Durrant, Armin W
Eric Nguyen, Michael Poli, Matthew G. Durrant, Armin W. Thomas, Brian Kang, Jeremy Sullivan, Madelena Y . Ng, Ashley Lewis, Aman Patel, Aaron Lou, Stefano Ermon, Stephen A. Baccus, Tina Hernandez-Boussard, Christopher Ré, Patrick D. Hsu, and Brian L. Hie. Sequence modeling and...
2024 doi
-
[24]
Esm all-atom: Multi-scale protein language model for unified molecular modeling.arXiv, June 2024
Kangjie Zheng, Siyu Long, Tianyu Lu, Junwei Y ang, Xinyu Dai, Ming Zhang, Zaiqing Nie, Wei-Ying Ma, and Hao Zhou. Esm all-atom: Multi-scale protein language model for unified molecular modeling.arXiv, June 2024. doi:10.48550/arXiv.2403.12995. arXiv:2403.12995 [q-bio]
-
[25]
Durrant, Jerome Ku, Michael Poli, Greg Brockman, Daniel Chang, Gabriel A
Garyk Brixi, Matthew G. Durrant, Jerome Ku, Michael Poli, Greg Brockman, Daniel Chang, Gabriel A. Gonzalez, Samuel H. King, David B. Li, Aditi T. Merchant, Mohsen Naghipourfar, Eric Nguyen, Chiara Ricci-Tam, David W. Romero, Gwanggyu Sun, Ali Taghibakshi, Anton Vorontsov, Bran...
2025
-
[26]
Lawrence Zitnick, Jerry Ma, and Rob Fergus
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the ...
2021 doi
-
[27]
Amini, Yisong Yue, Kevin K
Francesca-Zhoufan Li, Ava P . Amini, Yisong Yue, Kevin K. Y ang, and Alex X. Lu. Feature reuse and scaling: Understanding transfer learning with protein language models. bioRxiv, 2024. doi:10.1101/2024.02.05.578959
2024 doi
-
[28]
Ankh: Optimized protein language model unlocks general-purpose modelling.arXiv, January 2023
Ahmed Elnaggar, Hazem Essam, Wafaa Salah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Charlotte Rochereau, and Burkhard Rost. Ankh: Optimized protein language model unlocks general-purpose modelling.arXiv, January 2023. doi:10.48550/arXiv.2301.06568. arXiv:2301.06568 [cs]
-
[30]
Vernon, Almer van der Sloot, Benjamin Schulz, Sarath Chandar, and Christopher James Langmead
Quentin Fournier, Robert M. Vernon, Almer van der Sloot, Benjamin Schulz, Sarath Chandar, and Christopher James Langmead. Protein language models: Is scaling necessary?bioRxiv, page 2024.09.23.614603, September 2024. doi:10.1101/2024.09.23.614603. Halleeet al.| arXiv | Septemb...
2024 doi
-
[31]
Siddiqui, Albert Gu, Michael D
Krithik Ramesh, Sameed M. Siddiqui, Albert Gu, Michael D. Mitzenmacher, and Pardis C. Sabeti. Lyra: An efficient and expressive subquadratic architecture for modeling biological sequences.arXiv, March 2025. doi:10.48550/arXiv.2503.16351. arXiv:2503.16351 [cs]
2025 doi
- [32]
-
[33]
Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379 (6637):1123–1130, March 2023
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Y aniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, and Alexander Rives. Evolutionary-scale prediction of atomic-level ...
2023 doi
-
[34]
Calia, Chance A
Tudor-Stefan Cotet, Igor Krawczuk, Filippo Stocco, Noelia Ferruz, Anthony Gitter, Y oichi Kurumida, Lucas de Almeida Machado, Francesco Paesani, Cianna N. Calia, Chance A. Challacombe, Nikhil Haas, Ahmad Qamar, Bruno E. Correia, Martin Pacesa, Lennart Nickel, Kartic Subr, Leon...
2025
-
[35]
R. A. Fisher and Edward John Russell. On the mathematical foundations of theoretical statistics.Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 222(594-604):309–368, 1922. doi:10.1098/rsta.1922.0009
1922
-
[36]
The omg dataset: An open metagenomic corpus for mixed-modality genomic language modeling.bioRxiv, August 2024
Andre Cornman, Jacob West-Roberts, Antonio Pedro Camargo, Simon Roux, Martin Beracochea, Milot Mirdita, Sergey Ovchinnikov, and Yunha Hwang. The omg dataset: An open metagenomic corpus for mixed-modality genomic language modeling.bioRxiv, August 2024. doi:10.1101/2024.08.14.60...
2024 doi
- [37]
- [38]
-
[39]
Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.arXiv, March 2023
Pengcheng He, Jianfeng Gao, and Weizhu Chen. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.arXiv, March 2023. doi:10.48550/arXiv.2111.09543. arXiv:2111.09543 [cs]
-
[40]
Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference.arXiv, December 2024
Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. Smarter, better, faster, longer: A modern bidirectional e...
- [41]
- [42]
- [43]
-
[44]
Training compute-optimal protein language models.bioRxiv, 2024
Xingyi Cheng, Bo Chen, Pan Li, Jing Gong, Jie Tang, and Le Song. Training compute-optimal protein language models.bioRxiv, 2024. doi:10.1101/2024.06.06.597716. Pages: 2024.06.06.597716 Section: New Results
2024 doi
-
[45]
Esm cambrian: Revealing the mysteries of proteins with unsupervised learning.Evolutionary Scale Blog, December 2024
ESM Team. Esm cambrian: Revealing the mysteries of proteins with unsupervised learning.Evolutionary Scale Blog, December 2024. Evolutionary Scale, blog post
2024
-
[46]
Gleghorn
Logan Hallee, David Bichara, and Jason P . Gleghorn. Esm++: Efficient and hugging face compatible versions of the esm cambrian models.Hugging Face, 2024. doi: 10.57967/hf/3726
2024 doi
- [47]
- [48]
-
[49]
Roformer: Enhanced transformer with rotary position embedding.arXiv, November 2023
Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.arXiv, November 2023. arXiv:2104.09864 [cs]
2023 arXiv
-
[50]
Hu, Y elong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen
Edward J. Hu, Y elong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv, October 2021. doi:10.48550/arXiv.2106.09685. arXiv:2106.09685 [cs]
-
[51]
Karl Pearson. X. on the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling, July 1900
1900
-
[52]
Divergence measures based on the shannon entropy.IEEE Transactions on Information Theory, 37(1):145–151, January 1991
Jianhua Lin. Divergence measures based on the shannon entropy.IEEE Transactions on Information Theory, 37(1):145–151, January 1991
1991
-
[53]
Introduction to information retrieval.Cambridge UP, 2009
Christopher Manning, Prabhakar Raghavan, and Hinrich Schuetze. Introduction to information retrieval.Cambridge UP, 2009
2009
-
[54]
Gable, Tao Fang, Nadezhda T
Damian Szklarczyk, Rebecca Kirsch, Mikaela Koutrouli, Katerina Nastou, Farrokh Mehryary, Radja Hachilif, Annika L. Gable, Tao Fang, Nadezhda T. Doncheva, Sampo Pyysalo, Peer Bork, Lars J. Jensen, and Christian von Mering. The string database in 2023: protein-protein associatio...
2023 doi
-
[55]
Cd-hit: a fast program for clustering and comparing large sets of protein or nucleotide sequences.Bioinformatics (Oxford, England), 22 (13):1658–1659, July 2006
Weizhong Li and Adam Godzik. Cd-hit: a fast program for clustering and comparing large sets of protein or nucleotide sequences.Bioinformatics (Oxford, England), 22 (13):1658–1659, July 2006. ISSN 1367-4803. doi:10.1093/bioinformatics/btl158
2006 doi
-
[56]
Gleghorn
Logan Hallee and Jason P . Gleghorn. Protein-protein interaction prediction is achievable with large language models.bioRxiv, page 2023.06.07.544109, June 2023. doi: 10.1101/2023.06.07.544109
2023 doi
-
[57]
Ballard, Joshua Bambrick, Sebastian W
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J. Ballard, Joshua Bambrick, Sebastian W. Bodenstein, David A. Evans, Chia-Chun Hung, Michael O’Neill, David Reiman, Kathryn Tunyasuvunakool, Zachar...
2024
-
[58]
SaProt: Protein language modeling with structure-aware vocabulary.bioRxiv, 2023
Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. SaProt: Protein language modeling with structure-aware vocabulary.bioRxiv, 2023. doi: 10.1101/2023.10.01.560349. Pages: 2023.10.01.560349 Section: New Results
2023 doi
-
[59]
Protokens: Probabilistic vocabulary for compact and informative encodings of all-atom protein structures.bioRxiv, page 2023.11.27.568722, July 2024
Xiaohan Lin, Zhenyu Chen, Y anheng Li, Zicheng Ma, Chuanliu Fan, Ziqiang Cao, Shihao Feng, Yi Qin Gao, and Jun Zhang. Protokens: Probabilistic vocabulary for compact and informative encodings of all-atom protein structures.bioRxiv, page 2023.11.27.568722, July 2024. doi:10.110...
2023 doi
-
[60]
Khomtchouk
Logan Hallee and Bohdan B. Khomtchouk. Machine learning classifiers predict key genomic and evolutionary traits across the kingdoms of life.Scientific Reports, 13(1): 2088, 2023. ISSN 2045-2322. doi:10.1038/s41598-023-28965-7
2023 doi
-
[61]
Gleghorn, and Bohdan B
Logan Hallee, Rohan Kapur, Arjun Patel, Jason P . Gleghorn, and Bohdan B. Khomtchouk. Contrastive learning and mixture of experts enables precise vector embeddings in biological databases.Scientific Reports, 15(1):14953, April 2025. ISSN 2045-2322. doi:10.1038/s41598-025-98185-8
2025 doi
-
[62]
Carter, Samuel Curtis, Claudia Emerson, Jeffrey Gray, Ian C
Sarah R. Carter, Samuel Curtis, Claudia Emerson, Jeffrey Gray, Ian C. Haydon, Andrew Hebbeler, Claire Qureshi, Nicholas Randolph, Alexander Rives, and Lynda Stuart. Community values, guiding principles, and commitments for the responsible development of ai for protein design, ...
2024
-
[63]
Michael L. Waskom. seaborn: statistical data visualization.Journal of Open Source Software, 6(60):3021, 2021. doi:10.21105/joss.03021
2021 doi
- [64]
-
[65]
Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R
Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors.arXiv, July 2012. doi:10.48550/arXiv.1207.0580. arXiv:1207.0580 [cs]
-
[66]
J. A. Cuff and G. J. Barton. Evaluation and improvement of multiple sequence methods for protein secondary structure prediction.Proteins, 34(4):508–519, March 1999. ISSN 0887-3585. doi:10.1002/(sici)1097-0134(19990301)34:4<508::aid-prot10>3.0.co;2-4
1999 doi
-
[67]
Sixty-five years of the long march in protein secondary structure prediction: the final stretch?Briefings in Bioinformatics, 19(3):482–494, May 2018
Yuedong Y ang, Jianzhao Gao, Jihua Wang, Rhys Heffernan, Jack Hanson, Kuldip Paliwal, and Y aoqi Zhou. Sixty-five years of the long march in protein secondary structure prediction: the final stretch?Briefings in Bioinformatics, 19(3):482–494, May 2018. ISSN 1477-4054. doi:10.1...
2018 doi
-
[68]
Ghamary, Laura Vinué, Brahm J
Martin Pacesa, Lennart Nickel, Joseph Schmidt, Ekaterina Pyatova, Christian Schellhaas, Lucas Kissling, Ana Alcaraz-Serna, Y ehlin Cho, Kourosh H. Ghamary, Laura Vinué, Brahm J. Y achnin, Andrew M. Wollacott, Stephen Buckley, Sandrine Georgeon, Casper A. Goverde, Georgios N. H...
2024 doi
-
[69]
Chu, Harshnira Patani, Amy E
Vinicius Zambaldi, David La, Alexander E. Chu, Harshnira Patani, Amy E. Danson, Tristan O. C. Kwan, Thomas Frerix, Rosalia G. Schneider, David Saxton, Ashok Thillaisundaram, Zachary Wu, Isabel Moraes, Oskar Lange, Eliseo Papa, Gabriella Stanton, Victor Martin, Sukhdeep Singh, ...
-
[70]
Hutter, Fabian M
Iwan Zimmermann, Pascal Egloff, Cedric A.J. Hutter, Fabian M. Arnold, Peter Stohler, Nicolas Bocquet, Melanie N. Hug, Sylwia Huber, Martin Siegrist, Lisa Hetemann, Jennifer Gera, Samira Gmür, Peter Spies, Daniel Gygax, Eric R. Geertsma, Roger J.P . Dawson, and Markus A. Seeger...
2018 doi
-
[71]
C. D. Suraweera, M. G. Hinds, and M. Crystal Kvansakul. Crystal structures of epstein–barr virus bcl-2 homolog bhrf1 bound to bid and puma bh3 motif peptides.Viruses, 14(2222), 2022. ISSN 10. doi:10.3390/v14102222
2022 doi
-
[72]
Desbien, John W
Anthony L. Desbien, John W. Kappler, and Philippa Marrack. The epstein–barr virus bcl-2 homolog, bhrf1, blocks apoptosis by binding to a limited amount of bim. Proceedings of the National Academy of Sciences, 106:5663–5668, 2009. ISSN 14. doi:10.1073/pnas.0901036106
2009 doi
-
[73]
De novo design protein binders for mbp and gst tags.Biochemical and Biophysical Research Communications, 748:151322, February 2025
Jinlong Zhou, Yue Xiao, Quian Tang, Yunjun Y an, Dongqi Liu, and Houjin Zhang. De novo design protein binders for mbp and gst tags.Biochemical and Biophysical Research Communications, 748:151322, February 2025. ISSN 0006-291X. doi:10.1016/j.bbrc.2025.151322. Publisher: Elsevier
2025
-
[74]
Hicks, Agnidipta Ghosh, Tristin A
Wei Y ang, Derrick R. Hicks, Agnidipta Ghosh, Tristin A. Schwartze, Brian Conventry, Inna Goreshnik, Aza Allen, Samer F . Halabiya, Chan Johng Kim, Cynthia S. Hinck, David S. Lee, Asim K. Bera, Zhe Li, Yujia Wang, Thomas Schlichthaerle, Longxing Cao, Buwei Huang, Sarah Garrett...
2025
-
[75]
E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.Biometrics, 44(3):837–845, September 1988. ISSN 0006-341X
1988
-
[76]
Xu Sun and Weichao Xu. Fast implementation of delong’s algorithm for comparing the areas under correlated receiver operating characteristic curves.IEEE Signal Processing Letters, 21(11):1389–1393, November 2014. ISSN 1558-2361. doi:10.1109/LSP.2014.2337313. Halleeet al.| arXiv...
2014
-
[2024]
doi:10.1101/2024.07.30.605924
2024 doi
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.