{"id":"1e27cb12-4ecf-4142-a3b7-83ccf12a07d6","arxiv_id":"2501.10282","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey that categorizes protein language models by the knowledge they learn and reviews their applications, with no new experimental results.","lead":"This paper is a survey of protein language models (pLMs), grouping them by what they learn, from amino acid sequence patterns to structures, functions, and scientific text, and reviewing how these models are used for protein prediction and design. It is a useful orientation map for a fast-moving field, but it introduces no new models or experimental results.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 'effectively generalized' overstates the evidence: most cited pLM applications in §4 require task-specific heads, fine-tuning, or architectural additions, rather than zero-shot generalization.","rationale":"The reader's weakest assumption was coverage/selection bias: the survey may not give a representative map of the field. My concern is related but distinct: even within the set of papers the survey does cite, the evidence supports a weaker claim than the abstract makes. The word 'generalized' in the central claim implies zero-shot or near-zero-shot transfer, but Section 4 shows that most successes require substantial task-specific adaptation. This is an internal-evidence gap, not just a selection issue. It is load-bearing because the survey's main message—that pLMs are a generalizable substrate—depends on this interpretation. The concern does not invalidate the survey's usefulness as a taxonomy; rather, it argues for a more precise claim and for stating the adaptation requirements explicitly. The reader's CONDITIONAL verdict remains appropriate; my analysis provides additional, more specific justification for that verdict, so I leave the verdict unchanged.","tokens_in":49629,"tokens_out":4013,"duration_ms":45432,"concrete_test":"Classify every application cited in Section 4 (including Tables 1–4) into two categories: zero-shot use of a pre-trained pLM versus use requiring task-specific training (fine-tuning, learned prediction head, adapter, or additional architecture). If more than 80% fall into the second category, the abstract's 'effectively generalized' is unsupported and the claim should be revised. As a secondary check, examine three representative zero-shot results (e.g., ProteinGym likelihood-based fitness prediction) in the original papers to see whether they outperform non-pLM baselines and whether the effect is specifically attributable to the pLM.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract) is that pLMs 'skillfully grasp the foundational knowledge of proteins and can be effectively generalized to solve a diversity of sequence-structure-function reasoning problems.' The survey's own evidence in Section 4 does not support the strong reading of 'generalized.' For protein structure prediction, ESMFold, HelixFold-Single, OmegaFold, trRosettaX-Single, RGN2, and IgFold all require a trained folding trunk or prediction head on top of the pLM (Section 4.1, Figure 8). For function prediction, the vast majority of methods use the LM-as-encoder scheme with fine-tuning or a learned classifier; only likelihood-based fitness and mutation-effect prediction (ProteinGym, Brandes et al., Section 4.2.1) are genuinely zero-shot. For protein design, ProGen and ZymCTRL need control tags or fine-tuning for controllable generation (Section 4.3). Thus, the paper conflates 'adaptable via task-specific training' with 'generalization.' This is not a fatal internal contradiction, but it means the strongest claim is broader than the evidence. The survey also does not systematically report negative results or compare pLMs against strong non-pLM baselines, so the reader cannot assess whether pLMs are necessary or superior for these tasks. The claim should be either softened to 'adaptable across tasks' or explicitly scoped to zero-shot settings where such evidence exists.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a survey of protein language models (pLMs) and their applications in computational protein science. It proposes a taxonomy that divides pLMs into sequence-based, structure-and-function-enhanced, and multimodal models; reviews how pLMs are used for structure prediction, function prediction, and protein design; and discusses applications in antibody design, enzyme design, and drug discovery. The Abstract states that pLMs 'skillfully grasp the foundational knowledge of proteins and can be effectively generalized to solve a diversity of sequence-structure-function reasoning problems,' and the survey is organized to support this claim by mapping pLMs to downstream tasks.","tokens_in":49839,"tokens_out":5107,"duration_ms":49800,"significance":"If the survey's claims hold, it would provide a valuable entry point for researchers from both AI and biology backgrounds. The taxonomy is thoughtful and the coverage is unusually broad, including recent models such as ESM-3, DPLM-2, and xTrimoPGLM, as well as practical applications with wet-lab validation. The detailed tables (Tables 1–4) that list corpora, architectures, parameter counts, and pre-training objectives are a useful resource. The paper does not present original machine-checked proofs or reproducible code, but it cites primary sources for most claims, which is appropriate for a survey. The main contribution is organizational rather than experimental, and its usefulness depends on whether the selected literature is representative and whether the claims in the Abstract are calibrated to the evidence.","major_comments":[{"comment":"The central claim that pLMs are 'effectively generalized to solve a diversity of sequence-structure-function reasoning problems' is broader than the evidence presented in Section 4. In Section 4.1, all pLM-based structure prediction methods (ESMFold, HelixFold-Single, OmegaFold, trRosettaX-Single, RGN2, IgFold) require an additional trained folding trunk or prediction head. In Section 4.2, most function prediction methods use the LM-as-encoder scheme with a learned classifier, fine-tuning, or parameter-efficient fine-tuning; only likelihood-based fitness and mutation-effect prediction are described as zero-shot (Section 4.2.1). In Section 4.3, ProGen and ZymCTRL require control tags or fine-tuning for controllable generation. The survey thus conflates 'adaptable via task-specific training' with 'generalization.' I recommend softening the Abstract to 'adaptable across tasks' or explicitly scoping the generalization claim to the zero-shot settings where evidence exists.","section":"Abstract and Section 4"},{"comment":"The survey does not state its literature selection method. No databases searched, search dates, keywords, or inclusion/exclusion criteria are provided. This makes the taxonomy non-reproducible and leaves open the possibility that important pLMs were omitted or that the relative emphasis of topics is not representative. As the survey's central contribution is its categorization, a short methodology paragraph describing the search protocol and time window is needed.","section":"Section 3, first paragraph"},{"comment":"The sentence 'In investigations, OmegaFold, trRosettaX-Single, and RGN2 are all observed to outperform AlphaFold2 and RoseTTAFold on those orphan proteins and de novo designed proteins' is a specific quantitative claim with no citation. Please provide references for these comparisons or remove the claim. As written, it is a load-bearing assertion for the section's argument that pLM-based methods address the limitations of MSA-based approaches.","section":"Section 4.1, single-sequence structure prediction paragraph"},{"comment":"The statement 'the likelihoods inferred from pLMs correlate well with protein fitness [72, 253, 254]' is central to the zero-shot generalization narrative, yet no quantitative evidence is given. The survey should report representative correlation values (e.g., Spearman rho) from ProteinGym or the cited works, or state the range across benchmarks. Without those numbers, the claim is too vague to evaluate.","section":"Section 4.2.1"},{"comment":"The survey seldom reports how pLM-based methods compare with strong non-pLM baselines. For structure prediction, comparisons to AlphaFold2 and RoseTTAFold appear in Section 4.1, but for function prediction (Section 4.2) and protein design (Section 4.3) the text rarely mentions results from BLAST, HMMER, profile-based predictors, or classical machine-learning methods. Without this comparative context, the reader cannot judge whether pLMs are necessary or superior for these tasks. I recommend adding a comparative synthesis from the cited benchmarks (e.g., ProteinGym, FLIP, TAPE) or explicitly softening claims of pLM advantage.","section":"Section 4 (general)"}],"minor_comments":[{"comment":"The caption contains the typo 'Workfolw'; it should read 'Workflow'.","section":"Figure 8 caption"},{"comment":"The table header spells 'Functional' as 'Funcitional', and the scheme abbreviation 'Mull-Model Fine-Tuning' should be 'Full-Model Fine-Tuning'.","section":"Table 4"},{"comment":"The second framework in the caption is labeled 'LM-as-Encoder' but the text and Table 4 use 'LM-as-Predictor'; the caption should be corrected for consistency.","section":"Figure 9 caption"},{"comment":"The model 'paired-IgGen' is abbreviated 'p-IgGen' in the text but 'g-IgGen' in Table 1; the abbreviation should be unified.","section":"Section 3.1.1 and Table 1"},{"comment":"The Mamba architecture is cited via references [96] and [97], which are a survey and a recommendation paper; the original Mamba paper (Gu & Dao, reference [139]) should be cited at first mention.","section":"Section 3.1.1, BiMamba-S and Table 1"},{"comment":"The phrase 'reached the unanimous conclusion of non-optimal' is too strong for two cited studies; a softer formulation such as 'several recent studies have concluded' would be more accurate.","section":"Section 6.5"}],"recommendation":"major_revision","confidential_remarks":"The survey is timely and covers a wide range of recent work, but the Abstract's generalization claim is likely to draw criticism. I recommend the editor ask the authors to add a literature-selection methodology, soften or re-scope the central claim, and add citations for specific performance assertions. The reference list includes many preprints and bioRxiv entries; the authors should verify and update to published versions where available."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a solid, well-organized survey of protein language models, and its taxonomy is genuinely useful. If someone asks me how to get oriented in this fast-moving area, I'd point them to the categorization in Section 3 and the summary tables. The coverage of 2023-2024 models like ESM-3, xTrimoPGLM, and ProTrek is current, and the figures are clear. The authors deserve credit for structuring the field by what the models learn (sequence patterns, structure/function, external languages) rather than just architecture.\n\nThat said, the abstract oversells the results. It says pLMs 'can be effectively generalized to solve a diversity of sequence-structure-function reasoning problems.' But as the body of the survey itself shows, most applications in Section 4 use pLMs as encoders with task-specific prediction heads or fine-tuning. ESMFold and friends need a folding trunk; function prediction mostly uses learned classifiers; ProGen and ZymCTRL require control tags. The genuinely zero-shot cases are mostly likelihood-based fitness prediction. That's an important distinction, and the survey would be more accurate if the abstract said 'adaptable across tasks' rather than 'generalized.'\n\nThe other soft spot is typical for surveys: there is no stated literature search protocol or inclusion criteria, so the claim of comprehensive coverage rests on the authors' judgment. I don't see evidence of major omissions, but the lack of a method makes it hard to know. Some performance claims, like the correlation between pLM likelihood and fitness, are cited without critical re-evaluation, and negative results or comparisons to strong non-pLM baselines are not systematically reported. Minor issues: a few typos like 'Funcitional' in Table 4, and a duplicated sentence, but nothing serious.\n\nOverall, this is a useful orientation map for newcomers and a reasonable reference for researchers. It deserves a serious referee, but I'd encourage the authors to soften the generalization language and acknowledge the survey's selection choices.","headline":"A useful, well-organized survey of pLMs whose abstract overclaims generalization; worth refereeing after revision.","tokens_in":50370,"tokens_out":2442,"would_cite":false,"duration_ms":25671,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Protein language models—LLMs trained on amino acid sequences, structures, and scientific text—can grasp the “grammar” and “semantics” of proteins and be adapted to a broad range of structure prediction, function prediction, and protein…","keywords":["protein language models","large language models","protein structure prediction","protein function prediction","protein design","antibody design","enzyme design","drug discovery"],"falsifier":"A controlled benchmark that runs sequence-only, structure-enhanced, and multimodal pLMs on a fixed set of structure, function, and design tasks would settle the central claim: for example, if SaProt and ProLLaMA do not systematically beat ESM-2 on held-out tasks, the paper's hierarchy of protein knowledge fails, and if no pLM category transfers beyond task-specific models trained from scratch, the generalization story collapses.","tokens_in":49401,"feed_emoji":"🧬","tokens_out":5427,"duration_ms":50543,"temperature":0.7,"pith_summary":"This survey argues that protein language models (pLMs), LLMs trained on amino acid sequences and, in later versions, structures, functions, and text, learn the “grammar” and “semantics” of proteins and can be adapted to a wide range of sequence-structure-function problems. The authors organize pLMs by the kind of protein knowledge they master: sequence patterns, explicit structure and function information, and external scientific languages. They then trace how these models are used for structure prediction, function prediction, and protein design, including wet-lab-validated applications in antibody and enzyme design and drug-target interaction prediction. A sympathetic reader takes the paper as a map of a field in which pLMs are becoming a general substrate for protein science rather than task-specific tools.","feed_headline":"pLMs generalize across protein structure, function, and design","feed_subtitle":"A survey maps how LLMs learn protein grammar and apply it to structure prediction, protein design, and drug discovery.","key_machinery":"The load-bearing mechanism is the pLM itself, treated as a foundation model: a Transformer (or state-space) network pre-trained on large protein corpora so that amino acid tokens behave like words, with “grammar” in residue patterns and “semantics” in encoded structure and function. The paper identifies three technical routes that carry downstream transfer: (1) representation extraction, where frozen or fine-tuned pLM encodings feed prediction heads; (2) likelihood inference, where the model's probability of a mutant versus wild-type sequence is used as a zero-shot fitness score; and (3) prompting and instruction tuning, where a unified decoder answers protein questions or generates sequences under text or function control. Structure tokenization (VQ-VAE-derived tokens such as 3Di) is a notable sub-mechanism that lets 3D structure enter language-model training as discrete tokens.","core_discovery":"The paper's central claim is that protein language models “skillfully grasp the foundational knowledge of proteins and can be effectively generalized to solve a diversity of sequence-structure-function reasoning problems.” The evidence it assembles is a taxonomy: sequence-only pLMs (ESM-2, ProtGPT2, xTrimoPGLM) capture evolutionarily favored amino acid patterns; structure- and function-enhanced pLMs (SaProt, ESM-3) add explicit 3D and annotation knowledge; multimodal pLMs (ProLLaMA, BioT5) bridge protein sequences with natural language and molecule languages. On top of these foundations, the paper reports pLM-based single-sequence structure prediction comparable to MSA-based methods, zero-shot fitness and mutation-effect prediction, text-guided and condition-tagged protein generation, and ChatGPT-like protein question answering. The intended conclusion is that pLMs, not bespoke per-task models, now carry the main line of computational protein science.","pith_inferences":["Beyond the survey's claims, the taxonomy suggests that the next wave of pLMs will blur its three categories: sequence-only models will acquire structure tokens, and multimodal models will absorb molecule and text modalities, so the field's frontier becomes integration cost rather than architecture.","A testable extension of the paper's generalizability claim: on fixed benchmarks, pLM-based methods should dominate task-specific models trained from scratch whenever labeled data are scarce; a controlled comparison across ProteinGym-like tasks would settle this.","Because pLM likelihood is used as a proxy for evolutionary plausibility, species bias in training databases may leak into fitness predictions; adjusting training data composition is a direct extension of the redesign workflow the paper describes.","The survey's coverage assumption can be checked by re-running its categorization against a systematic search of recent pLMs; if major models fall outside the three categories, the proposed map needs revision."],"forward_implications":["Single-sequence structure prediction methods such as ESMFold can replace slow MSA searches for many proteins, making structure inference practical for orphan and fast-evolving proteins.","Zero-shot mutation-effect scoring by pLMs gives experimental labs a cheap first pass at fitness landscapes before deep mutational scanning.","Function prediction moves from many task-specific models to unified question-answering systems that answer property, annotation, site, and interaction questions with one model.","Conditional and text-guided pLMs extend protein design beyond redesign, generating de novo sequences, antibodies targeting new variants, and enzymes with improved stability.","Structure- and function-enhanced pLMs, including multi-track models like ESM-3, suggest that sequence, structure, and function can be treated as interchangeable token tracks in one generative model."],"supporting_citations":[{"why":"Supplies ESM-2 and ESMFold, the central evidence that a single-sequence pLM can predict atomic-level structure and replace MSA search.","marker":"[25]"},{"why":"ESM-1b establishes the foundational result that residue-residue contacts emerge from large-scale unsupervised protein sequence modeling.","marker":"[91]"},{"why":"ProtTrans shows that pLM representations transfer to downstream tasks and compete with MSA-based methods.","marker":"[56]"},{"why":"AlphaFold2 is the MSA-based baseline that pLM-based single-sequence structure prediction methods are compared against.","marker":"[8]"},{"why":"ESM-MSA-1b introduces MSA-aware pLMs and demonstrates co-evolution modeling within a language-model framework.","marker":"[28]"},{"why":"ESM-3 provides the multi-track sequence-structure-function token model used for de novo design of a distant fluorescent protein.","marker":"[30]"},{"why":"ProGen shows control-tag conditional generation of functional protein sequences across diverse families.","marker":"[35]"},{"why":"ProteinGym grounds the zero-shot fitness and mutation-effect prediction claims with large-scale benchmarks.","marker":"[160]"},{"why":"SaProt exemplifies structure-enhanced pLMs that integrate 3D structure tokens into the protein vocabulary.","marker":"[29]"}],"fun_headline_variants":["Protein LLMs master sequence, structure, and function","LLMs decode protein grammar for prediction and design","One protein LLM replaces many task-specific models","LLMs crack the protein code for structure and design"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey's usefulness rests on the assumption that its chosen models and papers give a representative map of the field; no search protocol or inclusion criteria are stated, so the taxonomy could miss important pLMs or misweight the landscape.","fun_headline_variants_meta":{"raw":{"variants":["Protein LLMs master sequence, structure, and function","LLMs decode protein grammar for prediction and design","One protein LLM replaces many task-specific models","LLMs crack the protein code for structure and design"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001095,"raw_usage":{"total_tokens":4610,"prompt_tokens":1023,"completion_tokens":3587,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":3526}},"tokens_in":639,"tokens_out":3587,"duration_ms":28823,"temperature":1.0,"reasoning_tokens":3526,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:15:54.154638+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled benchmark that runs sequence-only, structure-enhanced, and multimodal pLMs on a fixed set of structure, function, and design tasks would settle the central claim: for example, if SaProt and ProLLaMA do not systematically beat ESM-2 on held-out tasks, the paper's hierarchy of protein knowledge fails, and if no pLM category transfers beyond task-specific models trained from scratch, the generalization story collapses.","supporting_citations":[],"review_version":1}