Pith. sign in

REVIEW 4 major objections 5 minor 66 references

AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AMix-1's central claim is that a protein foundation model can be made test-time scalable by wrapping any scoreable fitness function in an evolutionary loop that proposes, verifies, and refines without retraining.

desk verdict Substantial systems paper with a genuine wet-lab AmeR result, but the test-time scaling headline is not supported by the experiments because the verifier is both selector and outcome metric. read the letter →

arxiv 2507.08920 v4 pith:CNLQR77C submitted 2025-07-11 q-bio.BM cs.AI

Changze Lv , Jiang Zhou , Siyu Long , Lihao Wang , Jiangtao Feng , Dongyu Xue , Yu Pei , Hao Wang
show 17 more authors
Zherui Zhang Yuchen Cai Zhiqiang Gao Ziyuan Ma Jiakai Hu Chaochen Gao Jingjing Gong Yuxuan Song Shuyi Zhang Xiaoqing Zheng Deyi Xiong Lei Bai Wanli Ouyang Ya-Qin Zhang Wei-Ying Ma Bowen Zhou Hao Zhou
This is my paper
classification q-bio.BMcs.AI
keywords proteinfoundationmodelBayesianflownetworkstest-timescalingdirectedevolutionin-contextlearninglawsdesignmultiplesequencealignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AMix-1 is a protein foundation model built on Bayesian flow networks, and the paper argues that it brings the large-language-model recipe of scaling laws, emergent abilities, in-context learning, and test-time scaling to protein design. The central claim is that once the model is trained, any scoreable objective—structural similarity, optimal temperature, pH, enzyme class, or catalytic activity—can be plugged into an evolutionary loop that repeatedly proposes variants, verifies them, and feeds the best back as a new profile prompt. The paper reports that this loop produces monotonic gains as the verification budget grows across six in silico directed-evolution tasks, without any parameter updates. A wet-lab experiment on the AmeR repressor supports the design pipeline, yielding a variant with up to 50 times the wild-type activity. If correct, this means protein engineers can trade verification compute for fitness and reuse one frozen model across many design goals.

What carries the argument

The load-bearing object is the profile-prompted proposal loop. A multiple sequence alignment is compressed into a position-wise frequency profile $P$, a categorical distribution over amino acids at each position; the frozen AMix-1 generator conditions on $P$ and samples candidate sequences. An external verifier $R(\cdot)$ scores each candidate, the top-$k$ are selected, a new profile is built from those winners, and the cycle repeats. What makes the loop scalable is that the proposal distribution is reshaped by prompt updating rather than gradient updates, so any verifier can be plugged in without retraining.

What would settle it

Run the EvoAMix-1 loop with an actual wet-lab activity assay as the verifier on a protein family such as AmeR, comparing measured top candidates against random-mutation baselines across increasing verification budgets. If higher budgets do not raise measured activity, or if the surrogate scores (ESMFold TM-score, CLEAN, Seq2Topt, Seq2pHopt, CLIPZyme) disagree with assay outcomes, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's discovery has four connected parts. First, the cross-entropy loss of AMix-1 follows a power law in compute and obeys a compute-optimal parametric scaling law, but only at intermediate noise levels; at extreme noise levels scaling breaks. Second, structural understanding emerges without structural supervision: as cross-entropy drops below a threshold, predicted structural confidence and structural similarity metrics jump sharply. Third, converting a multiple sequence alignment into a position-wise amino-acid frequency profile makes the frozen model a unified designer that preserves fold and function. Fourth, the evolutionary test-time scaling algorithm EvoAMix-1—propose, score with an external verifier, select the top-k, rebuild the profile from those winners, repeat—yields increasing top-k scores with increasing verifier calls across six in silico tasks, and the in-context design path is validated in the wet lab by the 50-fold AmeR variant.

Load-bearing premise

The test-time scaling claim rests on the verifier being a truthful fitness oracle, since the same in silico score used for selection is also the reported outcome metric; the paper notes in its conclusion that the loop has so far only been validated on simulated tasks using in silico metrics, lacking experimental assays.

Editorial extensions

If this is right

  • The predictive scaling law lets practitioners forecast loss and anticipate the onset of structural emergence before training large models, at least within the moderate noise regime.
  • MSA-profile conditioning turns one frozen model into a unified designer for structure preservation, enzyme class retention, and catalytic activity without task-specific fine-tuning.
  • EvoAMix-1 delivers monotonic design improvement as verification calls increase, matching or beating strong directed-evolution baselines in five of six tasks and avoiding the plateaus caused by restricted mutation schemes.
  • The wet-lab AmeR result shows that the in-context design path can produce functional proteins, with the best AMix-1 variant reaching roughly 50-fold repression over the wild type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I would test the loop with an actual wet-lab assay as the verifier, since the paper explicitly frames that as future work; if it works, the method becomes a general lab-in-the-loop engine for protein engineering.
  • The proposal-distribution perspective suggests a testable prediction: the entropy of the profile prompt should tighten over rounds, analogous to reward-model concentration in reinforcement learning.
  • If the verifier is noisy or biased, selection can amplify that bias; a natural extension is robustness-aware selection that accounts for verifier uncertainty, which the paper does not address.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces AMix-1, a protein foundation model based on Bayesian Flow Networks, and claims a systematic pathway comprising four components: predictive scaling laws, emergent structural abilities, MSA-profile-based in-context learning, and an evolutionary test-time scaling algorithm (EvoAMix-1). The authors pretrain a family of models from 8M to 1.7B parameters, report scaling-law fits for cross-entropy as a function of compute, model size, and token count, analyze emergent structure-related metrics on 20 CAMEO proteins, demonstrate in silico ICL on four design cases, and validate ICL with a wet-lab AmeR experiment claiming up to 50-fold activity improvement. For test-time scaling, they propose a propose-verify-update loop using plug-in verifiers (ESMFold TM-score, CLEAN, Seq2Topt, Seq2pHopt, CLIPZyme) and report monotonic performance gains with increasing verification budget across six in silico directed-evolution tasks. The paper explicitly acknowledges in Section 7 that the test-time scaling algorithm has only been validated on simulated tasks with in silico metrics and lacks experimental assays.

Significance. If the central claims hold, the paper would make a useful contribution: it provides a large pretrained BFN protein model family, a systematic scaling-law analysis specific to Bayesian Flow Networks, a training-free ICL interface based on MSA profiles, and a verification-budget-aware evolutionary loop that could, in principle, be coupled to wet-lab assays. The wet-lab AmeR result is a genuine asset, as are the authors' attempts to make the TTS loop task-agnostic through plug-in verifiers. The scaling-law analysis is presented with explicit functional forms and MRE metrics, and the paper ships detailed algorithmic descriptions and hyperparameter tables. However, several load-bearing empirical claims are currently supported by within-run curve fits, visual inspection of curves without error bars or statistical tests, and a test-time scaling evaluation in which the verifier used for selection is identical to the metric used for reporting; these issues currently limit the strength of the conclusions and require substantial revision.

major comments (4)
  1. [§6.3, Algorithm 3, Section 7] The central test-time scaling claim (Key Result ④) rests on a circular evaluation: the verifier R(·) is used both to select top-k candidates (Algorithm 3, lines 6 and 9) and to report the outcome metric in Figure 12. Because the reported curve is the running top-k over the accumulated history H, any stochastic proposer, including one with no adaptive mechanism, produces a monotonically non-decreasing score in expectation purely by extreme-value statistics. Section F.1 shows that random mutation achieves comparable or better reward than EvoAMix-1 on the Optimal Temperature and Optimal pH tasks, with only the pLDDT side panels differentiating the methods; pLDDT is not the reported scaling metric. Section 7 explicitly states that the test-time scaling algorithm 'has so far only been validated on simulated tasks using in silico metrics.' To support the paper's scaling claim, the authors should report a held-out fitness oracle that is not used in selection (e.g., a different structure predictor, an independent functional assay predictor, or wet-lab validation), and should additionally report per-round mean or median rewards rather than only the cumulative top-k. Without such evidence, the claim that AMix-1's design ability 'is scalable in test time' is not established.
  2. [§3.2, Eq. (6), Figure 3] The predictive scaling law in Eq. (6) is presented as a forecast, but the validation consists of red 'validation point' stars from the same training runs used to fit the curve. This is interpolation over the compute trajectory of each model, not prediction of held-out compute budgets or unseen model sizes. The claim in Key Result ① that cross-entropy is 'precisely predictable against compute, data quantity and model size' is therefore stronger than the evidence. In addition, the unscalable noise levels (α = 0.08 and 0.64 in Figure 13) are excluded after observing the poor fits, and the paper's own Table 2 marks α = 0.02 and 0.64 as unscalable with A = 0.000. The authors should either reframe the scaling-law contribution as a characterization of scaling behavior under moderate noise levels, or provide genuine out-of-distribution validation, e.g., fitting on smaller models and predicting the 1.7B model's loss.
  3. [§4.2, Figure 5] The emergent-ability analysis relies on visual inspection of curves over 20 CAMEO proteins, without error bars, confidence intervals, or statistical tests. The claim of a 'sharp increase' in structural metrics once cross-entropy drops below a threshold is not quantified, and no threshold estimation procedure or significance test is provided. Since Key Result ② is that structural understanding 'progressively emerges' with a 'consistent empirical alignment' to cross-entropy, the authors should report per-protein variability and a quantitative measure of emergence, such as a piecewise-linear fit with a breakpoint and a comparison against a null model of smooth improvement.
  4. [§5.3, Figure 9, Section G] The wet-lab AmeR result, which supports the headline 'up to 50× activity increase,' is presented as a single fold-repression value per method with no error bars, no replicate counts, and no statistical significance test. Figure 9 shows one bar per method. The text in Section G states that 'results demonstrated' the improvement but does not report the number of biological or technical replicates, the spread across the 40 candidates, or the distribution of fold-repression values. Given that the abstract and Key Result ③ rest on this comparison, the authors should provide replicate-level data, confidence intervals, and a statistical comparison against the EvoAI and single-mutant baselines.
minor comments (5)
  1. [Appendix A title] The appendix title 'Pretrainig Algorithm' contains a typo; it should read 'Pretraining Algorithm.'
  2. [§1, paragraph 2] The word 'successfuly' in the introduction is a typo; it should be 'successfully.'
  3. [§5.2, case studies] The in silico ICL case studies in Figure 7 are each based on a single protein family (or a small number of examples); reporting the number of generated sequences, the variance of TM-score/CLEAN/CLIPZyme metrics, and a comparison to a random-sequence or profile-free baseline would strengthen the generalization claims.
  4. [Reference [65]] Reference [65] is cited as Seq2pHopt but the URL points to a GitHub repository named 'Seq2Topt'; please verify the citation and correct it if the wrong tool is referenced.
  5. [Table 2] The 'Scalable' column in Table 2 uses check/cross marks with no explicit threshold; specify the criterion (e.g., A > 0 and Valid MRE below a cutoff) used to classify a noise level as scalable.

Circularity Check

1 steps flagged · score 6.0 of 10

Test-time scaling gains are reported on the same verifier used for top-k selection, making part of the improvement a construction.

  1. fitted input called prediction [Section 6.1, Algorithm 3; Section 6.3, Figure 12; Appendix E]
    "Algorithm 3: 'H ← H ∪(x_i, s_i); X ← Top-k(H); ... return Top-k(H).' Section 6.3: 'We report performance with respect to the verification budget, i.e., the number of verifier calls.' Appendix E: 'we compute the cumulative top-k mean score using the shared verifier.'"

    The test-time scaling 'prediction' is evaluated with the same function that drives selection. Algorithm 3 scores every candidate with R, stores it in H, selects top-k from H, and finally returns top-k(H); Figure 12 then plots mean top-5 task score under the same R as a function of verifier calls. Because H accumulates all generated sequences and the plotted value is the running top-k, the curve is a running maximum over an increasing sample: any stochastic proposer, including random mutation, yields monotonically non-decreasing expected R scores. The ablation in F.1 confirms that on Optimal Temperature and Optimal pH, random mutation matches or exceeds EvoAMix-1 on R, with only pLDDT separating the methods.

full rationale

The central test-time scaling claim (Key Result 4) reduces in part to a construction: the reported improvement is the running top-k of the same verifier used for selection, so monotonic gains are partly a statistical artifact independent of the proposer. This is mild-to-moderate circularity (score 6). However, the paper has independent grounding elsewhere: the wet-lab AmeR result (Section 5.3) validates the ICL design on real fitness, and the scaling law (Section 3) is fitted and validated on held-out token splits, not on the same data used for the claim. The self-citation to ProfileBFN (ref 53) is for methodology, not load-bearing for the scaling claim. The absence of a held-out fitness oracle, explicitly acknowledged in Section 7, is what prevents the test-time scaling result from being a fully self-contained prediction.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claims depend on fitted scaling-law constants, a post-hoc choice of scalable noise levels, assumptive surrogate verifiers, and the ProfileBFN-derived assumption that MSA profiles are sufficient prompts. These are the main costs paid upstream.

free parameters (5)
  • E0, a, b (compute scaling law coefficients) = e.g., E0=0.5870, a=6.6137, b=0.1803 for 1.7B at alpha=0.16; values vary by model and noise level
    Fitted to training FLOPs vs cross-entropy per model size per noise level (Eq. 6, Figure 3); used as the 'predictive' scaling law.
  • E, A, B, n, d (parametric scaling law) = for alpha=0.16: E=0.4638, A=0.2570, B=4.6740, n=0.0380, d=0.3332 (Table 2)
    Fitted across (N,D) configurations using Huber loss and L-BFGS (Eq. 7); basis for the compute-optimal frontier and the claim of predictability.
  • beta1 (final noise level) = 1.0
    Hand-chosen training hyperparameter in beta(t)=beta1*t^2 (Eq. 2, Section 3.1); scaling and emergence behavior depend on it.
  • Scalable noise levels alpha in {0.16, 0.32} = selected post hoc
    alpha=0.02, 0.08, 0.64 are labeled unscalable and excluded from the compute-optimal analysis (Sections 3.2-3.3, Appendix D); this post-hoc selection shapes the scaling-law conclusions.
  • EvoAMix-1 hyperparameters (N=50, k=5, T=10, inference steps 50-250, initial noise 0.90-0.99) = specified in Table 6
    Chosen experimental configuration, not fitted to data; the test-time scaling performance curves depend on these.
assumptions (6)
  • ad hoc to paper Power-law forms L(F)=E0+e^a F^b and L(N,D)=E+A N^-n + B D^-d are the correct functional forms.
    Borrowed from Kaplan and Chinchilla scaling literature (refs 19, 26) without derivation for BFN; validity is asserted via in-sample fit quality.
  • domain assumption ESMFold pLDDT and TM-score are faithful proxies for foldability and structural consistency.
    Used for all emergent-ability and structure-design evaluations (Sections 4, 5.2, 6.3); structural claims stand or fall on ESMFold's accuracy.
  • domain assumption CLEAN, CLIPZyme, Seq2Topt, and Seq2pHopt predictions are meaningful functional or biophysical fitness signals.
    Used both as verifiers in EvoAMix-1 and as outcome metrics in the TTS benchmarks (Table 6, Section 6.3); surrogate fidelity is not independently validated for these proteins.
  • domain assumption A per-position frequency profile P derived from an MSA is a sufficient prompt to guide generation to the target family (Eq. 9).
    Central to the ICL and TTS methods; assumes evolutionary constraints are captured by position-wise amino acid frequencies, a simplification inherited from ProfileBFN.
  • domain assumption Cross-entropy at a fixed noise level alpha is a causal 'ruler' for downstream generation performance.
    The emergent-ability analysis maps cross-entropy to pLDDT and TM-score; this correlation is assumed to imply control of generation quality.
  • domain assumption UniRef50 sequences after cropping to 1024 residues represent the protein design distribution.
    Training data choice (Section 3.1); cropping might truncate long proteins and reduce coverage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model." pith.science (2026). https://pith.science/paper/CNLQR77C

@misc{pith2026250708920,
  author       = {Pith},
  title        = {Pith review of: AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CNLQR77C}},
  note         = {Machine review of arXiv:2507.08920}
}
abstract

We introduce AMix-1, a powerful protein foundation model built on Bayesian Flow Networks and empowered by a systematic training methodology, encompassing pretraining scaling laws, emergent capability analysis, in-context learning mechanism, and test-time scaling algorithm. To guarantee robust scalability, we establish a predictive scaling law and reveal the progressive emergence of structural understanding via loss perspective, culminating in a strong 1.7-billion model. Building on this foundation, we devise a multiple sequence alignment (MSA)-based in-context learning strategy to unify protein design into a general framework, where AMix-1 recognizes deep evolutionary signals among MSAs and consistently generates structurally and functionally coherent proteins. This framework enables the successful design of a dramatically improved AmeR variant with an up to $50\times$ activity increase over its wild type. Pushing the boundaries of protein engineering, we further empower AMix-1 with an evolutionary test-time scaling algorithm for in silico directed evolution that delivers substantial, scalable performance gains as verification budgets are intensified, laying the groundwork for next-generation lab-in-the-loop protein design.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 26 canonical work pages

  1. [1]

    Improved protein structure prediction using potentials from deep learning.Nature, 577(7792):706–710, 2020

    Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Žídek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning.Nature, 577(7792):706–710, 2020

    2020

  2. [2]

    Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models.Nucleic acids research, 50(D1):D439–D444, 2022

    Mihaly Varadi, Stephen Anyango, Mandar Deshpande, Sreenath Nair, Cindy Natassia, Galabina Yordanova, David Yuan, Oana Stroe, Gemma Wood, Agata Laydon, et al. Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models.Nucleic acids research, 50(D1):D439–D444, 2022

    2022

  3. [3]

    Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, pages 1–3, 2024

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, pages 1–3, 2024

    2024

  4. [4]

    Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the National Academy of Sciences, 118(15):e2016239118, 2021

    2021

  5. [5]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    2023

  6. [6]

    Simulating 500 million years of evolution with a language model

    Tomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model. bioRxiv, pages 2024–07, 2024

    2024

  7. [7]

    Language models of protein sequences at the scale of evolution enable accurate structure prediction.bioRxiv, 2022

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction.bioRxiv, 2022

    2022

  8. [8]

    High-resolution de novo structure prediction from primary sequence.BioRxiv, pages 2022–07, 2022

    Ruidong Wu, Fan Ding, Rui Wang, Rui Shen, Xiwen Zhang, Shitong Luo, Chenpeng Su, Zuofan Wu, Qi Xie, Bonnie Berger, et al. High-resolution de novo structure prediction from primary sequence.BioRxiv, pages 2022–07, 2022

    2022

Show all 66 references
  1. [9]

    A method for multiple-sequence-alignment-free protein structure prediction using a protein language model.Nature Machine Intelligence, 5(10):1087–1096, October 2023

    Xiaomin Fang, Fan Wang, Lihang Liu, Jingzhou He, Dayong Lin, Yingfei Xiang, Kunrui Zhu, Xiaonan Zhang, Hua Wu, Hui Li, and Le Song. A method for multiple-sequence-alignment-free protein structure prediction using a protein language model.Nature Machine Intelligence, 5(10):1087...

    2023 doi

  2. [10]

    Learning inverse folding from millions of predicted structures

    Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors,Proc...

    2022

  3. [11]

    Accurate prediction of protein function using statistics-informed graph networks.Nature Communications, 15(1):6601, 2024

    Yaan J Jang, Qi-Qi Qin, Si-Yu Huang, Arun T John Peter, Xue-Ming Ding, and Benoît Kornmann. Accurate prediction of protein function using statistics-informed graph networks.Nature Communications, 15(1):6601, 2024

    2024

  4. [12]

    Genome-scale annotation of protein binding sites via language model and geometric deep learning.Elife, 13:RP93695, 2024

    Qianmu Yuan, Chong Tian, and Yuedong Yang. Genome-scale annotation of protein binding sites via language model and geometric deep learning.Elife, 13:RP93695, 2024

    2024

  5. [13]

    Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval

    Pascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena-Hurtado, Aidan N Gomez, Debora Marks, and Yarin Gal. Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. InInternational Conference on Machine Learning, pages 16990–...

    2022

  6. [14]

    Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023

    Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023

    2023

  7. [15]

    ESM-effect: An effective and efficient fine-tuning framework towards accurate prediction of mutation’s functional effect

    Moritz Glaser and Johannes Brägelmann. ESM-effect: An effective and efficient fine-tuning framework towards accurate prediction of mutation’s functional effect. InICLR 2025 Workshop on Machine Learning for Genomics Explorations, 2025. URLhttps://openreview.net/forum?id=kY3zOTMtjU. 22

    2025

  8. [16]

    Forcegen: End-to-end de novo protein generation based on nonlinear mechanical unfolding responses using a language diffusion model.Science Advances, 10(6):eadl4000, 2024

    Bo Ni, David L Kaplan, and Markus J Buehler. Forcegen: End-to-end de novo protein generation based on nonlinear mechanical unfolding responses using a language diffusion model.Science Advances, 10(6):eadl4000, 2024

    2024

  9. [17]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, TomHenighan, RewonChild, AdityaRamesh, DanielM.Ziegler, JeffreyWu,...

    2020 arXiv

  10. [18]

    Glm-130b: An open bilingual pre-trained model, 2023

    Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Peng Zhang, Yuxiao Dong, and Jie Tang. Glm-130b: An open bilingual pre-trained model, 2023. URLht...

    2023 arXiv

  11. [20]

    Zhang, Han Bao, Hanwei 23 Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei L...

    2025 arXiv

  12. [21]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025 arXiv

  13. [22]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  14. [23]

    Gemini: A family of highly capable multimodal models, 2025

    Gemini Team and et.al. Gemini: A family of highly capable multimodal models, 2025. URLhttps://arxiv.org/ abs/2312.11805. 24

    2025 arXiv

  15. [24]

    Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou

    Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically, 2017. URLhttps://arxiv.org/abs/1712.00409

    2017 arXiv

  16. [25]

    Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks,...

    2022 arXiv

  17. [26]

    Rae, Oriol Vinyals, and Laurent Sifre

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osin...

    2022 arXiv

  18. [27]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024. URLhttps://arxiv.org/abs/2408.03314

    2024 arXiv

  19. [28]

    Le, Christopher Ré, and Azalia Mirhoseini

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, 2024. URLhttps: //arxiv.org/abs/2407.21787

    2024 arXiv

  20. [29]

    Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models, 2025

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models, 2025. URLhttps://arxiv.org/ abs/2408.00724

    2025 arXiv

  21. [30]

    Bayesian flow networks.arXiv preprint arXiv:2308.07037, 2023

    Alex Graves, Rupesh Kumar Srivastava, Timothy Atkinson, and Faustino Gomez. Bayesian flow networks.arXiv preprint arXiv:2308.07037, 2023

    2023 arXiv

  22. [31]

    Molcraft: Structure-based drug design in continuous parameter space

    Yanru Qu, Keyue Qiu, Yuxuan Song, Jingjing Gong, Jiawei Han, Mingyue Zheng, Hao Zhou, and Wei-Ying Ma. Molcraft: Structure-based drug design in continuous parameter space. InInternational conference on machine learning, 2024

    2024

  23. [32]

    Uniref: comprehensive and non-redundant uniprot reference clusters.Bioinformatics, 23(10):1282–1288, 2007

    Baris E Suzek, Hongzhan Huang, Peter McGarvey, Raja Mazumder, and Cathy H Wu. Uniref: comprehensive and non-redundant uniprot reference clusters.Bioinformatics, 23(10):1282–1288, 2007

    2007

  24. [33]

    Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31 (6):926–932, 2015

    Baris E Suzek, Yuqi Wang, Hongzhan Huang, Peter B McGarvey, Cathy H Wu, and UniProt Consortium. Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31 (6):926–932, 2015

    2015

  25. [34]

    Protein generation with evolutionary diffusion: sequence is all you need.BioRxiv, pages 2023–09, 2023

    Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Alex X Lu, Nicolo Fusi, Ava P Amini, and Kevin K Yang. Protein generation with evolutionary diffusion: sequence is all you need.BioRxiv, pages 2023–09, 2023

    2023

  26. [35]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  27. [36]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  28. [37]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2018

    2018

  29. [38]

    Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1–113, 2023

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1–113, 2023. 25

    2023

  30. [39]

    Scaling instruction-finetuned language models.Journal of Machine Learning Research, 25(70):1–53, 2024

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models.Journal of Machine Learning Research, 25(70):1–53, 2024

    2024

  31. [40]

    Scaling laws for diffusion transformers.arXiv preprint arXiv:2410.08184, 2024

    Zhengyang Liang, Hao He, Ceyuan Yang, and Bo Dai. Scaling laws for diffusion transformers.arXiv preprint arXiv:2410.08184, 2024

    2024

  32. [41]

    Scaling properties of diffusion models for perceptual tasks

    Rahul Ravishankar, Zeeshan Patel, Jathushan Rajasegaran, and Jitendra Malik. Scaling properties of diffusion models for perceptual tasks. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 12945–12954, 2025

    2025

  33. [42]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  34. [43]

    Robust estimation of a location parameter

    Peter J Huber. Robust estimation of a location parameter. InBreakthroughs in statistics: Methodology and distribution, pages 492–518. Springer, 1992

    1992

  35. [44]

    Updating quasi-newton matrices with limited storage.Mathematics of computation, 35(151): 773–782, 1980

    Jorge Nocedal. Updating quasi-newton matrices with limited storage.Mathematics of computation, 35(151): 773–782, 1980

    1980

  36. [45]

    Emergent abilities of large language models.arXiv preprint arXiv:2206.07682, 2022

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models.arXiv preprint arXiv:2206.07682, 2022

    2022 arXiv

  37. [46]

    Understanding emergent abilities of language models from the loss perspective.arXiv preprint arXiv:2403.15796, 2024

    Zhengxiao Du, Aohan Zeng, Yuxiao Dong, and Jie Tang. Understanding emergent abilities of language models from the loss perspective.arXiv preprint arXiv:2403.15796, 2024

    2024 arXiv

  38. [47]

    Principles that govern the folding of protein chains.Science, 181(4096):223–230, 1973

    Christian B Anfinsen. Principles that govern the folding of protein chains.Science, 181(4096):223–230, 1973

    1973

  39. [48]

    Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    2021

  40. [49]

    Xavier Robin, Juergen Haas, Rafal Gumienny, Anna Smolinski, Gerardo Tauriello, and Torsten Schwede. Contin- uous automated model evaluation (cameo)—perspectives on the future of fully automated evaluation of structure prediction methods.Proteins: Structure, Function, and Bioin...

    1977

  41. [50]

    Diffusion language models are versatile protein learners.arXiv preprint arXiv:2402.18567, 2024

    Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Diffusion language models are versatile protein learners.arXiv preprint arXiv:2402.18567, 2024

    2024 arXiv

  42. [51]

    Barrett, Scott Cameron, Bora Guloglu, Matthew Greenig, Louis Robinson, Alex Graves, Liviu Copoiu, and Alexandre Laterre

    Timothy Atkinson, Thomas D. Barrett, Scott Cameron, Bora Guloglu, Matthew Greenig, Louis Robinson, Alex Graves, Liviu Copoiu, and Alexandre Laterre. Protein sequence modelling with bayesian flow networks.bioRxiv,

  43. [52]

    A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2022

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2022

    2022 arXiv

  44. [53]

    Steering protein family design through profile bayesian flow

    Jingjing Gong, Yu Pei, Siyu Long, Yuxuan Song, Zhe Zhang, Wenhao Huang, Ziyao Cao, Shuyi Zhang, Hao Zhou, and Wei-Ying Ma. Steering protein family design through profile bayesian flow. InThe Thirteenth International Conference on Learning Representations, ICLR. OpenReview.net, 2025

    2025

  45. [54]

    Enzyme function prediction using contrastive learning.Science, 379(6639):1358–1363, 2023

    Tianhao Yu, Haiyang Cui, Jianan Canal Li, Yunan Luo, Guangde Jiang, and Huimin Zhao. Enzyme function prediction using contrastive learning.Science, 379(6639):1358–1363, 2023

    2023

  46. [55]

    Clipzyme: Reaction-conditioned virtual screening of enzymes

    Peter Mikhael, Itamar Chinn, and Regina Barzilay. Clipzyme: Reaction-conditioned virtual screening of enzymes. InForty-first International Conference on Machine Learning, 2024

    2024

  47. [56]

    Genomic mining of prokaryotic repressors for orthogonal logic gates.Nature chemical biology, 10(2):99–105, 2014

    Brynne C Stanton, Alec AK Nielsen, Alvin Tamsir, Kevin Clancy, Todd Peterson, and Christopher A Voigt. Genomic mining of prokaryotic repressors for orthogonal logic gates.Nature chemical biology, 10(2):99–105, 2014

    2014

  48. [57]

    Evoai enables extreme compression and reconstruction of the protein sequence space

    Ziyuan Ma, Wenjie Li, Yunhao Shen, Yunxin Xu, Gengjiang Liu, Jiamin Chang, Zeju Li, Hong Qin, Boxue Tian, Haipeng Gong, et al. Evoai enables extreme compression and reconstruction of the protein sequence space. Nature Methods, 22(1):102–112, 2025. 26

    2025

  49. [58]

    Openai o1 system card, 2024

    OpenAI and et.al. Openai o1 system card, 2024. URLhttps://arxiv.org/abs/2412.16720

    2024 arXiv

  50. [59]

    s1: Simple test-time scaling, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025. URLhttps: //arxiv.org/abs/2501.19393

    2025 arXiv

  51. [60]

    Inference-time scaling for diffusion models beyond scaling denoising steps

    Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09732, 2025

    2025 arXiv

  52. [61]

    Active learning-assisted directed evolution.Nature Communications, 16 (1):714, 2025

    Jason Yang, Ravi G Lal, James C Bowden, Raul Astudillo, Mikhail A Hameedi, Sukhvinder Kaur, Matthew Hill, Yisong Yue, and Frances H Arnold. Active learning-assisted directed evolution.Nature Communications, 16 (1):714, 2025

    2025

  53. [62]

    Rapid in silico directed evolution by a protein language model with evolvepro.Science, page eadr6006, 2024

    Kaiyi Jiang, Zhaoqing Yan, Matteo Di Bernardo, Samantha R Sgrizzi, Lukas Villiger, Alisan Kayabolen, BJ Kim, Josephine K Carscadden, Masahiro Hiraizumi, Hiroshi Nishimasu, et al. Rapid in silico directed evolution by a protein language model with evolvepro.Science, page eadr6006, 2024

    2024

  54. [63]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  55. [64]

    Seq2topt: a sequence-based deep learning predictor of enzyme optimal temperature.Briefings in Bioinformatics, 26(2):bbaf114, 2025

    Sizhe Qiu, Bozhen Hu, Jing Zhao, Weiren Xu, and Aidong Yang. Seq2topt: a sequence-based deep learning predictor of enzyme optimal temperature.Briefings in Bioinformatics, 26(2):bbaf114, 2025

    2025

  56. [65]

    Seq2topt.https://github.com/SizheQiu/Seq2Topt, 2023

    Sizhe Qiu. Seq2topt.https://github.com/SizheQiu/Seq2Topt, 2023. commit ab12cd3, accessed 2025-07-04

    2023

  57. [66]

    fold repression

    Thanh VT Tran and Truong Son Hy. Protein design by directed evolution guided by large language models.IEEE Transactions on Evolutionary Computation, 2024. 27 Appendix A Pretrainig Algorithm A.1 Training The continuous-time discrete Bayesian flow loss, following the ProfileBFN ...

    2024

  58. [2024]

    URL https://www.biorxiv.org/content/early/2024/09/26/2024.09

    doi: 10.1101/2024.09.24.614734. URL https://www.biorxiv.org/content/early/2024/09/26/2024.09. 24.614734

    2024 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.