REVIEW 3 major objections 5 minor 39 references
Leveraging Large Language Models for enzymatic reaction prediction and characterization
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Fine-tuned Llama-3.1 models, trained with LoRA adapters, can predict enzyme classes, forward synthesis products, and retrosynthetic substrates from SMILES strings, with the 70B model reaching 91.7% accuracy on the first EC digit.
desk verdict A clean, honestly reported empirical benchmark of LoRA-fine-tuned Llama-3.1 on enzyme tasks, held back from acceptance by an uncontrolled multitask data-volume confound. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery has two parts. First, parameter-efficient fine-tuning via Low-Rank Adaptation (LoRA): the pretrained Llama-3.1 weights stay frozen and only small low-rank update matrices are learned, so the model gains task-specific behavior while the base model's general capabilities are preserved. Second, a multitask setup: the three task datasets are merged and one model is trained to emit EC numbers, products, and substrates, so each prediction can draw on shared enzymatic knowledge. All inputs and outputs are represented as text, with SMILES strings for molecules and dot-separated EC digits for enzymes, turning each task into a next-token generation problem. The data preparation, which groups reactions sharing the same product-EC or substrate-EC pair and rotates groups across tasks and splits, is what makes the single-task and multitask comparisons fair.
What would settle it
Run the same fine-tuning pipeline on a test set of enzyme reactions published after Llama-3.1's training data cutoff, or otherwise verifiably absent from public web corpora. If EC1 accuracy on that set collapses toward the 29.6% zero-shot baseline while accuracy on the original test set stays near 91.7%, the central claim of learned biochemical knowledge is refuted.
Extended reading notes
Core claim
The paper's central claim is that fine-tuned Llama-3.1 models capture enough biochemical knowledge to predict enzyme classes and, to a lesser degree, enzymatic reaction products and substrates. On the BRENDA subset, the 70B model reaches 91.7% macro-averaged accuracy for the first EC digit and 49.2% for three-digit EC numbers, and it generates valid SMILES in nearly all cases. Multitask fine-tuning on all three tasks simultaneously improves forward-synthesis match accuracy from 25.9% to 33.8% and retrosynthesis from 13.9% to 19.2%, which the authors attribute to shared enzymatic information. They also report that fine-tuning with as few as 200 training samples nearly doubles zero-shot EC1 accuracy, from 29.6% to 55.3%. In their reading, the residual gap to specialized baselines is a data-scale and coverage issue rather than a sign that the approach is unsound.
Load-bearing premise
The load-bearing premise is that the test reactions are genuinely new to the model; because the public BRENDA and ECREACT data may have appeared in Llama-3.1's pretraining, the reported accuracies could reflect memorization or format familiarity rather than learned biochemistry.
Editorial extensions
If this is right
- Fine-tuned 70B Llama-3.1 predicts the correct enzyme main class in 91.7% of test reactions, compared with 29.6% for zero-shot prompting of the same model.
- Multitask training is the strongest lever for synthesis tasks: it adds 7.9 percentage points for forward synthesis and 5.3 for retrosynthesis over single-task fine-tuning.
- With only 200 training examples, fine-tuning still reaches 55.3% EC1 accuracy, suggesting practical value in low-data biocatalysis settings.
- Fine-tuning on one enzyme task improves the model's zero-shot performance on the other two; for example, EC-only fine-tuning lifts forward-synthesis match accuracy to 12.9%.
- The model rarely produces invalid or unparseable SMILES under 5% for both synthesis tasks, so its errors are mostly chemically meaningful alternatives rather than grammar failures.
Reading between the lines
- If the pretraining-leakage concern is resolved and the multitask gains persist on truly unseen reactions, fine-tuned open LLMs could serve as cheap pre-screening tools for enzyme-substrate pairs, feeding candidate reactions to more expensive specialized predictors.
- The pattern that retrosynthesis benefits less than forward synthesis from multitask learning suggests that substrate prediction may need more diverse chemistry or explicit reaction templates; a testable extension is to add enzyme sequence or mechanism information to the prompt.
- A direct next experiment is a temporal split: train only on reactions recorded before a cutoff date and test on later ones. If accuracy holds, the biochemical generalization is real; if it collapses, the reported numbers would need recalibration.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper fine-tunes Llama-3.1 8B and 70B models with LoRA adapters on a BRENDA subset of the ECREACT dataset for three enzymatic tasks: EC number prediction, forward synthesis, and retrosynthesis. The authors compare single-task (ST) and multitask (MT) fine-tuning, evaluate low-data regimes and different LoRA configurations, and report that the 70B model reaches 91.7% EC1 accuracy and that multitask learning improves forward-synthesis match accuracy by 7.9 percentage points and retrosynthesis by 5.3 points over single-task models (Table 3). The paper concludes that fine-tuned LLMs capture biochemical knowledge and that multitask learning leverages shared enzymatic information. The authors also acknowledge limitations including possible pretraining contamination, dataset size constraints, and computational costs.
Significance. If the claims hold, the paper provides a useful, reproducible demonstration that open-weight LLMs adapted with PEFT can perform enzyme-related prediction tasks, with strengths including leakage-aware data splitting (grouping reactions by shared product/EC or substrate/EC pairs), evaluation over three seeds, and release of code and data. The honest limitations section is a positive feature. However, the most distinctive claim—that multitask learning yields synergy through shared biochemical knowledge—is currently confounded by training-set size, and the acknowledged pretraining contamination risk limits the interpretation of the absolute accuracy numbers. The paper is a solid empirical study, but the central multitask claim needs a matched-volume control before it can be accepted as stated.
major comments (3)
- [3.2, Tables 3 and 4] The multitask gains attributed to 'shared enzymatic information' are confounded by training-set size: the MT model is trained on the merged three ST datasets (~5400 examples), whereas each ST model is trained on ~1800 examples (Table 4). The +7.9 and +5.3 percentage-point improvements in Table 3 are therefore equally consistent with a pure data-volume effect. Please add a matched-volume control—for example, an ST model trained on a comparable number of examples drawn from the same union of tasks, or an MT model trained on a downsampled union—or explicitly rephrase the abstract and conclusions to stop claiming that the gains arise from leveraging shared enzymatic information.
- [3.5, Limitations] The acknowledged possibility that Llama-3.1's pretraining corpus includes BRENDA/ECREACT reactions means the absolute accuracies (e.g., 91.7% EC1 in Table 1 and 33.8% FS Match in Table 3) are not interpretable as biochemical generalization from fine-tuning alone. Because the paper's central claim is that fine-tuned LLMs 'capture biochemical knowledge,' this is a load-bearing issue, not merely a caveat. Please provide a contamination analysis (e.g., probing whether test reactions are memorized, or evaluating on temporally held-out or newly curated reactions) or substantially soften the generalization claim.
- [3.3 and Appendix 5.6] The XGBoost comparisons for forward and retrosynthesis are not apples-to-apples: XGBoost is evaluated on exact 256-bit fingerprint matching while the LLM is evaluated on canonical SMILES matching. This makes the Section 3.3 claim that 'our models outperform the XGBoost model' in the low-data regime difficult to interpret. Since the low-data section uses this comparison to argue that fine-tuning adds value, the evaluation protocols should be aligned (e.g., evaluate both methods on the same exact-match definition) or the claim should be qualified accordingly.
minor comments (5)
- [Section 3.1] There are several typos, including 'the the' in the first paragraph, 'uncorrect' instead of 'incorrect,' and 'settup' instead of 'setup'; please proofread.
- [Table 3] Table 3 does not report standard deviations or confidence intervals for the ST and MT values, despite Section 2.3 stating that all results are averaged over N=3 experiments with standard deviations reported where applicable; please add them or clarify why they are omitted.
- [Figure 6 caption] The caption says the model is given 'reactants and substrates,' but the EC prediction task is defined in Section 2.1 as predicting EC numbers from 'substrates and products'; the caption should say 'reactants and products.'
- [References [12] and [13]] References [12] and [13] both list the same title 'Improving language understanding by generative pre-training'; the second entry appears to be mis-titled and should be corrected.
- [Section 2.4] The phrase 'we explore two new LoRA setups' is confusing because these are not novel architectural proposals; consider rewording to 'two additional LoRA configurations.'
Circularity Check
No circularity: this is an empirical fine-tuning study with held-out test evaluation and no load-bearing self-citation or by-construction reduction.
full rationale
This is an empirical fine-tuning study, not a derivation. Reported metrics (EC accuracy, FS/RS match rates) are computed on a withheld 30% test split with group-level leakage control, and no parameter is fitted to the test set. The multitask claim is an experimental comparison, not a quantity equal to its own input: the MT model is trained on the merged ST datasets and evaluated on the same held-out groups, so the +7.9 and +5.3 point gains are measured outcomes, even though the absence of a data-volume-matched control means the 'shared enzymatic information' interpretation is not uniquely forced. That confound is a validity and experimental-design concern, not circularity. The acknowledged pretraining-contamination possibility (Section 3.5) is likewise a data-leakage threat, not a logical circularity. There are no load-bearing self-citations, no uniqueness theorems imported from the authors, and no ansatz smuggled in via citation. The paper is self-contained against external SOTA benchmarks, so the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (1)
- LoRA hyperparameters =
r=16, alpha=32, lr=0.002
assumptions (4)
- domain assumption EC numbers are adequate functional labels for enzymatic reactions
- domain assumption The BRENDA subset of ECREACT is accurately annotated
- domain assumption SMILES strings capture molecular identity for the tasks
- domain assumption The test split was not seen during Llama-3.1 pretraining
Cite this review
Pith. "Pith review of Leveraging Large Language Models for enzymatic reaction prediction and characterization." pith.science (2026). https://pith.science/paper/XYGLGOPH
@misc{pith2026250505616,
author = {Pith},
title = {Pith review of: Leveraging Large Language Models for enzymatic reaction prediction and characterization},
year = {2026},
howpublished = {\url{https://pith.science/paper/XYGLGOPH}},
note = {Machine review of arXiv:2505.05616}
}
read the original abstract
Predicting enzymatic reactions is crucial for applications in biocatalysis, metabolic engineering, and drug discovery, yet it remains a complex and resource-intensive task. Large Language Models (LLMs) have recently demonstrated remarkable success in various scientific domains, e.g., through their ability to generalize knowledge, reason over complex structures, and leverage in-context learning strategies. In this study, we systematically evaluate the capability of LLMs, particularly the Llama-3.1 family (8B and 70B), across three core biochemical tasks: Enzyme Commission number prediction, forward synthesis, and retrosynthesis. We compare single-task and multitask learning strategies, employing parameter-efficient fine-tuning via LoRA adapters. Additionally, we assess performance across different data regimes to explore their adaptability in low-data settings. Our results demonstrate that fine-tuned LLMs capture biochemical knowledge, with multitask learning enhancing forward- and retrosynthesis predictions by leveraging shared enzymatic information. We also identify key limitations, for example challenges in hierarchical EC classification schemes, highlighting areas for further improvement in LLM-driven biochemical modeling.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Birgit Wiltschi, Tomislav Cernava, Alexander Dennig, Meritxell Galindo Casas, Martina Geier, Steffen Gruber, and et al. Haberbauer, Marianne. Enzymes revolutionize the bioproduction of value-added compounds: From enzyme discovery to special applications. Biotechnology Advances 40: 107520, 2020
work page 2020
-
[2]
A. R. Alcántara, P. Domínguez de María, J. A. Littlechild, M. Schürmann, R. A. Sheldon, and R. Wohlgemuth. Biocatalysis as key to sustainable industrial chemistry. ChemSusChem, vol. 15, no. 9, p. e202102709, 2022
work page 2022
-
[3]
R. A. Sheldon. Green chemistry and biocatalysis: Engineering a sustainable future. Catalysis Today, vol. 431, p. 114571, 2024
work page 2024
-
[4]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. arXiv, https://doi.org/10.48550/arXiv.1706.03762, 2017. 13 A PREPRINT - MAY 12, 2025
-
[5]
P. Schwaller, T. Laino, T. Gaudin, P. Bolgar, C. Bekas, and A. A. Lee. Molecular transformer - a model for uncertainty-calibrated chemical reaction prediction. ACS Cent. Sci., vol. 5, no. 9, pp. 1572–1583, 2019
work page 2019
-
[6]
G. Pesciullesi, P. Schwaller, T. Laino, and J.-L. Reymond. Transfer learning enables the molecular transformer to predict regio- and stereoselective reactions on carbohydrates. Nat Commun, vol. 11, no. 1, p. 4874, 2020
work page 2020
- [7]
-
[8]
D. Kreutter, P. Schwaller, and J.-L. Reymond. Predicting enzymatic reactions with a molecular transformer.Chem. Sci., vol. 12, no. 25, pp. 8648–8659, 2021
work page 2021
Show all 39 references
-
[9]
Biocatalysed synthesis planning using data-driven learning
Daniel Probst, Matteo Manica, Yves Gaetan Nana Teukam, Alessandro Castrogiovanni, Federico Paratore, and Teodoro Laino. Biocatalysed synthesis planning using data-driven learning. Nature Communications 13, no. 1, 2022
2022
-
[10]
W. Qian, X. Wang, Y . Kang, P. Pan, T. Hou, and C.-Y . Hsieh. A general model for predicting enzyme functions based on enzymatic reactions. Journal of Cheminformatics, vol. 16, no. 1, p. 38, 2024
2024
-
[11]
Y . G. Nana Teukam, L. Kwate Dassi, M. Manica, D. Probst, P. Schwaller, and T. Laino. Language models can identify enzymatic binding sites in protein sequences. Computational and Structural Biotechnology Journal, vol. 23, pp. 1929–1937, 2024
1929
-
[12]
Radford and K
A. Radford and K. Narasimhan. Improving language understanding by generative pre-training. Semantic Scholar, 2018
2018
-
[13]
Radford, J
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Improving language understanding by generative pre-training. Semantic Scholar, 2019
2019
-
[14]
A. D. White et al. Assessment of chemistry knowledge in large language models that generate code. Digital Discovery, vol. 2, no. 2, pp. 368–376, 2023
2023
- [15]
- [16]
- [17]
- [18]
- [19]
- [20]
- [21]
- [22]
- [23]
- [24]
-
[25]
K. M. Jablonka, P. Schwaller, A. Ortega-Guerrero, and B. Smit. Leveraging large language models for predictive chemistry. Nat Mach Intell, pp. 1–9, 2024
2024
-
[26]
K. M. Jablonka et al. 14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon. Digital Discovery, vol. 2, no. 5, pp. 1233–1250, 2023
2023
-
[27]
Zheng et al
Z. Zheng et al. Shaping the water-harvesting behavior of metal–organic frameworks aided by fine-tuned gpt models. J. Am. Chem. Soc., vol. 145, no. 51, pp. 28284–28295, 2023
2023
- [29]
- [30]
-
[31]
Weininger
D. Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. J. Chem. Inf. Comput. Sci., vol. 28, no. 1, pp. 31–36, 1988
1988
-
[32]
Metanetx.org: a website and repository for accessing, analysing and manipulating metabolic networks
Ganter M, Bernard T, Moretti S, Stelling J, and Pagni M. Metanetx.org: a website and repository for accessing, analysing and manipulating metabolic networks. Bioinformatics Oxf Engl 29:815–816, 2013
2013
-
[33]
Rhea – a manually curated resource of biochemical reactions
Alcántara R et al. Rhea – a manually curated resource of biochemical reactions. Nucleic Acids Res 40:D754-760 24, 2012
2012
-
[34]
Pathbank: a comprehensive pathway database for model organisms
Wishart DS et al. Pathbank: a comprehensive pathway database for model organisms. Nucleic Acids Res 48:D470–D478 25, 2020
2020
-
[35]
Ida, Antje C., and Dietmar S
S. Ida, Antje C., and Dietmar S. Brenda, enzyme data and metabolic information—pubmed. Nucleic Acids Res. 30,47–49, 2002
2002
- [36]
- [37]
- [38]
-
[39]
Daylight Chemical Information Systems
Inc. Daylight Chemical Information Systems. Daylight theory manual: Fingerprints
-
[40]
molecule matching
T. Chen and C. Guestrin. Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD, 2016. 15 A PREPRINT - MAY 12, 2025 5 Appendix 5.1 Data preprocessing and analysis We implemented a series of preprocessing steps to ensure a fair split across training and te...
2016
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.