REVIEW 5 major objections 7 minor 44 references
zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literature
T0 review · 5 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A modular, human-in-the-loop pipeline extracts enzyme kinetic parameters, substrate SMILES, and molecular diagrams from scientific PDFs, outperforming existing baselines on a new expert-annotated P450 benchmark.
desk verdict The molecular image recognition results are the real, credible contribution; the headline table extraction numbers are internally inconsistent and unverifiable as written, so treat the P450 benchmark claims as conditional until data and split are released. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the multi-module pipeline. A table recognition stage built on PP-Structure/SLANet is fine-tuned in two stages, first on PubTabNet pretraining and then on domain-specific enzyme-kinetics tables, with fuzzy header-to-schema matching and value-unit pairing to turn messy tables into JSON entries. UniMolRec, an ensemble molecular image-to-SMILES recognizer, combines multiple state-of-the-art single models so that failures on one image style are compensated by another. An LLM-based semantic entity and relation extraction module assigns roles such as substrate, product, enzyme, and cofactor, and performs cross-modal integration. Wrapping these is an interactive annotation and active-learning module: human experts review and correct outputs, and the corrections are fed back to fine-tune the models, which the paper reports as yielding measurable accuracy gains. This modular design is what makes the platform extensible, since new models can be swapped in as they appear.
What would settle it
Give the released benchmark a single, pre-registered held-out split of P450 tables that were never used for fine-tuning, run the fine-tuned pipeline and the SLANet and MolScribe baselines on those tables, and compare. If the 3.9% table-accuracy and 3.0% relation-accuracy gains disappear, or if tracing the fine-tuning and evaluation sets shows overlap, the central claim of outperformance and self-improvement fails.
Extended reading notes
Core claim
On its own terms, the central discovery is that a hybrid, modular pipeline—fine-tuned table-structure OCR, an ensemble molecule recognizer, and LLM-based semantic parsing, wrapped in a human-in-the-loop annotation loop—can outperform the previous best methods on all three subtasks of enzyme-reaction extraction from the literature. The reported numbers are 89.9% table accuracy (vs 86.0% for SLANet), up to 99.1% on the UOB molecular image benchmark (vs 96.0% for MolScribe), and 94.2% role-assignment accuracy for linking molecules to their reaction roles. A second, closely related claim is that the system is self-improving: retraining the SLANet-based table module and the LLM reaction parser on the team's manually annotated P450 dataset lifted table accuracy by 3.9% and relation accuracy by 3.0%, evidence that the human-in-the-loop loop works in practice. The paper further claims the expert-annotated dataset itself—over 1,000 tables, approximately 5,000 biological fields, from 270 publications—is a community resource that enables both training and fair benchmarking.
Load-bearing premise
The load-bearing premise is that the manually annotated P450 benchmark is unbiased and reproducible: the labels are consistent across the ten annotators, and the data used to fine-tune the models do not overlap with the tables used to report accuracy, a split the paper does not document.
Editorial extensions
If this is right
- If the reported accuracies hold, enzyme kinetic databases could be back-filled from the uncurated literature at scale, giving kcat and Km prediction models far more training examples than current manually curated resources provide.
- Because the pipeline is modular, any improvement in table recognition, OCR, or molecule recognition can be dropped in without rebuilding the system; the paper's 3.9% and 3.0% refinement gains imply the platform's performance grows with its use.
- The released P450 benchmark gives the enzyme informatics community a standard, expert-annotated test set for comparing extraction tools, which was previously missing.
- With sequences, substrate SMILES, and kinetics captured in one structured record, the extracted data can feed downstream tasks such as enzyme function prediction, reaction feasibility checking, and knowledge-graph construction for biocatalysis.
Reading between the lines
- A natural extension the paper leaves untested is transfer across enzyme families: the table and molecule modules are family-agnostic, so applying the pipeline to non-P450 enzymes would mainly require re-tuning the LLM role-assignment module.
- The ensemble architecture of UniMolRec could be pushed further by treating inter-model disagreement as an uncertainty signal, so images on which recognizers disagree are automatically routed to human review rather than being resolved by majority vote.
- Because each structured record ties sequence, substrate SMILES, and measured kinetic parameters together, the released dataset may prove as valuable for training kcat and Km predictors as for benchmarking extraction tools; the paper frames it primarily as a benchmark, not as a modeling corpus.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents zERExtractor, a modular pipeline for extracting enzyme-catalyzed reaction data (e.g., kcat, Km, substrate SMILES, sequences, conditions) from scientific PDFs. The pipeline couples OCR-based table structure recognition (fine-tuned SLANet), a molecular image-to-SMILES ensemble called UniMolRec, and LLM-based relation extraction, with a human-in-the-loop annotation workflow. The authors report benchmark accuracies of 89.9% for table extraction, up to 99.1% for molecular image recognition, and 94.2% for relation role assignment, and state that they release a benchmark of over 1,000 annotated tables from 270 P450-related publications.
Significance. The problem of automatically converting enzyme kinetics literature into structured data is important, and the modular design with replaceable LLM components is timely. The molecular image recognition results (Table 2) are reported on standard benchmarks and appear competitive, and the authors have invested in a human-in-the-loop annotation system. If the table extraction and relation extraction results can be placed on a clearly described, leakage-free evaluation, the benchmark dataset would be a useful community resource. At present, however, the central numerical claims are not reproducible from the manuscript.
major comments (5)
- [Table Extraction Accuracy] The 89.9% accuracy claim is internally inconsistent. The text describes the evaluation set as "270 full-text articles, containing a total of 100 enzyme-related tables," but Table 3 is captioned "Performance Comparison on PubTabNet," and the cited baseline values (TableMaster 77.90%, LGPMA 65.74%, SLANet 86.0%) match published PubTabNet numbers. If 89.9% was measured on PubTabNet, it does not support the claimed P450 enzyme-table extraction performance; if it was measured on the 100 P450 tables, the baselines were not evaluated on the same benchmark, and the 3.9% gain is not a controlled comparison. Please specify the exact benchmark for Table 3 and re-run all baselines on the same test set.
- [Structured Extraction of Tabular Data / Model Refinement] The dataset size is reported inconsistently: the abstract says "over 1,000 fully annotated tables," the Results section says "over 800 tables" were annotated, later "175 high-quality tables" with 612 reaction entries are retained, and the evaluation benchmark contains "100 enzyme-related tables." No train/fine-tune/evaluation split is specified. Because the SLANet model was fine-tuned on tables "extracted from biochemical reaction documents" from the same 270-PDF corpus used for benchmarking, overlap between fine-tuning and evaluation tables is plausible and would inflate the reported accuracy. Please document the split explicitly and confirm that the evaluation tables were not used for fine-tuning.
- [Model Refinement through Human-Annotated Data] The claimed improvements of 3.9% in table field extraction and 3.0% in molecular relation recognition are referenced to Table 2, but Table 2 shows molecular image recognition results on synthetic and realistic datasets and contains no before/after fine-tuning comparison. The evaluation protocol for these improvements, including the test sets and baselines, is not described. This section does not currently support the "self-evolving" claim and needs a dedicated experimental description.
- [Molecular Graph Recognition and Relation] The relation extraction accuracy of 94.2% is reported without describing the evaluation set (size, composition, annotation procedure) or comparing to any baseline. The reference to OpenChemIE [23] in the phrase "manually annotated reaction benchmark[23]" is unexplained. As relation extraction is one of the three headline results, this number is currently unverifiable and needs a full description of the benchmark and metric.
- [Results (all benchmark sections)] None of the reported accuracies include error bars, confidence intervals, or significance tests. Given that the table extraction evaluation uses only 100 tables, the reported 3.9% improvement over SLANet could be within sampling variability; please provide at least binomial confidence intervals or per-table results.
minor comments (7)
- [Conclusion] The word "enzymetic" in the conclusion should be "enzymatic."
- [Molecular Graph Recognition and Relation] The text says "As shown in Table 1" when presenting the molecular image recognition results; the correct reference is Table 2 (Table 1 lists the schema). Similarly, "as shown in Table 2" in the Model Refinement section does not point to any relevant table.
- [Abstract / Data Availability] The abstract states that the authors "release" a large benchmark dataset, but no URL, repository, or data availability statement is provided anywhere in the manuscript; please add one.
- [References] Reference [40] has malformed author formatting with "Committee, P. (ed.)" duplicated; please fix the citation.
- [Table Extraction Accuracy] The accuracy metric "Acc" is defined only as "structural and semantic consistency between the predicted table and the ground truth annotation"; please specify the exact scoring rule (e.g., cell-level TEDS or exact match).
- [Structured Extraction of Tabular Data] The phrase "open-accessed publications" should be "open-access publications."
- [Data Preparation for Molecular Graph Recognition] For the in-house dataset of 1,355 manually labeled molecular images, please clarify whether any of these images are drawn from the evaluation benchmarks (e.g., UOB, USPTO) to rule out overlap between training and test data.
Circularity Check
Table-extraction accuracy is reported as a prediction but is measured on the same 270-PDF dataset used to fine-tune the model, with no documented held-out split; the 3.9% post-training improvement is described on the same dataset used for fitting.
-
fitted input called prediction
[Results, 'Model Refinement through Human-Annotated Data' and 'Table Extraction Accuracy'; Table 3]
"The dataset was constructed from 270 enzymology-related PDFs, yielding over 1,000 labeled tables and 5,000 annotated biological fields ... We used this dataset to fine-tune two critical components of our system. ... Post-training evaluation showed measurable improvements. The table field extraction accuracy improved by 3.9%, while molecular relation recognition improved by 3.0%."
In this passage the same dataset is both the fine-tuning input and the basis for the reported post-training improvement; no held-out split or independent test set is described. The earlier table-accuracy benchmark (100 enzyme-related tables from 270 full-text articles) likewise does not state that it is disjoint from the over-1,000-table fine-tuning set. Under the paper's own description, the 3.9% table-field improvement and the 89.9% accuracy are therefore measurements on data used to fit the model, i.e., training-set fit reported as a prediction rather than as an evaluation on new literature.
full rationale
This is an engineering report with no formal derivation chain, so most of the extraction claims rest on empirical benchmarks rather than on equations that could be circular by construction. The molecular image recognition component is benchmarked against external datasets (UOB, USPTO, CLEF, synthetic Indigo/ChemDraw), and the relation-extraction accuracy cites an external manually annotated reaction benchmark (OpenChemIE), so those components retain independent support. The table-extraction claim, however, is not independent: the paper states that the same 270-PDF, over-1,000-table dataset was used to fine-tune the SLANet-based extractor and then reports post-training improvements on that dataset, with no documented disjoint evaluation set. This is the central table-recognition result (89.9%, 3.9% gain over SLANet), and it is reinforced by internal inconsistencies: Table 3 is captioned as a PubTabNet comparison while the text describes a 100-table enzyme benchmark, and the dataset size is given as both over 800 and over 1,000 tables. Those inconsistencies are not themselves circularity, but they prevent the reader from verifying that the prediction was measured on unseen data. Because the table-extraction subclaim reduces, on the paper's own description, to a fit on the same corpus, the overall circularity score is 6, not higher, because the molecular and relation-extraction results are evaluated against external benchmarks and retain independent content.
Assumptions & free parameters
free parameters (3)
- Fine-tuning data sources and sizes =
Not reported precisely
- LLM prompts and role-assignment rules =
Not reported
- UniMolRec ensemble composition and thresholds =
Not reported
assumptions (3)
- domain assumption The manually annotated P450 benchmark labels are correct and consistent across ten annotators.
- domain assumption PubTabNet is an appropriate pretraining corpus, and the domain fine-tuning data do not overlap with the evaluation tables.
- domain assumption Standard OCR, table structure recognition, and image-to-SMILES pipelines work as claimed on the inputs.
Cite this review
Pith. "Pith review of zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literature." pith.science (2026). https://pith.science/paper/76VCKQMI
@misc{pith2026250809995,
author = {Pith},
title = {Pith review of: zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literature},
year = {2026},
howpublished = {\url{https://pith.science/paper/76VCKQMI}},
note = {Machine review of arXiv:2508.09995}
}
read the original abstract
The rapid expansion of enzyme kinetics literature has outpaced the curation capabilities of major biochemical databases, creating a substantial barrier to AI-driven modeling and knowledge discovery. We present zERExtractor, an automated and extensible platform for comprehensive extraction of enzyme-catalyzed reaction and activity data from scientific literature. zERExtractor features a unified, modular architecture that supports plug-and-play integration of state-of-the-art models, including large language models (LLMs), as interchangeable components, enabling continuous system evolution alongside advances in AI. Our pipeline combines domain-adapted deep learning, advanced OCR, semantic entity recognition, and prompt-driven LLM modules, together with human expert corrections, to extract kinetic parameters (e.g., kcat, Km), enzyme sequences, substrate SMILES, experimental conditions, and molecular diagrams from heterogeneous document formats. Through active learning strategies integrating AI-assisted annotation, expert validation, and iterative refinement, the system adapts rapidly to new data sources. We also release a large benchmark dataset comprising over 1,000 annotated tables and 5,000 biological fields from 270 P450-related enzymology publications. Benchmarking demonstrates that zERExtractor consistently outperforms existing baselines in table recognition (Acc 89.9%), molecular image interpretation (up to 99.1%), and relation extraction (accuracy 94.2%). zERExtractor bridges the longstanding data gap in enzyme kinetics with a flexible, plugin-ready framework and high-fidelity extraction, laying the groundwork for future AI-powered enzyme modeling and biochemical knowledge discovery.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[23]
Fan, V.et al.Openchemie: An infor- mation extraction toolkit for chem- istry literature.Journal of Chemical Information and Modeling64, 5521– 5534 (2024)
work page 2024
-
[1]
Nature Catalysis5, 662–672 (2022)
Li, F.et al.Deep learning-based k cat prediction enables improved enzyme- constrained model reconstruction. Nature Catalysis5, 662–672 (2022)
work page 2022
-
[2]
Wang, Z.et al.Robust enzyme dis- covery and engineering with deep learning using catapro.Nature com- munications16, 2736 (2025)
work page 2025
-
[3]
Yu, H., Deng, H., He, J., Keasling, J. D. & Luo, X. Unikp: a uni- fied framework for the prediction of enzyme kinetic parameters.Nature communications14, 8211 (2023)
work page 2023
-
[4]
Kroll, A., Rousset, Y., Hu, X.-P., Liebrand, N. A. & Lercher, M. J. Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning. Nature communications14, 4139 (2023)
work page 2023
-
[5]
Chang, A.et al.Brenda, the elixir core data resource in 2021: new devel- opments and updates.Nucleic acids research49, D498–D508 (2021)
work page 2021
-
[6]
Schomburg, I.et al.The brenda enzyme information system–from a database to an expert system.Jour- nal of biotechnology261, 194–206 (2017)
work page 2017
-
[7]
Wittig, U., Rey, M., Weidemann, A., Kania, R. & M¨ uller, W. Sabio- rk: an updated resource for man- ually curated biochemical reaction kinetics.Nucleic acids research46, D656–D660 (2018)
work page 2018
Show all 44 references
-
[8]
Fowler, D. M. & Fields, S. Deep mutational scanning: a new style of protein science.Nature methods11, 801–807 (2014)
2014
-
[9]
Uniprot: the universal protein knowl- edgebase in 2023 (2023)
2023
-
[10]
Kim, S.et al.Pubchem in 2021: new data content and improved web interfaces.Nucleic acids research49, D1388–D1395 (2021)
2021
-
[11]
Modeling enzyme kinet- ics: Current challenges and future perspectives for biocatalysis.Bio- chemistry63, 2533–2541 (2024)
Pleiss, J. Modeling enzyme kinet- ics: Current challenges and future perspectives for biocatalysis.Bio- chemistry63, 2533–2541 (2024)
2024
-
[12]
& Nielsen, J
Pornputtapong, N., Nookaew, I. & Nielsen, J. Human metabolic atlas: an online resource for human metabolism.Database2015, bav068 (2015). 13
2015
-
[13]
Hu, M.et al.Evaluation of large language models for discovery of gene set function.Nature methods22, 82– 91 (2025)
2025
-
[14]
Jin, Q., Leaman, R. & Lu, Z. Pubmed and beyond: biomedical literature search in the age of artificial intelli- gence.EBioMedicine100(2024)
2024
-
[15]
Dagdelen, J.et al.Structured infor- mation extraction from scientific text with large language models.Nature Communications15, 1418 (2024)
2024
-
[16]
Polak, M. P. & Morgan, D. Extract- ing accurate materials data from research papers with conversational language models and prompt engi- neering.Nature Communications15, 1569 (2024)
2024
-
[17]
A., MacKnight, R., Kline, B
Boiko, D. A., MacKnight, R., Kline, B. & Gomes, G. Autonomous chem- ical research with large language models.Nature624, 570–578 (2023)
2023
-
[18]
& Moghe, G
Smith, N., Yuan, X., Melissinos, C. & Moghe, G. Funcfetch: an llm-assisted workflow enables mining thousands of enzyme–substrate interactions from published manuscripts.Bioinformat- ics41, btae756 (2025)
2025
-
[19]
Lai, P.-T.et al.Enzchemred, a rich enzyme chemistry relation extraction dataset.Scientific Data11, 982 (2024)
2024
-
[20]
Jiang, J.et al.Enzyme co-scientist: Harnessing large language models for enzyme kinetic data extraction from literature.bioRxiv2025–03 (2025)
2025
-
[21]
& Farhadi, A
Redmon, J. & Farhadi, A. Yolo9000: better, faster, stronger (2017)
2017
-
[22]
arXiv preprint arXiv:2210.05391 (2022)
Li, C.et al.Pp-structurev2: A stronger document analysis system. arXiv preprint arXiv:2210.05391 (2022)
2022 arXiv
-
[24]
M.Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI(Simon and Schuster, 2021)
Monarch, R. M.Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI(Simon and Schuster, 2021)
2021
-
[25]
Luo, Q.et al.Benchmarking reverse docking through alphafold2 human proteome.Protein Science33, e5167 (2024)
2024
-
[26]
& Montanari, F
Clevert, D.-A., Le, T., Winter, R. & Montanari, F. Img2mol–accurate smiles recognition from molecular graphical depictions.Chemical sci- ence12, 14174–14181 (2021)
2021
-
[27]
Khokhlov, I., Krasnov, L., Fedorov, M. V. & Sosnin, S. Image2smiles: Transformer-based molecular opti- cal recognition engine.Chemistry- Methods2, e202100069 (2022)
2022
-
[28]
& Sakakibara, Y
Hirohara, M., Saito, Y., Koda, Y., Sato, K. & Sakakibara, Y. Con- volutional neural network based on smiles representation of compounds for detecting chemical motif.BMC bioinformatics19, 526 (2018)
2018
-
[29]
& van der Laan, M
Ju, C., Bibaut, A. & van der Laan, M. The relative performance of ensem- ble methods with deep convolutional neural networks for image classifica- tion.Journal of applied statistics45, 2800–2818 (2018). 14
2018
-
[30]
Qian, Y.et al.Molscribe: robust molecular structure recognition with image-to-graph generation.Jour- nal of Chemical Information and Modeling63, 1925–1934 (2023)
2023
-
[31]
O., Agea, M
Rajan, K., Brinkhaus, H. O., Agea, M. I., Zielesny, A. & Steinbeck, C. Decimer. ai: an open platform for automated optical chemical struc- ture identification, segmentation and recognition in scientific publications. Nature communications14, 5045 (2023)
2023
-
[32]
Cytochromes p450 as versatile biocatalysts.Journal of biotechnology124, 128–145 (2006)
Bernhardt, R. Cytochromes p450 as versatile biocatalysts.Journal of biotechnology124, 128–145 (2006)
2006
-
[33]
Gillam, E. M. Engineering cytochrome p450 enzymes.Chemical research in toxicology21, 220–231 (2008)
2008
-
[34]
Shaik, S.et al.P450 enzymes: Their structure, reactivity, and selec- tivity modeled by qm/mm calcula- tions.Chemical reviews110, 949– 1017 (2010)
2010
-
[35]
Huang, J.et al.A critical assessment of using chatgpt for extracting struc- tured data from clinical notes.NPJ digital medicine7, 106 (2024)
2024
-
[36]
& McQuaw, C
Staker, J., Marshall, K., Abel, R. & McQuaw, C. M. Molecular struc- ture extraction from documents using deep learning.Journal of chem- ical information and modeling59, 1017–1029 (2019)
2019
-
[37]
& Lee, H
Yoo, S., Kwon, O. & Lee, H. Image- to-graph transformers for chemical structure recognition (2022). Pub- lished in Nature Communications, Vol. 13, Issue 1, pp. 1234–1240
2022
-
[38]
Pavlov, D.et al.Indigo: univer- sal cheminformatics api.Journal of cheminformatics3, P4 (2011)
2011
-
[39]
Kim, S.et al.Pubchem substance and compound databases.Nucleic acids research44, D1202–D1213 (2016)
2016
-
[40]
& Yepes, A
Zhong, X., Tang, J. & Yepes, A. J. Committee, P. (ed.)Publaynet: Largest dataset ever for document layout analysis. (ed.Committee, P.) , Vol. 2019, 1015–1022 (IEEE, 2019)
2019
-
[41]
Ye, J.et al.Pingan-vcgroup’s solu- tion for icdar 2021 competition on scientific literature parsing task b: table recognition to html.arXiv preprint arXiv:2105.01848(2021)
2021 arXiv
-
[42]
(ed.) Lgpma: Complicated table structure recognition with local and global pyra- mid mask alignment
Qiao, L.et al.Committee, P. (ed.) Lgpma: Complicated table structure recognition with local and global pyra- mid mask alignment. (ed.Committee, P.)Proceedings of the International Conference on Document Analysis and Recognition (ICDAR), 99–114 (Springer, 2021)
2021
-
[43]
& Rybalkin, M
Smolov, V., Zentsev, F. & Rybalkin, M. Imago: Open-source toolkit for 2d chemical structure image recognition. (2011)
2011
-
[44]
Meng, J.et al.Boosting the pre- dictive performance with aqueous solubility dataset curation.Scientific Data9, 71 (2022). 15
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.