{"id":"398dd9e7-517a-4ed4-9f2b-79f2cfb38e81","arxiv_id":"2506.17963","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A two-stage conditional deep learning framework modulates enzyme and substrate embeddings toward catalysis-aware features, reporting better benchmark results than specialized enzyme-substrate predictors.","lead":"OmniESI is a deep learning system that predicts enzyme-substrate interactions across four tasks: kinetic parameters, substrate pairing, active site annotation, and mutation effects. It progressively adjusts general protein and molecule features in two conditional steps and reports state-of-the-art results on seven benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'consistently superior across seven benchmarks' claim rests on baseline numbers quoted from prior papers without reruns; the active-site table itself shows protocol mismatches, so the comparison is not fully controlled.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing point: the SOTA comparison inherits baseline numbers from prior publications without verifying that those numbers were produced under identical preprocessing, test subsets, and evaluation conditions. This concern is structurally distinct from the architecture itself and directly controls the paper's strongest claim. The supplementary material even provides concrete evidence of protocol mismatch for AEGAN and BLASTp in the active-site benchmark, demonstrating that 'raw results from a prior work' are not automatically aligned with OmniESI's evaluation. The small reported margins on several key metrics make this more than a theoretical worry: a 0.036 difference in R2 or a 0.016 difference in AUROC could plausibly be affected by input featurization, random seed, or evaluation details. The paper also lacks repeated-seed uncertainty quantification, which compounds the concern, but the baseline comparability issue is the more fundamental one because it affects every benchmark in the headline claim. If the baselines are rerun and the margins persist, the conditional verdict can be upgraded; if not, the SOTA claim should be revised. For these reasons, the reader's CONDITIONAL verdict stands, and no change in verdict is recommended.","tokens_in":20112,"tokens_out":4844,"duration_ms":58480,"concrete_test":"Obtain the released code or checkpoints for CatPred, UniKP, ESP, and EasIF A-ESM, and run each baseline on the exact test files and preprocessing pipeline used for OmniESI: same ESM-2 tokenization, same RDKit SMILES canonicalization, same sequence-identity filtered subsets, and same evaluation metrics script. Run OmniESI and each baseline with at least three random seeds. If the reported OmniESI margins over the rerun baselines shrink below the published differences or reverse on any of the seven benchmarks, the 'consistently superior' claim is not supported by the current evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim depends on the assumption that OmniESI's runs are directly comparable to the published baseline numbers. Section 2.2 and Supplementary Tables S1-S8 state that 'the raw results of the other models are taken from a prior work,' while OmniESI is evaluated with its own preprocessing: ESM-2-650M tokenization, RDKit SMILES canonicalization, and the original sequence-identity test subsets. Sharing the raw data splits is necessary but not sufficient for a fair comparison; the competitors must also share the exact test files, input encodings, and evaluation script. The paper does not rerun any competitor under these conditions. The risk is not hypothetical: Table S6 documents that AEGAN was evaluated without removing 225 test samples overlapping with its training set, and BLASTp used the entire SwissProt database as its alignment database, yet both are included in the same leaderboard. This shows that 'raw results from a prior work' can carry protocol differences that affect metrics. For the kinetic benchmarks, the reported margins are small (e.g., ID kcat R2 0.638 vs. 0.602; ESP overall AUROC 0.972 vs. 0.956), and sequence-identity filtering, featurization choices, or random seeds in the original papers could plausibly shift these numbers. Without independent reruns of the baselines, the headline claim that OmniESI 'consistently delivered superior performance' is not yet established; it is conditional on a comparison protocol that the paper does not verify.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OmniESI, a unified deep-learning framework for enzyme-substrate interaction prediction. It uses a frozen ESM-2-650M enzyme encoder and a trained GCN substrate encoder, followed by two conditional modules (BCFM and CCFM) that progressively modulate features toward a 'catalysis-aware' representation. The architecture is evaluated on four tasks: kinetic parameter regression (kcat, Km, Ki), enzyme-substrate pairing, active site annotation, and mutation-effect classification, using existing benchmark splits. The authors report that OmniESI outperforms or matches specialized baselines on most metrics, with the conditional modules adding only 0.16% of the total parameters.","tokens_in":20415,"tokens_out":6583,"duration_ms":66422,"significance":"If the reported comparisons are trustworthy, OmniESI is a valuable unified baseline and a useful architectural template: it couples strong pretrained encoders with lightweight task-agnostic conditioning, and it provides honest supplementary tables showing exceptions. The paper's strengths include public code/data links, reuse of established benchmark splits, transparent parameter accounting, and qualitative attention visualizations. However, the central 'consistently superior' claim depends on baseline numbers imported from previous papers without reruns, and the reported margins are small in several cases; the interpretability claim is also only qualitatively supported. These issues limit the current certainty of the headline claims but are addressable.","major_comments":[{"comment":"The headline SOTA claim rests on baseline metrics quoted from prior publications without rerunning those baselines under OmniESI's preprocessing and evaluation pipeline. The paper states in the supplementary that 'the raw results of the other models are taken from a prior work,' and Table S6 itself documents protocol mismatches: AEGAN was evaluated without removing 225 test samples overlapping its training set, and BLASTp used the entire SwissProt database as its alignment database. Since the reported margins are small (e.g., ID kcat R2 0.638 vs 0.602 in Table S1; ESP overall AUROC 0.972 vs 0.956 in Table S4), the comparison is not fully controlled. Please rerun the baselines under identical test files, featurization, and evaluation scripts, or provide a sensitivity analysis that quantifies how much protocol differences could shift the quoted numbers.","section":"§2.2, Supplementary Tables S1–S8"},{"comment":"The abstract's claim that OmniESI 'consistently delivered superior performance than state-of-the-art specialized methods across seven benchmarks' is stronger than the tables support. OmniESI is worse than CatPred on kcat OOD at 40% identity (R2 0.342 vs 0.365, Table S1), worse than CatPred on Km OOD at 60% and 40% (Table S2), and worse than EasIF A-ESM on the active-site 40–50% interval (F1 0.826 vs 0.836, Table S7). The text later correctly says 'best or second best on all metrics,' so the abstract and Discussion should be tempered to 'best or second best on most metrics' or equivalent.","section":"Abstract; §2.2; §2.3"},{"comment":"No uncertainty quantification is provided: every metric is a single run, with no standard deviations or significance tests. Several improvements are modest (e.g., ID kcat R2 0.638 vs 0.602, Table S1; active-site F1 0.907 vs 0.902 at 50–60%, Table S7), so the claimed advantages could be within training noise. Please report mean ± std over at least three to five seeds and, where appropriate, paired significance tests for the main comparisons.","section":"§4.6; Tables S1–S12"},{"comment":"The claim that the conditional networks 'internalize the fundamental patterns of catalytic efficiency' is supported only by two qualitative case studies and PCA visualizations, not by an independent quantitative validation. This is post hoc interpretation of attention weights. Please either weaken the claim to 'attention weights are qualitatively consistent with known catalytic mechanisms in selected cases' or add quantitative evidence, such as overlap between highlighted atoms/residues and experimentally annotated catalytic sites, with appropriate baselines.","section":"§2.3; §2.5; Abstract"},{"comment":"The mutation-effect evaluation is not yet convincing as a fair comparison. The test sets are carved from the same DMS experiments without an explicit position-disjoint or sequence-disjoint split, and the baseline UniKP is reported at chance level (MCC=0, ACC=0.5, Table S9). Because the 17 active sites and substrates are shared across training and test mutations, there is a risk of information leakage through shared mutation positions or substrates. Please clarify the split construction, run a position-disjoint cross-validation, and compare against at least one additional baseline beyond UniKP.","section":"§4.1.4; Tables S9–S11"}],"minor_comments":[{"comment":"Typographical errors should be fixed: 'domian transition' in the text near Eq. (2), 'OmniESi' in §2.2, and 'ceftazidimi' in §4.1.4.","section":"§2.2; §4.1.4; Eq. (2)"},{"comment":"The supplementary table captions render 'Table' as 'T able'; please correct the formatting.","section":"Supplementary Tables S1–S3, S7–S8"},{"comment":"The baseline name 'EasIF A-Saprot' in Table S7 is inconsistent with 'EasIF A-SaProt' used in the main text and Table S6.","section":"Table S7 vs Table S6"},{"comment":"The entry counts are arithmetically inconsistent: 212,111 + 572,282 = 784,393, but the text states 784,396 entries.","section":"§4.1.2"},{"comment":"References [4] and [9] cite the same paper by Black et al. and should be merged or one removed.","section":"References [4] and [9]"},{"comment":"The statement that performance is insensitive to architecture hyperparameters is not supported by any sensitivity sweep; either add a small study or soften the claim.","section":"§4.6"},{"comment":"Please state more prominently that the MCSA E-RXN CSA zero-shot results are used only for interpretability visualization, not as a predictive benchmark.","section":"§2.3"},{"comment":"Please provide a versioned release of the code, pretrained weights, and dataset access details to enable reproduction.","section":"Data and code availability"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's main risk is the unverified cross-paper baseline comparison. The authors are transparent about exceptions and the supplementary admits protocol mismatches, which is commendable but also indicates the comparison is not fully controlled. If the editor can commission a quick reproducibility check of the GitHub repo and a rerun of one or two baselines, that would substantially de-risk the revision. I do not see evidence of data fabrication; the concern is methodological, not ethical."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe genuinely new thing in this paper is the two-stage progressive conditioning: BCFM (bidirectional conditional feature modulation) and CCFM (catalysis-aware conditional feature modulation) on top of frozen ESM-2 and a trained GCN. That is a real architectural contribution, and the ablations show both modules help with negligible parameter overhead. The paper is also honest about where it loses—kcat OOD at 40% identity, Km OOD at 60/40%, active-site 40-50%—so the main tables mostly support the claims that OmniESI is best or second-best on most benchmarks.\n\nThe soft spot is the comparison protocol. The paper says the raw splits of CatPred-DB, ESP-DB, and SwissProt E-RXN ASA were reused, and baseline numbers are taken from prior work. That is not a fully controlled comparison. The supplementary itself exposes protocol mismatches: AEGAN was evaluated without removing 225 test samples overlapping its training set, and BLASTp used the entire SwissProt as its alignment database. So the active-site leaderboard is not strictly fair. For the kinetic and ESP benchmarks, reported margins are modest (kcat ID R2 0.638 vs. 0.602, ESP AUROC 0.972 vs. 0.956), and we don't know if preprocessing, featurization, or evaluation scripts were identical across runs. Without independent reruns of the baselines, the headline claim of 'consistently superior performance' is conditional, not established.\n\nTwo more smaller issues. There is no uncertainty quantification—no multiple seeds, no confidence intervals—so we can't tell whether the differences are noise. And the mutation effect evaluation uses a tiny test set with only UniKP as baseline, which is random on the single-point task; that comparison is weak.\n\nNone of this invalidates the architecture. The core engineering is sound, the writing is clear, and the parameter-efficiency claim is credible. The issues are addressable: rerun baselines under the same protocol, report seeds and error bars, add stronger baselines for mutation effects. This paper deserves a serious referee. I'd send it out, and I'd expect revision before acceptance.","headline":"A solid empirical paper with a genuinely new two-stage conditioning architecture, but its headline 'consistently superior' claim is not fully controlled because baselines are quoted from prior work without reruns.","tokens_in":20974,"tokens_out":2546,"would_cite":true,"duration_ms":26699,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Enzyme-substrate prediction improves when general embeddings are progressively conditioned toward catalytic features.","keywords":["enzyme-substrate interaction prediction","conditional deep learning","progressive feature modulation","enzyme kinetic parameters","active site annotation","mutation effect prediction","ESM-2","graph convolutional network"],"falsifier":"Run DLKcat, UniKP, CatPred, ESP, and the EasIF variants under OmniESI's exact preprocessing, encoders, and evaluation scripts; if the quoted metric gaps shrink to noise or reverse, the superiority claim is falsified. A second decisive check is an equal-parameter ablation in which BCFM and CCFM are replaced by plain cross-attention: if the $k_{\\text{cat}}$ and active-site gains disappear, the progressive conditioning is not the cause.","tokens_in":19910,"feed_emoji":"🧬","tokens_out":8891,"duration_ms":82948,"temperature":0.7,"pith_summary":"OmniESI sets out to establish that enzyme-substrate interactions are predicted better when the latent features of an enzyme and its substrate are progressively steered from general protein-molecule representations into a catalysis-aware representation by two conditional networks. The paper argues that existing methods either train encoders from scratch or freeze pretrained embeddings without imposing catalytic priors, so their features miss reaction-relevant residues and substrate functional groups and carry catalytically irrelevant noise. With one unified architecture and only task-specific output heads, OmniESI is reported to match or beat specialized state-of-the-art methods on seven benchmarks spanning kinetic parameter regression, enzyme-substrate pairing, active-site annotation, and mutation effect classification. The two conditioning modules together add roughly 0.16% of the parameters, and ablation studies attribute the gains to them. If the claim holds, a single parameter-efficient framework could replace a collection of task-specific enzyme prediction models.","feed_headline":"One framework predicts enzyme-substrate interactions across four tasks","feed_subtitle":"Two tiny conditioning modules retune latent features toward catalysis and lift one model past seven benchmarks.","key_machinery":"The machinery is a pair of conditional modulation modules inserted between frozen ESM-2/GCN encoders and task-specific heads. BCFM (bidirectional conditional feature modulation) uses a two-sided conditioning block that generates layer-normalization parameters from the partner's pooled representation, together with multi-scale convolutions, so that the enzyme's representation is conditioned on the substrate and vice versa. CCFM (catalysis-aware conditional feature modulation) uses the concatenated pair representation as a query to compute attention weights that highlight catalytically relevant residues and atoms. The load-bearing design choice is that the substrate conditions the enzyme representation and the enzyme conditions the substrate representation in the first module, and then the joint interaction representation conditions both sides in the second, so the latent features are gradually shifted from general protein-molecule space to catalysis-aware space.","core_discovery":"The central discovery is that decomposing enzyme-substrate interaction modeling into two conditional feature-modulation stages creates a catalysis-aware latent space that transfers across tasks. In the first stage, the BCFM module makes the enzyme and substrate embeddings condition each other bidirectionally, emphasizing reaction specificity; in the second, the CCFM module derives a rough interaction vector from the modulated pair and uses it to re-weight enzyme residues and substrate atoms, emphasizing the contacts that matter for catalysis. The authors report that this architecture consistently outperforms specialized baselines: for $k_{\\text{cat}}$ in the in-distribution CatPred-DB split it improves $R^2$ by 6.0% and MAE by 6.2% relative to CatPred; on the ESP overall test set it raises MCC by 0.079; on SwissProt E-RXN ASA active-site annotation it improves F1 by 4.4% and MCC by 2.7% relative to EasIF A-ESM; and on single-point CTX-M mutations it improves AUPRC and AUROC by 0.334 and 0.219 over UniKP. They also report that attention weights from the second conditional module highlight reactive bonds and functional groups consistent with known catalytic mechanisms, which they read as evidence that the network internalized patterns of catalytic efficiency rather than memorizing benchmark statistics.","pith_inferences":["Beyond the paper, if the progressive conditioning is the active ingredient, then adding BCFM and CCFM to existing predictors such as UniKP or CatPred should lift their performance by a comparable margin; that transplant experiment would isolate the mechanism from the rest of OmniESI.","Beyond the paper, the attention maps that flag reactive bonds and substrate atoms suggest a direct application: use the CCFM attention weights to propose candidate catalytic residues and reactive atoms for unannotated enzyme-substrate pairs, then validate on a larger M-CSA set than the two illustrative cases.","Beyond the paper, the fact that one architecture serves four tasks with only output heads and training hyperparameters changed points to natural extensions such as enantioselectivity or thermostability prediction, which the paper does not evaluate.","Beyond the paper, the steady performance drop as sequence identity falls to 40% suggests the conditioning modules correct feature misalignment but not distribution shift; pairing them with domain-invariant or meta-learning training is a concrete testable route to close that gap."],"forward_implications":["For enzyme kinetic parameters, the same network improves in-distribution predictions of $k_{\\text{cat}}$, $K_m$, and $K_i$ over DLKcat, UniKP, and CatPred, with the largest out-of-distribution gains on $K_i$.","For enzyme-substrate pairing, OmniESI outperforms ESP on the overall out-of-distribution test set and on the 0–40% and 40–60% sequence-identity intervals, not just near the training distribution.","For active-site annotation, OmniESI beats both EasIF variants on the overall SwissProt E-RXN ASA test set and on four of the five sequence-identity intervals, and its attention maps align with amide-hydrolysis and glycosylation reaction centers.","For mutation effects, OmniESI classifies single-point CTX-M mutations with 86.1% accuracy versus close-to-random UniKP, and improves MCC on both double-point epistasis datasets.","Ablation and parameter counts show that BCFM and CCFM each contribute to the gains and together add only about 0.16% of total parameters, making the progressive conditioning the active ingredient rather than a larger model."],"supporting_citations":[{"why":"supplies the CatPred-DB kinetic datasets, their raw train/test splits, and the CatPred baseline OmniESI must beat for $k_{\\text{cat}}$, $K_m$, and $K_i$.","marker":"[21]"},{"why":"supplies the ESP-DB pairing dataset, its out-of-distribution splits, and the ESP baseline for substrate-pairing comparison.","marker":"[22]"},{"why":"supplies the SwissProt E-RXN ASA active-site dataset, the EasIF baselines, and the MCSA E-RXN CSA test cases used for zero-shot attention visualization.","marker":"[23]"},{"why":"provides the frozen ESM-2-650M protein encoder whose embeddings are the starting point of the progressive modulation.","marker":"[26]"},{"why":"provides the graph convolutional substrate encoder that turns 2D molecular graphs into the molecule-side embeddings.","marker":"[34]"},{"why":"provides the UniKP baseline used for kinetic parameter and mutation-effect comparisons, and frames the unified multi-task benchmark.","marker":"[16]"},{"why":"supplies the CTX-M beta-lactamase deep mutational scanning data from which the single- and double-point mutation effect datasets are curated.","marker":"[58]"},{"why":"provides DLKcat as one of the kinetic-parameter baselines on CatPred-DB.","marker":"[12]"}],"fun_headline_variants":["Two tiny modules make one model beat specialized enzyme predictors","Progressive conditioning lifts enzyme-substrate predictions past seven benchmarks","Unified enzyme-substrate AI: two-stage conditioning, state-of-the-art across tasks","Small conditional tweaks give one model broad enzyme task mastery","One model, two conditioners, seven benchmarks beaten"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central comparison assumes that taking the raw data splits from the original benchmark papers while quoting the competing methods' reported numbers gives a like-for-like evaluation; if the baselines used different preprocessing, input encodings, or evaluation scripts, the reported seven-benchmark superiority may not survive a controlled rerun.","fun_headline_variants_meta":{"raw":{"variants":["Two tiny modules make one model beat specialized enzyme predictors","Progressive conditioning lifts enzyme-substrate predictions past seven benchmarks","Unified enzyme-substrate AI: two-stage conditioning, state-of-the-art across tasks","Small conditional tweaks give one model broad enzyme task mastery","One model, two conditioners, seven benchmarks beaten"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00025,"raw_usage":{"total_tokens":1624,"prompt_tokens":1087,"completion_tokens":537,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":703,"completion_tokens_details":{"reasoning_tokens":452}},"tokens_in":703,"tokens_out":537,"duration_ms":5942,"temperature":1.0,"reasoning_tokens":452,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:23:13.553191+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DLKcat, UniKP, CatPred, ESP, and the EasIF variants under OmniESI's exact preprocessing, encoders, and evaluation scripts; if the quoted metric gaps shrink to noise or reverse, the superiority claim is falsified. A second decisive check is an equal-parameter ablation in which BCFM and CCFM are replaced by plain cross-attention: if the $k_{\\text{cat}}$ and active-site gains disappear, the progressive conditioning is not the cause.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the CatPred-DB kinetic datasets, their raw train/test splits, and the CatPred baseline OmniESI must beat for $k_{\\text{cat}}$, $K_m$, and $K_i$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the ESP-DB pairing dataset, its out-of-distribution splits, and the ESP baseline for substrate-pairing comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the SwissProt E-RXN ASA active-site dataset, the EasIF baselines, and the MCSA E-RXN CSA test cases used for zero-shot attention visualization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the UniKP baseline used for kinetic parameter and mutation-effect comparisons, and frames the unified multi-task benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the CTX-M beta-lactamase deep mutational scanning data from which the single- and double-point mutation effect datasets are curated."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides DLKcat as one of the kinetic-parameter baselines on CatPred-DB."}],"review_version":1}