{"id":"cd315671-643d-4adc-a83b-51f5bf04aa6f","arxiv_id":"2502.01461","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Enzymatic reaction prediction improves when the enzyme representation is built from docking interaction scores and learned attention, rather than from a static protein embedding.","lead":"This paper introduces Docking-Aware Attention (DAA), a model that blends predicted protein-molecule docking scores with learned attention to create enzyme representations that change with each substrate. The authors report accuracy improvements on enzymatic reaction prediction, reaching about 62% on complex molecules and 55.5% on novel reactions, roughly 5 to 6 points above earlier methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DAA's physical-interaction claim rests on unstated protein-structure provenance and unvalidated DiffDock poses; without a pose-quality check the reported gains may be a molecule-specific feature rather than a physical prior.","rationale":"The central claim is not merely that a molecule-dependent protein token helps, but that docking-derived physical interaction scores create context-aware representations. The architecture's novelty rests on Eq. 2-3: DiffDock samples poses, Lennard-Jones scores are computed, and these scores are injected into attention. For this to be a physical prior, two conditions must hold: a correct 3D structure must exist for each ECREACT enzyme, and DiffDock poses must be accurate enough that per-residue LJ scores reflect true contacts. Neither is established. The manuscript never states the source of protein structures, the number of sampled poses, the LJ parameters, or the docking preprocessing, which is not a stylistic omission but a missing load-bearing support: without it the docking module cannot be reproduced and the physical interpretation cannot be checked. The ablation shows docking-only helps versus standard attention, but that comparison cannot distinguish physically meaningful poses from any molecule-dependent 3D perturbation. A random-score or random-pose control would settle attribution. This concern aligns with the reader's weakest assumption, and the conditional verdict remains appropriate: the empirical numbers may be correct, but the explanatory mechanism and reproducibility require independent verification.","tokens_in":872,"tokens_out":1039,"duration_ms":64095,"concrete_test":"Run the full DAA pipeline under two control conditions while keeping all hyperparameters fixed: (a) per-residue scores S_i from Eq. 3 randomly permuted across amino acids before smoothing; (b) DiffDock poses replaced by random rigid-body transforms of the substrate. If top-5 accuracy stays at or near 71.48% (or complex/novel subsets stay near 62.2/55.54), the physical docking signal is not the load-bearing component and the interpreted attention analysis is unsupported. If performance drops to the standard-attention level (~67.5%), pose quality becomes the remaining issue and should be validated by RMSD against known enzyme-substrate complexes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2.2 (Eqs. 2-3) makes DAA's novelty depend on DiffDock poses being accurate enough that per-residue Lennard-Jones scores reflect real contacts, but the paper never states where the ECREACT enzymes' 3D structures come from (PDB, AlphaFold, ESMFold) and never validates DiffDock on this enzyme-substrate set. ECREACT is a reaction dataset with SMILES and EC numbers, not a docking benchmark, and DiffDock's accuracy on metabolite-like substrates and blind docking is known to be worse than on its training benchmarks. If the sampled poses are mostly wrong, the scores S_i in Eq. 3 are not physical interaction strengths but deterministic functions of geometry noise; the 'docking-aware' attention then implements a learned molecule-dependent bias whose physical interpretation is unsupported. The ablation 'Docking-Only vs Standard Attention' (Table 2) does not resolve this: any per-molecule 3D feature, even random poses, could similarly shift attention and improve a sequence model. The paper also omits K, the Lennard-Jones parameters, smoothing initialization, and docking preprocessing details, so the experiment cannot be reproduced exactly. The missing provenance is the load-bearing gap: the central claim of 'physical interaction information' cannot be checked without it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Docking-Aware Attention (DAA), a mechanism that augments attention-based protein representations with per-residue interaction scores derived from DiffDock docking poses and Lennard-Jones potentials, in order to make protein representations context-dependent for enzymatic reaction prediction. The model is evaluated on the ECREACT dataset using a sequence-to-sequence T5 backbone with a protein embedding injected as a special token. The authors report improvements over baselines on overall accuracy (Top-5 71.48% vs 66.64% for EC Tokens), on a 'complex molecules' subset defined by Bertz Complexity Index >1500 (Top-5 62.20% vs 56.79%), and on an 'innovative reactions' split (Top-5 55.54% vs 49.45%). Ablation studies compare attention variants, embedding models, and token integration strategies, and the paper claims statistical significance for the main results via a two-proportion z-test.","tokens_in":12359,"tokens_out":4609,"duration_ms":41527,"significance":"If the central claim holds, DAA is a plausible and useful step toward context-dependent protein representations for enzymatic reaction prediction, and the idea of using docking-derived physical scores as an attention bias is a concrete and potentially transferable contribution. The paper ships open-source code and pre-trained models, and the ablation across embedding models (ESM3, ProtBERT, GearNet) is a strength, as it suggests the mechanism is not tied to a single encoder. However, the physical-interpretation claim rests on unverified docking pose quality and unstated protein structure provenance, and the statistical and split-definition gaps prevent full verification of the headline numbers. The core idea is defensible, but the current evidence is not yet sufficient to support the strong physical-interpretation and significance claims.","major_comments":[{"comment":"The physical-interpretation claim depends on DiffDock pose quality and on the availability of reliable protein 3D structures, neither of which is documented. Eq. (2) calls DiffDock(P, M, k) on ECREACT enzyme-substrate pairs, but ECREACT provides sequences and EC numbers rather than structures; the manuscript never states whether the structures come from PDB, AlphaFold, ESMFold, or homology models, and it does not validate DiffDock pose accuracy for these metabolite-like substrates. If the poses are largely inaccurate, the per-residue scores S_i in Eq. (3) are not physical interaction strengths but deterministic functions of geometry noise, making the 'docking-aware' component a learned molecule-dependent bias with an unsupported physical interpretation. Please specify the structure source and docking preprocessing, and include a pose-quality check or a negative control with randomized poses.","section":"Section 3.2.2, Eq. (2)-(3)"},{"comment":"Statistical significance is claimed for all bold entries in Table 1, but no p-values, confidence intervals, or test assumptions are reported. The Two-Proportion Z-Test is described only in general terms; the sample sizes for the All, Complex, and Novel subsets are not given, and no correction for multiple comparisons is mentioned. Please report exact p-values or confidence intervals for the headline comparisons (e.g., DAA vs. EC Tokens and DAA vs. ReactEmbed at Top-1 and Top-5 for each split), or temper the significance claims accordingly.","section":"Section 4.4.1"},{"comment":"The two headline evaluation splits are under-specified. The 'complex' split uses an ad hoc Bertz Complexity Index threshold of 1500 with no sensitivity analysis, and the 'innovative reactions' split is described only as 'target molecules not seen during training' without defining the molecular similarity criterion or the size of the subset. Because the abstract's main gains (62.2% vs. 56.79% and 55.54% vs. 49.45%) are computed on these splits, please define them precisely and show robustness to the threshold and split choices.","section":"Sections 5.1 and 5.2"},{"comment":"There is a notational inconsistency in the smoothing step and several free hyperparameters are omitted. Eq. (4) defines \\hat{S}_i = \\beta V_i + (1-\\beta) \\frac{1}{n} \\sum_j S_j, but V_i is not defined in the text, while Algorithm 1 (lines 8-10) uses V_i in the same expression and Eq. (3) defines S_i; this prevents exact reproduction of the interaction-profile computation. Please unify the S/V notation and report K (number of DiffDock samples), the Lennard-Jones epsilon and sigma values, the initialization of beta and gamma, and any docking preprocessing details.","section":"Section 3.2.3, Eq. (4), and Algorithm 1"}],"minor_comments":[{"comment":"The conclusion states 55.44% for innovative reactions, which is inconsistent with the 55.54% reported in the Abstract and Table 1; please correct the discrepancy.","section":"Section 7"},{"comment":"The ProtBERT base Top-5 value appears as 64.8 in Table 2 but 64.82 in Table 1, and the Addition row lists Top-3 as 57.113 with an extra digit; please standardize the decimal reporting.","section":"Table 2"},{"comment":"The statement that 'higher absolute values of S_i indicate stronger consistent interactions' is not immediately clear because the Lennard-Jones potential is signed, with positive values corresponding to repulsion; please clarify how the sign is handled in the attention bias.","section":"Section 3.2.2"},{"comment":"The ablation shows Docking-Only outperforming Standard Attention at Top-3 and Top-5 but not at Top-1; the text's claim that physical scores 'provide valuable signals' would benefit from a brief comment on this non-monotonic pattern.","section":"Section 5.3.1"},{"comment":"Reference [21] cites the USPTO patent assignment dataset, but the USPTO reaction dataset typically used for augmentation is a different resource (Lowe's reaction dataset); please cite the correct source for the reaction data.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the core idea is worth pursuing, but the missing structure provenance, pose-quality validation, and split/statistical details are load-bearing for the physical-interpretation and significance claims. I would like to see the revision address Major Comments 1-3 before this can be accepted. The anonymous code link should also be checked by the editor for accessibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about arXiv:2502.01461. First, the core idea — using docking-derived per-residue interaction scores as a dynamic bias in attention to make protein representations molecule-dependent — is genuinely new relative to the cited work. Second, the paper's empirical claims are plausible but under-supported: significance is asserted without reporting p-values, the docking poses are not validated on this enzyme-substrate set, and the protein structure source is unstated. That places it in the conditional bucket, not reject.\n\nWhat the paper does well: The architecture is simple and clean. It builds on ESM3 embeddings, samples K DiffDock poses, computes per-residue Lennard-Jones scores, smooths them with a learnable beta, and injects them with a learnable gamma into attention. The ablations are useful: standard attention vs docking-only vs full DAA, and DAA applied to three different base encoders (ESM3, ProtBERT, GearNet). The consistent improvement across those ablations suggests the mechanism is robust, not a one-off interaction with a particular embedding. The results on complex molecules and innovative reactions are presented clearly, and the code is claimed open-source.\n\nThe soft spots are proportionate. The most load-bearing is the missing validation of the docking signal. The paper does not say where the protein structures come from or whether DiffDock produces reliable poses on ECREACT's metabolite-like substrates. If the poses are mostly wrong, the 'physical interaction' scores are just deterministic functions of geometry noise — the method might still improve over baselines (any per-molecule 3D feature could shift attention), but the physical-interpretation claim would be unsupported. The paper should include a pose-quality check, or at least a sensitivity analysis with random poses. This is an addressable gap, not a fundamental error.\n\nThe statistical reporting is also thin: a Two-Proportion Z-Test is mentioned but no p-values, confidence intervals, or effect sizes are given. The 'innovative reaction' split is described only as 'target molecules not seen during training,' which leaves open how the split was constructed. And the Bertz complexity threshold (>1500) is arbitrary without a sensitivity analysis.\n\nIf these are fixed — provenance, pose validation, proper statistics, split details — I'd be confident in the claims. As it stands, the paper is a solid contribution to biocatalysis prediction: the mechanism is reusable, the experiments are suggestive, and the direction is worth pursuing.\n\nRecommendation: send it to peer review. It deserves serious referee time, not desk rejection.","headline":"A genuinely new mechanism for context-dependent protein representations, but the docking-quality assumption and statistical reporting need work before the numerical claims can be taken at face value.","tokens_in":12844,"tokens_out":2971,"would_cite":true,"duration_ms":26115,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Docking-aware attention generates dynamic, substrate-dependent protein representations that substantially improve enzymatic reaction prediction.","keywords":["Protein Representation Learning","Molecular Docking","Biocatalysis","Deep Learning","Computational Biology","Enzymatic Reaction Prediction","Context-Aware Attention"],"falsifier":"Replace the docking-derived interaction scores with random per-residue values while keeping the attention architecture and training protocol identical; if top-5 accuracy on complex molecules stays near 62.2% rather than falling to the static-embedding baseline, the docking signal itself is not the cause of the reported improvement.","tokens_in":11871,"feed_emoji":"🧬","tokens_out":7817,"duration_ms":63362,"temperature":0.7,"pith_summary":"The paper tries to establish that protein representations should not be static: the same enzyme behaves differently with different substrates, and a representation that reflects this can materially improve enzymatic reaction prediction. The proposed Docking-Aware Attention mechanism adds per-residue physical interaction scores, computed from sampled docking poses, to the attention logits of a protein encoder, so the embedding of a protein shifts with each molecular partner. On the ECREACT benchmark the method reports top-5 accuracy of 71.48% overall, 62.2% on complex molecules, and 55.54% on innovative reactions, outperforming static-representation baselines and EC-number tokens. If this holds, context-dependent embeddings could be a general ingredient for biocatalysis planning and other protein-molecule tasks.","feed_headline":"Docking-aware attention lifts enzymatic reaction prediction","feed_subtitle":"Per-substrate protein embeddings push top-5 accuracy to 62.2% on complex and 55.5% on novel reactions.","key_machinery":"The central object is the Docking-Aware Attention operator, defined as $\\text{Attention}(Q,K,V,S) = \\mathrm{softmax}\\left(\\frac{QK^T + \\gamma S}{\\sqrt{d}}\\right)V$, where $S$ is a smoothed, pose-averaged per-residue Lennard-Jones interaction profile of the protein–molecule pair and $\\gamma$ is a learned balance. It combines fixed learned attention patterns with a physical signal that varies by molecule, and it is what generates the paper's substrate-dependent protein representation.","core_discovery":"The central discovery is that biasing the attention mechanism with docking-derived interaction scores turns a fixed protein embedding into a context-dependent one: for each protein–molecule pair, the model samples several binding poses, averages per-residue Lennard-Jones potentials across them, smooths the profile, and adds it (with a learned weight $\\gamma$) to the pre-attention logits. The result is that the same protein receives different representations for different molecules, and those dynamic representations are what the paper credits for the reported gains, particularly on chemically complex or unseen reactions.","pith_inferences":["A cheap learned surrogate for the docking sampler could test how much of the gain comes from the physical scores versus the attention architecture's ability to condition on any per-residue signal.","The same mechanism should transfer to predicting protein-ligand binding affinity or enzyme substrate specificity, where the representation must likewise be partner-dependent.","Using a differentiable interaction module instead of sampled Lennard-Jones scores could make the physical bias trainable end-to-end and remove the need for a separate docking model at inference."],"forward_implications":["Substrate-dependent protein embeddings give the largest reported gains exactly where static models struggle: molecules with high Bertz complexity and reactions with novel products.","The docking-aware mechanism improves every base protein encoder tested (ESM3, ProtBERT, GearNet), suggesting the benefit transfers across embedding architectures.","Injecting the dynamic representation as a new token in the sequence-to-sequence model outperforms concatenation or addition, so how context is integrated matters.","The learned attention patterns are interpretable and differ for the same enzyme across reactions, which could make model predictions easier to audit.","The open-sourced code and models make it possible to extend DAA to other biocatalysis prediction settings, such as synthesis planning."],"supporting_citations":[{"why":"Supplies the sampled binding poses whose interatomic distances seed the Lennard-Jones interaction scores.","marker":"[6]"},{"why":"Provides the per-residue protein embeddings that Docking-Aware Attention pools into a context-dependent representation.","marker":"[14]"},{"why":"Contributes the ECREACT dataset of enzymatic reactions on which all models are trained and evaluated.","marker":"[25]"},{"why":"Establishes the transformer-based enzymatic reaction prediction task and provides the Chemical-Only baseline.","marker":"[18]"},{"why":"The strongest static-embedding baseline that DAA is compared against.","marker":"[30]"},{"why":"Defines the Bertz complexity index used to split the complex-molecule test set.","marker":"[2]"},{"why":"The structure-based static-embedding baseline, showing the limitation of fixed structural representations.","marker":"[37]"},{"why":"The sequence-based static-embedding baseline, helping to isolate the effect of context-awareness.","marker":"[3]"}],"fun_headline_variants":["Docking-aware attention adapts proteins per molecule","Protein embeddings that shift with the substrate","Docking-aware attention ups enzyme accuracy to 62% on complex molecules","Context-aware protein embeddings boost reaction forecasts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The sampled docking poses for the enzyme–substrate pairs in the dataset are accurate enough that the per-residue Lennard-Jones scores genuinely reflect which residues interact with the molecule.","fun_headline_variants_meta":{"raw":{"variants":["Docking-aware attention adapts proteins per molecule","Protein embeddings that shift with the substrate","Docking-aware attention ups enzyme accuracy to 62% on complex molecules","Context-aware protein embeddings boost reaction forecasts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000983,"raw_usage":{"total_tokens":4150,"prompt_tokens":899,"completion_tokens":3251,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":3191}},"tokens_in":515,"tokens_out":3251,"duration_ms":21191,"temperature":1.0,"reasoning_tokens":3191,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:12:27.272360+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the docking-derived interaction scores with random per-residue values while keeping the attention architecture and training protocol identical; if top-5 accuracy on complex molecules stays near 62.2% rather than falling to the static-embedding baseline, the docking signal itself is not the cause of the reported improvement.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the per-residue protein embeddings that Docking-Aware Attention pools into a context-dependent representation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the ECREACT dataset of enzymatic reactions on which all models are trained and evaluated."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the transformer-based enzymatic reaction prediction task and provides the Chemical-Only baseline."},{"cited_title":"ReactEmbed: A Plug-and-Play Module for Unifying Protein-Molecule Representations Guided by Biochemical Reaction Networks","cited_arxiv_id":"2501.18278","evidence_quote":"The strongest static-embedding baseline that DAA is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Bertz complexity index used to split the complex-molecule test set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The structure-based static-embedding baseline, showing the limitation of fixed structural representations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The sequence-based static-embedding baseline, helping to isolate the effect of context-awareness."}],"review_version":1}