{"id":"5d8cfb97-fb70-4d30-950e-4d015423f376","arxiv_id":"2507.06366","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Pre-training graph neural networks on DecoyDB, a 5.4-million-pose decoy dataset with RMSD annotations, improves protein-ligand binding affinity prediction accuracy, label efficiency, and generalization.","lead":"This paper introduces DecoyDB, a dataset of over 5.3 million computationally generated protein-ligand binding poses, each labeled by how far it deviates from the true binding structure. The authors show that pre-training graph neural networks on this dataset improves their downstream binding affinity predictions, especially when affinity labels are scarce.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Generalization claim may be leaky: §5.5 never states LP-PDBbind test complexes were excluded from DecoyDB pretraining, so Table 4's large gain could be memorization rather than generalization.","rationale":"The reader's weakest assumption—RMSD as a monotonic proxy for binding quality—is a real scientific concern, but it is not the single most load-bearing one. The paper's own ablation (Table 3) shows that the continuous two-category contrastive signal provides only a small benefit over a one-category loss, so the headline empirical gains could survive even if the RMSD weighting is imperfect. In contrast, the LP-PDBbind generalization experiment is the sole support for the 'generalizability' component of the central claim, and the manuscript contains an omitted verification: it never states that the LP-PDBbind test split was removed from DecoyDB pretraining. The dataset description only mentions removing overlaps with the fine-tuning binding affinity datasets and the PDBbind coresets. Because LP-PDBbind is a reorganization of PDBbind and DecoyDB is built from PDB, the overlap is not guaranteed to be empty, and the paper provides no formal or data-level check. This is not an accusation of intentional leakage; it is a missing statement and a missing check. The concrete test—computing the overlap and, if necessary, rerunning the experiment without overlapping pretraining data—directly settles the question. If the overlap is zero, the concern does not land and the reader's conditional verdict can stand. If the overlap is nonzero, the generalizability claim is unsupported and the overall verdict would need to move toward rejection or at least major revision. Because the concern is currently unresolved rather than demonstrated, keeping the CONDITIONAL verdict is appropriate, so the reader's verdict does not need to change in label; it does need this additional condition to be checked.","tokens_in":16239,"tokens_out":11167,"duration_ms":131084,"concrete_test":"Download DecoyDB ground-truth metadata (PDB ID, chain, ligand identifier) and the LP-PDBbind split metadata, or recompute the LP-PDBbind test split from PDBbind using the LP-PDBbind protocol. Compute exact overlap between DecoyDB complexes and the LP-PDBbind test set at the level of PDB ID + chain + ligand identifier. If overlap is nonzero, rerun the §5.5 experiment with all overlapping test complexes removed from DecoyDB pretraining; if the pre-train vs. no-pre-train RMSE gap in Table 4 collapses, the generalizability claim is an artifact. If overlap is zero, or the gap persists after removal, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim explicitly includes 'generalizability' (abstract, §5.5, Table 4), and the only evidence for it is the LP-PDBbind experiment. In §5 (Dataset description) the authors say they 'removed overlapping samples from fine-tuning binding affinity datasets' and specifically that the PDBbind2013/2016 coresets 'are also removed from our pre-training dataset.' Nowhere do they state that the LP-PDBbind split—a reorganized PDBbind dataset, likely based on a different PDBbind version than the fine-tuning sets used for Table 2—was excluded from DecoyDB. Since DecoyDB contains 61,104 PDB-derived complexes and 5.3M decoys, and LP-PDBbind is derived from PDBbind, overlap is structurally possible. If any LP-PDBbind test complex appears in DecoyDB (as a real complex or as the source of decoys), pretraining has already seen that test structure, and the Table 4 improvement (1.496 → 1.371) would reflect leakage, not better generalization. This is more decisive than the RMSD-monotonicity concern because it attacks the evaluation itself: a leak would invalidate one of the three headline claims regardless of how well the decoy signal works.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DecoyDB, a large-scale dataset of 61,104 protein-ligand complexes drawn from the PDB and 5,353,307 AutoDock Vina-generated decoy poses annotated with RMSD to the native ligand pose. It also proposes a graph contrastive pretraining framework that combines a two-category contrastive loss (with continuous negative weighting based on decoy RMSD and discrete negatives from other real complexes) with a denoising score matching (DSM) regularization. The pretrained encoders (GIGN, EGNN, SchNet, TorchMD-Net) are fine-tuned on PDBbind and evaluated on the PDBbind 2013/2016 coresets and the leakage-proof LP-PDBbind split. The paper claims consistent improvements in prediction accuracy, label efficiency, and generalizability from DecoyDB pretraining.","tokens_in":16445,"tokens_out":6618,"duration_ms":69435,"significance":"The dataset is a potentially valuable community resource: it is substantially larger than existing decoy-augmented pretraining sets, includes continuous RMSD annotations that enable finer-grained contrastive supervision, and is released with code and data. If the evaluation is demonstrated to be leakage-free and the loss functions are correctly specified, the reported gains across multiple GNN backbones and the label-efficiency results would constitute a useful contribution to structure-based binding-affinity prediction. However, the generalization claim currently rests on an unverified overlap assumption for LP-PDBbind, and several central equations are not correctly specified as written, so the results cannot be fully assessed in the present form.","major_comments":[{"comment":"The generalization claim in Table 4 rests entirely on the LP-PDBbind split, but the manuscript never states that LP-PDBbind complexes (or the decoys derived from them) were excluded from DecoyDB pretraining. The dataset description only reports removing overlapping samples from the PDBbind2016/2013 fine-tuning sets and removing the two coresets from pretraining. Since DecoyDB is built from PDB structures and LP-PDBbind is a reorganized PDBbind-derived split, structural overlap is plausible; if any LP-PDBbind test complex appears in DecoyDB, the Table 4 improvement (1.496 to 1.371) could reflect memorization rather than generalization. Please provide an explicit overlap analysis between DecoyDB and the LP-PDBbind train/validation/test splits, and remove any overlapping complexes from pretraining before reporting the generalization numbers.","section":"§5.5 and §5 (Dataset description)"},{"comment":"Figure 3 and the surrounding text report RMSE 'across the test datasets' while varying hyperparameters α and μ over the range 0.4 to 2.0. If α and μ were chosen by minimizing test-set RMSE, then the final Table 2 results for OURS are not independent test evaluations, and the comparison against baselines is unfair because the baselines are not similarly selected on the test coresets. The authors must state clearly whether α and μ were selected on the validation split (the 1,000-complex held-out set) or on the test coresets; if the latter, the experimental protocol must be revised, for example by selecting on validation and re-reporting the test numbers.","section":"§5.2, Figure 3"},{"comment":"Equation (2) defines β_{k,j} using d_{z_k,z_j}, which the text describes as 'the RMSD between embeddings z_k and z_j.' However, the RMSD computed in §4.1 is a spatial distance between ligand poses in input space, and it has no meaning for embedding vectors. The continuous negative weighting should reference the decoy's RMSD to the native pose (for example, d_{s_k,s_j} in input space), not an embedding distance. As written, the equation is not implementable and conflates input-space geometry with embedding-space geometry.","section":"§4.2, Eq. (2)"},{"comment":"Equation (4) defines the denoising score matching loss as the expectation of the vector difference ∂log f(s′)/∂x′ − (x − x′)/σ², with no squared norm. This is a vector-valued quantity and cannot serve as a scalar loss; standard DSM is E[||∂log f(s′)/∂x′ − (x − x′)/σ²||²]. The missing squared L2 norm is a mathematical error in a central component of the proposed method. Please correct the equation and confirm that the implementation matches the corrected form.","section":"§4.2, Eq. (4)"},{"comment":"Equation (1) is not the InfoNCE loss as claimed: the denominator sums over j ≠ i, excluding the positive sample i, whereas standard InfoNCE includes the positive sample in the denominator to normalize the softmax over all samples. Additionally, the expression 'sim(z_k,z_i^T/τ)' is dimensionally ill-formed; it should presumably be sim(z_k,z_i)/τ. Please restate the loss precisely and clarify whether the implementation follows the equation as written or the standard InfoNCE form, since this changes the training objective.","section":"§4.2, Eq. (1)"}],"minor_comments":[{"comment":"The superscript T on z_i in Eq. (1) is misplaced; the similarity should be written as sim(z_k, z_i)/τ.","section":"§4.2, Eq. (1)"},{"comment":"There is a duplicated word in 'by by randomly selecting'; it should read 'by randomly selecting.'","section":"§5, Dataset description"},{"comment":"The caption states that panels (a) and (c) show μ while panels (b) and (d) show α, but the panel labels and axis values could be misaligned; please clarify which panel corresponds to which hyperparameter and which dataset.","section":"Figure 3"},{"comment":"The RMSD calculation is described only as 'the spatial distances between the atoms of the decoy’s ligand and the corresponding atoms in the original ligand'; please specify whether this is computed over heavy atoms, how symmetry/atom-mapping ambiguities are handled, and whether any alignment is performed before the RMSD is calculated.","section":"§4.1"},{"comment":"The description of ConBAP states that it uses binary negative samples from decoys, but no ablation is reported that replaces the proposed continuous RMSD weighting with binary (unweighted) decoy negatives while keeping the same dataset; such an ablation would isolate the benefit of continuous weighting.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a machine-learning-for-drug-discovery venue, and the dataset release is a genuine contribution. The primary risk is evaluation leakage in the LP-PDBbind experiment; I would ask for a concrete overlap analysis before accepting the generalization claim. I would also ask the authors to verify that their implementation matches a corrected version of Eq. (1) and Eq. (4), since the printed equations appear to contain errors that make the method as described not fully implementable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: DecoyDB is a genuine resource and the continuous-RMSD contrastive loss is a real step beyond ConBAP, but the LP-PDBbind experiment may not be clean, and the hyperparameters look test-set tuned.\n\nWhat's new: 61K complexes from PDB, 5.35M Vina decoys with RMSD annotations, explicitly built outside PDBbind. That is useful. The two-category loss with alpha-weighted RMSD negatives is a sensible extension of the binary-decoy idea, and the DSM regularization, though underspecified, fits the local-minimum intuition. The main empirical claim on PDBbind coresets is supported: pretraining improves GIGN RMSE on both coresets, and the effect is larger with fewer labels. They ran five seeds and report standard deviations. That's solid work.\n\nSoft spots, in order of severity.\n\nFirst, the generalization claim. Section 5.5 reports a large gain on LP-PDBbind, but the paper never states that the LP-PDBbind test complexes were removed from DecoyDB. The coresets and fine-tuning sets are explicitly removed; LP-PDBbind is not mentioned. Since DecoyDB is built from PDB and LP-PDBbind is derived from PDBbind, overlap is structurally possible. As written, the Table 4 result could reflect memorization, not generalization. The authors need to state and verify exclusion. Until then, treat the generalizability claim as unproven.\n\nSecond, Figure 3 tunes alpha and mu directly on the PDBbind coresets. They vary the parameters and report RMSE on the test sets. That is test-set selection. It undermines the strength of the reported gains, though the gains over baselines are large enough that the main result probably survives.\n\nThird, technical details: Eq. (1) sums over j != i in the denominator without specifying whether positives are excluded; the beta weighting for positives is undefined. Eq. (4) is missing the norm squared in the DSM loss. And the score head and sigma are not described. These are fixable but need correction.\n\nWho should read this: groups working on structure-based affinity prediction and self-supervised learning for drug discovery. The dataset alone could be worth a citation.\n\nRecommendation: send it to review. The resource is valuable and the core results are plausible, but the leak check and the test-set tuning need to be addressed before the generalization claim is accepted.","headline":"A useful new decoy dataset and a plausible pretraining recipe, but the generalization claim needs a leak check and the hyperparameters look test-set tuned.","tokens_in":17074,"tokens_out":3282,"would_cite":true,"duration_ms":33726,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dataset of 5.4 million docking decoys, paired with a two-part contrastive loss, improves GNN-based protein-ligand binding affinity prediction across accuracy, label efficiency, and generalization.","keywords":["protein-ligand binding affinity prediction","graph contrastive learning","decoy structures","self-supervised pretraining","RMSD","AutoDock Vina","denoising score matching","PDBbind"],"falsifier":"Randomly permute the RMSD labels among decoys before computing the continuous weights in Eq. (2) while keeping everything else fixed; if test RMSE on the PDBbind 2013 and 2016 core sets does not move, the RMSD-based continuous weighting is not the source of the reported gains. A supporting check would measure whether Vina's docking score actually correlates with RMSD across DecoyDB decoys, since the graded negatives are meant to encode binding quality rather than geometry alone.","tokens_in":15990,"feed_emoji":"🧬","tokens_out":8893,"duration_ms":82732,"temperature":0.7,"pith_summary":"Protein-ligand binding affinity prediction is bottlenecked by scarce experimental labels: PDBbind has fewer than 20,000 labeled complexes. This paper claims that the bottleneck can be broken with self-supervised pretraining on unlabeled 3D structures, and proposes DecoyDB, a dataset of 61,104 high-resolution complexes augmented with 5,353,307 AutoDock Vina decoys, each annotated with RMSD from the native pose. It also proposes a contrastive pretraining objective with two negative categories: decoys weighted continuously by RMSD and other real complexes as binary negatives, plus a denoising score matching regularizer. Pretraining with this objective lowers GIGN's RMSE from 1.460 to 1.386 on the PDBbind 2013 core set and from 1.263 to 1.188 on the 2016 core set, and the gains are largest when fine-tuning labels are scarce. If true, this gives a practical route to using the far larger pool of unlabeled protein-ligand structures to improve drug-discovery scoring.","feed_headline":"Pretraining on 5.4M decoys cuts binding-prediction error ~6%","feed_subtitle":"A new dataset and contrastive loss let graph networks learn from unlabeled structures, with largest gains when labels are scarce.","key_machinery":"The carrying mechanism is a contrastive pretraining signal built from docking decoys. For each real complex anchor, near-native decoys (RMSD $\\le 2$ Å) act as positive pairs; far-from-native decoys are negative pairs whose contribution to the InfoNCE contrastive loss is weighted by normalized RMSD via $\\beta_{k,j} = \\alpha\\, d_{z_k,z_j}/d_{\\max}$ in Eq. (2), so worse poses are pushed farther from the anchor; other real complexes supply a second, binary negative category. A denoising score matching term adds Gaussian noise to ligand coordinates and trains the encoder to score the noised structure as if recovering the clean pose, enshrining the assumption that the native pose is an energy minimum. Together these two losses form $L = L_1 + \\mu L_2$, and this total objective is what pre-training optimizes before the regression head is fine-tuned on PDBbind labels.","core_discovery":"The paper's central claim is that a GNN encoder pre-trained on DecoyDB with this customized contrastive objective predicts binding affinity more accurately, with fewer labels, and on harder leakage-proof splits than the same encoder trained from scratch or with prior pretraining approaches. The claim is supported by consistent RMSE reductions across three base encoders—GIGN, EGNN, and SchNet—and by ablations showing that both the decoy-negative term and the denoising regularizer contribute, with the two-category loss slightly beating the one-category variant. The paper reads the result as evidence that structure-aware decoys, graded by RMSD, carry usable self-supervised signal about binding quality.","pith_inferences":["Because DecoyDB annotates every decoy with RMSD but not with an independent energy estimate, a natural untested extension is to re-weight negatives by a consensus of docking scores or by an energy function rather than by RMSD alone.","The dataset's scale (5.35M decoys over 61K complexes) could support other structure-based tasks such as pose ranking or docking-scoring benchmarks, although the experiments here only evaluate affinity prediction.","The improvement pattern suggests the method might transfer to other GNN architectures beyond the three tested, since the loss operates on embeddings rather than on architecture-specific features; this remains untested.","A specific testable extension is to shuffle decoy RMSD labels during pretraining: if core-set RMSE does not change, the continuous weighting is not the active ingredient."],"forward_implications":["Pre-trained models should be preferred over from-scratch training for binding affinity prediction when experimental labels are limited, since the reported gains are largest at 2,000–6,000 fine-tuning labels.","DecoyDB can serve as a pretraining corpus that is disjoint from PDBbind fine-tuning data, removing the leakage that previous decoy-based pretraining built from PDBbind complexes suffered.","The continuous RMSD weighting matters: replacing the two-category loss with a one-category loss raises RMSE slightly on both core sets, so grading negatives by deviation adds signal.","The DSM regularization term is not redundant with contrastive learning; removing it degrades performance, so both objectives should be kept in the pretraining loss.","Generalization to a leakage-proof split improves more than to a random split, implying the pretraining learns transferable interaction structure rather than memorizing benchmark biases."],"supporting_citations":[{"why":"Supplies AutoDock Vina 1.2, the docking program used to generate all 5,353,307 decoy poses in DecoyDB.","marker":"[Eberhardt et al., 2021]"},{"why":"Supplies the PDBbind fine-tuning labels and the two core sets (2013, 2016) used to measure RMSE and R.","marker":"[Wang et al., 2005]"},{"why":"Supplies GIGN, the main base encoder whose RMSE drops from 1.460 to 1.386 and from 1.263 to 1.188 after pretraining.","marker":"[Yang et al., 2023]"},{"why":"ConBAP is the closest prior decoy-based contrastive pretraining method and the baseline that the proposed framework must outperform on EGNN.","marker":"[Luo et al., 2024]"},{"why":"Provides the denoising score matching loss used as the DSM regularization term in Eq. (4).","marker":"[Zaidi et al., 2023]"},{"why":"Provides the LP-PDBbind leakage-proof split used to evaluate generalizability after pretraining.","marker":"[Li et al., 2024a]"},{"why":"Supplies Redocked2020, the decoy-augmented pretraining dataset used by the ConBAP baseline and the main existing comparison corpus.","marker":"[Francoeur et al., 2020]"},{"why":"Cited as the inspiration for treating the native complex as an energy minimum and adding denoising-based regularization.","marker":"[Jin et al., 2024]"}],"fun_headline_variants":["5.4M decoys sharpen protein-ligand affinity predictions","DecoyDB: self-supervised decoys cut binding error by 6%","Graph contrastive pretraining on decoys boosts label efficiency","New decoy dataset and loss improve binding affinity models","Pretrain on decoys, predict binding with fewer labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pre-training signal rests on the assumption that AutoDock Vina's generated poses are realistic enough to stand in for true binding and that a decoy's RMSD from the experimentally observed pose reliably tracks how poor its binding is; if a pose far from the native geometry can still bind well, or a pose close to it binds poorly, the contrastive labels mislead the encoder.","fun_headline_variants_meta":{"raw":{"variants":["5.4M decoys sharpen protein-ligand affinity predictions","DecoyDB: self-supervised decoys cut binding error by 6%","Graph contrastive pretraining on decoys boosts label efficiency","New decoy dataset and loss improve binding affinity models","Pretrain on decoys, predict binding with fewer labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00027,"raw_usage":{"total_tokens":1631,"prompt_tokens":959,"completion_tokens":672,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":585}},"tokens_in":575,"tokens_out":672,"duration_ms":7239,"temperature":1.0,"reasoning_tokens":585,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:07:07.310361+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Randomly permute the RMSD labels among decoys before computing the continuous weights in Eq. (2) while keeping everything else fixed; if test RMSE on the PDBbind 2013 and 2016 core sets does not move, the RMSD-based continuous weighting is not the source of the reported gains. A supporting check would measure whether Vina's docking score actually correlates with RMSD across DecoyDB decoys, since the graded negatives are meant to encode binding quality rather than geometry alone.","supporting_citations":[{"cited_title":"Autodock vina 1.2","cited_arxiv_id":null,"evidence_quote":"Supplies AutoDock Vina 1.2, the docking program used to generate all 5,353,307 decoy poses in DecoyDB."},{"cited_title":"The pdbbind database: methodologies and updates","cited_arxiv_id":null,"evidence_quote":"Supplies the PDBbind fine-tuning labels and the two core sets (2013, 2016) used to measure RMSE and R."},{"cited_title":"Pre-training via denoising for molecular property prediction","cited_arxiv_id":null,"evidence_quote":"Provides the denoising score matching loss used as the DSM regularization term in Eq. (4)."},{"cited_title":"Three-dimensional convolutional neural networks and a cross-docked data set for structure-based drug design","cited_arxiv_id":null,"evidence_quote":"Supplies Redocked2020, the decoy-augmented pretraining dataset used by the ConBAP baseline and the main existing comparison corpus."}],"review_version":1}