{"id":"3ba945db-163f-4a38-9d0b-23af8dc98860","arxiv_id":"2502.02371","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A pocket-prediction network with soft labels, ReLU output, and a five-model ensemble guides AutoDock Vina to 54.9% PoseBusters-valid top poses, beating DiffBindFR and approaching AlphaFold 3 on a time-split benchmark.","lead":"RAPID-Net is a deep learning tool that predicts where small molecules bind on proteins and feeds the predicted pockets to the AutoDock Vina docking program. On the PoseBusters benchmark it beats the DiffBindFR blind-docking method (54.9% vs 49.1% valid top poses) at a fraction of the compute, and it approaches AlphaFold 3 while using far fewer resources.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No train/test overlap analysis for PoseBusters/Astex; sc-PDB-derived training set likely includes classic PDB test entries, which could inflate reported docking accuracy.","rationale":"I read the paper in good faith: the work is a solid empirical contribution with a clear protocol, public code, and an honest disclosure that all evaluations use holo structures. The strongest claim—that pocket identification is a decisive driver and pose ranking is the bottleneck—is supported by the high sampling accuracy (92.2%) and the comparison against PUResNet/Kalasanty when all guide the same Vina engine. However, I found a more fundamental threat to the benchmark numbers. The training set is sc-PDB, a PDB-derived database. The Astex Diverse Set is a classic PDB-derived benchmark from 2007; it is very likely that many of its 85 entries are also present in sc-PDB. The paper explicitly excludes training overlaps for Coach420 and BU48 but is silent for Astex and PoseBusters. Without this exclusion, the reported accuracies could reflect the model having seen the exact binding sites during training. This is not a matter of interpretation or scope—it is a checkable data-hygiene issue that determines whether the headline numbers are evidence of generalization. The holo-structure limitation, by contrast, is disclosed and scoped as future work; it affects practical transfer but not the internal validity of the experiments. I therefore disagree with the reader's choice of weakest assumption, while agreeing that the paper should be CONDITIONAL: it must provide an overlap analysis and, if overlaps exist, re-evaluate on the disjoint subsets. If the overlap is negligible, the concern does not land and the original verdict stands.","tokens_in":24826,"tokens_out":11568,"duration_ms":106926,"concrete_test":"Download the curated sc-PDB training set from the public repository and the PoseBusters/Astex/Coach420/BU48 structures; compute overlap by PDB ID and by 90% sequence identity (as in the Tanimoto filter). For Astex and PoseBusters, rerun the full RAPID-Net-guided Vina protocol on the non-overlapping subset and compare Top-1 PB-valid rates to the published numbers. If the rates drop materially (e.g., >3 points) or the margin over DiffBindFR disappears, the benchmark results are contaminated and the central claim needs revision.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that precise pocket identification drives docking success rests on the PoseBusters and Astex benchmark numbers. However, the paper never states that the sc-PDB training set was cleaned of test-set overlap for these benchmarks. sc-PDB is a PDB-derived database of ligandable binding sites; the Astex Diverse Set (2007) is a curated subset of the PDB and almost certainly shares entries with sc-PDB. The authors demonstrate awareness of this issue in Section IX, where they exclude training-set structures for Coach420 and BU48, but no such exclusion is reported for Astex (Section VIII) or PoseBusters (Section VII). If any of these test complexes appear in training, the reported Top-1 accuracies, RMSD distributions, and PLI values are inflated by memorization. The PoseBench time split (structures deposited after 30 Sep 2021) is less exposed if the training snapshot predates it, so the AF3 comparison (53.1% vs 59.5%) may be robust, but the headline 54.9% PoseBusters and all Astex numbers are suspect. This is a concrete, checkable threat to the central claim—more fundamental than the acknowledged holo-structure limitation, which at least is disclosed.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RAPID-Net, a five-member ensemble of 3D U-Net-style convolutional networks for protein pocket prediction, trained on cavity labels from sc-PDB with soft Dice loss and ReLU output. Predictions are converted into AutoDock Vina search grids using hand-set expansion thresholds, and docking is evaluated on PoseBusters, Astex, Coach420, and BU48. The central claim is that precise pocket identification is the decisive factor in blind docking accuracy: RAPID-Net-guided Vina achieves 54.9% Top-1 PoseBusters-valid poses on PoseBusters versus 49.1% for DiffBindFR, 53.1% versus 59.5% for AlphaFold 3 on the PoseBench time split, and 92.2% sampling accuracy (at least one pose with RMSD<2 Å). The paper also reports per-run variability, PLI metrics, and qualitative allosteric-site case studies.","tokens_in":25050,"tokens_out":4827,"duration_ms":49708,"significance":"If the empirical results withstand scrutiny, the paper makes a useful practical contribution: a lightweight pocket predictor that can be plugged into standard docking pipelines, with public code and reproducible notebooks. The explicit separation of Top-1 accuracy from sampling accuracy is a valuable diagnostic, and the comparison against recent blind-docking and co-folding tools is informative. The main significance depends on the integrity of the benchmark evaluation, because the headline comparisons are the primary evidence for the claim that pocket identification, rather than pose ranking or receptor flexibility, is the bottleneck.","major_comments":[{"comment":"The manuscript reports no train/test overlap analysis for the PoseBusters and Astex benchmarks, although sc-PDB (Ref. 81) is the training set and Astex (Ref. 33) is a curated PDB-derived set. Section IX explicitly excludes training-set structures for Coach420 and BU48, but no equivalent exclusion or check is described for PoseBusters or Astex. If any Astex or older PoseBusters complexes appear in sc-PDB, the reported Top-1 accuracies, RMSD distributions, and PLI values in Sections VII–VIII would be inflated by memorization. The PoseBench time split (deposited after 30 September 2021) is less exposed if the training snapshot predates it, but the headline 54.9% PoseBusters number and the Astex numbers need a concrete overlap analysis or a revised, more cautious claim.","section":"Sections VII–VIII, Tables I–II"},{"comment":"The docking protocol depends on hand-set grid expansion thresholds of 2/5 Å for majority-voted pockets and 2/5/10/15 Å for minority-reported pockets. The 8FAV example in Fig. 5 shows that docking can succeed even when no predicted pocket overlaps the true ligand pose, purely because the expanded search grid covers the site. The paper does not report an ablation varying these thresholds or a control with a fixed large grid, so the central claim that precise pocket identification is the decisive driver of docking success is not cleanly separated from the effect of grid enlargement. Please report results for at least one alternative threshold set and clearly state the sensitivity of the headline metrics to this protocol choice.","section":"Section V, Fig. 5"},{"comment":"The comparison to DiffBindFR and AlphaFold 3 confounds pocket quality with the docking engine and protocol. No unguided AutoDock Vina baseline (docking over the whole protein with the same Exhaustiveness and num_modes settings) is reported. Since the central claim is that pocket identification drives docking success, the reader needs to see RAPID-Net-guided Vina versus unguided Vina under otherwise identical settings; without this baseline, part of the 54.9% result could reflect Vina's search behavior rather than RAPID-Net's pockets. The 'prior knowledge' Vina result of 93.8% sampling accuracy in Section VII is not an adequate substitute because it uses the true ligand coordinates to define the search box.","section":"Sections V and VII, Fig. 6"},{"comment":"The headline differences (54.9% vs 49.1% and 53.1% vs 59.5%) are reported without confidence intervals, bootstrap estimates, or repeated-run statistics at the docking level. The five RAPID-Net runs in Tables I–IV and Figs. 6 and 18 show substantial run-to-run variability in pocket coverage and PLI, so the ensemble docking accuracy may also vary with the random seeds or training runs. Please provide variability estimates for the primary Top-1 and sampling-accuracy metrics, or at least state explicitly whether the docking results were obtained from a single ensemble checkpoint.","section":"Section VII, Figs. 6 and 8"}],"minor_comments":[{"comment":"The notation max(x,y,z) and min(x,y,z) is ambiguous: these should be coordinate-wise extrema over the pocket atoms, not a scalar maximum or minimum of a single triple. Please clarify by writing e.g. x_max = max_i x_i, with analogous expressions for y and z.","section":"Eqs. (4)–(5)"},{"comment":"Section VI says the Top-1 RMSD is computed 'between the predicted and one of the true ligand poses if multiple true poses are available,' while Fig. 7's caption says RMSD to the closest one is reported. Please make the metric definition consistent throughout.","section":"Section VI and Fig. 7"},{"comment":"The descriptive claims about allosteric sites, exosites, and bridges are qualitative and based on visual inspection; please add a quantitative measure, such as residue-level overlap with annotated exosite or allosteric-site residues, to support these statements.","section":"Section X and Fig. 27"},{"comment":"The paper states in the introduction that all evaluations are performed on holo structures, but this important limitation is not restated in the metrics or benchmark sections; please state it explicitly alongside the PoseBusters and Astex results.","section":"Section VI"},{"comment":"Reference [85] contains the placeholder 'Accessed: YYYY-MM-DD'; please replace it with the actual access date.","section":"References"},{"comment":"The term 'minimally-reported pockets' in the Fig. 27 caption should be 'minority-reported pockets' for consistency with the rest of the text.","section":"Fig. 27 caption"}],"recommendation":"major_revision","confidential_remarks":"The train/test overlap concern is the most consequential issue. If the authors can demonstrate that sc-PDB was filtered against Astex and PoseBusters entries, the remaining issues (unguided baseline, threshold sensitivity, error bars) could be addressed in a revision. If the overlap is present, the central empirical claim would need substantial qualification and likely re-benchmarking."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"RAPID-Net is a practical, well-motivated pocket predictor: U-Net with soft Dice loss, ReLU output, a single SE block, and a five-model ensemble, integrated into a docking pipeline with multiple grid thresholds. The new ideas are small but real—the soft-label training is a sensible adaptation from medical imaging, and the ensemble-plus-threshold protocol is a clean way to convert pocket predictions into search boxes. The paper also ships code and data, which counts.\n\nThe strongest empirical contribution is the sampling-versus-ranking distinction. Showing that RAPID-Net-guided Vina finds a pose under 2 Å in 92.2% of cases, while Top-1 accuracy is much lower, is a useful diagnostic and gives the field a concrete target for rescoring methods.\n\nThe soft spot that matters: no train/test overlap check for PoseBusters or Astex. The authors exclude training-set structures for Coach420 and BU48, so they know about the issue. But sc-PDB is a PDB-derived database, and both PoseBusters and the Astex Diverse Set are curated PDB subsets. Without stating which sc-PDB release was used and removing overlapping entries, the 54.9% PoseBusters and all Astex numbers could be inflated by memorization. This is a concrete, checkable threat to the central claim. The PoseBench time split is less exposed if the training snapshot predates September 2021, but the paper doesn't say so, so even that comparison is not fully clean.\n\nOther gaps are less severe: no unguided Vina baseline, so the contribution of pocket prediction is not isolated; no error bars on the headline docking accuracies; and the holo-only evaluation is disclosed but limits the binding-site-agnostic framing. The qualitative distal-site claims are interesting but need quantitative backup.\n\nThe paper deserves a serious referee: the method is plausible, the code is public, and the question matters. But a referee should require the overlap analysis and an unguided Vina baseline before the benchmark claims can be accepted. I'd bring it to reading group only after those are on the table.","headline":"Solid pocket-guided docking work with a useful ranking-vs-sampling insight, but the headline benchmarks need a train/test overlap check before I'd trust the numbers.","tokens_in":25575,"tokens_out":3627,"would_cite":false,"duration_ms":35650,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RAPID-Net claims that precise binding-pocket identification, not ligand sampling or receptor flexibility, is the decisive factor in blind docking, and shows that a lightweight pocket predictor guiding AutoDock Vina beats DiffBindFR and…","keywords":["Protein-ligand interactions","Blind docking","Binding pocket prediction","Soft mask segmentation","Deep residual network","Convolutional neural network","PoseBusters benchmark","Ensemble pocket prediction"],"falsifier":"Run RAPID-Net plus AutoDock Vina on the PoseBusters complexes with each holo receptor replaced by its apo or AlphaFold-predicted structure, keeping the same pocket-to-grid protocol; if the Top-1 PoseBusters-valid rate falls far below 54.9%, the claim that pocket identification is the decisive driver would be shown to depend on seeing the bound pocket.","tokens_in":24638,"feed_emoji":"🧬","tokens_out":9410,"duration_ms":91079,"temperature":0.7,"pith_summary":"This paper argues that in binding-site-agnostic docking, the deciding factor is not how the ligand is sampled or how flexible the receptor is, but how precisely the binding pocket is located. To make that case, it introduces RAPID-Net, a deep residual network that predicts ligand-binding pockets as soft voxel masks and hands the resulting search box to AutoDock Vina. Guided by RAPID-Net, Vina reaches 54.9% Top-1 PoseBusters-valid poses on the PoseBusters benchmark, beating the flexible-receptor blind-docking tool DiffBindFR (49.1%), and 53.1% on the hard time split, within six points of AlphaFold 3 (59.5%) at a fraction of the cost. The authors also report that in 92.2% of cases at least one pose in the ensemble is correct, which they read as evidence that pose ranking, not sampling, is the main remaining bottleneck. If correct, the result makes accurate pocket finding the highest-leverage component of a practical blind-docking pipeline.","feed_headline":"Pocket finder lifts blind docking to 54.9% valid poses","feed_subtitle":"A lightweight pocket predictor guides Vina to near-AlphaFold 3 accuracy, cheaply enough for large-scale screening.","key_machinery":"The central object is RAPID-Net, a soft-segmentation 3D U-Net-style convolutional network that predicts a continuous per-voxel score for ligand-binding propensity rather than a binary pocket mask. Three design choices carry the argument: a ReLU activation in the final layer instead of sigmoid, a soft Dice loss based on L2 norms, and threshold-less training labels from VolSite/sc-PDB that extend beyond the 6.5 Å shell used by earlier predictors. Five independently trained replicas are combined by majority voting for high-confidence pockets and by 'minority-reported' union for recall; the resulting voxel sets define docking search grids of expandable size, which is what links pocket quality directly to downstream docking accuracy.","core_discovery":"The central claim is that precise pocket identification is the decisive driver of docking success in binding-site-agnostic settings, and that a lightweight, voxel-based pocket predictor can supply that precision to a standard rigid-receptor docking engine. RAPID-Net is a five-model ensemble of 3D U-Net-like convolutional networks trained on sc-PDB cavity labels; each model outputs a soft occupancy mask for 2 Å voxels, with a ReLU in the final layer and a soft Dice loss so the network learns pocket interiors versus boundaries rather than a binary yes/no. The predicted pockets are converted into search grids centered on the pocket, with size thresholds that expand to accommodate large ligands, and AutoDock Vina performs targeted docking on each grid. On PoseBusters, this scheme yields 54.9% Top-1 PB-valid poses versus 49.1% for DiffBindFR; on the PoseBench time split it reaches 53.1% versus 59.5% for AlphaFold 3; and in 92.2% of cases at least one sampled pose is within 2 Å RMSD, regardless of rank. The paper reads these numbers as showing that pocket localization currently outweighs receptor flexibility as the accuracy bottleneck, and that better pose reranking would be the next largest gain.","pith_inferences":["The paper does not report it, but a direct experiment is available: replacing holo receptors in PoseBusters with apo or AlphaFold-predicted structures would show how much of the 54.9% success depends on seeing the bound-state pocket.","If the sampling-versus-ranking gap holds, adding a learned rescoring function on top of RAPID-Net's own pockets should push Top-1 accuracy toward the 92.2% sampling ceiling, a much larger gain than further pocket refinement.","The minority-voted pockets, which recover allosteric and exosite regions with no direct ligand contact, could serve as a candidate generator for cryptic allosteric sites in drug discovery, a direction the paper mentions but does not develop.","Because grid thresholds are expanded to 15 Å for minority pockets, part of the docking success may come from generous search boxes; ablating threshold sizes per pocket would isolate how much of the gain comes from the precise pocket versus the enlarged grid."],"forward_implications":["If pocket localization is the dominant bottleneck, then upgrading the pocket predictor in any blind-docking pipeline—not only Vina-based ones—should transfer most of RAPID-Net's gain.","The 92.2% sampling accuracy implies an immediate ceiling: a rescoring function that selects the correct pose from the ensemble could raise Top-1 success toward that number, a gain larger than any reported pocket improvement.","Because RAPID-Net returns search grids rather than binding-site residues, the same predictions can be fed to flexible-receptor docking engines, which the paper argues would benefit from the same focused search.","Threshold-less training lets the model flag secondary and allosteric sites beyond the orthosteric pocket, so pocket-guided docking can be aimed at distal functional sites rather than only the main ligand site.","Large complexes that exceed co-folding model input limits, such as 8F4J, become dockable when the search space is reduced to a predicted pocket."],"supporting_citations":[{"why":"Supplies the docking engine whose sampling produces the pose ensembles evaluated throughout.","marker":"[12]"},{"why":"The flexible-receptor blind-docking baseline that RAPID-Net-guided Vina outperforms on PoseBusters.","marker":"[27]"},{"why":"Defines the PoseBusters benchmark, the RMSD<2 Å criterion, and the chemical-validity filter used for success.","marker":"[28]"},{"why":"Provides the PoseBench protocol and time-split subset used to compare with AlphaFold 3.","marker":"[31]"},{"why":"Voxel-based pocket prediction baseline and the training-data curation precedent (sc-PDB subset).","marker":"[69]"},{"why":"Segmentation-based pocket predictor used as a second baseline for docking and PLI comparisons.","marker":"[70]"},{"why":"sc-PDB supplies the training complexes and annotated binding sites.","marker":"[81]"},{"why":"VolSite defines the cavity pseudoatom labels used as soft training targets.","marker":"[82]"},{"why":"U-Net architecture is the base encoder-decoder structure that RAPID-Net adapts.","marker":"[76]"},{"why":"ReLU activation replaces sigmoid in the final layer, a core design change for soft pocket segmentation.","marker":"[90]"}],"fun_headline_variants":["RAPID-Net pocket guide pushes Vina past DiffBindFR","Lightweight pocket AI nearly matches AlphaFold 3 on docking","RAPID-Net uncovers allosteric pockets for targeted drug design","Cheap pocket predictor lifts blind docking to 54.9% valid poses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result rests on the assumption that pocket predictions trained and tested on ligand-bound (holo) structures will localize the right search region when the receptor is unbound or computationally predicted, where the pocket need not be pre-formed.","fun_headline_variants_meta":{"raw":{"variants":["RAPID-Net pocket guide pushes Vina past DiffBindFR","Lightweight pocket AI nearly matches AlphaFold 3 on docking","RAPID-Net uncovers allosteric pockets for targeted drug design","Cheap pocket predictor lifts blind docking to 54.9% valid poses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00042,"raw_usage":{"total_tokens":2284,"prompt_tokens":1191,"completion_tokens":1093,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":807,"completion_tokens_details":{"reasoning_tokens":1014}},"tokens_in":807,"tokens_out":1093,"duration_ms":10125,"temperature":1.0,"reasoning_tokens":1014,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T12:23:31.426177+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RAPID-Net plus AutoDock Vina on the PoseBusters complexes with each holo receptor replaced by its apo or AlphaFold-predicted structure, keeping the same pocket-to-grid protocol; if the Top-1 PoseBusters-valid rate falls far below 54.9%, the claim that pocket identification is the decisive driver would be shown to depend on seeing the bound pocket.","supporting_citations":[{"cited_title":"P.; Tiwary, P","cited_arxiv_id":null,"evidence_quote":"Voxel-based pocket prediction baseline and the training-data curation precedent (sc-PDB subset)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Segmentation-based pocket predictor used as a second baseline for docking and PLI comparisons."},{"cited_title":"Enhancing Open-World Bacterial Raman Spectra Identification by Feature Regularization for Im- proved Resilience against Unknown Classes.Chemical & Biomedical Imaging2024,2, 442–452","cited_arxiv_id":null,"evidence_quote":"sc-PDB supplies the training complexes and annotated binding sites."},{"cited_title":"sc- PDB: a 3D-database of ligandable binding sites—10 years on.Nucleic Acids Research2015,43, D399– D404","cited_arxiv_id":null,"evidence_quote":"VolSite defines the cavity pseudoatom labels used as soft training targets."},{"cited_title":"PGBind: pocket-guided explicit attention learning for protein– ligand docking.Briefings in Bioinformatics2024,25, bbae455","cited_arxiv_id":null,"evidence_quote":"U-Net architecture is the base encoder-decoder structure that RAPID-Net adapts."},{"cited_title":"Focal Loss for Dense Object Detection","cited_arxiv_id":null,"evidence_quote":"ReLU activation replaces sigmoid in the final layer, a core design change for soft pocket segmentation."}],"review_version":1}