{"id":"5e2444d3-66a4-4c79-8638-6c50be60e578","arxiv_id":"2510.24380","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"APEX factorizes a neural network's molecular predictions into per-building-block contributions, enabling near-exhaustive ranking of a 10-billion-compound synthesis library on a single GPU.","lead":"This paper presents a machine-learning search method that can rank more than 10 billion possible drug molecules in under a minute by treating each molecule's score as the sum of scores of its building blocks. A generalist might care because it attacks a bottleneck in virtual drug screening: current methods sample under 1 percent of huge chemical libraries, potentially missing the best compounds.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The additive factorization in Eq. 11 is the load-bearing assumption: the surrogate embedding is summed over R-groups with no cross-term, so top-k sets under the factorized objective can diverge from true docking top-k; only aggregate R2 is reported, not recall vs reconstruction error.","rationale":"The reader's weakest assumption identifies Eq. 11 as the critical point, and I agree. The central claim is not just that APEX is fast, but that its exhaustive but factorized search retrieves meaningful top-k sets. That claim depends on the factorized objective (16)/(17) being a faithful ordering of the true objective for the top-k region. The paper's Figure 7 is aggregate R2, which is necessary but not sufficient; top-k retrieval is a tail-sensitive operation. The absence of recall conditioned on reconstruction error is therefore the most direct gap. I also note the same concern is compounded by the risk of nested train/eval splits, since the surrogate is trained on the 1M CSL while recall is reported on the 12M CSL, which may contain many of the same products; this is a separate validation issue but the concrete test above includes a disjointness check. The runtime claims in Table 1 are credible and not in question. The verdict should remain conditional: the method is plausible, code and data are promised, and the concern can be settled by the stratified recall analysis.","tokens_in":10248,"tokens_out":9400,"duration_ms":86872,"concrete_test":"On the 12M benchmark, compute per-molecule L2 reconstruction error ||gθ(x) − \\hat g_λ(x)||2 and the factorized score residual for the target docking scores. Bin the 12M molecules by reconstruction-error quantile; within each bin, compute Recall-j-at-k for j=100 and k=100,000 against the ground-truth docking top-j. Also verify that the 1M training CSL is disjoint from the evaluation set (canonical SMILES intersection) and, if not, repeat the analysis on the held-out subset. If recall is high and flat across error quantiles, the additive factorization is adequate; if it drops sharply in the high-error bins, the aggregate R2 in Figure 7 masks a top-k failure mode.","verdict_should_be":"UNCHANGED","load_bearing_attack":"APEX solves (17) exactly, but the objective in (17) is obtained from the factorized surrogate \\hat g_λ(χ)=Σ_{(r,s)∈χ} u_{r,s} (Eq. 11), substituting for the actual embedding gθ(x). The surrogate gθ was trained without any additive-structure constraint, and docking scores depend on global molecular interactions (e.g., cross-R-group contacts), so the factorized predictions can reorder molecules relative to the surrogate's predictions even when the surrogate itself is accurate. The only support for this factorization is Figure 7, which reports aggregate R2 on a random sample of the 12M CSL. Aggregate R2 does not control top-k retrieval: a model can fit most molecules well while systematically mis-ranking the small upper-tail subpopulation that defines the top-k set. The paper reports no conditioning of Recall-j-at-k on per-molecule reconstruction error, no stratification by number of R-groups (2 vs 3 components), and no analysis of where the retrieved top-j sit in the reconstruction-error distribution. Without such evidence, the claim that the 30-second 10B search returns compounds close to the true top-k rests on an unvalidated additivity assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces APEX, a protocol for exhaustive virtual screening of combinatorial synthesis libraries (CSLs). APEX first trains a neural surrogate g_θ on a labeled subset of molecules to predict docking scores and physicochemical properties. It then trains a 'factorizer' that reconstructs the surrogate's d-dimensional embedding as a sum of per-R-group associative contributions u_{r,s}, so that any product's predicted score is a sum of cached synthon terms plus a bias. This factorization allows all products in the library to be evaluated on a GPU in a streaming fashion, and the paper claims exact retrieval of the (surrogate-defined) top-k set. The authors construct a 10B-compound CSL by BRICS fragmentation of ZINC22 molecules, fully enumerate a 12M-compound subset with AutoDock Vina docking scores on five targets and RDKit properties, and evaluate APEX's recall of ground-truth top-j sets on the 12M library. They also compare against Thompson sampling and report GPU runtimes below 30 seconds for top-1M retrieval on the 10B library under drug-likeness constraints.","tokens_in":10481,"tokens_out":7315,"duration_ms":72089,"significance":"If the methodological concerns are resolved, APEX would be a significant contribution: it replaces per-molecule expensive scoring with a cached additive factorization, making exhaustive ranking of ultra-large CSLs practical, and it ships an open benchmark, code, and trained model weights. The central idea of factorizing surrogate embeddings to enable exhaustive top-k search is attractive and the engineering (GPU top-k with chain-of-batches) is a real strength. However, the two load-bearing assumptions—that the additive factorization preserves ranking in the upper tail, and that the empirical evaluation is not contaminated by training/evaluation overlap—are not sufficiently established in the current manuscript.","major_comments":[{"comment":"The training and evaluation sets appear to be nested, which threatens the validity of the recall numbers in Figure 3. Section 2.1 says the authors generated a 1M-product library and a 12M-product library by 'uniformly downsampling each reaction'. If the 1M library is a subset of the 12M library, then exact molecules and their docking labels used to train the surrogate also appear in the evaluation set, and the factorizer is trained on their embeddings during the 12M factorizer training. This would allow memorization to inflate recall. The sentence 'to address data leakage concerns' does not address this overlap because a random downsample of a larger library typically contains many of the same products. The authors should either prove that the 1M and 12M sets are disjoint, construct a held-out disjoint subset (e.g., by holding out whole reactions or by constraining synthon combinations), or rerun the evaluation on a set guaranteed to contain no training molecules.","section":"Section 2.1 and Section 4.1"},{"comment":"The additive factorization in Eq. (11) is the sole mechanism that makes exhaustive search possible, but the paper does not establish that it preserves the ranking of the upper tail that defines top-k sets. Figure 7 reports aggregate R² of the factorized predictions on a random sample of the 12M library; aggregate R² is not sensitive to systematic misranking of a small subpopulation, and the top-k compounds are exactly such a subpopulation. The direct recall evaluation in Figure 3 is the right kind of evidence, but it is confounded by the possible train/test overlap in the previous comment. To support the 10B claims, the authors should report recall-j-at-k conditioned on per-molecule reconstruction error, stratify by the number of R-groups (two vs three components), and show where the retrieved top-j compounds sit in the reconstruction-error distribution relative to the full library. Without these diagnostics, one cannot tell whether the 30-second 10B search returns compounds that are close to the true top-k or merely products that happen to be well-approximated by the additive model.","section":"Section 3.2, Eq. (11); Figure 7"},{"comment":"The claimed robustness of the APEX factorization to 'errors in variables' is not supported. The argument appears to be that because the surrogate is trained with additive noise on embeddings (Eq. (4)), later replacing g_θ(x) with the factorized reconstruction ĝ_λ(χ) is a benign perturbation. But the factorizer is trained to minimize MSE (Eq. (12)), not to produce errors that match the isotropic noise p(ε); the reconstruction errors are structured, depend on the molecular graph, and can be systematic in the region of low docking scores that is most important for top-k retrieval. The paper should either provide a formal condition under which the substitution error behaves like the training-time noise, or soften the claim and instead state that this is an empirical approximation validated by the retrieval experiments.","section":"Section 3.3, Eqs. (4) and (13)"}],"minor_comments":[{"comment":"The main recall plot does not specify the values of j used, the number of repeated trials, or error bars; please include these details so the reader can assess the variability of the reported recall rates.","section":"Section 4.1 and Figure 3"},{"comment":"Please state explicitly whether the 1M and 12M libraries are generated as nested subsets of the same reaction grid, and if so, describe how the train/evaluation split avoids exact compound overlap.","section":"Section 2.1"},{"comment":"The embedding noise distribution p(ε) is not specified; please state the distribution and its variance, since the robustness claim in Section 3.3 depends on its properties.","section":"Section 3.1, Eq. (4)"},{"comment":"The table reports averages over five targets; please report per-target runtimes or standard deviations, and clarify whether the reported GPU time includes the final SMILES/DataFrame construction step.","section":"Section 4.3, Table 1"},{"comment":"The caption says 'docking scores (in bold)' but does not enumerate the 28 endpoints; please list the endpoints or provide a table so the reader can distinguish docking-score R² from physicochemical-property R².","section":"Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claims rest on the 12M evaluation, and the potential nesting of the 1M training set inside the 12M evaluation set is a serious concern that should be clarified before publication. The additive-factorization diagnostics (recall vs reconstruction error, per-component stratification) are also necessary to substantiate the 10B-library claims. The paper is otherwise well structured and the engineering contribution is credible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"APEX is worth reading. The paper's actual contribution is a practical protocol: a surrogate that factorizes molecular embeddings into per-R-group associative contributions, allowing exhaustive top-k retrieval over multibillion-compound libraries in seconds on a single GPU. The open 12M-compound benchmark with ground-truth docking scores across five targets is a real asset for the community. The runtime numbers are concrete and the implementation details (AIR top-k, chain-of-batches) are plausible.\n\nThe biggest weakness is the evaluation split. The 1M training set is a uniform downsample of the same reactions that make up the 12M evaluation library, so exact training molecules appear in the evaluation set. The authors say they addressed leakage, but a uniform downsample is not a disjoint split. If the surrogate has any memory of high-scoring training examples, recall-on-12M is inflated. This is fixable: hold out whole reactions or synthons, or at least report results on a truly disjoint split.\n\nThe second concern is the additivity assumption in Eq. 11. The factorizer is trained directly in that form, so the issue isn't an architectural mismatch; it's whether a sum of per-R-group vectors can reconstruct the surrogate's embedding well enough in the upper tail that defines top-k sets. The paper only reports aggregate R2 (Figure 7). That's not enough. A model can have high R2 overall and still systematically misorder the small subpopulation that matters. I'd want to see recall-j-at-k conditioned on reconstruction error, stratified by two- vs three-component reactions, and perhaps a comparison between the factorized top-k and the surrogate's own top-k on a subset where the surrogate can be run exhaustively.\n\nMinor: Figure 4 plots the 10B library's APEX-predicted scores against the 12M background; it's not a direct docking comparison, and it should be labeled as such. Also, error bars are missing from the recall and runtime tables.\n\nThe core idea holds up as a plausible engineering contribution, not just a trick. The citation to Pedawi et al. 2022 is appropriate — the factorization hierarchy is theirs — and the new parts (constraint-aware exhaustive GPU search, the benchmark) are clearly new.\n\nThis paper deserves a serious referee, but the current evidence doesn't yet support the headline 'exact retrieval of approximate top-k' without a disjoint split and tail-focused analysis. I'd send it out and ask for those experiments.","headline":"APEX is a credible method with a genuinely useful open benchmark, but its headline retrieval claims hinge on a nested train/eval split and an additivity assumption whose tail behavior is never directly tested.","tokens_in":11044,"tokens_out":3043,"would_cite":true,"duration_ms":26884,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a factorized surrogate embedding lets a single GPU retrieve the approximate top million compounds of a 10-billion-compound synthesis library in under 30 seconds.","keywords":["virtual screening","combinatorial synthesis libraries","top-k retrieval","neural network surrogate","factorized embeddings","docking scores","GPU acceleration","drug discovery"],"falsifier":"Measure, on the 12M enumerated library, the residual between the true docking score and the APEX additive score for each molecule. If, for molecules sharing one R-group, this residual is strongly predicted by the choice of the other R-groups, the additive model is missing cross-R-group interactions; a calculation showing that top-k recall at any k is not monotone in reconstruction error would likewise indicate the approximate objective selects for reconstruction accuracy rather than true docking quality.","tokens_in":10052,"feed_emoji":"🧪","tokens_out":9147,"duration_ms":78855,"temperature":0.7,"pith_summary":"The paper tries to establish that exhaustive virtual screening of ultra-large combinatorial synthesis libraries is possible despite their size. Its method, APEX, trains a lightweight neural surrogate for a scoring function, then factorizes the surrogate's embedding into per-building-block contributions so that every product in the library can be evaluated by summation rather than by a full network pass. The central claim is that this yields the exact top-k set of the factorized (hence approximate) objective, with retrieval of ground-truth top compounds far above random and competitive with Thompson sampling at matched budgets. If the claim holds, screening 10 billion compounds for the best million can run on a single consumer GPU in under thirty seconds under standard drug-likeness constraints.","feed_headline":"Sums of fragment scores rank 10B compounds in 30 seconds","feed_subtitle":"APEX turns docking scores into per-fragment contributions, so exhaustive top-k search becomes a GPU routine.","key_machinery":"The reaction factorizer: a stack of encoders (SynthonEncoder, RgroupEncoder, ReactionEncoder) paired with a SynthonValueEncoder and RgroupKeyEncoder that yield an associative contribution u_{r,s} = K_r v_s for assigning synthon s to R-group r. The factorization replaces the surrogate embedding g_θ(x) with the sum of u_{r,s} over the molecule's R-group assignments, and feeding that into the surrogate's linear head produces the APEX score as a sum of per-synthon contributions plus a bias. This makes scoring additive across building blocks, enabling all synthon contributions to be precomputed and cached as a |S|×d matrix; scoring a molecule then costs a few additions, and a streamed GPU top-k pass over every product in the library finishes in seconds.","core_discovery":"APEX's central claim is that a surrogate model whose tasks are linear heads on a shared molecular embedding can be factorized across the library's R-group structure. A pair of learned maps produces, for each R-group r and synthon s, an associative contribution u_{r,s} = K_r v_s, and the factorized embedding is the sum of these contributions over the molecule's R-group assignments. Because the objective and constraints are linear functions of the embedding, their factorized versions become sums of per-synthon terms v_{i,r,s}, so the score of each compound is computed with a few additions after the contributions are cached. The paper reports that, on a benchmark CSL of 12 million fully docked compounds, APEX's top-100,000 set recovers a substantially larger fraction of the ground-truth top-j compounds than random selection, matches or beats Thompson sampling with equal evaluation budgets, and that on a 10-billion-compound version of the library a top-1,000,000 retrieval completes in 10.9–24.3 seconds on a single Tesla T4 GPU. The reported scaling is made possible by replacing per-molecule neural evaluations with a streamed summation over cached contributions, using a GPU chain-of-batches top-k implementation.","pith_inferences":["If the factorization is as accurate as the reported aggregate R² suggests, the same cached contributions could serve any linear surrogate head, so the 30-second figure is not specific to docking; co-folding or ML affinity predictors with learnable embeddings would inherit the speedup.","The benchmark library was built by fragmenting ZINC22 lead-like compounds with BRICS, which may not reproduce the synthon-reuse and reaction-size distributions of commercial make-on-demand libraries; the factorization error, and hence the retrieval guarantee, is library-dependent until shown otherwise.","A natural diagnostic the paper does not report is recall as a function of per-molecule reconstruction error: if users could predict when the additive score is trustworthy for a given molecule, they could decide which retrieved compounds deserve re-docking without re-scoring the whole library.","Because the factorizer is trained with no labels on the target library, nothing forces it to preserve the docking-relevant geometry of the embedding; an explicit stress test with fragments whose activity depends on their partner context would map where the approximation breaks."],"forward_implications":["If the central claim holds, a screening query becomes declarative: specifying an objective and constraints returns the factorized-optimal top-k in seconds, and changing constraints does not require re-running docking or the surrogate.","Because every product in the library is scored rather than sampled, high-scoring compounds that stochastic search would miss are in principle retrievable, subject to the fidelity of the factorization.","At matched evaluation budgets APEX matches or beats Thompson sampling on top-j recall, and it does so with a runtime that does not grow with the number of compounds scored per budget.","The same cached synthon contributions serve any linear surrogate head, so the runtime cost of adding a new objective or constraint is negligible after the first factorized pass."],"supporting_citations":[{"why":"Provides the hierarchical CSL representation and the factorizer architecture APEX adapts.","marker":"Pedawi et al. (2022)"},{"why":"Gives the errors-in-variables rationale for using noise-robust embeddings in the factorized prediction.","marker":"Griliches (1974)"},{"why":"Supplies the GPU AIR top-k algorithm used in the chain-of-batches CUDA implementation.","marker":"Zhang et al. (2023)"},{"why":"Defines the Thompson sampling baseline and its warm-up schedule used in the comparison.","marker":"Klarich et al. (2024)"},{"why":"Defines the AutoDock Vina scoring function whose scores are the docking objective.","marker":"Trott & Olson (2010)"},{"why":"Provides the BRICS fragmentation rules used to construct the benchmark CSL from ZINC compounds.","marker":"Degen et al. (2008)"},{"why":"Supplies the ZINC22 molecule set from which the CSL building blocks are sampled.","marker":"Tingle et al. (2023)"},{"why":"Provides the five receptor structures and binding sites used for the docking benchmark.","marker":"García-Ortegón et al. (2022)"}],"fun_headline_variants":["APEX factorizes fragment scores to exhaustively rank 10B compounds","Fragment-sum surrogates let APEX enumerate 10B compounds in seconds","Linear fragment scores exhaustively rank 10B-molecule library in under a minute","Associative fragment sums cram exhaustive top-k into GPU seconds","APEX: per-synthon sums exhaustively screen 10-billion libraries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a molecule's docking-relevant embedding can be recovered as a sum of independent per-building-block contributions; if interactions between R-groups matter for docking scores, the additive reconstruction misorders molecules and the approximate top-k degrades.","fun_headline_variants_meta":{"raw":{"variants":["APEX factorizes fragment scores to exhaustively rank 10B compounds","Fragment-sum surrogates let APEX enumerate 10B compounds in seconds","Linear fragment scores exhaustively rank 10B-molecule library in under a minute","Associative fragment sums cram exhaustive top-k into GPU seconds","APEX: per-synthon sums exhaustively screen 10-billion libraries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0009,"raw_usage":{"total_tokens":3946,"prompt_tokens":1088,"completion_tokens":2858,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":704,"completion_tokens_details":{"reasoning_tokens":2760}},"tokens_in":704,"tokens_out":2858,"duration_ms":19923,"temperature":1.0,"reasoning_tokens":2760,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:41:34.247990+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, on the 12M enumerated library, the residual between the true docking score and the APEX additive score for each molecule. If, for molecules sharing one R-group, this residual is strongly predicted by the choice of the other R-groups, the additive model is missing cross-R-group interactions; a calculation showing that top-k recall at any k is not monotone in reconstruction error would likewise indicate the approximate objective selects for reconstruction accuracy rather than true docking quality.","supporting_citations":[{"cited_title":"ZINC -22: A free multi-billion-scale database of tangible compounds for ligand discovery","cited_arxiv_id":null,"evidence_quote":"Supplies the ZINC22 molecule set from which the CSL building blocks are sampled."},{"cited_title":"An efficient graph generative model for navigating ultra-large combinatorial synthesis libraries","cited_arxiv_id":null,"evidence_quote":"Provides the hierarchical CSL representation and the factorizer architecture APEX adapts."},{"cited_title":"Errors in variables and other unobservables","cited_arxiv_id":null,"evidence_quote":"Gives the errors-in-variables rationale for using noise-robust embeddings in the factorized prediction."},{"cited_title":"Thompson sampling: An efficient method for searching ultra-large synthesis on demand databases","cited_arxiv_id":null,"evidence_quote":"Defines the Thompson sampling baseline and its warm-up schedule used in the comparison."},{"cited_title":"A uto D ock V ina: Improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading","cited_arxiv_id":null,"evidence_quote":"Defines the AutoDock Vina scoring function whose scores are the docking objective."},{"cited_title":"On the art of compiling and using ``drug-like'' chemical fragment spaces","cited_arxiv_id":null,"evidence_quote":"Provides the BRICS fragmentation rules used to construct the benchmark CSL from ZINC compounds."}],"review_version":2}