{"id":"50c316b8-bab7-47f5-8198-dfc7c9e1ad82","arxiv_id":"1908.07209","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DeepScaffold generates valid, drug-like molecules that retain a given scaffold, and it extends scaffold-based generation to cyclic skeletons and side-chain property queries.","lead":"DeepScaffold is a deep-learning tool that grows new drug-like molecules around a molecular scaffold supplied by the chemist. It aims to help drug discovery by letting researchers design compounds that keep a desired core structure.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed guarantee that outputs always match the input scaffold is not enforced by the described action set: the 'connect two existing nodes' action in §2.4 could join two scaffold atoms, and no masking or post-hoc filter is described.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the scaffold-matching guarantee is asserted without an explicit constraint preventing the connect action from altering the scaffold. This concern is central because the paper's differentiator over earlier conditional generators is exactly the guarantee of scaffold containment. The existing validity metric does not test this property, and the absence of released code means the presence of a masking rule cannot be checked externally. The concern does not force rejection, because the guarantee could plausibly be enforced by an undocumented mask, but it makes the paper's central claim unverifiable as written. The verdict should remain CONDITIONAL, with the added condition that the authors demonstrate scaffold containment explicitly and clarify the action-masking logic. The reader's assessment already captures the core issue, so agreement is complete.","tokens_in":15768,"tokens_out":3540,"duration_ms":40179,"concrete_test":"Re-run or reimplement the §2.4 generator and apply RDKit's HasSubstructMatch(scaffold, molecule) to all generated samples (e.g., the 10,000 per test scaffold in §3.3.1 and all leave-one-out samples in Table 2); report the percentage of valid outputs that do not contain the input scaffold. Separately inspect the action-masking logic: enumerate calls of action 2 and verify that no bond can be placed between two atoms both present in the initial scaffold graph. If any substructure failure appears, the guarantee is disproven; if the mask is absent, the guarantee is unsupported by construction.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central assertion is that DeepScaffold 'guarantees' every generated molecule matches the scaffold query, because generation starts from the scaffold graph and only appends atoms, connects existing atoms, or terminates (§2.4). The weak point is the second action. The paper never states that the connect action is masked so that at least one endpoint is a newly added atom, nor that bonds between two original scaffold atoms are prohibited. If such a bond is sampled, the output no longer contains the input scaffold as a substructure; the scaffold has been topologically altered. The reported validity metric (§3.3.1) only checks RDKit sanitization, not substructure containment, so the 98.9% validity and the leave-one-out docking numbers do not bear on the guarantee. A learned policy might rarely or never choose the offending action, but 'guarantee' requires an architectural constraint, not empirical tendency. Since no code or data is released, the reader cannot verify whether such a mask exists. This is the most load-bearing concern because it targets the exact property that differentiates DeepScaffold from conditional SMILES generators; if the connect action is unmasked, the headline claim is false rather than merely under-evaluated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents DeepScaffold, a graph-based generative model for scaffold-directed de novo drug design. The system combines three components: a VAE-based model that assigns atom and bond types to cyclic skeletons (producing classical scaffolds), a scaffold-based molecule generator that grows side chains from a given scaffold graph, and a side-chain property filter based on pharmacophore-like queries. The authors evaluate the scaffold-diversification model on holdout scaffolds from ChEMBL and on three privileged GPCR scaffolds in a leave-one-out setting, reporting validity, uniqueness, property distributions, internal diversity, MMD, active-molecule reproduction rates, and DRD2 docking scores. The central claim is that the method guarantees that generated molecules match the input scaffold query and generalizes to new scaffolds.","tokens_in":15900,"tokens_out":3400,"duration_ms":34309,"significance":"If the scaffold-preservation guarantee were established, DeepScaffold would be a practically valuable contribution, extending earlier scaffold-based graph generative work by supporting multiple scaffold definitions, including cyclic skeletons and side-chain property constraints. The paper also contains a thoughtful treatment of evaluation methodology, including the use of a U-statistic for internal diversity and a bad-case analysis attributing high MMD scores to publication bias in ChEMBL. These are genuine strengths. However, the headline guarantee is not supported by the described architecture or by the reported metrics, as detailed in the major comments. The manuscript is therefore of interest to the molecular generative-modeling community but requires substantial revision before the central claim can be accepted.","major_comments":[{"comment":"The central guarantee that the output molecule always contains the input scaffold is not enforced by the described action set. The three allowed actions are 'append a new atom', 'connecting two existing nodes', and 'terminate'. The paper does not state that the connect action is masked so that at least one endpoint is a newly added atom, nor that bonds between two original scaffold atoms are prohibited. If such a bond is sampled, the scaffold is topologically altered and the output no longer contains the query as a substructure. Since the guarantee is load-bearing for the paper's main claim, please either describe the explicit masking/constraint mechanism or, if none exists, add a post-hoc substructure check and weaken the claim accordingly.","section":"§2.4, Figure 7b"},{"comment":"The reported average validity of 98.9% is computed using RDKit Chem.Sanitize, which checks chemical validity only, not whether the generated molecule contains the input scaffold as a substructure. Therefore the validity, uniqueness, and docking results do not provide evidence for the scaffold-match guarantee. A per-scaffold substructure-containment rate should be reported alongside validity to justify the 'guarantee' language.","section":"§3.3.1, Eq. (10)"},{"comment":"The cyclic-skeleton-to-scaffold component achieves only 82.5% validity and 61.0% reconstruction with the best β-VAE setting, yet no end-to-end evaluation of the full pipeline (cyclic skeleton → classical scaffold → final molecule) is presented. The abstract and introduction claim generation from cyclic skeletons as one of the model's capabilities, so the absence of an end-to-end metric leaves this aspect only partially validated. Please report the fraction of generated molecules that pass sanitization and contain the target skeleton.","section":"§2.3, Table 1"},{"comment":"The leave-one-out validation is performed on only three hand-picked privileged scaffolds, and no uncertainty estimates or statistical tests are reported. This is a very limited basis for the claim that the model generalizes to new scaffolds outside the training set. Please provide confidence intervals for the three cases or extend the leave-one-out evaluation to a larger random sample of scaffolds.","section":"§2.4.2, Table 2"}],"minor_comments":[{"comment":"The text states that the average validity is 98.9%, while the caption of Figure 9a reports 98.8%; please make these consistent.","section":"§3.3.1, Figure 9a"},{"comment":"The sentence 'tools for discovery of privileged scaffolds (such as CSE6 and5)' appears to have a malformed citation; please fix the reference formatting.","section":"§1, reference 5"},{"comment":"The pharmacophore-based filter is described only briefly; please specify how hydrogen-bond donor/acceptor counts are defined and how the side-chain size is measured, since these queries are part of the claimed functionality.","section":"§2.5"},{"comment":"The caption of Figure 13 lists active molecule reproduction rates that differ in precision from Table 2 (e.g., '5.17%' vs '0.0517'); please ensure consistent formatting between figure captions and tables.","section":"§3.3.6, Figure 13"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the scaffold-preservation guarantee, which is not established by the described action set or the reported metrics. This is fixable by adding a masking rule, reporting substructure-containment rates, or revising the claim. The paper does not release code or data, only a contact for licensing, which limits reproducibility for a claimed 'tool'. The relationship to Lim et al. [26] is acknowledged, but the novelty rests substantially on the scaffold-preservation property, so the guarantee issue should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: DeepScaffold genuinely extends scaffold-based generation beyond Lim et al., but the headline guarantee is undercut by a missing constraint. The scaffold-growing model itself looks sound; the skeleton-completion VAE and the pharmacophore filter are evaluated too thinly to carry the 'comprehensive tool' claim.\n\nWhat's new and good: the VAE that turns cyclic skeletons into classical scaffolds, the inclusion of HierS sub-scaffolds instead of only BM-scaffolds, and the option to query side-chain properties. The evaluation is more thorough than most: per-scaffold validity and uniqueness, MMD, internal diversity with U-statistics, a leave-one-out case study on three GPCR scaffolds, docking against DRD2, and an honest discussion of non-random-sampling bias in ChEMBL test sets. The observation that average validity correlates with scaffold size is a useful warning for future benchmarks. The self-citation to their earlier graph generator is appropriate, and they are transparent about borrowing the architecture.\n\nNow the soft spots, in proportion. The stress-test concern lands. Section 2.4 lists three actions: append a new atom, connect two existing nodes, or terminate. If 'connect two existing nodes' can connect two scaffold atoms, the scaffold changes and the guarantee fails. The paper does not describe any masking that forbids that, nor a post-hoc substructure check. Validity is only RDKit sanitization, which does not test scaffold containment. Since no code or data is released, we cannot verify whether a mask exists. This is the load-bearing flaw: the guarantee is the property that differentiates DeepScaffold from conditional SMILES generation.\n\nThe other gaps are real but more addressable. The cyclic skeleton pathway stops at scaffold reconstruction (82.5% validity, 61% reconstruction for the best VAE) and never evaluates end-to-end generation from skeleton to full molecule. The pharmacophore filter is described but never tested. And there is no numerical comparison with Lim et al., even though that is the closest prior method. None of these are fatal by themselves; together they mean the paper supports the scaffold-growing component well, but not the full 'comprehensive tool' claim.\n\nThe core scaffold-growing model is plausibly sound, the evaluation is thoughtful, and the paper is honest about its limitations. I would send it to peer review, but with a clear request: either add the masking constraint and demonstrate substructure containment on every output, or soften the guarantee to a claim about observed behavior. As written, the central promise is asserted, not proven.","headline":"A useful scaffold-growing generator with a credible evaluation story, but the advertised guarantee that outputs always contain the input scaffold is not actually enforced by the described action set.","tokens_in":16549,"tokens_out":2033,"would_cite":false,"duration_ms":20058,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A scaffold-first graph generator promises exact scaffold preservation in every output molecule.","keywords":["scaffold-based drug discovery","de novo molecule design","graph generative model","variational autoencoder","cyclic skeleton","classical scaffold","pharmacophore query","molecular docking"],"falsifier":"Instrument the decoder to log all actions during sampling: if a 'connect two existing atoms' action ever joins two atoms of the input scaffold, or any scaffold bond is altered, the exact-substructure guarantee fails. A more direct test is a subgraph isomorphism check between the queried scaffold and each generated molecule; any output that does not contain the scaffold as an exact substructure disproves the claim.","tokens_in":15467,"feed_emoji":"🧪","tokens_out":9018,"duration_ms":89478,"temperature":0.7,"pith_summary":"DeepScaffold tries to show that de novo molecule design can be anchored to a user-chosen scaffold instead of starting from an empty graph. The model accepts three kinds of queries: cyclic skeletons (ring-only graphs), classical BM-scaffolds with atom and bond types, and scaffolds with extra constraints on side-chain properties. Its central promise is that every generated molecule contains the queried scaffold as an exact substructure, and that the growth rules learned from training data transfer to scaffolds never seen during training. That matters because scaffold-based design is a standard medicinal-chemistry workflow, and earlier generative models either did not guarantee scaffold preservation or handled only one scaffold type.","feed_headline":"Molecule generator grows drugs from a user's scaffold","feed_subtitle":"Model accepts three scaffold types and docks generated molecules like known actives.","key_machinery":"The load-bearing mechanism is the sequential graph-expansion decoder. It starts from the molecular graph of the scaffold and at each step chooses one of three actions — append a new atom, connect two existing atoms, or stop — and no action in the described repertoire deletes or alters scaffold bonds, so the scaffold is meant to be an exact substructure of every completed molecule. Two auxiliary components extend the range of queries: a graph-convolutional variational autoencoder that maps cyclic skeletons to complete classical scaffolds by sampling atom and bond types, and a rule-based pharmacophore filter that keeps only generated molecules whose side chains satisfy user-specified size, hydrogen-bond-donor, and hydrogen-bond-acceptor constraints. The growth model itself is a 20-layer dense graph convolutional network with virtual bonds and virtual ring/linker nodes, trained by likelihood maximization with importance sampling.","core_discovery":"On its own terms, DeepScaffold's contribution is a generative architecture in which molecule construction is a graph-growing process seeded by the scaffold itself. A graph neural network is trained to extend the scaffold by appending atoms, connecting two existing atoms, or terminating, and the full molecule is built by repeated application of those actions. Because the design starts from the scaffold and never removes or changes scaffold atoms and bonds, the scaffold remains an exact substructure of every output molecule. For cyclic skeletons, a separate variational autoencoder first decorates the ring-only graph with atom and bond types to produce a classical scaffold, which then feeds the same growth process; side-chain queries on size, hydrogen-bond donors, and acceptors are applied as a post-filter. Across the test scaffolds the paper reports 98.9% average validity, outputs that are often more structurally diverse than the corresponding test molecules, and leave-one-out reproduction of known GPCR actives and drugs from privileged scaffolds.","pith_inferences":["If the missing connect-action constraint were made explicit and proven, the exact-substructure guarantee would become a formal invariant; until then, a skeptical reader should treat it as a design intent rather than a demonstrated theorem.","The same scaffold-seeded growth procedure could be applied to other graph-generation tasks that require a fixed subgraph to survive, such as constructing molecules around a covalent warhead or a conserved pharmacophore.","The paper's own bad-case analysis implies that literature-derived test sets are biased toward single-assay chemistry; constructing a deliberately unbiased scaffold benchmark would more directly test generalization than MMD against those sets.","The larger scatter in property distributions for generated molecules, read together with the bias analysis, suggests the model may be exploring legitimate chemical space rather than overfitting the side-chain preferences of a few publications—an interpretation that could be tested by prospective synthesis or by scoring on a diverse external library."],"forward_implications":["A chemist can hand the model a ring-only cyclic skeleton and receive a complete scaffold with atom and bond types filled in, then have full molecules grown from it.","Because generation starts from the scaffold graph itself, the model can in principle accept scaffolds that never appeared in training, not just memorized ones.","Ten thousand samples per privileged GPCR scaffold reproduced 5–16% of known actives and 13–25% of known drugs in leave-one-out tests, suggesting practical scaffold-based hit discovery.","Docking scores of generated molecules against DRD2 matched the test-set distribution and, for two of the three case-study scaffolds, beat random database molecules.","The observed positive correlation between scaffold size and validity and negative correlation with uniqueness means benchmarks for such models should compare per-scaffold metrics on a common scaffold set, not aggregate averages."],"supporting_citations":[{"why":"Provides the underlying conditional graph-generative architecture, the action set, and the importance-sampling training used by the scaffold-growing model.","marker":"[18]"},{"why":"The prior scaffold-growing model that DeepScaffold extends; supplies the BM-scaffold-only baseline and the direct-growth idea.","marker":"[26]"},{"why":"Supplies the 914,464-molecule drug-like dataset from which scaffolds and scaffold-molecule pairs are extracted.","marker":"[27]"},{"why":"Defines the classical scaffold (side chains stripped) that is one of the query types the model accepts.","marker":"[2]"},{"why":"Defines the hierarchical scaffold extraction that yields sub-scaffolds beyond the classical definition, expanding the training scaffold set.","marker":"[7]"},{"why":"Provides the variational autoencoder formalism used for the cyclic-skeleton-to-classical-scaffold transformer.","marker":"[30]"},{"why":"The beta-VAE variant used to counteract posterior collapse in the skeleton-to-scaffold model.","marker":"[32]"},{"why":"Supplies the internal-diversity definition whose U-statistic estimator the paper adopts.","marker":"[33]"},{"why":"Establishes the similarity coefficient used as a valid kernel, justifying the MMD comparison.","marker":"[38]"}],"fun_headline_variants":["Deep learning grows drugs from any scaffold","Scaffold-based AI designs novel drug molecules","AI tool builds drugs on user-defined scaffolds","Drug generator expands scaffolds into candidates","Grow drug-like molecules from core scaffolds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The exact-substructure guarantee rests on the assumption that the 'connect two existing atoms' action will never join two scaffold atoms or change a scaffold bond, yet no masking or constraint enforcing that is described.","fun_headline_variants_meta":{"raw":{"variants":["Deep learning grows drugs from any scaffold","Scaffold-based AI designs novel drug molecules","AI tool builds drugs on user-defined scaffolds","Drug generator expands scaffolds into candidates","Grow drug-like molecules from core scaffolds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1310,"prompt_tokens":874,"completion_tokens":436,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":372}},"tokens_in":490,"tokens_out":436,"duration_ms":5424,"temperature":1.0,"reasoning_tokens":372,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:22:51.888529+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Instrument the decoder to log all actions during sampling: if a 'connect two existing atoms' action ever joins two atoms of the input scaffold, or any scaffold bond is altered, the exact-substructure guarantee fails. A more direct test is a subgraph isomorphism check between the queried scaffold and each generated molecule; any output that does not contain the scaffold as an exact substructure disproves the claim.","supporting_citations":[{"cited_title":"Multi-objective de novo drug design with conditional graph generative model","cited_arxiv_id":null,"evidence_quote":"Provides the underlying conditional graph-generative architecture, the action set, and the importance-sampling training used by the scaffold-growing model."},{"cited_title":"Scaffold-based molecular design using graph generative model","cited_arxiv_id":"1905.13639","evidence_quote":"The prior scaffold-growing model that DeepScaffold extends; supplies the BM-scaffold-only baseline and the direct-growth idea."},{"cited_title":"J.; Bento, A","cited_arxiv_id":null,"evidence_quote":"Supplies the 914,464-molecule drug-like dataset from which scaffolds and scaffold-molecule pairs are extracted."},{"cited_title":"W.; Murcko, M","cited_arxiv_id":null,"evidence_quote":"Defines the classical scaffold (side chains stripped) that is one of the query types the model accepts."},{"cited_title":"J.; Janes, J.; Su, A","cited_arxiv_id":null,"evidence_quote":"Defines the hierarchical scaffold extraction that yields sub-scaffolds beyond the classical definition, expanding the training scaffold set."},{"cited_title":"P.; Welling, M","cited_arxiv_id":null,"evidence_quote":"Provides the variational autoencoder formalism used for the cyclic-skeleton-to-classical-scaffold transformer."},{"cited_title":"beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework","cited_arxiv_id":null,"evidence_quote":"The beta-VAE variant used to counteract posterior collapse in the skeleton-to-scaffold model."},{"cited_title":"ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity?","cited_arxiv_id":"1708.08227","evidence_quote":"Supplies the internal-diversity definition whose U-statistic estimator the paper adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the similarity coefficient used as a valid kernel, justifying the MMD comparison."}],"review_version":1}