REVIEW 4 major objections 4 minor 41 references
DeepScaffold: a comprehensive tool for scaffold-based de novo drug discovery using deep learning
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A scaffold-first graph generator promises exact scaffold preservation in every output molecule.
desk verdict A useful scaffold-growing generator with a credible evaluation story, but the advertised guarantee that outputs always contain the input scaffold is not actually enforced by the described action set. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the sequential graph-expansion decoder. It starts from the molecular graph of the scaffold and at each step chooses one of three actions — append a new atom, connect two existing atoms, or stop — and no action in the described repertoire deletes or alters scaffold bonds, so the scaffold is meant to be an exact substructure of every completed molecule. Two auxiliary components extend the range of queries: a graph-convolutional variational autoencoder that maps cyclic skeletons to complete classical scaffolds by sampling atom and bond types, and a rule-based pharmacophore filter that keeps only generated molecules whose side chains satisfy user-specified size, hydrogen-bond-donor, and hydrogen-bond-acceptor constraints. The growth model itself is a 20-layer dense graph convolutional network with virtual bonds and virtual ring/linker nodes, trained by likelihood maximization with importance sampling.
What would settle it
Instrument the decoder to log all actions during sampling: if a 'connect two existing atoms' action ever joins two atoms of the input scaffold, or any scaffold bond is altered, the exact-substructure guarantee fails. A more direct test is a subgraph isomorphism check between the queried scaffold and each generated molecule; any output that does not contain the scaffold as an exact substructure disproves the claim.
Extended reading notes
Core claim
On its own terms, DeepScaffold's contribution is a generative architecture in which molecule construction is a graph-growing process seeded by the scaffold itself. A graph neural network is trained to extend the scaffold by appending atoms, connecting two existing atoms, or terminating, and the full molecule is built by repeated application of those actions. Because the design starts from the scaffold and never removes or changes scaffold atoms and bonds, the scaffold remains an exact substructure of every output molecule. For cyclic skeletons, a separate variational autoencoder first decorates the ring-only graph with atom and bond types to produce a classical scaffold, which then feeds the same growth process; side-chain queries on size, hydrogen-bond donors, and acceptors are applied as a post-filter. Across the test scaffolds the paper reports 98.9% average validity, outputs that are often more structurally diverse than the corresponding test molecules, and leave-one-out reproduction of known GPCR actives and drugs from privileged scaffolds.
Load-bearing premise
The exact-substructure guarantee rests on the assumption that the 'connect two existing atoms' action will never join two scaffold atoms or change a scaffold bond, yet no masking or constraint enforcing that is described.
Editorial extensions
If this is right
- A chemist can hand the model a ring-only cyclic skeleton and receive a complete scaffold with atom and bond types filled in, then have full molecules grown from it.
- Because generation starts from the scaffold graph itself, the model can in principle accept scaffolds that never appeared in training, not just memorized ones.
- Ten thousand samples per privileged GPCR scaffold reproduced 5–16% of known actives and 13–25% of known drugs in leave-one-out tests, suggesting practical scaffold-based hit discovery.
- Docking scores of generated molecules against DRD2 matched the test-set distribution and, for two of the three case-study scaffolds, beat random database molecules.
- The observed positive correlation between scaffold size and validity and negative correlation with uniqueness means benchmarks for such models should compare per-scaffold metrics on a common scaffold set, not aggregate averages.
Reading between the lines
- If the missing connect-action constraint were made explicit and proven, the exact-substructure guarantee would become a formal invariant; until then, a skeptical reader should treat it as a design intent rather than a demonstrated theorem.
- The same scaffold-seeded growth procedure could be applied to other graph-generation tasks that require a fixed subgraph to survive, such as constructing molecules around a covalent warhead or a conserved pharmacophore.
- The paper's own bad-case analysis implies that literature-derived test sets are biased toward single-assay chemistry; constructing a deliberately unbiased scaffold benchmark would more directly test generalization than MMD against those sets.
- The larger scatter in property distributions for generated molecules, read together with the bias analysis, suggests the model may be exploring legitimate chemical space rather than overfitting the side-chain preferences of a few publications—an interpretation that could be tested by prospective synthesis or by scoring on a diverse external library.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents DeepScaffold, a graph-based generative model for scaffold-directed de novo drug design. The system combines three components: a VAE-based model that assigns atom and bond types to cyclic skeletons (producing classical scaffolds), a scaffold-based molecule generator that grows side chains from a given scaffold graph, and a side-chain property filter based on pharmacophore-like queries. The authors evaluate the scaffold-diversification model on holdout scaffolds from ChEMBL and on three privileged GPCR scaffolds in a leave-one-out setting, reporting validity, uniqueness, property distributions, internal diversity, MMD, active-molecule reproduction rates, and DRD2 docking scores. The central claim is that the method guarantees that generated molecules match the input scaffold query and generalizes to new scaffolds.
Significance. If the scaffold-preservation guarantee were established, DeepScaffold would be a practically valuable contribution, extending earlier scaffold-based graph generative work by supporting multiple scaffold definitions, including cyclic skeletons and side-chain property constraints. The paper also contains a thoughtful treatment of evaluation methodology, including the use of a U-statistic for internal diversity and a bad-case analysis attributing high MMD scores to publication bias in ChEMBL. These are genuine strengths. However, the headline guarantee is not supported by the described architecture or by the reported metrics, as detailed in the major comments. The manuscript is therefore of interest to the molecular generative-modeling community but requires substantial revision before the central claim can be accepted.
major comments (4)
- [§2.4, Figure 7b] The central guarantee that the output molecule always contains the input scaffold is not enforced by the described action set. The three allowed actions are 'append a new atom', 'connecting two existing nodes', and 'terminate'. The paper does not state that the connect action is masked so that at least one endpoint is a newly added atom, nor that bonds between two original scaffold atoms are prohibited. If such a bond is sampled, the scaffold is topologically altered and the output no longer contains the query as a substructure. Since the guarantee is load-bearing for the paper's main claim, please either describe the explicit masking/constraint mechanism or, if none exists, add a post-hoc substructure check and weaken the claim accordingly.
- [§3.3.1, Eq. (10)] The reported average validity of 98.9% is computed using RDKit Chem.Sanitize, which checks chemical validity only, not whether the generated molecule contains the input scaffold as a substructure. Therefore the validity, uniqueness, and docking results do not provide evidence for the scaffold-match guarantee. A per-scaffold substructure-containment rate should be reported alongside validity to justify the 'guarantee' language.
- [§2.3, Table 1] The cyclic-skeleton-to-scaffold component achieves only 82.5% validity and 61.0% reconstruction with the best β-VAE setting, yet no end-to-end evaluation of the full pipeline (cyclic skeleton → classical scaffold → final molecule) is presented. The abstract and introduction claim generation from cyclic skeletons as one of the model's capabilities, so the absence of an end-to-end metric leaves this aspect only partially validated. Please report the fraction of generated molecules that pass sanitization and contain the target skeleton.
- [§2.4.2, Table 2] The leave-one-out validation is performed on only three hand-picked privileged scaffolds, and no uncertainty estimates or statistical tests are reported. This is a very limited basis for the claim that the model generalizes to new scaffolds outside the training set. Please provide confidence intervals for the three cases or extend the leave-one-out evaluation to a larger random sample of scaffolds.
minor comments (4)
- [§3.3.1, Figure 9a] The text states that the average validity is 98.9%, while the caption of Figure 9a reports 98.8%; please make these consistent.
- [§1, reference 5] The sentence 'tools for discovery of privileged scaffolds (such as CSE6 and5)' appears to have a malformed citation; please fix the reference formatting.
- [§2.5] The pharmacophore-based filter is described only briefly; please specify how hydrogen-bond donor/acceptor counts are defined and how the side-chain size is measured, since these queries are part of the claimed functionality.
- [§3.3.6, Figure 13] The caption of Figure 13 lists active molecule reproduction rates that differ in precision from Table 2 (e.g., '5.17%' vs '0.0517'); please ensure consistent formatting between figure captions and tables.
Circularity Check
No significant circularity: the scaffold-based generator reuses the authors' prior graph-generative architecture, but the paper's central claims are independently evaluated by holdout/leave-one-out splits and external docking.
full rationale
DeepScaffold's scaffold-based generator is adapted from the authors' previous work [18] (e.g., 'The generation process of side-chain follows largely from our previous work.18'), but that citation supplies architecture and training details rather than the paper's conclusions. The scaffold-based capability is established by independent evaluation: holdout and leave-one-out splits exclude scaffold-molecule pairs from training, and docking against DRD2 is an external computational assay, so no fitted parameter is repackaged as a prediction. The CSK-to-scaffold VAE is trained with a reconstruction objective and evaluated on a held-out 20% split. The pharmacophore side-chain filter is an explicit post-processing step, not a hidden input. The only notable risk, namely that the 'connect two existing nodes' action could join two scaffold atoms and thereby undermine the advertised scaffold-match guarantee, is a correctness or implementation concern rather than circularity: nothing in the derivation assumes the conclusion, and no equation or metric reduces to its own input by construction.
Assumptions & free parameters
free parameters (5)
- beta in beta-VAE =
0.5 (also 0.1 tested)
- latent variable dimension =
10
- importance sampling k and alpha =
k=5, alpha=0.5
- QED threshold for dataset filtering =
0.5
- sampling mixture weight =
0.5
assumptions (6)
- standard math VAE evidence lower bound is a valid training objective for the skeleton-to-scaffold model (standard variational inference).
- standard math U-statistics internal diversity and MMD with Tanimoto kernel are unbiased, valid distributional distance metrics.
- domain assumption ChEMBL molecules with QED>0.5 are a representative training distribution for drug-like chemical space.
- domain assumption The defined action space (append atom, connect two atoms, terminate) can express all relevant side-chain modifications of a scaffold.
- domain assumption A scaffold remains unmodified during generation (guarantee of scaffold matching).
- domain assumption Docking score is a valid proxy for bioactivity against DRD2.
Cite this review
Pith. "Pith review of DeepScaffold: a comprehensive tool for scaffold-based de novo drug discovery using deep learning." pith.science (2026). https://pith.science/paper/5X3NLD32
@misc{pith2026190807209,
author = {Pith},
title = {Pith review of: DeepScaffold: a comprehensive tool for scaffold-based de novo drug discovery using deep learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/5X3NLD32}},
note = {Machine review of arXiv:1908.07209}
}
read the original abstract
The ultimate goal of drug design is to find novel compounds with desirable pharmacological properties. Designing molecules retaining particular scaffolds as the core structures of the molecules is one of the efficient ways to obtain potential drug candidates with desirable properties. We proposed a scaffold-based molecular generative model for scaffold-based drug discovery, which performs molecule generation based on a wide spectrum of scaffold definitions, including BM-scaffolds, cyclic skeletons, as well as scaffolds with specifications on side-chain properties. The model can generalize the learned chemical rules of adding atoms and bonds to a given scaffold. Furthermore, the generated compounds were evaluated by molecular docking in DRD2 targets and the results demonstrated that this approach can be effectively applied to solve several drug design problems, including the generation of compounds containing a given scaffold and de novo drug design of potential drug candidates with specific docking scores. Finally, a command line interface is created.
Figures
Figures from the paper (13 more)
Reference graph
Works this paper leans on
-
[1]
Computational Exploration of Molecular Scaffolds in Medicinal Chemistry
Hu, Y.; Stumpfe, D.; Bajorath, J. Computational Exploration of Molecular Scaffolds in Medicinal Chemistry. J Med Chem 2016, 59, 4062--76
work page 2016
-
[2]
Bemis, G. W.; Murcko, M. A. The properties of known drugs. 1. Molecular frameworks. J Med Chem 1996, 39, 2887--93
work page 1996
-
[3]
Barreiro, E. J. Privileged Scaffolds in Medicinal Chemistry; Royal Society of Chemistry: London, 2015
work page 2015
-
[4]
Varin, T.; Schuffenhauer, A.; Ertl, P.; Renner, S. Mining for bioactive scaffolds with scaffold networks: improved compound set enrichment from primary screening data. J Chem Inf Model 2011, 51, 1528--38
work page 2011
-
[5]
Identification of Bioactive Scaffolds Based on QSAR Models
Nakagawa, T.; Miyao, T.; Funatsu, K. Identification of Bioactive Scaffolds Based on QSAR Models. Mol Inform 2018, 37
work page 2018
-
[6]
N.; Zhang, J.-H.; Raman, P.; Ertl, P.; Schuffenhauer, A
Varin, T.; Gubler, H.; Parker, C. N.; Zhang, J.-H.; Raman, P.; Ertl, P.; Schuffenhauer, A. Compound set enrichment: a novel approach to analysis of primary HTS data. Journal of chemical information and modeling 2010, 50, 2067--2078
work page 2010
-
[7]
Wilkens, S. J.; Janes, J.; Su, A. I. HierS: hierarchical scaffold clustering using topological chemical graphs. Journal of medicinal chemistry 2005, 48, 3182--3193
work page 2005
-
[8]
A.; Schuffenhauer, A.; Scheck, M.; Wetzel, S.; Casaulta, M.; Odermatt, A.; Ertl, P.; Waldmann, H
Koch, M. A.; Schuffenhauer, A.; Scheck, M.; Wetzel, S.; Casaulta, M.; Odermatt, A.; Ertl, P.; Waldmann, H. Charting biologically relevant chemical space: a structural classification of natural products (SCONP). Proc Natl Acad Sci 2005, 102, 17272--7
work page 2005
Show all 41 references
-
[9]
A.; Waldmann, H
Schuffenhauer, A.; Ertl, P.; Roggo, S.; Wetzel, S.; Koch, M. A.; Waldmann, H. The scaffold tree--visualization of the scaffold universe by hierarchical scaffold classification. J Chem Inf Model 2007, 47, 47--58
2007
-
[10]
Scaffold Hunter: a comprehensive visual analytics framework for drug discovery
Schafer, T.; Kriege, N.; Humbeck, L.; Klein, K.; Koch, O.; Mutzel, P. Scaffold Hunter: a comprehensive visual analytics framework for drug discovery. J Cheminform 2017, 9, 28
2017
-
[11]
Deep learning
LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436--44
2015
-
[12]
Hu, Z.; Yang, Z.; Salakhutdinov, R.; Xing, E. P. On Unifying Deep Generative Models. International Conference for Learning Representations
-
[13]
Computer-based de novo design of drug-like molecules
Schneider, G.; Fechner, U. Computer-based de novo design of drug-like molecules. Nat Rev Drug Discov 2005, 4, 649--63
2005
-
[14]
Segler, M. H. S.; Kogej, T.; Tyrchan, C.; Waller, M. P. Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks. ACS Cent Sci 2018, 4, 120--131
2018
-
[15]
N.; Duvenaud, D.; Hernandez-Lobato, J
Gomez-Bombarelli, R.; Wei, J. N.; Duvenaud, D.; Hernandez-Lobato, J. M.; Sanchez-Lengeling, B.; Sheberla, D.; Aguilera-Iparraguirre, J.; Hirzel, T. D.; Adams, R. P.; Aspuru-Guzik, A. Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules. ACS Cent...
2018
-
[16]
Molecular de-novo design through deep reinforcement learning
Olivecrona, M.; Blaschke, T.; Engkvist, O.; Chen, H. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatics 2017, 9, 48
2017
-
[17]
SMILES, a chemical language and information system
Weininger, D. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. Journal of Chemical Information and Modeling 1988, 28, 31--36
1988
-
[18]
Multi-objective de novo drug design with conditional graph generative model
Li, Y.; Zhang, L.; Liu, Z. Multi-objective de novo drug design with conditional graph generative model. J Cheminform 2018, 10, 33
2018
-
[19]
GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders
Simonovsky, M.; Komodakis, N. GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders. arXiv:1610.02920 2017,
2017 arXiv
-
[20]
Learning Deep Generative Models of Graphs
Li, Y.; Vinyals, O.; Dyer, C.; Pascanu, R.; Battaglia, P. Learning Deep Generative Models of Graphs. arXiv:1803.03324 2017,
2017 arXiv
-
[21]
L.; Leskovec, J
You, J.; Ying, R.; Ren, X.; Hamilton, W. L.; Leskovec, J. GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models. International Conference on Machine Learning
-
[22]
Junction Tree Variational Autoencoder for Molecular Graph Generation
Jin, W.; Barzilay, R.; Jaakkola, T. Junction Tree Variational Autoencoder for Molecular Graph Generation. International Conference on Machine Learning
-
[23]
MolGAN: An implicit generative model for small molecular graphs
De Cao, N.; Kipf, T. MolGAN: An implicit generative model for small molecular graphs. arXiv:1805.11973 2018,
2018 arXiv
-
[24]
Latent Molecular Optimization for Targeted Therapeutic Design
Aumentado-Armstrong, T. Latent Molecular Optimization for Targeted Therapeutic Design. arXiv:1809.02032 2018,
2018 arXiv
-
[25]
QBMG: quasi-biogenic molecule generator with deep recurrent neural network
Zheng, S.; Yan, X.; Gu, Q.; Yang, Y.; Du, Y.; Lu, Y.; Xu, J. QBMG: quasi-biogenic molecule generator with deep recurrent neural network. J Cheminform 2019, 11, 5
2019
-
[26]
Lim, J.; Hwang, S.-Y.; Kim, S.; Moon, S.; Kim, W. Y. Scaffold-based molecular design using graph generative model. arXiv:1905.13639 2019,
1905 arXiv
-
[27]
J.; Bento, A
Gaulton, A.; Bellis, L. J.; Bento, A. P.; Chambers, J.; Davies, M.; Hersey, A.; Light, Y.; McGlinchey, S.; Michalovich, D.; Al-Lazikani, B.; Overington, J. P. ChEMBL: a large-scale bioactivity database for drug discovery. Nucleic Acids Res 2012, 40, D1100--7
2012
-
[28]
R.; Paolini, G
Bickerton, G. R.; Paolini, G. V.; Besnard, J.; Muresan, S.; Hopkins, A. L. Quantifying the chemical beauty of drugs. Nat Chem 2012, 4, 90--8
2012
-
[29]
Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions
Ertl, P.; Schuffenhauer, A. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. J Cheminform 2009, 1, 8
2009
-
[30]
P.; Welling, M
Kingma, D. P.; Welling, M. Auto-encoding variational bayes. International Conference for Learning Representations
-
[31]
R.; Vilnis, L.; Vinyals, O.; Dai, A
Bowman, S. R.; Vilnis, L.; Vinyals, O.; Dai, A. M.; Jozefowicz, R.; Bengio, S. Generating sentences from a continuous space. SIGNLL Conference on Computational Natural Language Learning
-
[32]
beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
Higgins, I.; Matthey, L.; Pal, A.; Burgess, C.; Glorot, X.; Botvinick, M.; Mohamed, S.; Lerchner, A. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. ICLR 2017, 2, 6
2017
-
[33]
ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity? arXiv:1708.08227 2017,
Benhenda, M. ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity? arXiv:1708.08227 2017,
2017 arXiv
-
[34]
Mmd gan: Towards deeper understanding of moment matching network
Li, C.-L.; Chang, W.-C.; Cheng, Y.; Yang, Y.; Póczos, B. Mmd gan: Towards deeper understanding of moment matching network. Advances in Neural Information Processing Systems. pp 2203--2213
-
[35]
H.; Taylor, G
Jiwoong Im, D.; Ma, A. H.; Taylor, G. W.; Branson, K. Quantitatively Evaluating GANs With Divergences Proposed for Training. International Conference on Learning Representations
-
[36]
Wasserstein GAN
Arjovsky, M.; Chintala, S.; Bottou, L. Wasserstein GAN. International Conference on Learning Representations
-
[37]
Frechet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery
Preuer, K.; Renz, P.; Unterthiner, T.; Hochreiter, S.; Klambauer, G. Frechet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery. J Chem Inf Model 2018, 58, 1736--1741
2018
-
[38]
Gower, J. C. A general coefficient of similarity and some of its properties. Biometrics 1971, 857--871
1971
-
[39]
Madras, B. K. History of the discovery of the antipsychotic dopamine D2 receptor: a basis for the dopamine hypothesis of schizophrenia. Journal of the History of the Neurosciences 2013, 22, 62--78
2013
-
[40]
P.; Salimans, T.; Duan, Y.; Dhariwal, P.; Schulman, J.; Sutskever, I.; Abbeel, P
Chen, X.; Kingma, D. P.; Salimans, T.; Duan, Y.; Dhariwal, P.; Schulman, J.; Sutskever, I.; Abbeel, P. Variational lossy autoencoder. International Conference on Learning Representations
-
[41]
6DTpDQ 2(C
Kingma, D. P.; Salimans, T.; Jozefowicz, R.; Chen, X.; Sutskever, I.; Welling, M. Improved variational inference with inverse autoregressive flow. Advances in Neural Information Processing Systems. pp 4743--4751 mcitethebibliography article.tex000066400000000000000000000070271...
1999
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.