REVIEW 4 major objections 7 minor 31 references
GeoScatt-GNN: A Geometric Scattering Transform-Based Graph Neural Network Model for Ames Mutagenicity Prediction
T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Scattering-based molecular features are claimed to lift Ames mutagenicity prediction to a 0.9812 AUC, ahead of a CNN baseline.
desk verdict Plausible scattering application, but the headline AUC is unverifiable because the pipeline never separates test molecules from feature construction and the meta-graph. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the geometric scattering transform on molecular graphs, built from tight Hann wavelets and diffusion wavelets, which produces multiscale, permutation-invariant node embeddings; the 2D scattering transform on molecule images plays a supporting role. The hybrid pipeline concatenates these scattering features with graph-isomorphism-network (GIN) embeddings and feeds the result into standard classifiers, while MOLG3-SAGE replaces the classifier with a graph-of-graphs: a fully connected meta-graph whose nodes are molecules, whose edge weights are Gaussian-kernel cosine similarities between scattering embeddings, and whose message passing is done by two GraphSAGE layers. The meta-graph is the mechanism that lets each molecule's prediction be informed by its similarity to every other molecule in the dataset.
What would settle it
Rebuild the meta-graph using only training molecules, assign molecules to train and test by scaffold clusters rather than at random, and rerun both MOLG3-SAGE and the LightGBM-plus-GIN pipeline; if the AUCs stay at roughly 0.9812 and 0.9622 while the CNN baseline stays at 0.954, the claim is supported, and if the gap narrows or inverts, the reported edge depends on split leakage and near-duplicate molecules.
Extended reading notes
Core claim
The central discovery claim is that multiscale wavelet scattering, applied in two versions (a 2D wavelet scattering transform on molecule images and a geometric scattering transform on molecular graphs), captures the structural and spatial information most relevant to mutagenicity better than previous representations. The paper reports that geometric scattering features alone, used with a simple classifier, already rival a CNN model (AUC 0.9507 versus 0.954), and that adding GIN embeddings pushes LightGBM to AUC 0.9812, above all listed baselines. It also claims that organizing the whole dataset as a fully connected meta-graph, in which each molecule is a node and edge weights are Gaussian kernels on cosine similarity of scattering embeddings, and then running GraphSAGE on that meta-graph, yields the best accuracy (0.9301) and MCC (0.8603). The paper frames this as evidence that scattering-based molecular representations set a new performance level for Ames mutagenicity prediction.
Load-bearing premise
The entire comparison assumes the test molecules never leak into the fully connected meta-graph used to train MOLG3-SAGE, and that an 80/20 random split is a fair approximation of external validation.
Editorial extensions
If this is right
- If the reported results are correct, scattering features offer a training-free, multiscale alternative to learned molecular fingerprints, with simple gradient-boosted trees matching or exceeding a purpose-built CNN.
- MOLG3-SAGE's fully connected meta-graph implies that molecule-to-molecule similarity, not just per-molecule structure, carries predictive signal for mutagenicity; retaining that relational information is what the paper credits for its best accuracy and MCC.
- The 2D scattering result implies that molecular images, when transformed rather than fed raw to a CNN, can outperform handcrafted descriptors, suggesting image-based screening pipelines need not abandon interpretable feature engineering.
- Because the scattering transforms are not mutagenicity-specific, the same pipeline is directly portable to other molecular property and toxicity endpoints, subject to revalidation.
Reading between the lines
- Editorial inference: the most informative follow-up test, not reported in the paper, is a scaffold-split evaluation in which the meta-graph is rebuilt from training molecules only; this would separate genuine generalization from reliance on near-duplicate molecules between train and test.
- Editorial inference: since both 2D and geometric scattering are parameter-light and training-free, the likely practical value is in small-data chemistry settings where deep models overfit; a useful extension would be measuring how the AUC gap changes as training set size shrinks.
- Editorial inference: the 0.9812 AUC may approach the ceiling of this benchmark, so the next meaningful comparison is against recent consensus and transformer-based predictors on larger external mutagenicity sets, and on time-split or scaffold-split validation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes scattering-transform-based representations for Ames mutagenicity prediction, combining 2D image scattering, geometric graph scattering (GGS), GIN embeddings, and a fully connected meta-graph GraphSAGE model (MOLG3-SAGE). Using the Hansen et al. benchmark (6,277 compounds after preprocessing and a random 80/20 split), it reports AUC 0.9812 for Lightgbm with GGS+GIN features and ACC 0.9301/AUC 0.9622 for MOLG3-SAGE, and claims state-of-the-art performance against published baselines. The central claim is empirical: scattering features improve mutagenicity screening over descriptors and over published CNN/GNN models.
Significance. Scattering transforms are a plausible and relatively unexplored representation for molecular property prediction, and the paper's claims are concrete and falsifiable: the reported AUCs could in principle be reproduced or refuted with the same data. If the results survive a clean, train-only evaluation protocol, the contribution would be useful to computational toxicology. However, as presented, the evaluation does not rule out test information entering feature construction and meta-graph message passing, and the baseline comparisons are not apples-to-apples. The headline numbers are therefore not currently trustworthy, and the significance of the work hinges on a re-run under a rigorous protocol.
major comments (4)
- [Section 4.3, Eqs. (15)-(18)] The meta-graph is defined over "n molecules" without any statement that test molecules are excluded from the node set and from the edge-weight construction, including the Gaussian-kernel normalization sigma. If test nodes and their pairwise distances participate in GraphSAGE message passing and in the global normalization, the evaluation is transductive, not the inductive comparison made against published models in Table 4. This directly affects the reported MOLG3-SAGE ACC 0.9301 and AUC 0.9622. Please specify and implement a train-only meta-graph construction, with sigma computed from training pairs only, and re-report all affected results.
- [Section 4.2.2 and Section 5] The Chi-squared feature selection on the 11,681-dimensional 2D scattering representation and the generation of GIN embeddings are not described as restricted to the training split. If feature selection or GIN training uses all 6,277 molecules or the test labels, then the Lightgbm GGS+GIN AUC 0.9812 and the 2D-scattering AUC 0.9128 are inflated by information leakage. The manuscript must state whether the GIN is supervised by mutagenicity labels, on which split it is trained, and whether the 4000-feature subset is selected on the training portion only; the experiments should then be rerun under that strict protocol.
- [Tables 3 and 4] The comparison to published baselines is not apples-to-apples. Baseline AUC values in Table 4 come from papers with different preprocessing of the Hansen data, different validation schemes (often cross-validation), and possibly different class distributions, whereas this paper uses a single random 80/20 split after its own filtering to 6,277 compounds. A higher AUC than a published number under different protocols does not establish superiority. Baselines should be rerun under the same split and metrics, or standard benchmark splits (e.g., scaffold split) should be used for all models.
- [Table 3] The central results are reported as point estimates from a single random split, with no standard deviations, no number of seeds, and no code. Given that the best reported AUC (0.9812) is close to other values in the same table (0.9796, 0.9748), the state-of-the-art claim is not statistically supported. Please report means and variances over multiple random splits, and ideally scaffold splits, and make code, seeds, and the exact data split available.
minor comments (7)
- [Abstract and Section 2] The text repeatedly refers to the "ZINC dataset" and "zinc toxicity," but the experiments use the Hansen et al. Ames mutagenicity benchmark; these terms should be corrected throughout.
- [Section 4.2.1 and Table 1] The text says the tight Hann wavelet transform is used with three scales (j = 3), while Table 1 lists the number of tight Hann scales as J = 4; the text and table should be aligned, and the meaning of "No. Tight HANN coefficients 7" should be explained.
- [Section 5] The paper says hyperparameters were optimized with 10-fold cross-validation on the training portion, and Table 3 is labeled "on a test set"; please clarify whether all Table 3 numbers come from the held-out 20% and, if so, state that the test set was used only once after model selection.
- [Section 3.2, Eq. (7)] Equation (7) contains an incomplete or malformed trigonometric expression, and the variables R and e_max are not defined in the text; please fix the equation and define all symbols.
- [Section 5] The description of the GIN architecture is unclear: it says three sequential graph isomorphism layers, then mentions input dimension 7 and "64 hidden layers," followed by fully connected layers yielding 128-dimensional embeddings. Please give the exact layer counts and dimensions.
- [References] Reference [28] is not the original Graph Isomorphism Network paper; the GIN method should be cited to Xu et al., "How Powerful are Graph Neural Networks?" (ICLR 2019).
- [Section 5] The descriptor name "EPCF6" appears to be a typo for "ECFP6."
Circularity Check
No significant circularity: the paper is an empirical benchmark study with no self-citation chain and no definitional reduction of predictions to inputs.
full rationale
The paper is an empirical modeling paper, not a derivation. Its central claims (Lightgbm with GGS+GIN embeddings reaching AUC 0.9812, and MOLG3-SAGE reaching ACC 0.9301) are measured on a held-out 20% test split, not derived from the feature definitions. The scattering transforms (2D WST and geometric graph scattering) are defined independently of the mutagenicity label in Eqs. (1)-(3), (5)-(11), and the machine learning and GNN models are trained rather than algebraically forced. The meta-graph edge weights in Eqs. (15)-(18) reuse the scattering embeddings that also serve as node features; this is feature reuse within a model, not a circular definition, because mutagenicity is not defined in terms of those embeddings. The paper contains no self-citations that carry the argument: citations [20]-[25] are external mathematical and methodological sources for wavelets and graph scattering. The main weaknesses that can be inferred from the text--ambiguous train/test separation for feature selection, the potentially transductive MOLG3-SAGE meta-graph, and the random 80/20 split without scaffold separation--are evaluation-protocol concerns, not circularity in the sense of a fitted parameter renamed as a prediction or an equation reducing to its own input. No circular step can be exhibited with the required quote and explicit reduction, so the appropriate score is 0.
Assumptions & free parameters
free parameters (6)
- Tight Hann wavelet scale count =
4 in Table 1; text says j=3 gave best results
- Diffusion wavelet layer count =
3
- 2D WST scales and orientations =
J=9, L=8
- 2D scattering feature subset size =
4000
- Gaussian diffusion kernel sigma =
standard deviation of the distance matrix
- MOLG3-SAGE hidden and embedding dimensions =
hidden 128, embedding 64
assumptions (5)
- standard math The wavelet scattering transform theory (Mallat) provides stable, informative representations.
- domain assumption Scattering coefficients preserve the chemical structural information needed for mutagenicity prediction.
- domain assumption Cosine similarity between scattering embeddings is a valid molecular similarity measure for the meta-graph edges.
- domain assumption Hansen et al. labels, after the clear-evidence rule, are reliable ground truth.
- ad hoc to paper Test molecules are not part of the meta-graph used to train MOLG3-SAGE.
Cite this review
Pith. "Pith review of GeoScatt-GNN: A Geometric Scattering Transform-Based Graph Neural Network Model for Ames Mutagenicity Prediction." pith.science (2026). https://pith.science/paper/XFCUAHYL
@misc{pith2026241115331,
author = {Pith},
title = {Pith review of: GeoScatt-GNN: A Geometric Scattering Transform-Based Graph Neural Network Model for Ames Mutagenicity Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/XFCUAHYL}},
note = {Machine review of arXiv:2411.15331}
}
read the original abstract
This paper tackles the pressing challenge of mutagenicity prediction by introducing three ground-breaking approaches. First, it showcases the superior performance of 2D scattering coefficients extracted from molecular images, compared to traditional molecular descriptors. Second, it presents a hybrid approach that combines geometric graph scattering (GGS), Graph Isomorphism Networks (GIN), and machine learning models, achieving strong results in mutagenicity prediction. Third, it introduces a novel graph neural network architecture, MOLG3-SAGE, which integrates GGS node features into a fully connected graph structure, delivering outstanding predictive accuracy. Experimental results on the ZINC dataset demonstrate significant improvements, emphasizing the effectiveness of blending 2D and geometric scattering techniques with graph neural networks. This study illustrates the potential of GNNs and GGS for mutagenicity prediction, with broad implications for drug discovery and chemical safety assessment.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
B. N. Ames, W. E. Durston, E. Yamasaki, F. D. Lee, Carcinogens are mutagens: a simple test system combining liver homogenates for activation and bacteria for detection, Proceedings of the National Academy of Sciences 70 (1973) 2281–2285
work page 1973
-
[2]
M. Honma, A. Kitazawa, A. Cayley, R. V . Williams, C. Barber, T. Hanser, R. Saiakhov, S. Chakravarti, G. J. Myatt, K. P. Cross, et al., Improvement of quantitative structure–activity relationship (qsar) tools for predicting ames mutagenicity: outcomes of the ames/qsar inter- national challenge project, Mutagenesis 34 (2019) 3–16
work page 2019
-
[3]
S. V . Feeney, R. Lui, D. Guan, S. Matthews, Multiple instance learning improves ames mutagenicity prediction for problematic molecular species, Chemical Research in Toxicology 36 (2023) 1227–1237
work page 2023
-
[4]
T. Li, Z. Liu, S. Thakkar, R. Roberts, W. Tong, Deepames: A deep learning-powered ames test predictive model with potential for regulatory application, Regulatory Toxicology and Pharmacology 144 (2023) 105486
work page 2023
-
[5]
S. Li, L. Zhang, H. Feng, J. Meng, D. Xie, L. Yi, I. T. Arkin, H. Liu, Mutagenpred-gcnns: a graph convolutional neural network-based classification model for mutagenicity prediction with data-driven molecular fingerprints, Interdisciplinary Sciences: Computational Life Sciences 13 (2021) 25–33
work page 2021
-
[6]
T. T. Van Tran, H. Tayara, K. T. Chong, Ampred-cnn: Ames mutagenicity prediction model based on convolutional neural networks, Computers in Biology and Medicine 176 (2024) 108560
work page 2024
- [7]
-
[8]
C. S. Chu, J. D. Simpson, P. M. O’Neill, N. G. Berry, Machine learning–predicting ames mutagenicity of small molecules, Journal of Molecular Graphics and Modelling 109 (2021) 108011
work page 2021
Show all 31 references
-
[9]
Venkatraman, Fp-admet: a compendium of fingerprint-based admet prediction models, Journal of cheminformatics 13 (2021) 1–12
V . Venkatraman, Fp-admet: a compendium of fingerprint-based admet prediction models, Journal of cheminformatics 13 (2021) 1–12
2021
-
[10]
N. K. Shinada, N. Koyama, M. Ikemori, T. Nishioka, S. Hitaoka, A. Hakura, S. Asakura, Y . Matsuoka, S. K. Palaniappan, Optimizing machine-learning models for mutagenicity prediction through better feature selection, Mutagenesis 37 (2022) 191–202
2022
-
[11]
C. Lou, H. Yang, H. Deng, M. Huang, W. Li, G. Liu, P. W. Lee, Y . Tang, Chemical rules for optimization of chemical mutagenicity via matched molecular pairs analysis and machine learning methods, Journal of Cheminformatics 15 (2023) 35
2023
-
[12]
Kumar, F
R. Kumar, F. U. Khan, A. Sharma, M. H. Siddiqui, I. B. Aziz, M. A. Kamal, G. M. Ashraf, B. S. Alghamdi, M. S. Uddin, A deep neural network–based approach for prediction of mutagenicity of compounds, Environmental Science and Pollution Research 28 (2021) 47641– 47650
2021
-
[13]
R. Lui, D. Guan, S. Matthews, Mechanistic task groupings enhance multitask deep learning of strain-specific ames mutagenicity, Chemical Research in Toxicology 36 (2023) 1248–1254
2023
-
[14]
Xiong, Z
G. Xiong, Z. Wu, J. Yi, L. Fu, Z. Yang, C. Hsieh, M. Yin, X. Zeng, C. Wu, A. Lu, et al., Admetlab 2.0: an integrated online platform for accurate and comprehensive predictions of admet properties, Nucleic acids research 49 (2021) W5–W14
2021
-
[15]
C. Hung, G. Gini, Qsar modeling without descriptors using graph convolutional neural networks: the case of mutagenicity prediction, Molecular diversity 25 (2021) 1283–1299
2021
-
[16]
J. Guo, Q. Liu, H. Guo, X. Lu, Ligandformer: A graph neural network for predicting compound property with robust interpretation, arXiv preprint arXiv:2202.10873 (2022)
2022 arXiv
-
[17]
Y . Wei, S. Li, Z. Li, Z. Wan, J. Lin, Interpretable-admet: a web service for admet prediction and optimization based on deep neural representation, Bioinformatics 38 (2022) 2863–2871
2022
-
[18]
J. M. Lilly, S. C. Olhede, Higher-order properties of analytic wavelets, IEEE Transactions on Signal Processing 57 (2008) 146–160
2008
-
[19]
Mallat, A wavelet tour of signal processing, Third edition, Academic Press (2008)
S. Mallat, A wavelet tour of signal processing, Third edition, Academic Press (2008)
2008
-
[20]
Mallat, Group invariant scattering, Communications on Pure and Applied Mathematics 65 (2012) 1331–1398
S. Mallat, Group invariant scattering, Communications on Pure and Applied Mathematics 65 (2012) 1331–1398
2012
-
[21]
F. Gama, A. Ribeiro, J. Bruna, Diffusion scattering transforms on graphs, arXiv preprint arXiv:1806.08829 (2018)
2018 arXiv
-
[22]
D. Zou, G. Lerman, Graph convolutional neural networks via scattering, Applied and Computational Harmonic Analysis 49 (2020) 1046– 1074
2020
-
[23]
R. R. Coifman, M. Maggioni, Diffusion wavelets, Applied and computational harmonic analysis 21 (2006) 53–94
2006
-
[24]
Hansen, S
K. Hansen, S. Mika, T. Schroeter, A. Sutter, A. Ter Laak, T. Steger-Hartmann, N. Heinrich, K.-R. Muller, Benchmark data set for in silico prediction of ames mutagenicity, Journal of chemical information and modeling 49 (2009) 2077–2081
2009
-
[25]
Jiang, Z
Q. Jiang, Z. Shen, Tight wavelet frames in low dimensions with canonical filters, Journal of Approximation Theory 196 (2015) 55–78
2015
-
[26]
Scarselli, M
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, G. Monfardini, The graph neural network model, IEEE transactions on neural networks 20 (2008) 61–80
2008
-
[27]
Hamilton, Z
W. Hamilton, Z. Ying, J. Leskovec, Inductive representation learning on large graphs, Advances in neural information processing systems 30 (2017)
2017
-
[28]
B.-H. Kim, J. C. Ye, Understanding graph isomorphism network for rs-fmri functional connectivity analysis, Frontiers in neuroscience 14 (2020) 630
2020
-
[29]
Banerjee, A
P. Banerjee, A. O. Eckert, A. K. Schrey, R. Preissner, Protox-ii: a webserver for the prediction of toxicity of chemicals, Nucleic acids research 46 (2018) W257–W263
2018
-
[30]
Karim, A
A. Karim, A. Mishra, M. H. Newton, A. Sattar, Efficient toxicity prediction via simple features using shallow neural networks and decision trees, Acs Omega 4 (2019) 1874–1888
2019
-
[31]
J. Rao, S. Zheng, Y . Lu, Y . Yang, Quantitative evaluation of explainable graph neural networks for molecular property prediction, Patterns 3 (2022). 18
2022
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.