REVIEW 3 major objections 5 minor 21 references
Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Clustering makes molecular-orbital machine learning up to 35,000 times faster at chemical accuracy.
desk verdict Solid, pragmatic MOB-ML scaling paper; the accuracy results hold up, but the headline speedup factors need a same-hardware baseline before they can be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the greedy regression-clustering algorithm of Späth applied to MOB feature vectors: it alternates ordinary-least-squares fits of each cluster with reassigning every data point to the nearest linear fit under the squared-residual distance metric, so clusters are chosen to be locally linear rather than merely compact (as K-means would do). This exploits the paper's empirical observation that diagonal pair-correlation energies vary smoothly and nearly linearly with MOB features such as the Fock matrix element. The workflow is completed by per-cluster LR/GPR regressors and a random forest classifier that assigns test feature vectors to clusters, with an ensemble of ten models averaged for final predictions.
What would settle it
Re-time both the clustered and unclustered MOB-ML pipelines on the same machine with the same core budget and the same serial/parallel setting, using the same 300-molecule training set; if the wall-clock ratio is much smaller than 35,000, the headline speedup is an artifact of the baseline timing.
Extended reading notes
Core claim
The central claim is that the correlation-energy landscape in MOB feature space is locally linear, and that an explicitly linear clustering can convert that structure into a large training-speed win without sacrificing accuracy. RC partitions the training feature/label pairs by minimizing total squared residual to an ensemble of linear fits; each cluster is regressed independently with LR or GPR; and an RFC predicts cluster membership for test vectors. With this pipeline, MOB-ML reaches chemical accuracy on QM7b-T with 300 training molecules, at reported training-time reductions of about 35,000-fold (RC/LR/RFC) and 4,500-fold (RC/GPR/RFC) versus the unclustered GPR baseline, with even better cost-accuracy tradeoff when the largest cluster is capped. The main remaining error source, by the authors' analysis, is the random forest classifier's imperfect assignment of borderline feature vectors, not the regression itself.
Load-bearing premise
The central efficiency claim assumes the unclustered Gaussian-process training times used for comparison were measured under the same hardware and parallelization conditions; if the baseline was slower in setup, the reported speedup factors are inflated.
Editorial extensions
If this is right
- MOB-ML training time drops from cubic scaling in the number of training pairs to roughly the cube of the largest cluster, enabling chemical accuracy with only seconds of parallelized training.
- With a perfect classifier, linear regression matches or beats GPR on this problem, so improved classifiers directly translate into better accuracy for the clustered MOB-ML approach.
- Capping the maximum cluster size lets additional training data improve accuracy without increasing the dominant GPR cost, since the largest cluster no longer grows.
- Models trained on small (up to seven heavy atoms) molecules retain transferability to larger (thirteen heavy atoms) molecules, with RC/GPR/RFC reaching the per-atom chemical-accuracy threshold that the unclustered baseline could not reach because of training cost.
- Other kernel-based regression methods with high training-time complexity, such as kernel ridge regression, would similarly benefit from clustering.
Reading between the lines
- A direct testable extension is to apply the same RC/LR/RFC recipe to other molecular properties that may share local linearity in MOB space, such as dipole moments or excitation energies; if local linearity is general, similar speedups should appear.
- Because the classifier is the error bottleneck, using soft or fuzzy cluster assignments at test time could remove both the misclassification penalty and the discontinuities the paper notes in predicted energies along molecular dynamics trajectories.
- The chemically intuitive clusters suggest RC could double as an unsupervised tool for mapping chemical space and selecting diverse training points, which would matter for active-learning settings beyond energy prediction.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces a regression-clustering (RC) extension of molecular-orbital-based machine learning (MOB-ML). The training data are partitioned into locally linear clusters via a greedy clusterwise linear regression algorithm; each cluster is then regressed independently with either linear regression (LR) or Gaussian process regression (GPR), and a random forest classifier (RFC) assigns new feature vectors to clusters. On the QM7b-T set, the authors report that RC/LR/RFC and RC/GPR/RFC reach 1 kcal/mol accuracy with about 300 training molecules, reduce wall-clock training time by roughly 35,000-fold and 4,500-fold relative to previous GPR-based MOB-ML, retain transferability to GDB-13-T, and benefit from capping the largest cluster size. The paper also presents evidence that the RC clusters correspond to chemically intuitive localized-orbital types and that the RFC introduces the dominant error, as shown by comparison with a 'perfect' classifier.
Significance. If the accuracy and training-cost claims survive scrutiny, this is a practically valuable contribution that directly addresses the O(M^3) training bottleneck of GPR-based MOB-ML while preserving the transferability advantage of MOB features. The study has several real strengths: predictions are averaged over 10 independently trained models with reported standard errors, the error decomposition using a perfect classifier cleanly isolates classification error, alternative classifiers are tested, and transferability from QM7b-T to GDB-13-T is explicitly evaluated. The main caveats are that the headline speedup factors compare against a baseline whose hardware and parallelization are unspecified, that the RC/LR/RFC speedup partly reflects a change of regressor rather than clustering alone, and that the number of clusters is selected using the evaluation test set. These issues are fixable with additional controlled benchmarks and validation, and the qualitative conclusions about accuracy and transferability are likely to remain.
major comments (3)
- [Section 4.4, Fig. 8, and Abstract] The wall-clock speedup factors reported in the Abstract and Section 4.4 (35,000-fold for RC/LR/RFC and 4,500-fold for RC/GPR/RFC) are computed relative to 'GPR without clustering' from Ref. 48, but the manuscript does not state the hardware, core count, serial/parallel execution, or whether hyperparameter optimization and I/O overhead are included in the baseline timing. Since the clustered pipeline explicitly uses 16 cores for per-cluster regression and 200 cores for RFC training, the reported ratios mix the effect of clustering with the effect of parallelization and hardware differences. A controlled benchmark on the same node with matched core counts, or a clear statement of the baseline conditions with a recomputed ratio, is required to support the headline training-cost claim.
- [Section 4.3, Fig. 6] The number of clusters for the diagonal (N=20) and off-diagonal (N=70) regressions is selected by inspecting prediction MAE on the held-out test portion of QM7b-T, and the same test set is then used to produce the learning curves in Figs. 7-10. This constitutes hyperparameter selection on the evaluation set and can optimistically bias the reported accuracies and transferability estimates. Please provide an internal validation procedure, such as nested cross-validation or a separate calibration set, or explicitly quantify how much the conclusions depend on the choice of N.
- [Section 4.4 and Fig. 8] The 35,000-fold speedup for RC/LR/RFC is not an apples-to-apples measure of the value of regression clustering because the baseline is GPR without clustering, so the comparison conflates a change of regression model (LR instead of GPR) with the clustering contribution. The 4,500-fold RC/GPR/RFC comparison is cleaner in this regard but still mixes clustering with parallelization, as noted in the first comment. Please report a same-regressor baseline, for example GPR without clustering parallelized on the same hardware, or LR without clustering, to separate the contributions of clustering, regression method, and parallelization.
minor comments (5)
- [Section 2.3, page 7] The text says 'i.d.d. sequence'; this should be 'i.i.d. sequence'.
- [Section 4.4, page 18] The model name 'RC/GRP/RFC' appears in the text and should be 'RC/GPR/RFC'.
- [Section 4.4, page 18] The sentence 'it has previously been shown than that MOB-ML ...' contains a typo ('than that' should be 'that').
- [Fig. 7 caption] The caption says 'with up to heavy seven atoms'; this should be 'with up to seven heavy atoms'.
- [Throughout] The unusual spacing in 'S CIKIT -LEARN' appears to be a formatting artifact; it should read 'scikit-learn'.
Circularity Check
No circular reasoning found: reported accuracies are evaluated on held-out molecules, and the perfect-classifier comparison is explicitly labeled impractical and used only for error decomposition.
full rationale
The derivation is self-contained with respect to the circularity criteria. The RC step (Section 2.3) optimizes the objective in Eq. 4 using only training labels, and the reported RC/LR/RFC and RC/GPR/RFC errors are computed on held-out molecules with cluster assignments made by an RFC trained on features and cluster labels exclusively from the training set. The only leakage-like device, the 'perfect classifer,' is explicitly not presented as a prediction: the paper states that it 'is not generally practical because it assumes prior knowledge of the reference energy labels for the test molecules' and that it 'should be regarded as a best case scenario' (Section 3.3); it is used only for error-source analysis, so it does not inject test information into the central claims. The local-linearity motivation is supported by direct numerical illustration rather than imported from the conclusion. MOB-ML features are reused from prior work (Ref. 48), but that is standard reuse of published methodology, not a step that reduces the new results to their own inputs. The wall-clock speedup comparison to 'GPR without clustering' from Ref. 48 raises a legitimate benchmark-comparability concern about hardware and parallelization, but that is a measurement/fairness issue, not a circularity: the clustered models are neither fitted to the baseline timings nor defined in terms of the claimed speedup. Therefore no load-bearing step in the derivation is equivalent to its inputs by construction.
Assumptions & free parameters
free parameters (4)
- number of clusters for diagonal regression (N_diag) =
20
- number of clusters for off-diagonal regression (N_offdiag) =
70
- RC convergence threshold =
1e-8 kcal^2/mol^2
- cluster size cap N_cap =
100, 200, 300, 500, 800
assumptions (4)
- standard math Nesbet's theorem allows exact decomposition of correlation energy into MO pair contributions (Eq. 1)
- domain assumption The pair correlation energy functional is universal and can be learned from MOB features
- domain assumption Pair correlation energies vary smoothly and approximately linearly with MOB feature values within clusters
- standard math Greedy RC algorithm (Algorithm 1) converges to clusters with low regression error
Cite this review
Pith. "Pith review of Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning." pith.science (2026). https://pith.science/paper/47KHNEZV
@misc{pith2026190902041,
author = {Pith},
title = {Pith review of: Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/47KHNEZV}},
note = {Machine review of arXiv:1909.02041}
}
read the original abstract
Machine learning (ML) in the representation of molecular-orbital-based (MOB) features has been shown to be an accurate and transferable approach to the prediction of post-Hartree-Fock correlation energies. Previous applications of MOB-ML employed Gaussian Process Regression (GPR), which provides good prediction accuracy with small training sets; however, the cost of GPR training scales cubically with the amount of data and becomes a computational bottleneck for large training sets. In the current work, we address this problem by introducing a clustering/regression/classification implementation of MOB-ML. In a first step, regression clustering (RC) is used to partition the training data to best fit an ensemble of linear regression (LR) models; in a second step, each cluster is regressed independently, using either LR or GPR; and in a third step, a random forest classifier (RFC) is trained for the prediction of cluster assignments based on MOB feature values. Upon inspection, RC is found to recapitulate chemically intuitive groupings of the frontier molecular orbitals, and the combined RC/LR/RFC and RC/GPR/RFC implementations of MOB-ML are found to provide good prediction accuracy with greatly reduced wall-clock training times. For a dataset of thermalized geometries of 7211 organic molecules of up to seven heavy atoms, both implementations reach chemical accuracy (1 kcal/mol error) with only 300 training molecules, while providing 35000-fold and 4500-fold reductions in the wall-clock training time, respectively, compared to MOB-ML without clustering. The resulting models are also demonstrated to retain transferability for the prediction of large-molecule energies with only small-molecule training data. Finally, it is shown that capping the number of training datapoints per cluster leads to further improvements in prediction accuracy with negligible increases in wall-clock training time.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Machine-learning approaches in drug discovery: Methods and applications
(1) Lavecchia, A. Machine-learning approaches in drug discovery: Methods and applications. Drug Discov. Today 2015, 20, 318–331. (2) Gawehn, E.; Hiss, J. A.; Schneider, G. Deep learning in drug discovery. Mol. Inform. 2016, 35, 3–14. (3) Popova, M.; Isayev, O.; Tropsha, A. Deep reinforcement learning for de novo drug design. Sci. Adv. 2018, 4, eaap7885. (...
work page 2015
-
[12]
(37) Ramakrishnan, R.; Hartmann, M.; Tapavicza, E.; von Lilienfeld, O. A. Electronic spectra from TDDFT and machine learning in chemical space. J. Chem. Phys. 2015, 143, 084111. (38) Gastegger, M.; Behler, J.; Marquetand, P. Machine learning molecular dynamics for the sim- ulation of infrared spectra. Chem. Sci. 2017, 8, 6924–6935. (39) Yao, K.; Herr, J. ...
work page 2015
-
[43]
(36) Smith, J. S.; Nebgen, B. T.; Zubatyuk, R.; Lubbers, N.; Devereux, C.; Barros, K.; Tretiak, S.; Isayev, O.; Roitberg, A. E. Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning. Nat. Commun. 2019, 10,
work page 2019
-
[53]
J.; Wolverton, C.; Hattrick-Simpers, J.; Mehta, A
(7) Ren, F.; Ward, L.; Williams, T.; Laws, K. J.; Wolverton, C.; Hattrick-Simpers, J.; Mehta, A. Accelerated discovery of metallic glasses through iteration of machine learning and high- throughput experiments. Sci. Adv. 2018, 4, eaaq1566. 24 (8) Butler, K. T.; Davies, D. W.; Cartwright, H.; Isayev, O.; Walsh, A. Machine learning for molecular and materia...
work page 2018
-
[275]
Algorithm 39: clusterwise linear regression
(56) Sp ¨ath, H. Algorithm 39: clusterwise linear regression. Computing 1979, 367–373. (57) Lloyd, S. Least squares quantization in PCM. IEEE Trans. Inf. Theory 1982, 28, 129–137. (58) Montavon, G.; Rupp, M.; Gobre, V .; Vazquez-Mayagoitia, A.; Hansen, K.; Tkatchenko, A.; M¨uller, K.-R.; von Lilienfeld, O. A. Machine learning of molecular electronic prope...
work page 1979
-
[360]
FCHL revisited: faster and more accurate quantum machine learning
(67) Cortes, C.; Jackel, L. D.; Solla, S. A.; Vapnik, V .; Denker, J. S. In Advances in Neural In- formation Processing Systems 6 ; Cowan, J. D., Tesauro, G., Alspector, J., Eds.; Morgan- Kaufmann, 1994; pp 327–334. (68) Baraldi, A.; Blonda, P. A survey of fuzzy clustering algorithms for pattern recognition. I. IEEE Trans. Syst. Man Cybern. B Cybern. 1999...
work page Pith review arXiv 1994
-
[513]
T.; Sz ´ekely, E.; Imbalzano, G.; Behler, J.; Cs ´anyi, G.; Ceriotti, M.; G ¨otz, A
26 (27) Nguyen, T. T.; Sz ´ekely, E.; Imbalzano, G.; Behler, J.; Cs ´anyi, G.; Ceriotti, M.; G ¨otz, A. W.; Paesani, F. Comparison of permutationally invariant polynomials, neural networks, and Gaus- sian approximation potentials in representing water interactions through many-body expan- sions. J. Chem. Phys. 2018, 148, 241725. (28) Fujikake, S.; Deringe...
work page 2018
-
[595]
(5) Mater, A. C.; Coote, M. L. Deep learning in chemistry. J. Chem. Inf. Model 2019, 59, 2545–
work page 2019
Show all 21 references
-
[618]
Local treatment of electron correlation
(50) Saebo, S.; Pulay, P. Local treatment of electron correlation. Annu. Rev. Phys. Chem. 1993, 44, 213–236. (51) Bartlett, R. J.; Watts, J. D.; Kucharski, S. A.; Noga, J. Non-iterative fifth-order triple and quadruple excitation energy corrections in correlated methods. Chem. ...
1993
-
[872]
human learning
(43) McGibbon, R. T.; Taube, A. G.; Donchev, A. G.; Siva, K.; Hern´andez, F.; Hargus, C.; Law, K.- H.; Klepeis, J. L.; Shaw, D. E. Improving the accuracy of Møller-Plesset perturbation theory with neural networks. J. Chem. Phys. 2017, 147, 161725. (44) Nudejima, T.; Ikabata, Y...
2017 arXiv
-
[1007]
(61) Pedregosa, F. et al. Scikit-learn: machine learning in python (v0.21.2). J. Mach. Learn. Res. 2011, 12,
2011
-
[1632]
29 (54) Szabo, A.; Ostlund, N. S. Modern Quantum Chemistry; Dover: Mineola, 1996; pp 231–239. (55) Sp ¨ath, H. Correction to algorithm 39: clusterwise linear regression. Computing 1979, 26,
1996
-
[1844]
T.; Arbabzadah, F.; Chmiela, S.; M¨uller, K.-R.; Tkatchenko, A
(25) Sch ¨utt, K. T.; Arbabzadah, F.; Chmiela, S.; M¨uller, K.-R.; Tkatchenko, A. Quantum-chemical insights from deep tensor neural networks. Nat. Commun. 2017, 8, 13890. (26) Wu, Z.; Ramsundar, B.; Feinberg, E. N.; Gomes, J.; Geniesse, C.; Pappu, A. S.; Leswing, K.; Pande, V ...
2017
-
[2006]
Decision forests: A unified framework for clas- sification, regression, density estimation, manifold learning and semi-supervised learning
(64) Criminisi, A.; Shotton, J.; Konukoglu, E. Decision forests: A unified framework for clas- sification, regression, density estimation, manifold learning and semi-supervised learning. F ound. Trends Comput. Graph. Vis.2012, 7, 81–227. (65) Fan, R.-E.; Chang, K.-W.; Hsieh, C.-...
2012
-
[2087]
Perspective: Machine learning potentials for atomistic simulations
(23) Behler, J. Perspective: Machine learning potentials for atomistic simulations. J. Chem. Phys. 2016, 145, 170901. (24) Paesani, F. Getting the right answers for the right reasons: toward predictive molecular simu- lations of water with many-body potential energy functions....
2016
-
[2269]
S.; Faber, F
(40) Christensen, A. S.; Faber, F. A.; von Lilienfeld, O. A. Operators in quantum machine learning: Response properties in chemical space. J. Chem. Phys. 2019, 150, 064105. (41) Ghosh, K.; Stuke, A.; Todorovi, M.; Jrgensen, P. B.; Schmidt, M. N.; Vehtari, A.; Rinke, P. Deep le...
2019
-
[2559]
Virtual screening of inorganic materials synthesis parameters with deep learning
(6) Kim, E.; Huang, K.; Jegelka, S.; Olivetti, E. Virtual screening of inorganic materials synthesis parameters with deep learning. npj Comput. Mater .2017, 3,
2017
-
[2903]
S.; Barros, K
(17) Lubbers, N.; Smith, J. S.; Barros, K. Hierarchical modeling of molecular energies using a deep neural network. J. Chem. Phys. 2018, 148, 241715. 25 (18) Bart ´ok, A. P.; Payne, M. C.; Kondor, R.; Cs ´anyi, G. Gaussian approximation potentials: the accuracy of quantum mech...
2018
-
[3404]
O.; Rupp, M.; von Lilienfeld, O
(22) Ramakrishnan, R.; Dral, P. O.; Rupp, M.; von Lilienfeld, O. A. Big data meets quantum chemistry approximations: the ∆-machine learning approach.J. Chem. Theory Comput. 2015, 11,
2015
-
[5776]
Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics
(30) Zhang, L.; Han, J.; Wang, H.; Car, R.; E, W. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett. 2018, 120, 143001. (31) Nandy, A.; Duan, C.; Janet, J. P.; Gugler, S.; Kulik, H. J. Strategies and software for machin...
2018 arXiv
-
[8732]
(60) Dunning, T. H. Gaussian basis sets for use in correlated molecular calculations. I. The atoms boron through neon and hydrogen. J. Chem. Phys. 1989, 90,
1989
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.