Fidelity of Machine Learned Potentials: Quantitative Assessment for Protonated Oxalate
Pith reviewed 2026-05-10 13:45 UTC · model grok-4.3
The pith
Two different machine-learned potentials for protonated oxalate agree closely on vibrational energies, infrared spectra, and hydrogen tunneling splittings.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Two widely applied machine-learned potential energy surfaces, one based on permutationally invariant polynomial regression and the other on the PhysNet neural network approach, are trained on the same dataset for protonated oxalate. When these surfaces are used in VSCF/VCI calculations of vibrational energies and wavefunctions, in direct comparison of infrared spectra, and in tunneling splitting calculations via ring polymer instanton theory, diffusion Monte Carlo, and the Q_im path method, the two surfaces yield results in excellent agreement with each other.
What carries the argument
Stress tests consisting of VSCF/VCI vibrational calculations, ring polymer instanton theory, diffusion Monte Carlo simulations, and the Q_im path method that evaluate the potentials on roughly one billion points across the 15-dimensional space.
If this is right
- Either potential can be used interchangeably for computing spectroscopic properties and tunneling rates in this system.
- Agreement on these quantities indicates that both approaches faithfully capture the potential beyond the training set points.
- The stress-test protocol provides a practical way to check fidelity for other machine-learned surfaces in high-dimensional spaces.
- The results support the use of either regression method for similar polyatomic molecules where extensive dynamical calculations are needed.
Where Pith is reading between the lines
- If similar agreement appears for other molecules, the need to maintain multiple independent machine-learned surfaces for cross-validation could decrease.
- The protocol could be extended to larger systems or to properties such as reaction rates that also sample distant regions of configuration space.
- Disagreement on only one of the three tunneling methods would point to a specific limitation in how a given potential handles rare but important geometries.
Load-bearing premise
The selected stress tests are sufficient to reveal any meaningful differences in how the two potentials represent the underlying surface throughout the full 15-dimensional space.
What would settle it
A clear numerical discrepancy, larger than method-specific uncertainties, appearing in the tunneling splitting values or in the vibrational energy levels computed from the two potentials.
Figures
read the original abstract
There has been a veritable explosion of methods and software to perform machine-learned regression on datasets of electronic energies and forces to develop high-dimensional machine learned potential energy surfaces (ML-PESs). A major, but not deeply-studied aspect is how well different ML-PESs represent the same dataset on which they are trained, beyond the standard fitting precision metrics. Here, this is examined in detail using several ''stress tests'', for two widely applied machine-learned potential approaches. One is based on permutationally invariant polynomial (PIP) linear least square regression and the other is the message-passing neural network PhysNet approach. These potentials and dipole moment surfaces are used in VSCF/VCI calculations of vibrational energies and wavefunctions. The energies from the two PESs are directly compared as are the IR spectra. In addition, tunneling splittings for the hydrogen transfer between two equivalent structures are reported from using three methods: ring polymer instanton theory, diffusion Monte Carlo simulations, and the $Q_{im}$ path method. These calculations require the evaluation of on the order of one billion energies that are widely dispersed in the 15-dimensional configurational space. The two PESs yield results for these quantities in excellent agreement with each other.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript examines the fidelity of two machine-learned potential energy surfaces (ML-PESs) for protonated oxalate (a 15-dimensional system) trained on the same electronic structure dataset: a permutationally invariant polynomial (PIP) linear least-squares model and the PhysNet message-passing neural network. Beyond standard fitting metrics, the authors apply multiple stress tests—VSCF/VCI calculations of vibrational energies and IR spectra, plus tunneling splittings obtained independently via ring-polymer instanton theory, diffusion Monte Carlo, and the Q_im path method—requiring ~10^9 energy evaluations widely dispersed across configuration space. The central claim is that the two PESs (and associated dipole surfaces) produce results in excellent quantitative agreement with each other on all tested observables.
Significance. If the reported agreement holds, the work provides concrete evidence that distinct ML architectures can faithfully represent the same underlying PES for high-dimensional quantum dynamics properties, including tunneling, without being limited by fitting precision alone. The multi-method cross-validation (~10^9 points) and use of independent quantum methods constitute a genuine advance over typical ML-PES validation. This strengthens confidence in deploying ML-PESs for spectroscopy and reaction dynamics in complex molecular systems.
minor comments (3)
- [Abstract/Introduction] The abstract and introduction would benefit from a brief explicit statement of the training-set size, energy range, and force RMSE values for both models to allow immediate comparison with the stress-test results.
- [VSCF/VCI calculations] In the VSCF/VCI section, clarify whether the same vibrational basis and mode-coupling scheme were used for both PESs or whether any re-optimization was performed; this would strengthen the direct comparability of the reported energies and spectra.
- [Figures and Tables] Figure captions for the IR spectra and tunneling-splitting tables should include the precise number of points sampled in each method and the coordinate ranges covered, to substantiate the claim of wide dispersion in the 15D space.
Simulated Author's Rebuttal
We thank the referee for the positive assessment of our manuscript and for recommending acceptance. The referee's summary correctly captures the central claim and the extensive stress-testing performed across vibrational properties and tunneling splittings.
Circularity Check
No significant circularity detected
full rationale
The paper's central claim is that two independently trained ML-PESs (PIP linear least-squares and PhysNet neural network) produce excellent agreement on VSCF/VCI vibrational energies/IR spectra and on tunneling splittings computed via three external methods (ring-polymer instanton, DMC, Q_im path). These stress tests evaluate ~10^9 points dispersed across the 15D space and are not derived from the PES fitting procedure itself. No load-bearing step reduces the agreement result to a fitted parameter, self-definition, or self-citation chain; the multi-method cross-validation supplies independent evidence. Minor self-citations to prior PIP or PhysNet work exist but are not invoked to justify the fidelity conclusion.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption VSCF/VCI calculations accurately reflect differences in PES quality for vibrational energies and wavefunctions
- domain assumption Ring polymer instanton theory, diffusion Monte Carlo, and Q_im path method provide reliable tunneling splittings from a given PES
Reference graph
Works this paper leans on
-
[1]
1250.13 1.70018 1241.07 27.9534 90.8098 3.45034 1213.17 29.7036 49.193 5.20051 1167.59 31.4538 20.038 6.9507 1106.19 33.2041 3.61036 8.70089 1031.33 34.2042 0. 10.4511 945.749 42.7052 135.185 12.2013 852.367 51.2561 541.933 13.9515 754.143 59.8069 1165.57 15.7017 653.949 68.3575 1958.58 17.452 554.487 76.9081 2889.29 19.2022 458.221 85.4587 3937.51 20.952...
work page 2041
-
[2]
Meuwly, WIREs Computational Molecular Science9, e1386 (2019)
M. Meuwly, WIREs Computational Molecular Science9, e1386 (2019)
work page 2019
-
[3]
S.ManzhosandT.J.Carrington,Chem.Rev.121,10187 (2021)
work page 2021
-
[5]
O. T. Unke, S. Chmiela, H. E. Sauceda, M. Gastegger, I. Poltavsky, K. T. Schütt, A. Tkatchenko, and K.-R. Müller, Chem. Rev.121, 10142 (2021)
work page 2021
-
[6]
D. Lu, H. Wang, M. Chen, L. Lin, R. Car, E. Weinan, W. Jia, and L. Zhang, Comput. Phys. Commun.259, 107624 (2021)
work page 2021
- [7]
-
[8]
O. T. Unke, M. Stöhr, S. Ganscha, T. Unterthiner, H. Maennel, S. Kashubin, D. Ahlin, M. Gastegger, L. Medrano Sandonas, J. T. Berryman,et al., Sci. Adv. 10, eadn4397 (2024)
work page 2024
-
[9]
V. Zaverkin, M. Ferraz, F. Alesiani, and M. Niepert, arXiv preprint arXiv:2508.10841 (2025)
-
[10]
A. Musaelian, S. Batzner, A. Johansson, L. Sun, C. J. Owen, M. Kornbluth, and B. Kozinsky, Nat. Comm.14, 579 (2023)
work page 2023
-
[11]
B. Kozinsky, A. Musaelian, A. Johansson, and S. Batzner, inProceedings of the International Con- ference for High Performance Computing, Networking, Storage and Analysis, SC ’23 (Association for Comput- ing Machinery, New York, NY, USA, 2023)
work page 2023
- [12]
-
[13]
T. C. Germann and K. Kadau, Int. J. Mod. Phys. C19, 1315 (2008)
work page 2008
-
[14]
Y. Shibuta, S. Sakane, E. Miyoshi, S. Okita, T. Takaki, and M. Ohno, Nat. Comm.8, 10 (2017)
work page 2017
-
[15]
L. Casalino, C. Seitz, J. Lederhofer, Y. Tsybovsky, I. A. Wilson, M. Kanekiyo, and R. E. Amaro, ACS Cent. Sci. 8, 1646 (2022)
work page 2022
- [16]
-
[17]
D. Ugarte La Torre, J. Jung, and Y. Sugita, inPro- ceedings of the International Conference on High Per- formance Computing in Asia-Pacific Region(2025) pp. 1–12
work page 2025
- [18]
-
[19]
S.-L. J. Lahey and C. N. Rowley, Chem. Sci.11, 2362 (2020)
work page 2020
-
[20]
M. Gastegger, K. T. Schütt, and K.-R. Müller, Chem. Sci.12, 11473 (2021)
work page 2021
-
[21]
T. J. Inizan, T. Plé, O. Adjoua, P. Ren, H. Gökcan, O. Isayev, L. Lagardère, and J.-P. Piquemal, Chem. Sci. 14, 5438 (2023)
work page 2023
-
[22]
K. Song, S. Käser, K. Töpfer, L. I. Vazquez-Salazar, and M. Meuwly, J. Chem. Phys.159(2023)
work page 2023
- [23]
-
[24]
K. Zinovjev, L. Hedges, R. Montagud Andreu, C. Woods, I. Tuñón, and M. W. van der Kamp, J. Chem. Theory Comput.20, 4514 (2024)
work page 2024
-
[25]
J. Kalayan, I. Ramzan, C. D. Williams, R. A. Bryce, and N. A. Burton, J. Comput. Chem.45, 1143 (2024)
work page 2024
-
[26]
I. Poltavsky, A. Charkin-Gorbulin, M. Puleva, G. Fon- seca, I. Batatia, N. J. Browning, S. Chmiela, M. Cui, J. T. Frank, S. Heinen, B. Huang, S. Käser, A. Kabylda, D. Khan, C. Müller, A. J. A. Price, K. Riedmiller, K. Töpfer, T. W. Ko, M. Meuwly, M. Rupp, G. Csányi, O. A. von Lilienfeld, J. T. Margraf, K.-R. Müller, and A. Tkatchenko, Chem. Sci.16, 3720 (2025)
work page 2025
-
[27]
I. Poltavsky, M. Puleva, A. Charkin-Gorbulin, G. Fon- seca, I. Batatia, N. J. Browning, S. Chmiela, M. Cui, J. T. Frank, S. Heinen, B. Huang, S. Käser, A. Kabylda, D. Khan, C. Müller, A. J. A. Price, K. Riedmiller, K. Töpfer, T. W. Ko, M. Meuwly, M. Rupp, G. Csányi, O. A. von Lilienfeld, J. T. Margraf, K.-R. Müller, and A. Tkatchenko, Chem. Sci.16, 3738 (2025)
work page 2025
-
[28]
J. Xia, Y. Zhang, and B. Jiang, Chem. Soc. Rev. 10.1039/D5CS00104H (2025), dOI: 10.1039/D5CS00104H
-
[29]
J. M. Bowman, C. Qu, R. Conte, A. Nandi, P. L. Hous- ton, and Q. Yu, J. Chem. Phys.162, 180901 (2025)
work page 2025
- [30]
-
[31]
W. Hwang, S. L. Austin, A. Blondel, E. D. Boittier, S. Boresch, M. Buck, J. Buckner, A. Caflisch, H.-T. Chang, X. Cheng, Y. K. Choi, J.-W. Chu, M. F. Crow- ley, Q. Cui, A. Damjanovic, Y. Deng, M. Devereux, X. Ding, M. F. Feig, J. Gao, D. R. Glowacki, J. E. I. Gonzales, M. B. Hamaneh, E. D. Harder, R. L. Hayes, J. Huang, Y. Huang, P. S. Hudson, W. Im, S. M...
work page 2024
-
[32]
M. E. Taylor and P. Stone, J. Mach. Learn. Res.10, 1633 (2009)
work page 2009
-
[33]
S. J. Pan and Q. Yang, IEEE Trans. Knowl. Data Eng. 22, 1345 (2009)
work page 2009
-
[34]
J. S. Smith, B. T. Nebgen, R. Zubatyuk, N. Lubbers, C. Devereux, K. Barros, S. Tretiak, O. Isayev, and A. E. Roitberg, Nat. Commun.10, 1 (2019)
work page 2019
- [35]
- [36]
-
[37]
H.-J. Werner, P. J. Knowles, G. Knizia, F. R. Manby, M. Schütz, P. Celani, W. Györffy, D. Kats, T. Korona, R. Lindh, A. Mitrushenkov, G. Rauhut, K. R. Shamasun- dar, T.B.Adler, R.D.Amos, S.J.Bennie, A.Bernhards- son, A. Berning, D. L. Cooper, M. J. O. Deegan, A. J. Dobbyn, F. Eckert, E. Goll, C. Hampel, A. Hesselmann, G. Hetzer, T. Hrenar, G. Jansen, C. K...
work page 2019
- [38]
-
[39]
V. Andreichev, S. Käser, E. L. Bocanegra, M. Salik, M. A. Johnson, and M. Meuwly, Phys. Chem. Chem. Phys. (2025)
work page 2025
-
[40]
B. J. Braams and J. M. Bowman, Int. Rev. Phys. Chem. 28, 577 (2009)
work page 2009
- [41]
-
[42]
Msa software with gradients,https://github.com/ szquchen/MSA-2.0(2019), accessed: 2019-01-20
work page 2019
-
[43]
P. L. Houston, C. Qu, Q. Yu, R. Conte, A. Nandi, J. K. Li, and J. M. Bowman, J. Chem. Phys.158, 044109 (2023)
work page 2023
-
[44]
R. Ramakrishnan, P. Dral, M. Rupp, and O. A. von Lilienfeld, J. Chem. Theory Comput.11, 2087 (2015)
work page 2087
- [45]
- [46]
-
[47]
J. M. Bowman, C. Qu, R. Conte, A. Nandi, P. L. Hous- ton, and Q. Yu, J. Chem. Theory and Comput.19, 1 (2023)
work page 2023
- [48]
-
[49]
P. L. Houston, C. Qu, Q. Yu, P. Pandey, R. Conte, A. Nandi, J. M. Bowman, and S. G. Kukolich, J. Chem. Theory Comput.20, 1821 (2024)
work page 2024
- [50]
-
[52]
A. Griewank and A. Walther,Evaluating Derivatives: Principles and Techniques of Algorithmic Differentia- tion, 2nd ed. (Society for Industrial and Applied Mathe- matics (SIAM), Philadelphia, PA, 2008)
work page 2008
-
[53]
Adam: A Method for Stochastic Optimization
D. Kingma and J. Ba, arXiv preprint arXiv:1412.6980 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[54]
S. J. Reddi, S. Kale, and S. Kumar, On the convergence of adam and beyond (2019), arXiv:1904.09237 [cs.LG]
work page Pith review arXiv 2019
-
[55]
O. T. Unke and M. Meuwly, J. Chem. Theory Comput. 15, 3678 (2019)
work page 2019
-
[56]
E. Boittier, K. Töpfer, M. Devereux, and M. Meuwly, Journal of Chemical Theory and Computation20, 8088 (2024)
work page 2024
- [57]
- [58]
- [59]
- [60]
- [61]
-
[62]
C. Qu, A. Nandi, P. L. Houston, and J. M. Bowman, J. Phys. Chem. Axx, xx (2026)
work page 2026
-
[63]
J. M. Bowman, S. Carter, and X. Huang, Int. Rev. Phys. Chem.22, 533 (2003)
work page 2003
-
[64]
Q. Yu and et al.,MULTIMODE, the n-mode Represen- tation of the Potential and Illustrations to IR Spectra of Glycine and Two Protonated Water Clusters., edited by J. M. Bowman (Vibrational Dynamics of Molecules, World Scientific, 2022)
work page 2022
-
[65]
C. Qu, T. C. Allison, P. L. Houston, R. Conte, A. Nandi, and J. M. Bowman, J. Chem. Phys.xxx, xxxx (2026), in revision
work page 2026
-
[66]
J. B. Anderson, J. Chem. Phys.63, 1499 (1975)
work page 1975
- [67]
-
[68]
J. B. Anderson, J. Chem. Phys.65, 4121 (1976)
work page 1976
-
[69]
Y. Wang, B. J. Braams, J. M. Bowman, S. Carter, and D. P. Tew, J. Chem. Phys.128, 224314 (2008)
work page 2008
-
[70]
J. O. Richardson and S. C. Althorpe, J. Chem. Phys. 134, 054109 (2011)
work page 2011
-
[71]
J. O. Richardson, Int. Rev. Phys. Chem.37, 171 (2018)
work page 2018
-
[72]
J. O. Richardson, Intern. Rev. Phys. Chem.37, 171 (2018)
work page 2018
-
[73]
J. E. Lawrence, J. Dušek, and J. O. Richardson, J. Chem. Phys.159, 014111 (2023)
work page 2023
- [74]
-
[75]
D. T. Colbert and W. H. Miller, J. Chem. Phys.96, 1982 (1992)
work page 1982
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.