REVIEW 5 major objections 7 minor 4 references
Graph Neural Network Force Fields (GPTFF-mol) for Organic Molecules from Optimization Trajectories (OpenGEM26)
T0 review · 5 major / 7 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read OpenGEM26, a new DFT dataset of 200,000 molecules and 4.4 million conformations with full optimization trajectories, supports a graph-neural-network potential that reaches 16 meV/molecule energy error and outperforms an established public p
desk verdict OpenGEM26 is a potentially valuable dataset resource, but the paper's central accuracy claims rest on an undescribed evaluation protocol and the data is not yet available. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The core mechanism is a graph neural network that represents a molecule as a graph (atoms as nodes, bonds as edges), constructs element-type embeddings and two-body and three-body interaction features within a cutoff radius, sums predicted atomic energies into total energy, and obtains forces by automatic differentiation of energy with respect to coordinates. The dataset's optimization trajectories are the enabling asset: they supply force labels and far-from-equilibrium configurations that standard equilibrium-only datasets lack.
What would settle it
Re-train the same model with a strict molecule-disjoint train/test split, so that no test molecule shares a heavy-atom skeleton with any training molecule, and recompute the energy mean absolute error; if the error rises materially above 16 meV/molecule, the generalization claims would need to be revised.
Extended reading notes
Core claim
The paper claims that the combination of a new DFT dataset (OpenGEM26) and a graph neural network potential (GPTFF-mol) yields an ML force field for organic molecules with up to ten heavy atoms (H, C, N, O, S, Cl) whose test-set energy MAE is 16 meV/molecule (0.82 meV/atom) and whose force predictions on the same test set outperform a widely used public neural-network potential. The dataset records complete DFT structure-optimization trajectories computed at the ωB97X-D/Def2-SVP and Def2-TZVP levels, so it contains many non-equilibrium conformations; the paper argues this is why the trained model reproduces physically meaningful torsional and tautomerization energy profiles even near transit
Load-bearing premise
The 16 meV/molecule accuracy assumes that the 200,000 molecules selected from the enumerated database are representative and that the test set does not contain near-duplicate conformations of training molecules; the paper does not describe its sampling or split.
Editorial extensions
If this is right
- If the claimed accuracy holds, the trained potential can be used for molecular structure optimization and high-throughput screening of small organic molecules at near-DFT accuracy with force-field-like speed.
- The dataset should enable training or refinement of other ML potentials, since it includes sulfur and chlorine and provides energy and force labels over complete optimization paths.
- The validation on butane rotation and keto-enol tautomerization indicates that the model can describe torsional strain and reaction barriers, including water-assisted proton transfer, not just equilibrium energies.
- The broader conformational coverage compared with the common benchmark suggests that ML potentials trained on OpenGEM26 should transfer better to molecules at distorted or reactive geometries.
- The high-precision dispersion-corrected reference calculations make the resource a candidate reference for studying dispersion-sensitive properties of sulfur- and chlorine-containing organics.
Reading between the lines
- Extension the authors do not pursue: the same graph architecture and trajectory-based data recipe could be scaled to a larger slice of enumerated chemical space, potentially yielding a universal small-molecule potential.
- The recorded optimization trajectories could support auxiliary learning targets beyond energy and forces, such as predicting optimization step sizes or learning a dynamics prior; this is not explored in the paper.
- The water-mediated tautomerization tests hint that the potential might be useful for studying explicit-solvent reaction paths, though extending it that far would require additional validation.
- Because the paper does not disclose its molecule-sampling or train/test splitting strategy, a conservative editorial inference is that the reported generalization is best read as a proof of concept until a molecule-disjoint evaluation is shown.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces OpenGEM26, a proposed large DFT-level dataset of 200,000 GDB-13 molecules (4.4 million conformations) with H, C, N, O, S, and Cl, up to ten heavy atoms, recorded along full structural optimization trajectories with abundant non-equilibrium geometries. All calculations are claimed at ωB97X-D/Def2-SVP and Def2-TZVP with dispersion corrections. Using this dataset, the authors train GPTFF-mol, a graph-neural-network potential, and report an energy MAE of 16 meV/molecule (0.82 meV/atom), force predictions superior to ANI-2x, and qualitative validation on butane rotation and keto-enol tautomerization. The paper also presents statistical comparisons suggesting that OpenGEM26 covers a broader conformational and chemical space than QM9.
Significance. If substantiated, the contribution is significant: existing benchmarks such as QM9 are mostly restricted to equilibrium structures and omit S/Cl, whereas OpenGEM26 is claimed to provide non-equilibrium trajectories with forces for a broader element set. The model code, training scripts, and pretrained weights are made available, which is a strength. However, the central quantitative claims (16 meV/molecule energy MAE, force superiority over ANI-2x, and broader conformational coverage) are currently not verifiable from the manuscript: the dataset itself is not released, the sampling rule from GDB-13 is not described, and the train/test split is unspecified. These issues directly affect the credibility of the headline numbers and the transferability claims.
major comments (5)
- [ML / Training results, Figure 3(a)-(c)] The reported 16 meV/molecule energy MAE and the force comparison cannot be verified because the train/test split is not described. The text never states whether the split is molecule-disjoint or conformation-disjoint, nor how the 200,000 molecules were sampled from GDB-13. Because each molecule contributes an MMFF→DFT optimization trajectory, adjacent frames are highly correlated; a conformation-level split would place near-training structures in the test set and inflate the accuracy. Please specify the split, the test-set size, and any similarity or duplicate filtering, and make the split file available. Without this, the headline 16 meV/molecule claim is not supported.
- [Dataset Methods] The selection rule for the 200,000 molecules from GDB-13 is absent. GDB-13 contains about 970 million molecules; 'molecules containing up to ten heavy atoms' is not an algorithm. If the subset is not random or stratified, the diversity comparisons with QM9 (Figures 1–2) and the transferability claims for S/Cl-containing organic molecules may be biased. State the exact sampling protocol (e.g., uniform random by formula, scaffold clustering, or active learning) and report the distribution of elements and molecular formulas in the selected subset.
- [ML / Training results, Figure 3(c)] The claim of 'superior force prediction performance compared with ANI-2x' is unquantified: no force MAE (in eV/Å) is given for either model in the text or Figure 3. In addition, ANI-2x is evaluated on a test set drawn from the OpenGEM26 distribution on which GPTFF-mol was trained; ANI-2x was not trained on OpenGEM26, so a larger force error may reflect distribution shift rather than inferior model quality. Please report numeric force MAEs, the evaluation code, and an out-of-distribution or cross-dataset benchmark where both models are compared fairly.
- [Dataset Methods / Abstract] The level of theory for the reference labels is ambiguous. The abstract and methods state calculations are at 'ωB97X-D/Def2-SVP and Def2-TZVP' and 'accuracy is within ωB97X-D/Def2-SVP for first step and then use Def2-TZVP to get more accurate optimized structures.' The paper never states whether the stored energies and forces are SVP or TZVP values, or both. Mixing levels in the training labels would undermine physical consistency of the trained potential. Specify the exact workflow (e.g., SVP optimization followed by TZVP single points) and which quantities are stored for each geometry.
- [Application, Figures 4 and 5] The application validations (butane rotation and keto-enol tautomerization) are qualitative and potentially in-distribution: butane and 2-hydroxypyridine/2-pyridone are small H/C/N/O molecules, and no statement excludes these molecules or their conformers from the OpenGEM26 training set. No numerical errors are reported for the energy profiles or reaction barriers. Because these tests are used to claim accurate dynamical behavior and barrier prediction, please add explicit molecule-exclusion checks, report quantitative errors (e.g., barrier MAE in meV), or soften the conclusion to reflect the qualitative nature of the demonstration.
minor comments (7)
- [Model / Eq. for force] The force formula is garbled: '𝐹⃗=−!"!#⃗' should read F = −∂E/∂r. Please typeset it correctly.
- [Figure 1(b)-(f)] The caption and main text swap the axis descriptions: the caption says the vertical axis represents bond length and the horizontal axis represents the logarithm of the number, while the text says the opposite. Please standardize.
- [References] Reference [42] (Huang, Dictionary of 140k GDB and ZINC derived AMONs) is incomplete: it lacks journal/volume/page/DOI information.
- [ML / Training results] The GNN hyperparameters (cutoff radius, embedding width, number of layers, learning rate, number of training epochs) are not given. Please list them in the text or point to a configuration file in the repository.
- [Dataset Methods] Typo: 'the accuracy is within ωB97X-D/Def2-SVP' should be 'the calculations are performed at the ωB97X-D/Def2-SVP level.'
- [Figure 3 caption] Please define 'test dataset' in the caption and provide the numerical MAE values shown in the parity and force plots; currently the reader must infer them from the figure.
- [Application / Butane rotation] Please state explicitly whether the butane rotation scans are rigid or relaxed. The current description and Figure 4 are ambiguous on this point.
Circularity Check
No demonstrated circularity: the central error metrics are empirical fits, not derivations, but the missing split definition and potentially in-sample validation leave the headline accuracy unverifiable.
full rationale
The paper's central claims are empirical rather than derivational. GPTFF-mol is trained on OpenGEM26, and the reported 16 meV/molecule energy MAE and force MAE are evaluated on an unspecified 'test set'. Training a model and reporting a holdout error is not circular in itself: the prediction is not equivalent to the training input by construction unless the test set overlaps or is strongly correlated with the training set. No equation in the paper reduces a predicted quantity to a fitted parameter. The only self-citation is the GPTFF architecture [58], but that citation is not used as evidence for the numerical accuracy claim; the reported test error is the evidence, so the self-citation is not load-bearing. The main concerns are omissions rather than demonstrated circular steps: (i) the paper never describes how 200,000 molecules were sampled from GDB-13 nor whether the train/test split is molecule-disjoint or conformation-disjoint; since each molecule contributes a full MMFF-to-DFT optimization trajectory, adjacent frames are highly correlated and a conformation-level split would inflate the reported MAE. (ii) The butane-rotation and 2-hydroxypyridine/2-pyridone tautomerization validations use small H/C/N/O molecules fully within the stated OpenGEM26 composition, and the paper never states that these molecules or conformations were excluded from training, so those validation curves are potentially in-sample. These are correctness and reproducibility risks, but the paper does not exhibit a specific reduction of a prediction to its own inputs, so they do not constitute proven circularity. Accordingly the circularity score is low, reflecting only a minor non-load-bearing self-citation and unresolved validation-protocol caveats.
Assumptions & free parameters
free parameters (2)
- GDB-13 subset sampling rule =
200,000 molecules (selection rule not reported)
- GNN hyperparameters (cutoff, embedding width, number of layers) =
not reported
assumptions (5)
- domain assumption GDB-13 enumerates the relevant chemical space
- domain assumption ωB97X-D/Def2-SVP + Def2-TZVP DFT provides reliable reference energies and forces
- domain assumption MMFF94 gen3d yields adequate initial geometries
- domain assumption GPTFF GNN architecture transfers to molecules
- domain assumption Train/test split is representative and non-leaky
Cite this review
Pith. "Pith review of Graph Neural Network Force Fields (GPTFF-mol) for Organic Molecules from Optimization Trajectories (OpenGEM26)." pith.science (2026). https://pith.science/paper/Y4T3KDZT
@misc{pith2026260721369,
author = {Pith},
title = {Pith review of: Graph Neural Network Force Fields (GPTFF-mol) for Organic Molecules from Optimization Trajectories (OpenGEM26)},
year = {2026},
howpublished = {\url{https://pith.science/paper/Y4T3KDZT}},
note = {Machine review of arXiv:2607.21369}
}
read the original abstract
Density functional theory (DFT) serves as a reliable tool for atomistic molecular simulations, while machine learning potentials have become powerful complements to balance accuracy and efficiency. In this work, we release OpenGEM26 (Open Generated Ensemble of Molecules, 2026), a large-scale dataset comprising 200,000 unique molecules and 4.4 million conformations composed of H, C, N, O, S and Cl with up to ten heavy atoms. All calculations are carried out at the {\omega}B97X-D/Def2-SVP and Def2-TZVP levels with dispersion corrections, and complete structural optimization trajectories and abundant non-equilibrium structures are recorded. Statistical analyses confirm that this dataset covers a broader conformational space than QM9 in terms of energy, bond lengths and bond angles. A graph neural network-based potential GPTFF-mol is trained using the new dataset, achieving an energy mean absolute error of 16 meV/molecule, which is equivalent to 0.82meV/atom, and superior force prediction performance compared with ANI-2x. Validated by butane rotation and keto-enol tautomerization tests, the model accurately describes molecular dynamical behaviors and reaction barriers at distorted geometries. This work provides a high-quality resource and robust ML potential for efficient simulations of sulfur- and chlorine-containing organic molecules.
Reference graph
Works this paper leans on
-
[16]
Cheng Z, Liu J, Jiang T, Chen M, Dai F, Gao Z, Ke G, Zhao Z and Ou Q 2023 Automatic Screen-out of Ir(III) Complex Emitters by Combined Machine Learning and Computational Analysis Adv. Opt. Mater. 11 2301093 [17] Fu F, Wang X, Zhang L, Yang Y, Chen J, Xu B, Ouyang C, Xu S, Dai F-Z and E W 2023 Unraveling the Mechanism of Phase Transformation and Structural...
2023
-
[32]
Blum L C and Reymond J-L 2009 970 Million Druglike Small Molecules for Virtual Screening in the Chemical Universe Database GDB-13 J. Am. Chem. Soc. 131 8732–3 [33] Ruddigkeit L, Van Deursen R, Blum L C and Reymond J-L 2012 Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17 J. Chem. Inf. Model. 52 2864–75 [34] Rupp ...
2009
-
[49]
Mindless
Korth M and Grimme S 2009 “Mindless” DFT Benchmarking J. Chem. Theory Comput. 5 993–1003 [50] Chai J-D and Head-Gordon M 2008 Long-range corrected hybrid density functionals with damped atom–atom dispersion corrections Phys. Chem. Chem. Phys. 10 6615 [51] Weigend F and Ahlrichs R 2005 Balanced basis sets of split valence, triple zeta valence and quadruple...
2009
-
[63]
Devereux C, Smith J S, Huddleston K K, Barros K, Zubatyuk R, Isayev O and Roitberg A E 2020 Extending the Applicability of the ANI Deep Learning Molecular Potential to Sulfur and Halogens J. Chem. Theory Comput. 16 4192–202 [64] Thölke P and De Fabritiis G 2022 TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials [65] Pelaez...
2020
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.