{"id":"5fdee8cc-5a2e-4700-a346-d19ddaf85baf","arxiv_id":"2607.21369","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"OpenGEM26 is a 200,000-molecule DFT dataset with 4.4 million conformations, and GPTFF-mol trained on it reports 16 meV/molecule energy error and force accuracy above ANI-2x.","lead":"The authors built OpenGEM26, a dataset of 200,000 small organic molecules with 4.4 million DFT-computed structures including distorted geometries and optimization paths, and trained a graph neural network force field on it. If the resource is as described, it gives chemists a larger, sulfur- and chlorine-aware training set and a potential for faster molecular dynamics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 16 meV/molecule energy MAE is not verifiable because the train/test split is unspecified; if the split is at the conformation level, trajectory correlation will inflate the reported accuracy.","rationale":"The reader's weakest assumption—that the sampling from GDB-13 and the train/test split are undisclosed and could render the generalization claims inflated—is precisely the most load-bearing concern. The paper's central contribution is a dataset and a trained potential; the headline 16 meV/molecule error is the quantitative evidence for that contribution. Without knowing whether the split is molecule-disjoint or conformation-disjoint, the error could be artificially low because optimization trajectories are highly autocorrelated. This is not a speculative issue: if a model sees many conformations of the same molecule during training, test conformations of that molecule are near-duplicates, so the test error measures interpolation within a known molecule's potential surface rather than generalization to new molecules. The paper provides no error bars, no hyperparameters, and no split metadata, so the reader cannot distinguish interpolation from generalization. The ANI-2x comparison is likewise made on the training distribution and is therefore not a fair head-to-head. The butane and keto-enol tests are qualitative and limited, so they do not rescue the quantitative claim. I agree with the reader's conditional verdict: the dataset and model may be valuable, but the central numbers are unverified pending data and split documentation. No additional concern is needed; this one is sufficient to justify the conditional verdict.","tokens_in":11405,"tokens_out":5005,"duration_ms":54400,"concrete_test":"Upon OpenGEM26 release, recompute the Figure 3 energy and force MAEs using a strictly molecule-disjoint split: group all conformations by canonical SMILES, randomly assign entire molecules to train/validation/test, and remove any test conformations within an RMSD threshold (e.g., 0.1 Å) of a training conformation. Compare the recomputed energy MAE with the reported 16 meV/molecule and the force MAE with ANI-2x. If the molecule-disjoint energy MAE is substantially higher (e.g., >2×) or the ANI-2x ranking reverses, the reported numbers are inflated by conformation leakage. As a secondary check, inspect the dataset metadata to confirm energy and force labels are computed at the same DFT level.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim—GPTFF-mol achieves 16 meV/molecule (0.82 meV/atom) energy MAE and superior force predictions versus ANI-2x—rests entirely on an undescribed evaluation protocol. In the 'ML / Training results' section, the paper reports errors on 'the test set' but never states whether the split is molecule-disjoint or conformation-disjoint, nor how the 200,000 molecules were sampled from the 970M-molecule GDB-13 space. This matters because each molecule contributes an entire MMFF→DFT optimization trajectory: adjacent frames are highly correlated, with similar geometries and energies. If the split was performed at the conformation level, or if near-duplicate conformations of training molecules appear in the test set, the energy and force MAEs are optimistic by an unknown and potentially large factor. The ANI-2x comparison in Figure 3(c) is also evaluated on this test set drawn from the OpenGEM26 training distribution; ANI-2x was not trained on OpenGEM26, so its higher force error may simply reflect distribution shift rather than inferior model quality. The butane-rotation and keto-enol IRC validations are qualitative and use only a few structures, so they do not constrain the 16 meV claim. Additionally, the paper never clarifies whether the DFT energy labels and force labels come from the same level of theory (SVP vs. TZVP), which would affect the physical consistency of the trained potential. The headline generalization claim is therefore not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OpenGEM26, a proposed large DFT-level dataset of 200,000 GDB-13 molecules (4.4 million conformations) with H, C, N, O, S, and Cl, up to ten heavy atoms, recorded along full structural optimization trajectories with abundant non-equilibrium geometries. All calculations are claimed at ωB97X-D/Def2-SVP and Def2-TZVP with dispersion corrections. Using this dataset, the authors train GPTFF-mol, a graph-neural-network potential, and report an energy MAE of 16 meV/molecule (0.82 meV/atom), force predictions superior to ANI-2x, and qualitative validation on butane rotation and keto-enol tautomerization. The paper also presents statistical comparisons suggesting that OpenGEM26 covers a broader conformational and chemical space than QM9.","tokens_in":11841,"tokens_out":6221,"duration_ms":63759,"significance":"If substantiated, the contribution is significant: existing benchmarks such as QM9 are mostly restricted to equilibrium structures and omit S/Cl, whereas OpenGEM26 is claimed to provide non-equilibrium trajectories with forces for a broader element set. The model code, training scripts, and pretrained weights are made available, which is a strength. However, the central quantitative claims (16 meV/molecule energy MAE, force superiority over ANI-2x, and broader conformational coverage) are currently not verifiable from the manuscript: the dataset itself is not released, the sampling rule from GDB-13 is not described, and the train/test split is unspecified. These issues directly affect the credibility of the headline numbers and the transferability claims.","major_comments":[{"comment":"The reported 16 meV/molecule energy MAE and the force comparison cannot be verified because the train/test split is not described. The text never states whether the split is molecule-disjoint or conformation-disjoint, nor how the 200,000 molecules were sampled from GDB-13. Because each molecule contributes an MMFF→DFT optimization trajectory, adjacent frames are highly correlated; a conformation-level split would place near-training structures in the test set and inflate the accuracy. Please specify the split, the test-set size, and any similarity or duplicate filtering, and make the split file available. Without this, the headline 16 meV/molecule claim is not supported.","section":"ML / Training results, Figure 3(a)-(c)"},{"comment":"The selection rule for the 200,000 molecules from GDB-13 is absent. GDB-13 contains about 970 million molecules; 'molecules containing up to ten heavy atoms' is not an algorithm. If the subset is not random or stratified, the diversity comparisons with QM9 (Figures 1–2) and the transferability claims for S/Cl-containing organic molecules may be biased. State the exact sampling protocol (e.g., uniform random by formula, scaffold clustering, or active learning) and report the distribution of elements and molecular formulas in the selected subset.","section":"Dataset Methods"},{"comment":"The claim of 'superior force prediction performance compared with ANI-2x' is unquantified: no force MAE (in eV/Å) is given for either model in the text or Figure 3. In addition, ANI-2x is evaluated on a test set drawn from the OpenGEM26 distribution on which GPTFF-mol was trained; ANI-2x was not trained on OpenGEM26, so a larger force error may reflect distribution shift rather than inferior model quality. Please report numeric force MAEs, the evaluation code, and an out-of-distribution or cross-dataset benchmark where both models are compared fairly.","section":"ML / Training results, Figure 3(c)"},{"comment":"The level of theory for the reference labels is ambiguous. The abstract and methods state calculations are at 'ωB97X-D/Def2-SVP and Def2-TZVP' and 'accuracy is within ωB97X-D/Def2-SVP for first step and then use Def2-TZVP to get more accurate optimized structures.' The paper never states whether the stored energies and forces are SVP or TZVP values, or both. Mixing levels in the training labels would undermine physical consistency of the trained potential. Specify the exact workflow (e.g., SVP optimization followed by TZVP single points) and which quantities are stored for each geometry.","section":"Dataset Methods / Abstract"},{"comment":"The application validations (butane rotation and keto-enol tautomerization) are qualitative and potentially in-distribution: butane and 2-hydroxypyridine/2-pyridone are small H/C/N/O molecules, and no statement excludes these molecules or their conformers from the OpenGEM26 training set. No numerical errors are reported for the energy profiles or reaction barriers. Because these tests are used to claim accurate dynamical behavior and barrier prediction, please add explicit molecule-exclusion checks, report quantitative errors (e.g., barrier MAE in meV), or soften the conclusion to reflect the qualitative nature of the demonstration.","section":"Application, Figures 4 and 5"}],"minor_comments":[{"comment":"The force formula is garbled: '𝐹⃗=−!\"!#⃗' should read F = −∂E/∂r. Please typeset it correctly.","section":"Model / Eq. for force"},{"comment":"The caption and main text swap the axis descriptions: the caption says the vertical axis represents bond length and the horizontal axis represents the logarithm of the number, while the text says the opposite. Please standardize.","section":"Figure 1(b)-(f)"},{"comment":"Reference [42] (Huang, Dictionary of 140k GDB and ZINC derived AMONs) is incomplete: it lacks journal/volume/page/DOI information.","section":"References"},{"comment":"The GNN hyperparameters (cutoff radius, embedding width, number of layers, learning rate, number of training epochs) are not given. Please list them in the text or point to a configuration file in the repository.","section":"ML / Training results"},{"comment":"Typo: 'the accuracy is within ωB97X-D/Def2-SVP' should be 'the calculations are performed at the ωB97X-D/Def2-SVP level.'","section":"Dataset Methods"},{"comment":"Please define 'test dataset' in the caption and provide the numerical MAE values shown in the parity and force plots; currently the reader must infer them from the figure.","section":"Figure 3 caption"},{"comment":"Please state explicitly whether the butane rotation scans are rigid or relaxed. The current description and Figure 4 are ambiguous on this point.","section":"Application / Butane rotation"}],"recommendation":"major_revision","confidential_remarks":"The manuscript could be a valuable contribution if the evaluation protocol is clarified. The central risk is that the 16 meV/molecule energy MAE and the force-superiority claim rest on an undocumented train/test split; if the split is at the conformation level, the reported numbers could be substantially overstated. I strongly recommend requiring the exact split definition, the GDB-13 sampling rule, and a public data deposit before acceptance. The ANI-2x comparison should also include a fair out-of-distribution component. These are fixable within the scope of a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Brief take: this is a dataset paper, and the dataset is the useful part. 200k GDB-13 molecules covering H/C/N/O/S/Cl with up to ten heavy atoms, 4.4M conformations, done at omegaB97X-D/Def2-SVP and TZVP with full optimization trajectories — that is a concrete extension beyond QM9 and ANI-1, especially for sulfur and chlorine and for non-equilibrium geometries. The authors are also straight about the model: GPTFF-mol reuses the GPTFF architecture; the novelty is the data, not the GNN.\n\nThe paper does the diversity comparison against QM9 reasonably well — bond lengths, angles, t-SNE on ANI-1x activations. That is honest evidence the dataset covers more conformational space. The butane and keto-enol validations are qualitative but not misleading as sanity checks.\n\nWhat is not solid is the numerical claim. The 16 meV/molecule and the force comparison with ANI-2x are presented without the test-set construction, the split (molecule-disjoint vs conformation-disjoint), the sampling rule from GDB-13, or error bars. Since the dataset consists of optimization trajectories, adjacent frames are strongly correlated; if the split is conformation-level or not molecule-disjoint, the MAE is optimistic by an unknown factor. The paper does not say. The ANI-2x comparison is on a test set drawn from OpenGEM26's own distribution, so a higher force error could just be distribution shift. Also the DFT level used for the energy labels vs force labels (SVP vs TZVP) is not clarified. The data release is deferred until after peer review, which makes all of this impossible to check.\n\nI would not call this a broken paper — the resource is real, and the methodology is standard. But the current manuscript does not support the headline numbers. What it needs is a revised version that states the sampling, the exact split, gives force MAE numbers, and releases the dataset to referees. With that, this becomes a useful benchmark addition to the ML force-field literature.\n\nRecommended: send it to peer review, but insist the dataset be made available for referees and the evaluation protocol documented. A serious referee can work with this.","headline":"OpenGEM26 is a potentially valuable dataset resource, but the paper's central accuracy claims rest on an undescribed evaluation protocol and the data is not yet available.","tokens_in":12269,"tokens_out":1720,"would_cite":false,"duration_ms":18172,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OpenGEM26, a new DFT dataset of 200,000 molecules and 4.4 million conformations with full optimization trajectories, supports a graph-neural-network potential that reaches 16 meV/molecule energy error and outperforms an established public p","keywords":["machine learning potentials","graph neural networks","molecular force fields","DFT datasets","non-equilibrium conformations","optimization trajectories","organic molecules","sulfur and chlorine chemistry"],"falsifier":"Re-train the same model with a strict molecule-disjoint train/test split, so that no test molecule shares a heavy-atom skeleton with any training molecule, and recompute the energy mean absolute error; if the error rises materially above 16 meV/molecule, the generalization claims would need to be revised.","tokens_in":11368,"feed_emoji":"🧪","tokens_out":4960,"duration_ms":53751,"temperature":0.7,"pith_summary":"OpenGEM26 is introduced as a new reference dataset of 200,000 distinct organic molecules and 4.4 million conformations, computed at a dispersion-corrected hybrid DFT level, that records complete structural-optimization trajectories and therefore includes many non-equilibrium geometries. The paper argues that this coverage is what lets a graph-neural-network potential, GPTFF-mol, reach an energy error of 16 meV/molecule (0.82 meV/atom) and force predictions that outperform a widely used existing open potential on the same test set. The authors validate the model by reproducing butane torsional barriers and keto-enol tautomerization energy profiles, including water-assisted cases. The point of the work is to show that trajectory-rich, sulfur- and chlorine-inclusive data can close the accuracy gap for ML potentials on reactive and distorted molecular structures.","feed_headline":"Graph-network force field reaches 16 meV/molecule on 4.4M structures","feed_subtitle":"Training on optimization trajectories with many off-equilibrium geometries yields DFT-level energies and forces for sulfur- and chlorine-con","key_machinery":"The core mechanism is a graph neural network that represents a molecule as a graph (atoms as nodes, bonds as edges), constructs element-type embeddings and two-body and three-body interaction features within a cutoff radius, sums predicted atomic energies into total energy, and obtains forces by automatic differentiation of energy with respect to coordinates. The dataset's optimization trajectories are the enabling asset: they supply force labels and far-from-equilibrium configurations that standard equilibrium-only datasets lack.","core_discovery":"The paper claims that the combination of a new DFT dataset (OpenGEM26) and a graph neural network potential (GPTFF-mol) yields an ML force field for organic molecules with up to ten heavy atoms (H, C, N, O, S, Cl) whose test-set energy MAE is 16 meV/molecule (0.82 meV/atom) and whose force predictions on the same test set outperform a widely used public neural-network potential. The dataset records complete DFT structure-optimization trajectories computed at the ωB97X-D/Def2-SVP and Def2-TZVP levels, so it contains many non-equilibrium conformations; the paper argues this is why the trained model reproduces physically meaningful torsional and tautomerization energy profiles even near transit","pith_inferences":["Extension the authors do not pursue: the same graph architecture and trajectory-based data recipe could be scaled to a larger slice of enumerated chemical space, potentially yielding a universal small-molecule potential.","The recorded optimization trajectories could support auxiliary learning targets beyond energy and forces, such as predicting optimization step sizes or learning a dynamics prior; this is not explored in the paper.","The water-mediated tautomerization tests hint that the potential might be useful for studying explicit-solvent reaction paths, though extending it that far would require additional validation.","Because the paper does not disclose its molecule-sampling or train/test splitting strategy, a conservative editorial inference is that the reported generalization is best read as a proof of concept until a molecule-disjoint evaluation is shown."],"forward_implications":["If the claimed accuracy holds, the trained potential can be used for molecular structure optimization and high-throughput screening of small organic molecules at near-DFT accuracy with force-field-like speed.","The dataset should enable training or refinement of other ML potentials, since it includes sulfur and chlorine and provides energy and force labels over complete optimization paths.","The validation on butane rotation and keto-enol tautomerization indicates that the model can describe torsional strain and reaction barriers, including water-assisted proton transfer, not just equilibrium energies.","The broader conformational coverage compared with the common benchmark suggests that ML potentials trained on OpenGEM26 should transfer better to molecules at distorted or reactive geometries.","The high-precision dispersion-corrected reference calculations make the resource a candidate reference for studying dispersion-sensitive properties of sulfur- and chlorine-containing organics."],"fun_headline_variants":["ML force field from DFT paths: 0.82 meV/atom on 4.4M conformers","Graph-net force field hits 16 meV/mol on 4.4M structures","Training on off-equilibrium geometries yields 0.82 meV/atom force field","GPTFF-mol: graph net for S/Cl organics from DFT trajectories","From DFT trajectories to 0.82 meV/atom for S/Cl molecules"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The 16 meV/molecule accuracy assumes that the 200,000 molecules selected from the enumerated database are representative and that the test set does not contain near-duplicate conformations of training molecules; the paper does not describe its sampling or split.","fun_headline_variants_meta":{"raw":{"variants":["ML force field from DFT paths: 0.82 meV/atom on 4.4M conformers","Graph-net force field hits 16 meV/mol on 4.4M structures","Training on off-equilibrium geometries yields 0.82 meV/atom force field","GPTFF-mol: graph net for S/Cl organics from DFT trajectories","From DFT trajectories to 0.82 meV/atom for S/Cl molecules"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001494,"raw_usage":{"total_tokens":5866,"prompt_tokens":808,"completion_tokens":5058,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":4942}},"tokens_in":552,"tokens_out":5058,"duration_ms":39017,"temperature":1.0,"reasoning_tokens":4942,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T07:37:22.187329+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-train the same model with a strict molecule-disjoint train/test split, so that no test molecule shares a heavy-atom skeleton with any training molecule, and recompute the energy mean absolute error; if the error rises materially above 16 meV/molecule, the generalization claims would need to be revised.","supporting_citations":[],"review_version":1}