{"id":"3d5c7939-ba87-4594-a811-1afcd5cfb7be","arxiv_id":"2603.16770","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Garnet, a graph neural network force field trained from quantum and experimental data without reusing existing force field parameters, matches established force fields on small-molecule, protein, and binding-free-energy benchmarks.","lead":"Researchers trained a graph neural network, Garnet, to assign all molecular dynamics force-field parameters for proteins and small molecules from quantum, condensed-phase, and NMR data. It matches established force fields on several benchmarks and shows that a double exponential potential trains more stably than Lennard-Jones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The NMR ensemble-reweighting gradient used for training is admitted to differ from the published equation, so the actual objective optimized in the protein-NMR component of the 'from scratch' claim is unverified.","rationale":"The reader identified the NMR reweighting code discrepancy as the weakest assumption; I agree it is the single most load-bearing concern. The paper's empirical claims are broadly supported by multiple benchmarks across small molecules, proteins, complexes, IDPs, and RBFE, with caveats that are mostly disclosed (IDP over-compaction, cmet exclusion, legacy reference trajectories). However, the 'from scratch' claim is specifically distinguished from prior work by the inclusion of protein NMR data and by not reusing legacy parameters. The legacy-trajectory dependence is disclosed and limited to early epochs, so it is a lesser issue. The NMR code discrepancy, by contrast, is an admitted mismatch between the method as described and the method as run; the actual optimized objective is not recoverable from the paper alone. This does not automatically overturn the empirical results, so CONDITIONAL remains the appropriate verdict rather than ACCEPT or REJECT. The proposed source-code inspection and finite-difference test is feasible because the repository is public and permissively licensed, and it would settle whether the implemented estimator is actually correct despite the wording, or whether the NMR training signal was applied differently than claimed.","tokens_in":31581,"tokens_out":7176,"duration_ms":71817,"concrete_test":"Inspect the GB3 NMR training code in the public Garnet repository, locate the function referenced by the 'mistake' comment, and extract the exact implemented gradient expression. Compare it against the correct reweighting estimator d⟨L⟩/dλ = −β(⟨L D dU/dλ⟩/⟨D⟩ − ⟨L D⟩⟨D dU/dλ⟩/⟨D⟩²) by running a finite-difference check on 3–5 parameters (e.g., one σ, one ε, one torsion k) using the same 200-snapshot subset. If the code matches the correct formula to numerical tolerance, the concern dissolves; if it differs, quantify the cosine and relative-norm error and determine whether the printed equation or the code is wrong and whether the discrepancy could materially alter the optimized parameters.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central 'from scratch' claim rests on training all force field parameters on three data modalities: QM, condensed-phase, and protein NMR. The NMR component uses ensemble reweighting to propagate gradients through a GB3 trajectory. The Methods explicitly state: 'Due to a mistake noticed after training, the actual code used differed from the above equation as described in the source code.' This is not a cosmetic remark: if the implemented gradient is biased, then the model was not optimized against the stated J-coupling and chemical-shift losses, and the paper does not specify what objective was actually optimized. The printed estimator is also not a complete specification, since the correct reweighting gradient requires a Z̃/Z normalization on both terms; the typeset equation does not transparently show the second normalization. This is an internal consistency and reproducibility problem, not a disagreement with field consensus. It matters because the NMR data are a distinguishing part of the pipeline: without a correct, described gradient, the claim that Garnet was trained 'on protein NMR data' is not currently established. The authors' disclosure is commendable, but the discrepancy itself is load-bearing. The benchmarks could still be valid, but the training story—and therefore the 'from scratch' claim—needs the code-level estimator to be reconciled with the published/correct estimator.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Garnet is a graph neural network that predicts all molecular mechanics parameters—bond, angle, torsion, partial charge, and non-bonded terms—from molecular topology, with no reuse of legacy force-field parameters. It is trained on DFT data (SPICE, GEMS, MACE-OFF), condensed-phase enthalpies of vaporisation/mixing, and GB3 NMR observables via ensemble reweighting, and it uses a double exponential non-bonded potential rather than Lennard-Jones. The authors benchmark Garnet on SPICE forces/energies, OpenFF Industry conformer minimisation, folded-protein and protein-complex simulations, IDP radii of gyration, water properties, and relative binding free energies with OpenFE. They report performance comparable to Amber14SB, Espaloma, and OpenFF, and in the RBFE setting similar to OpenFE and FEP+.","tokens_in":31960,"tokens_out":6859,"duration_ms":67743,"significance":"If the central claim holds, the work is significant: it demonstrates that a complete classical force field can be produced by an automated, data-driven pipeline, and it identifies the double exponential potential as a trainable alternative to Lennard-Jones. The paper's strengths include open code and data, a transparent functional-form comparison, and benchmarks spanning small molecules, proteins, complexes, IDPs, and binding free energies. However, the 'from scratch' claim is currently weakened by an explicit discrepancy between the published ensemble-reweighting equation and the code actually used for NMR training, and by post hoc exclusions in the SPICE and RBFE benchmarks. These issues are local and potentially fixable, but they need to be resolved before the headline claims can be fully accepted.","major_comments":[{"comment":"The text states: 'Due to a mistake noticed after training, the actual code used differed from the above equation as described in the source code.' Since the GB3 NMR data are one of the three training modalities that distinguish this work from prior GNN-based force fields, the paper must specify what objective was actually optimised. If the implemented gradient differs from the stated estimator, the claim that Garnet was trained on protein NMR data is not currently established from the manuscript alone. The printed equation also does not transparently show the normalisation on both terms of the reweighting gradient. Please provide the implemented estimator, a corrected display equation, and a numerical check (e.g., gradient comparison on a small system or a demonstration that the intended loss decreased) so that the training story is verifiable.","section":"Methods, Neural network training (ensemble reweighting equation)"},{"comment":"The main-text 'Overall' metrics are computed over eight targets, but Table 2 lists nine systems, with cmet relegated to Supplementary Table 3. The cmet series has a known net-charge-change failure, and Garnet's pairwise RMSE there is 8.26 kcal/mol versus 1.99 for OpenFE and 1.07 for FEP+. Excluding this failing target from the headline claim 'similar results to popular approaches for relative binding free energy predictions' is post hoc. Please report overall metrics including cmet, or pre-specify and justify the exclusion criteria. As written, the reader cannot distinguish a targeted benchmark from a favourable subset of the data.","section":"Results, Relative binding free energy benchmark; Supplementary Table 3; Figure 6"},{"comment":"The table reports only conformations that were 'successful with all four methods.' If one method fails on a substantial number of conformations, comparison on the common subset can be optimistic or biased in either direction. Please report the number of conformations excluded per method and per subset, and check whether the qualitative conclusions survive when each method is evaluated on the conformations it can handle individually. The current presentation does not allow the reader to assess the impact of this filtering.","section":"Table 1 (SPICE benchmark)"},{"comment":"GB3 was used during training and also serves as a folded-protein benchmark. The authors note that the good GB3 performance 'could be due to overfitting', which is appropriate, but this means GB3 is not independent evidence of transferability. The independent proteins (BPTI, HEWL, Ubq) should carry the folded-protein claim, and GB3 should be explicitly labelled as a train-set check in the figure or main text rather than presented as part of the validation set.","section":"Results, Protein benchmark (Figure 3B-C)"}],"minor_comments":[{"comment":"Typo in the proper-torsion regularisation loss: 'paramteri' should be 'parameter i'.","section":"Methods, Neural network training"},{"comment":"The manuscript uses openfe-v1.8.0 while the OpenFE Industry Benchmarking Project used openfe-v1.0.1. Please state whether this version difference changes the default protocol settings and justify that the comparison still isolates the force field rather than the software version.","section":"Methods, Relative binding free energy benchmark"},{"comment":"Figure 5C caption says 'models trained from scratch' for all variants, but the supplementary methods clarify that these runs used only reference trajectories. Please make this distinction in the main text to avoid ambiguity.","section":"Results, Different functional forms"}],"recommendation":"major_revision","confidential_remarks":"The paper is broadly sound and the benchmark suite is impressive, but the unresolved mismatch between the stated and implemented NMR training objective is the kind of issue that should be fixed before publication. The cmet exclusion is also a concern for the RBFE headline claim. I would not want to reject the work, as these are local issues, but they are load-bearing for the 'from scratch' and 'competitive RBFE' statements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper before anything else. First, the central claim is real: Garnet actually trains every MM parameter, including Lennard-Jones sigma/epsilon and the double-exponential global terms, from QM, condensed-phase, and protein NMR data. That is a step beyond Espaloma and friends, and the small-molecule, protein, and RBFE benchmarks are broad and mostly convincing. Second, the paper has a genuine soft spot in the NMR training component: the Methods admit the implemented ensemble-reweighting gradient differs from the printed equation, and the printed equation itself is not a complete spec. That matters because the NMR data are part of the 'from scratch' identity, but it is disclosed, the code is available, and the other two data modalities are not affected. The stress-test note is right to flag it, but I would not call it fatal; I would call it a reproducibility issue that needs a code-level reconciliation, not a reason to doubt the whole enterprise.\n\nWhat is actually new: the double-exponential functional form trained stably where Lennard-Jones failed, water parameters trained in-house, and a fully automated pipeline with a permissive license. The benchmarking is unusually thorough: DFT forces, minimization on the OpenFF Industry set, folded proteins, complexes, IDPs, and an eight-target RBFE comparison against OpenFE and FEP+. The results are honestly reported as comparable rather than superior, which is the right frame. The authors also openly flag the GB3 overfitting possibility, the cmet charge-change failure, the removed failed edges, and the early reliance on GAFF/Amber trajectories. That level of candor counts for a lot.\n\nSoft spots, in proportion: the NMR gradient discrepancy is the biggest one, but only because it is load-bearing for one of three training modalities. The GB3 leak (training and benchmarking on the same protein) is acknowledged and partially mitigated by the fact that GB3 performance is not better than Amber's. The RBFE story is slightly oversold in the abstract given that cmet is excluded from the headline metrics and the bace failed edges were removed; the supplementary table makes clear cmet is a real failure mode for net-charge changes, not a trivial edge case. The 'no legacy parameters' framing is technically true but softened by the reference trajectories early in training and the MBIS charges from SPICE, which encode DFT-derived choices. None of these sink the paper, but together they mean the abstract's 'without the use of existing parameters' is a bit strong.\n\nWho is this for? Anyone working on automated force field development, ML-parametrized classical potentials, or practical RBFE with open tools. It deserves a serious referee. The science is solid enough that the right outcome is peer review with requests for a corrected NMR gradient description, a clearer statement of which benchmarks include or exclude cmet, and ideally a reproducibility check of the training code. I would cite this in the next year and bring it to reading group.\n\nRecommendation: send to peer review. The NMR gradient issue should be resolved by pointing to the specific code lines and confirming the estimator is unbiased, but this is a strong, honest paper that advances the field.","headline":"A genuinely from-scratch GNN force field with real benchmarks, but the NMR training gradient discrepancy and a few post hoc exclusions keep it from being fully clean.","tokens_in":32407,"tokens_out":885,"would_cite":true,"duration_ms":12738,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Garnet learns a full biomolecular force field from scratch, with no legacy parameters","keywords":["force field","graph neural network","continuous atom typing","molecular dynamics","double exponential potential","ensemble reweighting","binding free energy","NMR training"],"falsifier":"Run the published training code and verify that the implemented ensemble-reweighting gradient matches the equation in the Methods, then retrain Garnet from scratch without the initial GAFF/Amber14SB conformations and compare its GB3 NMR benchmark metrics; if the gradient is biased or the initial conformations are essential, the central 'from scratch' claim would be undermined.","tokens_in":31463,"feed_emoji":"🧬","tokens_out":1926,"duration_ms":24161,"temperature":0.7,"pith_summary":"This paper claims that a graph neural network, Garnet, can assign every parameter needed for classical molecular dynamics simulations—bonds, angles, torsions, partial charges, and non-bonded interactions—for arbitrary proteins and small molecules, without reusing any existing force field parameters. The model is trained on quantum mechanical data, condensed-phase enthalpies, and protein NMR observables, using differentiable simulation and ensemble reweighting. The authors argue that the resulting force field performs comparably to established force fields on small-molecule conformational benchmarks, folded proteins, protein complexes, and disordered proteins, and that it gives relative binding free energy predictions similar to widely used protocols. If this holds, it would mean that manual, system-specific force field parameterization can be replaced by an automated, reproducible pipeline, and that new functional forms can be tested systematically rather than being locked in by historical parameter sets.","feed_headline":"Neural net builds a protein force field with no legacy parameters","feed_subtitle":"Garnet matches established force fields on small molecules, proteins, and binding free energies, and is trained end-to-end from data.","key_machinery":"The central object is the GNN-based continuous atom typing model, which predicts all force field parameters from bonding topology alone, using atom embeddings with a two-bond receptive field. The load-bearing mechanism is the double exponential potential, which replaces Lennard-Jones for non-bonded interactions and is trainable from scratch, and the ensemble reweighting gradient estimator that allows NMR data to backpropagate into the parameters through simulations. Training also splits DFT forces into intramolecular and intermolecular parts, weights the weak intermolecular signal more heavily, and co-trains water parameters without special treatment.","core_discovery":"The central discovery is that an end-to-end differentiable pipeline can train all molecular mechanics force field parameters from data, without seeding from existing parameters. Garnet uses a graph neural network whose continuous atom embeddings are converted into bond, angle, torsion, charge, and non-bonded parameters, and these parameters are optimized by running simulations during training and matching quantum mechanical forces and energies, experimental enthalpies, and NMR scalar couplings and chemical shifts. A key technical finding is that the conventional Lennard-Jones potential is unstable to train from scratch with this approach, whereas a double exponential potential—with two globa","pith_inferences":["The result implicitly challenges the assumption that decades of human tuning are needed for transferable force fields; a purely data-driven starting point may generalize to molecule classes that are underserved by existing parameter sets, such as intrinsically disordered proteins.","If the double exponential potential proves robust in other packages and for other molecule types, it may become a standard alternative to Lennard-Jones, and the paper's comparison of functional forms provides a template for benchmarking such changes fairly.","The identified weakness in handling net-charge-changing alchemical transformations suggests a concrete target: training force fields with charge-dependent or polarizable terms, or using free energy data directly as a training signal, which the authors note as a future direction.","The use of initial trajectories generated by existing force fields means the 'from scratch' claim is partially softened; a fully independent version, trained without any reference to GAFF or Amber14SB conformations, would be a stronger test of the approach."],"forward_implications":["If the claims hold, an open, reproducible pipeline can replace manual force field parameterization, enabling systematic exploration of new functional forms and rapid retraining as more data become available.","A single parameterization scheme covering proteins, small molecules, and water, with no molecule-specific tuning, would simplify and standardize molecular dynamics simulations across biological and chemical applications.","The finding that the double exponential potential is more amenable to automated training than Lennard-Jones suggests that future force field development should reconsider the standard non-bonded functional form.","Competitive relative binding free energy predictions with an automatically parameterized force field would lower the barrier to using rigorous alchemical calculations in drug discovery, especially for novel chemical matter.","The approach's reliance on experimental protein NMR data during training opens the door to directly optimizing force fields against biologically relevant observables, not just quantum mechanical energies."],"fun_headline_variants":["Neural net builds force field from scratch, no manual input","From scratch: AI force field for proteins and molecules","Garnet learns force field parameters entirely from data","Double exponential potential simplifies force field training"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The ensemble reweighting gradient estimator used to train on GB3 NMR data works correctly despite the paper's admission that the implemented code differed from the stated equation, and the initial training conformations came from existing force fields, so if the estimator is biased the model's optimization on NMR data is not as described.","fun_headline_variants_meta":{"raw":{"variants":["Neural net builds force field from scratch, no manual input","From scratch: AI force field for proteins and molecules","Garnet learns force field parameters entirely from data","Double exponential potential simplifies force field training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00024,"raw_usage":{"total_tokens":1303,"prompt_tokens":643,"completion_tokens":660,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":387,"completion_tokens_details":{"reasoning_tokens":598}},"tokens_in":387,"tokens_out":660,"duration_ms":7017,"temperature":1.0,"reasoning_tokens":598,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T17:58:33.042883+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the published training code and verify that the implemented ensemble-reweighting gradient matches the equation in the Methods, then retrain Garnet from scratch without the initial GAFF/Amber14SB conformations and compare its GB3 NMR benchmark metrics; if the gradient is biased or the initial conformations are essential, the central 'from scratch' claim would be undermined.","supporting_citations":[],"review_version":1}