{"id":"04f37305-7a68-4c63-982b-3fc37b451017","arxiv_id":"1908.08501","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A kernel ridge regression model trained on DFT-computed NMR parameters predicts 1H and 13C shifts and 1JCH couplings with errors comparable to DFT, fast enough for routine 3D structure screening.","lead":"IMPRESSION is a machine learning system that predicts NMR chemical shifts and one-bond carbon-hydrogen coupling constants from 3D molecular structures, trained on quantum chemical (DFT) calculations. It matches DFT accuracy for most environments while running in milliseconds, which could speed up molecular structure elucidation in chemistry.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SI training and test CSD lists share identical refcodes (AXADAF, IYASUW, LAFHEH, LILDEP), so the claimed independent test set appears contaminated; reported held-out errors need recomputation on a disjoint test set.","rationale":"The reader's verdict is CONDITIONAL, with training-set coverage as the weakest assumption. I agree coverage is a real boundary and is well documented by the outlier analysis, but I found a more direct threat to the quantified form of the central claim: the held-out test set may not be disjoint from training. The SI reference lists contain exact CSD code matches (AXADAF, IYASUW, LAFHEH, LILDEP). The adaptive sampling pipeline sampled the whole CSD superset, and no exclusion of the CSD-500 test structures is described, so the overlap appears real rather than a typo. Since KRR predicts by interpolating training environments, test-set leakage inflates apparent accuracy precisely for the entries that feed the headline error metrics. The effect could be small if only those four molecules overlap, but the protocol error invalidates the 'independent test set' framing until a recomputation is done. I also considered the test-set-tuned variance filters and the 10.91 Hz experimental offset in the 1JCH training targets; both are disclosed caveats that mostly affect the experimental comparison, whereas the training/test overlap affects the core ML-vs-DFT accuracy claim. I therefore keep the reader's CONDITIONAL verdict rather than accepting the reported numbers unconditionally, and the required condition is a disjointness check and recomputation.","tokens_in":25036,"tokens_out":11762,"duration_ms":124010,"concrete_test":"Build the training/test refcode lists from SI Section S8, compute their exact intersection, and for each intersection member confirm that the same molecular structure (same connectivity, and after DFT optimization) was used in both sets. Then remove every overlapping test molecule (and, as a robustness check, any test molecule whose graph matches a training molecule under a different refcode) and recompute the Fig. 2 and Section 2.2 MAE/RMSE values for δ1H, δ13C and 1JCH, both unfiltered and after the stated variance cutoffs. Compare to the published 0.23 ppm / 2.45 ppm / 0.87 Hz (unfiltered) and 0.23 ppm / 2.17 ppm / 0.81 Hz (filtered) figures; report the percentage change and the number of overlapping environments.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's quantified central claim rests on an 'independent' test set of 410 CSD-500 structures. However, the SI's training and test CSD reference lists (Section S8) both contain the exact refcodes AXADAF, IYASUW, LAFHEH and LILDEP. The adaptive-sampling procedure selected training structures from a 75,382-structure CSD superset (Section S1.4), and no statement says the CSD-500 test structures were excluded from that superset, so exact overlap is structurally possible and the lists indicate it occurred. Kernel ridge regression with FCHL/local kernels is a memory-based method: predictions for a training-like environment are strongly interpolated, so a molecule present in both sets can score artificially well. Even if the overlap is only about 1% of the test set, the claim that the reported MAE/RMSE values are held-out estimates is no longer supported as stated. The correct response is to compute the intersection programmatically, remove every overlapping molecule (including graph-isomorphic duplicates under different refcodes), and recompute all unfiltered and variance-filtered test metrics. If the values shift materially, the abstract's 'tens of milliseconds... near quantum chemical accuracy' claim must be restated with corrected numbers; if they do not shift, the paper's conclusions likely survive. This is a dataset-construction issue rather than a modeling assumption, and it takes priority over coverage and variance-filter caveats because it bears directly on the headline evaluation.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents IMPRESSION, a kernel ridge regression (KRR) model with FCHL representations for predicting 1H and 13C chemical shifts and 1JCH scalar couplings from 3D molecular structures. Training labels are DFT values computed at the omega-b97xd/6-311g(d,p) level for 882 adaptively sampled C,H,N,O,F-containing CSD structures; the test set is 410 structures from the CSD-500 set. The authors report test-set MAEs of 0.23 ppm (delta-1H), 2.45 ppm (delta-13C), and 0.87 Hz (1JCH), and a variance-based pre-prediction filter that further improves these values. They also compare against experimental datasets and demonstrate that IMPRESSION can discriminate strychnine diastereomers nearly as well as the underlying DFT method, in a fraction of the computational time.","tokens_in":25338,"tokens_out":3623,"duration_ms":38370,"significance":"If the held-out accuracy numbers survive a clean train/test split, this is a timely and useful contribution: it shows that a kernel-based ML model can reproduce DFT-quality NMR parameters for a nontrivial portion of organic chemical space at negligible computational cost, and the variance filter is a practical way to flag unreliable predictions. The paper includes genuinely useful elements: adaptive sampling to build the training set, a learning-curve analysis, explicit Gaussian input files, and a stereoisomer discrimination test that goes beyond simple error metrics. The experimental comparisons, while not fully independent because of the fitted offset discussed below, are a strength of the paper. The central claim is clear and, subject to the dataset-integrity issue, well supported by the reported numbers.","major_comments":[{"comment":"The claimed independent test set is contaminated. Section 2.1 states that the 410 CSD-500 structures are an independent test set, but the SI training and test CSD lists in Section S8 both contain the exact refcodes AXADAF, IYASUW, LAFHEH and LILDEP. In addition, Section S1.4 states that the initial 100 training structures were chosen at random from the CSD-500 test set. Although those 100 were discarded after the first active-learning round, they were used to train the models that selected subsequent structures, so the training procedure itself was informed by test-set molecules. Because KRR is a memory-based method, train/test overlap can artificially inflate the reported MAE and RMSE values. The authors must compute the set intersection programmatically, remove every overlapping molecule (including graph-isomorphic duplicates under different refcodes), and recompute all unfiltered and variance-filtered test metrics. If the numbers change materially, the abstract's 'near quantum chemical accuracy' claim must be restated with corrected figures.","section":"S1.4, S8, 2.1"},{"comment":"The variance-filter cutoffs (0.1 ppm, 5 ppm, 1 Hz) were selected using the test set itself: the text says 'We therefore identified variance cut-offs for IMPRESSION predictions that provide a good compromise between accuracy and excluded values for the test set.' This makes the filtered test-set MAE/RMSE values optimistically biased, because the filter thresholds were tuned to minimize error on the same data used for evaluation. The unfiltered test metrics are the primary evidence and should be reported as such; the filter cutoffs should be selected on a held-out portion of the training set or by nested cross-validation, and the filtered test metrics should be presented as exploratory rather than as the headline accuracy.","section":"2.2"},{"comment":"The 1JCH test and training labels are not pure DFT values. Section S1.3 states that a 10.91 Hz offset, derived from a comparison of DFT predictions to the 608 experimental couplings in Section 2.3, was applied to the DFT values in both the training and test datasets. Thus the reported 0.87 Hz MAE for 1JCH measures agreement with offset-corrected DFT, not with the raw quantum-chemical method. Moreover, the subsequent experimental comparison for 1JCH is not an independent test of the ML model, because the same experimental data were used to fit the offset. The authors should report the unfiltered raw-DFT test metrics as well, or clearly label the 1JCH results as 'DFT plus fitted offset' throughout the abstract and conclusions.","section":"S1.3, 2.3"}],"minor_comments":[{"comment":"Several citations in the SI are missing or rendered as question marks, e.g., 'Kernel Ridge Regression?' and 'Coulomb Matrix ?'. These placeholders should be replaced with proper references before resubmission.","section":"S1.1"},{"comment":"The terms MAE, RMSE, and MaxE are used throughout but only MAE is defined; please define all three the first time they appear.","section":"2.2"},{"comment":"The typo 'comprimise' appears in the sentence introducing the variance cutoffs; it should be 'compromise'.","section":"2.2"},{"comment":"The conclusion says 'it's underlying DFT method'; this should be 'its underlying DFT method'.","section":"3"},{"comment":"The SI tables and figures for diastereotopic proton assignment and outlier structures would benefit from a short sentence in each caption stating which level of theory the 'DFT' column refers to (omega-b97xd/6-311g(d,p) after the linear-scaling/offset correction).","section":"S4, S6"}],"recommendation":"major_revision","confidential_remarks":"The train/test overlap is the decisive issue. If the recomputed disjoint test-set metrics remain close to the reported values, the paper is likely acceptable after the variance-filter and 1JCH-offset caveats are addressed. If the metrics deteriorate substantially, the abstract's central claim will need to be weakened. I would ask the editor to request the authors to provide the exact intersection calculation and the recomputed metrics as part of the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, it is the first ML system I know of that predicts 1JCH couplings from 3D structures, and the product-of-kernels trick for the coupled C–H pair is sensible. Second, the central 'independent test set' is not actually disjoint from the training set: the SI lists for training and testing both contain AXADAF, IYASUW, LAFHEH, and LILDEP. And S1.4 says the initial 100 training structures were drawn from the CSD-500 test set, so the contamination is not just a typo. For a memory-based kernel method, identical molecules in training and test make the reported MAEs look better than they should. Four molecules may shift the headline numbers only slightly, but the claim as stated is no longer supported.\n\nWhat the paper does well: the learning curves, the comparison of three representations, the variance filter as a pre-prediction confidence measure, and the strychnine diastereomer test are all genuinely useful. The experimental validation on natural product structure revision is a good move, and the authors are honest about the large outliers (sp-hybridized atoms, pi systems) and about δ13C being weaker than δ1H and 1JCH.\n\nThe soft spots, in order:\n- The overlap above. This needs to be fixed by recomputing all test metrics after removing every overlapping molecule, including graph-isomorphic duplicates. If the numbers barely change, fine; if they change, the abstract needs correcting. This is a dataset-construction issue, not a modeling flaw.\n- The 10.91 Hz offset for 1JCH was fitted to the experimental dataset and then applied to the DFT training labels. The comparison to experiment therefore inherits that fitted offset, which makes the 'near DFT accuracy to experiment' claim for 1JCH less impressive than it looks. It is disclosed, but it is a fitting step on the validation data.\n- The variance filter cutoffs (0.1 ppm, 5 ppm, 1 Hz) were selected using the test set. That tunes the reported metrics to the test set. The effect is probably minor, but the honest fix is to choose cutoffs on training data alone.\n- No code or trained models are released. For an ML paper, that makes independent replication harder.\n\nThe central method — training on DFT labels, not experimental data — is the right call and is well executed. I would send this to peer review, but only with the requirement that the authors recompute on a genuinely disjoint test set and report both the overlapped and cleaned numbers. If the cleaned numbers hold up, this becomes a useful tool for structure elucidation.","headline":"IMPRESSION is a genuinely fast ML replacement for DFT NMR prediction, but the 'independent' test set overlaps the training set (four identical CSD refcodes), so the headline MAEs need recomputation.","tokens_in":25955,"tokens_out":2998,"would_cite":true,"duration_ms":29521,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A machine-learning model trained on quantum-chemical NMR data can predict 1H shifts, 13C shifts, and 1JCH couplings at DFT-level accuracy in milliseconds.","keywords":["NMR prediction","kernel ridge regression","chemical shifts","scalar couplings","density functional theory","3D molecular structure","adaptive sampling","variance filtering"],"falsifier":"Take a molecule containing sp-hybridized atoms or a strongly anisotropic pi environment, within the same element set, and check whether its IMPRESSION predictions pass the variance cutoffs while differing from high-level DFT by more than 10 ppm for 13C or more than 5 Hz for 1JCH; one clean low-variance, high-error case would refute the claim that the variance filter identifies all unreliable predictions.","tokens_in":24806,"feed_emoji":"⚗️","tokens_out":10437,"duration_ms":105050,"temperature":0.7,"pith_summary":"This paper establishes that a machine-learning system, IMPRESSION, trained on density-functional-theory (DFT) values rather than on experimental measurements, can predict three solution-state NMR parameters—proton and carbon chemical shifts ($\\delta^{1}$H, $\\delta^{13}$C) and one-bond carbon–proton couplings ($^{1}J_{\\mathrm{CH}}$)—from a three-dimensional molecular structure with accuracy comparable to the DFT calculation it was trained on, but in tens of milliseconds instead of hours or days. On an independent test set the model reaches mean absolute errors of 0.23 ppm, 2.45 ppm, and 0.87 Hz for the three parameters, and a variance-based filter flags environments whose predictions should not be trusted. Because the machine learns from computed data, it does not depend on large, error-free experimental databases and can be used for three-dimensional problems like conformation and stereoisomer selection. The practical claim is that routine NMR-based structure elucidation can obtain near-DFT accuracy without paying DFT's computational cost.","feed_headline":"NMR predictions match quantum chemistry in milliseconds","feed_subtitle":"Trained on computed shifts and couplings, it screens candidate structures and conformers in seconds instead of days.","key_machinery":"The machinery is kernel ridge regression over atomic chemical environments, using the FCHL representation, which describes a nucleus by the two- and three-body geometry of the atoms around it. The target value for an environment is a regularized weighted sum of kernel similarities to all training environments; for $^{1}J_{\\mathrm{CH}}$, the similarity is the product of the carbon-environment and hydrogen-environment kernels. Training environments were chosen by adaptive sampling: repeatedly predict the full candidate pool with several models, add the structures with the highest cross-validation variance, and recompute the DFT references for them. The same variance calculation, evaluated before a prediction is made, serves as the confidence filter that identifies environments outside the training coverage. All reference values come from DFT at the $\\omega$B97X-D/6-311g(d,p) level with gauge-invariant atomic orbitals and uncontracted basis functions, including all four spin–spin coupling contributions.","core_discovery":"The central claim is that a kernel machine trained on DFT-computed NMR parameters is a faithful surrogate for the DFT method itself: given the same three-dimensional structure, IMPRESSION reproduces the reference calculations for $\\delta^{1}$H, $\\delta^{13}$C, and $^{1}J_{\\mathrm{CH}}$ with test-set mean absolute errors of 0.23 ppm, 2.45 ppm, and 0.87 Hz, and root-mean-square errors of 0.35 ppm, 3.88 ppm, and 1.39 Hz. After applying a per-parameter variance filter, the model's agreement with experimental measurements on independent validation sets is essentially the same as the underlying DFT method's for proton shifts and one-bond couplings, and close for carbon shifts. The paper also shows that IMPRESSION reproduces DFT's ability to distinguish the correct diastereomer of a polycyclic alkaloid among 14 candidates from its $^{1}J_{\\mathrm{CH}}$ values, and to assign diastereotopic protons, which demonstrates that the three-dimensional sensitivity needed for stereochemical problems survives the machine-learning approximation.","pith_inferences":["Although the paper tests only carbon, hydrogen, nitrogen, oxygen, and fluorine chemistry, nothing in the kernel-ridge-regression mechanism is element-specific; training the same pipeline on DFT data for heteronuclei such as 15N, 19F, or 31P should work, provided a similarly diverse training superset can be assembled.","The variance-based confidence estimate could be used as a prior in Bayesian structure assignment: instead of simply capping errors, a likelihood could be built from the per-environment variance, a natural extension of the paper's geometric-mean error scoring.","The paper's own largest errors—up to 63 ppm for carbon shifts and 24.6 Hz for couplings—cluster around sp-hybridized atoms and atoms near pi systems, so a small deliberately diverse benchmark of such molecules would test whether the variance filter flags coverage failures reliably in practice."],"forward_implications":["Structure elucidation workflows that currently wait hours to days per candidate molecule can screen many conformers, diastereomers, or candidate structures in seconds, because the NMR prediction itself is no longer the bottleneck.","The variance filter gives each predicted value a usable confidence tag, so automated pipelines can send only uncertain predictions back to full DFT rather than recomputing everything.","Because the machine learns from the underlying DFT, any improvement in that DFT reference—better functionals, basis sets, or solvent models—should transfer directly to the machine's accuracy.","The learning curves indicate that increasing the training set roughly tenfold would approximately halve the current error, so the reported accuracy is not a ceiling.","Molecular-mechanics geometries, if paired with retrained models, could remove the expensive DFT geometry step as well, at the cost of a 30–50% increase in errors for some parameters."],"supporting_citations":[{"why":"supplies the independent 410-structure test set on which all reported test-set errors are measured.","marker":"[22]"},{"why":"the quantum-chemistry package used for all geometry optimizations and NMR reference calculations.","marker":"[24]"},{"why":"defines the DFT protocol (functionals, basis sets, and NMR keywords) whose accuracy IMPRESSION reproduces.","marker":"[26-30]"},{"why":"provides the linear-scaling method and reference compounds used to convert computed shieldings to chemical shifts for training and testing.","marker":"[10]"},{"why":"the adaptive sampling procedure that selects which structures enter the training set.","marker":"[33-35]"},{"why":"the large crystallographic database from which training molecules are drawn after element filtering.","marker":"[36]"},{"why":"the FCHL kernel representation of atomic environments used in the final model.","marker":"[42]"},{"why":"the experimental one-bond coupling dataset used to compare both DFT and IMPRESSION to measured values.","marker":"[23]"},{"why":"the experimental proton and carbon shift dataset used for the second experimental validation.","marker":"[44]"},{"why":"the earlier DFT demonstration that one-bond couplings can discriminate diastereomers, the benchmark IMPRESSION repeats.","marker":"[5]"}],"fun_headline_variants":["NMR parameters: ML matches DFT in milliseconds","DFT accuracy for NMR, delivered in milliseconds by ML","From days to milliseconds: ML surrogate for DFT NMR","3D NMR prediction: ML hits DFT grade in milliseconds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy claim rests on the 882 training structures—drawn by adaptive sampling from a large database of crystal structures of molecules containing only carbon, hydrogen, nitrogen, oxygen, and fluorine—being representative enough of the chemical environments users will encounter that poorly covered cases are rare and that cross-validation variance flags them; the paper's outlier analysis shows sp-hybridized atoms and atoms near pi systems are where this boundary shows.","fun_headline_variants_meta":{"raw":{"variants":["NMR parameters: ML matches DFT in milliseconds","DFT accuracy for NMR, delivered in milliseconds by ML","From days to milliseconds: ML surrogate for DFT NMR","3D NMR prediction: ML hits DFT grade in milliseconds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000521,"raw_usage":{"total_tokens":2495,"prompt_tokens":889,"completion_tokens":1606,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":1542}},"tokens_in":505,"tokens_out":1606,"duration_ms":15985,"temperature":1.0,"reasoning_tokens":1542,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:37:47.954641+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a molecule containing sp-hybridized atoms or a strongly anisotropic pi environment, within the same element set, and check whether its IMPRESSION predictions pass the variance cutoffs while differing from high-level DFT by more than 10 ppm for 13C or more than 5 Hz for 1JCH; one clean low-variance, high-error case would refute the claim that the variance filter identifies all unreliable predictions.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"the quantum-chemistry package used for all geometry optimizations and NMR reference calculations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"the earlier DFT demonstration that one-bond couplings can discriminate diastereomers, the benchmark IMPRESSION repeats."}],"review_version":1}