{"id":"c64c56cd-ac06-4a8a-9a45-30dbdc44f082","arxiv_id":"2501.01352","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A physics-informed neural network predicts nuclear binding energies to about 0.1 MeV and reproduces pairing and shell effects, with extrapolation tested against new AME2020 data.","lead":"This paper trains three neural networks to predict nuclear masses, combining liquid-drop and shell-model features with residual learning. The best variant reaches about 0.12 MeV accuracy on a test set and is used to study pairing and magic-number effects.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported RMSDs (0.122/0.191 MeV) are selected by monitoring the test set: early-stopping thresholds and architecture choices are tuned on the same nuclei later used to score the claim, so the headline numbers may be optimistically biased.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the early-stopping threshold is chosen by looking at test-set performance (Sec. III.A, Fig. 4), and the network architecture is also selected using test-set performance (Sec. II.D). My independent reading of the manuscript confirms this is the main threat to the central claim. The paper is otherwise honest about its procedure; it reports the test-set RMSD curves, the random-state choice, and the selected thresholds, which makes the issue a correctable evaluation-protocol problem rather than a hidden flaw. No independent machine-checked proof or released code exists to offset this concern. The WS4-informed residual learning is a plausible source of genuine accuracy, and the ablation tests are useful, but they do not quantify the bias introduced by test-set-based model selection. A proper three-way split with repeated random seeds and mean/standard-deviation reporting would settle the issue. If the re-evaluated numbers remain near 0.122/0.191 MeV, the claim stands; if they shift appreciably, the condition should be added to the verdict. Since the reader already assigned CONDITIONAL, my stress-test pass does not change that verdict.","tokens_in":18201,"tokens_out":2974,"duration_ms":31928,"concrete_test":"Hold out a true test set at the very beginning and never use it for threshold choice, architecture selection, or learning-rate adjustment. On the remaining data, perform, for example, ten random 80/20 splits; for each split, choose the early-stopping threshold and hidden-layer sizes using only a validation subset, then record mean ± std test RMSD for Method III in Case A and Case B. If the held-out RMSDs exceed the reported 0.122/0.191 MeV by more than about 0.03 MeV, or if seed-to-seed variance is large, the headline numbers are partly artifacts of test-set selection and require revision. If the re-estimated values reproduce the reported numbers within that tolerance, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central numerical claims (Method III test RMSD 0.122 MeV in Case A and 0.191 MeV in Case B) are supported by a protocol that consumes the test set during model selection. In Sec. II.D, the network structure is chosen by the rule: if training performance is similar, pick the structure that performs better on the test set. In Sec. III.A and Fig. 4, Methods II and III use early stopping where training is halted when the loss reaches a threshold 'as further reduction did not consistently improve the performance of the test set,' and the thresholds (1.28e-4, 5.97e-5, 1.68e-4, 3.80e-4) are selected from curves of test-set RMSD versus loss. The same test nuclei are therefore part of hyperparameter selection, so the quoted RMSDs are minima over stopping criteria rather than unbiased estimates of generalization. The single split (random_state = 42) and the absence of released code make the size of this selection bias unquantified. Because the paper's headline claim is precisely that the method reaches approximately 0.1 MeV, this is the load-bearing weakness: the real held-out accuracy could be materially worse, and the extrapolation claim is subject to the same test-set-based threshold selection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares three fully connected neural network (FCNN) methods for atomic mass prediction. Method I is a direct regression of experimental binding energies from macroscopic and shell-model input features. Method II uses a macroscopic-microscopic decomposition with a liquid-drop baseline and multi-output training on E_exp, E_LD, and E_exp - E_LD. Method III refines Method II by using WS4-derived Coulomb and deformation features. The authors report a test-set RMSD of 0.122 MeV on AME2020 for Method III (Case A) and an extrapolation RMSD of 0.191 MeV when training on AME2016 and testing on nuclei newly added in AME2020 (Case B). They also extend the training set with WS4 theoretical masses (Case C) and use predicted separation energies to claim reproduction of pairing and magic-number effects.","tokens_in":18461,"tokens_out":9077,"duration_ms":83804,"significance":"If the reported accuracy holds under an unbiased evaluation, the paper would provide a useful demonstration that a physics-informed FCNN with residual multi-output learning can represent the known mass surface at the ~0.1 MeV level and extrapolate moderately beyond the training region. The design choices are broadly sensible: macroscopic-microscopic features, residual learning, auxiliary outputs, and careful preprocessing in which scaling parameters are derived from the training set only are all strengths. The main limitation is that the reported test errors are selected using test-set information, so the headline numbers are optimistic and not yet reliable as generalization estimates. The magic-number 'reproduction' claim also needs to be reframed because the magic numbers are encoded directly in the input features. With a corrected validation protocol and clarified ablation, the work would be a solid contribution.","major_comments":[{"comment":"The architecture-selection rule in Sec. II.D ('if the training set performance is similar, we choose the one that performs better on the test set') and the early-stopping thresholds in Sec. III.A (e.g., 1.28e-4 for Method II and 5.97e-5 for Method III in Case A, and 1.68e-4 and 3.80e-4 in Case B) are chosen from curves of test-set RMSD versus loss shown in Fig. 4. Consequently the reported headline test RMSDs of 0.122 MeV (Case A) and 0.191 MeV (Case B) are minima over stopping criteria and architectures selected using the same nuclei on which they are then scored. They are therefore optimistically biased estimates of generalization. To support the accuracy claims, early stopping and architecture selection must be performed on a separate validation set, or via nested cross-validation, with the test set used exactly once. The current single split (random_state = 42) and the absence of released code make the size of this bias unquantifiable.","section":"Sec. II.D and III.A"},{"comment":"The ablation paragraph is internally inconsistent. It reports 'For Method I, the training and test RMSD are 0.756 MeV and 0.852 MeV' and corresponding numbers for Methods II and III, but Table III Case A gives baseline values of 0.135/0.204, 0.087/0.143, and 0.052/0.122 MeV for training/test for Methods I, II, and III, respectively. The paragraph does not state which feature is removed in each reported set of numbers, and the final sentence reports Method III training/test RMSD of 0.069/0.154 MeV after removing def, which cannot be reconciled with either the 0.052/0.122 MeV baseline or the preceding 0.172/0.196 MeV numbers. Since the ablation analysis is the evidence for the claim that the δnp and def inputs are indispensable, the paragraph must be rewritten to identify each ablation and to verify the numbers.","section":"Sec. III.A (ablation paragraph)"},{"comment":"The claimed reproduction of magic-number effects is substantially inherited from the input encoding. The valence features Vp and Vn in Eq. (5) are constructed using the prescribed magic-number list Z = 20, 28, 50, 82, 114, 120 and N = 20, 28, 50, 82, 126, 184, 198. Any regression model using these features has kinks at exactly those nucleon numbers, so the discontinuities in Sn, Sp, ΔSn, and ΔSp shown in Fig. 7 do not provide independent evidence for the magic numbers, including the newly proposed ones. The statement in Sec. IV that the three methods 'validated new magic number' is therefore overstated. To support such a claim, the authors would need to show that the network still develops the same kinks without Vp/Vn inputs, or that an alternative magic-number prescription fits notably worse; otherwise the claim should be softened to state that the model is consistent with the input magic numbers.","section":"Eq. (5) and Sec. III.C"}],"minor_comments":[{"comment":"The symbol Enn in Eq. (1) is not defined; it should be written as E_nn or E_net to denote the neural-network prediction.","section":"Sec. II.B, Eq. (1)"},{"comment":"The text says 'Theoretical binding energies, which are highly accurate, are excluded as output labels to avoid redundancy,' but Table I lists ELD as an output for both Methods II and III. This contradiction should be resolved, for example by clarifying that only the WS4 total binding energy is meant.","section":"Sec. II.C.2"},{"comment":"The text says 'In Case A, we trained and optimized two models,' but Case A compares three methods; this sentence should be corrected to avoid implying that Method I was not part of the comparison.","section":"Sec. III.C"},{"comment":"The FCNN used for the separation-energy predictions in Fig. 7 is described as having 60 neurons in the first hidden layer and 30 in the second, whereas Case A models use 80/40 and Case B models use 120/60; the text should specify which trained model generated Fig. 7 and why a different architecture was used.","section":"Sec. III.C"},{"comment":"There is a typo in 'Baysesian emulators'; it should be 'Bayesian emulators.'","section":"Sec. IV"}],"recommendation":"major_revision","confidential_remarks":"The central numerical claims of the manuscript are currently built on a protocol in which the test set is used to select architectures and early-stopping thresholds. I believe the work is potentially acceptable after the authors redo the evaluation with a proper validation split, clarify the ablation numbers, and soften the magic-number validation claim. The methodological core is plausible, so rejection is not warranted; major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely useful parts of this paper are the clean three-method comparison and the Case B extrapolation experiment (train on AME2016, test on nuclei newly added in AME2020). Multi-output residual learning on WS4-style features is a sensible idea, and the sensitivity analysis is a nice addition. The paper is well-written and cites the relevant ML-mass literature.\n\nThe soft spot is load-bearing. In Sec. II.D the network structure is chosen partly by test-set performance, and in Sec. III.A and Fig. 4 early stopping is halted when \"further reduction did not consistently improve the performance of the test set.\" Since the same nuclei are used for model selection and for reporting the 0.122 and 0.191 MeV RMSDs, those numbers are minima over stopping criteria rather than unbiased generalization estimates. A held-out validation set or repeated splits are needed before I would trust the accuracy claims. The ablation paragraph in Sec. III.A is also internally inconsistent with Table III (the stated RMSDs for Methods I, II, III differ from the table), and the magic-number reproduction is largely circular because the input features include valence numbers defined relative to exactly those prescribed magic numbers.\n\nNone of this means the paper is worthless. The method comparison is thoughtful, and the extrapolation test is a real attempt to address a genuine problem in the field. But the central quantitative claim is not supported by the current protocol. This is fixable with a proper validation set and, ideally, code release.\n\nWho gets value from this: people working on ML mass models, and experimental groups who want a quick benchmark of whether FCNN residual learning is worth trying for their isotope chains. It deserves a serious referee who will ask for the validation-set fix and the ablation cleanup, but it should not be taken at face value as a 0.1 MeV model.","headline":"The paper's headline RMSDs are likely optimistically selected because early stopping and architecture choices are tuned on the same test set that is later scored, so the 0.1 MeV claim is not yet established.","tokens_in":19039,"tokens_out":2209,"would_cite":false,"duration_ms":22294,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A physics-informed fully connected neural network reproduces known atomic masses to 0.122 MeV and predicts newly measured masses to 0.191 MeV.","keywords":["atomic mass prediction","fully connected neural network","macroscopic-microscopic model","residual learning","nuclear binding energy","magic numbers","mass extrapolation","AME2020"],"falsifier":"Retrain Method III on the same AME2016 data with a separate validation split used only for early stopping, then evaluate once on the untouched AME2020 additions. If the resulting extrapolation RMSD is materially larger than 0.191 MeV, or if the Case A test RMSD with validation-based stopping is materially larger than 0.122 MeV, the central accuracy claim does not hold.","tokens_in":17979,"feed_emoji":"⚛️","tokens_out":5395,"duration_ms":50379,"temperature":0.7,"pith_summary":"The paper tries to establish that a fully connected neural network, given macroscopic-microscopic features of the nucleus, can reproduce the known atomic mass surface to a test-set root-mean-square deviation of 0.122 MeV and can predict masses newly added between AME2016 and AME2020 to 0.191 MeV. The central design move is residual learning: instead of predicting the total binding energy directly, the network predicts the difference between the experimental binding energy and a liquid-drop-model baseline, while also outputting the baseline itself as an auxiliary label. If true, this would mean machine learning can act as a fast, accurate complement to mass models in regions where experimental data are sparse.","feed_headline":"A neural network reproduces nuclear masses to 0.122 MeV","feed_subtitle":"Physics-based features let the model also forecast newly measured masses to 0.191 MeV.","key_machinery":"The load-bearing mechanism is multi-output residual learning built on a macroscopic-microscopic baseline. The network takes liquid-drop and shell-model-inspired features, including valence nucleon numbers relative to magic numbers, and predicts three outputs: the experimental binding energy $E_{\\rm exp}$, the liquid-drop model binding energy $E_{\\rm LD}$, and their difference $E_{\\rm exp}-E_{\\rm LD}$. Training on the residual forces the network to learn the microscopic corrections that the baseline misses, which is what produces the sub-0.2 MeV accuracy.","core_discovery":"The paper's central claim is that Method III, a two-hidden-layer FCNN with WS4-derived input features and three output labels, achieves a test RMSD of 0.122 MeV on AME2020 and an extrapolation RMSD of 0.191 MeV when trained on AME2016 and tested on nuclei newly added in AME2020. It further claims that sensitivity and ablation analyses show the inputs act in a physically interpretable way, and that predicted neutron and proton separation energies reproduce pairing oscillations and cusps at magic numbers, including proposed superheavy magic numbers. The authors present this as evidence that a physics-informed neural network can validate existing mass data and extend modestly beyond the measured region.","pith_inferences":["The reported accuracy is selected by an early-stopping rule that tracks test-set performance; an independent validation-based retraining could yield larger errors than the headline numbers.","The extrapolation test covers only the small set of nuclei newly added between AME2016 and AME2020, so the 0.191 MeV figure should not be read as evidence for reliable predictions deep into unknown regions like the neutron drip line.","A natural transfer test would be to apply the same residual-learning setup to other nuclear observables, such as charge radii or beta-decay half-lives, using the best available theoretical baseline for each."],"forward_implications":["A model with test RMSD near 0.1 MeV can serve as a fast interpolator of the known mass surface, giving mass estimates for nuclei between measured points.","The extrapolation result suggests the same architecture can flag plausible masses for newly measured isotopes before an updated Atomic Mass Evaluation appears.","Pairing oscillations and magic-number kinks in predicted separation energies indicate the network can be used to search for shell closures in unexplored regions.","Because the method only needs a theoretical baseline and measured masses, it can be reapplied with any improved mass model as the baseline."],"supporting_citations":[{"why":"Supplies the WS4 macroscopic-microscopic mass model whose Coulomb term, deformation parameters, and binding-energy values Method III uses.","marker":"[24]"},{"why":"Provides the AME2020 experimental masses used as training and test targets in Cases A and C.","marker":"[64]"},{"why":"Provides the AME2016 training set for the extrapolation test in Case B.","marker":"[65]"},{"why":"Supplies the improved liquid-drop model formula and parameters that define the baseline binding energy $E_{\\rm LD}$ in Methods II and III.","marker":"[66]"},{"why":"Context for machine-learning mass predictions and the accuracy target set by r-process studies.","marker":"[20]"},{"why":"Earlier demonstration that neural networks can refine nuclear mass models, setting up the residual-learning approach extended here.","marker":"[35]"},{"why":"A recent artificial-neural-network mass prediction that the paper situates its three methods against.","marker":"[37]"}],"fun_headline_variants":["Neural network predicts atomic masses to 0.122 MeV","AI extrapolates atomic mass to new nuclei with 0.191 MeV error","Physics-informed neural net validates mass data to 0.122 MeV","Machine learning reproduces nuclear masses with 0.122 MeV accuracy","Neural network forecasts unseen nuclear masses within 0.191 MeV"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed accuracy assumes the test set was not used to tune the stopping criterion; because the early-stopping loss threshold was chosen by watching test-set error, the reported RMSD values may be optimistic.","fun_headline_variants_meta":{"raw":{"variants":["Neural network predicts atomic masses to 0.122 MeV","AI extrapolates atomic mass to new nuclei with 0.191 MeV error","Physics-informed neural net validates mass data to 0.122 MeV","Machine learning reproduces nuclear masses with 0.122 MeV accuracy","Neural network forecasts unseen nuclear masses within 0.191 MeV"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000299,"raw_usage":{"total_tokens":1678,"prompt_tokens":844,"completion_tokens":834,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":460,"completion_tokens_details":{"reasoning_tokens":741}},"tokens_in":460,"tokens_out":834,"duration_ms":7747,"temperature":1.0,"reasoning_tokens":741,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:29:07.808937+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain Method III on the same AME2016 data with a separate validation split used only for early stopping, then evaluate once on the untouched AME2020 additions. If the resulting extrapolation RMSD is materially larger than 0.191 MeV, or if the Case A test RMSD with validation-based stopping is materially larger than 0.122 MeV, the central accuracy claim does not hold.","supporting_citations":[{"cited_title":"On the rate of con- vergence of fully connected deep neural network regres- sion estimates,","cited_arxiv_id":null,"evidence_quote":"Provides the AME2016 training set for the extrapolation test in Case B."},{"cited_title":"Nuclear ground-state masses and deformations: FRDM(2012),","cited_arxiv_id":null,"evidence_quote":"Supplies the WS4 macroscopic-microscopic mass model whose Coulomb term, deformation parameters, and binding-energy values Method III uses."},{"cited_title":"Bayesian approach to model-based ex- trapolation of nuclear observables,","cited_arxiv_id":null,"evidence_quote":"Provides the AME2020 experimental masses used as training and test targets in Cases A and C."},{"cited_title":"Multilayer feedforward networks are universal approxi- mators,","cited_arxiv_id":null,"evidence_quote":"Supplies the improved liquid-drop model formula and parameters that define the baseline binding energy $E_{\\rm LD}$ in Methods II and III."},{"cited_title":"The AME2016 atomic mass evaluation (I). Evaluation of in- put data; and adjustment procedures,","cited_arxiv_id":null,"evidence_quote":"Context for machine-learning mass predictions and the accuracy target set by r-process studies."},{"cited_title":"Refining mass formulas for astrophysical applications: a Bayesian neu- ral network approach,","cited_arxiv_id":null,"evidence_quote":"Earlier demonstration that neural networks can refine nuclear mass models, setting up the residual-learning approach extended here."},{"cited_title":"Nuclear mass predictions based on convolutional neural network,","cited_arxiv_id":null,"evidence_quote":"A recent artificial-neural-network mass prediction that the paper situates its three methods against."}],"review_version":1}