REVIEW 3 major objections 5 minor 34 references
Robust extrapolation using physics-related activation functions in neural networks for nuclear masses
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A neural network that swaps generic computer-science activation functions for physics-related ones extrapolates nuclear masses to 396 keV RMS beyond the measured region, a threefold improvement over a conventional DenseNet, and reproduces…
desk verdict The core result is plausible, but the reported 396 keV extrapolation RMS is not a clean out-of-sample number because it combines the validation set used for model selection; a test-only number is needed before trusting the magnitude of the improvement. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is the PAF (physics-related activation function) network, a dense network whose hidden nodes apply a menu of physics-motivated functions: $1/N$, $1/Z$, $1/A$, $\log(x)$, $\sin(x)$, parity terms, $N/Z$, $Z/N$, $N^x$, $Z^x$, $A^{1/3}$, $A^{2/3}$, and $((N-Z)/A)x$, with inputs derived only from neutron number $N$ and proton number $Z$. L0 sparsity, implemented with hard-concrete stochastic gates, prunes weights and keeps the model far from over-parametrization. Two extra soft penalties applied during training carry the extrapolation: a bound penalty that keeps mass-excess predictions inside $[-0.1, 2]$ u in the unmeasured region, and a Garvey-Kelson penalty that discourages the six-mass combination $\Delta(N,Z)$ from deviating by more than 0.5 u. These components, rather than a larger network, are what let the model extrapolate.
What would settle it
Train a conventional ReLU DenseNet with exactly the same bound penalty, Garvey-Kelson penalty, and outer-ring validation protocol as the PAF network; if its extrapolation RMS falls from 1173 keV to near 396 keV, the extrapolation gain is caused by the physics penalties and not by the activation-function replacement, which would refute the paper's central attribution.
Extended reading notes
Core claim
The central claim, stated on the authors' own terms, is that a feed-forward network whose hidden nodes apply physics-related activation functions, rather than ReLU-like computer-science functions, can extrapolate nuclear masses beyond the measured landscape in a physically consistent way. Trained only on 1,736 of the 2,548 AME2020 masses and guided by a bound penalty plus a soft Garvey-Kelson penalty on randomly sampled extrapolation points, an ensemble of sixteen such networks predicts the outermost nuclei with 396 keV RMS error (N, Z > 0), compared with 1173 keV for a conventional DenseNet. The model is not given any existing mass-model prediction or magic-number input, yet its activation outputs show structure at magic numbers and its predicted separation-energy landscape terminates at two-nucleon drip lines close to the experimental ones. The authors therefore conclude that the extrapolation weakness of neural networks in this problem is dominated by the choice of nonlinear functions and by regularization of the unmeasured region, not by network architecture or data size.
Load-bearing premise
The load-bearing premise is that the Garvey-Kelson relation, an algebraic connection among six neighboring nuclear masses, stays approximately true for nuclei far outside the measured region, including near the drip lines; if it breaks down there, the training penalty pushes predictions toward wrong values and the reported extrapolation accuracy would not survive.
Editorial extensions
If this is right
- A neural network with no magic-number input produces the known shell structure: activation outputs display kinks at magic numbers, so the model offers an interpretable path to see where physics effects enter.
- The drip lines emerge even though most near-drip-line nuclei were excluded from training, implying the PAF network captures the physics needed to separate bound from unbound nuclei.
- Extrapolation at 396 keV on the outer ring of AME2020, without using any global mass model, suggests the method can supply mass predictions for nuclei beyond current measurements.
- The spread across the sixteen-model ensemble gives a per-nucleus uncertainty estimate, so the PAF approach can provide calibrated ranges for applications, not only central values.
- Omitting or altering the $N^x$ and $Z^x$ functions degrades extrapolation by 10 to 20 percent, indicating that these terms are what the nuclear mass system is most sensitive to.
Reading between the lines
- Beyond the paper, the same PAF recipe is a testable strategy for other nuclear observables such as charge radii, beta-decay half-lives, or fission barriers, where generic networks extrapolate poorly; if the gain transfers, activation-function engineering is a general route to physics extrapolation.
- We infer that an ablation separating the Garvey-Kelson penalty from the PAF activation functions would clarify whether the extrapolation gain comes mostly from the physics-guided regularization or from the functions themselves; the paper does not isolate these two contributions.
- Because the bound penalty's lower limit was chosen from the measured mass excess of $^{118}$Sn, the model's extrapolation is normalized to known extremes; predictions outside that range should be treated with caution until the bound is re-derived from a wider set of measured nuclei.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a neural-network model for nuclear masses that uses only N and Z as inputs, with physics-related activation functions (PAF) replacing conventional activations, L0 regularization for sparsity, and two training penalties: a bound penalty and a soft Garvey-Kelson constraint. Using AME2020 data divided into training, validation, and test sets for both interpolation and extrapolation, a 16-member ensemble is trained and evaluated. The central claim is that the PAF network extrapolates far better than conventional DenseNet and ConvNet architectures, with reported extrapolation RMS errors of 396 keV versus 1173 keV for DenseNet (N, Z > 0), and that it reproduces proton and neutron drip lines.
Significance. The physics-informed activation-function idea is a genuinely interesting and credible direction for extrapolation in nuclear mass tables. The authors are careful to avoid using global mass models or magic-number knowledge, and they provide a comparison with conventional networks and several mass models, along with useful interpretability checks. If the reported extrapolation performance survives an unbiased evaluation, the paper would be a valuable contribution to the machine-learning nuclear-physics literature. However, the headline extrapolation error combines the validation set used for model selection with the held-out test set, so the central quantitative claim is not yet convincingly established; a test-only evaluation is needed before the comparison can be trusted.
major comments (3)
- [Sec. IV and Table II] The headline extrapolation RMS in Table II is computed on the combined validation and test datasets, while Sec. IV states that model selection, early stopping, activation-function choices, and hyperparameters were tuned using the lowest RMS on the extrapolation validation set. The reported 396 keV therefore includes nuclei whose errors influenced the selected model, and it is not a fully out-of-sample estimate. Sec. III explicitly motivates separate test sets to avoid selecting a model biased on the validation datasets, but no test-only extrapolation RMS is reported anywhere. Please report test-only RMS for PAF, DenseNet, and ConvNet, and give validation and test RMS separately for both interpolation and extrapolation; if the test-only numbers differ materially from Table II, the main quantitative comparison must be revised.
- [Sec. V (AME2020 vs AME2012 evaluation)] The alternative evaluation using newly updated AME2020 data is incomplete. The text says 'We trained the same DNN and PAF networks,' but the numbers that follow are only for ConvNet (426 and 446 keV) and PAF (251 and 273 keV). Since the central comparison of the paper is PAF versus DenseNet, the absence of the DenseNet result makes the concluding sentence 'Clearly, the RMS errors on such data configuration do not fully reflect the poor extrapolation performance of DNNs' unsupported. Please include DenseNet (and ideally all compared models) in this evaluation or temper the conclusion.
- [Sec. II, Eqs. (3)-(5)] The Garvey-Kelson penalty is applied to random points in the extrapolation region with a fixed threshold G = 0.5 u, which implicitly assumes that the GK relation remains approximately valid far from stability, including near the drip lines. The paper gives no evidence for this assumption in the unmeasured region and no sensitivity analysis on G or on the bound range in Eq. (3). A concrete robustness check—for example, varying G and the bound limits, or measuring GK residuals on held-out outermost nuclei as a function of distance from the training region—would substantially strengthen the extrapolation claim. This is particularly relevant because the paper claims coverage up to the drip lines, where experimental constraints are sparse.
minor comments (5)
- [Table II] Please state the number of nuclei in the interpolation and extrapolation sets, separately for the validation and test splits and for the N, Z ≥ 8 and N, Z > 0 selections; the current table reports only total counts.
- [Fig. 4] The agreement of the predicted drip lines with experiment is presented qualitatively; a quantitative metric, such as the mean distance in N and Z between the predicted and experimental drip lines, would make the claim more precise.
- [Sec. V] The statement that 'DNNs with different activation functions, structure, and regularization also gave no significant improvements in the extrapolation regions' is not backed by a table or figure; please list the tested variants and their RMS values, or remove the claim.
- [Eq. (2)] The Iverson-bracket notation in the L0 norm is not defined; a one-sentence definition would help readers outside the machine-learning community.
- [Sec. IV and Fig. 3] The paper appropriately notes that the deep-ensemble uncertainties may be less well calibrated, but the caption of Fig. 3 should state explicitly that the outer-region trimming is based on an approximate ensemble-standard-deviation cutoff.
Circularity Check
Reported extrapolation RMS is a validation-selected metric, not a clean out-of-sample prediction.
-
fitted input called prediction
[Sec. IV (Training Details), Sec. V (Results), Table II caption; cf. Sec. III (Data Preparation)]
""During the tuning of the network details, such as choices of activation functions, we monitored the root mean square (RMS) errors on the validation datasets as well as the formation of drip lines." "Among all training epochs, we selected the DNN and PAF models that achieved the lowest RMS errors on the validation dataset for extrapolation." Table II caption: "It presents the evaluation results on the combined validation and test datasets.""
The paper's own Section III calls for separate test datasets "to avoid selecting a model biased on the validation datasets," but the only headline extrapolation RMS (396 keV vs 1173 keV) is reported on the combined validation and test sets. Since the PAF/DenseNet models, activation-function choices, and early-stopping epoch were selected by minimizing the extrapolation-validation RMS, the validation portion of Table II is the selection objective itself, not an independent prediction. No test-only extrapolation RMS is reported anywhere, so the claimed factor-of-three improvement cannot be separated from selection bias. The drip-line agreement was also monitored during tuning, making it partly a selected outcome as well.
full rationale
The clearest circularity is the evaluation-protocol one above: the main numerical claim is reported on validation+test combined after model selection on the extrapolation-validation set. This makes the headline extrapolation metric partly a fitted/selected quantity rather than a pure out-of-sample prediction. I did not find a self-citation chain that is load-bearing; Refs. [16], [17], and [22] are independent technical citations, and no uniqueness theorem from the authors is invoked. The Garvey-Kelson and bound penalties (Eqs. 3-5) are external physics priors applied to model outputs and random unmeasured points, not to the extrapolation labels, so they do not make the extrapolation RMS an identity by construction; whether GK remains valid near the drip lines is a correctness risk, not circularity. The magic-number emergence discussion is interpretive and not circular. Overall, the central claim has independent components, but the reported extrapolation improvement is contaminated by selection on the validation subset, so partial circularity is present.
Assumptions & free parameters
free parameters (4)
- L0 regularization strength lambda =
not reported
- Bound range (B1, B2) = (0.95, 1.05 u), i.e., mass excess in [-0.1, 2] u =
B1=0.95 u, B2=1.05 u
- Garvey-Kelson threshold G =
0.5 u
- Network architecture and activation-function set =
not fully specified
assumptions (4)
- domain assumption The Garvey-Kelson relation approximately holds in the extrapolation region, within +/- 0.5 u.
- domain assumption All nuclear mass excesses of interest lie within [-0.1, 2] u.
- domain assumption AME2020 experimental masses are accurate and representative.
- standard math The L0 hard-concrete stochastic gate method (Ref. [17]) behaves as specified and induces useful sparsity.
Cite this review
Pith. "Pith review of Robust extrapolation using physics-related activation functions in neural networks for nuclear masses." pith.science (2026). https://pith.science/paper/4U55RRMQ
@misc{pith2026250515363,
author = {Pith},
title = {Pith review of: Robust extrapolation using physics-related activation functions in neural networks for nuclear masses},
year = {2026},
howpublished = {\url{https://pith.science/paper/4U55RRMQ}},
note = {Machine review of arXiv:2505.15363}
}
read the original abstract
Given the importance of nuclear mass predictions, numerous models have been developed to extrapolate the measured data into unknown regions. While neural networks -- the core of modern artificial intelligence -- have been recently suggested as powerful methods, showcasing high predictive power in the measured region, their ability to extrapolate remains questionable. This limitation stems from their `black box' nature and large number of parameters entangled with nonlinear functions designed in the context of computer science. In this study, we demonstrate that replacing such nonlinear functions with physics-related functions significantly improves extrapolation performance and provides enhanced understanding of the model mechanism. Using only the information about neutron (N) and proton (Z) numbers without any existing global mass models or knowledge of magic numbers, we developed a highly accurate model that covers light nuclei (N, Z > 0) up to the drip lines. The extrapolation performance was rigorously evaluated using the outermost nuclei in the measurement landscape, and only the data in the inner region was used for training. We present details of the method and model, along with opportunities for future improvements.
Figures
Reference graph
Works this paper leans on
-
[1]
D. Lunney, J. M. Pearson, C. Thibault, Recent trends in the determination of nuclear masses, Rev. Mod. Phys. 75 (2003) 1021–1082. doi:10.1103/RevModPhys.75.1021
- [2]
-
[3]
J. Duflo, A. Zuker, Microscopic mass formulas, Phys. Rev. C 52 (1995) R23–R27. doi:10.1103/PhysRevC.52. R23
-
[4]
H. Koura, T. Tachibana, M. Uno, M. Yamada, Nuclidic mass formula on a spherical basis with an improved even- odd term, Progress of Theoretical Physics 113 (2) (2005) 305–325. doi:10.1143/PTP.113.305
-
[5]
N. Wang, M. Liu, X. Wu, J. Meng, Surface diffuse- ness correction in global mass formula, Physics Letters B 734 (2014) 215–219. doi:https://doi.org/10.1016/ j.physletb.2014.05.049
work page 2014
-
[6]
P. M¨ oller, A. J. Sierk, T. Ichikawa, H. Sagawa, Nuclear ground-state masses and deformations: FRDM(2012), Atomic Data and Nuclear Data Tables 109 (2016) 1–204. arXiv:1508.06294, doi:10.1016/j.adt.2015.10.002
arXiv 2012
-
[7]
R. Utama, J. Piekarewicz, H. B. Prosper, Nuclear mass predictions for the crustal composition of neutron stars: A bayesian neural network approach, Phys. Rev. C 93 (2016) 014311. doi:10.1103/PhysRevC.93.014311
-
[8]
Z. Niu, H. Liang, Nuclear mass predictions based on bayesian neural network approach with pairing and shell effects, Physics Letters B 778 (2018) 48–53. doi:https: //doi.org/10.1016/j.physletb.2018.01.002
Show all 34 references
-
[9]
A. E. Lovell, A. T. Mohan, T. M. Sprouse, M. R. Mumpower, Nuclear masses learned from a probabilis- tic neural network, Phys. Rev. C 106 (2022) 014305. doi:10.1103/PhysRevC.106.014305
2022 doi
-
[10]
M. R. Mumpower, T. M. Sprouse, A. E. Lovell, A. T. Mohan, Physically interpretable machine learning for nu- 6 clear masses, Phys. Rev. C 106 (2022) L021301. doi: 10.1103/PhysRevC.106.L021301
2022 doi
-
[11]
Boehnlein, M
A. Boehnlein, M. Diefenthaler, N. Sato, M. Schram, V. Ziegler, C. Fanelli, M. Hjorth-Jensen, T. Horn, M. P. Kuchera, D. Lee, W. Nazarewicz, P. Ostroumov, K. Orginos, A. Poon, X.-N. Wang, A. Scheinker, M. S. Smith, L.-G. Pang, Colloquium: Machine learning in nu- clear physics, ...
2022 doi
-
[12]
Barnard, L
E. Barnard, L. Wessels, Extrapolation and interpolati on in neural network classifiers, IEEE Control Systems Mag- azine 12 (5) (1992) 50–53. doi:10.1109/37.158898
1992 doi
-
[13]
Haley, D
P. Haley, D. Soloway, Extrapolation limitations of mul ti- layer feedforward neural networks, in: [Proceedings 1992] IJCNN International Joint Conference on Neural Net- works, Vol. 4, 1992, pp. 25–30 vol.4. doi:10.1109/ IJCNN.1992.227294
1992
-
[14]
Martius, C
G. Martius, C. H. Lampert, Extrapolation and learning equations (2016). arXiv:1610.02995
2016 arXiv
-
[15]
Y. Gal, L. Smith, Sufficient conditions for idealised mod - els to have no adversarial examples: a theoretical and empirical study with bayesian neural networks (2018). arXiv:1806.00667
2018 arXiv
-
[16]
C. H. Kim, K. Y. Chae, M. S. Smith, D. W. Bardayan, C. R. Brune, R. J. deBoer, D. Lu, D. Odell, Proba- bilistic neural networks for improved analyses with phe- nomenological r-matrix, Phys. Rev. C 110 (2024) 054609. doi:10.1103/PhysRevC.110.054609
2024 doi
-
[17]
Louizos, M
C. Louizos, M. Welling, D. P. Kingma, Learning sparse neural networks through l0 regularization (2018). arXiv: 1712.01312
2018 arXiv
-
[18]
M. M. Bejani, M. Ghatee, A systematic review on over- fitting control in shallow and deep neural networks, Ar- tificial Intelligence Review 54 (2021) 6391 – 6438
2021
-
[19]
Buhrmester, D
V. Buhrmester, D. M¨ unch, M. Arens, Analysis of ex- plainers of black box deep neural networks for com- puter vision: A survey, Machine Learning and Knowl- edge Extraction 3 (4) (2021) 966–989. doi:10.3390/ make3040048
2021
-
[20]
L. V. Jospin, H. Laga, F. Boussaid, W. Buntine, M. Ben- namoun, Hands-on bayesian neural networks—a tutorial for deep learning users, IEEE Computational Intelligence Magazine 17 (2) (2022) 29–48. doi:10.1109/MCI.2022. 3155327
2022 doi
-
[21]
Schmidt, H
M. Schmidt, H. Lipson, Distilling free-form natural la ws from experimental data, Science 324 (5923) (2009) 81–85. doi:10.1126/science.1165893
2009 doi
-
[22]
Specifically, the bound range of mass excesses was set to [−0.1, 2] u (atomic mass unit) by defining B1, B2=0.95, 1.05 u
was used to bound the magnitude of predictions on the extrapolation area in a reasonable range to avoid its drastic changes during the training: Lbound = ∑ i max(| fNN(xi; θ) − B1 | −B2, 0), (3) where the samples xi were randomly selected from the possible extrapolation region...
2024
-
[23]
Udrescu, M
S.-M. Udrescu, M. Tegmark, Ai feynman: A physics- inspired method for symbolic regression, Science Ad- vances 6 (16) (2020) eaay2631. doi:10.1126/sciadv. aay2631
2020 doi
-
[24]
Sahoo, C
S. Sahoo, C. Lampert, G. Martius, Learning equations fo r extrapolation and control, in: J. Dy, A. Krause (Eds.), Proceedings of the 35th International Conference on Ma- chine Learning, Vol. 80 of Proceedings of Machine Learn- ing Research, PMLR, Cambridge, MA, 2018, pp. 4442– 4450
2018
-
[25]
A. G´ eron, Hands-On Machine Learning with Scikit- Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 2nd Edition, O’Reilly Media, Inc., Sebastopol, CA, 2019
2019
-
[26]
Werner, A
M. Werner, A. Junginger, P. Hennig, G. Martius, In- formed equation learning (2021). arXiv:2105.06331
2021 arXiv
-
[27]
S. Kim, P. Y. Lu, S. Mukherjee, M. Gilbert, L. Jing, V. ˇCeperi´ c, M. Soljaˇ ci´ c, Integration of neural network- based symbolic regression in deep learning for scien- tific discovery, IEEE Transactions on Neural Networks and Learning Systems 32 (9) (2021) 4166–4177. doi: 10....
2021
-
[28]
K. P. Murphy, Probabilistic machine learning: an intro - duction, MIT press, Cambridge, MA, 2022
2022
-
[29]
M. Wang, W. Huang, F. Kondev, G. Audi, S. Naimi, The ame 2020 atomic mass evaluation (ii). tables, graphs and references*, Chinese Physics C 45 (3) (2021) 030003. doi:10.1088/1674-1137/abddaf
2021 doi
-
[30]
G. T. Garvey, I. Kelson, New nuclidic mass relation- ship, Phys. Rev. Lett. 16 (1966) 197–200. doi:10.1103/ PhysRevLett.16.197
1966
-
[31]
Lakshminarayanan, A
B. Lakshminarayanan, A. Pritzel, C. Blundell, Simple and scalable predictive uncertainty estimation using deep ensembles, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Sys- tems, Vol...
2017
-
[32]
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: Y. Bengio, Y. LeCun (Eds.), 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015
2015
-
[34]
Vaswani, N
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, I. Polosukhin, Atten- tion is all you need, in: I. Guyon, U. V. Luxburg, S. Ben- gio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Sys-...
2017
-
[210]
doi:https://doi.org/10.1016/0003-4916(74) 90299-1
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.