Pith. sign in

REVIEW 2 major objections 5 minor 55 references

Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A neural network can be built to satisfy conservation laws to machine precision without losing accuracy.

desk verdict A solid, honest paper on hard-constrained networks for climate emulation; the ACnet construction works, the nonlinear conversion-layer extension is genuinely new, and the main soft spot is an unverified but fixable assumption that the training data itself conserves energy, mass, and radiation exactly. read the letter →

arxiv 1909.00912 v5 pith:RY3GIRVD submitted 2019-09-03 physics.comp-ph physics.ao-ph

Tom Beucler , Michael Pritchard , Stephan Rasp , Jordan Ott , Pierre Baldi , Pierre Gentine This is my paper · ORCID
classification physics.comp-phphysics.ao-ph
keywords physics-informedneuralnetworksconservationlawsconstrainedregressionconvectiveparameterizationclimatemodelinganalyticconstraintsnetworkarchitecture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that neural-network emulators of physical systems can be forced to satisfy analytic constraints, such as conservation laws, without giving up accuracy. It does this by rewriting the constraints as a linear system $C[x;y]=0$ and adding fixed “constraints layers” that compute some outputs as residuals, so the constraints hold to machine precision. On a climate-modeling task where a network predicts convective heating and moistening, the architecture-constrained network keeps the unconstrained network’s mean-squared error to within 3 percent while cutting conservation violations from ordinary regression levels to about $10^{-9}$ in the dimensionless penalty. The paper also finds that enforcing constraints, whether in the loss or in the architecture, reduces error in the variables that appear in those constraints.

What carries the argument

The machinery is a constraint matrix $C$ together with fixed “constraints layers.” The paper starts with a possibly nonlinear constraint $c(x_0,y_0,z)=0$, rewrites it so that the constraint becomes linear in new variables, $C[x;y]=0$, and lets the network output only $p-n$ direct components; the constraints layers solve the linear system from bottom to top to fill in the remaining $n$ components exactly, making the network conservative by construction. For nonlinear constraints, such as those involving relative humidity, “conversion layers” transform variables before and after the network so the intermediate mapping is linearly constrained. Passing the loss gradients through these fixed layers during training is what distinguishes ACnet from simply post-processing residual outputs after training.

What would settle it

Take the four conservation equations and evaluate them on the model outputs used as the training targets; if those residuals are not effectively zero, say larger than $10^{-9}$ in the paper’s dimensionless penalty, then an ACnet trained on that data is being forced to conserve quantities its own training data does not conserve, and its constrained outputs can be compared against the raw targets to detect the resulting systematic bias.

Watch

Extended reading notes

Core claim

The central discovery is a constructive reduction: any set of analytic constraints on a regression can be rewritten as a linear constraint $C[x;y]=0$ by choosing auxiliary input and output variables, and then enforced exactly by appending fixed layers that solve the linear system for the $n$ “residual” outputs. Because gradients flow through these layers during training, the network learns under the constraint rather than having it imposed afterward. In the paper’s test case, a neural-network parameterization of convection that predicts heating, moistening, and radiative tendencies from a column’s thermodynamic state, this architecture (ACnet) satisfies four column-integrated conservation laws to within machine precision while its mean-squared error is within 3 percent of the best unconstrained network. A soft loss penalty (LCnet) improves conservation but never reaches machine precision, and the paper shows a systematic relation: the more a network violates a constraint, the larger its error on the outputs that appear in it, so enforcing constraints specifically improves those outputs.

Load-bearing premise

The derivation assumes the training “truth” outputs satisfy the conservation constraints exactly, so that network error is the only source of constraint violation; if the data itself violates conservation because of discretization or time averaging, exact enforcement would push the emulator away from its own labels.

Editorial extensions

If this is right

  • Climate emulators built this way cannot accumulate spurious energy or water over long integrations, removing a known drift source in machine-learning parameterizations.
  • Soft penalties are not enough: loss-constrained networks reduce conservation violations but leave nonzero residuals, so exact architectural constraints are the route to true machine-precision conservation.
  • The error reduction from constraints applies to variables appearing in the constraints, such as the total thermodynamic tendency, but not to intrinsically stochastic outputs like precipitation.
  • The same improvement holds on held-out test data, so the benefit is not merely overfitting to the training set.
  • Nonlinear constraints can be handled by conversion layers with a modest added optimization cost, opening the method to equations of state and kinetic-energy constraints.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The linearization trick is general: any constraint that can be expressed through auxiliary variables, for instance kinetic energy built from velocity components, could be handled the same way even if the original variables are nonlinearly related; the authors only sketch this direction.
  • If the training labels come from a discretized model that does not exactly conserve the target quantities, then hard architectural conservation would force the emulator to disagree with its own training data; computing the constraint residual on the raw training outputs would reveal whether this bias exists.
  • Exact conservation may also stabilize online coupling of machine-learning parameterizations inside a climate model by removing spurious source terms, but the paper does not run such coupled simulations, so this remains a testable prediction.
  • The same fixed-layer idea could enforce inequality constraints, such as positive liquid water, by inserting a positive-definite activation before the constraints layers; the paper sketches this but does not test it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents a systematic method for enforcing analytic constraints in neural networks, with applications to a super-parameterized climate model's convection parameterization. The authors introduce ACnet, which enforces linear conservation laws (energy, mass, longwave, shortwave) as hard architectural constraints, and compare it to unconstrained networks (UCnet) and loss-constrained networks (LCnet). They report two key results: ACnet achieves nearly the same MSE as the lowest-MSE UCnet (within 3%) while satisfying constraints to machine precision, and enforcing constraints reduces errors in variables that appear in the constraints, specifically the thermodynamic tendency THERMO. The paper also extends the approach to a nonlinear constraint via conversion layers and discusses inequality constraints.

Significance. If the results hold, the paper offers a practical and general strategy for building physically consistent neural-network emulators, which is important for climate modeling and other physics-based applications. The central derivation (Equations 1-9) is simple and correct under its stated assumption, train/validation/test splits are properly separated, and results are shown on all three sets. Strengths include open-source code, reproducible notebooks, and the fact that machine-precision conservation is a construction property of ACnet rather than a circular prediction. The main weakness is that the theoretical link between constraints and performance relies on an assumption about the truth data that is stated but never verified.

major comments (2)
  1. [Section II.C, Eq. (8)] The decomposition C[x, y_NN] = C[x, y_truth] + C[0, y_Err] assumes that the training targets satisfy the conservation constraints exactly. The paper never verifies this for the SP-CAM data, which are archived diagnostics on a fixed pressure grid (SM A.5). If the discrete conservation budgets do not close, then the residual outputs (e.g., low-level T and qv in SM B.1) are forced to compensate for data inconsistency, and Equation 9 is missing the C[x, y_truth] terms. Please compute C[x, y_truth] on the training, validation, and test sets and report its magnitude relative to the P values for UCnet (~90 W2m-4) and the linear baseline (~28 W2m-4). If the truth residual is not negligible, the theoretical interpretation of the cross-term and the claim that constraints violations are pure emulator error need revision, and the THERMO error reduction should be re-examined in that light.
  2. [Section III.E and SM B.2/B.3] The paper acknowledges that ACnet's residual outputs have systematically larger errors at the chosen residual level, and proposes weighting those outputs (beta > 1) to reduce the bias at the cost of overall MSE. However, the main-text key result (ACnet within 3% of UCnet, MSE 152 W2m-4 in SM Table IV) appears to refer to the standard implementation with beta=1, while the hyperparameter search in SM C.2.c identifies configurations with beta>1 and different residual indices that trade off total MSE against residual bias. The manuscript should clearly state which ACnet configuration underlies each key result and report the HP-optimized ACnet's performance, since the text suggests the residual bias is a known deficiency of the naive implementation.
minor comments (5)
  1. [Abstract and Section III.E] The phrase 'satisfying constraints to ~(10^-9%)' is dimensionally unclear; the penalty P is reported in W2m-4, so it would be clearer to write, for example, 'constraint violation of order 10^-10 W2m-4' or to state the relative magnitude with respect to typical values.
  2. [Equation (9)] The underbrace labels in Equation (9) are garbled in the formatted version, making the equation difficult to read; please re-typeset the equation with clean labels.
  3. [SM D.2] The reference to 'Table VIII, bottom-right cell' for ACnetNL's penalty is confusing because Table IX is the one with ACnetNL results; please correct the cross-reference.
  4. [Section III.E and Figure 3] The definitions of RESID and THERMO in Equation (13) are informal due to the ellipsis; please expand the equation or refer explicitly to the constraint rows in Equation (12) so that the quantities plotted in Figure 3b are unambiguous.
  5. [Abstract and Section III.E] The claim that enforcing constraints reduces errors in subsets of outputs is nuanced in the main text (it does not hold for individual components or for precipitation), but the abstract states it without qualification. Please add a brief qualifier or mention the limitations in the abstract.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the conservation constraints are externally derived and the reported performance comparisons are empirical.

full rationale

The paper's constraints are fixed analytic conservation laws derived from the SP-CAM/System for Atmospheric Modeling equations in SM A.1-A.4, not fitted targets or functions of the network's own predictions. The constraints matrix C in Eq. 12 is a constant matrix, and ACnet's exact satisfaction of C[x;y_NN]=0 is guaranteed by construction through the fixed residual layers of SM B.1, so the reported ~10^-9% conservation penalty is a construction property rather than a prediction derived from data. The paper's first key result is dominated by the empirical claim that ACnet is within 3% of UCnet in MSE on the test set, which is a genuine held-out measurement. The second key result, that enforcing constraints reduces error in variables appearing in the constraints, is presented as an empirical finding, and the paper explicitly disclaims any a-priori derivation: "Equation 9 does not provide a-priori predictions of performance, even for ACnets." The only vulnerable premise is the explicit assumption in Eq. 8 that the SP-CAM truth satisfies the conservation constraints exactly, but this is an openly stated assumption about data quality, not a self-referential reduction, and thus belongs to correctness risk rather than circularity. The self-citations to the authors' prior work (Refs. 17 and 35) are used for motivation and caveats, not as load-bearing derivations. The derivation chain is therefore self-contained and no circular step can be exhibited.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities are postulated. The free parameters are all hyperparameters tuned on validation data; the core conservation constraints are externally derived physical laws.

free parameters (4)
  • Conservation weight alpha = 0.00041 optimal; tested 0, 0.01, 0.25, 0.5, 0.75, 0.99
    Weight of the penalty term in LCnet loss (Equation 6); tuned by random hyperparameter search on validation MSE (SM C.2.b).
  • Residual loss multiplier beta = 1 to 20; beta <= 2 recommended
    Extra weight on residual outputs in ACnet loss (SM Equation 12); tuned jointly with residual indices in SM C.2.c.
  • Residual output indices (z1, z2) = Reference 29/29; hyperparameter search over 0 to 29, z1 <= 20 preferred
    Choice of vertical levels solved as residuals for mass and energy conservation (SM B.2 and C.2.c).
  • NN architecture hyperparameters = 5 layers x 512 nodes, Leaky ReLU 0.3, no dropout or batch norm, RMSprop, 20 epochs
    Baseline selected by UCnet hyperparameter search (SM C.2.a); shapes the within-3% performance claim.
assumptions (5)
  • domain assumption SP-CAM ground-truth tendencies satisfy the four conservation equations exactly.
    Stated before Equation 8 as 'assuming that constraints exactly hold for the truth'; load-bearing because exact enforcement can conflict with data if false.
  • domain assumption The conservation laws as written include all physically relevant terms for the subgrid convection parameterization (enthalpy, water, longwave, shortwave).
    SM A defines the included terms; a missing term would make the enforced balance physically wrong.
  • domain assumption The 30-level discretized pressure integrals preserve the continuous conservation laws.
    Used in SM A.5 to convert continuous integrals into linear constraints on discrete profiles.
  • domain assumption The saturation vapor pressure parameterization (SM D.3) accurately converts between relative and specific humidity.
    Needed for the nonlinear conversion-layer example in SM D.
  • standard math The constraints matrix C has full row rank n < m+p, so residual outputs are uniquely determined by row-echelon back-substitution.
    Stated in Section II.A and used in the ACnet construction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems." pith.science (2026). https://pith.science/paper/RY3GIRVD

@misc{pith2026190900912,
  author       = {Pith},
  title        = {Pith review of: Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RY3GIRVD}},
  note         = {Machine review of arXiv:1909.00912}
}
read the original abstract

Neural networks can emulate nonlinear physical systems with high accuracy, yet they may produce physically-inconsistent results when violating fundamental constraints. Here, we introduce a systematic way of enforcing nonlinear analytic constraints in neural networks via constraints in the architecture or the loss function. Applied to convective processes for climate modeling, architectural constraints enforce conservation laws to within machine precision without degrading performance. Enforcing constraints also reduces errors in the subsets of the outputs most impacted by the constraints.

Figures

Figures reproduced from arXiv: 1909.00912 by the authors.

Figure 1
Figure 1. FIG. 1. Framework to treat constrained regression problems using linearly-constrained NNs, with two examples: (1) A regression [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2. ACnet: Direct outputs are calculated using a stan [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. FIG. 3. (a) MSE and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 1
Figure 1. Figure 1: FIG. 1. For various residual levels: Squared error in convective moistening [PITH_FULL_IMAGE:figures/full_fig_p011_1.png]
Figure 2
Figure 2. Figure 2: FIG. 2. For various loss multipliers [PITH_FULL_IMAGE:figures/full_fig_p012_2.png]
Figure 3
Figure 3. Figure 3: FIG. 3. For various NN types and architectures: Coefficient of determination R [PITH_FULL_IMAGE:figures/full_fig_p014_3.png]
Figure 4
Figure 4. Figure 4: FIG. 4. Figure 3 reproduced for the training (top row) and validation (middle row) sets. (Bottom row) Convergence of the [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: FIG. 5. Training and validation MSE versus number of epochs from more than 200 models. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: FIG. 6. “Post-processing” UCnet: A standard NN is trained [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 8
Figure 8. Figure 8: The main result of this section is that we successfully enforced nonlinear constraints in NNs to excellent ap￾proximation, as can be seen by the constraints penalty of ACnetNL (Table VIII, bottom-right cell), which is 8 or￾ders of magnitude smaller than that of UCnetNL…
Figure 7
Figure 7. Figure 7: FIG. 7. (a) UCnet: Directly maps relative humidity inputs [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: FIG. 8. For UCnet, UCnet [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 45 canonical work pages

  1. [1]

    Baldi, Deep Learning in Science: Theory, Algorithms, and Applications (Cambridge University Press, Cam- bridge, UK, 2021) in press

    P. Baldi, Deep Learning in Science: Theory, Algorithms, and Applications (Cambridge University Press, Cam- bridge, UK, 2021) in press

    work page 2021

  2. [2]

    Reichstein, G

    M. Reichstein, G. Camps-Valls, B. Stevens, M. Jung, J. Denzler, N. Carvalhais, and Prabhat, Deep learning and process understanding for data-driven Earth system science, Nature 566, 195 (2019)

    work page 2019

  3. [3]

    K. J. Bergen, P. A. Johnson, M. V. De Hoop, and G. C. Beroza, Machine learning for data-driven discovery in solid Earth geoscience (2019)

    work page 2019

  4. [4]

    Karpatne, G

    A. Karpatne, G. Atluri, J. H. Faghmous, M. Steinbach, A. Banerjee, A. Ganguly, S. Shekhar, N. Samatova, and V. Kumar, Theory-guided data science: A new paradigm for scientific discovery from data, IEEE Transactions on Knowledge and Data Engineering 29, 2318 (2017)

    work page 2017

  5. [5]

    Willard, X

    J. Willard, X. Jia, S. Xu, M. Steinbach, and V. Ku- 6 mar, Integrating Physics-Based Modeling with Machine Learning: A Survey, (2020), arXiv:2003.04919

    arXiv 2020

  6. [6]

    M´ arquez-Neila, M

    P. M´ arquez-Neila, M. Salzmann, and P. Fua, Imposing Hard Constraints on Deep Networks: Promises and Lim- itations, (2017), arXiv:1706.02025

    arXiv 2017

  7. [7]

    Raissi, P

    M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics Informed Deep Learning (Part I): Data-driven Solu- tions of Nonlinear Partial Differential Equations, (2017), arXiv:1711.10561

    arXiv 2017

  8. [8]

    Bar-Sinai, S

    Y. Bar-Sinai, S. Hoyer, J. Hickey, and M. P. Brenner, Learning data-driven discretizations for partial differen- tial equations, Proceedings of the National Academy of Sciences 116, 15344 (2019)

    work page 2019

Show all 55 references
  1. [9]

    de Bezenac, A

    E. de Bezenac, A. Pajot, and P. Gallinari, Deep Learn- ing for Physical Processes: Incorporating Prior Scientific Knowledge, (2017), arXiv:1711.07970

    2017 arXiv

  2. [10]

    J. Ling, A. Kurzawski, and J. Templeton, Reynolds av- eraged turbulence modelling using deep neural networks with embedded invariance, Journal of Fluid Mechanics 807, 155 (2016)

    2016

  3. [11]

    J. L. Wu, H. Xiao, and E. Paterson, Physics-informed machine learning approach for augmenting turbulence models: A comprehensive framework, Physical Review Fluids 7, 074602 (2018)

    2018

  4. [12]

    L. Sun, H. Gao, S. Pan, and J. X. Wang, Surrogate mod- eling for fluid flows based on physics-constrained deep learning without simulation data, Computer Methods in Applied Mechanics and Engineering 361, 112732 (2020)

    2020

  5. [14]

    Karpatne, W

    A. Karpatne, W. Watkins, J. Read, and V. Ku- mar, Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling, (2017), arXiv:1710.11431

    2017 arXiv

  6. [15]

    X. Jia, J. Willard, A. Karpatne, J. Read, J. Zwart, M. Steinbach, and V. Kumar, Physics guided RNNs for modeling dynamical systems: A case study in simulating lake temperature profiles, in SIAM International Con- ference on Data Mining, SDM 2019 (2019) pp. 558–566, arXiv:1810.13075v2

    2019 arXiv

  7. [16]

    Raissi, A

    M. Raissi, A. Yazdani, and G. E. Karniadakis, Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations, Science 367, 1026 (2020)

    2020

  8. [17]

    Beucler, S

    T. Beucler, S. Rasp, M. Pritchard, and P. Gen- tine, Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling, (2019), arXiv:1906.06622

    2019 arXiv

  9. [18]

    A. K. Jain, J. Mao, and K. M. Mohiuddin, Artificial neu- ral networks: A tutorial (1996)

    1996

  10. [19]

    M. W. Gardner and S. R. Dorling, Artificial neural net- works (the multilayer perceptron) - a review of applica- tions in the atmospheric sciences, Atmospheric Environ- ment 32, 2627 (1998)

    1998

  11. [20]

    Palmer, G

    T. Palmer, G. Shutts, R. Hagedorn, F. Doblas-Reyes, T. Jung, and M. Leutbecher, Representing Model Uncer- tainty in Weather and Climate Prediction, Annual Re- view of Earth and Planetary Sciences 33, 163 (2005)

    2005

  12. [21]

    Schneider, J

    T. Schneider, J. Teixeira, C. S. Bretherton, F. Brient, K. G. Pressel, C. Sch¨ ar, and A. P. Siebesma, Climate goals and computing the future of clouds, Nature Climate Change 7, 3 (2017)

    2017

  13. [22]

    V. M. Krasnopolsky, M. S. Fox-Rabinovitz, and A. A. Be- lochitski, Using Ensemble of Neural Networks to Learn Stochastic Convection Parameterizations for Climate and Numerical Weather Prediction Models from Data Simu- lated by a Cloud Resolving Model, Advances in Artificial Neu...

    2013

  14. [24]

    S. Rasp, M. S. Pritchard, and P. Gentine, Deep learn- ing to represent sub-grid processes in climate mod- els, Proceedings of the National Academy of Sciences of the United States of America 115, 9684 (2018), arXiv:1806.04731

    2018 arXiv

  15. [25]

    N. D. Brenowitz and C. S. Bretherton, Prognostic Vali- dation of a Neural Network Unified Physics Parameteri- zation, Geophysical Research Letters 45, 6289 (2018)

    2018

  16. [26]

    Rasp, Coupled online learning as a way to tackle in- stabilities and biases in neural network parameterizations 10.5194/gmd-2019-319 (2019), arXiv:1907.01351

    S. Rasp, Coupled online learning as a way to tackle in- stabilities and biases in neural network parameterizations 10.5194/gmd-2019-319 (2019), arXiv:1907.01351

    2019 arXiv

  17. [27]

    Khairoutdinov, D

    M. Khairoutdinov, D. Randall, and C. DeMott, Sim- ulations of the Atmospheric General Circulation Using a Cloud-Resolving Model as a Superparameterization of Physical Processes, Journal of the Atmospheric Sciences 62, 2136 (2005)

    2005

  18. [28]

    M. S. Pritchard, C. S. Bretherton, and C. A. Demott, Restricting 32-128 km horizontal scales hardly affects the MJO in the Superparameterized Community Atmo- sphere Model v.3.0 but the number of cloud-resolving grid columns constrains vertical mixing, Journal of Ad- vances in M...

    2014

  19. [29]

    J. A. Andersen and Z. Kuang, Moist static energy bud- get of MJO-like disturbances in the atmosphere of a zon- ally symmetric aquaplanet, Journal of Climate 25, 2782 (2012)

    2012

  20. [30]

    Abadi, A

    M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Is- ard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Lev- enberg, D. Mane, R. Monga, S. Moore, D. Murray, C. Olah, ...

    2016 arXiv

  21. [31]

    Chollet, Keras (2015)

    F. Chollet, Keras (2015)

    2015

  22. [32]

    Tieleman, G

    T. Tieleman, G. E. Hinton, N. Srivastava, and K. Swer- sky, Lecture 6.5-rmsprop: Divide the gradient by a run- ning average of its recent magnitude, COURSERA: Neu- ral Networks for Machine Learning 4, 26 (2012)

    2012

  23. [33]

    D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, (2014), arXiv:1412.6980

    2014 arXiv

  24. [34]

    Hertel, J

    L. Hertel, J. Collado, P. Sadowski, J. Ott, and P. Baldi, Sherpa: Robust hyperparameter optimization for ma- chine learning, SoftwareX (2020), in press

    2020

  25. [35]

    Beucler, M

    T. Beucler, M. Pritchard, P. Gentine, and S. Rasp, To- wards Physically-consistent, Data-driven Models of Con- vection, (2020), arXiv:2002.08525

    2020 arXiv

  26. [36]

    pro- files

    J.-L. Wu, K. Kashinath, A. Albert, D. Chirila, Prab- hat, and H. Xiao, Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems, (2019), arXiv:1905.06841. Supplemental Material Enforcing Analytic Constraints in Neural-Networks...

    2019 arXiv

  27. [37]

    “Linear” is the multi-linear regression baseline, de- rived by replacing all of UCnet’s leaky rectified linear-unit activations with the identity function

  28. [38]

    our NN of lowest MSE (149 W 2m−4)

    “UCnet” is our best-performing NN, i.e. our NN of lowest MSE (149 W 2m−4). Despite its high per- formance, it violates conservation laws more than our multi-linear regression, motivating ACnet and LCnet

  29. [39]

    LCnet ( α = 0.01)

    “LCnet ( α = 0.01)” is our LCnet with strictly- positive conservation weight α of lowest MSE (151 W2m−4). The 1% conservation weight is enough to divide the mean penalty of “UCnet” by a factor 2.4 over the baseline validation dataset. Despite this improvement, samples at +1 st...

  30. [40]

    Its MSE is 152 W 2m−4, which is only 3W2m−4 more than our lowest-MSE UCnet

    “ACnet” is our reference ACnet described in SM B.1. Its MSE is 152 W 2m−4, which is only 3W2m−4 more than our lowest-MSE UCnet. We present the performance and constraints penalties of LCnets of varying conservation weight in Table V to bet- ter characterize the trade-off betwee...

  31. [41]

    residual

    NNs with nonlinear activation functions consistently outperform the multiple-linear regression baseline, while ACnets show a slight “residual” bias at the vertical level z = 29 (horizontal black line in SM Figure 3). Finally, as we produced Figure 3 using data from the test se...

  32. [42]

    residual

    and test (Line 5) datasets. Both “direct” MSEs are systematically to within 3% of each other, confirming that UCnets have been trained well enough to perform 9 0.0 0.5 1.0 Conservation weight 10 1 100 101 102 103 Loss ACnet ACnet UCnet LCnet (a) Performance vs Conservation MSE ...

    2000

  33. [43]

    A conversion layer ( RH↦→qv) calculating qv based on ( RH,T ) to convert x0 to x before ACnet

  34. [44]

    conversion lay- ers

    A conversion layer ( ˙qv↦→ ˙RH ) calculating ˙RH based on ( ˙qv, ˙T ) to convert y to y0 after ACnet. The resulting network, which we refer to as the nonlinear ACnet (ACnetNL for short), is depicted in SM Figure 7c. It is worth remarking that the idea of “conversion lay- ers” ...

  35. [45]

    An unconstrained network (UCnet),

  36. [46]

    An unconstrained network using the two conversion layers (RH↦→qv) and ( ˙qv↦→ ˙RH ) , referred to as the nonlinear UCnet (UCnetNL for short), to assess the effect of using conversion layers on optimization independently of their actual purpose to manage nonlinear constraints,

  37. [47]

    conver- sion

    A nonlinearly constrained network that exploits the two conversion layers to adapt the idea of ACnet to a nonlinear setting (ACnet NL). UCnetNL and ACnet NL are implemented using custom Tensorflow layers for the conversion layers. For each net- work type, we train three network...

    2000

  38. [48]

    M. F. Khairoutdinov and D. a. Randall, Cloud Resolving Modeling of the ARM Summer 1997 IOP: Model Formu- lation, Results, Uncertainties, and Sensitivities, Journal of the Atmospheric Sciences 60, 607 (2003)

    2003

  39. [49]

    W. D. Collins, P. J. Rasch, B. A. Boville, J. J. Hack, J. R. McCaa, D. L. Williamson, B. P. Briegleb, C. M. Bitz, S. J. Lin, and M. Zhang, The formulation and atmo- spheric simulation of the Community Atmosphere Model version 3 (CAM3), Journal of Climate 19, 2144 (2006)

    2006

  40. [50]

    Gentine, M

    P. Gentine, M. Pritchard, S. Rasp, G. Reinaudi, and G. Yacalis, Could Machine Learning Break the Convec- tion Parameterization Deadlock?, Geophysical Research Letters 45, 5742 (2018)

    2018

  41. [51]

    Yuval and P

    J. Yuval and P. A. O’Gorman, Stable machine-learning parameterization of subgrid processes for climate mod- eling at a range of resolutions, Nature Communications 11, 1 (2020)

    2020

  42. [52]

    Krizhevsky and G

    A. Krizhevsky and G. Hinton, Learning multiple lay- ers of features from tiny images.(2009), Cs.Toronto.Edu (2009)

    2009

  43. [53]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bern- stein, A. C. Berg, and L. Fei-Fei, ImageNet Large Scale Visual Recognition Challenge, International Journal of Computer Vision 115, 211 (2015), arXiv:1409.0575

    2015 arXiv

  44. [54]

    Hertel, P

    L. Hertel, P. Sadowski, J. Collado, and P. Baldi, Sherpa : Hyperparameter Optimization for Machine Learning Models, Conference on Neural Information Processing Systems (NIPS) (2018)

    2018

  45. [55]

    Bergstra and Y

    J. Bergstra and Y. Bengio, Random search for hyper- parameter optimization, Journal of Machine Learning Research 13, 281 (2012)

    2012

  46. [56]

    Bolton and L

    T. Bolton and L. Zanna, Applications of Deep Learning to Ocean Data Inference and Subgrid Parameterization, Journal of Advances in Modeling Earth Systems 11, 376 (2019)

    2019

  47. [57]

    M. F. Khairoutdinov and D. A. Randall, Cloud Resolving Modeling of the ARM Summer 1997 IOP: Model Formu- lation, Results, Uncertainties, and Sensitivities, Journal of the Atmospheric Sciences 60, 607 (2003)

    2003

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.