Pith. sign in

REVIEW 3 major objections 5 minor 34 references

Chaos into Order: Neural Framework for Expected Value Estimation of Stochastic Partial Differential Equations

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One feedforward network trained on averaged noise residuals can approximate the expected solution of a linear stochastic PDE.

desk verdict An honest, workmanlike empirical paper whose core mechanism is a one-line identity the authors never write down; the claim is plausible and the gaps are fixable. read the letter →

arxiv 2502.03670 v2 pith:2DAF55EN submitted 2025-02-05 cs.LG

Ísak Pétursson , María Óskarsdóttir This is my paper · ORCID
classification cs.LG MSC 60H1565M7568T07
keywords Physics-InformedNeuralNetworksStochasticPartialDifferentialEquationsExpectedvalueestimationMonteCarlosamplingheatequationMean-fieldapproximationHigh-dimensionalPDEsAdditivenoise
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that an ordinary physics-informed neural network, trained by averaging its PDE residual over many random noise samples, learns the expected value of the solution of a linear stochastic partial differential equation. The authors introduce the Learned Expectation Collapser (LEC), which samples space-time points and noise parameters together during training and minimizes a Monte Carlo-averaged residual loss. They test this on the stochastic heat equation with additive noise across 144 configurations, varying spatial dimension, noise process, forcing function, and number of samples. The empirical pattern is that the network tracks the analytical expectation closely in low dimensions, with relative L2 error roughly doubling every two dimensions, while larger Monte Carlo batches mainly stabilize training. The paper is explicit that this is an empirically supported hypothesis rather than a proven theorem.

What carries the argument

The load-bearing object is the Monte Carlo-averaged residual loss with the network input restricted to distribution parameters rather than noise realizations. At each training step the same network output $u_\theta(t,x)$ is scored against $m$ independent samples $W_i$, so the squared residual expands into the residual at the mean forcing plus a noise-variance term that does not depend on $u_\theta$. That decoupling is what lets the minimizer align with the solution of the mean PDE. The network architecture is a fully connected tanh network whose width and depth grow slowly with dimension, and the input includes the noise model's parameters ($\mu,\sigma$ etc.) sampled from fixed ranges, which lets one network represent a family of expectation solutions.

What would settle it

Train LEC on a zero-mean noise process with large variance, such as $\xi(t)=\sigma W_t$ with no drift, so that $\mathbb{E}[W(\xi(t))]=0$; the averaged residual loss then reduces to the deterministic heat residual plus a constant, so the network should reproduce the pure initial-boundary-value solution. If the learned function deviates from that solution, the minimizer is not the expectation.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a single feedforward network trained to minimize $$L = \frac{1}{m}\sum_{i=1}^m \|\partial_t u_\$\theta$ - \nu\$\Delta$ u_\$\theta$ - \vartheta G(x) W_i(\xi(t))\|^2 + \lambda_I \|u_\$\theta$(0,x)-\vartheta G(x)\|^2 + \lambda_B \|u_\$\theta$\|^2_{\partial\$\Omega$}$$ converges to $\mathbb{E}[u(t,x)]$, the expectation of the SPDE solution, even though the network never sees the noise path $W_i$ and the loss contains no explicit expectation. The paper verifies this for the stochastic heat equation with additive noise, using forcing maps $W(\xi)=\xi$, $W(\xi)=e^{-t}\xi$, and $W(\xi)=\xi^2$, and three noise processes whose analytical means are known. Across all 144 runs the network reduces loss and error, with best accuracy at dimension 2 (mean relative L2 error 0.032) and growing error at higher dimensions (0.47 at d=8). The authors state the result as a hypothesis confirmed empirically, not as a convergence theorem.

Load-bearing premise

The load-bearing premise is that the minimizer of the averaged squared residual equals the solution of the mean PDE; this requires the network output to be independent of the noise realization, so that the noise variance adds only a constant to the loss.

Editorial extensions

If this is right

  • If the claim holds, the expected solution of a linear SPDE with additive noise can be approximated by sampling the noise during training, without deriving $\mathbb{E}[W]$ or solving pathwise.
  • The same trained network can supply expectations across a range of noise-model parameters, because those parameters are inputs rather than fixed experiment constants.
  • Accuracy degrades predictably with spatial dimension, with relative L2 error roughly doubling every two dimensions, so the method's practical range is low-to-moderate dimension unless sampling or architecture changes are made.
  • Increasing the Monte Carlo batch size does not dramatically change final accuracy but visibly stabilizes the loss and error trajectories during training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The decoupling argument suggests the method should transfer to any linear SPDE with additive noise whose deterministic part is well posed, e.g. advection-diffusion or wave equations; the noise model would enter only through its mean and variance.
  • Because the loss minimizer depends on the noise only through its mean plus a constant variance, differences in accuracy between smooth and jump noise probably reflect optimization difficulty, not a change in the target expectation; testing with identical variance across noise models would separate the two.
  • A natural testable extension is to compare LEC's expectation against a Monte Carlo average of pathwise PINN solutions; if the two disagree, the single-network minimizer is capturing a different regression quantity than the true expectation.
  • The observed dimension scaling suggests that variance-reduced sampling of collocation points, or a better parameterization of the high-dimensional input, might push the method beyond d=8; the paper leaves this open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Learned Expectation Collapser (LEC), a physics-informed neural network trained on a Monte Carlo-averaged residual loss for linear SPDEs with additive noise. The central claim, stated in Sections 1 and 5, is that a single feedforward network trained by minimizing the average squared residual over stochastic realizations converges to the expected solution E[u(t,x)] of the SPDE. The method is tested on the stochastic heat equation with Dirichlet boundary conditions across spatial dimensions d = 2, 4, 6, 8, three noise models (Brownian motion with drift, Ornstein-Uhlenbeck, compound Poisson), three forcing maps, and four Monte Carlo sample sizes, totaling 144 configurations. Relative L2 errors are reported in Tables 1-4, showing errors that grow with dimension and with noise/forcing complexity. The paper includes a public code repository and derives analytical benchmark expectations in Appendix A.

Significance. If the central claim is correct, the observation is useful and somewhat surprising: a standard PINN can estimate the mean solution of a linear SPDE without deriving E[W] or solving pathwise, using only samples of the noise process. The paper's strengths are its reproducible public code, the breadth of the 144-configuration benchmark, and the fact that the underlying loss-decomposition identity for linear additive noise is correct. However, the manuscript is currently an empirical demonstration rather than a supported claim: the key identity that makes the stochastic residual minimizer equal to the mean-PDE minimizer is never stated, and no baseline PINN trained directly on the mean PDE is reported. These omissions leave the central claim load-bearing but unproven and empirically under-supported relative to the strength of the conclusion.

major comments (3)
  1. [Section 3.2, Eq. (3.2)] The central claim that minimizing the Monte Carlo-averaged residual yields E[u] rests on the pointwise identity E_W[(A u_theta - f - gW)^2] = (A u_theta - f - g E[W])^2 + g^2 Var(W), which holds when u_theta is independent of the noise realization W. This identity is never stated or proved in the paper. Please add an explicit derivation showing that the variance term is independent of u_theta, so the minimizers of the stochastic residual coincide with the minimizers of the deterministic mean-PDE residual, and state the regularity assumptions (finite second moments, integrability) needed for the argument. Without this derivation, the statement in Section 5 that vanilla PINNs 'converge to the expected solution' is a hypothesis, especially since Section 4.5 concedes the method 'lacks a formal convergence proof.'
  2. [Section 4, Tables 1-4] No baseline PINN trained directly on the mean PDE with source term vartheta G(x) E[W(xi(t))] is reported. Such a baseline is the natural control: if the stochastic residual training truly recovers E[u], it should match the accuracy of a PINN trained on the deterministic mean equation with the same architecture, collocation points, and training budget. Without this comparison, the reader cannot distinguish between the proposed stochastic-averaging mechanism and the trivial possibility that either training objective yields a similar approximate solution to the mean PDE. Please add this baseline and report the comparison, at least for a representative subset of configurations.
  3. [Section 4.1, Tables 1-2] The reported error statistics are averages over 36 configurations per dimension, not over independent training runs with different random seeds. The standard deviations in Tables 1-4 therefore reflect variation across noise models, forcing functions, and Monte Carlo settings, not the stochasticity of neural-network optimization. To support the claim that the model 'consistently' learns the expected solution, please either report the number of seeds per configuration and give mean +/- std over seeds for at least a representative subset, or explicitly state that each configuration was trained once and present the variability across seeds as a separate quantity.
minor comments (5)
  1. [Section 3.1] The equation after taking expectations appears typographically corrupted: 'partial_nu E[u(t,x)] / partial t' should read 'partial E[u(t,x)] / partial t = nu Delta E[u] + vartheta G(x) E[W(xi(t))]'. Please correct the notation.
  2. [Algorithm 1, line 25] The loss-weight assignment in the algorithm is inconsistent with Section 3.2: line 25 uses lambda_B for the initial-condition loss and lambda_I for the boundary loss, while the text defines lambda_I for initial and lambda_B for boundary. Please correct the algorithm and confirm that the code matches the intended weighting.
  3. [Section 2.1, reference [33]] The sentence 'Jin et al. [33] present fractional PINNs...' cites reference [33], which is a paper on masked autoencoders by Kim, Choi, Widemann, and Zohdi, not a fractional-PINN paper by Jin et al. Please check the citation and either correct the reference or the attribution.
  4. [Section 3.2] The phrase 'abalition studies' appears to be a typo for 'ablation studies'. Please proofread the manuscript for similar typographical errors.
  5. [Section 4.3, Table 3] The text says the exponential-linear forcing 'slightly outperforms linear in most cases,' but in d=2 the linear forcing has lower mean error (0.0294 vs 0.0308). Consider qualifying this statement to avoid over-generalization.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training loss does not encode the expected solution, benchmarks are derived independently, and no load-bearing self-citation appears.

full rationale

The paper's chain is self-contained with respect to its inputs. The LEC loss in Section 3.2 is defined purely from sampled noise realizations W^(i), the SPDE residual, and the initial/boundary conditions; it never uses the analytical expected solution E[u]. The evaluation target E[u] is derived independently in Appendix A from the linearity of expectation for Gaussian, Ornstein-Uhlenbeck, and compound-Poisson driving processes, using standard first- and second-moment formulas. No model parameter is fitted to E[u], and no result is imported from the authors' prior work; the cited references are external. The paper's central hypothesis that Monte-Carlo-averaged residual minimization converges to E[u] is asserted rather than proved, and for linear additive noise with a noise-independent network it follows from the elementary identity E_W[(Au - f - gW)^2] = (Au - f - gE[W])^2 + g^2 Var(W). Omitting that derivation is a correctness/completeness limitation, not circular reasoning, because the prediction is not built into the loss by construction. Therefore the circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on linearity of expectation, an unstated least-squares variance-decoupling lemma, and standard capacity and optimization assumptions. There are no invented physical entities. Several hand-chosen hyperparameters affect the reported accuracy but not the core convergence mechanism.

free parameters (6)
  • scaling factor theta = theta = sqrt(d) * log10(exp(d))
    Hand-chosen normalization to counteract decay of the sine-product solution magnitude with dimension; affects target scale and relative error.
  • initial and boundary loss weights lambda_I, lambda_B = 10
    Chosen to enforce initial and boundary conditions; no sensitivity study reported.
  • network width and depth scaling = rho(d)=floor(128*sqrt(d/2)), L(d)=round(6+2*log2(d/2))
    Capacity schedule chosen from preliminary ablations; no systematic study reported.
  • collocation point count n(d) = min(1000*(d/2)^2, 5000)
    Hand-chosen sampling budget per dimension; no convergence study in the number of collocation points.
  • training schedule = 10,000 epochs, learning rate 0.003
    Fixed schedule; no convergence criterion and no learning-rate study.
  • noise parameter sampling ranges = e.g., mu in (0.1,0.5), sigma in (0.1,1.0)
    Hand-chosen ranges over which the network learns a parametric family; results are only valid within these ranges.
assumptions (4)
  • standard math Expectation commutes with the linear differential operator and with initial and boundary conditions: E[du/dt] = dE[u]/dt and E[Delta u] = Delta E[u].
    Used in Section 3.1 to derive the mean PDE and the analytical benchmark solutions; requires integrability and regularity conditions that are not stated.
  • standard math For each fixed (t,x), the minimizer of E_W[(A u_theta - f - gW)^2] over the value of A u_theta equals f + gE[W], because the variance term is independent of A u_theta.
    This is the missing link that would justify the central hypothesis in Section 3.2, but the paper does not state or prove it.
  • domain assumption The feedforward network has sufficient capacity and the optimizer reaches a good global minimum.
    Standard PINN assumption; no universal approximation or optimization guarantee is provided.
  • domain assumption Sampled collocation points and Monte Carlo noise terms provide unbiased estimates of the continuous loss.
    Required for the stochastic loss to approximate the mean-field PDE residual; finite-sample effects are not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chaos into Order: Neural Framework for Expected Value Estimation of Stochastic Partial Differential Equations." pith.science (2026). https://pith.science/paper/2DAF55EN

@misc{pith2026250203670,
  author       = {Pith},
  title        = {Pith review of: Chaos into Order: Neural Framework for Expected Value Estimation of Stochastic Partial Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2DAF55EN}},
  note         = {Machine review of arXiv:2502.03670}
}
read the original abstract

Stochastic partial differential equations (SPDEs) describe the evolution of random processes over space and time, but their solutions are often analytically intractable and computationally expensive to estimate. In this paper, we propose the Learned Expectation Collapser (LEC), a physics-informed neural framework designed to approximate the expected value of linear SPDE solutions without requiring domain discretization. By leveraging randomized sampling of both space-time coordinates and noise realizations during training, LEC trains standard feedforward neural networks to minimize residual loss across multiple stochastic samples. We hypothesize and empirically confirm that this training regime drives the network to converge toward the expected value of the solution of the SPDE. Using the stochastic heat equation as a testbed, we evaluate performance across a diverse set of 144 experimental configurations that span multiple spatial dimensions, noise models, and forcing functions. The results show that the model consistently learns accurate approximations of the expected value of the solution in lower dimensions and a predictable decrease in accuracy with increased spatial dimensions, with improved stability and robustness under increased Monte Carlo sampling. Our findings offer new insight into how neural networks implicitly learn statistical structure from stochastic differential operators and suggest a pathway toward scalable, simulator-free SPDE solvers.

Figures

Figures reproduced from arXiv: 2502.03670 by the authors.

Figure 1
Figure 1. Model performance across different forcing functions. Each point represents a [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
Figure 2
Figure 2. Evolution of training loss (top) and relative [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 28 canonical work pages

  1. [1]

    S. G. Rosofsky, H. Al Majed, E. Huerta, Applications of physics in- formed neural operators, Machine Learning: Science and Technology 4 (2) (2023) 025022

    work page 2023

  2. [2]

    L. Cui, M. Fall, A coupled thermo–hydro-mechanical–chemical model for underground cemented tailings backfill, Tunnelling and Underground Space Technology 50 (2015) 396–414

    work page 2015

  3. [3]

    Scholz, F

    G. Scholz, F. Scholz, First-order differential equations in chemistry, ChemTexts 1 (2015) 1–12

    work page 2015

  4. [4]

    D. J. Duffy, Finite difference methods in financial engineering: a partial differential equation approach, John Wiley & Sons, 2013

    work page 2013

  5. [5]

    Black, M

    F. Black, M. Scholes, The pricing of options and corporate liabilities, Journal of political economy 81 (3) (1973) 637–654

    work page 1973

  6. [6]

    R. C. Merton, Theory of rational option pricing, The Bell Journal of economics and management science (1973) 141–183

    work page 1973

  7. [7]

    H. S. Carslaw, Introduction to the Mathematical Theory of the Conduc- tion of Heat in Solids, Macmillan and Company, limited, 1906

    work page 1906

  8. [8]

    Courant, D

    R. Courant, D. Hilbert, Methods of mathematical physics, volume 1, Vol. 1, John Wiley & Sons, 2008

    work page 2008

Show all 34 references
  1. [9]

    S. L. Heston, A closed-form solution for options with stochastic volatility with applications to bond and currency options, The review of financial studies 6 (2) (1993) 327–343

    1993

  2. [10]

    Risken, The Fokker-Planck Equation, Springer, 1996

    H. Risken, The Fokker-Planck Equation, Springer, 1996

    1996

  3. [11]

    Cercignani, C

    C. Cercignani, C. Cercignani, The boltzmann equation, Springer, 1988

    1988

  4. [12]

    I. T. Dimov, Monte Carlo methods for applied scientists, World Scien- tific, 2008

    2008

  5. [13]

    G. D. Smith, Numerical solution of partial differential equations: finite difference methods, Oxford university press, 1985. 22

    1985

  6. [14]

    O. C. Zienkiewicz, R. L. Taylor, The finite element method set, Elsevier, 2005

    2005

  7. [15]

    L. C. Evans, An introduction to stochastic differential equations, Vol. 82, American Mathematical Soc., 2012

    2012

  8. [16]

    Da Prato, J

    G. Da Prato, J. Zabczyk, Stochastic equations in infinite dimensions, Vol. 152, Cambridge university press, 2014

    2014

  9. [17]

    Bertini, N

    L. Bertini, N. Cancrini, The stochastic heat equation: Feynman-kac formula and intermittence, Journal of statistical Physics 78 (1995) 1377– 1401

    1995

  10. [18]

    von Renesse, M

    M.-K. von Renesse, M. Scheutzow, Existence and uniqueness of solutions of stochastic functional differential equations (2010)

    2010

  11. [19]

    Keese, A review of recent developments in the numerical solution of stochastic partial differential equations (stochastic finite elements), Univ.-Bibl., 2003

    A. Keese, A review of recent developments in the numerical solution of stochastic partial differential equations (stochastic finite elements), Univ.-Bibl., 2003

    2003

  12. [20]

    Marshall, Monte carlo methods for the solution of nonlinear partial differential equations, Computer physics communications 56 (1) (1989) 51–61

    G. Marshall, Monte carlo methods for the solution of nonlinear partial differential equations, Computer physics communications 56 (1) (1989) 51–61

    1989

  13. [21]

    Allen, S

    E. Allen, S. Novosel, Z. Zhang, Finite element and difference approxima- tion of some linear stochastic partial differential equations, Stochastics: An International Journal of Probability and Stochastic Processes 64 (1-

  14. [22]

    Raissi, P

    M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational physics 378 (2019) 686–707

    2019

  15. [23]

    Sirignano, K

    J. Sirignano, K. Spiliopoulos, Dgm: A deep learning algo- rithm for solving partial differential equations, Journal of Computational Physics 375 (2018) 1339–1364. doi:https: //doi.org/10.1016/j.jcp.2018.08.029. URL https://www.sciencedirect.com/science/article/pii/ S0021999118...

    2018 doi

  16. [24]

    L. Lyu, Z. Zhang, M. Chen, J. Chen, Mim: A deep mixed residual method for solving high-order partial differential equa- tions, Journal of Computational Physics 452 (2022) 110930. doi:https://doi.org/10.1016/j.jcp.2021.110930. URL https://www.sciencedirect.com/science/article/p...

    2022

  17. [25]

    Salvi, M

    C. Salvi, M. Lemercier, A. Gerasimovics, Neural stochastic pdes: Resolution-invariant learning of continuous spatiotemporal dynamics, Advances in Neural Information Processing Systems 35 (2022) 1333– 1344

    2022

  18. [26]

    Karumuri, R

    S. Karumuri, R. Tripathy, I. Bilionis, J. Panchal, Simulator-free solution of high-dimensional stochastic elliptic partial differential equations us- ing deep neural networks, Journal of Computational Physics 404 (2020) 109120

    2020

  19. [27]

    C. Beck, S. Becker, P. Cheridito, A. Jentzen, A. Neufeld, Deep learn- ing based numerical approximation algorithms for stochastic partial dif- ferential equations and high-dimensional nonlinear filtering problems (2020). arXiv:2012.01194. URL https://arxiv.org/abs/2012.01194

    2020

  20. [28]

    Zhang, L

    D. Zhang, L. Guo, G. E. Karniadakis, Learning in modal space: Solv- ing time-dependent stochastic pdes using physics-informed neural net- works, SIAM Journal on Scientific Computing 42 (2) (2020) A639– A665. arXiv:https://doi.org/10.1137/19M1260141, doi:10.1137/ 19M1260141. UR...

    2020 doi

  21. [29]

    C. Beck, M. Hutzenthaler, A. Jentzen, B. Kuckuck, An overview on deep learning-based approximation methods for partial differential equations, Discrete and Continuous Dynamical Systems - B 28 (6) (2023) 3697–3746. doi:10.3934/dcdsb.2022238. URL https://www.aimsciences.org/arti...

    2023 doi

  22. [30]

    L. Lyu, K. Wu, R. Du, J. Chen, Enforcing exact boundary and initial conditions in the deep mixed residual method (2020). arXiv:2008. 24 01491. URL https://arxiv.org/abs/2008.01491

    2020 arXiv

  23. [31]

    S. Wang, X. Yu, P. Perdikaris, When and why pinns fail to train: A neural tangent kernel perspective, Journal of Computational Physics 449 (2022) 110768

    2022

  24. [32]

    L. D. McClenny, U. M. Braga-Neto, Self-adaptive physics-informed neu- ral networks, Journal of Computational Physics 474 (2023) 111722

    2023

  25. [33]

    Y. Kim, Y. Choi, D. Widemann, T. Zohdi, A fast and accurate physics- informed neural network reduced order model with shallow masked au- toencoder, Journal of Computational Physics 451 (2022) 110841

    2022

  26. [34]

    Yuan, Y.-Q

    L. Yuan, Y.-Q. Ni, X.-Y. Deng, S. Hao, A-pinn: Auxiliary physics informed neural networks for forward and inverse problems of nonlin- ear integro-differential equations, Journal of Computational Physics 462 (2022) 111260. 25

    2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.