REVIEW 4 minor 1 cited by
Space-time error estimates for deep neural network approximations for differential equations
T0 review · 0 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read ReLU networks approximate Euler trajectories in time and space.
desk verdict Solid, fully written-up first space-time error estimates for ReLU-network Euler schemes; the central proof holds up, with the usual caveat that it is about Eulerization, not PDEs. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument is carried by an ANN calculus, a compositional algebra for building networks realizing sums, parallelizations, compositions, and Euler updates, together with explicit ReLU approximations of the square function x -> $x^{2}$ and of the product (t,x) -> tx. The product network is the space-time bridge: it converts each Euler step into a network operation, hat functions interpolate between grid times, and a discrete Gronwall estimate keeps the trajectory growth polynomial in ||x||. The rectifier activation a(x)=max{x,0} is used throughout.
What would settle it
Check the claim for d=1, N=1, T=1, with drift R(Phi)(x)=max{x,0}. The Euler map is Y_{t,x}=x+t*max{x,0}, so the network must realize the piecewise-quadratic map (t,x) -> x+t*max{x,0} on [0,1] x R. Compute the minimal ReLU network parameters needed to approximate this map to error epsilon and compare with the asserted bound P(Psi) <= C(1+|ln(epsilon)|^2); if the required parameter count grows faster than log(1/epsilon) as epsilon tends to 0, the epsilon-dependence in Theorem 1.1 fails.
Extended reading notes
Core claim
The main result, Theorem 3.12 with the special case Theorem 1.1, asserts that for every epsilon in (0,1], every dimension d, and every number N of Euler steps, there exist ReLU networks $Psi^{{epsilon,d,N}}$ such that for all t in [0,T] and x in R^d, ||$Y^{{d,N}}$_{t,x} - R($Psi^{{epsilon,d,N}}$)(t,x)|| <= C $d^{{1/2}}$ $N^{{3/2}}$ epsilon (1+||x||^3), while the network size satisfies P($Psi^{{epsilon,d,N}}$) <= C $d^{{16+8d}}$ $N^{6}$ (1+|ln(epsilon)|^2). Here $Y^{{d,N}}$ is the Euler approximation of an ordinary differential equation whose drift is itself given by a network with linear growth. The approximation is space-time in the strong sense: the same network realizes the solution map (t,x) -> Y_{t,x} continuously on [0,T] x R^d, and the error is uniform in t and x.
Load-bearing premise
The drift network must grow at most linearly, with ||R(Phi)(x)|| <= C(1+||x||), because that bound is the input to the Gronwall step that keeps the Euler trajectories finite and makes the a priori bounds enter the error estimate; if it is violated, the claimed uniform bounds can blow up.
Editorial extensions
If this is right
- For every tolerance epsilon, dimension d, and number of Euler steps N, a ReLU network of explicitly bounded size realizes the Euler map uniformly on [0,T] x R^d.
- The time direction is controlled by the same accuracy parameter epsilon as the spatial direction, closing the gap left by purely spatial estimates.
- The construction is explicit enough to be read as an algorithm: compose networks for the square, the product, hat functions, and the Euler update.
- The network size grows polynomially in N and logarithmically in 1/epsilon, but exponentially in the dimension d.
- The a priori bounds used in the proof are uniform in the perturbation sequence y, so the same network family covers perturbed Euler schemes in one stroke.
Reading between the lines
- The paper stops at Euler approximations of ODEs; the announced next step is to feed these space-time networks into PDE approximation schemes. A testable extension is to insert the constructed network into a splitting or backward-solver pipeline and check whether the observed error obeys the same epsilon-dependence.
- The size bound d^{16+8d} N^6 (1+|ln(epsilon)|^2) grows exponentially in d, so the result is a consistency and expressivity statement for fixed dimension, not a curse-of-dimensionality breakthrough; a follow-up would need to reduce this to a polynomial-in-d bound.
- Because the construction is explicit and deterministic, it provides a training-free baseline for neural emulation of differential-equation solvers: the weights can be hard-coded and evaluation cost compared against classical Euler integration.
- The requirement that the drift network grow at most linearly is inherited from the Gronwall step; weakening it would likely need a different a priori control and would change the polynomial-in-N factors in the error bound.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims to provide the first space-time error estimates for deep ReLU network approximations of Euler approximations of perturbed differential equations. The main result, Theorem 3.12, asserts that for an Euler scheme driven by a ReLU drift network with linear growth, there exist ReLU networks whose space-time map approximates the Euler flow with explicit error bounds of the form eps times a polynomial in d, N, and the a priori growth bound, together with explicit parameter-count bounds polynomial in N and d and quadratic in |log eps|. The proof develops an ANN calculus for compositions, parallelizations, and sums of ReLU networks; constructs product-approximation networks from a square-function approximant; and combines a posteriori error estimates via time hat-function interpolation with a discrete Gronwall a priori bound. Corollary 3.13 and Theorem 1.1 give simplified closed-form bounds, including the y=0 case stated in the introduction. The paper explicitly states that the estimates concern Euler approximations rather than the differential equation or PDE solution itself.
Significance. If correct, the result is a useful foundational contribution: it appears to be the first space-time error estimate, rather than a purely spatial one, for DNN approximation of an Euler scheme, and it supplies reusable ingredients with fully written proofs and explicitly tracked constants. The central statement is a conditional existence result with no free parameters: given any ReLU drift network, the constructed network and the error and size bounds are explicit. The weakest-assumption concern raised in review, namely the linear growth bound on the drift network, is not fragile: every fixed-depth ReLU network is globally Lipschitz, so the bound holds automatically with C equal to the network's Lipschitz constant, and the constant only enters through exp(CT). The main limitation, that the result concerns Euler approximations and not the exact ODE/PDE solution, is stated honestly in the abstract and introduction and does not undermine the central claim.
minor comments (4)
- [Proposition 3.5(v), Section 3.2.1] The displayed intermediate bound in Proposition 3.5(v) appears as "+ 1/(q-2) - 252", while the proof around Eq. (306) establishes the same expression with "+ 720/(q-2) - 252". The final bound is unaffected, but the statement should be corrected for consistency.
- [Section 2 and Proposition 3.10] The paper says that every result in Section 2 is proved in detail, yet Proposition 3.10 invokes the existence of identity networks I_d with D(I_d)=(d,2d,d) and R(I_d)=id by citing [25, Lemma 5.4], and Lemma 2.14 treats such networks as an assumption. Since the construction is elementary, either include it or explicitly acknowledge this external result in the self-containedness statement.
- [Theorem 1.1 and Corollary 3.13] The final constant is stated as "C in R", but the displayed inequalities are only meaningful when the constant is nonnegative; the statements should read "C in [0, infinity)".
- [Title and abstract] The title refers to "differential equations", but the theorem concerns Euler approximations rather than solutions of the differential equation; although the abstract and introduction make this clear, the title could be more precise, e.g., by mentioning Euler schemes.
Circularity Check
No significant circularity: the main theorem is a fully self-contained conditional existence result.
full rationale
I walked the derivation chain from the ANN calculus in Section 2 through the square-function and product approximations in Section 3.1–3.2 to the a posteriori estimate (Proposition 3.10), the Gronwall-based a priori bound (Lemma 3.11), and the main Theorem 3.12. Each step is proved in the paper with explicit constants, and the final estimate is a genuine conditional existence statement: given any ReLU drift network Φ with linear growth, the paper constructs a ReLU network Ψ whose realization approximates the Euler scheme of Φ in space-time with explicit error and parameter bounds. No parameter is fitted to the target quantity Y, and no equation is used as both input and output. The square-function approximant is built from an explicit hat-function interpolation (Lemmas 3.1–3.2), the product approximant from the polarization identity and the square approximant, and the Euler representation from a hat-function interpolation of the trajectory; all of these are independent constructions rather than restatements of the desired estimate. The paper does cite prior work from the same research group, notably [25] for an elementary identity-network lemma and for inspiration in the Euler representation, but the relevant identity network is a trivial ReLU construction and the Euler-representation argument is re-proved in the paper (Proposition 2.32 and Corollary 2.31). Such citations are not load-bearing in the sense of importing the target theorem. The acknowledged limitation—that the estimates concern Euler approximations rather than the differential-equation solution itself—is stated in the abstract and introduction and does not create circularity. No self-definitional step, fitted-input-called-prediction step, or imported-uniqueness argument was found.
Assumptions & free parameters
assumptions (5)
- standard math Discrete Gronwall inequality (Hutzenthaler et al. [23, Lemma 2.1])
- standard math Existence of a ReLU identity network I_d with D(I_d)=(d,2d,d) and R(I_d)(x)=x (Jentzen-Salimova-Welti [25, Lemma 5.4])
- standard math ReLU approximation of the square function on [0,1] with error ε and O(log ε^{-1}) parameters (Yarotsky [37], Elbrächter et al. [11])
- domain assumption Growth bound on the drift network: ||R(Φ)(x)|| ≤ C(1+||x||)
- domain assumption Size bound on the drift network: P(Φ_d) ≤ C d^d in Corollary 3.13 / Theorem 1.1
Cite this review
Pith. "Pith review of Space-time error estimates for deep neural network approximations for differential equations." pith.science (2026). https://pith.science/paper/VFEXWTVF
@misc{pith2026190803833,
author = {Pith},
title = {Pith review of: Space-time error estimates for deep neural network approximations for differential equations},
year = {2026},
howpublished = {\url{https://pith.science/paper/VFEXWTVF}},
note = {Machine review of arXiv:1908.03833}
}
abstract
Over the last few years deep artificial neural networks (DNNs) have very successfully been used in numerical simulations for a wide variety of computational problems including computer vision, image classification, speech recognition, natural language processing, as well as computational advertisement. In addition, it has recently been proposed to approximate solutions of partial differential equations (PDEs) by means of stochastic learning problems involving DNNs. There are now also a few rigorous mathematical results in the scientific literature which provide error estimates for such deep learning based approximation methods for PDEs. All of these articles provide spatial error estimates for neural network approximations for PDEs but do not provide error estimates for the entire space-time error for the considered neural network approximations. It is the subject of the main result of this article to provide space-time error estimates for DNN approximations of Euler approximations of certain perturbed differential equations. Our proof of this result is based (i) on a certain artificial neural network (ANN) calculus and (ii) on ANN approximation results for products of the form $[0,T]\times \mathbb{R}^d\ni (t,x)\mapsto tx\in \mathbb{R}^d$ where $T\in (0,\infty)$, $d\in \mathbb{N}$, which we both develop within this article.
Forward citations
Cited by 1 Pith paper
-
Deep neural network approximations for Monte Carlo algorithms
A general theorem shows that neural networks inherit the absence of the curse of dimensionality from any discrete Monte Carlo scheme they can emulate, with applications to Kolmogorov PDEs.
Reference graph
Works this paper leans on
-
[25]
Jentzen, A., Salimova, D., and Welti, T. A proof that deep artificial neural networks overcome the curse of dimensionality in the numer ical approx- imation of Kolmogorov partial differential equations with constant d iffusion and nonlinear drift coefficients. arXiv:1809.07321 (2018), 48 pages
arXiv 2018
-
[1]
Deep splitting method for parabolic PDEs
Beck, C., Becker, S., Cheridito, P., Jentzen, A., and Neufeld, A. Deep splitting method for parabolic PDEs. arXiv:1907.03452 (2019), 40 pages
arXiv 2019
-
[2]
Solving stochastic differential equations and Kolmogorov equations by means of deep learning
Beck, C., Becker, S., Grohs, P., Jaafari, N., and Jentzen, A. Solving stochastic differential equations and Kolmogorov equations by means of deep learning. arXiv:1806.00421 (2018), 56 pages
arXiv 2018
-
[3]
Beck, C., E, W., and Jentzen, A. Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Eq ua- tions and Second-order Backward Stochastic Differential Equatio ns. Journal of Nonlinear Science (Jan 2019). doi: 10.1007/s00332-018-9525-3
-
[4]
Becker, S., Cheridito, P., and Jentzen, A. Deep Optimal Stopping. Journal of Machine Learning Research 20 , 74 (2019), 1–25. doi: 10.3929/ethz- b-000344707. 83
doi:10.3929/ethz- 2019
-
[5]
Solving high- dimensional optimal stopping problems using deep learning
Becker, S., Cheridito, P., Jentzen, A., and Welti, T. Solving high- dimensional optimal stopping problems using deep learning. arXiv:1908.01602 (2019), 42 pages
arXiv 2019
-
[6]
Berg, J., and Nystr ¨om, K. A unified deep artificial neural network ap- proach to partial differential equations in complex geometries. Neurocomput- ing 317 (2018), 28–41. doi:10.1016/j.neucom.2018.06.056
-
[7]
Berner, J., Grohs, P., and Jentzen, A. Analysis of the generalization error: Empirical risk minimization over deep artificial neural networ ks over- comes the curse of dimensionality in the numerical approximation of B lack- Scholes partial differential equations. arXiv:1809.03062 (2018), 35 pages
arXiv 2018
Show all 37 references
-
[8]
Machine Learning for Semi Linear PDEs
Chan-W ai-Nam, Q., Mikael, J., and W arin, X. Machine Learning for Semi Linear PDEs. Journal of Scientific Computing 79 , 3 (Jun 2019), 1667–1712. doi: 10.1007/s10915-019-00908-3
2019 doi
-
[9]
Deep Learning-Based Numerical Meth- ods for High-Dimensional Parabolic Partial Differential Equations an d Back- ward Stochastic Differential Equations
E, W., Han, J., and Jentzen, A. Deep Learning-Based Numerical Meth- ods for High-Dimensional Parabolic Partial Differential Equations an d Back- ward Stochastic Differential Equations. Commun. Math. Stat. 5 , 4 (2017), 349–380. doi: 10.1007/s40304-017-0117-6
2017 doi
-
[10]
The deep Ritz method: A deep learning-based numer- ical algorithm for solving variational problems
E, W., and Yu, B. The deep Ritz method: A deep learning-based numer- ical algorithm for solving variational problems. arXiv:1710.00211 (2017), 14 pages
2017 arXiv
-
[11]
DNN Expression Rate Analysis of High-dimensional PDEs: Application to Op tion Pricing
Elbr¨achter, D., Grohs, P., Jentzen, A., and Schwab, C. DNN Expression Rate Analysis of High-dimensional PDEs: Application to Op tion Pricing. arXiv:1809.07669v2 (2018), 50 pages
2018 arXiv
-
[12]
F arahmand, A.-m., Nabi, S., and Nikovski, D. N. Deep reinforcement learning for partial differential equation control. American Control Conference (ACC) (2017), 3120–3127. doi: 10.23919/ACC.2017.7963427
2017
-
[13]
Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High dimensional BSDE s
Fujii, M., Takahashi, A., and Takahashi, M. Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High dimensional BSDE s. Asia-Pacific Financial Markets (Mar 2019). doi: 10.1007/s10690-019-09271-7
2019 doi
-
[14]
Deep Learning
Goodfellow, I., Bengio, Y., and Courville, A. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org
2016
-
[15]
Machine Learning for Pricing American Options in High Dimension
Goudenege, L., Molent, A., and Zanette, A. Machine Learning for Pricing American Options in High Dimension. arXiv:1903.11275 (2019), 11 pages. 84
2019 arXiv
-
[16]
A proof that artificial neural networks overcome the curse of dim ensionality in the numerical approximation of Black-Scholes partial differential e quations
Grohs, P., Hornung, F., Jentzen, A., and von Wurstemberger, P. A proof that artificial neural networks overcome the curse of dim ensionality in the numerical approximation of Black-Scholes partial differential e quations. arXiv:1809.02362 (2018), 124 pages
2018 arXiv
-
[17]
Deep Neural Network Approximation Theory
Grohs, P., Perekrestenko, D., Elbr ¨achter, D., and B ¨olcskei, H. Deep Neural Network Approximation Theory. arXiv:1901.02220 (2019), 60 pages
2019 arXiv
-
[18]
Solving high-dimensional partial differ- ential equations using deep learning
Han, J., Jentzen, A., and E, W. Solving high-dimensional partial differ- ential equations using deep learning. Proceedings of the National Academy of Sciences 115 , 34 (2018), 8505–8510. doi: 10.1073/pnas.1718942115
2018 doi
-
[19]
Convergence of the deep BSDE method for coupled FBSDEs
Han, J., and Long, J. Convergence of the deep BSDE method for coupled FBSDEs. arXiv:1811.01165 (2018), 26 pages
2018 arXiv
-
[20]
Deep Primal-Dual Algorithm for BSDEs: Appli- cations of Machine Learning to CV A and IM
Henry-Labordere, P. Deep Primal-Dual Algorithm for BSDEs: Appli- cations of Machine Learning to CV A and IM. Preprint, SSRN–id3071506 (2017), 16 pages
2017
-
[21]
Some machine learning schemes for high-dimensional nonlinear PDEs
Hur´ e, C., Pham, H., and W arin, X. Some machine learning schemes for high-dimensional nonlinear PDEs. arXiv:1902.01599 (2019), 33 pages
2019
-
[22]
Hutzenthaler, M., Jentzen, A., Kruse, T., and Nguyen, T. A. A proof that rectified deep neural networks overcome the curse of di- mensionality in the numerical approximation of semilinear heat equatio ns. arXiv:1901.10854 (2019), 24 pages
2019 arXiv
-
[23]
Over- coming the curse of dimensionality in the approximative pricing of finan cial derivatives with default risks
Hutzenthaler, M., Jentzen, A., and von Wurstemberger, P. Over- coming the curse of dimensionality in the approximative pricing of finan cial derivatives with default risks. arXiv:1903.05985 (2019), 71 pages
2019 arXiv
-
[24]
Deep PPDEs for rough local stochastic volatility
Jacquier, A., and Oumgari, M. Deep PPDEs for rough local stochastic volatility. arXiv:1906.02551 (2019), 21 pages
2019 arXiv
-
[26]
A theoretical analysis of deep neural networks and parametric PD Es
Kutyniok, G., Petersen, P., Raslan, M., and Schneider, R. A theoretical analysis of deep neural networks and parametric PD Es. arXiv:1904.00377 (2019), 42 pages. 85
2019 arXiv
-
[27]
Better Approximations of High Di- mensional Smooth Functions by Deep Neural Networks with Rectifie d Power Units
Li, B., Shanshan, T., and Yu, H. Better Approximations of High Di- mensional Smooth Functions by Deep Neural Networks with Rectifie d Power Units. arXiv:1903.05858v3 (2019), 21 pages
2019 arXiv
-
[28]
PDE-Net: Learning PDEs from Data
Long, Z., Lu, Y., Ma, X., and Dong, B. PDE-Net: Learning PDEs from Data. arXiv:1710.09668 (2017), 15 pages
2017 arXiv
-
[29]
O., Mishra, S., and Ray, D
Lye, K. O., Mishra, S., and Ray, D. Deep learning observables in computational fluid dynamics. arXiv:1903.03040 (2019), 57 pages
2019 arXiv
-
[30]
Neural Networks Trained to Solve Differential Equations Learn General Representations
Magill, M., Qureshi, F., and de Haan, H. Neural Networks Trained to Solve Differential Equations Learn General Representations. Advances in Neural Information Processing Systems (2018), 4075–4085
2018
-
[31]
Topological prop- erties of the set of functions generated by neural networks of fi xed size
Petersen, P., Raslan, M., and Voigtlaender, F. Topological prop- erties of the set of functions generated by neural networks of fi xed size. arXiv:1806.08459v2 (2018), 56 pages
2018 arXiv
-
[32]
Optimal approximation of piece- wise smooth functions using deep ReLU neural networks
Petersen, P., and Voigtlaender, F. Optimal approximation of piece- wise smooth functions using deep ReLU neural networks. arXiv:1709.05289 (2017), 54 pages
2017 arXiv
-
[33]
Neural networks-based backward scheme for fully nonlinear PDEs
Pham, H., and W arin, X. Neural networks-based backward scheme for fully nonlinear PDEs. arXiv:1908.00412v1 (2019), 15 pages
2019
-
[34]
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
Raissi, M. Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations. arXiv:1801.06637 (2018), 26 pages
2018 arXiv
-
[35]
Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-s um games of nonlinear stiff systems
Reisinger, C., and Zhang, Y. Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-s um games of nonlinear stiff systems. arXiv:1903.06652 (2019), 34 pages
2019 arXiv
-
[36]
DGM: A deep learning algorithm for solving partial differential equations
Sirignano, J., and Spiliopoulos, K. DGM: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics 375 (2018), 1339–1364. doi: 10.1016/j.jcp.2018.08.029
2018 doi
-
[37]
Error bounds for approximations with deep ReLU networks
Yarotsky, D. Error bounds for approximations with deep ReLU networks. Neural Networks 94 (2017), 103–114. doi: 10.1016/j.neunet.2017.07.002. 86
2017 doi
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.