Pith. sign in

REVIEW 3 major objections 5 minor 48 references

Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Stochastic PINN training converges linearly for wide two-layer nets

desk verdict First SGD convergence proof for PINNs, with a load-bearing but honestly flagged assumption on Gram-matrix positivity; the SGD half is solid, the SGF half has a fixable C^4 gap. read the letter →

arxiv 2508.21571 v2 pith:XZ2VMW4J submitted 2025-08-29 cs.LG cs.NAmath.NAstat.ML

Bangti Jin , Longjun Wu This is my paper · ORCID
classification cs.LGcs.NAmath.NAstat.ML MSC 68T0765N1260H10
keywords physics-informedneuralnetworksstochasticgradientdescentconvergenceanalysistangentkernelGrammatrixPoissonequationover-parameterizationPolyak-Lojasiewiczinequality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Stochastic gradient descent (SGD) and stochastic gradient flow (SGF) provably drive the empirical physics-informed neural network (PINN) loss to a global minimum at an exponential rate when a sufficiently wide two-layer network is trained to solve the Poisson equation. This paper supplies the convergence guarantee that was missing for stochastic optimizers, the ones used in practice, and it does so under a mild locally Lipschitz condition on the activation function. The result hinges on two Gram matrices built from the derivatives of the interior residual and boundary losses; their infinite-width limits are assumed to be strictly positive definite, and the proof shows they stay positive definite throughout training when the width is large and the step size is small. If the theorems are right, both discrete SGD and its continuous-time limit converge at the same linear rate, so the expected loss decays as predicted by the eigenvalues of those Gram matrices.

What carries the argument

The argument is carried by the Gram matrices $G_w(w,a)=D_w^\top D_w$ and $G_a(w,a)=D_a^\top D_a$, where $D_w$ and $D_a$ collect the derivatives of the interior residual vector $s$ and the boundary loss vector $h$ with respect to the weight vectors $w_r$ and the outer coefficients $a_r$, and by their infinite-width expectations $G^\infty_w$ and $G^\infty_a$. Assumption 2.2 postulates $\lambda_w = \lambda_{\min}(G^\infty_w)>0$ and $\lambda_a = \lambda_{\min}(G^\infty_a)>0$; the analysis then uses a stopping time $T$ that freezes the parameter deviations $\|w_r(t)-w_r(0)\|$, $|a_r(t)-a_r(0)|$ and the loss at reasonable levels. Inside $T$, Lemma 2.4 shows the finite-width Gram matrices keep eigenvalues at least $\lambda_w/2$ and $\lambda_a/2$, which yields the $\lambda_\theta$-Polyak–Lojasiewicz inequality $\|\nabla L\|^2 \ge \lambda_\theta L$; this inequality converts the gradient-norm bounds into the exponential loss decay. The static randomness of the Gaussian initialization is tamed with a concentration inequality for sums of sub-Weibull random variables, and the dynamic randomness is handled either by conditional-expectation bounds on the SGD iterates or, for SGF, by the exponential martingale and the Hessian trace estimates from Ito's formula.

What would settle it

Take a localized collection of collocation points in a square and a smooth activation allowed by Assumption 2.1 (for instance softplus or a RePU of power at least three), then compute or estimate the smallest eigenvalues of $G^\infty_w$ and $G^\infty_a$ as the width grows; if either eigenvalue vanishes or does not stay bounded away from zero as the sample set changes, the linear convergence claim loses its basis. A second check is to run mini-batch SGD on such a configuration and compare the empirical loss decay with the predicted rate $(1-\eta\lambda_\theta/2)^t$; a systematically slower decay would indicate a failure of Assumption 2.2 or of the uniform positive-definiteness step.

Watch

Extended reading notes

Core claim

The paper establishes Theorem 3.1 for SGD and Theorem 4.1 for SGF: under Assumptions 2.1 and 2.2, once the width $m$ is large enough and the step size $\eta$ is small enough, then with probability at least $1-2\delta$ over the random initialization, the stopping time $T$ is infinite with probability at least $(1-\tilde\delta)^2$ and the expected empirical loss satisfies $\mathbb{E}[L(t)] \leq (1-\eta\lambda_\theta/2)^t L(0)$ for all $t$ (SGD), and $\mathbb{E}[L(t)] \leq \exp(-\lambda_\theta t/2) L(0)$ (SGF), where $\lambda_\theta = \lambda_w + \lambda_a$ is the sum of the smallest eigenvalues of the two infinite-width Gram matrices. In other words, stochastic training of over-parameterized two-layer PINNs for a linear elliptic problem reaches the zero-loss regime at a guaranteed geometric rate, with the spectral gaps of the residual and boundary Gram matrices as the sole drivers of the convergence speed. The proof tracks the parameter deviations and the loss inside a carefully chosen stopping time, controls the initialization randomness with a sub-Weibull concentration inequality, and for the flow uses Ito's formula together with a strengthened Polyak–Lojasiewicz inequality to control the logarithm of the loss.

Load-bearing premise

The load-bearing premise is that the infinite-width Gram matrices for both the residual and boundary losses are strictly positive definite at initialization, with smallest eigenvalues $\lambda_w$ and $\lambda_a$; the paper assumes this condition instead of proving it for its general activation class, and the claimed linear rate is exactly the sum of those two eigenvalues.

Editorial extensions

If this is right

  • Mini-batch SGD reaches an $\epsilon$-accurate empirical loss in $O((1/\eta\lambda_\theta)\log(1/\epsilon))$ iterations, the same order as full-batch gradient descent but at lower per-iteration cost.
  • The width requirement grows polynomially in $\log(1/\delta)$ and inversely in the smallest Gram eigenvalues, quantifying how much over-parameterization is needed to absorb the noise of stochastic sampling.
  • Discrete SGD and continuous stochastic gradient flow share the same exponential decay rate $\lambda_\theta/2$, so the SDE model is a faithful continuous-time surrogate for the discrete analysis.
  • The conclusions hold for every activation that satisfies the locally Lipschitz condition of Assumption 2.1—including smooth activations and RePU with power at least three—provided the infinite Gram matrices are positive definite, which goes beyond the ReLU$^3$ setting of the earlier gradient-descent analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: because the proof uses only the linearity of the PDE through the affine-in-output residual, the same stopping-time argument should extend to other linear second-order elliptic and parabolic equations, e.g. the Helmholtz or heat equation, as long as the corresponding infinite-width Gram matrices are positive definite.
  • Inference: the rate identity $\lambda_\theta = \lambda_w + \lambda_a$ suggests a practical pre-training diagnostic: compute these two smallest eigenvalues for a proposed collocation set and activation; if either is tiny, training will be slow regardless of optimizer, and spectral enrichment of the samples could speed up convergence.
  • Inference: a natural empirical test of Theorem 3.1 is to measure the loss trajectories for widths $m$ and step sizes $\eta$ on either side of the thresholds in (16) and check that the exponent in the decay closely matches $\log(1-\eta\lambda_\theta/2)$, which would both support the theorem and reveal the sharpness of the constants.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies the training of a two-layer physics-informed neural network for the Poisson equation with Dirichlet boundary conditions. The empirical loss (Eq. (3)) is the sum of interior PDE-residual and boundary terms, and the network is trained by mini-batch SGD and by a continuous stochastic gradient flow (SGF). Under Assumption 2.1 (piecewise C^3 locally Lipschitz activations with polynomial growth of the local Lipschitz constants, Gaussian weights and Rademacher outputs at initialization) and Assumption 2.2 (strict positive definiteness of the expected initial Gram matrices G∞_w and G∞_a, with spectral gaps λ_w and λ_a), the paper claims linear convergence in expectation: Theorem 3.1 gives E[L(t)] ≤ (1−ηλθ/2)^t L(0) for SGD, and Theorem 4.1 gives E[L(t)] ≤ exp(−λθ t/2)L(0) for SGF, where λθ = λ_w + λ_a. The proofs follow the NTK/lazy-training template: sub-Weibull concentration inequalities control the initial loss and Gram matrices, a stopping time keeps the weights within a radius where the Gram matrices have a uniform positive lower bound, and exponential-martingale arguments handle the SGF noise.

Significance. If the results hold, this is a valuable contribution: it extends the deterministic gradient-descent analysis of Gao-Gu-Ng for ReLU^3 activations and the implicit-gradient-descent analysis of Xu et al. to stochastic algorithms, and it is among the first convergence guarantees for SGD in PINN training. The rates are explicit and depend only on the spectral gaps of infinite Gram matrices plus universal constants; no parameter is fitted to data. The appendix is detailed and largely self-contained, with careful uniform-in-width parameter control and stopping-time arguments, and these technical components are genuine strengths. The main caveats are that Assumption 2.2 is not proved for the advertised activation class and that the SGF proof needs C^4 regularity while Assumption 2.1 only provides C^3; both points affect the exact scope of the theorems and should be resolved before publication.

major comments (3)
  1. [Assumption 2.2 and Remark 2.2] Assumption 2.2 is the load-bearing premise: the contraction factors in Theorem 3.1 and Theorem 4.1 are exactly (1−ηλθ/2)^t and exp(−λθ t/2) with λθ = λw + λa, so if λw = 0 or λa = 0 for an allowed activation, no linear convergence follows. The paper, however, does not prove positive definiteness of G∞_w and G∞_a for the general class in Assumption 2.1. Remark 2.2 only sketches a linear-independence argument and cites [18,46], which cover ReLU^3 and nonpolynomial analytic three-times globally Lipschitz activations. The assumptions here are weaker: piecewise C^3 locally Lipschitz activations may be locally but not globally Lipschitz and may have unbounded growth of derivatives, e.g. σ(z) = z^3 + tanh(z). The authors should either prove positivity of the infinite Gram matrices for their full Assumption 2.1 class or explicitly narrow the scope of Theorems 3.1 and 4.1 to a class for which the cited arguments apply.
  2. [Section 4 and Appendix C.2 (Lemmas 4.1 and 4.3)] The stochastic-gradient-flow proof requires the fourth derivative of the activation. Assumption 2.1 only assumes σ is piecewise C^3 with a locally Lipschitz third derivative, but Lemma 4.3 and Appendix C.2 bound the Hessian using σ^(4). The paper itself acknowledges this in the opening paragraph of Section 4, yet Theorem 4.1 is still stated under Assumptions 2.1 and 2.2 without the extra regularity. This is fixable by adding a C^4 assumption (or a uniform bound on σ^(4)) for the SGF theorem, or by proving the Hessian bound with only C^3 data; as written, the theorem and its proof do not match.
  3. [Theorems 3.1 and 4.1, final displays] The conclusions are stated as unconditional expectations: 'E[L(t)] ≤ ...' for all t. The proofs, however, establish the decay only under the stopping time T = ∞ and on a high-probability initialization event; if T < ∞, the loss is no longer controlled by the proof. The statements should either condition the expectation on the event {T = ∞} / the high-probability initialization event, or provide an upper bound for the contribution of the failure event. Otherwise the displayed inequalities are stronger than what the argument establishes.
minor comments (5)
  1. [Abstract and reference list] The abstract says the result extends the existing GD analysis in reference [20], while the main text attributes this analysis to reference [18]; please make the reference numbering consistent.
  2. [Theorem 4.1] The theorem states 'for all t ∈ N' but the SGF model evolves in continuous time; the conclusion should be stated for t ≥ 0.
  3. [Appendix C.2, Hessian bound] The displayed estimates for the second derivatives are derived under the condition 'if B ≥ d', but Lemma 4.3 and Theorem 4.1 do not state this condition. Since B grows with m and d is fixed this is harmless asymptotically, but the statement should be made precise.
  4. [Equations (12)–(14)] The notation I and J is used both for the random index sets and for their cardinalities; please clarify explicitly that the subsets are drawn without replacement (if that is intended) and that the displayed unbiasedness uses the fixed sizes I = |I| and J = |J|.
  5. [Remark 2.2 and Section 5] The paper repeatedly advertises 'very generous assumptions on the activation function' and 'a general class of activation functions', but the positivity of the infinite Gram matrices is a nontrivial assumption that is not established for that class. The wording should be qualified accordingly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the SGD and SGF rates are conditional consequences of the explicitly assumed spectral gap lambda_theta, with no fitted input or self-citation reduction.

full rationale

The derivation chain is conditional rather than circular. Assumption 2.2 defines the infinite Gram matrices as the initialization expectation of the finite Gram matrices and postulates lambda_w, lambda_a > 0; Lemma 2.3 then shows by concentration that the finite initial Gram matrices inherit a positive lower bound, Lemma 2.4 gives a continuity bound that keeps the lower bound during training, and Lemmas 3.2 and 4.4 convert that lower bound into the claimed contraction factors (1 - eta*lambda_theta/2)^t L(0) and exp(-lambda_theta*t/2)L(0). The rate lambda_theta = lambda_w + lambda_a is an explicit input parameter of the theorem, not a quantity fitted from data or defined in terms of the decay it predicts, so the theorems do not reduce to their inputs by construction. The paper also does not rely on a load-bearing self-citation: the positivity citations [18,46] are by other author groups, and the proof of Assumption 2.2 for the paper's full activation class is explicitly only sketched in Remark 2.2. Two genuine non-circular weaknesses should be weighed: (i) Assumption 2.2 is not proved for the locally Lipschitz piecewise C^3 class of Assumption 2.1, and Section 5 concedes that the precise characterization of the smallest eigenvalues is 'still largely missing'; and (ii) the SGF analysis in Section 4 and Lemma 4.3 uses sigma^(4) although Assumption 2.1 only states C^3. Both are support gaps in the hypothesis-to-theorem chain, not circular reductions, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted to data and no new entities are introduced. The results depend on two unproved inputs: the positive-definiteness of the infinite Gram matrices and, for SGF, a fourth-order smoothness condition that is absent from Assumption 2.1. These are listed as axioms rather than derived facts.

assumptions (4)
  • domain assumption Activation smoothness (Assumption 2.1): σ is piecewise C^3 with locally Lipschitz derivatives up to order 3 and polynomially growing Lipschitz constants.
    All derivative bounds in the proofs rely on this. For the SGF theorem, Appendix C uses σ^(4), implying the assumption is implicitly strengthened; this mismatch is recorded as a red flag.
  • ad hoc to paper Infinite Gram positive definiteness (Assumption 2.2): G∞_w and G∞_a are positive definite with λw, λa > 0.
    The paper does not prove this for the general activation class and only points to [18,46], which cover ReLU^3 and nonpolynomial analytic activations. The convergence rate is proportional to λw + λa, making this assumption load-bearing.
  • domain assumption Bounded and uniformly sampled collocation points: ∥x_p∥, ∥y_q∥ ≤ 1, sampled i.i.d. from uniform distributions on Ω and ∂Ω.
    Used to bound extended sample norms (≤√2) and to apply concentration inequalities. Standard for NTK analyses of sampled losses.
  • domain assumption Existence/uniqueness of the SGF solution: the SDE (17) has a unique maximal local solution up to its random blow-up time (Section 4).
    Stated as a blanket assumption; the paper does not construct the solution. Standard SDE theory may cover it, but it is not verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation." pith.science (2026). https://pith.science/paper/XZ2VMW4J

@misc{pith2026250821571,
  author       = {Pith},
  title        = {Pith review of: Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XZ2VMW4J}},
  note         = {Machine review of arXiv:2508.21571}
}
read the original abstract

Physics informed neural networks (PINNs) represent a very popular class of neural solvers for partial differential equations. In practice, one often employs stochastic gradient descent type algorithms to train the neural network. Therefore, the convergence guarantee of stochastic gradient descent is of fundamental importance. In this work, we establish the linear convergence of stochastic gradient descent / flow in training over-parameterized two layer PINNs with a general class of activation functions for solving one model second-order elliptic problem, i.e., the Poisson equation, in the sense of high probability. These results extend the existing result [20] in which gradient descent was analyzed. The challenge of the analysis lies in handling the dynamic randomness introduced by stochastic optimization methods. The key of the analysis lies in ensuring the positive definiteness of suitable Gram matrices during the training. The analysis sheds insight into the dynamics of the optimization process, and provides guarantees on physics informed neural networks trained by stochastic algorithms.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 36 canonical work pages

  1. [1]

    Allen-Zhu, Y

    Z. Allen-Zhu, Y. Li, and Z. Song. A convergence theory for deep learning via over-parameterization. In International Conference on Machine Learning , pages 242–252. PMLR, 2019

    work page 2019

  2. [2]

    An and J

    J. An and J. Lu. Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks. Preprint arXiv:2304.09221, 2023

    arXiv 2023

  3. [3]

    C. Beck, M. Hutzenthaler, A. Jentzen, and B. Kuckuck. An overview on deep learning-based approximation methods for partial differential equations. Discrete Contin. Dyn. Syst. Ser. B , 28(6):3697–3746, 2023

    work page 2023

  4. [4]

    A. Bihlo. Improving physics-informed neural networks with meta-learned optimization. J. Mach. Learn. Res., 25:14, 26, 2024

    work page 2024

  5. [5]

    Bonfanti, G

    A. Bonfanti, G. Bruno, and C. Cipriani. The challenges of the nonlinear regime for physics-informed neural networks. Advances in Neural Information Processing Systems , 37:41852–41881, 2024

    work page 2024

  6. [6]

    R. H. Byrd, P. Lu, J. Nocedal, and C. Y. Zhu. A limited memory algorithm for bound constrained opti- mization. SIAM J. Sci. Comput. , 16(5):1190–1208, 1995

    work page 1995

  7. [7]

    Cao, Z.-C

    D. Cao, Z.-C. Guo, and L. Shi. Stochastic gradient descent for two-layer neural networks. Preprint, arXiv:2407.07670, 2024

    arXiv 2024

  8. [8]

    S. Cen, B. Jin, X. Li, and Z. Zhou. Imaging anisotropic conductivity from internal measurements with mixed least-squares deep neural networks. J. Comput. Phys. , 523:113648, 25, 2025

    work page 2025

Show all 48 references
  1. [9]

    Chatterjee

    S. Chatterjee. Convergence of gradient descent for deep neural networks. Preprint, arXiv:2203.16462, 2022

    2022

  2. [10]

    F. Chen, J. Huang, C. Wang, and H. Yang. Friedrichs learning: Weak solutions of partial differential equations via deep learning. SIAM J. Sci. Comput. , 45(3):A1271–A1299, 2023

    2023

  3. [11]

    Chizat, E

    L. Chizat, E. Oyallon, and F. Bach. On lazy training in differentiable programming. Advances in Neural Information Processing Systems, 32, 2019

    2019

  4. [12]

    Cuomo, V

    S. Cuomo, V. S. Di Cola, F. Giampaolo, G. Rozza, M. Raissi, and F. Piccialli. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. J. Sci. Comput., 92(3):88, 2022

    2022

  5. [13]

    De Ryck and S

    T. De Ryck and S. Mishra. Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning. Acta Numer., 33:633–713, 2024

    2024

  6. [14]

    S. Du, J. Lee, H. Li, L. Wang, and X. Zhai. Gradient descent finds global minima of deep neural networks. In International Conference on Machine Learning , pages 1675–1685. PMLR, 2019

    2019

  7. [15]

    S. S. Du, X. Zhai, B. Poczos, and A. Singh. Gradient descent provably optimizes over-parameterized neural networks. In International Conference on Learning Representations, 2019

    2019

  8. [16]

    W. E and B. Yu. The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Comm. Math. Stat. , 6(1):1–12, 2018

    2018

  9. [17]

    Eivazi, M

    H. Eivazi, M. Tahani, P. Schlatter, and R. Vinuesa. Physics-informed neural networks for solving Reynolds- averaged Navier–Stokes equations. Phys. Fluids, 34:075117, 2022

    2022

  10. [18]

    Y. Gao, Y. Gu, and M. Ng. Gradient descent finds the global optima of two-layer physics-informed neu- ral networks. In Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research, pages 10676–10707. PMLR, 2023

    2023

  11. [19]

    J. Han, A. Jentzen, and W. E. Solving high-dimensional partial differential equations using deep learning. Proc. Nat. Acad. Sci., 115(1):123–128, 2018

    2018

  12. [20]

    J. Han, J. Lu, and M. Zhou. Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion monte carlo like approach. J. Comput. Phys. , 423:109792, 2020

    2020

  13. [21]

    T. Hu, B. Jin, and Z. Zhou. Solving Poisson problems in polygonal domains with singularity enriched physics informed neural networks. SIAM J. Sci. Comput. , 46(4):C369–C398, 2024

    2024

  14. [22]

    Jacot, F

    A. Jacot, F. Gabriel, and C. Hongler. Neural tangent kernel: convergence and generalization in neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems , NIPS’18, pages 8580–8589, Red Hook, NY, USA, 2018

    2018

  15. [23]

    B. Jin, X. Li, Q. Quan, and Z. Zhou. Conductivity imaging from internal measurements with mixed least- squares deep neural networks. SIAM J. Imaging Sci. , 17(1):147–187, 2024

    2024

  16. [24]

    X. Jin, S. Cai, H. Li, and G. E. Karniadakis. NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. J. Comput. Phys. , 426:109951, 2021

    2021

  17. [25]

    Kazamaki

    N. Kazamaki. Continuous Exponential Martingales and BMO . Springer-Verlag, Berlin, 1994

    1994

  18. [26]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference for Learning Representations, San Diego, 2015

    2015

  19. [27]

    A. K. Kuchibhotla and A. Chakrabortty. Moving beyond sub-Gaussianity in high-dimensional statistics: applications in covariance estimation and linear regression. Inf. Inference, 11(4):1389–1456, 2022. 11

    2022

  20. [28]

    I. E. Lagaris, A. Likas, and D. I. Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Networks , 9(5):987–1000, 1998

    1998

  21. [29]

    Q. Li, C. Tai, and W. E. Stochastic modified equations and dynamics of stochastic gradient algorithms I: Mathematical foundations. J. Mach. Learn. Res. , 20:1–47, 2019

    2019

  22. [30]

    C. Liu, L. Zhu, and M. Belkin. Loss landscapes and optimization in over-parameterized non-linear systems and neural networks. Appl. Comput. Harmonic Anal. , 59:85–116, 2022

    2022

  23. [31]

    Lugosi and E

    G. Lugosi and E. Nualart. Convergence of continuous-time stochastic gradient descent with applications to linear deep neural networks. Preprint, arXiv:2409.07401, 2024

    2024

  24. [32]

    Luo and H

    T. Luo and H. Yang. Two-layer neural networks for partial differential equations: optimization and gen- eralization theory. In Handbook of Numerical Analysis, Volume 25 , pages 515–554. Elsevier, Amsterdam, 2024

    2024

  25. [33]

    L. Mirsky. A trace inequality of John von Neumann. Monatsh. Math., 79(4):303–306, 1974

    1974

  26. [34]

    Mishra and R

    S. Mishra and R. Molinaro. Estimates on the generalization error of physics-informed neural networks for approximating pdes. IMA J. Numer. Anal. , 43(1):1–43, 2023

    2023

  27. [35]

    Nguyen, M

    Q. Nguyen, M. Mondelli, and G. F. Montufar. Tight bounds on the smallest eigenvalue of the neural tangent kernel for deep ReLU networks. InInternational Conference on Machine Learning, pages 8119–8129. PMLR, 2021

    2021

  28. [36]

    Nießen and J

    J. Nießen and J. M¨ uller. Non-asymptotic analysis of projected gradient descent for physics-informed neural networks. Preprint, arXiv:2505.07311, 2025

    2025 arXiv

  29. [37]

    Raissi, P

    M. Raissi, P. Perdikaris, and G. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.J. Comput. Phys., 378:686–707, 2019

    2019

  30. [38]

    Rigollet and J.-C

    P. Rigollet and J.-C. H¨ utter. High-dimensional statistics. Preprint, arXiv:2310.19244, 2023

    2023 arXiv

  31. [39]

    Sirignano and K

    J. Sirignano and K. Spiliopoulos. DGM: A deep learning algorithm for solving partial differential equations. J. Comput. Phys. , 375:1339–1364, 2018

    2018

  32. [40]

    D. N. Tanyu, J. Ning, T. Freudenberg, N. Heilenk¨ otter, A. Rademacher, U. Iben, and P. Maass. Deep learning methods for partial differential equations and related parameter identification problems. Inverse Problems, 39(10):103001, 2023

    2023

  33. [41]

    J. F. Urb´ an, P. Stefanou, and J. A. Pons. Unveiling the optimization process of physics informed neural networks: how accurate and competitive can PINNs be? J. Comput. Phys. , 523:113656, 28, 2025

    2025

  34. [42]

    Vladimirova, S

    M. Vladimirova, S. Girard, H. Nguyen, and J. Arbel. Sub-weibull distributions: Generalizing sub-Gaussian and sub-exponential properties to heavier tailed distributions. Stat, 9(1):e318, 2020

    2020

  35. [43]

    S. Wang, H. Wang, and P. Perdikaris. On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks. Comput. Methods Appl. Mech. Eng. , 384:113938, 2021

    2021

  36. [44]

    S. Wang, X. Yu, and P. Perdikaris. When and why PINNs fail to train: A neural tangent kernel perspective. J. Comput. Phys. , 449:110768, 2022

    2022

  37. [45]

    X. Xu, T. Du, W. Kong, Y. Li, and Z. Huang. Convergence analysis of natural gradient descent for over- parameterized physics-informed neural networks. Preprint, arXiv:2408.00573v2, 2024

    2024 arXiv

  38. [46]

    X. Xu, T. Du, W. Kong, Y. Li, and Z. Huang. Convergence of implicit gradient descent for training two-layer physics-informed neural networks. Preprint, arXiv:2407.02827, 2024

    2024 arXiv

  39. [47]

    Y. Zang, G. Bao, X. Ye, and H. Zhou. Weak adversarial networks for high-dimensional partial differential equations. J. Comput. Phys. , 411:109409, 2020

    2020

  40. [48]

    s(t + 1) h(t + 1) − s(t) h(t) 2 2 · 1T>t|F t # ≤c2η2C4 BB12L(t). In summary, we obtain E [2L(t + 1)· 1T>t+1|F t] =E

    H. Zhang and J. Liu. Solving an inverse source problem by deep neural network method with convergence and error analysis. Inverse Problems, 39(7):075013, 2023. Appendix A. Proofs for section 2 A.1. Preliminary estimates. First we estimate the derivatives of the loss and then g...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.