Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

Non-Asymptotic Analysis of Projected Gradient Descent for Physics-Informed Neural Networks

T0 review · 4 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves a non-asymptotic convergence guarantee for projected gradient descent on physics-informed neural networks solving the Poisson equation, at a rate $O(1/\sqrt T + 1/\sqrt m + \epsilon_p)$, without over-parameterization.

desk verdict Right question, coherent strategy - but Lemma 4's linearization bound is off by a factor of m, so Theorem 6 is not established as written; the paper deserves referees and a serious revision. read the letter →

arxiv 2505.07311 v1 pith:RT2MKFUN submitted 2025-05-12 math.OC

Jonas Nie{ss}en , Johannes Müller This is my paper · ORCID
classification math.OC MSC 68T0765N12
keywords physics-informedneuralnetworkstangentkernelreproducingHilbertspaceprojectedgradientdescentPoissonequationnon-asymptoticconvergenceRademachercomplexityunder-parameterizedregime
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proves a finite-time convergence guarantee for projected gradient descent on a one-hidden-layer physics-informed neural network (PINN) trained to solve the Poisson equation. The main theorem bounds the time-averaged empirical loss by $O(1/\sqrt T + 1/\sqrt m + \epsilon_p)$ with high probability over the random initialization, where $T$ is the number of gradient steps, $m$ is the network width, and $\epsilon_p$ is the error of approximating the true solution by functions in a neural-tangent-kernel space with bounded transport map. The result does not require the network width to be much larger than the number of collocation points, and it permits fresh integration points at every iteration — the setting in which PINNs are usually trained. A companion generalization bound, built from Rademacher complexities of the network and its Laplacian, converts the loss bound into an overall $H^{1/2}$ error estimate. The authors present this as the first non-over-parameterized optimization guarantee for PINNs.

What carries the argument

The argument is carried by the Lyapunov potential $L(\theta)=\sum_{i=1}^m\|\theta_i-\bar\theta_i\|_2^2$, where $\bar\theta$ reconstructs an oracle approximant in $F_p$ from its transport map; nonexpansivity of the projection gives the drift inequality. Two bounds feed into it. Lemma 4 (the linearization error) states that inside the projection ball, the network and its Laplacian differ from their first-order Taylor expansions at initialization by at most $C\|x\|_2^2p^2/\sqrt m$, uniformly in $\theta$. Lemmas 5 and 12 use the random features $\Phi_{\mathrm{NTK}}(x;\theta)=x\sigma'(\theta^T x)$ to show that the linearized model approximates $\Delta u^*$ and the boundary trace of $u^*$ up to $O(\sqrt{\log(1/\delta)/m})$ plus the approximation error $\epsilon_p$. The symmetric initialization makes $F(x;\theta(0))=\Delta F(x;\theta(0))=0$, so the linear model starts at the zero function, and the projection keeps the gradient norm bounded while making the linearization error uniform.

What would settle it

Take a smooth activation satisfying Assumption 2, set $x=(1,\ldots,1)$, and put every weight on the boundary of its projection ball, $\theta_i=\theta_i(0)+p\sqrt m\,e_1$ for all $i$. For a sequence of widths $m=10^2,10^3,\ldots,10^6$, compute the normalized remainders $R_m=\sqrt m\,|\Delta F(x;\theta)-\sum_i\nabla_{\theta_i}\Delta F(x;\theta(0))^T(\theta_i-\theta_i(0))|$ and the analogous $F$-remainder. Lemma 4 predicts $R_m$ stays bounded uniformly in $m$ and in the direction of $\theta-\theta(0)$; a naive second-order Taylor estimate grows with $m$. If $R_m$ diverges, the linearization step in the proof of Theorem 6 is unsupported.

Watch

Extended reading notes

Core claim

At the center of the paper is Theorem 6. For a network $F(x;\theta)=m^{-1/2}\sum_{i=1}^m c_i\sigma(\theta_i^T x)$ with a smooth activation whose first four derivatives are bounded, symmetric initialization, and projection onto balls of radius $p\sqrt m$ around the initial weights, projected gradient descent with step size $\eta=1/\sqrt T$ satisfies, for any choice of integration sets $S_t$, $$\frac{1}{T}\sum_{t=0}^{T-1}E_{S_t}(\$\theta$(t)) \le \frac{C_1}{\sqrt T}+C_2\sqrt{\frac{\log(\max\{b_\$\Omega$,b_{\partial\$\Omega$}\}/\delta)}{m}}+C_3\epsilon_p$$ with probability at least $1-2\delta$ over the random initialization. Here $\epsilon_p$ is the best error with which $u^*$ can be represented by functions in the class $F_p$, the neural-tangent-RKHS ball of radius $p$. The paper then proves a Rademacher-complexity bound for the network and its Laplacian, giving a generalization bound for the best empirical-loss iterate, and combines both with the regularity estimate $\|u^*-F\|_{H^{1/2}}\lesssim\sqrt{E(\theta)}$ to obtain the overall error in Theorem 8. The intended contribution is to move PINN optimization theory from the regime of quadratic over-parameterization to the regime of arbitrary width and online sampling.

Load-bearing premise

Everything rests on the uniform linearization bound in Lemma 4: inside the projection ball of radius $p\sqrt m$, both the network and its Laplacian stay within $O(1/\sqrt m)$ of their first-order Taylor expansions at initialization, and if that bound is false, the drift inequality that produces Theorem 6 has no foundation.

Editorial extensions

If this is right

  • Because Theorem 6 holds for arbitrary integration sets $S_t$, a practitioner may re-sample collocation points at every gradient step and still have the same $1/\sqrt T + 1/\sqrt m$ guarantee; the proof does not need a fixed dataset.
  • The absence of over-parameterization means the width $m$ can be smaller than the number of collocation points — the regime reached as soon as the sampling density exceeds the network width — with the rate degrading only as $1/\sqrt m$.
  • When the solution $u^*$ is realizable, so $u^*\in F_\alpha$ for some $\alpha$, choosing $p\ge\alpha$ makes the approximation term $\epsilon_p$ vanish and yields an unconditional $O(1/\sqrt T + 1/\sqrt m)$ empirical-loss rate; the constants grow polynomially with $p$.
  • With iid uniform collocation points, the best-iterate generalization error is bounded by $C_4/\sqrt m + C_5\sqrt{\log(1/\delta)/\min\{b_\Omega,b_{\partial\Omega}\}}$, and combining this with Theorem 6 and the regularity estimate gives the overall $H^{1/2}$ error bound of Theorem 8.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I read the projection as the actual mechanism of the result: it confines the trajectory to the region where the uniform linearization bound holds. A testable consequence is that unprojected gradient descent should lose the promised rate whenever the weights leave the ball of radius $p\sqrt m$; comparing projected and unprojected runs with identical initialization and step size would show this.
  • The proof structure is generic in the PDE: the Poisson equation enters only through the Laplacian in the residual and a boundary trace term, and Assumption 2 only controls four derivatives of the activation. I would expect the same Lyapunov argument to carry over to other linear elliptic equations with the same second-order structure, with $\epsilon_p$ redefined for the corresponding differential
  • An empirical stress test of Lemma 4 would be to compute the normalized linearization remainder over the boundary of the projection ball for growing width. If it is not uniform in the direction of $\theta-\theta(0)$, the convergence claim might still hold, but the paper's proof would need a different linearization estimate.
  • The output weights are frozen at their Rademacher initialization; training both layers would require a different potential and a different function class, since $F_p$ is defined through hidden-weight transport maps only. Extending the guarantee to joint training is a natural next step not covered here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper analyzes projected gradient descent for two-layer physics-informed neural networks solving the Poisson equation. It claims a high-probability, non-asymptotic bound on the time-averaged empirical PINN loss of the form O(1/sqrt(T) + 1/sqrt(m) + epsilon_p), where epsilon_p is the approximation error of the true solution by functions in a neural-tangent-kernel RKHS class with bounded transport map. The proof strategy combines a Lyapunov drift argument, a uniform linearization error bound under a projection ball, approximation estimates for the linearized network, and Rademacher-complexity bounds for the network and its Laplacian. These results are then combined with a regularity-theory estimate to obtain an overall H^{1/2} error bound. The paper also reports a small numerical experiment on a two-dimensional Poisson problem.

Significance. If the main theorem were correct, the paper would make a significant contribution: it would provide one of the first non-over-parameterized optimization guarantees for PINNs, with explicit rates and without fitted constants. The use of an externally defined approximation error epsilon_p, rather than a circularly fitted quantity, is a genuine strength, as is the attempt to treat freshly sampled quadrature points. The Rademacher-complexity analysis for the Laplacian of the network is also a natural and potentially useful ingredient. However, the central proof contains multiple scaling errors, including a false uniform linearization bound, and these errors affect the main theorem, the generalization proposition, and the overall error theorem.

major comments (4)
  1. [Section 3.2, Lemma 4 (Eq. (19))] The uniform linearization bound in Eq. (19) is false under the stated projection radius. Since theta_i is only constrained to B_2(theta_i(0), p*sqrt(m)), the squared displacement |theta_ij - theta_ij(0)|^2 can be O(p^2 m), not O(1/m) as the proof marks in the appendix. Concrete counterexample: take d=1, sigma(t)=sin(t), theta_i(0)=0 for all i, and symmetric output weights c_i = +/-1/sqrt(m). Move only one neuron by setting theta_1 = p*sqrt(m), leaving its symmetric partner at 0. Then F(.;theta(0))=0 and Delta F(.;theta(0))=0, and the theta-gradient of Delta F at theta(0) is also zero, so the left-hand side of Eq. (19) at x=1 equals p^2 sqrt(m) |sin(p sqrt(m))|, while the right-hand side is C p^2/sqrt(m). For large m the inequality fails. The appendix proof's step '(theta_ij - theta_ij(0))^2 ≾ 1/m' is exactly where the argument breaks. Since Lemma 4 is the mechanism that converts the gradient inner product into 2 E_{S_t}(theta(t)) in the lower-bound part of the proof of Theorem 6, the Lyapunov drift inequality and the claimed rates do not follow.
  2. [Section 3.3, proof of Theorem 6, Eqs. (27)-(28)] The upper bound on the quantity (*) loses a factor of m. The per-neuron bound just before Eq. (27) gives ||grad_{theta_i} E_{S_t}(theta(t))||_2 <= c_4 + c_5 sqrt(m). The quantity (*) is the sum over i=1,...,m of the squares of these norms, so it is at most m(c_4 + c_5 sqrt(m))^2, not (c_4 + c_5)^2 as written in Eq. (28). This is not a cosmetic issue: after summation with eta = 1/sqrt(T), the term (1/4) eta^2 (*) would contribute O(m^2/sqrt(T)), which destroys the claimed C_1/sqrt(T) term independent of m.
  3. [Appendix, Lemma 12, Eq. (33)] Lemma 12 is internally inconsistent: the displayed computation for Eq. (33) ends with '<= p^2 C_Omega^2 sigma_2 sqrt(m)', which contradicts the lemma's claim that the quantity is ≾ 1/sqrt(m). Each of the m summands inside the outer sum is O(p^2/sqrt(m)) after the intermediate bounds, so the total is O(p^2 sqrt(m)) for fixed p. This estimate is used in the lower-bound step of Theorem 6 to control the difference between the gradient inner product at theta(t) and at theta(0); with an O(sqrt(m)) remainder, the conversion into 2 E_{S_t}(theta(t)) up to O(1/sqrt(m)) terms is not valid. The second estimate (34) is asserted with a bare appeal to Lemma 14 and has the same scaling risk.
  4. [Section 3.3, final display of the proof of Theorem 6] The claimed 1/sqrt(T) term is also not justified by the initial-potential term. With the reference weight theta chosen by Eq. (18), ||theta_i(0) - theta_i||_2 = ||v(theta_i(0))||_2 <= p, so L(theta(0)) = sum_i ||theta_i(0)-theta_i||_2^2 <= m p^2. Since eta = 1/sqrt(T), the term L(theta(0))/(4 eta T) is O(m p^2/sqrt(T)), not O(1/sqrt(T)) with a constant independent of m. The final displayed inequality introduces an undefined symbol 'alpha^2' for this term, and no m-independent bound is supplied.
minor comments (3)
  1. [Section 3.2, Lemma 5] The lemma states that the bounds hold 'for all x in Omega' with probability at least 1-delta, but the proof only applies Hoeffding's inequality pointwise in x. Some uniform covering or continuity argument is needed to justify the simultaneous statement over the uncountable set Omega.
  2. [Section 4, proof of Proposition 9] The sentence 'In the proof of Theorem 6 we have shown that ||grad_{theta_i} Delta F(x;theta(t))||_2 ≾ 1/sqrt(m)' is contradicted by the actual calculation in the proof of Theorem 6, which gives O(m + sqrt(m)) for general theta(t) in the projection ball. If the Rademacher argument only needs this gradient at theta(0), the statement should say so explicitly.
  3. [Section 3.4, numerical example] The experimental section does not state the truncation parameter a used for the initialization distribution, nor the precise initialization of the output weights, so the reported numerical results are not fully reproducible from the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction: the bounds are stated in terms of external quantities (T, m, epsilon_p) and standard regularity estimates; the only self-citation is non-load-bearing.

full rationale

The derivation chain is self-contained with respect to circularity. Theorem 6 (Eq. 24) bounds the time-averaged empirical loss by C1/sqrt(T) + C2 sqrt(log(max{b_Omega,b_dOmega}/delta)/m) + C3 epsilon_p, where epsilon_p is defined in Eq. (23) as the infimum over the NTK-RKHS class F_p of an L-infinity-type approximation error of the true solution u*. This epsilon_p is an external function-approximation quantity: it is not estimated from the empirical losses produced by Algorithm 1, and no constant in the bound is fitted to the data. The proof decomposes the drift into a linearization error (Lemma 4), a Monte Carlo approximation error (Lemma 5), and a gradient-drift term (Lemma 12); each is stated as an independent bound, and the Lyapunov argument combines them without using the desired bound as an input. Lemma 5 is a concentration statement about the random features at initialization, independent of the training trajectory. The total-error theorem uses [35, Theorem 8] to convert the exact PINN loss into an H^{1/2} error; although [35] is co-authored by one of the present authors, the same regularity estimate is also cited to [20,23,47], is externally checkable, and does not depend on the fitted values or on the optimization theorem, so the self-citation is not load-bearing. The numerical experiment is illustrative rather than a fitted prediction. The skeptical concern that Lemma 4's uniform linearization bound may be quantitatively wrong (displacements of size p*sqrt(m) are not of order 1/sqrt(m)) is a correctness/rigor issue; even if valid, it would make the theorem unproven, not circular, because Lemma 4 is not defined in terms of the theorem's conclusion. The explicitly deferred proof of Lemma 13 is likewise a rigor gap, not a circular reduction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the stated regularity of the activation, the symmetric initialization, uniform sampling for generalization, and the RKHS approximation class. There are no fitted free parameters or invented entities; the main burden is carried by the linearization-error estimate, which is where the proof appears to fail.

assumptions (5)
  • domain assumption The activation sigma has globally bounded derivatives up to fourth order (Assumption 2).
    Used throughout for the linearization and Rademacher estimates; excludes ReLU and powers of ReLU.
  • standard math The domain is C^{2,beta} and f in C^beta, so the Poisson problem has a unique C^{2,beta} solution and the loss controls the H^{1/2} error via regularity theory (Eq. (4) from [35]).
    Imported from classical PDE regularity and the cited regularity estimate.
  • ad hoc to paper Symmetric initialization: theta_{i+m/2}(0)=theta_i(0) and c_{i+m/2}=-c_i with c_i ~ +/-1/sqrt(m), so F(theta(0))=0 and Delta F(theta(0))=0.
    Made explicitly to simplify the convergence analysis; the paper acknowledges both restrictions are artificial.
  • domain assumption For the generalization part, integration points are sampled iid uniform on Omega and partial Omega.
    Matches common practice and permits the Rademacher and McDiarmid argument.
  • domain assumption The transport map v of the approximating target u in F_p has sup-norm bounded by p, and the target weights are constructed via Eq. (18).
    Standard RKHS and random-feature approximation assumption; defines epsilon_p.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Non-Asymptotic Analysis of Projected Gradient Descent for Physics-Informed Neural Networks." pith.science (2026). https://pith.science/paper/RT2MKFUN

@misc{pith2026250507311,
  author       = {Pith},
  title        = {Pith review of: Non-Asymptotic Analysis of Projected Gradient Descent for Physics-Informed Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RT2MKFUN}},
  note         = {Machine review of arXiv:2505.07311}
}
abstract

In this work, we provide a non-asymptotic convergence analysis of projected gradient descent for physics-informed neural networks for the Poisson equation. Under suitable assumptions, we show that the optimization error can be bounded by $\mathcal{O}(1/\sqrt{T} + 1/\sqrt{m} + \epsilon_{\text{approx}})$, where $T$ is the number of algorithm time steps, $m$ is the width of the neural network and $\epsilon_{\text{approx}}$ is an approximation error. The proof of our optimization result relies on bounding the linearization error and using this result together with a Lyapunov drift analysis. Additionally, we quantify the generalization error by bounding the Rademacher complexities of the neural network and its Laplacian. Combining both the optimization and generalization results, we obtain an overall error estimate based on an existing error estimate from regularity theory.

Figures

Figures reproduced from arXiv: 2505.07311 by the authors.

Figure 1
Figure 1. Shown are the 90% percentile of the empirical loss compared to dif￾ferent network widths m trained with projected stochastic gradient descent for T = m steps as well as the O [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation

    cs.LG 2025-08 conditional novelty 6.0 of 10

    SGD and stochastic gradient flow are proven to drive the empirical PINN loss for the Poisson equation to zero exponentially in expectation, for sufficiently wide two-layer networks.

Reference graph

Works this paper leans on

50 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [35]

    Notes on Exact Boundary Values in Residual Minimisation

    Johannes Müller and Marius Zeinhofer. “Notes on Exact Boundary Values in Residual Minimisation”. In: Proceedings of Mathematical and Scientific Machine Learning. Ed. by Bin Dong et al. Vol. 190. Proceedings of Machine Learning Research. PMLR, 2022, pp. 231–

    work page 2022

  2. [1]

    Physical activation functions (PAFs): An ap- proachformoreefficientinductionofphysicsintophysics-informedneuralnetworks(PINNs)

    Jassem Abbasi and Pål Østebø Andersen. “Physical activation functions (PAFs): An ap- proachformoreefficientinductionofphysicsintophysics-informedneuralnetworks(PINNs)”. In: Neurocomputing 608 (2024), p. 128352. issn: 0925-2312. doi: https : / / doi . org / 10.1016/j.neucom.2024.128352 . url: https://www.sciencedirect.com/science/ article/pii/S0925231224011238

    arXiv 2024

  3. [2]

    A novel framework for policy mirror descent with general parameterization and linear convergence

    Carlo Alfano, Rui Yuan, and Patrick Rebeschini. “A novel framework for policy mirror descent with general parameterization and linear convergence”. In: Advances in Neural Information Processing Systems36 (2024). REFERENCES 19

    work page 2024

  4. [3]

    Fine-grained analysis of optimization and generalization for overpa- rameterized two-layer neural networks

    Sanjeev Arora et al. “Fine-grained analysis of optimization and generalization for overpa- rameterized two-layer neural networks”. In:International Conference on Machine Learning. PMLR. 2019, pp. 322–332

    work page 2019

  5. [4]

    Learning theory from first principles

    Francis Bach. Learning theory from first principles. MIT press, 2024

    2024

  6. [5]

    Rademacher and Gaussian Complexities: Risk Bounds and Structural Results

    Peter Bartlett and Shahar Mendelson. “Rademacher and Gaussian Complexities: Risk Bounds and Structural Results”. In: vol. 3. 2001, pp. 224–240.isbn: 978-3-540-42343-0. doi: 10.1007/3-540-44581-1_15

    doi:10.1007/3-540-44581-1_15 2001

  7. [6]

    Three ways to solve partial differential equations with neural networks—A review

    Jan Blechschmidt and Oliver G Ernst. “Three ways to solve partial differential equations with neural networks—A review”. In:GAMM-Mitteilungen 44.2 (2021), e202100006

    work page 2021

  8. [7]

    Finite element methods of least-squares type

    Pavel B Bochev and Max D Gunzburger. “Finite element methods of least-squares type”. In: SIAM review40.4 (1998), pp. 789–837

    work page 1998

Show all 50 references
  1. [8]

    The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks

    Andrea Bonfanti, Giuseppe Bruno, and Cristina Cipriani. “The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks”. In:The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024. url: https://openreview.net/forum? id=FY6vPtITtE

    2024

  2. [9]

    Concentration inequalities

    Stéphane Boucheron, Gábor Lugosi, and Olivier Bousquet. “Concentration inequalities”. In: Summer school on machine learning. Springer, 2003, pp. 208–240

    2003

  3. [10]

    On the Spectral Bias of Neural Networks in the Neural Tangent Kernel Regime

    Benjamin Bowman. On the Spectral Bias of Neural Networks in the Neural Tangent Kernel Regime. University of California, Los Angeles, 2023

    2023

  4. [11]

    Physics-informed neural networks (PINNs) for fluid mechanics: A re- view

    Shengze Cai et al. “Physics-informed neural networks (PINNs) for fluid mechanics: A re- view”. In:Acta Mechanica Sinica37.12 (2021), pp. 1727–1738

    2021

  5. [12]

    Sample Complexity and Overparameterization Bounds for Temporal- Difference Learning With Neural Network Approximation

    Semih Cayci et al. “Sample Complexity and Overparameterization Bounds for Temporal- Difference Learning With Neural Network Approximation”. In:IEEE Transactions on Au- tomatic Control68.5 (2023), pp. 2891–2905.doi: 10.1109/TAC.2023.3234234

    2023

  6. [13]

    Lenaic Chizat and Francis Bach.On the Global Convergence of Gradient Descent for Over- parameterized Models using Optimal Transport. 2018. arXiv:1805.09545 [math.OC]

    2018 arXiv

  7. [14]

    Scientific machine learning through physics–informed neural net- works: Where we are and what’s next

    Salvatore Cuomo et al. “Scientific machine learning through physics–informed neural net- works: Where we are and what’s next”. In:Journal of Scientific Computing92.3 (2022), p. 88

    2022

  8. [15]

    Error estimates for physics- informed neural networks approximating the Navier–Stokes equations

    Tim De Ryck, Ameya D Jagtap, and Siddhartha Mishra. “Error estimates for physics- informed neural networks approximating the Navier–Stokes equations”. In:IMA Journal of Numerical Analysis44.1 (2024), pp. 83–119

    2024

  9. [16]

    Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs

    Tim De Ryck and Siddhartha Mishra. “Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs”. In:Advances in Computational Mathematics 48.6 (2022), p. 79

    2022

  10. [17]

    Gradient descent finds global minima of deep neural networks

    Simon Du et al. “Gradient descent finds global minima of deep neural networks”. In:Inter- national conference on machine learning. PMLR. 2019, pp. 1675–1685

    2019

  11. [18]

    Du et al

    Simon S. Du et al. Gradient Descent Provably Optimizes Over-parameterized Neural Net- works. 2019. arXiv:1810.02054 [cs.LG]

    2019 arXiv

  12. [19]

    AlgorithmsforsolvinghighdimensionalPDEs: from nonlinear Monte Carlo to machine learning

    WeinanE,JiequnHan,andArnulfJentzen.“AlgorithmsforsolvinghighdimensionalPDEs: from nonlinear Monte Carlo to machine learning”. In:Nonlinearity 35.1 (2021), p. 278

    2021

  13. [20]

    Lawrence C. Evans. Partial differential equations. Second. Vol. 19. Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2010, pp. xxii+749.isbn: 978-0-8218-4974-3. doi: 10.1090/gsm/019. url: https://doi.org/10.1090/gsm/019

    2010 doi

  14. [21]

    Gradient descent finds the global optima of two- layerphysics-informedneuralnetworks

    Yihang Gao, Yiqi Gu, and Michael Ng. “Gradient descent finds the global optima of two- layerphysics-informedneuralnetworks”.In: International Conference on Machine Learning. PMLR. 2023, pp. 10676–10707

    2023

  15. [22]

    An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning

    Lukas Gonon et al. “An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning”. In:arXiv preprint arXiv:2408.13222 (2024)

    2024 arXiv

  16. [23]

    Elliptic problems in nonsmooth domains

    Pierre Grisvard. Elliptic problems in nonsmooth domains. SIAM, 2011

    2011

  17. [24]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Clément Hongler. “Neural tangent kernel: Convergence and generalization in neural networks”. In:Advances in neural information processing sys- tems 31 (2018). 20 REFERENCES

    2018

  18. [25]

    Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks

    Ziwei Ji and Matus Telgarsky. “Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks”. In:International Con- ference on Learning Representations. 2020. url: https://openreview.net/forum?id= HygegyrYwH

    2020

  19. [26]

    Neural tangent kernels, transportation mappings, and universal approximation

    Ziwei Ji, Matus Telgarsky, and Ruicheng Xian. “Neural tangent kernels, transportation mappings, and universal approximation”. In:International Conference on Learning Repre- sentations. 2020. url: https://openreview.net/forum?id=HklQYxBKwS

    2020

  20. [27]

    DRMRevisited:ACompleteErrorAnalysis

    YulingJiaoetal.“DRMRevisited:ACompleteErrorAnalysis”.In: arXiv preprint arXiv:2407.09032 (2024)

    2024 arXiv

  21. [28]

    Physics-informed machine learning

    George Em Karniadakis et al. “Physics-informed machine learning”. In: Nature Reviews Physics 3.6 (2021), pp. 422–440

    2021

  22. [29]

    PINNACLE: PINN Adaptive ColLocation and Experimen- talpointsselection

    Gregory Kang Ruey Lau et al. “PINNACLE: PINN Adaptive ColLocation and Experimen- talpointsselection”.In: The Twelfth International Conference on Learning Representations

  23. [30]

    Probability in Banach Spaces: Isoperimetry and Processes

    Michel Ledoux and Michel Talagrand. “Probability in Banach Spaces: Isoperimetry and Processes”. In: 1991.url: https://api.semanticscholar.org/CorpusID:118526268

    1991

  24. [31]

    Chapter 11 - Two-layer neural networks for partial differential equations: optimization and generalization theory

    Tao Luo and Haizhao Yang. “Chapter 11 - Two-layer neural networks for partial differential equations: optimization and generalization theory”. In:Numerical Analysis Meets Machine Learning. Ed. by Siddhartha Mishra and Alex Townsend. Vol. 25. Handbook of Numerical Analysis. ISS...

    2024

  25. [32]

    Influence of Activation Functions on the Conver- gence of Physics-Informed Neural Networks for 1D Wave Equation

    Paweł Maczuga and Maciej Paszyński. “Influence of Activation Functions on the Conver- gence of Physics-Informed Neural Networks for 1D Wave Equation”. In:Computational Science – ICCS 2023. Ed. by Jiří Mikyška et al. Cham: Springer Nature Switzerland, 2023, pp. 74–88. isbn: 978...

    2023

  26. [33]

    On the method of bounded differences

    Colin McDiarmid. “On the method of bounded differences”. In:Surveys in Combinatorics, 1989: Invited Papers at the Twelfth British Combinatorial Conference. Ed. by J.Editor Siemons. London Mathematical Society Lecture Note Series. Cambridge University Press, 1989, pp. 148–188

    1989

  27. [34]

    Estimatesonthegeneralizationerrorofphysics- informed neural networks for approximating PDEs

    SiddharthaMishraandRobertoMolinaro.“Estimatesonthegeneralizationerrorofphysics- informed neural networks for approximating PDEs”. In:IMA Journal of Numerical Analysis 43.1 (2023), pp. 1–43

    2023

  28. [36]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. “Random features for large-scale kernel machines”. In: Advances in neural information processing systems20 (2007)

    2007

  29. [37]

    Uniform approximation of functions with random bases

    Ali Rahimi and Benjamin Recht. “Uniform approximation of functions with random bases”. In: 2008 46th annual allerton conference on communication, control, and computing. IEEE. 2008, pp. 555–561

    2008

  30. [38]

    Physics-informed neural net- works: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. “Physics-informed neural net- works: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations”. In:Journal of Computational physics378 (2019), pp. 686–707

    2019

  31. [39]

    Challenges in Training PINNs: A Loss Landscape Perspective

    Pratik Rathore et al. “Challenges in Training PINNs: A Loss Landscape Perspective”. In: Proceedings of the 41st International Conference on Machine Learning. Ed. by Ruslan Salakhutdinov et al. Vol. 235. Proceedings of Machine Learning Research. PMLR, 2024, pp. 42159–42191. url...

    2024

  32. [40]

    DGM: A deep learning algorithm for solving partial differential equations

    Justin Sirignano and Konstantinos Spiliopoulos. “DGM: A deep learning algorithm for solving partial differential equations”. In: Journal of computational physics 375 (2018), pp. 1339–1364. REFERENCES 21

    2018

  33. [41]

    Deep learning theory (DRAFT)

    Matus Telgarsky. Deep learning theory (DRAFT). 2023.url: https://mjt.cs.illinois. edu/dlt/two.pdf

    2023

  34. [42]

    Juan Diego Toscano et al.From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning. 2024. arXiv: 2410.13228 [cs.LG] . url: https://arxiv.org/abs/ 2410.13228

    2024 arXiv

  35. [43]

    Learning Specialized Activation Functions for Physics-Informed Neu- ral Networks

    Honghui Wang et al. “Learning Specialized Activation Functions for Physics-Informed Neu- ral Networks”. In: Communications in Computational Physics 34.4 (2023), pp. 869–906. issn: 1991-7120. doi: https://doi.org/10.4208/cicp.OA- 2023- 0058 . url: https: //global- sci.com/artic...

    2023 doi

  36. [44]

    An expert’s guide to training physics-informed neural networks

    Sifan Wang et al. “An expert’s guide to training physics-informed neural networks”. In: arXiv preprint arXiv:2308.08468(2023)

    2023 arXiv

  37. [45]

    Convergence of Implicit Gradient Descent for TrainingTwo-LayerPhysics-InformedNeuralNetworks

    Xianliang Xu, Zhongyi Huang, and Ye Li. “Convergence of Implicit Gradient Descent for TrainingTwo-LayerPhysics-InformedNeuralNetworks”.In: arXiv preprint arXiv:2407.02827 (2024)

    2024 arXiv

  38. [46]

    ConvergenceAnalysisofNaturalGradientDescentforOver-parameterized Physics-Informed Neural Networks

    XianliangXuetal.“ConvergenceAnalysisofNaturalGradientDescentforOver-parameterized Physics-Informed Neural Networks”. In:arXiv preprint arXiv:2408.00573(2024)

    2024 arXiv

  39. [47]

    Marius Zeinhofer, Rami Masri, and Kent-André Mardal.A Unified Framework for the Error Analysis of Physics-Informed Neural Networks. 2024. arXiv:2311.00529 [math.NA]

    2024 arXiv

  40. [48]

    Chengxi Zeng, Tilo Burghardt, and Alberto M Gambaruto.Feature Mapping in Physics- Informed Neural Networks (PINNs). 2024. arXiv: 2402.06955 [cs.LG] . url: https:// arxiv.org/abs/2402.06955

    2024 arXiv

  41. [240]

    url: https://proceedings.mlr.press/v190/muller22b.html

  42. [2024]

    url: https://openreview.net/forum?id=GzNaCp6Vcg

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.