Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Learn Singularly Perturbed Solutions via Homotopy Dynamics

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A homotopy path from easy to hard parameters lets neural networks learn singularly perturbed PDE solutions that direct training misses.

desk verdict A useful training trick for stiff PDEs, with experiments that look real, but the convergence theorem doesn't cover the algorithm that actually runs. read the letter →

arxiv 2502.00488 v3 pith:AG3UJZCY submitted 2025-02-01 cs.LG cs.NAmath.NA

Chuqi Chen , Yahong Yang , Yang Xiang , Wenrui Hao This is my paper · ORCID
classification cs.LGcs.NAmath.NA MSC 35B2565N3568T07
keywords singularlyperturbedPDEshomotopycontinuationphysics-informedneuralnetworkstangentkernelAllen–CahnequationHelmholtzBurgersoperatorlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Small parameters in singularly perturbed PDEs—interface thickness in Allen–Cahn, viscosity in Burgers, inverse wave number in Helmholtz—are precisely the regime where physics-informed neural networks lose accuracy, and this paper seeks to establish why and how to fix it. The authors argue that as $\varepsilon$ decreases, the smallest eigenvalue of the kernel governing gradient descent shrinks, so linear convergence slows from a rate like $\exp(-Cn^3t)$ to $\exp(-Ct/n)$. Their proposed remedy is homotopy dynamics: first train the network at a large, easy $\varepsilon_0$, then follow the solution branch $H(u(\varepsilon),\varepsilon)=0$ down to the target small $\varepsilon$ using a Davidenko-type ODE for the network parameters. If the convergence analysis is right, the same training pipeline handles sharp interfaces, high-dimensional high-frequency Helmholtz solutions, and Burgers operator learning with relative errors orders of magnitude below direct training.

What carries the argument

The central object is the homotopy residual $H(u,\varepsilon)=L_\varepsilon u-f(u)$, whose zero set defines the solution branch $u(\varepsilon)$. Following that branch is governed by the Davidenko equation $dH/d\varepsilon=0$, which the paper converts into homotopy dynamics for network parameters, $d\theta/d\varepsilon=-(H_u\nabla_\theta u)^\dagger H_\varepsilon$, using a Moore–Penrose inverse. Two numerical strategies implement it: forward Euler with the pseudo-inverse for small networks, and a homotopy-loss minimization for large networks, where each step minimizes a residual formed from the discretized $H_\varepsilon$ term together with the original PDE and boundary losses. The difficulty analysis is carried by the neural tangent kernel $K_\varepsilon=D_\varepsilon SS^\top D_\varepsilon^\top$: the smallest eigenvalue of this kernel bounds the gradient-descent convergence rate, so the behavior of $D_\varepsilon$ as a discrete differential operator determines when training becomes hard.

What would settle it

Run the paper's 1D Allen–Cahn protocol (Algorithm 1 with $\varepsilon_0=0.1$, $\varepsilon_n=0.01$) and record the residual $\|H(u_\theta(\varepsilon_k),\varepsilon_k)\|$ at every step; if the final residual is not driven to the scale reached by direct training at $\varepsilon_0$ (roughly $10^{-6}$), or if the final relative L2 error stays above the reported $8.08\times10^{-3}$, the practical convergence claim is not reproduced.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that the training difficulty of neural PDE solvers for the elliptic problem $-\varepsilon^2\Delta u=f(u)$ with boundary conditions is controlled by the smallest eigenvalue of the kernel $K_\varepsilon=D_\varepsilon SS^\top D_\varepsilon^\top$, where $D_\varepsilon$ discretizes the variation of the PDE operator and $S$ is the network Jacobian at collocation points. Theorem 4.1 shows that once $SS^\top$ is positive definite, the loss decays at rate $\exp(-\lambda_{\min}(K_\varepsilon)t/n)$, and in the Allen–Cahn example this rate deteriorates from roughly $\exp(-Cn^3t)$ at large $\varepsilon$ to $\exp(-Ct/n)$ at small $\varepsilon$. The paper's constructive claim is that this deterioration can be circumvented: treating $\varepsilon$ as a homotopy parameter, it derives the parameter dynamics $d\theta/d\varepsilon=-(H_u\nabla_\theta u)^\dagger H_\varepsilon$ and proves (Theorem 4.3) that if the starting network at $\varepsilon_0$ is already close to the exact solution in $H^2$ and the $\varepsilon$-steps obey a Lipschitz-based size bound, then the tracked approximation at the target $\varepsilon_n$ stays close to the exact solution. Experiments on 2D Allen–Cahn, a 20-dimensional Helmholtz equation, and Burgers operator learning report that this protocol reduces relative L2 error from order-one values to roughly $10^{-3}$–$10^{-2}$ where direct training fails.

Load-bearing premise

The load-bearing premise is that the network at the starting value $\varepsilon_0$ is already so accurate that the residual $H(u_\theta(\varepsilon),\varepsilon)$ stays negligible throughout the path, because the convergence theorem is proved for the exact solution branch $u(\varepsilon)$ and not for the parameter trajectory $\theta(\varepsilon)$ produced by the training algorithm.

Editorial extensions

If this is right

  • If the central claim is correct, physics-informed neural networks can solve singularly perturbed steady-state problems by a single continuation protocol: start at $\varepsilon_0\approx 0.1$ and decay $\varepsilon$ with step sizes chosen from a Lipschitz bound, instead of trying to minimize the sharp-interface loss directly.
  • For the tested 2D Allen–Cahn equation, the paper's homotopy loss reaches a relative L2 error of $8.78\times10^{-3}$ while original PINN training, curriculum training, sequence-to-sequence training, and resampling all stay above $8\times10^{-2}$.
  • At $d=20$ Helmholtz, the method keeps the relative L2 error near $5\times10^{-4}$ for $\varepsilon=1/50$, where classical training degrades to $9.98\times10^{-1}$; this indicates the approach is not limited to one spatial dimension.
  • In the operator-learning setting, a DeepONet trained with homotopy dynamics under an unsupervised loss infers Burgers steady states at $\varepsilon=0.05$ with relative error $2.52\times10^{-2}$ and shock-location error $1.2\times10^{-3}$, and the paper reports 0.2 s inference time versus over 2400 s for the finite-difference baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The convergence theorem is proved for the exact solution branch $u(\varepsilon)$, while the experiments optimize the homotopy loss for network parameters; a natural test of the paper's reach is whether the residual $H(u_\theta(\varepsilon),\varepsilon)$ is actually driven near zero at every intermediate $\varepsilon$-step, which the theorem assumes but does not prove.
  • The eigenvalue argument predicts a general ordering: any problem where $\lambda_{\max}(D_\varepsilon D_\varepsilon^\top)$ stays bounded away from zero as $\varepsilon\to0$ should not need homotopy, while problems where it collapses should benefit most; this could be checked on other singularly perturbed operators before running costly training.
  • Because the homotopy path resembles numerical continuation, the method could be paired with adaptive collocation near the evolving interface, reducing the number of residual points while keeping the tracked path accurate.
  • The strategy of using the previous $\varepsilon$-solution as a warm start matches curriculum learning; the paper's added value is the explicit step-size rule and convergence bound, which suggests other curriculum schedules for PDE parameters could be designed from eigenvalue data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a homotopy-dynamics training method for neural-network PDE solvers: first train at a large parameter ε0, then gradually decrease ε to a target ε* by tracking a Davidenko-style ODE for the network parameters (Eq. (8)) or by minimizing a homotopy loss (Eq. (10)). The authors claim two theoretical results: Theorem 4.1, that the smallest eigenvalue of a kernel Kε controls the convergence rate of direct training and that smaller ε makes training harder; and Theorem 4.3, that homotopy dynamics converges to the small-ε solution provided the initial network is accurate and the ε-steps are sufficiently small. Experiments on 1D/2D Allen–Cahn, a 20-dimensional Helmholtz equation, and Burgers operator learning report large accuracy improvements over several baselines.

Significance. The proposed method is practically appealing, and the experimental comparison is a genuine strength: the authors test against curriculum, time-sequential, and resampling baselines, report large L2RE reductions (e.g., Table 3), and cover problem settings such as high-dimensional Helmholtz and operator learning where homotopy training clearly helps. The paper also attempts to link the singular parameter ε to the NTK eigenvalue spectrum and to provide a step-size rule. However, the two central theoretical claims have serious gaps: Theorem 4.3 applies to an idealized Euler path of the exact PDE solution branch, not to the parameter trajectory generated by the implemented algorithm (Strategy 2), and Theorem 4.1's 'smaller ε is harder' conclusion is an overreading of an upper bound on λmin(Kε). The empirical results are strong enough to merit a major revision rather than rejection.

major comments (3)
  1. [Section 4.2, Theorem 4.3, Eq. (8), Eq. (10)] Theorem 4.3 is a statement about the exact solution branch u(ε) and its Euler approximation U(ε) defined in Eq. (23); it does not concern the neural-network parameter trajectory θ(ε) updated by Eq. (8) or trained with Eq. (10). Eq. (8) is obtained by differentiating the identity H(u(θ(ε)),ε)=0, but a finite network cannot satisfy this identity pointwise in ε; if r(ε)=H(u(θ(ε)),ε) is nonzero, the correct relation is H_u∇θu θ' + H_ε = r'(ε), and no argument in the paper controls r'(ε). The claim in Section 4.2 that Strategy 2 is 'merely an alternative approach for solving the linear system' is unsupported: minimizing LHom at finitely many collocation points neither drives the discrete residual of Eq. (7) to zero nor controls the H^2(Ω) norm in which Theorem 4.3 is formulated. Since all experiments in Section 5 except the 1D Allen–Cahn example use Strategy 2, the convergence guarantee does not apply to the algorithm evaluated in the tables.
  2. [Section 4.1, Eq. (20), Remark 4.2] The conclusion that smaller ε 'invariably' makes training harder does not follow from the inequalities of Theorem 4.1. Eq. (19) is a lower bound on the decay rate: L_H(θ(t)) ≤ L_H(θ(0)) exp(-λmin(Kε)t/n). Eq. (20) only bounds λmin(Kε) from above; for small ε this upper bound is O(1) rather than O(n^4), which means the worst-case guaranteed rate is slower, not that actual gradient-flow convergence is slower in any given run. Establishing increased training difficulty would require a lower bound on λmin(Kε) that decreases as ε→0, or a direct analysis of the loss landscape. The wording in Remark 4.2 and the paragraph following Eq. (21) overstates this as an impossibility statement.
  3. [Section 4.2, Eq. (24), Appendix B] The assumptions of Theorem 4.3 are not connected to the experimental step schedules. The condition τ ≪ 1 involves sup_k (ε_k-ε_{k+1})^2 ||u(ε_k)||_{H^4}; for singularly perturbed problems the H^4 norm typically grows as ε→0, so the permissible step size may be much smaller than the Δε = 0.1/0.01 used in, e.g., Appendix B.4 for the Helmholtz problem. The paper notes that P_ε may be large and that steps should be 'sufficiently small' but does not quantify the relation; therefore Theorem 4.3 is not verified for the reported runs.
minor comments (5)
  1. [Appendix A.2, Lemma A.7] Lemma A.7 is stated for an arbitrary matrix T; as written it is false for rectangular T (e.g., T ∈ R^{3×2}, A = diag(1,10,10)). The application in the proof of Theorem 4.1 uses T = D_ε^T, which is square and nonsingular under the theorem's assumptions, so Eq. (20) is not affected, but the lemma statement should be corrected to require T square and nonsingular.
  2. [Appendix B.4] Appendix B.4 lists the Helmholtz network architecture as [2,30,30,30,1], but Section 5.2 solves the equation in dimension d=20; the input width appears to be a typo.
  3. [Section 3.2, Eq. (10)] In Eq. (10), H_ε(u_θ(x_i), ε) is written with the homotopy parameter ε rather than the step index ε_k, and Δu_k is not defined in the displayed equation; this makes the discretization of the homotopy consistency term hard to parse.
  4. [Section 4.1, Eq. (13)] In Eq. (13), the symbol δ_{u_θ} H(u_θ(x_i), ε) is not defined; it appears to denote a functional derivative and should be defined explicitly.
  5. [Section 4.1, paragraph after Eq. (21)] The phrase 'the lower bound for the training speed is given by exp(−Ct/n)' is imprecise: Eq. (19) is an upper bound on the loss, not a lower bound on training speed; the authors should rephrase this in terms of guaranteed versus worst-case decay.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence theorem is a conditional classical Euler-error bound, and the gap between the exact-solution branch and the neural-network trajectory is a completeness issue rather than a circular derivation.

full rationale

The central theoretical result, Theorem 4.3, is not constructed from the method's own outputs. It is a standard Euler-method error estimate for the exact solution branch u(ε) satisfying H(u(ε),ε)=0 (Eq. 22), with assumptions e0 ≪ 1 and sufficiently small steps; the conclusion that u(εn)-U(εn) is small follows by the usual local-truncation-error recursion in Appendix A.3. This is conditional, not circular: an accurate starting point and small homotopy steps are hypotheses of a propagation guarantee, not relabeled conclusions. Theorem 4.1 is an NTK eigenvalue analysis of direct training, independent of the homotopy claim. The homotopy step schedule Δε is a hyperparameter, not a fitted constant dressed up as a prediction. Self-citations such as Chen & Hao 2019, Yang et al. 2025, Yang et al. 2023, Yang & He 2024, and Hao & Yang 2019 provide motivation, independently provable approximation bounds, or an externally corroborated steady-state uniqueness statement for Burgers' equation; none is invoked as an unverified uniqueness theorem that forces the paper's construction. The legitimate concern is one of completeness: Theorem 4.3 concerns U(ε), not the neural-network parameter trajectory θ(ε), and no argument shows that minimizing the empirical homotopy loss (Eq. 10) enforces Eq. (8) or controls the H^2 norm used in the theorem. That is an unproven gap between theory and algorithm, but it is not a circular reduction of the conclusion to the inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central method introduces no new physical entities. It rests on several unproved regularity, invertibility, positivity, and approximation assumptions, plus user-chosen weights and step schedules. The convergence theorem transfers most of the burden to e0 and small step sizes, which are not established for the actual neural network optimization.

free parameters (3)
  • lambda (boundary loss weight) = not reported
    Balances the boundary loss against the residual and homotopy terms in Eqs. (2) and (10); the value is chosen by hand and affects solution fidelity.
  • alpha (homotopy consistency weight) = not reported
    Weights the homotopy-dynamics consistency loss in Eq. (10); no value or sensitivity study is provided.
  • homotopy step schedule (Delta epsilon or Delta s, with refinement) = e.g. Delta epsilon = 0.001 for 1D Allen-Cahn; Delta s = 0.1 refined to 0.01 for 2D Allen-Cahn; Delta epsilon = 0.1…
    The step sizes are exactly the quantities Theorem 4.3 requires to be sufficiently small; they are chosen or refined by hand and no automated selection criterion is proven.
assumptions (5)
  • domain assumption D_epsilon is nonsingular and lambda_min(SS^T) > 0
    Theorem 4.1 requires these to conclude lambda_min(K_epsilon) > 0. Positivity of SS^T is argued for wide two-layer analytic activations via Proposition A.6, but D_epsilon nonsingular is not verified for small epsilon in Allen-Cahn, where diag(f'(u)) can approach -1.
  • domain assumption Assumption A.9: ||u - u*||_{H^2(Omega)} <= C L_c(theta)
    Used in Appendix A.4 to argue that the initial error e0 is small; the justification cites Grisvard for linear elliptic operators, while the paper applies it to nonlinear f(u) = u^3 - u without proof.
  • domain assumption h(u,epsilon) is Lipschitz and u(epsilon) lies in H^4 with bounded P_epsilon
    Theorem 4.3 requires a Lipschitz constant P_epsilon and a local truncation bound; P_epsilon can blow up as epsilon approaches zero, so the theorem only covers small-but-finite epsilon and does not address the limit epsilon -> 0.
  • domain assumption partial H / partial u is invertible along the whole homotopy path
    Eq. (22) and Theorem 4.3 require invertibility of the linearized operator at every epsilon; this is not established for the Allen-Cahn branches or for the Burgers shock solution.
  • domain assumption Approximation and generalization errors for the neural network are small
    The error decomposition in Appendix A.4 relies on approximation and Rademacher bounds from cited works (Yang et al. 2023; Luo and Yang 2020); the paper does not prove these bounds for the specific architectures and losses used in the experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learn Singularly Perturbed Solutions via Homotopy Dynamics." pith.science (2026). https://pith.science/paper/AG3UJZCY

@misc{pith2026250200488,
  author       = {Pith},
  title        = {Pith review of: Learn Singularly Perturbed Solutions via Homotopy Dynamics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AG3UJZCY}},
  note         = {Machine review of arXiv:2502.00488}
}
read the original abstract

Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for singularly perturbed problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in these singularly perturbed problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of these singularly perturbed problems. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving singularly perturbed differential equations.

Figures

Figures reproduced from arXiv: 2502.00488 by the authors.

Figure 1
Figure 1. Framework of homotopy dynamics for solving singularly perturbed problems. points, leading to substantial memory consumption, and the sampling process becomes increasingly complicated in high￾dimensional settings. The second strategy is the design of multiscale neural network architectures (Wang, 2020; Liu, 2020; Liu et al., 2024b; Hao et al., 2024; Wang et al., 2021; Huang et al., 2025). These approaches generally r… view at source ↗
Figure 2
Figure 2. Training curves for different values of ε in solving the 1D Allen-Cahn steady-state equation. As ε decreases, the training error increases, indicating that the training process becomes pro￾gressively more difficult. 2.3. Challenges in Training Neural Networks In this paper, we consider the following singularly perturbed elliptic problem: ( −ε 2∆u(x) = f(u), x ∈ Ω, u(x) = g(x), x ∈ ∂Ω, (3) where ε is a problem parame… view at source ↗
Figure 3
Figure 3. Evolution of the Homotopy dynamics for steady state 1D Allen-Cahn equation. The L2RE for ε = 0.01 is 8.08e − 3. system given in Eq. (7). However, for certain PDEs or larger neural networks, directly solving Eq. (7) is unstable because the term Hu∇θu contains many small singular values, caus￾ing conventional methods (e.g., using SVD) to incur large errors. Therefore, we opt to solve an optimization problem in the tra… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Largest eigenvalue of Dε (21) for different ε. A smaller ε results in a smaller largest eigenvalue of (21), leading to a slower convergence rate and increased difficulty in training. Therefore, we obtain dLH(θ) dt = ∇θLH(θ) dθ dt = − 1 n2 H(uθ(x), ε)SS⊤H⊤(uθ(x), ε) = −…
Figure 5
Figure 5. Figure 5: 2D Allen Cahn Equaiton. (Top) Evolution of the Homotopy Dynamcis. (Bottom) Plot for Cross-section of u(x, y) at y = 0.5 i.e., u(x, y = 0.5). The reference solution u∞(x) represents the ground truth steady-state solution. The L2RE is 8.78e − 3. Number of residual points…
Figure 6
Figure 6. Figure 6: 1D Burgers’ Equation (Operator Learning): Steady-state solutions for different initializations u0 under varying viscosity ε: (a) ε = 0.5, (b) ε = 0.1, (c) ε = 0.05. The results demonstrate that all final test solutions converge to the correct steady-state solution. (d)…
Figure 7
Figure 7. Figure 7: Solution for 1D Allen-Cahn equation for origin training [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Reference Solution for 2D Allen-Cahn equation. The result obtained using PINN is shown in the [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Solution for 2D Allen-Cahn equation for origin training. (1) (2) (3) (4) [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Numerical results for the 2D Allen–Cahn equation using the Curriculum training strategy. grid of 50 × 50 sample points and augment it by adding 5000 points in the vicinity of the sharp interface. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Numerical results for the 2D Allen–Cahn equation using the Sequence-to-sequence training strategy. B.4. High Dimension Helmholtz Equation Number of residual points nres = 10000 and number of boundary points nbc = 2000. Neural network architecture is a fully connected …
Figure 12
Figure 12. Figure 12: Numerical results for the 2D Allen–Cahn equation using the Resampling training strategy [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Largest eigenvalue of Dε (60) for different ε. A smaller ε results in a smaller largest eigenvalue of (60), leading to a slower convergence rate and increased difficulty in training. Dε = −ε 2∆dis + 1 d diag 1, . . . , 1) (60) B.5. High Frequency Function Approximatio…
Figure 14
Figure 14. Figure 14: High-frequency function sin(50πx) approximation: Comparison of loss curves between original evolution and homotopy evolution. The comparison shows that homotopy evolution effectively reduces the loss, successfully approximating the high-frequency function, while the o…
Figure 15
Figure 15. Figure 15: Largest eigenvalue of Dε (64) for different ε. A smaller ε results in a smaller largest eigenvalue of (64), leading to a slower convergence rate and increased difficulty in training [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Energy Dissipation Rate Guided Adaptive Sampling for Physics-Informed Neural Networks: Resolving Surface-Bulk Dynamics in Allen-Cahn Systems

    math.NA 2025-07 conditional novelty 6.0 of 10

    An energy-dissipation-rate-guided sampling strategy (EDRAS) improves PINN accuracy on thermodynamically consistent Allen-Cahn models, giving up to a sixfold relative MSE reduction over residual-based adaptive refineme...

Reference graph

Works this paper leans on

64 extracted references · 52 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Allen, S. M. and Cahn, J. W. Coherent and incoherent equilibria in iron-rich iron-aluminum alloys. Acta Metallurgica, 23 0 (9): 0 1017--1026, 1975

    work page 1975

  3. [3]

    A convergence theory for deep learning via over-parameterization

    Allen-Zhu, Z., Li, Y., and Song, Z. A convergence theory for deep learning via over-parameterization. In International conference on machine learning, pp.\ 242--252. PMLR, 2019

    work page 2019

  4. [4]

    S., Hu, W., Li, Z., Salakhutdinov, R

    Arora, S., Du, S. S., Hu, W., Li, Z., Salakhutdinov, R. R., and Wang, R. On exact computation with an infinitely wide neural net. Advances in neural information processing systems, 32, 2019

    work page 2019

  5. [5]

    Atkinson, K., Han, W., and Stewart, D. E. Numerical solution of ordinary differential equations, volume 81. John Wiley & Sons, 2009

    work page 2009

  6. [6]

    and Greville, T

    Ben-Israel, A. and Greville, T. N. Generalized inverses: theory and applications. Springer Science & Business Media, 2006

    work page 2006

  7. [7]

    Burgers, J. M. A mathematical model illustrating the theory of turbulence. Advances in applied mechanics, 1: 0 171--199, 1948

    work page 1948

  8. [8]

    and Gu, Q

    Cao, Y. and Gu, Q. Generalization error bounds of gradient descent for learning over-parameterized deep relu networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp.\ 3349--3356, 2020

    work page 2020

Show all 64 references
  1. [9]

    Automatic differentiation is essential in training neural networks for solving differential equations

    Chen, C., Yang, Y., Xiang, Y., and Hao, W. Automatic differentiation is essential in training neural networks for solving differential equations. arXiv preprint arXiv:2405.14099, 2024 a

    2024 arXiv

  2. [10]

    Quantifying training difficulty and accelerating convergence in neural network-based pde solvers

    Chen, C., Zhou, Q., Yang, Y., Xiang, Y., and Luo, T. Quantifying training difficulty and accelerating convergence in neural network-based pde solvers. arXiv preprint arXiv:2410.06308, 2024 b

    2024 arXiv

  3. [11]

    Bridging traditional and machine learning-based algorithms for solving PDEs : the random feature method

    Chen, J., Chi, X., Yang, Z., et al. Bridging traditional and machine learning-based algorithms for solving PDEs : the random feature method. J Mach Learn, 1: 0 268--98, 2022

    2022

  4. [12]

    and Hao, W

    Chen, Q. and Hao, W. A homotopy training algorithm for fully connected neural networks. Proceedings of the Royal Society A, 475 0 (2231): 0 20190662, 2019

    2019

  5. [13]

    S., Giampaolo, F., Rozza, G., Raissi, M., and Piccialli, F

    Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., and Piccialli, F. Scientific M achine L earning T hrough P hysics– I nformed N eural N etworks: W here W e A re and W hat’s N ext. J. Sci. Comput., 92 0 (3), 2022

    2022

  6. [14]

    and Wang, Y

    Dong, S. and Wang, Y. A method for computing inverse parametric PDE problems with random-weight neural networks. Journal of Computational Physics, 489: 0 112263, 2023

    2023

  7. [15]

    Gradient descent finds global minima of deep neural networks

    Du, S., Lee, J., Li, H., Wang, L., and Zhai, X. Gradient descent finds global minima of deep neural networks. In International conference on machine learning, pp.\ 1675--1685. PMLR, 2019

    2019

  8. [16]

    and Yu, B

    E, W. and Yu, B. The D eep R itz M ethod: A D eep L earning- B ased N umerical A lgorithm for S olving V ariational P roblems. Communications in Mathematics and Statistics, 6 0 (1): 0 1--12, 2018

    2018

  9. [17]

    Evans, L. C. Partial differential equations, volume 19. American Mathematical Society, 2022

    2022

  10. [18]

    Gradient descent finds the global optima of two-layer physics-informed neural networks

    Gao, Y., Gu, Y., and Ng, M. Gradient descent finds the global optima of two-layer physics-informed neural networks. In International Conference on Machine Learning, pp.\ 10676--10707. PMLR, 2023

    2023

  11. [19]

    Failure-informed adaptive sampling for pinns, part ii: combining with re-sampling and subset simulation

    Gao, Z., Tang, T., Yan, L., and Zhou, T. Failure-informed adaptive sampling for pinns, part ii: combining with re-sampling and subset simulation. Communications on Applied Mathematics and Computation, 6 0 (3): 0 1720--1741, 2024

    2024

  12. [20]

    A deep learning method for the dynamics of classic and conservative Allen-Cahn equations based on fully-discrete operators

    Geng, Y., Teng, Y., Wang, Z., and Ju, L. A deep learning method for the dynamics of classic and conservative Allen-Cahn equations based on fully-discrete operators. Journal of Computational Physics, 496: 0 112589, 2024

    2024

  13. [21]

    and Bengio, Y

    Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010

    2010

  14. [22]

    Elliptic problems in nonsmooth domains

    Grisvard, P. Elliptic problems in nonsmooth domains. SIAM, 2011

    2011

  15. [23]

    Solving high-dimensional partial differential equations using deep learning

    Han, J., Jentzen, A., and E, W. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115 0 (34): 0 8505--8510, 2018

    2018

  16. [24]

    and Yang, Y

    Hao, W. and Yang, Y. Convergence of a homotopy finite element method for computing steady states of burgers’ equation. ESAIM: Mathematical Modelling and Numerical Analysis, 53 0 (5): 0 1629--1644, 2019

    2019

  17. [25]

    P., Xi, Y., Xu, T., and Yang, Y

    Hao, W., Li, R. P., Xi, Y., Xu, T., and Yang, Y. Multiscale neural networks for approximating green's functions. arXiv preprint arXiv:2410.18439, 2024

    2024

  18. [26]

    Newton informed neural operator for solving nonlinear partial differential equations

    Hao, W., Liu, X., and Yang, Y. Newton informed neural operator for solving nonlinear partial differential equations. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2025

    2025

  19. [27]

    Mgno: Efficient parameterization of linear operators via multigrid

    He, J., Liu, X., and Xu, J. Mgno: Efficient parameterization of linear operators via multigrid. In 12th International Conference on Learning Representations, ICLR 2024, 2024

    2024

  20. [28]

    Methods of mathematical physics

    Hilbert, D. Methods of mathematical physics. CUP Archive, 1985

    1985

  21. [29]

    Frequency-adaptive multi-scale deep neural networks

    Huang, J., You, R., and Zhou, T. Frequency-adaptive multi-scale deep neural networks. Computer Methods in Applied Mechanics and Engineering, 437: 0 117751, 2025

    2025

  22. [30]

    E., Kevrekidis, I

    Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., and Yang, L. Physics-informed machine learning. Nature Reviews Physics, 3 0 (6): 0 422--440, 2021

    2021

  23. [31]

    and Kreiss, H.-O

    Kreiss, G. and Kreiss, H.-O. Convergence to steady state of solutions of burgers' equation. Applied Numerical Mathematics, 2 0 (3): 0 161--179, 1986. ISSN 0168-9274. doi:https://doi.org/10.1016/0168-9274(86)90026-7. URL https://www.sciencedirect.com/science/article/pii/0168927...

    1986

  24. [32]

    Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., and Mahoney, M. W. Characterizing possible failure modes in physics-informed neural networks. In Advances in Neural Information Processing Systems, 2021

    2021

  25. [33]

    Dosnet as a non-black-box pde solver: When deep learning meets operator splitting

    Lan, Y., Li, Z., Sun, J., and Xiang, Y. Dosnet as a non-black-box pde solver: When deep learning meets operator splitting. Journal of Computational Physics, 491: 0 112343, 2023

    2023

  26. [34]

    Lanthaler, S., Mishra, S., and Karniadakis, G. E. Error estimates for deeponets: A deep learning framework in infinite dimensions. Transactions of Mathematics and Its Applications, 6 0 (1): 0 tnac001, 2022

    2022

  27. [35]

    and Mathias, R

    Li, C.-K. and Mathias, R. The Lidskii-Mirsky-Wielandt theorem--additive and multiplicative versions. Numerische Mathematik, 81: 0 377--413, 1999

    1999

  28. [36]

    Z., and Zhu, J

    Li, W., Bazant, M. Z., and Zhu, J. Phase-field deeponet: Physics-informed deep operator neural network for fast simulations of pattern formation governed by gradient flows of free-energy functionals. Computer Methods in Applied Mechanics and Engineering, 416: 0 116299, 2023

    2023

  29. [37]

    Towards an understanding of residual networks using neural tangent hierarchy (nth)

    Li, Y., Luo, T., and Yip, N. Towards an understanding of residual networks using neural tangent hierarchy (nth). arXiv preprint arXiv:2007.03714, 2020

    2007 arXiv

  30. [38]

    B., Azizzadenesheli, K., liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A

    Li, Z., Kovachki, N. B., Azizzadenesheli, K., liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Fourier N eural O perator for P arametric P artial D ifferential E quations. In International Conference on Learning Representations, 2021

    2021

  31. [39]

    Deep nonparametric estimation of operators between infinite dimensional spaces

    Liu, H., Yang, H., Chen, M., Zhao, T., and Liao, W. Deep nonparametric estimation of operators between infinite dimensional spaces. Journal of Machine Learning Research, 25 0 (24): 0 1--67, 2024 a

    2024

  32. [40]

    Mitigating spectral bias for the multiscale operator learning

    Liu, X., Xu, B., Cao, S., and Zhang, L. Mitigating spectral bias for the multiscale operator learning. Journal of Computational Physics, 506: 0 112944, 2024 b . ISSN 0021-9991. doi:https://doi.org/10.1016/j.jcp.2024.112944. URL https://www.sciencedirect.com/science/article/pii...

    2024

  33. [41]

    Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains

    Liu, Z. Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains. Communications in Computational Physics, 28 0 (5), 2020

    2020

  34. [42]

    Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3 0 (3): 0 218--229, 2021 a

    2021

  35. [43]

    A priori generalization analysis of the deep Ritz method for solving high dimensional elliptic partial differential equations

    Lu, Y., Lu, J., and Wang, M. A priori generalization analysis of the deep Ritz method for solving high dimensional elliptic partial differential equations. In Conference on learning theory, pp.\ 3196--3241. PMLR, 2021 b

    2021

  36. [44]

    and Yang, H

    Luo, T. and Yang, H. Two-layer neural networks for partial differential equations: Optimization and generalization theory. arXiv preprint arXiv:2006.15733, 2020

    2006 arXiv

  37. [45]

    and Ghosh, S

    Mattey, R. and Ghosh, S. A novel sequential method to train physics informed neural networks for allen cahn and cahn hilliard equations. Computer Methods in Applied Mechanics and Engineering, 390: 0 114474, 2022

    2022

  38. [46]

    Morton, K. W. and Mayers, D. F. Numerical solution of partial differential equations: an introduction. Cambridge university press, 2005

    2005

  39. [47]

    Z., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., K \" o pf, A., Yang, E. Z., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Py T orch:...

    1912 arXiv

  40. [48]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

    Raissi, M., Perdikaris, P., and Karniadakis, G. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378: 0 686--707, 2019

    2019

  41. [49]

    Challenges in training pinns: A loss landscape perspective

    Rathore, P., Lei, W., Frangella, Z., Lu, L., and Udell, M. Challenges in training pinns: A loss landscape perspective. arXiv preprint arXiv:2402.01868, 2024

    2024 arXiv

  42. [50]

    Siegel, J. W. and Xu, J. Approximation rates for neural networks with general activation functions. Neural Networks, 128: 0 313--321, 2020

    2020

  43. [51]

    Local randomized neural networks with discontinuous Galerkin methods for partial differential equations

    Sun, J., Dong, S., and Wang, F. Local randomized neural networks with discontinuous Galerkin methods for partial differential equations. Journal of Computational and Applied Mathematics, 445: 0 115830, 2024

    2024

  44. [52]

    High-dimensional probability: An introduction with applications in data science, volume 47

    Vershynin, R. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018

    2018

  45. [53]

    Multi-scale deep neural network (mscalednn) methods for oscillatory stokes flows in complex domains

    Wang, B. Multi-scale deep neural network (mscalednn) methods for oscillatory stokes flows in complex domains. Communications in Computational Physics, 28 0 (5): 0 2139--2157, 2020

    2020

  46. [54]

    On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks

    Wang, S., Wang, H., and Perdikaris, P. On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 384: 0 113938, 2021

    2021

  47. [55]

    Wight, C. L. and Zhao, J. Solving Allen-Cahn and Cahn-Hilliard equations using the adaptive physics informed neural networks. arXiv preprint arXiv:2007.04542, 2020

    2007 arXiv

  48. [56]

    A variational analysis for the moving finite element method for gradient flows

    Xu, X. A variational analysis for the moving finite element method for gradient flows. arXiv preprint arXiv:2009.01393, 2020

    2009 arXiv

  49. [57]

    J., Zhang, Y., and Luo, T

    Xu, Z.-Q. J., Zhang, Y., and Luo, T. Overview frequency principle/spectral bias in deep learning. Communications on Applied Mathematics and Computation, pp.\ 1--38, 2024

    2024

  50. [58]

    DeepONet for solving PDEs : Generalization analysis in Sobolev training

    Yang, Y. DeepONet for solving PDEs : Generalization analysis in Sobolev training. arXiv preprint arXiv:2410.04344, 2024

    2024

  51. [59]

    and He, J

    Yang, Y. and He, J. Deeper or wider: A perspective from optimal generalization error with sobolev loss. Forty-first International Conference on Machine Learning, 2024

    2024

  52. [60]

    and Xiang, Y

    Yang, Y. and Xiang, Y. Approximation of functionals by neural network without curse of dimensionality. J Mach Learn, 1 (4): 0 342–372, 2022

    2022

  53. [61]

    Nearly optimal approximation rates for deep super relu networks on sobolev spaces

    Yang, Y., Wu, Y., Yang, H., and Xiang, Y. Nearly optimal approximation rates for deep super relu networks on sobolev spaces. arXiv preprint arXiv:2310.10766, 2023

    2023 arXiv

  54. [62]

    Homotopy relaxation training algorithms for infinite-width two-layer relu neural networks

    Yang, Y., Chen, Q., and Hao, W. Homotopy relaxation training algorithms for infinite-width two-layer relu neural networks. Journal of Scientific Computing, 102 0 (2): 0 40, 2025

    2025

  55. [63]

    Energy-dissipative evolutionary deep operator neural networks

    Zhang, J., Zhang, S., Shen, J., and Lin, G. Energy-dissipative evolutionary deep operator neural networks. Journal of Computational Physics, 498: 0 112638, 2024. ISSN 0021-9991. doi:https://doi.org/10.1016/j.jcp.2023.112638. URL https://www.sciencedirect.com/science/article/pi...

    2024

  56. [64]

    Annealed adaptive importance sampling method in pinns for solving high dimensional partial differential equations

    Zhang, Z., Li, J., and Liu, B. Annealed adaptive importance sampling method in pinns for solving high dimensional partial differential equations. Journal of Computational Physics, 521: 0 113561, 2025. ISSN 0021-9991. doi:https://doi.org/10.1016/j.jcp.2024.113561. URL https://w...

    2025

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.