Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features

Karl H. Johansson; Matthieu Barreau; Miguel Aguiar; Yulun Wu

REVIEW 4 major objections 4 minor 61 references

Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training physics-informed networks by alternating Fourier-enhanced basis updates with Levenberg-Marquardt coefficient fitting reduces relative L2 errors by up to two orders of magnitude on high-frequency and nonlinear PDEs, and the…

desk verdict The empirical method is worth a look, but the advertised global convergence theorem is unproven as written. read the letter →

arxiv 2608.05892 v1 pith:T7GDIBV3 submitted 2026-08-06 cs.LG cs.NAmath.NA

Yulun Wu , Matthieu Barreau , Miguel Aguiar , Karl H. Johansson This is my paper · ORCID

classification cs.LGcs.NAmath.NA

keywords physics-informedneuralnetworksspectralbiasLevenberg-MarquardtalgorithmalternatingoptimizationFourierfeaturesnonlinearleastsquarespartialdifferentialequationsglobalconvergence

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the two known failure modes of physics-informed neural networks, spectral bias and the entanglement of feature learning with coefficient fitting, can be addressed by splitting training into two alternating problems. The upper-level problem updates a Fourier-enhanced basis in the network's latent space, while the lower-level problem fits the projection coefficients with the Levenberg-Marquardt algorithm. The authors prove global convergence of the alternating scheme to critical points without requiring convexity of the lower-level problem, and report numerical results on Klein-Gordon, Korteweg-de Vries, high-frequency heat, lid-driven cavity, and viscous Burgers benchmarks in which the method reaches relative $L^2$ errors up to two orders of magnitude below state-of-the-art baselines. A reader should care because the proposed principle, decouple basis learning from coefficient fitting and solve the coefficient problem with a structure-exploiting least-squares solver, is directly aimed at the problems that make PINNs unreliable on oscillatory and nonlinear PDEs.

What carries the argument

The central object is the composite feature map $\psi_D=\gamma_D\circ z_\omega$: a random Fourier feature embedding applied to the last hidden layer rather than to the raw input coordinates, which enriches the basis with high-frequency components while keeping the basis adaptive through $\omega$. As $D\to\infty$ its inner product converges almost surely to a Gaussian RBF kernel in the latent space (Lemma 1), so the upper-level update acts as deep kernel learning that reshapes the kernel. The lower-level machinery is the Levenberg-Marquardt algorithm, which linearizes the residual and solves the strictly convex damped subproblem (17), whose closed-form normal-equation solution (18) defines the coefficient update; Lemma 2 bounds the linearization error quadratically in the residual, justifying the one or two inner iterations used with a warm-up initialization. Proposition 1, Lemma 3, and Theorem 1 then convert an inexact-descent condition into global convergence to critical points.

What would settle it

Run FALM-PINN on the 1D heat equation with $F=100$ using the paper's stated hyperparameters ($D=800$, $\sigma=10$, $N_f=15000$, $J$ with $\gamma=10^{-7}$, no warm-up) and compare the final relative $L^2$ error to the reported $6.8\times10^{-4}$; if the alternating phase does not reproduce an error below the best baseline's plateau near $10^{-1}$, the central accuracy claim is falsified.

Watch

Extended reading notes

Core claim

FALM-PINN's core claim is that applying a random Fourier feature map $\gamma_D$ to the network's last hidden layer $z_\omega$, giving the composite feature map $\psi_D=\gamma_D\circ z_\omega$, and then alternating gradient updates of $\omega$ with Levenberg-Marquardt solves of the coefficient vector $\beta$, removes the representation-coefficient coupling that limits PINN accuracy. The solution is written $u_{\omega,\beta}(x)=\Theta^\top\psi_D(x)$, and the lower-level problem minimizes the same physics loss by solving the damped normal equation $(J_j^\top J_j+\gamma_j I/2)\Delta\beta^{(j)}=-J_j^\top r(\beta^{(j)})$ at each inner iteration. The paper states that as $D\to\infty$ the feature inner product converges to a Gaussian kernel in the latent space (Lemma 1), that the LM linearization error is quadratic in the current residual (Lemma 2), and that the alternating sequence converges globally to critical points even for nonconvex lower-level problems (Theorem 1). For linear PDEs the lower-level problem becomes a single-step convex kernel ridge regression. On the tested benchmarks FALM-PINN reports relative $L^2$ errors such as $3.5\times10^{-5}$ on the 2D Klein-Gordon problem, $4.42\times10^{-4}$ on the 1D Korteweg-de Vries problem, $6.8\times10^{-4}$ on the $F=100$ heat equation, $2.78\times10^{-3}$ on lid-driven cavity flow, and $5.10\times10^{-5}$ on viscous Burgers, each below the compared baselines by one to two orders of magnitude.

Load-bearing premise

The load-bearing premise is that the upper-level updates are exact gradient steps with a small enough step size, the Levenberg-Marquardt damping stays large enough relative to the loss curvature at every inner step, and all iterates remain in a compact set; the implemented algorithm uses Adam, much smaller damping, and no compactness guarantee, so the convergence theorem does not directly cover the executed runs.

Editorial extensions

If this is right

For linear PDEs, the lower-level problem is convex and the Levenberg-Marquardt iteration converges in one step to the unique minimizer, so FALM-PINN reduces to a single closed-form kernel ridge regression and needs no warm-up phase or inner iterations.
For nonlinear and coupled PDE systems, $J=1$ or $J=2$ damped subproblems per outer iteration are reported sufficient to refine the coefficients, meaning the decoupling does not require a costly inner optimization loop.
The Fourier bandwidth $\sigma$ must be matched to the dominant frequency of the solution: the heat-equation ablation shows that for $F=200$ only $\sigma=50$ succeeds while $\sigma=1$ and $\sigma=10$ fail, which pins a practical tuning rule to Lemma 1.
The observed two-stage convergence, where the error drops sharply after switching from joint warm-up to alternating optimization, supports the claim that the accuracy gain comes from decoupling the basis from the coefficients rather than from additional network capacity.
If the convergence theorem holds, the decoupled scheme is guaranteed to reach critical points without convexity of the lower level, so the method is not limited to linear or scalar PDEs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The ablation shows that a fixed $\sigma$ fails outside a frequency band; a natural extension the authors leave implicit is an adaptive bandwidth scheduler that raises $\sigma$ as the residual's high-frequency content is detected, with the $F\in\{10,100,150,200\}$ table providing a quantitative benchmark.
For linear PDEs the lower level is an ordinary linear least-squares problem, so FALM-PINN can be read as an adaptive-basis spectral method; this suggests a direct comparison against classical spectral and multiscale solvers that the paper does not make.
The alternating design implies two timescales: the basis $\omega$ is updated slowly while the coefficients $\beta$ are fit essentially to convergence; testing whether a deliberately slow output-layer learning rate in joint training reproduces part of the gain would isolate how much of the improvement comes from decoupling versus from the LM solver.
Because the lower-level solve is built entirely on the residual Jacobian, the same decoupling could be applied to operator-learning or time-stepping PINN variants, a direction the paper mentions only as future work.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, the numerical core is solid and worth taking seriously. Second, the paper's central convergence claim does not hold as written, and the flaw is substantial, not cosmetic.

The method itself is a sensible extension of IFeF-PINN: decouple basis learning from coefficient fitting, and use Levenberg–Marquardt for the lower-level nonlinear least squares. That is a natural and practically reasonable idea. The experiments are decent for this subfield: five seeds, error bars, multiple benchmarks (Klein–Gordon, KdV, high-frequency heat, cavity flow, Burgers), and a clean comparison against IFeF-PINN with matched settings. The reported accuracy gains are large and plausible, and the ablation on bandwidth σ makes sense. Lemma 1 is correct. If I were working on PINN optimizers, I would cite this for the empirical recipe.

Now the soft spots, in order of severity.

The load-bearing problem is the invalid step in Proposition 1. The proof asserts that for some finite κ_J, ||β^(J) − β^(J−1)|| ≤ κ_J ||Σ Δβ^(j)|| = κ_J ||β_{k+1} − β_k||. That is false for arbitrary vectors: take Δβ^(0)=(1,0), Δβ^(1)=(−1,ε). Nothing in the LM update or the assumptions prevents such cancellation. Without that bound, Eq. (33) does not follow, and Theorem 1(iii)–(iv) are unsupported. The stress-test note gets this exactly right, and the problem is independent of the Adam-vs-gradient-descent mismatch.

Second, Lemma 2's stated bound omits a factor. The proof sums M·Nf components each with √(λ/Nf), giving M√(λ Nf), not the stated √(M λ). This is a lesser issue because Lemma 2 is not used in the convergence proof, but it is still an error in a lemma the paper relies on for intuition.

Third, the theory analyzes exact gradient descent (Algorithm 1 pseudocode) while the experiments use Adam. That is a common gap in ML papers, but it matters here because the abstract claims global convergence of the proposed training scheme. As written, the theorem covers a different algorithm than the one evaluated.

Finally, Assumption 1 (bounded iterates) is asserted with no mechanism, so even a repaired proof would need a new argument or a more modest claim.

My position: the empirical contribution deserves a fair referee. Send it to review, but the referees should require the authors to either fix the convergence proof or soften the claim substantially. The method is useful enough that the paper can be accepted on the numerics with an honest theoretical section; it should not be accepted with the current Theorem 1.

Referee Report

4 major / 4 minor

Summary. The paper proposes FALM-PINN, an alternating training framework for physics-informed neural networks that decouples a Fourier-enhanced basis update (upper-level problem) from a Levenberg–Marquardt coefficient fit (lower-level problem). The authors claim that this decoupling mitigates spectral bias and representation–coefficient coupling, and they report consistent relative L2 error reductions of one to two orders of magnitude over several PINN baselines on Klein–Gordon, Korteweg–de Vries, high-frequency heat, lid-driven cavity, and viscous Burgers benchmarks. The theoretical section proves a kernel approximation limit, a bound on the linearization error of LM, and a global convergence theorem to critical points for the alternating scheme. The numerical section reports five-seed means and standard deviations, includes a bandwidth ablation, and compares the lower-level LM solver against the authors' earlier IFeF-PINN method under matched settings.

Significance. If the algorithm and its convergence guarantee are as advertised, the decoupled LM formulation would be a practically useful and theoretically better-understood training principle for PINNs on high-frequency and nonlinear problems. The numerical comparisons are carefully executed: multiple challenging benchmarks, error bars over five independent trials, an ablation over the Fourier bandwidth, and a controlled comparison against IFeF-PINN with identical architecture and data. The kernel interpretation of the Fourier-enhanced features (Lemma 1) is a clean and correct observation. However, the central theoretical claim of global convergence is not substantiated as written: the proof of Proposition 1 contains an invalid norm inequality, and the theorem is proven only for an idealized gradient-descent update with large LM damping, while Algorithm 1 and the experiments use Adam and small damping. These gaps affect the paper's headline contribution, so the significance of the result currently rests on the empirical evidence rather than on the convergence theorem.

major comments (4)

[Section 3.4.3, Proposition 1, Eq. (33)] The step bounding the last inner increment by the total increment is invalid. From ∥∇βΦ(β^J)∥ ≤ b̃β∥β^J − β^(J−1)∥, the proof asserts that for J ≥ 1 there exists a finite κ_J with ∥β^(J) − β^(J−1)∥ ≤ κ_J ∥β^(J) − β^(0)∥. This does not follow from the stated assumptions: for increments Δβ^(0) = (1,0) and Δβ^(1) = (−1,ε), the norm of the second increment is about 1 while the norm of the sum is ε, so no finite κ_J exists uniformly as ε → 0. Nothing in the damped LM update (18) or Assumption 1 rules out such cancellation between inner steps. Consequently, Eq. (33) is unproven, and without Eq. (33) the gradient bound in Lemma 3 and the convergence conclusions Theorem 1(iii)–(iv) are unsupported. This failure is independent of the Adam-versus-gradient-descent mismatch; even for exact gradient descent with γ > L/2, the proof as written does not establish convergence to critical points.
[Algorithm 1 and Section 3.4.3, Lemma 3 (Eq. (37))] The convergence analysis does not cover the algorithm actually implemented. Lemma 3 uses the fixed-step gradient update ω_{k+1} − ω_k = −η_ω ∇_ω Φ(ω_k, β_{k+1}) to derive the sufficient decrease and gradient bounds, but Algorithm 1 specifies Adam for the upper-level update, and the experiments state that ω is optimized by Adam (e.g., Section 4.1). Similarly, Proposition 1 requires the LM damping to satisfy γ > L/2 at every inner iteration, while the experiments use fixed or decayed damping of order 10^−6 to 10^−5 without verifying the L-smoothness constant. As a result, Theorem 1 does not apply to the method whose performance is reported. The paper should either analyze Adam with an appropriate adaptive-step framework or state the convergence theorem for the exact algorithm executed in the experiments, with assumptions checked on the reported hyperparameters.
[Section 3.4.2, Lemma 2, Eq. (26)] The linearization-error bound has an incorrect N_f dependence. Summing the Taylor–Lagrange bound over the M N_f physics residual components gives (1/2) M M_N √(λ N_f) ∥Δβ∥², not (1/2) M_N √(M λ) ∥Δβ∥². The text's claim that the factor 1/√N_f 'cancels the √N_f growth' is wrong: there are N_f collocation points per equation, so the sum over components reintroduces √N_f, together with an extra factor of M rather than √M. The quadratic-in-residual structure is preserved, but the stated bound and its stated independence of N_f are not established. This should be corrected, and the consequences for the warm-up discussion and for the use of J ∈ {1,2} should be re-examined.
[Assumption 1 and Theorem 1] The 'global' convergence statement relies entirely on Assumption 1, which postulates that the iterates remain in a compact set B, but the paper provides no mechanism, update rule property, or damping schedule that guarantees this boundedness for the alternating sequence. Without such a guarantee, the accumulation-point argument in Theorem 1(iv) is conditional on an unverified assumption. The theorem should either be labeled as a conditional statement, or the proof should show that the algorithm prevents divergence, e.g., via a coercivity or regularization argument that is actually satisfied by Algorithm 1.

minor comments (4)

[Section 3.4.2] The symbol σ is used both for the Fourier bandwidth in Section 2.2 and for the smallest singular value in Lemma 2, Assumption 2; this is confusing and should be disambiguated.
[References] Reference [8] contains a typo: 'heterognous' should be 'heterogeneous'.
[Section 4.6] The comparison with IFeF-PINN is fair in design, but the text should clarify that IFeF-PINN is the authors' own prior method, since the closest-baseline claim in the abstract and introduction relies on that work.
[Section 3.1, Eq. (6)] The notation u_{ω,β}(x) = Θ^T ψ_D(x) is slightly misleading when m > 1, since Θ is an m × 2D matrix and ψ_D is a 2D vector; the intended column-wise interpretation is clear but should be written explicitly in the equation.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the derivation is self-contained, with only a minor non-load-bearing self-citation to IFeF-PINN; the proof gap in Proposition 1 is a correctness defect, not circularity.

full rationale

The central claims of the paper do not reduce by construction to their inputs. The numerical results are benchmarked against external baselines (RBA, PIKAN, SIREN, compleX-PINN, vanilla PINN) on standard PDE test problems with relative L2 error evaluated on independent test grids; no fitted parameter is renamed as a prediction. The Fourier-enhanced features are a standard random Fourier feature construction (Rahimi and Recht, Tancik et al.), and the bandwidth sigma is a hyperparameter swept in an ablation, not fit to the test error. The lower-level LM solver and upper-level gradient update minimize the same PINN composite loss, and the convergence theorem is an internal optimization claim supported by a deduction chain; although the proof of Proposition 1 contains a genuine norm-inequality gap concerning the constant kappa_J that undermines Theorem 1, that gap is a correctness risk rather than circularity, because the theorem is not obtained by assuming its conclusion. The only self-citation is IFeF-PINN [52], used to position the method and as a baseline with identical architecture; the cited linear single-step convergence statement is mathematically elementary and not load-bearing for the nonlinear central claim. Overall, no derivation step is equivalent to its inputs by definition, so the circularity score is low despite the self-citation. A score of 2 reflects the minor, non-load-bearing self-citation, not any circular derivation.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The main uncharged baggage is a set of theoretical conditions (compact iterates, full-rank Jacobian, large damping) that are neither certified nor compatible with the Adam-based implementation.

free parameters (6)

Fourier bandwidth sigma = 1, 10, 25, 50 depending on benchmark
Controls the frequency range of the Fourier-enhanced features; the ablation in Section 4.3 shows sigma must scale with target frequency F, so the reported accuracy depends on per-problem tuning.
LM damping gamma = 1e-6 to 1e-5 with decay schedules
Chosen per benchmark; Proposition 1 requires gamma > L/2 for the descent guarantee, which is never checked.
Inner LM iterations J = 1 or 2
Chosen empirically; the claim that one or two iterations suffice relies on Lemma 2.
Warm-up iterations Nwarm = 0, 5000, or 10000
Hand-selected per benchmark to give the lower-level solver a good initialization.
Number of Fourier features D = 800
Fixed hyperparameter; larger D approximates the RBF kernel better but is not adaptively chosen.
Loss weight lambda = 0.01 for physics, 1 for boundary and initial
Fixed manually across experiments; affects the balance of the least-squares objective.

assumptions (4)

standard math Bochner's theorem and the strong law of large numbers justify the almost-sure convergence of random Fourier features to the Gaussian kernel.
Used in Lemma 1 to interpret the Fourier-enhanced features as an RBF kernel in the latent space.
domain assumption The PDE operator N is twice continuously differentiable and iterates stay in a compact set where the Jacobian has full column rank with smallest singular value bounded below by sigma > 0.
Assumptions in Lemma 2; not verified for the PDE benchmarks.
ad hoc to paper The upper-level update is exact gradient descent with step size eta_omega < 2/L.
Lemma 3 and Theorem 1 use this update rule, but Algorithm 1 applies Adam, so the convergence proof does not cover the implemented algorithm.
ad hoc to paper The LM damping satisfies gamma > L/2 at every inner iteration.
Proposition 1 requires this for sufficient decrease, but the algorithm uses small fixed or decayed damping without checking this condition.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features." pith.science (2026). https://pith.science/paper/T7GDIBV3

@misc{pith2026260805892,
  author       = {Pith},
  title        = {Pith review of: Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T7GDIBV3}},
  note         = {Machine review of arXiv:2608.05892}
}

abstract

Physics-informed neural networks (PINNs) often fail to accurately resolve partial differential equations (PDEs) with high-frequency or multi-scale solutions, as well as strongly nonlinear problems. Two factors underlie this difficulty: spectral bias, the tendency of neural networks to underfit high-frequency features; and representation-coefficient coupling, the entanglement of representation learning and coefficient fitting within a single nonconvex optimization objective. In this work, we propose the Fourier-enhanced alternating Levenberg--Marquardt PINN (FALM-PINN), an optimization framework that decouples representation learning from coefficient fitting. The upper-level problem learns a Fourier-enhanced basis that enriches the latent space with high-frequency components, while the lower-level problem resolves the coupling by fitting the projection coefficients on this basis, solving a nonlinear least-squares problem with the Levenberg--Marquardt algorithm. The framework applies to general nonlinear and coupled PDE systems, and reduces to a single-step convex optimization problem for linear PDEs. We prove global convergence of the alternating training scheme in both cases. Numerical examples on multiple challenging high-frequency and nonlinear PDEs show that FALM-PINN achieves relative $L^2$ errors up to two orders of magnitude lower than state-of-the-art baselines.

Figures

Figures reproduced from arXiv: 2608.05892 by the authors.

**Figure 2.** Convergence trajectories on the 2D Klein–Gordon equation. Relative [PITH_FULL_IMAGE:figures/full_fig_p025_2.png] view at source ↗

**Figure 3.** Exact solution of the 2D Klein–Gordon equation at three time slices [PITH_FULL_IMAGE:figures/full_fig_p025_3.png] view at source ↗

Figures from the paper (15 more)

**Figure 4.** Figure 4: Point-wise absolute error maps in log scale on the 2D Klein–Gordon equation at [PITH_FULL_IMAGE:figures/full_fig_p026_4.png]

**Figure 5.** Figure 5: Reference solution of the 1D Korteweg–de Vries equation. [PITH_FULL_IMAGE:figures/full_fig_p027_5.png]

**Figure 6.** Figure 6: Relative L 2 error trajectories on the 1D Korteweg–de Vries equation. Curves denote the mean across five independent trials, and shaded bands indicate the min–max range. The vertical dashed line marks the transition from the warm-up phase (Nwarm = 5,000) to the alterna…

**Figure 7.** Figure 7: Predictions (left column) and absolute error maps in log scale (right column) [PITH_FULL_IMAGE:figures/full_fig_p030_7.png]

**Figure 8.** Figure 8: Exact solution of the 1D Heat equation. All MLP-based methods use a fully connected network with 5 hidden layers of width 80. Training proceeds for 50,000 iterations under Adam, with an initial learning rate of 1.5 × 10−3 exponentially decayed by a factor of 0.8 every …

**Figure 9.** Figure 9: Convergence trajectories on the 1D heat equation. Relative [PITH_FULL_IMAGE:figures/full_fig_p032_9.png]

**Figure 10.** Figure 10: Predicted solutions (left column) and point-wise absolute errors in log scale [PITH_FULL_IMAGE:figures/full_fig_p034_10.png]

**Figure 11.** Figure 11: Prediction slices u(x, t = 1) of FALM-PINN on the 1D heat equation for Fourierfeature bandwidths σ ∈ {1, 10, 25, 50}, at spatial frequencies F ∈ {200, 150, 100, 10} (top to bottom). The black curve is the exact solution. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_11.png]

**Figure 12.** Figure 12: Relative L 2 error trajectories on the 2D lid-driven cavity flow. Curves denote the mean across five independent trials; shaded bands indicate the min–max range. The vertical dashed line marks the transition from the warm-up phase (Nwarm = 5,000) to the alternating op…

**Figure 13.** Figure 13: Reference solution of the 2D lid-driven cavity flow. [PITH_FULL_IMAGE:figures/full_fig_p038_13.png]

**Figure 14.** Figure 14: Point-wise absolute error maps (log scale) on the 2D lid-driven cavity flow. [PITH_FULL_IMAGE:figures/full_fig_p039_14.png]

**Figure 15.** Figure 15: Relative L 2 error trajectories on the 1D viscous Burgers equation. Curves denote the mean across five independent trials, and shaded bands indicate the min–max range [PITH_FULL_IMAGE:figures/full_fig_p042_15.png]

**Figure 16.** Figure 16: Reference solution of the 1D viscous Burgers equation. [PITH_FULL_IMAGE:figures/full_fig_p042_16.png]

**Figure 17.** Figure 17: Predictions (left column) and error maps in log scale (right column) on the 1D [PITH_FULL_IMAGE:figures/full_fig_p043_17.png]

**Figure 18.** Figure 18: Prediction slices over time at fixed spatial positions [PITH_FULL_IMAGE:figures/full_fig_p044_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 45 canonical work pages

[1]

R. J. LeVeque, Finite difference methods for ordinary and partial dif- ferential equations: steady-state and time-dependent problems, SIAM, 2007

2007
[2]

O. C. Zienkiewicz, R. L. Taylor, P. Nithiarasu, J. Zhu, The finite element method, Vol. 3, Elsevier, 1977

work page 1977
[3]

P. G. Ciarlet, The finite element method for elliptic problems, SIAM, 2002

work page 2002
[4]

S. C. Brenner, L. R. Scott, The mathematical theory of finite element methods, Springer, 2008

work page 2008
[5]

J. P. Boyd, Chebyshev and Fourier spectral methods, Courier Corpora- tion, 2001

2001
[6]

Canuto, M

C. Canuto, M. Y. Hussaini, A. Quarteroni, T. A. Zang, Spectral meth- ods, Vol. 285, Springer, 2006

work page 2006
[7]

J. Shen, T. Tang, L.-L. Wang, Spectral methods: algorithms, analysis and applications, Vol. 41, Springer Science & Business Media, 2011

2011
[8]

Weinan, B

E. Weinan, B. Engquist, The heterognous multiscale methods, Commu- nications in Mathematical Sciences 1 (1) (2003) 87–132

work page 2003

Show all 61 references

[9]

Abdulle, E

A. Abdulle, E. Weinan, B. Engquist, E. Vanden-Eijnden, The heteroge- neous multiscale method, Acta Numerica 21 (2012) 1–87

2012
[10]

Efendiev, T

Y. Efendiev, T. Y. Hou, Multiscale finite element methods: theory and applications, Springer Science & Business Media, 2009

2009
[11]

Iserles, S

A. Iserles, S. P. Nørsett, Efficient quadrature of highly oscillatory inte- grals using derivatives, Proceedings of the Royal Society A: Mathemat- ical, Physical and Engineering Sciences 461 (2057) (2005) 1383–1399

2005
[12]

Raissi, P

M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378 (2019) 686–707. 47

2019
[13]

G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, L. Yang, Physics-informed machine learning, Nature Reviews Physics (2021)

2021
[14]

Z. Hu, K. Shukla, G. E. Karniadakis, K. Kawaguchi, Tackling the curse of dimensionality with physics-informed neural networks, Neural Net- works 176 (2024) 106369

2024
[15]

F. S. Costabal, S. Pezzuto, P. Perdikaris,δ-PINNs: Physics-informed neural networks on complex geometries, Engineering Applications of Ar- tificial Intelligence 127 (2024) 107324

2024
[16]

Raissi, A

M. Raissi, A. Yazdani, G. E. Karniadakis, Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations, Science 367 (6481) (2020) 1026–1030

2020
[17]

X. Jin, S. Cai, H. Li, G. E. Karniadakis, NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier- Stokes equations, Journal of Computational Physics 426 (2021) 109951

2021
[18]

Y. Chen, L. Lu, G. E. Karniadakis, L. Dal Negro, Physics-informed neural networks for inverse problems in nano-optics and metamaterials, Optics express 28 (8) (2020) 11618–11633

2020
[19]

L. Yang, X. Meng, G. E. Karniadakis, B-pinns: Bayesian physics- informed neural networks for forward and inverse pde problems with noisy data, Journal of Computational Physics 425 (2021) 109913

2021
[20]

Bastek, W

J.-H. Bastek, W. Sun, D. Kochmann, Physics-informed diffusion models, in: International Conference on Learning Representations, Vol. 2025, 2025, pp. 3360–3385

2025
[21]

Z. Wang, A. Harting, M. Barreau, M. M. Zavlanos, K. H. Johansson, Source-guided flow matching, arXiv preprint arXiv:2508.14807 (2025)

2025 arXiv
[22]

D. Shu, Z. Li, A. B. Farimani, A physics-informed diffusion model for high-fidelity flow field reconstruction, Journal of Computational Physics 478 (2023) 111972. 48

2023
[23]

Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, Fourier neural operator for parametric par- tialdifferentialequations, in: InternationalConferenceonLearningRep- resentations, 2021

2021
[24]

L. Lu, P. Jin, G. Pang, Z. Zhang, G. E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence 3 (3) (2021) 218–229

2021
[25]

Rahaman, A

N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, A. Courville, On the spectral bias of neural networks, in: International Conference on Machine Learning, PMLR, 2019, pp. 5301– 5310

2019
[26]

Z.-Q. J. Xu, L. Zhang, W. Cai, On understanding and overcoming spec- tral biases of deep neural network learning methods for solving PDEs, Journal of Computational Physics (2025) 113905

2025
[27]

Krishnapriyan, A

A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby, M. W. Mahoney, Char- acterizing possible failure modes in physics-informed neural networks, Advances in Neural Information Processing Systems 34 (2021) 26548– 26560

2021
[28]

S. Wang, S. Sankaran, P. Perdikaris, Respecting causality for training physics-informed neural networks, Computer Methods in Applied Me- chanics and Engineering 421 (2024) 116813

2024
[29]

S. Wang, X. Yu, P. Perdikaris, When and why PINNs fail to train: A neural tangent kernel perspective, Journal of Computational Physics 449 (2022) 110768

2022
[30]

Y. Song, H. Wang, H. Yang, M. L. Taccari, X. Chen, Loss-attentional physics-informed neural networks, Journal of Computational Physics 501 (2024) 112781

2024
[31]

S. J. Anagnostopoulos, J. D. Toscano, N. Stergiopulos, G. E. Karni- adakis, Residual-based attention in physics-informed neural networks, Computer Methods in Applied Mechanics and Engineering 421 (2024) 116805. 49

2024
[32]

C. Si, M. Yan, Convolution-weighting method for the physics-informed neural network: A primal-dual optimization perspective, Journal of Computational Physics 555 (2026) 113911

2026
[33]

C. Zhao, X. Xie, W. Chen, Casual attention: Adaptive enforcement of causalityinphysics-informedneuralnetworks, JournalofComputational Physics (2026) 115071

2026
[34]

C. Wu, M. Zhu, Q. Tan, Y. Kartha, L. Lu, A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks, Computer Methods in Applied Mechanics and Engi- neering 403 (2023) 115671

2023
[35]

W. Gao, C. Wang, Active learning based sampling for high-dimensional nonlinear partial differential equations, Journal of Computational Physics 475 (2023) 111848

2023
[36]

G. K. R. Lau, A. Hemachandra, S.-K. Ng, B. K. H. Low, PINNACLE: PINN adaptive collocation and experimental points selection, in: The Twelfth International Conference on Learning Representations, 2024

2024
[37]

Tancik, P

M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, Advances in Neural Information Processing Systems 33 (2020) 7537– 7547

2020
[38]

S. Wang, H. Wang, P. Perdikaris, On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks, Computer Methods in Applied Me- chanics and Engineering 384 (2021) 113938

2021
[39]

Sitzmann, J

V. Sitzmann, J. Martel, A. Bergman, D. Lindell, G. Wetzstein, Implicit neural representations with periodic activation functions, Advances in neural information processing systems 33 (2020) 7462–7473

2020
[40]

C. Si, M. Yan, X. Li, Z. Xia, Complex physics-informed neural network (2025). arXiv:2502.04917. URLhttps://arxiv.org/abs/2502.04917 50

2025 arXiv
[41]

X. Zeng, Y. Zhu, A nurbs-based parameterization physics-informed neu- ral network with an adaptive architecture for solving pdes, Journal of Computational Physics 562 (2026) 114980

2026
[42]

Z. Zhao, X. Ding, B. A. Prakash, PINNsFormer: A transformer-based framework for physics-informed neural networks, in: The Twelfth Inter- national Conference on Learning Representations, 2024

2024
[43]

Y. Wang, J. Sun, J. Bai, C. Anitescu, M. S. Eshaghi, X. Zhuang, T. Rabczuk, Y. Liu, Kolmogorov–arnold-informed neural network: A physics-informed deep learning framework for solving forward and in- verse problems based on kolmogorov–arnold networks, Computer Meth- ods in Appli...

2025
[44]

A. D. Jagtap, G. E. Karniadakis, Extended physics-informed neural net- works (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Communications in Computational Physics 28 (5) (2020)

2020
[45]

S. Wang, B. Li, Y. Chen, P. Perdikaris, Piratenets: Physics-informed deep learning with residual adaptive networks, Journal of Machine Learning Research 25 (402) (2024) 1–51

2024
[46]

A. H. Mustajab, H. Lyu, Z. Rizvi, F. Wuttke, Physics-informed neu- ral networks for high-frequency and multi-scale problems using transfer learning, Applied Sciences 14 (8) (2024) 3204

2024
[47]

Wang, C.-Y

Y. Wang, C.-Y. Lai, Multi-stage neural networks: Function approxima- tor of machine precision, Journal of Computational Physics 504 (2024) 112865

2024
[48]

E. C. Cyr, M. A. Gulian, R. G. Patel, M. Perego, N. A. Trask, Robust training and initialization of deep neural networks: An adaptive basis viewpoint, in: Mathematical and Scientific Machine Learning, PMLR, 2020, pp. 512–536

2020
[49]

Huang, Q.-Y

G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: theory and applications, Neurocomputing 70 (1-3) (2006) 489–501. 51

2006
[50]

Dwivedi, B

V. Dwivedi, B. Srinivasan, Physics informed extreme learning machine (pielm)–a rapid method for the numerical solution of partial differential equations, Neurocomputing 391 (2020) 96–118

2020
[51]

S. Dong, Z. Li, Local extreme learning machines and domain decom- position for solving linear and nonlinear partial differential equations, Computer Methods in Applied Mechanics and Engineering 387 (2021) 114129

2021
[52]

Y. Wu, M. Aguiar, K. H. Johansson, M. Barreau, Iterative training of physics-informed neural networks with fourier-enhanced features, arXiv preprint arXiv:2510.19399 (2025)

2025
[53]

Levenberg, A method for the solution of certain non-linear problems in least squares, Quarterly of applied mathematics 2 (2) (1944) 164–168

K. Levenberg, A method for the solution of certain non-linear problems in least squares, Quarterly of applied mathematics 2 (2) (1944) 164–168

1944
[54]

D.W.Marquardt, Analgorithmforleast-squaresestimationofnonlinear parameters, Journal of the society for Industrial and Applied Mathemat- ics 11 (2) (1963) 431–441

1963
[55]

A. G. Wilson, Z. Hu, R. Salakhutdinov, E. P. Xing, Deep kernel learning, in: Artificial Intelligence and Statistics, PMLR, 2016, pp. 370–378

2016
[56]

Rahimi, B

A. Rahimi, B. Recht, Random features for large-scale kernel machines, Advances in Neural Information Processing Systems 20 (2007)

2007
[57]

D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)

2014 arXiv
[58]

Coleman, Calculus on normed vector spaces, Choice Reviews Online (2013)

R. Coleman, Calculus on normed vector spaces, Choice Reviews Online (2013)

2013
[59]

Penwarden, A

M. Penwarden, A. D. Jagtap, S. Zhe, G. E. Karniadakis, R. M. Kirby, A unified scalable framework for causal sweeping strategies for physics-informed neural networks (PINNs) and their temporal de- compositions, Journal of Computational Physics 493 (2023) 112464. doi:10.1016/j.j...

2023
[60]

C. E. Shannon, Communication in the presence of noise, Proceedings of the IRE 37 (1) (2006) 10–21. 52

2006
[61]

Z. Hao, J. Yao, C. Su, H. Su, Z. Wang, F. Lu, Z. Xia, Y. Zhang, S. Liu, L. Lu, et al., Pinnacle: A comprehensive benchmark of physics-informed neural networks for solving pdes, Advances in Neural Information Pro- cessing Systems 37 (2024) 76721–76774. 53

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

R. J. LeVeque, Finite difference methods for ordinary and partial dif- ferential equations: steady-state and time-dependent problems, SIAM, 2007

2007

[2] [2]

O. C. Zienkiewicz, R. L. Taylor, P. Nithiarasu, J. Zhu, The finite element method, Vol. 3, Elsevier, 1977

work page 1977

[3] [3]

P. G. Ciarlet, The finite element method for elliptic problems, SIAM, 2002

work page 2002

[4] [4]

S. C. Brenner, L. R. Scott, The mathematical theory of finite element methods, Springer, 2008

work page 2008

[5] [5]

J. P. Boyd, Chebyshev and Fourier spectral methods, Courier Corpora- tion, 2001

2001

[6] [6]

Canuto, M

C. Canuto, M. Y. Hussaini, A. Quarteroni, T. A. Zang, Spectral meth- ods, Vol. 285, Springer, 2006

work page 2006

[7] [7]

J. Shen, T. Tang, L.-L. Wang, Spectral methods: algorithms, analysis and applications, Vol. 41, Springer Science & Business Media, 2011

2011

[8] [8]

Weinan, B

E. Weinan, B. Engquist, The heterognous multiscale methods, Commu- nications in Mathematical Sciences 1 (1) (2003) 87–132

work page 2003