Pith. sign in

REVIEW 3 major objections 5 minor 20 references

EMNGD claims that, under coercivity, its manifold-constrained step is exactly the best feasible approximation to the function-space Newton step, preserving parameter constraints without changing the PDE energy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:04 UTC pith:SMRXH7TY

load-bearing objection Solid geometry, correct theorem under coercivity, but the coercivity assumption fails for the experimental residual losses, and the headline empirical claim is not backed by matched comparisons. the 3 major comments →

arxiv 2607.22004 v1 pith:SMRXH7TY submitted 2026-07-24 cs.LG

Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers

Zhangyong Liang , Huanhuan Gao This is my paper
classification cs.LG MSC 65K1068T0749M15
keywords energy natural gradient descentmanifold optimizationneural PDE solversRiemannian retractionNewton projectionWoodbury identityNyström approximationphysics-informed neural networks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Energy manifold natural gradient descent (EMNGD) extends energy natural gradient methods from flat parameter spaces to Riemannian manifolds, so hard parameter constraints, such as normalized weight columns, are respected during optimization. The paper's central claim is that when the energy Hessian is coercive, the undamped EMNGD direction, pushed forward to function space, is exactly the best feasible approximation to the function-space Newton vector under the energy metric. This gives a principled way to impose parameter constraints without altering the residual or variational PDE energy. The paper also proves coordinate invariance, exact reduction to ordinary energy natural gradient descent in Euclidean space, global first-order convergence with Armijo backtracking, and an exact sample-space reformulation via the Woodbury identity. A receptive reader would care because the method supplies a geometric answer to constrained neural PDE training, and the reported experiments reach error levels far below first-order baselines.

Core claim

The paper's central claim is Theorem 6: assume D²E(P(x)) is symmetric, bounded, and coercive on the Hilbert space X, and let Nx be the function-space Newton vector solving Hx[Nx, v] = DE(P(x))[v] for all v. If the undamped EMNGD equation has a solution ηx, then the push-forward of that tangent vector satisfies Jxηx = Π^{Hx}_{Sx} Nx, where Sx = Jx(TxM) is the realized tangent subspace and the projection is taken in the energy inner product. In other words, the manifold-restricted natural gradient is not merely an approximation to the Newton step; it is the minimal energy-metric correction that remains feasible. For quadratic energies, this specializes to Jxηx equal to the projected current er

What carries the argument

The object carrying the argument is the damped pullback energy metric g^{E,λ}_x(ξ, ζ) = D²E(P(x))(Jxξ, Jxζ) + λ g⁰_x(ξ, ζ) on the tangent space TxM. The EMNGD direction is the unique tangent vector ηx solving g^{E,λ}_x(ηx, ζ) = dFx[ζ] for all ζ, equivalently ηx = (A^λ_x)⁻¹ grad⁰ F(x). Coercivity supplies the Hilbert-space projection characterization behind the main theorem. For quadratic residual energies or generalized Gauss–Newton pullbacks, the Woodbury (push-through) identity moves the solve to the sample-space kernel Kx = Jx Πx Jx^T without changing the damped direction, and the retraction Rx maps the tangent step back onto the parameter manifold.

Load-bearing premise

The load-bearing premise is that D²E(P(x)) is coercive on the whole Hilbert space X, but the paper's own note says the residual Poisson energy fails coercivity on H², so the main theorem does not formally cover the paper's main experiments.

What would settle it

Take a problem satisfying the coercivity premise, such as a variational Poisson energy on H¹, with a network parameterized on the oblique manifold. Solve the undamped EMNGD system, compute the energy-metric projection of a separately discretized Newton vector onto Jx(TxM), and compare as the discretization refines; disagreement beyond discretization error would falsify identity (33). Conversely, on the residual PINN Poisson setting where coercivity fails, check whether the identity still holds numerically: if it does, the coercivity condition is stronger than necessary; if it fails, the theore

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim is correct, constrained neural PDE solvers can inherit a Newton interpretation: the update is the energy-metric projection of the function-space Newton step onto feasible directions.
  • For quadratic energies, the EMNGD vector is the projected solution error, so each retracted step is a feasible move toward the projected solution correction.
  • Woodbury duality gives an exact sample-space solve whenever the number of residual samples is smaller than the parameter dimension, with the same direction as the parameter-space solve.
  • Nyström preconditioned Krylov solves recover the exact EMNGD direction after convergence, while sketch-and-solve admits a controlled direction error proportional to the kernel approximation error divided by damping squared.
  • Global first-order convergence with Armijo backtracking holds under metric equivalence and retraction smoothness, and inexact tangent solves with relative error below one remain descent directions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the main theorem's coercivity premise is not satisfied by the residual PINN energies used in the experiments—the paper itself notes the Poisson energy inner product is coercive on H^{1/2}, not H²—so the projection identity is formally unavailable for the benchmark setting; weakening the assumption to coercivity on the realized tangent subspace or to the Gauss–Newton pullback i
  • Editorial inference: the kernel Jx Πx Jx^T is a constrained analogue of the tangent kernel, and its spectrum can be tracked during training as a cheap diagnostic for when damping, Nyström rank, or residual subsampling dominates the direction error.
  • Editorial inference: the introductory commutator inequality suggests a concrete comparison: projected ambient ENGD should differ from intrinsic EMNGD exactly when the tangent projector fails to commute with the energy Hessian, and constrained-manifold experiments with normalized weights could quantify that gap.
  • Editorial inference: the paper's evidence-scope section states that most accuracy studies use the Euclidean specialization and that several tables are endpoint coverage rather than matched rankings; the empirical value of the manifold component itself therefore depends on future architecture-matched constrained experiments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Energy Manifold Natural Gradient Descent (EMNGD), an extension of energy natural gradient descent to Riemannian parameter manifolds. The method defines an energy-induced metric on tangent spaces of a parameter manifold, solves a damped linear system for the direction, and applies a retraction to stay on the manifold. The central theoretical result, Theorem 6, states that under a coercivity assumption on the energy Hessian, the push-forward of the undamped EMNGD direction equals the H-energy orthogonal projection of the function-space Newton vector onto the model's tangent-image. The paper also proves coordinate invariance, reduction to ENGD in Euclidean space, global first-order convergence with Armijo backtracking, robustness to inexact tangent solves, an exact Woodbury sample-space equivalence, and a Nyström direction-error bound. Experiments on Poisson, heat, and nonlinear PDE benchmarks report low relative L2 errors and favorable runtimes for EMNGD compared with several baselines.

Significance. If the main theorem and supporting results hold, the paper provides a clean geometric characterization of natural-gradient-type updates under parameter constraints, and the Woodbury/Nyström analysis yields a practical sample-space solver with an explicit error bound. The proofs are short and appear correct under the stated assumptions, and the authors explicitly list several limitations of their empirical setting. The significance is tempered, however, by two gaps: the coercivity assumption of the central theorem does not hold for the residual energies used in the main experiments, and the empirical comparison in the abstract is stronger than what the uncontrolled benchmark protocol supports. These issues are load-bearing for the paper's main claims and require attention before the contribution can be fully assessed.

major comments (3)
  1. [Theorem 6 vs. Section 5.3, Eq. (50), footnote 3] The central projection theorem assumes D²E(P(x)) is symmetric, bounded, and coercive on X. For the residual PINN losses used in the main experiments, this assumption fails: the Poisson energy inner product a(u,v)=∫ΔuΔv+∫_{∂Ω}uv (Eq. 50) is, by the paper's own footnote 3, not coercive on H²(Ω) but only on H^{1/2}(Ω). The exact Hessian of the residual loss (49) has exactly this form. Consequently, N_x in Eq. (32) may not exist uniquely, so the projection identity Eq. (33) lacks a well-defined target. Thus the theorem does not apply to the residual-form benchmarks; the theoretical characterization of EMNGD as a projected Newton step is only valid in the coercive (e.g., variational) setting. Since the algorithm's practical success on residual problems is a central selling point, this scope gap needs to be either closed by an alternative non-coercive analysis or the claims restricted.
  2. [Section 5.1.1, Tables 2–6; Section 6 'Evidence scope'] The abstract claims EMNGD 'achieves higher accuracy and faster convergence' than state-of-the-art baselines. The numerical evidence does not support such a general claim under controlled conditions. Table 2 is a single-seed run; Table 4 imports baselines with 257 parameters while the native EMNGD run uses 8,577 parameters (Section 5.3); Table 5 is described as 'endpoint coverage rather than an architecture-matched ranking'; Table 6 states 'Unequal stopping rules prevent a matched wall-clock or iteration-budget ranking'; and Section 5.1.1 says the studies are 'not wall-clock-matched comparisons.' Section 6 itself concedes that several tables use different architectures, stopping rules, or hardware. The empirical section should either be redone with matched architectures, budgets, and repeated seeds, or the abstract/conclusion should be tempered to report endpoint evidence only.
  3. [Section 5.1.1, Eq. (44); Section 6] The paper's novelty is the manifold extension, using the product-manifold parametrization of Eq. (44), yet the accuracy benchmarks use the Euclidean control M=R^p. The constrained product-manifold setting appears only in diagnostics (e.g., Section 5.2), not in the PDE accuracy comparisons. Section 6 says 'Architecture-matched tests on genuinely constrained neural PDE models will clarify the empirical value of the manifold component beyond the geometric guarantees.' As it stands, the experiments do not demonstrate that the Riemannian manifold component improves accuracy or convergence over the Euclidean specialization; the EMNGD label in Figures 4–6 covers the Euclidean case. The claimed advantage of the manifold construction is therefore unsubstantiated by the presented evidence.
minor comments (5)
  1. [Section 2, Figure 2] The legend lists 'GD ADAM BFGS L-BFGS Adam-L-BFGS' while the text and other captions use 'SGD'; the optimizer naming should be unified.
  2. [Eq. (18) and surrounding text] The Hilbert Gram matrix uses ∂_{θ_i}u_θ, but the realization map is P: M→X. Clarify whether θ denotes local coordinates or ambient parameters in this definition.
  3. [Tables 4 and 5] The imported EMNGD row in Table 4 (error 1.098×10⁻⁶) and the native EMNGD–Woodbury row in Table 5 (error 1.094×10⁻⁸) use different architectures and should be clearly cross-referenced in the text to avoid confusion.
  4. [Throughout] Several LaTeX artifacts remain, e.g., 'M¨uller' and 'M¨ uller' in text and references; a final proofread is needed.
  5. [Section 5.1.2] The hardware and precision details (double precision, RTX 5090 Laptop GPU) are stated once; consider repeating the precision setting in each table caption for self-containment.

Circularity Check

0 steps flagged

No circularity: EMNGD is defined by an independent energy-metric equation and Theorem 6 proves the Newton-projection identity under explicit coercivity assumptions; the coercivity gap is a correctness/scope issue, not a circular one.

full rationale

The EMNGD direction is defined in Definition 4 via equation (10), g^{E,λ}_x(η_x,ζ)=dF_x[ζ], with no reference to the function-space Newton vector. The Newton vector N_x is introduced independently in equation (32), H_x[N_x,v]=DE(P(x))[v], under the explicit coercivity assumption of Theorem 6. The theorem then derives J_x η_x = Π_{S_x}^{H_x} N_x using the chain rule and the Hilbert-space projection characterization; the equivalence is proven rather than assumed. The Woodbury identity (26) is a standard algebraic identity, and the Nyström error bound (29) follows from the resolvent identity; neither contains fitted parameters nor uses the target result as an input. The paper's own footnote 3 admits that the Poisson energy inner product is not coercive on H²(Ω), so Theorem 6 does not apply to the residual-form benchmarks; this is a validity and scope gap, not circular reasoning. There is also no load-bearing self-citation: the cited ENGD and coercivity results are by Müller and Zeinhofer, not by the present authors. The derivation chain is therefore self-contained and non-circular.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central theory rests on standard smooth-manifold assumptions (Assumption 1), on a uniform metric-equivalence and retraction-smoothness bound for convergence (Assumption 10), and on coercivity of the energy Hessian for the Newton projection theorem. Coercivity fails for the residual energies used in the experiments (footnote 3). The Woodbury/Nyström results additionally assume quadratic residual energies or GGN pullbacks. No new entities are introduced.

free parameters (3)
  • damping parameter λ_k = user-selected; values vary across experiments (e.g., λ in Figure 20 ranges 1e-7 to 1e-1)
    Positive damping is required for well-posedness of the tangent solve and appears in the definition of the EMNGD direction (Eq. 10, 15). It is a hyperparameter, not fitted to data, but the central convergence theory (Theorem 11) and Nyström bound (Prop. 5) depend on λ > 0.
  • Nyström rank ℓ = 900 in Fig. 11; not systematically reported otherwise
    Truncation rank for the sample-space kernel approximation; controls direction error in Prop. 5. Chosen by hand, affects accuracy–cost trade-off.
  • Armijo line-search parameters (c, β, α0) = not reported precisely
    Parameters of the backtracking procedure in Algorithm 1; theory requires c∈(0,1), β∈(0,1), α0>0 but exact values are not listed.
axioms (4)
  • standard math Assumption 1 (Geometric and analytic setting): M is a finite-dimensional smooth Riemannian manifold; E is twice Fréchet differentiable; P is twice differentiable; retraction R satisfies Rx(0)=x and dRx(0)=id.
    Defines the setting for all results; standard regularity assumptions for manifold optimization.
  • domain assumption Assumption 10 (Uniform metric equivalence and retraction smoothness): there exist m, M > 0 such that m||ξ||0^2 ≤ g^{E,λ}_x(ξ,ξ) ≤ M||ξ||0^2 on the sublevel set, and a Lipschitz-type bound on F∘R_x.
    Used in Theorem 11 to prove global first-order convergence; not verified in experiments.
  • domain assumption Coercivity of D²E(P(x)) on X (Theorem 6).
    Required for the existence of the function-space Newton vector Nx and the projection identity (33). The paper notes (footnote 3) that the Poisson residual energy inner product is not coercive on H²(Ω), so this axiom fails for a canonical example.
  • domain assumption Quadratic residual energy / GGN pullback for Woodbury identity (Section 3.1).
    The Woodbury duality and Nyström bounds assume the energy Gram is J^T J or J* J with a linear residual map (or GGN). Not valid for general nonlinear residuals.

pith-pipeline@v1.3.0-alltime-deepseek · 25616 in / 11521 out tokens · 107115 ms · 2026-08-01T06:04:06.362504+00:00 · methodology

0 comments
read the original abstract

Energy natural gradient descent (ENGD) aligns parameter updates with the curvature of an underlying function-space energy, but existing formulations assume an unconstrained Euclidean parameter domain. We introduce \EMNGDfull{}, a manifold optimization framework for physics-informed and variational neural PDE solvers whose parameters lie on a Riemannian manifold. EMNGD restricts the energy-induced quadratic model to feasible tangent directions and uses retractions to preserve parameter constraints throughout optimization. Under coercivity, we prove that the push-forward of the undamped EMNGD direction is the best feasible approximation to the function-space Newton vector in the energy metric. We establish coordinate invariance, exact reduction to ENGD in Euclidean space, global first-order convergence with Armijo backtracking, and robustness to inexact tangent solves. For quadratic residual energies and generalized Gauss--Newton pullbacks, the Woodbury identity transfers the tangent system to sample space without changing the direction. Nystr\"om approximation provides scalable sample-space solves with controlled direction error and recovers the exact direction after iterative convergence. On the evaluated neural PDE benchmarks, EMNGD achieves higher accuracy and faster convergence than the compared state-of-the-art baselines. Woodbury preserves the EMNGD direction, while scalable-solver diagnostics quantify the accuracy--cost trade-off of preconditioning and residual subsampling.

Figures

Figures reproduced from arXiv: 2607.22004 by Huanhuan Gao, Zhangyong Liang.

Figure 1
Figure 1. Figure 1: Schematic of an EMNGD update on a parameter manifold. A tangent step −αkηxk at xk is retracted to the feasible iterate xk+1. tangent point xek+1 = xk − αkηxk to the feasible iterate xk+1. The construction separates parameter constraints from function-space energy geometry. Contributions. The contributions are as follows: • Intrinsic energy manifold geometry. EMNGD extends ENGD from an uncon￾strained Euclid… view at source ↗
Figure 2
Figure 2. Figure 2: Relative L 2 errors of standard optimizers on one-dimensional PDE benchmarks. slows iterative solvers such as gradient descent [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: EMNGD update on a parameter manifold. From left to right, EMNGD solves the energy-metric equation in TxkM, retracts −αkηxk to xk+1 ∈ M, and repeats the update toward x ∗ . Then the coordinate vector v satisfies (G ϕ E + λGϕ 0 )v = b. (15) For λ > 0, the tangent system has a unique solution and the tangent vector ηx is independent of the chosen chart. For M = R p with the Euclidean metric, P(θ) = uθ, and Rθ… view at source ↗
Figure 4
Figure 4. Figure 4: Training loss (top) and final relative L 2 error (bottom) across PDE benchmarks. 10 0 10 1 10 2 10 3 10 4 10 5 Iteration 10 11 10 8 10 5 10 2 R elativ e L2 error Poisson equation 10 0 10 1 10 2 10 3 10 4 10 5 Iteration 10 11 10 8 10 5 10 2 Heat equation 10 0 10 1 10 2 10 3 10 4 10 5 Iteration 10 10 10 7 10 4 10 1 Nonlinear equation GD ADAM BFGS ENGD ENGD (woodbury) EMNGD [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figure 5
Figure 5. Figure 5: One-dimensional PDE benchmark. 5.1.2 Implementation Details We implement the solvers in JAX (Bradbury et al., 2018) with automatic differentiation. Least-squares solves use singular-value decomposition. BFGS uses jaxopt.BFGS. Unless stated otherwise, experiments run in double precision on one NVIDIA RTX 5090 Laptop GPU. The implementation is available at https://github.com/liangzhangyong/EMNGD. 5.2 Residua… view at source ↗
Figure 6
Figure 6. Figure 6: One-dimensional Poisson solutions and pointwise errors (log scale). [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Residual-Fisher geometry for hard-Dirichlet EMNGD. [PITH_FULL_IMAGE:figures/full_fig_p021_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Residual Jacobians for Woodbury EMNGD on two-dimensional Poisson. [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Layerwise contributions to the residual Gramian. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Sample-space residual-Gramian spectrum for 128 residual samples. [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: EMNGD with an exact Woodbury solve and rank-900 Nystr¨om preconditioning. Left: residual loss. Right: relative L 2 error. and the PINNs loss of the problem is L(θ) = 1 NΩ X NΩ i=1 (∆uθ(xi , yi) + f(xi , yi))2 + 1 N∂Ω N X∂Ω i=1 uθ(x b i , yb i ) 2 , (49) where {(xi , yi)}i=1,...,NΩ denote the interior collocation points and {(x b i , yb i )}i=1,...,N∂Ω denote the collocation points on ∂Ω. For the Poisson p… view at source ↗
Figure 12
Figure 12. Figure 12: Weight-sharing approximation for an EMNGD residual-Jacobian block. [PITH_FULL_IMAGE:figures/full_fig_p024_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Two-dimensional Poisson relative L 2 error across three parameter dimensions [PITH_FULL_IMAGE:figures/full_fig_p026_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Five-dimensional Poisson: relative L 2 error versus iteration and time. The solution is given by u ∗ (t, x) = exp  − π 2 t 4  sin(πx), 27 [PITH_FULL_IMAGE:figures/full_fig_p027_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Loss and relative L 2 error for five-dimensional Poisson. and the PINNs loss is L(θ) = 1 NΩT N XΩT i=1  ∂tuθ(ti , xi) − 1 4 ∂ 2 xuθ(ti , xi) 2 + 1 Nin X NΩ i=1 [PITH_FULL_IMAGE:figures/full_fig_p028_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Large-sample five-dimensional Poisson benchmark. [PITH_FULL_IMAGE:figures/full_fig_p029_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Metric comparison for five-dimensional Poisson. [PITH_FULL_IMAGE:figures/full_fig_p030_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: One-dimensional heat relative L 2 error across three parameter dimensions. 6 Discussion Feasible and energy geometry. EMNGD combines two distinct structures. The param￾eter manifold defines admissible local variations, and retractions preserve feasibility of finite updates. The pullback energy metric ranks the admissible variations by their function-space effects. EMNGD therefore minimizes the energy-indu… view at source ↗
Figure 19
Figure 19. Figure 19: Woodbury and Nystr¨om diagnostics for tangent-solver scaling and agreement. [PITH_FULL_IMAGE:figures/full_fig_p032_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Sample-kernel growth, damping sensitivity, and residual-subsampling effects. [PITH_FULL_IMAGE:figures/full_fig_p032_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Residual-Jacobian sensitivity to damping, sampling, and Nystr¨om rank. [PITH_FULL_IMAGE:figures/full_fig_p033_21.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 1 canonical work pages

  1. [7]

    On the promise of the stochastic generalized gauss-newton method for training dnns.arXiv preprint arXiv:2006.02409,

    Matilde Gargiani, Andrea Zanelli, Moritz Diehl, and Frank Hutter. On the promise of the stochastic generalized gauss-newton method for training dnns.arXiv preprint arXiv:2006.02409,

    Pith/arXiv arXiv 2006

  2. [8]

    An efficient greedy training algorithm for neural networks and applications in PDEs.arXiv preprint arXiv:2107.04466,

    Wenrui Hao, Xianlin Jin, Jonathan W Siegel, and Jinchao Xu. An efficient greedy training algorithm for neural networks and applications in PDEs.arXiv preprint arXiv:2107.04466,

    Pith/arXiv arXiv

  3. [9]

    Gauss-Newton natural gradient descent for physics-informed computational fluid dynamics.arXiv preprint arXiv:2402.10680,

    Anas Jnini, Flavio Vella, and Marius Zeinhofer. Gauss-Newton natural gradient descent for physics-informed computational fluid dynamics.arXiv preprint arXiv:2402.10680,

    Pith/arXiv arXiv

  4. [11]

    Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar

    URLhttps://doi.org/10.1007/s41884-018-0015-3. Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. InInternational Conference on Learning Representations,

    work page doi:10.1007/s41884-018-0015-3

  5. [13]

    Johannes M¨ uller and Marius Zeinhofer

    URL https://www.mis.mpg.de/publications/ preprints/2022/prepr2022-31.html. Johannes M¨ uller and Marius Zeinhofer. Error estimates for the deep ritz method with boundary penalty. InMathematical and Scientific Machine Learning, pages 215–230. PMLR, 2022a. Johannes M¨ uller and Marius Zeinhofer. Notes on exact boundary values in residual min- imisation. InM...

    2022

  6. [15]

    Efficient natural gradient descent methods for large-scale optimization problems.arXiv:2202.06236,

    Levon Nurbekyan, Wanzhou Lei, and Yunan Yang. Efficient natural gradient descent methods for large-scale optimization problems.arXiv:2202.06236,

    Pith/arXiv arXiv

  7. [16]

    Efficient subsampled gauss-newton and natural gradient methods for training neural networks.arXiv preprint arXiv:1906.02353,

    Yi Ren and Donald Goldfarb. Efficient subsampled gauss-newton and natural gradient methods for training neural networks.arXiv preprint arXiv:1906.02353,

    Pith/arXiv arXiv 1906

  8. [17]

    ¨Uber eine neue Methode zur L¨ osung gewisser Variationsprobleme der mathe- matischen Physik.Journal f¨ ur die reine und angewandte Mathematik (Crelles Journal), 1909(135):1–61,

    Walter Ritz. ¨Uber eine neue Methode zur L¨ osung gewisser Variationsprobleme der mathe- matischen Physik.Journal f¨ ur die reine und angewandte Mathematik (Crelles Journal), 1909(135):1–61,

    1909

  9. [19]

    Respecting causality is all you need for training physics-informed neural networks.arXiv preprint arXiv:2203.07404, 2022a

    Sifan Wang, Shyam Sankaran, and Paris Perdikaris. Respecting causality is all you need for training physics-informed neural networks.arXiv preprint arXiv:2203.07404, 2022a. Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why PINNs fail to train: A neural tangent kernel perspective.Journal of Computational Physics, 449:110768, 2022b. E Weinan, Jiequ...

    Pith/arXiv arXiv

  10. [20]

    Competitive physics informed networks

    Qi Zeng, Spencer H Bryngelson, and Florian Tobias Schaefer. Competitive physics informed networks. InICLR 2022 Workshop on Gamification and Multiagent Solutions,

    2022

  11. [2001]

    Neural operator: Learning maps between function spaces.arXiv preprint arXiv:2108.08481,

    Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces.arXiv preprint arXiv:2108.08481,

    Pith/arXiv arXiv

  12. [2002]

    An iteration count estimate for a mesh-dependent steepest descent method based on finite elements and Riesz inner product representation.arXiv preprint arXiv:1606.08069,

    Tobias Schwedes, Simon W Funke, and David A Ham. An iteration count estimate for a mesh-dependent steepest descent method based on finite elements and Riesz inner product representation.arXiv preprint arXiv:1606.08069,

    Pith/arXiv arXiv

  13. [2003]

    An overview on deep learning-based approximation methods for partial differential equations.arXiv preprint arXiv:2012.12348,

    Christian Beck, Martin Hutzenthaler, Arnulf Jentzen, and Benno Kuckuck. An overview on deep learning-based approximation methods for partial differential equations.arXiv preprint arXiv:2012.12348,

    Pith/arXiv arXiv 2012

  14. [2008]

    Geometry and convergence of natural policy gradi- ents.MPI MiS Preprint 31/2022,

    Johannes M¨ uller and Guido Mont´ ufar. Geometry and convergence of natural policy gradi- ents.MPI MiS Preprint 31/2022,

    2022

  15. [2018]

    Tianle Cai, Ruiqi Gao, Jikai Hou, Siyu Chen, Dong Wang, Di He, Zhihua Zhang, and Liwei Wang

    URL http://github.com/google/jax. Tianle Cai, Ruiqi Gao, Jikai Hou, Siyu Chen, Dong Wang, Di He, Zhihua Zhang, and Liwei Wang. Gram-gauss-newton method: Learning overparameterized neural networks for regression problems.arXiv preprint arXiv:1905.11675,

    Pith/arXiv arXiv 1905

  16. [2019]

    Efficient optimization of deep neural quantum states toward machine precision.arXiv preprint arXiv:2302.01941,

    Ao Chen and Markus Heyl. Efficient optimization of deep neural quantum states toward machine precision.arXiv preprint arXiv:2302.01941,

    Pith/arXiv arXiv

  17. [2021]

    Nystr¨ om approximation on manifolds.arXiv preprint arXiv:2605.14933,

    Hantao Nie, Bin Gao, Andi Han, Pratik Jawanpuria, Bamdev Mishra, and Zaiwen Wen. Nystr¨ om approximation on manifolds.arXiv preprint arXiv:2605.14933,

    Pith/arXiv arXiv

  18. [2022]

    Rethinking the impor- tance of sampling in physics-informed neural networks.arXiv preprint arXiv:2207.02338,

    34 Energy Manifold Natural Gradient Descent Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Anuj Karpatne. Rethinking the impor- tance of sampling in physics-informed neural networks.arXiv preprint arXiv:2207.02338,

    Pith/arXiv arXiv

  19. [2023]

    Kronecker-factored approximate curvature for physics-informed neural networks.arXiv preprint arXiv:2405.15603,

    Felix Dangel, Johannes M¨ uller, and Marius Zeinhofer. Kronecker-factored approximate curvature for physics-informed neural networks.arXiv preprint arXiv:2405.15603,

    Pith/arXiv arXiv

  20. [2024]

    Pso-pinn: Physics-informed neural networks trained with particle swarm optimization.arXiv preprint arXiv:2202.01943,

    Caio Davi and Ulisses Braga-Neto. Pso-pinn: Physics-informed neural networks trained with particle swarm optimization.arXiv preprint arXiv:2202.01943,

    Pith/arXiv arXiv