pith. sign in

arxiv: 2606.04018 · v2 · pith:KBLGFOMGnew · submitted 2026-06-01 · 🧮 math.NA · cs.NA

The Coercivity Gap in Neural PDE Solvers: Parameter Escape and Functional Convergence

Enrique Zuazua This is my paper

Pith reviewed 2026-06-30 10:53 UTC · model grok-4.3

classification 🧮 math.NA cs.NA
keywords neural PDE solverscoercivity gapelliptic variational problemsparameter spacefunctional convergenceneuron condensationPINN methodsGaussian wave-packets
0
0 comments X

The pith

Even when an elliptic energy is coercive in function space, its restriction to a neural ansatz can lose coercivity in parameters while the states still converge to the PDE solution.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper distinguishes the geometry of neural parameters from the convergence of the physical states they produce when approximating elliptic PDE solutions variationally. It establishes that non-closed neural manifolds and neuron condensation can destroy coercivity in parameter space even if the original energy is coercive and strictly convex. Despite this, the state functions can stay bounded and converge strongly to the exact solution. The mechanism is shown explicitly for Gaussian wave-packet approximations, with rates derived, and the same state-level stability is argued to hold for PINN residual methods and HYCO hybrids. Regularization approaches are discussed as remedies.

Core claim

The central claim is that the restriction of a coercive elliptic energy to a nonlinear neural ansatz may fail to be coercive in parameter space due to non-closedness of the approximation manifold and neuron condensation that generates limiting profiles outside the fixed ansatz class, yet the associated state functions remain bounded and converge strongly to the exact PDE solution. This is proven for Gaussian wave-packet approximations of a model elliptic problem in the whole space, with explicit convergence rates, and the state-level principle is shown to extend to residual-minimization methods of PINN type and to HYCO-type hybrid methods.

What carries the argument

Non-closedness of neural approximation manifolds that permits neuron condensation to limiting profiles outside the fixed ansatz class.

If this is right

  • State functions remain bounded and converge strongly to the PDE solution even when parameters escape to infinity.
  • Explicit convergence rates hold for Gaussian wave-packet approximations of the model problem.
  • The same state-level stability principle applies directly to residual-minimization methods of PINN type.
  • The principle also applies to HYCO-type hybrid methods.
  • Relaxation and Tikhonov regularization can restore well-posedness at the parameter level.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Training procedures may succeed by tracking state convergence rather than parameter boundedness.
  • Similar coercivity gaps are likely in other nonlinear approximation families that admit condensation.
  • The distinction motivates state-aware stopping criteria or hybrid regularizers that act on the output functions.

Load-bearing premise

Neural approximation manifolds are non-closed and permit neuron condensation that produces limiting profiles outside the fixed ansatz class.

What would settle it

A concrete counter-example in which a Gaussian wave-packet neural ansatz for the model elliptic problem produces states that fail to converge strongly to the exact solution while parameters escape would falsify the central claim.

Figures

Figures reproduced from arXiv: 2606.04018 by Enrique Zuazua.

Figure 1
Figure 1. Figure 1: The coercivity gap. Parameter sequences may escape in the nonlinear ansatz, while the corre￾sponding state functions remain controlled by the coercive PDE energy and may converge strongly in the natural energy space. resulting optimization problems are typically nonconvex and may fail to be coercive with respect to the neural parameters; see, for instance, [4, 12, 21]. The central theme of this article is … view at source ↗
Figure 2
Figure 2. Figure 2: The explicit collision in state space for d = 1. The two-neuron difference quotient qh = (G(· + h) − G(· − h))/(2h) approaches the smooth state G′ as the centers collapse and the weights grow like h −1 . The lower panel makes the same point quantitatively: the parameter norm increases while the state error decreases. Proposition 4.1 (Nondegenerate collision). As h ↓ 0, qh −→ ∂1G strongly in H1 (R d ), wher… view at source ↗
read the original abstract

We study neural approximation of elliptic PDE solutions from a variational perspective. The central point is the distinction between the geometry of neural parameters and the convergence of the corresponding physical states. Even when the original elliptic energy is coercive and strictly convex in the natural energy space, its restriction to a nonlinear neural ansatz may fail to be coercive in parameter space. This failure is caused by non-closedness of neural approximation manifolds and by condensation of neurons, which may generate limiting profiles outside the fixed ansatz class. Nevertheless, the associated state functions may remain bounded and converge strongly to the exact PDE solution. We prove this mechanism for Gaussian wave-packet approximations of a prototypical elliptic model in the whole space, derive convergence rates, and explain how the same state-level stability principle applies to residual minimization methods of PINN type, and HYCO-type hybrid methods. We also discuss relaxation and Tikhonov regularization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper claims that the restriction of a coercive, strictly convex elliptic energy to a nonlinear neural ansatz need not be coercive in parameter space, owing to non-closedness of the approximation manifold and neuron condensation that can produce limiting profiles outside the ansatz class. Nevertheless, the associated state functions remain bounded and converge strongly to the exact PDE solution. The mechanism is proved for Gaussian wave-packet approximations of a model elliptic problem on the whole space, with explicit convergence rates derived; the same state-level stability is then invoked for residual-minimization methods of PINN type and for HYCO-type hybrids. Relaxation and Tikhonov regularization are also discussed.

Significance. If the central distinction between parameter escape and functional convergence holds, the work supplies a useful theoretical lens for understanding why neural PDE solvers can succeed even when the restricted energy lacks coercivity. The explicit Gaussian-wave-packet construction furnishes a concrete, analyzable example, while the extension to residual methods offers direct guidance for PINN-type algorithms. Derivation of convergence rates adds quantitative content that is often missing from neural-PDE analyses.

minor comments (2)
  1. Abstract: the acronym 'HYCO' is introduced without expansion or reference; a parenthetical definition or citation would improve accessibility for readers outside the immediate subfield.
  2. The transition from the Gaussian-wave-packet analysis to the general residual-minimization setting (presumably §5 or §6) would benefit from an explicit statement of the hypotheses under which the state-level stability carries over verbatim.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of the manuscript, the recognition of its significance, and the recommendation of minor revision. No specific major comments were raised in the report.

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained

full rationale

The paper distinguishes non-coercivity of the restricted energy on the neural parameter manifold (due to non-closedness and neuron condensation) from strong convergence of state functions. This separation rests on standard variational properties of elliptic energies and an explicit Gaussian wave-packet analysis that produces limiting profiles outside the ansatz while states remain bounded. No step reduces a claimed prediction or convergence result to a fitted quantity, self-definition, or load-bearing self-citation chain. The argument is independent of the target result and does not invoke uniqueness theorems or ansatzes from prior author work in a circular manner.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review performed on abstract only; full list of background assumptions and any ad-hoc choices cannot be extracted.

axioms (1)
  • domain assumption The elliptic energy functional is coercive and strictly convex in the natural energy space.
    Stated as the starting point whose restriction to neural ansatzes is then analyzed.

pith-pipeline@v0.9.1-grok · 5678 in / 1244 out tokens · 48819 ms · 2026-06-30T10:53:39.691684+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 12 canonical work pages · 3 internal anchors

  1. [1]

    Bach,Breaking the curse of dimensionality with convex neural networks, J

    F. Bach,Breaking the curse of dimensionality with convex neural networks, J. Mach. Learn. Res.18 (2017), 1–53

    2017

  2. [2]

    Bellomo, F

    N. Bellomo, F. Brezzi and E. Zuazua,New trends in mathematics for scientific machine learning, Math. Models Methods Appl. Sci.36(2026), no. 8, 1615–1621, doi:10.1142/S0218202526020033

    work page doi:10.1142/s0218202526020033 2026

  3. [3]

    Bertoluzza, E

    S. Bertoluzza, E. Burman and C. He,W AN discretization of PDEs: Best approximation, stabilization, and essential boundary conditions, SIAM J. Sci. Comput.46(2024), C688–C715

    2024

  4. [4]

    M. D. Buhmann,Radial basis functions, Acta Numer.9(2000), 1–38

    2000

  5. [5]

    Z. Chen, L. Huang, M. Yang and S. Zhou,Regularity of second-order elliptic PDEs in spectral Barron spaces, arXiv:2602.19381, 2026

    work page arXiv 2026

  6. [6]

    Chizat and F

    L. Chizat and F. Bach,On the global convergence of gradient descent for over-parameterized models using optimal transport, Adv. Neural Inf. Process. Syst.31(2018), 3036–3046

    2018

  7. [7]

    Dal Maso,An Introduction to Γ-Convergence, Progress in Nonlinear Differential Equations and Their Applications, vol

    G. Dal Maso,An Introduction to Γ-Convergence, Progress in Nonlinear Differential Equations and Their Applications, vol. 8, Birkhäuser, Boston, 1993

    1993

  8. [8]

    De Ryck, A

    T. De Ryck, A. D. Jagtap and S. Mishra,Error estimates for physics-informed neural networks approxi- mating the Navier–Stokes equations, IMA J. Numer. Anal.44(2024), 83–119. 30

    2024

  9. [9]

    Doumèche, G

    N. Doumèche, G. Biau and C. Boyer,On the convergence of physics-informed neural networks, Bernoulli 31(2025), 2127–2151

    2025

  10. [10]

    Dondl, J

    P. Dondl, J. Müller and M. Zeinhofer,Uniform convergence guarantees for the Deep Ritz method for nonlinear problems, Adv. Contin. Discrete Models2022(2022), article 49

    2022

  11. [11]

    W. E and B. Yu,The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat.6(2018), 1–12

    2018

  12. [12]

    Fernández and E

    D. Fernández and E. Zuazua,Quantitative conditioning and parameter escape for Gaussian neural PDE solvers, work in preparation

  13. [13]

    Gazoulis, I

    D. Gazoulis, I. Gkanis and C. G. Makridakis,On the stability and convergence of physics informed neural networks, IMA J. Numer. Anal. (2025), article draf090, doi:10.1093/imanum/draf090

    work page doi:10.1093/imanum/draf090 2025

  14. [14]

    Girosi and T

    F. Girosi and T. Poggio,Networks and the best approximation property, Biol. Cybern.63(1990), 169–176

    1990

  15. [15]

    L. I. Ignat and E. Zuazua,Optimal convergence rates for the finite element approximation of the Sobolev constantarXiv:2504.09637, (2025), Foundations of Computational Mathematics, to appear

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [16]

    arXiv preprint arXiv:2509.14123

    L. Liverani, T. Steynberg and E. Zuazua,HYCO: Hybrid-cooperative learning for data-driven PDE modeling, arXiv:2509.14123, 2025

    work page arXiv 2025

  17. [17]

    HYCO: A Formalism for Hybrid-Cooperative PDE Modelling

    L. Liverani and E. Zuazua,HYCO: A formalism for hybrid-cooperative PDE modelling, arXiv:2602.23859, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  18. [18]

    P. C. Kainen, V. Kůrková and A. Vogt,Approximation by neural networks is not continuous, Neurocom- puting29(1999), 47–56

    1999

  19. [19]

    G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang and L. Yang,Physics-informed machine learning, Nat. Rev. Phys.3(2021), 422–440

    2021

  20. [20]

    Non-Uniqueness of Solutions in Neural Variational Methods

    A. Langer,Non-uniqueness of solutions in neural variational methods, arXiv:2605.08877, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  21. [21]

    Q.-T. Le, R. Gribonval and E. Riccietti,Does a sparse ReLU network training problem always admit an optimum?, Adv. Neural Inf. Process. Syst.36(2023)

    2023

  22. [22]

    Liu and E

    K. Liu and E. Zuazua,Moments, time inversion and source identification for the heat equation, Inverse Problems42(2026), 015009

    2026

  23. [23]

    Lloyd,Least squares quantization in PCM, IEEE Trans

    S. Lloyd,Least squares quantization in PCM, IEEE Trans. Inform. Theory28(1982), 129–137

    1982

  24. [24]

    Luo, Z.-Q

    T. Luo, Z.-Q. J. Xu, Z. Ma and Y. Zhang,Phase diagram for two-layer ReLU neural networks at infinite- width limit, J. Mach. Learn. Res.22(2021), 1–47

    2021

  25. [25]

    Luo and H

    T. Luo and H. Yang,Two-layer neural networks for partial differential equations: Optimization and generalization theory, Handbook Numer. Anal.25(2024), 515–554

    2024

  26. [26]

    W. R. Madych and S. A. Nelson,Bounds on multivariate polynomials and exponential error estimates for multiquadric interpolation, J. Approx. Theory70(1992), 94–114

    1992

  27. [27]

    Mahan, E

    S. Mahan, E. J. King and A. Cloninger,Nonclosedness of sets of neural networks in Sobolev spaces, Neural Netw.137(2021), 85–96

    2021

  28. [28]

    Mishra and R

    S. Mishra and R. Molinaro,Estimates on the generalization error of physics-informed neural networks for approximating PDEs, IMA J. Numer. Anal.43(2023), 1–43

    2023

  29. [29]

    Müller and M

    J. Müller and M. Zeinhofer,Deep Ritz revisited, arXiv:1912.03937, 2019

    work page arXiv 1912

  30. [30]

    Müller and M

    J. Müller and M. Zeinhofer,Achieving high accuracy with PINNs via energy natural gradient descent, Proc. Mach. Learn. Res.202(2023), 25471–25485. 31

    2023

  31. [31]

    Petersen, M

    P. Petersen, M. Raslan and F. Voigtlaender,Topological properties of the set of functions generated by neural networks of fixed size, Found. Comput. Math.21(2021), 375–444

    2021

  32. [32]

    Raissi, P

    M. Raissi, P. Perdikaris and G. E. Karniadakis,Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys.378(2019), 686–707

    2019

  33. [33]

    Rojas, P

    S. Rojas, P. Maczuga, J. Muñoz-Matute, D. Pardo and M. Paszyński,Robust variational physics-informed neural networks, Comput. Methods Appl. Mech. Engrg.425(2024), 116904, doi:10.1016/j.cma.2024.116904

    work page doi:10.1016/j.cma.2024.116904 2024

  34. [34]

    Savarese, I

    P. Savarese, I. Evron, D. Soudry and N. Srebro,How do infinite width bounded norm networks look in function space?, Proc. Mach. Learn. Res.99(2019), 2667–2690

    2019

  35. [35]

    Y. Shin, J. Darbon and G. E. Karniadakis,On the convergence of physics-informed neural networks for linear second-order elliptic and parabolic type PDEs, Commun. Comput. Phys.28(2020), 2042–2074

    2020

  36. [36]

    S. Wang, S. Sankaran, H. Wang and P. Perdikaris,An expert’s guide to training physics-informed neural networks, arXiv:2308.08468, 2023

    work page arXiv 2023

  37. [37]

    Wendland,Scattered Data Approximation, Cambridge University Press, Cambridge, 2005

    H. Wendland,Scattered Data Approximation, Cambridge University Press, Cambridge, 2005

    2005

  38. [38]

    Wiener,Tauberian theorems, Ann

    N. Wiener,Tauberian theorems, Ann. of Math.33(1932), 1–100

    1932

  39. [39]

    H. Zhou, Q. Zhou, T. Luo, Y. Zhang and Z.-Q. J. Xu,Towards understanding the condensation of neural networks at initial training, Adv. Neural Inf. Process. Syst.35(2022), 2184–2196

    2022

  40. [40]

    arXiv preprint arXiv:2505.14002

    W. Zhao and T. Luo,Convergence guarantees for gradient-based training of neural PDE solvers: From linear to nonlinear PDEs, arXiv:2505.14002, 2025

    work page arXiv 2025

  41. [41]

    Zuazua,Propagation, observation, and control of waves approximated by finite difference methods, SIAM Rev.47(2005), 197–243

    E. Zuazua,Propagation, observation, and control of waves approximated by finite difference methods, SIAM Rev.47(2005), 197–243

    2005

  42. [42]

    arXiv preprint arXiv:2003.11834

    E. Zuazua,Asymptotic behavior of scalar convection–diffusion equations, arXiv:2003.11834, 2020. 32

    work page arXiv 2003