pith. sign in

arxiv: 2605.04126 · v2 · pith:CXUV5FHRnew · submitted 2026-05-05 · 💻 cs.LG · cs.NA· math.NA

Simultaneous CNN Approximation on Manifolds with Applications to Boundary Value Problems

Hanfei Zhou , Lei Shi This is my paper

Pith reviewed 2026-07-01 00:10 UTC · model grok-4.3

classification 💻 cs.LG cs.NAmath.NA
keywords convolutional neural networksRiemannian manifoldsSobolev approximationboundary value problemsphysics-informed neural networksLaplace-Beltrami eigenmodestrace norms
0
0 comments X

The pith

CNNs approximate functions on manifolds at rates set by intrinsic dimension and solve elliptic boundary problems with a spectral loss.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proves that single- and multichannel convolutional neural networks can approximate target functions on compact Riemannian manifolds in Sobolev norms, where the convergence rates depend on the manifold's intrinsic dimension and the difference in smoothness between the target and the network class. It constructs a physics-informed training framework whose boundary loss expands the residual in Laplace-Beltrami eigenmodes on the boundary and weights them by Sobolev trace factors so that the penalty matches the natural trace norm required by elliptic theory. This construction separates approximation error, generalization error, and spectral truncation error, and numerical tests on hemispheres and half-tori show one-to-two-order gains over standard PINNs when boundary data contain high frequencies.

Core claim

Convolutional neural networks achieve simultaneous Sobolev approximation on manifolds with rates governed by intrinsic dimension and smoothness gap; the associated spectral boundary loss, formed by eigenmode expansion and trace-norm weights, aligns with the H^{2s-1/2} boundary norm for 2s-order elliptic problems and yields an error decomposition consistent with fast-rate generalization bounds.

What carries the argument

Spectral boundary loss that expands the boundary residual in boundary Laplace-Beltrami eigenmodes and penalizes the coefficients by Sobolev trace weights.

If this is right

  • Approximation and generalization errors decouple from spectral truncation error under the stated loss.
  • Both single-channel and multichannel CNNs inherit the same dimension-dependent rates.
  • The loss permits FFT-based or precomputed implementations without singular double integrals or auxiliary boundary extensions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same eigenmode-weighted loss could be tested on other linear elliptic operators whose natural boundary norms admit spectral characterizations.
  • If the rates truly track intrinsic dimension, the method should show larger gains on data supported on lower-dimensional submanifolds embedded in higher-dimensional space.
  • Precomputed eigenbases on standard manifolds make the approach immediately usable for any 2s-order problem whose trace space is captured by the same weights.

Load-bearing premise

The chosen eigenmode expansion together with the trace weights reproduces the exact Sobolev trace norm of the boundary residual without further manifold regularity or discretization mismatch.

What would settle it

A manifold and target function where measured CNN approximation rates fail to improve when intrinsic dimension drops, or where the spectral-loss method shows no accuracy gain over standard PINN training on high-frequency boundary data.

Figures

Figures reproduced from arXiv: 2605.04126 by Hanfei Zhou, Lei Shi.

Figure 1
Figure 1. Figure 1: Empirical convergence curves (log–log) and fitted slopes on two manifolds view at source ↗
Figure 2
Figure 2. Figure 2: We compare the function error |uθ − u| and Laplacian error |∆Muθ − ∆Mu| across all four settings. To study the optimization behavior, we track the test RelL 2 and Rel H2 errors over epochs on both MS and MT. With all other hyperparameters fixed, we compare the Sobolev penalty with the standard L 2 penalty. Empirically, the Sobolev penalty L H3/2 bnd yields faster 13 view at source ↗
Figure 2
Figure 2. Figure 2: Pointwise error visualization at N = 4096: left/right subplots correspond to the spatial distributions of |uθ−u| and |∆Muθ−∆Mu|, comparing four combinations of boundary penalty and manifold type. error decay in the early stage and smoother convergence trajectories view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of test L 2 error versus epoch under different interior sample sizes. (a) RelL 2 , N = 16384 (b) RelL 2 , N = 32768 (c) Rel H2s , N = 16384 (d) Rel H2s , N = 32768 view at source ↗
Figure 4
Figure 4. Figure 4: Channel-width sweep results on the upper-torus manifold view at source ↗
read the original abstract

This paper develops convolutional neural network (CNN) methods for simultaneous Sobolev approximation and elliptic boundary value problems on compact Riemannian manifolds. We prove approximation estimates for single- and multichannel CNNs, with rates governed by the intrinsic dimension and the smoothness gap. Motivated by elliptic stability, we propose a physics-informed CNN framework with a spectral boundary loss. The boundary residual is expanded in boundary Laplace--Beltrami eigenmodes and penalized by Sobolev trace weights, matching the natural \(\mathcal H^{2s-1/2}(\partial\mathcal M^d)\) trace norm for \(2s\)-order elliptic problems. This avoids smooth auxiliary constructions for exact boundary enforcement and singular Sobolev--Slobodeckij double integrals, while allowing FFT-based or precomputed spectral implementations. We also derive an error decomposition separating approximation, generalization, and spectral truncation errors, showing that the proposed loss is aligned with localized fast-rate generalization analysis. Numerical experiments on the upper hemisphere and upper half-torus demonstrate improved accuracy, convergence, and stability over standard PINNs, with one to two orders of magnitude gains for high-frequency boundary data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript develops CNN-based methods for simultaneous Sobolev approximation on compact Riemannian manifolds and for solving 2s-order elliptic boundary value problems. It proves approximation rates for single- and multichannel CNNs governed by intrinsic dimension and smoothness gap, proposes a physics-informed framework whose spectral boundary loss expands the residual in boundary Laplace-Beltrami eigenmodes and applies Sobolev trace weights to match the natural H^{2s-1/2}(∂M^d) trace norm, derives an error decomposition separating approximation, generalization, and truncation terms, and reports numerical gains of one to two orders of magnitude over standard PINNs on the upper hemisphere and upper half-torus, especially for high-frequency boundary data.

Significance. If the norm-equivalence claim and the supporting analysis hold, the work supplies a theoretically grounded alternative to auxiliary boundary constructions or singular double-integral penalties, together with explicit rates and a decomposition aligned with fast-rate generalization. The combination of manifold CNN approximation theory, spectral loss design, and reproducible numerical comparisons on non-Euclidean domains constitutes a substantive contribution to physics-informed learning on manifolds.

major comments (2)
  1. [Abstract; spectral boundary loss section] Abstract and the section introducing the spectral boundary loss: the claim that the eigenmode expansion penalized by Sobolev trace weights is equivalent (with manifold-independent constants) to the full H^{2s-1/2}(∂M^d) trace norm is asserted without a detailed verification of eigenbasis completeness or reproduction of the fractional seminorm on manifolds of only C^{2s} regularity. This equivalence is load-bearing for both the elliptic-stability motivation and the subsequent error decomposition.
  2. [Error decomposition section] Error-decomposition section: the separation into approximation, generalization, and spectral-truncation errors, together with the assertion that the proposed loss aligns with localized fast-rate generalization, presupposes that the spectral loss controls the stability term with constants independent of the manifold and the frequency content. If the trace-norm equivalence fails to hold at the required strength, the decomposition no longer supports the claimed alignment.
minor comments (2)
  1. [Introduction] The notation distinguishing intrinsic dimension d, manifold dimension, and the order 2s should be introduced with a single consistent table or paragraph early in the manuscript.
  2. [Numerical experiments] Numerical-experiment figures would benefit from explicit labels indicating the Sobolev index s and the highest frequency retained in the boundary data for each curve.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments on the manuscript. We address the two major comments point by point below and will revise the paper accordingly.

read point-by-point responses

  1. Referee: [Abstract; spectral boundary loss section] Abstract and the section introducing the spectral boundary loss: the claim that the eigenmode expansion penalized by Sobolev trace weights is equivalent (with manifold-independent constants) to the full H^{2s-1/2}(∂M^d) trace norm is asserted without a detailed verification of eigenbasis completeness or reproduction of the fractional seminorm on manifolds of only C^{2s} regularity. This equivalence is load-bearing for both the elliptic-stability motivation and the subsequent error decomposition.

    Authors: We appreciate the referee highlighting the need for explicit verification. The completeness of the Laplace-Beltrami eigenbasis holds on compact Riemannian manifolds of class C^{2s} by standard results in spectral geometry. The weighted spectral penalty is constructed to match the trace norm via the spectral characterization of the fractional Sobolev space on the boundary. We acknowledge that the manuscript asserted this equivalence without a self-contained proof. In revision we will add a dedicated appendix that verifies eigenbasis completeness, reproduces the fractional seminorm, and confirms manifold-independent constants under the stated regularity, citing the relevant spectral theory. revision: yes

  2. Referee: [Error decomposition section] Error-decomposition section: the separation into approximation, generalization, and spectral-truncation errors, together with the assertion that the proposed loss aligns with localized fast-rate generalization, presupposes that the spectral loss controls the stability term with constants independent of the manifold and the frequency content. If the trace-norm equivalence fails to hold at the required strength, the decomposition no longer supports the claimed alignment.

    Authors: The error decomposition and its alignment with fast-rate generalization analysis rest on the stability constants furnished by the spectral loss. Once the appendix establishes the required norm equivalence with manifold- and frequency-independent constants, the decomposition holds as stated. We will revise the error-decomposition section to reference the new appendix explicitly and to clarify the resulting control on the stability term. revision: yes

Circularity Check

0 steps flagged

No circularity: derivations and claims remain independent of inputs

full rationale

The abstract and provided text describe new CNN approximation estimates on manifolds with rates depending on intrinsic dimension and smoothness gap, plus a spectral boundary loss constructed via Laplace-Beltrami eigenmode expansion and Sobolev weights. The error decomposition into approximation, generalization, and truncation terms is presented as derived from the framework rather than presupposed. No quoted equations reduce a result to a fitted parameter renamed as prediction, a self-citation chain, or a definitional equivalence. The matching to the trace norm is asserted as a design choice aligned with elliptic stability, not shown to be tautological. This is the common case of a self-contained analysis.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Only abstract available, so ledger is necessarily incomplete; the domain assumption of compact Riemannian manifolds is explicit, while choices such as Sobolev trace weights and eigenmode truncation order are not quantified as fitted parameters.

axioms (1)
  • domain assumption The manifold is a compact Riemannian manifold
    Stated directly in the abstract as the setting for the CNN approximation and boundary value problems.

pith-pipeline@v0.9.1-grok · 5724 in / 1401 out tokens · 37256 ms · 2026-07-01T00:10:52.540178+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 5 canonical work pages · 1 internal anchor

  1. [1]

    Convergence of diffusion models under the manifold hypothesis in high-dimensions.arXiv preprint arXiv:2409.18804, 2024

    Iskander Azangulov, George Deligiannidis, and Judith Rousseau. Convergence of diffusion models under the manifold hypothesis in high-dimensions.arXiv preprint arXiv:2409.18804, 2024

    work page arXiv 2024

  2. [2]

    Representation learning: A review and new perspectives.IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013

    Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives.IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013

    2013

  3. [3]

    Enforc- ing dirichlet boundary conditions in physics-informed neural networks and variational physics-informed neural networks.Heliyon, 9(8), 2023

    Stefano Berrone, Claudio Canuto, Moreno Pintore, and Natarajan Sukumar. Enforc- ing dirichlet boundary conditions in physics-informed neural networks and variational physics-informed neural networks.Heliyon, 9(8), 2023

    2023

  4. [4]

    Convergence and error control of consistent PINNs for elliptic PDEs.IMA journal of numerical analysis, 46(1):90–148, 2026

    Andrea Bonito, Ronald DeVore, Guergana Petrova, and Jonathan W Siegel. Convergence and error control of consistent PINNs for elliptic PDEs.IMA journal of numerical analysis, 46(1):90–148, 2026

    2026

  5. [5]

    Reproducing kernel Hilbert spaces on manifolds: Sobolev and diffusion spaces.Analysis and Applications, 19(03):363–396, 2021

    Ernesto De Vito, Nicole M¨ ucke, and Lorenzo Rosasco. Reproducing kernel Hilbert spaces on manifolds: Sobolev and diffusion spaces.Analysis and Applications, 19(03):363–396, 2021

    2021

  6. [6]

    Nonlinear approximation.Acta Numerica, 7:51–150, 1998

    Ronald A DeVore. Nonlinear approximation.Acta Numerica, 7:51–150, 1998

    1998

  7. [7]

    Theory of deep convolu- tional neural networks II: Spherical analysis.Neural Networks, 131:154–162, 2020

    Zhiying Fang, Han Feng, Shuo Huang, and Ding-Xuan Zhou. Theory of deep convolu- tional neural networks II: Spherical analysis.Neural Networks, 131:154–162, 2020

    2020

  8. [8]

    Scattered data interpolation on embedded sub- manifolds with restricted positive definite kernels: Sobolev error estimates.SIAM Journal on Numerical Analysis, 50(3):1753–1776, 2012

    Edward Fuselier and Grady B Wright. Scattered data interpolation on embedded sub- manifolds with restricted positive definite kernels: Sobolev error estimates.SIAM Journal on Numerical Analysis, 50(3):1753–1776, 2012

    2012

  9. [9]

    Evarist M. Gin´ e. The addition formula for the eigenfunctions of the Laplacian.Advances in Mathematics, 18(1):102–107, 1975

    1975

  10. [10]

    Gaussian Error Linear Units (GELUs)

    D Hendrycks. Gaussian error linear units (GELUs).arXiv preprint arXiv:1606.08415, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  11. [11]

    Trace regularity PINNs: EnforcingH 1/2(∂Ω) for bound- ary data.arXiv preprint arXiv:2510.16817, 2025

    Doyoon Kim and Junbin Song. Trace regularity PINNs: EnforcingH 1/2(∂Ω) for bound- ary data.arXiv preprint arXiv:2510.16817, 2025

    work page arXiv 2025

  12. [12]

    Imagenet classification with deep convolutional neural networks.Advances in neural information processing systems, 25, 2012

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks.Advances in neural information processing systems, 25, 2012

    2012

  13. [13]

    Solving PDEs on spheres with physics-informed convolutional neural networks.Applied and Computational Harmonic Analysis, 74:101714, 2025

    Guanhang Lei, Zhen Lei, Lei Shi, Chenyu Zeng, and Ding-Xuan Zhou. Solving PDEs on spheres with physics-informed convolutional neural networks.Applied and Computational Harmonic Analysis, 74:101714, 2025

    2025

  14. [14]

    Understanding deep convolutional networks.Philosophical Trans- actions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374 (2065):20150203, 2016

    St´ ephane Mallat. Understanding deep convolutional networks.Philosophical Trans- actions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374 (2065):20150203, 2016

    2065

  15. [15]

    Consistent PINNs for higher-order elliptic PDEs.Inter- national Journal for Numerical Methods in Engineering, 127(7):e70320, 2026

    Shiv Mishra and Arbaz Khan. Consistent PINNs for higher-order elliptic PDEs.Inter- national Journal for Numerical Methods in Engineering, 127(7):e70320, 2026. 23

    2026

  16. [16]

    A novel Galerkin method for solving PDEs on the sphere using highly localized kernel bases.Mathematics of Computation, 86(303):197–231, 2017

    Francis Narcowich, Stephen Rowe, and Joseph Ward. A novel Galerkin method for solving PDEs on the sphere using highly localized kernel bases.Mathematics of Computation, 86(303):197–231, 2017

    2017

  17. [17]

    Diffusion models are minimax opti- mal distribution estimators

    Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax opti- mal distribution estimators. InICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2023

    2023

  18. [18]

    Springer Science & Business Media, 2012

    Allan Pinkus.N-widths in Approximation Theory, volume 7. Springer Science & Business Media, 2012

    2012

  19. [19]

    Sharp lower bounds on the manifold widths of Sobolev and Besov spaces.Journal of Complexity, 85:101884, 2024

    Jonathan W Siegel. Sharp lower bounds on the manifold widths of Sobolev and Besov spaces.Journal of Complexity, 85:101884, 2024

    2024

  20. [20]

    Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks.Computer Methods in Applied Mechanics and Engineering, 389:114333, 2022

    Natarajan Sukumar and Ankit Srivastava. Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks.Computer Methods in Applied Mechanics and Engineering, 389:114333, 2022

    2022

  21. [21]

    Two-point homogeneous spaces.Annals of Mathematics, 55(1): 177–191, 1952

    Hsien-Chung Wang. Two-point homogeneous spaces.Annals of Mathematics, 55(1): 177–191, 1952

    1952

  22. [22]

    Cambridge University Press, 2004

    Holger Wendland.Scattered Data Approximation, volume 17. Cambridge University Press, 2004

    2004

  23. [23]

    On the rates of convergence for learning with convolutional neural networks.SIAM Journal on Mathematics of Data Science, 7 (4):1755–1772, 2025

    Yunfei Yang, Han Feng, and Ding-Xuan Zhou. On the rates of convergence for learning with convolutional neural networks.SIAM Journal on Mathematics of Data Science, 7 (4):1755–1772, 2025

    2025

  24. [24]

    Deep distributed convolutional neural networks: Universality.Analysis and Applications, 16(06):895–919, 2018

    Ding-Xuan Zhou. Deep distributed convolutional neural networks: Universality.Analysis and Applications, 16(06):895–919, 2018

    2018

  25. [25]

    Theory of deep convolutional neural networks: Downsampling.Neural Networks, 124:319–327, 2020

    Ding-Xuan Zhou. Theory of deep convolutional neural networks: Downsampling.Neural Networks, 124:319–327, 2020

    2020

  26. [26]

    Universality of deep convolutional neural networks.Applied and Com- putational Harmonic Analysis, 48(2):787–794, 2020

    Ding-Xuan Zhou. Universality of deep convolutional neural networks.Applied and Com- putational Harmonic Analysis, 48(2):787–794, 2020

    2020

  27. [27]

    Expressive power of deep networks on manifolds: Simultaneous approximation.arXiv preprint arXiv:2509.09362, 2025

    Hanfei Zhou and Lei Shi. Expressive power of deep networks on manifolds: Simultaneous approximation.arXiv preprint arXiv:2509.09362, 2025

    work page arXiv 2025

  28. [28]

    SSBE-PINN: A sobolev boundary scheme boosting stability and accuracy in elliptic/parabolic PDE learning.arXiv preprint arXiv:2508.10322, 2025

    Qixuan Zhou, Chuqi Chen, Tao Luo, and Yang Xiang. SSBE-PINN: A sobolev boundary scheme boosting stability and accuracy in elliptic/parabolic PDE learning.arXiv preprint arXiv:2508.10322, 2025. 24

    work page arXiv 2025