pith. sign in

arxiv: 2508.19347 · v3 · submitted 2025-08-26 · 🧮 math.NA · cs.NA· math.FA

Neural operators for solving nonlinear inverse problems

Otmar Scherzer , Thi Lan Nhi Vu , Jikai Yan This is my paper

Pith reviewed 2026-05-18 20:32 UTC · model grok-4.3

classification 🧮 math.NA cs.NAmath.FA
keywords neural operatorsTikhonov regularizationinverse problemsill-posed operator equationsSobolev spacesoperator approximationnonlinear inverse problemsregularization theory
0
0 comments X

The pith

Tikhonov regularization with neural operators solves ill-posed nonlinear inverse problems by balancing approximation errors, regularization parameters, and noise.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that neural operators trained on input-output pairs can act as surrogates for unknown forward operators in Tikhonov regularization for infinite-dimensional ill-posed equations. Error balancing between the neural approximation, the regularization parameter, and data noise yields stable reconstructions, with the analysis extended to Sobolev and Lebesgue spaces to support the required function-space estimates. The work also addresses selecting appropriate network architectures during training and demonstrates the method through numerical experiments on nonlinear problems.

Core claim

When a neural operator approximates the true forward operator with controlled error in Sobolev or Lebesgue spaces, Tikhonov regularization converges to a stable solution of the inverse problem provided the regularization parameter is chosen to balance the approximation error, regularization strength, and noise level.

What carries the argument

Neural operators serving as surrogates inside Tikhonov regularization, with approximation properties extended from continuous functions to Sobolev and Lebesgue spaces.

Load-bearing premise

Once trained on available pairs, the neural operator approximates the true forward operator accurately enough in the Sobolev or Lebesgue spaces needed for the error-balancing analysis.

What would settle it

A case where increasing neural-operator capacity or decreasing noise fails to reduce reconstruction error according to the predicted balancing rates.

read the original abstract

We consider solving a probably infinite dimensional operator equation, where the operator is not modeled by physical laws but is specified indirectly via training pairs of the input-output relation of the operator. Neural operators have proven to be efficient to approximate infinite dimensional operators. In this paper we analyze Tikhonov regularization with neural operators as surrogates for solving ill-posed operator equations. The analysis is based on balancing approximation errors of neural operators, regularization parameters, and noise. Moreover, we extend the approximation properties of neural operators from sets of continuous functions to Sobolev and Lebesgue spaces, which is crucial for solving inverse problems and we discuss the problem of finding an appropriate network structure of neural operators (training). Finally, we present some numerical experiments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper analyzes Tikhonov regularization for ill-posed nonlinear operator equations where the forward operator is learned from input-output training pairs via neural operators. The central claim is that balancing the neural-operator approximation error, the regularization parameter, and the noise level yields convergent reconstructions; the work extends neural-operator approximation theory from continuous functions to Sobolev and Lebesgue spaces, discusses architecture selection during training, and presents numerical experiments.

Significance. If the error-balancing argument can be made rigorous with explicit rates, the framework would usefully connect data-driven operator learning to classical regularization theory for inverse problems in which no explicit physical model is available. The Sobolev-space extension is a necessary technical step for applying the theory to typical inverse-problem settings.

major comments (2)
  1. [Abstract / balancing-errors paragraph] Abstract and the paragraph on balancing errors: the convergence analysis requires that the neural-operator approximation error ||F − F_N|| can be driven below any fixed multiple of the noise level δ by increasing network size or training data. The stated extension of approximation results to W^{k,p} spaces is qualitative; no quantitative generalization bound or rate in the Sobolev norm is derived or cited that would guarantee the required decay relative to δ. Without this, the balancing argument remains formal.
  2. [Numerical experiments] Numerical-experiments section: the reported results lack error bars, baseline comparisons (standard Tikhonov, other surrogates), and explicit measurement of the realized approximation error ||F − F_N|| in the Sobolev norm used in the analysis. This weakens the empirical support for the claimed error balance.
minor comments (1)
  1. [Abstract] The abstract should state the precise assumptions on the distribution of the training pairs that are needed for the Sobolev-space approximation result to hold.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. We address each major comment below and indicate the revisions we will make to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract / balancing-errors paragraph] Abstract and the paragraph on balancing errors: the convergence analysis requires that the neural-operator approximation error ||F − F_N|| can be driven below any fixed multiple of the noise level δ by increasing network size or training data. The stated extension of approximation results to W^{k,p} spaces is qualitative; no quantitative generalization bound or rate in the Sobolev norm is derived or cited that would guarantee the required decay relative to δ. Without this, the balancing argument remains formal.

    Authors: We thank the referee for this observation. Our extension to Sobolev and Lebesgue spaces proves that neural operators are dense in the appropriate operator norms, which justifies that the approximation error ||F − F_N|| can be made arbitrarily small by increasing network size or training data. The balancing argument is therefore rigorous under the explicit assumption that this error is controlled relative to δ; we will revise the manuscript to state this assumption more clearly in the abstract and analysis sections and to note that quantitative rates would require additional assumptions on the training process or the target operator, which we leave for future work. revision: partial

  2. Referee: [Numerical experiments] Numerical-experiments section: the reported results lack error bars, baseline comparisons (standard Tikhonov, other surrogates), and explicit measurement of the realized approximation error ||F − F_N|| in the Sobolev norm used in the analysis. This weakens the empirical support for the claimed error balance.

    Authors: We agree that these additions would improve the empirical section. In the revised manuscript we will report error bars obtained from multiple independent training runs with different random seeds. We will also include comparisons against other neural-operator architectures as surrogates. Finally, we will compute and display the realized approximation error ||F − F_N|| in the Sobolev norm for the trained networks to directly demonstrate the error balance achieved in the experiments. revision: yes

Circularity Check

0 steps flagged

No circularity: analysis treats neural-operator error as external input

full rationale

The paper's central claim rests on balancing three error sources (neural-operator approximation error, regularization parameter, and noise) within Tikhonov regularization, together with an extension of approximation results from C^0 to Sobolev/Lebesgue spaces. These steps are presented as independent mathematical contributions; the approximation error ||F - F_N|| is introduced as an external quantity to be balanced rather than derived from the paper's own fitted quantities or equations. No self-citation is invoked to justify a uniqueness theorem or to smuggle an ansatz, and no prediction is obtained by renaming a fitted parameter. The derivation therefore remains self-contained against external benchmarks of regularization theory and function-space approximation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The analysis assumes standard functional-analytic properties of neural operators and the well-posedness of the regularized problem once the surrogate is fixed; no new entities are introduced.

axioms (1)
  • domain assumption Neural operators can be trained to approximate the forward operator with controllable error in Sobolev and Lebesgue spaces
    Invoked when extending approximation properties and balancing errors for the regularization analysis.

pith-pipeline@v0.9.0 · 5650 in / 1252 out tokens · 33515 ms · 2026-05-18T20:32:33.730854+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · 1 internal anchor

  1. [1]

    Sobolev Spaces

    R. A. Adams. “Sobolev Spaces”. Pure and Applied Mathematics 65. New York: Academic Press,

    work page

  2. [2]

    isbn: 9780080873817 (cited on pages 5, 11)

    work page

  3. [3]

    Sobolev Spaces

    R. A. Adams and F. J. J. F. “Sobolev Spaces”. 2nd ed. Pure and Applied Mathematics. Amsterdam: Elsevier, 2003 (cited on page 13)

    work page 2003

  4. [4]

    A Data-Driven Iteratively Regularized Landweber Iteration

    A. Aspri, S. Banert, O. Öktem, and O. Scherzer. “A Data-Driven Iteratively Regularized Landweber Iteration”. In:Numerical Functional Analysis and Optimization41.10 (Mar. 2020), pp. 1190–1227. issn: 0163-0563. doi: 10.1080/01630563.2020.1740734 (cited on page 6)

    work page doi:10.1080/01630563.2020.1740734 2020

  5. [5]

    Data Driven Reconstruction Using Frames and Riesz Bases

    A. Aspri, L. Frischauf, Y. Korolev, and O. Scherzer. “Data Driven Reconstruction Using Frames and Riesz Bases”. In:Deterministic and Stochastic Optimal Control and Inverse Problems. Ed. by B. Jadamba, A. A. Khan, S. Migórski, and M. Sama. CRC Press, 2021, pp. 303–318.doi: 10.1201/9781003050575-13 (cited on page 5)

    work page doi:10.1201/9781003050575-13 2021

  6. [6]

    Spectral Function Space Learning and Numerical Linear Algebra Networks for Solving Linear Inverse Problems

    A. Aspri, L. Frischauf, and O. Scherzer. “Spectral Function Space Learning and Numerical Linear Algebra Networks for Solving Linear Inverse Problems”. Preprint on ArXiv 2408.10690. 2024 (cited on pages 6, 9)

    work page arXiv 2024

  7. [7]

    Data driven regularization by projection

    A. Aspri, Y. Korolev, and O. Scherzer. “Data driven regularization by projection”. In:Inverse Problems 36.12 (Dec. 2020), p. 125009.issn: 0266-5611. doi: 10.1088/1361-6420/abb61b (cited on pages 5, 6)

    work page doi:10.1088/1361-6420/abb61b 2020

  8. [8]

    Behavior of the error of the approximate solutions of boundary value problems for linear elliptic operators by Galerkin’s and finite difference methods

    J. P. Aubin. “Behavior of the error of the approximate solutions of boundary value problems for linear elliptic operators by Galerkin’s and finite difference methods”. In:Annali della Scuala Normale Superiore di Pisa. Classe di Scienze21.4 (1967), pp. 599–637 (cited on page 3)

    work page 1967

  9. [9]

    Universal approximation bounds for superpositions of a sigmoidal function

    A. R. Barron. “Universal approximation bounds for superpositions of a sigmoidal function”. In:IEEE Transactions on Information Theory39.3 (1993), pp. 930–945.issn: 0018-9448. doi: 10.1109/18. 256500 (cited on page 8)

    work page doi:10.1109/18 1993

  10. [10]

    Approximations of continuous functionals by neural networks with application to dynamic systems

    T. Chen and H. Chen. “Approximations of continuous functionals by neural networks with application to dynamic systems”. In:IEEE Transactions on Neural Networks4.6 (1993), pp. 910–918.doi: 10.1109/72.286886 (cited on pages 2, 5, 7, 9)

    work page doi:10.1109/72.286886 1993

  11. [11]

    Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems

    T. Chen and H. Chen. “Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems”. In:IEEE Transactions on Neural Networks6.4 (1995), pp. 911–917.doi: 10.1109/72.392253 (cited on pages 2, 5, 7, 10, 11)

    work page doi:10.1109/72.392253 1995

  12. [12]

    The Finite Element Method for Elliptic Problems

    P. G. Ciarlet. “The Finite Element Method for Elliptic Problems”. Amsterdam: North-Holland, 1978 (cited on page 3)

    work page 1978

  13. [13]

    Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of Control, Signals and Systems 2 (4) (1989) 303–314.doi: 10.1007/BF02551274

    G. Cybenko. “Approximation by superpositions of a sigmoidal function”. In:Mathematics of Control, Signals, and Systems2.4 (1989), pp. 303–314.doi: 10.1007/bf02551274 (cited on pages 5, 7, 9)

    work page doi:10.1007/bf02551274 1989

  14. [14]

    An iterative thresholding algorithm for linear inverse problems with a sparsity constraint

    I. Daubechies, M. Defrise, and C. De Mol. “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint”. In:Communications on Pure and Applied Mathematics57.11 (2004), pp. 1413–1457.issn: 0010-3640. doi: 10.1002/cpa.20042 (cited on page 2)

    work page doi:10.1002/cpa.20042 2004

  15. [15]

    Convergence rates for Tikhonov regularisation of nonlinear ill-posed problems

    H. W. Engl, K. Kunisch, and A. Neubauer. “Convergence rates for Tikhonov regularisation of nonlinear ill-posed problems”. In:Inverse Problems5.3 (1989), pp. 523–540.issn: 0266-5611 (cited on pages 3, 4, 9)

    work page 1989

  16. [16]

    Partial Differential Equations

    L. C. Evans. “Partial Differential Equations”. Second. Vol. 19. Graduate Studies in Mathematics. Providence, RI: American Mathematical Society, 2010.isbn: 978-0-8218-4974-3 (cited on page 13)

    work page 2010

  17. [17]

    The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind

    C. W. Groetsch. “The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind”. Boston: Pitman, 1984 (cited on page 9). 16

    work page 1984

  18. [18]

    Addendum on data driven regularization by projection

    M. Hanke and O. Scherzer. “Addendum on data driven regularization by projection”. Preprint on ArXiv 2508.07709. 2025 (cited on page 6)

    work page arXiv 2025

  19. [19]

    Regularization of Nonlinear Inverse Problems – From Functional Analysis to Data-Driven Approaches

    C. Kirisits, B. Mejri, S. Pereverzev, O. Scherzer, and C. Shi. “Regularization of Nonlinear Inverse Problems – From Functional Analysis to Data-Driven Approaches”. Preprint on ArXiv 2506.17465. 2025 (cited on page 5)

    work page arXiv 2025

  20. [20]

    Fourier Neural Operator for Parametric Partial Differential Equations

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandku- mar. “Fourier Neural Operator for Parametric Partial Differential Equations”. Preprint on ArXiv 2010.08895. 2020. doi: arxiv:2010.08895 (cited on page 2)

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  21. [21]

    Learning nonlinear opera- tors via DeepONet based on the universal approximation theorem of operators.Nature Machine Intelligence, 3: 218–229, 2021

    L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis. “Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators”. In:Nature Machine Intelligence3.3 (2021), pp. 218–229.doi: 10.1038/s42256-021-00302-5 (cited on pages 2, 3, 5, 7)

    work page doi:10.1038/s42256-021-00302-5 2021

  22. [22]

    Tikhonov regularization for non-linear ill-posed problems: optimal convergence rates and finite-dimensional approximation

    A. Neubauer. “Tikhonov regularization for non-linear ill-posed problems: optimal convergence rates and finite-dimensional approximation”. In:Inverse Problems5.4 (1989), pp. 541–557.issn: 0266-5611 (cited on pages 2, 9)

    work page 1989

  23. [23]

    Finite-dimensional approximation of Tikhonov regularized solutions of nonlinear ill-posed problems

    A. Neubauer and O. Scherzer. “Finite-dimensional approximation of Tikhonov regularized solutions of nonlinear ill-posed problems”. In:Numerical Functional Analysis and Optimization11.1-2 (1990), pp. 85–99.issn: 0163-0563. doi: 10.1080/01630569008816362 (cited on pages 2–4, 9, 12, 14)

    work page doi:10.1080/01630569008816362 1990

  24. [24]

    Ein Kriterium für die Quasi-Optimalität des Ritzschen Verfahrens

    J. Nitsche. “Ein Kriterium für die Quasi-Optimalität des Ritzschen Verfahrens”. In:Numerische Mathematik 11.4 (1968), pp. 346–348.issn: 0029-599X. doi: 10.1007/bf02166687 (cited on page 3)

    work page doi:10.1007/bf02166687 1968

  25. [25]

    Discretization of variational regularization in Banach spaces

    C. Pöschl, E. Resmerita, and O. Scherzer. “Discretization of variational regularization in Banach spaces”. In:Inverse Problems 26.10 (2010), p. 105017. issn: 0266-5611. doi: 10 . 1088 / 0266 - 5611/26/10/105017 (cited on page 2)

    work page 2010

  26. [26]

    Physica E: Low-dimensional Systems and Nanostructures106, 208–238 (2019) https://doi.org/10.1016/j

    M. Raissi, P. Perdikaris, and G. E. Karniadakis. “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations”. In: Journal of Computational Physics378 (2019), pp. 686–707.issn: 0021-9991. doi: 10.1016/j. jcp.2018.10.045 (cited on page 2)

    work page doi:10.1016/j 2019

  27. [27]

    Nonlinear total variation based noise removal algorithms

    L. I. Rudin, S. Osher, and E. Fatemi. “Nonlinear total variation based noise removal algorithms”. In: Physica D. Nonlinear Phenomena60.1–4 (1992), pp. 259–268 (cited on page 1)

    work page 1992

  28. [28]

    Variational Methods in Imaging

    O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, and F. Lenzen. “Variational Methods in Imaging”. Applied Mathematical Sciences 167. New York: Springer, 2009.isbn: 978-0-387-30931-6. doi: 10.1007/978-0-387-69277-7 (cited on page 2)

    work page doi:10.1007/978-0-387-69277-7 2009

  29. [29]

    Gauss–Newton method for solving linear inverse problems with neural network coders

    O. Scherzer, B. Hofmann, and Z. Nashed. “Gauss–Newton method for solving linear inverse problems with neural network coders”. In:Sampling Theory, Signal Processing, and Data Analysis21.2 (2023). doi: 10.1007/s43670-023-00066-6 (cited on pages 3, 5)

    work page doi:10.1007/s43670-023-00066-6 2023

  30. [30]

    Regularization methods in Banach spaces

    T. Schuster, B. Kaltenbacher, B. Hofmann, and K. S. Kazimierski. “Regularization methods in Banach spaces”. Radon Series on Computational and Applied Mathematics 10. Berlin, Boston: De Gruyter, 2012. xii+283.doi: 10.1515/9783110255720 (cited on page 2)

    work page doi:10.1515/9783110255720 2012

  31. [31]

    Learning the solution operator of parametric partial differential equations with physics-informed DeepONets

    S. Wang, H. Wang, and P. Perdikaris. “Learning the solution operator of parametric partial differential equations with physics-informed DeepONets”. In:Science Advances7.40 (2021). doi: 10.1126/ sciadv.abi8605 (cited on page 2)

    work page 2021