Pith. sign in

REVIEW 4 major objections 6 minor 79 references

Orthogonal greedy algorithm for linear operator learning with shallow neural network

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Orthogonal greedy algorithms train shallow neural networks to learn linear operators by estimating their Green's-function kernels, with proved convergence rates that numerical tests match.

desk verdict A useful point-wise greedy variant and a fixable but real gap in the kernel-estimation theory. read the letter →

arxiv 2501.02791 v1 pith:E37SSQEN submitted 2025-01-06 math.NA cs.LGcs.NA

Ye Lin , Jiwei Jia , Young Ju Lee , Ran Zhang This is my paper · ORCID
classification math.NAcs.LGcs.NA MSC 41A6565D1568T07
keywords orthogonalgreedyalgorithmGreen'sfunctionoperatorlearningkernelestimationshallowneuralnetworksemi-innerproductpoint-wiseweakapproximation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that orthogonal greedy algorithms (OGA) can train shallow neural networks to learn linear operators from data by estimating the operator's kernel—the Green's function—directly, without stochastic gradient descent. The key move is a new data-dependent semi-inner product defined by averages over the training pairs, which lets the greedy dictionary search measure how well a candidate kernel reproduces the observed responses. The authors prove that weak OGA in this product converges at rate $n^{-1/2-(2k+1)/(4d)}$ for directly estimating the $2d$-dimensional kernel, and that a point-wise variant (PW-OGA), which fits $d$-dimensional slices of the kernel, improves the rate to $n^{-1/2-(2k+1)/(2d)}$. In their numerical experiments the observed rates match these exponents and beat the deep operator-learning baselines by orders of magnitude in accuracy. If correct, the result turns operator learning into a dictionary approximation problem with certified convergence rates and a transparent dependence on data diversity.

What carries the argument

The load-bearing object is the data-dependent semi-inner product of Eq. (20), built from the training forcing functions $\{f_j\}$ and the integral-transform pairing $G\star f$. Together with a randomized discrete dictionary of $\pm\text{ReLU}^k$ ridge functions (Eq. (38)), it converts kernel estimation into a weak orthogonal greedy algorithm: at each step one searches the random dictionary for the neuron maximizing the semi-inner product against the current residual, then projects orthogonally onto the span of the selected neurons, solving the small linear system of Eq. (50) for the coefficients. The point-wise variant (PW-OGA) applies the same loop to each kernel slice $G_s(y)=G(y|x_s)$, so each subproblem is $d$-dimensional function approximation instead of $2d$-dimensional kernel approximation, which is why its rate has the factor-two improvement.

What would settle it

Train PW-OGA on forcing functions sampled from a low-dimensional subspace (e.g., only even functions or low-frequency modes), then evaluate the learned operator on a forcing function with a significant component outside that subspace; if the predicted response is wrong even though the training semi-norm error is at machine precision, the semi-norm certificate fails for out-of-span kernels. Concretely, one can check whether $\epsilon_G$ remains large or diverges while $\epsilon_u$ is small, using the SVD of the forcing data matrix to predict which kernel components are unobservable.

Watch

Extended reading notes

Core claim

The central claim is that linear operator learning is equivalent to estimating the kernel of an integral operator, and that this estimation can be performed by the orthogonal greedy algorithm with randomized dictionaries of ReLU$^k$ ridge functions, provided the Hilbert inner product is replaced by the data-dependent semi-inner product $$\langle G_1, G_2\rangle_H = \frac{1}{N}\sum_{j=1}^N (G_1\star f_j, G_2\star f_j)_{$L^{2}$(\$\Omega$)}.$$ In this product, a candidate kernel is judged by how well its integral transforms reproduce the training responses; the null space of the induced seminorm consists of kernels that are invisible to the training forcing functions. The paper proves that weak OGA converges in this seminorm (or norm, when the forcing data span the discrete forcing space) with the near-optimal exponent $n^{-1/2-(2k+1)/(4d)}$, and that the point-wise variant that learns each slice $G(y|x_s)$ separately attains the faster exponent $n^{-1/2-(2k+1)/(2d)}$. The same estimate transfers to the inference error for predicted responses, and the numerical sections report that both algorithms achieve the predicted rates while improving on deep operator-learning baselines by orders of magnitude.

Load-bearing premise

The load-bearing premise is that the true kernel $G$ can be replaced by its piecewise-linear discrete surrogate $\hat{G}$ with negligible modeling error, and that the training forcing functions span the discrete forcing space so that the new 'inner product' is positive definite; if either fails, zero semi-norm error no longer implies the kernel is learned.

Editorial extensions

If this is right

  • OGA-based kernel estimation gives a training procedure for shallow neural networks that avoids gradient descent and comes with explicit convergence-rate certificates in the data-dependent semi-norm.
  • Direct kernel OGA converges at $n^{-1/2-(2k+1)/(4d)}$ and PW-OGA at $n^{-1/2-(2k+1)/(2d)}$, so in every dimension the point-wise method is provably faster for the same neuron count (once the number of response points $m_u$ is below $n^{(2k+1)/(4d)}$).
  • The inference error for unseen responses obeys the same rate as the kernel estimation error, so accuracy on predicted solutions is controlled by the same theory.
  • A larger null space $\mathcal{N}$ in the semi-norm speeds up convergence of the response error $\epsilon_u$ but can cause kernel error $\epsilon_G$ to diverge; the paper documents this trade-off and ties it to linear dependence of the forcing data.
  • On the paper's 1D-3D Poisson, Helmholtz, and cosine-kernel tests, both methods reach the predicted optimal rates and improve on the three deep operator-learning baselines by orders of magnitude.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because positivity of the semi-inner product is guaranteed only when the training forcing functions span the discrete forcing space, a kernel component lying outside that span is invisible; a natural stress test is to evaluate the learned operator on forcing functions outside the training span, where the semi-norm certificates say nothing.
  • The paper's own SVD experiment on the forcing data matrix suggests a practical diagnostic: the rank profile of the data matrix predicts when $\epsilon_G$ will diverge and could serve as an early-stopping or data-augmentation criterion for PW-OGA.
  • The faster point-wise rate hints that splitting the kernel into slices could be combined with mesh or domain decomposition, yielding a naturally parallel operator learner; such a combination is not studied in the paper.
  • A testable extension would be to add probing forcing functions (e.g., localized bumps or random projections) to the training set to shrink the null space and thereby convert semi-norm convergence into genuine kernel convergence; the paper does not propose this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes two orthogonal greedy algorithms for linear operator learning with shallow ReLU networks: OGA, which approximates the Green's function directly in a data-dependent semi-inner product, and PW-OGA, which estimates the kernel pointwise as a collection of d-dimensional slice functions. The authors prove approximation-rate estimates for both algorithms (Theorems 1 and 2), claiming rates of order n^{-1/2-(2k+1)/(4d)} and n^{-1/2-(2k+1)/(2d)} respectively, and support these claims with numerical experiments on 1D, 2D, and 3D problems, comparing against GL, FNO, and DON baselines.

Significance. If the kernel-estimation rates were valid for the true Green's function, this would be a valuable contribution: it would supply the first greedy-algorithm-based method for operator learning with rigorous rates that avoid nonconvex optimization. The pointwise reduction from a 2d-dimensional to a d-dimensional approximation problem is a sensible and practically useful idea, and the numerical results for PW-OGA's response error ϵ_u are substantially better than the reported baselines in several 2D and 3D cases. The paper also deserves credit for explicitly acknowledging that its bilinear form is only a semi-inner product and for demonstrating the associated overfitting phenomenon numerically. However, as explained in the major comments, the central theoretical claim — that the method approximates the kernel itself — is not established: the rates control only the response error on the training forcing space, and the numerical claims about ϵ_G are not supported by the theory.

major comments (4)
  1. [Sec. 3.3 and Sec. 3.6, Lemma 1 and Eq. (25)] Lemma 1's positivity proof is invalid for a genuine kernel. The statement that G⋆f_{j_l}=0 for m_f linearly independent f_j implies G=0 is only true after identifying G with the finite-dimensional piecewise-linear surrogate Ĝ of Eq. (22), an identification that is made silently. For a true Green's function, any kernel whose y-section lies in the orthogonal complement of span{f_j} in L2(Ω) gives zero semi-norm without being zero; the paper itself concedes this in Eq. (21). Consequently, ⟨·,·⟩_H in Eq. (20) is only a semi-inner product on the kernel space, and Theorem 1's bound (54) controls the response error on the training forcing space, not the kernel error. The decomposition (25) separates out the modeling error ∥G−Ĝ∥_H but supplies no quantitative bound on it; declaring it negligible does not establish convergence of the kernel. This undermines the abstract's claim that the algorithm approximates the Green's function and invalidates the report of ϵ_G rates as consequences of Theorem 1.
  2. [Sec. 3.6, Lemma 5, Eq. (64)] Lemma 5 is not a valid chain of inequalities. The middle term inf_{ψ∈N}∥G−Gn+ψ∥_{L2(Ω×Ω)} contains a free variable Gn that is not quantified by an infimum or supremum. Moreover, the second inequality, inf_ψ ∥G−Gn+ψ∥_{L2} ≤ inf_{μ_n∈Σ_{n,M}(D)} ∥G−μ_n∥_{L2}, does not follow from Lemma 2 and is generally false for a fixed suboptimal iterate Gn. Lemma 2 gives ∥G−μ_n∥_H ≤ ∥G−μ_n+ψ∥ for each μ_n and ψ, which shows only that the infimum over Gn of the semi-norm error is bounded by the quotient norm of a best L2 approximant. The intended conclusion that the OGA error is bounded by the L2 approximation rate requires a different argument, and the paper does not provide one. This is load-bearing because it is the step that connects the semi-norm rate to the standard shallow-network approximation rate.
  3. [Sec. 3.6, proof of Theorem 1, Eq. (58)] The proof of Theorem 1 treats ⟨·,·⟩_H as a definite inner product, but the theorem claims validity for the semi-norm case. The recursion (58) divides by ∥Gn−Pn−1Gn∥^2_H; in a semi-inner product this quantity can be zero for a nonzero vector, since span{g_1,...,g_n} may contain elements with zero H-norm. Remark 1 states that one should interpret the projection as projection onto N⊥, but this is not implemented in the proof, and no argument shows that the dictionary elements selected by Algorithm 3 have positive H-norm after such a projection, nor that the orthogonal projection P_n is well-defined and contractive in the semi-norm. Thus the proof does not cover the advertised semi-norm setting.
  4. [Sec. 4, Table 3 and Figs. 9–10] The numerical evidence does not support the paper's general claim that both OGA and PW-OGA outperform baselines by orders of magnitude. Table 3 shows that for the 2D Poisson and Helmholtz problems, direct OGA returns ϵ_u = 1.75e-1 and 3.79e-1 on the disk and H-domain, respectively, which is worse than GL (4.05e-2 and 7.13e-2) and sometimes worse than DON. The impressive improvements in Table 3 are specific to PW-OGA, not to OGA. Furthermore, Figs. 9(b) and 10 show that ϵ_G diverges or flattens as the number of neurons increases; the text acknowledges overfitting in Sec. 4.2.2 but still states that both ϵ_u and ϵ_G achieve the optimal rate. The rate claims should be restricted to PW-OGA's ϵ_u in the pre-overfitting regime, and the ϵ_G claims should be removed or substantially qualified.
minor comments (6)
  1. [Sec. 3.3, Eq. (22)] The definition of the surrogate class G is ambiguous: the condition G≈ f∼ = (...) should state that it holds for all f in the finite-dimensional sampling space, and the precise relation between piecewise-linear interpolation and the discrete matrix G≈ is not given.
  2. [Sec. 3.6, proof of Theorem 1] In the line after Eq. (56), 'sup_{G∈D}|⟨rn−1,G⟩_H|' should read 'sup_{g∈D}|⟨rn−1,g⟩_H|' to avoid confusion between the target kernel and dictionary elements.
  3. [Sec. 3.6, Lemma 4] The notation ε_n(co(K))_H is not defined; the entropy number in Eq. (51) is introduced for a set A⊂H, but the subscript H in Lemma 4 is unexplained.
  4. [Sec. 3.8, Algorithm 4] The loop 'for s = 1,2,...' should be 'for s = 1,...,m_u' with m_u defined as the number of response points; otherwise the algorithm appears to have an infinite outer loop.
  5. [Fig. 9 and Fig. 10 captions] The captions say 'pair-wise kernel estimation' but the method is introduced as 'point-wise kernel estimation' (PW-OGA); the terminology should be made consistent.
  6. [Sec. 3.6, Eq. (30)] The quotient norm ∥G∥_{L2(Ω×Ω)}/N is written with a division sign that is not standard notation; the authors should define it explicitly as inf_{ψ∈N}∥G+ψ∥_{L2(Ω×Ω)}.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the convergence rates are derived from external dictionary-entropy and WOGA bounds, and the numerical rate comparisons are not fitted inputs.

full rationale

The derivation chain is self-contained in the sense relevant to circularity. Theorem 1 obtains the weak-OGA rate from the standard recursion (55)-(63) together with the metric-entropy Lemma 4, both imported from external works [24,26,27,65,66]; no parameter is fitted to the training data to produce the rate. Theorem 2 applies the same WOGA argument slicewise, with the d-dimensional dictionary rate taken from the external bound [28]. The data-dependent semi-inner product (20) is used both to define the greedy selection and to measure the error, which is a standard approximation-theoretic setup rather than a case of predicting a fitted quantity. The paper explicitly acknowledges that (20) is only a semi-norm (21) and that positivity in Lemma 1 holds only after the identification G=G_hat in (22), with the modeling error declared negligible in (25); this limits the claim about true Green's functions but is a stated modeling assumption, not a circular reduction. The only self-citation, [57], appears in a list of related Green's-function-learning methods and is not load-bearing. The numerical optimal-rate lines are comparisons against theoretical slopes, not fits that enter the derivation, and the overfitting discussion in Sec. 4.3.1 is an explicit limitation rather than a circular step.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No physical entities are introduced. The semi-inner product is a mathematical construction, not a postulated entity. The main hidden load is the identification of G with a piecewise-linear surrogate, which makes Lemma 1 true only on a finite-dimensional space.

free parameters (1)
  • Random dictionary size N_R = 512
    Chosen by hand for all reported experiments; it controls the quality of the argmax step but does not enter the theoretical rate.
assumptions (4)
  • domain assumption The target kernel G is replaced by its piecewise-linear interpolation G-hat with negligible modeling error.
    Section 3.3, Eq. (22) and the following paragraph; needed for Lemma 1 positivity and for the discrete linear algebra that defines the inner product.
  • domain assumption The training forcing functions {f_j} contain at least m_f linearly independent elements and are normalized.
    Lemma 1; makes the data-dependent bilinear form an inner product on the finite-dimensional surrogate space.
  • standard math Sharp approximation rates for ReLU^k shallow networks in L2 (Siegel and Xu [28]) are valid.
    Used to convert the abstract entropy bound in Theorem 1 into the explicit n^(-1/2-(2k+1)/(4d)) rate in Lemma 5.
  • standard math The weak OGA convergence theorem of Xu and Xu [27] and Li and Siegel [26] is valid.
    Theorem 1 follows the proof strategy of these prior works rather than re-deriving the recursion from first principles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Orthogonal greedy algorithm for linear operator learning with shallow neural network." pith.science (2026). https://pith.science/paper/E37SSQEN

@misc{pith2026250102791,
  author       = {Pith},
  title        = {Pith review of: Orthogonal greedy algorithm for linear operator learning with shallow neural network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E37SSQEN}},
  note         = {Machine review of arXiv:2501.02791}
}
read the original abstract

Greedy algorithms, particularly the orthogonal greedy algorithm (OGA), have proven effective in training shallow neural networks for fitting functions and solving partial differential equations (PDEs). In this paper, we extend the application of OGA to the tasks of linear operator learning, which is equivalent to learning the kernel function through integral transforms. Firstly, a novel greedy algorithm is developed for kernel estimation rate in a new semi-inner product, which can be utilized to approximate the Green's function of linear PDEs from data. Secondly, we introduce the OGA for point-wise kernel estimation to further improve the approximation rate, achieving orders of accuracy improvement across various tasks and baseline models. In addition, we provide a theoretical analysis on the kernel estimation problem and the optimal approximation rates for both algorithms, establishing their efficacy and potential for future applications in PDEs and operator learning tasks.

Figures

Figures reproduced from arXiv: 2501.02791 by the authors.

Figure 1
Figure 1. Schematic of Green Learning Like many neural network-based operator learning methods, the approximation errors are mainly arise from nu￾merical integration and optimization[10], which prevent achieving the theoretical order of accuracy. Currently, the most widely used approach for training neural networks is stochastic gradient descent(SGD) or its variant, such as ADAM[8], often combined with a learning rate schedul… view at source ↗
Figure 2
Figure 2. exact kernel G and its approximant Gˆ with h = 0.1 Let Gn denote the n-neuron approximation of G. Then, we have the following inequality: ∥G − Gn∥H ≤ ∥G − Gˆ∥H + ∥Gˆ − Gn∥H. (25) (25) shows that the error in linear operator learning with shallow neural network consists of two components. The first is the modeling error, ∥G − Gˆ∥H, which represents the difference between the true kernel G and Gˆ. The second component… view at source ↗
Figure 3
Figure 3. (a) Kernel function. (b) Slices of kernel function. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Result of 1D Poisson problem: (a) The training dynamics of OGA method. (b) Exact Green’s function. (c) Learned Green’s function. (d) [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Result of 1D Helmholtz problem: (a) The training dynamics of OGA method. (b) Exact Green’s function. (c) Learned Green’s function. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Computational domain of 2D problems: (a) unit disk domain. (b) H shaped domain. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Training dynamics of cosine kernel integral: (a) relative [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Training dynamics of 2D Poisson and Helmholtz problems: (a) Poisson problems. (b) Helmholtz problems [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Training dynamics of 2D test problems with pair-wise kernel estimation. (a) relative [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Training dynamics of 2D test problems with pair-wise kernel estimation. (a) Poisson problems. (b) Helmholtz problems [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Sample result of 2D poisson problem on unit disk: (a) Reference FEM result [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Sample result of 2D helmholtz problem on unit disk: (a) Reference FEM result [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Sample result of 2D poisson problem on H shaped domain: (a) Reference FEM result [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Sample result of 2D helmholtz problem on H shaped domain: (a) Reference FEM result [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Training dynamics(ϵu) of 3D test problems with pair-wise kernel estimation. 4.3.1. Overfitting treatment for PW-OGA Theoretically, both ϵu and ϵG should converge under optimal approximation rate (42) with a sufficiently large and diverse dataset, i.e., when N = {0}. H…
Figure 16
Figure 16. Figure 16: Sample responses of 3D smooth kernel: (a) Reference response [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]
Figure 17
Figure 17. Figure 17: Sample responses of 3D oscillatory singular kernel: (a) Reference response [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]
Figure 18
Figure 18. Figure 18: Overfitting phenomenon analysis: (a) Singular values of [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

79 extracted references · 55 canonical work pages

  1. [1]

    Physics-informed machine learning

    George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021

    2021

  2. [2]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019

    2019

  3. [3]

    Dgm: A deep learning algorithm for solving partial di fferential equations

    Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial di fferential equations. Journal of computational physics, 375:1339–1364, 2018

    work page 2018

  4. [4]

    The deep ritz method: a deep learning-based numerical algorithm for solving variational problems

    Bing Yu et al. The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018

    2018

  5. [5]

    Data-driven discovery of green’s functions with human-understandable deep learning

    Nicolas Boull ´e, Christopher J Earls, and Alex Townsend. Data-driven discovery of green’s functions with human-understandable deep learning. Scientific reports, 12(1):4824, 2022

    work page 2022

  6. [6]

    Learning nonlinear operators via deeponet based on the universal approximation theorem of operators

    Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218–229, 2021

    2021

  7. [7]

    Stuart, and Anima Anandkumar

    Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In 9th International Conference on Learning Representa- tions, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021

    work page 2021

  8. [8]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    arXiv 2014

Show all 79 references
  1. [9]

    Practical methods of optimization

    Roger Fletcher. Practical methods of optimization. John Wiley & Sons, 2000

    2000

  2. [10]

    Elliptic pde learning is provably data-e fficient

    Nicolas Boull ´e, Diana Halikias, and Alex Townsend. Elliptic pde learning is provably data-e fficient. Proceedings of the National Academy of Sciences, 120(39):e2303904120, 2023

    2023

  3. [12]

    Lamperski and Tyler Lekang

    Andrew G. Lamperski and Tyler Lekang. Approximation with random shallow relu networks with applications to model reference adaptive control. ArXiv, abs/2403.17142, 2024

    2024 arXiv

  4. [13]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Neural Information Processing Systems, 2007

    2007

  5. [14]

    A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics

    E Weinan, Chao Ma, and Lei Wu. A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics. Science China Mathematics, 63:1235 – 1258, 2019

    2019

  6. [15]

    Transferable neural networks for partial di fferential equations

    Zezhong Zhang, Feng Bao, Lili Ju, and Guannan Zhang. Transferable neural networks for partial di fferential equations. Journal of Scientific Computing, 99(1):2, 2024

    2024

  7. [16]

    Local extreme learning machines and domain decomposition for solving linear and nonlinear partial di ffer- ential equations

    Suchuan Dong and Zongwei Li. Local extreme learning machines and domain decomposition for solving linear and nonlinear partial di ffer- ential equations. Computer Methods in Applied Mechanics and Engineering, 387:114129, 2021

    2021

  8. [17]

    Extreme learning machine: Theory and applications

    Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew. Extreme learning machine: Theory and applications. Neurocomputing, 70(1):489– 501, 2006. Neural Networks

    2006

  9. [18]

    A nonoverlapping domain decomposition method for extreme learning machines: Elliptic problems

    Chang-Ock Lee, Youngkyu Lee, and Byungeun Ryoo. A nonoverlapping domain decomposition method for extreme learning machines: Elliptic problems. ArXiv, abs/2406.15959, 2024

    2024 arXiv

  10. [19]

    Finite neuron method and convergence analysis

    Jinchao Xu. Finite neuron method and convergence analysis. Communications in Computational Physics, 28:1707–1745, 2020

    2020

  11. [20]

    A neuron-wise subspace correction method for the finite neuron method

    Jongho Park, Jinchao Xu, and Xiaofeng Xu. A neuron-wise subspace correction method for the finite neuron method. CoRR, abs/2211.12031, 2022

    2022

  12. [21]

    Barron, Albert Cohen, Wolfgang Dahmen, and Ronald A

    Andrew R. Barron, Albert Cohen, Wolfgang Dahmen, and Ronald A. DeV ore. Approximation and learning by greedy algorithms.The Annals of Statistics, 36(1):64 – 94, 2008

    2008

  13. [22]

    Y .C. Pati, R. Rezaiifar, and P.S. Krishnaprasad. Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In Proceedings of 27th Asilomar Conference on Signals, Systems and Computers, pages 40–44 vol.1, 1993. 23

    1993

  14. [24]

    Weak greedy algorithms

    Vladimir N Temlyakov. Weak greedy algorithms. Advances in Computational Mathematics, 12(2):213–227, 2000

    2000

  15. [25]

    Greedy training algorithms for neural networks and applications to pdes

    Jonathan W Siegel, Qingguo Hong, Xianlin Jin, Wenrui Hao, and Jinchao Xu. Greedy training algorithms for neural networks and applications to pdes. Journal of Computational Physics, 484:112084, 2023

    2023

  16. [27]

    Randomized greedy algorithms for neural network optimization

    Jinchao Xu and Xiaofeng Xu. Randomized greedy algorithms for neural network optimization. arXiv preprint arXiv:2407.17763, 2024

    2024 arXiv

  17. [28]

    Sharp bounds on the approximation rates, metric entropy, and n-widths of shallow neural networks

    Jonathan W Siegel and Jinchao Xu. Sharp bounds on the approximation rates, metric entropy, and n-widths of shallow neural networks. Foundations of Computational Mathematics, 24(2):481–537, 2024

    2024

  18. [29]

    Mionet: Learning multiple-input operators via tensor product

    Pengzhan Jin, Shuai Meng, and Lu Lu. Mionet: Learning multiple-input operators via tensor product. SIAM Journal on Scientific Computing, 44(6):A3490–A3514, 2022

    2022

  19. [30]

    Learning the solution operator of parametric partial di fferential equations with physics- informed DeepONets

    Sifan Wang, Hanwen Wang, and Paris Perdikaris. Learning the solution operator of parametric partial di fferential equations with physics- informed DeepONets. Science Advances, 7(40):eabi8605, 2021

    2021

  20. [31]

    Improved architectures and training algorithms for deep operator networks

    Sifan Wang, Hanwen Wang, and Paris Perdikaris. Improved architectures and training algorithms for deep operator networks. Journal of Scientific Computing, 92(2):35, 2022

    2022

  21. [32]

    Adaptive operator learning for infinite-dimensional bayesian inverse problems

    Zhiwei Gao, Liang Yan, and Tao Zhou. Adaptive operator learning for infinite-dimensional bayesian inverse problems. SIAM/ASA Journal on Uncertainty Quantification, 12(4):1389–1423, 2024

    2024

  22. [33]

    Ib-uq: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning

    Ling Guo, Hao Wu, Yan Wang, Wenwen Zhou, and Tao Zhou. Ib-uq: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning. Journal of Computational Physics, 510:113089, 2024

    2024

  23. [34]

    Neural operator: Learning maps between function spaces with applications to pdes

    Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1–97, 2023

    2023

  24. [35]

    Multipole graph neural operator for parametric partial differential equations

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhattacharya, and Anima Anandkumar. Multipole graph neural operator for parametric partial differential equations. Advances in Neural Information Processing Systems, 33:6755– 6766, 2020

    2020

  25. [36]

    Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M

    Zongyi Li, Nikola B. Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations. CoRR, abs/2003.03485, 2020

    2003 arXiv

  26. [37]

    Fourier neural operator with learned deformations for pdes on general geometries

    Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1–26, 2023

    2023

  27. [38]

    U-NO: U-shaped neural operators

    Md Ashiqur Rahman, Zachary E Ross, and Kamyar Azizzadenesheli. U-NO: U-shaped neural operators. Transactions on Machine Learning Research, 2023

    2023

  28. [39]

    Factorized fourier neural operators

    Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In The Eleventh International Conference on Learning Representations, 2023

    2023

  29. [40]

    Learning deep implicit fourier neural operators (ifnos) with applications to heterogeneous material modeling

    Huaiqian You, Quinn Zhang, Colton J Ross, Chung-Hao Lee, and Yue Yu. Learning deep implicit fourier neural operators (ifnos) with applications to heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering, 398:115296, 2022

    2022

  30. [41]

    Choose a transformer: Fourier or galerkin

    Shuhao Cao. Choose a transformer: Fourier or galerkin. Advances in neural information processing systems, 34:24924–24940, 2021

    2021

  31. [42]

    Transformer meets boundary value inverse problems

    Ruchi Guo, Shuhao Cao, and Long Chen. Transformer meets boundary value inverse problems. In The Eleventh International Conference on Learning Representations, 2022

    2022

  32. [43]

    Gnot: A general neural operator transformer for operator learning

    Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning , pages 12556–12569. PMLR, 2023

    2023

  33. [44]

    Learning operators with coupled attention

    Georgios Kissas, Jacob H Seidman, Leonardo Ferreira Guilhoto, Victor M Preciado, George J Pappas, and Paris Perdikaris. Learning operators with coupled attention. Journal of Machine Learning Research, 23(215):1–63, 2022

    2022

  34. [45]

    Mesh-independent operator learning for partial di fferential equations

    Seungjun Lee. Mesh-independent operator learning for partial di fferential equations. In ICML 2022 2nd AI for Science Workshop, 2022

    2022

  35. [46]

    Scalable transformer for pde surrogate modeling

    Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for pde surrogate modeling. Advances in Neural Information Processing Systems, 36, 2024

    2024

  36. [47]

    Mgnet: A unified framework of multigrid and convolutional neural network.Science china mathematics, 62:1331– 1354, 2019

    Juncai He and Jinchao Xu. Mgnet: A unified framework of multigrid and convolutional neural network.Science china mathematics, 62:1331– 1354, 2019

    2019

  37. [48]

    MgNO: E fficient parameterization of linear operators via multigrid

    Juncai He, Xinliang Liu, and Jinchao Xu. MgNO: E fficient parameterization of linear operators via multigrid. In The Twelfth International Conference on Learning Representations, 2024

    2024

  38. [49]

    An enhanced v-cycle mgnet model for operator learning in numerical partial di fferential equations

    Jianqing Zhu, Juncai He, and Qiumei Huang. An enhanced v-cycle mgnet model for operator learning in numerical partial di fferential equations. Computational Geosciences, pages 1–12, 2023

    2023

  39. [50]

    Fv-mgnet: Fully connected v-cycle mgnet for interpretable time series forecasting

    Jianqing Zhu, Juncai He, Lian Zhang, and Jinchao Xu. Fv-mgnet: Fully connected v-cycle mgnet for interpretable time series forecasting. Journal of Computational Science, 69:102005, 2023

    2023

  40. [51]

    Mod-net: A machine learning approach via model-operator-data network for solving pdes

    Lulu Zhang, Tao Luo, Yaoyu Zhang, Zhi-Qin John Xu, and Zheng Ma. Mod-net: A machine learning approach via model-operator-data network for solving pdes. Communications in Computational Physics, 32(2):299–335, 2022

    2022

  41. [52]

    Deepgreen: deep learning of green’s functions for nonlinear boundary value problems

    Craig R Gin, Daniel E Shea, Steven L Brunton, and J Nathan Kutz. Deepgreen: deep learning of green’s functions for nonlinear boundary value problems. Scientific reports, 11(1):21614, 2021

    2021

  42. [53]

    Bi-greennet: learning green’s functions by boundary integral network

    Guochang Lin, Fukai Chen, Pipi Hu, Xiang Chen, Junqing Chen, Jun Wang, and Zuoqiang Shi. Bi-greennet: learning green’s functions by boundary integral network. Communications in Mathematics and Statistics, 11(1):103–129, 2023

    2023

  43. [54]

    Deep surrogate model for learning green’s function associated with linear reaction-diffusion operator

    Junqing Ji, Lili Ju, and Xiaoping Zhang. Deep surrogate model for learning green’s function associated with linear reaction-diffusion operator. arXiv preprint arXiv:2310.03642, 2023

    2023 arXiv

  44. [55]

    Deep generalized green’s functions

    Rixi Peng, Juncheng Dong, Jordan Malof, Willie J Padilla, and Vahid Tarokh. Deep generalized green’s functions. arXiv preprint arXiv:2306.02925, 2023

    2023 arXiv

  45. [56]

    Learning green’s functions of linear reaction-di ffusion equations with application to 24 fast numerical solver

    Yuankai Teng, Xiaoping Zhang, Zhu Wang, and Lili Ju. Learning green’s functions of linear reaction-di ffusion equations with application to 24 fast numerical solver. In Mathematical and Scientific Machine Learning, pages 1–16. PMLR, 2022

    2022

  46. [57]

    Green multigrid network

    Ye Lin, Young Ju Lee, and Jiwei Jia. Green multigrid network. arXiv preprint arXiv:2407.03593, 2024

    2024 arXiv

  47. [58]

    Yuwen Li and Jonathan W. Siegel. Entropy-based convergence rates of greedy algorithms. Mathematical Models and Methods in Applied Sciences, 34(05):779–802, 2024

    2024

  48. [59]

    Remarques sur un r ´esultat non publi´e de b

    Gilles Pisier. Remarques sur un r ´esultat non publi´e de b. maurey. S´ eminaire d’Analyse fonctionnelle (dit” Maurey-Schwartz”), pages 1–12, 1981

    1981

  49. [60]

    Gerald B. Folland. Real Analysis: Modern Techniques and Their Applications, 2nd Edition. Wiley, 2013

    2013

  50. [61]

    Multilevel multi-integration algorithm for acoustics

    Isa ´ıas Hern´andez Ram´ırez. Multilevel multi-integration algorithm for acoustics. Phd thesis - research ut, graduation ut, University of Twente, Netherlands, September 2005

    2005

  51. [62]

    Mallat and Zhifeng Zhang

    S.G. Mallat and Zhifeng Zhang. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41(12):3397– 3415, 1993

    1993

  52. [63]

    A simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training

    Lee K Jones. A simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training. The annals of Statistics, pages 608–613, 1992

    1992

  53. [64]

    V . M. Tikhomirov.ϵ-Entropy andϵ-Capacity of Sets In Functional Spaces, pages 86–170. Springer Netherlands, Dordrecht, 1993

    1993

  54. [65]

    Siegel and Jinchao Xu

    Jonathan W. Siegel and Jinchao Xu. Optimal convergence rates for the orthogonal greedy algorithm. IEEE Transactions on Information Theory, 68(5):3354–3361, 2022

    2022

  55. [66]

    Entropy-based convergence rates of greedy algorithms

    Yuwen Li and Jonathan Siegel. Entropy-based convergence rates of greedy algorithms. M3AS, 34(5), 2024

    2024

  56. [67]

    Universal approximation bounds for superpositions of a sigmoidal function

    Andrew R Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information theory, 39(3):930–945, 1993

    1993

  57. [68]

    Hinging hyperplanes for regression, classification, and function approximation

    Leo Breiman. Hinging hyperplanes for regression, classification, and function approximation. IEEE Transactions on Information Theory , 39(3):999–1013, 1993

    1993

  58. [69]

    Approximation by combinations of relu and squared relu ridge functions with \ellˆ 1 and\ellˆ 0 controls

    Jason M Klusowski and Andrew R Barron. Approximation by combinations of relu and squared relu ridge functions with \ellˆ 1 and\ellˆ 0 controls. IEEE Transactions on Information Theory, 64(12):7649–7656, 2018

    2018

  59. [70]

    Tighter sparse approximation bounds for relu neural networks

    Carles Domingo-Enrich and Youssef Mroueh. Tighter sparse approximation bounds for relu neural networks. arXiv preprint arXiv:2110.03673, 2021

    2021 arXiv

  60. [71]

    On the activation function dependence of the spectral bias of neural networks

    Qingguo Hong, Jonathan W Siegel, Qinyang Tan, and Jinchao Xu. On the activation function dependence of the spectral bias of neural networks. arXiv preprint arXiv:2208.04924, 2022

    2022 arXiv

  61. [72]

    Bridging traditional and machine learning-based algorithms for solving pdes: the random feature method

    Jingrun Chen, Xurong Chi, Zhouwang Yang, et al. Bridging traditional and machine learning-based algorithms for solving pdes: the random feature method. J Mach Learn, 1:268–98, 2022

    2022

  62. [73]

    Can physics-informed neural networks beat the finite element method? IMA Journal of Applied Mathematics, page hxae011, 2024

    Tamara G Grossmann, Urszula Julia Komorowska, Jonas Latz, and Carola-Bibiane Sch ¨onlieb. Can physics-informed neural networks beat the finite element method? IMA Journal of Applied Mathematics, page hxae011, 2024

    2024

  63. [74]

    Why shallow networks struggle with approximating and learning high frequency: A numerical study

    Shijun Zhang, Hongkai Zhao, Yimin Zhong, and Haomin Zhou. Why shallow networks struggle with approximating and learning high frequency: A numerical study. arXiv preprint arXiv:2306.17301, 2023

    2023 arXiv

  64. [75]

    Monte Carlo methods in statistical physics, volume 7

    Kurt Binder, David M Ceperley, J-P Hansen, MH Kalos, DP Landau, D Levesque, H Mueller-Krumbhaar, D Stau ffer, and J-J Weis. Monte Carlo methods in statistical physics, volume 7. Springer Science & Business Media, 2012

    2012

  65. [76]

    Gaussian processes for machine learning , volume 2

    Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning , volume 2. MIT press Cambridge, MA, 2006

    2006

  66. [77]

    A Driscoll, N

    T. A Driscoll, N. Hale, and L. N. Trefethen. Chebfun Guide. Pafnuty Publications, 2014

    2014

  67. [78]

    Gmsh: A 3-d finite element mesh generator with built-in pre-and post-processing facilities

    Christophe Geuzaine and Jean-Franc ¸ois Remacle. Gmsh: A 3-d finite element mesh generator with built-in pre-and post-processing facilities. International journal for numerical methods in engineering, 79(11):1309–1331, 2009

    2009

  68. [79]

    Baratta, Joseph P

    Igor A. Baratta, Joseph P. Dean, Jørgen S. Dokken, Michal Habera, Jack S. Hale, Chris N. Richardson, Marie E. Rognes, Matthew W. Scroggs, Nathan Sime, and Garth N. Wells. DOLFINx: The next generation FEniCS problem solving environment, December 2023

    2023

  69. [80]

    Gaussianrandomfields.jl: A julia package to generate and sample from gaussian random fields

    Pieterjan Robbe. Gaussianrandomfields.jl: A julia package to generate and sample from gaussian random fields. Journal of Open Source Software, 8(89):5595, 2023

    2023

  70. [81]

    DeepXDE: A deep learning library for solving di fferential equations

    Lu Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. DeepXDE: A deep learning library for solving di fferential equations. SIAM Review, 63(1):208–228, 2021

    2021

  71. [82]

    Rational neural networks

    Nicolas Boull ´e, Yuji Nakatsukasa, and Alex Townsend. Rational neural networks. Advances in neural information processing systems , 33:14243–14253, 2020. Appendix A. Appendix Appendix A.1. Deep operator network baselines We implement the Deep operator network(DON) using the D...

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.