Pith. sign in

REVIEW 2 major objections 4 minor 122 references

Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Many neural-network layers become resolution-agnostic when their discrete sums are replaced by integrals with quadrature weights.

desk verdict A useful and honest synthesis of neural-operator recipes from standard NN layers; not a new result, but the cleanest practical guide I've seen, with thin-but-illustrative experiments. read the letter →

arxiv 2506.10973 v1 pith:WLAZJR2K submitted 2025-06-12 cs.LG cs.AIcs.NAmath.FAmath.NA

Julius Berner , Miguel Liu-Schiaffini , Jean Kossaifi , Valentin Duruisseaux , Boris Bonev , Kamyar Azizzadenesheli , Anima Anandkumar This is my paper
classification cs.LGcs.AIcs.NAmath.FAmath.NA MSC 68T0765D32
keywords neuraloperatorsoperatorlearningdiscretization-agnosticquadratureweightsgraphFourierfunctionspacespartialdifferentialequations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Many well-known neural-network layers—fully-connected, convolutional, graph, and transformer attention—can be converted into neural operators, mappings between function spaces, by a simple recipe: replace the discrete sum over neighboring points with an integral over a fixed patch of the domain, and approximate that integral by a sum with quadrature weights. The paper argues this replacement is what makes a model agnostic to the discretization, so the same trained model can be evaluated at arbitrary resolutions with errors that shrink as the mesh is refined. This matters for scientific machine learning because PDE solvers produce data on adaptive meshes, and operators that generalize across resolutions can exploit multi-resolution training data and do zero-shot super-resolution. The claim is supported by experiments on the Navier-Stokes equations, where a Fourier neural operator trained at one resolution keeps its accuracy at other resolutions while a U-Net and a vision transformer do not.

What carries the argument

The carrying identity is Eq. (19), the graph neural operator (GNO): $g(y_j)=\sum_{i:\,x_i\in D(y_j)} K(x_i,y_j,f(x_i),f(y_j))\,\Delta_i$, which approximates the integral operator $\int_{D(y_j)} K(x,y_j,f(x),f(y_j))\,\mathrm{d}x$. The quadrature weights $\Delta_i$ (e.g., cell volumes from a Delaunay triangulation of the point cloud, or grid spacings on regular grids) make the discrete sum converge to the integral as the discretization is refined; fixed domain subsets $D(y_j)$ (radius balls, or the whole domain for global operators) keep the receptive field independent of resolution. Additional machinery includes pointwise operators for activations and liftings, spectral convolution via the Fourier transform for global convolutions, and inner-product encoders with parametric decoders for encoder-decoder layers. The recipe's role is to convert each finite-dimensional layer into a discretization-convergent layer, so the whole architecture inherits resolution-agnostic behavior.

What would settle it

On an irregular point cloud, take a discontinuous integrand such as a step function and compare the Delaunay-triangulation quadrature estimate against the true integral as the number of points grows; if the quadrature error does not tend to zero, the Riemann-sum assumption behind Eq. (19) is violated. In the operator setting, train a GNO-style model on a PDE with shock or discontinuous coefficient data at two resolutions and evaluate zero-shot at intermediate resolutions: the claim predicts relative error should decrease with resolution, so a systematic divergence or plateau would falsify it.

Watch

Extended reading notes

Core claim

The paper's central claim is that most neural network architectures can be viewed as graph neural networks, and that turning them into neural operators requires one structural change: replace the sum over neighbors, $g_j=\sum_{i\in\mathrm{Neighb}_j}K_{ij}(f_i,f_j)$, with an integral over a fixed subset $D(y_j)$ of the spatial domain, $g(y_j)=\int_{x\in D(y_j)} K(x,y_j,f(x),f(y_j))\,\mathrm{d}x$, and then discretize that integral with quadrature weights $\Delta_i$, giving Eq.~(19). Because the neighborhood $D(y_j)$ is defined in the domain rather than by index proximity, the receptive field stays fixed as the resolution changes, and because the sum carries quadrature weights, the discrete model converges to the integral operator as discretizations are refined. The paper shows this recipe subsumes MLPs turned into integral transforms, convolutional operators with learned kernels, Fourier spectral convolutions, graph neural operators, resolution-agnostic self-attention, and encoder-decoder operators with finite-dimensional latents. Empirically, on the Navier-Stokes benchmark, models built this way generalize across resolutions, whereas standard U-Nets, vision transformers, and multi-resolution-augmented neural networks do not.

Load-bearing premise

The recipe assumes that the fields aggregated over each neighborhood are regular enough that Riemann-sum quadrature with the chosen point weights converges to the true integral, so for sharply discontinuous or highly irregular fields the quadrature weights from a Delaunay triangulation may not converge and the discretization-agnostic property could fail.

Editorial extensions

If this is right

  • Any architecture expressible as a message-passing graph neural network can be converted into a neural operator by using domain-based neighborhoods and quadrature-weighted aggregation, making it applicable to arbitrary point clouds.
  • A neural operator trained at one resolution can be evaluated at higher and lower resolutions with only discretization error, enabling zero-shot super-resolution without retraining.
  • Multi-resolution training of ordinary neural networks helps at the resolutions seen during training but does not confer reliable generalization to unseen resolutions.
  • Interpolating inputs and outputs to a fixed resolution defines a naive neural operator but discards the high-resolution information in the input, limiting accuracy.
  • Losses, normalization statistics, and inner products used for training should be computed with quadrature weights so that the training objective itself is resolution-agnostic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the recipe is as general as claimed, porting mature architectures—segmentation U-Nets, vision transformers, and graph networks—into operator form with quadrature weights should reproduce their empirical strengths in scientific applications, something the paper tests only for a few cases.
  • The quadrature-weight analysis implies that on regular uniform grids, where weights cancel in self-attention, standard transformers are already discretization-consistent; the practical gap should appear mainly on irregular or adaptive meshes.
  • A testable extension is to apply the recipe to point-cloud data with strongly non-uniform density and compare quadrature-weighted GNOs against unweighted message passing; the paper predicts the weighted version will generalize across samplings while the unweighted one will not.
  • One could audit existing neural operators by checking whether their discrete layers converge to the claimed integral operator under refinement; any layer whose sums lack quadrature weights is a candidate for the same collapse-to-pointwise failure described for CNNs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes a set of design principles for neural operators (discretization-agnostic behavior, fixed parameter count, and universal approximation) and derives a recipe for converting standard neural-network layers into operator layers. The recipe replaces index-based sums by quadrature-weighted sums over subsets of the underlying domain, parametrizes kernels by learnable functions, and fixes receptive fields independently of resolution. These ideas are applied to MLPs, CNNs, GNNs, transformers, and encoder-decoder architectures, recovering and unifying existing models such as GNO, FNO, DeepONet, and transformer neural operators. Experiments on a Navier-Stokes benchmark compare U-Net, ViT, FNO, OFormer, and composite operators across resolutions, and the appendices provide numerical integration details, an error decomposition for zero-shot resolution transfer, and experimental details.

Significance. The paper's main value is a unifying and practically actionable framework: it shows that many existing neural-operator architectures arise from one conversion recipe, gives clean, self-contained derivations in Section 3, and ships code. The paper also makes the useful point that naive discretizations of CNNs/GNNs have resolution-dependent receptive fields, and its experiments illustrate the practical importance of quadrature weights and fixed receptive fields. However, the central claims are stated more strongly than the evidence supports: the discretization-agnostic guarantee is established only under regularity conditions that are not flagged in the main text, and the empirical section rests on one dataset without reported seed variance. These issues are fixable with explicit caveats and additional experiments, so the paper is, in my view, a solid candidate after revision.

major comments (2)
  1. [Section 3.8, Eq. (19); Appendix A.5] The central claim that Eq. (19) yields a discretization-agnostic operator is established only under regularity conditions that are not stated where the claim is made. Appendix A.5 guarantees Riemann-sum convergence for continuous integrands K, and Appendix A.4 defines F as subspaces of continuous functions; but the paper's motivation includes PDE applications with shocks, interfaces, and piecewise-constant coefficients, for which the composition K(x,y_j,f(x),f(y_j)) can be discontinuous. For such inputs the Delaunay-based quadrature weights of Algorithm 1 are not analyzed, and the refinement limit of the empirical sum may depend on the discretization. Section 6 acknowledges the need to quantify and control discretization error in practice, but Eq. (19) and the surrounding summary are presented without this qualification. Please add an explicit regularity assumption, or an analysis for piecewise-continuous data, to the statement of the recipe and flag the limitation in the main text.
  2. [Section 5.2, Figure 5; Appendix C] The empirical section is the main support for the practical claims that quadrature weights and fixed receptive fields improve cross-resolution generalization, but it reports results on a single Navier-Stokes dataset and does not state the number of seeds or show error bars. The text claims in Section 5.2 that the experiments 'empirically verify three main principles,' which is too strong without variance information, particularly for conclusions about mixed-resolution training and kernel interpolation at unseen resolutions. Please report multiple seeds with standard deviations, or temper the claim to 'illustrate' rather than 'verify.'
minor comments (4)
  1. [Section 1, Contribution 1] The contribution statement says the design principles are 'necessary for well-posed operator learning,' but Section 2 motivates them as desirable properties and does not prove necessity. This wording overstates the formal content; either prove the necessity claim or soften it to 'central to' well-posed operator learning.
  2. [Section 3.8, Figure 2 caption] There is a typo: '(spherial) FNOs' should be '(spherical) FNOs.'
  3. [Section 3.4, Eq. (11)] The notation in Eq. (11) uses f(x) and f(y_j) inside the integral, but the surrounding text switches between 'f(x)' and 'f(xi)' when discussing discretizations; this is understandable but could be clarified by using a dummy variable consistently, e.g., f(x) for the continuous integrand and f(x_i) for the discretized sum.
  4. [Appendix A.5, Eq. (31)] The Monte-Carlo quadrature derivation is correct but the notation for the quadrature weight underbraces is visually confusing; a short sentence defining Delta_i = (n p(x_i))^{-1} would improve readability.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the conversion recipe is derived self-contained from discrete layers to integral operators, and the quadrature/regularity caveats are limitations rather than circular inputs.

full rationale

We traced the claimed derivation chain from discrete layers (Eqs. (3), (7), (10), (12), (17)) to continuous operators (Eqs. (4), (8), (11), (13), (18)) and back to the discretized graph neural operator (Eq. (19)). Each step replaces index-based weights with evaluations of learnable kernel functions and inserts quadrature weights; the continuous limit is a genuine integral and the discretized form is a Riemann-sum approximation whose convergence is stated in Appendix A.5 for continuous integrands. The discretization-agnostic property is therefore obtained by construction from a standard convergence theorem, not from a fitted parameter or from a self-referential uniqueness claim. The numerical experiments in Section 5.2 compare independent architectures on the Navier-Stokes benchmark and do not rename any fitted quantity as a prediction. The paper does cite heavily from its own authors (e.g., Kovachki et al. 2023 for operator approximation and convergence results, Liu-Schiaffini et al. 2024 for LocalConv/DiffConv layers, and Azizzadenesheli et al. 2024 for applications), but these citations supply background, baselines, and previously established theoretical results; removing them would not collapse the derivation because the core equations are derived in the text. The Appendix A.5 regularity caveat (convergence guaranteed for continuous K) is a genuine limitation for discontinuous or highly oscillatory inputs, but it is a correctness/robustness concern rather than a circular step. Overall, no step in the derivation reduces by construction to its own input; the score of 2 reflects the presence of minor, non-load-bearing self-citations, not actual circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities, forces, or dimensions; all layers derive from existing mathematical structures such as integral operators and Fourier bases. The central recipe does not introduce ad hoc free parameters; the experimental hyperparameters (number of modes, hidden channels, radius cutoff) are standard model configuration choices, not parameters fitted to data in support of the derivation.

assumptions (4)
  • standard math Universal approximation of neural operators by architectures such as FNO and GNO (Kovachki et al., 2023; Lanthaler et al., 2022).
    The paper treats universal approximation as a defining property and relies on it to argue the recipe produces useful models (Section 2.1, Appendix B).
  • standard math Quadrature rules based on partitions or Delaunay triangulations converge to Riemann integrals for continuous integrands.
    The discretization-agnostic operators in Eqs. (4), (8), (11), and (13) assume the weighted sums converge to the corresponding integrals as resolution increases (Appendix A.5).
  • domain assumption Input functions are continuous and can be adequately represented by point evaluations.
    The paper's framework assumes each input function is known through its values on a point cloud, and that refining the cloud recovers the function (Section 2, Appendix A.1).
  • domain assumption The Navier-Stokes force-to-vorticity experiment is representative of operator learning tasks that benefit from discretization-agnostic models.
    The empirical conclusions about generalization across resolutions are drawn from a single PDE task with one dataset split (Section 5.2, Appendix C).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning." pith.science (2026). https://pith.science/paper/WLAZJR2K

@misc{pith2026250610973,
  author       = {Pith},
  title        = {Pith review of: Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WLAZJR2K}},
  note         = {Machine review of arXiv:2506.10973}
}
read the original abstract

A wide range of scientific problems, such as those described by continuous-time dynamical systems and partial differential equations (PDEs), are naturally formulated on function spaces. While function spaces are typically infinite-dimensional, deep learning has predominantly advanced through applications in computer vision and natural language processing that focus on mappings between finite-dimensional spaces. Such fundamental disparities in the nature of the data have limited neural networks from achieving a comparable level of success in scientific applications as seen in other fields. Neural operators are a principled way to generalize neural networks to mappings between function spaces, offering a pathway to replicate deep learning's transformative impact on scientific problems. For instance, neural operators can learn solution operators for entire classes of PDEs, e.g., physical systems with different boundary conditions, coefficient functions, and geometries. A key factor in deep learning's success has been the careful engineering of neural architectures through extensive empirical testing. Translating these neural architectures into neural operators allows operator learning to enjoy these same empirical optimizations. However, prior neural operator architectures have often been introduced as standalone models, not directly derived as extensions of existing neural network architectures. In this paper, we identify and distill the key principles for constructing practical implementations of mappings between infinite-dimensional function spaces. Using these principles, we propose a recipe for converting several popular neural architectures into neural operators with minimal modifications. This paper aims to guide practitioners through this process and details the steps to make neural operators work in practice. Our code can be found at https://github.com/neuraloperator/NNs-to-NOs

Figures

Figures reproduced from arXiv: 2506.10973 by the authors.

Figure 1
Figure 1. Illustration of a neural operator. The input is a function f ∈ F that can be given at different discretizations (xi) n i=1. The output is a function g ∈ G that can be queried at different points (yj ) m j=1. between function spaces parametrized by a finite number of parameters. They should be discretization￾agnostic and be able to approximate any sufficiently regular operator with arbitrarily low error. 2. Using the… view at source ↗
Figure 2
Figure 2. Pipeline of converting neural networks to neural operators. Graph neural network (GNN) and convolutional layers can be converted into well-posed neural operator layers through a sequence of simple modifications. GNO refers to the graph neural operator (Li et al., 2020b), “Spec. conv.” refers to a spectral convolution as used in Fourier neural operators (FNOs) (Li et al., 2020a), and Local FNO refers to a FNO supplem… view at source ↗
Figure 3
Figure 3. Visualizing the need for quadrature weights. Aggregating function values at irregularly￾spaced points without proper quadrature weights (e.g., taking the mean, i.e., ∆ = 1/n, as in the top figure) adds more weight to densely sampled areas. When increasing the resolution, the output depends on the chosen refinement of the discretization and does not have a unique limit (top). The use of quadrature weights leads to co… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Illustration of collapsing receptive fields with a nearest neighbors strategy. The figure shows the values of the input function f (blue) which influence the output function g at a point y when using a nearest neighbors strategy (e.g. as in convolutional and graph neur…
Figure 5
Figure 5. Figure 5: Relative L 2 -errors (on unseen test data) when training different methods on the Navier-Stokes equations (see Appendix C for details). Although FNO and OFormer are only trained on resolution 128, they achieve approximately the same error for higher and lower resolutio…
Figure 6
Figure 6. Figure 6: Sample input and output functions and two discretizations for learning the mapping from forcing [PITH_FULL_IMAGE:figures/full_fig_p037_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

122 extracted references · 25 canonical work pages

  1. [1]

    Junaid Ahmed and Alhassan S. Yasin. Graph neural operators for classification of spatial transcriptomics data. arXiv preprint arXiv:2302.00658, 2023

    work page Pith review arXiv 2023

  2. [2]

    Scalable second order optimization for deep learning

    Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, and Yoram Singer. Scalable second order optimization for deep learning. arXiv preprint arXiv:2002.09018, 2020

    arXiv 2002

  3. [3]

    Neural operators for accelerating scientific simulations and design

    Kamyar Azizzadenesheli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiaffini, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, pp.\ 1--9, 2024

    2024

  4. [4]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014

    arXiv 2014

  5. [5]

    Representation Equivalent Neural Operators: a Framework for Alias-free Operator Learning

    Francesca Bartolucci, Emmanuel de B \'e zenac, Bogdan Raoni \'c , Roberto Molinaro, Siddhartha Mishra, and Rima Alaifari. Are neural operators really neural operators? frame theory meets operator learning. arXiv preprint arXiv:2305.19913, 2023

    work page Pith review arXiv 2023

  6. [6]

    Model reduction and neural networks for parametric PDEs

    Kaushik Bhattacharya, Bamdad Hosseini, Nikola B Kovachki, and Andrew M Stuart. Model reduction and neural networks for parametric PDEs . The SMAI journal of computational mathematics, 7: 0 121--157, 2021

    2021

  7. [7]

    Spherical Fourier neural operators: learning stable dynamics on the sphere

    Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical Fourier neural operators: learning stable dynamics on the sphere. Proceedings of the 40th International Conference on Machine Learning (ICML), 202: 0 2806--2823, 2023

    2023

  8. [8]

    A mathematical guide to operator learning

    Nicolas Boull \'e and Alex Townsend. A mathematical guide to operator learning. arXiv preprint arXiv:2312.14688, 2023

    arXiv 2023

Show all 122 references
  1. [9]

    Lie point symmetry data augmentation for neural PDE solvers

    Johannes Brandstetter, Max Welling, and Daniel E Worrall. Lie point symmetry data augmentation for neural PDE solvers. In International Conference on Machine Learning, pp.\ 2241--2256. PMLR, 2022

    2022

  2. [10]

    Message passing neural PDE solvers

    Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural PDE solvers. arXiv preprint arXiv:2202.03376, 2023

    2023 arXiv

  3. [11]

    Geometric deep learning: Grids, groups, graphs, geodesics, and gauges

    Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veli c kovi \'c . Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021

    2021 arXiv

  4. [12]

    Continuum attention for neural operators

    Edoardo Calvello, Nikola B Kovachki, Matthew E Levine, and Andrew M Stuart. Continuum attention for neural operators. arXiv preprint arXiv:2406.06486, 2024

    2024

  5. [13]

    LNO : Laplace neural operator for solving differential equations

    Qianying Cao, Somdatta Goswami, and George Em Karniadakis. LNO : Laplace neural operator for solving differential equations. arXiv preprint arXiv:2303.10528, 2023

    2023 arXiv

  6. [14]

    Choose a transformer: Fourier or Galerkin

    Shuhao Cao. Choose a transformer: Fourier or Galerkin . Advances in neural information processing systems, 34: 0 24924--24940, 2021

    2021

  7. [15]

    Carey, L

    N. Carey, L. Zanisi, S. Pamela, V. Gopakumar, J. Omotani, J. Buchanan, J. Brandstetter, F. Paischer, G. Galletti, and P. Setinek. Data efficiency and long-term prediction capabilities for neural operator surrogate models of edge plasma simulations. arXiv preprint arXiv:2502.17...

    2025 arXiv

  8. [16]

    Alpak, Farah Hariri, Clement Etienam, Pandu Devarakota, Anima Anandkumar, and Detlef Hohl

    Anirban Chandra, Marius Koch, Suraj Pawar, Aniruddha Panda, Kamyar Azizzadenesheli, Jeroen Snippe, Faruk O. Alpak, Farah Hariri, Clement Etienam, Pandu Devarakota, Anima Anandkumar, and Detlef Hohl. Fourier neural operator based surrogates for CO _2 storage in realistic geolog...

    2025 arXiv

  9. [17]

    CROM : Continuous reduced-order modeling of PDEs using implicit neural representations

    Peter Yichen Chen, Jinxu Xiang, Dong Heon Cho, Yue Chang, GA Pershing, Henrique Teles Maia, Maurizio Chiaramonte, Kevin Carlberg, and Eitan Grinspun. CROM : Continuous reduced-order modeling of PDEs using implicit neural representations. arXiv preprint arXiv:2206.02607, 2022

    2022 arXiv

  10. [18]

    Equivariant neural operator learning with graphon convolution

    Chaoran Cheng and Jian Peng. Equivariant neural operator learning with graphon convolution. Advances in Neural Information Processing Systems, 36, 2024

    2024

  11. [19]

    Rethinking attention with performers

    Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. Rethinking attention with performers. arXiv preprint arXiv:2009.14794, 2020

    2009 arXiv

  12. [20]

    Fourier neural operator for fluid flow in small-shape 2D simulated porous media dataset

    A Choubineh, J Chen, DA Wood, F Coenen, and F Ma. Fourier neural operator for fluid flow in small-shape 2D simulated porous media dataset. Algorithms, 16 0 (1): 0 24, 2023

    2023

  13. [21]

    Kirby, and Sarang Joshi

    Haocheng Dai, Michael Penwarden, Robert M. Kirby, and Sarang Joshi. Neural operator learning for ultrasound tomography inversion. arXiv preprint arXiv:2304.03297, 2023

    2023 arXiv

  14. [22]

    Generic bounds on the approximation error for physics-informed (and) operator learning

    Tim De Ryck and Siddhartha Mishra. Generic bounds on the approximation error for physics-informed (and) operator learning. Advances in Neural Information Processing Systems, 35: 0 10945--10958, 2022

    2022

  15. [23]

    Nonlinear approximation

    Ronald A DeVore. Nonlinear approximation. Acta numerica, 7: 0 51--150, 1998

    1998

  16. [24]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv...

    2010 arXiv

  17. [25]

    Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric . In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019

    2019

  18. [26]

    Incremental spatial and spectral learning of neural operators for solving large-scale PDEs

    Robert Joseph George, Jiawei Zhao, Jean Kossaifi, Zongyi Li, and Anima Anandkumar. Incremental spatial and spectral learning of neural operators for solving large-scale PDEs . arXiv preprint arXiv:2211.15188, 2022

    2022 arXiv

  19. [27]

    Understanding the difficulty of training deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp.\ 249--256. JMLR Workshop and Conference Proceedings, 2010

    2010

  20. [28]

    A Kaczmarz -inspired approach to accelerate the optimization of neural network wavefunctions

    Gil Goldshlager, Nilin Abrahamsen, and Lin Lin. A Kaczmarz -inspired approach to accelerate the optimization of neural network wavefunctions. arXiv preprint arXiv:2401.10190, 2024

    2024 arXiv

  21. [29]

    Plasma surrogate modelling using fourier neural operators

    Vignesh Gopakumar, Stanislas Pamela, Lorenzo Zanisi, Zongyi Li, Ander Gray, Daniel Brennand, Nitesh Bhatia, Gregory Stathopoulos, Matt Kusner, Marc Peter Deisenroth, Anima Anandkumar, the JOREK Team, and MAST Team. Plasma surrogate modelling using fourier neural operators. Nuc...

    2024

  22. [30]

    Theory-to-practice gap for neural networks and neural operators

    Philipp Grohs, Samuel Lanthaler, and Margaret Trautner. Theory-to-practice gap for neural networks and neural operators. arXiv preprint arXiv:2503.18219, 2025

    2025 arXiv

  23. [31]

    Adaptive Fourier neural operators: Efficient token mixers for transformers

    John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, and Bryan Catanzaro. Adaptive Fourier neural operators: Efficient token mixers for transformers. arXiv preprint arXiv:2111.13587, 2021

    2021 arXiv

  24. [32]

    Multiwavelet-based operator learning for differential equations

    Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. Multiwavelet-based operator learning for differential equations. Advances in neural information processing systems, 34: 0 24048--24062, 2021

    2021

  25. [33]

    Towards multi-spatiotemporal-scale generalized PDE modeling

    Jayesh K Gupta and Johannes Brandstetter. Towards multi-spatiotemporal-scale generalized PDE modeling. arXiv preprint arXiv:2209.15616, 2022

    2022 arXiv

  26. [34]

    GNOT : A general neural operator transformer for operator learning

    Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. GNOT : A general neural operator transformer for operator learning. In International Conference on Machine Learning, pp.\ 12556--12569. PMLR, 2023 a

    2023

  27. [35]

    Pinnacle: A comprehensive benchmark of physics-informed neural networks for solving PDEs

    Zhongkai Hao, Jiachen Yao, Chang Su, Hang Su, Ziao Wang, Fanzhi Lu, Zeyu Xia, Yichi Zhang, Songming Liu, Lu Lu, et al. Pinnacle: A comprehensive benchmark of physics-informed neural networks for solving PDEs . arXiv preprint arXiv:2306.08827, 2023 b

    2023 arXiv

  28. [36]

    DPOT : Auto-regressive denoising operator transformer for large-scale PDE pre-training

    Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, and Jun Zhu. DPOT : Auto-regressive denoising operator transformer for large-scale PDE pre-training. In Forty-first International Conference on Machine Learning, 2024

    2024

  29. [37]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp.\ 1026--1034, 2015

    2015

  30. [38]

    Group equivariant Fourier neural operators for partial differential equations

    Jacob Helwig, Xuan Zhang, Cong Fu, Jerry Kurtin, Stephan Wojtowytsch, and Shuiwang Ji. Group equivariant Fourier neural operators for partial differential equations. arXiv preprint arXiv:2306.05697, 2023

    2023 arXiv

  31. [39]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015

    2015 arXiv

  32. [40]

    A unified model for compressed sensing mri across undersampling patterns

    Armeet Singh Jatyani, Jiayun Wang, Aditi Chandrashekar, Zihui Wu, Miguel Liu-Schiaffini, Bahareh Tolooshams, and Anima Anandkumar. A unified model for compressed sensing mri across undersampling patterns. arXiv preprint arXiv:2410.16290, 2024

    arXiv

  33. [41]

    Perfception: Perception using radiance fields

    Yoonwoo Jeong, Seungjoo Shin, Junha Lee, Chris Choy, Anima Anandkumar, Minsu Cho, and Jaesik Park. Perfception: Perception using radiance fields. In Advances in Neural Information Processing Systems, volume 35, pp.\ 26105--26121, 2022

    2022

  34. [42]

    Jiang, Z

    P. Jiang, Z. Yang, J. Wang, C. Huang, P. Xue, T.C. Chakraborty, et al. Efficient super-resolution of near-surface climate modeling using the Fourier neural operator. Journal of Advances in Modeling Earth Systems, 15, 2023

    2023

  35. [43]

    Resolution-invariant image classification based on Fourier neural operators

    Samira Kabri, Tim Roith, Daniel Tenbrinck, and Martin Burger. Resolution-invariant image classification based on Fourier neural operators. In International Conference on Scale Space and Variational Methods in Computer Vision, pp.\ 236--249. Springer, 2023

    2023

  36. [44]

    Multi-task learning using uncertainty to weigh losses for scene geometry and semantics

    Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 7482--7491, 2018

    2018

  37. [45]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  38. [46]

    Multi-grid tensorized Fourier neural operator for high-resolution PDEs

    Jean Kossaifi, Nikola Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar. Multi-grid tensorized Fourier neural operator for high-resolution PDEs . arXiv preprint arXiv:2310.00120, 2023

    2023 arXiv

  39. [47]

    A library for learning neural operators

    Jean Kossaifi, Nikola Kovachki, Zongyi Li, David Pitt, Miguel Liu-Schiaffini, Valentin Duruisseaux, Robert Joseph George, Boris Bonev, Kamyar Azizzadenesheli, Julius Berner, and Anima Anandkumar. A library for learning neural operators. arXiv preprint arXiv:2412.10354, 2024

    2024

  40. [48]

    On universal approximation and error bounds for Fourier neural operators

    Nikola Kovachki, Samuel Lanthaler, and Siddhartha Mishra. On universal approximation and error bounds for Fourier neural operators. Journal of Machine Learning Research, 22 0 (290): 0 1--76, 2021

    2021

  41. [49]

    Neural operator: Learning maps between function spaces with applications to PDEs

    Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to PDEs . Journal of Machine Learning Research, 24 0 (89): 0 1--97, 2023

    2023

  42. [50]

    Data complexity estimates for operator learning

    Nikola B Kovachki, Samuel Lanthaler, and Hrushikesh Mhaskar. Data complexity estimates for operator learning. arXiv preprint arXiv:2405.15992, 2024 a

    2024 arXiv

  43. [51]

    Operator learning: Algorithms and analysis

    Nikola B Kovachki, Samuel Lanthaler, and Andrew M Stuart. Operator learning: Algorithms and analysis. arXiv preprint arXiv:2402.15715, 2024 b

    2024 arXiv

  44. [52]

    FourCastNet : Accelerating global high-resolution weather forecasting using adaptive Fourier neural operators

    Thorsten Kurth, Shashank Subramanian, Peter Harrington, Jaideep Pathak, Morteza Mardani, David Hall, Andrea Miele, Karthik Kashinath, and Anima Anandkumar. FourCastNet : Accelerating global high-resolution weather forecasting using adaptive Fourier neural operators. Proceeding...

    2023

  45. [53]

    Artificial neural networks for solving ordinary and partial differential equations

    Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 9 0 (5): 0 987--1000, 1998

    1998

  46. [54]

    Learning skillful medium-range global weather forecasting

    Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 382 0 (6677): 0 1416--1421, 2023

    2023

  47. [55]

    Operator learning with PCA-Net : upper and lower complexity bounds

    Samuel Lanthaler. Operator learning with PCA-Net : upper and lower complexity bounds. arXiv preprint arXiv:2303.16317, 2023

    2023 arXiv

  48. [56]

    Error estimates for DeepONets : A deep learning framework in infinite dimensions

    Samuel Lanthaler, Siddhartha Mishra, and George E Karniadakis. Error estimates for DeepONets : A deep learning framework in infinite dimensions. Transactions of Mathematics and Its Applications, 6 0 (1): 0 tnac001, 2022

    2022

  49. [57]

    The nonlocal neural operator: Universal approximation

    Samuel Lanthaler, Zongyi Li, and Andrew M Stuart. The nonlocal neural operator: Universal approximation. arXiv preprint arXiv:2304.13221, 2023

    2023 arXiv

  50. [58]

    Discretization error of Fourier neural operators

    Samuel Lanthaler, Andrew M Stuart, and Margaret Trautner. Discretization error of Fourier neural operators. arXiv preprint arXiv:2405.02221, 2024

    2024

  51. [59]

    Layer normalization

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. ArXiv e-prints, pp.\ arXiv--1607, 2016

    2016

  52. [60]

    Fourier neural operator approach to large eddy simulation of three-dimensional turbulence

    Zhijie Li, Wenhui Peng, Zelong Yuan, and Jianchun Wang. Fourier neural operator approach to large eddy simulation of three-dimensional turbulence. Theoretical and Applied Mechanics Letters, 12 0 (6): 0 100389, 2022 a

    2022

  53. [61]

    Transformer for partial differential equations' operator learning

    Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning. arXiv preprint arXiv:2205.13671, 2022 b

    2022 arXiv

  54. [62]

    Scalable transformer for PDE surrogate modeling

    Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for PDE surrogate modeling. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  55. [63]

    Fourier neural operator for parametric partial differential equations

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020 a

    2010 arXiv

  56. [64]

    Neural operator: Graph kernel network for partial differential equations

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020 b

    2003 arXiv

  57. [65]

    Multipole graph neural operator for parametric partial differential equations

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhattacharya, and Anima Anandkumar. Multipole graph neural operator for parametric partial differential equations. Advances in Neural Information Processing Systems, 33: 0 6755--6766, 2020 c

    2020

  58. [66]

    Learning chaotic dynamics in dissipative systems

    Zongyi Li, Miguel Liu-Schiaffini, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Learning chaotic dynamics in dissipative systems. Advances in Neural Information Processing Systems, 35: 0 16768--16781, 2022 c

    2022

  59. [67]

    Fourier neural operator with learned deformations for PDEs on general geometries

    Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for PDEs on general geometries. Journal of Machine Learning Research, 24 0 (388): 0 1--26, 2023

    2023

  60. [68]

    Geometry-informed neural operator for large-scale 3d PDEs

    Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Mohammad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry-informed neural operator for large-scale 3d PDEs . Advances in Neural Information Processing Systems...

    2024

  61. [69]

    Physics-informed neural operator for learning partial differential equations

    Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 1 0 (3): 0 1--27, 2024 c

    2024

  62. [70]

    Graph metanetworks for processing diverse neural architectures

    Derek Lim, Haggai Maron, Marc T Law, Jonathan Lorraine, and James Lucas. Graph metanetworks for processing diverse neural architectures. In International Conference on Learning Representations, 2024

    2024

  63. [71]

    Lin, Julius Berner, Valentin Duruisseaux, David Pitt, Daniel Leibovici, Jean Kossaifi, Kamyar Azizzadenesheli, and Anima Anandkumar

    Ryan Y. Lin, Julius Berner, Valentin Duruisseaux, David Pitt, Daniel Leibovici, Jean Kossaifi, Kamyar Azizzadenesheli, and Anima Anandkumar. Enabling automatic differentiation with mollified graph neural operators. arXiv preprint arXiv:2504.08277, 2025

    2025

  64. [72]

    Beyond regular grids: Fourier -based neural operators on arbitrary domains

    Levi Lingsch, Mike Y Michelis, Emmanuel de Bezenac, Sirani M Perera, Robert K Katzschmann, and Siddhartha Mishra. Beyond regular grids: Fourier -based neural operators on arbitrary domains. arXiv preprint arXiv:2305.19663, 2023

    2023

  65. [73]

    Tipping point forecasting in non-stationary dynamics on function spaces

    Miguel Liu-Schiaffini, Clare E Singer, Nikola Kovachki, Tapio Schneider, Kamyar Azizzadenesheli, and Anima Anandkumar. Tipping point forecasting in non-stationary dynamics on function spaces. arXiv preprint arXiv:2308.08794, 2023

    2023 arXiv

  66. [74]

    Neural operators with localized integral and differential kernels

    Miguel Liu-Schiaffini, Julius Berner, Boris Bonev, Thorsten Kurth, Kamyar Azizzadenesheli, and Anima Anandkumar. Neural operators with localized integral and differential kernels. arXiv preprint arXiv:2402.16845, 2024

    2024 arXiv

  67. [75]

    DeepONet : Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators

    Lu Lu, Pengzhan Jin, and George Em Karniadakis. DeepONet : Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193, 2019

    1910 arXiv

  68. [76]

    Effective approaches to attention-based neural machine translation

    Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025, 2015

    2015 arXiv

  69. [77]

    Huge ensembles part i: Design of ensemble weather forecasts using spherical Fourier neural operators

    Ankur Mahesh, William Collins, Boris Bonev, Noah Brenowitz, Yair Cohen, Joshua Elms, Peter Harrington, Karthik Kashinath, Thorsten Kurth, Joshua North, et al. Huge ensembles part i: Design of ensemble weather forecasts using spherical Fourier neural operators. arXiv preprint a...

    2024 arXiv

  70. [78]

    Fourier continuation for exact derivative computation in physics-informed neural operators

    Haydn Maust, Zongyi Li, Yixuan Wang, Daniel Leibovici, Oscar Bruno, Thomas Hou, and Anima Anandkumar. Fourier continuation for exact derivative computation in physics-informed neural operators. arXiv preprint arXiv:2211.15960, 2022

    2022

  71. [79]

    NeRF : Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF : Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, pp.\ 405--421, 2020

    2020

  72. [80]

    Position: Optimization in SciML should employ the function space geometry

    Johannes M \"u ller and Marius Zeinhofer. Position: Optimization in SciML should employ the function space geometry. In Forty-first International Conference on Machine Learning, 2024

    2024

  73. [81]

    Instant neural graphics primitives with a multiresolution hash encoding

    Thomas M \"u ller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics, 41 0 (4): 0 1--15, 2022

    2022

  74. [82]

    The well: a large-scale collection of diverse physics simulations for machine learning

    Ruben Ohana, Michael McCabe, Lucas Meyer, Rudy Morel, Fruzsina Agocs, Miguel Beneitez, Marsha Berger, Blakesly Burkhart, Stuart Dalziel, Drummond Fielding, et al. The well: a large-scale collection of diverse physics simulations for machine learning. Advances in Neural Informa...

    2024

  75. [83]

    Pamela, N

    S.J.P. Pamela, N. Carey, J. Brandstetter, R. Akers, L. Zanisi, J. Buchanan, V. Gopakumar, M. Hoelzl, G. Huijsmans, K. Pentland, T. James, and G. Antonucci. Neural-parareal: Self-improving acceleration of fusion MHD simulations using time-parallelisation and neural operators. C...

    2025

  76. [84]

    Robust confinement state classification with uncertainty quantification through ensembled data-driven methods

    Yoeri Poels, Cristina Venturini, Alessandro Pau, Olivier Sauter, Vlado Menkovski, the TCV team, and the WPTE team. Robust confinement state classification with uncertainty quantification through ensembled data-driven methods. arXiv preprint arXiv:2502.17397, 2025

    2025 arXiv

  77. [85]

    Generative adversarial neural operators

    Md Ashiqur Rahman, Manuel A Florez, Anima Anandkumar, Zachary E Ross, and Kamyar Azizzadenesheli. Generative adversarial neural operators. arXiv preprint arXiv:2205.03017, 2022 a

    2022 arXiv

  78. [86]

    U-NO : U-shaped neural operators

    Md Ashiqur Rahman, Zachary E Ross, and Kamyar Azizzadenesheli. U-NO : U-shaped neural operators. arXiv preprint arXiv:2204.11127, 2022 b

    2022 arXiv

  79. [87]

    Pretraining codomain attention neural operators for solving multiphysics PDEs

    Md Ashiqur Rahman, Robert Joseph George, Mogab Elleithy, Daniel Leibovici, Zongyi Li, Boris Bonev, Colin White, Julius Berner, Raymond A Yeh, Jean Kossaifi, et al. Pretraining codomain attention neural operators for solving multiphysics PDEs . arXiv preprint arXiv:2403.12553, 2024

    2024 arXiv

  80. [88]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378: 0 686--707, 2019

    2019

  81. [89]

    Convolutional neural operators for robust and accurate learning of PDEs

    Bogdan Raonic, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de B \'e zenac. Convolutional neural operators for robust and accurate learning of PDEs . In Advances in Neural Information Processing Systems, vol...

    2024

  82. [90]

    Anoop Krishnan

    Meer Mehran Rashid, Tanu Pittie, Souvik Chakraborty, and N.M. Anoop Krishnan. Learning the stress-strain fields in digital composites using Fourier neural operator. iScience, 25 0 (11): 0 105452, 2022

    2022

  83. [91]

    An introduction to partial differential equations, volume 13

    Michael Renardy and Robert C Rogers. An introduction to partial differential equations, volume 13. Springer Science & Business Media, 2006

    2006

  84. [92]

    U-Net : Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net : Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention, pp.\ 234--241. Springer, 2015

    2015

  85. [93]

    NOMAD : Nonlinear manifold decoders for operator learning

    Jacob Seidman, Georgios Kissas, Paris Perdikaris, and George J Pappas. NOMAD : Nonlinear manifold decoders for operator learning. Advances in Neural Information Processing Systems, 35: 0 5601--5613, 2022

    2022

  86. [94]

    Variational autoencoding neural operators

    Jacob H Seidman, Georgios Kissas, George J Pappas, and Paris Perdikaris. Variational autoencoding neural operators. arXiv preprint arXiv:2302.10351, 2023

    2023 arXiv

  87. [95]

    Koupa \

    Louis Serrano, Lise Le Boudec, Armand Kassa \" Koupa \" , Thomas X Wang, Yuan Yin, Jean-No \"e l Vittaut, and Patrick Gallinari. Operator learning with neural fields: Tackling PDEs on general geometries. arXiv preprint arXiv:2306.07266, 2023

    2023 arXiv

  88. [96]

    Parallel physics-informed neural networks via domain decomposition

    Khemraj Shukla, Ameya D Jagtap, and George Em Karniadakis. Parallel physics-informed neural networks via domain decomposition. Journal of Computational Physics, 447: 0 110683, 2021

    2021

  89. [97]

    DGM : A deep learning algorithm for solving partial differential equations

    Justin Sirignano and Konstantinos Spiliopoulos. DGM : A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375: 0 1339--1364, 2018

    2018

  90. [98]

    Implicit neural representations with periodic activation functions

    Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. Advances in neural information processing systems, 33: 0 7462--7473, 2020

    2020

  91. [99]

    RoFormer : Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. RoFormer : Enhanced transformer with rotary position embedding. Neurocomputing, 568: 0 127063, 2024

    2024

  92. [100]

    Operator learning: A statistical perspective

    Unique Subedi and Ambuj Tewari. Operator learning: A statistical perspective. arXiv preprint arXiv:2504.03503, 2025

    2025 arXiv

  93. [101]

    Accelerating time-reversal imaging with neural operators for real-time earthquake locations

    Hongyu Sun, Yan Yang, Kamyar Azizzadenesheli, Robert W Clayton, and Zachary E Ross. Accelerating time-reversal imaging with neural operators for real-time earthquake locations. arXiv preprint arXiv:2210.06636, 2022

    2022 arXiv

  94. [102]

    Wavelet neural operator: a neural operator for parametric partial differential equations

    Tapas Tripura and Souvik Chakraborty. Wavelet neural operator: a neural operator for parametric partial differential equations. arXiv preprint arXiv:2205.02191, 2022

    2022 arXiv

  95. [103]

    Instance normalization: The missing ingredient for fast stylization

    D Ulyanov. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016

    2016 arXiv

  96. [104]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  97. [105]

    Graph attention networks

    Petar Veli c kovi \'c , Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  98. [106]

    Ultrasound lung aeration map via physics-aware neural operators

    Jiayun Wang, Oleksii Ostras, Masashi Sode, Bahareh Tolooshams, Zongyi Li, Kamyar Azizzadenesheli, Gianmarco Pinton, and Anima Anandkumar. Ultrasound lung aeration map via physics-aware neural operators. arXiv preprint arXiv:2501.01157, 2025

    2025 arXiv

  99. [107]

    Learning the solution operator of parametric partial differential equations with physics-informed DeepONets

    Sifan Wang, Hanwen Wang, and Paris Perdikaris. Learning the solution operator of parametric partial differential equations with physics-informed DeepONets . Science advances, 7 0 (40), 2021

    2021

  100. [108]

    An expert's guide to training physics-informed neural networks

    Sifan Wang, Shyam Sankaran, Hanwen Wang, and Paris Perdikaris. An expert's guide to training physics-informed neural networks. arXiv preprint arXiv:2308.08468, 2023

    2023 arXiv

  101. [109]

    Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson. U-FNO: an enhanced Fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163: 0 104180, 2022

    2022

  102. [110]

    Gege Wen, Zongyi Li, Qirui Long, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson. Real-time high-resolution CO2 geological storage prediction using nested fourier neural operators. Energy Environ. Sci., 16: 0 1732--1741, 2023

    2023

  103. [111]

    FNOSeg3D : Resolution-robust 3D image segmentation with Fourier neural operator

    Ken CL Wong, Hongzhi Wang, and Tanveer Syeda-Mahmood. FNOSeg3D : Resolution-robust 3D image segmentation with Fourier neural operator. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pp.\ 1--5. IEEE, 2023

    2023

  104. [112]

    Equivariant graph neural operator for modeling 3d dynamics

    Minkai Xu, Jiaqi Han, Aaron Lou, Jean Kossaifi, Arvind Ramanathan, Kamyar Azizzadenesheli, Jure Leskovec, Stefano Ermon, and Anima Anandkumar. Equivariant graph neural operator for modeling 3d dynamics. arXiv preprint arXiv:2401.11037, 2024

    2024 arXiv

  105. [113]

    Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling

    Qidong Yang, Paula Harder, Venkatesh Ramesh, Alex Hernandez-Garcia, Daniela Szwarcman, Prasanna Sattigeri, Campbell D Watson, and David Rolnick. Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling . In ICLR 2023 Workshop on Tackling Climate Change with M...

    2023

  106. [114]

    Seismic wave propagation and inversion with neural operators

    Yan Yang, Angela F Gao, Jorge C Castellanos, Zachary E Ross, Kamyar Azizzadenesheli, and Robert W Clayton. Seismic wave propagation and inversion with neural operators. The Seismic Record, 1 0 (3): 0 126--134, 2021

    2021

  107. [115]

    Guided diffusion sampling on function spaces with applications to PDEs

    Jiachen Yao, Abbas Mammadov, Julius Berner, Gavin Kerrigan, Jong Chul Ye, Kamyar Azizzadenesheli, and Anima Anandkumar. Guided diffusion sampling on function spaces with applications to PDEs . arXiv preprint arXiv:2505.17004, 2025

    2025

  108. [116]

    Learning deep implicit Fourier neural operators ( IFNOs ) with applications to heterogeneous material modeling

    Huaiqian You, Quinn Zhang, Colton J Ross, Chung-Hao Lee, and Yue Yu. Learning deep implicit Fourier neural operators ( IFNOs ) with applications to heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering, 398: 0 115296, 2022

    2022

  109. [117]

    Neural born series operator for biomedical ultrasound computed tomography

    Zhijun Zeng, Yihang Zheng, Youjia Zheng, Yubing Li, Zuoqiang Shi, and He Sun. Neural born series operator for biomedical ultrasound computed tomography. arXiv preprint arXiv:2312.15575, 2023

    2023 arXiv

  110. [118]

    Fixup initialization: Residual learning without normalization

    Hongyi Zhang, Yann N Dauphin, and Tengyu Ma. Fixup initialization: Residual learning without normalization. arXiv preprint arXiv:1901.09321, 2019

    1901 arXiv

  111. [119]

    ZerO initialization: Initializing neural networks with only zeros and ones

    Jiawei Zhao, Florian Sch \"a fer, and Anima Anandkumar. ZerO initialization: Initializing neural networks with only zeros and ones. arXiv preprint arXiv:2110.12661, 2021

    2021 arXiv

  112. [120]

    Fast sampling of diffusion models via operator learning

    Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, and Anima Anandkumar. Fast sampling of diffusion models via operator learning. In International Conference on Machine Learning, pp.\ 42390--42402. PMLR, 2023

    2023

  113. [121]

    Permutation equivariant neural functionals

    Allan Zhou, Kaien Yang, Kaylee Burns, Adriano Cardace, Yiding Jiang, Samuel Sokota, J Zico Kolter, and Chelsea Finn. Permutation equivariant neural functionals. Advances in neural information processing systems, 36: 0 24966--24992, 2023

    2023

  114. [122]

    Brady, Paul W

    Tingtao Zhou, Xuan Wan, Daniel Zhengyu Huang, Zongyi Li, Zhiwei Peng, Anima Anandkumar, John F. Brady, Paul W. Sternberg, and Chiara Daraio. AI -aided geometric design of anti-infection catheters. Science Advances, 10 0 (1), 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.