Pith. sign in

REVIEW 4 major objections 6 minor 57 references

GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A geometric correction to the state-space scan lets a Mamba-based neural operator cut PDE solution error by up to 58.9 percent.

desk verdict GeoMaNO has a clever architectural idea and strong-looking results, but the headline SOTA numbers appear to be best-of-22 ablations selected on the test set, so the gains are probably inflated. read the letter →

arxiv 2505.12020 v1 pith:5ZE3RFFR submitted 2025-05-17 cs.LG cs.AI

Xi Han , Jingwei Zhang , Dimitris Samaras , Fei Hou , Hong Qin This is my paper · ORCID
classification cs.LGcs.AI
keywords neuraloperatorPDEsurrogatestate-spacemodelMambageometriccorrectioncross-scanDarcyflowNavier-Stokes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to build a neural operator—a learned mapping from PDE parameters to solutions—that is both accurate and efficient on regular grids. It claims that Mamba-style state-space models, which process sequences in linear time, can outperform the Transformer-based operators that currently dominate the field, provided the geometry of the grid is respected. To that end it introduces GeoMaNO, whose GeoMamba-SSM module adds a learnable or fixed geometric correction that suppresses duplicate hidden states produced by the four-way cross-scan, and, for 2D problems, replaces the one-dimensional scan with a two-dimensional state-space recurrence. On the standard Darcy flow and Navier-Stokes benchmarks the paper reports relative L2 errors of 0.0036 and 0.0370, beating the previous state of the art by 36.8% and 58.9%, with lower inference time and GPU memory. If these numbers hold, fast and cheap surrogate PDE solvers become significantly more accurate on the grid-based problems that dominate engineering practice.

What carries the argument

The load-bearing object is the GeoMamba-SSM module, a selective state-space model whose output subtracts an input-adaptive correction term, $-R_sB_sx$, from the hidden-state readout in each scan direction. The subtraction is the mechanism that removes the fourfold duplication of $Bx$ created when a grid is scanned in four directions and then merged. For two-dimensional PDEs the module uses a two-dimensional recurrence that keeps vertical neighbors adjacent in the hidden state, making the decay of each contribution a Manhattan distance instead of a flattened row-major distance. The ablation study attributes the accuracy gain to these two geometric choices, with the best fixed correction patterns chosen per benchmark.

What would settle it

Re-run the comparison with a held-out validation set: select the geometric-correction configuration by validation error, then report test error. If the chosen GeoMaNO does not beat the previous best model by roughly 58.9% on Navier-Stokes and 36.8% on Darcy flow, the reported state-of-the-art claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the geometric form of the state-space scan, not just the backbone choice, is what unlocks accuracy on regular-grid PDEs. GeoMamba-SSM writes the output as $y^{(d)}[t]=\sum_s(C_s^{(d)}[t]h_s^{(d)}[t]-R_s^{(d)}[t]B_s^{(d)}[t]x^{(d)}[t])$, so the duplicated contribution $Bx$ that appears four times under the four-way cross-scan is dampened by a correction coefficient $R_s$ that can be fixed or learned. For two-dimensional problems it further replaces the flattened 1D recursion with the 2D recurrence $g_s[i,j]=A_s[i,j]g_s[i,j-1]+B_s[i,j]x[i,j]$ and $h_s[i,j]=A_s[i,j]h_s[i-1,j]+g_s[i,j]$, which makes the hidden-state decay depend on Manhattan distance rather than row-major linear distance. With these two mechanisms GeoMaNO reports relative L2 errors of 0.0370 on Navier-Stokes and 0.0036 on Darcy flow, improvements of 58.9% and 36.8% over the prior best model, alongside reduced training and inference time and memory.

Load-bearing premise

The reported gains depend on choosing the best geometric-correction configuration from the ablation table while using the same test set for the final numbers, and the paper describes no separate validation split for that selection.

Editorial extensions

If this is right

  • On the two benchmarks tested, a state-space backbone with geometric correction outperforms the previous best Transformer-based operator in both accuracy and efficiency, suggesting linear-complexity backbones can replace quadratic attention for grid PDEs.
  • The large error reductions imply that the four-way cross-scan's hidden-state duplication was a real source of error, not a cosmetic artifact, and other multi-scan Mamba models may carry the same overhead.
  • The 2D recurrence preserves spatial adjacency for 2D PDEs, but the paper reports it fails on the 3D Navier-Stokes domain, so dimension-matched scan geometry is needed rather than one universal scan.
  • The hardware-aware tiling described in the appendix applies the correction at the thread level, so the geometric fix does not add memory traffic; GPU memory drops substantially on Darcy flow.
  • If the reported gains are reproducible, PDE surrogate modeling on regular grids can move from quadratic-complexity attention to linear-complexity scanning without sacrificing accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper shows GeoMamba-SSM also improves ImageNet-100 accuracy over vanilla Mamba-SSM inside a vision backbone, the duplicate-state correction is likely transferable to non-PDE multi-scan state-space models; this is an inference beyond the paper's PDE claim.
  • The correction coefficient is only a per-scan constant or a scalar learned coefficient; an input-dependent $R_s$ could dampen duplicates where they actually occur, a testable variant the paper does not explore.
  • Extending the 2D recurrence to three dimensions, which the paper names as future work, is the direct test of whether the same geometric principle carries to volumetric and time-dependent PDEs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes GeoMaNO, a neural operator built on a Mamba-style state-space model augmented with a geometric correction designed to dampen duplicate hidden states caused by four-way cross-scans. The architecture is applied to the standard THUML PDE benchmarks: Darcy flow (2D elliptic) and Navier-Stokes (3D parabolic/hyperbolic). The authors claim state-of-the-art accuracy, improving on the previous best Transolver by 58.9% on Navier-Stokes and 36.8% on Darcy flow, and also improved efficiency (training time, inference time, memory). The paper includes architectural ablations, scalability studies, error bars for the main results, and an additional ImageNet-100 experiment demonstrating generalization of the geometric correction module. The mathematical preliminaries and the kernel-integral/SSM equivalence are credited to prior work, while the geometric correction and 2D SSM representation are presented as the main novelties.

Significance. If the empirical claims hold, GeoMaNO would be a noteworthy contribution: it combines linear-complexity selective SSMs with a geometrically motivated correction, addressing a recognized limitation of Transformer-based neural operators on regular grids. The paper ships error bars for its main numbers and ablation tables over a reasonably sized configuration space, and the ImageNet-100 transfer experiment is a useful sanity check for the geometric-correction mechanism. However, the load-bearing evidence for the headline improvements is currently jeopardized by an experimental protocol that appears to select the best of many ablation configurations on the same test set, by an internal contradiction in the efficiency tables, and by an inconsistency between the mathematical formulation and the provided implementation pseudo-code. These issues must be resolved before the claimed state-of-the-art results can be accepted.

major comments (4)
  1. [Sec. A.1, Table 1, Table 3] The manuscript never describes a validation split; Sec. A.1 states only that 1000 samples are used for training and 200 for testing. The headline numbers in Table 1 (0.0370 for Navier-Stokes, 0.0036 for Darcy) exactly coincide with the best rows of the ablation table (Table 3): 'Vanilla, Yes, Learnable' for N-S and '2D, None, 0011' for Darcy. If, as the text suggests, all ablation configurations were evaluated on the same 200 test samples, then the reported 'state-of-the-art' gains are the result of test-set selection over 22 configurations, which systematically biases the reported errors downward relative to a pre-specified model. Please rerun the experiments with a proper validation split, select the configuration on validation, and report only the test-set performance of the selected model, or otherwise document that model selection is performed without access to the test labels.
  2. [Table 2(a) and Table 7(b)] The memory consumption for Transolver on the Navier-Stokes task is reported as 91.85 MB in Table 2(a) but as 5.21 MB in Table 7(b) of the appendix. The two tables cannot both be correct. Since efficiency is one of the paper's core claims, this internal contradiction makes the memory comparison unreliable: if the 5.21 MB figure is the true one, then GeoMaNO's 70.44 MB is substantially larger and the claimed 23.3% memory saving is false; if the 91.85 MB figure is correct, the appendix needs a correction. The authors must identify the error and provide consistent numbers for all efficiency metrics.
  3. [Eq. (9), Eq. (10), and Algorithm 2 line 24] The geometric correction in the main equations subtracts R_s B_s x from the output (Eqs. 9 and 10), while Algorithm 2 computes y += Cs*(hs - Rs*x), omitting the multiplication by B_s. Since the correction term is the central theoretical novelty, this discrepancy between the formal definition and the implementation pseudo-code is load-bearing. The authors need to clarify which form is actually implemented and used in the experiments, and align the mathematical formulation with the algorithmic description.
  4. [Sec. 5.2 and Sec. A.4] The accuracy comparison in Table 1 reuses baseline numbers from their original papers (e.g., the Transolver error 0.0900 is taken from the Transolver paper) rather than rerunning them under the same training and evaluation protocol as GeoMaNO. This is a fairness concern, especially when juxtaposed with the same-environment efficiency comparison in Table 2, which does run Transolver locally. Please rerun at least the strongest baselines (Transolver, ONO, FactFormer, LSM) under the same environment, data split, and evaluation metric, and report their errors alongside GeoMaNO so that the claimed 58.9% and 36.8% gains are statistically matched.
minor comments (6)
  1. [Sec. 1, last paragraph] There is a typo: 'empolyed' should be 'employed'.
  2. [Appendix B, Eq. (16)] Eq. (16) is missing the factor v(y) in the integrand; the correct kernel integral should be \int C e^{Ax} B e^{-Ay} v(y) dy, which is what Eq. (17) later uses. Please fix the typo.
  3. [Appendix B, Eq. (18)] In the derivation of h'(x), the lower integration limit changes from -infinity to 0 without explanation; the limit should remain -infinity for the conclusion h'(x) = A h(x) + B v(x) to hold.
  4. [Sec. 5.1] The main-text model configuration does not explicitly state which positional-encoding and geometric-correction settings are used for the final GeoMaNO model on each benchmark; these are only inferable from Table 3. Please state the final configuration in the main text for reproducibility.
  5. [Algorithm 2] The tensor shapes in the 'Require' block are inconsistent with the usage in the loop: cAt is declared as (N,ED) but is used with spatial subscripts [kh,kw], and cBt is declared as (B,H,W) without the dstate dimension. Please clarify the exact shapes and broadcasting semantics.
  6. [Table 4] The Darcy loss is reported as LrL2 + 0.1Lg, but the spatial gradient regularizer Lg is not defined in the main text; please define it where the training configuration is introduced.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline SOTA figures are the best rows of the paper's own ablation table selected on the same test set, so the reported gains partially reduce to test-set selection.

  1. fitted input called prediction [Table 1 vs. Table 3; Sec. 5.1 (Implementation Details) and Sec. A.1 (Benchmarks)]
    "GeoMaNO (Ours) 0.0370 0.0036 [Table 1]; Vanilla ✓ Learnable 0.03701 [Table 3(b)]; 2D None 0011 0.00362 [Table 3(a)]; 'Regarding the GeoMaNO layers, we employ the vanilla GeoMamba-SSM for Navier-Stokes and 2DGeoMamba-SSM for Darcy flow.'"

    The Table 1 headline numbers are exactly the best rows of Table 3 (Navier-Stokes: Vanilla+Learnable, 0.03701 vs. reported 0.0370; Darcy: 2D+0011, 0.00362 vs. reported 0.0036). Section 5.1 states the main model uses the vanilla SSM for Navier-Stokes and the 2D SSM for Darcy, i.e., the configurations that win the ablation search. Section A.1 specifies only 1000 training and 200 testing samples and never mentions a validation split, so the Table 3 grid appears to be evaluated on the same test set used for Table 1. The reported SOTA is therefore the minimum over the 22 ablation configurations, a test-set-selected quantity presented as an independent measured prediction.

full rationale

The mathematical derivation chain is largely self-contained: the SSM/kernel-integral equivalence in Sec. B is explicitly credited to Zheng et al. [6], and the 2D SSM recursions in Eq. 10 and Sec. C are derived in the text from the published 2DMamba formulation [10]. Neither of these citations smuggles in the paper's PDE benchmark claims; the novelty (geometric correction R_s and application to neural operators) is stated and ablated directly. No uniqueness theorem or self-citation is used to forbid alternatives. The only substantive circularity concern is empirical: the paper's headline SOTA errors coincide exactly with the best rows of its own ablation table, while no validation split is described, so the best-of-22 configuration is selected on the test set and then reported as the performance of 'GeoMaNO'. That makes the headline accuracy gain partially a constructed, test-set-minimum quantity rather than an independent prediction of a fixed configuration. Under the fitted-input-called-prediction pattern, this warrants a partial circularity score; it is not a fully circular derivation, because the method itself and its theoretical framing are not reduced to the empirical result.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim relies on standard operator-learning assumptions, a heuristic correction mechanism, and benchmark data from prior work. The main free parameters are the Rs configuration and a few training hyperparameters; no new physical entities are introduced.

free parameters (3)
  • Rs geometric correction configuration (fixed or learnable) = Darcy: fixed 0011; Navier-Stokes: learnable
    The paper ablates fixed configurations 0001, 0011, 0111 and a learnable version, and the best performing variant on each benchmark is used for the main results (Table 1, Table 3). This is a hyperparameter selected after seeing benchmark performance.
  • Gradient regularization weight on Darcy flow = 0.1
    The Darcy loss is L_rL2 + 0.1 L_g (Appendix Table 4), chosen following ONO rather than derived; it affects the reported Darcy error.
  • Model depth, embedding dimension, SSM dstate count = Depth 8; ED 256 (NS) and 64 (Darcy); N=16
    Standard hyperparameters taken from Transolver; they affect results but are not the paper's contribution.
assumptions (4)
  • standard math Mamba-SSM recurrences are equivalent to neural operator kernel integral operators (credited to Zheng et al. [6]).
    Repeated in Sec. B; treated as a proven external result, not re-derived.
  • ad hoc to paper The four-way cross-scan merge duplicates hidden information additively and the geometric correction removes the duplication as intended.
    Sec. 4.3 asserts this for a simplified constant-A,B model; the actual CrossMerge operation and the effect of R_s are not analyzed in general.
  • domain assumption The standard benchmarks (Darcy flow, Navier-Stokes) and the ground-truth solvers in the THUML repository are accurate and representative.
    Sec. 5.1 and A.1 rely on the benchmark datasets without independent verification.
  • domain assumption Baseline results quoted from prior papers (e.g., Transolver) are directly comparable to GeoMaNO's runs.
    Sec. A.4 states Transolver results in Table 1 are taken from the original paper; comparability of training protocol, resolution, and hardware is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations." pith.science (2026). https://pith.science/paper/5ZE3RFFR

@misc{pith2026250512020,
  author       = {Pith},
  title        = {Pith review of: GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5ZE3RFFR}},
  note         = {Machine review of arXiv:2505.12020}
}
read the original abstract

The neural operator (NO) framework has emerged as a powerful tool for solving partial differential equations (PDEs). Recent NOs are dominated by the Transformer architecture, which offers NOs the capability to capture long-range dependencies in PDE dynamics. However, existing Transformer-based NOs suffer from quadratic complexity, lack geometric rigor, and thus suffer from sub-optimal performance on regular grids. As a remedy, we propose the Geometric Mamba Neural Operator (GeoMaNO) framework, which empowers NOs with Mamba's modeling capability, linear complexity, plus geometric rigor. We evaluate GeoMaNO's performance on multiple standard and popularly employed PDE benchmarks, spanning from Darcy flow problems to Navier-Stokes problems. GeoMaNO improves existing baselines in solution operator approximation by as much as 58.9%.

Figures

Figures reproduced from arXiv: 2505.12020 by the authors.

Figure 1
Figure 1. Overview of the novel GeoMaNO architecture. (1) The input function a(x) 2 is lifted and patchified to a higher-dimensional latent representation by the geometry-aware encoder layer E; (2) T stacked GeoMaNO layers M1 · · · MT (middle) performs the kernel integral via GeoMamba kernels (bottom); (3) Each GeoMamba kernel performs geometry-aware Mamba scans, combined with skip-connections and non-linear activations; and … view at source ↗
Figure 2
Figure 2. The four-way cross-scan pattern. Left: In the data preparation stage, the input grid is traversed in four orders, and the traversal results are stacked at an additional order dimension. Right: For each slice along the order dimension, we process it with our GeoMamba-SSM module (shown in blue color). As a special case, for 2D PDEs, in GeoMamba-SSM, we employ a 2D SSM representation with geometric rigor (shown in gree… view at source ↗
Figure 3
Figure 3. Illustration of our GeoMamba-SSM variants. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: GeoMaNO’s scalability. We ablate model depth and embedding dimensions, and report the [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Illustration of 1D v.s. 2D scanning paths and spatial discrepancy. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Illustration of the typical GPU Memory hierarchy. The typical GPU memory hierarchy. [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Illustration of Mamba’s 1D scanning operator. Here, [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Comparasion of ERFs on ImageNet100 classification tasks. Pixels with higher intensity [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 53 canonical work pages

  1. [1]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” inAdvances in Neural Information Processing Systems, 2017

    work page 2017

  2. [2]

    Gnot: a general neural operator transformer for operator learning,

    Z. Hao, Z. Wang, H. Su, C. Ying, Y . Dong, S. Liu, Z. Cheng, J. Song, and J. Zhu, “Gnot: a general neural operator transformer for operator learning,” in Proceedings of the 40th International Conference on Machine Learning, 2023

    work page 2023

  3. [3]

    Transolver: a fast transformer solver for pdes on general geometries,

    H. Wu, H. Luo, H. Wang, J. Wang, and M. Long, “Transolver: a fast transformer solver for pdes on general geometries,” in Proceedings of the 41st International Conference on Machine Learning, 2024

    work page 2024

  4. [4]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” in Proceedings of the First Conference on Language Modeling, 2024

    work page 2024

  5. [5]

    Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality,

    T. Dao and A. Gu, “Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality,” inProceeding of the 41st International Conference on Machine Learning, 2024

    work page 2024

  6. [6]

    Alias-free mamba neural operator,

    J. Zheng, W. Li, N. Xu, J. Zhu, X. Lin, and X. Zhang, “Alias-free mamba neural operator,” in Advances in Neural Information Processing Systems, vol. 37, 2024

    work page 2024

  7. [7]

    State-space models are accurate and efficient neural operators for dynamical systems,

    Z. Hu, N. A. Daryakenari, Q. Shen, K. Kawaguchi, and G. E. Karniadakis, “State-space models are accurate and efficient neural operators for dynamical systems,” in Computing Research Repository, vol. abs/2409.03231, 2025

    arXiv 2025

  8. [8]

    Vmamba: Visual state space model,

    Y . Liu, Y . Tian, Y . Zhao, H. Yu, L. Xie, Y . Wang, Q. Ye, and Y . Liu, “Vmamba: Visual state space model,” in Advances in Neural Information Processing Systems, 2024

    work page 2024

Show all 57 references
  1. [9]

    Vision mamba: Efficient vi- sual representation learning with bidirectional state space model,

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: Efficient vi- sual representation learning with bidirectional state space model,” in Proceedings of the 41st International Conference on Machine Learning, 2024

    2024

  2. [10]

    2DMamba: Efficient state space model for image representation with applications on giga-pixel whole slide image classification,

    J. Zhang, A. T. Nguyen, X. Han, V . Q.-H. Trinh, H. Qin, D. Samaras, and M. S. Hosseini, “2DMamba: Efficient state space model for image representation with applications on giga-pixel whole slide image classification,” in Proceedings of 2025 IEEE/CVF Conference on Computer Vis...

    2025

  3. [11]

    PDE solving standard benchmark

    THUML, “PDE solving standard benchmark.” https://github.com/thuml/Transolver/ tree/main/PDE-Solving-StandardBenchmark , 2025

    2025

  4. [12]

    Learning nonlinear operators via deeponet based on the universal approximation theorem of operators,

    L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis, “Learning nonlinear operators via deeponet based on the universal approximation theorem of operators,” in Nature Machine Intelligence, pp. 218–229, 2021

    2021

  5. [13]

    Fourier Neural Operator for Parametric Partial Differential Equations,

    Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandku- mar, “Fourier Neural Operator for Parametric Partial Differential Equations,” inProceedings of the 9th International Conference on Learning Representations, 2021

    2021

  6. [14]

    U-fno – an enhanced fourier neural operator-based deep-learning model for multiphase flow,

    G. Wen, Z. Li, K. Azizzadenesheli, A. Anandkumar, and S. M. Benson, “U-fno – an enhanced fourier neural operator-based deep-learning model for multiphase flow,” Advances in Water Resources, vol. 163, pp. 104–180, 2022

    2022

  7. [15]

    Adaptive fourier neu- ral operators: Efficient token mixers for transformers,

    J. Guibas, M. Mardani, Z. Li, A. Tao, A. Anandkumar, and B. Catanzaro, “Adaptive fourier neu- ral operators: Efficient token mixers for transformers,” in Proceedings of the 10th International Conference on Learning Representations, 2022

    2022

  8. [16]

    Incremental spatial and spectral learning of neural operators for solving large-scale PDEs,

    R. J. George, J. Zhao, J. Kossaifi, Z. Li, and A. Anandkumar, “Incremental spatial and spectral learning of neural operators for solving large-scale PDEs,” Transactions on Machine Learning Research, 2024. 11

    2024

  9. [17]

    Four- castnet: A global data-driven high-resolution weather model using adaptive fourier neural operators,

    J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli, P. Hassanzadeh, K. Kashinath, and A. Anandkumar, “Four- castnet: A global data-driven high-resolution weather model using adaptive fourier neural oper...

    2022

  10. [18]

    Factorized fourier neural operators,

    A. Tran, A. P. Mathews, L. Xie, and C. S. Ong, “Factorized fourier neural operators,” in Proceedings of the 11th International Conference on Learning Representations, 2023

    2023

  11. [19]

    Neural operator: Graph kernel network for partial differential equations,

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Neural operator: Graph kernel network for partial differential equations,” in Proceedings of ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020

    2020

  12. [20]

    Multipole graph neural operator for parametric partial differential equations,

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Multipole graph neural operator for parametric partial differential equations,” in Advances in Neural Information Processing Systems, 2020

    2020

  13. [21]

    Parallel prefix computation,

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Parallel prefix computation,” Journal of Machine Learning Research, vol. 89, pp. 1–97, 2023

    2023

  14. [22]

    Improved operator learning by orthogonal attention,

    Z. Xiao, Z. Hao, B. Lin, Z. Deng, and H. Su, “Improved operator learning by orthogonal attention,” in Proceedings of the 41st International Conference on Machine Learning, vol. 235, 2024

    2024

  15. [24]

    Towards general neural surrogate solvers with specialized neural accelerators,

    C. Mao, R. Lupoiu, T. Dai, M. Chen, and J. A. Fan, “Towards general neural surrogate solvers with specialized neural accelerators,” in Proceedings of the 41st International Conference on Machine Learning, 2024

    2024

  16. [25]

    HT-net: Hierarchical transformer based operator learning model for multiscale PDEs,

    X. Liu, B. Xu, and L. Zhang, “HT-net: Hierarchical transformer based operator learning model for multiscale PDEs,” 2023

    2023

  17. [26]

    Reformer: The efficient transformer,

    N. Kitaev, L. Kaiser, and A. Levskaya, “Reformer: The efficient transformer,” inProceedings of the 8th International Conference on Learning Representations, 2020

    2020

  18. [27]

    Rethinking attention with performers,

    K. M. Choromanski, V . Likhosherstov, D. Dohan, X. Song, A. Gane, T. Sarlos, P. Hawkins, J. Q. Davis, A. Mohiuddin, L. Kaiser, D. B. Belanger, L. J. Colwell, and A. Weller, “Rethinking attention with performers,” in Proceedings of the 9th International Conference on Learning R...

    2021

  19. [28]

    Choose a transformer: Fourier or galerkin,

    S. Cao, “Choose a transformer: Fourier or galerkin,” in Advances in Neural Information Processing Systems, 2021

    2021

  20. [29]

    Mamba neural operator: Who wins? transformers vs. state-space models for pdes,

    C.-W. Cheng, J. Huang, Y . Zhang, G. Yang, C.-B. Schönlieb, and A. I. Aviles-Rivero, “Mamba neural operator: Who wins? transformers vs. state-space models for pdes,” in Computing Research Repository, vol. abs/2410.02113, 2024

  21. [30]

    Learning to Optimize Multigrid PDE Solvers,

    D. Greenfeld, M. Galun, R. Kimmel, I. Yavneh, and R. Basri, “Learning to Optimize Multigrid PDE Solvers,” in Proceedings of the 36th International Conference on Machine Learning, 2019

    2019

  22. [31]

    UGrid: An efficient-and-rigorous neural multigrid solver for linear PDEs,

    X. Han, F. Hou, and H. Qin, “UGrid: An efficient-and-rigorous neural multigrid solver for linear PDEs,” in Proceedings of the 41st International Conference on Machine Learning, vol. 235, 2024

    2024

  23. [32]

    Combining recurrent, convolutional, and continuous-time models with linear state space layers,

    A. Gu, I. Johnson, K. Goel, K. Saab, T. Dao, A. Rudra, and C. Ré, “Combining recurrent, convolutional, and continuous-time models with linear state space layers,”Advances in Neural Information Processing Systems, 2021

    2021

  24. [33]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. Ré, “Efficiently modeling long sequences with structured state spaces,” in Computing Research Repository, 2021

    2021

  25. [34]

    Diagonal state spaces are as effective as structured state spaces,

    A. Gupta, A. Gu, and J. Berant, “Diagonal state spaces are as effective as structured state spaces,” Advances in Neural Information Processing Systems, vol. 35, pp. 22982–22994, 2022. 12

    2022

  26. [35]

    What makes convolutional models great on long sequence modeling?,

    Y . Li, T. Cai, Y . Zhang, D. Chen, and D. Dey, “What makes convolutional models great on long sequence modeling?,” in Computing Research Repository, vol. abs/2210.09298, 2022

    2022 arXiv

  27. [36]

    Plainmamba: Improving non-hierarchical mamba in visual recognition,

    C. Yang, Z. Chen, M. Espinosa, L. Ericsson, Z. Wang, J. Liu, and E. J. Crowley, “Plainmamba: Improving non-hierarchical mamba in visual recognition,” in Proceedings of the 35th British Machine Vision Conference, 2024

    2024

  28. [37]

    Quadmamba: Learning quadtree-based selective scan for visual state space model,

    F. Xie, W. Zhang, Z. Wang, and C. Ma, “Quadmamba: Learning quadtree-based selective scan for visual state space model,” Advances in Neural Information Processing Systems, 2024

    2024

  29. [38]

    Neural operator: learning maps between function spaces with applications to pdes,

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Neural operator: learning maps between function spaces with applications to pdes,”The Journal of Machine Learning Research, vol. 24, no. 1, 2023

    2023

  30. [39]

    Patches are all you need?,

    A. Trockman and J. Z. Kolter, “Patches are all you need?,” inProceedings of the 11th Interna- tional Conference on Learning Representations, 2023

    2023

  31. [40]

    Feed-forward neural networks,

    G. Bebis and M. Georgiopoulos, “Feed-forward neural networks,” IEEE Potentials, vol. 13, no. 4, pp. 27–31, 1994

    1994

  32. [41]

    Koopman operator dynamical models: Learning, analysis and control,

    P. Bevanda, S. Sosnowski, and S. Hirche, “Koopman operator dynamical models: Learning, analysis and control,” Annual Reviews in Control, vol. 52, pp. 197–212, 2021

    2021

  33. [42]

    Perceiver: General perception with iterative attention,

    A. Jaegle, F. Gimeno, A. Brock, A. Zisserman, O. Vinyals, and J. Carreira, “Perceiver: General perception with iterative attention,” in Proceedings of the 38th International Conference on Machine Learning, 2021

    2021

  34. [43]

    Layer normalization,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” in Proceedings of the 3rd International Conference for Learning Representations, 2016

    2016

  35. [44]

    Multiwavelet-based operator learning for differential equations,

    G. Gupta, X. Xiao, and P. Bogdan, “Multiwavelet-based operator learning for differential equations,” in Advances in Neural Information Processing Systems, 2021

    2021

  36. [45]

    Fourier neural operator with learned deforma- tions for pdes on general geometries,

    Z. Li, D. Z. Huang, B. Liu, and A. Anandkumar, “Fourier neural operator with learned deforma- tions for pdes on general geometries,” Journal of Machine Learning Research, vol. 24, no. 388, pp. 1–26, 2023

    2023

  37. [46]

    U-NO: U-shaped neural operators,

    M. A. Rahman, Z. E. Ross, and K. Azizzadenesheli, “U-NO: U-shaped neural operators,” Transactions on Machine Learning Research, 2023

    2023

  38. [47]

    Solving high-dimensional pdes with latent spectral models,

    H. Wu, T. Hu, H. Luo, J. Wang, and M. Long, “Solving high-dimensional pdes with latent spectral models,” in Proceedings of the 40th International Conference on Machine Learning, 2023

    2023

  39. [48]

    Transformer for partial differential equations’ operator learning,

    Z. Li, K. Meidani, and A. B. Farimani, “Transformer for partial differential equations’ operator learning,” Transactions on Machine Learning Research, 2023

    2023

  40. [49]

    Scalable transformer for PDE surrogate modeling,

    Z. Li, D. Shu, and A. B. Farimani, “Scalable transformer for PDE surrogate modeling,” in Advances in Neural Information Processing Systems, 2023

    2023

  41. [50]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009

    2009

  42. [51]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inProceedings of the 7th International Conference on Learning Representations, 2019

    2019

  43. [52]

    Super-convergence: very fast training of neural networks using large learning rates,

    L. N. Smith and N. Topin, “Super-convergence: very fast training of neural networks using large learning rates,” in Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, vol. 11006, p. 1100612, International Society for Optics and Photonics, 2019

    2019

  44. [53]

    A more accurate scharfetter-gummel algorithm of electron transport for semi- conductor and gas discharge simulation,

    A. Kulikovsky, “A more accurate scharfetter-gummel algorithm of electron transport for semi- conductor and gas discharge simulation,” Journal of Computational Physics, vol. 119, no. 1, pp. 149–155, 1995. 13

    1995

  45. [54]

    FlashAttention: Fast and memory-efficient exact attention with IO-awareness,

    T. Dao, D. Y . Fu, S. Ermon, A. Rudra, and C. Ré, “FlashAttention: Fast and memory-efficient exact attention with IO-awareness,” in Advances in Neural Information Processing Systems, 2022

    2022

  46. [55]

    FlashAttention-2: Faster attention with better parallelism and work partitioning,

    T. Dao, “FlashAttention-2: Faster attention with better parallelism and work partitioning,” in Proceedings of the 12th International Conference on Learning Representations, 2024

    2024

  47. [56]

    Parallel prefix computation,

    R. E. Ladner and M. J. Fischer, “Parallel prefix computation,” Journal of the ACM, vol. 27, no. 4, pp. 831–838, 1980

    1980

  48. [57]

    Scan primitives for gpu computing,

    S. Sengupta, M. Harris, Y . Zhang, and J. D. Owens, “Scan primitives for gpu computing,” in Proceedings of the 22nd ACM SIGGRAPH/EUROGRAPHICS Symposium on Graphics Hardware, 2007

    2007

  49. [58]

    Understanding the effective receptive field in deep convolutional neural networks,

    W. Luo, Y . Li, R. Urtasun, and R. Zemel, “Understanding the effective receptive field in deep convolutional neural networks,” inAdvances in Neural Information Processing Systems, 2016. 14 Supplementary Materials GeoMaNO: Geometric Mamba Neural Operator for Partial Differentia...

    2016

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.