REVIEW 4 major objections 7 minor 36 references
Graph-Based Operator Learning from Limited Data on Irregular Domains
T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper proposes GOLA, a graph-based operator learner with a learnable Fourier encoder, and claims it outperforms grid-based and graph-based baselines on four 2D PDE benchmarks, with the largest gains in data-scarce regimes.
desk verdict The architecture is a reasonable incremental combination, but the headline outperformance claim rests on an unfair comparison: FNO, DeepONet, and GCN are evaluated on the full grid while GOLA is evaluated on 1000 random points. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the learnable Fourier encoder. For a set of $M$ learnable frequency vectors $\omega_m\in\mathbb{R}^2$, the encoder builds complex exponential basis functions $\varphi_m(x)=e^{2\pi i\langle\omega_m,x\rangle}$, computes Fourier coefficients $\hat{u}_{c,m}=\frac{1}{N}\sum_i f_{c,i}\,\overline{\varphi_m(x_i)}$ over the sampled points, filters these coefficients through learnable complex weights $W$, and reconstructs node features by the inverse transform $h=\mathrm{Re}(v)$. Because the basis functions are evaluated directly at the sampled coordinates, the encoder is defined on arbitrary point locations rather than a grid. These spectral node features then feed an attention-enhanced graph neural network: message passing aggregates local neighbor information, and multi-head self-attention with normalized keys and values captures global dependencies. The paper's complexity statement is $O(MNC + NkC^2 + NkC)$ for $N$ points, $M$ Fourier modes, $C$ channels, and $k$ average neighbors.
What would settle it
Run GOLA on inputs sampled from a deliberately nonuniform distribution, such as points concentrated near domain boundaries or drawn from a strongly peaked density on the unit square, and compare relative $L^2$ error against the uniform-sampling baseline; if the unweighted average in Eq. (4) is biased, the reconstructed input field will not converge to the true $f$ as $N$ grows and the test error should degrade in a way that density-weighted quadrature would correct.
Extended reading notes
Core claim
On its own terms, the paper establishes that a graph neural operator with a spectral input encoder can approximate PDE solution operators on irregularly sampled point clouds, and that this architecture beats the baselines it is compared against in the tested settings. With 30 training samples, GOLA achieves relative $L^2$ errors of 0.1554, 0.3707, 0.0803, and 0.0747 on the four benchmarks, edging out the best grid-based baseline (FNO) on each task. With 100 training samples and 1000 sample points, the reported reductions over GKN are 34.38%, 20.65%, 47.69%, and 57.95%. The measured trend is monotone in the expected direction: more training data and denser sampling both lower test error, and the model continues to improve when evaluated at test densities (up to 4000 points) beyond its training density (1000 points). The paper interprets these results as evidence that combining global spectral information with local attention-based message passing yields a continuous, data-efficient approximator that adapts to non-Euclidean geometries.
Load-bearing premise
The load-bearing premise is that a plain average over the sampled points recovers the input function's true Fourier coefficients; the paper only tests points drawn uniformly from a grid, so it has not shown the encoder is unbiased for genuinely nonuniform or clustered sampling. If that premise fails, the reconstructed node features no longer represent the input field and the learned operator cannot be trusted.
Editorial extensions
If this is right
- Training GOLA on 20–40 PDE solution pairs already brings relative $L^2$ error down sharply on all four benchmarks, so the method is usable when generating many training solutions is expensive.
- The model generalizes across resolutions: after training on 1000 sampled points per input, test error keeps decreasing as evaluation density rises to 4000 points, meaning predictions remain consistent on finer discretizations.
- Spectral encoding is doing real work: graph-only GCN and kernel-only GKN are consistently worse than GOLA, and the gap grows on Eikonal and Nonlinear Diffusion.
- GOLA's advantage over GKN is strongest in the data-scarce and high-density regime, suggesting the architecture's inductive bias matters most when the training set cannot compensate for model weaknesses.
Reading between the lines
- I infer that the encoder's unbiasedness depends on sampling uniformity: since Eq. (4) is an unweighted average, genuinely clustered or adaptively sampled points would bias the Fourier coefficients, and a density-aware quadrature weight would be needed before the irregular-domain claim is fully demonstrated.
- I infer the same encoder-decoder split could be transplanted to inverse problems where inputs arrive from fixed sparse sensors, treating sensor locations as graph nodes; the paper's resolution-generalization results suggest the learned operator would not be tied to a particular sensor layout.
- I infer that the method's advantage will not be uniform across PDE classes: the reported gains over GKN are large and growing with density for Eikonal and Nonlinear Diffusion but smaller and less consistent for Advection, hinting that smoothness or global coupling of the solution moderates the benefit.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GOLA, an operator-learning architecture that combines a graph constructed over sampled spatial points, a learnable Fourier-type spectral encoder (Eqs. 2-6), message-passing layers, and multi-head self-attention. The goal is to learn PDE solution operators from limited data on irregular or nonuniform point sets. The authors evaluate the model on four 2D PDE benchmarks (Darcy Flow, Advection, Eikonal, Nonlinear Diffusion) across training sizes of 5-100 and sampling densities of 20-1000, and claim that GOLA consistently outperforms DeepONet, FNO, GCN, and GKN. A proposition in Section 4 asserts universal operator approximation.
Significance. If the experimental claims held under a common evaluation protocol, the contribution would be of practical interest: the combination of a learnable spectral encoder with attention-enhanced GNNs is natural for mesh-free operator learning, and the paper covers an unusually wide grid of training sizes and sampling densities, including resolution-shift and data-efficiency studies in Tables 3-8 and the Appendix. The GOLA-versus-GKN comparisons use the same 1000-point evaluation protocol for both models and show consistent improvements, which is a genuine strength. The reported complexity O(MNC)+O(NkC^2) and the parameter count (2,900,249) are useful practical details. The chief weakness is that the headline comparisons against FNO, DeepONet, and GCN are not computed on the same evaluation points, so the central claim of consistent outperformance is not supported by the evidence as presented, and no variance information is reported.
major comments (4)
- [§5, Table 1; Eq. (16)] The evaluation protocol for the headline comparison is not common across models. The text states that 'We test DeepONet, FNO, GCN on standard grid data, and test GKN and our model GOLA on 1000 sample data randomly selected from grid data.' The relative L2 error in Eq. (16) is therefore evaluated on a full 128x128 grid for FNO/DeepONet/GCN but on only 1000 points for GOLA/GKN. A smaller error on a smaller evaluation set does not establish better accuracy, and the Section 5 claim that GOLA 'consistently achieves the lowest test error across all four benchmarks' is not supported. All models should be evaluated on identical test point sets (the same 1000-point subset or the full grid), and both numbers should be reported.
- [§5, Tables 1-8 and Figures 2-6] No random seeds, repetitions, or error bars are reported anywhere. Because the training sets are tiny (5-100 samples) and the input points are randomly subsampled, run-to-run variability is likely substantial; the 20.65-57.95% improvements over GKN in Table 2 could sit within this variability. The authors should train with at least three to five seeds and report mean relative L2 errors with standard deviations for each table and figure.
- [§3.3, Eq. (4) and §5] The Fourier coefficients in Eq. (4) are an unweighted 1/N average, which is an unbiased estimator only under approximately uniform sampling. The title and abstract claim support for irregular domains and nonuniform samples, but every experiment in Section 5 subsamples uniformly from a 128x128 grid over the unit square; no experiment uses a nonuniform density or a non-Cartesian geometry. For genuine nonuniformity the estimator becomes biased, and the reconstruction in Eq. (6) has no stated convergence guarantee; whether end-to-end training could compensate is not analyzed. The authors should add a nonuniform-sampling experiment (e.g., clustered or adaptively sampled points) together with a quadrature-weighted version of Eq. (4), or restrict the claim to uniform random subsampling.
- [§4, Proposition] The proof misuses Fourier completeness. The completeness of the Fourier basis in L2(Ω) is a statement about the fixed integer harmonics, but the architecture uses a finite set of learnable frequencies {ω_m} with coefficients computed by the unweighted average in Eq. (4), which are not the L2 projection coefficients for an arbitrary point set. The assertion that the encoder 'can approximate the functional input f to arbitrary precision' does not follow from the cited result. In addition, pointwise sampling of an arbitrary f in L2(Ω) is not well-defined because L2 functions are equivalence classes, and the asserted convergence of interpolation over an increasingly dense point cloud requires regularity assumptions that are not stated. The proposition should be reformulated (e.g., for continuous functions with a uniform sampling scheme or proper quadrature), or the proof should be replaced by a citation to existing neural-operator universal approximation theorems.
minor comments (7)
- [§3.3, Eqs. (2)-(6)] The notation is confusing: u is used both for the PDE solution (Eqs. (1) and (16)) and for the Fourier-domain quantities in Eqs. (4)-(5). Please rename the spectral coefficients (e.g., â).
- [§3.5, Eq. (12)] The sentence 'where where G_h ∈ R^{d_h×d_h}, w is calculated by the number of points' contains a duplicated 'where', and the weight w(y_j) is never defined; specify the normalization, e.g., w(y_j)=1/N.
- [§3.4, Eq. (8)] The displayed formula for m̂ is garbled ('m̂=||(...)'); the concatenation symbol and the four aggregated statistics should be typeset cleanly.
- [§2 and References] The related-work paragraph is informal ('There are many latest research about...'), and the bibliography contains duplicated entries (Sanchez-Gonzalez et al., Hegde et al., Liu et al. each appear twice) as well as an 'Anonymous (2025)' entry that is not citable.
- [§5, dataset description] The paper does not describe how the training and test data were generated (PDE solvers, simulation resolution, parameter distributions, test-set composition); this information is needed for reproducibility and to confirm that the 100 test cases are distinct from the training cases.
- [§5, complexity discussion] Parameter counts are given only for GOLA (2,900,249); reporting the parameter counts and wall-clock times of the baselines would make the efficiency comparison meaningful.
- [Figure 1] Figure 1 is not described in the text; a brief walk-through of the pipeline (graph construction, Fourier encoder, message passing, attention, output projection) would help the reader.
Circularity Check
No significant circularity: the empirical claims are trained and tested on external data, and the theoretical sketch invokes external universal-approximation results rather than the paper's own fitted parameters or self-citations.
full rationale
The paper's central claims are empirical: GOLA is trained on generated PDE datasets and evaluated on held-out test problems, so the reported relative L2 errors (Eq. 16) are not derived from any fitted constant of the model itself. The theoretical proposition in Section 4 is a generic universal-approximation argument that invokes external results—the operator universal approximation theorem of Lu et al. (2019), GNN universal approximation results of Xu et al. (2019) and Morris et al. (2019), and the completeness of Fourier bases in L2(Ω). None of these citations is from the present authors, and none is used to forbid alternatives or to justify the architecture in a way that presupposes the conclusion. The Fourier encoder equations (4)-(6) are defined as a projection and reconstruction, not as a quantity that is fitted and then renamed a prediction; the learnable complex coefficients W are trained on data and the outputs are compared against independent solution fields. No equation in the paper reduces by construction to its own input, and no fitted parameter is relabeled as a prediction. Potential weaknesses—such as the unweighted 1/N average in Eq. (4) being biased for nonuniform sampling, and the baseline comparison in Table 1 using different evaluation grids for different models—are genuine experimental and correctness risks, but they are not instances of circular reasoning under the definitions used here. Accordingly, no circular step is identified and the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- Number of Fourier modes M =
not reported
- Graph edge threshold r =
not reported
- Attention heads H and hidden dimensions =
not reported
assumptions (4)
- standard math The standard Fourier basis is complete in L2(Omega)
- domain assumption GNNs are universal approximators for functions on graphs
- domain assumption Randomly sampled points from a uniform grid represent the continuous domain for training and evaluation
- ad hoc to paper The unweighted 1/N average in Eq. (4) yields valid Fourier coefficients for nonuniform samples
Cite this review
Pith. "Pith review of Graph-Based Operator Learning from Limited Data on Irregular Domains." pith.science (2026). https://pith.science/paper/QRXUJVBK
@misc{pith2026250518923,
author = {Pith},
title = {Pith review of: Graph-Based Operator Learning from Limited Data on Irregular Domains},
year = {2026},
howpublished = {\url{https://pith.science/paper/QRXUJVBK}},
note = {Machine review of arXiv:2505.18923}
}
read the original abstract
Operator learning seeks to approximate mappings from input functions to output solutions, particularly in the context of partial differential equations (PDEs). While recent advances such as DeepONet and Fourier Neural Operator (FNO) have demonstrated strong performance, they often rely on regular grid discretizations, limiting their applicability to complex or irregular domains. In this work, we propose a Graph-based Operator Learning with Attention (GOLA) framework that addresses this limitation by constructing graphs from irregularly sampled spatial points and leveraging attention-enhanced Graph Neural Netwoks (GNNs) to model spatial dependencies with global information. To improve the expressive capacity, we introduce a Fourier-based encoder that projects input functions into a frequency space using learnable complex coefficients, allowing for flexible embeddings even with sparse or nonuniform samples. We evaluated our approach across a range of 2D PDEs, including Darcy Flow, Advection, Eikonal, and Nonlinear Diffusion, under varying sampling densities. Our method consistently outperforms baselines, particularly in data-scarce regimes, demonstrating strong generalization and efficiency on irregular domains.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2020). Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895
arXiv 2020
-
[2]
Lu, L., Jin, P., and Karniadakis, G. E. (2019). Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193
arXiv 2019
-
[3]
Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics , 378:686--707
2019
-
[4]
Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., and Anandkumar, A. (2021). Physics-informed neural operator for learning partial differential equations. arXiv preprint arXiv:2111.03794
arXiv 2021
-
[5]
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2020). Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485
arXiv 2020
-
[6]
Kipf, T. N. and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR)
work page 2016
-
[7]
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. (2017). Graph attention networks. In International Conference on Learning Representations (ICLR)
work page 2017
-
[8]
Battaglia, P. W. et al. (2018). Relational inductive biases, deep learning, and graph networks. Nature , 557(7707):528--536
work page 2018
Show all 36 references
-
[9]
Sanchez-Gonzalez, A. et al. (2020). Learning to simulate complex physics with graph networks. ICML
2020
-
[10]
Liu, W. et al. (2021). Physics-informed graph neural networks for learning and solving partial differential equations. NeurIPS Workshop on ML for Physical Sciences
2021
-
[11]
Bar-Sinai, Y. et al. (2019). Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences , 116(31):15344--15349
2019
-
[12]
Brandstetter, J. et al. (2022). Message passing neural PDE solver. In International Conference on Learning Representations (ICLR)
2022
-
[13]
Wang, L. et al. (2022). PDE-GCN: Learning PDEs on graphs. In International Conference on Machine Learning (ICML)
2022
-
[14]
Hegde, C. et al. (2022). Graph neural networks for learning PDEs from sparse and irregular data. NeurIPS
2022
-
[15]
and Kehl, R
Markidis, S. and Kehl, R. (2022). Neural operators are graph neural networks. arXiv preprint arXiv:2209.04899
2022 arXiv
-
[16]
Lam, R. et al. (2024). GraphCast: Learning skillful medium-range global weather forecasting. Science , 383(6674):346--351
2024
-
[17]
Jiang, Y., Li, Z., Chen, Q., et al. (2023). Tokenized neural operators. arXiv preprint arXiv:2305.11674
2023 arXiv
-
[18]
Zhang, X. et al. (2024). Attention-based Fourier neural operator for parametric PDEs. arXiv preprint arXiv:2401.04501
2024 arXiv
-
[19]
Patel, A. K. et al. (2023). Transformers generalize neural operators. arXiv preprint arXiv:2310.05372
2023 arXiv
-
[20]
B., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A
Li, Z., Kovachki, N. B., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A. M., and Anandkumar, A. (2023). Graph neural operator for parametric partial differential equations. In International Conference on Learning Representations (ICLR) . URL: https://openreview.net/...
2023
-
[21]
Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., and Battaglia, P. (2020). Learning mesh-based simulation with graph networks. In International Conference on Machine Learning (ICML) . URL: https://arxiv.org/abs/2006.09262
2020 arXiv
-
[22]
Liu, Y., Lütjens, B., Azizzadenesheli, K., and Anandkumar, A. (2022). U-Netformer: A U-Net style transformer for solving PDEs. arXiv preprint arXiv:2206.11832
2022 arXiv
-
[23]
Zhou, Q., Sun, L., and Lin, Z. (2023). Tokenformer: A token mixing architecture for solving PDEs. arXiv preprint arXiv:2310.02271
2023 arXiv
-
[24]
Xiao, Z., Hao, Z., Lin, B., Deng, Z., and Su, H. (2024). Improved operator learning by orthogonal attention. In Proceedings of the 41st International Conference on Machine Learning , pages 54288--54299. PMLR. URL: https://proceedings.mlr.press/v235/xiao24c.html
2024
-
[25]
F., Preciado, V
Kissas, G., Seidman, J., Guilhoto, L. F., Preciado, V. M., Pappas, G. J., and Perdikaris, P. (2022). Learning operators with coupled attention. Journal of Machine Learning Research , 23(152):1--63. URL: http://jmlr.org/papers/v23/21-1521.html
2022
-
[26]
and Townsend, A
Boullé, N. and Townsend, A. (2024). A mathematical guide to operator learning. Handbook of Numerical Analysis , 25:83--125. DOI: 10.1016/bs.hna.2024.05.003. URL: https://doi.org/10.1016/bs.hna.2024.05.003
2024 doi
-
[27]
Xu, M., Han, J., Lou, A., Kossaifi, J., Ramanathan, A., Azizzadenesheli, K., Leskovec, J., Ermon, S., and Anandkumar, A. (2024). Equivariant graph neural operator for modeling 3D dynamics. arXiv preprint arXiv:2401.11037 . URL: https://arxiv.org/abs/2401.11037
2024 arXiv
-
[28]
and Gu, G
Jin, Z. and Gu, G. X. (2023). Leveraging graph neural networks and neural operator techniques for high-fidelity mesh-based physics simulations. AIP Advances , 13(4):046109. DOI: 10.1063/5.0141469. URL: https://pubs.aip.org/aip/aml/article/1/4/046109/2923152
2023 doi
-
[29]
Cuomo, S., Di Cola, V., Giampaolo, F., Rozza, G., Raissi, M., and Piccialli, F. (2022). Physics-informed machine learning: A survey on problems, methods, and applications. ACM Computing Surveys (CSUR) , 55(1):1--38. DOI: 10.1145/3501297
2022 doi
-
[30]
B., Lanthaler, S., and Stuart, A
Kovachki, N. B., Lanthaler, S., and Stuart, A. M. (2024). Operator learning: Algorithms and analysis. arXiv preprint arXiv:2402.15715
2024 arXiv
-
[31]
and Townsend, A
Boull \'e , N. and Townsend, A. (2023). A mathematical guide to operator learning. arXiv preprint arXiv:2312.14688
2023 arXiv
-
[32]
Nelsen, N. H. and Stuart, A. M. (2024). Operator learning using random features: A tool for scientific computing. arXiv preprint arXiv:2408.06526
2024 arXiv
-
[33]
Cauchy random features for operator learning in Sobolev space
Anonymous (2025). Cauchy random features for operator learning in Sobolev space. arXiv preprint arXiv:2503.00300
2025 arXiv
-
[34]
Batlle, P., Darcy, M., Hosseini, B., and Owhadi, H. (2023). Kernel methods are competitive for operator learning. arXiv preprint arXiv:2304.13202
2023 arXiv
-
[35]
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. (2019). How powerful are graph neural networks? In International Conference on Learning Representations (ICLR) . URL: https://openreview.net/forum?id=ryGs6iA5Km
2019
-
[36]
L., Lenssen, J
Morris, C., Ritzert, M., Fey, M., Hamilton, W. L., Lenssen, J. E., Rattan, G., and Grohe, M. (2019). Weisfeiler and Leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, number 01, pages 4602--4609
2019
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.