Pith. sign in

REVIEW 2 major objections 3 minor 44 references

Least-squares refit of the final readout improves Fourier neural operator accuracy on random obstacle-to-solution maps.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 02:17 UTC pith:TL7SF4FU

load-bearing objection The paper adds a post-training least-squares refit to a trained FNO for obstacle-to-solution maps, but the performance claims rest on details not visible in the abstract. the 2 major comments →

arxiv 2606.29436 v1 pith:TL7SF4FU submitted 2026-06-28 math.NA cs.LGcs.NA

Fourier Neural Operators with Least-Squares Readout Refit for Learning Random Obstacle-to-Solution Maps

Chenhui Zhu , Fei Wang This is my paper
classification math.NA cs.LGcs.NA
keywords Fourier neural operatorsleast-squares readout refitobstacle problemsvariational inequalitiesoperator learningrandom obstacle fieldsfree boundary problems
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies learning the map from sampled random obstacle fields on a grid directly to the solutions of elliptic variational inequalities, without an explicit stochastic parametrization of the input. After an FNO is trained end-to-end, its nonlinear backbone is frozen and the final affine readout is replaced by the solution of the induced linear least-squares problem over the full training set and all grid points. This produces the empirically optimal linear head for the fixed features. Numerical comparisons on two obstacle ensembles show that the refitted model (FNO-LS) outperforms vanilla FNO, DeepONet variants, and a two-stage baseline, with the largest gains on higher-amplitude obstacles that induce more complex contact sets.

Core claim

After end-to-end training of an FNO on random obstacle-to-solution pairs, freezing the nonlinear backbone and recomputing only the final affine readout via least-squares minimization over all training samples and grid points yields the squared-error optimal linear head for those features, improving average field accuracy, contact-set recovery, and obstacle-violation metrics especially when the backbone is informative but not fully converged.

What carries the argument

The post-training least-squares readout refit, which solves the linear least-squares problem for the final affine layer on the frozen FNO features while leaving the nonlinear representation unchanged.

Load-bearing premise

The features learned by the frozen FNO backbone remain fixed and useful for new obstacle realizations without introducing bias from the particular training samples or grid.

What would settle it

Apply the refit on a training ensemble, then evaluate whether the refitted model produces higher error than the original end-to-end FNO on a fresh ensemble of obstacle realizations drawn from the same distribution but not used in training or refitting.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The refit improves average field accuracy, contact-set recovery, and obstacle-violation metrics at low additional cost.
  • Gains are largest for higher-amplitude obstacles with more complex contact geometry.
  • The method is effective when the FNO backbone is informative but not fully converged.
  • It applies directly to grid-sampled inputs without needing an explicit truncated stochastic parametrization of the random fields.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same refit step could be tested on other operator-learning backbones whose nonlinear layers are already trained.
  • The approach may extend to other free-boundary problems governed by variational inequalities on random domains.
  • Performance on obstacle realizations lying outside the amplitude range of the training ensemble would test generalization limits.
  • Pairing the refit with larger or more diverse training sets could further reduce any residual readout bias.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper studies operator learning for maps from random self-affine obstacle fields to solutions of elliptic variational inequalities. It proposes a post-training least-squares readout refit (FNO-LS) in which a standard FNO is trained end-to-end, its nonlinear backbone is frozen, and the final affine readout is recomputed by solving the empirical linear least-squares problem over all training samples and grid points. The authors compare FNO-LS against DeepONet, POD-DeepONet, a two-stage DeepONet baseline, and vanilla FNO on two obstacle ensembles differing in amplitude, claiming that FNO-LS yields the strongest performance in average field accuracy, contact-set recovery, and obstacle-violation metrics, especially for higher-amplitude cases with complex contact geometry, at negligible extra cost.

Significance. If the numerical improvements are reproducible and generalize, the least-squares refit supplies a simple, architecture-agnostic post-processing step that extracts the optimal linear readout for any fixed nonlinear feature map. This is particularly relevant for contact problems whose solution depends on an unknown free boundary, where explicit stochastic parametrizations are cumbersome. The approach also illustrates that, when the backbone features are already informative, further gains can be obtained without retraining the expensive nonlinear layers.

major comments (2)
  1. [§3.2] §3.2 (Least-Squares Readout Refit): the description states that the readout is obtained by solving the induced linear least-squares problem over training samples and grid points, yet supplies no information on regularization, conditioning, or the numerical solver employed. Without this, it is impossible to verify that the reported “empirical squared-error optimal” readout is uniquely defined or reproducible, and the performance advantage of FNO-LS cannot be isolated from implementation choices.
  2. [§4] §4 (Numerical Experiments): the central claim that FNO-LS improves generalization rests on test-set metrics alone; no train/test gap after the refit, no ablation on training-set size, and no grid-resolution sensitivity study are reported. Consequently it remains unclear whether the observed gains on higher-amplitude obstacles arise from better utilization of the frozen features or from overfitting to training-specific contact patterns.
minor comments (3)
  1. [Abstract] The abstract asserts quantitative superiority but contains no numerical values, error bars, or sample counts; these should be added so that the performance claim can be assessed at a glance.
  2. [§2] Notation for the obstacle field, contact set, and variational inequality should be introduced once in §2 and used consistently; several symbols appear to be redefined in the numerical section.
  3. [Figures in §4] Figure captions for the contact-set and violation plots should explicitly state the number of test realizations and the precise definition of the plotted error measures.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. The two major comments identify important gaps in reproducibility and validation that we can address through clarifications and modest additional reporting. We outline our responses below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [§3.2] §3.2 (Least-Squares Readout Refit): the description states that the readout is obtained by solving the induced linear least-squares problem over training samples and grid points, yet supplies no information on regularization, conditioning, or the numerical solver employed. Without this, it is impossible to verify that the reported “empirical squared-error optimal” readout is uniquely defined or reproducible, and the performance advantage of FNO-LS cannot be isolated from implementation choices.

    Authors: We agree that the numerical details of the least-squares solve must be stated explicitly for reproducibility. In our implementation the readout weights were obtained via the standard dense linear least-squares solver (numpy.linalg.lstsq / LAPACK gelsd) applied to the tall feature matrix assembled from all training samples and all grid points; no explicit regularization was used. We observed that the condition number of the normal matrix remained moderate (typically O(10^3–10^4)) because the FNO feature dimension is modest relative to the number of equations. In the revised manuscript we will insert a short paragraph in §3.2 that (i) names the solver, (ii) states the absence of regularization, and (iii) reports the observed condition numbers on both obstacle ensembles. This change will make the procedure fully reproducible without altering any numerical results. revision: yes

  2. Referee: [§4] §4 (Numerical Experiments): the central claim that FNO-LS improves generalization rests on test-set metrics alone; no train/test gap after the refit, no ablation on training-set size, and no grid-resolution sensitivity study are reported. Consequently it remains unclear whether the observed gains on higher-amplitude obstacles arise from better utilization of the frozen features or from overfitting to training-specific contact patterns.

    Authors: The referee is correct that we did not report the training error after the refit or perform the suggested ablations. Because the refit solves an over-determined linear system whose solution is optimal for the training features, the training error necessarily decreases; the relevant question is whether the test error also improves. We will add a compact table in §4 that lists both training and test L2 errors (and contact-set metrics) before and after the readout refit for both obstacle ensembles. This will quantify the train/test gap and show that the refit reduces test error without inflating the gap. A systematic ablation on training-set size and grid resolution would require a new campaign of experiments whose computational cost exceeds the scope of the present study; we will therefore note this limitation and list it as future work rather than perform the full study in the revision. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical LS refit is explicit training-set optimization with test evaluation

full rationale

The paper describes a standard post-training procedure: train FNO end-to-end, freeze the nonlinear layers, then solve an explicit linear least-squares problem for the final affine readout using all training samples and grid points. Performance claims rest on numerical comparisons (field accuracy, contact-set recovery, obstacle-violation metrics) against baselines on held-out obstacle realizations. No derivation, theorem, or first-principles result is asserted that reduces by construction to its own inputs; the LS step is defined directly from the training data rather than being renamed or forced as a prediction. No self-citations, uniqueness theorems, or ansatzes appear in the provided text. The method is self-contained as an empirical enhancement whose generalization is tested externally to the fit.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are stated. The method implicitly relies on the existence of a well-defined linear readout space after feature extraction and on the training set being representative of the target distribution.

pith-pipeline@v0.9.1-grok · 5811 in / 1249 out tokens · 30575 ms · 2026-06-30T02:17:21.961731+00:00 · methodology

0 comments
read the original abstract

We study operator learning for random obstacle-to-solution maps arising from elliptic variational inequalities with finite-band self-affine random obstacle fields. Instead of introducing an explicit truncated stochastic parametrization of the random input, we learn the map directly from sampled obstacle realizations on a fixed grid. This problem is challenging because the solution is governed not only by the obstacle field itself, but also by the induced contact set and free-boundary geometry. We introduce a post-training least-squares readout refit for the Fourier neural operator (FNO). After the FNO is trained end to end, its nonlinear backbone is frozen and the final affine readout is recomputed by solving the induced linear least-squares problem over all training samples and grid points. The refit yields the empirical squared-error optimal readout for the learned frozen features while leaving the nonlinear representation unchanged. We compare vanilla DeepONet, POD-DeepONet, a two-stage DeepONet baseline, FNO, and FNO with least-squares readout refit (FNO-LS) on two obstacle ensembles with different amplitude levels. Numerical results show that FNO-LS achieves the strongest overall performance among the tested models, particularly for higher-amplitude obstacles with more complex contact geometry. The method improves average field accuracy, contact-set recovery, and obstacle-violation metrics at low additional cost, especially when the FNO backbone is informative but not fully converged. These results suggest that least-squares readout refit is a simple and effective post-training enhancement for learning random obstacle-to-solution maps.

Figures

Figures reproduced from arXiv: 2606.29436 by Chenhui Zhu, Fei Wang.

Figure 1
Figure 1. Figure 1: Schematic of the DeepONet architecture. The branch network encodes the sampled [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Schematic of the Fourier neural operator architecture. The lifting operator [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Two-stage FNO training and post-training readout refit. In Stage 1, the full [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Representative realizations of the random obstacle field for two amplitude levels: [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Best-case FNO-100 prediction on the test set in Example 1, selected by the smallest [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of the reference and predicted contact sets for the best-case sample [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Worst-case FNO-100 prediction on the test set in Example 1, selected by the largest [PITH_FULL_IMAGE:figures/full_fig_p022_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of the reference and predicted contact sets for the worst-case sample [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Best-case FNO-100 prediction on the test set in Example 2, selected by the smallest [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of the reference and predicted contact sets for the best-case sample [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Worst-case FNO-100 prediction on the test set in Example 2, selected by the [PITH_FULL_IMAGE:figures/full_fig_p026_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Comparison of the reference and predicted contact sets for the worst-case sample [PITH_FULL_IMAGE:figures/full_fig_p027_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 4 canonical work pages · 2 internal anchors

  1. [1]

    Alphonse, M

    A. Alphonse, M. Hinterm¨ uller, A. Kister, C. H. Lun, and C. Sirotenko, A neural net- work approach to learning solutions of a class of elliptic variational inequalities, arXiv: 27 2411.18565, 2024

    work page internal anchor Pith review arXiv 2024

  2. [2]

    Atkinson and W

    K. Atkinson and W. Han,Theoretical Numerical Analysis: A Functional Analysis Framework, Third Edition, Springer, New York, 2009

    2009

  3. [3]

    Babuˇ ska and P

    I. Babuˇ ska and P. Chatzipantelidis, On solving elliptic stochastic partial differential equations,Computer Methods in Applied Mechanics and Engineering,191(2002), 4093– 4122

    2002

  4. [4]

    Babuˇ ska, R

    I. Babuˇ ska, R. Tempone, and G. E. Zouraris, Galerkin finite element approximations of stochastic elliptic partial differential equations,SIAM Journal on Numerical Analysis, 42(2004), 800–825

    2004

  5. [5]

    Babuˇ ska, F

    I. Babuˇ ska, F. Nobile, and R. Tempone, A stochastic collocation method for elliptic partial differential equations with random input data,SIAM Journal on Numerical Analysis,45(2007), 1005–1034

    2007

  6. [6]

    El Bahja, J

    H. El Bahja, J. C. Hauffen, P. Jung, B. Bah, and I. Karambal, A physics-informed neural network framework for modeling obstacle-related equations,Nonlinear Dynamics,113 (2025), 12533–12544

    2025

  7. [7]

    Bierig and A

    C. Bierig and A. Chernov, Convergence analysis of multilevel Monte Carlo variance estimators and application for random obstacle problems,Numerische Mathematik,130 (2015), 579–613

    2015

  8. [8]

    Cheng, X

    X. Cheng, X. Shen, X. Wang, and K. Liang, A deep neural network-based method for solving obstacle problems,Nonlinear Analysis: Real World Applications,72(2023), 103864

    2023

  9. [9]

    M. Darehmiraki, A deep learning approach for the obstacle problem, InProceedings of Academia-Industry Consortium for Data Science: AICDS 2020, Springer Nature, Singapore, 2022, 179–188

    2020

  10. [10]

    Duvaut and J.-L

    G. Duvaut and J.-L. Lions,Inequalities in Mechanics and Physics, Springer-Verlag, Berlin–New York, 1976

    1976

  11. [11]

    C. Eck, J. Jaruˇ sek, and M. Krbec,Unilateral Contact Problems: Variational Methods and Existence Theorems, Monographs and Textbooks in Pure and Applied Mathematics, Vol. 270, CRC Press, Boca Raton, 2005

    2005

  12. [12]

    Eigel, C

    M. Eigel, C. Heiß, and J. E. Sch¨ utte, Multi-level neural networks for high-dimensional parametric obstacle problems, arXiv: 2504.05026, 2025. 28

    work page arXiv 2025

  13. [13]

    Forster and R

    R. Forster and R. Kornhuber, A polynomial chaos approach to stochastic variational inequalities,Journal of Numerical Mathematics,18(2010), 235–255

    2010

  14. [14]

    Y. Gao, Y. Song, Z. Tan, H. Yue, and S. Zeng, Prox-PINNs: A Deep Learning Algo- rithmic Framework for Elliptic Variational Inequalities, arXiv: 2505.14430, 2025

    work page arXiv 2025

  15. [15]

    Glowinski,Numerical Methods for Nonlinear Variational Problems, Springer-Verlag, New York, 1984

    R. Glowinski,Numerical Methods for Nonlinear Variational Problems, Springer-Verlag, New York, 1984

    1984

  16. [16]

    Glowinski, J.-L

    R. Glowinski, J.-L. Lions, and R. Tr´ emoli` eres,Numerical Analysis of Variational In- equalities, North-Holland, Amsterdam, 1981

    1981

  17. [17]

    Han and B

    W. Han and B. D. Reddy,Plasticity: Mathematical Theory and Numerical Analysis, Second Edition, Springer, New York, 2013

    2013

  18. [18]

    Han and M

    W. Han and M. Sofonea,Quasistatic Contact Problems in Viscoelasticity and Viscoplas- ticity, Studies in Advanced Mathematics, Vol. 30, American Mathematical Society, Providence, RI; International Press, Somerville, MA, 2002

    2002

  19. [19]

    Hlav´ aˇ cek, J

    I. Hlav´ aˇ cek, J. Haslinger, J. Neˇ cas, and J. Lov´ ıˇ sek,Solution of Variational Inequalities in Mechanics, Springer-Verlag, New York, 1988

    1988

  20. [20]

    H¨ ueber and B

    S. H¨ ueber and B. I. Wohlmuth, A primal-dual active set strategy for non-linear multi- body contact problems,Computer Methods in Applied Mechanics and Engineering,194 (2005), 3147–3166

    2005

  21. [21]

    K¨ arkk¨ ainen, K

    T. K¨ arkk¨ ainen, K. Kunisch, and P. Tarvainen, Augmented Lagrangian active set meth- ods for obstacle problems,Journal of Optimization Theory and Applications,119(2003), 499–533

    2003

  22. [22]

    Kikuchi and J

    N. Kikuchi and J. T. Oden,Contact Problems in Elasticity: A Study of Variational Inequalities and Finite Element Methods, SIAM, Philadelphia, 1988

    1988

  23. [23]

    Kinderlehrer and G

    D. Kinderlehrer and G. Stampacchia,An Introduction to Variational Inequalities and Their Applications, SIAM, Philadelphia, 2000

    2000

  24. [24]

    Kornhuber, C

    R. Kornhuber, C. Schwab, and M. W. Wolf, Multilevel Monte Carlo finite element methods for stochastic elliptic variational inequalities,SIAM Journal on Numerical Analysis,52(2014), 1243–1268

    2014

  25. [25]

    N. B. Kovachki, S. Lanthaler, and S. Mishra, On universal approximation and error bounds for Fourier neural operators,Journal of Machine Learning Research,22(2021), 1–76. 29

    2021

  26. [26]

    N. B. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. M. Stuart, and A. Anandkumar, Neural operator: Learning maps between function spaces with applications to PDEs,Journal of Machine Learning Research,24(2023), 1–97

    2023

  27. [27]

    Lanthaler and A

    S. Lanthaler and A. M. Stuart, The parametric complexity of operator learning,IMA Journal of Numerical Analysis,46(2026), 647–712

    2026

  28. [28]

    Lee and Y

    S. Lee and Y. Shin, On the Training and Generalization of Deep Operator Networks, SIAM Journal on Scientific Computing,46(2024), C273–C296

    2024

  29. [29]

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. M. Stuart, and A. Anandkumar, Fourier neural operator for parametric partial differential equations, In International Conference on Learning Representations, 2021

    2021

  30. [30]

    L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis, Learning nonlinear opera- tors via DeepONet based on the universal approximation theorem of operators,Nature Machine Intelligence,3(2021), 218–229

    2021

  31. [31]

    L. Lu, X. Meng, S. Cai, Z. Mao, S. Goswami, Z. Zhang, and G. E. Karniadakis, A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data,Computer Methods in Applied Mechanics and Engineering,393 (2022), 114778

    2022

  32. [32]

    B. N. J. Persson, Contact mechanics for randomly rough surfaces,Surface Science Re- ports,61(2006), 201–227

    2006

  33. [33]

    J. F. Rodrigues,Obstacle Problems in Mathematical Physics, North-Holland, Amster- dam, 1987

    1987

  34. [34]

    Schwab and R

    C. Schwab and R. A. Todor, Karhunen–Lo` eve approximation of random fields by gener- alized fast multipole methods,Journal of Computational Physics,217(2006), 100–122

    2006

  35. [35]

    Schwab and A

    C. Schwab and A. Stein, Deep solution operators for variational inequalities via proximal neural networks,Research in the Mathematical Sciences,9(2022), 36

    2022

  36. [36]

    R. A. Todor and C. Schwab, Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients,IMA Journal of Numerical Analysis,27 (2007), 232–261

    2007

  37. [37]

    S. Wang, H. Wang, and P. Perdikaris, Learning the solution operator of parametric partial differential equations with physics-informed DeepONets,Science Advances,7 (2021), eabi8605. 30

    2021

  38. [38]

    F. Wang, W. Han, and X. Cheng, Discontinuous Galerkin methods for solving elliptic variational inequalities,SIAM Journal on Numerical Analysis,48(2010), 708–733

    2010

  39. [39]

    F. Wang, W. Han, and X. Cheng, Discontinuous Galerkin methods for solving a qua- sistatic contact problem,Numerische Mathematik,126(2014), 771–800

    2014

  40. [40]

    Wang and H

    F. Wang and H. Wei, Virtual element method for simplified friction problem,Applied Mathematics Letters,85(2018), 125–131

    2018

  41. [41]

    Wang and H

    F. Wang and H. Wei, Virtual element methods for the obstacle problem,IMA Journal of Numerical Analysis,40(2020), 708–728

    2020

  42. [42]

    Xiu, Efficient collocational approach for parametric uncertainty analysis,Communi- cations in Computational Physics,2(2007), 293–309

    D. Xiu, Efficient collocational approach for parametric uncertainty analysis,Communi- cations in Computational Physics,2(2007), 293–309

    2007

  43. [43]

    X. E. Zhao, W. Hao, and B. Hu, Two neural-network-based methods for solving elliptic obstacle problems,Chaos, Solitons & Fractals,161(2022), 112313

    2022

  44. [44]

    C. Zhu, F. Wang, and W. Han, Numerical Analysis of Stochastic Elliptic Variational Inequalities of the First Kind, arXiv: 2604.25111, 2026. 31

    work page internal anchor Pith review Pith/arXiv arXiv 2026