Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

MscaleFNO: Multi-scale Fourier Neural Operator Learning for Oscillatory Function Spaces

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a weighted sum of parallel FNO branches, each fed a scaled copy of both the coordinate and the input function, removes FNO's spectral bias and lets it learn oscillatory maps like sin(20a) to 1e-4 relative error…

desk verdict MscaleFNO is a plausible architecture with large reported gains, but the sine-activation confound means the multiscale mechanism itself isn't actually demonstrated. read the letter →

arxiv 2412.20183 v1 pith:UDHII3BC submitted 2024-12-28 math.NA cs.NA

Zhilin You , Zhenli Xu , Wei Cai This is my paper · ORCID
classification math.NAcs.NA MSC 68T0735J0565T50
keywords Fourierneuraloperatormulti-scalearchitecturespectralbiasoscillatoryfunctionspacesHelmholtzequationlearninghigh-frequencyapproximationwavescattering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MscaleFNO, a multi-scale Fourier neural operator for learning maps between oscillatory functions, such as the coefficient-to-solution map of the Helmholtz equation. The central claim is that a weighted sum of parallel FNO branches, each processing scaled copies of the spatial coordinate and the input function, decomposes the operator's output into frequency bands and thereby removes the spectral bias that makes a normal FNO fail on high-frequency targets. In the reported tests, MscaleFNO reaches around 1e-4 relative test error on u = sin(20a) where normal FNO stays at O(1), and stays below 1e-2 on Helmholtz maps where normal FNO degrades to roughly 0.2 (M=200) or 0.7 (L=10). If this holds, it gives a parameter-efficient way to learn oscillatory operators in wave scattering and related settings.

What carries the argument

The load-bearing object is the scaled-input branch sum of Eq. (32): N parallel FNO sub-networks, each fed (c_i x, c_i a(x)) instead of (x, a(x)), with trainable scale factors c_i, combination weights gamma_i, and sine activation in all Fourier layers. The idea, carried over from MscaleDNN, is that a large c_i compresses a high-frequency pattern into a low-frequency one that the sub-network can learn, and the weighted sum reconstructs the full spectrum of the operator output. In the paper's view, smaller c_i branches capture low-frequency components and larger c_i branches capture high-frequency ones.

What would settle it

Train MscaleFNO with all c_i fixed to 1 while keeping the parallel branches and sine activation, and also a variant that scales only x (not a(x)), on the sin(20a) task; if either variant matches the reported 1e-4 accuracy, then the claim that simultaneous input-function and coordinate scaling drives the improvement is false.

Watch

Extended reading notes

Core claim

The paper's discovery claim is that the architecture u(x) = sum_i gamma_i FNO_theta(c_i x, c_i a(x))(x), with trainable scales c_i and weights gamma_i, gives a frequency decomposition of the operator image and cures the spectral bias of FNO. The authors verify this on two families of problems: a synthetic nonlinear map u = sin(ma(x)) meant to mimic the Green's-function dependence on wavenumber, and the 1-D Helmholtz scattering map from wave-number perturbation to field. In both, MscaleFNO with roughly the same or fewer parameters than the normal FNO reaches relative errors around 1e-4 to below 1e-2 while normal FNO stalls or degrades as frequency content grows.

Load-bearing premise

The load-bearing premise is that stretching both the coordinate and the input function by the same factor inside each branch separates the operator's output into frequency bands, asserted by analogy with MscaleDNN and supported only by one trained example's spectra rather than by derivation or an ablation that isolates the scaling mechanism.

Editorial extensions

If this is right

  • MscaleFNO makes oscillatory operator learning tractable at parameter counts comparable to normal FNO: the reported models have fewer parameters yet much lower errors.
  • The method keeps the FNO's Fourier-layer machinery intact, so existing FNO implementations can be wrapped in parallel branches without changing the spectral convolution core.
  • Because the synthetic sin(ma) map stands in for the Green's function's dependence on wave number, success there suggests the mechanism transfers to genuine wave-scattering maps, as the Helmholtz tests show.
  • The paper's out-of-distribution test at L=10 indicates the learned multi-scale decomposition generalizes beyond the training distribution of input perturbations.
  • Extension to higher-dimensional Helmholtz equations and inverse medium problems is the paper's stated next step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An ablation that scales only x and not a(x), or that keeps scaling but replaces sine activation with GELU, would isolate whether the reported gains come from the multiscale scaling mechanism or from the simultaneous architectural changes; the paper does not run this ablation.
  • The branch-spectrum evidence in Fig. 11 comes from a single trained network at M=200; a seed-averaged study would show whether the frequency decomposition by scale is stable.
  • If the scaling mechanism is what matters, the initial scales {c_i} could be chosen adaptively from the input's DFT rather than hand-set, which the paper does not explore.
  • The current evidence is one-dimensional; the paper's own future-work mention of higher dimensions leaves open whether scaling both arguments remains separable in 2-D and 3-D.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MscaleFNO, an architecture of N parallel FNO branches whose inputs (spatial coordinate and input function) are scaled by factors {c_i} and whose outputs are combined with trainable weights {gamma_i}. The method is motivated by MscaleDNN, and the central claim is that the scaled parallel branches decompose the operator image into frequency bands, reducing the spectral bias of FNO for oscillatory maps. The numerical experiments cover u = sin(m a(x)) and maps from a coefficient omega(x) to the solution of a 1-D Helmholtz equation; the reported relative L2 errors are often orders of magnitude smaller than those of a normal FNO with a comparable parameter count. The paper is empirical: no convergence or frequency-decomposition theorem is proved.

Significance. If the reported gains survive a controlled comparison, MscaleFNO would be a simple and useful architecture for operator learning in the high-frequency regime. The paper has concrete strengths: consistent large improvements on five examples, explicit parameter-count comparisons, an out-of-distribution generalization test (Example 4.5), and a spectral visualization of branch contributions (Fig. 11). The main weakness is that the experiments change the activation and the scaling simultaneously, so the mechanism behind the gains is not identified; the analogy to MscaleDNN is suggestive but not a derivation. The central claim is therefore plausible but not yet established.

major comments (3)
  1. [Section 3.2, Eq. (32), Section 4] The reported improvement is confounded by an unstated activation change. Section 3.2 states that throughout all Fourier layers MscaleFNO employs the sine activation function sigma(x) = sin(x), whereas the normal FNO baseline is specified in Section 2 with GELU in the MLP (Eq. (22)) and projection (Eq. (19)) layers, and Section 4 does not state the normal FNO's Fourier-layer activation. The large error reductions in Figs. 4, 9, 14, and 16 may therefore be caused by the sine activation alone, which is known to enhance high-frequency representational capacity, rather than by the multiscale scaling. This is load-bearing for the paper's central claim. I request a same-activation control: train the normal FNO with sine activation in its Fourier layers, and also train an MscaleFNO variant with all c_i = 1 (eight branches, sine activation) to isolate the scaling mechanism.
  2. [Section 3.2, Eq. (32)] The analogy to MscaleDNN in Section 3.1 does not justify the proposed scaling. In MscaleDNN, Eq. (31) applies the scaled coordinate as the argument of the network output, f_theta_i(alpha_i x), which directly compresses the frequency content of the network's response. In Eq. (32), each branch evaluates FNO_theta(c_i x, c_i a(x))(x) at the original x, only scaling the inputs; no argument shows that this yields a frequency decomposition of the operator image. The spectral plot in Fig. 11 illustrates branch spectra for one trained model, but because the scales and weights are trained and the branches use sine activation, it does not establish that the scaling mechanism is responsible. A derivation for operator maps, or at least an ablation that varies only {c_i} while fixing activation and branch architecture, is needed to support the statement that subnetworks with larger c_i capture high-frequency components.
  3. [Section 4, Examples 4.1-4.4] The initial scale sets are hand-picked per problem, and no selection criterion or sensitivity study is reported. The sets differ across examples: {1,10,20,40,60,80,100,120} for Example 4.1 and most of Example 4.2, {1,40,80,100,120,140,180,200} for M = 200, and {1,4,8,10,12,14,18,20} for Example 4.3; Examples 4.4 and 4.5 do not state the scales used. These choices encode knowledge of the target frequency bands, which weakens the claim that the method provides a general solution to spectral bias. The paper should either specify a principled scale-selection rule, test sensitivity to the scale set, or use adaptive or automatic initialization.
minor comments (6)
  1. [Example 4.2] The text reads "M = 1 , 20, 40, 80, 100" but the first reported case is M = 10 as shown in Fig. 9; this appears to be a typo.
  2. [Fig. 11 caption] The caption contains the typo "differet" and should read "different."
  3. [Example 4.5] The phrase "over intervel [−10, 10]" should be "over the interval [−10, 10]."
  4. [Section 5] In the conclusion, "Helmhotlz" should be "Helmholtz."
  5. [Section 3.2] The manuscript does not report the final trained values of the scale parameters {c_i} and combination weights {gamma_i}; since the branches in Fig. 11 are labeled by initial scales, reporting the trained scales would clarify which frequency bands each branch actually captures after training.
  6. [General] No code or data availability statement is provided, which limits reproducibility for an empirical architecture study.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: the central error comparisons are independent benchmarks; one mild by-construction diagnostic (Fig. 11) is presented as evidence but is not load-bearing.

  1. self definitional [Section 3.2 Eq. (32); Section 4.1, Fig. 11]
    "Fig. 11 presents the discrete Fourier transform of the contributions {γi FNOθi(cix, cia(x))}8 i=1 from different subnetworks in MscaleFNO at M = 200, revealing a systematic pattern in the frequency decomposition across different scales. Specifically, the subnetworks with increasing scale parameters exhibit enhanced capability in capturing high-frequency components"

    Eq. (32) defines each branch by the scaled input pair (cix, cia(x)); after the linear lifting (Eq. 15) and the sine activation in the Fourier layers, the phase of the branch's internal features carries ci x and ci a(x), so larger ci directly gives the branch access to higher-frequency basis functions. The observed ordering of branch spectra in Fig. 11 is therefore a consequence of the scaling definition, not an independent discovery that the architecture decomposes the operator by frequency. This step is weak evidence, but it is non-load-bearing: the main claims rest on held-out relative L2 errors of MscaleFNO versus normal FNO, which are independent of this diagnostic.

full rationale

The paper is an empirical architecture study, not a derivation, and its central comparisons (MscaleFNO vs normal FNO on u=sin(20a), multi-frequency maps, and Helmholtz scattering with L up to 10) are evaluated on held-out test sets with matched parameter counts, so no result reduces to its training input by construction. The MscaleDNN material in Section 3.1 is cited as inspiration, not as a theorem that forces Eq. (32), and the self-citations are background rather than load-bearing. The hand-chosen initial scale sets are hyperparameters that encode prior knowledge of target frequency bands; they are not fitted parameters renamed as predictions, and the normal FNO baseline has the same Fourier-mode capacity yet fails, so the comparison retains independent content. The only mild circularity-adjacent element is the Fig. 11 'spectral decomposition' diagnostic, which is essentially the designed scaling mechanism showing itself; since the quantitative results do not depend on this diagnostic, the overall circularity score is 1 rather than 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on transferring the frequency principle from scalar DNNs to FNOs and on hand-selected scale initialization; these are assumed rather than demonstrated. No new physical entities are introduced.

free parameters (2)
  • Initial scale factors {c_i} = {1,10,20,40,60,80,100,120} for M<=100; {1,40,80,100,120,140,180,200} for M=200; {1,4,8,10,12,14,18,20} for Helmholtz L=1
    Hand-chosen per test problem based on expected frequency content; no sensitivity study is given, and these values directly control which frequencies each branch can emphasize.
  • Trained combination weights {gamma_i} = learned during training
    The branch outputs are blended with trainable weights, so the final frequency mix is partly fit to the training data rather than prescribed by the architecture.
assumptions (3)
  • domain assumption The frequency principle, or spectral bias, of DNNs extends to FNOs mapping between function spaces.
    Section 3.1 motivates MscaleDNN using the frequency principle, and Section 3.2 extends it to operator learning without proof or a dedicated experiment.
  • ad hoc to paper Scaling the input as (c_i x, c_i a(x)) in each branch yields a frequency decomposition of the output analogous to the scalar MscaleDNN case.
    Eq. (32) postulates this architecture; only Fig. 11 shows one trained example's branch spectra, not a general mechanism or an ablation.
  • standard math Standard Fourier analysis results, including the convolution theorem and the Green's function decomposition for the Helmholtz equation.
    Used in Section 2.1 Eq. (10) for the FNO integral kernel and in Section 3.3 Eqs. (36)-(42) for the scattering solution representation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MscaleFNO: Multi-scale Fourier Neural Operator Learning for Oscillatory Function Spaces." pith.science (2026). https://pith.science/paper/UDHII3BC

@misc{pith2026241220183,
  author       = {Pith},
  title        = {Pith review of: MscaleFNO: Multi-scale Fourier Neural Operator Learning for Oscillatory Function Spaces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UDHII3BC}},
  note         = {Machine review of arXiv:2412.20183}
}
read the original abstract

In this paper, a multi-scale Fourier neural operator (MscaleFNO) is proposed to reduce the spectral bias of the FNO in learning the mapping between highly oscillatory functions, with application to the nonlinear mapping between the coefficient of the Helmholtz equation and its solution. The MscaleFNO consists of a series of parallel normal FNOs with scaled input of the function and the spatial variable, and their outputs are shown to be able to capture various high-frequency components of the mapping's image. Numerical methods demonstrate the substantial improvement of the MscaleFNO for the problem of wave scattering in the high-frequency regime over the normal FNO with a similar number of network parameters.

Figures

Figures reproduced from arXiv: 2412.20183 by the authors.

Figure 1
Figure 1. The FNO architecture Truncation mechanism of Rt. Considering that domain D is discretized with n ∈ N points such that vt ∈ R dv×n . When the Fast Fourier Transform (FFT) is applied to vt , it produces F(vt) ∈ R dv×n . To achieve computational efficiency, we truncate the Fourier spectrum to retain only the kmax lowest frequency modes, where kmax < n. The truncation operation is implemented through a complex-valued we… view at source ↗
Figure 2
Figure 2. The MscaleFNO architecture u (x) = X N i=1 γi FNOθm [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. The profile of the input function a(x) and its DFT The relative testing error curves is shown in [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: Error curves of different models during the training process [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Predicted solution by the normal FNO (left) and MscaleFNO [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: DFT of predicted solution by normal FNO (left) and MscaleFNO [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: The profile of the input function a(x) (left) and the DFT of a(x) (right) 0 100 200 300 400 500 modes 400 200 0 200 400 Amplitude (a) M = 10 0 100 200 300 400 500 modes 600 400 200 0 200 Amplitude (b) M = 20 0 100 200 300 400 500 modes 400 200 0 200 400 Amplitude (c) M…
Figure 8
Figure 8. Figure 8: DFT of representative exact solution u(x) for different M frequencies as M grows. This spectral distribution manifests itself in the spatial domain as an increasingly oscillatory behavior [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Error curves of different models during the training process under [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: M = 200: Predicted solution by normal FNO (left) and MscaleFNO (right) with zoomed-in inset for x ∈ [−0.18, −0.12] MscaleFNO in capturing the fine details of the solutions [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: M = 200: Spectral contributions of MscaleFNO subnetworks corresponding to differet initial scales 4.2 Mapping between scatterer properties and solution for Helmholtz Equation We consider the 1-D Helmholtz equation of (34) with a 2 (x) = λ 2 + cω(x) for a scatterer wit…
Figure 12
Figure 12. Figure 12: illustrates the characteristic profiles of input function ω(x) and the corresponding solution u(x) is shown in [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: The profile of the exact solution u(x) from ω(x) in [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Error curves of different models during the training process [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Characteristic solutions of the Helmholtz equation in spatial space [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: Error curves under different values of L (Epoch=100) [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: L = 10: Predicted solution by normal FNO (left) and MscaleFNO (right) with zoomed-in inset for x ∈ [−0.2, 0.2] [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: L = 10: DFT of predicted solution by normal FNO (left) and MscaleFNO (right) with zoomed-in inset for modes ∈ [1000, 1100] the entire spectrum, as evidenced by the zoomed-in inset (frequencies 1000- 1100). Example 4.5. To evaluate the generalization capability of the …
Figure 19
Figure 19. Figure 19: L = 10: (a) Predicted solution of normal FNO against exact solution with zoomed-in inset for x ∈ [−0.2, 0.2] and (b) The DFT of u(x) with zoomed-in inset for modes ∈ [1000, 1100] 10 5 0 5 10 x 10 0 10 Value (a) u(x) Exact solution MscaleFNO 0.2 0.0 0.2 10 0 10 0 1000 …
Figure 20
Figure 20. Figure 20: L = 10: (a) Predicted solution of MscaleFNO against exact solution with zoomed-in inset for x ∈ [−0.2, 0.2] and (b) The DFT of u(x) with zoomed-in inset for modes ∈ [1000, 1100] Acknowledgement Z. You and Z. Xu are financially supported by the National Natural Science…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On understanding and overcoming spectral biases of deep neural network learning methods for solving PDEs

    math.NA 2025-01 conditional

    A survey of methods to overcome spectral bias in deep neural network solvers for PDEs, with an emphasis on the authors' own MscaleDNN and PhaseDNN approaches.

Reference graph

Works this paper leans on

17 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Boull´ e, C

    N. Boull´ e, C. J. Earls, A. Townsend,Data-driven Discovery of Green ’s Functions with Human-understandable Deep Learning, Scientific Reports, 12:4824, 2022

    work page 2022

  2. [2]

    Boull´ e and A

    N. Boull´ e and A. Townsend,Chapter 3 - A mathematical guide to oper- ator learning, in Numerical Analysis Meets Machine Learning ,S. Mishra and A. Townsend, Eds., Handbook of Numerical Analysis, Elsevier, 25:83-125, 2024

    work page 2024

  3. [3]

    W. Cai. Deterministic, Stochastic, and Deep Learning Methods for Computational Electromagnetics, Second Edition, Springer, 2025. 21

    work page 2025

  4. [4]

    W. Cai, X. Li, L. Liu, A Phase Shift Deep Neural Network for High Fre- quency Approximation and Wave Problems , SIAM Journal on Scientific Computing, 42(5):A3285-A3312, 2020

    work page 2020

  5. [5]

    T. Chen, H. Chen, Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , IEEE Transactions on Neural Networks, 6(4):911- 917, 1995

    work page 1995

  6. [6]

    C. R. Gin, D. E. Shea, S. L. Brunton, J. N. Kutz, DeepGreen: Deep Learning of Green ’s Functions for Nonlinear Boundary Value Problems, Scientific Reports, 11:21614, 2021

    work page 2021

  7. [7]

    N. B. Kovachki, Z.-Y. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. M. Stuart, A. Anandkumar, Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs , J. Mach. Learn. Res., 24(89):1-97, 2023

    work page 2023

  8. [8]

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, Fourier Neural Operator for Parametric Partial Differential Equations , in International Conference on Learning Representations, 2021a

    work page

Show all 17 references
  1. [9]

    X. Liu, B. Xu, S. Cao, and L. Zhang, Mitigating spectral bias for the multiscale operator learning, arXiv preprint arXiv:2210.10890, 2024

    2024 arXiv

  2. [10]

    Z. Liu, W. Cai, Z.-Q. J. Xu, Multi-Scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Do- mains, Communications in Computational Physics, 28(5):1970–2001, 2020

    1970

  3. [11]

    L. Lu, P. Jin, G. Pang, Z. Zhang, G. E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, 3(3):218–229, 2021

    2021

  4. [12]

    Oommen, A

    V. Oommen, A. Bora, Z. Zhang, G. E. Karniadakis, Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling, arXiv preprint arXiv:2409.08477

    arXiv

  5. [13]

    Rahaman, A

    N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, On the spectral bias of neural networks , in International Conference on Machine Learning, PMLR, 2019, pp. 5301–5310

    2019

  6. [14]

    Ronneberger, P

    O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, in: N. Navab, J. Hornegger, W. Wells, A. Frangi (Eds.), Medical Image Computing and Computer-Assisted 22 Intervention – MICCAI 2015 , Lecture Notes in Computer Science, vol. 9...

    2015

  7. [15]

    Z.-Q. J. Xu, Y. Zhang, T. Luo, Y. Xiao, Z. Ma, Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks, Communications in Computational Physics, 28(5):1746–1767, 2020

    2020

  8. [16]

    A. I. Zayed, A Convolution and Product Theorem for the Fractional Fourier Transform, IEEE Signal Processing Letters, 5(4):101-103, 1998

    1998

  9. [17]

    Zhang, W

    L. Zhang, W. Cai, Z.-Q. J. Xu, A Correction and Comments on ”Multi- Scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains” , Communications in Computational Physics, 33(5):1509-1513, 2023. 23

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.