{"id":"b1657b19-6f76-4d4a-b1fe-c28eef0aba65","arxiv_id":"2412.20183","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"MscaleFNO, a weighted sum of Fourier neural operators with scaled inputs, substantially outperforms standard FNO on learned 1-D mappings between oscillatory functions, including Helmholtz coefficient-to-solution maps.","lead":"This paper introduces MscaleFNO, a Fourier neural operator built from several parallel branches that each receive a differently scaled version of the input function and spatial coordinate. On 1-D oscillatory mapping and Helmholtz scattering tests, it reports much lower test error than a standard FNO with similar parameter count, especially in high-frequency regimes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MscaleFNO changes input scaling and sine activation simultaneously; without a same-activation baseline the reported gains are not attributable to the multiscale mechanism.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the multiscale scaling mechanism is not isolated from the simultaneous switch to sine activation, and the claim rests on an analogy rather than a derivation or ablation. I checked the parameter counts: Eq. (33) gives 1,035,544 for MscaleFNO and Eq. (24) gives 1,164,001 for normal FNO, so the comparison is parameter-fair as claimed. The numerical gains are large and the architecture is plausible, but the central causal attribution is not yet established. A matched-activation ablation is a cheap, decisive test, so the appropriate verdict is CONDITIONAL rather than ACCEPT or REJECT. No concerns about author conduct or internal arithmetic arose; the issue is purely about confounding variables and missing controls.","tokens_in":13099,"tokens_out":3392,"duration_ms":36633,"concrete_test":"Run the Example 4.2 M=200 comparison with four matched configurations: (1) MscaleFNO with the paper's initial scales and sine activation; (2) MscaleFNO with all c_i=1 (no input scaling) and sine activation; (3) MscaleFNO with the paper's scales but GELU in the Fourier layers; (4) normal FNO with sine activation in the Fourier layers. Keep parameter counts, optimizer, epochs, seeds, and data identical. If configuration (2) or (4) reaches roughly the same 1e-2 relative test error as configuration (1), the multiscale scaling is not the driver; if only (1) succeeds, the scaling mechanism is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that the parallel scaled-input branches in Eq. (32) reduce spectral bias and yield the large error reductions reported in Section 4. However, the comparison is confounded: Section 3.2 states 'Throughout all Fourier layers, we employ the sine activation function σ(x)=sin(x)' for MscaleFNO, while Section 4 specifies only that both models use Adam with the same learning rate and comparable parameter counts. It never states the activation used in the normal FNO Fourier layers, and the projection layer of the normal FNO is described earlier with GELU. Thus the normal FNO baseline plausibly uses GELU or ReLU, so MscaleFNO differs from it by two simultaneous changes: the scaling (c_i x, c_i a(x)) and the sine activation. Sine activation is itself known to introduce high-frequency representational capacity, so the reported gains cannot be attributed to multiscale scaling without a same-activation control. The analogy to MscaleDNN in Section 3.1 is also not a derivation: MscaleDNN composes the network output with the scaled coordinate fθi(α_i x), whereas Eq. (32) evaluates each branch at the original x while scaling its inputs, and no argument shows this produces a frequency decomposition of the operator image. With hand-picked scale sets per problem and no ablation, the load-bearing premise that scaling is the causal mechanism remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MscaleFNO, an architecture of N parallel FNO branches whose inputs (spatial coordinate and input function) are scaled by factors {c_i} and whose outputs are combined with trainable weights {gamma_i}. The method is motivated by MscaleDNN, and the central claim is that the scaled parallel branches decompose the operator image into frequency bands, reducing the spectral bias of FNO for oscillatory maps. The numerical experiments cover u = sin(m a(x)) and maps from a coefficient omega(x) to the solution of a 1-D Helmholtz equation; the reported relative L2 errors are often orders of magnitude smaller than those of a normal FNO with a comparable parameter count. The paper is empirical: no convergence or frequency-decomposition theorem is proved.","tokens_in":13413,"tokens_out":8454,"duration_ms":80182,"significance":"If the reported gains survive a controlled comparison, MscaleFNO would be a simple and useful architecture for operator learning in the high-frequency regime. The paper has concrete strengths: consistent large improvements on five examples, explicit parameter-count comparisons, an out-of-distribution generalization test (Example 4.5), and a spectral visualization of branch contributions (Fig. 11). The main weakness is that the experiments change the activation and the scaling simultaneously, so the mechanism behind the gains is not identified; the analogy to MscaleDNN is suggestive but not a derivation. The central claim is therefore plausible but not yet established.","major_comments":[{"comment":"The reported improvement is confounded by an unstated activation change. Section 3.2 states that throughout all Fourier layers MscaleFNO employs the sine activation function sigma(x) = sin(x), whereas the normal FNO baseline is specified in Section 2 with GELU in the MLP (Eq. (22)) and projection (Eq. (19)) layers, and Section 4 does not state the normal FNO's Fourier-layer activation. The large error reductions in Figs. 4, 9, 14, and 16 may therefore be caused by the sine activation alone, which is known to enhance high-frequency representational capacity, rather than by the multiscale scaling. This is load-bearing for the paper's central claim. I request a same-activation control: train the normal FNO with sine activation in its Fourier layers, and also train an MscaleFNO variant with all c_i = 1 (eight branches, sine activation) to isolate the scaling mechanism.","section":"Section 3.2, Eq. (32), Section 4"},{"comment":"The analogy to MscaleDNN in Section 3.1 does not justify the proposed scaling. In MscaleDNN, Eq. (31) applies the scaled coordinate as the argument of the network output, f_theta_i(alpha_i x), which directly compresses the frequency content of the network's response. In Eq. (32), each branch evaluates FNO_theta(c_i x, c_i a(x))(x) at the original x, only scaling the inputs; no argument shows that this yields a frequency decomposition of the operator image. The spectral plot in Fig. 11 illustrates branch spectra for one trained model, but because the scales and weights are trained and the branches use sine activation, it does not establish that the scaling mechanism is responsible. A derivation for operator maps, or at least an ablation that varies only {c_i} while fixing activation and branch architecture, is needed to support the statement that subnetworks with larger c_i capture high-frequency components.","section":"Section 3.2, Eq. (32)"},{"comment":"The initial scale sets are hand-picked per problem, and no selection criterion or sensitivity study is reported. The sets differ across examples: {1,10,20,40,60,80,100,120} for Example 4.1 and most of Example 4.2, {1,40,80,100,120,140,180,200} for M = 200, and {1,4,8,10,12,14,18,20} for Example 4.3; Examples 4.4 and 4.5 do not state the scales used. These choices encode knowledge of the target frequency bands, which weakens the claim that the method provides a general solution to spectral bias. The paper should either specify a principled scale-selection rule, test sensitivity to the scale set, or use adaptive or automatic initialization.","section":"Section 4, Examples 4.1-4.4"}],"minor_comments":[{"comment":"The text reads \"M = 1 , 20, 40, 80, 100\" but the first reported case is M = 10 as shown in Fig. 9; this appears to be a typo.","section":"Example 4.2"},{"comment":"The caption contains the typo \"differet\" and should read \"different.\"","section":"Fig. 11 caption"},{"comment":"The phrase \"over intervel [−10, 10]\" should be \"over the interval [−10, 10].\"","section":"Example 4.5"},{"comment":"In the conclusion, \"Helmhotlz\" should be \"Helmholtz.\"","section":"Section 5"},{"comment":"The manuscript does not report the final trained values of the scale parameters {c_i} and combination weights {gamma_i}; since the branches in Fig. 11 are labeled by initial scales, reporting the trained scales would clarify which frequency bands each branch actually captures after training.","section":"Section 3.2"},{"comment":"No code or data availability statement is provided, which limits reproducibility for an empirical architecture study.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a numerical analysis venue as an empirical architecture contribution, and the magnitude of the reported gains makes the approach worth pursuing. However, the confound between the sine activation and the multiscale scaling must be resolved in revision; I would not require new theory, but the same-activation ablation is essential. I would also request a statement on code and data availability if the journal's policy allows it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you work on spectral bias in operator learning. The proposal is a clean extension: parallel FNO branches on scaled inputs (c_i x, c_i a(x)) plus a trained weighted sum. That architecture is not in the cited literature, so it's new. The paper does what a good empirical paper should: it reports relative L2 test errors, shows spectra, includes a generalization test, and keeps parameter counts comparable. The gains are big and consistent—MscaleFNO reaches 1e-4 on sin(20a) where FNO stays O(1), and stays below 1e-2 on Helmholtz maps where FNO degrades to 0.2 or 0.7.\n\nBut the attribution is not established. Section 3.2 states that throughout all Fourier layers the sine activation is used, while the normal FNO baseline is described elsewhere with GELU. So the comparison changes two things at once: the scaling and the activation. Sine is known to help represent high frequencies, so the reported gains could come from activation alone. The paper never runs a same-activation control. That's the softest spot, and it's load-bearing because the whole point is the multiscale mechanism. The MscaleDNN analogy in Section 3.1 is heuristic, not a derivation: it scales the output coordinate of each subnetwork, while Eq. (32) scales both inputs and evaluates at original x, and no argument shows this yields a frequency decomposition of the operator image. Fig. 11 is one trained example, not an ablation. The hand-picked initial scale sets per problem are minor; scales are hyperparameters, but a seed study would help. No code or data is provided, which is a reproducibility gap but not a fatal one.\n\nThe central idea might well be right. The evidence is consistent and the architecture is plausible. But as it stands, the paper demonstrates that this specific combination of multiscale inputs and sine activation works, not that the multiscale scaling is what does the work. A same-activation baseline and an ablation (e.g., sine activation without scaling, different scale sets) would fix it.\n\nWho is it for? Researchers working on Fourier neural operators or spectral-bias mitigation. It deserves a serious referee, but the referee should require the control experiments before acceptance. I'd bring it to a reading group to discuss the confound.","headline":"MscaleFNO is a plausible architecture with large reported gains, but the sine-activation confound means the multiscale mechanism itself isn't actually demonstrated.","tokens_in":13882,"tokens_out":2234,"would_cite":true,"duration_ms":20283,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","35J05","65T50"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a weighted sum of parallel FNO branches, each fed a scaled copy of both the coordinate and the input function, removes FNO's spectral bias and lets it learn oscillatory maps like sin(20a) to 1e-4 relative error…","keywords":["Fourier neural operator","multi-scale architecture","spectral bias","oscillatory function spaces","Helmholtz equation","operator learning","high-frequency approximation","wave scattering"],"falsifier":"Train MscaleFNO with all c_i fixed to 1 while keeping the parallel branches and sine activation, and also a variant that scales only x (not a(x)), on the sin(20a) task; if either variant matches the reported 1e-4 accuracy, then the claim that simultaneous input-function and coordinate scaling drives the improvement is false.","tokens_in":12909,"feed_emoji":"🌊","tokens_out":8272,"duration_ms":67200,"temperature":0.7,"pith_summary":"The paper proposes MscaleFNO, a multi-scale Fourier neural operator for learning maps between oscillatory functions, such as the coefficient-to-solution map of the Helmholtz equation. The central claim is that a weighted sum of parallel FNO branches, each processing scaled copies of the spatial coordinate and the input function, decomposes the operator's output into frequency bands and thereby removes the spectral bias that makes a normal FNO fail on high-frequency targets. In the reported tests, MscaleFNO reaches around 1e-4 relative test error on u = sin(20a) where normal FNO stays at O(1), and stays below 1e-2 on Helmholtz maps where normal FNO degrades to roughly 0.2 (M=200) or 0.7 (L=10). If this holds, it gives a parameter-efficient way to learn oscillatory operators in wave scattering and related settings.","feed_headline":"Multi-scale FNO hits 1e-4 where standard FNO stalls at O(1)","feed_subtitle":"Parallel scaled FNO branches learn wave scattering maps that ordinary FNO misses, down to 1e-2 error.","key_machinery":"The load-bearing object is the scaled-input branch sum of Eq. (32): N parallel FNO sub-networks, each fed (c_i x, c_i a(x)) instead of (x, a(x)), with trainable scale factors c_i, combination weights gamma_i, and sine activation in all Fourier layers. The idea, carried over from MscaleDNN, is that a large c_i compresses a high-frequency pattern into a low-frequency one that the sub-network can learn, and the weighted sum reconstructs the full spectrum of the operator output. In the paper's view, smaller c_i branches capture low-frequency components and larger c_i branches capture high-frequency ones.","core_discovery":"The paper's discovery claim is that the architecture u(x) = sum_i gamma_i FNO_theta(c_i x, c_i a(x))(x), with trainable scales c_i and weights gamma_i, gives a frequency decomposition of the operator image and cures the spectral bias of FNO. The authors verify this on two families of problems: a synthetic nonlinear map u = sin(ma(x)) meant to mimic the Green's-function dependence on wavenumber, and the 1-D Helmholtz scattering map from wave-number perturbation to field. In both, MscaleFNO with roughly the same or fewer parameters than the normal FNO reaches relative errors around 1e-4 to below 1e-2 while normal FNO stalls or degrades as frequency content grows.","pith_inferences":["An ablation that scales only x and not a(x), or that keeps scaling but replaces sine activation with GELU, would isolate whether the reported gains come from the multiscale scaling mechanism or from the simultaneous architectural changes; the paper does not run this ablation.","The branch-spectrum evidence in Fig. 11 comes from a single trained network at M=200; a seed-averaged study would show whether the frequency decomposition by scale is stable.","If the scaling mechanism is what matters, the initial scales {c_i} could be chosen adaptively from the input's DFT rather than hand-set, which the paper does not explore.","The current evidence is one-dimensional; the paper's own future-work mention of higher dimensions leaves open whether scaling both arguments remains separable in 2-D and 3-D."],"forward_implications":["MscaleFNO makes oscillatory operator learning tractable at parameter counts comparable to normal FNO: the reported models have fewer parameters yet much lower errors.","The method keeps the FNO's Fourier-layer machinery intact, so existing FNO implementations can be wrapped in parallel branches without changing the spectral convolution core.","Because the synthetic sin(ma) map stands in for the Green's function's dependence on wave number, success there suggests the mechanism transfers to genuine wave-scattering maps, as the Helmholtz tests show.","The paper's out-of-distribution test at L=10 indicates the learned multi-scale decomposition generalizes beyond the training distribution of input perturbations.","Extension to higher-dimensional Helmholtz equations and inverse medium problems is the paper's stated next step."],"supporting_citations":[{"why":"Supplies the FNO architecture that MscaleFNO extends and the comparison baseline.","marker":"[8]"},{"why":"Supplies the multi-scale DNN scaling argument that the paper extends to operators.","marker":"[10]"},{"why":"Corrects and refines that MscaleDNN scaling argument and is cited alongside [10].","marker":"[17]"},{"why":"Establishes the spectral bias of deep networks that motivates the design.","marker":"[13]"},{"why":"Supplies the frequency principle used to justify low-frequency learning.","marker":"[15]"},{"why":"Prior operator-learning method addressing spectral bias that the paper distinguishes itself from.","marker":"[9]"}],"fun_headline_variants":["Multi-scale FNO cures spectral bias in oscillatory maps","Parallel scaled FNOs catch high-frequency wave maps FNO misses","MscaleFNO: splitting frequency scales beats plain FNO on Helmholtz","Trainable scale branches let FNO resolve high-frequency scattering","Beyond spectral bias: multi-scale FNO for oscillatory functions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that stretching both the coordinate and the input function by the same factor inside each branch separates the operator's output into frequency bands, asserted by analogy with MscaleDNN and supported only by one trained example's spectra rather than by derivation or an ablation that isolates the scaling mechanism.","fun_headline_variants_meta":{"raw":{"variants":["Multi-scale FNO cures spectral bias in oscillatory maps","Parallel scaled FNOs catch high-frequency wave maps FNO misses","MscaleFNO: splitting frequency scales beats plain FNO on Helmholtz","Trainable scale branches let FNO resolve high-frequency scattering","Beyond spectral bias: multi-scale FNO for oscillatory functions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000174,"raw_usage":{"total_tokens":1221,"prompt_tokens":820,"completion_tokens":401,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":436,"completion_tokens_details":{"reasoning_tokens":312}},"tokens_in":436,"tokens_out":401,"duration_ms":4256,"temperature":1.0,"reasoning_tokens":312,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:28:09.759304+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MscaleFNO with all c_i fixed to 1 while keeping the parallel branches and sine activation, and also a variant that scales only x (not a(x)), on the sin(20a) task; if either variant matches the reported 1e-4 accuracy, then the claim that simultaneous input-function and coordinate scaling drives the improvement is false.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the FNO architecture that MscaleFNO extends and the comparison baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the multi-scale DNN scaling argument that the paper extends to operators."},{"cited_title":"Zhang, W","cited_arxiv_id":null,"evidence_quote":"Corrects and refines that MscaleDNN scaling argument and is cited alongside [10]."},{"cited_title":"Rahaman, A","cited_arxiv_id":null,"evidence_quote":"Establishes the spectral bias of deep networks that motivates the design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the frequency principle used to justify low-frequency learning."}],"review_version":1}