Pith. sign in

REVIEW 3 major objections 4 minor 40 references

By reversing the order of truncation and distributed exchange, DTST computes exactly the retained Fourier modes as a sum of local partial DFTs, making spectral-layer communication depend on mode count rather than grid size—and the paper rep

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 02:12 UTC pith:O73GDK23

load-bearing objection A genuinely useful distributed-spectral-transform idea with clean, exact math, but the abstract oversells the communication reduction at scale: the 'under 6%' claim only holds at 4 GPUs. the 3 major comments →

arxiv 2607.14394 v1 pith:O73GDK23 submitted 2026-07-15 cs.DC

DRIFT: Direct Reduced Fourier Transforms for Distributed Spectral Neural Operators

Sana Taghipour Anvari , David Kaeli This is my paper
classification cs.DC
keywords Fourier Neural OperatorsDistributed Truncated Spectral Transformpartial DFTmode truncationcommunication-avoiding algorithmsAllReducespectral methodsPDE surrogates
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that for Fourier Neural Operators and other truncated spectral methods, the standard distributed FFT—which communicates the full spatial grid even though most coefficients are immediately discarded—can be replaced by the Distributed Truncated Spectral Transform (DTST). In DTST, each GPU computes only the retained frequency modes locally, using partial DFTs, and two global collectives (an AllReduce and an AllGather) combine the results with a payload that depends only on the mode count, not on spatial resolution. The paper shows this is mathematically exact, identical to a full distributed FFT followed by truncation, and reports 38–64× forward-pass speedups and a 37× training speedup over the distributed FNO baseline on 4–32 GPUs, with communication dropping from 97% to under 6% of forward-pass time. A sympathetic reader should care because this directly attacks the dominant scaling bottleneck for learned PDE surrogates at high resolution.

Core claim

The central claim is that the truncated global spectrum is exactly the sum of per-GPU partial spectra: X̂[k] = Σ_p X̂_p[k]. Because the DFT is linear, truncation and aggregation commute, so instead of computing a full distributed FFT (which requires multiple dense all-to-all repartitions of the full N^d tensor) and then discarding all but M modes, each GPU can evaluate a partial DFT over only the retained modes on its own spatial block, sum those partial spectra with an AllReduce, apply sharded spectral weights, and reassemble the convolved spectrum with an AllGather. This makes the per-layer communication volume O(M), independent of spatial resolution, with O(log P) latency. The paper verif

What carries the argument

The key object is the per-dimension DFT basis matrix B_i, whose rows are the twiddle factors for the retained modes; multiplying a local spatial axis by B_i contracts it from N_loc^i to |S_i| modes. A phase offset p_i·N_loc^i encodes each GPU's global position, and separability of the multidimensional DFT lets the transform be applied as sequential per-dimension matrix multiplications. Processing non-distributed dimensions first progressively compresses the tensor, making the partial DFT cheaper overall than a full FFT at typical mode counts. The communication pair that carries the argument is one AllReduce (to sum partial spectra) and one AllGather (to reassemble the sharded convolved spect

Load-bearing premise

The load-bearing premise is aggressive mode truncation: k_max must stay far below N, so the retained spectrum M=(2k_max)^d is tiny compared with the full grid; if a spectral method keeps a large share of modes, the partial-DFT computation crosses over to being more expensive than the full FFT (the paper places the crossover near k_max≈10 on N=128) and the communication savings shrink, as the paper's own k_max=16 measurements show.

What would settle it

Run a single DTST layer at N=128 with k_max=64 (so k_max/N=0.5) on 32 GPUs and compare its wall-clock time against a distributed FFT followed by truncation: if the AllReduce/AllGather pair plus the basis-matrix multiplies is not faster than the four all-to-all repartitions of the baseline, then the claim that the advantage persists beyond aggressive truncation fails. The crossover is already predictable from the paper's communication and FLOP models by evaluating them at M≈N^d.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any spectral method that retains only M of the N^d frequency modes can replace a distributed FFT with DTST, cutting per-layer communication from O(N^d/P) all-to-all data movement to O(M) AllReduce/AllGather traffic.
  • The forward and backward passes incur identical collective costs, so training speedups track inference speedups; the paper measures a 37× end-to-end wall-clock training speedup with essentially unchanged convergence.
  • The communication savings grow with spatial resolution and dimensionality, since the fraction of communicated data that is actually used is (2k_max/N)^d, which shrinks rapidly as N or d grows.
  • At larger GPU counts, the baseline all-to-all latency scales as O(P) while the AllReduce latency scales only as O(log P), so the communication advantage is projected to widen beyond the tested 32 GPUs.
  • Because DTST is exact and requires only that the retained mode set be known in advance, it is a drop-in replacement for the distributed FFT in truncated spectral layers without changing model outputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The same reversal should apply to other truncated spectral transforms beyond learned surrogates—spherical harmonic transforms in climate models, spectral-element solvers, or any band-limited convolution on distributed data—wherever only a small set of modes is physically meaningful.
  • Editorial: The paper's own dimension-ordering analysis implies a roughly 30% reduction in partial-DFT FLOPs by contracting the largest local dimension first; overlapping the AllReduce with later partial-DFT stages would likely push the communication-bound crossover to even larger GPU counts.
  • Editorial: Since spectral weights are already sharded across GPUs, a natural extension is to shard the channel dimension as well, trading the AllGather's replicated spectrum for an additional AllReduce and reducing per-GPU spectral memory at the cost of one more collective.
  • Editorial: The method's exactness suggests a direct test on non-neural spectral solvers: replace distributed FFTs in an existing parallel PDE code with DTST and measure whether the two-collective payload remains smaller than the all-to-all repartitions once k_max/N is chosen realistically for that application.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces the Distributed Truncated Spectral Transform (DTST), which computes only the retained Fourier modes of a spatially distributed tensor by having each GPU evaluate a local partial DFT and then combining partial spectra via an AllReduce; an AllGather reassembles the convolved spectrum after per-GPU spectral weight multiplication. The authors prove that the result equals the truncated global DFT (Eqs. 5–7), analyze communication cost as O(M) payload and O(log P) latency, implement it in the DRIFT library for Fourier Neural Operators, and report large speedups over a distributed FNO baseline on a 3D+time PDEBench problem.

Significance. The core mathematical contribution is sound: Eq. (7) is an exact application of DFT linearity, and the O(M) communication bound is derived without fitted parameters. The work is valuable because it shows a principled way to avoid dense all-to-all redistribution in truncated spectral methods, and the GPU implementation with separable basis matrices is a concrete practical step. The main concerns are empirical and presentation-related: the headline communication fraction and exactness claims are not supported by the paper's own data, and the compute-efficiency claim needs direct timing evidence rather than FLOP counts alone.

major comments (3)
  1. [Abstract; Section VI-E (Fig. 9)] The abstract's claim that DRIFT reduces communication time from 97% to "under 6%" of forward-pass time is contradicted by the paper's own measurements. Summing the per-phase values in Fig. 9 on the 128^3 grid gives AR+AG fractions of 5.1/148.3 = 3.4% at P=4, but 13/87.7 = 14.8% at P=8, 15/56.5 = 26.5% at P=16, and 20/43.4 = 46.1% at P=32. Thus "under 6%" holds only at P=4; at the largest scale reported, DRIFT is already communication-bound, as Section VII's Limitations also states. This does not invalidate the DTST derivation, but the headline performance claim needs to be conditioned on P and k_max (or removed).
  2. [Section VI-C (Fig. 7)] The numerical exactness claims are stated inconsistently. Fig. 7 reports a relative Frobenius error of 3.2e-14 between the FFT+truncation and partial-DFT spectra, yet the same figure states that the distributed full-model comparison on P=16 GPUs is "bitwise identical (relative L2 = 0)". Since the forward pass contains GELU nonlinearities and pointwise operations that amplify rounding differences, 3.2e-14 input differences cannot produce bitwise-identical outputs. The claim should be "identical to machine precision" or the two numbers must be reconciled; as written, the reader cannot tell which comparison used which transform path.
  3. [Sections V-B and VII (Table II, crossover claim)] The claim that the partial-DFT gemm path is cheaper than cuFFT is based on FLOP counts (Table II), and the crossover threshold in Section VII is stated as "k_max ≈ 10 (k_max/N ≈ 0.08)" without supporting measurement. FLOP counts do not determine runtime on GPUs; a direct timing comparison of the gemm stages vs cuFFT at the same N and k_max is needed to support Contribution 3 and the crossover statement. The end-to-end speedups are encouraging, but they conflate communication savings with compute efficiency.
minor comments (4)
  1. [Section VI-E (Fig. 9)] The heatmap is labeled "per-block" but the phase times sum to the full forward-pass times in Table V (e.g., 43.9 ms at P=32 vs 43.4 ms). Clarify whether these are per-forward-pass or per-block; the current label is misleading.
  2. [Section I and Section III (Eq. 4)] The useful communicated fraction is given as (k_max/N)^d in Section I but as (2 k_max/N)^d in Eq. (4). The factor of 2 should be consistent.
  3. [Abstract and Table V] The abstract's speedup range "38–64×" is not fully supported by the data: Table V gives 37.7× at P=32 on 128^3, and the weak-scaling results in Fig. 10 range from 24× to 43×. Please qualify the range (e.g., strong scaling on 128^3 with k_max=8) or adjust the numbers.
  4. [Algorithm 1] Line 7 returns GELU(y0 + yp), while Eq. (1) uses a generic activation σ. State explicitly that σ = GELU in this implementation.

Circularity Check

1 steps flagged

Central DTST derivation is self-contained (DFT linearity); only minor in-sample fitting in the collective-time model validation prevents a perfect score.

specific steps
  1. fitted input called prediction [Section VI-A, Table I (communication model validation)]
    "For the AllReduce and AllGather, per-block fits on the inter-node configurations (P≥8, 2–8 nodes) give T_AR = 1.07 + 0.29 log2 P and T_AG = 0.44 + 0.54 log2 P (ms/block), which confirms the log2 P scaling predicted by the theoretical model with prediction errors below 8%."

    The alpha and beta constants in T_AR and T_AG are fitted to the measured collective times on the same inter-node configurations (P=8,16,32) that Table I reports as model predictions. The below-8% prediction errors are therefore in-sample residuals of the fit, not out-of-sample validation of Eq. (16)'s log-P scaling. This is a statistical circularity in the performance-model validation, but it is not load-bearing for the main exactness proof or for the measured speedups, which are reported directly from timings.

full rationale

The core claim of the paper — DTST yields the truncated global DFT by summing per-GPU partial DFTs — is derived transparently from the linearity of the DFT in Eqs. (5)–(7): the global sum over the full spatial domain is split over disjoint local domains, and each partial sum is computed locally. No parameter is fitted to make this identity hold; M is a configuration hyperparameter, and the AllReduce/AllGather payload is computed from the retained-mode set. The O(M) communication-volume and O(log P) latency statements follow from the alpha-beta model (Eqs. (8), (15), (16)) and are consistent with measured phase times; the only circularity found is the in-sample fit of the AR/AG latency constants in the Section VI-A model validation, which affects the model-prediction table but not the central derivation. The paper's numerical-exactness section has an internal wording inconsistency (relative Frobenius error 3.2e-14 above Fig. 7 vs. bitwise identical relative L2=0 below it), but that is a correctness/consistency issue, not a circular derivation. Self-citations (e.g., DaggerFFT [25]) appear only in related work and are not load-bearing. Overall, the central claims are self-contained; the minor in-sample model fitting warrants a score of 1, not higher.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central exactness claim rests only on DFT linearity (Eq. 5–7), which is classical and self-contained. The speedup and cost claims additionally rest on the α–β collective model, a contiguous-block decomposition, a no-overlap baseline for DFNO, and a gemm-vs-cuFFT FLOP model. No new physical entities are postulated. The fitted constants (AR/AG log-P fits, all-to-all α–β) are calibration of the validation model, not inputs to the claimed result.

free parameters (5)
  • AllReduce latency fit intercept a0 = 1.07 ms/block
    Fitted intercept in T_AR = a0 + a1·log2 P (ms/block), Section VI-A. Calibrates the cost model against measured AllReduce times; not an input to the claimed result.
  • AllReduce latency slope a1 = 0.29 ms/log2 P
    Fitted slope confirming log2 P scaling of AllReduce (Section VI-A). Validation-only calibration.
  • AllGather latency fit intercept b0 = 0.44 ms/block
    Fitted intercept in T_AG = b0 + b1·log2 P (Section VI-A). Validation-only calibration.
  • AllGather latency slope b1 = 0.54 ms/log2 P
    Fitted slope for AllGather scaling (Section VI-A). Validation-only calibration.
  • Baseline all-to-all α–β parameters = Implied by Table I predictions (all-to-all model within 3.3% of measured)
    The α–β latency/bandwidth values for the DFNO all-to-all model are calibrated to measured collective times (Table I). They validate Eq. (17) but are not used to derive the DTST exactness or O(M) communication claims.
axioms (5)
  • standard math Linearity of the DFT: the global spectrum is the sum of per-subdomain partial DFTs (Eqs. 5–7).
    The entire mathematical content of DTST. Splitting the spatial sum over disjoint GPU subdomains and summing is exact by linearity; no approximation is introduced.
  • standard math α–β communication model with AllReduce latency O(log2 P) and all-to-all latency O(P) (Eqs. 8–10, cited [27–30]).
    The asymptotic communication claims (O(log P) vs O(P) latency, O(M) vs O(N^d) bandwidth) rest on this standard model of MPI collectives.
  • domain assumption Spatial domain decomposition into contiguous blocks with phase-offset basis matrices B_i (Eq. 18).
    The local partial DFT assumes each GPU owns a contiguous block of the grid with local extents N_i^loc = N_i/P_i; the phase offset p_i·N_i^loc encodes global position. This is the data-distribution contract of the method.
  • domain assumption Baseline comparison premise: the DFNO baseline performs four non-overlapped all-to-all repartitions per block, so 97% of its forward time is communication (Section VI-A, Fig. 2).
    The headline 38–64× speedup is measured against this specific baseline. A baseline with communication–computation overlap or fused all-to-all would reduce the measured speedup. The paper does not compare against overlapping variants.
  • domain assumption FLOP model for FFT vs gemm: cuFFT costs (5/2)N·log2 N and the partial DFT costs 2KN per dimension (Section V-B).
    The claim that progressive-compression gemm is 13% cheaper than cuFFT at k_max=8 (Table II) uses this constant-factor FLOP model; the crossover at k_max≈10 (Section VII) is an empirical estimate from the same model.

pith-pipeline@v1.3.0-alltime-deepseek · 17425 in / 18675 out tokens · 170207 ms · 2026-08-02T02:12:01.473814+00:00 · methodology

0 comments
read the original abstract

Fourier Neural Operators (FNOs) learn solution operators for partial differential equations and offer orders of magnitude speedup over traditional numerical solvers at inference time, which makes them attractive surrogates for high-resolution computational physics. Scaling FNOs to high-resolution spatial grids requires distributing the data across GPUs, but the distributed FFT at the core of each spectral layer requires multiple dense all-to-all collectives that communicate the full spatial tensor, only for most coefficients to be discarded immediately. We introduce the Distributed Truncated Spectral Transform (DTST), which reverses this order. Each GPU computes only a small subset of frequency modes used by the spectral convolution locally via a partial DFT, and two collectives combine the results with a payload that depends only on this mode count, not the spatial resolution. DTST produces spectral coefficients identical to the standard distributed FFT with truncation, while providing both spatial data parallelism and spectral weight model parallelism. We present DRIFT, a GPU implementation of DTST for distributed Fourier Neural Operators, using separable per-dimension basis matrices and efficient GPU-to-GPU communication. On a 3D+time FNO across 4--32 GPUs, on up to 8 nodes (4 GPUs/node), DRIFT achieves a forward-pass speedup of 38--64$\times$ and a 37$\times$ training speedup over the distributed FNO baseline, reducing communication time from 97\% to under 6\% of the forward-pass time, with growing speedups at higher resolution.

Figures

Figures reproduced from arXiv: 2607.14394 by David Kaeli, Sana Taghipour Anvari.

Figure 1
Figure 1. Figure 1: Standard distributed FNO layer. 4 8 16 32 Number of GPUs 0 2000 4000 6000 Forward-pass time (ms) 6.88s 3.32s 1.75s 0.92s (a) Absolute time 0 20 40 60 80 100 Fraction of forward-pass time (%) 4 8 16 Number of GPUs 32 97.6% 2.4% 97.4% 2.6% 97.3% 2.7% 97.0% 3.0% (b) Communication fraction Communication Compute [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Communication vs. compute breakdown of the baseline [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The DRIFT layer architecture. Each GPU applies a local partial DFT first along the fully local dimensions (Z, T), then [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Data-centric view of the DRIFT pipeline with [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Partial DFT via gemm for one spatial dimension (N=128, K=16). The basis matrix B ∈ C K×N with Bk,j = ω k·j N contains only the rows corresponding to the K frequency modes: 8 low (k=0, . . . , 7) and 8 high (k=120, . . . , 127). with P = PxPy. In both cases, the basis matrix (Eq. (18)) adapts automatically through the phase offset piNloc i , which shifts the DFT twiddle factors so that each GPU’s local indi… view at source ↗
Figure 6
Figure 6. Figure 6: Progressive tensor compression across gemm stages. Each stage contracts one dimension (red arrow) via a basis￾matrix multiply, reducing the tensor that subsequent stages operate on. Blue blocks denote full-sized dimensions; green blocks denote already-contracted dimensions (K=16). By stage 4, the tensor has been compressed in three dimensions, reducing the operational volume by 60× relative to stage 1. wou… view at source ↗
Figure 7
Figure 7. Figure 7: Above: Spectral coefficient comparison on PDEBench [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: DRIFT strong scaling across five grid sizes. Left: [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Per-block phase breakdown for DRIFT at 1283 (left) and 643 (right) grids. Modes (8, 8, 8, 16), dv=20, 4 GPUs per node, one rank per GPU. from compute-bound to communication-bound explains the decreasing speedup over DFNO at higher GPU counts in Table V. Even so, the absolute communication time remains below 20 ms, over 44× lower than DFNO’s 891 ms. The spectral convolution remains negligible at all scales … view at source ↗
Figure 11
Figure 11. Figure 11: Effect of kmax on forward-pass time and speedup across 4, 16, 32 GPU counts. DRIFT’s advantage grows as fewer modes are retained. 1283 grid, dv=20, 4 blocks, 4 GPUs per node, one rank per GPU. 20 40 60 80 100 Epoch 10 1 10 0 10 1 MSE Loss DFNO DRIFT 0 2 4 6 Wall-clock time (hours) 10 1 10 0 10 1 MSE Loss DFNO: 6.8h DRIFT: 11min DFNO DRIFT [PITH_FULL_IMAGE:figures/full_fig_p010_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Training convergence on PDEBench 3D compressible [PITH_FULL_IMAGE:figures/full_fig_p010_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 12 canonical work pages · 3 internal anchors

  1. [1]

    Neural operator: Learning maps between function spaces with applications to PDEs,

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and A. Anandkumar,“Neural operator: Learning maps between function spaces with applications to PDEs,”J. Mach. Learn. Res., vol. 24, no. 89, pp. 1–97, 2023. https://doi.org/10.48550/arXiv.2108.08481

    Pith/arXiv arXiv doi:10.48550/arxiv.2108.08481 2023

  2. [2]

    Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators,

    L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis, “Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators,”Nature Mach. Intell., vol. 3, pp. 218–229, 2021. https://doi.org/10.1038/s42256-021-00302-5

    doi:10.1038/s42256-021-00302-5 2021

  3. [3]

    Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets

    S. Wang, H. Wang, and P. Perdikaris, “Learning the solution operator of parametric partial differential equations with physics-informed Deep- ONets,”Sci. Adv., 2021. https://doi.org/10.48550/arXiv.2103.10974

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2103.10974 2021

  4. [4]

    Fourier neural operator for parametric partial differential equations,

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Fourier neural operator for parametric partial differential equations,” inProc. Int. Conf. Learn. Representations (ICLR), 2021. https://doi.org/10.48550/arXiv.2010.08895

    Pith/arXiv arXiv doi:10.48550/arxiv.2010.08895 2021

  5. [5]

    Fourier neu- ral operators explained: A practical perspective,

    V . Duruisseaux, J. Kossaifi, and A. Anandkumar, “Fourier neu- ral operators explained: A practical perspective,”arXiv preprint arXiv:2512.01421, 2025. https://doi.org/10.48550/arXiv.2512.01421

    doi:10.48550/arxiv.2512.01421 2025

  6. [6]

    Physics-informed neural operator for learning partial differential equations,

    Z. Li, H. Zheng, N. Kovachki, D. Jin, H. Chen, B. Liu, K. Azizzade- nesheli, and A. Anandkumar, “Physics-informed neural operator for learning partial differential equations,”ACM/IMS J. Data Sci., vol. 1, no. 3, 2024. https://doi.org/10.48550/arXiv.2111.03794

    Pith/arXiv arXiv doi:10.48550/arxiv.2111.03794 2024

  7. [7]

    U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow,

    G. Wen, Z. Li, K. Azizzadenesheli, A. Anandkumar, and S. M. Benson, “U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow,”Adv. Water Resour., vol. 163, p. 104180,

  8. [8]

    Factorized Fourier neural operators,

    A. Tran, A. Mathews, L. Xie, and C. S. Ong, “Factorized Fourier neural operators,” inProc. Int. Conf. Learn. Representations (ICLR), 2023. https://doi.org/10.48550/arXiv.2111.13802

    Pith/arXiv arXiv doi:10.48550/arxiv.2111.13802 2023

  9. [9]

    Multi-grid tensorized Fourier neural operator for high-resolution PDEs,

    J. Kossaifi, N. Kovachki, K. Azizzadenesheli, and A. Anandkumar, “Multi-grid tensorized Fourier neural operator for high-resolution PDEs,” https://doi.org/10.48550/arXiv.2310.00120

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.00120

  10. [10]

    Tucker-FNO: Tensor Tucker-Fourier neural operator and its universal approximation theory,

    G. Zhou, Z. Zeng, Y . Luo, Q. Xie, and D. Meng, “Tucker-FNO: Tensor Tucker-Fourier neural operator and its universal approximation theory,” in(ICLR), 2026. https://openreview.net/pdf?id=UJvkXnuozY

    2026

  11. [11]

    D-FNO: A decomposed Fourier neural operator for large-scale parametric partial differential equations,

    K. Li and W. Ye, “D-FNO: A decomposed Fourier neural operator for large-scale parametric partial differential equations,”Comput. Methods Appl. Mech. Eng., 2025. https://doi.org/10.1016/j.cma.2025.117732

    arXiv 2025

  12. [12]

    Adaptive Fourier neural operators: Efficient token mixers for transformers,

    J. Guibas, M. Mardani, Z. Li, A. Tao, A. Anandkumar, and B. Catan- zaro, “Adaptive Fourier neural operators: Efficient token mixers for transformers,” inProc. Int. Conf. Learn. Representations (ICLR), 2022. https://doi.org/10.48550/arXiv.2111.13587

    Pith/arXiv arXiv doi:10.48550/arxiv.2111.13587 2022

  13. [13]

    FourCastNet: Acceler- ating global high-resolution weather forecasting using adaptive Fourier neural operators,

    T. Kurth, S. Subramanian, P. Harrington, J. Pathak, M. Mardani, D. Hall, A. Miele, K. Kashinath, and A. Anandkumar, “FourCastNet: Acceler- ating global high-resolution weather forecasting using adaptive Fourier neural operators,” inProc. Platform for Advanced Scientific Computing Conf. (PASC), 2023. https://doi.org/10.1145/3592979.3593412

    arXiv 2023

  14. [14]

    TurboFNO: High-Performance Fourier Neural Operator with Fused FFT-GEMM-iFFT on GPU

    S. Wu, Y . Zhai, H. Dai, H. Zhao, Y . Zhu, H. Hu, and Z. Chen, “TurboFNO: High-performance Fourier neural operator with fused FFT- GEMM-iFFT on GPU,” inProc. Int. Conf. High Performance Comput- ing, Networking, Storage and Analysis (SC), 2025. https://doi.org/10.48550/arXiv.2504.11681

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2504.11681 2025

  15. [15]

    Model-parallel Fourier neural oper- ators as learned surrogates for large-scale parametric PDEs,

    T. J. Grady II, R. Khan, M. Louboutin, Z. Yin, P. A. Witte, R. Chandra, R. J. Hewett, and F. J. Herrmann, “Model-parallel Fourier neural oper- ators as learned surrogates for large-scale parametric PDEs,”Comput. Geosci., 2023. https://doi.org/10.1016/j.cageo.2023.105402

    arXiv 2023

  16. [16]

    A Linear Algebraic Approach to Model Parallelism in Deep Learning

    R. J. Hewett and T. J. Grady II, “A linear algebraic approach to model parallelism in deep learning,”arXiv preprint arXiv:2006.03108, 2020. https://doi.org/10.48550/arXiv.2006.03108

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2006.03108 2006

  17. [17]

    On the communication complexity of 3D FFTs and its implications for exascale,

    K. Czechowski, C. Battaglino, C. McClanahan, K. Iyer, P.-K. Yeung, and R. Vuduc, “On the communication complexity of 3D FFTs and its implications for exascale,” inProc. ACM Int. Conf. Supercomputing,

  18. [18]

    Scalability issues in FFT computation,

    A. Ayala, S. Tomov, M. Stoyanov, and J. Dongarra, “Scalability issues in FFT computation,” inParallel Computing Technologies, Springer, 2021, pp. 279–287. https://doi.org/10.1007/978-3-030-86359-3 21

    work page doi:10.1007/978-3-030-86359-3 2021

  19. [19]

    heFFTe: Highly efficient FFT for exascale,

    A. Ayala, S. Tomov, A. Haidar, and J. Dongarra, “heFFTe: Highly efficient FFT for exascale,” inProc. Int. Conf. Computational Science (ICCS), 2020. https://doi.org/10.1007/978-3-030-50371-0 19

    work page doi:10.1007/978-3-030-50371-0 2020

  20. [20]

    Impacts of multi-GPU MPI collective communications on large FFT computation,

    A. Ayala, S. Tomov, X. Luo, H. Shaiek, A. Haidar, G. Bosilca, and J. Dongarra, “Impacts of multi-GPU MPI collective communications on large FFT computation,” inProc. IEEE/ACM Workshop on Exascale MPI (ExaMPI), 2019. https://doi.org/10.1109/ExaMPI49596.2019.00007

    arXiv 2019

  21. [21]

    Implementation of parallel FFT algorithms on distributed memory machines with a minimum overhead of communication,

    J.-M. Calvin, “Implementation of parallel FFT algorithms on distributed memory machines with a minimum overhead of communication,”Par- allel Comput., 1997. https://doi.org/10.1016/S0167-8191(96)00039-7

    work page doi:10.1016/s0167-8191(96)00039-7 1997

  22. [22]

    PFFT: An extension of FFTW to massively parallel archi- tectures,

    M. Pippig, “PFFT: An extension of FFTW to massively parallel archi- tectures,”SIAM J. Sci. Comput., vol. 35, no. 3, pp. C213–C236, 2013. https://doi.org/10.1137/120885887

    work page doi:10.1137/120885887 2013

  23. [23]

    2DECOMP&FFT – A highly scalable 2D decom- position library and FFT interface,

    N. Li and S. Laizet, “2DECOMP&FFT – A highly scalable 2D decom- position library and FFT interface,” inCray User Group Conf., 2010. https://api.semanticscholar.org/CorpusID:62453043

    2010

  24. [24]

    Low communication FMM-accelerated FFT on GPUs,

    A. Abdelfattahet al., “Low communication FMM-accelerated FFT on GPUs,” inProc. Int. Conf. High Performance Computing, Networking, Storage and Analysis (SC), 2017 https://doi.org/10.1145/3126908.3126919

    arXiv 2017

  25. [25]

    DaggerFFT: A distributed FFT framework using task scheduling in Julia,

    S. Taghipour Anvari, J. Samaroo, M. Raayai Ardakani, and D. Kaeli, “DaggerFFT: A distributed FFT framework using task scheduling in Julia,” inProc. IEEE Int. Parallel and Distributed Processing Symp. (IPDPS), 2026. https://doi.org/10.48550/arXiv.2601.12209

    work page doi:10.48550/arxiv.2601.12209 2026

  26. [26]

    Automatic generation of mappings for distributed Fourier operations,

    D. Popovici, B. Wu, J. Shalf, and M. Kong, “Automatic generation of mappings for distributed Fourier operations,” inProc. Int. Conf. High Performance Computing, Networking, Storage and Analysis (SC), 2025. https://doi.org/10.1145/3712285.3759869

    arXiv 2025

  27. [27]

    Kumar, A

    V . Kumar, A. Grama, A. Gupta, and G. Karypis,Introduction to Parallel Computing, 2nd ed. Addison-Wesley, 2003

    2003

  28. [28]

    Toward performance models of MPI implementations for understanding application scaling issues,

    T. Hoefler, W. Gropp, R. Thakur, and J. L. Tr ¨aff, “Toward performance models of MPI implementations for understanding application scaling issues,” inProc. European MPI Users’ Group Meeting (EuroMPI), 2010, pp. 21–30. https://doi.org/10.1007/978-3-642-15646-5 3

    work page doi:10.1007/978-3-642-15646-5 2010

  29. [29]

    Collective communication: Theory, practice, and experience,

    E. Chan, M. Heimlich, A. Purkayastha, and R. van de Geijn, “Collective communication: Theory, practice, and experience,”Concurrency and Computation: Practice and Experience, vol. 19, no. 13, pp. 1749–1783,

  30. [30]

    Performance analysis of MPI collective operations,

    J. Pje ˇsivac-Grbovi´c, T. Angskun, G. Bosilca, G. E. Fagg, E. Gabriel, and J. J. Dongarra, “Performance analysis of MPI collective operations,” Cluster Computing, 2007. https://doi.org/10.1007/s10586-007-0012-0

    work page doi:10.1007/s10586-007-0012-0 2007

  31. [31]

    An algorithm for the evaluation of finite trigonometric series,

    G. Goertzel, “An algorithm for the evaluation of finite trigonometric series,”The American Mathematical Monthly, vol. 65, no. 1, pp. 34–35,

  32. [32]

    The communication challenge for MPP: Intel Paragon and Meiko CS-2,

    R. W. Hockney, “The communication challenge for MPP: Intel Paragon and Meiko CS-2,”Parallel Computing, vol. 20, no. 3, pp. 389–398,

  33. [33]

    PDEBench: An extensive benchmark for scientific machine learning,

    M. Takamoto, T. Praditia, R. Lebert, M. N. Holme, T. Luber, R. Lippe, C. Meinikheim, R. Klamt, T. Friederich, and others, “PDEBench: An extensive benchmark for scientific machine learning,” inProc. Advances in Neural Information Processing Systems (NeurIPS), 2022. https://doi.org/10.48550/arXiv.2210.07182

    Pith/arXiv arXiv doi:10.48550/arxiv.2210.07182 2022

  34. [34]

    The spectral element method: An efficient tool to simulate the seismic response of 2D and 3D geological structures,

    D. Komatitsch and J.-P. Vilotte, “The spectral element method: An efficient tool to simulate the seismic response of 2D and 3D geological structures,”Bull. Seismol. Soc. Am., vol. 88, no. 2, pp. 368–392, 1998. https://doi.org/10.1785/BSSA0880020368

    doi:10.1785/bssa0880020368 1998

  35. [35]

    A fast spherical harmonics transform for global NWP and climate models,

    N. P. Wedi, M. Hamrud, and G. Mozdzynski, “A fast spherical harmonics transform for global NWP and climate models,”Mon. Weather Rev., vol. 141, no. 10, pp. 3450–3461, 2013. https://doi.org/10.1175/MWR-D-13-00016.1

    work page doi:10.1175/mwr-d-13-00016.1 2013

  36. [1958]

    https://doi.org/10.2307/2310304

    work page doi:10.2307/2310304

  37. [1994]

    https://doi.org/10.1016/S0167-8191(06)80021-9

    doi:10.1016/s0167-8191(06)80021-9

  38. [2007]

    https://doi.org/10.1002/cpe.1206

    doi:10.1002/cpe.1206

  39. [2012]

    https://doi.org/10.1145/2304576.2304604

    arXiv

  40. [2022]

    https://doi.org/10.48550/arXiv.2109.03697

    Pith/arXiv arXiv doi:10.48550/arxiv.2109.03697