REVIEW 20 cited by
Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
read the original abstract
Vision transformers have delivered tremendous success in representation learning. This is primarily due to effective token mixing through self attention. However, this scales quadratically with the number of pixels, which becomes infeasible for high-resolution inputs. To cope with this challenge, we propose Adaptive Fourier Neural Operator (AFNO) as an efficient token mixer that learns to mix in the Fourier domain. AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution. This principle was previously used to design FNO, which solves global convolution efficiently in the Fourier domain and has shown promise in learning challenging PDEs. To handle challenges in visual representation learning such as discontinuities in images and high resolution inputs, we propose principled architectural modifications to FNO which results in memory and computational efficiency. This includes imposing a block-diagonal structure on the channel mixing weights, adaptively sharing weights across tokens, and sparsifying the frequency modes via soft-thresholding and shrinkage. The resulting model is highly parallel with a quasi-linear complexity and has linear memory in the sequence size. AFNO outperforms self-attention mechanisms for few-shot segmentation in terms of both efficiency and accuracy. For Cityscapes segmentation with the Segformer-B3 backbone, AFNO can handle a sequence size of 65k and outperforms other efficient self-attention mechanisms.
Forward citations
Cited by 20 Pith papers
-
Structure Before Collapse: Transient semantic geometry in next-token prediction
Semantic geometry emerges transiently early in next-token prediction training before collapsing to Neural Collapse symmetry in synthetic settings with latent semantic factors.
-
CHASM: Cross-frequency Harmonized Axis-Separable Mixing for Spectral Token Operators
CHASM introduces a cross-frequency harmonized axis-separable spectral mixer using a shared channel eigenbasis plus per-frequency positive gains, yielding consistent gains over same-backbone baselines in medical and na...
-
Latent Fourier Transform
LatentFT uses latent-space Fourier transforms and frequency masking in diffusion autoencoders to enable timescale-specific manipulation of musical structure in generative models.
-
Non-markovian neural quantum propagator and its application to the simulation of ultrafast nonlinear spectra
A machine learning model called neural quantum propagator is introduced to efficiently solve non-Markovian quantum dynamics described by HEOM and applied to simulate spectra of the FMO complex.
-
Chiaroscuro Attention: Spending Compute in the Dark
CHIAR-Former routes tokens via spectral entropy to DCT mixing or attention, yielding 35-40% FLOP savings at 400M parameters with modest perplexity increase on WikiText-103.
-
Uncovering Extreme Event Mechanisms for Prediction and Control with Sensitivity-Balanced Projections
CoBRAS projections via backpropagation capture sensitivity to enable data-driven prediction and control of extreme events in chaotic dynamical systems including turbulence, oscillator networks, and rogue waves.
-
Skillful high-resolution weather forecasting independent of physical models
ObsCast produces skillful short-term high-resolution weather analyses and forecasts over the contiguous US and Europe using only observational data, outperforming operational NWP without relying on NWP-derived data fo...
-
AOT-POT: Adaptive Operator Transformation for Large-Scale PDE Pre-training
AOT-POT adaptively reshapes complex PDE solution operators via input-dependent transformations and parallel stream mixing to enable effective large-scale pre-training, yielding SOTA results on 12 benchmarks with minim...
-
U-HNO: A U-shaped Hybrid Neural Operator with Sparse-Point Adaptive Routing for Non-stationary PDE Dynamics
U-HNO uses adaptive per-point routing in a U-shaped hybrid architecture to achieve state-of-the-art accuracy on PDE benchmarks with sharp localized features.
-
Don't Fix the Basis -- Learn It: Spectral Representation with Adaptive Basis Learning for PDEs
ABLE learns a spatially adaptive Parseval frame from data via an ancillary density to replace fixed bases in spectral neural operators for PDEs.
-
Large-eddy simulation nets (LESnets) based on physics-informed neural operator for wall-bounded turbulence
LESnets integrates LES equations and the law of the wall into F-FNO to enable data-free, stable long-term predictions of wall-bounded turbulence at Re_tau up to 1000 on coarse grids, matching traditional LES accuracy ...
-
Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting
ESFM is a single open foundation model that unifies heterogeneous Earth data sources and forecasts missing regions while preserving inter-variable physical relationships.
-
Multi-Scale Wavelet Transformers for Operator Learning of Dynamical Systems
Multi-scale wavelet transformers learn operator dynamics of chaotic systems in the wavelet domain, yielding lower errors and higher spectral fidelity on benchmarks and ERA5 climate data.
-
NOWS: Neural Operator Warm Starts for Accelerating Iterative Solvers
Neural operators supply warm-start guesses that cut iteration counts and runtime by up to 90% in Krylov solvers for PDEs while retaining the original methods' convergence guarantees.
-
Estimating Parameter Fields in Multi-Physics PDEs from Scarce Measurements
Neptune infers spatiotemporal parameter fields in PDEs from as few as 45 sparse measurements using independent coordinate neural networks, outperforming PINNs and neural operators with lower errors and better extrapolation.
-
Foundations of Practical Quantum Advantage in Quantum-Informed Machine Learning for Predicting Chaos
Develops k-indexed Q-Priors for quantum-informed ML on chaotic systems, proving a two-stage quantum-classical separation in measurement complexity and demonstrating it in turbulent flow and weather forecasting workflows.
-
Caracal: Causal Architecture via Spectral Mixing
Caracal is a Fourier-based sequence mixing architecture that achieves causal autoregressive modeling with standard operators and competitive performance on long sequences.
-
Euler-inspired Decoupling Neural Operator for Efficient Pansharpening
EDNO redefines pansharpening as a frequency-domain functional mapping that decouples fusion via Euler-inspired polar coordinates into explicit phase-rotation simulation and implicit spectral modeling for improved efficiency.
-
Physics-guided spatiotemporal neural models for fuel density prediction
Adding physics-guided loss terms to ConvLSTM, AFNONet, and ViViT improves fuel density prediction accuracy and stability over purely data-driven baselines on simulated prescribed-fire data.
-
A Geometry-Aware Triplane Field Network for Vehicle Aerodynamic Prediction
GTF-Net combines triplane features, AFNO spectral mixing, CNN refinement, and explicit geometric encodings to reduce relative L2 error on vehicle pressure and shear stress prediction versus prior baselines.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.