Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 reserved pith:6QWJAYDErecord.jsonopen to challenge →
read the original abstract
We show that passing input points through a simple Fourier feature mapping enables a multilayer perceptron (MLP) to learn high-frequency functions in low-dimensional problem domains. These results shed light on recent advances in computer vision and graphics that achieve state-of-the-art results by using MLPs to represent complex 3D objects and scenes. Using tools from the neural tangent kernel (NTK) literature, we show that a standard MLP fails to learn high frequencies both in theory and in practice. To overcome this spectral bias, we use a Fourier feature mapping to transform the effective NTK into a stationary kernel with a tunable bandwidth. We suggest an approach for selecting problem-specific Fourier features that greatly improves the performance of MLPs for low-dimensional regression tasks relevant to the computer vision and graphics communities.
This paper has not been read by Pith yet.
Forward citations
Cited by 26 Pith papers
-
Sampling the Schwinger Model with Gauge-Equivariant Diffusion
A gauge-equivariant diffusion model samples Schwinger model configurations, yielding unbiased observables matching MCMC and qualitatively less topological freezing than HMC.
-
One Generator, Any Process: LLM-Conditioning for the LHC
LLM embeddings condition generative networks for LHC events, yielding faster convergence, higher quality, and generalization to unseen processes.
-
Physics informed operator learning of parameter dependent spectra
DeepOPiraKAN learns parameter-to-spectrum mappings via operator learning and achieves relative errors of O(10^{-6}) to O(10^{-4}) for Kerr black hole quasinormal modes up to n=7 when benchmarked against Leaver's method.
-
Deep learning-based phase-field modelling of brittle fracture in anisotropic media
A variational physics-informed neural network solves higher-order anisotropic phase-field fracture models by minimizing total energy with B-spline enriched trial functions.
-
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
NeuS introduces a bias-free volume rendering method for signed distance function representations to reconstruct accurate surfaces from 2D images.
-
Self-Supervised Implicit CEST Reconstruction via Physics-Informed Lorentz Encoding
Embedding learnable Lorentzian basis functions into the spectral encoding layer of an implicit neural representation enables high-fidelity CEST Z-spectrum reconstruction from as few as 21 frequency offsets.
-
Confidence-feedback-weighted graph matching network: online-offline laser-induced damage site matching under complex interference
A confidence-feedback-weighted graph matching network achieves 96.36% F1-score on damage site matching by using matchability confidence to weight edge features and applying geometric consistency and hard-example mining.
-
One Generator, Any Process: LLM-Conditioning for the LHC
LLM embeddings condition a generative transformer to enable faster convergence, better performance, and generalization to unseen LHC processes using a single model.
-
Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation
MiTaS fuses multi-resolution tactile data from GelSight and Evetac sensors with vision using modality-specific stems and transformer fusion to condition flow-matching policies, reporting 80% average success on five co...
-
IV-Net: A neural network for elliptic PDEs with random and highly varying coefficients
IV-Net is a multigrid-inspired convolutional neural operator that approximates solutions to linear elliptic PDEs with high-contrast coefficients and shows better accuracy than POD and other neural operators on heterog...
-
RoPeSLR: 3D RoPE-driven Sparse-LowRank Attention for Efficient Diffusion Transformers
RoPeSLR combines 3D RoPE-guided sparse attention with head-wise low-rank parameterization to achieve sub-quadratic complexity in DiTs while preserving distance awareness for efficient ultra-long video synthesis.
-
LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators
LiFT factorizes 3D medical volume synthesis into per-slice 2D generation and inter-slice trajectory learning, using a tri-planar drifting loss for unconditional coherence and a z-context mixer for paired translation tasks.
-
MoMo: Conditioned Contrastive Representation Learning for Preference-Modulated Planning
MoMo conditions contrastive representations and prediction operators on user preferences via FiLM and low-rank modulation to enable continuous modulation of plan safety while preserving inference efficiency.
-
MoMo: Conditioned Contrastive Representation Learning for Preference-Modulated Planning
MoMo uses Feature-Wise Linear Modulation and low-rank neural modulation to condition contrastive planning representations on user preferences while preserving inference efficiency and probability density ratios.
-
From Characterization To Construction: Generative Quantum Circuit Synthesis from Gate Set Tomography Data
A generative QMLC framework tokenizes GST data, embeds it via curriculum-trained set-vision transformers into a context-aware latent space, and uses diffusion models to synthesize circuits conditioned on desired measu...
-
When PINNs Go Wrong: Pseudo-Time Stepping Against Spurious Solutions
PINNs fail on spurious solutions admitted by the residual loss; adaptive pseudo-time stepping with Jacobian-based step selection improves accuracy and robustness on PDE benchmarks.
-
A Deep Ritz Method for High-Dimensional Steady States of the Cahn-Hilliard Equation
A Deep Ritz method with augmented Lagrangian and Fourier feature mappings computes high-dimensional steady states of the Cahn-Hilliard equation and identifies multiple nontrivial phase separation patterns.
-
AE-ViT: Stable Long-Horizon Parametric Partial Differential Equations Modeling
AE-ViT combines a convolutional autoencoder with a latent-space transformer and multi-stage parameter plus coordinate injection to deliver stable long-horizon predictions for parametric PDEs, cutting relative rollout ...
-
Monte Carlo Event Generation with Continuous Normalizing Flows
Continuous normalizing flows improve unweighting efficiency in Monte Carlo event generation for high-jet-multiplicity collider processes by factors up to 184, with wall-time gains of about ten when combined with coupl...
-
Cell-induced densification and tether formation in fibrous extracellular matrices with biomimetic physics-informed neural networks
Bio-PINNs with a near-to-far curriculum and deformation-uncertainty proxy recover cell-induced densified phases and tether morphologies more reliably than standard adaptive PINN baselines in single-cell and multicellu...
-
Nomic Embed: Training a Reproducible Long Context Text Embedder
Nomic AI produced and open-sourced a reproducible 8192-context English text embedder that exceeds OpenAI Ada-002 and text-embedding-3-small performance on MTEB short-context and LoCo long-context benchmarks.
-
General-Purpose Nonlinear Function Approximation via Linear Integrated Photonics
Optical random Fourier feature mapping enables approximation of tenth-order polynomials, special functions, neural activations, and high-dimensional softmax in linear silicon photonic hardware.
-
FEDONet : Fourier-Embedded DeepONet for Spectrally Accurate Operator Learning
FEDONet augments DeepONet with Fourier-embedded trunk networks using random Fourier features, yielding lower L2 reconstruction errors than standard DeepONet on Burgers', 2D Poisson, Eikonal, Allen-Cahn, and Kuramoto-S...
-
Flow Matching for Convective-Scale Precipitation Downscaling
Flow matching produces better spatial structure than diffusion models for convective precipitation downscaling but underestimates heavy rainfall amounts.
-
Machine Learning-based Separation of the He I 10830{\AA} Chromospheric Signal: Quantitative Analysis of Chromosphere-Corona Intensity in the Quiet Sun
CNN separation of He I 10830Å chromospheric signal from photospheric contamination in quiet Sun reveals R ≈ -0.84 anti-correlation with 304Å and magnetic-field-dependent EUV coupling.
-
A Deep Ritz Method for High-Dimensional Steady States of the Cahn-Hilliard Equation
A deep learning framework computes multiple steady states of the Cahn-Hilliard equation in high dimensions using Fourier feature mappings and an augmented Lagrangian for mass conservation.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.