pith. sign in

hub

On the Spectral Bias of Neural Networks

10 Pith papers cite this work. Polarity classification is still indexing.

10 Pith papers citing it
abstract

Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with $100\%$ accuracy. In this work, we present properties of neural networks that complement this aspect of expressivity. By using tools from Fourier analysis, we show that deep ReLU networks are biased towards low frequency functions, meaning that they cannot have local fluctuations without affecting their global behavior. Intuitively, this property is in line with the observation that over-parameterized networks find simple patterns that generalize across data samples. We also investigate how the shape of the data manifold affects expressivity by showing evidence that learning high frequencies gets \emph{easier} with increasing manifold complexity, and present a theoretical understanding of this behavior. Finally, we study the robustness of the frequency components with respect to parameter perturbation, to develop the intuition that the parameters must be finely tuned to express high frequency functions.

hub tools

citation-role summary

background 3

citation-polarity summary

roles

background 3

polarities

background 3

clear filters

representative citing papers

FEDONet : Fourier-Embedded DeepONet for Spectrally Accurate Operator Learning

cs.LG · 2025-09-15 · conditional · novelty 5.0

FEDONet augments DeepONet with Fourier-embedded trunk networks using random Fourier features, yielding lower L2 reconstruction errors than standard DeepONet on Burgers', 2D Poisson, Eikonal, Allen-Cahn, and Kuramoto-Sivashinsky equations across dataset sizes and noise levels.

citing papers explorer

Showing 1 of 1 citing paper after filters.

  • FEDONet : Fourier-Embedded DeepONet for Spectrally Accurate Operator Learning cs.LG · 2025-09-15 · conditional · none · ref 51 · internal anchor

    FEDONet augments DeepONet with Fourier-embedded trunk networks using random Fourier features, yielding lower L2 reconstruction errors than standard DeepONet on Burgers', 2D Poisson, Eikonal, Allen-Cahn, and Kuramoto-Sivashinsky equations across dataset sizes and noise levels.