Derives exact Frobenius norm imbalance identity for deep nonlinear networks, classifies activations into four classes, and obtains critical-depth escape time law τ★ = Θ(ε^{-(r-2)}) from reduction to scalar ODE on permutation-symmetric submanifold.
hub
On the Spectral Bias of Neural Networks
10 Pith papers cite this work. Polarity classification is still indexing.
10
Pith papers citing it
abstract
Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with $100\%$ accuracy. In this work, we present properties of neural networks that complement this aspect of expressivity. By using tools from Fourier analysis, we show that deep ReLU networks are biased towards low frequency functions, meaning that they cannot have local fluctuations without affecting their global behavior. Intuitively, this property is in line with the observation that over-parameterized networks find simple patterns that generalize across data samples. We also investigate how the shape of the data manifold affects expressivity by showing evidence that learning high frequencies gets \emph{easier} with increasing manifold complexity, and present a theoretical understanding of this behavior. Finally, we study the robustness of the frequency components with respect to parameter perturbation, to develop the intuition that the parameters must be finely tuned to express high frequency functions.
hub tools
citation-role summary
background 3
citation-polarity summary
roles
background 3polarities
background 3representative citing papers
Neural networks optimized solely on crossing symmetry reconstruct CFT correlators from minimal input data to few-percent accuracy across generalized free fields, minimal models, Ising, N=4 SYM, and AdS diagrams.
A variational physics-informed neural network solves higher-order anisotropic phase-field fracture models by minimizing total energy with B-spline enriched trial functions.
A hybrid neural policy operating in impulse space enables physics-based characters to track exaggerated, dynamically infeasible motions that standard DRL methods cannot stabilize.
MoMo conditions contrastive representations and prediction operators on user preferences via FiLM and low-rank modulation to enable continuous modulation of plan safety while preserving inference efficiency.
A new scale-aware diagnostic framework shows that unconstrained diffusion generative models exhibit structural freezing and instability instead of smooth physical responses under multiscale perturbations.
Neural networks trained on crossing symmetry accurately reconstruct conformal correlators from minimal inputs due to alignment between their spectral bias and CFT smoothness.
FEDONet augments DeepONet with Fourier-embedded trunk networks using random Fourier features, yielding lower L2 reconstruction errors than standard DeepONet on Burgers', 2D Poisson, Eikonal, Allen-Cahn, and Kuramoto-Sivashinsky equations across dataset sizes and noise levels.
Beta Sampling uses spectral analysis to select critical denoising steps in diffusion models, outperforming uniform sampling on FID and IS metrics.
citing papers explorer
-
A Theory of Saddle Escape in Deep Nonlinear Networks
Derives exact Frobenius norm imbalance identity for deep nonlinear networks, classifies activations into four classes, and obtains critical-depth escape time law τ★ = Θ(ε^{-(r-2)}) from reduction to scalar ODE on permutation-symmetric submanifold.
-
Neural Spectral Bias and Conformal Correlators I: Introduction and Applications
Neural networks optimized solely on crossing symmetry reconstruct CFT correlators from minimal input data to few-percent accuracy across generalized free fields, minimal models, Ising, N=4 SYM, and AdS diagrams.
-
Deep learning-based phase-field modelling of brittle fracture in anisotropic media
A variational physics-informed neural network solves higher-order anisotropic phase-field fracture models by minimizing total energy with B-spline enriched trial functions.
-
Neural Assistive Impulses: Synthesizing Exaggerated Motions for Physics-based Characters
A hybrid neural policy operating in impulse space enables physics-based characters to track exaggerated, dynamically infeasible motions that standard DRL methods cannot stabilize.
-
MoMo: Conditioned Contrastive Representation Learning for Preference-Modulated Planning
MoMo conditions contrastive representations and prediction operators on user preferences via FiLM and low-rank modulation to enable continuous modulation of plan safety while preserving inference efficiency.
-
Scale-Aware Adversarial Analysis: A Diagnostic for Generative AI in Multiscale Complex Systems
A new scale-aware diagnostic framework shows that unconstrained diffusion generative models exhibit structural freezing and instability instead of smooth physical responses under multiscale perturbations.
-
Neural Networks Reveal a Universal Bias in Conformal Correlators
Neural networks trained on crossing symmetry accurately reconstruct conformal correlators from minimal inputs due to alignment between their spectral bias and CFT smoothness.
-
FEDONet : Fourier-Embedded DeepONet for Spectrally Accurate Operator Learning
FEDONet augments DeepONet with Fourier-embedded trunk networks using random Fourier features, yielding lower L2 reconstruction errors than standard DeepONet on Burgers', 2D Poisson, Eikonal, Allen-Cahn, and Kuramoto-Sivashinsky equations across dataset sizes and noise levels.
-
Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysis
Beta Sampling uses spectral analysis to select critical denoising steps in diffusion models, outperforming uniform sampling on FID and IS metrics.
- Theory of the Frequency Principle for General Deep Neural Networks