Rigorous proofs establish convergence of gradient descent for a fixed-bias ReLU network in 1D along with spectral bias, and introduce the full-wave rectified exponential (FReX) activation with similar convergence properties.
On the spectral bias of neural networks
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Mathematical analysis of one-layer neural network with fixed biases, a new activation function and other observations
Rigorous proofs establish convergence of gradient descent for a fixed-bias ReLU network in 1D along with spectral bias, and introduce the full-wave rectified exponential (FReX) activation with similar convergence properties.