Weight-decay-regularized two-layer ReLU networks need width exponential in the number of samples for a benign loss landscape, and small initialization can still converge to spurious minima.
Early alignment in two-layer networks training is a two-edged sword 10.48550/arxiv.2401.10791
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
extension 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
extension 1polarities
extend 1representative citing papers
citing papers explorer
-
Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization
Weight-decay-regularized two-layer ReLU networks need width exponential in the number of samples for a benign loss landscape, and small initialization can still converge to spurious minima.