During training, the effective feature rank of a network's last hidden layer grows in a staircase pattern alongside loss drops, and initializing for high rank accelerates training.
Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equations
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
$\epsilon$-rank and the Staircase Phenomenon: New Insights into Neural Network Training Dynamics
During training, the effective feature rank of a network's last hidden layer grows in a staircase pattern alongside loss drops, and initializing for high rank accelerates training.