A per-parameter decomposition of training loss change shows that learning is noisy, with only about half of parameters helping per step, some layers hurting overall, and learning spikes synchronized across layers.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
LCA: Loss Change Allocation for Neural Network Training
A per-parameter decomposition of training loss change shows that learning is noisy, with only about half of parameters helping per step, some layers hurting overall, and learning spikes synchronized across layers.