At initialization, recentering plus ReLU in a batch-normalized network drives representations toward a single cluster plus one outlier in an orthogonal direction, with partial theoretical support.
cc/paper/2009/file/5751ec3e9a4feab575962e78e006250d-Paper.pdf
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Batch Normalization Decomposed
At initialization, recentering plus ReLU in a batch-normalized network drives representations toward a single cluster plus one outlier in an orthogonal direction, with partial theoretical support.