At initialization, recentering plus ReLU in a batch-normalized network drives representations toward a single cluster plus one outlier in an orthogonal direction, with partial theoretical support.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Batch Normalization Decomposed
At initialization, recentering plus ReLU in a batch-normalized network drives representations toward a single cluster plus one outlier in an orthogonal direction, with partial theoretical support.