Residual connections align cross-layer gradients while symmetry-breaking activations prevent rotational drift, causing principal singular vectors of adjacent layers to align.
Feature learning as align- ment: a structural property of gradient descent in non-linear neural networks
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.LG 2years
2026 2representative citing papers
Derives three-force decomposition of squared weight norm under AdamW and validates it on Pythia-70M models, plus spline recovery of alignment force from checkpoints.
citing papers explorer
-
Why Geometric Continuity Emerges in Deep Neural Networks: Residual Connections and Rotational Symmetry Breaking
Residual connections align cross-layer gradients while symmetry-breaking activations prevent rotational drift, causing principal singular vectors of adjacent layers to align.
-
Weibull Weight-Scale Parameter Evolution under AdamW Training Dynamics
Derives three-force decomposition of squared weight norm under AdamW and validates it on Pythia-70M models, plus spline recovery of alignment force from checkpoints.