Using the unrestricted feature model with MSE loss, the work shows that bias regularization drives one-hot class means from simplex ETF to orthogonal frame and that the classifier bias centers arbitrary label encodings.
Cross entropy versus label smoothing: A neural collapse perspective.arXiv preprint arXiv:2402.03979
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.LG 2verdicts
UNVERDICTED 2representative citing papers
Neural regression collapse occurs when last-layer feature intrinsic dimension falls below target intrinsic dimension, creating over-compressed and under-compressed regimes that govern generalization based on data quantity and noise.
citing papers explorer
-
The role of class encoding in neural collapse
Using the unrestricted feature model with MSE loss, the work shows that bias regularization drives one-hot class means from simplex ETF to orthogonal frame and that the classifier bias centers arbitrary label encodings.
-
Geometric Analysis of Neural Regression Collapse via Intrinsic Dimension
Neural regression collapse occurs when last-layer feature intrinsic dimension falls below target intrinsic dimension, creating over-compressed and under-compressed regimes that govern generalization based on data quantity and noise.