REVIEW 7 cited by
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Neural Collapse: A Review on Modelling Principles and Generalization
read the original abstract
Deep classifier neural networks enter the terminal phase of training (TPT) when training error reaches zero and tend to exhibit intriguing Neural Collapse (NC) properties. Neural collapse essentially represents a state at which the within-class variability of final hidden layer outputs is infinitesimally small and their class means form a simplex equiangular tight frame. This simplifies the last layer behaviour to that of a nearest-class center decision rule. Despite the simplicity of this state, the dynamics and implications of reaching it are yet to be fully understood. In this work, we review the principles which aid in modelling neural collapse, followed by the implications of this state on generalization and transfer learning capabilities of neural networks. Finally, we conclude by discussing potential avenues and directions for future research.
Forward citations
Cited by 7 Pith papers
-
Optimal Representations for Generalized Contrastive Learning with Imbalanced Datasets
In generalized contrastive learning with imbalanced classes, optimal representations collapse to class means whose angular geometry is determined by class proportions via convex optimization, and extreme imbalance cau...
-
How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse
In multi-label neural collapse, terminal geometry is controlled by the centered label covariance spectrum κ_m derived from label distribution moments, with higher-multiplicity prototypes following class-frequency-weig...
-
Learning from almost nothing: How neural networks survive heavy input corruption
Infinite-width MLPs implement a nearest-class-mean prototype classifier as their leading-order decision rule under heavy attribute noise, explaining observed robustness in experiments.
-
Superposition Yields Robust Neural Scaling
Strong superposition causes neural loss to scale as the inverse of model dimension due to geometric feature overlaps, explaining scaling laws for broad frequency distributions.
-
Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment
A two-stage alignment framework that first fuses visual modalities (RGB, flow, skeleton) then introduces text, achieving 21% SRCC improvement on a new clinical AQA dataset and gains on two public benchmarks.
-
Manifold-Constrained Adversarial Training for Long-Tailed Robustness via Geometric Alignment
MCAT improves adversarial robustness on long-tailed datasets by constraining examples to class manifolds and enforcing balanced geometric separation.
-
There Will Be a Scientific Theory of Deep Learning
A mechanics of the learning process is emerging in deep learning theory, characterized by dynamics, coarse statistics, and falsifiable predictions across idealized settings, limits, laws, hyperparameters, and universa...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.