SGD on neural network weights induces a BBP phase transition that detaches signal eigenvalues from the random bulk, yielding an analytically solvable phase diagram for trainability in a linear teacher-student model.
Bbp phase transition for an extensive number of outliers
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
Random-matrix theory helps disentangle signal from noise in large data sets. We analyze rectangular $p \times q$ matrices $W = W_0 + M$ in which the noise $M$ generates a Marchenko-Pastur bulk, whereas the signal $W_0$ injects an extensive set of degenerate singular values. Keeping $\mathrm{rank}$ $W_0/q$ finite as $p,q \to \infty$, we show that the trace of the resolvent of $W^{\top} W$ obeys a quartic equation for one degenerate signal, yielding an exact spectral density, and derive explicit asymptotics in the strong-signal regime. We map out a detailed generalized Baik-Ben Arous-P\'ech\'e (BBP) phase diagram and clarify how a finite density of spikes reshapes the bulk edges. We further derive a $1/3$-scaling law for the critical signal strength in terms of the rank ratio for rectangular matrices in the finite-to-extensive-rank crossover. Numerical simulations validate the theory and illustrate its relevance for high-dimensional inference tasks with multiple degenerate signals and more general signal distributions.
citation-role summary
background 1
citation-polarity summary
fields
cond-mat.dis-nn 2years
2026 2verdicts
UNVERDICTED 2roles
background 1polarities
background 1representative citing papers
A two-level DMFT tracks bulk and outlier spectral dynamics in wide networks, predicting width-consistent outlier growth and hyperparameter transfer under muP scaling for deep linear nets while noting bulk restructuring for large-output tasks.
citing papers explorer
-
Spectral phase transitions and trainability in neural network learning dynamics
SGD on neural network weights induces a BBP phase transition that detaches signal eigenvalues from the random bulk, yielding an analytically solvable phase diagram for trainability in a linear teacher-student model.
-
Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
A two-level DMFT tracks bulk and outlier spectral dynamics in wide networks, predicting width-consistent outlier growth and hyperparameter transfer under muP scaling for deep linear nets while noting bulk restructuring for large-output tasks.