REVIEW 5 cited by
Universality for the global spectrum of random inner-product kernel matrices in the polynomial regime
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
We consider certain large random matrices, called random inner-product kernel matrices, which are essentially given by a nonlinear function $f$ applied entrywise to a sample-covariance matrix, $f(X^TX)$, where $X \in \mathbb{R}^{d \times N}$ is random and normalized in such a way that $f$ typically has order-one arguments. We work in the polynomial regime, where $N \asymp d^\ell$ for some $\ell > 0$, not just the linear regime where $\ell = 1$. Earlier work by various authors showed that, when the columns of $X$ are either uniform on the sphere or standard Gaussian vectors, and when $\ell$ is an integer (the linear regime $\ell = 1$ is particularly well-studied), the bulk eigenvalues of such matrices behave in a simple way: They are asymptotically given by the free convolution of the semicircular and Mar\v{c}enko-Pastur distributions, with relative weights given by expanding $f$ in the Hermite basis. In this paper, we show that this phenomenon is universal, holding as soon as $X$ has i.i.d. entries with all finite moments. In the case of non-integer $\ell$, the Mar\v{c}enko-Pastur term disappears (its weight in the free convolution vanishes), and the spectrum is just semicircular.
Forward citations
Cited by 5 Pith papers
-
On the edge eigenvalues of sparse random geometric graphs
For Gaussian-sampled random geometric graphs in the sparse regime, the first nontrivial edge eigenvalues of the scaled random-walk Laplacian converge in probability to 2(k-1)/sigma^2-type eigenvalues of a weighted Lap...
-
Spectra of high-dimensional sparse random geometric graphs
Under mild dimension conditions, the empirical spectral distribution of sparse high-dimensional random geometric graphs matches the semicircle law or the Erdős-Renyi limit.
-
Marchenko-Pastur law for tensor powers of exchangeable unconditional vectors
For exchangeable and sign-symmetric base vectors, the empirical spectrum of sample covariance matrices of their d-fold tensor powers converges almost surely to the Marchenko-Pastur law.
-
Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling
The limiting eigenvalue distribution of the two-layer NTK in the quadratic scaling n/(dp) tends to a Marchenko-Pastur map applied to a deterministic measure depending on the activation and output weights.
-
Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws
For a high-dimensional Gaussian sequence model, empirical risk minimization in single-head tied attention has exactly computable test error, interpolation and recovery thresholds, and a singular-value spectrum that be...
Discussion (0). Continue with ORCID to comment.