REVIEW 6 cited by
Estimating the Spectral Density of Large Implicit Matrices
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Many important problems are characterized by the eigenvalues of a large matrix. For example, the difficulty of many optimization problems, such as those arising from the fitting of large models in statistics and machine learning, can be investigated via the spectrum of the Hessian of the empirical loss function. Network data can be understood via the eigenstructure of a graph Laplacian matrix using spectral graph theory. Quantum simulations and other many-body problems are often characterized via the eigenvalues of the solution space, as are various dynamic systems. However, naive eigenvalue estimation is computationally expensive even when the matrix can be represented; in many of these situations the matrix is so large as to only be available implicitly via products with vectors. Even worse, one may only have noisy estimates of such matrix vector products. In this work, we combine several different techniques for randomized estimation and show that it is possible to construct unbiased estimators to answer a broad class of questions about the spectra of such implicit matrices, even in the presence of noise. We validate these methods on large-scale problems in which graph theory and random matrix theory provide ground truth.
Forward citations
Cited by 6 Pith papers
-
A Defense of the Quadratic Model
Local Taylor-expanded quadratic models reproduce a 150M-parameter LLM's validation loss for up to 10% of training late in the run, and LLM pretraining operates within a factor of 2 of a stochastic or deterministic edg...
-
Spectral density estimation for normal matrices
Algorithm approximates spectral density of normal matrices to EMD O(1/m + log m/sqrt(n)) with m matvec queries and proves Omega(1/epsilon) lower bound even for symmetric case.
-
pop-cosmos: Disentangling galaxy properties from observables using data-driven approaches
Rest-frame optical galaxy SEDs from a 16-parameter SPS model are captured by five disentangled VAE latents (mass, young stars, dust, soft/hard ionization); metallicity and age are not independent drivers.
-
It's all In the (Exponential) Family: An Equivalence between Maximum Likelihood Estimation and Control Variates for Sketching Algorithms
In exponential families where the target parameters equal the mean parameters, optimal control variate estimators and maximum likelihood estimators have identical asymptotic variance, giving a fixed-point algorithm fo...
-
Spectral Estimation with Free Decompression
Free decompression evolves a small submatrix spectrum into an estimate of a large matrix spectrum using a PDE derived from free probability, the Nica-Speicher free compression theorem.
-
pop-cosmos: Galaxy size evolution across structural and star-formation classifications in COSMOS-Web
Galaxy size-mass relations exhibit double power-law breaks at different pivot masses for quiescent versus bulge-dominated samples, coinciding with AGN activity scales.
Discussion (0). Continue with ORCID to comment.