Estimation of Population Linear Spectral Statistics by Marchenko--Pastur Inversion

Ben Deitmar

arxiv: 2504.03390 · v6 · pith:DB46JPYFnew · submitted 2025-04-04 · 🧮 math.ST · stat.TH

Estimation of Population Linear Spectral Statistics by Marchenko--Pastur Inversion

Ben Deitmar This is my paper

Pith reviewed 2026-05-22 21:31 UTC · model grok-4.3

classification 🧮 math.ST stat.TH

keywords linear spectral statisticsMarchenko-Pastur lawhigh-dimensional estimationnonparametric statisticscovariance estimationrandom matrix theory

0 comments

The pith

Inverting the Marchenko-Pastur law produces estimators for population linear spectral statistics with rate O(n^{ε-1}) when dimension grows with sample size.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a method to estimate linear spectral statistics of the population covariance matrix by inverting the Marchenko-Pastur law applied to the sample covariance. In the regime where dimension d and sample size n satisfy d/n approaching a positive constant, this approach achieves a convergence rate of O(n^{ε-1}) for any ε>0 in a fully nonparametric setting. This rate is claimed to be the first proven of its kind for such general conditions. For Gaussian observations, the method further admits a central limit theorem with n-rate scaling. A reader would care because accurate estimation of spectral properties is key to understanding high-dimensional data structures like covariances.

Core claim

The paper introduces an inversion technique based on the Marchenko-Pastur law to estimate population linear spectral statistics from sample data. When d/n → c >0, this estimator achieves convergence rate O(n^{ε-1}) for any ε>0 in general nonparametric settings, and for Gaussian data it satisfies a CLT with normalization factor n.

What carries the argument

Marchenko-Pastur inversion: recovering population spectral functionals by inverting the integral relation given by the limiting eigenvalue distribution of the sample covariance.

If this is right

The estimator converges faster than previous methods in high-dimensional nonparametric cases.
It enables consistent recovery of population traces of functions of the covariance matrix.
For Gaussian data the n-scaled error is asymptotically normal, permitting inference.
The approach requires only the validity of the Marchenko-Pastur limit rather than parametric assumptions on the distribution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The inversion technique could be adapted to other random-matrix limiting laws that arise under different dependence structures.
It may improve downstream tasks such as high-dimensional principal component analysis when sample and dimension sizes are comparable.
Testing the procedure on non-Gaussian heavy-tailed data would reveal whether the central limit theorem extends beyond the Gaussian case.

Load-bearing premise

The high-dimensional regime d/n → c >0 must hold and the data-generating process must satisfy the conditions for the Marchenko-Pastur law to apply in a nonparametric setting.

What would settle it

A sequence of simulations or datasets with d/n → c >0 where the estimation error for a linear spectral statistic fails to decay at rate n^{ε-1} for small ε>0 would falsify the convergence claim.

read the original abstract

A new method of estimating population linear spectral statistics from high-dimensional data is introduced. When the dimension $d$ grows with the sample size $n$ such that $\frac{d}{n} \to c>0$, the proposed method is the first with proven convergence rate of $\mathcal{O}(n^{\varepsilon - 1})$ for any $\varepsilon > 0$ in a general nonparametric setting. For Gaussian data, a CLT for the estimation error with normalization factor $n$ is shown.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper claims the first near-parametric rate for nonparametric linear spectral statistics estimation via MP inversion in the d/n→c regime, but the uniform rate likely requires unstated eigenvalue support bounds that standard inversion arguments need.

read the letter

The new contribution is an estimator for population linear spectral statistics that inverts the Marchenko-Pastur map applied to the empirical spectral measure. It targets the high-dimensional regime where d/n converges to a positive constant and reports an O(n^{ε-1}) rate that holds uniformly over a nonparametric class, plus a central limit theorem at rate n when the data are Gaussian. That rate is the headline result and would be useful if it survives the details, since most existing work either assumes more structure on the spectrum or settles for slower rates in fully nonparametric settings. The approach is direct and avoids heavy parametric modeling, which is a plus for the subfield. The main soft spot is exactly the one flagged in the stress test. Standard bounds on the stability of the MP inversion operator require the population spectral measure to stay away from zero and infinity; without those restrictions the error terms do not decay at the claimed speed uniformly. The abstract gives no indication of truncation, regularization, or explicit support conditions that would restore the rate, so the nonparametric claim rests on whether the paper quietly imposes those conditions or proves a stronger result. The Gaussian CLT is narrower but at least standard. Overall the work is aimed at readers already comfortable with random matrix techniques in statistics who want sharper rates for spectral functionals. The math looks like it could be checked in a referee report, and the claim is specific enough that a serious editor should send it out rather than desk-reject. I would bring it to a reading group to see the proofs on the inversion error.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces an estimator for population linear spectral statistics (LSS) obtained by inverting the Marchenko-Pastur map applied to the empirical spectral measure of the sample covariance. In the regime d/n → c > 0 it claims the first convergence rate of O(n^{ε−1}) (any ε>0) that holds uniformly over a general nonparametric class of population spectral measures, together with a CLT at rate n for Gaussian observations.

Significance. If the uniform rate and CLT are valid under the stated nonparametric conditions, the work would supply the first nearly parametric rate for this class of functionals in high dimensions without parametric restrictions on the spectrum, which is a notable technical achievement.

major comments (2)

[main convergence theorem / rate statement] The claimed O(n^{ε−1}) rate in a fully general nonparametric setting (no support restrictions) rests on stability of the MP inversion operator. Standard arguments for such stability require the population measure to be supported away from 0 and ∞; the manuscript should identify the precise section or theorem where this is relaxed or where truncation/regularization is shown not to degrade the rate.
[CLT section] The CLT is stated only for Gaussian data. The manuscript should clarify whether the same normalization n remains valid under the weaker moment conditions used for the rate result, or whether the Gaussian assumption is essential for the CLT.

minor comments (1)

[introduction / notation] Notation for the inversion operator and the class of admissible measures should be introduced earlier and used consistently.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. We address the two major points below and will revise the manuscript to improve clarity on these technical aspects.

read point-by-point responses

Referee: [main convergence theorem / rate statement] The claimed O(n^{ε−1}) rate in a fully general nonparametric setting (no support restrictions) rests on stability of the MP inversion operator. Standard arguments for such stability require the population measure to be supported away from 0 and ∞; the manuscript should identify the precise section or theorem where this is relaxed or where truncation/regularization is shown not to degrade the rate.

Authors: The uniform stability of the Marchenko-Pastur inversion operator over the nonparametric class is established in Proposition 2.4, which requires only moment bounds rather than explicit support restrictions. In the proof of the main rate result (Theorem 3.1), we introduce a truncation of the empirical spectral measure at levels n^{-α} and n^α (with α small) whose contribution is controlled by the tail bounds available under the nonparametric assumptions; the truncation error is shown to be o(n^{-1+ε}) uniformly, so that the inversion stability applies to the truncated measure without rate degradation. We will add an explicit remark after Theorem 3.1 referencing this truncation argument to make the relaxation transparent. revision: yes
Referee: [CLT section] The CLT is stated only for Gaussian data. The manuscript should clarify whether the same normalization n remains valid under the weaker moment conditions used for the rate result, or whether the Gaussian assumption is essential for the CLT.

Authors: The CLT in Theorem 4.1 is proved under Gaussianity because the argument relies on the exact joint law of the sample eigenvalues (via the Wishart ensemble) to obtain the limiting variance and the n-rate normalization. Under the weaker (4+δ)-moment conditions sufficient for the O(n^{ε-1}) rate, the same normalization does not necessarily produce a non-degenerate Gaussian limit, and the current proof technique does not extend. We will revise the discussion following Theorem 4.1 to state explicitly that the Gaussian assumption is essential for the n-rate CLT while the convergence rate holds more generally. revision: yes

Circularity Check

0 steps flagged

Derivation is self-contained; no reduction to inputs by construction.

full rationale

The paper presents a new MP-inversion estimator for population linear spectral statistics and states a convergence rate result under the high-dimensional regime. No quoted step equates the claimed O(n^{ε-1}) rate or the estimator itself to a fitted parameter or prior self-citation by definition. The central claim rests on stability properties of the inversion map applied to the empirical spectral measure, which is an independent analytic argument rather than a renaming or tautological fit. Self-citations, if present, are not load-bearing for the rate proof itself.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no information on free parameters, axioms, or invented entities; full text required for ledger.

pith-pipeline@v0.9.0 · 5596 in / 983 out tokens · 41068 ms · 2026-05-22T21:31:37.362796+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel (J-cost uniqueness) unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Marchenko–Pastur equation ... s = ∫ λ / (λ(1−c z s −c)−z) dH(λ) (Lemma 1.1); inversion via φ_ν,c(z) = −1/s_ν(z) and admissible curves γ_n for Cauchy formula (Cor. 2.9)
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

rate O(n^{ε−1}) for holomorphic g on data-driven domain D̂(τ,κ,n) (Thm 2.5)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.