REVIEW 60 references
Estimating eigenvectors and eigenspaces of covariance matrices: Optimal Bounds and Conditions for Consistency
T0 review · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read The paper proves matching upper and lower bounds for the error of sample eigenvectors and eigenspaces, and shows the error's order is governed by the effective rank and the signal-to-gap ratio rather than the ambient dimension d.
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the pair (r_eff, λ_p/δ(p)): the effective rank (sum of eigenvalues over the largest) and the signal-to-gap ratio (the p-th eigenvalue divided by the distance to its nearest neighbour). The argument uses a contour integral representation of the difference between the two resolvents (zI-M)^{-1} and (zI-M̃)^{-1}, expands the integrand in powers of the noise matrix E = M̃ - M, and groups the many resulting terms with a combinatorial profile scheme. Two variance parameters carry the dominant lower bound: VarS(p,γ_p), the variance of the sample cross-term (u_p^T Y)(Y^T u_γp), and s_p, the weighted average of the variances of the entries (u_p^T Y)(Y^T u_i).
What would settle it
Take d large, M with a clear spike λ_1=1 and all other eigenvalues small, and set Y to have iid Rademacher entries with the leading eigenvector u_1 aligned with a coordinate axis. Then VarS(1,2)=0, so the lower-bound term in Theorem 2.5 vanishes. Simulate n large and measure ‖ũ_1ũ_1^T - u_1u_1^T‖. If the error is asymptotically much smaller than λ_1/(√n δ(1)) in the small-effective-rank regime, the claimed matching lower bound (with a universal constant independent of the eigenvector alignment) is false. Conversely, with Gaussian entries the predicted rate should appear.
Extended reading notes
Core claim
The central claim is that the estimation error for the p-th eigenvector projector and the p-dimensional eigenspace projector is determined, up to a constant factor, by the ratio of the effective rank r_eff = tr(M)/λ_1 and the signal-to-gap ratio λ_p/δ(p), where δ(p) is the distance from λ_p to the nearest eigenvalue. In the small-effective-rank regime the error is of order min{λ_p/(√n δ(p)), 1}; in the medium regime it is of order λ_p/(√n δ(p)); in the large regime it is of order √(r_eff/n). The paper further claims necessary and sufficient conditions for consistency: the estimator is consistent exactly when the relevant error quantity tends to 0, and this holds precisely when n grows with r
Load-bearing premise
The matching lower bound and the 'if and only if' consistency claims rely on the variance quantities VarS(p,γ_p) and s_p being bounded below by a positive constant; if the entry distribution and eigenvector alignment make these variances zero or vanishing, the lower-bound theorem and the necessity part of the consistency conditions can fail.
Editorial extensions
If this is right
- If the effective rank is small (r_eff ≤ r λ_p/δ(p)), the error never exceeds a constant, and consistency is equivalent to λ_p/(√n δ(p)) → 0; no dependence on d appears.
- If the effective rank is large (r_eff > r (λ_p/δ(p))²), the gap between eigenvalues stops mattering and the error is √(r_eff/n); consistency is equivalent to n ≫ r_eff.
- In the medium regime, error and consistency both hinge on λ_p/(√n δ(p)), but only after n ≥ C r_eff λ_p/δ(p).
- A single set of conditions covers every index p, including p growing with d, and eigenspaces of dimension p, so the theory is not restricted to the leading eigenvector.
- Matching lower bounds imply that no perturbation-based upper bound can be improved by more than a constant factor in the covered regimes.
Reading between the lines
- The paper's lower-bound theorems assume the variance terms VarS(p,γ_p) and s_p are bounded away from zero; if a specific factor distribution and eigenvector alignment made them vanish (e.g., Rademacher entries aligned with coordinate axes), the 'if and only if' consistency conditions would likely fail, and the true error could be far smaller—so the sharp transition is distribution-dependent, not u
- One practical reading: the quantity r_eff·λ_p/δ(p) is the effective sample size threshold below which consistent eigenvector estimation is impossible; practitioners could estimate this threshold from data and check whether their n is above it.
- Because the deterministic core of the proof only needs bounds on the noise norm and skewness parameters, the same order-of-magnitude results should transfer to missing entries, additive noise, and heavy-tailed inputs with bounded fourth moments—an extension the paper states but leaves for later work.
- A testable extension of the phase transition: in a spiked model with known λ_p and δ(p), the observed error should drop as λ_p/(√n δ(p)) until n crosses C r_eff λ_p/δ(p), then switch to the √(r_eff/n) rate; measuring this kink would verify the predicted threshold.
Editorial analysis
A structured set of objections, weighed in public.
Circularity Check
Central F_s expansions are imported from the authors' own unpublished preprint [52]; no fitted/definitional circularity, but the derivation chain is not self-contained.
-
self citation load bearing
[Section 6.3.1 and Section 6.3.3 (estimates (19), (23), (28); Lemmas 6.1–6.3)]
"Next, using the explicit computation of F1 from [52, Section 7.2, pp. 32–33], we obtain ... By [52, Section 7.2 - Estimates (50)(51)(52)], we further have ∥M1∥ ≤2√rx/δ(p) ... In [52, Lemmas 7.2-7.4], Tran and Vu proved the following lemmas, bounding ∥R Γ M(α;β)dz∥ with respect to the above three types."
The proofs of Theorems 2.4 and 2.5 reduce the eigenvector error to the sum of F_s and then bound F_1, F_2, and F_s (s≥3) using explicit computations and Lemmas 7.2–7.4 taken verbatim from [52]. These estimates are not proved in the present paper; they are the core technical mechanism that produces the claimed matching upper and lower bounds. Since [52] is an unpublished preprint by the same two authors and is not replaced here by an independent derivation, the central load-bearing step of the derivation chain is a self-citation whose correctness is not independently established in this text.
full rationale
There is no parameter-fitting or definitional circularity: the bounds are expressed directly in terms of population quantities (λ_p, δ(p), r_eff, VarS, s_p) and are not fitted to the data, and the lower-bound argument uses the CLT and Paley–Zygmund rather than assuming the target error. The main non-circular correctness concern is that the matching lower bounds and the consistency 'iff' statements require VarS(p,γp) and s_p to be bounded below by a positive constant; this is stated only in Section 4 ('we assume that (5) holds and VarS(p,γp), sp ≥ c'), not in Theorems 1.1–1.6, 2.1–2.3, or 3.1–3.3 and Corollaries 1.1–1.3. With Rademacher coordinates and suitably aligned eigenvectors, VarS can vanish, so those lower bounds need not hold. That is a missing assumption / correctness risk, not a circular step. The reason the score is 4 rather than lower is that the paper leans on the authors' own unpublished preprint [52] for the decisive F_s estimates, making the derivation chain dependent on a load-bearing self-citation. The central claim still has independent content beyond [52]—the covariance-specific variance computations, the three-regime analysis, and the consistency corollaries—so this is not a 6+ 'reduces by construction' circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption Y has 8-wise independent sub-Gaussian entries with mean 0, variance 1, and ∥y_i∥_{ψ_2} ≤ K, with K,r,κ_p = O(1).
- ad hoc to paper The combinatorial expansion bounds from the authors' earlier work (Tran–Vu, arXiv:2409.20207, [52]) — Lemmas 6.1–6.3 and the estimates of F_1, F_2, F_s in Section 6.3 — are correct.
- ad hoc to paper The variance parameters VarS(p,γ_p) and s_p are bounded away from zero (positive lower bound c>0).
- standard math The matrix concentration bound of Koltchinskii–Lounici [28] and Zhivotovskiy [59]: with probability ≥1−e^{−t}, ∥M̃−M∥ ≤ 20Kλ1√((4r_eff+t)/n) for n ≥ 4r_eff+t.
Cite this review
Pith. "Pith review of Estimating eigenvectors and eigenspaces of covariance matrices: Optimal Bounds and Conditions for Consistency." pith.science (2026). https://pith.science/paper/SPJQRA36
@misc{pith2026260723964,
author = {Pith},
title = {Pith review of: Estimating eigenvectors and eigenspaces of covariance matrices: Optimal Bounds and Conditions for Consistency},
year = {2026},
howpublished = {\url{https://pith.science/paper/SPJQRA36}},
note = {Machine review of arXiv:2607.23964}
}
abstract
Let $X = [ \xi_1, \,\, \xi_2,...\,\, ,\xi_d]^\top$ be a zero-mean random vector of large dimension $d$ ($d \rightarrow \infty$) with (hidden) covariance matrix $M = (m_{ij})_{1 \leq i, j \leq d},$ where $m_{ij} = m_{ji} = \textbf{Cov}(\xi_i, \xi_j).$ Let $X_1, X_2, \dots, X_n$ be $n$ iid samples of $X$. Consider the sample covariance matrix $$\textstyle \tilde{M} := \frac{1}{n} \sum_{i=1}^{n} X_i X_i^\top.$$ In practice, one frequently uses the eigenvectors and eigenspaces of $\tilde M$ as estimators for those of $M$. A central task is to provide an error analysis for these estimators. In this paper, we provide an optimal error analysis, obtaining upper and lower bounds of matching order of magnitude, for a wide range of parameters $d$ and $n$, under mild assumptions on $M$. As corollaries, we obtain new necessary and sufficient conditions for the consistency of the estimators. In these conditions, we only require the number of samples $n$ to depend linearly on the effective rank of $M$, which can be much smaller than the dimension $d$.
Reference graph
Works this paper leans on
-
[1]
Adamczak, A
R. Adamczak, A. Litvak, A. Pajor, and N. Tomczak-Jaegermann,Quantitative estimates of the conver- gence of the empirical covariance matrix in log-concave ensembles, Journal of the American Mathematical Society23(2010), no. 2, 535–561
2010
-
[2]
Agarwal, K
R.P. Agarwal, K. Perera, and S. Pinelas,An introduction to complex analysis, Springer Science & Business Media, 2011. 32 PHUC TRAN, V AN VU
2011
-
[3]
A.Mehrotra, P. Tran, V. Vu, and M. Zampetakis,Improved guarantees for heterogeneous treatment-effect estimation via matrix completion, arXiv preprint: 2605.30319 (2026)
arXiv 2026
-
[4]
Anderson,An introduction to multivariate statistical analysis, Wiley, Hoboken, NJ, third edition, 2003
T.W. Anderson,An introduction to multivariate statistical analysis, Wiley, Hoboken, NJ, third edition, 2003
2003
-
[5]
Bai and J
Z. Bai and J. Yao,Central limit theorems for eigenvalues in a spiked population model, Ann. Inst. Henri Poincare Probab. Stat.44(2008), no. 3, 447–474
2008
-
[6]
Multivariate Anal106(2012), 167–177
,On sample eigenvalues in a generalized spiked population model, J. Multivariate Anal106(2012), 167–177
2012
-
[7]
Z. Bai, J. Yao, and S. Zhang,Large sample covariance matrices and high-dimensional data analysis, Camb. Ser. Stat. Probab. Math., Cambridge University Press, 2015
2015
-
[8]
Baik and J
J. Baik and J. Silverstein,Eigenvalues of large sample covariance matrices of spiked population models, J. Multivariate Anal.97(2006), no. 6, 1382–1408
2006
Show all 60 references
-
[9]
Bhatia,Matrix analysis, Grad
R. Bhatia,Matrix analysis, Grad. Texts in Math., Springer, New York, NY, 2013
2013
-
[10]
Bloemendal, A
A. Bloemendal, A. Knowles, H. Yau, and J. Yin,On the principal components of sample covariance matrices, Probab. Theory and Related Fields164(2016), no. 1–2, 459–552
2016
-
[11]
Bunea and L
F. Bunea and L. Xiao,On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA, Bernoulli21(2015), no. 2, 1200–1230
2015
-
[12]
Y. Chen, C. Chen, and J. Fan,Asymmetry helps: Eigenvalue and eigenvector analyses of asymmetrically perturbed low-rank matrices, Ann. Statist.49(2021), no. 1, 435
2021
-
[13]
Dauxois, A
J. Dauxois, A. Pousse, and Y. Romain,Asymptotic theory for the principal component analysis of a vector random function: some applications to statistical inference, J. Multivariate Anal.12(1982), no. 1, 136– 154
1982
-
[14]
Fan and W
J. Fan and W. Wang,Asymptotics of empirical eigenstructure for high dimensional spiked covariance, Ann. Statist.45(2017), no. 3, 1342–1374
2017
-
[15]
Higham,Functions of matrices: Theory and computation, SIAM, 2008
N.J. Higham,Functions of matrices: Theory and computation, SIAM, 2008. See§1.3 and§7.6 for the Cauchy–Dunford integral
2008
-
[16]
Hotelling,Analysis of a complex of statistical variables into principal components., Journal of Educa- tional Psychology24(1933), no
H. Hotelling,Analysis of a complex of statistical variables into principal components., Journal of Educa- tional Psychology24(1933), no. 6, 417
1933
-
[17]
Jirak and M
M. Jirak and M. Wahl,Perturbation bounds for eigenspaces under a relative gap condition, Proc. Amer. Math. Soc.148(2020), no. 2, 479–494
2020
-
[18]
Math.412 (2023)
,Relative perturbation bounds with applications to empirical covariance operators, Adv. Math.412 (2023). Paper No. 108808, 59 pp
2023
-
[19]
Johnstone,On the distribution of the largest eigenvalue in principal components analysis, Ann
I. Johnstone,On the distribution of the largest eigenvalue in principal components analysis, Ann. Statist. 29(2001), no. 2, 295–327
2001
-
[20]
Johnstone and A
I. Johnstone and A. Lu,Sparse principal component analysis, arXiv (2009). arXiv:0901.4392
2009 arXiv
-
[21]
Johnstone and D
I. Johnstone and D. Paul,PCA in high dimensions: An orientation, Proceedings of the IEEE106(2018), no. 8, 1277–1292
2018
-
[22]
Jolliffe,Principal component analysis, International encyclopedia of statistical science, 2011, pp
I. Jolliffe,Principal component analysis, International encyclopedia of statistical science, 2011, pp. 1094– 1096
2011
-
[23]
Jung and J.S
S. Jung and J.S. Marron,PCA consistency in high dimension, low sample size context, Ann. Statist.37 (2009), no. 6B, 4104–4130
2009
-
[24]
El Karoui and A
N. El Karoui and A. d’Aspremont,Second order accurate distributed eigenvector computation for extremely large matrices, Electron. J. Stat.4(2010), no. 10, 1345–1385
2010
-
[25]
Kato,Perturbation theory for linear operators, Classics in Mathematics, Springer, 1980
T. Kato,Perturbation theory for linear operators, Classics in Mathematics, Springer, 1980
1980
-
[26]
Koltchinskii and E
V. Koltchinskii and E. Gin´ e,Random matrix approximation of spectra of integral operators, Bernoulli6 (2000), no. 1, 113–167
2000
-
[27]
Koltchinskii and K
V. Koltchinskii and K. Lounici,Asymptotics and concentration bounds for bilinear forms of spectral projectors of sample covariance, Ann. Inst. Henri Poincare Probab. Stat.52(2016), no. 4, 1976–2013
2016
-
[28]
1, 110–133
,Concentration inequalities and moment bounds for sample covariance operators, Bernoulli23 (2017), no. 1, 110–133
2017
-
[29]
Koltchinskii and D
V. Koltchinskii and D. Xia,Perturbation of linear forms of singular vectors under Gaussian noise, Progr. Probab.71(2016), 397–423
2016
-
[30]
Lorenz,Empirical orthogonal functions and statistical weather prediction(1956)
E.N. Lorenz,Empirical orthogonal functions and statistical weather prediction(1956)
1956
-
[31]
Martinsson and J
P-G. Martinsson and J. Tropp,Randomized numerical linear algebra: foundations & algorithms (2020) (2020). arXiv:2002.01387
2020 arXiv
-
[32]
Mas and L
A. Mas and L. Menneteau,Perturbation approach applied to the asymptotic study of random operators, High dimensional probability iii, 2003, pp. 127–134. ESTIMATING EIGENVECTORS AND EIGENSPACES OF COV ARIANCE MATRICES 33
2003
-
[33]
Mestre,On the asymptotic behavior of the sample estimates of eigenvalues and eigenvectors of covari- ance matrices, IEEE Transactions on Signal Processing56(2008), no
X. Mestre,On the asymptotic behavior of the sample estimates of eigenvalues and eigenvectors of covari- ance matrices, IEEE Transactions on Signal Processing56(2008), no. 11, 5353–5368
2008
-
[34]
Nadler,Finite sample approximation results for principal component analysis: A matrix perturbation approach, Ann
B. Nadler,Finite sample approximation results for principal component analysis: A matrix perturbation approach, Ann. Statist.36(2008), no. 6, 2791–2817
2008
-
[35]
O’Rourke, V
S. O’Rourke, V. Vu, and K. Wang,Random perturbation of low rank matrices: Improving classical bounds, Linear Algebra Appl.540(2018), no. 1, 26–59
2018
-
[36]
,Matrices with Gaussian noise: Optimal estimates for singular subspace perturbation, IEEE Trans. Inform. Theory70(2023), no. 3, 1978–2002
2023
-
[37]
Paul,Asymptotics of sample eigenstructure for a large dimensional spiked covariance model, Statist
D. Paul,Asymptotics of sample eigenstructure for a large dimensional spiked covariance model, Statist. Sinica17(2007), no. 4, 1617–1642
2007
-
[38]
Pearson,On lines and planes of closest fit to systems of points in space, The London, Edinburgh, and Dublin philosophical magazine and journal of science2(1901), no
K. Pearson,On lines and planes of closest fit to systems of points in space, The London, Edinburgh, and Dublin philosophical magazine and journal of science2(1901), no. 11, 559–572
1901
-
[39]
Rudelson and R
M. Rudelson and R. Vershynin,Sampling from large matrices: an approach through geometric functional analysis, J. ACM54(2007), no. 4, 21–es
2007
-
[40]
,Hanson-Wright inequality and sub-Gaussian concentration, Electron. Commun. Probab.18 (2013), no. 82, 1–9
2013
-
[41]
,Small ball probabilities for linear images of high-dimensional distributions, Int. Math. Res. Not. 2015(2015), no. 19, 9594–9617
2015
-
[42]
D. Shen, H. Shen, H. Zhu, and JS. Marron,Surprising asymptotic conical structure in critical sample eigen-directions, arXiv:1303.6171 (2013)
2013 arXiv
-
[43]
D. Shen, H. Shen, H. Zhu, and J.S. Marron,The statistics and mathematics of high dimension low sample size asymptotics, Statistica Sinica26(2016), no. 4, 1747
2016
-
[44]
Srivastava and R
N. Srivastava and R. Vershynin,The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics, Ann. Statist.5(2013), 3081–3111
2013
-
[45]
Stein and R
E. Stein and R. Shakarchi,Complex analysis, Princeton Lectures in Analysis II, Princeton University Press, 2003
2003
-
[46]
Tikhomirov,Sample covariance matrices of heavy-tailed distributions, Int
K. Tikhomirov,Sample covariance matrices of heavy-tailed distributions, Int. Math. Res. Not.2018 (2018), no. 20, 6254–6289
2018
-
[47]
Tran and V
L. Tran and V. Vu,Fast exact recovery of noisy matrix from few entries: the infinity norm approach, Proceedings of the 39th Conference on Neural Information Processing Systems, 2025. arXiv: 2501.19224v1
2025 arXiv
-
[48]
P. Tran, N. K. Vishnoi, and V. Vu,Spectral perturbation bounds for low-rank approximation with appli- cations to privacy, Proceedings of the 39th Conference on Neural Information Processing Systems, 2025. arXiv: 2510.25670
2025
-
[49]
Tran and V
P. Tran and V. Vu,New perturbation bounds for low rank approximation of matrices: Beyond eckart- young-mirsky, arXiv preprint: 2511.08875 (2025)
2025
-
[50]
,Davis–Kahan theorem under a moderate gap condition, Commun. Contemp. Math.28(2026), no. 1. Article no. 2550035, doi: 10.1142/S021919972550035X
2026 doi
-
[51]
,Eigenvalue stability and new perturbation bounds for the extremal eigenvalues of a matrix, arXiv preprint: 2603.19758 (2026)
2026 arXiv
-
[52]
arXiv preprint: 2409.20207
,New matrix perturbation bounds with relative norm: Perturbation of eigenspaces(2026). arXiv preprint: 2409.20207
2026
-
[53]
Vishnoi,Perturbation bounds for low-rank inverse approximations under noise, Proceedings of the 39th conference on neural information processing systems (neurips 2025), 2025
Phuc Tran and Nisheeth K. Vishnoi,Perturbation bounds for low-rank inverse approximations under noise, Proceedings of the 39th conference on neural information processing systems (neurips 2025), 2025
2025
-
[54]
Tropp,An introduction to matrix concentration inequalities, Found
J. Tropp,An introduction to matrix concentration inequalities, Found. Trends Mach. Learn.8(2015), no. 1-2, 1–230
2015
-
[55]
Vershynin,Introduction to the non-asymptotic analysis of random matrices, Compressed sensing: The- ory and applications, 2012, pp
R. Vershynin,Introduction to the non-asymptotic analysis of random matrices, Compressed sensing: The- ory and applications, 2012, pp. 210–268
2012
-
[56]
Press, 2019
,High dimensional probability, Cambridge Univ. Press, 2019
2019
-
[57]
Wainwright,High-dimensional statistics: A non-asymptotic viewpoint, Camb
M. Wainwright,High-dimensional statistics: A non-asymptotic viewpoint, Camb. Ser. Stat. Probab. Math., Cambridge Univ. Press, 2019
2019
-
[58]
Xia and F
D. Xia and F. Zhou,The sup-norm perturbation of HOSVD and low-rank tensor denoising, J. Mach. Learn. Res.20(2019), no. 61, 42
2019
-
[59]
Zhivotovskiy,Dimension-free bounds for sums of independent matrices and simple tensors via the variational principle, Electron
N. Zhivotovskiy,Dimension-free bounds for sums of independent matrices and simple tensors via the variational principle, Electron. J. Probab.29(2024), 1–28
2024
-
[60]
Z. Zhu, T. Wang, and R.J. Samworth,High-dimensional principal component analysis with heterogeneous missingness, J. R. Stat. Soc. Ser. B Stat. Methodol.84(2022), no. 5, 2000–2031. 34 PHUC TRAN, V AN VU AppendixA.Derivation of Theorems 2.1, 2.2, and 2.3 from Theorem 2.4 and The...
2022
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.