REVIEW 2 major objections 5 minor 52 references
Joint estimation of high-dimensional spiked covariance matrices via a partially shared subspace
T0 review · 2 major / 5 minor · reviewed 2026-07-10 · grok-4.5
Pith's one-line read When two high-dimensional covariances share only part of their spiked subspace, you can recover that shared part, optimally pool it, and improve the target covariance estimate.
desk verdict Solid high-dim RMT paper that generalizes partial subspace sharing and gives a usable pooling estimator; the main soft spot is the unproved leap from oracle consistency to the iterative algorithm they actually run. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The partially shared subspace (PSS) model together with the almost-sure limits of sample principal angles (Lemma 2 / Theorem 3): shared singular values converge to a product of attenuation factors times a rotation, distinct singular values converge separately, and a debiased cutoff separates them, driving the iterative rank-and-index estimator and the optimal pooling weight.
What would settle it
Generate data from the PSS model with known shared rank and mixed index sets under proportional growth; check whether Algorithm 1 recovers the true shared rank and index sets with probability approaching one, and whether the pooled covariance's Frobenius error falls below the target-only spiked estimator at the rate predicted by the asymptotic loss formula.
Extended reading notes
Core claim
Under the partially shared subspace (PSS) model, the shared rank and the index sets of the shared spiked eigenvectors can be recovered consistently from the singular values and principal angles of the two sample spiked subspaces; the shared projection can then be estimated by a weighted sum of the two sample projections whose optimal weight has a closed form that depends on relative sample sizes and relative spike strengths, yielding an asymptotic efficiency gain over target-only estimation and a consistent high-dimensional contrastive subspace.
Load-bearing premise
The shared directions must produce sample singular values that stay strictly larger than those of the distinct directions, and the alternating algorithm that uses those cutoffs is assumed to inherit the consistency proved only for its oracle versions.
Editorial extensions
If this is right
- Target covariance estimates in high dimensions can be improved by any related background dataset that shares even a non-leading subspace, without requiring the whole eigenbasis or parameter proximity.
- Negative transfer is self-limiting: if no shared structure exists the estimated shared rank collapses to zero.
- Contrastive dimension reduction gains its first high-dimensional asymptotic guarantees via the PSS distinct subspace.
- Portfolio risk estimates during regime shifts can exploit pre-shift returns as background while isolating crisis-specific factors.
- Tumor-versus-normal gene-expression analysis can separate organ-level shared variation from disease-specific directions with quantified error.
Reading between the lines
- The same principal-angle separation idea could extend to more than two groups if a multi-subspace angle or joint-and-individual variation measure replaces the two-matrix singular values.
- Combining the optimal shared-projection pooling with existing optimal eigenvalue shrinkage would likely produce a still tighter covariance estimator for downstream tasks such as LDA or GMV portfolios.
- The method itself can serve as a diagnostic: a stable zero shared-rank estimate is evidence against a shared-subspace hypothesis, something many common-PCA procedures cannot detect.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a partially shared subspace (PSS) spiked covariance model for two high-dimensional datasets that share an r_S-dimensional subspace of unknown rank and arbitrary spectral position while retaining distinct spikes. Under proportional growth (Assumptions 1–2) and a shared/distinct singular-value separation condition (Assumption 3), it derives almost-sure limits for principal angles between sample eigenspaces (Lemma 2, Theorem 3), constructs oracle estimators for shared rank and index sets that are a.s. consistent (Theorems 5–6), and proposes an iterative Algorithm 1 that alternates between them. A weighted pooled projection estimator of the shared subspace is shown to have asymptotic Frobenius loss L(α) with closed-form optimal weight α* (Theorem 7), yielding a plug-in target covariance estimator and a high-dimensional contrastive dimension-reduction estimator from the distinct subspace. Simulations under the PSS model, misspecification, and degenerate endpoints, plus two real-data illustrations (COVID-era GMV portfolios and LGG gene expression), support the claims.
Significance. If the results hold, the paper supplies a clean geometric alternative to proximity-based transfer learning for high-dimensional covariances and the first contrastive dimension-reduction procedure with asymptotic guarantees in the proportional-growth regime. The RMT foundations (extensions of Johnstone/Paul/BBP) are carefully derived, the optimal pooling weight is closed-form and interpretable, and the framework strictly generalizes existing common-PC and multi-group models (Remark 1). The simulations are extensive (including misspecification and degenerate endpoints) and the real-data examples are coherent. The main practical deliverable—an implementable joint estimator of shared rank, indices, and the pooled covariance—would be useful for portfolio construction, multi-study genomics, and related settings where background data are abundant but only partially aligned.
major comments (2)
- End of §3.2 and Theorems 5–6: the paper itself states that oracle consistency of ˆr*_S and ˆΨ*_X, ˆΨ*_Y “do not automatically guarantee the consistency of the empirical estimator ˆr_S output by Algorithm 1,” and that termination is not universally guaranteed (only that the finite-state map eventually cycles or fixes). The subsequent asymptotic loss L(α), optimal weight α*, and efficiency claims for ˆΣ_X and the distinct-subspace CDR estimator all attach to the procedure that is actually run. Either a consistency argument for the alternating map under Assumptions 1–3, or a high-probability bound that it recovers the oracle fixed point, is needed before the central estimation claims can be regarded as fully established.
- Assumption 3 and Discussion §6 / Appendix A4.2: the separation condition ϕ_cX(λ_rS,S)ϕ_cY(γ_rS,S) > σ_1(Φ_X,D Q_D^⊤ U_D Φ_Y,D) is load-bearing for Theorems 5–6 and for the cutoff used in Algorithm 1. Finite-sample underestimation of the cutoff is only patched by an ad-hoc margin ε whose three candidates (0, 1/√(n_X+n_Y), parametric bootstrap) have no uniformly best choice. The paper should either supply a data-driven, theoretically justified rule for ε or quantify the probability that the unadjusted cutoff recovers r_S under the stated assumptions, so that practitioners know when the procedure is reliable.
minor comments (5)
- Notation for debiased spikes (˜λ, ˜γ) and the debiasing map d(ℓ,c) appears in §3.2 without an explicit display of the inverse of the BBP map; a short displayed equation would help readers.
- Figure 1 and Table 1: the mixed-position, small-sample cells show substantial underestimation of r_S by PSS; a brief discussion of when the method is expected to struggle would improve interpretability.
- Appendix A1 Algorithm 1: the initialization ˆΨ_X = {r_X}, ˆΨ_Y = {r_Y} is natural but could be motivated more explicitly (why the largest indices rather than, e.g., a random or top-k start).
- Related-work paragraph on transfer learning: a short comparison of the geometric PSS condition versus the usual Frobenius/sparsity proximity conditions would clarify the novelty claim.
- Typos / polish: “efficiency” (ligature issues), “difficulty”, and a few missing spaces around math operators appear in the main text and appendix.
Circularity Check
No significant circularity: asymptotic claims rest on external RMT limits (Paul, Baik–Silverstein, Johnstone) and standard singular-value continuity; the iterative Algorithm 1 is an unproved practical bridge, not a definitional reduction of any claimed result.
full rationale
The paper’s load-bearing derivations (Lemma 2, Theorems 3–7) start from classical single-sample spiked asymptotics (Proposition 1, citing Paul 2007 / Baik–Silverstein) and extend them via elementary almost-sure arguments (inner-product decompositions, block-diagonalization of singular values, continuity of singular values, and the CS decomposition). The oracle estimators ˆr*_S and ˆΨ* are shown consistent under Assumptions 1–3 by direct comparison of limiting singular-value multisets; the approximate loss L†(α) and optimal weight α* follow by expanding the Frobenius distance of the weighted projection sum and collecting the known ϕc factors. No parameter is fitted to data and then re-used as a “prediction” of a related quantity; no uniqueness theorem or ansatz is imported from the authors’ own prior work; the model strictly generalizes (rather than renames) earlier common-subspace models. The only interdependence is the chicken-and-egg between rank and index sets inside Algorithm 1, which the paper itself flags as unproved for the empirical (non-oracle) estimator. That is a gap in the consistency theory, not a circular derivation of any stated theorem. Hence the derivation chain is self-contained against external RMT benchmarks and scores 0.
Assumptions & free parameters
free parameters (3)
- cutoff margin ε
- spike counts rX, rY and bulk levels
- pooling weight α (plug-in ˆα)
assumptions (4)
- domain assumption Data are i.i.d. Gaussian with spiked covariances ΣX, ΣY (unit bulk after rescaling).
- domain assumption Proportional growth p/nX→cX, p/nY→cY with all spikes above BBP thresholds (Assumptions 1–2).
- ad hoc to paper Assumption 3: ϕcX(λrS,S)ϕcY(γrS,S) > σ1(ΦX,D Q⊤D UD ΦY,D).
- standard math Standard singular-value / projection perturbation facts (Stewart–Sun, Vershynin concentration lemmas).
invented entities (2)
-
Partially shared subspace (PSS) model
independent evidence
-
Oracle-then-iterative shared rank/index estimators (ˆr∗S, ˆΨ∗, Algorithm 1)
Cite this review
Pith. "Pith review of Joint estimation of high-dimensional spiked covariance matrices via a partially shared subspace." pith.science (2026). https://pith.science/paper/JIU5QSKO
@misc{pith2026260708123,
author = {Pith},
title = {Pith review of: Joint estimation of high-dimensional spiked covariance matrices via a partially shared subspace},
year = {2026},
howpublished = {\url{https://pith.science/paper/JIU5QSKO}},
note = {Machine review of arXiv:2607.08123}
}
read the original abstract
Statistical analysis of high-dimensional data is often hampered by limited sample sizes, yet auxiliary datasets from related sources are often readily available. When two such datasets share part of their covariance structure, but not all of it, exploiting the shared part can substantially improve estimation. We propose a spiked covariance model that explicitly captures this partial sharing: two datasets share a subspace of unknown rank and arbitrary position in the spectrum, while each retains its own distinct spiked directions. The model treats the two datasets symmetrically and strictly generalizes existing models for shared covariance structure. We develop a complete estimation procedure that includes joint estimation of the shared subspace and its rank, a closed-form pooling weight for combining the two datasets, and asymptotic guarantees derived from random matrix theory in the proportional-growth regime. The framework also resolves a gap in contrastive dimension reduction by providing a principled estimator for high-dimensional settings. We illustrate the methodology on portfolio construction during the early COVID-19 pandemic and on contrastive analysis of brain tumor gene expression.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
Annals of Statistics , volume=
Optimal shrinkage of eigenvalues in the spiked covariance model , author=. Annals of Statistics , volume=
-
[3]
Journal of Machine Learning Research , volume=
Shared subspace models for multi-group covariance estimation , author=. Journal of Machine Learning Research , volume=
- [4]
-
[5]
Annals of Statistics , volume=
On the distribution of the largest eigenvalue in principal components analysis , author=. Annals of Statistics , volume=. 2001 , publisher=
work page 2001
-
[6]
Journal of the American Statistical Association , volume=
Estimation of the number of spiked eigenvalues in a covariance matrix by bulk eigenvalue matching analysis , author=. Journal of the American Statistical Association , volume=. 2023 , publisher=
work page 2023
-
[7]
SIAM Journal on Matrix Analysis and Applications , volume=
Schubert varieties and distances between subspaces of different dimensions , author=. SIAM Journal on Matrix Analysis and Applications , volume=. 2016 , publisher=
work page 2016
-
[8]
Asymptotics of sample eigenstructure for a large dimensional spiked covariance model , author=. Statistica Sinica , volume=
Show all 52 references
-
[9]
Journal of Multivariate Analysis , volume=
Eigenvalues of large sample covariance matrices of spiked population models , author=. Journal of Multivariate Analysis , volume=. 2006 , publisher=
2006
-
[10]
Vershynin, Roman , title =
-
[11]
2019 , publisher=
Probability: Theory and examples , author=. 2019 , publisher=
2019
-
[12]
Nature Communications , volume=
Exploring patterns enriched in a dataset with contrastive principal component analysis , author=. Nature Communications , volume=. 2018 , publisher=
2018
-
[13]
The Annals of Applied Statistics , volume=
Probabilistic contrastive dimension reduction for case-control study data , author=. The Annals of Applied Statistics , volume=. 2024 , publisher=
2024
-
[14]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Unsupervised learning with contrastive latent variable models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[15]
Advances in Neural Information Processing Systems , volume=
Contrastive dimension reduction: When and how? , author=. Advances in Neural Information Processing Systems , volume=
-
[16]
Common principal components in
Flury, Bernhard N , journal=. Common principal components in. 1984 , publisher=
1984
-
[17]
Biometrika , volume=
Two generalizations of the common principal component model , author=. Biometrika , volume=. 1987 , publisher=
1987
-
[18]
Deflation methods for sparse
Mackey, Lester , journal=. Deflation methods for sparse
-
[19]
and Sun, Ji-guang , publisher =
Stewart, Gilbert W. and Sun, Ji-guang , publisher =. Matrix perturbation theory , year =
-
[20]
2007 , publisher=
Applied multivariate statistical analysis , author=. 2007 , publisher=
2007
-
[21]
Electronic Journal of Statistics , volume =
Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation , author=. Electronic Journal of Statistics , volume =
-
[22]
arXiv preprint arXiv:2510.11847 , year=
Contrastive dimension reduction: A systematic review , author=. arXiv preprint arXiv:2510.11847 , year=
-
[23]
Journal of the American Statistical Association , volume=
Transfer learning under high-dimensional generalized linear models , author=. Journal of the American Statistical Association , volume=. 2023 , publisher=
2023
-
[24]
Proceedings of the International Conference on Artificial Intelligence and Statistics , pages=
Transfusion: Covariate-shift robust transfer learning for high-dimensional regression , author=. Proceedings of the International Conference on Artificial Intelligence and Statistics , pages=. 2024 , organization=
2024
-
[25]
arXiv preprint arXiv:2501.02411 , year=
Transfer learning via regularized linear discriminant analysis , author=. arXiv preprint arXiv:2501.02411 , year=
-
[26]
2023 , school=
Transfer learning in classification and regression with summary statistics , author=. 2023 , school=
2023
-
[27]
Duan, Junting and Pelger, Markus and Xiong, Ruoxuan , journal=. Target. 2024 , publisher=
2024
-
[28]
arXiv preprint arXiv:2403.07431 , year=
Knowledge transfer across multiple principal component analysis studies , author=. arXiv preprint arXiv:2403.07431 , year=
-
[29]
2024 , institution=
Transfer learning for covariance matrix estimation: Optimality and Adaptivity , author=. 2024 , institution=
2024
-
[30]
Journal of the American Statistical Association , volume=
Between-groups comparison of principal components , author=. Journal of the American Statistical Association , volume=. 1979 , publisher=
1979
-
[31]
Biometrika , volume=
Common principal component subspaces in two groups , author=. Biometrika , volume=. 1988 , publisher=
1988
-
[32]
Biometrika , volume=
Some tests for common principal component subspaces in several groups , author=. Biometrika , volume=. 1991 , publisher=
1991
-
[33]
Biometrics , volume=
Semiparametric partial common principal component analysis for covariance matrices , author=. Biometrics , volume=. 2021 , publisher=
2021
-
[34]
Personalized
Shi, Naichen and Al Kontar, Raed , journal=. Personalized
-
[35]
2022 , howpublished =
Shi, Naichen and Al Kontar, Raed , title =. 2022 , howpublished =
2022
-
[36]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Contrastive functional principal component analysis , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[37]
and Gasalberti, D
Karsonovich, T. and Gasalberti, D. P. and Rayi, A. , title =. StatPearls [Internet] , publisher =. 2025 , month = jan, url =
2025
-
[38]
Visualizing and interpreting cancer genomics data via the
Goldman, Mary J and Craft, Brian and Hastie, Mim and Repe. Visualizing and interpreting cancer genomics data via the. Nature Biotechnology , volume=. 2020 , publisher=
2020
-
[39]
Dimension estimation in noisy
Ulfarsson, Magnus O., and Solo, Victor , journal=. Dimension estimation in noisy. 2008 , publisher=
2008
-
[40]
2009 , publisher=
Modern Portfolio Theory and Investment Analysis , author=. 2009 , publisher=
2009
-
[41]
The Journal of Finance , volume=
Risk reduction in large portfolios: Why imposing the wrong constraints helps , author=. The Journal of Finance , volume=. 2003 , publisher=
2003
-
[42]
The examination of
Horvath, Dominik and Wang, Yung-Lin , journal=. The examination of. 2021 , publisher=
2021
-
[43]
Analysis of risk correlations among stock markets during the
Wu, JunFeng and Zhang, Chao and Chen, Yun , journal=. Analysis of risk correlations among stock markets during the. 2022 , publisher=
2022
-
[44]
Scientific Reports , volume=
Principal components analysis and the reported low intrinsic dimensionality of gene expression microarray data , author=. Scientific Reports , volume=. 2016 , publisher=
2016
-
[45]
Nature Biotechnology , volume=
A global map of human gene expression , author=. Nature Biotechnology , volume=. 2010 , publisher=
2010
-
[46]
Identifying patterns differing between high-dimensional datasets with generalized contrastive
de Oliveira, Eliezyer Fermino and Garg, Pranjal and Hjerling-Leffler, Jens and Batista-Brito, Renata and Sjulson, Lucas , journal=. Identifying patterns differing between high-dimensional datasets with generalized contrastive. 2025 , publisher=
2025
-
[47]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Probabilistic principal component analysis , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 1999 , publisher=
1999
-
[48]
Journal of Multivariate Analysis , volume=
Angle-based joint and individual variation explained , author=. Journal of Multivariate Analysis , volume=. 2018 , publisher=
2018
-
[49]
Journal of Machine Learning Research , volume=
High-dimensional linear discriminant analysis classifier for spiked covariance model , author=. Journal of Machine Learning Research , volume=
-
[50]
Tony and Li, Hongzhe , title =
Li, Sai and Cai, T. Tony and Li, Hongzhe , title =. Journal of the American Statistical Association , volume =. 2023 , publisher =
2023
-
[51]
Journal of Machine Learning Research , volume =
Ren, Mingyang and Zhen, Yaoming and Wang, Junhui , title =. Journal of Machine Learning Research , volume =
-
[52]
Hendy, Sharon and Dar, Yehuda , journal=
Reviewed July 10, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.