Distributed Estimation and Gap-Free Analysis of Canonical Correlations

Canyi Chen; Liping Zhu

REVIEW 3 major objections 5 minor 38 references

Distributed Estimation and Gap-Free Analysis of Canonical Correlations

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A distributed, communication-efficient algorithm estimates canonical correlations at pooled-data accuracy without assuming a gap between successive correlations.

desk verdict Worth a careful referee: a solid distributed CCA algorithm with a gap-free bound, but the main theorem's proof has a fixable yet load-bearing mismatch in Lemma B.3. read the letter →

arxiv 2412.17792 v1 pith:37YIG4CT submitted 2024-12-23 stat.CO

Canyi Chen , Liping Zhu This is my paper · ORCID

classification stat.CO MSC 62H20

keywords canonicalcorrelationanalysisdistributedestimationgap-freeboundshift-and-invertpreconditioningcommunication-efficientalgorithmmulti-roundoptimizationtop-Ldirections

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Canonical correlation analysis finds the linear combinations of two variable sets that move together, and doing it on data spread over many machines normally forces a choice between communicating large covariance matrices or accepting biased one-shot estimates. This paper proposes a multi-round distributed algorithm whose rounds exchange only vectors, and proves that its output converges linearly to the estimate a single pooled machine would produce. The proof works without the usual requirement that the top canonical correlations be separated by a nonzero gap, an assumption the paper shows is violated in three common image datasets. A sympathetic reader can take the central claim as: distributed CCA can match pooled-data accuracy at moderate communication cost, and the guarantee degrades gracefully, through a tunable threshold $\delta$, as the correlations cluster together.

What carries the argument

The machinery is shift-and-invert preconditioning for CCA, executed in a distributed star network. Shift-and-invert preconditioning is an iterative method that turns a top-eigenvector problem into repeated linear-system solves: the paper writes the CCA objective as a convex quadratic whose Hessian is $\hat H=\begin{pmatrix}\bar\rho_1\hat\Sigma_x & -\hat\Sigma_{x,y}\\ -\hat\Sigma_{x,y}^T & \bar\rho_1\hat\Sigma_y\end{pmatrix}$, so one Newton step is applying $(\bar\rho_1 I-\hat C)^{-1}$ to a vector rather than computing an SVD. In each outer round, the central machine collects the local gradient vectors $\hat g_{k,j}$ (each of dimension $d=d_x+d_y$) and updates using the first machine's Hessian $\hat H_1$ as a surrogate for the pooled Hessian; Lemma B.3 shows this inner loop contracts at rate $8\kappa/(\gamma\omega)$. The gap-free component is the error metric (3.1): instead of requiring $\hat\rho_1-\hat\rho_2>0$, the paper measures error against all pooled directions with correlation at most $(1-\delta)\hat\rho_1$, and Lemma B.2 shows each outer iteration shrinks the sum of those squared projections by a factor $128\omega^2/(\delta^2\hat\rho_1^2)$. That ratio, together with the surrogate-Hessian approximation, is what produces the contraction base $576\kappa/(\gamma\delta\hat\rho_1)$.

What would settle it

Simulate CCA data with a controlled spectral gap, split it across K machines, and on each run measure $\kappa=\|\hat T_1-\hat T\|$ from the first machine's sample covariance versus the pooled one. Set the local sample size n so that $576\kappa/(\gamma\delta\hat\rho_1)\ge 1$; then Theorem 1's condition fails and the observed error curve should stop decreasing (or diverge) with outer iterations. A run with n just above the threshold should show the linear contraction predicted by Corollary 1, so the threshold itself is observable.

Watch

Extended reading notes

Core claim

The paper establishes that top canonical directions can be computed in a distributed star network so that the estimator matches the pooled estimator up to the same statistical rate, with no explicit gap between successive canonical correlations. The main result (Theorem 1, simplified in Corollary 1) bounds the distance between the distributed and pooled estimates by a gap-free error: for any $\delta\in(0,1)$, the squared projections of the distributed direction onto pooled directions whose correlations are at most $(1-\delta)\hat\rho_1$ are $O_p\{(128\omega^2/(\delta^2\hat\rho_1^2))^T + (1/(1-128\omega^2/(\delta\hat\rho_1)^2))(64\kappa^2/(\gamma^2\omega^2))^{T'}\}$. Choosing $\omega=(\kappa\delta\hat\rho_1/\gamma)^{1/2}/3$ and $T'=T$ reduces the bound to $O_p\{(576\kappa/(\gamma\delta\hat\rho_1))^T\}$, a linear contraction whenever the first machine's spectral deviation $\kappa=\|\hat T_1-\hat T\|$ is small. Corollary 2 converts this into a co-variability guarantee: $(\hat u^\text{dist})^T\hat\Sigma_{x,y}(\hat v^\text{dist})\ge (1-\delta)(1-\varepsilon)\hat\rho_1$, so the distributed pair captures almost all of the pooled top-pair co-variability even when consecutive correlations coincide. Theorem 2 and its corollaries extend the same linear rate to the top-$L$ subspaces, replacing the classical $\sin\Theta$ gap condition with an enlarged-subspace statement.

Load-bearing premise

The load-bearing premise is that the first machine's local sample is large enough that the difference between its covariance structure and the pooled one (the paper's $\kappa$) stays small relative to the target correlation strength $\gamma\delta\hat\rho_1$, since otherwise the contraction ratio $576\kappa/(\gamma\delta\hat\rho_1)$ exceeds one and the linear convergence to pooled accuracy is not guaranteed.

Editorial extensions

If this is right

After $O(\log(1/\varepsilon))$ outer rounds, the distributed estimate reaches a pooled-level error; each round transmits only $d$-dimensional vectors, so the communication cost is $O(TT'dK)$ and the algorithm qualifies as communication-efficient under the vector-transmission standard.
The gap-free bound remains meaningful when the top canonical correlations are tied: the error is controlled by the tunable threshold $\delta$, and Corollary 2 guarantees the distributed pair captures at least $(1-\delta)(1-\varepsilon)$ of the pooled top-pair co-variability.
For the top-$L$ setting, Theorem 2 extends the same linear rate to subspaces without requiring $\hat\rho_L-\hat\rho_{L+1}>0$; Corollaries 4-6 provide a gap-free analogue of the classical $\sin\Theta$ theorem and show each estimated direction's captured co-variability is within a $(1-\delta)$ factor of the pooled one.
The proof conditions hold with overwhelming probability under sub-Gaussian data, and the same algorithm admits a fully stochastic initialization (no local SVD), so the convergence guarantee is not tied to an expensive initial eigen-decomposition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's 'no stringent restriction on K' is conditional: for fixed total sample size N, increasing K eventually makes n too small to keep $576\kappa/(\gamma\delta\hat\rho_1)<1$. An implicit consequence is that the method has a maximum number of usable machines at a given N, and the paper does not spell out its closed form.
The same enlargement-of-target trick, measuring error against all pooled directions within a $\delta$-neighborhood of the top correlation, could be transplanted to distributed PCA and generalized eigendecomposition, where explicit eigen-gaps are also commonly assumed; that transfer is not explored in the paper.
A practitioner-facing diagnostic follows directly: estimate $\hat\kappa=\|\hat T_1-\hat T\|$ from local and pooled covariances; if $576\hat\kappa/(\hat\gamma\delta\hat\rho_1)$ is near or above 1, the central machine should pull more data before trusting the output, or $\delta$ should be increased.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a useful paper on distributed CCA, with a real algorithmic contribution and a proof gap in one lemma that is repairable but load-bearing as written. Don't desk-reject it; send it to a referee, but ask that referee to check Lemma B.3 carefully.

What's new: a multi-round distributed algorithm for CCA that communicates only vectors, uses shift-and-invert with a local Hessian, and gives a gap-free convergence bound matching the pooled rate. The gap-free analysis is borrowed from Allen-Zhu and Li, but the distributed setting with local Hessian approximations is not in the prior literature. The bound in Corollary 1 is the kind of result people will want. The simulations and the three benchmark applications are honest and show the algorithm works.

Soft spots, in order of seriousness.

1. Lemma B.3. The theorem defines κ = ||T̂1 − T̂||. The lemma statement recalls that definition. The proof, however, writes ||Ĥ1 − Ĥ|| = κ and bounds the Hessian difference by covariance deviations. These are not the same quantity. The whitened cross-covariance T can be well-conditioned even when the covariance blocks differ in directions with little cross-covariance, so a bound on ||T̂1 − T̂|| does not control ||Ĥ1 − Ĥ||. As written, the inner-loop contraction (B.7) is not proven under the theorem's assumptions. That matters because Theorem 1 uses Lemma B.3 to set the inner-loop error and recurse. The fix looks straightforward: add a covariance concentration bound linking the two quantities, which will put an explicit per-machine sample-size condition into the theorem. But as written, the main bound doesn't follow.

2. The abstract's claim of no stringent restrictions on the number of machines is overstated. The theory needs n large enough that κ = o(δ γ ρ1). For fixed N, that limits K. The paper's own remark at the end of Section 3 concedes the per-machine sample must be large; the abstract should say "much weaker restrictions than one-shot methods" rather than "no stringent restrictions."

3. Minor: no code or error bars in the experiments. The log-error tables are suggestive, not definitive.

Overall: the core idea is sound, the writing is clear, and the citations are appropriate. The proof gap is the one thing that must be fixed before publication. I'd accept this for peer review and ask for a revision that repairs Lemma B.3 and recalculates the constants.

Referee Report

3 major / 5 minor

Summary. The paper proposes a multi-round distributed canonical correlation analysis (CCA) algorithm. The method constructs canonical directions through a convex, shift-and-invert preconditioned formulation and approximates the pooled Newton step using the first machine's Hessian together with averaged local gradients, requiring only O(dK) communication per inner iteration. The main theoretical contribution is a gap-free analysis: Theorem 1 and Corollary 1 claim that, under conditions involving the per-machine spectral deviation κ = ||T̂1 − T̂||, the distributed top canonical direction converges linearly to the pooled estimate with error Op((576κ/(γδρ̂1))^T), without requiring an explicit gap between canonical correlations. Corollaries 2–6 extend this to co-variability capture and top-L subspace guarantees. Simulations and three image benchmarks compare the method favorably with naive and whitened divide-and-conquer baselines.

Significance. If the proof gaps are repaired, the paper would make a useful contribution: a communication-efficient distributed CCA algorithm that transmits vectors rather than covariance matrices, combined with a gap-free analysis that avoids the commonly imposed eigengap assumption. The paper also provides an explicit concentration lemma (Lemma A.1) and is candid in the final remark that only the first machine's local sample size needs to be sufficiently large. The central claims are plausible and the numerical results support the algorithm's practical promise, but the proof of the main convergence theorem currently contains a load-bearing gap that must be fixed before the advertised rates are established.

major comments (3)

[Appendix B, Lemma B.3 (Eq. B.7)] Lemma B.3 is stated for κ = ||T̂1 − T̂||, exactly as in Theorem 1, but the proof contains the line '||Ĥ1 − Ĥ|| def = κ' and then bounds Ĥ1 − Ĥ by covariance deviations. No inequality is proved that controls ||Ĥ1 − Ĥ|| in terms of ||T̂1 − T̂||; the map from the covariance blocks to T = Σx^{-1/2}ΣxyΣy^{-1/2} does not have a dimension-free Lipschitz inverse in directions orthogonal to Σxy. Consequently the contraction factor 8κ/(γω) in (B.7) is not established under the theorem's assumptions. This is load-bearing because Theorem 1's proof uses ε_{T'} from Lemma B.3 in the recursion leading to (B.8), and Corollary 1's base 576κ/(γδρ̂1) depends on that factor. The gap appears repairable by adding a high-probability covariance-concentration bound, but the repair will introduce an explicit per-machine sample-size condition and should be reflected in the statements of Theorem 1 and Corollary 1.
[Appendix B, Lemma B.3 proof] The displayed identity '||I − M_{ρ1,1}M_ρ^{-1}|| = ||I − Ĥ1^{-1}Ĥ||' is not valid as written. From (B.1)–(B.2), M_{ρ1,1}M_ρ^{-1} = SĤ1^{-1}ĤS^{-1} with S = diag(Σ̂x^{1/2}, Σ̂y^{1/2}); S does not commute with Ĥ unless the cross-covariance is zero, so the spectral norm of I − SĤ1^{-1}ĤS^{-1} is not equal to that of I − Ĥ1^{-1}Ĥ. The proof needs an additional bound on ‖S(I − Ĥ1^{-1}Ĥ)S^{-1}‖, which introduces factors of the empirical covariance condition number and again affects the contraction factor in (B.7).
[Section 3.1 (Corollary 1) and Section 6] The claim that the method 'does not impose stringent restrictions on the number of machines' needs qualification. Corollary 1's linear-rate condition is κ = op(γδρ̂1) with κ = Op((d log²d/n)^{1/2}); in the uniformly split setting n = N/K, this requires n ≫ d log²d/(γδρ̂1)² and hence K ≪ N(γδρ̂1)²/(d log²d). The final remark in Section 6 correctly notes that the first machine's sample size must be sufficiently large, but it does not state the fixed-total-sample-size consequence, and Table 1 shows the estimate deteriorating as K grows for fixed iteration counts. Please state this trade-off explicitly so that the advertised flexibility in K can be calibrated against the per-machine sample-size requirement.

minor comments (5)

[Title page] The manuscript contains JMLR template artifacts, including 'Journal of Machine Learning Research 23 (2022) 1-35', 'Editor: My editor', and '©2022 Author One and Author Two'; these should be removed before any formal submission.
[Theorem 1 and Eq. (B.8)] The inner-loop term in Theorem 1 has a constant 1, while the proof in (B.8) has a constant 512 in front of (64κ²/(γ²ω²))^{T'}; the constants should be reconciled.
[Figure 1 caption] The caption appears to be missing the panel label for MFEAT; it reads '(A): MMILL (B): MNIST MFEAT' rather than labeling all three panels.
[Section 2.2 and Lemma B.2] The matrix B = diag(Σ̂x, Σ̂y) is introduced in the proof of Lemma B.2 but not in the methodology section; defining it where the local gradients are first discussed would improve readability.
[Abstract] The abstract says the distributed estimate achieves 'the same convergence rate as if all observations were pooled together', but the theorems compare the distributed estimate with the pooled estimator, not with the population canonical directions; this should be clarified to avoid overstatement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the distributed CCA bound is derived from independent concentration and spectral-perturbation lemmas; the gap-free accuracy measure is not fitted to the claimed output and the main proof does not rely on self-citation.

full rationale

The derivation of Corollary 1 is self-contained once the external concentration result of Gao et al. (2019) is granted: κ = ||T̂1−T̂|| is an independent spectral-deviation parameter bounded by Lemma A.1, and the initial condition (3.2) and the condition 2κ ≤ ω are shown to hold with overwhelming probability rather than being imposed as the conclusion. The outer-loop contraction (Lemma B.2) is obtained from the eigendecomposition of (ρ1I − Ĉ)^{-1}, and the inner-loop analysis (Lemma B.3) targets the difference between the local-Hessian and pooled-Hessian updates; neither step fits a parameter to the left-hand side of (3.3). Corollaries 2 and 5 are conditional implications that convert the projection bound into a co-variability lower bound via spectral decomposition, which is a mathematical consequence rather than a circular restatement. The cited Allen-Zhu and Li (2017) results are prior external work used for the top-L extension and gap-free Wedin bound; there is no self-citation chain carrying the main claim. A non-circular correctness concern is that Lemma B.3's proof writes ||Ĥ1 − Ĥ|| def = κ after defining κ = ||T̂1 − T̂||, so the contraction as written is not explicitly connected to the theorem's κ; this is a proof gap, not an equivalence of output to input.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard sub-Gaussian and positive-definiteness assumptions, a user-chosen δ, an unspecified constant c0, and the per-machine sample size condition encoded in κ. No entities are invented.

free parameters (3)

δ (relative gap threshold) = user-specified in (0,1)
The gap-free error metric and the convergence rate both depend on δ; smaller δ captures more co-variability but slows convergence. It is chosen by the user, not fitted to data.
c0 in ω = c0 (d log^2 d / n)^{1/2} = unspecified positive constant
Algorithm 1 sets ρ1_bar = σmax(\hat{T}_1) + 1.5 c0 (d log^2 d/n)^{1/2}; the theorem only requires ω ≥ 2κ, so c0 must be large enough, an implementation tuning choice.
T and T' (outer and inner iterations) = e.g., T=50, T'=10 in simulations
The convergence bound requires T and T' large enough; the user must choose them, and the simulations fix them.

assumptions (5)

domain assumption Samples (x_i, y_i) are sub-Gaussian
Used in Lemma A.1 and throughout for concentration of covariance estimates; stated in Section 2.1.
domain assumption Normalization max{σmax(Σx), σmax(Σy)} ≤ 1 and γ = min{σmin(Σx), σmin(Σy)} > 0
Justified by CCA invariance to linear transformations; needed for the constants in the bounds.
domain assumption Data are split uniformly into K machines of size n
The distributed setting and the definition of κ = ||\hat{T}_1 - \hat{T}|| use per-machine sample size; Section 2.2.
domain assumption Initial estimator from machine 1 satisfies condition (3.2)
Required by Theorem 1; Lemma A.1 proves it holds with overwhelming probability when n is large, but it is still an assumption on the initial iterate.
domain assumption The pooled empirical whitened cross-covariance \hat{T} exists and the covariance estimates are invertible with ||\hatΣ_{x,1}^{-1}|| ≤ 2/γ etc.
Used in Lemma A.1 and B.3 proofs to bound norms; holds w.h.p. under sub-Gaussianity and large n.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Distributed Estimation and Gap-Free Analysis of Canonical Correlations." pith.science (2026). https://pith.science/paper/37YIG4CT

@misc{pith2026241217792,
  author       = {Pith},
  title        = {Pith review of: Distributed Estimation and Gap-Free Analysis of Canonical Correlations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37YIG4CT}},
  note         = {Machine review of arXiv:2412.17792}
}

read the original abstract

Massive data analysis calls for distributed algorithms and theories. We design a multi-round distributed algorithm for canonical correlation analysis. We construct principal directions through the convex formulation of canonical correlation analysis and use the shift-and-invert preconditioning iteration to expedite the convergence rate. This distributed algorithm is communication-efficient. The resultant estimate achieves the same convergence rate as if all observations were pooled together, but does not impose stringent restrictions on the number of machines. We take a gap-free analysis to bypass the widely used yet unrealistic assumption of an explicit gap between the successive canonical correlations in the canonical correlation analysis. Extensive simulations and applications to three benchmark image data are conducted to demonstrate the empirical performance of our proposed algorithms and theories.

Figures

Figures reproduced from arXiv: 2412.17792 by the authors.

**Figure 2.** The horizontal axis stands for the number of outer i [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗

**Figure 3.** The horizontal axis stands for the number of outer i [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: The horizontal axis stands for 1/δ, and the vertical axis stands for the logarithmic errors of the naive divide-and-conquer ( ), the whitened divide-and-conquer ( ), the pooled ( ) and the distributed ( ) estimates. 4.3 The number of local machines We evaluate how the …

**Figure 5.** Figure 5: The horizontal axis stands for the logarithmic num [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 34 canonical work pages

[1]

Lazysvd: Even faster svd decomposition yet without agonizing pain

Zeyuan Allen-Zhu and Yuanzhi Li. Lazysvd: Even faster svd decomposition yet without agonizing pain. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper/2016/file/c6e19e830859f2cb9f7c8f8cacb8d2a6-Paper.pdf

work page 2016
[2]

Doubly accelerated methods for faster CCA and generalized eigendecomposition

Zeyuan Allen-Zhu and Yuanzhi Li. Doubly accelerated methods for faster CCA and generalized eigendecomposition. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 98--106. PMLR, 06--11 Aug 2017. URL https://proceedings.mlr.press/v70/al...

work page 2017
[3]

T. W. Anderson. Asymptotic Theory for Canonical Correlation Analysis . Journal of Multivariate Analysis, 70 0 (1): 0 1--29, July 1999

work page 1999
[4]

T. W. Anderson. An Introduction to Multivariate Statistical Analysis. Wiley Series in Probability and Statistics. Wiley-Interscience , Hoboken, N.J , 3rd ed edition, 2003

work page 2003
[5]

A probabilistic interpretation of canonical correlation analysis

Francis Bach and Michael Jordan. A probabilistic interpretation of canonical correlation analysis. Technical Report 688, University of California , Dept. of Statistics , May 2005

work page 2005
[6]

Canonical correlation coefficients of high-dimensional Gaussian vectors: Finite rank case

Zhigang Bao, Jiang Hu, Guangming Pan, and Wang Zhou. Canonical correlation coefficients of high-dimensional Gaussian vectors: Finite rank case. The Annals of Statistics, 47 0 (1): 0 612--640, February 2019

work page 2019
[7]

Tony Cai and Anru Zhang

T. Tony Cai and Anru Zhang. Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics. The Annals of Statistics, 46 0 (1): 0 60--89, February 2018

work page 2018
[8]

Lee, He Li, and Yun Yang

Xi Chen, Jason D. Lee, He Li, and Yun Yang. Distributed Estimation for Principal Component Analysis : An Enlarged Eigenspace Analysis . Journal of the American Statistical Association, 117 0 (540): 0 1775--1786, October 2022

work page 2022

Show all 38 references

[9]

Yang, Chris J

Zhehui Chen, Lin F. Yang, Chris J. Li, and Tuo Zhao. Dropping convexity for more efficient and scalable online multiview learning. arXiv:1702.08134, September 2019

2019 arXiv
[10]

Tackling Small Eigen-Gaps : Fine-Grained Eigenvector Estimation and Inference Under Heteroscedastic Noise

Chen Cheng, Yuting Wei, and Yuxin Chen. Tackling Small Eigen-Gaps : Fine-Grained Eigenvector Estimation and Inference Under Heteroscedastic Noise . IEEE Transactions on Information Theory, 67 0 (11): 0 7380--7419, November 2021

2021
[11]

Multi-view learning of word embeddings via CCA

Paramveer Dhillon, Dean P Foster, and Lyle Ungar. Multi-view learning of word embeddings via CCA . In J. Shawe-Taylor , R. Zemel, P. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc. , 2011

2011
[12]

UCI machine learning repository, 2017

Dheeru Dua and Casey Graff. UCI machine learning repository, 2017

2017
[13]

Challenges of big data analysis

Jianqing Fan, Fang Han, and Han Liu. Challenges of big data analysis. National Science Review, 1 0 (2): 0 293--314, June 2014

2014
[14]

Distributed estimation of principal eigenspaces

Jianqing Fan, Dong Wang, Kaizheng Wang, and Ziwei Zhu. Distributed estimation of principal eigenspaces. The Annals of Statistics, 47 0 (6): 0 3009--3031, December 2019

2019
[15]

Communication- Efficient Accurate Statistical Estimation

Jianqing Fan, Yongyi Guo, and Kaizheng Wang. Communication- Efficient Accurate Statistical Estimation . Journal of the American Statistical Association, pages 1--11, September 2021

2021
[16]

Chao Gao, Zongming Ma, Zhao Ren, and Harrison H. Zhou. Minimax estimation in sparse canonical correlation analysis. The Annals of Statistics, 43 0 (5): 0 2168--2197, 2015

2015
[17]

Chao Gao, Zongming Ma, and Harrison H. Zhou. Sparse CCA : Adaptive estimation and computational barriers. The Annals of Statistics, 45 0 (5): 0 2074--2101, October 2017

2017
[18]

Stochastic Canonical Correlation Analysis

Chao Gao, Dan Garber, Nathan Srebro, Jialei Wang, and Weiran Wang. Stochastic Canonical Correlation Analysis . Journal of Machine Learning Research, 20 0 (167): 0 1--46, 2019

2019
[19]

Fast and simple PCA via convex optimization

Dan Garber and Elad Hazan. Fast and simple PCA via convex optimization. arXiv:1509.05647, November 2015

2015 arXiv
[20]

Communication-efficient algorithms for distributed stochastic principal component analysis

Dan Garber, Ohad Shamir, and Nathan Srebro. Communication-efficient algorithms for distributed stochastic principal component analysis. In International Conference on Machine Learning , pages 1203--1212. PMLR , July 2017

2017
[21]

Golub and Charles F

Gene H. Golub and Charles F. Van Loan. Matrix Computations . Number 3 in Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press , Baltimore , 3 edition, 1983

1983
[22]

Golub and Hongyuan Zha

Gene H. Golub and Hongyuan Zha. The canonical correlations of matrix pairs and their numerical computation. In Avner Friedman, Willard Miller, Adam Bojanczyk, and George Cybenko, editors, Linear Algebra for Signal Processing , volume 69, pages 27--49. Springer New York , New Y...

1995
[23]

Hardoon, Sandor Szedmak, and John Shawe-Taylor

David R. Hardoon, Sandor Szedmak, and John Shawe-Taylor . Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16 0 (12): 0 2639--2664, December 2004

2004
[24]

Hestenes and E

M. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49: 0 409--436, 1952

1952
[25]

Relations between two sets of variates

Harold Hotelling. Relations between two sets of variates. Biometrika, 28 0 (3/4): 0 321--377, 1936

1936
[26]

Accelerating stochastic gradient descent using predictive variance reduction

Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., ...

2013
[27]

Jordan, Jason D

Michael I. Jordan, Jason D. Lee, and Yun Yang. Communication-efficient distributed statistical inference. Journal of the American Statistical Association, 114 0 (526): 0 668--681, 2019

2019
[28]

Lecun, L

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998

1998
[29]

One-shot distributed algorithm for generalized eigenvalue problem

Kexin Lv, Fan He, Xiaolin Huang, Jie Yang, and Liming Chen. One-shot distributed algorithm for generalized eigenvalue problem. arXiv:2010.11625, October 2020

2010 arXiv
[30]

Subspace perspective on canonical correlation analysis: Dimension reduction and minimax rates

Zhuang Ma and Xiaodong Li. Subspace perspective on canonical correlation analysis: Dimension reduction and minimax rates. Bernoulli, 26 0 (1): 0 432--470, February 2020

2020
[31]

Finding linear structure in large datasets with scalable canonical correlation analysis

Zhuang Ma, Yichao Lu, and Dean Foster. Finding linear structure in large datasets with scalable canonical correlation analysis. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Le...

2015
[32]

Communication-efficient distributed optimization using an approximate newton-type method

Ohad Shamir, Nati Srebro, and Tong Zhang. Communication-efficient distributed optimization using an approximate newton-type method. In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machi...

2014
[33]

Cees G. M. Snoek, Marcel Worring, Jan C. van Gemert , Jan-Mark Geusebroek, and Arnold W. M. Smeulders. The challenge problem for automated detection of 101 semantic concepts in multimedia. In Proceedings of the 14th ACM International Conference on Multimedia , MM '06, pages 42...

2006
[34]

Wainwright

Martin J. Wainwright. High- Dimensional Statistics : A Non-Asymptotic Viewpoint . Cambridge University Press , first edition, February 2019

2019
[35]

Efficient globally convergent stochastic optimization for canonical correlation analysis

Weiran Wang, Jialei Wang, Dan Garber, Dan Garber, and Nati Srebro. Efficient globally convergent stochastic optimization for canonical correlation analysis. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems...

2016
[36]

Y. Yu, T. Wang, and R. J. Samworth. A useful variant of the Davis Kahan theorem for statisticians. Biometrika, 102 0 (2): 0 315--323, June 2015

2015
[37]

Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J

Matei Zaharia, Reynold S. Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J. Franklin, Ali Ghodsi, Joseph Gonzalez, Scott Shenker, and Ion Stoica. Apache spark: A unified engine for big data processin...

2016
[38]

A general framework for robust testing and confidence regions in high-dimensional quantile regression

Tianqi Zhao, Mladen Kolar, and Han Liu. A general framework for robust testing and confidence regions in high-dimensional quantile regression. arXiv:1412.8724, March 2015

2015 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Lazysvd: Even faster svd decomposition yet without agonizing pain

Zeyuan Allen-Zhu and Yuanzhi Li. Lazysvd: Even faster svd decomposition yet without agonizing pain. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper/2016/file/c6e19e830859f2cb9f7c8f8cacb8d2a6-Paper.pdf

work page 2016

[2] [2]

Doubly accelerated methods for faster CCA and generalized eigendecomposition

Zeyuan Allen-Zhu and Yuanzhi Li. Doubly accelerated methods for faster CCA and generalized eigendecomposition. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 98--106. PMLR, 06--11 Aug 2017. URL https://proceedings.mlr.press/v70/al...

work page 2017

[3] [3]

T. W. Anderson. Asymptotic Theory for Canonical Correlation Analysis . Journal of Multivariate Analysis, 70 0 (1): 0 1--29, July 1999

work page 1999

[4] [4]

T. W. Anderson. An Introduction to Multivariate Statistical Analysis. Wiley Series in Probability and Statistics. Wiley-Interscience , Hoboken, N.J , 3rd ed edition, 2003

work page 2003

[5] [5]

A probabilistic interpretation of canonical correlation analysis

Francis Bach and Michael Jordan. A probabilistic interpretation of canonical correlation analysis. Technical Report 688, University of California , Dept. of Statistics , May 2005

work page 2005

[6] [6]

Canonical correlation coefficients of high-dimensional Gaussian vectors: Finite rank case

Zhigang Bao, Jiang Hu, Guangming Pan, and Wang Zhou. Canonical correlation coefficients of high-dimensional Gaussian vectors: Finite rank case. The Annals of Statistics, 47 0 (1): 0 612--640, February 2019

work page 2019

[7] [7]

Tony Cai and Anru Zhang

T. Tony Cai and Anru Zhang. Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics. The Annals of Statistics, 46 0 (1): 0 60--89, February 2018

work page 2018

[8] [8]

Lee, He Li, and Yun Yang

Xi Chen, Jason D. Lee, He Li, and Yun Yang. Distributed Estimation for Principal Component Analysis : An Enlarged Eigenspace Analysis . Journal of the American Statistical Association, 117 0 (540): 0 1775--1786, October 2022

work page 2022