REVIEW 2 major objections 7 minor 28 references
Block subspace expansions for eigenvalues and eigenvectors approximation
T0 review · 2 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper proves that the optimal block expansion of a search subspace is obtained by applying the pseudoinverse of the residual to the target subspace, and that the resulting iterates converge geometrically in the Hermitian case.
desk verdict Necessary non-degeneracy condition missing from the abstract; otherwise the block optimal expansion and convergence bounds hold up. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the residual matrix $R=(I-P_{\mathcal V})AV$, whose range is the part of $A(\mathcal V)$ lying outside $\mathcal V$, together with the subspace $\mathcal N=(I-P_{\mathcal V})P_{\mathcal S}(\mathcal X)$ obtained by projecting the target $\mathcal X$ onto $\mathcal S=\mathcal V+A(\mathcal V)$ and then removing the $\mathcal V$-component. The identity $\mathcal S=\mathcal V\oplus (I-P_{\mathcal V})A(\mathcal V)$ turns every expanded space $\mathcal V+A(\mathcal W)$ into an orthogonal sum, so the optimal choice becomes $\mathcal W=\mathcal V R^{\dagger}(\mathcal X)$, where $R^{\dagger}$ is the Moore–Penrose pseudoinverse. The convergence analysis reduces angles to singular values of $X_\perp^*V(X^*V)^{\dagger}$ and amplifies the spectral gap with the linear polynomial $x-(\lambda_{d+1}+\lambda_n)/2$ for the optimal iterates, and with Chebyshev polynomials for block Krylov spaces.
What would settle it
Take $A=\mathrm{diag}(1,0)$, $d=1$, $\mathcal X=\mathrm{span}\{e_1\}$, and $\mathcal V=\mathrm{span}\{e_2\}$: then $A(\mathcal V)=\{0\}$, every iterate stays in $\mathrm{span}\{e_2\}$, and $\tan\Theta(\mathcal X,\mathcal V_t)=\infty$, so the bound of Theorem 3.8 fails exactly when its hypothesis $\mathcal X\cap\mathcal V^\perp=\{0\}$ is violated. To test the optimality formula itself, enumerate all unit-norm $w\in\mathcal V$ in a small random instance and compare the minimum of the principal angle of $\mathcal V+A(\mathcal W)$ over $\dim\mathcal W\le d$ with the angle of $\mathcal V+A(\mathcal V R^\dagger(\mathcal X))$; any mismatch would disprove Theorem 3.2.
Extended reading notes
Core claim
The central discovery is that the optimal block subspace expansion problem has a closed-form solution. Let $\mathcal S=\mathcal V+A(\mathcal V)$ and $\mathcal N=(I-P_{\mathcal V})P_{\mathcal S}(\mathcal X)$. Then $\mathcal V\oplus\mathcal N$ is the closest subspace to $\mathcal X$ among all spaces of the form $\mathcal V+A(\mathcal W)$ with $\mathcal W\subset\mathcal V$ and $\dim\mathcal W\le d$, in the componentwise sense of principal angles; a subspace $\mathcal W_0$ attaining this optimum can be chosen as $\mathcal W_0=\mathcal V R^{\dagger}(\mathcal X)$, uniquely when $\mathcal V\cap A^{-1}(\mathcal V)=\{0\}$ and $\dim\mathcal N=d$. This is Theorem 3.2 and Corollary 3.4. In the Hermitian case, Algorithm 3.1 produces $\mathcal V_t\subset\mathcal V_{t-1}+A(\mathcal V_{t-1})$ with $\Theta(\mathcal X,\mathcal V_t)=\Theta(\mathcal X,\mathcal V_{t-1}+A(\mathcal V_{t-1}))$, and if $\mathcal X\cap\mathcal V^\perp=\{0\}$ then every unitarily invariant norm of $\tan\Theta(\mathcal X,\mathcal V_t)$ satisfies the geometric bound of Theorem 3.8. Block Krylov spaces $\mathcal K_t=\mathcal V+A\mathcal V+\cdots+A^t\mathcal V$ satisfy a related Chebyshev bound that takes oversampling into account (Theorem 3.9).
Load-bearing premise
The load-bearing premise for the convergence theorem is that $\mathcal X\cap\mathcal V^\perp=\{0\}$, meaning no target eigenvector is orthogonal to the starting subspace; otherwise that direction is invisible to every iterate $\mathcal V_t$, the tangent angles stay infinite, and the claimed geometric decay cannot hold, and the paper gives no probability estimate for this 'generic' condition.
Editorial extensions
If this is right
- At each step the optimal subspace $\mathcal V_t$ lies inside the block Krylov space $\mathcal K_t$ but has dimension $r+dt$ instead of $r(t+1)$; the angle to $\mathcal X$ equals that of the full one-step block Krylov expansion, giving the same angular quality in a much smaller space (Remark 3.5).
- For Hermitian $A$ with $\lambda_d>\lambda_{d+1}$ and $\mathcal X\cap\mathcal V^\perp=\{0\}$, the tangent of the largest principal angle decays geometrically with rate $(\lambda_{d+1}-\lambda_n)/((\lambda_d-\lambda_n)+(\lambda_d-\lambda_{d+1}))$ in every unitarily invariant norm (Theorem 3.8).
- The block Krylov sequence $\mathcal K_t$ converges with a rate that improves with the oversampling $\rho=r-d$ via a free parameter $p\in[d,r]$; this refines the bound one would get from $\mathcal V_t\subset\mathcal K_t$ alone (Theorem 3.9).
- Replacing the unknown $\mathcal X$ in Algorithm 3.1 by a Rayleigh–Ritz or refined Rayleigh–Ritz approximation gives a computable algorithm that, in the numerical examples, stays competitive with block Krylov while storing substantially fewer vectors (Section 5).
- Setting $d=1$ recovers the previously known optimal expansion of Ye and Jia, so the results form a genuine block extension of that line of work (Remark 3.3).
Reading between the lines
- Because the optimal expansion requires the target $\mathcal X$, its practical role is as a benchmark; a natural extension would be to feed the same projection formula with a randomized sketch of $\mathcal X$ and prove a probabilistic error bound.
- The hypothesis $\mathcal X\cap\mathcal V^\perp=\{0\}$ is called generic but carries no probability estimate; combining the paper's geometric rate with standard concentration bounds for random Gaussian $\mathcal V$ would turn Theorem 3.8 into a quantitative high-probability statement.
- The improved Chebyshev separation estimate in Proposition 6.7 is stronger than the corresponding bound used in earlier randomized block Krylov analyses, so it could sharpen known convergence constants for dominant-subspace and singular-value approximation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the problem of expanding a current subspace V, of dimension r >= d, to V + A(W0) with W0 a subspace of V of dimension at most d, so as to approximate a d-dimensional A-invariant target subspace X in the sense of principal angles. It proves the existence of such optimal W0 and gives an explicit characterization in terms of the pseudo-inverse R†(X) (Theorem 3.2, Corollary 3.4). It then defines Algorithm 3.1, an iterative theoretical procedure that is step-wise optimal but depends on the unknown X, and shows that in the Hermitian case with X equal to the top-d eigenspace and λd > λd+1, the angle between X and the iterates converges linearly to zero under the non-degeneracy condition X∩V⊥={0} (Theorem 3.8). A related convergence bound for block Krylov subspaces is given in Theorem 3.9. Computable versions via Rayleigh-Ritz and refined Rayleigh-Ritz are proposed as Algorithm 3.2, and numerical experiments compare the algorithms on diagonal matrices with different eigenvalue decays.
Significance. The paper provides a natural generalization of the single-vector optimal expansion results of Ye and Jia to the block case d > 1, with a clean proof strategy based on the subspace PS(X) and pseudo-inverse formulas. The explicit characterization of all optimal expansions and the embedding of the optimal subspaces in block Krylov spaces are valuable structural insights. The proofs are detailed, and the auxiliary technical results (e.g., Theorem 6.2) are of independent interest. The paper is honest about the non-computability of the theoretical algorithm and presents the computable algorithm as a heuristic benchmark. Despite the issues noted below, the core mathematical claims in Theorems 3.2 and 3.8 are well supported.
major comments (2)
- [Abstract / Theorem 3.8] The abstract states that the iterative sequence approximates X arbitrarily well when A is Hermitian and X is simple, but it omits the necessary hypothesis X∩V⊥={0} that appears in Theorem 3.8. This condition is not merely technical: if a nonzero x∈X is orthogonal to V0, then for Hermitian A, x is orthogonal to A(V0) and therefore to every iterate V_t, so the iterates cannot converge to X (indeed, the corresponding principal angle stays π/2 and its tangent is undefined). The theorem statement is correct, but the abstract's unqualified claim is false. Please include the condition, or an explicit 'generic' qualifier, in the abstract and in any summary of the convergence result.
- [Theorem 3.9 / Proposition 4.7] The statement of Theorem 3.9 allows any d ≤ p ≤ r but does not require λ_{p+1} > λ_n. The bound in Eq. (8) contains (λ_{p+1} − λ_n) in the denominator, and the proof invokes Proposition 4.7 item 2, which uses the Chebyshev polynomial T_t((x−λ_n)/(λ_{p+1}−λ_n)). When λ_{p+1}=λ_n these expressions are undefined, so the theorem as stated is not meaningful for all allowed p. Add the hypothesis λ_{p+1} > λ_n to Theorem 3.9 and Proposition 4.7 item 2, or specify a limiting interpretation for the degenerate case.
minor comments (7)
- [Section 2] The phrase 'it's Moore–Penrose pseudo-inverse' should be 'its Moore–Penrose pseudo-inverse'.
- [Sections 3.3 and 5] The name 'Raleigh-Ritz' is misspelled; it should be 'Rayleigh-Ritz'.
- [Figure 3] The caption 'Eliptical decay models' should read 'Elliptical decay models'.
- [Section 3.3] The sentence 'Based on the previous ideas we consider the following' before Algorithm 3.2 is incomplete and should be finished or removed.
- [Theorem 3.9, Eq. (8)] The typesetting of the exponent 3^{−t·min{...}} in Eq. (8) is ambiguous; please format the expression so that the argument of min and the exponentiation are clear.
- [Sections 3.2 and 5] The paper calls the hypotheses X∩V⊥={0} and span{x1,...,xr}∩V⊥={0} 'generic' but gives no probability estimate. Since the numerical section uses a single Gaussian draw per configuration, a brief statement or reference on the probability of the non-degeneracy condition would strengthen the presentation.
- [Section 6.4] In Proposition 6.5, 'Chebysehv polynomials' should be 'Chebyshev polynomials'.
Circularity Check
No significant circularity: optimal expansions are explicitly non-computable and the convergence bounds are derived from spectral data, not fitted or self-cited.
full rationale
The central theoretical construction (Theorem 3.2, Corollary 3.4, Algorithm 3.1) chooses W0 from the projection of the target subspace X onto S = V + A(V), and the paper states plainly that this makes the expansion optimal but not computable ('although the expansions V + A(W0) are optimal, they are not computable'). This is not a hidden prediction: it is an explicit optimization characterization, and no fitted quantity is renamed as a result. The quantitative convergence claims (Theorems 3.8 and 3.9) are derived from the eigenvalues λi of A through polynomial filter estimates (Proposition 4.7), with constants such as (λd+1 − λn)/((λd − λn) + (λd − λd+1)) computed from A, not fitted to the numerical examples. The stated assumptions X∩V⊥ = {0} and span{x1,...,xr}∩V⊥ = {0} are genuine non-degeneracy conditions needed for the tangent of principal angles to be finite; they are correctness conditions, not circularity. References to Jia [10], Ye [27] and Zhu-Knyazev [28] are external prior results used as tools, and no load-bearing self-citation chain appears. The numerical section benchmarks the stated algorithms and upper bounds; it does not supply the theorems.
Assumptions & free parameters
assumptions (7)
- standard math Interlacing inequalities and monotonicity of principal angles under subspace inclusion (Proposition 4.1)
- standard math Zhu-Knyazev identity: for orthonormal bases, tangents of principal angles are singular values of X_⊥^* V (X^* V)^†, and the inequality in Theorem 4.5/6.2
- standard math Chebyshev polynomial growth estimates (Propositions 6.6 and 6.7)
- domain assumption Target subspace X is the span of eigenvectors for the d largest eigenvalues with λd > λd+1
- domain assumption Non-degeneracy: X ∩ V⊥ = {0} in Theorem 3.8 and span{x1..xr} ∩ V⊥ = {0} in Theorem 3.9
- domain assumption Unstated condition λp+1 > λn for the Chebyshev rescaling in Proposition 4.7 and Theorem 3.9
- domain assumption Exact arithmetic in the theoretical algorithms; no roundoff analysis is included
Cite this review
Pith. "Pith review of Block subspace expansions for eigenvalues and eigenvectors approximation." pith.science (2026). https://pith.science/paper/723ZARWL
@misc{pith2026241114578,
author = {Pith},
title = {Pith review of: Block subspace expansions for eigenvalues and eigenvectors approximation},
year = {2026},
howpublished = {\url{https://pith.science/paper/723ZARWL}},
note = {Machine review of arXiv:2411.14578}
}
abstract
Let $A\in\mathbb C^{n\times n}$ and let $\mathcal X\subset \mathbb C^n$ be an $A$-invariant subspace with $\dim \mathcal X=d\geq 1$, corresponding to exterior eigenvalues of $A$. Given an initial subspace $\mathcal V\subset \mathbb C^n$ with $\dim \mathcal V=r\geq d$, we search for expansions of $\mathcal V$ of the form $\mathcal V+A(\mathcal W_0)$, where $\mathcal W_0\subset \mathcal V$ is such that $\dim \mathcal W_0\leq d$ and such that the expanded subspace is closer to $\mathcal X$ than the initial $\mathcal V$. We show that there exist (theoretical) optimal choices of such $\mathcal W_0$, in the sense that $\theta_i(\mathcal X,\mathcal V+A(\mathcal W_0))\leq \theta_i(\mathcal V+A(\mathcal W))$ for every $\mathcal W\subset \mathcal V$ with $\dim \mathcal W\leq d$, where $\theta_i(\mathcal X,\mathcal T)$ denotes the $i$-th principal angle between $\mathcal X$ and $\mathcal T$, for $1\leq i\leq d\leq \dim \mathcal T$. We relate these optimal expansions to block Krylov subspaces generated by $A$ and $\mathcal V$. We also show that the corresponding iterative sequence of subspaces constructed in this way approximate $\mathcal X$ arbitrarily well, when $A$ is Hermitian and $\mathcal X$ is simple. We further introduce computable versions of this construction and compute several numerical examples that show the performance of the computable algorithms and test our convergence analysis.
Figures
Reference graph
Works this paper leans on
-
[1]
R. Bhatia, Matrix analysis. Graduate Texts in Mathematics, 169. Springer-Verlag, New York, 1997
work page 1997
-
[2]
P. Drineas, I.F. Ipsen, E.M. Kontopoulou, M. Magdon-Ismail, Structural convergence results for approximation of dominant subspaces from block Krylov spaces. SIAM J. Matrix Anal. Appl. 39 (2018), no. 2, 567-586
work page 2018
-
[3]
Gu, Subspace Iteration Randomization and Singular Value Problems
M. Gu, Subspace Iteration Randomization and Singular Value Problems. SIAM Journal on Scientific Computing, vol. 37, n.3, January 2015, pp. A1139-73., 10.1137/130938700
-
[4]
Hochstenbach, A Jacobi-Davidson type SVD method
M. Hochstenbach, A Jacobi-Davidson type SVD method. Copper Mountain Conference (2000). SIAM J. Sci. Comput. 23 (2001), no. 2, 606-628
work page 2000
-
[5]
Z. Jia, The convergence of generalized Lanczos methods for large unsymmetric eigenproblems, SIAM Journal on Matrix Analysis and Applications 16 (1995), 843–862
work page 1995
-
[6]
Z. Jia, Refined iterative algorithms based on Arnoldi’s process for unsymmetric eigenproblems, Linear Algebra Appl., 259 (1997), pp. 1-23
work page 1997
-
[7]
Z. Jia, Generalized block Lanczos methods for large unsymmetric eigenproblems, Numerische Mathematik 80 (1998), 171–189
work page 1998
-
[8]
Jia, Some theoretical comparisons of refined Ritz vectors and Ritz vectors, Science in China Ser
Z. Jia, Some theoretical comparisons of refined Ritz vectors and Ritz vectors, Science in China Ser. A Math., 47 (2004), Supp. 222-233
work page 2004
Show all 28 references
-
[9]
Jia, The convergence of harmonic Ritz values, harmonic Ritz vectors and refined harmonic Ritz vectors, Math
Z. Jia, The convergence of harmonic Ritz values, harmonic Ritz vectors and refined harmonic Ritz vectors, Math. Comput., 74 (2005), pp. 1441-1456
2005
-
[10]
Jia, Theoretical and computable optimal subspace expansions for matrix eigenvalue problems
Z. Jia, Theoretical and computable optimal subspace expansions for matrix eigenvalue problems. SIAM J. Matrix Anal. Appl. 43 (2022), no. 2, 584-604
2022
-
[11]
Jia and C
Z. Jia and C. Li, Inner iterations in the shift-invert residual Arnoldi method and the Jacobi– Davidson method, Sci. China Math., 57 (2014), pp. 1733-1752
2014
-
[12]
Jia and C
Z. Jia and C. Li, Harmonic and refined harmonic shift-invert residual Arnoldi and Jacobi– Davidson methods for interior eigenvalue problems, J. Comput. Appl. Math., 282 (2015), pp. 83-97
2015
-
[13]
Jia, G.W
Z. Jia, G.W. Stewart, An analysis of the Rayleigh-Ritz method for approximating eigenspaces. Math. Comp. 70 (2001), no. 234, 637-647
2001
-
[14]
Lee, Residual Arnoldi method: theory, package and experiments, Ph
C. Lee, Residual Arnoldi method: theory, package and experiments, Ph. D thesis, TR-4515, Department of Computer Science, University of Maryland at College Park, 2007
2007
-
[15]
Lee and G
C. Lee and G. W. Stewart, Analysis of the residual Arnoldi method, TR-4890, Department of Computer Science, University of Maryland at College Park, 2007
2007
-
[16]
Qiu, Li, Yanxia Zhang, Chi-Kwong Li, Unitarily Invariant Metrics on the Grassmann Space. SIAM J. Matrix Anal. Appl. 27 (2005), no.2, 10.1137/040607605
2005 doi
-
[17]
J. C. Mason, D. C. Handscomb, Chebyshev polynomials. Chapman & Hall/CRC, Boca Raton, FL, 2003
2003
-
[18]
Cameron Musco, Christopher Musco, Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition, 2015, https://arxiv.org/abs/1504.05477
2015 arXiv
-
[19]
Saad, Numerical Methods for Large Eigenvalue Problems, Revised Version, SIAM, Philadephia, PA, 2011
Y. Saad, Numerical Methods for Large Eigenvalue Problems, Revised Version, SIAM, Philadephia, PA, 2011
2011
-
[20]
40, n.o 1, January 2019, pp
Saibaba, Arvind K., Randomized Subspace Iteration: Analysis of Canonical Angles and Unitarily Invariant Norms, SIAM Journal on Matrix Analysis and Applications, vol. 40, n.o 1, January 2019, pp. 23-48
2019
-
[21]
G.; Van der Vorst, Henk A
Sleijpen, Gerard L. G.; Van der Vorst, Henk A. A Jacobi-Davidson iteration method for linear eigenvalue problems. SIAM J. Matrix Anal. Appl. 17 (1996), no. 2, 401–425
1996
-
[22]
G.; Van der Vorst, Henk A
Sleijpen, Gerard L. G.; Van der Vorst, Henk A. A Jacobi-Davidson iteration method for linear eigenvalue problems. SIAM Rev. 42 (2000), no. 2, 267-293
2000
-
[23]
G. W. Stewart, Matrix Algorithms II: Eigensystems, SIAM, Philadelphia, PA, 2001
2001
-
[24]
Van der Vorst, Computational Methods for Large Eigenvalue Problems, Elsevier, North Hollands, 2002
H. Van der Vorst, Computational Methods for Large Eigenvalue Problems, Elsevier, North Hollands, 2002
2002
-
[25]
S. Wang, Z. Zhang, Tong Zhang, Improved Analysis of the Randomized Power Method and Block Lanczos Method (2015) https://arxiv.org/abs/1508.06429
2015 arXiv
-
[26]
Wu and L
G. Wu and L. Zhang, On expansion of search subspaces for large non-Hermitian eigenprob- lems, Linear Algebra Appl., 454 (2014), pp. 107-129
2014
-
[27]
Ye, Optimal expansion of subspaces for eigenvector approximations
Q. Ye, Optimal expansion of subspaces for eigenvector approximations. Linear Algebra Appl. 428 (2008), no. 4, 911-918
2008
-
[28]
Zhu, A.V
P. Zhu, A.V. Knyazev, Angles between subspaces and their tangents. J. Numer. Math. 21 (2013), no. 4, 325-340. 27
2013
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.