REVIEW 3 major objections 4 minor 36 references
Randomized Krylov methods for inverse problems
T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Randomized Golub-Kahan bidiagonalization extends randomized Krylov solvers to rectangular inverse problems, producing regularized solutions comparable to deterministic Krylov methods when the sketch dimensions are large enough.
desk verdict A solid methods paper: the rGK factorization and its three solvers are genuinely new, the derivations hold up in exact arithmetic, and the main gaps are unquantified stability of the sketched Gram-Schmidt and missing runtime evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the randomized Golub-Kahan (rGK) factorization, built by the randomized Gram-Schmidt process with two sketching matrices, $\Theta^{(n)}$ for vectors in $\mathbb{R}^n$ and $\Theta^{(m)}$ for vectors in $\mathbb{R}^m$. At iteration $k$ it maintains stacks of sketched basis vectors and computes coefficients by sketched inner products, yielding the two factorizations in (2.14). These factorizations carry the argument because each solver is obtained by imposing a different condition on the residual or normal-equations residual and then reading off the projected problem from $M_k$ and $T_{k+1}$; the hybrid variants regularize those projected problems with Tikhonov terms and evaluate parameter-choice functionals cheaply.
What would settle it
Run Algorithm 2.2 on a rectangular inverse problem, such as the seismic tomography example with $A$ of size $131072\times 65536$, while increasing the sketch dimensions $\ell_m/\ell_n$ across a range and recording the relative reconstruction error and the orthogonality measure $\|I - V_k^\top V_k\|_F$ per iteration; if even at large sketch sizes the randomized solution error does not approach the deterministic LSQR error or the basis orthogonality degrades linearly with iteration count, the claimed accuracy equivalence fails.
Extended reading notes
Core claim
The central claim is that the randomized Golub-Kahan algorithm, Algorithm 2.2, gives exact-arithmetic factorizations $AV_k = U_{k+1}M_k$ and $A^\top U_{k+1} = V_{k+1}T_{k+1}$ for a rectangular matrix $A$, where the columns of $V$ and $U$ are orthogonal with respect to the sketched inner products and $M_k$, $T_{k+1}$ are upper Hessenberg and upper triangular. From these factorizations the paper derives new solvers rLSQR, rCGLS, and rLSMR that generalize the corresponding deterministic Krylov methods: rLSQR minimizes the sketched residual norm over the Krylov subspace, rLSMR minimizes the sketched normal-equations residual, and rCGLS enforces a different orthogonality condition that has no short recurrence. The same framework yields hybrid Tikhonov-regularized variants in which the regularization parameter can be chosen at each iteration by the discrepancy principle or weighted GCV, because the approximation subspace does not depend on that parameter.
Load-bearing premise
The load-bearing premise is that the sketching matrices accurately preserve inner products over the Krylov subspaces generated during iteration; if the sketch size is too small, the basis vectors are not sufficiently orthogonal and the iterations become erratic.
Editorial extensions
If this is right
- Rectangular inverse problems, including underdetermined and overdetermined ones, can now be attacked with randomized Krylov methods, not only square systems as with rGMRES.
- When the sketch dimensions are chosen large enough, rLSQR, rCGLS, and rLSMR reproduce the semiconvergence behavior of standard LSQR, CGLS, and LSMR, so early termination still acts as regularization.
- Hybrid randomized methods inherit the key property of deterministic hybrids: the approximation subspace is independent of the regularization parameter, so $\lambda$ can be tuned at each iteration at negligible extra cost.
- For problems needing only few iterations, such as high noise or severe ill-conditioning, smaller sketch sizes suffice, widening the regime of computational savings.
- The rGK projected matrices approximate the dominant singular values of $A$, with rCGLS and rLSMR giving better approximations than rLSQR, which affects both iterative and hybrid regularization behavior.
Reading between the lines
- A natural testable extension, not pursued in the paper, is an adaptive scheme that enlarges the sketch dimensions when the sketched basis vectors lose orthogonality, rather than fixing the sketch size a priori.
- Because the paper leaves the theoretical properties of the low-rank approximations from rGK open, establishing those properties could open the same factorizations to matrix-function estimation or uncertainty quantification in Bayesian inverse problems.
- The paper notes that storage grows linearly with iterations because short recurrences are lost; combining rGK with the recycling strategies already mentioned for hybrid methods could keep memory bounded in long runs.
- The observed sensitivity to sketch size suggests that a practical implementation should monitor the sketched orthogonality and terminate before semiconvergence deterioration, an operational rule the paper does not fully specify.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops randomized Krylov subspace methods for large-scale linear inverse problems with rectangular forward operators A. It extends the randomized Gram-Schmidt approach of Balabanov and Grigori to a randomized Golub-Kahan (rGK) algorithm, which uses two sketching matrices and produces the factorizations AV_k = U_{k+1}M_k and A^T U_{k+1} = V_{k+1}T_{k+1}. On top of these factorizations the authors derive randomized LSQR, CGLS, and LSMR solvers and hybrid Tikhonov-regularized variants with automatic parameter selection via discrepancy principle and weighted GCV. The paper also reviews the randomized Arnoldi/GMRES setting and presents numerical experiments for image deblurring and seismic tomography.
Significance. If the algorithmic claims hold, this is a useful contribution: it extends randomized inner-product-free Krylov techniques from square to rectangular inverse problems, provides new projected solvers with exact-arithmetic derivations, and introduces hybrid variants with adaptive regularization. The derivations of the factorizations and projected systems in Section 2 are careful and self-contained, and the paper is honest about its scope, including the lack of a theoretical analysis of the low-rank approximations. The numerical experiments do show that the methods can work when the sketch sizes are sufficiently large. The main weakness is that the central accuracy and optimality claims depend on the preservation of sketched orthonormality by an un-reorthogonalized classical Gram-Schmidt process, and this is not analyzed or tested. The efficiency motivation is also supported only by flop counts, not by runtime experiments.
major comments (3)
- [Section 2.2, Algorithm 2.2, Eqs. (2.14), (2.19)-(2.27)] The derivations of rLSQR, rCGLS, and rLSMR are exact only if the columns of Θ(m)U_{k+1} and Θ(n)V_{k+1} are orthonormal, since identities such as (2.20) and (2.27) use that orthonormality explicitly. Algorithm 2.2 computes these bases by a single classical Gram-Schmidt sweep with coefficients and norms obtained from sketched vectors (steps 13-17 and 21-25). In floating-point arithmetic, classical Gram-Schmidt can lose orthogonality at a rate governed by the conditioning of the basis, and for severely ill-conditioned inverse problems this can make the factorizations (2.14) inaccurate. The only evidence offered for the stability of the simplified orthogonalization is the undocumented empirical remark in Section 2.1 that no substantial differences were observed on a variety of inverse problems. Since the residual-optimality properties of the proposed solvers collapse when the sketched orthonormality is lost, the paper should either add a reorthogonalized or otherwise stabilized variant, report quantitative measurements of the loss of Θ-orthogonality, or provide an error analysis that bounds the effect of this loss on the factorizations and projected solutions. The erratic behavior at small sketch sizes in Figures 5 and 7 is consistent with this concern, but the paper does not distinguish it from insufficient sketching.
- [Section 4, cost claims in Section 2.2, Eq. (2.16)] The claimed computational advantage of rGK over GKB and reorthogonalized GKB is based entirely on flop-count estimates and the synthetic cost comparisons in Figure 1. The numerical experiments in Section 4 report only relative reconstruction errors and do not include any wall-clock times, iteration timings, or comparisons of actual runtime for rGK versus GKB/ro-GKB. Because the main motivation stated in the introduction is the reduction of computational cost, the paper needs at least a basic runtime validation on the reported test problems to confirm that the flop-count advantages translate into practice, especially given the full reorthogonalization and storage requirements of the randomized bases.
- [Section 3, Eqs. (3.15)-(3.16)] The discrepancy principle and weighted GCV are adapted to randomized hybrid methods by replacing the true residual with the sketched residual Θ(m)(b - Ax_k(λ)). No error analysis is provided that relates the λ selected by these sketched criteria to the λ that would be selected with the true residual, and the numerical results in Figure 11 show that the GCV-based parameter can visibly overestimate the optimal value. Since the adaptive selection of λ is one of the advertised features of the hybrid methods, the heuristic nature of these modifications should be stated more prominently, or a perturbation bound in terms of the embedding quality ε and the sketch dimensions should be supplied.
minor comments (4)
- [Section 3, Eq. (3.16)] The numerator of the WGCV functional is written as n ||Θ(m)(b - Ax_k(λ))||^2, but the standard GCV formula for an m-dimensional data space uses the factor m; since the denominator traces involve the m×m matrix AA_reg^†(λ), the factor n appears to be a typo and should be m or justified explicitly.
- [Section 4.1, Figure 5 and surrounding text] The sentence 'Similar results are observed for rGMRES-GCV (shown in the right plot of Figure 4)' should refer to Figure 5, since Figure 4 is the comparison with deterministic GMRES and Figure 5 is the one with varying ℓ_n for hybrid rGMRES with GCV.
- [Section 4.1] There is a typo in 'we use the SHRT' which should read 'SRHT'.
- [Section 2.3, Eqs. (2.24)-(2.26)] The notation e_{k+1}^T for the k×(k+1) matrix [I_k, 0]T_{k+1} is confusing, because e_{k+1} is elsewhere used for the (k+1)-st canonical basis vector; a different symbol, for instance T_{k+1}^{(k)}, would improve readability.
Circularity Check
No significant circularity: the new solver derivations are algebraic consequences of the rGK factorizations, and the flagged stability caveats are evidence gaps rather than circular reasoning.
full rationale
The central derivation chain in Sections 2.2 and 2.3 is self-contained. Algorithm 2.2 computes the rGK coefficients directly from sketched inner products, so the factorizations AV_k = U_{k+1}M_k and A^T U_{k+1} = V_{k+1}T_{k+1} in (2.14) hold by construction in exact arithmetic, and the Θ-orthonormality of the sketched basis vectors is enforced by the same recursion. The optimality statements (2.20), (2.24), and (2.26)-(2.27) therefore follow by substitution from these factorizations together with the definition of the algorithm; no fitted parameter is later relabelled as a prediction, and no result is imported from the authors' prior work to force the conclusions. The rGMRES residual bound in Section 2.1 is a direct consequence of the stated (ε,δ,K+1)-oblivious embedding assumption, not of a self-citation. The paper's self-citations are to standard hybrid projection and weighted GCV tools, which are independent, externally established methods rather than load-bearing proofs of the new randomized factorizations. Per the reviewing rule, I flag two non-circular evidence gaps: Section 2.1's remark 'we did not observe substantial differences when testing on a variety of inverse problems' supports the un-reorthogonalized classical Gram-Schmidt choice without presenting those tests, and Section 5 states that the complexity analysis and the theoretical properties of the rGK low-rank approximations are still future work. These are stability and evidence limitations, not circularity, and they do not make the derivation equivalent to its inputs.
Assumptions & free parameters
free parameters (2)
- Sketch dimension ℓ_n (for Θ(n)) =
13106 (5% of n) for deblurring; 482 (0.7% of n) for tomography, per (2.18)
- Sketch dimension ℓ_m (for Θ(m)) =
512 (0.4% of m) for tomography, per (2.18)
assumptions (4)
- domain assumption The sketching matrices Θ(m) and Θ(n) satisfy the (ε,δ,K+1)-oblivious subspace embedding property (equations (2.2)-(2.3)) with high probability.
- standard math Exact arithmetic is assumed in the factorizations (2.7) and (2.14) and in the derivations of the projected problems (2.19), (2.24), (2.26).
- domain assumption Standard parameter selection rules (discrepancy principle, weighted GCV) remain appropriate when applied to sketched residuals in (3.15)-(3.16).
- domain assumption The inverse problems under consideration satisfy standard assumptions for iterative regularization (e.g., discrete Picard condition), so that early termination or Tikhonov regularization yields meaningful solutions.
Cite this review
Pith. "Pith review of Randomized Krylov methods for inverse problems." pith.science (2026). https://pith.science/paper/QRPBYXQF
@misc{pith2026250820269,
author = {Pith},
title = {Pith review of: Randomized Krylov methods for inverse problems},
year = {2026},
howpublished = {\url{https://pith.science/paper/QRPBYXQF}},
note = {Machine review of arXiv:2508.20269}
}
read the original abstract
In this paper we develop randomized Krylov subspace methods for efficiently computing regularized solutions to large-scale linear inverse problems. Building on the recently developed randomized Gram-Schmidt process, where sketched inner products are used to estimate inner products of high-dimensional vectors, we propose a randomized Golub-Kahan approach that works for general rectangular matrices. We describe new iterative solvers based on the randomized Golub-Kahan approach and show how they can be used for solving inverse problems with rectangular matrices, thus extending the capabilities of the recently proposed randomized GMRES method. We also consider hybrid projection methods that combine iterative projection methods, based on both the randomized Arnoldi and randomized Golub-Kahan factorizations, with Tikhonov regularization, where regularization parameters can be selected automatically during the iterative process. Numerical results from image deblurring and seismic tomography show the potential benefits of these approaches.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[1]
N. Ailon and E. Liberty , Fast dimension reduction using Rademacher series on dual BCH codes, Discrete Comput. Geom., 42 (2009), p. 615
work page 2009
-
[2]
H. Avron, P. Maymounkov, and S. Toledo , Blendenpik: Supercharging LAPACK’s least- squares solver, SIAM Journal on Scientific Computing, 32 (2010), pp. 1217–1236, https: //doi.org/10.1137/090767911
-
[3]
J. Baglama, D. Calvetti, and L. Reichel , Iterative methods for the computation of a few eigenvalues of a large symmetric matrix , BIT Numerical Mathematics, 36 (1996), pp. 400– 421
work page 1996
-
[4]
O. Balabanov and L. Grigori , Randomized Gram–Schmidt process with application to GMRES, SIAM Journal on Scientific Computing, 44 (2022), pp. A1450–A1474, https: //doi.org/10.1137/20M138870X
-
[5]
A. N. Brown, J. Chung, J. G. Nagy, and M. Sabat ´e Landman, Inner-product free Krylov methods for large-scale inverse problems , SIAM Journal on Scientific Computing, (2025), pp. S161–S182
work page 2025
-
[6]
A. N. Brown, M. Sabat ´e Landman, and J. G. Nagy , H-CMRH: An inner product free hybrid Krylov method for large-scale inverse problems , SIAM Journal on Matrix Analysis and Applications, 46 (2025), pp. 232–255, https://doi.org/10.1137/24M1634874
-
[7]
D. Calvetti, S. Morigi, L. Reichel, and F. Sgallari, An L-ribbon for large underdetermined linear discrete ill-posed problems, Numerical Algorithms, 25 (2000), pp. 89–107
work page 2000
-
[8]
A. Chowdhury, J. Yang, and P. Drineas, An iterative, sketching-based framework for ridge regression, in Proceedings of the 35th International Conference on Machine Learning, J. Dy and A. Krause, eds., vol. 80 of Proceedings of Machine Learning Research, PMLR, 10–15 Jul 2018, pp. 989–998
work page 2018
Show all 36 references
-
[9]
Chung and S
J. Chung and S. Gazzola , Flexible Krylov methods for ℓ p regularization, SIAM Journal on Scientific Computing, 41 (2019), pp. S149–S171
2019
-
[10]
Chung and S
J. Chung and S. Gazzola, Computational methods for large-scale inverse problems: A survey on hybrid projection methods , SIAM Review, 66 (2024), pp. 205–284
2024
-
[11]
Chung, J
J. Chung, J. G. Nagy, and D. P. O’Leary , A weighted GCV method for Lanczos hybrid regularization, Elec. Trans. Numer. Anal., 28 (2008), pp. 149–167
2008
-
[12]
Chung and K
J. Chung and K. Palmer, A hybrid LSMR algorithm for large-scale Tikhonov regularization , SIAM Journal on Scientific Computing, 37 (2015), pp. S562–S580
2015
-
[13]
Cortinovis, D
A. Cortinovis, D. Kressner, and Y. Nakatsukasa , Speeding up Krylov subspace methods for computing via randomization , SIAM Journal on Matrix Analysis and Applications, 45 (2024), pp. 619–633
2024
-
[14]
Frommer and P
A. Frommer and P. Maass , Fast CG-based methods for Tikhonov-Phillips regularization , SIAM Journal on Scientific Computing, 20 (1999), pp. 1831–1850
1999
-
[15]
Gazagnadou, M
N. Gazagnadou, M. Ibrahim, and R. M. Gower , Ridgesketch: A fast sketching based solver for large scale ridge regression , SIAM Journal on Matrix Analysis and Applications, 43 26 CHUNG AND GAZZOLA (2022), pp. 1440–1468, https://doi.org/10.1137/21M1422963
2022 doi
-
[16]
Gazzola, P
S. Gazzola, P. C. Hansen, and J. G. Nagy, IR Tools: a MATLAB package of iterative regu- larization methods and large-scale test problems, Numer. Algorithms, 81 (2019), p. 773–811, https://doi.org/10.1007/s11075-018-0570-7
2019 doi
-
[17]
Gazzola and M
S. Gazzola and M. S. Landman , Krylov methods for inverse problems: Surveying classi- cal, and introducing new, algorithmic approaches , Mitteilungen der Gesellschaft f¨ ur Ange- wandte Mathematik und Mechanik, 43 (2020)
2020
-
[18]
Gazzola and P
S. Gazzola and P. Novati , Inheritance of the discrete Picard condition in Krylov subspace methods, BIT Numerical Mathematics, 56 (2016), pp. 893–918
2016
-
[19]
Gazzola, P
S. Gazzola, P. Novati, and M. R. Russo , On Krylov projection methods and Tikhonov regularization, Electron. Trans. Numer. Anal., 44 (2015), pp. 83–123
2015
-
[20]
P. C. Hansen, Discrete Inverse Problems: Insight and Algorithms , SIAM, Philadelphia, 2010
2010
-
[21]
C. Iyer, C. Carothers, and P. Drineas , Randomized sketching for large-scale sparse ridge regression problems, in 2016 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (ScalA), 2016, pp. 65–72, https://doi.org/10.1109/ScalA.2016.013
2016 doi
-
[22]
T. K. Jensen and P. C. Hansen , Iterative regularization with minimum-residual methods , BIT, 47 (2007), pp. 103–120
2007
-
[23]
Jiang, J
J. Jiang, J. Chung, and E. de Sturler, Hybrid projection methods with recycling for inverse problems, SIAM Journal on Scientific Computing, (2021), pp. S146–S172
2021
-
[24]
M. S. Landman, A. N. Brown, J. Chung, and J. G. Nagy , Randomized and inner-product free Krylov methods for large-scale inverse problems, Numerical Algorithms, (2025), https: //doi.org/10.1007/s11075-025-02167-w
2025 doi
-
[25]
Martinsson and J
P.-G. Martinsson and J. A. Tropp , Randomized numerical linear algebra: Founda- tions and algorithms , Acta Numerica, 29 (2020), p. 403–572, https://doi.org/10.1017/ S0962492920000021
2020
-
[26]
Meier and Y
M. Meier and Y. Nakatsukasa, Randomized algorithms for Tikhonov regularization in linear least squares, 2022, https://arxiv.org/abs/2203.07329
2022 arXiv
-
[27]
Meier, Y
M. Meier, Y. Nakatsukasa, A. Townsend, and M. Webb, Are sketch-and-precondition least squares solvers numerically stable? , SIAM Journal on Matrix Analysis and Applications, 45 (2024), pp. 905–929, https://doi.org/10.1137/23M1551973
2024 doi
-
[28]
X. Meng, M. A. Saunders, and M. W. Mahoney , LSRN: A parallel iterative solver for strongly over- or underdetermined systems , SIAM Journal on Scientific Computing, 36 (2014), pp. C95–C118, https://doi.org/10.1137/120866580
2014 doi
-
[29]
Musco and C
C. Musco and C. Musco , Randomized block Krylov methods for stronger and faster approxi- mate singular value decomposition, Advances in neural information processing systems, 28 (2015)
2015
-
[30]
J. Nagy, K. Palmer, and L. Perrone , Iterative methods for image deblurring: A MATLAB object oriented approach, Numerical Algorithms, 36 (2004), pp. 73–93
2004
-
[31]
Nakatsukasa and J
Y. Nakatsukasa and J. A. Tropp, Fast and accurate randomized algorithms for linear systems and eigenvalue problems , SIAM Journal on Matrix Analysis and Applications, 45 (2024), pp. 1183–1214, https://doi.org/10.1137/23M1565413
2024 doi
-
[32]
Saad, Iterative Methods for Sparse Linear Systems , SIAM, Philadelphia, 2nd ed., 2003
Y. Saad, Iterative Methods for Sparse Linear Systems , SIAM, Philadelphia, 2nd ed., 2003
2003
-
[33]
A. K. Saibaba, J. Chung, and K. Petroske , Efficient Krylov subspace methods for uncer- tainty quantification in large Bayesian linear inverse problems , Numerical Linear Algebra with Applications, 27 (2020), p. e2325
2020
-
[34]
J. A. Tropp , Randomized block Krylov methods for approximating extreme eigenvalues , Nu- merische Mathematik, 150 (2022), pp. 217–255
2022
-
[35]
Van Aarle, W
W. Van Aarle, W. J. Palenstijn, J. De Beenhouwer, T. Altantzis, S. Bals, K. J. Baten- burg, and J. Sijbers , The ASTRA toolbox: A platform for advanced algorithm develop- ment in electron tomography, Ultramicroscopy, 157 (2015), pp. 35–47
2015
-
[36]
D. P. Woodruff, Sketching as a tool for numerical linear algebra , Foun. Trends Theor. Com- put. Sci., 10 (2014), pp. 1–157
2014
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.