REVIEW 2 major objections 5 minor 47 references
Adaptive, Matrix-Free Low-Rank Approximation
T0 review · 2 major / 5 minor · reviewed 2026-07-10 · grok-4.5
Pith's one-line read Adaptive matrix-free QB algorithms meet fixed error tolerances down to machine precision while returning near-optimal ranks.
desk verdict Solid engineering paper that closes a real matrix-free gap with machine-precision residual tracking and rank pruning; spectral claims are only order-of-magnitude, as the authors themselves note. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The randomised residual indicator (||Y_i||_F or ||B_i||_2 obtained from the current sketch) together with the subsequent matrix-free rank-pruning step that discards the lowest-energy rows or singular directions of the last block.
What would settle it
Run the algorithms on a matrix whose singular values decay slowly enough that the look-ahead block of size 16–32 fails to capture the dominant residual subspace; if the returned residual then systematically exceeds the prescribed tolerance by more than the modest constant predicted by the theory, the claim is false.
Extended reading notes
Core claim
A family of four adaptive, matrix-free randomised QB algorithms (two Frobenius-norm, two spectral-norm, each with an adjoint-free counterpart) solves the fixed-tolerance low-rank approximation problem to machine precision by reading residual norms from already-computed random sketches and by pruning the final block after the tolerance is crossed, thereby decoupling BLAS-3 block size from the numerical rank.
Load-bearing premise
A modest fixed block size and a single Gaussian sketch already give a reliable enough proxy for residual norms and for the singular values needed by pruning, even though a rigorous subspace embedding would demand a block size that grows like one over tolerance squared.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a family of adaptive, matrix-free randomized QB algorithms for fixed-tolerance low-rank approximation of operators accessible only via matvecs. Four variants are proposed (matrix-free and adjoint-free, each in Frobenius and spectral norms). The main technical ingredients are (i) residual-norm indicators read from the same Gaussian sketches already used for range finding (Hutchinson-style for Frobenius; look-ahead block spectral norm for the 2-norm), remaining usable down to machine precision; (ii) a cheap post-iteration rank-pruning step that decouples BLAS-3 block size from final rank; and (iii) adjoint-free range-finder variants. Theoretical support is given via classical randomized-norm and subspace-embedding arguments (Theorems 2.2–2.3, 3.4, 4.3). Numerical experiments on three synthetic spectra (exponential, polynomial, S-shaped) compare against randQB_EI, randQB_HMT, and the truncated SVD, showing ranks near the t-SVD optimum and reliable tolerance satisfaction where prior adaptive methods fail or over-estimate rank.
Significance. The work closes a genuine practical gap: no previously published method is simultaneously matrix-free, rank-adaptive, and accurate to machine precision in both Frobenius and spectral norms. The rank-pruning idea is a clean, low-cost fix for the block-size dilemma that arises in blocked randomized QB, and the adjoint-free variants are useful when only Q is needed. Public MATLAB code that reproduces all experiments is a clear strength. The Frobenius algorithms rest on fully standard, parameter-free probabilistic identities and an exact Pythagorean pruning identity (Theorem 3.4); the spectral algorithms are more approximate but still produce substantially tighter ranks than the Halko–Martinsson–Tropp adaptive range finder. The contribution is therefore of immediate interest for matrix-free applications (implicit Hessians, H-matrix construction, ST-HOSVD) and is ready for use once the claim language is aligned with the spectral caveats already present in the body.
major comments (2)
- Abstract and §1 claim that the proposed methods meet the prescribed tolerance “with high probability” for both norms. For the Frobenius variants this is justified by the unbiased Hutchinson estimator and Theorem 3.4. For the spectral variants the body is more careful: Theorem 4.3 only guarantees an error ≤√(η²+1)ε, Remark 4.4 notes that η is uncontrolled for modest b, and the paragraph after Algorithm 4.3 explicitly weakens the AF spectral claim to “≲ε / on the order of the tolerance” because a genuine ε-subspace embedding would require b∼1/ε². The abstract and the opening claim should be qualified so that they match the spectral analysis already given in §4; otherwise the strongest claim overstates what is proved.
- §4.1–4.2 and Algorithm 4.1: the look-ahead indicator ||B_i||_2 is used both as a stopping criterion and (via the pruned last block) as the basis for the final residual bound. Theorem 4.3 absorbs the gap into the factor η from Theorem 2.3, but no quantitative guidance is given on how large b must be for η to stay near 1 on the spectra of interest. The numerical tables (e.g., Matrix 2, ε=10^{-3}, MF-Spec relative error 1.11×10^{-3}) show occasional mild overshoots. A short additional experiment or remark that reports the observed ratio ||E||_2 / ||B_i||_2 across the three test matrices would make the practical reliability of the spectral indicator transparent and would strengthen the central claim for the spectral methods.
minor comments (5)
- Remark 3.3 and the related-work discussion of Gorman et al. [17] and Pearce et al. [37] correctly credit prior use of Hutchinson-style residual estimates; it would help the reader if the precise novelty of the pruning step (Algorithm 3.2 / 4.2) relative to those works were stated in one sentence in §1.2.
- Figures 1–6: the right-hand panels that plot the error indicator versus iteration are useful, but the vertical axis ranges and the “fail” markers for randQB_EI could be explained more explicitly in the captions so that a reader skimming the figures understands the precision-wall phenomenon without returning to the text.
- Notation: the residual is written both E_i and E_{i-1} in different places; a single consistent indexing convention (e.g., always the residual after i blocks) would reduce minor confusion when reading Algorithms 3.1 and 4.1 side by side.
- §5.1: the choice p=4 (Frobenius) versus p=2 (spectral) for Matrix 2 is well motivated for stressing the baselines, but a one-sentence note that the proposed methods themselves are not restricted to these exponents would prevent a casual reader from thinking the algorithms are spectrum-specific.
- Typographical: “Nystr¨ om” appears with a broken umlaut in §1.1; “randQB HMT” / “randQB EI” spacing is inconsistent with the later “randQB_MF_Fro” style used in the algorithms.
Circularity Check
No circularity: algorithms rest on external randomized-NLA primitives (Hutchinson, subspace embeddings) whose guarantees are independent of the paper’s own numerics or definitions.
full rationale
The derivation chain is self-contained and non-circular. Residual indicators (||Yi||F for Frobenius, look-ahead ||Bi||2 or ||Yi||2 for spectral) are justified by classical external results (Hutchinson’s unbiased estimator Thm 2.2 / [25]; subspace-embedding singular-value preservation Lemma 2.4 / [32]; QB projection bounds Thm 2.3 / [42]) that do not depend on the paper’s algorithms or test matrices. Rank-pruning identities (Thm 3.4 Pythagorean additivity of row norms; Eckart–Young truncation of the final block) follow directly from orthonormality of the QB factors already constructed, not from any fitted quantity or self-referential definition. Block size b and oversampling are chosen a priori (b=16/32); no free parameters are fitted to the three synthetic spectra. Relative-norm estimates of ||A|| are likewise obtained from the same independent Gaussian sketches already required for range finding (Remarks 3.1, 4.1, 4.5). The only mild reuse—employing the range-finding sketch also as residual estimator—is an efficiency observation, not a definitional loop. Spectral claims are explicitly weakened by the authors themselves to “on the order of” the tolerance when b is modest (remark after Alg. 4.3; Thm 4.3), so no over-claim is smuggled in. Numerical tables serve only as empirical validation against t-SVD and prior adaptive baselines; they are not used to derive or calibrate any theoretical statement. Consequently the strongest claim (adaptive matrix-free QB meeting a prescribed tolerance down to machine precision with near-optimal rank) does not reduce to its own inputs by construction.
Assumptions & free parameters
free parameters (2)
- block size b =
16 or 32 (user-chosen)
- oversampling / safety constants (implicit p, η)
assumptions (4)
- standard math Hutchinson’s unbiased Frobenius-norm estimator (Thm. 2.2) and its concentration for modest sample size b
- standard math Gaussian subspace embeddings preserve singular values up to (1±ε) factors (Lemma 2.4)
- standard math The QB residual after i-1 blocks is orthogonal to the newly computed basis block Qi (used in Thm. 3.4 and the Pythagorean argument of Thm. 4.3)
- domain assumption A modest fixed block size already captures the dominant singular subspace of the residual with high probability, so that ||Bi||₂ ≈ ||Ei-1||₂
Cite this review
Pith. "Pith review of Adaptive, Matrix-Free Low-Rank Approximation." pith.science (2026). https://pith.science/paper/L3DGTDII
@misc{pith2026260706758,
author = {Pith},
title = {Pith review of: Adaptive, Matrix-Free Low-Rank Approximation},
year = {2026},
howpublished = {\url{https://pith.science/paper/L3DGTDII}},
note = {Machine review of arXiv:2607.06758}
}
abstract
We study fixed-tolerance low-rank approximation in the matrix-free setting, where a matrix or linear operator $\mathbf{A}$ is accessible only through matrix-vector products and its rank must be determined adaptively to meet a prescribed error tolerance. We introduce a family of adaptive, matrix-free randomized QB algorithms. A randomized error indicator estimates the residual norm -- in either the Frobenius or the spectral norm -- directly from a random sketch, remaining accurate down to machine precision. A matrix-free rank-pruning step decouples the computational block size from the final rank, so that large, BLAS-3-friendly blocks can be used without over-estimating the rank, and an adjoint-free variant returns the orthonormal basis using only the forward operator. Across test matrices with diverse singular-value decays, the proposed methods attain ranks close to the truncated-SVD optimum while meeting the prescribed tolerance with high probability.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Bebendorf , Approximation of boundary element matrices , Numerische Mathematik, 86 (2000), pp
M. Bebendorf , Approximation of boundary element matrices , Numerische Mathematik, 86 (2000), pp. 565–589
work page 2000
-
[2]
M. Bebendorf, Hierarchical Matrices: A Means to Efficiently Solve Elliptic Boundary Value Problems, Springer, 2008
work page 2008
-
[3]
E. K. Bjarkason , Pass-efficient randomized algorithms for low-rank matrix approximation using any number of views , 2018, https://arxiv.org/abs/1804.07531, https://arxiv.org/ abs/1804.07531
work page Pith review arXiv 2018
-
[4]
T. Bui-Thanh, O. Ghattas, J. Martin, and G. Stadler , A computational framework for infinite-dimensional bayesian inverse problems part i: The linearized case, with application to global seismic inversion , SIAM Journal on Scientific Computing, 35 (2013), pp. A2494– A2523
work page 2013
-
[5]
Carrel, Randomized methods for dynamical low-rank approximation , 2024, https://arxiv
B. Carrel, Randomized methods for dynamical low-rank approximation , 2024, https://arxiv. org/abs/2410.17091, https://arxiv.org/abs/2410.17091. 24 ARNEL I. SMITH, ELLY DO, AND CHAO CHEN
-
[6]
S. Chandrasekaran, P. Dewilde, M. Gu, and N. Somasunderam , On the numerical rank of the off-diagonal blocks of schur complements of discretized elliptic pdes , SIAM Journal on Matrix Analysis and Applications, 31 (2010), pp. 2261–2290
work page 2010
-
[7]
Y. Chen, E. N. Epperly, J. A. Tropp, and R. J. Webber , Randomly pivoted cholesky: Practical approximation of a kernel matrix with few entry evaluations , Communications on Pure and Applied Mathematics, 78 (2025), pp. 995–1041
work page 2025
- [8]
Show all 47 references
-
[9]
De Lathauwer, B
L. De Lathauwer, B. De Moor, and J. Vandewalle , A multilinear singular value decom- position, SIAM journal on Matrix Analysis and Applications, 21 (2000), pp. 1253–1278
2000
-
[10]
Y. Dong, C. Chen, P.-G. Martinsson, and K. Pearce , Robust blockwise random pivoting: Fast and accurate adaptive interpolative decomposition, SIAM Journal on Matrix Analysis and Applications, 46 (2025), pp. 1791–1815
2025
-
[11]
J. A. Duersch and M. Gu , Randomized qr with column pivoting , SIAM Journal on Scientific Computing, 39 (2017), pp. C263–C291
2017
-
[12]
H. P. Flath, L. C. Wilcox, V. Akc ¸elik, J. Hill, B. van Bloemen Waanders, and O. Ghat- tas, Fast algorithms for bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial hessian approximations , SIAM Journal on Scientific Computing, ...
2011
-
[13]
Ghysels, T
P. Ghysels, T. Mary, X. S. Li, K. Turkiyyah, and S. Soubh , Robust and accurate stopping criteria for adaptive randomized sampling in matrix-free HSS construction , SIAM Journal on Scientific Computing, 38 (2016), pp. S323–S347
2016
-
[14]
Gittens and M
A. Gittens and M. Mahoney, Revisiting the nystrom method for improved large-scale machine learning, in International Conference on Machine Learning, PMLR, 2013, pp. 567–575
2013
-
[15]
G. H. Golub and C. F. Van Loan , Matrix Computations , The Johns Hopkins University Press, Baltimore, 4th ed., 2013
2013
-
[16]
G. H. Golub and C. F. Van Loan , Matrix Computations , Johns Hopkins University Press, 4th ed., 2013
2013
-
[17]
Gorman, G
C. Gorman, G. Ch ´avez, P. Ghysels, T. Mary, F.-H. Rouet, and X. S. Li , Robust and accurate stopping criteria for adaptive randomized sampling in matrix-free hierarchically semiseparable construction, SIAM Journal on Scientific Computing, 41 (2019), pp. S61– S85
2019
-
[18]
Gratton and D
S. Gratton and D. Titley-Peloquin , Improved bounds for small-sample estimation , SIAM Journal on Matrix Analysis and Applications, 39 (2018), pp. 922–931
2018
-
[19]
Gu, Subspace iteration randomization and singular value problems , SIAM Journal on Sci- entific Computing, 37 (2015), pp
M. Gu, Subspace iteration randomization and singular value problems , SIAM Journal on Sci- entific Computing, 37 (2015), pp. A1139–A1173
2015
-
[20]
Gu and S
M. Gu and S. C. Eisenstat , Efficient algorithms for computing a rank-revealing QR factor- ization, SIAM Journal on Scientific Computing, 17 (1996), pp. 848–869
1996
-
[21]
Hackbusch, Hierarchical Matrices: Algorithms and Analysis , Springer, 2015
W. Hackbusch, Hierarchical Matrices: Algorithms and Analysis , Springer, 2015
2015
-
[22]
Halko, P.-G
N. Halko, P.-G. Martinsson, and J. A. Tropp , Finding structure with randomness: Proba- bilistic algorithms for constructing approximate matrix decompositions , SIAM review, 53 (2011), pp. 217–288
2011
-
[23]
Heavner, C
N. Heavner, C. Chen, A. Gopal, and P.-G. Martinsson, Efficient algorithms for computing rank-revealing factorizations on a gpu , Numerical Linear Algebra with Applications, 30 (2023), p. e2515
2023
-
[24]
Heldring, E
A. Heldring, E. Ubeda, and J. M. Rius , On the accuracy of the Adaptive Cross Approxima- tion algorithm, in XI Iberian Meeting on Computational Electromagnetics, 2016, pp. 74–75
2016
-
[25]
Hutchinson , A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines, Communication in Statistics- Simulation and Computation, 18 (1989), pp
M. Hutchinson , A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines, Communication in Statistics- Simulation and Computation, 18 (1989), pp. 1059–1076, https://doi.org/10.1080/03610919008812866
1989 doi
-
[26]
Levitt and P.-G
J. Levitt and P.-G. Martinsson , Linear-complexity black-box randomized compression of rank-structured matrices, SIAM Journal on Scientific Computing, 46 (2024), pp. A1747– A1763, https://doi.org/10.1137/22m1528574, http://dx.doi.org/10.1137/22M1528574
2024 doi
-
[27]
L. Lin, J. Lu, and L. Ying , Fast construction of hierarchical matrix representation from matrix-vector multiplication , Journal of Computational Physics, 230 (2011), pp. 4071– 4087
2011
-
[28]
Liu and Y
Q. Liu and Y. Yu , Efficient adaptive randomized algorithms for fixed-threshold low-rank ma- trix approximation , 2025, https://arxiv.org/abs/2508.07553, https://arxiv.org/abs/2508. 07553
2025 arXiv
-
[29]
Martinsson , Compressing rank-structured matrices via randomized sampling , SIAM Journal on Scientific Computing, 38 (2016), pp
P.-G. Martinsson , Compressing rank-structured matrices via randomized sampling , SIAM Journal on Scientific Computing, 38 (2016), pp. A2572–A2594
2016
-
[30]
Martinsson, G
P.-G. Martinsson, G. Quintana-Orti, and N. Heavner , randutv: A blocked randomized 25 algorithm for computing a rank-revealing utv factorization , ACM Transactions on Mathe- matical Software (TOMS), 45 (2019), pp. 1–26
2019
-
[31]
Martinsson, G
P.-G. Martinsson, G. Quintana Ort´I, N. Heavner, and R. Van De Geijn, Householder qr factorization with randomization for column pivoting (hqrrp) , SIAM Journal on Scientific Computing, 39 (2017), pp. C96–C115
2017
-
[32]
P. G. Martinsson and J. Tropp , Randomized numerical linear algebra: Foundations and algorithms, Acta Numerica, 29 (2020), pp. 403–572
2020
-
[33]
Martinsson and S
P.-G. Martinsson and S. Voronin , A randomized blocked algorithm for efficiently comput- ing rank-revealing factorizations of matrices , SIAM Journal on Scientific Computing, 38 (2016), pp. S485–S507, https://doi.org/10.1137/15M1026080
2016 doi
-
[34]
Meier and Y
M. Meier and Y. Nakatsukasa, Fast randomized numerical rank estimation for numerically low-rank matrices, Linear Algebra and its Applications, 686 (2024), pp. 1–32
2024
-
[35]
Nakatsukasa, Fast and stable randomized low-rank matrix approximation , arXiv preprint arXiv:2009.11392, (2020)
Y. Nakatsukasa, Fast and stable randomized low-rank matrix approximation , arXiv preprint arXiv:2009.11392, (2020)
2009 arXiv
-
[36]
C. H. Paige, The computation of eigenvalues and eigenvectors of very large sparse matrices , Ph.D. dissertation, London University, (1971)
1971
-
[37]
Pearce, C
K. Pearce, C. Chen, Y. Dong, and P.-G. Martinsson , Adaptive parallelizable algorithms for interpolative decompositions via partially pivoted lu , Numerical Linear Algebra with Applications, 32 (2025), p. e70002
2025
-
[38]
K. J. Pearce and P.-G. Martinsson , Randomized algorithms for low-rank matrix and ten- sor decompositions, 2025, https://arxiv.org/abs/2512.05286, https://arxiv.org/abs/2512. 05286
2025
-
[39]
A. S. D. Perrotolo, Y. Diouane, S. G ¨urol, and X. Vasseur , A unified error analysis for randomized low-rank approximation with application to data assimilation , 2024, https: //arxiv.org/abs/2405.04811, https://arxiv.org/abs/2405.04811
2024
-
[40]
Pritchard, T
N. Pritchard, T. Park, Y. Nakatsukasa, and P.-G. Martinsson, Fast rank adaptive CUR via a recycled small sketch , 2025, https://arxiv.org/abs/2509.21963, https://arxiv.org/ abs/2509.21963
2025
-
[41]
Saad, Numerical Methods for Large Eigenvalue Problems , SIAM, 2011
Y. Saad, Numerical Methods for Large Eigenvalue Problems , SIAM, 2011
2011
-
[42]
A. K. Saibaba , Randomized subspace iteration: Analysis of canonical angles and unitarily invariant norms, SIAM Journal on Matrix Analysis and Applications, 40 (2019), pp. 23– 48
2019
-
[43]
Vannieuwenhoven, R
N. Vannieuwenhoven, R. Vandebril, and K. Meerbergen , A new strategy for the robotic truncation of a higher-order singular value decomposition , SIAM Journal on Scientific Computing, 34 (2012), pp. A1027–A1052
2012
-
[44]
Xia, Randomized sparse direct solvers, SIAM Journal on Matrix Analysis and Applications, 34 (2013), pp
J. Xia, Randomized sparse direct solvers, SIAM Journal on Matrix Analysis and Applications, 34 (2013), pp. 197–227
2013
-
[45]
W. Xu, W. Shen, C. Liu, and Z. Jia, A novel adaptive low-rank matrix approximation method for image compression and reconstruction, 2025, https://arxiv.org/abs/2506.22713, https: //arxiv.org/abs/2506.22713
2025 arXiv
-
[46]
Yesypenko and P.-G
A. Yesypenko and P.-G. Martinsson , Randomized strong recursive skeletonization: Si- multaneous compression and LU factorization of hierarchical matrices using matrix– vector products, Journal of Scientific Computing, 106 (2026), https://doi.org/10.1007/ s10915-025-03157-9
2026
-
[47]
W. Yu, Y. Gu, and Y. Li , Efficient randomized algorithms for the fixed-precision low-rank matrix approximation , SIAM Journal on Matrix Analysis and Applications, 39 (2018), pp. 1339–1359, https://doi.org/10.1137/17M1141977
2018 doi
Reviewed July 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.