REVIEW 4 major objections 5 minor 43 references
Towards an Efficient Shifted Cholesky QR for Applications in Model Order Reduction using pyMOR
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that recomputing the shift from the current Gramian makes shifted Cholesky QR deliver machine-precision orthogonality even for ill-conditioned matrices with $\kappa_2(A)=10^{20}$, and that the resulting updating and…
desk verdict A genuinely useful, honestly benchmarked variant of shifted Cholesky QR, but the robustness claim rests on an unguarded eigenvalue estimate and should be tightened before it is advertised as generally robust. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the shifted Gramian identity $X + \sigma I = \tilde R^T \tilde R$, where $X = A^T A$; from the Cholesky factor $\tilde R$ the algorithm forms $Q \leftarrow Q\tilde R^{-1}$ and accumulates $R$. The robustness mechanism is the recomputed shift $\sigma = \max\{11(mn + n(n+1))u\|X\|_2, 2u\}$, evaluated from a Lanczos estimate of the largest eigenvalue of the current Gramian whenever the Cholesky step breaks down. For the update mode, Proposition 2.1 supplies the identity $R_2^T R_2 = A_2^T A_2 - A_2^T Q_1 Q_1^T A_2$, which converts the orthogonalization of new columns $A_2$ against an existing orthonormal basis $Q_1$ into a small $p \times p$ Cholesky problem. The panel scheme pnCholQR(r) applies that update block by block, keeping each Gramian small enough for cache and cutting the flop count by up to a factor of two when $n^3 \ll m$.
What would settle it
Construct a matrix with condition number $\kappa_2(A) = 10^{20}$ whose Gramian has a largest eigenvalue concentrated in a direction that the Lanczos process with $v_0 = \mathbf{1}$ underestimates, run rsCholQR with the Listing 1 eigsh call, and check the loss of orthogonality $\varepsilon_{\mathrm{LOO}}(Q)$; if the safeguard lets the Cholesky step break down or $\varepsilon_{\mathrm{LOO}}$ exceeds $10^{-13}$, the claimed robustness fails on that input.
Extended reading notes
Core claim
The central claim, stated in Section 4, is that rsCholQR and pnCholQR(r) are numerically robust even for matrix condition numbers as high as $\kappa_2(A) = 10^{20} \gg 1/u$, while the earlier sCholQR3 and iter-sCholQR variants work reliably only for $\kappa_2(A) < 1/u$. The mechanism is to recompute the shift $\sigma = \max\{11(mn + n(n+1))u\|X\|_2, 2u\}$ from the current Gramian iterate $X$ instead of reusing the initial estimate. With the shift matched to the current iterate, the repeated Cholesky correction drives $Q^T Q$ close to the identity in the reported experiments, with loss of orthogonality comparable to that of an LAPACK-based QR routine. The same mechanism is embedded in a column-update scheme and in a panel scheme that splits the input into $r$ blocks; the paper reports that both keep the robustness and, depending on hardware and matrix properties, can beat SciPy QR in runtime.
Load-bearing premise
Everything rests on the shift formula $\sigma = \max\{11(mn + n(n+1))u\|X\|_2, 2u\}$ staying in a narrow window, even though $\|X\|_2$ is only coarsely estimated by a Lanczos call with tolerance $10^{-2}$: too small an estimate can let the shifted Cholesky step break down, and a stale or oversized shift can quietly destroy orthogonality.
Editorial extensions
If this is right
- If the claim holds, adaptive MOR loops can extend an orthonormal basis with appended vectors while keeping orthogonality near machine precision, using only vector-wise operations and no entrywise access to the underlying data.
- The panel scheme gives a tunable cost-quality trade-off: as the number of panels grows, the flop count approaches half of rsCholQR's in the tall-skinny regime, so a MOR code can choose panel width to match cache size without losing robustness.
- When the condition number of the input is unknown, the paper recommends rsCholQR and pnCholQR(r) as the safer defaults among Cholesky QR variants, while sCholQR3 and iter-sCholQR remain cheaper choices for $\kappa_2(A) < 1/u$.
- On the notebook hardware the Cholesky variants can be faster than SciPy QR, and on the server the panel variants overtake SciPy QR for large column counts, so the communication-avoiding structure pays off in practice as well as in flop counts.
Reading between the lines
- The authors leave implicit that the factor-10 escalation in Algorithm 2 is the weakest point of the update scheme; if that heuristic fails on some input, a provably safe shift bound would be needed, and the factor 11 in the shift formula is a plausible starting point for such a bound.
- The authors leave implicit that their robustness evidence is entirely empirical: a testable follow-up is to certify the eigsh estimate with an upper bound, or to replace it with a deterministic power iteration, so that the safeguard $\max\{11(\cdots)u\|X\|_2, 2u\}$ is guaranteed rather than heuristic.
- The authors leave implicit that the synthetic SVD-generated test matrices may not reproduce the nearly dependent snapshots of real PDE simulations; applying the panel scheme to snapshot vectors produced by different finite-element backends would test whether it can replace the rank-revealing behavior of the default modified Gram-Schmidt routine.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes three modifications to shifted Cholesky QR for MOR applications in pyMOR: rsCholQR recomputes the shift from the current Gramian instead of reusing an initial estimate; CholQRUpdate provides a QR-updating scheme; pnCholQR(r) splits the input into panels and applies the update iteratively. The authors benchmark these against SciPy QR, MGS2, sCholQR3, and iter-sCholQR on two hardware systems and several VectorArray backends, using random SVD-generated matrices with condition numbers up to 10^20. They claim that the new methods are numerically robust for these extreme condition numbers and can be competitive in runtime, while also giving a flop-count analysis of the proposed algorithms.
Significance. If the robustness claim holds, the paper offers a practically useful communication-avoiding orthogonalization method for MOR, especially in the adaptive/updating mode where only vector-wise access is available. The paper's strengths are its reproducible Zenodo artifacts, careful benchmarking over multiple backends and hardware, direct comparison against standard baselines, and a flop-count analysis. However, the robustness claim rests on an unanalyzed spectral-norm estimate and on a heuristic escalation loop, so the contribution is currently an experimental proposal rather than a fully established numerical method.
major comments (4)
- [§4 and Algorithm 1] The central claim that rsCholQR is numerically robust for κ2(A)=10^20 is load-bearing but depends on the unguarded norm estimate in Listing 1. If the starting vector v0=ones(n) has near-zero overlap with the dominant eigenspace of X=A^T A, the eigsh estimate can be orders of magnitude too small, the shift σ collapses to the 2u floor, and the call chol(X+σI) on line 7 of Algorithm 1 can break down with no retry or escalation in the outer loop. Since Section 4 recommends rsCholQR precisely when the condition number is unknown, the paper should either add an escalation/fallback analogous to Algorithm 2's inner while loop, use a norm estimate robust to starting vectors, or restrict the robustness claim to inputs for which the eigsh estimate is reliable.
- [§2.2, Algorithm 2 lines 8-10] The factor-10 escalation in the inner while loop is acknowledged to be a heuristic, and the manuscript offers no evidence or analysis that it terminates or that the resulting shift preserves accuracy. Because pnCholQR(r)'s robustness inherits from this escalation, the paper should at least report the distribution of inner-loop iterations and failure rates over the benchmark, or provide a heuristic justification with a bound on the number of escalations necessary before a Cholesky factor is obtained.
- [§3.3 and §4] The robustness claim is validated only on synthetic SVD-generated matrices with random orthogonal factors. For MOR data, structured linear dependence and a dominant eigenspace aligned away from the initial Lanczos vector are plausible, and those cases are exactly where the shift recomputation can fail. The paper should either include adversarial or application-derived test matrices, such as POD snapshots or IRKA bases, or add a formal convergence and backward-stability analysis for the recomputed shift; without one of these, the conclusion in Section 4 overstates the evidence.
- [§3.1 vs Algorithms 1-3] The pseudocodes for rsCholQR and pnCholQR(r) do not include the maximum of 10 outer iterations that the benchmark implementation enforces, nor do they state what is returned when the loop exits at the cap without satisfying the stopping criterion. This discrepancy matters because the robustness claim in Section 4 is effectively 'converges within 10 iterations on the tested random matrices'; the paper should specify the failure mode for non-convergence and align the pseudocode with the implementation.
minor comments (5)
- [Novelty statement] The novelty statement contains a typo: 'imrpove' should be 'improve'.
- [§2.1] The word 'Lanzcos' should be spelled 'Lanczos'.
- [Table 3 and Listing 1] Table 3 refers to 'scipy.linalg.eigsh', but Listing 1 uses 'scipy.sparse.linalg.eigsh'; the naming should be made consistent.
- [Figure 3b] The y-axis label 'Runtime [s] / m' is ambiguous; writing 'Runtime per vector length (s/m)' would be clearer.
- [Algorithm 1 line 5] Using an assignment inside the 'if' condition is unconventional; a try/catch or an explicit breakdown flag would improve readability.
Circularity Check
No significant circularity: robustness claims rest on external benchmarks and in-paper proofs.
full rationale
The central robustness claim for rsCholQR and pnCholQR(r) is an empirical finding supported by comparisons against external baselines (SciPy QR, MGS2, and sCholQR3 / iter-sCholQR from [21]); no fitted parameter is relabeled as a prediction. The shift formula in Algorithm 1 is a proposed safeguard motivated by experiments, and Algorithm 2's factor-10 escalation is explicitly acknowledged as an unresolved heuristic. The only self-citation is Algorithm 2's origin in [35] by the second author, but the correctness of the update is re-derived in this paper via Proposition 2.1, which contains a full proof, so [35] is not load-bearing. The coarse eigsh norm estimate in Listing 1 is a legitimate robustness risk, but it is a correctness and analysis gap, not a circular step, because success is not derived from that estimate by construction. Overall, the paper is self-contained against external benchmarks and exhibits no circular reduction of its claims to their own inputs.
Assumptions & free parameters
free parameters (4)
- escalation_factor_10 =
10
- eigsh_tolerance =
1e-2
- stopping_tolerance =
1e-13
- shift_floor =
2u
assumptions (4)
- domain assumption The shift bound σ = 11(mn+n(n+1))u||X||_2 from [21] guarantees that the shifted Cholesky factorization of X + σ I succeeds in floating-point arithmetic.
- domain assumption The test matrices A := U Σ V^T with log-equidistant singular values in [1, 10^20] are representative of matrices encountered in MOR applications.
- standard math The IEEE double precision roundoff model with unit roundoff u ≈ 1.11e-16 applies, and the algorithms' operations follow standard BLAS/LAPACK rounding behavior.
- domain assumption For MOR, linearly dependent vectors in the new block can be handled by shifting rather than dropped; the 'lucky breakdown' discussion in Section 2.2 argues rank-deficiency manifests as zero diagonal entries, and shifting is used as a workaround.
Cite this review
Pith. "Pith review of Towards an Efficient Shifted Cholesky QR for Applications in Model Order Reduction using pyMOR." pith.science (2026). https://pith.science/paper/MDGSVBJS
@misc{pith2026250707788,
author = {Pith},
title = {Pith review of: Towards an Efficient Shifted Cholesky QR for Applications in Model Order Reduction using pyMOR},
year = {2026},
howpublished = {\url{https://pith.science/paper/MDGSVBJS}},
note = {Machine review of arXiv:2507.07788}
}
read the original abstract
Many model order reduction (MOR) methods rely on the computation of an orthonormal basis of a subspace onto which the large full order model is projected. Numerically, this entails the orthogonalization of a set of vectors. The nature of the MOR process imposes several requirements for the orthogonalization process. Firstly, MOR is oftentimes performed in an adaptive or iterative manner, where the quality of the reduced order model, i.e., the dimension of the reduced subspace, is decided on the fly. Therefore, it is important that the orthogonalization routine can be executed iteratively. Secondly, one possibly has to deal with high-dimensional arrays of abstract vectors that do not allow explicit access to entries, making it difficult to employ so-called `orthogonal triangularization algorithms' such as Householder QR. For these reasons, (modified) Gram-Schmidt-type algorithms are commonly used in MOR applications. These methods belong to the category of `triangular orthogonalization' algorithms that do not rely on elementwise access to the vectors and can be easily updated. Recently, algorithms like shifted Cholesky QR have gained attention. These also belong to the aforementioned category and have proven their aptitude for MOR algorithms in previous studies. A key benefit of these methods is that they are communication-avoiding, leading to vastly superior performance on memory-bandwidth-limited problems and parallel or distributed architectures. This work formulates an efficient updating scheme for Cholesky QR algorithms and proposes an improved shifting strategy for highly ill-conditioned matrices. The proposed algorithmic extensions are validated with numerical experiments on a laptop and computation server.
Reference graph
Works this paper leans on
-
[35]
A. J. R. Pelling and E. Sarradj , Adaptive Reduced Order Modelling of Discrete-Time Systems with Input-Output Dead Time , arXiv.org, (2025), https://arxiv.org/abs/2506.08870
work page Pith review arXiv 2025
-
[1]
M. S. Alnaes, J. Blechta, J. Hake, A. Jo- hansson, B. Kehlet, A. Logg, C. N. Richard- son, J. Ring, M. E. Rognes, and G. N. Wells , The FEniCS project version 1.5 , Archive of Nu- merical Software, 3 (2015), https://doi.org/10. 11588/ans.2015.100.20553
work page 2015
-
[2]
A. C. Antoulas , Approximation of Large-Scale Dynamical Systems , Advances in Design and Con- trol, Society for Industrial and Applied Mathematics, Philadelphia, 2005
work page 2005
-
[3]
ARPACK-NG contributors , ARPACK-NG, https://github.com/opencollab/arpack-ng
-
[4]
P. Benner, A. Cohen, M. Ohlberger, and K. Willcox , eds., Model Reduction and Approxi- mation: Theory and Algorithms , no. 15 in Compu- tational Science and Engineering, Society for Indus- trial and Applied Mathematics, Philadelphia, 2017, https://doi.org/10.1137/1.9781611974829
-
[5]
P. Benner, S. Grivet-Talocia, A. Quar- teroni, G. Rozza, W. H. A. Schilders, and L. M. Silveira, eds., Model Order Reduction. Vol- ume 1: System- and Data-Driven Methods and Al- gorithms, De Gruyter, Berlin, 2021, https://doi. org/10.1515/9783110498967
- [6]
-
[7]
P. Benner, S. Grivet-Talocia, A. Quar- teroni, G. Rozza, W. H. A. Schilders, and L. M. Silveira, eds., Model Order Reduction. Vol- ume 3: Applications , De Gruyter, Berlin, 2021, https://doi.org/10.1515/9783110499001
Show all 43 references
-
[8]
Bindhak, Code and data - Towards an Efficient Shifted Cholesky-QR for Applications in Model Order Reduction using pyMOR , 2025, https://doi.org/ 10.5281/zenodo.15729514
M. Bindhak, Code and data - Towards an Efficient Shifted Cholesky-QR for Applications in Model Order Reduction using pyMOR , 2025, https://doi.org/ 10.5281/zenodo.15729514
2025 doi
-
[9]
Bischof and C
C. Bischof and C. V an Loan , The WY Rep- resentation for Products of Householder Matrices , SIAM J. Sci. Statist. Comput., 8 (1987), pp. s2–s13, https://doi.org/10.1137/0908009
1987 doi
-
[10]
Blackford and J
S. Blackford and J. Dongarra , LAPACK Working Note 41 . https://netlib.org/lapack/ lawnspdf/lawn41.pdf, June 1999. Version 3.0
1999
-
[11]
Carson, K
E. Carson, K. Lund, Y. Ma, and E. Oktay , Reorthogonalized pythagorean variants of block clas- sical Gram-Schmidt , SIAM J. Matrix Anal. Appl., 46 (2025), pp. 310–340, https://doi.org/10.1137/ 24M1658723
2025
-
[12]
Carson, K
E. Carson, K. Lund, and M. Rozlo ˇzn´ık, The stability of block variants of classical Gram-Schmidt , SIAM J. Matrix Anal. Appl., 42 (2021), pp. 1365– 1380, https://doi.org/10.1137/21M1394424
2021 doi
-
[13]
Carson, K
E. Carson, K. Lund, M. Rozlo ˇzn´ık, and S. Thomas , Block Gram-Schmidt algorithms and their stability properties , Linear Algebra Appl., 638 (2022), pp. 150–195, https://doi.org/10.1016/j. laa.2021.12.017
2022 doi
-
[14]
Harris and K
Charles R. Harris and K. Jarrod Millman and St ´efan J. van der W alt and Ralf Gom- mers and Pauli Virtanen and David Cour- napeau and Eric Wieser and Julian Tay- lor and Sebastian Berg and Nathaniel J. Smith and Robert Kern and Matti Picus and Stephan Hoyer and Marten H. van ...
2020
-
[15]
S. Chellappa, A posteriori Error Estimation and Adaptivity for Model Order Reduction of Large- Scale Systems , Dissertation, Otto-von-Guericke- Universit¨ at, Madgeburg, Germany, 2022,https:// doi.org/10.25673/101396
2022 doi
-
[16]
, https: //github.com/conda/conda
conda contributors , conda: A system-level, bi- nary package and environment manager running on all major operating systems and platforms. , https: //github.com/conda/conda
-
[17]
Demmel, L
J. Demmel, L. Grigori, M. Hoemmen, and J. Langou, Communication-optimal parallel and se- quential QR and LU factorizations , SIAM J. Sci. Comput., 34 (2012), pp. A206–A239, https://doi. org/10.1137/080731992
2012 doi
-
[18]
R. W. Freund , Model reduction methods based on Krylov subspaces , Acta Numer., 12 (2003), pp. 267–319, https://doi.org/10.1017/ S0962492902000120
2003
-
[19]
Fritze, S
R. Fritze, S. Rave, F. Schindler, P. Mlinari´c, L. Balicki, and H. Kleikamp, pyMOR, Dec. 2024, https://doi.org/10.5281/zenodo.14536112. Preprint. 2025-08-24 M. Bindhak, A. J. R. Pelling, J. Saak: Towards Efficient Shifted Cholesky QR for pyMOR 16
2024 doi
-
[20]
T. Fukaya, An investigation into the impact of the structured QR kernel on the overall performance of the TSQR algorithm , in Proceedings of the Inter- national Conference on High Performance Comput- ing in Asia-Pacific Region, HPCAsia ’19, New York, NY, USA, 2019, Association...
2019 doi
-
[21]
Fukaya, R
T. Fukaya, R. Kannan, Y. Nakatsukasa, Y. Yamamoto, and Y. Yanagisawa , Shifted Cholesky QR for Computing the QR Factorization of Ill-Conditioned Matrices , SIAM J. Sci. Comput., 42 (2020), pp. A477–A503, https://doi.org/10. 1137/18M1218212
2020
-
[22]
Fukaya, Y
T. Fukaya, Y. Nakatsukasa, and Y. Ya- mamoto, A Cholesky QR type algorithm for com- puting tall-skinny QR factorization with column piv- oting, in 2024 IEEE International Parallel and Dis- tributed Processing Symposium (IPDPS), IEEE, May 2024, pp. 63–75, https://doi.org/10.110...
2024
-
[23]
Fukaya, Y
T. Fukaya, Y. Nakatsukasa, Y. Yanagisawa, and Y. Yamamoto , CholeskyQR2: A Simple and Communication-Avoiding Algorithm for Computing a Tall-Skinny QR Factorization on a Large-Scale Parallel System , in 2014 5th Workshop on Lat- est Advances in Scalable Algorithms for Large- Sc...
2014
-
[24]
G. H. Golub and C. F. V an Loan, Matrix Com- putations, Johns Hopkins Studies in the Mathemati- cal Sciences, Johns Hopkins University Press, Balti- more, fourth ed., 2013
2013
-
[25]
E. J. Grimme , Krylov projection methods for model reduction , Ph.D. Thesis, Univ. of Illinois at Urbana-Champaign, USA, 1997, https://perso.uclouvain.be/paul.vandooren/ ThesisGrimme.pdf
1997
-
[26]
Gugercin, A
S. Gugercin, A. C. Antoulas, and C. Beat- tie, H2 Model Reduction for Large-Scale Linear Dy- namical Systems , SIAM J. Matrix Anal. Appl., 30 (2008), pp. 609–638, https://doi.org/10.1137/ 060666123
2008
-
[27]
Lanczos , An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J
C. Lanczos , An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Research Nat. Bur. Standards, 45 (1950), pp. 255–282
1950
-
[28]
R. B. Lehoucq, D. C. Sorensen, and C. Yang , ARPACK Users’ Guide , Society for Industrial and Applied Mathematics, 1998, https://doi.org/10. 1137/1.9780898719628
1998
-
[29]
Logg, K.-A
A. Logg, K.-A. Mardal, G. N. Wells, et al. , Automated Solution of Differential Equations by the Finite Element Method , Springer, 2012, https:// doi.org/10.1007/978-3-642-23099-8
2012 doi
-
[30]
R. Milk, S. Rave, and F. Schindler , py- MOR – Generic Algorithms and Interfaces for Model Order Reduction , SIAM J. Sci. Comput., 38 (2016), pp. S194–S216, https://doi.org/10.1137/ 15M1026614
2016
-
[31]
Miniforge contributors , Miniforge, https:// github.com/conda-forge/miniforge
-
[32]
Minster, A
R. Minster, A. K. Saibaba, J. Kar, and A. Chakrabortty, Efficient Algorithms for Eigen- system Realization Using Randomized SVD , SIAM J. Matrix Anal. Appl., 42 (2021), pp. 1045–1072, https://doi.org/10.1137/20M1327616
2021 doi
-
[33]
NGSolve contributors , NGSolve, https:// github.com/NGSolve/ngsolve
-
[34]
OpenBLAS contributors, OpenBLAS, https:// github.com/OpenMathLib/OpenBLAS
-
[36]
, py- MOR – Model Order Reduction with Python , https: //pymor.org
pyMOR developers and contributors. , py- MOR – Model Order Reduction with Python , https: //pymor.org
-
[37]
J. Saak , Efficient Numerical Solution of Large Scale Algebraic Matrix Equations in PDE Control and Model Order Reduction , Dissertation, Tech- nische Universit¨ at Chemnitz, Chemnitz, Germany, July 2009, http://nbn-resolving.de/urn:nbn: de:bsz:ch1-200901642
2009
-
[38]
Sch¨oberl, Joachim, NETGEN an advancing front 2D/3D-mesh generator based on abstract rules, Com- puting and Visualization in Science, 1 (1997), pp. 41– 52
1997
-
[39]
Schreiber and C
R. Schreiber and C. V an Loan , A Storage- Efficient $WY$ Representation for Products of Householder Transformations, SIAM J. Sci. Statist. Comput., 10 (1989), pp. 53–57, https://doi.org/ 10.1137/0910005
1989 doi
-
[40]
Sirovich , Turbulence and the dynamics of co- herent structures
L. Sirovich , Turbulence and the dynamics of co- herent structures. parts I-III , Quart. Appl. Math., 45 (1987), pp. 561–590, http://www.jstor.org/ stable/43637457
1987
-
[41]
D. C. Sorensen , Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Preprint. 2025-08-24 M. Bindhak, A. J. R. Pelling, J. Saak: Towards Efficient Shifted Cholesky QR for pyMOR 17 Eigenvalue Calculations , Springer Nether- lands, Dordrecht, 1997, pp. 119–165, https: /...
2025 doi
-
[42]
L. N. Trefethen and D. Bau, III , Chapter II: QR Factorization and Least Squares , in Numerical Linear Algebra, Other Titles in Applied Mathemat- ics, Society for Industrial and Applied Mathematics, Jan. 1997, pp. 41–85, https://doi.org/10.1137/ 1.9780898719574.ch2
1997
-
[43]
Virtanen, R
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der W alt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Lar- son, C. J. Carey, ˙I. Polat, Y. F...
2020
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.