REVIEW 1 major objections 3 minor 34 references
Any nonincreasing convergence curves are simultaneously possible for GMRES and weighted GMRES, as well as for left and right preconditioned GMRES
T0 review · 1 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper proves that any two nonincreasing convergence curves can be realized together: plain GMRES and weighted GMRES, or left- and right-preconditioned GMRES, can be made to follow different prescribed residual histories on one system…
desk verdict Solid new results on simultaneous convergence-curve prescription for weighted and preconditioned GMRES, with a small patchable gap in the converse of Theorem 15. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The residual decrease vector $g$ and the nested residual basis $W$ carry the argument. A convergence curve is recorded as $g_j=\sqrt{\|r_{j-1}\|^2-\|r_j\|^2}$, so prescribing a curve is prescribing $g$. Two nested bases $W$ and $\widetilde W$ of the Krylov residual spaces are related by $\widetilde W = W T$ with $T$ block upper triangular of the form (5); enforcing the right-hand side $b=Wg=\widetilde W\tilde g$ gives $g=T\tilde g$, and requiring $\widetilde W$ to be $M$-orthonormal forces $TT^*=W^*M^{-1}W$, which turns the eigenvalues of $M$ into singular values of $T$. This chain of identities is what all the equivalence theorems hinge on.
What would settle it
Run the paper's constructive proof on a random admissible triple, for instance n=2 with M=diag(1,1/4), g=(2,1)^T, and tilde g=(1,1)^T, which satisfies the Theorem 15 condition, building the system via the prescribed-basis version of the classical construction and then computing the actual I-GMRES and M-GMRES residual norms. If either residual sequence deviates from the prescribed curve, the converse direction is false. A simpler check is to enumerate all unitary matrices W for a small n and ask whether the classical construction can realize W as the GMRES basis; the first failure would refute the theorem's converse.
Extended reading notes
Core claim
The paper's central discovery is a transfer principle for prescribed GMRES convergence. In the residual decrease vector formalism, where a convergence curve is encoded by $g$ with entries $\sqrt{\|r_{i-1}\|^2-\|r_i\|^2}$, two curves $g$ and $\tilde g$ can be realized simultaneously by I-GMRES and M-GMRES, for a fixed Hermitian positive definite weight $M$, if and only if there is a nonsingular block upper triangular matrix $T$ such that $g=T\tilde g$ and the singular values of $T$ are the inverse square roots of the eigenvalues of $M$. The analogous statement for a fixed preconditioner $H$ is that left and right preconditioned GMRES realize prescribed curves exactly when the same triangular condition holds with singular values of $T$ equal to the reciprocals of the singular values of $H$. In both settings the eigenvalues of the (preconditioned) system matrix can be prescribed arbitrarily, generalizing the classical one-curve theorem. When the weight or preconditioner is itself free, any two same-length nonincreasing curves are simultaneously possible.
Load-bearing premise
The proof that the algebraic condition is sufficient assumes you can build a linear system whose GMRES residual directions are exactly a pre-chosen set of orthonormal vectors; the classical theorem cited only guarantees some set of directions, and the paper does not pin down the strengthened version it relies on.
Editorial extensions
If this is right
- For a fixed weight matrix, the possible weighted-GMRES curves compatible with a given unweighted curve are exactly parameterized by triangular matrices whose singular values are fixed by the eigenvalues of the weight; every such matrix yields a system realizing both curves.
- When the weight is not fixed, any two same-length nonincreasing curves are simultaneously realizable, and the system matrix can carry any prescribed eigenvalues.
- Left and right preconditioned GMRES are interchangeable in a strong sense: any two curves are simultaneously possible, and for a fixed preconditioner the realizability condition is the same singular-value condition on the curves.
- For any system and preconditioner there is another system and preconditioner with the same preconditioned spectrum where the left/right convergence roles are exchanged, so neither side is inherently faster.
- The Euclidean and weighted residual norms of the two methods can differ by a factor bounded by the square root of the condition number of the weight, and the paper's constructions show this bound can be approached, so the inner product choice alone can drastically change the observed convergence history.
Reading between the lines
- Beyond the paper: the singular-value condition in Theorem 15 is algorithmic in nature, so it could be used as a design tool to search for a weight matrix that pushes weighted GMRES toward a desired curve while preserving a given unweighted curve.
- Beyond the paper: because weighted GMRES with weight $M=P^*P$ is equivalent to split preconditioning by $(P,P^{-1})$, the same flexibility should extend to split preconditioners, with the left factor controlling the norm and the right factor controlling the operator.
- Beyond the paper: the full flexibility proved here likely does not survive restarting or blocking, since restart cycles couple the nested bases across cycles; testing the same pair of curves under restarted or block GMRES would isolate the mechanism that makes the unrestricted setting so permissive.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies prescribed convergence curves for GMRES in the Euclidean inner product (I-GMRES), in a weighted inner product (M-GMRES), and for left/right/split preconditioned GMRES. Building on the Greenbaum–Pták–Strakoš theorem (Theorem 5), it proves: (i) for any fixed (A,b) and any admissible decrease vector g̃, there exists an hpd weight M such that M-GMRES realizes g̃ (Theorem 10); (ii) any two curves of the same length can be realized simultaneously by I-GMRES and M-GMRES with prescribed spectrum (Corollary 11), together with a characterization of the weights that allow a fixed M to realize a given curve (Theorem 13); (iii) a necessary and sufficient condition, in terms of a block-triangular T with prescribed singular values and g = T g̃, for a fixed M to allow simultaneous realization (Theorem 15); and (iv) analogous results for left/right preconditioned GMRES (Theorems 21–25), including a reversal construction (Corollary 24). The paper also gives residual-norm bounds relating I- and M-GMRES (Theorem 16) and contains numerical illustrations. The proofs are largely constructive and are accompanied by open-source MATLAB code.
Significance. If the main theorems are established, the paper makes a genuine contribution to the inverse convergence-analysis literature. The results quantify how much the choice of inner product or the side of preconditioning can alter GMRES convergence independently of the spectrum, and the block-triangular singular-value condition in Theorem 15 is a new and checkable criterion. The paper is also honest about the fact that the characterization in Theorem 15 is not very transparent and analyzes it in Section 2.3. Strengths include the constructive proofs, the clean structural Lemmas 7 and 8, the explicit characterization theorems, and the openly available reproducible MATLAB code. The main weakness is a missing justification in the converse of Theorem 15, where a basis-prescribed version of the GPS construction is needed; as written that proof is incomplete, although the gap is repairable by a short unitary-similarity argument.
major comments (1)
- [Section 2.2, proof of Theorem 15, after Eq. (17)] The converse direction applies Theorem 5 to the vector b := Wg and then asserts that the first m columns of ~W := WT form a nested basis of AK_m(A,b). Theorem 5 as stated guarantees only the existence of some system (A,b) realizing g; it does not guarantee that the prescribed unitary W is a nested residual basis for the constructed A. The same gap is inherited by Theorem 25, whose proof invokes Theorem 15. The gap is repairable: take (A0,b0) from Theorem 5 with a nested orthonormal basis V (so b0 = Vg and A0 K_i(A0,b0) = span{v1,...,vi}), and set A := W V* A0 V W*. Then A has the prescribed eigenvalues, b = Wg, and the first m columns of W form a nested basis of AK_m(A,b), so the subsequent verification of the M-orthonormality of WT goes through. Please insert this argument, or cite a basis-prescribed version explicitly such as the constructive proof in [8, Th. 3].
minor comments (3)
- [Title and Abstract] The simultaneous-prescription theorems (Corollary 11 and Theorem 22) require the two curves to have the same length, i.e. the same GMRES breakdown iteration. The title's unqualified claim that any nonincreasing convergence curves are simultaneously possible is stronger than the theorems; please qualify the statement.
- [Section 4.2, Experiment 5] The sentence 'the scaled vectors (sqrt(mu_i)/beta w_i)_i form an M-orthonormal nested basis' is not consistent with the definitions M = beta M0 and b = (1/sqrt(n)) Q1; with W = Q, the normalized vectors are w_i/sqrt(beta mu_i) and the corresponding squared decrease is beta mu_i/n. Please correct the scaling expression (the displayed formulas for ||~r_i||_M are consistent with the corrected normalization).
- [Lemma 8] The proof of Lemma 8 is only a sketch ('justified by nestedness'). Since Lemma 8 underlies both Theorem 13 and Theorem 15, a short block-by-block derivation of the upper-triangular structure of T would improve readability and make the paper more self-contained.
Circularity Check
No significant circularity; the derivation is anchored in the external GPS theorem and explicit constructions, with a contextual self-citation and a non-circular proof gap in Theorem 15's converse.
full rationale
The derivation chain is anchored in the external Greenbaum-Ptak-Strakos theorem (Theorem 5, cited to [13]) and in self-contained structural lemmas (Lemmas 6-8). Theorems 10, 13, 15, 22, 23 and 25 construct the weight matrix M, the system (A,b), and the preconditioner H explicitly from the prescribed residual decrease vectors; no parameter is fitted to a subset of data and then renamed as a prediction. The only directly relevant self-citation is [30], used as context ('See also the literature review in [30]' and 'see also [30, Sec. 3.1]'); it is not load-bearing for any theorem. I therefore find no step where the claimed conclusion is assumed as an input. One correctness caveat should be separated from circularity: in the converse direction of Theorem 15 (Section 2.2, after equation (17)), the proof chooses a unitary W satisfying TT* = W* M^{-1} W, sets b := Wg, invokes Theorem 5, and then asserts that ~W := WT has its first m columns forming a nested basis of AK_m(A,b). The quoted Theorem 5 guarantees existence of some A realizing the curve g with this b, but it does not state that a prescribed unitary W can be made the nested residual basis. The needed basis-preserved version is true (by unitary similarity of the GPS construction), so this is an omitted justification, not a circular reduction. Theorem 25 inherits the same caveat, but this does not raise the circularity score.
Assumptions & free parameters
assumptions (6)
- domain assumption Greenbaum-Ptak-Strakos theorem (Theorem 5): for any residual decrease vector g of length m and any n nonzero complex numbers lambda_i, there exists a nonsingular A with eigenvalues lambda_i and a vector b such that GMRES(A,b) realizes g.
- domain assumption A strengthened, basis-prescribed version of the GPS theorem: given any unitary W whose first m columns are to be the nested residual basis, one can construct (A,b) with prescribed eigenvalues realizing a given curve g.
- standard math The GMRES residual decrease vector formalism: for any nested orthonormal basis w_i of the residual Krylov space, |<b,w_i>| = sqrt(||r_{i-1}||^2 - ||r_i||^2) (Lemma 7 and equation (2)).
- standard math Spectral theorem and singular value decomposition: hpd matrices can be diagonalized by unitary similarity; TT* and M^{-1} sharing eigenvalues implies existence of unitary W with TT* = W*M^{-1}W.
- standard math Existence of Cholesky factorization M = P*P for any hpd M.
- domain assumption The Krylov breakdown index m = dim K_n(A,b) is independent of the inner product.
Cite this review
Pith. "Pith review of Any nonincreasing convergence curves are simultaneously possible for GMRES and weighted GMRES, as well as for left and right preconditioned GMRES." pith.science (2026). https://pith.science/paper/5MYYJPGX
@misc{pith2026250617193,
author = {Pith},
title = {Pith review of: Any nonincreasing convergence curves are simultaneously possible for GMRES and weighted GMRES, as well as for left and right preconditioned GMRES},
year = {2026},
howpublished = {\url{https://pith.science/paper/5MYYJPGX}},
note = {Machine review of arXiv:2506.17193}
}
read the original abstract
The convergence of the GMRES linear solver is notoriously hard to predict. A particularly enlightening result by [Greenbaum, Pt\'ak, Strako\v{s}, 1996] is that, given any convergence curve, one can build a linear system for which GMRES realizes that convergence curve. What is even more extraordinary is that the eigenvalues of the problem matrix can be chosen arbitrarily. We build upon this idea to derive novel results about weighted GMRES. We prove that for any linear system and any prescribed convergence curve, there exists a weight matrix M for which weighted GMRES (i.e., GMRES in the inner product induced by M) realizes that convergence curve, and we characterize the form of M. Additionally, we exhibit a necessary and sufficient condition on M for the simultaneous prescription of two convergence curves, one realized by GMRES in the Euclidean inner product, and the other in the inner product induced by M. These results are then applied to infer some properties of preconditioned GMRES when the preconditioner is applied either on the left or on the right. For instance, we show that any two convergence curves are simultaneously possible for left and right preconditioned GMRES.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
M. Arioli, V. Pták, and Z. Strakoš. Krylov sequences of maximal length and convergence of GMRES.BIT Numerical Mathematics, 38(4):636–643, 1998.doi:10.1007/BF02510405
-
[2]
A comparative study of sparse approximate inverse preconditioners.Appl
Michele Benzi and Miroslav Tůma. A comparative study of sparse approximate inverse preconditioners.Appl. Numer. Math., 30(2-3):305–340, 1999.doi:10.1016/S0168-9274(98)00118-4
-
[3]
X.-C. Cai and O. B. Widlund. Domain decomposition algorithms for indefinite elliptic problems.SIAM J. Sci. Stat. Comput., 13(1):243–258, 1992.doi:10.1137/0913013
doi:10.1137/0913013 1992
-
[4]
T. F. Chan, E. Chow, Y. Saad, and M. C. Yeung. Preserving symmetry in preconditioned Krylov subspace methods.SIAM J. Sci. Comput., 20(2):568–581, 1999.doi:10.1137/S1064827596311554
-
[5]
T. A. Davis and Y. Hu. The University of Florida sparse matrix collection.ACM Trans. Math. Softw., 38(1):25,
-
[6]
K. Du, J. Duintjer Tebbens, and G. Meurant. Any admissible harmonic Ritz value set is possible for GMRES. ETNA - Electronic Transactions on Numerical Analysis, 27:37–56, 2017.doi:10.1553/etna_vol47s37
-
[7]
J. Duintjer Tebbens and G. Meurant. Any Ritz Value Behavior Is Possible for Arnoldi and for GMRES.SIAM Journal on Matrix Analysis and Applications, 33(3):958–978, 2012.doi:10.1137/110843666
-
[8]
J. Duintjer Tebbens and G. Meurant. Prescribing the behavior of early terminating GMRES and Arnoldi iterations.Numerical Algorithms, 65(1):69–90, 2014.doi:10.1007/s11075-013-9695-x
Show all 34 references
-
[9]
S. C. Eisenstat, H. C. Elman, and M. H. Schultz. Variational iterative methods for nonsymmetric systems of linear equations.SIAM J. Numer. Anal., 20:345–357, 1983.doi:10.1137/0720023
1983 doi
-
[10]
M. Embree. Extending elman’s bound for gmres, 2025. URL:https://arxiv.org/abs/2312.15022,arXiv: 2312.15022
2025 arXiv
-
[11]
Embree, R
M. Embree, R. B. Morgan, and H. V. Nguyen. Weighted Inner Products for GMRES and GMRES-DR.SIAM Journal on Scientific Computing, 39(5):S610–S632, 2017.doi:10.1137/16M1082615
2017 doi
-
[12]
A. Essai. Weighted FOM and GMRES for solving nonsymmetric linear systems.Numerical Algorithms, 18(3):277–292, 1998.doi:10.1023/A:1019177600806
1998 doi
-
[13]
Greenbaum, V
A. Greenbaum, V. Pták, and Z. Strakoš. Any Nonincreasing Convergence Curve is Possible for GMRES.SIAM Journal on Matrix Analysis and Applications, 17(3):465–469, 1996.doi:10.1137/S0895479894275030
1996 doi
-
[14]
Güttel and J
S. Güttel and J. Pestana. Some observations on weighted GMRES.Numerical Algorithms, 67(4):733–752, 2014. doi:10.1007/s11075-013-9820-x
2014 doi
-
[15]
R. A. Horn and C. R. Johnson.Matrix analysis.Cambridge: Cambridge University Press, 2nd ed. edition, 2013
2013
-
[16]
A different approach to bounding the minimal residual norm in Krylov methods
I Ipsen. A different approach to bounding the minimal residual norm in Krylov methods. Technical report, Citeseer, 1998
1998
-
[17]
I. C. F. Ipsen. Expressions and bounds for the GMRES residual.BIT, 40(3):524–535, 2000.doi:10.1023/A: 1022371814205. 22 P. Matalon and N. Spillane Simultaneously prescribed GMRES curves
2000 doi
-
[18]
Klawonn and G
A. Klawonn and G. Starke. Block triangular preconditioners for nonsymmetric saddle point problems: Field-of- values analysis.Numer. Math., 81(4):577–594, 1999.doi:10.1007/s002110050405
1999 doi
-
[19]
Kubínová and K
M. Kubínová and K. M. Soodhalter. Admissible and attainable convergence behavior of block Arnoldi and GMRES.SIAM J. Matrix Anal. Appl., 41(2):464–486, 2020.doi:10.1137/19M1272469
2020 doi
-
[20]
Liesen, M
J. Liesen, M. Rozlozník, and Z. Strakos. Least squares residuals and minimal residual methods.SIAM J. Sci. Comput., 23(5):1503–1525, 2002.doi:10.1137/S1064827500377988
2002 doi
-
[21]
N. S. Mendelsohn. Some elementary properties of ill conditioned matrices and linear equations.Am. Math. Mon., 63:285–295, 1956.doi:10.2307/2310510
1956 doi
-
[22]
G. Meurant. Prescribing the error in GMRES. unpublished technical report, 2011. URL:https://www. gerard-meurant.fr/error_2011.pdf
2011
-
[23]
GMRESandtheArioli, Pták, andStrakošparametrization.BIT Numerical Mathematics, 52(3):687– 702, 2012.doi:10.1007/s10543-012-0374-4
G.Meurant. GMRESandtheArioli, Pták, andStrakošparametrization.BIT Numerical Mathematics, 52(3):687– 702, 2012.doi:10.1007/s10543-012-0374-4
2012 doi
-
[24]
Pestana.Nonstandard Inner Products and Preconditioned Iterative Methods
J. Pestana.Nonstandard Inner Products and Preconditioned Iterative Methods. PhD thesis, University of Oxford, 2011
2011
-
[25]
Pestana and A
J. Pestana and A. J. Wathen. On the choice of preconditioner for minimum residual methods for non-Hermitian matrices.Journal of Computational and Applied Mathematics, 249:57–68, 2013.doi:10.1016/j.cam.2013.02. 020
2013 doi
-
[26]
Saad.Iterative methods for sparse linear systems.Philadelphia, PA: SIAM Society for Industrial and Applied Mathematics, 2nd ed
Y. Saad.Iterative methods for sparse linear systems.Philadelphia, PA: SIAM Society for Industrial and Applied Mathematics, 2nd ed. edition, 2003
2003
-
[27]
Saad and M
Y. Saad and M. H. Schultz. GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems.SIAM Journal on Scientific and Statistical Computing, 7(3):856–869, 1986.doi:10.1137/ 0907058
1986
-
[28]
Sarkis and D
M. Sarkis and D. B. Szyld. Optimal left and right additive Schwarz preconditioning for minimal residual methods with Euclidean and energy norms.Comput. Methods Appl. Mech. Eng., 196(8):1612–1621, 2007.doi: 10.1016/j.cma.2006.03.027
2007 doi
-
[29]
Nečas Cent
Jennifer Scott and Miroslav Tůma.Algorithms for sparse linear systems. Nečas Cent. Ser. Cham: Springer, 2023.doi:10.1007/978-3-031-25820-6
2023 doi
-
[30]
Do you precondition on the left or on the right?, 2026
Nicole Spillane, Pierre Matalon, and Daniel B Szyld. Do you precondition on the left or on the right?, 2026. (Accepted into the proceedings of the DD29 conference). URL:https://arxiv.org/abs/2512.05160,arXiv: 2512.05160
2026
-
[31]
G. Starke. Field-of-values analysis of preconditioned iterative methods for nonsymmetric elliptic problems. Numer. Math., 78(1):103–117, 1997.doi:10.1007/s002110050306
1997 doi
-
[32]
Vecharynski and J
E. Vecharynski and J. Langou. Any admissible cycle-convergence behavior is possible for restarted GMRES at its initial cycles.Numerical Linear Algebra with Applications, 18(3):499–511, 2011.doi:10.1002/nla.739
2011 doi
-
[33]
A. Wathen. Preconditioning and convergence in the right norm.International Journal of Computer Mathematics, 84(8):1199–1209, 2007.doi:10.1080/00207160701355961. 23
2007 doi
-
[2011]
Id/No 1.doi:10.1145/2049662.2049663
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.