pith. sign in

arxiv: 2604.27971 · v1 · submitted 2026-04-30 · 🧮 math.NA · cs.NA

Flexible GMRES converges in two phases

Stefan G\"uttel , Lauri Nyman This is my paper

Pith reviewed 2026-05-07 04:53 UTC · model grok-4.3

classification 🧮 math.NA cs.NA
keywords flexible GMRESFGMRESresidual boundconvergence phasesiterative methodspreconditioninginner solver toleranceKrylov methods
0
0 comments X

The pith

Flexible GMRES exhibits two distinct convergence phases whose character is set by the residual tolerance of the inner preconditioner.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper derives a sharp upper bound on the residual norms produced by flexible GMRES. The bound shows that convergence occurs in two phases whose rates depend on how accurately each inner preconditioner solve is carried out. With tight inner tolerances the overall process behaves essentially like geometric convergence at a constant rate. With looser inner tolerances an initial slower phase appears before the residual drops more rapidly. The authors prove the bound cannot be improved by constructing an explicit example in which the residual meets the bound at every step.

Core claim

The central claim is a sharp upper bound on the residuals of flexible GMRES that depends explicitly on the sequence of inner residual tolerances. The bound reveals that FGMRES convergence consists of an early phase limited by the inner tolerances followed by a later phase whose rate matches standard GMRES behavior once the inner solves are sufficiently accurate. The bound is attained with equality for a specially constructed matrix and sequence of inner tolerances, confirming sharpness.

What carries the argument

The sharp upper bound on residual norms derived from the flexible Arnoldi relation and the controlled inner residual tolerances; the bound tracks how the varying inner solves affect residual reduction at each outer iteration.

If this is right

  • The number of outer iterations needed can be estimated in advance from the chosen inner tolerances.
  • A constant tight inner tolerance produces nearly steady geometric convergence without a visible slow phase.
  • Looser inner tolerances produce a clear initial plateau followed by faster reduction once the accumulated effect of the inner solves improves.
  • The bound supplies a theoretical basis for choosing or adapting inner tolerances to minimize total computational work.
  • Observed plateaus in practical FGMRES runs can be explained as the first phase of the two-phase pattern.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same two-phase structure may appear in other flexible or inexact Krylov methods that vary the accuracy of preconditioner applications.
  • The bound could be used inside iterative solvers to predict when to tighten or relax the inner tolerance dynamically.
  • Similar sharp bounds might be obtainable for restarted flexible GMRES or for methods that combine flexible and inexact preconditioning.
  • In large-scale applications the analysis offers a way to trade inner solve cost against the length of the slow initial phase.

Load-bearing premise

The inner preconditioner must be driven exactly to the user-specified residual tolerance at every outer step and the flexible iteration must obey the standard residual-update relations without extra rounding or matrix-specific effects.

What would settle it

A concrete matrix, right-hand side, and sequence of inner tolerances for which the observed residual norm at some outer iteration exceeds the value given by the derived upper bound.

Figures

Figures reproduced from arXiv: 2604.27971 by Lauri Nyman, Stefan G\"uttel.

Figure 1
Figure 1. Figure 1 view at source ↗
Figure 5
Figure 5. Figure 5 view at source ↗
Figure 5
Figure 5. Figure 5 view at source ↗
read the original abstract

We derive a sharp upper bound on the residuals produced by the flexible GMRES (FGMRES) method. The bound shows that FGMRES exhibits two phases of convergence depending on the residual tolerance of the inner preconditioner. For small tolerances, the convergence of FGMRES is practically geometric with a constant rate throughout, while for looser tolerances the two-phase behavior becomes more pronounced. We also show that the derived bound cannot be improved and construct an example for which it becomes an equality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript derives a sharp upper bound on the residual norms produced by flexible GMRES (FGMRES). The bound is parameterized by the inner preconditioner's residual tolerance and takes a two-phase form (initial rapid decrease followed by a slower rate) when the tolerance is loose; for tight tolerances the bound reduces to essentially geometric convergence. The authors prove the bound cannot be improved in general and construct a specific example in which the FGMRES residual norm equals the bound at every outer iteration.

Significance. If the derivation holds, the result supplies a rigorous, tolerance-dependent explanation for the two-phase convergence frequently observed when FGMRES is paired with inexact or variable inner solvers. The explicit construction of an equality case is a clear strength, establishing that the bound is optimal rather than merely permissive. This could inform practical selection of inner tolerances and the design of stopping criteria for inner iterations.

major comments (2)
  1. [Abstract] Abstract and the statement of the main result: the claim that the derived upper bound 'shows that FGMRES exhibits two phases of convergence' is not automatically justified by an upper bound alone. An inequality ||r_k|| ≤ B(k, tol) where B has a two-phase shape only guarantees that the residual cannot exceed B; it does not force the actual residual sequence to display the second phase unless additional relations (e.g., that the flexible Arnoldi residual identity becomes an equality under the given inner tolerance) are established for general inputs. The manuscript should either (i) prove that the bound is attained or nearly attained for a broad class of problems, or (ii) rephrase the claim to refer to the worst-case or envelope behavior permitted by the bound.
  2. [Section 4 and Section 5] Section 4 (derivation of the bound) and the equality example in Section 5: the proof that the bound is sharp relies on a specially constructed matrix and inner-solver sequence. It is unclear whether the two-phase shape is realized only in this contrived case or whether the same limiting behavior occurs for generic matrices when the inner tolerance is fixed but loose. A brief discussion of the conditions under which the residual actually tracks the bound (rather than lying strictly below it) would strengthen the central claim.
minor comments (2)
  1. [Section 2] Notation for the inner tolerance (denoted tol or ε in different places) should be unified and introduced once in Section 2.
  2. [Figure 1] Figure 1 (or the plot of the bound versus iteration) would benefit from an overlay of actual FGMRES residual curves for the equality example to visually confirm that the bound is attained.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive suggestions. We agree that the abstract and main result statements require clarification to avoid implying that every FGMRES run exhibits two-phase convergence. We will revise to present the bound as a sharp worst-case envelope. Point-by-point responses are below.

read point-by-point responses

  1. Referee: [Abstract] Abstract and the statement of the main result: the claim that the derived upper bound 'shows that FGMRES exhibits two phases of convergence' is not automatically justified by an upper bound alone. An inequality ||r_k|| ≤ B(k, tol) where B has a two-phase shape only guarantees that the residual cannot exceed B; it does not force the actual residual sequence to display the second phase unless additional relations (e.g., that the flexible Arnoldi residual identity becomes an equality under the given inner tolerance) are established for general inputs. The manuscript should either (i) prove that the bound is attained or nearly attained for a broad class of problems, or (ii) rephrase the claim to refer to the worst-case or envelope behavior permitted by the bound.

    Authors: We agree that the upper bound does not force two-phase behavior in every instance. The bound is a sharp characterization of the worst-case residual norm, and the equality example constructed in Section 5 demonstrates that the two-phase shape is attained for specific problems and inner-solver sequences. We will revise the abstract and the statement of the main result to refer explicitly to the worst-case or envelope behavior permitted by the bound, following the referee's option (ii). No claim of attainment for a broad class of problems will be made. revision: yes

  2. Referee: [Section 4 and Section 5] Section 4 (derivation of the bound) and the equality example in Section 5: the proof that the bound is sharp relies on a specially constructed matrix and inner-solver sequence. It is unclear whether the two-phase shape is realized only in this contrived case or whether the same limiting behavior occurs for generic matrices when the inner tolerance is fixed but loose. A brief discussion of the conditions under which the residual actually tracks the bound (rather than lying strictly below it) would strengthen the central claim.

    Authors: The equality example is constructed specifically to prove sharpness, i.e., that the bound cannot be improved in general. For generic matrices the residual will typically lie strictly below the bound. We will add a concise discussion after the equality example in Section 5 that extracts from the proof the precise conditions (saturation of the inner residual tolerances together with specific choices of the correction vectors that make the flexible Arnoldi relations equalities) under which the actual residual can track the bound. This clarifies when the two-phase envelope is approached without asserting that it occurs for all generic matrices. revision: yes

Circularity Check

0 steps flagged

Derivation of sharp residual bound for FGMRES is self-contained mathematical analysis

full rationale

The paper derives an upper bound on FGMRES residuals directly from the definition of the flexible Arnoldi process, the outer residual relation, and the user-specified inner residual tolerance. No parameters are fitted to data and then presented as predictions. No self-citations are invoked to justify load-bearing uniqueness or ansatz choices. Sharpness is established by constructing a specific example in which the bound is attained, which is a standard and non-circular technique for proving tightness. The two-phase behavior is a consequence of the functional form of the derived bound rather than a redefinition or renaming of an input quantity. The derivation chain is independent of the target result and does not reduce to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review performed on abstract only; no explicit free parameters, invented entities, or non-standard axioms are stated. The bound presumably rests on standard properties of GMRES residual relations and the definition of flexible preconditioning.

axioms (1)
  • domain assumption Standard residual relations and orthogonality properties hold for the flexible GMRES iteration with varying preconditioners
    The two-phase bound derivation relies on these properties of the method as described in the abstract.

pith-pipeline@v0.9.0 · 5365 in / 1441 out tokens · 100564 ms · 2026-05-07T04:53:11.787531+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    Balay, S

    S. Balay, S. Abhyankar, M. Adams, J. Brown, P. Brune, K. Buschelman, L. Dalcin, A. Dener, V. Eijkhout, W. Gropp, et al. PETSc Users Manual. 2019

    2019

  2. [2]

    Bavier, M

    E. Bavier, M. Hoemmen, S. Rajamanickam, and H. Thornquist. Amesos2 and Belos: Direct and iterative solvers for large sparse linear systems.Sci. Program., 20(3):241–255, 2012

    2012

  3. [3]

    Beckermann, S

    B. Beckermann, S. A. Goreinov, and E. E. Tyrtyshnikov. Some remarks on the Elman estimate for GMRES.SIAM J. Matrix Anal. Appl., 27(3):772–778, 2005

    2005

  4. [4]

    M. Benzi. Preconditioning techniques for large linear systems: a survey.J. Comput. Phys., 182(2):418–477, 2002

    2002

  5. [5]

    P. Brown. A theoretical comparison of the Arnoldi and GMRES algorithms.SIAM J. Sci. Stat. Comput., 12(1):58–78, 1991

    1991

  6. [6]

    D. A. Knoll and D. E. Keyes. Jacobian-free Newton–Krylov methods: a survey of approaches and applications.J. Comput. Phys., 193(2):357–397, 2004

    2004

  7. [7]

    H. C. Elman.Iterative Methods for Large Sparse Nonsymmetric Systems of Linear Equations. PhD thesis, Department of Mathematics, Yale University, New Haven, CT, 1982

    1982

  8. [8]

    Ge and F

    L. Ge and F. Sotiropoulos. A numerical method for solving the 3d unsteady incompressible Navier–Stokes equations in curvilinear domains with complex immersed boundaries.J. Comput. Phys., 225(2):1782–1809, 2007

    2007

  9. [9]

    Stabilizing randomized GMRES through flexible GMRES

    S. G¨ uttel and J. W. Pearson. Stabilizing randomized GMRES through flexible GMRES. Tech- nical Report arXiv:2506.18408, arXiv, 2025

    work page internal anchor Pith review arXiv 2025

  10. [10]

    Y. Jang, L. Grigori, E. Martin, and C. Content. Randomized flexible GMRES with deflated restarting.Numer. Alg., 98(1):431–465, 2025

    2025

  11. [11]

    Z. Li, G. Dong, and X. Yang. Onset of wake meandering for a floating offshore wind turbine under side-to-side motion.J. Fluid Mech., 934:A29, 2022

    2022

  12. [12]

    C. Liu, F. Roters, and D. Raabe. Role of grain-level chemo-mechanics in composite cathode degradation of solid-state lithium batteries.Nat. Commun., 15(1):7970, 2024

    2024

  13. [13]

    C. Liu, P. Shanthraj, M. Diehl, F. Roters, S. Dong, J. Dong, W. Ding, and D. Raabe. An integrated crystal plasticity–phase field model for spatially resolved twin nucleation, prop- agation, and growth in hexagonal materials.Int. J. Plast., 106:203–227, 2018

    2018

  14. [14]

    R. B. Morgan. GMRES with deflated restarting.SIAM J. Sci. Comput., 24(1):20–37, 2002

    2002

  15. [15]

    Y. Saad. A flexible inner-outer preconditioned GMRES algorithm.SIAM J. Sci. Comput., 14(2):461–469, 1993

    1993

  16. [16]

    Saad.Iterative Methods for Sparse Linear Systems, 2nd edition

    Y. Saad.Iterative Methods for Sparse Linear Systems, 2nd edition. SIAM, Philadelphia, PA, 2003

    2003

  17. [17]

    Saad.Numerical Methods for Large Eigenvalue Problems: Revised Edition

    Y. Saad.Numerical Methods for Large Eigenvalue Problems: Revised Edition. SIAM, 2011

    2011

  18. [18]

    Saad and M

    Y. Saad and M. Schultz. GMRES: A generalized minimal residual algorithm for solving non- symmetric linear systems.SIAM J. Sci. Stat. Comput., 7(3):856–869, 1986

    1986

  19. [19]

    Simoncini and D

    V. Simoncini and D. B. Szyld. Flexible inner-outer Krylov subspace methods.SIAM J. Numer. Anal., 40(6):2219–2239, 2002

    2002

  20. [20]

    Simoncini and D

    V. Simoncini and D. B. Szyld. Theory of inexact Krylov subspace methods and applications to scientific computing.SIAM J. Sci. Comput., 25(2):454–477, 2003

    2003

  21. [21]

    Simoncini and D

    V. Simoncini and D. B. Szyld. Recent computational developments in Krylov subspace methods for linear systems.Numer. Linear Algebra Appl., 14(1):1–59, 2007

    2007

  22. [22]

    C. Vuik. New insights in GMRES-like methods with variable preconditioners.J. Comput. Appl. Math., 61(2):189–204, 1995

    1995