Convergence analysis and proof of acceleration for NGMRES applied to the Picard iteration for Navier-Stokes equations

Leo G Rebholz; Yunhui He

arxiv: 2604.12922 · v1 · submitted 2026-04-14 · 🧮 math.NA · cs.NA

Convergence analysis and proof of acceleration for NGMRES applied to the Picard iteration for Navier-Stokes equations

Yunhui He , Leo G Rebholz This is my paper

Pith reviewed 2026-05-10 14:21 UTC · model grok-4.3

classification 🧮 math.NA cs.NA

keywords NGMRESPicard iterationNavier-Stokesconvergence analysisnonlinear accelerationLipschitz constantleast-squares optimization

0 comments

The pith

NGMRES accelerates Picard iteration for Navier-Stokes by scaling its Lipschitz constant with the optimization gain.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a convergence analysis for nonlinear GMRES (NGMRES) used to accelerate the Picard iteration on the Navier-Stokes equations. It proves that NGMRES of arbitrary depth reduces the Lipschitz constant of the underlying Picard map by exactly the gain factor obtained from the least-squares optimization step inside the accelerator. This supplies the first rigorous identification of the mechanism that produces acceleration and shows the estimates remain sharp in practice while enabling convergence in regimes where the plain Picard method diverges.

Core claim

The central claim is that NGMRES applied to the Picard iteration for Navier-Stokes equations scales the Picard Lipschitz constant by the gain of the optimization problem arising in the NGMRES least-squares step. The proof holds for general depth once the optimal norm for that least-squares problem is identified, and numerical tests confirm that the resulting convergence bounds are tight while the accelerated iteration succeeds even when the unaccelerated Picard iteration fails to converge.

What carries the argument

The optimization gain produced by the least-squares problem inside NGMRES, which directly multiplies and thereby reduces the Picard Lipschitz constant.

If this is right

The Picard iteration converges whenever the NGMRES optimization gain is sufficiently smaller than the reciprocal of the original Lipschitz constant.
The same scaling relation holds for any depth of NGMRES, so deeper histories produce at least as strong acceleration.
The method remains effective on problems where the plain Picard iteration diverges.
The derived convergence bounds match observed rates closely enough to serve as practical a-priori estimates.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same gain-based scaling argument could be tested on other fixed-point iterations arising in nonlinear PDEs.
Identifying the optimal norm once per problem class might allow pre-tuned NGMRES variants that avoid repeated norm searches.
Because the proof isolates the gain as the sole source of acceleration, any other accelerator that achieves a comparable gain should produce similar convergence improvement.

Load-bearing premise

The analysis requires that an optimal norm exists for the NGMRES least-squares problem and that the resulting optimization gain is well-defined and positive.

What would settle it

Compute the effective Lipschitz constant of the Picard map on a concrete Navier-Stokes problem both with and without NGMRES; if the measured contraction factor is not equal to the unaccelerated factor times the observed optimization gain, the scaling claim is false.

Figures

Figures reproduced from arXiv: 2604.12922 by Leo G Rebholz, Yunhui He.

**Figure 2.** Figure 2: The plots above show (top) convergence of NGMRES for NSE 2D driven cavity for varying [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗

**Figure 3.** Figure 3: The plot above show convergence results for [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗

**Figure 4.** Figure 4: The plots above show convergence of NGMRES-Picard for NSE 2D driven cavity for varying [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗

**Figure 5.** Figure 5: The plot above shows the Re=1000 3D driven cavity velocity solution displayed as midsliceplanes of velocity. 0 20 40 60 80 100 Iteration k 10-6 10-4 10-2 100 || g(u k) ||V / Re=400 Picard m=0 m=5 m=10 m=20 0 20 40 60 80 100 Iteration k 0.4 0.5 0.6 0.7 0.8 0.9 1 Re=400 m=0 k m=0 ||g(uk )||V //||g(uk-1)||V / m=10 k m=10 ||g(uk )||V //||g(uk-1)||V / 0 20 40 60 80 100 Iteration k 10-6 10-4 10-2 100 || g(u k) … view at source ↗

**Figure 6.** Figure 6: The plots above show (left) convergence of NGMRES-Picard for NSE 3D driven cavity, for [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗

**Figure 7.** Figure 7: The plots above show convergence of NGMRES-Picard for NSE 3D driven cavity for varying [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗

**Figure 8.** Figure 8: Shown above is the artery mesh (before the barycenter refinement is applied) restricted to [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗

**Figure 9.** Figure 9: Shown above are contour slices of the velocity magnitude for the [PITH_FULL_IMAGE:figures/full_fig_p020_9.png] view at source ↗

**Figure 10.** Figure 10: The plots above show convergence behavior of NGMRES-Picard for 3D artery model, for [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗

read the original abstract

We consider nonlinear GMRES (NGMRES) as an acceleration technique for the Navier-Stokes Picard iteration, a direction that has not previously been explored. We identify the optimal norm for the least squares optimization problem arising in the NGMRES algorithm, and establish a convergence analysis for NGMRES with general depth that proves NGMRES scales the Picard Lipschitz constant by the gain of the optimization problem. To our knowledge, this is the first convergence proof for NGMRES that identifies the mechanism responsible for convergence acceleration. Numerical experiments demonstrate that the convergence estimates are remarkably sharp. In addition, NGMRES greatly improves the performance of the Picard iteration, even in cases where the unaccelerated iteration diverges.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

NGMRES gets a convergence proof for the NS Picard case, but the key scaling step may not hold when the base iteration diverges.

read the letter

The paper's main contribution is a convergence analysis for nonlinear GMRES accelerating the Picard iteration on the Navier-Stokes equations. It claims this is the first proof that identifies how NGMRES works by scaling the Lipschitz constant of the Picard map with the gain from its least-squares optimization. The new part is applying NGMRES here and deriving the scaling result for general depth. They pick what they call the optimal norm for the least-squares problem and show the acceleration comes from that gain factor. The numerical tests are a plus: they indicate the bounds are sharp and that NGMRES speeds things up or makes them converge when the plain Picard method does not. The potential issue is whether that optimal norm actually delivers a gain less than one in the cases that matter. The proof assumes the norm makes the least-squares well-posed with gain g < 1 while keeping the Picard operator Lipschitz with constant L. For Navier-Stokes, the Picard iteration is typically only contractive under small data assumptions or in particular norms, and it often diverges at moderate Reynolds numbers. The paper does not appear to give explicit conditions that guarantee g stays below one uniformly or when the unaccelerated version fails. If the gain can reach or exceed one in those regimes, the claimed mechanism for acceleration does not fully apply. This work is for numerical analysts and CFD practitioners who use or study iterative methods for fluid problems. A reader interested in acceleration techniques or convergence theory for nonlinear solvers will get something from the proof sketch and the experiments. It is solid enough on the surface to deserve referee time, even though the norm question will likely require more detail in revision. I recommend sending it out for peer review.

Referee Report

1 major / 2 minor

Summary. The manuscript develops a convergence analysis for nonlinear GMRES (NGMRES) used to accelerate the Picard iteration for the incompressible Navier-Stokes equations. It identifies an optimal norm for the least-squares subproblem in NGMRES and proves that, for general depth, NGMRES scales the Lipschitz constant of the Picard operator by the gain factor arising from the optimization. This is presented as the first convergence proof that explicitly identifies the acceleration mechanism. Numerical experiments are included to demonstrate that the theoretical estimates are sharp and that NGMRES improves convergence even in regimes where the unaccelerated Picard iteration diverges.

Significance. If the central result holds, the work supplies a valuable theoretical explanation for the observed acceleration of NGMRES on nonlinear fixed-point iterations arising from Navier-Stokes discretizations. The explicit scaling relation between the Picard Lipschitz constant and the optimization gain provides a clear mechanism that could inform the design of other acceleration strategies in computational fluid dynamics. The reported sharpness of the numerical estimates is a positive feature that strengthens the contribution.

major comments (1)

[convergence analysis (main theorem)] The central convergence theorem (as summarized in the abstract and introduction) states that NGMRES multiplies the Picard Lipschitz constant L by an optimization gain g < 1 in the identified optimal norm. However, the analysis does not appear to supply an explicit condition or construction guaranteeing that this same norm keeps L finite and forces g < 1 uniformly when the plain Picard iteration diverges (e.g., at high Reynolds number). This point is load-bearing for the claim that acceleration occurs in diverging cases.

minor comments (2)

[introduction] The term 'depth' for the NGMRES parameter is used in the abstract and introduction but would benefit from an early, self-contained definition before the algorithm is presented.
[numerical experiments] Numerical figures would be clearer if the captions explicitly noted which quantities are being compared to demonstrate sharpness of the estimates.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their careful reading and for acknowledging the potential significance of the explicit acceleration mechanism identified in our work. We address the major comment on the convergence analysis below.

read point-by-point responses

Referee: The central convergence theorem (as summarized in the abstract and introduction) states that NGMRES multiplies the Picard Lipschitz constant L by an optimization gain g < 1 in the identified optimal norm. However, the analysis does not appear to supply an explicit condition or construction guaranteeing that this same norm keeps L finite and forces g < 1 uniformly when the plain Picard iteration diverges (e.g., at high Reynolds number). This point is load-bearing for the claim that acceleration occurs in diverging cases.

Authors: We agree that the central theorem assumes the Picard operator has finite Lipschitz constant L in the identified optimal norm for the least-squares subproblem and then proves that NGMRES produces an effective Lipschitz constant of gL, where g is the optimization gain (strictly less than 1 whenever the minimization yields improvement). The analysis does not supply an explicit a priori construction or uniform bound on the norm that guarantees L remains finite while ensuring gL < 1 in regimes where the unaccelerated Picard iteration diverges (L ≥ 1 in standard norms at high Reynolds number). This is a fair observation regarding the scope of the theorem. The manuscript's claim of improvement in diverging cases rests on the numerical experiments, which demonstrate that NGMRES accelerates convergence even when plain Picard diverges, with the theoretical estimates remaining sharp. The contribution of the theorem is the explicit identification of the scaling mechanism itself, which applies whenever a suitable norm exists. We will revise the manuscript to state the assumptions of the theorem more explicitly and to clarify that acceleration in diverging regimes is established numerically while the theory explains the underlying contraction mechanism when the iteration is Lipschitz in the chosen norm. This is a partial revision. revision: partial

Circularity Check

0 steps flagged

No significant circularity in the claimed derivation

full rationale

The paper's central result derives the scaling of the Picard Lipschitz constant by the NGMRES optimization gain directly from the definition of the least-squares subproblem and the structure of the iteration update. This is a mathematical identity under the stated assumptions on the norm and gain, not a reduction of the output to fitted inputs or a self-referential definition. No load-bearing self-citations, imported uniqueness theorems, or ansatzes are present in the provided abstract and summary; the analysis remains self-contained and does not rename known results or force predictions by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The abstract does not introduce new parameters, entities, or ad hoc axioms beyond standard numerical analysis assumptions for convergence of iterative methods.

pith-pipeline@v0.9.0 · 5416 in / 1096 out tokens · 49591 ms · 2026-05-10T14:21:33.303082+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages

[1]

[1]D. G. Anderson,Iterative procedures for nonlinear integral equations, J. Assoc. Comput. Mach., 12 (1965), pp. 547–560. [2]D. Arnold and J. Qin,Quadratic velocity/linear pressure Stokes elements, in Advances in Computer Methods for Partial Differential Equations VII, R. Vichnevetsky, D. Knight, and G. Richter, eds., IMACS, 1992, pp. 28–34. [3]M. Benzi a...

work page 1965
[2]

Chang and I.-L

[6]Q. Chang and I.-L. Chern,Acceleration methods for total variation-based image denoising, SIAM Journal on Scientific Computing, 25 (2003), pp. 982–994. [7]T. Charnyi, T. Heister, M. Olshanskii, and L. Rebholz,On conservation laws of Navier-Stokes Galerkin discretizations, Journal of Computational Physics, 337 (2017), pp. 289–

work page 2003
[3]

De Sterck,A nonlinear GMRES optimization algorithm for canonical tensor decomposi- tion, SIAM Journal on Scientific Computing, 34 (2012), pp

21 [8]H. De Sterck,A nonlinear GMRES optimization algorithm for canonical tensor decomposi- tion, SIAM Journal on Scientific Computing, 34 (2012), pp. A1351–A1379. [9]H. De Sterck,Steepest descent preconditioning for nonlinear GMRES optimization, Numer- ical Linear Algebra with Applications, 20 (2013), pp. 453–471. [10]H. De Sterck and Y. He,On the asympt...

work page 2012
[4]

Greif and Y

[19]C. Greif and Y. He,Convergence properties of nonlinear GMRES applied to linear systems, SIAM Journal on Matrix Analysis and Applications, 47 (2026), pp. 330–352. [20]P. Gresho and R. Sani,Incompressible Flow and the Finite Element Method, vol. 2, Wiley,

work page 2026
[5]

Guzman and L

[21]J. Guzman and L. Scott,The Scott-Vogelius finite elements revisited, Math. Comp., 88 (2019), pp. 515–529. [22]W. W. Hager and H. Zhang,A survey of nonlinear conjugate gradient methods, Pacific journal of Optimization, 2 (2006), pp. 35–58. [23]E. Hawkins and L. Rebholz,On the choice of optimization norm for Anderson acceleration of the Picard iteration...

work page arXiv 2019
[6]

[30]V. John, A. Linke, C. Merdon, M. Neilan, and L. G. Rebholz,On the divergence constraint in mixed finite element methods for incompressible flows, SIAM Review, 59 (2017), pp. 492–544. [31]W. Layton,An Introduction to the Numerical Analysis of Viscous Incompressible Flows, SIAM, Philadelphia,

work page 2017
[7]

Layton, C

[32]W. Layton, C. Manica, M. Neda, M. A. Olshanskii, and L. Rebholz,On the accuracy of the rotation form in simulations of the Navier-Stokes equations, Journal of Computational Physics, 228 (2009), pp. 3433–3447. [33]C. W. Oosterlee and T. W ashio,Krylov subspace acceleration of nonlinear multigrid with application to recirculating flows, SIAM Journal on ...

work page 2009
[8]

Pollock and L

[34]S. Pollock and L. Rebholz,Anderson acceleration for contractive and noncontractive operators, IMA Journal of Numerical Analysis, 41 (2021), pp. 2841–2872. [35]S. Pollock and L. Rebholz,Filtering for Anderson acceleration, SIAM Journal on Scien- tific Computing, 45 (2023), pp. A1571–A1590. [36]S. Pollock and L. Rebholz,Anderson Acceleration for Numeric...

work page 2021
[9]

Pollock, L

[37]S. Pollock, L. Rebholz, X. Tu, and M. Xiao,Analysis of the Picard-Newton iteration for the Navier-Stokes equations: global stability and quadratic convergence, Journal of Scientific Computing, 14 (2025), pp. 1–23. [38]S. Pollock, L. Rebholz, and M. Xiao,Anderson-accelerated convergence of Picard iter- ations for incompressible Navier-Stokes equations,...

work page 2025
[10]

[39]A. N. Riseth,Objective acceleration for unconstrained optimization, Numerical Linear Alge- bra with Applications, 26 (2019), p. e2216. [40]J. Savage and K. Chen,An improved and accelerated non-linear multigrid method for total- variation denoising, International Journal of Computer Mathematics, 82 (2005), pp. 1001–

work page 2019
[11]

[41]L. R. Scott and M. Vogelius,Conforming finite element methods for incompressible and nearly incompressible continua, in Large-scale computations in fluid mechanics, Part 2, vol. 22- 2 of Lectures in Applied Mathematics, Amer. Math. Soc., 1985, pp. 221–244. [42]R. Temam,Navier-Stokes equations, Elsevier, North-Holland,

work page 1985
[12]

[43]J. M. Tempestti, S. Kim, B. Lindsey, and A. Veneziani,A pseudo-spectral method for wall shear stress estimation from Doppler ultrasound imaging in coronary arteries, Cardiovas- cular Engineering and Technology, 15 (2024), pp. 647–666. [44]M. C. Thompson and J. H. Ferziger,An adaptive multigrid technique for the incompress- ible Navier-Stokes equations...

work page 2024
[13]

Wong and A

[47]K. Wong and A. Baker,A 3D incompressible Navier-Stokes velocity-vorticity weak form finite element algorithm, International Journal for Numerical Methods in Fluids, 38 (2002), pp. 99–123. [48]S. Zhang,A new family of stable mixed finite elements for the 3D Stokes equations, Math. Comp., 74 (2005), pp. 543–554. 23

work page 2002

[1] [1]

[1]D. G. Anderson,Iterative procedures for nonlinear integral equations, J. Assoc. Comput. Mach., 12 (1965), pp. 547–560. [2]D. Arnold and J. Qin,Quadratic velocity/linear pressure Stokes elements, in Advances in Computer Methods for Partial Differential Equations VII, R. Vichnevetsky, D. Knight, and G. Richter, eds., IMACS, 1992, pp. 28–34. [3]M. Benzi a...

work page 1965

[2] [2]

Chang and I.-L

[6]Q. Chang and I.-L. Chern,Acceleration methods for total variation-based image denoising, SIAM Journal on Scientific Computing, 25 (2003), pp. 982–994. [7]T. Charnyi, T. Heister, M. Olshanskii, and L. Rebholz,On conservation laws of Navier-Stokes Galerkin discretizations, Journal of Computational Physics, 337 (2017), pp. 289–

work page 2003

[3] [3]

De Sterck,A nonlinear GMRES optimization algorithm for canonical tensor decomposi- tion, SIAM Journal on Scientific Computing, 34 (2012), pp

21 [8]H. De Sterck,A nonlinear GMRES optimization algorithm for canonical tensor decomposi- tion, SIAM Journal on Scientific Computing, 34 (2012), pp. A1351–A1379. [9]H. De Sterck,Steepest descent preconditioning for nonlinear GMRES optimization, Numer- ical Linear Algebra with Applications, 20 (2013), pp. 453–471. [10]H. De Sterck and Y. He,On the asympt...

work page 2012

[4] [4]

Greif and Y

[19]C. Greif and Y. He,Convergence properties of nonlinear GMRES applied to linear systems, SIAM Journal on Matrix Analysis and Applications, 47 (2026), pp. 330–352. [20]P. Gresho and R. Sani,Incompressible Flow and the Finite Element Method, vol. 2, Wiley,

work page 2026

[5] [5]

Guzman and L

[21]J. Guzman and L. Scott,The Scott-Vogelius finite elements revisited, Math. Comp., 88 (2019), pp. 515–529. [22]W. W. Hager and H. Zhang,A survey of nonlinear conjugate gradient methods, Pacific journal of Optimization, 2 (2006), pp. 35–58. [23]E. Hawkins and L. Rebholz,On the choice of optimization norm for Anderson acceleration of the Picard iteration...

work page arXiv 2019

[6] [6]

[30]V. John, A. Linke, C. Merdon, M. Neilan, and L. G. Rebholz,On the divergence constraint in mixed finite element methods for incompressible flows, SIAM Review, 59 (2017), pp. 492–544. [31]W. Layton,An Introduction to the Numerical Analysis of Viscous Incompressible Flows, SIAM, Philadelphia,

work page 2017

[7] [7]

Layton, C

[32]W. Layton, C. Manica, M. Neda, M. A. Olshanskii, and L. Rebholz,On the accuracy of the rotation form in simulations of the Navier-Stokes equations, Journal of Computational Physics, 228 (2009), pp. 3433–3447. [33]C. W. Oosterlee and T. W ashio,Krylov subspace acceleration of nonlinear multigrid with application to recirculating flows, SIAM Journal on ...

work page 2009

[8] [8]

Pollock and L

[34]S. Pollock and L. Rebholz,Anderson acceleration for contractive and noncontractive operators, IMA Journal of Numerical Analysis, 41 (2021), pp. 2841–2872. [35]S. Pollock and L. Rebholz,Filtering for Anderson acceleration, SIAM Journal on Scien- tific Computing, 45 (2023), pp. A1571–A1590. [36]S. Pollock and L. Rebholz,Anderson Acceleration for Numeric...

work page 2021

[9] [9]

Pollock, L

[37]S. Pollock, L. Rebholz, X. Tu, and M. Xiao,Analysis of the Picard-Newton iteration for the Navier-Stokes equations: global stability and quadratic convergence, Journal of Scientific Computing, 14 (2025), pp. 1–23. [38]S. Pollock, L. Rebholz, and M. Xiao,Anderson-accelerated convergence of Picard iter- ations for incompressible Navier-Stokes equations,...

work page 2025

[10] [10]

[39]A. N. Riseth,Objective acceleration for unconstrained optimization, Numerical Linear Alge- bra with Applications, 26 (2019), p. e2216. [40]J. Savage and K. Chen,An improved and accelerated non-linear multigrid method for total- variation denoising, International Journal of Computer Mathematics, 82 (2005), pp. 1001–

work page 2019

[11] [11]

[41]L. R. Scott and M. Vogelius,Conforming finite element methods for incompressible and nearly incompressible continua, in Large-scale computations in fluid mechanics, Part 2, vol. 22- 2 of Lectures in Applied Mathematics, Amer. Math. Soc., 1985, pp. 221–244. [42]R. Temam,Navier-Stokes equations, Elsevier, North-Holland,

work page 1985

[12] [12]

[43]J. M. Tempestti, S. Kim, B. Lindsey, and A. Veneziani,A pseudo-spectral method for wall shear stress estimation from Doppler ultrasound imaging in coronary arteries, Cardiovas- cular Engineering and Technology, 15 (2024), pp. 647–666. [44]M. C. Thompson and J. H. Ferziger,An adaptive multigrid technique for the incompress- ible Navier-Stokes equations...

work page 2024

[13] [13]

Wong and A

[47]K. Wong and A. Baker,A 3D incompressible Navier-Stokes velocity-vorticity weak form finite element algorithm, International Journal for Numerical Methods in Fluids, 38 (2002), pp. 99–123. [48]S. Zhang,A new family of stable mixed finite elements for the 3D Stokes equations, Math. Comp., 74 (2005), pp. 543–554. 23

work page 2002