REVIEW 3 major objections 3 minor 1 cited by
The IC smoother inside a multigrid V-cycle can be stored and applied at far lower precision than the level-transfer operations, because its rounding error scales with the condition number of the incomplete-Cholesky factor rather than with t
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 23:40 UTC pith:LPN3V44I
load-bearing objection Genuine theoretical extension of mixed-precision multigrid analysis, but the headline half-precision experiments violate the paper's own Theorem 5.3 conditions. the 3 major comments →
Mixed precision multigrid with smoothing based on incomplete Cholesky factorization
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the relative finite-precision error of one V-cycle is bounded by a sum over levels in which the IC smoother contributes a term proportional to the condition number of the IC factor, while the other level operations contribute a term proportional to the square root of the condition number of the system matrix. In the paper's notation, Λ_V = Λ_0 + Σ_j [3‖A_j‖Λ_{Mj} + ε̇_j κ_{A_j}^{1/2}(C_{1,j}‖M_j‖ + C_{2,j})], with Λ_{Mj} for IC smoothing bounded by roughly 2(εR_j + εS_j(m_{L_j,εS}+1/2)) κ_{L_j} ‖L_j^{-1}‖². Because the smoother error is controlled by κ_{L_j} and the level-operation error by κ_{A_j}^{1/2}, in settings where κ_{L_j} (times its constants) is smaller th
What carries the argument
The key object is the per-level error split in the V-cycle bound: the smoothing error enters as 3‖A_j‖Λ_{M_j}, while the residual, restriction, prolongation, and correction operations enter as ε̇_j κ_{A_j}^{1/2}(C_{1,j}‖M_j‖ + C_{2,j}). For IC smoothing, Λ_{M_j} is bounded via a perturbed-triangular-solve analysis: rounding the IC factor to εR and solving with εS gives a factor proportional to (εR + εS(m_{L_j}+1/2)) κ_{L_j} ‖L_j^{-1}‖². This mechanism separates the smoother's precision requirement from the conditioning of the system matrix, and it is exactly what permits the IC smoother to run at lower precision than the other level operations.
Load-bearing premise
All conclusions are conditional on the exact-arithmetic smoother and V-cycle being contractions (error factor strictly less than one); the paper says it cannot prove this and only verifies it numerically for its Poisson test problems, so a non-contracting smoother would make the finite-precision bound vacuous.
What would settle it
A concrete check is to compute, for a chosen PDE and IC fill-in, the exact smoother contraction factor ‖I − MA‖_A, the condition number κ_L of the IC factor, and κ_A^{1/2}. If a problem can be exhibited where ‖I − MA‖_A ≥ 1, the derived contraction factor ρ_V + Λ_V is vacuous; if a problem with κ_L ≥ κ_A^{1/2} still permits a tenfold lower IC precision without changing the iteration count, the bound's scaling is not the controlling mechanism.
If this is right
- On each level, the precision for IC smoothing can be chosen independently of the precision for residual, restriction, prolongation, and correction.
- When κ_{L_j} is small relative to κ_{A_j}^{1/2}, the IC smoothing precision can be significantly lower than the level-operation precision without changing the V-cycle contraction factor.
- The per-level precision requirements can be tuned individually: coarse levels can run at lower precision than fine levels, and changing IC fill-in alters the required smoothing precision through the properties of L_j.
- For the Poisson problems tested, mixed-precision IC smoothing with the same iteration count as uniform double precision yields up to 1.43× speedup, down to 0.714 relative energy, and up to 16% memory reduction.
- The same analysis applies when the V-cycle is used as a preconditioner; PCG with low-precision IC smoothing shows similar speedups and energy savings, confirming that the smoother's precision budget is separate from the level operations.
Where Pith is reading between the lines
- This suggests that for a broad class of elliptic problems, the dominant precision requirement on fine levels may shift from the smoother to the level-transfer operations, so profiling should focus on the residual/restriction/prolongation path rather than the triangular solves.
- The same bounding structure likely extends to other smoothers such as ILU, Gauss-Seidel, or SOR, where the smoother's error contribution would involve a condition number of the smoother or its factors, giving a template for deciding which smoothers to run at low precision.
- Since the analysis ignores rounding errors from computing the IC factorization itself, a natural extension is to compute the factorization directly in the low precision; if the behavior matches the bound, the memory and time savings would grow further.
- The bound is worst case, so the true precision headroom for IC smoothing may be even larger than the theory predicts, and runtime quantisation effects such as mixed-format triangular solves could be exploited further.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a mixed-precision formulation of the multigrid V-cycle in which the finite-precision behavior of the smoother and the coarsest-level solver is modeled through generic relative-error assumptions rather than by assigning a single unit roundoff to all operations. A two-grid bound (Theorem 3.1) and a V-cycle bound (Theorem 4.1) are derived; the distinctive structural feature is that the smoother error enters through ∥A_j∥Λ_{M_j}, not through κ_{A_j}^{1/2}. The paper then specializes the smoother to incomplete Cholesky smoothing: Theorem 5.3 bounds the finite-precision error of a mixed-precision triangular substitution step under the condition η_L κ_L < 1/2, and Section 6 combines this with the V-cycle bound to conclude that, when κ_{L_j} is smaller than κ_{A_j}^{1/2}, the IC smoothing can be run at lower precision than the residual/restriction/prolongation/correction operations. This is illustrated in 1D simulated-precision experiments and in 3D GPU experiments with Ginkgo, where the d-s-h-sh and s-s-h-sh variants achieve a 1.43× speedup and about 0.71 relative energy while preserving the iteration count.
Significance. The paper's modular treatment of mixed-precision multigrid is a useful contribution. The central structural insight—that the smoothing error term is multiplied by ∥A_j∥ rather than by κ_{A_j}^{1/2}—appears robust and has clear practical consequences for choosing equilibria between ε_S/ε_R and ˙ε. The proofs are detailed and mostly standard, and the experiments are reproducible (Zenodo DOI is provided). The analysis of the IC smoother, in particular the use of the sparse substitution backward-error bound with m_L rather than matrix size, is a nice step. The main caveat is that the paper's own headline GPU experiments appear to lie outside the validity regime of Theorem 5.3, and the exact-arithmetic contraction assumption (4.1) is explicitly left unverified; these points weaken the claimed interaction between theory and experiments, though they do not invalidate the theoretical derivation itself.
major comments (3)
- [Section 9, Table 1/Table 2, and Theorem 5.3] The central bound (6.2) relies on Theorem 5.3, which requires m_L ε_S < 1 and η_L κ_L < 1/2 with η_L = ε_R + ε_S m_{L,ε_S} + ε_R ε_S m_{L,ε_S}. For the 3D experiment the paper reports max_j m_{L_j} = 1215. For the variants d-s-h-sh and s-s-h-sh, if the triangular solve is genuinely performed at half precision (ε_S = 2^{-11}), then m_L ε_S ≈ 0.59, m_{L,ε_S} ≈ 2985, and even with ε_R = single precision η_L ≈ 1.46, so η_L κ_L > 1/2 for any κ_L ≥ 1. The theorem's condition is therefore violated for the very variants that produce the headline speedups. If, instead, the 'single-half mix' implementation performs the substitution arithmetic in single precision and only stores intermediate values in half precision, then the half precision is a storage effect not captured by the ε_S term in the error model. In either case, the paper needs to report η_L κ_L for every GPU variant, including h-s-h-sh
- [Section 6, eq. (6.1), and the numerical verification of assumption (4.1)] The entire finite-precision V-cycle analysis perturbs a method that is assumed to contract in exact arithmetic: assumption (4.1), ∥I_j − M_j A_j∥_{A_j} < 1, is needed for Theorems 3.1 and 4.1. The paper explicitly states in Section 6 that it is 'not able to theoretically verify' this assumption. Since the h-s-h-sh variant in Section 9 stagnates at rel. res. 1.5·10^{-2} or 2.5·10^{-2}, at least in that case the exact-arithmetic contraction premise is violated (or the finite-precision perturbation is large). For the convergent low-precision variants, the paper does not report any numerical verification of (4.1). A precondition check of ∥I_j − M_j A_j∥_{A_j} (e.g., via a few Lanczos steps) for the actual IC factors used in Tables 2 and 3 would make the connection between theory and experiments much stronger; without it, the same-iteration-count results are only heuristic evidence. This conc
- [Theorem 3.1, statement and proof] The constant ξ is defined inconsistently. In the theorem statement, ξ = ∥A_C^{-1}∥^{1/2}∥A^{-1}∥^{1/2}; in the proof, immediately after eq. (3.23), ξ is written as ∥A∥^{1/2}/∥A_C∥^{1/2}. Neither matches the derivation of K9: to obtain κ_A^{1/2} ξ = ∥A∥^{1/2}∥A_C^{-1}∥^{1/2}, one needs ξ = ∥A_C^{-1}∥^{1/2}/∥A^{-1}∥^{1/2}. Since C1 and C2 in (3.6) depend on ξ, the numerical value of the bound in Theorem 3.1, and consequently the constants in Theorem 4.1, are affected. This is a local but nontrivial error in a central displayed result and must be corrected.
minor comments (3)
- [Section 6, eq. (6.2)] Eq. (6.1) defines Λ_{M_j} = 2(ε_R + ε_S(m_{L,ε_S}+1/2))κ_{L_j}∥L_j^{-1}∥². Substituting into Theorem 4.1's term 3∥A_j∥Λ_{M_j} would give a factor 6 in the first sum of (6.2), but (6.2) writes 3(ε_R + ε_S(...)) without the factor 2. The simplified 1D estimate in Section 8 correctly uses 6, so (6.2) should be reconciled with (6.1) and Theorem 4.1.
- [Section 5.2, proof of Theorem 5.3] In the displayed bounds for K2 and the final accumulated error, the proof writes κ_T and m_{T,ε_S}; the symbol should be κ_L and m_{L,ε_S}. This is a notational slip in an otherwise clear derivation.
- [Table 1] The four-letter variant names are not fully explained. The table header lists only three precision columns (residual etc., L storage, triangular solve), but the variants have four letters (e.g., d-s-h-sh). The meaning of the fourth letter—apparently related to the 'single-half mix' storage mode—should be made explicit in the table caption or in the surrounding text.
Circularity Check
No significant circularity: derived bounds are self-contained and not reduced to fitted parameters or self-citations.
full rationale
The derivation chain is a standard forward error analysis: the paper postulates general finite-precision assumptions (3.3)-(3.4) and (4.2)-(4.3), proves Theorems 3.1 and 4.1 line-by-line (with the proof of Theorem 3.1 and Appendix B), then derives the IC-smoother bound (5.5) from Lemma 5.2, which is itself proved in Appendix C. The final estimate (6.2) is an algebraic consequence of (4.1), (5.5), and the V-cycle bound, with explicit constants C1,j, C2,j and no parameter fitted to the experiments. The observation that epsilon_R_j and epsilon_S_j can be lower than epsilon_dot_j when kappa_Lj is smaller than kappa^{1/2}_{Aj} is a direct reading of the displayed terms, not an input disguised as a conclusion. The experimental sections measure dS_min and ddot_min after the theory and do not calibrate it. The paper explicitly states the limitation of assumption (4.1) in Section 6 ('We note that we are not able to theoretically verify the assumption (4.1) It can, however, be verified numerically in concrete settings.'), and the conditions of Theorem 5.3 (mL*epsilon_S < 1, eta_L*kappa_L < 1/2) are stated as hypotheses; whether the half-precision experiments satisfy them is a validity/verification concern, not a circularity. References [16,17,22] are foundational but are not self-citations of the present authors, and the proofs are largely reproduced; other self-references [15,26,27,28] are background or implementation-related, not load-bearing. No step reduces the claimed result to its own inputs.
Axiom & Free-Parameter Ledger
free parameters (3)
- ICT(dpt) drop tolerance =
5e-3
- Value filtering thresholds after scaling =
5e-16 (system matrices), 5e-12 (prolongation)
- Coarsest-level CG tolerance for h-s-h-sh variant =
1e-4 relative residual, max 100 iterations
axioms (9)
- standard math Standard floating-point rounding model (2.5)–(2.8): component-wise rounding bounds and operation bounds for vector/matrix arithmetic.
- domain assumption Galerkin condition A_{j-1} = P_j^⊤ A_j P_j and restriction = prolongation^⊤.
- domain assumption Exact-precision smoother contraction ∥I_j − M_jA_j∥_{A_j} < 1 (eq. (4.1); eq. (3.1) for two-grid).
- domain assumption Exact-precision V-cycle contraction with uniform factor ρV < 1 for all sub-grids (eq. (4.4)).
- domain assumption Smoother and coarsest-solver finite-precision error models (4.2)–(4.3) with constants ΛMj and Λ0.
- standard math Smallness conditions mLεS < 1 and ηLκL < 1/2 for the triangular-solve perturbation bounds.
- domain assumption Finite precision errors from computing the IC factorization itself are ignored.
- standard math No overflow/underflow during computation.
- standard math Triangular solves are performed via substitution.
read the original abstract
Multigrid methods are popular iterative methods for solving large-scale sparse systems of linear equations. We present a mixed precision formulation of the multigrid V-cycle method with general assumptions on the finite precision errors coming from the application of coarsest-level solver and smoothing. Inspired by existing analysis, we derive a bound on the relative finite precision error of the V-cycle which gives insight into how the finite precision errors from the individual components of the method may affect the overall finite precision error. We use the result to study V-cycle methods with smoothing based on incomplete Cholesky factorization. The results imply that in certain settings, the precisions used for applying the incomplete Cholesky smoothing can be significantly lower than the precision used for computing the residual, restriction, prolongation, and correction on a given level. We perform numerical experiments using simulated floating point arithmetic with the MATLAB Advanpix toolbox as well as experiments on GPUs using the Ginkgo library. The experiments illustrate the theoretical findings and show that in the considered settings, the incomplete Cholesky smoothing can be applied in relatively low precisions, resulting in significant improvements in the execution time (up to 28% less), energy savings (up to 26% less) and required memory (up to 16% less) in comparison with the uniform double precision variant.
Forward citations
Cited by 1 Pith paper
-
Iterative Poisson Solvers for Self-gravity with the GPU Code Astaroth
GPU-accelerated iterative Poisson solvers for self-gravity are implemented and tested in Astaroth, achieving convergence and timing performance comparable to existing methods while supporting production-scale astrophy...
Reference graph
Works this paper leans on
-
[1]
Advanpix Multiprecision Computing Toolbox for MATLAB, version 5.1.0.15432 , Advanpix LLC., Yokohama, Japan, https://www.advanpix.com/
-
[2]
Abdelfattah, H
A. Abdelfattah, H. Anzt, E. G. Boman, E. Carson, T. Cojean, J. Dongarra, A. Fox, M. Gates, N. J. Higham, X. S. Li, J. Loe, P. Luszczek, S. Pranesh, S. Rajaman- ickam, T. Ribizel, B. F. Smith, K. Swirydowicz, S. Thomas, S. Tomov, Y. M. Tsai, and U. M. Yang , A survey of numerical linear algebra methods utilizing mixed-precision arithmetic, The Internationa...
2021
-
[3]
M. S. Alnaes, J. Blechta, J. Hake, et al. , The FEniCS project version 1.5 , Archive of Numerical Software, 3 (2015)
2015
-
[4]
H. Anzt, T. Cojean, G. Flegar, F. G¨obel, T. Gr¨utzmacher, P. Nayak, T. Ribizel, Y. M. Tsai, and E. S. Quintana-Ort ´ı, Ginkgo: A Modern Linear Operator Algebra Frame- work for High Performance Computing , ACM Transactions on Mathematical Software, 48 (2022), pp. 2:1 – 2:33
2022
-
[5]
Brandt and O
A. Brandt and O. E. Livne , Multigrid Techniques, Society for Industrial and Applied Math- ematics, 2011
2011
-
[6]
W. L. Briggs, V. E. Henson, and S. F. McCormick , A Multigrid Tutorial , SIAM, Philadel- phia, PA, second ed., 2000
2000
-
[7]
Carson and N
E. Carson and N. J. Higham, Accelerating the Solution of Linear Systems by Iterative Refine- ment in Three Precisions , SIAM Journal on Scientific Computing, 40 (2018), pp. A817– A847
2018
-
[8]
Cojean, P
T. Cojean, P. Nayak, T. Ribizel, N. Beams, Y.-H. M. Tsai, M. Koch, F. G ¨obel, T. Gr¨utzmacher, and H. Anzt, Ginkgo - A math library designed to accelerate Exascale Computing Project science applications , The International Journal of High Performance Computing Applications, (2024)
2024
-
[9]
Drzisga, A
D. Drzisga, A. W agner, and B. Wohlmuth, A Matrix-Free ILU Realization Based on Sur- rogates, SIAM Journal on Scientific Computing, 45 (2023), pp. C304–C329
2023
-
[10]
N. J. Higham , Accuracy and Stability of Numerical Algorithms , Society for Industrial and Applied Mathematics, second ed., 2002
2002
-
[11]
N. J. Higham and T. Mary , Mixed precision algorithms in numerical linear algebra , Acta Numerica, 31 (2022), p. 347–414
2022
-
[12]
N. J. Higham, S. Pranesh, and M. Zounon , Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems , SIAM Journal on Scientific Computing, 41 (2019), pp. A2536–A2551
2019
-
[13]
Kettler, Analysis and comparison of relaxation schemes in robust multigrid and precon- ditioned conjugate gradient methods , in Multigrid Methods, W
R. Kettler, Analysis and comparison of relaxation schemes in robust multigrid and precon- ditioned conjugate gradient methods , in Multigrid Methods, W. Hackbusch and U. Trot- tenberg, eds., Berlin, Heidelberg, 1982, Springer Berlin Heidelberg, pp. 502–534
1982
-
[14]
Kettler and P
R. Kettler and P. Wesseling, Aspects of multigrid methods for problems in three dimensions, Applied Mathematics and Computation, 19 (1986), pp. 159–168
1986
-
[15]
N. Kohl, S. F. McCormick, and R. Tamstorf , Multigrid Methods Using Block Floating Point Arithmetic, SIAM Journal on Scientific Computing, 46 (2024), pp. S202–S224
2024
-
[16]
S. F. McCormick, J. Benzaken, and R. Tamstorf , Algebraic error analysis for mixed- precision multigrid solvers , SIAM Journal on Scientific Computing, 43 (2021), pp. S392– S419
2021
-
[17]
S. F. McCormick and R. Tamstorf , Rounding-Error Analysis of Multigrid V -Cycles, SIAM Journal on Scientific Computing, 46 (2024), pp. S88–S95
2024
-
[18]
Notay, Convergence Analysis of Perturbed Two-Grid and Multigrid Methods, SIAM Journal on Numerical Analysis, 45 (2007), pp
Y. Notay, Convergence Analysis of Perturbed Two-Grid and Multigrid Methods, SIAM Journal on Numerical Analysis, 45 (2007), pp. 1035–1044
2007
-
[19]
Notay, Algebraic Theory of Two-Grid Methods, Numerical Mathematics: Theory, Methods and Applications, 8 (2015), pp
Y. Notay, Algebraic Theory of Two-Grid Methods, Numerical Mathematics: Theory, Methods and Applications, 8 (2015), pp. 168–198
2015
-
[20]
Saad , Iterative Methods for Sparse Linear Systems , Society for Industrial and Applied Mathematics, second ed., 2003
Y. Saad , Iterative Methods for Sparse Linear Systems , Society for Industrial and Applied Mathematics, second ed., 2003
2003
-
[21]
Scott and M
J. Scott and M. T ˚uma, Algorithms for sparse linear systems , Springer Nature, 2023
2023
-
[22]
Tamstorf, J
R. Tamstorf, J. Benzaken, and S. F. McCormick , Discretization-Error-Accurate Mixed- Precision Multigrid Solvers , SIAM Journal on Scientific Computing, 43 (2021), pp. S420– S447
2021
-
[23]
Thomas, A
S. Thomas, A. Carr, P. Mullowney, K. ´Swirydowicz, and M. Day, Scaled ILU smoothers for Navier–Stokes pressure projection , International Journal for Numerical Methods in Fluids, 96 (2024), pp. 537–560
2024
-
[24]
Tielen, M
R. Tielen, M. M ¨oller, D. G ¨oddeke, and C. Vuik , p-multigrid methods and their compar- MIXED PRECISION MULTIGRID WITH IC SMOOTHING 25 ison to h-multigrid methods within isogeometric analysis , Computer Methods in Applied Mechanics and Engineering, 372 (2020), p. 113347
2020
-
[25]
Trottenberg, C
U. Trottenberg, C. W. Oosterlee, and A. Schuller , Multigrid, Academic Press, London, 2001
2001
-
[26]
Tsai, Portable Mixed Precision Algebraic Multigrid on High Performance GPUs , PhD thesis, Karlsruher Institut f¨ ur Technologie (KIT), 2024
Y.-H. Tsai, Portable Mixed Precision Algebraic Multigrid on High Performance GPUs , PhD thesis, Karlsruher Institut f¨ ur Technologie (KIT), 2024
2024
-
[27]
Y.-H. M. Tsai, N. Beams, and H. Anzt , Mixed Precision Algebraic Multigrid on GPUs , in Parallel Processing and Applied Mathematics, R. Wyrzykowski, J. Dongarra, E. Deelman, and K. Karczewski, eds., Cham, 2023, Springer International Publishing, pp. 113–125
2023
-
[28]
Y.-H. M. Tsai, N. Beams, and H. Anzt, Three-precision algebraic multigrid on GPUs, Future Generation Computer Systems, 149 (2023), pp. 280–293
2023
-
[29]
Wesseling, A robust and efficient multigrid method , in Multigrid Methods, W
P. Wesseling, A robust and efficient multigrid method , in Multigrid Methods, W. Hackbusch and U. Trottenberg, eds., Berlin, Heidelberg, 1982, Springer Berlin Heidelberg, pp. 614– 630
1982
-
[30]
Wesseling , Theoretical and practical aspects of a multigrid method , SIAM Journal on Scientific and Statistical Computing, 3 (1982), pp
P. Wesseling , Theoretical and practical aspects of a multigrid method , SIAM Journal on Scientific and Statistical Computing, 3 (1982), pp. 387–407
1982
-
[31]
Yserentant , Old and new convergence proofs for multigrid methods , Acta Numerica, 2 (1993), pp
H. Yserentant , Old and new convergence proofs for multigrid methods , Acta Numerica, 2 (1993), pp. 285–326
1993
-
[32]
Y. Zong, P. Yu, H. Huang, and W. Xue , FP16 Acceleration in Structured Multigrid Precon- ditioner for Real-World Applications , in Proceedings of the 53rd International Conference on Parallel Processing, ICPP ’24, New York, NY, USA, 2024, Association for Computing Machinery, p. 52–62. Appendix A. Derivation of inequalities (2.1) - (2.4). Ad. (2.1): ∥v∥2 A...
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.