Pith. sign in

REVIEW 4 major objections 7 minor 27 references

Deterministic and Probabilistic Rounding Error Analysis for Mixed-Precision Arithmetic on Modern Computing Units

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Treating rounding errors as random rather than worst-case cuts accumulated error bounds for tensor-core matrix multiplication by roughly a factor of ten.

desk verdict The deterministic parts are fine, but the paper's headline probabilistic Tensor Core bound is a deterministic bound in disguise, with an undefined gamma_tilde and a probability that can go negative. read the letter →

arxiv 2411.18747 v1 pith:ZALDMPMA submitted 2024-11-27 stat.CO

Sahil Bhola , Karthik Duraisamy This is my paper · ORCID
classification stat.CO MSC 65G50
keywords roundingerroranalysismixed-precisionarithmeticprobabilisticboundsfusedmultiply-addtensorcoresfloating-pointmatrixmultiplicationvariance-informed
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper develops rounding-error bounds for three building blocks of numerical computing: fused multiply-add (FMA), mixed-precision FMA, and matrix-matrix multiplication on tensor cores. It argues that treating each rounding error as an independent random variable, instead of assuming the worst case, yields much tighter error estimates while keeping a probability guarantee. The headline numerical result is a nearly tenfold tightening of the accumulated-error bound for tensor-core GEMM compared with deterministic analysis. This matters because low-precision hardware is increasingly used for large scientific and machine-learning computations, and tighter bounds let users choose precision levels with more confidence.

What carries the argument

The carrying object is the VIBEA product bound, Lemma 2.2, which is imported from the authors' companion work. For $n$ independent rounding errors drawn uniformly from $[-u, u]$, it replaces the deterministic product estimate $1 + \theta$ with $|\theta| \le \gamma_n = n u/(1 - n u)$ by a random product $1 + \tilde{\theta}$ whose magnitude is bounded by $\tilde{\gamma}_n(\lambda)$ with probability at least $p_b(\lambda, u, n)$, a tail bound that involves the variance of the logarithm terms. Because $\tilde{\gamma}_n(\lambda)$ grows like $\sqrt{n}$ rather than linearly in $n$, the probabilistic bound remains much smaller for large $n$. The tensor-core analysis additionally relies on the assumption that each short-precision product is represented exactly in the intermediate precision, which removes the multiplication rounding error from the count.

What would settle it

Run the same fp16 tensor-core GEMM with inputs whose products do not fit in the intermediate precision, repeat the experiment many times, and tally how often the true forward error exceeds the Theorem 3.9 bound; an exceedance rate larger than the theorem's claimed probability would refute the independence and exact-product premises.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that variance-informed probabilistic backward error analysis (VIBEA) produces a rounding-error bound for tensor-core matrix multiplication that is roughly an order of magnitude smaller than the deterministic bound, while still carrying an explicit probability of success. In the empirical setup with fp16 inputs, fp32 accumulation, and matrices of size $m=2^{10}$, $t=2^{15}$, $n=2^{3}$, the deterministic bound is around $\mathcal{O}(10^2)$ and the probabilistic bound around $\mathcal{O}(10^1)$, whereas the actual observed maximum forward error is $\mathcal{O}(10^{-2})$. The same machinery is applied to FMA and mixed-precision FMA, where the deterministic and probabilistic bounds coincide when no precision change occurs, and where mixed-precision operation costs about $10^4$ times more error while roughly doubling arithmetic intensity.

Load-bearing premise

The probabilistic bounds all inherit Lemma 2.2's assumption that each rounding error is an independent random variable uniformly distributed between $-u$ and $u$, and the tensor-core analysis additionally assumes each short-precision product is stored exactly in the intermediate precision; if rounding in the hardware is deterministic or correlated, or if a product exceeds that intermediate precision, the stated probability guarantee can fail.

Editorial extensions

If this is right

  • For a single multiply-accumulate operation, FMA gives the tightest bounds, while mixed-precision FMA has about $10^4$ times larger error but nearly twice the arithmetic intensity.
  • For tensor-core GEMM with fp16 inputs and fp32 accumulation, the probabilistic bound is about ten times tighter than the deterministic bound when inputs are random.
  • Both kinds of tensor-core bound overestimate the true maximum forward error by roughly $10^5$ on the tested random data, so they are safe but loose.
  • Because the probabilistic bound scales with $\sqrt{n}$ instead of $n u/(1 - n u)$, the gap between probabilistic and deterministic estimates widens as the number of operations grows.
  • The bounds are formulated as backward-error bounds when no precision change occurs and as forward-error bounds for the full GEMM, so they can be compared with input uncertainty to judge reliability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the $\sqrt{n}$ scaling holds beyond random inputs, mixed-precision linear-algebra libraries could replace worst-case safety margins with probabilistic error budgets, letting larger problems run in fp8 or lower precision than deterministic analysis would allow.
  • The paper does not report the value of $\lambda$ used in the experiments, so the tenfold comparison cannot be reproduced from the text; reporting $\lambda$ and its sensitivity would make the bound usable and checkable.
  • The same machinery could be applied to stochastic rounding, where the independence assumption is closer to actual hardware behavior, potentially making the probabilistic guarantee nearly exact rather than an overestimate.
  • For structured or correlated data, such as low-rank matrices, the independence assumption is likely to fail; testing these inputs would show how far the guarantee reaches beyond the uniform random case.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The manuscript derives deterministic (DBEA) and probabilistic (VIBEA) backward/forward rounding error bounds for fused multiply-add, mixed-precision FMA, and NVIDIA Tensor Core matrix multiplication. The probabilistic analysis relies on Lemma 2.2, imported from the authors' prior work [23], and the paper claims in the abstract and in Section 4.2 that the VIBEA bounds for Tensor Core GEMM are nearly an order of magnitude tighter than the deterministic ones. The deterministic lemmas are straightforward applications of the standard gamma_n bounds, and the Tensor Core deterministic theorem follows the block-FMA analysis of Blanchard et al. The experimental section reports MAC and GEMM error distributions on an H100 GPU using cuBLAS.

Significance. If the central probabilistic claim were substantiated, the paper would be a useful reference for mixed-precision error analysis, especially because it provides a concrete Tensor Core experiment and a cuBLAS code snippet. The paper also correctly emphasizes that the assumption of errorless multiplication in Tensor Cores changes the deterministic bound from gamma_n to gamma_{n-1} relative to earlier work. However, the novel contribution is not self-contained as written: the key quantity gamma_tilde_n(lambda) is never defined, lambda is not reported, and Theorem 3.9 as printed reduces to the deterministic Theorem 3.7. The advertised order-of-magnitude improvement therefore cannot be checked from the text.

major comments (4)
  1. [Section 3.3.2, Theorem 3.9] The proof of Theorem 3.9 states that after relaxing the bounds one obtains |theta_tilde| <= gamma_tilde_{n-1}^{ubar} and |theta_tilde| <= gamma_tilde_q^{uFMA}, but the displayed bound in Theorem 3.9 uses the deterministic gamma_{n-1}^{ubar} and gamma_q^{uFMA} factors. As printed, Theorem 3.9 is identical to the deterministic Theorem 3.7 and cannot justify the claimed VIBEA tightening. If the tilde-gamma version was intended, then gamma_tilde_n(lambda) must be defined, since Lemma 2.2 defines only the probability pb and not gamma_tilde.
  2. [Section 3.3.2, Theorems 3.9 and 3.10] The stated probability expression 1 - m*t * sum_{i,j} (2 - pb(lambda, ubar, c1) - pb(lambda, uFMA, c2)) is not guaranteed to lie in [0,1]. For the experimental dimensions m=2^10, t=2^15, n=2^3, the prefactor m*t times the inner sum has magnitude on the order of 2*m*t*n ~ 5.5e8 unless pb is extremely close to 1. The manuscript must specify lambda and verify that the expression is positive; otherwise the bound is vacuous. This affects both Theorem 3.9 and Theorem 3.10.
  3. [Section 2.1, Lemma 2.2] The paper is not self-contained because gamma_tilde_n(lambda), used in Lemmas 3.3 and 3.6 and Theorems 3.9 and 3.10, is never defined. Lemma 2.2 defines only the success probability pb(λ, u, n), and the definition of gamma_tilde is left to the authors' prior arXiv preprint [23]. Additionally, the i.i.d. U[-u,u] assumption on the rounding errors is asserted without discussion of whether it is appropriate for Tensor Core accumulation, where the same input data may induce correlated rounding events.
  4. [Section 4.2 and Figure 6] The claimed order-of-magnitude improvement of VIBEA over DBEA is not reproducible. The text does not report the value of lambda used in the experiments, does not give the formula for gamma_tilde_n(lambda), and, as noted above, the printed Theorem 3.9 is identical to its deterministic counterpart. The numerical statement is also internally inconsistent: the text says the bounds are 'u_fp32 x 10^11 ~ O(10^2)' and 'O(10^1)', but u_fp32 x 10^11 is about 6e3, not O(10^2).
minor comments (7)
  1. [Theorems 3.7-3.10] The phrase 'computed computed approximation' appears repeatedly and should be corrected to 'computed approximation'.
  2. [Section 3.3.2, after Lemma 2.2] The line 'theta_tilde_q^{uFMA} <= |gamma_tilde_q|' should read '|theta_tilde_q^{uFMA}| <= gamma_tilde_q^{uFMA}', with absolute values on the random variable.
  3. [Section 3.3.2, probabilistic derivation] In the expression for Delta y_j, the symbols theta still appear without tildes even though the preceding line has converted them to theta_tilde; this makes the derivation hard to follow.
  4. [Lemma 3.1] The proof writes 'z_hat = fma(a,b,c)(1+delta)', which double-counts the rounding for a correctly rounded FMA; the final bound is correct when interpreted as fl(a*b+c) = (a*b+c)(1+delta), but the notation should be cleaned up.
  5. [Listing 1] The text refers to 'Algorithm 1' for the cuBLAS code snippet, but the snippet is actually Listing 1; the cross-reference should be fixed.
  6. [Section 4.2] The notation is inconsistent: Algorithm 1 defines A in R^{m x n} and B in R^{n x t}, while Section 4.2 defines A in R^{m x t} and B in R^{t x n}; this makes it difficult to map the experimental parameters to the theorem.
  7. [Conclusions] The phrase 'nearly O(104) times larger' should be written as 'O(10^4)' with proper superscript formatting.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the probabilistic tool is an imported lemma with independent assumptions and the experiments are external; the Theorem 3.9 gamma/tilde-gamma mismatch is a reproducibility defect, not a circular step.

full rationale

The derivation chain is not circular. The core probabilistic input, Lemma 2.2, is imported from the authors' prior work [23], but it is a standalone lemma with stated assumptions—'δ1, . . . , δn represent n i.i.d. random variables distributed uniformly as U[−u,u]'—that do not include any target bound of this paper, so under the review rules it counts as independent support rather than load-bearing self-citation. The deterministic bounds (Lemmas 3.1–3.6, Theorems 3.7–3.8) are derived in-paper from the IEEE rounding model (eq. (3)) and Higham's Lemma 2.1, and the Tensor Core deterministic analysis is a re-derivation of the cited framework [25] under the stated 'errorless multiplication' assumption. The numerical comparisons are external: fp64 is treated as the ground truth and cuBLAS on an NVIDIA H100 is measured, so there is no fitted parameter being relabeled as a prediction. The manuscript does contain a serious non-circular defect: in Section 3.3.2, after invoking Lemma 2.2 the proof writes '|Δyj| ≤ (γ̄u n−1 + γuFMA q + γ̄u n−1 ∗ γuFMA q)|yj|' and Theorem 3.9 states the same deterministic-gamma bound, while the accompanying text and Theorem 3.10's ζ use tilde-gamma quantities; γ̃_n(λ) itself is never defined and the λ used to produce Figure 6 is not reported. This makes the advertised nearly-order-of-magnitude VIBEA tightening non-reproducible from the printed equations, but it is an internal inconsistency and missing-definition problem, not a derivation that reduces by construction to its own input.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central bounds rest on the standard IEEE rounding model, the deterministic gamma_n bound, the authors' own VIBEA lemma, and the errorless-product assumption for Tensor cores. No new physical entities are introduced. The main missing quantity is gamma_tilde_n(lambda), which is never defined in the paper.

free parameters (1)
  • lambda (probabilistic bound parameter) = not stated in the paper
    All VIBEA bounds and probability expressions depend on lambda, but no value is reported in the experiments and no formula for gamma_tilde_n(lambda) is given, so the probabilistic bounds are uninterpretable as written.
assumptions (5)
  • domain assumption Rounding model fl(z) = z(1+delta) with |delta| <= u (Eq. 3)
    Standard IEEE rounding model used in all lemmas and theorems.
  • standard math Deterministic product bound gamma_n = n u / (1 - n u) for n u < 1 (Lemma 2.1)
    Standard backward error result from Higham; the condition n u < 1 is not checked in the Tensor core experiments.
  • domain assumption Rounding errors delta_i are i.i.d. U[-u,u] and gamma_tilde_n(lambda) bounds their product with probability p_b (Lemma 2.2)
    The VIBEA probabilistic model is imported from the authors' prior work [23] and is not derived in this paper.
  • domain assumption Tensor core multiplication a times b introduces no rounding error because the product fits in the intermediate precision
    Used throughout Section 3.3; valid for fp16 times fp16 into fp32 but restricts generality.
  • domain assumption Blocked GEMM Algorithm 1 accurately describes cuBLAS Tensor core behavior
    Theorems 3.7 through 3.10 rely on this decomposition and rounding pattern.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deterministic and Probabilistic Rounding Error Analysis for Mixed-Precision Arithmetic on Modern Computing Units." pith.science (2026). https://pith.science/paper/ZALDMPMA

@misc{pith2026241118747,
  author       = {Pith},
  title        = {Pith review of: Deterministic and Probabilistic Rounding Error Analysis for Mixed-Precision Arithmetic on Modern Computing Units},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZALDMPMA}},
  note         = {Machine review of arXiv:2411.18747}
}
read the original abstract

Modern computer architectures support low-precision arithmetic, which present opportunities for the adoption of mixed-precision algorithms to achieve high computational throughput and reduce energy consumption. As a growing number of scientific computations leverage specialized hardware accelerators, the risk of rounding errors increases, potentially compromising the reliability of models. This shift towards hardware-optimized, low-precision computations highlights the importance of rounding error analysis to ensure that performance gains do not come at the expense of accuracy, especially in high-stakes scientific applications. In this work, we conduct rounding error analysis on widely used operations such as fused multiply-add (FMA), mixed-precision FMA (MPFMA), and NVIDIA Tensor cores. We present a deterministic and probabilistic approach to quantifying the accumulated rounding errors. Numerical experiments are presented to perform the multiply and accumulate operation (MAC) and matrix-matrix multiplication using Tensor cores with random data. We show that probabilistic bounds produce tighter estimates by nearly an order of magnitude compared to deterministic ones for matrix-matrix multiplication.

Figures

Figures reproduced from arXiv: 2411.18747 by the authors.

Figure 1
Figure 1. Schematic of backward and forward errors. Solid line: exact; dashed line: computed. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Schematic of the mixed-precision FMA operation. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Schematic for Tensor core matrix multiply and accumulate. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: An illustration of the Tensor core kernel. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Empirical distribution for the forward error ( [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Empirical distribution for the forward error ( [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 26 canonical work pages

  1. [25]

    Blanchard, N

    P. Blanchard, N. J. Higham, F. Lopez, T. Mary, S. Pranesh, Mixed precision block fused multiply- add: Error analysis and application to gpu tensor cores, SIAM Journal on Scientific Computing 42 (3) (2020) C124–C141

    work page 2020

  2. [23]

    Bhola, K

    S. Bhola, K. Duraisamy, Variance-informed rounding uncertainty analysis for floating-point sta- tistical models, arXiv preprint arXiv:2404.12556 (2024)

    work page arXiv 2024

  3. [1]

    Gupta, A

    S. Gupta, A. Agrawal, K. Gopalakrishnan, P. Narayanan, Deep learning with limited numerical precision, in: International conference on machine learning, PMLR, 2015, pp. 1737–1746

    work page 2015

  4. [2]

    N. Wang, J. Choi, D. Brand, C.-Y. Chen, K. Gopalakrishnan, Training deep neural networks with 8-bit floating point numbers, Advances in neural information processing systems 31 (2018)

    work page 2018

  5. [3]

    Hubara, M

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, Y. Bengio, Quantized neural networks: Training neural networks with low precision weights and activations, Journal of Machine Learning Research 18 (187) (2018) 1–30

    work page 2018

  6. [4]

    E. A. Paxton, M. Chantry, M. Kl¨ ower, L. Saffin, T. Palmer, Climate modeling in low precision: Effects of both deterministic and stochastic rounding, Journal of Climate 35 (4) (2022) 1215–1229

    work page 2022

  7. [5]

    Kimpson, E

    T. Kimpson, E. A. Paxton, M. Chantry, T. Palmer, Climate-change modelling at reduced floating- point precision with stochastic rounding, Quarterly Journal of the Royal Meteorological Society 149 (752) (2023) 843–855

    work page 2023

  8. [6]

    Hatfield, M

    S. Hatfield, M. Chantry, P. D¨ uben, T. Palmer, Accelerating high-resolution weather models with deep-learning hardware, in: Proceedings of the platform for advanced scientific computing con- ference, 2019, pp. 1–11

    work page 2019

Show all 27 references
  1. [7]

    N. J. Higham, S. Pranesh, M. Zounon, Squeezing a matrix into half precision, with an application to solving linear systems, SIAM journal on scientific computing 41 (4) (2019) A2536–A2551

    2019

  2. [8]

    Haidar, H

    A. Haidar, H. Bayraktar, S. Tomov, J. Dongarra, N. J. Higham, Mixed-precision iterative refine- ment using tensor cores on gpus to accelerate solution of linear systems, Proceedings of the Royal Society A 476 (2243) (2020) 20200110

    2020

  3. [9]

    Haidar, P

    A. Haidar, P. Wu, S. Tomov, J. Dongarra, Investigating half precision arithmetic to accelerate dense linear system solvers, in: Proceedings of the 8th workshop on latest advances in scalable algorithms for large-scale systems, 2017, pp. 1–8

    2017

  4. [10]

    Abdelfattah, H

    A. Abdelfattah, H. Anzt, E. G. Boman, E. Carson, T. Cojean, J. Dongarra, A. Fox, M. Gates, N. J. Higham, X. S. Li, et al., A survey of numerical linear algebra methods utilizing mixed-precision arithmetic, The International Journal of High Performance Computing Applications 35...

    2021

  5. [11]

    Buttari, J

    A. Buttari, J. Dongarra, J. Langou, J. Langou, P. Luszczek, J. Kurzak, Mixed precision iterative refinement techniques for the solution of dense linear systems, The International Journal of High Performance Computing Applications 21 (4) (2007) 457–466

    2007

  6. [12]

    Y. Liu, X. Liu, E. Wu, Real-time 3d fluid simulation on gpu with complex obstacles, in: 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings., IEEE, 2004, pp. 247–256

    2004

  7. [13]

    P. R. Rinaldi, E. Dari, M. J. V´ enere, A. Clausse, A lattice-boltzmann solver for 3d fluid simulation on gpu, Simulation Modelling Practice and Theory 25 (2012) 163–171

    2012

  8. [14]

    M. Karp, F. Liu, R. Stanly, S. Rezaeiravesh, N. Jansson, P. Schlatter, S. Markidis, Uncertainty quantification of reduced-precision time series in turbulent channel flow, in: Proceedings of the SC’23 Workshops of The International Conference on High Performance Computing, Netw...

    2023

  9. [15]

    Lienhart, A

    G. Lienhart, A. Kugel, R. Manner, Using floating-point arithmetic on fpgas to accelerate scien- tific n-body simulations, in: Proceedings. 10th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, IEEE, 2002, pp. 182–191

    2002

  10. [16]

    N. J. Higham, Accuracy and stability of numerical algorithms, SIAM, 2002

    2002

  11. [17]

    Von Neumann, H

    J. Von Neumann, H. H. Goldstine, Numerical inverting of matrices of high order (1947)

    1947

  12. [18]

    T. E. Hull, J. R. Swenson, Tests of probabilistic models for propagation of roundoff errors, Com- munications of the ACM 9 (2) (1966) 108–113

    1966

  13. [19]

    Henrici, Test of probabilistic models for the propagation of roundoff errors, Communications of the ACM 9 (6) (1966) 409–410

    P. Henrici, Test of probabilistic models for the propagation of roundoff errors, Communications of the ACM 9 (6) (1966) 409–410

    1966

  14. [20]

    Tienari, A statistical model of roundoff error for varying length floating-point arithmetic, BIT Numerical Mathematics 10 (3) (1970) 355–365

    M. Tienari, A statistical model of roundoff error for varying length floating-point arithmetic, BIT Numerical Mathematics 10 (3) (1970) 355–365

    1970

  15. [21]

    I. C. Ipsen, H. Zhou, Probabilistic error analysis for inner products, SIAM journal on matrix analysis and applications 41 (4) (2020) 1726–1741

    2020

  16. [22]

    N. J. Higham, T. Mary, Sharper probabilistic backward error analysis for basic linear algebra kernels with random data, SIAM Journal on Scientific Computing 42 (5) (2020) A3427–A3446

    2020

  17. [24]

    IEEE, Ieee standard for floating-point arithmetic, ieee std 754-2019 (revision of ieee 754-2008), Institute of Electrical and Electronics Engineers New York, 2019

    2019

  18. [26]

    Dahlqvist, R

    F. Dahlqvist, R. Salvia, G. A. Constantinides, A probabilistic approach to floating-point arith- metic, in: 2019 53rd Asilomar Conference on Signals, Systems, and Computers, IEEE, 2019, pp. 596–602

    2019

  19. [27]

    Y. Fang, L. Chen, Probabilistic rounding error analysis from a statistical perspective, arXiv preprint arXiv:2405.07537 (2024). 15

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.