{"id":"4307fd1a-4df0-41a1-b6ef-430d7bc4316c","arxiv_id":"2411.18747","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Rounding error bounds for FMA, mixed-precision FMA and Tensor cores are derived deterministically and probabilistically, with probabilistic bounds about an order of magnitude tighter in a Tensor core GEMM test.","lead":"This paper derives deterministic and probabilistic rounding error bounds for fused multiply-add, mixed-precision FMA, and NVIDIA Tensor core matrix multiplication. It reports that the probabilistic bounds are roughly ten times tighter than deterministic bounds in a matrix multiplication experiment, though both are still far looser than the actual observed errors.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.9's printed probabilistic Tensor Core bound uses deterministic gamma factors, and gamma_tilde_n(lambda) is never defined; the claimed order-of-magnitude VIBEA tightening is therefore not supported by the text.","rationale":"The deterministic FMA, MPFMA, and Tensor Core derivations in Sections 3.1 and 3.2 appear standard, and I see no need to object to them. The serious problem is concentrated entirely in the probabilistic Tensor Core result that carries the abstract's headline claim. The reader's weakest assumption identified the undefined gamma_tilde and lambda as the core reproducibility gap; my stress-test finds an even more direct textual inconsistency: Theorem 3.9, the main probability theorem used for Figure 6, is printed with deterministic gamma symbols rather than the tilde-gamma quantities introduced in the preceding proof. If read literally, the VIBEA bound equals the DBEA bound and no order-of-magnitude improvement follows. If read as a typo, the missing definition of gamma_tilde and the missing lambda still prevent any numerical check, and the union-bound probability formula appears vacuous at the reported matrix sizes for reasonable lambda values. Either way, the central claim is not established by the text, so the reader's REJECT verdict stands. I do not treat the missing definitions as evidence of bad faith; they are concrete omissions in an otherwise coherent deterministic analysis.","tokens_in":13669,"tokens_out":9359,"duration_ms":86332,"concrete_test":"Re-derive Theorem 3.9 from Section 3.3.2 retaining tilde-gamma factors throughout the final relaxation, then obtain the exact definition of gamma_tilde_n(lambda) from arXiv:2404.12556 and the lambda used in Figure 6. Evaluate the resulting probability expression for m=2^10, t=2^15, n=2^3 and u=u_fp32 over a range of lambda values. If Theorem 3.9 remains a deterministic-gamma bound, or if gamma_tilde or lambda is unavailable, or if no lambda yields both a bound at least 10x tighter than DBEA and a nonnegative stated probability, the headline claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Figure 6 claim that VIBEA bounds are tighter than DBEA by nearly an order of magnitude for Tensor Core matrix-matrix multiplication. The load-bearing step is Theorem 3.9. In Section 3.3.2, the proof correctly starts with |tilde_theta| <= tilde_gamma from Lemma 2.2, but the displayed bound immediately afterward is |Delta y_j| <= (gamma_{n-1}^{ubar} + gamma_q^{uFMA} + gamma_{n-1}^{ubar} * gamma_q^{uFMA}) |y_j|, and Theorem 3.9 states exactly this deterministic-gamma expression. As printed, Theorem 3.9 is identical to the deterministic Theorem 3.7 and cannot produce the claimed probabilistic tightness. If the gammas were intended to be tilde-gammas, the paper never defines gamma_tilde_n(lambda) and never reports the lambda used in the experiments; Lemma 2.2 defines only the probability pb and not gamma_tilde. Furthermore, for the experimental dimensions m=2^10, t=2^15, n=2^3, the union-bound probability in Theorem 3.9 contains a factor m*t times a sum over n block terms; unless lambda is chosen very large, the stated probability is negative. Thus the central probabilistic claim is not reproducible from the manuscript.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript derives deterministic (DBEA) and probabilistic (VIBEA) backward/forward rounding error bounds for fused multiply-add, mixed-precision FMA, and NVIDIA Tensor Core matrix multiplication. The probabilistic analysis relies on Lemma 2.2, imported from the authors' prior work [23], and the paper claims in the abstract and in Section 4.2 that the VIBEA bounds for Tensor Core GEMM are nearly an order of magnitude tighter than the deterministic ones. The deterministic lemmas are straightforward applications of the standard gamma_n bounds, and the Tensor Core deterministic theorem follows the block-FMA analysis of Blanchard et al. The experimental section reports MAC and GEMM error distributions on an H100 GPU using cuBLAS.","tokens_in":13935,"tokens_out":7604,"duration_ms":65519,"significance":"If the central probabilistic claim were substantiated, the paper would be a useful reference for mixed-precision error analysis, especially because it provides a concrete Tensor Core experiment and a cuBLAS code snippet. The paper also correctly emphasizes that the assumption of errorless multiplication in Tensor Cores changes the deterministic bound from gamma_n to gamma_{n-1} relative to earlier work. However, the novel contribution is not self-contained as written: the key quantity gamma_tilde_n(lambda) is never defined, lambda is not reported, and Theorem 3.9 as printed reduces to the deterministic Theorem 3.7. The advertised order-of-magnitude improvement therefore cannot be checked from the text.","major_comments":[{"comment":"The proof of Theorem 3.9 states that after relaxing the bounds one obtains |theta_tilde| <= gamma_tilde_{n-1}^{ubar} and |theta_tilde| <= gamma_tilde_q^{uFMA}, but the displayed bound in Theorem 3.9 uses the deterministic gamma_{n-1}^{ubar} and gamma_q^{uFMA} factors. As printed, Theorem 3.9 is identical to the deterministic Theorem 3.7 and cannot justify the claimed VIBEA tightening. If the tilde-gamma version was intended, then gamma_tilde_n(lambda) must be defined, since Lemma 2.2 defines only the probability pb and not gamma_tilde.","section":"Section 3.3.2, Theorem 3.9"},{"comment":"The stated probability expression 1 - m*t * sum_{i,j} (2 - pb(lambda, ubar, c1) - pb(lambda, uFMA, c2)) is not guaranteed to lie in [0,1]. For the experimental dimensions m=2^10, t=2^15, n=2^3, the prefactor m*t times the inner sum has magnitude on the order of 2*m*t*n ~ 5.5e8 unless pb is extremely close to 1. The manuscript must specify lambda and verify that the expression is positive; otherwise the bound is vacuous. This affects both Theorem 3.9 and Theorem 3.10.","section":"Section 3.3.2, Theorems 3.9 and 3.10"},{"comment":"The paper is not self-contained because gamma_tilde_n(lambda), used in Lemmas 3.3 and 3.6 and Theorems 3.9 and 3.10, is never defined. Lemma 2.2 defines only the success probability pb(λ, u, n), and the definition of gamma_tilde is left to the authors' prior arXiv preprint [23]. Additionally, the i.i.d. U[-u,u] assumption on the rounding errors is asserted without discussion of whether it is appropriate for Tensor Core accumulation, where the same input data may induce correlated rounding events.","section":"Section 2.1, Lemma 2.2"},{"comment":"The claimed order-of-magnitude improvement of VIBEA over DBEA is not reproducible. The text does not report the value of lambda used in the experiments, does not give the formula for gamma_tilde_n(lambda), and, as noted above, the printed Theorem 3.9 is identical to its deterministic counterpart. The numerical statement is also internally inconsistent: the text says the bounds are 'u_fp32 x 10^11 ~ O(10^2)' and 'O(10^1)', but u_fp32 x 10^11 is about 6e3, not O(10^2).","section":"Section 4.2 and Figure 6"}],"minor_comments":[{"comment":"The phrase 'computed computed approximation' appears repeatedly and should be corrected to 'computed approximation'.","section":"Theorems 3.7-3.10"},{"comment":"The line 'theta_tilde_q^{uFMA} <= |gamma_tilde_q|' should read '|theta_tilde_q^{uFMA}| <= gamma_tilde_q^{uFMA}', with absolute values on the random variable.","section":"Section 3.3.2, after Lemma 2.2"},{"comment":"In the expression for Delta y_j, the symbols theta still appear without tildes even though the preceding line has converted them to theta_tilde; this makes the derivation hard to follow.","section":"Section 3.3.2, probabilistic derivation"},{"comment":"The proof writes 'z_hat = fma(a,b,c)(1+delta)', which double-counts the rounding for a correctly rounded FMA; the final bound is correct when interpreted as fl(a*b+c) = (a*b+c)(1+delta), but the notation should be cleaned up.","section":"Lemma 3.1"},{"comment":"The text refers to 'Algorithm 1' for the cuBLAS code snippet, but the snippet is actually Listing 1; the cross-reference should be fixed.","section":"Listing 1"},{"comment":"The notation is inconsistent: Algorithm 1 defines A in R^{m x n} and B in R^{n x t}, while Section 4.2 defines A in R^{m x t} and B in R^{t x n}; this makes it difficult to map the experimental parameters to the theorem.","section":"Section 4.2"},{"comment":"The phrase 'nearly O(104) times larger' should be written as 'O(10^4)' with proper superscript formatting.","section":"Conclusions"}],"recommendation":"major_revision","confidential_remarks":"The paper's central probabilistic claim depends on the authors' own unpublished preprint [23], and the main theorems are not self-contained. The editor may wish to verify that [23] is available and, ideally, peer-reviewed before evaluating the contribution. The deterministic part is sound but mostly standard."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take the deterministic analysis as competent and the probabilistic conclusion as unsupported. The FMA/MPFMA lemmas are standard algebra, and the Tensor Core derivation under an errorless multiply correctly recovers a factor of n-1 compared to reference [25]. The experiment is straightforward and honestly notes that both bounds overestimate the true error by about 10^5.\n\nThe soft spots are load-bearing. The abstract claims VIBEA bounds are tighter by an order of magnitude for matrix-matrix multiplication, but Theorem 3.9, the stated probabilistic bound, is exactly the deterministic bound from Theorem 3.7: the proof relaxes the tilde-theta bounds to tilde-gamma but then writes plain gammas in the final inequality. So as printed, the theorem gives no probabilistic tightening. The same section never defines gamma_tilde_n(lambda) or reports lambda, so even if the gammas were intended to be tildes, the bound is not checkable. The stated probability expression has an m*t factor; for the experimental sizes m=2^10, t=2^15, n=2^3, the union-bound term exceeds 1, making the probability negative for any reasonable lambda. Theorem 3.10 compounds this by invoking lemma 2.1 while using tilde-gammas in the bound. These are not cosmetic issues; they invalidate the main claim.\n\nThe reader's moderate-confidence reject is justified; the stress-test catches the crux. I would not cite this result until the probabilistic proof is corrected and the missing definitions are supplied. For peer review, I would desk reject the current version, but a careful revision that fixes the gamma/tilde-gamma slip, defines gamma_tilde_n, reports lambda, and rescales the union bound could make the paper worth a second look. The deterministic material alone is not novel or strong enough to warrant review.","headline":"The deterministic parts are fine, but the paper's headline probabilistic Tensor Core bound is a deterministic bound in disguise, with an undefined gamma_tilde and a probability that can go negative.","tokens_in":14461,"tokens_out":4442,"would_cite":false,"duration_ms":39033,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65G50"],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating rounding errors as random rather than worst-case cuts accumulated error bounds for tensor-core matrix multiplication by roughly a factor of ten.","keywords":["rounding error analysis","mixed-precision arithmetic","probabilistic error bounds","fused multiply-add","tensor cores","floating-point arithmetic","matrix multiplication","variance-informed bounds"],"falsifier":"Run the same fp16 tensor-core GEMM with inputs whose products do not fit in the intermediate precision, repeat the experiment many times, and tally how often the true forward error exceeds the Theorem 3.9 bound; an exceedance rate larger than the theorem's claimed probability would refute the independence and exact-product premises.","tokens_in":13470,"feed_emoji":"🔢","tokens_out":9718,"duration_ms":160769,"temperature":0.7,"pith_summary":"This paper develops rounding-error bounds for three building blocks of numerical computing: fused multiply-add (FMA), mixed-precision FMA, and matrix-matrix multiplication on tensor cores. It argues that treating each rounding error as an independent random variable, instead of assuming the worst case, yields much tighter error estimates while keeping a probability guarantee. The headline numerical result is a nearly tenfold tightening of the accumulated-error bound for tensor-core GEMM compared with deterministic analysis. This matters because low-precision hardware is increasingly used for large scientific and machine-learning computations, and tighter bounds let users choose precision levels with more confidence.","feed_headline":"Probabilistic rounding bounds beat worst-case by ~10x on tensor cores","feed_subtitle":"Treating rounding errors as random tightens tensor-core GEMM error bounds by an order of magnitude.","key_machinery":"The carrying object is the VIBEA product bound, Lemma 2.2, which is imported from the authors' companion work. For $n$ independent rounding errors drawn uniformly from $[-u, u]$, it replaces the deterministic product estimate $1 + \\theta$ with $|\\theta| \\le \\gamma_n = n u/(1 - n u)$ by a random product $1 + \\tilde{\\theta}$ whose magnitude is bounded by $\\tilde{\\gamma}_n(\\lambda)$ with probability at least $p_b(\\lambda, u, n)$, a tail bound that involves the variance of the logarithm terms. Because $\\tilde{\\gamma}_n(\\lambda)$ grows like $\\sqrt{n}$ rather than linearly in $n$, the probabilistic bound remains much smaller for large $n$. The tensor-core analysis additionally relies on the assumption that each short-precision product is represented exactly in the intermediate precision, which removes the multiplication rounding error from the count.","core_discovery":"On the paper's own terms, the central claim is that variance-informed probabilistic backward error analysis (VIBEA) produces a rounding-error bound for tensor-core matrix multiplication that is roughly an order of magnitude smaller than the deterministic bound, while still carrying an explicit probability of success. In the empirical setup with fp16 inputs, fp32 accumulation, and matrices of size $m=2^{10}$, $t=2^{15}$, $n=2^{3}$, the deterministic bound is around $\\mathcal{O}(10^2)$ and the probabilistic bound around $\\mathcal{O}(10^1)$, whereas the actual observed maximum forward error is $\\mathcal{O}(10^{-2})$. The same machinery is applied to FMA and mixed-precision FMA, where the deterministic and probabilistic bounds coincide when no precision change occurs, and where mixed-precision operation costs about $10^4$ times more error while roughly doubling arithmetic intensity.","pith_inferences":["If the $\\sqrt{n}$ scaling holds beyond random inputs, mixed-precision linear-algebra libraries could replace worst-case safety margins with probabilistic error budgets, letting larger problems run in fp8 or lower precision than deterministic analysis would allow.","The paper does not report the value of $\\lambda$ used in the experiments, so the tenfold comparison cannot be reproduced from the text; reporting $\\lambda$ and its sensitivity would make the bound usable and checkable.","The same machinery could be applied to stochastic rounding, where the independence assumption is closer to actual hardware behavior, potentially making the probabilistic guarantee nearly exact rather than an overestimate.","For structured or correlated data, such as low-rank matrices, the independence assumption is likely to fail; testing these inputs would show how far the guarantee reaches beyond the uniform random case."],"forward_implications":["For a single multiply-accumulate operation, FMA gives the tightest bounds, while mixed-precision FMA has about $10^4$ times larger error but nearly twice the arithmetic intensity.","For tensor-core GEMM with fp16 inputs and fp32 accumulation, the probabilistic bound is about ten times tighter than the deterministic bound when inputs are random.","Both kinds of tensor-core bound overestimate the true maximum forward error by roughly $10^5$ on the tested random data, so they are safe but loose.","Because the probabilistic bound scales with $\\sqrt{n}$ instead of $n u/(1 - n u)$, the gap between probabilistic and deterministic estimates widens as the number of operations grows.","The bounds are formulated as backward-error bounds when no precision change occurs and as forward-error bounds for the full GEMM, so they can be compared with input uncertainty to judge reliability."],"supporting_citations":[{"why":"Supplies the deterministic product bound $\\gamma_n = n u/(1 - n u)$ used in every deterministic lemma and theorem.","marker":"[16]"},{"why":"Provides the VIBEA Lemma 2.2 with the probability guarantee $p_b(\\lambda, u, n)$ that all probabilistic bounds inherit.","marker":"[23]"},{"why":"Establishes the deterministic tensor-core error analysis framework that the paper specialises to the exact-product case and extends probabilistically.","marker":"[25]"},{"why":"Earlier probabilistic backward error results for random data that motivate the $\\sqrt{n}$-scaled replacement for $\\gamma_n$.","marker":"[22]"},{"why":"Defines the floating-point rounding model $\\mathrm{fl}(z)=z(1+\\delta)^\\rho$ with $|\\delta| \\le u$ on which all the proofs rest.","marker":"[24]"}],"fun_headline_variants":["Tighter error bounds: probabilistic beats deterministic by 10x","Probability tightens rounding bounds 10x on tensor cores","Rounding error bounds: probabilistic wins by an order of magnitude","Probabilistic analysis cuts tensor-core error bounds 10x","Worst-case bounds looser than probabilistic by 10x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The probabilistic bounds all inherit Lemma 2.2's assumption that each rounding error is an independent random variable uniformly distributed between $-u$ and $u$, and the tensor-core analysis additionally assumes each short-precision product is stored exactly in the intermediate precision; if rounding in the hardware is deterministic or correlated, or if a product exceeds that intermediate precision, the stated probability guarantee can fail.","fun_headline_variants_meta":{"raw":{"variants":["Tighter error bounds: probabilistic beats deterministic by 10x","Probability tightens rounding bounds 10x on tensor cores","Rounding error bounds: probabilistic wins by an order of magnitude","Probabilistic analysis cuts tensor-core error bounds 10x","Worst-case bounds looser than probabilistic by 10x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1445,"prompt_tokens":904,"completion_tokens":541,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":457}},"tokens_in":520,"tokens_out":541,"duration_ms":5572,"temperature":1.0,"reasoning_tokens":457,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:56:28.034302+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same fp16 tensor-core GEMM with inputs whose products do not fit in the intermediate precision, repeat the experiment many times, and tally how often the true forward error exceeds the Theorem 3.9 bound; an exceedance rate larger than the theorem's claimed probability would refute the independence and exact-product premises.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the deterministic product bound $\\gamma_n = n u/(1 - n u)$ used in every deterministic lemma and theorem."},{"cited_title":"Bhola, K","cited_arxiv_id":null,"evidence_quote":"Provides the VIBEA Lemma 2.2 with the probability guarantee $p_b(\\lambda, u, n)$ that all probabilistic bounds inherit."},{"cited_title":"Blanchard, N","cited_arxiv_id":null,"evidence_quote":"Establishes the deterministic tensor-core error analysis framework that the paper specialises to the exact-product case and extends probabilistically."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier probabilistic backward error results for random data that motivate the $\\sqrt{n}$-scaled replacement for $\\gamma_n$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the floating-point rounding model $\\mathrm{fl}(z)=z(1+\\delta)^\\rho$ with $|\\delta| \\le u$ on which all the proofs rest."}],"review_version":1}