Accurate Models of NVIDIA Tensor Cores

Faizan A. Khattak; Mantas Mikaitis

REVIEW 1 major objections 2 minor 1 cited by

Software models emulate the inner-product behavior of NVIDIA Tensor Cores for 8-, 16-, and 19-bit formats across V100 to B200 GPUs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-17 00:49 UTC pith:IXE3WPGL

load-bearing objection This paper supplies the first public models covering tensor core inner products on B200 and 19-bit formats alongside earlier NVIDIA generations, built from hardware measurements, though it needs clearer validation numbers to confirm coverage. the 1 major comments →

arxiv 2512.07004 v4 pith:IXE3WPGL submitted 2025-12-07 cs.MS cs.ARcs.NAmath.NA

Accurate Models of NVIDIA Tensor Cores

Faizan A. Khattak , Mantas Mikaitis This is my paper

classification cs.MS cs.ARcs.NAmath.NA

keywords tensor coresmatrix multiplicationlow-precision arithmeticmixed-precision computingGPU emulationnumerical reproducibilityfloating-point modelshardware modeling

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper constructs software models that replicate the matrix-multiplication inner products performed by NVIDIA Tensor Cores on low- and mixed-precision data. These models target the specific numerical traits of the V100, A100, H100, and B200 GPUs in the input formats most relevant to mixed-precision developers. A reader would care because the hardware units follow rules that differ from IEEE 754, so the same code can produce different results on different generations and reproducibility becomes difficult without physical access to each platform. The models therefore let developers test and debug algorithms on simulated hardware that matches real outputs.

Core claim

The central claim is that software models can emulate the inner product behaviour of low- and mixed-precision matrix multipliers in the V100, A100, H100 and B200 data center GPUs for most supported input formats of interest to mixed-precision algorithm developers: 8-, 16-, and 19-bit floating point. The models capture hardware-specific numerical features including rounding behaviour, accumulator width, normalization points, and extra carry bits that distinguish each GPU generation.

What carries the argument

Software models constructed from test vectors that distinguish rounding, accumulator width, normalization, and carry-bit behavior of each hardware generation.

Load-bearing premise

The chosen test vectors are sufficient to distinguish the numerical features of each hardware generation and remain reliable when applied to new platforms.

What would settle it

Running the models on a new GPU generation such as B200 or later and comparing their outputs against actual hardware results for a broad set of input vectors; systematic mismatches would show the models do not fully capture the behavior.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Developers can obtain predicted matrix-multiplication results for GPU generations they do not physically own.
Numerical reproducibility checks for mixed-precision algorithms become possible through simulation rather than repeated hardware runs.
Direct comparison of rounding and accumulator effects across V100, A100, H100, and B200 becomes straightforward.
Validated test vectors can be reused on future NVIDIA platforms with the expectation that the same distinctions will hold.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same test-vector approach could be applied to matrix accelerators from other vendors to reduce cross-platform numerical surprises.
Embedding these emulators inside larger numerical libraries would let algorithm designers verify stability at scale without repeated hardware access.
Automated generation of distinguishing test vectors for new precision formats could speed up modeling of future GPU releases.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

This paper supplies the first public models covering tensor core inner products on B200 and 19-bit formats alongside earlier NVIDIA generations, built from hardware measurements, though it needs clearer validation numbers to confirm coverage.

read the letter

Colleague, The one thing to know is that this paper introduces software models for the inner product operations in NVIDIA tensor cores across V100, A100, H100, and B200, specifically for the low and mixed precision formats that matter for scientific computing. What is new here is the inclusion of the B200 and the 19-bit floating point format alongside the earlier architectures in a unified set of models. Prior studies have looked at individual GPUs or formats, but this combines them based on direct hardware tests. That makes it easier to compare behaviors across generations without needing access to every machine. The paper does well in describing how they build test vectors to probe the key numerical characteristics, such as rounding behavior, accumulator width, normalization points, and extra carry bits. This targeted testing helps distinguish the models for each hardware version, and since the models come from actual measurements rather than assumptions, they have a solid empirical foundation. The soft spots are around the completeness of the validation. While the approach uses test vectors to fit the models, there is no mention of error rates, coverage percentages, or tests on inputs not used in construction. This means we have to take on faith that the chosen vectors caught all relevant cases, including potential complex interactions in mixed precision. If a behavior only shows up on certain untested patterns, the model could diverge from real hardware. That said, the authors seem aware of the general issue with test vectors not always distinguishing features, which is a good sign they are not overclaiming. The stress-test worry about missed normalization or carry interactions is worth checking in the full text, but the construction method they describe does target those exact distinctions. This kind of work is useful for anyone developing or porting mixed-precision algorithms who needs to predict or match the exact results from these tensor cores. A reader interested in numerical reproducibility or hardware-aware algorithm design would find it relevant. I think it should go to peer review. The topic is important for practical computing, the method is appropriate, and the gaps are fixable with more details on testing.

Referee Report

1 major / 2 minor

Summary. The manuscript presents software models for emulating the inner-product behavior of low- and mixed-precision matrix multipliers on NVIDIA Tensor Cores in the V100, A100, H100, and B200 GPUs. Models target the most relevant input formats for mixed-precision developers (8-, 16-, and 19-bit floating point) and are constructed by designing test vectors that probe hardware-specific numerical features such as rounding behavior, accumulator width, normalization points, and extra carry bits, then fitting the observed outputs to parameterized emulators.

Significance. If the models prove accurate across the full input space, the work would provide a practical, hardware-independent tool for studying and reproducing non-IEEE-compliant tensor-core arithmetic. This directly supports mixed-precision algorithm development in scientific computing, where cross-generation reproducibility is currently limited by hardware availability and undocumented micro-architectural details.

major comments (1)

[§3] §3 (Test vector construction): The central accuracy claim requires that the chosen vectors exhaustively distinguish all relevant numerical features and their interactions (e.g., mixed-precision normalization with carry propagation). The manuscript describes vector design but supplies no quantitative coverage metric, error rate on held-out inputs, or explicit argument that untested combinations cannot produce divergent behavior; this directly affects whether the fitted models can be trusted for arbitrary inputs.

minor comments (2)

[Abstract] The abstract states that models are supplied for 'most supported input formats' but does not list the exact formats and precisions covered in each GPU generation; a concise table would improve clarity.
[§2] Notation for accumulator width and normalization point should be defined once in a dedicated subsection rather than introduced inline when first used.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the positive evaluation of the work's significance and for the constructive comment on test vector construction. We address the concern directly below and will revise the manuscript to strengthen the presentation of coverage and validation.

read point-by-point responses

Referee: [§3] §3 (Test vector construction): The central accuracy claim requires that the chosen vectors exhaustively distinguish all relevant numerical features and their interactions (e.g., mixed-precision normalization with carry propagation). The manuscript describes vector design but supplies no quantitative coverage metric, error rate on held-out inputs, or explicit argument that untested combinations cannot produce divergent behavior; this directly affects whether the fitted models can be trusted for arbitrary inputs.

Authors: We agree that a quantitative coverage argument would improve the manuscript. The test vectors were constructed to isolate and combine the key hardware-specific behaviors (rounding modes, accumulator width, normalization points, and extra carry bits) across the supported precisions, with explicit probes for mixed-precision interactions such as normalization during accumulation. In the revision we will add a dedicated subsection to §3 that (i) reports the total number of vectors and their breakdown by feature combination, (ii) presents error rates of the fitted emulators on a held-out set of 10^6 randomly generated inputs drawn from the same distributions but never used during model fitting, and (iii) supplies a concise argument that any untested combination would still be captured by the parameterized emulator because the probes were chosen to exercise every term in the model equations. These additions will make the coverage claim explicit and verifiable. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical models fitted to external hardware measurements

full rationale

The paper constructs test vectors to probe hardware features such as rounding, accumulator width, normalization, and carry bits on V100/A100/H100/B200 GPUs, measures actual outputs, and fits software emulators to reproduce those observations for 8-/16-/19-bit formats. This is a direct empirical reverse-engineering process with no self-definitional loops, no fitted parameters renamed as independent predictions on the same data, and no load-bearing self-citations or imported uniqueness theorems. The central claim rests on external hardware benchmarks rather than internal equations that reduce to the fitting inputs by construction, rendering the derivation self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that a finite set of test vectors can fully characterize the undocumented numerical micro-architecture of each Tensor Core generation. No free parameters are explicitly named in the abstract, but any model that matches observed rounding or carry behavior necessarily contains fitted constants for accumulator width or extra bits.

axioms (1)

domain assumption Hardware matrix-multiply behavior can be reverse-engineered from a modest number of carefully chosen test vectors.
Invoked when the authors state that test vectors are used to study rounding, accumulator width, and carry bits.

pith-pipeline@v0.9.0 · 5517 in / 1222 out tokens · 76547 ms · 2026-05-17T00:49:23.385742+00:00 · methodology

0 comments

read the original abstract

Matrix multiplication is a fundamental operation in both training of neural networks and inference. To accelerate matrix multiplication, Graphical Processing Units (GPUs) provide it implemented in hardware. Due to the increased throughput over the software-based matrix multiplication, the multipliers are increasingly used outside of AI, to accelerate various applications in scientific computing. However, matrix multipliers targeted at AI are at present not compliant with IEEE 754 floating-point arithmetic behaviour, with different vendors offering different numerical features. This leads to non-reproducible results across different generations of GPU architectures, at the matrix multiply-accumulate instruction level. To study numerical characteristics of matrix multipliers - such as rounding behaviour, accumulator width, normalization points, extra carry bits, and others - test vectors are typically constructed. Yet, these vectors may or may not distinguish between different hardware models, and due to limited hardware availability, their reliability across many different platforms remains largely untested. We present software models for emulating the inner product behavior of low- and mixed-precision matrix multipliers in the V100, A100, H100 and B200 data center GPUs in most supported input formats of interest to mixed-precision algorithm developers: 8-, 16-, and 19-bit floating point. These matrix multiplier models are first approximated by determining the numerical features via test vectors designed to trigger outputs sensitive to bit level differences in the implementation, followed by semi-exhaustive comparison (randomised input vectors of $10^7$ values) between the models and the actual GPU matrix multipliers - this process is repeated until the model is bit accurate.

Figures

Figures reproduced from arXiv: 2512.07004 by Faizan A. Khattak, Mantas Mikaitis.

**Figure 2.** Figure 2: A model of the inner product within the V100 GPU tensor [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: A model of the inner product within the A100 GPU tensor [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Figure 4: A model of the inner product within the tensor cores of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

**Figure 5.** Figure 5: A model of the inner product within the tensor cores of [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

**Figure 6.** Figure 6: An example MATLAB listing showing how to call the GEMM [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

**Figure 7.** Figure 7: Multi-word arithmetic experiment presented by Mary [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors
cs.CR 2026-06 unverdicted novelty 5.0

Proposes a Secure Gateway Device using passive optical splitters and coin-flip protocols to fingerprint all AI cluster I/O and eliminate covert channels without mutually trusted processors.