pith. sign in

arxiv: 2606.00279 · v2 · pith:Y3RXRW6Nnew · submitted 2026-05-29 · 💻 cs.CR · cs.LG

Bit-Exact AI Inference Verification Without Performance Tradeoffs

Naci Cankaya This is my paper

Pith reviewed 2026-06-28 21:44 UTC · model grok-4.3

classification 💻 cs.CR cs.LG
keywords AI inference verificationbitwise precisionGPU emulationLLM auditingfloating-point determinismrounding error signaturesadversarial verificationsoftware-only re-computation
0
0 comments X

The pith

Bit-exact LLM inference verification works across different NVIDIA GPUs via software emulation alone.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that modern inference engines produce deterministic outputs that can be re-computed exactly on different hardware. This holds when the required computation information is available and atomic functions are not called in the backend. A software-only emulation achieves bitwise-precise matches across multiple GPU variants without forcing determinism flags that reduce performance. Accumulated rounding errors then function as a signature of the specific software and hardware setup. The approach removes a barrier to verifying AI workloads against covert adversaries who might otherwise exploit non-determinism for hidden modifications or steganography.

Core claim

We demonstrate that such bitwise-precise re-computation does not require access to identical hardware, via a software-only emulation of LLM inference across multiple NVIDIA GPU variants. Thus, accumulated rounding errors can be an auditable signature of the software and hardware setup used for inference, instead of a constraint on verifiability.

What carries the argument

Software-only emulation of inference engines that replicates exact computation sequences for bitwise matching across GPU variants.

If this is right

  • Rounding errors become an auditable signature of the exact inference setup rather than an obstacle to verification.
  • Verification of AI claims proceeds without setting performance-compromising determinism flags.
  • Covert adversaries lose degrees of freedom to hide modifications or perform unreported batch computations.
  • Approximate output matching is no longer required for credible auditing of monitored AI workloads.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same emulation principle could support detection of steganographic content by checking for exact output divergence.
  • Auditors might require disclosure of the re-computation metadata as a standard compliance step.
  • The method points toward routine cross-hardware consistency checks becoming feasible for governance of deployed models.

Load-bearing premise

The right information must be available for re-computation and no atomic functions are called in the backend of the inference engines.

What would settle it

An experiment showing that the software emulation produces different bit patterns from the original run on a different GPU variant, even when the required information is supplied and atomic functions are avoided.

Figures

Figures reproduced from arXiv: 2606.00279 by Naci Cankaya.

Figure 1
Figure 1. Figure 1: Four summation topologies that compute c + P i pi. All produce the same result in exact arithmetic, but not in general under floating-point rounding. Adapted from (Xie et al., 2025). attention blocks requires modeling special function units (SFUs) for exponentials, reciprocals, and square roots. Soft￾ware is also a source of discrepancy: trigonometric functions in RoPE, for example, depend on the specific … view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of top-5 logprobs (average across measure￾ments at the last three token positions of the first batch element) across different batch sizes for decode inference in vLLM. We see that some changes in batch size can leave batch element 0’s outputs unchanged. Such ”equivalence classes” were consistent across prompts for any given fixed sequence length, but they be￾came rarer with increasing sequence … view at source ↗
read the original abstract

Verifying claims about AI workloads is a prerequisite for credible AI governance of covert adversaries (who comply with monitoring only when detection likelihood is high), yet the apparent non-determinism of GPU floating-point arithmetic forces auditors to accept approximate output matches. Covert adversaries can exploit unverifiable degrees of freedom in monitored computation. Attack vectors include steganography, unreported modification of inference software, and covert computation via unreported batch elements. Empirically, we analyze how modern inference engines (vLLM, HF transformers) produce deterministic but non-invariant outputs, without needing to set performance-compromising determinism flags, if the right information is available for re-computation and no atomic functions are called in the backend. We demonstrate that such bitwise-precise re-computation does not require access to identical hardware, via a software-only emulation of LLM inference across multiple NVIDIA GPU variants. Thus, accumulated rounding errors can be an auditable signature of the software and hardware setup used for inference, instead of a constraint on verifiability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript claims that modern LLM inference engines (vLLM, Hugging Face Transformers) produce deterministic but hardware-variant outputs without performance-compromising determinism flags, provided sufficient re-computation information is available and no atomic functions are called in the backend. It demonstrates that bitwise-precise re-computation across NVIDIA GPU variants is achievable via software-only emulation, allowing accumulated rounding errors to serve as an auditable signature of the specific software/hardware setup rather than a barrier to verification. This is motivated by needs in AI governance against covert adversaries exploiting unverifiable degrees of freedom such as steganography or unreported modifications.

Significance. If the empirical demonstration and its premises hold, the work would be significant for AI security and verifiable computation, offering a practical route to exact, hardware-agnostic auditing of inference without performance tradeoffs. The software-only emulation approach and reframing of floating-point non-invariance as a signature are notable strengths that could support credible monitoring of covertly non-compliant AI systems.

major comments (1)
  1. [Abstract] Abstract: The central claim that bitwise-precise re-computation and cross-hardware determinism are possible is conditioned on the premise that 'no atomic functions are called in the backend of the inference engines.' No code inspection, kernel analysis, or empirical verification is supplied to establish the absence of atomic operations (e.g., atomicAdd) from critical paths in vLLM or HF transformers. This assumption is load-bearing; its violation would render the determinism, emulation, and auditable-signature arguments unsupported.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their careful reading of the manuscript and for highlighting this important point about the load-bearing assumption in our claims. We address the comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim that bitwise-precise re-computation and cross-hardware determinism are possible is conditioned on the premise that 'no atomic functions are called in the backend of the inference engines.' No code inspection, kernel analysis, or empirical verification is supplied to establish the absence of atomic operations (e.g., atomicAdd) from critical paths in vLLM or HF transformers. This assumption is load-bearing; its violation would render the determinism, emulation, and auditable-signature arguments unsupported.

    Authors: We agree that the absence of atomic operations in critical paths is a load-bearing assumption for the determinism and emulation results. While our empirical results demonstrate consistent bitwise outputs (which would be disrupted by non-deterministic atomic usage in reductions or accumulations), this does not constitute direct verification. In the revised manuscript we will add an explicit code inspection and kernel analysis subsection documenting that atomic functions (e.g., atomicAdd) are not invoked in the matrix-multiplication, reduction, and normalization paths of the vLLM and Hugging Face Transformers backends used in our experiments. This will be supported by references to the relevant CUDA kernel sources and call graphs. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical demonstration independent of self-referential inputs or definitions.

full rationale

The paper presents an empirical analysis of inference engine behavior (vLLM, HF transformers) and a software-only emulation demonstration across GPU variants. The central claim is conditioned on premises about re-computation information availability and absence of atomic functions, but these are stated as empirical conditions rather than derived via equations, fitted parameters renamed as predictions, or self-citation chains. No load-bearing steps reduce to self-definitional constructs, fitted inputs called predictions, or ansatzes smuggled via citation. The work is self-contained against external benchmarks (observed engine outputs), with no evidence of the enumerated circularity patterns. This is the expected outcome for an observation-based paper.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests primarily on a domain assumption about the behavior of specific inference engines under particular conditions; no free parameters or invented entities are introduced in the abstract.

axioms (1)
  • domain assumption Modern inference engines produce deterministic but non-invariant outputs when the right information is available and no atomic functions are called.
    This is presented as an empirical observation that enables the re-computation approach.

pith-pipeline@v0.9.1-grok · 5698 in / 1261 out tokens · 24736 ms · 2026-06-28T21:44:33.621525+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 5 canonical work pages · 2 internal anchors

  1. [1]

    Journal of Cryptology , volume=

    Security Against Covert Adversaries: Efficient Protocols for Realistic Adversaries , author=. Journal of Cryptology , volume=

  2. [2]

    Verifying International Agreements on

    Baker, Mauricio and Kulp, Gabriel and Marks, Oliver and Brundage, Miles and Heim, Lennart , journal=. Verifying International Agreements on

  3. [3]

    and Soder, Lisa and Wei, Kevin , journal=

    Reuel, Anka and Bucknall, Ben and Casper, Stephen and Hadfield, Gillian K. and Soder, Lisa and Wei, Kevin , journal=. Open Problems in Technical

  4. [4]

    Harack, Ben and Trager, Robert F. and Reuel, Anka and Manheim, David and Brundage, Miles and Aarne, Onni and Scher, Aaron and Pan, Yanliang and Xiao, Jenny and Loke, Kristy and Adan, Sumaya Nur and Bas, Guillem and Caputo, Nicholas A. and Morse, Julia C. and Ahuja, Janvi and Duan, Isabella and Egan, Janet and Bucknall, Ben and Rosen, Brianna and Araujo, R...

  5. [5]

    Mechanisms to Verify International Agreements About

    Scher, Aaron and Thiergart, Lisa , institution=. Mechanisms to Verify International Agreements About. 2024 , note=

    2024

  6. [6]

    and Reed, Tom and Miller, Jack William and Barnett, Peter , journal=

    Wasil, Akash R. and Reed, Tom and Miller, Jack William and Barnett, Peter , journal=. Verification Methods for International

  7. [7]

    and Mikaitis, Mantas , journal=

    Khattak, Faizan A. and Mikaitis, Mantas , journal=. Accurate Models of

  8. [8]

    Xie, Peichen and Wang, Yang and Yang, Fan and Yang, Mao , journal=

  9. [9]

    Bit-Accurate Modeling of

    Xie, Peichen and Xu, Shuotao and Wang, Yang and Yang, Fan and Yang, Mao , journal=. Bit-Accurate Modeling of

  10. [10]

    Li, Xinyi and Li, Ang and Fang, Bo and Swirydowicz, Kevin and Laguna, Ignacio and Gopalakrishnan, Ganesh , booktitle=

  11. [11]

    and Lopez, Florent and Mary, Theo and Pranesh, Srikara , journal=

    Blanchard, Pierre and Higham, Nicholas J. and Lopez, Florent and Mary, Theo and Pranesh, Srikara , journal=. Mixed Precision Block Fused Multiply-Add: Error Analysis and Application to

  12. [12]

    ACM Computing Surveys , volume=

    What Every Computer Scientist Should Know About Floating-Point Arithmetic , author=. ACM Computing Surveys , volume=

  13. [13]

    Fasi, Massimiliano and Mikaitis, Mantas , journal=

  14. [14]

    Verifiable Machine Learning: A Survey of Zero-Knowledge Proofs for

    Kang, Daniel and others , journal=. Verifiable Machine Learning: A Survey of Zero-Knowledge Proofs for

  15. [15]

    2025 , howpublished=

    Qwen3 Technical Report , author=. 2025 , howpublished=

    2025

  16. [16]

    Impacts of Floating-Point Non-Associativity on Reproducibility for

    Shanmugavelu, Sanjif and Taillefumier, Maxime and Culver, Christopher and Hernandez, Oscar and Coletti, Mark and Sedova, Ada , booktitle=. Impacts of Floating-Point Non-Associativity on Reproducibility for

  17. [17]

    Yuan, Jiayi and Li, Hao and Ding, Xinheng and Xie, Wenya and Li, Yu-Jhe and Zhao, Wentian and Wan, Kun and Shi, Jing and Hu, Xia and Liu, Zirui , journal=. Give Me

  18. [18]

    2025 , howpublished=

    Defeating Nondeterminism in. 2025 , howpublished=

    2025

  19. [19]

    2025 , howpublished=

    Towards Deterministic Inference in. 2025 , howpublished=

    2025

  20. [20]

    2025 , journal =

    Karvonen, Adam and Reuter, Daniel and Rinberg, Roy and Marks, Luke and Garriga-Alonso, Adri\`. arXiv preprint arXiv:2511.20621 , year=

    work page arXiv

  21. [21]

    Verifying

    Rinberg, Roy and Karvonen, Adam and Hoover, Alexander and Reuter, Daniel and Warr, Keri , journal=. Verifying

  22. [22]

    Sun, Haochen and Li, Jason and Zhang, Hongyang , booktitle=. zk

  23. [23]

    2024 , howpublished=

    Solving Reproducibility Challenges in Deep Learning and. 2024 , howpublished=

    2024

  24. [24]

    Fingerprinting All

    Cankaya, Naci and Kry\'. Fingerprinting All. 2026 , note=

    2026

  25. [25]

    Poisoning Attacks on

    Souly, Alexandra and Rando, Javier and Chapman, Ed and Davies, Xander and Hasircioglu, Burak and Shereen, Ezzeldin and Mougan, Carlos and Mavroudis, Vasilios and Jones, Erik and Hicks, Chris and Carlini, Nicholas and Gal, Yarin and Kirk, Robert , journal=. Poisoning Attacks on

  26. [26]

    Poisoning web- scale training datasets is practical,

    Poisoning Web-Scale Training Datasets is Practical , author=. arXiv preprint arXiv:2302.10149 , year=

    work page arXiv

  27. [27]

    Proofs of useful work from arbitrary matrix multipli- cation.CoRR, abs/2504.09971, 2025

    Proofs of Useful Work from Arbitrary Matrix Multiplication , author=. arXiv preprint arXiv:2504.09971 , year=

    work page arXiv

  28. [28]

    2025 , howpublished=

    2025

  29. [29]

    2026 , howpublished=

    2026

  30. [30]

    2023 , howpublished=

    glibc floating point math functions provide slightly different results between. 2023 , howpublished=

    2023

  31. [31]

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces , author=. arXiv preprint arXiv:2312.00752 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  32. [32]

    , institution=

    Jia, Zhe and Maggioni, Marco and Smith, Jeffrey and Scarpazza, Daniele P. , institution=. Dissecting the. 2019 , note=

    2019

  33. [33]

    and Mishra, Asit K

    Jog, Adwait and Kayiran, Onur and Pai, Ashutosh and Kandemir, Mahmut T. and Mishra, Asit K. and Iyer, Ravishankar and Das, Chita R. , booktitle=. Managing. 2014 , organization=

    2014

  34. [34]

    2025 , howpublished=

    State of. 2025 , howpublished=

    2025

  35. [35]

    Scaling Laws for Neural Language Models

    Scaling Laws for Neural Language Models , author=. arXiv preprint arXiv:2001.08361 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  36. [36]

    2024 , howpublished=

    2024

  37. [37]

    2025 , howpublished =

    2025

  38. [38]

    2020 , institution =

    2020

  39. [39]

    2022 , institution =

    2022

  40. [40]

    2025 , institution =

    2025

  41. [41]

    2019 , pages =

    IEEE Std 754-2019 (Revision of IEEE 754-2008) , title =. 2019 , pages =

    2019

  42. [42]

    Shah, Jay and Bikshandi, Ganesh and Zhang, Ying and Thakkar, Vijay and Ramani, Pradeep and Dao, Tri , booktitle =

  43. [43]

    2026 , month = apr, type =

    2026