Pith. sign in

REVIEW 3 major objections 4 minor 32 references

This paper claims that INT8 post-training quantization makes ARM edge inference dispatch-invariant, with byte-identical outputs across microarchitectures and execution providers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 00:01 UTC pith:VCXFM6BR

load-bearing objection A genuinely new and mostly solid empirical result on INT8 reproducibility across ARM parts, but the title's 'dispatch-invariant' goes a bit beyond what the timing evidence actually proves. the 3 major comments →

arxiv 2607.23227 v1 pith:VCXFM6BR submitted 2026-07-25 cs.ET cs.PF

INT8 Quantization Makes ARM Edge Inference Dispatch-Invariant

Sebasti\'an A. Cruz Romero , Shenied E. Maldonado Guerra This is my paper
classification cs.ET cs.PF
keywords INT8 quantizationQDQ post-training quantizationARM edge inferencemicrokernel dispatchnumerical reproducibilityequivalence classesINT32 accumulationARM Cortex-A
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

On ARM edge hardware, the numerical outputs of a fixed neural network do not depend on which ARM microarchitecture executes them, provided the model is quantized to INT8 in the QDQ format. Across devices spanning Cortex-A53, A72, and A76, FP32 inference is bit-identical when only the hardware changes, but changes with the execution provider; INT8 collapses both axes to a single equivalence class. The authors trace this to a structural property: discrete-grid inputs make any convolution's arithmetic exact and order-independent, and the quantizer at each layer boundary restores that precondition. They verify the prediction on production CNNs, where every intermediate accumulator and final output is byte-identical between devices running different ARM kernel families. If right, the result tells practitioners that INT8 is the reproducible mode on ARM fleets and that the meaningful variation axis is precision, not microarchitecture.

Core claim

The central discovery is that INT8 post-training quantization makes inference dispatch-invariant on ARM: under the TFLite/XNNPACK CPU path, MobileNetV2 and ResNet50V2 produce byte-identical outputs on Cortex-A76 and Cortex-A72 across 500 ImageNet images per model, including every intermediate INT32 accumulator and INT8 tensor on a 50-image subset. The explanatory claim is H1+H2: a conv or gemm whose input lies on the quantization grid is deterministic regardless of which kernel executes it, and a QuantizeLinear node after every layer keeps inputs on the grid. The operator-level reason is that both ARM paths - SDOT and the NEON multiply-accumulate fallback - accumulate exactly into INT32 with

What carries the argument

Two properties, H1 and H2. H1 (discrete-grid absorption) is the claim that if every element of a conv/gemm's input lies exactly on the quantization grid (scale s, zero-point z), then the operator's FP32 output is byte-identical across execution providers. H2 (requantization ratchet) is the claim that every QuantizeLinear node in a QDQ graph emits a tensor on that grid, so the precondition is restored at every layer boundary. The carrying mechanism is the exact INT32 accumulator: ARM's SDOT and NEON/SMULL paths both compute integer dot products exactly, without the saturating INT16 intermediate that x86's PMADDUBSW introduces; because integer addition is associative, SIMD width and reduction-

Load-bearing premise

The claim that the two devices demonstrably execute different microkernel families rests on timing signatures and documented dispatch rules rather than direct instruction-level proof; if both devices actually ran the same code, the byte-identical outputs would demonstrate determinism but not dispatch-invariance.

What would settle it

Run the same INT8 TFLite models on a Cortex-A76 and Cortex-A72 while using perf stat to count SDOT instructions on the A76; if the A76 shows no SDOT execution, the dispatch-difference premise fails. Conversely, an adversarially chosen input or a channel dimension beyond the stated bound that triggers an INT32 overflow or a saturating intermediate would break byte-identity and falsify the generalization.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On a mixed ARM fleet running the same INT8 QDQ model under a runtime with exact INT32 kernels, outputs are byte-identical across microarchitectures; continuous-output consumers such as calibration, anomaly scoring, audit logs, and A/B comparisons get consistency across hardware.
  • FP32 remains provider-dependent even on a single microarchitecture: switching execution provider can flip outputs on every image, so FP32 reproducibility claims must be scoped to the provider, not to the hardware.
  • INT8's label changes relative to FP32 are identical on every tested ARM device, so no hardware-specific label drift exists in quantized production CNNs.
  • Hardware fingerprinting that relies on output divergence cannot distinguish ARM microarchitectures through INT8 CNN inference; auditing must target FP32 inference, autoregressive generation, or timing side channels.
  • The x86 saturating-INT16 mechanism is the boundary: platforms using PMADDUBSW can fragment INT8 outputs, while platforms with direct INT32 accumulation - ARM and VNNI - should not.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If H1+H2 is structural rather than specific to the tested runtimes, byte-identity should extend to other ARM runtimes and to untested microarchitectures such as Cortex-A55, A78, and Oryon; this is directly testable without new theory.
  • The proof's bound suggests a crisp sufficient condition for cross-platform bit-exact INT8 inference: any kernel family with exact INT32 accumulation and no saturating intermediates will exhibit the same invariance, regardless of SIMD-width differences.
  • The FP32 execution-provider divergence on fixed hardware provides a practical check for whether a runtime's advertised provider is actually active: if outputs are identical across providers, dispatch likely fell back silently.
  • A direct instruction-count confirmation would strengthen the dispatch claim, but byte-identity itself is a determinism result that holds even if the kernel families were identical; the dispatch-invariance reading is the stronger, less certain part.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies numerical reproducibility of neural-network inference on ARM edge hardware. It reports, on four Raspberry Pi devices spanning Cortex-A53/A72/A76 under ONNX Runtime, that FP32 CPU-EP outputs are byte-identical across microarchitectures, while switching to the XNNPACK execution provider on a fixed A76 device produces divergent FP32 outputs on all 1,000 CIFAR-10 images (mean remaining precision 14.97 of 23 mantissa bits). INT8 QDQ post-training quantization is reported to collapse both axes: all five conditions produce byte-identical outputs. The authors propose a structural explanation, H1+H2: discrete-grid inputs make Conv/Gemm dispatch-deterministic, and QuantizeLinear at each layer boundary restores that precondition. They then test the prediction on MobileNetV2 and ResNet50V2 under TensorFlow Lite/XNNPACK, reporting byte-identical final outputs and byte-identical intermediate INT8/INT32 tensors between Cortex-A76 (SDOT) and Cortex-A72 (NEON) over 500 ImageNet images per model, and attribute the x86 failure mode to PMADDUBSW saturating INT16 intermediates.

Significance. If the central claim holds, the paper makes a valuable and practical contribution: it delimits the Schlögl et al. EQC/RP phenomenon on ARM, identifies INT8 QDQ as a reproducible serving mode, and gives a concrete instruction-level reason why x86 and ARM differ. The empirical evidence is unusually strong in one respect: the production-CNN byte-identity results include layer-by-layer INT32 accumulator comparisons over 50 images per model (109 tensors for MobileNetV2, 156 for ResNet50V2), and the limitation sections are candid. The parameter-free Prop. 2-style arithmetic argument is a genuine attempt at a mechanistic explanation. The main weakness is that the headline generalization to 'different microkernel families demonstrably execute' rests on indirect timing evidence that the authors themselves acknowledge is not definitive.

major comments (3)
  1. [§5.1 and §5.2] The central claim that byte-identity holds across different ARM microkernel families is not empirically secured. The only evidence that the A76 and A72 devices ran different XNNPACK kernels is the timing asymmetry: 5.04× INT8/FP32 speedup on A76 versus ~1.0× on A72. As the authors admit, no perf stat was run and the Pi 5 thermal state (88–91 °C) adds latency variability. The alternative explanation — that both devices ran the same NEON/SMULL kernel family and the timing difference reflects A76's higher throughput — is not excluded. In that case the byte-identical outputs demonstrate same-code determinism, not dispatch-invariance. This is load-bearing because the abstract and introduction state that 'different ARM microkernel families demonstrably execute.' The authors should either provide direct dispatch confirmation (e.g., perf stat or instruction-level traces) or explicitly relabel th
  2. [§5.3, Prop. 2] Proposition 2's proof contains an incorrect product bound. It states that for (a_k, b_k) ∈ [−128, 127]^2, |a_k b_k| ≤ 127×128 = 16,256, but the maximum absolute product in that domain is (−128)·(−128) = 16,384. With 16,384 per product, K = 131,072 gives 16,384 × 131,072 = 2^31, which exceeds the INT32 positive limit. The theorem as stated is therefore not proven. This is easily fixable: either restrict weights to [−127, 127] as XNNPACK does, or lower the stated K bound to 131,071. Because the tested models have K ≤ 2048, the empirical conclusions are unaffected, but the proposition's claim of a general bound should be corrected.
  3. [§4.2, Def. 1 / Prop. 1] The explanatory status of H1 needs clarification. Definition 1 defines discrete-grid absorption as the property that the FP32 output is byte-identical under the compared providers, and Proposition 1 is then a restatement of that property plus the requantization assumption. As the paper itself notes, Proposition 1 is 'an empirical property, not a theorem.' The independent mechanistic content comes from the localization experiment and from Prop. 2. I recommend the authors state more explicitly that H1 is a shorthand for the empirical finding, not a prediction derived from first principles, so readers do not mistake a definition for an explanation.
minor comments (4)
  1. [Abstract and §5.1] The abstract says 'timing evidence confirms SDOT dispatch on A76 and NEON multiply-accumulate on A72,' but §5.1 explicitly calls the timing evidence 'strong but indirect.' Please align the language; 'consistent with' rather than 'confirms' would be more accurate unless direct confirmation is added.
  2. [Introduction] Typo in first paragraph: 'rutn' should be 'run.'
  3. [§5.3, Eq. (1)] The proof would be clearer if it stated explicitly that the bound applies to the running accumulator after each SDOT instruction, and if it noted that the XNNPACK symmetric-weight constraint [−127, 127] is what makes the 16,256 product bound valid for the actual kernel.
  4. [Table 2] The distinction between 'EP (intended)' and 'EP (resolved)' is useful, but the table caption could note that the resolved provider was parsed from telemetry; this is stated in the text but not in the caption.

Circularity Check

0 steps flagged

No significant circularity: the byte-identity result is an out-of-sample measurement, Prop. 2 is a parameter-free arithmetic bound, and no load-bearing self-citation is present.

full rationale

The paper's central empirical claim—that INT8 QDQ inference is byte-identical between Cortex-A76 SDOT and Cortex-A72 NEON paths on MobileNetV2 and ResNet50V2—is not fitted or derived from the claimed explanation. The H1+H2 invariant is an empirical decomposition measured on a separate small CNN (CifarSmall), and the production-CNN test uses 500 unseen ImageNet images per model with all intermediate tensors compared. No constants are tuned to force the result. Proposition 2 is a parameter-free arithmetic bound (K ≤ 131,072, with products bounded by 16,256) showing that both ARM kernel families compute the same exact INT32 sum; this is an independent mathematical statement, not a restatement of the data. The paper is transparent that Proposition 1 is stated as an empirical property, not a theorem, and that direct perf-stat confirmation of microkernel dispatch was not performed. Those are correctness/evidence limitations, not circularity. There are no self-citations: the cited EQC/RP, XNNPACK, oneDNN, and ARM references are external. The small-CNN/provider results are honestly bounded, and the silent-fallback artifact is excluded from positive evidence. The only mild framing concern is that H1 and H2 are named after the byte-identity properties they are used to explain, but because the production measurement is independent and Prop. 2 carries the arithmetic weight, this does not reduce the derivation to its inputs.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The central claim rests on standard exact-integer arithmetic plus two domain assumptions about XNNPACK/ARM (no saturating intermediates; actually-different microkernels) and one graph-level empirical invariant (H1+H2) that is stated rather than formally proved. No physical entities are introduced; H1/H2 are named empirical invariants, treated as axioms here.

free parameters (1)
  • Per-tensor/per-layer quantization scales and zero-points = e.g., s=0.003922, z=-128 for CifarSmall input QDQ; per-layer values from calibration
    These are fitted during post-training quantization and define the discrete grid G(s,z) that H1 refers to; they are standard QDQ parameters rather than ad hoc to the paper, and the invariance claim is not sensitive to their exact values.
axioms (4)
  • domain assumption Integer addition is exact and associative in the INT32 accumulator; ARM SDOT and NEON/SMLAL/SADALP paths have no saturating intermediate (no PMADDUBSW analogue).
    Prop. 2 relies on the ARM Architecture Reference Manual and XNNPACK kernel source; this is plausible but not independently verified in the preprint.
  • domain assumption The A76 and A72 devices actually executed different XNNPACK kernel families (SDOT vs NEON multiply-accumulate).
    Supported only by timing signatures (5.04x vs ~1.0x INT8/FP32 speedup), not by perf stat; authors call this indirect.
  • ad hoc to paper H1: any Conv/Gemm with inputs on an int8 grid is byte-identical across dispatch paths.
    Defined in §4.3 from two-provider observations on CifarSmall; generalized to a structural invariant without a formal dispatch trace.
  • ad hoc to paper H2: QuantizeLinear at every layer boundary restores the grid precondition.
    Observed on the nine QDQ hooks in CifarSmall; assumed to hold for all QDQ graphs of the listed operator set.

pith-pipeline@v1.3.0-alltime-deepseek · 13346 in / 16534 out tokens · 143550 ms · 2026-08-01T00:01:35.472663+00:00 · methodology

0 comments
read the original abstract

On x86, kernel dispatch fragments the outputs of the same neural network into many equivalence classes across hardware. We ask whether the same fragmentation governs ARM edge inference, where most edge ML actually runs. Across four Raspberry Pi devices spanning Cortex-A53, A72, and A76 under ONNX Runtime CPU, microarchitecture is not observable in the outputs of a fixed FP32 CNN. Holding hardware constant at Cortex-A76 and switching only the execution provider, FP32 outputs disagree on every CIFAR-10 image with a mean remaining precision of 14.97 of 23 mantissa bits. INT8 QDQ post-training quantization collapses both axes to a single equivalence class. We trace this to a structural property of QDQ graphs that we call H1+H2: discrete-grid inputs make any Conv dispatch-deterministic (H1) and QuantizeLinear at every layer boundary preserves that precondition (H2). H1+H2 predicts that bit-exact agreement should extend to production CNNs under runtimes that confirmably exercise different ARM microkernels. We verify this on MobileNetV2 and ResNet50V2 under TensorFlow Lite with XNNPACK, where timing evidence confirms SDOT dispatch on A76 and NEON multiply-accumulate on A72 yet every intermediate INT32 accumulator and every final output is byte-identical across 500 ImageNet images per model. We then identify the specific x86 mechanism that breaks the same invariant on x86, namely PMADDUBSW saturating INT16 intermediates, which has no ARM analogue. The Schl\"ogl et al. divergence phenomenon is delineated rather than contradicted. For practitioners deploying quantized CNNs across heterogeneous ARM fleets, the operational consequence is direct. INT8 inference is the reproducible mode and the relevant behavioral variation axis is precision, not microarchitecture.

Figures

Figures reproduced from arXiv: 2607.23227 by Sebasti\'an A. Cruz Romero, Shenied E. Maldonado Guerra.

Figure 1
Figure 1. Figure 1: Distribution of remaining precision over 1,000 CIFAR-10 test images for the four distinct condition-and-format combinations on Cortex-A76. Only the FP32 XNNPACK condition departs from byte-identity with the reference. The other three collapse to the RP = 23 ceiling. Conditions C, D, and E under CPU EP coincide with Condition B and are summarized in [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Per-layer mean RP across the three quantization scenarios in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Timing evidence for different XNNPACK microkernel dispatch on production CNNs. INT8 inference on A76 (Condition A) is 5.04× faster than FP32 (Condition B) for ResNet50V2, exceeding the 4× speedup from weight compression alone. On A72 (Condition C), INT8 and FP32 latency are nearly identical (137.80 ms versus approximately 138 ms). The asymmetric signature is consistent with SDOT throughput on A76 and its a… view at source ↗
Figure 4
Figure 4. Figure 4: INT8 GEMM accumulation paths on ARM and x86. ARM NEON widens INT8 operands transiently to INT16 via SMULL and SMLAL, then accumulates into INT32 via SADALP with no saturation at any step. ARM SDOT computes four INT8 products and accumulates into INT32 in one cycle with no saturation. The x86 PMADDUBSW instruction produces saturating INT16 output when the sum of two uint8 by int8 products exceeds 32,767, re… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 7 linked inside Pith

  1. [1]

    Colbert, A

    I. Colbert, A. Pappalardo, and J. Petri-Koenig. A2Q: Accumulator-aware quan- tization with guaranteed overflow avoidance. InProc. 2023 IEEE/CVF Int. Conf. Computer Vision (ICCV), pages 16989–16998, Paris, France, Oct. 2023. doi: 10.1109/ICCV51070.2023.01558. URLhttps://openaccess.thecvf.com/ content/ICCV2023/papers/Colbert_A2Q_Accumulator-Aware_Quantizati...

    arXiv 2023

  2. [2]

    Demmel and H

    J. Demmel and H. D. Nguyen. Fast reproducible floating-point summation. InProc. 21st IEEE Symposium on Computer Arithmetic (ARITH), pages 163–172, Austin, TX, USA, Apr. 2013. doi: 10.1109/ARITH.2013.9

    doi:10.1109/arith.2013.9 2013

  3. [4]

    Dukhan and The XNNPACK Authors

    M. Dukhan and The XNNPACK Authors. XNNPACK: High-efficiency floating-point and quantized neural network inference on ARM, x86, WebAssembly, and RISC-V plat- forms. Google AI / TensorFlow Blog, 2021. URLhttps://blog.tensorflow.org/ 2020/07/accelerating-tensorflow-lite-xnnpack-integration.html. Accessed: 2026-06-18

    2021

  4. [5]

    Goldberg

    D. Goldberg. What every computer scientist should know about floating-point arith- metic.ACM Computing Surveys, 23(1):5–48, Mar. 1991. doi: 10.1145/103162.103163

    arXiv 1991

  5. [6]

    D. Gope, J. Beu, V. Ganesh, and M. Mattina. ARM Cortex-A INT8 optimizations for edge ML inference: A survey of microkernel dispatch and instruction selection. arXiv preprint arXiv:2501.00032, Jan. 2025. URLhttps://arxiv.org/abs/2501.00032

    Pith/arXiv arXiv 2025

  6. [7]

    C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. InProc. 34th Int. Conf. Machine Learning (ICML), volume 70 ofProceedings of Machine Learning Research, pages 1321–1330, Sydney, NSW, Australia, Aug. 2017. PMLR. URLhttps://proceedings.mlr.press/v70/guo17a.html

    2017

  7. [8]

    K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In B. Leibe, J. Matas, N. Sebe, and M. Welling, editors,Computer Vision – ECCV 2016, volume 9908 ofLecture Notes in Computer Science, pages 630–645, Cham, Switzerland,

    2016

  8. [9]

    N. J. Higham.Accuracy and Stability of Numerical Algorithms. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, USA, 2nd edition, 2002. ISBN 978-0-89871-521-7. doi: 10.1137/1.9780898718027

    doi:10.1137/1.9780898718027 2002

  9. [10]

    IEEE standard for floating-point arithmetic

    IEEE Computer Society. IEEE standard for floating-point arithmetic. IEEE Std 754- 2019 (Revision of IEEE Std 754-2008), July 2019

    2019

  10. [11]

    oneDNN developer guide: INT8 inference and quantization

    Intel Corporation. oneDNN developer guide: INT8 inference and quantization. Intel oneAPI Documentation, 2024. URLhttps://oneapi-src.github.io/oneDNN/dev_ guide_int8_computations.html. Accessed: 2026-06-18

    2024

  11. [12]

    Jacob, S

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko. Quantization and training of neural networks for efficient integer- arithmetic-only inference. InProc. 2018 IEEE/CVF Conf. Computer Vision and Pat- tern Recognition (CVPR), pages 2704–2713, Salt Lake City, UT, USA, June 2018. doi: 10.1109/CVPR.2018.00286

    arXiv 2018

  12. [13]

    S. R. Jain, A. Gural, M. Wu, and C. H. Dick. Efficient computation reduction in bayesian neural networks through feature decomposition and memorization. arXiv preprint arXiv:2006.10226, June 2020. URLhttps://arxiv.org/abs/2006.10226. 12

    Pith/arXiv arXiv 2006

  13. [14]

    Krishnamoorthi

    R. Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342, June 2018. URLhttps://arxiv.org/ abs/1806.08342

    Pith/arXiv arXiv 2018

  14. [15]

    Krizhevsky

    A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Department of Computer Science, University of Toronto, Toronto, ON, Canada, 2009. URLhttps://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf

    2009

  15. [16]

    ONNX Runtime documentation: Quantize ONNX mod- els

    Microsoft Corporation. ONNX Runtime documentation: Quantize ONNX mod- els. ONNX Runtime Documentation, 2024. URLhttps://onnxruntime.ai/docs/ performance/model-optimizations/quantization.html. Accessed: 2026-06-18

    2024

  16. [17]

    Nagel, R

    M. Nagel, R. A. Amjad, M. van Baalen, C. Louizos, and T. Blankevoort. Up or down? adaptive rounding for post-training quantization. InProc. 37th Int. Conf. Machine Learning (ICML), volume 119 ofProceedings of Machine Learning Research, pages 7197–7206. PMLR, July 2020. URLhttps://proceedings.mlr.press/v119/ nagel20a.html

    2020

  17. [18]

    cuDNN developer guide: Reproducibility (determin- ism)

    NVIDIA Corporation. cuDNN developer guide: Reproducibility (determin- ism). NVIDIA Technical Documentation, 2024. URLhttps://docs.nvidia.com/ deeplearning/cudnn/developer-guide/index.html#reproducibility. Accessed: 2026-06-18

    2024

  18. [19]

    H. V. Pham, S. Qian, J. Wang, T. Lutellier, J. Rosenthal, L. Tan, Y. Yu, and N. Na- gappan. Problems and opportunities in training deep learning software systems: An analysis of variance. InProc. 35th IEEE/ACM Int. Conf. Automated Software Engi- neering (ASE ’20), pages 771–783, Virtual Event, Australia, Sept. 2020. ACM. doi: 10.1145/3324884.3416545

    arXiv 2020

  19. [20]

    V. J. Reddi, C. Cheng, D. Kanter, P. Mattson, G. Schmuelling, C.-J. Wu, B. Ander- son, M. Breughe, M. Charlebois, W. Chou, R. Chukka, C. Coleman, S. Davis, P. Deng, G. Diamos, J. Duke, D. Fick, J. S. Gardner, I. Hubara, S. Idgunji, T. B. Jablin, J. Jiao, T. St. John, P. Kanwar, D. Lee, J. Liao, A. Lokhmotov, F. Massa, P. Meng, P. Micike- vicius, C. Osborn...

    arXiv 2020

  20. [21]

    Sán Adrián, P

    D. Sán Adrián, P. Llopis, J. Garcia-Blas, and J. Carretero. The cambrian explosion of edge AI hardware: Benchmarking heterogeneous microarchitectures for INT8 inference. arXiv preprint arXiv:2506.11728, June 2025. URLhttps://arxiv.org/abs/2506. 11728

    Pith/arXiv arXiv 2025

  21. [22]

    Sandler, A

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. MobileNetV2: In- verted residuals and linear bottlenecks. InProc. 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), pages 4510–4520, Salt Lake City, UT, USA, June 2018. doi: 10.1109/CVPR.2018.00474

    arXiv 2018

  22. [23]

    Schlögl, T

    A. Schlögl, T. Kupek, and R. Böhme. Forensicability of deep neural network inference pipelines. InProc. 2021 IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), pages 2515–2519, Toronto, ON, Canada (virtual), June 2021. doi: 10.1109/ ICASSP39728.2021.9414301

    arXiv 2021

  23. [24]

    Schlögl, T

    A. Schlögl, T. Kupek, and R. Böhme. iNNformant: Boundary samples as telltale watermarks. InProc. 2021 ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec ’21), pages 81–86, Virtual Event, Belgium, June 2021. ACM. doi: 10.1145/3437880.3460411

    arXiv 2021

  24. [25]

    Schlögl, N

    A. Schlögl, N. Hofer, and R. Böhme. Causes and effects of unanticipated numerical deviations in neural network inference frameworks. InProc. 37th Int. Conf. Neural In- formation Processing Systems (NeurIPS ’23), volume 36, pages 56095–56107, New Or- leans, LA, USA, Dec. 2023. URLhttps://papers.neurips.cc/paper_files/paper/ 13 2023/hash/af076c3bdbf935b81d8...

    2023

  25. [26]

    Sculley, G

    D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaud- hary, M. Young, J.-F. Crespo, and D. Dennison. Hidden technical debt in machine learning systems. InAdvances in Neural Information Process- ing Systems 28 (NIPS 2015), pages 2503–2511, Montréal, QC, Canada, Dec

    2015

  26. [27]

    Shanmugavelu, M

    S. Shanmugavelu, M. Taillefumier, C. Culver, O. Hernandez, M. Coletti, and A. Se- dova. Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications. InProc. SC ’24 Workshops of the Int. Conf. High Performance Computing, Network, Storage, and Analysis (SC-W ’24), pages 170–179, Atlanta, GA, USA, Nov. 2024. IEEE Pre...

    arXiv 2024

  27. [28]

    H. Wu, P. Judd, X. Zhang, M. Isaev, and P. Micikevicius. Integer quantization for deep learning inference: Principles and empirical evaluation. arXiv preprint arXiv:2004.09602, Apr. 2020. URLhttps://arxiv.org/abs/2004.09602

    Pith/arXiv arXiv 2004

  28. [29]

    J. Yuan, H. Li, X. Ding, W. Xie, Y.-J. Li, W. Zhao, K. Wan, J. Shi, X. Hu, and Z. Liu. Understanding and mitigating numerical sources of nondeterminism in LLM inference. InAdvances in Neural Information Processing Systems 38 (NeurIPS 2025), volume 38, pages 169819–169851, San Diego, CA, USA, Dec. 2025. URLhttps://openreview. net/forum?id=Q3qAsZAEZw. Oral ...

    arXiv 2025

  29. [30]

    Hardwareandsoftware platform inference

    C.Zhang, H.Foerster, R.D.Mullins, Y.Zhao, andI.Shumailov. Hardwareandsoftware platform inference. InProc. 42nd Int. Conf. Machine Learning (ICML), volume 267 ofProceedings of Machine Learning Research, Vancouver, BC, Canada, July 2025. PMLR. URLhttps://proceedings.mlr.press/v267/zhang25u.html. Also available as arXiv:2411.05197. 14

    Pith/arXiv arXiv 2025

  30. [2015]

    URLhttps://papers.nips.cc/paper/2015/hash/ 86df7dcfd896fcaf2674f757a2463eba-Abstract.html

    Curran Associates, Inc. URLhttps://papers.nips.cc/paper/2015/hash/ 86df7dcfd896fcaf2674f757a2463eba-Abstract.html

    2015

  31. [2016]

    doi: 10.1007/978-3-319-46493-0\_38

    Springer International Publishing. doi: 10.1007/978-3-319-46493-0\_38

    doi:10.1007/978-3-319-46493-0

  32. [2019]

    URLhttps://arxiv.org/abs/1907.02129

    Pith/arXiv arXiv 1907