Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Late Breaking Result: FPGA-Based Emulation and Fault Injection for CNN Inference Accelerators

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes an FPGA-based platform that makes fault-tolerance analysis of CNN inference accelerators run an order of magnitude faster than software simulation by emulating a real NVDLA accelerator and injecting faults directly into…

desk verdict Useful open-source FPGA fault-injection platform for NVDLA, but the key claim that it emulates NVDLA faithfully is unvalidated, so the accuracy-drop results are not yet trustworthy. read the letter →

arxiv 2501.12818 v1 pith:VTWO5VTZ submitted 2025-01-22 cs.AR

Filip Masar , Vojtech Mrazek , Lukas Sekanina This is my paper · ORCID
classification cs.AR
keywords FPGAemulationfaultinjectionCNNinferenceacceleratorNVDLAtoleranceanalysisResNet-18MACunithardwarereliability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes an FPGA-based emulation platform that lets researchers run fault-injection experiments directly on a hardware implementation of a CNN inference accelerator, rather than on a software model. In the case study, an NVDLA accelerator executing ResNet-18 on CIFAR-10 is equipped with logic that can override the output of any multiplier, emulating stuck-at or pulse faults, and the resulting classification-accuracy drop is measured. The reported platform performs 217 inferences per second over the whole network, while the software baseline completes only 5.8 simulations per second for two convolutional layers, making the approach about an order of magnitude faster. The hardware overhead of the fault-injection logic is small: 18 additional look-up tables for a fixed error value and well under a percent of logic resources for a dynamically selected value.

What carries the argument

The load-bearing mechanism is a fault injector attached to the 18-bit output of every multiplier inside the MAC units. Controlled through an AXI4-Lite interface, it can override each multiplier's output with zero (stuck-at fault) or a constant (pulse fault), and by changing the source code other fault models can be added. Around this, the platform combines the NVDLA accelerator mapped to FPGA logic, the Tengine framework converting the pre-trained CNN into an execution plan, and ARM software that orchestrates inference and injection. The fault-injection logic's small footprint (roughly 18 LUTs for a fixed constant, 0.71% LUT and 0.31% FF increase for a dynamically selected value) is what makes inserting fault-injection hardware into the accelerator cheap enough for a practical analysis platform.

What would settle it

Run the identical fault-injection campaign (same network, same weights, same multiplier positions and error values) on a cycle-accurate NVDLA simulator or actual NVDLA silicon and compare per-multiplier accuracy drops; a material discrepancy in the drops or in which multipliers are most sensitive would show the emulator is not a faithful fault model.

Watch

Extended reading notes

Core claim

The paper's central claim is that fault-tolerance analysis of CNN inference accelerators can be moved from software simulation to an FPGA emulator without losing the architecture-level detail that matters. The authors build an 8-bit NVDLA accelerator on a Zynq UltraScale+ SoC using the NVDLA Xilinx FPGA Mapping tool, connect it to the Tengine runtime so ResNet-18 runs with the pre-trained CIFAR-10 weights, and insert an 18-bit fault injector at the output of each of the 64 multipliers in the eight MAC units. Injecting constant errors 0, 1, and -1 into one to seven multipliers shows accuracy drop growing with the number of affected multipliers, with some multiplier positions more sensitive than others. The performance argument is that the emulator sustains 217 inferences per second for the entire ResNet-18, versus 5.8 simulations per second for two layers in a recent software framework, and the FPGA implementation also outperforms single-thread ARM and Ryzen software inference.

Load-bearing premise

The platform's results are meaningful only if the FPGA version of NVDLA, modified to add fault injection, responds to multiplier faults the same way the real NVDLA silicon would.

Editorial extensions

If this is right

  • Full-network fault-injection campaigns become feasible: 217 inferences per second on ResNet-18 means thousands of injected-fault scenarios can be swept in minutes rather than hours.
  • The negligible resource overhead (18 LUTs for a fixed error, under 1% for a variable error) means the fault-injection logic does not disturb accelerator timing or inference latency (4.59 ms per image unchanged).
  • The observed accuracy drop depends mainly on how many multipliers fail, not on whether the injected constant is 0, 1, or -1.
  • Because specific multiplier positions respond differently, reliability analysis at the individual-multiplier level can identify the most vulnerable parts of a MAC array.
  • The platform can be extended to other fault models and, as the authors state, used to validate software fault-tolerance tools and to study error-correcting or redundant mechanisms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the emulator's fault response is faithful to real NVDLA hardware, the platform could replace software simulators as the reference for fault-tolerance studies; the paper does not yet demonstrate this fidelity.
  • The reported throughput comparison is between complete-network emulation and a two-layer software simulation, so the exact speedup for a like-for-like workload remains to be quantified; a direct comparison on the same layers would tighten the claim.
  • The current configuration disables Winograd convolution, weight compression, and SRAM caching; faults in those features could produce different vulnerability maps, and testing with them enabled would show whether the ranking of sensitive multipliers is stable.
  • With the fault injector configurable per multiplier, an obvious next experiment is a full exhaustive sweep over all 64 multipliers and all error values, which would replace the 210 random samples with a complete vulnerability map.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes an FPGA-based emulation platform for fault tolerance (FT) analysis of CNN inference accelerators. The platform is built by mapping the open-source NVDLA accelerator to a Zynq UltraScale+ SoC, adding fault-injection logic at the outputs of the 8x8 multiplier array, and using Tengine to convert a Caffe-trained ResNet-18 network into an execution plan. The authors report synthesis and performance results, area overhead of the fault-injection logic, and a first case study measuring classification accuracy drop on CIFAR-10 when constant errors (0, 1, -1) are injected into one or more multipliers. They claim the FPGA-based FT analysis is an order of magnitude faster than software emulation, citing 217 inferences per second for the entire ResNet-18 versus 5.8 simulations per second for two layers in the SAFFIRA framework.

Significance. If the emulator faithfully represents the target accelerator, the platform would be a valuable open-source tool for fast, hardware-level fault-injection experiments on a realistic NVDLA-like architecture. The concrete synthesis and timing measurements in Table I are a useful contribution, and the public repository enables reproducibility. However, the significance currently rests on two unverified pillars: the fidelity of the FPGA-mapped NVDLA to the real accelerator, and the fairness of the speed comparison against software simulation. The fault-tolerance results, while suggestive, lack the statistical grounding needed to support claims about multiplier-specific susceptibility.

major comments (3)
  1. [Section II (Fidelity of the NVDLA emulation)] The accelerator is described as an NVDLA created with the NVDLA Xilinx FPGA Mapping tool [9] and, in the authors' own words, 'excluding features like SRAM cache, reshaping, Winograd convolution, and weight compression.' The paper does not validate that the resulting FPGA implementation behaves like the real NVDLA under faults. Because the fault injector overrides the 18-bit output of each multiplier after synthesis, any retiming, pipelining, or multiplier re-mapping performed by the FPGA toolchain could change which physical operation corresponds to a synthesized multiplier. Without comparison against a cycle-accurate NVDLA simulator or silicon measurements, the accuracy-drop curves in Figs. 2 and 3 may be artifacts of the FPGA mapping. This is load-bearing because the central claim is that the platform performs FT analysis for NVDLA; as written, the results characterize a custom FPGA accelerator, not necessarily NVDLA.
  2. [Section IV (Speedup claim)] The claim that the FT analysis 'is an order of magnitude faster than software emulation' compares 217 inferences per second for the whole ResNet-18 on FPGA with 5.8 simulations per second for 'only two convolutional layers' in SAFFIRA [4]. These are not matched workloads: the comparison mixes a full-network inference on hardware with a partial-network software simulation, and no end-to-end fault-injection scenario is defined for either system. To support the speedup claim, the authors should compare the same CNN, the same number of layers, the same bit-width, and the same fault-injection campaign (or equivalent per-image and per-fault costs), or measure SAFFIRA on the full ResNet-18. As reported, the order-of-magnitude statement is not established.
  3. [Section III (Statistical validity of fault-injection results)] The paper reports 'results obtained from 210 FIs' for Fig. 2 and per-multiplier accuracy drops in Fig. 3, but it does not state how many fault injections were performed per condition (e.g., per number of affected multipliers, per multiplier position), how many test images were used, or whether repeated trials show variance. The heat map in Fig. 3 appears to be based on one fault per multiplier position, and the claim that 'some multipliers exhibit greater sensitivity to faults' and that the last multiplier of MAC 1 is the most critical is made without confidence intervals or a statistical test. The difference could easily be within random variation of the test set. This undermines the specific susceptibility conclusions, though it does not invalidate the overall platform concept.
minor comments (5)
  1. [Section II] The fault-injection mechanism is described as overriding the multiplier output 'with either zero or a constant value, emulating thus a stuck-at error or a pulse fault, respectively.' A pulse fault is normally a transient event; the paper should clarify whether the override lasts for a single inference, for all inferences, or for a specific time window, as this materially changes the fault model.
  2. [Table I] The rows for 'NVDLA + FI (constant error)' and 'NVDLA + FI (variable error)' report the same 4.59 ms inference time as the base NVDLA. It would be helpful to state explicitly whether this timing includes any fault-injection setup or reconfiguration overhead, and which fault was injected in the timed run.
  3. [Section III, Fig. 2] The box plots would be more informative if the caption stated the number of random multiplier selections per condition and the number of images used to compute accuracy, so that the spread of the boxes can be interpreted as sampling variability rather than only a distribution of fault effects.
  4. [Abstract / Section II] The abstract says that 'for a given CNN model, hardware accelerator architecture, and FT analysis target, an FPGA-based CNN implementation is generated,' but the paper demonstrates only one fixed architecture and one CNN. The authors should clarify what parts of the flow are automated and what manual steps remain, especially regarding the fault-injection logic generation for a new network or accelerator.
  5. [Throughout] There are minor typographical issues, including 'single-threat ARM implementation' in Section III which should be 'single-thread,' and the reference to SAFFIRA in Section IV correctly identifies [4] but the speed comparison should also cite the specific configuration used for the 5.8 simulations/s figure.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all reported results are direct measurements or benchmark comparisons, with no fitted parameter or self-citation chain forcing the conclusions.

full rationale

The paper reports an FPGA-based fault-injection platform and presents measured results: inference latency, FPGA resource usage, and classification-accuracy drops under injected multiplier faults. These are direct empirical measurements of the implemented system, not predictions derived from fitted parameters. The accuracy-drop curves in Figs. 2 and 3 are consequences of physically injecting faults into the FPGA implementation and measuring classification accuracy; the injected fault value is an input, not an output derived from the measured accuracy. The speed comparison with SAFFIRA (217 inferences per second versus 5.8 simulations per second for two layers) is a benchmark comparison between different systems, and no quantity in the comparison is defined in terms of the other. The paper's use of the NVDLA Xilinx FPGA Mapping tool and the Tengine framework is a construction choice, and the lack of validation of the emulator's fidelity against real NVDLA silicon is a correctness or validity concern, not a circularity concern. No load-bearing self-citations appear, and the central claims do not reduce by definition to their inputs. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper contains no fitted constants; the fault values (0, 1, -1), the number of affected multipliers, and the selected MAC positions are experimental inputs, not free parameters. All load-bearing premises are domain assumptions about the faithfulness of the emulation and the fault model.

assumptions (3)
  • domain assumption The FPGA-mapped NVDLA implementation faithfully represents the NVDLA accelerator architecture for fault-tolerance analysis.
    The platform builds the accelerator with the NVDLA Xilinx FPGA Mapping tool [9] and modifies the software stack, but no experiment validates that its fault response matches the NVDLA ASIC or a trusted NVDLA model.
  • domain assumption Overriding a multiplier output with 0, 1, or -1 models meaningful stuck-at and pulse faults in NVDLA MAC units.
    The fault injector replaces the 18-bit multiplier output with a constant; the paper does not justify that this matches realistic fault distributions in NVDLA multipliers.
  • domain assumption The Tengine-pretrained ResNet-18 with 75.5% CIFAR-10 accuracy is the correct reference for measuring accuracy drop.
    Accuracy drop is computed against this model from the Tengine Model Zoo, which is not independently verified in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Late Breaking Result: FPGA-Based Emulation and Fault Injection for CNN Inference Accelerators." pith.science (2026). https://pith.science/paper/VTWO5VTZ

@misc{pith2026250112818,
  author       = {Pith},
  title        = {Pith review of: Late Breaking Result: FPGA-Based Emulation and Fault Injection for CNN Inference Accelerators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VTWO5VTZ}},
  note         = {Machine review of arXiv:2501.12818}
}
read the original abstract

A new field programmable gate array (FPGA)-based emulation platform is proposed to accelerate fault tolerance analysis of inference accelerators of convolutional neural networks (CNN). For a given CNN model, hardware accelerator architecture, and FT analysis target, an FPGA-based CNN implementation is generated (with the help of the Tengine framework), and fault injection logic is added. In our first case study, we report how the classification accuracy drop depends on the faults injected into multipliers used in Multiply-and-Accumulate Units of NVDLA inference accelerator executing ResNet-18 CNN. The FT analysis emulated on Zynq UltraScale+ SoC is an order of magnitude faster than software emulation.

Figures

Figures reproduced from arXiv: 2501.12818 by the authors.

Figure 2
Figure 2. Accuracy drop depending on the number of affected [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The accuracy drop after injection of (a) 0, (b) 1, and [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. STAR: Astrocyte-Inspired State-Augmented Repair for Supervised Memristive AI Hardware Systems

    cs.ET 2026-07 conditional novelty 5.0 of 10

    STAR restores accuracy of memristive neural networks after stuck-at faults by adding a repair nudge during equilibrium-propagation retraining that pulls layer states toward pre-fault class-conditional activation targets.

Reference graph

Works this paper leans on

12 extracted references · 10 canonical work pages · cited by 1 Pith paper

  1. [9]

    L. Wang. (2021) CN NVDLA xilinx FPGA mapping . [Online]. Available: https://leiblog.wang/NVDLA-Xilinx-FPGA-Mapping/

    work page 2021

  2. [4]

    Taheri, M

    M. Taheri, M. Daneshtalab et al., `` SAFFIRA : a framework for assessing the reliability of systolic-array-based dnn accelerators,'' in DDECS '24', 2024

    work page 2024

  3. [1]

    M. H. Ahmadilivani, M. Taheri et al., ``A systematic literature review on hardware reliability assessment methods for deep neural networks,'' ACM Comput. Surv. , vol. 56, no. 6, pp. 141:1--141:39, 2024

    work page 2024

  4. [2]

    M. H. Ahmadilivani, M. Barbareschi et al., ``Special session: Approximation and fault resiliency of dnn accelerators,'' in VTS '23, 2023

    work page 2023

  5. [3]

    Y. He, P. Balaprakash et al., `` FIdelity : Efficient resilience analysis framework for deep learning accelerators,'' in MICRO '20, 2020

    work page 2020

  6. [5]

    Pappalardo, A

    S. Pappalardo, A. Ruospo et al., `` A Fault Injection Framework for AI Hardware Accelerators ,'' in LATS '23, 2023

    work page 2023

  7. [6]

    Sharma, J

    H. Sharma, J. Park et al., ``From high-level deep neural models to fpgas,'' in MICRO '16', 2016

    work page 2016

  8. [7]

    Y.-H. Chen, T. Krishna et al., ``Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks,'' IEEE Journal of Solid-State Circuits, vol. 52, no. 1, pp. 127--138, 2017

    work page 2017

Show all 12 references
  1. [8]

    [Online]

    `` NVIDIA deep learning accelerator (NVDLA) ,'' 2021. [Online]. Available: http://nvdla.org

    2021

  2. [10]

    EN Tengine

    OPEN AI LAB . EN Tengine . [Online]. Available: https://github.com/OAID/Tengine

  3. [11]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write ...

  4. [12]

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.