An Efficient Sparse Hardware Accelerator for Spike-Driven Transformer

Qiwei Dong; Siyu Zhang; Wendong Mao; Zhengke Li; Zhongfeng Wang

REVIEW 3 major objections 5 minor 16 references

An Efficient Sparse Hardware Accelerator for Spike-Driven Transformer

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that encoding where spikes fire turns linear, maxpooling, and self-attention into address comparison, giving 307.2 GSOP/s at 25.6 GSOP/W.

desk verdict A solid FPGA accelerator for the Spike-driven Transformer with a genuinely new dual-spike attention module, but the headline speedup and energy numbers rest on an apples-to-oranges comparison that peer review should force the authors to fix. read the letter →

arxiv 2501.07825 v1 pith:QMUDS34R submitted 2025-01-14 cs.AR

Zhengke Li , Wendong Mao , Siyu Zhang , Qiwei Dong , Zhongfeng Wang This is my paper · ORCID

classification cs.AR

keywords spikingneuralnetworksspike-driventransformerhardwareacceleratorFPGAimplementationsparsecomputationpositionencodingself-attentionlow-powerinference

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Spike-driven Transformers run on binary spike events, yet prior SNN accelerators still read zero entries and mostly target convolutional or fully connected networks. The paper tries to show that encoding each fired spike's position in memory lets a hardware accelerator skip all non-spike values and execute linear layers, maxpooling, and spike-driven self-attention through address comparison and sparse addition. On a Xilinx Virtex UltraScale FPGA running the Spike-driven Transformer on CIFAR-10, the design reports a peak throughput of 307.2 GSOP/s and an energy efficiency of 25.6 GSOP/W, up to 13.24x and 1.33x better than three prior SNN accelerators. If true, this is a concrete path toward running transformer-scale spiking networks on low-power hardware.

What carries the argument

The Spike Encoding Array (SEA) converts each fired spike into its token position and stores the encoded spikes in an Encoded Spike SRAM, so downstream units treat every stored entry as valid and never test for zeros. The Spike Mask-Add Module (SMAM) compares address pairs from spiking Q and K, accumulates matches along tokens, thresholds to build the attention mask, and applies it to encoded V spikes. The Spike Linear Unit (SLU) uses each spike position to index a weight buffer and sums selected weights without multiplication. The shared mechanism is that all computation becomes address comparison and addition.

What would settle it

Run the same Spike-driven Transformer and CIFAR-10 benchmark on the same FPGA while fixing one SOP definition across all compared accelerators and measuring power at the board; if the 13.24x throughput or 1.33x energy ratios do not survive that setup, the central quantitative comparison collapses.

Watch

Extended reading notes

Core claim

The central claim is that a spike can be represented by its position so that zero-valued activations never enter computation. Linear layers reduce to selecting and accumulating weights at encoded spike addresses; maxpooling reduces to checking whether any fired spike's position falls inside a pooling window, with overlap reused across windows; self-attention reduces to comparing two sets of spike addresses, accumulating matches along the token dimension, and thresholding the result to form a mask that gates the value spikes. The report states that this yields 307.2 GSOP/s peak throughput and 25.6 GSOP/W energy efficiency on the Spike-driven Transformer with CIFAR-10, and that the design is unique among the compared accelerators in handling dual spike inputs for self-attention.

Load-bearing premise

The entire quantitative comparison depends on counting synaptic operations the same way in the new accelerator and in the three earlier designs, even though they run different networks on different datasets and FPGAs.

Editorial extensions

If this is right

Inference energy and latency scale with the number of fired spikes rather than the tensor dimensions, because non-spike positions never enter the datapath.
The same encoding unit serves maxpooling, linear layers, and self-attention, so a single accelerator core can map a full Spike-driven Transformer, not just convolutional or fully connected SNNs.
Self-attention with two spike inputs costs only address comparison and token-wise accumulation, making attention masks much cheaper than matrix multiplication.
On the reported FPGA implementation, peak throughput of 307.2 GSOP/s and 25.6 GSOP/W indicate that spiking transformers can run under edge-style hardware constraints.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The position-encoding trick should transfer to other attention mechanisms where keys and queries are sparse or event-driven, such as token pruning in vision transformers, though the paper does not test that.
The reported energy advantage may not survive when off-chip weight and activation movement dominates, because the design spends extra BRAMs and logic on encoding and temporal storage; a board-level power measurement would be the more demanding test.
A natural next experiment is to apply the same encoder to ImageNet-scale spiking transformers or language tasks and check whether 10-bit quantized accuracy of 94.87% on CIFAR-10 degrades.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take: the architecture is real and worth a look, but the headline numbers are not yet supported. The position-encoding method is a reasonable adaptation of address-event representation to spike-driven transformer layers. The genuinely new pieces are handling dual spike inputs in self-attention (SMAM) and applying encoded spikes to maxpooling. The modules are described concretely, the 1536-PE / 200 MHz arithmetic is internally consistent, and the implementation is fully on an FPGA, which is more than many accelerator papers do.

The soft spot is the evaluation. Table I mixes networks (FC, CNN, Transformer), datasets (MNIST, MLND, CIFAR-10), and FPGA platforms (Zynq-7000, Kintex, Virtex). The Virtex UltraScale is roughly 10x the LUT budget of the TCAD and AICAS baselines. The paper defines SOP as 'a spike traversing a unique synapse' but never shows the baselines count SOPs the same way. A Transformer workload with high sparsity can produce high SOP/s while doing qualitatively different operations (address comparisons, mask-adds) than the MAC-dominated CNN baselines. So the 13.24x speedup and 1.33x energy numbers are not established. They are plausible upper bounds, not measured improvements.

Two smaller issues: power comes from Vivado estimation, not measurement, and the baseline energy numbers may come from different estimation flows. And the 94.87% CIFAR-10 accuracy is reported after quantization but there is no comparison to the original Spike-driven Transformer accuracy, so the quantization cost is unknown. No code or bitstream is provided, which makes independent verification harder.

Overall, the central architectural idea is solid and the paper is clearly written. The comparison problem is fixable in principle: run a matched workload or clearly state that the comparison is across different operating points. I would send this to peer review, not desk reject it, but the reviewers should push for a fair comparison or a revision that drops the 'up to' ratios.

You could cite it if you work on SNN accelerator design; otherwise it is not going to change your work. I would bring it to a reading group focused on efficient transformer hardware.

Referee Report

3 major / 5 minor

Summary. The paper proposes a sparse hardware accelerator for the Spike-driven Transformer. The key idea is to encode the position information of active spikes and skip non-spike values, so that linear, maxpooling, and spike-driven self-attention computations are performed using address comparison and sparse addition rather than dense multiply-accumulate. The accelerator is implemented on a Xilinx Virtex UltraScale FPGA and evaluated on the Spike-driven Transformer with CIFAR-10, reporting a peak throughput of 307.2 GSOP/s, an energy efficiency of 25.6 GSOP/W, and up to 13.24x and 1.33x improvements over prior SNN accelerators.

Significance. The work addresses a real gap: most prior SNN accelerators target convolutional or fully connected spiking networks, while the Spike-driven Transformer requires handling dual spike inputs in self-attention. The proposed position-encoding scheme and the associated SMU, SLU, and SMAM modules are a plausible way to exploit sparsity in transformer-style SNNs. The paper also provides an RTL implementation with resource utilization and frequency details, which is a concrete strength. If the comparative performance claims were rigorously established, this would be a useful contribution to SNN hardware design. However, as presented, the headline speedup and energy-efficiency ratios rest on a comparison across different networks, datasets, and FPGA platforms, without a unified definition of the reported synaptic-operation counts.

major comments (3)

[IV-B, Table I] The headline improvements of 13.24x and 1.33x are computed from Table I, which compares designs running different networks (FC, CNN, Transformer), different datasets (MNIST, MLND, CIFAR-10), and different FPGA families (Kintex UltraScale, Zynq-7000, Zynq UltraScale, Virtex UltraScale). The paper defines a synaptic operation as 'a spike traversing a unique synapse', but it does not show that the ISCAS [14], TCAD [15], and AICAS [16] baselines use the same SOP accounting, nor that their peak throughput values are measured under comparable sparsity assumptions. Because the proposed design replaces multiply-accumulate operations with address comparisons and mask-adds, equal SOP counts may not represent equal computational work. The 13.24x and 1.33x ratios are therefore not established by the data as presented; a controlled comparison with a unified SOP definition or a matched workload is required.
[IV-A, IV-B] The energy-efficiency comparison is load-bearing but relies on inconsistent power methodology. Section IV-A states that hardware utilization and power are estimated using Vivado 2023.2, while the baseline values in Table I come from other published designs, where it is not stated whether power is measured on-chip or estimated, and on different FPGA platforms. The claimed 1.33x energy improvement over the AICAS [16] baseline depends on comparing an estimated power number with an independently reported number. The authors should either measure power on the same board using a consistent methodology or provide a sensitivity analysis showing that the 1.33x ratio is robust to plausible differences in power accounting.
[IV-B] The derivation of the peak throughput value 307.2 GSOP/s is not explained in the paper. The number is consistent with 1,536 parallel neurons operating at 200 MHz, but that equivalence assumes each neuron contributes one synaptic operation per cycle in the peak case. Since the architecture is designed to skip non-spike values, the relationship between this peak rate and actual workload sparsity needs to be stated explicitly. Without this, the reader cannot determine whether the comparison in Table I is peak-to-peak, peak-to-average, or something else, and the central throughput claim remains ambiguous.

minor comments (5)

[I, IV-A] There is a typo in 'Xilinx Virtex UltarScale' in Section I and in the Experimental Setup; it should read 'UltraScale'. Also, 'Cifar-10' is used inconsistently and should be 'CIFAR-10' throughout.
[III-C, Fig. 4] Figure 4 contains garbled glyphs in the PDF text and the comparator data path is difficult to follow. The text says the comparator 'temporarily stores the larger address' during the scan of encoded spikes, but the exact ordering and termination conditions of the comparison loop are not fully specified. Please regenerate the figure and clarify the algorithm, ideally with a small pseudocode listing.
[Table I] The footnote for the ISCAS [14] baseline says the reported values are an 'average performance under different conditions'. Please specify which conditions are averaged and why an average is the appropriate point of comparison for a peak-throughput and peak-efficiency table.
[IV-A] The paper reports 94.87% accuracy after 10-bit weight/activation and 8-bit encoded-spike quantization, but it does not give the accuracy of the unquantized Spike-driven Transformer on CIFAR-10. Adding this baseline would help the reader assess the quantization cost and the practical feasibility of the accelerator.
[References] Reference [6] contains a typo: 'ASPLOPS' should be 'ASPLOS'. Please check all reference titles for similar errors.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the paper's claims rest on measured FPGA implementation and externally published baselines; the only self-citation is non-load-bearing component reuse.

full rationale

The paper's derivation chain is architectural rather than formal: it proposes a spike-position encoding scheme and implements SMU, SMAM, and SLU units that perform maxpooling, spike-driven self-attention, and linear operations using address comparison and addition. The headline quantitative claims (307.2 GSOP/s, 25.6 GSOP/W, 13.24x and 1.33x improvements) are measured on a Xilinx Virtex UltraScale implementation and compared against separately published SNN accelerators ([14]--[16]). No parameter is fitted to the reported metrics, no predicted ratio is defined in terms of the fit, and no uniqueness theorem or ansatz is imported from the authors' own prior work to force the design. The single self-citation ([13], used for the Tile Engine convolution component) is a component reuse; the paper's novel encoding and spike-computation units do not reduce to that citation, and the comparison ratios are not derived from it. Concerns that Table I may not be apples-to-apples because SOP definitions, workloads, and FPGA families differ across baselines are benchmarking-fairness issues, not circularity: they do not show that the paper's output is equivalent to its input by construction. The validation is performed only on the Spike-driven Transformer whose sparsity motivates the design, but that is an evaluation-scope choice, not a self-referential derivation. Overall, no circular step can be quoted from the paper; the slight caution warranting a score above 0 is the presence of the self-citation and the conditional nature of the efficiency estimate, not a reduction of the central claim to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No invented physical entities are introduced; all components are arithmetic and logic units. The design relies on fixed quantization widths, the standard LIF model, and the accuracy of Vivado power estimation and cross-paper SOP counting.

free parameters (3)

Quantization bit widths = 10-bit weights/activations, 8-bit encoded spikes
Hand-picked to reach the reported 94.87% CIFAR-10 accuracy; no sensitivity analysis or baseline accuracy is given.
Parallel neuron count = 1536
Peak throughput equals 1536 neurons times 200 MHz; this architectural capacity choice is not justified relative to the compared designs.
Clock frequency = 200 MHz
Implementation target that directly affects all throughput and power numbers; no comparison at matched frequencies is provided.

assumptions (4)

domain assumption The LIF neuron model in Equations (1)-(3) is the correct abstraction for the target Spike-driven Transformer.
The accelerator implements the model from [11]; if the deployed model differs, hardware coverage and efficiency claims change.
domain assumption Encoding spike positions preserves the numerical semantics of the transformer exactly under finite-precision quantization.
No formal proof is given; the design relies on address comparison being equivalent to the Hadamard product of binary spike matrices.
domain assumption Vivado power estimates are representative of operating power.
Energy efficiency is computed from synthesis power estimation, not board measurements, and this is not validated against baseline methods.
domain assumption The compared accelerators count synaptic operations (SOPs) identically.
Throughput and energy ratios in Table I assume SOP is a commensurable metric across designs, but the paper does not verify this.

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Efficient Sparse Hardware Accelerator for Spike-Driven Transformer." pith.science (2026). https://pith.science/paper/QMUDS34R

@misc{pith2026250107825,
  author       = {Pith},
  title        = {Pith review of: An Efficient Sparse Hardware Accelerator for Spike-Driven Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QMUDS34R}},
  note         = {Machine review of arXiv:2501.07825}
}

abstract

Recently, large models, such as Vision Transformer and BERT, have garnered significant attention due to their exceptional performance. However, their extensive computational requirements lead to considerable power and hardware resource consumption. Brain-inspired computing, characterized by its spike-driven methods, has emerged as a promising approach for low-power hardware implementation. In this paper, we propose an efficient sparse hardware accelerator for Spike-driven Transformer. We first design a novel encoding method that encodes the position information of valid activations and skips non-spike values. This method enables us to use encoded spikes for executing the calculations of linear, maxpooling and spike-driven self-attention. Compared with the single spike input design of conventional SNN accelerators that primarily focus on convolution-based spiking computations, the specialized module for spike-driven self-attention is unique in its ability to handle dual spike inputs. By exclusively utilizing activated spikes, our design fully exploits the sparsity of Spike-driven Transformer, which diminishes redundant operations, lowers power consumption, and minimizes computational latency. Experimental results indicate that compared to existing SNNs accelerators, our design achieves up to 13.24$\times$ and 1.33$\times$ improvements in terms of throughput and energy efficiency, respectively.

Figures

Figures reproduced from arXiv: 2501.07825 by the authors.

**Figure 2.** The architecture of the proposed SEA. When the output of the adder [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Illustration of the calculation process of the SMU. The red and black [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: Illustration of the SLU. (a) The calculation process of the SLU [PITH_FULL_IMAGE:figures/full_fig_p003_5.png]

**Figure 4.** Figure 4: The details of SMAM. (a) Data paths of SMAM. (b) The logic of [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 6.** Figure 6: The average sparsity of SDSA and subsequent linear layers. [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

[14]

An Event-driven Spiking Neural Network Accelerator with On-chip Sparse Weight,

Y . Kuang, X. Cui, C. Zou, Y . Zhong, Z. Dai and Z. Wang, “An Event-driven Spiking Neural Network Accelerator with On-chip Sparse Weight,” IEEE Int. Symp. on Circuits and Syst. (ISCAS), Austin, TX, pp. 3468-3472, 2022

work page 2022
[15]

Skydiver: A Spiking Neural Network Accelerator Exploiting Spatio-Temporal Workload Balance,

Q. Chen, C. Gao, X. Fang and H. Luan, “Skydiver: A Spiking Neural Network Accelerator Exploiting Spatio-Temporal Workload Balance,” IEEE Trans. Computer-Aided Design Integr. Circuits Syst., vol. 41, no. 12, pp. 5732-5736, Dec. 2022

work page 2022
[16]

Q. Chen, C. Sun, C. Gao, X. Fang and H. Luan, ”FrameFire: Enabling Efficient Spiking Neural Network Inference for Video Segmentation,” 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS), Hangzhou, China, 2023, pp. 1-5

work page 2023
[1]

Handwritten digit recognition with a back-propagation network,

B.B. Le Cun, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel, “Handwritten digit recognition with a back-propagation network,” Proc. Adv. Neural Inf. Process. Syst., pp. 396-404, 1989

work page 1989
[2]

Vaswani, N

A. Vaswani, N. Shazeer, N.Parmar. “Attention is All yo u Need,“ Proc. Adv. Neural Inf. Process. Syst., pp. 5998–6008, Jun. 2017

work page 2017
[3]

Networks of spiking neurons: The third generation of neural network models,

W. Maass, “Networks of spiking neurons: The third generation of neural network models,” Neural Netw. vol. 10, no. 9, pp. 1659–1671, Dec. 1997

work page 1997
[4]

Eyeriss: An Energy- Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,

Y . Chen, K. Tushar, J. S. Emer,and S. Vivienne, “Eyeriss: An Energy- Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,” IEEE Journal of Solid-State Circuits, vol. 52, no. 1, pp. 127-138, 2017

work page 2017
[5]

SpAtten: Efficient sparse attention architecture with cascade token and head pruning,

H. Wang, Z. Zhang, and S. Han, “SpAtten: Efficient sparse attention architecture with cascade token and head pruning,” in Proc. IEEE Int. Symp. High Perform. Comput. Archit. (HPCA), Feb. 2021, pp. 97–110

work page 2021

Show all 16 references

[6]

DOTA: Detect and omit weak attentions for scalable Transformer acceleration,

Z. Qu, L. Liu, F. Tu, Z. Chen, Y . Ding, and Y . Xie, “DOTA: Detect and omit weak attentions for scalable Transformer acceleration,” in Proc. 27th ACM Int. Conf. Architectural Support Program. Languages Operating Syst. (ASPLOPS), Feb. 2022, pp. 14–26

2022
[7]

FEAS: A Faster Event-Driven Accelerator Supporting Inhibitory Spiking Neural Net- work,

S. Li, L. Gong, T. Wang, C. Wang, and X. Zhou. “FEAS: A Faster Event-Driven Accelerator Supporting Inhibitory Spiking Neural Net- work,” Int. Symp. PPAP. Xi’an, Dec. 2021

2021
[8]

Fast and Energy-Efficient SNN Processor With Adaptive Clock/Event-Driven Computation Scheme and Online Learning,

S. Li, Z. Zhang, R. Mao, J Xiao, L. Chang, and J. Zhou. “Fast and Energy-Efficient SNN Processor With Adaptive Clock/Event-Driven Computation Scheme and Online Learning,” IEEE Trans. Circuits Syst I. pp. 1543-1552, April 2021

2021
[9]

FireFly: A High- Throughput Hardware Accelerator for Spiking Neural Networks With Efficient DSP and Memory Optimization,

J. Li, G. Shen, D. Zhao, Q. Zhang and Y . Zeng, “FireFly: A High- Throughput Hardware Accelerator for Spiking Neural Networks With Efficient DSP and Memory Optimization,” IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 31, no. 8, pp. 1178-1191, Aug. 2023

2023
[10]

Efficient Hard- ware Acceleration of Sparsely Active Convolutional Spiking Neural Networks,

J. Sommer, M. A. ¨Ozkan, O. Keszocze and J. Teich, “Efficient Hard- ware Acceleration of Sparsely Active Convolutional Spiking Neural Networks,” IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., vol. 41, no. 11, pp. 3767-3778, Nov. 2022

2022
[11]

Spike-driven transformer,

M. Yao, J. Hu, Z. Zhou, L. Yuan, Y . Tian, B. Xu, et.al, “Spike-driven transformer,” Adv. Neural Inf. Process. Syst. New Orleans, Dec. 2023

2023
[12]

Theoretical neuroscience: Computational and mathematical modeling of neural systems,

P. Dayan and L. Abbott, “Theoretical neuroscience: Computational and mathematical modeling of neural systems,” J. Cognit. Neurosci., vol. 15, no. 1, pp. 154–155, 2003

2003
[13]

A Unified Acceleration Solution Based on Deformable Network for Image Pixel Processing,

W. Mao, Z. Su, J. Luo and Z. Wang, “A Unified Acceleration Solution Based on Deformable Network for Image Pixel Processing,” IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 70, no. 9, pp. 3629-3633, Sept. 2023

2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [14]

An Event-driven Spiking Neural Network Accelerator with On-chip Sparse Weight,

Y . Kuang, X. Cui, C. Zou, Y . Zhong, Z. Dai and Z. Wang, “An Event-driven Spiking Neural Network Accelerator with On-chip Sparse Weight,” IEEE Int. Symp. on Circuits and Syst. (ISCAS), Austin, TX, pp. 3468-3472, 2022

work page 2022

[2] [15]

Skydiver: A Spiking Neural Network Accelerator Exploiting Spatio-Temporal Workload Balance,

Q. Chen, C. Gao, X. Fang and H. Luan, “Skydiver: A Spiking Neural Network Accelerator Exploiting Spatio-Temporal Workload Balance,” IEEE Trans. Computer-Aided Design Integr. Circuits Syst., vol. 41, no. 12, pp. 5732-5736, Dec. 2022

work page 2022

[3] [16]

Q. Chen, C. Sun, C. Gao, X. Fang and H. Luan, ”FrameFire: Enabling Efficient Spiking Neural Network Inference for Video Segmentation,” 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS), Hangzhou, China, 2023, pp. 1-5

work page 2023

[4] [1]

Handwritten digit recognition with a back-propagation network,

B.B. Le Cun, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel, “Handwritten digit recognition with a back-propagation network,” Proc. Adv. Neural Inf. Process. Syst., pp. 396-404, 1989

work page 1989

[5] [2]

Vaswani, N

A. Vaswani, N. Shazeer, N.Parmar. “Attention is All yo u Need,“ Proc. Adv. Neural Inf. Process. Syst., pp. 5998–6008, Jun. 2017

work page 2017

[6] [3]

Networks of spiking neurons: The third generation of neural network models,

W. Maass, “Networks of spiking neurons: The third generation of neural network models,” Neural Netw. vol. 10, no. 9, pp. 1659–1671, Dec. 1997

work page 1997

[7] [4]

Eyeriss: An Energy- Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,

Y . Chen, K. Tushar, J. S. Emer,and S. Vivienne, “Eyeriss: An Energy- Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,” IEEE Journal of Solid-State Circuits, vol. 52, no. 1, pp. 127-138, 2017

work page 2017

[8] [5]

SpAtten: Efficient sparse attention architecture with cascade token and head pruning,

H. Wang, Z. Zhang, and S. Han, “SpAtten: Efficient sparse attention architecture with cascade token and head pruning,” in Proc. IEEE Int. Symp. High Perform. Comput. Archit. (HPCA), Feb. 2021, pp. 97–110

work page 2021