Pith. sign in

REVIEW 4 major objections 6 minor 65 references

Bishop: Sparsified Bundling Spiking Transformers on Heterogeneous Cores with Error-Constrained Pruning

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dedicated hardware built around token-time bundles can make spiking transformer inference roughly six times faster and six times more energy-efficient than prior spiking neural network accelerators, without sacrificing accuracy.

desk verdict A genuinely new spiking-transformer accelerator design with a correct error-bound pruning argument, but the headline speedup is hard to audit because the PTB baseline configuration is reported too thinly. read the letter →

arxiv 2505.12281 v1 pith:UJBQYK4H submitted 2025-05-18 cs.NE

Boxun Xu , Yuxuan Yin , Vikram Iyer , Peng Li This is my paper · ORCID
classification cs.NE
keywords SpikingNeuralNetworksNeuromorphicAcceleratorsTransformersHW/SWCo-DesignToken-TimeBundleSparsity-AwareTrainingError-ConstrainedPruningHeterogeneousAccelerator
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that spiking transformers, which are accurate but computationally heavy models that combine spikes with attention, can be made practical on edge hardware if the architecture is designed around the structure of their data. Its central proposal is the Token-Time Bundle (TTB), a block of tokens across several time steps that becomes the basic unit of computation, exposing multi-bit weight reuse and bundle-level sparsity. Two algorithmic pieces reinforce this: a training loss that pushes firing sparsity to occur in whole bundles, and an error-bounded pruning method that removes spiking queries and keys whose contribution to the attention map is guaranteed small. In the authors' cycle-accurate, 28nm synthesis evaluation, the resulting Bishop accelerator runs an average of 5.91x faster and uses 6.11x less energy than a prior spiking systolic-array accelerator while preserving or improving accuracy across five datasets. A reader should care because spiking transformers have no dedicated accelerator before this design, yet their attention layers dominate total compute as token counts grow.

What carries the argument

Token-Time Bundle (TTB): a container that packs a set of $B_{S_n}$ tokens over $B_{S_t}$ time points for one output feature, serving as the unit of work mapped to each processing element. The TTB turns the binary spiking activations of a transformer's MLP, projection, and attention layers into structured chunks whose multi-bit weights can be reused both inside the bundle and across bundles. On top of this container sit three coordinated mechanisms: a stratifier that routes dense TTB workloads to a systolic dense core and sparse ones to a sparse core with flexible reduction, a Bundle Sparsity-Aware (BSA) training loss that minimizes the number of active TTBs, and Error-Constrained TTB Pruning (ECP), which drops spiking query and key bundle rows whose accumulated score is provably below a threshold. In the attention core, the binary Q and K values are combined by AND gates and accumulated, eliminating multi-bit multipliers and making attention-map computation the architecture's main source of gain.

What would settle it

Reproduce the comparison on an independent cycle-accurate simulator or silicon, matching Bishop and PTB in die area, memory bandwidth, and process node, and run the paper's five models with fixed, untuned BSA/ECP hyperparameters; if Bishop's end-to-end speedup over PTB falls substantially below 5.91x, or its energy improvement below 6.11x, the central claim is refuted.

Watch

Extended reading notes

Core claim

The paper claims to be the first dedicated hardware accelerator and HW/SW co-design framework for spiking transformers. Its central discovery is that the right unit of work for these models is a spatiotemporal block of binary spikes rather than a single spike or a single token-step: bundling tokens and time steps into a TTB exposes multi-bit weight reuse, and tagging each bundle as active or inactive turns inherent firing sparsity into a structured, skippable computation. On the algorithm side, the paper shows that a training penalty on the number of active bundles creates the structured sparsity the hardware needs, and that because spiking queries and keys are binary, pruning a row of query or key bundles can be done with a guaranteed error bound, which then cascades to shrink the attention map, value reads, and output writes. The measured consequence is an average 5.91x speedup and 6.11x energy-efficiency improvement over PTB across CIFAR10, CIFAR100, ImageNet-100, DVS-Gesture, plus a keyword-spotting task, with accuracy held or slightly improved.

Load-bearing premise

The headline numbers rest on the fairness of the evaluation: both Bishop and the PTB baseline are implemented in the authors' own cycle-accurate simulator with equal processing resources, so if the PTB configuration understates what a well-tuned general SNN accelerator could do on transformer workloads, or the simulator overestimates Bishop's data reuse, the 5.91x and 6.11x figures would shrink.

Editorial extensions

If this is right

  • Spiking transformer inference becomes practical on edge silicon: the synthesized 28nm Bishop design runs at 500 MHz with 2.96 mm² area and 627 mW peak power, while beating an edge GPU by an average of 299x in latency.
  • The attention bottleneck is the largest source of gain: with ECP, on average only 15.5% of attention computation remains, cutting attention-layer energy by 83.76% and latency by 43.92%.
  • Sparsity can be engineered at the bundle level: BSA training turns scattered spikes into whole inactive TTBs (52.2% of input features in one CIFAR10 model have no active TTBs), letting hardware skip them wholesale.
  • The accelerator works across different token/feature ratios: speedup over PTB is 6.71x on CIFAR10 where features dominate tokens ($D \gg N$) and 7.73x on ImageNet-100 where tokens dominate features ($N > D$).
  • Pruning binary queries and keys with error bounds can slightly improve accuracy while cutting work, e.g., +1.22% on CIFAR10 at an appropriate threshold.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the ECP threshold counts active bundles, the attention savings should grow superlinearly with sequence length, making the approach more valuable for long-sequence spiking transformers than for the short-token vision models evaluated.
  • Editorial inference: the reported gains use per-dataset choices of the BSA weight $\lambda$ and the ECP threshold $\theta_p$; a robustness test with these hyperparameters fixed across all datasets would show how much of the 5.91x/6.11x is intrinsic to the architecture rather than to per-task tuning.
  • Editorial inference: the error-bound argument relies on binary Q and K, not on LIF dynamics, so the same pruning scheme could transfer to other binary-activation attention models beyond spiking networks.
  • Editorial inference: the paper prunes values only indirectly, after query and key pruning; adding a direct bundle-level sparsity objective for V and the attention output could yield additional energy savings on top of what Bishop reports.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Bishop, a heterogeneous accelerator and HW/SW co-design framework for spiking transformers. The key idea is the Token-Time Bundle (TTB), which packs spiking activations across tokens and time points to enable multi-bit weight reuse and structured sparsity. Bishop comprises a stratifier that routes high-density workloads to a dense core and low-density workloads to a sparse core, a dedicated attention core that computes spiking attention with AND/accumulate (AAC) and select-accumulate (SAC) operations instead of multipliers, and an associated training pipeline (BSA) plus an error-constrained pruning method (ECP) that prunes binary queries, keys, and values with a claimed error bound. The paper evaluates Bishop on four vision datasets and one speech command dataset using a custom cycle-accurate simulator and 28nm RTL synthesis, reporting on average a 5.91x speedup and 6.11x energy-efficiency improvement over the PTB baseline while maintaining or improving model accuracy.

Significance. If the quantitative claims hold, Bishop would be a meaningful contribution to neuromorphic hardware and efficient spiking-transformer inference. The TTB abstraction is a clean way to expose spatiotemporal data reuse and structured sparsity, and the heterogeneous dense/sparse core decomposition is well motivated by workload profiling. The ECP insight that binary Q/K makes the attention-score magnitude bounded by the active-bundle count is simple, correct for the score matrix, and gives the pruning scheme a theoretical grounding that ANN attention pruning lacks. The paper also provides useful design-space exploration of stratification thresholds and bundle volumes, plus concrete 28nm synthesis results. The main risks are that the headline comparisons rest on a baseline whose configuration is incompletely reported and on an unpublished simulator, and that the per-dataset tuning of lambda and theta_p means the reported gains correspond to tuned models rather than a configuration-free architecture.

major comments (4)
  1. [6.1] The fairness of the headline comparison rests on the claim that PTB and Bishop are configured with the same number of PEs and the same register/compute resources, but the paper never reports PTB's PE count, array dimensions, dataflow, or memory partition. Section 6.1 states Bishop has a 512-PE dense core, a 512-PE attention core, up to 128 sparse-core units, and a 512-neuron spike generator, which implies more than 2x the PEs of a single 512-PE baseline. Please report PTB's full configuration, or provide an ablation that matches PE counts and compute resources, so that the 5.91x/6.11x gains can be attributed to the proposed mechanisms rather than to additional parallelism.
  2. [Abstract / 6.2] The abstract claims an average 5.91x speedup and 6.11x energy improvement over previous SNN accelerators, but Section 6.2 computes this average over only four of the five evaluated models; the fifth model (Google SC, 4.0x over PTB) is excluded, and including it would lower the average to about 5.5x. Please either include Model 5 in the reported average or explicitly state that the average is over the four vision models.
  3. [5.1] The 'well-defined error bound' claimed for ECP bounds only the pre-scaling attention score S (each pruned S entry is < theta_p,Q), not the final attention output Y = S * V or its LIF output. Since each Y entry sums over N token scores, with binary V the output error can be as large as theta_p,Q * N; the paper does not derive an end-to-end bound. Please state the exact error metric and provide the corresponding bound, or revise the claim to say the bound applies to the attention score matrix.
  4. [4.1] The BSA loss L_bsp in Eq. (10) is a sum of L0 bundle norms, which are non-differentiable, but the paper does not describe the surrogate gradient or relaxation used to backpropagate through this loss; the integration of ECP into the training pipeline ('ECP-aware training') is also not specified. Without these details the training procedure is not reproducible.
minor comments (6)
  1. [Title / figures] The title and several figure labels contain stray spaces and character artifacts (e.g., 'B undling', 'P runing', and 'uni000...' sequences in Figures 12 and 13); please proofread and regenerate the figures.
  2. [2.1] Equation (6) applies the scaling factor s to the score before multiplication with V; the attention-core description should clarify whether Mode 2's SAC units operate on the scaled S or the raw S, and where the shifter is applied.
  3. [6.1] The phrase 'same amount of register and compute resources' is ambiguous because Bishop's PEs use MUX/AND-based SAC/AAC operations while PTB's PEs use multipliers; please define the resource metric precisely.
  4. [6.3] The paper reports that ECP improves accuracy on several datasets (e.g., +1.24% on CIFAR10), but it does not discuss whether the reported accuracy numbers are averaged over multiple runs or seeds; please state the statistical significance.
  5. [6.1] The evaluation uses an unpublished analytic simulator for the dense/attention cores and STONNE for the sparse core; please clarify which parts are cycle-accurate and which are analytical, and whether the simulator has been validated against RTL.
  6. [7] Related work correctly mentions ref. [51] (Xu et al., 3D integration), but the novelty claim of being the 'first dedicated hardware accelerator' should be more carefully qualified relative to this prior spiking transformer accelerator, beyond the stated lack of token-time packing.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: Bishop's speedups are measured in simulation, ECP's error bound is presented as a derived property of binary Q/K arithmetic, and self-citations appear only as baselines or background, not as load-bearing premises.

full rationale

The paper's central claims are empirical hardware results, not predictions reverse-engineered from fitted constants. The 5.91x/6.11x comparisons come from an analytic cycle-accurate simulator and 28nm synthesis, with the PTB baseline implemented and synthesized in the same paper, so the comparison is not a self-citation standing in for evidence. BSA training optimizes TTB-level sparsity and then the accelerator skips inactive TTBs; this is the intended co-design mechanism and is evaluated by simulation rather than being true by construction. ECP's pruning is justified by the binary values of Q and K: the paper states that if a Q bundle row has fewer than θ_p active bundles, the corresponding scores in S=QK^T are bounded by θ_p. Whether or not the tightness of that bound is debatable, it is presented as a derived inequality from the binary arithmetic, not as the pruning threshold renamed as a result. The per-dataset λ and θ_p values are hyperparameters chosen for accuracy/sparsity trade-offs, and the accuracy and energy numbers are measured afterward, so they are not fitted inputs masquerading as predictions. Self-citations to PTB [27] and the authors' earlier 3D spiking-transformer work [51] are background and contrast; the novelty claim may be debatable given [51], but that is a scope/correctness concern, not circularity. Overall, no step in the derivation reduces to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central performance claims depend on several hand-set hyperparameters (BSA weight lambda, ECP threshold theta_p, stratification threshold theta_s, TTB bundle volume) and on standard modeling assumptions (LIF dynamics, spiking attention formulation, CACTI energy models). The TTB container is an invented abstraction with no external falsifiable handle outside the simulated architecture.

free parameters (4)
  • BSA sparsity regularization weight lambda = 1.0 (CIFAR10), 0.5 (CIFAR100), 0.3 (ImageNet-100), 1.0 (DVS-Gesture-128)
    Section 6.1 sets lambda per dataset to balance accuracy and TTB sparsity; the reported hardware gains depend on these values.
  • ECP pruning threshold theta_p (queries and keys) = 6 (CIFAR10, CIFAR100, ImageNet-100, Google SC), 10 (DVS-Gesture-128)
    Section 6.1 sets thresholds 'without compromising accuracy'; Fig 14 shows the accuracy/energy tradeoff is threshold-dependent, so the numbers are tuned per dataset.
  • Stratification threshold theta_s = per-layer values, e.g. roughly 50% dense / 50% sparse split for ImageNet-100 (Sec 6.4)
    Section 6.5.1: near-optimal EDP occurs when workloads balance; theta_s is a per-layer tuning parameter.
  • TTB bundle volume (BSt x BSn) = 4-8 chosen as near-optimal (e.g., (2,4), (4,14) tested)
    Section 6.5.2: bundle volume is swept and the range 4-8 selected; this choice affects weight reuse and memory traffic.
assumptions (5)
  • domain assumption LIF neuron model with leak and threshold reset (Eq 1-2)
    Standard spiking neuron model from [20]; underlies all spiking activations.
  • domain assumption Spiking self-attention formulation (Eq 3-8) with LIF before output projection and power-of-two scale s
    Adopted from [53, 64]; Bishop optimizes this exact computation, including LIF before the output projection.
  • standard math ECP error bound: S = QK^T with binary Q,K implies pruning rows with fewer than theta active bundles bounds all pruned scores by theta
    A mathematical consequence of binary matrices, stated in Sec 5.1 and Fig 7.
  • domain assumption Fidelity of simulation tools: CACTI 7.0, DDR4 model, STONNE, and the authors' analytic cycle-accurate simulator
    The reported energy/latency numbers inherit the assumptions of these tools (Sec 6.1).
  • domain assumption Accuracy preservation under BSA and ECP at the selected thresholds
    The claim 'while delivering higher accuracy' is empirical and depends on the trained models and the chosen lambda and theta_p (Sec 6.3, Fig 14).
invented entities (2)
  • Token-Time Bundle (TTB)
    purpose: Containers bundling BSn tokens over BSt time points to enable weight reuse and structured sparsity.
    A data-organization abstraction introduced in Sec 3; it has no external measurable signature beyond the simulated architecture.
  • Bishop hardware cores (stratifier, dense core, sparse core, attention core)
    purpose: Heterogeneous processing elements for stratified dense/sparse MLP and projection workloads plus attention.
    RTL synthesized in-house but no fabricated chip or open netlist; performance claims are simulation-based.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bishop: Sparsified Bundling Spiking Transformers on Heterogeneous Cores with Error-Constrained Pruning." pith.science (2026). https://pith.science/paper/UJBQYK4H

@misc{pith2026250512281,
  author       = {Pith},
  title        = {Pith review of: Bishop: Sparsified Bundling Spiking Transformers on Heterogeneous Cores with Error-Constrained Pruning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJBQYK4H}},
  note         = {Machine review of arXiv:2505.12281}
}
read the original abstract

We present Bishop, the first dedicated hardware accelerator architecture and HW/SW co-design framework for spiking transformers that optimally represents, manages, and processes spike-based workloads while exploring spatiotemporal sparsity and data reuse. Specifically, we introduce the concept of Token-Time Bundle (TTB), a container that bundles spiking data of a set of tokens over multiple time points. Our heterogeneous accelerator architecture Bishop concurrently processes workload packed in TTBs and explores intra- and inter-bundle multiple-bit weight reuse to significantly reduce memory access. Bishop utilizes a stratifier, a dense core array, and a sparse core array to process MLP blocks and projection layers. The stratifier routes high-density spiking activation workload to the dense core and low-density counterpart to the sparse core, ensuring optimized processing tailored to the given spatiotemporal sparsity level. To further reduce data access and computation, we introduce a novel Bundle Sparsity-Aware (BSA) training pipeline that enhances not only the overall but also structured TTB-level firing sparsity. Moreover, the processing efficiency of self-attention layers is boosted by the proposed Error-Constrained TTB Pruning (ECP), which trims activities in spiking queries, keys, and values both before and after the computation of spiking attention maps with a well-defined error bound. Finally, we design a reconfigurable TTB spiking attention core to efficiently compute spiking attention maps by executing highly simplified "AND" and "Accumulate" operations. On average, Bishop achieves a 5.91x speedup and 6.11x improvement in energy efficiency over previous SNN accelerators, while delivering higher accuracy across multiple datasets.

Figures

Figures reproduced from arXiv: 2505.12281 by the authors.

Figure 1
Figure 1. Bishop: the first accelerator architecture and SW/HW co-design framework dedicated to spiking transformers. TTBs empowers us to harness structured data reuse and firing spar￾sity inherent in key computations within the transformer across both time and space. Our Bishop architecture ❶ is heterogeneously comprised of a TTB stratifier, a TTB dense core, a TTB sparse core, and a TTB attention core. The TTB stratifier ef… view at source ↗
Figure 2
Figure 2. Spiking transformer model architecture with multi [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The FLOPs breakdown of a spiking transformer [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: (a) The conventional approach lacks parallel processing in time and space, processing each token at each timepoint [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 6
Figure 6. Figure 6: Spiking activities at the output projection layer in [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Error-constrained TT bundle pruning (ECP) for queries (Q), keys (K), attention scores (S), values (V), and outputs (Y) [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 9
Figure 9. Figure 9: The overall heterogeneous Bishop Architecture [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: (a) The original spiking workload 𝑋. (b) The strat￾ified dense workload 𝑋𝐷 and sparse workload 𝑋𝑆 ; weight indices are coordinated to dispatch 𝑊𝐷 and 𝑊𝑆 onto the TT￾Bundle dense/TT-Bundle core, respectively. Concurrently, the coordinated weights are passed from left t…
Figure 11
Figure 11. Figure 11: Normalized latency and energy comparison of [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 17
Figure 17. Figure 17: Power/area breakdown of the proposed Bishop accelerator. 6.5.2 Impact of TTB Bundle Volume. As shown in [PITH_FULL_IMAGE:figures/full_fig_p012_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 48 canonical work pages

  1. [1]

    A low power, fully event-based gesture recognition system,

    A. Amir, B. Taba, D. Berg, T. Melano, J. McKinstry, C. Di Nolfo, T. Nayak, A. An- dreopoulos, G. Garreau, M. Mendoza et al. , “A low power, fully event-based gesture recognition system, ” inProceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 7243–7252

    work page 2017

  2. [2]

    Spikingbert: Distilling bert to train spiking language models using implicit differentiation,

    M. Bal and A. Sengupta, “Spikingbert: Distilling bert to train spiking language models using implicit differentiation, ” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 10, 2024, pp. 10 998–11 006

    work page 2024

  3. [3]

    Cacti 7: New tools for interconnect exploration in innovative off-chip memories,

    R. Balasubramonian, A. B. Kahng, N. Muralimanohar, A. Shafiee, and V. Srinivas, “Cacti 7: New tools for interconnect exploration in innovative off-chip memories, ” ACM Trans. Archit. Code Optim. , vol. 14, no. 2, jun 2017. [Online]. Available: https://doi.org/10.1145/3085572

    doi:10.1145/3085572 2017

  4. [4]

    Pruning of Deep Spiking Neural Networks through Gradient Rewiring

    Y. Chen, Z. Yu, W. Fang, T. Huang, and Y. Tian, “Pruning of deep spiking neural networks through gradient rewiring, ”arXiv preprint arXiv:2105.04916, 2021

    work page Pith review arXiv 2021

  5. [5]

    Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning,

    S. S. Chowdhury, N. Rathi, and K. Roy, “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning, ” inEuropean Conference on Computer Vision . Springer, 2022, pp. 709–726

    work page 2022

  6. [6]

    Transformer-xl: Attentive language models beyond a fixed-length context,

    Z. Dai, Z. Yang, Y. Yang, J. G. Carbonell, Q. V. Le, and R. Salakhutdinov, “Transformer-xl: Attentive language models beyond a fixed-length context, ” in Annual Meeting of the Association for Computational Linguistics , 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:57759363

    work page 2019

  7. [7]

    Flashattention: Fast and memory- efficient exact attention with io-awareness,

    T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré, “Flashattention: Fast and memory- efficient exact attention with io-awareness, ” Advances in Neural Information Processing Systems, vol. 35, pp. 16 344–16 359, 2022

    work page 2022

  8. [8]

    ViTALiTy: Unifying low- rank and sparse approximation for vision transformer acceleration with a linear taylor attention,

    J. Dass, S. Wu, H. Shi, C. Li, Z. Ye, Z. Wang, and Y. Lin, “ViTALiTy: Unifying low- rank and sparse approximation for vision transformer acceleration with a linear taylor attention, ” in2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, pp. 415–428

    work page

Show all 65 references
  1. [9]

    Loihi: A neuromorphic manycore processor with on-chip learning,

    M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain et al., “Loihi: A neuromorphic manycore processor with on-chip learning, ”Ieee Micro, vol. 38, no. 1, pp. 82–99, 2018

    2018

  2. [10]

    A neural attention model for speech command recognition,

    D. C. De Andrade, S. Leo, M. L. D. S. Viana, and C. Bernkopf, “A neural attention model for speech command recognition, ”arXiv preprint arXiv:1808.08929, 2018

    2018 arXiv

  3. [11]

    Truenorth: Accelerating from zero to 64 million neurons in 10 years,

    M. V. DeBole, B. Taba, A. Amir, F. Akopyan, A. Andreopoulos, W. P. Risk, J. Kusnitz, C. O. Otero, T. K. Nayak, R. Appuswamy et al., “Truenorth: Accelerating from zero to 64 million neurons in 10 years, ”Computer, vol. 52, no. 5, pp. 20–29, 2019

    2019

  4. [12]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database, ” in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255

    2009

  5. [13]

    Compre- hensive snn compression using admm optimization and activity regularization,

    L. Deng, Y. Wu, Y. Hu, L. Liang, G. Li, X. Hu, Y. Ding, P. Li, and Y. Xie, “Compre- hensive snn compression using admm optimization and activity regularization, ” IEEE Transactions on Neural Networks and Learning Systems , vol. 34, no. 6, pp. 2791–2805, 2023

    2023

  6. [14]

    Comprehensive snn compression using admm optimization and activity regularization,

    ——, “Comprehensive snn compression using admm optimization and activity regularization, ”IEEE Transactions on Neural Networks and Learning Systems , vol. 34, no. 6, pp. 2791–2805, 2023

    2023

  7. [15]

    Temporal efficient training of spiking neural network via gradient re-weighting,

    S. Deng, Y. Li, S. Zhang, and S. Gu, “Temporal efficient training of spiking neural network via gradient re-weighting, ”arXiv preprint arXiv:2202.11946, 2022

    2022 arXiv

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , “An image is worth 16x16 words: Transformers for image recognition at scale, ” arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  9. [17]

    Adaptable butterfly accelerator for attention-based NNs via hardware and algorithm co-design,

    H. Fan, T. Chau, S. I. Venieris, R. Lee, A. Kouris, W. Luk, N. D. Lane, and M. S. Abdelfattah, “Adaptable butterfly accelerator for attention-based NNs via hardware and algorithm co-design, ” in 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, pp...

    2022

  10. [18]

    Taskfusion: An efficient transfer learning architecture with dual delta sparsity for multi-task natural language processing,

    Z. Fan, Q. Zhang, P. Abillama, S. Shoouri, C. Lee, D. T. Blaauw, H. Kim, and D. Sylvester, “Taskfusion: An efficient transfer learning architecture with dual delta sparsity for multi-task natural language processing, ” in Proceedings of the 50th Annual International Symposium ...

    2023

  11. [19]

    Deep residual learning in spiking neural networks,

    W. Fang, Z. Yu, Y. Chen, T. Huang, T. Masquelier, and Y. Tian, “Deep residual learning in spiking neural networks, ”Advances in Neural Information Processing Systems, vol. 34, pp. 21 056–21 069, 2021

    2021

  12. [20]

    Gerstner and W

    W. Gerstner and W. M. Kistler,Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press, 2002

    2002

  13. [21]

    Single-word speech recognition with convolutional neural networks on raw waveforms,

    P. Jansson, “Single-word speech recognition with convolutional neural networks on raw waveforms, ” 2018

    2018

  14. [22]

    20.5 c-transformer: A 2.6-18.1j/token homogeneous dnn-transformer/spiking-transformer processor with big-little network and implicit weight generation for large language models,

    S. Kim, S. Kim, W. Jo, S. Kim, S. Hong, and H.-J. Yoo, “20.5 c-transformer: A 2.6-18.1j/token homogeneous dnn-transformer/spiking-transformer processor with big-little network and implicit weight generation for large language models, ” in 2024 IEEE International Solid-State Ci...

    2024

  15. [23]

    Dynamic spike bundling for energy-efficient spiking neural networks,

    S. Krithivasan, S. Sen, S. Venkataramani, and A. Raghunathan, “Dynamic spike bundling for energy-efficient spiking neural networks, ” in 2019 IEEE/ACM In- ternational Symposium on Low Power Electronics and Design (ISLPED) , 2019, pp. 1–6

    2019

  16. [24]

    Enabling spike-based backpropagation for training deep neural network architectures,

    C. Lee, S. S. Sarwar, P. Panda, G. Srinivasan, and K. Roy, “Enabling spike-based backpropagation for training deep neural network architectures, ” Frontiers in neuroscience, vol. 14, p. 497482, 2020

    2020

  17. [25]

    Flexon: A flexible digital neuron for efficient spiking neural network simulations,

    D. Lee, G. Lee, D. Kwon, S. Lee, Y. Kim, and J. Kim, “Flexon: A flexible digital neuron for efficient spiking neural network simulations, ” in2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) . IEEE, 2018, pp. 275–288

    2018

  18. [26]

    Reconfigurable dataflow optimization for spatiotemporal spiking neural computation on systolic array accelerators,

    J.-J. Lee and P. Li, “Reconfigurable dataflow optimization for spatiotemporal spiking neural computation on systolic array accelerators, ” in 2020 IEEE 38th International Conference on Computer Design (ICCD) , 2020, pp. 57–64. 13 ISCA ’25, June 21–25, 2025, Tokyo, Japan Boxun ...

    2020

  19. [27]

    Parallel time batching: Systolic-array acceleration of sparse spiking neural computation,

    J.-J. Lee, W. Zhang, and P. Li, “Parallel time batching: Systolic-array acceleration of sparse spiking neural computation, ” in2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2022, pp. 317–330

    2022

  20. [28]

    Converting artificial neural networks to spik- ing neural networks via parameter calibration,

    Y. Li, S. Deng, X. Dong, and S. Gu, “Converting artificial neural networks to spik- ing neural networks via parameter calibration, ”arXiv preprint arXiv:2205.10121, 2022

    2022 arXiv

  21. [29]

    Differentiable spike: Rethink- ing gradient-descent for training spiking neural networks,

    Y. Li, Y. Guo, S. Zhang, S. Deng, Y. Hai, and S. Gu, “Differentiable spike: Rethink- ing gradient-descent for training spiking neural networks, ”Advances in Neural Information Processing Systems, vol. 34, pp. 23 426–23 439, 2021

    2021

  22. [30]

    H2learn: High-efficiency learning accelerator for high-accuracy spiking neural networks,

    L. Liang, Z. Qu, Z. Chen, F. Tu, Y. Wu, L. Deng, G. Li, P. Li, and Y. Xie, “H2learn: High-efficiency learning accelerator for high-accuracy spiking neural networks, ” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 41, no. 11, pp. 4782–4796, 2021

    2021

  23. [31]

    Activen: A scalable and flexibly- programmable event-driven neuromorphic processor,

    X. Liu, Z. Pu, P. Qu, W. Zheng, and Y. Zhang, “Activen: A scalable and flexibly- programmable event-driven neuromorphic processor, ” in2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 2024, pp. 1122– 1137

    2024

  24. [32]

    30.2 a 22nm 0.26 nw/synapse spike-driven spiking neural network processing unit using time-step-first dataflow and sparsity-adaptive in-memory computing,

    Y. Liu, Y. Ma, N. Shang, T. Zhao, P. Chen, M. Wu, J. Ru, T. Jia, L. Ye, Z. Wanget al., “30.2 a 22nm 0.26 nw/synapse spike-driven spiking neural network processing unit using time-step-first dataflow and sparsity-adaptive in-memory computing, ” in 2024 IEEE International Solid-...

    2024

  25. [33]

    Sanger: A co-design framework for enabling sparse attention using reconfigurable architecture,

    L. Lu, Y. Jin, H. Bi, Z. Luo, P. Li, T. Wang, and Y. Liang, “Sanger: A co-design framework for enabling sparse attention using reconfigurable architecture, ” in MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture , 2021, pp. 977–991

    2021

  26. [34]

    Networks of spiking neurons: The third generation of neural network models,

    W. Maass, “Networks of spiking neurons: The third generation of neural network models, ”Neural Networks, vol. 10, no. 9, pp. 1659–1671, 1997

    1997

  27. [35]

    Stellar: Energy-efficient and low- latency snn algorithm and hardware co-design with spatiotemporal computation,

    R. Mao, L. Tang, X. Yuan, Y. Liu, and J. Zhou, “Stellar: Energy-efficient and low- latency snn algorithm and hardware co-design with spatiotemporal computation, ” in 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2024, pp. 172–185

    2024

  28. [36]

    Stonne: Enabling cycle-level microarchitectural simulation for dnn inference accelerators,

    F. Muñoz-Matrínez, J. L. Abellán, M. E. Acacio, and T. Krishna, “Stonne: Enabling cycle-level microarchitectural simulation for dnn inference accelerators, ” in2021 IEEE International Symposium on Workload Characterization (IISWC) , 2021

    2021

  29. [37]

    Spinalflow: An architecture and dataflow tailored for spiking neural networks,

    S. Narayanan, K. Taht, R. Balasubramonian, E. Giacomin, and P.-E. Gaillardon, “Spinalflow: An architecture and dataflow tailored for spiking neural networks, ” in Proceedings of the ACM/IEEE 47th Annual International Symposium on Computer Architecture, ser. ISCA ’20. IEEE Pres...

    2020

  30. [38]

    SIGMA: A sparse and irregular GEMM accelerator with flexible interconnects for DNN training,

    E. Qin, A. Samajdar, H. Kwon, V. Nadella, S. Srinivasan, D. Das, B. Kaul, and T. Krishna, “SIGMA: A sparse and irregular GEMM accelerator with flexible interconnects for DNN training, ” in IEEE International Symposium on High Performance Computer Architecture, HPCA 2020, San D...

    2020

  31. [39]

    DOTA: detect and omit weak attentions for scalable transformer acceleration,

    Z. Qu, L. Liu, F. Tu, Z. Chen, Y. Ding, and Y. Xie, “DOTA: detect and omit weak attentions for scalable transformer acceleration, ” inProceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS ’22....

  32. [40]

    Stdp-based pruning of connections and weight quantization in spiking neural networks for energy-efficient recognition,

    N. Rathi, P. Panda, and K. Roy, “Stdp-based pruning of connections and weight quantization in spiking neural networks for energy-efficient recognition, ”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 38, no. 4, pp. 668–677, 2019

    2019

  33. [41]

    Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization,

    N. Rathi and K. Roy, “Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization, ”IEEE Transactions on Neural Networks and Learning Systems , vol. 34, no. 6, pp. 3174–3182, 2021

    2021

  34. [42]

    Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation,

    N. Rathi, G. Srinivasan, P. Panda, and K. Roy, “Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation, ” arXiv preprint arXiv:2005.01807, 2020

    2005 arXiv

  35. [43]

    Towards spike-based machine intelligence with neuromorphic computing,

    K. Roy, A. Jaiswal, and P. Panda, “Towards spike-based machine intelligence with neuromorphic computing, ”Nature, vol. 575, p. 607–617, 2019

    2019

  36. [44]

    Towards spike-based machine intelligence with neuromorphic computing,

    ——, “Towards spike-based machine intelligence with neuromorphic computing, ” Nature, vol. 575, no. 7784, pp. 607–617, 2019

    2019

  37. [45]

    Going deeper in spiking neural networks: Vgg and residual architectures,

    A. Sengupta, Y. Ye, R. Wang, C. Liu, and K. Roy, “Going deeper in spiking neural networks: Vgg and residual architectures, ”Frontiers in neuroscience, vol. 13, p. 95, 2019

    2019

  38. [46]

    Nebula: A neuromorphic spin-based ultra-low power architecture for snns and anns,

    S. Singh, A. Sarma, N. Jao, A. Pattnaik, S. Lu, K. Yang, A. Sengupta, V. Narayanan, and C. R. Das, “Nebula: A neuromorphic spin-based ultra-low power architecture for snns and anns, ” in2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). IEEE, 20...

    2020

  39. [47]

    Skipper: Enabling efficient snn training through activation- checkpointing and time-skipping,

    S. Singh, A. Sarma, S. Lu, A. Sengupta, M. T. Kandemir, E. Neftci, V. Narayanan, and C. R. Das, “Skipper: Enabling efficient snn training through activation- checkpointing and time-skipping, ” inProceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitectu...

    2023

  40. [48]

    Spatten: Efficient sparse attention architecture with cascade token and head pruning,

    H. Wang, Z. Zhang, and S. Han, “Spatten: Efficient sparse attention architecture with cascade token and head pruning, ” in2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2021, pp. 97–110

    2021

  41. [49]

    Masked spiking trans- former,

    Z. Wang, Y. Fang, J. Cao, Q. Zhang, Z. Wang, and R. Xu, “Masked spiking trans- former, ” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 1761–1771

    2023

  42. [50]

    Speech commands: A dataset for limited-vocabulary speech recogni- tion,

    P. Warden, “Speech commands: A dataset for limited-vocabulary speech recogni- tion, ”arXiv preprint arXiv:1804.03209, 2018

    2018 arXiv

  43. [51]

    Spiking transformer hardware accelerators in 3d integration,

    B. Xu, J. Hwang, P. Vanna-iampikul, S. K. Lim, and P. Li, “Spiking transformer hardware accelerators in 3d integration, ” inProceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design , 2024, pp. 1–9

    2024

  44. [52]

    Deep residual spiking neural network for keyword spotting in low-resource settings

    Q. Yang, Q. Liu, and H. Li, “Deep residual spiking neural network for keyword spotting in low-resource settings. ” inInterspeech, 2022, pp. 3023–3027

    2022

  45. [53]

    Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next-generation neuromorphic chips,

    M. Yao, J. Hu, T. Hu, Y. Xu, Z. Zhou, Y. Tian, B. Xu, and G. Li, “Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next-generation neuromorphic chips, ”arXiv preprint arXiv:2404.03663, 2024

    2024 arXiv

  46. [54]

    Spike-driven transformer,

    M. Yao, J. Hu, Z. Zhou, L. Yuan, Y. Tian, B. Xu, and G. Li, “Spike-driven transformer, ” arXiv preprint arXiv:2307.01694, 2023

    2023 arXiv

  47. [55]

    Loas: Fully temporal-parallel dataflow for dual-sparse spiking neural networks,

    R. Yin, Y. Kim, D. Wu, and P. Panda, “Loas: Fully temporal-parallel dataflow for dual-sparse spiking neural networks, ” in2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 2024, pp. 1107–1121

    2024

  48. [56]

    Mint: Multiplier-less integer quantization for energy efficient spiking neural networks,

    R. Yin, Y. Li, A. Moitra, and P. Panda, “Mint: Multiplier-less integer quantization for energy efficient spiking neural networks, ” in2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 2024, pp. 830–835

    2024

  49. [57]

    Sata: Sparsity-aware training accelerator for spiking neural networks,

    R. Yin, A. Moitra, A. Bhattacharjee, Y. Kim, and P. Panda, “Sata: Sparsity-aware training accelerator for spiking neural networks, ”IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems , vol. 42, no. 6, pp. 1926–1938, 2023

    1926

  50. [58]

    Vitcod: Vision transformer acceleration via dedicated algorithm and accelerator co-design,

    H. You, Z. Sun, H. Shi, Z. Yu, Y. Zhao, Y. Zhang, C. Li, B. Li, and Y. Lin, “Vitcod: Vision transformer acceleration via dedicated algorithm and accelerator co-design, ” in IEEE International Symposium on High- Performance Computer Architecture, HPCA 2023, Montreal, QC, Canada...

    2023

  51. [59]

    Castling-ViT: Compressing self-attention via switching towards linear-angular attention at vision transformer inference,

    H. You, Y. Xiong, X. Dai, B. Wu, P. Zhang, H. Fan, P. Vajda, and Y. C. Lin, “Castling-ViT: Compressing self-attention via switching towards linear-angular attention at vision transformer inference, ” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)...

    2023

  52. [60]

    Spiking transformers for event-based single object tracking,

    J. Zhang, B. Dong, H. Zhang, J. Ding, F. Heide, B. Yin, and X. Yang, “Spiking transformers for event-based single object tracking, ” in2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022, pp. 8791–8800

    2022

  53. [61]

    Temporal spike sequence learning via backpropagation for deep spiking neural networks,

    W. Zhang and P. Li, “Temporal spike sequence learning via backpropagation for deep spiking neural networks, ” inAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, virtual, 2020

    2020

  54. [62]

    Alisa: Accelerating large language model inference via sparsity-aware kv caching,

    Y. Zhao, D. Wu, and J. Wang, “Alisa: Accelerating large language model inference via sparsity-aware kv caching, ”arXiv preprint arXiv:2403.17312, 2024

    2024 arXiv

  55. [63]

    Going deeper with directly-trained larger spiking neural networks,

    H. Zheng, Y. Wu, L. Deng, Y. Hu, and G. Li, “Going deeper with directly-trained larger spiking neural networks, ” inProceedings of the AAAI conference on artificial intelligence, vol. 35, no. 12, 2021, pp. 11 062–11 070

    2021

  56. [64]

    Spikformer: When spiking neural network meets transformer,

    Z. Zhou, Y. Zhu, C. He, Y. Wang, S. YAN, Y. Tian, and L. Yuan, “Spikformer: When spiking neural network meets transformer, ” inThe Eleventh International Conference on Learning Representations , 2023

    2023

  57. [65]

    Spikegpt: Generative pre-trained language model with spiking neural networks,

    R.-J. Zhu, Q. Zhao, G. Li, and J. K. Eshraghian, “Spikegpt: Generative pre-trained language model with spiking neural networks, ” 2023. 14

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.