Pith. sign in

REVIEW 3 major objections 6 minor 71 references

Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Spike activations in SNNs cluster into binary patterns, and Phi converts most online multiply-accumulates into precomputed lookups, reporting 3.45x speedup and 4.93x energy efficiency over prior accelerators.

desk verdict Phi's pattern-plus-correction decomposition is a genuine new idea for SNN acceleration, but the headline speedups depend on counting L1 PWP fetches as free, and the paper's own numbers don't support 5.8x over bit sparsity. read the letter →

arxiv 2505.10909 v1 pith:SONCESMX submitted 2025-05-16 cs.AR cs.AI

Chiyue Wei , Bowen Duan , Cong Guo , Jingyang Zhang , Qingyue Song , Hai "Helen" Li , Yiran Chen This is my paper · ORCID
classification cs.ARcs.AI
keywords spikingneuralnetworkshierarchicalsparsitypattern-basedcomputingSNNacceleratoralgorithm-hardwareco-designpattern-awarefine-tuningbit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Spiking neural networks communicate through binary 0/1 spikes, and current accelerators save work only by skipping zeros. This paper claims that the ones are not randomly placed: rows of each activation tile cluster around a small set of repeating binary patterns. Phi exploits that structure by decomposing every activation matrix into a Level 1 matrix whose rows are pre-calibrated patterns (computed once offline with the weights) and a Level 2 correction matrix of rare +1/−1 elements, so that adding the two levels recovers the original activation exactly. Only the correction matrix is computed online, and it is far sparser than the raw spike matrix: 96.80% element sparsity versus 83.63% bit sparsity. The paper reports a 3.45x speedup and a 4.93x energy-efficiency gain over the best previous SNN accelerator, with a lossless variant that exactly reproduces the original outputs.

What carries the argument

The working identity is the exact decomposition $A = A^{(1)} + A^{(2)}$, where $A^{(1)}$ is the vector-sparse Level 1 matrix (each row is one of $q=128$ pre-calibrated binary patterns of length $k=16$, or all zeros) and $A^{(2)}$ is the element-sparse Level 2 matrix of values 0, +1, and -1. Then $AW = A^{(1)}W + A^{(2)}W$, and $A^{(1)}W$ is a lookup of precomputed Pattern-Weight Products. The patterns come from a k-means clustering run on calibration activations with Hamming distance as the metric, with one-hot and all-zero rows filtered out; pattern assignment chooses, per activation row, the pattern with the smallest Hamming distance, and falls back to raw bit sparsity if the correction would be denser than the original row. A pattern-aware fine-tuning regularizer then trains activations toward the calibrated patterns, raising Level 2 sparsity. The hardware side is a pattern matcher that finds the best pattern per row, an L1 processor that fetches and accumulates PWPs with a prefetcher that drops unreferenced patterns, and an L2 processor that packs sparse rows and sums them with a reconfigurable adder tree.

What would settle it

Run the Phi pipeline on a binary activation workload in which each row is a distinct pattern, with no reuse within a tile, using the tile size $k=16$ and $q=128$ from the paper's setup, and measure end-to-end cycles and DRAM energy: if the Level 1 processor becomes dominated by fetching a unique Pattern-Weight Product per row, the claimed 3.45x speedup and 4.93x energy gain would not survive, and the paper's own random-matrix stress test would show only the 2.7x theoretical bound.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is empirical: SNN activations form distinct clusters rather than a uniform 0/1 fog, and those clusters can be captured by very few prototypes. With a tile width of $k=16$ and $q=128$ patterns per tile, the Level 1 patterns cover almost all activation density, leaving a Level 2 correction matrix of density $3.05\%$ compared with $16.37\%$ for the original bit matrix. Because the Level 1 result is the product of a pattern row with weight rows, it can be precomputed offline as Pattern-Weight Products; the online arithmetic is just the sparse correction product plus accumulation of the fetched products. The bidirectional corrections use both $+1$ (activation has a 1 where the pattern has 0) and $-1$ (activation has a 0 where the pattern has 1), preserving exactness. The measured result is a $3.45\times$ runtime speedup and $4.93\times$ energy-efficiency improvement over the state-of-the-art SNN accelerator, and a theoretical $4.5\times$ speedup over bit sparsity and $38\times$ over the dense baseline. Skipping the pattern-aware fine-tuning gives a lossless mode that reproduces the original SNN outputs exactly.

Load-bearing premise

The gains rest on the assumption that retrieving a precomputed pattern-weight product is much cheaper than computing that product online; if real activations reuse each pattern only rarely, the retrieval becomes the dominant cost, and the paper's own numbers show even after prefetching the weight-plus-product traffic sits at roughly three times the dense baseline.

Editorial extensions

If this is right

  • With the lossless variant, an SNN can be served exactly with the same algorithmic output while replacing most multiply-accumulates by memory lookups; accuracy-critical deployments do not have to trade correctness for speed.
  • The same 128-pattern, width-16 configuration worked across convolutional SNNs and spiking transformers on vision and language tasks, with theoretical speedups of 4.5x over bit sparsity and 38x over dense; the pattern structure is not specific to one architecture.
  • Because even random binary matrices give a 2.7x theoretical speedup over bit sparsity, the two-level decomposition is a general property of binary matrices, so the optimizer applies to any binary activation stream, not just trained SNNs.
  • PAFT trades a small accuracy decrease for 1.26x more runtime improvement, so the framework spans a lossless point and a lossy point; the lambda hyperparameter chooses where to land.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same two-level split transfers to any binary matrix product; a practical next step is applying it to bit-sliced quantized DNNs, where each bitplane is already a binary matrix, so each plane could use its own calibrated pattern set.
  • The reported 27.73% PWP utilization means most stored patterns in a tile are never referenced; reducing q per tile or sharing patterns across tiles directly attacks the dominant DRAM traffic term and is a cheaper lever than adding more on-chip buffer.
  • A stress test not run in the paper: measure whether pattern IDs recur across consecutive timesteps of the same layer; if they do, the pattern matcher could cache those IDs and cut the preprocessing energy that is currently amortized at 75.5x.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Phi, an algorithm-hardware co-design framework that exploits regular patterns in binary SNN spike activations. The activation matrix is decomposed exactly into a Level 1 matrix of pre-defined patterns (processed by offline-computed Pattern-Weight Products) and a Level 2 element-wise sparse correction matrix with entries in {+1,-1}. A k-means-based calibration stage selects patterns, and an optional pattern-aware fine-tuning (PAFT) step trades a small accuracy loss for higher Level 2 sparsity. A dedicated accelerator is described with a preprocessor, L1/L2 processors, and a spiking neuron array. The authors report a 3.45x speedup and 4.93x energy-efficiency improvement over the Stellar SNN accelerator, and theoretical speedups of 4.5-5.8x over bit sparsity.

Significance. The central decomposition A = P + C is exact by construction, so the lossless version of Phi (without PAFT) correctly reproduces the original SNN outputs. The idea of converting activation-pattern matches into offline lookups is novel and potentially applicable beyond SNNs, e.g., to bit-sliced DNNs. The paper ships a substantial evaluation across six models and several datasets, includes a SystemVerilog implementation with 28nm synthesis, and uses a design-space exploration for tile size, pattern count, and buffer capacity. If the performance and energy claims survive scrutiny, this would be a meaningful advance in SNN acceleration. However, the current manuscript contains load-bearing inconsistencies in the operation-count model and in the headline numbers that prevent full confidence.

major comments (3)
  1. [Table 4 / Sec. 5.6 / Sec. 4.4] The theoretical speedups in Table 4 are computed as Bit Density divided by the sum of L2:+1 and L2:-1 densities (e.g., 10.6/1.8 = 5.8x for VGG16/CIFAR100). This counts all Level 1 work as free. The L1 processor described in Sec. 4.4 retrieves a Pattern-Weight Product and accumulates it into the L1 partial-sum buffer for each assigned pattern row; that is one N-wide vector accumulation per assigned row, which is the same type and cost as one accumulation in a bit-sparse baseline. The paper does not state whether the simulator's 242.80 GOP/s figure includes these Level 1 accumulations, so the connection between the op-count model and the measured 3.45x/4.93x claims cannot be checked. Moreover, under the assignment rule of Sec. 3.1, rows not assigned a pattern are retained in the Level 2 matrix with their original bit density; the reported L2 densities (e.g., 1.8% total for VGG16/CIFAR100) and the 49.34% pattern-index sparsity reported in Sec. 4.4 appear mutually inconsistent, since the unassigned rows alone would contribute roughly 0.49 x 10.6% = 5.2% to L2 density. The theoretical speedup and the simulator results should be recomputed with an explicit online-operation count that includes Level 1 pattern-row accumulations and unassigned rows.
  2. [Sec. 5.3.1 / Sec. 5.3.2 / Table 2] The text-reported speedups and energy ratios do not match Table 2. For throughput, the text says Phi outperforms Stellar by 3.45x, but Table 2 gives 242.80/58.11 = 4.18x. For energy efficiency, the text reports 4.93x over Stellar and 13.48x over PTB, but Table 2 gives 285.81/61.71 = 4.63x and 285.81/10.65 = 26.84x. The abstract repeats the 3.45x/4.93x figures. Since the headline claims are not reproducible from the paper's own table, the authors should identify which configuration produced which numbers and correct the inconsistency.
  3. [Sec. 5.1 (baseline methodology)] The comparison against Stellar, which is the key reference for the headline claims, is made by relying on 'results reported in the paper' rather than by simulating Stellar in the same framework. With different array sizes, dataflows, memory hierarchies, or workload assumptions, the speedup and energy ratios can change substantially. The authors should either re-implement Stellar in their simulator or state explicitly the Stellar configuration (array shape, dataflow, buffer capacities, frequency, technology, and memory model) used for comparison, and justify that it is compatible with the Phi evaluation.
minor comments (6)
  1. [Abstract / Sec. 1] The phrase 'novelpattern-based' appears in the abstract and in Section 1 with a missing space; it should read 'novel pattern-based'.
  2. [Fig. 8] The annotation values on top of the bars (e.g., 42.2, 15.32x, 1) are not defined in the caption or in the text; please state what these numbers represent.
  3. [Table 2] The PTB row lists area as '-' without explanation; please either fill in the value or explicitly state that it is not available.
  4. [Sec. 3.2] The calibration workflow says a small subset of the training data is used, but the subset size and sampling procedure are not reported; please specify them.
  5. [Sec. 5.1] The text says models are obtained from open-source repositories 'on GitHub in the paper,' but no URLs or repository identifiers are given; please add the references.
  6. [Fig. 12(b)] The caption 'Effect of reducing memory traffic' does not identify which bars correspond to the prefetcher configuration; the figure and text should be clearer about the three traffic categories shown.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the lossless decomposition is correct by construction and performance claims are measured against external baselines, with only a definitional theoretical-speedup caveat.

full rationale

Phi's derivation is self-contained: patterns are calibrated by k-means on a training subset and applied to test activations, PWPs are genuinely precomputed offline, and the L2 {+1,-1} residual guarantees that the sum of Level 1 and Level 2 activations equals the original activation matrix, making the no-PAFT variant lossless by construction. The headline 3.45x and 4.93x numbers come from a cycle/energy simulator benchmarked against external accelerators (Eyeriss, SpinalFlow, PTB, SATO, Stellar), not from the paper's own definitions. PAFT's sparsity gain is the explicit optimization objective (minimizing the Hamming-distance regularization R in Sec. 3.3), so reporting improved L2 sparsity after PAFT is a disclosed design trade-off rather than a hidden prediction. The 'theoretical 4.5x speedup' in Sec. 5.6/Table 4 is transparently a density ratio and is labeled 'theoretical'; it is not the evidence for the simulated speedups. Self-citations (Prosperity [60], Transitive Array [20]) appear only as related work/discussion and are not load-bearing. A real but non-circular concern is that Sec. 4.4 describes the L1 processor as performing PWP 'retrieval and accumulation' online, so the free-L1 accounting used in the theoretical speedup may overstate op reduction; this is an accounting/correctness question, not a circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on standard architecture-simulation practice plus two domain assumptions: calibration transfer from training to test activations, and the memory-for-compute trade-off for pattern-weight products. No new physical entities are introduced; the only invention is a data representation called phi sparsity.

free parameters (6)
  • pattern_count_per_partition q = 128
    Chosen by design-space exploration to balance Level 2 density against memory access growth (Fig. 7c); the speedup numbers depend on this choice.
  • partition_width k = 16
    Selected because k=16 gives lowest total density and balanced L1/L2 workload (Fig. 7a-b).
  • PAFT_learning_rate = 1e-5 to 1e-3 (searched)
    Searched per model to balance accuracy and pattern alignment; directly influences the reported accuracy and sparsity trade-off.
  • PAFT_lambda = 0.01 to 1 (searched)
    Regularization weight in the PAFT loss; controls the trade-off between accuracy and Level 2 sparsity.
  • PAFT_epochs = 5
    Empirically chosen fine-tuning duration to achieve the reported 1.26x runtime improvement.
  • onchip_buffer_sizes = 240KB total (4KB pack, 16KB weight, 64KB PWP, 28KB pattern ID, 128KB PSum)
    Chosen to balance DRAM power against buffer power and area in Fig. 7d.
assumptions (5)
  • domain assumption Training-data activation distribution represents test distribution for pattern calibration
    Sec. 3.2 states that a small subset of training data suffices for calibration; evidence is t-SNE of one layer of one model (Fig. 9a).
  • standard math k-means centers rounded to binary are near-optimal patterns for maximizing Level 2 sparsity
    Used in Sec. 3.2; Hamming distance directly ties cluster centers to Level 2 sparsity.
  • domain assumption The LIF neuron model captures the evaluated SNNs
    Sec. 2.1 states the work focuses on LIF; results on Spikformer and SpikeBERT depend on this model choice.
  • domain assumption The simulator models cycle-level behavior and power accurately
    Sec. 5.1 describes an unshipped simulator based on a widely adopted framework, so its accuracy cannot be independently checked.
  • domain assumption Surrogate gradient or implicit differentiation is used to backpropagate through spike activations in PAFT
    Sec. 3.3 says PAFT uses backpropagation, but the exact mechanism for binary activations is unspecified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks." pith.science (2026). https://pith.science/paper/SONCESMX

@misc{pith2026250510909,
  author       = {Pith},
  title        = {Pith review of: Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SONCESMX}},
  note         = {Machine review of arXiv:2505.10909}
}
abstract

Spiking Neural Networks (SNNs) are gaining attention for their energy efficiency and biological plausibility, utilizing 0-1 activation sparsity through spike-driven computation. While existing SNN accelerators exploit this sparsity to skip zero computations, they often overlook the unique distribution patterns inherent in binary activations. In this work, we observe that particular patterns exist in spike activations, which we can utilize to reduce the substantial computation of SNN models. Based on these findings, we propose a novel \textbf{pattern-based hierarchical sparsity} framework, termed \textbf{\textit{Phi}}, to optimize computation. \textit{Phi} introduces a two-level sparsity hierarchy: Level 1 exhibits vector-wise sparsity by representing activations with pre-defined patterns, allowing for offline pre-computation with weights and significantly reducing most runtime computation. Level 2 features element-wise sparsity by complementing the Level 1 matrix, using a highly sparse matrix to further reduce computation while maintaining accuracy. We present an algorithm-hardware co-design approach. Algorithmically, we employ a k-means-based pattern selection method to identify representative patterns and introduce a pattern-aware fine-tuning technique to enhance Level 2 sparsity. Architecturally, we design \textbf{\textit{Phi}}, a dedicated hardware architecture that efficiently processes the two levels of \textit{Phi} sparsity on the fly. Extensive experiments demonstrate that \textit{Phi} achieves a $3.45\times$ speedup and a $4.93\times$ improvement in energy efficiency compared to state-of-the-art SNN accelerators, showcasing the effectiveness of our framework in optimizing SNN computation.

Figures

Figures reproduced from arXiv: 2505.10909 by the authors.

Figure 1
Figure 1. Activation visualization for (a) normal distribution, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Phi Sparsity. 2.4.2 Hardware Challenges. Dynamic Activation Patterns. Al￾though patterns can be pre-determined, activations are dynamically generated during inference. This dynamic nature poses fundamen￾tal challenges for hardware in the efficient implementation of Phi. Therefore, activation identification and processing must be per￾formed at runtime with high throughput without impeding compu￾tational processing. S… view at source ↗
Figure 3
Figure 3. Phi architecture overview. 4 Phi Architecture Design 4.1 Overview We present Phi, a novel architecture dedicated to our proposed Phi sparsity, which efficiently addresses the challenges in exploiting Phi sparsity. Our Phi architecture dynamically generates Phi sparsity infor￾mation and efficiently processes two distinct levels of sparsity. The overview of the Phi architecture is shown in [PITH_FULL_IMAGE:figures/fu… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: (a) Matcher 1, the operation 1011−1110 = 0, -1, 0, 1 produces a candidate sparse map. Popcount operations [2] are then performed on the difference to count nonzero elements in the candidate sparse map and on the original activation to establish a baseline case (represe…
Figure 5
Figure 5. Figure 5: Level 2 Processor P0 W3 W4 P1 W5 W9 P2 W6 Row1 Row3 Row4 p w 0 w p 3 4 1 w w 5 p w 9 2 6 1 3 Index Row ID Label 32 + + … + (a) Level 2 Sparse Pack Reconfigurable Adder Tree PE PE PE PE PE PE PE 2-1 2-1 4 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Reconfigurable Adder Tree row and the row indices, which are essential for the L2 processor’s operation. Packer. The packer optimizes row packing density while mini￾mizing buffer access conflicts, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Design space exploration [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Speedup (normalized by Spiking Eyeriss) and energy (normalized by [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 11
Figure 11. Figure 11: PAFT accuracy results. 0 0.8 1.6 2.4 3.2 4 Weight Traffic VGG16 ResNet18 Spikformer SDT SpikeBERT SpikingBERT Geomean Norm. Dense Memory Traffic Norm. Phi (w/o Prefetch) Memory Traffic Nor. Phi (w Prefetch) Memory Traffic 9.0 9.0 9.0 9.0 9.0 9.0 9.0 0 0.4 0.8 1.2 1.6 …
Figure 10
Figure 10. Figure 10: Element density with and without PAFT. area to minimize memory access. Due to converting computation into memory access, the buffer ranks as the highest power consumer, respectively. Additionally, the area and power associated with the L2 processor are larger than tho…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 46 canonical work pages

  1. [1]

    IEEE Standard for SystemVerilog: Unified Hardware Design, Specification, and Verification Language

    2017. IEEE Standard for SystemVerilog: Unified Hardware Design, Specification, and Verification Language. https://doi.org/10.1109/IEEESTD.2018.8299595

    work page arXiv 2017

  2. [2]

    Amogh Agrawal, Akhilesh Jaiswal, Deboleena Roy, Bing Han, Gopalakrishnan Srinivasan, Aayush Ankit, and Kaushik Roy. 2019. Xcel-RAM: Accelerating Binary Neural Networks in High-Throughput SRAM Compute Arrays.IEEE Transactions on Circuits and Systems I: Regular Papers66, 8 (2019), 3064–3076

    work page 2019

  3. [3]

    Jorge Albericio, Alberto Delmás, Patrick Judd, Sayeh Sharify, Gerard O’Leary, Roman Genov, and Andreas Moshovos. 2017. Bit-pragmatic deep neural network computing. InProceedings of the 50th annual IEEE/ACM international symposium on microarchitecture. 382–394

    work page 2017

  4. [4]

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    work page

  5. [5]

    Malyaban Bal and Abhronil Sengupta. 2024. Spikingbert: Distilling bert to train spiking language models using implicit differentiation. InProceedings of the AAAI conference on artificial intelligence, Vol. 38. 10998–11006

    work page 2024

  6. [6]

    A. N. Burkitt. 2006. A Review of the Integrate-and-Fire Neuron Model: I. Homo- geneous Synaptic Input.Biological Cybernetics95, 1 (2006), 1–19

    work page 2006

  7. [7]

    Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, and Olivier Temam. 2014. Diannao: A small-footprint high-throughput accel- erator for ubiquitous machine-learning.ACM SIGARCH Computer Architecture News42, 1 (2014), 269–284

    work page 2014

  8. [8]

    Yuzong Chen, Jian Meng, Jae-sun Seo, and Mohamed S Abdelfattah. 2024. BBS: Bi- directional bit-level sparsity for deep learning acceleration. In2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 551–564

    work page 2024

Show all 71 references
  1. [9]

    Yu-Hsin Chen, Tushar Krishna, Joel S Emer, and Vivienne Sze. 2016. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks.IEEE journal of solid-state circuits52, 1 (2016), 127–138

    2016

  2. [10]

    William J Dally and Brian Towles. 2001. Route packets, not wires: on-chip inteconnection networks. InProceedings of the 38th annual design automation conference. 684–689

    2001

  3. [11]

    Alexey Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929(2020)

    2020 arXiv

  4. [12]

    R David Evans, Lufei Liu, and Tor M Aamodt. 2020. Jpeg-act: accelerating deep learning via transform-based lossy compression. In2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). IEEE, 860–873

    2020

  5. [13]

    Mingyu Gao, Xuan Yang, Jing Pu, Mark Horowitz, and Christos Kozyrakis. 2019. Tangram: Optimized coarse-grained dataflow for scalable nn accelerators. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating S...

    2019

  6. [14]

    Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. 2014. Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press

    2014

  7. [15]

    Samanwoy Ghosh-Dastidar and Hojjat Adeli. 2009. Spiking neural networks. International journal of neural systems19, 04 (2009), 295–308

    2009

  8. [16]

    Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. InProceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 249–256

    2010

  9. [17]

    Cong Guo, Feng Cheng, Zhixu Du, James Kiessling, Jonathan Ku, Shiyu Li, Ziru Li, Mingyuan Ma, Tergel Molom-Ochir, Benjamin Morris, et al. 2025. A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models.IEEE Circuits and Systems Magazine25, 1 (202...

    2025

  10. [18]

    Cong Guo, Bo Yang Hsueh, Jingwen Leng, Yuxian Qiu, Yue Guan, Zehuan Wang, Xiaoying Jia, Xipeng Li, Minyi Guo, and Yuhao Zhu. 2020. Accelerating sparse dnn models without hardware-support via tile-wise sparsity. InSC20: International Conference for High Performance Computing, N...

    2020

  11. [19]

    Cong Guo, Jiaming Tang, Weiming Hu, Jingwen Leng, Chen Zhang, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. 2023. Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization. InProceedings of the 50th Annual International Symposium on Com...

    2023

  12. [20]

    Cong Guo, Chiyue Wei, Jiaming Tang, Bowen Duan, Song Han, Hai Li, and Yiran Chen. 2025. Transitive Array: An Efficient GEMM Accelerator with Result Reuse. arXiv preprint arXiv:2504.16339(2025)

    2025 arXiv

  13. [21]

    Cong Guo, Fengchen Xue, Jingwen Leng, Yuxian Qiu, Yue Guan, Weihao Cui, Quan Chen, and Minyi Guo. 2024. Accelerating sparse dnns based on tiled gemm. IEEE Trans. Comput.(2024)

    2024

  14. [22]

    Cong Guo, Chen Zhang, Jingwen Leng, Zihan Liu, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. 2022. Ant: Exploiting adaptive numerical data type for low-bit deep neural network quantization. In2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 1414–1433

    2022

  15. [23]

    Hassaballah, Saleh Omran, and Youssef B

    M. Hassaballah, Saleh Omran, and Youssef B. Mahdy. 2008. A Review of SIMD Multimedia Extensions and their Usage in Scientific and Engineering Applications. Comput. J.51, 6 (2008), 630–649

    2008

  16. [24]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. InProceedings of the IEEE international conference on computer vision. 1026–1034

    2015

  17. [25]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778

    2016

  18. [26]

    Weiming Hu, Haoyan Zhang, Cong Guo, Yu Feng, Renyang Guan, Zhendong Hua, Zihan Liu, Yue Guan, Minyi Guo, and Jingwen Leng. 2025. M-ANT: Efficient Low- bit Group Quantization for LLMs via Mathematically Adaptive Numerical Type. In2025 IEEE International Symposium on High Perfor...

    2025

  19. [27]

    Weizhe Hua, Yuan Zhou, Christopher De Sa, Zhiru Zhang, and G Edward Suh

  20. [28]

    Sagar Imambi, Kolla Bhanu Prakash, and GR Kanagachidambaresan. 2021. Py- Torch.Programming with TensorFlow: solution for edge computing applications (2021), 87–104

    2021

  21. [29]

    Eugene M Izhikevich. 2003. Simple model of spiking neurons.IEEE Transactions on neural networks14, 6 (2003), 1569–1572

    2003

  22. [30]

    Herve Jégou, Matthijs Douze, and Cordelia Schmid. 2011. Product Quantization for Nearest Neighbor Search.IEEE Transactions on Pattern Analysis and Machine Intelligence33, 1 (2011), 117–128

    2011

  23. [31]

    Alireza Khodamoradi, Kristof Denolf, and Ryan Kastner. 2021. S2n2: A fpga accelerator for streaming spiking neural networks. InThe 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 194–205

    2021

  24. [32]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  25. [33]

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning.nature 521, 7553 (2015), 436–444

    2015

  26. [34]

    Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pen- nington, and Jascha Sohl-Dickstein. 2017. Deep neural networks as gaussian processes.arXiv preprint arXiv:1711.00165(2017)

    2017 arXiv

  27. [35]

    Jeong-Jun Lee, Wenrui Zhang, and Peng Li. 2022. Parallel time batching: Systolic- array acceleration of sparse spiking neural computation. In2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 317–330

    2022

  28. [36]

    Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. 2017. Cifar10- dvs: an event-stream dataset for object classification.Frontiers in neuroscience11 (2017), 309

    2017

  29. [37]

    Shang Li, Zhiyuan Yang, Dhiraj Reddy, Ankur Srivastava, and Bruce Jacob. 2020. DRAMsim3: A cycle-accurate, thermal-capable DRAM simulator.IEEE Computer Architecture Letters19, 2 (2020), 106–109

    2020

  30. [38]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and accel- eration.Proceedings of Machine Learning and Systems6 (2...

    2024

  31. [39]

    Fangxin Liu, Wenbo Zhao, Zongwu Wang, Yongbiao Chen, Tao Yang, Zhezhi He, Xiaokang Yang, and Li Jiang. 2022. Sato: spiking neural network acceleration via temporal-oriented dataflow and architecture. InProceedings of the 59th ACM/IEEE Design Automation Conference. 1105–1110

    2022

  32. [40]

    Hang Lu, Liang Chang, Chenglong Li, Zixuan Zhu, Shengjian Lu, Yanhuan Liu, and Mingzhe Zhang. 2021. Distilling bit-level sparsity parallelism for general purpose deep learning acceleration. InMICRO-54: 54th Annual IEEE/ACM Inter- national Symposium on Microarchitecture. 963–976

    2021

  33. [41]

    Changze Lv, Tianlong Li, Jianhan Xu, Chenxi Gu, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, and Xuanjing Huang. 2023. Spikebert: A language spik- former trained with two-stage knowledge distillation from bert.arXiv preprint arXiv:2308.15122(2023)

    2023 arXiv

  34. [42]

    Ruixin Mao, Lin Tang, Xingyu Yuan, Ye Liu, and Jun Zhou. 2024. Stellar: Energy-Efficient and Low-Latency SNN Algorithm and Hardware Co-Design with Spatiotemporal Computation. In2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 172–185

    2024

  35. [43]

    Naveen Muralimanohar, Rajeev Balasubramonian, and Norman P Jouppi. 2009. CACTI 6.0: A tool to model large caches.HP laboratories27 (2009), 28

    2009

  36. [44]

    Surya Narayanan, Karl Taht, Rajeev Balasubramonian, Edouard Giacomin, and Pierre-Emmanuel Gaillardon. 2020. Spinalflow: An architecture and dataflow tailored for spiking neural networks. In2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). IEEE, 349–362

    2020

  37. [45]

    Pearson, A

    Martin J. Pearson, A. G. Pipe, B. Mitchinson, K. Gurney, C. Melhuish, I. Gilhespy, and M. Nibouche. 2007. Implementing Spiking Neural Networks for Real-Time Signal-Processing and Control Applications: A Model-Validated FPGA Approach. IEEE Transactions on Neural Networks18, 5 (...

    2007

  38. [46]

    Eric Qin, Ananda Samajdar, Hyoukjun Kwon, Vineet Nadella, Sudarshan Srini- vasan, Dipankar Das, Bharat Kaul, and Tushar Krishna. 2020. Sigma: A sparse and irregular gemm accelerator with flexible interconnects for dnn training. In 2020 IEEE International Symposium on High Perf...

    2020

  39. [47]

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors.nature323, 6088 (1986), 533–536

    1986

  40. [48]

    Hardik Sharma, Jongse Park, Naveen Suda, Liangzhen Lai, Benson Chau, Joon Kyung Kim, Vikas Chandra, and Hadi Esmaeilzadeh. 2018. Bit fusion: Bit- level dynamically composable architecture for accelerating deep neural network. In2018 ACM/IEEE 45th Annual International Symposium...

    2018

  41. [49]

    Man Shi, Vikram Jain, Antony Joseph, Maurice Meijer, and Marian Verhelst

  42. [50]

    Ajay Shrestha and Ausif Mahmood. 2019. Review of Deep Learning Algorithms and Architectures.IEEE Access7 (2019), 53040–53065

    2019

  43. [51]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556(2014)

    2014 arXiv

  44. [52]

    In2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)

    BitWave: Exploiting column-based bit-level sparsity for deep learning acceleration. In2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 732–746

  45. [53]

    Christoph Stöckl and Wolfgang Maass. 2021. Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. Nature Machine Intelligence3, 3 (2021), 230–238

    2021

  46. [54]

    Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S. Emer. 2017. Efficient Processing of Deep Neural Networks: A Tutorial and Survey.Proc. IEEE105, 12 (2017), 2295–2329

    2017

  47. [55]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. InProceedings of the 2013 conference on empirical methods in natural language ...

    2013

  48. [56]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research9, 11 (2008)

    2008

  49. [57]

    Gregory K Wallace. 1991. The JPEG still picture compression standard.Commun. ACM34, 4 (1991), 30–44

    1991

  50. [58]

    Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothée Masquelier, and Anthony Maida. 2019. Deep learning in spiking neural networks. Neural networks111 (2019), 47–63

    2019

  51. [59]

    Yang Wang, Chen Zhang, Zhiqiang Xie, Cong Guo, Yunxin Liu, and Jingwen Leng

  52. [60]

    Chiyue Wei, Cong Guo, Feng Cheng, Shiyu Li, Hao Frank Yang, Hai Helen Li, and Yiran Chen. 2025. Prosperity: Accelerating Spiking Neural Networks via Product Sparsity. In2025 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 806–820

    2025

  53. [61]

    Hanrui Wang, Zhekai Zhang, and Song Han. 2021. Spatten: Efficient sparse atten- tion architecture with cascade token and head pruning. In2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 97–110

    2021

  54. [62]

    Xingrun Xing, Zheng Zhang, Ziyi Ni, Shitao Xiao, Yiming Ju, Siqi Fan, Yequan Wang, Jiajun Zhang, and Guoqi Li. 2024. SpikeLM: Towards General Spike- Driven Language Modeling via Elastic Bi-Spiking Mechanisms.arXiv preprint arXiv:2406.03287(2024)

    2024 arXiv

  55. [63]

    Yue Yang, Xumeng Zhang, Pei Chen, Lingli Cheng, Yanting Ding, Chao Li, Jie Yu, and Qi Liu. 2024. Fully Integrated Memristive Hodgkin-Huxley Neurons With Homeostatic Plasticity.IEEE Electron Device Letters45, 11 (2024), 2225–2228

    2024

  56. [64]

    Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. 2024. Spike-driven transformer.Advances in neural information processing systems36 (2024)

    2024

  57. [65]

    Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference.arXiv preprint arXiv:1704.05426(2017)

    2017 arXiv

  58. [66]

    Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. 2022. Spikformer: When spiking neural network meets trans- former.arXiv preprint arXiv:2209.15425(2022)

    2022 arXiv

  59. [67]

    Rui-Jie Zhu, Malu Zhang, Qihang Zhao, Haoyu Deng, Yule Duan, and Liang-Jian Deng. 2024. TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks.IEEE Transactions on Neural Networks and Learning Systems(2024), 1–14

    2024

  60. [69]

    MD Zeiler. 2014. Visualizing and Understanding Convolutional Networks. In European conference on computer vision/arXiv, Vol. 1311

    2014

  61. [2019]

    InProceedings of the 52nd Annual IEEE/ACM international sympo- sium on microarchitecture

    Boosting the performance of CNN accelerators with dynamic fine-grained channel gating. InProceedings of the 52nd Annual IEEE/ACM international sympo- sium on microarchitecture. 139–150

  62. [2021]

    In2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)

    Dual-side sparse tensor core. In2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA). IEEE, 1083–1095

  63. [2024]

    Quarot: Outlier-free 4-bit inference in rotated llms.Advances in Neural Information Processing Systems37 (2024), 100213–100240

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.