Pith. sign in

REVIEW 5 major objections 6 minor 29 references

Trimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Heterogeneous quantization cuts spiking vision transformer weights to an average of about 2-4 effective bits per layer, cutting storage and estimated energy by roughly an order of magnitude while keeping accuracy within 1%.

desk verdict A credible idea for heterogeneous quantization of spiking transformers, but the reported numbers are internally inconsistent and the equations are sloppy; needs major revision before the quantitative claims can be trusted. read the letter →

arxiv 2412.05505 v1 pith:2BGTUN2F submitted 2024-12-07 cs.NE cs.AI

Boxun Xu , Yufei Song , Peng Li This is my paper · ORCID
classification cs.NEcs.AI
keywords spikingneuralnetworksvisiontransformersheterogeneousquantizationlayer-wisearchitecturesearchpower-of-twoedgedeploymentenergyefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a spiking vision transformer does not need uniform 32-bit weights to stay accurate. It introduces SpikeHQ, which chooses for every layer, through a differentiable search, one of a small set of quantizers: 2-bit or 4-bit uniform, 2-bit or 4-bit power-of-two, or 32-bit float. On DVS Gesture and CIFAR10-DVS, the method keeps accuracy within about 1 percentage point of the full-precision model while reporting storage compression of 8.71x to 10.19x and energy reduction of 5.69x to 10.2x under an analytical hardware model. If these results hold, large spiking transformers become far more plausible targets for edge and neuromorphic deployment, where memory and energy budgets are tight.

What carries the argument

The load-bearing machinery is SpikeHQ, a bi-level optimization that treats each layer's quantization choice as an architectural parameter. A supernet (a network containing all candidate layer variants in parallel) is built by summing, for each composite layer, the outputs of all candidate quantized versions weighted by a Gumbel-softmax selection probability, a differentiable approximation to sampling from a discrete distribution, so the discrete search over quantization schemes is relaxed into a continuous one. The outer optimization updates selection probabilities to minimize a total loss $L = L_{\text{acc}} \times L_{\text{hw}}^\beta$, where the hardware term accumulates per-layer costs from an analytic 45nm CMOS model, while the inner optimization trains the weights under the selected quantization using a straight-through estimator, which treats the rounding operator as its own gradient in the backward pass. The final per-layer scheme is read off as the highest-probability choice, and the power-of-two option is hardware-relevant because it lets multiplications be replaced by shifts and adds.

What would settle it

Run the quantized spiking transformer produced by SpikeHQ on a real 45nm-class edge accelerator or neuromorphic chip and compare measured inference energy against the model's prediction. If the measured energy does not drop by roughly 5.69x-10.2x relative to the full-precision model, or if DRAM access or control overheads dominate the measured cost, the central energy claim would be refuted.

Watch

Extended reading notes

Core claim

The central discovery is that quantization precision is not a uniform resource across a spiking transformer: different layers tolerate very different bit widths, and a search over per-layer choices can exploit that heterogeneity. The paper claims that a spiking transformer trained with quantization-aware heterogeneous quantization retains essentially full accuracy while running at an average effective resolution of roughly 2 to 4 bits per weight, with storage and estimated energy reduced by about an order of magnitude. It further reports an empirical pattern in which query and key projections tolerate lower precision than value and output projections, while the tokenizer's first convolution and the classification head are the layers that most need higher precision. These layer-level findings are presented as evidence about where spiking transformers carry excess precision.

Load-bearing premise

The load-bearing premise is that the analytical 45nm hardware-cost model in Eq. (7), with the per-operation energy constants from Table 1, correctly predicts the energy and storage cost of running the quantized spiking transformer; all reported energy reductions are computed from this model, not from hardware measurements.

Editorial extensions

If this is right

  • If correct, spiking transformer inference on 45nm-class edge hardware can run at roughly 2-4 effective bits per weight, with storage and energy costs reduced by about an order of magnitude.
  • Accuracy losses under 1% on DVS Gesture and CIFAR10-DVS indicate that quantization-aware heterogeneous search can outperform naive uniform low-bit quantization for spiking transformers.
  • The reported layer-level precision pattern suggests that future spiking transformer designs can concentrate precision in tokenizer, classification head, and value/output projections while using cheaper 2-bit or power-of-two operations elsewhere.
  • The shift-and-add implementation of power-of-two quantized layers connects spiking transformer compression to low-power digital accelerator designs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The paper's finding that the tokenizer dominates post-quantization energy suggests that redesigning the input tokenizer, not just quantizing it, may be the next lever for further savings.
  • Inference: The experiments cover small two-block spiking transformers, so the same search should be tested on deeper or wider models before claiming the compression ratios scale.
  • Inference: Because the energy numbers come from an analytic model rather than measured chips, a hardware measurement study would be the natural next check on whether the claimed 5.69x-10.2x reductions survive real DRAM access, routing, and control overheads.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces SpikeHQ, a differentiable neural architecture search method for heterogeneous layer-wise quantization of spiking vision transformers. The search space includes 32-bit floating point, 2-bit and 4-bit uniform quantization, and 2-bit and 4-bit power-of-two quantization. The optimization objective combines the cross-entropy loss with an analytic hardware cost model, and the discrete selection is relaxed via Gumbel softmax in a bi-level optimization. Experiments are reported on DVS128, CIFAR10-DVS, and N-Caltech101, with tables and figures giving accuracy, average bit width, storage, and energy under different values of the hardware-loss weight beta.

Significance. The topic is timely and the direction is promising: if the method works as claimed, it would be a useful step toward deploying spiking vision transformers on resource-constrained edge devices. The paper is the first to propose heterogeneous (mixed-scheme, mixed-precision) quantization search for spiking neural networks, and it provides an interesting empirical observation that query/key projections tolerate lower precision than value/output projections, and that the tokenizer and classification head need higher precision. The analytic cost model is explicitly stated and the per-operation energy constants are taken from an external source, which is a strength. However, the paper's headline quantitative claims are internally inconsistent: the abstract's numbers do not match Table 2, the energy/storage reductions are evaluated with the same cost model that the search optimizes, and the quantizer equations as printed are not mathematically well-formed. These issues currently prevent the contribution from being considered established.

major comments (5)
  1. [Abstract and Section 1 vs. Table 2] The abstract and Section 1 claim 'less than a 1% accuracy drop' on DVS Gesture and CIFAR10-DVS with average effective resolutions of 3.14-3.67 bits (abstract) or 2.33-3.24 bits (Section 1). Table 2 contradicts this: DVS128 has accuracy drops of -1.04 at beta=1.5 and -1.39 at beta=2.0, and CIFAR10-DVS has drops of -1.20 at beta=1.0 and -1.10 at beta=2.0. The rows that do satisfy the '<1%' condition have average bit widths of 3.07/2.33 (DVS128, beta=0.5/1.0) and 3.24/2.52/2.85 (CIFAR10-DVS, beta=0.5/1.5/1.0), not 3.14-3.67. The abstract's central accuracy claim is therefore not supported by the paper's own data.
  2. [Abstract and Section 1 vs. Table 2] The energy and storage reduction numbers in the abstract do not correspond to any single configuration in Table 2. The abstract states energy reductions of 5.69x, 8.72x, and 10.2x while maintaining accuracies of 85.3%, 97.57%, and 80.4%. To match those accuracies, the relevant rows are N-Caltech101 beta=1.5 (energy 17.57%, i.e., 5.69x), DVS128 beta=1.0 (energy 11.61%, i.e., 8.61x), and CIFAR10-DVS beta=1.5 (energy 11.48%, i.e., 8.71x). The value 10.2x appears only at DVS128 beta=2.0, whose accuracy is 96.87%, and at CIFAR10-DVS beta=2.0, whose accuracy is 80.10%. No row yields 8.72x. The claimed storage compression of 8.71x-10.19x also does not match Table 2, where storage percentages range from 6.58% to 11.85%, corresponding to 8.44x to 15.20x. The headline numbers are assembled from different rows and misrepresent the results.
  3. [Section 2.2, Eqs. (1)-(6)] The quantizer definitions are not mathematically well-formed. In Eq. (1), the clamping condition is applied to the raw value theta rather than to the scaled-and-shifted value theta/s + z, and the zero-point z is never defined; as written, the mapping from floating point to an integer grid is not computable. In Eq. (3), max(theta) is treated as a scalar even though theta is a tensor, and the expression lacks an absolute value, so the scale factor for negative weights is undefined. In Eq. (5), the interval bounds use expressions like 2^{-2b-1+1}, whose meaning is unclear and which do not define a valid partition of the real line. Because the entire method is built on these quantizers, the paper cannot be reproduced from the equations alone.
  4. [Section 3.1, Eq. (7)] The hardware cost model charges C_dram per stored bit (#bits), not per memory access, and it does not model DRAM access granularity, activation memory traffic, on-chip SRAM behavior, control overhead, or data movement between compute units and memory. All reported energy reductions are computed from this model, which is also the L_hw term that the search minimizes. Consequently, the reported 'energy reduction' is in part an echo of the optimization objective rather than an independently measured result. The paper should either validate the model against a hardware simulator or measurements, or explicitly reframe all energy claims as 'estimated cost under the assumptions of Eq. (7)', with a sensitivity analysis to the per-operation constants.
  5. [Section 4, Table 2] The experimental results are reported without error bars or multiple seeds. The claimed 'less than 1%' accuracy preservation relies on differences that are comparable to typical run-to-run variance in training spiking transformers, so a single run per configuration is insufficient to establish that accuracy is maintained within 1%. Additionally, the storage percentage column is exactly avg_bits/32 (e.g., 2.33/32 = 7.28%), so the storage compression ratio is a direct arithmetic restatement of the selected bit widths rather than a validated memory-system result; this should be acknowledged when presenting storage savings.
minor comments (6)
  1. [Abstract and Section 1] The abstract states an average effective resolution of 3.14-3.67 bits, while Section 1 states 2.33-3.24 bits; these two ranges should be reconciled, and both should match the rows in Table 2 that actually support the 'less than 1% drop' claim.
  2. [Table 2 and Section 4.2] Section 4.2 says the method reduces energy and storage 'to as low as 9.73% (10.42x) and 6.58% (15.20x)'. In Table 2, the 9.73% energy cell is for CIFAR10-DVS beta=2.0 and corresponds to 10.28x, not 10.42x; the 6.58% storage cell is for the same row and corresponds to 15.20x. The '10.42x' does not match any cell in the table.
  3. [Section 2.2.1] The notation in Eq. (1)-(2) is inconsistent: the zero-point z is introduced but never used in the dequantization formula, and the relationship between theta_int, theta, s, and z should be stated explicitly with a concrete numerical example.
  4. [Section 3.2, Eq. (14)] The total loss L_total = L_acc * L_hw^beta mixes a cross-entropy loss with a hardware cost raised to a power beta; the units and scaling behavior are not discussed, and the choice of beta values (0.5, 1.0, 1.5, 2.0) is not justified. A brief explanation of why a multiplicative product is used, rather than a weighted sum, would improve readability.
  5. [References and Figures] Several references are incomplete or inconsistently cited (e.g., 'Vaswani et al.' and 'Zhou et al.' without years in the text), and Figure 4 is difficult to read because the pie-chart labels and percentages are small; the figure should be enlarged or redrawn.
  6. [Reproducibility] The paper does not mention whether code or trained models will be released, and it does not report the random seeds, exact Gumbel temperature schedule, or the search-space details beyond the five candidate configurations. Providing these details would be necessary for independent verification.

Circularity Check

2 steps flagged · score 6.0 of 10

Partial circularity: reported energy and storage savings are computed from the same hardware objective being minimized and from the chosen bit widths, rather than from independent hardware measurement.

  1. fitted input called prediction [Section 3.3, Eqs. (13)-(14); Section 4.2, Table 2]
    "The total hardware loss of the supernet is obtained by summing up all layer-wise hardware costs based on (7) weighted by the corresponding quantization selection probabilities: Lhw = ... (13) We define the total loss to be minimized based on a combination of cross-entropy model accuracy loss Lacc and hardware loss Lhw with a user-specified β trading off between the two: Ltotal = Lacc × Lβhw. (14) ... SpikeHQ significantly reduces the energy and storage requirements of the full precision models to as low as 9.73% (10.42×) and 6.58% (15.20×), respectively."

    The energy percentages reported in Table 2 are computed with the same analytic hardware cost C_HW of Eq. (7). Eq. (13) aggregates C_HW into L_hw, and Eq. (14) makes L_hw the hardware term of the loss minimized by the architecture search. The layer-wise bit widths are fitted to minimize exactly this quantity, and the reported 5.69x-10.2x energy reduction is then read back from that same objective. The decrease is therefore an echo of the optimized objective rather than an independent hardware measurement; the external constants from You et al. anchor the model, but the empirical force of the energy claim is by construction.

  2. self definitional [Abstract; Section 4.2; Table 2]
    "It attains a model compression rate of 8.71x-10.19x for standard floating-point spiking transformers. ... The average number of bits used for weight parameters is reduced from 32 to 2.16bits."

    The compression range 8.71x-10.19x is exactly 32 divided by the average bit widths (3.67 to 3.14 bits), and the storage percentages in Table 2 are the average bit counts divided by 32 (e.g., 2.33/32 = 7.28%). Thus the storage-reduction claim is a restatement of the layer-wise bit widths selected by the search, not an independently validated storage outcome. It is definitional accounting from the chosen bit widths rather than a separate derived result.

full rationale

The accuracy comparisons use external neuromorphic datasets and full-precision baselines from Zhou et al., so the central feasibility claim of preserving accuracy after heterogeneous quantization retains independent benchmark content. No load-bearing self-citation chain or imported uniqueness theorem is present; the authors' prior spiking-transformer papers are background. The main circularity is confined to the hardware-overhead headline: quantization configurations are optimized by minimizing L_hw in Eqs. (13)-(14), and the same C_HW model of Eq. (7) is then used to report the achieved energy reduction, a fitted-input-called-prediction pattern. The 8.71x-10.19x compression figure is likewise the arithmetic inverse of the selected average bit count, hence definitional. Separately, the abstract's 'less than 1% accuracy drop' is contradicted by Table 2 entries such as -1.04 and -1.39 on DVS128 and -1.20 on CIFAR10-DVS, and the quantizer definitions in Eqs. (1)-(6) contain correctness issues, but those are validity/consistency concerns rather than circularity. Because the independent accuracy benchmark remains and the hardware constants are external, the circularity is partial, giving a score of 6.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The compressed model's performance is evaluated against a full-precision baseline, but the energy and storage claims rest entirely on the external op-cost table and the per-layer quantization choices selected by the search. The main choices not justified independently are the hardware cost model, the candidate quantizer set, and the beta and temperature schedules.

free parameters (3)
  • beta (hardware loss weight) = swept over 0, 0.5, 1.0, 1.5, 2.0
    Scales the hardware cost term in L_total (Eq. 14); all reported accuracy, energy, and storage points are indexed by this hand-chosen trade-off parameter.
  • Gumbel temperature lambda schedule = not specified numerically
    Initial and final temperatures and the decay schedule are described only qualitatively, and the selected quantization schemes depend on this schedule.
  • Quantization candidate set = FP32, 2b uniform, 4b uniform, 2b power-of-two, 4b power-of-two
    The search space is a design choice; the achievable average bit widths and compression rates are constrained to this set.
assumptions (5)
  • domain assumption The energy cost table from You et al. (Table 1) accurately models real hardware energy on 45nm CMOS for spiking transformer inference.
    Used directly in Eq. (7) to compute every storage and energy saving; if these constants or the op-count model are wrong, the energy reductions change.
  • domain assumption Spiking transformer weight distributions are approximately symmetric, motivating power-of-two quantization.
    Section 2.2.2 states this; the power-of-two quantizer relies on it and the search may select it.
  • domain assumption Spiking activations are robust to weight imprecision, so low-bit quantization preserves accuracy.
    Section 2.2 uses the binary activation property as justification; this is not proven for the tested models.
  • standard math Gumbel softmax and the straight-through estimator give valid gradients for discrete quantization choices.
    Standard techniques accepted from the literature; the paper applies them without extending them.
  • ad hoc to paper The uniform quantizer in Eq. (1) and power-of-two quantizer in Eq. (5) are correctly implemented as intended in the experiments.
    The formulas as printed are malformed or ambiguous; experiments assume a specific implementation that the text does not uniquely specify.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Trimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search." pith.science (2026). https://pith.science/paper/2BGTUN2F

@misc{pith2026241205505,
  author       = {Pith},
  title        = {Pith review of: Trimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2BGTUN2F}},
  note         = {Machine review of arXiv:2412.05505}
}
read the original abstract

Spiking Neural Networks (SNNs) are amenable to deployment on edge devices and neuromorphic hardware due to their lower dissipation. Recently, SNN-based transformers have garnered significant interest, incorporating attention mechanisms akin to their counterparts in Artificial Neural Networks (ANNs) while demonstrating excellent performance. However, deploying large spiking transformer models on resource-constrained edge devices such as mobile phones, still poses significant challenges resulted from the high computational demands of large uncompressed high-precision models. In this work, we introduce a novel heterogeneous quantization method for compressing spiking transformers through layer-wise quantization. Our approach optimizes the quantization of each layer using one of two distinct quantization schemes, i.e., uniform or power-of-two quantification, with mixed bit resolutions. Our heterogeneous quantization demonstrates the feasibility of maintaining high performance for spiking transformers while utilizing an average effective resolution of 3.14-3.67 bits with less than a 1% accuracy drop on DVS Gesture and CIFAR10-DVS datasets. It attains a model compression rate of 8.71x-10.19x for standard floating-point spiking transformers. Moreover, the proposed approach achieves a significant energy reduction of 5.69x, 8.72x, and 10.2x while maintaining high accuracy levels of 85.3%, 97.57%, and 80.4% on N-Caltech101, DVS-Gesture, and CIFAR10-DVS datasets, respectively.

Figures

Figures reproduced from arXiv: 2412.05505 by the authors.

Figure 1
Figure 1. Heterogeneous Quantization Compression on Spiking Vision Transformers. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. SpikeHQ: proposed heterogeneous quantization by bi-level neural architecture search. whose size depends on b: θint =    0, θ < 0 ⌊ θ s ⌉ + z, 0 ≤ θ ≤ 2 b − 1 2 b − 1, θ > 2 b − 1 (1) θ ≈ ˆθ = s(θint − z). (2) 2.2.2 Power-of-Two Quantizer Power-of-two quantization is symmetric with a power-of-two scale factor s. Scaling with s can be efficiently realized by performing bit-shifting k times. It is a suitable quant… view at source ↗
Figure 3
Figure 3. Evolution of quantization scheme selection probabilities for the last layer of spiking transformers [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Breakdown of normalized energy consumption and storage overhead of spiking transformers across [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Distribution of Weight parameter value before(top figures) and after(bottom figures) applying [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 23 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Estimating or propagating gradients through stochastic neurons

    Yoshua Bengio. Estimating or propagating gradients through stochastic neurons. CoRR, abs/1305.2982, 2013

    arXiv 2013

  3. [3]

    Davies et al

    M. Davies et al. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro 2018

    work page 2018

  4. [4]

    Loihi: A neuromorphic manycore processor with on-chip learning

    Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38 0 (1): 0 82--99, 2018

    work page 2018

  5. [5]

    Truenorth: Accelerating from zero to 64 million neurons in 10 years

    Michael V DeBole, Brian Taba, Arnon Amir, Filipp Akopyan, Alexander Andreopoulos, William P Risk, Jeff Kusnitz, Carlos Ortega Otero, Tapan K Nayak, Rathinakumar Appuswamy, et al. Truenorth: Accelerating from zero to 64 million neurons in 10 years. Computer, 52 0 (5): 0 20--29, 2019

    work page 2019

  6. [6]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR 2021

    work page 2021

  7. [7]

    Auto-nba: Efficient and effective search over the joint space of networks, bitwidths, and accelerators

    Yonggan Fu, Yongan Zhang, Yang Zhang, David Cox, and Yingyan Lin. Auto-nba: Efficient and effective search over the joint space of networks, bitwidths, and accelerators. In ICML 2021

    work page 2021

  8. [8]

    Garrick, G

    O. Garrick, G. Orchard, N. Le Thi, X. Xu, X. Chen, C. Posch, and N.V. Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. In Frontiers in Neuroscience 2015

    work page 2015

Show all 29 references
  1. [9]

    Categorical reparameterization with gumbel-softmax

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In ICLR 2017

    2017

  2. [10]

    Parallel time batching: Systolic-array acceleration of sparse spiking neural computation

    Jeong-Jun Lee, Wenrui Zhang, and Peng Li. Parallel time batching: Systolic-array acceleration of sparse spiking neural computation. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pp.\ 317--330, 2022. doi:10.1109/HPCA53966.2022.00031

    2022

  3. [11]

    H. Li, H. Liu, X. Ji, G. Li, and L. Shi. Cifar10-dvs: An event-stream dataset for object classification. In Frontiers in Neuroscience 2017, 2017

    2017

  4. [12]

    DARTS: differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: differentiable architecture search. In ICLR 2019, a

    2019

  5. [13]

    Ecoformer: Energy-saving attention with linear complexity

    Jing Liu, Zizheng Pan, Haoyu He, Jianfei Cai, and Bohan Zhuang. Ecoformer: Energy-saving attention with linear complexity. In NeurIPS 2022, b

    2022

  6. [14]

    Fast neural networks without multipliers

    Michele Marchesi, Gianni Orlandi, Francesco Piazza, and Aurelio Uncini. Fast neural networks without multipliers. IEEE transactions on Neural Networks, 4 0 (1): 0 53--62, 1993

    1993

  7. [15]

    Merolla, John V

    Paul A. Merolla, John V. Arthur, Rodrigo Alvarez-Icaza, Andrew S. Cassidy, Jun Sawada, Filipp Akopyan, Bryan L. Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, Bernard Brezzo, Ivan Vo, Steven K. Esser, Rathinakumar Appuswamy, Brian Taba, Arnon Amir, Myron D. Flickner, William ...

    2014

  8. [16]

    A white paper on neural network quantization

    Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart van Baalen, and Tijmen Blankevoort. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295, 2021. URL https://arxiv.org/abs/2106.08295

    2021 arXiv

  9. [17]

    Q-spinn: A framework for quantizing spiking neural networks

    Rachmad Vidya Wicaksana Putra and Muhammad Shafique. Q-spinn: A framework for quantizing spiking neural networks. In IJCNN 2021

    2021

  10. [18]

    An approach to the application of shift-and-add algorithms on engineering and industrial processes

    Jose-Luis Sanchez-Romero, Antonio Jimeno-Morenilla, Rafael Molina-Carmona, and Jose Perez-Martinez. An approach to the application of shift-and-add algorithms on engineering and industrial processes. Mathematical and Computer Modelling, 57 0 (7-8): 0 1800--1806, 2013

    2013

  11. [19]

    Adder attention for vision transformer

    Han Shu, Jiahao Wang, Hanting Chen, Lin Li, Yujiu Yang, and Yunhe Wang. Adder attention for vision transformer. In NeurIPS 2021

    2021

  12. [20]

    Hardware efficient weight-binarized spiking neural networks

    Chengcheng Tang and Jie Han. Hardware efficient weight-binarized spiking neural networks. In DATE 2023

    2023

  13. [21]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neurips 2017

    2017

  14. [22]

    Dista: Denoising spiking transformer with intrinsic plasticity and spatiotemporal attention

    Boxun Xu, Hejia Geng, Yuxuan Yin, and Peng Li. Dista: Denoising spiking transformer with intrinsic plasticity and spatiotemporal attention. arXiv preprint arXiv:2311.09376, 2023

    2023 arXiv

  15. [23]

    Ds2ta: Denoising spiking transformer with attenuated spatiotemporal attention

    Boxun Xu, Hejia Geng, Yuxuan Yin, and Peng Li. Ds2ta: Denoising spiking transformer with attenuated spatiotemporal attention. arXiv preprint arXiv:2409.15375, 2024 a

    2024 arXiv

  16. [24]

    Spiking transformer hardware accelerators in 3d integration

    Boxun Xu, Junyoung Hwang, Pruek Vanna-iampikul, Sung Kyu Lim, and Peng Li. Spiking transformer hardware accelerators in 3d integration. arXiv preprint arXiv:2411.07397, 2024 b

    2024 arXiv

  17. [25]

    Adaptive equalizer using finite-bit power-of-two quantizer

    Ping Xue and Bede Liu. Adaptive equalizer using finite-bit power-of-two quantizer. IEEE transactions on acoustics, speech, and signal processing, 34 0 (6): 0 1603--1611, 1986

    1986

  18. [26]

    Shiftaddnet: A hardware-inspired deep network

    Haoran You, Xiaohan Chen, Yongan Zhang, Chaojian Li, Sicheng Li, Zihao Liu, Zhangyang Wang, and Yingyan Lin. Shiftaddnet: A hardware-inspired deep network. In NeurIPS 2020

    2020

  19. [27]

    Temporal spike sequence learning via backpropagation for deep spiking neural networks

    Wenrui Zhang and Peng Li. Temporal spike sequence learning via backpropagation for deep spiking neural networks. In NeurIPS 2020

    2020

  20. [28]

    Spikformer: When spiking neural network meets transformer

    Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. In ICLR 2023

    2023

  21. [29]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.