Pith. sign in

REVIEW 4 major objections 6 minor 53 references

FAS: Fast ANN-SNN Conversion for Spiking Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that pretrained large language models can be converted into spiking neural networks that run in as few as four to sixteen timesteps while keeping accuracy close to the original model, and that the conversion works for…

desk verdict Solid two-stage SNN conversion recipe for LLMs, but the abstract's headline energy and accuracy numbers are narrower than they look. read the letter →

arxiv 2502.04405 v2 pith:CYWW53FR submitted 2025-02-06 cs.LG cs.AIcs.CL

Long Chen , Xiaotian Song , Andy Song , BaDong Chen , Jiancheng Lv , Yanan Sun This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords spikingneuralnetworksANN-SNNconversionlargelanguagemodelslow-latencyinferencetemporalerrorcalibrationquantizationclip-floor-shiftenergy-efficientLLMneuron-wise
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FAS is a two-stage recipe for turning pretrained LLMs into spiking neural networks without training a spiking model from scratch. The first stage fine-tunes all parameters of the pretrained model after swapping its activation for the Quantization Clip-Floor-Shift (QCFS) function, which makes analog activations match what spiking neurons can represent. The second stage calibrates the converted network, first adjusting each layer's firing threshold and initial membrane potential, then tuning each neuron individually so its firing rate tracks the ANN's activation. The paper reports that this yields GLUE results close to BERT at four timesteps, WikiText-103 perplexity 16.84 at sixteen timesteps, and competitive zero-shot performance from converted OPT-7B, PaliGemma-3B, and LLaVA-1.5-7B; the energy savings are claimed on the spiking components only.

What carries the argument

The load-bearing identity is the Integrate-and-Fire rate map $r^l(T) = \mathrm{clip}(\frac{\theta^l}{T}\lfloor \frac{T W^l r^{l-1}(T)+v^l(0)}{\theta^l}\rfloor, 0, \theta^l)$, which says a layer's spike rate is a clipped, shifted step function of the previous layer's rate. FAS trains the ANN to mimic this step function by replacing its activation with QCFS, and then exploits the fact that the step size is $\theta^l/T$: with low activation values, most neurons need fewer than $T/2$ spikes, so their thresholds can be lowered without changing the representable values. The coarse-to-fine calibration, layer-wise scaling of thresholds and initial potentials followed by neuron-wise backpropagation on those two quantities alone, is the mechanism that shrinks the residual temporal error, with the activation-align MSE and logits KL losses keeping the spiking network aligned to the ANN.

What would settle it

Measure the full end-to-end energy of a FAS-converted model on hardware that accounts for spiking and non-spiking layers at 8 timesteps; if the spiking-model total is not a small fraction of the ANN's total, the 96.63% claim does not hold at system level. A second check is to compute theoretical spike counts $\tau_{\mathrm{theor}}$ for a different LLM and dataset: if more than a few percent of neurons exceed $T/2$, the threshold-lowering step that FAS's second stage relies on will not generalize.

Watch

Extended reading notes

Core claim

The central claim is that ANN-to-SNN conversion error for LLMs splits into two treatable parts: quantization/clipping error, caused by discretizing activations, and temporal error, caused by the irregular arrival of spikes. FAS's first stage removes most of the first kind by fine-tuning the pretrained LLM with QCFS activations, so the ANN itself is trained to behave like an ideal spiking network. The paper's insight for the second kind is empirical: after this fine-tuning, about 99% of neurons need at most T/2 spikes to represent their activations over T timesteps, so lowering thresholds is safe for most neurons but not all; FAS therefore lowers thresholds layer-wise and then optimizes per-neuron thresholds and initial membrane potentials with backpropagation through time, using the ANN's activations and logits as targets. The result, as reported, is a converted SNN whose accuracy matches or exceeds the original ANN on several tasks, at four to sixteen timesteps.

Load-bearing premise

The energy numbers assume fixed costs of 0.9 pJ per addition and 4.6 pJ per multiply-accumulate and count only the spiking layers, so if the non-spiking parts—softmax, layer normalization, embeddings, and calibration overhead—consume a large share of deployment energy, the reported reductions shrink.

Editorial extensions

If this is right

  • At 4-16 timesteps, converted spiking LLMs could be deployed with inference latency far below the hundreds of timesteps typical of earlier spiking language models.
  • Because FAS starts from pretrained weights, converting a new LLM costs one fine-tuning run and a calibration pass, not a full spiking training run.
  • The same two-stage error decomposition may apply to other deep sequence models, not just Transformers, whenever their activations are bounded and mostly small.
  • If the spiking-layer energy figures hold, running the converted networks on spike-based hardware would replace dense multiply-accumulate operations with additions that scale with observed spike rates, reported below 0.64 in the GPT-2 analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline 96.63% energy reduction is not an end-to-end claim: embeddings, softmax, layer normalization, and any offset-spike machinery sit outside the spiking-layer accounting, so a full-system energy comparison could shrink the advantage considerably.
  • The same threshold-lowering logic suggests a testable extension: let each layer or neuron choose its timesteps dynamically based on its activation magnitude, since the T/2 observation is data-dependent rather than architectural.
  • Conversion quality likely tracks calibration-data coverage; probing the converted models on out-of-distribution inputs would reveal whether the calibration generalizes or is tuned to the benchmark distribution.
  • For multimodal models, FAS's reported gains on HallusionBench raise the question of whether conversion changes hallucination patterns, which accuracy alone does not show.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces FAS, a two-stage ANN-to-SNN conversion method for large language models. Stage 1 replaces the activation function of a pretrained LLM with the QCFS quantization clip-floor-shift function and fine-tunes all parameters; stage 2 performs layer-wise and then neuron-wise calibration of firing thresholds and initial membrane potentials, using the ANN's activations and logits as alignment targets. Experiments are reported on GLUE tasks with BERT, language modeling with GPT-2, zero-shot reasoning with OPT-7B, and vision-language tasks with PaliGemma-3B and LLaVA-1.5-7B, at 1-32 timesteps. The paper claims state-of-the-art conversion performance, low latency, and a 96.63% energy reduction at eight timesteps, with source code provided.

Significance. The method is a reasonable and potentially valuable contribution: it avoids training an SNN from scratch, uses a two-stage calibration that is computationally light (only thresholds and initial potentials are optimized), and it is evaluated across a broader set of language and vision-language models than most prior ANN-SNN conversion work. The release of source code, the ablations of LWC/NWC, rho, lambda, and L, and the use of ANN activations as a teacher signal are all strengths. However, the headline claims are currently stated more strongly than the evidence supports: the '3% higher' accuracy is a single-task result, and the 96.63% energy figure excludes non-spiking components such as embeddings, softmax, LayerNorm, and vision encoders. If the authors scope the claims and add system-level energy analysis or repeated-run statistics, the contribution would be a solid empirical paper.

major comments (4)
  1. [Abstract; Table 3] The abstract's claim that 'FAS only takes eight timesteps to achieve an accuracy of 3% higher than that of the OPT-7B model' is not supported as a general statement. At T=8 in Table 3, FAS is higher than OPT-7B on only one of seven tasks (COPA: 84 vs 81), equal on OpenbookQA (27.60 vs 27.60), and lower on the remaining five tasks (e.g., PIQA 72.74 vs 76.26, Winogrande 60.30 vs 65.43, WSC 77.29 vs 82.05). Please rephrase the claim to refer to the specific task or report an aggregate (e.g., mean accuracy or matched-pair comparison) instead of implying a global improvement.
  2. [Section 5.7; Appendix I; Table 3] The headline energy reduction of 96.63% is computed only for the spiking parts of the model. Appendix I explicitly states that the comparison is made 'solely on the energy consumption of the spiking parts and their corresponding parts in the ANN,' which excludes embeddings, softmax, LayerNorm, and, for vision-language models, the vision encoder. Section 5.7 shows that spiking these non-linearities with UGO leads to accuracy drops (e.g., QQP falls from 90.38 to 89.54) and does not report energy numbers for the full converted model. The abstract's system-level energy claim is therefore not established; please provide full-model energy estimates that include all operations, or explicitly scope the 96.63% figure to the spiking layers.
  3. [Tables 1-5, 18-20] No error bars, confidence intervals, or repeated-seed results are reported. Several headline comparisons are very small (e.g., Table 2 shows FAS at 90.75 vs BERT at 90.66 at T=16, and Table 18 shows differences below 0.5% on several GLUE tasks). These differences may be within run-to-run variance, especially given the full-parameter fine-tuning and calibration procedures. Please report mean and standard deviation over at least three runs for the key claims, or provide a statistical significance test, so that the 'surpasses the ANN' and 'state-of-the-art' wording is justified.
  4. [Equations (5)-(6); Section 4.3] The definitions of the theoretical spike count and theoretical maximum spike count are under-specified. In Eq. (5), psi_i is defined as 'Max(tau_theor)' without stating whether the maximum is taken over time, over the dataset, or both; Eq. (6) then uses tau_real and tau_theor as if they were well-defined random variables. Because Observation 1 and the threshold-lowering rationale in Stage 2 rest on these quantities, please formalize their domains and state the assumptions under which Eq. (6) approximates the temporal error.
minor comments (6)
  1. [Section 4.1] There is a typo: 'detailes' should be 'details'.
  2. [Section 5.4; Appendix J] The model name is spelled inconsistently as 'Pailgemma-3B' in Section 5.4 and 'PaliGemma-3B' elsewhere; use one consistent spelling.
  3. [Tables 2 and 21] The baseline name is written as both 'SRP' and 'SPR'; the abbreviation should be consistent (the reference [17] uses SPR).
  4. [Equation (9)] The temperature parameter in the logits loss is denoted T, which conflicts with the number of timesteps T used throughout the paper; please rename the temperature (e.g., tau).
  5. [Figure 3(a)] The caption labels 'case:1 w/' and 'case:1 w/o' are cryptic; please spell out what 'w/' and 'w/o' mean (with/without optimization).
  6. [Table 18] The cell '87.977/87558' appears to be a typo (likely '87.977/87.558'); also Table 11 contains 'epcoh' instead of 'epoch'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: FAS's calibration is a fitted teacher-student procedure evaluated on held-out benchmarks; the partial energy accounting is a scope caveat, not a circular reduction.

full rationale

The derivation chain is self-contained. Stage 2 calibrates thresholds and initial membrane potentials by minimizing a teacher-student loss (Eqs. 8-10) on calibration data; this is a fitting procedure, not a prediction derived from the target result. The reported accuracies (Tables 1, 3, 4, 5, 12, 13) are measured on held-out evaluation sets, and Table 7's ablation shows that the calibration components independently improve test perplexity and accuracy, so the results are not forced by the training objective. The temporal-error analysis (Eq. 6 and Observations 1-3) is a motivating heuristic supported by empirical distributions and ablations, not a theorem whose conclusion is assumed as input. The energy claim in Appendix I explicitly restricts the comparison to 'the energy consumption of the spiking parts and their corresponding parts in the ANN'; excluding embeddings, softmax, and LayerNorm is an accounting-scope limitation that weakens the system-level claim but does not make the ratio circular. The one self-citation ([38]) is a related-work pointer in the introduction and is not load-bearing for the FAS derivation or the experimental claims. Therefore, no circular step is identified.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central method relies on standard SNN conversion assumptions plus two paper-specific modeling choices (threshold lowering, energy accounting). No fundamentally new physical entities or forces are introduced. The main free parameters are calibration hyperparameters tuned on validation data.

free parameters (5)
  • alpha (threshold scaling) = 0.6
    Layer-wise threshold scaling factor. The paper says it can be determined from theoretical spike counts but sets it to 0.6 for BERT and GPT-2 (Appendix F).
  • beta (initial membrane potential scaling) = 0.1
    Scaling factor for initial membrane potentials relative to threshold, set to 0.1 for BERT and GPT-2 (Appendix F).
  • lambda1, lambda2 (loss weights) = 1:0.0012 for GPT-2; 1:1 for vision-language; lambda2 searched 0.2-1 for BERT
    Weights for activation align and logits losses, chosen by performance (Tables 9, 14, Appendix G.3).
  • rho (calibration steps) = rho=2 best for BERT; rho=T best for GPT-2
    Number of timesteps used in neuron-wise calibration; selected per model based on performance (Tables 6, 13).
  • L (quantization levels) = 8 for GPT-2 main results
    QCFS quantization parameter; L=8 gives the best trade-off in Table 8.
assumptions (4)
  • domain assumption IF neuron spike-rate formula Eq. (3)/(14) approximates ANN activation
    Standard in ANN-SNN conversion; relies on assuming v_l(T) lies in [0, theta) and ignoring reset details; stated in Section 3.3 and Appendix A.
  • domain assumption QCFS activation is a good substitute for ReLU/GELU after fine-tuning
    Borrowed from Bu et al. [6]; the paper assumes fine-tuning with QCFS eliminates QC errors before Stage 2.
  • ad hoc to paper Temporal error can be reduced by lowering thresholds and calibrating initial potentials
    Introduced in Observations 1-3 and Eq. (6); it motivates Stage 2 but is not derived from first principles.
  • domain assumption Energy model with 0.9 pJ/AC and 4.6 pJ/MAC applied only to spiking parts
    From Li et al. [24]; the paper explicitly excludes non-spiking MAC operations, which limits the energy claim (Appendix I).

how reviews work

0 comments
Cite this review

Pith. "Pith review of FAS: Fast ANN-SNN Conversion for Spiking Large Language Models." pith.science (2026). https://pith.science/paper/CYWW53FR

@misc{pith2026250204405,
  author       = {Pith},
  title        = {Pith review of: FAS: Fast ANN-SNN Conversion for Spiking Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CYWW53FR}},
  note         = {Machine review of arXiv:2502.04405}
}
read the original abstract

Spiking Large Language Models have been shown as a good alternative to LLMs in various scenarios. Existing methods for creating Spiking LLMs, i.e., direct training and ANN-SNN conversion, often suffer from performance degradation and relatively high computational costs. To address these issues, we propose a novel Fast ANN-SNN conversion strategy (FAS) that transforms LLMs into spiking LLMs in two stages. The first stage employs a full-parameter fine-tuning of pre-trained models, so it does not need any direct training from scratch. The second stage introduces a coarse-to-fine calibration method to reduce conversion errors and improve accuracy. Experiments on both language and vision-language tasks across four different scales of LLMs demonstrate that FAS can achieve state-of-the-art performance yet with significantly reduced inference latency and computational costs. Notably, FAS only takes eight timesteps to achieve an accuracy of 3\% higher than that of the OPT-7B model, while reducing energy consumption by 96.63\%. The source code is available at https://github.com/lc783/FAS

Figures

Figures reproduced from arXiv: 2502.04405 by the authors.

Figure 1
Figure 1. Performance of ANN-SNN conver￾sion methods on GPT-2 for WikiText-103. When dealing with LLMs, several problems arise. First, the training cost of LLMs is significantly higher than that of CNNs, making existing training-from￾scratch conversion methods ineffective in the sce￾nario. This is because existing NN-SNN conver￾sion methods do not fully utilize the pre-trained weights [17, 42]. As a result, they lead to high … view at source ↗
Figure 2
Figure 2. The overall framework of the proposed FAS method. QC errors is composed of the quan￾tization error and the clipping error. 4 Methodology 4.1 Overall Framework As discussed in Section 3.3, the quantization er￾rors and clipping errors come from the process of discretizing the continuous ANN activation function, and the temporal errors are caused by using the disordered temporal sequences to generate spikes. The propos… view at source ↗
Figure 3
Figure 3. Illustration of our observations. Observation 1. By lowering the thresholds when τtheor < T, the temporal error can be reduced. The maximum activation value θ l in the SNN is aligned with the upper activation bound λ l of the ANN, thereby eliminating clipping error. However, when a l is lower than θ l , θ l can be set in the range [a l , θl ] without affecting the firing rate mapping to a l , provided that other dat… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Impact of parameter ρ Parameter ρ [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The effectiveness of FAS for threshold and initial membrane potentials optimization. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Relationship between Errors and Co￾sine Similarity 0 1000 2000 3000 4000 5000 6000 7000 Step 16.5 17.0 17.5 18.0 18.5 19.0 19.5 Perplexity GPT FAS(Global Performance) [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 8
Figure 8. Figure 8: Firing rate visualization of GPT-2 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Specifically, in both Case 1 and Case 2, FAS succeeded yet PaliGemma failed. The cases [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 9
Figure 9. Figure 9: Case Studies on Vision-Language Models 22 [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 29 canonical work pages

  1. [1]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 1(2):3, 2023

    arXiv 2023

  2. [2]

    Spikingbert: Distilling bert to train spiking language models using implicit differentiation

    Malyaban Bal and Abhronil Sengupta. Spikingbert: Distilling bert to train spiking language models using implicit differentiation. In AAAI Conference on Artificial Intelligence, 2023

    work page 2023

  3. [3]

    Spikingbert: Distilling bert to train spiking language models using implicit differentiation

    Malyaban Bal and Abhronil Sengupta. Spikingbert: Distilling bert to train spiking language models using implicit differentiation. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 10998–11006, 2024

    work page 2024

  4. [4]

    Paligemma: A versatile 3b vlm for transfer

    Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726, 2024

    arXiv 2024

  5. [5]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  6. [6]

    Optimized Potential Initialization for Low-latency Spiking Neural Networks

    Tong Bu, Jianhao Ding, Zhaofei Yu, and Tiejun Huang. Optimized potential initialization for low-latency spiking neural networks. ArXiv, abs/2202.01440, 2022

    work page Pith review arXiv 2022

  7. [7]

    Optimal ann-snn conversion for high-accuracy and ultra-low-latency spiking neural networks

    Tong Bu, Wei Fang, Jianhao Ding, PengLin Dai, Zhaofei Yu, and Tiejun Huang. Optimal ann-snn conversion for high-accuracy and ultra-low-latency spiking neural networks. arXiv preprint arXiv:2303.04347, 2023

    arXiv 2023

  8. [8]

    Spiking deep convolutional neural networks for energy-efficient object recognition

    Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, 113:54–66, 2015

    work page 2015

Show all 53 references
  1. [9]

    Loihi: A neuromorphic manycore processor with on-chip learning

    Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38(1):82–99, 2018

    2018

  2. [10]

    The growing energy footprint of artificial intelligence

    Alex de Vries. The growing energy footprint of artificial intelligence. Joule, 2023

    2023

  3. [11]

    Optimal conversion of conventional artificial neural networks to spiking neural networks

    Shi-Wee Deng and Shi Gu. Optimal conversion of conventional artificial neural networks to spiking neural networks. ArXiv, abs/2103.00476, 2021

    2021 arXiv

  4. [12]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019

    2019

  5. [13]

    Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing

    Peter Udo Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. 2015 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2015

    2015

  6. [14]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  7. [15]

    Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network

    Han et al. Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13555–13564, 2020

    2020

  8. [16]

    Deep spiking neural network: Energy efficiency through time based coding

    Bing Han and Kaushik Roy. Deep spiking neural network: Energy efficiency through time based coding. In European conference on computer vision, pages 388–404. Springer, 2020

    2020

  9. [17]

    Reducing ann-snn conversion error through residual membrane potential

    Zecheng Hao, Tong Bu, Jianhao Ding, Tiejun Huang, and Zhaofei Yu. Reducing ann-snn conversion error through residual membrane potential. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11–21, 2023

    2023

  10. [18]

    Bridging the gap between anns and snns by calibrating offset spikes

    Zecheng Hao, Jianhao Ding, Tong Bu, Tiejun Huang, and Zhaofei Yu. Bridging the gap between anns and snns by calibrating offset spikes. ArXiv, abs/2302.10685, 2023

    2023 arXiv

  11. [19]

    Hinton, Oriol Vinyals, and Jeffrey Dean

    Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. ArXiv, abs/1503.02531, 2015. 10

    2015 arXiv

  12. [20]

    The information pathways hypothesis: Transformers are dynamic self- ensembles

    Md Shamim Hussain. The information pathways hypothesis: Transformers are dynamic self- ensembles. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 810–821, 2023

    2023

  13. [21]

    Spatio-temporal approximation: A training-free snn conversion for transformers

    Yizhou Jiang, Kunlin Hu, Tianren Zhang, Haichuan Gao, Yuqian Liu, Ying Fang, and Feng Chen. Spatio-temporal approximation: A training-free snn conversion for transformers. In The Twelfth International Conference on Learning Representations, 2024

    2024

  14. [22]

    Training deep spiking neural networks using backpropagation

    Junhaeng Lee, Tobi Delbrück, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10, 2016

    2016

  15. [23]

    Efficient and accurate conversion of spiking neural network with burst spikes

    Yang Li and Yi Zeng. Efficient and accurate conversion of spiking neural network with burst spikes. In International Joint Conference on Artificial Intelligence, 2022

    2022

  16. [24]

    A free lunch from ann: Towards efficient, accurate spiking neural networks calibration

    Yuhang Li, Shi-Wee Deng, Xin Dong, Ruihao Gong, and Shi Gu. A free lunch from ann: Towards efficient, accurate spiking neural networks calibration. ArXiv, abs/2106.06984, 2021

    2021 arXiv

  17. [25]

    Error-aware conversion from ann to snn via post-training parameter calibration

    Yuhang Li, Shikuang Deng, Xin Dong, and Shi Gu. Error-aware conversion from ann to snn via post-training parameter calibration. International Journal of Computer Vision, pages 1–24, 2024

    2024

  18. [26]

    Learnable surrogate gradient for direct training spiking neural networks

    Shuang Lian, Jiangrong Shen, Qianhui Liu, Ziming Wang, Rui Yan, and Huajin Tang. Learnable surrogate gradient for direct training spiking neural networks. In IJCAI, pages 3002–3010, 2023

    2023

  19. [27]

    Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    2024

  20. [28]

    Fineweb-edu, May 2024

    Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. Fineweb-edu, May 2024

    2024

  21. [29]

    Spikebert: A language spikformer learned from bert with knowledge distillation, 2024

    Changze Lv, Tianlong Li, Jianhan Xu, Chenxi Gu, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, and Xuanjing Huang. Spikebert: A language spikformer learned from bert with knowledge distillation, 2024

    2024

  22. [30]

    Spiking convolutional neural networks for text classification

    Changze Lv, Jianhan Xu, and Xiaoqing Zheng. Spiking convolutional neural networks for text classification. In International Conference on Learning Representations, 2023

    2023

  23. [31]

    A million spiking-neuron integrated circuit with a scalable communication network and interface

    Paul A Merolla, John V Arthur, Rodrigo Alvarez-Icaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan, Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197...

    2014

  24. [32]

    Neftci, Hesham Mostafa, and Friedemann Zenke

    Emre O. Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51–63, 2019

    2019

  25. [33]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  26. [34]

    Theory and tools for the conversion of analog to spiking convolutional neural networks

    Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, and Michael Pfeiffer. Theory and tools for the conversion of analog to spiking convolutional neural networks. arXiv preprint arXiv:1612.04052, 2016

    2016 arXiv

  27. [35]

    Conversion of continuous-valued deep networks to efficient event-driven networks for image classification

    Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in neuroscience, 11:294078, 2017

    2017

  28. [36]

    Wang, Chiao Liu, and Kaushik Roy

    Abhronil Sengupta, Yuting Ye, Robert Y . Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in Neuroscience, 13, 2018

    2018

  29. [37]

    Astrocyte-enabled advancements in spiking neural networks for large language modeling.ArXiv, abs/2312.07625, 2023

    Guobin Shen, Dongcheng Zhao, Yiting Dong, Yang Li, Jindong Li, Kang Sun, and Yi Zeng. Astrocyte-enabled advancements in spiking neural networks for large language modeling.ArXiv, abs/2312.07625, 2023

    2023 arXiv

  30. [38]

    One-step spiking transformer with a linear complexity

    Xiaotian Song, Andy Song, Rong Xiao, and Yanan Sun. One-step spiking transformer with a linear complexity. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 3142–3150, 2024

    2024

  31. [39]

    diagnosis please

    Yuki Sonoda, Ryo Kurokawa, Yuta Nakamura, Jun Kanzawa, Mariko Kurokawa, Yuji Ohizumi, Wataru Gonoi, and Osamu Abe. Diagnostic performances of gpt-4o, claude 3 opus, and gemini 1.5 pro in “diagnosis please” cases. Japanese journal of radiology, pages 1–5, 2024. 11

    2024

  32. [40]

    Learning general purpose distributed sentence representations via large scale multi-task learning

    Sandeep Subramanian, Adam Trischler, Yoshua Bengio, and Christopher J Pal. Learning general purpose distributed sentence representations via large scale multi-task learning. In International Conference on Learning Representations, 2018

    2018

  33. [41]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In BlackboxNLP@EMNLP, 2018

    2018

  34. [42]

    Toward high-accuracy and low-latency spiking neural networks with two-stage optimization

    Ziming Wang, Yuhao Zhang, Shuang Lian, Xiaoxin Cui, Rui Yan, and Huajin Tang. Toward high-accuracy and low-latency spiking neural networks with two-stage optimization. IEEE Transactions on Neural Networks and Learning Systems, 2023

    2023

  35. [43]

    Spatio-temporal backpropagation for training high-performance spiking neural networks

    Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in Neuroscience, 12, 2017

    2017

  36. [44]

    Spikelm: Towards general spike-driven language modeling via elastic bi-spiking mechanisms

    Xingrun Xing, Zheng Zhang, Ziyi Ni, Shitao Xiao, Yiming Ju, Siqi Fan, Yequan Wang, Jiajun Zhang, and Guoqi Li. Spikelm: Towards general spike-driven language modeling via elastic bi-spiking mechanisms. arXiv preprint arXiv:2406.03287, 2024

    2024 arXiv

  37. [45]

    Training spiking neural networks with local tandem learning.Advances in Neural Information Processing Systems, 35:12662–12676, 2022

    Qu Yang, Jibin Wu, Malu Zhang, Yansong Chua, Xinchao Wang, and Haizhou Li. Training spiking neural networks with local tandem learning.Advances in Neural Information Processing Systems, 35:12662–12676, 2022

    2022

  38. [46]

    Spike- driven transformer

    Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. Spike- driven transformer. Advances in neural information processing systems, 36, 2024

    2024

  39. [47]

    Spike-based dynamic computing with asynchronous sensing-computing neuromorphic chip

    Man Yao, Ole Richter, Guangshe Zhao, Ning Qiao, Yannan Xing, Dingheng Wang, Tianxiang Hu, Wei Fang, Tugba Demirci, Michele De Marchi, et al. Spike-based dynamic computing with asynchronous sensing-computing neuromorphic chip. Nature Communications, 15(1):4464, 2024

    2024

  40. [48]

    The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks

    Friedemann Zenke and Tim P V ogels. The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks. Neural computation, 33(4):899–925, 2021

    2021

  41. [49]

    Opt: Open pre-trained transformer language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  42. [50]

    Minigpt-4: En- hancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  43. [51]

    Spikegpt: Generative pre-trained language model with spiking neural networks

    Rui-Jie Zhu, Qihang Zhao, Guoqi Li, and Jason K Eshraghian. Spikegpt: Generative pre-trained language model with spiking neural networks. arXiv preprint arXiv:2302.13939, 2023. 12 A The Spike Rate of SNNs This section present the details to get the functional representation of...

    2023 arXiv

  44. [52]

    FAS achieves SOTA performance across all time steps, surpassing the LLM.In particular, the accuracy of FAS with 16 time steps using the BERT model exceeds that of its ANN counterpart and other SOTA methods

  45. [53]

    For example, on GPT-2 models, the energy consumption of FAS and QCFC is 7.04% and 8.99% under 4 and 8 time steps, respectively

    FAS runs fast, especially under similar energy consumption. For example, on GPT-2 models, the energy consumption of FAS and QCFC is 7.04% and 8.99% under 4 and 8 time steps, respectively. As a result, FAS achieves lower perplexity, and the reduced number of time steps results ...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.