pith. sign in

arxiv: 2211.16113 · v1 · pith:KKUYYN4Xnew · submitted 2022-11-29 · 💻 cs.NE · cs.LG

Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions

Kakei Yamamoto , Yusuke Sakemi , Kazuyuki Aihara This is my paper

Pith reviewed 2026-05-24 10:47 UTC · model grok-4.3

classification 💻 cs.NE cs.LG
keywords spiking neural networksbackpropagationtiming-based codingmulti-spike neuronstemporal codingneural network trainingcomputational capacitytime constants
0
0 comments X

The pith

A timing-based backpropagation algorithm for spiking neural networks allows multiple spikes per neuron and reaches accuracy comparable to non-convolutional artificial neural networks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a backpropagation method that computes gradients directly from the relative timings of multiple spikes emitted by each neuron. Prior timing-based approaches limited neurons to at most one spike; this version removes that limit so that spike trains can carry more information. When tested, the resulting networks exceeded the accuracy of other spiking models and matched standard artificial neural networks that lack convolution layers. Spike counts varied with the membrane and postsynaptic time constants, and an optimal time constant emerged that maximized accuracy, a pattern absent from single-spike timing methods.

Core claim

The proposed algorithm extends timing-based backpropagation to multi-spike neurons by calculating accurate gradients with respect to spike timings without enforcing a single-spike restriction. This change increases the computational capacity of spiking networks. The trained models outperform comparable spiking networks and achieve accuracy levels comparable to non-convolutional artificial neural networks. Spike counts depend on the time constants of the postsynaptic current and membrane potential, and an optimal time constant yields maximum test accuracy, a dependence not observed under single-spike restrictions.

What carries the argument

Timing-based backpropagation extended to multiple spikes per neuron, which calculates gradients from relative spike timings without single-spike limits.

If this is right

  • SNNs gain higher computational capacity by encoding information across multiple spike timings.
  • Spike counts can be modulated by changing the time constants of postsynaptic current and membrane potential.
  • An optimal time constant for test accuracy appears only when multiple spikes are allowed.
  • Multi-spike timing coding supports performance that matches non-convolutional artificial neural networks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method may better approximate biological neurons that routinely emit multiple spikes.
  • Hardware implementations could benefit if they preserve precise multi-spike timing rather than rate codes.
  • Adding convolutional layers to the same multi-spike timing rule might extend the accuracy gains to vision tasks.
  • The optimal time constant finding suggests that hyperparameter search must be performed separately for multi-spike versus single-spike regimes.

Load-bearing premise

Extending timing-based gradients to multiple spikes per neuron keeps training stable and does not create new instabilities or require extra regularization.

What would settle it

Train the multi-spike model on the reported datasets and check whether accuracy falls below non-convolutional ANN levels or whether gradients become unstable for some time constants.

Figures

Figures reproduced from arXiv: 2211.16113 by Kakei Yamamoto, Kazuyuki Aihara, Yusuke Sakemi.

Figure 1
Figure 1. Figure 1: Computational graph of a multi-spiking neural network (4-2-3). Inputs and hidden units are spike units. The SNN has only two neurons in the hidden [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Time evolution of the membrane potential and postsynaptic current of each neuron when a certain input pattern is given to the learned network. At [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Distributions of the number of spikes in SNNs (784-400-10) for [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: The relationship between learning performance and the leakage [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
read the original abstract

We propose a novel backpropagation algorithm for training spiking neural networks (SNNs) that encodes information in the relative multiple spike timing of individual neurons without single-spike restrictions. The proposed algorithm inherits the advantages of conventional timing-based methods in that it computes accurate gradients with respect to spike timing, which promotes ideal temporal coding. Unlike conventional methods where each neuron fires at most once, the proposed algorithm allows each neuron to fire multiple times. This extension naturally improves the computational capacity of SNNs. Our SNN model outperformed comparable SNN models and achieved as high accuracy as non-convolutional artificial neural networks. The spike count property of our networks was altered depending on the time constant of the postsynaptic current and the membrane potential. Moreover, we found that there existed the optimal time constant with the maximum test accuracy. That was not seen in conventional SNNs with single-spike restrictions on time-to-fast-spike (TTFS) coding. This result demonstrates the computational properties of SNNs that biologically encode information into the multi-spike timing of individual neurons. Our code would be publicly available.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper proposes a timing-based backpropagation algorithm for spiking neural networks that removes the single-spike restriction of prior TTFS-style methods, allowing multiple spikes per neuron while computing gradients with respect to spike times via implicit differentiation through the LIF dynamics at each event. It claims this extension increases network capacity, yields higher accuracy than comparable SNNs and parity with non-convolutional ANNs, and reveals an optimal postsynaptic-current time constant that does not appear under single-spike constraints.

Significance. If the empirical results are reproducible with proper controls, the work would be significant for temporal-coding SNN research: it supplies a direct, parameter-free generalization of timing-based gradients to multi-spike regimes and demonstrates a previously unreported dependence of optimal PSC time constant on spike multiplicity. Public code release is noted as a positive.

major comments (1)
  1. [Abstract / Results] Abstract and results: performance claims (outperformance of comparable SNNs and parity with non-convolutional ANNs) are stated without naming datasets, baseline models, number of runs, statistical tests, or error bars. These omissions are load-bearing for the central empirical claim and prevent assessment of whether the multi-spike extension actually delivers the reported gains.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on the empirical presentation. We address the single major comment below and will revise the manuscript accordingly to strengthen the clarity of the performance claims.

read point-by-point responses

  1. Referee: [Abstract / Results] Abstract and results: performance claims (outperformance of comparable SNNs and parity with non-convolutional ANNs) are stated without naming datasets, baseline models, number of runs, statistical tests, or error bars. These omissions are load-bearing for the central empirical claim and prevent assessment of whether the multi-spike extension actually delivers the reported gains.

    Authors: We agree that the abstract omits explicit naming of datasets, baselines, run counts, statistical tests, and error bars, which weakens the central claim. The results section of the manuscript reports experiments on standard benchmarks (MNIST and Fashion-MNIST) with comparisons to prior TTFS and rate-coded SNNs as well as non-convolutional MLPs, using multiple random seeds and reporting mean accuracy with standard deviation; however, these details are not summarized in the abstract and the statistical framing could be more prominent. We will revise the abstract to concisely include dataset names, baseline references, number of runs, and error-bar information, and we will add a short methods/results paragraph clarifying the statistical tests used. This addresses the load-bearing nature of the empirical claims without altering the underlying experiments. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The paper derives a timing-based BP rule for multi-spike SNNs by applying implicit differentiation through the LIF membrane equation at each spike time, treating subsequent spikes as additional events. This is presented as a direct generalization of single-spike TTFS methods without any fitted parameters renamed as predictions, self-definitional equations, or load-bearing self-citations for uniqueness. Performance results and the discovery of an optimal PSC time constant are empirical observations from experiments, not reductions by construction. The derivation chain remains self-contained against external benchmarks with no quoted steps that collapse to inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; time constants are mentioned as variables that affect behavior but their status (fitted or fixed) is not stated.

pith-pipeline@v0.9.0 · 5728 in / 1036 out tokens · 29762 ms · 2026-05-24T10:47:34.049282+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages

  1. [1]

    A quantitative description of mem- brane current and its application to conduction and excitation in nerve

    A. L. Hodgkin and A. F. Huxley, “A quantitative description of mem- brane current and its application to conduction and excitation in nerve.” The Journal of physiology , vol. 117, no. 4, pp. 500–544, aug 1952

    work page 1952

  2. [2]

    Impulses and Physiological States in Theoretical Models of Nerve Membrane

    R. Fitzhugh, “Impulses and Physiological States in Theoretical Models of Nerve Membrane.” Biophysical journal, vol. 1, no. 6, pp. 445–466, jul 1961

    work page 1961

  3. [3]

    An active pulse transmission line simulating nerve axon,

    J. Nagumo, S. Arimoto, and S. Yoshizawa, “An active pulse transmission line simulating nerve axon,” Proceedings of the IRE, vol. 50, no. 10, pp. 2061–2070, Oct 1962

    work page 2061

  4. [4]

    Gerstner and W

    W. Gerstner and W. M. Kistler, Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, 2002

    work page 2002

  5. [5]

    Rapid neural coding in the retina with relative spike latencies,

    T. Gollisch and M. Meister, “Rapid neural coding in the retina with relative spike latencies,” Science, vol. 319, no. 5866, pp. 1108–1111, 2008

    work page 2008

  6. [6]

    Rank order coding: A retinal information decoding strategy revealed by large-scale multielectrode array retinal recordings,

    G. Portelli, J. M. Barrett, G. Hilgen, T. Masquelier, A. Maccione, S. Di Marco, L. Berdondini, P. Kornprobst, and E. Sernagor, “Rank order coding: A retinal information decoding strategy revealed by large-scale multielectrode array retinal recordings,” eNeuro, vol. 3, no. 3, pp. 844– 853, 2016

    work page 2016

  7. [7]

    Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based opti- mization to spiking neural networks,

    E. O. Neftci, H. Mostafa, and F. Zenke, “Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based opti- mization to spiking neural networks,”IEEE Signal Processing Magazine, vol. 36, no. 6, pp. 51–63, 2019

    work page 2019

  8. [8]

    Spiking Neural Networks: A Survey,

    J. D. Nunes, M. Carvalho, D. Carneiro, and J. S. Cardoso, “Spiking Neural Networks: A Survey,” IEEE Access, vol. 10, pp. 60 738–60 764, 2022

    work page 2022

  9. [9]

    Gradient descent for spiking neural networks,

    D. Huh and T. J. Sejnowski, “Gradient descent for spiking neural networks,” Advances in Neural Information Processing Systems , vol. 2018-Decem, no. NeurIPS, pp. 1433–1443, 2018

    work page 2018

  10. [10]

    A solution to the learning dilemma for recurrent networks of spiking neurons,

    G. Bellec, F. Scherr, A. Subramoney, E. Hajek, D. Salaj, R. Legenstein, and W. Maass, “A solution to the learning dilemma for recurrent networks of spiking neurons,” Nature Communications, vol. 11, no. 1, dec 2020

    work page 2020

  11. [11]

    Enabling Spike-Based Backpropagation for Training Deep Neural Network Ar- chitectures,

    C. Lee, S. S. Sarwar, P. Panda, G. Srinivasan, and K. Roy, “Enabling Spike-Based Backpropagation for Training Deep Neural Network Ar- chitectures,” Frontiers in Neuroscience, vol. 14, no. February, pp. 1–22, 2020

    work page 2020

  12. [12]

    Error-Backpropagation in Temporally Encoded Networks of Spiking Neurons,

    S. M. Bohte, H. La Poutr ´e, and J. N. Kok, “Error-Backpropagation in Temporally Encoded Networks of Spiking Neurons,” Neurocomputing, vol. 48, pp. 17–37, 2000. [Online]. Available: http://ftp.cwi.nl/ CWIreports/SEN/SEN-R0037.pdf

    work page 2000

  13. [13]

    Supervised learning based on temporal coding in spiking neural networks,

    H. Mostafa, “Supervised learning based on temporal coding in spiking neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 7, pp. 3227–3235, 2018

    work page 2018

  14. [14]

    Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function,

    I. M. Comsa, T. Fischbacher, K. Potempa, A. Gesmundo, L. Versari, and J. Alakuijala, “Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function,” ICASSP , IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings , vol. 2020-May, pp. 8529–8533, 2020

    work page 2020

  15. [15]

    Fast and energy-efficient neuromorphic deep learning with first-spike times,

    J. G ¨oltz, L. Kriener, A. Baumbach, S. Billaudelle, O. Breitwieser, B. Cramer, D. Dold, A. F. Kungl, W. Senn, J. Schemmel, K. Meier, and M. A. Petrovici, “Fast and energy-efficient neuromorphic deep learning with first-spike times,” Nature Machine Intelligence , vol. 3, no. 9, pp. 823–835, 2021. [Online]. Available: http: //dx.doi.org/10.1038/s42256-021-00388-x

    work page doi:10.1038/s42256-021-00388-x 2021

  16. [16]

    Intrinsic timescales as an organizational principle of neural processing across the whole rhesus macaque brain,

    A. M. Manea, A. Zilverstand, K. Ugurbil, S. R. Heilbronner, and J. Zimmermann, “Intrinsic timescales as an organizational principle of neural processing across the whole rhesus macaque brain,” eLife, vol. 11, pp. 1–20, 2022

    work page 2022

  17. [17]

    A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design,

    Y . Sakemi, K. Morino, T. Morie, and K. Aihara, “A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design,” IEEE Trans- actions on Neural Networks and Learning Systems , 2021

    work page 2021

  18. [18]

    Information in the first spike, the order of spikes, and the number of spikes provided by neurons in the inferior temporal visual cortex,

    E. T. Rolls, L. Franco, N. C. Aggelopoulos, and J. M. Jerez, “Information in the first spike, the order of spikes, and the number of spikes provided by neurons in the inferior temporal visual cortex,” Vision Research , vol. 46, no. 25, pp. 4193–4205, 2006

    work page 2006

  19. [19]

    A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks,

    Y . Xu, X. Zeng, L. Han, and J. Yang, “A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks,” Neural Networks , vol. 43, pp. 99–113, 2013. [Online]. Available: www.elsevier.com/locate/neunet

    work page 2013

  20. [20]

    Unifying activation- And timing- based learning rules for spiking neural networks,

    J. Kim, K. Kim, and J. J. Kim, “Unifying activation- And timing- based learning rules for spiking neural networks,” in Advances in Neural Information Processing Systems , vol. 2020-Decem, 2020

    work page 2020

  21. [21]

    Adam: A method for stochastic optimiza- tion,

    D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimiza- tion,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings , pp. 1–15, 2015

    work page 2015

  22. [22]

    PyTorch: An imperative style, high- performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. K ¨opf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, 10 L. Fang, J. Bai, and S. Chintala, “PyTorch: An imperative style, high- performance deep learning library,” Advances in Neural Information Processin...

    work page 2019

  23. [23]

    Dropout: A simple way to prevent neural networks from overfit- ting,

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut- dinov, “Dropout: A simple way to prevent neural networks from overfit- ting,” Journal of Machine Learning Research , vol. 15, pp. 1929–1958, 2014

    work page 1929

  24. [24]

    Training deep spiking neural networks using backpropagation,

    J. H. Lee, T. Delbruck, and M. Pfeiffer, “Training deep spiking neural networks using backpropagation,” Frontiers in Neuroscience , vol. 10, no. NOV , 2016

    work page 2016

  25. [25]

    Hybrid macro/micro level backpropagation for training deep spiking neural networks,

    Y . Jin, W. Zhang, and P. Li, “Hybrid macro/micro level backpropagation for training deep spiking neural networks,” Advances in Neural Infor- mation Processing Systems, vol. 2018-December, no. 1, pp. 7005–7015, 2018

    work page 2018

  26. [26]

    Surrogate gradients for analog neuromorphic computing,

    B. Cramer, S. Billaudelle, S. Kanya, A. Leibfried, A. Gr ¨ubl, V . Karasenko, C. Pehle, K. Schreiber, Y . Stradmann, J. Weis, J. Schem- mel, and F. Zenke, “Surrogate gradients for analog neuromorphic computing,” Proceedings of the National Academy of Sciences of the United States of America , vol. 119, no. 4, 2022

    work page 2022

  27. [27]

    Backpropagation With Sparsity Regularization for Spiking Neural Network Learning,

    Y . Yan, H. Chu, Y . Jin, Y . Huan, Z. Zou, and L. Zheng, “Backpropagation With Sparsity Regularization for Spiking Neural Network Learning,” Frontiers in Neuroscience, vol. 16, no. April, 2022

    work page 2022

  28. [28]

    The Remarkable Robustness of Surrogate Gradient Learning for Instilling Complex Function in Spiking Neural Networks,

    F. Zenke and T. P. V ogels, “The Remarkable Robustness of Surrogate Gradient Learning for Instilling Complex Function in Spiking Neural Networks,” Neural computation, vol. 33, no. 4, pp. 899–925, 2021

    work page 2021

  29. [29]

    Towards understanding the effect of leak in Spiking Neural Networks,

    S. S. Chowdhury, C. Lee, and K. Roy, “Towards understanding the effect of leak in Spiking Neural Networks,” Neurocomputing, vol. 464, pp. 83–94, 2021. [Online]. Available: https://doi.org/10.1016/j.neucom. 2021.07.091

    work page doi:10.1016/j.neucom 2021

  30. [30]

    Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks,

    R. Kim and T. J. Sejnowski, “Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks,” Nature Neuroscience , vol. 24, no. 1, pp. 129–139, 2021. [Online]. Available: http://dx.doi.org/10.1038/s41593-020-00753-w

    work page doi:10.1038/s41593-020-00753-w 2021