Pith. sign in

REVIEW 3 major objections 4 minor 43 references

A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HYPR is a training algorithm for recurrent spiking neural networks that parallelizes approximate online gradient learning over sequence segments, achieving constant memory independent of sequence length and, with oscillatory neuron…

desk verdict Solid algorithmic contribution with a genuinely parallel e-prop reformulation; the 'low gap' empirical claim is real but confounded and needs a cleaner comparison. read the letter →

arxiv 2506.14464 v1 pith:57NYZEY6 submitted 2025-06-17 cs.NE cs.AIcs.LG

Maximilian Baronig , Yeganeh Bahariasl , Ozan Özdenizci , Robert Legenstein This is my paper
classification cs.NEcs.AIcs.LG
keywords recurrentspikingneuralnetworksonlinelearningeligibilitypropagatione-propbackpropagationthroughtimeassociativescanoscillatoryneuronmodelsconstantmemorytraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces HYPR, a training algorithm for recurrent spiking neural networks that aims to combine the two things gradient-based sequence training usually cannot have together: online, sequence-length-independent memory and GPU parallelism. The core move is to rewrite the per-neuron eligibility updates of approximate forward learning as a linear state-space recurrence, so that cumulative parameter updates over a sequence segment can be computed with associative scans instead of a step-by-step forward pass. The authors show that the resulting cumulative updates are mathematically equivalent to fully online e-prop, while being computable up to 108 times faster on a GPU and with memory that depends only on the chosen subsequence length, not on total sequence length. On benchmarks with oscillatory spiking neuron models, HYPR comes within a few percentage points of backpropagation-through-time, a much smaller gap than earlier approximate forward-gradient methods. A careful reader would care because this points toward practical online training of recurrent spiking networks on long or infinite streams, on standard hardware and potentially on neuromorphic systems.

What carries the argument

The load-bearing object is the eligibility matrix $e_i^t$ of approximate forward learning, recast with per-step Jacobians $A_i^t$ and parameter derivatives $\delta_i^t$ into the linear state-space form $e_i^t = A_i^t e_i^{t-1} + \delta_i^t$, with the approximate parameter gradient $\tilde{\nabla}_{\theta_i}^t = B_i^t e_i^t$. Because this recurrence is linear in $e$, it can be unrolled and evaluated over a segment of length $\lambda$ with associative scans: the cumulative transition matrices $\phi^{\lambda:t} = \prod_{k=t}^{\lambda} A_i^k$ are obtained for all $t$ in parallel, and the backward-accumulated vectors $q_i^t = q_i^{t+1}A_i^{t+1} + dL^t/ds_i^t$ are obtained by a reverse scan. The final combination $[\tilde{\nabla}_{\theta_i}]_{1:\lambda} = q_i^0 e_i^0 + \sum_t q_i^t \delta_i^t$ reproduces the cumulative e-prop update without ever materializing the intermediate eligibility matrices; low-rank factorization of $\delta_i^t$ keeps memory at $O(\lambda)$ rather than $O(T)$. This machinery is what makes the memory constant in total sequence length and the computation parallelizable over time.

What would settle it

Reproduce the BPTT baselines on SHD, ECG, and sMNIST using the original released code and hyperparameters for each neuron model, then measure the gap to HYPR; if the gap is much larger than the reported few points, the 'unprecedentedly low gap' claim collapses. A second decisive check is to compare HYPR's cumulative parameter updates with fully online e-prop on the same network and seed: any numerical difference would falsify the Appendix F equivalence, and measuring GPU memory at fixed $\lambda$ while growing input length would test the constant-memory claim.

Watch

Extended reading notes

Core claim

The central claim is that approximate online forward learning and parallel backpropagation-through-time can be fused without changing the learning rule. Given a neuron with state $s_i^t$, the eligibility matrix $e_i^t$ follows the linear recurrence $e_i^t = A_i^t e_i^{t-1} + \delta_i^t$ with $A_i^t = \partial s_i^t/\partial s_i^{t-1}$ and $\delta_i^t = \partial s_i^t/\partial \theta_i$, and the approximate gradient is $\tilde{\nabla}_{\theta_i}^t = B_i^t e_i^t$ with $B_i^t = (dL^t/dy_i^t)(\partial y_i^t/\partial s_i^t)$. This is a linear state-space model even when the neuron itself is highly nonlinear, because the nonlinearity lives inside the Jacobians, which are linear first-order approximations. HYPR splits the input into subsequences, computes state-transition products $\phi^{\lambda:t}$ and back-accumulated vectors $q^t$ with associative scans in parallel, and combines them to form the cumulative approximate parameter gradient $[\tilde{\nabla}_{\theta_i}]_{1:\lambda} = q_i^0 e_i^0 + \sum_{t=1}^{\lambda} q_i^t \delta_i^t$. The result is mathematically equivalent to fully online e-prop but with the per-step materialization of large eligibility matrices replaced by parallel scans over low-rank factors. The paper's discovery is that this equivalence holds for a wide class of spiking neuron models, and that with oscillatory models the accuracy gap to BPTT nearly disappears.

Load-bearing premise

The claim that HYPR nearly matches BPTT rests on the BPTT baselines being comparably strong, and the authors report that they could not exactly reproduce the published BPTT accuracies with their own code.

Editorial extensions

If this is right

  • HYPR can train an RSNN on arbitrarily long input streams on a single GPU, because memory grows only with subsequence length $\lambda$, not with total sequence length.
  • Because its cumulative updates are equivalent to fully online e-prop, HYPR retains e-prop's infinite training context and online spirit while running up to 108 times faster in the reported medium-size setup.
  • Oscillatory neuron models such as BRF and SE-adLIF train particularly well under HYPR, reducing the accuracy gap to BPTT on SHD, ECG, and sMNIST to roughly one to three percentage points.
  • HYPR is the first approximate forward-learning RSNN method reported to work on sCIFAR and Pathfinder-E, though with a larger gap to BPTT there.
  • Recurrent connections are still used by HYPR-trained networks: removing them lowers accuracy for both BPTT and HYPR, so the method does not collapse to a feedforward learner despite ignoring recurrent gradient pathways.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same linear-SSM reformulation likely applies to other forward-gradient algorithms whose updates are built from eligibility-like quantities, so the associative-scan trick may extend beyond e-prop to broader families of online recurrent learning rules; the paper itself does not explore this.
  • A testable prediction is that other neuron models with rich internal or oscillatory dynamics will show similarly small HYPR-to-BPTT gaps, while simpler leaky or threshold-adaptation models will not; choosing neuron models for forward-gradient trainability could become a design principle.
  • If the equivalence and constant-memory properties hold at scale, HYPR could serve as a fast surrogate for studying e-prop-style learning on long sequences, and could motivate simplified variants tailored for neuromorphic hardware, though the paper notes that pure forward propagation may be simpler to implement on such hardware.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes HYPR, a segment-wise training algorithm for recurrent spiking neural networks that combines approximate forward eligibility propagation (e-prop-style) with parallel, scan-based computation over subsequences. The authors show that cumulative HYPR parameter updates are mathematically equivalent to cumulative e-prop updates (Appendix F), that eligibility products can be computed via associative scans in O(log λ) time per subsequence, and that memory use is constant with respect to the total sequence length. Experiments on a long-delay cue task report a 108× speedup over e-prop and constant memory, and benchmark runs on SHD, ECG, sMNIST, sCIFAR, and Pathfinder-E report BPTT-competitive accuracies, especially for oscillatory neuron models (BRF, SE-adLIF).

Significance. If the empirical claims hold, the paper makes a useful contribution: it addresses the sequentiality bottleneck of forward-propagation learning for RSNNs without sacrificing the constant-memory, infinite-context property of e-prop. The core algorithmic idea is elegant, the equivalence derivation in Appendix F is sound, and the availability of code plus 5-seed evaluations strengthens reproducibility. The main unresolved issue is the controlled comparison to BPTT, which is load-bearing for the headline claim of an unprecedentedly low performance gap; that issue is fixable within the manuscript's scope.

major comments (3)
  1. [Section 5.2; Appendix M.1; Tables A4–A7] The HYPR-versus-BPTT benchmark comparison is not controlled. Appendix M.1 states that BPTT uses the sum-of-softmax loss where applicable, while HYPR is forced to use a summative per-timestep cross-entropy because sum-of-softmax has no per-timestep decomposition. In addition, Tables A4–A7 report different hyperparameters for HYPR and BPTT, e.g., ECG SE-adLIF learning rate 0.01 vs 0.005 and batch size 64 vs 32 (Table A5), sMNIST BRF learning rate 0.1 vs 0.01 (Table A6), and sCIFAR t0 500 vs 300 and surrogate DG vs SLAYER (Table A7). The reported accuracy gap therefore mixes the algorithm choice with a change of loss objective and with different tuning. Since the 'unprecedentedly low gap' claim is a central empirical contribution, the authors should report a matched comparison, at minimum BPTT with the same per-timestep loss, and ideally both algorithms under the same loss, surrogate, learning rate, and schedule.
  2. [Table 1 footnote] The ALIF BPTT baselines are not reproduced: the published accuracies are 90.4/85.9/98.7, while the reproduced BPTT accuracies are 85.61/84.24/97.57. Since HYPR is compared against the authors' own reproduced BPTT numbers, the statement that HYPR is 'mostly on par' with BPTT and the claim of an unprecedentedly low gap may be overstated. The footnote argues that the discrepancy is not critical for a relative comparison, but it is critical for the specific claim that approximate forward learning approaches BPTT; the authors should either reproduce the published baselines or explicitly quantify the HYPR-to-published-BPTT gap.
  3. [Section 5.2; Appendix M.2] The subsequence length λ is not reported for any benchmark experiment, and no choice of λ is listed in Tables A4–A7. This matters for two reasons. First, λ controls the trade-off between parallelism and memory, so the claimed constant-memory property is only meaningful if λ is fixed and reported. Second, without λ, a reader cannot tell whether the reported accuracies were obtained in a near-online configuration or with a subsequence equal to the full sequence, in which case HYPR would reduce to a parallelized approximate BPTT variant. Please report λ, and the number of subsequences, for every benchmark configuration.
minor comments (4)
  1. [Introduction] The phrase 'complex neuon state dynamics' contains a typo and should read 'complex neuron state dynamics'.
  2. [Appendix F] Equation (A11) uses the notation Δθti for the cumulative approximate parameter gradient, whereas the main text and Eq. (6) use ∇~θti; please unify the notation.
  3. [Appendix G] The associative-scan construction is defined on the reversed sequence [pλ,...,p0]; the text should state explicitly how the returned sequence is ordered so that the implementation can be checked against Eq. (A12).
  4. [Appendix M.4] The code repository is referenced, but there is no version tag or commit hash; adding one would make the reproducibility claim verifiable.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: HYPR's core is an algebraic reformulation of e-prop with external benchmark comparisons; flagged caveats affect empirical comparison fairness, not circularity.

full rationale

HYPR's claimed derivation chain is algebraic, not circular. It starts from the e-prop eligibility recurrence (Eq. 4), rewrites it as the linear SSM of Eqs. (5)-(6), unrolls the eligibility matrix (Eqs. (9)-(11)), and reorders the cumulative APG sum in Appendix F to obtain the backward-scan form of Eqs. (13)-(15) with q_t defined recursively in Eq. (15). Each of these steps is an equality manipulation from stated definitions; the equivalence to fully-online e-prop is proven in-paper, not imported by citation. The runtime/memory claims are measured against actual implementations (Fig. 2), not derived from fitted quantities. The self-citations to e-prop [12] and SE-adLIF [15] supply the baseline algorithm and neuron models, but the central equivalence proof would hold for any model in the class of Eq. (1)-(2), so the self-citations are not load-bearing. Passages that a reviewer must weigh are present: the Table 1 asterisk admits the ALIF BPTT baselines were not reproduced; Appendix M.1 admits different loss functions were used for BPTT (sum-of-softmax) vs HYPR (per-timestep CE); and lambda is not reported for the benchmarks. These are real threats to the strength of the 'low gap' empirical claim and to the claim that HYPR is evaluated online, but they are comparison-fairness issues, not instances of a prediction reducing to its inputs by construction. Under the stated hard rules, such baseline-quality concerns belong to correctness risk rather than circularity. Accordingly no circular step is identified; the score of 1 reflects the presence of minor self-citation without load-bearing role.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The algorithm introduces no new physical entities or fitted constants; all free parameters are standard machine-learning hyperparameters. The central derivation relies on standard associative-scan mathematics and standard SNN modeling assumptions. No invented entities are postulated.

free parameters (5)
  • Subsequence length lambda = not reported for benchmark experiments
    Chosen trade-off between memory O(lambda) and parallelization efficiency. Affects speed and memory, not correctness, but its absence in Tables A4-A7 complicates reproduction.
  • Learning rates per algorithm = varies, e.g., BRF sMNIST BPTT 0.1 vs HYPR 0.01; SE-adLIF ECG BPTT 0.01 vs HYPR 0.005
    Hand-tuned separately for HYPR and BPTT in several experiments (Appendix M.2), which could bias the relative performance comparison.
  • t0 (ignored initial time steps) = e.g., sCIFAR BPTT 500 vs HYPR 300
    Changes the temporal window of the loss and can affect reported accuracy; differs between algorithms in some tasks.
  • Surrogate gradient function = e.g., sCIFAR BRF BPTT double Gaussian vs HYPR SLAYER
    Surrogate choice affects gradient quality and can alter performance; different choices across algorithms make comparisons less clean.
  • Gradient clipping magnitude = e.g., ECG BRF BPTT none vs HYPR 1.0
    Clipping differs between algorithms in several runs, another hand-chosen factor that could influence final accuracy.
assumptions (4)
  • standard math Associativity of state-transition matrix products and correctness of the associative scan algorithm.
    Used in Eqs. (9)-(14) and Appendix G to parallelize accumulation of eligibility and q-vector recurrences; Blelloch's scan requires an associative operator, which matrix multiplication is.
  • domain assumption Surrogate gradient approximation replaces the non-differentiable Heaviside output function.
    Section 3.1 and Appendix M.1; standard practice in SNN training because the true derivative of a spike is undefined or zero almost everywhere.
  • domain assumption Neuron state transition f is differentiable with respect to state and input so Jacobians can be computed by autodiff.
    Appendix E computes Jacobians via jac() for all neuron models; spike resets are handled through surrogate gradients, making the dynamics effectively differentiable.
  • domain assumption The loss is a summative per-timestep loss L = sum_t L_t, allowing the decomposition in Eq. (12).
    Appendix M.1 states that the sum-of-softmax loss is not compatible with HYPR, so all HYPR experiments use a per-timestep CE loss. This restricts HYPR to losses decomposable over time.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networks." pith.science (2026). https://pith.science/paper/57NYZEY6

@misc{pith2026250614464,
  author       = {Pith},
  title        = {Pith review of: A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/57NYZEY6}},
  note         = {Machine review of arXiv:2506.14464}
}
read the original abstract

Recurrent spiking neural networks (RSNNs) can be implemented very efficiently in neuromorphic systems. Nevertheless, training of these models with powerful gradient-based learning algorithms is mostly performed on standard digital hardware using Backpropagation through time (BPTT). However, BPTT has substantial limitations. It does not permit online training and its memory consumption scales linearly with the number of computation steps. In contrast, learning methods using forward propagation of gradients operate in an online manner with a memory consumption independent of the number of time steps. These methods enable SNNs to learn from continuous, infinite-length input sequences. Yet, slow execution speed on conventional hardware as well as inferior performance has hindered their widespread application. In this work, we introduce HYbrid PRopagation (HYPR) that combines the efficiency of parallelization with approximate online forward learning. Our algorithm yields high-throughput online learning through parallelization, paired with constant, i.e., sequence length independent, memory demands. HYPR enables parallelization of parameter update computation over the sub sequences for RSNNs consisting of almost arbitrary non-linear spiking neuron models. We apply HYPR to networks of spiking neurons with oscillatory subthreshold dynamics. We find that this type of neuron model is particularly well trainable by HYPR, resulting in an unprecedentedly low task performance gap between approximate forward gradient learning and BPTT.

Figures

Figures reproduced from arXiv: 2506.14464 by the authors.

Figure 1
Figure 1. a Generic neuron model framework considered in this work. We differentiate between intra-layer neuron recurrence through explicit recurrent weighted connections (magenta) and im￾plicit recurrence through the state-to-state transition (blue). b The local gradient [ds t i /dθi ] local is given by the pathway through the states of neuron i without considering indirect influence through outputs of other neurons. The mag… view at source ↗
Figure 2
Figure 2. Comparison of execution time and memory usage for BPTT and HYPR on the cue task. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 35 canonical work pages

  1. [1]

    Networks of spiking neurons: the third generation of neural network models

    Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural Networks, 10(9):1659–1671, 1997

    work page 1997

  2. [2]

    Neuronal dynamics: From single neurons to networks and models of cognition

    Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition . Cambridge University Press, 2014

    2014

  3. [3]

    A review of spiking neuromorphic hardware communication systems

    Aaron R Young, Mark E Dean, James S Plank, and Garrett S Rose. A review of spiking neuromorphic hardware communication systems. IEEE Access, 7:135606–135620, 2019

    work page 2019

  4. [4]

    Combining recurrent, convolutional, and continuous-time models with linear state space layers

    Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher R´e. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in Neural Information Processing Systems , 34:572–585, 2021

    work page 2021

  5. [5]

    Hippo: Recurrent mem- ory with optimal polynomial projections

    Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher R ´e. Hippo: Recurrent mem- ory with optimal polynomial projections. Advances in Neural Information Processing Systems, 33:1474–1487, 2020

    work page 2020

  6. [6]

    Diagonal state spaces are as effective as struc- tured state spaces

    Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are as effective as struc- tured state spaces. Advances in Neural Information Processing Systems , 35:22982–22994, 2022

    work page 2022

  7. [7]

    Resurrecting recurrent neural networks for long sequences

    Antonio Orvieto, Samuel L Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De. Resurrecting recurrent neural networks for long sequences. In Inter- national Conference on Machine Learning , pages 26670–26698. PMLR, 2023

    work page 2023

  8. [8]

    Parallel spiking unit for efficient training of spiking neural networks

    Yang Li, Yinqian Sun, Xiang He, Yiting Dong, Dongcheng Zhao, and Yi Zeng. Parallel spiking unit for efficient training of spiking neural networks. In 2024 International Joint Conference on Neural Networks (IJCNN) , pages 1–8. IEEE, 2024

    work page 2024

Show all 43 references
  1. [9]

    PRF: Parallel resonate and fire neuron for long sequence learning in spiking neural networks

    Yulong Huang, Zunchang Liu, Changchun Feng, Xiaopeng Lin, Hongwei Ren, Haotian Fu, Yue Zhou, Hong Xing, and Bojun Cheng. PRF: Parallel resonate and fire neuron for long sequence learning in spiking neural networks. arXiv preprint arXiv:2410.03530, 2024

    arXiv

  2. [10]

    Channel-wise parallelizable spiking neuron with multiplication- free dynamics and large temporal receptive fields

    Peng Xue, Wei Fang, Zhengyu Ma, Zihan Huang, Zhaokun Zhou, Yonghong Tian, Timoth ´ee Masquelier, and Huihui Zhou. Channel-wise parallelizable spiking neuron with multiplication- free dynamics and large temporal receptive fields. arXiv preprint arXiv:2501.14490, 2025

    2025

  3. [11]

    P-spikessm: Harnessing probabilistic spiking state space models for long-range dependency tasks

    Malyaban Bal and Abhronil Sengupta. P-spikessm: Harnessing probabilistic spiking state space models for long-range dependency tasks. In The Thirteenth International Conference on Learning Representations (ICLR), 2025

    2025

  4. [12]

    A solution to the learning dilemma for recurrent networks of spiking neurons

    Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Leg- enstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11(1):3625, July 2020

    2020

  5. [13]

    Williams and David Zipser

    Ronald J. Williams and David Zipser. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks. Neural Computation, 1(2):270–280, June 1989

    1989

  6. [14]

    ReckOn: A 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales

    Charlotte Frenkel and Giacomo Indiveri. ReckOn: A 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales. In IEEE International Solid-State Circuits Conference (ISSCC) , volume 65, pages 1–3, 2022

    2022

  7. [15]

    Advanc- ing spatio-temporal processing in spiking neural networks through adaptation

    Maximilian Baronig, Romain Ferrand, Silvester Sabathiel, and Robert Legenstein. Advanc- ing spatio-temporal processing in spiking neural networks through adaptation. arXiv preprint arXiv:2408.07517, 2025

    2025 arXiv

  8. [16]

    A surrogate gradient spiking baseline for speech com- mand recognition

    Alexandre Bittar and Philip N Garner. A surrogate gradient spiking baseline for speech com- mand recognition. Frontiers in Neuroscience, 16:865897, 2022

    2022

  9. [17]

    Balanced resonate-and- fire neurons

    Saya Higuchi, Sebastian Kairat, Sander Bohte, and Sebastian Otte. Balanced resonate-and- fire neurons. In International Conference on Machine Learning (ICML) , pages 18305–18323. PMLR, 2024. 24

    2024

  10. [18]

    An efficient gradient-based algorithm for on-line training of recurrent network trajectories

    Ronald J Williams and Jing Peng. An efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural Computation, 2(4):490–501, 1990

    1990

  11. [19]

    Unbiased online recurrent optimization

    Corentin Tallec and Yann Ollivier. Unbiased online recurrent optimization. In International Conference On Learning Representation (ICLR) , 2018

    2018

  12. [20]

    Approximating real-time recurrent learning with random kronecker factors

    Asier Mujika, Florian Meier, and Angelika Steger. Approximating real-time recurrent learning with random kronecker factors. Advances in Neural Information Processing Systems, 31, 2018

    2018

  13. [21]

    Practical real time recurrent learning with a sparse approximation

    Jacob Menick, Erich Elsen, Utku Evci, Simon Osindero, Karen Simonyan, and Alex Graves. Practical real time recurrent learning with a sparse approximation. InInternational Conference On Learning Representation (ICLR), 2021

    2021

  14. [22]

    David Silver, Anirudh Goyal, Ivo Danihelka, Matteo Hessel, and H. V . Hasselt. Learning by directional gradient descent. In International Conference On Learning Representation (ICLR), 2022

    2022

  15. [23]

    Online training through time for spiking neural networks.Advances in Neural Information Processing Systems, 35:20717–20730, 2022

    Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Online training through time for spiking neural networks.Advances in Neural Information Processing Systems, 35:20717–20730, 2022

    2022

  16. [24]

    A fixed size storage o(n3) time complexity learning algorithm for fully recurrent continually running networks

    J ¨urgen Schmidhuber. A fixed size storage o(n3) time complexity learning algorithm for fully recurrent continually running networks. Neural Computation, 4(2):243–248, 1992

    1992

  17. [25]

    Exploring the promise and limits of real-time recurrent learning

    Kazuki Irie, Anand Gopalakrishnan, and J ¨urgen Schmidhuber. Exploring the promise and limits of real-time recurrent learning. In The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  18. [26]

    Online learning of long-range dependencies

    Nicolas Zucchet, Robert Meier, Simon Schug, Asier Mujika, and Joao Sacramento. Online learning of long-range dependencies. Advances in Neural Information Processing Systems , 36:10477–10493, 2023

    2023

  19. [27]

    Parallel Spiking Neurons with High Efficiency and Abil- ity to Learn Long-term Dependencies

    Wei Fang, Zhaofei Yu, Zhaokun Zhou, Ding Chen, Yanqi Chen, Zhengyu Ma, Timoth ´ee Masquelier, and Yonghong Tian. Parallel Spiking Neurons with High Efficiency and Abil- ity to Learn Long-term Dependencies. Advances in Neural Information Processing Systems , 36:53674–53687, 2023

    2023

  20. [28]

    Esser, Paul A

    Steven K. Esser, Paul A. Merolla, John V . Arthur, Andrew S. Cassidy, Rathinakumar Ap- puswamy, Alexander Andreopoulos, David J. Berg, Jeffrey L. McKinstry, Timothy Melano, Davis R. Barch, Carmelo di Nolfo, Pallab Datta, Arnon Amir, Brian Taba, Myron D. Flickner, and Dharmendr...

    2016

  21. [29]

    Long short-term memory and learning-to-learn in networks of spiking neurons

    Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-learn in networks of spiking neurons. Advances in Neural Information Processing Systems, 31, 2018

    2018

  22. [30]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  23. [31]

    Legendre memory units: Continuous-time representation in recurrent neural networks

    Aaron V oelker, Ivana Kaji´c, and Chris Eliasmith. Legendre memory units: Continuous-time representation in recurrent neural networks. Advances in Neural Information Processing Sys- tems, 32, 2019

    2019

  24. [32]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  25. [33]

    Blelloch

    Guy E. Blelloch. Prefix sums and their applications. Technical Report CMU-CS-90-190, School of Computer Science, Carnegie Mellon University, November 1990

    1990

  26. [34]

    Simplified state space layers for sequence modeling

    Jimmy TH Smith, Andrew Warrington, and Scott Linderman. Simplified state space layers for sequence modeling. In The Eleventh International Conference on Learning Representations (ICLR), 2023. 25

    2023

  27. [35]

    Accurate and efficient time-domain classi- fication with adaptive spiking recurrent neural networks

    Bojian Yin, Federico Corradi, and Sander M Boht´e. Accurate and efficient time-domain classi- fication with adaptive spiking recurrent neural networks. Nature Machine Intelligence, 3(10): 905–913, 2021

    2021

  28. [36]

    The Hei- delberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks

    Benjamin Cramer, Yannik Stradmann, Johannes Schemmel, and Friedemann Zenke. The Hei- delberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems , 33(7):2744–2757, 2022

    2022

  29. [37]

    A database for evaluation of algorithms for measurement of qt and other waveform intervals in the ecg

    Pablo Laguna, Roger G Mark, A Goldberg, and George B Moody. A database for evaluation of algorithms for measurement of qt and other waveform intervals in the ecg. In Computers in Cardiology, pages 673–676. IEEE, 1997

    1997

  30. [38]

    The MNIST database of handwritten digits

    Yann LeCun. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/, 1998

    1998

  31. [39]

    Long range arena: A benchmark for efficient transformers

    Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler. Long range arena: A benchmark for efficient transformers. arXiv preprint arXiv:2011.04006, 2020

    2011 arXiv

  32. [40]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009

    2009

  33. [41]

    Slayer: Spike layer error reassignment in time

    Sumit B Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. Ad- vances in Neural Information Processing Systems , 31, 2018

    2018

  34. [42]

    SGDR: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations (ICLR) , 2022

    2022

  35. [43]

    JAX: composable transformations of Python+NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http: //github.com/jax-ml/jax. 26

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.