REVIEW 3 major objections 5 minor 41 references
Three factor delay learning rules for spiking neural networks
T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read This paper proposes three-factor learning rules that let LIF spiking networks learn synaptic and axonal delays online, achieving accuracy close to offline backpropagation on speech recognition tasks.
desk verdict A genuinely useful online delay-learning rule for SNNs, but the printed delay gradient has a sign/notation error that must be corrected. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the three-factor update in Equation 8: the delay gradient for a synapse decomposes into a top-down error signal and a local eligibility trace that recursively tracks the sensitivity of the neuron's output to the delay. Because the delayed spike is not differentiable, the paper replaces the Dirac spike train in Equation 10 with a Gaussian kernel in Equation 11 and uses its derivative in Equation 12; this surrogate supplies the missing gradient and is only used during learning, not inference. The recursive eligibility trace allows the computation to happen online with a ring buffer, despite the kernel being nominally non-causal.
What would settle it
Run the same training pipeline with a causal exponential or truncated causal Gaussian kernel of matched width replacing Eq. 11; if delays still learn and accuracy gains survive, the method does not depend on non-causal smoothing as the paper argues. Conversely, verify Eq. 12 by finite differences on Eq. 11; if the sign or normalization is wrong, the reported gradient is not the one being implemented.
Extended reading notes
Core claim
The central discovery is that delay parameters — both synaptic delays at individual connections and axonal delays at virtual input neurons — can be updated online using a three-factor rule whose eligibility trace is computed with a Gaussian surrogate for the spike train. Although shifting a spike by a delay is discontinuous in the delay parameter, representing the delayed spike as a Gaussian kernel makes the derivative well-defined; the product of this derivative with the surrogate spiking nonlinearity and the error signal gives a gradient-equivalent update. In experiments, this local online rule closely matches offline backpropagation for fully connected feedforward and recurrent networks,
Load-bearing premise
The Gaussian surrogate in Eq. 11 and its derivative in Eq. 12 correctly capture the sensitivity of a postsynaptic neuron to a shift in spike timing, even though the true spike-delay relationship is a discontinuous shift and the surrogate kernel is non-causal; causal exponential kernels did not support learning.
Editorial extensions
If this is right
- On the SHD speech dataset, online delay learning reaches 92.64% test accuracy, within 0.15 percentage points of the 92.79% offline backpropagation baseline under the same network.
- Adding learnable delays improves accuracy by up to 20% over weights-only networks; jointly learning weights and delays adds up to 14% at comparable parameter counts, with the largest gains in small and sparse models.
- Because delay updates are computed from recursively maintained eligibility traces rather than unrolled sequences, learning can proceed sample-by-sample in real time, removing update locking.
- A sparse model with learnable delays can match a denser weights-only model's accuracy with a 6.6x smaller model and 67% lower inference latency in the comparison reported.
- On-chip memory planning changes: axonal delays add about 8.3% memory overhead, synaptic delays about 57.7%, but the delays' accuracy gains pay off once the network is sparse enough.
Reading between the lines
- The consistent failure of causal kernels reported in the paper suggests a testable design principle: non-causal smoothing may be what makes delay gradients informative, so hardware that wants causal operation would need to approximate or accept a learning-accuracy penalty.
- If axonal delay updates accumulate error because each axonal parameter pools contributions from many postsynaptic neurons, then switching to time-to-first-spike coding, where delays attach to hidden neurons rather than input neurons, could make those updates local and more accurate — a natural extension.
- Across setups, accuracy on SHD saturates around 92.5% even when SRNNs double parameters; this hints that delay learning is supplying the temporal feature extraction that additional recurrent capacity would otherwise provide, and that scaling to harder tasks would likely need more layers rather than more parameters per layer.
- The memory-overhead figures imply a concrete design rule: delay learning is memory-favorable only in sparse regimes; testing at which sparsity each delay type breaks even (43% for axonal, 74% for synaptic per the paper) turns the result into an engineering guideline.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes three-factor learning rules for simultaneously learning synaptic and axonal delays in single-hidden-layer LIF-based SNNs and SRNNs. Delays are treated as parameters of a Gaussian surrogate spike kernel; the eligibility trace is computed recursively from the derivative of this kernel with respect to the delay, and a top-down error signal from the readout layer provides the third factor. The method is evaluated on the SHD and SSC keyword-spotting datasets, with comparisons to a DCLS/BPTT offline baseline, weights-only baselines, fixed-delay controls, and sparse-network variants. The paper reports that delays improve accuracy by up to 20%, that joint weight-delay learning yields up to 14% higher accuracy at comparable parameter counts, and that the online method matches offline BPTT to within 0.15% on SHD.
Significance. If the central derivation is correct and reproducible, this is a timely and practically relevant contribution to online, on-device learning in neuromorphic systems. The experimental design has several strengths: results are reported with 95% confidence intervals from repeated runs; the online method is compared directly to an offline DCLS/BPTT baseline; fixed-delay controls isolate the contribution of delay learning; and the memory-footprint analysis for on-chip implementation is useful. However, the central delay-gradient equation contains a sign/notation error, and a key kernel hyperparameter is not reported. These issues must be resolved before the claimed results can be independently reproduced from the text.
major comments (3)
- [Eq. (12), Sec. III-D] Equation (12) is stated as the derivative of the Gaussian surrogate with respect to the delay parameters, but as printed the left-hand side is d x(t-D_ji)/dt and the right-hand side is negative of the time derivative of the Gaussian. Since x(t-D_ji) = G(t - t_k - D_ji), the correct derivative with respect to D_ji is + (t - t_k - D_ji)/(sqrt(2 pi) sigma^3) * exp(-(t - t_k - D_ji)^2/(2 sigma^2)), which is the negative of the printed right-hand side. This derivative enters the eligibility trace in Eq. (9) and hence the update rule in Eq. (8). A reader implementing Eq. (12) exactly as written would update delays in the opposite direction of the intended gradient. The reported experimental results must therefore have been obtained with a corrected derivative. This is a load-bearing reproducibility defect in the central learning rule; please correct Eq. (12) and explicitly confirm the sign use
- [Sec. III-D / IV-B] The Gaussian width sigma in Eq. (11) is never specified. The magnitude and direction of all delay gradients depend directly on sigma, and the paper reports that causal kernels 'did not support effective learning', so the choice of sigma is not a neutral detail. Without a reported value for sigma (or a sigma schedule, if used), the experiments in Table I cannot be reproduced from the text. Please add the sigma value used for each experiment and, ideally, a short sensitivity analysis.
- [Sec. VI / Table I] The paper claims a 6.6x model size reduction and 67% lower inference latency relative to the DCLS baseline, but Table I shows the same FC-128 configuration with 32.3k parameters for both the BP baseline (row 1) and the online method (row 4) in the synaptic-delay condition. The basis for the 6.6x reduction is therefore unclear from the reported data. Please specify the exact baseline architecture and parameter count used for this claim, or revise the claim so that it is directly supported by Table I or a companion table.
minor comments (5)
- [Eq. (9)] The notation in Eq. (9) is inconsistent: the eligibility trace is written as e_t_ji = partial z_t_j / partial v_t'_j, but the prime on t' is not defined and the subsequent bracket includes partial v_t_j / partial D_ji. Please rewrite with consistent time indices, e.g., partial z_t_j / partial v_t_j times the recursive trace.
- [Eq. (10)-(11)] Equation (10) writes x^{t-D_ji}_i as a sum of Dirac deltas, but in the discrete-time dynamics of Eq. (3) the input is a sequence indexed by time. Please clarify the relationship between the continuous-time kernel notation and the discrete-time update used in the implementation.
- [Table I / Sec. V] The text states that the weights-only SRNN baseline is 85.72%, while Table I row 9 reports 85.77% +/- 0.94%. Please correct the inconsistency.
- [Sec. VI-A] There is a typo, 'Section Section VI', and the discussion of ring-buffer sizes would benefit from a more precise explanation of when the buffer depth is 'kernel length' versus 'maximum delay'.
- [Sec. IV-B] The paper attributes the large weights-only performance gap between online and DCLS/BPTT to analytical LIF dynamics versus Euler integration, but no direct comparison or citation is given for that claim in this context. Please add a reference or a small control experiment to support this attribution.
Circularity Check
No significant circularity: the delay-learning result is validated against external offline BPTT baselines and fixed-delay controls.
full rationale
The paper's central claim is that the proposed three-factor rules allow online learning of synaptic and axonal delays and match offline backpropagation accuracy. The delay-gradient computation in Sec. III-D uses a Gaussian surrogate (Eq. 11) and its derivative (Eq. 12). This surrogate is an explicit modeling choice, justified by comparative preliminary experiments (Gaussian vs. causal exponential vs. double-exponential kernels), not a term defined in terms of the target result. The learned delays Dji are independently parameterized, clamped, and evaluated against (1) a weights-only baseline, (2) fixed-delay controls, and (3) an external DCLS/BPTT offline baseline from prior work; Table I's headline accuracies are therefore not fitted to the targets. No load-bearing step relies on the present authors' prior work: the reference list contains no self-citations by Vassallo or TaheriNejad, and e-prop is cited as an external algorithm (Ref. [28]) whose formulation is extended, not assumed. The only notable issue is that Eq. (12) is printed as a time derivative d x/dt with a negative sign while the surrounding text says it is the derivative with respect to delay Dji; that is a correctness/reproducibility concern (the correct delay derivative has the opposite sign), but it does not make the claimed result equivalent to its inputs by construction. The non-causal kernel and empirical hyperparameter selection are acknowledged limitations, not circularity. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (5)
- Gaussian kernel width σ =
not reported
- Delay learning rate =
1e-2
- Weight learning rate =
1e-4
- Maximum delay Dmax =
25 timesteps (250 ms)
- Surrogate gradient width γ_pd =
0.3
assumptions (4)
- domain assumption LIF membrane dynamics with reset-by-subtraction (Eq. 5) are a sufficient neuron model for delay learning.
- ad hoc to paper Dirac-delta spike trains can be replaced by a Gaussian kernel to obtain delay gradients (Eq. 11).
- domain assumption The eligibility-trace recursion in Eq. (9) yields gradients equivalent enough to BPTT for delay parameters.
- domain assumption Non-causal kernel information can be made available in an online setting via ring buffers.
Cite this review
Pith. "Pith review of Three factor delay learning rules for spiking neural networks." pith.science (2026). https://pith.science/paper/PGOKCA7P
@misc{pith2026260100668,
author = {Pith},
title = {Pith review of: Three factor delay learning rules for spiking neural networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/PGOKCA7P}},
note = {Machine review of arXiv:2601.00668}
}
read the original abstract
Spiking Neural Networks (SNNs) are dynamical systems that operate on spatiotemporal data, yet their learnable parameters are often limited to synaptic weights, contributing little to temporal pattern recognition. Learnable parameters that delay spike times can improve classification performance in temporal tasks, but existing methods rely on large networks and offline learning, making them unsuitable for real-time operation in resource-constrained environments. In this paper, we introduce synaptic and axonal delays to leaky integrate and fire (LIF)-based feedforward and recurrent SNNs, and propose three-factor learning rules to simultaneously learn delay parameters online. We employ a smooth Gaussian surrogate to approximate spike derivatives exclusively for the eligibility trace calculation, and together with a top-down error signal determine parameter updates. Our experiments show that incorporating delays improves accuracy by up to 20% over a weights-only baseline, and for networks with similar parameter counts, jointly learning weights and delays yields up to 14% higher accuracy. On the SHD speech recognition dataset, our method achieves similar accuracy to offline backpropagation-based approaches. Compared to state-of-the-art methods, it reduces model size by 6.6x and inference latency by 67%, with only a 2.4% drop in classification accuracy. Our findings benefit the design of power and area-constrained neuromorphic processors by enabling on-device learning and lowering memory requirements.
Figures
Reference graph
Works this paper leans on
-
[1]
Bittar and P
A. Bittar and P. N. Garner. A surrogate gradient spiking baseline for speech command recognition.Frontiers in Neuroscience, 16:865897, August 2022
2022
-
[2]
Yinet al.Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks.Nature Machine Intelligence, 3(10):905–913, October 2021
B. Yinet al.Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks.Nature Machine Intelligence, 3(10):905–913, October 2021
2021
-
[3]
Deckerset al.Co-learning synaptic delays, weights and adaptation in spiking neural networks.Frontiers in Neuroscience, 18:1360300, April 2024
L. Deckerset al.Co-learning synaptic delays, weights and adaptation in spiking neural networks.Frontiers in Neuroscience, 18:1360300, April 2024
2024
-
[4]
Hammouamriet al.Learning delays in spiking neural networks using dilated convolutions with learnable spacings
I. Hammouamriet al.Learning delays in spiking neural networks using dilated convolutions with learnable spacings. InThe Twelfth International Conference on Learning Representations, 2024
2024
-
[5]
Sunet al.Learnable axonal delay in spiking neural networks improves spoken word recognition.Frontiers in Neuroscience, 17:1275944, November 2023
P. Sunet al.Learnable axonal delay in spiking neural networks improves spoken word recognition.Frontiers in Neuroscience, 17:1275944, November 2023
2023
-
[6]
E. O. Neftciet al.Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks.IEEE Signal Processing Magazine, 36(6):51–63, November 2019
2019
-
[7]
B. Crameret al.The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks.IEEE Transactions on Neural Networks and Learning Systems, 33(7):2744–2757, July 2022
2022
-
[8]
B. L. Sabatini and W. G. Regehr. Timing of neurotransmission at fast synapses in the mammalian brain.Nature, 384(6605):170–172, November 1996
1996
Show all 41 references
-
[9]
Debanneet al.Axon Physiology.Physiological Reviews, 91(2):555– 602, April 2011
D. Debanneet al.Axon Physiology.Physiological Reviews, 91(2):555– 602, April 2011
2011
-
[10]
London and M
M. London and M. H ¨ausser. DENDRITIC COMPUTATION.Annual Review of Neuroscience, 28(1):503–532, July 2005
2005
-
[11]
S. B. Shrestha and G. Orchard. Slayer: spike layer error reassignment in time. InProceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, pp. 1419–1428, Red Hook, NY , USA, 2018. Curran Associates Inc
2018
-
[12]
Sunet al.Axonal Delay as a Short-Term Memory for Feed Forward Deep Spiking Neural Networks
P. Sunet al.Axonal Delay as a Short-Term Memory for Feed Forward Deep Spiking Neural Networks. InICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8932–8936, Singapore, Singapore, May 2022. IEEE
2022
-
[13]
Khalfaoui-Hassaniet al.Dilated convolution with learnable spacings
I. Khalfaoui-Hassaniet al.Dilated convolution with learnable spacings. InThe Eleventh International Conference on Learning Representations, 2023
2023
-
[14]
M ´esz´aroset al.Learning Delays Through Gradients and Structure: Emergence of Spatiotemporal Patterns in Spiking Neural Networks, November 2024
B. M ´esz´aroset al.Learning Delays Through Gradients and Structure: Emergence of Spatiotemporal Patterns in Spiking Neural Networks, November 2024. arXiv:2407.18917 [cs]
2024 arXiv
-
[15]
T. C. Wunderlich and C. Pehle. Event-based backpropagation can compute exact gradients for spiking neural networks.Scientific Reports, 11(1):12829, June 2021
2021
-
[16]
M´esz´aroset al.Efficient Event-based Delay Learning in Spiking Neural Networks, June 2025
B. M´esz´aroset al.Efficient Event-based Delay Learning in Spiking Neural Networks, June 2025. arXiv:2501.07331 [cs]
2025 arXiv
-
[17]
Nowotnyet al.Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks.Neuromorphic Computing and Engineering, 5(1):014001, March 2025
T. Nowotnyet al.Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks.Neuromorphic Computing and Engineering, 5(1):014001, March 2025
2025
-
[18]
Jaderberget al.Decoupled neural interfaces using synthetic gradients
M. Jaderberget al.Decoupled neural interfaces using synthetic gradients. In D. Precup and Y . W. Teh, editors,Proceedings of the 34th International Conference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pp. 1627–1635. PMLR, 06–11 Aug 2017
2017
-
[19]
R. J. Williams and D. Zipser. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks.Neural Computation, 1(2):270– 280, June 1989
1989
-
[20]
Benzinget al.Optimal Kronecker-sum approximation of real time recurrent learning
F. Benzinget al.Optimal Kronecker-sum approximation of real time recurrent learning. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 ofProceedings of Machine Learning Research, pp. 604–613. PMLR, 09...
2019
-
[21]
Tallec and Y
C. Tallec and Y . Ollivier. Unbiased online recurrent optimization. In International Conference on Learning Representations, 2018
2018
-
[22]
Gerstneret al.Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules
W. Gerstneret al.Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules. Frontiers in Neural Circuits, 12:53, July 2018
2018
-
[23]
D. E. Rumelhartet al.Learning representations by back-propagating errors.Nature, 323(6088):533–536, 1986
1986
-
[24]
Liaoet al.How important is weight symmetry in backpropagation? In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pp
Q. Liaoet al.How important is weight symmetry in backpropagation? In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pp. 1837–1844. AAAI Press, 2016
2016
-
[25]
Bi and M.-m
G.-q. Bi and M.-m. Poo. Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsy- naptic Cell Type.The Journal of Neuroscience, 18(24):10464–10472, December 1998
1998
-
[26]
Marschallet al.A unified framework of online learning algorithms for training recurrent neural networks.J
O. Marschallet al.A unified framework of online learning algorithms for training recurrent neural networks.J. Mach. Learn. Res., 21(1), January 2020
2020
-
[27]
Zenke and S
F. Zenke and S. Ganguli. SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks.Neural Computation, 30(6):1514–1541, June 2018
2018
-
[28]
Bellecet al.A solution to the learning dilemma for recurrent networks of spiking neurons.Nature Communications, 11(1):3625, July 2020
G. Bellecet al.A solution to the learning dilemma for recurrent networks of spiking neurons.Nature Communications, 11(1):3625, July 2020
2020
-
[29]
Kaiseret al.Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE).Frontiers in Neuroscience, 14:424, May 2020
J. Kaiseret al.Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE).Frontiers in Neuroscience, 14:424, May 2020
2020
-
[30]
Bohnstinglet al.Online Spatio-Temporal Learning in Deep Neural Networks.IEEE Transactions on Neural Networks and Learning Systems, pp
T. Bohnstinglet al.Online Spatio-Temporal Learning in Deep Neural Networks.IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, 2022
2022
-
[31]
L. Abbott. Lapicque’s introduction of the integrate-and-fire model neuron (1907).Brain Research Bulletin, 50(5-6):303–304, November 1999
1907
-
[32]
Gerstneret al
W. Gerstneret al. Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press, 1 edition, July 2014
2014
-
[33]
G ¨oltzet al.DelGrad: Exact event-based gradients in spiking networks for training delays and weights, December 2024
J. G ¨oltzet al.DelGrad: Exact event-based gradients in spiking networks for training delays and weights, December 2024. arXiv:2404.19165 [cs]
2024 arXiv
-
[34]
Wanget al.A Delay Learning Algorithm Based on Spike Train Kernels for Spiking Neurons.Frontiers in Neuroscience, 13:252, March 2019
X. Wanget al.A Delay Learning Algorithm Based on Spike Train Kernels for Spiking Neurons.Frontiers in Neuroscience, 13:252, March 2019
2019
-
[35]
P. Warden. Speech commands: A dataset for limited-vocabulary speech recognition.CoRR, abs/1804.03209, 2018
2018 arXiv
-
[36]
Fanget al.Spikingjelly: An open-source machine learning in- frastructure platform for spike-based intelligence.Science Advances, 9(40):eadi1480, 2023
W. Fanget al.Spikingjelly: An open-source machine learning in- frastructure platform for spike-based intelligence.Science Advances, 9(40):eadi1480, 2023
2023
-
[37]
Baroniget al.Advancing spatio-temporal processing through adaptation in spiking neural networks.Nature Communications, 16(1):5776, July 2025
M. Baroniget al.Advancing spatio-temporal processing through adaptation in spiking neural networks.Nature Communications, 16(1):5776, July 2025
2025
-
[38]
Sch ¨oneet al.Scalable Event-by-Event Processing of Neuromorphic Sensory Signals with Deep State-Space Models
M. Sch ¨oneet al.Scalable Event-by-Event Processing of Neuromorphic Sensory Signals with Deep State-Space Models. In2024 International Conference on Neuromorphic Systems (ICONS), pp. 124–131, Arlington, V A, USA, July 2024. IEEE
2024
-
[39]
Soydanet al.S7: Selective and Simplified State Space Layers for Sequence Modeling, October 2024
T. Soydanet al.S7: Selective and Simplified State Space Layers for Sequence Modeling, October 2024. arXiv:2410.03464 [cs]
2024 arXiv
-
[40]
Frenkel and G
C. Frenkel and G. Indiveri. ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescales. In2022 IEEE International Solid- State Circuits Conference (ISSCC), pp. 1–3, San Francisco, CA, USA, February
-
[41]
Davieset al.Loihi: A Neuromorphic Manycore Processor with On- Chip Learning.IEEE Micro, 38(1):82–99, January 2018
M. Davieset al.Loihi: A Neuromorphic Manycore Processor with On- Chip Learning.IEEE Micro, 38(1):82–99, January 2018
2018
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.