REVIEW 3 major objections 5 minor 14 references
Efficient Synaptic Delay Implementation in Digital Event-Driven AI Accelerators
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A two-FIFO circular queue makes synaptic-delay memory scale with activation sparsity, not network size.
desk verdict SCDQ is a genuine architectural novelty with sound queue-level math, but the paper's headline memory advantage over ring buffers is not demonstrated on its own measurements because activation sparsity α is never reported and measured memory on Seneca exceeds Loihi's in every tested network. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Shared Circular Delay Queue (SCDQ): a pair of FIFOs, the pre-processing queue (PRQ) and post-processing queue (POQ), arranged in a loop with a buffer swap triggered by an end-of-timestep event. Each event is tagged with a delay counter and may be forwarded, stored, or both, letting one presynaptic event exit at several different delay steps. The zero-skipping delay-forwarding filter maintains a binary matrix, $WVU$, that records whether any non-zero weight remains for a given presynaptic neuron and delay level, so useless events can be dropped before they consume queue capacity. This combination is the mechanism that turns delay memory from quadratic-in-$D$ scaling into $O(\alpha I D)$ scaling.
What would settle it
Compute the average activation density $\alpha$ of the trained 48-48-20 model on the Spiking Heidelberg Digits test set. If $\alpha > 0.25$, then the SCDQ stored-event count $\alpha \cdot 48 \cdot (2\cdot 60-1)$ would exceed the equivalent ring-buffer count for Loihi's configuration, and the claimed memory crossover would not hold for that model.
Extended reading notes
Core claim
The central claim is that SCDQ stores delayed synaptic events in two interconnected FIFOs, the pre-processing queue and post-processing queue, which swap buffers at each end-of-timestep. Incoming events carry a delay counter and can orbit the pair, exiting at multiple future timesteps, which supports per-synapse delays rather than only axonal delays. The memory overhead is $\alpha \cdot I \cdot (2D-1)$, scaling as $O(\alpha I D)$, versus $\tfrac{1}{2}\alpha I (D^2+D)$ for a shared delay queue. A pruning filter using a binary $WVU$ matrix drops events whose delayed synapses are all zero, so activation sparsity directly shrinks queue occupancy. Measured on Seneca, the circuit implementation of SCDQ adds 2–3% to inference energy and cuts latency several-fold compared with a software delay queue on the same core.
Load-bearing premise
The memory advantage over ring buffers depends on the trained models having activation density $\alpha$ at or below the crossover threshold (0.25 versus Loihi, 0.5 versus SpiNNaker), and the paper does not report $\alpha$ for its own models.
Editorial extensions
If this is right
- Memory for synaptic delays becomes a function of activation sparsity, so training procedures that encourage sparse firing directly reduce hardware cost.
- At activation density $\alpha \leq 0.25$, SCDQ uses less delay memory than Loihi-style ring buffers at 64 delay steps, and at $\alpha \leq 0.5$ less than SpiNNaker-style ring buffers.
- Axonal pruning gives larger hardware savings than per-synapse pruning alone, because pruning an entire axon lets events drop from the queue before their maximum delay.
- A circuit implementation of SCDQ improves energy per inference by 3.1–3.5× and latency by 3.5–4.3× over a software implementation on the same processor core, with the delay hardware contributing only 2–3% of inference energy.
- Sharing an SCDQ across layers mapped to one compute core makes delay memory depend on the number of cores, not the number of layers.
Reading between the lines
- The paper does not report the actual activation density $\alpha$ of its trained models, so the crossover claims against Loihi and SpiNNaker are conditional; measuring $\alpha$ per layer would turn them into a testable prediction.
- The single-FIFO pointer variant sketched in the paper would reduce memory to $\alpha \cdot I \cdot D$, roughly doubling the advantage; this is a natural next hardware target.
- If delay-trained models commonly reach $\alpha < 0.25$, SCDQ-type structures could become the default delay primitive in edge accelerators, making delay parameterization a standard model knob.
- Because SCDQ lets an event exit at several timepoints, it can express per-synapse delay patterns that axonal-only shared queues cannot, which may enable new delay-parameterized model families.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the Shared Circular Delay Queue (SCDQ), a two-FIFO circular structure for implementing synaptic delays in digital event-driven neuromorphic accelerators. It derives an analytical memory overhead of α·I·(2D−1) for SCDQ, compares it with the shared delay queue's O(α·I·D²) and with ring-buffer approaches, and reports a hardware implementation integrated into the Seneca processor. The authors evaluate four spiking network models on the SHD benchmark, reporting software/hardware fidelity, energy, latency, memory, and area, and conclude that the Delay IP adds only 2–3% of inference energy while improving latency compared to a software implementation.
Significance. If the SCDQ memory formula holds, the result is valuable: delay memory overhead becomes linear in the number of delay levels and presynaptic neurons, modulated by activation density, rather than quadratic in D. The hardware implementation, including the W V U pruning filter and the SRAM-based queue, is a concrete contribution, and the measured 2–3% energy overhead of the Delay IP is a useful, clearly quantified data point. The fidelity comparison between PyTorch and Seneca is a sound validation step, and the area reduction from flip-flop to SRAM memory (81%) is convincingly reported. However, the broader claims of memory, energy, and latency advantage over existing systems are not yet fully supported by the experiments as presented.
major comments (3)
- [Section V (Discussion), Table II] The claim that SCDQ beats Loihi's ring buffer when α≤0.25 is not validated empirically, because α is never reported for the four evaluated networks. The only indirect evidence is the SRAM sizing in Section IV-C: the 2048-word SRAM holds an observed maximum of 1596 events; for the 48-48-20 network with I=48 and D=60, the paper's own formula α·I·(2D−1) gives α≈0.28, just above the crossover threshold. This is consistent with Table II, where total Seneca memory exceeds Loihi's in all four networks. Please report α per model and per layer, and provide a controlled comparison of delay-queue memory only, rather than total core memory.
- [Section III-A, Section IV-C] The queue-capacity formula uses α as an average activation density, but a hardware queue must be sized for worst-case occupancy. The SRAM capacity is chosen from an observed maximum over a 100-sample test set; no backpressure, overflow handling, or upper bound on spike rate is described. If a burst exceeds the 2048-event capacity, the SCDQ will either drop events or stall the core, and neither behavior is discussed. The paper should either specify a hard spike-rate cap, provide a worst-case capacity analysis, or describe and validate the overflow policy.
- [Section IV-B, Table II] The energy and latency comparison with Loihi is not controlled: Loihi runs on 1 core with int8 quantization while Seneca runs on 3 cores with bfloat16. The reported 1.2–1.9× lower energy efficiency and the 'less than 0.5×' latency advantage therefore cannot be attributed to SCDQ versus ring buffers. Please provide a normalized comparison (e.g., per-core energy, energy per event, or a same-resource/same-precision mapping) before claiming that SCDQ improves latency or closes the energy gap.
minor comments (5)
- [Section III-A] The text contains 'SCDC' instead of 'SCDQ' and refers to 'Figure ??' for the block diagram; the figure reference should be resolved.
- [Section II] The Shared Delay Queue formula writes ∑_{d=1}^D(D−d) = ½(D²+D), but the sum from d=1 to D of (D−d) is D(D−1)/2; please correct the indexing or the closed form.
- [Section V] The symbol α is called 'activation sparsity' but α=1 denotes every neuron active; this is activation density, not sparsity. The text also contains the typo 'no spasity'.
- [Figures 5 and 6] The captions state that activations are shown 'on Seneca, Loihi and PyTorch', but the figures contain only PyTorch and Seneca panels; either add Loihi panels or correct the captions.
- [Table I] The 'Consistency against PyTorch' row for PyTorch is trivially 100%; clarify that this is self-consistency and define the consistency metric.
Circularity Check
No load-bearing circularity: the SCDQ memory formula is an analytic queue property, the cross-architecture comparisons use external Loihi/TrueNorth constraints, and the self-citations are background support; the unmeasured activation sparsity is an evidence gap, not a circular reduction.
full rationale
The paper's central analytical result, the SCDQ memory overhead alpha*I*(2D-1), is presented as a property of the queue's PRQ/POQ operation (Section III-A) and is used in Section V to compute crossover conditions against TrueNorth, Loihi, and SpiNNaker. These cross-architecture comparisons use external published constraints (TrueNorth 16 timesteps/256 neurons; Loihi 64 delays/48 neurons/8-bit weights), not quantities fitted in this paper, so the 'if alpha <= 0.25' lower-memory claim is a conditional arithmetic consequence rather than a fitted prediction. The self-citations, [6] (delay training/empirical study) and [12] (Seneca platform), support the training framework and the evaluation platform but are not the source of the memory-scaling derivation; [6] is an externally published empirical study, and [12] is the hardware used for measurements, so neither is load-bearing circular evidence. The main gap is that alpha is defined in Section II but never reported for the evaluated models, and Table II shows measured Seneca total memory exceeding Loihi's in all four networks; this makes the crossover condition unverified, but missing evidence is a correctness/risk concern, not an equivalence-by-construction. No step in the derivation reduces a predicted quantity to a fitted input or to a self-citation chain, so no circular step is identified.
Assumptions & free parameters
free parameters (1)
- activation sparsity α =
not reported; implied around 0.3-0.6 by queue usage in the 48-48-20 model
assumptions (4)
- domain assumption Synaptic delays can be modeled as D separate trainable synapses per pre-post pair (framework of Patino et al. [6]).
- domain assumption Activation sparsity α is a meaningful minimum sparsity that remains stable during inference, and the same α applies across all timesteps and layers in the capacity formula.
- domain assumption Netlist-based power estimation in a 22nm GlobalFoundry process with Cadence Joules is representative of real silicon energy to within the claimed 15%.
- domain assumption The event count in the queue is bounded by the allocated SRAM (2048 words) for the evaluated workloads.
Cite this review
Pith. "Pith review of Efficient Synaptic Delay Implementation in Digital Event-Driven AI Accelerators." pith.science (2026). https://pith.science/paper/AYNFFZH3
@misc{pith2026250113610,
author = {Pith},
title = {Pith review of: Efficient Synaptic Delay Implementation in Digital Event-Driven AI Accelerators},
year = {2026},
howpublished = {\url{https://pith.science/paper/AYNFFZH3}},
note = {Machine review of arXiv:2501.13610}
}
read the original abstract
Synaptic delay parameterization of neural network models have remained largely unexplored but recent literature has been showing promising results, suggesting the delay parameterized models are simpler, smaller, sparser, and thus more energy efficient than similar performing (e.g. task accuracy) non-delay parameterized ones. We introduce Shared Circular Delay Queue (SCDQ), a novel hardware structure for supporting synaptic delays on digital neuromorphic accelerators. Our analysis and hardware results show that it scales better in terms of memory, than current commonly used approaches, and is more amortizable to algorithm-hardware co-optimizations, where in fact, memory scaling is modulated by model sparsity and not merely network size. Next to memory we also report performance on latency area and energy per inference.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
F. Akopyan, J. Sawada, A. Cassidy, R. Alvarez-Icaza, J. Arthur, P. Merolla, N. Imam, Y . Nakamura, P. Datta, G. Nam, B. Taba, M. Beakes, B. Brezzo, J. B. Kuang, R. Manohar, W. P. Risk, B. Jackson, and D. S. Modha. Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE Transac- tions on Computer-Aided Design of In...
work page 2015
-
[2]
M. Davies, N Srinivasa, T-H. Lin, G. Chinya, Y . Cao, S.H. Choday, G. Dimou, P. Joshi, N. Imam, and S. Jain. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro , 38(1):82–99, 2018
work page 2018
-
[3]
M.M. Khan, D.R. Lester, L.A. Plana, A. Rast, X. Jin, E. Painkras, and S.B. Furber. Spinnaker: mapping neural networks onto a massively- parallel chip multiprocessor. In 2008 IEEE International Joint Con- ference on Neural Networks (IEEE World Congress on Computational Intelligence), pages 2849–2856. Ieee, 2008
work page 2008
-
[4]
C. Stoelzel, Y . Bereshpolova, J-M. Alonso, and H. Swadlow. Axonal conduction delays, brain state, and corticogeniculate communication. The Journal of Neuroscience , 37, 05 2017
work page 2017
-
[5]
J.A Starzyk, L. Maciura, and A. Horzyk. Associative memories with synaptic delays. IEEE Transactions on Neural Networks and Learning Systems, 31(1), 2020
work page 2020
-
[6]
A. Pati ˜no-Saucedo, A. Yousefzadeh, G. Tang, F. Corradi, B. Linares- Barranco, and M. Sifalakis. Empirical study on the efficiency of spiking neural networks with axonal delays, and algorithm-hardware benchmarking. In 2023 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 2023
work page 2023
- [7]
-
[8]
I. Hammouamri, I. Khalfaoui-Hassani, and T. Masquelier. Learning delays in spiking neural networks using dilated convolutions with learnable spacings. arXiv preprint arXiv:2306.17670 , 2023
arXiv 2023
Show all 14 references
-
[9]
X. Wang, X. Lin, and X. Dang. A delay learning algorithm based on spike train kernels for spiking neurons. Frontiers in Neuroscience, 13, 2019
2019
-
[10]
P. Sun, E. Eqlimi, Y . Chua, P. Devos, and D. Botteldooren. Adaptive axonal delays in feedforward spiking neural networks for accurate spoken word recognition. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023
2023
-
[11]
Lazzaro, J
J. Lazzaro, J. Wawrzynek, M. Mahowald, M. Sivilotti, and D. Gillespie. Silicon auditory processors as computer peripherals. IEEE Transactions on Neural Networks , 4(3):523–528, 1993
1993
-
[12]
G. Tang, K. Vadivel, Y . Xu, R. Bilgic, K. Shidqi, P Detterer, S. Traferro, M. Konijnenburg, M. Sifalakis, G-J. van Schaik, and A. Yousefzadeh. Seneca: building a fully digital neuromorphic processor, design trade- offs and challenges. Frontiers in Neuroscience, 17, 2023
2023
-
[13]
Morrison, C
A. Morrison, C. Mehring, T. Geisel, AD. Aertsen, and M. Diesmann. Advancing the boundaries of high-connectivity network simulation with distributed computing. Neural computation, 17(8), 2005
2005
-
[14]
Cramer, Y Stradmann, J Schemmel, and F
B. Cramer, Y Stradmann, J Schemmel, and F. Zenke. The heidelberg spiking data sets for the systematic evaluation of spiking neural net- works. IEEE Transactions on Neural Networks and Learning Systems , 33(7), 2022
2022
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.