REVIEW 3 major objections 5 minor 25 references
Realtime-Capable Hybrid Spiking Neural Networks for Neural Decoding of Cortical Activity
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that a hybrid spiking neural network can be the best-reported decoder of cortical activity on the Primate Reaching task, while a compressed version of the same architecture remains realtime-capable for neuroprosthetic…
desk verdict Solid SNN benchmark result, but the realtime claim is analytically derived, not measured, and the pretraining protocol needs explicit confirmation of test-set exclusion. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the hybrid decoder architecture together with a closed-form calculation of its streaming latency. The network compresses an input spike sequence in time through two or three temporal convolutional layers whose kernel sizes double per layer, processes the resulting keypoints through recurrently connected LIF units, and linearly interpolates the decoded velocities back to the original sequence length. The latency and execution rate are derived by working backward from one output keypoint to find the buffer size in the first convolutional layer needed to produce it; the paper's identity is that latency equals half that initial buffer size plus one time step, because training uses centered kernels. This receptive-field arithmetic (summarized in Algorithms 1–4) is what turns the design constraint 'decode within 100 ms' into concrete kernel sizes: two convolutional layers with kernels 9 and 18 yield 96 ms.
What would settle it
Run RTnet or sRTnet on the intended neuromorphic or hybrid hardware with a streaming input and measure the wall-clock time from the arrival of an input spike window to the emission of the corresponding velocity update; if the end-to-end delay exceeds 100 ms or the sustained update rate falls below 10 Hz, the paper's realtime-capability claim is refuted. Independently, retraining BMnet on the same splits with the same hyperparameters and checking whether test $R^2$ remains above 0.710 would settle the accuracy-lead claim.
Extended reading notes
Core claim
On the task of decoding finger-tip velocity from motor and somatosensory cortex spike trains, the paper's central claim is that a hybrid SNN achieves state-of-the-art accuracy while still admitting a realtime-capable compressed form. The uncompressed model, BMnet, reaches test $R^2 = 0.717 \pm 0.004$, surpassing the best previously published SNN by 0.019 and the best ANN by 0.007, making it the best-performing model reported on this task. The realtime version, RTnet, uses two convolutional layers with kernels of size 9 and 18 instead of three layers with sizes 31, 62, and 124, reducing the receptive field from 652 points to 46 points and the latency from 1308 ms to 96 ms. With weight and buffer quantization and spike regularization, the small version sRTnet keeps $R^2 = 0.675 \pm 0.011$—the best among co-optimized models—while cutting memory roughly by two thirds. The paper argues, on the basis of its buffer-size arithmetic, that this model satisfies the realtime criteria of at most 100 ms latency and at least 10 Hz execution rate.
Load-bearing premise
The whole realtime-capable verdict rests on an analytical latency formula, not on a measured system: the 96 ms figure assumes that buffering and receptive-field arithmetic are the only contributors to latency, so if actual communication, spike-dispatch, or memory-access overhead shows up on hardware, the realtime claim can fail even though the accuracy result stands.
Editorial extensions
If this is right
- If BMnet's $R^2$ is reproduced by independent training runs, the hybrid SNN becomes the accuracy leader on the Primate Reaching task among both spiking and non-spiking decoders, retiring the assumption that event-based models must sacrifice accuracy.
- The realtime-capable sRTnet would allow velocity updates every 16 ms with 96 ms end-to-end delay, fast enough for closed-loop prosthetic control without perceptible lag, assuming the computed latency transfers to hardware.
- Because sRTnet keeps event-based spiking units, it remains compatible with low-power neuromorphic execution in a wireless implant, which is the stated end goal.
- The finding that pruning causes a sharp accuracy drop suggests the current compression ceiling is set by unstructured pruning methods, not by the architecture itself.
- The ablations on Loco files imply that systems with tight memory budgets can drop somatosensory input with modest accuracy loss, while the best accuracy uses both M1 and S1.
Reading between the lines
- The 96 ms figure is an analytical estimate: actual deployment on neuromorphic hardware must also pay spike-dispatch, communication, and memory-access overheads, so the realtime-capable claim should be read as 'the architecture leaves room for implementation overhead' until measured end-to-end.
- The accuracy gap between BMnet and RTnet (0.717 vs 0.685) suggests the large temporal context, not the spiking recurrence, carries most of the decoding benefit; a fair test would be a non-spiking version of the same two-layer receptive field.
- The sharp pruning failure points to a possibly simple extension: structured or gradual pruning with sparsity-aware retraining might recover much of sRTnet's footprint without the observed cliff, and could be tested without new data.
- If the 96 ms latency does hold on hardware, the same receptive-field arithmetic could be reused to size convolutional front ends for other neural decoders whose latency budgets differ, e.g., 50 ms for more reactive control.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a hybrid spiking neural network (SNN) architecture for decoding finger velocity from cortical spike trains in the Primate Reaching task of the 2024 Neural Decoding Challenge. Three models are introduced: BMnet, a large model optimized for R²; RTnet, a reduced-latency variant; and sRTnet, a compressed version of RTnet using weight and buffer quantization. The authors report that BMnet achieves a test R² of 0.717±0.004, which they state surpasses both the prior SNN state of the art (Liu et al., 0.698±0.002) and the prior ANN state of the art (Wang et al., 0.710±0.050). They also claim that RTnet and sRTnet are realtime-capable, with a latency of 96 ms and an execution rate of 62.5 Hz, based on an analytical receptive-field calculation using buffer-size algorithms. The paper further analyzes the contribution of somatosensory cortex data and discusses compression, quantisation, and future deployment on neuromorphic hardware.
Significance. If the accuracy and latency claims were fully supported, the paper would make a useful contribution: it demonstrates that a hybrid SNN can achieve competitive decoding accuracy on a standardized benchmark and that a variant of the model is algorithmically compatible with the 100 ms / 10 Hz realtime envelope often cited for motor-control BMIs. The benchmarking on the public Primate Reaching dataset and the comparison with previously published SNN and ANN results are valuable, and the paper is generally clear about its architectural choices and limitations. However, the central claims currently rest on two pillars that need reinforcement: the claimed superiority over ANN state of the art is not statistically justified by the reported error bars, and the realtime-capable verdict is derived from an analytical latency model rather than measured on target hardware. Both issues are load-bearing for the paper's headline conclusions.
major comments (3)
- [Section IV-F, Table II] The pretraining procedure is described as using "all available recording sessions" for each primate, but the text does not state whether the six evaluation files (I1-I3, L1-L3) were excluded from pretraining. Since these files are part of the same dataset and the same animals, including them in pretraining could leak information and inflate the reported test R² values in Tables II and III. Please specify the exact session-level split, confirm that the evaluation files were held out, and, if they were not, retrain with a clean split and report the resulting numbers.
- [Table II, Section V-A] The claim that BMnet surpasses the ANN state of the art (Wang et al.) is not supported by the reported uncertainty. The difference is 0.717 - 0.710 = 0.007 in test R², while the Wang et al. result carries an uncertainty of ±0.050; thus the gap is well within one standard deviation of the competing method. To support the statement that BMnet is "the best-performing model yet," the authors should provide a statistical comparison, e.g., confidence intervals on the difference, a significance test, or a per-file breakdown. Alternatively, the claim should be softened to "comparable to" or "within error bars of" the ANN state of the art, while retaining the stronger claim with respect to SNNs, where the gap (0.717 vs. 0.698±0.002) is larger.
- [Section IV-C, Section IV-D.2, Table IV] The realtime-capable claim (96 ms latency, 62.5 Hz execution rate) is derived entirely from an analytical latency model: latency is defined as "half of the buffer size of the initial layer + 1 step" and the 96 ms figure comes from receptive-field arithmetic for two convolutional layers with kernels 9 and 18. This is a lower bound on one component of the end-to-end latency; it does not include input acquisition and spike encoding, communication between the convolutional and spiking parts, compute time on the target platform, interpolation scheduling, or output delivery. The paper itself states that deployment on neuromorphic hardware such as Loihi2 is future work, and no wall-clock measurement or cycle-accurate simulation is reported. Please either provide an end-to-end latency measurement or simulation on the target hardware, or explicitly relabel the 96 ms figure as an algorithmic lower bound and avoid the binary "Realtime-Capable" verdict in Table IV until it is validated.
minor comments (5)
- [Table III caption] The caption contains a typo: "adresses" should be "addresses".
- [Table IV caption] The caption contains a typo: "wether" should be "whether".
- [Section IV-B] The hyperparameter search is performed on the L2 file only, and the resulting hyperparameters are then used for all models reported in Tables II and III. Since L2 is one of the six evaluation files, the authors should clarify whether the final test R² values are computed with the same hyperparameters for all files or with per-file tuning; if the latter, the comparison with prior work is less straightforward.
- [Section IV-C] The sentence "The size determined by half of the buffer size of the initial layer + 1 step represents the model's latency" is ambiguous: it could mean half of the buffer size plus one step, or half of (buffer size plus one step). Please clarify the intended arithmetic, especially as this formula is the basis for the 96 ms claim.
- [General] The paper would benefit from stating whether code and trained model checkpoints are publicly available, as this would strengthen reproducibility of the benchmark numbers.
Circularity Check
No load-bearing circularity; the accuracy claim is externally benchmarked and the realtime claim is a stated design specification, though latency is derived rather than measured.
full rationale
The central accuracy claim (Table II: BMnet 0.717±0.004) is evaluated on the external Primate Reaching dataset against other groups' models (e.g., Liu et al. bigRSNN, Wang et al., NeuroBench baselines), so it does not reduce to a fitted input or to the paper's own definitions. Hyperparameters are optimized on one file (L2) and then applied to the challenge evaluation; this is standard model selection, not a prediction forced by construction. The realtime-capability claim is a design-specification loop rather than a circular derivation: Sec. IV-C defines latency analytically via receptive-field/buffer arithmetic (Algorithms 1–4), and Sec. IV-D.2 chooses kernel sizes 9 and 18 so that the computed latency is 96 ms under that definition; the definition is debatable because actual deployment is explicitly future work (Sec. VI: 'Future work will focus on implementing the networks presented in this work on neuromorphic hardware, e.g., Loihi2'), but that is a measurement/validity limitation, not circularity. Self-citations to [16] and [22] provide architectural context and an optimization metric, but the benchmark and latency calculations do not use those citations as the evidence for their own conclusions. No equation in the paper is equivalent to its input by construction, and no fitted parameter is renamed as a prediction.
Assumptions & free parameters
free parameters (4)
- Architecture hyperparameters (N_C, k, N_L, N_LIF, L_seq, r_int) =
BMnet: 40, 31, 2, 56, 4096, 8; RTnet/sRTnet: 40, 9, 2, 56, 4096, 4
- Spike regularization strength λS =
2.87e-3
- Weight regularization strength λW =
1.71e-6
- Fixed-point quantization bit widths (weights, buffers) =
1-7 integer bits and 1-4 fractional bits with sign bit
assumptions (4)
- domain assumption NeuroBench and the Primate Reaching dataset provide a fair, comparable evaluation across models.
- ad hoc to paper Half the initial-layer buffer size plus one step equals algorithmic latency.
- domain assumption Linear interpolation error is negligible for r_int <= 8.
- domain assumption The hybrid architecture from [16] is a correct and suitable starting point.
Cite this review
Pith. "Pith review of Realtime-Capable Hybrid Spiking Neural Networks for Neural Decoding of Cortical Activity." pith.science (2026). https://pith.science/paper/JRKKEK5D
@misc{pith2026250613400,
author = {Pith},
title = {Pith review of: Realtime-Capable Hybrid Spiking Neural Networks for Neural Decoding of Cortical Activity},
year = {2026},
howpublished = {\url{https://pith.science/paper/JRKKEK5D}},
note = {Machine review of arXiv:2506.13400}
}
read the original abstract
Intra-cortical brain-machine interfaces (iBMIs) present a promising solution to restoring and decoding brain activity lost due to injury. However, patients with such neuroprosthetics suffer from permanent skull openings resulting from the devices' bulky wiring. This drives the development of wireless iBMIs, which demand low power consumption and small device footprint. Most recently, spiking neural networks (SNNs) have been researched as potential candidates for low-power neural decoding. In this work, we present the next step of utilizing SNNs for such tasks, building on the recently published results of the 2024 Grand Challenge on Neural Decoding Challenge for Motor Control of non-Human Primates. We optimize our model architecture to exceed the existing state of the art on the Primate Reaching dataset while maintaining similar resource demand through various compression techniques. We further focus on implementing a realtime-capable version of the model and discuss the implications of this architecture. With this, we advance one step towards latency-free decoding of cortical spike trains using neuromorphic technology, ultimately improving the lives of millions of paralyzed patients.
Figures
Reference graph
Works this paper leans on
-
[1]
Prevalence and causes of paralysis—united states, 2013,
B. S. Armour, E. A. Courtney-Long, M. H. Fox, H. Fredine, and A. Cahill, “Prevalence and causes of paralysis—united states, 2013,” American journal of public health, vol. 106, no. 10, pp. 1855–1857, 2016
work page 2013
-
[2]
World Health Organization (WHO), “Spinal cord injury.,” 2024. https: //www.who.int/news-room/fact-sheets/detail/spinal-cord-injury, [Ac- cessed: Nov. 27, 2024]
work page 2024
-
[3]
Brain–machine interfaces: past, present and future,
M. A. Lebedev and M. A. Nicolelis, “Brain–machine interfaces: past, present and future,”TRENDS in Neurosciences, vol. 29, no. 9, pp. 536–546, 2006
work page 2006
-
[4]
Brain-machine interfaces: an overview,
M. Lebedev, “Brain-machine interfaces: an overview,”Translational Neuroscience, vol. 5, pp. 99–110, 2014
work page 2014
-
[5]
Independent mobility achieved through a wireless brain-machine interface,
C. Libedinsky, R. So, Z. Xu, T. K. Kyar, D. Ho, C. Lim, L. Chan, Y . Chua, L. Yao, J. H. Cheong,et al., “Independent mobility achieved through a wireless brain-machine interface,”PLoS One, vol. 11, no. 11, p. e0165773, 2016
work page 2016
-
[6]
C. Pandarinath, P. Nuyujukian, C. H. Blabe, B. L. Sorice, J. Saab, F. R. Willett, L. R. Hochberg, K. V . Shenoy, and J. M. Henderson, “High performance communication by people with paralysis using an intracortical brain-computer interface,”elife, vol. 6, p. e18554, 2017
work page 2017
-
[7]
J. D. Simeral, T. Hosman, J. Saab, S. N. Flesher, M. Vilela, B. Franco, J. N. Kelemen, D. M. Brandman, J. G. Ciancibello, P. G. Rezaii, et al., “Home use of a percutaneous wireless intracortical brain- computer interface by individuals with tetraplegia,”IEEE Transactions on Biomedical Engineering, vol. 68, no. 7, pp. 2313–2325, 2021
work page 2021
-
[8]
W. M. Reichert,Indwelling neural implants: strategies for contending with the in vivo environment. CRC Press, 2007
work page 2007
Show all 25 references
-
[9]
A brain-machine interface operating with a real-time spiking neural network control algorithm,
J. Dethier, P. Nuyujukian, C. Eliasmith, T. Stewart, S. Elasaad, K. V . Shenoy, and K. A. Boahen, “A brain-machine interface operating with a real-time spiking neural network control algorithm,”Advances in neural information processing systems, vol. 24, 2011
2011
-
[10]
An energy-efficient spiking neural network for finger velocity decoding for implantable brain- machine interface,
J. Liao, L. Widmer, X. Wang, A. Di Mauro, S. R. Nason-Tomaszewski, C. A. Chestek, L. Benini, and T. Jang, “An energy-efficient spiking neural network for finger velocity decoding for implantable brain- machine interface,” in2022 IEEE 4th International Conference on Artificial ...
2022
-
[11]
The combination of brain-computer interfaces and artificial intelligence: applications and challenges,
X. Zhang, Z. Ma, H. Zheng, T. Li, K. Chen, X. Wang, C. Liu, L. Xu, X. Wu, D. Lin,et al., “The combination of brain-computer interfaces and artificial intelligence: applications and challenges,”Annals of translational medicine, vol. 8, no. 11, 2020
2020
-
[12]
Human motor decoding from neural signals: a review,
W.-k. Tam, T. Wu, Q. Zhao, E. Keefer, and Z. Yang, “Human motor decoding from neural signals: a review,”BMC Biomedical Engineering, vol. 1, pp. 1–22, 2019
2019
-
[13]
Towards spike-based machine in- telligence with neuromorphic computing,
K. Roy, A. Jaiswal, and P. Panda, “Towards spike-based machine in- telligence with neuromorphic computing,”Nature, vol. 575, no. 7784, pp. 607–617, 2019
2019
-
[14]
Grand challenge on neural decoding for motor control of non-human primates,
B. Zhou, P.-S. V . Sun, J. Yik, C. Frenkel, V . J. Reddi, and A. Basu, “Grand challenge on neural decoding for motor control of non-human primates,” in2024 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1–5, IEEE, 2024
2024
-
[15]
Non- human primate reaching with multichannel sensorimotor cortex elec- trophysiology,
J. E. O’Doherty, M. M. Cardoso, J. G. Makin, and P. N. Sabes, “Non- human primate reaching with multichannel sensorimotor cortex elec- trophysiology,”Zenodo http://doi. org/10.5281/zenodo, vol. 583331, 2017
2017 doi
-
[16]
Hybrid spiking neural networks for low-power intra-cortical brain-machine inter- faces,
A. Vasilache, J. Krausse, K. Knobloch, and J. Becker, “Hybrid spiking neural networks for low-power intra-cortical brain-machine inter- faces,” in2024 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1–5, IEEE, 2024
2024
-
[17]
Decoding finger velocity from cortical spike trains with recurrent spiking neural networks,
T. Liu, J. Gygax, J. Rossbroich, Y . Chua, S. Zhang, and F. Zenke, “Decoding finger velocity from cortical spike trains with recurrent spiking neural networks,” in2024 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1–5, IEEE, 2024
2024
-
[18]
Leveraging recurrent neural networks for predicting motor movements from primate motor cortex neural recordings,
Y . Wang, Z. Wang, and S.-C. Liu, “Leveraging recurrent neural networks for predicting motor movements from primate motor cortex neural recordings,” in2024 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1–5, IEEE, 2024
2024
-
[19]
The neurobench framework for benchmarking neuromorphic computing algorithms and systems,
J. Yik, K. Van den Berghe, D. den Blanken, Y . Bouhadjar, M. Fabre, P. Hueber, W. Ke, M. A. Khoei, D. Kleyko, N. Pacik-Nelson, et al., “The neurobench framework for benchmarking neuromorphic computing algorithms and systems,”Nature Communications, vol. 16, no. 1, p. 1545, 2025
2025
-
[20]
Optuna: A next-generation hyperparameter optimization framework,
T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,”25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2623–2631, 2019
2019
-
[21]
Long-latency reflexes account for limb biomechanics through several supraspinal pathways,
I. L. Kurtzer, “Long-latency reflexes account for limb biomechanics through several supraspinal pathways,”Frontiers in Integrative Neu- roscience, vol. 8, Jan. 2015. Publisher: Frontiers
2015
-
[22]
On metric-driven development of embedded neuromorphic ai,
J. Krausse, M. Neher, I. Fuerst-Walter, C. Weigelt, T. Harbaum, K. Knobloch, and J. Becker, “On metric-driven development of embedded neuromorphic ai,” in2024 IEEE 37th International System- on-Chip Conference (SOCC), pp. 1–6, IEEE, 2024
2024
-
[23]
Exploring neuronal leakage for spiking neural networks on event-driven hardware,
J. Krausse, S. Nitzsche, B. Pachideh, C. Weigelt, K. Knobloch, and J. Becker, “Exploring neuronal leakage for spiking neural networks on event-driven hardware,” in2024 IEEE/ACM International Conference on Neuromorphic Systems (ICONS), IEEE, 2024
2024
-
[24]
Rapid control and feedback rates enhance neu- roprosthetic control,
M. M. Shanechi, A. L. Orsborn, H. G. Moorman, S. Gowda, S. Dangi, and J. M. Carmena, “Rapid control and feedback rates enhance neu- roprosthetic control,”Nature communications, vol. 8, no. 1, p. 13825, 2017
2017
-
[25]
Efficient neuromorphic signal pro- cessing with loihi 2,
G. Orchard, E. P. Frady, D. B. D. Rubin, S. Sanborn, S. B. Shrestha, F. T. Sommer, and M. Davies, “Efficient neuromorphic signal pro- cessing with loihi 2,” in2021 IEEE Workshop on Signal Processing Systems (SiPS), pp. 254–259, IEEE, 2021. VII. SUPPLEMENTARYINFORMATION Algorit...
2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.