REVIEW 5 major objections 5 minor 32 references
Cross Knowledge Distillation between Artificial and Spiking Neural Networks
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Cross Knowledge Distillation lets an ANN teacher lift an SNN to 97.13% on N-Caltech101.
desk verdict New cross-modal/cross-architecture distillation with consistent gains, but the headline SOTA comes from a test-set sweep and should be re-reported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is an indirect, phased distillation pipeline: at each timestep, a vanilla KL-divergence logits distillation loss transfers knowledge from an ANN teacher to the static (RGB) stream of an SNN; a domain-alignment loss defined as the timestep-averaged CKA dissimilarity between same-category static and dynamic features, plus a TET classification regularization term, aligns the two streams inside the weight-shared SNN; and a non-linear replacement probability gradually swaps static inputs for DVS inputs while a sigmoidal gate switches off the ANN distillation at a chosen epoch. The CKA similarity per timestep is what converts ANN-derived knowledge into a form usable by the DVS stream.
What would settle it
Train the same CKD pipeline on an event-based dataset whose classes are distinguished mainly by motion or temporal structure, and compare against a DVS-only baseline while measuring per-timestep CKA values; if accuracy does not improve or the alignment score fails to track the gain, the domain-alignment assumption fails.
Extended reading notes
Core claim
The central claim is that an SNN can be made to perform nearly as well as an ANN on event-based image classification by routing the ANN's knowledge through an intermediate RGB-DVS hybrid stream inside the student. Instead of distilling DVS features directly from an ANN, CKD feeds paired RGB and DVS data of the same category into a weight-shared SNN, aligns their feature distributions at every timestep with a CKA-based domain-alignment loss, and applies logits-based knowledge distillation from the ANN teacher to the static stream only. A replacement schedule gradually increases the proportion of DVS data in the hybrid stream, while a phase-switching function turns off the ANN distillation once the stream becomes too dynamic. The paper demonstrates the method on N-Caltech101 and CEP-DVS, reporting accuracy of 97.13% and 40.20% respectively, and argues that this closes most of the gap between SNNs and ANNs on these tasks.
Load-bearing premise
The argument depends on per-timestep CKA alignment between static RGB-derived and dynamic DVS features actually transferring task-discriminative temporal information; if aligning the two streams distorts or discards the temporal cues that make DVS data valuable, the indirect distillation would hurt rather than help.
Editorial extensions
If this is right
- SNNs trained with CKD can reach accuracy within about a point of their ANN teacher on event-camera classification, suggesting neuromorphic models need not trade away task accuracy for energy efficiency.
- Abundant RGB data and pretrained ANN teachers can substitute for scarce labeled DVS data, lowering the collection-cost barrier for event-based vision.
- The phase-switching schedule means the RGB teacher matters most early in training; later training can be dominated by the native DVS modality without losing the transferred knowledge.
- Because distillation happens in logits rather than layer-wise features, CKD relaxes architectural constraints and can be applied to different SNN backbones, as shown with VGGSNN and Spiking ResNet-18.
- The same hybrid-stream design is positioned by the authors as a foundation for action recognition and video-based detection, where temporal information is richer.
Reading between the lines
- One implication the paper leaves implicit is a curriculum interpretation: the hybrid stream teaches the SNN static appearance first and then hands over to dynamic input; a testable extension is to vary the replacement curve to see whether the optimal schedule depends on how much temporal structure the target dataset carries.
- A direct diagnostic the paper does not report is whether the per-timestep CKA alignment actually rises during training; tracking the domain-alignment loss separately would show whether the accuracy gain comes from alignment or merely from the classification loss on augmented RGB-DVS pairs.
- If the method generalizes, the same RGB-hybrid-DVS scheme could transfer ANN knowledge to SNNs for tasks where paired static and event data are harder to obtain, such as optical flow or video anomaly detection, by using synthetic or weakly paired modalities.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes Cross Knowledge Distillation (CKD), a training scheme for SNNs on event-based (DVS) image classification that simultaneously addresses cross-modality and cross-architecture transfer. A teacher ANN (WRN101-2) processes RGB data, and logits KD is applied to the static stream of a shared SNN; the static and dynamic streams are aligned with a CKA-based domain-alignment loss, and a cubic replacement schedule progressively mixes DVS data into the static stream. A sigmoid phase-switching function turns off KD after a threshold epoch. Experiments on N-Caltech101 and CEP-DVS report top-1 accuracies of 97.13% and 40.20%, claimed to be new SOTA and an improvement over the Knowledge-Transfer baseline [10].
Significance. If the reported gains are real, this is a practically useful contribution: it is, to the authors' knowledge, the first ANN-to-SNN distillation that bridges both modality and architecture, and the CKA-plus-replacement design is a plausible way to transfer RGB knowledge into a DVS-trained SNN. Strengths include error bars over three seeds, an ablation over KD loss variants, and a commitment to release code. However, the headline accuracy numbers are selected from a test-set hyperparameter sweep, the comparison is not controlled for the extra RGB/teacher supervision, and the ablation does not isolate the proposed components; these issues currently prevent the SOTA claim from being accepted at face value.
major comments (5)
- [Fig. 4, Table I, Section IV.C] The headline accuracies (97.13% on N-Caltech101 and 40.20% on CEP-DVS) are maxima selected from the grid of phase-switching hyperparameters (eth, k) shown in Fig. 4, and Table I reports the best of three seeds in parentheses. No validation split is described before this model selection, and the baseline Knowledge-Transfer method is not given the same test-set sweep. Selecting the maximum over the grid shown in Fig. 4 on relatively small datasets can inflate the apparent gain over the baseline (3.68 percentage points in the abstract) by a non-negligible amount. The SOTA claim is therefore not established for a fixed configuration; the authors should either use a validation split for hyperparameter selection and report the resulting test accuracy, or report the full distribution of grid results (mean and standard deviation) and apply the same protocol to the baselines.
- [Section III.A-B, Table II] No ablation isolates the contributions of the three proposed components: the CKA domain-alignment loss LDA, the semantic replacement schedule Preplace, and the phased KD loss. Table II varies only the KD loss function (DKD, LumiNet, DIST, KD) while keeping LDA and replacement fixed, and Fig. 4 varies only eth and k. Consequently, the reported gains over the baseline could come entirely from the added ANN teacher and RGB data rather than from the proposed cross-modality alignment or replacement mechanism. The authors should add ablations that remove LDA, remove replacement, and remove KD separately (and in combination), with the same teacher and data budget.
- [Section IV.A, Table I] The experimental setup is inconsistent as reported. Section IV.A states that the CEP-DVS model is trained with 6 time steps, but Table I lists timesteps 10 for all CEP-DVS rows; in addition, the baseline Knowledge-Transfer accuracy is given as 93.45% in the abstract and Introduction, but as 93.18 ± 0.38 (93.33*) in Table I. These discrepancies affect the fairness and reproducibility of the comparison and must be resolved before the numerical claims can be assessed.
- [Eq. (5)-(6), Section III.A.1] The domain-alignment loss is not fully specified. The CKA formula is never written out (only cited), and it is unclear whether CKA is applied to the flattened feature maps or to the penultimate-layer vectors, which layer is used, and how the learnable coefficients θt are initialized or constrained. In addition, Lcls-e appears both inside LDA in Eq. (6) and again in the total loss Lall = LS + Lcls-e, so the classification loss for the dynamic stream is weighted twice; the authors should clarify whether this double counting is intended and how the weights are balanced.
- [Table I, Section IV.B] The comparison in Table I is not controlled for additional supervision. CKD uses an ImageNet-pretrained WRN101-2 teacher and RGB training pairs, while none of the compared baselines use an ANN teacher or RGB data. The accuracy gap relative to these baselines therefore conflates the proposed distillation mechanism with access to a stronger external teacher and an additional modality. A comparison against a same-setting baseline (e.g., [10] augmented with the same teacher but without the proposed losses) would be needed to support the SOTA claim as stated.
minor comments (5)
- [Section III.B.1] In the text following Eq. (9), 'the the Kullback-Leibler divergence' should read 'the Kullback-Leibler divergence', and in Section III.B.2 'the LKD is denied as shown in (9)' should read 'the LKD is defined as shown in (9)'.
- [Eq. (8)] The notation for the replacement schedule should specify whether bi is 0-based or 1-based and whether ec starts at 0 or 1, since this changes the exact mixing schedule and affects reproducibility.
- [References] Reference [29] is cited as 'Wide ResNet101 2 (WRN101 2)' but the listed title is 'SpinalNet: Deep neural network with gradual input'; this mismatch should be corrected or the correct WRN101-2 reference supplied.
- [Fig. 3] The t-SNE visualizations would benefit from axis labels, a note on perplexity or number of iterations, and a statement of which layer's features are visualized; the qualitative claim about 'tighter clusters' is otherwise hard to verify.
- [Table I] The footnote 'The best accuracy is shown in parentheses' is ambiguous for the CKD rows, because the parenthetical value is simultaneously the best of three seeds and the maximum over the Fig. 4 hyperparameter grid; the two selection steps should be reported separately.
Circularity Check
No derivational circularity; mild benchmark circularity because headline accuracies are selected from a test-set hyperparameter sweep.
-
fitted input called prediction
[Section IV-C, Fig. 4, and Table I]
"On the N-Caltech101, peaks (97.13%) occur at eth = 149.5, k= 100 and eth = 119.5, k= 100 ... On the CEP-DVS, peak (40.2%) are observed at small values of k, with eth = 89.5, k= 0.001 ... The best accuracy is shown in parentheses."
The reported SOTA numbers are the best points of a grid search over the phase-switching parameters (eth,k) of Eq. 10, and Table I then reports the best of three seeds in parentheses. No validation split is fixed before model selection, so 97.13% and 40.20% are extrema over configurations evaluated on the test sets rather than the expected accuracy of a predetermined configuration. The claim 'outperforms current State-of-the-Art methods' is therefore partially forced by selecting the maximum over the swept grid; however, every ablation point exceeds the baseline, so the method retains independent empirical support.
full rationale
The derivation chain is not circular in the equation sense: the losses (CKA-based LDA in Eqs. 5-6, vanilla KD in Eq. 9, and the switching weight in Eqs. 10-11) define a training procedure rather than being fitted to reproduce the reported accuracies, and no key result depends on a self-citation or a uniqueness theorem. The central empirical claim is nevertheless weakened by test-set hyperparameter selection: the headline 97.13%/40.20% accuracies are the peaks of the Fig. 4 (eth,k) sweep with best-of-seeds reported, so these numbers are selection statistics rather than fixed-configuration expectations. Because all swept configurations still beat the baseline, this is benchmark circularity of modest severity rather than a fully forced result.
Assumptions & free parameters
free parameters (6)
- alpha =
1
- beta =
1
- replacement exponent =
3
- switching threshold eth =
149.5 (N-Caltech101), 89.5 (CEP-DVS)
- switching slope k =
100 (N-Caltech101), 0.001 (CEP-DVS)
- KD temperature tau =
not specified
assumptions (4)
- domain assumption Surrogate gradient enables backpropagation through spiking neurons.
- ad hoc to paper CKA is a valid differentiable objective for aligning feature distributions across modalities.
- domain assumption Knowledge distilled from an RGB-trained ANN via logits is beneficial to an SNN processing DVS data through a hybrid stream.
- ad hoc to paper The replacement schedule (Eq. 8) and phased switching (Eq. 10) ensure stable training without harming the DVS stream.
Cite this review
Pith. "Pith review of Cross Knowledge Distillation between Artificial and Spiking Neural Networks." pith.science (2026). https://pith.science/paper/W4RWCVOI
@misc{pith2026250709269,
author = {Pith},
title = {Pith review of: Cross Knowledge Distillation between Artificial and Spiking Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/W4RWCVOI}},
note = {Machine review of arXiv:2507.09269}
}
read the original abstract
Recently, Spiking Neural Networks (SNNs) have demonstrated rich potential in computer vision domain due to their high biological plausibility, event-driven characteristic and energy-saving efficiency. Still, limited annotated event-based datasets and immature SNN architectures result in their performance inferior to that of Artificial Neural Networks (ANNs). To enhance the performance of SNNs on their optimal data format, DVS data, we explore using RGB data and well-performing ANNs to implement knowledge distillation. In this case, solving cross-modality and cross-architecture challenges is necessary. In this paper, we propose cross knowledge distillation (CKD), which not only leverages semantic similarity and sliding replacement to mitigate the cross-modality challenge, but also uses an indirect phased knowledge distillation to mitigate the cross-architecture challenge. We validated our method on main-stream neuromorphic datasets, including N-Caltech101 and CEP-DVS. The experimental results show that our method outperforms current State-of-the-Art methods. The code will be available at https://github.com/ShawnYE618/CKD
Figures
Reference graph
Works this paper leans on
-
[10]
An efficient knowledge transfer strategy for spiking neural networks from static to event domain,
Xiang He, Dongcheng Zhao, Yang Li, Guobin Shen, Qingqun Kong, and Yi Zeng, “An efficient knowledge transfer strategy for spiking neural networks from static to event domain,” in Proceedings of the AAAI Conference on Artificial Intelligence , 2024, vol. 38, pp. 512–520
work page 2024
-
[1]
Networks of spiking neurons: the third generation of neural network models,
Wolfgang Maass, “Networks of spiking neurons: the third generation of neural network models,” Neural networks, vol. 10, no. 9, pp. 1659–1671, 1997
work page 1997
-
[2]
Towards spike-based machine intelligence with neuromorphic computing,
Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda, “Towards spike-based machine intelligence with neuromorphic computing,” Na- ture, vol. 575, no. 7784, pp. 607–617, 2019
work page 2019
-
[3]
Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip,
Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al., “Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip,” IEEE transactions on computer-aided design of integrated circuits and systems , vol. 34, no. 10, pp. 153...
work page 2015
-
[4]
Loihi: A neuromorphic manycore processor with on-chip learning,
Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al., “Loihi: A neuromorphic manycore processor with on-chip learning,” Ieee Micro, vol. 38, no. 1, pp. 82–99, 2018
work page 2018
-
[5]
Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks
Yufei Guo, Yuanpei Chen, Xiaode Liu, Weihang Peng, Yuhan Zhang, Xuhui Huang, and Zhe Ma, “Ternary spike: Learning ternary spikes for spiking neural networks,” arXiv preprint arXiv:2312.06372 , 2023
work page Pith review arXiv 2023
-
[6]
UCF-Crime-DVS: A Novel Event-Based Dataset for Video Anomaly Detection with Spiking Neural Networks
Yuanbin Qian, Shuhan Ye, Chong Wang, Xiaojie Cai, Jiangbo Qian, and Jiafei Wu, “Ucf-crime-dvs: A novel event-based dataset for video anomaly detection with spiking neural networks,” arXiv preprint arXiv:2503.12905, 2025
work page Pith review arXiv 2025
-
[7]
Distilling the knowledge in a neural network,
Geoffrey Hinton, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015
arXiv 2015
Show all 32 references
-
[8]
Lasnn: Layer- wise ann-to-snn distillation for effective and efficient training in deep spiking neural networks,
Di Hong, Jiangrong Shen, Yu Qi, and Yueming Wang, “Lasnn: Layer- wise ann-to-snn distillation for effective and efficient training in deep spiking neural networks,” arXiv preprint arXiv:2304.09101 , 2023
2023 arXiv
-
[9]
Respike: Residual frames-based hybrid spik- ing neural networks for efficient action recognition,
Shiting Xiao, Yuhang Li, Youngeun Kim, Donghyun Lee, and Priyadarshini Panda, “Respike: Residual frames-based hybrid spik- ing neural networks for efficient action recognition,” arXiv preprint arXiv:2409.01564, 2024
2024 arXiv
-
[11]
Converting static image datasets to spiking neuromorphic datasets using saccades,
Orchard Garrick, Jayawant Ajinkya, Gregory K. Cohen, and Thakor Nitish, “Converting static image datasets to spiking neuromorphic datasets using saccades,” Frontiers in Neuroscience , vol. 9, no. 178, 2015
2015
-
[12]
Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories,
Li Fei-Fei, Rob Fergus, and Pietro Perona, “Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories,” Computer Vision and Pattern Recognition Workshop, 2004
2004
-
[13]
Reduction of class activation uncertainty with background information,
H M Dipu Kabir, “Reduction of class activation uncertainty with background information,” 2024
2024
-
[14]
Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski, Neuronal dynamics: From single neurons to networks and models of cognition, Cambridge University Press, 2014
2014
-
[15]
Unsupervised event-based learning of optical flow, depth, and egomotion,
Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, and Kostas Dani- ilidis, “Unsupervised event-based learning of optical flow, depth, and egomotion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 989–997
2019
-
[16]
Hots: a hierarchy of event-based time-surfaces for pattern recognition,
Xavier Lagorce, Garrick Orchard, Francesco Galluppi, Bertram E Shi, and Ryad B Benosman, “Hots: a hierarchy of event-based time-surfaces for pattern recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 7, pp. 1346–1359, 2016
2016
-
[17]
Hats: Histograms of averaged time surfaces for robust event-based object classification,
Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman, “Hats: Histograms of averaged time surfaces for robust event-based object classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 1731–1740
2018
-
[18]
Restructuring the teacher and student in self-distillation,
Yujie Zheng, Chong Wang, Chenchen Tao, Sunqi Lin, Jiangbo Qian, and Jiafei Wu, “Restructuring the teacher and student in self-distillation,” IEEE Transactions on Image Processing , 2024
2024
-
[19]
Decoupled knowledge distillation,
Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang, “Decoupled knowledge distillation,” 2022
2022
-
[20]
Distill vision transformers to cnns via teacher collaboration,
Sunqi Lin, Chong Wang, Yujie Zheng, Chenchen Tao, Xinmiao Dai, and Yuqi Li, “Distill vision transformers to cnns via teacher collaboration,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 5925–5929
2024
-
[21]
Distilling spikes: Knowledge distillation in spiking neural networks,
Ravi Kumar Kushawaha, Saurabh Kumar, Biplab Banerjee, and Rajbabu Velmurugan, “Distilling spikes: Knowledge distillation in spiking neural networks,” in 2020 25th International Conference on Pattern Recognition (ICPR), 2021, pp. 4536–4543
2020
-
[22]
Similarity of neural network representations revisited,
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton, “Similarity of neural network representations revisited,” in International conference on machine learning . PMLR, 2019, pp. 3519– 3529
2019
-
[23]
Temporal efficient training of spiking neural network via gradient re-weighting,
Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu, “Temporal efficient training of spiking neural network via gradient re-weighting,” in International Conference on Learning Representations , 2022
2022
-
[24]
Learning from images: A distillation learning framework for event cameras,
Yongjian Deng, Hao Chen, Huiying Chen, and Youfu Li, “Learning from images: A distillation learning framework for event cameras,” IEEE Transactions on Image Processing , vol. 30, pp. 4919–4931, 2021
2021
-
[25]
Neuromorphic data augmentation for training spiking neural networks,
Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, and Priyadarshini Panda, “Neuromorphic data augmentation for training spiking neural networks,” 2022
2022
-
[26]
Eventmix: An efficient data augmentation strategy for event-based learning,
Guobin Shen, Dongcheng Zhao, and Yi Zeng, “Eventmix: An efficient data augmentation strategy for event-based learning,” Information Sciences, vol. 644, pp. 119170, 2023
2023
-
[27]
An unsupervised stdp-based spiking neural network inspired by biologically plausible learning rules and connections,
Yiting Dong, Dongcheng Zhao, Yang Li, and Yi Zeng, “An unsupervised stdp-based spiking neural network inspired by biologically plausible learning rules and connections,” Neural Networks , vol. 165, pp. 799– 808, 2023
2023
-
[28]
Improving stability and performance of spiking neural networks through enhancing temporal consistency,
Dongcheng Zhao, Guobin Shen, Yiting Dong, Yang Li, and Yi Zeng, “Improving stability and performance of spiking neural networks through enhancing temporal consistency,” Pattern Recognition, vol. 159, pp. 111094, 2025
2025
-
[29]
Spinalnet: Deep neural network with gradual input,
H M Dipu Kabir, Moloud Abdar, Abbas Khosravi, Seyed Moham- mad Jafar Jalali, Amir F. Atiya, Saeid Nahavandi, and Dipti Srinivasan, “Spinalnet: Deep neural network with gradual input,” IEEE Transactions on Artificial Intelligence , vol. 4, no. 5, pp. 1165–1177, Oct. 2023
2023
-
[30]
Braincog: A spiking neural network based, brain-inspired cognitive intelligence engine for brain-inspired ai and brain simulation,
Yi Zeng, Dongcheng Zhao, Feifei Zhao, Guobin Shen, Yiting Dong, Enmeng Lu, Qian Zhang, Yinqian Sun, Qian Liang, Yuxuan Zhao, et al., “Braincog: A spiking neural network based, brain-inspired cognitive intelligence engine for brain-inspired ai and brain simulation,” Patterns, v...
2023
-
[31]
Luminet: The bright side of perceptual knowledge distillation,
Md Ismail Hossain, MM Elahi, Sameera Ramasinghe, Ali Cheraghian, Fuad Rahman, Nabeel Mohammed, and Shafin Rahman, “Luminet: The bright side of perceptual knowledge distillation,” arXiv preprint arXiv:2310.03669, 2023
2023 arXiv
-
[32]
Knowledge distillation from a stronger teacher,
Tao Huang, Shan You, Fei Wang, Chen Qian, and Chang Xu, “Knowledge distillation from a stronger teacher,” arXiv preprint arXiv:2205.10536, 2022
2022 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.