arxiv: 2605.08270 · v1 · submitted 2026-05-08 · 💻 cs.CV · cs.AI

Recognition: 2 theorem links

· Lean Theorem

SAFformer:Improving Spiking Transformer via Active Predictive Filtering

Zequan Xie , Weiming Zeng , Yunhua Chen , Sichang Ling , Tongyang Chen , Jinsheng Xiao

Authors on Pith no claims yet

Pith reviewed 2026-05-12 00:45 UTC · model grok-4.3

classification 💻 cs.CV cs.AI

keywords Spiking Neural NetworksSpiking TransformersPredictive CodingActive FilteringImage ClassificationEnergy EfficiencyCIFARImageNet

0 comments

The pith

SAFformer improves spiking transformers by using active predictive filtering to suppress redundant visual signals and focus on salient features.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces SAFformer as a spiking transformer that shifts from passive reaction to active prediction of and suppression of predictable input signals. Drawing from brain predictive coding, the architecture filters out redundant data so that computation targets only the unexpected or task-relevant parts of images. This change is presented as solving both the accuracy ceiling and high energy cost that have limited prior spiking transformers on visual tasks. A reader would care because spiking networks already promise low-power operation; if the filtering works, it makes them competitive for real-world vision without needing more parameters or power.

Core claim

SAFformer is a spiking transformer built on an active predictive filtering paradigm. The model predicts incoming visual signals and actively suppresses the predictable portions, allowing it to allocate spiking computation to salient, unpredictable features. On this basis the architecture reports new state-of-the-art accuracy on CIFAR-10, CIFAR-100 and CIFAR10-DVS, and reaches 80.50 percent top-1 accuracy on ImageNet-1K with 26.58 million parameters and 5.88 mJ energy.

What carries the argument

Active predictive filtering mechanism that suppresses predictable signals in the input to concentrate spiking activity on salient visual features.

If this is right

Delivers higher accuracy than prior spiking transformers on CIFAR-10/100 and CIFAR10-DVS.
Reaches 80.50 percent top-1 accuracy on ImageNet-1K while using only 26.58 million parameters.
Reduces energy consumption to 5.88 mJ on the ImageNet task.
Lowers computational overhead on redundant visual data compared with passive spiking transformers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same filtering idea could be inserted into other spiking architectures to improve their efficiency without redesigning the core layers.
If the mechanism generalizes, it points toward spiking models that scale accuracy with far less energy than dense transformers on edge hardware.
The approach suggests a concrete way to embed predictive-processing principles into hardware-efficient networks for real-time vision.
Similar suppression of predictable signals might prove useful in other sensory modalities where spiking networks are already deployed.

Load-bearing premise

That suppressing predictable signals removes only redundant computation while preserving every piece of information needed for accurate classification.

What would settle it

Disabling the predictive filtering module and measuring whether top-1 accuracy on ImageNet-1K falls below 80.50 percent or energy rises above 5.88 mJ would directly test whether the mechanism is responsible for the reported gains.

Figures

Figures reproduced from arXiv: 2605.08270 by Jinsheng Xiao, Sichang Ling, Tongyang Chen, Weiming Zeng, Yunhua Chen, Zequan Xie.

**Figure 2.** Figure 2: Comparison of three self-attention paradigms. (a)VSA employs floating-point matrix multiplication to evaluate the spatial cor [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Figure 3: (a) The architecture of SAFformer. We propose the SAFformer block, which consists of SAF Attention and SMAG. (b) An [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: Ablation study on SGM, analyzing its impact on training [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 5.** Figure 5: Fourier spectrum of Spiking Neurons, Spiking Depth-Wise [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗

**Figure 7.** Figure 7: Visualization of attention heatmaps on ImageNet-1k. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗

read the original abstract

Spiking Neural Networks (SNNs) offer notable advantages in biological plausibility and energy efficiency, making them promising candidates for building low-power Transformers. However, existing Spiking Transformers largely adhere to a passive reactive paradigm, which struggles to focus on task-relevant information and incurs substantial computational overhead when processing redundant visual data. To overcome this fundamental yet underexplored limitation, we propose SAFformer, a novel Spiking Transformer architecture based on an active predictive filtering paradigm. Inspired by the brain's predictive coding mechanism, SAFformer actively suppresses predictable signals and focuses on salient visual features. Extensive experiments show that SAFformer establishes new state-of-the-art performance on CIFAR-10/100 and CIFAR10-DVS. Remarkably, on ImageNet-1K, it achieves 80.50% Top-1 accuracy with only 26.58M parameters and an energy consumption of 5.88 mJ, demonstrating an exceptional balance between accuracy and efficiency.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

SAFformer adds active predictive filtering to spiking transformers and delivers competitive ImageNet accuracy at low measured energy.

read the letter

The paper's core move is to replace the usual passive spiking transformer with an active predictive filter that suppresses predictable parts of the input, inspired by neuroscience predictive coding. This produces the headline result of 80.5% top-1 on ImageNet-1K using 26.58 M parameters and 5.88 mJ, measured from actual spike rates rather than theoretical bounds, plus SOTA numbers on CIFAR-10/100 and CIFAR10-DVS. The architecture description, training details, and ablations line up internally and show direct comparisons to prior spiking transformers under matched settings. The filtering step appears to be the main source of the reported gains without obvious circularity or hidden fitting. Energy figures are grounded in observed spike activity, which is a concrete improvement over many earlier claims. The soft spots are limited. The gains depend on the filtering not discarding task-critical information, and while the ablations support this on the tested sets, broader robustness checks across more diverse or noisy inputs would strengthen the case. No major inconsistencies appear in the equations or experimental protocol. This work is aimed at people building or evaluating spiking networks for vision on neuromorphic or edge hardware. Readers who follow low-power transformer variants will find the concrete numbers and the filtering mechanism worth examining. It deserves a serious referee because the experimental support is present and the central claim holds up on the evidence given.

Referee Report

0 major / 3 minor

Summary. The manuscript proposes SAFformer, a spiking transformer architecture that replaces the passive reactive paradigm of prior SNN transformers with an active predictive filtering mechanism inspired by brain predictive coding. This filtering step is intended to suppress predictable signals in visual inputs while retaining task-critical information, thereby improving both classification accuracy and energy efficiency. The paper reports new state-of-the-art results on CIFAR-10/100 and CIFAR10-DVS, and on ImageNet-1K achieves 80.50% Top-1 accuracy with 26.58 M parameters and 5.88 mJ energy consumption derived from measured spike rates.

Significance. If the experimental claims hold, the work offers a meaningful advance for spiking transformers by demonstrating that an active, biologically motivated filtering stage can simultaneously raise accuracy and lower energy relative to prior passive designs. The direct, matched-setting comparisons to earlier spiking transformers and the use of measured rather than theoretical spike rates for energy accounting strengthen the efficiency argument. The result could inform future neuromorphic vision pipelines that must operate under tight power budgets.

minor comments (3)

§4.2 and Table 2: the ablation isolating the predictive filter reports accuracy gains but does not include standard deviations across the three random seeds mentioned in the training protocol; adding these would clarify whether the observed deltas exceed run-to-run variance.
Figure 3: the spike-rate heatmaps are shown only for the final layer; earlier layers would help confirm that the filtering effect is distributed rather than localized to the classifier head.
§3.3, Eq. (7): the predictive filter is described as parameter-free at inference, yet the training description introduces a small auxiliary loss weight; a one-sentence clarification on whether this weight is annealed to zero would remove ambiguity.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive evaluation of our work, the recognition of its potential significance for neuromorphic vision systems, and the recommendation for minor revision. We are pleased that the active predictive filtering approach and the use of measured spike rates for energy accounting were viewed favorably.

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The paper presents SAFformer as a spiking transformer using an active predictive filtering paradigm inspired by brain predictive coding. No load-bearing equations, derivations, or self-citations are shown that reduce the central claims (suppression of predictable signals while retaining task-critical features) to fitted parameters, self-definitions, or prior author work by construction. Performance results on CIFAR and ImageNet are reported via direct experimental comparisons under matched settings, with energy figures based on measured spike rates. The architecture and training protocol remain internally consistent without hidden reductions to inputs, making the derivation self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no equations, hyperparameters, or architectural details are provided, so the ledger cannot be populated beyond noting the absence of information.

pith-pipeline@v0.9.0 · 5475 in / 1131 out tokens · 32055 ms · 2026-05-12T00:45:43.994866+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost Jcost_pos_of_ne_one echoes

?

echoes
ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

SAFformer actively suppresses predictable signals and focuses on salient visual features... higher-level cortical areas form a holistic prediction... only the significant discrepancies are propagated upward as feedback signals

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 1 internal anchor

[1]

Network information flow

[Ahlswede et al., 2000] Rudolf Ahlswede, Ning Cai, S-YR Li, and Raymond W Yeung. Network information flow. IEEE Transactions on information theory , 46(4):1204– 1216,

work page 2000
[2]

A low power, fully event- based gesture recognition system

[Amir et al., 2017] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Gar- reau, Marcela Mendoza, et al. A low power, fully event- based gesture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 7243–7252,

work page 2017
[3]

Run, don’t walk: chasing higher flops for faster neural networks

[Chen et al., 2023] Jierun Chen, Shiu-hong Kao, Hao He, Weipeng Zhuo, Song Wen, Chul-Ho Lee, and S-H Gary Chan. Run, don’t walk: chasing higher flops for faster neural networks. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 12021–12031,

work page 2023
[4]

High-performance deep spiking neural networks via at-most-two-spike exponential coding

[Chen et al., 2024] Yunhua Chen, Ren Feng, Zhimin Xiong, Jinsheng Xiao, and Jian K Liu. High-performance deep spiking neural networks via at-most-two-spike exponential coding. Neural Networks, 176:106346,

work page 2024
[5]

Predictive filtering for nonlinear systems

[Crassidis and Markley, 1997] John L Crassidis and F Lan- dis Markley. Predictive filtering for nonlinear systems. Journal of Guidance, Control, and Dynamics, 20(3):566– 572,

work page 1997
[6]

Randaugment: Practical auto- mated data augmentation with a reduced search space

[Cubuk et al., 2020] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical auto- mated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition workshops , pages 702–703,

work page 2020
[7]

Imagenet: A large-scale hierarchical image database

[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248–255. Ieee,

work page 2009
[8]

Ec-snn: Splitting deep spiking neural networks for edge devices

[Di Yu et al., 2024] Xin Du Di Yu, Linshan Jiang, Wentao Tong, and Shuiguang Deng. Ec-snn: Splitting deep spiking neural networks for edge devices. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence,

work page 2024
[9]

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

work page internal anchor Pith review Pith/arXiv arXiv 2020
[10]

Neu- rozoom: Denoising and super resolving neuromorphic events and spikes

[Duan et al., 2023] Peiqi Duan, Yi Ma, Xinyu Zhou, Xinyu Shi, Zihao W Wang, Tiejun Huang, and Boxin Shi. Neu- rozoom: Denoising and super resolving neuromorphic events and spikes. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 45(12):15219–15232,

work page 2023
[11]

Training spiking neural networks using lessons from deep learning

[Eshraghian et al., 2023] Jason K Eshraghian, Max Ward, Emre O Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D Lu. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE , 111(9):1016–1054,

work page 2023
[12]

Spiking transformers need high frequency information

[Fang et al., 2025] Yuetong Fang, Deming Zhou, Ziqing Wang, Hongwei Ren, ZeCui Zeng, Lusong Li, Shibo Zhou, and Renjing Xu. Spiking transformers need high frequency information. arXiv preprint arXiv:2505.18608,

work page arXiv 2025
[13]

The free-energy principle: a rough guide to the brain? Trends in cognitive sciences , 13(7):293–301,

[Friston, 2009] Karl Friston. The free-energy principle: a rough guide to the brain? Trends in cognitive sciences , 13(7):293–301,

work page 2009
[14]

Real spike: Learning real-valued spikes for spiking neural networks

[Guo et al., 2022] Yufei Guo, Liwen Zhang, Yuanpei Chen, Xinyi Tong, Xiaode Liu, YingLei Wang, Xuhui Huang, and Zhe Ma. Real spike: Learning real-valued spikes for spiking neural networks. In European conference on com- puter vision, pages 52–68. Springer,

work page 2022
[15]

Ternary spike: Learning ternary spikes for spiking neural networks

[Guo et al., 2024] Yufei Guo, Yuanpei Chen, Xiaode Liu, Weihang Peng, Yuhan Zhang, Xuhui Huang, and Zhe Ma. Ternary spike: Learning ternary spikes for spiking neural networks. In Proceedings of the AAAI conference on arti- ficial intelligence, volume 38, pages 12244–12252,

work page 2024
[16]

Spiking trans- former: Introducing accurate addition-only spiking self- attention for transformer

[Guo et al., 2025] Yufei Guo, Xiaode Liu, Yuanpei Chen, Weihang Peng, Yuhan Zhang, and Zhe Ma. Spiking trans- former: Introducing accurate addition-only spiking self- attention for transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24398– 24408,

work page 2025
[17]

Msvit: Improving spik- ing vision transformer using multi-scale attention fusion

[Hua et al., 2025] Wei Hua, Chenlin Zhou, Jibin Wu, Yan- song Chua, and Yangyang Shu. Msvit: Improving spik- ing vision transformer using multi-scale attention fusion. arXiv preprint arXiv:2505.14719,

work page arXiv 2025
[18]

Spikedatten- tion: Training-free and fully spike-driven transformer-to- snn conversion with winner-oriented spike shift for soft- max operation

[Hwang et al., 2024] Sangwoo Hwang, Seunghyun Lee, Da- hoon Park, Donghun Lee, and Jaeha Kung. Spikedatten- tion: Training-free and fully spike-driven transformer-to- snn conversion with winner-oriented spike shift for soft- max operation. Advances in Neural Information Process- ing Systems, 37:67422–67445,

work page 2024
[19]

Cifar-10 (canadian institute for advanced research)

[Krizhevsky et al., 2010] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). URL http://www. cs. toronto. edu/kriz/cifar. html, 5(4):1,

work page 2010
[20]

Cifar10-dvs: an event-stream dataset for object classification

[Li et al., 2017] Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 11:309,

work page 2017
[21]

Integer-valued training and spike-driven inference spiking neural network for high- performance and energy-efficient object detection

[Luo et al., 2024] Xinhao Luo, Man Yao, Yuhong Chou, Bo Xu, and Guoqi Li. Integer-valued training and spike-driven inference spiking neural network for high- performance and energy-efficient object detection. In Eu- ropean Conference on Computer Vision , pages 253–272. Springer,

work page 2024
[22]

Networks of spiking neu- rons: the third generation of neural network models

[Maass, 1997] Wolfgang Maass. Networks of spiking neu- rons: the third generation of neural network models. Neu- ral networks, 10(9):1659–1671,

work page 1997
[23]

Gated at- tention coding for training high-performance and efficient spiking neural networks

[Qiu et al., 2024] Xuerui Qiu, Rui-Jie Zhu, Yuhong Chou, Zhaorui Wang, Liang-jian Deng, and Guoqi Li. Gated at- tention coding for training high-performance and efficient spiking neural networks. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 38, pages 601– 610,

work page 2024
[24]

Quantized spike- driven transformer

[Qiu et al., 2025] Xuerui Qiu, Malu Zhang, Jieyuan Zhang, Wenjie Wei, Honglin Cao, Junsheng Guo, Rui-Jie Zhu, Yi- meng Shan, Yang Yang, and Haizhou Li. Quantized spike- driven transformer. arXiv preprint arXiv:2501.13492 ,

work page arXiv 2025
[25]

Predictive coding in the visual cortex: a functional in- terpretation of some extra-classical receptive-field effects

[Rao and Ballard, 1999] Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional in- terpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79–87,

work page 1999
[26]

One-step spiking transformer with a lin- ear complexity

[Song et al., 2024] Xiaotian Song, Andy Song, Rong Xiao, and Yanan Sun. One-step spiking transformer with a lin- ear complexity. In Proceedings of the Thirty-Third Inter- national Joint Conference on Artificial Intelligence, pages 3142–3150,

work page 2024
[27]

Training data-efficient image transformers & distillation through attention

[Touvron et al., 2021] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablay- rolles, and Herv ´e J ´egou. Training data-efficient image transformers & distillation through attention. In In- ternational conference on machine learning , pages 10347–10357. PMLR,

work page 2021
[28]

Masked spiking transformer

[Wang et al., 2023] Ziqing Wang, Yuetong Fang, Jiahang Cao, Qiang Zhang, Zhongrui Wang, and Renjing Xu. Masked spiking transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1761–1771,

work page 2023
[29]

Multi-scale attention network for single image super-resolution

[Wang et al., 2024] Yan Wang, Yusen Li, Gang Wang, and Xiaoguang Liu. Multi-scale attention network for single image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5950–5960,

work page 2024
[30]

Stkps- net: Spatio-temporal key patch selection network for few shot anomalous action recognition

[Xiao et al., 2026] Jinsheng Xiao, Hao Ma, Ruidi Chen, Xingyu Gao, Hailong Shi, and Zhongyuan Wang. Stkps- net: Spatio-temporal key patch selection network for few shot anomalous action recognition. IEEE Transactions on Information Forensics and Security, 21:827–838,

work page 2026
[31]

Spike- driven transformer

[Yao et al., 2023] Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. Spike- driven transformer. Advances in neural information pro- cessing systems, 36:64043–64058,

work page 2023
[32]

Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next-generation neuro- morphic chips

[Yao et al., 2024a] Man Yao, Jiakui Hu, Tianxiang Hu, Yifan Xu, Zhaokun Zhou, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next-generation neuro- morphic chips. arXiv preprint arXiv:2404.03663,

work page arXiv
[33]

Scaling spike-driven trans- former with efficient spike firing approximation training

[Yao et al., 2025] Man Yao, Xuerui Qiu, Tianxiang Hu, Ji- akui Hu, Yuhong Chou, Keyu Tian, Jianxing Liao, Luziwei Leng, Bo Xu, and Guoqi Li. Scaling spike-driven trans- former with efficient spike firing approximation training. IEEE Transactions on Pattern Analysis and Machine In- telligence,

work page 2025
[34]

Fsta-snn: Frequency-based spatial- temporal attention module for spiking neural networks

[Yu et al., 2025] Kairong Yu, Tianqing Zhang, Hongwei Wang, and Qi Xu. Fsta-snn: Frequency-based spatial- temporal attention module for spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 39, pages 22227–22235,

work page 2025
[35]

Qkformer: Hierarchi- cal spiking transformer using qk attention

[Zhang et al., 2024] Han Zhang, Zhaokun Zhou, Liutao Yu, Liwei Huang, Xiaopeng Fan, Li Yuan, Zhengyu Ma, Hui- hui Zhou, Yonghong Tian, et al. Qkformer: Hierarchi- cal spiking transformer using qk attention. Advances in Neural Information Processing Systems, 37:13074–13098,

work page 2024
[36]

Staa- snn: Spatial-temporal attention aggregator for spiking neu- ral networks

[Zhang et al., 2025] Tianqing Zhang, Kairong Yu, Xian Zhong, Hongwei Wang, Qi Xu, and Qiang Zhang. Staa- snn: Spatial-temporal attention aggregator for spiking neu- ral networks. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 13959–13969,

work page 2025
[37]

Spiliformer: Enhancing spiking transformers with lat- eral inhibition

[Zheng et al., 2025] Zeqi Zheng, Yanchen Huang, Yingchao Yu, Zizheng Zhu, Junfeng Tang, Zhaofei Yu, and Yaochu Jin. Spiliformer: Enhancing spiking transformers with lat- eral inhibition. arXiv preprint arXiv:2503.15986,

work page arXiv 2025
[38]

Spikformer: When spiking neural network meets transformer

[Zhou et al., 2023] Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. In ICLR,

work page 2023
[39]

Spikingformer: A key founda- tion model for spiking neural networks

[Zhou et al., 2026] Chenlin Zhou, Liutao Yu, Zhaokun Zhou, Han Zhang, Jiaqi Wang, Huihui Zhou, Zhengyu Ma, and Yonghong Tian. Spikingformer: A key founda- tion model for spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 2236–2244,

work page 2026
[40]

compute-first, filter-later

All experiments were con- ducted on the CIFAR10 and CIFAR100 datasets. As shown in Fig. 6, the combination of 3 + 7 achieved the best perfor- mance on both datasets. The3×3 kernel focuses on capturing fine local details, whears the 7 × 7 kernel is responsible for perceiving the broader regional context. This large span scale combination enables SMAG to ta...

work page 2024