pith. sign in

arxiv: 2605.09276 · v1 · submitted 2026-05-10 · 💻 cs.LG · cs.CV

Uncertainty-Aware Token Importance Estimation in Spiking Transformers

Wenxuan Liu , Zecheng Hao , Tong Bu , Yuran Wang , Zhaofei Yu This is my paper

Pith reviewed 2026-05-12 04:27 UTC · model grok-4.3

classification 💻 cs.LG cs.CV
keywords spiking transformerstoken pruninguncertainty estimationDirichlet distributionneuromorphic visiontoken importancetemporal dynamicsinference efficiency
0
0 comments X

The pith

Spiking transformers can reduce redundant tokens by measuring how their uncertainty about the final class prediction changes across spiking steps.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that token representations in spiking transformers form gradually over multiple spiking steps, so importance should be judged by how each token's uncertainty about the class decision evolves rather than by single-step activation strength. It models each token's class evidence with a Dirichlet distribution and derives an importance score from the mean and fluctuation of that uncertainty across steps. This score allows pruning of low-contribution tokens during inference without retraining. The approach matters because it directly targets the energy and latency costs that arise when every token is processed at every time step in neuromorphic hardware. Experiments indicate the method yields more favorable accuracy-efficiency curves than prior response-based pruning rules on both static and event-based vision tasks.

Core claim

Tokens exhibit heterogeneous uncertainty trajectories over spiking steps; temporally aggregated uncertainty statistics, obtained by modeling token-wise class evidence with a Dirichlet distribution and summarizing its mean and fluctuation, provide an effective cue for distinguishing informative tokens from redundant ones, enabling more accurate token reduction than activation-magnitude or feature-similarity criteria.

What carries the argument

Uncert, a training-free plug-and-play framework that fits a Dirichlet distribution to each token's class evidence and computes a temporal uncertainty score from the distribution's mean and fluctuation across spiking steps.

Load-bearing premise

That the mean and fluctuation of uncertainty derived from the Dirichlet fit on class evidence accurately reflect a token's actual contribution to the final class prediction.

What would settle it

A controlled pruning experiment on the same benchmarks where tokens ranked by this uncertainty score are removed yet final accuracy falls below the level achieved by magnitude-based or similarity-based pruning at identical token counts.

Figures

Figures reproduced from arXiv: 2605.09276 by Tong Bu, Wenxuan Liu, Yuran Wang, Zecheng Hao, Zhaofei Yu.

Figure 1
Figure 1. Figure 1: Observation of Uncert. (a) Existing token impor [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Motivation of Uncert. Left: Temporal uncertainty [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The overall framework of the proposed Uncert, including the basic spiking transformer backbone (top) and the spiking [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualization of temporal token uncertainty. The [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Pruning visualization of Uncert (QKFormer) on DVS [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Merge visualization of Uncert (QKFormer) on [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison across stages and methods. Here, stage [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
read the original abstract

Spiking transformers have shown strong potential for neuromorphic vision, yet their token processing across multiple spiking steps still introduces substantial redundancy and inference cost. Existing token reduction methods mainly rely on response based cues, such as activation magnitude, firing statistics, or feature similarity. Although effective, these criteria do not explicitly characterize token importance from the perspective of temporally evolving class evidence. In spiking transformers, token representations are progressively formed across multiple spiking steps rather than determined at a single instant, suggesting that token importance should be evaluated not only by instantaneous responses but also by temporal uncertainty patterns. Our key observation is that tokens exhibit heterogeneous uncertainty trajectories over time, and that their temporally aggregated uncertainty statistics provide an effective cue for distinguishing informative tokens from redundant ones. Motivated by this, we propose Uncert, a training free and plug and play token importance estimation framework for spiking transformers. Specifically, Uncert models token wise class evidence with a Dirichlet distribution and summarizes each token temporal uncertainty using its mean and fluctuation across spiking steps, yielding an uncertainty aware importance score for token reduction during inference. Experiments on both static and neuromorphic benchmarks show that Uncert achieves favorable accuracy and efficiency tradeoffs, with the most consistent gains observed under token pruning. Further analysis reveals a clear empirical connection between temporal uncertainty patterns and token contribution, offering new insights into token dynamics in spiking transformers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces Uncert, a training-free and plug-and-play token importance estimation method for spiking transformers. Token-wise class evidence is modeled via a Dirichlet distribution; each token's temporal uncertainty is then summarized by its mean and fluctuation across spiking steps to produce an importance score used for pruning redundant tokens at inference time. Experiments on static and neuromorphic vision benchmarks report favorable accuracy-efficiency trade-offs relative to response-based baselines, with the strongest gains under token pruning, and include analysis linking uncertainty trajectories to token contribution.

Significance. If the empirical link between the proposed uncertainty statistics and actual token contribution holds under rigorous controls, the work supplies a new, training-free lens on token dynamics in spiking networks that could improve efficiency in neuromorphic hardware without retraining. The plug-and-play design and focus on temporally evolving class evidence are clear strengths.

major comments (2)
  1. [Abstract and Experiments section] The central claim—that temporally aggregated Dirichlet uncertainty statistics (mean and fluctuation) provide a cue that distinguishes tokens contributing to the final class prediction—rests on an untested causal assumption. The abstract and experimental sections report accuracy gains under pruning, yet no ablation isolates the uncertainty component from simpler statistics such as firing rate or activation magnitude; without such controls it remains unclear whether the modeling choice is load-bearing or incidental.
  2. [Abstract and §3 (method description)] The manuscript states that tokens exhibit heterogeneous uncertainty trajectories and that these statistics yield an effective importance score, but the provided description and experimental claims lack direct validation (e.g., correlation or causal intervention studies) that the uncertainty trajectory is not merely a proxy for other response cues. This weakens support for the key observation.
minor comments (2)
  1. [Method] Clarify the precise parameterization of the Dirichlet distribution and the exact formulas used to compute mean and fluctuation from the per-step evidence vectors.
  2. [Experiments] Add error bars, number of runs, and statistical significance tests to all reported accuracy-efficiency trade-off results.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and insightful comments, which highlight important opportunities to strengthen the empirical support for our claims. We address each major comment below and will incorporate the suggested analyses in the revised manuscript.

read point-by-point responses

  1. Referee: [Abstract and Experiments section] The central claim—that temporally aggregated Dirichlet uncertainty statistics (mean and fluctuation) provide a cue that distinguishes tokens contributing to the final class prediction—rests on an untested causal assumption. The abstract and experimental sections report accuracy gains under pruning, yet no ablation isolates the uncertainty component from simpler statistics such as firing rate or activation magnitude; without such controls it remains unclear whether the modeling choice is load-bearing or incidental.

    Authors: We agree that the manuscript would benefit from explicit ablations that isolate the contribution of the Dirichlet-modeled uncertainty statistics from simpler response-based measures. While the current experiments compare Uncert against existing response-based token pruning methods and report favorable trade-offs, they do not directly replace the uncertainty mean/fluctuation with firing rate or activation magnitude within the same scoring framework. In the revised manuscript, we will add these component ablations on the vision benchmarks, reporting accuracy and efficiency when using only firing-rate-based importance versus the full uncertainty-aware score. This will clarify whether the temporal uncertainty modeling is load-bearing. revision: yes

  2. Referee: [Abstract and §3 (method description)] The manuscript states that tokens exhibit heterogeneous uncertainty trajectories and that these statistics yield an effective importance score, but the provided description and experimental claims lack direct validation (e.g., correlation or causal intervention studies) that the uncertainty trajectory is not merely a proxy for other response cues. This weakens support for the key observation.

    Authors: We acknowledge that the current analysis linking uncertainty trajectories to token contribution is primarily observational and does not include the quantitative correlation or intervention studies suggested. The manuscript reports an empirical connection and superior pruning performance, but does not explicitly correlate uncertainty scores with per-token prediction impact or perform interventions that swap uncertainty for magnitude-based cues. We will add these validations in the revised version, including Pearson correlations between uncertainty statistics and token removal effects on class evidence, plus the component ablations noted above. These additions will directly address whether the uncertainty trajectory provides information beyond simpler cues. revision: yes

Circularity Check

0 steps flagged

No significant circularity in the proposed token importance estimation framework

full rationale

The paper introduces Uncert as a training-free, plug-and-play procedure that models token-wise class evidence via Dirichlet distributions and aggregates temporal uncertainty (mean and fluctuation) into an importance score for pruning. This construction is explicitly definitional for the estimator rather than a derivation that reduces predictions or results to fitted inputs or self-citations by construction. No equations are presented that equate the output importance score to its inputs tautologically, and the central claims rest on empirical validation across benchmarks rather than self-referential logic. The framework remains self-contained against external benchmarks without load-bearing self-citation chains or ansatz smuggling.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard probabilistic modeling assumptions for class evidence and the empirical link between uncertainty trajectories and token utility; no free parameters or new entities are specified in the abstract.

axioms (1)
  • domain assumption Token representations in spiking transformers evolve over multiple spiking steps, and class evidence for each token can be modeled with a Dirichlet distribution.
    Invoked to justify the uncertainty-aware importance score.

pith-pipeline@v0.9.0 · 5544 in / 1139 out tokens · 70875 ms · 2026-05-12T04:27:38.339476+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

38 extracted references · 38 canonical work pages

  1. [1]

    Berg, Timothy Melano, Jeffrey L

    Arnon Amir, Brian Taba, David J. Berg, Timothy Melano, Jeffrey L. McKinstry, Carmelo di Nolfo, Tapan K. Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, Jeff Kusnitz, Michael DeBole, Steven K. Esser, Tobi Delbrück, Myron Flickner, and Dharmendra S. Modha. 2017. A Low Power, Fully Event- Based Gesture Recognition System. InProc. IEEE/CVF ...

    work page 2017

  2. [2]

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feicht- enhofer, and Judy Hoffman. 2023. Token Merging: Your ViT But Faster. InProc. Int. Conf. Learn. Represent

    work page 2023

  3. [3]

    Tong Bu, Xinyu Shi, and Zhaofei Yu. [n. d.]. Activity Pruning for Efficient Spiking Neural Networks. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems

    work page

  4. [4]

    Yuetong Fang, Ziqing Wang, Lingfeng Zhang, Jiahang Cao, Honglei Chen, and Renjing Xu. 2024. Spiking wavelet transformer. InEuropean conference on com- puter vision. Springer, 19–37

    work page 2024

  5. [5]

    Yuetong Fang, Deming Zhou, Ziqing Wang, Hongwei Ren, ZeCui Zeng, Lusong Li, Shibo Zhou, and Renjing Xu. 2025. Spiking neural networks need high-frequency information. InNeurIPS,. 1–22

    work page 2025

  6. [6]

    2002.Spiking neuron models: Single neurons, populations, plasticity

    Wulfram Gerstner and Werner M Kistler. 2002.Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press

    work page 2002

  7. [7]

    Zecheng Hao, Qichao Ma, Kang Chen, Yi Zhang, Zhaofei Yu, and Tiejun Huang

    work page

  8. [8]

    InInternational Conference on Machine Learning

    Faster and Stronger: When ANN-SNN Conversion Meets Parallel Spiking Calculation. InInternational Conference on Machine Learning

    work page

  9. [9]

    Zihan Huang, Wei Fang, Tong Bu, Peng Xue, Zecheng Hao, Wenxuan Liu, Yuan- hong Tang, Zhaofei Yu, and Tiejun Huang. 2025. Differential coding for training- free ann-to-snn conversion. InInternational Conference on Machine Learning

    work page 2025

  10. [10]

    Zihan Huang, Xinyu Shi, Zecheng Hao, Tong Bu, Jianhao Ding, Zhaofei Yu, and Tiejun Huang. 2024. Towards high-performance spiking transformers from ann to snn conversion

    work page 2024

  11. [11]

    Eugene M Izhikevich. 2004. Which model to use for cortical spiking neurons? IEEE Transactions on Neural Networks(2004)

    work page 2004

  12. [12]

    Donghwa Kang, Youngmoon Lee, Eun-Kyu Lee, Brent Kang, Jinkyu Lee, and Hyeongboo Baek. 2024. AT-SNN: Adaptive Tokens for Vision Transformer on Spiking Neural Network.arXiv preprint arXiv:2408.12293(2024)

    work page arXiv 2024

  13. [13]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    work page 2009

  14. [14]

    Donghyun Lee, Yuhang Li, Youngeun Kim, Shiting Xiao, and Priyadarshini Panda

    work page

  15. [15]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Spiking transformer with spatial-temporal attention. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13948–13958

    work page

  16. [16]

    James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, and Santiago Ontañón. 2022. FNet: Mixing Tokens with Fourier Transforms. InProc. North Am. Chapter Assoc. Comput. Linguist.4296–4313

    work page 2022

  17. [17]

    Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. 2017. Cifar10- dvs: an event-stream dataset for object classification.Frontiers in Neuroscience (2017)

    work page 2017

  18. [18]

    Yanghao Li, Chao-Yuan Wu, Haoqi Fan, Karttikeya Mangalam, Bo Xiong, Jitendra Malik, and Christoph Feichtenhofer. 2022. MViTv2: Improved Multiscale Vision Transformers for Classification and Detection. InProc. IEEE/CVF Conf. Comput. Vis. Pattern Recog.4794–4804

    work page 2022

  19. [19]

    Wenxuan Liu, Xuemei Jia, Xian Zhong, Kui Jiang, Xiaohan Yu, and Mang Ye. 2025. Dynamic and Static Mutual Fitting for Action Recognition.Pattern Recog.157 (2025), 110948

    work page 2025

  20. [20]

    Yue Liu, Shanlin Xiao, Bo Li, and Zhiyi Yu. 2024. SparseSpikformer: A Co-Design Framework for Token and Weight Pruning in Spiking Transformer. InICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 6410–6414

    work page 2024

  21. [21]

    Wolfgang Maass. 1997. Networks of spiking neurons: the third generation of neural network models.Neural Networks(1997)

    work page 1997

  22. [22]

    Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. 2021. DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification. InAdv. Neural Inform. Process. Syst.13937–13949

    work page 2021

  23. [23]

    1976.A Mathematical Theory of Evidence

    G Shafer. 1976.A Mathematical Theory of Evidence. Princeton University Press

    work page 1976

  24. [24]

    Xinyu Shi, Zecheng Hao, and Zhaofei Yu. 2024. SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks. InComputer Vision and Pattern Recognition

    work page 2024

  25. [25]

    William E Vinje and Jack L Gallant. 2000. Sparse coding and decorrelation in primary visual cortex during natural vision.Science287, 5456 (2000), 1273–1276

    work page 2000

  26. [26]

    Wenjie Wei, Xiaolong Zhou, Malu Zhang, Ammar Belatreche, Qian Sun, Yimeng Shan, Dehao Zhang, Zijian Zhou, Zeyu Ma, Yang Yang, and Haizhou Li. 2026. TP-Spikformer: Token Pruned Spiking Transformer. InInternational Conference on Learning Representations (ICLR)

    work page 2026

  27. [27]

    Man Yao, JiaKui Hu, Tianxiang Hu, Yifan Xu, Zhaokun Zhou, Yonghong Tian, Xu Bo, and Guoqi Li. 2024. Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips. InInternational Conference on Learning Representations

    work page 2024

  28. [28]

    Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. 2023. Spike-driven transformer.Advances in neural information processing systems36 (2023), 64043–64058

    work page 2023

  29. [29]

    Man Yao, Xuerui Qiu, Tianxiang Hu, Jiakui Hu, Yuhong Chou, Keyu Tian, Jianxing Liao, Luziwei Leng, Bo Xu, and Guoqi Li. 2025. Scaling spike-driven transformer with efficient spike firing approximation training.IEEE Transactions on Pattern Analysis and Machine Intelligence47, 4 (2025), 2973–2990

    work page 2025

  30. [30]

    Man Yao, Ole Richter, Guangshe Zhao, Ning Qiao, Yannan Xing, Dingheng Wang, Tianxiang Hu, Wei Fang, Tugba Demirci, Michele De Marchi, et al. 2024. Spike- based dynamic computing with asynchronous sensing-computing neuromorphic chip.Nature Communications(2024)

    work page 2024

  31. [31]

    Takashi Yoshida and Kenichi Ohki. 2020. Natural images are reliably repre- sented by sparse and variable populations of neurons in visual cortex.Nature communications(2020)

    work page 2020

  32. [32]

    Wang Zeng, Sheng Jin, Wentao Liu, Chen Qian, Ping Luo, Wanli Ouyang, and Xiaogang Wang. 2022. Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer. InProc. IEEE/CVF Conf. Comput. Vis. Pattern Recog.11091–11101

    work page 2022

  33. [33]

    Xian Zhong, Shengwang Hu, Wenxuan Liu, Wenxin Huang, Jianhao Ding, Zhaofei Yu, and Tiejun Huang. 2024. Towards Low-latency Event-based Visual Recogni- tion with Hybrid Step-wise Distillation Spiking Neural Networks. InProceedings of the 32nd ACM International Conference on Multimedia(Melbourne VIC, Aus- tralia)(MM ’24). 9828–9836

    work page 2024

  34. [34]

    Chenlin Zhou, Han Zhang, Zhaokun Zhou, Liutao Yu, Liwei Huang, Xiaopeng Fan, Li Yuan, Zhengyu Ma, Huihui Zhou, and Yonghong Tian. 2024. Qkformer: Hi- erarchical spiking transformer using qk attention.Advances in Neural Information Conference acronym ’26, Month DD–DD, 2026, City, Country Trovato et al. Processing Systems37 (2024), 13074–13098

    work page 2024

  35. [35]

    Zhaokun Zhou, Kaiwei Che, Wei Fang, Keyu Tian, Yuesheng Zhu, Shuicheng Yan, Yonghong Tian, and Li Yuan. 2024. Spikformer v2: Join the high accuracy club on imagenet with an snn ticket.arXiv preprint arXiv:2401.02020(2024)

    work page arXiv 2024

  36. [36]

    Zhaokun Zhou, Kaiwei Che, Jun Niu, Man Yao, Guoqi Li, Li Yuan, Guibo Luo, and Yuesheng Zhu. 2024. Spatial-temporal spiking feature pruning in spiking transformer.IEEE Transactions on Cognitive and Developmental Systems(2024)

    work page 2024

  37. [37]

    Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. 2022. Spikformer: When spiking neural network meets trans- former.arXiv preprint arXiv:2209.15425(2022)

    work page arXiv 2022

  38. [38]

    Zhengyang Zhuge, Peisong Wang, Xingting Yao, and Jian Cheng. 2024. Towards Efficient Spiking Transformer: A Token Sparsification Framework for Training and Inference Acceleration. InProceedings of the 41st International Conference on Machine Learning. Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    work page 2024