Pith. sign in

REVIEW 4 major objections 7 minor 87 references

SPACT18: Spiking Human Action Recognition Benchmark Dataset with Complementary RGB and Thermal Modalities

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read SPACT18 introduces the first spike-camera action-recognition benchmark with synchronized RGB and thermal video.

desk verdict Genuinely new spike-camera action recognition dataset with RGB and thermal, but the 'synchronized' claim lacks timing evidence and the test-split numbers don't add up; deserves review after a revision pass. read the letter →

arxiv 2507.16151 v1 pith:6YETPMOE submitted 2025-07-22 cs.CV cs.AI

classification cs.CVcs.AI
keywords spikecameravideoactionrecognitionspikingneuralnetworksmultimodalfusionthermalimagingANN-SNNconversionneuromorphicdatasethumanactivity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SPACT18, a video action-recognition benchmark built from native spike-camera recordings, and pairs every spike stream with synchronized RGB and thermal video of the same 18 daily actions performed by 44 participants, yielding 3,168 clips per modality. Its central claim is that this is the first action-recognition dataset captured with a spike camera, filling a gap left by event-camera datasets and by spike datasets devoted to low-level tasks such as reconstruction, optical flow, and depth. The authors also propose a rate-preserving compression algorithm that reduces the raw 100,000-time-step spike streams to 10k and 1k versions, and they report ANN, ANN-SNN-converted, and directly trained SNN baselines across all modalities. If the benchmark is sound, it gives the spiking community a standardized testbed for energy-efficient video understanding and for studying how spiking, RGB, and thermal signals complement each other.

What carries the argument

The load-bearing object is the spike stream $S(x,y,k)\in\{0,1\}^{H\times W\times T}$, produced by the integrate-and-fire pixel circuit $V_{i,j}(t)=\int_{t_{\text{last}}}^{t}\alpha I_{i,j}(\gamma)d\gamma$, with a spike fired when the accumulated charge reaches threshold $\theta$ and a polling interval $\tau=50\,\mu$s. Carrying the argument is the compression algorithm: it partitions the 100k time steps into non-overlapping intervals of length $d$, computes each interval's firing rate $r[i]$, and feeds these rates into an integrate-and-fire neuron whose output is the compressed spike train; Lemma 1 shows that for constant-rate input the original and compressed trains have the same limiting firing rate. This compression converts raw streams into 10k ($d=10$) and 1k ($d=100$) versions that fit into ANN and SNN training, and the rate-encoding of those compressed versions supplies the input used for all ANN baselines.

What would settle it

Record a fast, visible event (for example a hand clap or a flashing light) in the same studio setup and check its onset in all three streams; if the detected onset offset between the 60 Hz RGB, 8.7 Hz thermal, and 20 kHz spike streams varies by more than the corresponding frame interval across trials, the claimed synchronization does not hold and the fusion numbers in Table 4 cannot be attributed to aligned multimodality.

Watch

Extended reading notes

Core claim

SPACT18 is claimed to be the first multimodal video action recognition dataset recorded by a spike camera, using a sensor that emits a binary spike whenever accumulated light at a pixel crosses a threshold, sampled at 20,000 Hz. The 44 subjects each performed 18 actions in two sessions, and the authors segment every session into 3,168 short clips per sensor type, keeping a subject-wise train/validation/test split. On these data the paper reports that rate-encoded 10k spiking input reaches 84.7% accuracy with UniFormer, nearly matching RGB color (85.2%) and thermal color (85.7%), while the more aggressively compressed 1k version drops to 74.4%; direct SNN training lags far behind (best 58.16% with TET-ResNet), and ANN-SNN conversion of MC3 regains accuracy only at 1024–2048 simulation steps. The paper's conclusion is that SPACT18 exposes both the promise of native spiking video and the current bottleneck: efficient direct SNN training and low-latency ANN-SNN conversion.

Load-bearing premise

The dataset's value as a synchronized three-modality benchmark depends on the assumption that the spike, RGB, and thermal cameras were truly aligned in time, yet the paper describes only manual segmentation and quality checks, with no calibration, alignment, or timing-error analysis.

Editorial extensions

If this is right

  • Storing a spike-camera video dataset is tractable: the raw 3.8 TB compresses via LZMA to 425 GB, and the 10k and 1k compressed versions to 42.43 GB and 4.15 GB respectively, so the proposed compression makes public release practical.
  • Spiking data trained as rate-encoded 10k reaches 84.7% with UniFormer, so native spike-camera video can support competitive action recognition without a conversion step that discards temporal structure.
  • Compressing spikes too aggressively is not free: 1k rate-encoded spikes drop to 74.4% on the same model, showing that compression ratio is a first-order hyperparameter for spiking video.
  • Direct SNN training on this benchmark tops out near 58%, and ANN-SNN conversion needs hundreds to thousands of time steps to approach ANN accuracy, so the dataset isolates latency and training efficiency as the central open problems for spiking video.
  • Hybrid ANN-SNN fusion with thermal plus spiking reaches 87.54%, the best result in the paper, indicating that multimodal input can compensate for the weakness of either stream alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same studio protocol could be rerun with an event camera in place of the spike camera to produce a controlled spike-vs-event comparison, which the current fixed setup cannot provide.
  • The compression lemma's invariance of limiting firing rate for constant input suggests a more general design principle: any temporal downsampling that preserves local spike rates keeps first-order statistics, and could be extended to learned or adaptive interval lengths for non-stationary inputs.
  • If explicit timestamps and calibration are added to the released data, SPACT18 could double as a testbed for self-supervised cross-modal learning, where spike streams supervise reconstruction of RGB or thermal frames, an application the paper mentions only as future low-level work.
  • The fact that thermal alone beats RGB on most baselines implies that temperature signatures carry distinctive action information in controlled indoor settings; this would need re-testing outdoors or under variable lighting before generalizing to realistic deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper introduces SPACT18, a multimodal action recognition dataset recorded with a spike camera (20,000 Hz), an RGB camera (60 Hz), and a thermal camera (8.7 Hz) from 44 subjects performing 18 actions in two sessions, yielding 3,168 videos per modality after splitting each 10-second clip in half. The authors propose a temporal compression algorithm that converts raw 100k-step spike trains into 10k and 1k versions, provide ANN baselines (X3D, C2D, I3D, SlowFast, UniFormer) on RGB, thermal, and rate-encoded spike data, and report SNN direct-training, ANN-SNN conversion, and Respike hybrid fusion results. The main claims are that SPACT18 is the first spike-camera VAR dataset, that the modalities are synchronized, and that the compression preserves critical temporal information while reducing latency.

Significance. If the synchronization and split issues are resolved, SPACT18 would be a useful public resource: it is the first action-recognition benchmark captured with a spike camera rather than an event camera, it pairs spike data with RGB and thermal modalities, it includes raw 100k-step spike streams alongside compressed versions, and it provides baselines across ANN, ANN-SNN, direct SNN, and hybrid models. The compression lemma is proved for constant-rate inputs, and the dataset release is a concrete contribution. However, the value of the benchmark as a synchronized multimodal resource and the quantitative claims about compression both depend on evidence that is currently missing, so the significance is conditional.

major comments (4)
  1. [Section 3.1-3.2, Table 9] The central claim of 'synchronized RGB and thermal modalities' is not supported. The hardware section lists native rates of 20,000 Hz, 60 Hz, and 8.7 Hz, but Section 3.2 describes only manual inspection, segmentation into 18 sub-videos, and halving; there is no mention of hardware synchronization, timestamps, inter-camera calibration, or timing-error analysis. With a thermal frame period of roughly 115 ms, a one-frame offset can shift the action window, so the fusion results in Table 4 and the benchmark's synchronized attribute are unverifiable. The authors should report the synchronization procedure and quantify alignment error, or reframe the dataset as co-recorded rather than synchronized.
  2. [Section 4, Figures 9-11] The test split is internally inconsistent. The text states an 80/10/10 subject-wise split of 3,168 videos, which implies about 317 test videos, but each confusion matrix sums to 576 test samples (32 per class), i.e., about 18% of the data. Since 44 subjects cannot be divided into an exact 10% subject-wise split, and Table 1 reports 1,584 samples while the text and Table 7 report 3,168, the exact number of test subjects and per-class sample counts must be stated and the tables/figures recomputed with a consistent split. The ambiguity also affects comparability of all accuracy numbers.
  3. [Section 3.2.1, Tables 3-5] The compression claim is not validated against the raw data. All spike experiments use the compressed 10k or 1k versions, and ANN experiments additionally use rate encoding; no baseline is run on the raw 100k-step spike stream. Consequently, the reported accuracy differences between 10k and 1k cannot be separated from losses introduced by compression, and the raw data's value for SNN benchmarking is not demonstrated. At minimum, one ANN or SNN baseline on a truncated or sampled version of the raw stream should be included, or a clear infeasibility argument with pilot results should be provided.
  4. [Section 3.2.1, Algorithm 1] The pseudocode is not reproducible as written: it initializes u but not v, uses v in the loop before definition, and returns Su rather than s'. The membrane accumulation variable and its initial value (presumably v ← 0) must be stated, and the returned array should be named consistently. This is a core part of the compression contribution, so the pseudocode should be corrected.
minor comments (7)
  1. [Section 1] The phrase 'textural spatiotemporal details than event cameras' should read 'more textural spatiotemporal details than event cameras'.
  2. [Tables 3 and 6] The input-size notation such as '50x2242' and '100x2002' is ambiguous; clarify that dimensions are frames × height × width, e.g., 50×224×224.
  3. [Table 2] The two rate-encoded rows have identical sizes; explain why the rate-encoded versions of 10k and 1k have the same storage footprint.
  4. [Table 4] The standalone 'Spiking 10k' row appears to be the Respike single-modality baseline but is not labeled as such; the row grouping should be made explicit.
  5. [Figures 7-11] The captions contain typos ('Unif ormerB', 'X3_M'), and Figures 9-11 are not referenced in the main text.
  6. [Section 6.2] The word 'devloped' should be 'developed'.
  7. [References] The reference list has duplicate or overlapping entries for Carreira & Zisserman (2017a,b), Feichtenhofer et al. (2019a,b), and Tran et al. (2018a,b,c); these should be consolidated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation is present: the dataset construction, compression algorithm, and baseline evaluations are independent of one another, and the synchronization gap is an evidentiary concern rather than a circular step.

full rationale

The paper's central deliverables—SPACT18 and its compressed variants—are constructed independently of the reported accuracy numbers. The compression algorithm (Section 3.2.1, Algorithm 1 and Equation 2) is a deterministic preprocessing transform with a standalone mathematical claim (Lemma 1) about rate preservation for constant-rate IF spike trains; the lemma is proven in Appendix A.4 and does not depend on any experimental result. The ANN, SNN, and hybrid results in Tables 3 through 6 are evaluations on a fixed 80/10/10 subject-wise split, not predictions obtained by fitting parameters to a target quantity, so there is no fitted-input-called-prediction pattern. The few references to prior work by the same authors (Jiang et al., 2023; Wu et al., 2024) are contextual citations in the ANN-SNN conversion discussion, not load-bearing uniqueness or correctness justifications. The main weakness is different: the claim of 'synchronized RGB and thermal modalities' (abstract, Section 1, and Table 7) is asserted without hardware-synchronization, timestamp, or alignment-error analysis in Section 3.2, which would undermine the multimodal fusion results if the streams are actually misaligned. That is a missing-evidence or correctness risk, not a circularity, because the dataset content does not reduce to the claimed fusion accuracy by construction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The dataset's scientific value rests on the sensor model, an unverified synchronization assumption, and untested generalization of the compression theorem. No free parameters are fitted to a target prediction, and no new physical entities are introduced.

free parameters (1)
  • Compression factor d = 10 (for 10k) and 100 (for 1k)
    Chosen by hand; determines the temporal resolution of the compressed spike trains and affects all downstream results.
assumptions (4)
  • domain assumption The spike camera sensor behaves as an integrate-and-fire accumulator (Equation 1).
    The dataset and compression analysis rely on this sensor model, which is standard in the spike camera literature.
  • ad hoc to paper The three camera streams are temporally synchronized.
    No alignment procedure or timing error analysis is given, yet multimodal fusion experiments assume synchronization.
  • domain assumption Compression preserves spike rates for real spike trains.
    Lemma 1 only proves rate preservation for constant input; real spike data is not constant, and this generalization is untested.
  • domain assumption The manual segmentation into action clips is correct and labels are reliable.
    Section 3.2 says manual inspection was used to fix anomalies, but no labeling accuracy statistics are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SPACT18: Spiking Human Action Recognition Benchmark Dataset with Complementary RGB and Thermal Modalities." pith.science (2026). https://pith.science/paper/6YETPMOE

@misc{pith2026250716151,
  author       = {Pith},
  title        = {Pith review of: SPACT18: Spiking Human Action Recognition Benchmark Dataset with Complementary RGB and Thermal Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6YETPMOE}},
  note         = {Machine review of arXiv:2507.16151}
}
read the original abstract

Spike cameras, bio-inspired vision sensors, asynchronously fire spikes by accumulating light intensities at each pixel, offering ultra-high energy efficiency and exceptional temporal resolution. Unlike event cameras, which record changes in light intensity to capture motion, spike cameras provide even finer spatiotemporal resolution and a more precise representation of continuous changes. In this paper, we introduce the first video action recognition (VAR) dataset using spike camera, alongside synchronized RGB and thermal modalities, to enable comprehensive benchmarking for Spiking Neural Networks (SNNs). By preserving the inherent sparsity and temporal precision of spiking data, our three datasets offer a unique platform for exploring multimodal video understanding and serve as a valuable resource for directly comparing spiking, thermal, and RGB modalities. This work contributes a novel dataset that will drive research in energy-efficient, ultra-low-power video understanding, specifically for action recognition tasks using spike-based data.

Figures

Figures reproduced from arXiv: 2507.16151 by the authors.

Figure 1
Figure 1. Sample output frame from the spike camera for each action of the same subject. Texture [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. summarizes an overview of the data collection process. Compression Algorithm Data Post-Processing Compressed 10 k Compressed 1 k Storage [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The experimental setup consists of the following components: (1) a spike camera for event [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Sample output frame from each camera for the same participant and action. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the proposed spike stream compression method. The original spike stream [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Reconstructed, with TFP, sample output frame from the spike camera for the three versions [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The figure shows visual results from the [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: The figure shows visual results from the [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Confusion Matrix for MC3 baseline model on Spiking-10k Data: The matrix illustrates the model’s performance in classifying various activities, with strong diagonal values indicating accurate predictions and some off-diagonal misclassifications, particularly in similar …
Figure 10
Figure 10. Figure 10: Confusion Matrix for X3DM Model on RGB Data. It shows strong performance in activity classification, with most predictions aligning well with actual labels, though some confusion exists, particularly in close activities like ’drums’ and ’guitar.’ 22 [PITH_FULL_IMAGE:…
Figure 11
Figure 11. Figure 11: Confusion Matrix for UniformerB model on Thermal Data: This matrix demonstrates the model’s best overall performance, with most activities being accurately classified, particularly for ’running,’ ’walking,’ and ’jumping,’ though minor confusion is observed in activiti…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 48 canonical work pages

  1. [1]

    Making sense of neuromorphic event data for human action recognition

    Salah Al-Obaidi, Hiba Al-Khafaji, and Charith Abhayaratne. Making sense of neuromorphic event data for human action recognition. IEEE Access, 9: 0 82686--82700, 2021. doi:10.1109/ACCESS.2021.3085708

  2. [2]

    A. Amir, B. Taba, et al. A low power, fully event-based gesture recognition system. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 7388--7397, 2021

  3. [3]

    Vivit: A video vision transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. Vivit: A video vision transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 6836--6846, 2021. doi:10.1109/ICCV48922.2021.00676

  4. [4]

    Is space-time attention all you need for video understanding? In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139, pp.\ 813--824

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139, pp.\ 813--824. PMLR, 2021

  5. [5]

    Graph-based spatio-temporal feature learning for neuromorphic vision sensing

    Yin Bi, Aaron Chadha, Alhabib Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos. Graph-based spatio-temporal feature learning for neuromorphic vision sensing. IEEE Transactions on Image Processing, 29: 0 9084--9098, 2020. doi:10.1109/TIP.2020.3023597

  6. [6]

    Digit Recognition using Multimodal Spiking Neural Networks

    William Bjorndahl, Jack Easton, Austin Modoff, Eric C. Larson, Joseph Camp, and Prasanna Rangarajan. Digit recognition using multimodal spiking neural networks, 2024. URL https://arxiv.org/abs/2409.00552

  7. [7]

    Optimal ann-snn conversion for high-accuracy and ultra-low-latency spiking neural networks, 2023

    Tong Bu, Wei Fang, Jianhao Ding, PengLin Dai, Zhaofei Yu, and Tiejun Huang. Optimal ann-snn conversion for high-accuracy and ultra-low-latency spiking neural networks, 2023. URL https://arxiv.org/abs/2303.04347

  8. [8]

    Spiking deep convolutional neural networks for energy-efficient object recognition

    Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vision, 113 0 (1): 0 54–66, May 2015. ISSN 0920-5691. doi:10.1007/s11263-014-0788-3. URL https://doi.org/10.1007/s11263-014-0788-3

Show all 87 references
  1. [9]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 6299--6308. IEEE, 2017 a . doi:10.1109/CVPR.2017.502

  2. [10]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 6299--6308, 2017 b

  3. [11]

    A short note about kinetics-600, 2018

    Joao Carreira, Eric Noland, Andras Banki-Horvath, Chloe Hillier, and Andrew Zisserman. A short note about kinetics-600, 2018. URL https://arxiv.org/abs/1808.01340

  4. [12]

    A short note on the kinetics-700 human action dataset, 2022

    Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zisserman. A short note on the kinetics-700 human action dataset, 2022. URL https://arxiv.org/abs/1907.06987

  5. [13]

    Heterogeneous recurrent spiking neural network for spatio-temporal classification

    Biswadeep Chakraborty and Saibal Mukhopadhyay. Heterogeneous recurrent spiking neural network for spatio-temporal classification. Frontiers in Neuroscience, 17, 2023. ISSN 1662-453X. doi:10.3389/fnins.2023.994517. URL https://www.frontiersin.org/journals/neuroscience/articles/...

  6. [14]

    Self-supervised mutual learning for dynamic scene reconstruction of spiking camera

    Shiyan Chen, Chaoteng Duan, Zhaofei Yu, Ruiqin Xiong, and Tiejun Huang. Self-supervised mutual learning for dynamic scene reconstruction of spiking camera. In Lud De Raedt (ed.), Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-2...

  7. [15]

    Spikenvs: Enhancing novel view synthesis from blurry images via spike camera, 2024

    Gaole Dai, Zhenyu Wang, Qinwen Xu, Ming Lu, Wen Chen, Boxin Shi, Shanghang Zhang, and Tiejun Huang. Spikenvs: Enhancing novel view synthesis from blurry images via spike camera, 2024. URL https://arxiv.org/abs/2404.06710

  8. [16]

    Temporal efficient training of spiking neural network via gradient re-weighting

    Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient re-weighting. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=_XNtisL32jv

  9. [17]

    Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer

    Peter U. Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In 2015 International Joint Conference on Neural Networks (IJCNN), pp.\ 1--8, 2015. doi...

  10. [18]

    Spike camera and its coding methods

    Siwei Dong, Tiejun Huang, and Yonghong Tian. Spike camera and its coding methods. In DCC, pp.\ 1--10. Peking University, 2017

  11. [19]

    Bullying10k: A large-scale neuromorphic dataset towards privacy-preserving bullying recognition

    Yiting Dong, Yang Li, Dongcheng Zhao, Guobin Shen, and Yi Zeng. Bullying10k: A large-scale neuromorphic dataset towards privacy-preserving bullying recognition. In Neural Information Processing Systems, 2023. URL https://api.semanticscholar.org/CorpusID:259203896

  12. [20]

    Led: A large-scale real-world paired dataset for event camera denoising

    Yuxing Duan. Led: A large-scale real-world paired dataset for event camera denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 25637--25647, June 2024

  13. [21]

    X3d: Expanding architectures for efficient video recognition, 2020

    Christoph Feichtenhofer. X3d: Expanding architectures for efficient video recognition, 2020. URL https://arxiv.org/abs/2004.04730

  14. [22]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 6202--6211, 2019 a

  15. [23]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 6202--6211, 2019 b . doi:10.1109/ICCV.2019.00632

  16. [24]

    Spikecodec: An end-to-end learned compression framework for spiking camera, 2023

    Kexiang Feng, Chuanmin Jia, Siwei Ma, and Wen Gao. Spikecodec: An end-to-end learned compression framework for spiking camera, 2023. URL https://arxiv.org/abs/2306.14108

  17. [25]

    Tall: Temporal activity localization via language query

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp.\ 5267--5276, 2017

  18. [26]

    Action recognition and benchmark using event cameras

    Yue Gao, Jiaxuan Lu, Siqi Li, Nan Ma, Shaoyi Du, Yipeng Li, and Qionghai Dai. Action recognition and benchmark using event cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 0 (12): 0 14081--14097, 2023. doi:10.1109/TPAMI.2023.3300741

  19. [27]

    Ava: A video dataset of spatio-temporally localized atomic visual actions

    Chunhui Gu, Chen Sun, David A Ross, Carl Vondrick, Caroline Pantofaru, Yeqing Li, Sudheendra Vijayanarasimhan, George Toderici, Simone Ricco, Rahul Sukthankar, Cordelia Schmid, and Jitendra Malik. Ava: A video dataset of spatio-temporally localized atomic visual actions. In Pr...

  20. [28]

    Y. Guo, X. Huang, and Z. Ma. Direct learning-based deep spiking neural networks: A review. Frontiers in Neuroscience, 17: 0 1209795, 2023

  21. [29]

    Activitynet: A large-scale video benchmark for human activity understanding

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 961--970, 2015. doi:10.1109/CVPR.2015.7298698

  22. [30]

    Optical flow estimation for spiking camera

    Liwen Hu, Rui Zhao, Ziluo Ding, Lei Ma, Boxin Shi, Ruiqin Xiong, and Tiejun Huang. Optical flow estimation for spiking camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 17844--17853. IEEE, 2022. URL https://github.com/Acn...

  23. [31]

    Advancing spiking neural networks toward deep residual learning

    Yifan Hu, Lei Deng, Yujie Wu, Man Yao, and Guoqi Li. Advancing spiking neural networks toward deep residual learning. IEEE Transactions on Neural Networks and Learning Systems, pp.\ 1--15, 2024. doi:10.1109/TNNLS.2024.3355393

  24. [32]

    in the wild

    Haroon Idrees, Amir R. Zamir, Yu-Gang Jiang, Alex Gorban, Ivan Laptev, Rahul Sukthankar, and Mubarak Shah. The thumos challenge on action recognition for videos “in the wild”. Computer Vision and Image Understanding, 155: 0 1--23, 2017. ISSN 1077-3142. doi:https://doi.org/10.1...

  25. [33]

    A unified optimization framework of ANN - SNN conversion: Towards optimal mapping from activation values to firing rates

    Haiyan Jiang, Srinivas Anumasa, Giulia De Masi, Huan Xiong, and Bin Gu. A unified optimization framework of ANN - SNN conversion: Towards optimal mapping from activation values to firing rates. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato,...

  26. [34]

    The kinetics human action video dataset, 2017

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, Mustafa Suleyman, and Andrew Zisserman. The kinetics human action video dataset, 2017. URL https://arxiv.org/abs/1705.06950

  27. [35]

    Kuehne, H

    H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre. Hmdb: A large video database for human motion recognition. In 2011 International Conference on Computer Vision, pp.\ 2556--2563, 2011. doi:10.1109/ICCV.2011.6126543

  28. [36]

    Cifar10-dvs: An event-stream dataset for object classification

    Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: An event-stream dataset for object classification. Frontiers in Neuroscience, 11, 2017. ISSN 1662-453X. doi:10.3389/fnins.2017.00309. URL https://www.frontiersin.org/journals/neuroscience/articles/10...

  29. [37]

    Uniformer: Unified transformer for efficient spatiotemporal representation learning

    Kunchang Li, Yali Wang, Peng Gao, Guanglu Song, Yu Liu, Hongsheng Li, and Yu Qiao. Uniformer: Unified transformer for efficient spatiotemporal representation learning. In International Conference on Learning Representations (ICLR), 2022. doi:10.48550/arXiv.2201.04676

  30. [38]

    Uniformerv2: Unlocking the potential of image vits for video understanding

    Kunchang Li, Duo He, Yifan Xue, Yali Wang, Yu Qiao, and Jifeng Dai. Uniformerv2: Unlocking the potential of image vits for video understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 12345--12354, 2023. doi:10.1109/ICCV.2023.00123

  31. [39]

    Ntu rgb+d 120: A large-scale benchmark for 3d human activity understanding

    Jun Liu, Amir Shahroudy, Mauricio Perez, Gang Wang, Ling-Yu Duan, and Alex C Kot. Ntu rgb+d 120: A large-scale benchmark for 3d human activity understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42 0 (10): 0 2684--2701, 2020

  32. [40]

    Event-based action recognition using motion information and spiking neural networks

    Qianhui Liu, Dong Xing, Huajin Tang, De Ma, and Gang Pan. Event-based action recognition using motion information and spiking neural networks. In International Joint Conference on Artificial Intelligence, 2021. URL https://api.semanticscholar.org/CorpusID:237101117

  33. [41]

    Energy-efficient distributed spiking neural network for wireless edge intelligence

    Yanzhen Liu, Zhijin Qin, and Geoffrey Li. Energy-efficient distributed spiking neural network for wireless edge intelligence. IEEE Transactions on Wireless Communications, PP: 0 1--1, 09 2024. doi:10.1109/TWC.2024.3374549

  34. [42]

    Fineaction: A fine-grained video dataset for temporal action localization

    Yi Liu, Limin Wang, Yali Wang, Xiao Ma, and Yu Qiao. Fineaction: A fine-grained video dataset for temporal action localization. IEEE Transactions on Image Processing (TIP), 2022 a

  35. [43]

    Video swin transformer

    Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 3202--3211, 2022 b . doi:10.1109/CVPR52688.2022.00323

  36. [44]

    Neuromorphic benchmark datasets for pedestrian detection, action recognition, and fall detection

    Shu Miao, Guang Chen, Xiangyu Ning, Yang Zi, Kejia Ren, Zhenshan Bing, and Alois C Knoll. Neuromorphic benchmark datasets for pedestrian detection, action recognition, and fall detection. Frontiers in neurorobotics, 13: 0 38, 2019

  37. [45]

    Learning to recognize actions from limited training examples using a recurrent spiking neural model

    Priyadarshini Panda and Narayan Srinivasa. Learning to recognize actions from limited training examples using a recurrent spiking neural model. Frontiers in Neuroscience, 12, 2018. ISSN 1662-453X. doi:10.3389/fnins.2018.00126. URL https://www.frontiersin.org/journals/neuroscie...

  38. [46]

    LZMA SDK (software development kit), 2013

    Igor Pavlov. LZMA SDK (software development kit), 2013. Available at: https://www.7-zip.org/sdk.html

  39. [47]

    R. Poppe. A survey on vision-based human action recognition. Image and Vision Computing, 28 0 (6): 0 976--990, 2010

  40. [48]

    Stdp based unsupervised multimodal learning with cross-modal processing in spiking neural networks

    Nitin Rathi and Kaushik Roy. Stdp based unsupervised multimodal learning with cross-modal processing in spiking neural networks. IEEE Transactions on Emerging Topics in Computational Intelligence, 5: 0 143--153, 2021. URL https://api.semanticscholar.org/CorpusID:70168107

  41. [49]

    K. Roy, A. Jaiswal, and P. Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575: 0 607--617, 2019

  42. [50]

    Fusing event-based camera and radar for slam using spiking neural networks with continual stdp learning

    Ali Safa, Tim Verbelen, Ilja Ocket, André Bourdoux, Hichem Sahli, Francky Catthoor, and Georges Gielen. Fusing event-based camera and radar for slam using spiking neural networks with continual stdp learning. In 2023 IEEE International Conference on Robotics and Automation (IC...

  43. [51]

    Convolutional spiking neural networks for spatio-temporal feature extraction

    Alireza Samadzadeh, Fatemeh Sadat Tabatabaei Far, Ali Javadi, Ahmad Nickabadi, and Morteza Haghir Chehreghani. Convolutional spiking neural networks for spatio-temporal feature extraction. Neural Processing Letters, 55: 0 6979--6995, 2020. URL https://api.semanticscholar.org/C...

  44. [52]

    Schuldt, I

    C. Schuldt, I. Laptev, and B. Caputo. Recognizing human actions: a local svm approach. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., volume 3, pp.\ 32--36 Vol.3, 2004. doi:10.1109/ICPR.2004.1334462

  45. [53]

    Finegym: A hierarchical video dataset for fine-grained action understanding

    Dian Shao, Yue Zhao, Bo Dai, and Dahua Lin. Finegym: A hierarchical video dataset for fine-grained action understanding. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020

  46. [54]

    Two-stream convolutional networks for action recognition in videos

    Karen Simonyan and Andrew Zisserman. Two-stream convolutional networks for action recognition in videos. In Advances in Neural Information Processing Systems (NeurIPS), pp.\ 568--576, 2014

  47. [55]

    Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012. URL https://arxiv.org/abs/1212.0402

  48. [56]

    Learning spatiotemporal features with 3d convolutional networks

    Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp.\ 4489--4497, 2015. doi:10.1109/ICCV.2015.510

  49. [57]

    A closer look at spatiotemporal convolutions for action recognition

    Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 6450--6459, 2018 a . doi:10.1109...

  50. [58]

    A closer look at spatiotemporal convolutions for action recognition, 2018 b

    Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition, 2018 b . URL https://arxiv.org/abs/1711.11248

  51. [59]

    A closer look at spatiotemporal convolutions for action recognition

    Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 6450--6459, 2018 c

  52. [60]

    A new ann-snn conversion method with high accuracy, low latency and good robustness

    Bingsen Wang, Jian Cao, Jue Chen, Shuo Feng, and Yuan Wang. A new ann-snn conversion method with high accuracy, low latency and good robustness. In Edith Elkind (ed.), Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23 , pp.\ 3...

  53. [61]

    Temporal segment networks: Towards good practices for deep action recognition

    Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks: Towards good practices for deep action recognition. In European Conference on Computer Vision, 2016. URL https://api.semanticscholar.org/CorpusID:5711057

  54. [62]

    Dailydvs-200: A comprehensive benchmark dataset for event-based action recognition

    Qi Wang, Zhou Xu, Yuming Lin, Jingtao Ye, Hongsheng Li, Guangming Zhu, Syed Afaq Ali Shah, Mohammed Bennamoun, and Liang Zhang. Dailydvs-200: A comprehensive benchmark dataset for event-based action recognition. arXiv preprint arXiv:2407.05106, 2024 a

  55. [63]

    Temporal spiking recurrent neural network for action recognition

    Wei Wang, Siyuan Hao, Yunchao Wei, Shengtao Xiao, Jiashi Feng, and Nicu Sebe. Temporal spiking recurrent neural network for action recognition. IEEE Access, 7: 0 117165--117175, 2019. doi:10.1109/ACCESS.2019.2936604

  56. [64]

    Hardvs: Revisiting human activity recognition with dynamic vision sensors, 2022 a

    Xiao Wang, Zongzhen Wu, Bo Jiang, Zhimin Bao, Lin Zhu, Guoqi Li, Yaowei Wang, and Yonghong Tian. Hardvs: Revisiting human activity recognition with dynamic vision sensors, 2022 a . URL https://arxiv.org/abs/2211.09648

  57. [65]

    Event stream-based visual object tracking: A high-resolution benchmark dataset and a novel baseline

    Xiao Wang, Shiao Wang, Chuanming Tang, Lin Zhu, Bo Jiang, Yonghong Tian, and Jin Tang. Event stream-based visual object tracking: A high-resolution benchmark dataset and a novel baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp....

  58. [66]

    Event-enhanced multi-modal spiking neural network for dynamic obstacle avoidance

    Yang Wang, Bo Dong, Yuji Zhang, Yunduo Zhou, Haiyang Mei, Ziqi Wei, and Xin Yang. Event-enhanced multi-modal spiking neural network for dynamic obstacle avoidance. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, pp.\ 3138–3148, New York, NY, USA,...

  59. [67]

    Learning stereo depth estimation with bio-inspired spike cameras

    Yixuan Wang, Jianing Li, Lin Zhu, Xijie Xiang, Tiejun Huang, and Yonghong Tian. Learning stereo depth estimation with bio-inspired spike cameras. In 2022 IEEE International Conference on Multimedia and Expo (ICME), pp.\ 1--6, 2022 b . doi:10.1109/ICME52920.2022.9859975

  60. [68]

    Deep learning-based video action recognition: A review

    Mohd Hanief Wani and Arman Rasool Faridi. Deep learning-based video action recognition: A review. In 2022 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), pp.\ 243--249, 2022. doi:10.1109/ICCCIS56430.2022.10037736

  61. [69]

    Ftbc: Forward temporal bias correction for optimizing ann-snn conversion, 2024

    Xiaofeng Wu, Velibor Bojkovic, Bin Gu, Kun Suo, and Kai Zou. Ftbc: Forward temporal bias correction for optimizing ann-snn conversion, 2024. URL https://arxiv.org/abs/2403.18388

  62. [70]

    Learning super-resolution reconstruction for high temporal resolution spike stream

    Xijie Xiang, Lin Zhu, Jianing Li, Yixuan Wang, Tiejun Huang, and Yonghong Tian. Learning super-resolution reconstruction for high temporal resolution spike stream. IEEE Transactions on Circuits and Systems for Video Technology, 33 0 (1): 0 16--29, 2023. doi:10.1109/TCSVT.2021.3130147

  63. [71]

    Respike: Residual frames-based hybrid spiking neural networks for efficient action recognition, 2024

    Shiting Xiao, Yuhang Li, Youngeun Kim, Donghyun Lee, and Priyadarshini Panda. Respike: Residual frames-based hybrid spiking neural networks for efficient action recognition, 2024. URL https://arxiv.org/abs/2409.01564

  64. [72]

    Converting artificial neural networks to ultralow-latency spiking neural networks for action recognition

    Hong You, Xian Zhong, Wenxuan Liu, Qi Wei, Wenxin Huang, Zhaofei Yu, and Tiejun Huang. Converting artificial neural networks to ultralow-latency spiking neural networks for action recognition. IEEE Transactions on Cognitive and Developmental Systems, 16 0 (4): 0 1533--1545, 20...

  65. [73]

    Svformer: A direct training spiking transformer for efficient video action recognition

    Liutao Yu, Liwei Huang, Chenlin Zhou, Han Zhang, Zhengyu Ma, and Yonghong Tian. Svformer: A direct training spiking transformer for efficient video action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2024

  66. [74]

    High-accuracy and energy-efficient action recognition with deep spiking neural network

    Jingren Zhang, Jingjing Wang, Xie Di, and Shiliang Pu. High-accuracy and energy-efficient action recognition with deep spiking neural network. In Mohammad Tanveer, Sonali Agarwal, Seiichi Ozawa, Asif Ekbal, and Adam Jatowt (eds.), Neural Information Processing, pp.\ 279--292, ...

  67. [75]

    Spike transformer: Monocular depth estimation for spiking camera

    Jiyuan Zhang, Lulu Tang, Zhaofei Yu, Jiwen Lu, and Tiejun Huang. Spike transformer: Monocular depth estimation for spiking camera. In Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VII, pp.\ 34–52, Berlin, Heidel...

  68. [76]

    Learning temporal-ordered representation for spike streams based on discrete wavelet transforms

    Jiyuan Zhang, Shanshan Jia, Zhaofei Yu, and Tiejun Huang. Learning temporal-ordered representation for spike streams based on discrete wavelet transforms. In AAAI Conference on Artificial Intelligence, 2023 b . URL https://api.semanticscholar.org/CorpusID:259629663

  69. [77]

    Motion estimation for spike camera data sequence via spike interval analysis

    Jing Zhao, Ruiqin Xiong, Rui Zhao, Jin Wang, Siwei Ma, and Tiejun Huang. Motion estimation for spike camera data sequence via spike interval analysis. In 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp.\ 371--374, 2020. doi:10.1109/V...

  70. [78]

    Spk2imgnet: Learning to reconstruct dynamic scene from continuous spike stream

    Jing Zhao, Ruiqin Xiong, Hangfan Liu, Jian Zhang, and Tiejun Huang. Spk2imgnet: Learning to reconstruct dynamic scene from continuous spike stream. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 11991--12000, 2021. doi:10.1109/CVPR46437.2021.01182

  71. [79]

    Learning optical flow from continuous spike streams

    Rui Zhao, Ruiqin Xiong, Jing Zhao, Zhaofei Yu, Xiaopeng Fan, and Tiejun Huang. Learning optical flow from continuous spike streams. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ 7...

  72. [80]

    Boosting spike camera image reconstruction from a perspective of dealing with spike fluctuations

    Rui Zhao, Ruiqin Xiong, Jing Zhao, Jian Zhang, Xiaopeng Fan, Zhaofei Yu, and Tiejun Huang. Boosting spike camera image reconstruction from a perspective of dealing with spike fluctuations. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 2495...

  73. [81]

    Spike-based motion estimation for object tracking through bio-inspired unsupervised learning

    Yajing Zheng, Zhaofei Yu, Song Wang, and Tiejun Huang. Spike-based motion estimation for object tracking through bio-inspired unsupervised learning. IEEE Transactions on Image Processing, 32: 0 335--349, 2023. doi:10.1109/TIP.2022.3228168

  74. [82]

    A retina-inspired sampling method for visual texture reconstruction

    Lin Zhu, Siwei Dong, Tiejun Huang, and Yonghong Tian. A retina-inspired sampling method for visual texture reconstruction. In IEEE International Conference on Multimedia and Expo (ICME), pp.\ 1--5. Peking University, 2019

  75. [83]

    Retina-like visual image reconstruction via spiking neural model

    Lin Zhu, Siwei Dong, Jianing Li, Tiejun Huang, and Yonghong Tian. Retina-like visual image reconstruction via spiking neural model. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 1435--1443, 2020. doi:10.1109/CVPR42600.2020.00151

  76. [84]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  77. [85]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  78. [86]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  79. [87]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.