REVIEW 4 major objections 6 minor 95 references
DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A 22M-parameter distilled 'nano' student trained from a 157M-parameter Video-FocalNet Base teacher keeps most of the teacher's top-1 accuracy on five action-recognition benchmarks at about one-seventh the compute.
desk verdict A useful distillation recipe for a 22M video model, but the SOTA claim is overstated and the teacher-pretraining protocol puts the external comparisons on shaky ground. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrier of the argument is the spatio-temporal focal modulation block, a self-attention-free interaction in which the output at a position is $y_i = q(x_i) \odot m_s \odot m_t$, the query projection multiplied element-wise by a spatial modulator and a temporal modulator built from hierarchical depthwise and pointwise convolutions with gated aggregation. This replaces the quadratic token-to-token attention of video transformers with convolution and multiplication operations, which is what makes the 22M-parameter student feasible. The second mechanism is forward-KL distillation: the student minimizes $L_{\mathrm{kd}} = \mathrm{KLD}(\sigma(p/\tau)\,\|\,\sigma(q_\theta/\tau))\cdot\tau^2$ against the teacher's softened logits, combined with cross-entropy as $L_{\mathrm{total}} = \alpha L_{\mathrm{kd}} + (1-\alpha)L_{\mathrm{ce}}$, with $\alpha=0.3$ and temperature $\tau=10$ found optimal. The teacher, Video-FocalNet Base, has embedding dimension 128 and 24 blocks; the student has embedding dimension 96 and 5 blocks distributed as [1,1,2,1] across four stages.
What would settle it
Take the same 22M-parameter student and train it against a teacher pretrained only on ImageNet-1K, with no target-dataset pretraining and every other hyperparameter identical; if top-1 accuracy on SSV2 or Kinetics-400 falls to or below the undestilled student's level, the distillation gains claimed here depend on leaking the evaluation distribution through teacher pretraining.
Extended reading notes
Core claim
The paper claims that spatio-temporal focal modulation — a convolutional mechanism that modulates each query token by element-wise products of spatial and temporal context aggregates rather than computing pairwise attention — can be scaled down to a 22M-parameter student that, supervised by forward-KL distillation from a 157M-parameter teacher, reaches top-1 accuracy of 86.6% on UCF50, 88.4% on UCF101, 82.7% on HMDB51, 70.8% on SSV2, and 83.1% on Kinetics-400 using only 8 input frames. Within the Video-FocalNet family, the student is more than seven times smaller than the Base teacher and stays within a few points of it on every benchmark. Against the published methods listed in the paper, DVFL-Net reports the highest top-1 accuracy on UCF50, UCF101, HMDB51, and SSV2, and ranks just behind its own teacher on SSV2 and Kinetics-400. The authors therefore frame the contribution as an optimal accuracy-efficiency balance rather than a pure accuracy win.
Load-bearing premise
The results assume the teacher is pretrained on the very dataset the student is later tested on, while most comparison methods relied on external pretraining, so the reported gap may partly reflect in-domain supervision rather than the distilled architecture itself.
Editorial extensions
If this is right
- A 22M-parameter, 27-GFLOP student stays within a few tenths of a point of the 157M-parameter, 220-GFLOP teacher on SSV2 and Kinetics-400, so near-transformer accuracy does not require transformer-scale compute.
- Distillation improves the student on all five benchmarks, with gains of 5.2, 5.9, 11.1, 6.1, and 5.6 top-1 points, and the largest gain lands on the smallest dataset (HMDB51), indicating that soft teacher labels help most where labeled data are scarce.
- At 8 frames and 224x224 crops the student tops the paper's listed comparisons on four datasets, implying heavy frame sampling and multi-crop inference are not necessary for competitive accuracy.
- Peak training memory of 5,600 MB per epoch versus 21,950 MB for the teacher on UCF101 means the student fits compute budgets the teacher cannot, which the paper uses to argue for on-device deployment.
Reading between the lines
- The paper's 'state-of-the-art' wording is best read as top of its own comparison lists at far lower cost: the tables themselves place the teacher ahead on SSV2 (71.1 vs 70.8) and Kinetics-400 (83.6 vs 83.1), so the robust claim is parity at one-seventh the compute, not outright supremacy.
- Because the teacher is pretrained on each target dataset (Section IV.C.1) while most listed baselines use external pretraining, a natural test is to distill from a teacher pretrained only on ImageNet-1K; the reported margins would probably shrink if that in-domain advantage were removed.
- The ablation varies $\alpha$ and $\tau$ but never the divergence direction; comparing forward KL with reverse or symmetric KL on the same teacher-student pair would isolate whether the soft labels themselves, or the specific asymmetry, drive the gains.
- The outsized HMDB51 gain (11.1 points) suggests a transferable recipe: for small or fine-grained datasets, a same-domain pretrained teacher may matter more than the student's architectural capacity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes VFL-Net, a 22M-parameter 'nano' variant of the Video-FocalNet architecture, and DVFL-Net, a version trained by offline knowledge distillation from Video-FocalNet Base (157M) using forward KL divergence on soft logits plus cross-entropy. Experiments on UCF50, UCF101, HMDB51, SSV2, and Kinetics-400 report Top-1/Top-5 accuracy, compare against Video-FocalNet variants and published HAR methods, and include ablations over alpha, temperature, and training duration. The paper's headline claim is state-of-the-art Top-1 accuracy across all five datasets at roughly 7x lower computational cost.
Significance. If the reported accuracy-efficiency trade-off survives a matched evaluation protocol, the paper offers a useful lightweight baseline for edge video recognition and a clean demonstration that distillation helps the compact student (VFL-Net vs DVFL-Net, Table VIII). The authors release code and report cost metrics (GFLOPs, memory, training time), which are assets. However, the significance currently rests on overstated SOTA language and on a comparison protocol that is not aligned with most listed baselines.
major comments (4)
- [IV.C.1 and Tables VIII-XIII] The teacher initialization protocol undermines external comparisons. The text states that the teacher is 'first pretrained on a dataset from the datasets used in this work' and that the student is then 'trained from the scratch under the supervision of the teacher (pretrained on the same dataset)'. For UCF50, UCF101, and HMDB51, this means the teacher has already seen that benchmark's training split before producing soft targets; for Kinetics-400, teacher pretraining and KD training use the same training set. Most methods in Tables IX-XIII use external pretraining (Kinetics or ImageNet) or different frame counts, so the reported gains (+5.2 to +11.1) and the superiority claims are not attributable solely to architecture and KD. Please rerun with a teacher pretrained only on Kinetics-400 (or ImageNet) for the smaller datasets, and report baselines under identical frame and crop counts.
- [Section I and Tables XII-XIII] The state-of-the-art claim is contradicted by the paper's own numbers. DVFL-Net achieves 70.8% Top-1 on SSV2 while Video-FocalNet-B achieves 71.1% (Table XII), and 83.1% on Kinetics-400 versus 83.6% for Video-FocalNet-B (Table XIII). Since the teacher itself outperforms the student on these two benchmarks, the statement that DVFL-Net 'achieves state-of-the-art Top-1 accuracy across the UCF50, UCF101, HMDB51, SSV2, and Kinetics-400 datasets' cannot stand. The authors should replace it with a defensible claim such as competitive accuracy at substantially lower GFLOPs, or define a restricted comparison category (for example, among models using 8 frames) and verify that claim.
- [Tables III-VIII and IV.C.2] Pretraining labels for VFL-Net and DVFL-Net are inconsistent and obscure the resource comparison. Every table lists 'ImageNet-1K' under Pretrain for VFL-Net/DVFL-Net, but IV.C.2 states that the student is 'trained from the scratch' and that only the teacher initializes from ImageNet-1K. If the student indeed receives no ImageNet pretraining, the 'Pretrain' column is wrong; if it does, the text and tables need correction. The teacher's target-dataset pretraining protocol also needs to be stated in each table footnote so that the comparison is transparent.
- [III.B and IV.H] The claimed contribution of forward KL divergence is not supported by any ablation against the standard reverse KL or a symmetric variant. The method section emphasizes forward KL, and the distillation equation (2) is one directional choice, but the ablations vary only alpha and tau. Since conventional KD with reverse KL would likely also improve VFL-Net, the experiments do not establish that the forward direction is responsible for the gains in Table VIII. Please add a direct comparison (forward KL versus reverse KL versus both) with the same alpha and tau values.
minor comments (6)
- [Section I and IV.A] The text says 'three publicly available action recognition datasets' and 'three HAR benchmarks' but then lists five datasets; the wording should be corrected to five.
- [III.B] The notation in the distillation section contains corrupted symbols: the teacher and student distributions are 'denoted by √' and '⨿θ', and Eq. (2) is not typeset correctly; these should be fixed to standard p' and q'_theta notation.
- [Throughout] There are several typos and spacing issues, including 'tudies' in the introduction, 'Expending Eq. 3' in III.B, and 'F ormulation' in the III.A heading.
- [Table XIII] The Video-FocalNet-B row cites reference [82], which is the Video Swin Transformer paper; it should cite reference [11].
- [Table XI] The parameter count for CF-IIH is listed as '186.M', which should presumably be '186M'.
- [Section V] The conclusion states that the teacher has '175 million parameters' while Table I and Section III.B say 157M; these numbers should be aligned.
Circularity Check
No significant circularity: the distilled-student accuracy is measured on held-out test labels, and the teacher's in-domain pretraining is a comparison-validity caveat rather than a circular reduction.
full rationale
DVFL-Net's claimed derivation chain consists of (i) adopting Video-FocalNet Base [11] as the teacher; (ii) defining a shallower VFL-Net student (Table I); (iii) optimizing the student with L_total = alpha * KLD(p' || q'_theta) * tau^2 + (1 - alpha) * CE(y, q'_theta) (Eqs. 2-6); and (iv) reporting top-1/top-5 accuracy on the test splits of UCF50, UCF101, HMDB51, SSV2, and Kinetics-400. None of these steps defines its output in terms of its input: the KD gain (e.g., 82.7 vs. 71.6 on HMDB51) is an empirical measurement on held-out test labels, not a quantity encoded in the loss or architecture, so no 'prediction' reduces to a fit by construction. The main caveat is Section IV.C.1: 'The teacher model (i.e., Video-FocalNet Base) is first pretrained on a dataset from the datasets used in this work (i.e., UCF50, UCF101, HMDB51, SSV2, and Kinetics-400).' Because the teacher is pretrained on the target benchmark, the soft-target supervision is not an independent source of external knowledge, and the comparison with published baselines that use external pretraining is not fully matched. This is a genuine comparison-validity limitation, but it is not circular: the student's accuracy is still measured against held-out ground-truth labels, and the internal VFL-Net-versus-DVFL-Net ablation controls for the KD mechanism under a fixed protocol. The paper's own Tables XII and XIII additionally show DVFL-Net trailing Video-FocalNet-B on SSV2 (70.8 vs. 71.1) and Kinetics-400 (83.1 vs. 83.6), contradicting the Section I 'state-of-the-art across ... datasets' claim; that is a correctness/fairness issue, not a circularity issue. The only self-citation, ViT-ReT [73], appears as a SOTA baseline and is not load-bearing to the central efficiency-accuracy claim. Overall, the central derivation is self-contained; score is low (2) to reflect the in-domain teacher-pretraining caveat and the minor non-load-bearing self-citation, not because any result is equivalent to its inputs by definition.
Assumptions & free parameters
free parameters (4)
- KD loss weight alpha =
0.3
- KD temperature tau =
10
- Student depth configuration =
[1,1,2,1], embedding dim 96
- Number of input frames =
8
assumptions (4)
- domain assumption Video-FocalNet's spatio-temporal focal modulation is an effective building block for both teacher and student.
- domain assumption The teacher, fine-tuned on each target dataset, provides soft labels that transfer useful knowledge to the student beyond the ground-truth labels.
- domain assumption Standard training recipe (SGD, cosine schedule, 120 epochs, batch size 8) converges for all datasets.
- domain assumption The reported top-1 accuracies are comparable across methods even though pretraining, frame counts, and evaluation views differ.
Cite this review
Pith. "Pith review of DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition." pith.science (2026). https://pith.science/paper/HUD6CCV2
@misc{pith2026250712426,
author = {Pith},
title = {Pith review of: DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/HUD6CCV2}},
note = {Machine review of arXiv:2507.12426}
}
read the original abstract
The landscape of video recognition has evolved significantly, shifting from traditional Convolutional Neural Networks (CNNs) to Transformer-based architectures for improved accuracy. While 3D CNNs have been effective at capturing spatiotemporal dynamics, recent Transformer models leverage self-attention to model long-range spatial and temporal dependencies. Despite achieving state-of-the-art performance on major benchmarks, Transformers remain computationally expensive, particularly with dense video data. To address this, we propose a lightweight Video Focal Modulation Network, DVFL-Net, which distills spatiotemporal knowledge from a large pre-trained teacher into a compact nano student model, enabling efficient on-device deployment. DVFL-Net utilizes knowledge distillation and spatial-temporal feature modulation to significantly reduce computation while preserving high recognition performance. We employ forward Kullback-Leibler (KL) divergence alongside spatio-temporal focal modulation to effectively transfer both local and global context from the Video-FocalNet Base (teacher) to the proposed VFL-Net (student). We evaluate DVFL-Net on UCF50, UCF101, HMDB51, SSV2, and Kinetics-400, benchmarking it against recent state-of-the-art methods in Human Action Recognition (HAR). Additionally, we conduct a detailed ablation study analyzing the impact of forward KL divergence. The results confirm the superiority of DVFL-Net in achieving an optimal balance between performance and efficiency, demonstrating lower memory usage, reduced GFLOPs, and strong accuracy, making it a practical solution for real-time HAR applications.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Quo vadis, action recognition? a new model and the kinetics dataset,
J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6299–6308
2017
-
[2]
Spatiotemporal residual networks for video action recognition,
R. Christoph and F. A. Pinz, “Spatiotemporal residual networks for video action recognition,” Advances in neural information processing systems, vol. 2, pp. 3468–3476, 2016
2016
-
[3]
Learning spatiotemporal features with 3d convolutional networks,
D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497
2015
-
[4]
Large-scale video classification with convolutional neural networks,
A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, “Large-scale video classification with convolutional neural networks,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, pp. 1725–1732
2014
-
[5]
Beyond short snippets: Deep networks for video classification,
J. Yue-Hei Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici, “Beyond short snippets: Deep networks for video classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 4694–4702
2015
-
[6]
Two-stream convolutional networks for action recognition in videos,
K. Simonyan and A. Zisserman, “Two-stream convolutional networks for action recognition in videos,” Advances in neural information processing systems, vol. 27, 2014
2014
-
[7]
Action recognition using deep 3d cnns with sequential feature aggregation and attention,
F. Anvarov, D. Kim, and B. Song, “Action recognition using deep 3d cnns with sequential feature aggregation and attention,” Electronics, vol. 9, p. 147, 2020
2020
-
[8]
A closer look at spatiotemporal convolutions for action recognition,
D. Tran, H. Wang, L. Torresani, J. Ray, Y . LeCun, and M. Paluri, “A closer look at spatiotemporal convolutions for action recognition,” 2018
2018
Show all 95 references
-
[9]
Human action recognition in videos using convolution long short-term memory network with spatio-temporal networks,
A. Sarabu and A. K. Santra, “Human action recognition in videos using convolution long short-term memory network with spatio-temporal networks,” Emerging Science Journal, vol. 5, pp. 25–33, 2021
2021
-
[10]
Action recognition in videos using pre-trained 2d convolutional neural networks,
J. Kim and C. S. Won, “Action recognition in videos using pre-trained 2d convolutional neural networks,” IEEE Access, vol. 8, pp. 60 179–60 188, 2020
2020
-
[11]
Video-focalnets: Spatio-temporal focal modulation for video action recognition,
S. T. Wasim, M. U. Khattak, M. Naseer, S. Khan, M. Shah, and F. S. Khan, “Video-focalnets: Spatio-temporal focal modulation for video action recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 13 778–13 789
2023
-
[12]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS, 2017
2017
-
[13]
Morph: flexible acceleration for 3d cnn-based video understanding,
K. Hegde, R. Agrawal, Y . Yao, and C. W. Fletcher, “Morph: flexible acceleration for 3d cnn-based video understanding,” 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), vol. 26, pp. 933–946, 2018
2018
-
[14]
Video swin transformer,
Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” 2021
2021
-
[15]
Is space-time attention all you need for video understanding?
G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding?” in ICML, 2021
2021
-
[16]
Vivit: A video vision transformer,
A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lu ˇci´c, and C. Schmid, “Vivit: A video vision transformer,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 6836–6846
2021
-
[17]
Video swin transformer,
Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” in CVPR, 2022. 15
2022
-
[18]
Multiview transformers for video recognition,
S. Yan, X. Xiong, A. Arnab, Z. Lu, M. Zhang, C. Sun, and C. Schmid, “Multiview transformers for video recognition,” in CVPR, 2022
2022
-
[19]
Vivit: a video vision transformer,
A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lu ˇci´c, and C. Schmid, “Vivit: a video vision transformer,” 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021
2021
-
[21]
A short note on the kinetics-700 human action dataset,
J. Carreira, E. Noland, C. Hillier, and A. Zisserman, “A short note on the kinetics-700 human action dataset,” in arXiv preprint arXiv:1907.06987, 2019
1907 arXiv
-
[22]
The" something something
R. Goyal, S. Ebrahimi Kahou, V . Michalski, J. Materzynska, S. West- phal, H. Kim, V . Haenel, I. Fruend, P. Yianilos, M. Mueller-Freitaget al., “The" something something" video database for learning and evaluating visual common sense,” in ICCV, 2017
2017
-
[23]
Tsnet: token sparsification for efficient video transformer,
H. Wang, W. Zhang, and G. Liu, “Tsnet: token sparsification for efficient video transformer,” Applied Sciences, vol. 13, p. 10633, 2023
2023
-
[24]
Dualformer: local- global stratified transformer for efficient video recognition,
Y . Liang, P. Zhou, R. Zimmermann, and S. Yan, “Dualformer: local- global stratified transformer for efficient video recognition,” 2021
2021
-
[25]
Aerobics action recognition algorithm based on three-dimensional convolutional neural network and multilabel clas- sification,
Q. Wang and M. Wang, “Aerobics action recognition algorithm based on three-dimensional convolutional neural network and multilabel clas- sification,” Scientific Programming, vol. 2021, pp. 1–8, 2021
2021
-
[26]
Tsm: temporal shift module for efficient video understanding,
J. Lin, C. Gan, and S. Han, “Tsm: temporal shift module for efficient video understanding,” 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019
2019
-
[27]
Multi-stream interaction networks for human action recognition,
H. Wang, B. Yu, J. Li, L. Zhang, and D. Chen, “Multi-stream interaction networks for human action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 5, pp. 3050–3060, 2021
2021
-
[28]
Spatio- temporal adaptive network with bidirectional temporal difference for action recognition,
Z. Li, J. Li, Y . Ma, R. Wang, Z. Shi, Y . Ding, and X. Liu, “Spatio- temporal adaptive network with bidirectional temporal difference for action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 9, pp. 5174–5185, 2023
2023
-
[29]
Agpn: Action granularity pyramid network for video action recognition,
Y . Chen, H. Ge, Y . Liu, X. Cai, and L. Sun, “Agpn: Action granularity pyramid network for video action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 8, pp. 3912– 3923, 2023
2023
-
[30]
Mawkdn: A multimodal fusion wavelet knowledge distillation approach based on cross-view attention for action recognition,
Z. Quan, Q. Chen, M. Zhang, W. Hu, Q. Zhao, J. Hou, Y . Li, and Z. Liu, “Mawkdn: A multimodal fusion wavelet knowledge distillation approach based on cross-view attention for action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 10, ...
2023
-
[31]
Convolutional neural networks or vision transformers: who will win the race for action recognitions in visual data?
O. Moutik, H. Sekkat, S. Tigani, A. Chehri, R. Saadane, T. A. Tchakoucht, and A. Paul, “Convolutional neural networks or vision transformers: who will win the race for action recognitions in visual data?” Sensors, vol. 23, p. 734, 2023
2023
-
[32]
Decoupled knowledge distillation,
B. Zhao, Q. Cui, R. Song, Y . Qiu, and J. Liang, “Decoupled knowledge distillation,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
2022
-
[33]
Knowledge distil- lation in video-based human action recognition: an intuitive approach to efficient and flexible model training,
F. Camarena, M. Gonzalez-Mendoza, and L. Chang, “Knowledge distil- lation in video-based human action recognition: an intuitive approach to efficient and flexible model training,” Journal of Imaging, vol. 10, p. 85, 2024
2024
-
[34]
Tomato leaf disease recognition based on multi-task distillation learning,
B. Liu, S. Wei, F. Zhang, N. Guo, H. Fan, and W. Yao, “Tomato leaf disease recognition based on multi-task distillation learning,” Frontiers in Plant Science, vol. 14, 2024
2024
-
[35]
Videoadviser: video knowledge distillation for multimodal transfer learning,
Y . Wang, D. Zeng, S. Wada, and S. Kurihara, “Videoadviser: video knowledge distillation for multimodal transfer learning,” IEEE Access, vol. 11, pp. 51 229–51 240, 2023
2023
-
[36]
Generative model- based feature knowledge distillation for action recognition,
G. Wang, P. Zhao, Y . Shi, C. Zhao, and S. Yang, “Generative model- based feature knowledge distillation for action recognition,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 15 474– 15 482, 2024
2024
-
[37]
Distillation of human-object interaction contexts for action recognition,
M. Almushyti and F. W. Li, “Distillation of human-object interaction contexts for action recognition,” 2021
2021
-
[38]
Gaussian error linear units (gelus),
D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415, 2016
2016 arXiv
-
[39]
Recognizing 50 human action categories of web videos,
K. K. Reddy and M. Shah, “Recognizing 50 human action categories of web videos,” Machine vision and applications, vol. 24, no. 5, pp. 971–981, 2013
2013
-
[40]
Ucf101: A dataset of 101 human actions classes from videos in the wild,
K. Soomro, “Ucf101: A dataset of 101 human actions classes from videos in the wild,” arXiv preprint arXiv:1212.0402, 2012
2012 arXiv
-
[41]
Hmdb: a large video database for human motion recognition,
H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “Hmdb: a large video database for human motion recognition,” in 2011 International conference on computer vision. IEEE, 2011, pp. 2556– 2563
2011
-
[42]
The" something something
R. Goyal, S. Ebrahimi Kahou, V . Michalski, J. Materzynska, S. West- phal, H. Kim, V . Haenel, I. Fruend, P. Yianilos, M. Mueller-Freitaget al., “The" something something" video database for learning and evaluat- ing visual common sense,” in Proceedings of the IEEE internation...
2017
-
[43]
The kinetics human action video dataset,
W. Kay, J. Carreira, K. Simonyan, B. Zhang, C. Hillier, S. Vijaya- narasimhan, F. Viola, T. Green, T. Back, P. Natsev et al., “The kinetics human action video dataset,” arXiv preprint arXiv:1705.06950, 2017
2017 arXiv
-
[44]
Uniformer: Unified transformer for efficient spatiotemporal representation learning,
K. Li, Y . Wang, P. Gao, G. Song, Y . Liu, H. Li, and Y . Qiao, “Uniformer: Unified transformer for efficient spatiotemporal representation learning,” arXiv preprint arXiv:2201.04676, 2022
2022 arXiv
-
[45]
Going deeper with convolutions,
C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V . Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in CVPR, 2015
2015
-
[46]
Making sense of neuromorphic event data for human action recognition,
S. Al-Obaidi, H. Al-Khafaji, and C. Abhayaratne, “Making sense of neuromorphic event data for human action recognition,” IEEE Access, vol. 9, pp. 82 686–82 700, 2021
2021
-
[47]
Human action recognition using dis- tance transform and entropy based features,
P. Ramya and R. Rajeswari, “Human action recognition using dis- tance transform and entropy based features,” Multimedia Tools and Applications, vol. 80, no. 6, pp. 8147–8173, 2021
2021
-
[48]
Human action recognition using hybrid deep evolving neural networks,
P. Dasari, L. Zhang, Y . Yu, H. Huang, and R. Gao, “Human action recognition using hybrid deep evolving neural networks,” in 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022, pp. 1–8
2022
-
[49]
Simple-action-guided dictionary learning for complex action recognition,
F. Liu, X. Xu, X. Xing, K. Guo, and L. Wang, “Simple-action-guided dictionary learning for complex action recognition,” Neurocomputing, vol. 501, pp. 387–396, 2022
2022
-
[50]
Human activity classification using the 3dcnn architecture,
R. Vrskova, R. Hudec, P. Kamencay, and P. Sykora, “Human activity classification using the 3dcnn architecture,” Applied Sciences, vol. 12, no. 2, p. 931, 2022
2022
-
[51]
Fast classification and action recognition with event-based imaging,
C. Liu, X. Qi, E. Y . Lam, and N. Wong, “Fast classification and action recognition with event-based imaging,”IEEE access, vol. 10, pp. 55 638– 55 649, 2022
2022
-
[52]
Spatio-temporal features based human action recognition using convolutional long short-term deep neural network,
A. S. Saif, E. D. Wollega, and S. A. Kalevela, “Spatio-temporal features based human action recognition using convolutional long short-term deep neural network,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 5, 2023
2023
-
[53]
Human action recognition using multi-stream fusion and hybrid deep neural networks,
S. Chopra, L. Zhang, and M. Jiang, “Human action recognition using multi-stream fusion and hybrid deep neural networks,” in 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2023, pp. 4852–4858
2023
-
[54]
Self-supervised video representation learning by uncovering spatio-temporal statistics,
J. Wang, J. Jiao, L. Bao, S. He, W. Liu, and Y .-H. Liu, “Self-supervised video representation learning by uncovering spatio-temporal statistics,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3791–3806, 2021
2021
-
[55]
Enhancing self-supervised video representation learning via multi-level feature optimization,
R. Qian, Y . Li, H. Liu, J. See, S. Ding, X. Liu, D. Li, and W. Lin, “Enhancing self-supervised video representation learning via multi-level feature optimization,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 7990–8001
2021
-
[56]
Videomoco: Contrastive video representation learning with temporally adversarial examples,
T. Pan, Y . Song, T. Yang, W. Jiang, and W. Liu, “Videomoco: Contrastive video representation learning with temporally adversarial examples,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 11 205–11 214
2021
-
[57]
Action recognition from a single coded image,
S. Kumawat, T. Okawara, M. Yoshida, H. Nagahara, and Y . Yagi, “Action recognition from a single coded image,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4109–4121, 2022
2022
-
[58]
Tclr: Temporal contrastive learning for video representation,
I. Dave, R. Gupta, M. N. Rizve, and M. Shah, “Tclr: Temporal contrastive learning for video representation,” Computer Vision and Image Understanding, vol. 219, p. 103406, 2022
2022
-
[59]
Learn2augment: learning to composite videos for data augmentation in action recognition,
S. N. Gowda, M. Rohrbach, F. Keller, and L. Sevilla-Lara, “Learn2augment: learning to composite videos for data augmentation in action recognition,” in European conference on computer vision. Springer, 2022, pp. 242–259
2022
-
[60]
Learning from temporal gradient for semi-supervised action recognition,
J. Xiao, L. Jing, L. Zhang, J. He, Q. She, Z. Zhou, A. Yuille, and Y . Li, “Learning from temporal gradient for semi-supervised action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3252–3262
2022
-
[61]
Preserve pre- trained knowledge: Transfer learning with self-distillation for action recognition,
Y . Zhou, Z. He, K. Lu, G. Wang, and G. Wang, “Preserve pre- trained knowledge: Transfer learning with self-distillation for action recognition,” arXiv preprint arXiv:2205.00506, 2022
2022 arXiv
-
[62]
Extreme low- resolution action recognition with confident spatial-temporal attention transfer,
Y . Bai, Q. Zou, X. Chen, L. Li, Z. Ding, and L. Chen, “Extreme low- resolution action recognition with confident spatial-temporal attention transfer,” International Journal of Computer Vision, vol. 131, no. 6, pp. 1550–1565, 2023
2023
-
[63]
Self-supervised video-based action recognition with disturbances,
W. Lin, X. Ding, Y . Huang, and H. Zeng, “Self-supervised video-based action recognition with disturbances,” IEEE Transactions on Image Processing, vol. 32, pp. 2493–2507, 2023. 16
2023
-
[64]
Spatial-temporal exclusive capsule network for open set action recognition,
Y . Feng, J. Gao, S. Yang, and C. Xu, “Spatial-temporal exclusive capsule network for open set action recognition,” IEEE Transactions on Multimedia, vol. 25, pp. 9464–9478, 2023
2023
-
[65]
Sv- former: Semi-supervised video transformer for action recognition,
Z. Xing, Q. Dai, H. Hu, J. Chen, Z. Wu, and Y .-G. Jiang, “Sv- former: Semi-supervised video transformer for action recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 18 816–18 826
2023
-
[66]
Actionhub: a large-scale action video description dataset for zero-shot action recog- nition,
J. Zhou, J. Liang, K.-Y . Lin, J. Yang, and W.-S. Zheng, “Actionhub: a large-scale action video description dataset for zero-shot action recog- nition,” arXiv preprint arXiv:2401.11654, 2024
2024 arXiv
-
[67]
Self-supervised learning via multi-transformation classification for action recognition,
D.-Q. Vu, N. Le, and J.-C. Wang, “Self-supervised learning via multi-transformation classification for action recognition,” in 2024 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). IEEE, 2024, pp. 1–6
2024
-
[68]
Semi-supervised action recog- nition with dynamic temporal information fusion,
H. Qian, J. Zhang, Z. Shi, and Y . Zhang, “Semi-supervised action recog- nition with dynamic temporal information fusion,” Neurocomputing, p. 128683, 2024
2024
-
[69]
Spatiotemporal contrastive video representation learning,
R. Qian, T. Meng, B. Gong, M.-H. Yang, H. Wang, S. Belongie, and Y . Cui, “Spatiotemporal contrastive video representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6964–6974
2021
-
[70]
Representation learning for compressed video action recognition via attentive cross- modal interaction with motion enhancement,
B. Li, J. Chen, D. Zhang, X. Bao, and D. Huang, “Representation learning for compressed video action recognition via attentive cross- modal interaction with motion enhancement,” in Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2022, pp. 1–8
2022
-
[71]
Motion-driven visual tempo learn- ing for video-based action recognition,
Y . Liu, J. Yuan, and Z. Tu, “Motion-driven visual tempo learn- ing for video-based action recognition,” IEEE Transactions on Image Processing, vol. 31, pp. 4104–4116, 2022
2022
-
[72]
Learning spatiotemporal and motion features in a unified 2d network for action recognition,
M. Wang, J. Xing, J. Su, J. Chen, and Y . Liu, “Learning spatiotemporal and motion features in a unified 2d network for action recognition,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 3, pp. 3347–3362, 2022
2022
-
[73]
Vit-ret: Vision and recurrent transformer neural networks for human activity recognition in videos,
J. Wensel, H. Ullah, and A. Munir, “Vit-ret: Vision and recurrent transformer neural networks for human activity recognition in videos,” IEEE Access, 2023
2023
-
[74]
Spatial-temporal interleaved net- work for efficient action recognition,
S. Jiang, H. Zhang, Y . Qi, and Q. Liu, “Spatial-temporal interleaved net- work for efficient action recognition,” IEEE Transactions on Industrial Informatics, 2024
2024
-
[75]
A hybrid transformer framework for efficient activity recog- nition using consumer electronics,
A. Hussain, S. U. Khan, N. Khan, M. W. Bhatt, A. Farouk, J. Bhola, and S. W. Baik, “A hybrid transformer framework for efficient activity recog- nition using consumer electronics,” IEEE Transactions on Consumer Electronics, 2024
2024
-
[76]
A knowledge-based hierarchical causal inference network for video action recognition,
Y . Liu, F. Liu, L. Jiao, Q. Bao, L. Li, Y . Guo, and P. Chen, “A knowledge-based hierarchical causal inference network for video action recognition,” IEEE Transactions on Multimedia, 2024
2024
-
[77]
Is space-time attention all you need for video understanding?
G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding?” in ICML, vol. 2, no. 3, 2021, p. 4
2021
-
[78]
Vidtr: Video transformer without convolutions,
Y . Zhang, X. Li, C. Liu, B. Shuai, Y . Zhu, B. Brattoli, H. Chen, I. Marsic, and J. Tighe, “Vidtr: Video transformer without convolutions,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 13 577–13 587
2021
-
[79]
Keeping your eye on the ball: Tra- jectory attention in video transformers,
M. Patrick, D. Campbell, Y . Asano, I. Misra, F. Metze, C. Feichtenhofer, A. Vedaldi, and J. F. Henriques, “Keeping your eye on the ball: Tra- jectory attention in video transformers,” Advances in neural information processing systems, vol. 34, pp. 12 493–12 506, 2021
2021
-
[80]
Multiscale vision transformers,
H. Fan, B. Xiong, K. Mangalam, Y . Li, Z. Yan, J. Malik, and C. Feichten- hofer, “Multiscale vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 6824–6835
2021
-
[81]
Multiview transformers for video recognition,
S. Yan, X. Xiong, A. Arnab, Z. Lu, M. Zhang, C. Sun, and C. Schmid, “Multiview transformers for video recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 3333–3343
2022
-
[82]
Video swin transformer,
Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 3202–3211
2022
-
[83]
Mvitv2: Improved multiscale vision transformers for classification and detection,
Y . Li, C.-Y . Wu, H. Fan, K. Mangalam, B. Xiong, J. Malik, and C. Feichtenhofer, “Mvitv2: Improved multiscale vision transformers for classification and detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4804– 4814
2022
-
[84]
A novel spatio-temporal-wise network for action recognition,
Z. Cai, “A novel spatio-temporal-wise network for action recognition,” IEEE Access, vol. 11, pp. 49 071–49 080, 2023
2023
-
[85]
D-tsm: Discriminative temporal shift module for action recognition,
S. Lee and S. Hong, “D-tsm: Discriminative temporal shift module for action recognition,” in 2023 20th international conference on ubiquitous robots (UR). IEEE, 2023, pp. 133–136
2023
-
[86]
Scene adaptive mechanism for action recognition,
C. Wu, X.-J. Wu, T. Xu, and J. Kittler, “Scene adaptive mechanism for action recognition,” Computer Vision and Image Understanding, vol. 238, p. 103854, 2024
2024
-
[87]
Sta+: Spatiotemporal adaptation with adaptive model selection for video action recognition,
M. Li, C. Zhang, and X. Zheng, “Sta+: Spatiotemporal adaptation with adaptive model selection for video action recognition,” in 2024 IEEE 4th International Conference on Digital Twins and Parallel Intelligence (DTPI). IEEE, 2024, pp. 177–182
2024
-
[88]
Short-term action learning for video action recognition,
L. Ting-Long, “Short-term action learning for video action recognition,” IEEE Access, vol. 12, pp. 30 867–30 875, 2024
2024
-
[89]
Tea: Temporal excitation and aggregation for action recognition,
Y . Li, B. Ji, X. Shi, J. Zhang, B. Kang, and L. Wang, “Tea: Temporal excitation and aggregation for action recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 909–918
2020
-
[90]
Movinets: Mobile video networks for efficient video recogni- tion,
D. Kondratyuk, L. Yuan, Y . Li, L. Zhang, M. Tan, M. Brown, and B. Gong, “Movinets: Mobile video networks for efficient video recogni- tion,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16 020–16 030
2021
-
[91]
Timebal- ance: Temporally-invariant and temporally-distinctive video represen- tations for semi-supervised action recognition,
I. R. Dave, M. N. Rizve, C. Chen, and M. Shah, “Timebal- ance: Temporally-invariant and temporally-distinctive video represen- tations for semi-supervised action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2341–2352
2023
-
[92]
Dilated multi-temporal modeling for action recognition,
T. Zhang, Y . Wu, and X. Li, “Dilated multi-temporal modeling for action recognition,” Applied Sciences, vol. 13, no. 12, p. 6934, 2023
2023
-
[93]
Learning discriminative spatio- temporal representations for semi-supervised action recognition,
Y . Wang, S. Zhou, K. Xia, and L. Wang, “Learning discriminative spatio- temporal representations for semi-supervised action recognition,” arXiv preprint arXiv:2404.16416, 2024
2024 arXiv
-
[94]
Discrimina- tive segment focus network for fine-grained video action recognition,
B. Sun, X. Ye, T. Yan, Z. Wang, H. Li, and Z. Wang, “Discrimina- tive segment focus network for fine-grained video action recognition,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 20, no. 7, pp. 1–20, 2024
2024
-
[95]
Temporal difference attention for action recog- nition,
H. Zhang and Y . Xia, “Temporal difference attention for action recog- nition,” in 2024 4th International Conference on Neural Networks, Information and Communication (NNICE). IEEE, 2024, pp. 648–652
2024
-
[96]
An efficient motion visual learning method for video action recognition,
B. Wang, F. Chang, C. Liu, W. Wang, and R. Ma, “An efficient motion visual learning method for video action recognition,” Expert Systems with Applications, vol. 255, p. 124596, 2024. Hayat Ullah received his Bachelor’s degree in Computer Science from Islamia College University...
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.