Pith. sign in

REVIEW 4 major objections 6 minor 95 references

DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read A 22M-parameter distilled 'nano' student trained from a 157M-parameter Video-FocalNet Base teacher keeps most of the teacher's top-1 accuracy on five action-recognition benchmarks at about one-seventh the compute.

desk verdict A useful distillation recipe for a 22M video model, but the SOTA claim is overstated and the teacher-pretraining protocol puts the external comparisons on shaky ground. read the letter →

arxiv 2507.12426 v2 pith:HUD6CCV2 submitted 2025-07-16 cs.CV

classification cs.CV
keywords humanactionrecognitionknowledgedistillationspatio-temporalfocalmodulationlightweightvideomodeltransformersforwardKLdivergencecompressionon-devicedeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DVFL-Net is a 22-million-parameter action-recognition model that shrinks the Video-FocalNet architecture to a five-block 'nano' configuration and then trains it by distilling spatio-temporal knowledge from the 157-million-parameter Video-FocalNet Base teacher using forward Kullback-Leibler divergence. The paper's central claim is that this distilled student keeps the teacher's top-1 accuracy largely intact across UCF50, UCF101, HMDB51, SSV2, and Kinetics-400 while cutting parameters roughly seven-fold and GFLOPs from 220 to 27, making video recognition practical on devices that cannot host a large transformer. Distillation adds 5.2 to 11.1 accuracy points over the same student trained alone, with the largest gain on the smallest dataset (HMDB51). The claim matters because it tests whether a cheap convolutional focal-modulation backbone can absorb a large video model's knowledge instead of paying self-attention's quadratic cost.

What carries the argument

The carrier of the argument is the spatio-temporal focal modulation block, a self-attention-free interaction in which the output at a position is $y_i = q(x_i) \odot m_s \odot m_t$, the query projection multiplied element-wise by a spatial modulator and a temporal modulator built from hierarchical depthwise and pointwise convolutions with gated aggregation. This replaces the quadratic token-to-token attention of video transformers with convolution and multiplication operations, which is what makes the 22M-parameter student feasible. The second mechanism is forward-KL distillation: the student minimizes $L_{\mathrm{kd}} = \mathrm{KLD}(\sigma(p/\tau)\,\|\,\sigma(q_\theta/\tau))\cdot\tau^2$ against the teacher's softened logits, combined with cross-entropy as $L_{\mathrm{total}} = \alpha L_{\mathrm{kd}} + (1-\alpha)L_{\mathrm{ce}}$, with $\alpha=0.3$ and temperature $\tau=10$ found optimal. The teacher, Video-FocalNet Base, has embedding dimension 128 and 24 blocks; the student has embedding dimension 96 and 5 blocks distributed as [1,1,2,1] across four stages.

What would settle it

Take the same 22M-parameter student and train it against a teacher pretrained only on ImageNet-1K, with no target-dataset pretraining and every other hyperparameter identical; if top-1 accuracy on SSV2 or Kinetics-400 falls to or below the undestilled student's level, the distillation gains claimed here depend on leaking the evaluation distribution through teacher pretraining.

Watch

Extended reading notes

Core claim

The paper claims that spatio-temporal focal modulation — a convolutional mechanism that modulates each query token by element-wise products of spatial and temporal context aggregates rather than computing pairwise attention — can be scaled down to a 22M-parameter student that, supervised by forward-KL distillation from a 157M-parameter teacher, reaches top-1 accuracy of 86.6% on UCF50, 88.4% on UCF101, 82.7% on HMDB51, 70.8% on SSV2, and 83.1% on Kinetics-400 using only 8 input frames. Within the Video-FocalNet family, the student is more than seven times smaller than the Base teacher and stays within a few points of it on every benchmark. Against the published methods listed in the paper, DVFL-Net reports the highest top-1 accuracy on UCF50, UCF101, HMDB51, and SSV2, and ranks just behind its own teacher on SSV2 and Kinetics-400. The authors therefore frame the contribution as an optimal accuracy-efficiency balance rather than a pure accuracy win.

Load-bearing premise

The results assume the teacher is pretrained on the very dataset the student is later tested on, while most comparison methods relied on external pretraining, so the reported gap may partly reflect in-domain supervision rather than the distilled architecture itself.

Editorial extensions

If this is right

  • A 22M-parameter, 27-GFLOP student stays within a few tenths of a point of the 157M-parameter, 220-GFLOP teacher on SSV2 and Kinetics-400, so near-transformer accuracy does not require transformer-scale compute.
  • Distillation improves the student on all five benchmarks, with gains of 5.2, 5.9, 11.1, 6.1, and 5.6 top-1 points, and the largest gain lands on the smallest dataset (HMDB51), indicating that soft teacher labels help most where labeled data are scarce.
  • At 8 frames and 224x224 crops the student tops the paper's listed comparisons on four datasets, implying heavy frame sampling and multi-crop inference are not necessary for competitive accuracy.
  • Peak training memory of 5,600 MB per epoch versus 21,950 MB for the teacher on UCF101 means the student fits compute budgets the teacher cannot, which the paper uses to argue for on-device deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's 'state-of-the-art' wording is best read as top of its own comparison lists at far lower cost: the tables themselves place the teacher ahead on SSV2 (71.1 vs 70.8) and Kinetics-400 (83.6 vs 83.1), so the robust claim is parity at one-seventh the compute, not outright supremacy.
  • Because the teacher is pretrained on each target dataset (Section IV.C.1) while most listed baselines use external pretraining, a natural test is to distill from a teacher pretrained only on ImageNet-1K; the reported margins would probably shrink if that in-domain advantage were removed.
  • The ablation varies $\alpha$ and $\tau$ but never the divergence direction; comparing forward KL with reverse or symmetric KL on the same teacher-student pair would isolate whether the soft labels themselves, or the specific asymmetry, drive the gains.
  • The outsized HMDB51 gain (11.1 points) suggests a transferable recipe: for small or fine-grained datasets, a same-domain pretrained teacher may matter more than the student's architectural capacity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes VFL-Net, a 22M-parameter 'nano' variant of the Video-FocalNet architecture, and DVFL-Net, a version trained by offline knowledge distillation from Video-FocalNet Base (157M) using forward KL divergence on soft logits plus cross-entropy. Experiments on UCF50, UCF101, HMDB51, SSV2, and Kinetics-400 report Top-1/Top-5 accuracy, compare against Video-FocalNet variants and published HAR methods, and include ablations over alpha, temperature, and training duration. The paper's headline claim is state-of-the-art Top-1 accuracy across all five datasets at roughly 7x lower computational cost.

Significance. If the reported accuracy-efficiency trade-off survives a matched evaluation protocol, the paper offers a useful lightweight baseline for edge video recognition and a clean demonstration that distillation helps the compact student (VFL-Net vs DVFL-Net, Table VIII). The authors release code and report cost metrics (GFLOPs, memory, training time), which are assets. However, the significance currently rests on overstated SOTA language and on a comparison protocol that is not aligned with most listed baselines.

major comments (4)
  1. [IV.C.1 and Tables VIII-XIII] The teacher initialization protocol undermines external comparisons. The text states that the teacher is 'first pretrained on a dataset from the datasets used in this work' and that the student is then 'trained from the scratch under the supervision of the teacher (pretrained on the same dataset)'. For UCF50, UCF101, and HMDB51, this means the teacher has already seen that benchmark's training split before producing soft targets; for Kinetics-400, teacher pretraining and KD training use the same training set. Most methods in Tables IX-XIII use external pretraining (Kinetics or ImageNet) or different frame counts, so the reported gains (+5.2 to +11.1) and the superiority claims are not attributable solely to architecture and KD. Please rerun with a teacher pretrained only on Kinetics-400 (or ImageNet) for the smaller datasets, and report baselines under identical frame and crop counts.
  2. [Section I and Tables XII-XIII] The state-of-the-art claim is contradicted by the paper's own numbers. DVFL-Net achieves 70.8% Top-1 on SSV2 while Video-FocalNet-B achieves 71.1% (Table XII), and 83.1% on Kinetics-400 versus 83.6% for Video-FocalNet-B (Table XIII). Since the teacher itself outperforms the student on these two benchmarks, the statement that DVFL-Net 'achieves state-of-the-art Top-1 accuracy across the UCF50, UCF101, HMDB51, SSV2, and Kinetics-400 datasets' cannot stand. The authors should replace it with a defensible claim such as competitive accuracy at substantially lower GFLOPs, or define a restricted comparison category (for example, among models using 8 frames) and verify that claim.
  3. [Tables III-VIII and IV.C.2] Pretraining labels for VFL-Net and DVFL-Net are inconsistent and obscure the resource comparison. Every table lists 'ImageNet-1K' under Pretrain for VFL-Net/DVFL-Net, but IV.C.2 states that the student is 'trained from the scratch' and that only the teacher initializes from ImageNet-1K. If the student indeed receives no ImageNet pretraining, the 'Pretrain' column is wrong; if it does, the text and tables need correction. The teacher's target-dataset pretraining protocol also needs to be stated in each table footnote so that the comparison is transparent.
  4. [III.B and IV.H] The claimed contribution of forward KL divergence is not supported by any ablation against the standard reverse KL or a symmetric variant. The method section emphasizes forward KL, and the distillation equation (2) is one directional choice, but the ablations vary only alpha and tau. Since conventional KD with reverse KL would likely also improve VFL-Net, the experiments do not establish that the forward direction is responsible for the gains in Table VIII. Please add a direct comparison (forward KL versus reverse KL versus both) with the same alpha and tau values.
minor comments (6)
  1. [Section I and IV.A] The text says 'three publicly available action recognition datasets' and 'three HAR benchmarks' but then lists five datasets; the wording should be corrected to five.
  2. [III.B] The notation in the distillation section contains corrupted symbols: the teacher and student distributions are 'denoted by √' and '⨿θ', and Eq. (2) is not typeset correctly; these should be fixed to standard p' and q'_theta notation.
  3. [Throughout] There are several typos and spacing issues, including 'tudies' in the introduction, 'Expending Eq. 3' in III.B, and 'F ormulation' in the III.A heading.
  4. [Table XIII] The Video-FocalNet-B row cites reference [82], which is the Video Swin Transformer paper; it should cite reference [11].
  5. [Table XI] The parameter count for CF-IIH is listed as '186.M', which should presumably be '186M'.
  6. [Section V] The conclusion states that the teacher has '175 million parameters' while Table I and Section III.B say 157M; these numbers should be aligned.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the distilled-student accuracy is measured on held-out test labels, and the teacher's in-domain pretraining is a comparison-validity caveat rather than a circular reduction.

full rationale

DVFL-Net's claimed derivation chain consists of (i) adopting Video-FocalNet Base [11] as the teacher; (ii) defining a shallower VFL-Net student (Table I); (iii) optimizing the student with L_total = alpha * KLD(p' || q'_theta) * tau^2 + (1 - alpha) * CE(y, q'_theta) (Eqs. 2-6); and (iv) reporting top-1/top-5 accuracy on the test splits of UCF50, UCF101, HMDB51, SSV2, and Kinetics-400. None of these steps defines its output in terms of its input: the KD gain (e.g., 82.7 vs. 71.6 on HMDB51) is an empirical measurement on held-out test labels, not a quantity encoded in the loss or architecture, so no 'prediction' reduces to a fit by construction. The main caveat is Section IV.C.1: 'The teacher model (i.e., Video-FocalNet Base) is first pretrained on a dataset from the datasets used in this work (i.e., UCF50, UCF101, HMDB51, SSV2, and Kinetics-400).' Because the teacher is pretrained on the target benchmark, the soft-target supervision is not an independent source of external knowledge, and the comparison with published baselines that use external pretraining is not fully matched. This is a genuine comparison-validity limitation, but it is not circular: the student's accuracy is still measured against held-out ground-truth labels, and the internal VFL-Net-versus-DVFL-Net ablation controls for the KD mechanism under a fixed protocol. The paper's own Tables XII and XIII additionally show DVFL-Net trailing Video-FocalNet-B on SSV2 (70.8 vs. 71.1) and Kinetics-400 (83.1 vs. 83.6), contradicting the Section I 'state-of-the-art across ... datasets' claim; that is a correctness/fairness issue, not a circularity issue. The only self-citation, ViT-ReT [73], appears as a SOTA baseline and is not load-bearing to the central efficiency-accuracy claim. Overall, the central derivation is self-contained; score is low (2) to reflect the in-domain teacher-pretraining caveat and the minor non-load-bearing self-citation, not because any result is equivalent to its inputs by definition.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical results rely on two hand-chosen hyperparameters (alpha=0.3, tau=10), a hand-designed student depth schedule, and a set of domain assumptions about the teacher and the comparability of benchmark numbers. No new physical or mathematical entities are introduced.

free parameters (4)
  • KD loss weight alpha = 0.3
    Selected by grid search over {0.3, 0.5, 0.7} and reported as optimal on the same datasets (Section IV.H, Figure 7); the reported numbers use this hand-chosen value.
  • KD temperature tau = 10
    Selected by grid search over {5, 10, 15} and reported as optimal across datasets (Section IV.H, Figure 8); affects soft-target smoothness.
  • Student depth configuration = [1,1,2,1], embedding dim 96
    Hand-designed nano configuration 'investigated' by the authors (Table I); no principled search or cost function given.
  • Number of input frames = 8
    Set to 8 following prior work; the paper claims efficiency partly due to fewer frames, but the frame count is a free design choice (Section IV.C).
assumptions (4)
  • domain assumption Video-FocalNet's spatio-temporal focal modulation is an effective building block for both teacher and student.
    The method inherits the entire architecture from [11] without re-validating the focal modulation mechanism; performance depends on this prior model being strong.
  • domain assumption The teacher, fine-tuned on each target dataset, provides soft labels that transfer useful knowledge to the student beyond the ground-truth labels.
    Section III.B relies on response-based distillation from this in-domain teacher; if the teacher's soft labels are not informative, the KD gains vanish.
  • domain assumption Standard training recipe (SGD, cosine schedule, 120 epochs, batch size 8) converges for all datasets.
    Adopted from [11] and [44]; no convergence analysis is provided for the nano architecture.
  • domain assumption The reported top-1 accuracies are comparable across methods even though pretraining, frame counts, and evaluation views differ.
    SOTA tables compare numbers from papers with heterogeneous setups, which the manuscript does not control or discuss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition." pith.science (2026). https://pith.science/paper/HUD6CCV2

@misc{pith2026250712426,
  author       = {Pith},
  title        = {Pith review of: DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HUD6CCV2}},
  note         = {Machine review of arXiv:2507.12426}
}
read the original abstract

The landscape of video recognition has evolved significantly, shifting from traditional Convolutional Neural Networks (CNNs) to Transformer-based architectures for improved accuracy. While 3D CNNs have been effective at capturing spatiotemporal dynamics, recent Transformer models leverage self-attention to model long-range spatial and temporal dependencies. Despite achieving state-of-the-art performance on major benchmarks, Transformers remain computationally expensive, particularly with dense video data. To address this, we propose a lightweight Video Focal Modulation Network, DVFL-Net, which distills spatiotemporal knowledge from a large pre-trained teacher into a compact nano student model, enabling efficient on-device deployment. DVFL-Net utilizes knowledge distillation and spatial-temporal feature modulation to significantly reduce computation while preserving high recognition performance. We employ forward Kullback-Leibler (KL) divergence alongside spatio-temporal focal modulation to effectively transfer both local and global context from the Video-FocalNet Base (teacher) to the proposed VFL-Net (student). We evaluate DVFL-Net on UCF50, UCF101, HMDB51, SSV2, and Kinetics-400, benchmarking it against recent state-of-the-art methods in Human Action Recognition (HAR). Additionally, we conduct a detailed ablation study analyzing the impact of forward KL divergence. The results confirm the superiority of DVFL-Net in achieving an optimal balance between performance and efficiency, demonstrating lower memory usage, reduced GFLOPs, and strong accuracy, making it a practical solution for real-time HAR applications.

Figures

Figures reproduced from arXiv: 2507.12426 by the authors.

Figure 1
Figure 1. Model’s Top-1 Accuracy vs Model’s Parameters Cost: We evaluate the performance of our DVFL-Net model against different variants of the Video-FocalNet [11] architecture for video action recognition task. The comparison is based on top-1 accuracy vs model’s parameters cost, using the UCF101 dataset. Our DVFL-Net demonstrates competitive performance compared to its Video-FocalNet counterparts. processing. CNNs have lon… view at source ↗
Figure 2
Figure 2. The figure presents the knowledge distillation framework between the teacher (Video-FocalNet Base) and student (VFL [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The architectural overview of teacher and student models. Both the teacher and student models follows a four-stage architecture, with each stage containing a patch embedding layer and Video-FocalNet blocks. The teacher model consists of 24 Video-FocalNet blocks, distributed as 2, 2, 18, and 2 blocks across stages one, two, three, and four, respectively. In contrast, the student model is more lightweight, containing … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The Spatio-Temporal Focal Modulation layer, pro￾cessing spatial and temporal information independently. It separates spatial and temporal encoding, allowing each to be modeled distinctly before being combined for enhanced spatio￾temporal representation. that the studen…
Figure 5
Figure 5. Figure 5: Visual illustration of VFLNetkd’s top-5 predictions across eight distinct action videos sampled from various classes of UCF101 dataset, including Bench Press, Golf Swing, Basketball, Mixing, Playing Guitar, Skiing, Playing Tabla, and Pullups. Each video thumbnail is ov…
Figure 6
Figure 6. Figure 6: Visual illustration of spatio-temporal focal modula￾tion of three sample videos (i.e., Mixing, Bench Press, and Basketball) from UCF101 dataset. The brighter regions in yellow and green correspond to areas of significant action, reflecting where our DVFL-Net focuses it…
Figure 8
Figure 8. Figure 8: Impact of τ on Top-1 accuracy of VFL-Net across UCF101, UCF50, and HMDB51 datasets. The ablation study highlights per￾formance variations with different τ values [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Impact of training duration on model’s performance in knowledge distillation setting using UCF101 dataset. 4) Impact of Training Durability on KD Performance: To assess the impact of training duration on model performance in a knowledge distillation setting, we trained…
Figure 10
Figure 10. Figure 10: Visual illustration of the spatial and temporal focal modulation achieved by our DVFL-Net on sample videos from [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Memory Utilization across models on UCF101 dataset. Our DVFL-Net achieves the lowest memory usage and computational cost, demonstrating efficiency improvements over Video-FocalNet variants. stance, DVFL-Net completes training on the SSV2 dataset in just 84.16 hours, w…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

95 extracted references · 72 canonical work pages

  1. [1]

    Quo vadis, action recognition? a new model and the kinetics dataset,

    J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6299–6308

  2. [2]

    Spatiotemporal residual networks for video action recognition,

    R. Christoph and F. A. Pinz, “Spatiotemporal residual networks for video action recognition,” Advances in neural information processing systems, vol. 2, pp. 3468–3476, 2016

  3. [3]

    Learning spatiotemporal features with 3d convolutional networks,

    D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497

  4. [4]

    Large-scale video classification with convolutional neural networks,

    A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, “Large-scale video classification with convolutional neural networks,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, pp. 1725–1732

  5. [5]

    Beyond short snippets: Deep networks for video classification,

    J. Yue-Hei Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici, “Beyond short snippets: Deep networks for video classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 4694–4702

  6. [6]

    Two-stream convolutional networks for action recognition in videos,

    K. Simonyan and A. Zisserman, “Two-stream convolutional networks for action recognition in videos,” Advances in neural information processing systems, vol. 27, 2014

  7. [7]

    Action recognition using deep 3d cnns with sequential feature aggregation and attention,

    F. Anvarov, D. Kim, and B. Song, “Action recognition using deep 3d cnns with sequential feature aggregation and attention,” Electronics, vol. 9, p. 147, 2020

  8. [8]

    A closer look at spatiotemporal convolutions for action recognition,

    D. Tran, H. Wang, L. Torresani, J. Ray, Y . LeCun, and M. Paluri, “A closer look at spatiotemporal convolutions for action recognition,” 2018

Show all 95 references
  1. [9]

    Human action recognition in videos using convolution long short-term memory network with spatio-temporal networks,

    A. Sarabu and A. K. Santra, “Human action recognition in videos using convolution long short-term memory network with spatio-temporal networks,” Emerging Science Journal, vol. 5, pp. 25–33, 2021

  2. [10]

    Action recognition in videos using pre-trained 2d convolutional neural networks,

    J. Kim and C. S. Won, “Action recognition in videos using pre-trained 2d convolutional neural networks,” IEEE Access, vol. 8, pp. 60 179–60 188, 2020

  3. [11]

    Video-focalnets: Spatio-temporal focal modulation for video action recognition,

    S. T. Wasim, M. U. Khattak, M. Naseer, S. Khan, M. Shah, and F. S. Khan, “Video-focalnets: Spatio-temporal focal modulation for video action recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 13 778–13 789

  4. [12]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS, 2017

  5. [13]

    Morph: flexible acceleration for 3d cnn-based video understanding,

    K. Hegde, R. Agrawal, Y . Yao, and C. W. Fletcher, “Morph: flexible acceleration for 3d cnn-based video understanding,” 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), vol. 26, pp. 933–946, 2018

  6. [14]

    Video swin transformer,

    Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” 2021

  7. [15]

    Is space-time attention all you need for video understanding?

    G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding?” in ICML, 2021

  8. [16]

    Vivit: A video vision transformer,

    A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lu ˇci´c, and C. Schmid, “Vivit: A video vision transformer,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 6836–6846

  9. [17]

    Video swin transformer,

    Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” in CVPR, 2022. 15

  10. [18]

    Multiview transformers for video recognition,

    S. Yan, X. Xiong, A. Arnab, Z. Lu, M. Zhang, C. Sun, and C. Schmid, “Multiview transformers for video recognition,” in CVPR, 2022

  11. [19]

    Vivit: a video vision transformer,

    A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lu ˇci´c, and C. Schmid, “Vivit: a video vision transformer,” 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021

  12. [21]

    A short note on the kinetics-700 human action dataset,

    J. Carreira, E. Noland, C. Hillier, and A. Zisserman, “A short note on the kinetics-700 human action dataset,” in arXiv preprint arXiv:1907.06987, 2019

  13. [22]

    The" something something

    R. Goyal, S. Ebrahimi Kahou, V . Michalski, J. Materzynska, S. West- phal, H. Kim, V . Haenel, I. Fruend, P. Yianilos, M. Mueller-Freitaget al., “The" something something" video database for learning and evaluating visual common sense,” in ICCV, 2017

  14. [23]

    Tsnet: token sparsification for efficient video transformer,

    H. Wang, W. Zhang, and G. Liu, “Tsnet: token sparsification for efficient video transformer,” Applied Sciences, vol. 13, p. 10633, 2023

  15. [24]

    Dualformer: local- global stratified transformer for efficient video recognition,

    Y . Liang, P. Zhou, R. Zimmermann, and S. Yan, “Dualformer: local- global stratified transformer for efficient video recognition,” 2021

  16. [25]

    Aerobics action recognition algorithm based on three-dimensional convolutional neural network and multilabel clas- sification,

    Q. Wang and M. Wang, “Aerobics action recognition algorithm based on three-dimensional convolutional neural network and multilabel clas- sification,” Scientific Programming, vol. 2021, pp. 1–8, 2021

  17. [26]

    Tsm: temporal shift module for efficient video understanding,

    J. Lin, C. Gan, and S. Han, “Tsm: temporal shift module for efficient video understanding,” 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019

  18. [27]

    Multi-stream interaction networks for human action recognition,

    H. Wang, B. Yu, J. Li, L. Zhang, and D. Chen, “Multi-stream interaction networks for human action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 5, pp. 3050–3060, 2021

  19. [28]

    Spatio- temporal adaptive network with bidirectional temporal difference for action recognition,

    Z. Li, J. Li, Y . Ma, R. Wang, Z. Shi, Y . Ding, and X. Liu, “Spatio- temporal adaptive network with bidirectional temporal difference for action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 9, pp. 5174–5185, 2023

  20. [29]

    Agpn: Action granularity pyramid network for video action recognition,

    Y . Chen, H. Ge, Y . Liu, X. Cai, and L. Sun, “Agpn: Action granularity pyramid network for video action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 8, pp. 3912– 3923, 2023

  21. [30]

    Mawkdn: A multimodal fusion wavelet knowledge distillation approach based on cross-view attention for action recognition,

    Z. Quan, Q. Chen, M. Zhang, W. Hu, Q. Zhao, J. Hou, Y . Li, and Z. Liu, “Mawkdn: A multimodal fusion wavelet knowledge distillation approach based on cross-view attention for action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 10, ...

  22. [31]

    Convolutional neural networks or vision transformers: who will win the race for action recognitions in visual data?

    O. Moutik, H. Sekkat, S. Tigani, A. Chehri, R. Saadane, T. A. Tchakoucht, and A. Paul, “Convolutional neural networks or vision transformers: who will win the race for action recognitions in visual data?” Sensors, vol. 23, p. 734, 2023

  23. [32]

    Decoupled knowledge distillation,

    B. Zhao, Q. Cui, R. Song, Y . Qiu, and J. Liang, “Decoupled knowledge distillation,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

  24. [33]

    Knowledge distil- lation in video-based human action recognition: an intuitive approach to efficient and flexible model training,

    F. Camarena, M. Gonzalez-Mendoza, and L. Chang, “Knowledge distil- lation in video-based human action recognition: an intuitive approach to efficient and flexible model training,” Journal of Imaging, vol. 10, p. 85, 2024

  25. [34]

    Tomato leaf disease recognition based on multi-task distillation learning,

    B. Liu, S. Wei, F. Zhang, N. Guo, H. Fan, and W. Yao, “Tomato leaf disease recognition based on multi-task distillation learning,” Frontiers in Plant Science, vol. 14, 2024

  26. [35]

    Videoadviser: video knowledge distillation for multimodal transfer learning,

    Y . Wang, D. Zeng, S. Wada, and S. Kurihara, “Videoadviser: video knowledge distillation for multimodal transfer learning,” IEEE Access, vol. 11, pp. 51 229–51 240, 2023

  27. [36]

    Generative model- based feature knowledge distillation for action recognition,

    G. Wang, P. Zhao, Y . Shi, C. Zhao, and S. Yang, “Generative model- based feature knowledge distillation for action recognition,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 15 474– 15 482, 2024

  28. [37]

    Distillation of human-object interaction contexts for action recognition,

    M. Almushyti and F. W. Li, “Distillation of human-object interaction contexts for action recognition,” 2021

  29. [38]

    Gaussian error linear units (gelus),

    D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415, 2016

  30. [39]

    Recognizing 50 human action categories of web videos,

    K. K. Reddy and M. Shah, “Recognizing 50 human action categories of web videos,” Machine vision and applications, vol. 24, no. 5, pp. 971–981, 2013

  31. [40]

    Ucf101: A dataset of 101 human actions classes from videos in the wild,

    K. Soomro, “Ucf101: A dataset of 101 human actions classes from videos in the wild,” arXiv preprint arXiv:1212.0402, 2012

  32. [41]

    Hmdb: a large video database for human motion recognition,

    H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “Hmdb: a large video database for human motion recognition,” in 2011 International conference on computer vision. IEEE, 2011, pp. 2556– 2563

  33. [42]

    The" something something

    R. Goyal, S. Ebrahimi Kahou, V . Michalski, J. Materzynska, S. West- phal, H. Kim, V . Haenel, I. Fruend, P. Yianilos, M. Mueller-Freitaget al., “The" something something" video database for learning and evaluat- ing visual common sense,” in Proceedings of the IEEE internation...

  34. [43]

    The kinetics human action video dataset,

    W. Kay, J. Carreira, K. Simonyan, B. Zhang, C. Hillier, S. Vijaya- narasimhan, F. Viola, T. Green, T. Back, P. Natsev et al., “The kinetics human action video dataset,” arXiv preprint arXiv:1705.06950, 2017

  35. [44]

    Uniformer: Unified transformer for efficient spatiotemporal representation learning,

    K. Li, Y . Wang, P. Gao, G. Song, Y . Liu, H. Li, and Y . Qiao, “Uniformer: Unified transformer for efficient spatiotemporal representation learning,” arXiv preprint arXiv:2201.04676, 2022

  36. [45]

    Going deeper with convolutions,

    C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V . Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in CVPR, 2015

  37. [46]

    Making sense of neuromorphic event data for human action recognition,

    S. Al-Obaidi, H. Al-Khafaji, and C. Abhayaratne, “Making sense of neuromorphic event data for human action recognition,” IEEE Access, vol. 9, pp. 82 686–82 700, 2021

  38. [47]

    Human action recognition using dis- tance transform and entropy based features,

    P. Ramya and R. Rajeswari, “Human action recognition using dis- tance transform and entropy based features,” Multimedia Tools and Applications, vol. 80, no. 6, pp. 8147–8173, 2021

  39. [48]

    Human action recognition using hybrid deep evolving neural networks,

    P. Dasari, L. Zhang, Y . Yu, H. Huang, and R. Gao, “Human action recognition using hybrid deep evolving neural networks,” in 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022, pp. 1–8

  40. [49]

    Simple-action-guided dictionary learning for complex action recognition,

    F. Liu, X. Xu, X. Xing, K. Guo, and L. Wang, “Simple-action-guided dictionary learning for complex action recognition,” Neurocomputing, vol. 501, pp. 387–396, 2022

  41. [50]

    Human activity classification using the 3dcnn architecture,

    R. Vrskova, R. Hudec, P. Kamencay, and P. Sykora, “Human activity classification using the 3dcnn architecture,” Applied Sciences, vol. 12, no. 2, p. 931, 2022

  42. [51]

    Fast classification and action recognition with event-based imaging,

    C. Liu, X. Qi, E. Y . Lam, and N. Wong, “Fast classification and action recognition with event-based imaging,”IEEE access, vol. 10, pp. 55 638– 55 649, 2022

  43. [52]

    Spatio-temporal features based human action recognition using convolutional long short-term deep neural network,

    A. S. Saif, E. D. Wollega, and S. A. Kalevela, “Spatio-temporal features based human action recognition using convolutional long short-term deep neural network,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 5, 2023

  44. [53]

    Human action recognition using multi-stream fusion and hybrid deep neural networks,

    S. Chopra, L. Zhang, and M. Jiang, “Human action recognition using multi-stream fusion and hybrid deep neural networks,” in 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2023, pp. 4852–4858

  45. [54]

    Self-supervised video representation learning by uncovering spatio-temporal statistics,

    J. Wang, J. Jiao, L. Bao, S. He, W. Liu, and Y .-H. Liu, “Self-supervised video representation learning by uncovering spatio-temporal statistics,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3791–3806, 2021

  46. [55]

    Enhancing self-supervised video representation learning via multi-level feature optimization,

    R. Qian, Y . Li, H. Liu, J. See, S. Ding, X. Liu, D. Li, and W. Lin, “Enhancing self-supervised video representation learning via multi-level feature optimization,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 7990–8001

  47. [56]

    Videomoco: Contrastive video representation learning with temporally adversarial examples,

    T. Pan, Y . Song, T. Yang, W. Jiang, and W. Liu, “Videomoco: Contrastive video representation learning with temporally adversarial examples,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 11 205–11 214

  48. [57]

    Action recognition from a single coded image,

    S. Kumawat, T. Okawara, M. Yoshida, H. Nagahara, and Y . Yagi, “Action recognition from a single coded image,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4109–4121, 2022

  49. [58]

    Tclr: Temporal contrastive learning for video representation,

    I. Dave, R. Gupta, M. N. Rizve, and M. Shah, “Tclr: Temporal contrastive learning for video representation,” Computer Vision and Image Understanding, vol. 219, p. 103406, 2022

  50. [59]

    Learn2augment: learning to composite videos for data augmentation in action recognition,

    S. N. Gowda, M. Rohrbach, F. Keller, and L. Sevilla-Lara, “Learn2augment: learning to composite videos for data augmentation in action recognition,” in European conference on computer vision. Springer, 2022, pp. 242–259

  51. [60]

    Learning from temporal gradient for semi-supervised action recognition,

    J. Xiao, L. Jing, L. Zhang, J. He, Q. She, Z. Zhou, A. Yuille, and Y . Li, “Learning from temporal gradient for semi-supervised action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3252–3262

  52. [61]

    Preserve pre- trained knowledge: Transfer learning with self-distillation for action recognition,

    Y . Zhou, Z. He, K. Lu, G. Wang, and G. Wang, “Preserve pre- trained knowledge: Transfer learning with self-distillation for action recognition,” arXiv preprint arXiv:2205.00506, 2022

  53. [62]

    Extreme low- resolution action recognition with confident spatial-temporal attention transfer,

    Y . Bai, Q. Zou, X. Chen, L. Li, Z. Ding, and L. Chen, “Extreme low- resolution action recognition with confident spatial-temporal attention transfer,” International Journal of Computer Vision, vol. 131, no. 6, pp. 1550–1565, 2023

  54. [63]

    Self-supervised video-based action recognition with disturbances,

    W. Lin, X. Ding, Y . Huang, and H. Zeng, “Self-supervised video-based action recognition with disturbances,” IEEE Transactions on Image Processing, vol. 32, pp. 2493–2507, 2023. 16

  55. [64]

    Spatial-temporal exclusive capsule network for open set action recognition,

    Y . Feng, J. Gao, S. Yang, and C. Xu, “Spatial-temporal exclusive capsule network for open set action recognition,” IEEE Transactions on Multimedia, vol. 25, pp. 9464–9478, 2023

  56. [65]

    Sv- former: Semi-supervised video transformer for action recognition,

    Z. Xing, Q. Dai, H. Hu, J. Chen, Z. Wu, and Y .-G. Jiang, “Sv- former: Semi-supervised video transformer for action recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 18 816–18 826

  57. [66]

    Actionhub: a large-scale action video description dataset for zero-shot action recog- nition,

    J. Zhou, J. Liang, K.-Y . Lin, J. Yang, and W.-S. Zheng, “Actionhub: a large-scale action video description dataset for zero-shot action recog- nition,” arXiv preprint arXiv:2401.11654, 2024

  58. [67]

    Self-supervised learning via multi-transformation classification for action recognition,

    D.-Q. Vu, N. Le, and J.-C. Wang, “Self-supervised learning via multi-transformation classification for action recognition,” in 2024 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). IEEE, 2024, pp. 1–6

  59. [68]

    Semi-supervised action recog- nition with dynamic temporal information fusion,

    H. Qian, J. Zhang, Z. Shi, and Y . Zhang, “Semi-supervised action recog- nition with dynamic temporal information fusion,” Neurocomputing, p. 128683, 2024

  60. [69]

    Spatiotemporal contrastive video representation learning,

    R. Qian, T. Meng, B. Gong, M.-H. Yang, H. Wang, S. Belongie, and Y . Cui, “Spatiotemporal contrastive video representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6964–6974

  61. [70]

    Representation learning for compressed video action recognition via attentive cross- modal interaction with motion enhancement,

    B. Li, J. Chen, D. Zhang, X. Bao, and D. Huang, “Representation learning for compressed video action recognition via attentive cross- modal interaction with motion enhancement,” in Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2022, pp. 1–8

  62. [71]

    Motion-driven visual tempo learn- ing for video-based action recognition,

    Y . Liu, J. Yuan, and Z. Tu, “Motion-driven visual tempo learn- ing for video-based action recognition,” IEEE Transactions on Image Processing, vol. 31, pp. 4104–4116, 2022

  63. [72]

    Learning spatiotemporal and motion features in a unified 2d network for action recognition,

    M. Wang, J. Xing, J. Su, J. Chen, and Y . Liu, “Learning spatiotemporal and motion features in a unified 2d network for action recognition,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 3, pp. 3347–3362, 2022

  64. [73]

    Vit-ret: Vision and recurrent transformer neural networks for human activity recognition in videos,

    J. Wensel, H. Ullah, and A. Munir, “Vit-ret: Vision and recurrent transformer neural networks for human activity recognition in videos,” IEEE Access, 2023

  65. [74]

    Spatial-temporal interleaved net- work for efficient action recognition,

    S. Jiang, H. Zhang, Y . Qi, and Q. Liu, “Spatial-temporal interleaved net- work for efficient action recognition,” IEEE Transactions on Industrial Informatics, 2024

  66. [75]

    A hybrid transformer framework for efficient activity recog- nition using consumer electronics,

    A. Hussain, S. U. Khan, N. Khan, M. W. Bhatt, A. Farouk, J. Bhola, and S. W. Baik, “A hybrid transformer framework for efficient activity recog- nition using consumer electronics,” IEEE Transactions on Consumer Electronics, 2024

  67. [76]

    A knowledge-based hierarchical causal inference network for video action recognition,

    Y . Liu, F. Liu, L. Jiao, Q. Bao, L. Li, Y . Guo, and P. Chen, “A knowledge-based hierarchical causal inference network for video action recognition,” IEEE Transactions on Multimedia, 2024

  68. [77]

    Is space-time attention all you need for video understanding?

    G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding?” in ICML, vol. 2, no. 3, 2021, p. 4

  69. [78]

    Vidtr: Video transformer without convolutions,

    Y . Zhang, X. Li, C. Liu, B. Shuai, Y . Zhu, B. Brattoli, H. Chen, I. Marsic, and J. Tighe, “Vidtr: Video transformer without convolutions,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 13 577–13 587

  70. [79]

    Keeping your eye on the ball: Tra- jectory attention in video transformers,

    M. Patrick, D. Campbell, Y . Asano, I. Misra, F. Metze, C. Feichtenhofer, A. Vedaldi, and J. F. Henriques, “Keeping your eye on the ball: Tra- jectory attention in video transformers,” Advances in neural information processing systems, vol. 34, pp. 12 493–12 506, 2021

  71. [80]

    Multiscale vision transformers,

    H. Fan, B. Xiong, K. Mangalam, Y . Li, Z. Yan, J. Malik, and C. Feichten- hofer, “Multiscale vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 6824–6835

  72. [81]

    Multiview transformers for video recognition,

    S. Yan, X. Xiong, A. Arnab, Z. Lu, M. Zhang, C. Sun, and C. Schmid, “Multiview transformers for video recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 3333–3343

  73. [82]

    Video swin transformer,

    Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 3202–3211

  74. [83]

    Mvitv2: Improved multiscale vision transformers for classification and detection,

    Y . Li, C.-Y . Wu, H. Fan, K. Mangalam, B. Xiong, J. Malik, and C. Feichtenhofer, “Mvitv2: Improved multiscale vision transformers for classification and detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4804– 4814

  75. [84]

    A novel spatio-temporal-wise network for action recognition,

    Z. Cai, “A novel spatio-temporal-wise network for action recognition,” IEEE Access, vol. 11, pp. 49 071–49 080, 2023

  76. [85]

    D-tsm: Discriminative temporal shift module for action recognition,

    S. Lee and S. Hong, “D-tsm: Discriminative temporal shift module for action recognition,” in 2023 20th international conference on ubiquitous robots (UR). IEEE, 2023, pp. 133–136

  77. [86]

    Scene adaptive mechanism for action recognition,

    C. Wu, X.-J. Wu, T. Xu, and J. Kittler, “Scene adaptive mechanism for action recognition,” Computer Vision and Image Understanding, vol. 238, p. 103854, 2024

  78. [87]

    Sta+: Spatiotemporal adaptation with adaptive model selection for video action recognition,

    M. Li, C. Zhang, and X. Zheng, “Sta+: Spatiotemporal adaptation with adaptive model selection for video action recognition,” in 2024 IEEE 4th International Conference on Digital Twins and Parallel Intelligence (DTPI). IEEE, 2024, pp. 177–182

  79. [88]

    Short-term action learning for video action recognition,

    L. Ting-Long, “Short-term action learning for video action recognition,” IEEE Access, vol. 12, pp. 30 867–30 875, 2024

  80. [89]

    Tea: Temporal excitation and aggregation for action recognition,

    Y . Li, B. Ji, X. Shi, J. Zhang, B. Kang, and L. Wang, “Tea: Temporal excitation and aggregation for action recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 909–918

  81. [90]

    Movinets: Mobile video networks for efficient video recogni- tion,

    D. Kondratyuk, L. Yuan, Y . Li, L. Zhang, M. Tan, M. Brown, and B. Gong, “Movinets: Mobile video networks for efficient video recogni- tion,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16 020–16 030

  82. [91]

    Timebal- ance: Temporally-invariant and temporally-distinctive video represen- tations for semi-supervised action recognition,

    I. R. Dave, M. N. Rizve, C. Chen, and M. Shah, “Timebal- ance: Temporally-invariant and temporally-distinctive video represen- tations for semi-supervised action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2341–2352

  83. [92]

    Dilated multi-temporal modeling for action recognition,

    T. Zhang, Y . Wu, and X. Li, “Dilated multi-temporal modeling for action recognition,” Applied Sciences, vol. 13, no. 12, p. 6934, 2023

  84. [93]

    Learning discriminative spatio- temporal representations for semi-supervised action recognition,

    Y . Wang, S. Zhou, K. Xia, and L. Wang, “Learning discriminative spatio- temporal representations for semi-supervised action recognition,” arXiv preprint arXiv:2404.16416, 2024

  85. [94]

    Discrimina- tive segment focus network for fine-grained video action recognition,

    B. Sun, X. Ye, T. Yan, Z. Wang, H. Li, and Z. Wang, “Discrimina- tive segment focus network for fine-grained video action recognition,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 20, no. 7, pp. 1–20, 2024

  86. [95]

    Temporal difference attention for action recog- nition,

    H. Zhang and Y . Xia, “Temporal difference attention for action recog- nition,” in 2024 4th International Conference on Neural Networks, Information and Communication (NNICE). IEEE, 2024, pp. 648–652

  87. [96]

    An efficient motion visual learning method for video action recognition,

    B. Wang, F. Chang, C. Liu, W. Wang, and R. Ma, “An efficient motion visual learning method for video action recognition,” Expert Systems with Applications, vol. 255, p. 124596, 2024. Hayat Ullah received his Bachelor’s degree in Computer Science from Islamia College University...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.