Pith. sign in

REVIEW 3 major objections 5 minor 28 references

VideoSEMA claims that a cheap split space-time attention — a Mamba-like spatial block plus softmax temporal attention — matches or beats much heavier video transformers and Mamba models, and proves that split space-time attention can exactl

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

VideoSEMA uses SEMA spatial attention plus softmax temporal attention to outperform larger video transformers and Mamba models on K400/SSv2 and degrade less at 1024² resolution.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection Promising efficient video block with a theory that misses the implemented architecture. the 3 major comments →

arxiv 2607.14711 v2 pith:MWZITRRC submitted 2026-07-16 cs.CV cs.AI

VideoSEMA: a scalable and efficient Mamba-like attention for video understanding

Nhat Thanh Tran , Fanghui Xue , Shuai Zhang , Jiancheng Lyu , Yunling Zheng , Yingyong Qi , Jack Xin This is my paper
classification cs.CV cs.AI
keywords video understandingaction recognitionMamba-like attentionsplit space-time attentionSEMAKinetics-400Something-Something v2high-resolution scalability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VideoSEMA aims to show that video action recognition does not need a heavy full space-time transformer. It splits attention into a spatial pass using SEMA — a windowed softmax attention running in parallel with a global average, a Mamba-like mechanism that is linear in spatial resolution — and a temporal pass using ordinary softmax attention over the frame dimension. The paper reports that on Kinetics-400 and Something-Something v2, this 31M-parameter model beats larger Vision Transformer and Mamba baselines on top-1 accuracy, and that when evaluated at 1024² resolution without fine-tuning, it loses far less accuracy than VideoMamba. The accompanying theory shows that split space-time attention is exactly equivalent to full space-time attention whenever the temporal and spatial score matrices have rank at most the head dimensions, with a low-rank approximation bound otherwise.

Core claim

Split space-time attention can be made cheap in space and exact in principle. For each query, the full space-time attention weight η_ij decomposes uniquely into a temporal marginal α_j = Σ_i η_ij and a spatial conditional β_{i|j} = η_ij / α_j, so the split output equals the full output provided the temporal and spatial branches can realize those two distributions. Theorem 4.1 proves they can whenever the rank of the target temporal score matrix is at most dT and the rank of the spatial score matrix is at most dS, because low-rank query-key factorizations then reproduce the scores. When those rank conditions are violated, Theorem 4.3 bounds the split-vs-full gap by the L1 approximation errors

What carries the argument

The SEMA block is the spatial engine: a softmax attention restricted to a local window of tokens, computed in parallel with a global average of all value tokens, the average term standing in for Mamba's exponential-forgetting global recurrence. The temporal engine is a full softmax attention over T frames, applied at each spatial location. The glue is Theorem 4.1's marginal-conditional factorization: any full attention coefficient matrix can be written as α_j β_{i|j}, and if the induced score matrices Rφ (M×T) and Sφ (M×NT) have rank at most dT and dS, dot-product query/key matrices realize the split exactly. This is what turns the cheaper split computation into a provably faithful surrogate

Load-bearing premise

For the theoretical claim, the load-bearing premise is that the temporal and spatial score matrices have rank at most the head dimensions — a condition the paper itself concedes is 'quite strict' for long sequences (Remark 4.5), so in practice the low-rank approximation regime of Theorem 4.3 is the more likely mode. For the empirical claim, the load-bearing premise is that the comparison baselines were trained under fairly comparable budgets and pretraining; the paper states

What would settle it

Train or load a VideoSEMA model and extract the implied temporal scores (M×T) and spatial scores (M×NT) — or approximate them from the attention distributions. If the effective rank of the spatial score matrix exceeds the spatial head dimension dS on real inputs, the exact-equivalence theorem does not explain the model's accuracy. Likewise, re-training the architecture with temporal softmax replaced by a bidirectional Mamba scan on K400 would test whether the reported 5-point ablation gap is the source of the main results.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • At the reported settings, VideoSEMA (31M params, 87G FLOPs) reaches 82.4 top-1 on K400 at 16×224², above MViTv2-S (35M, 64G, 81.0) and VideoMamba-S (26M, 68G, 80.8) with similar or lower compute.
  • Because split attention is an exact marginal-conditional decomposition of full attention under the rank condition, the split architecture does not inherently sacrifice modeling power, only the ability to realize very high-rank score matrices.
  • Without fine-tuning, VideoSEMA holds up much better than VideoMamba when frames are enlarged from 224² to 1024² (54.6 vs 40.8), suggesting the global-averaging branch preserves context that a sequential scan loses.
  • The ablation showing temporal softmax attention (81.3) beating a temporal Mamba scan (76.3) on K400 indicates the sequential order of Mamba is a poor fit for short-clip temporal reasoning, so full temporal attention remains the right choice for T ≤ 32.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The rank-condition theorem suggests a design recipe beyond this paper: if a full-attention video model's score matrices have small effective rank, one can replace it with a split attention whose head dimensions match the rank and recover the same behavior at far lower cost; measuring those ranks on existing models would test how often this applies.
  • The high-resolution robustness at 1024² without fine-tuning likely extends to other tasks that increase input size at inference, such as video detection or segmentation on large frames, where preserving global context matters.
  • For videos longer than the 32 frames tested here, replacing temporal softmax with dilated or sparse attention is the paper's stated plan; the equivalence theorem suggests the same rank argument would transfer to sparse temporal score matrices, giving a principled bound on the approximation.
  • A direct test of the temporal-attention claim would be to freeze the spatial branch and swap the temporal softmax for a bidirectional scan on a longer-video dataset; if accuracy drops as it does in the ablation, the temporal softmax is the active ingredient.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes VideoSEMA, a video classification architecture that combines SEMA spatial attention (local window attention plus global average) with softmax temporal attention in a split space-time design. The claimed contributions are empirical: better accuracy than heavier ViT/Mamba models on K400 and SSv2, and a robustness advantage over VideoMamba when input resolution is increased from 224^2 to 1024^2 without fine-tuning. The paper also makes a theoretical claim: under rank conditions, split space-time attention is exactly equivalent to full space-time attention (Theorem 4.1), with a low-rank approximation bound (Theorem 4.3). The manuscript includes complexity analysis, experiments on two benchmarks, an ablation of the temporal component, and qualitative examples.

Significance. If the empirical claims hold, VideoSEMA is an attractive efficient backbone for video understanding, particularly for high-resolution inputs: the complexity is linear in spatial resolution, and the reported 13.8-point advantage over VideoMamba at 1024^2 without fine-tuning is a concrete, falsifiable result. The paper also gives a clean exposition of SEMA and a low-rank error bound that is useful in principle. However, the central theoretical claim is not established for the implemented architecture: Theorem 4.1 proves a realizability result for an idealized factorized attention that is more general than the VideoSEMA block. The paper's value therefore currently rests on the experiments, which would need stronger controlled comparisons to fully support the headline 'outperforms' claims.

major comments (3)
  1. [§4, Theorem 4.1 vs. Eqs. (8)–(10)] Theorem 4.1 does not apply to the implemented VideoSEMA block. The theorem's target spatial score matrix Sφ ∈ R^{M×NT} (Eq. 23) has a row for every original query m and a column for every (key frame, spatial key) pair, meaning each query m is assumed to compute a spatial attention distribution inside every temporal frame j. The implemented block does not do this: Eq. (9) restricts spatial attention for query (t,h,w) to keys in the same frame t, and Eq. (10) then mixes the spatial outputs at the same spatial location across time. Consequently, for a query (t,h,w) and a key in a different frame i≠t, the spatial coefficient is the one computed by the query at (i,h,w), not by (t,h,w). The conditional distributions β_{m,:|j} defined in Eq. (19) generally vary with the query's temporal index m_t; the implemented spatial branch cannot see this variation for j≠m_t. Even when the rank conditions
  2. [§4.2, Eqs. (19)–(21); abstract] The equality in Eq. (21) is a marginal-conditional identity: α and β are defined from the full attention weights η, so the split output equals the full output by construction. The substantive content is the rank-condition realizability, and that content is disconnected from the actual architecture, as detailed above. Remark 4.5 already concedes that the rank condition is strict in the long-sequence regime, but the abstract presents the equivalence without this caveat. The authors should either prove a realizability result for the actual topology (e.g., with β_{m,i|j} replaced by β_{m_j,i|j} where m_j=(j,h,w) is the query actually used by the spatial branch for frame j) or clearly reposition Theorem 4.1 as an analysis of an idealized split attention and temper the abstract's claim accordingly.
  3. [§5, Tables 1–2] The headline empirical comparisons are based on numbers reported in prior papers (e.g., VideoMamba-M/S from [13], TimeSformer, MViTv2) rather than a common training harness. The text claims 'similar training methodologies and computational budgets' (Sec. 5.3.1) and 'fair comparison' (Sec. 5.2), but it does not specify the training recipe, augmentation, epoch counts, or inference protocol of the baselines. Since the central positive claim is that VideoSEMA outperforms heavier models, this leaves room for training-recipe confounds. Please either retrain the main baselines under the same protocol or provide a detailed table of hyperparameters and explicitly state that differences may reflect training recipes. At minimum, report seed variance or multiple runs.
minor comments (5)
  1. [§5.3.1] The sentence 'This demonstrates that VideoSEMA is a' is incomplete; finish or delete.
  2. [§3.2] The term 'Mamba-like' is used without a precise definition of the retained macro-architecture. Equation (2) defines SEMA as window attention plus a broadcast average, which is not by itself a Mamba macro-architecture. Clarify what structure of Mamba is preserved.
  3. [Abstract and throughout] There are several typos and inconsistent names: 'Mam ba' in the abstract, 'Sema' in Table 1 caption, 'Y unling' in the author block, and 'V -JEP A 2' in the reference list. Run a spellcheck and unify the notation (VideoSEMA vs. VideoSema).
  4. [Fig. 2] The scheme label 'T+S' and the caption 'SEMA Split Space-Time Attention (T+window+average)' are inconsistent with the order in Eqs. (8)–(10), where spatial SEMA is applied before temporal attention. Use consistent notation, e.g., 'SEMA + T'.
  5. [§5.4, Table 3] The Mamba temporal baseline is reported at 76.3, much lower than the other options. Without details on kernel size, directionality, and tuning budget, it is hard to judge whether this is an under-tuned comparator. Report these details or soften the conclusion that attention is outright best.

Circularity Check

1 steps flagged

Split/full equivalence theorem reduces to a marginal-conditional identity; empirical benchmarks are independent.

specific steps
  1. self definitional [Section 4.2, Eqs. (19)-(21) and Theorem 4.1]
    "define αmj = ∑ N l=1 ηmlj, β mi|j = ηmij/αmj. (19) ... α mjβmi|j = ηmij. (20) Thus the split output equals the full output for every query: A(m) ST = ... = A(m) T +S. (21) Therefore split space time attention is an exact marginal-conditional decomposition of each row of the full attention matrix."

    The target temporal and spatial score matrices are constructed directly from the full-attention matrix η that the theorem claims to reproduce: α is the row marginal and β is the conditional, so Eq. (21) is an identity, not a property derived from the split architecture. Theorem 4.1 then only adds a rank condition ensuring dot-product realization of these constructed scores. In addition, the constructed S^φ_{m,(j,i)} requires spatial scores between query m and every key frame j, whereas the implemented block in Eq. (9) restricts spatial attention to the query's own frame, so the theorem's equivalence is for a different factorized operator than VideoSEMA.

full rationale

The empirical claims are self-contained: VideoSEMA is trained and evaluated on K400 and SSv2, and the reported comparisons to VideoMamba and other models are standard benchmark numbers, not quantities fitted from VideoSEMA's own outputs. No fitted parameter is relabeled as a prediction. The theoretical Section 4 is where a reduction-by-construction occurs: α and β are defined as marginals/conditionals of η, making Eq. (21) an identity; the rank-condition theorem adds realization content but also silently assumes spatial scores from every query to every key frame, which the actual spatial-first split in Eqs. (8)-(10) never computes. The SEMA approximation is inherited from the authors' prior work [24], so it is a self-citation, but the video-level results do not rest on a self-citation chain. Overall: partial circularity in the stated theoretical equivalence, while the empirical core remains independent; hence score 4.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The empirical model has no fitted constants beyond standard training hyperparameters. The theoretical equivalence result, however, depends on rank assumptions that are stated but unverified, and the SEMA approximation is taken from a self-cited prior paper. No new physical or conceptual entities are introduced.

free parameters (2)
  • window size w (spatial window attention) = 7 (example)
    Spatial SEMA in Eq. (9) is parameterized by a window index set I(h,w); the paper gives 'e.g. window size of 7' and inherits the choice from SEMA/Swin. Accuracy and complexity depend on it.
  • temporal/spatial head dims d_T, d_S = not specified
    Theorem 4.1 requires dT ≥ rank(Rφ) and dS ≥ rank(Sφ). No evidence is given that the trained heads satisfy these inequalities; Remark 4.5 concedes exact rank conditions are 'quite strict'.
axioms (4)
  • domain assumption The SEMA global-averaging term in Eq. (2) approximates full softmax attention as n→∞ (from [24])
    The spatial branch's global information relies on this self-cited approximation guarantee; the paper does not re-derive it.
  • domain assumption The learned dot-product score matrices can realize the target temporal/spatial score matrices Rφ and Sφ whenever rank conditions hold
    Theorem 4.1 assumes factorization of Rφ and Sφ and then that the network's Q,K projections can represent those factors (Eq. 31); the col(Qτ)⊂col(y) condition is not guaranteed.
  • ad hoc to paper Rank inequalities dT ≥ rank(Rφ) and dS ≥ rank(Sφ) hold (or approximately hold) for the actual trained attention coefficients
    The exact equivalence in Theorem 4.1 depends on these inequalities; Remark 4.5 states they are unlikely for long sequences, so the practical claim rests on the unverified low-rank approximation.
  • standard math φ is invertible on its positive range (e.g., exp for softmax)
    Used to invert normalized coefficients into score matrices in Theorem 4.1 and Remark 4.2.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of VideoSEMA: a scalable and efficient Mamba-like attention for video understanding." pith.science (2026). https://pith.science/paper/MWZITRRC

@misc{pith2026260714711,
  author       = {Pith},
  title        = {Pith review of: VideoSEMA: a scalable and efficient Mamba-like attention for video understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MWZITRRC}},
  note         = {Machine review of arXiv:2607.14711}
}
Share X Bluesky LinkedIn Reddit HN
abstract

We present for video understanding (classification) a split space-time attention model, VideoSEMA, consisting of a scalable and efficient Mamba-like attention (SEMA) block in space and a softmax temporal attention in time. In each frame, SEMA attention applies a local window attention in parallel with a global averaging in a Mamba macro-architecture, which is called Mamba-like. Under certain rank conditions, we prove that the computationally cheaper split space-time attention is equivalent to full space-time attention. On benchmark K400 data sets, VideoSEMA out-performs heavier vision transformer and Mamba models. On benchmark SSv2 data, VideoSEMA leads in top-1 accuracy among models of similar parameter sizes. As image resolution scales up from standard $224^2$ to $1024^2$ on K400 and without fine-tuning, VideoSEMA degrades much more gracefully than VideoMamba in accuracy. It is promising to extend VideoSEMA to longer videos with a dilated/sparse temporal attention.

Figures

Figures reproduced from arXiv: 2607.14711 by Fanghui Xue, Jack Xin, Jiancheng Lyu, Nhat Thanh Tran, Shuai Zhang, Yingyong Qi, Yunling Zheng.

Figure 1
Figure 1. Figure 1: Overview of VideoSEMA macro-structure. still performed globally, thus the total computational complexity of VideoSEMA is O((T 2HW + T HW w)C), linear in spatial resolution HW. In the datasets of this paper, the number of video frames is at most 32 (i.e. T ≤ 32 in Tab. 1, and T ≤ 16 in Tab. 2 and Tab. 3). The main contribution to the complexity comes from spatial resolution HW = (224)2 . With a linear compl… view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of space time attention types. For i [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison of VideoSEMA and VideoMam [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 6 linked inside Pith

  1. [1]

    V -jepa 2: Self-supervised video models enable unde rstanding, prediction and planning

    Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba, Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinh a, Artem Zholus, Sergio Ar- naud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, Piotr Bojanowski, V asil Khalidov, Patrick Labatut, Francisco Massa, Marc Szafrani ec, Kapil Krishnakumar, Y ong L...

    Pith/arXiv arXiv 2025

  2. [2]

    Is sp ace-time attention all you need for video understanding? In Proceedings of the International Conference on Machine Lea rning (ICML), July 2021

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is sp ace-time attention all you need for video understanding? In Proceedings of the International Conference on Machine Lea rning (ICML), July 2021

    2021

  3. [3]

    J. Ding, S. Ma, L. Dong, X. Zhang, S. Huang, W . Wang, N. Zhen g, and F. Wei. Longnet: Scaling transformers to 1,000,000,000 tokens. arXiv preprint arXiv:2307.02486 , 2023

    Pith/arXiv arXiv 2023

  4. [4]

    Multiscale vision transformers

    Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Y anghao Li, Zh icheng Y an, Jitendra Malik, and Christoph Feichtenhofer. Multiscale vision transformers . In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 6824–6835, October 2021

    2021

  5. [5]

    X3d: Expanding architecture s for efficient video recognition

    Christoph Feichtenhofer. X3d: Expanding architecture s for efficient video recognition. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and P attern Recognition (CVPR) , June 2020

    2020

  6. [6]

    Glowinski, S

    R. Glowinski, S. J. Osher, and W . Yin. Splitting methods i n communication, imaging, science, and engineering. Springer, 2017. 13

    2017

  7. [7]

    something some- thing

    Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski , Joanna Materzynska, Susanne Westphal, Heuna Kim, V alentin Haenel, Ingo Fruend, Peter Yi anilos, Moritz Mueller-Freitag, Florian Hoppe, Christian Thurau, Ingo Bax, and Roland Memis evic. The “something some- thing” video database for learning and evaluating visual co mmon sense. In 2017 IEEE Inter- ...

    2017

  8. [8]

    Mamba: Linear-time sequence model ing with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence model ing with selective state spaces. In First Conference on Language Modeling , 2024

    2024

  9. [9]

    Hippo: Recurrent mem- ory with optimal polynomial projections

    Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christ opher Ré. Hippo: Recurrent mem- ory with optimal polynomial projections. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 1474–1487. Curran Associates, Inc., 2020

    2020

  10. [10]

    Demystify Mamba in Visio n: A Linear Attention Perspective

    Dongchen Han, Ziyi Wang, Zhuofan Xia, Yizeng Han, Yifan Pu, Chunjiang Ge, Jun Song, Shiji Song, Bo Zheng, and Gao Huang. Demystify Mamba in Visio n: A Linear Attention Perspective. NeurIPS, 2024

    2024

  11. [11]

    The kinetics human action video dataset

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, C hloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natse v, Mustafa Suleyman, and An- drew Zisserman. The kinetics human action video dataset. arXiv:1705.06950, 2017

    Pith/arXiv arXiv 2017

  12. [12]

    CT-Net: Channel tensoriza- tion network for video classification

    Kunchang Li, Xianhang Li, Y ali Wang, Jun Wang, and Y u Qia o. CT-Net: Channel tensoriza- tion network for video classification. In International Conference on Learning Representations, 2021

    2021

  13. [13]

    Video- Mamba: State space model for efficient video understanding

    Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Y ali Wang, Lim in Wang, and Y u Qiao. Video- Mamba: State space model for efficient video understanding. arXiv:2403.06977, in ECCV , 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Uniformer: Unified transformer for efficient spatial-tempo ral representation learning

    Kunchang Li, Y ali Wang, Gao Peng, Guanglu Song, Y u Liu, H ongsheng Li, and Y u Qiao. Uniformer: Unified transformer for efficient spatial-tempo ral representation learning. In Inter- national Conference on Learning Representations , 2022

    2022

  15. [15]

    MViTv2: Improved multiscale vis ion transformers for classification and detection

    Y anghao Li, Chao-Y uan Wu, Haoqi Fan, Karttikeya Mangalam, Bo Xiong, Jitendra Malik, and Christoph Feichtenhofer. MViTv2: Improved multiscale vis ion transformers for classification and detection. In Proceedings of the IEEE/CVF Conference on Computer Vision a nd Pattern Recognition (CVPR), pages 4804–4814, June 2022

    2022

  16. [16]

    Swin transformer: Hierarchical vision transformer us ing shifted windows

    Ze Liu, Y utong Lin, Y ue Cao, Han Hu, Yixuan Wei, Zheng Zha ng, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer us ing shifted windows. In Proceed- ings of the IEEE/CVF International Conference on Computer V ision (ICCV) , pages 10012– 10022, October 2021

    2021

  17. [17]

    Video swin transformer

    Ze Liu, Jia Ning, Y ue Cao, Yixuan Wei, Zheng Zhang, Steph en Lin, and Han Hu. Video swin transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision a nd Pattern Recognition (CVPR), pages 3202–3211, June 2022

    2022

  18. [18]

    A primal-dual framework for transformers and neural networ ks

    Tan Nguyen, Tam Nguyen, Nhat Ho, Andrea Bertozzi, Richa rd Baraniuk, and Stanley Osher. A primal-dual framework for transformers and neural networ ks. in Proc. of ICLR , 2023

    2023

  19. [19]

    Henriques

    Mandela Patrick, Dylan Campbell, Y uki Asano, Ishan Mis ra, Florian Metze, Christoph Feicht- enhofer, Andrea V edaldi, and Joao F. Henriques. Keeping you r eye on the ball: Trajectory attention in video transformers. In A. Beygelzimer, Y . Daup hin, P . Liang, and J. Wortman V aughan, editors,Advances in Neural Information Processing Systems , 2021

    2021

  20. [20]

    G. Strang. On the construction and comparison of differ ence schemes. SIAM Journal on Numerical Analysis, 5(3):506–517, 1968

    1968

  21. [21]

    WLiT: Windows and lin- ear transformer for video action recognition

    Ruochen Sun, Tian Zhang, Yiming Wan, Feng Zhang, and Jia n Wei. WLiT: Windows and lin- ear transformer for video action recognition. Sensors (Basel, Switzerland) , 23(3):1616, 2023

    2023

  22. [22]

    A mathematical explanation of transformers

    Xue-Cheng Tai, Hao Liu, Lingfeng Li, and Raymond H Chan. A mathematical explanation of transformers. arXiv:2510.03989, 2025. 14

    Pith/arXiv arXiv 2025

  23. [23]

    VideoM AE: Masked autoencoders are data-efficient learners for self-supervised video pre- training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. VideoM AE: Masked autoencoders are data-efficient learners for self-supervised video pre- training. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Pro- cessing Systems, volume 35, pages 10078–10093. Curran Associates, Inc., 20 22

  24. [24]

    SEMA: a scalable and efficient mamba like attent ion via token localization and averaging

    Nhat Thanh Tran, Fanghui Xue, Shuai Zhang, Jiancheng Ly u, Y unling Zheng, Yingyong Qi, and Jack Xin. SEMA: a scalable and efficient mamba like attent ion via token localization and averaging. arXiv:2506.08297; in Proc. of ICML , 2026

    Pith/arXiv arXiv 2026

  25. [25]

    Make your training flexible: Towards deployment-efficient v ideo models

    Chenting Wang, Kunchang Li, Tianxiang Jiang, Xiangyu Z eng, Yi Wang, and Limin Wang. Make your training flexible: Towards deployment-efficient v ideo models. In Proceedings of the IEEE/CVF International Conference on Computer Vision ( ICCV), pages 23880–23891, October 2025

    2025

  26. [26]

    Tdn: Temp oral difference networks for efficient action recognition

    Limin Wang, Zhan Tong, Bin Ji, and Gangshan Wu. Tdn: Temp oral difference networks for efficient action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1895–1904, June 2021

    1904

  27. [27]

    Internvideo2: Sca ling foundation models for multimodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Y u, Yinan He, Gu o Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Y ansong Shi, Tianxiang Jiang, Songze Li, Ji lan Xu, Hongjie Zhang, Yifei Huang, Y u Qiao, Y ali Wang, and Limin Wang. Internvideo2: Sca ling foundation models for multimodal video understanding. In Aleš Leonardis, Eli sa Ricci, Stefan Roth, Olga Rus- sa...

    2024

  28. [28]

    Zheng, Z

    Y . Zheng, Z. Xu, F. Xue, B. Y ang, J. Lyu, S. Zhang, Y . Qi, an d J. Xin. AFIDAF: Alternating Fourier and Image Domain Adaptive Filters as an Efficient Alt ernative to Attention in ViTs. International Symposium of Visual Computing, Reno, NV , 15046:17–30, 2024. 15

    2024

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.