Pith. sign in

REVIEW 11 cited by

Is Space-Time Attention All You Need for Video Understanding?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2102.05095 v4 pith:ZMAETS7E submitted 2021-02-09 cs.CV

Gedas Bertasius , Heng Wang , Lorenzo Torresani This is my paper
classification cs.CV
keywords videoattentionaccuracytimesformerappliedbestclassificationdesign
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present a convolution-free approach to video classification built exclusively on self-attention over space and time. Our method, named "TimeSformer," adapts the standard Transformer architecture to video by enabling spatiotemporal feature learning directly from a sequence of frame-level patches. Our experimental study compares different self-attention schemes and suggests that "divided attention," where temporal attention and spatial attention are separately applied within each block, leads to the best video classification accuracy among the design choices considered. Despite the radically new design, TimeSformer achieves state-of-the-art results on several action recognition benchmarks, including the best reported accuracy on Kinetics-400 and Kinetics-600. Finally, compared to 3D convolutional networks, our model is faster to train, it can achieve dramatically higher test efficiency (at a small drop in accuracy), and it can also be applied to much longer video clips (over one minute long). Code and models are available at: https://github.com/facebookresearch/TimeSformer.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 1,359 citations worldwide. Full citation record

  1. Incentivizing Vision Language Models to Search for Long Video Question Answering

    cs.CV 2026-07 conditional novelty 7.0 of 10

    RL post-training of a VLM agent with neuro-symbolic temporal-logic rewards for evidence retrieval raises Pass@1 by up to 8% and Pass@4 by 15% on long-video QA.

  2. Towards Open-Vocabulary Multimodal 3D Object Detection with Attributes

    cs.CV 2025-08 conditional novelty 6.0 of 10

    OVODA combines a 3DETR-style detector with a frozen foundation model to detect novel objects and attributes in 3D scenes, and the OVAD dataset adds spatial and motion attribute labels to nuScenes.

  3. GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs

    cs.CV 2026-07 conditional novelty 5.0 of 10

    GMoT's gated motion tokens improve multimodal LLM micro-gesture recognition on iMiGUE and SMG, with limited support for reasoning-grounding claims.

  4. A Space-Time Transformer for Precipitation Nowcasting

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A full space-time attention video transformer recast as 64-class rainfall prediction with log-frequency class weighting won the Weather4Cast 2025 Cumulative Rainfall challenge (CRPS 3.135).

  5. Fine-Tuning Video Transformers for Word-Level Bangla Sign Language: A Comparative Analysis for Classification Tasks

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Off-the-shelf video transformers (VideoMAE, ViViT, TimeSformer) fine-tuned on Bangla sign language videos reach 95.5% top-1 accuracy on BdSLW60 and 81.04% on the BdSLW401 front subset.

  6. MOOSE: Pay Attention to Temporal Dynamics for Video Understanding via Optical Flows

    cs.CV 2025-06 reject novelty 5.0 of 10

    MOOSE fuses frozen DINOv2 image features with optical flow features via cross-attention and causal aggregation, reporting 70.84% Kinetics-400 and 65.23% SSv2 top-1 accuracy, alongside interpretable attention maps.

  7. CascadeFormer: A Family of Two-stage Cascading Transformers for Skeleton-based Human Action Recognition

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A masked-pretrained skeleton transformer with a second fine-tuning transformer and cross-attention fusion reaches 94.66% on Penn Action, 91.16% on N-UCLA, and 81.01%/88.17% on NTU RGB+D 60 cross-subject/cross-view.

  8. Comparing Learning Paradigms for Egocentric Video Summarization

    cs.CV 2025-06 reject novelty 4.0 of 10

    A prompt-engineered GPT-4o (quality score 64.95) outperformed Shotluck Holmes (61.19) and TAC-SUM (58.43) on a 21-video egocentric summary evaluation, though all scores were modest.

  9. The evolution of AI from image interpretation toward scientific inference in nanoparticle electron microscopy

    cond-mat.mtrl-sci 2026-07 conditional novelty 3.5 of 10

    AI for nanoparticle TEM/STEM has progressed from detection and segmentation to physics-informed restoration, 2D-to-3D inference, and spatiotemporal analysis of in situ dynamics, with remaining gaps in benchmarking and...

  10. MVP: Winning Solution to SMP Challenge 2025 Video Track

    cs.CV 2025-07 conditional novelty 3.0 of 10

    MVP, a pipeline using XCLIP video features, user metadata, and a CatBoost regressor, won the SMP Challenge 2025 Video Track with a MAPE of 0.1754.

  11. Data-Efficient Challenges in Visual Inductive Priors: A Retrospective

    cs.CV 2025-06 conditional novelty 3.0 of 10

    A retrospective of four data-limited computer vision challenges finds that ensembles and heavy augmentation, not novel inductive priors, drove winning performance.

Pith tools