Pith. sign in

REVIEW 7 cited by

UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2201.04676 v3 pith:KSOGMIUN submitted 2022-01-12 cs.CV

Kunchang Li , Yali Wang , Peng Gao , Guanglu Song , Yu Liu , Hongsheng Li , Yu Qiao This is my paper
classification cs.CV
keywords localuniformerdependencyredundancyspatiotemporalaccuracyachievesglobal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

It is a challenging task to learn rich and multi-scale spatiotemporal semantics from high-dimensional videos, due to large local redundancy and complex global dependency between video frames. The recent advances in this research have been mainly driven by 3D convolutional neural networks and vision transformers. Although 3D convolution can efficiently aggregate local context to suppress local redundancy from a small 3D neighborhood, it lacks the capability to capture global dependency because of the limited receptive field. Alternatively, vision transformers can effectively capture long-range dependency by self-attention mechanism, while having the limitation on reducing local redundancy with blind similarity comparison among all the tokens in each layer. Based on these observations, we propose a novel Unified transFormer (UniFormer) which seamlessly integrates merits of 3D convolution and spatiotemporal self-attention in a concise transformer format, and achieves a preferable balance between computation and accuracy. Different from traditional transformers, our relation aggregator can tackle both spatiotemporal redundancy and dependency, by learning local and global token affinity respectively in shallow and deep layers. We conduct extensive experiments on the popular video benchmarks, e.g., Kinetics-400, Kinetics-600, and Something-Something V1&V2. With only ImageNet-1K pretraining, our UniFormer achieves 82.9%/84.8% top-1 accuracy on Kinetics-400/Kinetics-600, while requiring 10x fewer GFLOPs than other state-of-the-art methods. For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60.9% and 71.2% top-1 accuracy respectively. Code is available at https://github.com/Sense-X/UniFormer.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EPFL-Smart-Kitchen-30: Densely annotated cooking dataset with 3D kinematics to challenge video and language models

    cs.CV 2025-06 conditional novelty 7.0 of 10

    EPFL-Smart-Kitchen-30 is a 29.7-hour multimodal cooking dataset with 60k action segments and four benchmarks, including a kinematic-focused VQA benchmark that shows current video-language models struggle with hand and...

  2. SPACT18: Spiking Human Action Recognition Benchmark Dataset with Complementary RGB and Thermal Modalities

    cs.CV 2025-07 conditional novelty 6.0 of 10

    SPACT18 is claimed to be the first action recognition dataset captured with a spike camera, paired with synchronized RGB and thermal video.

  3. CXR-LT 2024: A MICCAI challenge on long-tailed, multi-label, and zero-shot disease classification from chest X-ray

    cs.CV 2025-06 conditional novelty 6.0 of 10

    CXR-LT 2024 provides a new large chest X-ray benchmark with 45 labels and three tasks, and reports that top models achieve mAP of 0.28 to 0.53 on long-tailed tasks but only 0.11 to 0.13 on zero-shot unseen diseases.

  4. FuseMamba-VD: Dual Branch VideoMamba with Gated Class Token Fusion for Violence Detection

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A dual-branch VideoMamba with gated class-token fusion achieves 95.85% accuracy on a newly combined violence-detection benchmark and 74.13% on DVD, with about half the parameters and FLOPs of the CUE-Net baseline.

  5. DeSPITE: Exploring Contrastive Deep Skeleton-Pointcloud-IMU-Text Embeddings for Advanced Point Cloud Human Activity Understanding

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A contrastive model aligns LiDAR point clouds, skeletons, IMU signals, and text in one embedding space, enabling cross-modal matching and improving point cloud activity recognition.

  6. CA3D: Convolutional-Attentional 3D Nets for Efficient Video Activity Recognition on the Edge

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A compact spatio-temporal network mixing convolutions and linear-complexity temporal attention reaches strong accuracy on UCF101, HMDB51, and Kinetics400 with a 7M-parameter model and float16 training.

  7. DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition

    cs.CV 2025-07 conditional novelty 3.0 of 10

    DVFL-Net, a 22M-parameter distilled video model, reaches near-teacher accuracy on five action recognition benchmarks with 27 GFLOPs, but its claimed state-of-the-art status is not fully supported by the reported numbers.

Pith tools