Pith. sign in

REVIEW 17 cited by

UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2211.09552 v1 pith:2COMO4YJ submitted 2022-11-17 cs.CV

Kunchang Li , Yali Wang , Yinan He , Yizhuo Li , Yi Wang , Limin Wang , Yu Qiao This is my paper
classification cs.CV
keywords videouniformervitsuniformerv2learningarmingavailablefamily
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Learning discriminative spatiotemporal representation is the key problem of video understanding. Recently, Vision Transformers (ViTs) have shown their power in learning long-term video dependency with self-attention. Unfortunately, they exhibit limitations in tackling local video redundancy, due to the blind global comparison among tokens. UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format. However, this model has to require a tiresome and complicated image-pretraining phrase, before being finetuned on videos. This blocks its wide usage in practice. On the contrary, open-sourced ViTs are readily available and well-pretrained with rich image supervision. Based on these observations, we propose a generic paradigm to build a powerful family of video networks, by arming the pretrained ViTs with efficient UniFormer designs. We call this family UniFormerV2, since it inherits the concise style of the UniFormer block. But it contains brand-new local and global relation aggregators, which allow for preferable accuracy-computation balance by seamlessly integrating advantages from both ViTs and UniFormer. Without any bells and whistles, our UniFormerV2 gets the state-of-the-art recognition performance on 8 popular video benchmarks, including scene-related Kinetics-400/600/700 and Moments in Time, temporal-related Something-Something V1/V2, untrimmed ActivityNet and HACS. In particular, it is the first model to achieve 90% top-1 accuracy on Kinetics-400, to our best knowledge. Code will be available at https://github.com/OpenGVLab/UniFormerV2.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. NeuroLip: An Event-driven Spatiotemporal Learning Framework for Cross-Scene Lip-Motion-based Visual Speaker Recognition

    cs.CV 2026-04 conditional novelty 7.0 of 10

    NeuroLip delivers an event-driven framework for cross-scene lip-motion speaker recognition, reaching over 71% accuracy on unseen viewpoints and 76% in low light while outperforming baselines by at least 8.54%.

  2. V-Nutri: Dish-Level Nutrition Estimation from Egocentric Cooking Videos

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    V-Nutri fuses final-dish features with cooking-process keyframes from egocentric videos to improve dish-level calorie and macronutrient estimation over single-image baselines.

  3. VideoChat: Chat-Centric Video Understanding

    cs.CV 2023-05 conditional novelty 7.0 of 10

    VideoChat integrates video models and LLMs via a learnable interface for chat-based spatiotemporal and causal video reasoning, trained on a new video-centric instruction dataset.

  4. Decoding Children's Gait Behavior

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A new 1,185-video pediatric gait dataset with EVGS labels, plus a VideoMAE-based model, reaches ~84% average accuracy on 34 clinical gait items — far above MLLMs and prior gait models.

  5. PhiZero: A World Model Built Around Physical Language

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A self-supervised discrete physical-language bottleneck plus a VLM reasoner lets a world model predict state transitions before rendering video, improving physical coherence and enabling zero-shot motion transfer.

  6. Rethinking the Readout: Unlocking Video Backbones for AI-Generated Video Detection

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Replacing the global-pooling readout of a frozen video backbone with a velocity-gated, per-channel-magnitude readout improves AI-generated video detection cross-generator accuracy by several AUC points.

  7. CAM-VFD: Cross-Attention Multimodal Video Forgery Detection

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    CAM-VFD detects video forgeries by using cross-attention to identify contradictions between CLIP appearance, VideoMAE motion, and MiDaS depth features.

  8. DVAR: Adversarial Multi-Agent Debate for Video Authenticity Detection

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    DVAR turns video authenticity detection into an iterative debate between a generative hypothesis agent and a natural mechanism agent, resolved via minimum description length and a knowledge base for better generalizat...

  9. MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models

    cs.CV 2025-01 unverdicted novelty 6.0 of 10

    MotionBench is a new benchmark showing poor fine-grained motion understanding in VLMs and proposes TE Fusion to improve performance with higher frame rates.

  10. MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

    cs.CV 2023-11 accept novelty 6.0 of 10

    MVBench is a benchmark of 20 temporal video understanding tasks built by transforming static tasks into dynamic ones, with VideoChat2 outperforming prior MLLMs by over 15%.

  11. InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

    cs.CV 2023-07 unverdicted novelty 6.0 of 10

    InternVid supplies 7M videos and LLM captions to train ViCLIP, which reaches leading zero-shot action recognition and competitive retrieval performance.

  12. Spatio-Temporal Fusion Model for Standard View Classification of Echocardiographic Videos

    cs.CV 2026-06 conditional novelty 5.0 of 10

    The authors release the EV9V dataset of 5138 videos and introduce an uncertainty-aware CNN-LSTM model that achieves competitive performance on standard echocardiographic view classification.

  13. ConvFormer3D-TAP: Phase/Uncertainty-Aware Front-End Fusion for Cine CMR View Classification Pipelines

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    ConvFormer3D-TAP classifies six cine CMR views at 96% accuracy using 3D conv tokenization, multiscale attention, and uncertainty-aware multi-clip fusion on 150k sequences.

  14. OmniVec2 -- A Novel Transformer based Network for Large Scale Multimodal and Multitask Learning

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A shared-backbone transformer with pairwise modality training reports top results across 25 datasets spanning 12 modalities.

  15. Uncertainty-aware Diffusion and Reinforcement Learning for Joint Plane Localization and Anomaly Diagnosis in 3D Ultrasound

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A diffusion-plus-reinforcement-learning system jointly localizes the coronal plane and diagnoses congenital uterine anomalies in 3D ultrasound, with an uncertainty adjustment reaching 94.58% classification accuracy an...

  16. CogVLM2: Visual Language Models for Image and Video Understanding

    cs.CV 2024-08 conditional novelty 5.0 of 10

    CogVLM2 family achieves state-of-the-art results on image and video understanding benchmarks through improved visual expert architecture, higher resolution inputs, and automated temporal grounding for videos.

  17. InternVideo: General Video Foundation Models via Generative and Discriminative Learning

    cs.CV 2022-12 unverdicted novelty 5.0 of 10

    InternVideo combines masked video modeling and video-language contrastive learning into a single foundation model that reaches state-of-the-art results on 39 video datasets including 91.1% top-1 on Kinetics-400.

Pith tools