REVIEW 6 cited by
VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Scale is the primary factor for building a powerful foundation model that could well generalize to a variety of downstream tasks. However, it is still challenging to train video foundation models with billions of parameters. This paper shows that video masked autoencoder (VideoMAE) is a scalable and general self-supervised pre-trainer for building video foundation models. We scale the VideoMAE in both model and data with a core design. Specifically, we present a dual masking strategy for efficient pre-training, with an encoder operating on a subset of video tokens and a decoder processing another subset of video tokens. Although VideoMAE is very efficient due to high masking ratio in encoder, masking decoder can still further reduce the overall computational cost. This enables the efficient pre-training of billion-level models in video. We also use a progressive training paradigm that involves an initial pre-training on a diverse multi-sourced unlabeled dataset, followed by a post-pre-training on a mixed labeled dataset. Finally, we successfully train a video ViT model with a billion parameters, which achieves a new state-of-the-art performance on the datasets of Kinetics (90.0% on K400 and 89.9% on K600) and Something-Something (68.7% on V1 and 77.0% on V2). In addition, we extensively verify the pre-trained video ViT models on a variety of downstream tasks, demonstrating its effectiveness as a general video representation learner. The code and model is available at \url{https://github.com/OpenGVLab/VideoMAEv2}.
Forward citations
Cited by 6 Pith papers
-
The TIME Machine: On The Power of Motion for Efficient Perception
TIME is a motion-based embedding from point tracks, trained only on synthetic data via masked autoencoding, that matches state-of-the-art video model performance with up to 10,000x less training data.
-
Parallel Decoding Distillation for Fast Image and Video Generation
A trajectory-based distillation method trains a student to predict multiple mean velocities per network evaluation, enabling 4-8 step generation with competitive quality and improved diversity.
-
Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation
Kandinsky 5.0 is a released family of image and video generation models (up to 19B parameters) that uses flow matching, a CrossDiT architecture, sparse attention, and multi-stage training to produce high-quality 10-se...
-
CuriosAI Submission to the EgoExo4D Proficiency Estimation Challenge 2025
On EgoExo4D proficiency estimation, a two-stage pipeline with zero-shot scenario recognition and per-scenario, per-view VideoMAE classifiers (47.8% validation) outperforms a Sapiens-2B multi-task model (43.6%).
-
A Survey on Efficiency Optimization Techniques for DNN-based Video Analytics: Process Systems, Algorithms, and Applications
A bottom-up survey of efficiency optimization techniques for DNN-based video analytics, spanning storage, computing, algorithms, and applications.
-
Infinite Video Understanding
The paper argues that video understanding research should aim at processing streams of arbitrary, unbounded duration and outlines the challenges, directions, and metrics needed.
Discussion (0). Sign in to comment.