Pith. sign in

REVIEW 3 cited by

Masked Feature Prediction for Self-Supervised Visual Pre-Training

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2112.09133 v2 pith:ZA7RQHMN submitted 2021-12-16 cs.CV cs.LG

Chen Wei , Haoqi Fan , Saining Xie , Chao-Yuan Wu , Alan Yuille , Christoph Feichtenhofer This is my paper · ORCID
classification cs.CVcs.LG
keywords featuremaskedmaskfeatresultsvisualapproachinputmodels
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present Masked Feature Prediction (MaskFeat) for self-supervised pre-training of video models. Our approach first randomly masks out a portion of the input sequence and then predicts the feature of the masked regions. We study five different types of features and find Histograms of Oriented Gradients (HOG), a hand-crafted feature descriptor, works particularly well in terms of both performance and efficiency. We observe that the local contrast normalization in HOG is essential for good results, which is in line with earlier work using HOG for visual recognition. Our approach can learn abundant visual knowledge and drive large-scale Transformer-based models. Without using extra model weights or supervision, MaskFeat pre-trained on unlabeled videos achieves unprecedented results of 86.7% with MViT-L on Kinetics-400, 88.3% on Kinetics-600, 80.4% on Kinetics-700, 39.8 mAP on AVA, and 75.0% on SSv2. MaskFeat further generalizes to image input, which can be interpreted as a video with a single frame and obtains competitive results on ImageNet.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Token Enhancing for Vision Representation Learning

    cs.CV 2024-11 conditional novelty 6.0 of 10

    Training a vision transformer with multiple auxiliary tokens and distilling them into a single global token improves self-supervised representation quality with no additional inference cost.

  2. Self-supervised Spatial-Temporal Learner for Precipitation Nowcasting

    cs.LG 2024-12 conditional novelty 5.0 of 10

    SpaT-SparK, a SparK-based masked-image-modeling pretrainer with a translation network, reduces pMSE for short-term precipitation nowcasting but sacrifices recall and skill scores versus the smaller SmaAt-UNet.

  3. A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning

    cs.MM 2024-11 conditional novelty 3.0 of 10

    A review that categorizes deep audio-visual correlation learning methods by architectures, objective functions, datasets, and evaluation metrics, and points to missing standardized benchmarks.

Pith tools