Pith. sign in

REVIEW 8 cited by

SimMIM: A Simple Framework for Masked Image Modeling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2111.09886 v2 pith:3YQV62P5 submitted 2021-11-18 cs.CV

SimMIM: A Simple Framework for Masked Image Modeling

Zhenda Xie , Zheng Zhang , Yue Cao , Yutong Lin , Jianmin Bao , Zhuliang Yao , Qi Dai , Han Hu This is my paper
classification cs.CV
keywords imagemaskedapproachdesignsframeworkimagenet-1kmodelingsimmim
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper presents SimMIM, a simple framework for masked image modeling. We simplify recently proposed related approaches without special designs such as block-wise masking and tokenization via discrete VAE or clustering. To study what let the masked image modeling task learn good representations, we systematically study the major components in our framework, and find that simple designs of each component have revealed very strong representation learning performance: 1) random masking of the input image with a moderately large masked patch size (e.g., 32) makes a strong pre-text task; 2) predicting raw pixels of RGB values by direct regression performs no worse than the patch classification approaches with complex designs; 3) the prediction head can be as light as a linear layer, with no worse performance than heavier ones. Using ViT-B, our approach achieves 83.8% top-1 fine-tuning accuracy on ImageNet-1K by pre-training also on this dataset, surpassing previous best approach by +0.6%. When applied on a larger model of about 650 million parameters, SwinV2-H, it achieves 87.1% top-1 accuracy on ImageNet-1K using only ImageNet-1K data. We also leverage this approach to facilitate the training of a 3B model (SwinV2-G), that by $40\times$ less data than that in previous practice, we achieve the state-of-the-art on four representative vision benchmarks. The code and models will be publicly available at https://github.com/microsoft/SimMIM.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

    cs.CV 2026-06 unverdicted novelty 6.0

    MIMFlow is an end-to-end model that routes semantic latents through a normalizing flow while a decoder handles high-frequency pixels, reporting FID 2.50 and 71.3% linear probing accuracy on ImageNet 256x256 with 128 tokens.

  2. MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

    cs.CV 2026-06 accept novelty 6.0

    End-to-end masked-image VAE plus normalizing flow yields FID 2.50 on ImageNet 256 with 128 tokens and higher linear-probe accuracy than unmasked counterparts.

  3. MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

    cs.CV 2026-06 unverdicted novelty 6.0

    MIMFlow uses a VAE on masked images to feed semantic latents to a normalizing flow while a decoder handles high-frequency details, reporting FID 2.50 and 71.3% linear probing on ImageNet 256x256 with 128 tokens.

  4. Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training

    cs.CV 2026-05 unverdicted novelty 6.0

    Transcoda achieves state-of-the-art zero-shot OMR with an 18.46% OMR-NED error rate on synthetic scores and 63.97% on historical Polish scans using a 59M model trained in 6 hours via synthetic data, kern normalization...

  5. Revisiting Feature Prediction for Learning Visual Representations from Video

    cs.CV 2024-02 conditional novelty 6.0

    V-JEPA models trained only on feature prediction from 2 million public videos achieve 81.9% on Kinetics-400, 72.2% on Something-Something-v2, and 77.9% on ImageNet-1K using frozen ViT-H/16 backbones.

  6. Vision Transformers Need Registers

    cs.CV 2023-09 unverdicted novelty 6.0

    Adding register tokens to Vision Transformers eliminates high-norm background artifacts and raises state-of-the-art performance on dense visual prediction tasks.

  7. CoCa: Contrastive Captioners are Image-Text Foundation Models

    cs.CV 2022-05 accept novelty 6.0

    CoCa unifies contrastive and generative pretraining in one image-text model to reach 86.3% zero-shot ImageNet accuracy and new state-of-the-art results on multiple downstream benchmarks.

  8. Expectation and Acoustic Neural Network Representations Enhance Music Identification from Brain Activity

    cs.AI 2026-03 unverdicted novelty 5.0

    Separating acoustic and expectation ANN representations as teacher targets improves EEG music identification beyond baselines and seed ensembles.