Pith Number

pith:VKM6NKHH

pith:2021:VKM6NKHHDFRCQ4RKL4VN7CV4VZ

not attested not anchored not stored refs resolved

Masked Autoencoders Are Scalable Vision Learners

Kaiming He, Piotr Doll\'ar, Ross Girshick, Saining Xie, Xinlei Chen, Yanghao Li

Masked autoencoders learn scalable vision features by reconstructing heavily masked image patches.

arxiv:2111.06377 v3 · 2021-11-11 · cs.CV

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{VKM6NKHHDFRCQ4RKL4VN7CV4VZ}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Our scalable approach allows for learning high-capacity models that generalize well: e.g., a vanilla ViT-Huge model achieves the best accuracy (87.8%) among methods that use only ImageNet-1K data.

C2weakest assumption

That masking a high proportion of the input (e.g. 75%) yields a nontrivial and meaningful self-supervisory task whose difficulty drives useful feature learning rather than trivial solutions.

C3one line summary

Masked autoencoders with asymmetric encoder-decoder and 75% masking ratio enable scalable self-supervised pre-training of vision transformers, achieving 87.8% ImageNet-1K accuracy with ViT-Huge using only unlabeled data.

References

74 extracted · 74 resolved · 5 Pith anchors

[1] Layer Normalization 2016 · arXiv:1607.06450

[2] BEiT: BERT Pre-Training of Image Transformers 2021 · arXiv:2106.08254

[3] Self-organizing neural network that discovers surfaces in random-dot stereograms 1992

[4] Language mod- els are few-shot learners 2020

[5] Emerging properties in self-supervised vision transformers 2021

Formal links

2 machine-checked theorem links

Cited by

31 papers in Pith

The TIME Machine: On The Power of Motion for Efficient Perception

KFC-W: Generating 3D-Consistent Videos from Unposed Internet Photos

Escaping Plato's Cave: JAM for Aligning Independently Trained Vision and Language Models

SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning

C$^2$FG: Control Classifier-Free Guidance via Score Discrepancy Analysis

Receipt and verification

First computed	2026-05-17T23:38:48.776710Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

aa99e6a8e7196228722a5f2adf8abcae7a418c36a55f77187f94a3a23e1c1656

Aliases

arxiv: 2111.06377 · arxiv_version: 2111.06377v3 · doi: 10.48550/arxiv.2111.06377 · pith_short_12: VKM6NKHHDFRC · pith_short_16: VKM6NKHHDFRCQ4RK · pith_short_8: VKM6NKHH

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/VKM6NKHHDFRCQ4RKL4VN7CV4VZ \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: aa99e6a8e7196228722a5f2adf8abcae7a418c36a55f77187f94a3a23e1c1656

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "6238adccafeedeef2db5d7eb1dd165dcc2a780a0709a21bf0621578993f81067",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CV",
    "submitted_at": "2021-11-11T18:46:40Z",
    "title_canon_sha256": "ea6c7158eab8cc332ab8f2aacdb3daf765d105415ff67a4b06ad26fe5613a9a1"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2111.06377",
    "kind": "arxiv",
    "version": 3
  }
}