PR-MIM recovers the accuracy lost when masked image modeling throws away tokens by reconstructing them with a lightweight convolution and spreading kept tokens, making pre-training 28% cheaper and 36% lighter without hurting ViT-B/16 performance.
Exploring simple siamese rep- resentation learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
PR-MIM: Delving Deeper into Partial Reconstruction in Masked Image Modeling
PR-MIM recovers the accuracy lost when masked image modeling throws away tokens by reconstructing them with a lightweight convolution and spreading kept tokens, making pre-training 28% cheaper and 36% lighter without hurting ViT-B/16 performance.