OCL pre-trains vision transformers by contrasting two non-overlapping randomly masked views of each image, reaching 85.8% ImageNet fine-tuning accuracy with ViT-L/16 in 133 hours on 4 A100s.
DropPos: Pre- Training Vision Transformers by Reconstructing Dropped Positions
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
One Leaf Reveals the Season: Occlusion-Based Contrastive Learning with Semantic-Aware Views for Efficient Visual Representation
OCL pre-trains vision transformers by contrasting two non-overlapping randomly masked views of each image, reaching 85.8% ImageNet fine-tuning accuracy with ViT-L/16 in 133 hours on 4 A100s.