Masked autoencoders with asymmetric encoder-decoder and 75% masking ratio enable scalable self-supervised pre-training of vision transformers, achieving 87.8% ImageNet-1K accuracy with ViT-Huge using only unlabeled data.
Self- supervised pretraining of visual features in the wild.CoRR, abs/2103.01988
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
fields
cs.CV 6representative citing papers
Self-supervised ViTs show emergent semantic segmentation and 78.3% k-NN accuracy on ImageNet; DINO reaches 80.1% linear evaluation with ViT-Base.
Franca introduces nested Matryoshka clustering and positional disentanglement in a transparent SSL pipeline to deliver open-source vision models competitive with closed proprietary systems.
Adding register tokens to Vision Transformers eliminates high-norm background artifacts and raises state-of-the-art performance on dense visual prediction tasks.
SMIT, which combines masked image modeling with self-distillation, delivers the highest segmentation accuracy, fastest convergence, and best few-shot performance across nine CT and MRI tasks compared to contrastive and rotation-based SSL methods.
Pith review generated a malformed one-line summary.
citing papers explorer
-
Masked Autoencoders Are Scalable Vision Learners
Masked autoencoders with asymmetric encoder-decoder and 75% masking ratio enable scalable self-supervised pre-training of vision transformers, achieving 87.8% ImageNet-1K accuracy with ViT-Huge using only unlabeled data.
-
Emerging Properties in Self-Supervised Vision Transformers
Self-supervised ViTs show emergent semantic segmentation and 78.3% k-NN accuracy on ImageNet; DINO reaches 80.1% linear evaluation with ViT-Base.
-
Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learning
Franca introduces nested Matryoshka clustering and positional disentanglement in a transparent SSL pipeline to deliver open-source vision models competitive with closed proprietary systems.
-
Vision Transformers Need Registers
Adding register tokens to Vision Transformers eliminates high-norm background artifacts and raises state-of-the-art performance on dense visual prediction tasks.
-
Benchmarking transferability of SSL pretraining to same and different modality segmentation tasks
SMIT, which combines masked image modeling with self-distillation, delivers the highest segmentation accuracy, fastest convergence, and best few-shot performance across nine CT and MRI tasks compared to contrastive and rotation-based SSL methods.
-
DINOv2: Learning Robust Visual Features without Supervision
Pith review generated a malformed one-line summary.