REVIEW 2 cited by
Watching the World Go By: Representation Learning from Unlabeled Videos
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent single image unsupervised representation learning techniques show remarkable success on a variety of tasks. The basic principle in these works is instance discrimination: learning to differentiate between two augmented versions of the same image and a large batch of unrelated images. Networks learn to ignore the augmentation noise and extract semantically meaningful representations. Prior work uses artificial data augmentation techniques such as cropping, and color jitter which can only affect the image in superficial ways and are not aligned with how objects actually change e.g. occlusion, deformation, viewpoint change. In this paper, we argue that videos offer this natural augmentation for free. Videos can provide entirely new views of objects, show deformation, and even connect semantically similar but visually distinct concepts. We propose Video Noise Contrastive Estimation, a method for using unlabeled video to learn strong, transferable single image representations. We demonstrate improvements over recent unsupervised single image techniques, as well as over fully supervised ImageNet pretraining, across a variety of temporal and non-temporal tasks. Code and the Random Related Video Views dataset are available at https://www.github.com/danielgordon10/vince
Forward citations
Cited by 2 Pith papers
-
MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning
MoSiC clusters dense point tracks in videos and propagates the cluster assignments along the tracks, improving DINOv2's dense representations by 1 to 6 percent on segmentation and in-context benchmarks.
-
Self-Supervised Spatial Correspondence Across Modalities
Dense pixel-level correspondence across visual modalities (RGB, depth, thermal, sketch, style) can be learned from unlabeled videos via cycle-consistent contrastive random walks.
Discussion (0). Sign in to comment.