REVIEW 4 cited by
Debiased Contrastive Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
A prominent technique for self-supervised representation learning has been to contrast semantically similar and dissimilar pairs of samples. Without access to labels, dissimilar (negative) points are typically taken to be randomly sampled datapoints, implicitly accepting that these points may, in reality, actually have the same label. Perhaps unsurprisingly, we observe that sampling negative examples from truly different labels improves performance, in a synthetic setting where labels are available. Motivated by this observation, we develop a debiased contrastive objective that corrects for the sampling of same-label datapoints, even without knowledge of the true labels. Empirically, the proposed objective consistently outperforms the state-of-the-art for representation learning in vision, language, and reinforcement learning benchmarks. Theoretically, we establish generalization bounds for the downstream classification task.
Forward citations
Cited by 4 Pith papers
-
Fusion Embedding: A Unified Embedding Space for Text, Image, Video, and Audio
Trained connectors and audio-only gated adapters integrate audio into a frozen vision-language embedding space, preserving base outputs bit-exactly and yielding emergent audio-image retrieval.
-
I-Con: A Unifying Framework for Representation Learning
One KL-divergence loss over conditional neighbor distributions is claimed to cover 23 representation learning methods, and its debiasing variant improves frozen-feature ImageNet clustering.
-
TRUST: Leveraging Text Robustness for Unsupervised Domain Adaptation
TRUST adapts a vision model to an unlabeled target domain by generating pseudo-labels from captions, weighting them by caption-based uncertainty, and aligning image and text features with a soft contrastive loss, repo...
-
Image Classification Using a Diffusion Model as a Pre-Training Model
Representation-conditioned diffusion pre-training improves hematoma classification accuracy by +6.15% and F1 by +13.60% over DINOv2 on a 179-image brain CT test set.
Discussion (0). Continue with ORCID to comment.