REVIEW 1 cited by
On the Memorization Properties of Contrastive Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
On the Memorization Properties of Contrastive Learning
read the original abstract
Memorization studies of deep neural networks (DNNs) help to understand what patterns and how do DNNs learn, and motivate improvements to DNN training approaches. In this work, we investigate the memorization properties of SimCLR, a widely used contrastive self-supervised learning approach, and compare them to the memorization of supervised learning and random labels training. We find that both training objects and augmentations may have different complexity in the sense of how SimCLR learns them. Moreover, we show that SimCLR is similar to random labels training in terms of the distribution of training objects complexity.
Forward citations
Cited by 1 Pith paper
-
MultiMem: Measuring and Mitigating Memorization in Multi-Modal Contrastive Learning
Introduces MultiMem as the first metric for memorization in multi-modal contrastive learning, identifies cross-modal misalignment (text dominant) as key driver, and shows targeted augmentations reduce it while improvi...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.