REVIEW 5 cited by
I-Con: A Unifying Framework for Representation Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
I-Con: A Unifying Framework for Representation Learning
read the original abstract
As the field of representation learning grows, there has been a proliferation of different loss functions to solve different classes of problems. We introduce a single information-theoretic equation that generalizes a large collection of modern loss functions in machine learning. In particular, we introduce a framework that shows that several broad classes of machine learning methods are precisely minimizing an integrated KL divergence between two conditional distributions: the supervisory and learned representations. This viewpoint exposes a hidden information geometry underlying clustering, spectral methods, dimensionality reduction, contrastive learning, and supervised learning. This framework enables the development of new loss functions by combining successful techniques from across the literature. We not only present a wide array of proofs, connecting over 23 different approaches, but we also leverage these theoretical results to create state-of-the-art unsupervised image classifiers that achieve a +8% improvement over the prior state-of-the-art on unsupervised classification on ImageNet-1K. We also demonstrate that I-Con can be used to derive principled debiasing methods which improve contrastive representation learners.
Forward citations
Cited by 5 Pith papers
-
Understanding Self-Supervised Learning via Latent Distribution Matching
Self-supervised learning is cast as latent distribution matching that aligns representations to a model while enforcing uniformity, unifying multiple SSL families and proving identifiability for predictive variants ev...
-
Understanding Self-Supervised Learning via Latent Distribution Matching
Self-supervised learning is recast as latent distribution matching that unifies multiple SSL families and yields a sampling-free Kalman-based predictor plus an identifiability proof for predictive variants under mild ...
-
Understanding Self-Supervised Learning via Latent Distribution Matching
Self-supervised learning can be understood as latent distribution matching, and under a Gaussian predictive model this yields identifiable representations up to affine transformations.
-
Learning Minimal Representations of Fermionic Ground States
Autoencoders trained on Hubbard ground-state measurement vectors show a sharp reconstruction threshold at L−1 latent dimensions, and the decoder can be used as a variational ansatz for energy minimization.
-
Understanding Self-Supervised Learning via Latent Distribution Matching
Self-supervised learning is recast as latent distribution matching that unifies ICA with contrastive and predictive methods and derives an identifiable nonlinear Bayesian filtering model for timeseries.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.