Lifting Architectural Constraints of Injective Flows
Reviewed by Pithpith:6EEL43ISopen to challenge →
read the original abstract
Normalizing Flows explicitly maximize a full-dimensional likelihood on the training data. However, real data is typically only supported on a lower-dimensional manifold leading the model to expend significant compute on modeling noise. Injective Flows fix this by jointly learning a manifold and the distribution on it. So far, they have been limited by restrictive architectures and/or high computational cost. We lift both constraints by a new efficient estimator for the maximum likelihood loss, compatible with free-form bottleneck architectures. We further show that naively learning both the data manifold and the distribution on it can lead to divergent solutions, and use this insight to motivate a stable maximum likelihood training objective. We perform extensive experiments on toy, tabular and image data, demonstrating the competitive performance of the resulting model.
This paper has not been read by Pith yet.
Forward citations
Cited by 3 Pith papers
-
Understanding Self-Supervised Learning via Latent Distribution Matching
Self-supervised learning is cast as latent distribution matching that aligns representations to a model while enforcing uniformity, unifying multiple SSL families and proving identifiability for predictive variants ev...
-
Understanding Self-Supervised Learning via Latent Distribution Matching
Self-supervised learning is recast as latent distribution matching that unifies multiple SSL families and yields a sampling-free Kalman-based predictor plus an identifiability proof for predictive variants under mild ...
-
Understanding Self-Supervised Learning via Latent Distribution Matching
Self-supervised learning is recast as latent distribution matching that unifies ICA with contrastive and predictive methods and derives an identifiable nonlinear Bayesian filtering model for timeseries.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.