REVIEW 5 cited by
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
read the original abstract
We construct an unsupervised learning model that achieves nonlinear disentanglement of underlying factors of variation in naturalistic videos. Previous work suggests that representations can be disentangled if all but a few factors in the environment stay constant at any point in time. As a result, algorithms proposed for this problem have only been tested on carefully constructed datasets with this exact property, leaving it unclear whether they will transfer to natural scenes. Here we provide evidence that objects in segmented natural movies undergo transitions that are typically small in magnitude with occasional large jumps, which is characteristic of a temporally sparse distribution. We leverage this finding and present SlowVAE, a model for unsupervised representation learning that uses a sparse prior on temporally adjacent observations to disentangle generative factors without any assumptions on the number of changing factors. We provide a proof of identifiability and show that the model reliably learns disentangled representations on several established benchmark datasets, often surpassing the current state-of-the-art. We additionally demonstrate transferability towards video datasets with natural dynamics, Natural Sprites and KITTI Masks, which we contribute as benchmarks for guiding disentanglement research towards more natural data domains.
Forward citations
Cited by 5 Pith papers
-
Mechanistic Independence: A Principle for Identifiable Disentangled Representations
Mechanistic independence criteria yield identifiability of latent subspaces under nonlinear mixing by focusing on action-based independence rather than latent distributions, with a hierarchy and graph-theoretic view o...
-
MOSAIC: Module Discovery via Sparse Additive Identifiable Causal Learning for Scientific Time Series
MOSAIC recovers identifiable latent variables and their sparse associated observations in scientific time series by combining temporal causal representation learning with support recovery through a sparse additive decoder.
-
MoVA: Learning Asymmetric Dual Projections for Modular Long Video-Text Alignment
MoVA introduces modular asymmetric dual projections to handle temporal misalignment and semantic asymmetry in long video-text alignment.
-
Cloning Deterministic Worlds: The Critical Role of Latent Geometry in Long-Horizon World Models
GRWM uses temporal contrastive learning to geometrically regularize latent spaces in world models for high-fidelity cloning of deterministic 3D worlds.
-
There Will Be a Scientific Theory of Deep Learning
A mechanics of the learning process is emerging in deep learning theory, characterized by dynamics, coarse statistics, and falsifiable predictions across idealized settings, limits, laws, hyperparameters, and universa...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.