Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that diverse-skill pretraining in reinforcement learning recovers the ground-truth states of the environment up to a linear transformation, providing the first identifiability guarantee for representation learning in RL.

desk verdict A sound conditional identifiability result for CSF, but the paper overstates its reach: the key assumptions are supported by empirical plots of learned features, not ground-truth state differences, so the guarantee is not actually shown to apply to CSF's optimization. read the letter →

arxiv 2507.14748 v1 pith:WKPADHTA submitted 2025-07-19 cs.LG cs.AIstat.ML

Patrik Reizinger , Bálint Mucsányi , Siyuan Guo , Benjamin Eysenbach , Bernhard Schölkopf , Wieland Brendel This is my paper
classification cs.LGcs.AIstat.ML
keywords mutualinformationskilllearningidentifiablerepresentationcontrastivesuccessorfeaturesnonlinearICAreinforcementunsuperviseddiscoverylinearidentifiabilitycross-entropyobjective
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to prove why mutual information skill learning (MISL) works in reinforcement learning, using the Contrastive Successor Features (CSF) method as the case study. The central claim is that when an agent learns a diverse set of skills by maximizing the inner product between skill vectors and feature differences, the learned features provably match the environment's ground-truth states up to a linear transformation. This is the first identifiability guarantee for representation learning in RL, and it explains why specific design choices—an inner-product critic, a contrastive (cross-entropy) objective, and diversity rather than entropy—are the ones that succeed. If true, it means self-supervised RL pretraining is not just a heuristic for exploration but a principled way to invert the environment's data-generating process.

What carries the argument

The central object is the Contrastive Successor Features (CSF) objective, in which a critic $q(z_i|\phi(o),\phi(o'))$ is trained with a contrastive (cross-entropy) loss to infer the skill $z_i$ from consecutive observations via the inner product $(\phi(o')-\phi(o))^\top z_i$. The load-bearing identity is the identifiability theorem (Theorem 1, adapted from nonlinear ICA): under a von Mises-Fisher conditional distribution for feature differences, a uniform marginal on the hypersphere, and diverse skills that span $\mathbb{R}^d$, a globally optimal encoder and linear classifier yield $\phi(o')-\phi(o) = A(s'-s)$. Proposition 1 extends this to the features themselves, $\phi(o) = A s$, using the inner product parametrization. This theorem is what converts the heuristic of 'learning diverse skills' into a provable recovery of the ground-truth states.

What would settle it

In a new simulated environment with known ground-truth states, record the learned feature differences and test whether they follow the assumed von Mises-Fisher distribution with a uniform marginal on the hypersphere, and whether the skill can be decoded from the state pair. If a case is found where those assumptions clearly fail but the $R^2$ between learned features and ground truth still approaches 1, then the theorem's premises are not necessary for the observed identifiability; if the $R^2$ collapses whenever the assumptions fail, they are the load-bearing ingredient.

Watch

Extended reading notes

Core claim

Under Assumption 1, the paper proves that the features learned by CSF are identifiable up to a linear map: if a continuous encoder and a linear classifier globally minimize the contrastive cross-entropy objective, then $\phi(o') - \phi(o) = A(s' - s)$ for a linear map $A \in \mathbb{R}^{d\times d}$, and because the critic uses an inner product parametrization, the same $A$ relates the features themselves, $\phi(o) = A s$. The proof adapts a known nonlinear-ICA identifiability result to the RL setting, interpreting skills as auxiliary variables and the required 'diverse policies' as the sufficient variability condition. The paper further shows that maximum-entropy policies cannot be diverse, that the choice of mutual information objective $I(s,s';z)$ versus $I(s;z)$ matters for the geometry of the latent space, and that a finite skill set must span the state space for identifiability. Experiments in MuJoCo and DeepMind Control confirm that the learned features and feature differences achieve high $R^2$ with ground-truth states, both from states and from pixels.

Load-bearing premise

The proof assumes that the learned skill-conditioned policies are genuinely diverse, meaning an ideal discriminator can always infer which skill produced a given state transition; if the policies ever fail to be diverse, the identifiability conclusion collapses.

Editorial extensions

If this is right

  • The learned CSF features can be used to read out the ground-truth states of the MDP with a linear map, which should make downstream task solving and planning easier.
  • Designing MISL methods away from these ingredients—for example replacing the inner-product critic with a more expressive one or adding a maximum-entropy regularizer—should degrade identifiability and, correspondingly, zero-shot transfer.
  • A finite set of skills is sufficient for identifiability only if the skill vectors form an affine generator system of the state space; using too few or non-spanning skills breaks the guarantee and reduces exploration.
  • The distinction between optimizing $I(s,s';z)$ and $I(s;z)$ is not cosmetic: only the transition-based mutual information imposes the locality constraint that keeps consecutive-state embeddings close but distinct.
  • In pixel-based environments, feature identifiability is still achieved, though the linear relationship for feature differences is weaker than for states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to test whether the same linear identifiability holds when the generator $g$ is non-injective or discontinuous; if empirical $R^2$ stays high, the injectivity assumption could be relaxed.
  • The paper's link between policy diversity and ICA's sufficient variability implies that adding an explicit diversity term to the policy loss, rather than letting it emerge from the reward, could make identifiability hold in environments where exploration is initially poor.
  • If the linear map $A$ is consistent across seeds and tasks, the learned features could serve as a fixed coordinate system for an agent, letting a downstream policy be trained directly in the ground-truth state space.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper analyzes Contrastive Successor Features (CSF), a mutual-information skill learning (MISL) method, through the lens of identifiable representation learning. It adapts an identifiability theorem from Reizinger et al. [49] to the RL setting, stating that under Assumption 1—which includes von Mises-Fisher skill-conditioned state differences, a uniform marginal over differences, and diverse policies—CSF's encoder recovers the ground-truth states up to a linear transformation. The authors also present Lemma 1 against maximum-entropy policies, discuss implications of different MI objectives, and empirically evaluate CSF in MuJoCo and DeepMind Control using R² scores between learned features and ground-truth states, state coverage, and oracle returns.

Significance. The paper is the first to attempt an identifiability guarantee for representation learning in RL, and it draws a useful connection between MISL and nonlinear ICA. The conditional theorem is imported and sound, and the practical insight about entropy regularization is interesting. However, the applicability of the theorem to CSF is not established: the assumptions concern the ground-truth state differences, while the paper's feasibility argument uses learned feature differences, and the diversity assumption is assumed rather than demonstrated. The work is therefore best viewed as a promising theoretical framework with a plausibility study, rather than a proof that CSF in practice identifies the ground truth.

major comments (4)
  1. [§3.1, Assumption 1 feasibility items (ii) and (iii)] The paper cites [71, Fig. 2] to justify Assumption 1(ii)-(iii), but that figure concerns the learned feature differences φ(o')−φ(o), whereas the assumption is about the ground-truth state differences s'−s. Since φ is unknown and not yet proven linear at the point of justification, the distribution of φ(o')−φ(o) does not provide evidence about the distribution of s'−s. This is circular: using the learned features to verify assumptions about the true states presupposes the linear identifiability the theorem aims to establish. The authors should either derive these properties from the training objective or measure them directly on the ground-truth states.
  2. [§3.1, Assumption 1(iv) and Lemma 1] Assumption 1(iv) asserts that an ideal discriminator can uniquely map (s'−s) to z, i.e., that the learned policies are diverse. Lemma 1 only rules out the maximum-entropy policy as a failure mode; it does not show that the policies obtained by optimizing the reward in Eq. (3) are diverse in the sense of Definition 1. The paragraph following Lemma 1 in the feasibility section is an intuitive argument, not a proof. The theorem requires this assumption to hold for the actual data distribution, so the paper needs to either prove it under additional conditions or empirically verify it (e.g., by training a discriminator on ground-truth state pairs from the learned policies and measuring its accuracy).
  3. [§3.1, Proposition 1 proof] The proof claims that an offset is impossible 'as they lie on S^{d-1}', but Assumption 1 only constrains the differences (s'−s) to the sphere, not the states s themselves. From φ(o')−φ(o)=A(s'−s) one can at most conclude φ(o)=As+c for a constant c. The identifiability statement should be amended to 'up to an affine transformation', or an additional assumption (e.g., that the features and states are centered, or that a reference state is known) must be added. As written, Proposition 1 is not proven.
  4. [Abstract and §3.1] The paper claims that 'CSF can provably recover the environment's ground-truth features up to a linear transformation'. What is actually proven is a conditional statement: if Assumption 1 holds, then an encoder and linear classifier that globally minimize the cross-entropy objective are linearly identifiable. The paper does not show that CSF's objective produces data satisfying Assumption 1, and the empirical R² scores are consistent with but do not verify the assumptions. The claim is therefore stronger than the evidence supports; the manuscript should separate the conditional theorem from the claim of CSF's practical identifiability.
minor comments (4)
  1. [§4 vs Figure 3 caption] The text says error bars represent two standard deviations unless otherwise noted, but the caption of Figure 3 says 'Error bars represent one standard deviation.' Please reconcile this discrepancy.
  2. [§3.1, feasibility (i)] The statement that 'this setup is sufficient, but not necessary: a set of discrete skills also leads to high identifiability scores (Fig. 4)' is misleading without qualification: Fig. 4 shows high R² only for large discrete skill sets (100, 500), while small sets perform poorly. Please rephrase to clarify that diversity in terms of spanning the sphere is what matters.
  3. [§3.2, Lemma 1 proof] The proof of Lemma 1 is informal, particularly the step 'by reordering the terms, we get' and the final contradiction. It would benefit from a more explicit derivation of the expression for the reward under a skill-independent policy.
  4. [§4, linear fit] The text states that a linear map A is fit by minimizing ||s − Aφ(o)||²₂. Given the potential offset issue in Proposition 1, the paper should state whether an intercept was included in the regression or justify why it is not needed.

Circularity Check

2 steps flagged · score 4.0 of 10

CSF identifiability proof is conditional on Assumption 1, whose feasibility is supported by plots of the learned features phi(o')-phi(o) rather than the ground-truth state differences s'-s, and whose diversity premise is asserted via the CSF objective rather than derived.

  1. ansatz smuggled in via citation [Sec. 3.1, 'Assumption feasibility' for Assumption 1(ii)-(iii)]
    "(ii) Empirical observations show that the learned features ϕ(o′) − ϕ(o) follow a vMF for CSF [71, Fig. 2(a-b)] (iii) Empirical observations show the learned features ϕ(o′) − ϕ(o) follow a uniform marginal on the hypersphere for CSF [71, Fig. 2(c)]"

    Assumptions 1(ii)-(iii) are conditions on the ground-truth feature differences p(s′−s|z). The cited evidence from [71] is about the learned encoder outputs ϕ(o′)−ϕ(o). At this point in the derivation Thm 1 has not yet been applied, so ϕ is not known to be a linear function of s. Using the learned differences to validate distributional assumptions about s′−s assumes the very linear identifiability the theorem is meant to establish; the vMF/uniform conditions are thus imported from the same group's empirical plots of the learned representation rather than verified on the DGP quantities the theorem requires.

  2. self definitional [Sec. 3.1, 'Assumption feasibility' for Assumption 1(iv); Lemma 1]
    "The policy optimizes [ϕ(o′) − ϕ(o)]⊤ zi. As we show in Lem. 1, a maximum entropy policy is not diverse. Applying our argument to pairs of skills, one can always improve diversity if the state transitions depend on any one of those two skills (as opposed to no skill dependence, which is implied by a maximum entropy policy), which implies that each state transition can be mapped to a single skill. Refer to Lem. 1 for details."

    Assumption 1(iv) is exactly the diversity premise the theorem needs: an ideal discriminator can uniquely map (s′−s) to z. The feasibility argument asserts that optimizing the CSF reward produces this property and concludes 'each state transition can be mapped to a single skill.' But Lemma 1 only rules out the maximum-entropy policy as one failure mode; it does not show that the learned skill-conditioned policies are diverse for ground-truth state differences. The desired uniqueness condition is restated as a consequence of the reward objective rather than derived, making the identifiability guarantee for CSF conditional on a premise that is assumed as part of the proof's input.

full rationale

The main theorem (Thm. 1) is imported from Reizinger et al. [49], which shares authors with this paper; I do not treat that import as circular by itself, because [49] is a parameter-free theoretical statement with stated assumptions that do not include the CSF conclusion. The circularity is in the application to CSF. The 'Assumption feasibility' paragraph validates Assumption 1(ii)-(iii), which are conditions on ground-truth s′−s, using [71, Fig. 2] observations of the learned ϕ(o′)−ϕ(o). Before Thm. 1 is proved, ϕ is an arbitrary learned encoder and is not known to be linear in s, so observations about ϕ(o′)−ϕ(o) cannot be transferred to s′−s without assuming the linear identifiability that is the paper's conclusion. Likewise, Assumption 1(iv) is the diversity premise; the feasibility argument asserts the CSF reward yields it, yet Lemma 1 only excludes the maximum-entropy policy and does not establish that every state transition is uniquely mapped to a skill. The paper's own Limitations section concedes that the result 'is based on the observations of Zheng et al. [71]' for the assumptions. The MuJoCo/DMC R2 experiments are external benchmarks and would be meaningful if the assumptions were independently verified, but they do not repair the self-referential assumption validation. A separate, non-circular correctness concern: Proposition 1's proof claims an offset is impossible because states 'lie on S^{d−1}', but the difference equation ϕ(o′)−ϕ(o)=A(s′−s) is invariant under adding a constant to ϕ, and the stated assumptions do not constrain ϕ to the sphere, so the passage from difference identifiability to pointwise identifiability is not justified as written. Overall: partial circularity in the premise-to-application step, score 4.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The theoretical result is conditional on strong distributional and diversity assumptions inherited from nonlinear ICA, and the central theorem is imported from a self-cited prior paper. No new entities are introduced, and no free parameters are fitted in the derivation.

assumptions (6)
  • domain assumption Transition differences (s' - s) given skill z follow a von Mises-Fisher distribution on the hypersphere with mean direction z (Assumption 1(ii)).
    Required to apply the nonlinear ICA identifiability theorem from [49]; the paper justifies it by citing empirical plots of learned features in [71], but it is not derived from environment dynamics or the CSF objective.
  • domain assumption The marginal distribution of state differences (s' - s) is uniform on the hypersphere (Assumption 1(iii)).
    Needed by the theorem; support is empirical from [71].
  • domain assumption Skill-conditioned policies are diverse: an ideal discriminator can uniquely map (s' - s) to z (Assumption 1(iv)).
    This is the key premise connecting skill learning to ICA auxiliary-variable identifiability. Lemma 1 only shows max-entropy policies are not diverse; it does not show that CSF produces diverse policies.
  • domain assumption The generator g: S^{d-1} -> R^D is continuous and injective (Assumption 1(vi)).
    Standard in nonlinear ICA; for state-based experiments g is the identity, for pixel inputs this is assumed without testing.
  • domain assumption The critic and encoder reach the global optimum of the cross-entropy objective, and a linear classifier exists (Assumption 1(v)).
    The theorem requires global optimality and a linear classifier; deep networks only approximately satisfy this.
  • standard math Theorem 1 of Reizinger et al. [49] is correct.
    The paper's central result is quoted verbatim from [49], a paper sharing authors; the proof is not reproduced here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning." pith.science (2026). https://pith.science/paper/WKPADHTA

@misc{pith2026250714748,
  author       = {Pith},
  title        = {Pith review of: Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WKPADHTA}},
  note         = {Machine review of arXiv:2507.14748}
}
read the original abstract

Self-supervised feature learning and pretraining methods in reinforcement learning (RL) often rely on information-theoretic principles, termed mutual information skill learning (MISL). These methods aim to learn a representation of the environment while also incentivizing exploration thereof. However, the role of the representation and mutual information parametrization in MISL is not yet well understood theoretically. Our work investigates MISL through the lens of identifiable representation learning by focusing on the Contrastive Successor Features (CSF) method. We prove that CSF can provably recover the environment's ground-truth features up to a linear transformation due to the inner product parametrization of the features and skill diversity in a discriminative sense. This first identifiability guarantee for representation learning in RL also helps explain the implications of different mutual information objectives and the downsides of entropy regularizers. We empirically validate our claims in MuJoCo and DeepMind Control and show how CSF provably recovers the ground-truth features both from states and pixels.

Figures

Figures reproduced from arXiv: 2507.14748 by the authors.

Figure 1
Figure 1. The success of mutual information skill learning (MISL) [71] can be explained by learning identifiable features: The MISL method Contrastive Successor Features (CSF) [71] uses uniformly drawn skills on the hypersphere to learn a skill-conditioned policy by maximizing skill diversity via (3), effectively covering S d−1 (spanning R d ). CSF learns an encoder ϕ to map obser￾vations o to features ϕ(o), and uses an inner… view at source ↗
Figure 2
Figure 2. Implications of optimizing different mutual information objectives I(s, s′ ; zi) vs. I(s; zi): the reward function and the feature learning objective impose different inductive biases on the structure of the latent space. Left: using feature ϕ(s ′ ) − ϕ(o) to parametrize I(s, s′ ; zi) ensures that the embeddings of consecutive states are close but distinct. Right: optimizing I(s; z) does not impose a “locality” cons… view at source ↗
Figure 3
Figure 3. CSF identifies the underlying states in MuJoCo and DMC up to a linear transformation. Top: Identifiability of both features ϕ(o) and feature differences ϕ(o ′ ) − ϕ(o), measured by the R2 score (higher is better); Middle: state coverage, indicating exploratory behavior; Bottom: oracle return indicating zero-shot task transfer performance. Error bars represent one standard deviation. 4 Experiments Setup. We use the c… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The effect of latent space dimen￾sionality on state identifiability and zero-shot task transfer in the Ant environment. Lin￾ear identifiability requires that the feature space has at least as many dimensions as the true state (cf. Assum. 1(v)). In contrast, a smaller l…
Figure 6
Figure 6. Figure 6: The effect of skill diversity on state identifiability and zero-shot skill transfer in the Ant environment. Skills are sampled from p(z) at the start of pretraining and kept fixed throughout, except in the ‘sampling’ case where skills are redrawn from p(z) during train…
Figure 7
Figure 7. Figure 7: The effect of latent space dimensionality on state identifiability and coverage in the Ant environment. Linear identifiability requires that the feature space has at least as many dimensions as the true state (cf. Assum. 1(v)). D Acronyms ELBO evidence lower bound CL C…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization

    cs.LG 2026-05 unverdicted novelty 8.0 of 10

    GCRL and MISL are unified as control maximization, with three inequivalent GCRL formulations each matched to a MISL objective via bounds on goal-sensitivity.

Reference graph

Works this paper leans on

77 extracted references · 44 canonical work pages · cited by 1 Pith paper

  1. [49]

    V ogt, Randall Balestriero, Wieland Brendel, and David Klindt

    Patrik Reizinger, Alice Bizeul, Attila Juhos, Julia E. V ogt, Randall Balestriero, Wieland Brendel, and David Klindt. Cross-Entropy Is All You Need To Invert the Data Generating Process. October 2024. URL https://openreview.net/forum?id=hrqNOxpItr. 1, 2, 3, 4, 5, 6, 9, 15

    work page 2024

  2. [71]

    CausalV AE: Structured Causal Disentanglement in Variational Autoencoder, December 2023

    Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. CausalV AE: Structured Causal Disentanglement in Variational Autoencoder, December 2023. URL http: //arxiv.org/abs/2004.08697. arXiv:2004.08697 [cs]. 4

    arXiv 2023

  3. [1]

    Variational option discovery algorithms

    Joshua Achiam, Harrison Edwards, Dario Amodei, and Pieter Abbeel. Variational option discovery algorithms. arXiv preprint arXiv:1807.10299, 2018. 1, 3

    arXiv 2018

  4. [2]

    A Probabilistic Model Behind Self-Supervised Learning

    Alice Bizeul, Bernhard Schölkopf, and Carl Allen. A Probabilistic Model to explain Self- Supervised Representation Learning, February 2024. URL http://arxiv.org/abs/2402. 01399. arXiv:2402.01399 [cs, stat]. 3

    work page Pith review arXiv 2024

  5. [3]

    Large-Scale Study of Curiosity-Driven Learning

    Yuri Burda, Harri Edwards, Deepak Pathak, Amos Storkey, Trevor Darrell, and Alexei A Efros. Large-Scale Study of Curiosity-Driven Learning. page 15. 3

    work page

  6. [4]

    A Simple Framework for Contrastive Learning of Visual Representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. arXiv:2002.05709 [cs, stat], June 2020. URL http://arxiv.org/abs/2002.05709. arXiv: 2002.05709. 3

    arXiv 2002

  7. [5]

    Varia- tional Empowerment as Representation Learning for Goal-Conditioned Reinforcement Learning

    Jongwook Choi, Archit Sharma, Honglak Lee, Sergey Levine, and Shixiang Shane Gu. Varia- tional Empowerment as Representation Learning for Goal-Conditioned Reinforcement Learning. In Proceedings of the 38th International Conference on Machine Learning, pages 1953–1963. PMLR, July 2021. URL https://proceedings.mlr.press/v139/choi21b.html. ISSN: 2640-3498. 1

    work page 1953

  8. [6]

    Analyse des liaisons de probabilité

    George Darmois. Analyse des liaisons de probabilité. In Proc. Int. Stat. Conferences 1947, page 231, 1951. 3

    work page 1947

Show all 77 references
  1. [7]

    A Survey of State Representation Learning for Deep Reinforcement Learning

    Ayoub Echchahed and Pablo Samuel Castro. A Survey of State Representation Learning for Deep Reinforcement Learning. Transactions on Machine Learning Research, March 2025. ISSN 2835-8856. URL https://openreview.net/forum?id=gOk34vUHtz. 3

    2025

  2. [8]

    Diversity is All You Need: Learning Skills without a Reward Function, October 2018

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is All You Need: Learning Skills without a Reward Function, October 2018. URL guodefinetti. arXiv:1802.06070 [cs]. 1, 3, 4, 5

    2018 arXiv

  3. [9]

    Salakhutdi- nov

    Benjamin Eysenbach, Tianjun Zhang, Sergey Levine, and Russ R. Salakhutdi- nov. Contrastive Learning as Goal-Conditioned Reinforcement Learning. Ad- vances in Neural Information Processing Systems , 35:35603–35620, December

  4. [10]

    Contrastive Representations Make Planning Easy

    Benjamin Eysenbach, Vivek Myers, Sergey Levine, and Ruslan Salakhutdinov. Contrastive Representations Make Planning Easy. December 2023. URL https://openreview.net/ forum?id=W0bhHvQK60. 3, 4

    2023

  5. [11]

    Wichmann

    Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, November 2020. ISSN 2522-5839. doi: 10.1038/ s42256-020-00257...

    2020

  6. [12]

    Variational Intrinsic Control

    Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational Intrinsic Control. February 2017. URL https://openreview.net/forum?id=Skc-Fo4Yg. 1, 3

    2017

  7. [13]

    Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Schölkopf

    Luigi Gresele, Paul K. Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Schölkopf. The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA. arXiv:1905.06642 [cs, stat], August 2019. URL http://arxiv.org/abs/ 1905.06642. arXiv: 190...

    1905 arXiv

  8. [14]

    World Models

    David Ha and Jürgen Schmidhuber. World Models. March 2018. doi: 10.5281/zenodo.1207631. URL http://arxiv.org/abs/1803.10122. arXiv:1803.10122 [cs, stat]. 1 10

    2018 arXiv

  9. [15]

    Fast Task Inference with Variational Intrinsic Successor Features

    Steven Hansen, Will Dabney, Andre Barreto, David Warde-Farley, Tom Van de Wiele, and V olodymyr Mnih. Fast Task Inference with Variational Intrinsic Successor Features. September

  10. [16]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020. 4

    2020

  11. [17]

    Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA

    Aapo Hyvarinen and Hiroshi Morioka. Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA. arXiv:1605.06336 [cs, stat], May 2016. URL http://arxiv. org/abs/1605.06336. arXiv: 1605.06336. 3, 4

    2016 arXiv

  12. [18]

    Aapo Hyvarinen, Hiroaki Sasaki, and Richard E. Turner. Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning. arXiv:1805.08651 [cs, stat], February 2019. URL http://arxiv.org/abs/1805.08651. arXiv: 1805.08651. 1, 3, 4, 8

    2019 arXiv

  13. [19]

    Nonlinear independent component analysis: Existence and uniqueness results

    Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks, 12(3):429–439, April 1999. ISSN 0893-6080. doi: 10.1016/S0893-6080(98)00140-3. URL https://www.sciencedirect.com/science/ article/pii/S0893608098001...

    1999 doi

  14. [20]

    Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

    Hermanni Hälvä, Sylvain Le Corff, Luc Lehéricy, Jonathan So, Yongjie Zhu, Elisabeth Gassiat, and Aapo Hyvarinen. Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA. arXiv:2106.09620 [cs, stat] , June 2021. URL http://arxiv.org/abs/ 2106.09620. ar...

    2021 arXiv

  15. [21]

    Occam’s Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?, June 2024

    Mark Ibrahim, David Klindt, and Randall Balestriero. Occam’s Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?, June 2024. URL http://arxiv. org/abs/2406.10743. arXiv:2406.10743 [cs]. 4, 5, 9

    2024 arXiv

  16. [22]

    Variational Autoencoders and Nonlinear ICA: A Unifying Framework

    Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational Autoencoders and Nonlinear ICA: A Unifying Framework. In International Conference on Artificial Intelligence and Statistics , pages 2207–2217. PMLR, June 2020. URL http: //proceedings.mlr.press/v1...

    2020

  17. [23]

    Kingma, and Aapo Hyvärinen

    Ilyes Khemakhem, Ricardo Pio Monti, Diederik P. Kingma, and Aapo Hyvärinen. ICE- BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA. arXiv:2002.11537 [cs, stat] , October 2020. URL http://arxiv.org/abs/2002.11537. arXiv: 2002.11537. 3

    2002 arXiv

  18. [24]

    Unsupervised Reinforcement Learning with Contrastive Intrinsic Con- trol

    Michael Laskin, Hao Liu, Xue Bin Peng, Denis Yarats, Aravind Rajeswaran, and Pieter Abbeel. Unsupervised Reinforcement Learning with Contrastive Intrinsic Con- trol. Advances in Neural Information Processing Systems , 35:34478–34491, Decem- ber 2022. URL https://proceedings.ne...

    2022

  19. [25]

    Asano, Taco Cohen, and Efstratios Gavves

    Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M. Asano, Taco Cohen, and Efstratios Gavves. CITRIS: Causal Identifiability from Temporal Intervened Sequences, June 2022. URL http://arxiv.org/abs/2202.03169. Number: arXiv:2202.03169 arXiv:2202.03169 [cs, stat]. 4

    2022 arXiv

  20. [26]

    Asano, Taco Cohen, and Efstratios Gavves

    Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M. Asano, Taco Cohen, and Efstratios Gavves. iCITRIS: Causal Representation Learning for Instantaneous Temporal Effects. July

  21. [27]

    A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals, August 2024

    Grace Liu, Michael Tang, and Benjamin Eysenbach. A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals, August 2024. URL http://arxiv.org/abs/2408.05804. arXiv:2408.05804 [cs]. 1

    2024 arXiv

  22. [28]

    Self-Supervised Learning via Maximum Entropy Coding

    Xin Liu, Zhongdao Wang, Yali Li, and Shengjin Wang. Self-Supervised Learning via Maximum Entropy Coding. 2022. doi: 10.48550/ARXIV .2210.11464. URL https://arxiv.org/abs/ 2210.11464. Publisher: arXiv Version Number: 1. 3 11

    2022 doi arXiv

  23. [29]

    URL https://openreview.net/forum?id=xeDKTZsZ7Z7. 4

  24. [30]

    Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

    Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations. In International Conference on Machine Learn- ing, pages 4114–412...

    2019

  25. [31]

    Weakly-Supervised Disentanglement Without Compromises

    Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. Weakly-Supervised Disentanglement Without Compromises. arXiv:2002.02886 [cs, stat] , October 2020. URL http://arxiv.org/abs/2002.02886. arXiv: 2002.02886. 3

    2002 arXiv

  26. [32]

    Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning, January 2023

    Yuejiang Liu, Alexandre Alahi, Chris Russell, Max Horn, Dominik Zietlow, Bernhard Schölkopf, and Francesco Locatello. Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning, January 2023. URL http://arxiv.org/abs/2301.05169. arXiv:2301.05169 [cs]. 4

    2023 arXiv

  27. [33]

    Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning

    Shakir Mohamed and Danilo Jimenez Rezende. Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https://proceedings.neurips. cc/paper/2015/ha...

    2015

  28. [34]

    Connectivity-contrastive learning: Combining causal discovery and representation learning for multimodal data

    Hiroshi Morioka and Aapo Hyvarinen. Connectivity-contrastive learning: Combining causal discovery and representation learning for multimodal data. In Proceedings of The 26th Interna- tional Conference on Artificial Intelligence and Statistics, pages 3399–3426. PMLR, April 2023...

    2023

  29. [35]

    ALAN: Autonomously Exploring Robotic Agents in the Real World, February 2023

    Russell Mendonca, Shikhar Bahl, and Deepak Pathak. ALAN: Autonomously Exploring Robotic Agents in the Real World, February 2023. URL http://arxiv.org/abs/2302.06604. arXiv:2302.06604 [cs, eess]. 3

    2023 arXiv

  30. [36]

    Representation Learning with Contrastive Predictive Coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748 [cs, stat], January 2019. URL http://arxiv.org/ abs/1807.03748. arXiv: 1807.03748. 4

    2019 arXiv

  31. [37]

    The Linear Representation Hypothesis and the Geometry of Large Language Models, November 2023

    Kiho Park, Yo Joong Choe, and Victor Veitch. The Linear Representation Hypothesis and the Geometry of Large Language Models, November 2023. URL http://arxiv.org/abs/2311. 03658. arXiv:2311.03658 [cs, stat]. 2

    2023 arXiv

  32. [38]

    Independent Innovation Analysis for Nonlinear Vector Autoregressive Process

    Hiroshi Morioka, Hermanni Hälvä, and Aapo Hyvärinen. Independent Innovation Analysis for Nonlinear Vector Autoregressive Process. arXiv:2006.10944 [cs, stat], February 2021. URL https://arxiv.org/abs/2006.10944. arXiv: 2006.10944. 3

    2006 arXiv

  33. [39]

    Lipschitz- constrained Unsupervised Skill Discovery

    Seohong Park, Jongwook Choi, Jaekyeom Kim, Honglak Lee, and Gunhee Kim. Lipschitz- constrained Unsupervised Skill Discovery. October 2021. URL https://openreview.net/ forum?id=BGvt0ghNgA. 3, 5

    2021

  34. [40]

    Lipschitz- constrained unsupervised skill discovery

    Seohong Park, Jongwook Choi, Jaekyeom Kim, Honglak Lee, and Gunhee Kim. Lipschitz- constrained unsupervised skill discovery. In International Conference on Learning Representa- tions, 2022. 3

    2022

  35. [41]

    The Geometry of Categorical and Hierarchical Concepts in Large Language Models, June 2024

    Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. The Geometry of Categorical and Hierarchical Concepts in Large Language Models, June 2024. URL http://arxiv.org/abs/ 2406.01506. arXiv:2406.01506 [cs, stat]. 1

    2024 arXiv

  36. [42]

    METRA: Scalable Unsupervised RL with Metric-Aware Abstraction, March 2024

    Seohong Park, Oleh Rybkin, and Sergey Levine. METRA: Scalable Unsupervised RL with Metric-Aware Abstraction, March 2024. URL http://arxiv.org/abs/2310.08887. arXiv:2310.08887. 1, 3, 4, 5 12

    2024 arXiv

  37. [43]

    Pardalos

    Seonho Park and Panos M. Pardalos. Deep Data Density Estimation through Donsker-Varadhan Representation. arXiv:2104.06612 [cs, math], April 2021. URL http://arxiv.org/abs/ 2104.06612. arXiv: 2104.06612. 1

    2021 arXiv

  38. [44]

    Controllability-Aware Unsupervised Skill Discovery, June 2023

    Seohong Park, Kimin Lee, Youngwoon Lee, and Pieter Abbeel. Controllability-Aware Unsupervised Skill Discovery, June 2023. URL http://arxiv.org/abs/2302.05103. arXiv:2302.05103. 4, 5

    2023 arXiv

  39. [45]

    Efros, and Trevor Darrell

    Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-Driven Exploration by Self-Supervised Prediction. In 2017 IEEE Conference on Computer Vi- sion and Pattern Recognition Workshops (CVPRW) , pages 488–489, Honolulu, HI, USA, July 2017. IEEE. ISBN 978-...

    2017

  40. [46]

    Causality: Models, Reasoning, and Inference

    Judea Pearl. Causality: Models, Reasoning, and Inference . Cambridge University Press, Cambridge, 2 edition, 2009. ISBN 978-0-511-80316-1. doi: 10.1017/CBO9780511803161. URL http://ebooks.cambridge.org/ref/id/CBO9780511803161. 4

    2009 doi

  41. [47]

    Self-Supervised Exploration via Disagree- ment

    Deepak Pathak, Dhiraj Gandhi, and Abhinav Gupta. Self-Supervised Exploration via Disagree- ment. page 10. 1, 3, 4

  42. [48]

    Jacobian-based Causal Discovery with Nonlinear ICA

    Patrik Reizinger, Yash Sharma, Matthias Bethge, Bernhard Schölkopf, Ferenc Huszár, and Wieland Brendel. Jacobian-based Causal Discovery with Nonlinear ICA. Transactions on Machine Learning Research, April 2023. ISSN 2835-8856. URL https://openreview.net/ forum?id=2Yo9xqR6Ab. 4

    2023

  43. [50]

    An Inter- ventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysis, November 2023

    Goutham Rajendran, Patrik Reizinger, Wieland Brendel, and Pradeep Ravikumar. An Inter- ventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysis, November 2023. URL http://arxiv.org/abs/2311.18048. arXiv:2311.18048 [cs, eess, stat]. 1...

    2023 arXiv

  44. [51]

    Geoffrey Roeder, Luke Metz, and Diederik P. Kingma. On Linear Identifiability of Learned Representations. arXiv:2007.00810 [cs, stat] , July 2020. URL http://arxiv.org/abs/ 2007.00810. arXiv: 2007.00810. 1, 2, 3, 4

    2007 arXiv

  45. [52]

    Zimmermann, and Wieland Brendel

    Evgenia Rusak, Patrik Reizinger, Attila Juhos, Oliver Bringmann, Roland S. Zimmermann, and Wieland Brendel. InfoNCE: Identifying the Gap Between Theory and Practice, June 2024. URL http://arxiv.org/abs/2407.00143. arXiv:2407.00143 [cs, stat]. 3, 4, 5

    2024 arXiv

  46. [53]

    Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning

    Patrik Reizinger, Siyuan Guo, Ferenc Huszár, Bernhard Schölkopf, and Wieland Brendel. Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning. October 2024. URL https://openreview.net/forum?id=k03mB41vyM. 1, 5, 8

    2024

  47. [54]

    Towards Causal Representation Learning

    Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalch- brenner, Anirudh Goyal, and Yoshua Bengio. Towards Causal Representation Learning. arXiv:2102.11107 [cs], February 2021. URL http://arxiv.org/abs/2102.11107. arXiv: 2102.11107 version: 1. 1, 4

    2021 arXiv

  48. [55]

    Planning to Explore via Self-Supervised World Models

    Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, and Deepak Pathak. Planning to Explore via Self-Supervised World Models. arXiv:2005.05960 [cs, stat], June 2020. URL http://arxiv.org/abs/2005.05960. arXiv: 2005.05960. 3

    2005 arXiv

  49. [56]

    Regularity as Intrinsic Reward for Free Play

    Cansu Sancaktar, Justus Piater, and Georg Martius. Regularity as Intrinsic Reward for Free Play. Advances in Neural Information Processing Systems , 36, Decem- ber 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/ c529dba08a146ea8d6cf715ae8930cbe-Abstract-C...

    2023

  50. [57]

    What Do We Maximize in Self-Supervised Learning?, July 2022

    Ravid Shwartz-Ziv, Randall Balestriero, and Yann LeCun. What Do We Maximize in Self-Supervised Learning?, July 2022. URL http://arxiv.org/abs/2207.10081. arXiv:2207.10081 [cs]. 3

    2022 arXiv

  51. [58]

    MIT press, 2000

    Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman.Causation, prediction, and search. MIT press, 2000. 4

    2000

  52. [59]

    Dynamics- Aware Unsupervised Discovery of Skills, February 2020

    Archit Sharma, Shixiang Gu, Sergey Levine, Vikash Kumar, and Karol Hausman. Dynamics- Aware Unsupervised Discovery of Skills, February 2020. URL http://arxiv.org/abs/ 1907.01657. arXiv:1907.01657 [cs, stat]. 1, 3, 4, 5 13

    2020 arXiv

  53. [60]

    Optimistic Active Exploration of Dynamical Systems

    Lenart Treven, Cansu Sancaktar, Sebastian Blaes, Stelian Coros, and Andreas Krause. Optimistic Active Exploration of Dynamical Systems. Advances in Neural Information Processing Systems, 36, December 2023. URL https://proceedings.neurips.cc/paper_files/paper/ 2023/hash/77b5aaf...

    2023

  54. [61]

    Rubenstein, Sylvain Gelly, and Mario Lucic

    Michael Tschannen, Josip Djolonga, Paul K. Rubenstein, Sylvain Gelly, and Mario Lucic. On Mutual Information Maximization for Representation Learning, January 2020. URL http: //arxiv.org/abs/1907.13625. arXiv:1907.13625 [cs]. 2

    2020 arXiv

  55. [62]

    Reinforcement learning: An introduction, volume 1

    Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998. 3

    1998

  56. [63]

    Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap, May 2022

    Yifei Wang, Qi Zhang, Yisen Wang, Jiansheng Yang, and Zhouchen Lin. Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap, May 2022. URL http://arxiv.org/abs/2203.13457. arXiv:2203.13457 [cs, stat]. 6

    2022 arXiv

  57. [64]

    Unsupervised control through non-parametric discriminative rewards.arXiv preprint arXiv:1811.11359, 2018

    David Warde-Farley, Tom Van de Wiele, Tejas Kulkarni, Catalin Ionescu, Steven Hansen, and V olodymyr Mnih. Unsupervised control through non-parametric discriminative rewards.arXiv preprint arXiv:1811.11359, 2018. 3

    2018 arXiv

  58. [65]

    Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

    Tongzhou Wang and Phillip Isola. Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere. arXiv:2005.10242 [cs, stat], November 2020. URL http://arxiv.org/abs/2005.10242. arXiv: 2005.10242. 3

    2005 arXiv

  59. [66]

    Correlation and causation

    Sewall Wright. Correlation and causation. Journal of Agricultural Research, (7), 1921. 8

    1921

  60. [67]

    Yu, and Dahua Lin

    Zhirong Wu, Yuanjun Xiong, Stella X. Yu, and Dahua Lin. Unsupervised Feature Learning via Non-Parametric Instance Discrimination. pages 3733–3742, 2018. URL https://openaccess.thecvf.com/content_cvpr_2018/html/Wu_Unsupervised_ Feature_Learning_CVPR_2018_paper.html. 4

    2018

  61. [68]

    Causal Component Analysis, October 2023

    Liang Wendong, Armin Keki ´c, Julius von Kügelgen, Simon Buchholz, Michel Besserve, Luigi Gresele, and Bernhard Schölkopf. Causal Component Analysis, October 2023. URL http://arxiv.org/abs/2305.17225. arXiv:2305.17225 [cs, stat]. 1, 3, 8

    2023 arXiv

  62. [69]

    Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning, June 2025

    Yucheng Yang, Tianyi Zhou, Qiang He, Lei Han, Mykola Pechenizkiy, and Meng Fang. Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning, June 2025. URL http://arxiv.org/abs/2506. 10629. arXiv:2506.10629 [cs]. 3

    2025 arXiv

  63. [70]

    Contrastive Difference Predic- tive Coding, October 2023

    Chongyi Zheng, Ruslan Salakhutdinov, and Benjamin Eysenbach. Contrastive Difference Predic- tive Coding, October 2023. URL http://arxiv.org/abs/2310.20141. arXiv:2310.20141 [cs]. 3, 4

    2023

  64. [72]

    Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel

    Roland S. Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive Learning Inverts the Data Generating Process. arXiv:2102.08850 [cs], February 2021. URL http://arxiv.org/abs/2102.08850. arXiv: 2102.08850. 1, 3, 4, 5 14 A Impact Statement ...

    2021 arXiv

  65. [74]

    Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning, December 2024

    Chongyi Zheng, Jens Tuyls, Joanne Peng, and Benjamin Eysenbach. Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning, December 2024. URL http://arxiv. org/abs/2412.08021. arXiv:2412.08021 [cs]. 1, 2, 3, 5, 6, 8, 9, 15

    2024

  66. [76]

    for any i ̸= j the system {zi − zj} is now a generator system of Rd

  67. [77]

    the invertible linear image of an affine generator system is also an affine generator system. C Experimental Details and Further Results C.1 Compute Resources All experiments were run in a compute cluster using an Intel Xeon Gold CPU (16 cores, 2.9 GHz) and NVIDIA RTX 2080 Ti ...

  68. [2019]

    URL https://openreview.net/forum?id=BJeAHkrYDS. 1, 4, 5

  69. [2022]

    1, 3, 4, 5, 9

    URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/ e7663e974c4ee7a2b475a4775201ce1f-Abstract-Conference.html . 1, 3, 4, 5, 9

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.