Pith. sign in

REVIEW 3 major objections 6 minor 50 references

Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing the VAE's rotationally invariant Gaussian prior with an $L^p$-nested subspace prior removes rotational unidentifiability and yields better disentanglement at comparable reconstruction quality than $\beta$-VAE and $\beta$-TCVAE.

desk verdict Lp-nested ISA prior is a genuine step for disentangled VAEs and the empirical trade-off gains look real, but 'resolves unidentifiability' is an overclaim and the near-Gaussian exponents need sensitivity analysis. read the letter →

arxiv 1909.05063 v1 pith:X7RDQJ4M submitted 2019-09-05 stat.ML cs.LG

Jan Stühmer , Richard E. Turner , Sebastian Nowozin This is my paper · ORCID
classification stat.MLcs.LG
keywords variationalautoencoderdisentangledrepresentationlearningindependentsubspaceanalysisLp-nestedsymmetricdistributionsstructuredlatentpriorsidentifiabilitybeta-VAEmutualinformationgap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most work on disentangling variational autoencoders reweights the objective, as in $\beta$-VAE; this paper argues that such reweighting works by amplifying known variational-inference biases---over-pruning latent components and over-orthogonalizing their effects---and so inevitably trades reconstruction quality for disentanglement. The alternative it proposes is to change the model rather than the objective: use an $L^p$-nested symmetric prior that divides the latent space into independent subspaces, and whose density is not rotationally invariant. The paper claims that this prior removes the rotational unidentifiability of the standard normal prior, that it supports disentangled representations even with the unmodified ELBO ($\beta=1$), and that it shifts the disentanglement-versus-reconstruction trade-off compared with $\beta$-VAE and $\beta$-TCVAE on dSprites, 3D Faces, and cars3d.

What carries the argument

The load-bearing object is the $L^p$-nested symmetric distribution used as the latent prior, in its depth-two independent-subspace-analysis form: the density is a radial function of an outer $L^{p_0}$ norm whose arguments are inner $L^{p_1}$ norms over groups of latent coordinates, making the groups independent of one another while leaving coordinates inside a group coupled through a shared radius. Its log-density has a closed form (a radial density divided by a surface-area factor built from gamma functions), so the KL term of the ELBO can be estimated by Monte Carlo and the reparameterization trick is untouched; exact sampling from the prior is also available via a $\beta$-Dirichlet algorithm, which the paper needs for generation. The decisive property is asymmetry: the prior is not rotationally invariant, breaking the rotational degeneracy of the Gaussian prior, and the exponents $p_0$ and $p_1$ control whether the isocontours are axis-aligned (platykurtic) or diagonal (leptokurtic), which is how the prior expresses a preference for how a factor such as a sprite's $(x,y)$ position is encoded.

What would settle it

Train ISA-VAE on a synthetic dataset whose factors are known, rotate the data (or the factor axes) by 45 degrees, and retrain: if the learned latent axes rotate with the data and the MIG score is unchanged, the prior imposes only a soft coordinate preference rather than an identifiable axis structure. A second check is to set $p_1$ equal to $p_0$ so the prior is fully factorized within subspaces and see whether the disentanglement advantage disappears.

Watch

Extended reading notes

Core claim

The central claim is that the standard VAE's isotropic Gaussian prior is the real obstacle to disentanglement. Because that prior is rotationally invariant, any rotation of the latent space can be absorbed into the decoder's mapping, so the latent axes are statistically indistinguishable and no amount of objective reweighting can identify them. The paper's proposal is to place an $L^p$-nested symmetric distribution, specialized to an independent subspace analysis (ISA) layout, over the latents: the prior density is a radial function of a cascade of $L^p$ norms, so the latent vector splits into a hierarchy of independent vector spaces while coordinates within a subspace share a common radius, and the isocontours are not rotationally symmetric. With this prior, the unmodified ELBO at $\beta=1$ already achieves a higher mutual information gap (MIG) than the standard VAE, and across $\beta$ values ISA-VAE reaches higher MIG at comparable or better reconstruction quality than $\beta$-VAE and $\beta$-TCVAE; learning the per-subspace exponents during training improves the trade-off further.

Load-bearing premise

The claimed advantage depends on setting the ISA layout and the exponents $p_0$ and $p_1$ close to the true factor structure, and the paper gives no identifiability proof; it fixes $p_0=2.1$ and chooses $p_1$ per dataset with knowledge of the true factors (Section 4), and because $p_0$ is near 2, the prior remains nearly rotationally invariant inside each subspace, so the symmetry-breaking may be weaker than claimed.

Editorial extensions

If this is right

  • ISA-VAE with the unmodified ELBO ($\beta=1$) achieves higher MIG scores than the standard VAE, so disentanglement does not require reweighting the objective when the prior is structured.
  • On dSprites and 3D Faces, baseline methods reach their best MIG scores only at poor reconstruction quality, whereas ISA-VAE reaches comparable or higher MIG at lower $\beta$, preserving reconstruction quality.
  • Learning the exponents $p_1,\ldots,p_k$ during training, starting from 2.0 with $p_0$ fixed at 2.1, improves the disentanglement-reconstruction trade-off further, with learned exponents remaining slightly above 2.0 and distinct from $p_0$.
  • Because the prior's log-density is available in closed form and the encoder remains a diagonal Gaussian, the method can be dropped into an existing VAE implementation by changing only the KL term.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's identifiability argument leans on the prior being non-factorized, but it does not prove identifiability; a natural extension would be to test whether the learned subspace partition is invariant across random seeds and across different ISA layouts of the same total dimension.
  • If true factors are genuinely low-dimensional subspaces rather than single coordinates, standard one-latent-per-factor metrics like MIG may understate what ISA-VAE learns; a two-dimensional factor such as $(x,y)$ position would live inside one subspace, suggesting a subspace-level disentanglement metric.
  • The same tree-of-norms construction extends naturally to deeper hierarchies, which could represent factors that are themselves hierarchically organized; a testable extension is whether a three-level prior matches datasets with grouped factors better than the depth-two ISA layout.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper argues that the standard normal prior's rotational invariance makes VAE latent representations unidentifiable, and that objective-modification approaches such as beta-VAE only amplify variational inference biases (orthogonalization and pruning). As an alternative, the authors propose ISA-VAE, which uses an Lp-nested symmetric prior (Eqs. 7-11) to define independent subspaces in latent space. They claim this non-factorized, non-rotationally-invariant prior resolves the unidentifiability problem, facilitates disentanglement even at beta=1, and improves the disentanglement-reconstruction trade-off relative to beta-VAE and beta-TCVAE. The empirical evaluation on dSprites, 3D Faces, and cars3d reports MIG and reconstruction quality, with 50 runs per setting on the first two datasets, and additionally explores learning the exponents during training.

Significance. If established, the result would be valuable: it offers a principled alternative to objective-function modification, provides exact sampling and density evaluation for a flexible family of priors, and includes instructive toy analyses of variational biases in linear models (Appendix B). The experimental protocol is extensive for its time (50 runs per setting with standard errors), and the idea of using non-Gaussian structured priors to break latent-space symmetry is worth pursuing. However, the load-bearing claims of resolving unidentifiability and achieving unsupervised gains require stronger support: the paper does not release code, does not report significance tests, and relies on dataset-informed exponent choices. The conceptual contribution is therefore promising but not yet fully substantiated.

major comments (3)
  1. [Sections 2.2 and 3.2; Eq. (9)] The claim that the ISA prior 'resolves the problem of unidentifiability of the standard VAE normal prior' is not established. Section 2.2 only notes that the Locatello et al. impossibility proof assumes a factorized prior, which the ISA prior is not; that observation alone does not establish identifiability of the individual latent factors. In Eq. (9), the density depends on the coordinates within each subspace only through the L_p1 norm of that subspace, and with the values used in Section 4 (p1 = 2.2 on dSprites, p1 = 1.9 on 3D Faces) and the learned exponents in Fig. 7 (1.95-2.17), this within-subspace density is nearly L2-spherically symmetric. Consequently, rotations inside a subspace are almost unobservable in the prior, and no proof or quantitative sensitivity analysis is provided to show that the small anisotropy is sufficient to identify individual latent dimensions rather than only the subspace up to rotation. This is central to the paper's first advertised advantage.
  2. [Section 4, 'Choosing the Exponents'] The reported MIG advantage is entangled with supervised selection of the prior exponents. The paper chooses p0 = 2.1 globally and then fixes p1 = 2.2 for dSprites because a platykurtic distribution fits the known square-shaped (x,y) distribution, and p1 = 1.9 for 3D Faces because it 'provided better results.' Since the true generative factors are known in these synthetic benchmarks, this is dataset-informed tuning of the prior shape, and the unsupervised claim cannot be separated from that tuning without a sensitivity analysis. I ask for a systematic study of performance as p0 and p1 vary (including values farther from 2), an ablation in which the exponents are set without using the known factors (e.g., from a validation split or a fixed default), and a report of the variance across seeds for each exponent setting.
  3. [Section 4 and Figures 5-6] The wording 'significantly mitigates the trade-off' is not backed by statistical tests. The paper reports standard errors over 50 runs per setting on dSprites and 3D Faces (and 15 runs for the learned-exponent experiments in Appendix A.2), but no pairwise significance tests are given, and the number of runs is not stated for every reported scatter point. Given that the main empirical conclusion is a comparative improvement, the authors should report confidence intervals or tests (e.g., bootstrap or permutation) for the MIG differences at matched beta, and should release the code and exact run counts to allow independent verification.
minor comments (6)
  1. [Section 4] The manuscript states that source code will be made available, but no repository link is provided; please include a public URL with the exact training and evaluation scripts.
  2. [Section 3.2, Eq. (11)] The symbol n in the radial density psi_0(v_0) is not explicitly defined in the main text; it should be identified as the dimension of z (or of the root subspace) when Eq. (11) is introduced, and the scale parameter s should also be defined.
  3. [Figure 5 caption] The caption in Fig. 5(c) contains the typo 'unmodefied' for 'unmodified', and the number of experiments per beta for the learned-exponent runs is only given in Appendix A.2; the main text should state these sample sizes where the scatter plots are first discussed.
  4. [Section 4.1] The claim of higher mean, median and maximum quantile in Fig. 5a is made verbally; please provide a small table with the exact MIG quantiles for VAE and ISA-VAE to support reproducibility.
  5. [Appendix B] The factor-analysis experiments in B.3-B.5 show that beta does not change the inferred directions in the linear model, while the ICA experiment in B.8 shows beta=5 prunes a component; the manuscript could state more explicitly that these two results together imply that beta has different effects in different regimes, which is exactly why the toy analyses are not by themselves a full explanation of beta-VAE behaviour.
  6. [Section 3.2] The text uses 'Kulback-Leibler' instead of 'Kullback-Leibler' in the ISA-VAE paragraph; please correct the typo.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the ISA-VAE claims are empirical comparisons plus an asserted symmetry argument, not results forced by construction or by self-citation.

full rationale

This paper does not contain a derivation chain that reduces to its own inputs. The central empirical claims—higher MIG scores and an improved disentanglement/reconstruction trade-off—are obtained by running ISA-VAE, beta-VAE, and beta-TCVAE under matched hyperparameters and measuring the resulting MIG and reconstruction losses; they are measured outcomes, not quantities that are equal to the prior or to the chosen exponents by construction. The only potentially dataset-informed choice is the per-dataset exponent selection in Section 4 ('Choosing the Exponents'), where p1 = 2.2 is chosen for dSprites and p1 = 1.9 for 3D faces with knowledge of the true factor distributions. However, the paper does not present the resulting MIG scores as 'predictions' from a fitted parameter, so this is a benchmarking/external-validity concern rather than a circularity. The identifiability claim is asserted from the non-rotational invariance of the Lp-nested prior and from the observation that the Locatello et al. impossibility proof requires a factorized prior; this is an external theorem, not a self-citation, and the non-factorization of the ISA prior is stated explicitly. No load-bearing self-citations appear: the Turner and Sahani reference supports the VI-bias discussion but is not used to derive the main result. No equation-level equivalence, fitted-parameter-renamed-as-prediction, or uniqueness result imported from the authors' own prior work was found. Accordingly, the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new entities are postulated. The prior is a known family from ICA literature. The core of the contribution is empirical, so the ledger shows mostly hyperparameters and the assumption that the data factors are independent/groupable.

free parameters (4)
  • p0 (outer norm exponent) = 2.1 (fixed)
    Hand-chosen to yield a platykurtic distribution over subspaces; Section 4 'Choosing the Exponents'.
  • p1 (subspace norm exponent) = 2.2 (dSprites), 1.9 (3D faces); learned from 2.0 in learned-exponents variant
    Chosen per dataset to match the distribution of generative factors; also learned during training.
  • ISA layout (number and sizes of subspaces) = l0=5, l_i=4 or 5 across figures
    Hand-designed to provide enough independent subspaces for the dataset's factors; Section 4 'Choosing the ISA-layout'.
  • beta (regularizer weight) = swept from 0.5 to 6.0
    Shared with baselines; central comparison is across this sweep.
assumptions (5)
  • standard math Lp-nested symmetric distributions are valid densities with the given surface area formula (Sinz & Bethge 2010).
    Used to define the ISA prior in Eqs. 7-9.
  • domain assumption The Locatello et al. impossibility result for unsupervised disentanglement applies only to priors that factorize over every latent dimension.
    Invoked in Section 2.2 to argue the proposed non-factorized prior can escape unidentifiability; this is an interpretation of the cited theorem, not a proof for this model.
  • domain assumption Mutual Information Gap (MIG) is a valid and sufficient measure of disentanglement.
    MIG from Chen et al. [6] is used to compare methods; validity of MIG as the target metric is assumed.
  • domain assumption Monte Carlo estimation of the KL with a fixed Gaussian encoder and this prior is unbiased and low enough variance for training.
    Used in Eq. 12 and in training; no variance analysis is provided.
  • ad hoc to paper The true generative factors are independent or groupable into independent subspaces.
    The whole disentanglement objective assumes independent factors; the ISA prior further assumes a subspace grouping, which is a modeling assumption about the data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations." pith.science (2026). https://pith.science/paper/X7RDQJ4M

@misc{pith2026190905063,
  author       = {Pith},
  title        = {Pith review of: Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X7RDQJ4M}},
  note         = {Machine review of arXiv:1909.05063}
}
read the original abstract

Recently there has been an increased interest in unsupervised learning of disentangled representations using the Variational Autoencoder (VAE) framework. Most of the existing work has focused largely on modifying the variational cost function to achieve this goal. We first show that these modifications, e.g. beta-VAE, simplify the tendency of variational inference to underfit causing pathological over-pruning and over-orthogonalization of learned components. Second we propose a complementary approach: to modify the probabilistic model with a structured latent prior. This prior allows to discover latent variable representations that are structured into a hierarchy of independent vector spaces. The proposed prior has three major advantages: First, in contrast to the standard VAE normal prior the proposed prior is not rotationally invariant. This resolves the problem of unidentifiability of the standard VAE normal prior. Second, we demonstrate that the proposed prior encourages a disentangled latent representation which facilitates learning of disentangled representations. Third, extensive quantitative experiments demonstrate that the prior significantly mitigates the trade-off between reconstruction loss and disentanglement over the state of the art.

Figures

Figures reproduced from arXiv: 1909.05063 by the authors.

Figure 1
Figure 1. The modified ELBO objective of β-VAE emphasizes orthogonalization and pruning with increasing β-weight of the Kullback Leibler divergence term. 2.2 Related Work Recent work has shown an increased interest into learn￾ing of interpretable representations. In addition to the work mentioned already, we briefly review some of the influential papers: [9] present a variant of a GAN that encourages an interpretable latent r… view at source ↗
Figure 2
Figure 2. Iso-contours of the L p -nested function exam￾ple in equation 6 for combinations of p0, p1 ∈ {1, 2, 9}. 2.4 β-VAE Emphasizes Orthogonalization and Pruning What happens to these biases in the β-VAE general￾ization when β > 1? The short answer is that they grow. This can be understood by considering coordi￾nate ascent of the modified objective. With θ fixed, optimising q finds a solution that is closer to the prior di… view at source ↗
Figure 3
Figure 3. Leptokurtic and platykurtic priors encourage different orientations of the encoding of the (x,y) location of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Tree representation of L p -nested distributions. a) Tree of the example provided in Eq. 6. b) Tree corresponding to an L p -nested ISA model. Kulback-Leibler divergence of the ELBO-objective can be estimated by Monte-Carlo sampling. This leads to an ELBO-objective of …
Figure 5
Figure 5. Figure 5: Comparison of the different approaches for different values of [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Comparison on the cars3d dataset. Scatter [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Histogram of learned exponents on the 3d faces dataset. To identify the different subspaces we choose [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Disentangled representations for representative models of the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Disentangled representations for representative models of the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Disentangled representations for representative models of the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Disentangled representations for models representative for the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Disentangled representations for models representative for the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Disentangled representations for models representative for the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Disentangled representations for models representative for the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Disentangled representations for models representative for the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Disentangled representations for models representative for the upper quantile of MIG scores for [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 39 canonical work pages

  1. [1]

    Learning factorial codes by pre- dictability minimization,

    J. Schmidhuber, “Learning factorial codes by pre- dictability minimization,” Neural Computation , vol. 4, no. 6, pp. 863–879, 1992

    work page 1992

  2. [2]

    A survey of inductive biases for factorial representation-learning,

    K. Ridgeway, “A survey of inductive biases for factorial representation-learning,” arXiv preprint arXiv:1612.05299, 2016

    arXiv 2016

  3. [3]

    Emergence of invariance and disentangling in deep representations,

    A. Achille and S. Soatto, “Emergence of invariance and disentangling in deep representations,” arXiv preprint arXiv:1706.01350, 2017

    arXiv 2017

  4. [4]

    β-vae: Learning basic visual concepts with a con- strained variational framework,

    I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glo- rot, M. Botvinick, S. Mohamed, and A. Lerchner, “β-vae: Learning basic visual concepts with a con- strained variational framework,” 2016

    work page 2016

  5. [5]

    Un- derstanding disentangling in β-vae,

    C. P. Burgess, I. Higgins, A. Pal, L. Matthey, N. Watters, G. Desjardins, and A. Lerchner, “Un- derstanding disentangling in β-vae,” in Learning Jan St¨ uhmer, Richard E. T urner, Sebastian Nowozin Disentangled Representations: From Perception to Control Workshop, 2017

    work page 2017

  6. [6]

    Isolating sources of disentanglement in variational autoencoders,

    T. Q. Chen, X. Li, R. Grosse, and D. Duvenaud, “Isolating sources of disentanglement in variational autoencoders,” arXiv preprint arXiv:1802.04942 , 2018

    arXiv 2018

  7. [7]

    Disentangling by factoris- ing,

    H. Kim and A. Mnih, “Disentangling by factoris- ing,” arXiv preprint arXiv:1802.05983 , 2018

    arXiv 2018

  8. [8]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in International Conference on Learning Representations, 2014

    2014

Show all 50 references
  1. [9]

    Infogan: Inter- pretable representation learning by information maximizing generative adversarial nets,

    X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Inter- pretable representation learning by information maximizing generative adversarial nets,” in Ad- vances in neural information processing systems , pp. 2172–2180, 2016

    2016

  2. [10]

    Sugiyama, T

    M. Sugiyama, T. Suzuki, and T. Kanamori, Den- sity ratio estimation in machine learning . Cam- bridge University Press, 2012

    2012

  3. [11]

    Variational inference of disentangled latent con- cepts from unlabeled observations,

    A. Kumar, P. Sattigeri, and A. Balakrishnan, “Variational inference of disentangled latent con- cepts from unlabeled observations,” arXiv preprint arXiv:1711.00848, 2017

    2017 arXiv

  4. [12]

    Generative models of visually grounded imagina- tion,

    R. Vedantam, I. Fischer, J. Huang, and K. Murphy, “Generative models of visually grounded imagina- tion,” arXiv preprint arXiv:1705.10762 , 2017

    2017 arXiv

  5. [14]

    Deep convolutional inverse graph- ics network,

    T. D. Kulkarni, W. F. Whitney, P. Kohli, and J. Tenenbaum, “Deep convolutional inverse graph- ics network,” in Advances in Neural Information Processing Systems 28 (C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, eds.), pp. 2539–2547, Curran Associates, Inc., 2015

    2015

  6. [15]

    Semi-supervised learning with deep generative models,

    D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling, “Semi-supervised learning with deep generative models,” in Advances in Neural Infor- mation Processing Systems, pp. 3581–3589, 2014

    2014

  7. [16]

    Learning to disentangle factors of variation with manifold interaction,

    S. Reed, K. Sohn, Y. Zhang, and H. Lee, “Learning to disentangle factors of variation with manifold interaction,” in International Conference on Ma- chine Learning, pp. 1431–1439, 2014

    2014

  8. [17]

    Automatic differentiation in machine learning: a survey.,

    A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, “Automatic differentiation in machine learning: a survey.,” Journal of machine learning research, vol. 18, no. 153, pp. 1–153, 2017

    2017

  9. [18]

    Transforming auto-encoders,

    G. E. Hinton, A. Krizhevsky, and S. D. Wang, “Transforming auto-encoders,” in International Conference on Artificial Neural Networks , pp. 44– 51, Springer, 2011

    2011

  10. [19]

    Unpaired image-to-image translation using cycle- consistent adversarial networks,

    J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle- consistent adversarial networks,” arXiv preprint, 2017

    2017

  11. [20]

    Unsupervised learning of spatiotem- porally coherent metrics,

    R. Goroshin, J. Bruna, J. Tompson, D. Eigen, and Y. LeCun, “Unsupervised learning of spatiotem- porally coherent metrics,” in Proceedings of the IEEE international conference on computer vision, pp. 4086–4093, 2015

    2015

  12. [21]

    Unsuper- vised learning of disentangled and interpretable representations from sequential data,

    W.-N. Hsu, Y. Zhang, and J. Glass, “Unsuper- vised learning of disentangled and interpretable representations from sequential data,” in Advances in neural information processing systems, pp. 1878– 1889, 2017

    2017

  13. [22]

    Unsupervised learning of disentangled representations from video,

    E. L. Denton et al. , “Unsupervised learning of disentangled representations from video,” in Ad- vances in Neural Information Processing Systems , pp. 4414–4423, 2017

    2017

  14. [23]

    Vae with a vamp- prior,

    J. M. Tomczak and M. Welling, “Vae with a vamp- prior,” arXiv preprint arXiv:1705.07120 , 2017

    2017 arXiv

  15. [24]

    The loracs prior for vaes: Letting the trees speak for the data,

    S. Vikram, M. D. Hoffman, and M. J. Johnson, “The loracs prior for vaes: Letting the trees speak for the data,” arXiv preprint arXiv:1810.06891 , 2018

    2018 arXiv

  16. [25]

    Chal- lenging common assumptions in the unsupervised learning of disentangled representations,

    F. Locatello, S. Bauer, M. Luˇ ci´ c, G. R¨ atsch, S. Gelly, B. Sch¨ olkopf, and O. F. Bachem, “Chal- lenging common assumptions in the unsupervised learning of disentangled representations,” in Inter- national Conference on Machine Learning , 2019

    2019

  17. [26]

    Vari- ational autoencoders pursue pca directions (by accident),

    M. Rolinek, D. Zietlow, and G. Martius, “Vari- ational autoencoders pursue pca directions (by accident),” in Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition, pp. 12406–12415, 2019

    2019

  18. [27]

    Two problems with variational expectation maximisation for time- series models,

    R. E. Turner and M. Sahani, “Two problems with variational expectation maximisation for time- series models,” in Bayesian Time series models (D. Barber, T. Cemgil, and S. Chiappa, eds.), ch. 5, pp. 109–130, Cambridge University Press, 2011

    2011

  19. [28]

    Im- portance weighted autoencoders,

    Y. Burda, R. Grosse, and R. Salakhutdinov, “Im- portance weighted autoencoders,” arXiv preprint arXiv:1509.00519, 2015

    2015 arXiv

  20. [29]

    Inference suboptimality in variational autoencoders,

    C. Cremer, X. Li, and D. Duvenaud, “Inference suboptimality in variational autoencoders,” arXiv preprint arXiv:1801.03558, 2018. Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations

    2018 arXiv

  21. [30]

    The generalized gaus- sian mixture model using ica,

    T.-w. Lee and M. Lewicki, “The generalized gaus- sian mixture model using ica,” 08 2000

    2000

  22. [31]

    Self- adaptive blind source separation based on activa- tion functions adaptation,

    L. Zhang, A. Cichocki, and S.-i. Amari, “Self- adaptive blind source separation based on activa- tion functions adaptation,” IEEE Transactions on Neural Networks, vol. 15, no. 2, pp. 233–244, 2004

    2004

  23. [32]

    Efficient coding of natural sounds,

    M. S. Lewicki, “Efficient coding of natural sounds,” Nature neuroscience, vol. 5, no. 4, p. 356, 2002

    2002

  24. [33]

    Lp-nested symmetric distributions,

    F. Sinz and M. Bethge, “Lp-nested symmetric distributions,” Journal of Machine Learning Re- search, vol. 11, no. Dec, pp. 3409–3451, 2010

    2010

  25. [34]

    G. E. Box and G. C. Tiao, Bayesian Inference in Statistical Analysis . Reading,Mass.: Addison- Wesley, 1973

    1973

  26. [35]

    dsprites: Disentanglement testing sprites dataset

    L. Matthey, I. Higgins, D. Hassabis, and A. Ler- chner, “dsprites: Disentanglement testing sprites dataset.” https://github.com/deepmind/dsprites- dataset/, 2017

    2017

  27. [36]

    Character- ization of the p-generalized normal distribution,

    F. Sinz, S. Gerwinn, and M. Bethge, “Character- ization of the p-generalized normal distribution,” Journal of Multivariate Analysis , vol. 100, no. 5, pp. 817–820, 2009

    2009

  28. [37]

    Emergence of phase-and shift-invariant features by decomposi- tion of natural images into independent feature subspaces,

    A. Hyv¨ arinen and P. Hoyer, “Emergence of phase-and shift-invariant features by decomposi- tion of natural images into independent feature subspaces,” Neural computation, vol. 12, no. 7, pp. 1705–1720, 2000

    2000

  29. [38]

    Complex cell pool- ing and the statistics of natural images,

    A. Hyv¨ arinen and U. K¨ oster, “Complex cell pool- ing and the statistics of natural images,” Network: Computation in Neural Systems , vol. 18, no. 2, pp. 81–100, 2007

    2007

  30. [39]

    Hier- archical modeling of local image features through Lp-nested symmetric distributions,

    F. H. Sinz, E. P. Simoncelli, and M. Bethge, “Hier- archical modeling of local image features through Lp-nested symmetric distributions,” in Advances in neural information processing systems, pp. 1696– 1704, 2009

    2009

  31. [40]

    Modeling and inference with υ-spherical distri- butions,

    C. Fernandez, J. Osiewalski, and M. F. Steel, “Modeling and inference with υ-spherical distri- butions,” Journal of the American Statistical As- sociation, vol. 90, no. 432, pp. 1331–1340, 1995

    1995

  32. [41]

    To- wards a definition of disentangled representations,

    I. Higgins, D. Amos, D. Pfau, S. Racani` ere, L. Matthey, D. J. Rezende, and A. Lerchner, “To- wards a definition of disentangled representations,” CoRR, vol. abs/1812.02230, 2018

    2018 arXiv

  33. [42]

    Stick- ing the landing: Simple, lower-variance gradi- ent estimators for variational inference,

    G. Roeder, Y. Wu, and D. K. Duvenaud, “Stick- ing the landing: Simple, lower-variance gradi- ent estimators for variational inference,” in Ad- vances in Neural Information Processing Systems 30 (I. Guyon, U. V. Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan, and ...

    2017

  34. [43]

    IEEE, A 3D Face Model for Pose and Illumination Invariant Face Recognition, (Genova, Italy), 2009

    2009

  35. [44]

    Deep visual analogy-making,

    S. E. Reed, Y. Zhang, Y. Zhang, and H. Lee, “Deep visual analogy-making,” in Advances in Neural In- formation Processing Systems 28 (C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Gar- nett, eds.), pp. 1252–1260, Curran Associates, Inc., 2015

    2015

  36. [45]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Confer- ence on Learning Representations, 2015. Jan St¨ uhmer, Richard E. T urner, Sebastian Nowozin A Appendix A.1 Sampling from Lp-nested Symmetric Distributions We reproduce the sampling ...

    2015

  37. [46]

    Sample v0 from a beta distribution β[n, 1]

  38. [47]

    Obtain coordinates on the Lp-nested sphere within the positive orthant by si↦→ s 1 pi i = ˜ui (the exponentiation is taken component-wise)

    For each inner node i of the tree associated with f, sample the auxiliary variable si from a Dirichlet distribution Dir [ ni,1 pi ,..., ni,l1 pi ] where ni,k are the number of leaves in the subtree under node i,k . Obtain coordinates on the Lp-nested sphere within the positive...

  39. [48]

    The components of vi,1:li constitute the radii for the layer direct below them

    Transform these samples to Cartesian coordinates byvi· ˜ui = vi,1:li for each inner node, starting from the root node and descending to lower layers. The components of vi,1:li constitute the radii for the layer direct below them. If i = 0, the radius had been sampled in step 1

  40. [49]

    Normalize x to get a uniform sample from the sphere u = x f (x)

    Once the two previous steps have been repeated until no inner node is left, we have a sample x from the uniform distribution in the positive quadrant. Normalize x to get a uniform sample from the sphere u = x f (x)

  41. [50]

    Sample a new radius ˜v0 from the radial distribution of the target radial distribution ψ0 and obtain the sample via ˜x = ˜v0· u

  42. [51]

    Multiply each entry xi of ˜x by and independent sample zi from the uniform distribution over {−1, 1}. Algorithm 1: Exact sampling algorithm forLp-nested symmetric distributions from [33] A.2 Learned Exponents An interesting question when learning the exponents of the prior is,...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.