pith. sign in

arxiv: 2606.07058 · v1 · pith:PDP7JUUTnew · submitted 2026-06-05 · 💻 cs.LG · cs.CV· math.AT· stat.ML

Constructing VAE Latent Spaces with Prescribed Topology

Jilles S. van Hulst , Jakub M. Tomczak , W.P.M.H. Heemels , Duarte J. Antunes This is my paper

Pith reviewed 2026-06-27 22:59 UTC · model grok-4.3

classification 💻 cs.LG cs.CVmath.ATstat.ML
keywords variational autoencoderslatent space topologyproduct covering spacestopological mismatchmanifold learningKL divergencereparametrization trick
0
0 comments X

The pith

A constructive framework lets VAEs use latent spaces whose topology matches data manifolds that admit product covering spaces.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows how to build variational autoencoders whose priors align with the topology of data lying on manifolds such as tori, cylinders, Möbius strips, Klein bottles, and real projective spaces. Standard Gaussian priors create a mismatch that degrades reconstruction when data has periodic or bounded structure. The method factorizes distributions over elementary factors like circles, intervals, and lines, producing closed-form decoupled KL divergences so each factor can be shaped independently. Coordinate transformations let ordinary neural networks output the required non-Euclidean parameters while keeping gradients smooth. Experiments on synthetic manifolds and rotated or cyclically shifted MNIST confirm that topology-aware models outperform the Gaussian baseline across regularization strengths.

Core claim

The authors present a constructive mathematical framework that resolves the topological mismatch for all manifolds that admit a product covering space, expressed as products of circles, intervals, or lines or as quotients of such products by a finite symmetry group. Factorized distributions over the elementary factors produce product topologies with closed-form, decoupled KL divergences, allowing each latent factor to be shaped independently while training remains tractable. Reparametrizable encoder-prior pairs are catalogued for periodic, bounded, and unbounded supports, and coordinate transformations enable standard networks to output the necessary parameters. For quotient manifolds the de

What carries the argument

Product covering space realized through factorized distributions over elementary factors (circles, intervals, lines) or finite quotients thereof, with decoupled KL terms and group-invariant decoder inputs.

If this is right

  • KL regularization aligns with the true data manifold rather than imposing Euclidean structure.
  • Each latent factor can be regularized and sampled independently without coupling terms.
  • Training stays tractable because all KL divergences remain closed-form.
  • Identified points on quotient manifolds produce identical decoder outputs by construction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same factorized construction could be tested on time-series data with periodic components to check whether reconstruction of cycles improves.
  • Anchor constraints might be varied to create controlled topological holes and measure effects on generation diversity.
  • The coordinate transformations could be reused in other latent-variable models that need non-Euclidean parameter outputs.

Load-bearing premise

The data manifold must admit a product covering space built from products of circles, intervals, or lines or quotients of such products by a finite symmetry group.

What would settle it

A dataset whose manifold lacks any product covering space of the required form where the proposed priors produce worse reconstructions or higher KL mismatch than a standard Gaussian prior at matched regularization strength.

Figures

Figures reproduced from arXiv: 2606.07058 by Duarte J. Antunes, Jakub M. Tomczak, Jilles S. van Hulst, W.P.M.H. Heemels.

Figure 1
Figure 1. Figure 1: Pipeline for designing a topology-aware VAE, illustrated with the running example of a M e [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Standard VAE architecture. The encoder networks [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: VAE architecture for topology shaping. The encoder maps observations [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Learned latent representations for the three synthetic experiments (columns: true factors, Gaussian baseline, topology-aware VAE, topology-aware [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: KL sweep for the three synthetic topologies. Each subplot varies [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Rotated MNIST experiment (R2 × S 1 latent space, digit classes 6 and 7). Columns: ground truth, Gaussian VAE (ℓ = 3), Mixed-Circle VAE, and Mixed-Circle VAE (anchored). First row: latent coordinates colored by the true rotation angle θ. for the Mixed-Circle VAEs, the circular coordinates (cos zθ, sin zθ) are plotted; for the Gaussian, a UMAP projection of the full latent space is shown instead; for the gro… view at source ↗
Figure 7
Figure 7. Figure 7: Shifted MNIST experiment (R2 × T 2 latent space, digit classes 3, 4, and 7). Columns: ground truth, Gaussian VAE (ℓ = 4), Mixed-Torus VAE, and Mixed-Torus VAE (anchored). First row: latent coordinates colored by the true shift angle θ1. The ground-truth column plots (θ1, θ2) directly. The Mixed-Torus VAEs plot the learned circular coordinates (zθ1 , zθ2 ). The Gaussian column shows a UMAP projection of the… view at source ↗
read the original abstract

Variational autoencoders (VAEs) learn low-dimensional latent representations of high-dimensional data. When the data lies on a manifold with non-Euclidean topology, the standard Gaussian prior introduces a topological mismatch that degrades reconstruction quality and prevents faithful representation. We present a constructive mathematical framework that resolves this mismatch for all manifolds that admit a product covering space. These are manifolds expressible as products of elementary factors (circles, intervals, or lines) or as quotients of such products by a finite symmetry group. The class includes cylinders, tori, M\"{o}bius strips, Klein bottles, and real projective spaces. Factorized distributions over the elementary factors yield product topologies with closed-form, decoupled KL divergences, so that each latent factor can be shaped independently while keeping training tractable. We catalogue reparametrizable encoder-prior pairs for periodic, bounded, and unbounded supports, and provide coordinate transformations that allow standard neural networks to output non-Euclidean parameters with smooth gradients. For quotient manifolds, the decoder receives group-invariant features of the covering-space coordinates, so that identified points produce identical outputs. Anchor constraints fix the coordinate system relative to the data or create soft topological holes. Experiments on synthetic manifolds and real-image datasets (rotated and cyclically shifted MNIST) confirm that a topology-matched prior aligns KL regularization with the data manifold. The resulting topology-aware models outperform the Gaussian baseline at all practically relevant regularization strengths. The code is available at https://github.com/JvHulst/VAE-Topology.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper claims to present a constructive mathematical framework for VAEs that resolves the topological mismatch between standard Gaussian priors and data manifolds admitting product covering spaces. These manifolds are defined as products of elementary factors (circles, intervals, lines) or quotients by finite symmetry groups, including cylinders, tori, Möbius strips, Klein bottles, and real projective spaces. The framework uses factorized distributions for decoupled KL divergences, catalogues reparametrizable encoder-prior pairs, provides coordinate transformations and group-invariant decoders, and uses anchor constraints. Experiments on synthetic data and modified MNIST datasets demonstrate superior performance over Gaussian baselines.

Significance. If the framework is correctly implemented for the claimed class of manifolds, it offers a tractable way to prescribe non-trivial topologies in VAE latent spaces, which could significantly improve representation learning for data with periodic, bounded, or quotient structures. The provision of code supports reproducibility and practical adoption.

major comments (1)
  1. [Abstract] Abstract: The assertion that real projective spaces are included in the class of manifolds admitting product covering spaces from 1D elementary factors is inconsistent. The universal cover of RP^n for n > 1 is the n-sphere S^n, which is not homeomorphic to any product of circles, intervals, or lines (as these are aspherical or have different topological invariants like Euler characteristic). The definition does not appear to accommodate this, creating an internal mismatch in the stated scope of the framework.
minor comments (1)
  1. [Abstract] Abstract: The phrase 'all practically relevant regularization strengths' is vague without specific values or figure references.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for identifying this inconsistency in the abstract. The comment is correct, and we will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The assertion that real projective spaces are included in the class of manifolds admitting product covering spaces from 1D elementary factors is inconsistent. The universal cover of RP^n for n > 1 is the n-sphere S^n, which is not homeomorphic to any product of circles, intervals, or lines (as these are aspherical or have different topological invariants like Euler characteristic). The definition does not appear to accommodate this, creating an internal mismatch in the stated scope of the framework.

    Authors: We agree with the referee. The framework is restricted to manifolds whose covering spaces are products of the listed 1D elementary factors (or finite quotients of such products). RP^1 is diffeomorphic to S^1 and therefore covered, but for n>1 the universal cover S^n lies outside this class. The inclusion of 'real projective spaces' in the abstract is therefore erroneous. We will remove this phrase from the abstract and from any similar statements in the introduction. The remainder of the claimed class (cylinders, tori, Möbius strips, Klein bottles) remains valid under the stated definition, as each admits a product covering space of the required form. revision: yes

Circularity Check

0 steps flagged

No circularity; constructive framework is self-contained

full rationale

The paper advances a constructive mathematical framework defining product covering spaces from elementary 1D factors, deriving factorized distributions with closed-form decoupled KL terms, and cataloguing reparametrizable encoder-prior pairs via explicit coordinate transformations. These steps rely on standard distributions and group actions rather than any fitted parameter renamed as a prediction, self-citation chain, or definitional loop. The central claim of resolving VAE topology mismatch follows directly from the stated constructions without reducing to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Relies on standard concepts from algebraic topology; no new entities introduced.

axioms (1)
  • domain assumption Manifolds admit a product covering space if expressible as products of circles, intervals, or lines or quotients thereof by finite groups.
    This defines the class of manifolds the framework applies to, as stated in the abstract.

pith-pipeline@v0.9.1-grok · 5829 in / 1199 out tokens · 31768 ms · 2026-06-27T22:59:51.997286+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    Auto-Encoding Variational Bayes,

    D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” Cambridge Explorations in Arts and Sciences, vol. 2, no. 1, pp. 1–14, dec 2013

    2013

  2. [2]

    Stochastic backprop- agation and approximate inference in deep generative models,

    D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic backprop- agation and approximate inference in deep generative models,”31st International Conference on Machine Learning, ICML 2014, vol. 4, pp. 3057–3070, 2014

    2014

  3. [3]

    Hyperspherical Variational Auto-Encoders,

    T. R. Davidson, L. Falorsi, N. De Cao, T. Kipf, and J. M. Tomczak, “Hyperspherical Variational Auto-Encoders,”34th Conference on Un- certainty in Artificial Intelligence 2018, UAI 2018, vol. 2, pp. 856–865, apr 2018

    2018

  4. [4]

    Increasing Expressivity of a Hyperspherical V AE,

    T. R. Davidson, J. M. Tomczak, and E. Gavves, “Increasing Expressivity of a Hyperspherical V AE,” inNeurIPS 2019 Workshop on Bayesian Deep Learning, vol. 1, no. 1, oct 2019, pp. 1–8

    2019

  5. [5]

    Explorations in Homeomorphic Variational Auto-Encoding,

    L. Falorsi, P. de Haan, T. R. Davidson, N. De Cao, M. Weiler, P. Forr´e, and T. S. Cohen, “Explorations in Homeomorphic Variational Auto-Encoding,” inICML workshop on Theoretical Foundations and Applications of Deep Generative Models, jul 2018

    2018

  6. [6]

    Continuous Hierarchical Representations with Poincar´e Variational Auto-Encoders,

    E. Mathieu, C. Le Lan, C. J. Maddison, R. Tomioka, and Y . W. Teh, “Continuous Hierarchical Representations with Poincar´e Variational Auto-Encoders,” inAdvances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc., 2019

    2019

  7. [7]

    Variational Autoencoders with Riemannian Brownian Motion Priors,

    D. Kalatzis, D. Eklund, G. Arvanitidis, and S. Hauberg, “Variational Autoencoders with Riemannian Brownian Motion Priors,” inICML 2020, aug 2020

    2020

  8. [8]

    Diffusion Variational Autoencoders,

    L. A. Perez Rey, V . Menkovski, and J. Portegies, “Diffusion Variational Autoencoders,” inProceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, vol. 2021-Janua. California: International Joint Conferences on Artificial Intelligence Organization, jul 2020, pp. 2704–2710

    2021

  9. [9]

    A wrapped normal distribution on hyperbolic space for gradient-based learning,

    Y . Nagano, S. Yamaguchi, Y . Fujita, and M. Koyama, “A wrapped normal distribution on hyperbolic space for gradient-based learning,” 36th International Conference on Machine Learning, ICML 2019, pp. 8242–8251, 2019

    2019

  10. [10]

    Isometric Quotient Variational Auto-Encoders for Structure-Preserving Represen- tation Learning,

    I. Huh, C. Jeong, J. M. Choe, Y . G. Kim, and D. S. Kim, “Isometric Quotient Variational Auto-Encoders for Structure-Preserving Represen- tation Learning,”Advances in Neural Information Processing Systems, vol. 36, no. NeurIPS, pp. 1–13, 2023

    2023

  11. [11]

    Equivariant Priors for Compressed Sensing with Unknown Orientation,

    A. Kuzina, K. Pratik, F. V . Massoli, and A. Behboodi, “Equivariant Priors for Compressed Sensing with Unknown Orientation,”Proceedings of Machine Learning Research, vol. 162, pp. 11 753–11 771, 2022

    2022

  12. [12]

    Flows for simultaneous manifold learning and density estimation,

    J. Brehmer and K. Cranmer, “Flows for simultaneous manifold learning and density estimation,”Advances in Neural Information Processing Systems, nov 2020

    2020

  13. [13]

    Equivariant Neural Rendering,

    E. Dupont, M. A. Bautista, A. Colburn, A. Sankar, C. Guestrin, J. Susskind, and Q. Shan, “Equivariant Neural Rendering,”37th Interna- tional Conference on Machine Learning, ICML 2020, vol. PartF16814, pp. 2741–2750, dec 2020

    2020

  14. [14]

    Topological Au- toencoders,

    M. Moor, M. Horn, B. Rieck, and K. Borgwardt, “Topological Au- toencoders,”37th International Conference on Machine Learning, ICML 2020, vol. PartF16814, pp. 7001–7011, may 2021

    2020

  15. [15]

    V AE with a VampPrior,

    J. M. Tomczak and M. Welling, “V AE with a VampPrior,”Proceedings of Machine Learning Research, vol. 84, feb 2018

    2018

  16. [16]

    On Incorporating Inductive Biases into V AEs,

    N. Miao, E. Mathieu, N. Siddharth, Y . W. Teh, and T. Rainforth, “On Incorporating Inductive Biases into V AEs,”ICLR 2022 - 10th International Conference on Learning Representations, pp. 1–22, feb 2022

    2022

  17. [17]

    Stereochem- istry of polypeptide chain configurations,

    G. Ramachandran, C. Ramakrishnan, and V . Sasisekharan, “Stereochem- istry of polypeptide chain configurations,”Journal of Molecular Biology, vol. 7, no. 1, pp. 95–99, jul 1963

    1963

  18. [18]

    A generative, probabilistic model of local protein structure,

    W. Boomsma, K. V . Mardia, C. C. Taylor, J. Ferkinghoff-Borg, A. Krogh, and T. Hamelryck, “A generative, probabilistic model of local protein structure,”Proceedings of the National Academy of Sciences, vol. 105, no. 26, pp. 8932–8937, jul 2008

    2008

  19. [19]

    S. M. LaValle,Planning Algorithms. Cambridge University Press, may 2006, vol. 9780521862

    2006

  20. [20]

    On the Local Behavior of Spaces of Natural Images,

    G. Carlsson, T. Ishkhanov, V . de Silva, and A. Zomorodian, “On the Local Behavior of Spaces of Natural Images,”International Journal of Computer Vision, vol. 76, no. 1, pp. 1–12, jan 2008

    2008

  21. [21]

    Deep Orientation Uncertainty Learning based on a Bingham Loss,

    I. Gilitschenski, R. Sahoo, W. Schwarting, A. Amini, S. Karaman, and D. Rus, “Deep Orientation Uncertainty Learning based on a Bingham Loss,” inInternational Conference on Learning Representations, 2020

    2020

  22. [22]

    J. M. Lee,Introduction to Topological Manifolds, 2nd ed., ser. Graduate Texts in Mathematics. Springer, 2011, vol. 202

    2011

  23. [23]

    T. M. Cover and J. A. Thomas,Elements of Information Theory, 2nd ed. Wiley, 2006

    2006

  24. [24]

    beta-V AE: Learning Basic Visual Concepts with a Constrained Variational Framework,

    I. Higgins, L. Matthey, A. Pal, C. P. Burgess, X. Glorot, M. M. Botvinick, S. Mohamed, and A. Lerchner, “beta-V AE: Learning Basic Visual Concepts with a Constrained Variational Framework,” inInter- national Conference on Learning Representations, 2017

    2017

  25. [25]

    Variational Inference with Normalizing Flows,

    D. J. Rezende and S. Mohamed, “Variational Inference with Normalizing Flows,”32nd International Conference on Machine Learning, ICML 2015, vol. 2, pp. 1530–1538, may 2015

    2015

  26. [26]

    Categorical Reparameterization with Gumbel-Softmax,

    E. Jang, S. Gu, and B. Poole, “Categorical Reparameterization with Gumbel-Softmax,”5th International Conference on Learning Represen- tations, ICLR 2017 - Conference Track Proceedings, pp. 1–13, aug 2017

    2017

  27. [27]

    The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables,

    C. J. Maddison, A. Mnih, and Y . W. Teh, “The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables,”5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, pp. 1–20, mar 2017

    2017

  28. [28]

    Learning Disentangled Joint Continuous and Discrete Representations,

    E. Dupont, “Learning Disentangled Joint Continuous and Discrete Representations,”Advances in Neural Information Processing Systems, vol. 2018-Decem, no. 2016, pp. 710–720, oct 2018

    2018

  29. [29]

    Sturmfels,Algorithms in Invariant Theory, 2nd ed

    B. Sturmfels,Algorithms in Invariant Theory, 2nd ed. Springer, 2008

    2008

  30. [30]

    A Discriminative Feature Learning Approach for Deep Face Recognition,

    Y . Wen, K. Zhang, Z. Li, and Y . Qiao, “A Discriminative Feature Learning Approach for Deep Face Recognition,” inECCV 2016, vol. 44, no. 4, 2016, pp. 499–515

    2016

  31. [31]

    Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

    N. Dilokthanakul, P. A. M. Mediano, M. Garnelo, M. C. H. Lee, H. Salimbeni, K. Arulkumaran, and M. Shanahan, “Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders,”arXiv preprint arXiv:1611.02648, jan 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  32. [32]

    Importance weighted au- toencoders,

    Y . Burda, R. Grosse, and R. Salakhutdinov, “Importance weighted au- toencoders,”4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, pp. 1–14, 2016

    2016

  33. [33]

    Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis,

    J. B. Kruskal, “Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis,”Psychometrika, vol. 29, no. 1, pp. 1–27, mar 1964

    1964

  34. [34]

    Understanding disentangling inβ-V AE,

    C. P. Burgess, I. Higgins, A. Pal, L. Matthey, N. Watters, G. Desjardins, and A. Lerchner, “Understanding disentangling inβ-V AE,” inNeurIPS 2017 Workshop on Learning Disentangled Representations, no. Nips, apr 2018. APPENDIXA KL DIVERGENCEDERIVATIONS This appendix provides the proof of Proposition 1, closed- form KL divergences for the distribution pairs...

    2017

  35. [35]

    For each factor of eZ, select functionsf 1,

    Choose a basis of coordinate functions for the covering space. For each factor of eZ, select functionsf 1, . . . , fm that naturally parameterize it: •Circular factorS 1:use{cosθ,sinθ}. •Interval factor[0,1]:use{h− 1 2 }. •Unbounded factorRor[0,∞):use{z}(or{z−c} centered at a symmetry-compatible pointc). 16 The full basis for the covering space is the uni...

  36. [36]

    , fm up to some degreed

    Form monomials inf 1, . . . , fm up to some degreed

  37. [37]

    ApplyRto each monomial

  38. [38]

    Discard any result that vanishes identically, and remove linearly dependent features

  39. [39]

    If not, increasedand repeat

    Verify that the surviving features separateG-orbits (i.e., ρ(z1) =ρ(z 2)impliesz 1 ∼z 2). If not, increasedand repeat. Proposition 3 guarantees termination at some finite degree d. All components ofρare polynomials or trigonometric polynomials by construction, so they are smooth and their gradients are available in closed form for backpropagation. A. Appl...