Pith. sign in

REVIEW 2 major objections 2 minor 50 references

Disentanglement with Holographic Reduced Representations

T0 review · 2 major / 2 minor · reviewed 2026-06-27 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Unbinding in holographic reduced representations induces approximately independent symbol-value pairs and supplies a per-slot capacity bound for disentanglement.

desk verdict HRR disentanglement gives a vector-sum latent with a claimed capacity bound, but the independence proof is for the fixed operator and likely does not carry to the trained network. read the letter →

arxiv 2606.09725 v1 pith:7QEEMWAE submitted 2026-06-08 cs.LG

Jhonny J. Velasquez Olivera , Christo K. Thomas , Walid Saad This is my paper · ORCID
classification cs.LG
keywords disentanglementholographicreducedrepresentationsHRRunbindinginductivebiassymboliccapacityboundneuralnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes an unsupervised algorithm that embeds holographic reduced representations inside neural networks to produce disentangled factors of variation. It treats the latent code as a sum of vector symbols rather than a set of scalar dimensions and relies on the unbinding operation to extract individual symbol-value pairs. The central theoretical result is a proof that unbinding creates approximately independent pairs together with an information-theoretic bound on the number of distinct symbolic concepts that fit reliably into each slot. This construction yields competitive scores on standard disentanglement metrics and latent-traversal tests while showing greater robustness to additive noise than conventional autoencoder baselines. The approach therefore supplies both an empirical method and a quantitative account of the inductive bias that favors separation of compositional concepts.

What carries the argument

The HRR unbinding operation, which extracts a value vector from a summed holographic representation by circular correlation with a symbol vector.

What would settle it

Train the model on a dataset with known factors, apply unbinding to the learned codes, and compute mutual information between the recovered symbol and value vectors; if the pairs remain strongly dependent, the independence claim fails.

Watch

Extended reading notes

Core claim

The HRR unbinding operation, when placed inside a differentiable network, induces approximately independent symbol-value pairs; an information-theoretic analysis of the unbinding channel proves this independence property and supplies a per-slot capacity bound that quantifies how many distinct symbolic concepts can be encoded without destructive interference.

Load-bearing premise

The unbinding step can be inserted into an ordinary differentiable neural network while keeping the symbolic separation properties intact.

Editorial extensions

If this is right

  • The summed-vector latent code is more robust to noise than scalar-dimensional latent codes across a range of signal-to-noise ratios.
  • Reconstruction quality remains stable even when the input is corrupted by noise that would degrade standard disentangled autoencoders.
  • The per-slot capacity bound limits the number of reliably separable symbolic concepts that can be stored in one representation.
  • Latent units become explicit vector symbols rather than independent scalar dimensions, supporting compositional relationships among factors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The capacity bound could be used to choose the dimensionality of the HRR space for a target number of factors before training begins.
  • The same unbinding mechanism might be tested on datasets whose factors have explicit compositional structure to check whether the symbolic encoding improves generalization.
  • Because the representations remain fully differentiable, the method could be inserted into larger pipelines that combine neural perception with symbolic reasoning steps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes an unsupervised disentanglement method that embeds holographic reduced representations (HRR) into a neural network, using the unbinding operation as an inductive bias for separating symbolic factors of variation. It claims to prove that unbinding produces approximately independent symbol-value pairs, derives a per-slot capacity bound quantifying reliable encoding of distinct concepts, reports competitive performance against VAE/GAN baselines on latent traversals and disentanglement metrics, and shows greater robustness to noise than scalar-latent autoencoders. Representations are formed by summed vector bindings rather than low-dimensional scalar latents.

Significance. If the independence result and capacity bound are shown to survive end-to-end gradient training and the empirical gains are reproducible, the work supplies a concrete symbolic mechanism with an information-theoretic account of its disentangling bias, together with a noise-robust alternative to continuous latent models. The vector-sum representation and explicit capacity analysis are distinctive strengths that could inform hybrid neuro-symbolic architectures.

major comments (2)
  1. [§4] §4 (theoretical analysis): The independence proof and per-slot capacity bound are derived under the standard algebraic assumptions of the isolated HRR channel (high-dimensional random vectors, circular convolution, additive noise). The manuscript does not demonstrate that the vectors learned by joint back-propagation on reconstruction and disentanglement losses continue to satisfy those distributional assumptions at convergence; without such verification the bound does not automatically transfer to the trained model.
  2. [Experimental section (Tables 2–3 and Figure 4)] Experimental section (Tables 2–3 and Figure 4): competitiveness is asserted on standard disentanglement benchmarks, yet the precise architecture used to embed the differentiable HRR binding/unbinding operators (including how the binding weights are parameterized and whether they remain fixed or are optimized) is not specified in sufficient detail to reproduce the claimed separation property or to confirm that the capacity bound remains predictive of observed performance.
minor comments (2)
  1. [§3] Notation for the binding and unbinding operators should be introduced once with explicit equations rather than relying on prose descriptions.
  2. [Abstract and §5] The abstract states that the method yields 'competitive results'; the corresponding quantitative deltas and statistical significance tests should be reported in the main text rather than only in supplementary material.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We respond to each major comment below and indicate planned revisions.

read point-by-point responses

  1. Referee: [§4] §4 (theoretical analysis): The independence proof and per-slot capacity bound are derived under the standard algebraic assumptions of the isolated HRR channel (high-dimensional random vectors, circular convolution, additive noise). The manuscript does not demonstrate that the vectors learned by joint back-propagation on reconstruction and disentanglement losses continue to satisfy those distributional assumptions at convergence; without such verification the bound does not automatically transfer to the trained model.

    Authors: The referee is correct that the independence result and capacity bound are derived under the standard HRR assumptions and that the manuscript provides no explicit post-training verification that the learned vectors continue to satisfy them. We will add an appendix with empirical checks (e.g., distributional statistics and correlation measurements on the converged vectors) to support transfer of the bounds. revision: yes

  2. Referee: Experimental section (Tables 2–3 and Figure 4): competitiveness is asserted on standard disentanglement benchmarks, yet the precise architecture used to embed the differentiable HRR binding/unbinding operators (including how the binding weights are parameterized and whether they remain fixed or are optimized) is not specified in sufficient detail to reproduce the claimed separation property or to confirm that the capacity bound remains predictive of observed performance.

    Authors: We agree that the current description lacks sufficient implementation detail for reproducibility. The revised manuscript will expand the experimental section to specify the network architecture, confirm that binding weights are learned end-to-end by gradient descent (not fixed), and describe the differentiable implementations of binding and unbinding. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: mathematical proof of unbinding properties stands independently of training dynamics

full rationale

The paper's central theoretical contribution is an information-theoretic analysis proving that the HRR unbinding operation induces approximately independent symbol-value pairs, together with a derived per-slot capacity bound. This is presented as a direct mathematical result on the algebraic properties of the unbinding channel under standard high-dimensional vector assumptions, not as a statistical fit to data or a quantity obtained by optimizing the same loss used in the neural network experiments. No equations or claims in the abstract reduce the bound to a fitted parameter, a self-citation chain, or a renaming of an input; the proof is offered as an independent account of the inductive bias. Empirical results on disentanglement metrics are reported separately and do not feed back into the capacity derivation. The derivation chain is therefore self-contained against external benchmarks.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

The abstract does not introduce new free parameters, invented entities, or non-standard axioms; the work relies on the existing mathematical properties of holographic reduced representations and standard information-theoretic channel analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Disentanglement with Holographic Reduced Representations." pith.science (2026). https://pith.science/paper/7QEEMWAE

@misc{pith2026260609725,
  author       = {Pith},
  title        = {Pith review of: Disentanglement with Holographic Reduced Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7QEEMWAE}},
  note         = {Machine review of arXiv:2606.09725}
}
read the original abstract

Disentanglement, the separation of factors of variation in data using neural networks, remains a long-standing challenge in machine learning. Prior work has addressed this problem with variational autoencoders and generative adversarial networks that incorporate ideas from variational inference and information-theoretic constraints. In contrast to methods that rely on continuous representations, we propose a design that treats disentangled representations as symbolic structures, motivated by the compositional relationships among the concepts that make up samples from a distribution. However, learning discrete symbolic structures with neural networks while maintaining differentiability is difficult and often requires complex architectures. To address this, we introduce an unsupervised learning algorithm that uses holographic reduced representations (HRR) for neural disentanglement. We show that the HRR unbinding operation provides an inductive bias for separating factors and yields competitive results against baselines, as measured by latent traversals and disentanglement metrics. We complement these empirical findings with an information-theoretic analysis of the HRR unbinding channel. We prove that unbinding induces approximately independent symbol-value pairs and derive a per-slot capacity bound that quantifies how many distinct symbolic concepts can be reliably encoded, giving a quantitative account of the inductive bias toward disentanglement. The resulting representations differ from standard autoencoder-based models, in that their latent units are vectors that are summed together, rather than scalar dimensions of a low-dimensional latent vector. We show that this HRR representation is more robust to noise than other disentangled representations and maintains reconstruction quality across a range of SNRs.

Figures

Figures reproduced from arXiv: 2606.09725 by the authors.

Figure 1
Figure 1. Average performance of each model, as measured by PSNR, under varying levels of [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Each row is a visualization of how reconstruction quality degrades as noise intensity [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Latent component swaps performance for highest performing model relative to InfoM score. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Latent interpolation swaps components progressively until the representation is fully [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Mean and variance over training iterations, for both the codebook vectors and the latent [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Codebook usage over training iterations, measured by perplexity. We can see that the model [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: InfoM performance over different combinations of codebook size and latent dimension [PITH_FULL_IMAGE:figures/full_fig_p035_7.png]
Figure 8
Figure 8. Figure 8: InfoE performance over different combinations of codebook size and latent dimension [PITH_FULL_IMAGE:figures/full_fig_p035_8.png]
Figure 9
Figure 9. Figure 9: InfoC performance over different combinations of codebook size and latent dimension [PITH_FULL_IMAGE:figures/full_fig_p036_9.png]
Figure 10
Figure 10. Figure 10: Each row is a visualization of how reconstruction quality degrades as noise intensity [PITH_FULL_IMAGE:figures/full_fig_p039_10.png]
Figure 11
Figure 11. Figure 11: Each row is a visualization of how reconstruction quality degrades as noise intensity [PITH_FULL_IMAGE:figures/full_fig_p039_11.png]
Figure 12
Figure 12. Figure 12: Each row is a visualization of how reconstruction quality degrades as noise intensity [PITH_FULL_IMAGE:figures/full_fig_p040_12.png]
Figure 13
Figure 13. Figure 13: Latent component swaps performance for highest performing model relative to InfoM [PITH_FULL_IMAGE:figures/full_fig_p041_13.png]
Figure 14
Figure 14. Figure 14: Latent component swaps performance for highest performing model relative to InfoM [PITH_FULL_IMAGE:figures/full_fig_p041_14.png]
Figure 15
Figure 15. Figure 15: Latent component swaps performance for highest performing model relative to InfoM [PITH_FULL_IMAGE:figures/full_fig_p042_15.png]
Figure 16
Figure 16. Figure 16: Latent interpolation swaps components progressively until the representation is fully [PITH_FULL_IMAGE:figures/full_fig_p043_16.png]
Figure 17
Figure 17. Figure 17: Latent interpolation swaps components progressively until the representation is fully [PITH_FULL_IMAGE:figures/full_fig_p043_17.png]
Figure 18
Figure 18. Figure 18: Latent interpolation swaps components progressively until the representation is fully [PITH_FULL_IMAGE:figures/full_fig_p043_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 11 canonical work pages

  1. [1]

    doi: 10.1109/TPAMI.2013.50

    Y . Bengio, A. Courville, and P. Vincent. Representation Learning: A Review and New Per- spectives.IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798–1828, August 2013. ISSN 0162-8828, 2160-9292. doi: 10.1109/TPAMI.2013.50

    work page doi:10.1109/tpami.2013.50 2013

  2. [2]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes, December 2022

    2022

  3. [3]

    Beta-V AE: Learning basic visual concepts with a constrained variational framework

    Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. Beta-V AE: Learning basic visual concepts with a constrained variational framework. InInternational Conference on Learning Representations, 2017

    2017

  4. [4]

    Ricky T. Q. Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating Sources of Disentanglement in Variational Autoencoders. InAdvances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018

    2018

  5. [5]

    Disentangling by factorising

    Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In Jennifer Dy and Andreas Krause, editors,Proceedings of the 35th International Conference on Machine Learning, volume 80 ofProceedings of Machine Learning Research, pages 2649–2658. PMLR, July 2018

    2018

  6. [6]

    Learning Deep Disentangled Embeddings With the F-Statistic Loss

    Karl Ridgeway and Michael C Mozer. Learning Deep Disentangled Embeddings With the F-Statistic Loss. InAdvances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018

    2018

  7. [7]

    Cian Eastwood and Christopher K. I. Williams. A framework for the quantitative evaluation of disentangled representations. InInternational Conference on Learning Representations, 2018

    2018

  8. [8]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. InAdvances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014

    2014

Show all 50 references
  1. [9]

    InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Ad- versarial Nets

    Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Ad- versarial Nets. InAdvances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016

    2016

  2. [10]

    Kyle Hsu, Will Dorrell, James C. R. Whittington, Jiajun Wu, and Chelsea Finn. Disentanglement via latent quantization. InThirty-Seventh Conference on Neural Information Processing Systems, 2023

    2023

  3. [11]

    Neural Discrete Representation Learning

    Aaron van den Oord, Oriol Vinyals, and koray kavukcuoglu. Neural Discrete Representation Learning. InAdvances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017

    2017

  4. [12]

    A sober look at the unsupervised learning of disentangled representations and their evaluation.Journal of Machine Learning Research, 21(209):1–62, 2020

    Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. A sober look at the unsupervised learning of disentangled representations and their evaluation.Journal of Machine Learning Research, 21(209):1–62, 2020

    2020

  5. [13]

    Kovalev, and Aleksandr I

    Alexandr Korchemnyi, Alexey K. Kovalev, and Aleksandr I. Panov. Symbolic disentangled representations for images, 2024

    2024

  6. [14]

    Rachkovskij, Evgeny Osipov, and Abbas Rahimi

    Denis Kleyko, Dmitri A. Rachkovskij, Evgeny Osipov, and Abbas Rahimi. A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations.ACM Comput. Surv., 55(6):130:1–130:40, December 2022. ISSN 0360-0300. doi: 10.1145/3538531

    2022 doi

  7. [15]

    An encoding framework for binarized images using hyperdimensional computing.Frontiers in Big Data, 7, June 2024

    Laura Smets, Werner Van Leekwijck, Ing Jyh Tsang, and Steven Latré. An encoding framework for binarized images using hyperdimensional computing.Frontiers in Big Data, 7, June 2024. ISSN 2624-909X. doi: 10.3389/fdata.2024.1371518. 11

    2024 doi

  8. [16]

    Hierarchical hyperdimen- sional computing for energy efficient classification

    Mohsen Imani, Chenyu Huang, Deqian Kong, and Tajana Rosing. Hierarchical hyperdimen- sional computing for energy efficient classification. InProceedings of the 55th Annual Design Automation Conference, Dac ’18, New York, NY , USA, 2018. Association for Computing Machinery. ISB...

    2018 doi

  9. [17]

    Olshausen, Yulia Sandamirskaya, Friedrich T

    Alpha Renner, Lazar Supic, Andreea Danielescu, Giacomo Indiveri, Bruno A. Olshausen, Yulia Sandamirskaya, Friedrich T. Sommer, and E. Paxon Frady. Neuromorphic visual scene understanding with resonator networks.Nature Machine Intelligence, 6(6):641–652, June

  10. [18]

    doi: 10.1038/s42256-024-00848-0

    ISSN 2522-5839. doi: 10.1038/s42256-024-00848-0. URL https://www.nature. com/articles/s42256-024-00848-0

    doi

  11. [19]

    A Simple Framework for Contrastive Learning of Visual Representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. InProceedings of the 37th International Conference on Machine Learning, pages 1597–1607. PMLR, November 2020

    2020

  12. [20]

    Unsupervised representation learning with deep convolutional generative adversarial networks

    Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, Ma...

    2016

  13. [21]

    Emerging Properties in Self-Supervised Vision Transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jegou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging Properties in Self-Supervised Vision Transformers. In2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9630–9640, October

  14. [22]

    doi: 10.1109/ICCV48922.2021.00951

    2021 doi

  15. [23]

    Self-Supervised Learning from Images with a Joint- Embedding Predictive Architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-Supervised Learning from Images with a Joint- Embedding Predictive Architecture. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2023 doi

  16. [24]

    Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap Your Own Latent - A New Ap...

    2020

  17. [25]

    Masked Autoencoders Are Scalable Vision Learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked Autoencoders Are Scalable Vision Learners. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979–15988, June 2022. doi: 10.1109/CVPR52688. 2022.01553

    2022 doi

  18. [26]

    Efficient Estimation of Word Representations in Vector Space, September 2013

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space, September 2013. URL http://arxiv.org/abs/1301

    2013

  19. [27]

    arXiv:1301.3781 [cs]

    arXiv

  20. [28]

    Learning Transferable Visual Models From Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of ...

    2021

  21. [29]

    A neuro- vector-symbolic architecture for solving Raven’s progressive matrices.Nature Machine Intelli- gence, 5(4):363–375, March 2023

    Michael Hersche, Mustafa Zeqiri, Luca Benini, Abu Sebastian, and Abbas Rahimi. A neuro- vector-symbolic architecture for solving Raven’s progressive matrices.Nature Machine Intelli- gence, 5(4):363–375, March 2023. ISSN 2522-5839. doi: 10.1038/s42256-023-00630-8

    2023 doi

  22. [30]

    Learning with Holographic Reduced Representations

    Ashwinkumar Ganesan, Hang Gao, Sunil Gandhi, Edward Raff, Tim Oates, James Holt, and Mark McLean. Learning with Holographic Reduced Representations. InAdvances in Neural Information Processing Systems, volume 34, pages 25606–25620. Curran Associates, Inc., 2021. 12

    2021

  23. [31]

    Towards generalization in subitizing with neuro-symbolic loss using holographic reduced representations

    Mohammad Mahmudul Alam, Edward Raff, and Tim Oates. Towards generalization in subitizing with neuro-symbolic loss using holographic reduced representations. InNeuro- symbolic learning and reasoning in the era of large language models, 2023. URL https: //openreview.net/forum?id...

    2023

  24. [32]

    Recasting Self-Attention with Holographic Reduced Representations

    Mohammad Mahmudul Alam, Edward Raff, Stella Biderman, Tim Oates, and James Holt. Recasting Self-Attention with Holographic Reduced Representations. InProceedings of the 40th International Conference on Machine Learning, pages 490–507. PMLR, July 2023

    2023

  25. [33]

    T.A. Plate. Holographic reduced representations.IEEE Transactions on Neural Networks, 6(3): 623–641, May 1995. ISSN 1941-0093. doi: 10.1109/72.377968

    1995 doi

  26. [34]

    Tripod: Three complementary inductive biases for disentangled representation learning

    Kyle Hsu, Jubayer Ibn Hamid, Kaylee Burns, Chelsea Finn, and Jiajun Wu. Tripod: Three complementary inductive biases for disentangled representation learning. InProceedings of the 41st International Conference on Machine Learning, ICML’24, Vienna, Austria, 2024. JMLR.org

    2024

  27. [35]

    Estimating or propagating gradients through stochastic neurons for conditional computation, 2013

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation, 2013

    2013

  28. [36]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  29. [37]

    3D shapes dataset, 2018

    Chris Burgess and Hyunjik Kim. 3D shapes dataset, 2018

    2018

  30. [38]

    On the transfer of inductive bias from simulation to the real world: a new disen- tanglement dataset

    Muhammad Waleed Gondal, Manuel Wuthrich, Djordje Miladinovic, Francesco Locatello, Martin Breidt, Valentin V olchkov, Joel Akpo, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. On the transfer of inductive bias from simulation to the real world: a new disen- tanglement d...

    2019

  31. [39]

    NVIDIA Research Projects, November 2025

    NVlabs/High-res-disentanglement-datasets. NVIDIA Research Projects, November 2025

    2025

  32. [40]

    The role of disentanglement in generalisation

    Milton Llera Montero, Casimir JH Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrey Bowers. The role of disentanglement in generalisation. InInternational Conference on Learning Representations, 2021

    2021

  33. [41]

    Neural systematic binder

    Gautam Singh, Yeongbin Kim, and Sungjin Ahn. Neural systematic binder. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  34. [42]

    PyTorch: An Imperative Style, High-Performance Deep Learning Library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  35. [43]

    +x 1c0c2 +x 2c0c1 +x 1c0c1 +x2c1c2 +x 1c1c2 +x 2c0c2 x1(c2 0 +c 2 1 +c 2

  36. [44]

    +x 0c0c1 +x 2c0c2 +x 0c0c2 +x2c1c2 +x 0c1c2 +x 2c0c1 x2(c2 0 +c 2 1 +c 2

  37. [45]

    d−1X k=0 d−1X m=0 s(i) k s(ℓ) m (vℓ)k+a−m ! · d−1X k′=0 d−1X m′=0 s(i) k′ s(ℓ) m′(vℓ)k′+b−m′ !# =E

    +x 0c0c1 +x 1c1c2 +x 0c1c2 +x1c0c2 +x 0c0c2 +x 1c0c1   = "x0(1 +ξ) +η 0 x1(1 +ξ) +η 1 x2(1 +ξ) +η 2 # = (1 +ξ)x+η. 15 Here, ξ= (c 2 0 +· · ·+c 2 d)−1 . Through the central limit theorem and by assuming independence between ci and xi, we can show that ξ∼ N(0,2/d) , and...

  38. [46]

    Initializing the symbol vectors, S, with an improved scheme proposed in a prior work [27]

  39. [47]

    Setting the codebook,C, to be static

  40. [48]

    Setting the symbol vectors,Sto be learnable

  41. [49]

    Removing the pre-processing network described in Section 3.1

  42. [50]

    H Complete disentanglement results The full disentanglement results for each dataset, with the mean and 95% confidence intervals described in Appendix E, can be found in Table 12

    Directly keeping approximate-HRR vectors in the codebook, as opposed to the parameterized approach described in the previous paragraph. H Complete disentanglement results The full disentanglement results for each dataset, with the mean and 95% confidence intervals described in...

Pith tools

Reviewed June 27, 2026 · model on record in the stance chip above.