Pith. sign in

REVIEW 3 major objections 5 minor 29 references

Compositional Generalization via Forced Rendering of Disentangled Latents

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fully disentangled latents are not sufficient: rendering factors into the output pixel space is what restores compositional generalization.

desk verdict Clean toy demonstration that disentangled latents don't guarantee OOD composition, with two fixes that work, but the 're-entanglement' mechanism relies on a non-invariant metric that needs fixing before the explanation is taken as established. read the letter →

arxiv 2501.18797 v2 pith:VUT7IF36 submitted 2025-01-30 cs.LG cs.AIstat.ML

Qiyao Liang , Daoyuan Qian , Liu Ziyin , Ila Fiete This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords compositionalgeneralizationdisentangledrepresentationsout-of-distributionGaussianbumpgenerationJacobianmetrictensorlow-rankembeddingregularizationstripedataaugmentationmemorizationvscomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attacks the common belief that a disentangled, factorized latent representation is enough to make a neural network compose those factors in new combinations. In a controlled 2D Gaussian bump task, the authors feed the decoder fully disentangled encodings of the x and y coordinates of a bump, yet standard CNN and MLP decoders still fail when asked to generate bumps at held-out combinations of x and y: the networks re-entangle the factors in deeper layers and instead memorize the training images, superposing them when prompted with out-of-distribution inputs. The paper's positive claim is that robust compositional generalization appears once the disentangled factors are rendered directly into the full output (pixel) domain, either by low-rank regularized embedding filters that learn stripe-like per-coordinate templates, or by training on isolated 1D stripes plus a few bumps. If this is right, it redirects design from disentangling the bottleneck to disentangling the output representation, and it offers a concrete, data-efficient recipe for extrapolation.

What carries the argument

The central object is the Jacobian-based metric tensor g(x,y)=J(x,y)^\top J(x,y) of a layer's input-output map, which measures how infinitesimal displacements in the disentangled x and y inputs are stretched and mixed in the layer's output space; the scalar factorization score 1-|g_{xy}|/(|g_{xx}|+|g_{yy}|) quantifies layer-by-layer re-entanglement. The enabling mechanism is forced rendering: making the disentangled factors live as spatially structured 2D patterns in output/pixel space, implemented either by low-rank SVD-style embedding matrices with entropy and variance penalties or by training on 1D stripe images that isolate each factor. This preserves factorization through the decoder and permits additive composition that later layers refine into bump intersections.

What would settle it

Run the same 2D Gaussian bump task with a network whose first layer is a fixed invertible nonlinear mixing, such as a normalizing-flow block, applied to the fully disentangled (x,y) inputs. If the Jacobian-based factorization score drops sharply after that layer but the model still composes out-of-distribution bumps correctly, then layer-wise re-entanglement of the input representation is not the operative failure and the proposed diagnostic is misleading.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that factorization in an abstract latent space does not survive the downstream decoder. Tracking the Jacobian-based metric tensor layer by layer, the authors observe progressive manifold warping that erodes the factorization score of the representation, and a kernel analysis shows that out-of-distribution outputs match a binary factorized kernel that maps each coordinate independently to nearest in-distribution samples, i.e. a superposition of memorized images. The remedy is forced rendering: when the per-coordinate factors are expanded into the same 2D pixel space as the target images, through embedding matrices regularized to be low-rank with entropy and variance penalties, or through a data curriculum that trains on 1D Gaussian stripes before or alongside bumps, the network composes zero-shot in out-of-distribution regions and learns with data scaling that is linear in the image size N rather than cubic.

Load-bearing premise

The load-bearing premise is that the Jacobian-based factorization score measures true re-entanglement; that score is not invariant under the arbitrary invertible transformations the paper's own definition of factorization permits, so a nonlinear but factorized representation could be reported as entangled.

Editorial extensions

If this is right

  • Standard CNN and MLP decoders will fail out-of-distribution composition even with perfectly disentangled inputs, and the failure mode is predictable as superposition of in-distribution training images.
  • Low-rank regularization of per-coordinate embedding filters produces stripe-like pixel-space templates and enables compositional extrapolation, including into corner out-of-distribution regions.
  • Training on isolated 1D stripes enables zero-shot composition of 2D bumps; adding a linear number of 2D bump examples suffices, giving data scaling of O(N) instead of O(N^3).
  • Layer-by-layer Jacobian metric and volume diagnostics show factorization eroding with depth in failing networks and staying flat in successful stripe-trained or regularized networks.
  • The same superposition strategy appears in single-dimension interpolation and extrapolation, as shown by the MNIST rotation experiments, suggesting the mechanism is not specific to compositional tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the render-into-output-space principle generalizes, architecture search for compositional models should focus on the geometry of last-layer representations rather than the bottleneck; a testable extension is whether transformer or attention models with spatial, stripe-like factor embeddings inherit this benefit.
  • The paper sharpens the interpretation of prior stripe-training results: stripes help not only by learning disentangled latents but by placing factor structure in the pixel domain, where the decoder cannot easily re-entangle it.
  • A more robust diagnostic would be the minimum factorization score over all invertible reparametrizations; computing that could distinguish genuine topological entanglements from simple axis misalignment in reported failure cases.
  • In image domains beyond bumps, one could replace abstract latent codes with spatial factor maps, such as per-axis coordinate grids injected as input channels, and test whether out-of-distribution composition improves as the paper's principle predicts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies compositional generalization in a synthetic 2D Gaussian "bump" generation task. It shows that CNN and MLP decoders, even when given fully disentangled (x,y) input encodings, fail to generate correct out-of-distribution (OOD) bump positions; instead they tend to generate superpositions of memorized in-distribution images. The authors attribute this failure to a layer-by-layer "re-entanglement" of factors, measured via the Jacobian metric tensor and a scalar factorization score, and argue that deeper-layer manifold warping destroys the input factorization. They then propose two remedies: low-rank regularization of per-factor embedding matrices, and training on isolated 1D stripe patterns. Both are reported to restore compositional OOD generalization, with claimed data-efficiency scaling of roughly linear in image size for the stripe-based approach versus cubic for standard bump-only training. The manuscript includes extensive appendices, a conceptual framework for factorization/compositionality, and a link to code.

Significance. If the mechanistic story were validated, this would be an important clarification of the limits of disentangled representations and a practical recipe for achieving compositional generalization in controlled settings. The core empirical observations—failure with disentangled inputs, success with the two fixes—are documented with figures, ablations, and code, and they are likely reproducible. The paper also provides a broad conceptual framework for factorization in Appendix A. However, the central mechanistic claim that deeper layers "re-entangle" originally factorized representations is currently supported only by a coordinate-dependent measure that is inconsistent with the paper's own definition of factorization. If this measure can be replaced or validated, the paper would make a substantial contribution; as it stands, the explanation is not fully demonstrated.

major comments (3)
  1. [Section 3.4, Eq. (7), Appendix A.2] The factorization score in Eq. (7) is not invariant under the invertible reparametrizations allowed by the paper's own general definition of factorization in Appendix A.2. For example, for a 2D representation α(x,y) = (x+y, y), the invertible map T(u,v) = (u-v, v) recovers the factorized coordinates (x,y), so the representation is factorized according to the paper's definition. Yet its Jacobian is [[1,1],[0,1]], giving g_xy = 1, g_xx = 1, g_yy = 2, and Eq. (7) returns 2/3 rather than 1. Thus a factorized representation can be scored as entangled. Consequently, the "layer-by-layer erosion of factorization" and the volume metric maps in Fig. 2(c-f) may reflect coordinate choice rather than an intrinsic loss of factorization. Since Section 3.4 and the abstract use this erosion as the cause of OOD failure, this is a load-bearing issue. I recommend either adopting an invariant measure (e.g., checking whether a block-diagonalizing invertible map exists at each layer, or computing the minimal possible cross-terms under invertible transformations), or validating Eq. (7) on simple known factorized and entangled functions before using it to support the re-entanglement mechanism.
  2. [Section 4.2, Fig. 4(c)] The data-scaling claim that bump-only training requires O(N^3) samples while stripe-based training requires O(N) samples is presented as evidence that the approach "breaks the curse of dimensionality." However, this rests on a small synthetic task with image size N up to 28 and a fitted power law, with no reported confidence intervals, error bars, or statistical comparisons across seeds. The 90%-accuracy threshold based on the darkest-pixel location is also a coarse metric that may behave differently as N grows. If this scaling statement remains in the paper, it should either be supported by more systematic scaling experiments and uncertainty quantification, or explicitly framed as a qualitative observation about this toy setting.
  3. [Section 3.3, Fig. 1(e)] The claim that non-compositional models implement a "binary factorized kernel" rests on a visual comparison between the model's similarity matrix and the idealized K_binary in Fig. 1(e). No quantitative agreement measure (e.g., correlation coefficient, normalized mutual information, or the decomposition error of D((x,y)_OOD) ≈ D(x_superposed) + D(y_superposed)) is provided. Given that this superposition mechanism is used to argue for a general "memorization by superposition" strategy, I ask for a quantitative evaluation of how well the proposed binary kernel and the superposition equation fit the actual model behavior on held-out OOD inputs.
minor comments (5)
  1. [Appendix E] The references to "Fig. 6(a)" and "Fig. 6(b)" for the stripe dataset appear to be incorrect; the stripe dataset is shown in Fig. 7(a) and sample OOD outputs in Fig. 7(b). Please correct these cross-references.
  2. [Appendix B.1] There is a typo: "wraped" should be "warped".
  3. [Fig. 4(e)] The caption contains "trained trained on a dataset"; the duplicate "trained" should be removed.
  4. [Appendix D] The values of η1, η2, and the rank bound r used in Eqs. (10)-(12) are not reported in the experimental details. Please include them, or the selection procedure, so the regularization experiments can be reproduced.
  5. [Section 3.2] The statement "While not shown, models trained with disentangled 1-hot and positional input encodings also fail" is unverifiable as written; either provide the corresponding results in the appendix or soften the claim.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the failure diagnosis and the proposed remedies are empirical and independently tested; the only flagged concern is a metric-validity caveat, not a circular reduction.

full rationale

The paper's central claims are supported by direct experiments rather than by fitting a parameter and then predicting that same fitted quantity. The OOD failure is measured as MSE between generated and ground-truth images on withheld (x,y) combinations (Fig. 1b-c), the superposition phenomenon is visually and quantitatively characterized (Fig. 1d-e), and the proposed interventions—low-rank embedding regularization and stripe-augmented training—are evaluated on held-out OOD regions (Figs. 3-4). Equation (1) is a standard Schmidt decomposition applied to kernels, not a result whose conclusion is presupposed by its definition. The stripe-training idea is attributed to prior self-citations (Liang et al. 2024a,b), but the present paper independently instantiates and tests that idea in its own CNN/MLP experiments, so the self-citations are not load-bearing. The only substantive concern is that the Jacobian-based factorization metric in Eq. (7) is not invariant under the arbitrary invertible transformations allowed by the paper's own general definition of factorization in App. A.2, so the mechanistic claim of layer-by-layer re-entanglement (Sec. 3.4, Fig. 2) may be an artifact of coordinate choice. That is a validity or correctness issue, not a circularity: the 're-entanglement' conclusion is not equivalent to the metric's definition by construction, and the direct OOD failure evidence and the success of the proposed remedies do not depend on that metric. Accordingly, no step of the derivation reduces to its own inputs, and the paper is best characterized as self-contained empirical work with a measurement caveat.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the experimental construction of disentangled inputs, the standard kernel decomposition, and the Jacobian-based factorization metric. No new physical entities are introduced. The metric's validity is the largest unexamined assumption.

free parameters (3)
  • eta_1 (entropy penalty weight) = Not reported in text
    Controls the entropy penalty on singular values in Eq (10)/(12); its value is needed to reproduce the low-rank regularization result and is left to the code repository.
  • eta_2 (variance penalty weight) = Not reported in text
    Controls the variance penalty in Eq (11)/(12); hand-chosen, specific value deferred to code.
  • rank bound r = Not reported in text
    Upper bound on the rank of each embedding matrix in Eq (8)/(44); chosen by hand and affects the expressiveness/regularization trade-off.
assumptions (4)
  • standard math Positive-semidefinite symmetric square-integrable kernels admit a Schmidt-type sum-of-product decomposition (Eq 1).
    Used in Section 3.3 and Appendix A.4 to justify the binary factorized kernel comparison. This is a standard result for suitable kernels, though the discrete Gram matrix version is an approximation.
  • domain assumption The 1-hot, bump and ramp input encodings used in Figures 1-2 are 'disentangled' because x and y are carried by disjoint neurons.
    The paper's whole setup depends on accepting these encodings as factorized; Section 3.1 states this by construction. If the relevant notion of disentanglement is different (e.g. requires invariant coordinates under all transformations), the negative result is less surprising.
  • ad hoc to paper Layer-wise factorization is measured by the Jacobian metric tensor (Eq 5) and the scalar factorization score (Eq 7), which are assumed to reflect the network's true factorization.
    Section 3.4 uses these to conclude that deeper layers 'warp and remix' factors. However, the paper's own general definition (Appendix A.2) allows arbitrary invertible transformations, under which g_xy is not invariant. The metric's validity for detecting entanglement is therefore an unverified assumption.
  • domain assumption The qualitative agreement between the binary factorized kernel and the empirical similarity matrix (Fig 1(e)) is taken as evidence for the memorization/superposition mechanism.
    Section 3.3 interprets the visual match as evidence that failing models combine memorized ID samples. This is a modeling-interpretation assumption, not a quantitative test.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Compositional Generalization via Forced Rendering of Disentangled Latents." pith.science (2026). https://pith.science/paper/VUT7IF36

@misc{pith2026250118797,
  author       = {Pith},
  title        = {Pith review of: Compositional Generalization via Forced Rendering of Disentangled Latents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VUT7IF36}},
  note         = {Machine review of arXiv:2501.18797}
}
read the original abstract

Composition-the ability to generate myriad variations from finite means-is believed to underlie powerful generalization. However, compositional generalization remains a key challenge for deep learning. A widely held assumption is that learning disentangled (factorized) representations naturally supports this kind of extrapolation. Yet, empirical results are mixed, with many generative models failing to recognize and compose factors to generate out-of-distribution (OOD) samples. In this work, we investigate a controlled 2D Gaussian "bump" generation task with fully disentangled (x,y) inputs, demonstrating that standard generative architectures still fail in OOD regions when training with partial data, by re-entangling latent representations in subsequent layers. By examining the model's learned kernels and manifold geometry, we show that this failure reflects a "memorization" strategy for generation via data superposition rather than via composition of the true factorized features. We show that when models are forced-through architectural modifications with regularization or curated training data-to render the disentangled latents into the full-dimensional representational (pixel) space, they can be highly data-efficient and effective at composing in OOD regions. These findings underscore that disentangled latents in an abstract representation are insufficient and show that if models can represent disentangled factors directly in the output representational space, it can achieve robust compositional generalization.

Figures

Figures reproduced from arXiv: 2501.18797 by the authors.

Figure 1
Figure 1. Various disentangled input/latent encodings fail to support compositional generalization. (a) shows the experimental setup for the 2D Gaussian bump toy experiment, where the scalar (x, y) coordinate pair is encoded into disentangled representation (population-based vs. ramp-based encodings), which is then fed to a decoder-only architecture to generate a N × N grayscale image of a single 2D Gaussian bump centered at … view at source ↗
Figure 2
Figure 2. Cause of failure of compositional generalization despite disentangled inputs/latents: input memorization by decoder undoes factorization. (a-b) show the linear probe metrics of the learned representation as a function of x and y for bump- and ramp￾input-encodings, and (c-d) show the factorization score (Eq. 7), and (e-f) show the volume metric log10(dv), all as a function of layer depth, for models trained with bump… view at source ↗
Figure 3
Figure 3. Inducing compositional generalization through architectural rendering and regularization constraints. (a) Schematic of architecturally forced rendering of the initially disentangled representations (with 1-hot input encodings) into a space matching the output space (disentangled processing). (b) Sampled embedding activations corresponding to x = 14 and y = 14 for networks trained without and with regularization, res… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Inducing generalizable composition by training single disentangled factors to render (data curriculum). (a) Sample grayscale images of 1D Gaussian “stripes” at x = 14 and y = 14. (b) Generated OOD output as a function of number of Gaussian bumps (all outside the upper …
Figure 5
Figure 5. Figure 5: The action of the neural net can be imagined as wrapping of the (originally flat) native space (left) into the embedding space (right). Volume Element. The local volume element is given by dv(x) = q det g(x)  dx1 . . . dxN . (39) This indicates how the network “stretc…
Figure 6
Figure 6. Figure 6: Visualization of x and y embedding matrices without and with regularization. C. Low-rank Regularization for Compositional Generalization In our generative approach, each input factor (e.g. the one-hot code for coordinate x or y) is mapped to a two-dimensional embedding…
Figure 7
Figure 7. Figure 7: Augmenting 2D Gaussian bump dataset with 1D Gaussian stripes enables compositional generalization. (a) shows the 1D Gaussian stripe dataset for N = 28. (b) shows 4 sample generated OOD images for models trained on 1D Gaussian stripes + p% ID 2D Gaussian bumps subsample…
Figure 8
Figure 8. Figure 8: Full similarity/distance/overlap matrices visualization across the entire ID-OOD data distribution, compared between networks with bump (top) and rate (bottom) input encodings. E. Dataset Augmentation One way of improving compositional generalization in our synthetic t…
Figure 9
Figure 9. Figure 9: MNIST digit “4” image rotation experiment with OOD range of 160◦ − 200◦ . (a) shows the reconstruction MSE as a function of rotation angle. (b) shows the 2D UMAP embeddings for representations at different layers of the network. (c) and (d) shows the generated rotated …
Figure 10
Figure 10. Figure 10: Comparison of generated OOD images for different network depths between networks with bump-based (left) and ramp-based (right) input encodings. (middle plot) or x (right plot). Across the layers, we observe that early-layer neurons primarily capture stripe-like featur…
Figure 11
Figure 11. Figure 11: Evolution of manifold density and linear probe as a function of layer depth for different CNN network depths (bump encoding) [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]
Figure 12
Figure 12. Figure 12: Evolution of manifold density and linear probe as a function of layer depth for different CNN network depths (rate encoding). 24 [PITH_FULL_IMAGE:figures/full_fig_p024_12.png]
Figure 13
Figure 13. Figure 13: Evolution of manifold density and linear probe as a function of layer depth for different MLP network depths (bump encoding) [PITH_FULL_IMAGE:figures/full_fig_p025_13.png]
Figure 14
Figure 14. Figure 14: Evolution of manifold density and linear probe as a function of layer depth for different MLP network depths (bump encoding). 25 [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]
Figure 15
Figure 15. Figure 15: Neural tuning curves for sample neurons across different layers of a 4-layer CNN trained on a dataset of only stripes (left) and stripes + 100 bumps (right). 26 [PITH_FULL_IMAGE:figures/full_fig_p026_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 22 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    C., and Vincent, P

    Bengio, Y., Courville, A. C., and Vincent, P. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. , 35 0 (8): 0 1798--1828, 2013. doi:10.1109/TPAMI.2013.50. URL https://doi.org/10.1109/TPAMI.2013.50

    doi:10.1109/tpami.2013.50 2013

  3. [3]

    P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A

    Burgess, C. P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A. Understanding disentangling in \( \) -vae. CoRR, abs/1804.03599, 2018. URL http://arxiv.org/abs/1804.03599

    arXiv 2018

  4. [4]

    Q., Li, X., Grosse, R

    Chen, T. Q., Li, X., Grosse, R. B., and Duvenaud, D. Isolating sources of disentanglement in variational autoencoders. In Bengio, S., Wallach, H. M., Larochelle, H., Grauman, K., Cesa - Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, D...

    work page 2018

  5. [5]

    Infogan: Interpretable representation learning by information maximizing generative adversarial nets

    Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Lee, D. D., Sugiyama, M., von Luxburg, U., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information P...

    work page 2016

  6. [6]

    H., and Amari, S

    Cichocki, A., Zdunek, R., Phan, A. H., and Amari, S. Nonnegative Matrix and Tensor Factorizations - Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, 2009. ISBN 978-0-470-74666-0. doi:10.1002/9780470747278. URL https://doi.org/10.1002/9780470747278

    work page doi:10.1002/9780470747278 2009

  7. [7]

    N., Hawes, N., and Posner, I

    DeMoss, B., Sapora, S., Foerster, J. N., Hawes, N., and Posner, I. The complexity dynamics of grokking. CoRR, abs/2412.09810, 2024. doi:10.48550/ARXIV.2412.09810. URL https://doi.org/10.48550/arXiv.2412.09810

    arXiv doi:10.48550/arxiv.2412.09810 2024

  8. [8]

    and Williams, C

    Eastwood, C. and Williams, C. K. I. A framework for the quantitative evaluation of disentangled representations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. URL https://openreview.net/forum?id=By-7dz-AZ

    work page 2018

Show all 29 references
  1. [9]

    Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., and Lempitsky, V. S. Domain-adversarial training of neural networks. In Csurka, G. (ed.), Domain Adaptation in Computer Vision Applications, Advances in Computer Vision and Pattern ...

    2017 doi

  2. [10]

    P., Schwartz, A

    Georgopoulos, A. P., Schwartz, A. B., and Kettner, R. E. Neuronal population coding of movement direction. Science, 233 0 (4771): 0 1416--1419, 1986. doi:10.1126/science.3749889

    1986 doi

  3. [11]

    P., Glorot, X., Botvinick, M

    Higgins, I., Matthey, L., Pal, A., Burgess, C. P., Glorot, X., Botvinick, M. M., Mohamed, S., and Lerchner, A. beta-vae: Learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, Fra...

    2017

  4. [12]

    P., Glorot, X., Botvinick, M

    Higgins, I., Matthey, L., Pal, A., Burgess, C. P., Glorot, X., Botvinick, M. M., Mohamed, S., and Lerchner, A. beta-vae: Learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, Fra...

    2017

  5. [13]

    and Ganguli, S

    Kamb, M. and Ganguli, S. An analytic theory of creativity in convolutional diffusion models. CoRR, abs/2412.20292, 2024. doi:10.48550/ARXIV.2412.20292. URL https://doi.org/10.48550/arXiv.2412.20292

    2024 doi arXiv

  6. [14]

    and Mnih, A

    Kim, H. and Mnih, A. Disentangling by factorising. In Dy, J. G. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm \" a ssan, Stockholm, Sweden, July 10-15, 2018 , volume 80 of Proceedings of Machine Learning Res...

    2018

  7. [15]

    Lake, B. M. and Baroni, M. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In Dy, J. G. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm \" a ssan, ...

    2018

  8. [16]

    Liang, Q., Liu, Z., and Fiete, I. R. Do diffusion models learn semantically meaningful and efficient representations? In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024 a . URL https://openreview.net/forum?id=pAAMFQiAuD

    2024

  9. [17]

    How diffusion models learn to factorize and compose

    Liang, Q., Liu, Z., Ostrow, M., and Fiete, I. How diffusion models learn to factorize and compose. In 38th Conference on Neural Information Processing Systems, NeurIPS 2024, 2024 b

    2024

  10. [18]

    and Stachenfeld, K

    Lippl, S. and Stachenfeld, K. When does compositional structure yield compositional generalization? A kernel theory. CoRR, abs/2405.16391, 2024. doi:10.48550/ARXIV.2405.16391. URL https://doi.org/10.48550/arXiv.2405.16391

    2024 doi arXiv

  11. [19]

    a tsch, G., Gelly, S., Sch \

    Locatello, F., Bauer, S., Lucic, M., R \" a tsch, G., Gelly, S., Sch \" o lkopf, B., and Bachem, O. Challenging common assumptions in the unsupervised learning of disentangled representations. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International...

    2019

  12. [20]

    Lost in latent space: Examining failures of disentangled models at combinatorial generalisation

    Montero, M., Bowers, J., Ponte Costa, R., Ludwig, C., and Malhotra, G. Lost in latent space: Examining failures of disentangled models at combinatorial generalisation. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Informa...

    2022

  13. [21]

    L., Ludwig, C

    Montero, M. L., Ludwig, C. J. H., Costa, R. P., Malhotra, G., and Bowers, J. S. The role of disentanglement in generalisation. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. URL https://open...

    2021

  14. [22]

    S., Dick, R

    Okawa, M., Lubana, E. S., Dick, R. P., and Tanaka, H. Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task. CoRR, abs/2310.09336, 2023. doi:10.48550/ARXIV.2310.09336. URL https://doi.org/10.48550/arXiv.2310.09336

    2023 doi arXiv

  15. [23]

    and Nadel, L

    O'Keefe, J. and Nadel, L. The Hippocampus as a Cognitive Map. Oxford University Press, Oxford, UK, 1978

    1978

  16. [24]

    Shadlen, M. N. and Newsome, W. T. The variable discharge of cortical neurons: implications for connectivity, computations, and coding. The Journal of Neuroscience, 18 0 (10): 0 3870--3896, 1998. doi:10.1523/JNEUROSCI.18-10-03870.1998

    1998

  17. [25]

    Compositional generalization from first principles

    Wiedemer, T., Mayilvahanan, P., Bethge, M., and Brendel, W. Compositional generalization from first principles. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neura...

    2023

  18. [26]

    Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language

    Xu, Z., Niethammer, M., and Raffel, C. Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural In...

    2022

  19. [27]

    Vector-based representation is the key: A study on disentanglement and compositional generalization

    Yang, T., Wang, Y., Lan, C., Lu, Y., and Zheng, N. Vector-based representation is the key: A study on disentanglement and compositional generalization. CoRR, abs/2305.18063, 2023. doi:10.48550/ARXIV.2305.18063. URL https://doi.org/10.48550/arXiv.2305.18063

    2023 doi arXiv

  20. [28]

    A., Yang, S., Rubinfien, J

    Zavatone - Veth, J. A., Yang, S., Rubinfien, J. A., and Pehlevan, C. Neural networks learn to magnify areas near decision boundaries. CoRR, abs/2301.11375, 2023. doi:10.48550/ARXIV.2301.11375. URL https://doi.org/10.48550/arXiv.2301.11375

    2023 doi

  21. [29]

    D., and Ermon, S

    Zhao, S., Ren, H., Yuan, A., Song, J., Goodman, N. D., and Ermon, S. Bias and generalization in deep generative models: An empirical study. In Bengio, S., Wallach, H. M., Larochelle, H., Grauman, K., Cesa - Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Pr...

    2018

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.