Pith. sign in

REVIEW 3 major objections 5 minor 80 references

Identifiable Object Representations under Spatial Ambiguities

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that object representations learned from multiple camera views remain identifiable up to a permutation of the objects and an affine transformation, even when objects are partly or fully occluded in individual views, and…

desk verdict VISA's architecture and datasets are useful, but the identifiability proof doesn't hold as written due to a flawed aggregation lemma. read the letter →

arxiv 2506.07806 v1 pith:7U5ZALYD submitted 2025-06-09 cs.LG cs.CV

Avinash Kori , Francesca Toni , Ben Glocker This is my paper
classification cs.LGcs.CV
keywords object-centricrepresentationlearningmulti-viewidentifiabilityGaussianmixturemodelsslotattentionocclusionviewinvarianceprobabilisticgenerative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to prove that a multi-view object-centric learner can recover identifiable object content — the same latent representation up to a permutation of the objects and an affine transformation — even when objects are partially or fully occluded in some views, and without any viewpoint annotations. Identifiability of this kind is what makes object-centric representations dependable building blocks for downstream reasoning, and the authors position their result as the first formal guarantee for object representations under occlusion with multi-view observational data. The route runs through Gaussian mixtures: each view's slot posterior is a spatial Gaussian mixture, slots are aligned across views, and the content aggregator combines them so that the content posterior is again a Gaussian mixture, which brings the aggregate within reach of an existing identifiability theorem for mixture-based generative models. The paper supports the theory with synthetic visualizations and benchmarks on CLEVR-MV, GQN, and GSO, plus two new large multi-view datasets, MV-MOVIC and MV-MOVID.

What carries the argument

The load-bearing object is the aggregate content posterior constructed by VISA, the paper's View-Invariant Slot Attention. Each view's image is mapped through a learned inverse view transformation $T_{\theta_v}^{-1}$ into a shared implicit space, where probabilistic slot attention fits a spatial Gaussian mixture model over slots; matched slots are aligned with a permutation estimated by Hungarian matching and combined across views by a convex combination whose weights come from the slot mixing coefficients. Lemma F.3 shows that such a convex combination of view-specific slot Gaussian mixtures is again a Gaussian mixture, and Lemma 5.2 shows the dataset-wide aggregate is a non-degenerate global mixture with $MK$ components. With the mixing function assumed weakly injective (Assumption F.4), the identifiability theorem for Gaussian-mixture generative models applies and yields the affine-and-permutation equivalence of Theorem 5.3.

What would settle it

Train VISA on scenes in which the same object appears in several views under a per-scene correlated nuisance, such as an illumination or blur pattern shared across all views and correlated with the object's appearance, and then measure whether matched slots for one object index across views have negligible empirical covariance $\mathbb{E}[s_k^1 (s_k^2)^\top]$; a non-negligible covariance together with two independent training runs whose content posteriors are not related by a single permutation and affine map would refute Theorem 5.3. More directly, on a dataset with known generative factors, train two independent VISA models and test whether the recovered content distributions are related by exactly one permutation and one affine transformation, since any recovered pair requiring a non-affine correspondence between runs falsifies the claimed equivalence.

Watch

Extended reading notes

Core claim

The central claim is Theorem 5.3: for any nonempty subset $A$ of views, the aggregate content $c$ is identifiable up to the $\sim_s$ equivalence relation — two learned models are equivalent when their inverse mappings agree after a permutation of object indices, an affine matrix, and a shift — provided the view-specific slot posterior is a non-degenerate Gaussian mixture. Theorem 5.4 adds invariance: models trained on different viewpoint subsets, each satisfying the viewpoint-sufficiency condition that the union of views covers every object in the scene, recover aggregate content in the same equivalence class. Theorem 5.5 adds approximate equivariance: a homeomorphic transformation applied to the scene induces a corresponding homeomorphic transformation of the learned view representation. The authors claim these are the first identifiability guarantees for object-centric representations under occlusion in a multi-view setting, with no view annotations and no paired viewpoint-image data required.

Load-bearing premise

The proof requires that slots describing the same object in different views be statistically uncorrelated, because that is what makes a convex combination of the per-view Gaussian mixtures a Gaussian mixture again; the paper never states this condition as an assumption.

Editorial extensions

If this is right

  • Identifiable slots from independent training runs can be passed to downstream modules that assume stable object identities, such as compositional generation or causal discovery, with only a fixed affine correction needed to align runs.
  • Occlusion is handled by construction: an object hidden in one view still contributes its content through the views where it is visible, and the guarantee holds as long as the union of views covers every object, not every view.
  • Because viewpoint information is a learned global latent rather than an annotation, the method transfers to scenes with unknown or inconsistent camera poses, which the out-of-domain MV-MOVIC and MV-MOVID evaluations probe.
  • The identifiability guarantee is not tied to one decoder family; the paper verifies it with additive MLP and spatial-broadcasting decoders and with non-additive transformer decoders, subject to weak injectivity of the mixing function.
  • Viewpoint sufficiency replaces the stronger requirement that every object be visible in every view, so partial occlusion in individual views is explicitly allowed by the theory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The unstated uncorrelated-slots condition is the link most likely to break on real data: if object appearance is coherently correlated across views, such as specular material seen by several cameras, the aggregated posterior can leave the Gaussian-mixture family and the proved route collapses even if the architecture still works empirically.
  • A natural extension the paper does not pursue is to treat the identifiable content space as a fixed feature bank for transfer: because ambiguity is resolved up to one global affine map, a single calibration step per new environment should place its slots in a shared coordinate frame.
  • The affine ambiguity marks a boundary for downstream use: tasks that need metric-faithful representations, such as exact 3D coordinates or scale-sensitive measurements, would have to pin down the affine map, for instance by anchoring one object of known size.
  • The view-warm-up heuristic, which randomly decodes from view-specific slots early in training, suggests a testable hypothesis: the uncorrelatedness condition may be induced by training dynamics rather than guaranteed by the model class, so removing warm-up should measurably degrade identifiability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes VISA, a multi-view object-centric representation learning method that aggregates view-specific slot representations (each modeled as a spatial Gaussian mixture model) into a view-invariant content representation, without using viewpoint annotations. The central theoretical claim is that the aggregate content representation is identifiable up to permutation and affine transformation (Theorem 5.3), with additional invariance and approximate equivariance results (Theorems 5.4 and 5.5). The proof strategy is to show that the aggregate content posterior is a non-degenerate Gaussian mixture model and then invoke existing identifiability results from Kivva et al. (2022) and Kori et al. (2024). The authors also present experiments on synthetic data, standard benchmarks, and two new datasets, MV-MOVI-C and MV-MOVI-D.

Significance. If the theoretical claims were fully established, this would be the first identifiable object-centric representation result for multi-view scenes under occlusion without auxiliary viewpoint information, and the two new datasets would be a useful community resource. The paper also gives empirical evidence that their method improves over single-view baselines in terms of slot mean correlation coefficient. However, the theoretical core is not sound as written: the key lemmas used to establish the Gaussian-mixture form of the aggregate posterior contain unjustified steps, and the main identifiability theorem therefore does not follow. Since the identifiability guarantee is the paper's headline contribution, the gaps are load-bearing.

major comments (3)
  1. [Appendix F, Lemma F.3, Eq. (36)] The proof of Lemma F.3 uses the identity E[s1_k (s2_k)^T] = μ1_k μ2_k^T to evaluate the cross term in the covariance of the convex combination. This identity holds only if the same-index slots from different views are uncorrelated. The lemma statement does not include this condition, and it is not stated as an assumption anywhere in the paper (Assumption 5.1 concerns viewpoint sufficiency, and Assumption F.4 concerns weak injectivity). In the VISA generative model, slots for the same object across views are both derived from the same latent content c and are therefore generally correlated; indeed, exploiting this shared content is the purpose of the aggregation. Without the uncorrelatedness condition, the covariance of the aggregate slot is not given by Eq. (37), and the claim that the aggregate content posterior q(c|s) is a K-component GMM is not established. Consequently, the application of the Kivva et al. (2022) identifiability theorem in Theorem 5.3 lacks its required premise.
  2. [Section 5, Lemma 5.2, Eq. (55)] The proof of Lemma 5.2 collapses the per-scene posterior q(z^A | x^A_i) to a Dirac delta at its mean (Eq. (55)), which is a heuristic approximation, not a derivation. The text acknowledges this and attempts a 'general case' derivation in Eqs. (58)-(64), but that derivation is also problematic: the product of GMMs across |A| views in Eq. (60) would, in general, have K^{|A|} components, not K, unless an alignment/independence condition is imposed. The assertion that aligned components reduce the product to K components is not justified. Therefore, the non-degenerate MK-component GMM form of the aggregate posterior in Eq. (10), which is the foundation for Theorem 5.3, is not rigorously established.
  3. [Section 5, Theorem 5.3 proof, Eqs. (69)-(85)] The proof of Theorem 5.3 begins with the statement 'Based on the results of (Kori et al., 2024) we know that when p(s) is aggregate posterior of q(s|x), p(s) is identifiable up to ∼s equivalence' and then says that p(c) is identifiable 'using similar arguments' without providing the detailed argument. Since the GMM structure of p(c) is not established (per the two preceding comments), the conclusion does not follow. Moreover, the explicit derivation of p(c) in Eqs. (77)-(80) is hard to follow: Eq. (80) writes the density as N(v; ...) instead of N(c; ...), and the summation indices and ranges are incorrect. The proof thus leaves the central claim unsupported.
minor comments (5)
  1. [Abstract] The sentence 'However, addressing challenges presents both theoretical and practical difficulties' is grammatically incomplete and should be rephrased.
  2. [Appendix F, Lemma F.3] The notation is inconsistent: s1 and s2 are introduced as sets of K random vectors, but then treated as random variables with a GMM distribution. Additionally, the aggregator in Eq. (7) uses mixing-coefficient-dependent weights, whereas Lemma F.3 considers arbitrary convex weights w1, w2; the relationship between the two should be clarified.
  3. [Definition 3.1] The equivalence relation is written as f^{-1}_{θ1}(x;v) = P(f^{-1}_{θ2}(x;v) H + a), which mixes a permutation P with an affine map H and offset a. It would be clearer to define an affine transformation A(z) = H z + a and write the relation as f^{-1}_{θ1}(x;v) = P(A(f^{-1}_{θ2}(x;v))).
  4. [Section 6] In Tables 2 and 4, the INV-SMCC and MCC entries for SA, PSA, and other single-view baselines are left as '-', which is understandable since these models do not estimate view information, but the caption should state explicitly that these metrics are undefined for those baselines rather than forcing the reader to infer it.
  5. [Throughout] There are several typos, such as 'spacial' (Section 1) and 'allows our approach to resolve' (Abstract). A careful proofreading pass is recommended.

Circularity Check

3 steps flagged · score 6.0 of 10

Theorem 5.3's aggregate-content identifiability is inherited from the same authors' slot-identifiability result and is made to look like a GMM proof only via an unstated cross-view uncorrelatedness assumption in Lemma F.3 plus a Dirac-delta collapse in Lemma 5.2.

  1. self citation load bearing [Theorem 5.3 proof, Section 5 (first sentence of the proof)]
    "Based on the results of (Kori et al., 2024) we know that when p(s) is aggregate posterior of q(s|x), p(s) is identifiable up to ∼s equivalence. Additionally, based on lemma 5.2 we know that both q(s|x) and q(c|s) are a non-degenerate GMM with valid probability distribution. Using similar arguments in (Kori et al., 2024; Kivva et al., 2022) we show that p(c) and p(s) are identifiable up to ∼s equivalence."

    The central identifiability claim for aggregate content c is not proved from the model's assumptions alone: the proof starts by importing slot identifiability from the same authors' prior work (Kori et al., 2024) and then says the content case follows by 'similar arguments'. Thus the load-bearing step of the main theorem reduces to a self-citation whose content is not re-derived or independently verified. If the cited slot-identifiability result were removed, Theorem 5.3 would have no proof of the p(s) premise and hence no route to p(c).

  2. other [Lemma F.3, Eq. (36), Appendix F]
    "Then, substitute known expectations: E[s_i_k(s_i_k)^T] = Σ_i,k + μ_i,k μ_i,k^T (35); E[s1_k(s2_k)^T] = μ1,k μ2,k^T (36)."

    The claimed GMM form of the aggregate content posterior, which is the premise needed to invoke the Kivva et al. identifiability theorem, is obtained by substituting E[s1_k s2_k^T] = μ1,k μ2,k^T. That identity holds only if slots for the same object index across different views are uncorrelated, but no such assumption is stated anywhere in the paper. In the VISA generative model, slots for the same object across views are both derived from the same content c and should generally be correlated. Without this hidden independence condition, w1 s1_k + w2 s2_k is not necessarily Gaussian, and the resulting distribution need not be a K-component GMM; the identifiability conclusion is therefore forced by an unstated assumption rather than derived.

1 more flagged steps
  1. other [Lemma 5.2 proof, Eq. (55), Appendix F]
    "≈ ∫ 1/M Σ_{i=1}^M δ(z^A − μ(x^A_i)) · Σ_{k=1}^K π_k(x^A_i) N(z^A; μ_k(x^A_i), σ^2_k(x^A_i)) dz^A (55)"

    To establish that the aggregate posterior p(c) is a non-degenerate GMM, the proof replaces the slot posterior q(z^A | x^A_i) by a Dirac delta at its mean. This collapses the uncertainty in the slot posterior and turns the product of Gaussians into a GMM by flat. The approximation is not justified and is exactly what makes the number of mixture components remain K instead of growing. The non-degenerate GMM premise of Theorem 5.3 is thus manufactured by an unverified collapse rather than being a consequence of the stated modelling assumptions, making the theorem's conclusion depend on a step that assumes the desired structure.

full rationale

The paper's advertised contribution is a proof that aggregate content representations are identifiable under occlusions and view ambiguities. Walking the derivation chain, the proof of Theorem 5.3 begins with the same group's slot-identifiability result (Kori et al., 2024) and then extends it to content using Lemma 5.2 and Lemma F.3. The self-citation is load-bearing because no independent proof of the slot-level premise is supplied, and the content-level conclusion is presented as following from 'similar arguments'. The aggregation step is supposed to show that q(c|s) is a K-component GMM, but Lemma F.3's proof uses E[s1_k s2_k^T] = μ1_k μ2_k^T, i.e., cross-view uncorrelatedness for the same object, which is never stated as an assumption and is inconsistent with the model's shared-content generative process. Lemma 5.2 further replaces the slot posterior by a Dirac delta at its mean, collapsing uncertainty to obtain the desired GMM form. These two steps are not merely missing details: they are the steps that make the aggregate content posterior a non-degenerate GMM, which is the exact condition required to import the Kivva et al. identifiability theorem. As written, therefore, the central theoretical guarantee is partly inherited from a self-citation and partly forced by hidden assumptions and approximations, rather than derived from the stated generative model. The empirical sections are extensive and provide supporting evidence, but they do not repair the circular or assumption-dependent structure of the proof. Overall score 6 reflects partial circularity: the conclusion does not fully reduce to the inputs by construction, but the main theorem's validity rests on an unstated independence condition and a self-cited slot-identifiability result.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central identifiability theorem rests on (a) the prior PSA identifiability result of the same group, (b) the unstated assumption that view-specific slots for the same object are uncorrelated across views (Lemma F.3), (c) viewpoint sufficiency, and (d) weak injectivity of piecewise affine mixing functions. The uncorrelatedness assumption is introduced ad hoc and is load-bearing; it is not justified anywhere in the paper.

free parameters (3)
  • Number of slots K = 7 for CLEVR-MV, GSO, MV-MOVI; 4 for GQN
    Chosen by hand per dataset; the GMM component count equals the assumed maximum number of objects, and the theory takes K as given.
  • Number of views V per dataset = 10 for CLEVR-MV and GQN; 8 for GSO; 5 for MV-MOVI
    Dataset design choice. Figure 12 shows identifiability metrics increase with V, so V is an effective hyperparameter controlling the strength of the claimed guarantees.
  • View warm-up probability and duration = 0.5 probability, 100,000 iterations (Appendix G.5)
    Ad hoc training schedule to stabilize the content aggregator; not derived from the theory and not analyzed.
assumptions (6)
  • standard math Aggregate posterior q(c) is the optimal prior p(c) (Hoffman & Johnson 2016; Kori et al. 2024)
    Invoked in Lemma 5.2 to equate the marginal of per-scene posteriors with the model prior; standard ELBO surgery result.
  • domain assumption Viewpoint sufficiency (Assumption 5.1): the union of objects visible across the views in A contains all objects in the environment
    Needed for Theorems 5.4 and 5.5 (invariance and equivariance). The authors acknowledge in Section 7 that it is strong and may not hold in practice; they only test it indirectly on MV-MOVID.
  • domain assumption Weak injectivity of the mixing function (Assumption F.4)
    Inherited from Kivva et al. 2022; required for the generic-point and measure arguments. The authors note in Section 7 that weak injectivity may not hold for all architectures.
  • domain assumption Mixing functions are piecewise affine (LeakyReLU networks)
    Used to invoke Kivva et al. Theorems F.8 to F.10. Deviations from piecewise affinity, such as LayerNorm in some MLP decoders, are not analyzed.
  • ad hoc to paper Slots across views are independent in the aggregate posterior
    Implicit in Eqn. 70 where p(s^A) is written as a product over views. Not stated as an assumption; without it the content aggregation does not provably yield the GMM needed for identifiability.
  • ad hoc to paper Within Lemma F.3, slots from different views for the same object index are uncorrelated (E[s1_k s2_k^T] = μ1_k μ2_k^T)
    Assumed in the covariance computation of the convex combination of GMMs. Likely false for correlated slots of the same object; this is the most fragile step of the theory.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identifiable Object Representations under Spatial Ambiguities." pith.science (2026). https://pith.science/paper/7U5ZALYD

@misc{pith2026250607806,
  author       = {Pith},
  title        = {Pith review of: Identifiable Object Representations under Spatial Ambiguities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7U5ZALYD}},
  note         = {Machine review of arXiv:2506.07806}
}
read the original abstract

Modular object-centric representations are essential for *human-like reasoning* but are challenging to obtain under spatial ambiguities, *e.g. due to occlusions and view ambiguities*. However, addressing challenges presents both theoretical and practical difficulties. We introduce a novel multi-view probabilistic approach that aggregates view-specific slots to capture *invariant content* information while simultaneously learning disentangled global *viewpoint-level* information. Unlike prior single-view methods, our approach resolves spatial ambiguities, provides theoretical guarantees for identifiability, and requires *no viewpoint annotations*. Extensive experiments on standard benchmarks and novel complex datasets validate our method's robustness and scalability.

Figures

Figures reproduced from arXiv: 2506.07806 by the authors.

Figure 1
Figure 1. (a) Occlusion Ambiguity: the orange object, which is occluded by the blue object, could be any of the six plausible objects shown on the right. (b) View Ambiguity: the blue object is observed from two different viewpoints (represented with a red arrow and a dot), leading to a change in its overall shape. In general, identifiable representations resolve ambiguities by determining the most plausible object under occlu… view at source ↗
Figure 2
Figure 2. Identifiability across a number of views measured with Slot Mean Correlation Coefficient (SMCC). O4 in x 1 and x 2 can be interpreted as a cube, but only after considering x 3 we can conclude that being a pyramid. A handful of approaches, including MULMON(Li et al., 2020), DYMON(Li et al., 2021), OCLOC(Yuan et al., 2024), have considered multiple viewpoints for extracting object representations. Additionally, method… view at source ↗
Figure 4
Figure 4. Graphical model for multi-view probabilistic slot at￾tention: For every image in a dataset a view v ∈ R dv ∼ p(v), this view is used to compute transformation Tθv . Similarly, desired number (< K) of content representations c ∈ R N×ds are sampled content distribution p(c). Finally, the image x is generated using the transformed content Tθv (c) and view v. where P ∈ P ⊆ {0, 1} K×K is a permutation matrix, H ∈ R d×d i… view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Identifiability of q(c). The top row indicates individual feature distribution across five different runs. The bottom row reflects the feature feature distribution, which we use as a proxy for multi-dimensional features given Lemma F.2. As observed, mean feature distri…
Figure 6
Figure 6. Figure 6: Viewpoint invariance for q(c). The top and bottom row indicates individual feature levels and mean feature distributions, respectively. Each columns reflect marginalised aggregate content distribution q(c) when trained with different view pairs {(blue, red), (green, bl…
Figure 7
Figure 7. Figure 7: Data generating process: The figure illustrates 3D point cloud data in the first row, with camera location highlighted in red, blue, and green arrow. Following rows indicates projected images and point cloud as observed from red, blue, and green cameras, respectively. …
Figure 8
Figure 8. Figure 8: Identifiability of q(c) and q(s). Estimated marginalised slot distribution (q(s)–blue contours) and marginalised content distribution (q(c)–orange contours, across 4 runs of VISA. This provides strong evidence of recovery of the latent space up to affine transformation…
Figure 9
Figure 9. Figure 9: Viewpoint invariance for q(c). Estimated marginalised aggregate content distribution q(c) when trained with different view pairs {(green, red), (red, blue), (green, blue)} are illustrated in later figures. As the resulting distributions with different datasets only var…
Figure 10
Figure 10. Figure 10: Visual illustrations of benchmark results on CLEVR-AUG dataset. G.6 Hyperparameters In [PITH_FULL_IMAGE:figures/full_fig_p029_10.png]
Figure 11
Figure 11. Figure 11: Visual illustrations of benchmark results on CLEVR-MV dataset. features rather than the original image as proposed in (Seitzer et al., 2022). For most of hyperparameters we use the values suggested by (Locatello et al., 2020b; Seitzer et al., 2022), based on their abl…
Figure 12
Figure 12. Figure 12: Influence of Number of viewpoints on identifiability for synthetic datasets. (a) (b) [PITH_FULL_IMAGE:figures/full_fig_p031_12.png]
Figure 13
Figure 13. Figure 13: Visual illustrations of benchmark results on MVMOVI-C dataset with 2 views. G.7 Computational Resources We run all our experiments on a cluster with a Nvidia NVIDIA L40 48GB GPU cards. Our training usually takes between eight hours to a couple of days, depending on th…
Figure 14
Figure 14. Figure 14: Visual illustrations of benchmark results on MVMOVI-C dataset with 3 views [PITH_FULL_IMAGE:figures/full_fig_p032_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

80 extracted references · 50 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Interventional Causal Representation Learning

    Ahuja, K., Wang, Y., Mahajan, D., and Bengio, Y. Interventional causal representation learning. arXiv preprint arXiv:2209.11924, 2022

    work page Pith review arXiv 2022

  3. [3]

    D., and Tenenbaum, J

    Arsalan Soltani, A., Huang, H., Wu, J., Kulkarni, T. D., and Tenenbaum, J. B. Synthesizing 3d shapes via modeling multi-view depth maps and silhouettes with deep generative networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 1511--1519, 2017

    work page 2017

  4. [4]

    W., Hamrick, J

    Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018

    arXiv 2018

  5. [5]

    Representation learning: A review and new perspectives

    Bengio, Y., Courville, A., and Vincent, P. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35 0 (8): 0 1798--1828, 2013

    2013

  6. [6]

    o lkopf, B., von K \

    Brady, J., Zimmermann, R. S., Sharma, Y., Sch \"o lkopf, B., von K \"u gelgen, J., and Brendel, W. Provably learning object-centric representations. arXiv preprint arXiv:2305.14229, 2023

    arXiv 2023

  7. [7]

    Brehmer, J., De Haan, P., Lippe, P., and Cohen, T. S. Weakly supervised causal representation learning. Advances in Neural Information Processing Systems, 35: 0 38319--38331, 2022

    2022

  8. [8]

    Function classes for identifiable nonlinear independent component analysis

    Buchholz, S., Besserve, M., and Sch \"o lkopf, B. Function classes for identifiable nonlinear independent component analysis. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022

    work page 2022

Show all 80 references
  1. [9]

    P., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M., and Lerchner, A

    Burgess, C. P., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M., and Lerchner, A. Monet: Unsupervised scene decomposition and representation. arXiv preprint arXiv:1901.11390, 2019

    1901 arXiv

  2. [10]

    Emerging properties in self-supervised vision transformers

    Caron, M., Touvron, H., Misra, I., J \'e gou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 9650--9660, 2021

    2021

  3. [11]

    L., and Levine, S

    Chang, M., Griffiths, T. L., and Levine, S. Object representations as fixed points: Training iterative refinement algorithms with implicit differentiation. arXiv preprint arXiv:2207.00787, 2022

    2022 arXiv

  4. [12]

    Roots: Object-centric representation and rendering of 3d scenes

    Chen, C., Deng, F., and Ahn, S. Roots: Object-centric representation and rendering of 3d scenes. Journal of Machine Learning Research, 22 0 (259): 0 1--36, 2021

    2021

  5. [13]

    and Pineau, J

    Crawford, E. and Pineau, J. Exploiting spatial invariance for scalable unsupervised object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp.\ 3684--3692, 2020

    2020

  6. [14]

    Daunhawer, I., Bizeul, A., Palumbo, E., Marx, A., and Vogt, J. E. Identifiability results for multimodal contrastive learning. arXiv preprint arXiv:2303.09166, 2023

    2023 arXiv

  7. [15]

    A., Garnelo, M., Lee, M

    Dilokthanakul, N., Mediano, P. A., Garnelo, M., Lee, M. C., Salimbeni, H., Arulkumaran, K., and Shanahan, M. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648, 2016

    2016 arXiv

  8. [16]

    B., and Vanhoucke, V

    Downs, L., Francis, A., Koenig, N., Kinman, B., Hickman, R., Reymann, K., McHugh, T. B., and Vanhoucke, V. Google scanned objects: A high-quality dataset of 3d scanned household items. In 2022 International Conference on Robotics and Automation (ICRA), pp.\ 2553--2560. IEEE, 2022

    2022

  9. [17]

    and Williams, C

    Eastwood, C. and Williams, C. K. I. A framework for the quantitative evaluation of disentangled representations. In International Conference on Learning Representations, 2018

    2018

  10. [18]

    Slot order matters for compositional scene understanding

    Emami, P., He, P., Ranka, S., and Rangarajan, A. Slot order matters for compositional scene understanding. arXiv preprint arXiv:2206.01370, 2022

    2022 arXiv

  11. [19]

    R., Jones, O

    Engelcke, M., Kosiorek, A. R., Jones, O. P., and Posner, I. Genesis: Generative scene inference and sampling with object-centric latent representations. arXiv preprint arXiv:1907.13052, 2019

    1907 arXiv

  12. [20]

    Genesis-v2: Inferring unordered object representations without iterative refinement

    Engelcke, M., Parker Jones, O., and Posner, I. Genesis-v2: Inferring unordered object representations without iterative refinement. Advances in Neural Information Processing Systems, 34: 0 8085--8094, 2021

    2021

  13. [21]

    A., Jimenez Rezende, D., Besse, F., Viola, F., Morcos, A

    Eslami, S. A., Jimenez Rezende, D., Besse, F., Viola, F., Morcos, A. S., Garnelo, M., Ruderman, A., Rusu, A. A., Danihelka, I., Gregor, K., et al. Neural scene representation and rendering. Science, 360 0 (6394): 0 1204--1210, 2018

    2018

  14. [22]

    D., Lagnado, D

    Gerstenberg, T., Goodman, N. D., Lagnado, D. A., and Tenenbaum, J. B. A counterfactual simulation model of causal judgments for physical events. Psychological review, 128 0 (5): 0 936, 2021

    2021

  15. [23]

    M., Schulz, L

    Gopnik, A., Glymour, C., Sobel, D. M., Schulz, L. E., Kushnir, T., and Danks, D. A theory of causal learning in children: causal maps and bayes nets. Psychological review, 111 0 (1): 0 3, 2004

    2004

  16. [24]

    L., Kabra, R., Watters, N., Burgess, C., Zoran, D., Matthey, L., Botvinick, M., and Lerchner, A

    Greff, K., Kaufman, R. L., Kabra, R., Watters, N., Burgess, C., Zoran, D., Matthey, L., Botvinick, M., and Lerchner, A. Multi-object representation learning with iterative variational inference. In International Conference on Machine Learning, pp.\ 2424--2433. PMLR, 2019

    2019

  17. [25]

    On the binding problem in artificial neural networks

    Greff, K., Van Steenkiste, S., and Schmidhuber, J. On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208, 2020

    2012 arXiv

  18. [26]

    J., Gnanapragasam, D., Golemo, F., Herrmann, C., et al

    Greff, K., Belletti, F., Beyer, L., Doersch, C., Du, Y., Duckworth, D., Fleet, D. J., Gnanapragasam, D., Golemo, F., Herrmann, C., et al. Kubric: A scalable dataset generator. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 3749--3761, 2022

    2022

  19. [27]

    K., Mehrjou, A., Locatello, F., and Sch \"o lkopf, B

    Gresele, L., Rubenstein, P. K., Mehrjou, A., Locatello, F., and Sch \"o lkopf, B. The incomplete rosetta stone problem: Identifiability results for multi-view nonlinear ica. In Uncertainty in Artificial Intelligence, pp.\ 217--227. PMLR, 2020

    2020

  20. [28]

    and Zisserman, A

    Hartley, R. and Zisserman, A. Multiple view geometry in computer vision. Cambridge university press, 2003

    2003

  21. [29]

    beta- VAE : Learning basic visual concepts with a constrained variational framework

    Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. beta- VAE : Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2017

    2017

  22. [30]

    Some demonstrations of the effects of structural descriptions in mental imagery

    Hinton, G. Some demonstrations of the effects of structural descriptions in mental imagery. Cognitive Science, 3 0 (3): 0 231--250, 1979

    1979

  23. [31]

    How to represent part-whole hierarchies in a neural network

    Hinton, G. How to represent part-whole hierarchies in a neural network. Neural Computation, pp.\ 1--40, 2022

    2022

  24. [32]

    Hoffman, M. D. and Johnson, M. J. Elbo surgery: yet another way to carve up the variational evidence lower bound. In Workshop in Advances in Approximate Bayesian Inference, NIPS, volume 1, 2016

    2016

  25. [33]

    F., and Niebles, J

    Hsieh, J.-T., Liu, B., Huang, D.-A., Fei-Fei, L. F., and Niebles, J. C. Learning to decompose and disentangle representations for video prediction. Advances in neural information processing systems, 31, 2018

    2018

  26. [34]

    and Pajunen, P

    Hyv \"a rinen, A. and Pajunen, P. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12 0 (3): 0 429--439, 1999

    1999

  27. [35]

    Nonlinear ica using auxiliary variables and generalized contrastive learning

    Hyvarinen, A., Sasaki, H., and Turner, R. Nonlinear ica using auxiliary variables and generalized contrastive learning. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89, pp.\ 859--868. PMLR, 2019

    2019

  28. [36]

    Spatial transformer networks

    Jaderberg, M., Simonyan, K., Zisserman, A., et al. Spatial transformer networks. Advances in neural information processing systems, 28, 2015

    2015

  29. [37]

    Variational autoencoders and nonlinear ica: A unifying framework

    Khemakhem, I., Kingma, D., Monti, R., and Hyvarinen, A. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pp.\ 2207--2217. PMLR, 2020 a

    2020

  30. [38]

    Ice-beem: Identifiable conditional energy-based deep models based on nonlinear ica

    Khemakhem, I., Monti, R., Kingma, D., and Hyvarinen, A. Ice-beem: Identifiable conditional energy-based deep models based on nonlinear ica. In Advances in Neural Information Processing Systems, volume 33, 2020 b . URL https://proceedings.neurips.cc/paper_files/paper/2020/file/...

    2020

  31. [39]

    Ice-beem: Identifiable conditional energy-based deep models based on nonlinear ica

    Khemakhem, I., Monti, R., Kingma, D., and Hyvarinen, A. Ice-beem: Identifiable conditional energy-based deep models based on nonlinear ica. Advances in Neural Information Processing Systems, 33: 0 12768--12778, 2020 c

    2020

  32. [40]

    and Mnih, A

    Kim, H. and Mnih, A. Disentangling by factorising. In Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, 2018

    2018

  33. [41]

    Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  34. [42]

    F., Mahendran, A., Stone, A., Sabour, S., Heigold, G., Jonschkowski, R., Dosovitskiy, A., and Greff, K

    Kipf, T., Elsayed, G. F., Mahendran, A., Stone, A., Sabour, S., Heigold, G., Jonschkowski, R., Dosovitskiy, A., and Greff, K. Conditional object-centric learning from video. arXiv preprint arXiv:2111.12594, 2021

    2021 arXiv

  35. [43]

    Identifiability of deep generative models without auxiliary information

    Kivva, B., Rajendran, G., Ravikumar, P., and Aragam, B. Identifiability of deep generative models without auxiliary information. Advances in Neural Information Processing Systems, 35: 0 15687--15701, 2022

    2022

  36. [45]

    Kori, A., Locatello, F., Toni, F., Glocker, B., and Ribeiro, F. D. S. Identifiable object centric representations via probabilistic slot attention. arXiv preprint arXiv:2307.09437, 2024

    2024 arXiv

  37. [46]

    W., and Posner, I

    Kosiorek, A., Kim, H., Teh, Y. W., and Posner, I. Sequential attend, infer, repeat: Generative modelling of moving objects. Advances in Neural Information Processing Systems, 31, 2018

    2018

  38. [47]

    Additive decoders for latent variables identification and cartesian-product extrapolation

    Lachapelle, S., Mahajan, D., Mitliagkas, I., and Lacoste-Julien, S. Additive decoders for latent variables identification and cartesian-product extrapolation. arXiv preprint arXiv:2307.02598, 2023

    2023 arXiv

  39. [48]

    M., Ullman, T

    Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. Building machines that learn and think like people. Behavioral and brain sciences, 40: 0 e253, 2017

    2017

  40. [49]

    Learning object-centric representations of multi-object scenes from multiple views

    Li, N., Eastwood, C., and Fisher, R. Learning object-centric representations of multi-object scenes from multiple views. Advances in Neural Information Processing Systems, 33: 0 5656--5666, 2020

    2020

  41. [50]

    A., Hu, W., Sun, Z., and Fisher, R

    Li, N., Raza, M. A., Hu, W., Sun, Z., and Fisher, R. Object-centric representation learning with generative spatial-temporal factorization. Advances in neural information processing systems, 34: 0 10772--10783, 2021

    2021

  42. [51]

    V., Sun, W., Singh, G., Deng, F., Jiang, J., and Ahn, S

    Lin, Z., Wu, Y.-F., Peri, S. V., Sun, W., Singh, G., Deng, F., Jiang, J., and Ahn, S. Space: Unsupervised object-oriented scene representation via spatial attention and decomposition. arXiv preprint arXiv:2001.02407, 2020

    2001 arXiv

  43. [52]

    Slotlifter: Slot-guided feature lifting for learning object-centric radiance fields

    Liu, Y., Jia, B., Chen, Y., and Huang, S. Slotlifter: Slot-guided feature lifting for learning object-centric radiance fields. In European Conference on Computer Vision, pp.\ 270--288. Springer, 2025

    2025

  44. [53]

    Challenging common assumptions in the unsupervised learning of disentangled representations

    Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Sch \"o lkopf, B., and Bachem, O. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning, pp.\ 4114--4124. PMLR, 2019

    2019

  45. [54]

    a tsch, G., Sch \

    Locatello, F., Poole, B., R \"a tsch, G., Sch \"o lkopf, B., Bachem, O., and Tschannen, M. Weakly-supervised disentanglement without compromises. In International Conference on Machine Learning, pp.\ 6348--6359. PMLR, 2020 a

    2020

  46. [55]

    Object-centric learning with slot attention

    Locatello, F., Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., Dosovitskiy, A., and Kipf, T. Object-centric learning with slot attention. Advances in Neural Information Processing Systems, 33: 0 11525--11538, 2020 b

    2020

  47. [56]

    Rotating features for object discovery

    L \"o we, S., Lippe, P., Locatello, F., and Welling, M. Rotating features for object discovery. Advances in Neural Information Processing Systems, 36, 2024

    2024

  48. [57]

    Unsupervised discovery of object-centric neural fields

    Luo, R., Yu, H.-X., and Wu, J. Unsupervised discovery of object-centric neural fields. arXiv preprint arXiv:2402.07376, 2024

    2024 arXiv

  49. [58]

    Object-centric architectures enable efficient causal representation learning

    Mansouri, A., Hartford, J., Zhang, Y., and Bengio, Y. Object-centric architectures enable efficient causal representation learning. arXiv preprint arXiv:2310.19054, 2023

    2023 arXiv

  50. [59]

    Marcus, G. F. The algebraic mind: Integrating connectionism and cognitive science. MIT press, 2003

    2003

  51. [60]

    Iterative amortized inference

    Marino, J., Yue, Y., and Mandt, S. Iterative amortized inference. In International Conference on Machine Learning, pp.\ 3403--3412. PMLR, 2018

    2018

  52. [61]

    Mathieu, E., Rainforth, T., Siddharth, N., and Teh, Y. W. Disentangling disentanglement in variational autoencoders. In Proceedings of the 36th International Conference on Machine Learning, 2019

    2019

  53. [62]

    P., Tancik, M., Barron, J

    Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65 0 (1): 0 99--106, 2021

    2021

  54. [63]

    o lkopf, B. and von K \

    Sch \"o lkopf, B. and von K \"u gelgen, J. From statistical to causal learning. Proceedings of the International Congress of Mathematicians, 2022

    2022

  55. [64]

    Bridging the gap to real-world object-centric learning

    Seitzer, M., Horn, M., Zadaianchuk, A., Zietlow, D., Xiao, T., Simon-Gabriel, C.-J., He, T., Zhang, Z., Sch \"o lkopf, B., Brox, T., et al. Bridging the gap to real-world object-centric learning. arXiv preprint arXiv:2209.14860, 2022

    2022 arXiv

  56. [65]

    Illiterate dall-e learns to compose

    Singh, G., Deng, F., and Ahn, S. Illiterate dall-e learns to compose. arXiv preprint arXiv:2110.11405, 2021

    2021 arXiv

  57. [66]

    Neural block-slot representations

    Singh, G., Kim, Y., and Ahn, S. Neural block-slot representations. arXiv preprint arXiv:2211.01177, 2022

    2022 arXiv

  58. [67]

    B., Kemp, C., Griffiths, T

    Tenenbaum, J. B., Kemp, C., Griffiths, T. L., and Goodman, N. D. How to grow a mind: Statistics, structure, and abstraction. science, 331 0 (6022): 0 1279--1285, 2011

    2011

  59. [68]

    Geometry-aware neural rendering

    Tobin, J., Zaremba, W., and Abbeel, P. Geometry-aware neural rendering. Advances in Neural Information Processing Systems, 32, 2019

    2019

  60. [69]

    Relational neural expectation maximization: Unsupervised discovery of objects and their interactions

    Van Steenkiste, S., Chang, M., Greff, K., and Schmidhuber, J. Relational neural expectation maximization: Unsupervised discovery of objects and their interactions. arXiv preprint arXiv:1802.10353, 2018

    2018 arXiv

  61. [70]

    Investigating object compositionality in generative adversarial networks

    Van Steenkiste, S., Kurach, K., Schmidhuber, J., and Gelly, S. Investigating object compositionality in generative adversarial networks. Neural Networks, 130: 0 309--325, 2020

    2020

  62. [71]

    N., Kaiser, ., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  63. [72]

    u gelgen, J., Sharma, Y., Gresele, L., Brendel, W., Sch \

    Von K \"u gelgen, J., Sharma, Y., Gresele, L., Brendel, W., Sch \"o lkopf, B., Besserve, M., and Locatello, F. Self-supervised learning with data augmentations provably isolates content from style. Advances in neural information processing systems, 34: 0 16451--16467, 2021

    2021

  64. [73]

    Slot-vae: Object-centric scene generation with slot attention

    Wang, Y., Liu, L., and Dauwels, J. Slot-vae: Object-centric scene generation with slot attention. arXiv preprint arXiv:2306.06997, 2023

    2023 arXiv

  65. [74]

    and Paige, B

    Willetts, M. and Paige, B. I don't need u: Identifiable non-linear ica without side information. arXiv preprint arXiv:2106.05238, 2021

    2021 arXiv

  66. [75]

    Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling

    Wu, J., Zhang, C., Xue, T., Freeman, B., and Tenenbaum, J. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Advances in neural information processing systems, 29, 2016

    2016

  67. [76]

    Nonlinear ICA using volume-preserving transformations

    Yang, X., Wang, Y., Sun, J., Zhang, X., Zhang, S., Li, Z., and Yan, J. Nonlinear ICA using volume-preserving transformations. In International Conference on Learning Representations, 2022

    2022

  68. [77]

    Unsupervised object-centric learning from multiple unspecified viewpoints

    Yuan, J., Chen, T., Shen, Z., Li, B., and Xue, X. Unsupervised object-centric learning from multiple unspecified viewpoints. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  69. [78]

    S., Sharma, Y., Schneider, S., Bethge, M., and Brendel, W

    Zimmermann, R. S., Sharma, Y., Schneider, S., Bethge, M., and Brendel, W. Contrastive learning inverts the data generating process. In International Conference on Machine Learning, pp.\ 12979--12990. PMLR, 2021

    2021

  70. [79]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  71. [80]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  72. [81]

    masked decoders

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.