Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

Slot-based object-centric visual representations make robot manipulation policies generalize better under lighting, texture, and distractor shifts than global or dense features, even without task-specific pretraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 07:00 UTC pith:INQNCSDR

load-bearing objection Useful large-scale benchmark of slot-based object-centric representations for manipulation, but the causal claim is not yet clean: the token-count confound is not controlled. the 5 major comments →

arxiv 2601.21416 v2 pith:INQNCSDR submitted 2026-01-29 cs.RO

Spotlighting Task-Relevant Features: Object-Centric Representations for Better Generalization in Robotic Manipulation

Alexandre Chapin (1) , Bruno Machado (1) , Emmanuel Dellandr\'ea (1) , Liming Chen (1) ((1) LIRIS) This is my paper
classification cs.RO
keywords visual representationobject-centric representationsslot attentionrobotic manipulationgeneralization under distribution shiftimitation learningrepresentation pretraining
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether the structure of a robot's visual representation—a single pooled vector for the whole image, per-region dense features, or an intermediate set of object-like slots—determines how well an imitation-learned policy keeps working when the scene changes. Across seven pretrained visual models, two simulation benchmarks, and a real robot arm with four tabletop tasks, the authors find that slot-based object-centric representations match or beat dense-feature baselines in-domain and generalize substantially better under lighting changes, novel textures, and distractors. The advantage appears even when the slot model has never been trained on robot data, and grows when it is pretrained on a large collection of real robot demonstrations. A sympathetic reader would take this as evidence that the inductive bias of grouping the image into objects is itself doing work, independent of the strength of the underlying vision backbone. If correct, representation design—not just pretraining data scale—is a usable lever for making manipulation policies tolerate the messiness of the real world.

Core claim

The central discovery is that inserting a slot-based object-centric layer between a frozen vision backbone and a transformer policy changes how well the policy tolerates visual distribution shift. Using DINOSAUR*, which runs Slot Attention on top of frozen DINOv2 features and adds a temporal transformer to propagate slots across frames, the policies match or exceed dense-feature baselines on the training distribution and beat every global and dense baseline under shift, with the smallest relative performance drop. Pretraining the same model on a large mixture of real-world robot demonstrations (DINOSAUR-Rob*) pushes the gains further and makes it the top performer across simulation and real

What carries the argument

The load-bearing component is Slot Attention, a differentiable module that turns a set of dense feature tokens into a small fixed number of object-like vectors ('slots') through iterative cross-attention with a competitive softmax, so that each slot specializes in a distinct part of the image. The paper builds DINOSAUR* by applying this slot layer to features from a frozen self-supervised vision backbone (DINOv2) and adding a transformer layer that passes slot information between timesteps for temporal coherence; DINOSAUR-Rob* is the same architecture given an additional pretraining stage on real robotic demonstration videos. These slots become the visual tokens that the policy transformer a

Load-bearing premise

The entire advantage rests on the assumption that Slot Attention, applied to frozen DINOv2 features, actually separates task-relevant objects from background and distractors under the tested shifts—the paper measures task success, not slot purity, and its limitations section concedes that some slots capture background or distractors.

What would settle it

Give the policy the same frozen backbone but replace Slot Attention with a fixed random grouping of dense tokens into the same number of vectors; if generalization under lighting, texture, and distractor shifts remains just as high, the object-centric structure is not the cause. Alternatively, compute per-slot attention masks under a distractor shift and check whether the slots that drive successful actions are the ones aligned with the manipulated object; if performance survives when object-aligned slots are removed, the filtering story fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Object-centric representations give robot-policy designers a practical middle ground between one-vector summaries and dense pixel-level features, with better out-of-distribution success rates and no task-specific tuning.
  • Pretraining a slot-based model on large quantities of real robot demonstration data is worthwhile, contradicting earlier assumptions that object-centric methods do not benefit from large-scale pretraining.
  • In real-world deployment, a frozen vision backbone plus a slot layer can keep a manipulation policy working when lighting, texture, and clutter change, even if the slot model has never seen robot video.
  • The failure of a naive segmentation baseline suggests that masks alone, without spatial coordinates or end-to-end training, are insufficient; object structure needs to be learned jointly with the policy.
  • The results point to representation structure as an independent factor in generalization, separate from the choice of vision backbone or pretraining data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is background filtering, then slot purity under shift should predict policy success; one could instrument the Slot Attention maps to test whether the slots that drive successful rollouts are the ones aligned with the manipulated object.
  • The results suggest a design heuristic: representation granularity should match the object structure of the task, a principle that may transfer to other control problems beyond tabletop manipulation.
  • The paper's own limitation section implies that adding semantic grounding to slots could close the remaining distractor gap, since some slots are currently allocated to background or distractors.
  • A natural testable extension is to sweep the number of slots and the backbone scale to find where object-centric structure stops being beneficial; the paper does not explore this boundary.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper compares global, dense, and slot-based object-centric visual representations for robotic manipulation policies. The authors introduce an adapted Slot Attention module (DINOSAUR*) pretrained on COCO and optionally on robot video (DINOSAUR-Rob*), and evaluate seven frozen visual representations within a BAKU-based policy architecture on MetaWorld, LIBERO, and a real-world Franka setup. Generalization is tested under distractors, texture changes, and lighting shifts. The central claim is that slot-based object-centric representations outperform global and dense representations in generalization settings, even without task-specific pretraining, and that robot-video pretraining amplifies these benefits.

Significance. If the empirical claim holds, the paper provides evidence that a structured, object-centric bottleneck is a useful inductive bias for robust robotic manipulation, and it would be one of the first large-scale demonstrations of this in realistic control tasks. The study has several strengths: a unified policy architecture with frozen encoders to isolate representation choice, a COCO-only version of the object-centric model as a control for robot-data exposure, a focus analysis, and open-sourced evaluation code. However, the causal interpretation is currently undermined by a token-count confound and by insufficient statistical support for the real-world results.

major comments (5)
  1. [Table III, Section V-B] Real-world generalization results are reported as point estimates without error bars, confidence intervals, or significance tests; each condition has only 12 rollouts. With n=12, a difference such as DINOSAUR* 0.28 vs DINOv2 0.07 corresponds to roughly 3 vs 1 successful rollouts, and the DINOSAUR-Rob* 0.41 vs 0.07 gap is still based on small counts. Report per-task counts, confidence intervals, and at least a bootstrap or binomial test before claiming that SOCR policies 'consistently achieve the highest success rates' in the real world.
  2. [Section III, Tables II-III] SOCR policies feed exactly K slot tokens to the BAKU transformer, while dense baselines feed N patch tokens and global baselines feed one token. Because BAKU is a transformer, its behavior depends on sequence length; the observed gains may be due to the information bottleneck or sequence-length regularization rather than object-centric structure. No non-object-centric K-token control is run. SAM+DINOv2 is not a valid control because it is excluded from generalization analysis and was deliberately stripped of spatial information (Section IV-b). Add a control that maps the same dense features to K learned tokens without slot constraints (e.g., a linear projection or learned query bottleneck) and show that the object-centric version outperforms it.
  3. [Section V-A, LIBERO] The paper states that LIBERO 'does not introduce any distributional shifts for evaluation' and reports only in-domain LIBERO results. Nevertheless, Figure 4 and the conclusion claim 'consistently superior performance ... under distributional shifts' across all environments. As written, LIBERO only supports in-domain performance. Either add LIBERO shift experiments (e.g., texture/lighting/clutter variants) or restrict the generalization claim to MetaWorld and the real world.
  4. [Section III-b vs Table III] DINOSAUR-Rob* is pretrained on BridgeData V2, Fractal, and DROID, all robot-manipulation video datasets that may share visual and embodiment distribution with the real-world Franka evaluation. The claim that robot pretraining 'amplifies' SOCR benefits is therefore confounded by pretraining data distribution. The paper lacks a robot-pretrained dense or global baseline trained on the same mixture. Add such a baseline (or use an existing robot-pretrained dense representation, e.g., MCR) to isolate the effect of object-centric structure from data exposure.
  5. [Section VII vs Section VI] The paper attributes SOCR generalization to slots 'filtering task-irrelevant background', but Section VII concedes that slots can bind background and distractors, and no quantitative slot-purity or objectness metric is reported under distribution shifts. Provide segmentation or slot-binding metrics under the tested shifts to substantiate the proposed mechanism rather than relying on anecdotal visualizations.
minor comments (6)
  1. [Section VI] Typo: 'Figure 1 shows thant' should be 'shows that'.
  2. [Abstract / Introduction] The acronym SBOCR is used in the abstract and introduction, while SOCR is used elsewhere; standardize to one term.
  3. [Table I] The SAM+DINOv2 row is labeled 'SM' (segmentation). Clarify in the table caption that this baseline is excluded from generalization analysis, as stated in Section V-B.
  4. [Section IV-b] The SAM+DINOv2 baseline is described as 'naïve' because it removes spatial information. This design choice is justified later in Section V-A but should be stated earlier to avoid confusion about its representational capacity.
  5. [Section III] Hyperparameters for Slot Attention (slot count K, number of iterations, temporal transformer architecture, training details) are not reported in the main text. Add a table or appendix describing these settings.
  6. [Figure 4] The label 'METaworld' is misspelled, and the red numbers indicating relative drops are hard to read; increase font size and caption contrast.

Circularity Check

0 steps flagged

No significant circularity: the central comparison is an open benchmark with frozen external baselines and a COCO-only object-centric control.

full rationale

This paper is an empirical benchmarking study rather than a derivation chain. The central claim—that slot-based object-centric representations improve generalization—is supported by direct comparisons against external, frozen visual encoders (ResNet, R3M, DINOv2, VC-1, Theia) using a shared policy architecture. The key paired comparison between DINOSAUR* and DINOv2 holds the backbone fixed and adds only the Slot Attention layer, so the improvement is not definitionally built into the comparison. DINOSAUR-Rob* is pretrained on robot video, which creates a distributional-overlap concern, but the paper also reports DINOSAUR*, pretrained only on COCO, which outperforms the dense/global baselines and thus independently supports the object-centric structure claim. No equation equates a predicted quantity with a fitted input, and no load-bearing result is justified solely by self-citation (the reference list contains no author self-citations). The acknowledged limitation that some slots bind background or distractors weakens the proposed filtering mechanism but does not make the empirical result circular. The SAM+DINOv2 exclusion is a design choice, not a circular reduction. The token-count confound noted by skeptics is a legitimate alternative explanation but is not circularity: the paper does not define object-centric success in terms of token count. Overall, the evaluation is externally falsifiable and the central comparison is not forced by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical entities are postulated. The free parameters are model/pretraining choices, not fitted physical constants. The key domain assumptions concern transfer of slot decomposition, fairness of the comparison, and the statistical sufficiency of the real-world rollout counts.

free parameters (4)
  • Slot count K
    Number of slots used by Slot Attention is not reported in the paper; it controls decomposition granularity and the token sequence fed to the policy.
  • Slot Attention iterations
    Iterative refinement steps for slot binding are not specified; they affect decomposition quality and compute.
  • Temporal transformer hyperparameters
    The new temporal layer in DINOSAUR* (layers, heads, hidden dims) is not specified, though it is central to the video extension.
  • Robot mixture pretraining composition = 188k trajectories from BridgeData V2, Fractal, DROID
    The manual choice of datasets and their proportions is a free design decision; DINOSAUR-Rob*'s performance depends on it.
axioms (5)
  • domain assumption Slot Attention pretraining on COCO transfers object decomposition to robot manipulation scenes.
    DINOSAUR* is pretrained on COCO images and used frozen for manipulation; cross-domain transfer of slot structure is assumed rather than demonstrated per-scene.
  • domain assumption Identical frozen encoders and the same BAKU/ACT observation trunk give a fair comparison across global, dense, and slot features.
    Fairness relies on the transformer treating all token types equivalently; no representation-specific tuning or capacity adjustment is described.
  • domain assumption Robot-video pretraining (BridgeData V2, Fractal, DROID) improves downstream performance on the held-out Franka tasks.
    Assumes visual overlap between pretraining mixture and test tasks; no disjoint-domain ablation is provided.
  • domain assumption 12 real-world rollouts per task per condition are sufficient to rank policies.
    No confidence intervals or repeated seeds are reported for real-world results; small-n ranking may reflect noise.
  • domain assumption MetaWorld texture/lighting/distractor shifts are representative of real-world distribution shifts.
    Generalization conclusions partly rest on simulated shifts standing in for real-world appearance changes.

pith-pipeline@v1.3.0-alltime-deepseek · 104 in / 9922 out tokens · 108949 ms · 2026-08-03T07:00:32.824438+00:00 · methodology

0 comments
read the original abstract

The generalization capabilities of robotic manipulation policies are heavily influenced by the choice of visual representations. Existing approaches typically rely on representations extracted from pre-trained encoders, using two dominant types of features: global features, which summarize an entire image via a single pooled vector, and dense features, which preserve a patch-wise embedding from the final encoder layer. While widely used, both feature types mix task-relevant and irrelevant information, leading to poor generalization under distribution shifts, such as changes in lighting, textures, or the presence of distractors. In this work, we explore an intermediate structured alternative: Slot-Based Object-Centric Representations (SBOCR), which group dense features into a finite set of object-like entities. This representation permits to naturally reduce the noise provided to the robotic manipulation policy while keeping enough information to efficiently perform the task. We benchmark a range of global and dense representations against intermediate slot-based representations, across a suite of simulated and real-world manipulation tasks ranging from simple to complex. We evaluate their generalization under diverse visual conditions, including changes in lighting, texture, and the presence of distractors. Our findings reveal that SBOCR-based policies outperform dense and global representation-based policies in generalization settings, even without task-specific pretraining. These insights suggest that SBOCR is a promising direction for designing visual systems that generalize effectively in dynamic, real-world robotic environments.

Figures

Figures reproduced from arXiv: 2601.21416 by Alexandre Chapin (1), Bruno Machado (1), Emmanuel Dellandr\'ea (1), Liming Chen (1) ((1) LIRIS).

Figure 1
Figure 1. Figure 1: Overview of visual representations. (Left) We use a set of pre-trained visual models with different latent-space structures: global, dense, and object-centric. Dense representations are extracted from one of the encoder’s layers (CNN or ViT) before linear projection, while global representations are obtained after pooling operations. Slot-based object-centric representations emerge from an additional Slot-… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the robotic manipulation policy architecture. We use a pre-trained visual model to extract visual features from raw images. These features are then combined with other modalities (e.g., language instructions, proprioception) in an observation trunk with an additional action token. Finally, a policy head predicts the next action given the additional token. We experiment with different types of v… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of real-world setup. We evaluate the different visual models on a Franka robotic arm on four tabletop manipulation tasks (From left to right): Stacking bowls into a pan, Opening a drawer placing a screwdriver inside and closing the drawer, Putting cans into a bin and Placing plates into a dish rack. as the backbone. This choice ensures that differences in performance can be attributed to the repre… view at source ↗
Figure 4
Figure 4. Figure 4: Overall success rate on in-domain and generalization scenarios. Mean success rate over all tasks for each visual model on MetaWorld (left), LIBERO (middle) and Real robot using Franka (right). Green dot: in-domain performance, Orange dot: average performance over all generalization scenarios (distractors, novel textures, lighting changes). Red number: relative drop in performance from in-domain to generali… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. See Less, Specify More: Visual Evidence Budgets for Generalizable VLAs

    cs.RO 2026-06 unverdicted novelty 6.0

    S2 improves generalization in vision-language-action models by using goal-preserving refined language guidance and explicit visual evidence budgets, raising mean subtask success from 54.2% to 79.0% on eight real-robot...

  2. More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learning

    cs.RO 2026-07 conditional novelty 5.0

    Frozen object-centric SPOT slots beat matched dense DINO features by 22.4% held-out success on ManiSkill3 PickCube; structure, not token count, drives the gain.

Reference graph

Works this paper leans on

62 extracted references · 50 linked inside Pith · cited by 2 Pith papers

  1. [1]

    R3m: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3m: A universal visual representation for robot manipulation,” 2022. [Online]. Available: https://arxiv.org/abs/2203.12601

    Pith/arXiv arXiv 2022

  2. [2]

    Where are we in the search for an artificial visual cortex for embodied intelligence?

    A. Majumdar, K. Yadav, S. Arnaud, et al. , “Where are we in the search for an artificial visual cortex for embodied intelligence?”

  3. [3]

    Vip: Towards universal visual reward and representation via value-implicit pre-training,

    Y . J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V . Kumar, and A. Zhang, “Vip: Towards universal visual reward and representation via value-implicit pre-training,” 2023. [Online]. Available: https://arxiv.org/abs/2210.00030

    Pith/arXiv arXiv 2023

  4. [4]

    Theia: Distilling diverse vision foundation models for robot learning,

    J. Shang, K. Schmeckpeper, B. B. May, M. V . Minniti, T. Kelestemur, D. Watkins, and L. Herlant, “Theia: Distilling diverse vision foundation models for robot learning,” 2024. [Online]. Available: https://arxiv.org/abs/2407.20179

    Pith/arXiv arXiv 2024

  5. [5]

    Real-world robot learning with masked visual pre-training,

    I. Radosavovic, T. Xiao, S. James, P. Abbeel, J. Malik, and T. Darrell, “Real-world robot learning with masked visual pre-training,” 2022. [Online]. Available: https://arxiv.org/abs/2210.03109

    Pith/arXiv arXiv 2022

  6. [6]

    Principles of object perception,

    E. S. Spelke, “Principles of object perception,” Cognitive science , vol. 14, no. 1, pp. 29–56, 1990

    1990

  7. [7]

    Building machines that learn and think like people,

    B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman, “Building machines that learn and think like people,” 2016. [Online]. Available: https://arxiv.org/abs/1604.00289

    Pith/arXiv arXiv 2016

  8. [8]

    Robots pre- train robots: Manipulation-centric robotic representation from large- scale robot dataset,

    G. Jiang, Y . Sun, T. Huang, H. Li, Y . Liang, and H. Xu, “Robots pre- train robots: Manipulation-centric robotic representation from large- scale robot dataset,” arXiv preprint arXiv:2410.22325 , 2024

    Pith/arXiv arXiv 2024

  9. [9]

    The unsurprising effectiveness of pre-trained vision models for control,

    S. Parisi, A. Rajeswaran, S. Purushwalkam, and A. Gupta, “The unsurprising effectiveness of pre-trained vision models for control,”

  10. [10]

    What makes pre-trained visual representations successful for robust manipulation?

    K. Burns, Z. Witzel, J. I. Hamid, T. Yu, C. Finn, and K. Hausman, “What makes pre-trained visual representations successful for robust manipulation?” 2023. [Online]. Available: https://arxiv.org/abs/2312.12444

    Pith/arXiv arXiv 2023

  11. [11]

    Object- centric learning with slot attention,

    F. Locatello, D. Weissenborn, T. Unterthiner, A. Mahendran, G. Heigold, J. Uszkoreit, A. Dosovitskiy, and T. Kipf, “Object- centric learning with slot attention,” 2020. [Online]. Available: https://arxiv.org/abs/2006.15055

    Pith/arXiv arXiv 2020

  12. [12]

    Bridging the gap to real-world object-centric learning,

    M. Seitzer, M. Horn, A. Zadaianchuk, et al. , “Bridging the gap to real-world object-centric learning,” 2023. [Online]. Available: https://arxiv.org/abs/2209.14860

    Pith/arXiv arXiv 2023

  13. [13]

    An investigation into pre-training object-centric representations for reinforcement learning,

    J. Yoon, Y .-F. Wu, H. Bae, and S. Ahn, “An investigation into pre-training object-centric representations for reinforcement learning,”

  14. [14]

    Visuomotor control in multi-object scenes using object-aware representations,

    N. Heravi, A. Wahid, C. Lynch, P. Florence, T. Armstrong, J. Tompson, P. Sermanet, J. Bohg, and D. Dwibedi, “Visuomotor control in multi-object scenes using object-aware representations,”

  15. [15]

    Entity-centric reinforcement learning for object manipulation from pixels,

    D. Haramati, T. Daniel, and A. Tamar, “Entity-centric reinforcement learning for object manipulation from pixels,” 2024. [Online]. Available: https://arxiv.org/abs/2404.01220

    Pith/arXiv arXiv 2024

  16. [16]

    Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration,

    N. Watters, L. Matthey, M. Bosnjak, C. P. Burgess, and A. Lerchner, “Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration,” 2019. [Online]. Available: https://arxiv.org/abs/1905.09275

    Pith/arXiv arXiv 2019

  17. [17]

    Conditional object-centric learning from video,

    T. Kipf, G. F. Elsayed, A. Mahendran, A. Stone, S. Sabour, G. Heigold, R. Jonschkowski, A. Dosovitskiy, and K. Greff, “Conditional object-centric learning from video,” 2022. [Online]. Available: https://arxiv.org/abs/2111.12594

    Pith/arXiv arXiv 2022

  18. [18]

    Available: https://arxiv.org/abs/2205.06333

    [Online]. Available: https://arxiv.org/abs/2205.06333

    Pith/arXiv arXiv

  19. [19]

    Improved baselines with momentum contrastive learning,

    X. Chen, H. Fan, R. Girshick, and K. He, “Improved baselines with momentum contrastive learning,” 2020. [Online]. Available: https://arxiv.org/abs/2003.04297

    Pith/arXiv arXiv 2020

  20. [20]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” 2021. [Online]. Available: https://arxiv.org/abs/2104. 14294

    2021

  21. [21]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, et al. , “Dinov2: Learning robust visual features without supervision,” 2024. [Online]. Available: https://arxiv.org/abs/2304.07193

    Pith/arXiv arXiv 2024

  22. [22]

    Zero-shot object-centric representation learning,

    A. Didolkar, A. Zadaianchuk, A. Goyal, M. Mozer, Y . Bengio, G. Martius, and M. Seitzer, “Zero-shot object-centric representation learning,” 2024. [Online]. Available: https://arxiv.org/abs/2408.09162

    Pith/arXiv arXiv 2024

  23. [23]

    Ego4d: Around the world in 3,000 hours of egocentric video,

    K. Grauman, A. Westbury, E. Byrne, et al. , “Ego4d: Around the world in 3,000 hours of egocentric video,” 2022. [Online]. Available: https://arxiv.org/abs/2110.07058

    Pith/arXiv arXiv 2022

  24. [24]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, et al. , “Imagenet large scale visual recognition challenge,” 2015. [Online]. Available: https://arxiv.org/abs/1409.0575

    Pith/arXiv arXiv 2015

  25. [25]

    An unbiased look at datasets for visuo-motor pre-training,

    S. Dasari, M. K. Srirama, U. Jain, and A. Gupta, “An unbiased look at datasets for visuo-motor pre-training,” 2023. [Online]. Available: https://arxiv.org/abs/2310.09289

    Pith/arXiv arXiv 2023

  26. [26]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, et al. , “Learning transferable visual models from natural language supervision,” 2021. [Online]. Available: https://arxiv.org/abs/2103.00020

    Pith/arXiv arXiv 2021

  27. [27]

    Sold: Slot object-centric latent dynamics models for relational manipulation learning from pixels,

    M. Mosbach, J. N. Ewertz, A. Villar-Corrales, and S. Behnke, “Sold: Slot object-centric latent dynamics models for relational manipulation learning from pixels,” 2025. [Online]. Available: https://arxiv.org/abs/2410.08822

    Pith/arXiv arXiv 2025

  28. [28]

    Explainable image recognition via enhanced slot-attention based classifier,

    B. Wang, L. Li, J. Zhang, Y . Nakashima, and H. Nagahara, “Explainable image recognition via enhanced slot-attention based classifier,” 2024. [Online]. Available: https://arxiv.org/abs/2407.05616

    Pith/arXiv arXiv 2024

  29. [29]

    Simone: View-invariant, temporally-abstracted object representations via unsupervised video decomposition,

    R. Kabra, D. Zoran, G. Erdogan, L. Matthey, A. Creswell, M. Botvinick, A. Lerchner, and C. P. Burgess, “Simone: View-invariant, temporally-abstracted object representations via unsupervised video decomposition,” 2021. [Online]. Available: https://arxiv.org/abs/2106.03849

    Pith/arXiv arXiv 2021

  30. [30]

    Carformer: Self-driving with learned object-centric representations,

    S. Hamdan and F. G ¨uney, “Carformer: Self-driving with learned object-centric representations,” 2024. [Online]. Available: https: //arxiv.org/abs/2407.15843

    Pith/arXiv arXiv 2024

  31. [31]

    Object-centric slot diffusion,

    J. Jiang, F. Deng, G. Singh, and S. Ahn, “Object-centric slot diffusion,” 2023. [Online]. Available: https://arxiv.org/abs/2303.10834

    Pith/arXiv arXiv 2023

  32. [32]

    Slotdiffusion: Object-centric generative modeling with diffusion models,

    Z. Wu, J. Hu, W. Lu, I. Gilitschenski, and A. Garg, “Slotdiffusion: Object-centric generative modeling with diffusion models,” 2023. [Online]. Available: https://arxiv.org/abs/2305.11281

    Pith/arXiv arXiv 2023

  33. [33]

    Illiterate dall-e learns to compose,

    G. Singh, F. Deng, and S. Ahn, “Illiterate dall-e learns to compose,”

  34. [34]

    Monet: Unsupervised scene decomposition and representation,

    C. P. Burgess, L. Matthey, N. Watters, R. Kabra, I. Higgins, M. Botvinick, and A. Lerchner, “Monet: Unsupervised scene decomposition and representation,” 2019. [Online]. Available: https: //arxiv.org/abs/1901.11390

    Pith/arXiv arXiv 2019

  35. [35]

    Simple unsupervised object-centric learning for complex and naturalistic videos,

    G. Singh, Y .-F. Wu, and S. Ahn, “Simple unsupervised object-centric learning for complex and naturalistic videos,” 2022. [Online]. Available: https://arxiv.org/abs/2205.14065

    Pith/arXiv arXiv 2022

  36. [36]

    Object-centric learning for real-world videos by predicting temporal feature similarities,

    A. Zadaianchuk, M. Seitzer, and G. Martius, “Object-centric learning for real-world videos by predicting temporal feature similarities,”

  37. [37]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, et al. , “Segment anything,” 2023. [Online]. Available: https://arxiv.org/abs/2304.02643

    Pith/arXiv arXiv 2023

  38. [38]

    Available: https://arxiv.org/abs/2110.11405

    [Online]. Available: https://arxiv.org/abs/2110.11405

    Pith/arXiv arXiv

  39. [39]

    Savi++: Towards end-to-end object- centric learning from real-world videos,

    G. F. Elsayed, A. Mahendran, S. van Steenkiste, K. Greff, M. C. Mozer, and T. Kipf, “Savi++: Towards end-to-end object- centric learning from real-world videos,” 2022. [Online]. Available: https://arxiv.org/abs/2206.07764

    Pith/arXiv arXiv 2022

  40. [40]

    Task-oriented hierarchical object decomposition for visuomotor control,

    J. Qian, Y . Li, B. Bucher, and D. Jayaraman, “Task-oriented hierarchical object decomposition for visuomotor control,” 2024. [Online]. Available: https://arxiv.org/abs/2411.01284

    Pith/arXiv arXiv 2024

  41. [41]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Doll ´ar, “Microsoft coco: Common objects in context,” 2015. [Online]. Available: https://arxiv.org/abs/1405.0312

    Pith/arXiv arXiv 2015

  42. [42]

    Available: https://arxiv.org/abs/2306.04829

    [Online]. Available: https://arxiv.org/abs/2306.04829

    Pith/arXiv arXiv

  43. [43]

    Rt-1: Robotics transformer for real-world control at scale,

    A. Brohan, N. Brown, J. Carbajal, et al. , “Rt-1: Robotics transformer for real-world control at scale,” 2023. [Online]. Available: https://arxiv.org/abs/2212.06817

    Pith/arXiv arXiv 2023

  44. [44]

    Composing pre-trained object-centric representations for robotics from

    J. Shi, J. Qian, Y . J. Ma, and D. Jayaraman, “Composing pre-trained object-centric representations for robotics from ”what” and ”where” foundation models,” 2024. [Online]. Available: https: //arxiv.org/abs/2404.13474

    Pith/arXiv arXiv 2024

  45. [45]

    Learning generalizable manipulation policies with object-centric 3d representations,

    Y . Zhu, Z. Jiang, P. Stone, and Y . Zhu, “Learning generalizable manipulation policies with object-centric 3d representations,” 2023. [Online]. Available: https://arxiv.org/abs/2310.14386

    Pith/arXiv arXiv 2023

  46. [46]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” 2023. [Online]. Available: https://arxiv.org/abs/2304.13705

    Pith/arXiv arXiv 2023

  47. [47]

    Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,

    B. Warner, A. Chaffin, B. Clavi ´e, et al. , “Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,” 2024. [Online]. Available: https://arxiv.org/abs/2412.13663

    Pith/arXiv arXiv 2024

  48. [48]

    Bridgedata v2: A dataset for robot learning at scale,

    H. Walke, K. Black, A. Lee, et al. , “Bridgedata v2: A dataset for robot learning at scale,” 2024. [Online]. Available: https: //arxiv.org/abs/2308.12952

    Pith/arXiv arXiv 2024

  49. [49]

    Libero: Benchmarking knowledge transfer for lifelong robot learning,

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learning,” arXiv preprint arXiv:2306.03310, 2023

    Pith/arXiv arXiv 2023

  50. [50]

    Droid: A large-scale in-the-wild robot manipulation dataset,

    A. Khazatsky, K. Pertsch, S. Nair, et al. , “Droid: A large-scale in-the-wild robot manipulation dataset,” 2024. [Online]. Available: https://arxiv.org/abs/2403.12945

    Pith/arXiv arXiv 2024

  51. [51]

    Baku: An efficient transformer for multi-task policy learning,

    S. Haldar, Z. Peng, and L. Pinto, “Baku: An efficient transformer for multi-task policy learning,” 2024. [Online]. Available: https: //arxiv.org/abs/2406.07539

    Pith/arXiv arXiv 2024

  52. [52]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, et al. , “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” arXiv preprint arXiv:2303.05499, 2023

    Pith/arXiv arXiv 2023

  53. [53]

    Tracking anything with decoupled video segmentation,

    H. K. Cheng, S. W. Oh, B. Price, A. Schwing, and J.-Y . Lee, “Tracking anything with decoupled video segmentation,” in ICCV, 2023

    2023

  54. [54]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,

    T. Yu, D. Quillen, Z. He, R. Julian, A. Narayan, H. Shively, A. Bellathur, K. Hausman, C. Finn, and S. Levine, “Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,” 2021. [Online]. Available: https://arxiv.org/abs/1910.10897

    Pith/arXiv arXiv 2021

  55. [55]

    Quantifying attention flow in transformers,

    S. Abnar and W. Zuidema, “Quantifying attention flow in transformers,” 2020. [Online]. Available: https://arxiv.org/abs/2005. 00928

    2020

  56. [56]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015. [Online]. Available: https://arxiv.org/abs/ 1512.03385

    Pith/arXiv arXiv 2015

  57. [57]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021. [Online]. Available: https://arxiv.org/abs/2010.11929

    Pith/arXiv arXiv 2021

  58. [60]

    Grad-cam: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision, vol. 128, no. 2, p. 336–359, Oct. 2019. [Online]. Available: http://dx.doi.org/10.1007/s11263-019-01228-7

    doi:10.1007/s11263-019-01228-7 2019

  59. [62]

    Temporally consistent object-centric learning by contrasting slots,

    A. Manasyan, M. Seitzer, F. Radovic, G. Martius, and A. Zadaianchuk, “Temporally consistent object-centric learning by contrasting slots,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2025

    2025

  60. [2022]

    Available: https://arxiv.org/abs/2203.03580

    [Online]. Available: https://arxiv.org/abs/2203.03580

    Pith/arXiv arXiv

  61. [2023]

    Available: https://arxiv.org/abs/2302.04419

    [Online]. Available: https://arxiv.org/abs/2302.04419

    Pith/arXiv arXiv

  62. [2024]

    Available: https://arxiv.org/abs/2303.18240

    [Online]. Available: https://arxiv.org/abs/2303.18240

    Pith/arXiv arXiv