REVIEW 7 cited by
SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Learning a latent dynamics model provides a task-agnostic representation of an agent's understanding of its environment. Leveraging this knowledge for model-based reinforcement learning (RL) holds the potential to improve sample efficiency over model-free methods by learning from imagined rollouts. Furthermore, because the latent space serves as input to behavior models, the informative representations learned by the world model facilitate efficient learning of desired skills. Most existing methods rely on holistic representations of the environment's state. In contrast, humans reason about objects and their interactions, predicting how actions will affect specific parts of their surroundings. Inspired by this, we propose Slot-Attention for Object-centric Latent Dynamics (SOLD), a novel model-based RL algorithm that learns object-centric dynamics models in an unsupervised manner from pixel inputs. We demonstrate that the structured latent space not only improves model interpretability but also provides a valuable input space for behavior models to reason over. Our results show that SOLD outperforms DreamerV3 and TD-MPC2 - state-of-the-art model-based RL algorithms - across a range of benchmark robotic environments that require relational reasoning and manipulation capabilities. Videos are available at https://slot-latent-dynamics.github.io/.
Forward citations
Cited by 7 Pith papers
-
Grounding Spatial Relations in a Compact World Model: Instruction Leakage and a Goal-Free Dynamics Fix
Goal-conditioned world models transcribe instructions instead of perceiving spatial relations when the instruction names the scored quantity, and removing the goal from the dynamics fixes it.
-
TaskSense: Focusing on What Matters in World Models
TaskSense filters visual observations with latent-conditioned stochastic spatial attention before encoding, improving world-model control under visual distractions relative to DreamerV3.
-
Learning visual representations for compositional analysis of artworks and photographs
A comparison of object-centric graph models versus fine-tuned foundation models for visual composition analysis, showing fine-tuning wins with data but sacrifices interpretability.
-
Spotlighting Task-Relevant Features: Object-Centric Representations for Better Generalization in Robotic Manipulation
Slot-based object-centric visual representations, especially with robot-video pretraining, improve out-of-distribution generalization of robotic manipulation policies compared to global and dense pre-trained features.
-
Object-Centric Representations Improve Policy Generalization in Robot Manipulation
Slot-based object-centric representations, especially a video model pretrained on robot data, improve policy generalization under visual distribution shifts in simulated and real-world manipulation tasks.
-
ES-Merging: Biological MLLM Merging via Embedding Space Signals
ES-Merging estimates layer-wise and element-wise merge coefficients from coarse- and fine-grained embedding signals and claims better cross-modal reasoning and single-modal knowledge preservation than parameter-space merging.
-
Towards Conscious Service Robots
Service robots should be built with a two-system cognitive architecture, including a global workspace and metacognitive monitoring, to generalize to novel situations.
Discussion (0). Continue with ORCID to comment.