REVIEW 21 cited by
Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Modern machine learning systems rely on large datasets to attain broad generalization, and this often poses a challenge in robot learning, where each robotic platform and task might have only a small dataset. By training a single policy across many different kinds of robots, a robot learning method can leverage much broader and more diverse datasets, which in turn can lead to better generalization and robustness. However, training a single policy on multi-robot data is challenging because robots can have widely varying sensors, actuators, and control frequencies. We propose CrossFormer, a scalable and flexible transformer-based policy that can consume data from any embodiment. We train CrossFormer on the largest and most diverse dataset to date, 900K trajectories across 20 different robot embodiments. We demonstrate that the same network weights can control vastly different robots, including single and dual arm manipulation systems, wheeled robots, quadcopters, and quadrupeds. Unlike prior work, our model does not require manual alignment of the observation or action spaces. Extensive experiments in the real world show that our method matches the performance of specialist policies tailored for each embodiment, while also significantly outperforming the prior state of the art in cross-embodiment learning.
Forward citations
Cited by 21 Pith papers
-
The One RING: a Robotic Indoor Navigation Generalist
A simulation-trained policy that randomizes robot body and camera configurations generalizes zero-shot to real robots it has never seen.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
SeeSE3: Emergence of 3D Space in Vision Features
Self-supervised vision features, especially DINOv2, contain a subspace that a small trained adapter can map to 3D camera motion, enabling pose estimation and latent-space navigation without explicit 3D reconstruction.
-
Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
Preserving pretrained VLM features with layer-wise distillation plus supervising the language head on discretized action directions improves OOD generalization of VLA policies on LIBERO, CALVIN, and a real xArm7.
-
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
A modular benchmark of 100 dexterous manipulation tasks across 3 arms and 6 hands with 3,180 demonstrations reveals that current policies (Diffusion Policy, DP3, OpenVLA, π0.5) achieve only 34% mean success, exposing ...
-
Cross-Embodiment Robot Manipulation via a Unified Hand Action Space
UHAS maps hand actions to deformations of a shared unit sphere and recovers joint commands via cascade IK, enabling multi-hand RL, zero-shot transfer, and modest real-world cube reorientation on LEAP and Allegro.
-
Layout-independent actuation allocator for fin-actuated marine robots
A GNN-Transformer-MDN allocator with differentiable physics refinement zero-shot maps target wrenches to fin commands across unseen layouts and actuator failures.
-
InDex: Empowering VLA Models with Intent-Conditioned Arm-Hand Coordination for Dexterous Manipulation
InDex adapts VLA models to high-DoF dexterous manipulation via intent-conditioned fine-tuning and a decoupled diffusion head, outperforming monolithic baselines in simulation tasks with minimal data.
-
Scalable and General Whole-Body Control for Cross-Humanoid Locomotion
A single policy trained with physics-consistent morphological randomization and a canonical 32-joint representation zero-shot transfers across 12 simulated and 7 real humanoid robots.
-
Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation
Low within-subdataset diversity and large between-subdataset differences cause shortcut learning in generalist robot policies, and targeted augmentation can mitigate it.
-
Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining
Pretraining a modular transformer policy on human demonstrations then finetuning on a small robot dataset improves success on six real quadruped manipulation tasks, including out-of-distribution objects.
-
Shared Modular Recurrence in Contextual MDPs for Universal Morphology Control
A shared per-limb recurrent network improves zero-shot generalization of universal robot controllers to unseen morphologies in MuJoCo.
-
CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World
CordViP achieves strong real-world dexterous manipulation by feeding a diffusion policy with pose-tracked 3D object models and hand point clouds, pretrained on contact maps and arm-hand coordination.
-
Learning Real-World Action-Video Dynamics with Heterogeneous Masked Autoregression
HMA is a masked autoregressive transformer that predicts future video and actions across many robot embodiments, running up to 15x faster than prior diffusion-based video simulators while matching or improving visual ...
-
SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuum
A simulation-only training pipeline using Gaussian Splatting scenes and MPC demonstrations produces drone policies that transfer zero-shot to real flight.
-
From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons
A single MLLM-based agent, finetuned with cross-domain supervision and online RL, achieves strong zero-shot generalization across manipulation, navigation, games, UI control, and planning.
-
SPARSE Data, Rich Results: Few-Shot Semi-Supervised Learning via Class-Conditioned Image Translation
A GAN framework that translates unlabeled medical images between classes and fuses ensemble, time-averaged pseudo-labels outperforms six prior GAN semi-supervised methods on MedMNIST at 5-50 labels per class.
-
Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion
A single diffusion-plus-residual-RL policy, trained on four robot morphologies using zero-padded observations and actions, outperforms per-robot PPO baselines in simulation and transfers to real robots.
-
Taking Flight with Dialogue: Enabling Natural Language Control for PX4-based Drone Agent
An open-source ROS2/PX4 framework using locally hosted LLMs and VLMs enables natural language drone commands, with the best simulated mission success rate at 40%.
-
Physical AI Governance: From Theory to Practice Across Life Cycle
A survey that organizes Physical AI governance into five principles and a five-stage lifecycle, with stage-specific operational practices.
-
Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
A review that categorizes large-model-empowered embodied AI into hierarchical and end-to-end decision-making, imitation and reinforcement learning, and world models.
Discussion (0). Continue with ORCID to comment.