Pith. sign in

REVIEW 3 major objections 6 minor 6 cited by

A single particle-based world model — trained on simulated robot hands and real human hands — can plan object manipulation for novel robot hands it never trained on, without fine-tuning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 00:22 UTC pith:XJBDDHZ7

load-bearing objection Solid simulation scaling study of particle-based cross-embodiment world models, but the real-robot 'effective control' claim is under-supported by the current metrics. the 3 major comments →

arxiv 2511.01177 v3 pith:XJBDDHZ7 submitted 2025-11-03 cs.RO

Scaling Cross-Embodiment World Models for Dexterous Manipulation

Zihao He , Bo Ai , Tongzhou Mu , Yulin Liu , Weikang Wan , Jiawei Fu , Yilun Du , Henrik I. Christensen
show 1 more author
Hao Su
This is my paper
classification cs.RO
keywords cross-embodiment learningworld modelsparticle-based dynamicsdexterous manipulationmodel-predictive controlembodiment scalinggraph neural networkssim-to-real transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that a single learned world model can serve as a shared brain across physically different hands. Its central claim is that when hands and objects are represented as sets of 3D particles, and actions as particle displacement fields, the underlying interaction dynamics become embodiment-invariant — so a model trained on some hands can predict object motion for hands it has never touched. The authors train a graph-based dynamics model on random interaction data from simulated robot hands and real human hands, then use it for model-predictive control on two physical hands with different degrees of freedom. They report three results: prediction error falls as more training embodiments are added; mixing simulation and human data beats either alone; and the same model drives real reshaping of plasticine on a 6-DoF and a 12-DoF hand with no per-embodiment fine-tuning. If correct, this offers a path to generalist manipulation skills that learn from heterogeneous, human-in-the-loop data.

Core claim

The paper's central claim is that the true transition function T — how object state evolves under hand motion — is embodiment-invariant, and that this invariance can be exploited by expressing hands and objects as sets of 3D particles with actions as end-effector particle displacement fields. In this shared geometric space, a single graph-based dynamics model can be trained by supervised regression on random interaction data gathered from multiple simulated robot hands (6–24 DoF) and from real human hand meshes reconstructed from multi-view cameras. The authors then use the same model inside a cross-entropy model-predictive controller, sampling joint actions, converting them to particle disp

What carries the argument

The central object is a unified particle-based state–action space: hands are sampled into sets of 3D particles, objects into their own particle sets, and an action is the displacement field of the end-effector particles between consecutive timesteps. This abstracts away joint spaces and kinematic structures. The learned world model is a graph neural network (specifically DPI-Net style) that treats particles as nodes in a radius graph and propagates local interaction information through message passing, giving spatial locality and equivariance to translation, rotation, and permutation. At deployment, only a forward kinematics model is needed to map joint commands into particle displacements,

Load-bearing premise

The load-bearing premise is the conjecture that the physical interaction process — object motion induced by contact — is identical across embodiments when expressed in particle space; the paper supports this only empirically on a few hands and two tasks.

What would settle it

Train on all but one hand and run the same MPC on the held-out hand; if that hand has fundamentally different actuation (e.g., underactuated tendon-driven vs. fully actuated) or finger count, and the object-tracking error is no better than a random-action baseline, the shared-dynamics claim fails. Concretely: hold out a two-finger gripper from a training set of multifingered hands; if the learned particle model cannot push or reshape an object to a target with MPC, the universality premise is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Adding more training embodiments improves zero-shot prediction on unseen hands, to the point where five training hands can match or beat direct training on the target hand.
  • Co-training on simulated robot data and real human data at roughly a 1:1 ratio outperforms either source alone, so simulation data can act as a regularizer for real human data.
  • The same particle-dynamics model, without fine-tuning, enables model-predictive control on two real hands with different kinematics and degrees of freedom (6 and 12), successfully reshaping deformable objects.
  • Deformable tasks with large contact surfaces benefit more from embodiment diversity than rigid pushing, suggesting the approach is most useful for contact-rich manipulation.
  • Because only forward kinematics is required at deployment, the method can scale to new hardware without motion retargeting or expert demonstrations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If particle-space dynamics are truly universal, the same representation should also transfer to non-anthropomorphic end effectors such as parallel grippers or soft manipulators, provided enough training diversity; the authors do not demonstrate this, but it follows directly from their premise.
  • The scaling results suggest a practical recipe for building generalist dynamics models: pool heterogeneous interaction data in a shared geometric space and let perception quality (hand and object reconstruction) become the main bottleneck rather than policy learning.
  • A testable consequence: prediction error on a held-out hand should keep decreasing as the number and diversity of training embodiments grows; if the error plateaus or rises at larger scales or more dissimilar embodiments, the shared-dynamics premise has limits.
  • The 1:1 co-training optimum is likely task- and data-dependent; whether it holds for other tasks, simulators, or data volumes is an open empirical question.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an embodiment-agnostic state/action representation for dexterous manipulation: hands are represented as sets of 3D particles, actions as end-effector particle displacements, and a GNN-based world model is trained jointly on simulated robot data and real human demonstrations. The learned dynamics model is then used with CEM-based MPC to control novel robot hands without fine-tuning. Three main claims are made: (i) increasing the number of training embodiments improves prediction on unseen hands; (ii) co-training simulated and real data outperforms either data source alone; (iii) the same particle-based model enables effective real-world control on two robot hands with different kinematics and DoF. The simulation scaling study enumerates all subsets of six hands, holds out the target hand, and reports MSE with confidence intervals. Real-world experiments compare a co-trained model against a human-only model on the Ability Hand and XHand for plasticine reshaping, reporting Chamfer/EMD errors and qualitative snapshots.

Significance. The particle-space representation is conceptually clean and the simulation scaling study is the strongest part of the paper: it systematically varies the number of training embodiments, holds out the target, and presents confidence intervals over all subsets, which is a rigorous way to study cross-embodiment scaling. The paper also makes a concrete, falsifiable claim that co-training simulation and real human data helps, and it demonstrates a working MPC pipeline on two physical hands. If the real-world control claim were backed by proper baselines and success metrics, this would be a solid contribution to cross-embodiment learning and model-based manipulation. The current weakness is that the headline 'effective control' result rests on a thin real-robot evaluation, and the co-training ratio selection introduces a potential selection bias.

major comments (3)
  1. [Section IV-D, Table I] The claim of 'effective cross-embodiment deployment' is not empirically established. The real-robot evaluation reports only Chamfer/EMD errors on two hands, with no comparison to a no-op policy, a random-action baseline, or the initial error before manipulation. The reported improvements of the co-trained model over human-only are 3–6%, which, despite non-overlapping confidence intervals, may be practically negligible. Without a task-success measure (e.g., a shape-match threshold or a per-trial success rate) and without the pre-manipulation error, the absolute CD/EMD values are uninterpretable. Figure 4 provides only qualitative snapshots. This is a load-bearing gap because finding (iii) and the abstract's 'effective control' claim depend on it.
  2. [Section IV-C, Figure 5 and Table I] The 1:1 sim-to-real data ratio is selected from Figure 5, which is evaluated on the same held-out human interactions that are used to support the 'co-training outperforms either alone' finding. The authors acknowledge that these evaluation values are approximations, but if the ratio is chosen by inspecting the same plot, the reported advantage is selection-biased. To make the co-training claim rigorous, the authors should select hyperparameters on a separate validation split (or use nested/selection-aware evaluation) and then report performance on a truly held-out split. Additionally, Table I omits a simulation-only baseline, which is needed to show the complementary benefit of real human data.
  3. [Section IV-B, Figure 2] The embodiment-scaling claim is established only for prediction MSE on unseen hands, not for the downstream MPC performance that motivates the paper. The 'zero-shot strength at x=5' conclusion is about the dynamics model's predictive accuracy; it does not show that MPC with the scaled model achieves better task success on a held-out hand. Without a simulated MPC evaluation across training-subset sizes, the link between the scaling law and the paper's central control-transfer goal remains indirect. I recommend adding a task-level MPC evaluation in simulation (even for a subset of target hands) to demonstrate that the prediction-error gains translate into control gains.
minor comments (6)
  1. [Section III-C] The DPI-Net architecture description omits key hyperparameters (number of encoder/decoder layers, hidden dimensions, radius threshold for edge construction, number of message-passing steps). These details are needed for reproducibility.
  2. [Section III-D] Planning hyperparameters are not reported. Please provide the CEM population size, number of iterations, planning horizon H, and the cost-weighting details.
  3. [Section IV-C, Figure 5] Please clarify how the sim:real ratio is computed (number of trajectories? number of timesteps?) and whether the '1:1' ratio is defined per task or globally. This affects the interpretation of the co-training recipe.
  4. [Section IV-D, Table I] The text states 'five trials per shape, for a total of 20 runs,' but the table reports only one mean and CI per hand/model. Please report per-shape results or explain how the CI is computed across the 20 runs.
  5. [Section I and III-A] The assertion that 'the underlying physical interaction process, captured by T, is universal' is a conjecture, as the paper itself notes. The experiments only cover two task families and a limited set of hands. Please add an explicit limitations paragraph stating that universality is not proven and that generalization to other contact regimes, object materials, and task families is untested.
  6. [Section IV-A] The paper lists a 'URDF variant of the Shadow Hand without its forearm' as a separate embodiment. Please clarify whether this is a genuinely different embodiment or a modification of the same kinematic chain, and justify counting it as an independent data source.

Circularity Check

0 steps flagged

No significant circularity: the central cross-embodiment scaling and transfer claims are evaluated on held-out embodiments and do not reduce to their inputs.

full rationale

The paper's main claimed derivation—that a particle-based world model trained on diverse embodiments transfers to unseen hands—is not circular. In simulation (Section IV-B), each target hand is held out and models are trained on subsets of the remaining hands, with MSE on the unseen hand reported in Figure 2; this is an external, held-out evaluation, not a fitted parameter renamed as a prediction. The real-world deployment (Section IV-D) evaluates models on physical Ability and XHand hardware, providing out-of-sample evidence that the same learned model can drive MPC. The particle state/action abstraction is a design choice, and the fact that end-effector displacement is provided as an action (Section III-A) makes hand-particle prediction partly tautological, but the paper's claims concern object dynamics and control effectiveness, which are learned and measured, not defined. The co-training recipe (Section IV-C) selects a 1:1 sim/real ratio using held-out human data; while this is model selection on a validation metric, the paper transparently states that these evaluations are approximations and then checks the chosen model on real hardware, so the direction of the co-training benefit is not forced by construction. The citation of the authors' prior 'embodiment scaling law' [3] is not load-bearing: the scaling result is established by the paper's own Figure 2. The main weaknesses—lack of no-op/random-action baselines in Table I, no success-rate metric, and the inclusion of simulated Ability/XHand data in the co-trained model—undermine the strength of the 'effective control' and 'co-training' conclusions, but they are experimental rigor issues, not circular derivation. No self-definitional, fitted-input-as-prediction, or self-citation-chain reduction is present.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The ledger shows the paper's contribution is empirical rather than derivational. The particle-space invariance and the sufficiency of the sampling/architecture choices are assumed, not proven; the main fitted quantity is the sim:real mixing ratio, tuned on the evaluation set. No new physical entities are introduced.

free parameters (5)
  • sim_to_real_data_ratio = 1:1 (searched over {0, 0.2, 0.5, 0.8, 1.0})
    Figure 5 sweeps five mixing ratios; the 1:1 ratio giving the best held-out human prediction error is then used for all real-robot deployments. This is a hyperparameter selected on the evaluation set, not derived.
  • particle_counts = not specified (N_e and N_o)
    The number of particles per hand/object is a design choice that affects graph density and model capacity; the paper does not report values or a sensitivity study (Section III-A).
  • radius-graph_radius_and_message_steps = not specified
    Neighbor radius for graph construction and number of message-passing steps L are not reported; model behavior and generalization depend on them (Section III-C).
  • motion_primitive_parameterization = hand-specified for each task
    Planning actions are restricted to hand-designed primitives (Fingers Pinch, Palm Press, Thumb Pinch) parameterized by z-rotation, translation, and thumb-DoF amounts (Section III-D); planning performance depends on these choices.
  • planning_hyperparameters = not specified
    CEM iterations, number of action samples, planning horizon H, and cost weights are not reported, making control results hard to reproduce.
axioms (5)
  • domain assumption Environment dynamics are embodiment-invariant when expressed in particle space
    Stated as the core conjecture in Section I and formalized in Section III-A ('the underlying physical interaction process, captured by T, is universal'). It is the load-bearing premise of the method.
  • domain assumption Particle sets with FPS preserve the geometry relevant to contact dynamics
    Section III-B samples hand and object meshes with farthest point sampling; the paper assumes this discretization loses no information needed for prediction.
  • domain assumption Graph neural network locality/equivariance inductive biases transfer across embodiments
    Section III-C adopts DPI-Net based on locality and equivariance; the paper does not ablate this architecture choice against alternatives.
  • domain assumption SAPIEN and Rewarped simulators provide realistic contact and material dynamics for transfer
    Section IV-A uses SAPIEN for rigid pushing and Rewarped for deformable objects; all scaling-law claims are computed inside these simulators, and sim-to-real transfer is assumed to be meaningful.
  • domain assumption Human hand motion in the chosen primitives lies on a low-dimensional manifold useful for robot control
    Section III-D invokes [12] and restricts planning to three primitives; the paper assumes these cover the task-relevant interaction space.

pith-pipeline@v1.3.0-alltime-deepseek · 10630 in / 10837 out tokens · 116212 ms · 2026-08-04T00:22:32.285902+00:00 · methodology

0 comments
read the original abstract

Cross-embodiment learning seeks to build generalist robots that learn from and operate across diverse morphologies, but differences in kinematics and action spaces hinder data sharing and control transfer. We ask: What structure can be shared across embodiments despite these differences? We argue that the physical interactions they induce can be modeled in a shared geometric space, allowing world models to provide a common interface for learning and control. To realize this idea, we represent human and robot hands as sets of 3D particles and define actions as end-effector particle displacement fields. This representation abstracts away embodiment-specific joint spaces while preserving the geometry and motion relevant to physical interaction. We train a graph-based world model on random interaction data from diverse simulated robot hands and real human hands, and integrate it with model-predictive control for deployment on new hardware. Experiments on rigid and deformable manipulation reveal three findings: increasing the diversity of training embodiments improves generalization to unseen hands; appropriately combining simulated and real-world data outperforms either source alone; and the same learned model enables effective control on robotic hands with distinct kinematics and degrees of freedom. These results position particle-based world models as a shared interface for learning from and for heterogeneous embodiments.

Figures

Figures reproduced from arXiv: 2511.01177 by Bo Ai, Hao Su, Henrik I. Christensen, Jiawei Fu, Tongzhou Mu, Weikang Wan, Yilun Du, Yulin Liu, Zihao He.

Figure 1
Figure 1. Figure 1: Overall framework. Our key idea is to represent both embodiments and objects as 3D particles, and actions as end-effector particle displacement fields. These state–action abstractions unify data and control across embodiments. (a) We train world models on random interaction data from diverse robot hands in simulation and from human demonstrations in the real world. (b) At deployment, joint action samples a… view at source ↗
Figure 2
Figure 2. Figure 2: Scaling trends in cross-embodiment world model learning. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cross-embodiment setups in simulation and the real [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results of cross-embodiment deployment [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Evaluating training recipes for bridging simulation [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ${\pi}_{0.7}$: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities

    cs.LG 2026-04 unverdicted novelty 7.0

    π₀.₇ is a steerable generalist robotic model that uses rich multimodal prompts including language, subgoal images, and performance metadata to achieve out-of-the-box generalization across tasks and robot bodies.

  2. EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World

    cs.RO 2026-04 conditional novelty 6.5

    A 1,362-hour multi-source egocentric human dataset and multi-lab study show co-training improves robot manipulation, but only when aligned human–robot data anchors transfer from diverse human data.

  3. LACE: Latent Visual Representation for Cross-Embodiment Learning

    cs.RO 2026-05 unverdicted novelty 6.0

    LACE aligns human-robot visual features via semantic distribution matching on corresponding body parts plus Gram loss, yielding 65% better zero-shot policy transfer than baseline DINO.

  4. EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World

    cs.RO 2026-04 unverdicted novelty 6.0

    EgoVerse releases 1,362 hours of standardized egocentric human data across 1,965 tasks and shows via multi-lab experiments that robot policy performance scales with human data volume when the data aligns with robot ob...

  5. One Hand to Rule Them All: Canonical Representations for Unified Dexterous Manipulation

    cs.RO 2026-02 unverdicted novelty 6.0

    A unified parameter space and canonical URDF enable cross-embodiment dexterous grasping policies with 81.9% zero-shot success on unseen hands like the 3-finger LEAP Hand.

  6. PhysCoRe: Physics-Corrected Residual World Models for Material-Aware Deformable Dynamics

    cs.RO 2026-07 conditional novelty 5.0

    PhysCoRe uses a differentiable MPM simulator with neural material inference and residual velocity correction, and reports more accurate future prediction on real deformable-object manipulation than optimization baselines.

Reference graph

Works this paper leans on

54 extracted references · 1 canonical work pages · cited by 5 Pith papers

  1. [1]

    A Review of Learning-Based Dynamics Models for Robotic Manipulation

    Bo Ai et al. “A Review of Learning-Based Dynamics Models for Robotic Manipulation”. In:Science Robotics(2025).DOI: 10.1126/scirobotics.adt1497

    doi:10.1126/scirobotics.adt1497 2025

  2. [2]

    RoboPack: Learning Tactile-Informed Dynam- ics Models for Dense Packing

    Bo Ai et al. “RoboPack: Learning Tactile-Informed Dynam- ics Models for Dense Packing”. In:Robotics: Science and Systems. 2024

    2024

  3. [3]

    Towards Embodiment Scaling Laws in Robot Locomotion

    Bo Ai et al. “Towards Embodiment Scaling Laws in Robot Locomotion”. In:Conference on Robot Learning (CoRL) (2025).URL:https : / / arxiv . org / abs / 2505 . 05753

    2025

  4. [4]

    π 0: A Vision-Language-Action Flow Model for General Robot Control

    Kevin Black et al. “π 0: A Vision-Language-Action Flow Model for General Robot Control”. In:CoRR abs/2410.24164 (2024).DOI:10.48550/ARXIV.2410. 24164. arXiv:2410 . 24164.URL:https : / / doi . org/10.48550/arXiv.2410.24164

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.24164 2024

  5. [5]

    One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Loco- motion

    Nico Bohlinger et al. “One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Loco- motion”. In:Conference on Robot Learning(2024)

    2024

  6. [6]

    Vividex: Learning vision-based dexter- ous manipulation from human videos

    Zerui Chen et al. “Vividex: Learning vision-based dexter- ous manipulation from human videos”. In:arXiv preprint arXiv:2404.15709(2024)

    Pith/arXiv arXiv 2024

  7. [7]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi et al. “Diffusion policy: Visuomotor policy learning via action diffusion”. In:The International Jour- nal of Robotics Research(2024).DOI:10 . 1177 / 02783649241273668

    2024

  8. [8]

    Learning universal policies via text-guided video generation

    Yilun Du et al. “Learning universal policies via text-guided video generation”. In:Advances in Neural Information Pro- cessing Systems (NeurIPS)(2023)

    2023

  9. [9]

    Visual foresight: Model-based deep reinforcement learning for vision-based robotic control

    Frederik Ebert et al. “Visual foresight: Model-based deep reinforcement learning for vision-based robotic control”. In: arXiv:1812.00568(2018)

    Pith/arXiv arXiv 2018

  10. [10]

    AnyDexGrasp: General Dexterous Grasping for Different Hands with Human-level Learning Efficiency

    Hao-Shu Fang et al. “AnyDexGrasp: General Dexterous Grasping for Different Hands with Human-level Learning Efficiency”. In:arXiv preprint arXiv:2502.16420(2025)

    Pith/arXiv arXiv 2025

  11. [11]

    RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot

    Hao-Shu Fang et al. “RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot”. In:2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 2024, pp. 653–660

    2024

  12. [12]

    The GRASP Taxonomy of Human Grasp Types

    Thomas Feix et al. “The GRASP Taxonomy of Human Grasp Types”. In:IEEE Trans. Hum. Mach. Syst.46.1 (2016), pp. 66–77

    2016

  13. [13]

    Dextreme: Transfer of agile in-hand manipulation from simulation to reality

    Ankur Handa et al. “Dextreme: Transfer of agile in-hand manipulation from simulation to reality”. In:2023 IEEE In- ternational Conference on Robotics and Automation (ICRA). IEEE. 2023, pp. 5977–5984

    2023

  14. [14]

    Learning Particle-Based World Model from Human for Robot Dexterous Manipulation

    Zhengdong Hong et al. “Learning Particle-Based World Model from Human for Robot Dexterous Manipulation”. In: 3rd RSS Workshop on Dexterous Manipulation: Learning and Control with Diverse Data. 2025.URL:https : / / openreview.net/forum?id=3zuBUoic2n

    2025

  15. [15]

    The foundation of efficient robot learning

    Leslie Pack Kaelbling. “The foundation of efficient robot learning”. In:Science369.6506 (2020), pp. 915–916.DOI: 10 . 1126 / science . aaz7597. eprint:https : / / www.science.org/doi/pdf/10.1126/science. aaz7597.URL:https://www.science.org/doi/ abs/10.1126/science.aaz7597

    doi:10.1126/science 2020

  16. [16]

    Poisson surface reconstruction

    Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. “Poisson surface reconstruction”. In:Proceedings of the fourth Eurographics symposium on Geometry processing. V ol. 7. 4. 2006

    2006

  17. [17]

    OpenVLA: An Open-Source Vision- Language-Action Model

    Moo Jin Kim et al. “OpenVLA: An Open-Source Vision- Language-Action Model”. In:Conference on Robot Learn- ing, 6-9 November 2024, Munich, Germany. Ed. by Pulkit Agrawal, Oliver Kroemer, and Wolfram Burgard. V ol. 270. Proceedings of Machine Learning Research. PMLR, 2024, pp. 2679–2713.URL:https : / / proceedings . mlr . press/v270/kim25c.html

    2024

  18. [18]

    DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differen- tiable Physics

    Sizhe Li et al. “DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differen- tiable Physics”. In:ICLR. OpenReview.net, 2023

    2023

  19. [19]

    Learning Particle Dynamics for Manipu- lating Rigid Bodies, Deformable Objects, and Fluids

    Yunzhu Li et al. “Learning Particle Dynamics for Manipu- lating Rigid Bodies, Deformable Objects, and Fluids”. In: ICLR (Poster). OpenReview.net, 2019

    2019

  20. [20]

    Learning Visible Connectivity Dynamics for Cloth Smoothing

    Xingyu Lin et al. “Learning Visible Connectivity Dynamics for Cloth Smoothing”. In:Conference on Robot Learning (CoRL). 2021

    2021

  21. [21]

    Contact-invariant optimization for hand manipulation

    Igor Mordatch, Zoran Popovi ´c, and Emanuel Todorov. “Contact-invariant optimization for hand manipulation”. In: Proceedings of the ACM SIGGRAPH/Eurographics sympo- sium on computer animation. 2012, pp. 137–144

    2012

  22. [22]

    Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collab- oration

    Abby O’Neill et al. “Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collab- oration”. In:IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13- 17, 2024. IEEE, 2024, pp. 6892–6903.DOI:10 . 1109 / ICRA57147.2024.10611477.URL:https://doi. org/10.1109/ICRA57147.2024.10611477

    arXiv 2024

  23. [23]

    Octo: An Open-Source Generalist Robot Policy

    Octo Model Team et al. “Octo: An Open-Source Generalist Robot Policy”. In:Proceedings of Robotics: Science and Systems. Delft, Netherlands, 2024

    2024

  24. [24]

    Global Planning for Contact-Rich Ma- nipulation via Local Smoothing of Quasi-Dynamic Contact Models

    Tao Pang et al. “Global Planning for Contact-Rich Ma- nipulation via Local Smoothing of Quasi-Dynamic Contact Models”. In:IEEE Transactions on Robotics39.6 (2023), pp. 4691–4711.DOI:10.1109/TRO.2023.3300230

    arXiv 2023

  25. [25]

    GET-Zero: Graph Embod- iment Transformer for Zero-shot Embodiment Generaliza- tion

    Austin Patel and Shuran Song. “GET-Zero: Graph Embod- iment Transformer for Zero-shot Embodiment Generaliza- tion”. In:2025 IEEE International Conference on Robotics and Automation (ICRA). 2025

    2025

  26. [26]

    In-hand object rotation via rapid motor adaptation

    Haozhi Qi et al. “In-hand object rotation via rapid motor adaptation”. In:Conference on Robot Learning. PMLR. 2023, pp. 1722–1732

    2023

  27. [27]

    DexMV: Imitation Learning for Dexter- ous Manipulation from Human Videos

    Yuzhe Qin et al. “DexMV: Imitation Learning for Dexter- ous Manipulation from Human Videos”. In:ECCV (39). V ol. 13699. Lecture Notes in Computer Science. Springer, 2022, pp. 570–587

    2022

  28. [28]

    Humanoid Policy ˜ Human Policy

    Ri-Zhao Qiu et al. “Humanoid Policy ˜ Human Policy”. In: arXiv preprint arXiv:2503.13441(2025)

    arXiv 2025

  29. [29]

    In-hand dexterous manipulation of piecewise- smooth 3-d objects

    Daniela Rus. “In-hand dexterous manipulation of piecewise- smooth 3-d objects”. In:The International Journal of Robotics Research18.4 (1999), pp. 355–381

    1999

  30. [30]

    Articulated hands: Force control and kinematic issues

    J Kenneth Salisbury and John J Craig. “Articulated hands: Force control and kinematic issues”. In:The International journal of Robotics research1.1 (1982), pp. 4–17

    1982

  31. [31]

    LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning

    Kenneth Shaw, Ananye Agarwal, and Deepak Pathak. “LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning”. In:Robotics: Science and Sys- tems (RSS)(2023)

    2023

  32. [32]

    Bimanual dexterity for complex tasks

    Kenneth Shaw et al. “Bimanual dexterity for complex tasks”. In:arXiv preprint arXiv:2411.13677(2024)

    Pith/arXiv arXiv 2024

  33. [33]

    RoboCook: Long-Horizon Elasto- Plastic Object Manipulation with Diverse Tools

    Haochen Shi et al. “RoboCook: Long-Horizon Elasto- Plastic Object Manipulation with Diverse Tools”. In:CoRL. V ol. 229. Proceedings of Machine Learning Research. PMLR, 2023, pp. 642–660

    2023

  34. [34]

    RoboCraft: Learning to see, simulate, and shape elasto-plastic objects in 3D with graph networks

    Haochen Shi et al. “RoboCraft: Learning to see, simulate, and shape elasto-plastic objects in 3D with graph networks”. In:Int. J. Robotics Res.43.4 (2024), pp. 533–549

    2024

  35. [35]

    RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks

    Haochen Shi et al. “RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks”. In: Robotics: Science and Systems. 2022

    2022

  36. [36]

    Dexterous contact-rich manipulation via the contact trust region

    HJ Suh et al. “Dexterous contact-rich manipulation via the contact trust region”. In:arXiv preprint arXiv:2505.02291 (2025)

    arXiv 2025

  37. [37]

    How to Grow a Mind: Statistics, Structure, and Abstraction

    Joshua B. Tenenbaum et al. “How to Grow a Mind: Statistics, Structure, and Abstraction”. In:Science331.6022 (2011), pp. 1279–1285

    2011

  38. [38]

    Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation

    Tongxuan Tian et al. “Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation”. In: arXiv preprint arXiv:2503.11999(2025)

    Pith/arXiv arXiv 2025

  39. [39]

    LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations

    Weikang Wan et al. “LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations”. In:arXiv preprint arXiv:2508.17547(2025)

    Pith/arXiv arXiv 2025

  40. [40]

    Unidexgrasp++: Improving dexterous grasping policy learning via geometry-aware curriculum and iterative generalist-specialist learning

    Weikang Wan et al. “Unidexgrasp++: Improving dexterous grasping policy learning via geometry-aware curriculum and iterative generalist-specialist learning”. In:Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, pp. 3891–3902

    2023

  41. [41]

    Dynamic-Resolution Model Learning for Object Pile Manipulation

    Yixuan Wang et al. “Dynamic-Resolution Model Learning for Object Pile Manipulation”. In:Robotics: Science and Systems. 2023

    2023

  42. [42]

    D(R,O) Grasp: A Unified Representa- tion of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping

    Zhenyu Wei et al. “D(R,O) Grasp: A Unified Representa- tion of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping”. In:arXiv preprint arXiv:2410.01702 (2024)

    Pith/arXiv arXiv 2024

  43. [43]

    CAGE: Causal Attention Enables Data-Efficient Generalizable Robotic Manipulation

    Shangning Xia et al. “CAGE: Causal Attention Enables Data-Efficient Generalizable Robotic Manipulation”. In: arXiv preprint arXiv:2410.14974(2024)

    Pith/arXiv arXiv 2024

  44. [44]

    SAPIEN: A SimulAted Part-based Inter- active ENvironment

    Fanbo Xiang et al. “SAPIEN: A SimulAted Part-based Inter- active ENvironment”. In:The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 2020

    2020

  45. [45]

    Stabilizing Rein- forcement Learning in Differentiable Multiphysics Simula- tion

    Eliot Xing, Vernon Luk, and Jean Oh. “Stabilizing Rein- forcement Learning in Differentiable Multiphysics Simula- tion”. In:International Conference on Learning Representa- tions (ICLR)(2025)

    2025

  46. [46]

    Pushing the Limits of Cross- Embodiment Learning for Manipulation and Navigation

    Jonathan Heewon Yang et al. “Pushing the Limits of Cross- Embodiment Learning for Manipulation and Navigation”. In: Robotics: Science and Systems XX, Delft, The Netherlands, July 15-19, 2024. Ed. by Dana Kulic et al. 2024.DOI:10. 15607 / RSS . 2024 . XX . 093.URL:https : / / doi . org/10.15607/RSS.2024.XX.093

    work page doi:10.15607/rss.2024.xx.093 2024

  47. [47]

    Multi-view Hand Reconstruction with a Point-Embedded Transformer

    Lixin Yang et al. “Multi-view Hand Reconstruction with a Point-Embedded Transformer”. In:CoRRabs/2408.10581 (2024)

    Pith/arXiv arXiv 2024

  48. [48]

    Learning interactive real-world simu- lators

    Mengjiao Yang et al. “Learning interactive real-world simu- lators”. In:International Conference on Learning Represen- tations (ICLR). 2024

    2024

  49. [49]

    Ruihan Yang et al.EgoVLA: Learning Vision-Language- Action Models from Egocentric Human Videos. 2025. arXiv: 2507.12440 [cs.RO].URL:https://arxiv.org/ abs/2507.12440

    Pith/arXiv arXiv 2025

  50. [50]

    DexterityGen: Foundation Con- troller for Unprecedented Dexterity

    Zhao-Heng Yin et al. “DexterityGen: Foundation Con- troller for Unprecedented Dexterity”. In:arXiv preprint arXiv:2502.04307(2025)

    Pith/arXiv arXiv 2025

  51. [51]

    AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation

    Kaifeng Zhang et al. “AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation”. In:Robotics: Science and Systems. 2024

    2024

  52. [52]

    Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling

    Mingtong Zhang, Kaifeng Zhang, and Yunzhu Li. “Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling”. In:8th Annual Conference on Robot Learning. 2024

    2024

  53. [53]

    Dexdlo: Learning goal-conditioned dexterous policy for dynamic manipulation of deformable linear objects

    Sun Zhaole, Jihong Zhu, and Robert B Fisher. “Dexdlo: Learning goal-conditioned dexterous policy for dynamic manipulation of deformable linear objects”. In:2024 IEEE international conference on robotics and automation (ICRA). IEEE. 2024, pp. 16009–16015

    2024

  54. [54]

    RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

    Brianna Zitkovich et al. “RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control”. In: CoRL. V ol. 229. Proceedings of Machine Learning Research. PMLR, 2023, pp. 2165–2183

    2023