REVIEW 5 major objections 5 minor 92 references
DR-LfD claims long-horizon manipulation needs only tens of demonstrations per skill if demonstrations are decomposed into contact-defined atomic skills and recombined by a task-and-motion planner.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 02:32 UTC pith:K3ERWTFL
load-bearing objection A promising TAMP-IL hybrid whose central data-complexity claim is plausible but rests on an untested unimodal contact-pose assumption; worth a careful referee, not a desk reject. the 5 major comments →
Decompose and Reorganize: Planning with Primitives and Visuomotor Policies Learned from Demonstrations
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On its own terms, the contribution is a complete 'decompose-and-reorganize' pipeline in which a vision-language model assists contact-based segmentation of demonstrations into skills, each skill is wrapped as a TAMP-compatible stream function (with learned primitives, keypose predictors, and safety checks), and an online perceive-plan-act loop re-plans locally after each step. The paper reports that this system outperforms end-to-end imitation baselines on spatially out-of-distribution tabletop tasks, reaches 100% success in peg insertion where baselines drop to 2–12%, and on a public bimanual benchmark achieves with 100 demonstrations per skill what end-to-end policies need 1000 demonstrati
What carries the argument
The load-bearing object is the contact-aware scene graph and its contact signatures: for each skill, a precondition graph and an effect graph that abstract away stochastic intermediate contact states while fixing the initiation and termination conditions. These signatures let TAMP chain skills learned from different demonstration sets, because the unimodal contact-pose assumption makes handed-off gripper–object states interchangeable. The supporting machinery is a set of network-integrated stream functions — attach/detach primitives, uni/bi-manual keypose streams, and a post-grasp bridge — that expose learned skills to the planner as conditional samplers, together with SO(3)-equivariant diff
Load-bearing premise
The framework depends on the assumption that for a common gripper–object contact edge, relative contact poses are similar across skills and demonstrations and follow a single unimodal distribution, so that skills learned from different demonstrations can be sequenced without retraining.
What would settle it
Collect demonstrations of a single skill where the gripper grasps the same object in two qualitatively different ways (e.g., top grasp versus side grasp), run the decomposition, and check whether the contact-pose clusters overlap. If they form two distinct modes, the unimodal assumption fails, and the chained primitives will hand off gripper–object states the policy never saw, producing missed handoffs or failed insertions.
If this is right
- The demonstration budget for a new long-horizon task grows with the number of skill types, not with sequence length.
- Skills can be improved in isolation: swapping a weak skill for a better-trained variant does not require recollecting demos or retraining the whole task.
- Out-of-distribution object placements are handled by first moving objects into in-distribution states with object-centric primitives, then starting the visuomotor policy from a keypose it has seen.
- Reachability and collision-safety constraints are enforced at plan time, with failed checks feeding back to the planner so it can insert regrasps or obstacle relocations.
- On a public bimanual benchmark, 100 demonstrations per skill outperform end-to-end imitation trained on 1000 demonstrations.
Where Pith is reading between the lines
- If the unimodal contact-pose assumption were relaxed by adding contact-mode arguments to the grasp predicate, the same framework could handle skills that grasp the same object in qualitatively different ways; this is the paper's own stated relaxation path, but it is not implemented.
- The decomposition principle suggests a complementary scaling route for foundation behavior models: instead of one monolithic policy memorizing full sequences, pretrained local skills can be wrapped as planner streams and chained by TAMP, which may improve long-horizon reliability without more data.
- A direct test of the framework's central claim would be to measure success rate versus number of demonstrations per skill for a fixed composite task; the paper reports results at specific budgets but does not plot the scaling curve.
- The object-centric equivariance advantage may shrink outside tabletop settings where objects are not stably segmented or where global context matters; extending to mobile or in-hand manipulation would clarify the boundary.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DR-LfD, a hybrid framework that decomposes human demonstrations into atomic skills using VLM-assisted, contact-aware scene-graph analysis, learns each skill either as an SO(3)-equivariant object-centric primitive or as a visuomotor policy, and wraps all skills as PDDLStream-compatible stream functions so a sampling-based TAMP planner can recombine them for novel long-horizon tasks. The central claim is that this decomposition changes the demonstration burden from exponential in the length of the skill sequence to roughly linear in the number of distinct skill types, with limited data per skill. The manuscript supports this claim with real-world ALOHA experiments (Peg-in-Hole variants, Object Handoff, Screwdriver Packing, Cup-sleeve Insertion, and two combined long-horizon tasks) and simulation benchmarks (LIBERO and DexMimicGen), plus specialized constraint-handling scenarios for reachability and collision safety.
Significance. If the central claim holds, DR-LfD is a meaningful step toward data-efficient long-horizon manipulation: it provides a concrete mechanism for sequencing closed-loop visuomotor policies with geometric reasoning, avoids 6-DoF pose estimation in learned primitives, and introduces network-integrated stream functions that are useful abstractions for TAMP with learned skills. The paper also contains refreshingly honest and detailed failure/limitation discussions (Section VII, Appendices B and D), including the dependence on manual VLM checking, the exponential TAMP search cost, and the depth-sensing requirement. The architecture is coherent and the experiments are broad, but as discussed below, the headline scaling claim and the key composability assumption are not yet empirically established at the level the paper claims.
major comments (5)
- [Section III-B] The unimodal contact-pose assumption is the load-bearing premise for skill recombination, but it is stated and never tested. The paper reports no measurement of the within-edge contact-pose distribution for a common gripper-object edge across different skills. The DexMimicGen ablation (Table VIII, DR-LfD (Planned Contact): 0% and 35%) demonstrates exactly the failure mode this assumption rules out: contact poses that deviate from the demonstration distribution produce OOD wrist-camera observations and policy failure. To support the central data-complexity claim, the authors should either measure that across all skill sets used in Sections VI-B and VI-C the common gripper-object edges have overlapping, unimodal contact-pose distributions, or extend the AtGrasp predicate with explicit contact modes (as the paper itself mentions) and show that TAMP can plan over those modes.
- [Abstract, Section I, Table VIII] The headline claim that DR-LfD 'transforms the learning problem from one requiring exponential demonstration data over possible skill sequences to one whose demonstration burden scales with the number of distinct skill types' is asserted rather than demonstrated. No experiment varies the number of demonstrations per skill or the length/combination of skills and measures success as a function of data. Table VIII compares 100 vs 1000 demos for end-to-end baselines but not for DR-LfD's own per-skill data budget, and Section VI-C fixes 20 demos per skill without a data curve. Please add a scaling experiment (e.g., per-skill demo counts vs composite success for 1, 2, and 3 skills, or sequence length vs required demos), or clearly restrict the claim to a structural architectural argument.
- [Section VI-C, Fig. 10] The Three-Tapes task, one of the two headline long-horizon composition results, has no reported success rate or trial count; only the 21-step skeleton is described. The Cup-Sponge-Screwdriver task reports a 50% success rate (and later 35%/50% in the clean-background substitution) but no per-subtask failure analysis. These two tasks are the direct evidence for Q3 ('Can DR-LfD complete tasks with novel goals that require multiple steps?'), and as reported they do not quantitatively establish the composability claim. Please report per-task success over a stated number of trials, and include per-subgoal completion statistics (the paper already computes 'average completed steps' for the substitution study, so this is feasible).
- [Tables IV–VIII] All success rates are point estimates over 10 or 20 trials with no confidence intervals or significance tests. With 20 trials, the difference between 90% and 100% is within binomial noise; with 10 trials, 10-percentage-point differences are inconclusive. Given the strong claims of outperforming DP, SDP, ACT, and a pretrained VLA baseline, the headline comparative claims need at least Wilson or bootstrap confidence intervals, or raw per-trial outcomes, for the main comparisons. This is especially important in Table V, where DR-LfD and π_{0.5} differ by 5–10 percentage points in several settings.
- [Appendix D, Section IV-A1] The VLM grounding stage is not fully automated: the authors state that 'we manually check the correctness of VLM output before the next data processing step' and that task descriptions were manually crafted when not provided. This is disclosed honestly, but it means the 'VLM-assisted' decomposition pipeline requires a human in the loop, and the claimed scalability to new tasks inherits that manual cost. At minimum, the paper should state this limitation prominently in the main text (not only in Appendix D), and ideally report the effect of human-corrected vs raw VLM output on downstream success rates.
minor comments (5)
- [Section VII-C] The discussion of depth-sensing limitations is good, but it appears only after the experiments; consider moving a sentence about the depth requirement into Section IV-B where the object-centric primitives are introduced.
- [Section V-B2] The MDF safety map is built by replaying demonstrations that were also used to train the policy. This is not circular in the damaging sense, but it may be optimistic if the policy deviates from the demonstrations during deployment. Please add one sentence acknowledging this and why the min-over-time construction is still conservative in the intended use.
- [Section VI-A2] The real-world baseline π_{0.5} is described only as 'a stronger pretrained behavior model.' Since the comparison with this baseline is used to make a general claim about VLA models and OOD generalization, please give the exact model, checkpoint, and fine-tuning protocol.
- [Table XI] The VLM SGCorr scores for Screwdriver Packing (0.50) and LIBERO-long (0.53) are notably lower than the other tasks; this is informative, but the reader is left wondering how these grounding errors propagate. A short paragraph in Appendix D connecting the low SGCorr cases to the reported success rates would help.
- [Throughout] There are several spacing/hyphenation artifacts (e.g., 'DR-LfD(D...' in the abstract, 'aframework' in the introduction). A careful proofread is needed before publication.
Circularity Check
Framework is not circular in its central decomposition claim; two minor self-referential definitions (MDF safety envelope, keypose admissibility) are present but not load-bearing.
specific steps
-
fitted input called prediction
[Section V-B.2 (Policy Safety Constraint), Eq. (3)]
"We replay the demonstration during T a in simulation. At each timestep t∈T a, we voxelize the workspace and record the voxels occupied by the manipulator or related objects as an occupancy map O t. ... A minimum distance field (MDF) is then obtained by aggregating across timesteps, updating each voxel in the workspace with the minimum distance value encountered."
The IsSafePolicy test stream certifies safety by checking whether obstacles lie inside the MDF, but the MDF is constructed from the same demonstrations that trained the visuomotor policy. Thus 'safe' is defined as 'inside the policy's own demonstration swept volume'; the safety certification reduces to the training envelope rather than independently predicting the stochastic policy's collision-freeness. This is a self-referential constraint, though it is a minor component and the paper's real-world safety experiments provide external evidence.
-
self definitional
[Section IV-C (Learning Visuomotor Policies and Keyposes)]
"To this end, we design a keypose predictor Φ kp to produce admissible initiation and termination conditions for each segmented skill. ... We learn the distribution of keyposes in both initial and terminal contact states, i.e., P(q a.Gpre , qa.Geff |λ a), conditioned on the skill description λ a."
The keypose predictor's notion of 'admissible' is defined as being inside the distribution of keyposes extracted from the very demonstrations used to train the visuomotor policy. Therefore, ensuring in-distribution initiation is true by construction: the predictor reproduces the policy's own training distribution. Appendix B makes this explicit: the primitive and keypose predictors 'jointly drive the system observation O to follow the same distribution as that of the policy learned for a.' This is a self-consistency argument, though the paper's successful task rollouts supply independent empirical support.
full rationale
The central claim of DR-LfD is not circular. The data-complexity argument—that decomposing demonstrations into atomic skills and letting TAMP sequence them shifts the demonstration burden from exponential in sequence length to linear in skill types—follows from the architecture and is not an equation that reduces the output to a fitted constant. The learned policies, primitives, and keypose predictors are all trained from demonstrations, but the TAMP skeleton and skill sequencing are computed by a planner, and the empirical benchmarks (LIBERO, DexMimicGen, real-world ALOHA tasks, and ablation comparisons) provide independent content. The two identified items are minor self-referential definitions: the MDF safety map is built by replaying the demonstrations that define the policy, so IsSafePolicy certifies staying inside the training swept volume; and the keypose predictor's 'admissible' states are, by construction, the keypose distribution of the same demonstrations. These are honest design choices and are tested empirically, so they do not undermine the central decomposition claim. No load-bearing self-citation chain or imported uniqueness theorem is present; self-citations such as [15], [17], and [74] are peripheral. Overall, the paper's derivation is largely self-contained, with only minor circular content.
Axiom & Free-Parameter Ledger
free parameters (4)
- Contact distance threshold
- In-hand velocity alignment threshold
- Reachability map thresholds
- MDF safety distance threshold
axioms (6)
- domain assumption Unimodal, repeatable contact-pose distribution across skills
- domain assumption Factorized observation model P(O|s) = P(q|s_h) P(I_wrist|s_h-o) P(I_3rd|s_h, s_o)
- domain assumption Conditional independence of skill trajectories given point cloud and language
- domain assumption Contact signatures Gpre/Geff are sufficient for skill preconditions and effects
- domain assumption Perception pipeline accuracy
- ad hoc to paper VLM grounding is correct after manual checking
read the original abstract
Successfully automating dexterous, long-horizon robotic manipulation requires frameworks capable of both high-level reasoning and fine-grained execution. Traditional task and motion planning (TAMP), while excellent at symbolic planning, is often brittle in contact-rich operations. Simultaneously, imitation learning (IL), while effective in manipulation tasks with visual feedback, is limited by its low capability in spatial generalization and multi-stage operation. To reconcile their complementary strengths and limitations, we propose DR-LfD (Decomposed and Reorganized Skills Learned from Demonstrations), a framework that seamlessly integrates visuomotor policies into a TAMP-gated decision-making system. Based on contact relationships, DR-LfD decomposes human demonstrations into atomic skills, which are reproduced as visuomotor policies or object-centric primitives. The initiation, termination, and constraints of the visuomotor policies are carefully modeled and implemented in a TAMP-compatible form, enabling reorganization of skills learned from different sources. DR-LfD transforms the learning problem from one requiring exponential demonstration data over possible skill sequences to one whose demonstration burden scales with the number of distinct skill types, with limited data for each skill. Through comprehensive real-world and simulation benchmarking across diverse scenarios, we demonstrate the strong performance of DR-LfD on tasks involving multiple steps, unseen setups, and physical constraints. Project website: https://dr-lfd.github.io/DR-LfD-website.
Figures
Reference graph
Works this paper leans on
-
[1]
An incremental constraint-based framework for task and motion planning,
N. T. Dantam, Z. K. Kingston, S. Chaudhuri, and L. E. Kavraki, “An incremental constraint-based framework for task and motion planning,” The International Journal of Robotics Research, vol. 37, no. 10, pp. 1134–1151, 2018
2018
-
[2]
Task and motion planning for execution in the real,
T. Pan, R. Shome, and L. E. Kavraki, “Task and motion planning for execution in the real,”IEEE Transactions on Robotics, vol. 40, pp. 3356– 3371, 2024
2024
-
[3]
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,
T. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,” inRobotics: Science and Systems XIX. Robotics: Science and Systems Foundation, Jul. 2023
2023
-
[4]
GELLO: A general, low-cost, and intuitive teleoperation framework for robot manipulators,
P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel, “GELLO: A general, low-cost, and intuitive teleoperation framework for robot manipulators,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 12 156–12 163
2024
-
[5]
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion,
C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion,” inRobotics: Science and Systems XIX. Robotics: Science and Systems Foundation, Jul. 2023
2023
-
[6]
Implicit behavioral cloning,
P. Florence, C. Lynch, A. Zeng, O. A. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mordatch, and J. Tompson, “Implicit behavioral cloning,” inConference on Robot Learning. PMLR, 2022, pp. 158– 168
2022
-
[7]
What matters in learning from offline human demonstrations for robot manipulation,
A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart ´ın-Mart´ın, “What matters in learning from offline human demonstrations for robot manipulation,” inConference on Robot Learning. PMLR, 2021, pp. 1678–1690
2021
-
[8]
3D Diffuser Actor: Policy diffusion with 3d scene representations,
T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki, “3D Diffuser Actor: Policy diffusion with 3d scene representations,” inConference on Robot Learning. PMLR, 2024, pp. 1949–1974
2024
-
[9]
3D Diffusion Policy: Generalizable visuomotor policy learning via simple 3D repre- sentations,
Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3D Diffusion Policy: Generalizable visuomotor policy learning via simple 3D repre- sentations,” inICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024
2024
-
[10]
π0: A vision-language-action flow model for general robot control,
K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichteret al., “π0: A vision-language-action flow model for general robot control,” inRobotics: Science and Systems, 2025
2025
-
[11]
ALOHA Unleashed: A simple recipe for robot dexterity,
T. Z. Zhao, J. Tompson, D. Driess, P. Florence, K. Ghasemipour, C. Finn, and A. Wahid, “ALOHA Unleashed: A simple recipe for robot dexterity,” inConference on Robot Learning. PMLR, 2024, pp. 1910–1924
2024
-
[12]
RDT-1B: A diffusion foundation model for bimanual manipulation,
S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “RDT-1B: A diffusion foundation model for bimanual manipulation,” inInternational Conference on Learning Representations (ICLR), 2025
2025
-
[13]
A careful exami- nation of large behavior models for multitask dexterous manipulation,
J. Barreiros, A. Beaulieu, A. Bhat, R. Cory, E. Cousineau, H. Dai, C.-H. Fang, K. Hashimoto, M. Z. Irshad, M. Itkinaet al., “A careful exami- nation of large behavior models for multitask dexterous manipulation,” Science Robotics, vol. 11, no. 113, p. eaea6201, 2026
2026
-
[14]
A Bimanual Manipulation Taxonomy,
F. Krebs and T. Asfour, “A Bimanual Manipulation Taxonomy,”IEEE Robot. Autom. Lett., vol. 7, no. 4, pp. 11 031–11 038, Oct. 2022
2022
-
[15]
BiKC: Keypose-conditioned consistency policy for bimanual robotic manipulation,
D. Yu, H. Xu, Y . Chen, Y . Ren, and J. Pan, “BiKC: Keypose-conditioned consistency policy for bimanual robotic manipulation,” inInternational Workshop on the Algorithmic Foundations of Robotics (WAFR), 2024
2024
-
[16]
Combined task and motion planning for a dual-arm robot to use a suction cup tool,
H. Chen, W. Wan, and K. Harada, “Combined task and motion planning for a dual-arm robot to use a suction cup tool,” in2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids). IEEE, 2019, pp. 446–452
2019
-
[17]
Enabling versatility and dexterity of the dual- arm manipulators: A general framework toward universal cooperative manipulation,
Y . Ren, Z. Zhou, Z. Xu, Y . Yang, G. Zhai, M. Leibold, F. Ni, Z. Zhang, M. Buss, and Y . Zheng, “Enabling versatility and dexterity of the dual- arm manipulators: A general framework toward universal cooperative manipulation,”IEEE Transactions on Robotics, vol. 40, 2024
2024
-
[18]
Efficient task/motion planning for a dual-arm robot from language instructions and cooking images,
K. Takata, T. Kiyokawa, I. G. Ramirez-Alpizar, N. Yamanobe, W. Wan, and K. Harada, “Efficient task/motion planning for a dual-arm robot from language instructions and cooking images,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 12 058–12 065
2022
-
[19]
Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Tasks,
J. Gao, Z. Tao, N. Jaquier, and T. Asfour, “Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Tasks,” in2024 IEEE International Conference on Robotics and Automation (ICRA), 2024
2024
-
[20]
Gripper keypose and object pointflow as interfaces for bimanual robotic manipulation,
Y . Yang, Z. Cai, Y . Tian, J. Zeng, and J. Pang, “Gripper keypose and object pointflow as interfaces for bimanual robotic manipulation,” in Robotics: Science and Systems, 2025
2025
-
[21]
Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning,
Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. J. Fan, and Y . Zhu, “Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning,” in2025 IEEE Interna- tional Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 16 923–16 930
2025
-
[22]
Mimicgen: A data generation system for scalable robot learning using human demonstrations,
A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox, “Mimicgen: A data generation system for scalable robot learning using human demonstrations,” inConference on Robot Learning. PMLR, 2023, pp. 1820–1864
2023
-
[23]
T. Chen, Z. Chen, B. Chen, Z. Cai, Y . Liu, Z. Li, Q. Liang, X. Lin, Y . Ge, Z. Guet al., “Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation,”arXiv preprint arXiv:2506.18088, 2025
Pith/arXiv arXiv 2025
-
[24]
Novel demonstration generation with gaussian splatting enables robust one-shot manipulation,
S. Yang, W. Yu, J. Zeng, J. Lv, K. Ren, C. Lu, D. Lin, and J. Pang, “Novel demonstration generation with gaussian splatting enables robust one-shot manipulation,” inRobotics: Science and Systems, 2025
2025
-
[25]
Equibot: SIM(3)-Equivariant diffusion policy for generalizable and data efficient learning,
J. Yang, Z.-a. Cao, C. Deng, R. Antonova, S. Song, and J. Bohg, “Equibot: SIM(3)-Equivariant diffusion policy for generalizable and data efficient learning,” inConference on Robot Learning. PMLR, 2024
2024
-
[26]
Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning,
Z. Xue, S. Deng, Z. Chen, Y . Wang, Z. Yuan, and H. Xu, “Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning,” inRobotics: Science and Systems, 2025
2025
-
[27]
You only teach once: Learn one-shot bimanual robotic manipulation from video demonstrations,
H. Zhou, R. Wang, Y . Tai, Y . Deng, G. Liu, and K. Jia, “You only teach once: Learn one-shot bimanual robotic manipulation from video demonstrations,” inRobotics: Science and Systems, 2025
2025
-
[28]
Diffusion-EDFs: Bi-Equivariant denoising generative modeling on SE(3) for visual robotic manipulation,
H. Ryu, J. Kim, H. An, J. Chang, J. Seo, T. Kim, Y . Kim, C. Hwang, J. Choi, and R. Horowitz, “Diffusion-EDFs: Bi-Equivariant denoising generative modeling on SE(3) for visual robotic manipulation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18 007–18 018
2024
-
[29]
RiEMann: Near real-time SE(3)-equivariant robot manipulation without point cloud segmentation,
C. Gao, Z. Xue, S. Deng, T. Liang, S. Yang, L. Shao, and H. Xu, “RiEMann: Near real-time SE(3)-equivariant robot manipulation without point cloud segmentation,” inConference on Robot Learning. PMLR, 2024
2024
-
[30]
3d equivariant visuomotor policy learning via spherical projection,
B. Hu, D. Wang, D. Klee, H. Tian, X. Zhu, H. Huang, R. Platt, and R. Walters, “3d equivariant visuomotor policy learning via spherical projection,” inAdvances in Neural Information Processing Systems, 2025
2025
-
[31]
A practical guide for incorporating symmetry in diffusion policy,
D. Wang, B. Hu, S. Song, R. Walters, and R. Platt, “A practical guide for incorporating symmetry in diffusion policy,” inAdvances in Neural Information Processing Systems, 2025
2025
-
[32]
Plan-seq- learn: Language model guided rl for solving long horizon robotics tasks,
M. Dalal, T. Chiruvolu, D. Chaplot, and R. Salakhutdinov, “Plan-seq- learn: Language model guided rl for solving long horizon robotics tasks,” inInternational Conference on Learning Representations (ICLR), 2024
2024
-
[33]
League: Guided skill learning and abstraction for long-horizon manipulation,
S. Cheng and D. Xu, “League: Guided skill learning and abstraction for long-horizon manipulation,”IEEE Robotics and Automation Letters, 2023
2023
-
[34]
STAP: Sequencing task- agnostic policies,
C. Agia, T. Migimatsu, J. Wu, and J. Bohg, “STAP: Sequencing task- agnostic policies,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 7951–7958
2023
-
[35]
Logic-skill program- ming: An optimization-based approach to sequential skill planning,
T. Xue, A. Razmjoo, S. Shetty, and S. Calinon, “Logic-skill program- ming: An optimization-based approach to sequential skill planning,” in Robotics: Science and Systems, 2024
2024
-
[36]
Spire: Synergistic planning, imitation, and reinforcement learning for long- horizon manipulation,
Z. Zhou, A. Garg, D. Fox, C. Garrett, and A. Mandlekar, “Spire: Synergistic planning, imitation, and reinforcement learning for long- horizon manipulation,” inConference on Robot Learning. PMLR, 2024, pp. 2347–2371. 18
2024
-
[37]
NOD-TAMP: Multi- step manipulation planning with neural object descriptors,
S. Cheng, C. R. Garrett, A. Mandlekar, and D. Xu, “NOD-TAMP: Multi- step manipulation planning with neural object descriptors,” inCoRL 2023 Workshop on Learning Effective Abstractions for Planning (LEAP), 2023
2023
-
[38]
Logic learning from demonstrations for multi-step manipulation tasks in dynamic environ- ments,
Y . Zhang, T. Xue, A. Razmjoo, and S. Calinon, “Logic learning from demonstrations for multi-step manipulation tasks in dynamic environ- ments,”IEEE Robotics and Automation Letters, 2024
2024
-
[39]
Generative factor chaining: Coor- dinated manipulation with diffusion-based factor graph,
U. A. Mishra, Y . Chen, and D. Xu, “Generative factor chaining: Coor- dinated manipulation with diffusion-based factor graph,” inICRA 2024 Workshop - Back to the Future: Robot Learning Going Probabilistic, 2024
2024
-
[40]
Human-in-the-loop task and motion planning for imitation learning,
A. Mandlekar, C. R. Garrett, D. Xu, and D. Fox, “Human-in-the-loop task and motion planning for imitation learning,” inConference on Robot Learning. PMLR, 2023, pp. 3030–3060
2023
-
[41]
Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,
C. Garrett, A. Mandlekar, B. Wen, and D. Fox, “Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,” inConference on Robot Learning. PMLR, 2024, pp. 2750–2790
2024
-
[42]
Learning compositional behaviors from demonstration and language,
W. Liu, N. Nie, R. Zhang, J. Mao, and J. Wu, “Learning compositional behaviors from demonstration and language,” in8th Annual Conference on Robot Learning, 2024
2024
-
[43]
Learning a thousand tasks in a day,
K. Dreczkowski, P. Vitiello, V . V osylius, and E. Johns, “Learning a thousand tasks in a day,”Science Robotics, vol. 10, no. 108, p. eadv7594, 2025
2025
-
[44]
Learning Reusable Manipulation Strategies,
J. Mao, T. Lozano-P ´erez, J. B. Tenenbaum, and L. P. Kaelbling, “Learning Reusable Manipulation Strategies,” in7th Annual Conference on Robot Learning, Aug. 2023
2023
-
[45]
InterPreT: Interactive predicate learning from language feedback for generalizable task planning,
M. Han, Y . Zhu, S.-C. Zhu, Y . N. Wu, and Y . Zhu, “InterPreT: Interactive predicate learning from language feedback for generalizable task planning,” inRobotics: Science and Systems, 2024
2024
-
[46]
UniDomain: Pretraining a unified PDDL domain from real-world demonstrations for generalizable robot task planning,
H. Ye, Y . Xiao, C. Lu, and P. Cai, “UniDomain: Pretraining a unified PDDL domain from real-world demonstrations for generalizable robot task planning,” inAdvances in Neural Information Processing Systems, 2025
2025
-
[47]
Vlm see, robot do: Human demo video to robot action plan via vision language model,
B. Wang, J. Zhang, S. Dong, I. Fang, and C. Feng, “Vlm see, robot do: Human demo video to robot action plan via vision language model,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 17 215–17 222
2025
-
[48]
Do as i can, not as i say: Grounding language in robotic affordances,
M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausmanet al., “Do as i can, not as i say: Grounding language in robotic affordances,” inConference on Robot Learning. PMLR, 2022, pp. 287–318
2022
-
[49]
Progprompt: Generating situated robot task plans using large language models,
I. Singh, V . Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using large language models,” in2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 11 523– 11 530
2023
-
[50]
Differentiable physics and stable modes for tool-use and manipulation planning,
M. A. Toussaint, K. R. Allen, K. A. Smith, and J. B. Tenenbaum, “Differentiable physics and stable modes for tool-use and manipulation planning,” 2018
2018
-
[51]
Object-centric task and motion planning in dynamic environments,
T. Migimatsu and J. Bohg, “Object-centric task and motion planning in dynamic environments,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 844–851, 2020
2020
-
[52]
Sampling-based methods for factored task and motion planning,
C. R. Garrett, T. Lozano-P ´erez, and L. P. Kaelbling, “Sampling-based methods for factored task and motion planning,”The International Journal of Robotics Research, vol. 37, no. 13-14, pp. 1796–1825, 2018
2018
-
[53]
PDDLStream: Integrating symbolic planners and blackbox sam- plers via optimistic adaptive planning,
——, “PDDLStream: Integrating symbolic planners and blackbox sam- plers via optimistic adaptive planning,” inProceedings of the interna- tional conference on automated planning and scheduling, vol. 30, 2020, pp. 440–448
2020
-
[54]
Long-horizon manipulation of unknown objects via task and motion planning with estimated affordances,
A. Curtis, X. Fang, L. P. Kaelbling, T. Lozano-P ´erez, and C. R. Garrett, “Long-horizon manipulation of unknown objects via task and motion planning with estimated affordances,” in2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022, pp. 1940–1946
2022
-
[55]
Unseen object instance segmentation for robotic environments,
C. Xie, Y . Xiang, A. Mousavian, and D. Fox, “Unseen object instance segmentation for robotic environments,”IEEE Transactions on Robotics, vol. 37, no. 5, pp. 1343–1359, 2021
2021
-
[56]
High precision grasp pose detection in dense clutter,
M. Gualtieri, A. Ten Pas, K. Saenko, and R. Platt, “High precision grasp pose detection in dense clutter,” in2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp. 598–605
2016
-
[57]
6-DoF Graspnet: Varia- tional grasp generation for object manipulation,
A. Mousavian, C. Eppner, and D. Fox, “6-DoF Graspnet: Varia- tional grasp generation for object manipulation,” inProceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 2901– 2910
2019
-
[58]
Gen6D: Generalizable model-free 6-DoF object pose estimation from rgb images,
Y . Liu, Y . Wen, S. Peng, C. Lin, X. Long, T. Komura, and W. Wang, “Gen6D: Generalizable model-free 6-DoF object pose estimation from rgb images,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 298–315
2022
-
[59]
POPE: 6- DoF promptable pose estimation of any object in any scene with one reference,
Z. Fan, P. Pan, P. Wang, Y . Jiang, D. Xu, and Z. Wang, “POPE: 6- DoF promptable pose estimation of any object in any scene with one reference,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 7771–7781
2024
-
[60]
FoundationPose: Unified 6D pose estimation and tracking of novel objects,
B. Wen, W. Yang, J. Kautz, and S. Birchfield, “FoundationPose: Unified 6D pose estimation and tracking of novel objects,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 17 868–17 879
2024
-
[61]
Unified task and motion planning us- ing object-centric abstractions of motion constraints,
A. Agostini and J. Piater, “Unified task and motion planning us- ing object-centric abstractions of motion constraints,”arXiv preprint arXiv:2312.17605, 2023
arXiv 2023
-
[62]
Grounded SAM: Assembling open-world models for diverse visual tasks,
T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yanet al., “Grounded SAM: Assembling open-world models for diverse visual tasks,”arXiv preprint arXiv:2401.14159, 2024
Pith/arXiv arXiv 2024
-
[63]
Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model,
H. K. Cheng and A. G. Schwing, “Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model,” inEuropean conference on computer vision. Springer, 2022, pp. 640–658
2022
-
[64]
D (r, o) grasp: A unified representation of robot and object interaction for cross-embodiment dexterous grasping,
Z. Wei, Z. Xu, J. Guo, Y . Hou, C. Gao, Z. Cai, J. Luo, and L. Shao, “D (r, o) grasp: A unified representation of robot and object interaction for cross-embodiment dexterous grasping,” in2025 IEEE International Conference on Robotics and Automation (ICRA), 2025
2025
-
[65]
Vector Neurons: A general framework for SO(3)-equivariant networks,
C. Deng, O. Litany, Y . Duan, A. Poulenard, A. Tagliasacchi, and L. J. Guibas, “Vector Neurons: A general framework for SO(3)-equivariant networks,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 12 200–12 209
2021
-
[66]
Bert: Pre-training of deep bidirectional transformers for language understanding,
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inPro- ceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), 2019
2019
-
[67]
EFEM: Equivariant neural field expectation maximization for 3D object seg- mentation without scene supervision,
J. Lei, C. Deng, K. Schmeckpeper, L. Guibas, and K. Daniilidis, “EFEM: Equivariant neural field expectation maximization for 3D object seg- mentation without scene supervision,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 4902–4912
2023
-
[68]
Denoising Diffusion Probabilistic Models,
J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models,” inAdvances in Neural Information Processing Systems, vol. 33. Curran Associates, Inc., 2020, pp. 6840–6851
2020
-
[69]
Should ebms model the energy or the score?
T. Salimans and J. Ho, “Should ebms model the energy or the score?” inEnergy Based Models Workshop-ICLR 2021, 2021
2021
-
[70]
Generative modeling by estimating gradients of the data distribution,
Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,”Advances in neural information processing systems, vol. 32, 2019
2019
-
[71]
Score-Based Generative Modeling through Stochastic Differential Equations,
Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” inInternational Conference on Learning Rep- resentations, Oct. 2020
2020
-
[72]
Coast: Constraints and streams for task and motion planning,
B. Vu, T. Migimatsu, and J. Bohg, “Coast: Constraints and streams for task and motion planning,” in2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 14 875–14 881
2024
-
[73]
Schedulestream: Temporal planning with samplers for gpu-accelerated multi-arm task and motion planning & scheduling,
C. Garrett and F. Ramos, “Schedulestream: Temporal planning with samplers for gpu-accelerated multi-arm task and motion planning & scheduling,” in2026 IEEE International Conference on Robotics and Automation (ICRA), 2026
2026
-
[74]
Gpu-accelerated incremental euclidean distance transform for online motion planning of mobile robots,
Y . Chen, S. Lai, J. Cui, B. Wang, and B. M. Chen, “Gpu-accelerated incremental euclidean distance transform for online motion planning of mobile robots,”IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6894–6901, 2022
2022
-
[75]
One demo is worth a thousand trajectories: Action-view augmentation for visuomotor policies,
C. Pan, L. Liang, D. Bauer, E. Cousineau, B. Burchfiel, S. Feng, and S. Song, “One demo is worth a thousand trajectories: Action-view augmentation for visuomotor policies,” in9th Annual Conference on Robot Learning, 2025
2025
-
[76]
Rail: Reachability-aided imitation learning for safe policy execution,
W. Jung, D. Anthony, U. A. Mishra, N. R. Arachchige, M. Bronars, D. Xu, and S. Kousik, “Rail: Reachability-aided imitation learning for safe policy execution,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 3582–3589
2025
-
[77]
Libero: Benchmarking knowledge transfer for lifelong robot learning,
B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learning,”Advances in Neural Information Processing Systems, vol. 36, pp. 44 776–44 791, 2023
2023
-
[78]
Grasp pose detection in point clouds,
A. Ten Pas, M. Gualtieri, K. Saenko, and R. Platt, “Grasp pose detection in point clouds,”The International Journal of Robotics Research, vol. 36, no. 13-14, pp. 1455–1473, 2017
2017
-
[79]
M2t2: Multi-task masked transformer for object-centric pick and place,
W. Yuan, A. Murali, A. Mousavian, and D. Fox, “M2t2: Multi-task masked transformer for object-centric pick and place,” in7th Annual Conference on Robot Learning, 2023. 19
2023
-
[80]
TiPToP: A modular open-vocabulary planning system for robotic manipulation,
W. Shen, N. Kumar, S. Chintalapudi, J. Wang, C. Watson, E. Hu, J. Cao, D. Jayaraman, L. P. Kaelbling, and T. Lozano-P ´erez, “TiPToP: A modular open-vocabulary planning system for robotic manipulation,” arXiv preprint arXiv:2603.09971, 2026
Pith/arXiv arXiv 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.