Pith. sign in

REVIEW 4 major objections 6 minor 5 cited by

Constraint-Preserving Data Generation for Visuomotor Policy Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CP-Gen claims that a single expert demonstration can generate the training data needed for visuomotor policies to generalize across novel object geometries and poses, including zero-shot sim-to-real.

desk verdict Keypoint-trajectory constraints buy real geometry generalization in imitation data generation, but the core assumption is untested and the reporting has gaps; worth a serious referee. read the letter →

arxiv 2508.03944 v1 pith:ISWI67OZ submitted 2025-08-05 cs.RO cs.AI

Kevin Lin , Varun Ragunath , Andrew McAlinden , Aaditya Prasad , Jimmy Wu , Yuke Zhu , Jeannette Bohg This is my paper
classification cs.ROcs.AI
keywords visuomotorpolicylearningdatagenerationkeypoint-trajectoryconstraintsone-shotimitationzero-shotsim-to-realrobotmanipulationobjectgeometrygeneralizationdiffusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CP-Gen claims that one expert demonstration, expanded through keypoint-trajectory constraints, is enough to train visuomotor policies that handle unseen object shapes and poses and transfer from simulation to the real world. The method decomposes the source trajectory into free-space motions and robot skills, expresses each skill as a requirement that keypoints on the robot or grasped object follow a trajectory defined in a task-relevant object's frame, and then generates new demonstrations by sampling geometry and pose transforms while solving for robot configurations that keep the keypoints on track. If the claim holds, it replaces the need for large teleoperated datasets in many manipulation tasks. The paper reports that policies trained this way average 77% success, versus 50% for the best baseline.

What carries the argument

The central object is the keypoint-trajectory constraint. For each skill segment, actor keypoints—points fixed on the robot gripper or a grasped object—are required to track a target keypoint trajectory expressed in the local frame of a task-relevant object. When the object's geometry or pose changes, CP-Gen applies the sampled transform to the object and its associated keypoints or trajectory, then solves for robot joint configurations that minimize the distance between forward-kinematic actor keypoints and transformed target keypoints, with a temporal smoothness penalty. This turns geometry-aware data generation into an optimization problem, with collision-free motion planning between skil

What would settle it

Run the keypoint-matching optimization on a tight-tolerance task such as Wine Glass Spiral Hanging, but execute the resulting joint trajectories in a simulator with contacts and Coulomb friction before any success filtering, and count failures where the keypoints follow the reference yet the object jams or slips. A high failure rate at zero mismatch would refute the assumption that the constraint preservation suffices.

Watch

Extended reading notes

Core claim

On its own terms, CP-Gen establishes that representing robot skills as keypoint-trajectory constraints makes automated data generation geometry-aware. Instead of replaying pose-transformed rigid actions, the method transforms the object and its associated keypoints under the sampled geometry, then solves per timestep for joint configurations that make forward-kinematic actor keypoints track the transformed reference trajectory. Because these constraints are defined relative to task-relevant objects, a single source demonstration can be multiplied into a thousand synthetic demonstrations spanning new geometries and poses, and those demonstrations are sufficient to train closed-loop visuomotor

Load-bearing premise

Everything rests on assuming that if the generated robot motion tracks the transformed keypoint trajectory exactly, the skill will actually succeed; contact forces, friction, and grasp slip could break this even when keypoints match.

Editorial extensions

If this is right

  • If the central claim is right, a single teleoperated demonstration can seed a large training dataset for tasks that previously required months of collection.
  • Generated data should let policies generalize across object aspect ratios and shape variations, not just rigid poses, because the constraint itself is transformed with the geometry.
  • The benchmark numbers suggest CP-Gen's advantage grows when geometry variations are present, while it matches pose-only methods on tasks where only poses vary.
  • The same pipeline, using keypoint matching and motion planning, yields zero-shot sim-to-real policies on multi-stage, non-prehensile, and tight-tolerance manipulation tasks.
  • The method assumes a success detector exists per task; the quality of generated data depends on the accuracy of that detector.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The keypoint formulation is object-relative, so it should carry over to bimanual or mobile manipulation as long as actor keypoints can be annotated, even though the paper demonstrates single-arm results.
  • Automating keypoint selection with learned keypoint detectors would likely remove the main manual bottleneck and could make CP-Gen a drop-in data engine for new tasks without extra teleoperation.
  • Because the data generation is constraint-based, the same machinery could also be used to check or filter candidate demonstrations, not just to generate them.
  • The fixed skill sequence limits task-level variation; coupling CP-Gen with a task-level planner could generate demonstrations for longer-horizon tasks at no additional human cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. CP-Gen is a data-generation method for closed-loop visuomotor policies. From one expert demonstration, it segments the trajectory into free-space motion and skill segments, annotates keypoints on the robot/grasped object, and encodes each skill as a keypoint-trajectory constraint in a task-relevant object frame. New demonstrations are created by sampling object pose and geometry transforms, transforming the target keypoint trajectories, solving Eq. (2) for joint configurations with L-BFGS-B, and motion planning between skills; failed rollouts are removed by a task-specific success detector. A Diffusion Policy is trained on 1000 successful generated demos. The paper reports evaluations on eight MimicGen tasks, eight geometry-generalization variants, and four real-world tasks, comparing with MimicGen and DemoGen.

Significance. If correct, the result is significant: it suggests that one expert demo, combined with keypoint-trajectory constraints and a simulator, can replace hundreds of teleoperated demonstrations for tasks involving pose and scale geometry variation. The paper's strengths include the clean formulation, broad simulation evaluation with error bars, real-world validation, honest limitations, and ablations that separate pose vs geometry generalization. The evaluation metric is not circular: success detectors are shared across methods and the reported quantity is held-out policy success. The main gaps are evidential: the sufficiency of the keypoint constraint is assumed rather than analyzed; data-generation filtering may bias coverage; and the real-world trial counts are not reported. These concerns are addressable in revision.

major comments (4)
  1. [Section 4 (keypoint-trajectory constraint); Eq. (2)] The central premise is stated as an assumption: 'adapting and preserving this keypoint-trajectory constraint to a new scene leads to successful execution.' This is not derived, and Eq. (2) only minimizes keypoint error with a smoothness penalty; it does not enforce exact preservation. The success detector filters obvious failures, but the mechanism is not validated. I recommend adding a failure-mode analysis (keypoint error vs. contact/friction failures) and tempering 'preserving' to 'encouraging' unless exact preservation is shown.
  2. [Table 3(a); Fig. 3] On Geometry variants, data-generation success is only 62% (vs. 89% for Pose Only). The final 1000-demo training set therefore excludes 38% of sampled scenes, and rejection is likely correlated with transform magnitude near the scale-range extremes in Fig. 3. The paper reports only aggregate policy success, so the claim that policies generalize across the full sampled geometry range is not supported. Please report policy success stratified by scale/geometry difficulty, or otherwise demonstrate that retained demos cover the intended distribution.
  3. [Section 5.2, Table 2] The four real-world tasks are a key part of the contribution, but the number of trials per task is not reported. Without N, the standard errors (e.g., 0.80±0.13, 0.00±0.00) are uninterpretable and the zero-shot sim2real claim cannot be assessed. Please report trial counts and per-geometry results.
  4. [Section 5.1; Fig. 3] The simulation geometry generalization is limited to non-uniform scaling of the same object meshes. The abstract/conclusions say 'novel object geometries' without this qualifier, which overstates the evidence. Please state this scope explicitly in the claims; the Limitations paragraph already acknowledges category-level shape changes as future work.
minor comments (6)
  1. [General] Please add a code/data availability statement; the project page is mentioned but the manuscript does not state whether code is released.
  2. [Section 4.2/4.3] Details of the L-BFGS-B initialization, the value/selection of lambda in Eq. (2), and the motion planner configuration are missing; include them in an appendix for reproducibility.
  3. [Section 5.1 (DemoGen baseline)] The DemoGen baseline is reimplemented as 'CP-Gen without the keypoint trajectory constraints.' Clarify how closely this matches the original DemoGen and cite the exact version used, since the comparison is central to the ablation claim.
  4. [Section 4.2] The paper assumes a fixed end-effector-to-grasped-object frame transformation. This assumption is important and should be discussed in the Limitations; real grasps often vary in position/orientation.
  5. [Figure 2] Figure 2 is dense, and the in-figure annotation asking to enlarge points appears to be an editing artifact. The figure should be cleaned and the font/point sizes increased.
  6. [Section 5.1, Table 1] The abstract and text quote aggregate success rates (e.g., 85% vs 63%) that are averages over the RGB and D+S modalities. State this averaging explicitly in the table caption or text to avoid confusion.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the main result is an externally evaluated empirical claim, not an identity or a fitted-input prediction.

full rationale

The paper's derivation chain is: segment a source trajectory (Section 4.1), represent skills as keypoint-trajectory constraints (Eq. 1), adapt those constraints under sampled geometry/pose transforms, solve inverse kinematics (Eq. 2), plan motions, filter with a success detector, and train a Diffusion Policy. The only load-bearing premise is stated explicitly in Section 4: 'We assume that adapting and preserving this keypoint-trajectory constraint to a new scene leads to successful execution.' This is a hypothesis being tested, not an equation that defines the evaluation metric. Generated data are filtered by a success detector 'which we assume exists for each task'; the detector is applied identically to CP-Gen, MimicGen, and DemoGen, and the reported metric is task success on held-out geometry/pose resets, not keypoint-constraint satisfaction. No parameter is fitted to the evaluation outcomes, and no 'prediction' is just a renamed fit. The self-citations (e.g., DROID, DexMimicGen, Equivact, Text2motion in Related Work and Limitations) are contextual, not load-bearing. The Limitations section acknowledges remaining assumptions (manual keypoint annotation, fixed skill sequence, need for a geometric transform between object instances), which are generality limitations, not circularity. The Table 3(a) coverage concern (62% vs 89% data-generation success on geometry variants) is a potential validity/coverage risk, but it does not make the result circular.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The manual keypoint and skill annotations, per-task scale ranges, and lambda are hand-set inputs that the results depend on. They are not derived from data or physics, and they are not reported with enough detail for independent reproduction.

free parameters (3)
  • lambda (temporal smoothness weight in Eq. 2) = not reported
    Balances keypoint matching against joint-space smoothness; hand-set and unspecified in the manuscript.
  • Per-task uniform scale sampling ranges (e.g., [0.6,1.4] for StackThreeG, [0.9,1.1] for ThreadingG) = varies by task
    Chosen manually per task (Figure 3); these define the geometry-augmentation distribution and therefore the difficulty of the Geometry Generalization benchmark.
  • Manual keypoint selection, reference-object choice, and skill segmentation = hand-annotated
    No automated selection; the method's success depends on the annotator picking keypoints that capture the task constraint. Listed as a limitation by the authors.
assumptions (5)
  • domain assumption An expert demonstration can be segmented into free-space motion segments and robot skill segments without harm to task success.
    Used throughout Section 4 (decomposition of tau_src); misclassification would break the motion-plan/skill-optimization split.
  • domain assumption Preserving the keypoint-trajectory constraint under sampled object geometry and pose transforms is sufficient for successful task execution in the new scene.
    Explicitly stated in Section 4: 'We assume that adapting and preserving this keypoint-trajectory constraint to a new scene leads to successful execution.' The method and all downstream policy evaluations depend on this.
  • domain assumption A reliable success detector exists for each task and is used to filter generated demonstrations.
    Section 4: 'after filtering out any failed demonstrations using a success detector (which we assume exists for each task)'.
  • domain assumption The transformation from the end-effector frame to the grasped object frame is fixed while the object is grasped.
    Assumed in Section 4.2 when computing actor keypoints in the world frame via forward kinematics; grasp slip would make generated trajectories inconsistent.
  • standard math L-BFGS-B can reliably solve the per-timestep keypoint-matching optimization (Eq. 2) to a configuration that tracks the target trajectory.
    The method relies on gradient-based optimization from SciPy; no guarantee of global optimality is given, and the smoothness weight lambda is not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Constraint-Preserving Data Generation for Visuomotor Policy Learning." pith.science (2026). https://pith.science/paper/ISWI67OZ

@misc{pith2026250803944,
  author       = {Pith},
  title        = {Pith review of: Constraint-Preserving Data Generation for Visuomotor Policy Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ISWI67OZ}},
  note         = {Machine review of arXiv:2508.03944}
}
read the original abstract

Large-scale demonstration data has powered key breakthroughs in robot manipulation, but collecting that data remains costly and time-consuming. We present Constraint-Preserving Data Generation (CP-Gen), a method that uses a single expert trajectory to generate robot demonstrations containing novel object geometries and poses. These generated demonstrations are used to train closed-loop visuomotor policies that transfer zero-shot to the real world and generalize across variations in object geometries and poses. Similar to prior work using pose variations for data generation, CP-Gen first decomposes expert demonstrations into free-space motions and robot skills. But unlike those works, we achieve geometry-aware data generation by formulating robot skills as keypoint-trajectory constraints: keypoints on the robot or grasped object must track a reference trajectory defined relative to a task-relevant object. To generate a new demonstration, CP-Gen samples pose and geometry transforms for each task-relevant object, then applies these transforms to the object and its associated keypoints or keypoint trajectories. We optimize robot joint configurations so that the keypoints on the robot or grasped object track the transformed keypoint trajectory, and then motion plan a collision-free path to the first optimized joint configuration. Experiments on 16 simulation tasks and four real-world tasks, featuring multi-stage, non-prehensile and tight-tolerance manipulation, show that policies trained using CP-Gen achieve an average success rate of 77%, outperforming the best baseline that achieves an average of 50%.

Figures

Figures reproduced from arXiv: 2508.03944 by the authors.

Figure 1
Figure 1. CP-Gen uses one expert demonstration and keypoint-trajectory constraints to generate diverse demonstrations in simulation involving novel object geometries and poses, enabling large-scale policy train￾ing and zero-shot sim-to-real transfer. For the Wine Glass Spiral Hanging task, the first keypoint-trajectory is anchored to the wine glass and constrains the motion of points on the end effector. The second keypoint￾t… view at source ↗
Figure 2
Figure 2. CP-Gen Method. In the Source Data Processing stage (top), starting from an expert demonstration τsrc, we (a) segment the trajectory into free-space motion and skill segments, (b) annotate keypoints on the robot or grasped object (dubbed actor keypoints), and (c) convert each skill segment into a keypoint-trajectory con￾straint by extracting a keypoint-trajectory expressed in the frame of a task-relevant object. Afte… view at source ↗
Figure 3
Figure 3. Simulation Tasks and Geometry Generalization Variants. Top: Tasks from the MimicGen bench￾mark [6] and object geometries sampled from our proposed Geometry Generalization task variants. Bottom: Uniform scale sampling ranges applied in Geometry Generalization task variants. 4.2 Skill Segment Adaptation Given a skill segment τskill and a current scene observation, our goal is to adapt the original keypoint￾trajectory … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Real World Tasks. We evaluate on four challenging real-world tasks and show that policies trained on CP-Gen generated simulation datasets can transfer zero-shot to the real world. demonstrations. MimicGen first parses each source demonstration into object-centric subta…

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DockAnywhere: Data-Efficient Visuomotor Policy Learning for Mobile Manipulation via Novel Demonstration Generation

    cs.RO 2026-04 unverdicted novelty 7.0 of 10

    DockAnywhere lifts single demonstrations to diverse docking points via structure-preserving augmentation and point-cloud spatial editing to improve viewpoint generalization in visuomotor policies for mobile manipulation.

  2. ShapeGen: Robotic Data Generation for Category-Level Manipulation

    cs.RO 2026-04 unverdicted novelty 6.0 of 10

    ShapeGen generates shape-diverse 3D robotic manipulation demonstrations without simulators by curating a functional shape library and applying a minimal-annotation pipeline for novel, physically plausible data.

  3. A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

    cs.RO 2026-04 unverdicted novelty 6.0 of 10

    Sim-and-real co-training for robot policies is driven primarily by balanced cross-domain representation alignment and secondarily by domain-dependent action reweighting.

  4. Learning in ImaginationLand: Omnidirectional Policies through 3D Generative Models (OP-Gen)

    cs.RO 2025-09 conditional novelty 6.0 of 10

    A robot policy trained on one real demonstration plus AI-generated 3D views succeeds from novel initial poses, including opposite-side starts, across six real manipulation tasks.

  5. ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning

    cs.RO 2025-12 conditional novelty 5.0 of 10

    ReinforceGen uses imitation learning, RL fine-tuning of skill policies, and real-time pose replanning to reach over 80% success on five long-horizon Robosuite tasks from only 10 human demonstrations.

Reference graph

Works this paper leans on

54 extracted references · 35 canonical work pages · cited by 5 Pith papers

  1. [1]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. In Proceedings of Robotics: Science and Systems, Daegu, Republic of Korea, July 2023. doi:10.15607/RSS.2023.XIX.016

    doi:10.15607/rss.2023.xix.016 2023

  2. [2]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. In Conference on Robot Learning (CoRL), 2024

    work page 2024

  3. [3]

    T. Z. Zhao, J. Tompson, D. Driess, P. Florence, K. Ghasemipour, C. Finn, and A. Wahid. Aloha unleashed: A simple recipe for robot dexterity. arXiv preprint arXiv:2410.13126, 2024

    arXiv 2024

  4. [4]

    Aldaco, T

    J. Aldaco, T. Armstrong, R. Baruch, J. Bingham, S. Chan, K. Draper, D. Dwibedi, C. Finn, P. Florence, S. Goodrich, et al. Aloha 2: An enhanced low-cost hardware for bimanual teleop- eration. arXiv preprint arXiv:2405.02292, 2024

    arXiv 2024

  5. [5]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, P. D. Fagan, J. Hejna, M. Itkina, M. Lepert, Y . J. Ma, P. T. Miller, J. Wu, S. Belkhale, S. Dass, H. Ha, A. Jain, A. Lee, Y . Lee, M. Memmel, S. Park, I. Radosavovic, K. Wang, A. Zhan, K. Black, C. Chi, K. B. Hatch, S. Lin, J. ...

    2024

  6. [6]

    Mandlekar, S

    A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. In 7th Annual Conference on Robot Learning, 2023. 9

    work page 2023

  7. [7]

    C. R. Garrett, A. Mandlekar, B. Wen, and D. Fox. Skillgen: Automated demonstration genera- tion for efficient skill learning and deployment. In 8th Annual Conference on Robot Learning,

    work page

  8. [8]

    Jiang, Y

    Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y . Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

Show all 54 references
  1. [9]

    Ameperosa, J

    E. Ameperosa, J. A. Collins, M. Jain, and A. Garg. Rocoda: Counterfactual data augmentation for data-efficient robot learning from demonstrations. arXiv preprint arXiv:2411.16959, 2024

    2024 arXiv

  2. [10]

    Z. Xue, S. Deng, Z. Chen, Y . Wang, Z. Yuan, and H. Xu. Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning. arXiv preprint arXiv:2502.16932 , 2025

    2025 arXiv

  3. [11]

    D. Wang, R. Walters, and R. Platt. SO (2)-Equivariant Reinforcement Learning. In Interna- tional Conference on Learning Representations, 2022

    2022

  4. [12]

    D. Wang, R. Walters, X. Zhu, and R. Platt. Equivariant Q Learning in Spatial Action Spaces. In 5th Annual Conference on Robot Learning, 2021

    2021

  5. [13]

    Huang, D

    H. Huang, D. Wang, A. Tangri, R. Walters, and R. Platt. Leveraging Symmetries in Pick and Place. The International Journal of Robotics Research, 2023

    2023

  6. [14]

    Simeonov, Y

    A. Simeonov, Y . Du, Y .-C. Lin, A. R. Garcia, L. P. Kaelbling, T. Lozano-P´erez, and P. Agrawal. SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields. In Conference on Robot Learning, pages 835–846. PMLR, 2023

    2023

  7. [15]

    C. Pan, B. Okorn, H. Zhang, B. Eisner, and D. Held. TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulation. In Conference on Robot Learning , pages 1783–1792. PMLR, 2023

    2023

  8. [16]

    Huang, D

    H. Huang, D. Wang, X. Zhu, R. Walters, and R. Platt. Edge Grasp Network: A Graph-Based SE(3)-invariant Approach to Grasp Detection. In International Conference on Robotics and Automation (ICRA), 2023

    2023

  9. [17]

    S. Liu, M. Xu, P. Huang, X. Zhang, Y . Liu, K. Oguchi, and D. Zhao. Continual Vision-based Reinforcement Learning with Group Symmetries. In Conference on Robot Learning , pages 222–240. PMLR, 2023

    2023

  10. [18]

    M. Jia, D. Wang, G. Su, D. Klee, X. Zhu, R. Walters, and R. Platt. SEIL: Simulation-augmented Equivariant Imitation Learning. In International Conference on Robotics and Automation (ICRA), 2023

    2023

  11. [19]

    S. Kim, B. Lim, Y . Lee, and F. C. Park. Se (2)-equivariant pushing dynamics models for tabletop object manipulations. In Conference on Robot Learning , pages 427–436. PMLR, 2023

    2023

  12. [20]

    Kohler, A

    C. Kohler, A. S. Srikanth, E. Arora, and R. Platt. Symmetric models for visual force policy learning. arXiv preprint arXiv:2308.14670, 2023

    2023 arXiv

  13. [21]

    H. H. Nguyen, A. Baisero, D. Klee, D. Wang, R. Platt, and C. Amato. Equivariant reinforce- ment learning under partial observability. InConference on Robot Learning, pages 3309–3320. PMLR, 2023

    2023

  14. [22]

    Nguyen, T

    H. Nguyen, T. Kozuno, C. C. Beltran-Hernandez, and M. Hamaya. Symmetry-aware reinforce- ment learning for robotic assembly under partial observability with a soft wrist.arXiv preprint arXiv:2402.18002, 2024

    2024 arXiv

  15. [23]

    Eisner, Y

    B. Eisner, Y . Yang, T. Davchev, M. Vecerik, J. Scholz, and D. Held. Deep SE(3)-equivariant geometric reasoning for precise placement tasks. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=2inBuwTyL2

    2024

  16. [24]

    Huang, D

    H. Huang, D. Wang, R. Walters, and R. Platt. Equivariant Transporter Network. In Robotics: Science and Systems, 2022. 10

    2022

  17. [25]

    Simeonov, Y

    A. Simeonov, Y . Du, A. Tagliasacchi, J. B. Tenenbaum, A. Rodriguez, P. Agrawal, and V . Sitz- mann. Neural descriptor fields: Se (3)-equivariant object representations for manipulation. In ICRA, 2022

    2022

  18. [26]

    H. Ryu, H. in Lee, J.-H. Lee, and J. Choi. Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  19. [27]

    Huang, O

    H. Huang, O. Howell, X. Zhu, D. Wang, R. Walters, and R. Platt. Fourier Transporter: Bi- Equivariant Robotic Manipulation in 3D. arXiv preprint arXiv:2401.12046, 2024

    2024 arXiv

  20. [28]

    X. Zhu, D. Wang, O. Biza, G. Su, R. Walters, and R. Platt. Sample Efficient Grasp Learning Using Equivariant Models. In Robotics: Science and Systems, 2022

    2022

  21. [29]

    X. Zhu, D. Wang, G. Su, O. Biza, R. Walters, and R. Platt. On robot grasp learning using equivariant models. Autonomous Robots, 47(8):1175–1193, 2023

    2023

  22. [30]

    D. Wang, S. Hart, D. Surovik, T. Kelestemur, H. Huang, H. Zhao, M. Yeatman, J. Wang, R. Walters, and R. Platt. Equivariant diffusion policy. In 8th Annual Conference on Robot Learning, 2024. URL https://openreview.net/forum?id=wD2kUVLT1g

    2024

  23. [31]

    Yang, Z.-a

    J. Yang, Z.-a. Cao, C. Deng, R. Antonova, S. Song, and J. Bohg. Equibot: Sim(3)-equivariant diffusion policy for generalizable and data efficient learning. In 8th Annual Conference on Robot Learning, 2024

    2024

  24. [32]

    J. Yang, C. Deng, J. Wu, R. Antonova, L. Guibas, and J. Bohg. Equivact: Sim(3)-equivariant visuomotor policies beyond rigid object manipulation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 9249–9255. IEEE, 2024

    2024

  25. [33]

    Hoque, A

    R. Hoque, A. Mandlekar, C. Garrett, K. Goldberg, and D. Fox. Intervengen: Interventional data generation for robust and data-efficient robot imitation learning, 2024

    2024

  26. [34]

    Huang, C

    W. Huang, C. Wang, Y . Li, R. Zhang, and L. Fei-Fei. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. In 8th Annual Conference on Robot Learning, 2024. URL https://openreview.net/forum?id=9iG3SEbMnL

    2024

  27. [35]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  28. [36]

    Gpt-4 technical report, 2023

    OpenAI. Gpt-4 technical report, 2023

    2023

  29. [37]

    Manuelli, W

    L. Manuelli, W. Gao, P. Florence, and R. Tedrake. kpam: Keypoint affordances for category- level robotic manipulation. In ISRR, 2019

    2019

  30. [38]

    Z. Qin, K. Fang, Y . Zhu, F.-F. Li, and S. Savarese. Keto: Learning keypoint representations for tool manipulation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 7278–7285. IEEE, 2020

    2020

  31. [39]

    Sundaresan, J

    P. Sundaresan, J. Grannen, B. Thananjeyan, A. Balakrishna, M. Laskey, K. Stone, J. E. Gonza- lez, and K. Goldberg. Learning rope manipulation policies using dense object descriptors trained on synthetic depth data. In Proceedings of the IEEE International Conference on Robotic...

    2020

  32. [40]

    Manuelli, Y

    L. Manuelli, Y . Li, P. Florence, and R. Tedrake. Keypoints into the future: Self-supervised cor- respondence in model-based reinforcement learning. arXiv preprint arXiv:2009.05085, 2020

    2009 arXiv

  33. [41]

    R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing , 16(5):1190–1208, 1995. doi:10.1137/ 0916069. URL https://doi.org/10.1137/0916069. 11

    1995 doi

  34. [42]

    Virtanen, R

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey,˙I. Polat, Y . Feng...

    2020

  35. [43]

    Sundaralingam, S

    B. Sundaralingam, S. K. S. Hari, A. Fishman, C. Garrett, K. V . Wyk, V . Blukis, A. Millane, H. Oleynikova, A. Handa, F. Ramos, N. Ratliff, and D. Fox. curobo: Parallelized collision-free minimum-jerk robot motion generation, 2023

    2023

  36. [44]

    Thomason, Z

    W. Thomason, Z. Kingston, and L. E. Kavraki. Motions in microseconds via vectorized sampling-based planning. In IEEE International Conference on Robotics and Automation . URL http://arxiv.org/abs/2309.14545

    arXiv

  37. [45]

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Proceedings of Robotics: Science and Systems (RSS), 2023

    2023

  38. [46]

    Dalal, M

    M. Dalal, M. Liu, W. Talbott, C. Chen, D. Pathak, J. Zhang, and R. Salakhutdinov. Local policies enable zero-shot long-horizon manipulation. International Conference of Robotics and Automation, 2025

    2025

  39. [47]

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  40. [48]

    Huang, F

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022

    2022 arXiv

  41. [49]

    K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg. Text2motion: From natural language instructions to feasible plans. Autonomous Robots, 47(8):1345–1365, 2023

    2023

  42. [50]

    B. Liu, Y . Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477, 2023

    2023 arXiv

  43. [51]

    L. P. Kaelbling and T. Lozano-P ´erez. Hierarchical task and motion planning in the now. In 2011 IEEE International Conference on Robotics and Automation , pages 1470–1477, 2011. doi:10.1109/ICRA.2011.5980391

    2011

  44. [52]

    C. R. Garrett, T. Lozano-P´erez, and L. P. Kaelbling. Pddlstream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 30, pages 440–448, 2020

    2020

  45. [53]

    C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kaelbling, and T. Lozano-P´erez. Integrated task and motion planning. Annual review of control, robotics, and autonomous systems, 4:265–293, 2021. 12

    2021

  46. [2024]

    URL https://openreview.net/forum?id=YOFrRTDC6d

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.