Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A learned value function makes robot grasping reactive in clutter

desk verdict Value-guided MPC for grasping is an impressive empirical system, but the missing ablation against a geometric cost means the central contribution is not yet isolated. read the letter →

arxiv 2509.06201 v1 pith:BSWNSIIM submitted 2025-09-07 cs.RO cs.AIcs.LG

Jun Yamada , Adithyavairavan Murali , Ajay Mandlekar , Clemens Eppner , Ingmar Posner , Balakumar Sundaralingam This is my paper
classification cs.ROcs.AIcs.LG
keywords modelpredictivecontrolgraspingvaluefunctionclosed-loopmanipulationpointcloudofflinereinforcementlearningmotionplanningclutteredscenes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Grasp-MPC is a closed-loop 6-DoF grasping policy: instead of committing to a precomputed grasp trajectory, the robot re-plans every control step using a learned value function as the task cost. The value function is trained on 2.1 million synthetic reach-to-grasp trajectories spanning 8,515 objects, with sparse labels that mark whether a motion plan reached an annotated feasible grasp. Because the value function consumes a segmented point cloud of the target object and the gripper pose relative to its center, it transfers to new objects and to cluttered scenes without retraining. In simulation it nearly matches an open-loop oracle when grasp poses are perfect (74.9% vs 79.0%) and beats all baselines when poses are noisy or predicted (60.3% and 67.2%); in real cluttered scenes it reaches 74.4% vs 41.1% for the open-loop pipeline. The paper's point is that the main failure mode of open-loop grasping—grasp-prediction error and object movement—can be handled by control rather than by better prediction alone.

What carries the argument

The load-bearing object is the learned grasp-value function V(x_t), trained with the Bellman error objective on 2M synthetic trajectories. Its input is a centered segmented point cloud of the target object, encoded by a point-cloud network, concatenated with the end-effector pose relative to the point-cloud centroid; its output is a positive scalar approximating expected discounted cost-to-go. In the MPC loop this value is summed over the horizon as the task cost, with weight 1000, alongside the planner's default world-collision, self-collision, and smoothness costs. The value function is what gives MPC a notion of 'this trajectory ends in a good grasp' beyond geometric distance to a possibl

What would settle it

Relabel the same 2M trajectories using a physics simulation that actually closes the gripper and lifts the object, retrain the value function, and rerun the simulated and real evaluations. If grasp success does not improve—or if the physics-labeled value function performs no better than the planner-labeled one—then the planner-success proxy is not the limiting factor; if it improves substantially, the reported gains depend on that proxy rather than on the MPC loop itself.

Watch

Extended reading notes

Core claim

The central claim is that a value function trained offline on motion-planned reach-to-grasp trajectories can serve as the task cost inside model predictive control, and that this makes closed-loop visual grasping work for novel objects in clutter. The value function predicts expected discounted cost-to-go from a segmented target point cloud and the gripper pose relative to the object centroid, trained by Bellman-error minimization with sparse costs that are zero only near a feasible grasp goal. At deployment, MPC minimizes the planner's default costs plus a discounted sum of this learned value over a 30-step horizon, so it steers toward states that predict grasp success rather than toward a

Load-bearing premise

The load-bearing assumption is that a trajectory should be labeled a successful grasp if motion planning reaches a grasp pose annotated as feasible and the gripper joints are within 5 mm of the goal—without ever checking in physics simulation whether the gripper actually lifts the object.

Editorial extensions

If this is right

  • Closed-loop grasping can recover from grasp-pose errors up to ±2 cm and ±18°, the regime that breaks open-loop methods; this directly targets the stated main failure mode of grasping in clutter.
  • Because MPC itself is the policy, no behavioral policy needs to be extracted from the offline data; the value function can be reused with any sampling-based MPC and any added constraint cost.
  • The value function transfers from synthetic empty scenes to real cluttered scenes without physics simulation in the training labels, so large-scale data generation via motion planning is sufficient for generalization.
  • Since labels are only success/failure, the value function can be fine-tuned with real-world demonstrations or with physics-simulated relabeling without changing the architecture or the MPC loop.
  • The approach is not limited to grasping: the same value-function-plus-MPC pattern applies to any manipulation task for which demonstrations can be generated by motion planning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: use the trained value function to score and select among candidate grasp poses before execution, rather than trusting the prediction model's ranking; the same value function could serve as a grasp-quality filter.
  • The phase analysis hints that grasps chosen by value-guided MPC survive a subsequent retrieve motion more often (99.8% vs 87.5% with predicted poses); if that holds, the value function implicitly encodes grasp stability and could be reused as a post-grasp transport-quality predictor.
  • Because the training labels never check physics, the reported numbers are likely a lower bound on what the same pipeline could achieve; relabeling the 2M trajectories with a quick physics grasp check is the cheapest high-value follow-up.
  • The moving-object experiment (60% success on 30 trials) is promising but small; scaling perturbations and using continuous object motion would test whether the value function degrades gracefully or fails abruptly under distribution shift.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Grasp-MPC, a closed-loop 6-DoF visual grasping framework that combines model predictive control with a learned value function. A value function is trained on a synthetic dataset of about 2.1M trajectories generated by CuRobo motion planning from perturbed pre-grasp poses to annotated grasp poses across 8,515 Objaverse objects, with sparse success labels based on whether the motion plan reaches a feasible grasp pose. At deployment, an off-the-shelf grasp predictor (M2T2) proposes target poses, a motion planner drives the robot to a pre-grasp pose, and MPPI in CuRobo minimizes the learned value cost together with collision and smoothness costs. The method is evaluated in FetchBench-style simulation (5,400 problems) and in three real-world tabletop/shelf scenes (30 trials per condition), reporting grasp success of 74.9% with ground-truth poses, 60.3% under pose noise, 67.2% with M2T2 poses, and 74.4% overall in the real world, outperforming open-loop, diffusion policy, transformer policy, and IQL baselines.

Significance. If the reported results are fully supported, the paper makes a valuable empirical contribution: it demonstrates that a value function trained from large-scale synthetic motion-planned trajectories can serve as an MPC task cost for closed-loop grasping in clutter, with real-world validation on novel objects. The scale of the dataset (2M trajectories, 115M states) and the use of both successful and failed trajectories are strengths, as are the modular integration with collision avoidance and the evaluation across multiple scene types. The central novelty, however, is the learned value cost, and the paper does not currently isolate this contribution from the closed-loop MPC mechanics. The simulation results are extensive, but the real-world claims rest on small samples without uncertainty quantification. The label definition used to train the value function is also a correctness risk that deserves direct scrutiny. The overall idea is promising and worth pursuing, but the evidence as presented is not yet conclusive for the strongest claims.

major comments (3)
  1. [§IV-C, Eq. (5)-(6); §V-B and §V-C] The central contribution is the learned value function as the grasp cost in MPC, yet there is no ablation that replaces V_theta with a geometric distance-to-grasp cost within the same MPPI/CuRobo framework. The comparisons in Section V are against OSC (open-loop linear execution), Transformer/DP policies, and IQL; none of these isolates the value function. The claim in Section I that geometric distance costs are sensitive to prediction errors and fail to exploit MPC's closed-loop capability is therefore never directly tested. It is plausible that closed-loop MPPI with collision avoidance and smoothness costs, tracking the same pre-grasp-to-grasp motion, would produce most of the reported gains over OSC, with the learned value contributing little. Please add an ablation that uses the same MPC and hyperparameters but replaces Eq. (5) with a cost such as ||T_ee - T_grasp||^2 (or a similarly
  2. [§IV-A, Eq. (1)] The training labels define a trajectory as successful when the motion plan reaches the annotated grasp pose and the pose is marked feasible by the GraspGen/ACRONYM annotation process, not when the grasp is physically validated. The text states explicitly: 'We do not validate these trajectories with simulation to accelerate data collection.' This means the value function may be learning motion-planning reachability to an annotated pose rather than physical grasp success (e.g., contact stability, slip, gripper-object interference). The real-world results mitigate this concern for the evaluated scenes, but they do not disambiguate whether the value function's guidance is superior because it reaches physically good poses or because it simply tracks planned reachable poses. For a stronger claim, please provide evidence on the correlation between the motion-planning-based labels and physical g
  3. [§VI-B and Fig. 8] The real-world experiments use only 30 trials per scene condition (5 objects x 3 poses x 2 trials per method) and report point estimates without confidence intervals or significance tests. For example, the empty-table condition is 50.0% vs 86.7%, the cluttered-table 43.3% vs 67.7%, and the shelf 30.0% vs 70.0%; with n=30 per condition, these differences have very wide confidence intervals. The '33.3% improvement' headline in the abstract is therefore not statistically well grounded. Please report per-trial results, confidence intervals (e.g., Wilson intervals), and, where possible, a significance test or effect-size measure. This is load-bearing for the real-world claims, which are a key part of the paper's contribution.
minor comments (5)
  1. [§V-B] The text says 'Grasp-MPC (73.6%)' when comparing with IQL, but Figure 5 and the subsequent text report 74.9% for the ground-truth condition. One of these is a typo and should be corrected.
  2. [§IV-A] The translation noise is written as 'U(−0.04cm,0.04cm)'. If this is intended to be meters or centimeters, the notation is inconsistent with the surrounding numerical values (0.04 m would be 4 cm; 0.04 cm is 0.4 mm). Please clarify the units.
  3. [Figure 5 caption] The caption's numeric list is not visually keyed to the bars in a way that is easy to parse. Please add value labels directly above each bar or restructure the caption by group (ground-truth, noisy, M2T2) for readability.
  4. [Appendix D] There are typos in the appendix text: 'retreive pose' and 'consistenly' should be 'retrieve' and 'consistently'. Also, reference [19] and [44] both cite the same MPPI paper, and [49] and [55] both cite Grounding DINO; duplicate references should be consolidated.
  5. [§V-A] The evaluation metric excludes trials where motion planning to the pre-grasp pose failed, and Appendix D shows this reachability varies by method (e.g., ~84-87%). The paper should state more prominently that the headline success rates are conditional on successful pre-grasp reaching, since the absolute task-success rate is lower for all methods.

Circularity Check

0 steps flagged · score 1.0 of 10

No meaningful circularity: the learned value function is evaluated against physical grasp success on novel objects, not against its own training labels; self-citations to CuRobo/GraspGen/M2T2 are infrastructure, not the result.

full rationale

The paper's central claim is that a value function trained on 2M synthetic grasp trajectories can serve as an MPC cost for closed-loop grasping. The training label (Eq. 1) is explicitly a proxy: a trajectory is successful if motion planning reaches the goal joint configuration of a GraspGen-annotated feasible grasp pose, and the authors state 'We do not validate these trajectories with simulation to accelerate data collection' (Sec. IV-A). The evaluation, in contrast, measures physical grasp success by lifting an object at least 1 cm in Isaac Sim, or by lifting and returning to home in the real world. Thus the reported success rates are not determined by the training label by construction; the value function must generalize from reachability/feasibility labels to actual physical grasping on novel FetchBench/ACRONYM objects and real objects. The paper does rely on tools co-authored by the authors — CuRobo for motion planning/MPC, GraspGen for grasp annotations, and M2T2 for predicted grasp poses — but these are used as infrastructure and external baselines; the test grasp annotations and physical outcomes are not fitted to the value function. No uniqueness theorem or ansatz is imported from self-citations to rule out alternatives. The most serious concern is experimental, not circular: the paper never ablates the learned value function against a geometric distance-to-grasp cost within the same CuRobo/MPPI framework, so the marginal contribution of the learned value component is not isolated. That omission undermines a strong attribution claim but does not make the derivation circular. The paper also openly acknowledges the proxy-label limitation and suggests physics-validated labels as future work, which further supports that the result is not being presented as a tautology.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the proxy success label for training data, which is not physics-validated, and on the transfer of a value function from empty scenes to clutter. No physically new entities are introduced; the learned value function is a standard neural model.

free parameters (6)
  • cost weight omega = 1000
    Weights the learned value cost against CuRobo default costs in Eq. 6; no sensitivity analysis provided.
  • pre-grasp offset = 15cm data, 10cm real, 6cm sim
    Distance along approach vector to define pre-grasp poses; affects the state distribution and task difficulty.
  • success threshold epsilon = 5e-3
    Joint-position tolerance in Eq. 1 defines terminal success states for the value function.
  • discount factor gamma = 0.99
    Discount in Bellman target; standard but chosen without ablation.
  • training noise ranges = U(-0.04,0.04) m and U(-0.04pi,0.04pi) rad
    Added to pre-grasp poses in data generation to increase coverage.
  • MPC horizon and particles = horizon=30, num_particles=400
    MPPI hyperparameters from Table I; chosen for real-time performance.
assumptions (5)
  • domain assumption The robot can track joint position/velocity/acceleration targets accurately with an inverse dynamics controller
    Stated in Section III as an assumption for the MPC dynamics model.
  • domain assumption Motion-planning success to a feasible grasp pose is a valid proxy for physical grasp success
    Section IV-A: 'We do not validate these trajectories with simulation'; Eq. 1 labels trajectories as successful solely based on reaching the goal joint configuration.
  • domain assumption The value function trained on empty scenes transfers to cluttered scenes
    The training environment is empty (Section IV-A), but deployment includes shelves, baskets, and clutter; the paper relies on this transfer.
  • domain assumption Point cloud segmentation (SAM-Track) provides clean target object point clouds in real world
    Section VI-A describes the perception system; the value function's generalization depends on accurate segmentation.
  • domain assumption The test objects, while novel, are from a distribution similar to the Objaverse training set
    Training uses Objaverse; evaluation uses FetchBench/ACRONYM objects, which may share a distribution but this is not guaranteed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control." pith.science (2026). https://pith.science/paper/BSWNSIIM

@misc{pith2026250906201,
  author       = {Pith},
  title        = {Pith review of: Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BSWNSIIM}},
  note         = {Machine review of arXiv:2509.06201}
}
read the original abstract

Grasping of diverse objects in unstructured environments remains a significant challenge. Open-loop grasping methods, effective in controlled settings, struggle in cluttered environments. Grasp prediction errors and object pose changes during grasping are the main causes of failure. In contrast, closed-loop methods address these challenges in simplified settings (e.g., single object on a table) on a limited set of objects, with no path to generalization. We propose Grasp-MPC, a closed-loop 6-DoF vision-based grasping policy designed for robust and reactive grasping of novel objects in cluttered environments. Grasp-MPC incorporates a value function, trained on visual observations from a large-scale synthetic dataset of 2 million grasp trajectories that include successful and failed attempts. We deploy this learned value function in an MPC framework in combination with other cost terms that encourage collision avoidance and smooth execution. We evaluate Grasp-MPC on FetchBench and real-world settings across diverse environments. Grasp-MPC improves grasp success rates by up to 32.6% in simulation and 33.3% in real-world noisy conditions, outperforming open-loop, diffusion policy, transformer policy, and IQL approaches. Videos and more at http://grasp-mpc.github.io.

Figures

Figures reproduced from arXiv: 2509.06201 by the authors.

Figure 1
Figure 1. Grasp-MPC overview. A large-scale synthetic grasp trajectory dataset is generated in simulation using a motion planner, collecting only trajectories between pre-grasp and ground-truth grasp poses across 8K Objaverse objects. A value function is trained using a sparse cost label given the target object’s point cloud and the end-effector pose. The learned value function is used in an MPC framework, enabling robust and… view at source ↗
Figure 2
Figure 2. Grasp-MPC Pipeline. Grasp-MPC seamlessly integrates off-the-shelf grasp prediction and motion planning with an MPC that incorporates a learned grasp value function, enabling grasping in cluttered scenes. The pipeline involves: (1) predicting grasp and pre-grasp poses using a fixed offset and filtering out in-collision poses via IK; (2) planning a trajectory to a collision￾free pre-grasp pose; and (3) executing actio… view at source ↗
Figure 3
Figure 3. Visualization of a learned value function. (a) Costs [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Simulated environments. Grasp-MPC is extensively evaluated in FetchBench [14] environments. noise. The value function is trained with a mini-batch size of 1536, where 32 distinct object point clouds are sampled and 48 states are sampled for each object point cloud. The…
Figure 5
Figure 5. Figure 5: Grasp performance comparison across different methods given ground truth, noisy, and M2T2-predicted grasp poses. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Grasp-MPC execution in the Table Clutter scene. Grasp-MPC effectively grasps a novel object from the bin [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Representative real-world environments: (Left) Table [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 9
Figure 9. Figure 9: Grasp-MPC execution for moving objects. Grasp-MPC adapts in real time to track and grasp moving target objects, capabilities that open-loop approaches lack. object, resulting in 30 trials total [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Grasp success rate for each scene type given grasp [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Ablation on Ensemble showing Grasp Success and [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Grasp Success across different λ values. Cgrasp(xh∈H, a¨h∈H) = log( X K 1 exp( 1 λ Gi(xh∈H, a¨h∈H))) (7) Impact of Value Function Ensembles [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Split of fetchbench task into phases with successful trials at each phase. Each bar represents the number of successful [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: Grasp-MPC trains a value function to serve as a cost function within MPC. The PointNet++ encoder takes a segmented point cloud as input, and the state encoder takes an end-effector pose with respect to the mean of the object point cloud T EE obj as input. Features fro…
Figure 15
Figure 15. Figure 15: Representative scenes in FetchBench [14] for grasping in clutter. We replace Isaac Gym [54] used as the underlying [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 16
Figure 16. Figure 16: Example collision voxel visualization using NVBlox. [PITH_FULL_IMAGE:figures/full_fig_p013_16.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping

    cs.RO 2026-05 unverdicted novelty 6.0 of 10

    GraspGen-X extends diffusion 6-DOF grasping to cross-embodiment via swept-volume gripper encoding, trained on procedural grippers and 2B grasps, claiming best zero-shot generalization to novel grippers in sim and real tests.

Reference graph

Works this paper leans on

55 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [1]

    M2t2: Multi-task masked transformer for object-centric pick and place,

    W. Yuan, A. Murali, A. Mousavian, and D. Fox, “M2t2: Multi-task masked transformer for object-centric pick and place,” inConference on Robot Learning, 2023

    work page 2023

  2. [2]

    6-dof graspnet: Variational grasp generation for object manipulation,

    A. Mousavian, C. Eppner, and D. Fox, “6-dof graspnet: Variational grasp generation for object manipulation,” inIEEE International Conference on Computer Vision, 2019, pp. 2901–2910

    work page 2019

  3. [3]

    Contact- graspnet: Efficient 6-dof grasp generation in cluttered scenes,

    M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox, “Contact- graspnet: Efficient 6-dof grasp generation in cluttered scenes,” inIEEE International Conference on Robotics and Automation. IEEE, 2021, pp. 13 438–13 444

    work page 2021

  4. [4]

    Grasp Diffusion Network: Learning Grasp Generators from Partial Point Clouds with Diffusion Models in SO(3)xR3

    J. Carvalho, A. T. Le, P. Jahr, Q. Sun, J. Urain, D. Koert, and J. Peters, “Grasp diffusion network: Learning grasp generators from partial point clouds with diffusion models in so(3)xr3,”arXiv preprint arXiv:2412.08398, 2024

    work page Pith review arXiv 2024

  5. [5]

    Scalable deep reinforcement learning for vision-based robotic manipulation,

    D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V . Vanhouckeet al., “Scalable deep reinforcement learning for vision-based robotic manipulation,” in Conference on Robot Learning. PMLR, 2018, pp. 651–673

    work page 2018

  6. [6]

    Goal-auxiliary actor-critic for 6d robotic grasping with point clouds,

    L. Wang, Y . Xiang, W. Yang, A. Mousavian, and D. Fox, “Goal-auxiliary actor-critic for 6d robotic grasping with point clouds,” inConference on Robot Learning. PMLR, 2022, pp. 70–80

    work page 2022

  7. [7]

    Dextrah- rgb: Visuomotor policies to grasp anything with dexterous hands,

    R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk, “Dextrah- rgb: Visuomotor policies to grasp anything with dexterous hands,”arXiv preprint arXiv:2412.01791, 2024

    arXiv 2024

  8. [8]

    DextrAH-g: Pixels-to-action dexterous arm-hand grasping with geometric fabrics,

    T. G. W. Lum, M. Matak, V . Makoviychuk, A. Handa, A. Allshire, T. Hermans, N. D. Ratliff, and K. V . Wyk, “DextrAH-g: Pixels-to-action dexterous arm-hand grasping with geometric fabrics,” inConference on Robot Learning, 2024

    work page 2024

Show all 55 references
  1. [9]

    Objaverse-xl: A universe of 10m+ 3d objects,

    M. Deitke, R. Liu, M. Wallingford, H. Ngo, O. Michel, A. Kusupati, A. Fan, C. Laforte, V . V oleti, S. Y . Gadreet al., “Objaverse-xl: A universe of 10m+ 3d objects,”Neural Information Processing Systems, vol. 36, 2024

    2024

  2. [10]

    Graspldm: Generative 6-dof grasp synthesis using latent diffusion models,

    K. R. Barad, A. Orsula, A. Richard, J. Dentler, M. Olivares-Mendez, and C. Martinez, “Graspldm: Generative 6-dof grasp synthesis using latent diffusion models,”IEEE Access, 2024

    2024

  3. [11]

    Graspnet-1billion: A large- scale benchmark for general object grasping,

    H.-S. Fang, C. Wang, M. Gou, and C. Lu, “Graspnet-1billion: A large- scale benchmark for general object grasping,” inIEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 441–11 450

    2020

  4. [12]

    Acronym: A large-scale grasp dataset based on simulation,

    C. Eppner, A. Mousavian, and D. Fox, “Acronym: A large-scale grasp dataset based on simulation,” inIEEE International Conference on Robotics and Automation. IEEE, 2021, pp. 6222–6227

    2021

  5. [13]

    Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations,

    S. Song, A. Zeng, J. Lee, and T. A. Funkhouser, “Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations,” IEEE Robotics and Automation Letters, vol. 5, pp. 4978–4985, 2019

    2019

  6. [14]

    Fetchbench: A simulation benchmark for robot fetching,

    B. Han, M. Parakh, D. Geng, J. A. Defay, G. Luyang, and J. Deng, “Fetchbench: A simulation benchmark for robot fetching,” inConference on Robot Learning. PMLR, 2025, pp. 3053–3071

    2025

  7. [15]

    Motion planner augmented reinforcement learn- ing for robot manipulation in obstructed environments,

    J. Yamada, Y . Lee, G. Salhotra, K. Pertsch, M. Pflueger, G. Sukhatme, J. Lim, and P. Englert, “Motion planner augmented reinforcement learn- ing for robot manipulation in obstructed environments,” inConference on Robot Learning. PMLR, 2021, pp. 589–603

    2021

  8. [16]

    Efficient skill acquisition for insertion tasks in obstructed environments,

    J. Yamada, J. Collins, and I. Posner, “Efficient skill acquisition for insertion tasks in obstructed environments,” inProceedings of the 6th Annual Learning for Dynamics and Control Conference, vol. 242. PMLR, 2024, pp. 615–627

    2024

  9. [17]

    Imitating task and motion planning with visuomotor transformers,

    M. Dalal, A. Mandlekar, C. Garrett, A. Handa, R. Salakhutdinov, and D. Fox, “Imitating task and motion planning with visuomotor transformers,” 2023

    2023

  10. [18]

    Autonomous helicopter aerobatics through apprenticeship learning,

    P. Abbeel, A. Coates, and A. Ng, “Autonomous helicopter aerobatics through apprenticeship learning,”The International Journal of Robotics Research, vol. 29, pp. 1608 – 1639, 2010

    2010

  11. [20]

    Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,

    J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim, “Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,” inIEEE/RSJ International Conference on Intelligent Robots and Systems, 2018, pp. 1–9

    2018

  12. [21]

    Neural motion fields: Encoding grasp trajectories as implicit value functions,

    Y .-C. Chen, A. Murali, B. Sundaralingam, W. Yang, A. Garg, and D. Fox, “Neural motion fields: Encoding grasp trajectories as implicit value functions,”arXiv preprint arXiv:2206.14854, 2022

    2022 arXiv

  13. [22]

    Deepmpc: Learning deep latent features for model predictive control,

    I. Lenz, R. A. Knepper, and A. Saxena, “Deepmpc: Learning deep latent features for model predictive control,” inRobotics: Science and Systems, 2015

    2015

  14. [23]

    Deep visual foresight for planning robot motion,

    C. Finn and S. Levine, “Deep visual foresight for planning robot motion,” inIEEE International Conference on Robotics and Automation, 2017, pp. 2786–2793

    2017

  15. [24]

    From pixels to torques: Policy learning with deep dynamical models,

    N. Wahlstr ¨om, T. B. Sch ¨on, and M. P. Deisenroth, “From pixels to torques: Policy learning with deep dynamical models,”arXiv preprint arXiv:1502.02251, 2015

    2015 arXiv

  16. [25]

    Embed to control: A locally linear latent dynamics model for control from raw images,

    M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller, “Embed to control: A locally linear latent dynamics model for control from raw images,”Neural Information Processing Systems, vol. 28, 2015

    2015

  17. [26]

    Visual foresight: Model-based deep reinforcement learning for vision-based robotic control,

    F. Ebert, C. Finn, S. Dasari, A. Xie, A. Lee, and S. Levine, “Visual foresight: Model-based deep reinforcement learning for vision-based robotic control,”arXiv preprint arXiv:1812.00568, 2018

    2018 arXiv

  18. [27]

    Learning latent dynamics for planning from pixels,

    D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning latent dynamics for planning from pixels,” in International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. ...

    2019

  19. [28]

    Temporal difference learning for model predictive control,

    N. A. Hansen, H. Su, and X. Wang, “Temporal difference learning for model predictive control,” inInternational Conference on Machine Learning. PMLR, 2022, pp. 8387–8406

    2022

  20. [29]

    Dy- namic non-prehensile object transport via model-predictive reinforce- ment learning,

    N. Jawale, B. Boots, B. Sundaralingam, and M. Bhardwaj, “Dy- namic non-prehensile object transport via model-predictive reinforce- ment learning,” 2024

    2024

  21. [30]

    Value function approximation and model predictive control,

    M. Zhong, M. Johnson, Y . Tassa, T. Erez, and E. Todorov, “Value function approximation and model predictive control,” in2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, 2013, pp. 100–107

    2013

  22. [31]

    Plan online, learn offline: Efficient learning and exploration via model- based control,

    K. Lowrey, A. Rajeswaran, S. Kakade, E. Todorov, and I. Mordatch, “Plan online, learn offline: Efficient learning and exploration via model- based control,”arXiv preprint arXiv:1811.01848, 2018

    2018 arXiv

  23. [32]

    Learning sampling distributions for model predictive control,

    J. Sacks and B. Boots, “Learning sampling distributions for model predictive control,” inConference on Robot Learning. PMLR, 2023, pp. 1733–1742

    2023

  24. [33]

    D-cubed: Latent dif- fusion trajectory optimisation for dexterous deformable manipulation,

    J. Yamada, S. Zhong, J. Collins, and I. Posner, “D-cubed: Latent dif- fusion trajectory optimisation for dexterous deformable manipulation,” 2024

    2024

  25. [34]

    Advantage-weighted regression: Simple and scalable off-policy reinforcement learning,

    X. B. Peng, A. Kumar, G. Zhang, and S. Levine, “Advantage-weighted regression: Simple and scalable off-policy reinforcement learning,”arXiv preprint arXiv:1910.00177, 2019

    1910 arXiv

  26. [35]

    Offline reinforcement learning with implicit q-learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit q-learning,”arXiv preprint arXiv:2110.06169, 2021

    2021 arXiv

  27. [36]

    Conservative q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,”Neural Information Processing Sys- tems, vol. 33, pp. 1179–1191, 2020

    2020

  28. [37]

    Is value learning really the main bottleneck in offline rl?

    S. Park, K. Frans, S. Levine, and A. Kumar, “Is value learning really the main bottleneck in offline rl?”arXiv preprint arXiv:2406.09329, 2024

    2024 arXiv

  29. [38]

    Storm: An integrated framework for fast joint- space model-predictive control for reactive manipulation,

    M. Bhardwaj, B. Sundaralingam, A. Mousavian, N. D. Ratliff, D. Fox, F. Ramos, and B. Boots, “Storm: An integrated framework for fast joint- space model-predictive control for reactive manipulation,” inConference on Robot Learning. PMLR, 2022, pp. 750–759

    2022

  30. [39]

    Graspgen: A diffusion-based framework for 6-dof grasping with on-generator training,

    A. Murali, B. Sundaralingam, Y .-W. Chao, J. Yamada, W. Yuan, M. Carlson, F. Ramos, S. Birchfield, D. Fox, and C. Eppner, “Graspgen: A diffusion-based framework for 6-dof grasping with on-generator training,”arXiv preprint arXiv:2507.13097, 2025

    2025 arXiv

  31. [40]

    Curobo: Parallelized collision-free minimum-jerk robot motion gen- eration,

    B. Sundaralingam, S. K. S. Hari, A. Fishman, C. Garrett, K. Van Wyk, V . Blukis, A. Millane, H. Oleynikova, A. Handa, F. Ramoset al., “Curobo: Parallelized collision-free minimum-jerk robot motion gen- eration,”arXiv preprint arXiv:2310.17274, 2023

    2023 arXiv

  32. [41]

    R. S. Sutton and A. G. Barto,Reinforcement Learning: An Introduction. The MIT Press, 2018. [Online]. Available: http: //incompleteideas.net/book/the-book-2nd.html

    2018

  33. [42]

    Adversarially trained actor critic for offline reinforcement learning,

    C.-A. Cheng, T. Xie, N. Jiang, and A. Agarwal, “Adversarially trained actor critic for offline reinforcement learning,” inProceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. ...

    2022

  34. [43]

    Anygrasp: Robust and efficient grasp perception in spatial and temporal domains,

    H.-S. Fang, C. Wang, H. Fang, M. Gou, J. Liu, H. Yan, W. Liu, Y . Xie, and C. Lu, “Anygrasp: Robust and efficient grasp perception in spatial and temporal domains,”IEEE Transactions on Robotics (T-RO), 2023

    2023

  35. [44]

    Aggressive driving with model predictive path integral control,

    G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” inIEEE International Conference on Robotics and Automation, 2016, pp. 1433– 1440

    2016

  36. [45]

    Pointnet++: Deep hierar- chical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierar- chical feature learning on point sets in a metric space,”arXiv preprint arXiv:1706.02413, 2017

    2017 arXiv

  37. [46]

    A unified approach for motion and force control of robot manipulators: The operational space formulation,

    O. Khatib, “A unified approach for motion and force control of robot manipulators: The operational space formulation,”IEEE Journal on Robotics and Automation, vol. 3, no. 1, pp. 43–53, 1987

    1987

  38. [47]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”The International Journal of Robotics Research, 2023

    2023

  39. [48]

    Segment and track anything,

    Y . Cheng, L. Li, Y . Xu, X. Li, Z. Yang, W. Wang, and Y . Yang, “Segment and track anything,”arXiv preprint arXiv:2305.06558, 2023

    2023 arXiv

  40. [49]

    Grounding dino: Marrying dino with grounded pre- training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhuet al., “Grounding dino: Marrying dino with grounded pre- training for open-set object detection,”arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  41. [50]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll´ar, and R. Girshick, “Segment anything,”arXiv preprint arXiv:2304.02643, 2023

    2023 arXiv

  42. [51]

    nvblox: Gpu-accelerated incremental signed distance field mapping,

    A. Millane, H. Oleynikova, E. Wirbel, R. Steiner, V . Ramasamy, D. Tingdahl, and R. Siegwart, “nvblox: Gpu-accelerated incremental signed distance field mapping,”arXiv preprint arXiv:2311.00626, 2024

    2024 arXiv

  43. [52]

    Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,

    C. Garrett, A. Mandlekar, B. Wen, and D. Fox, “Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,” inConference on Robot Learning, 2024

    2024

  44. [53]

    On representa- tion of 3d rotation in the context of deep learning,

    V . Pravdov ´a, L. Gajdo ˇsech, H. Ali, and V . Kocur, “On representa- tion of 3d rotation in the context of deep learning,”arXiv preprint arXiv:2410.10350, 2024

    2024 arXiv

  45. [54]

    Isaac gym: High performance gpu-based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handaet al., “Isaac gym: High performance gpu-based physics simulation for robot learning,”arXiv preprint arXiv:2108.10470, 2021

    2021 arXiv

  46. [55]

    Dino: Detr with improved denoising anchor boxes for end-to- end object detection,

    H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. M. Ni, and H.-Y . Shum, “Dino: Detr with improved denoising anchor boxes for end-to- end object detection,”arXiv preprint arXiv:2203.03605, 2022. APPENDIX A. Full point cloud vs partial point cloud observations. Grasp-MPCrob...

    2022 arXiv

  47. [256]

    After the set abstraction layers, the output passes through three fully connected layers with sizes 512,256, and128

    The third abstraction layer skips FPS, instead grouping all points together, and employs a local PointNet with layer sizes of256,256, and512. After the set abstraction layers, the output passes through three fully connected layers with sizes 512,256, and128. Between these laye...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.