Pith. sign in

REVIEW 3 major objections 4 minor 70 references

Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidance

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Contact guidance lifts robot imitation by 20.8 percent on average across six manipulation tasks.

desk verdict Coherent contact-guided hierarchical diffusion policy with a genuinely useful prompt-guidance capability, but the 20.8% headline is a best-of-variants artifact and the main Q-learning variant loses on Square; worth reviewing after the claims are recalibrated. read the letter →

arxiv 2411.12982 v1 pith:VHO62OBJ submitted 2024-11-20 cs.RO

Dexin Wang , Chunsheng Liu , Faliang Chang , Yichen Xu This is my paper · ORCID
classification cs.RO
keywords hierarchicaldiffusionpolicycontactguidanceimitationlearningrobotmanipulationQ-learningprompt3Dpointcloudconditioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hierarchical Diffusion Policy (HDP) claims that end-to-end diffusion policies underperform on contact-rich manipulation because they never explicitly model where the robot should touch the object. The paper splits the problem into two conditional diffusion processes: a high-level Guider predicts the next objective contact from observations including 3D point clouds, and a low-level Actor generates an action sequence toward that contact, trained by behavior cloning plus a Q-learning term that rewards reaching the contact. Across six simulated and real tasks, the authors report an average 20.8% improvement over Diffusion Policy, with the largest gains on contact-rich and high-precision tasks, plus new capabilities: interpretable subgoals, human prompt guidance, and a faster snapshot-gradient training scheme. If correct, the paper establishes that contact-level subgoals, rather than end poses alone, are a useful decomposition for manipulation imitation learning.

What carries the argument

The central object is the objective contact $C_t$: the expected contact position between the end-effector and the object at time $t$, or the finger position when already in contact, used as the high-level subgoal. Three networks carry the argument: the Guider, a conditional denoising diffusion process predicting $C_t$; the Actor, a conditional denoising diffusion process predicting the action sequence toward $C_t$; and the Critic, a Q-function whose reward is positive when the robot reaches the objective contact within $T_r$ steps. The phased-objective-contacts algorithm decides which contacts in the demonstrations are real subgoals versus misoperations, and the snapshot-gradient optimization strategy computes the Q-learning gradient at only one denoising timestep per iteration via oneshot denoising (Eq. 11), which the paper reports gives a 4x training speedup and a 24% memory reduction. A modified PointNet 3D encoder, stripped of normalization and input transformations, supplies pose- and size-sensitive object observations. These components together carry the decomposition claim that contact planning and contact-conditioned trajectory generation are learnable as separate diffusion processes.

What would settle it

Take a demonstration set for one task that includes two distinct successful contact strategies whose intermediate object poses are within the thresholds $\tau_l$ and $\tau_r$ but whose later outcomes differ. If Algorithm 1 merges these phases and erases one contact, and HDP trained on the merged labels performs worse than HDP trained with both contacts kept, the pose-similarity assumption is falsified.

Watch

Extended reading notes

Core claim

HDP establishes that a manipulation policy can be substantially improved by making the predicted contact between robot and object an explicit intermediate variable. The high-level Guider models the multimodal distribution of objective contacts $p(C_t|O_t)$ as a conditional denoising diffusion process conditioned on observations; the low-level Actor models $p(A_t|O_t,C_t)$ and is optimized to minimize diffusion noise prediction error while maximizing the Critic's Q-value for reaching the contact within $T_r$ steps (Eq. 9). A labeling algorithm (Algorithms 1-4) turns raw demonstrations into phased objective contacts and sparse rewards by erasing contacts that do not change object pose, which the authors argue removes misoperations. The authors report that this contact-guided hierarchy outperforms end-to-end Diffusion Policy on all six benchmarks, with an average improvement of 20.8%, and that the resulting trajectories are more interpretable and can be steered in real time by human-specified prompts.

Load-bearing premise

The labeling algorithm assumes that any contact phase whose ending object pose closely matches the neighboring phase is a misoperation that can be erased; if similar poses can arise from different legitimate contact strategies, the Guider learns to discard valid contacts and the Critic's rewards are built on wrong labels.

Editorial extensions

If this is right

  • Contact-rich tasks become more learnable: on the Tilt task, contact guidance adds an average 7.9% over the same policy without objective contacts, while the end-to-end baseline saturates at a lower success rate.
  • Human operators can steer a deployed policy in real time by specifying a prompt contact without retraining; in the Move-T experiment, prompting raised stage-2 success from 29% to 71%.
  • Decomposing multimodal action distributions into contact planning and contact-conditioned trajectory generation lets a single policy express multiple valid strategies, such as pulling left or pushing right, instead of collapsing onto one mode.
  • Fewer demonstrations are needed: the gains of HDP over Diffusion Policy are largest in low-sample regimes, and the snapshot-gradient oneshot denoising is more robust when training data is scarce.
  • The same contact-as-subgoal representation handles rigid and deformable objects and different end-effectors, since contacts generalize across gripper geometry better than end poses do.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the contact-as-subgoal formulation suggests a natural path to multi-finger dexterous manipulation, where a single end pose is insufficient but multiple contact points can be predicted jointly; the authors mention this as future work.
  • Beyond the paper: because the reward and labels are defined by object-pose similarity, the method's success on deformable objects may rely on hand-recorded immediate contacts; a testable extension would be a learned contact-quality model that scores candidate contacts rather than thresholding pose similarity.
  • Beyond the paper: prompt guidance opens a practical interface for correcting policies online; one could evaluate whether sparse human interventions during deployment improve long-horizon success rates beyond the three-prompt limit tested here.
  • Beyond the paper: the 20.8% average is across six self-selected tasks; the strongest evidence for the decomposition claim would be a benchmark comparison on contact-rich tasks where the ground-truth misoperations are known, so the labeling assumption can be checked independently.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Hierarchical Diffusion Policy (HDP), a two-level imitation learning method for contact-rich manipulation. A high-level Guider network, modeled as a conditional diffusion process, predicts objective contacts from observations including 3D point clouds; a low-level Actor network, also a conditional diffusion process, generates action sequences conditioned on the predicted contact. A Critic network provides a Q-value used to augment the behavior-cloning loss with a Q-learning term. To construct training contacts, the paper introduces a phased-objective-contact algorithm that deletes contacts associated with object-pose-similar phases, which it identifies as misoperations. Additional contributions include a modified PointNet 3D encoder, snapshot gradient optimization, and prompt guidance for human intervention. The paper reports simulation results on Can, Square, Tilt, and Push-T and real-world results on Move-T and Cloth Unfold, claiming an average improvement of 20.8% over Diffusion Policy.

Significance. If the central claim were supported, the paper would make a useful contribution: decomposing manipulation into contact prediction and contact-conditioned trajectory generation, with a Q-value bonus, is a plausible route to improving imitation learning in contact-rich regimes, and the prompt-guidance mechanism is a genuinely attractive controllability feature. The paper also provides unusually detailed empirical documentation: multiple training-sample counts, three seeds, 50 initial conditions, per-variant ablations, and real-world tests on both rigid and deformable objects. The code and data are promised publicly. However, the headline comparison is not tied to any single configuration, and the paper's own Section VI-C concedes a task where the proposed contact guidance hurts performance. These issues are fixable in revision but the claims as written overstate what the evidence shows.

major comments (3)
  1. [Abstract, Tables II and III, Section VI-C] The headline "average improvement of 20.8%" is not attributable to any fixed HDP configuration. The rows labeled "HDP Improvement" in Tables II and III are per-cell maxima over three variants, HDP(No-OC), HDP(eta=0), and HDP(eta=0.001). For example, on Square-ph(50), the eta=0.001 variant scores 0.74/0.60, below the Diffusion Policy baseline of 0.78/0.62; the reported 19.4% improvement comes from the No-OC average of 0.74/0.62, not from the proposed contact-guided Q-learning variant. Similarly, the 64.3% improvement on Square-mh(50) comes from No-OC, not from the Q-learning variant. The manuscript also contains an internal inconsistency: the abstract and introduction state 20.8%, while Section VI-C reports an average improvement of 17.0%. The central claim must be reported per configuration and per task, with the averaging rule stated explicitly, and the abstract must not present the best of three variants as a single method's improvement.
  2. [Table II note and Section VI-B] The comparison target is not the published Diffusion Policy used in the abstract's claim. The note under Table II states: "We reduce the size of the Diffusion Policy network and use the same backbone network in HDP to accelerate training, resulting in slightly lower performance compared to the results in [9]." The abstract nevertheless says HDP "significantly outperforms the existing state-of-the-art imitation learning method Diffusion Policy." A comparison against a deliberately weakened reimplementation cannot support a superiority claim over the state of the art unless the published numbers from [9] are also reported and the discrepancy is discussed. At minimum, the claim should be scoped to "Diffusion Policy with the reduced backbone used here."
  3. [Section VI-C, Algorithm 1, Algorithm 3, Eq. (14)] The paper's own Section VI-C undermines the load-bearing assumption behind phased objective contacts. It states that for precise tasks such as Square, "many inadvertent touches that change the object's position which cannot be removed by Algorithm 1, causing the Guider to learn objective contacts that reproduce these mistakes," and that contact guidance gives an average decrease of 2.1% on Square when comparing HDP(eta=0.001) to No-OC. Since Eq. (14)'s pose-similarity criterion is what decides which contacts are considered misoperations and are erased from the Guider's targets and the Critic's rewards, this admission means the central preprocessing step can fail on a benchmark included in the headline average. The paper should report results with and without the phased-contact construction on all tasks, discuss when the similarity assumption is invalid, and avoid the universal claim that contact guidance improves performance.
minor comments (4)
  1. [Section VI-C] The sentence "Our results show that Diffusion Policy significantly improves state-of-the-art performance across the board" appears to mean "HDP"; as written it contradicts the paper's own comparison. The term "significantly" is also used without statistical tests, so it should be replaced with a quantitative statement.
  2. [Appendix A, Cloth Unfold] The appendix states that for the cloth task "Algorithm 2 struggles to model the object and accurately compute objective contacts," so the authors fall back to immediate contacts. Because the real-world Cloth Unfold result is presented as evidence that HDP handles deformable objects, the main text should state clearly that the proposed phased-contact mechanism was not used for that task and that the reported result is for a degraded version of the method.
  3. [Abstract] The GitHub URL contains a space, "https://github.com/dexin-wang/Hierarchical Diffusion Policy"; it should be a single hyphenated or percent-encoded URL.
  4. [Section IV-C, Eq. (15)] The noise augmentation in Eq. (15) has four hyperparameters (mu1, mu2, tau_n1, tau_n2) that are reported only in the appendix tables; a sentence explaining the role of each would improve reproducibility, since the appendix table packs them into a single row.

Circularity Check

1 steps flagged · score 2.0 of 10

No meaningful circularity: the core derivation is self-contained and benchmarked on external tasks; the only genuine self-reference is the Tilt task, whose demonstrations and label thresholds are inherited from the authors' prior MRLNM work.

  1. self citation load bearing [Sec. VI-A2 (Tilt), Sec. VI-C, Appendix A (hyperparameters), Tables II/III]
    "Tilt: adapted from [30], requires pushing a cube that is too large to grasp from above to a distant wall, tilting it along the wall, and grasping it from the side. ... The demonstrations are collected using the reinforcement learning method MRLNM [30] trained on this task. ... tau_l and tau_r are set according to the MRLNM [30] to align with the average magnitude of the action."

    The largest contact-guidance gains (Tilt: 51.0%, 70.8%, 39.7% in Table III) are measured on a task whose demonstrations are produced by the authors' own RL policy MRLNM [30], and the contact-label thresholds tau_l and tau_r used in Eq. 14 and Algorithm 4 are also taken from [30]. The 'objective contacts' are therefore not independently sourced labels but a re-encoding of the behavior of the authors' own prior policy with its own thresholds; the margin over Diffusion Policy on this task partly measures how well HDP re-fits that self-generated demonstration policy rather than an external ground truth. Since Sec.

full rationale

The core derivation chain (Guider Eqs. 5-6, Actor Eqs. 7-11, Critic Eqs. 12-13, contact-label Algorithms 1-4) is self-contained: contact targets are generated from demonstrations by explicit pose-similarity rules, and the policy is trained against those targets. The reported success rates are evaluated on held-out initial conditions and on external or fixed datasets (Robomimic Can/Square, Push-T from IBC, plus the authors' real Move-T and Cloth tasks), so there is no by-construction equivalence between the fitted contact labels and the evaluation metric. The Q-learning bonus is a reweighted imitation signal derived from the same demonstrations, but that is the paper's stated design rather than a disguised prediction. The weaknesses identified elsewhere in the manuscript, such as the 'HDP Improvement' rows being cell-wise maxima over variants, the reduced-size Diffusion Policy baseline, and cases where Algorithm 1 fails (Square inadvertent touches, cloth fallback to immediate contacts), are correctness and reporting concerns rather than circularity. The single genuine self-reference is the Tilt task, whose demonstrations and thresholds come from the authors' prior MRLNM [30], which warrants the low score of 2.

Assumptions & free parameters 5 free parameters · 6 assumptions · 1 invented entities

The central claim rests on three families of inputs the reader does not pay for upstream: (1) hand-set thresholds (tau_l, tau_r, tau_f) and coefficients (eta, noise scales) that shape the contact labels and rewards; (2) domain assumptions that contacts are sufficient subgoals and that pose similarity detects misoperations, both of which the paper's own ablations partly contradict (Square regression, cloth fallback); (3) standard DDPM machinery and offline-RL conventions. No new physical entities are posited; the objective contact is an abstract construct whose evidence is only end-task success. Total: 5 free parameters, 6 stated axioms, 1 invented construct.

free parameters (5)
  • Pose similarity thresholds tau_l and tau_r = Per task: 0.02 m and 10 deg (Can, Square, Tilt); 3 and 1 (Push-T); 0.002 m and 1 deg (Move-T)
    Hand-set per task and inherited from the authors' MRLNM [30]; they drive phase merging in Algorithm 3 via Eq. 14, hence determine Guider targets and Critic rewards.
  • Contact-reaching threshold tau_f = One quarter of end-effector size per task (e.g., 0.004 m for Can/Square, 7.5 for Push-T)
    Defines the reward in Algorithm 4: r = R if a finger is within tau_f of the contact within Tr steps; hand-set per task.
  • Q-learning loss coefficient eta = 0.001 (Table IX)
    Selected as 'optimal for most of the tasks we tested' (Sec. IV-D); ablations (Fig. 4) show eta shapes the BC/Q-learning trade-off and sensitivity grows as demonstration count shrinks.
  • Critic augmentation noise scales mu1, mu2, tau_n1, tau_n2 = mu1=0.005, mu2=0.1, tau_n1=1 (Table IX); minor noise capped at half a finger width
    Eqs. 15-16 generate the negative examples that make the Critic distinguish good from poor trajectories; hand-set, task-agnostic values.
  • Maximum reward R = 10 (Table VIII)
    Scales the Q-learning target in Eqs. 12-13; fixed per task and effectively absorbed into eta.
assumptions (6)
  • standard math Standard DDPM forward and reverse process equations (Eqs. 1-4) and the noise-prediction training target.
    Both layers are CDDPMs; correctness of the generative model is assumed from Ho et al. [59] and Diffusion Policy [9].
  • domain assumption A contact position is a sufficient subgoal for conditioning the low-level trajectory generator.
    Actor models p(At | Ot, Ct) (Eq. 7); if the contact does not capture the task-relevant interaction (e.g., deformable cloth), the method degrades and the authors fall back to immediate contacts (Appendix A).
  • domain assumption Object pose similarity identifies misoperations; contacts in merged phases can be erased without hurting performance.
    Algorithm 3 uses Eq. 14 to delete contacts when consecutive phase end-poses are similar. The paper's own Sec. VI-C shows this fails for inadvertent touches that change pose in the Square task.
  • domain assumption Reaching the objective contact (finger within tau_f within Tr steps) is a valid reward proxy for task progress.
    Algorithm 4 defines r = R only on this geometric condition (Eqs. 12-13); no object-state verification is used in the reward.
  • domain assumption Demonstration data is sufficient to train the Critic offline; the target Q-value uses the demonstrated trajectory At+Tr, not the policy trajectory.
    Eq. 13 replaces the on-policy tuple with a demonstration tuple, a fitted Q-evaluation shortcut whose validity rests on the demonstration distribution covering the policy distribution.
  • domain assumption Prebuilt object model plus ICP alignment provides usable point clouds in the real Move-T task.
    Sec. VII-A generates point clouds by transforming the pre-constructed T-block cloud to ICP-estimated poses, avoiding raw perception noise; deformable-object perception is reduced to four marker keypoints (Sec. VII-B).
invented entities (1)
  • Objective contact C_t (per-finger contact position plus contact flag, zero for non-contacting fingers)
    purpose: High-level subgoal predicted by the Guider and used to condition the Actor's trajectory generation and to define the Critic's reward.
    A modeling construct introduced by the paper; its only validation is downstream task success in Tables II, III, V, and VII. No measurement or falsifiable prediction outside the task evaluation confirms that contacts are the correct subgoal representation, and the authors fall back to immediate contacts for deformable objects (Appendix A).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidance." pith.science (2026). https://pith.science/paper/VHO62OBJ

@misc{pith2026241112982,
  author       = {Pith},
  title        = {Pith review of: Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VHO62OBJ}},
  note         = {Machine review of arXiv:2411.12982}
}
read the original abstract

Decision-making in robotics using denoising diffusion processes has increasingly become a hot research topic, but end-to-end policies perform poorly in tasks with rich contact and have limited controllability. This paper proposes Hierarchical Diffusion Policy (HDP), a new imitation learning method of using objective contacts to guide the generation of robot trajectories. The policy is divided into two layers: the high-level policy predicts the contact for the robot's next object manipulation based on 3D information, while the low-level policy predicts the action sequence toward the high-level contact based on the latent variables of observation and contact. We represent both level policies as conditional denoising diffusion processes, and combine behavioral cloning and Q-learning to optimize the low level policy for accurately guiding actions towards contact. We benchmark Hierarchical Diffusion Policy across 6 different tasks and find that it significantly outperforms the existing state of-the-art imitation learning method Diffusion Policy with an average improvement of 20.8%. We find that contact guidance yields significant improvements, including superior performance, greater interpretability, and stronger controllability, especially on contact-rich tasks. To further unlock the potential of HDP, this paper proposes a set of key technical contributions including snapshot gradient optimization, 3D conditioning, and prompt guidance, which improve the policy's optimization efficiency, spatial awareness, and controllability respectively. Finally, real world experiments verify that HDP can handle both rigid and deformable objects.

Figures

Figures reproduced from arXiv: 2411.12982 by the authors.

Figure 1
Figure 1. Inference Process of Hierarchical Diffusion Policy. Compared to the Diffusion Policy that generates operation trajectories end-to-end, HDP introduces objective contacts, predicted by the Guider network or provided by humans, to guide the trajectory generation. performance beyond that of the expert. Meanwhile, most imi￾tation learning methods struggle to handle multimodal action distributions because they are modeled… view at source ↗
Figure 2
Figure 2. Hierarchical Diffusion Policy Overview. (a) At time step t during inference, the Guider takes the latest To steps of observation data Ot as input and predicts objective contact Ct, the Actor takes observation data Ot and objective contact Ct as input and predicts Tp steps of actions, of which Ta steps of actions are executed on the robot without re-planning. During training, in addition to minimizing the prediction … view at source ↗
Figure 3
Figure 3. Phased objective contacts. The algorithm consists of three steps: recording object subgoals and contacts, merging misoperations and no-contact operations, and configuring ob￾jective contacts. Misoperations include ⃝1 overshooting the movement and ⃝2 not moving or barely moving the object. The algorithm eliminates contacts related to misoperations by detecting object pose similarity, reducing suboptimal objective con… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Q-learning Ablation Study. HDP with oneshot denoising is better and robust than iterative denoising. A larger coefficient leads to greater optimization intensity and more optimization conflicts. The sensitivity of HDP to coefficient changes and gradient imbalance cause…
Figure 5
Figure 5. Figure 5: Multimodal behavior. In the initial state, the robot can pull the left side or push the right side to move the object to the target position (black wireframe). The warm and cold curves represent the motion trajectories of the left and right fingers of the gripper, resp…
Figure 6
Figure 6. Figure 6: Prompt guidance. At the given state, the end-effector (orange) can either go left or right to push the block. The blue cross is the manually designated target contact, i.e. prompt. Hierarchical Diffusion Policy can sequentially generate trajectories towards each prompt…
Figure 7
Figure 7. Figure 7: Different Objective Contact (OC) Schemes. In the no-OC scheme, the Actor’s condition is only observation. Compared with no-OC scheme, contact guidance improves the stability of the results. Compared with nearest-OC scheme, the phased-OC scheme improves the success rate…
Figure 8
Figure 8. Figure 8: Q-learning Action Steps Ablation Study. There is no significant difference in the success rate as Tr increased [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Simulation Tasks. The Can and Square tasks require grasping, picking, and placing operations. The Tilt task re￾quires pushing, tilting, grasping, and picking operations. The Push-T task requires pushing operations. 2) Tilt: adapted from [30], requires pushing a cube th…
Figure 10
Figure 10. Figure 10: Realworld Move-T Experiment Setup. (a) Hardware setup. The robot needs to precisely (b) push or (c) clamp the T-shaped block to the target region, and (d) then move the end-effector to the end-zone. outperforms Diffusion Policy, with an average improvement of 55.4%. A…
Figure 11
Figure 11. Figure 11: Prompt Guidance in Move-T Task. Each row represents an execution example. When the robot is about to push the object off the table or fall into trajectory oscillation (within the yellow circle), a prompt can guide the robot to complete the task. The crosses in the sec…
Figure 12
Figure 12. Figure 12: IoU Distribution of Realworld Move-T Experiment. The IoU distribution of HDP is more balanced. The IoU of Diffusion Policy tends to be at either extreme, indicating that Diffusion Policy either completely fails or performs almost perfectly. visual interface to replace…
Figure 13
Figure 13. Figure 13: Robustness Test for hierarchical Diffusion Policy. Left: The Block is blocked by a towel and can still be moved to the target area. Middle: HDP immediately aborts heading to the end-zone, returning the block to goal state upon detecting block shift. Right: HDP immedia…
Figure 14
Figure 14. Figure 14: Cloth Unfolding Experiment Setup. The robot needs to use either ⃝1 picking or ⃝2 pushing to unfold the folded cloth, and sometimes a combination of these two modes of operation is required. The yellow markers are used to detect the four corner positions. Failure Count…
Figure 15
Figure 15. Figure 15: Visual interface of Move-T task. The red and green circles indicate the positions of the two fingers of the gripper, and the crosses indicate the prompts. The weights of the target critic ϕ ′ are obtained by calculat￾ing the EMA of the critic weights ϕ, as follows: ϕ …
Figure 16
Figure 16. Figure 16: Example of each test task. For Move-T task, we show running examples of two policies trained on demonstrations containing only pushing or grasping operation modes, respectively [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 43 canonical work pages

  1. [9]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” arXiv preprint arXiv:2303.04137 , 2023

    arXiv 2023

  2. [1]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al. , “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  3. [2]

    Exploration in deep rein- forcement learning: A survey,

    P. Ladosz, L. Weng, M. Kim, and H. Oh, “Exploration in deep rein- forcement learning: A survey,” Information Fusion , vol. 85, pp. 1–22, 2022

    work page 2022

  4. [3]

    Imitation learning: Progress, taxonomies and challenges,

    B. Zheng, S. Verma, J. Zhou, I. W. Tsang, and F. Chen, “Imitation learning: Progress, taxonomies and challenges,” IEEE transactions on neural networks and learning systems , vol. 35, no. 5, pp. 6322–6337, 2024

    work page 2024

  5. [4]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502 , 2020

    arXiv 2010

  6. [5]

    Improved denoising diffusion proba- bilistic models,

    A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion proba- bilistic models,” in International conference on machine learning , pp. 8162–8171. PMLR, 2021

    work page 2021

  7. [6]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10 684–10 695, 2022

    work page 2022

  8. [7]

    Diffusion policies as an expres- sive policy class for offline reinforcement learning,

    Z. Wang, J. J. Hunt, and M. Zhou, “Diffusion policies as an expres- sive policy class for offline reinforcement learning,” arXiv preprint arXiv:2208.06193, 2022

    arXiv 2022

Show all 70 references
  1. [8]

    Planning with diffusion for flexible behavior synthesis,

    M. Janner, Y . Du, J. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” in International Conference on Machine Learning, pp. 9902–9915. PMLR, 2022

    2022

  2. [10]

    Recent trends in task and motion planning for robotics: A survey,

    H. Guo, F. Wu, Y . Qin, R. Li, K. Li, and K. Li, “Recent trends in task and motion planning for robotics: A survey,” ACM Computing Surveys, vol. 55, no. 13s, pp. 1–36, 2023

    2023

  3. [11]

    Sampling-based motion planning: A comparative review,

    A. Orthey, C. Chamzas, and L. E. Kavraki, “Sampling-based motion planning: A comparative review,” Annual Review of Control, Robotics, and Autonomous Systems , vol. 7, 2023

    2023

  4. [12]

    Hybrid hierarchical learn- ing for solving complex sequential tasks using the robotic manipulation network roman,

    E. Triantafyllidis, F. Acero, Z. Liu, and Z. Li, “Hybrid hierarchical learn- ing for solving complex sequential tasks using the robotic manipulation network roman,” Nature Machine Intelligence , vol. 5, no. 9, pp. 991– 1005, 2023

    2023

  5. [13]

    Reinforcement learning in robotic applications: a comprehensive survey,

    B. Singh, R. Kumar, and V . P. Singh, “Reinforcement learning in robotic applications: a comprehensive survey,” Artificial Intelligence Review , vol. 55, no. 2, pp. 945–990, 2022

    2022

  6. [14]

    A survey on offline reinforcement learning: Taxonomy, review, and open problems,

    R. F. Prudencio, M. R. Maximo, and E. L. Colombini, “A survey on offline reinforcement learning: Taxonomy, review, and open problems,” IEEE Transactions on Neural Networks and Learning Systems , 2023

    2023

  7. [15]

    A hierarchical framework for long horizon planning of object-contact trajectories,

    B. Aceituno and A. Rodriguez, “A hierarchical framework for long horizon planning of object-contact trajectories,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pp. 189–196. IEEE, 2022

    2022

  8. [16]

    Enhancing dexterity in robotic manipulation via hierarchical contact exploration,

    X. Cheng, S. Patil, Z. Temel, O. Kroemer, and M. T. Mason, “Enhancing dexterity in robotic manipulation via hierarchical contact exploration,” IEEE Robotics and Automation Letters, vol. 9, no. 1, pp. 390–397, 2023

    2023

  9. [17]

    Hierarchical learning of robotic contact policies,

    M. Simoni ˇc, A. Ude, and B. Nemec, “Hierarchical learning of robotic contact policies,” Robotics and computer-integrated manufacturing , vol. 86, p. 102657, 2024

    2024

  10. [18]

    A learning based hierarchical control framework for human-robot collaboration,

    Z. Jin, A. Liu, W.-A. Zhang, L. Yu, and C.-Y . Su, “A learning based hierarchical control framework for human-robot collaboration,” IEEE Transactions on Automation Science and Engineering , vol. 20, DOI 10.1109/TASE.2022.3161993, no. 1, pp. 506–517, 2023

    2022

  11. [19]

    Multi-stage cable routing through hierarchical imitation learning,

    J. Luo, C. Xu, X. Geng, G. Feng, K. Fang, L. Tan, S. Schaal, and S. Levine, “Multi-stage cable routing through hierarchical imitation learning,” IEEE Transactions on Robotics , 2024

    2024

  12. [20]

    Bottom-up skill discovery from un- segmented demonstrations for long-horizon robot manipulation,

    Y . Zhu, P. Stone, and Y . Zhu, “Bottom-up skill discovery from un- segmented demonstrations for long-horizon robot manipulation,” IEEE Robotics and Automation Letters , vol. 7, no. 2, pp. 4126–4133, 2022

    2022

  13. [21]

    Hierarchical reinforcement learning with universal policies for multi- step robotic manipulation,

    X. Yang, Z. Ji, J. Wu, Y .-K. Lai, C. Wei, G. Liu, and R. Setchi, “Hierarchical reinforcement learning with universal policies for multi- step robotic manipulation,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 9, pp. 4727–4741, 2021

    2021

  14. [22]

    Residual skill policies: Learning an adaptable skill-based action space for rein- forcement learning for robotics,

    K. Rana, M. Xu, B. Tidd, M. Milford, and N. S ¨underhauf, “Residual skill policies: Learning an adaptable skill-based action space for rein- forcement learning for robotics,” in Conference on Robot Learning , pp. 2095–2104. PMLR, 2023

    2023

  15. [23]

    Compositional foundation models for hierarchical planning,

    A. Ajay, S. Han, Y . Du, S. Li, A. Gupta, T. Jaakkola, J. Tenenbaum, L. Kaelbling, A. Srivastava, and P. Agrawal, “Compositional foundation models for hierarchical planning,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  16. [24]

    Chain-of- thought predictive control,

    Z. Jia, F. Liu, V . Thumuluri, L. Chen, Z. Huang, and H. Su, “Chain-of- thought predictive control,” arXiv preprint arXiv:2304.00776 , 2023

    2023 arXiv

  17. [25]

    Do as i can, not as i say: Grounding language in robotic affordances,

    A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian et al., “Do as i can, not as i say: Grounding language in robotic affordances,” in Conference on robot learning, pp. 287–318. PMLR, 2023

    2023

  18. [26]

    Deep hierarchical planning from pixels,

    D. Hafner, K.-H. Lee, I. Fischer, and P. Abbeel, “Deep hierarchical planning from pixels,” Advances in Neural Information Processing Systems, vol. 35, pp. 26 091–26 104, 2022

    2022

  19. [27]

    Adjacency constraint for efficient hierarchical reinforcement learning,

    T. Zhang, S. Guo, T. Tan, X. Hu, and F. Chen, “Adjacency constraint for efficient hierarchical reinforcement learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 4, pp. 4152– 4166, 2022

    2022

  20. [28]

    Planning irregular object packing via hierarchical reinforcement learning,

    S. Huang, Z. Wang, J. Zhou, and J. Lu, “Planning irregular object packing via hierarchical reinforcement learning,” IEEE Robotics and Automation Letters, vol. 8, no. 1, pp. 81–88, 2022

    2022

  21. [29]

    Hierarchical diffusion policy for kinematics-aware multi-task robotic manipulation,

    X. Ma, S. Patidar, I. Haughton, and S. James, “Hierarchical diffusion policy for kinematics-aware multi-task robotic manipulation,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18 081–18 090, 2024. 17 Fig. 16: Example of each test...

    2024

  22. [30]

    Multi-stage reinforcement learning for non-prehensile manipulation,

    D. Wang, C. Liu, F. Chang, H. Huan, and K. Cheng, “Multi-stage reinforcement learning for non-prehensile manipulation,” IEEE Robotics and Automation Letters , 2024

    2024

  23. [31]

    Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,

    T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, “Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,” in 2018 IEEE international conference on robotics and automation (ICRA) , pp. 5628–5635. IEEE, 2018

    2018

  24. [32]

    Self-supervised correspon- dence in visuomotor policy learning,

    P. Florence, L. Manuelli, and R. Tedrake, “Self-supervised correspon- dence in visuomotor policy learning,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 492–499, 2019

    2019

  25. [33]

    The magical benchmark for robust imitation,

    S. Toyer, R. Shah, A. Critch, and S. Russell, “The magical benchmark for robust imitation,” Advances in Neural Information Processing Systems , vol. 33, pp. 18 284–18 295, 2020

    2020

  26. [34]

    Transporter networks: Rearranging the visual world for robotic manipulation,

    A. Zeng, P. Florence, J. Tompson, S. Welker, J. Chien, M. Attarian, T. Armstrong, I. Krasin, D. Duong, V . Sindhwani et al. , “Transporter networks: Rearranging the visual world for robotic manipulation,” in Conference on Robot Learning , pp. 726–747. PMLR, 2021

    2021

  27. [35]

    Speedfolding: Learning efficient bimanual folding of garments,

    Y . Avigal, L. Berscheid, T. Asfour, T. Kr ¨oger, and K. Goldberg, “Speedfolding: Learning efficient bimanual folding of garments,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–8. IEEE, 2022

    2022

  28. [36]

    What matters in learning from offline human demonstrations for robot manipulation,

    A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart ´ın-Mart´ın, “What matters in learning from offline human demonstrations for robot manipulation,” in Conference on Robot Learning , pp. 1678–1690. PMLR, 2022

    2022

  29. [37]

    Implicit behavioral cloning,

    P. Florence, C. Lynch, A. Zeng, O. A. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mordatch, and J. Tompson, “Implicit behavioral cloning,” in Conference on Robot Learning , pp. 158–168. PMLR, 2022

    2022

  30. [38]

    Improved contrastive di- vergence training of energy-based models,

    Y . Du, S. Li, J. Tenenbaum, and I. Mordatch, “Improved contrastive di- vergence training of energy-based models,” in International Conference on Machine Learning , pp. 2837–2848. PMLR, 2021

    2021

  31. [39]

    Learning the stein discrepancy for training and evaluating energy-based models without sampling,

    W. Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Duvenaud, and R. Zemel, “Learning the stein discrepancy for training and evaluating energy-based models without sampling,” in International Conference on Machine Learning, pp. 3732–3747. PMLR, 2020

    2020

  32. [40]

    Offline rein- forcement learning with realizability and single-policy concentrability,

    W. Zhan, B. Huang, A. Huang, N. Jiang, and J. Lee, “Offline rein- forcement learning with realizability and single-policy concentrability,” in Conference on Learning Theory , pp. 2730–2775. PMLR, 2022

    2022

  33. [41]

    Adversarially trained actor critic for offline reinforcement learning,

    C.-A. Cheng, T. Xie, N. Jiang, and A. Agarwal, “Adversarially trained actor critic for offline reinforcement learning,” in International Confer- ence on Machine Learning , pp. 3852–3878. PMLR, 2022

    2022

  34. [42]

    Action- quantized offline reinforcement learning for robotic skill learning,

    J. Luo, P. Dong, J. Wu, A. Kumar, X. Geng, and S. Levine, “Action- quantized offline reinforcement learning for robotic skill learning,” in Conference on Robot Learning , pp. 1348–1361. PMLR, 2023

    2023

  35. [43]

    Off-policy deep reinforcement learning without exploration,

    S. Fujimoto, D. Meger, and D. Precup, “Off-policy deep reinforcement learning without exploration,” in International conference on machine learning, pp. 2052–2062. PMLR, 2019

    2019

  36. [44]

    Awac: Accelerating online reinforcement learning with offline datasets,

    A. Nair, A. Gupta, M. Dalal, and S. Levine, “Awac: Accelerating online reinforcement learning with offline datasets,” arXiv preprint arXiv:2006.09359, 2020

    2006 arXiv

  37. [45]

    Eligibility traces for off-policy policy evaluation,

    D. Precup, R. S. Sutton, and S. P. Singh, “Eligibility traces for off-policy policy evaluation,” in Proceedings of the Seventeenth International Conference on Machine Learning , pp. 759–766, 2000

    2000

  38. [46]

    Gendice: Generalized offline estimation of stationary values,

    R. Zhang, B. Dai, L. Li, and D. Schuurmans, “Gendice: Generalized offline estimation of stationary values,” in International Conference on Learning Representations

  39. [47]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” in International conference on machine learning, pp. 1861–1870. PMLR, 2018

    2018

  40. [48]

    Conservative q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 1179–1191, 2020

    2020

  41. [49]

    An optimistic perspec- tive on offline reinforcement learning,

    R. Agarwal, D. Schuurmans, and M. Norouzi, “An optimistic perspec- tive on offline reinforcement learning,” in International conference on machine learning, pp. 104–114. PMLR, 2020

    2020

  42. [50]

    Cascaded diffusion models for high fidelity image generation,

    J. Ho, C. Saharia, W. Chan, D. J. Fleet, M. Norouzi, and T. Salimans, “Cascaded diffusion models for high fidelity image generation,” Journal of Machine Learning Research , vol. 23, no. 47, pp. 1–33, 2022

    2022

  43. [51]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021

    2021

  44. [52]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,

    J. Z. Wu, Y . Ge, X. Wang, S. W. Lei, Y . Gu, Y . Shi, W. Hsu, Y . Shan, X. Qie, and M. Z. Shou, “Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 7623– 7633, 2023

    2023

  45. [53]

    Sora: A review on background, technology, limitations, and opportunities of large vision models,

    Y . Liu, K. Zhang, Y . Li, Z. Yan, C. Gao, R. Chen, Z. Yuan, Y . Huang, H. Sun, J. Gao et al. , “Sora: A review on background, technology, limitations, and opportunities of large vision models,” arXiv preprint arXiv:2402.17177, 2024

    2024 arXiv

  46. [54]

    Lion: Latent point diffusion models for 3d shape generation,

    A. Vahdat, F. Williams, Z. Gojcic, O. Litany, S. Fidler, K. Kreis et al., “Lion: Latent point diffusion models for 3d shape generation,” Advances in Neural Information Processing Systems , vol. 35, pp. 10 021–10 039, 2022

    2022

  47. [55]

    Diffusion probabilistic models for 3d point cloud generation,

    S. Luo and W. Hu, “Diffusion probabilistic models for 3d point cloud generation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 2837–2845, 2021

    2021

  48. [56]

    Train- ing diffusion models with reinforcement learning,

    K. Black, M. Janner, Y . Du, I. Kostrikov, and S. Levine, “Train- ing diffusion models with reinforcement learning,” arXiv preprint arXiv:2305.13301, 2023

    2023 arXiv

  49. [57]

    Is conditional generative modeling all you need for de- cision making?

    A. Ajay, Y . Du, A. Gupta, J. B. Tenenbaum, T. S. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for de- cision making?” in The Eleventh International Conference on Learning Representations

  50. [58]

    Crossway diffusion: Im- proving diffusion-based visuomotor policy via self-supervised learning,

    X. Li, V . Belagali, J. Shang, and M. S. Ryoo, “Crossway diffusion: Im- proving diffusion-based visuomotor policy via self-supervised learning,” arXiv preprint arXiv:2307.01849 , 2023

    2023 arXiv

  51. [59]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  52. [60]

    Receding horizon control of nonlinear systems,

    D. Q. Mayne and H. Michalska, “Receding horizon control of nonlinear systems,” in Proceedings of the 27th IEEE Conference on Decision and Control, pp. 464–465. IEEE, 1988

    1988

  53. [61]

    Pointnet: Deep learning on point sets for 3d classification and segmentation,

    C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 652–660, 2017

    2017

  54. [62]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems , vol. 30, 2017

    2017

  55. [63]

    Diffusion policies for out-of- distribution generalization in offline reinforcement learning,

    S. E. Ada, E. Oztop, and E. Ugur, “Diffusion policies for out-of- distribution generalization in offline reinforcement learning,” IEEE Robotics and Automation Letters , 2024

    2024

  56. [64]

    Behavior transformers: Cloning k modes with one stone,

    N. M. Shafiullah, Z. Cui, A. A. Altanzaya, and L. Pinto, “Behavior transformers: Cloning k modes with one stone,” Advances in neural information processing systems , vol. 35, pp. 22 955–22 968, 2022

    2022

  57. [65]

    V oxposer: Composable 3d value maps for robotic manipulation with language models,

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei, “V oxposer: Composable 3d value maps for robotic manipulation with language models,” in Conference on Robot Learning, pp. 540–562. PMLR, 2023

    2023

  58. [66]

    Palm-e: An embodied multimodal language model,

    D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu et al., “Palm-e: An embodied multimodal language model,” in International Conference on Machine Learning, pp. 8469–8488. PMLR, 2023

    2023

  59. [67]

    Language models as zero-shot trajectory generators,

    T. Kwon, N. D. Palo, and E. Johns, “Language models as zero-shot trajectory generators,” IEEE Robotics and Automation Letters , vol. 9, DOI 10.1109/LRA.2024.3410155, no. 7, pp. 6728–6735, 2024

    2024

  60. [68]

    On the sample complexity of actor-critic method for reinforcement learning with function approxima- tion,

    H. Kumar, A. Koppel, and A. Ribeiro, “On the sample complexity of actor-critic method for reinforcement learning with function approxima- tion,” Machine Learning, vol. 112, no. 7, pp. 2433–2467, 2023

    2023

  61. [69]

    3d diffusion policy,

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3d diffusion policy,” arXiv preprint arXiv:2403.03954 , 2024

    2024 arXiv

  62. [70]

    On the continuity of rotation representations in neural networks,

    Y . Zhou, C. Barnes, J. Lu, J. Yang, and H. Li, “On the continuity of rotation representations in neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 5745–5753, 2019

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.