Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DexHandDiff claims that diffusion planners for dexterous manipulation fail on contact-rich tasks because they generate objects as if they could move on their own, and that jointly denoising hand states, object states, and actions with…

desk verdict Empirically solid goal-adaptive dexterous manipulation planner; the guidance math and the dynamics-model evidence need revision before acceptance. read the letter →

arxiv 2411.18562 v6 pith:XBJMXHTH submitted 2024-11-27 cs.RO cs.CVcs.LG

Zhixuan Liang , Yao Mu , Yixiao Wang , Tianxing Chen , Wenqi Shao , Wei Zhan , Masayoshi Tomizuka , Ping Luo
show 1 more author
Mingyu Ding
This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords dexterousmanipulationdiffusionplanningghoststatesclassifier-guideddynamicsmodelgoaladaptationLLM-generatedguidancecontact-rich
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DexHandDiff claims that diffusion planners for dexterous hands fail in contact-rich tasks because they generate either actions or object states alone. Generating object states directly lets objects appear to move without hand contact, which the paper calls ghost states; generating only actions cannot steer toward new goals without new data. The paper instead jointly denoises hand states, object states, and actions, using a learned dynamics model plus phase-aware guidance to keep state-action pairs physically consistent. On a door-opening model trained only on 90-degree demonstrations, DexHandDiff reports 59.2% average success across goal-shifted door tasks versus 29.5% for the next best baseline, and 70.7% average success over ten in-domain and goal-adaptive task variants. The significance is that one demonstration dataset can be repurposed to new goals and even reversed tasks, with LLM-written guidance functions replacing hand-designed rewards.

What carries the argument

The load-bearing object is the joint state-action diffusion trajectory, where the denoised object is a concatenated sequence of hand states, object states, and actions. Multiple guidance objectives are combined as a product of experts, so the reverse-process gradient is the sum of individual guidance gradients. A separately learned dynamics model supplies a consistency energy, penalizing state-action pairs whose predicted next state disagrees with demonstrated dynamics. A contact-distance phase switch applies alignment guidance before contact and adds goal, dynamics, and physical-constraint guidance after contact, with the physical constraint implemented as a Heaviside bound on per-step object motion; in-hand tasks use goal, finger-motion, dynamics, and constraint guidance. A two-stage LLM process generates the differentiable guidance code from environment descriptions and task instructions.

What would settle it

Measure the learned dynamics model's one-step prediction error on guided trajectories for an out-of-distribution goal, such as 30-degree door opening, by comparing its predicted next state to the simulator's next state; if trajectories that still succeed contain large dynamics errors, then the dynamics guide is not doing the load-bearing work and ghost-state reduction should instead be attributed to the contact-phase mask.

Watch

Extended reading notes

Core claim

The central discovery is that the ghost-state failure is not a diffusion artifact but a controllability mismatch: object degrees of freedom are not directly actuated, so a diffusion model that independently denoises them invents object motion. DexHandDiff instead models the joint state-action trajectory and couples states and actions through a dynamics consistency energy in the sampling objective. A dual-phase schedule first guides the hand to the contact point, so no object motion is attempted before contact, and then guides the coupled hand-object system to the goal, with a step-size constraint bounding per-step object change. This lets a model trained only on 90-degree door opening reach 70.0% success on 30-degree opening and 40.0% on the left-hemisphere pen reorientation, against 16.7% and 23.3% for the strongest baselines on those tasks. The paper also shows LLM-generated guidance functions come within a few points of hand-crafted ones, reducing human tuning iterations from roughly twenty to about five.

Load-bearing premise

The learned dynamics model is trained only on the original demonstrations, so it must stay accurate for the out-of-distribution states and actions that goal-guided sampling produces; if it is wrong there, the dynamics guidance will push generated trajectories away from physical reality instead of preserving it.

Editorial extensions

If this is right

  • A single diffusion checkpoint can handle multiple goals of the same task if guidance is available, removing the need to recollect demonstrations for each new target configuration.
  • The state-action coupling plus dynamics guide reduces the gap between predicted and simulated hand-object states by roughly half on adapted tasks, supporting the claim that ghost states are suppressed.
  • LLM-generated guidance functions make new dexterous goals addressable through language instructions, reducing manual reward engineering from about twenty iterations to about five.
  • The architecture is light enough for real-time control: 3.96M parameters and 3.27 GFLOPS, running at 5-7 Hz natively and about 36 Hz with a faster solver and command interpolation.
  • There is a tradeoff between adaptation and in-domain accuracy: classifier-free baselines score 100% on the exact training goal (door 90 degrees) while DexHandDiff scores 90%, suggesting users trade some training-distribution performance for goal flexibility.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never validates the learned dynamics model's prediction error on out-of-distribution states, so the dynamics guidance is the most unexamined load-bearing component; an ensemble or a finetuning pass on augmented data would be a direct stress test.
  • Because each guidance function depends on explicit contact-point geometry, extending the method to novel objects likely requires automatic contact-point detection or letting the LLM infer contact structure directly from the environment description.
  • The in-domain accuracy penalty (90% versus 100% on the training goal) suggests users should expect a small performance tax when using goal-adaptive guidance, and scheduling guidance strength by task familiarity could recover some of it.
  • LLM-generated guidance lags hand-crafted guidance on door 30 degrees and pen half-side reorientation, so prompt and template refinement remains a clear lever for closing the remaining gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DexHandDiff, an interaction-aware diffusion planner for dexterous manipulation. The method jointly diffuses over concatenated state-action trajectories and combines classifier-guided goal conditioning with a learned dynamics model, a dual-phase pre-contact/post-contact guidance mechanism, and LLM-generated guidance functions. The authors evaluate on five Adroit/Shadow Hand tasks with goal-adaptive variants, reporting a 59.2% average success rate across door-opening goal shifts (Table 2) and a 70.7% average across five tasks (Table 3), and argue that the approach reduces 'ghost states' by maintaining state-action consistency.

Significance. If the proposed mechanism is made formally sound, the paper would make a useful contribution to diffusion-based planning for contact-rich manipulation. The empirical evaluation is broad and the goal-adaptivity results, if reproducible and not dominated by tuned guidance hyperparameters, would be a meaningful advance over state-only and action-only diffusion baselines. The combination of dual-phase guidance, dynamics consistency, and LLM-generated guidance functions is a novel and potentially practical pipeline. The paper also provides a clear diagnosis of the ghost-state failure mode in state-based diffusion. However, the current presentation leaves two load-bearing technical points underspecified: the differentiability of the Heaviside-based guidance terms and the validity of the learned dynamics model outside the training distribution. These need to be resolved before the central claims can be assessed as fully supported.

major comments (3)
  1. [Section 5.1, Eqs. (14) and (16)] The behaviour models h_penalty and h_finger are defined using Heaviside step functions. The classifier-guided formalism in Eqs. (7)-(8) and the product-of-experts derivation require the guidance gradient g = -Σ ∇_τ ε_i(τ,c) to be well-defined, which in turn requires the energies ε_i to be differentiable (or at least Lipschitz, as assumed in Section 3.3). The gradient of a Heaviside function is a Dirac measure, not a function; the statement that applying Eq. (11) makes ε_penalty 'a Dirac delta function that directly sets value when satisfying the constraints' does not resolve this, because a delta measure cannot be used as a gradient term in the Gaussian reverse-process update. The authors should either replace Eqs. (14) and (16) with smooth surrogate penalties that are actually used in implementation (e.g., sigmoid or soft-threshold terms), or provide a rigorous treatment of how step-function behaviour models are incorporated. This is load-bearing because the entire dual-guidance mechanism is presented as gradient-based classifier guidance.
  2. [Section 5.1, Eq. (13), and Appendix D] The phase-transition threshold δ1 and the penalty thresholds δ2 and δ3 are never reported, despite being central to the dual-phase guidance; the text says a 'smooth transition mask' is applied, but Eq. (13) is a hard inequality based on δ1. In addition, Appendix D states that the guidance scale α is task-dependent, selected from {500, 1000, 2000}, but the per-task values used to produce Tables 2 and 3 are not given. Because these parameters directly control the strength of the guidance terms that produce the reported success rates, the results are not reproducible without these values, and it is unclear whether the reported averages reflect robust behaviour or manually tuned guidance gains.
  3. [Section 5.1, Eq. (12), and Section 6.5, Table 6] The learned dynamics model T(s,a), trained only on demonstration data, is used as the dynamics-consistency guidance ε_dyn at inference on out-of-distribution goals (e.g., door at 30°, door closing, pen left-hemisphere rotation, hammer half-drive). The paper never validates T's prediction error on these OOD settings, so it is not established that ε_dyn provides a correct penalty rather than a corrupting one. The paper's own ablation in Table 6 shows that Dyn+Joint (31.7%) barely improves over Joint S&A (30.8%), and Dyn-guide alone gives the same average as Diffuser (27.5%). Moreover, the analysis in Section 6.1 reports that 8 of 14 failures on the 70° task opened to 90°, which is consistent with learned dynamics overriding the goal guidance. The authors should report T's prediction error against the simulator on OOD rollouts, or provide an OOD ablation with and without ε_dyn, to support the claim that dynamics-based guidance enables robust adaptability.
minor comments (4)
  1. [References, [30]] Reference [30] is listed as 'Dexdiffuser: Interaction-aware diffusion planning for adaptive dexterous manipulation, arXiv:2411.18562, 2024', which is the same arXiv identifier as the present manuscript. Citing your own prior version as if it were independent external work is misleading; this reference should either be removed or clearly presented as a self-citation with an explicit description of the relationship.
  2. [Section 6.3, Table 4] The 'ghost state' metric in Table 4 is underspecified: the text does not define what 'predicted' and 'simulated' states are, how the L2 distance is computed and normalized beyond 'per dimension', or why Conditional DP is excluded. Without this detail, the quantitative claim that DexHandDiff 'nearly halves' the baseline's gap is hard to interpret.
  3. [Section 6.1] The phrase 'over 3 tries for 10 random seeds' is ambiguous about the total number of evaluation episodes per task. Clarify whether this means 3 episodes per seed (30 episodes total) or some other configuration, and report the total episode count.
  4. [Table 2] There is a typographical formatting issue in the Decision Diffuser row for Open 50°, where '3.3 ± 4.7' appears as '3 .3 ± 4.7'.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the only signal is a minor non-load-bearing self-citation (Ref. [30] is the paper's own arXiv ID).

full rationale

The derivation chain in DexHandDiff is self-contained in the relevant sense. The claimed predictions (success rates on door 30/50/70/110/close, pen half-side, hammer half-drive, etc.) are obtained from simulator rollouts under held-out goal configurations and are compared against external baselines (Diffuser, Decision Diffuser, Diffusion Policy, Conditional DP). The guidance terms in Eqs. 10-15 are defined as objective and constraint energies over the trajectory, not as fitted values of the reported success metric; the learned dynamics model T(s,a) in Eq. 12 is an auxiliary consistency constraint trained on demonstrations, and its use in inference is a methodological choice rather than a renamed version of the evaluation target. The one circularity-adjacent artifact is reference [30], which cites the paper's own arXiv preprint (arXiv:2411.18562) as prior work in the related-work list; this is a self-citation but it is not load-bearing anywhere in the derivation or experimental logic. The Appendix's classifier-guidance theorem is a standard Bayes and Taylor-approximation argument supported by [15,28] and restated, not imported as a uniqueness theorem. The unvalidated out-of-distribution accuracy of the dynamics model is a legitimate correctness risk, but that is an empirical robustness concern, not a circular reduction of the kind where Eq. X equals Eq. Y by construction.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

No new physical entities are introduced. The core depends on hand-set thresholds, per-task guidance scaling, and an unvalidated learned dynamics model, which are the main free parameters and assumptions.

free parameters (6)
  • Guidance scale alpha = 500, 1000, or 2000 per task
    Selected based on empirical performance (Appendix D); controls how strongly guidance steers denoising; exact value per task not reported.
  • Phase transition threshold delta1 = not reported in paper; 0.1 in sample code
    Palm-object distance threshold in Eq. 13 that switches between pre-contact and post-contact guidance; critical to dual-phase behavior.
  • Object state change threshold delta2 = not reported
    Used in Eq. 14 penalty to cap per-step object movement; value not given.
  • Finger motion threshold delta3 = not reported
    Used in Eq. 16 to require finger joint movement; value not given.
  • Reward scaling factor targets = e.g., first reward ~12, goal reward ~30, dynamics ~1.2 (door prompt)
    Baked into LLM prompts (Appendix E/F); determines relative weighting of guidance terms; tuned by prompt design.
  • Inference planning horizon = 8 for door/block, 32 for hammer/pen
    Receding horizon control; differs from training horizon 32 (Appendix D).
assumptions (6)
  • standard math Classifier-guided diffusion mean shift requires Lipschitz smoothness and first-order Taylor expansion of log p(y|tau)
    Appendix A Theorems A.1/A.2; standard in prior classifier guidance literature.
  • standard math Product of experts h_i can be combined multiplicatively and normalizing constants ignored
    Eq. 10 in Section 5.1; standard product-of-experts assumption.
  • domain assumption Learned dynamics model T(s,a) approximates the true simulator at inference
    Eq. 12; needed for dynamics-consistency guidance; no OOD validation.
  • ad hoc to paper Tasks decompose into pre-contact and post-contact phases determined by palm-object distance
    Eq. 13; threshold delta1; used for door, hammer, etc.; in-hand tasks use single phase.
  • ad hoc to paper Heaviside-based behavior models (Eqs. 14, 16) are valid differentiable guidance objectives
    The paper treats hard indicators as energy functions; gradients are zero almost everywhere, so the guidance mechanism is not well-defined.
  • domain assumption D4RL and TQC+HER demonstrations are sufficiently expert and diverse
    Used to train diffusion and dynamics models; coverage determines OOD adaptability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation." pith.science (2026). https://pith.science/paper/XBJMXHTH

@misc{pith2026241118562,
  author       = {Pith},
  title        = {Pith review of: DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XBJMXHTH}},
  note         = {Machine review of arXiv:2411.18562}
}
read the original abstract

Dexterous manipulation with contact-rich interactions is crucial for advanced robotics. While recent diffusion-based planning approaches show promise for simple manipulation tasks, they often produce unrealistic ghost states (e.g., the object automatically moves without hand contact) or lack adaptability when handling complex sequential interactions. In this work, we introduce DexHandDiff, an interaction-aware diffusion planning framework for adaptive dexterous manipulation. DexHandDiff models joint state-action dynamics through a dual-phase diffusion process which consists of pre-interaction contact alignment and post-contact goal-directed control, enabling goal-adaptive generalizable dexterous manipulation. Additionally, we incorporate dynamics model-based dual guidance and leverage large language models for automated guidance function generation, enhancing generalizability for physical interactions and facilitating diverse goal adaptation through language cues. Experiments on physical interaction tasks such as door opening, pen and block re-orientation, object relocation, and hammer striking demonstrate DexHandDiff's effectiveness on goals outside training distributions, achieving over twice the average success rate (59.2% vs. 29.5%) compared to existing methods. Our framework achieves an average of 70.7% success rate on goal adaptive dexterous tasks, highlighting its robustness and flexibility in contact-rich manipulation.

Figures

Figures reproduced from arXiv: 2411.18562 by the authors.

Figure 1
Figure 1. (a) Previous diffusers directly apply goal guidance to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Framework of DexHandDiff. DexHandDiff employs joint state-action diffusion with interaction-aware guidance. Before interaction (top middle), guidance aligns the hand to the object contact point. Upon contact (bottom middle), additional guidance steers both hand and object states towards the goal (“&” means state concatenation at input level), enforcing physical constraints and avoiding ghost states. A learned dynami… view at source ↗
Figure 4
Figure 4. Visualization results of goal-adaptive tasks by DexHandDiff. For each task, training data sample (with orange stroke) is followed by inference on novel goals beyond the training data. In the Door task, DexHandDiff guides the door to new target angle (30◦ ) and holds the door in position when the hand releases, which cannot be attained by simply truncating actions from 90◦ training data. DexHandDiff avoids ghost stat… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Visualization of goal-adaptive door manipulation. Despite training only on 90◦ demonstrations, DexHandDiff adapts to various target angles (30◦ -110◦ ) and door closing, maintaining stable control and physical consistency throughout the motion sequence. C.1. Goal Adapt…
Figure 6
Figure 6. Figure 6: Visualization of pen manipulation tasks. Top: right-half re-orientation (training distribution). Middle: left-half re-orientation, requiring challenging large-arc rotation from the initial horizontal-right position. Bottom: dynamic goal tracking where target yaw angle …
Figure 7
Figure 7. Figure 7: Visualization of hammer and block manipulation tasks. Top two rows: full and partial nail-driving tasks, demonstrating precise control over interaction depth. Bottom two rows: block orientation tasks with quaternion-based pose control, showing adaptation to both positi…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World

    cs.RO 2025-02 conditional novelty 6.0 of 10

    CordViP achieves strong real-world dexterous manipulation by feeding a diffusion policy with pose-tracked 3D object models and hand point clouds, pretrained on contact maps and arm-hand coordination.

  2. G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulation

    cs.RO 2024-11 conditional novelty 6.0 of 10

    G3Flow improves diffusion policies for robotic manipulation by maintaining a complete, object-centric 3D semantic representation, called semantic flow, during interaction.

Reference graph

Works this paper leans on

99 extracted references · 56 canonical work pages · cited by 2 Pith papers

  1. [30]

    Dexdiffuser: Interaction-aware dif- fusion planning for adaptive dexterous manipulation

    Zhixuan Liang, Yao Mu, Yixiao Wang, Fei Ni, Tianxing Chen, Wenqi Shao, Wei Zhan, Masayoshi Tomizuka, Ping Luo, and Mingyu Ding. Dexdiffuser: Interaction-aware dif- fusion planning for adaptive dexterous manipulation. arXiv preprint arXiv:2411.18562, 2024. 2

    arXiv 2024

  2. [1]

    Is conditional gen- erative modeling all you need for decision making? In The Eleventh International Conference on Learning Representa- tions, 2023

    Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B Tenenbaum, Tommi S Jaakkola, and Pulkit Agrawal. Is conditional gen- erative modeling all you need for decision making? In The Eleventh International Conference on Learning Representa- tions, 2023. 1, 2, 3, 4, 6, 7

    2023

  3. [2]

    Solving rubik’s cube with a robot hand

    Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Ma- teusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113, 2019. 1

    arXiv 1910

  4. [3]

    Hind- sight experience replay

    Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. Hind- sight experience replay. Advances in neural information pro- cessing systems, 30, 2017. 6

    2017

  5. [4]

    Learning dexterous in-hand manipulation

    OpenAI: Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al. Learning dexterous in-hand manipulation. The Inter- national Journal of Robotics Research, 39(1):3–20, 2020. 1

    2020

  6. [5]

    Claude 3.5 sonnet, 2024

    Anthropic. Claude 3.5 sonnet, 2024. Available at: https://www.anthropic.com/news/claude-3- 5-sonnet. 8

    2024

  7. [6]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158, 2024. 2

    arXiv

  8. [7]

    Roboscript: Code generation for free-form manipulation tasks across real and simulation

    Junting Chen, Yao Mu, Qiaojun Yu, Tianming Wei, Silang Wu, Zhecheng Yuan, Zhixuan Liang, Chao Yang, Kaipeng Zhang, Wenqi Shao, et al. Roboscript: Code generation for free-form manipulation tasks across real and simulation. arXiv preprint arXiv:2402.14623, 2024. 3

    arXiv 2024

Show all 99 references
  1. [8]

    A system for gen- eral in-hand object re-orientation

    Tao Chen, Jie Xu, and Pulkit Agrawal. A system for gen- eral in-hand object re-orientation. In Conference on Robot Learning, pages 297–307. PMLR, 2022. 1

    2022

  2. [9]

    G3flow: Generative 3d semantic flow for pose-aware and generalizable object ma- nipulation

    Tianxing Chen, Yao Mu, Zhixuan Liang, Zanxin Chen, Shijia Peng, Qiangyu Chen, Mingkun Xu, Ruizhen Hu, Hongyuan Zhang, Xuelong Li, et al. G3flow: Generative 3d semantic flow for pose-aware and generalizable object ma- nipulation. arXiv preprint arXiv:2411.18369, 2024. 2

    2024 arXiv

  3. [10]

    Towards human-level bimanual dexterous manipulation with reinforcement learn- ing

    Yuanpei Chen, Tianhao Wu, Shengjie Wang, Xidong Feng, Jiechuan Jiang, Zongqing Lu, Stephen McAleer, Hao Dong, Song-Chun Zhu, and Yaodong Yang. Towards human-level bimanual dexterous manipulation with reinforcement learn- ing. Advances in Neural Information Processing Systems ,...

    2022

  4. [11]

    Bi-dexhands: Towards human-level bimanual dexter- ous manipulation

    Yuanpei Chen, Yiran Geng, Fangwei Zhong, Jiaming Ji, Jiechuang Jiang, Zongqing Lu, Hao Dong, and Yaodong Yang. Bi-dexhands: Towards human-level bimanual dexter- ous manipulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 2

    2023

  5. [12]

    Vividex: Learning vision-based dexterous manipula- tion from human videos

    Zerui Chen, Shizhe Chen, Cordelia Schmid, and Ivan Laptev. Vividex: Learning vision-based dexterous manipula- tion from human videos. arXiv preprint arXiv:2404.15709,

    arXiv

  6. [13]

    Dextransfer: Real world multi-fingered dexterous grasp- ing with minimal human demonstrations

    Zoey Qiuyu Chen, Karl Van Wyk, Yu-Wei Chao, Wei Yang, Arsalan Mousavian, Abhishek Gupta, and Dieter Fox. Dextransfer: Real world multi-fingered dexterous grasp- ing with minimal human demonstrations. arXiv preprint arXiv:2209.14284, 2022. 2

    2022 arXiv

  7. [14]

    Diffusion policy: Visuomotor policy learning via action dif- fusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action dif- fusion. The International Journal of Robotics Research, page 02783649241273668, 2023. 1, 2, 3, 4, 6, 7

    2023

  8. [15]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 3, 1

    2021

  9. [16]

    Learning universal policies via text-guided video genera- tion

    Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video genera- tion. Advances in Neural Information Processing Systems , 36, 2024. 2

    2024

  10. [17]

    On the theory of stochastic processes, with particular reference to applications

    William Feller. On the theory of stochastic processes, with particular reference to applications. In Selected Papers I , pages 769–798. Springer, 2015. 3

    2015

  11. [18]

    D4rl: Datasets for deep data-driven rein- forcement learning

    Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven rein- forcement learning. arXiv preprint arXiv:2004.07219, 2020. 6

    2004 arXiv

  12. [19]

    Reset-free reinforcement learning via multi-task learning: Learning dexterous manipulation behaviors with- out human intervention

    Abhishek Gupta, Justin Yu, Tony Z Zhao, Vikash Kumar, Aaron Rovinsky, Kelvin Xu, Thomas Devlin, and Sergey Levine. Reset-free reinforcement learning via multi-task learning: Learning dexterous manipulation behaviors with- out human intervention. In 2021 IEEE International Conf...

    2021

  13. [20]

    Omnih2o: Universal and dexterous human- to-humanoid whole-body teleoperation and learning

    Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous human- to-humanoid whole-body teleoperation and learning. arXiv preprint arXiv:2406.08858, 2024. 2

    2024 arXiv

  14. [21]

    Training products of experts by mini- mizing contrastive divergence

    Geoffrey E Hinton. Training products of experts by mini- mizing contrastive divergence. Neural computation, 14(8): 1771–1800, 2002. 4

    2002

  15. [22]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  16. [23]

    Generalization in dexterous manipulation via geometry-aware multi-task learning

    Wenlong Huang, Igor Mordatch, Pieter Abbeel, and Deepak Pathak. Generalization in dexterous manipulation via geometry-aware multi-task learning. arXiv preprint arXiv:2111.03062, 2021. 1

    2021 arXiv

  17. [24]

    Planning with diffusion for flexible behavior syn- thesis

    Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior syn- thesis. In International Conference on Machine Learning , pages 9902–9915. PMLR, 2022. 1, 2, 3, 4, 5, 6, 7, 8 9

    2022

  18. [25]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. 3

    2015

  19. [26]

    Controlling overestimation bias with truncated mixture of continuous distributional quantile crit- ics

    Arsenii Kuznetsov, Pavel Shvechikov, Alexander Grishin, and Dmitry Vetrov. Controlling overestimation bias with truncated mixture of continuous distributional quantile crit- ics. In International Conference on Machine Learning, pages 5556–5566. PMLR, 2020. 6

    2020

  20. [27]

    Code as policies: Language model programs for embodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 9493–9500. IEEE, 2023. 3

    2023

  21. [28]

    Adaptdiffuser: Diffusion mod- els as adaptive self-evolving planners

    Zhixuan Liang, Yao Mu, Mingyu Ding, Fei Ni, Masayoshi Tomizuka, and Ping Luo. Adaptdiffuser: Diffusion mod- els as adaptive self-evolving planners. In International Con- ference on Machine Learning, pages 20725–20745. PMLR,

  22. [29]

    Skilldiffuser: Interpretable hi- erarchical planning via skill abstractions in diffusion-based task execution

    Zhixuan Liang, Yao Mu, Hengbo Ma, Masayoshi Tomizuka, Mingyu Ding, and Ping Luo. Skilldiffuser: Interpretable hi- erarchical planning via skill abstractions in diffusion-based task execution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2024

  23. [31]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongx- uan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022. 8

    2022 arXiv

  24. [32]

    Grasping di- verse objects with simulated humanoids

    Zhengyi Luo, Jinkun Cao, Sammy Christen, Alexander Winkler, Kris Kitani, and Weipeng Xu. Grasping di- verse objects with simulated humanoids. arXiv preprint arXiv:2407.11385, 2024. 2

    2024 arXiv

  25. [33]

    Eureka: Human-level reward design via coding large language models

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931, 2023. 3, 6

    arXiv

  26. [34]

    Dexvip: Learning dexterous grasping with human hand pose priors from video

    Priyanka Mandikal and Kristen Grauman. Dexvip: Learning dexterous grasping with human hand pose priors from video. In Conference on Robot Learning , pages 651–661. PMLR,

  27. [35]

    Robocodex: Multimodal code generation for robotic behavior synthesis

    Yao Mu, Junting Chen, Qing-Long Zhang, Shoufa Chen, Qiaojun Yu, GE Chongjian, Runjian Chen, Zhixuan Liang, Mengkang Hu, Chaofan Tao, et al. Robocodex: Multimodal code generation for robotic behavior synthesis. In Forty-first International Conference on Machine Learning, 2024. 3

    2024

  28. [36]

    Deep dynamics models for learning dexter- ous manipulation

    Anusha Nagabandi, Kurt Konolige, Sergey Levine, and Vikash Kumar. Deep dynamics models for learning dexter- ous manipulation. In Conference on Robot Learning, pages 1101–1112. PMLR, 2020. 2

    2020

  29. [37]

    Metadiffuser: Diffusion model as conditional planner for offline meta-rl

    Fei Ni, Jianye Hao, Yao Mu, Yifu Yuan, Yan Zheng, Bin Wang, and Zhixuan Liang. Metadiffuser: Diffusion model as conditional planner for offline meta-rl. In International Con- ference on Machine Learning, pages 26087–26105. PMLR,

  30. [38]

    Multi-goal reinforcement learning: Challenging robotics environments and request for research,

    Matthias Plappert, Marcin Andrychowicz, Alex Ray, Bob McGrew, Bowen Baker, Glenn Powell, Jonas Schneider, Josh Tobin, Maciek Chociej, Peter Welinder, Vikash Kumar, and Wojciech Zaremba. Multi-goal reinforcement learning: Challenging robotics environments and request for research,

  31. [39]

    Markov decision processes: discrete stochastic dynamic programming

    Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 1994. 3

    1994

  32. [40]

    Dexmv: Imitation learning for dexterous manipulation from human videos

    Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu, Hanwen Jiang, Rui- han Yang, Yang Fu, and Xiaolong Wang. Dexmv: Imitation learning for dexterous manipulation from human videos. In European Conference on Computer Vision, pages 570–587. Springer, 2022. 2

    2022

  33. [42]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations

    Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giu- lia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087, 2017. 2

    2017 arXiv

  34. [43]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Pa...

    2015

  35. [44]

    Shadow robot

    Shadow Robot Company. Shadow robot. https://www. shadowrobot.com/, 2024. Accessed: 2024-11-14. 6

    2024

  36. [45]

    Robotic telekinesis: Learning a robotic hand imita- tor by watching humans on youtube

    Aravind Sivakumar, Kenneth Shaw, and Deepak Pathak. Robotic telekinesis: Learning a robotic hand imita- tor by watching humans on youtube. arXiv preprint arXiv:2202.10448, 2022. 2

    2022 arXiv

  37. [46]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems , pages 5026–5033. IEEE, 2012. 4

    2012

  38. [47]

    Unidexgrasp++: Im- proving dexterous grasping policy learning via geometry- aware curriculum and iterative generalist-specialist learning

    Weikang Wan, Haoran Geng, Yun Liu, Zikang Shan, Yaodong Yang, Li Yi, and He Wang. Unidexgrasp++: Im- proving dexterous grasping policy learning via geometry- aware curriculum and iterative generalist-specialist learning. In Proceedings of the IEEE/CVF International Conference ...

    2023

  39. [48]

    Dexcap: Scalable and portable mocap data collection system for dexterous manipulation

    Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, and C Karen Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation. arXiv preprint arXiv:2403.07788, 2024. 2

    2024 arXiv

  40. [49]

    Heaviside step function

    Eric W Weisstein. Heaviside step function. https://mathworld. wolfram. com/, 2002. 5

    2002

  41. [50]

    Dexdiffuser: Generating dexterous grasps with diffu- sion models

    Zehang Weng, Haofei Lu, Danica Kragic, and Jens Lun- dell. Dexdiffuser: Generating dexterous grasps with diffu- sion models. arXiv preprint arXiv:2402.02989, 2024. 2, 1 10

    2024 arXiv

  42. [51]

    Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots

    Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang, Zhix- uan Liang, Zeyu Lu, Ying Shan, and Ping Luo. Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. arXiv preprint arXiv:2405.07990, 2024. 3

    2024 arXiv

  43. [52]

    Unidexfpm: Uni- versal dexterous functional pre-grasp manipulation via dif- fusion policy

    Tianhao Wu, Yunchong Gan, Mingdong Wu, Jingbo Cheng, Yaodong Yang, Yixin Zhu, and Hao Dong. Unidexfpm: Uni- versal dexterous functional pre-grasp manipulation via dif- fusion policy. arXiv preprint arXiv:2403.12421, 2024. 1, 2, 5

    2024 arXiv

  44. [53]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. In Pro- ceedings of the European conference on computer vision (ECCV), pages 3–19, 2018. 3

    2018

  45. [54]

    Text2reward: Reward shaping with language models for re- inforcement learning

    Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. Text2reward: Reward shaping with language models for re- inforcement learning. In The Twelfth International Confer- ence on Learning Representations, 2024. 3, 6

    2024

  46. [55]

    Dexterous manipulation for multi-fingered robotic hands with reinforcement learning: A review

    Chunmiao Yu and Peng Wang. Dexterous manipulation for multi-fingered robotic hands with reinforcement learning: A review. Frontiers in Neurorobotics, 16:861825, 2022. 2

    2022

  47. [56]

    Learning fine-grained bimanual manipulation with low-cost hardware

    Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705, 2023. 2

    2023 arXiv

  48. [57]

    Learning diverse bimanual dexterous manipula- tion skills from human demonstrations

    Bohan Zhou, Haoqi Yuan, Yuhui Fu, and Zongqing Lu. Learning diverse bimanual dexterous manipula- tion skills from human demonstrations. arXiv preprint arXiv:2410.02477, 2024. 2

    2024 arXiv

  49. [58]

    Dexterous manipulation with deep reinforcement learning: Efficient, general, and low- cost

    Henry Zhu, Abhishek Gupta, Aravind Rajeswaran, Sergey Levine, and Vikash Kumar. Dexterous manipulation with deep reinforcement learning: Efficient, general, and low- cost. In 2019 International Conference on Robotics and Au- tomation (ICRA), pages 3651–3657. IEEE, 2019. 2 11 D...

    2019

  50. [60]

    In manipulation tasks involving interaction with an object, such as opening a door, hammer striking, note that we cannot directly control the object’s state

    The guidance function should be differentiable, which computes a scalar reward indicating how well each intermediate trajectory aligns with the task objectives. In manipulation tasks involving interaction with an object, such as opening a door, hammer striking, note that we ca...

  51. [65]

    [optional] extra constraint of the robot, which is often implied by the task instruction ... Environment Description: class BaseEnv(gym.Env): self.hand : AdroitHand # The Adroit Shadow Hand used in the environment self.door : Door # The Door object in the environment self.dt :...

  52. [71]

    Write a guidance function for a diffusion-based planner that helps the Adroit Shadow Hand open the door to 30 degrees (pi/6 radians)

    The transition between Phase 1 and Phase 2 by using a grasp mask to determine if the hand has successfully grasped the object. Use a condition like ‘mask = torch.norm(palm_pos[:, 0, :] - handle_pos[:, 0, :], p=2, dim=1) < 0.1‘ to switch from guiding only the hand to guiding bo...

  53. [75]

    For some term, use ‘torch.mean()‘ to accumulate reward over the horizon

    All the reward including the goal achieving reward should be across all horizon steps. For some term, use ‘torch.mean()‘ to accumulate reward over the horizon. For terms where the last dimension is 1 (such as angles), we should use torch.squeeze to remove that dimension before...

  54. [76]

    Use ‘self.scaling_factors‘ as an empty dictionary by default. If the scaling factor for any reward component does not exist, initialize it adaptively to make that first reward term in batch approximately 12 initially, except for the goal-achieving reward (make the reward 30) a...

  55. [80]

    Ensure that the guidance function uses soft interpolation for targets, e.g., smoothly guiding the door hinge angle towards soft goals over the trajectory horizon like ‘interpolated_angle = (1 - alpha) * current_angle + alpha * target_angle‘. 7 E.3. Hand Pen Task Prompt Example...

  56. [82]

    In manipulation tasks involving interaction with an object, such as rotating a pen, note that we cannot directly control the object’s state

    The guidance function should be differentiable, which computes a scalar reward indicating how well each intermediate trajectory aligns with the task objectives. In manipulation tasks involving interaction with an object, such as rotating a pen, note that we cannot directly con...

  57. [85]

    regularization of the object’s state change (e.g., encourage the hand joint movement to enhance interaction with the object)

  58. [87]

    [optional] extra constraint of the robot, which is often implied by the task instruction ... Environment Description: class BaseEnv(gym.Env): self.hand : AdroitHand # The Adroit Shadow Hand used in the environment self.pen : Pen # The Pen object in the environment self.target ...

  59. [88]

    All angles are expressed in radians. 9

  60. [92]

    Write a guidance function for a diffusion-based planner that helps the Adroit Shadow Hand rotate the pen to the desired target orientation

    Use L2 distance via ‘torch.norm(,p=2)‘ to calculate all the difference instead of mse loss or ‘torch.abs‘. For terms where the last dimension is 1 (such as angles), we should use torch.squeeze to remove that dimension before calculating the norm at dimension 1, rather than dim...

  61. [94]

    The function should return the ‘reward‘ as a torch.Tensor of shape ‘(B,)‘

    Then write a differentiable guidance function that guides the planner to generate actions smoothly based on the current normed state and action, with the function prototype as ‘def guidance_fn(self, normed_obs, normed_actions, dyn_model=None, without_normalizer=False, desired_...

  62. [95]

    For some term, use ‘torch.mean()‘ to accumulate reward over the horizon

    All the reward including the goal achieving reward should be across all horizon steps. For some term, use ‘torch.mean()‘ to accumulate reward over the horizon

  63. [96]

    You should first normalize the direction vector and then use inner product to calculate the similarity between two orientations

    Use input ‘desired_pen‘ as the target rotation, but you should reshape it by ‘target_rotation = desired_pen[..., -3:].reshape(batch_size, 1, 3).repeat(1, horizon, 1)‘. You should first normalize the direction vector and then use inner product to calculate the similarity betwee...

  64. [97]

    You encourage the hand joint movement to enhance interaction with the object

    Don’t directly use actions to penalize the reward, but you can use the difference between the current and previous hand joint states to penalize the reward. You encourage the hand joint movement to enhance interaction with the object

  65. [98]

    Use ‘self.scaling_factors‘ as an empty dictionary by default. If the scaling factor for any reward component does not exist, initialize it adaptively to make that first reward term in batch approximately 1 initially, except for the the dynamics reward (make it 2.)

  66. [101]

    When you writing code, you can also add some comments as your thought, like this: ‘‘‘ # Here unnormalize the observations if a normalizer is provided # Here use ‘torch.norm‘ to compute the L2 distance between the current and target angles for the door hinge ‘‘‘ Few-shot hint:

  67. [102]

    If use soft goals, don’t calculate another hard goal reward

    Ensure that the guidance function uses soft interpolation for targets, e.g., smoothly guiding the pen orientation towards soft goals over the trajectory horizon like ‘interpolated_angle = (1 - alpha) * current_obj_orien + alpha * desired_orien‘. If use soft goals, don’t calcul...

  68. [103]

    Only consider the smoothness of the hand joint movement

    No smoothness reward for the pen movement. Only consider the smoothness of the hand joint movement. E.4. Hand Hammer Task Prompt Example You are an expert in robotics, diffusion model, reinforcement learning, and code generation. We are going to use an Adroit Shadow Hand to co...

  69. [104]

    The guidance function is used to steer the sampling process toward desired outcomes during the reverse diffusion process

  70. [105]

    In manipulation tasks involving interaction with an object, such as opening a door, hammer striking, note that we cannot directly control the object’s state

    The guidance function should be differentiable, which computes a scalar reward indicating how well each intermediate trajectory aligns with the task objectives. In manipulation tasks involving interaction with an object, such as opening a door, hammer striking, note that we ca...

  71. [106]

    difference between the current state of the object and its goal state

  72. [107]

    dynamics constraints to ensure the interactions between the hand and the object are physically plausible

  73. [108]

    regularization of the object’s state change (e.g., limiting the hinge state change of a door to avoid abrupt movements)

  74. [109]

    [optional] extra constraint of the target object, which is often implied by the task instruction

  75. [110]

    [optional] extra constraint of the robot, which is often implied by the task instruction ... Environment Description: class BaseEnv(gym.Env): self.hand : AdroitHand # The Adroit Shadow Hand used in the environment self.hammer : Hammer # The Hammer object in the environment sel...

  76. [111]

    All angles are expressed in radians

  77. [112]

    The normed_obs is gotten from ‘normed_obs = get_obs()‘

    The input ‘normed_obs‘ is a tensor with shape (B, H, obs_dim), ‘normed_actions‘ is a tensor with shape (B, H, act_dim), where B is the batch size, H is the horizon length. The normed_obs is gotten from ‘normed_obs = get_obs()‘

  78. [113]

    If you need to match the observations or actions to some explicit value and if not without_normalizer, you should unnormalize them using ‘self.unnormalize(normed_obs, is_obs=True)‘

  79. [114]

    Only consider it in phase 2

    If ‘dyn_model‘ is provided, please call ‘self.cal_dyn_reward(state=normed_obs, action=normed_actions)‘ to calculates the reward for dynamics inconsistency (a scalar value) between generated states and actions. Only consider it in phase 2. Pay attention the input should be norm...

  80. [115]

    Use L2 distance via ‘torch.norm(,p=2)‘ to calculate all the difference instead of mse loss or ‘torch.abs‘

  81. [116]

    Write a guidance function for a diffusion-based planner that helps the Adroit Shadow Hand grasp the hammer and only drive half nail into the board

    The transition between Phase 1 and Phase 2 by using a grasp mask to determine if the hand has successfully grasped the object. Use a condition like ‘mask = torch.norm(palm_pos[:, 0, :] - handle_pos[:, 0, :], p=2, dim=1) < 0.1‘ to switch from guiding only the hand to guiding bo...

  82. [117]

    Please think step by step and explain what it means in the context of this environment

  83. [118]

    The function should return the ‘reward‘ as a torch.Tensor of shape ‘(B,)‘

    Then write a differentiable guidance function that guides the planner to generate actions smoothly based on the current normed state and action, with the function prototype as ‘def guidance_fn(self, normed_obs, normed_actions, dyn_model=None, without_normalizer=False)‘. The fu...

  84. [119]

    In Phase 2, guide both the object toward the final task goal

    Make sure the guidance aligns with the two phases: In Phase 1, only calculate a pre-grasp reward to guide the hand closer to the object. In Phase 2, guide both the object toward the final task goal. Ensure object velocity constraints are applied to regulate object state changes

  85. [120]

    For some term, use ‘torch.mean()‘ to accumulate reward over the horizon

    All the reward including the goal achieving reward should be across all horizon steps. For some term, use ‘torch.mean()‘ to accumulate reward over the horizon. For terms where the last dimension is 1 (such as angles), we should use torch.squeeze to remove that dimension before...

  86. [121]

    Ensure that all operations are compatible with PyTorch tensors and the function is differentiable

    Take care of variables’ type, never use functions or variables not provided. Ensure that all operations are compatible with PyTorch tensors and the function is differentiable. Do not use any absolute value operation and inplace operations, e.g. ‘x += 1‘, ‘x[0] = 1‘, using ‘x =...

  87. [122]

    Pay attention to the physical meaning of each dimension in the observation and action data as explained in the environment description above

  88. [123]

    When you writing code, you can also add some comments as your thought, like this: ‘‘‘ # Here unnormalize the observations if a normalizer is provided # Here use ‘torch.norm‘ to compute the L2 distance between the current and target angles for the door hinge # Here cauculate th...

  89. [124]

    Use nail insertion displacement (range from -0.01 No Drive to 0.09 Full Drive) as a measure of nail insertion degree

  90. [125]

    Constraint both the hammer qpos changes and the hammer position & rotation changes (pos+rot as 1 item) to avoid abrupt movements

  91. [126]

    Use ‘self.scaling_factors‘ as an empty dictionary by default. If the scaling factor for any reward component does not exist, initialize it adaptively to make that first reward term in batch approximately 6 initially, except for the hammer qpos change constraint reward (make th...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.