Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

REFINE-DP jointly fine-tunes a diffusion-policy planner and an RL controller, pushing humanoid loco-manipulation success from roughly 50–70% pre-trained to over 90% in simulation and 50–70%+ on hardware.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:11 UTC pith:3YPHO5GG

load-bearing objection Useful incremental method for joint fine-tuning of DP planner and low-level controller, but the 'no privileged state' claim is contradicted by the MoCap setup. the 3 major comments →

arxiv 2603.13707 v3 pith:3YPHO5GG submitted 2026-03-14 cs.RO cs.AIcs.LG

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

Zhaoyuan Gu , Yipu Chen , Zimeng Chai , Alfred Cueva , Thong Nguyen , Yifan Wu , Huishu Xue , Minji Kim
show 6 more authors
Isaac Legene Fukang Liu KyoungMok Kim Ayan Barula Yongxin Chen Ye Zhao
This is my paper
classification cs.RO cs.AIcs.LG
keywords diffusion policyRL fine-tuninghumanoidloco-manipulationhierarchical controlsim-to-realPPOdoor traversal
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the reason offline-trained diffusion planners fail on humanoid loco-manipulation is a distribution mismatch between the planner's commands and the low-level controller's ability to track them, and that the fix is to fine-tune both together with reinforcement learning rather than collect more demonstrations. The proposed REFINE-DP keeps the diffusion policy as a high-level planner that outputs base velocities and hand poses, and an RL policy as the whole-body controller; a PPO-based diffusion-policy gradient updates the planner while the controller is simultaneously adapted to the evolving command distribution. On a T1 humanoid, the method raises success rates from 50–70% pre-trained to over 90% in simulation, cuts the demonstration data needed to reach 90% from about 1,000 trajectories to 50, extends to out-of-distribution initial conditions with a curriculum, and achieves 50–70%+ success in real-world door traversal and box transport. The broader claim is that RL fine-tuning, not data scaling, is the key to making behavior-cloned planners reliable on high-dimensional dynamic robots.

Core claim

REFINE-DP jointly optimizes two policies in a hierarchical planner–controller stack: a pre-trained diffusion policy (DP) that emits low-dimensional commands (base velocity, hand poses, gripper) and an RL-based loco-manipulation controller that tracks them. The DP is fine-tuned via DPPO, treating each denoising step as a decision step in an augmented MDP so that a PPO-style policy gradient can be applied; the low-level controller is fine-tuned with the same rollouts using its own reward. Because the planner's commands are a moving target rather than the stationary commands used in controller pre-training, joint optimization brings the controller's inputs in-distribution, improving tracking ac

What carries the argument

The central mechanism is the joint optimization loop (Algorithm 1): a diffusion-process-augmented MDP that lets a PPO-style gradient update the implicit diffusion-policy density, while the same environment rollouts feed a second PPO update to the low-level controller. This co-adaptation keeps the planner's command distribution and the controller's tracking capability aligned, which is what reduces compounding error and jitter. The hierarchical decomposition itself — DP outputting base velocity and hand poses instead of full-body joint commands — is the enabling simplification that makes the action space small enough for diffusion modeling and teleoperation.

Load-bearing premise

The hardware successes and the 'no privileged state' claim rest on a motion-capture system that streams ground-truth robot and object poses to the planner at 90 Hz; strip that sensing away and the reported real-world success rates are not established.

What would settle it

Run the same REFINE-DP pipeline on the T1 with the MoCap system removed and only onboard cameras or depth sensors providing object poses; if the door-traversal and box-transport success rates fall below the MoCap-conditioned numbers (or the robot fails to start), the central claim that RL fine-tuning alone yields reliable real-world loco-manipulation is falsified. A cheaper check: inject increasing pose noise into the MoCap stream in simulation and measure the success-rate cliff.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If REFINE-DP is correct, RL fine-tuning can substitute for large demonstration datasets: a planner pre-trained on 50 trajectories reaches about 95% success after fine-tuning, where pre-training alone would need roughly 1,000.
  • Joint optimization reduces fine-tuning iterations by about half (20 vs 40 to reach 90% success) and cuts orientation tracking error by up to 50%, implying the controller must be adapted along with the planner to get both success and motion quality.
  • The curriculum-based domain randomization extends the planner's coverage to initial conditions spanning 320%, 125%, and 600% of pre-training ranges, so fine-tuning, not more data, is presented as the route to out-of-distribution robustness.
  • On hardware, the fine-tuned policy re-attempts failed manipulations and re-plans when the object moves, behaviors the pre-trained policy lacks, which supports the claim that fine-tuning adds a recovery mechanism.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own numbers imply a perception bottleneck: the 50–70% real-world success rates are achieved with a 90 Hz MoCap stream of ground-truth torso and object poses, so the method's practical margin over baselines on hardware is untested under the onboard-vision conditions that a deployed humanoid would face.
  • If the joint-optimization story generalizes, it suggests a recipe for other hierarchical learning systems: whenever a high-level policy is fine-tuned, the low-level executor should be fine-tuned in the same loop; otherwise the planner adapts to a fixed controller's quirks and overcommands, as the paper observes when fine-tuning the DP alone degrades tracking.
  • A testable extension is to apply REFINE-DP to vision-based planners (the paper lists this as future work); the same co-adaptation argument predicts that a vision-conditioned DP would also benefit from low-level controller fine-tuning, but the MoCap-free gap would need to be closed first.
  • The curriculum result suggests a general tool: when sparse rewards make direct fine-tuning collapse, progressive randomization of initial conditions (a 'start-state curriculum') can unlock out-of-distribution generalization; this is a concrete mechanism others could adopt.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. REFINE-DP proposes a hierarchical framework for humanoid loco-manipulation in which a pre-trained diffusion policy (DP) generates high-level base-velocity and hand-pose commands, and a low-level RL policy tracks these commands. The paper's main contribution is an RL fine-tuning stage that combines DPPO for the DP with PPO updates for the low-level controller, either alone or jointly. The authors report over 90% success in simulation across four tasks, improved data efficiency versus pre-training alone, OOD robustness via curriculum-based domain randomization, and real-world success rates above 70% for door traversal and above 50% for box transportation. The DPPO augmented-MDP formalism in Section III-C is a faithful adaptation of prior work, and the hierarchical decomposition is sensible. The main empirical claim to be checked is the real-world autonomy claim, since the hardware experiments rely on an external MoCap system for all object and torso pose observations.

Significance. If the results hold, the paper makes a useful practical contribution: it shows that RL fine-tuning can substantially improve the task success rate of a demonstration-trained diffusion planner, and that additionally adapting the low-level controller improves command tracking and motion smoothness. The reported data-efficiency result (50 trajectories plus fine-tuning reaching roughly 95% success where pre-training alone needs about 1,000 trajectories) is striking and worth reporting carefully. The paper also provides a clean comparison against pre-trained DPs, MLP baselines, MLP-FT, and residual RL, which is valuable for the community. However, the real-world autonomy claim is not established as stated: the state-based DP receives ground-truth-like object and torso poses from a MoCap system, and the conclusion explicitly lists vision as future work. This does not invalidate the core method, but it materially narrows the scope of the hardware demonstration. The paper's central derivation is coherent and no circularity was found, but the empirical support needs stronger statistical reporting before the quantitative claims can be fully assessed.

major comments (3)
  1. [Abstract / Section IV-A] The supplied abstract claims REFINE-DP 'enables real-world execution without privileged state information,' but Section IV-A states that the DP relies on pre-processed poses of the robot torso and target object captured by a motion capture system streamed at 90 Hz, with no onboard perception or object-pose estimator described. Section V lists 'incorporating vision modality in the planner' as future work. Thus the hardware success rates (>70% door, >50% box) are demonstrations with an external ground-truth-like state source, not autonomous perception-based execution. The claim should be revised to state the sensing assumption explicitly, and the hardware results should be described as execution under MoCap-provided object/torso state, not as 'without privileged state information.' This is a load-bearing framing issue for the real-world contribution.
  2. [Section IV-C / Section IV-D / Figures 4-6] All quantitative success rates and tracking metrics are presented as point estimates with no confidence intervals, seeds, or trial counts. Section IV-D reports 'exceeds 70%' and 'above 50%' on hardware without stating how many trials were performed, the success criterion, or the variance across runs. Since the central claims—over 90% simulation success, the margin over baselines, the up-to-50% orientation-error reduction, and the 10-20% hardware speedups—rest on these numbers, the paper should report trial counts, per-seed standard deviations or confidence intervals, and ideally success-rate intervals (e.g., Wilson intervals). Without this, the reader cannot distinguish a robust effect from a small-sample artifact.
  3. [Section III-D / Algorithm 1] The joint optimization procedure updates the low-level policy with PPO using a replay buffer D, but PPO is an on-policy algorithm and the paper does not describe importance-weighting or any off-policy correction for the replay samples. More importantly, when the low-level controller is updated, the environment transition P(S_{t+1}|S_t,A_t^0) changes, so the augmented-MDP objective in Eq. (6), which assumes a fixed MDP, is no longer an unbiased policy-gradient estimator for the current dynamics. The paper should either provide the off-policy correction and discuss the non-stationarity, or present the procedure as an approximate/alternating scheme and justify it with ablations. This point is load-bearing for the claim that 'joint optimization' is a principled part of REFINE-DP rather than an ad hoc alternating update.
minor comments (4)
  1. [Section III-A] The paper says 'Detailed observations and rewards are on our website.' For a journal article, at least the reward terms and observation definitions should be in the paper or a permanent supplement; otherwise the low-level controller and its joint optimization are not reproducible.
  2. [Section IV-D] The hardware section reports 'average speedups of around 10% and 20%' but does not specify how these were measured or how many trials each number comes from. Please clarify whether these are the same trials as the success-rate experiments.
  3. [Figure 6] The caption says the figure shows time in both simulation and hardware experiments, but the text only provides hardware speedup numbers. Please either report the corresponding hardware task-completion times or change the caption to avoid ambiguity.
  4. [Section IV-B / Equation for MLP-FT] The stochastic MLP fine-tuning baseline is described with a mean-reverting update, but it is not clear how the policy-gradient objective is computed for this implicitly defined stochastic process. A brief derivation or a pointer to the reference would help the reader assess the fairness of this baseline.

Circularity Check

0 steps flagged

No circular derivation; the MoCap dependence contradicts the 'no privileged state' claim but is not a circularity defect.

full rationale

The derivation chain is not circular. The DP pre-training objective (Eq. 2) and the DPPO augmented-MDP policy-gradient update (Eqs. 4-6) are taken from external work [6] and are standard formulations; REFINE-DP's claimed contribution is the joint update of the high-level DP and low-level controller (Alg. 1), and the reported success rates, OOD coverage, data-efficiency curves, and motion-tracking errors are measured from simulator/hardware rollouts against task success and tracking metrics that are not defined in terms of the fitted parameters. No prediction is the value of a fitted parameter renamed. Self-citations [7],[8],[18] appear but are contextual references (RL fine-tuning example, survey, motion-prior generation) and are not load-bearing: no uniqueness theorem and no fitted quantity is imported from them. The strongest caveat is experimental, not circular: the abstract's claim of real-world execution 'without privileged state information' is undercut by Section IV-A, which states the state-based DP uses MoCap-streamed torso/object poses at 90 Hz, and the Conclusion lists 'incorporating vision modality in the planner' as future work; but that is a support/validity gap regarding autonomy, not a reduction of the derivation to its inputs.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No new physical entities or forces are introduced. The method relies on standard ML components, simulator fidelity, MoCap availability, and hand-selected curriculum and randomization parameters. The free parameters are mostly design choices that directly shape the reported success rates.

free parameters (6)
  • OOD randomization ranges = radial distance 320%, polar angle 125%, heading angle 600% of pre-training ranges
    Hand-selected to define the out-of-distribution test; no independent criterion given, and the pre-trained vs fine-tuned comparison depends on these ranges.
  • Curriculum threshold and increment = 90% success-rate threshold, 10% range increase per stage
    Chosen so that fine-tuning reliably reaches the target success; this makes the OOD result partly a function of tuning rather than a fixed prediction.
  • Hardware command clamps = locomotion speed clamped to 0.2 m/s, hand speed to 0.05 m/s
    Imposed to improve real-world success; directly influences the reported hardware success rates and is not part of a principled derivation.
  • Low-level reward weights and observation details = not disclosed; deferred to project website
    The low-level controller pre-training and joint optimization depend on these weights; without them the results are not reproducible from the paper.
  • Demonstration dataset composition = 50 teleoperated trajectories augmented with heuristic-planner rollouts to 1000 trajectories
    The mix of teleoperated and automatically generated data is a design choice that affects pre-training quality and the subsequent fine-tuning comparison.
  • PPO / DPPO hyperparameters = not specified
    Learning rates, batch sizes, GAE parameters, and denoising-step counts are not given, making the fine-tuning procedure not fully reproducible from the paper.
axioms (5)
  • domain assumption IsaacLab simulation dynamics are faithful enough to the Booster T1 for pre-training, fine-tuning, and zero-shot transfer.
    The entire method is validated in IsaacLab and then transferred to hardware; if the simulator does not model contact, actuation, or perception noise well, the reported success rates are not meaningful. Invoked throughout Section IV.
  • domain assumption The velocity-to-footstep planner reliably converts DP base-velocity commands into executable footstep commands.
    Section III-A states 'We use a velocity-to-footstep planner to convert the velocity command into a sequence of footstep commands.' The low-level controller's performance depends on this hand-designed component, which is not analyzed for error propagation.
  • domain assumption MoCap provides reliable ground-truth torso and object poses at 90 Hz during real-world execution.
    Section IV-A relies on a MoCap workstation for planner observations. If the MoCap system is unavailable, noisy, or considered privileged state, the hardware results and the abstract's 'without privileged state information' claim would not hold.
  • standard math The DPPO augmented-MDP construction correctly turns the implicit diffusion policy into a likelihood-based policy-gradient objective.
    Section III-C adopts DPPO [6] as a given; the paper does not re-derive or verify its correctness in this hierarchical setting.
  • domain assumption A sparse task-success reward is a sufficient learning signal when starting from a pre-trained policy with 50-70% success.
    Section IV-C claims pre-training is essential and that sparse rewards rarely provide informative signal from random initialization. This is an empirical assumption about the reward landscape, not a proven fact.

pith-pipeline@v1.3.0-alltime-deepseek · 12335 in / 11022 out tokens · 116844 ms · 2026-08-02T18:11:44.641058+00:00 · methodology

0 comments
read the original abstract

Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-environment dynamics and long-horizon tasks. While diffusion policies (DPs) show promise for learning from demonstrations, deploying them on humanoids poses critical challenges: the motion planner trained offline is decoupled from the loco-manipulation controller, leading to poor command tracking, compounding distribution shift, and task failures. The common approach of scaling demonstration data is prohibitively expensive for high-dimensional humanoid systems. To address this challenge, we present REFINE-DP (REinforcement learning FINE-tuning of Diffusion Policy), a hierarchical framework that jointly optimizes a DP motion planner and an RL-based loco-manipulation controller. The DP is fine-tuned via a PPO-based diffusion policy gradient to improve task success rate, while the controller is simultaneously updated to accurately track the planner's evolving command distribution, reducing the distributional mismatch that degrades motion quality. We validate REFINE-DP on a humanoid robot performing loco-manipulation tasks, including door traversal and long-horizon object transport. REFINE-DP achieves an over 90% success rate in simulation, even in out-of-distribution cases not seen in the pre-training data, and enables real-world execution without privileged state information. Our proposed method substantially outperforms pre-trained DP baselines and demonstrates that RL fine-tuning is key to reliable humanoid loco-manipulation. https://refine-dp.github.io/REFINE-DP/

Figures

Figures reproduced from arXiv: 2603.13707 by Alfred Cueva, Ayan Barula, Fukang Liu, Huishu Xue, Isaac Legene, KyoungMok Kim, MinJi Kim, Thong Nguyen, Ye Zhao, Yifan Wu, Yipu Chen, Yongxin Chen, Zhaoyuan Gu, Zimeng Chai.

Figure 1
Figure 1. Figure 1: Our pipeline consists of three stages: (a) Data Collection, where expert demonstrations are collected in a source [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The sim-to-real RL of the loco-manipulation control [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The robot autonomously executes loco-manipulation tasks: (a) door opening, (b) object transportation. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of success rates across tasks in Sec. IV-A [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Joint optimization achieves better end effector tracking [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: Specifically, we evaluate upper-body tracking perfor [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of success rate between scaling pre [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. EgoEngine: From Egocentric Human Videos to High-Fidelity Dexterous Robot Demonstrations

    cs.RO 2026-06 unverdicted novelty 7.0

    EgoEngine transforms egocentric human videos into high-fidelity robot data enabling zero-shot visuomotor dexterous policy learning without real-robot demonstrations.

  2. Learning Asynchronous Upper-body Task-space Trajectory Tracking Policy for Humanoid Robots

    cs.RO 2026-06 unverdicted novelty 4.0

    An asynchronous upper-body task-space tracking policy for humanoids is learned via teacher-student distillation with sliding-window rewards and MPC post-training, showing better low-rate tracking than baselines in sim...

Reference graph

Works this paper leans on

37 extracted references · 4 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Tai- loring solution accuracy for fast whole-body model predictive control of legged robots,

    C. Khazoom, S. Hong, M. Chignoli, E. Stanger-Jones, and S. Kim, “Tai- loring solution accuracy for fast whole-body model predictive control of legged robots,”IEEE Robotics and Automation Letters, vol. 9, no. 12, pp. 11 074–11 081, 2024

    2024

  2. [2]

    Seec: Stable end- effector control with model-enhanced residual learning for humanoid loco-manipulation,

    J. Jang, Z. Wang, Z. Zhou, F. Wu, and Y . Zhao, “Seec: Stable end- effector control with model-enhanced residual learning for humanoid loco-manipulation,”IEEE International Conference on Robotics, Au- tomation (ICRA), 2026

    2026

  3. [3]

    Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning

    T. Heet al., “Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning.” inCoRL, vol. 270, 2024, pp. 1516–1540

    2024

  4. [4]

    Humanplus: Hu- manoid shadowing and imitation from humans,

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Hu- manoid shadowing and imitation from humans,” inAnnual Conference on Robot Learning, 2024

    2024

  5. [5]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”The International Journal of Robotics Research, vol. 44, no. 10-11, pp. 1684–1704, 2025

    2025

  6. [6]

    Diffusion policy policy optimization,

    A. Z. Renet al., “Diffusion policy policy optimization,” inThe Thir- teenth International Conference on Learning Representations, 2025

    2025

  7. [7]

    Ppf: Pre-training and preservative fine-tuning of humanoid locomotion via model-assumption- based regularization,

    H. Jung, Z. Gu, Y . Zhao, H.-W. Park, and S. Ha, “Ppf: Pre-training and preservative fine-tuning of humanoid locomotion via model-assumption- based regularization,”IEEE Robotics and Automation Letters, vol. 10, no. 11, pp. 11 466–11 473, 2025

    2025

  8. [8]

    Humanoid locomotion and manipulation: Current progress and challenges in control, planning, and learning,

    Z. Guet al., “Humanoid locomotion and manipulation: Current progress and challenges in control, planning, and learning,”IEEE/ASME Trans- actions on Mechatronics. DOI: 10.1109/TMECH.2025.3579247, 2025

    arXiv 2025

  9. [9]

    Twist2: Scalable, portable, and holistic humanoid data collection system,

    Y . Zeet al., “Twist2: Scalable, portable, and holistic humanoid data collection system,”arXiv preprint arXiv:2511.02832, 2025

    arXiv 2025

  10. [10]

    Falcon: Learning force-adaptive humanoid loco- manipulation,

    Y . Zhanget al., “Falcon: Learning force-adaptive humanoid loco- manipulation,” 2025

    2025

  11. [11]

    Mobile-television: Predictive motion priors for humanoid whole-body control,

    C. Luet al., “Mobile-television: Predictive motion priors for humanoid whole-body control,”arXiv preprint arXiv:2412.07773, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit,

    Q. Ben, F. Jia, J. Zeng, J. Dong, D. Lin, and J. Pang, “HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit,” inProceedings of Robotics: Science and Systems, LosAngeles, CA, USA, June 2025

    2025

  13. [13]

    Wococo: Learning whole-body humanoid control with sequential contacts,

    C. Zhang, W. Xiao, T. He, and G. Shi, “Wococo: Learning whole-body humanoid control with sequential contacts,” inAnnual Conference on Robot Learning, 2024

    2024

  14. [14]

    Curiosity-driven learning of joint locomotion and manipulation tasks,

    C. Schwarke, V . Klemm, M. v. d. Boon, M. Bjelonic, and M. Hutter, “Curiosity-driven learning of joint locomotion and manipulation tasks,” inProceedings of the 7th Conference on Robot Learning, vol. 229, 2023, pp. 2594–2610

    2023

  15. [15]

    Learning agile soccer skills for a bipedal robot with deep reinforcement learning,

    T. Haarnojaet al., “Learning agile soccer skills for a bipedal robot with deep reinforcement learning,”Science Robotics, vol. 9, no. 89, 2024

    2024

  16. [16]

    Opening the sim-to-real door for humanoid pixel-to- action policy transfer,

    H. Xueet al., “Opening the sim-to-real door for humanoid pixel-to- action policy transfer,” 2025

    2025

  17. [17]

    Sim-to-real learning for humanoid box loco-manipulation,

    J. Dao, H. Duan, and A. Fern, “Sim-to-real learning for humanoid box loco-manipulation,” inInternational Conference on Robotics and Automation, 2023

    2023

  18. [18]

    Opt2skill: Imitating dynamically-feasible whole-body trajectories for versatile humanoid loco-manipulation,

    F. Liuet al., “Opt2skill: Imitating dynamically-feasible whole-body trajectories for versatile humanoid loco-manipulation,”IEEE Robotics and Automation Letters, vol. 10, no. 11, pp. 12 261–12 268, 2025

    2025

  19. [19]

    Hier- archical planning and control for box loco-manipulation,

    Z. Xie, J. Tseng, S. Starke, M. van de Panne, and C. K. Liu, “Hier- archical planning and control for box loco-manipulation,”Proc. ACM Comput. Graph. Interact. Tech., vol. 6, no. 3, 2023

    2023

  20. [20]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” inICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023

    2023

  21. [21]

    Visualmimic: Visual humanoid loco-manipulation via motion tracking and generation,

    S. Yin, Y . Ze, H.-X. Yu, C. K. Liu, and J. Wu, “Visualmimic: Visual humanoid loco-manipulation via motion tracking and generation,” 2025

    2025

  22. [22]

    Hdmi: Learning interactive humanoid whole-body control from human videos,

    H. Weng, Y . Li, N. Sobanbabu, Z. Wang, Z. Luo, T. He, D. Ramanan, and G. Shi, “Hdmi: Learning interactive humanoid whole-body control from human videos,” 2025

    2025

  23. [23]

    From imitation to refinement – residual rl for precise assembly,

    L. Ankile, A. Simeonov, I. Shenfeld, M. Torne, and P. Agrawal, “From imitation to refinement – residual rl for precise assembly,” 2024

    2024

  24. [24]

    Rfs: Reinforce- ment learning with residual flow steering for dexterous manipulation,

    E. Su, T. Westenbroek, A. Nagabandi, and A. Gupta, “Rfs: Reinforce- ment learning with residual flow steering for dexterous manipulation,” 2026

    2026

  25. [25]

    Residual off-policy rl for finetuning behavior cloning policies,

    L. Ankile, Z. Jiang, R. Duan, G. Shi, P. Abbeel, and A. Nagabandi, “Residual off-policy rl for finetuning behavior cloning policies,” 2025

    2025

  26. [26]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”Computing Research Repository, vol. abs/1707.06347, 2017

    Pith/arXiv arXiv 2017

  27. [27]

    Efficient online reinforcement learning for diffusion policy,

    H. Ma, T. Chen, K. Wang, N. Li, and B. Dai, “Efficient online reinforcement learning for diffusion policy,” 2025

    2025

  28. [28]

    π RL: Online rl fine-tuning for flow-based vision- language-action models,

    K. Chenet al., “π RL: Online rl fine-tuning for flow-based vision- language-action models,” 2026

    2026

  29. [29]

    Isaac lab: A gpu-accelerated simulation framework for multi-modal robot learning,

    M. Mittalet al., “Isaac lab: A gpu-accelerated simulation framework for multi-modal robot learning,”arXiv preprint arXiv:2511.04831, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Beamdojo: Learning agile humanoid locomotion on sparse footholds,

    H. Wang, Z. Wang, J. Ren, Q. Ben, T. Huang, W. Zhang, and J. Pang, “Beamdojo: Learning agile humanoid locomotion on sparse footholds,” arXiv preprint arXiv:2502.10363, 2025. IEEE ROBOTICS AND AUTOMATION LETTERS 9

    Pith/arXiv arXiv 2025

  31. [31]

    Re- inforcement learning-based footstep control for humanoid robots on complex terrain,

    W. Suliman, E. Davydenko, E. Chaikovskaia, and R. Gorbachev, “Re- inforcement learning-based footstep control for humanoid robots on complex terrain,”IEEE Access, 2025

    2025

  32. [32]

    Deepmimic: example-guided deep reinforcement learning of physics-based character skills,

    X. B. Peng, P. Abbeel, S. Levine, and M. van de Panne, “Deepmimic: example-guided deep reinforcement learning of physics-based character skills,”ACM Transactions On Graphics, vol. 37, no. 4, 2018

    2018

  33. [33]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840– 6851, 2020

    2020

  34. [34]

    High- dimensional continuous control using generalized advantage estimation,

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estimation,” inInternational Conference on Learning Representations, 2016

    2016

  35. [35]

    Improved Denoising Diffusion Proba- bilistic Models,

    A. Q. Nichol and P. Dhariwal, “Improved Denoising Diffusion Proba- bilistic Models,” inProceedings of the 38th International Conference on Machine Learning. PMLR, Jul. 2021, pp. 8162–8171

    2021

  36. [36]

    Stageact: Stage-conditioned imitation for robust humanoid door opening,

    M. Lee, D. K. Kim, J. K. Bandi, M. Smith, A. Liao, A. akbar Agha- mohammadi, and S. Omidshafiei, “Stageact: Stage-conditioned imitation for robust humanoid door opening,” 2025

    2025

  37. [37]

    A behavior architecture for fast humanoid robot door traversals,

    D. Calvertet al., “A behavior architecture for fast humanoid robot door traversals,”Robotics and Autonomous Systems, vol. 195, p. 105217, 2026

    2026