Pith. sign in

REVIEW 4 major objections 4 minor 31 references

Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A deep reinforcement learning curriculum that randomizes contact softness and ground unevenness in simulation lets a single proprioception-only policy walk a full-size humanoid over compliant and uneven real terrain with no per-terrain…

desk verdict Real humanoid sim-to-real result with honest limits; the core transfer claim stands, but the evidence is thin and the terrain proxy is rough. read the letter →

arxiv 2504.13619 v1 pith:VVLF4LYJ submitted 2025-04-18 cs.RO

Rohan P. Singh , Mitsuharu Morisawa , Mehdi Benallegue , Zhaoming Xie , Fumio Kanehiro This is my paper · ORCID
classification cs.RO
keywords sim-to-realtransferbipedallocomotionhumanoidrobotdeepreinforcementlearningcompliantterrainunevendomainrandomizationclock-basedgait
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that one blind walking policy, trained entirely in simulation from joint, inertial, and motor-current feedback, can carry a full-size humanoid across grass, pavement, a soft mattress, and cushion foam without per-terrain tuning. The recipe is a two-phase curriculum: first learn a clock-based gait on flat rigid ground, then fine-tune the same network while randomizing how softly the feet contact the ground and how uneven the ground profile is. The authors further argue that letting the policy nudge the phase of its own gait clock produces aperiodic stepping and improves robustness on uneven terrain in simulation. If the paper is right, life-sized bipedal robots can handle deformable and irregular ground without terrain sensing or explicit surface identification.

What carries the argument

Two mechanisms carry the argument. The first is a two-phase terrain curriculum: pretrain on flat, rigid ground, then fine-tune while randomizing the contact softness between feet and ground (the simulator's contact time constant over a stiff-to-spring-like range) and the heightfield's z-position from a flat floor at $-4$ cm to a fully uneven surface at $0$ cm, with terrain randomization disabled during double support. The second is a clock-modulation action: the policy outputs a scalar phase offset, clipped to $\pm 5$ control timesteps, that updates the cyclic phase variable $\phi$ via $\phi_{t+1} = \phi_t + \mathrm{clip}(a_{\delta\phi}, -5, 5) + 1$, so swing and stance durations can shorten or lengthen and the nominal 2 s gait becomes adaptive and aperiodic.

What would settle it

Measure the vertical sinkage and ground-reaction force under the robot's foot on each real test surface and compare them with the simulator's soft-contact behavior across the randomized range; if a real surface whose measured behavior falls inside that range repeatedly makes the transferred policy fail, the randomization proxy is refuted. Failing that, a single successful traversal of an obstacle above the 4 cm training height in simulation but not on the robot would separate the sim-to-real gap from the curriculum's coverage.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single end-to-end policy with only proprioceptive observations transfers zero-shot from simulation to the real HRP-5P humanoid and walks on both compliant and uneven surfaces. Training proceeds in two phases: a base policy for standing, stepping in place, and walking forward is learned on flat rigid ground, then fine-tuned on a curriculum that randomizes the soft-contact time constant of the feet and the vertical offset of a pre-generated heightfield, from flat to fully uneven. Real-robot experiments with the transferred policy succeeded in 6 of 9 indoor trials across rigid uneven blocks, a soft gym mattress, and cushion foam, and covered roughly 25 m on a paved street and 30 m on irregular grass outdoors. The paper also claims that augmenting the action space with a phase-shift action for the cyclic clock, clipped to $\pm 5$ control timesteps, leads to aperiodic swing and stance durations, higher training reward, and longer mean episode survival on simulated terrain with 5–7 cm peak unevenness; hardware deployment of this clock-control variant was left for future work.

Load-bearing premise

The load-bearing premise is that the simulated ranges of ground softness and height irregularity are good enough stand-ins for real grass, mattress foam, pavement, and uneven blocks that a policy trained on them works on the real surfaces with no adjustment.

Editorial extensions

If this is right

  • A single trained network can replace terrain-specific gait timing and gain tuning for surfaces within the simulated range of compliance and unevenness.
  • Randomizing contact softness during training is sufficient to make the policy place the foot adaptively on both stiff floors and deformable surfaces.
  • Letting the policy change its clock phase extends survival on simulated uneven terrain beyond the 4 cm training height, indicating that variable cadence is itself a robustness mechanism.
  • Because the same network handles standing, stepping in place, and walking forward, mode switching does not require separate controllers.
  • The 4 cm training cap on unevenness translates directly into a real-world obstacle-height limit, which the paper expects to raise by training with larger heightfield offsets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Adding exteroception at deployment, such as vision or foot pressure, could plausibly push the same curriculum past the 4 cm obstacle ceiling without changing the control structure.
  • The clock-modulation mechanism is a general lever for any rhythmic locomotion task where contact timing is disturbed, and the learned preference for shorter cycles (about 1.55 s forward, 1.71 s stepping) suggests cadence is tuned to command speed.
  • The paper's reported failures on slopes and in standing mode point to a concrete next experiment: slope randomization during training should remove those failure modes if the compliance-unevenness curriculum is the active ingredient.
  • A mapping from measured real-surface sinkage and ground-reaction force to simulator contact parameters would let the same curriculum transfer to other humanoid platforms without the trial-and-error used to choose the randomization ranges.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents a two-phase sim-to-real reinforcement-learning pipeline for the life-sized HRP-5P humanoid. Phase 1 trains a base policy on a flat, rigid floor; Phase 2 fine-tunes it on randomized compliant contact parameters (MuJoCo solref in [0.02, 0.4]) and randomized heightfield terrain unevenness (z-offset in (-4, 0] cm), together with intra-episode dynamics randomization. The resulting 12-dimensional-action policy is deployed zero-shot on the real robot using only proprioceptive feedback, with reports of 6/9 successful indoor trials over rigid irregular blocks, a mattress, and cushion foam, plus two outdoor walks on grass and a paved street. The paper additionally proposes a 13-dimensional-action clock-control policy that predicts phase-variable offsets to achieve aperiodic gaits; this policy is evaluated only in simulation, where it shortens the average gait cycle and yields larger mean episode lengths on unevenness heights up to 7 cm. The authors release training and evaluation code.

Significance. If the central claims hold, this is a valuable systems result: it demonstrates that a life-sized electrically actuated humanoid can walk over real compliant and uneven surfaces without exteroception or terrain-specific tuning, and it offers a concrete mechanism (clock-signal modulation) for aperiodic gait. The paper's strengths are genuine: real-robot demonstrations on a challenging platform, an ablation comparing flat-floor, uneven-rigid, fixed-compliance, and randomized-compliance training, and public code for reproducibility. The main limitations are that the simulated compliance/unevenness proxy is heuristic rather than measured, the clock-control advantage in Table V is not statistically supported, and the real-robot trial set is small and underspecified. These issues do not invalidate the demonstrations, but they do limit how strongly the robustness and mechanism claims can be stated.

major comments (4)
  1. [Section IV-C and Section IV-B] The load-bearing premise for sim-to-real transfer is that the randomized MuJoCo contact and terrain model is an adequate proxy for real mattress, foam, grass, and uneven ground. Section IV-C states that the solref range was 'determined through trial-and-error and visually inspecting the contact behavior of the simulated robot,' and Section IV-B admits that heightfield z-randomization can move the ground under the support foot and produce upward thrust that is 'unlikely to happen in the real world.' No force-penetration, sinkage, or stiffness measurements of the real terrains are reported. Because the policy could be learning to reject arbitrary contact impulses rather than terrain-appropriate compliance, this premise is not established. Please add either (i) quantitative evidence connecting the simulated contact parameters to the real terrains used, (ii) a sensitivity analysis showing the deployed policy's performance is robust across a wider range of solref and heightfield settings, or (iii) a clearly stated scoping of the claim to robustness within the simulated perturbation class rather than validated terrain-adaptive behavior on real hardware.
  2. [Table V and Section VI-C] The clock-control advantage is the paper's second contribution, but Table V reports only mean episode lengths under a 10 s cap, with no standard deviations, confidence intervals, or significance tests. At 5 cm unevenness the difference is 9.675 s versus 9.725 s, which is well within plausible sampling variation; at 4 cm the two means are identical. The 100-episode means alone do not establish that clock control improves robustness on uneven terrain. Please report per-condition variance or raw episode distributions, perform a statistical comparison, and justify mean-time-to-termination as a robustness metric given the 10 s episode cap. The training reward curves in Figure 3(a) are also averaged over only three seeds without error bars, so the claim of convergence to a higher reward is not fully supported.
  3. [Section VI-B and Section VII] The real-robot evaluation is the empirical core of the paper, but Section VI-B reports 6/9 indoor successes and two outdoor walks without specifying the trial protocol: how many consecutive attempts were made, whether successes were first-attempt, how terrain order and operator commands varied, and whether any trials were discarded. The three reported failures are attributed to slope standing and obstacles above 4 cm, which are exactly the distribution edges of the training data; this is informative, but it also means the 'robust walking' claim is supported by a small and potentially selected sample. Please report per-terrain trial counts and selection criteria, add more trials if possible, and clarify whether the relaxed safety limits were used for all subsequent trials. Otherwise the robustness claim should be tempered to 'promising initial demonstrations.'
  4. [Section V, Section VI-C, and Limitations] The paper's abstract and Section V imply that clock modulation improves robustness in a general sense, but the Limitations paragraph in Section VI-C states that real-robot deployment of the clock-control policy is 'significantly more difficult' and that the sim-to-real gap 'may be higher' than for the default policy. Since no real-robot clock-control results are presented, the contribution must be explicitly scoped as a simulation-only finding. As written, readers may infer a hardware benefit that the manuscript itself does not claim to have demonstrated. Please revise the abstract, Section V, and the conclusion to state that the clock-control advantage is shown in simulation only, with real-robot transfer deferred.
minor comments (4)
  1. [Section III-A, III-B, IV-D] There are several typographical errors: 'proprotional' should be 'proportional', 'initilialized' should be 'initialized', and 'expained' should be 'explained'; the Introduction also uses 'quit standing' where 'quiet standing' is presumably intended.
  2. [Section III-A and Table III] The reward function is not specified in the paper; the text refers to [24] and the public code for the precise terms. Given that the code is released, this is acceptable, but for self-containedness the authors should either list all reward terms and weights in an appendix or include them in the supplementary material.
  3. [Figure 1 caption] The caption says the panels are ordered 'clockwise' but does not make the panel order obvious; please label each panel with the terrain type or otherwise clarify the ordering.
  4. [Section VI-C, Figure 4] The claim that the policy 'mainly makes positive corrections' to the clock should be quantified (e.g., mean and distribution of a_delta_phi), rather than stated from inspection of a single rollout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the terrain-randomized training claim is validated by real-robot transfer and out-of-distribution simulation tests, not by construction.

full rationale

The paper's central claim is that a two-phase curriculum with randomized foot-contact compliance (solref in 0.02-0.4) and heightfield z-position (-4 to 0 cm) produces a single proprioceptive policy that transfers to real compliant and uneven terrain. This claim is evaluated empirically, not derived from the training inputs: real HRP-5P trials (6/9 indoor successes, two outdoor traversals) and ablations (baseline vs. uneven vs. fixed compliance vs. terrain-randomized) test the policy on terrain not seen during training. The clock-control claim is also tested out-of-distribution: Table V reports mean episode lengths at 4-7 cm unevenness although training used a 4 cm ceiling, so the improvement is not forced by the training distribution. The only substantive self-citation is adoption of the reward function and training hyperparameters from the authors' prior work [24]; this is transparent, code-linked, and independent of the terrain-robustness contribution. The paper's own limitations, such as solref chosen through trial-and-error and the heightfield 'upward thrust' artifact, undermine external validity of the sim-to-real proxy, but that is a correctness or robustness concern, not a circularity: no prediction is defined in terms of its fitted inputs, and no load-bearing step reduces to a self-citation chain.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

All load-bearing premises are domain assumptions about simulator fidelity and sensor sufficiency; the paper provides no measurement grounding for them. The free parameters are randomization ranges and bounds chosen by hand; none are fitted to real-robot data, which keeps circularity low but leaves transfer validity as an unvalidated assumption.

free parameters (6)
  • Foot compliance solref range = [0.02, 0.4]
    Set by trial-and-error and visual inspection of simulated contact; not measured against real surfaces. The whole soft-terrain transfer claim depends on this range.
  • Training unevenness height cap = 4 cm
    Limits terrain difficulty; real failures on obstacles above 4 cm and on slopes are explained by this cap, so robustness is only claimed within this range.
  • Dynamics randomization ranges = damping 0.2-5, friction 2-8 N-m, mass 0.95-1.05x, CoM +/-0.01 m
    Hand-chosen to hedge against unmodeled real dynamics; not derived from system identification, and the clock-control policy's sim-to-real gap is said to be higher.
  • Clock offset clip = +/-5 timesteps (0.125 s)
    Conservative bound on phase shift per step; the adaptive gait frequencies observed in the paper depend on this clip.
  • Nominal gait cycle L = 2 s at 40 Hz (80 timesteps)
    Fixed clock for training; aperiodic gaits are measured relative to this baseline period.
  • Reward weights = Table III (from [24])
    Hand-tuned in prior work; reused unchanged for terrain walking, so the learned behavior is partly determined by these weights.
assumptions (5)
  • domain assumption MuJoCo soft-contact time constant is a valid surrogate for real surface compliance
    Section IV-A randomizes the solref time constant only; no validation against measured deformation of grass, foam, or mattress is provided.
  • domain assumption Heightfield randomization creates realistic unevenness without spurious thrust
    Section IV-B disables randomization in double support to mitigate an acknowledged unphysical upward thrust artifact of the method.
  • domain assumption Instantaneous proprioception suffices for implicit terrain identification
    The policy is a feedforward MLP with no observation history and no exteroceptive input, per Section III-A and Section VII.
  • domain assumption MuJoCo model plus in-episode dynamics randomization is close enough for zero-shot transfer
    Section IV-D lists randomization ranges but gives no system identification; the paper notes a higher sim-to-real gap for the clock-control policy, indicating this assumption is imperfect.
  • domain assumption Reward terms from [24] are adequate for terrain walking
    Table III reuses the prior reward function unchanged; no terrain-specific reward terms are added, so the learned behavior inherits the prior objective's adequacy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/VVLF4LYJ

@misc{pith2026250413619,
  author       = {Pith},
  title        = {Pith review of: Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VVLF4LYJ}},
  note         = {Machine review of arXiv:2504.13619}
}
read the original abstract

For the deployment of legged robots in real-world environments, it is essential to develop robust locomotion control methods for challenging terrains that may exhibit unexpected deformability and irregularity. In this paper, we explore the application of sim-to-real deep reinforcement learning (RL) for the design of bipedal locomotion controllers for humanoid robots on compliant and uneven terrains. Our key contribution is to show that a simple training curriculum for exposing the RL agent to randomized terrains in simulation can achieve robust walking on a real humanoid robot using only proprioceptive feedback. We train an end-to-end bipedal locomotion policy using the proposed approach, and show extensive real-robot demonstration on the HRP-5P humanoid over several difficult terrains inside and outside the lab environment. Further, we argue that the robustness of a bipedal walking policy can be improved if the robot is allowed to exhibit aperiodic motion with variable stepping frequency. We propose a new control policy to enable modification of the observed clock signal, leading to adaptive gait frequencies depending on the terrain and command velocity. Through simulation experiments, we show the effectiveness of this policy specifically for walking over challenging terrains by controlling swing and stance durations. The code for training and evaluation is available online at https://github.com/rohanpsingh/LearningHumanoidWalking. Demo video is available at https://www.youtube.com/watch?v=ZgfNzGAkk2Q.

Figures

Figures reproduced from arXiv: 2504.13619 by the authors.

Figure 1
Figure 1. HRP-5P humanoid bipedal locomotion (clockwise) on flat rigid floor, soft cushion, uneven inclined blocks, paved street, and grass using learned policy. The same RL policy was used for all terrains without any parameter tuning in between experiments. Lifter and overhead crane serve as a failsafe, the ropes are slack and robot is not provided external support. must preferably be adaptive — to adapt the gait according … view at source ↗
Figure 2
Figure 2. Overview of our training framework. (L) We propose to train a feedforward RL agent while exposing it to randomized dynamics parameters in the first phase and then, additionally, randomized uneven and compliant terrains in the second phase. The policy achieves zero-shot sim-to-real transfer on the real HRP-5P. (R) We also propose an augmented policy that can make clock signal modifications for regulating the stepping… view at source ↗
Figure 4
Figure 4. Evolution of the phase variable for a 20-second episode rollout on flat, rigid terrain for walking forward and stepping in-place with a clock￾control policy. Policy phase offset predictions (top) and the actual scalar phase variable (bottom) are both normalized by a fixed period L = 80 for better visualization. a robust success rate. Our real robot experiments also help validate the use of current feedback for achie… view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Behavior analysis of Clock Control (in simulation). (a) Training reward curves for the clock-control policy and the default policy averaged over 3 training sessions with separate random seeds. Both policies are trained on randomized terrains starting from a regular ter…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 22 canonical work pages

  1. [26]

    Blind bipedal stair traversal via sim-to-real reinforcement learning,

    J. Siekmann, K. Green, J. Warila, A. Fern, and J. Hurst, “Blind bipedal stair traversal via sim-to-real reinforcement learning,” arXiv preprint arXiv:2105.08328, 2021

    arXiv 2021

  2. [1]

    Biped walking stabilization based on linear inverted pendulum tracking,

    S. Kajita, M. Morisawa, K. Miura, S. Nakaoka, K. Harada, K. Kaneko, F. Kanehiro, and K. Yokoi, “Biped walking stabilization based on linear inverted pendulum tracking,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2010, pp. 4489– 4496

    work page 2010

  3. [2]

    Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control,

    S. Caron, A. Kheddar, and O. Tempier, “Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control,” in 2019 International Conference on Robotics and Automation (ICRA) . IEEE, 2019, pp. 277–283

    work page 2019

  4. [3]

    Realization of biped walking on soft ground with stabilization control based on gait analysis,

    K. Hashimoto, H.-j. Kang, M. Nakamura, E. Falotico, H.-o. Lim, A. Takanishi, C. Laschi, P. Dario, and A. Berthoz, “Realization of biped walking on soft ground with stabilization control based on gait analysis,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2012, pp. 2064–2069

    work page 2012

  5. [4]

    Terrain classification and locomotion parameters adaptation for humanoid robots using force/torque sensing,

    K. Walas, D. Kanoulas, and P. Kryczka, “Terrain classification and locomotion parameters adaptation for humanoid robots using force/torque sensing,” in 2016 IEEE-RAS 16th International Confer- ence on Humanoid Robots (Humanoids) . IEEE, 2016, pp. 133–140

    work page 2016

  6. [5]

    Sim-to-real learning of all common bipedal gaits via periodic reward composition,

    J. Siekmann, Y . Godse, A. Fern, and J. Hurst, “Sim-to-real learning of all common bipedal gaits via periodic reward composition,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 7309–7315

    2021

  7. [6]

    Humanoid robot hrp-5p: An electrically actuated humanoid robot with high-power and wide-range joints,

    K. Kaneko, H. Kaminaga, T. Sakaguchi, S. Kajita, M. Morisawa, I. Kumagai, and F. Kanehiro, “Humanoid robot hrp-5p: An electrically actuated humanoid robot with high-power and wide-range joints,” IEEE Robotics and Automation Letters , vol. 4, no. 2, pp. 1431–1438, 2019

    work page 2019

  8. [7]

    Biped walking pattern generation by using preview control of zero-moment point,

    S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi, and H. Hirukawa, “Biped walking pattern generation by using preview control of zero-moment point,” in 2003 IEEE international conference on robotics and automation (Cat. No. 03CH37422) , vol. 2. IEEE, 2003, pp. 1620–1626

    work page 2003

Show all 31 references
  1. [8]

    Hybrid zero dynamics of planar biped walkers,

    E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek, “Hybrid zero dynamics of planar biped walkers,” IEEE transactions on automatic control, vol. 48, no. 1, pp. 42–56, 2003

    2003

  2. [9]

    Dynamic walking on compliant and uneven terrain using dcm and passivity-based whole-body control,

    G. Mesesan, J. Englsberger, G. Garofalo, C. Ott, and A. Albu- Sch¨affer, “Dynamic walking on compliant and uneven terrain using dcm and passivity-based whole-body control,” in 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids) . IEEE, 2019, pp. 25–32

    2019

  3. [10]

    Inverse dynamics control of compliant hybrid zero dynamic walking,

    J. Reher and A. D. Ames, “Inverse dynamics control of compliant hybrid zero dynamic walking,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 2040–2047

    2021

  4. [11]

    Design of a compliant bipedal walking controller for the darpa robotics challenge,

    M. A. Hopkins, R. J. Griffin, A. Leonessa, B. Y . Lattimer, and T. Furukawa, “Design of a compliant bipedal walking controller for the darpa robotics challenge,” in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) . IEEE, 2015, pp. 831–837

    2015

  5. [12]

    Walking control of bipedal robot on soft ground considering ground reaction force,

    M. Komuta, Y . Abe, and S. Katsura, “Walking control of bipedal robot on soft ground considering ground reaction force,” in 2017 IEEE/SICE International Symposium on System Integration (SII) . IEEE, 2017, pp. 318–323

    2017

  6. [13]

    Development of a static sinkage model for a biped robot on loose soil,

    S. Komizunai, A. Konno, S. Abiko, and M. Uchiyama, “Development of a static sinkage model for a biped robot on loose soil,” in 2010 IEEE/SICE International Symposium on System Integration . IEEE, 2010, pp. 61–66

    2010

  7. [14]

    A study on the benefits of using variable stiffness feet for humanoid walking on rough terrains,

    I. Frizza, H. Kaminaga, K. Ayusawa, P. Fraisse, and G. Venture, “A study on the benefits of using variable stiffness feet for humanoid walking on rough terrains,” in 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids) . IEEE, 2022, pp. 427–434

    2022

  8. [15]

    Learning quadrupedal locomotion over challenging terrain,

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,” Science Robotics, vol. 5, no. 47, 2020. [Online]. Available: https://robotics. sciencemag.org/content/5/47/eabc5986

    2020

  9. [16]

    Learning robust perceptive locomotion for quadrupedal robots in the wild,

    T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,” Science Robotics , vol. 7, no. 62, p. eabk2822, 2022

    2022

  10. [17]

    Learning quadrupedal locomotion on deformable terrain,

    S. Choi, G. Ji, J. Park, H. Kim, J. Mun, J. H. Lee, and J. Hwangbo, “Learning quadrupedal locomotion on deformable terrain,” Science Robotics, vol. 8, no. 74, p. eade2256, 2023

    2023

  11. [18]

    Learning locomotion skills for cassie: Iterative design and sim-to- real,

    Z. Xie, P. Clary, J. Dao, P. Morais, J. Hurst, and M. van de Panne, “Learning locomotion skills for cassie: Iterative design and sim-to- real,” in Proc. Conference on Robot Learning (CORL 2019) , 2019

    2019

  12. [19]

    Learning memory-based control for human-scale bipedal locomotion,

    J. Siekmann, S. Valluri, J. Dao, L. Bermillo, H. Duan, A. Fern, and J. Hurst, “Learning memory-based control for human-scale bipedal locomotion,” arXiv preprint arXiv:2006.02402 , 2020

    2006 arXiv

  13. [20]

    Learning linear policies for robust bipedal locomotion on terrains with varying slopes,

    L. Krishna, U. A. Mishra, G. A. Castillo, A. Hereid, and S. Kolathaya, “Learning linear policies for robust bipedal locomotion on terrains with varying slopes,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2021, pp. 5159–5164

    2021

  14. [21]

    Learning humanoid locomotion with transformers,

    I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Learning humanoid locomotion with transformers,” arXiv preprint arXiv:2303.03381 , 2023

    2023 arXiv

  15. [22]

    Deepwalk: Omnidirectional bipedal gait by deep reinforcement learning,

    D. Rodriguez and S. Behnke, “Deepwalk: Omnidirectional bipedal gait by deep reinforcement learning,” in 2021 IEEE international conference on robotics and automation (ICRA) . IEEE, 2021, pp. 3033–3039

    2021

  16. [23]

    Sim-to-real learning of robust compliant bipedal locomotion on torque sensor-less gear-driven humanoid,

    S. Masuda and K. Takahashi, “Sim-to-real learning of robust compliant bipedal locomotion on torque sensor-less gear-driven humanoid,” arXiv preprint arXiv:2204.03897 , 2022

    2022 arXiv

  17. [24]

    Learning bipedal walking for humanoids with current feedback,

    R. P. Singh, Z. Xie, P. Gergondet, and F. Kanehiro, “Learning bipedal walking for humanoids with current feedback,” arXiv preprint arXiv:2303.03724, 2023

    2023 arXiv

  18. [25]

    Torque-based deep reinforcement learning for task-and-robot agnostic learning on bipedal robots using sim-to-real transfer,

    D. Kim, G. Berseth, M. Schwartz, and J. Park, “Torque-based deep reinforcement learning for task-and-robot agnostic learning on bipedal robots using sim-to-real transfer,” arXiv preprint arXiv:2304.09434 , 2023

    2023 arXiv

  19. [27]

    Learning bipedal walking on planned footsteps for humanoid robots,

    R. P. Singh, M. Benallegue, M. Morisawa, R. Cisneros, and F. Kane- hiro, “Learning bipedal walking on planned footsteps for humanoid robots,” in 2022 IEEE-RAS 21st International Conference on Hu- manoid Robots (Humanoids) . IEEE, 2022, pp. 686–693

    2022

  20. [28]

    Learning natural locomotion behaviors for humanoid robots using human bias,

    C. Yang, K. Yuan, S. Heng, T. Komura, and Z. Li, “Learning natural locomotion behaviors for humanoid robots using human bias,” IEEE Robotics and Automation Letters , vol. 5, no. 2, pp. 2610–2617, 2020

    2020

  21. [29]

    Mujoco: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2012, pp. 5026–5033

    2012

  22. [30]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  23. [31]

    mc-mujoco: Simulating articulated robots with fsm controllers in mujoco,

    R. P. Singh, P. Gergondet, and F. Kanehiro, “mc-mujoco: Simulating articulated robots with fsm controllers in mujoco,” in 2023 IEEE/SICE International Symposium on System Integration (SII) . IEEE, 2023, pp. 1–5

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.