Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Pretraining on a single rigid body and transferring through a mass-inertia homotopy yields faster, more stable learning of dynamic quadruped maneuvers than baselines, and the policies deploy on a real robot.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:15 UTC pith:ATABJ6PQ

load-bearing objection A solid, clearly-written RL transfer method with credible real-robot demos; the homotopy story is somewhat over-sold and the evidence is thinner than the prose, but it deserves a serious referee. the 4 major comments →

arxiv 2512.24698 v2 pith:ATABJ6PQ submitted 2025-12-31 cs.RO

Dynamic Policy Learning for Legged Robot with Simplified Model Pretraining and Model-Homotopy-Inspired Transfer

Dongyun Kang , Min-Gyu Kim , Tae-Gyu Song , Hajun Kim , Sehoon Ha , Hae-Won Park This is my paper
classification cs.RO
keywords model homotopysingle rigid body modelreinforcement learninglegged locomotioncontinuation methodsim-to-real transferquadruped robotbackflip
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the gap between a simplified single-rigid-body (SRB) simulator and a full-body legged robot can be bridged by a continuation path rather than by tracking or imitation. After pretraining a policy in the SRB environment, the method gradually increases leg mass and inertia while trimming the trunk to keep total mass constant, walking the policy through a sequence of intermediate models to the full-body environment. Compared to direct fine-tuning, an imitation-transfer baseline, and vanilla RL, this model-homotopy transfer converges faster, reaches higher normalized returns, and shows much less run-to-run variation; for the wall-assisted backflip it converges twice as fast as imitation transfer and returns 19% higher reward. The framework successfully generates flips, wall-assisted jumps, and wall-assisted backflips in simulation and deploys the trot and backflip policies on a real quadruped.

Core claim

The central discovery is that a policy pretrained on a single-rigid-body model can be transferred to full-body dynamics by linearly interpolating mass and inertia parameters between the trunk and legs through a homotopy parameter λ (from 0.01 to 1 over 900 iterations). This parametric transition keeps the total mass constant while progressively re-introducing leg inertia, swing dynamics, and contact effects. In experiments, the model-homotopy transfer is the fastest to converge across all tested motions, attains the highest normalized return, and has the tightest distribution across five seeds. For the wall-assisted backflip, the method attains a 19.0% higher normalized return than imitation

What carries the argument

The load-bearing device is the model homotopy: a family of simulators indexed by λ in which leg mass and inertia scale as λ times their full-body values, while the trunk's mass, inertia, and center-of-mass are simultaneously interpolated to keep the total mass and inertia of the system constant. At small λ the environment behaves essentially like the idealized SRB model with massless legs; at λ=1 it is the full-body model. The policy is trained with a standard policy-gradient reinforcement learning algorithm while λ is linearly increased from 0.01 to 1 over 900 iterations, followed by fine-tuning in the full-body environment. The control interface remains the same across the path—ground-reac

Load-bearing premise

The linear mass-and-inertia redistribution in the paper is assumed to be a valid continuation path—at every intermediate λ the interpolated simulator is physically meaningful and the optimal policy changes slowly enough that the reinforcement learning algorithm can track it from the SRB solution; the paper provides no analysis of the reward or policy landscape along λ and no sensitivity study for the hand-selected 900-iteration linear schedule.

What would settle it

A concrete check: run the same SRB pretraining and transfer protocol but replace the linear mass-inertia interpolation with a step change that goes directly from λ=0.01 to λ=1 after a single pretraining phase; if normalized return and convergence time are statistically indistinguishable, the homotopy path is not doing the claimed work. Alternatively, measure the policy's normalized return at intermediate λ values during training: if there is a λ at which the success rate dips to near zero, the continuation path is not smooth and the outcome depends on a fortuitous schedule rather than a princi

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the homotopy path is a valid continuation, the same SRB pretraining can seed many different dynamic behaviors by reusing the simplified model and switching only the task reward, removing the need for demonstration data or heavy reward shaping.
  • The method's seed-to-seed consistency suggests that the continuation path eliminates most of the variance that the model gap injects into policy transfer, a direct benefit for reproducible robot training.
  • Because the transferred policy preserves reactive behaviors learned during pretraining, it is more robust to external disturbances than an imitation-based policy, which is a concrete advantage for real-world deployment.
  • The backflip on real hardware implies that the homotopy-transferred policy accounts for leg inertia and generates sufficient angular momentum, something the SRB policy cannot do, indicating the transfer adapts the simplified motion rather than merely tracking it.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One could test whether the homotopy itself or merely the gradual curriculum causes the speed-up by comparing against a schedule that jumps directly to full-body dynamics after one intermediate point; if convergence is similar, the intermediate path may not be the active ingredient.
  • A reverse homotopy—starting from full-body and moving toward SRB—could quantify which dynamic effects (leg inertia, swing collisions, contact transients) are hardest for the policy to adapt to, providing a diagnostic tool for other model gaps.
  • Extending the homotopy to contact parameters (e.g., friction, restitution, contact noise) in the same continuation style might further narrow the sim-to-real gap for terrains where surface properties dominate.
  • If the linear λ schedule is a lucky hand-tuned choice, an adaptive schedule driven by the policy's success rate per λ could make the method more principled and robust, and would also test the paper's assumption that the optimal policy changes slowly along the path.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a two-stage reinforcement learning framework for generating dynamic quadrupedal motions. In Stage 1, a policy is pretrained in a Single Rigid Body (SRB) simulation with task-specific contact plans, Bezier swing trajectories, and a hand-designed reward table. In Stage 2, the policy is transferred to a full-body simulator by a continuation procedure in which the trunk and leg mass, inertia, and center-of-mass parameters are linearly interpolated with a parameter lambda from near-zero leg inertia to the full-body values. The method is evaluated on gaits, backflip, sideflip, yawspin, wall-assisted turn/jump/backflip, compared with Direct Transfer, Imitation Transfer, and Vanilla RL over five seeds, and deployed on a Unitree Go1 for trotting and backflip. The central claim is that the homotopy transfer yields faster convergence, higher final return, and better stability than the baselines.

Significance. If the central claim is fully substantiated, the paper makes a useful practical contribution: it shows that simplified-model pretraining plus a mass-inertia continuation path can produce complex dynamic behaviors without demonstrations or heavy reward tuning, and the real-robot backflip is a non-trivial validation. The paper's strengths include a broad set of dynamic tasks, a reasonable baseline set (Direct Transfer, Imitation Transfer, Vanilla RL), five training seeds, disturbance robustness tests, and real-hardware deployment with quantitative trajectory comparisons. The main weakness is that the mechanism underpinning the claimed advantage -- the validity of the homotopy path -- is not directly measured, and the statistical reporting is limited to raw max-min ranges rather than standard errors or confidence intervals. The contribution is therefore conditional on additional path-level and statistical evidence.

major comments (4)
  1. [Section IV-A vs Section III-A/III-B1] The proposed homotopy does not actually connect the two environments used for pretraining and transfer. The SRB pretraining environment is a separate idealized simulator with massless legs, ideal point contacts, no leg collisions, and no swing-leg PD, whereas the full-body environment includes leg geometry, contact models, collision, and task-space PD. The equations in Section IV-A (m_leg = lambda*m_leg,full, etc.) only rescale inertial parameters inside the full-body simulator; at lambda=0.01 the environment is a full-body simulation with nearly massless legs, not the SRB model. Thus the path does not continuously bridge the SRB environment to the full-body environment; the SRB policy is an external warm start and the continuation is an inertia curriculum inside the target simulator. This is load-bearing for the claimed 'principled bridge' and should be clarified or the path redefined t
  2. [Section IV-A and Section V-B] No evidence is reported that the optimal policy or achievable return changes slowly along the continuation parameter lambda. The frozen SRB-pretrained policy's success or return as a function of lambda is never measured, and no intermediate learning curves or policy evaluation at lambda values such as 0.2, 0.5, or 0.8 are shown. The linear schedule (lambda from 0.01 to 1 over 900 iterations) is a hand-selected hyperparameter with no sensitivity study. With only endpoint comparisons, the faster convergence claim could be an artifact of a tuned curriculum rather than evidence for a homotopy mechanism. Please report return/success along lambda for the frozen pretrained policy and after short adaptation, and include at least one ablation of the schedule (duration or shape) or a theoretical justification.
  3. [Section V-B, Fig. 4] The main quantitative comparisons are based on five seeds, but only raw max-min ranges are reported; no standard deviations, confidence intervals, or individual-run plots (except the wall-assisted backflip learning curve) are provided. The specific claim that Model Homotopy Transfer achieves a 19.0% higher normalized return and converges twice as fast as Imitation Transfer rests on single summary numbers. Please report mean +/- std (or confidence intervals) for all tasks and seeds, and state whether the ordering of methods is consistent across seeds. This is necessary to support the 'superior stability' and 'faster convergence' conclusions.
  4. [Section V-B2, Fig. 5] The disturbance robustness comparison is only between Model Homotopy Transfer and Imitation Transfer; Direct Transfer and Vanilla RL are omitted, and the reported success rates are averages over 100 disturbance directions with no variance or statistical summary. Since robustness is presented as a supporting advantage of the method, please either include the other baselines or clearly scope the claim to the comparison shown, and report variability across seeds or disturbance samples.
minor comments (6)
  1. [Section IV-A] Typo: 'paramter' should be 'parameter'.
  2. [Figure 2 caption] Typo: 'Model Homotopy Tranfer' should be 'Transfer'.
  3. [Table I] The notation 'rw w - (alpha * tw) * alpha' is ambiguous; define alpha and tw clearly, and specify whether the reward is active only in the air phase.
  4. [Appendix A, Table III] The target orientation notation such as 'Rz = -xhat' is nonstandard and could be confused with a rotation matrix entry; please define the Euler-angle or quaternion convention used.
  5. [Section V-C, Fig. 7] Panel (b) lacks axis labels on the second subplot; the time axis appears to be missing, making the joint-angle comparison hard to read.
  6. [Section VI] The conclusion appropriately acknowledges the reliance on predefined contact sequences and nominal swing trajectories; this limitation should also be mentioned in the abstract or introduction so readers are not misled about the degree of automation.

Circularity Check

0 steps flagged

No circularity: all headline results are empirical RL comparisons; no prediction is equivalent by construction to a fitted input or self-cited premise.

full rationale

No load-bearing circular step was found. The paper's central claim is that SRB pretraining followed by the model homotopy transfer converges faster and more stably than Direct Transfer, Imitation Transfer, and Vanilla RL (Sec. V-B). This is an empirical result measured by normalized return, converged iteration, and max-min range (Fig. 4), not a quantity derived from the method's own fitted parameters. The homotopy path is explicitly defined by the interpolation equations in Sec. IV-A (leg mass/inertia scaled by lambda, trunk mass/inertia/CoM linearly interpolated to keep total mass constant); this is a constructive definition, not a self-definitional relation in which the output is its input. The lambda schedule (linear 0.01 to 1 over 900 iterations) is a hand-chosen curriculum hyperparameter; lacking a sensitivity study is a validation weakness, but no equation in the paper reduces the reported speed-up to that choice by construction. Self-citations ([4], [9], [34], [35], [39], [40]) appear only as background or as implementation details (e.g., concurrent estimator, motor operating region model) and do not carry the core homotopy-transfer claim or forbid alternatives. The paper's own limitation statement in Sec. VI ('our framework currently relies on predefined contact sequences and nominal swing trajectories') is an acknowledged engineering scope limit, not a circular reduction. The reviewer concern that no frozen-policy success-versus-lambda measurement is reported concerns whether the continuation path is physically meaningful; that is an empirical validity question, not a circularity. Accordingly, the derivation chain is self-contained and the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The method relies on several hand-tuned free parameters (λ schedule, reward weights, contact plans, PD gains) and on domain assumptions about model fidelity and the validity of the continuation path. No new physical entities are introduced. The counts here indicate that the paper's contribution is a training recipe whose components are largely chosen by hand rather than derived from first principles.

free parameters (4)
  • Continuation schedule (lambda) = linear 0.01 to 1 over 900 iterations
    The hand-chosen schedule for how quickly the model transitions from SRB-like to full-body. No sensitivity study is provided, and the training result depends on this choice.
  • Reward weights and shapes (a, b, u, l) for each task = Table I values
    Each motion has its own hand-tuned reward parameters. These directly shape the learned behavior and are not derived from any principle.
  • Contact plans and Bezier swing-trajectory control points = Appendix A values
    Predefined contact sequences and nominal swing trajectories are task-specific hand-engineering. The authors acknowledge this as a limitation in the conclusion.
  • PD gains for swing leg control = Kp=500, Kd=5
    Fixed gains chosen for task-space PD control during swing; the paper does not describe a tuning procedure.
axioms (4)
  • domain assumption Single Rigid Body model with massless legs, ideal point contacts, and GRF control is a useful and valid approximation for pretraining.
    Used throughout Section III-A and III-B to define the pretraining environment; the fidelity of this approximation is assumed, not derived.
  • domain assumption The full-body simulation is a faithful model of the real robot, including rotor inertia, latency, and motor operating regions (Section V-C).
    Sim-to-real transfer validity depends on the simulation being accurate; the simulator itself is not named, so this cannot be independently checked.
  • ad hoc to paper The mass-inertia redistribution path (Section IV-A) yields a smooth continuation of the policy optimization landscape.
    The central premise of the method. The paper assumes that linearly interpolating leg and trunk mass/inertia produces a learnable sequence of environments without analyzing the reward landscape or providing a theoretical guarantee.
  • standard math PPO with the described network and 100 parallel environments can reliably optimize the policies over the chosen horizon.
    Standard RL assumptions about PPO convergence and hyperparameter choices; not proven in the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 11926 in / 8605 out tokens · 92307 ms · 2026-08-03T13:15:13.446141+00:00 · methodology

0 comments
read the original abstract

Generating dynamic motions for legged robots remains a challenging problem. While reinforcement learning has achieved notable success in various legged locomotion tasks, producing highly dynamic behaviors often requires extensive reward tuning or high-quality demonstrations. Leveraging reduced-order models can help mitigate these challenges. However, the model discrepancy poses a significant challenge when transferring policies to full-body dynamics environments. In this work, we introduce a continuation-based learning framework that combines simplified model pretraining and model-homotopy-inspired transfer to efficiently generate and refine complex dynamic behaviors. First, we pretrain the policy using a single rigid body model to capture core motion patterns in a simplified environment. Next, we employ a continuation strategy to progressively transfer the policy to the full-body environment, minimizing performance loss. To define the continuation path, we introduce a parametric transition path from the single rigid body model to the full-body model by gradually redistributing mass and inertia between the trunk and legs. The proposed method achieves faster convergence and demonstrates superior stability during the transfer process compared to baseline methods. Our framework is validated on a range of dynamic tasks, including flips and wall-assisted maneuvers, and is successfully deployed on a real quadrupedal robot.

Figures

Figures reproduced from arXiv: 2512.24698 by Dongyun Kang, Hae-Won Park, Hajun Kim, Min-Gyu Kim, Sehoon Ha, Tae-Gyu Song.

Figure 1
Figure 1. Figure 1: Conceptual illustration of the proposed model homotopy [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed learning framework. The motion policy is first pretrained on a simplified SRB model to learn core motion [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Snapshots of wall-assisted backflip. The red lines represent the CoM trajectories over time. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of training performance across various methods [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of robustness across disturbances. Each heatmap [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Snapshots of the backflip execution on the Unitree Go1 robot. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Quantitative analysis of real-world deployment. (a) Trot velocity [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Contact plan of wall-assisted motions Phase ptarget Rtarget Bound (0) (0.4, 0, 0.3) - Air (1) (0.4, 0, 0.3) - Jump Up (2) (0.85, 0, 0.6) - Air (3) (0.85, 0, 0.6) Rz = −xˆ Wall (4) (0.2, 0, 0.3) Rz = −xˆ Air (5) (0.2, 0, 0.3) Rz = zˆ, Rx = −xˆ Landing (6) (0.0, 0, 0.3) Rz = zˆ, Rx = −xˆ TABLE III: Keyframe targets of the wall-assisted turn. To effectively explore the interaction with the wall, the swing tra… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. WARL: Wrench-Augmented Reinforcement Learning for Task-Agnostic Learning in Legged Robots

    cs.RO 2026-07 conditional novelty 4.0

    Adding a simulated torso wrench during early RL training and gradually removing it lets a quadruped learn six locomotion tasks with a shared reward, yielding a joint-only policy in simulation.

Reference graph

Works this paper leans on

43 extracted references · 5 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Synthesis and stabilization of complex behaviors through online trajectory optimization,

    Y . Tassa, T. Erez, and E. Todorov, “Synthesis and stabilization of complex behaviors through online trajectory optimization,” in2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012, pp. 4906–4913

    2012

  2. [2]

    Discovery of complex behaviors through contact-invariant optimization,

    I. Mordatch, E. Todorov, and Z. Popovi ´c, “Discovery of complex behaviors through contact-invariant optimization,”ACM Transactions on Graphics (ToG), vol. 31, no. 4, pp. 1–8, 2012

    2012

  3. [3]

    Multicontact locomotion of legged robots,

    J. Carpentier and N. Mansard, “Multicontact locomotion of legged robots,” IEEE Transactions on Robotics, vol. 34, no. 6, pp. 1441–1460, 2018

    2018

  4. [4]

    Contact-implicit mpc: Controlling diverse quadruped motions without pre-planned contact modes or trajectories,

    G. Kim, D. Kang, J.-H. Kim, S. Hong, and H.-W. Park, “Contact-implicit mpc: Controlling diverse quadruped motions without pre-planned contact modes or trajectories,”arXiv preprint arXiv:2312.08961, 2023

    Pith/arXiv arXiv 2023

  5. [5]

    Fast online trajectory optimization for the bipedal robot cassie

    T. Apgar, P. Clary, K. Green, A. Fern, and J. W. Hurst, “Fast online trajectory optimization for the bipedal robot cassie.” inRobotics: Science and Systems, vol. 101. Pittsburgh, Pennsylvania, USA, 2018, p. 14

    2018

  6. [6]

    Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,

    J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim, “Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,” in2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2018, pp. 1–9

    2018

  7. [7]

    Gait and trajectory optimization for legged systems through phase-based end- effector parameterization,

    A. W. Winkler, C. D. Bellicoso, M. Hutter, and J. Buchli, “Gait and trajectory optimization for legged systems through phase-based end- effector parameterization,”IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1560–1567, 2018

    2018

  8. [8]

    Real-time constrained nonlinear model predictive control on so (3) for dynamic legged locomotion,

    S. Hong, J.-H. Kim, and H.-W. Park, “Real-time constrained nonlinear model predictive control on so (3) for dynamic legged locomotion,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 3982–3989

    2020

  9. [9]

    Dynamically- consistent trajectory optimization for legged robots via contact point decomposition,

    S. Kim, H. Kim, G. Kim, M.-G. Kim, and H.-W. Park, “Dynamically- consistent trajectory optimization for legged robots via contact point decomposition,”arXiv preprint arXiv:2510.24069, 2025

    arXiv 2025

  10. [10]

    Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control,

    D. Kim, J. Di Carlo, B. Katz, G. Bledt, and S. Kim, “Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control,”arXiv preprint arXiv:1909.06586, 2019

    Pith/arXiv arXiv 1909

  11. [11]

    Whole-body motion planning with centroidal dynamics and full kinematics,

    H. Dai, A. Valenzuela, and R. Tedrake, “Whole-body motion planning with centroidal dynamics and full kinematics,” in2014 IEEE-RAS International Conference on Humanoid Robots. IEEE, 2014, pp. 295– 302

    2014

  12. [12]

    Momentum- aware trajectory optimization and control for agile quadrupedal locomo- tion,

    Z. Zhou, B. Wingo, N. Boyd, S. Hutchinson, and Y . Zhao, “Momentum- aware trajectory optimization and control for agile quadrupedal locomo- tion,”IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 7755–7762, 2022

    2022

  13. [13]

    Staged contact optimization: Combining contact-implicit and multi-phase hybrid trajectory optimiza- tion,

    M. R. Turski, J. Norby, and A. M. Johnson, “Staged contact optimization: Combining contact-implicit and multi-phase hybrid trajectory optimiza- tion,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 2376–2383

    2023

  14. [14]

    Contact-timing and trajectory optimization for 3d jumping on quadruped robots,

    C. Nguyen and Q. Nguyen, “Contact-timing and trajectory optimization for 3d jumping on quadruped robots,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 11 994–11 999

    2022

  15. [15]

    Learning quadrupedal locomotion over challenging terrain,

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Science robotics, vol. 5, no. 47, p. eabc5986, 2020

    2020

  16. [16]

    Anymal parkour: Learning agile navigation for quadrupedal robots,

    D. Hoeller, N. Rudin, D. Sako, and M. Hutter, “Anymal parkour: Learning agile navigation for quadrupedal robots,”Science Robotics, vol. 9, no. 88, p. eadi7566, 2024

    2024

  17. [17]

    High-speed control and navigation for quadrupedal robots on complex and discrete terrain,

    H. Kim, H. Oh, J. Park, Y . Kim, D. Youm, M. Jung, M. Lee, and J. Hwangbo, “High-speed control and navigation for quadrupedal robots on complex and discrete terrain,”Science Robotics, vol. 10, no. 102, p. eads6192, 2025

    2025

  18. [18]

    Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization,

    M. Bogdanovic, M. Khadiv, and L. Righetti, “Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization,”Frontiers in Robotics and AI, vol. 9, p. 854212, 2022

    2022

  19. [19]

    Learning agile robotic locomotion skills by imitating animals,

    X. B. Peng, E. Coumans, T. Zhang, T.-W. Lee, J. Tan, and S. Levine, “Learning agile robotic locomotion skills by imitating animals,”arXiv preprint arXiv:2004.00784, 2020

    Pith/arXiv arXiv 2004

  20. [20]

    Deepmimic: Example-guided deep reinforcement learning of physics-based character skills,

    X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne, “Deepmimic: Example-guided deep reinforcement learning of physics-based character skills,”ACM Transactions On Graphics (TOG), vol. 37, no. 4, pp. 1–14, 2018

    2018

  21. [21]

    Amp: Adversarial motion priors for stylized physics-based character control,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character control,” ACM Transactions on Graphics (ToG), vol. 40, no. 4, pp. 1–20, 2021

    2021

  22. [22]

    Learning agile skills via adversarial imitation of rough partial demonstrations,

    C. Li, M. Vlastelica, S. Blaes, J. Frey, F. Grimminger, and G. Mar- tius, “Learning agile skills via adversarial imitation of rough partial demonstrations,” inConference on Robot Learning. PMLR, 2023, pp. 342–352

    2023

  23. [23]

    Learning spring mass locomotion: Guiding policies with a reduced-order model,

    K. Green, Y . Godse, J. Dao, R. L. Hatton, A. Fern, and J. Hurst, “Learning spring mass locomotion: Guiding policies with a reduced-order model,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3926–3932, 2021

    2021

  24. [24]

    Optimizing bipedal maneuvers of single rigid-body models for reinforcement learning,

    R. Batke, F. Yu, J. Dao, J. Hurst, R. L. Hatton, A. Fern, and K. Green, “Optimizing bipedal maneuvers of single rigid-body models for reinforcement learning,” in2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids). IEEE, 2022, pp. 714–721

    2022

  25. [25]

    Opt-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors,

    Y . Fuchioka, Z. Xie, and M. Van de Panne, “Opt-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 5092–5098

    2023

  26. [26]

    Glide: Generalizable quadrupedal locomotion in diverse environments with a centroidal model,

    Z. Xie, X. Da, B. Babich, A. Garg, and M. v. de Panne, “Glide: Generalizable quadrupedal locomotion in diverse environments with a centroidal model,” inInternational Workshop on the Algorithmic Foundations of Robotics. Springer, 2022, pp. 523–539

    2022

  27. [27]

    Learning to brachiate via simplified model imitation,

    D. Reda, H. Y . Ling, and M. Van De Panne, “Learning to brachiate via simplified model imitation,” inACM SIGGRAPH 2022 Conference Proceedings, 2022, pp. 1–9

    2022

  28. [28]

    Reinforcement learning for reduced- order models of legged robots,

    Y .-M. Chen, H. Bui, and M. Posa, “Reinforcement learning for reduced- order models of legged robots,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 5801–5807

    2024

  29. [29]

    Adaptive tracking of a single- rigid-body character in various environments,

    T. Kwon, T. Gu, J. Ahn, and Y . Lee, “Adaptive tracking of a single- rigid-body character in various environments,” inSIGGRAPH Asia 2023 Conference Papers, 2023, pp. 1–11

    2023

  30. [30]

    E. L. Allgower and K. Georg,Numerical continuation methods: an introduction. Springer Science & Business Media, 2012, vol. 13

    2012

  31. [31]

    Learning symmetric and low-energy locomotion,

    W. Yu, G. Turk, and C. K. Liu, “Learning symmetric and low-energy locomotion,”ACM Transactions on Graphics (TOG), vol. 37, no. 4, pp. 1–12, 2018

    2018

  32. [32]

    Robot parkour learning,

    Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao, “Robot parkour learning,” inConference on Robot Learning (CoRL), 2023

    2023

  33. [33]

    Not only rewards but also constraints: Applications on legged robot locomotion,

    Y . Kim, H. Oh, J. Lee, J. Choi, G. Ji, M. Jung, D. Youm, and J. Hwangbo, “Not only rewards but also constraints: Applications on legged robot locomotion,”IEEE Transactions on Robotics, 2024

    2024

  34. [34]

    A learning framework for diverse legged robot locomotion using barrier-based style rewards,

    G. Kim, Y .-H. Lee, and H.-W. Park, “A learning framework for diverse legged robot locomotion using barrier-based style rewards,”arXiv preprint arXiv:2409.15780, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    Imitating and finetuning model predictive control for robust and symmetric quadrupedal locomotion,

    D. Youm, H. Jung, H. Kim, J. Hwangbo, H.-W. Park, and S. Ha, “Imitating and finetuning model predictive control for robust and symmetric quadrupedal locomotion,”IEEE Robotics and Automation Letters, 2023

    2023

  36. [36]

    Probabilistic homotopy optimization for dynamic motion planning,

    S. Pardis, M. Chignoli, and S. Kim, “Probabilistic homotopy optimization for dynamic motion planning,”arXiv preprint arXiv:2408.12490, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    Generating families of optimally actuated gaits from a legged system’s energetically conservative dynam- ics,

    M. Raff, N. Rosa, and C. D. Remy, “Generating families of optimally actuated gaits from a legged system’s energetically conservative dynam- ics,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 8866–8872

    2022

  38. [38]

    A survey on curriculum learning,

    X. Wang, Y . Chen, and W. Zhu, “A survey on curriculum learning,”IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 9, pp. 4555–4576, 2021

    2021

  39. [39]

    Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,

    G. Ji, J. Mun, H. Kim, and J. Hwangbo, “Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4630–4637, 2022

    2022

  40. [40]

    Shin, T.-G

    Y .-H. Shin, T.-G. Song, G. Ji, and H.-W. Park, “Reinforcement learning for high-speed quadrupedal locomotion with motor operating region constraints: Mitigating motor model discrepancies through torque clipping in realistic motor operating region,”IEEE Robotics & Automation Magazine, 2024. APPENDIX A. Motion Task Specifications This appendix provides the...

    2024

  41. [41]

    Gait Parameters:Gait motions are defined by periodic stance and swing phases, each with a duration of 0.2 s. The nominal swing trajectory is defined by three control points: the current foot position in the local frame, a target position determined by the Raibert heuristic, and a midpoint with a clearance of 0.1 m

  42. [42]

    Key air phase timings are summarized in Table II

    Flip Motion Parameters:Flips are composed of takeoff, air, and landing phases over a total duration of 2 s. Key air phase timings are summarized in Table II. The nominal swing trajectory is parameterized as a Bezier curve with three control points: the current foot position, (0,0,−0.2) and (0,0,−0.3) in the local frame. TABLE II: Contact Plan Parameters f...

  43. [43]

    The contact plan is illustrated in Fig

    Wall-Assisted Motion Parameters:Wall-assisted motions are designed as a series of bounding gaits over a 2 s duration. The contact plan is illustrated in Fig. 8. Each motion is guided by keyframe position and orientation targets that vary across phases. The targets for the wall-assisted turn are summarized in Table III. Other wall-assisted motions can be e...