Pith. sign in

REVIEW 4 major objections 6 minor 23 references

This paper argues that injecting a 6D wrench into the action space of legged-robot reinforcement learning makes exploration efficient enough to learn diverse tasks with shared rewards, and that a success-rate-driven curriculum can fade the

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:12 UTC pith:NLMOS3KV

load-bearing objection WARL shows a plausible trick for legged RL exploration, but it never actually demonstrates the wrench can be removed—every quantitative success happens under a residual wrench. the 4 major comments →

arxiv 2607.24036 v1 pith:NLMOS3KV submitted 2026-07-27 cs.RO

WARL: Wrench-Augmented Reinforcement Learning for Task-Agnostic Learning in Legged Robots

Keita Yoneda , Kento Kawaharazuka , Kei Okada This is my paper
classification cs.RO
keywords reinforcement learninglegged robotsaction space designwrenchcurriculum learningexplorationquadruped locomotionsim-to-real transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Typical reinforcement learning for legged robots lets the policy act only through joint commands, and the complex, contact-dependent mapping from joints to body motion makes exploration slow and fragile. WARL instead adds a six-dimensional force-and-torque wrench applied directly to the torso as an exploratory action, so that Gaussian noise moves the body through space immediately. A Switching Curriculum alternates between training with the wrench and re-adapting the joint policy under progressively attenuated saved wrench sequences, ending with a policy that never uses the wrench. The paper reports that this recipe learns all six tested tasks—hurdle jump, ledge jump, gap leap, island traverse, backflip, barrel roll—on two different quadruped designs with a shared reward per task family and no terrain curriculum. It also reports a trade-off: wrench-driven exploration can bypass the robot's body, as in the island task where the learned motion skips usable footholds and jumps the whole gap.

Core claim

The paper's central discovery is that the action space is the primary bottleneck in legged-robot reinforcement learning, and that a wrench—an abstraction of the net force and torque acting on the torso—is an action representation that collapses the gap between the action space and the task space. Directly applying policy-generated forces and torques to the torso allows early exploration to range tens of meters instead of about one meter, and the learned motions are then transferred to a joint-only policy through a curriculum. The strength of the claim is demonstrated by solving six tasks that normally demand per-task reward shaping or terrain curricula, using one shared reward per task famil

What carries the argument

The key machinery is the wrench-augmented action space plus the Switching Curriculum. The wrench policy outputs a 7-dimensional command—three force components, three torque components, and a scale factor—which is clipped, scaled by a curriculum coefficient α, and applied to the torso in the global frame; the joint policy outputs target joint angles. The Switching Curriculum alternates between a Joint+Wrench phase, where both policies learn and successful wrench sequences are stored, and a Joint-only Adaptation phase, where the frozen wrench is replayed as a decaying median of saved sequences while only the joint policy updates. The curriculum coefficient starts at α=1 and is multiplied by σ=

Load-bearing premise

The load-bearing premise is that a motion discovered under an artificially large torso wrench (up to 1000 N and 100 Nm on an 18 kg robot) can be gradually reproduced by the robot's own joints, so that replaying attenuated wrench sequences and eventually removing the wrench yields a joint-only policy that still performs the motion.

What would settle it

Record the wrench sequences used in the Joint+Wrench phase and compute, for the final joint-only policy (α≈0), the torso acceleration it must create; check whether that acceleration can be produced by ground-reaction forces within the 24 Nm joint-torque limits. A cheaper, directly observable test: rerun the IslandTraverse task with the wrench scale reduced from (1000 N, 100 Nm) to a value matching feasible leg wrench—if the robot then uses the two intermediate footholds instead of jumping the entire 1.4 m gap, the paper's trade-off claim is confirmed and its embodiment concern is concrete.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If WARL works as reported, a single reward function per task family and a fixed terrain are enough to learn jump, gap, and dynamic acrobatic tasks on a quadruped, removing much of the per-task reward engineering and terrain-curriculum design that currently dominates legged-robot RL.
  • The learned joint-only policy is a candidate for real-robot deployment: it is distilled into a proprioceptive student policy that reproduces hurdle, ledge, and gap motions in a second simulator, so the wrench is an exploration aid, not a runtime input.
  • The Switching Curriculum is not a minor add-on: the ablation shows that omitting it leaves a wrench-dependent policy with inconsistent success, while the baseline without any wrench stays near zero success on the hurdle task.
  • The 'No wrench' reward penalty measurably reduces wrench dependence and shrinks the success-rate drop at each phase switch, giving practitioners a cheap lever to control the exploration/exploitation balance.
  • Wrench-based exploration has a documented failure mode: in the island-traverse task the robot learns to jump the full 1.4 m gap instead of using the two 0.2 m footholds, so spatial exploration gains can come at the cost of embodiment-aware, terrain-utilizing behaviors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural calibration rule implied by the paper's own trade-off: set the wrench scale to the maximum wrench the legs can actually produce from ground-reaction forces (roughly joint-torque limit × stance geometry), rather than the arbitrary 1000 N/100 Nm used here; one testable prediction is that a smaller, embodiment-consistent scale would restore foothold use in IslandTraverse while keeping most
  • The wrench can be read as a body-independent action representation, and the paper's single-embodiment experiments leave open a testable transfer experiment: train a wrench policy once on one quadruped and initialize the joint policy of the other with the replayed wrench sequences, checking whether the saved wrench knowledge transfers across morphologies.
  • Because the wrench is applied in global coordinates using torso orientation, the method is not tied to the quadruped morphology; the same exploration-plus-decay recipe could be tried on bipedal or multi-legged systems, but the embodiment trade-off is likely to be more severe for underactuated systems with fewer legs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes WARL, a reinforcement-learning framework for legged robots that augments the action space with an external torso wrench (force and torque) to improve exploration. A Switching Curriculum alternates between a Joint+Wrench phase, in which both a wrench policy and a joint policy are trained, and a Joint-only Adaptation phase, in which saved wrench sequences are replayed with an exponentially decaying coefficient. The authors claim that this yields a final joint-only policy that succeeds on four goal-reaching tasks (HurdleJump, LedgeJump, GapLeap, IslandTraverse) and two dynamic tasks (Backflip, Barrel-Roll) using the same per-family reward and no terrain curriculum, on two quadruped models in Isaac Gym. An ablation on HurdleJump supports the role of the Switching Curriculum and wrench reward, and qualitative MuJoCo snapshots are shown for three tasks after teacher-student distillation.

Significance. If the central claim holds, WARL would be a useful contribution to legged-robot RL: it would show that a directly applied wrench action space plus a success-rate-based curriculum can substantially reduce task-specific reward and curriculum engineering, and the paper honestly identifies an important failure mode (wrench-driven behaviors that bypass the embodiment). The paper also ships a multi-task, multi-robot simulation study and an ablation, which is a reasonable amount of evidence for a first report. However, the current evidence does not fully establish the central zero-wrench claim: no success rates are reported with the wrench disabled in the final policy, and the only zero-wrench demonstration is qualitative. The proposed method is nevertheless defensible as a framework, and the missing evidence is within the scope of a revision.

major comments (4)
  1. [Alg. 1, lines 13–18; Sec. III-C] The Switching Curriculum's phase-success criterion ζ is evaluated in an environment that still applies the attenuated saved wrench σ w_t (line 16). A phase can therefore terminate with a high ζ while the joint policy still depends on residual wrench assistance. The paper's central claim is that WARL 'ultimately' acquires a joint-only policy, but no zero-wrench success rate is reported for any of the six tasks. The only zero-wrench evidence is qualitative MuJoCo snapshots in Sec. V-C for HurdleJump, LedgeJump, and GapLeap, and the student policy is not quantitatively evaluated. Please report for all six tasks the success rate of the final π_j with the wrench fully disabled, and the same for the distilled student π_s. This is the load-bearing test of Contribution 1.
  2. [Sec. V-A; Sec. VI-a] The IslandTraverse result is direct evidence that wrench-based exploration can produce motions that are not embodiment-feasible: the authors report that the policy 'directly jumped over the 1.4 m gap without using footholds' because wrench-based exploration dominated. Given the wrench scale (1000 N, 100 Nm) far exceeds the joint torque limits (24 Nm), this is a plausible failure mode in any task. The authors acknowledge the trade-off, but they do not quantify how often such exploitation occurs or whether the final joint-only policy actually reproduces the reported successes. A concrete test would be to evaluate the final zero-wrench policy's motion against an inverse-dynamics or joint-torque feasibility check, or at least to report the wrench-norm and contact-force statistics on successful episodes for each task.
  3. [Sec. V-B] The ablation study is performed only on HurdleJump, and the success-rate curves shown are only for this single task. Since the paper's main contribution is 'versatile learning' across tasks and robots, the ablation should be reported on at least a subset of the other tasks (e.g., GapLeap and Backflip) to show that the Switching Curriculum is not task-specific. In addition, the related-work section explicitly contrasts WARL with EFGCL [18], ZEST [19], and A2CF [20], but none of these methods is compared empirically. Without such a comparison, the claim that wrench-guided exploration is more effective than prior wrench-based assistance is not substantiated.
  4. [Sec. IV-D] The choice of (f_max, τ_max) = (1000 N, 100 Nm) is justified only by the exploratory experiment in Sec. II-B. A wrench of this magnitude is considerably larger than what the robot's joints can generate (max joint torque 24 Nm), which makes it unsurprising that wrench-based exploration dominates. No sensitivity analysis is provided for this scale, although it is a free parameter of the method. A sweep over smaller wrench scales, or a reasoning about why this scale is necessary for the dynamic tasks, would strengthen the claim that the method is not relying on an unrealistically large external force.
minor comments (6)
  1. [Abstract] Typo: 'wrenche' should be 'wrench'.
  2. [Fig. 4] The caption says 'Transition of rewards' but the text describes learning curves; please clarify whether the y-axis is episode reward or success rate, as the same figure includes 'reward' in the caption.
  3. [Sec. V-C] There is a text-encoding artifact: 'Teacher ˆa€“Student' should read 'Teacher–Student'. Please fix.
  4. [Table II and Table III] The 'No wrench' reward term is actually a wrench-usage penalty; the name is confusing and might be better rendered as 'Wrench penalty' or 'No-wrench preference'.
  5. [Sec. V-B, Fig. 7] The legend and text refer to 'success rate' but the axis is not labeled in the figure; adding a y-axis label would improve readability.
  6. [References] Some references are dated 2026 (e.g., [18], [19]). If these are preprints, please include the arXiv identifiers and access dates in the bibliography.

Circularity Check

0 steps flagged

No significant circularity: WARL's claims are empirical, and its self-citations are non-load-bearing platform/reference citations.

full rationale

This paper does not present a formal derivation chain that could reduce to its inputs. Its central claims are empirical: wrench-augmented exploration improves learning efficiency, the Switching Curriculum removes wrench dependence, and the resulting policy generalizes across six tasks. Each of these is tested through independent simulation experiments, learning curves, ablations, and MuJoCo execution snapshots. The only self-citations are [10] (the KLEIYN robot platform) and [18] (the authors' prior EFGCL method), and neither is load-bearing: the robot is an experimental testbed, not a premise that defines the result, and EFGCL is cited as related prior work rather than as the justification for WARL's claims. The wrench-scale parameters (1000 N, 100 Nm) are selected from the paper's own exploratory experiment in Sec. II-B, which is a design choice, not a prediction derived from the target results. The skeptical concern that curriculum success is measured under an attenuated wrench rather than at exactly w=0 is an empirical validation gap: the paper itself flags the embodiment trade-off in Sec. VI and the IslandTraverse result in Sec. V-A, acknowledging that wrench-driven motions may not be embodiment-feasible. That is honest limitation reporting, not circular reasoning. No equation or fitted parameter is renamed as a prediction, no uniqueness theorem is imported from self-citations, and no derivation is equivalent to its inputs by construction. Therefore the appropriate finding is no significant circularity, score 0.

Axiom & Free-Parameter Ledger

8 free parameters · 4 axioms · 1 invented entities

The central claim rests on several domain assumptions about wrench-based exploration being a valid proxy and about curriculum thresholds and reward weights being universally applicable; none of these are derived or validated on hardware. Free parameters are hand-selected and could affect the reported six-task success.

free parameters (8)
  • f_max, tau_max = 1000 N, 100 Nm
    Selected from the random-exploration experiment in Fig. 2 (Sec. IV-D); defines the strength of wrench exploration that the curriculum must eventually remove.
  • lambda_wrench = 0.5
    Fixed scaling factor for wrench actions before clipping (Sec. III-B).
  • gamma, delta, sigma, epsilon = 0.6, 2e-6, 0.8, 0.01
    Success threshold, success-change threshold, decay coefficient, and termination threshold for the Switching Curriculum (Sec. IV-D).
  • N_succ, eta, kappa = 100, 0.9, 100.0
    Buffer size, median update weight, and force-torque balance factor for saving wrench sequences (Sec. III-D).
  • reward weights = See Tables II-III
    Hand-set weights for target position, stand pose, stand DOF, angular velocity, collision, and no-wrench penalties.
  • success criteria = |p-p*|<0.2 m, |theta|<45 deg (goal); |theta|<22.5 deg, |dz|<0.1 m (dynamic)
    Define what counts as success and drive the success-rate curriculum.
  • standing height z* = 0.43 m KLEIYN, 0.3 m Go2
    Per-robot normalization, claimed as the only robot-specific tuning.
  • w_cur = 0.01 * 0.9999^i
    Iterative curriculum weight for angular velocity, collision, and no-wrench penalties.
axioms (4)
  • domain assumption A wrench applied to the torso link is a legitimate exploratory action whose effects can be annealed to zero without destroying learned behavior.
    Entire Switching Curriculum (Sec. III-C) depends on this; Sec. V-A's IslandTraverse result—skipping footholds—shows wrench-generated motions can be unphysical for the embodiment.
  • ad hoc to paper Success-rate thresholds (gamma=0.6, |Delta zeta|<delta) reliably indicate that a skill is learned and it is safe to reduce wrench.
    Threshold values are chosen in Sec. IV-D; no sensitivity analysis is provided, so the curriculum switching schedule is assumed robust to these numbers.
  • domain assumption The common reward across tasks is sufficient to encode the goal; no task-specific shaping is required.
    The 'task-agnostic' claim rests on this; the reward tables still use task-specific terms (target position/orientation, dynamic-task angle) and success thresholds (Sec. IV-C).
  • domain assumption Teacher-student distillation from privileged to proprioceptive observations preserves the wrench-learned policy's performance.
    Sec. III-E and V-C rely on standard sim-to-sim distillation; no hardware validation or distillation-loss details are given.
invented entities (1)
  • External torso wrench (f, tau) applied in simulation no independent evidence
    purpose: Augment the action space for exploration; later attenuated via curriculum to yield a joint-only policy
    A wrench is not a physical actuator available on KLEIYN or Go2; it exists only as a simulated force on the torso link (Sec. III-B). Real-robot applicability requires the joint-only policy to obviate it, which is not validated on hardware.

pith-pipeline@v1.3.0-alltime-deepseek · 10333 in / 13298 out tokens · 112510 ms · 2026-07-31T23:12:10.690347+00:00 · methodology

0 comments
read the original abstract

While reinforcement learning for legged robots has achieved high motor performance, it has been constrained by the limited exploration capability of actions confined to the joint space. To address this issue, this study proposes a new method, Wrench-Augmented Reinforcement Learning (WARL), which introduces a wrenche (force and torque) into the action space. The proposed method combines wrench-guided exploration with a success rate-based curriculum mechanism to expand exploration capabilities in the early stages of learning, with the ultimate goal of acquiring behaviors based solely on joint control. Experiments using a quadruped robot demonstrated that WARL can learn robustly across diverse terrains and motor tasks without requiring terrain-specific reward adjustments or complex curriculum designs. Furthermore, an ablation study verified the effectiveness of the Switching Curriculum, which gradually eliminates the wrench. On the other hand, we also show that introducing a wrench can encourage behaviors that do not sufficiently exploit the robot's physical embodiment. These findings suggest that while wrench-based exploration enhancement is effective for improving learning efficiency, designing it in a way that is consistent with the robot's physical structure is a critical future challenge.

Figures

Figures reproduced from arXiv: 2607.24036 by Kei Okada, Keita Yoneda, Kento Kawaharazuka.

Figure 1
Figure 1. Figure 1: Overview of Wrench-Augmented Reinforcement Learning (WARL). [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Learning flow of WARL. Joint+Wrench learning promotes exploration and stores successful wrench sequences. Joint-only adaptation reduces wrench dependence using a decayed coefficient α. ration while ultimately acquiring a joint-only control policy. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Learned wrench and joint actions in HurdleJump at iteration 500. Wrench assistance is applied to clear the obstacle, while joint control is coordinated to lift the feet. 1. joint+wrench learning 1. joint+wrench learning 2. joint-only adaptation |𝒘| = 0 No Wrench Phase Success Rate and Curriculum Weight in the HurdleJump Task [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: [a-f]-1: Transition of rewards when learning six tasks with KLEIYN [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 7
Figure 7. Figure 7: Results of the ablation study of the proposed method on the [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Snapshots of the learned policy executed on the MuJoCo simulator. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 5 linked inside Pith

  1. [1]

    Learning agile and dynamic motor skills for legged robots,

    J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,”Science Robotics, vol. 4, no. 26, 2019

    2019

  2. [2]

    Robot parkour learning,

    Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao, “Robot parkour learning,”arXiv preprint arXiv:2309.05665, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    High-speed control and navigation for quadrupedal robots on complex and discrete terrain,

    H. Kim, H. Oh, J. Park, Y . Kim, D. Youm, M. Jung, M. Lee, and J. Hwangbo, “High-speed control and navigation for quadrupedal robots on complex and discrete terrain,”Science Robotics, vol. 10, no. 102, p. eads6192, 2025

    2025

  4. [4]

    Learning to walk in minutes using massively parallel deep reinforcement learning,

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in Proceedings of the 2022 Conference on Robot Learning, 2022, pp. 91–100

    2022

  5. [5]

    Curriculum-based reinforcement learning for quadrupedal jumping: A reference-free design,

    V . Atanassov, J. Ding, J. Kober, I. Havoutis, and C. Della Santina, “Curriculum-based reinforcement learning for quadrupedal jumping: A reference-free design,”IEEE Robotics&Automation Magazine, 2024

    2024

  6. [6]

    Rapid locomotion via reinforcement learning,

    G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal, “Rapid locomotion via reinforcement learning,”The International Journal of Robotics Research, vol. 43, no. 4, pp. 572–587, 2024

    2024

  7. [7]

    Learning-based legged locomotion: State of the art and future per- spectives,

    S. Ha, J. Lee, M. van de Panne, Z. Xie, W. Yu, and M. Khadiv, “Learning-based legged locomotion: State of the art and future per- spectives,”The International Journal of Robotics Research, vol. 44, no. 8, pp. 1396–1427, 2025

    2025

  8. [8]

    Robust ladder climbing with a quadrupedal robot,

    D. V ogel, R. Baines, J. Church, J. Lotzer, K. Werner, and M. Hutter, “Robust ladder climbing with a quadrupedal robot,”arXiv preprint arXiv:2409.17731, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Robust quadruped jumping via deep reinforcement learning,

    G. Bellegarda, C. Nguyen, and Q. Nguyen, “Robust quadruped jumping via deep reinforcement learning,”Robotics and Autonomous Systems, vol. 182, p. 104799, 2024

    2024

  10. [10]

    Kleiyn: A quadruped robot with an active waist for both locomotion and wall climbing,

    K. Yoneda, K. Kawaharazuka, T. Suzuki, T. Hattori, and K. Okada, “Kleiyn: A quadruped robot with an active waist for both locomotion and wall climbing,”arXiv preprint arXiv:2507.06562, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks,

    R. Mart ´ın-Mart´ın, M. A. Lee, R. Gardner, S. Savarese, J. Bohg, and A. Garg, “Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks,” in2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2019, pp. 1010–1017

    2019

  12. [12]

    Learning whole-body loco-manipulation for omni-directional task space pose tracking with a wheeled-quadrupedal-manipulator,

    K. Jiang, Z. Fu, J. Guo, W. Zhang, and H. Chen, “Learning whole-body loco-manipulation for omni-directional task space pose tracking with a wheeled-quadrupedal-manipulator,”IEEE Robotics and Automation Letters, vol. 10, no. 2, pp. 1481–1488, 2024

    2024

  13. [13]

    On the role of the action space in robot manipulation learning and sim-to-real transfer,

    E. Aljalbout, F. Frank, M. Karl, and P. van der Smagt, “On the role of the action space in robot manipulation learning and sim-to-real transfer,”IEEE Robotics and Automation Letters, vol. 9, no. 6, pp. 5895–5902, 2024

    2024

  14. [14]

    Learning task space actions for bipedal locomotion,

    H. Duan, J. Dao, K. Green, T. Apgar, A. Fern, and J. Hurst, “Learning task space actions for bipedal locomotion,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 1276–1282

    2021

  15. [15]

    Cpg-rl: Learning central pattern generators for quadruped locomotion,

    G. Bellegarda and A. Ijspeert, “Cpg-rl: Learning central pattern generators for quadruped locomotion,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 12 547–12 554, 2022

    2022

  16. [16]

    Dy- namic policy learning for legged robot with simplified model pretrain- ing and model homotopy transfer,

    D. Kang, M.-G. Kim, T.-G. Song, H. Kim, S. Ha, and H.-W. Park, “Dy- namic policy learning for legged robot with simplified model pretrain- ing and model homotopy transfer,”arXiv preprint arXiv:2512.24698, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Learning robust perceptive locomotion for quadrupedal robots in the wild,

    T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,”Science Robotics, vol. 7, no. 62, 2022

    2022

  18. [18]

    Efgcl: Learning dy- namic motion through spotting-inspired external force-guided curricu- lum learning,

    K. Yoneda, K. Kawaharazuka, and K. Okada, “Efgcl: Learning dy- namic motion through spotting-inspired external force-guided curricu- lum learning,”IEEE Robotics and Automation Letters, vol. 11, no. 5, pp. 5907–5913, 2026

    2026

  19. [19]

    Zest: Zero-shot embodied skill transfer for athletic robot control,

    J. P. Sleiman, H. Li, A. Adu-Bredu, R. Deits, A. Kumar, K. Bergamin, M. Bhardwaj, S. Biddlestone, N. Burger, M. A. Estradaet al., “Zest: Zero-shot embodied skill transfer for athletic robot control,”arXiv preprint arXiv:2602.00401, 2026

    arXiv 2026

  20. [20]

    Learning motion skills with adaptive assistive curriculum force in humanoid robots,

    Z. Cao, Y . Zhang, B. Nie, H. Lin, H. Li, and Y . Gao, “Learning motion skills with adaptive assistive curriculum force in humanoid robots,” arXiv preprint arXiv:2506.23125, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017

    2017

  22. [22]

    Gpu-accelerated robotic simulation for distributed reinforce- ment learning,

    J. Liang, V . Makoviychuk, A. Handa, N. Chentanez, M. Macklin, and D. Fox, “Gpu-accelerated robotic simulation for distributed reinforce- ment learning,” 2018

    2018

  23. [23]

    MuJoCo: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “MuJoCo: A physics engine for model-based control,” inProceedings of the 2012 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems, 2012, pp. 5026– 5033

    2012