Jump-Start Reinforcement Learning with Self-Evolving Priors for Extreme Monopedal Locomotion

Guojian Zhan; Shengbo Eben Li; Shiqi Liu; Tao Zhang; Yao Lyu; Ziang Zheng

REVIEW 4 major objections 4 minor 29 references

Jump-Start Reinforcement Learning with Self-Evolving Priors for Extreme Monopedal Locomotion

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a self-bootstrapping RL curriculum, JumpER, lets a simulated quadruped hop on one leg over 60 cm gaps, irregular stairs, and stepping stones spaced 15–35 cm apart.

desk verdict A solid simulation demonstration of monopedal hopping on extreme terrain, but the 'no handcrafted reward shaping' claim is contradicted by Table I and the real-world claim needs hardware evidence. read the letter →

arxiv 2507.01243 v1 pith:MAO2B6ZE submitted 2025-07-01 cs.RO cs.LG

Ziang Zheng , Guojian Zhan , Shiqi Liu , Yao Lyu , Tao Zhang , Shengbo Eben Li This is my paper

classification cs.ROcs.LG

keywords reinforcementlearningmonopedalhoppingquadrupedlocomotionjump-startself-evolvingpriorscurriculumextremeterrainunderactuated

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a quadruped can be trained to hop across extreme terrain while using only one leg for contact, a regime of extreme underactuation where ordinary reinforcement learning collapses because early rollouts are unstable and rewards are sparse. The proposed method, JumpER, attacks this by bootstrapping: at each stage, the policy learned in the previous stage is frozen and used as a guide that jump-starts the next policy, so the robot generates its own curriculum without expert demonstrations or handcrafted reward shaping. In simulation, the final policy hops over gaps up to 60 cm, irregular stairs, and stepping stones spaced 15–35 cm apart, outperforming vanilla and pretrained PPO baselines on terrain success. The authors claim this is the first robust monopedal hopping of a quadruped on unpredictable terrain; whether the simulated policy transfers to physical hardware remains untested.

What carries the argument

The mechanism is the self-evolving prior: at each training stage $i$, the policy $\pi^{(i-1)}$ from the previous stage is frozen and used as the guidance policy $\pi_g^{(i)}$, and a new policy $\pi_\theta^{(i)}$ is trained by first following the guide for the first $h$ steps (in practice, for a number of rollout patches that is annealed down over training) and then acting on its own. This converts the jump-start RL idea into a curriculum that needs no external expert: the guide only needs to be slightly competent on the previous task, and its short successful segments give the learner the exploration signal that sparse rewards otherwise hide. Two practical techniques support the chain: gradients are computed over many short parallel rollout patches rather than full episodes, and the number of guide patches is annealed via $n_t = \max(0, n_0 - \lfloor t/m \rfloor)$. The three-stage curriculum around this mechanism—modality, observation, and objective—is what makes each prior a meaningful stepping stone.

What would settle it

Put the final policy on a physical quadruped and set it before a 60 cm gap with the requirement that only the designated leg touch the ground; if the robot cannot consistently clear the gap without another leg or body part contacting the ground, the paper's claim of robust monopedal hopping on unpredictable terrains is refuted. For a purely simulated falsification, remove the stage-by-stage bootstrap and train the goal-reaching policy directly from a random policy: if it matches JumpER's success rate, the proposed prior mechanism is not the cause of the reported gains.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the dual challenge of extreme underactuation (one-legged hopping) and extreme terrain can be broken by a three-stage curriculum: a modality transform that turns a bipedal standing prior into a monopedal hopping policy, an observation transform that adds terrain heightmaps to proprioception, and an objective transform that switches from velocity tracking to goal reaching. At each stage the previous policy, frozen, acts as the guidance policy for the first portion of each rollout, and control then transfers to the new learning policy; the guided portion is annealed away as training progresses. The paper reports that this chain yields high success rates on wide gaps, stairs, and stepping stones where vanilla PPO, pretrained PPO, and dense-reward baselines plateau or fail, and it claims to be the first demonstration of robust monopedal hopping of a quadruped on unpredictable terrain.

Load-bearing premise

The load-bearing premise is that the physics simulation used for all experiments faithfully reproduces the contact-rich, underactuated dynamics of a real quadruped, so a policy trained in simulation will transfer to hardware; the conclusion asserts practical utility without any real-robot test.

Editorial extensions

If this is right

A quadruped with only one functional leg could still be trained to reach a goal location, giving a concrete fallback behavior for leg failure or severe mechanical damage.
The three-stage transform recipe—action modality, observation space, and task objective—can be applied to other sparse-reward underactuated tasks without handcrafted shaping terms.
The guided-rollout bootstrapping supplies the missing exploration signal in early training, so tasks that previously required dense rewards or expert priors become trainable from scratch with an internally generated prior.
Emergent behaviors reported by the authors, such as trunk tilting before a jump and mid-air posture adjustment, indicate that the curriculum discovers motor strategies that are not explicitly encoded in the reward.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial: the same bootstrap-annealing scheme could be tried on other underactuated balance problems, for example a biped with one locked knee or a quadrotor losing one rotor, where a weak intermediate policy exists but no expert final policy does.
Editorial: the paper's simulation-only evidence leaves the hardware question open; an immediate extension is to repeat the training with domain randomization and measure the sim-to-real gap on the same gap and stepping-stone courses.
Editorial: removing the annealing schedule and keeping a fixed number of guide patches throughout training would show how much of the reported gain comes from the self-evolving prior chain versus the scheduling trick.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing you should know: this is a simulation-only monopedal hopping result that bundles curriculum learning with jump-start RL, and the empirical win over baselines is large and consistent. The problem is real—quadruped with a single functioning leg on gaps and stepping stones—and I don't know of prior work that trains this exact capability. The staged curriculum is sensible: first single-leg balance on flat ground, then add terrain observations, then switch from velocity tracking to goal reaching. The patch-level scheduling is a nice engineering touch. The baselines are fairly chosen (vanilla PPO, pretrained PPO, dense-reward variants), and the 5-run mean/std reporting is honest.

Where it gets soft: the abstract says 'without relying on external expert priors or handcrafted reward shaping,' but Table I is full of shaping: R_out_platform, R_reach_far, posture penalties, contact penalties. Even if you argue these are task-agnostic, they are handcrafted. That overclaim should be fixed. More important, there is no ablation isolating the self-evolving prior from the staged curriculum. The staged curriculum alone might be doing most of the work, and we need that experiment to believe the method's core claim. The simulation-only status is also a real limitation: no domain randomization, no hardware, and the termination condition (any non-designated ground contact ends the episode) is a clean contact oracle you don't have on a physical robot. Yet the conclusion says 'practical utility in real-world applications.' That's unsupported. Missing citations to kickstarting and progressive networks are a minor issue; the self-evolving prior is a close relative.

Bottom line: this deserves a serious referee, but the revision should tone down the shaping claim, add an ablation, and either cut the real-world sentence or add hardware tests. For someone in legged locomotion, it's a useful reference; for a general RL audience, the method is incremental. I'd accept it for peer review but expect heavy revision.

Referee Report

4 major / 4 minor

Summary. The paper proposes JumpER, a multi-stage jump-start reinforcement learning framework in which the guidance policy for each stage is the policy learned in the previous stage (a 'self-evolving prior'). This is combined with a three-stage curriculum that changes the action modality (bipedal to monopedal), the observation space (proprioception to terrain-aware), and the task objective (velocity tracking to goal reaching). The method is evaluated in IsaacLab on a quadruped required to hop on a single leg over flat, rough, sloped, stair, wide-gap, and stepping-stone terrains. The authors report that JumpER outperforms vanilla PPO and pretrained-PPO baselines on most metrics, and they conclude that the approach enables robust monopedal hopping on extreme terrains, with gap crossings up to 60 cm and stepping-stone distances from 15 to 35 cm.

Significance. If the claims were fully supported, the paper would describe a notable empirical advance: stable monopedal hopping on discontinuous, underactuated terrain is a genuinely difficult locomotion problem, and the staged curriculum plus self-generated guidance is a plausible recipe for tackling sparse-reward regimes. The paper includes useful positive features: a clearly described curriculum, mean-plus-minus-standard-deviation results over five runs, several baselines including dense-reward variants, and an anonymous code link. However, the central claims currently outrun the evidence in three ways: all results are simulation-only, the contribution of the self-evolving prior is not isolated from the curriculum, and the description of the reward design is internally inconsistent with the stated 'no handcrafted reward shaping' claim. The paper is therefore best viewed as a promising method paper whose core empirical assertions still need either additional experiments or careful restatement.

major comments (4)

[Section IV-A, Section V] All capability results are generated in IsaacLab without any hardware validation, domain randomization, actuator latency or torque limits, sensor noise, or friction/restitution variation. The conclusion states that the results indicate 'practical utility in real-world applications,' but for a contact-rich, underactuated task such as monopedal hopping, the discrepancy between simulation and reality is exactly where these unmodeled quantities matter most. The training termination condition also provides a clean contact oracle ('any contact with any non-designated leg terminates the episode') that is not directly observable on real hardware. Please either add hardware experiments or a detailed sim-to-real analysis, or revise the abstract and conclusion to state that the demonstrated capability is currently simulation-only.
[Section III, Table II] The paper does not include an ablation that separates the effect of the self-evolving prior from the effect of the staged curriculum. In Table II, JumpER is always the full combination of the three-stage curriculum and self-evolving guidance, while the baselines are either trained from scratch, pretrained once, or fine-tuned with dense rewards. To support the central claim that iterative bootstrapping of the previous policy is what stabilizes training, the authors should compare JumpER against (a) the same three-stage curriculum with a fixed external guidance policy, and (b) the same three-stage curriculum with no jump-start guidance at all. Without these controls, the reported gains could be attributed entirely to the staged task progression.
[Section I, Section III-B, Table I] The abstract and introduction claim that JumpER works 'without relying on external expert priors or handcrafted reward shaping,' but this is contradicted by the method as described. Stage 1 explicitly begins from a 'stable bipedal standing policy [3]' as the first guidance policy, which is an externally trained prior. In addition, Table I contains dense reward terms such as 'R_out platform 1{|x|>3}10.0' and 'R_reach far exp(-||x||)-0.5,' and Section IV-A states that dense shaping rewards are adopted as a benchmark. Please clarify which reward terms are actually used in each stage of JumpER, which are used only in the baseline comparisons, and whether the 'no handcrafted reward shaping' claim refers to a specific sparse-reward variant. If JumpER itself uses the dense terms, the claim should be revised.
[Section IV-D, Section V] The paper repeatedly describes the terrains as 'unpredictable' and 'robust,' but the evaluation is performed on the same randomized training curriculum used for learning, not on held-out terrain distributions or out-of-distribution configurations. A policy can have high success on the training distribution without generalizing to genuinely unseen terrain parameters. For the robustness and generalization claims to be supported, the authors should evaluate on held-out terrain seeds, unseen gap/stone configurations, or a separate test set, and report both mean success and diversity of terrain samples.

minor comments (4)

[Table II] In the T5 wide-gap and T6 stepping-stone rows, the JumpER entries contain a stray leading '-' before the reach-return value (e.g., '-0.80±0.06'), which appears to be a formatting artifact that should be corrected.
[Section III-C, Eq. (5)] The scheduling parameter m is described as 'the number of environment steps between scheduling updates,' but the text and Fig. 4 refer to 'training steps' and 'iterations.' Please use consistent units for n0, m, and t, and clarify how patch boundaries align with the gradient computation described in the patch-level gradient paragraph.
[Section IV-A] The description of the evaluation metrics includes a metric called 'leg-on-air reward R_Air' that measures 'distance from the target handstand height.' The task is monopedal hopping, not handstand, and the relationship between these metrics and the actual task is never defined. Please either define these metrics for the monopedal-hopping setting or remove the leftover handstand-related phrasing.
[Section III-B] Stage 1 is described as transitioning from a 'partial bipedal prior' to a monopedal policy, and the text mentions 'bipedal standing policy [3]' as the first-stage guidance. It would help to state explicitly how the bipedal policy is obtained and what 'partial bipedal' means, given that the robot is a quadruped and the task requires a single designated leg to be in contact.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the self-evolving prior is algorithmic bootstrapping, and the 60 cm gap results are empirical training outcomes, not predictions fitted by construction.

full rationale

This is an empirical reinforcement-learning paper rather than a formal derivation, so the standard circularity test—whether Eq. X reduces to Eq. Y by construction—has little purchase. The central mechanism, JumpER, defines the guidance policy at stage i as the frozen policy from stage i-1 (Eq. 4), and the three-stage curriculum transforms action modality, observation, and objective. This is a legitimate bootstrapping procedure: the 'self-evolving prior' is an algorithmic input, not a fitted constant later relabeled as a prediction. The reported capabilities (60 cm gaps, 15-35 cm stepping stones) are success rates measured on the same curriculum used for training; that is a limitation on generalization claims but not a circular argument, because the comparison baselines are trained and evaluated under identical conditions. Self-citations exist—[8] is used for terrain curriculum details and observation structure, and [21] for random-annealing jump-start—but none of these is load-bearing for the central contribution; they do not forbid alternatives or import an unverified uniqueness theorem. The main weakness is external validity: all experiments run in IsaacLab with no hardware transfer or domain randomization, so 'practical utility in real-world applications' is not established. This is a sim-to-real correctness risk, not circularity. No quoted passage exhibits a definitional equivalence or a fitted-input-called-prediction reduction, so the score reflects only the presence of minor, non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central result is empirical and depends mainly on hand-tuned reward terms, curriculum design, and simulation fidelity. No new physical entities or formal axioms are introduced.

free parameters (4)

Reward weights (Table I) = Various (e.g., termination -200, dense goal -0.5)
Hand-tuned to make training stable; the paper does not report a sensitivity analysis.
Guidance patch count n0 = 2
Number of initial guide patches per episode, chosen by hand.
Scheduling interval m = 300 iterations
Environment steps between scheduling updates, chosen by hand.
Stage count tau = 3
Number of bootstrap stages, selected for the monopedal task.

assumptions (4)

domain assumption IsaacLab/IsaacGym simulation accurately captures the contact dynamics and underactuation of a real quadruped robot.
All experiments are simulated; the conclusion asserts practical real-world utility without hardware validation.
domain assumption The previous stage policy is a reliable and safe guide for the next stage.
If the prior policy is poor, jump-starting from it could bias the learner; no analysis of this failure mode is given.
standard math PPO with GAE is a sufficiently stable optimizer for the monopedal task.
The paper builds on PPO and RSL-RL without questioning their applicability.
ad hoc to paper The three-stage curricular decomposition (modality, observation, objective) is a valid path to the final task.
This decomposition is hand-designed for monopedal hopping and is not derived from first principles.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Jump-Start Reinforcement Learning with Self-Evolving Priors for Extreme Monopedal Locomotion." pith.science (2026). https://pith.science/paper/MAO2B6ZE

@misc{pith2026250701243,
  author       = {Pith},
  title        = {Pith review of: Jump-Start Reinforcement Learning with Self-Evolving Priors for Extreme Monopedal Locomotion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MAO2B6ZE}},
  note         = {Machine review of arXiv:2507.01243}
}

read the original abstract

Reinforcement learning (RL) has shown great potential in enabling quadruped robots to perform agile locomotion. However, directly training policies to simultaneously handle dual extreme challenges, i.e., extreme underactuation and extreme terrains, as in monopedal hopping tasks, remains highly challenging due to unstable early-stage interactions and unreliable reward feedback. To address this, we propose JumpER (jump-start reinforcement learning via self-evolving priors), an RL training framework that structures policy learning into multiple stages of increasing complexity. By dynamically generating self-evolving priors through iterative bootstrapping of previously learned policies, JumpER progressively refines and enhances guidance, thereby stabilizing exploration and policy optimization without relying on external expert priors or handcrafted reward shaping. Specifically, when integrated with a structured three-stage curriculum that incrementally evolves action modality, observation space, and task objective, JumpER enables quadruped robots to achieve robust monopedal hopping on unpredictable terrains for the first time. Remarkably, the resulting policy effectively handles challenging scenarios that traditional methods struggle to conquer, including wide gaps up to 60 cm, irregularly spaced stairs, and stepping stones with distances varying from 15 cm to 35 cm. JumpER thus provides a principled and scalable approach for addressing locomotion tasks under the dual challenges of extreme underactuation and extreme terrains.

Figures

Figures reproduced from arXiv: 2507.01243 by the authors.

**Figure 2.** Overview of JumpER framework for achieving robust monopedal locomotion on extreme terrains. In the three-stage [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Overview of the training playground, which includes [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Training curves on Balancing task. Our JumpER (red) outperforms both scratch and naive pretraining baselines. The observed performance drops at 300 and 600 iterations are attributed to jump scheduling. This destabilization is promptly corrected by ongoing guidance and…

**Figure 5.** Figure 5: Visualization of monopedal hopping on extreme terrains. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 16 canonical work pages

[3]

Extreme parkour with legged robots,

X. Cheng, K. Shi, A. Agarwal, and D. Pathak, “Extreme parkour with legged robots,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 11 443–11 450

2024
[1]

Advances in real-world applications for legged robots,

C. D. Bellicoso, M. Bjelonic, L. Wellhausen, K. Holtmann, F. G ¨unther, M. Tranzatto, P. Fankhauser, and M. Hutter, “Advances in real-world applications for legged robots,”Journal of Field Robotics, vol. 35, no. 8, pp. 1311–1326, 2018

2018
[2]

Mul- timodality robotic systems: Integrated combined legged-aerial mobility for subterranean search-and-rescue,

B. Lindqvist, S. Karlsson, A. Koval, I. Tevetzidis, J. Halu ˇska, C. Kanellakis, A.-a. Agha-mohammadi, and G. Nikolakopoulos, “Mul- timodality robotic systems: Integrated combined legged-aerial mobility for subterranean search-and-rescue,”Robotics and Autonomous Sys- tems, vol. 154, p. 104134, 2022

work page 2022
[4]

Learning agile loco- motion on risky terrains,

C. Zhang, N. Rudin, D. Hoeller, and M. Hutter, “Learning agile loco- motion on risky terrains,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 11 864– 11 871

2024
[5]

S. E. Li,Reinforcement Learning for Sequential Decision and Optimal Control. Springer Verlag, Singapore, 2023

2023
[6]

Rambo: Rl-augmented model-based optimal control for whole-body loco- manipulation,

J. Cheng, D. Kang, G. Fadini, G. Shi, and S. Coros, “Rambo: Rl-augmented model-based optimal control for whole-body loco- manipulation,”arXiv preprint arXiv:2504.06662, 2025

arXiv 2025
[7]

Enhance generality by model-based reinforcement learning and domain ran- domization,

G. Zhan, Y . Lyu, S. E. Li, Y . Jiang, X. Zhang, and L. Tao, “Enhance generality by model-based reinforcement learning and domain ran- domization,” in2023 7th CAA International Conference on Vehicular Control and Intelligence (CVCI). IEEE, 2023, pp. 1–6

work page 2023
[8]

Transferable latent-to-latent locomotion policy for efficient and versatile motion control of diverse legged robots,

Z. Zheng, G. Zhan, B. Shuai, S. Qin, J. Li, T. Zhang, and S. E. Li, “Transferable latent-to-latent locomotion policy for efficient and versatile motion control of diverse legged robots,”2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025

work page 2025

Show all 29 references

[9]

Walking with terrain reconstruction: Learning to traverse risky sparse footholds,

R. Yu, Q. Wang, Y . Wang, Z. Wang, J. Wu, and Q. Zhu, “Walking with terrain reconstruction: Learning to traverse risky sparse footholds,” arXiv preprint arXiv:2409.15692, 2024

2024 arXiv
[10]

Learning robust perceptive locomotion for quadrupedal robots in the wild,

T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,”Science robotics, vol. 7, no. 62, p. eabk2822, 2022

2022
[11]

Learning quadrupedal locomotion over challenging terrain,

J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Science robotics, vol. 5, no. 47, p. eabc5986, 2020

2020
[12]

World model-based perception for visual legged locomotion,

H. Lai, J. Cao, J. Xu, H. Wu, Y . Lin, T. Kong, Y . Yu, and W. Zhang, “World model-based perception for visual legged locomotion,”arXiv preprint arXiv:2409.16784, 2024

2024 arXiv
[13]

Underactuated robotics: Learning, planning, and control for efficient and agile machines course notes for mit 6.832,

R. Tedrake, “Underactuated robotics: Learning, planning, and control for efficient and agile machines course notes for mit 6.832,”Working draft edition, vol. 3, no. 4, p. 2, 2009

2009
[14]

Task- space riccati feedback based whole body control for underactuated legged locomotion,

S. Yang, Z. Hong, S. Li, P. Wensing, W. Zhang, and H. Chen, “Task- space riccati feedback based whole body control for underactuated legged locomotion,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 6826–6831

2024
[15]

Meta-learning for fast adaptive locomotion with uncertainties in environments and robot dynamics,

T. Anne, J. Wilkinson, and Z. Li, “Meta-learning for fast adaptive locomotion with uncertainties in environments and robot dynamics,” in2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021, pp. 4568–4575

2021
[16]

Multi-task learning of active fault-tolerant controller for leg failures in quadruped robots,

T. Hou, J. Tu, X. Gao, Z. Dong, P. Zhai, and L. Zhang, “Multi-task learning of active fault-tolerant controller for leg failures in quadruped robots,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 9758–9764

2024
[17]

Towards fault-tolerant quadruped loco- motion with reinforcement learning,

D. Liu, J. Yin, and S. See, “Towards fault-tolerant quadruped loco- motion with reinforcement learning,” in2024 IEEE Conference on Artificial Intelligence (CAI). IEEE, 2024, pp. 1438–1441

2024
[18]

Acl: Action learner for fault-tolerant quadruped locomotion control,

T. Xu, Y . Cheng, P. Shen, L. Zhao, C. Engineering,et al., “Acl: Action learner for fault-tolerant quadruped locomotion control,”arXiv preprint arXiv:2503.21401, 2025

2025 arXiv
[19]

Learning agile bipedal motions on a quadrupedal robot,

Y . Li, J. Li, W. Fu, and Y . Wu, “Learning agile bipedal motions on a quadrupedal robot,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 9735–9742

2024
[20]

Jump-start reinforcement learning,

I. Uchendu, T. Xiao, Y . Lu, B. Zhu, M. Yan, J. Simon, M. Bennice, C. Fu, C. Ma, J. Jiao, S. Levine, and K. Hausman, “Jump-start reinforcement learning,” inProceedings of the 40th International Conference on Machine Learning. PMLR, 2023, pp. 34 556–34 583

2023
[21]

Rocket landing control with random annealing jump start reinforcement learning,

Y . Jiang, Y . Yang, Z. Lan, G. Zhan, S. E. Li, Q. Sun, J. Ma, T. Yu, and C. Zhang, “Rocket landing control with random annealing jump start reinforcement learning,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 14 026– 14 033

2024
[22]

A transformation-aggregation framework for state representation of au- tonomous driving systems,

G. Zhan, Y . Jiang, S. E. Li, Y . Lyu, X. Zhang, and Y . Yin, “A transformation-aggregation framework for state representation of au- tonomous driving systems,”IEEE Transactions on Intelligent Trans- portation Systems, vol. 25, no. 7, pp. 7311–7322, 2024

2024
[23]

Proximal policy optimization algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

2017 arXiv
[24]

High- dimensional continuous control using generalized advantage estima- tion,

J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estima- tion,” inProceedings of the International Conference on Learning Representations (ICLR), 2016

2016
[25]

Isaac gym: High performance gpu based physics simulation for robot learning,

V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Mack- lin, D. Hoeller, N. Rudin, A. Allshire, A. Handa,et al., “Isaac gym: High performance gpu based physics simulation for robot learning,” inThirty-fifth Conference on Neural Information Processing Systems Datase...

2021
[26]

Orbit: A unified simulation framework for interactive robot learning environments,

M. Mittal, C. Yu, Q. Yu, J. Liu, N. Rudin, D. Hoeller, J. L. Yuan, R. Singh, Y . Guo, H. Mazhar, A. Mandlekar, B. Babich, G. State, M. Hutter, and A. Garg, “Orbit: A unified simulation framework for interactive robot learning environments,”IEEE Robotics and Automa- tion Letter...

2023
[27]

Learning to walk in minutes using massively parallel deep reinforcement learning,

N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” inPro- ceedings of the 5th Conference on Robot Learning, ser. Proceedings of Machine Learning Research, vol. 164. PMLR, 2022, pp. 91–100

2022
[28]

Conformal symplectic optimization for stable reinforcement learn- ing,

Y . Lyu, X. Zhang, S. E. Li, J. Duan, L. Tao, Q. Xu, L. He, and K. Li, “Conformal symplectic optimization for stable reinforcement learn- ing,”IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 6, pp. 11 049–11 063, 2025

2025
[29]

Curriculum-based reinforcement learning for quadrupedal jumping: A reference-free design,

V . Atanassov, J. Ding, J. Kober, I. Havoutis, and C. Della Santina, “Curriculum-based reinforcement learning for quadrupedal jumping: A reference-free design,”IEEE Robotics & Automation Magazine, 2024

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [3]

Extreme parkour with legged robots,

X. Cheng, K. Shi, A. Agarwal, and D. Pathak, “Extreme parkour with legged robots,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 11 443–11 450

2024

[2] [1]

Advances in real-world applications for legged robots,

C. D. Bellicoso, M. Bjelonic, L. Wellhausen, K. Holtmann, F. G ¨unther, M. Tranzatto, P. Fankhauser, and M. Hutter, “Advances in real-world applications for legged robots,”Journal of Field Robotics, vol. 35, no. 8, pp. 1311–1326, 2018

2018

[3] [2]

Mul- timodality robotic systems: Integrated combined legged-aerial mobility for subterranean search-and-rescue,

B. Lindqvist, S. Karlsson, A. Koval, I. Tevetzidis, J. Halu ˇska, C. Kanellakis, A.-a. Agha-mohammadi, and G. Nikolakopoulos, “Mul- timodality robotic systems: Integrated combined legged-aerial mobility for subterranean search-and-rescue,”Robotics and Autonomous Sys- tems, vol. 154, p. 104134, 2022

work page 2022

[4] [4]

Learning agile loco- motion on risky terrains,

C. Zhang, N. Rudin, D. Hoeller, and M. Hutter, “Learning agile loco- motion on risky terrains,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 11 864– 11 871

2024

[5] [5]

S. E. Li,Reinforcement Learning for Sequential Decision and Optimal Control. Springer Verlag, Singapore, 2023

2023

[6] [6]

Rambo: Rl-augmented model-based optimal control for whole-body loco- manipulation,

J. Cheng, D. Kang, G. Fadini, G. Shi, and S. Coros, “Rambo: Rl-augmented model-based optimal control for whole-body loco- manipulation,”arXiv preprint arXiv:2504.06662, 2025

arXiv 2025

[7] [7]

Enhance generality by model-based reinforcement learning and domain ran- domization,

G. Zhan, Y . Lyu, S. E. Li, Y . Jiang, X. Zhang, and L. Tao, “Enhance generality by model-based reinforcement learning and domain ran- domization,” in2023 7th CAA International Conference on Vehicular Control and Intelligence (CVCI). IEEE, 2023, pp. 1–6

work page 2023

[8] [8]

Transferable latent-to-latent locomotion policy for efficient and versatile motion control of diverse legged robots,

Z. Zheng, G. Zhan, B. Shuai, S. Qin, J. Li, T. Zhang, and S. E. Li, “Transferable latent-to-latent locomotion policy for efficient and versatile motion control of diverse legged robots,”2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025

work page 2025