Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation

Ayonga Hereid; Chengyang Peng; Keith A. Redmill; Sankalp Agrawal; Shiting Gong; Zhihao Zhang

REVIEW 4 major objections 5 minor 1 cited by

Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-level navigation scheme—a bootstrapped RL subgoal planner plus a linear MPC gait controller—lets a simulated humanoid robot steer through cluttered environments with 93.3% success in the reported tests.

desk verdict A useful hierarchical RL+MPC navigation stack for Digit with a strong bootstrapping effect, but the headline 'significantly improves' rests on four trials and a missing held-out comparison against the model-based expert. read the letter →

arxiv 2506.02206 v1 pith:KXFBMM6H submitted 2025-06-02 cs.RO

Chengyang Peng , Zhihao Zhang , Shiting Gong , Sankalp Agrawal , Keith A. Redmill , Ayonga Hereid This is my paper

classification cs.RO

keywords humanoidrobotnavigationreinforcementlearningsubgoalplanningmodelpredictivecontrollinearinvertedpendulumdatabootstrappingobstacleavoidancesoftactor-critic

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a two-level navigation scheme—an RL policy that chooses a local waypoint each walking step, plus a linear MPC that turns those waypoints into foot placements—lets a bipedal humanoid robot navigate cluttered rooms in real time. The authors test the claim in simulation on the Digit humanoid and report that the learned subgoal planner reaches the goal in 93.3% of the seen test scenarios and 89.3% of 25 harder unseen scenarios, outperforming the model-based expert that supplied its training data. If the claim holds, it resolves a practical trade-off: nonlinear whole-body optimization is too slow for closed-loop navigation, while geometric planners ignore the robot's dynamics. The proposed combination keeps the high-level decision fast (about 1.3 ms per subgoal) and the low-level gait computation fast (about 0.94 ms), while the bootstrapping trick initializes RL with expert demonstrations instead of starting from random exploration.

What carries the argument

The load-bearing object is the two-level decomposition of navigation into subgoal selection and gait generation, glued by the robot-centric polar action space $(d_c, \phi_c)$ and the linearization in Eq. (16). The high level is an MDP whose state includes a $64 \times 64$ local occupancy grid and reduced step-to-step center-of-mass states; the policy is a Gaussian Soft Actor-Critic actor over subgoals, with rewards for goal progress, heading alignment, smoothness, forward velocity, and obstacle clearance. The low level is a linear MPC built on the 3D linear inverted pendulum model, whose constant-CoM-height and constant-centroidal-momentum assumptions make the step-to-step dynamics a discrete linear system; the MPC's quadratic cost penalizes distance from the commanded subgoal while respecting velocity, reachability, and maneuverability constraints. The data-bootstrapping mechanism fills the replay buffer with 10,000 state-action transitions from RRT+LMPC, initially sampling 80% from these demonstrations and decaying to fully online samples, which is what allows the RL policy to learn high-performance navigation rather than circling until time-out.

What would settle it

Re-run the reported trials while logging the actual center-of-mass height and centroidal angular momentum during MPC tracking; if the height deviates substantially from the assumed constant $H = 1$ m when the robot executes sharp commanded turns, the LIP model's guarantee is broken, and a scenario with a higher density of forced rotations should show success rates falling toward the LMPC baseline.

Watch

Extended reading notes

Core claim

The central claim is that a learned one-step subgoal planner, trained with the Soft Actor-Critic algorithm and bootstrapped on offline demonstrations from an RRT-guided linear LIP-MPC controller, can serve as the high-level navigation policy for a bipedal humanoid while the low-level MPC maintains stable walking. At each step the policy outputs a subgoal in robot-centric polar coordinates (distance $d_c$ and heading $\phi_c$); the low-level MPC then solves a linear-constraint quadratic program that chooses swing-foot stepping positions and a turning rate. The turning rate is fixed in advance by $\omega_k = \phi_c/(N T)$, which converts a nonlinear optimization into a fast one solvable at 20 Hz or faster. The reported simulations show 93.3% success on the training distribution and 89.3% on an unseen eight-obstacle test set, compared with 90% for the RRT+LMPC expert and 14% for the best learning-only baseline (Conservative Q-Learning, CQL).

Load-bearing premise

Everything rests on the low-level planner being able to execute the turn implied by a subgoal's heading, because the MPC fixes the turning schedule via $\omega_k = \phi_c/(N T)$ instead of optimizing heading jointly with foot placement; if the robot cannot track that scheduled turn while holding its center of mass near constant height, the gait fails and the reported success rates do not transfer.

Editorial extensions

If this is right

A real-time humanoid navigation loop can be assembled from a 1.3 ms learned subgoal update plus a 0.94 ms linear MPC, both comfortably within the 20 Hz gait-control requirement the paper uses.
The high-level planner needs only a local occupancy map, so navigation does not require solving a global path-planning problem at every step.
Bootstrapping from expert demonstrations changes the Soft Actor-Critic result from 9% success in the seen test set to 93.3%, so the demonstration dataset is load-bearing, not merely a training accelerator.
The trained policy generalizes off the training distribution, keeping 89.3% success on unseen scenarios that all contain the maximum eight obstacles.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An extension the paper leaves implicit: the same subgoal abstraction could be paired with a nonlinear or whole-body MPC; if the abstraction is doing the work, success rates should transfer while low-level agility improves.
The comparison with CQL suggests offline-only learning saturates below the hybrid recipe; a testable prediction is that adding more demonstration data alone will not close the gap unless online interaction is also allowed.
Because the action space limits subgoal heading to $\pm \pi/4$ and the low level fixes the turning schedule through Eq. (16), the policy may struggle in environments that force repeated sharp turns; measuring success rate against cumulative turning demand would probe that boundary.
The subgoal is updated at 2.5 Hz, once per walking step, so the local occupancy map is refreshed at that rate; dynamic obstacles moving between updates would be the first serious stress test for the local-map-only state.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper is a practical integration paper, not a new algorithm paper. The new bit is coupling a SAC-based subgoal planner with a linear MPC gait controller for Digit, and bootstrapping the replay buffer with RRT+LMPC demonstrations. The components are known, but the specific stack is new and the evaluation is in MuJoCo.

What it does well: the system is clearly described, hyperparameters are complete, and the bootstrapping effect is large. SAC alone gets 9% success on the seen test, SAC+Demo gets 93.3%. That's a real, reproducible-looking result. The modular design is sensible, and the paper is honestly written.

The soft spots are evaluation rigor. The abstract says 'significantly improves' compared to both model-based and learning-based methods. Against learning baselines, the margin is large. But against the model-based expert, RRT+LMPC, the margin is 93.3 vs 90.0 on the seen test, over four trials with overlapping standard deviations and no significance test. The unseen test (Table V) is the only held-out evaluation, and it omits RRT+LMPC entirely, so the adaptability claim relative to the model-based method is untested. Also, the unseen set is not a sample from the training distribution (all 25 cases have 8 obstacles and a fixed goal), so it's a stress test, not a generalization estimate. The low-level MPC details are deferred to [20], which makes independent verification of Eq. (16) and the constraints difficult. That's a citation-pattern issue, not misconduct, but it matters.

The central idea — that bootstrapping model-based demonstrations helps RL for this task — holds up. I don't see a load-bearing flaw in the math, but the headline claim is over-stated for the current evidence.

I'd send this to peer review, with a request for statistical tests, a held-out comparison against RRT+LMPC, and code release. It's useful for people working on hierarchical legged navigation, and I'd likely cite it as an example of demonstration bootstrapping in humanoid navigation.

Referee Report

4 major / 5 minor

Summary. The paper proposes a hierarchical navigation framework for the Agility Robotics Digit humanoid in simulation. A high-level Soft Actor-Critic policy receives a 64×64 local occupancy map plus robot and goal states and outputs a robot-centric subgoal (distance and heading), while a low-level linear MPC based on a step-to-step LIP model computes foot placements and turning rates to track that subgoal. Training is bootstrapped by seeding the replay buffer with 10,000 transitions collected from an RRT+LMPC expert, then continuing with online SAC. The authors compare the method against two model-based baselines (LMPC and RRT+LMPC) and three learning baselines (CQL, PPO, SAC) in a MuJoCo environment, reporting a 93.3% success rate on the training-set test scenarios and 89.3% on 25 newly generated high-difficulty scenarios, versus 90.0% for RRT+LMPC on the seen set and 14.0% for the best learning-only baseline on the unseen set.

Significance. If the claimed performance advantage is statistically and experimentally robust, the paper would demonstrate a practical division of labor: a learned one-step subgoal planner that uses only local perception, paired with a lightweight linear-MPC gait controller, can outperform both a purely geometric RRT+MPC planner and learning-only high-level planners for bipedal navigation. The paper has several genuine strengths: the hierarchical decomposition is clean and well motivated; the reward structure is explicit and fully parameterized; the simulation study includes four learning algorithms and three model-based variants; computation times for the MPC and the RL planner are reported; and a video is provided. The main weakness is that the central quantitative claim of 'significantly improves' is not backed by statistical testing or by a model-based comparison on held-out environments, so the headline result could change under a more careful evaluation.

major comments (4)

[Section IV.A, Table III] The abstract's claim that the method 'significantly improves navigation success rates' is not supported by the reported statistics. The seen-test success margin is 93.3 ± 2.1% versus 90.0 ± 1.4% for RRT+LMPC, computed from only four trials, with overlapping standard deviations and no significance test or per-trial counts. A difference of 3.3 percentage points over four trials may easily be explained by sampling noise. Please report the raw per-trial outcomes and provide a suitable test (for example, an exact binomial test on aggregate counts, or a paired permutation test across trials) before making the 'significantly improves' claim.
[Section IV.B, Tables IV and V] The held-out generalization comparison in Table V omits the model-based baselines LMPC and RRT+LMPC, even though the abstract and Section IV.A explicitly claim superiority over 'the original model-based method.' The seen-environment comparison in Table III is not a generalization test because it uses the same 50 environments used for training. To support the adaptability claim, the authors need to run LMPC and RRT+LMPC on the same 25 unseen environments and report their success rates and rewards alongside the learning-based methods.
[Section IV, testing-unseen setup] The 'testing-unseen' environment is not a sample from the training distribution, so it cannot estimate a typical generalization gap. All 25 scenarios contain exactly eight obstacles, start at (0,0), and share the same goal at (10,10), whereas the training distribution contains 0, 1, 2, 6, or 8 obstacles with randomized goal positions. The current protocol is a useful stress test, but the paper should either resample unseen scenarios from the same randomization procedure as training, or explicitly label the results as a high-difficulty stress test and avoid wording that suggests an estimate of average generalization performance.
[Section II.B, Eq. (16)] The low-level MPC prescribes the entire turning schedule as ω_k = φ_c/(N T), leaving the MPC to optimize only foot placements around this predetermined heading ramp. The paper does not separately evaluate whether the commanded heading change is actually tracked by the LIP-based gait, nor does it report tracking errors or fall rates attributable to the low-level controller. Because the success metric conflates high-level subgoal selection with low-level feasibility, a demonstration that the commanded turns are dynamically feasible under the LIP assumptions (for example, a tracking-error or stability analysis, or a sensitivity study over the action bounds d_c and φ_c) is needed to support the claim that the framework generalizes to more dynamic or uncertain environments.

minor comments (5)

[Table I] The table heading says 'Reply buffer size' but should read 'Replay buffer size.'
[Throughout] The text inconsistently uses 'trial' and 'trail' (for example, 'results from one specific trail' and 'only considering successful trails' in Section IV.A). Please standardize to 'trial.'
[Equations (9)-(11)] The velocity reward subcomponents are written as rvx and rvy, which read as products r·vx and r·vy in the typeset PDF. Please rename them (for example, r_vx and r_vy) for clarity.
[Section IV.A, Real-Time Computation] The RL planner is reported to compute a subgoal in 1.3 ± 0.5 ms but is deliberately run at 2.5 Hz. The paper would benefit from a one-sentence justification of why such a low update rate is appropriate given the MPC step duration of 0.4 s and the observed computation time.
[General] The authors state that training is performed on a laptop with an RTX 4060 GPU, but do not report wall-clock training time or whether all 10,000 episodes were used per learning baseline. Reporting these details would improve reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the subgoal RL policy is measured against re-run baselines, and the self-cited low-level controller is empirically validated in the same simulator rather than assumed.

full rationale

The paper's success claim is empirical, not derived: SAC+Demo is trained with the reward in Eq. (5) and then evaluated by independent success-rate metrics in Tables III-V, so the result is not an identity with the training objective. The low-level LIP-MPC and the RRT+LMPC demonstration generator are inherited from the authors' prior work [20]/[10]; this is substantial self-citation, but the paper does not ask the reader to take those components on faith. It re-runs LMPC and RRT+LMPC as baselines in the same MuJoCo environment (Table III, 84% and 90% success) before reporting 93.3% for the proposed method, so the comparison is a benchmark rather than a circular proof. The bootstrapped demonstrations seed the replay buffer, but SAC is not an imitation learner and the final policy could in principle underperform RRT+LMPC, so the 93.3% vs 90.0% margin is not forced by construction. Eq. (16) is a stated design choice for converting a subgoal heading to a turning rate, not a prediction derived from the fitted policy. The paper's own limitation statement (Conclusion: 'there remains room for improving trajectory optimization and enhancing robustness and generalization') concerns generalization and hardware transfer, not circularity. No self-definitional, fitted-input, uniqueness-import, or ansatz-smuggling step was found. Concerns about missing model-based baselines on unseen tests and overlapping error bars are validity issues, not circularity. Score 2 reflects the multiple non-load-bearing self-citations, not actual circular reasoning.

Assumptions & free parameters 11 free parameters · 6 assumptions · 0 invented entities

The framework rests on a large hand-tuned reward design and on a low-level controller whose details are mostly inherited from the authors' prior work. These choices, rather than a first-principles derivation, determine the reported performance.

free parameters (11)

Reward weight vector w = [0.2, 0.1, 0.25, 0.2, 0.25]
Eq. (5) weights six sub-rewards; chosen by hand without sensitivity analysis.
Goal proximity reward parameters [ag, bg] = [2.33, 0.3]
Eq. (6); scaling for distance-progress reward.
Heading penalty scale a_theta = 1.39
Eq. (7); penalizes heading error with cubic term.
Action smoothness balance q_a = 0.5
Eq. (8); balances progress versus action-change penalty.
Velocity reward parameters [qv, av, bv] = [0.7, 15, 0.5]
Eqs. (9)-(10); balances forward versus lateral velocity rewards.
Collision avoidance scaling zeta = 0.4
Eq. (12); barrier-condition blending in collision reward.
Terminal reward and penalties = 60, -80, -70
Eq. (13); hand-chosen to balance goal-reaching against failures.
Demonstration schedule = 80% demos, 20% online initially; 10,000 demo transitions
Section III; the bootstrapping schedule and demo count are chosen by hand and are not ablated.
Subgoal action bounds (d_c, phi_c) = [0,3] m, [-pi/4, pi/4]
Section II-A; limits are chosen from physical capability before evaluation.
MPC horizon and step duration (N, T) = N=3, T=0.4 s
Section II-B; fixed prediction horizon and step time used by all baselines.
Local occupancy map geometry = 64x64 grid, 4.5 m forward, 1.5 m back, 6 m lateral
Section II-A; perception range is a design choice that bounds what the policy can see.

assumptions (6)

domain assumption CoM height and centroidal momentum remain constant, yielding a linear inverted pendulum model.
Section II.B takes this standard bipedal modeling assumption as the basis for the step-to-step dynamics in Eq. (14).
domain assumption Step-to-step discrete dynamics with a constant step duration T and passive stance ankle are accurate enough for Digit at the commanded speeds.
Section II.B uses constant T and impact resets to define the discrete linear system; this is an approximation for the real underactuated robot.
ad hoc to paper The turning rate can be prespecified as omega_k = phi_c/(N T) while the MPC optimizes only foot placements.
Eq. (16) decouples heading control from foot placement optimization; this is introduced for this paper and is not derived from the dynamics.
ad hoc to paper Reward terms and hand-picked weights induce the intended navigation behavior.
Section II-A defines six reward components with fixed weights; no proof or sensitivity analysis shows these choices are necessary or sufficient.
domain assumption The MuJoCo Digit simulation is a valid proxy for evaluating navigation success and stability.
All validation is in simulation; hardware effects such as friction, compliance, and sensing noise are not considered.
domain assumption The 10,000 RRT+LMPC demonstration transitions are a representative and useful initialization for the RL policy.
Section III; no analysis shows the demonstrations cover the state space or that the 80%-to-100% schedule is optimal.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation." pith.science (2026). https://pith.science/paper/KXFBMM6H

@misc{pith2026250602206,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KXFBMM6H}},
  note         = {Machine review of arXiv:2506.02206}
}

read the original abstract

Safe and real-time navigation is fundamental for humanoid robot applications. However, existing bipedal robot navigation frameworks often struggle to balance computational efficiency with the precision required for stable locomotion. We propose a novel hierarchical framework that continuously generates dynamic subgoals to guide the robot through cluttered environments. Our method comprises a high-level reinforcement learning (RL) planner for subgoal selection in a robot-centric coordinate system and a low-level Model Predictive Control (MPC) based planner which produces robust walking gaits to reach these subgoals. To expedite and stabilize the training process, we incorporate a data bootstrapping technique that leverages a model-based navigation approach to generate a diverse, informative dataset. We validate our method in simulation using the Agility Robotics Digit humanoid across multiple scenarios with random obstacles. Results show that our framework significantly improves navigation success rates and adaptability compared to both the original model-based method and other learning-based methods.

Figures

Figures reproduced from arXiv: 2506.02206 by the authors.

**Figure 1.** A humanoid robot navigates around clustered environment using our proposed sub-goal orientated framework. underscoring the need for robust navigation methods [8]–[10]. Model-based methods are commonly used in bipedal locomotion control, formulating and solving constrained optimization problems to integrate path planning with robot dynamics. Many studies have developed trajectory optimization controllers based on … view at source ↗

**Figure 2.** Overall structure of the proposed hierarchical framework for humanoid navigation. The high-level RL-based planner [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Five representative environments from the 50 ran [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: The navigation performance of LMPC, RRT-LMPC, [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Comparing training performance of four different learning-based approaches. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

RAVEN: Reinforcement-Adaptive Visibility-Graph Planning for Robust Humanoid Navigation with Collision-Free MPC
cs.RO 2026-07 conditional novelty 5.0 of 10

Reinforcement learning that adjusts obstacle-inflation radii in a visibility-graph planner improves humanoid navigation robustness under delay and noise, beating a static MPC baseline and an end-to-end RL policy in a ...

Reference graph

Works this paper leans on

35 extracted references · 27 canonical work pages · cited by 1 Pith paper

[20]

Real-time safe bipedal robot navigation using linear discrete control barrier functions,

C. Peng, V . Paredes, G. A. Castillo, and A. Hereid, “Real-time safe bipedal robot navigation using linear discrete control barrier functions,” in To appear in 2025 IEEE International Conference on Robotics and Automation (ICRA) , 2025

work page 2025
[1]

Introduction of the Foot Placement Estimator: A Dynamic Measure of Balance for Bipedal Robotics,

D. L. Wight, E. G. Kubica, and D. W. L. Wang, “Introduction of the Foot Placement Estimator: A Dynamic Measure of Balance for Bipedal Robotics,” Journal of Computational and Nonlinear Dynamics , vol. 3, no. 1, p. 011009, 11 2007

work page 2007
[2]

Navigation planning for legged robots in challenging terrain,

M. Wermelinger, P. Fankhauser, R. Diethelm, P. Kr ¨usi, R. Siegwart, and M. Hutter, “Navigation planning for legged robots in challenging terrain,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2016, pp. 1184–1189

work page 2016
[3]

Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot,

S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Per- menter, T. Koolen, P. Marion, and R. Tedrake, “Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot,” Autonomous robots, vol. 40, pp. 429–455, 2016

2016
[4]

Exact cell decomposition of arrangements used for path planning in robotics,

N. Sleumer and N. Tschichold-G ¨urmann, “Exact cell decomposition of arrangements used for path planning in robotics,” Technical Re- port/ETH Zurich, Department of Computer Science , vol. 329, 1999

work page 1999
[5]

An overview of autonomous mobile robot path planning algorithms,

N. Sariff and N. Buniyamin, “An overview of autonomous mobile robot path planning algorithms,” in 2006 4th student conference on research and development. IEEE, 2006, pp. 183–188

work page 2006
[6]

Path planning and trajectory planning algorithms: A general overview,

A. Gasparetto, P. Boscariol, A. Lanzutti, and R. Vidoni, “Path planning and trajectory planning algorithms: A general overview,” Motion and Operation Planning of Robotic Systems: Background and Practical Approaches, pp. 3–27, 2015

work page 2015
[7]

Confidence random tree- based algorithm for mobile robot path planning considering the path length and safety,

Y . N. Kim, D. W. Ko, and I. H. Suh, “Confidence random tree- based algorithm for mobile robot path planning considering the path length and safety,”International Journal of Advanced Robotic Systems, vol. 16, no. 2, p. 1729881419838179, 2019

work page 2019

Show all 35 references

[8]

Optimization-based motion planning for legged robots,

A. W. Winkler, “Optimization-based motion planning for legged robots,” Ph.D. dissertation, ETH Zurich, 2018

2018
[9]

Integrated task and motion planning for safe legged navigation in partially observable environments,

A. Shamsah, Z. Gu, J. Warnke, S. Hutchinson, and Y . Zhao, “Integrated task and motion planning for safe legged navigation in partially observable environments,” IEEE Transactions on Robotics , vol. 39, no. 6, pp. 4913–4934, 2023

2023
[10]

Unified path and gait planning for safe bipedal robot navigation,

C. Peng, V . Paredes, and A. Hereid, “Unified path and gait planning for safe bipedal robot navigation,” arXiv preprint arXiv:2403.17347 , 2024

2024 arXiv
[11]

Fast direct multiple shooting algorithms for optimal robot control,

M. Diehl, H. G. Bock, H. Diedam, and P.-B. Wieber, “Fast direct multiple shooting algorithms for optimal robot control,” in Fast mo- tions in biomechanics and robotics: optimization and feedback control. Springer, 2006, pp. 65–93

2006
[12]

Using optimization to create self-stable human-like running,

K. Mombaur, “Using optimization to create self-stable human-like running,” Robotica, vol. 27, no. 3, pp. 321–330, 2009

2009
[13]

Whole-body motion planning with centroidal dynamics and full kinematics,

H. Dai, A. Valenzuela, and R. Tedrake, “Whole-body motion planning with centroidal dynamics and full kinematics,” in 2014 IEEE-RAS International Conference on Humanoid Robots . IEEE, 2014, pp. 295–302

2014
[14]

The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation,

S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa, “The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation,” in Proceedings 2001 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems. Expanding the Societal Rol...

2001
[15]

Bipedal walking control based on capture point dynamics,

J. Englsberger, C. Ott, M. A. Roa, A. Albu-Sch ¨affer, and G. Hirzinger, “Bipedal walking control based on capture point dynamics,” in 2011 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2011, pp. 4420–4427

2011
[16]

Nonlinear model predictive control for rough-terrain robot hopping,

M. Rutschmann, B. Satzinger, M. Byl, and K. Byl, “Nonlinear model predictive control for rough-terrain robot hopping,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2012, pp. 1859–1864

2012
[17]

Perceptive locomotion through nonlinear model-predictive control,

R. Grandia, F. Jenelten, S. Yang, F. Farshidian, and M. Hutter, “Perceptive locomotion through nonlinear model-predictive control,” IEEE Transactions on Robotics , vol. 39, no. 5, pp. 3402–3421, 2023

2023
[18]

Model predictive control for dynamic footstep adjustment using the divergent component of motion,

R. J. Griffin and A. Leonessa, “Model predictive control for dynamic footstep adjustment using the divergent component of motion,” in2016 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2016, pp. 1763–1768

2016
[19]

A sequential mpc approach to reactive planning for bipedal robots using safe corridors in highly cluttered environments,

K. S. Narkhede, A. M. Kulkarni, D. A. Thanki, and I. Poulakakis, “A sequential mpc approach to reactive planning for bipedal robots using safe corridors in highly cluttered environments,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11 831–11 838, 2022

2022
[21]

Apprenticeship learning via inverse rein- forcement learning,

P. Abbeel and A. Y . Ng, “Apprenticeship learning via inverse rein- forcement learning,” in Proceedings of the twenty-first international conference on Machine learning , 2004, p. 1

2004
[22]

Apprenticeship learning using linear programming,

U. Syed, M. Bowling, and R. E. Schapire, “Apprenticeship learning using linear programming,” in Proceedings of the 25th international conference on Machine learning , 2008, pp. 1032–1039

2008
[23]

Generative adversarial imitation learning,

J. Ho and S. Ermon, “Generative adversarial imitation learning,” Advances in neural information processing systems , vol. 29, 2016

2016
[24]

Goal-oriented obstacle avoid- ance with deep reinforcement learning in continuous action space,

R. Cimurs, J. H. Lee, and I. H. Suh, “Goal-oriented obstacle avoid- ance with deep reinforcement learning in continuous action space,” Electronics, vol. 9, no. 3, p. 411, 2020

2020
[25]

Where to go next: Learning a subgoal recommendation policy for navigation in dynamic environments,

B. Brito, M. Everett, J. P. How, and J. Alonso-Mora, “Where to go next: Learning a subgoal recommendation policy for navigation in dynamic environments,”IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4616–4623, 2021

2021
[26]

Robot navigation in constrained pedestrian environments using reinforcement learning,

C. P ´erez-D’Arpino, C. Liu, P. Goebel, R. Mart ´ın-Mart´ın, and S. Savarese, “Robot navigation in constrained pedestrian environments using reinforcement learning,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 1140–1146

2021
[27]

A hierarchical deep reinforcement learning framework with high efficiency and generalization for fast and safe navigation,

W. Zhu and M. Hayashibe, “A hierarchical deep reinforcement learning framework with high efficiency and generalization for fast and safe navigation,” IEEE Transactions on industrial Electronics , vol. 70, no. 5, pp. 4962–4971, 2022

2022
[28]

Drl-vo: Learning to navigate through crowded dynamic scenes using velocity obstacles,

Z. Xie and P. Dames, “Drl-vo: Learning to navigate through crowded dynamic scenes using velocity obstacles,” IEEE Transactions on Robotics, vol. 39, no. 4, pp. 2700–2719, 2023

2023
[29]

Efficient deep reinforcement learning with imitative expert priors for autonomous driving,

Z. Huang, J. Wu, and C. Lv, “Efficient deep reinforcement learning with imitative expert priors for autonomous driving,” IEEE Transac- tions on Neural Networks and Learning Systems , vol. 34, no. 10, pp. 7391–7403, 2022

2022
[30]

Pre-training goal-based models for sample-efficient reinforcement learning,

H. Yuan, Z. Mu, F. Xie, and Z. Lu, “Pre-training goal-based models for sample-efficient reinforcement learning,” in The Twelfth International Conference on Learning Representations , 2024

2024
[31]

Deep q- learning from demonstrations,

T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, I. Osband, et al. , “Deep q- learning from demonstrations,” in Proceedings of the AAAI conference on artificial intelligence , vol. 32, no. 1, 2018

2018
[32]

Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” in International conference on machine learning . Pmlr, 2018, pp. 1861–1870

2018
[33]

Template model inspired task space learning for robust bipedal locomotion,

G. A. Castillo, B. Weng, S. Yang, W. Zhang, and A. Hereid, “Template model inspired task space learning for robust bipedal locomotion,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 8582–8589

2023
[34]

Conservative q- learning for offline reinforcement learning,

A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q- learning for offline reinforcement learning,” Advances in neural in- formation processing systems , vol. 33, pp. 1179–1191, 2020

2020
[35]

Proximal policy optimization algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

2017 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [20]

Real-time safe bipedal robot navigation using linear discrete control barrier functions,

C. Peng, V . Paredes, G. A. Castillo, and A. Hereid, “Real-time safe bipedal robot navigation using linear discrete control barrier functions,” in To appear in 2025 IEEE International Conference on Robotics and Automation (ICRA) , 2025

work page 2025

[2] [1]

Introduction of the Foot Placement Estimator: A Dynamic Measure of Balance for Bipedal Robotics,

D. L. Wight, E. G. Kubica, and D. W. L. Wang, “Introduction of the Foot Placement Estimator: A Dynamic Measure of Balance for Bipedal Robotics,” Journal of Computational and Nonlinear Dynamics , vol. 3, no. 1, p. 011009, 11 2007

work page 2007

[3] [2]

Navigation planning for legged robots in challenging terrain,

M. Wermelinger, P. Fankhauser, R. Diethelm, P. Kr ¨usi, R. Siegwart, and M. Hutter, “Navigation planning for legged robots in challenging terrain,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2016, pp. 1184–1189

work page 2016

[4] [3]

Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot,

S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Per- menter, T. Koolen, P. Marion, and R. Tedrake, “Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot,” Autonomous robots, vol. 40, pp. 429–455, 2016

2016

[5] [4]

Exact cell decomposition of arrangements used for path planning in robotics,

N. Sleumer and N. Tschichold-G ¨urmann, “Exact cell decomposition of arrangements used for path planning in robotics,” Technical Re- port/ETH Zurich, Department of Computer Science , vol. 329, 1999

work page 1999

[6] [5]

An overview of autonomous mobile robot path planning algorithms,

N. Sariff and N. Buniyamin, “An overview of autonomous mobile robot path planning algorithms,” in 2006 4th student conference on research and development. IEEE, 2006, pp. 183–188

work page 2006

[7] [6]

Path planning and trajectory planning algorithms: A general overview,

A. Gasparetto, P. Boscariol, A. Lanzutti, and R. Vidoni, “Path planning and trajectory planning algorithms: A general overview,” Motion and Operation Planning of Robotic Systems: Background and Practical Approaches, pp. 3–27, 2015

work page 2015

[8] [7]

Confidence random tree- based algorithm for mobile robot path planning considering the path length and safety,

Y . N. Kim, D. W. Ko, and I. H. Suh, “Confidence random tree- based algorithm for mobile robot path planning considering the path length and safety,”International Journal of Advanced Robotic Systems, vol. 16, no. 2, p. 1729881419838179, 2019

work page 2019