Pith. sign in

REVIEW 4 major objections 5 minor 19 references

A learning layer that adjusts obstacle inflation in a visibility-graph planner makes a humanoid robot navigate more robustly under control delay and sensor noise than either a fixed-geometry MPC or a pure end-to-end RL policy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:31 UTC pith:RCN3KPHU

load-bearing objection A sensible hybrid-planner idea—RL adapting visibility-graph inflation radii—is let down by single-scenario, no-variance evaluation; still worth a serious referee. the 4 major comments →

arxiv 2607.15701 v1 pith:RCN3KPHU submitted 2026-07-17 cs.RO

RAVEN: Reinforcement-Adaptive Visibility-Graph Planning for Robust Humanoid Navigation with Collision-Free MPC

Ruochen Hou , Shiqi Wang , Beom Jun Kim , Hanzhang Fang , Mehak Singal , Dennis W. Hong This is my paper
classification cs.RO
keywords humanoid navigationreinforcement learningmodel predictive controlvisibility graphobstacle inflationsim-to-real transfercollision avoidancehierarchical planning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes RAVEN, a hierarchical navigation system for humanoid robots in which a reinforcement-learning meta-policy does not generate commands but instead chooses per-obstacle inflation radii used to build a visibility graph. By enlarging or shrinking obstacles, the learned policy reshapes the free-space geometry and thus changes which shortest path the graph planner selects, compensating for the overshoot and tracking errors introduced by control delay and estimation noise. A collision-free model predictive controller then tracks the chosen path while explicitly enforcing velocity, acceleration, and obstacle-avoidance constraints. In simulation with a 0.06 s actuation delay, RAVEN keeps maximum obstacle penetration at 0.03 m while achieving the shortest average path (9.33 m) and fastest completion (11.58 s) of the three compared methods, and hardware trials on a bipedal robot show trajectories that transfer more consistently than the pure RL policy.

Core claim

The central claim is that learned adaptation of a geometric planner parameter—obstacle inflation—is an effective, interpretable middle ground between rigid classical planning and monolithic end-to-end reinforcement learning. The meta-policy observes the delayed, noisy robot state and outputs an inflation radius for each obstacle in a visibility-graph planner; larger radii push the shortest path further from obstacles, smaller radii allow it to cut closer. Since the graph always returns the shortest path under the current geometry, the RL search is constrained to a family of shortest-path solutions, which the paper argues both guides exploration and preserves global efficiency. The downstream

What carries the argument

The load-bearing object is the adaptive obstacle-inflation radius in the Dynamic Augmented Visibility Graph. A visibility graph is a graph whose vertices include obstacle vertices and the start and goal, with edges between any two mutually visible vertices; shortest paths through it are piecewise straight and tangent to inflated obstacles. RAVEN lets a reinforcement-learning meta-policy choose each obstacle's inflation radius continuously, so the learned layer effectively morphs the graph's geometry and therefore the topology of the planned path. The collision-free MPC layer tracks the planned trajectory through a quadratic program that enforces velocity and acceleration limits and linearize

Load-bearing premise

The learned inflation policy transfers from simulation to the physical robot because the simulation faithfully reproduces the robot's control delay, observation noise, and low-level tracking behavior; if the real tracking-error distribution differs from the simulated one, the claimed real-world robustness is not established.

What would settle it

Run the trained RAVEN policy on the physical humanoid for repeated trials with the same start, goal, and obstacle layout under the nominal 0.06 s delay, and measure per-trial maximum obstacle penetration and success rate; if the median penetration exceeds the 0.03 m safety bound or trials fail at a rate not observed in simulation, the central robustness claim is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Under 0.06 s actuation delay, RAVEN keeps maximum obstacle penetration at 0.03 m while the fixed-inflation MPC baseline reaches 0.128 m, supporting the claim that geometric adaptation suppresses delay-induced overshoot.
  • RAVEN's average path length (9.33 m) and time to completion (11.58 s) are better than both the classical baseline (11.25 m, 12.32 s) and the end-to-end RL baseline (9.80 m, 12.21 s), showing that learning over shortest-path structures preserves efficiency while adding robustness.
  • Because the MPC layer, not a learned network, enforces velocity and obstacle constraints, the framework retains explicit safety guarantees and runs at about 100 Hz, fast enough for real-time humanoid navigation.
  • The asymmetric actor-critic design—the actor sees delayed noisy states while the critic sees privileged clean states—is intended to make the learned inflation policy transfer from simulation to the physical robot.
  • The approach suggests that learning at the level of geometric planning parameters, rather than control weights or raw commands, can be a scalable way to add adaptability to classical navigation stacks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the same mechanism of learning geometric parameters rather than control weights could be applied to other graph-based or sampling-based planners, for example adapting connection radii or sample distributions in a probabilistic roadmap, with similar interpretability.
  • A natural testable extension is to train the meta-policy with moving obstacles; the learned inflation schedule could then be examined to see whether the robot pre-inflates obstacles on the side where control delay is largest.
  • The paper's hardware evidence is a single qualitative trajectory comparison; a quantitative multi-trial deployment measuring per-trial penetration depth and success rate would directly substantiate the sim-to-real claim.
  • Since control delay is the dominant disturbance, a fixed analytic rule that inflates obstacles as a function of current speed and heading rate might capture much of the benefit; comparing RAVEN against such a hand-designed schedule would isolate what the learned adaptation adds beyond a well-chosen static inflation policy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. RAVEN is a hierarchical navigation framework for humanoid robots. A PPO meta-policy observes robot, goal, and obstacle states and outputs per-obstacle inflation radii used by the DA VG visibility-graph planner; the resulting shortest path is tracked by a cf-MPC layer with linearized obstacle constraints and a low-level locomotion policy. The paper compares RAVEN with a fixed-radius DA VG-cfMPC baseline and an end-to-end RL baseline under zero delay and 0.06 s delay in a single simulation scenario (Table V), and reports a qualitative hardware demonstration on the T1 robot. The central claim is that RL-adapted geometric graph construction is more robust and more efficient than both static MPC and pure RL while retaining interpretability.

Significance. The idea of using RL to modify obstacle inflation in a visibility graph is a genuinely different integration point from prior work that tunes MPC cost weights or generates subgoals; it has the potential to be interpretable and sample-efficient because the graph shortest-path structure provides a planning prior. Strengths include a JAX/GPU implementation with parallel training, a deliberate attempt to equalize total training time between RAVEN and the pure RL baseline, and a real-hardware deployment with a low-level locomotion policy. No circularity is apparent: the learned inflation behavior is not forced by the reward construction. However, the evidence base is too narrow to support the broad robustness and sim-to-real claims made in the paper.

major comments (4)
  1. [§V-A, Table V] The quantitative support for the main claim rests on a single scenario: one start pose, one goal pose, and three fixed obstacles. The table reports 'Average' metrics with no number of episodes, no seeds, no standard deviations, and no significance tests. The observed margins (e.g., 9.33 m vs 9.80 m path length and 11.58 s vs 12.21 s completion time under 0.06 s delay) could easily be within run-to-run variance. Since the RL meta-policy is trained on randomized episodes, a held-out, multi-seed evaluation over varied obstacle layouts and start/goal configurations is necessary before claiming superiority.
  2. [§III-A and §V-A, Table V(b)] The term 'collision-free MPC' is contradicted by the reported 'Ave. Max Obstacle Penetration Depth' of 0.03 m for RAVEN and 0.128 m for the baseline. The constraint in §III-A contains a slack variable δ_j ≥ 0, so it is a soft constraint that permits penetration. The paper should either rename this layer or explicitly frame the guarantee as bounded penetration under a chosen safety margin; otherwise the title, abstract, and conclusion overstate the safety property.
  3. [§V-B] The hardware demonstration is a single qualitative trajectory comparison: no success criteria, no repeated trials, and no quantitative metrics such as completion time, penetration depth, or failure count. The text concludes 'more reliable navigation' and 'consistent performance across simulation and real-world' from Fig. 5, but the figure only shows curves. Additionally, the hardware start pose (3.4, -1.0, 1.57) differs from the simulation scenario in Table V (3.0, -1.0, -1.57), making it unclear what the simulated rollout in Fig. 5 was compared against. This section does not support the sim-to-real transfer claim.
  4. [§IV-A, §V-A] The only classical baseline uses a fixed inflation radius of 1 m, with no sensitivity analysis or systematic tuning. A central claim is that automatic adaptation is preferable to manual tuning, but a more favorable fixed radius, or a small set of tuned radii, might close much of the gap. Reporting an ablation over fixed inflation values, and ideally a comparison against RL-tuned MPC cost weights, would make the contribution of geometric adaptation load-bearing rather than an artifact of a single baseline setting.
minor comments (5)
  1. [§III-A] The obstacle-avoidance inequality should be numbered, and all symbols (V_{k,j}, R_obs,j, δ_j) should be defined immediately after the equation.
  2. [Table V] The table should state the number of evaluation episodes and seeds for each method, and the authors should consider reporting standard deviations or confidence intervals.
  3. [References] Reference [13] lacks publication venue, year, and bibliographic details; please complete it.
  4. [Throughout] The name is written inconsistently as 'RAVEN' in the title and 'RA VEN' in the body; pick one convention.
  5. [§V-A] The term 'penetration depth' is used without definition; define it consistently with the soft-constraint slack variables.

Circularity Check

0 steps flagged

No circular derivation: RL-adapted inflation is an open empirical result; the self-citation to the prior DAVG-cfMPC pipeline is legitimate reuse, not load-bearing.

full rationale

The paper's derivation chain is not circular. The only learned component is the RL meta-policy, which maps observations to inflation radii through an affine transformation; its behavior is determined by PPO training against a reward that includes time, path length, collision, penetration, and action-rate penalties. Table V then reports those same quantities, but this is standard evaluation of a trained policy rather than a quantity equal to the policy output by construction. The learned inflation schedule is not derivable from the reward alone: the reward does not specify which radii to select, the visibility graph still computes shortest paths, and the MPC still enforces its own constraints, so the outcome remains an empirical result. The only self-citation is [4], the authors' prior DAVG-cfMPC pipeline, used as the base planner and as the MPC baseline; this is a peer-reviewed reusable component and is not invoked to establish the new adaptive-inflation claim, so it is not load-bearing. The manuscript's use of a single 'representative navigation scenario' in Section V-A and the qualitative hardware comparison in Section V-B are validity/statistical-support concerns, not circularity steps.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claim depends on hand-tuned reward weights, inflation bounds, MPC costs, and the fidelity of the simulation-to-real transfer; no new physical entities are introduced.

free parameters (6)
  • RAVEN reward weights = w_time=-5.0, w_path=-20.0, w_col=-8.0, w_inside=-80.0, w_ar=-0.5, R_succ=5000, R_fall=-50000
    Hand-chosen to balance efficiency and safety; directly shape the learned inflation behavior. No sensitivity analysis is reported (Table II).
  • Inflation radius bounds r_min, r_max = not reported
    Define the allowable range of the RL action mapping (Sec III-B2); chosen by hand and affect achievable path topology. Values not stated.
  • Baseline fixed inflation radius = 1.0 m
    Manual baseline uses a fixed 1 m radius (Sec IV-A); a different choice could change relative performance.
  • Success/penalty thresholds = epsilon_p=0.2, epsilon_psi=0.2, r_pen=1.0
    Given in Table IV caption; define success and collision-penalty circles. Chosen without sensitivity analysis; likely shared with RAVEN though not stated.
  • Delay/noise injection parameters = 0.06 s delay; noise not specified
    Evaluation uses 0.06 s actuation delay (Sec V-A); training delay/noise distribution is not fully specified, affecting sim-to-real transfer.
  • MPC cost weights and horizon = not reported
    Inherited from prior DA VG-cfMPC work [4]; affect tracking tightness and penetration depth. Not stated in this paper.
axioms (5)
  • domain assumption Visibility graph shortest path yields a kinematically feasible reference trajectory.
    Sec III-A; the planner routes the robot around inflated obstacles assuming point/humanoid footprint geometry.
  • domain assumption Linearized soft collision constraints with slack delta_j in the QP are sufficient to keep the robot collision-free during execution.
    Sec III-A Eq (1); the paper later reports nonzero penetration (0.03 m), so this axiom is only approximately satisfied.
  • domain assumption Booster Gym locomotion policy tracks MPC velocity commands on the T1 robot.
    Sec V-B; the low-level policy is taken from [19] and assumed to work as a velocity controller.
  • domain assumption MJX simulation with injected delay and noise faithfully replicates the T1's real-world tracking behavior for training and evaluation.
    Sec IV-B and V-B; sim-to-real transfer of the learned inflation policy relies on this.
  • domain assumption PPO converges to a generalizable policy under the given reward and observation space.
    Sec IV-B; standard practice, no convergence guarantee, and no evaluation over randomized environments.

pith-pipeline@v1.3.0-alltime-deepseek · 10736 in / 12372 out tokens · 96758 ms · 2026-08-01T22:31:46.269439+00:00 · methodology

0 comments
read the original abstract

Humanoid navigation in dynamic environments requires long-horizon planning while respecting short-horizon dynamic and safety constraints. Classical visibility-graph planners combined with model predictive control (MPC) can efficiently generate collision-free trajectories, but their performance depends on manually tuned parameters and accurate system modeling. In real robotic systems, control delays, state-estimation noise, and locomotion uncertainties can cause overshoot and constraint violations even when the nominal path is geometrically optimal. We propose RAVEN, a hierarchical reinforcement learning (RL)-MPC framework for robust humanoid navigation. Unlike prior approaches that use learning to tune cost weights or replace planning entirely, RAVEN employs RL to adapt the geometric construction of a visibility-graph planner by modifying obstacle inflation and related graph parameters. By directly reshaping the free-space geometry, the learned planner alters the topology of the global path to compensate for delay and tracking imperfections. A collision-free MPC layer then tracks the planned trajectory while explicitly enforcing velocity bounds and obstacle-avoidance constraints. By training under realistic delays and observation noise, RAVEN learns planning adaptations that improve robustness while retaining explicit long-horizon geometric planning and constrained optimization, in contrast to end-to-end learning approaches. We evaluate RAVEN against a manually tuned visibility-graph MPC baseline and a pure RL navigation policy. Results demonstrate reduced overshoot near obstacles, improved robustness in narrow passages, and more reliable navigation under delay and noise. These findings indicate that reinforcement-adaptive graph construction combined with constrained MPC provides an effective and interpretable alternative to end-to-end learning for robust humanoid navigation.

Figures

Figures reproduced from arXiv: 2607.15701 by Beom Jun Kim, Dennis W. Hong, Hanzhang Fang, Mehak Singal, Ruochen Hou, Shiqi Wang.

Figure 1
Figure 1. Figure 1: The RAVEN framework deployed on the T1 Booster biped navigating [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the RAVEN architecture. The framework consists of [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of how adapting obstacle size changes the visibility graph [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of how RAVEN adapts the planned path. Red dots [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of real-world and simulated trajectories under the same [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 10 linked inside Pith

  1. [1]

    Compass: Cross-embodiment mobility policy via residual rl and skill synthesis,

    W. Liu, H. Zhao, C. Li, Y . Deng, J. Biswas, S. Pouya, and Y . Chang, “Compass: Cross-embodiment mobility policy via residual rl and skill synthesis,” 2025. [Online]. Available: https://arxiv.org/abs/2502.16372

    arXiv 2025

  2. [2]

    Anymal parkour: Learning agile navigation for quadrupedal robots,

    D. Hoeller, N. Rudin, D. Sako, and M. Hutter, “Anymal parkour: Learning agile navigation for quadrupedal robots,” 2023. [Online]. Available: https://arxiv.org/abs/2306.14874

    Pith/arXiv arXiv 2023

  3. [3]

    Robot parkour learning,

    Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao, “Robot parkour learning,” 2023. [Online]. Available: https://arxiv.org/abs/2309.05665

    Pith/arXiv arXiv 2023

  4. [4]

    Model predictive control with visibility graphs for humanoid path planning and tracking against adversarial opponents,

    R. Hou, G. I. Fernandez, M. Zhu, and D. W. Hong, “Model predictive control with visibility graphs for humanoid path planning and tracking against adversarial opponents,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 14 520–14 526

    2025

  5. [5]

    Lto: lazy trajectory opti- mization with graph-search planning for high dof robots in cluttered environments,

    Y . Shirai, X. Lin, A. Mehta, and D. Hong, “Lto: lazy trajectory opti- mization with graph-search planning for high dof robots in cluttered environments,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 7533–7539

    2021

  6. [6]

    Reinforcement learning with data bootstrapping for dynamic subgoal pursuit in humanoid robot navigation,

    C. Peng, Z. Zhang, S. Gong, S. Agrawal, K. A. Redmill, and A. Hereid, “Reinforcement learning with data bootstrapping for dynamic subgoal pursuit in humanoid robot navigation,” 2025. [Online]. Available: https://arxiv.org/abs/2506.02206

    Pith/arXiv arXiv 2025

  7. [7]

    Hierarchical learning-enhanced mpc for safe crowd navigation with heterogeneous constraints,

    H. Liu, Y . Feng, W. Dong, K. Fan, C. Wang, and Y . Gao, “Hierarchical learning-enhanced mpc for safe crowd navigation with heterogeneous constraints,” 2025. [Online]. Available: https: //arxiv.org/abs/2506.09859

    Pith/arXiv arXiv 2025

  8. [8]

    Robot navigation with reinforcement learned path generation and fine-tuned motion control,

    L. Zhang, Z. Hou, J. Wang, Z. Liu, and W. Li, “Robot navigation with reinforcement learned path generation and fine-tuned motion control,” IEEE Robotics and Automation Letters, vol. 8, no. 8, pp. 4489–4496, 2023

    2023

  9. [9]

    Residual reinforcement learning for robot control,

    T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, “Residual reinforcement learning for robot control,” 2018. [Online]. Available: https: //arxiv.org/abs/1812.03201

    Pith/arXiv arXiv 2018

  10. [10]

    Mpc-net: A first principles guided policy search,

    J. Carius, F. Farshidian, and M. Hutter, “Mpc-net: A first principles guided policy search,”IEEE Robotics and Automation Letters, vol. 5, no. 2, p. 2897–2904, Apr. 2020. [Online]. Available: http://dx.doi.org/10.1109/LRA.2020.2974653

    arXiv 2020

  11. [11]

    Learning deep control policies for autonomous aerial vehicles with mpc-guided policy search,

    T. Zhang, G. Kahn, S. Levine, and P. Abbeel, “Learning deep control policies for autonomous aerial vehicles with mpc-guided policy search,” 2016. [Online]. Available: https://arxiv.org/abs/1509.06791

    Pith/arXiv arXiv 2016

  12. [12]

    Hybrid classical/rl local planner for ground robot navigation,

    V . D. Sharma, J. Lee, M. Andrews, and I. Hadži ´c, “Hybrid classical/rl local planner for ground robot navigation,” 2024. [Online]. Available: https://arxiv.org/abs/2410.03066

    Pith/arXiv arXiv 2024

  13. [13]

    Safe reinforcement learning via statistical model predictive shielding

    O. Bastani, S. Li, and A. Xu, “Safe reinforcement learning via statistical model predictive shielding.”

  14. [14]

    Real-time neural mpc: Deep learning model predictive control for quadrotors and agile robotic platforms,

    T. Salzmann, E. Kaufmann, J. Arrizabalaga, M. Pavone, D. Scaramuzza, and M. Ryll, “Real-time neural mpc: Deep learning model predictive control for quadrotors and agile robotic platforms,”IEEE Robotics and Automation Letters, vol. 8, no. 4, p. 2397–2404, Apr. 2023. [Online]. Available: http://dx.doi.org/10.1109/LRA.2023.3246839

    arXiv 2023

  15. [15]

    Dynamic tube mpc: Learning tube dynamics with massively parallel simulation for robust safety in practice,

    W. D. Compton, N. Csomay-Shanklin, C. Johnson, and A. D. Ames, “Dynamic tube mpc: Learning tube dynamics with massively parallel simulation for robust safety in practice,” 2024. [Online]. Available: https://arxiv.org/abs/2411.15350

    Pith/arXiv arXiv 2024

  16. [16]

    Automated tuning of nonlinear model predictive controller by reinforcement learning,

    M. Mehndiratta, E. Camci, and E. Kayacan, “Automated tuning of nonlinear model predictive controller by reinforcement learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 3016–3021

    2018

  17. [17]

    A safe reinforcement learning driven weights-varying model predictive control for autonomous vehicle motion control,

    B. Zarrouki, M. Spanakakis, and J. Betz, “A safe reinforcement learning driven weights-varying model predictive control for autonomous vehicle motion control,” 2024. [Online]. Available: https://arxiv.org/abs/2402.02624

    Pith/arXiv arXiv 2024

  18. [18]

    Actor- critic model predictive control: Differentiable optimization meets reinforcement learning for agile flight,

    A. Romero, E. Aljalbout, Y . Song, and D. Scaramuzza, “Actor- critic model predictive control: Differentiable optimization meets reinforcement learning for agile flight,” 2025. [Online]. Available: https://arxiv.org/abs/2306.09852

    arXiv 2025

  19. [19]

    Booster gym: An end-to-end reinforcement learning framework for humanoid robot locomotion,

    Y . Wang, P. Chen, X. Han, F. Wu, and M. Zhao, “Booster gym: An end-to-end reinforcement learning framework for humanoid robot locomotion,”arXiv preprint arXiv:2506.15132, 2025

    Pith/arXiv arXiv 2025