{"id":"76cbe366-5ce5-49b9-8c12-54c2eb7a1df0","arxiv_id":"2506.02206","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":11,"one_line_summary":"A hierarchical controller pairing a reinforcement-learned subgoal planner with a linear-inverted-pendulum walking controller improves simulated Digit humanoid navigation success over model-based and learning baselines.","lead":"The authors used an AI planner to choose short walking targets and a stability controller to reach them, letting a simulated two-legged robot weave through obstacles. The paper is worth a look because it shows a practical way to combine learned decision-making with physics-based walking control, though the comparisons are small and simulation-only.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'significantly improves' claim is not yet supported: the model-based baseline is absent from unseen tests, and the seen-test margin (93.3 vs 90.0) rests on four trials with overlapping error bars.","rationale":"After reading the paper, I looked first at the low-level controller because Eq. (16) fixes the turn-rate schedule and the MPC constraints are deferred to a prior paper. That assumption could limit agility, but the simulations already exercise it on 50 training maps, and the reported success rates suggest the simplification works in the tested regime. The load-bearing weakness is elsewhere: the central claim is comparative and quantitative, but the comparison is statistically thin and asymmetric. The only model-based baseline appears in Table III, which is over the training scenarios; Table V's held-out evaluation omits RRT+LMPC. Thus 'adaptability compared to the original model-based method' has no direct evidence. The 3.3-point seen margin is also within one standard deviation, and no significance test is reported. These are fixable with a modest additional evaluation, so I keep the reader's conditional verdict rather than moving to reject. The paper does report hyperparameters and computing times, which is helpful, but the missing baseline comparison is sufficiently important that the abstract's strongest claim should not be accepted as established.","tokens_in":11244,"tokens_out":5953,"duration_ms":57666,"concrete_test":"Run RRT+LMPC on the 25 unseen scenarios of Table V under the same evaluation conditions, with at least 10 seeds to account for RRT randomness, and compare its success rate to SAC+Demo's 89.3%. If RRT+LMPC's 95% confidence interval overlaps 89.3%, the central claim of improved adaptability over the model-based method is unsupported. Additionally, rerun the full comparison on a held-out 50-scenario set drawn from the training distribution to confirm that the seen-test margin is not training-set overfitting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the abstract's claim that the framework 'significantly improves navigation success rates and adaptability compared to both the original model-based method and other learning-based methods,' the paper needs (i) a model-based comparison on held-out environments and (ii) statistical evidence that the advantage is not noise. Neither is present. Table III compares LMPC, RRT+LMPC, and the proposed method only on the 'testing-seen' environment, which is explicitly the 50 training scenarios; the margin over RRT+LMPC is 93.3±2.1% vs 90.0±1.4% over four trials, with overlapping standard deviations and no significance test. Table V, the only held-out evaluation, reports learning baselines but omits RRT+LMPC, so the 'adaptability compared to the original model-based method' component is untested. Also, the unseen set is not a sample from the training distribution (all 25 cases have 8 obstacles and a fixed goal at (10,10)), so it cannot estimate a typical generalization gap even though it is a useful stress test. These gaps, rather than any internal inconsistency in Eq. (16), are the weakest point: the central quantitative claim would fail if a held-out comparison showed RRT+LMPC matching or beating SAC+Demo.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hierarchical navigation framework for the Agility Robotics Digit humanoid in simulation. A high-level Soft Actor-Critic policy receives a 64×64 local occupancy map plus robot and goal states and outputs a robot-centric subgoal (distance and heading), while a low-level linear MPC based on a step-to-step LIP model computes foot placements and turning rates to track that subgoal. Training is bootstrapped by seeding the replay buffer with 10,000 transitions collected from an RRT+LMPC expert, then continuing with online SAC. The authors compare the method against two model-based baselines (LMPC and RRT+LMPC) and three learning baselines (CQL, PPO, SAC) in a MuJoCo environment, reporting a 93.3% success rate on the training-set test scenarios and 89.3% on 25 newly generated high-difficulty scenarios, versus 90.0% for RRT+LMPC on the seen set and 14.0% for the best learning-only baseline on the unseen set.","tokens_in":11625,"tokens_out":3257,"duration_ms":33743,"significance":"If the claimed performance advantage is statistically and experimentally robust, the paper would demonstrate a practical division of labor: a learned one-step subgoal planner that uses only local perception, paired with a lightweight linear-MPC gait controller, can outperform both a purely geometric RRT+MPC planner and learning-only high-level planners for bipedal navigation. The paper has several genuine strengths: the hierarchical decomposition is clean and well motivated; the reward structure is explicit and fully parameterized; the simulation study includes four learning algorithms and three model-based variants; computation times for the MPC and the RL planner are reported; and a video is provided. The main weakness is that the central quantitative claim of 'significantly improves' is not backed by statistical testing or by a model-based comparison on held-out environments, so the headline result could change under a more careful evaluation.","major_comments":[{"comment":"The abstract's claim that the method 'significantly improves navigation success rates' is not supported by the reported statistics. The seen-test success margin is 93.3 ± 2.1% versus 90.0 ± 1.4% for RRT+LMPC, computed from only four trials, with overlapping standard deviations and no significance test or per-trial counts. A difference of 3.3 percentage points over four trials may easily be explained by sampling noise. Please report the raw per-trial outcomes and provide a suitable test (for example, an exact binomial test on aggregate counts, or a paired permutation test across trials) before making the 'significantly improves' claim.","section":"Section IV.A, Table III"},{"comment":"The held-out generalization comparison in Table V omits the model-based baselines LMPC and RRT+LMPC, even though the abstract and Section IV.A explicitly claim superiority over 'the original model-based method.' The seen-environment comparison in Table III is not a generalization test because it uses the same 50 environments used for training. To support the adaptability claim, the authors need to run LMPC and RRT+LMPC on the same 25 unseen environments and report their success rates and rewards alongside the learning-based methods.","section":"Section IV.B, Tables IV and V"},{"comment":"The 'testing-unseen' environment is not a sample from the training distribution, so it cannot estimate a typical generalization gap. All 25 scenarios contain exactly eight obstacles, start at (0,0), and share the same goal at (10,10), whereas the training distribution contains 0, 1, 2, 6, or 8 obstacles with randomized goal positions. The current protocol is a useful stress test, but the paper should either resample unseen scenarios from the same randomization procedure as training, or explicitly label the results as a high-difficulty stress test and avoid wording that suggests an estimate of average generalization performance.","section":"Section IV, testing-unseen setup"},{"comment":"The low-level MPC prescribes the entire turning schedule as ω_k = φ_c/(N T), leaving the MPC to optimize only foot placements around this predetermined heading ramp. The paper does not separately evaluate whether the commanded heading change is actually tracked by the LIP-based gait, nor does it report tracking errors or fall rates attributable to the low-level controller. Because the success metric conflates high-level subgoal selection with low-level feasibility, a demonstration that the commanded turns are dynamically feasible under the LIP assumptions (for example, a tracking-error or stability analysis, or a sensitivity study over the action bounds d_c and φ_c) is needed to support the claim that the framework generalizes to more dynamic or uncertain environments.","section":"Section II.B, Eq. (16)"}],"minor_comments":[{"comment":"The table heading says 'Reply buffer size' but should read 'Replay buffer size.'","section":"Table I"},{"comment":"The text inconsistently uses 'trial' and 'trail' (for example, 'results from one specific trail' and 'only considering successful trails' in Section IV.A). Please standardize to 'trial.'","section":"Throughout"},{"comment":"The velocity reward subcomponents are written as rvx and rvy, which read as products r·vx and r·vy in the typeset PDF. Please rename them (for example, r_vx and r_vy) for clarity.","section":"Equations (9)-(11)"},{"comment":"The RL planner is reported to compute a subgoal in 1.3 ± 0.5 ms but is deliberately run at 2.5 Hz. The paper would benefit from a one-sentence justification of why such a low update rate is appropriate given the MPC step duration of 0.4 s and the observed computation time.","section":"Section IV.A, Real-Time Computation"},{"comment":"The authors state that training is performed on a laptop with an RTX 4060 GPU, but do not report wall-clock training time or whether all 10,000 episodes were used per learning baseline. Reporting these details would improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The evaluation relies heavily on the authors' own prior work for the LMPC formulation [20] and the nonlinear MPC baseline [10], and the manuscript refers readers elsewhere for the detailed MPC constraints. For a journal submission, it would be advisable to require that the MPC formulation be made self-contained in an appendix or that code and environment definitions be released, since otherwise an independent assessment of the central comparison is difficult. The statistical and held-out-baseline gaps identified in the major comments are fixable within the scope of the paper, so I do not recommend rejection, but the current evidence does not support the headline 'significantly improves' claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is a practical integration paper, not a new algorithm paper. The new bit is coupling a SAC-based subgoal planner with a linear MPC gait controller for Digit, and bootstrapping the replay buffer with RRT+LMPC demonstrations. The components are known, but the specific stack is new and the evaluation is in MuJoCo.\n\nWhat it does well: the system is clearly described, hyperparameters are complete, and the bootstrapping effect is large. SAC alone gets 9% success on the seen test, SAC+Demo gets 93.3%. That's a real, reproducible-looking result. The modular design is sensible, and the paper is honestly written.\n\nThe soft spots are evaluation rigor. The abstract says 'significantly improves' compared to both model-based and learning-based methods. Against learning baselines, the margin is large. But against the model-based expert, RRT+LMPC, the margin is 93.3 vs 90.0 on the seen test, over four trials with overlapping standard deviations and no significance test. The unseen test (Table V) is the only held-out evaluation, and it omits RRT+LMPC entirely, so the adaptability claim relative to the model-based method is untested. Also, the unseen set is not a sample from the training distribution (all 25 cases have 8 obstacles and a fixed goal), so it's a stress test, not a generalization estimate. The low-level MPC details are deferred to [20], which makes independent verification of Eq. (16) and the constraints difficult. That's a citation-pattern issue, not misconduct, but it matters.\n\nThe central idea — that bootstrapping model-based demonstrations helps RL for this task — holds up. I don't see a load-bearing flaw in the math, but the headline claim is over-stated for the current evidence.\n\nI'd send this to peer review, with a request for statistical tests, a held-out comparison against RRT+LMPC, and code release. It's useful for people working on hierarchical legged navigation, and I'd likely cite it as an example of demonstration bootstrapping in humanoid navigation.","headline":"A useful hierarchical RL+MPC navigation stack for Digit with a strong bootstrapping effect, but the headline 'significantly improves' rests on four trials and a missing held-out comparison against the model-based expert.","tokens_in":12109,"tokens_out":2222,"would_cite":true,"duration_ms":19990,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-level navigation scheme—a bootstrapped RL subgoal planner plus a linear MPC gait controller—lets a simulated humanoid robot steer through cluttered environments with 93.3% success in the reported tests.","keywords":["humanoid robot navigation","reinforcement learning","subgoal planning","model predictive control","linear inverted pendulum","data bootstrapping","obstacle avoidance","soft actor-critic"],"falsifier":"Re-run the reported trials while logging the actual center-of-mass height and centroidal angular momentum during MPC tracking; if the height deviates substantially from the assumed constant $H = 1$ m when the robot executes sharp commanded turns, the LIP model's guarantee is broken, and a scenario with a higher density of forced rotations should show success rates falling toward the LMPC baseline.","tokens_in":11052,"feed_emoji":"🤖","tokens_out":9965,"duration_ms":93380,"temperature":0.7,"pith_summary":"This paper tries to establish that a two-level navigation scheme—an RL policy that chooses a local waypoint each walking step, plus a linear MPC that turns those waypoints into foot placements—lets a bipedal humanoid robot navigate cluttered rooms in real time. The authors test the claim in simulation on the Digit humanoid and report that the learned subgoal planner reaches the goal in 93.3% of the seen test scenarios and 89.3% of 25 harder unseen scenarios, outperforming the model-based expert that supplied its training data. If the claim holds, it resolves a practical trade-off: nonlinear whole-body optimization is too slow for closed-loop navigation, while geometric planners ignore the robot's dynamics. The proposed combination keeps the high-level decision fast (about 1.3 ms per subgoal) and the low-level gait computation fast (about 0.94 ms), while the bootstrapping trick initializes RL with expert demonstrations instead of starting from random exploration.","feed_headline":"A learned subgoal planner steers a humanoid to 93% navigation success","feed_subtitle":"A two-level RL-plus-MPC policy beats both its model-based expert and pure learning baselines in cluttered simulation.","key_machinery":"The load-bearing object is the two-level decomposition of navigation into subgoal selection and gait generation, glued by the robot-centric polar action space $(d_c, \\phi_c)$ and the linearization in Eq. (16). The high level is an MDP whose state includes a $64 \\times 64$ local occupancy grid and reduced step-to-step center-of-mass states; the policy is a Gaussian Soft Actor-Critic actor over subgoals, with rewards for goal progress, heading alignment, smoothness, forward velocity, and obstacle clearance. The low level is a linear MPC built on the 3D linear inverted pendulum model, whose constant-CoM-height and constant-centroidal-momentum assumptions make the step-to-step dynamics a discrete linear system; the MPC's quadratic cost penalizes distance from the commanded subgoal while respecting velocity, reachability, and maneuverability constraints. The data-bootstrapping mechanism fills the replay buffer with 10,000 state-action transitions from RRT+LMPC, initially sampling 80% from these demonstrations and decaying to fully online samples, which is what allows the RL policy to learn high-performance navigation rather than circling until time-out.","core_discovery":"The central claim is that a learned one-step subgoal planner, trained with the Soft Actor-Critic algorithm and bootstrapped on offline demonstrations from an RRT-guided linear LIP-MPC controller, can serve as the high-level navigation policy for a bipedal humanoid while the low-level MPC maintains stable walking. At each step the policy outputs a subgoal in robot-centric polar coordinates (distance $d_c$ and heading $\\phi_c$); the low-level MPC then solves a linear-constraint quadratic program that chooses swing-foot stepping positions and a turning rate. The turning rate is fixed in advance by $\\omega_k = \\phi_c/(N T)$, which converts a nonlinear optimization into a fast one solvable at 20 Hz or faster. The reported simulations show 93.3% success on the training distribution and 89.3% on an unseen eight-obstacle test set, compared with 90% for the RRT+LMPC expert and 14% for the best learning-only baseline (Conservative Q-Learning, CQL).","pith_inferences":["An extension the paper leaves implicit: the same subgoal abstraction could be paired with a nonlinear or whole-body MPC; if the abstraction is doing the work, success rates should transfer while low-level agility improves.","The comparison with CQL suggests offline-only learning saturates below the hybrid recipe; a testable prediction is that adding more demonstration data alone will not close the gap unless online interaction is also allowed.","Because the action space limits subgoal heading to $\\pm \\pi/4$ and the low level fixes the turning schedule through Eq. (16), the policy may struggle in environments that force repeated sharp turns; measuring success rate against cumulative turning demand would probe that boundary.","The subgoal is updated at 2.5 Hz, once per walking step, so the local occupancy map is refreshed at that rate; dynamic obstacles moving between updates would be the first serious stress test for the local-map-only state."],"forward_implications":["A real-time humanoid navigation loop can be assembled from a 1.3 ms learned subgoal update plus a 0.94 ms linear MPC, both comfortably within the 20 Hz gait-control requirement the paper uses.","The high-level planner needs only a local occupancy map, so navigation does not require solving a global path-planning problem at every step.","Bootstrapping from expert demonstrations changes the Soft Actor-Critic result from 9% success in the seen test set to 93.3%, so the demonstration dataset is load-bearing, not merely a training accelerator.","The trained policy generalizes off the training distribution, keeping 89.3% success on unseen scenarios that all contain the maximum eight obstacles."],"supporting_citations":[{"why":"Provides the nonlinear LIP-MPC baseline that motivates the need for a faster linear formulation and is the original model-based method the paper improves on.","marker":"[10]"},{"why":"Defines the 3D linear inverted pendulum model that yields the discrete linear step-to-step dynamics used in the low-level MPC.","marker":"[14]"},{"why":"Supplies the real-time linear LIP-MPC controller and the collision-avoidance reward used to build the demonstration expert and the high-level reward.","marker":"[20]"},{"why":"Introduces the demonstration-bootstrapping idea that the paper applies to initialize the replay buffer with expert transitions.","marker":"[31]"},{"why":"Defines the Soft Actor-Critic algorithm used to train the high-level subgoal policy.","marker":"[32]"},{"why":"Documents the 20 Hz real-time gait-control requirement that the paper uses to judge whether the low-level MPC is fast enough.","marker":"[33]"},{"why":"Provides the Conservative Q-Learning offline-RL baseline against which the bootstrapped online policy is compared.","marker":"[34]"},{"why":"Provides the Proximal Policy Optimization baseline used to illustrate the sample-inefficiency of on-policy learning in this task.","marker":"[35]"}],"fun_headline_variants":["RL subgoal planner with data bootstrapping hits 93% navigation success","Bootstrap RL subgoal selection steers humanoid to 93% success in clutter","Hierarchical RL-MPC navigation: learned subgoals beat model-based expert","Data bootstrapping accelerates RL subgoal learning for humanoid navigation","Learned subgoal planner with bootstrap data beats RRT expert in humanoid nav"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the low-level planner being able to execute the turn implied by a subgoal's heading, because the MPC fixes the turning schedule via $\\omega_k = \\phi_c/(N T)$ instead of optimizing heading jointly with foot placement; if the robot cannot track that scheduled turn while holding its center of mass near constant height, the gait fails and the reported success rates do not transfer.","fun_headline_variants_meta":{"raw":{"variants":["RL subgoal planner with data bootstrapping hits 93% navigation success","Bootstrap RL subgoal selection steers humanoid to 93% success in clutter","Hierarchical RL-MPC navigation: learned subgoals beat model-based expert","Data bootstrapping accelerates RL subgoal learning for humanoid navigation","Learned subgoal planner with bootstrap data beats RRT expert in humanoid nav"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000725,"raw_usage":{"total_tokens":3240,"prompt_tokens":923,"completion_tokens":2317,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":2215}},"tokens_in":539,"tokens_out":2317,"duration_ms":18430,"temperature":1.0,"reasoning_tokens":2215,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:28:50.557553+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the reported trials while logging the actual center-of-mass height and centroidal angular momentum during MPC tracking; if the height deviates substantially from the assumed constant $H = 1$ m when the robot executes sharp commanded turns, the LIP model's guarantee is broken, and a scenario with a higher density of forced rotations should show success rates falling toward the LMPC baseline.","supporting_citations":[{"cited_title":"Unified Path and Gait Planning for Safe Bipedal Robot Navigation","cited_arxiv_id":"2403.17347","evidence_quote":"Provides the nonlinear LIP-MPC baseline that motivates the need for a faster linear formulation and is the original model-based method the paper improves on."},{"cited_title":"The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation,","cited_arxiv_id":null,"evidence_quote":"Defines the 3D linear inverted pendulum model that yields the discrete linear step-to-step dynamics used in the low-level MPC."},{"cited_title":"Real-time safe bipedal robot navigation using linear discrete control barrier functions,","cited_arxiv_id":null,"evidence_quote":"Supplies the real-time linear LIP-MPC controller and the collision-avoidance reward used to build the demonstration expert and the high-level reward."},{"cited_title":"Deep q- learning from demonstrations,","cited_arxiv_id":null,"evidence_quote":"Introduces the demonstration-bootstrapping idea that the paper applies to initialize the replay buffer with expert transitions."},{"cited_title":"Template model inspired task space learning for robust bipedal locomotion,","cited_arxiv_id":null,"evidence_quote":"Documents the 20 Hz real-time gait-control requirement that the paper uses to judge whether the low-level MPC is fast enough."},{"cited_title":"Conservative q- learning for offline reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the Conservative Q-Learning offline-RL baseline against which the bootstrapped online policy is compared."}],"review_version":1}