{"id":"15b0ed6b-4f4a-453f-b86f-873489321619","arxiv_id":"2507.18138","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A modular residual learning framework, with a reinforcement-learned footstep and gait-phase correction and a supervised-learned dynamics correction, makes a nominal MPC quadruped controller robust to payloads, disturbances, and out-of-distribution conditions.","lead":"The paper combines a standard model-predictive controller for a four-legged robot with two small learning modules, one that adjusts foot placement and one that corrects the robot's motion model. This makes the robot keep its balance under heavy payloads and pushes, where the plain controller fails, while needing less training than full end-to-end reinforcement learning.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Residual-dynamics deployment relies on an unvalidated autoregressive prediction loop; the learned residual's closed-loop behavior in simulation is never checked before it is credited for the hardware robustness results.","rationale":"The reader's weakest assumption identifies the residual-dynamics sim-to-real transfer as the load-bearing risk, and I agree that this is the central soft spot. My stress-test pass sharpens the mechanism: the regressor in Eq. (16) is trained on labels from Eq. (14) but deployed with an autoregressive input, f_res,k-1, that during deployment is the network's own past output rather than the analytical label. This train/serve input distribution shift can produce self-sustaining residual errors that are then injected into the MPC dynamics in Eq. (19). The paper provides only indirect evidence through one set of hardware trials, which are single runs without error bars, and the simulation ablations in Table IV use the analytical residual, not the learned regressor. Thus the central claim that the learned residual module itself robustifies the nominal controller is not yet directly supported. This concern does not contradict the paper's evidence; it calls for one additional validation. The reader's CONDITIONAL verdict already captures this need, so I recommend no change to the verdict, with the concrete simulation check added as a condition for acceptance. I marked agreement as 'partial' because the reader identified the same broad transfer risk, while my concern is a more specific, testable mechanism within that risk.","tokens_in":12402,"tokens_out":3154,"duration_ms":38446,"concrete_test":"Run the following simulation experiment with the same domain randomization as Table I, the same MPC setup, and the same trained residual footstep policy: replace the analytical filtered residual from Eq. (15) in constraint (19) with the trained pi_RD output computed as in Eq. (16)-(17), including the autoregressive input f_res,k-1, and run the Table III protocol (6 kg payload, then external disturbances). Compare roll-angle RMS and success rates against res-dyn and res-all using the analytical residual. In the same simulation, log per-step prediction error ||pi_RD(oRD) - f_res_analytical|| and the time trace of the closed-loop residual. If the prediction error or the residual magnitude grows over time, the autoregressive feedback is the cause; if performance matches the analytical-residual version, the sim-to-real transfer concern is substantially weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-D's residual dynamics module is the load-bearing component for the payload and disturbance results, and its transfer from simulation to real hardware rests on an unvalidated autoregressive loop. The training label in Eq. (14) is an analytical finite-difference residual computed from simulator states, then low-pass filtered by Eq. (15). The deployed regressor in Eq. (16) instead predicts that residual from oRD (Eq. (17)), which includes f_res,k-1 — the network's own previous prediction, not the analytical residual. Because the input distribution at deployment is therefore generated by the regressor itself, any prediction error is fed back and can accumulate into a spurious additive term in the MPC constraint (19). The paper reports only downstream hardware success (Table III, single trials per condition) and never shows that pi_RD's outputs match the analytical residual in distribution, nor that the closed loop with the predicted residual is stable over time. This missing validation is precisely what must hold for the central claim that the learned dynamics correction, rather than the nominal MPC or footstep module alone, enables robustness under heavy payloads and disturbances.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a modular residual learning framework for quadruped locomotion. An RL-trained residual footstep and gait-phase module augments a heuristic footstep planner, while a supervised residual-dynamics network predicts low-frequency model discrepancy for a convex MPC built on the single rigid body model. The nominal MPC remains the core policy, and the learned modules act as auxiliary corrections. The authors evaluate the framework in simulation and on a Unitree Go1, reporting improved robustness to payloads and disturbances, reduced sensitivity to MPC cost weights, better out-of-distribution consistency than an end-to-end RL baseline, and higher learning efficiency than several residual-RL baselines.","tokens_in":12639,"tokens_out":7918,"duration_ms":85813,"significance":"If the claims hold, the framework is a pragmatic contribution to hybrid model-based/learning-based control: it retains the consistency and safety of convex MPC while using small, modular learned corrections to cover model mismatch. The modular division of labor (RL for discrete foothold/phase decisions, supervised learning for continuous dynamics residuals) is a sensible design that reduces the RL search space. Strengths of the paper include the relatively large simulation ablations (100 runs per condition in Table IV), multiple baselines including end-to-end RL and residual-augmented MPC variants, domain randomization, and a hardware feasibility demonstration. The main weakness is that the sim-to-real transfer of the learned residual-dynamics regressor is not directly validated: no closed-loop simulation with the neural regressor is reported, and the hardware evidence consists of single trials. The significance is therefore conditional on closing that validation gap.","major_comments":[{"comment":"The deployed residual-dynamics regressor is autoregressive: the observation o_RD in Eq. (17) contains \\hat f_res,k-1, which at runtime is the network's own previous output rather than the analytical filtered residual from Eq. (15). The regression labels, by contrast, are computed from simulator states via Eq. (14) and filtered with Eq. (15). The manuscript does not state whether the simulation results in Table IV and Fig. 5 use the analytical residual or the learned network. If they use the analytical residual, the only direct evidence for the learned regressor is the single-trial hardware data in Table III and Fig. 6. Either way, the paper needs a closed-loop simulation evaluation with the neural regressor in the MPC loop, a quantitative comparison of its predictions to the analytical residual (for example, normalized RMSE and a spectral check), and evidence that the autoregressive input distribution at deployment does not drift outside the training distribution. This validation is load-bearing because the hardware payload and disturbance robustness results are attributed to the residual dynamics module.","section":"Sec. III-D, Eqs. (14)-(17); Fig. 2"},{"comment":"All hardware results are single trials without repeated runs, error bars, or explicit trial counts. Given that the central robustness claim is supported by these demonstrations, the paper should report at least the number of trials and success/failure outcomes per condition, or provide repeated trials with variance. This is especially important for the headline numbers in Table III, where the 0.0695 rad roll RMS under disturbance is a single number from one run, and for the out-of-distribution comparison in Fig. 6, which is presented without any quantitative error metric beyond the plotted trajectory.","section":"Sec. IV-B, Table III; Sec. IV-D, Fig. 6"},{"comment":"The text states that the proposed method consistently achieves a high success rate, but at M_p = 1.75M the fpos-phase-dynRL baseline achieves 88% success versus 76% for res-all. Moreover, no confidence intervals or statistical tests are reported for the 100-run success rates, so the claimed superiority over this baseline is not established. The comparison should either be accompanied by uncertainty estimates or the claim should be softened to 'comparable success with lower RMS error and better learning efficiency.'","section":"Sec. IV-E, Table IV"}],"minor_comments":[{"comment":"The success-rate criterion is not defined. Please specify what constitutes a failure (for example, trunk contact, fall detection, or timeout) and how it is detected in simulation and on hardware.","section":"Sec. IV-C and Sec. IV-E"},{"comment":"The reward weights c1 and c2 in Eq. (9) are never given. Reporting their values is necessary for reproducibility.","section":"Sec. III-C, Eq. (9)"},{"comment":"The learning curves appear to come from a single training run. Please report the number of seeds and show variance or error bars, since the learning-efficiency claim rests on this figure.","section":"Sec. IV-E, Fig. 7"},{"comment":"The baseline implementations (fpos-phase-dynRL, jpos-dynRL, phase, resdyn-window) are described too briefly to be reproduced. Please provide network architectures, input/output dimensions, and training hyperparameters, or cite public code.","section":"Sec. IV-E"},{"comment":"Please clarify how the 10 million residual-dynamics training samples were collected: which policy was used, at what sampling rate, and whether the observation in Eq. (17) during training contains the analytical \\hat f_res,k-1 or the network's own previous prediction.","section":"Sec. III-D"},{"comment":"The out-of-distribution hardware test in Fig. 6 is qualitative. Reporting velocity-tracking RMS error for both controllers with and without the sandbag would make the comparison quantitative.","section":"Sec. IV-D"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on legged locomotion or MPC+learning hybrids. The new thing here is the specific split: an RL-trained residual footstep/phase module plus a supervised-learned residual dynamics module, low-pass filtered and treated as constant over the MPC horizon. That combination is not in the cited literature, and the paper does a decent job showing it helps. The simulation ablations (Table IV, 100 runs per condition) and the weight-sweep in Fig. 5 are real evidence. I also give credit for the honest conclusion that broader evaluation is needed.\n\nThe main soft spot is exactly the stress-test concern: the residual dynamics network is deployed in an autoregressive loop (its input includes its own previous output), and the paper never checks whether that loop keeps the predicted residual close to the analytical residual in closed loop. That matters because the residual dynamics are credited with much of the robustness, especially under payload. The hardware success is encouraging but indirect. Still, I wouldn't call it fatal – the system works, and the ablation shows the dynamics module contributes. It's a missing diagnostic, not a contradiction.\n\nOther soft spots, in proportion: the hardware results are single trials with no variance, so the numbers like 0.0212 vs 0.0538 are anecdotes, not measurements. Several tuning values (reward weights c1,c2, PD gains) are absent, which hurts replication. The learning curves in Fig. 7 look like single runs, so the \"learning efficiency\" claim is weaker than it appears.\n\nWho it's for: people building hybrid MPC-RL controllers for quadrupeds will get concrete ideas. It deserves a serious referee – I would send it to review, not desk reject, but with the expectation that the residual-loop validation and reproducibility details be addressed. My own verdict would be a conditional accept, not a full accept as is.","headline":"A genuinely modular residual-learning result with credible simulation ablations and a real hardware demo, but the residual-dynamics loop is undertested in closed loop and the hardware trials are single shots.","tokens_in":13194,"tokens_out":1582,"would_cite":true,"duration_ms":19020,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By pairing a reinforcement-learned footstep correction with a supervised-learned dynamics correction, this paper argues, a nominal convex MPC quadruped controller can handle heavy payloads, external kicks, and out-of-distribution…","keywords":["legged robots","quadruped locomotion","model predictive control","residual learning","reinforcement learning","supervised learning","robust control","out-of-distribution generalization"],"falsifier":"During a repeated 6 kg payload and push-recovery trial, compare the network-predicted residual with the measured residual, defined as the difference between the nominal model's one-step prediction and the observed state change over the same control step. If the predicted residual is not significantly correlated with the measured residual at the moments when the robot regains balance, then the proposed compensation mechanism is not what is stabilizing the robot.","tokens_in":12131,"feed_emoji":"🦿","tokens_out":8221,"duration_ms":87717,"temperature":0.7,"pith_summary":"This paper tries to show that a conventional model-based quadruped controller, built around a simplified rigid-body model and convex model-predictive control, can be made reliable under exactly the uncertainties that break it: heavy payloads, external pushes, and uneven terrain. The method adds two small learned correction modules instead of replacing the controller. A reinforcement-learned module adjusts where each foot lands and how long each stance lasts, while a supervised-learned module predicts the difference between the nominal model and what actually happens, feeding that correction into the MPC's prediction. A sympathetic reader would care because the design keeps the predictable, constraint-respecting core of model-based control while shifting only the model-mismatch burden onto learning; the authors report that this reduces tuning sensitivity, improves training efficiency, and keeps a real robot balanced where the nominal controller fails.","feed_headline":"Two residual modules let an MPC quadruped survive payloads and kicks","feed_subtitle":"The nominal convex-MPC controller alone falls under a 6 kg load; adding learned corrections keeps it balancing.","key_machinery":"The central object is a pair of residual modules inserted into the nominal loop. A residual footstep module, trained with proximal policy optimization, outputs per-leg foothold and gait-phase corrections to the heuristic footstep planner. A residual dynamics module, trained with supervised learning, takes recent inertial, joint, and torque history and outputs the low-pass-filtered, 10 Hz difference between the nominal model's prediction and the observed state change. That residual term is added as a constant offset to the MPC's dynamics constraint over the 0.1 s horizon, so the optimization stays convex and fast. The design's work is to route each kind of model error to the appropriate learning paradigm, reinforcement learning for discrete contact choices and supervised regression for smooth continuous mismatch, while keeping the nominal controller as the safety and feasibility backbone.","core_discovery":"The central claim is that model mismatch in legged locomotion can be decomposed into two separate correction problems, discrete contact decisions and continuous dynamics errors, and that each can be learned by the method best suited to it, with both residuals layered on top of a nominal convex MPC. Under this design the paper reports that the full system stays balanced through a 6 kg payload and added disturbances, while the vanilla MPC fails in the payload condition and a dynamics-only variant fails when kicks are added; in simulation it achieves the highest success rates across payload levels and the lowest RMS velocity error in most conditions, and it outperforms an end-to-end RL baseline when the payload is placed outside the training distribution. The paper also claims the residual correction term can be predicted from proprioceptive history alone, making hardware deployment feasible without accurate contact or ground-reaction-force estimates.","pith_inferences":["Editorial inference: because the residual is low-pass filtered at 10 Hz and treated as constant over the 0.1 s horizon, the method is tuned for slow, quasi-static model errors; high-frequency impacts or terrain compliance would be smoothed out, so extending to dynamic or deformable terrain would likely require a higher cutoff or an in-horizon residual model.","Editorial inference: the paper's decomposition suggests a recipe for other model-based controllers: separate discrete decisions such as footholds, gait, or contact sequence from continuous model error, and choose the learning tool per component rather than learning one monolithic policy.","Editorial inference: the residual dynamics network is trained on simulation labels and never validated against measured real-world residuals, so a direct extension would be to log the analytical residual on hardware and fine-tune or validate the regressor online; this would also reveal whether the demonstrated stability is truly due to the predicted residual."],"forward_implications":["If the results hold, the same nominal MPC can be kept as a safety layer while small learned residuals compensate its model mismatch, avoiding a full end-to-end replacement and its sample costs.","Robots using this scheme should tolerate heavier payloads and pushes than the baseline MPC, and should do so without re-tuning the MPC's cost weights.","Because the dynamics residual is predicted from sensor history alone, deployment does not require high-fidelity estimates of ground reaction forces or contact states.","The modular split should make training converge faster than end-to-end RL or joint-space residual RL, as measured by the paper's learning curves.","Outside the training distribution, such as a payload attached to the legs rather than the trunk, velocity tracking should degrade less than for an end-to-end RL policy."],"supporting_citations":[{"why":"Supplies the nominal convex-MPC formulation with single-rigid-body dynamics that the residuals are layered onto.","marker":"[3]"},{"why":"Defines the heuristic footstep planner and gait-phase update that the residual footstep module corrects.","marker":"[2]"},{"why":"Provides the proximal-policy-optimization algorithm used to train the residual footstep policy.","marker":"[35]"},{"why":"Provides the physics simulation used to collect training data and to label the residual dynamics.","marker":"[36]"},{"why":"Serves as the end-to-end RL baseline for out-of-distribution tests and learning-efficiency comparison.","marker":"[7]"},{"why":"Serves as the RL-augmented MPC baseline with joint-space and dynamics residuals for the ablation study.","marker":"[29]"},{"why":"Used as the online residual-regression baseline the paper compares against as resdyn-window.","marker":"[16]"},{"why":"Used as the learned gait-transition baseline in the comparative analysis.","marker":"[23]"}],"fun_headline_variants":["Two learned residuals boost MPC for quadruped robustness","Modular residual learning makes MPC quadruped kick-proof","Hybrid control: model-based MPC plus learned residuals for legged robots","Residual modules fix MPC mismatches for real quadruped","Learned corrections keep MPC quadruped balanced under loads"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method relies on a neural network trained purely in simulation to predict the real robot's model error from its own recent sensor readings, and on that error staying roughly constant over the 0.1 s planning horizon; neither assumption is directly verified on the hardware.","fun_headline_variants_meta":{"raw":{"variants":["Two learned residuals boost MPC for quadruped robustness","Modular residual learning makes MPC quadruped kick-proof","Hybrid control: model-based MPC plus learned residuals for legged robots","Residual modules fix MPC mismatches for real quadruped","Learned corrections keep MPC quadruped balanced under loads"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00063,"raw_usage":{"total_tokens":2864,"prompt_tokens":854,"completion_tokens":2010,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":470,"completion_tokens_details":{"reasoning_tokens":1930}},"tokens_in":470,"tokens_out":2010,"duration_ms":13161,"temperature":1.0,"reasoning_tokens":1930,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:17:57.890388+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"During a repeated 6 kg payload and push-recovery trial, compare the network-predicted residual with the measured residual, defined as the difference between the nominal model's one-step prediction and the observed state change over the same control step. If the predicted residual is not significantly correlated with the measured residual at the moments when the robot regains balance, then the proposed compensation mechanism is not what is stabilizing the robot.","supporting_citations":[{"cited_title":"Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,","cited_arxiv_id":null,"evidence_quote":"Supplies the nominal convex-MPC formulation with single-rigid-body dynamics that the residuals are layered onto."},{"cited_title":"Per-contact iteration method for solving contact dynamics,","cited_arxiv_id":null,"evidence_quote":"Provides the physics simulation used to collect training data and to label the residual dynamics."},{"cited_title":"Learning agile locomotion and adaptive be- haviors via rl-augmented mpc,","cited_arxiv_id":null,"evidence_quote":"Serves as the RL-augmented MPC baseline with joint-space and dynamics residuals for the ablation study."},{"cited_title":"Fast and efficient locomotion via learned gait transitions,","cited_arxiv_id":null,"evidence_quote":"Used as the learned gait-transition baseline in the comparative analysis."}],"review_version":1}