REVIEW 3 major objections 4 minor 18 references
MULE: Multi-terrain and Unknown Load Adaptation for Effective Quadrupedal Locomotion
T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read A two-policy RL controller lets a quadruped carry changing payloads across terrain without explicit load estimation.
desk verdict A useful two-phase adaptive RL idea with real hardware evidence, but the GRF reward's training-time use of the true payload mass and the lack of quantitative ablations keep the central claim from being fully established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the two-phase PPO training scheme plus a corrective action $\Delta a_t$ produced by an adaptive policy $\pi_\phi$ that receives the nominal observations augmented with estimated foot forces. Foot forces are estimated from joint torques via the Jacobian pseudoinverse $f = (J(\theta)^T)^\dagger \tau$, and the adaptive policy is rewarded by a GRF tracking term $r_{\mathrm{GRF}}$ that encourages total foot forces above $(m_r + m_p)g$ whenever height is below command. The combined action $a_t + \Delta a_t$ is applied to the environment, and the adaptive reward weights are set so that the corrective policy focuses on height stabilization and ground reaction force generation while the nominal policy keeps velocity tracking.
What would settle it
Retrain the adaptive policy with the GRF tracking reward threshold set to robot mass only, $(m_r + 0)g$, instead of $(m_r + m_p)g$, then deploy under a 6 kg payload: if height tracking error grows to the baseline's level, the reported adaptation was enabled by training-time knowledge of the true payload rather than by proprioceptive adaptation.
Extended reading notes
Core claim
The central claim is that an adaptive policy, trained in a second phase alongside a frozen nominal policy, produces corrective actions whose magnitude tracks the payload: when the body height drops below the commanded height, the adaptive policy increases the estimated foot forces and restores tracking. The paper validates this by comparing the adaptive controller against a baseline on flat ground, stairs, and slopes, and by showing on hardware that the baseline becomes unstable or stalls under high payloads while the adaptive controller continues. The authors claim this works without explicit payload parameter estimation and without predefined gait schedules, relying instead on the corrective policy's learned response to observed state and estimated foot forces.
Load-bearing premise
The training reward for corrective actions uses the true payload weight in its force threshold, even though the deployed controller never sees the payload; if the learned response actually relies on that hidden information, the claim of unknown-load adaptation is much weaker.
Editorial extensions
If this is right
- On flat ground, the adaptive controller reduces body-height tracking error that otherwise grows with payload, while velocity tracking stays comparable to the baseline.
- On stairs, the baseline controller stalls under high payloads, while the adaptive controller maintains forward progression and lower height and velocity tracking errors.
- On hardware, the adaptive controller carries progressive loads up to 10 kg with stable locomotion, whereas the baseline shows foot scuffing and instability at higher payloads.
- The norm of the corrective action rises and falls with each payload change, suggesting the policy acts as an implicit load detector without being explicitly told to intervene.
- Because no gait schedule or manual gain tuning is used, the same two-phase recipe could apply to other command-tracking tasks beyond height and velocity.
Reading between the lines
- The GRF tracking reward uses the true payload mass $m_p$ in its threshold during training, so the decisive control experiment would be to retrain the adaptive policy with a payload-invariant threshold; the paper does not report this ablation.
- Since foot forces are estimated from commanded torques through a Jacobian pseudoinverse, the policy's corrective behavior may degrade on soft or slippery ground where the commanded torque-to-force mapping is unreliable; that setting is outside the reported tests.
- The corrective-policy architecture is agnostic to what causes the tracking error, so the same two-phase design could be trained for external pushes, uneven step heights, or actuator degradation without changing the framework.
- A testable extension is to provide the latent context vector from the encoder to the adaptive policy and train the two policies end-to-end, allowing the nominal policy's state estimate to change rather than only offsetting actions.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an adaptive reinforcement learning framework for quadruped locomotion under unknown payloads and varying terrains. A Phase-1 nominal policy is trained for baseline locomotion, and a Phase-2 adaptive policy is trained to output corrective actions using an augmented observation that includes estimated foot forces. Training is done in Isaac Gym on a Unitree Go1 model, and deployment is demonstrated on a physical Go1 with static and dynamic payloads. The central claim is that the adaptive controller consistently outperforms a DreamWaQ-based baseline in tracking body height and velocity commands on flat ground, slopes, and stairs.
Significance. If the empirical claims hold, the two-phase training scheme is a practical alternative to explicit payload estimation and to conservative domain randomization, with a hardware demonstration on a real quadruped. The paper also correctly identifies a relevant gap: model-based payload adaptation often relies on predefined gaits or trajectory generators. However, the evidence provided is mostly qualitative raw time series, with no quantitative summary statistics, error bars, or ablations, so the strength of the central claim is not yet commensurate with the level of the assertion. The paper does not ship code or reproducible training configurations, which limits independent verification.
major comments (3)
- [III.A and III.B] The central empirical claim that the adaptive controller 'consistently outperformed' the baseline is supported only by qualitative descriptions and raw time-series plots (Figures 4, 5, and 6). No means, standard deviations, error bars, RMSE/MAE values, or success rates are reported for height or velocity tracking, and no trial counts are given for the hardware experiments. This is load-bearing: without aggregated quantitative results, the reader cannot assess the statistical significance or the magnitude of the claimed improvement.
- [II.E, GRF tracking reward] The GRF tracking reward uses the exact payload mass in the threshold (m_r + m_p)g. Although the deployed policy does not observe m_p, the training-time reward provides a precise payload-dependent force target, and the policy's observation includes the same estimated foot forces that appear in the reward. The claim of 'unknown load adaptation' is therefore not fully separated from training-time access to the payload. The authors should provide an ablation that removes m_p from the reward (e.g., a fixed or mass-independent threshold), an analysis of robustness to force-estimator bias or noise, and a quantitative comparison between estimated and ground-truth foot forces. Without such evidence, the paper does not establish that the learned corrective behavior is a genuine load-adaptation mechanism rather than a reward-shaped response to a known force threshold.
- [III.A, baseline comparison] The baseline is described only as 'DreamWaQ, with the base mass randomized in the range [0, 10] kg.' No details are given about the baseline's training configuration, architecture, reward weights, observation space, or training budget, making it impossible to judge whether the comparison is fair. In addition, the paper does not compare the adaptive policy against its own Phase-1 nominal policy without the corrective term, which would be a more direct ablation of the proposed contribution. The claim of consistent superiority is also weakened by the absence of quantitative results for slopes in the main text and by the lack of hardware plots for slopes and stairs.
minor comments (4)
- [II.D, observations] The observation vector is written as o_t = [w_t, g_t, c_t, θ_t, θ̇_t, a_{t-1}], but the text uses 'wt' and 'gt' in the description; please use consistent notation for the angular velocity and gravity vector.
- [III.A, Figure 5] The caption refers to 'flat red segments' in the curves, but the figure itself is not included in the text and the meaning of the red segments is not defined. Please clarify what the red color indicates and describe the failure mode explicitly.
- [III.B, hardware experiments] The hardware section reports qualitative observations for slopes and stairs but does not show any plots or metrics for those terrains. Given the abstract's claim of 'across all settings,' quantitative hardware results for slopes and stairs should be included or the scope stated more precisely.
- [General] The reference list contains several self-citations and recent arXiv preprints, but no code or data availability statement is given. For an empirical RL paper with a project website, releasing training configurations and evaluation scripts would greatly improve reproducibility.
Circularity Check
No circularity: the paper's central claim is an empirical RL comparison, and no load-bearing step reduces to its own inputs by construction.
full rationale
The central claim is empirical: an adaptive RL controller is compared against a DreamWaQ-based baseline in Isaac Gym and on a Unitree Go1, and the reported improvements in height and velocity tracking are benchmark results rather than outputs derived from the method's assumptions. The two-phase PPO training and the hand-specified reward functions do not define the claimed outcome in terms of the inputs; the adaptive corrective policy is trained to maximize a reward, not fitted to reproduce a precomputed prediction. The CE Net loss is explicitly imported from external prior work ([9]), and the only self-citation ([12], PIP-Loco) appears in a background list of RL locomotion methods and is not load-bearing for any of the paper's claims. The GRF tracking reward r_GRF in Section II.E does use the true payload mass mp in its force threshold (mr+mp)g, so the training reward encodes payload information that the deployed policy never observes; this is a legitimate ablation and experimental-design concern, but it is not circularity, because the paper does not derive or predict mp and the empirical comparison is not true by construction. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no known result is repackaged as new. The claimed behavior remains an empirical finding, so the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- GRF reward coefficients =
0.75 and 0.50
- Payload sampling range =
initial [0,1] kg, resampled [0,2.5] kg every 4 s
- Hardware PD gains =
kp=20.0, kd=0.5
assumptions (3)
- domain assumption The Jacobian pseudoinverse foot force estimate f=(J^T)^+ tau is sufficiently accurate for the adaptive policy's observations.
- domain assumption Simulation in Isaac Gym with a pre-trained actuator network (from [8][18]) transfers to the Unitree Go1 without additional sim-to-real tuning.
- domain assumption The nominal policy retains its Phase-1 behavior when trained jointly in Phase 2 with the adaptive policy and separate rewards.
Cite this review
Pith. "Pith review of MULE: Multi-terrain and Unknown Load Adaptation for Effective Quadrupedal Locomotion." pith.science (2026). https://pith.science/paper/QRC3WHWB
@misc{pith2026250500488,
author = {Pith},
title = {Pith review of: MULE: Multi-terrain and Unknown Load Adaptation for Effective Quadrupedal Locomotion},
year = {2026},
howpublished = {\url{https://pith.science/paper/QRC3WHWB}},
note = {Machine review of arXiv:2505.00488}
}
read the original abstract
Quadrupedal robots are increasingly deployed for load-carrying tasks across diverse terrains. While Model Predictive Control (MPC)-based methods can account for payload variations, they often depend on predefined gait schedules or trajectory generators, limiting their adaptability in unstructured environments. To address these limitations, we propose an Adaptive Reinforcement Learning (RL) framework that enables quadrupedal robots to dynamically adapt to both varying payloads and diverse terrains. The framework consists of a nominal policy responsible for baseline locomotion and an adaptive policy that learns corrective actions to preserve stability and improve command tracking under payload variations. We validate the proposed approach through large-scale simulation experiments in Isaac Gym and real-world hardware deployment on a Unitree Go1 quadruped. The controller was tested on flat ground, slopes, and stairs under both static and dynamic payload changes. Across all settings, our adaptive controller consistently outperformed the controller in tracking body height and velocity commands, demonstrating enhanced robustness and adaptability without requiring explicit gait design or manual tuning.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[12]
Pip-loco: A proprioceptive infinite horizon planning framework for quadrupedal robot locomotion,
A. Shirwatkar, N. Saxena, K. Chandra, and S. Kolathaya, “Pip-loco: A proprioceptive infinite horizon planning framework for quadrupedal robot locomotion,” 2024
2024
-
[1]
Online payload identification for quadruped robots,
G. Tournois, M. Focchi, A. Del Prete, R. Orsolino, D. G. Caldwell, and C. Semini, “Online payload identification for quadruped robots,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4889–4896, 2017
work page 2017
-
[2]
Online learning of unknown dynamics for model-based controllers in legged locomotion,
Y . Sun, W. L. Ubellacker, W.-L. Ma, X. Zhang, C. Wang, N. V . Csomay-Shanklin, M. Tomizuka, K. Sreenath, and A. D. Ames, “Online learning of unknown dynamics for model-based controllers in legged locomotion,” IEEE Robotics and Automation Letters , vol. 6, no. 4, pp. 8442–8449, 2021
2021
-
[3]
Adaptive Force-based Control for Legged Robots
M. Sombolestan, Y . Chen, and Q. Nguyen, “Adaptive force-based control for legged robots,” CoRR, vol. abs/2011.06236, 2020
work page Pith review arXiv 2011
-
[4]
Robust convex model predictive control for quadruped locomotion under uncertainties,
S. Xu, L. Zhu, H.-T. Zhang, and C. P. Ho, “Robust convex model predictive control for quadruped locomotion under uncertainties,” IEEE Transactions on Robotics , vol. 39, no. 6, pp. 4837–4854, 2023
work page 2023
-
[5]
Adaptive clf-mpc with application to quadrupedal robots,
M. V . Minniti, R. Grandia, F. Farshidian, and M. Hutter, “Adaptive clf-mpc with application to quadrupedal robots,” IEEE Robotics and Automation Letters, vol. 7, p. 565–572, Jan. 2022
work page 2022
-
[6]
Learning agile locomotion and adaptive behaviors via rl-augmented mpc,
Y . Chen and Q. Nguyen, “Learning agile locomotion and adaptive behaviors via rl-augmented mpc,” 2024
work page 2024
-
[7]
Rma: Rapid motor adaptation for legged robots,
A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,” 2021
2021
Show all 18 references
-
[8]
Walk these ways: Tuning robot control for generalization with multiplicity of behavior,
G. B. Margolis and P. Agrawal, “Walk these ways: Tuning robot control for generalization with multiplicity of behavior,” 2022
2022
-
[9]
Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning,
I. M. A. Nahrendra, B. Yu, and H. Myung, “Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning,” 2023
2023
-
[10]
Learning to walk in minutes using massively parallel deep reinforcement learning,
N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” CoRR, vol. abs/2109.11978, 2021
2021 arXiv
-
[11]
Hybrid internal model: Learning agile legged locomotion with simulated robot response,
J. Long, Z. Wang, Q. Li, L. Cao, J. Gao, and J. Pang, “Hybrid internal model: Learning agile legged locomotion with simulated robot response,” in The Twelfth International Conference on Learning Representations, 2024
2024
-
[13]
Domain randomization via entropy maximization,
G. Tiboni, P. Klink, J. Peters, T. Tommasi, C. D’Eramo, and G. Chal- vatzaki, “Domain randomization via entropy maximization,” 2024
2024
-
[14]
Revisiting domain ran- domization via relaxed state-adversarial policy optimization,
Y .-H. Lien, P.-C. Hsieh, and Y .-S. Wang, “Revisiting domain ran- domization via relaxed state-adversarial policy optimization,” in Pro- ceedings of the 40th International Conference on Machine Learning (A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarle...
2023
-
[15]
Ro- bust model based reinforcement learning using L1 adaptive control,
M. Sung, S. H. Karumanchi, A. Gahlawat, and N. Hovakimyan, “Ro- bust model based reinforcement learning using L1 adaptive control,” 2024
2024
-
[16]
Safe reinforcement learning with dual robustness,
Z. Li, C. Hu, Y . Wang, Y . Yang, and S. E. Li, “Safe reinforcement learning with dual robustness,” 2023
2023
-
[17]
Proximal policy optimization algorithms,
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017
2017
-
[18]
Learning agile and dynamic motor skills for legged robots,
J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,” Science Robotics , vol. 4, Jan. 2019
2019
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.