REVIEW 4 major objections 6 minor 1 cited by
One simulation-trained policy can walk a 50 kg quadruped it never encountered, provided the joint controller gains are sampled broadly during training.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 11:01 UTC pith:5AWTGZZC
load-bearing objection Useful empirical comparison of PD-gain sampling strategies with a real zero-shot ANYmal deployment, but the headline claim about baselines is under-supported by the interpolated reimplementations. the 4 major comments →
Sampling Strategies for Robust Universal Quadrupedal Locomotion Policies
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a policy's robustness to unseen quadruped morphologies depends more on the distribution of joint PD gains seen in training than on the distribution of robot shapes. When gains are sampled uniformly over a wide range, a single policy trained on four reference quadrupeds with randomized masses, leg geometries, and centre-of-mass offsets can walk a real 50 kg quadruped at zero shot, while policies trained with linear or polynomial mass-to-gain mappings or with tight interpolation around nominal gains fail to walk on that hardware. The paper further shows that a particle-filter adaptive sampler, which reweights robot configurations according to whether command-tracking
What carries the argument
The load-bearing mechanism is the joint PD controller law tau = Kp(q_des − q) − Kd qdot, with the 24 gains (Kp, Kd per joint) treated as training parameters. The paper's novel instrument is a particle filter (sequential importance resampling) over robot configurations: each configuration's importance weight depends on whether linear and zero-velocity command tracking lie in the interval [0.4,0.9], and the filter resamples configurations via a random nearest-neighbour walk, while an adaptive sampling range SR grows or shrinks with aggregate tracking performance. This mechanism concentrates training on configurations that are neither too easy nor too hard, and it is paired with a full-range un
Load-bearing premise
The conclusion that wide-range PD gain sampling is necessary rests on the baselines being reimplemented faithfully; since the authors interpolated the baselines' mass-to-gain mappings and excluded some from hardware testing because of unsafe gains, a mis-reimplementation could make full-range sampling look necessary when it is not.
What would settle it
Run the comparison with the original implementations/checkpoints of the mass-mapped and narrow-gain baselines on the same 50 kg hardware quadruped; if any of them walks reliably at zero shot, the claim that wide full-range PD gain sampling is required is refuted. A cheaper falsifier: train a policy with wide uniform gains but without the particle filter and check whether hardware tracking errors remain in the same range.
If this is right
- A single policy can be deployed zero-shot across quadrupeds from 12 kg to 50 kg, eliminating per-robot retraining.
- Controllers whose gains are tied to robot mass (linearly or polynomially) or kept near nominal values will not walk on heavy platforms, regardless of morphology diversity.
- The success-rate plateau over PD gain combinations is wider with adaptive particle-filter sampling, implying greater tolerance to gain miscalibration on hardware.
- Exposing the velocity estimator to the full gain range early in training improves its accuracy, while adaptive sampling gives slightly better velocity-command tracking.
Where Pith is reading between the lines
- A simpler ablation, keeping morphology sampling fixed and toggling only the gain distribution, could confirm whether wide gain randomization is the active ingredient; if it is, the particle-filter complexity may be unnecessary for small robots.
- Wide gain randomization may act as a regularizer that forces the policy to rely on proprioceptive state rather than memorized dynamics; measuring the policy's sensitivity to gain perturbations after training could test this.
- Because the baselines were reimplemented with interpolated gain schedules and some were excluded from hardware testing due to unsafe gain ranges, a reproduction using the original baselines' own code is needed before taking 'necessity' literally.
- The same adaptive particle-filter scheme could be applied to other controller properties (torque limits, friction, latency) to see if robustness benefits generalize beyond gains.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies how to sample robot morphology parameters and joint PD gains when training a single 'universal' quadrupedal locomotion policy in RaiSim. It compares three PD-gain sampling strategies—mass-mapped linear/polynomial gains (GenLoco/MorAL style), narrow nominal/interpolated gains (URMA style), and broad uniform or particle-filter-adaptive sampling—and reports sim-to-sim robustness on A1 and ANYmal models plus zero-shot hardware deployment on ANYmal. The authors claim that broad, mass-independent PD-gain randomization is necessary to close the sim-to-real gap, and that their particle-filter-adaptive variant yields the most robust uniform policy.
Significance. If the comparative evidence were solid, the paper would make a useful empirical contribution to universal legged-locomotion policy learning: a real ANYmal zero-shot deployment with a single policy is a valuable data point, and the explicit comparison of gain-sampling strategies is well-motivated. The paper also ships no code or checkpoints, but the hardware evaluation and the breadth of simulated perturbations are genuine strengths. However, the central claim depends on fair reproduction of prior methods and on statistically supported hardware comparisons; both are currently under-supported.
major comments (4)
- [Section V-A; Tables III–IV] The headline conclusion—that baselines 'fail to transfer reliably' and that full-range PD-gain sampling is required—rests on reimplementations of GenLoco, ManyQuadrupeds, MorAL, and URMA. Section V-A describes the baseline gains as 'interpolate[d] between nominal values' (GenLoco, ManyQuadrupeds) or 'interpolated from the values given in the original work' (URMA), but no interpolation formula, nominal reference set, or validation against original code/checkpoints is reported. If the effective baseline gain distributions differ from the published algorithms, the comparison may reflect the reimplementation rather than the original methods. Please provide exact interpolation schemes, quantify the resulting gain distributions, and, where feasible, compare against original checkpoints or code.
- [Section V-A; Table III] The hardware evidence for baseline failure is not actually measured. GenLoco and ManyQuadrupeds are excluded a priori by the Kd > 1.5 safety cutoff, so their 'failure to transfer reliably' is asserted, not demonstrated. MorAL and URMA are evaluated with modified deployment gains (85.0:1.0) rather than their published nominal gains (82:5 and 80:2), and the nominal-gain rows are listed as NA. Moreover, Section V-A states that MorAL and URMA 'achieved stable standing but were unable to realize walking,' yet Table III reports walking RMSE for 'MorAL adaptive' and 'URMA adaptive.' Please clarify which policies actually walked, under which gains, and distinguish method re-tuning from failure of the original method.
- [Eq. (10), Section IV-D2] The particle-filter weight definition is not well-formed as written. Eq. (10) appears to mix a sum over commands with indicator conditions on Tr_lin and Tr_zero, and the expression cannot be parsed unambiguously. Since the adaptive particle-filter weighting is a core methodological contribution, please rewrite Eq. (10) as a precise definition of w_k^i, including normalization and the exact role of the 1/2 factor and the summation index 'cmd.'
- [Section V-B; Tables III–IV] The quantitative support is statistically thin. Section V-B reports SR* with NT=100 but gives no confidence intervals; Tables III and IV report single-point RMSE values with no trial counts, run durations, or error bars. Several comparisons in Table III (e.g., Particle Filt adaptive 85.0:0.5 vs Uniform 85.0:0.5) are within plausible noise. Please report per-condition trial counts, standard deviations/confidence intervals, and statistical tests for the central claims, especially the ANYmal hardware comparisons.
minor comments (6)
- [Section II-B] ManyQuadrupeds is described as relying on 'GPG' (Central Pattern Generators); this should be 'CPG'.
- [Throughout] 'SR' is used both for the sampling range (Eq. 9) and for the success rate SR* in Section V-B. The notation is confusing; consider renaming one of them.
- [Figure 4] ManyQuadrupeds is 'removed' from Fig. 4 because it 'performed worse than GenLoco.' Since ManyQuadrupeds is a named baseline in the comparison, the omission should be justified with data rather than a parenthetical removal.
- [Table II] The learning rate is listed as 'adaptive' with no value or schedule; please specify the initial value and decay schedule to make training reproducible.
- [References] The comparison relies heavily on reference [28] (an arXiv preprint) for the base architecture and hyperparameters. If a peer-reviewed version exists, cite it; otherwise state more explicitly which components are taken from the preprint.
- [Throughout] Spelling of 'RaiSim' is inconsistent ('Raisim' in several places); ensure consistent capitalization.
Circularity Check
No circular derivation: empirical RL comparison; self-citation of PAL is not load-bearing.
full rationale
The paper's central claim is that broad PD-gain randomization, especially with adaptive particle-filter sampling, produces a more robust universal locomotion policy. This is established by training PPO policies and measuring held-out simulation success rates (SR*) and hardware command-tracking RMSE against external baselines. No equation in the paper fits a parameter to the evaluation metric and then reports that fit as a prediction. The adaptive SR and particle-filter weighting (Eqs. 8-10) use the policy's command-tracking performance to choose which robot configurations to sample during training; this is a training-time curriculum, not a construction of the evaluation outcome. The architecture and reward follow the authors' prior PAL work [28], but that self-citation is not load-bearing for the comparison: the claimed advantage is tested against GenLoco, ManyQuadrupeds, MorAL, and URMA, and the hardware/simulation results are reported directly. The main weaknesses are reproducibility/fairness concerns - baseline PD gains are interpolated rather than taken from released checkpoints, and GenLoco/ManyQuadrupeds were excluded from hardware by a derivative-gain safety threshold - but these are correctness risks, not circularity. I find no step where a 'prediction' reduces to its input by definition.
Axiom & Free-Parameter Ledger
free parameters (6)
- SR curriculum thresholds (Eq. 9) =
increase if Tr_lin>0.65 and Tr_zero>0.55; decrease if Tr_lin<0.55 or Tr_zero<0.40; step ±0.01; initial SR=0.1
- Particle filter performance interval (Eq. 10) =
Tr_lin, Tr_zero in [0.4, 0.9]
- Particle filter/curriculum sample split =
n_u = n_r = n_w = 10% of n_p
- Deployment PD gains on ANYmal (Table III) =
Kp ∈ {80, 82.5, 85}, Kd ∈ {0.5, 1.0}
- Per-joint PD gain perturbation =
c_PDij * clip(1+ε, 0.95, 1.05), ε~N(0,1)
- Reward weights (Table I) =
weights from PAL [28] (e.g., base height -20, air time -6, velocity 3.0)
axioms (6)
- domain assumption RaiSim physics is a valid stand-in for real quadruped dynamics for sim-to-real transfer.
- domain assumption Four reference models (A1, Aliengo, ANYmal B, ANYmal C) plus uniform randomization cover the morphology space needed for a 'universal' policy.
- domain assumption Baseline methods are faithfully re-implemented/interpolated from their papers so that performance differences reflect sampling strategy rather than implementation artifacts.
- standard math PPO with the given hyperparameters converges to a policy representative of each sampling strategy.
- ad hoc to paper Manual selection of deployment PD gains on ANYmal (Kp 80-85, Kd 0.5-1.0) is representative of each trained policy's operational range.
- domain assumption A robot that can stand collision-free for 2 s in RaiSim is a valid training configuration.
read the original abstract
This work focuses on sampling strategies of configuration variations for generating robust universal locomotion policies for quadrupedal robots. We investigate the effects of sampling physical robot parameters and joint proportional-derivative gains to enable training a single reinforcement learning policy that generalizes to multiple parameter configurations. Three fundamental joint gain sampling strategies are compared: parameter sampling with (1) linear and polynomial function mappings of mass-to-gains, (2) performance-based adaptive filtering, and (3) uniform random sampling. We improve the robustness of the policy by biasing the configurations using nominal priors and reference models. All training was conducted using the RaiSim simulation environment, tested in simulation on a range of diverse quadrupeds, and zero-shot deployed onto hardware using the ANYmal quadruped robot. Compared to multiple baseline implementations, our results demonstrate the need for significant joint controller gains randomization for robust closing of the sim-to-real gap.
Figures
Forward citations
Cited by 1 Pith paper
-
Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots
Empirical comparison of blind, critic-perceptive, and fully perceptive variants of morphology-aware RL locomotion controllers shows critic-only perception improves robustness over blind baselines while remaining more ...
Reference graph
Works this paper leans on
-
[1]
Crocoddyl: An efficient and versatile framework for multi-contact optimal control,
C. Mastalli, R. Budhiraja, W. Merkt, G. Saurel, B. Hammoud, M. Naveau, J. Carpentier, S. Vijayakumar, and N. Mansard, “Crocoddyl: An efficient and versatile framework for multi-contact optimal control,” in2020 International Conference on Robotics and Automation. IEEE. [Online]. Available: http://arxiv.org/abs/1909. 04947
1909
-
[2]
Perceptive locomotion through nonlinear model-predictive control,
R. Grandia, F. Jenelten, S. Yang, F. Farshidian, and M. Hutter, “Perceptive locomotion through nonlinear model-predictive control,” in2023 Transactions on Robotics. IEEE. [Online]. Available: https://ieeexplore.ieee.org/document/10138309/
-
[3]
Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao, “Robot parkour learning,” in2023 Conference on Robot Learning. [Online]. Available: http://arxiv.org/abs/2309.05665
-
[4]
High-speed control and navigation for quadrupedal robots on complex and discrete terrain,
H. Kim, H. Oh, J. Park, Y . Kim, D. Youm, M. Jung, M. Lee, and J. Hwangbo, “High-speed control and navigation for quadrupedal robots on complex and discrete terrain,” in2025 Robotics and Au- tomation Letters. IEEE
-
[5]
F. Jenelten, J. He, F. Farshidian, and M. Hutter, “DTC: Deep tracking control,” in2024 Science Robotics, vol. 9. [Online]. Available: https://www.science.org/doi/10.1126/scirobotics.adh5401
-
[6]
Robotics
U. Robotics. A1. [Online]. Available: https://www.unitree.com/en/a1/
-
[7]
ANYmal d
ANYbotics. ANYmal d. [Online]. Available: https://www.anybotics. com/robotics/anymal/
-
[8]
Learning agile and dynamic motor skills for legged robots| science robotics. [Online]. Available: https://www.science.org/doi/full/ 10.1126/scirobotics.aau5872
-
[9]
Learning robust perceptive locomotion for quadrupedal robots in the wild,
T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,” in2022 Science Robotics, vol. eabk2822. [Online]. Available: https://www.science.org/doi/10.1126/scirobotics.abk2822
-
[10]
Learning quadrupedal locomotion on deformable terrain,
S. Choi, G. Ji, J. Park, H. Kim, J. Mun, J. H. Lee, and J. Hwangbo, “Learning quadrupedal locomotion on deformable terrain,” in2023 Science Robotics, vol. 8. [Online]. Available: https://www.science.org/doi/10.1126/scirobotics.ade2256
-
[11]
Learning robust autonomous navigation and locomotion for wheeled-legged robots,
J. Lee, M. Bjelonic, A. Reske, L. Wellhausen, T. Miki, and M. Hutter, “Learning robust autonomous navigation and locomotion for wheeled-legged robots,” in2024 Science Robotics. [Online]. Available: http://arxiv.org/abs/2405.01792
-
[12]
Learning coor- dinated badminton skills for legged manipulators,
Y . Ma, A. Cramariuc, F. Farshidian, and M. Hutter, “Learning coor- dinated badminton skills for legged manipulators,” in2025 Science Robotics, vol. 10
-
[13]
Learning agile robotic locomotion skills by imitating animals,
X. B. Peng, E. Coumans, T. Zhang, T. W. E. Lee, J. Tan, and S. Levine, “Learning agile robotic locomotion skills by imitating animals,” in arXiv preprint arXiv:2004.00784
Pith/arXiv arXiv 2004
-
[14]
Learning agile skills via adversarial imitation of rough partial demonstrations,
C. Li, M. Vlastelica, S. Blaes, J. Frey, F. Grimminger, and G. Mar- tius, “Learning agile skills via adversarial imitation of rough partial demonstrations,” in2023 Conference on Robot Learning
-
[15]
Adversarial motion priors make good substitutes for complex reward functions,
A. Escontrela, X. B. Peng, W. Yu, T. Zhang, A. Iscen, K. Goldberg, and P. Abbeel, “Adversarial motion priors make good substitutes for complex reward functions,” in2022 International Conference on Intelligent Robots and Systems. IEEE. [Online]. Available: https://ieeexplore.ieee.org/document/9981973/
-
[16]
OPT-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors,
Y . Fuchioka, Z. Xie, and M. v. d. Panne, “OPT-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors,” in 2023 International Conference on Robotics and Automation. IEEE. [Online]. Available: http://arxiv.org/abs/2210.01247
Pith/arXiv arXiv 2023
-
[17]
Sim- to-real transfer of robotic control with dynamics randomization,
X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim- to-real transfer of robotic control with dynamics randomization,” in 2018 International Conference on Robotics and Automation. IEEE. [Online]. Available: https://ieeexplore.ieee.org/document/8460528/
arXiv 2018
-
[18]
Learning locomotion skills for cassie: Iterative design and sim-to- real,
Z. Xie, P. Clary, J. Dao, P. Morais, J. Hurst, and M. Van De Panne, “Learning locomotion skills for cassie: Iterative design and sim-to- real,” in2020 Conference on Robotic Learning
-
[19]
One policy to control them all:shared modular policies for agent-agnostic control,
W. Huang, I. Mordatch, and D. Pathak, “One policy to control them all:shared modular policies for agent-agnostic control,” in2020 International Conference on Machine Learning
-
[20]
MetaMorph: Learning universal controllers with transformers,
A. Gupta, L. Fan, S. Ganguli, and L. Fei-Fei, “MetaMorph: Learning universal controllers with transformers,” in2022 International Conference on Learning Representations. [Online]. Available: https: //openreview.net/forum?id=Opmqtk GvYL
-
[21]
AnyMorph: Learning transferable polices by inferring agent morphology,
B. Trabucco, M. Phielipp, and G. Berseth, “AnyMorph: Learning transferable polices by inferring agent morphology,” in2022 International conference on machine learning. PMLR. [Online]. Available: http://arxiv.org/abs/2206.12279
-
[22]
Model-agnostic meta-learning for fast adaptation of deep networks,
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in2017 International conference on machine learning. PMLR
-
[23]
Meta reinforcement learning for optimal design of legged robots,
A. Belmonte-Baeza, J. Lee, G. Valsecchi, and M. Hutter, “Meta reinforcement learning for optimal design of legged robots,” in2022 Robotics and Automation Letters, vol. 7. IEEE. [Online]. Available: https://ieeexplore.ieee.org/document/9910025/
-
[24]
GenLoco: Generalized locomotion controllers for quadrupedal robots,
G. Feng, H. Zhang, Z. Li, X. B. Peng, B. Basireddy, L. Yue, Z. Song, L. Yang, Y . Liu, K. Sreenath, and S. Levine, “GenLoco: Generalized locomotion controllers for quadrupedal robots,” in 2022 Conference on robot learning. PMLR. [Online]. Available: http://arxiv.org/abs/2209.05309
Pith/arXiv arXiv 2022
-
[25]
ManyQuadrupeds: Learning a single locomotion policy for diverse quadruped robots,
M. Shafiee, G. Bellegarda, and A. Ijspeert, “ManyQuadrupeds: Learning a single locomotion policy for diverse quadruped robots,” inarXiv preprint arXiv:2310.10486. arXiv. [Online]. Available: http://arxiv.org/abs/2310.10486
-
[26]
Z. Luo, Y . Dong, X. Li, R. Huang, Z. Shu, E. Xiao, and P. Lu, “MorAL: Learning morphologically adaptive locomotion controller for quadrupedal robots on challenging terrains,” in 2024 Robotics and Automation Letters. IEEE. [Online]. Available: https://ieeexplore.ieee.org/document/10463132/
arXiv 2024
-
[27]
One policy to run them all: an end-to-end learning approach to multi-embodiment locomotion,
N. Bohlinger, G. Czechmanowski, M. Krupka, P. Kicki, K. Walas, J. Peters, and D. Tateo, “One policy to run them all: an end-to-end learning approach to multi-embodiment locomotion,” in2024 Conference on Robot Learning. [Online]. Available: http://arxiv.org/abs/2409.06366
-
[28]
D. Rytz, S. Choi, W. Yu, W. Merkt, J. Hwangbo, and I. Havoutis, “Reference free platform adaptive locomotion for quadrupedal robots using a dynamics conditioned policy,” inarXiv preprint arXiv:2505.16042. [Online]. Available: http: //arxiv.org/abs/2505.16042
-
[29]
Learning quadrupedal locomotion over challenging terrain,
J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,” in2020 Science Robotics, vol. 5. [Online]. Available: https://www.science. org/doi/10.1126/scirobotics.abc5986
-
[30]
RaiSim – v1.1.7 documentation
R. Tech, “RaiSim – v1.1.7 documentation.” [Online]. Available: https://raisim.com/
-
[31]
ANYmal - a highly mobile and dynamic quadrupedal robot,
M. Hutter, C. Gehring, D. Jud, A. Lauber, C. D. Bellicoso, V . Tsounis, J. Hwangbo, K. Bodie, P. Fankhauser, M. Bloesch, R. Diethelm, S. Bachmann, A. Melzer, and M. Hoepflinger, “ANYmal - a highly mobile and dynamic quadrupedal robot,” in2016 International Con- ference on Intelligent Robots and Systems. IEEE, ISSN: 2153-0866
-
[32]
Ackerman
E. Ackerman. ANYbotics introduces sleek new ANYmal c quadruped. [Online]. Available: https://spectrum.ieee.org/ anybotics-introduces-sleek-new-anymal-c-quadruped
-
[33]
R. Tech. Articulated systems — RaiSim v1.1.7 documentation. [Online]. Available: https://raisim.com/sections/ArticulatedSystem. html
-
[34]
Sim-to-real: Learning agile locomotion for quadruped robots,
J. Tan, T. Zhang, E. Coumans, A. Iscen, Y . Bai, D. Hafner, S. Bohez, and V . Vanhoucke, “Sim-to-real: Learning agile locomotion for quadruped robots,” in2018 Robotics: Science and Systems. MIT Press Journals. [Online]. Available: http://arxiv.org/abs/1804.10332
-
[35]
Dynamics randomization revisited: A case study for quadrupedal locomotion,
Z. Xie, X. Da, M. van de Panne, B. Babich, and A. Garg, “Dynamics randomization revisited: A case study for quadrupedal locomotion,” in2021 IEEE International Conference on Robotics and Automation. [Online]. Available: https://ieeexplore.ieee.org/document/9560837/
-
[36]
Proximal policy optimization algorithms,
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” inarXiv preprint. [Online]. Available: https://arxiv.org/abs/1707.06347
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.