REVIEW 3 major objections 7 minor 31 references
Discovery of skill switching criteria for learning agile quadruped locomotion
T0 review · 3 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read This paper claims that a quadruped robot can learn trotting, bounding, galloping, and fall recovery in one policy, with the distances that trigger each gait switch found automatically by an outer optimization loop rather than hand-set.
desk verdict A plausible hierarchical RL recipe for multi-gait quadruped locomotion with a real galloping and fall-recovery demo, but the evaluation is so thin that the central claim is not yet supported quantitatively. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the multiplicative composition of pre-trained Gaussian skill policies, following the form $\pi(a | s, g) = \frac{1}{Z(s,g)} \prod_{i=1}^n \pi_i(a | s)^{w_i(s,g)}$, where $w_i$ are nonnegative weights from a gating network and $Z$ normalizes. Because the primitives are Gaussian, the composite is again Gaussian with closed-form mean and precision updates, avoiding the blending artifacts of additive mixtures. The gating network receives the robot state plus the normalized distance to the goal and outputs weights summing to one; the expert policies stay frozen. The second mechanism is the outer-loop optimization: gait switch distances $x_1, x_2$ enter the reference foot-contact reward (trot for $|d| < x_1$, bound for $x_1 \le |d| < x_2$, gallop for $|d| \ge x_2$) and are updated by CMA-ES alongside inner-loop reinforcement learning of the gating network.
What would settle it
Run the learned multi-skill policy on a real Unitree A1 for repeated long-duration goal-tracking episodes (e.g., at least 60 seconds including a sustained gallop phase) using the same joystick goal commands as the paper; count how often the robot falls or needs to invoke the recovery skill during galloping. If the robot cannot complete most episodes without a fall, the claim that the framework naturally produces stable galloping transitions in the real world is not supported.
Extended reading notes
Core claim
The central discovery is that gait switch criteria can be discovered automatically by treating them as decision variables in an optimization problem that runs outside the motor-learning loop. Two distances, x1 and x2, separating trot-to-bound and bound-to-gallop, are inserted into the reference foot-contact reward used to train a high-level gating network; covariance-matrix adaptation (CMA-ES) then updates x1 and x2 to maximize goal-tracking reward. The learned values in the reported run are 2.2 m and 4.3 m, and they are not directly where transitions occur: they shape the reward, while the gating network learns natural transition behavior. The framework's second claim is that multiplicative composition of pre-trained Gaussian skill policies, with weights produced by the gating network, yields a smooth composite policy that outperforms manually switching between single-skill policies, which failed at the bound-to-gallop transition in the real world.
Load-bearing premise
The whole real-world result rests on the assumption that simulation-trained expert policies and the learned velocity estimator transfer to the physical Unitree A1 with no adaptation, even though the paper's own limitations section reports that this transfer is imperfect and makes sustained galloping unstable.
Editorial extensions
If this is right
- A robot operator no longer needs to hand-select when a quadruped changes gait; the criteria emerge from optimizing task reward.
- Because the expert policies are frozen, adding a new skill requires only training a new single-skill expert and letting the gating network learn how to blend it in, rather than relearning a monolithic policy from scratch.
- Fall recovery can live inside the same composite policy as high-speed gaits, so the robot can go directly from a failure to trotting without a separately triggered recovery controller.
- The optimized switch criteria are coupled to the task's cost function, so changing the objective (energy, time, terrain) should produce different switching behavior without changing the learning architecture.
Reading between the lines
- A natural next test is to feed the gating network body-position-based world-frame goals, as used in simulation; if galloping becomes stable with that input, the paper's stated real-world limitation is a feedback issue rather than a fundamental policy limitation.
- The same outer-loop discovery could be applied to other discrete decisions in legged control, such as when to switch between walking modes on stairs or when to trigger a recovery motion, whenever a threshold can be encoded in the reward.
- The claim that switch criteria are 'automatically discovered' is specific to the chosen reward and optimization horizon; with a different goal-tracking reward the discovered distances would change, so the criteria are not intrinsic properties of the robot or the gaits.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a hierarchical framework for multi-skill quadruped locomotion. Low-level SAC policies for fall recovery, trotting, bounding, and galloping are trained separately using foot-contact-pattern rewards; a high-level gating network composes these fixed expert policies multiplicatively to track a goal; and an outer CMA-ES loop optimizes two distance thresholds (x1 and x2) that activate different foot-contact-pattern rewards during the multi-skill training. The authors report simulation training and a real-world deployment on a Unitree A1 with a joystick-based command, showing trot-bounding-galloping transitions and fall recovery, plus a qualitative comparison against a manual-switch baseline.
Significance. If substantiated, the framework would be a useful contribution: it demonstrates a way to combine multiple dynamic gaits without reference motions and to tune a small number of behavioral switching parameters via outer-loop optimization while reusing fixed expert policies. The multiplicative composition formulation (Eqs. 2-4) and the explicit outer-loop CMA-ES formulation are clear and reproducible in principle, and the optimized thresholds (x1=2.2 m, x2=4.3 m) constitute a falsifiable quantitative prediction about where gait transitions should occur. The paper is also explicitly honest about its main limitation, namely that real-world galloping is not reliable over long periods, which is a strength of the presentation. However, the key real-world claims currently rest on a single qualitative demonstration and on a command-representation mismatch that the authors themselves acknowledge, so the significance is not yet fully supported.
major comments (3)
- [Sections 2.2.4 and 4.2] The optimized criteria x1 and x2 are parameters of the reward function, not directly validated as the distances at which gait transitions actually occur. Section 4.2 states that "the optimized gait switch criteria are not exactly where the gait transitions occur in practice since they are only incorporated in reward functions." The paper never reports the actual distances at which the gating network switches dominant expert weights in the deployed policy, nor does it compare the optimized criteria against fixed or manually chosen criteria in an ablation. Because the title and abstract label these thresholds as "discovered skill switching criteria," the claim is not yet supported. I request that the authors extract transition distances from the real-world logs (e.g., from the weight traces in Fig. 7b) and compare them to x1 and x2, and that they include an ablation with fixed thresholds to show that the outer-loop optimization changes the resulting behavior in a measurable way.
- [Sections 4.3 and 4.4] The real-world evaluation consists of a single 20-second joystick-driven trajectory (Figs. 4, 6, 7, 9) and one failed manual-switch baseline run (Section 4.4). There are no repeated trials, no success rates, no per-gait speed or duration statistics, no transition-smoothness metrics, and no error bars. Section 5 concedes that galloping "cannot ensure 100% success rate ... for very long periods" in the real world. Since the abstract and contributions claim successful real-world trotting, bounding, galloping, and their natural transitions, the evidence is anecdotal. I request repeated trials with clearly defined success criteria (e.g., sustained galloping for a minimum duration without fall, successful transition between each gait pair), and quantitative comparison with the manual-switch baseline (e.g., failure rates, maximum continuous galloping duration, and velocity-profile variance).
- [Sections 4.1 and 5] The sim-to-real validation is weakened by a known input-representation mismatch. Training uses goal positions in the world frame (Section 2.2), while real-world deployment uses a joystick-based normalized relative goal distance in the robot heading frame (Section 4.1), which Section 5 admits "is not possible to reproduce the same goal commands as in simulation via joystick." The velocity estimator is also trained purely in simulation and is explicitly out-of-distribution during galloping (Section 5). The most distinctive part of the paper's claim, sustained and natural galloping with transitions, is exactly the regime where the authors acknowledge the sim-to-real gap is largest. To make the central claim load-bearing, the paper should either train and evaluate the multi-skill policy with the same command representation used in deployment, or provide a quantitative analysis of the velocity-estimation error and its effect on the gating network's behavior during galloping.
minor comments (7)
- [Eq. (1)] The notation "bx" in the RBF formula appears to be a typo for \hat{x}; please fix the typography and ensure the hat is rendered consistently in Table 1.
- [Section 2.2.4] The symbols r_hz and r_phi in the full target-following reward rg = r_hz r_phi(8r_pg + 4r_vg + 4r_phig) are not defined in the text; since they multiply the entire reward, their definitions are needed for reproducibility.
- [Section 2.1.3] The sentence "It should be noted that the order of the gait types is not fixed" is confusing immediately after "we assume trotting, bounding and galloping gait as speed increases"; please clarify whether the framework can accommodate a different or learned ordering of gait types.
- [Section 2.2.1] The gating network is described as receiving a "normalized distance" to the goal, but Section 4.1 describes a two-axis normalized relative goal command in the robot heading frame; please state the exact input dimensionality and the normalization used.
- [Section 3] The text "we aim to find gait switch criteria x1m, x2m" appears to contain a typo (the subscript "m"); it should likely read x1, x2. Please correct.
- [Introduction] The claim "to the best of our knowledge, our work is the first multi-skill learning and optimization framework ... (especially galloping)" should be more carefully qualified, since AllGaits (Bellegarda et al., 2024) already reports galloping and transitions, albeit with a different mechanism; the authors should add a sentence that clearly distinguishes the contribution (e.g., outer-loop optimization of switch criteria without a bio-inspired cost).
- [Section 2.2.1] The gating network weights are required to sum to one and be non-negative (Eq. 2), but the output activation (e.g., softmax) and how zero weights are handled are not specified; please add a sentence.
Circularity Check
No significant circularity: the gait switch criteria are transparently optimized reward parameters, not derived predictions, and no load-bearing self-citation is used.
full rationale
The paper's derivation chain is explicit and non-circular. Single-skill policies are pre-trained with SAC using contact-pattern rewards (Sec. 2.1), the multi-skill policy is formed by multiplicative composition of these fixed expert policies (Eqs. 2-4), and a high-level gating network is trained with reward r = 0.6rg + 0.2rf + 0.2re (Eq. 5). The switch criteria x1 and x2 are introduced in Sec. 2.2.4 as thresholds that select which reference foot-contact pattern is rewarded, and then optimized in Sec. 3 by CMA-ES against the accumulated goal-tracking reward rg. The paper openly states that 'the optimized gait switch criteria are not exactly where the gait transitions occur in practice since they are only incorporated in reward functions,' so the criteria are not being passed off as independently predicted transition locations. They are reward-shaping hyperparameters found by a genuine outer-loop optimization, not fitted values renamed as predictions. No self-citation is load-bearing: Yu et al. 2023 is cited for state-feature selection and Yang et al. 2020 for related failure-recovery work, but neither is used to justify the central claim. The real-world galloping limitation is a robustness and evidence-quality concern, not a circularity. No specific equation reduces to its own input by construction, and no uniqueness theorem is imported from the authors' prior work.
Assumptions & free parameters
free parameters (5)
- Gait switch criteria x1, x2 =
x1 = 2.2 m, x2 = 4.3 m after CMA-ES
- Multi-skill reward mixing weights (0.6 rg, 0.2 rf, 0.2 re) =
0.6, 0.2, 0.2
- RBF shape parameters alpha in Table 1 =
e.g., -2.35, -51.16, -18.42, -0.004, -0.032, -460.50, -7.47
- Single-skill reward weights in Table 2 =
Twelve weights per task, e.g., 0.189/0.189/0.114 for recovery
- CMA-ES initial thresholds and hyperparameters =
initial x1=2.0 m, x2=5.0 m, population 50, sigma 1.0 m
assumptions (5)
- domain assumption The animal-inspired foot contact patterns in Fig. 3 are appropriate references for the Unitree A1.
- domain assumption Pre-trained single-skill policies remain valid when composed multiplicatively under a gating network and transferred to the real robot.
- domain assumption The simulation-trained velocity estimator transfers to real-world out-of-distribution galloping motions.
- domain assumption The hand-designed reward structure with RBF terms and weights induces the intended gaits and stable goal tracking.
- standard math Multiplicative composition of Gaussian policies yields a valid Gaussian policy (Peng et al., 2019).
Cite this review
Pith. "Pith review of Discovery of skill switching criteria for learning agile quadruped locomotion." pith.science (2026). https://pith.science/paper/5Y53KNO5
@misc{pith2026250206676,
author = {Pith},
title = {Pith review of: Discovery of skill switching criteria for learning agile quadruped locomotion},
year = {2026},
howpublished = {\url{https://pith.science/paper/5Y53KNO5}},
note = {Machine review of arXiv:2502.06676}
}
read the original abstract
This paper develops a hierarchical learning and optimization framework that can learn and achieve well-coordinated multi-skill locomotion. The learned multi-skill policy can switch between skills automatically and naturally in tracking arbitrarily positioned goals and recover from failures promptly. The proposed framework is composed of a deep reinforcement learning process and an optimization process. First, the contact pattern is incorporated into the reward terms for learning different types of gaits as separate policies without the need for any other references. Then, a higher level policy is learned to generate weights for individual policies to compose multi-skill locomotion in a goal-tracking task setting. Skills are automatically and naturally switched according to the distance to the goal. The proper distances for skill switching are incorporated in reward calculation for learning the high level policy and updated by an outer optimization loop as learning progresses. We first demonstrated successful multi-skill locomotion in comprehensive tasks on a simulated Unitree A1 quadruped robot. We also deployed the learned policy in the real world showcasing trotting, bounding, galloping, and their natural transitions as the goal position changes. Moreover, the learned policy can react to unexpected failures at any time, perform prompt recovery, and resume locomotion successfully. Compared to discrete switch between single skills which failed to transition to galloping in the real world, our proposed approach achieves all the learned agile skills, with smoother and more continuous skill transitions.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...
-
[2]
Bellegarda, G., Shafiee, M., and Ijspeert, A. (2024). Allgaits: Learning all quadruped gaits and transitions. arXiv preprint arXiv:2411.04787
arXiv 2024
-
[3]
C., Yu, W., Zhang, T., Freeman, D., Lee, K.-H., Lee, L., Saliceti, S., Zhuang, V., et al
Caluwaerts, K., Iscen, A., Kew, J. C., Yu, W., Zhang, T., Freeman, D., Lee, K.-H., Lee, L., Saliceti, S., Zhuang, V., et al. (2023). Barkour: Benchmarking animal-level agility with quadruped robots. arXiv preprint arXiv:2305.14654
arXiv 2023
-
[4]
A., Zhou, C., and Tsagarakis, N
Castano, J. A., Zhou, C., and Tsagarakis, N. (2019). Design a fall recovery strategy for a wheel-legged quadruped robot using stability feature space. In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO) , pages 41--46. IEEE
work page 2019
-
[5]
Chen, A. S., Lessing, A. M., Tang, A., Chada, G., Smith, L., Levine, S., and Finn, C. (2024). Commonsense reasoning for legged robot adaptation with vision-language models. arXiv preprint arXiv:2407.02666
arXiv 2024
-
[6]
Cheng, X., Shi, K., Agarwal, A., and Pathak, D. (2024). Extreme parkour with legged robots. In International Conference
work page 2024
-
[7]
Fu, Z., Kumar, A., Malik, J., and Pathak, D. (2021). Minimizing energy consumption leads to the emergence of gaits in legged robots. In Conference on Robot Learning . PMLR
work page 2021
-
[8]
Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning , pages 1861--1870. PMLR
2018
Show all 31 references
-
[9]
Hansen, N. (2016). The cma evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772
2016 arXiv
-
[10]
He, T., Zhang, C., Xiao, W., He, G., Liu, C., and Shi, G. (2024). Agile but safe: Learning collision-free high-speed legged locomotion. In Robotics: Science and Systems
2024
-
[11]
Hoyt, D. F. and Taylor, C. R. (1981). Gait and the energetics of locomotion in horses. Nature , 292(5820):239--240
1981
-
[12]
B., Shao, S., Nikolic, B., and Sreenath, K
Huang, X., Chi, Y., Wang, R., Li, Z., Peng, X. B., Shao, S., Nikolic, B., and Sreenath, K. (2024). Diffuseloco: Real-time legged locomotion control with diffusion from offline datasets. In Conference on Robot Learning
2024
-
[13]
Humphreys, J., Li, J., Wan, Y., Gao, H., and Zhou, C. (2023). Bio-inspired gait transitions for quadruped locomotion. IEEE Robotics and Automation Letters
2023
-
[14]
Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., and Hutter, M. (2019). Learning agile and dynamic motor skills for legged robots. Science Robotics , 4(26):eaau5872
2019
-
[15]
A., Jordan, M
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., and Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural computation , 3(1):79--87
1991
-
[16]
Ji, G., Mun, J., Kim, H., and Hwangbo, J. (2022). Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion. IEEE Robotics and Automation Letters , 7(2):4630--4637
2022
-
[17]
J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al
Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al. (2024). Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246
2024 arXiv
-
[18]
Liang, B., Sun, L., Zhu, X., Zhang, B., Xiong, Z., Li, C., Sreenath, K., and Tomizuka, M. (2024). Adaptive energy regularization for autonomous gait transition and energy-efficient quadruped locomotion. arXiv preprint arXiv:2403.20001
2024 arXiv
-
[19]
Margolis, G. B. and Agrawal, P. (2023). Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning , pages 22--31. PMLR
2023
-
[20]
L., Merkt, W., Papatheodorou, A., Havoutis, I., and Posner, I
Mitchell, A. L., Merkt, W., Papatheodorou, A., Havoutis, I., and Posner, I. (2024). Gaitor: Learning a unified representation across gaits for real-world quadruped locomotion. In Conference on Robot Learning . PMLR
2024
-
[21]
L., Yu, W., Posner, I., and Havoutis, I
O'Mahoney, R., Mitchell, A. L., Yu, W., Posner, I., and Havoutis, I. (2024). Offline adaptation of quadruped locomotion using diffusion models. arXiv preprint arXiv:2411.08832
2024 arXiv
-
[22]
and Ishiguro, A
Owaki, D. and Ishiguro, A. (2017). A quadruped robot exhibiting spontaneous gait transitions from walking to trotting to galloping. Scientific reports , 7(1):1--10
2017
-
[23]
B., Chang, M., Zhang, G., Abbeel, P., and Levine, S
Peng, X. B., Chang, M., Zhang, G., Abbeel, P., and Levine, S. (2019). Mcp: Learning composable hierarchical control with multiplicative compositional policies. Advances in Neural Information Processing Systems , 32
2019
-
[24]
Reske, A., Carius, J., Ma, Y., Farshidian, F., and Hutter, M. (2021). Imitation learning from mpc for quadrupedal multi-gait control. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 5014--5020. IEEE
2021
-
[25]
Shafiee, M., Bellegarda, G., and Ijspeert, A. (2024). Viability leads to the emergence of gait transitions in learning agile quadrupedal locomotion on challenging terrains. Nature Communications , 15(1):3073
2024
-
[26]
Shao, Y., Jin, Y., Liu, X., He, W., Wang, H., and Yang, W. (2021). Learning free gait transition for quadruped robots via phase-guided controller. IEEE Robotics and Automation Letters
2021
-
[27]
Tang, Y., Yu, W., Tan, J., Zen, H., Faust, A., and Harada, T. (2023). Saytap: Language to quadrupedal locomotion. arXiv preprint arXiv:2306.07580
2023 arXiv
-
[28]
Yang, C., Yuan, K., Zhu, Q., Yu, W., and Li, Z. (2020). Multi-expert learning of adaptive legged locomotion. Science Robotics , 5(49):eabb2174
2020
-
[29]
Yang, Y., Zhang, T., Coumans, E., Tan, J., and Boots, B. (2022). Fast and efficient locomotion via learned gait transitions. In Conference on Robot Learning , pages 773--783. PMLR
2022
-
[30]
J., and Li, Z
Yu, W., Yang, C., McGreavy, C., Triantafyllidis, E., Bellegarda, G., Shafiee, M., Ijspeert, A. J., and Li, Z. (2023). Identifying important sensory feedback for learning locomotion skills. Nature Machine Intelligence , 5(8):919--932
2023
-
[31]
Zhuang, Z., Fu, Z., Wang, J., Atkeson, C., Schwertfeger, S., Finn, C., and Zhao, H. (2023). Robot parkour learning. In Conference on Robot Learning , pages 73--92. PMLR
2023
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.