REVIEW 3 major objections 7 minor 28 references
Using coordinated foreleg manipulation and hind-leg support, a simulation-trained hierarchical controller enables a hexapod to push objects of unknown size, weight, and friction to specified poses on real hardware.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 14:38 UTC pith:C4RLKUOR
load-bearing objection A credible hexapod pushing system with strong simulation evidence but anecdotal real-world validation; the central claim outruns the hardware data. the 3 major comments →
HeLoM: Hierarchical Learning for Whole-Body Loco-Manipulation by a Hexapod Robot
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that whole-body, non-prehensile pushing by a hexapod can be decomposed into two learned policies that work together: a Planner that decides where to place the forelegs and how to move the body, and a Controller that tracks those decisions while keeping the robot upright. The Planner is rewarded for bringing the object to the goal, but only after both forelegs are in contact, and it also receives penalties for object tilt, foot slip, and erratic actions. The Controller is trained first with smoothness regularization and domain randomization, then frozen while the Planner is trained on top. The paper reports a 95.6% success rate over 1000 simulated trials and successful re
What carries the argument
The load-bearing mechanism is the hierarchical decomposition into Planner and Controller networks, both trained with reinforcement learning. The Planner runs at 50 Hz and outputs desired body velocities and target foreleg foot positions; the Controller maps proprioceptive observations and latent estimates to 18 joint position targets. Three design choices carry the argument: a reward that gives a bonus for simultaneous contact of both forelegs, encouraging coordinated bi-foot pushing; a term that penalizes object tilt around the x/y axes to keep the pushing line near the center of mass and prevent tipping; and a curriculum over target lateral displacement and orientation that starts easy and
Load-bearing premise
The central claim rests on the assumption that randomized physics parameters and PD gain scaling in simulation adequately model the real hexapod's Series Elastic Actuator nonlinearities, so a policy trained entirely in simulation with no fine-tuning will behave on the real robot as it does in simulation.
What would settle it
Deploy the exact HeLoM policy on a real box with mass 10 kg and surface friction around 0.5 (well inside the randomized ranges) and measure success rate over 20 independent trials; if the real-world success is below 50% while simulation reports 95.6% for the same distribution, the zero-shot transfer claim collapses.
If this is right
- Hexapod robots can perform useful manipulation of large, heavy, irregular objects without a dedicated arm, using only coordinated leg forces.
- Zero-shot sim-to-real transfer is feasible for contact-rich pushing on platforms with compliant actuators, provided the domain randomization is broad enough.
- The hierarchical planner/controller split may generalize to other multi-legged platforms and to non-prehensile tasks beyond pushing, such as rolling or dragging.
- A command curriculum is necessary for learning long-range pushing with large orientation changes; without it, simulated success drops from 95.6% to 78.7%.
- The robot can adapt online to external disturbances and object geometry it has never seen, suggesting the learned policy is not merely memorizing a single pushing strategy.
Where Pith is reading between the lines
- If the framework's success stems from the hexapod's redundant legs, the same hierarchy may transfer to other legged robots, but with fewer legs the stability margin would be thinner; a direct comparison would test that hypothesis.
- The paper's reported failures on lightweight, low-friction objects suggest the policy lacks fine force control at the low end of the distribution; adding force feedback or a different terminal reward is a testable fix.
- The reliance on an external motion capture system means the perception problem is deferred; a version with onboard cameras or LiDAR would need to tolerate pose-estimation noise, which is an open extension not addressed here.
- A conservative reading of 'unknown physical properties' is that they vary within the randomized ranges; the paper does not claim generalization to, say, 30 kg boxes or very slippery surfaces.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HeLoM, a hierarchical reinforcement-learning framework for whole-body loco-manipulation by a hexapod robot. A high-level Planner outputs commanded base velocities and foreleg foot positions; a low-level Controller, based on DreamWaQ, tracks these commands while maintaining locomotion stability. Both levels are trained in Isaac Gym with domain randomization and a command-based curriculum, then deployed zero-shot on the HEBI Daisy hexapod. Simulation evaluation over 1000 randomized trials reports a 95.6% success rate with an average completion time of 12.1 s, alongside ablations showing the importance of the curriculum and key reward terms. Real-world experiments qualitatively demonstrate pushing of four boxes with varying sizes, masses, materials, and center-of-mass locations, as well as recovery from external disturbances. The central claim is that the framework can stably push objects of varying sizes and unknown physical properties to designated goal poses in the real world.
Significance. If the central claim is fully substantiated, HeLoM would be a meaningful advance in legged loco-manipulation: it is among the first hexapod-specific frameworks to use coordinated foreleg interaction and hind-leg propulsion for non-prehensile pushing, requiring no dedicated arm and no real-world fine-tuning. The paper has clear strengths: the hierarchical decomposition with a frozen low-level controller is principled; the command-based curriculum is a sensible response to sparse exploration; the domain randomization over physical parameters, PD gains, and object properties is well motivated; and the simulation study is strong in scope, with 1000 randomized trials and ablations isolating the curriculum and key rewards. The main weakness is that the paper's headline claim is about real-world capability, but the real-world evidence is purely qualitative. The simulation metrics cannot by themselves establish the sim-to-real transfer claims, especially given the paper's own statement that Daisy's series elastic actuators enlarge the sim-to-real gap.
major comments (3)
- [Section IV.B and Figs. 5-7] The central abstract claim — that HeLoM can 'stably push objects of varying sizes and unknown physical properties to designated goal poses in the real world' — is not quantitatively supported by the hardware section. Figs. 5-7 show individual qualitative sequences, but there is no trial count, no success rate, no final pose error, no completion time, and no comparison condition on hardware. Table VI reports a 95.6% success rate over 1000 simulation trials, but this cannot be extrapolated to the real robot. The authors should define the same success criteria used in simulation (position error below 0.05 m, orientation error below 5 degrees) and report success rate, error statistics, and trial counts for the real-world pushing task, including the disturbance-recovery scenario.
- [Section III.C.1 and Table IV] The paper states that Daisy's Series Elastic Actuators have nonlinearities and parameter uncertainties that 'further enlarge the sim-to-real gap,' and the proposed mitigation is domain randomization over PD gains and physical parameters. However, no quantitative evidence is provided that this mitigation suffices on hardware. Fig. 4 shows a qualitative velocity/foot-tracking comparison with and without the smoothness loss, but it does not measure task-level pushing performance. The zero-shot deployment claim depends on the effectiveness of this transfer mechanism, so the hardware evaluation should include quantitative tracking errors and task success rates, ideally with a comparison against a non-adapted baseline, to demonstrate that the domain randomization actually closes the gap.
- [Section IV.B.1-2 and Section V] The real-world experiments rely on an external optical motion capture system for object and robot poses, and the conclusion explicitly concedes that this 'limits its use to controlled environments.' The abstract, however, presents the real-world result without this caveat. This is an evidence gap rather than an internal inconsistency, but it should be addressed: either the abstract should qualify the claim to 'controlled indoor environments with external pose feedback,' or the paper should provide additional real-world evidence showing the controller works when such external infrastructure is unavailable.
minor comments (7)
- [Title] The title reads 'in Hexapod Robot'; grammatically it should be 'for a Hexapod Robot' or 'with a Hexapod Robot.'
- [Section III.C.2] The command-based curriculum is described only qualitatively. For reproducibility, specify the number of curriculum levels, the exact thresholds for y-displacement and orientation, and the expansion schedule.
- [Section III.C.1, Eq. (2)] The smoothness-loss equation uses hyperparameters λ_π and λ_V but their values are not reported. Please provide the values used in training.
- [Table II] The notation `rdis,obj tar` appears with inconsistent subscripts, and the penalty term `P([a_min - a_t]^+ + [a_t - a_max]^+)` is not formally defined. Clarify whether this is an indicator or a hinge penalty and define the bracket notation.
- [Fig. 4] The velocity and foot-position tracking plots lack explicit axis labels, units, and quantitative error metrics. Adding these would make the comparison with and without smoothness loss more informative.
- [Section IV.B.2] The sentence 'e) is the object specifications and experimental test parameters' is a sentence fragment; rephrase. Also, consider stating explicitly whether the four boxes in Fig. 6 are the same as those used for the quantitative evaluation in Table VI, if applicable.
- [Section II.A] There is a typographical error: 'B’ezier' should be 'Bézier.'
Circularity Check
No significant circularity: the paper's claims rest on trained-policy evaluation and external baselines, not on self-referential derivation.
full rationale
HeLoM is an empirical RL system: the high-level Planner and low-level Controller are trained in Isaac Gym with PPO, and the paper's central claims are supported by held-out simulation success rates (95.6% over 1000 randomized trials, Table VI) and by qualitative real-world pushing demonstrations (Figs. 5–7). There is no analytic derivation chain in which an output is defined in terms of an input, and no fitted parameter is later renamed as a prediction. The simulation success criterion (position error < 0.05 m, orientation error < 5°) is the same task definition used in training, but that is standard, disclosed benchmarking rather than circularity: the RL policy is not constructed to satisfy that criterion by hand; it is optimized against a reward function and then evaluated under the same task metric, which is an empirical measurement. Domain randomization over friction, mass, COM, inertia, and PD gains (Tables IV–V) is a known sim-to-real mitigation, not a hidden way of encoding the real-world result. Self-citations appear in related work ([1], [4], [19]) and are not load-bearing for the central claim; the controller architecture is taken from the external DreamWaQ work [25], and the smoothness loss from external L2C2 [27]. The paper's own conclusion explicitly concedes that the current implementation requires an external motion capture system, which limits deployment to controlled environments; this is a limitation disclosure, not a circular step. The skeptic's concern—that real-world support is anecdotal without quantitative success rates or pose errors—is an evidence-strength issue, not a circularity issue. I find no step where a prediction reduces by construction to its inputs.
Axiom & Free-Parameter Ledger
free parameters (5)
- Planner reward weights =
Table II; e.g., r_dis_obj_tar weight 3, terminal +10, dual-contact bonus +0.3
- Controller reward weights =
Table III
- Command bounds for Planner during training =
v_cmd x [0,0.5] m/s, y [-0.3,0.3] m/s, yaw [-1,1] rad/s; forefoot offsets x [-0.25,0.25], y [-0.1,0.2], z [-0.2,0.2] m
- Success and reset thresholds =
success at 0.05 m / 5 deg / 25 s; tilt reset at 40 deg
- Domain randomization ranges =
Tables IV and V
axioms (5)
- domain assumption Simulation fidelity: Isaac Gym contact dynamics and PD control, with randomized physical parameters, are close enough to Daisy's real SEA-driven dynamics for zero-shot sim-to-real transfer.
- domain assumption Ground-truth object and robot poses are available at 50 Hz from an external motion capture system during deployment.
- domain assumption The low-level Controller can faithfully execute Planner commands, so freezing it and training Planner on top is a valid decomposition.
- domain assumption PPO with the specified rewards converges to a policy whose success reflects task competence rather than reward hacking.
- domain assumption Privileged object properties (mass, COM, inertia, friction) can be given only to the critic, while the actor must generalize over the randomized distribution.
read the original abstract
In nature, animals often need to move/manipulate objects comparable in weight/size to their own bodies. Compared to grasping and carrying, pushing provides a more straightforward and efficient non-prehensile manipulation strategy, avoiding complex grasp design while leveraging direct contact to regulate an object's pose during interaction. Achieving effective pushing, however, requires both sufficient manipulation capability and stable whole-body coordination, which is particularly challenging when dealing with heavy or irregular objects. To address these challenges, we propose HeLoM, a learning-based hierarchical whole-body manipulation framework for hexapod robots that exploits coordinated multi-limb control and is applicable to multi-legged robotic systems. Inspired by the cooperative strategies of multi-legged insects, our framework leverages multiple contact points and high degrees of freedom to enable efficient and dynamic whole-body coordination during object interaction. HeLoM's high-level planner plans pushing behaviors, while its low-level controller maintains locomotion stability and generates dynamically consistent joint actions. This design enables the robot to maintain balance while executing continuous and controllable pushing behaviors through coordinated foreleg interaction and supportive hind-leg propulsion. We validate the effectiveness of HeLoM through both simulation and real-world experiments. Results show that our framework can stably push objects of varying sizes and unknown physical properties to designated goal poses in the real world.
Figures
Reference graph
Works this paper leans on
-
[1]
Legged robots for object manipulation: A review,
Y . Gong, G. Sun, A. Nair, A. Bidwai, R. CS, J. Grezmak, G. Sartoretti, and K. A. Daltorio, “Legged robots for object manipulation: A review,” Frontiers in Mechanical Engineering, vol. 9, p. 1142421, 2023
2023
-
[2]
Learning goal-oriented non-prehensile pushing in cluttered scenes,
N. Dengler, D. Großklaus, and M. Bennewitz, “Learning goal-oriented non-prehensile pushing in cluttered scenes,” in2022 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 1116–1122
2022
-
[3]
Learning whole-body manipulation for quadrupedal robot,
S. Jeon, M. Jung, S. Choi, B. Kim, and J. Hwangbo, “Learning whole-body manipulation for quadrupedal robot,”IEEE Robotics and Automation Letters, vol. 9, no. 1, pp. 699–706, 2023
2023
-
[4]
Sata: Safe and adaptive torque- based locomotion policies inspired by animal learning,
P. Li, H. Li, G. Sun, J. Cheng, X. Yang, G. Bellegarda, M. Shafiee, Y . Cao, A. Ijspeert, and G. Sartoretti, “Sata: Safe and adaptive torque- based locomotion policies inspired by animal learning,”arXiv preprint arXiv:2502.12674, 2025
Pith/arXiv arXiv 2025
-
[5]
Whole-body end- effector pose tracking,
T. Portela, A. Cramariuc, M. Mittal, and M. Hutter, “Whole-body end- effector pose tracking,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 11 205–11 211
2025
-
[6]
I. Dadiotis, M. Mittal, N. Tsagarakis, and M. Hutter, “Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning,”arXiv preprint arXiv:2502.01546, 2025
arXiv 2025
-
[7]
Rambo: Rl- augmented model-based whole-body control for loco-manipulation,
J. Cheng, D. Kang, G. Fadini, G. Shi, and S. Coros, “Rambo: Rl- augmented model-based whole-body control for loco-manipulation,” IEEE Robotics and Automation Letters, 2025
2025
-
[8]
Nature’s all-in-one: Multi- tasking robots inspired by dung beetles,
B. Leung, S. Gorb, and P. Manoonpong, “Nature’s all-in-one: Multi- tasking robots inspired by dung beetles,”Advanced Science, vol. 11, no. 47, p. 2408080, 2024
2024
-
[9]
Contact optimization for non-prehensile loco-manipulation via hierarchical model predictive control,
A. Rigo, Y . Chen, S. K. Gupta, and Q. Nguyen, “Contact optimization for non-prehensile loco-manipulation via hierarchical model predictive control,”arXiv preprint arXiv:2210.03442, 2022
Pith/arXiv arXiv 2022
-
[10]
Hierarchical adaptive loco- manipulation control for quadruped robots,
M. Sombolestan and Q. Nguyen, “Hierarchical adaptive loco- manipulation control for quadruped robots,”arXiv preprint arXiv:2209.13145, 2022
Pith/arXiv arXiv 2022
-
[11]
Loco-manipulation control for arm- mounted quadruped robots: Dynamic and kinematic strategies,
G. Xin, F. Zeng, and K. Qin, “Loco-manipulation control for arm- mounted quadruped robots: Dynamic and kinematic strategies,”Ma- chines, vol. 10, no. 8, p. 719, 2022
2022
-
[12]
Roloma: Robust loco-manipulation for quadruped robots with arms,
H. Ferrolho, V . Ivan, W. Merkt, I. Havoutis, and S. Vijayakumar, “Roloma: Robust loco-manipulation for quadruped robots with arms,” Autonomous Robots, vol. 47, no. 8, pp. 1463–1481, 2023
2023
-
[13]
Multi-contact heavy object pushing with a centaur-type humanoid robot: Planning and control for a real demonstrator,
M. P. Polverini, A. Laurenzi, E. M. Hoffman, F. Ruscelli, and N. G. Tsagarakis, “Multi-contact heavy object pushing with a centaur-type humanoid robot: Planning and control for a real demonstrator,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 859–866, 2020
2020
-
[14]
Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,
J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim, “Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,” in2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2018, pp. 1–9
2018
-
[15]
Dribblebot: Dynamic legged manipulation in the wild,
Y . Ji, G. B. Margolis, and P. Agrawal, “Dribblebot: Dynamic legged manipulation in the wild,”arXiv preprint arXiv:2304.01159, 2023
Pith/arXiv arXiv 2023
-
[16]
Deep whole-body control: learning a unified policy for manipulation and locomotion,
Z. Fu, X. Cheng, and D. Pathak, “Deep whole-body control: learning a unified policy for manipulation and locomotion,” inConference on Robot Learning. PMLR, 2023, pp. 138–149
2023
-
[17]
Learning force con- trol for legged manipulation,
T. Portela, G. B. Margolis, Y . Ji, and P. Agrawal, “Learning force con- trol for legged manipulation,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 15 366–15 372
2024
-
[18]
Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,
H. Ha, Y . Gao, Z. Fu, J. Tan, and S. Song, “Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,”arXiv preprint arXiv:2407.10353, 2024
Pith/arXiv arXiv 2024
-
[19]
Joint-space cpg for safe foothold plan- ning and body pose control during locomotion and climbing,
G. Sun and G. Sartoretti, “Joint-space cpg for safe foothold plan- ning and body pose control during locomotion and climbing,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9889–9896, 2022
2022
-
[20]
Autonomous obstacle legipulation with a hexapod robot,
B. Lu, B. Tam, and N. Kottege, “Autonomous obstacle legipulation with a hexapod robot,”arXiv preprint arXiv:2011.06227, 2020
Pith/arXiv arXiv 2011
-
[21]
Design and multimodal locomotion plan of a hexapod robot with improved knee joints,
K. Xu, R. Qin, C. Chen, G. Dong, J. Chen, and X. Ding, “Design and multimodal locomotion plan of a hexapod robot with improved knee joints,”Journal of Field Robotics, vol. 41, no. 5, pp. 1279–1296, 2024
2024
-
[22]
De- sign and control of a novel leg-arm multiplexing mobile operational hexapod robot,
T. Chen, Y . Li, X. Rong, G. Zhang, H. Chai, J. Bi, and Q. Wang, “De- sign and control of a novel leg-arm multiplexing mobile operational hexapod robot,”IEEE Robotics and Automation Letters, vol. 7, no. 1, pp. 382–389, 2021
2021
-
[23]
Pedipulate: Enabling manipulation skills using a quadruped robot’s leg,
P. Arm, M. Mittal, H. Kolvenbach, and M. Hutter, “Pedipulate: Enabling manipulation skills using a quadruped robot’s leg,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 5717–5723
2024
-
[24]
Learning multi-agent loco-manipulation for long-horizon quadrupedal pushing,
Y . Feng, C. Hong, Y . Niu, S. Liu, Y . Yang, and D. Zhao, “Learning multi-agent loco-manipulation for long-horizon quadrupedal pushing,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 14 441–14 448
2025
-
[25]
I. Nahrendra, B. Yu, and H. Myung, “Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning,”arXiv preprint arXiv:2301.10602, 2023
Pith/arXiv arXiv 2023
-
[26]
Proximal policy optimization algorithms,
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017
Pith/arXiv arXiv 2017
-
[27]
L2c2: Locally lipschitz continuous constraint towards stable and smooth reinforcement learning,
T. Kobayashi, “L2c2: Locally lipschitz continuous constraint towards stable and smooth reinforcement learning,” in2022 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 4032–4039
2022
-
[28]
Learning to walk in minutes using massively parallel deep reinforcement learning,
N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in Conference on robot learning. PMLR, 2022, pp. 91–100
2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.