REVIEW 4 major objections 5 minor 22 references
Replacing the policy network with a Mamba encoder makes humanoid walking RL faster, stabler, and cheaper to actuate.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 15:46 UTC pith:JDW2Z2LS
load-bearing objection Mamba encoder shows real learning-efficiency gains for humanoid walking, but the energy-saving claims are confounded by slower gait and need a speed-matched re-evaluation. the 4 major comments →
HuMam: Humanoid Motion Control via End-to-End Deep Reinforcement Learning with Mamba
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a state-centric Mamba encoder—processing a token pair of robot-centric and external footstep states through gated state-space dynamics—yields more efficient, stable, and physically economical humanoid walking policies than a feedforward baseline trained under identical settings. The discovery is an application result: the Mamba structure, not new rewards or simulators, is credited for faster convergence, lower cross-seed variance, and reduced actuation cost across forward, backward, lateral, curved, and standing tasks.
What carries the argument
The single-layer Mamba encoder with selective gating (state-space update equations) that mixes two tokens: robot-centric state (leg joint positions/velocities, base orientation/angular velocity) and external state (two oriented footstep targets plus a continuous phase clock). The encoder's output feeds shared policy and value heads optimized with PPO; actions are joint position targets executed by a low-gain PD loop at 1000 Hz. A six-term reward shapes contact, swing, step placement, posture, height, and upper-body stability.
Load-bearing premise
The policy assumes the two upcoming footstep targets it receives are correct and always available; the Mamba backbone's demonstrated gains apply only under these oracle plans, with online footstep planning and feasibility checks deferred to future work.
What would settle it
Train the same Mamba and feedforward policies with footstep targets randomly jittered or drawn from a distribution not seen in training, and measure the return gap; if the feedforward baseline matches or beats Mamba under target noise, the claimed advantage is not about feature fusion but about overfitting to clean plans. A more direct test: replace the Mamba layer with a linear layer of equal parameter count and check whether the sample-efficiency and torque-reduction gains persist.
If this is right
- If the Mamba advantage holds, a lightweight single-layer state-space encoder could replace heavier recurrent or attention-based backbones in onboard humanoid controllers with limited compute and memory.
- The reported 9.6% mean and 9.1% peak torque reductions, plus lower energy per meter, could translate to longer battery life and reduced mechanical stress on physical robots.
- The state-centric interface (robot states plus two planned footsteps) is compatible with adding perception later without changing the policy's input structure.
- The sample-efficiency gains (up to 42.5% fewer samples to reach a target return) could reduce training cost for new gaits or terrains.
- The consistent improvement across distinct walking modes suggests the Mamba backbone generalizes across gait patterns, not just one trajectory type.
Where Pith is reading between the lines
- The observed benefits may stem from Mamba's selective gating acting as an input-dependent feature mixer, effectively learning to weight robot versus footstep information per state; a controlled ablation replacing Mamba with a single gated linear layer of equal parameter count would test this mechanism directly.
- If the advantage persists under noisy or partially occluded footstep targets, the encoder likely learns robust representations rather than memorizing exact plan coordinates; this could be checked by perturbing target positions at evaluation time.
- The same backbone may benefit other contact-rich legged robots or whole-body manipulation tasks with multimodal state inputs, though this is untested and speculative.
- Because the paper only compares against one feedforward baseline, the relative benefit over other sequence models (GRU, Transformer) under the same compute envelope remains open; the authors themselves list this comparison as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents HuMam, an end-to-end RL framework for humanoid locomotion that uses a single-layer Mamba encoder to fuse robot-centric proprioceptive states with two planned footstep targets and a continuous phase clock. The policy outputs 12 joint-position targets tracked by a PD controller and is trained with PPO under a six-term reward. Experiments are conducted in mc-mujoco on the JVRC-1 humanoid across forward, backward, lateral, curved walking, and standing, comparing HuMam against a feedforward baseline with identical training settings. The central claims are that the Mamba backbone improves learning efficiency, training stability, and final returns, while also reducing power consumption and torque peaks.
Significance. If the empirical claims hold, the paper makes a modest but useful contribution: it is the first demonstration of a Mamba-style state-space encoder as the fusion backbone for end-to-end humanoid locomotion RL, and it provides a systematic comparison against a strong feedforward baseline. The paper is clearly written, the method is straightforward, and the training/evaluation setup is shared across both arms of the comparison, which limits circularity. However, the absence of speed control and speed reporting in the energy/torque evaluations, combined with missing seed counts and statistical tests, currently weakens the quantitative support for the headline contributions. The work does not provide code, checkpoints, or hardware validation, so the claims rest entirely on simulation-based comparisons.
major comments (4)
- [§IV-C, Table VI, Eq. (5)] Speed confound in energy/torque claims: The reward in Eq. (5) contains no velocity or progress term, and step achievement (§IV-B.c) only requires a foot to remain within 20 cm of the target for one single-support duration. Recovering walking speed from Table VI as power ÷ (J/m) gives, for curved walking, HuMam at 95.17/1303 ≈ 0.073 m/s versus Baseline at 131.93/1055 ≈ 0.125 m/s—a 42% speed deficit. Forward walking is also slower (≈0.256 vs 0.272 m/s). Since joint torques and actuation power scale with gait speed, the reported reductions in power, energy per meter, and torque peaks may simply reflect a slower gait rather than the Mamba encoder. The authors must report walking speed for every task and provide a speed-matched controlled evaluation (e.g., matched commanded speed or evaluation at the same achieved speed) before claiming 'reducing power consumption and torque peaks'.
- [§IV-D, Tables III–IV, Fig. 3] Lack of statistical support: The paper reports learning curves 'across seeds' and standard deviations, but never states the number of seeds. No statistical significance tests or confidence intervals are provided for the small differences in Table IV (e.g., average total reward 0.737 vs 0.728). Moreover, for curved and forward walking the Baseline actually obtains higher total reward (0.677 vs 0.672 and 0.694 vs 0.668, respectively), which contradicts the unqualified 'consistently improves' claim. Add the seed count, per-seed results, and significance tests or effect sizes; without these, the central comparison is not rigorously established.
- [§IV-C, Table V] Torque comparison is not speed-matched: Table V reports average and peak torques only for forward walking, and no walking speed is given. Given the speed differences inferred from Table VI, the reported 9.6% average and 9.1% peak torque reductions could be consequences of slower gait. The authors should either report torque at matched gait speeds or include speed as a covariate; otherwise the 'smoother actuation' and 'energy-saving' conclusions are not supported.
- [Algorithm 1, §IV-B.d] Domain randomization is unspecified: Algorithm 1 samples domain-randomization parameters ξ, and the text mentions dynamics, sensors, init state, and target jitter, but the actual distributions and ranges are never provided. This is a reproducibility gap and also weakens the robustness interpretation. Provide the full DR specification (e.g., noise magnitudes, friction ranges, mass distributions) or state explicitly that DR was not used.
minor comments (5)
- [Table VI] Energy metrics are reported without error bars or confidence intervals, even though the paper acknowledges variability elsewhere. Add error bars for J/m, W, and W/kg, or explain why they are omitted.
- [§IV-B.c] The phrase 'one single-support duration' is used to define when a step is scored but is never defined. Clarify how this duration is determined and how target advancement is triggered.
- [§III-C, Eqs. (15)–(16)] The state-space encoder equations use σ(·) for 'elementwise gates', but standard Mamba uses specific nonlinearities (e.g., SiLU, softmax for selection). Clarify the exact gating functions and tensor dimensions of W_A, W_B, W_C, and W_D, since the equations as written are not self-contained.
- [Table V] In the text, the right hip pitch (R HIP P) is described as showing a 'substantial reduction in both average and peak torque', but the table shows average −16.4% and peak +41.1%. Verify the reported numbers or rephrase; the current wording is misleading for that joint.
- [Fig. 3] The caption says 'across scenarios', but it is not clear whether the learning curves are aggregated over all tasks or shown per task. Label the panels or explain the aggregation.
Circularity Check
No circularity: the paper is a controlled empirical comparison of two architectures trained under identical rewards and hyperparameters, with all performance, torque, and energy numbers measured in simulation.
full rationale
I walked the derivation chain and found no step in which a claimed output is equivalent by construction to an input, nor any load-bearing self-citation. The central claim is an empirical comparison: HuMam (single-layer Mamba encoder) versus a feedforward baseline trained with the same observation space, the same six-term reward (Eq. 5), and the same PPO hyperparameters (Table II). Because the reward weights are shared by both arms, they do not force the comparison outcome. The Mamba encoder equations (15)-(17) are generic gated state-space updates; no performance, stability, torque, or energy result is derived from those equations, so there is no self-definitional reduction. Energy and torque metrics (Section IV-C, Tables V and VI) are measured in simulation and compared across policies; no fitted parameter is renamed as a prediction, and no external benchmark value is imported from the authors' own prior work. The reference list contains no paper by the present authors, so there is no self-citation chain. The only substantive caveat is an experimental confound, not circularity: the reward (Eq. 5) contains no velocity or progress term, and step scoring in Section IV-B.c is based on holding a foot within a 20 cm radius for one single-support duration, so the reported energy savings could partly reflect slower gait rather than the Mamba backbone. That is a validity threat to the energy-efficiency claim, but it is not an equation-level reduction of a prediction to its inputs. The paper also explicitly lists online footstep planning with feasibility checks as future work, which acknowledges the oracle-footstep limitation rather than hiding it. Therefore the circularity score is 0.
Axiom & Free-Parameter Ledger
free parameters (6)
- Reward coefficients (alpha_force, alpha_vel, alpha_step, alpha_orient, alpha_height, alpha_upper) =
0.15, 0.15, 0.45, 0.05, 0.05, 0.05
- Phase clock period L (Eq. 4) =
not reported
- PD gains of the low-level controller =
not reported
- Footstep geometry (step length, foot spread) =
0.25 m, 0.12 m
- Termination height and step-scoring radius =
0.60 m, 0.20 m
- Domain randomization distributions xi =
unspecified
axioms (5)
- domain assumption Current observation is a sufficient statistic for the locomotion MDP (Markovian state)
- domain assumption Footstep plans are provided externally and are accurate
- domain assumption mc-mujoco's JVRC-1 dynamics faithfully model a real humanoid
- domain assumption The feedforward baseline [20] is a strong, fairly re-implemented comparison
- standard math PPO converges to a good policy for this MDP
Cite this review
Pith. "Pith review of HuMam: Humanoid Motion Control via End-to-End Deep Reinforcement Learning with Mamba." pith.science (2026). https://pith.science/paper/JDW2Z2LS
@misc{pith2026250918046,
author = {Pith},
title = {Pith review of: HuMam: Humanoid Motion Control via End-to-End Deep Reinforcement Learning with Mamba},
year = {2026},
howpublished = {\url{https://pith.science/paper/JDW2Z2LS}},
note = {Machine review of arXiv:2509.18046}
}
read the original abstract
End-to-end reinforcement learning (RL) for humanoid locomotion is appealing for its compact perception-action mapping, yet practical policies often suffer from training instability, inefficient feature fusion, and high actuation cost. We present HuMam, a state-centric end-to-end RL framework that employs a single-layer Mamba encoder to fuse robot-centric states with oriented footstep targets and a continuous phase clock. The policy outputs joint position targets tracked by a low-level PD loop and is optimized with PPO. A concise six-term reward balances contact quality, swing smoothness, foot placement, posture, and body stability while implicitly promoting energy saving. On the JVRC-1 humanoid in mc-mujoco, HuMam consistently improves learning efficiency, training stability, and overall task performance over a strong feedforward baseline, while reducing power consumption and torque peaks. To our knowledge, this is the first end-to-end humanoid RL controller that adopts Mamba as the fusion backbone, demonstrating tangible gains in efficiency, stability, and control economy.
Figures
Reference graph
Works this paper leans on
-
[1]
Advancements in humanoid robots: A comprehensive review and future prospects,
Y . Tong, H. Liu, and Z. Zhang, “Advancements in humanoid robots: A comprehensive review and future prospects,”IEEE/CAA Journal of Automatica Sinica, vol. 11, no. 2, pp. 301–328, 2024
2024
-
[2]
Reinforcement learning in robotic applications: a comprehensive survey,
B. Singh, R. Kumar, and V . P. Singh, “Reinforcement learning in robotic applications: a comprehensive survey,” Artificial Intelligence Review, vol. 55, no. 2, pp. 945–990, 2022
2022
-
[3]
A comprehensive survey on humanoid robot development,
S. Saeedvand, M. Jafari, H. S. Aghdasi, and J. Baltes, “A comprehensive survey on humanoid robot development,” The Knowledge Engineering Review, vol. 34, p. e20, 2019
2019
-
[4]
Teleoperation of humanoid robots: A survey,
K. Darvish, L. Penco, J. Ramos, R. Cisneros, J. Pratt, E. Yoshida, S. Ivaldi, and D. Pucci, “Teleoperation of humanoid robots: A survey,”IEEE Transactions on Robotics, vol. 39, no. 3, pp. 1706–1727, 2023
2023
-
[5]
Learning agile and dynamic motor skills for legged robots,
J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,” Science Robotics, vol. 4, no. 26, p. eaau5872,
-
[6]
Learning quadrupedal locomotion over challenging terrain,
J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Science Robotics, vol. 5, no. 47, p. eabc5986, 2020. [Online]. Available: https: //www.science.org/doi/10.1126/scirobotics.abc5986
-
[7]
Learning to walk in minutes using massively parallel deep reinforcement learning,
N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” inRobotics: Science and Systems (RSS). PMLR, 2022, pMLR V ol. 164. [Online]. Available: https://arxiv.org/abs/2109.11978
Pith/arXiv arXiv 2022
-
[8]
Rma: Rapid motor adaptation for legged robots,
A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,”arXiv preprint arXiv:2107.04034, 2021. IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, SEPTEMBER 2025 10 Fig. 8: Joint Torques of Forward Walking Task
Pith/arXiv arXiv 2021
-
[9]
Sim-to- real learning of all common bipedal gaits via periodic reward composition,
J. Siekmann, Y . Godse, A. Fern, and J. Hurst, “Sim-to- real learning of all common bipedal gaits via periodic reward composition,” in2021 IEEE International Con- ference on Robotics and Automation (ICRA). IEEE, 2021, pp. 7309–7315
2021
-
[10]
Reinforcement learning for robust parameterized locomotion control of bipedal robots,
Z. Li, X. Cheng, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Reinforcement learning for robust parameterized locomotion control of bipedal robots,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 2811–2817
2021
-
[11]
Blind bipedal stair traversal via sim-to-real reinforce- ment learning,
J. Siekmann, K. Green, J. Warila, A. Fern, and J. Hurst, “Blind bipedal stair traversal via sim-to-real reinforce- ment learning,”arXiv preprint arXiv:2105.08328, 2021
Pith/arXiv arXiv 2021
-
[12]
Learning whole-body motor skills for humanoids,
C. Yang, W. Zhang, C. G. Atkeson, and D. W. Hong, “Learning whole-body motor skills for humanoids,” arXiv preprint arXiv:2002.02991, 2020. [Online]. Available: https://arxiv.org/abs/2002.02991
Pith/arXiv arXiv 2002
-
[13]
Deepwalk: Omnidirectional bipedal gait by deep reinforcement learning,
D. Rodriguez, W. Merkt, Y . Demiris, E. Valassakis, and M. Fallon, “Deepwalk: Omnidirectional bipedal gait by deep reinforcement learning,” in2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 1276–1282. [Online]. Available: https://arxiv.org/abs/2106.00534
Pith/arXiv arXiv 2021
-
[14]
Robust feedback motion policy design using reinforcement learning on a 3d digit bipedal robot,
G. A. Castillo, B. Weng, W. Zhang, and A. Hereid, “Robust feedback motion policy design using reinforcement learning on a 3d digit bipedal robot,”arXiv preprint arXiv:2103.15309, 2021. [Online]. Available: https://arxiv.org/abs/2103.15309
Pith/arXiv arXiv 2021
-
[15]
Allsteps: Curriculum-driven learning of stepping stone skills,
Z. Xie, H. Y . Ling, N. H. Kim, and M. van de Panne, “Allsteps: Curriculum-driven learning of stepping stone skills,”Computer Graphics Forum, vol. 39, no. 8, pp. 213–224, 2020. [Online]. Available: https: //arxiv.org/abs/2005.04323
Pith/arXiv arXiv 2020
-
[16]
Learning dynamic bipedal walking across stepping stones,
H. Duan, A. Malik, M. S. Gadde, J. Dao, A. Fern, and J. Hurst, “Learning dynamic bipedal walking across stepping stones,”arXiv preprint arXiv:2205.01807, 2022. [Online]. Available: https://arxiv.org/abs/2205.01807
Pith/arXiv arXiv 2022
-
[17]
Proximal policy optimization algorithms,
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017
Pith/arXiv arXiv 2017
-
[18]
mc-mujoco: Simulating articulated robots with fsm controllers in mujoco,
R. P. Singh, P. Gergondet, and F. Kanehiro, “mc-mujoco: Simulating articulated robots with fsm controllers in mujoco,” in2023 IEEE/SICE International Symposium on System Integration (SII). IEEE, 2023, pp. 1–5
2023
-
[19]
Proposal of inspection and rescue tasks for tunnel disasters—task development of japan virtual robotics challenge,
M. Okugawa, K. Oogane, M. Shimizu, Y . Ohtsubo, T. Kimura, T. Takahashi, and S. Tadokoro, “Proposal of inspection and rescue tasks for tunnel disasters—task development of japan virtual robotics challenge,” in2015 IEEE international symposium on safety, security, and rescue robotics (SSRR). IEEE, 2015, pp. 1–2
2015
-
[20]
Learning bipedal walking on planned footsteps for humanoid robots,
R. P. Singh, M. Benallegue, M. Morisawa, R. Cis- neros, and F. Kanehiro, “Learning bipedal walking on planned footsteps for humanoid robots,” in2022 IEEE- RAS 21st International Conference on Humanoid Robots (Humanoids). IEEE, 2022, pp. 686–693
2022
-
[21]
Anytime search-based footstep planning with suboptimality bounds,
A. Hornung, A. Dornbush, M. Likhachev, and M. Ben- newitz, “Anytime search-based footstep planning with suboptimality bounds,” in2012 12th IEEE-RAS Inter- national Conference on Humanoid Robots (Humanoids 2012), 2012, pp. 674–679
2012
-
[2019]
Available: https://www.science.org/doi/ 10.1126/scirobotics.aau5872
[Online]. Available: https://www.science.org/doi/ 10.1126/scirobotics.aau5872
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.