Pith. sign in

REVIEW 4 major objections 3 minor 18 references

Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Modeling a biped's coupled knee-ankle linkage as a closed kinematic chain, rather than a serial chain, determines whether a learned controller falls under pushes or walks reliably in the real world.

desk verdict Plausible but under-supported: the closed-chain RL result is interesting, but the baseline is likely unfair and the headline 81% number leaks from a table inconsistency. read the letter →

arxiv 2507.10164 v1 pith:U7ALWPQ4 submitted 2025-07-14 cs.RO

Egor Maslennikov , Eduard Zaliaev , Nikita Dudorov , Oleg Shamanin , Karanov Dmitry , Gleb Afanasev , Alexey Burkov , Egor Lygin
show 2 more authors
Simeon Nedelchev Evgeny Ponomarev
This is my paper
classification cs.RO
keywords bipedallocomotionclosedkinematicchainsreinforcementlearningsim-to-realtransferparallelmechanismsadversarialtrainingsymmetry-awareTopArobot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when a bipedal robot's legs contain a closed kinematic chain—here, a coupled knee-ankle linkage—training the policy on a simplified serial-chain model is the main reason sim-to-real transfer fails. The authors train three policies on their TopA robot: one with open-chain (serial) kinematics, one with the true closed-chain model, and one with closed-chain modeling plus adversarial command and force perturbations. Across velocity-tracking, push-resistance, and real-world walking tests, the closed-chain policies outperform the serial one, and the adversarially trained version walks a 100-meter indoor course in roughly half the time of the non-adversarial closed-chain policy. The intended upshot is that faithful modeling of parallel mechanisms, not just more domain randomization or better policy architectures, is a practical necessity for learning bipedal locomotion that transfers.

What carries the argument

The load-bearing object is the closed kinematic chain itself, expressed as a linear coupling between motor and joint coordinates. For TopA's leg, $q_{\text{joint}} = [[1,0],[-1,1]] q_{\text{motor}}$ ties knee and ankle motion together, so a command to one motor moves both joints; the authors use this constraint directly in simulation rather than treating the leg as two independent serial joints. Carrying the argument are three training components layered on top of a PPO actor-critic policy: a symmetry loss that mirrors observations and actions across the sagittal plane, weight decay on the last actor layer to avoid extreme position targets, and a cyclic adversarial training scheme that perturbs commanded velocities, gravity readings, and foot forces. The closed-chain model is what the ablation studies compare against, while the symmetry, decay, and adversary components are what separate the C and C+A policies.

What would settle it

Train the serial-baseline policy (O) and the closed-chain policy (C) with identical observation and action spaces, network architecture, rewards, and domain-randomization ranges, then compare time-to-fall and velocity-tracking MAE under the same push distribution; if O's average fall time approaches C+A's 9.71 s or its rotation-tracking MAE falls below the reported gap, the claim that serial simplification is the dominant impairment is falsified. Repeating the comparison with the open-chain model augmented by artificially coupled noise on knee and ankle would isolate whether the benefit comes from model fidelity or from extra regularization.

Watch

Extended reading notes

Core claim

The paper's central claim is that explicitly modeling closed kinematic chains in the simulator is essential for effective reinforcement learning of bipedal locomotion, because the motor-to-joint coupling, friction, and control-space effects of parallel mechanisms are not captured by serial approximations. On TopA, whose knee and ankle are coupled by the motor-to-joint map $[q_k, q_a]_j^T = [[1,0],[-1,1]][q_k,q_a]_m^T$, the closed-chain policy (C) and the closed-chain-plus-adversarial policy (C+A) both beat the open-chain policy (O) in mean tracking error, torque effort, time-to-fall under torso pushes, and command-noise tolerance; the largest tracking improvement is about 81% on the rotation task, and average fall time rises from 4.77 s (O) to 8.48 s (C) to 9.71 s (C+A). On the physical robot, O fails to complete a 100 m course, while C takes about 6 minutes and C+A about 3.1 minutes. The authors infer that the serial-model simplification is the dominant cause of the sim-to-real gap for such platforms.

Load-bearing premise

The central comparison assumes that the only difference between the serial and closed-chain policies is the kinematic model, so that the measured performance gap is caused by modeling the closed chain rather than by a mismatch in action space, observation vector, network size, reward, or randomization between the two setups.

Editorial extensions

If this is right

  • A policy that sees true coupled knee-ankle dynamics is more torque-efficient: closed-chain policies report lower mean torque than the serial baseline across most tasks, so closed-chain modeling can reduce actuator effort, not only improve tracking.
  • Closed-chain modeling alone buys most of the fall-time improvement (4.77 to 8.48 s), and adversarial training adds the remainder (to 9.71 s) plus a wider, more symmetric stability region in the push-perturbation heatmaps.
  • Adversarial command perturbations are what make the policy usable under noisy operator inputs: C drops to 12% success at high command noise, while C+A holds 73%.
  • In real-world deployment, the serial-model policy is not a degraded alternative but a failure mode: it cannot complete a 100 m course, whereas both closed-chain policies do.
  • For designers of legged robots with parallel mechanisms, the results imply that simulator fidelity to closed-chain constraints should be treated as part of the control policy's training data, not as an optional accuracy refinement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference (editorial): the 81% figure is reported on the rotation task, where the serial policy falls; the authors do not claim uniform gains, so readers should expect the benefit to concentrate on tasks that stress the coupled ankle-knee coordination rather than on simple forward walking.
  • Inference (editorial): a testable extension is to train the serial baseline with substantially stronger domain randomization over friction, inertia, and motor friction; if the gap narrows, part of the closed-chain benefit may be an implicit augmentation of the dynamics distribution rather than model fidelity per se.
  • Inference (editorial): because the coupling matrix is linear and holonomic, the same argument should apply to other closed-chain bipeds and to parallel wrists or fingers, but the quantitative gains will depend on how strongly the constraint couples the actuators—a weakly coupled chain may show little benefit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper presents an RL framework for a custom bipedal robot, TopA, whose legs contain closed kinematic chains (a coupled knee-ankle mechanism). The authors train policies with an open-loop kinematic model (O), a closed-loop model (C), and a closed-loop model with adversarial training (C+A). They report that C and C+A achieve substantially better velocity tracking, robustness to pushes and command noise, and successful real-world walking, with a claimed improvement of up to 81% in tracking accuracy. The central claim is that explicitly modeling closed kinematic chains in training is essential for effective sim-to-real transfer in bipedal locomotion.

Significance. If the central claim is supported, the result is practically significant: most humanoid and legged robots use some form of parallel mechanism, and this paper provides a concrete demonstration that ignoring the coupling in simulation harms learned policies. The paper also contributes a robustness recipe (symmetry loss, weight decay, adversarial training) applied to a closed-chain platform, and it includes a real-robot validation. The significance, however, hinges on the validity of the O versus C comparison, which currently has an important gap. The contribution could become valuable after a proper control experiment, but as written the evidence is not yet convincing.

major comments (4)
  1. [III-B and Eq. (5)] The open-loop baseline O is not a proper control for the invertible coupling in Eq. (5). Since the matrix M = [[1,0],[-1,1]] has determinant 1 and inverse [[1,0],[1,1]], commands from a serial policy can be transformed to motor commands by applying M^{-1} at deployment. The paper never deploys O with this inverse transform; it uses O's raw outputs. Consequently, the observed gap could be due to an omitted coordinate transformation at execution rather than to a failure of RL training with simplified dynamics. To support the central claim, please run O with the inverse action transform (both in simulation and on the robot) and report whether the gap persists. If O+M^{-1} matches C, the claim 'accurate modeling of closed kinematic chains is essential' is not supported by these experiments.
  2. [Table IV] Table IV contains an internal inconsistency: in the 'Walk forward' block, the C row is marked with an asterisk whose footnote reads 'Failure due to fall'. This contradicts the narrative that C is the improved closed-loop model and also contradicts the paper's own description of Table IV. Moreover, the '81% better tracking accuracy' claim in the conclusion appears to be derived from the rotate tasks, where the O baseline is also marked as having fallen. If a policy fell during an episode, the MAE numbers are not comparable to those of a stable policy. Please correct the table and clarify how per-episode MAE is aggregated for episodes in which the robot falls.
  3. [III-B and III-D] The simulation results (Tables IV-V, Fig. 5) are reported without repeated seeds or error bars, and the real-world validation is a single demonstration without multiple trials or quantitative success criteria. Given that the paper's headline claims are quantitative (81% improvement, 104% fall-time improvement, 73% vs 12% success), the absence of variance information makes it impossible to assess the reliability of these numbers. Please add at least three seeds for the ablation and robustness experiments, report mean±std or confidence intervals, and describe how many real-world runs were performed.
  4. [III-B] The paper states that O and C share 'identical reward structures..., domain randomization parameters..., weight decay regularization, and symmetry considerations,' but it does not specify whether the state and action spaces are identical under the open-loop model. Since the state vector includes motor positions (q_t), if O treats these as joint positions, the policy's observation semantics differ from C. Please state explicitly whether the observation vector and the action vector are the same for O and C, and if they differ, how this is handled in the comparison.
minor comments (3)
  1. [Throughout] There are several typos and grammatical errors, e.g., 'he experimental results' at the start of Section IV and the garbled sentence in Section III-A: 'The control policy π operates at 50 Hz and provides position commands for all actuators of the robot: at ∼ qcmd as well as other robot software runs on an onboard PC.' Please proofread carefully.
  2. [References] Reference [10] is formatted incorrectly: 'T. M. F. Shi, C. Zhang' should be 'T. M. F. Shi and C. Zhang', and the title and author list need to be verified against the published source.
  3. [Fig. 5] The heatmaps in Figure 5 use a color scale that begins at 2 or 3 rather than 0, which may exaggerate differences near the floor; consider plotting the full range from 0 to 10 for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim rests on empirical sim and real-robot comparisons, not on fitted parameters or self-citations.

full rationale

This manuscript is an empirical comparison. The claim that accurate modeling of closed kinematic chains is essential is supported by ablations O/C/C+A evaluated in MuJoCo (Table IV, Fig. 5) and in real-world trials (Section III-D), not by a derivation from the kinematic model itself. The motor-to-joint coupling in Eq. 5 is an input model, and the improvement of C over O is measured behavior, not an algebraic consequence of the equations. The skeptic's observation that O might be improved by applying M^{-1} at deployment is an omitted counterfactual and a threat to internal validity, but it is not circularity: no equation in the paper reduces the claimed prediction to an input parameter, no fitted parameter is renamed as a prediction, and no load-bearing argument relies on self-citation. The adversarial training is evaluated under perturbations related to those used during training, which is a standard robustness-evaluation design rather than a definitional equivalence. The C* fall inconsistency in Table IV is a correctness or reporting issue, not a circular step. Therefore no significant circularity is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities; TopA is a physical robot, not a postulated construct. The ledger instead tracks hand-tuned training choices and the domain assumptions that the linear coupling map and simulator fidelity are sufficient.

free parameters (5)
  • Reward weights (Table II) = e.g., linear velocity tracking 3.0, angular velocity tracking 1.5, action rate -2.0, joint limits -5.0
    Hand-set weights define the gait reward; no sensitivity analysis is reported, so the learned behavior and downstream comparisons depend on this tuning.
  • Domain randomization ranges (Table III) = friction scale 0.75-1.25, mass scale 0.9-1.1, push velocity 0-2.5 m/s, etc.
    Sim-to-real transfer and robustness results depend on these chosen ranges, and they were not swept.
  • Symmetry loss and weight decay coefficients (Eq. 4) = not reported
    The combined loss is L = L_actor + L_critic + L_symmetry + L_decay, but the multiplicative weights for L_symmetry and lambda in L_decay are omitted, leaving the training objective underspecified.
  • LKF parameters for linear velocity estimation = not specified
    The linear Kalman filter was tuned against OptiTrack ground truth; these values affect the observations fed to the policy and are not reported.
  • PPO and network hyperparameters (Table I) = learning rate 3e-5, clip 0.2, GAE lambda 0.95, discount 0.99, layers [512,256,128]
    The learning algorithm and architecture choices affect policy performance and were selected without a reported sweep; they are part of the training recipe that produces the reported results.
assumptions (4)
  • domain assumption The linear motor-to-joint map (Eq. 5) plus the simulator's multibody dynamics is sufficient to capture all closed-chain effects relevant to locomotion, including joint coupling, friction, and motor-space control.
    The central claim rests on this: if dynamic effects require more than a kinematic map, the comparison to the open-loop model is incomplete. Invoked in Section III-A.
  • domain assumption The open-loop (O) policy and closed-loop (C/C+A) policies are comparable despite the different kinematic models, with no change in observation/action representation or training details.
    Section III-B states all policies used identical rewards, domain randomization, weight decay, and symmetry, but does not show that the action spaces are commensurate.
  • domain assumption IsaacLab and MuJoCo simulations, after actuator parameter identification, faithfully predict real TopA behavior closely enough for conclusions about sim-to-real transfer.
    The real-world validation is interpreted as confirming the simulation results, which presupposes simulator fidelity. Invoked in Sections II-B and III-A.
  • domain assumption The XZ-plane symmetry map used in the symmetry loss is a valid symmetry for TopA's dynamics.
    Symmetry loss assumes left-right symmetry in observations and actions; unmodeled asymmetries in mass distribution or actuator properties could bias the policy. Invoked in Section II-C.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains." pith.science (2026). https://pith.science/paper/U7ALWPQ4

@misc{pith2026250710164,
  author       = {Pith},
  title        = {Pith review of: Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U7ALWPQ4}},
  note         = {Machine review of arXiv:2507.10164}
}
read the original abstract

Developing robust locomotion controllers for bipedal robots with closed kinematic chains presents unique challenges, particularly since most reinforcement learning (RL) approaches simplify these parallel mechanisms into serial models during training. We demonstrate that this simplification significantly impairs sim-to-real transfer by failing to capture essential aspects such as joint coupling, friction dynamics, and motor-space control characteristics. In this work, we present an RL framework that explicitly incorporates closed-chain dynamics and validate it on our custom-built robot TopA. Our approach enhances policy robustness through symmetry-aware loss functions, adversarial training, and targeted network regularization. Experimental results demonstrate that our integrated approach achieves stable locomotion across diverse terrains, significantly outperforming methods based on simplified kinematic models.

Figures

Figures reproduced from arXiv: 2507.10164 by the authors.

Figure 1
Figure 1. The TopA bipedal robot. [8], these methods address symptoms rather than resolving the fundamental challenge of accurately modeling closed kinematic chains. In parallel with these modeling challenges, the RL com￾munity has made significant advances in policy learning tech￾niques with direct applications to bipedal locomotion. No￾table developments include leveraging morphological sym￾metry for more natural gaits [9],… view at source ↗
Figure 2
Figure 2. Policy training scheme. TABLE III: Overview of Domain Randomization. Parameter Unit Range Operator Friction with ground - [0.75, 1.25] scale Link masses - [0.9, 1.1] scale Push velocity m/s [0.0, 2.5] add Linear velocity m/s [-0.1, 0.1] add Angular velocity rad/s [-0.2, 0.2] add Motor positions rad [-0.01, 0.01] add Motor velocities rad/s [-1.5, 1.5] add C. Robustness enhancement techniques While the core training m… view at source ↗
Figure 3
Figure 3. An overview of our cyclic adversarial training. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The TopA walks in real world. A. Hardware platform TopA is primarily constructed from 3D-printed plastic components, with aluminum rods providing structural rein￾forcement. The robot features 5-DOF legs, including hip roll, hip yaw, hip pitch, knee pitch, and ankle pit…
Figure 5
Figure 5. Figure 5: Average time to fall under applied external force. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 11 canonical work pages

  1. [1]

    Design and Control of a Bipedal Robotic Character

    R. Grandia, E. Knoop, M. A. Hopkins, G. Wiedebach, J. Bishop, S. Pickles, D. M ¨uller, and M. B ¨acher, “Design and Control of a Bipedal Robotic Character,” Jan. 2025, arXiv:2501.05204. [Online]. Available: http://arxiv.org/abs/2501.05204

    work page Pith review arXiv 2025

  2. [2]

    Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior

    G. B. Margolis and P. Agrawal, “Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior,” Dec. 2022, arXiv:2212.03238. [Online]. Available: http://arxiv.org/ abs/2212.03238

    work page Pith review arXiv 2022

  3. [3]

    Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking,

    B. v. Marum, A. Shrestha, H. Duan, P. Dugar, J. Dao, and A. Fern, “Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking,” Aug. 2024, arXiv:2404.19173. [Online]. Available: http://arxiv.org/abs/2404.19173

    arXiv 2024

  4. [4]

    Berkeley humanoid: A research platform for learning-based control,

    Q. Liao, B. Zhang, X. Huang, X. Huang, Z. Li, and K. Sreenath, “Berkeley humanoid: A research platform for learning-based control,” 2024

    work page 2024

  5. [5]

    HiLo: Learning Whole-Body Human-like Locomotion with Motion Tracking Controller

    Q. Zhang, C. Weng, G. Li, F. He, and Y . Cai, “HiLo: Learning Whole-Body Human-like Locomotion with Motion Tracking Controller,” Feb. 2025, arXiv:2502.03122. [Online]. Available: http://arxiv.org/abs/2502.03122

    work page Pith review arXiv 2025

  6. [6]

    Exbody2: Advanced expressive humanoid whole-body control,

    M. Ji, X. Peng, F. Liu, J. Li, G. Yang, X. Cheng, and X. Wang, “Exbody2: Advanced expressive humanoid whole-body control,”arXiv preprint arXiv:2412.13196, 2024

    arXiv 2024

  7. [7]

    Real-world humanoid locomotion with reinforcement learning,

    I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Real-world humanoid locomotion with reinforcement learning,” Science robotics, vol. 9, p. eadi9579, 04 2024

    work page 2024

  8. [8]

    Feedback control for cassie with deep reinforcement learning,

    Z. Xie, G. Berseth, P. Clary, J. Hurst, and M. van de Panne, “Feedback control for cassie with deep reinforcement learning,” in2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2018, pp. 1241–1246

    work page 2018

Show all 18 references
  1. [9]

    On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis,

    D. Ordonez-Apraez, M. Martin, A. Agudo, and F. Moreno-Noguer, “On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis,” arXiv preprint arXiv:2302.10433 , 2023

    2023 arXiv

  2. [10]

    Rethinking robustness assessment: Adver- sarial attacks on learning-based quadrupedal locomotion controllers,

    T. M. F. Shi, C. Zhang, “Rethinking robustness assessment: Adver- sarial attacks on learning-based quadrupedal locomotion controllers,” arXiv preprint arXiv:2405.12424v2 , 2024

    2024 arXiv

  3. [11]

    Learn- ing humanoid locomotion over challenging terrain,

    I. Radosavovic, S. Kamat, T. Darrell, and J. Malik, “Learn- ing humanoid locomotion over challenging terrain,” arXiv preprint arXiv:2410.03654, 2024

    2024 arXiv

  4. [12]

    Proximal Policy Optimization Algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” Aug. 2017, arXiv:1707.06347 [cs]. [Online]. Available: http://arxiv.org/abs/ 1707.06347

    2017 arXiv

  5. [13]

    Learning humanoid locomotion with perceptive internal model,

    J. Long, J. Ren, M. Shi, Z. Wang, T. Huang, P. Luo, and J. Pang, “Learning humanoid locomotion with perceptive internal model,” arXiv preprint arXiv:2411.14386 , 2024

    2024 arXiv

  6. [14]

    Learning symmetry and low-energy locomotion,

    W. Yu, G. Turk, and C. K. Liu, “Learning symmetry and low-energy locomotion,” arXiv preprint arXiv:1801.08093 , 2018

    2018 arXiv

  7. [15]

    Humanoid whole-body locomotion on narrow terrain via dynamic balance and reinforcement learning,

    W. Xie, C. Bai, J. Shi, J. Yang, Y . Ge, W. Zhang, and X. Li, “Humanoid whole-body locomotion on narrow terrain via dynamic balance and reinforcement learning,” arXiv preprint arXiv:2502.17219 , 2025

    2025 arXiv

  8. [16]

    Mit cheetah 3: Design and control of a robust, dynamic quadruped robot,

    M. Powell, B. Katz, J. Carlo, P. Wensing, and S. Kim, “Mit cheetah 3: Design and control of a robust, dynamic quadruped robot,” 10 2018

    2018

  9. [17]

    Robot operating system 2: Design, architecture, and uses in the wild,

    S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” Science Robotics , vol. 7, no. 66, p. eabm6074,

  10. [2022]

    Available: https://www.science.org/doi/abs/10.1126/ scirobotics.abm6074

    [Online]. Available: https://www.science.org/doi/abs/10.1126/ scirobotics.abm6074

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.