Pith. sign in

REVIEW 3 major objections 5 minor 38 cited by

RSL-RL argues that a compact, GPU-only RL library centered on PPO and DAgger-style behavior cloning can meet most robotics research needs, including sim-to-real transfer.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 17:34 UTC pith:LETPGUWX

load-bearing objection RSL-RL is a real, widely used library and the paper explains it clearly, but the high-throughput claim is asserted, not shown, and the validation rests on a chain of prior papers. the 3 major comments →

arxiv 2509.10771 v1 pith:LETPGUWX submitted 2025-09-13 cs.RO cs.LG

RSL-RL: A Learning Library for Robotics Research

Clemens Schwarke , Mayank Mittal , Nikita Rudin , David Hoeller , Marco Hutter This is my paper
classification cs.RO cs.LG
keywords reinforcement learningroboticspolicy distillationPPObehavior cloningGPU-only trainingsim-to-real transfersparse-reward exploration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

RSL-RL is an open-source reinforcement learning library whose stated purpose is to serve robotics research specifically, in contrast to general-purpose RL frameworks. The paper argues that a compact, modifiable codebase built around PPO plus DAgger-style behavior cloning, with GPU-only training and robotics-specific auxiliary techniques, is enough to cover a large share of robot learning practice. It claims this design yields high throughput in large-scale simulation and has been validated through its use in published simulation and real-world robot results. The practical point is that robotics researchers should be able to take one small library, modify a few files, and go from idea to trained policy in minutes rather than assembling heavy infrastructure.

Core claim

The paper argues that a deliberately narrow RL library can serve the robotics community better than a broad framework. Concretely, RSL-RL implements exactly two algorithms—PPO for on-policy learning and a DAgger-style behavior-cloning routine for distilling a trained teacher into a deployable student—plus symmetry augmentation and a partial-state curiosity reward. The framework is organized into three independently modifiable components (Runner, Algorithm, Network) with an environment interface that passes observations as nested tensor containers, and training runs entirely on GPU with native multi-GPU and multi-node support. The claimed payoff is that this combination has produced walking p

What carries the argument

The load-bearing piece is the three-part architecture—Runner, Algorithm, Network—with a same-step-reset environment interface and nested-tensor observation containers. This structure is what makes the codebase modifiable in a few files and keeps the training loop GPU-only and batched. The PPO implementation carries the throughput; the DAgger-style behavior-cloning loop carries sim-to-real distillation; symmetry augmentation and subset-state curiosity carry the robotics-specific sample-efficiency gains.

Load-bearing premise

The central claim assumes that the throughput and real-world successes credited to RSL-RL by the cited prior applications reflect the library as packaged, not bespoke modifications each project made.

What would settle it

Run the repository's default PPO example on the same GPU simulator and hardware class as the paper's cited 'walking in minutes' result, and measure wall-clock time until a walking gait appears. If training takes hours rather than minutes, or if multi-GPU scaling is not observed, the high-throughput premise fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the paper is right, a robotics lab can train a locomotion or manipulation policy on a single GPU in minutes and deploy it on hardware through the built-in teacher-student distillation path.
  • New robot-learning tasks can be tackled by editing at most the runner, algorithm, and network files, lowering the barrier to reproducing and extending published results.
  • Because the library integrates with several GPU-accelerated simulators, it can serve as a common baseline that makes published robotics results easier to compare.
  • The included symmetry augmentation and curiosity reward become reusable building blocks, reducing manual reward shaping on sparse-reward and symmetric-robot problems.
  • Researchers who need to benchmark many RL algorithms would still require a broader tool; the paper is explicit that RSL-RL is intentionally not designed for that.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If RSL-RL's throughput claim is general, its simple interface could make it a natural testbed for scaling studies: sweeping batch size and GPU count to find where wall-clock performance saturates would be a direct extension the paper does not report.
  • The partial-state curiosity variant suggests a broader design principle: intrinsic rewards can focus on task-relevant degrees of freedom rather than the full state; testing this on non-locomotion sparse-reward tasks would be a cheap next experiment.
  • The paper's emphasis on PPO plus distillation implies that many real deployments do not need off-policy or model-based algorithms—a hypothesis worth testing by comparing policy performance and engineering cost on the same hardware.
  • Because no standalone imitation-learning path is included, extending the behavior-cloning routine into a demonstration-only pipeline would let the library serve tasks where no RL-trained teacher exists.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents RSL-RL, a PyTorch-based reinforcement-learning library aimed at robotics researchers. It describes a deliberately minimal design with three extension points (runners, algorithms, networks), two main algorithms (PPO and DAgger-style behavior cloning), auxiliary techniques (symmetry augmentation and RND-based curiosity), logging and distributed-training utilities, and integration with several GPU-accelerated simulators. The manuscript asserts that the library is compact, easily modifiable, high-throughput, and validated in both simulation and real-world experiments. However, these assertions are not accompanied by any quantitative measurements, benchmarks, training curves, or runtime comparisons within the paper itself; validation is deferred to a list of external publications, many from the same research group. The paper includes a brief discussion of intended use and limitations.

Significance. RSL-RL is a real, publicly available library that has been used in several well-known robotics results, and the paper provides a useful high-level description of its design and components. If the central claim of high-throughput, validated utility were supported by measurements, the paper would be a valuable system description for robotics practitioners selecting a lightweight RL codebase. As it stands, however, the paper functions more as an announcement and an adoption list than as a self-contained evaluation. The absence of direct benchmarks and version pinning creates a reproducibility gap that prevents the reader from verifying the current release against the claimed properties. The authors are in a strong position to close this gap by adding experiments and precise version metadata.

major comments (3)
  1. [Abstract and §1 (Introduction)] The abstract and the bullet point 'High-throughput training' in §1 assert that RSL-RL achieves high-throughput performance, but the manuscript contains no quantitative evidence: no samples-per-second or wall-clock measurements, no hardware description, no training curves, and no comparison with existing libraries such as rl_games, CleanRL, or TorchRL. Section 4 lists external papers, but those cannot validate the current release without a version/commit pin. Please add a benchmark section, at minimum for a canonical legged-locomotion task, reporting throughput and training time, and give the exact commit or release version of the code used.
  2. [§4 (Applications in Research)] The validation-by-publication argument is structurally weak because most cited applications originate from the same group (e.g., Rudin et al. 2022, Schwarke et al. 2023, Hoeller et al. 2024, He et al. 2025). No mapping is provided between each cited work and the RSL-RL version/API described in this paper, so the current codebase could have diverged from those forks. To support the claim of 'effectiveness validated,' state the version or commit used in each external study and include at least one reproducibility test or independent evaluation against the current release.
  3. [§2.2 (Auxiliary Techniques)] The RND modification is described only qualitatively: 'computes the reward using only a subset of the full state of the system.' There is no equation, no detail about how the subset is selected, and no ablation or reference supporting the claimed benefit over standard RND. Since this is presented as a distinguishing feature, please specify the formulation formally and provide evidence, or cite a publication that contains the full details.
minor comments (5)
  1. [Abstract] The repository URL appears as 'rsl rl' because of a line break; fix the URL formatting so it reads 'https://github.com/leggedrobotics/rsl_rl'.
  2. [Figure 1] The figure has no detailed caption. It would help to annotate the data flow between Runners, Algorithms, and Networks, and to indicate which components users typically modify.
  3. [References] Several bibliography entries contain encoding artifacts, e.g., the author name 'Jo ˜AG ¸ o GM Ara˜Aˇ sjo' in the CleanRL reference. Please regenerate or clean the reference list.
  4. [§3 (Implementation Details)] The term 'same-step reset mode' in the VecEnv interface is not defined. Since this is part of the environment contract, give a brief explanation or a pointer to Gymnasium's terminology.
  5. [§2.3 (Utilities)] The claim of 'distributed training over multiple nodes and multi-GPUs' has no configuration details or measured scaling behavior. At minimum, describe the launching mechanism and any known limitations.

Circularity Check

0 steps flagged

No significant circularity; the paper's performance and validation claims rest on external published applications, not on an in-paper derivation that reduces to its inputs.

full rationale

This manuscript is a software description, not a derivation or prediction paper. The central claims—compact design, high-throughput GPU-only training, and demonstrated utility—are presented as properties of an open-source codebase. The paper does not fit any parameter, define a quantity in terms of the quantity it claims to predict, or invoke a uniqueness theorem. Its strongest empirical assertions, such as 'RSL-RL achieves high-throughput performance' and 'effectiveness has been validated in both simulation benchmarks and in real-world robotic experiments,' are supported in Section 4 by a list of externally published peer-reviewed papers, including several from the same ETH/NVIDIA group. That reliance on prior work is a normal evidentiary pattern for a library paper: the cited papers contain their own experiments, benchmarks, and sim-to-real results, and they use the library as a tool rather than presupposing the library's performance claims. The overlap in authorship is a credibility and reproducibility concern, not a logical circularity. The absence of an in-manuscript throughput benchmark, a version pin, or a reproduction script is a verification gap—correctness risk, not a circularity risk. Because no load-bearing step reduces by construction to an input or to an unverified self-citation chain, the paper receives a circularity score of 0.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The paper is a software description, so it has no fitted constants or invented physical entities. Its load-bearing assumptions are domain choices about what robotics researchers need and about the validity of the citation chain used as evidence.

axioms (4)
  • domain assumption Compactness and modifiability are the primary needs of robotics researchers using RL libraries.
    The paper's design philosophy (§1) prioritizes compact code; this is an assumption about user needs, not empirically established in the paper.
  • domain assumption PPO and DAgger-style BC are sufficient for the majority of robotics RL tasks.
    The library deliberately includes only these two algorithms (§2.1, §5), assuming they cover the community's needs.
  • domain assumption GPU-only training is the appropriate target; CPU support is unnecessary.
    Stated in Abstract and §2.3 as a design choice, with no comparison to CPU or mixed pipelines.
  • domain assumption Prior publications cited in §4 used the library in a way that validates its design.
    The paper uses these citations as evidence without reproducing their experiments.

pith-pipeline@v1.3.0-alltime-deepseek · 5925 in / 10309 out tokens · 107312 ms · 2026-08-04T17:34:45.449737+00:00 · methodology

0 comments
read the original abstract

RSL-RL is an open-source Reinforcement Learning library tailored to the specific needs of the robotics community. Unlike broad general-purpose frameworks, its design philosophy prioritizes a compact and easily modifiable codebase, allowing researchers to adapt and extend algorithms with minimal overhead. The library focuses on algorithms most widely adopted in robotics, together with auxiliary techniques that address robotics-specific challenges. Optimized for GPU-only training, RSL-RL achieves high-throughput performance in large-scale simulation environments. Its effectiveness has been validated in both simulation benchmarks and in real-world robotic experiments, demonstrating its utility as a lightweight, extensible, and practical framework to develop learning-based robotic controllers. The library is open-sourced at: https://github.com/leggedrobotics/rsl_rl.

Figures

Figures reproduced from arXiv: 2509.10771 by Clemens Schwarke, David Hoeller, Marco Hutter, Mayank Mittal, Nikita Rudin.

Figure 1
Figure 1. Figure 1: Overview of the framework. RSL-RL consists of three main com￾ponents: Runners, Algorithms, and Networks, which can be easily modified in￾dependently. The framework comes with support for common logging choices and useful extensions for robotics. ©2025 Clemens Schwarke, Mayank Mittal, Nikita Rudin, David Hoeller, and Marco Hutter. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. arXiv:2… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 38 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. EgoEngine: From Egocentric Human Videos to High-Fidelity Dexterous Robot Demonstrations

    cs.RO 2026-06 unverdicted novelty 7.0

    EgoEngine transforms egocentric human videos into high-fidelity robot data enabling zero-shot visuomotor dexterous policy learning without real-robot demonstrations.

  2. Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation

    cs.RO 2026-05 unverdicted novelty 7.0

    CoP tactile representation with differentiable calibration enables zero-shot sim-to-real transfer and outperforms binary and raw-taxel baselines on peg-in-hole insertion and ball balancing with a multi-fingered hand.

  3. Betting for Sim-to-Real Performance Evaluation

    cs.RO 2026-04 unverdicted novelty 7.0

    Betting mechanisms can yield provably more accurate and efficient estimates of real-world robot behavior than Monte Carlo sampling under specified conditions, with practical approximations demonstrated on synthetic da...

  4. HALO: Hybrid Auto-encoded Locomotion with Learned Latent Dynamics, Poincar\'e Maps, and Regions of Attraction

    cs.RO 2026-04 unverdicted novelty 7.0

    HALO learns latent reduced-order models with Poincaré maps for hybrid locomotion dynamics, allowing Lyapunov-based regions of attraction to be lifted from latent space to the full-order system.

  5. Bounded Ratio Reinforcement Learning

    cs.LG 2026-04 conditional novelty 7.0

    BRRL derives an analytic optimal policy for regularized constrained RL that guarantees monotonic improvement and yields the BPO algorithm that matches or exceeds PPO.

  6. Open-DiffLoco: Open-Source Differentiable Learning for Deployable Blind Quadruped Locomotion

    cs.RO 2026-08 conditional novelty 6.0

    Differentiable simulation with SHAC and a simplified reward set trains deployable blind quadruped policies that transfer to a real Unitree Go2, plus a new critic-Jacobian supervision method (JAVE).

  7. Learning Adaptive Multi-Task Guidance, Navigation, and Control via Hypernetworks

    cs.RO 2026-07 conditional novelty 6.0

    A hypernetwork maps continuous physics-informed task embeddings to shared actor-critic weights, mastering four orbital GNC tasks and composing novel ones without retraining, with sim-to-real on a floating platform.

  8. Sling2Sim2Real: One-Shot Elastic System Identification for Non-Destructive Slingshot Policy Learning

    cs.RO 2026-07 conditional novelty 6.0

    A one-shot Real2Sim2Real framework fits five elastic parameters from a single non-destructive interaction and zero-shot transfers a slingshot RL policy to a real Franka arm.

  9. Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

    cs.RO 2026-07 conditional novelty 6.0

    UHAS maps hand actions to deformations of a shared unit sphere and recovers joint commands via cascade IK, enabling multi-hand RL, zero-shot transfer, and modest real-world cube reorientation on LEAP and Allegro.

  10. Learning Locomotion on Discrete Terrain via Minimal Proximity Sensing

    cs.RO 2026-06 unverdicted novelty 6.0

    Foot-mounted proximity sensors provide pre-contact feedback that, when integrated into RL, improves quadruped traversal robustness on discrete terrain with reliable sim-to-real transfer.

  11. MAPL: Multi-Objective Preference Learning for Robot Locomotion

    cs.RO 2026-06 unverdicted novelty 6.0

    MAPL trains quadruped locomotion policies from LLM-generated multi-objective trajectory preferences and matches or exceeds expert-designed reward performance in four environments without manual reward engineering.

  12. Rotation-Aware Point-Cloud Embeddings for Vision-Based In-Hand Reorientation

    cs.RO 2026-06 unverdicted novelty 6.0

    Learns rotation-aware point-cloud embeddings calibrated to SO(3) geodesic error, enabling model-free RL for vision-based in-hand reorientation without pose or flow inputs.

  13. TAGA: Terrain-aware Active Gaze Learning for Generalizable Agile Humanoid Locomotion

    cs.RO 2026-06 unverdicted novelty 6.0

    TAGA learns terrain-aware active gaze behaviors for humanoid robots via RL alone, enabling generalizable locomotion with 1.2m real-world gap traversal.

  14. HORIZON: Recoverability-Governed Curriculum for Physical-Domain Scaling

    cs.RO 2026-06 unverdicted novelty 6.0

    HORIZON is a recoverability-governed checkpointed frontier curriculum for on-policy physical-domain scaling on quadruped locomotion that identifies three regularities: uneven widening, non-monotonic composition, and t...

  15. S-Cheetah: A Novel Quadrupedal Robot with a 3-DOF Active Spine Learning Agile Locomotion

    cs.RO 2026-05 unverdicted novelty 6.0

    A quadruped robot with a three-degree-of-freedom active spine reaches 6.9 m/s top speed and 7.2 rad/s turning rate via an RL framework that rewards spine engagement and gallop gaits.

  16. ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation

    cs.RO 2026-04 unverdicted novelty 6.0

    A framework using 3D Gaussian Splatting for visual domain randomization enables robust monocular RGB-based dexterous in-hand reorientation on real hardware for multiple objects under varied lighting.

  17. PriPG-RL: Privileged Planner-Guided Reinforcement Learning for Partially Observable Systems with Anytime-Feasible MPC

    cs.LG 2026-04 unverdicted novelty 6.0

    PriPG-RL trains RL policies for POMDPs by distilling knowledge from a privileged anytime-feasible MPC planner into a P2P-SAC policy, improving sample efficiency and performance in partially observable robotic navigation.

  18. FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control

    cs.LG 2026-04 unverdicted novelty 6.0

    FlashSAC improves training speed and final performance of off-policy RL on high-dimensional robot tasks by reducing update frequency, increasing model scale, and bounding norms to limit critic error accumulation.

  19. FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control

    cs.LG 2026-04 unverdicted novelty 6.0

    FlashSAC scales up Soft Actor-Critic with fewer updates, larger models, higher data throughput, and norm bounds to deliver faster, more stable training than PPO on high-dimensional robot control tasks across dozens of...

  20. SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows

    cs.RO 2026-02 unverdicted novelty 6.0

    SERNF achieves sample-efficient real-world fine-tuning of multimodal dexterous policies by pairing exact-likelihood normalizing flow policies with action-chunked value critics.

  21. SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows

    cs.RO 2026-02 conditional novelty 6.0

    SERNF fine-tunes dexterous manipulation policies on real hardware by pairing normalizing-flow policies with action-chunked critics and conservative off-policy RL.

  22. Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning

    cs.RO 2025-11 unverdicted novelty 6.0

    Isaac Lab is a unified GPU-native platform combining high-fidelity physics, photorealistic rendering, multi-frequency sensors, domain randomization, and learning pipelines for scalable multi-modal robot policy training.

  23. Pretraining in Actor-Critic Reinforcement Learning for Locomotion

    cs.RO 2025-10 conditional novelty 6.0

    Pretraining an inverse dynamics model on task-agnostic exploration data and using its weights to initialize PPO actor and critic improves sample efficiency by ~37% and final performance by ~7% across nine simulated lo...

  24. RANDPOL: Parameter-Efficient End-to-End Quadruped Locomotion via Randomized Policy Learning

    cs.LG 2025-05 unverdicted novelty 6.0

    RANDPOL achieves effective quadruped locomotion by training only the final linear readout of a randomly initialized and fixed neural network policy, matching PPO results with reduced parameters and enabling zero-shot ...

  25. PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball

    cs.RO 2026-07 conditional novelty 5.0

    Link-CBF training with masked onboard depth lets a humanoid dodge whole-body contact near oracle rates in sim and on 95% of real throws, while stronger Joint-CBF needs better ball observability.

  26. Learning Locomotion on Discrete Terrain via Minimal Proximity Sensing

    cs.RO 2026-06 unverdicted novelty 5.0

    Foot-mounted infrared proximity sensors supply pre-contact signals that, when incorporated into RL, improve quadrupedal traversal of discrete terrain such as gaps and stepping stones.

  27. PPO-EAL: Exact Augmented Lagrangian Proximal Policy Optimization for Safe Robotic Control

    cs.RO 2026-06 unverdicted novelty 5.0

    PPO-EAL integrates exact augmented Lagrangian optimization into PPO for safe robotic control, with claimed theoretical guarantees and better empirical safety-performance tradeoffs on several robot benchmarks including...

  28. CTS-MoE: Implicit Terrain Adaptation via Mixture-of-Experts for Perceptive Locomotion

    cs.RO 2026-06 unverdicted novelty 5.0

    CTS-MoE combines a dense MoE actor with perception-based gating and a multi-critic architecture to enable adaptive perceptive locomotion on discontinuous terrain in a single-stage teacher-student training setup.

  29. Redesigning Regularization for Effective Policy Smoothing

    cs.RO 2026-06 unverdicted novelty 5.0

    Redesigned regularization addresses implementation gaps in policy smoothing for RL, yielding smoother motions with improved performance and robustness on a quadruped robot in sim-to-real settings.

  30. MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds

    cs.RO 2026-06 unverdicted novelty 5.0

    MARCH combines simplified-model trajectory generation with CLF-guided teacher RL and vision-policy distillation to enable stable humanoid locomotion over sparse terrain with better sample efficiency than pure model-fr...

  31. HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers

    cs.RO 2026-06 unverdicted novelty 5.0

    HANDOFF is a distilled mixture-of-experts humanoid whole-body controller that follows a compact task-space interface, matches SOTA velocity tracking, provides large manipulation workspace on Unitree G1, and supports V...

  32. Bridging the Gap: Enabling Soft Actor Critic for High Performance Legged Locomotion

    cs.RO 2026-05 unverdicted novelty 5.0

    Targeted changes to policy initialization, critic targets, and return estimation let SAC match PPO performance across legged locomotion tasks in massively parallel simulation.

  33. Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy

    cs.RO 2026-05 unverdicted novelty 5.0

    Terrain-consistent reference modulation during RL training yields SE(2)-controllable humanoid locomotion policies that improve tracking in simulation and enable over 70 m closed-loop autonomous navigation on rough ter...

  34. End-to-End Crop Row Navigation via LiDAR-Based Deep Reinforcement Learning

    cs.RO 2025-09 conditional novelty 5.0

    Raw 3D LiDAR, compressed into flattened voxel maps, trains a reinforcement learning policy that reliably follows straight crop rows in simulation and degrades on curvier rows.

  35. WaveLander: A Generalizable Hierarchical Control Framework for UAV Landing on Wave-Disturbed Platforms via Reinforcement Learning

    cs.RO 2026-07 unverdicted novelty 4.0

    WaveLander is a hierarchical RL control system for UAV landing on wave-disturbed platforms that uses RL for vertical velocity reference and conventional controller for attitude and lateral stability, showing robust pe...

  36. Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient

    cs.RO 2026-05 unverdicted novelty 4.0

    SDPG is a new on-policy visual RL algorithm that estimates gradients via stochastic perturbations of rollouts, achieving faster training and lower memory use than baselines on visual MuJoCo tasks while adding new robo...

  37. Robotic Strawberry Harvesting with Robust Vision and Deep Reinforcement Learning based Sim-to-Real Control

    cs.RO 2026-05 conditional novelty 4.0

    A modified YOLO segmentation model plus sim-trained PPO control yields 84.3% overall success harvesting 281 strawberries in greenhouse trials on a real UR10e manipulator.

  38. The Unified Autonomy Stack: Toward a Blueprint for Generalizable Robot Autonomy

    cs.RO 2026-05 accept novelty 4.0

    An open-sourced Unified Autonomy Stack fuses LiDAR, radar, vision and inertial data with sampling-based planning and control barrier functions to deliver resilient autonomy on aerial and ground robots in challenging r...

Reference graph

Works this paper leans on

43 extracted references · 6 linked inside Pith · cited by 35 Pith papers

  1. [1]

    Tensorflow: Large-scale machine learning on heterogeneous distributed systems

    Mart \' n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv e-prints, pages arXiv--1603, 2016

    2016

  2. [2]

    Pedipulate: Enabling manipulation skills using a quadruped robot’s leg

    Philip Arm, Mayank Mittal, Hendrik Kolvenbach, and Marco Hutter. Pedipulate: Enabling manipulation skills using a quadruped robot’s leg. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 5717--5723. IEEE, 2024

    2024

  3. [3]

    Experiment tracking with weights and biases, 2020

    Lukas Biewald. Experiment tracking with weights and biases, 2020. URL https://www.wandb.com/. Software available from wandb.com

    2020

  4. [4]

    Torchrl: A data-driven decision-making library for pytorch, 2023

    Albert Bou, Matteo Bettini, Sebastian Dittert, Vikash Kumar, Shagun Sodhani, Xiaomeng Yang, Gianni De Fabritiis, and Vincent Moens. Torchrl: A data-driven decision-making library for pytorch, 2023

    2023

  5. [5]

    Exploration by random network distillation

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. In International Conference on Learning Representations (ICLR), pages 1--17, 2019

    2019

  6. [6]

    Cat: Constraints as terminations for legged locomotion reinforcement learning

    Elliot Chane-Sane, Pierre-Alexandre Leziart, Thomas Flayols, Olivier Stasse, Philippe Sou \`e res, and Nicolas Mansard. Cat: Constraints as terminations for legged locomotion reinforcement learning. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 13303--13310. IEEE, 2024

    2024

  7. [7]

    Extreme parkour with legged robots

    Xuxin Cheng, Kexin Shi, Ananye Agarwal, and Deepak Pathak. Extreme parkour with legged robots. Conference on Robot Learning (CoRL), 2023

    2023

  8. [8]

    Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning

    Ioannis Dadiotis, Mayank Mittal, Nikos Tsagarakis, and Marco Hutter. Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

  9. [9]

    Deep whole-body control: learning a unified policy for manipulation and locomotion

    Zipeng Fu, Xuxin Cheng, and Deepak Pathak. Deep whole-body control: learning a unified policy for manipulation and locomotion. In Conference on Robot Learning (CoRL), pages 138--149. PMLR, 2023

    2023

  10. [10]

    Dextreme: Transfer of agile in-hand manipulation from simulation to reality

    Ankur Handa, Arthur Allshire, Viktor Makoviychuk, Aleksei Petrenko, Ritvik Singh, Jingzhou Liu, Denys Makoviichuk, Karl Van Wyk, Alexander Zhurkevich, Balakumar Sundaralingam, et al. Dextreme: Transfer of agile in-hand manipulation from simulation to reality. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 5977...

    2023

  11. [11]

    Attention-based map encoding for learning generalized legged locomotion

    Junzhe He, Chong Zhang, Fabian Jenelten, Ruben Grandia, Moritz B \"a cher, and Marco Hutter. Attention-based map encoding for learning generalized legged locomotion. Science Robotics, 10 0 (105): 0 eadv3604, 2025

    2025

  12. [12]

    Hover: Versatile neural whole-body controller for humanoid robots

    Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, et al. Hover: Versatile neural whole-body controller for humanoid robots. arXiv preprint arXiv:2410.21229, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Anymal parkour: Learning agile navigation for quadrupedal robots

    David Hoeller, Nikita Rudin, Dhionis Sako, and Marco Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots. Science Robotics, 9 0 (88): 0 eadi7566, 2024

    2024

  14. [14]

    The 37 implementation details of proximal policy optimization

    Shengyi Huang, Rousslan Fernand Julien Dossa, Antonin Raffin, Anssi Kanervisto, and Weixun Wang. The 37 implementation details of proximal policy optimization. In ICLR Blog Track, 2022 a . URL https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/. https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/

    2022

  15. [15]

    Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms

    Shengyi Huang, Rousslan Fernand Julien Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and Jo \ A G o GM Ara \ A s jo. Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research (JMLR), 23 0 (274): 0 1--18, 2022 b

    2022

  16. [16]

    Learning agile and dynamic motor skills for legged robots

    Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter. Learning agile and dynamic motor skills for legged robots. Science Robotics, 4 0 (26): 0 eaau5872, 2019

    2019

  17. [17]

    Learning robust autonomous navigation and locomotion for wheeled-legged robots

    Joonho Lee, Marko Bjelonic, Alexander Reske, Lorenz Wellhausen, Takahiro Miki, and Marco Hutter. Learning robust autonomous navigation and locomotion for wheeled-legged robots. Science Robotics, 9 0 (89): 0 eadi9641, 2024

    2024

  18. [18]

    Marladona-towards cooperative team play using multi-agent reinforcement learning

    Zichong Li, Filip Bjelonic, Victor Klemm, and Marco Hutter. Marladona-towards cooperative team play using multi-agent reinforcement learning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

  19. [19]

    Rllib: Abstractions for distributed reinforcement learning

    Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael Jordan, and Ion Stoica. Rllib: Abstractions for distributed reinforcement learning. In International Conference on Machine Learning (ICML), pages 3053--3062, 2018

    2018

  20. [20]

    rl-games: A high-performance framework for reinforcement learning

    Denys Makoviichuk and Viktor Makoviychuk. rl-games: A high-performance framework for reinforcement learning. https://github.com/Denys88/rl_games, May 2021

    2021

  21. [21]

    Isaac gym: High performance gpu based physics simulation for robot learning

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance gpu based physics simulation for robot learning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021

    2021

  22. [22]

    Margolis and Pulkit Agrawal

    Gabriel B. Margolis and Pulkit Agrawal. Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning (CoRL), volume 205, pages 22--31. PMLR, 2023

    2023

  23. [23]

    Learning robust perceptive locomotion for quadrupedal robots in the wild

    Takahiro Miki, Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7 0 (62), 2022

    2022

  24. [24]

    Orbit: A unified simulation framework for interactive robot learning environments

    Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, et al. Orbit: A unified simulation framework for interactive robot learning environments. IEEE Robotics and Automation Letters (RA-L), 8 0 (6): 0 3740--3747, 2023

    2023

  25. [25]

    Symmetry considerations for learning task symmetric robot policies

    Mayank Mittal, Nikita Rudin, Victor Klemm, Arthur Allshire, and Marco Hutter. Symmetry considerations for learning task symmetric robot policies. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 7433--7439. IEEE, 2024

    2024

  26. [26]

    neptune.ai: experiment tracker, 2024

    neptune.ai . neptune.ai: experiment tracker, 2024. URL https://neptune.ai

    2024

  27. [27]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019

    2019

  28. [28]

    Stable-baselines3: Reliable reinforcement learning implementations

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research (JMLR), 22 0 (268): 0 1--8, 2021

    2021

  29. [29]

    A reduction of imitation learning and structured prediction to no-regret online learning

    St \'e phane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627--635. JMLR Workshop and Conference Proceedings, 2011

    2011

  30. [30]

    Learning to walk in minutes using massively parallel deep reinforcement learning

    Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning (CoRL), pages 91--100, 2022

    2022

  31. [31]

    Parkour in the wild: Learning a general and extensible agile locomotion policy using multi-expert distillation and rl fine-tuning

    Nikita Rudin, Junzhe He, Joshua Aurand, and Marco Hutter. Parkour in the wild: Learning a general and extensible agile locomotion policy using multi-expert distillation and rl fine-tuning. International Journal of Robotics Research (IJRR), 2025

    2025

  32. [32]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv e-prints, pages arXiv--1707, 2017

    2017

  33. [33]

    Curiosity-driven learning of joint locomotion and manipulation tasks

    Clemens Schwarke, Victor Klemm, Matthijs Van der Boon, Marko Bjelonic, and Marco Hutter. Curiosity-driven learning of joint locomotion and manipulation tasks. In Conference on Robot Learning (CoRL), volume 229, pages 2594--2610, 2023

    2023

  34. [34]

    skrl: Modular and flexible library for reinforcement learning

    Antonio Serrano-Munoz, Dimitrios Chrysostomou, Simon B gh, and Nestor Arana-Arexolaleiba. skrl: Modular and flexible library for reinforcement learning. Journal of Machine Learning Research (JMLR), 24 0 (254): 0 1--9, 2023

    2023

  35. [35]

    Reinforcement learning algorithms: A brief survey

    Ashish Kumar Shakya, Gopinatha Pillai, and Sohom Chakrabarty. Reinforcement learning algorithms: A brief survey. Expert Systems with Applications, 231: 0 120495, 2023

    2023

  36. [36]

    Guided reinforcement learning for robust multi-contact loco-manipulation

    Jean-Pierre Sleiman, Mayank Mittal, and Marco Hutter. Guided reinforcement learning for robust multi-contact loco-manipulation. In Conference on Robot Learning (CoRL), 2024

    2024

  37. [37]

    Gymnasium: A standard interface for reinforcement learning environments

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goul \ a o, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Advanced skills through multiple adversarial motion priors in reinforcement learning

    Eric Vollenweider, Marko Bjelonic, Victor Klemm, Nikita Rudin, Joonho Lee, and Marco Hutter. Advanced skills through multiple adversarial motion priors in reinforcement learning. arXiv preprint arXiv:2203.14912, 2022

    Pith/arXiv arXiv 2022

  39. [39]

    Improving long-range navigation with spatially-enhanced recurrent memory via end-to-end reinforcement learning

    Fan Yang, Per Frivik, David Hoeller, Chen Wang, Cesar Cadena, and Marco Hutter. Improving long-range navigation with spatially-enhanced recurrent memory via end-to-end reinforcement learning. arXiv preprint arXiv:2506.05997, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Mujoco playground

    Kevin Zakka, Baruch Tabanpour, Qiayuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A Kahrs, et al. Mujoco playground. Robotics: Science and Systems (RSS), 2025

    2025

  41. [41]

    Learning agile locomotion on risky terrains

    Chong Zhang, Nikita Rudin, David Hoeller, and Marco Hutter. Learning agile locomotion on risky terrains. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11864--11871. IEEE, 2024 a

    2024

  42. [42]

    Wococo: Learning whole-body humanoid control with sequential contacts

    Chong Zhang, Wenli Xiao, Tairan He, and Guanya Shi. Wococo: Learning whole-body humanoid control with sequential contacts. arXiv preprint arXiv:2406.06005, 2024 b

    Pith/arXiv arXiv 2024

  43. [43]

    Penalized proximal policy optimization for safe reinforcement learning

    Linrui Zhang, Li Shen, Long Yang, Shixiang Chen, Bo Yuan, Xueqian Wang, and Dacheng Tao. Penalized proximal policy optimization for safe reinforcement learning. arXiv preprint arXiv:2205.11814, 2022

    Pith/arXiv arXiv 2022