A proprioceptive humanoid policy trained with slope-adaptive ZMP regularization plus biomechanical reward gating traverses outdoor grass slopes to 32.1° without online exteroception.
hub
Fasttd3: Simple, fast, and capable reinforcement learning for humanoid control
15 Pith papers cite this work. Polarity classification is still indexing.
15
Pith papers citing it
abstract
Reinforcement learning (RL) has driven significant progress in robotics, but its complexity and long training times remain major bottlenecks. In this report, we introduce FastTD3, a simple, fast, and capable RL algorithm that significantly speeds up training for humanoid robots in popular suites such as HumanoidBench, IsaacLab, and MuJoCo Playground. Our recipe is remarkably simple: we train an off-policy TD3 agent with several modifications -- parallel simulation, large-batch updates, a distributional critic, and carefully tuned hyperparameters. FastTD3 solves a range of HumanoidBench tasks in under 3 hours on a single A100 GPU, while remaining stable during training. We also provide a lightweight and easy-to-use implementation of FastTD3 to accelerate RL research in robotics.
hub tools
citation-role summary
background 2
baseline 1
citation-polarity summary
representative citing papers
ACPO claims an exact decentralized decomposition of the cooperative MARL joint policy gradient via serialized agent-chained beliefs, but a key proof step is invalid, so the central claim is not established.
AnyBody distills a privileged teacher tracker into a latent unit-sphere representation and uses a masked transformer to drive humanoid control from arbitrary keypoint subsets.
UniLab is a CPU/GPU heterogeneous system for robot RL training using MuJoCoUni and MotrixSim backends that reports 3-10x end-to-end efficiency improvements and cross-platform compatibility beyond CUDA.
Hypernetworks map a forcing parameter directly to policy weights in an RL framework, enabling unified stabilization of the Kuramoto-Sivashinsky equation across regimes with KAN architectures showing strongest extrapolation.
FlashSAC improves training speed and final performance of off-policy RL on high-dimensional robot tasks by reducing update frequency, increasing model scale, and bounding norms to limit critic error accumulation.
ExpertGen generates high-success expert policies in simulation from imperfect priors by freezing a diffusion behavior model and optimizing its initial noise via RL, then distills them for real-robot deployment.
FastDSAC enables state-of-the-art maximum entropy RL for high-dimensional humanoid control via entropy redistribution per dimension and improved continuous value estimation.
ViTacFormer learns a cross-modal visuo-tactile latent space with autoregressive tactile prediction and an easy-to-hard curriculum, then uses the representation for imitation learning that yields ~50% higher success and the first reported 11-stage, 2.5-minute autonomous dexterous tasks.
MR.Q combines predictive auxiliary tasks with high-capacity value functions in a model-free architecture to achieve strong multitask RL performance without planning.
Non-uniform replay helps most when replay volume is low; high-entropy sampling remains important, and a truncated geometric distribution delivers better sample efficiency with negligible overhead.
Dyna-SAuR learns scalable safety filters and policies from an uncertainty-aware model, cutting failures by two orders of magnitude on CartPole and MuJoCo Walker tasks.
REPPO is an on-policy RL method that combines pathwise policy gradients with relative entropy constraints to achieve stable training and high sample efficiency without replay buffers.
FastDSAC adds a truncated Gaussian policy constraint to distributional actor-critic methods to preserve network plasticity and accelerate training for scalable humanoid locomotion in parallel sampling setups.
ReFPO adds explicit Reflow regularization to FPO, stabilizing PPO-style training and supporting high-fidelity one-step inference across GridWorld, MuJoCo, and Humanoid tasks.
citing papers explorer
-
Physics-Guided Biomechanical Gait Adaptation for Humanoid Locomotion on Extreme Sloped Terrains
A proprioceptive humanoid policy trained with slope-adaptive ZMP regularization plus biomechanical reward gating traverses outdoor grass slopes to 32.1° without online exteroception.
-
ACPO: Agent-Chained Policy Optimization for Multi-Agent Reinforcement Learning
ACPO claims an exact decentralized decomposition of the cooperative MARL joint policy gradient via serialized agent-chained beliefs, but a key proof step is invalid, so the central claim is not established.
-
AnyBody: Free-Form Whole-Body Humanoid Control from Arbitrary Keypoint Guidance
AnyBody distills a privileged teacher tracker into a latent unit-sphere representation and uses a masked transformer to drive humanoid control from arbitrary keypoint subsets.
-
UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms
UniLab is a CPU/GPU heterogeneous system for robot RL training using MuJoCoUni and MotrixSim backends that reports 3-10x end-to-end efficiency improvements and cross-platform compatibility beyond CUDA.
-
Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEs
Hypernetworks map a forcing parameter directly to policy weights in an RL framework, enabling unified stabilization of the Kuramoto-Sivashinsky equation across regimes with KAN architectures showing strongest extrapolation.
-
FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control
FlashSAC improves training speed and final performance of off-policy RL on high-dimensional robot tasks by reducing update frequency, increasing model scale, and bounding norms to limit critic error accumulation.
-
ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priors
ExpertGen generates high-success expert policies in simulation from imperfect priors by freezing a diffusion behavior model and optimizing its initial noise via RL, then distills them for real-robot deployment.
-
FastDSAC: Unlocking the Potential of Maximum Entropy RL in High-Dimensional Humanoid Control
FastDSAC enables state-of-the-art maximum entropy RL for high-dimensional humanoid control via entropy redistribution per dimension and improved continuous value estimation.
-
ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulation
ViTacFormer learns a cross-modal visuo-tactile latent space with autoregressive tactile prediction and an easy-to-hard curriculum, then uses the representation for imitation learning that yields ~50% higher success and the first reported 11-stage, 2.5-minute autonomous dexterous tasks.
-
Representation Learning Enables Scalable Multitask Deep Reinforcement Learning
MR.Q combines predictive auxiliary tasks with high-capacity value functions in a model-free architecture to achieve strong multitask RL performance without planning.
-
When Does Non-Uniform Replay Matter in Reinforcement Learning?
Non-uniform replay helps most when replay volume is low; high-entropy sampling remains important, and a truncated geometric distribution delivers better sample efficiency with negligible overhead.
-
Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty
Dyna-SAuR learns scalable safety filters and policies from an uncertainty-aware model, cutting failures by two orders of magnitude on CartPole and MuJoCo Walker tasks.
-
Relative Entropy Pathwise Policy Optimization
REPPO is an on-policy RL method that combines pathwise policy gradients with relative entropy constraints to achieve stable training and high sample efficiency without replay buffers.
-
FastDSAC: Enhancing Policy Plasticity via Constrained Exploration for Scalable Humanoid Locomotion
FastDSAC adds a truncated Gaussian policy constraint to distributional actor-critic methods to preserve network plasticity and accelerate training for scalable humanoid locomotion in parallel sampling setups.
-
ReFPO: Reflow Regularization for Flow Matching Policy Gradients
ReFPO adds explicit Reflow regularization to FPO, stabilizing PPO-style training and supporting high-fidelity one-step inference across GridWorld, MuJoCo, and Humanoid tasks.