REVIEW 5 cited by
Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose to address quadrupedal locomotion tasks using Reinforcement Learning (RL) with a Transformer-based model that learns to combine proprioceptive information and high-dimensional depth sensor inputs. While learning-based locomotion has made great advances using RL, most methods still rely on domain randomization for training blind agents that generalize to challenging terrains. Our key insight is that proprioceptive states only offer contact measurements for immediate reaction, whereas an agent equipped with visual sensory observations can learn to proactively maneuver environments with obstacles and uneven terrain by anticipating changes in the environment many steps ahead. In this paper, we introduce LocoTransformer, an end-to-end RL method that leverages both proprioceptive states and visual observations for locomotion control. We evaluate our method in challenging simulated environments with different obstacles and uneven terrain. We transfer our learned policy from simulation to a real robot by running it indoors and in the wild with unseen obstacles and terrain. Our method not only significantly improves over baselines, but also achieves far better generalization performance, especially when transferred to the real robot. Our project page with videos is at https://rchalyang.github.io/LocoTransformer/ .
Forward citations
Cited by 5 Pith papers
-
StairMaster: Learning to Conquer Risky Hollow Stairs for Agile Quadrupedal Robots
StairMaster trains an RL policy that lets a Unitree Go2 quadruped climb hollow stairs up to 55 degrees via zero-shot sim-to-real transfer using cross-attention, SRU memory, and active-perception rewards.
-
QuadKAN: KAN-Enhanced Quadruped Motion Control via End-to-End Reinforcement Learning
A KAN-based spline policy for vision-guided quadruped locomotion improves return, distance, and collision avoidance over MLP baselines in PyBullet simulation.
-
Generalized Locomotion in Out-of-distribution Conditions with Robust Transformer
A transformer with body tokenization and consistent dropout generalizes to unseen leg damages and sensor noise while trained on limited dynamics and clean observations.
-
DPL: Depth-only Perceptive Humanoid Locomotion via Realistic Depth Synthesis and Cross-Attention Terrain Reconstruction
Combining a blind-backbone policy, cross-attention terrain reconstruction from depth plus proprioception, and realistic synthetic depth with noise enables depth-only full-sized humanoid locomotion over stairs, slopes,...
-
Hierarchical Reinforcement Learning and Value Optimization for Challenging Quadruped Locomotion
A hierarchical quadruped controller uses online optimization over the low-level policy's value function to choose footstep targets, improving normalized reward and reducing collisions over an end-to-end baseline witho...
Discussion (0). Sign in to comment.