REVIEW 12 cited by
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In recent years, significant progress has been made in the field of robotic reinforcement learning (RL), enabling methods that handle complex image observations, train in the real world, and incorporate auxiliary data, such as demonstrations and prior experience. However, despite these advances, robotic RL remains hard to use. It is acknowledged among practitioners that the particular implementation details of these algorithms are often just as important (if not more so) for performance as the choice of algorithm. We posit that a significant challenge to widespread adoption of robotic RL, as well as further development of robotic RL methods, is the comparative inaccessibility of such methods. To address this challenge, we developed a carefully implemented library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment, a high-quality controller for a widely-adopted robot, and a number of challenging example tasks. We provide this library as a resource for the community, describe its design choices, and present experimental results. Perhaps surprisingly, we find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation between 25 to 50 minutes of training per policy on average, improving over state-of-the-art results reported for similar tasks in the literature. These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent recovery and correction behaviors. We hope that these promising results and our high-quality open-source implementation will provide a tool for the robotics community to facilitate further developments in robotic RL. Our code, documentation, and videos can be found at https://serl-robot.github.io/
Forward citations
Cited by 12 Pith papers
-
Prediction with Action: Visual Policy Learning via Joint Denoising Process
PAD jointly denoises future images and robot actions in a single diffusion transformer, using video co-training to improve multi-task imitation learning.
-
Adaptation of Generalist Robot Policies with Minimal Data
MiDAS, a two-stage recipe of one-demo behavior cloning plus residual online RL on a frozen VLA backbone, reaches high success from a single demonstration in simulation and improves real-world bimanual manipulation.
-
Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation
MPAIL2 demonstrates real-world manipulation learning from observation alone, without rewards or action labels, plus positive online transfer.
-
Robust Peg-in-Hole Assembly under Uncertainties via Compliant and Interactive Contact-Rich Manipulation
A vision-free, learning-free compliant manipulation system performs peg-in-hole insertion at clearances tighter than the robot's own precision, formalized as composed manipulation funnels that shrink perception and ex...
-
Fabrica: Dual-Arm Assembly of General Multi-Part Objects via Integrated Planning and Learning
A dual-arm robotic system combining hierarchical planning with equivariant residual RL policies demonstrates multi-part assembly of five-to-nine-part objects, with strong step-level but weaker end-to-end real-world success.
-
Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
Warm-start RL (WSRL) fine-tunes offline-pretrained RL agents online with no offline data retention, using 5,000 warm-up rollouts from the frozen pre-trained policy followed by standard high-UTD SAC.
-
SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training
Simulation-pretrained policies, with digital-twin demos for critic bootstrapping and action proposals, cut real-world RL training time while reaching near-perfect success on three manipulation tasks.
-
mimic-one: a Scalable Model Recipe for General Purpose Robot Dexterity
mimic-one reports up to 93.3% out-of-distribution success on three real-world dexterous tasks using a diffusion policy, a custom 16-DoF hand, and a teleoperation data-collection recipe with self-correction trajectories.
-
CrayonRobo: Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation
CrayonRobo trains a vision-language-action model to read colored 2D prompt overlays (contact point, end-effector axes, movement direction) and output SE(3) contact poses, enabling step-by-step and long-horizon robotic...
-
Improving Vision-Language-Action Model with Online Reinforcement Learning
Alternating online RL on a frozen vision-language backbone with supervised fine-tuning on collected successes improves a VLA policy's task success and generalization.
-
Simulation-Aided Policy Tuning for Black-Box Robot Learning
An adaptive Bayesian policy-search method (HCI-GIBO/S-HCI-GIBO) uses simulator data to reduce real-robot queries while aiming for high-confidence policy improvements.
-
Integrating Model-based Control and RL for Sim2Real Transfer of Tight Insertion Policies
A hybrid policy combining a potential-field controller with residual reinforcement learning, trained only in simulation, achieves high zero-shot success on sub-millimeter insertion tasks in the real world.
Discussion (0). Continue with ORCID to comment.