REVIEW 5 cited by
EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Most reinforcement learning (RL) methods focus on learning optimal policies over low-level action spaces. While these methods can perform well in their training environments, they lack the flexibility to transfer to new tasks. Instead, RL agents that can act over useful, temporally extended skills rather than low-level actions can learn new tasks more easily. Prior work in skill-based RL either requires expert supervision to define useful skills, which is hard to scale, or learns a skill-space from offline data with heuristics that limit the adaptability of the skills, making them difficult to transfer during downstream RL. Our approach, EXTRACT, instead utilizes pre-trained vision language models to extract a discrete set of semantically meaningful skills from offline data, each of which is parameterized by continuous arguments, without human supervision. This skill parameterization allows robots to learn new tasks by only needing to learn when to select a specific skill and how to modify its arguments for the specific task. We demonstrate through experiments in sparse-reward, image-based, robot manipulation environments that EXTRACT can more quickly learn new tasks than prior works, with major gains in sample efficiency and performance over prior skill-based RL. Website at https://www.jessezhang.net/projects/extract/.
Forward citations
Cited by 5 Pith papers
-
TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics
The log-probability a VLM assigns to 'True' for 'does this video prefix complete the task?' is used as a zero-shot dense progress reward that outperforms GVL on open-source models.
-
STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning
Sub-trajectory retrieval with vision foundation models and dynamic time warping improves few-shot robot imitation learning over state-based and full-trajectory retrieval.
-
Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control
Asynchronous dual-frequency hierarchical imitation learning with VQ skills and training-only latent diffusion improves compositional language-conditioned robot control and reduces skill codebook collapse.
-
N2M: Bridging Navigation and Manipulation by Learning Pose Preference from Rollout
N2M predicts preferable base poses for manipulation policies from ego-centric point clouds, learned from rollouts, lifting success from 3% to 54% in the PnPCounterToCab task.
-
SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Training
SLAC learns a latent action space in a low-fidelity simulator and uses it for real-world reinforcement learning, solving whole-body mobile manipulation tasks in under an hour without demonstrations.
Discussion (0). Continue with ORCID to comment.