REVIEW 19 cited by
SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulation
read the original abstract
Real-world tasks such as garment manipulation and table rearrangement demand robots to perform generalizable, highly precise, and long-horizon actions. Although imitation learning has proven to be an effective approach for teaching robots new skills, large amounts of expert demonstration data are still indispensible for these complex tasks, resulting in high sample complexity and costly data collection. To address this, we propose Semantic Keypoint Imitation Learning (SKIL), a framework which automatically obtains semantic keypoints with the help of vision foundation models, and forms the descriptor of semantic keypoints that enables efficient imitation learning of complex robotic tasks with significantly lower sample complexity. In real-world experiments, SKIL doubles the performance of baseline methods in tasks such as picking a cup or mouse, while demonstrating exceptional robustness to variations in objects, environmental changes, and distractors. For long-horizon tasks like hanging a towel on a rack where previous methods fail completely, SKIL achieves a mean success rate of 70\% with as few as 30 demonstrations. Furthermore, SKIL naturally supports cross-embodiment learning due to its semantic keypoints abstraction. Our experiments demonstrate that even human videos bring considerable improvement to the learning performance. All these results demonstrate the great success of SKIL in achieving data-efficient generalizable robotic learning. Visualizations and code are available at: https://skil-robotics.github.io/SKIL-robotics/.
Forward citations
Cited by 19 Pith papers
-
RoBoSR: Structured Scene Representations for Embodied Robotic Reasoning
RoBoSR uses structured object-centric scene graphs as an intermediate representation to enable causal reasoning and subtask planning in embodied robotics, outperforming baselines on benchmarks and real demos.
-
MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models
MaskWAM unifies mask prompting and prediction in world-action models via Mixture of Transformers to improve robotic policy generalization on language-ambiguous tasks.
-
From a Single Demonstration to a General Policy for Contact-Rich Manipulation
A one-shot LfD framework abstracts a single demonstration into environmental-constraint primitives, then uses self-exploration, human corrections, and compliant recovery to produce a policy that generalizes across pos...
-
GAP: Geometric Anchor Pre-training for Data-Efficient Visuomotor Learning of Manipulation Tasks
GAP pre-trains the spatial adapter on a lightweight simulated proxy task with free object masks to generate repeatable geometric keypoints, yielding higher success rates than baselines in low-data robotic manipulation...
-
SID: Sliding into Distribution for Robust Few-Demonstration Manipulation
SID achieves approximately 90% success on six real-world manipulation tasks with only two demonstrations under out-of-distribution initializations, with less than 10% performance drop under distractors and disturbances.
-
HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactions
HeteroGenManip decouples grasp localization from interaction planning using task-conditioned foundation models and multi-model diffusion policies, delivering 31% average gains in broad simulation tasks and 36.7% in fo...
-
HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactions
A task-conditioned two-stage system decouples grasp localization from interaction trajectory planning using specialized foundation models to improve generalization across heterogeneous object types.
-
A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics
A two-level hierarchical vector quantization tokenizer that clusters actions spatially and temporally achieves new state-of-the-art results in in-context imitation learning for robotics.
-
From Video to Control: A Survey of Learning Manipulation Interfaces from Temporal Visual Data
Video-to-robot control methods cluster into three interface families, and the field’s main bottleneck is grounding video-derived predictions into dependable closed-loop robot behavior.
-
On the Generalization Capabilities, Design Choices and Limitations of Keypoint Imitation Learning
KIL using foundation model keypoints reaches 75% success on five manipulation tasks, beating RGB (47%) but matching S2-diffusion (73%), with generalization tests on unseen objects via over 2000 real-world rollouts.
-
FocalPolicy: Frequency-Optimized Chunking and Locally Anchored Flow Matching for Coherent Visuomotor Policy
FocalPolicy introduces frequency-optimized chunking and locally anchored flow matching with a foresight composite objective to improve inter-chunk coherence in visuomotor policies for manipulation tasks.
-
FocalPolicy: Frequency-Optimized Chunking and Locally Anchored Flow Matching for Coherent Visuomotor Policy
FocalPolicy introduces frequency-optimized chunking and locally anchored flow matching with a foresight composite objective to reduce inter-chunk discontinuities in visuomotor policies.
-
A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics
A two-level hierarchical vector-quantization tokenizer that recovers actions and timestamps sets a claimed SOTA for in-context robot imitation learning.
-
A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics
A two-level vector quantization tokenizer that jointly reconstructs robot actions and timestamps to improve in-context imitation learning performance on manipulation benchmarks.
-
From Video to Control: A Survey of Learning Manipulation Interfaces from Temporal Visual Data
A survey introduces an interface-centric taxonomy for video-to-control methods in robotic manipulation and identifies the robotics integration layer as the central open challenge.
-
From World Models to World Action Models: A Concise Tutorial for Robotics
World models are action-conditioned predictors of task-relevant futures; world action models couple those futures to robot actions via four paradigms: imagine-then-execute, feature-conditioned, joint, and auxiliary pr...
-
From World Models to World Action Models: A Concise Tutorial for Robotics
A tutorial that categorizes world models into observation-space and state-space types and outlines four paradigms for world action models connecting predictions to robot actions.
-
From World Models to World Action Models: A Concise Tutorial for Robotics
A tutorial defining world models and world action models for robotics, with design axes and a four-paradigm taxonomy of prediction-action coupling.
-
From World Models to World Action Models: A Concise Tutorial for Robotics
A tutorial taxonomizes world models for robotics into observation-space and state-space types and introduces world action models via four paradigms linking predictions to executable actions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.