REVIEW 5 cited by
VIP: Vision Instructed Pre-training for Robotic Manipulation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The effectiveness of scaling up training data in robotic manipulation is still limited. A primary challenge in manipulation is the tasks are diverse, and the trained policy would be confused if the task targets are not specified clearly. Existing works primarily rely on text instruction to describe targets. However, we reveal that current robotic data cannot train policies to understand text instruction effectively, and vision is much more comprehensible. Therefore, we introduce utilizing vision instruction to specify targets. A straightforward implementation is training a policy to predict the intermediate actions linking the current observation and a future image. Nevertheless, a single future image does not describe the task target in insufficient detail. To handle this problem, we propose to use sparse point flows to provide more detailed information. Extensive tasks are designed based on real and simulated environments to evaluate the effectiveness of our vision instructed pre-training (VIP) method. The results indicate VIP improves the performance on diverse tasks significantly, and the derived policy can complete competitive tasks like ``opening the lid of a tightly sealed bottle''.
Forward citations
Cited by 5 Pith papers
-
LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation
A 0.5B parameter latent world-action model trains end-to-end on a single GPU and reaches 90.48% average success on 50 RoboTwin 2.0 tasks with a language-free Visual Transition Token for task specification.
-
SAM3D-Guided Object-Centric Representation Alignment for Vision-Language-Action Models
Training-time alignment of π0 visual features with object-centric SAM3D 3D features improves VLA manipulation performance while keeping RGB-language-only inference.
-
Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation
Lift3D-VLA integrates 3D point cloud encoding and temporal action modeling into Vision-Language-Action models, achieving higher success rates on simulated and real-world robotic manipulation tasks.
-
Choose What to Manipulate: Revealing Data Scaling Laws in Bounding-Box Guided Policies for Semantic Manipulation
A bounding-box-conditioned diffusion policy shows a power-law improvement with the number of object classes in training data, reaching about 85% success on four semantic manipulation tasks.
-
Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space
Breaking long manipulation tasks into atomic subtasks and collecting demonstrations from varied starting poses improves imitation learning success using fewer demonstration frames.
Discussion (0). Continue with ORCID to comment.