REVIEW 5 cited by
M2T2: Multi-Task Masked Transformer for Object-centric Pick and Place
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
With the advent of large language models and large-scale robotic datasets, there has been tremendous progress in high-level decision-making for object manipulation. These generic models are able to interpret complex tasks using language commands, but they often have difficulties generalizing to out-of-distribution objects due to the inability of low-level action primitives. In contrast, existing task-specific models excel in low-level manipulation of unknown objects, but only work for a single type of action. To bridge this gap, we present M2T2, a single model that supplies different types of low-level actions that work robustly on arbitrary objects in cluttered scenes. M2T2 is a transformer model which reasons about contact points and predicts valid gripper poses for different action modes given a raw point cloud of the scene. Trained on a large-scale synthetic dataset with 128K scenes, M2T2 achieves zero-shot sim2real transfer on the real robot, outperforming the baseline system with state-of-the-art task-specific models by about 19% in overall performance and 37.5% in challenging scenes where the object needs to be re-oriented for collision-free placement. M2T2 also achieves state-of-the-art results on a subset of language conditioned tasks in RLBench. Videos of robot experiments on unseen objects in both real world and simulation are available on our project website https://m2-t2.github.io.
Forward citations
Cited by 5 Pith papers
-
GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation
A vision-language model fine-tuned on 379k synthetic task-grasp pairs predicts task-appropriate grasp points, improving real-world task-oriented grasping over previous methods.
-
COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping
COMBO-Grasp trains a stabilizing constraint policy and an RL grasping policy, then refines the constraint pose with value-function gradients, improving bimanual grasping of occluded objects in simulation and real world.
-
SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation
SAM2Act reports 86.8% average success across 18 RLBench tasks, and the memory variant SAM2Act+ reaches 94.3% on the new MemoryBench tasks.
-
Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation
Lift3D uses task-aware depth reconstruction and mapped 2D positional embeddings to let pretrained 2D vision transformers act as 3D point-cloud manipulation policies, beating prior methods on average.
-
CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordance
Adding a visual and textual chain-of-affordance reasoning step to a vision-language-action model improves robot manipulation success rates and generalization in the paper's evaluations.
Discussion (0). Continue with ORCID to comment.