REVIEW 9 cited by
RVT: Robotic View Transformer for 3D Object Manipulation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
For 3D object manipulation, methods that build an explicit 3D representation perform better than those relying only on camera images. But using explicit 3D representations like voxels comes at large computing cost, adversely affecting scalability. In this work, we propose RVT, a multi-view transformer for 3D manipulation that is both scalable and accurate. Some key features of RVT are an attention mechanism to aggregate information across views and re-rendering of the camera input from virtual views around the robot workspace. In simulations, we find that a single RVT model works well across 18 RLBench tasks with 249 task variations, achieving 26% higher relative success than the existing state-of-the-art method (PerAct). It also trains 36X faster than PerAct for achieving the same performance and achieves 2.3X the inference speed of PerAct. Further, RVT can perform a variety of manipulation tasks in the real world with just a few ($\sim$10) demonstrations per task. Visual results, code, and trained model are provided at https://robotic-view-transformer.github.io/.
Forward citations
Cited by 9 Pith papers
-
ARGUS: Aligning Robot Scene Geometry Under Shifting Views with Large 3D Vision Models
A preprocessing pipeline that reconstructs a 3D point cloud from RGB images and re-renders it from a fixed viewpoint improves viewpoint robustness and data efficiency for vision-based robot policies.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
A modular benchmark of 100 dexterous manipulation tasks across 3 arms and 6 hands with 3,180 demonstrations reveals that current policies (Diffusion Policy, DP3, OpenVLA, π0.5) achieve only 34% mean success, exposing ...
-
Geometry-Aware Motion Latents for Learning Robust Manipulation Policies
Predicting future 3D pointmaps forces discrete motion latents to encode physical geometric transformations, improving single-view robot manipulation over 2D/static-3D baselines.
-
ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model
A hierarchical leader-follower Gaussian world model improves multi-task bimanual manipulation success rates over prior single-arm-based methods.
-
CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity
CodeDiffuser uses vision-language-model-generated code to build 3D attention maps that condition a diffusion policy, improving success on ambiguous language manipulation tasks compared with end-to-end baselines.
-
MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation
MALMM, a three-agent LLM framework with per-step environment feedback, achieves 81% average success on nine RLBench tasks versus 50% for a single-agent LLM baseline, in zero-shot settings.
-
QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Models
Adding an auxiliary quantized-depth-token prediction task to a VLA policy improves manipulation success rates on LIBERO, Simpler, and real-robot pick-and-place tasks versus the open-pi-zero baseline.
-
Advances in Transformers for Robotic Applications: A Review
A survey of Transformer applications in robotic perception, planning, control, human-robot interaction, and reinforcement learning.
Discussion (0). Continue with ORCID to comment.