Pith. sign in

REVIEW 5 cited by

3D-MVP: 3D Multiview Pretraining for Robotic Manipulation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.18158 v2 pith:2BSMLSOV submitted 2024-06-26 cs.RO cs.CV

3D-MVP: 3D Multiview Pretraining for Robotic Manipulation

Shengyi Qian , Kaichun Mo , Valts Blukis , David F. Fouhey , Dieter Fox , Ankit Goyal This is my paper
classification cs.RO cs.CV
keywords pretrainingd-mvpmanipulationmaskedmulti-viewrobotictransformervisual
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Recent works have shown that visual pretraining on egocentric datasets using masked autoencoders (MAE) can improve generalization for downstream robotics tasks. However, these approaches pretrain only on 2D images, while many robotics applications require 3D scene understanding. In this work, we propose 3D-MVP, a novel approach for 3D Multi-View Pretraining using masked autoencoders. We leverage Robotic View Transformer (RVT), which uses a multi-view transformer to understand the 3D scene and predict gripper pose actions. We split RVT's multi-view transformer into visual encoder and action decoder, and pretrain its visual encoder using masked autoencoding on large-scale 3D datasets such as Objaverse. We evaluate 3D-MVP on a suite of virtual robot manipulation tasks and demonstrate improved performance over baselines. Our results suggest that 3D-aware pretraining is a promising approach to improve generalization of vision-based robotic manipulation policies. Project site: https://jasonqsy.github.io/3DMVP

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Learning 3D Reconstruction with Priors in Test Time

    cs.CV 2026-04 unverdicted novelty 7.0

    Test-time constrained optimization incorporates priors into pre-trained multiview transformers via self-supervised losses and penalty terms to improve 3D reconstruction accuracy.

  2. Self-supervised Learning Of Visual Pose Estimation Without Pose Labels By Classifying LED States

    cs.RO 2025-09 conditional novelty 7.0

    A self-supervised training scheme where a network estimates a robot's relative pose by predicting the on/off states of its LEDs, needing no pose labels or CAD model.

  3. ProGAL-VLA: Grounded Alignment through Prospective Reasoning in Vision-Language-Action Models

    cs.RO 2026-04 unverdicted novelty 6.0

    ProGAL-VLA uses 3D graphs, symbolic sub-goals, and a Grounding Alignment Contrastive loss to ground actions on verified embeddings, raising robustness from 30.3% to 71.5% and ambiguity AUROC to 0.81 on robotic benchmarks.

  4. Fake or Real, Can Robots Tell? Evaluating VLM Robustness to Domain Shift in Single-View Robotic Scene Understanding

    cs.RO 2025-06 unverdicted novelty 6.0

    VLMs caption real objects effectively but degrade on 3D-printed fakes in robotic scenes, while some standard metrics fail to detect the factual errors from this domain shift.

  5. R3D: Revisiting 3D Policy Learning

    cs.CV 2026-04 unverdicted novelty 5.0

    A transformer 3D encoder plus diffusion decoder architecture, with 3D-specific augmentations, outperforms prior 3D policy methods on manipulation benchmarks by improving training stability.