{"arxiv_id":"2610.08780","title":"DepthWorld: 3D World Model for Robot Manipulation","abstract":"World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning. All of these uses depend on faithful 3D geometry, yet current video-based world models are trained on RGB alone and produce rollouts that look correct frame-by-frame but do not compose into a consistent 3D world. Closing this gap requires progress on two fronts: large-scale 3D supervision for manipulation, and an architecture that can absorb it without disturbing strong pretrained video priors. We introduce a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics. Applied to the DROID dataset, this yields DROID-3D, a calibrated 3D dataset providing dense metric depth and recalibrated multi-view extrinsics (achieving <0.7 px reprojection error on 90% of episodes for external cameras). We then train DepthWorld, a Stable Video Diffusion-based world model that jointly predicts multi-view RGB and depth via spatial latent tiling, leaving the pretrained Variational Autoencoder (VAE) unchanged. Depth supervision improves RGB prediction itself by +1.48 dB PSNR over an identical RGB-only baseline at equal training budget, while simultaneously yielding accurate metric depth for downstream geometric reasoning.","primary_cat":"cs.RO","cross_cats":["cs.AI","cs.CV"],"submitted_at":"2026-10-06T17:59:00+00:00","latest_version":1,"source":"arxiv","source_label":"arXiv","source_url":null,"has_pdf":true,"has_source":false,"verdict":"CONDITIONAL","confidence":"HIGH","one_line_summary":"A calibration pipeline turns DROID into a robot-grounded metric-depth dataset, and joint RGB-depth training of a diffusion world model improves RGB prediction while providing metric depth.","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","upvotes":0,"downvotes":0,"pith_score":0,"inbound_count":1,"_links":{"html":"https://pith.science/paper/2610.08780","verdict":"/api/v1/verdicts/2610.08780"}}