Robotics · arXiv:2610.08780 v1 · submitted 6 Oct 2026 · 51 references
DepthWorld: 3D World Model for Robot Manipulation
The paper says
“World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning.”
more of the abstract
All of these uses depend on faithful 3D geometry, yet current video-based world models are trained on RGB alone and produce rollouts that look correct frame-by-frame but do not compose into a consistent 3D world. Closing this gap requires progress on two fronts: large-scale 3D supervision for manipulation, and an architecture that can absorb it without disturbing strong pretrained video priors. We introduce a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics. Applied to the DROID dataset, this yields DROID-3D, a calibrated 3D dataset providing dense metric depth and recalibrated multi-view extrinsics (achieving <0.7 px reprojection error on 90% of episodes for external cameras). We then train DepthWorld, a Stable Video Diffusion-based world model that jointly predicts multi-view RGB and depth via spatial latent tiling, leaving the pretrained Variational Autoencoder (VAE) unchanged. Depth supervision improves RGB prediction itself by +1.48 dB PSNR over an identical RGB-only baseline at equal training budget, while simultaneously yielding accurate metric depth for downstream geometric reasoning.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 7 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. The calibration contribution is real and well-evidenced, but the headline RGB gain is confounded: RGB+D runs twice the latent tokens per step, and the paper never runs the token-matched control that would clean the attribution. This line is wrong
4 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 51 references say.
The paper says, at §4.1 and Appendix D.3 (Table 2, sentence b157)
The review did not quote this passage.
Pith says: Objection 1 of 4 · major
The headline +1.48 dB PSNR gain is confounded by model capacity. DepthWorld encodes six tiles (RGB and depth for each of three views) into a 72×80 joint latent, while the RGB-only baseline uses three RGB tiles in a 72×40 grid; this doubles the spatial token count and, with attention in the U-Net, roughly doubles per-step compute.
more of objection 1
Appendix D.3 calls the baseline a controlled comparison with the 'same set, split, architecture, and training budget,' but the training budget is never defined as steps versus FLOPs, and the two models do not receive the same compute per step at equal step count. No control replaces the three depth tiles with noise, a constant, or duplicated RGB while keeping the token grid at 72×80. Until such a control is run, the +1.48 dB gain cannot be attributed to depth supervision rather than to the additional learned capacity, and the Limitations paragraph (b153) does not acknowledge this confound.
The paper says, at Appendix C.8 and Appendix A.1 (Table 2, sentence b343)
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.74):
“The depth metrics are not: each model is scored against the depth source it was trained on, so they mea- sure how faithfully a model reproduces its own supervision rather than its absolute metric accuracy. 11 Table A2:Ablation: Spatial latent tiling vs. other integration architectures, at∼70k training steps under the autoregressive rollout protocol (Appendix D.3).”
Pith says: Objection 2 of 4 · major
The metric-depth claim is only sparsely grounded and is otherwise self-referential. All DROID-3D depth, for both training and evaluation, comes from the learned S2M2 disparity network; the depth metrics in Table 2 and Appendix C are computed against the VAE round-trip of the same S2M2 depth, so they measure self-reproduction of the training signal, exactly as the authors note in b343.
more of objection 2
Independent metric grounding is provided for five in-lab ChArUco capture sets and 50 REALM scenes, but 71,100 episodes remain validated only against S2M2 itself. The conclusion that DepthWorld produces 'accurate metric depth for downstream geometric reasoning' therefore holds only under the untested assumption that S2M2 depth is metrically unbiased on the unvalidated DROID scenes. I would like to see either a larger independent depth-validation set or a clearer separation between the calibrated-grounding claim and the S2M2-reproduction claim in the abstract and Section 5.
The paper says, at §5 and Appendix D.4 (Table A5)
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.83):
“Second, DepthWorld predicts depth in the latent space of the SVD V AE, whose encode–decode round trip alone has an error floor that PointWorld’s point representation does not have; how we account for this is described below.”
Pith says: Objection 3 of 4 · major
The comparison with PointWorld is qualified in the text but the table presentation makes it easy to over-read. In Table A5, PointWorld is scored against raw ground-truth depth while DepthWorld is scored against VAE-round-tripped ground truth, and Appendix D.4 notes that the VAE round trip alone displaces the ground truth by about 45 mm on moving points.
more of objection 3
This 'own-ceiling' protocol is disclosed and argued carefully, but the headline numbers (e.g., whole-scene Chamfer 8→9 mm for DepthWorld vs 8→18 mm for PointWorld) mix two different targets. Since the off-diagonal comparisons are said to reverse the winner, the summary in Section 5 should state explicitly that the comparative ranking is conditional on each model being scored above its own representational floor, and the main text should not present Table A5 as a direct accuracy comparison.
The paper says, at §5, Table 2
The review did not quote this passage.
Pith says: Objection 4 of 4 · major
Several reported differences between RGB+D and RGB+D PM-DPT are very small (24.09 vs 24.11 dB PSNR; external AbsRel 0.0765 vs 0.0782), and no variance across seeds or confidence intervals is reported anywhere. Given that the PM-DPT contribution is advertised as a refinement, single-run differences of this size are not interpretable. Reporting at least two seeds or bootstrap intervals for the main Table 2 metrics would make the qualitative claims actionable.
5 minor objections
The paper says, at §1, Introduction
Pith says: minor objection 1 of 5
There is a duplicated word in 'kinematically decoupled from from the robot’s own coordinate frame'; please remove the second 'from.'
The paper says, at Figure 1 caption
Pith says: minor objection 2 of 5
The caption says DepthWorld 'capitalizes on the depth inputs,' but the inference protocol described in Appendix C.7 initializes the history buffer from the first frame and the model is not given ground-truth depth at rollout. Please clarify whether depth is an input at inference or only a training target, and adjust the caption wording accordingly.
show 3 more
The paper says, at Table 2 caption and §5
Pith says: minor objection 3 of 5
The phrase 'specially on wrist metrics' should be 'especially on wrist metrics.' Also, the depth-quality columns in Table 2 are evaluated against the VAE round-trip of the S2M2 depth; this caveat appears in Appendix C.8 but should be repeated in the Table 2 caption or Section 5 protocol for readers who do not read the appendix.
The paper says, at Appendix D.4 and D.5
Pith says: minor objection 4 of 5
The text uses unresolved placeholder citations for CoTracker3, DBSCAN, and CogVideoX-5b-I2V ('[?]'). These need to be replaced with proper references before publication.
The paper says, at Appendix C.2
Pith says: minor objection 5 of 5
The statement 'Spatial tiling performs better: it improves every RGB metric on both views' refers to Table A2, but that table compares joint RGB-depth architectures and does not include a token-matched RGB-only baseline; please add a sentence making this limitation explicit in the appendix as well as in the main text.
What would settle it
Retrain an RGB-only control on the 72×80 grid, filling the three depth-tile slots with (i) per-step i.i.d. Gaussian noise, (ii) a constant gray image, and (iii) a duplicate of the same view's RGB tile, keeping step count, batch size, and optimizer identical to RGB+D. Report external-view PSNR on the same 256 held-out trajectories and FLOPs/step for each variant. If any no-depth control moves PSNR materially from 22.63 toward 24.09, the headline is a capacity/token-count effect; only if all remain near 22.63 can 'depth supervision improves RGB' be accepted.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 7 Oct 2026
- Reader holds up, with conditions
- Skeptic disagrees
- Editor wrote the verdict above
- Referee major revision: 4 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The real product here is the calibration pipeline, and it's good. Pooling all episodes of one physical robot into a single factor graph to separate shared kinematic parameters (hand-eye, joint offsets) from per-scene extrinsics is a genuine scaling insight: the Schur-complement reduction to a 13x13 global solve is clean, and the results back it up, 4.9 mm / 0.39 deg against ChArUco ground truth versus 72 mm / 3.0 deg for the PointWorld-style per-scene baseline. The calibration table (23x EE, 13x WE, 8x RD, +37 Mask IoU) is convincing, and the own-ceiling protocol in the PointWorld comparison is honest, careful evaluation work. DROID-3D, if released, is a real asset for the subfield.
The headline RGB claim is where I part company with the framing. The stress-test note is right: RGB-only runs three latent tiles per timestep, RGB+D runs six. Same nominal budget is not the same compute — attention scales with token count — and the paper never defines 'budget' as steps or FLOPs, and never runs the control that would settle it: an RGB-only model with the depth tiles replaced by noise, a constant, or duplicated RGB. The +1.48 dB is a real measured difference, but attributing it to depth supervision rather than extra capacity is untested, and the Limitations section does not mention the confound. One nuance in the paper's favor: the depth-source ablation (Table A3) shows bad depth actively hurts RGB (-2 dB), so the depth signal is doing something. But the S2M2-vs-ZED-NEURAL row, same token count, near-identical RGB at 24.07 vs 24.26 dB, is equally consistent with the gain being mostly tokens. Needs a matched-token control before I'd trust the causal story.
Other soft spots, in proportion. No code or data shipped, and DROID-3D itself unreleased, so nothing is independently reproducible. All depth metrics are scored against the S2M2 depth used in training — disclosed in C.8 — but the 'accurate metric depth at 71,000 episodes' claim rests on only five ChArUco sets and fifty REALM scenes of external grounding. The unresolved [?] citations for CoTracker3, CogVideoX-5b-I2V, and DBSCAN in Appendices D.4-D.5 break replication of the baseline protocols. The dual-branch comparison is admitted to be not 1:1.
Verdict: send to a serious referee, yes. The calibration contribution stands on its own even if the RGB-attribution story collapses. I would ask for a token-matched RGB-only control and code or data before accepting.
The strongest objection
The central empirical claim (b157) is the +1.48 dB PSNR gain of RGB+D over an 'identical' RGB-only baseline, but the comparison is not confound-free. Section 4.1 describes RGB+D as a joint latent of 72×80 made of six tiles (RGB and depth for each of three views); the RGB-only baseline has only three RGB tiles, so the latent grid is half as wide in tokens and the U-Net's per-step compute (notably attention) is correspondingly smaller. Appendix D.3 calls the baseline retrained 'on the same set, split, architecture, and training budget', but 'budget' is never defined as steps vs FLOPs; if it means steps, the depth model receives substantially more compute per step. No ablation equalizes token count while withholding depth information: Table A2 compares joint RGB-depth architectures (dual-branch, channel expansion, tiling), not an RGB-only model with the depth tiles replaced by noise, a constant, or duplicated RGB. The claimed attribution of the RGB gain to depth supervision therefore rests on an untested assumption. The paper's Limitations (b153) do not mention this confound.
I read the paper's center of gravity as the claim that depth supervision improves RGB prediction while yielding depth, because that is what b157 calls 'the central empirical finding' and what the architecture section is built around. The reader's weakest assumption (S2M2 self-reproduction) is real and is explicitly acknowledged at b343 and Appendix C.8; it constrains the absolute-depth claim but does not touch the RGB gain. The token-count confound is different: it directly threatens the controlled RGB comparison and is not acknowledged anywhere. The paper has genuine independent support—ChArUco and REALM calibration validation, a retrained equal-split baseline, and a plausible architectural story via Marigold—so I would not reject the paper. But the headline needs one additional control before the attribution is proven. I therefore keep the verdict CONDITIONAL, with the added condition that the authors supply a token-matched, no-depth baseline. My concern is not about novelty or style; it is about whether the single controlled experiment in the paper actually isolates the variable it names.
The referee's report
If the claims hold, the paper is a substantial practical contribution: it upgrades a widely used real-robot manipulation dataset with dense metric depth and robot-grounded multi-view calibration, and it shows a way to inject 3D supervision into a pretrained video-diffusion world model without destructive architectural modification. The ChArUco/REALM validation of the calibration pipeline (4.9 mm / 0.39 deg median extrinsics error) is a strong piece of evidence, and the paper is unusually candid about caveats such as the VAE round-trip evaluation floor and the held-out-trajectory (not cross-embodiment) scope. The main risk is that the headline RGB improvement is not yet attributable to depth supervision because the RGB+D model also has twice the latent token count of the RGB-only baseline; this needs a token-matched control before the central claim can be accepted at face value.
Pith's reading of the paper (a machine's summary, not the authors')
The paper tries to establish that a video world model for robot manipulation can produce rollouts that compose into a single coherent 3D scene, not just visually plausible frames. To get there it builds two things: DROID-3D, a recalibration of the DROID dataset that supplies dense metric depth and robot-grounded camera poses for over 70,000 episodes, and DepthWorld, a video-diffusion world model that predicts multi-view RGB and depth together. The central empirical claim is that adding depth as a joint prediction target improves RGB prediction itself by +1.48 dB PSNR at equal training budget, so geometric supervision is not a cost to appearance prediction but a benefit. If correct, this opens a route to world models whose rollouts can be read off for policy evaluation, improvement, and planning.
What carries the argument. Spatial latent tiling is the architectural device that carries the argument: each view's RGB and depth are encoded as separate images by the untouched SVD VAE and placed side-by-side in a 72x80 latent grid, so the single pretrained U-Net denoises both modalities jointly with only the spatial position embeddings extended. This is what allows depth supervision to be added without destructive re-initialization of the pretrained video backbone. The companion mechanism is the joint factor graph, an SE(3)-parameterized robust loss over all episodes of a robot whose Schur-complement solve reduces to a 13x13 global problem per iteration, cleanly separating per-scene camera corrections from shared kinematic parameters. A DPT point-map head initialized from VGGT lifts the predicted RGB-depth latents into robot base coordinates and supervises cross-view agreement directly in 3D.
Core claim. The core claim is that 3D-consistent world-model rollouts are attainable without sacrificing the pretrained video prior. On the data side, the paper shows that a multi-view stereo teleoperation collection with a known robot kinematic model can be lifted into supervision-quality 3D: S2M2 learned stereo disparity supplies per-view metric depth, and a joint factor graph pooling all episodes of one physical robot separates per-scene extrinsic noise from shared kinematic errors (hand-eye correction and joint offset), grounding the rig in the robot's base frame. On the modeling side, DepthWorld encodes each view's RGB and depth independently with the unmodified SVD variational autoencoder and tiles them side-by-side in a wider latent grid, so a single pretrained U-Net denoises both modalities together. The paper reports that this joint target beats an identical RGB-only baseline by +1.48 dB PSNR on external views and +1.02 dB on wrist views while producing metric depth, with a lightweight DPT point-map head supervised in the robot base frame further sharpening wrist-view depth.
The premise it rests on. Every depth number in the paper — the calibration targets that define DROID-3D and the ground truth that all DepthWorld depth accuracy is scored against — comes from the learned S2M2 stereo disparity, which is checked against real metric ground truth on only five in-lab ChArUco capture sets and 50 REALM scenes out of 71,100 total.
If the paper is right (Pith's inferences, not the authors')
World models for manipulation can be trained so their rollouts carry internally consistent geometry, making occlusion and contact reasoning possible where RGB-only models fail.
Supervising a second modality need not trade off against the first: depth supervision improved the RGB signal, so geometric training targets can act as a free gain.
The calibration approach transfers to any multi-view stereo teleoperation corpus with a known URDF, so other large manipulation datasets could be lifted into 3D without new capture hardware.
Because tiling exploits the VAE's latent capacity rather than re-initializing weights, the pattern could transfer to other pretrained video diffusion backbones.
The recalibrated extrinsics have a measurable downstream effect: point-map supervision trained against DROID-3D poses rather than factory poses improves wrist-view depth accuracy (AbsRel 0.233 to 0.220).
The paper reports that the calibration is far more robust than per-scene refinement, with no analogous per-scene failure mode to the baseline's roughly 18% non-convergence.
The +1.48 dB RGB gain is scored against real RGB pixels, so it would survive even if S2M2 depth were metrically biased; it requires only that depth is an information-bearing auxiliary signal. A cleaner control than the paper runs would corrupt or permute the depth targets and check the gain disappears.
If tiling works for depth, it likely generalizes to any modality the pretrained VAE can encode — normals, segmentation, optical flow — giving a generic recipe for joint-modality world models.
The tracked-point and Chamfer comparisons with PointWorld hint that depth-image world models drift less on static scenes than point-cloud world models; whether that holds over longer horizons and harder scenes is left open.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning. All of these uses depend on faithful 3D geometry, yet current video-based world models are trained on RGB alone and produce rollouts that look correct frame-by-frame but do not compose into a consistent 3D world. Closing this gap requires progress on two fronts: large-scale 3D supervision for manipulation, and an architecture that can absorb it without disturbing strong pretrained video priors. We introduce a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics. Applied to the DROID dataset, this yields DROID-3D, a calibrated 3D dataset providing dense metric depth and recalibrated multi-view extrinsics (achieving <0.7 px reprojection error on 90% of episodes for external cameras). We then train DepthWorld, a Stable Video Diffusion-based world model that jointly predicts multi-view RGB and depth via spatial latent tiling, leaving the pretrained Variational Autoencoder (VAE) unchanged. Depth supervision improves RGB prediction itself by +1.48 dB PSNR over an identical RGB-only baseline at equal training budget, while simultaneously yielding accurate metric depth for downstream geometric reasoning.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-07
The authors' main claim, as the classifier picked it out 1.00
“Depth supervision improves RGB prediction itself by +1.48 dB PSNR over an identical RGB-only baseline at equal training budget, while simultaneously yielding accurate metric depth for downstream geometric reasoning.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.16 · human participants 0.05 · controlled comparison 0.94 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.93 · mentions code or data availability 0.05
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning. | background 1.00 | 0.05 | 0.04 | asserted 0.99 |
| All of these uses depend on faithful 3D geometry, yet current video-based world models are trained on RGB alone and produce rollouts that look correct frame-by-frame but do not compose into a consistent 3D world. | gap 0.93 | 0.11 | 0.06 | asserted 1.00 |
| Closing this gap requires progress on two fronts: large-scale 3D supervision for manipulation, and an architecture that can absorb it without disturbing strong pretrained video priors. | gap 0.90 | 0.07 | 0.07 | asserted 0.98 |
| We introduce a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics. | method 0.89 | 0.96 | 0.05 | asserted 1.00 |
| Applied to the DROID dataset, this yields DROID-3D, a calibrated 3D dataset providing dense metric depth and recalibrated multi-view extrinsics (achieving <0.7 px reprojection error on 90% of episodes for external cameras). | result 0.69 | 0.96 | 0.08 | asserted 1.00 |
| We then train DepthWorld, a Stable Video Diffusion-based world model that jointly predicts multi-view RGB and depth via spatial latent tiling, leaving the pretrained Variational Autoencoder (VAE) unchanged. | method 0.79 | 0.95 | 0.12 | asserted 1.00 |
| Depth supervision improves RGB prediction itself by +1.48 dB PSNR over an identical RGB-only baseline at equal training budget, while simultaneously yielding accurate metric depth for downstream geometric reasoning. | result 1.00 | 0.95 | 0.50 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.96
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-07: 1 request, 5,821 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (4)
Figures are the authors', reproduced from the paper.
References (51)
- [1] Y . Guo, L. X. Shi, J. Chen, and C. Finn. Ctrl-world: A controllable generative world model for robot manipulation.arXiv preprint arXiv:2510.10125, 2025 on PithPith's reviewarXiv
- [2] J. Bardhan, P. Drozdik, J. Sivic, and V . Petrik. Persistent robot world models: Stabilizing multi-step rollouts via reinforcement learning.arXiv preprint arXiv:2603.25685, 2026 on PithPith's reviewarXiv
- [3] Y . Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. Tenenbaum, D. Schuurmans, and P. Abbeel. Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023 on Pith
- [4] H. Zhen, Q. Sun, H. Zhang, J. Li, S. Zhou, Y . Du, and C. Gan. Tesseract: learning 4d embodied world models.arXiv preprint arXiv:2504.20995, 2025 on PithPith's reviewarXiv
- [5] L. Russell, A. Hu, L. Bertoni, G. Fedoseev, J. Shotton, E. Arani, and G. Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving.arXiv preprint arXiv:2503.20523, 2025 on PithPith's reviewarXiv
- [6] S. Wang, V . Leroy, Y . Cabon, B. Chidlovskii, and J. Revaud. Dust3r: Geometric 3d vision made easy. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20697–20709, 2024 on Pith
- [7] V . Leroy, Y . Cabon, and J. Revaud. Grounding image matching in 3d with mast3r. InEuropean conference on computer vision, pages 71–91. Springer, 2024 on Pith
- [8] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. Vggt: Visual ge- ometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025 on Pith
- [9] Y . Wang, J. Zhou, H. Zhu, W. Chang, Y . Zhou, Z. Li, J. Chen, J. Pang, C. Shen, and T. He. pi3: Permutation-equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025 on PithPith's reviewarXiv
- [10] H. Lin, S. Chen, J. Liew, D. Y . Chen, Z. Li, G. Shi, J. Feng, and B. Kang. Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647, 2025 on PithPith's reviewarXiv
- [11] N. Keetha, N. M ¨uller, J. Sch¨onberger, L. Porzi, Y . Zhang, T. Fischer, A. Knapitsch, D. Zauss, E. Weber, N. Antunes, et al. Mapanything: Universal feed-forward metric 3d reconstruction. arXiv preprint arXiv:2509.13414, 2025 on PithPith's reviewarXiv
- [12] C. Yeshwanth, Y .-C. Liu, M. Nießner, and A. Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023 on Pith
- [13] G. Baruch, Z. Chen, A. Dehghan, T. Dimry, Y . Feigin, P. Fu, T. Gebauer, B. Joffe, D. Kurz, A. Schwartz, et al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data.arXiv preprint arXiv:2111.08897, 2021. 10 on PithPith's reviewarXiv
- [14] J. Reizenstein, R. Shapovalov, P. Henzler, L. Sbordone, P. Labatut, and D. Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10901– 10911, 2021 on Pith
- [15] Z. Li and N. Snavely. Megadepth: Learning single-view depth prediction from internet photos. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2041–2050, 2018 on Pith
- [16] Y . Yao, Z. Luo, S. Li, J. Zhang, Y . Ren, L. Zhou, T. Fang, and L. Quan. Blendedmvs: A large- scale dataset for generalized multi-view stereo networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1790–1799, 2020 on Pith
- [17] S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. Turner, E. Un- dersander, W. Galuba, A. Westbury, A. X. Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai.arXiv preprint arXiv:2109.08238, 2021 on PithPith's reviewarXiv
- [18] A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024 on PithPith's reviewarXiv
- [19] F. Zhu, H. Wu, S. Guo, Y . Liu, C. Cheang, and T. Kong. Irasim: A fine-grained world model for robot manipulation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9834–9844, 2025 on Pith
- [20] F. Jiang, Y . Chen, K. Xu, Y . Liu, H. Wang, Z. Shen, J. Lu, S. Huang, Y . Wang, C. Xie, et al. Robowm-bench: A benchmark for evaluating world models in robotic manipulation.arXiv preprint arXiv:2604.19092, 2026 on PithPith's reviewarXiv
show 31 more
- [21] R. Ranftl, A. Bochkovskiy, and V . Koltun. Vision transformers for dense prediction. InPro- ceedings of the IEEE/CVF international conference on computer vision, pages 12179–12188, 2021 on Pith
- [22] T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely. Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018 on PithPith's reviewarXiv
- [23] A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y . Levi, Z. En- glish, V . V oleti, A. Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023 on PithPith's reviewarXiv
- [24] B. Ke, A. Obukhov, S. Huang, N. Metzger, R. C. Daudt, and K. Schindler. Repurposing diffusion-based image generators for monocular depth estimation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9492–9502, 2024 on Pith
- [25] W. Hu, X. Gao, X. Li, S. Zhao, X. Cun, Y . Zhang, L. Quan, and Y . Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2005–2015, 2025 on Pith
- [26] C. Zhang, Z. Wu, G. Lu, Y . Tang, and Z. Wang. imowm: Taming interactive multi-modal world model for robotic manipulation.arXiv preprint arXiv:2510.09036, 2025 on PithPith's reviewarXiv
- [27] E. Pallotta, S. M. Azar, S. Li, O. Zatsarynna, and J. Gall. Syncvp: joint diffusion for syn- chronous multi-modal video prediction. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 13787–13797, 2025 on Pith
- [28] H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V . My- ers, M. J. Kim, M. Du, et al. Bridgedata v2: A dataset for robot learning at scale. InConference on Robot Learning, pages 1723–1736. PMLR, 2023. 11 on Pith
- [29] A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022 on PithPith's reviewarXiv
- [30] A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE International Conference on Robotics and Automati... on Pith
- [31] H.-S. Fang, H. Fang, Z. Tang, J. Liu, C. Wang, J. Wang, H. Zhu, and C. Lu. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot.arXiv preprint arXiv:2307.00595, 2023 on PithPith's reviewarXiv
- [32] S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots.arXiv preprint arXiv:2406.02523, 2024 on PithPith's reviewarXiv
- [33] R. Gong, J. Huang, Y . Zhao, H. Geng, X. Gao, Q. Wu, W. Ai, Z. Zhou, D. Terzopoulos, S.-C. Zhu, et al. Arnold: A benchmark for language-grounded task learning with continuous states in realistic 3d scenes. InProceedings of the IEEE/CVF International Conference on Computer Visi... on Pith
- [34] W. Huang, Y .-W. Chao, A. Mousavian, M.-Y . Liu, D. Fox, K. Mo, and L. Fei-Fei. Point- world: Scaling 3d world models for in-the-wild robotic manipulation.arXiv preprint arXiv:2601.03782, 2026 on PitharXiv
- [35] J. Min, Y . Jeon, J. Kim, and M. Choi. S2m2: Scalable stereo matching model for reliable depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 26729–26739, 2025 on Pith
- [36] J. Edstedt, D. Nordstr ¨om, Y . Zhang, G. B¨okman, J. Astermark, V . Larsson, A. Heyden, F. Kahl, M. Wadenb¨ack, and M. Felsberg. Roma v2: Harder better faster denser feature matching.arXiv preprint arXiv:2511.15706, 2025 on PithPith's reviewarXiv
- [37] D. Nordstr ¨om, J. Edstedt, G. B ¨okman, J. Astermark, A. Heyden, V . Larsson, M. Wadenb¨ack, M. Felsberg, and F. Kahl. Loma: Local feature matching revisited.arXiv preprint arXiv:2604.04931, 2026 on PithPith's reviewarXiv
- [38] M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023 on PithPith's reviewarXiv
- [39] J. T. Barron. A general and adaptive robust loss function. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4331–4339, 2019 on Pith
- [40] D. Barath, J. Noskova, M. Ivashechkin, and J. Matas. Magsac++, a fast, reliable and accurate robust estimator. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1304–1312, 2020 on Pith
- [41] C. Yuan, S. Joshi, S. Zhu, H. Su, H. Zhao, and Y . Gao. Roboengine: Plug-and-play robot data augmentation with semantic robot segmentation and background generation. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7622– on Pith
- [42] M. Sedlacek, P. Yefanov, G. Ponimatkin, J. Bardhan, S. Pilc, M. Fourmy, E. Kazakos, C. G. Snoek, J. Sivic, and V . Petrik. Realm: A real-to-sim validated benchmark for generalization in robotic manipulation.IEEE Robotics and Automation Letters, 2026. 12 on Pith
- [43] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004 on Pith
- [44] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018 on Pith
- [45] B. Wen, M. Trepte, J. Aribido, J. Kautz, O. Gallo, and S. Birchfield. Foundationstereo: Zero- shot stereo matching. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5249–5260, 2025 on Pith
- [46] C. Ryali, Y .-T. Hu, D. Bolya, C. Wei, H. Fan, P.-Y . Huang, V . Aggarwal, A. Chowdhury, O. Poursaeed, J. Hoffman, et al. Hiera: A hierarchical vision transformer without the bells- and-whistles. InInternational conference on machine learning, pages 29441–29454. PMLR, 2023 on Pith
- [47] D. G. Lowe. Object recognition from local scale-invariant features. InProceedings of the seventh IEEE international conference on computer vision, volume 2, pages 1150–1157. Ieee, 1999 on Pith
- [48] D. DeTone, T. Malisiewicz, and A. Rabinovich. Superpoint: Self-supervised interest point detection and description. InProceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 224–236, 2018 on Pith
- [49] P. Lindenberger, P.-E. Sarlin, and M. Pollefeys. Lightglue: Local feature matching at light speed. InProceedings of the IEEE/CVF international conference on computer vision, pages 17627–17638, 2023 on Pith
- [50] P.-E. Sarlin, A. Unagar, M. Larsson, H. Germain, C. Toft, V . Larsson, M. Pollefeys, V . Lepetit, L. Hammarstrand, F. Kahl, et al. Back to the feature: Learning robust camera localization from pixels to pose. InProceedings of the IEEE/CVF conference on computer vision and patt... on Pith
- [51] A. ˇS. Mikeˇst´ıkov´a, M. Fourmy, M. Cifka, J. Sivic, and V . Petrik. Alignpose: Generalizable 6d pose estimation via multi-view feature-metric alignment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14626–14636, 2026. 13 Supplemen... on Pith
Cited by 1 Pith paper
Reviewed papers on Pith that cite this one. The full citation record
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment