Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

RaySt3R: Predicting Novel Depth Maps for Zero-Shot Object Completion

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RaySt3R claims that 3D shape completion from a single RGB-D image can be solved by predicting depth, masks, and confidence for novel query rays and merging them, beating prior methods by 20–44% in chamfer distance.

desk verdict A solid and well-executed incremental advance: the ray-map view-synthesis formulation for object completion works, but the paper needs a depth-noise ablation and cleaner hyperparameter hygiene before the zero-shot claim is fully convincing. read the letter →

arxiv 2506.05285 v1 pith:LZYGWAXX submitted 2025-06-05 cs.CV

Bardienus P. Duisterhof , Jan Oberst , Bowen Wen , Stan Birchfield , Deva Ramanan , Jeffrey Ichnowski This is my paper
classification cs.CV
keywords 3DshapecompletionnovelviewsynthesisRGB-Dtransformerzero-shotgeneralizationmulti-objectscenesconfidence-awaredepthsim-to-real
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RaySt3R argues that complete 3D shapes of all objects in a scene can be reconstructed from a single masked RGB-D image by treating shape completion as novel-view synthesis. Instead of predicting voxels or point clouds directly, a feedforward transformer takes the novel viewpoint as a ray map and outputs depth, object mask, and per-pixel confidence for that view, then predictions from multiple views are merged into a full shape. The method is trained on 1.1 million synthetic scenes with 12 million views without volumetric supervision, and is evaluated zero-shot on real-world benchmarks where the paper reports chamfer-distance improvements of 20–44% over the strongest baseline.

What carries the argument

The central object is the query ray map $R$, an $H \times W$ array of normalized ray directions that encodes a novel viewpoint. It is fed as queries in cross-attention to a transformer whose keys combine a point map of the input RGB-D and DINOv2 visual features. The argument is carried by a confidence-aware depth loss, which learns per-pixel confidence without explicit supervision, and by the view-merging rule that multiplies an occlusion mask, a thresholded predicted mask, and a confidence threshold to decide which predicted points enter the final shape.

What would settle it

Re-run the YCB-Video or HOPE evaluation with synthetic sensor noise added to the input depth (e.g., Gaussian noise whose standard deviation matches a consumer depth camera, alone and combined with dropout holes), and check whether the reported 20–44% chamfer-distance margins survive; the occlusion assumption in Eq. 6 would fail if performance collapses at realistic noise levels.

Watch

Extended reading notes

Core claim

The central claim is that multi-object 3D shape completion reduces to predicting, for arbitrary query rays, the depth of the surface that first intersects the ray, together with a mask and a confidence score. The input is a masked RGB-D image plus a query ray map defining a novel viewpoint; the model uses a vision transformer with frozen DINOv2 features for the RGB, self-attention on the input point map and the ray map, cross-attention between the two, and DPT heads for depth, mask, and confidence. Predictions from 22 sampled views are fused by an occlusion test against the input depth and mask, a thresholded predicted object mask, and a thresholded confidence score. The paper reports state-of-the-art chamfer distance and F1@10mm on the OctMAE, YCB-Video, HOPE, and HomebrewedDB benchmarks, with the weakest standard deviation among methods, despite zero-shot transfer from synthetic training.

Load-bearing premise

The merging step assumes the input depth map and foreground mask are accurate enough to decide which novel-view points are occluded; real depth noise could remove true geometry or keep hallucinated points, and the paper tests mask noise but not input-depth noise.

Editorial extensions

If this is right

  • Robots can obtain complete object geometry for grasping and manipulation from a single RGB-D frame, with inference under 1.2 seconds and no multi-view capture or volumetric grid.
  • Shape completion no longer requires volumetric 3D supervision; training on pairs of RGB-D views plus query rays is enough, which makes data scaling substantially cheaper.
  • A single confidence threshold trades accuracy against completeness, giving downstream systems a direct knob for outlier tolerance.
  • Because predictions are per-view in camera coordinates, the output aligns naturally with pose-estimation and motion-planning pipelines, unlike canonical-space generators.
  • The approach scales in compute via the number of queried views, so deployment can trade reconstruction density against latency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ray-map query formulation is generic: the same network could be asked for depth along arbitrary rays outside the sampled view sphere, e.g., from an arbitrary camera pose, without retraining; the paper does not explicitly explore this.
  • Adding realistic depth dropout and sensor noise to the input during training is a direct, untested extension that would target the weakest assumption of the merging step.
  • The per-ray confidence scores could double as an active-view-selection signal: a system could query extra views precisely where confidence is low, reducing hallucination in unseen regions; the paper does not investigate this loop.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. RaySt3R recasts single-RGB-D multi-object 3D shape completion as novel-view depth prediction. Given a foreground-masked RGB-D image and a set of query rays, a DINOv2-conditioned ViT with cross-attention predicts per-pixel depth, object mask, and confidence for the novel views. Predictions from 22 views are fused using an occlusion test against the input depth (Eq. 6), predicted-mask thresholding, and a confidence threshold. The model is trained on 1.1M synthetic scenes from FoundationPose and OctMAE and evaluated zero-shot on the OctMAE test split and on YCB-Video, HOPE, and HomebrewedDB, reporting state-of-the-art chamfer distance and F1@10mm, with gains of 20-44% over OctMAE. The paper also provides ablations of training data, architecture components, and the merging procedure.

Significance. If the reported results hold, RaySt3R is a strong practical method for zero-shot multi-object shape completion: it is fast (1.2 s on a 4090), does not require per-object alignment or iterative refinement, and generalizes from synthetic data to real cluttered scenes. The paper's strengths include a large-scale synthetic training dataset, a systematic ablation of the view-merging pipeline (Table 3), standard-deviation reporting (Table 4), a mask-noise sensitivity analysis (Fig. 9), and the promise of open-sourcing the dataset. The central novelty—training a feedforward transformer to predict confidence-aware depth for arbitrary query rays and fusing them—is clearly presented and well-motivated.

major comments (3)
  1. [Section 4.4, Eq. (6); Table 3] The occlusion filter in Eq. (6) assumes the input depth map D_input is an accurate measurement of the visible foreground: every novel-view point projecting behind D_input is retained as newly revealed geometry, and every point in front is discarded. Table 3 shows that this component is load-bearing (removing it degrades OctMAE-test CD from 5.21 to 7.69), yet the paper only ablates noise in the input mask (Fig. 9), never noise or systematic bias in D_input. The real-world evaluation datasets contain noisy and incomplete depth, and the claimed 20-44% gains over OctMAE on those datasets could be sensitive to depth errors: if D_input reads too large, true novel-view surfaces are incorrectly discarded; if it reads too small, hallucinated points behind the measured surface survive. Please add a depth-noise ablation (e.g., Gaussian depth perturbation, holes, and a constant bias) on the real-world datasets, or otherwise provide evidence that the merging step is robust to realistic sensor noise.
  2. [Section 5.4 and Figure 4] Several hyperparameters are selected using the evaluation benchmarks themselves, which weakens the zero-shot claim. The confidence threshold τ=5 is justified by Figure 4, where the curve is averaged over 'all real-world datasets'—precisely the datasets used for the main evaluation—and the view-sampling parameters are set to λ_bb=1.3 and λ_cam=0.7 for all real-world datasets but λ_bb=2.5 and λ_cam=1.2 for OctMAE, i.e., adapted per evaluation dataset. Because the headline claim is zero-shot generalization, tuning on the test benchmarks gives the method an advantage that a truly zero-shot user would not have. Please either (a) select all hyperparameters on a held-out validation split disjoint from the test frames, or (b) provide a sensitivity analysis showing that the reported improvements hold over a wide range of τ, λ_bb, and λ_cam. The current presentation does not rule out that part of the 20-44% gain comes from per-dataset tuning.
  3. [Sections 5.1 and 5.2] The method is trained on synthetic data from the OctMAE dataset and evaluated on the 'evaluation split of the OctMAE dataset test split', but the paper does not state whether the test scenes or objects were excluded from training. If the OctMAE test split contains scenes or meshes that also appear in the training data, the synthetic benchmark would be optimistic and the reported 5.21 mm CD on this split would need to be re-interpreted. Please clarify the exact split protocol, including whether any object meshes from the test set are present in the training set, and if so, report results on a held-out split with unseen objects.
minor comments (4)
  1. [Section 4.3] The text says 'We don't include the input point map in our predictions as it likely contains noise and artifacts. Instead, query RaySt3R with the input view and include it in our predictions.' This appears contradictory and should be reworded to clarify whether the raw input point map is excluded from the final merged cloud while the predicted depth for the input view is included, or whether the input view is handled differently from other views.
  2. [Table 4] The table reports standard deviations as subscripts, but the caption says 'STD' without specifying the unit of aggregation; please state whether the standard deviation is computed over frames within each dataset, and over which number of samples, so the reader can interpret the ± values correctly.
  3. [Eq. (6)] The notation (p_n,i,j)_z is used for the z-component of the projected point; please define this explicitly in the text, and also clarify the handling of points whose projection falls outside the input image boundaries or exactly on the foreground-background boundary.
  4. [Acknowledgements and Supplement 8.3.2] There are minor typos: 'memebers' should be 'members' in the acknowledgements, and 'perfromance' should be 'performance' in Section 8.3.2. Please proofread; a few other small grammatical issues exist in the supplement.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivation-level circularity: RaySt3R's central claim rests on held-out external benchmarks, not on a self-referential fit.

full rationale

The paper's derivation is an empirical pipeline, not a reduction of outputs to inputs. RaySt3R is trained with Eqs. (3)-(5) to predict novel-view depth, masks, and confidence from a masked RGB-D input and query ray map; those predictions are functions of learned weights, not algebraic consequences of the input depth. The occlusion filter in Eq. (6) uses the input depth and mask as geometric priors to decide which predicted points are newly visible, but it does not construct the predicted depth values, so it is not circular. The headline result is judged against the held-out OctMAE test split and the external real-world YCB-Video, HOPE, and HomebrewedDB benchmarks, with baseline numbers taken from prior work; no target metric is defined by RaySt3R's own output. Self-citations such as MASt3R-SFM [9] and FoundationPose [50] appear as architectural inspiration and training-data sources, but they are not used as unverified load-bearing premises or uniqueness theorems. The per-dataset view-sampling parameters and confidence threshold are chosen with some lookahead to the reported benchmarks, which is a test-set-tuning caveat rather than a circular derivation. Overall, no specific equation or fitted input is renamed as a prediction.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The method is empirical; the central claim rests on a trained model and manual merging rules. No new physical entities are introduced. The key free parameters are hyperparameters, several of which are tuned per evaluation dataset. The axioms are the standard camera model, the representativeness of the synthetic training data, the usefulness of DINOv2 features, and the correctness of the hand-crafted fusion. None of these is machine-checked.

free parameters (6)
  • Confidence threshold tau = 5
    Selected on the evaluation datasets via Figure 4, balancing accuracy and completeness; used in Eq. (7).
  • View sphere radius factor lambda_bb = 1.3 (real-world), 2.5 (OctMAE)
    Tunable per dataset, Section 5.4, controls how far query views are sampled from the object bounding box.
  • Minimum camera distance factor lambda_cam = 0.7 (real-world), 1.2 (OctMAE)
    Tunable per dataset, Section 5.4, clips the view radius to avoid degenerate small boxes.
  • Confidence loss weight alpha = 0.2
    Set during training in Section 5.4 for the depth loss in Eq. (3).
  • Mask loss weight lambda_mask = 0.1
    Set during training in Section 5.4 to combine depth and mask losses in Eq. (5).
  • Number of query views = 22
    Used at inference to sample novel views, Section 5.4.
assumptions (6)
  • standard math Pinhole camera model: unprojection X = K^{-1}[iD, jD, D]^T and projection p = K P h(q) correctly relate image pixels to 3D points.
    Used in Section 4.1 for the input point map and in Section 4.4 Eq. (6) for occlusion checking.
  • domain assumption Synthetic training distribution (GSO/Objaverse objects in random scenes) is representative enough to generalize to real cluttered scenes.
    Sections 5.1 and 5.6; zero-shot generalization depends on this transfer.
  • domain assumption Frozen DINOv2 features provide a useful visual prior for predicting depth and geometry from novel views.
    Section 4.1 and ablation Table 2 show removing DINOv2 worsens performance; the method's success relies on it.
  • domain assumption The hand-crafted occlusion and mask merging (Eqs. 6 and 7) yields a complete and accurate surface when filtered by confidence and predicted masks.
    Section 4.4; Table 3 ablation supports the design, but the merging rule is not formally derived.
  • domain assumption Ground-truth meshes and depth values in the evaluation datasets are reliable references for Chamfer distance and F1.
    Section 5.2 and Section 8.3.1; all metrics are computed against these references.
  • domain assumption The confidence scores learned without explicit supervision are a valid proxy for per-pixel depth error and can be thresholded to improve the final reconstruction.
    Section 4.2 Eq. (3) and Section 5.8; Figure 4 supports the proxy, but the threshold is tuned on the evaluation data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RaySt3R: Predicting Novel Depth Maps for Zero-Shot Object Completion." pith.science (2026). https://pith.science/paper/LZYGWAXX

@misc{pith2026250605285,
  author       = {Pith},
  title        = {Pith review of: RaySt3R: Predicting Novel Depth Maps for Zero-Shot Object Completion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LZYGWAXX}},
  note         = {Machine review of arXiv:2506.05285}
}
read the original abstract

3D shape completion has broad applications in robotics, digital twin reconstruction, and extended reality (XR). Although recent advances in 3D object and scene completion have achieved impressive results, existing methods lack 3D consistency, are computationally expensive, and struggle to capture sharp object boundaries. Our work (RaySt3R) addresses these limitations by recasting 3D shape completion as a novel view synthesis problem. Specifically, given a single RGB-D image and a novel viewpoint (encoded as a collection of query rays), we train a feedforward transformer to predict depth maps, object masks, and per-pixel confidence scores for those query rays. RaySt3R fuses these predictions across multiple query views to reconstruct complete 3D shapes. We evaluate RaySt3R on synthetic and real-world datasets, and observe it achieves state-of-the-art performance, outperforming the baselines on all datasets by up to 44% in 3D chamfer distance. Project page: https://rayst3r.github.io

Figures

Figures reproduced from arXiv: 2506.05285 by the authors.

Figure 1
Figure 1. RaySt3R is a method for zero-shot 3D shape completion from a single foreground-masked RGB-D [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The architecture of RaySt3R. RaySt3R takes a single RGB-D image and foreground mask as input, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results of RaySt3R in real-world multi-object scenes. For each scene, we select a subset [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (14 more)
Figure 5
Figure 5. Figure 5: The impact of the confidence threshold on the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: RaySt3R chamfer distance distribution compared against several baselines. The results suggest [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 8
Figure 8. Figure 8: 8.2.2 Mask ablation We have demonstrated RaySt3R outperforms the baselines in the real world, even with imperfect masks. It remains unclear how sensitive our method is to the input mask [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 7
Figure 7. Figure 7: Rotation ablation: Chamfer Distance averaged over the evaluation split of HomebrewedDB [19], HOPE [40], and YCB-Video [54] with different numbers of steps during the rotational alignment grid search. 0.8 1.0 1.2 Scaling Factor 20 40 Chamfer-L1 HOPE HomebrewedDB YCB-Vid…
Figure 8
Figure 8. Figure 8: Scaling ablation: Chamfer distance averaged over the evaluation split of HomebrewedDB [19], HOPE [40], and YCB-Video [54] for different initial scaling factors. the foreground marked as background. We study noise in the range of 0 to 0.2, meaning 0% to 20% of the foreg…
Figure 9
Figure 9. Figure 9: RaySt3R input mask noise ablation. The results suggest RaySt3R is more robust against false positive [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Highlight on relative object placement and aspect ratio. The examples suggest related works such as [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Comparison of different methods on scene [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Comparison of different methods on scene [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: Comparison of different methods on scene [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Comparison of different methods on scene [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Comparison of different methods on scene [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Comparison of different methods on scene [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]
Figure 17
Figure 17. Figure 17: Qualitative results for RGB predictions from ViewCrafter [ [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Learning Physics-Guided Residual Dynamics for Deformable Object Simulation

    cs.RO 2026-07 conditional novelty 6.0 of 10

    Physics-guided residual dynamics, a spring-mass simulator plus a network that predicts velocity corrections, yields the most accurate deformable-object simulation in the paper's real-world tests.

Reference graph

Works this paper leans on

57 extracted references · 44 canonical work pages · cited by 1 Pith paper

  1. [1]

    Agarwal, G

    A. Agarwal, G. Singh, B. Sen, T. Lozano-Pérez, and L. P. Kaelbling. Scenecomplete: Open- world 3d scene completion in complex real world environments for robot manipulation, 2024

    work page 2024

  2. [2]

    J. T. Barron and J. Malik. Intrinsic scene properties from a single rgb-d image. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 17–24, 2013

    work page 2013

  3. [3]

    Boulch and R

    A. Boulch and R. Marlet. Poco: Point convolution for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6302–6314, 2022

    work page 2022

  4. [4]

    Bozic, P

    A. Bozic, P. Palafox, J. Thies, A. Dai, and M. Nießner. Transformerfusion: Monocular rgb scene reconstruction using transformers. Advances in Neural Information Processing Systems , 34:1403–1414, 2021

    work page 2021

  5. [5]

    C. Choy, J. Gwak, and S. Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In CVPR, 2019

    work page 2019

  6. [6]

    Dalal, M

    M. Dalal, M. Liu, W. Talbott, C. Chen, D. Pathak, J. Zhang, and R. Salakhutdinov. Local policies enable zero-shot long-horizon manipulation. International Conference of Robotics and Automation, 2025

    2025

  7. [7]

    Deitke, D

    M. Deitke, D. Schwenk, J. Salvador, L. Weihs, O. Michel, E. VanderBilt, L. Schmidt, K. Ehsani, A. Kembhavi, and A. Farhadi. Objaverse: A universe of annotated 3d objects. CVPR, 2022

    work page 2022

  8. [8]

    Downs, A

    L. Downs, A. Francis, N. Koenig, B. Kinman, R. Hickman, K. Reymann, T. B. McHugh, and V . Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. In ICRA, 2022. 17

    work page 2022

Show all 57 references
  1. [9]

    Duisterhof, L

    B. Duisterhof, L. Zust, P. Weinzaepfel, V . Leroy, Y . Cabon, and J. Revaud. Mast3r-sfm: a fully-integrated solution for unconstrained structure-from-motion, 2024

    2024

  2. [10]

    El-Nouby, M

    A. El-Nouby, M. Klein, S. Zhai, M. A. Bautista, V . Shankar, A. Toshev, J. M. Susskind, and A. Joulin. Scalable pre-training of large autoregressive image models. In Proceedings of the 41st International Conference on Machine Learning , ICML’24. JMLR.org, 2024

    2024

  3. [11]

    H. Fan, H. Su, and L. J. Guibas. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 605–613, 2017

    2017

  4. [12]

    R. Gao*, A. Holynski*, P. Henzler, A. Brussee, R. Martin-Brualla, P. P. Srinivasan, J. T. Barron, and B. Poole*. Cat3d: Create anything in 3d with multi-view diffusion models. Advances in Neural Information Processing Systems, 2024

    2024

  5. [13]

    K. He, X. Chen, S. Xie, Y . Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. CVPR, 2022

    2022

  6. [14]

    J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  7. [15]

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet. Video diffusion models. Advances in Neural Information Processing Systems , 35:8633–8646, 2022

    2022

  8. [16]

    J. Hou, A. Dai, and M. Nießner. Revealnet: Seeing behind objects in rgb-d scans. In CVPR, 2020

    2020

  9. [17]

    Huang, L

    H. Huang, L. Fu, M. Danielczuk, C. M. Kim, Z. Tam, J. Ichnowski, A. Angelova, B. Ichter, and K. Goldberg. Mechanical search on shelves with efficient stacking and destacking of objects. In Robotics Research, pages 205–221, Cham, 2023. Springer Nature Switzerland

    2023

  10. [18]

    Iwase, K

    S. Iwase, K. Liu, V . Guizilini, A. Gaidon, K. Kitani, R. Ambrus, and S. Zakharov. Zero-shot multi-object scene completion, 2024

    2024

  11. [19]

    Kaskman, S

    R. Kaskman, S. Zakharov, I. Shugurov, and S. Ilic. Homebreweddb: Rgb-d dataset for 6d pose estimation of 3d objects. ICCVW, 2019

    2019

  12. [20]

    Kolesnikov, A

    A. Kolesnikov, A. Dosovitskiy, D. Weissenborn, G. Heigold, J. Uszkoreit, L. Beyer, M. Minderer, M. Dehghani, N. Houlsby, S. Gelly, T. Unterthiner, and X. Zhai. An image is worth 16x16 words: Transformers for image recognition at scale. 2021

    2021

  13. [21]

    Ladicky, O

    L. Ladicky, O. Saurer, S. Jeong, F. Maninchedda, and M. Pollefeys. From point clouds to mesh using regression. In Proceedings of the IEEE International Conference on Computer Vision , pages 3893–3902, 2017

    2017

  14. [22]

    J. Li, K. Han, P. Wang, Y . Liu, and X. Yuan. Anisotropic convolutional networks for 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3351–3359, 2020

    2020

  15. [23]

    R. Li, B. Zhang, Z. Li, F. Tombari, and P. Wonka. Lari: Layered ray intersections for single-view 3d geometric reasoning. In arXiv preprint arXiv:2504.18424, 2025

    2025 arXiv

  16. [24]

    Y . Li, Z. Yu, C. Choy, C. Xiao, J. M. Alvarez, S. Fidler, C. Feng, and A. Anandkumar. V oxformer: Sparse voxel transformer for camera-based 3d semantic scene completion. In CVPR, 2023

    2023

  17. [25]

    R. Liu, R. Wu, B. Van Hoorick, P. Tokmakov, S. Zakharov, and C. V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023

    2023

  18. [26]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  19. [27]

    Mahler, M

    J. Mahler, M. Matl, V . Satish, M. Danielczuk, B. DeRose, S. McKinley, and K. Goldberg. Learning ambidextrous robot grasping policies. Science Robotics, 4(26):eaau4984, 2019. 18

    2019

  20. [28]

    Melas-Kyriazi, C

    L. Melas-Kyriazi, C. Rupprecht, and A. Vedaldi. Pc2: Projection-conditioned point cloud diffusion for single-image 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12923–12932, 2023

    2023

  21. [29]

    Mescheder, M

    L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4460–4470, 2019

    2019

  22. [30]

    Mishani, H

    I. Mishani, H. Feddock, and M. Likhachev. Constant-time motion planning with anytime refine- ment for manipulation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), page 10337–10343. IEEE, May 2024

    2024

  23. [31]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, R. Howes, P.-Y . Huang, H. Xu, V . Sharma, S.-W. Li, W. Galuba, M. Rabbat, M. Assran, N. Ballas, G. Synnaeve, I. Misra, H. Jegou, J. Mairal, P. Labatut,...

    2023

  24. [32]

    J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 165–174, 2019

    2019

  25. [33]

    S. Peng, M. Niemeyer, L. Mescheder, M. Pollefeys, and A. Geiger. Convolutional occupancy networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 523–540. Springer, 2020

    2020

  26. [34]

    Ranftl, A

    R. Ranftl, A. Bochkovskiy, and V . Koltun. Vision transformers for dense prediction. InICCV, 2021

    2021

  27. [35]

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024

    2024

  28. [36]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  29. [37]

    Saharia, W

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gon- tijo Lopes, B. Karagol Ayan, T. Salimans, et al. Photorealistic text-to-image diffusion mod- els with deep language understanding. Advances in neural information processing systems , 35:3647...

    2022

  30. [38]

    Somani, T

    A. Somani, T. S. Huang, and S. D. Blostein. Least-squares fitting of two 3-D point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 698–700, 1987

    1987

  31. [39]

    Z. Tang, B. Sundaralingam, J. Tremblay, B. Wen, Y . Yuan, S. Tyree, C. Loop, A. Schwing, and S. Birchfield. Rgb-only reconstruction of tabletop scenes for collision-free manipulator control. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 1778–1...

    2023

  32. [40]

    Tyree, J

    S. Tyree, J. Tremblay, T. To, J. Cheng, T. Mosier, J. Smith, and S. Birchfield. 6-dof pose estimation of household objects for robotic manipulation: An accessible dataset and benchmark. In International Conference on Intelligent Robots and Systems (IROS) , 2022

    2022

  33. [41]

    Vahdat, F

    A. Vahdat, F. Williams, Z. Gojcic, O. Litany, S. Fidler, K. Kreis, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems , 35:10021–10039, 2022

    2022

  34. [42]

    V oleti, C.-H

    V . V oleti, C.-H. Yao, M. Boss, A. Letts, D. Pankratz, D. Tochilkin, C. Laforte, R. Rombach, and V . Jampani. Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion. In European Conference on Computer Vision, pages 439–457. Springe...

    2024

  35. [43]

    J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  36. [44]

    P.-S. Wang, Y . Liu, and X. Tong. Deep octree-based cnns with output-guided skip connections for 3d shape and scene completion. In CVPRW, 2020

    2020

  37. [45]

    Q. Wang, Y . Zhang, A. Holynski, A. A. Efros, and A. Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501.12387, 2025

    2025 arXiv

  38. [46]

    S. Wang, V . Leroy, Y . Cabon, B. Chidlovskii, and J. Revaud. Dust3r: Geometric 3d vision made easy. In CVPR, 2024

    2024

  39. [47]

    Weber, N

    E. Weber, N. Müller, Y . Kant, V . Agrawal, M. Zollhöfer, A. Kanazawa, and C. Richardt. Fillerbuster: Multi-view scene completion for casual captures, 2025. arXiv:2502.05175

    2025

  40. [48]

    B. Wen, W. Lian, K. Bekris, and S. Schaal. Catgrasp: Learning category-level task-relevant grasping in clutter from simulation. In 2022 International Conference on Robotics and Automa- tion (ICRA), pages 6401–6408. IEEE, 2022

    2022

  41. [49]

    B. Wen, C. Mitash, B. Ren, and K. E. Bekris. se (3)-tracknet: Data-driven 6d pose tracking by calibrating image residuals in synthetic domains. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 10367–10373. IEEE, 2020

    2020

  42. [50]

    B. Wen, W. Yang, J. Kautz, and S. Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. In CVPR, 2024

    2024

  43. [51]

    C.-Y . Wu, J. Johnson, J. Malik, C. Feichtenhofer, and G. Gkioxari. Multiview compressive coding for 3D reconstruction. In CVPR, 2023

    2023

  44. [52]

    K. Wu, F. Liu, Z. Cai, R. Yan, H. Wang, Y . Hu, Y . Duan, and K. Ma. Unique3d: High-quality and efficient 3d mesh generation from a single image, 2024

    2024

  45. [53]

    Xiang, Z

    J. Xiang, Z. Lv, S. Xu, Y . Deng, R. Wang, B. Zhang, D. Chen, X. Tong, and J. Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506, 2024

    2024 arXiv

  46. [54]

    Xiang, T

    Y . Xiang, T. Schmidt, V . Narayanan, and D. Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. 2018

    2018

  47. [55]

    X. Yan, L. Lin, N. J. Mitra, D. Lischinski, D. Cohen-Or, and H. Huang. Shapeformer: Transformer-based shape completion via sparse representation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

    2022

  48. [56]

    K. Yao, L. Zhang, X. Yan, Y . Zeng, Q. Zhang, L. Xu, W. Yang, J. Gu, and J. Yu. CAST: Component-aligned 3d scene reconstruction from an RGB image. In arXiv:2502.12894, 2025

    2025 arXiv

  49. [57]

    W. Yu, J. Xing, L. Yuan, W. Hu, X. Li, Z. Huang, X. Gao, T.-T. Wong, Y . Shan, and Y . Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02048, 2024. 20

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.