Pith. sign in

REVIEW 5 major objections 5 minor 57 references

BoxFusion: Reconstruction-Free Open-Vocabulary 3D Object Detection via Real-Time Multi-View Box Fusion

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BoxFusion claims open-vocabulary 3D object detection can run in real time from an RGB-D stream without dense reconstruction, and reports state-of-the-art online results on CA-1M and ScanNetV2.

desk verdict A clever reconstruction-free fusion system with a genuine new task, but the headline CA-1M numbers may be inflated by training-set overlap and need a clean-split re-run. read the letter →

arxiv 2506.15610 v3 pith:VEXXCCUD submitted 2025-06-18 cs.CV

Yuqing Lan , Chenyang Zhu , Zhirui Gao , Jiazhao Zhang , Yihan Cao , Renjiao Yi , Yijie Wang , Kai Xu This is my paper
classification cs.CV
keywords open-vocabulary3Dobjectdetectionreconstruction-freeperceptionmulti-viewboxfusionparticlefilteringoptimizationRGB-DstreamingCubifyAnythingCLIPonline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that dense 3D reconstruction is unnecessary for accurate open-vocabulary 3D object detection from an RGB-D video stream. Instead, single-view 3D box proposals from a pretrained foundation model are associated across views and fused by an IoU-guided random-optimization step into globally consistent boxes, with semantics supplied by CLIP. The authors report state-of-the-art results among online methods on CA-1M and ScanNetV2, including an AP15 of 31.22 on CA-1M versus 9.17 for the best online baseline, while running above 20 FPS and using 7.0GB of GPU memory. If correct, this means embodied systems can perceive objects in large indoor spaces without paying the cost of building and storing a dense map.

What carries the argument

The load-bearing machinery is the multi-view box fusion module. Given a global box $G_t^i$ with position $p=(x,y,z)$ and shape $s=(l,w,h)$, it maximizes the summed 2D IoU between the convex hull of the global box's projected corners and the hulls of the per-view candidate boxes (Eq. 5). Because this objective is highly nonlinear, the optimization is carried out by particle filtering with a pre-sampled particle swarm template (PST): particles in 6D space are scored by the objective, the best ones are moved and rescaled, and the procedure iterates until convergence. The association module feeds this optimizer with candidate lists: 3D NMS associates spatially overlapping oriented boxes by sampled convex-hull IoU, and a 2D projective-IoU matcher associates small objects whose proposals are adjacent but not overlapping.

What would settle it

Held-out-view test: for each detected object, fuse proposals from a subset of views, then project the fused box into a view that was excluded from fusion and measure its 2D IoU with the visible object; if the fused box is not more accurate than that view's own single-view proposal, the reported gains come from self-consistency, not true 3D accuracy.

Watch

Extended reading notes

Core claim

The central claim is that a sparse, box-only scene representation — the position, scale, and open-vocabulary semantics of each object — is sufficient for online 3D perception, and that multi-view consistency can be enforced by optimizing a 6D box (position and shape) so that its projections align with per-view proposals. On the paper's own terms, BoxFusion establishes that reconstruction-free detection outperforms both offline point-cloud methods and online reconstruction-based methods on CA-1M and ScanNetV2, and that the gap is largest on fine-grained and small objects. The authors attribute the result to two mechanisms: association via 3D NMS plus 2D projective correspondence, which recovers small objects that spatial-overlap strategies miss, and particle-filter random optimization, which fuses boxes without the cost of dense reconstruction.

Load-bearing premise

The fusion objective only maximizes agreement with the system's own per-view proposals, which the paper itself concedes are 3D-biased with scale uncertainty, so the central assumption is that this bias is small or symmetric enough that multi-view consensus is closer to the true object than any single view; if the bias is systematic, the fused boxes inherit it and the AP gains may reflect self-consistency rather than accuracy.

Editorial extensions

If this is right

  • Online 3D detection can run at over 20 FPS with 7.0GB of GPU memory, because no dense reconstruction or point-cloud storage is needed.
  • The box-only representation is enough to support downstream embodied tasks such as scene-graph generation and collision detection, at least in static indoor scenes.
  • Fine-grained and small objects (e.g., wall switches, remotes) that point-cloud pipelines typically miss become detectable because association is driven by 2D correspondence as well as 3D overlap.
  • Deployment scales to large environments exceeding 1000 square meters, but only while the scene remains static; moving people or objects produce redundant and inaccurate boxes, and tightly stacked objects remain ambiguous.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the reported gains should be tested with held-out views — fusing proposals from one subset of views and checking accuracy against a view not used in fusion — since the optimization maximizes agreement with the system's own proposals, which the paper admits are biased in 3D.
  • Editorial inference: Eq. 6 as printed has a sign error relative to Eq. 5: it exponentiates the negative summed IoU, which would reward smaller overlap, so the published likelihood cannot be what the optimization actually maximizes unless the implementation differs from the text.
  • Editorial inference: extending the framework with dynamic-object tracking or instance-aware motion filtering could address the static-scene limitation, and the object-centric memory produced by box fusion could feed navigation or manipulation policies directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes BoxFusion, a reconstruction-free online open-vocabulary 3D object detection system. Given a streaming posed RGB-D input, the method uses Cubify Anything to produce single-view 3D box proposals, CLIP to attach open-vocabulary semantics, an association module based on 3D NMS and 2D projective IoU matching, and a particle-filter-based random optimization that fuses multi-view proposals into a single global 3D box by maximizing multi-view 2D IoU. The authors report state-of-the-art results among online methods on CA-1M (class-agnostic AP15 of 31.22 vs. 9.17 for EmbodiedSAM) and ScanNetV2 (AP15 of 37.46 vs. 31.39 for OnlineAnySeg), with an average runtime above 20 FPS and 7.0 GB GPU memory. The central claim is that dense 3D reconstruction is unnecessary for accurate open-vocabulary 3D detection from posed RGB-D streams.

Significance. If the results hold, the paper makes a valuable practical point: a sparse, box-only scene representation can support open-vocabulary 3D detection at real-time rates and low memory footprint, which is relevant for embodied navigation and manipulation. The evaluation is anchored to external ground-truth boxes, and the ablation studies and runtime measurements are useful. The paper also promises code release. The significance is currently tempered by two correctness issues in the published equations and by a potential evaluation-leakage concern on CA-1M, where the frozen proposal generator appears to be trained on the same benchmark that is used for validation. Because the ScanNetV2 result is less exposed to that concern, the central paradigm is still defensible, but the CA-1M headline claim needs clarification or re-evaluation.

major comments (5)
  1. [Sec. 4.1, Sec. 3.2, Table 1] The CA-1M column in Table 1 is the strongest support for the headline claim, but the manuscript never states whether the 107 CA-1M validation scenes are disjoint from the training data of Cubify Anything, which is described in Sec. 3.2 as trained on the large-scale dataset introduced in the same reference [19] that defines CA-1M. If the validation scenes overlap the training distribution, the 31.22 AP15 figure largely measures seen-object bias rather than the reconstruction-free fusion paradigm. Please state the split explicitly, and if overlap exists, re-evaluate on a disjoint held-out set or report CA-1M results with a proposal generator not trained on that benchmark.
  2. [Sec. 3.4, Eq. (6)] The likelihood is written as exp(-(1/xi) * sum_j H(alpha_ij, beta_ij)). Since Eq. (5) maximizes sum_j H, maximizing this likelihood actually minimizes the IoU sum, which is the opposite of the stated objective. The sign preceding the summation should be positive, or the expression should be framed as an energy to be minimized. As published, the equation contradicts the optimization described in the text and in Figure 4.
  3. [Sec. 3.3.1, Eq. (2)] The expression labeled IoU in Eq. (2) is not an IoU. The numerator sums points in the union Gi union Gj, while the denominator sums indicators for Gi and Gj separately, which counts points in the intersection twice. For two identical boxes the formula evaluates to 0.5 rather than 1.0. The correct sampled IoU should use an intersection count in the numerator and a union count in the denominator.
  4. [Sec. 3.3, Sec. 3.4, implementation details] The method depends on at least eight parameters (tau_3d, tau_2d, tau_r, tau_t, xi, N_pst, O_n, tau_box), but none of their values are reported, and the paper defers details to supplementary material that is not part of this submission. This prevents reproduction of the reported AP, FPS, and memory numbers. Please report all parameter values and a brief sensitivity analysis of Table 1 to these parameters.
  5. [Sec. 3.4, Table 2] The fusion objective optimizes agreement with the system's own per-view proposals, and the paper itself notes these proposals are 'biased from the ground truth in 3D with scale uncertainty.' Because the reported AP is computed against external ground-truth boxes, this is not circular, but it does mean the fusion can only remove inconsistency, not correct a systematic per-view bias. To support the claim that multi-view fusion improves accuracy, please add a per-view baseline (e.g., AP of unfused single-view proposals) and an analysis of fused versus single-view box errors.
minor comments (5)
  1. [Sec. 3.4] There is a typo: 'parametes' should be 'parameters'. Also, Eq. (4) writes T^{-1}_t, but since the sum in Eq. (5) runs over views j, the projection should presumably use T^{-1}_j for the j-th view.
  2. [Sec. 4.2, Table 1 caption] The note that OnlineAnySeg 'segments images in an offline manner' means the FPS comparison with that baseline is not an apples-to-apples online comparison; please make this qualification more prominent and define precisely which stages are included in each reported FPS.
  3. [Sec. 4.1] The text says text embeddings are taken for 'over 300' categories from reference [24], but [24] is the COCO paper, which defines 80 categories; please clarify the actual category vocabulary and where the text prompts come from.
  4. [Sec. 4.4, Table 2] The ablation Table 2 reports results on 'six scenes of CA-1M and eight scenes of ScanNetV2' with 'Ours' at 35.49 AP15, which does not match any entry in Table 1; please clarify how the subset scenes are combined and why the full-model AP differs from the main table.
  5. [Figure 7] The phrase 'the while arrows' should be 'the white arrows'.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity: the fusion objective maximizes agreement with the system's own per-view proposals, but every headline AP number is evaluated against external ground-truth boxes, so the central claim is independently grounded. The main caveats are CA-1M training-provenance overlap and an Eq. 6 sign error, neither of which is a definitional circularity.

full rationale

The paper's derivation chain is not circular. Proposals come from the externally pretrained Cubify Anything and CLIP, and the association and fusion modules are hand-designed geometric operations with no parameters fitted to the evaluation labels. The fusion objective in Eq. 5 maximizes the 2D IoU between the fused global box and the system's own per-view proposals, so it enforces multi-view self-consistency by construction; however, all reported AP numbers in Table 1 are computed against ground-truth boxes on CA-1M and ScanNetV2, so the headline state-of-the-art claim is independently grounded. Self-citations to RoseFusion, RemixFusion, MIPS-Fusion, and OnlineAnySeg are technique-inspiration or baseline references, not load-bearing premises. The ablation tables also compare against external GT, so the claimed contributions do not reduce to fitting their own inputs. Two non-circular caveats should be weighed separately. First, Cubify Anything is described as trained on a large-scale dataset introduced in the same reference [19] that defines CA-1M, and the paper does not disclose whether the 107 CA-1M validation scenes are disjoint from Cubify Anything's training data; if overlapping, the large CA-1M margin could reflect training-distribution familiarity rather than the reconstruction-free fusion paradigm. This is a benchmark-provenance risk, not a circularity. Second, Eq. 6 writes the likelihood as exp(-1/xi * sum H), which is inconsistent with Eq. 5's maximization of sum H; this is a correctness bug in the optimization write-up, but the surrounding text and Eq. 5 consistently describe maximizing agreement, so it does not change the circularity verdict. Overall, no step in the paper's derivation is equivalent to its own input by definition, and the score is accordingly low.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The method has six unspecified hand-tuned parameters that control association and fusion, and it builds on the untested reliability of Cubify Anything and given camera poses. No new physical entities are introduced; the only invented construct is the pre-sampled particle swarm template, which has no external evidence beyond the paper's own results.

free parameters (6)
  • 3D NMS threshold tau_3d = not reported
    IoU threshold in Eq. 2 for associating overlapping 3D boxes; manually chosen and not specified.
  • 2D projective IoU threshold tau_2d = not reported
    Threshold in Eq. 3 for associating small objects via projected box overlap; not specified.
  • camera direction and translation thresholds tau_r, tau_t = not reported
    Used in spatial association to decide whether a box is added to a candidate list; not specified.
  • likelihood temperature xi = not reported
    Temperature in Eq. 6 controlling particle weighting; not specified.
  • particle count N_pst and sample points O_n = not reported
    Number of pre-sampled particles and number of sample points for convex hull IoU; not specified.
  • box-count threshold tau_box = not reported
    Minimum number of candidate boxes that triggers fusion optimization; not specified.
assumptions (4)
  • domain assumption Cubify Anything produces reliable metric 3D bounding boxes for single RGB-D images
    The full pipeline depends on the quality of these proposals; the paper relies on a third-party VFM without local verification.
  • domain assumption Camera poses are given and accurate
    All fusion and projection use posed RGB-D inputs; pose errors degrade the method, and the paper does not model pose uncertainty.
  • standard math Pinhole camera projection model
    Used in Eq. 3 and Eq. 4 for projecting 3D boxes to 2D hulls.
  • domain assumption Particle filtering with pre-sampled swarm templates converges to a good optimum of Eq. 5
    The optimization is heuristic; no convergence guarantee is provided, and the sign of Eq. 6 is inconsistent with Eq. 5.
invented entities (1)
  • Pre-sampled particle swarm template (PST)
    purpose: Defines the search distribution for the 6D box parameters in the fusion optimization
    An algorithmic construct, not a physical entity; its only evidence is the method's own performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BoxFusion: Reconstruction-Free Open-Vocabulary 3D Object Detection via Real-Time Multi-View Box Fusion." pith.science (2026). https://pith.science/paper/VEXXCCUD

@misc{pith2026250615610,
  author       = {Pith},
  title        = {Pith review of: BoxFusion: Reconstruction-Free Open-Vocabulary 3D Object Detection via Real-Time Multi-View Box Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VEXXCCUD}},
  note         = {Machine review of arXiv:2506.15610}
}
read the original abstract

Open-vocabulary 3D object detection has gained significant interest due to its critical applications in autonomous driving and embodied AI. Existing detection methods, whether offline or online, typically rely on dense point cloud reconstruction, which imposes substantial computational overhead and memory constraints, hindering real-time deployment in downstream tasks. To address this, we propose a novel reconstruction-free online framework tailored for memory-efficient and real-time 3D detection. Specifically, given streaming posed RGB-D video input, we leverage Cubify Anything as a pre-trained visual foundation model (VFM) for single-view 3D object detection by bounding boxes, coupled with CLIP to capture open-vocabulary semantics of detected objects. To fuse all detected bounding boxes across different views into a unified one, we employ an association module for correspondences of multi-views and an optimization module to fuse the 3D bounding boxes of the same instance predicted in multi-views. The association module utilizes 3D Non-Maximum Suppression (NMS) and a box correspondence matching module, while the optimization module uses an IoU-guided efficient random optimization technique based on particle filtering to enforce multi-view consistency of the 3D bounding boxes while minimizing computational complexity. Extensive experiments on ScanNetV2 and CA-1M datasets demonstrate that our method achieves state-of-the-art performance among online methods. Benefiting from this novel reconstruction-free paradigm for 3D object detection, our method exhibits great generalization abilities in various scenarios, enabling real-time perception even in environments exceeding 1000 square meters.

Figures

Figures reproduced from arXiv: 2506.15610 by the authors.

Figure 1
Figure 1. The state-of-the art methods leverage dense reconstruc [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Method Overview of BoxFusion. (a) Given online RGB-D images with camera poses, we use Cubify Anything to generate [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visualization of multi-view box fusion using random [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: 2D example of particle filtering optimization (PFO) us [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Gallery of 3D object detection on CA-1M and ScanNetV2. Our method is reconstruction-free, and the mesh is semi-transparent. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visualization of online 3D object detections on 42898849 of CA-1M. The first row shows the detected bounding boxes alongside [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Detailed qualitative comparison on ScanNetV2 and CA-1M. Our method achieves comprehensive detection of both common [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Ablation studies of different fusion strategies. Improve [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Open-vocabulary object retrieval with user-specific text [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Typical failure cases. (a) Two humans are walking, [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 41 canonical work pages

  1. [19]

    Cubify anything: Scaling in- door 3d object detection

    Justin Lazarow, David Griffiths, Gefen Kohavi, Francisco Crespo, and Afshin Dehghan. Cubify anything: Scaling in- door 3d object detection. arXiv preprint arXiv:2412.04458,

    arXiv

  2. [1]

    Omni3d: A large benchmark and model for 3d object detection in the wild

    Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3d: A large benchmark and model for 3d object detection in the wild. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 13154–13164, 2023. 2, 6

    work page 2023

  3. [2]

    Coda: Col- laborative novel box discovery and cross-modal alignment for open-vocabulary 3d object detection

    Yang Cao, Zeng Yihan, Hang Xu, and Dan Xu. Coda: Col- laborative novel box discovery and cross-modal alignment for open-vocabulary 3d object detection. Advances in Neu- ral Information Processing Systems, 36:71862–71873, 2023. 2

    work page 2023

  4. [3]

    Cognav: Cognitive pro- cess modeling for object goal navigation with llms

    Yihan Cao, Jiazhao Zhang, Zhinan Yu, Shuzhen Liu, Zheng Qin, Qin Zou, Bo Du, and Kai Xu. Cognav: Cognitive pro- cess modeling for object goal navigation with llms. arXiv preprint arXiv:2412.10439, 2024. 1

    arXiv 2024

  5. [4]

    Collabo- rative novel object discovery and box-guided cross-modal alignment for open-vocabulary 3d object detection

    Yang Cao, Yihan Zeng, Hang Xu, and Dan Xu. Collabo- rative novel object discovery and box-guided cross-modal alignment for open-vocabulary 3d object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence,

    work page

  6. [5]

    A hierarchical graph network for 3d object detection on point clouds

    Jintai Chen, Biwen Lei, Qingyu Song, Haochao Ying, Danny Z Chen, and Jian Wu. A hierarchical graph network for 3d object detection on point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 392–401, 2020. 2

    work page 2020

  7. [6]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 6

    2017

  8. [7]

    Graph-to-3d: End-to-end generation and ma- nipulation of 3d scenes using scene graphs

    Helisa Dhamo, Fabian Manhardt, Nassir Navab, and Fed- erico Tombari. Graph-to-3d: End-to-end generation and ma- nipulation of 3d scenes using scene graphs. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 16352–16361, 2021. 6

    work page 2021

Show all 57 references
  1. [8]

    Disarm: Displacement aware relation mod- ule for 3d detection

    Yao Duan, Chenyang Zhu, Yuqing Lan, Renjiao Yi, Xinwang Liu, and Kai Xu. Disarm: Displacement aware relation mod- ule for 3d detection. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 16980–16989, 2022. 2

    2022

  2. [9]

    3d-mpa: Multi-proposal ag- gregation for 3d semantic instance segmentation

    Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, and Matthias Nießner. 3d-mpa: Multi-proposal ag- gregation for 3d semantic instance segmentation. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9031–9040, 2020. 2

    2020

  3. [10]

    Generic objects as pose probes for few- shot view synthesis

    Zhirui Gao, Renjiao Yi, Chenyang Zhu, Ke Zhuang, Wei Chen, and Kai Xu. Generic objects as pose probes for few- shot view synthesis. IEEE Transactions on Circuits and Sys- tems for Video Technology, 2025. 1

    2025

  4. [11]

    Training an open-vocabulary monocular 3d detection model without 3d data

    Rui Huang, Henry Zheng, Yan Wang, Zhuofan Xia, Marco Pavone, and Gao Huang. Training an open-vocabulary monocular 3d detection model without 3d data. Advances in Neural Information Processing Systems, 37:72145–72169,

  5. [12]

    Particle filter with swarm move for optimiza- tion

    Chunlin Ji, Yangyang Zhang, Mengmeng Tong, and Shengx- iang Yang. Particle filter with swarm move for optimiza- tion. In International Conference on Parallel Problem Solv- ing from Nature, pages 909–918. Springer, 2008. 6

    2008

  6. [13]

    Open-vocabulary 3d semantic segmentation with foundation models

    Li Jiang, Shaoshuai Shi, and Bernt Schiele. Open-vocabulary 3d semantic segmentation with foundation models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21284–21294, 2024. 1

    2024

  7. [14]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2, 3

    2023

  8. [15]

    Pycuda and pyopencl: A scripting-based approach to gpu run-time code generation

    Andreas Kl ¨ockner, Nicolas Pinto, Yunsup Lee, Bryan Catan- zaro, Paul Ivanov, and Ahmed Fasih. Pycuda and pyopencl: A scripting-based approach to gpu run-time code generation. Parallel computing, 38(3):157–174, 2012. 6

    2012

  9. [16]

    Open3dsg: Open- vocabulary 3d scene graphs from point clouds with queryable objects and open-set relationships

    Sebastian Koch, Narunas Vaskevicius, Mirco Colosi, Pe- dro Hermosilla, and Timo Ropinski. Open3dsg: Open- vocabulary 3d scene graphs from point clouds with queryable objects and open-set relationships. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2024

  10. [17]

    Arm3d: Attention-based re- lation module for indoor 3d object detection

    Yuqing Lan, Yao Duan, Chenyi Liu, Chenyang Zhu, Yueshan Xiong, Hui Huang, and Kai Xu. Arm3d: Attention-based re- lation module for indoor 3d object detection. Computational Visual Media, 8(3):395–414, 2022. 2

    2022

  11. [18]

    Remixfusion: Residual-based mixed representation for large-scale online rgb-d reconstruction

    Yuqing Lan, Chenyang Zhu, Shuaifeng Zhi, Jiazhao Zhang, Zhoufeng Wang, Renjiao Yi, Yijie Wang, and Kai Xu. Remixfusion: Residual-based mixed representation for large-scale online rgb-d reconstruction. arXiv preprint arXiv:2507.17594, 2025. 6

    2025

  12. [20]

    Patch- work++: Fast and robust ground segmentation solving par- tial under-segmentation using 3D point cloud

    Seungjae Lee, Hyungtae Lim, and Hyun Myung. Patch- work++: Fast and robust ground segmentation solving par- tial under-segmentation using 3D point cloud. In Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. , pages 13276– 13283, 2022. 6

    2022

  13. [21]

    Ground- ing image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and J´erˆome Revaud. Ground- ing image matching in 3d with mast3r. In European Confer- ence on Computer Vision, pages 71–91. Springer, 2024. 5

    2024

  14. [22]

    Grass: Generative recursive autoencoders for shape structures

    Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. Grass: Generative recursive autoencoders for shape structures. ACM Transactions on Graphics (TOG), 36(4):1–14, 2017. 1

    2017

  15. [23]

    Prompting depth anything for 4k resolution accurate metric depth estimation

    Haotong Lin, Sida Peng, Jingxiao Chen, Songyou Peng, Ji- aming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou, and Bingyi Kang. Prompting depth anything for 4k resolution accurate metric depth estimation. 2024. 1

    2024

  16. [24]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceeding...

    2014

  17. [25]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2024

  18. [26]

    Group-free 3d object detection via transformers

    Ze Liu, Zheng Zhang, Yue Cao, Han Hu, and Xin Tong. Group-free 3d object detection via transformers. InProceed- ings of the IEEE/CVF international conference on computer vision, pages 2949–2958, 2021. 2, 4

    2021

  19. [27]

    Open-vocabulary point-cloud object detection without 3d an- notation

    Yuheng Lu, Chenfeng Xu, Xiaobao Wei, Xiaodong Xie, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. Open-vocabulary point-cloud object detection without 3d an- notation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1190–1199,

  20. [28]

    Oa-cnns: Omni- adaptive sparse cnns for 3d semantic segmentation

    Bohao Peng, Xiaoyang Wu, Li Jiang, Yukang Chen, Heng- shuang Zhao, Zhuotao Tian, and Jiaya Jia. Oa-cnns: Omni- adaptive sparse cnns for 3d semantic segmentation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21305–21315, 2024. 1

    2024

  21. [29]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660,

  22. [30]

    Deep hough voting for 3d object detection in point clouds

    Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas. Deep hough voting for 3d object detection in point clouds. In proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 9277–9286, 2019. 1, 2, 4, 6

    2019

  23. [31]

    Imvotenet: Boosting 3d object detection in point clouds with image votes

    Charles R Qi, Xinlei Chen, Or Litany, and Leonidas J Guibas. Imvotenet: Boosting 3d object detection in point clouds with image votes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4404–4413, 2020. 2

    2020

  24. [32]

    High quality entity segmentation

    Lu Qi, Jason Kuen, Tiancheng Shen, Jiuxiang Gu, Wenbo Li, Weidong Guo, Jiaya Jia, Zhe Lin, and Ming-Hsuan Yang. High quality entity segmentation. In 2023 IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 4024–4033. IEEE, 2023. 3

    2023

  25. [33]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  26. [34]

    Fcaf3d: Fully convolutional anchor-free 3d object detection

    Danila Rukhovich, Anna V orontsova, and Anton Konushin. Fcaf3d: Fully convolutional anchor-free 3d object detection. In European Conference on Computer Vision , pages 477–

  27. [35]

    Tr3d: Towards real-time indoor 3d object detection

    Danila Rukhovich, Anna V orontsova, and Anton Konushin. Tr3d: Towards real-time indoor 3d object detection. In 2023 IEEE International Conference on Image Processing (ICIP), pages 281–285. IEEE, 2023. 1, 2, 6, 7

    2023

  28. [36]

    Clip-fields: Weakly supervised semantic fields for robotic memory

    Nur Muhammad Mahi Shafiullah, Chris Paxton, Lerrel Pinto, Soumith Chintala, and Arthur Szlam. Clip-fields: Weakly supervised semantic fields for robotic memory. arXiv preprint arXiv:2210.05663, 2022. 3

    2022 arXiv

  29. [37]

    Mips-fusion: Multi-implicit-submaps for scalable and robust online neural rgb-d reconstruction.ACM Transactions on Graphics (TOG), 42(6):1–16, 2023

    Yijie Tang, Jiazhao Zhang, Zhinan Yu, He Wang, and Kai Xu. Mips-fusion: Multi-implicit-submaps for scalable and robust online neural rgb-d reconstruction.ACM Transactions on Graphics (TOG), 42(6):1–16, 2023. 6

    2023

  30. [38]

    Onlineanyseg: On- line zero-shot 3d segmentation by visual foundation model guided 2d mask merging

    Yijie Tang, Jiazhao Zhang, Yuqing Lan, Yulan Guo, Dezun Dong, Chenyang Zhu, and Kai Xu. Onlineanyseg: On- line zero-shot 3d segmentation by visual foundation model guided 2d mask merging. arXiv preprint arXiv:2503.01309,

    arXiv

  31. [39]

    Spatiallm: Large language model for spatial understanding

    ManyCore Research Team. Spatiallm: Large language model for spatial understanding. https://github. com/manycore-research/SpatialLM , 2025. 3, 6, 7, 8

    2025

  32. [40]

    Appa- 3d: an autonomous 3d path planning algorithm for uavs in unknown complex environments

    Jintao Wang, Zuyi Zhao, Jiayi Qu, and Xingguo Chen. Appa- 3d: an autonomous 3d path planning algorithm for uavs in unknown complex environments. Scientific Reports, 14(1): 1231, 2024. 1

    2024

  33. [41]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697– 20709, 2024. 5

    2024

  34. [42]

    Ov-uni3detr: Towards unified open- vocabulary 3d object detection via cycle-modality propaga- tion

    Zhenyu Wang, Yali Li, Taichi Liu, Hengshuang Zhao, and Shengjin Wang. Ov-uni3detr: Towards unified open- vocabulary 3d object detection via cycle-modality propaga- tion. In European Conference on Computer Vision , pages 73–89. Springer, 2024. 2

    2024

  35. [43]

    Mlcvnet: Multi-level con- text votenet for 3d object detection

    Qian Xie, Yu-Kun Lai, Jing Wu, Zhoutao Wang, Yiming Zhang, Kai Xu, and Jun Wang. Mlcvnet: Multi-level con- text votenet for 3d object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10447–10456, 2020. 2

    2020

  36. [44]

    Embodiedsam: Online segment any 3d thing in real time

    Xiuwei Xu, Huangxing Chen, Linqing Zhao, Ziwei Wang, Jie Zhou, and Jiwen Lu. Embodiedsam: Online segment any 3d thing in real time. arXiv preprint arXiv:2408.11811,

    arXiv

  37. [45]

    Memory-based adapters for online 3d scene perception

    Xiuwei Xu, Chong Xia, Ziwei Wang, Linqing Zhao, Yueqi Duan, Jie Zhou, and Jiwen Lu. Memory-based adapters for online 3d scene perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 21604–21613, 2024. 2, 3

    2024

  38. [46]

    M 2 diffuser: Diffusion-based trajectory optimization for mobile manipulation in 3d scenes

    Sixu Yan, Zeyu Zhang, Muzhi Han, Zaijin Wang, Qi Xie, Zhitian Li, Zhehan Li, Hangxin Liu, Xinggang Wang, and Song-Chun Zhu. M 2 diffuser: Diffusion-based trajectory optimization for mobile manipulation in 3d scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence,

  39. [47]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 1

    2024

  40. [48]

    Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu. Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023. 3

    2023 arXiv

  41. [49]

    Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation

    Hang Yin, Xiuwei Xu, Zhenyu Wu, Jie Zhou, and Jiwen Lu. Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation. Advances in Neural Information Processing Systems, 37:5285–5307, 2024. 1

    2024

  42. [50]

    Detect anything 3d in the wild

    Hanxue Zhang, Haoran Jiang, Qingsong Yao, Yanan Sun, Renrui Zhang, Hao Zhao, Hongyang Li, Hongzi Zhu, and Zetong Yang. Detect anything 3d in the wild. arXiv preprint arXiv:2504.07958, 2025. 3

    2025

  43. [51]

    Rosefusion: random optimization for online dense recon- struction under fast camera motion

    Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, and Kai Xu. Rosefusion: random optimization for online dense recon- struction under fast camera motion. ACM Transactions on Graphics (TOG), 40(4):1–17, 2021. 4, 6

    2021

  44. [52]

    Asro- dio: Active subspace random optimization based depth iner- tial odometry

    Jiazhao Zhang, Yijie Tang, He Wang, and Kai Xu. Asro- dio: Active subspace random optimization based depth iner- tial odometry. IEEE Transactions on Robotics, 39(2):1496– 1508, 2022. 4

    2022

  45. [53]

    Gamma: Graspability-aware mobile manipulation policy learning based on online grasping pose fusion

    Jiazhao Zhang, Nandiraju Gireesh, Jilong Wang, Xiaomeng Fang, Chaoyi Xu, Weiguang Chen, Liu Dai, and He Wang. Gamma: Graspability-aware mobile manipulation policy learning based on online grasping pose fusion. In 2024 IEEE International Conference on Robotics and Automation (I...

    2024

  46. [54]

    Navgpt: Explicit reasoning in vision-and-language navigation with large lan- guage models

    Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-and-language navigation with large lan- guage models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7641–7649, 2024. 1

    2024

  47. [55]

    V oxelnet: End-to-end learning for point cloud based 3d object detection

    Yin Zhou and Oncel Tuzel. V oxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 4490–4499, 2018. 1, 2

    2018

  48. [493]

    1, 2, 6, 7, 8

    Springer, 2022. 1, 2, 6, 7, 8

    2022

  49. [2025]

    2, 3, 4, 6, 7, 8, 10

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.