Pith. sign in

REVIEW 4 major objections 5 minor 65 references

OpenM3D is the first multi-view open-vocabulary 3D object detector trained without human annotations, using graph-embedding pseudo boxes and CLIP-aligned voxel features to detect and classify objects from RGB images alone at 0.3 seconds per

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A single-stage image-based detector that, trained with pseudo boxes from SAM segments and CLIP features, detects and classifies arbitrary indoor objects in 3D at 0.3 seconds per scene.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The pseudo-box pipeline and efficiency are real; the accuracy-over-SoTA claim is not yet supported by the comparisons in the paper. the 4 major comments →

arxiv 2508.20063 v1 pith:U4XWGHQA submitted 2025-08-27 cs.CV

OpenM3D: Open Vocabulary Multi-view Indoor 3D Object Detection without Human Annotations

Peng-Hao Hsu , Ke Zhang , Fu-En Wang , Tao Tu , Ming-Feng Li , Yu-Lun Liu , Albert Y. C. Chen , Min Sun
show 1 more author
Cheng-Hao Kuo
This is my paper
classification cs.CV
keywords open-vocabulary 3D object detectionmulti-view imagespseudo label generationgraph embeddingCLIP alignmentsingle-stage detectorindoor scenesScanNet200
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that open-vocabulary 3D object detection in indoor scenes can be done without human annotations and without needing depth or point clouds at inference. It introduces OpenM3D, a single-stage detector built on image-induced voxel features, trained with class-agnostic 3D boxes and a voxel-semantic alignment loss. The 3D boxes come from a new pseudo-label pipeline that clusters per-view 2D segments across all viewpoints using graph embeddings, achieving higher precision and recall than existing pseudo boxes from OV-3DET and SAM3D. At inference, OpenM3D takes only multi-view RGB images and matches voxel features against text embeddings, reaching 0.3 seconds per scene and beating two-stage baselines on ScanNet200 and ARKitScenes.

Core claim

The central claim is that a single-stage multi-view image-based detector can perform open-vocabulary 3D detection without human annotations and without 3D sensors at test time, and that it beats stronger-looking two-stage pipelines in both accuracy and speed. The paper constructs this detector by (1) generating class-agnostic 3D pseudo boxes through graph-embedding-based clustering of SAM segments lifted into 3D, refined by ground-truth mesh segmentation, and (2) aligning the detector's voxel features with CLIP segment features during training, so that the same features localize objects and classify them against arbitrary text prompts. On ScanNet200 and ARKitScenes, the resulting pseudo boxe

What carries the argument

The graph embedding-based 3D pseudo box generation: each 2D segment (from SAM) is a node, lifted to a partial 3D segment; edges mark overlapping segments in the same voxel; DeepWalk embeddings are clustered by K-means into complete 3D segments, and mesh segmentation refinement fuses these with ground-truth mesh segments. This supplies the class-agnostic training targets. The voxel-semantic alignment loss then pulls each voxel feature toward the CLIP embedding of the 2D segment that projects into it, enabling open-vocabulary classification at inference by cosine similarity with text embeddings.

Load-bearing premise

The training pipeline uses ground-truth scene meshes to refine pseudo-box boundaries and to standardize 3D coordinates; if such meshes are unavailable, the pseudo boxes get noisier and the reported accuracy gains could shrink.

What would settle it

Run the full training on ScanNet200 with the mesh-based refinement (MSR and coordinates standardization) removed or replaced by a noisy reconstructed mesh, and compare AP@25 against the reported 26.92 and against the 19.53 obtained when training on OV-3DET boxes. If the gap largely disappears, the result depends on ground-truth mesh access rather than on the graph-embedding clustering itself.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Indoor 3D detection no longer requires depth sensors or point-cloud processing at inference; only multi-view RGB images and camera poses are needed.
  • Open-vocabulary classification can be folded into a single pass of the detector, eliminating the costly second-stage CLIP image encoder used by two-stage baselines.
  • The pseudo-box pipeline provides a general recipe for turning noisy per-view segmentation into coherent 3D object proposals without human labels.
  • The method's speed (0.3 seconds per scene) makes real-time open-vocabulary 3D perception feasible for robotics and augmented reality.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The training pipeline still depends on high-quality ground-truth meshes (for mesh segmentation refinement and coordinate standardization), so the 'no human annotations' claim really means 'no human 3D boxes or class labels' given dataset meshes; removing this mesh dependence is a natural next step.
  • The method's gains over SAM3D come partly from considering all viewpoints jointly; extending the graph-embedding idea to temporal or multi-session data could help with long-horizon 3D aggregation.
  • Because the detector's open-vocabulary performance tracks CLIP's semantic granularity, better text-image alignment or a different alignment target could yield larger improvements than better pseudo boxes.
  • A direct testable extension is replacing ground-truth meshes with an off-the-shelf multi-view reconstruction during training, which would separate the contribution of the mesh from the contribution of the graph clustering.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. OpenM3D proposes a single-stage, multi-view image-based 3D object detector for open-vocabulary indoor detection. Training uses no human box/class annotations: SAM 2D segments are lifted to partial 3D segments, merged across viewpoints through a graph-embedding/K-means pipeline, optionally refined with mesh-based graph-cut segments, and converted to axis-aligned 3D pseudo boxes. The detector is trained with a class-agnostic localization loss on these boxes and a voxel-semantic alignment loss that matches voxel features to CLIP embeddings from the 2D segments. At inference only multi-view RGB images and camera poses are needed; classification is performed by matching average voxel features in each box to CLIP text embeddings. Experiments on ScanNet200 and ARKitScenes report higher pseudo-box precision/recall than OV-3DET and SAM3D, better class-agnostic AP, and high speed (0.3 s per scene).

Significance. If the results hold, the paper makes a useful contribution by removing 3D sensors at inference and human box annotations at training, while retaining a fast single-stage pipeline. The pseudo-box generation study is the strongest part: it shows clear precision gains (e.g., 32.07% vs 14.48% for SAM3D at IoU@0.25 on ScanNet200) and consistent class-agnostic detector improvements. The paper is also transparent in treating pseudo-box evaluation on the training set as a diagnostic rather than as evidence of test-set circularity. However, the headline accuracy claim against existing OV detectors is not yet empirically established: the main ScanNet200 benchmark lacks end-to-end comparisons with published OV detectors, and the ScanNetv2 comparison in Table 4 is asymmetric. The training-time dependence on ground-truth meshes is also not disclosed and ablated. These are fixable with additional experiments or a more guarded claim, but they are load-bearing for the paper's central message.

major comments (4)
  1. [§5.4, Table 4] The claim that OpenM3D is 'comparable' to or better than prior point-cloud-based OV detectors is not supported by the evidence as presented. The footnote says † methods were evaluated with OV-3DET's pseudo-boxes, while OpenM3D's evaluation uses ground-truth 3D boxes from ScanNetv2 in its multi-view setting. That is an asymmetric protocol: the prior numbers reflect pseudo-box errors, while OpenM3D's numbers reflect GT-box supervision at evaluation. Moreover, Table 3 on the main ScanNet200 benchmark only includes OpenM3D trained on different pseudo boxes and the self-constructed S2D baselines. Please add end-to-end results for OV-3DET, CoDA, and ImOV3D on the same ScanNet200 split and with the same GT-box evaluation, or substantially soften the 'superior accuracy' claim.
  2. [§3 'Mesh Segmentation Refinement' and App. B.1 'Coordinates Standardization'] The training protocol uses the ground-truth mesh of each training scene in two places: (i) partial 3D segments are fused to mesh vertices by voxelization + KNN, and (ii) mesh-based graph-cut segments are used to re-label and refine the image-derived complete 3D segments. This is an extra dataset input beyond the announced 'posed RGB-D images are given' setting and likely contributes to the pseudo-box quality. The weaker MSR gains on ARKitScenes suggest mesh-quality sensitivity. Please ablate the pipeline without mesh inputs and state this dependence explicitly in the abstract/contributions; otherwise the 'without human annotations' framing overstates the method's transferability to mesh-free RGB-D scans.
  3. [§5.4, Tables 2-3] The ScanNet200 open-vocabulary comparison is limited to (i) OpenM3D retrained on OV-3DET/SAM3D pseudo boxes and (ii) S2D, which reuses OpenM3D's class-agnostic detector. These are useful ablations, but they do not establish superiority over existing end-to-end OV 3D detectors. The margins among the main rows are small (4.23 vs 4.04 vs 3.92 mAP@25), and with no error bars it is unclear whether these differences are meaningful. Please add published OV detectors to the same benchmark or narrow the paper's claims to pseudo-box quality, speed, and comparisons against the constructed baselines.
  4. [Tables 1-4, 8-10] All reported numbers are single-run point estimates. Given the stochastic components (DeepWalk, K-means, detector training) and the small margins in the main tables, the significance of the improvements cannot be assessed. Please report mean±std over at least 3 seeds for the key comparisons, or provide a statistical test.
minor comments (5)
  1. [§5.4] The text says S2D shows a drop in mAR@25 'from 0.15 to 0.10', but Table 3 reports 15.12 and 10.05 (percentages). Please fix the units/notation.
  2. [§5.1 vs §5.4] S2D is described as '7 times slower' in §5.1 and 'sixfold' in §5.4. Make the factor consistent (Table 7 implies 7×).
  3. [Eq. (2)-(3)] The overlap measure uses the smaller segment in the denominator (i.e., IoU with min rather than union). Please clarify whether θ was tuned and report sensitivity to θ and to the K-means cluster count K=100.
  4. [Abstract/Introduction] The 'first multi-view open-vocabulary 3D object detector' claim needs qualification: related work includes LeRF (image-only, but requiring per-scene reconstruction at inference) and ImOV3D (trained from images, point cloud at inference). Define the comparison boundary explicitly.
  5. [Limitation (§C)] The limitation section only discusses CLIP classification. The ground-truth mesh dependence and the absence of end-to-end comparisons with existing OV detectors should be acknowledged there as well.

Circularity Check

0 steps flagged

No significant circularity: the training/inference chain is self-contained and the detector is evaluated on held-out scenes; the main caveats are reliance on the authors' own ImGeoNet backbone and self-constructed baselines, which are evidence concerns rather than circular derivation.

full rationale

The paper's derivation chain is not circular. Pseudo-boxes are generated by lifting SAM 2D segments to 3D, building a graph over overlapping partial segments, embedding with DeepWalk, clustering with K-means, and refining with mesh-based segments; the class-agnostic detector is then trained on those pseudo-boxes and the voxel-semantic alignment loss. The detector is evaluated on the held-out ScanNet200/ARKitScenes test splits, while the pseudo-box quality is assessed on the training set as a diagnostic, so no test labels leak into the pseudo-box generation or the detector training. The voxel-semantic alignment trains voxel features toward CLIP image embeddings during training, and inference matches average voxel features to CLIP text embeddings; this is standard cross-modal alignment, not a self-definitional loop, because the CLIP text and image encoders are pre-trained externally and are not fitted to the test set. The use of the ground-truth mesh in coordinates standardization and mesh segmentation refinement is a training-time geometric prior; it may weaken the 'without human annotations' framing, but it does not make the test-time predictions equal to the training inputs. The comparison against S2D and the depth-estimation baseline is an ablation that shares the same class-agnostic detector with OpenM3D, so it does not constitute an independent benchmark, and Table 4's footnote reveals an asymmetric evaluation against OV-3DET/CoDA/ImOV3D; these are limitations in empirical support, not circular derivations. The admitted self-citation to ImGeoNet [50] is disclosed and the architecture was independently published and evaluated, so it is not load-bearing circularity. Overall, no equation reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The central pipeline introduces no new physical entities. It does introduce several hand-chosen hyperparameters that directly control pseudo-box quality (edge threshold, cluster count, size filters, vocabulary substitution), and it relies on the availability of a reconstructed mesh and multiple off-the-shelf models whose errors propagate into the training signal.

free parameters (5)
  • edge overlap threshold θ (Eq. 2) = not specified
    Threshold for placing an edge between partial 3D segments; chosen by hand, no value or sensitivity analysis given.
  • K-means cluster count K = 100 for all scenes
    Fixed number of clusters for grouping partial segments into complete 3D segments; scene-independent.
  • minimum point counts for boxes = 300 points (ScanNet200), 500 points (ARKitScenes)
    Heuristic filters to remove small/less visible boxes; affects recall and precision.
  • maximum box volume = 8.5 m^3
    Heuristic to discard boxes unlikely to represent whole objects.
  • CLIP segment prompt vocabulary = 364 categories from OV-3DET
    Each 2D segment's CLIP embedding is replaced with the closest category text embedding, which injects a fixed vocabulary into training features.
axioms (4)
  • domain assumption Ground-truth reconstructed mesh is available for training and used for coordinate standardization and mesh segmentation refinement (Sec. 3, B.1).
    The pseudo-box pipeline depends on the dataset mesh; without it, coordinate standardization and refinement cannot be applied.
  • domain assumption Pre-trained models (SAM, CLIP, Mask DINO, DeepWalk) are reliable for their roles (Sec. 3-4).
    No re-training or domain adaptation of these components; errors propagate into pseudo boxes.
  • domain assumption Posed RGB-D training images have accurate depth and camera poses (Sec. 3).
    Back-projection (Eq. 1) assumes depth and pose accuracy; noisy depth would degrade clustering and alignment.
  • domain assumption CLIP feature space is an adequate semantic space for open-vocabulary classification (Sec. 4.2).
    The method inherits CLIP's biases and limitations; the paper's limitation section acknowledges confusion among semantically similar objects.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of OpenM3D: Open Vocabulary Multi-view Indoor 3D Object Detection without Human Annotations." pith.science (2026). https://pith.science/paper/U4XWGHQA

@misc{pith2026250820063,
  author       = {Pith},
  title        = {Pith review of: OpenM3D: Open Vocabulary Multi-view Indoor 3D Object Detection without Human Annotations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U4XWGHQA}},
  note         = {Machine review of arXiv:2508.20063}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Open-vocabulary (OV) 3D object detection is an emerging field, yet its exploration through image-based methods remains limited compared to 3D point cloud-based methods. We introduce OpenM3D, a novel open-vocabulary multi-view indoor 3D object detector trained without human annotations. In particular, OpenM3D is a single-stage detector adapting the 2D-induced voxel features from the ImGeoNet model. To support OV, it is jointly trained with a class-agnostic 3D localization loss requiring high-quality 3D pseudo boxes and a voxel-semantic alignment loss requiring diverse pre-trained CLIP features. We follow the training setting of OV-3DET where posed RGB-D images are given but no human annotations of 3D boxes or classes are available. We propose a 3D Pseudo Box Generation method using a graph embedding technique that combines 2D segments into coherent 3D structures. Our pseudo-boxes achieve higher precision and recall than other methods, including the method proposed in OV-3DET. We further sample diverse CLIP features from 2D segments associated with each coherent 3D structure to align with the corresponding voxel feature. The key to training a highly accurate single-stage detector requires both losses to be learned toward high-quality targets. At inference, OpenM3D, a highly efficient detector, requires only multi-view images for input and demonstrates superior accuracy and speed (0.3 sec. per scene) on ScanNet200 and ARKitScenes indoor benchmarks compared to existing methods. We outperform a strong two-stage method that leverages our class-agnostic detector with a ViT CLIP-based OV classifier and a baseline incorporating multi-view depth estimator on both accuracy and speed.

Figures

Figures reproduced from arXiv: 2508.20063 by Albert Y. C. Chen, Cheng-Hao Kuo, Fu-En Wang, Ke Zhang, Ming-Feng Li, Min Sun, Peng-Hao Hsu, Tao Tu, Yu-Lun Liu.

Figure 1
Figure 1. Figure 1: Graph Embedding-Based 3D Pseudo Box Generation. Given multi-view images, we utilize an off-the-shelf class-agnostic 2D instance segmentation approach to produce 2D segments S 2D = {n 2D j } (see color-coded 2D masks in (a)). Each 2D segment is then lifted in 3D to form a partial 3D segment n 3D j following Eq. 1 (see (b)). Next, we build a graph in which each partial segment n 3D j is a node and we determi… view at source ↗
Figure 2
Figure 2. Figure 2: Overview. OpenM3D learns class-agnostic 3D box prediction and open-vocabulary (OV) assignments during training and only needs multi-view RGB images to infer OV 3D boxes. The bottom branch is our class-agnostic 3D object detector (Sec. 4.1), where we build the 3D voxel features based on ImGeoNet. During training, given a set of RGBD images and their corresponding poses, we back-project 2D features from imag… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative Results of OpenM3D on ScanNet200 and ARKitScenes. Given multi-view images and corresponding cam￾era poses, OpenM3D can detect objects by arbitrary text prompts towards open-vocabulary detection. The color-coded boxes corre￾spond to different object classes. We show a subset of text prompts used in the ImageNet dataset and specific prompts [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: mAP25 and mAR25 in various class numbers from 18 to 189 classes. As the class number gets larger, OpenM3D scores consistently higher mAP (Left) and mAR (Right) compared with the “Strong Two-stage” method. datasets depending on the dataset domains. OpenM3D pre￾dicts classes by matching the 3D voxel feature with the text embeddings of class names wrapped in multiple prompts, such as “A photo of a {}”, and se… view at source ↗
Figure 5
Figure 5. Figure 5: Visualize Pseudo Boxes of OpenM3D on ScanNet200. We visualize our 3D pseudo boxes using two different volume sizes (small and medium). In this visualization, cyan represents false positives, while magenta represents true positives matching the GT boxes at IoU@0.25. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Localizing Novel Object with Pseudo Box on ScanNet200. Ground Truth OpenM3D Ground Truth OpenM3D Text Prompt ‘Classify this image, is it an large indoor {shelf}.' 'Label this image with the indoor {pillow}.' 'a photo of a nice {door}.' 'a photo of many{office chair}.' 'a photo of a my {backpack}.' 'a photo of a clean {chair}.' 'Identify and label the indoor {door} in this image.' 'Classify this image as de… view at source ↗
Figure 7
Figure 7. Figure 7: More Qualitative Results of OpenM3D on ScanNet200. We show general text prompts used in the ImageNet dataset, as well as prompts from specific text. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Novel and tail predictions in OpenM3D [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: 3R baseline qualitative result. Comparison between (left) ground-truth ScanNet scene, (middle) VGGT 3D reconstruc￾tion using only RGB images and poses, and (right) OVIR-3D seg￾mentation result on the VGGT output. The reconstruction lacks fine-grained indoor geometry, resulting in inaccurate 2D–3D match￾ing and degraded segmentation quality. tion (81 s). Unlike others, it avoids costly CLIP inference and de… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 61 canonical work pages · 1 internal anchor

  1. [1]

    Multi-view depth estimation by fusing single-view depth probability with multi-view geometry

    Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Multi-view depth estimation by fusing single-view depth probability with multi-view geometry. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 6

    work page 2022

  2. [2]

    Arkitscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Bran- don Joffe, Daniel Kurz, Arik Schwartz, and Elad Shul- man. Arkitscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. In Advances in Neural Information Processing Systems (NeurIPS), 2021. 2, 6

    work page 2021

  3. [3]

    Coda: Collaborative novel box discovery and cross-modal alignment for open-vocabulary 3d object detection

    Yang Cao, Yihan Zeng, Hang Xu, and Dan Xu. Coda: Collaborative novel box discovery and cross-modal alignment for open-vocabulary 3d object detection. In Advances in Neural Information Processing Systems (NeurIPS), 2023. 3, 7, 16

    work page 2023

  4. [4]

    End-to-end object detection with trans- formers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with trans- formers. In European Conference on Computer Vision (ECCV), 2020. 2

    work page 2020

  5. [5]

    Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 2, 5, 12, 16

    work page 2017

  6. [6]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009. 7, 8

    work page 2009

  7. [7]

    V oxel r-cnn: To- wards high performance voxel-based 3d object detec- tion

    Jiajun Deng, Shaoshuai Shi, Peiwei Li, Wengang Zhou, Yanyong Zhang, and Houqiang Li. V oxel r-cnn: To- wards high performance voxel-based 3d object detec- tion. In AAAI Conference on Artificial Intelligence (AAAI), 2021. 2

    work page 2021

  8. [8]

    Effi- cient graph-based image segmentation

    Pedro F Felzenszwalb and Daniel P Huttenlocher. Effi- cient graph-based image segmentation. International Journal of Computer Vision (IJCV), 59, 2004. 4

    work page 2004

  9. [10]

    Scaling open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scaling open-vocabulary image segmentation with image-level labels. In European Conference on Com- puter Vision (ECCV), 2022. 1

    work page 2022

  10. [11]

    Open-vocabulary object detection via vision and lan- guage knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and lan- guage knowledge distillation. In International Con- ference on Learning Representations (ICLR), 2022. 1, 2

    work page 2022

  11. [12]

    Generative sparse detection networks for 3d single-shot object detection

    JunYoung Gwak, Christopher Choy, and Silvio Savarese. Generative sparse detection networks for 3d single-shot object detection. In European Confer- ence on Computer Vision (ECCV), 2020. 2

    work page 2020

  12. [13]

    Semantic abstraction: Open- world 3d scene understanding from 2d vision-language models

    Huy Ha and Shuran Song. Semantic abstraction: Open- world 3d scene understanding from 2d vision-language models. In Conference on Robot Learning (CoRL) ,

    work page

  13. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 7

    work page 2016

  14. [15]

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view

    Junjie Huang, Guan Huang, Zheng Zhu, and Da- long Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view. arXiv preprint arXiv:2112.11790, 2021. 2

    Pith/arXiv arXiv 2021

  15. [16]

    Tenenbaum, Celso Miguel de Melo, Madhava Krishna, Liam Paull, Florian Shkurti, and An- tonio Torralba

    Krishna Murthy Jatavallabhula, Alihusein Kuwajer- wala, Qiao Gu, Mohd Omama, Tao Chen, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, Ayush Tewari, Joshua B. Tenenbaum, Celso Miguel de Melo, Madhava Krishna, Liam Paull, Florian Shkurti, and An- tonio Torralba. Conceptfusion: Open-set multimodal 3d mapping. In Robotics: Science and Systems (RSS),

    work page

  16. [17]

    Scaling up visual and vision- language representation learning with noisy text su- pervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision- language representation learning with noisy text su- pervision. In International Conference on Machine Learning (ICML), 2021. 1

    work page 2021

  17. [18]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 19729–19739, 2023. 3

    work page 2023

  18. [19]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. In IEEE International Conference on Computer Vision (ICCV), 2023. 1, 6, 12, 15, 16

    work page 2023

  19. [20]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 8

    work page 2009

  20. [21]

    Pointpillars: Fast encoders for object detection from point clouds

    Alex H Lang, Sourabh V ora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2

    work page 2019

  21. [22]

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation

    Feng Li, Hao Zhang, Huaizhe Xu, Shilong Liu, Lei Zhang, Lionel M Ni, and Heung-Yeung Shum. Mask dino: Towards a unified transformer-based framework for object detection and segmentation. In IEEE Con- 9 ference on Computer Vision and Pattern Recognition (CVPR), 2023. 6

    work page 2023

  22. [23]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal trans- formers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal trans- formers. In European Conference on Computer Vision (ECCV), 2022. 2

    work page 2022

  23. [24]

    Focal loss for dense object detec- tion

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detec- tion. In IEEE International Conference on Computer Vision (ICCV), 2017. 5

    work page 2017

  24. [25]

    Petr: Position embedding transformation for multi-view 3d object detection

    Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection. In European Confer- ence on Computer Vision (ECCV), 2022. 2

    work page 2022

  25. [26]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019. 7

    work page 2019

  26. [27]

    High-quality entity segmentation

    Qi Lu, Jason Kuen, Shen Tiancheng, Gu Jiuxiang, Guo Weidong, Jia Jiaya, Lin Zhe, and Yang Ming-Hsuan. High-quality entity segmentation. In IEEE Interna- tional Conference on Computer Vision (ICCV), 2023. 8, 15, 16

    work page 2023

  27. [28]

    Ovir-3d: Open- vocabulary 3d instance retrieval without training on 3d data

    Shiyang Lu, Haonan Chang, Eric Pu Jing, Abdeslam Boularias, and Kostas Bekris. Ovir-3d: Open- vocabulary 3d instance retrieval without training on 3d data. In 7th Annual Conference on Robot Learning,

    work page

  28. [29]

    Open-vocabulary point-cloud object detection without 3d annotation

    Yuheng Lu, Chenfeng Xu, Xiaobao Wei, Xiaodong Xie, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. Open-vocabulary point-cloud object detection without 3d annotation. In IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2023. 1, 2, 3, 5, 6, 7, 12, 15, 16

    work page 2023

  29. [30]

    V oxel transformer for 3d object detection

    Jiageng Mao, Yujing Xue, Minzhe Niu, Haoyue Bai, Jiashi Feng, Xiaodan Liang, Hang Xu, and Chunjing Xu. V oxel transformer for 3d object detection. InIEEE International Conference on Computer Vision (ICCV),

    work page

  30. [31]

    Atlas: End-to-end 3d scene reconstruction from posed images

    Zak Murez, Tarrence Van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, and Andrew Rabinovich. Atlas: End-to-end 3d scene reconstruction from posed images. In European Conference on Computer Vision (ECCV), 2020. 2

    work page 2020

  31. [32]

    3d object detection with pointformer

    Xuran Pan, Zhuofan Xia, Shiji Song, Li Erran Li, and Gao Huang. 3d object detection with pointformer. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 2

    work page 2021

  32. [33]

    Openscene: 3d scene understanding with open vocabularies

    Songyou Peng, Kyle Genova, Chiyu ”Max” Jiang, Andrea Tagliasacchi, Marc Pollefeys, and Thomas Funkhouser. Openscene: 3d scene understanding with open vocabularies. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1, 3

    work page 2023

  33. [34]

    Deepwalk: Online learning of social representations

    Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 701–710, 2014. 6

    work page 2014

  34. [35]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems (NeurIPS), 2017. 2

    work page 2017

  35. [36]

    Frustum pointnets for 3d object detection from rgb-d data

    Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from rgb-d data. InIEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2018

    work page 2018

  36. [37]

    Deep hough voting for 3d object detection in point clouds

    Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas. Deep hough voting for 3d object detection in point clouds. In IEEE International Conference on Computer Vision (ICCV), 2019. 2, 5

    work page 2019

  37. [38]

    Learning transferable vi- sual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable vi- sual models from natural language supervision. In In- ternational Conference on Machine Learning (ICML),

    work page

  38. [39]

    Im- proved visual-semantic alignment for zero-shot object detection

    Shafin Rahman, Salman Khan, and Nick Barnes. Im- proved visual-semantic alignment for zero-shot object detection. In AAAI Conference on Artificial Intelligence (AAAI), 2020. 2

    work page 2020

  39. [40]

    Language-grounded indoor 3d semantic segmentation in the wild

    David Rozenberszki, Or Litany, and Angela Dai. Language-grounded indoor 3d semantic segmentation in the wild. In European Conference on Computer Vision (ECCV), 2022. 2, 6

    work page 2022

  40. [41]

    Fcaf3d: fully convolutional anchor-free 3d object detection

    Danila Rukhovich, Anna V orontsova, and Anton Konushin. Fcaf3d: fully convolutional anchor-free 3d object detection. In European Conference on Computer Vision (ECCV), 2022. 2

    work page 2022

  41. [42]

    Imvoxelnet: Image to voxels projection for monocular and multi-view general-purpose 3d ob- ject detection

    Danila Rukhovich, Anna V orontsova, and Anton Konushin. Imvoxelnet: Image to voxels projection for monocular and multi-view general-purpose 3d ob- ject detection. In Winter Conference on Applications of Computer Vision (WACV), 2022. 1, 2, 4

    work page 2022

  42. [43]

    Pointrcnn: 3d object proposal generation and detection from point cloud

    Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrcnn: 3d object proposal generation and detection from point cloud. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2

    work page 2019

  43. [44]

    From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network

    Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network. IEEE Transactions on Pat- tern Analysis and Machine Intelligence (TPAMI), 2020. 2 10

    work page 2020

  44. [45]

    Point-gnn: Graph neural network for 3d object detection in a point cloud

    Weijing Shi and Raj Rajkumar. Point-gnn: Graph neural network for 3d object detection in a point cloud. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 2

    work page 2020

  45. [46]

    Sumner, Marc Pollefeys, Federico Tombari, and Francis Engel- mann

    Ayc ¸a Takmaz, Elisabetta Fedele, Robert W. Sumner, Marc Pollefeys, Federico Tombari, and Francis Engel- mann. Openmask3d: Open-vocabulary 3d instance segmentation. In Advances in Neural Information Pro- cessing Systems (NeurIPS), 2023. 1, 2, 3

    work page 2023

  46. [47]

    Mv-dust3r+: Single-stage scene reconstruc- tion from sparse views in 2 seconds

    Zhenggang Tang, Yuchen Fan, Dilin Wang, Hongyu Xu, Rakesh Ranjan, Alexander Schwing, and Zhicheng Yan. Mv-dust3r+: Single-stage scene reconstruc- tion from sparse views in 2 seconds. arXiv preprint arXiv:2412.06974, 2024. 15

    Pith/arXiv arXiv 2024

  47. [48]

    Fcos: A simple and strong anchor-free object detector

    Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: A simple and strong anchor-free object detector. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020. 4

    work page 2020

  48. [49]

    CrossDTR: Cross-view and Depth-guided Transformers for 3D Object Detection

    Ching-Yu Tseng, Yi-Rong Chen, Hsin-Ying Lee, Tsung-Han Wu, Wen-Chin Chen, and Winston Hsu. Crossdtr: Cross-view and depth-guided trans- formers for 3d object detection. arXiv preprint arXiv:2209.13507, 2022. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  49. [50]

    Imgeonet: Image-induced geometry-aware voxel repre- sentation for multi-view 3d object detection

    Tao Tu, Shun-Po Chuang, Yu-Lun Liu, Cheng Sun, Ke Zhang, Donna Roy, Cheng-Hao Kuo, and Min Sun. Imgeonet: Image-induced geometry-aware voxel repre- sentation for multi-view 3d object detection. In IEEE International Conference on Computer Vision (ICCV),

    work page

  50. [51]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, An- drea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 15

    work page 2025

  51. [52]

    Detr3d: 3d object detection from multi-view images via 3d-to-2d queries

    Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In Conference on Robot Learning (CoRL), 2022. 2

    work page 2022

  52. [53]

    Nerf-det: Learning geometry-aware volumetric representation for multi-view 3d object detection

    Chenfeng Xu, Bichen Wu, Ji Hou, Sam Tsai, Ruilong Li, Jialiang Wang, Wei Zhan, Zijian He, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka. Nerf-det: Learning geometry-aware volumetric representation for multi-view 3d object detection. In IEEE International Conference on Computer Vision (ICCV), 2023. 1, 2

    work page 2023

  53. [54]

    A simple base- line for open-vocabulary semantic segmentation with pre-trained vision-language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple base- line for open-vocabulary semantic segmentation with pre-trained vision-language model. In European Con- ference on Computer Vision (ECCV), 2022. 1

    work page 2022

  54. [55]

    Second: Sparsely embedded convolutional detection

    Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 2018. 2

    work page 2018

  55. [56]

    Pixor: Real-time 3d object detection from point clouds

    Bin Yang, Wenjie Luo, and Raquel Urtasun. Pixor: Real-time 3d object detection from point clouds. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 2

    work page 2018

  56. [57]

    Imov3d: Learn- ing open-vocabulary point clouds 3d object detection from only 2d images

    Timing Yang, Yuanliang Ju, and Li Yi. Imov3d: Learn- ing open-vocabulary point clouds 3d object detection from only 2d images. NeurIPS 2024, 2024. 3, 7

    work page 2024

  57. [58]

    Sam3d: Segment anything in 3d scenes

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu. Sam3d: Segment anything in 3d scenes. arXiv preprint arXiv:2306.03908, 2023. 2, 5, 6, 7, 12, 15, 16

    Pith/arXiv arXiv 2023

  58. [59]

    Std: Sparse-to-dense 3d object detector for point cloud

    Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Std: Sparse-to-dense 3d object detector for point cloud. In IEEE International Conference on Computer Vision (ICCV), 2019. 2

    work page 2019

  59. [60]

    3dssd: Point-based 3d single stage object detector

    Zetong Yang, Yanan Sun, Shu Liu, and Jiaya Jia. 3dssd: Point-based 3d single stage object detector. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2020. 2

    work page 2020

  60. [61]

    Open-vocabulary object detection us- ing captions

    Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih-Fu Chang. Open-vocabulary object detection us- ing captions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 2

    work page 2021

  61. [62]

    Pointclip: Point cloud understanding by clip

    Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hong- sheng Li. Pointclip: Point cloud understanding by clip. arXiv preprint arXiv:2112.02413, 2021. 2

    Pith/arXiv arXiv 2021

  62. [63]

    H3dnet: 3d object detection using hybrid ge- ometric primitives

    Zaiwei Zhang, Bo Sun, Haitao Yang, and Qixing Huang. H3dnet: 3d object detection using hybrid ge- ometric primitives. In European Conference on Com- puter Vision (ECCV), 2020. 2

    work page 2020

  63. [64]

    Iou loss for 2d/3d object detection

    Dingfu Zhou, Jin Fang, Xibin Song, Chenye Guan, Junbo Yin, Yuchao Dai, and Ruigang Yang. Iou loss for 2d/3d object detection. In International Conference on 3D Vision (3DV), 2019. 4

    work page 2019

  64. [65]

    Detecting twenty- thousand classes using image-level supervision

    Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Kr¨ahenb¨uhl, and Ishan Misra. Detecting twenty- thousand classes using image-level supervision. In European Conference on Computer Vision (ECCV) ,

    work page

  65. [66]

    V oxelnet: End-to-end learn- ing for point cloud based 3d object detection

    Yin Zhou and Oncel Tuzel. V oxelnet: End-to-end learn- ing for point cloud based 3d object detection. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2018. 2 A. Visualization This section showcases visualizations of 3D pseudo boxes generated by our method, along with additional qualitative results from OpenM3D. Visualize 3D Pseudo ...

    work page 2018

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.