Pith. sign in

REVIEW 2 major objections 7 minor 98 references

Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understanding

T0 review · 2 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MPEC's masked point-entity contrast pretrains 3D encoders to align object-level point features with language, reaching 66.0% foreground mIoU on ScanNet for open-vocabulary segmentation.

desk verdict Solid data-efficiency results and a clean ablation, but the zero-shot Matterport3D claim is undermined by HM3D training overlap. read the letter →

arxiv 2504.19500 v1 pith:7S6U3XMO submitted 2025-04-28 cs.CV cs.CL

Yan Wang , Baoxiong Jia , Ziyu Zhu , Siyuan Huang This is my paper · ORCID
classification cs.CVcs.CL
keywords open-vocabulary3Dsemanticsegmentationpoint-entitycontrastivelearningvision-languagepretrainingmaskedpointcontrastentity-to-languagealignmentzero-shottransferScanNetdata-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MPEC is a pretraining method for 3D point-cloud encoders aimed at open-vocabulary scene understanding: after training, the same encoder can label objects by arbitrary language descriptions without category-specific fine-tuning and can be fine-tuned for segmentation, grounding, captioning, and question answering. The paper argues that prior methods either align points to language through 2D images and lose 3D structure, or learn 3D contrastive features with no notion of objects, so neither yields features that both understand language and separate individual instances. MPEC adds the missing object level: it masks two views of a scene, matches points that fall in the same proposed entity mask across views, contrasts them against other entities, and then aligns each entity's merged point features with generated captions and spatial referrals. On ScanNet this yields 66.0% foreground mIoU and 81.3% foreground mAcc, and the same weights transfer to eight datasets from low-level perception to high-level reasoning.

What carries the argument

The load-bearing mechanism is masked point-entity contrast. For each scene an off-the-shelf instance segmenter proposes entity masks; two augmented views are generated with complementary random grid masking; a sparse-convolution 3D U-Net encodes both views; and a point-to-entity contrastive loss compares each point in one view with the mean feature of its corresponding entity in the other view, treating other entities and the background as negatives. A second loss, entity-to-language contrast, averages matched point features across the two views, projects them through a lightweight vision-language adapter, and aligns them with text embeddings from CLIP, a frozen text encoder pretrained on image-text pairs, using per-entity captions and spatial referrals. The text-to-entity direction uses cross-entropy while the entity-to-text direction uses binary cross-entropy, because several descriptions can refer to the same object. The combined contrastive loss trains the 3D encoder and adapter end-to-end while the text encoder stays frozen.

What would settle it

Train MPEC on the same scenes using ground-truth instance masks in place of the proposed masks and hand-verified captions in place of generated text; if the reported 66.0% foreground mIoU stays essentially unchanged, pseudo-label quality is not the mechanism, while a material change would show the result depends on the unverified training signal.

Watch

Extended reading notes

Core claim

The paper's central discovery is that entity-level contrastive pretraining, not just point-level or region-level contrast, produces 3D features that are simultaneously language-aligned and instance-discriminative. Concretely, MPEC reaches state-of-the-art open-vocabulary 3D semantic segmentation on ScanNet with 66.0% foreground mIoU and 81.3% foreground mAcc, surpassing the previous best method by 3.0 and 6.5 points respectively. It also improves zero-shot transfer on unseen scene datasets, handles long-tail categories on ScanNet200 with 10.8% foreground mIoU and 27.4% foreground mAcc, and, after fine-tuning, improves closed-set segmentation, instance segmentation, and several 3D vision-language reasoning tasks over previous pretraining baselines. The data-efficiency experiments show a large gain: with 1% of ScanNet scenes, fine-tuned semantic segmentation reaches 40.8% mIoU versus 30.7% for the strongest prior contrastive pretraining.

Load-bearing premise

The load-bearing premise is that the off-the-shelf entity masks and the generated text descriptions are accurate enough that same-mask points form one object and each text describes that object; the paper does not directly measure the noise in these pseudo-labels.

Editorial extensions

If this is right

  • Open-vocabulary segmentation becomes a direct readout: at test time, category names are fed to the frozen text encoder and each point is labeled by its best-matching category, so no category-specific training is needed.
  • The pretrained encoder transfers to closed-set perception: fine-tuning it beats prior self-supervised 3D pretraining on ScanNet200 semantic segmentation (31.8 versus 30.0 mIoU) and instance segmentation (31.6 versus 27.5 mAP@0.5).
  • Data-scarce fine-tuning improves markedly: semantic segmentation with 1% of ScanNet scenes reaches 40.8% mIoU, up from 30.7% for the strongest prior method, and with only 20 labeled points per scene reaches 62.9% mIoU.
  • High-level 3D vision-language tasks also gain: using the learned encoder as the backbone of a promptable-query model improves grounding accuracy on ScanRefer, Sr3D, and Multi3DRefer, raises captioning CIDEr to 80.2, and lifts SQA3D accuracy to 47.5%.
  • Ablations indicate both text types matter: removing either captions or spatial referrals lowers ScanNet foreground mIoU, showing that appearance descriptions and relational descriptions contribute separately.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The noise in the pseudo-labels is never measured; a direct test is to retrain with ground-truth instance masks or hand-checked captions and compare the reported ScanNet numbers.
  • The paper reports that replacing its frozen text encoder with a trainable one raises zero-shot grounding from 17.0% to 42.6%; the likely interpretation is that the remaining bottleneck for fine-grained spatial referrals is the text side, not the 3D encoder.
  • Because the training masks come from an off-the-shelf segmenter, the learned encoder could be used to refine its own proposals and retrain in an iterative loop, potentially improving open-vocabulary instance segmentation further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper proposes MPEC, a masked point-entity contrastive learning method for open-vocabulary 3D scene understanding. It trains a 3D sparse U-Net and a vision-language adapter using two contrastive objectives: (i) point-to-entity contrast across two augmented and masked views of a scene, guided by off-the-shelf entity mask proposals, and (ii) entity-to-language contrast between merged 3D point features and CLIP text embeddings of generated captions and referrals. The model is pre-trained on multiple real and synthetic datasets from SceneVerse and evaluated on open-vocabulary semantic segmentation on ScanNet, ScanNet200, SceneVerse-val, and Matterport3D, as well as via fine-tuning on perception and reasoning benchmarks. The paper reports state-of-the-art ScanNet OV-SemSeg results (66.0% f-mIoU, 81.3% f-mAcc), large gains in data-efficiency settings, and consistent improvements after fine-tuning on most reasoning tasks.

Significance. If the reported results hold, MPEC provides a strong recipe for open-vocabulary 3D encoders: entity-level contrast under cross-view masking appears to improve instance discrimination while preserving semantic alignment, and the data-efficiency gains are practically valuable (e.g., 40.8 vs. 30.7 mIoU at 1% ScanNet-LR). The paper includes thorough ablations on model design, text type, and data composition, and the main ScanNet OV-SemSeg result outperforms prior work under the same SPUNet backbone. However, the zero-shot Matterport3D claim requires scene-level overlap verification with the HM3D training data, and the fine-tuning gains in Table 6 are small and unreplicated, so the overarching zero-shot and transferability claims are not yet fully supported.

major comments (2)
  1. [§3.3 / Table 3] Table 3 marks MPEC as 'Zero-Shot' on Matterport3D, but §3.3 states that MPEC is trained on HM3D from SceneVerse, and Table 9 confirms HM3D is part of the default training mixture. HM3D and Matterport3D are both derived from Matterport scans and are known to overlap at the level of source environments. The sentence 'MPEC omits Matterport3D during training' only verifies that the dataset named Matterport3D was not loaded; it does not rule out that Matterport3D test scenes appeared in the HM3D training subset. Since the reported 47.7 f-mIoU and 69.8 f-mAcc are the paper's only direct evidence for zero-shot generalization to unseen real scenes, the authors must provide a scene-level overlap check between their HM3D training split and the Matterport3D test set, and either re-evaluate on a provably disjoint set or remove the zero-shot label and adjust the abstract's claim accordingly.
  2. [Table 6 / §4.3] The paper claims 'consistent and notable improvements' on high-level reasoning tasks, but Table 6 shows no improvement on Nr3D (66.7 vs. 66.7), a small decrease on Scan2Cap (80.2 vs. 80.3), and only +0.4 on SQA3D (47.5 vs. 47.1). No error bars or multiple-seed results are reported anywhere in the paper, so these differences are likely within run-to-run noise. The abstract and conclusion should either be supported by repeated runs or scaled back to the tasks with clear gains (ScanRefer, Sr3D, Multi3DRefer). I recommend adding variance estimates to at least the main tables (Tables 1, 3, 6).
minor comments (7)
  1. [§3.3, Training Data] The paper never quantifies the noise in the entity masks or the generated text descriptions. Since these define the positive pairs for both contrastive losses, a sentence on estimated noise or a sensitivity analysis would improve confidence in the training signal.
  2. [Supp. Table A.4] The zero-shot grounding results with frozen CLIP (17.0 overall) are far below those of trainable-BERT models (52.9–59.2), which is consistent with the paper's attribution to CLIP but should be prominently acknowledged in the main text, since Fig. 3 already notes the CLIP limitation.
  3. [References] References [26] and [27] are the same paper (3D-LLM) duplicated; please remove one.
  4. [§4.1, third paragraph] The text contains a garbled symbol '„10%' that should be typeset as 'approximately 10%'.
  5. [§4.3] The word 'ScaNet200' appears once and should be 'ScanNet200'.
  6. [Fig. 1] The radar chart and its labels are very small and hard to read in print; please enlarge or provide the numbers in a table.
  7. [Supp. §A] The paper does not specify how the NT sampled text descriptions are selected and whether sampling is uniform over captions and referrals; please clarify in the implementation details.

Circularity Check

1 steps flagged · score 6.0 of 10

Matterport3D zero-shot row reduces to HM3D training input; the rest of the derivation is otherwise self-contained.

  1. fitted input called prediction [Section 4.2 (Table 3) and Section 3.3 (Training Data, Table 9)]
    "It’s worth noting that, compared with OpenScene [57] and OV3D [38], our MPEC omits Matterport3D during training. ... We also leverage the generated paired 3D-VL data from other real 3D scene datasets, including 3RScan [66], HM3D [60] and MultiScan [50] in SceneVerse as additional training data."

    Tab. 3 marks the Matterport3D row as Zero-Shot (✓) and the text claims MPEC 'omits Matterport3D during training,' but Sec. 3.3 and Tab. 9 show MPEC is trained on HM3D from SceneVerse. HM3D is a Matterport-captured scan collection whose source buildings/scans overlap with the Matterport3D benchmark. Omitting the dataset named 'Matterport3D' is therefore not the same as omitting Matterport3D scenes: a subset of the test scans can be present in the training set. Under that overlap, the reported 47.7 f-mIoU / 69.8 f-mAcc are not an out-of-distribution zero-shot prediction; they are produced by a model already fitted to the same scene geometry, so that part of the abstract's 'superior zero-shot scene understanding' claim reduces to a trained input rather than a held-out generalization.

full rationale

MPEC's core derivation is self-contained: the point-to-entity loss (Eq. 6) and entity-to-language loss (Eq. 13) are contrastive objectives trained on external point clouds and pseudo-labels, and the ScanNet OV-SemSeg result is measured on a public benchmark and is not a restatement of the loss. Using the frozen CLIP text encoder both as training target and at evaluation is the intended alignment, not a definitional shortcut, because evaluation supplies category names rather than the training captions/referrals; the ScanNet20/200 and SceneVerse-val results remain externally comparable. Self-citations (SceneVerse, PQ3D) provide training data and mask proposals, but these are publicly released resources and do not by themselves force the benchmark scores. The concrete reduction is the Matterport3D zero-shot row: training includes HM3D, a Matterport-sourced collection with known scene overlap with Matterport3D, so the 'Zero-Shot' label in Tab. 3 is unsupported unless scene-level disjointness is verified. This makes the 47.7/69.8 numbers a partially fitted input rather than a clean zero-shot prediction; the rest of the paper's claims are not reduced.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities; it is a training objective. The main load-bearing input is the quality of auto-generated pseudo-labels (entity masks and text), plus the assumption that CLIP's text space is sufficient for the open-vocabulary alignment.

free parameters (4)
  • alpha (text-to-entity CE loss weight) = 1.0
    Set empirically in supplement A to balance the scale of cross-entropy and binary cross-entropy losses in Eq. 13.
  • beta (entity-to-text BCE loss weight) = 6.0
    Set empirically in supplement A; the authors state the values are chosen to balance the two loss scales.
  • temperature tau
    Temperature in the contrastive losses (Eqs. 4, 5, 11); the paper never reports its value, so a reader cannot reproduce the softmax sharpness.
  • NT sampled text descriptions per scene = 64
    Number of text descriptions sampled per scene during training, set in supplement A; affects the entity-to-language contrastive batch composition.
assumptions (4)
  • domain assumption Entity mask proposals from the off-the-shelf model [97] correctly group points into objects across views.
    Invoked in Sec. 3.1 and 3.3: the point-to-entity loss treats mask-defined entities as ground-truth groups, so mask errors directly corrupt the training target.
  • domain assumption Text descriptions from GPT-4V and scene graphs are semantically correct and refer to the intended entities.
    Invoked in Sec. 3.2 and 3.3: the entity-to-language contrast assumes each description matches exactly one entity mask; mismatched text would create false negatives.
  • domain assumption Frozen CLIP text features are an adequate semantic space for open-vocabulary 3D concepts.
    Invoked throughout Sec. 3.2 and in the evaluation protocol of Sec. 4.1; the paper's own Tab. A.4 shows CLIP is a major bottleneck for spatial referrals, but category-level alignment is still assumed sufficient.
  • domain assumption Augmented views preserve entity identity, so cross-view positives are meaningful.
    Invoked in Sec. 3.1: the cross-view contrast assumes the same mask index in both views corresponds to the same physical object after spatial, photometric, and sampling augmentations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understanding." pith.science (2026). https://pith.science/paper/7S6U3XMO

@misc{pith2026250419500,
  author       = {Pith},
  title        = {Pith review of: Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7S6U3XMO}},
  note         = {Machine review of arXiv:2504.19500}
}
read the original abstract

Open-vocabulary 3D scene understanding is pivotal for enhancing physical intelligence, as it enables embodied agents to interpret and interact dynamically within real-world environments. This paper introduces MPEC, a novel Masked Point-Entity Contrastive learning method for open-vocabulary 3D semantic segmentation that leverages both 3D entity-language alignment and point-entity consistency across different point cloud views to foster entity-specific feature representations. Our method improves semantic discrimination and enhances the differentiation of unique instances, achieving state-of-the-art results on ScanNet for open-vocabulary 3D semantic segmentation and demonstrating superior zero-shot scene understanding capabilities. Extensive fine-tuning experiments on 8 datasets, spanning from low-level perception to high-level reasoning tasks, showcase the potential of learned 3D features, driving consistent performance gains across varied 3D scene understanding tasks. Project website: https://mpec-3d.github.io/

Figures

Figures reproduced from arXiv: 2504.19500 by the authors.

Figure 1
Figure 1. Qualitative and Quantitative Analysis of MPEC Results. MPEC achieves state-of-the-art on open-vocabulary 3D semantic segmentation. Compared with OpenScene [57], MPEC is more robust to tail classes, visual ambiguity, and detailed descriptions, e.g., spatial referral. The radar chart highlights the performance advantages of MPEC across various 3D scene understanding tasks. Abstract Open-vocabulary 3D scene understandi… view at source ↗
Figure 2
Figure 2. The Overall Pipeline of MPEC. Given a 3D point cloud as input, we predict entity mask proposals and generate text descriptions for each entity. Then different views of the scene are randomly masked and we replace the masked point features with learnable embeddings. A 3D UNet then extracts per-point features. Guided by the entity masks, cross-view point-to-entity contrastive learning is conducted to enforce cross-vie… view at source ↗
Figure 3
Figure 3. Qualitative Results on ScanNet [14]. We show relatively good and bad cases in blue and red. 4.2. Zero-shot Transfer and Long-tail Scenarios Experiment Settings To assess the generalization capa￾bility of learned representations to unseen scenes and lan￾guage descriptions, we evaluate the f-mIoU and f-mAcc on OV-SemSeg in two settings: • Zero-shot Transfer: For zero-shot transfer experiments, we evaluate MPEC on the … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

98 extracted references · 50 canonical work pages

  1. [1]

    Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes. InECCV, pages 422–440. Springer, 2020. 2, 7, 8

    2020

  2. [2]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebo- tar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 1

    arXiv 2022

  3. [3]

    Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. InNIPSD, 2021. 8

    2021

  4. [4]

    3djcg: A unified framework for joint dense captioning and visual grounding on 3d point clouds

    Daigang Cai, Lichen Zhao, Jing Zhang, Lu Sheng, and Dong Xu. 3djcg: A unified framework for joint dense captioning and visual grounding on 3d point clouds. InCVPR, pages 16464–16473, 2022. 3

    2022

  5. [5]

    Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158, 2017

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158, 2017. 6, 7

    arXiv 2017

  6. [6]

    Scanrefer: 3d object localization in rgb-d scans using natural language

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language. InECCV, pages 202–221. Springer, 2020. 2, 7, 8

    2020

  7. [7]

    Clip2scene: Towards label-efficient 3d scene under- standing by clip

    Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wenping Wang. Clip2scene: Towards label-efficient 3d scene under- standing by clip. InCVPR, pages 7020–7030, 2023. 2

    2023

  8. [8]

    Scan2cap: Context-aware dense captioning in rgb-d scans

    Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X Chang. Scan2cap: Context-aware dense captioning in rgb-d scans. InCVPR, pages 3193–3203, 2021. 2, 7, 8

    2021

Show all 98 references
  1. [9]

    Unit3d: A unified transformer for 3d dense captioning and visual grounding

    Zhenyu Chen, Ronghang Hu, Xinlei Chen, Matthias Nießner, and Angel X Chang. Unit3d: A unified transformer for 3d dense captioning and visual grounding. InICCV, pages 18109– 18119, 2023. 3

    2023

  2. [10]

    Cat-seg: Cost aggregation for open-vocabulary semantic segmentation

    Seokju Cho, Heeseong Shin, Sunghwan Hong, Anurag Arnab, Paul Hongsuck Seo, and Seungryong Kim. Cat-seg: Cost aggregation for open-vocabulary semantic segmentation. In CVPR, pages 4113–4123, 2024. 2

    2024

  3. [11]

    4d spatio-temporal convnets: Minkowski convolutional neural networks

    Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. InCVPR, pages 3075–3084, 2019. 5, 13

    2019

  4. [12]

    Pointcept: A codebase for point cloud perception research

    Pointcept Contributors. Pointcept: A codebase for point cloud perception research. https://github.com/ Pointcept/Pointcept, 2023. 13

    2023

  5. [13]

    Spconv: Spatially sparse convolu- tion library

    Spconv Contributors. Spconv: Spatially sparse convolu- tion library. https://github.com/traveller59/ spconv, 2022. 13

    2022

  6. [14]

    Scannet: Richly- annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly- annotated 3d reconstructions of indoor scenes. InCVPR, pages 5828–5839, 2017. 2, 6, 7, 8, 13, 14, 15, 16, 17

    2017

  7. [15]

    Procthor: Large-scale embodied ai using procedural generation

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation. In NeurIPS, pages 5982–5994. Curran Associates, Inc., 2022. 8

    2022

  8. [16]

    Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018. 14

    2018 arXiv

  9. [17]

    Pla: Language-driven open-vocabulary 3d scene understanding

    Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. Pla: Language-driven open-vocabulary 3d scene understanding. InCVPR, pages 7010–7019, 2023. 2, 5, 6, 7

    2023

  10. [18]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 6

    2010 arXiv

  11. [19]

    Foundation models in robotics: Applications, challenges, and the fu- ture.The International Journal of Robotics Research, page 02783649241281508, 2023

    Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, et al. Foundation models in robotics: Applications, challenges, and the fu- ture.The International Journal of Robotics Research, page 0...

    2023

  12. [20]

    Scene-llm: Extending language model for 3d visual understanding and reasoning.arXiv preprint arXiv:2403.11401, 2024

    Rao Fu, Jingyu Liu, Xilun Chen, Yixin Nie, and Wen- han Xiong. Scene-llm: Extending language model for 3d visual understanding and reasoning.arXiv preprint arXiv:2403.11401, 2024. 3

    2024 arXiv

  13. [21]

    Scaling open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scaling open-vocabulary image segmentation with image-level labels. InECCV, pages 540–557. Springer, 2022. 2

    2022

  14. [22]

    3d semantic segmentation with submanifold sparse convolutional networks

    Benjamin Graham, Martin Engelcke, and Laurens Van Der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. InCVPR, pages 9224–9232,

  15. [23]

    Concept- graphs: Open-vocabulary 3d scene graphs for perception and planning

    Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. Concept- graphs: Open-vocabulary 3d scene graphs for perception and planning. InICRA, pages 5021–5028. IEEE, 2024. 1

    2024

  16. [24]

    Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following.arXiv preprint arXiv:2309.00615, 2023

    Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xi- anzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following.arXiv preprint ...

    2023 arXiv

  17. [25]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. InCVPR, pages 16000–16009, 2022. 3

    2022

  18. [26]

    3d-llm: Injecting the 3d world into large language models.NIPS, 36:20482– 20494, 2023

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models.NIPS, 36:20482– 20494, 2023. 3

    2023

  19. [27]

    3d-llm: Injecting the 3d world into large language models.NIPS, 36:20482– 20494, 2023

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models.NIPS, 36:20482– 20494, 2023. 1

    2023

  20. [28]

    Multiply: A multisensory object- centric embodied large language model in 3d world

    Yining Hong, Zishuo Zheng, Peihao Chen, Yian Wang, Jun- yan Li, and Chuang Gan. Multiply: A multisensory object- centric embodied large language model in 3d world. InCVPR, pages 26406–26416, 2024. 3

    2024

  21. [29]

    Exploring data-efficient 3d scene understanding with 9 contrastive scene contexts

    Ji Hou, Benjamin Graham, Matthias Nießner, and Saining Xie. Exploring data-efficient 3d scene understanding with 9 contrastive scene contexts. InCVPR, pages 15587–15597,

  22. [30]

    Chat-scene: Bridging 3d scene and large language models with object identifiers

    Haifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang, Runsen Xu, Tai Wang, Luping Liu, Xize Cheng, Yang Zhao, Jiangmiao Pang, et al. Chat-scene: Bridging 3d scene and large language models with object identifiers. InNIPS, 2024. 3

    2024

  23. [31]

    An embodied generalist agent in 3d world.arXiv preprint arXiv:2311.12871, 2023

    Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baox- iong Jia, and Siyuan Huang. An embodied generalist agent in 3d world.arXiv preprint arXiv:2311.12871, 2023. 1, 3

    2023 arXiv

  24. [32]

    Unveiling the mist over 3d vision-language under- standing: Object-centric evaluation with chain-of-analysis

    Jiangyong Huang, Baoxiong Jia, Yan Wang, Ziyu Zhu, Xiongkun Linghu, Qing Li, Song-Chun Zhu, and Siyuan Huang. Unveiling the mist over 3d vision-language under- standing: Object-centric evaluation with chain-of-analysis. In CVPR, 2025. 2, 3

    2025

  25. [33]

    Spatio-temporal self-supervised representation learning for 3d point clouds

    Siyuan Huang, Yichen Xie, Song-Chun Zhu, and Yixin Zhu. Spatio-temporal self-supervised representation learning for 3d point clouds. InICCV, pages 6535–6545, 2021. 3

    2021

  26. [34]

    Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation

    Zhening Huang, Xiaoyang Wu, Xi Chen, Hengshuang Zhao, Lei Zhu, and Joan Lasenby. Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation. InECCV, pages 169–185. Springer, 2025. 2

    2025

  27. [35]

    Sceneverse: Scaling 3d vision-language learning for grounded scene un- derstanding.arXiv preprint arXiv:2401.09340, 2024

    Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. Sceneverse: Scaling 3d vision-language learning for grounded scene un- derstanding.arXiv preprint arXiv:2401.09340, 2024. 2, 3, 5, 6, 7, 8, 14

    2024 arXiv

  28. [36]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. InICML, pages 4904–

  29. [37]

    Pointgroup: Dual-set point grouping for 3d instance segmentation

    Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi- Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point grouping for 3d instance segmentation. InCVPR, pages 4867–4876,

  30. [38]

    Open-vocabulary 3d semantic segmentation with foundation models

    Li Jiang, Shaoshuai Shi, and Bernt Schiele. Open-vocabulary 3d semantic segmentation with foundation models. InCVPR, pages 21284–21294, 2024. 2, 5, 6, 7

    2024

  31. [39]

    Robin3d: Improving 3d large lan- guage model via robust instruction tuning.arXiv preprint arXiv:2410.00255, 2024

    Weitai Kang, Haifeng Huang, Yuzhang Shang, Mubarak Shah, and Yan Yan. Robin3d: Improving 3d large lan- guage model via robust instruction tuning.arXiv preprint arXiv:2410.00255, 2024. 3

    2024 arXiv

  32. [40]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InICCV, pages 4015–4026, 2023. 2

    2023

  33. [41]

    Language-driven semantic seg- mentation.arXiv preprint arXiv:2201.03546, 2022

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and René Ranftl. Language-driven semantic seg- mentation.arXiv preprint arXiv:2201.03546, 2022. 2, 6

    2022 arXiv

  34. [42]

    Maniptrans: Efficient dexterous bimanual manipula- tion transfer via residual learning

    Kailin Li, Puhao Li, Tengyu Liu, Yuyang Li, and Siyuan Huang. Maniptrans: Efficient dexterous bimanual manipula- tion transfer via residual learning. InCVPR, 2025. 1

    2025

  35. [43]

    Dense multimodal alignment for open-vocabulary 3d scene understanding

    Ruihuang Li, Zhengqiang Zhang, Chenhang He, Zhiyuan Ma, Vishal M Patel, and Lei Zhang. Dense multimodal alignment for open-vocabulary 3d scene understanding. InECCV, pages 416–434. Springer, 2024. 2

    2024

  36. [44]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 3

    2023

  37. [45]

    Building interactable replicas of complex articulated objects via gaussian splatting

    Yu Liu, Baoxiong Jia, Ruijie Lu, Junfeng Ni, Song-Chun Zhu, and Siyuan Huang. Building interactable replicas of complex articulated objects via gaussian splatting. InICLR, 2025. 1

    2025

  38. [46]

    Movis: Enhancing multi-object novel view synthesis for indoor scenes

    Ruijie Lu, Yixin Chen, Junfeng Ni, Baoxiong Jia, Yu Liu, Di- wen Wan, Gang Zeng, and Siyuan Huang. Movis: Enhancing multi-object novel view synthesis for indoor scenes. InCVPR,

  39. [47]

    Ovir-3d: Open-vocabulary 3d instance retrieval without training on 3d data

    Shiyang Lu, Haonan Chang, Eric Pu Jing, Abdeslam Boular- ias, and Kostas Bekris. Ovir-3d: Open-vocabulary 3d instance retrieval without training on 3d data. InCoRL, pages 1610–

  40. [48]

    Sqa3d: Sit- uated question answering in 3d scenes.arXiv preprint arXiv:2210.07474, 2022

    Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Sit- uated question answering in 3d scenes.arXiv preprint arXiv:2210.07474, 2022. 2, 7, 8

    2022 arXiv

  41. [49]

    When llms step into the 3d world: A survey and meta-analysis of 3d tasks via multi-modal large language models.arXiv preprint arXiv:2405.10255, 2024

    Xianzheng Ma, Yash Bhalgat, Brandon Smart, Shuai Chen, Xinghui Li, Jian Ding, Jindong Gu, Dave Zhenyu Chen, Songyou Peng, Jia-Wang Bian, et al. When llms step into the 3d world: A survey and meta-analysis of 3d tasks via multi-modal large language models.arXiv preprint arXiv:2...

    2024

  42. [50]

    Multiscan: Scalable rgbd scanning for 3d environments with articulated objects

    Yongsen Mao, Yiming Zhang, Hanxiao Jiang, Angel Chang, and Manolis Savva. Multiscan: Scalable rgbd scanning for 3d environments with articulated objects. InNIPS, 2022. 5, 6, 8

    2022

  43. [51]

    Occupancy-mae: Self-supervised pre-training large- scale lidar point clouds with masked occupancy autoencoders

    Chen Min, Liang Xiao, Dawei Zhao, Yiming Nie, and Bin Dai. Occupancy-mae: Self-supervised pre-training large- scale lidar point clouds with masked occupancy autoencoders. IEEE Transactions on Intelligent Vehicles, 2023. 3

    2023

  44. [52]

    Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance

    Phuc Nguyen, Tuan Duc Ngo, Evangelos Kalogerakis, Chuang Gan, Anh Tran, Cuong Pham, and Khoi Nguyen. Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance. InCVPR, 2024. 2

    2024

  45. [53]

    Phyrecon: Physically plausible neural scene recon- struction

    Junfeng Ni, Yixin Chen, Bohan Jing, Nan Jiang, Bin Wang, Bo Dai, Puhao Li, Yixin Zhu, Song-Chun Zhu, and Siyuan Huang. Phyrecon: Physically plausible neural scene recon- struction. InNeurIPS, 2024. 1

    2024

  46. [54]

    Decompositional neural scene reconstruction with generative diffusion prior

    Junfeng Ni, Yu Liu, Ruijie Lu, Zirui Zhou, Song-Chun Zhu, Yixin Chen, and Siyuan Huang. Decompositional neural scene reconstruction with generative diffusion prior. InCVPR,

  47. [55]

    Open x-embodiment: Robotic learning datasets and rt-x mod- els.arXiv preprint arXiv:2310.08864, 2023

    Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, et al. Open x-embodiment: Robotic learning datasets and rt-x mod- els.arXiv preprint arXiv:2310.08864, 2023. 1

    arXiv

  48. [56]

    Gpt-4 with vision (gpt-4v) system card, 2023

    OpenAI. Gpt-4 with vision (gpt-4v) system card, 2023. 5

    2023

  49. [57]

    Openscene: 3d scene understanding with open vocabularies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasac- chi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. InCVPR, pages 815–824, 2023. 1, 2, 6, 7, 14

    2023

  50. [58]

    Shapellm: Universal 3d object understanding for embodied interaction

    Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. InECCV, pages 214–238. Springer, 2025. 3 10

    2025

  51. [59]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InICML, pages 8748–8763. PMLR, 2021. 2, 4, 5, 6, 13, 14

    2021

  52. [60]

    Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai.arXiv preprint arXiv:2109.08238, 2021

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Un- dersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai.arXiv preprin...

    2021 arXiv

  53. [61]

    Language- grounded indoor 3d semantic segmentation in the wild

    David Rozenberszki, Or Litany, and Angela Dai. Language- grounded indoor 3d semantic segmentation in the wild. In ECCV, pages 125–141. Springer, 2022. 6, 7, 8, 13

    2022

  54. [62]

    Mask3d: Mask trans- former for 3d semantic instance segmentation

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d: Mask trans- former for 3d semantic instance segmentation. InICRA, pages 8216–8223. IEEE, 2023. 2

    2023

  55. [63]

    Open- mask3d: open-vocabulary 3d instance segmentation

    Ayça Takmaz, Elisabetta Fedele, Robert W Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. Open- mask3d: open-vocabulary 3d instance segmentation. InNIPS, pages 68367–68390, 2023. 2

    2023

  56. [64]

    Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors

    Yuan Tang, Xu Han, Xianzhi Li, Qiao Yu, Yixue Hao, Long Hu, and Min Chen. Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors. In ACMMM, pages 6617–6626, 2024. 3

    2024

  57. [65]

    Geomae: Masked geometric target prediction for self-supervised point cloud pre-training

    Xiaoyu Tian, Haoxi Ran, Yue Wang, and Hang Zhao. Geomae: Masked geometric target prediction for self-supervised point cloud pre-training. InCVPR, pages 13570–13580, 2023. 3

    2023

  58. [66]

    Rio: 3d object instance re- localization in changing indoor environments

    Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias Nießner. Rio: 3d object instance re- localization in changing indoor environments. InICCV, 2019. 5, 8

    2019

  59. [67]

    Groupcontrast: Semantic-aware self-supervised representation learning for 3d understanding

    Chengyao Wang, Li Jiang, Xiaoyang Wu, Zhuotao Tian, Bo- hao Peng, Hengshuang Zhao, and Jiaya Jia. Groupcontrast: Semantic-aware self-supervised representation learning for 3d understanding. InCVPR, pages 4917–4928, 2024. 2, 3, 7, 8, 13, 15

    2024

  60. [68]

    Open vocabulary 3d scene understanding via geometry guided self-distillation

    Pengfei Wang, Yuxi Wang, Shuai Li, Zhaoxiang Zhang, Zhen Lei, and Lei Zhang. Open vocabulary 3d scene understanding via geometry guided self-distillation. InECCV, pages 442–

  61. [69]

    Embodiedscan: A holistic multi- modal 3d perception suite towards embodied ai

    Tai Wang, Xiaohan Mao, Chenming Zhu, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, et al. Embodiedscan: A holistic multi- modal 3d perception suite towards embodied ai. InCVPR,

  62. [70]

    T-mae: Temporal masked autoencoders for point cloud representation learning.arXiv preprint arXiv:2312.10217, 2023

    Weijie Wei, Fatemeh Karimi Nejadasl, Theo Gevers, and Martin R Oswald. T-mae: Temporal masked autoencoders for point cloud representation learning.arXiv preprint arXiv:2312.10217, 2023. 3

    2023 arXiv

  63. [71]

    Clipself: Vision trans- former distills itself for open-vocabulary dense prediction

    Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Xiangtai Li, Wentao Liu, and Chen Change Loy. Clipself: Vision trans- former distills itself for open-vocabulary dense prediction. arXiv preprint arXiv:2310.01403, 2023. 2

    2023 arXiv

  64. [72]

    Masked scene contrast: A scalable framework for unsuper- vised 3d representation learning

    Xiaoyang Wu, Xin Wen, Xihui Liu, and Hengshuang Zhao. Masked scene contrast: A scalable framework for unsuper- vised 3d representation learning. InCVPR, pages 9415–9424,

  65. [73]

    Sed: A simple encoder-decoder for open-vocabulary semantic segmentation

    Bin Xie, Jiale Cao, Jin Xie, Fahad Shahbaz Khan, and Yanwei Pang. Sed: A simple encoder-decoder for open-vocabulary semantic segmentation. InCVPR, pages 3426–3436, 2024. 2

    2024

  66. [74]

    Pointcontrast: Unsupervised pre- training for 3d point cloud understanding

    Saining Xie, Jiatao Gu, Demi Guo, Charles R Qi, Leonidas Guibas, and Or Litany. Pointcontrast: Unsupervised pre- training for 3d point cloud understanding. InECCV, pages 574–591. Springer, 2020. 2, 3, 8, 13

    2020

  67. [75]

    Simmim: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. InCVPR, pages 9653–9663, 2022. 3

    2022

  68. [76]

    Sampro3d: Locating sam prompts in 3d for zero-shot scene segmentation.arXiv preprint arXiv:2311.17707, 2023

    Mutian Xu, Xingyilang Yin, Lingteng Qiu, Yang Liu, Xin Tong, and Xiaoguang Han. Sampro3d: Locating sam prompts in 3d for zero-shot scene segmentation.arXiv preprint arXiv:2311.17707, 2023. 2

    2023 arXiv

  69. [77]

    Maskclus- tering: View consensus based mask graph clustering for open- vocabulary 3d instance segmentation

    Mi Yan, Jiazhao Zhang, Yan Zhu, and He Wang. Maskclus- tering: View consensus based mask graph clustering for open- vocabulary 3d instance segmentation. InCVPR, pages 28274– 28284, 2024. 2

    2024

  70. [78]

    3d vision and language pretraining with large-scale synthetic data.arXiv preprint arXiv:2407.06084,

    Dejie Yang, Zhu Xu, Wentao Mo, Qingchao Chen, Siyuan Huang, and Yang Liu. 3d vision and language pretraining with large-scale synthetic data.arXiv preprint arXiv:2407.06084,

    arXiv

  71. [79]

    Gd-mae: gener- ative decoder for mae pre-training on lidar point clouds

    Honghui Yang, Tong He, Jiaheng Liu, Hua Chen, Boxi Wu, Binbin Lin, Xiaofei He, and Wanli Ouyang. Gd-mae: gener- ative decoder for mae pre-training on lidar point clouds. In CVPR, pages 9403–9414, 2023. 3

    2023

  72. [80]

    3d-grand: A million-scale dataset for 3d-llms with better grounding and less hallucination.arXiv preprint arXiv:2406.05132, 2024

    Jianing Yang, Xuweiyi Chen, Nikhil Madaan, Madhavan Iyen- gar, Shengyi Qian, David F Fouhey, and Joyce Chai. 3d-grand: A million-scale dataset for 3d-llms with better grounding and less hallucination.arXiv preprint arXiv:2406.05132, 2024. 2

    2024 arXiv

  73. [81]

    Regionplc: Regional point-language contrastive learning for open-world 3d scene understanding

    Jihan Yang, Runyu Ding, Weipeng Deng, Zhe Wang, and Xiaojuan Qi. Regionplc: Regional point-language contrastive learning for open-world 3d scene understanding. InCVPR,

  74. [82]

    Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu. Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023. 2

    2023 arXiv

  75. [83]

    Metascenes: Towards automated replica creation for real-world 3d scans

    Huangyue Yu, Baoxiong Jia, Yixin Chen, Yandan Yang, Puhao Li, Rongpeng Su, Jiaxin Li, Qing Li, Wei Liang, Song- Chun Zhu, Tengyu Liu, and Siyuan Huang. Metascenes: Towards automated replica creation for real-world 3d scans. InCVPR, 2025. 1, 2

    2025

  76. [84]

    Point-bert: Pre-training 3d point cloud transformers with masked point modeling

    Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. InCVPR, pages 19313–19322, 2022. 2, 3

    2022

  77. [85]

    Clip2: Contrastive language-image- point pretraining from real-world point cloud data

    Yihan Zeng, Chenhan Jiang, Jiageng Mao, Jianhua Han, Chao- qiang Ye, Qingqiu Huang, Dit-Yan Yeung, Zhen Yang, Xiao- dan Liang, and Hang Xu. Clip2: Contrastive language-image- point pretraining from real-world point cloud data. InCVPR, pages 15244–15253, 2023. 6

    2023

  78. [86]

    Vision-language pre-training with object contrastive learning for 3d scene understanding

    Taolin Zhang, Sunan He, Tao Dai, Zhi Wang, Bin Chen, and Shu-Tao Xia. Vision-language pre-training with object contrastive learning for 3d scene understanding. InAAAI, pages 7296–7304, 2024. 3 11

    2024

  79. [87]

    Multi3drefer: Grounding text description to multiple 3d ob- jects

    Yiming Zhang, ZeMing Gong, and Angel X Chang. Multi3drefer: Grounding text description to multiple 3d ob- jects. InICCV, pages 15225–15236, 2023. 2, 7, 8

    2023

  80. [88]

    Self-supervised pretraining of 3d features on any point-cloud

    Zaiwei Zhang, Rohit Girdhar, Armand Joulin, and Ishan Misra. Self-supervised pretraining of 3d features on any point-cloud. InICCV, pages 10252–10263, 2021. 3

    2021

  81. [89]

    3d-vla: A 3d vision-language-action generative world model.arXiv preprint arXiv:2403.09631, 2024

    Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d vision-language-action generative world model.arXiv preprint arXiv:2403.09631, 2024. 3

    2024 arXiv

  82. [90]

    Structured3d: A large photo-realistic dataset for structured 3d modeling

    Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou. Structured3d: A large photo-realistic dataset for structured 3d modeling. InECCV, pages 519–

  83. [91]

    Closed- loop open-vocabulary mobile manipulation with gpt-4v.arXiv preprint arXiv:2404.10220, 2024

    Peiyuan Zhi, Zhiyuan Zhang, Muzhi Han, Zeyu Zhang, Zhi- tian Li, Ziyuan Jiao, Baoxiong Jia, and Siyuan Huang. Closed- loop open-vocabulary mobile manipulation with gpt-4v.arXiv preprint arXiv:2404.10220, 2024. 1

    2024 arXiv

  84. [92]

    Extract free dense labels from clip

    Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from clip. InECCV, pages 696–712. Springer,

  85. [93]

    Uni3d: Exploring unified 3d representation at scale.arXiv preprint arXiv:2310.06773,

    Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale.arXiv preprint arXiv:2310.06773,

    arXiv

  86. [94]

    Detecting twenty-thousand classes using image-level supervision

    Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähen- bühl, and Ishan Misra. Detecting twenty-thousand classes using image-level supervision. InECCV, pages 350–368. Springer, 2022. 2

    2022

  87. [95]

    Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness.arXiv preprint arXiv:2409.18125, 2024

    Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness.arXiv preprint arXiv:2409.18125, 2024. 3

    2024 arXiv

  88. [96]

    3d-vista: Pre-trained transformer for 3d vision and text alignment

    Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, and Qing Li. 3d-vista: Pre-trained transformer for 3d vision and text alignment. InICCV, pages 2911–2921, 2023. 3, 14

    2023

  89. [97]

    Unifying 3d vision-language understanding via prompt- able queries.arXiv preprint arXiv:2405.11442, 2024

    Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, and Qing Li. Unifying 3d vision-language understanding via prompt- able queries.arXiv preprint arXiv:2405.11442, 2024. 3, 5, 7, 13 12 Masked Point-Entity Contrast for Open-...

    2024 arXiv

  90. [2021]

    2, 3, 7, 8, 13, 14, 15

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.