Pith. sign in

REVIEW 4 major objections 49 references

Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM

T0 review · 4 major / 0 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single point-cloud model can answer 3D questions with both a mask of the referred region and real-world measurements.

desk verdict Solid systems paper that actually ships joint point grounding + metric dialogue; the soft spot is viewport-auto-lifted metric GT, not the architecture. read the letter →

arxiv 2607.05493 v1 pith:LLNCEPXM submitted 2026-07-06 cs.CV

Amol Harsh , Zongyan Han , Jean Lahoud , Ye Liu , Rao Muhammad Anwer , Hisham Cholakkal , Salman Khan , Fahad Khan This is my paper
classification cs.CV
keywords 3Dpointgroundingmetricspatialreasoninglargemultimodalmodelspart-levelsegmentationgroundedmeasurementpoint-cloudLMMScanNetopen-vocabulary
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Natural-language questions about indoor 3D scenes are only useful when the answer can be checked against the geometry and when sizes and distances are given in meters, not vague comparisons. Existing systems either chat without pointing at the scene or produce masks without holding a metric conversation. This paper argues that both capabilities can live in one model: Ground3D-LMM takes a colored point cloud (and optionally an RGB view), replies in text, and when a special grounding token is emitted, produces a point-level mask of the referred object or part while reporting dimensions, clearances, and distances in physical units. The authors formalize 3D Grounded Measurement as the joint task of predicting the referred region and its metric quantities, release a large ScanNet/ScanNet++-based dataset of roughly 2.5 million question-answer pairs spanning eight subtasks at object and part granularity, and show that the model outperforms open-vocabulary segmentation and 3D reasoning baselines on their benchmark as well as on Reason3D and ScanRefer. A sympathetic reader cares because verifiable, metric dialogue is what robots, AR agents, and assistive systems need before language about 3D space becomes actionable.

What carries the argument

Point features are projected as tokens into an LMM alongside text (and optional image tokens); when the model emits a special <SEG> trigger tied to a phrase, a lightweight segmentation head predicts the corresponding point-level mask while the language model verbalizes metric quantities computed over the grounded region.

What would settle it

On the manually verified test scenes, check whether predicted masks with IoU above 0.3 also yield metric answers whose absolute percentage error and delta-success rate remain high when independent physical measurements of the same objects and parts are taken with a tape measure or laser rangefinder in the original rooms.

Watch

Extended reading notes

Core claim

Ground3D-LMM is a unified point-cloud large multimodal model that, from a colored point cloud and an optional RGB image, produces conversational answers paired with point-level 3D masks and metric numeric outputs at both object and part granularity, including multi-object queries; the accompanying 3D Grounded Measurement task and Ground3D dataset make this joint capability measurable and show the model is a strong baseline.

Load-bearing premise

Metric labels and spatial relations derived from visible, viewport-limited point sets after automatic 2D-to-3D lifting and large-scale vision-language synthesis are accurate enough to train and evaluate the model as ground truth.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper proposes Ground3D-LMM, a unified point-cloud LMM that, given a colored point cloud and optional RGB view, produces conversational text answers together with point-level 3D masks (via a <SEG> trigger and segmentation head) and metric numeric outputs at object and part granularity, including multi-object queries. It formalizes the 3D Grounded Measurement task (joint referred-region mask + physical quantities in real units), introduces the Ground3D dataset (~2.5M QA pairs on ScanNet/ScanNet++ with object/part masks, multi-turn dialogue, and a manually filtered test split), and reports gains over image baselines, UniSeg3D, Reason3D, SD-VLM, and MLLM-For3D on Ground3D, Reason3D, and ScanRefer, with ablations on data scale, joint mask+metric supervision, and mask noise.

Significance. If the joint grounding-and-metric claims hold under reliable physical supervision, the work fills a clear gap between conversational 3D LMMs (text-only) and grounding/segmentation models (non-interactive, non-metric), with direct relevance to robotics clearances, AR/VR measurement, and assistive part-level interaction. Strengths that should be credited include: a large multi-task dataset with object/part granularity and multi-turn structure; public release of data and model; a simple, reproducible architecture (sparse 3D U-Net + Qwen3-VL-4B + SEG head); strong external instance-level grounding results on ScanRefer (Table 6: Acc@0.25 55.73 / mIoU 38.72 in 3D-only); and ablations (Table 7) showing mutual benefit of joint mask+metric supervision and robustness to noisy masks. These make the paper a useful systems and benchmark contribution even if metric fidelity needs tighter validation.

major comments (4)
  1. Section 3.2 Stages 3–4 and the verification paragraph: metric attributes (OBB length/width/thickness, 5th-percentile closest-point distances, camera-frame left/right and closer/farther) are computed only on viewport-visible, depth-lifted point sets after 5%–95% outlier cleanup, not on complete object meshes. The text states metrics match “what the model perceives.” This is load-bearing for the central “metric-aware / real-world units” claim and for GM-δ. Without an independent physical audit (e.g., comparison of viewport OBBs/clearances to ScanNet mesh/instance dimensions or multi-view fused geometry on a held-out subset, with error histograms by occlusion and part type), low APE / high δ / GM-δ largely measure agreement with a synthetic, partial-view target. External ScanRefer/Reason3D gains validate grounding, not metric fidelity. Please add such a validation study and discuss systemat
  2. Table 4 and Supplementary Table 8: even the best 3D+2D model reports Mean APE of ~74–88% on the evaluation subsets, with δ success rates of 34–44% and GM-δ ≈ 42.69. These absolute numbers are modest for a claim of actionable metric measurement (size, thickness, clearance). The paper should (i) report APE/δ stratified by quantity type (length vs thickness vs inter-object distance), object vs part, and occlusion level; (ii) define what constitutes acceptable metric error for the intended applications; and (iii) place GM-δ (currently deferred to the supplement) in the main evaluation section with the same stratification, since it is the paper’s own joint success metric.
  3. Section 5.3 / Tables 2–3: several baselines are only partially adapted to the proposed task. Reason3D is a reasoning-based semantic segmenter and UniSeg3D an open-vocabulary segmenter; neither is trained for multi-object metric dialogue or part-level grounded measurement on Ground3D. The image baseline back-projects Grounded-SAM masks through depth. Gains over these systems support the value of the unified interface, but do not fully isolate whether improvements come from the architecture, the large synthetic training set, or task mismatch. Please either (a) fine-tune or re-prompt the strongest open baselines on Ground3D training data under a common protocol, or (b) clearly frame comparisons as “task-adapted baselines” and add a controlled ablation that freezes the LMM and varies only the grounding/metric heads.
  4. Dataset circularity (Section 3.2 Stage 5 and filtering): training QA is synthesized by Qwen3-VL conditioned on the same Stage-4 attributes used as labels, with VLM-based filtering and manual checks only on the 128-scene test subset (~26% removed). This is acceptable for a new benchmark if disclosed, but the main text should quantify residual label noise on the test set (inter-annotator or human-vs-pipeline metric disagreement rates, phrase–mask consistency rates) and report whether metric numbers in answers are copied from context (as the generation prompts require) versus recomputed by the model at inference. Without that, it is hard to separate memorization of synthetic templates from geometric reasoning.

Circularity Check

1 steps flagged · score 1.0 of 10

Empirical LMM paper with no derivation circularity; train/eval share synthetic geometric GT by design, but claims are not forced by construction and external benchmarks are independent.

  1. other [Section 3.2 Stages 3–4; Evaluation Metrics (GM-δ); verification paragraph]
    "All metrics are defined over the visible 3D geometry from the current viewport rather than the full object mesh, matching what the model perceives and reflecting embodied and robotics settings. ... A prediction is counted as successful only when its mask IoU with the ground truth exceeds 0.3 and its metric prediction satisfies δ≤1.25."

    Metric GT (OBB size, clearances, depth/left-right) and the GM-δ success criterion are both produced by the same viewport-limited geometric pipeline used to build training labels. Metric accuracy on Ground3D therefore partly measures agreement with that construction rather than independent physical survey. This is ordinary synthetic-GT coupling, not a fitted free parameter renamed as a prediction, and external ScanRefer/Reason3D results do not depend on it.

full rationale

Ground3D-LMM is a supervised multimodal system paper, not a first-principles derivation. The load-bearing claims are (i) a model that jointly emits text, point masks, and metric numbers, and (ii) strong results on the authors’ 3D Grounded Measurement task plus Reason3D and ScanRefer. Masks and metric targets (OBB dimensions, 5th-percentile clearances, camera-frame relations) are computed once from ScanNet/ScanNet++ depth and used as ordinary supervised labels; the model is not given those scalars as inputs and then asked to “predict” them, nor is any free parameter fitted to the reported APE/δ/GM-δ and then re-reported as a prediction. Evaluation on Reason3D and ScanRefer after fine-tuning is external to the synthetic metric pipeline and therefore not self-referential. Author-overlapping citations (e.g., PARIS-3D) appear only as related work, not as uniqueness theorems that force the method. The only mild self-containment is that metric success on Ground3D measures agreement with the authors’ viewport-limited geometric construction—standard for synthetic GT, not circular reduction of a claimed derivation. Score 1 reflects that soft train/eval coupling without elevating it to fitted-input or self-definitional circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 3 invented entities

The work is empirical systems research. It rests on standard deep-learning practice, ScanNet geometry, and the assumption that automatically lifted and VLM-synthesized labels are good enough after filtering. No new physical entities or free physical constants are introduced; free parameters are ordinary training hyperparameters and geometric thresholds used in data construction.

free parameters (5)
  • Learning rates (point encoder 1e-6, LMM 1e-5, heads 1e-4) and AdamW weight decay 0.05
    Chosen stratified rates and decay; not derived from theory; affect convergence of the reported numbers.
  • Loss weights λ_seg = λ_text = 1.0
    Set without extensive tuning; balance mask vs language objectives.
  • IoU threshold 0.3 and δ≤1.25 for GM-δ success
    Hand-chosen success criteria that define the joint metric.
  • Outlier percentiles (5%–95%) and distance-transform erosion for mask cleanup
    Geometric thresholds in Stage 3 that determine which points become ground-truth masks.
  • Manual removal rate (~26%) plus VLM filter (~3%) on evaluation scenes
    Human and secondary-VLM filtering decisions that shape the final test distribution.
assumptions (4)
  • domain assumption Visible viewport geometry (not full object meshes) is the correct reference for metric answers in embodied settings.
    Stated in Stage 4; if full-mesh metrics were required, reported sizes/distances would change.
  • domain assumption Qwen3-VL-generated open-vocabulary object/part proposals and QA pairs, after rule and VLM filtering, are sufficiently accurate for supervision.
    Core of Stages 1 and 5; synthetic labels drive both training and much of evaluation.
  • domain assumption Sparse 3D U-Net superpoint features plus a linear projector can be aligned into a 4B LMM token space for joint language and geometry reasoning.
    Architectural premise of Sections 4.2–4.3.
  • standard math Standard BCE+Dice and next-token cross-entropy losses are adequate multi-task objectives for mask and text.
    Section 4.5; conventional losses, not re-derived.
invented entities (3)
  • 3D Grounded Measurement task independent evidence
    purpose: Joint evaluation protocol requiring both a point mask and metric quantities for the same referent.
    New task definition; evaluation construct rather than a physical entity. Independent evidence is the public dataset and metrics.
  • Ground3D dataset (~2.5M QA pairs with object/part masks and metrics) independent evidence
    purpose: Provide training and test supervision for grounded metric dialogue.
    Constructed resource; value depends on label quality after automatic synthesis and partial manual verification.
  • <SEG> trigger token + segmentation head for point masks
    purpose: Couple free-form LMM text generation to point-level mask prediction.
    Engineering mechanism similar to prior grounded LMMs (e.g., LISA-style); not a new physical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM." pith.science (2026). https://pith.science/paper/LLNCEPXM

@misc{pith2026260705493,
  author       = {Pith},
  title        = {Pith review of: Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LLNCEPXM}},
  note         = {Machine review of arXiv:2607.05493}
}
read the original abstract

Natural-language queries about 3D environments become actionable when responses are verifiable and metric. Verifiability requires explicit grounding to the referred 3D region, while metric answers report physical measurements in real-world units (e.g., size, thickness, clearance, and distance). Existing 3D large multimodal models (LMMs) approaches remain limited: conversational systems typically respond without explicit 3D grounding, while 3D grounding models are not designed for interactive, metric-aware dialogue. In this paper, we present Ground3D-LMM, a unified model that takes a point cloud and an optional RGB image as input and supports 3D spatial conversation with (i) point-grounded responses and (ii) metric numeric outputs at both object and part granularity, including multi-object queries. To evaluate this intersection of grounding and measurement, we define the 3D Grounded Measurement task, which requires predicting the referred 3D region and the corresponding metric quantities in real-world units. We introduce a large-scale dataset built on ScanNet and ScanNet++ datasets with dense object and part annotations and roughly 2.5M question-answer pairs spanning eight tasks, along with a manually verified test set. Extensive experiments on multiple datasets and tasks show that our proposed Ground3D-LMM model provides a strong baseline for grounded, metric-aware 3D conversational understanding. Our dataset and model are publicly available.

Figures

Figures reproduced from arXiv: 2607.05493 by the authors.

Figure 1
Figure 1. Ground3D-LMM answers open-vocabulary object/part questions with grounded (3D segmentation) and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. 3.1 Dataset Subtasks Ground3D dataset is organized as a set of complementary subtasks that cover grounded conversation and metric reasoning at both object and part granularity: 3D metric estimation: report object/part dimensions derived from the referred region; 3D grounded measurement: jointly return a mask and the requested measurement for the same referent; 3D distance queries: compute distances/clearances betwee… view at source ↗
Figure 2
Figure 2. Ground3D statistics and annotation pipeline. Starting from ScanNet [12] and ScanNet++ [42] RGB-D sequences, we sample representative frames, detect object/part candidates, generate and refine 2D masks, lift them to point-level 3D supervision via depth back-projection, and compute metric attributes and spatial relations. We then synthesize grounded QA at object/part level, scene-relational level, and multi-turn dialo… view at source ↗
Figures from the paper (10 more)
Figure 3
Figure 3. Figure 3: Ground3D-LMM overview. Given a colored point cloud and an optional RGB image, the model answers in text and outputs point-level masks when <SEG> is triggered. and global size ranking via OBB volume. All metrics are defined over the visible 3D geometry from the current …
Figure 4
Figure 4. Figure 4: Qualitative results of Ground3D-LMM across six representative object- and part-level sub-tasks. (Zoom [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 1
Figure 1. Figure 1: Semantic embedding map (UMAP) ScanNet/ScanNet++ taxonomy Ground3D labels (open-vocab) [PITH_FULL_IMAGE:figures/full_fig_p015_1.png]
Figure 5
Figure 5. Figure 5: Semantic embedding map (UMAP). CLIP text embeddings of ScanNet/ScanNet++ taxonomy labels (blue) and Ground3D labels (orange) projected to 2D using UMAP. Quantitative evidence: nearest-neighbor similarity to the taxonomy. While the UMAP visualization provides intuitive …
Figure 2
Figure 2. Figure 2: Nearest-neighbor similarity distribution (semantic novelty) [PITH_FULL_IMAGE:figures/full_fig_p016_2.png]
Figure 6
Figure 6. Figure 6: Nearest-neighbor similarity distribution (semantic novelty). Blue: leave-one-out NN cosine similarity within the ScanNet/ScanNet++ taxonomy. Orange: NN cosine similarity from each Ground3D label to its nearest taxonomy label. Dashed vertical lines mark baseline-driven …
Figure 7
Figure 7. Figure 7: Qualitative results of Ground3D-LMM on 3D multi-turn conversation. (Zoom in for better visualization.) [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Additional qualitative results of Ground3D-LMM on a six-turn 3D conversation. Note that multi-turn [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results of Ground3D-LMM on all object-level tasks. (Zoom in for better visualization.) [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results of Ground3D-LMM on all part-level tasks. (Zoom in for better visualization.) [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 6 linked inside Pith

  1. [1]

    Referit3D: Neural listeners for fine-grained 3D object identification in real-world scenes

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. Referit3D: Neural listeners for fine-grained 3D object identification in real-world scenes. InEuropean conference on computer vision, pages 422–440. Springer, 2020

    2020

  2. [2]

    Kestrel: 3D multimodal llm for part-aware grounded description

    Mahmoud Ahmed, Junjie Fei, Jian Ding, Eslam Mohamed Bakr, and Mohamed Elhoseiny. Kestrel: 3D multimodal llm for part-aware grounded description. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8973–8983, 2025

    2025

  3. [3]

    Scanqa: 3D question answering for spatial scene understanding

    Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3D question answering for spatial scene understanding. Inproceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19129–19139, 2022

    2022

  4. [4]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    arXiv 2025

  5. [5]

    OPEN-YOLO 3D: Towards fast and accurate open-vocabulary 3D instance segmentation

    Mohamed El Amine Boudjoghra, Angela Dai, Jean Lahoud, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, and Fahad Shahbaz Khan. OPEN-YOLO 3D: Towards fast and accurate open-vocabulary 3D instance segmentation. In13th International Conference on Learning Representations, ICLR 2025, pages 26618–26631. International Conference on Learning Representations, I...

    2025

  6. [6]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465, 2024

    2024

  7. [7]

    Scanrefer: 3D object localization in rgb-d scans using natural language

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3D object localization in rgb-d scans using natural language. InECCV, pages 202–221. Springer, 2020

    2020

  8. [8]

    SD-VLM: Spatial measuring and understanding with depth-encoded vision-language models.arXiv preprint arXiv:2509.17664, 2025

    Pingyi Chen, Yujing Lou, Shen Cao, Jinhui Guo, Lubin Fan, Yue Wu, Lin Yang, Lizhuang Ma, and Jieping Ye. SD-VLM: Spatial measuring and understanding with depth-encoded vision-language models.arXiv preprint arXiv:2509.17664, 2025

    arXiv 2025

Show all 49 references
  1. [9]

    SpatialRGPT: Grounded spatial reasoning in vision-language models.Advances in Neural Information Processing Systems, 37:135062–135093, 2024

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. SpatialRGPT: Grounded spatial reasoning in vision-language models.Advances in Neural Information Processing Systems, 37:135062–135093, 2024

    2024

  2. [10]

    4D spatio-temporal convnets: Minkowski con- volutional neural networks

    Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4D spatio-temporal convnets: Minkowski con- volutional neural networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3075–3084, 2019

    2019

  3. [11]

    Spconv: Spatially sparse convolution library

    Spconv Contributors. Spconv: Spatially sparse convolution library. https://github.com/traveller59/ spconv, last accessed 2026/06/26, 2022

    2026

  4. [12]

    Scannet: Richly-annotated 3D reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3D reconstructions of indoor scenes. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017

    2017

  5. [13]

    MM-Spatial: Exploring 3D spatial understanding in multimodal LLMs, 2025

    Erik Daxberger, Nina Wenzel, David Griffiths, Haiming Gang, Justin Lazarow, Gefen Kohavi, Kai Kang, Marcin Eichner, Yinfei Yang, Afshin Dehghan, and Peter Grasch. MM-Spatial: Exploring 3D spatial understanding in multimodal LLMs, 2025

    2025

  6. [14]

    Segpoint: Segment any point cloud via large language model

    Shuting He, Henghui Ding, Xudong Jiang, and Bihan Wen. Segpoint: Segment any point cloud via large language model. InECCV, pages 349–367. Springer, 2024

    2024

  7. [15]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InICLR, 2022. 12

    2022

  8. [16]

    MLLM-For3D: Adapting multimodal large language model for 3D reasoning segmentation

    Jiaxin Huang and et al. MLLM-For3D: Adapting multimodal large language model for 3D reasoning segmentation. InNeurIPS, 2025

    2025

  9. [17]

    Reason3D: Searching and reasoning 3D segmentation via large language model.arXiv preprint arXiv:2405.17427, 2024

    Kuan-Chih Huang, Xiangtai Li, Lu Qi, Shuicheng Yan, and Ming-Hsuan Yang. Reason3D: Searching and reasoning 3D segmentation via large language model.arXiv preprint arXiv:2405.17427, 2024

    2024 arXiv

  10. [18]

    Text-guided graph neural networks for referring 3D instance segmentation.AAAI, 35(2):1610–1618, May 2021

    Pin-Hao Huang, Han-Hung Lee, Hwann-Tzong Chen, and Tyng-Luh Liu. Text-guided graph neural networks for referring 3D instance segmentation.AAAI, 35(2):1610–1618, May 2021

    2021

  11. [19]

    Openclip

    Gabriel Ilharco, Mitchell Wortsman, and et al. Openclip. https://github.com/mlfoundations/open_ clip, last accessed 2026/06/26, 2021

    2026

  12. [20]

    Bottom up top down detection transformers for language grounding in images and point clouds

    Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, and Katerina Fragkiadaki. Bottom up top down detection transformers for language grounding in images and point clouds. arXiv preprint arXiv:2112.08879, 2021

    2021 arXiv

  13. [21]

    Intent3D: 3D object detection in rgb-d scans based on human intention.arXiv preprint arXiv:2405.18295, 2024

    Weitai Kang, Mengxue Qu, Jyoti Kini, Yunchao Wei, Mubarak Shah, and Yan Yan. Intent3D: 3D object detection in rgb-d scans based on human intention.arXiv preprint arXiv:2405.18295, 2024

    2024 arXiv

  14. [22]

    Paris3D: Reasoning-based 3D part segmentation using large multimodal model

    Amrin Kareem, Jean Lahoud, and Hisham Cholakkal. Paris3D: Reasoning-based 3D part segmentation using large multimodal model. InEuropean Conference on Computer Vision, pages 466–482. Springer, 2024

    2024

  15. [23]

    Seeground: See and ground for zero-shot open-vocabulary 3D visual grounding

    Rong Li, Shijie Li, Lingdong Kong, Xulei Yang, and Junwei Liang. Seeground: See and ground for zero-shot open-vocabulary 3D visual grounding. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 3707–3717, 2025

    2025

  16. [24]

    Instance segmentation in 3D scenes using semantic superpoint tree networks

    Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan, and Kui Jia. Instance segmentation in 3D scenes using semantic superpoint tree networks. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2783–2792, 2021

    2021

  17. [25]

    Partslip: Low-shot part segmentation for 3D point clouds via pretrained image-language models

    Minghua Liu, Yinhao Zhu, Hong Cai, Shizhong Han, Zhan Ling, Fatih Porikli, and Hao Su. Partslip: Low-shot part segmentation for 3D point clouds via pretrained image-language models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2173...

    2023

  18. [26]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  19. [27]

    SQA3D: Situated question answering in 3D scenes

    Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. SQA3D: Situated question answering in 3D scenes. InInternational Conference on Learning Representations, 2023

    2023

  20. [28]

    Umap: Uniform manifold approximation and projection for dimension reduction.arXiv preprint arXiv:1802.03426, 2018

    Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction.arXiv preprint arXiv:1802.03426, 2018

    2018 arXiv

  21. [29]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In2016 fourth international conference on 3D vision (3DV), pages 565–571. IEEE, 2016

    2016

  22. [30]

    Open3DIS: Open-vocabulary 3D instance segmentation with 2D mask guidance

    Phuc Nguyen, Tuan Duc Ngo, Evangelos Kalogerakis, Chuang Gan, Anh Tran, Cuong Pham, and Khoi Nguyen. Open3DIS: Open-vocabulary 3D instance segmentation with 2D mask guidance. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4018–4028, 2024

    2024

  23. [31]

    GPT-4 technical report, 2023

    OpenAI. GPT-4 technical report, 2023

    2023

  24. [32]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, and et al. Learning transferable visual models from natural language supervision. InICML, 2021

    2021

  25. [33]

    SAM 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. SAM 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    2024 arXiv

  26. [34]

    Grounded SAM: Assembling open-world models for diverse visual tasks, 2024

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded SAM: Assembling open-world models for diverse visual tasks, 2024

    2024

  27. [35]

    Efficient 3D semantic segmentation with superpoint transformer

    Damien Robert, Hugo Raguet, and Loic Landrieu. Efficient 3D semantic segmentation with superpoint transformer. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 13

    2023

  28. [36]

    LAION-5B: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION...

    2022

  29. [37]

    N3D-VLM: Native 3D grounding enables accurate spatial reasoning in vision-language models

    Yuxin Wang, Lei Ke, Boqiang Zhang, Tianyuan Qu, Hanxun Yu, Zhenpeng Huang, Meng Yu, Dan Xu, and Dong Yu. N3D-VLM: Native 3D grounding enables accurate spatial reasoning in vision-language models. arXiv preprint arXiv:2512.16561, 2025

    2025

  30. [38]

    3D-STMN: Dependency-driven superpoint-text matching network for end-to-end 3D referring expression segmentation

    Changli Wu, Yiwei Ma, Qi Chen, Haowei Wang, Gen Luo, Jiayi Ji, and Xiaoshuai Sun. 3D-STMN: Dependency-driven superpoint-text matching network for end-to-end 3D referring expression segmentation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages ...

    2024

  31. [39]

    Eda: Explicit text-decoupling and dense alignment for 3D visual grounding

    Yanmin Wu, Xinhua Cheng, Renrui Zhang, Zesen Cheng, and Jian Zhang. Eda: Explicit text-decoupling and dense alignment for 3D visual grounding. InCVPR, pages 19231–19242, 2023

    2023

  32. [40]

    S 2-MLLM: Boosting spatial reasoning capability of mllms for 3D visual grounding with structural guidance.arXiv preprint arXiv:2512.01223, 2025

    Beining Xu, Siting Zhu, Zhao Jin, Junxian Li, and Hesheng Wang. S 2-MLLM: Boosting spatial reasoning capability of mllms for 3D visual grounding with structural guidance.arXiv preprint arXiv:2512.01223, 2025

    2025

  33. [41]

    A unified framework for 3D scene understanding.Advances in Neural Information Processing Systems, 37:59468–59490, 2024

    Wei Xu, Chunsheng Shi, Sifan Tu, Xin Zhou, Dingkang Liang, and Xiang Bai. A unified framework for 3D scene understanding.Advances in Neural Information Processing Systems, 37:59468–59490, 2024

    2024

  34. [42]

    Scannet++: A high-fidelity dataset of 3D indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3D indoor scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023

    2023

  35. [43]

    Inst3d-lmm: Instance-aware 3D scene understanding with multi-modal instruction tuning

    Hanxun Yu, Wentong Li, Song Wang, Junbo Chen, and Jianke Zhu. Inst3d-lmm: Instance-aware 3D scene understanding with multi-modal instruction tuning. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 14147–14157, 2025

    2025

  36. [44]

    Instancerefer: Cooperative holistic understanding for visual grounding on point clouds through instance multi-level contextual referring

    Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Sheng Wang, Zhen Li, and Shuguang Cui. Instancerefer: Cooperative holistic understanding for visual grounding on point clouds through instance multi-level contextual referring. InProceedings of the IEEE/CVF International Confer...

    2021

  37. [45]

    MM1.5: Methods, analysis & insights from multimodal LLM fine-tuning, 2024

    Haotian Zhang, Mingfei Gao, Zhe Gan, Philipp Dufter, Nina Wenzel, Forrest Huang, Dhruti Shah, Xianzhi Du, Bowen Zhang, Yanghao Li, Sam Dodge, Keen You, Zhen Yang, Aleksei Timofeev, Mingze Xu, Hong-You Chen, Jean-Philippe Fauconnier, Zhengfeng Lai, Haoxuan You, Zirui Wang, Afsh...

    2024

  38. [46]

    near the door

    Yiming Zhang, ZeMing Gong, and Angel X Chang. Multi3drefer: Grounding text description to multiple 3D objects. InICCV, pages 15225–15236, 2023. 14 Supplementary Material of Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM A Semantic Novelty Beyond t...

    2023

  39. [47]

    One about the *overall size* (length x width x thickness)

  40. [48]

    One about *thickness*

  41. [49]

    to the left of the table

    One about *length* Keep variety in phrasing across objects/frames, but the numeric values must be copied exactly from the Compact Object Context. Do NOT invent or modify numbers. The context includes an object_id for the labeled/bounded instance in the scene. Use it ONLY inter...

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.