Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that spatial reasoning can be bolted onto any neural 3D representation by decomposing language with an LLM and querying a hierarchical feature field.

desk verdict Solid engineering combination with a clean LLM-driven decomposition idea, but the self-referential SAM-based evaluation means the headline numbers are not yet trustworthy evidence of spatial reasoning. read the letter →

arxiv 2507.06719 v1 pith:G4KS7G33 submitted 2025-07-09 cs.CV cs.RO

Zhenyang Liu , Sixiao Zheng , Siyu Chen , Cairong Zhao , Longfei Liang , Xiangyang Xue , Yanwei Fu This is my paper
classification cs.CVcs.RO
keywords Open-vocabulary3DvisualgroundingSpatialreasoningLanguagefieldsNeuralrepresentationLLMquerydecompositionHierarchicalfeaturefieldpropertiesInstance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes SpatialReasoner, a framework that gives neural 3D scene representations the ability to answer spatial-language queries such as "the book on the chair." Its central claim is that language-field methods fail on such queries because they lack spatial reasoning on both sides of the problem: CLIP cannot reliably parse implicit relations in long queries, and the scene field stores only object semantics, not the geometry and appearance that distinguish one instance from another. SpatialReasoner addresses the first gap by fine-tuning an LLM to decompose a query into target, anchor, and spatial relation, and the second by building a hierarchical feature field that mixes CLIP language features with instance features and is augmented by opacity and color. On the extended LERF spatial-reasoning benchmark, integrating the framework into NeRF, Instant-NGP, and 3DGS raises overall localization accuracy to 83.4, 87.8, and 91.7 percent, respectively, roughly double the previous language-field methods. A sympathetic reader would take the contribution as evidence that spatial reasoning can be layered on top of existing neural representations rather than requiring a new representation.

What carries the argument

The load-bearing mechanism is the combination of an LLM-based query decomposer and a visual-properties-enhanced hierarchical feature field. The decomposer turns a free-form query into three instructions: target category, anchor category, and spatial relation, so that the vision side never has to infer the relation from raw CLIP text matching. The hierarchical field contains a language field and an instance field; both map 3D position, physical scale, opacity, and color to embeddings, and the instance field is trained with a margin-based contrastive loss so that objects sharing the same semantic label, such as two books, receive distinct instance embeddings. An instance graph then merges candidate regions before the spatial relation is checked, and the four supported relation types are Horizontal Proximity, Vertical Proximity, Support, and Allocentric relations.

What would settle it

Recompute the extended LERF spatial-reasoning accuracy using human-annotated 3D bounding boxes as ground truth while keeping the proposed method unchanged; if the gap over the language-field baselines narrows substantially or vanishes, the result rests on shared mask supervision rather than on spatial reasoning.

Watch

Extended reading notes

Core claim

The central claim is that existing language-field methods localize objects directly from the whole query and therefore fail at spatial relations, and that the failure can be fixed by adding spatial reasoning in both the language and the scene. SpatialReasoner fine-tunes a small LLM to parse a query into target, anchor, and relation instructions, and constructs a hierarchical feature field with a language field and an instance field, both conditioned on visual properties—opacity and color—extracted from the neural reconstruction. Querying these fields hierarchically activates candidate targets and anchors, an instance graph merges related candidates, and the spatial relation is explicitly verified to pick the final instance. The paper reports that this design integrates into NeRF, Instant-NGP, and 3DGS and reaches overall localization accuracies of 83.4, 87.8, and 91.7 percent on the extended LERF spatial-reasoning benchmark, outperforming the language-field baselines it compares against.

Load-bearing premise

The evaluation's ground-truth boxes on the extended LERF dataset are produced by the automatic segmentation model on rendered novel views, and the same segmentation model generates the masks that supervise the proposed fields, so the comparison assumes those automatic masks are an unbiased measure of correct localization.

Editorial extensions

If this is right

  • Integrating SpatialReasoner into NeRF, Instant-NGP, or 3DGS yields localization accuracy of 83.4, 87.8, and 91.7 percent on the extended LERF spatial-reasoning benchmark, roughly double the best language-field baseline.
  • Because the framework only needs posed images plus the same neural representation already used for reconstruction, it brings spatial-relation grounding to any scene that can be reconstructed as a radiance field or Gaussian splatting.
  • Separating query parsing from scene reasoning means the language-side LLM can be improved or swapped without retraining the 3D fields, and vice versa.
  • The contrastively supervised instance field is what lets the method distinguish identical objects such as two books, so relational grounding no longer collapses to category matching.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same decompose-then-query design could be transferred to 2D referring segmentation or video object grounding, where relational phrases such as "left of" or "behind" also break pure CLIP feature matching.
  • A testable extension the paper leaves implicit is a full ablation of the LLM: replacing it with a fixed rule-based parser would show how much of the gain comes from language-side decomposition versus scene-side hierarchical fields.
  • The shared use of the automatic segmentation model for both supervision and evaluation suggests the strongest next check is a human-annotated spatial-relation benchmark, which would separate genuine spatial understanding from segmentation bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes SpatialReasoner, a framework for open-vocabulary 3D visual grounding that augments neural representation--based language fields with an LLM fine-tuned to decompose queries into target, anchor, and spatial relation, and a hierarchical feature field (language plus instance) that incorporates opacity and color as visual properties and SAM masks as supervision. The pipeline is instantiated on NeRF, Instant-NGP, and 3DGS and evaluated on an extended LERF dataset, the Replica dataset, and a newly introduced Re3D dataset. The central claim is that adding LLM-driven spatial relation decomposition and a visual-properties-enhanced hierarchical feature field lets language-field methods outperform prior work on spatial-relation queries, with reported accuracies of 83.4%, 87.8%, and 91.7% on the extended LERF benchmark for the three backbones.

Significance. If the claims were established, the contribution would be practically useful: it offers a modular upgrade path for existing NeRF/3DGS language fields to handle spatial queries, and the decomposition into target, anchor, and relation is a plausible design. The paper also reports experiments across three backbones and includes ablations of visual properties, the instance field, and the instance graph. However, the evaluation is currently not independent of the training signal, and the spatial-relation verification mechanism is unspecified. These issues must be addressed before the empirical claims can be taken as evidence for the advertised spatial reasoning capability.

major comments (3)
  1. [Sec. 4.1 and Sec. 3.3] The extended LERF evaluation is confounded with the method's training signal. Section 4.1 states that ground-truth labels for the extended LERF benchmark are generated by rendering novel views and using SAM. Section 3.3 uses the same SAM automatic mask generator to create the mask supervision for the language field (Eq. 4) and the instance field (Eq. 5), and to compute physical scales used in candidate generation. As a result, the reported 83.4--91.7% accuracies in Table 1 and 85.3--92.8% mIoU values in Table 2 largely measure how well the method reproduces the mask family it was optimized to fit, not whether it resolves spatial relations such as "the book on the chair." Because no human-annotated ground truth or standard benchmark such as ScanRefer, Sr3D, or Nr3D is reported, the quantitative support for the central spatial-reasoning claim is currently undermined. Please re-evaluate with independent human-annotated boxes or masks, or on an existing benchmark, and report results separately for each of the four spatial relations.
  2. [Sec. 3.4] The core mechanism for verifying spatial relations is not described. The text states that SpatialReasoner "considers four spatial relations (Horizontal Proximity, Vertical Proximity, Support, Allocentric)" and that "the specific object is determined accordingly," but no algorithm, thresholds, geometric tests, or decision rules are given. Equation (6) similarly defines the instance graph edge set through an unspecified threshold on affine differences, without stating how the threshold is chosen or how connected components are converted into final candidates. Since spatial relation verification is the load-bearing component for the central claim, this under-specification prevents the reader from assessing whether the method performs spatial reasoning in 3D. Please provide the full procedure, including parameters, pseudocode, and a failure analysis.
  3. [Sec. 3.2 and Sec. 4.1] The paper fine-tunes an LLM on Sr3D and Sr3D++ and mentions that the number of instructions is set to 3 for those benchmarks, but it reports no quantitative results on these human-annotated datasets or on ScanRefer/Nr3D. The newly introduced Re3D dataset is captured with the same Polycam pipeline and evaluated with the same protocol as the extended LERF benchmark, so it does not break the SAM circularity. Adding results on at least one standard human-annotated 3D visual grounding benchmark is necessary to demonstrate that the reported gains are not an artifact of the evaluation protocol. The paper should also report LLM decomposition accuracy on held-out queries and describe the fine-tuning data and instruction templates to rule out trivial overlap with the test queries.
minor comments (6)
  1. [Figures 2 and 3] There are typos in the figure text: "mask-extraceted" in Figure 2 and "localozation" in Figure 3; these should be corrected.
  2. [Eq. (6)] The notation in Eq. (6) is unclear: the dimension of V, the meaning of the tensor product, and the norm used should be defined explicitly, and the threshold for defining edges from the affine differences should be stated.
  3. [Sec. 3.4] The phrase "the number of instructions (n) is adapted per benchmark" is not instantiated for LERF, Replica, or Re3D; please specify n and the actual instruction templates for each dataset.
  4. [Sec. 3.2 and Sec. 4.1] The method description refers to fine-tuning ChatGPT, while the implementation uses TinyLlama; this discrepancy should be reconciled.
  5. [Table 4] The claim that the method improves performance "without significantly increasing the inference cost" is not supported for the NeRF variant: SpatialReasoner(NeRF) has a single-view activation time of 1.43 s, more than 35 times slower than LangSplat's 0.04 s.
  6. [Sec. 4.1] The metric definitions should specify how the rendered relevance map is thresholded to compute mIoU against the annotated ground-truth mask, and the Replica head/common/tail split should be described with dataset statistics.

Circularity Check

1 steps flagged · score 6.0 of 10

Extended-LERF evaluation is self-referential: SAM provides both the mask supervision and the ground-truth boxes, so the reported spatial-reasoning gains partly measure SAM-mask fit.

  1. fitted input called prediction [Sec. 4.1 (Datasets, LERF) and Sec. 3.3 (Supervision Generation, Eq. 5)]
    "To assess localization performance under spatial language queries, we enhance LERF by rendering novel views and generating ground truth labels using the Segment Anything Model (SAM). ... SpatialReasoner first utilizes the automatic mask generator of SAM [14] to generate object masks from training views. ... Localization accuracy considers a label a success if the highest relevance pixel falls inside the annotated box."

    The extended-LERF 'annotated' boxes/masks are produced by SAM, and the same automatic SAM mask generator supplies the supervision for the proposed method: SAM masks are used to extract CLIP features, compute physical scales, and define the mask-contrastive instance loss (Eq. 5). The metric then counts a hit when the highest-relevance pixel falls inside a SAM-generated box, and mIoU overlaps the rendered relevance map with SAM masks. Hence a model optimized to concentrate relevance within SAM mask interiors on training views is rewarded on novel views for reproducing that same mask family, independent of whether it actually resolved the spatial relation (e.g., which book is on the chair).

full rationale

The quantitative support for the central claim rests on Tables 1-5, but the main benchmark is confounded in a way that makes part of the reported gain circular. The paper's own Sec. 4.1 states that ground-truth labels on the extended LERF are generated by SAM, and Sec. 3.3 states that SAM's automatic mask generator supplies the masks used to supervise the language and instance fields, including the mask-contrastive loss in Eq. 5. Because the metric counts a success when the highest-relevance pixel falls inside the SAM-generated annotated box, and mIoU is measured against SAM masks, the evaluation rewards the very mask distribution the method was optimized to fit. This makes the advantage over LangSplat and LERF on LERF partly self-referential. I do not find a load-bearing self-citation chain: ReasonGrounder is a related-work citation, not a source of the framework's validity, and no uniqueness theorem is imported. The LLM decomposition and the geometric spatial-relation verification are genuine, non-circular components, so the central mechanism is not entirely equivalent to its input. However, the absence of human-annotated ground-truth benchmarks (e.g., ScanRefer, Sr3D, or Nr3D) means the central spatial-reasoning claim is supported mainly by the SAM-vs-SAM evaluation. Score 6 reflects a partial circularity in the evaluation design, not a wholesale fabricated derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on pretrained foundation models (CLIP, SAM, TinyLlama), standard volume rendering, and a handcrafted four-way spatial relation taxonomy. The unstated thresholds for the instance graph and spatial rules are free parameters that materially affect results. No new physical entities are introduced.

free parameters (4)
  • instance graph edge threshold
    Edges in the instance graph are defined based on pairwise distances A (Eq. 6), but the threshold for edge creation is never stated, so the merging behavior is a hand-chosen value not reported.
  • spatial relation rule thresholds
    The four spatial relations (Horizontal Proximity, Vertical Proximity, Support, Allocentric) are applied without specifying the geometric thresholds or test procedures used to decide whether two candidate boxes satisfy the relation.
  • instruction count n = 3 for Sr3D/Sr3D++
    The number of decomposed instructions is adapted per benchmark; for Sr3D it is set to 3 (target, anchor, relation). This is a hand-set design choice.
  • instance loss margin lambda_in
    The margin constant in Eq. (5) controls how far apart different instances must be in embedding space; its value is not reported.
assumptions (5)
  • standard math Volume rendering integration for feature fields
    The method relies on the volume rendering equation (Eq. 1 and Eq. 4) to aggregate point features into pixel features; this is standard in NeRF literature.
  • domain assumption CLIP text-image embedding alignment
    The language field assumes CLIP embeddings of query phrases and image regions are comparable, so similarity scores (Eq. 2) are meaningful for grounding.
  • domain assumption Depth deprojection accuracy
    Physical scales are obtained by deprojecting SAM mask pixels using depth from the trained neural representation; this assumes the reconstruction depth is accurate enough for scale estimation.
  • ad hoc to paper Four-way spatial relation taxonomy
    The system reduces all possible spatial relations to Horizontal Proximity, Vertical Proximity, Support, and Allocentric, which is a domain-specific closed set not derived from first principles.
  • domain assumption Fine-tuned LLM parses queries correctly
    The method assumes the fine-tuned TinyLlama reliably decomposes arbitrary free-form queries into target, anchor, and relation, even for phrases outside the Sr3D template distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding." pith.science (2026). https://pith.science/paper/G4KS7G33

@misc{pith2026250706719,
  author       = {Pith},
  title        = {Pith review of: A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4KS7G33}},
  note         = {Machine review of arXiv:2507.06719}
}
read the original abstract

Open-vocabulary 3D visual grounding aims to localize target objects based on free-form language queries, which is crucial for embodied AI applications such as autonomous navigation, robotics, and augmented reality. Learning 3D language fields through neural representations enables accurate understanding of 3D scenes from limited viewpoints and facilitates the localization of target objects in complex environments. However, existing language field methods struggle to accurately localize instances using spatial relations in language queries, such as ``the book on the chair.'' This limitation mainly arises from inadequate reasoning about spatial relations in both language queries and 3D scenes. In this work, we propose SpatialReasoner, a novel neural representation-based framework with large language model (LLM)-driven spatial reasoning that constructs a visual properties-enhanced hierarchical feature field for open-vocabulary 3D visual grounding. To enable spatial reasoning in language queries, SpatialReasoner fine-tunes an LLM to capture spatial relations and explicitly infer instructions for the target, anchor, and spatial relation. To enable spatial reasoning in 3D scenes, SpatialReasoner incorporates visual properties (opacity and color) to construct a hierarchical feature field. This field represents language and instance features using distilled CLIP features and masks extracted via the Segment Anything Model (SAM). The field is then queried using the inferred instructions in a hierarchical manner to localize the target 3D instance based on the spatial relation in the language query. Extensive experiments show that our framework can be seamlessly integrated into different neural representations, outperforming baseline models in 3D visual grounding while empowering their spatial reasoning capability.

Figures

Figures reproduced from arXiv: 2507.06719 by the authors.

Figure 1
Figure 1. We propose SpatialReasoner for neural representa￾tion: prior language field methods localize instances directly from complex user queries but fail to capture spatial rela￾tions in both the language query and the environment (left). Our SpatialReasoner instead utilizes a large language model (LLM) and a hierarchical feature field to think and look “step by step” (right). Crucially, reasoning through an LLM—such as sp… view at source ↗
Figure 2
Figure 2. The overall pipeline of SpatialReasoner framework. SpatialReasoner fine-tunes an LLM to decompose language [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparisons of spatial reasoning capability. Results demonstrate that our SpatialReasoner achieves spatial [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons of 3D visual grounding capability. Results demonstrate that our SpatialReasoner achieves [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Ablation Study on visual properties with qualitative [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Ablation Study on instance graph construction with [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. QuASH: Using Natural-Language Heuristics to Query Visual-Language Robotic Maps

    cs.RO 2025-10 conditional novelty 6.0 of 10

    Querying VLM robot maps with an SVM trained on LLM-generated synonym/antonym embeddings outperforms cosine-threshold and single-antonym baselines on images and OpenSeg maps, but not consistently on LSeg maps.

Reference graph

Works this paper leans on

52 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. 2020. Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16 . Springer, 422–440

    2020

  3. [3]

    Robert A Brebin, Loren Carpenter, and Pat Hanrahan. 1998. Volume rendering. In Seminal graphics: pioneering efforts that shaped the field . 363–372

    work page 1998

  4. [4]

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. 2020. Scanrefer: 3d object localization in rgb-d scans using natural language. In European conference on computer vision. Springer, 202–221

    2020

  5. [5]

    Nianchen Deng, Zhenyi He, Jiannan Ye, Budmonde Duinkharjav, Praneeth Chakravarthula, Xubo Yang, and Qi Sun. 2022. Fov-nerf: Foveated neural radi- ance fields for virtual reality. IEEE Transactions on Visualization and Computer Graphics 28, 11 (2022), 3854–3864

    work page 2022

  6. [6]

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. 2022. Scaling open- vocabulary image segmentation with image-level labels. In European Conference on Computer Vision. Springer, 540–557

    2022

  7. [7]

    Michael A Goodrich, Alan C Schultz, et al. 2008. Human–robot interaction: a survey. Foundations and Trends ® in Human–Computer Interaction 1, 3 (2008), 203–275

    work page 2008

  8. [8]

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. 2021. Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921 (2021)

    arXiv 2021

Show all 52 references
  1. [9]

    Huy Ha and Shuran Song. 2022. Semantic abstraction: Open-world 3d scene understanding from 2d vision-language models. arXiv preprint arXiv:2207.11514 (2022)

    2022 arXiv

  2. [10]

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 2023. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems 36 (2023), 20482– 20494

    2023

  3. [11]

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis

  4. [12]

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. 2023. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 19729–19739

    2023

  5. [13]

    Chung Min Kim, Mingxuan Wu, Justin Kerr, Ken Goldberg, Matthew Tancik, and Angjoo Kanazawa. 2024. GARField: Group Anything with Radiance Fields. arXiv preprint arXiv:2401.09419 (2024)

    2024 arXiv

  6. [14]

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al

  7. [15]

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al

  8. [16]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4015–4026

  9. [17]

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. 2023. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

    2023

  10. [18]

    Haitao Lin, Yanwei Fu, and Xiangyang Xue. 2023. PourIt!: Weakly-supervised Liquid Perception from a Single Image for Visual Closed-Loop Robotic Pouring. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 241– 251

    2023

  11. [19]

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and René Ranftl

  12. [20]

    Zhenyang Liu, Yikai Wang, Sixiao Zheng, Tongying Pan, Longfei Liang, Yanwei Fu, and Xiangyang Xue. 2025. ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning. arXiv preprint arXiv:2503.23297 (2025)

    2025 arXiv

  13. [21]

    Shiyang Lu, Haonan Chang, Eric Pu Jing, Abdeslam Boularias, and Kostas Bekris

  14. [22]

    Timo Lüddecke and Alexander Ecker. 2022. Image segmentation using text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7086–7096

    2022

  15. [23]

    Yichen Liu, Benran Hu, Junkai Huang, Yu-Wing Tai, and Chi-Keung Tang. 2023. Instance neural radiance field. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision . 787–796

    2023

  16. [24]

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 1 (2021), 99–106

    2021

  17. [25]

    Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. In- stant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG) 41, 4 (2022), 1–15

    2022

  18. [26]

    In Conference on Robot Learning

    Ovir-3d: Open-vocabulary 3d instance retrieval without training on 3d data. In Conference on Robot Learning . PMLR, 1610–1620

  19. [27]

    Jelena Novosel, Prashanth Viswanath, and Bruno Arsenali. 2019. Boosting se- mantic segmentation with multi-task self-supervised learning for autonomous driving applications. In Proc. of NeurIPS-Workshops, Vol. 3

    2019

  20. [28]

    Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. 2016. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 11–20

    2016

  21. [29]

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister

  22. [30]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  23. [31]

    Simon Niedermayr, Josef Stumpfegger, and Rüdiger Westermann. 2024. Com- pressed 3d gaussian splatting for accelerated novel view synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10349– 10358

    2024

  24. [32]

    Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, and Sergey Levine. 2021. Ving: Learning open-world navigation with visual goals. In 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 13215– 13222

    2021

  25. [33]

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. 2023. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 815–824

    2023

  26. [34]

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. 2019. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797 (2019)

    2019 arXiv

  27. [35]

    LangSplat: 3D Language Gaussian Splatting.arXiv preprint arXiv:2312.16084 (2023)

    2023 arXiv

  28. [36]

    Suhani Vora, Noha Radwan, Klaus Greff, Henning Meyer, Kyle Genova, Mehdi SM Sajjadi, Etienne Pot, Andrea Tagliasacchi, and Daniel Duckworth. 2021. Nesf: Neural semantic fields for generalizable semantic segmentation of 3d scenes. arXiv preprint arXiv:2111.13260 (2021)

    2021 arXiv

  29. [37]

    Fabio Remondino, Ali Karami, Ziyang Yan, Gabriele Mazzacca, Simone Rigon, and Rongjun Qin. 2023. A critical analysis of NeRF-based 3D reconstruction. Remote Sensing 15, 14 (2023), 3585

    2023

  30. [38]

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, and Shalini De Mello. 2023. Open-vocabulary panoptic segmentation with text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2955–2966

    2023

  31. [39]

    Thomas B Sheridan. 2016. Human–robot interaction: status and challenges. Human factors 58, 4 (2016), 525–532

    2016

  32. [40]

    Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Sheng Wang, Zhen Li, and Shuguang Cui. 2021. Instancerefer: Cooperative holistic understanding for visual grounding on point clouds through instance multi-level contextual referring. In Proceedings of the IEEE/CVF International...

    2021

  33. [41]

    Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar. 2022. Block- nerf: Scalable large scene neural view synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern ...

    2022

  34. [42]

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024. Tinyllama: An open-source small language model. arXiv preprint arXiv:2401.02385 (2024)

    2024 arXiv

  35. [43]

    Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. 2022. Cris: Clip-driven referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 11686–11695

    2022

  36. [44]

    Luowei Zhou, Yannis Kalantidis, Xinlei Chen, Jason J Corso, and Marcus Rohrbach

  37. [45]

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg

  38. [48]

    Hao Zhang, Fang Li, and Narendra Ahuja. 2024. Open-NeRF: Towards Open Vo- cabulary NeRF Decomposition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 3456–3465

    2024

  39. [50]

    Xiaoshuai Zhang, Sai Bi, Kalyan Sunkavalli, Hao Su, and Zexiang Xu. 2022. Nerfu- sion: Fusing radiance fields for large-scale scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 5449–5458

    2022

  40. [2016]

    In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14

    Modeling context in referring expressions. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 . Springer, 69–85

    2016

  41. [2017]

    International journal of computer vision 123 (2017), 32–73

    Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision 123 (2017), 32–73

    2017

  42. [2019]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Grounded video description. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 6578–6587

  43. [2022]

    arXiv preprint arXiv:2201.03546 (2022)

    Language-driven semantic segmentation. arXiv preprint arXiv:2201.03546 (2022)

    2022 arXiv

  44. [2023]

    ACM Transac- tions on Graphics 42, 4 (2023), 1–14

    3d gaussian splatting for real-time radiance field rendering. ACM Transac- tions on Graphics 42, 4 (2023), 1–14

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.