Pith. sign in

REVIEW 3 major objections 5 minor 53 references

Towards Terrain-Aware Task-Driven 3D Scene Graph Generation in Outdoor Environments

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a single metric-semantic point cloud built from LiDAR and camera data can be prompted with natural-language queries to generate task-driven, terrain-aware 3D scene graphs, and reports qualitative campus results as a…

desk verdict A useful workshop-level feasibility demo of terrain-aware outdoor 3DSGs; the terrain layer is closed-set, not open-set, which tempers the 'task-agnostic open-set' claim but doesn't sink the core graph-generation idea. read the letter →

arxiv 2506.06562 v1 pith:XSWD7IFG submitted 2025-06-06 cs.RO

Chad R Samuelson , Timothy W McLain , Joshua G Mangelson This is my paper
classification cs.RO
keywords 3Dscenegraphsoutdoorroboticsmetric-semanticmappingCLIPembeddingsterrainsegmentationgeneralizedVoronoidiagramtask-drivenopen-setclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper makes an early attempt to bring 3D scene graphs, so far mostly an indoor technology, to outdoor environments. Its central claim is that a single task-agnostic metric-semantic point cloud, built once from LiDAR and camera data, can later be queried with natural-language prompts to generate task-driven 3D scene graphs that include both object nodes and terrain-aware place nodes. The paper demonstrates the bottom three layers of a planned five-layer graph on a campus scene: a metric-semantic point cloud, prompted object bounding boxes, and per-terrain generalized Voronoi diagrams that act as place nodes. The reported evidence is qualitative, but the authors argue it establishes feasibility for outdoor deployment.

What carries the argument

The load-bearing mechanism is cosine similarity between CLIP text-prompt embeddings and per-point accumulated CLIP image embeddings, with fixed thresholds selecting object and terrain points. For place nodes, a generalized Voronoi diagram (GVD) is computed per terrain type from a 2D binary grid via the brushfire algorithm, then pruned and refined through node insertion and edge splitting, following the Hydra-style procedure; the resulting GVD nodes and edges serve as the terrain-aware place-node layer.

What would settle it

Run the full pipeline on a held-out outdoor scene with per-point ground-truth labels, and compute precision/recall or mean intersection-over-union for the prompted classes (trees, cars; sidewalk, grass, asphalt). If the cosine-similarity thresholding yields near-chance separation, or if the GVD place nodes do not align with the actual drivable terrain in a qualitative map overlay, the feasibility claim is refuted.

Watch

Extended reading notes

Core claim

The paper claims that an outdoor 3D scene graph can be produced in two phases. In phase one, a metric-semantic point cloud is built by fusing a LiDAR-inertial SLAM trajectory with CLIP embeddings of FastSAM segment masks and fine-tuned YOLOv11 terrain labels, storing an averaged semantic embedding per point. In phase two, a task prompt is embedded with CLIP's text encoder, and points whose cosine similarity exceeds α=0.28 for objects or 0.95 for terrain are assigned to the corresponding class; DBSCAN clustering turns object points into bounding-box nodes, while per-terrain binary grids are processed with a brushfire distance transform into a generalized Voronoi diagram whose nodes and edges become terrain-aware place nodes. Qualitative results show detected trees and cars and GVD place nodes for sidewalk, grass, and asphalt, supporting the paper's stated claim that outdoor 3DSG generation is feasible.

Load-bearing premise

The load-bearing premise is that CLIP cosine similarity between a point's averaged embedding and the task prompt is a reliable signal for assigning points to semantic classes, with the object threshold α=0.28 and the terrain threshold 0.95 held fixed; if those similarities do not cleanly separate outdoor categories, both the object nodes and the GVD place nodes are built from mislabeled points.

Editorial extensions

If this is right

  • A single metric-semantic map can be stored once and reused for multiple tasks, since task-specific graphs are generated by prompting rather than by re-mapping.
  • Terrain can be represented as topological place nodes, enabling navigation planners to reason about drivable surfaces rather than raw points.
  • Open-set object queries (e.g., 'tree', 'car') can be answered directly from the point cloud without retraining, using a text prompt.
  • The demonstrated three-layer graph is a step toward the full five-layer hierarchy; region and map nodes are left for future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the two-phase approach scales, it implies a 'map once, query many' paradigm for field robotics in which downstream tasks need never alter the underlying map.
  • The terrain threshold of 0.95 is suspiciously close to 1.0 because the terrain prompts reuse the exact YOLO labels; this suggests terrain semantics may not be truly open-set, and the GVD layer inherits any bias in the terrain classifier.
  • A testable extension would be to measure how many camera views are needed for the per-point averaged embedding to stabilize, and whether an uncertainty-aware threshold performs better than the fixed α values.
  • The GVD place-node construction could transfer to other terrain-like classes (dirt trails, puddles, curbs) if open-set label embeddings are available, connecting directly to off-road navigation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents a two-phase pipeline for building outdoor 3D scene graphs: first, a task-agnostic metric-semantic point cloud is constructed by fusing LiDAR and camera data, associating CLIP embeddings with each point; second, given a task, the point cloud is prompted to extract object nodes (via cosine similarity and DBSCAN clustering) and terrain-aware place nodes (via per-terrain 2D generalized Voronoi diagrams). The method is evaluated on a campus outdoor scene with qualitative results showing object detections and terrain-colored point clouds, integrated into a three-level scene graph. The central claim is that this demonstrates preliminary feasibility of outdoor, task-driven, terrain-aware 3DSGs.

Significance. If the approach were fully validated, the idea of a reusable task-agnostic metric-semantic map for outdoor 3DSGs would be a valuable contribution, and the use of GVD-based place nodes for terrain is a sensible extension of indoor methods. The system design is clean, and the paper honestly reports several failure modes (missed objects, duplicate boxes, unclassified points). However, the current evidence is only qualitative, and the terrain layer is not actually open-set: it reuses the exact closed-set YOLOv11 labels stored at mapping time. Thus the two headline contributions—open-set task-agnostic mapping and terrain-aware place nodes—are only partially demonstrated, and the terrain-aware GVD contribution lacks independent validation.

major comments (3)
  1. [§III-A.2, §IV-C] The terrain semantics are not open-set. In §III-A.2, terrain regions are segmented by a fine-tuned YOLOv11 and represented by CLIP text embeddings of the fixed labels; in §IV-C, the terrain prompts ('sidewalk', 'grass', 'asphalt') are stated to 'exactly match the same CLIP text embeddings from the YOLOv11 model labels,' with a 0.95 threshold chosen to accommodate averaging. This makes the terrain classification and the resulting GVD place nodes of §III-B.3 a self-consistency lookup rather than an open-set query. To support the open-set claim, the authors should demonstrate terrain prompting with prompts not seen during mapping (e.g., 'paved path', 'lawn'), or explicitly reframe the terrain contribution as closed-set place-node generation.
  2. [§IV] The feasibility claim rests entirely on qualitative visualizations (Figs. 4–6). There are no quantitative metrics—no precision/recall, IoU, or graph-accuracy numbers—and no comparison to existing outdoor 3DSG methods (e.g., Greve et al. [26] or Steinke et al. [10]). Adding even basic counts of true/false positive object detections, a manually labeled terrain accuracy score, and a sensitivity analysis of the thresholds (α, 0.95, DBSCAN parameters) would substantially strengthen the load-bearing claim that the pipeline works.
  3. [§IV-C, §III-B.3] The 0.95 terrain threshold is justified by the fact that the prompts are identical to the stored embeddings, and the paper explicitly admits this. This means the reported terrain separation does not establish that the map stores reusable, class-agnostic semantic information. The paper should measure how classification degrades when prompts are paraphrased or combined (e.g., 'sidewalk surface', 'grass area') and should report the distribution of cosine similarities for terrain points versus non-terrain points, so that the choice of 0.95 is not purely anecdotal.
minor comments (5)
  1. [References] Reference [49] is titled 'YOLOv8: A novel object detection algorithm...' but the text repeatedly refers to 'YOLOv11'; please align the reference with the actual model used.
  2. [§III-B.3] 'V oronoi' contains a stray space; please correct to 'Voronoi'.
  3. [§IV-B] In Figure 5, the caption says 'A few failed cases are shown' but does not point to them in the figure; adding markers or arrows would make the qualitative claims easier to verify.
  4. [§III-B.3] The GVD refinement thresholds (deviation distance, max node distance, iteration count) are not specified anywhere; please list their values in the experimental setup section.
  5. [§III-B.4] The AIB algorithm is mentioned without a citation; please provide a reference for the Agglomerative Information Bottleneck method.

Circularity Check

1 steps flagged · score 6.0 of 10

Terrain "open-set" prompting is a closed-set label lookup: the 0.95 cosine prompts exactly match CLIP text embeddings stored from YOLOv11 labels, so the terrain-aware place nodes inherit that circularity; object prompting remains genuinely external.

  1. self definitional [Sec. III-A.2 and Sec. IV-C (Terrain-Aware 3D Scene Graph)]
    "Each terrain text label from the YOLO model is also encoded using CLIP's text encoder to obtain corresponding semantic embeddings. ... To highlight the terrain types, we prompt the point cloud with the class names "sidewalk," "grass," and "asphalt," coloring all matching points with a cosine similarity above 0.95. This high threshold is chosen because the prompts exactly match the same CLIP text embeddings from the YOLOv11 model labels during the metric-semantic point cloud generation phase, but is below 1.0 because of the effects of averaging other CLIP embeddings with these points."

    In Phase 1, terrain point semantics are not class-agnostic: terrain regions are segmented by a fine-tuned YOLOv11 network and represented by CLIP text embeddings of those fixed labels. In Phase 2, the 'terrain prompt' uses the same CLIP text prompts ('sidewalk,' 'grass,' 'asphalt') with a 0.95 cosine threshold that the paper states was chosen precisely because the prompts exactly match the stored YOLOv11 label embeddings. The cosine similarity above 0.95 therefore mostly measures a point's stored embedding against itself (modulo multi-view averaging), so the terrain classification shown in Figs. 4b and 6a is a closed-set label lookup rather than an open-set query. The terrain-aware GVD place nodes (Sec.

full rationale

The derivation chain is mostly self-contained: LIO-SAM provides an independent geometric map, FastSAM generates class-agnostic masks, CLIP supplies external embeddings, DBSCAN clusters points, and the GVD construction is a geometry-only computation. The object queries 'image of a tree' and 'image of a car' are not stored during mapping and therefore provide genuine open-set evidence, even if only qualitatively evaluated. The one significant circular step is the terrain semantic layer. Section III-A.2 stores CLIP text embeddings of the YOLOv11 terrain labels as the semantic content of terrain points, and Section IV-C then 'prompts' with the same class names using a 0.95 threshold that the paper explicitly justifies by noting that the prompts exactly match the same stored CLIP text embeddings. This makes the terrain classification a lookup of labels put in at mapping time rather than an open-set query. Because the paper's second contribution is the terrain-aware place-node layer, and that layer is built from this binary terrain classification, the terrain-aware contribution is not independently demonstrated. The circularity is substantial but partial: the object layer and the geometrical GVD computation retain independent content, and there is no load-bearing self-citation or uniqueness-import chain. Score 6 reflects one central 'prediction' (terrain semantic segmentation) reducing by construction while other components remain independent.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on sensor calibration, CLIP discriminability, mask transfer, and the suitability of 2D GVDs for outdoor place topology. It also depends on several hand-set thresholds; the terrain query is close to self-referential because the same CLIP embeddings stored during mapping are used as queries.

free parameters (5)
  • alpha (object cosine similarity threshold) = 0.28
    Used to assign points to object classes; chosen from qualitative tuning, not derived (Sec. III-B.2, IV-C).
  • terrain cosine similarity threshold = 0.95
    Set near 1.0 because prompts equal stored YOLO label embeddings; accounts for averaging but remains a hand-picked cutoff (Sec. IV-C).
  • DBSCAN parameters (eps, min_samples) = not reported
    Controls object bounding-box clusters; no values or sensitivity analysis given (Sec. III-B.2).
  • GVD refinement thresholds (deviation distance, max node distance, iteration count) = not reported
    Controls where new GVD nodes are inserted; values omitted (Sec. III-B.3).
  • YOLOv11 terrain fine-tuning data and training configuration = not reported
    The terrain classifier is fine-tuned on unspecified data; the trained model is a fitted component (Sec. III-A.2).
assumptions (4)
  • domain assumption Accurate global metric point cloud from LIO-SAM
    The pipeline back-projects and associates semantics using LIO-SAM poses; poor localization or calibration would corrupt point-level labels. Invoked in Sec. III-A.1.
  • domain assumption CLIP embeddings are semantically aligned across images, text, and 3D points
    Open-set object and terrain retrieval relies on cosine similarity between averaged CLIP features and prompts. Invoked in Sec. III-B.2.
  • domain assumption FastSAM masks correspond to coherent objects and their mask-level CLIP embeddings can be transferred to LiDAR points via back-projection
    The paper projects masks into 3D and stores their embeddings per point; it later notes border-projection artifacts (Sec. IV-B).
  • ad hoc to paper A 2D per-terrain GVD is a valid place-node representation for outdoor navigation
    The terrain-aware place layer is defined by this GVD; no experiment connects GVD nodes to navigation outcomes. Sec. III-B.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Terrain-Aware Task-Driven 3D Scene Graph Generation in Outdoor Environments." pith.science (2026). https://pith.science/paper/XSWD7IFG

@misc{pith2026250606562,
  author       = {Pith},
  title        = {Pith review of: Towards Terrain-Aware Task-Driven 3D Scene Graph Generation in Outdoor Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XSWD7IFG}},
  note         = {Machine review of arXiv:2506.06562}
}
read the original abstract

High-level autonomous operations depend on a robot's ability to construct a sufficiently expressive model of its environment. Traditional three-dimensional (3D) scene representations, such as point clouds and occupancy grids, provide detailed geometric information but lack the structured, semantic organization needed for high-level reasoning. 3D scene graphs (3DSGs) address this limitation by integrating geometric, topological, and semantic relationships into a multi-level graph-based representation. By capturing hierarchical abstractions of objects and spatial layouts, 3DSGs enable robots to reason about environments in a structured manner, improving context-aware decision-making and adaptive planning. Although most recent work has focused on indoor 3DSGs, this paper investigates their construction and utility in outdoor environments. We present a method for generating a task-agnostic metric-semantic point cloud for large outdoor settings and propose modifications to existing indoor 3DSG generation techniques for outdoor applicability. Our preliminary qualitative results demonstrate the feasibility of outdoor 3DSGs and highlight their potential for future deployment in real-world field robotic applications.

Figures

Figures reproduced from arXiv: 2506.06562 by the authors.

Figure 1
Figure 1. Preliminary implementation of a task-driven terrain [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Task-Agnostic Metric-Semantic Mapping. Mapping steps: (1) Extract terrain and class-agnostic semantics from RGB imagery. (2) Generate a global 3D point-cloud map using LIO-SAM. (3) Fuse these results to produce a CLIP-based semantic embedding for each sparse point in the 3D point-cloud map. (b) Task-Driven 3D Scene Graph Generation. Upon receipt of a task specification, a unique 3DSG structured specifically to s… view at source ↗
Figure 3
Figure 3. GVD process: (a) Initial occupancy grid of sidewalk [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Terrain point-cloud results: (a) LIO-SAM path over [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Matching images associated with the “image of a [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Demonstration of the promptable nature of task [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 42 canonical work pages

  1. [26]

    Collaborative dynamic 3D scene graphs for automated driving,

    E. Greve, M. B ¨uchner, N. V ¨odisch, W. Burgard, and A. Valada, “Collaborative dynamic 3D scene graphs for automated driving,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 11 118–11 124

    work page 2024

  2. [10]

    Collaborative Dynamic 3D Scene Graphs for Open-Vocabulary Urban Scene Understanding

    T. Steinke, M. B ¨uchner, N. V ¨odisch, and A. Valada, “Collaborative dynamic 3D scene graphs for open- vocabulary urban scene understanding,” 2025. [Online]. Available: https://arxiv.org/abs/2503.08474

    work page Pith review arXiv 2025

  3. [1]

    ORB-SLAM2: an open-source SLAM system for monocular, stereo and Presented at the 2025 IEEE ICRA Workshop on Field Robotics RGB-D cameras,

    R. Mur-Artal and J. D. Tard ´os, “ORB-SLAM2: an open-source SLAM system for monocular, stereo and Presented at the 2025 IEEE ICRA Workshop on Field Robotics RGB-D cameras,”IEEE Transactions on Robotics, vol. 33, no. 5, pp. 1255–1262, 2017

    work page 2025

  4. [2]

    iSAM2: Incremental smoothing and mapping with fluid relinearization and incremental vari- able reordering,

    M. Kaess, H. Johannsson, R. Roberts, V . Ila, J. Leonard, and F. Dellaert, “iSAM2: Incremental smoothing and mapping with fluid relinearization and incremental vari- able reordering,” inProceedings of the IEEE Interna- tional Conference on Robotics and Automation (ICRA), 2011, pp. 3281–3288

    work page 2011

  5. [3]

    VINS-Mono: A robust and versatile monocular visual-inertial state estimator,

    T. Qin, P. Li, and S. Shen, “VINS-Mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004–1020, 2018

    work page 2018

  6. [4]

    Pairwise consistent measurement set maximization for robust multi-robot map merging,

    J. G. Mangelson, D. Dominic, R. M. Eustice, and R. Vasudevan, “Pairwise consistent measurement set maximization for robust multi-robot map merging,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 2916– 2923

    work page 2018

  7. [5]

    LIO-SAM: Tightly-coupled lidar inertial odometry via smoothing and mapping,

    T. Shan, B. Englot, D. Meyers, W. Wang, C. Ratti, and R. Daniela, “LIO-SAM: Tightly-coupled lidar inertial odometry via smoothing and mapping,” inProceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 5135–5142

    work page 2020

  8. [6]

    ConceptGraphs: Open-vocabulary 3D scene graphs for perception and planning,

    Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa, C. Gan, C. M. De Melo, J. B. Tenenbaum, A. Torralba, F. Shkurti, and L. Paull, “ConceptGraphs: Open-vocabulary 3D scene graphs for perception and planning,” inProceedings of the IEEE International Conference on Robotics and Automation (IC...

    work page 2024

Show all 53 references
  1. [7]

    Hierarchical open-vocabulary 3D scene graphs for language-grounded robot navigation,

    A. Werby, C. Huang, M. B ¨uchner, A. Valada, and W. Burgard, “Hierarchical open-vocabulary 3D scene graphs for language-grounded robot navigation,” Robotics: Science and Systems, 2024

    2024

  2. [8]

    WildLMa: Long horizon loco- manipulation in the wild,

    R.-Z. Qiu, Y . Song, X. Peng, S. A. Suryadevara, G. Yang, M. Liu, M. Ji, C. Jia, R. Yang, X. Zou, and X. Wang, “WildLMa: Long horizon loco- manipulation in the wild,” 2024. [Online]. Available: https://arxiv.org/abs/2411.15131

    2024 arXiv

  3. [9]

    OrionNav: Online planning for robot autonomy with context-aware LLM and open- vocabulary semantic scene graphs,

    V . N. Devarakonda, R. G. Goswami, A. U. Kaypak, N. Patel, R. Khorrambakht, P. Krishnamurthy, and F. Khorrami, “OrionNav: Online planning for robot autonomy with context-aware LLM and open- vocabulary semantic scene graphs,” 2024. [Online]. Available: https://arxiv.org/abs/2410.06239

    2024 arXiv

  4. [11]

    3D scene graph: A struc- ture for unified semantics, 3D space, and camera,

    I. Armeni, Z.-Y . He, A. Zamir, J. Gwak, J. Malik, M. Fischer, and S. Savarese, “3D scene graph: A struc- ture for unified semantics, 3D space, and camera,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 5663–5672

    2019

  5. [12]

    Kimera: an open-source library for real-time metric- semantic localization and mapping,

    A. Rosinol, M. Abate, Y . Chang, and L. Carlone, “Kimera: an open-source library for real-time metric- semantic localization and mapping,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2020, pp. 1689–1696

    2020

  6. [13]

    3D dynamic scene graphs: Actionable spatial perception with places, objects, and humans

    A. Rosinol, A. Gupta, M. Abate, J. Shi, and L. Carlone, “3D dynamic scene graphs: Actionable spatial perception with places, objects, and humans.” [Online]. Available: https://arxiv.org/abs/2002.06289

    2002 arXiv

  7. [14]

    Hydra: A real- time spatial perception system for 3D scene graph construction and optimization,

    N. Hughes, Y . Chang, and L. Carlone, “Hydra: A real- time spatial perception system for 3D scene graph construction and optimization,”Robotics: Science and Systems (RSS), 2022

    2022

  8. [15]

    S-Graphs+: Real-time localization and mapping leveraging hierarchical representations,

    H. Bavle, J. L. Sanchez-Lopez, M. Shaheer, J. Civera, and H. V oos, “S-Graphs+: Real-time localization and mapping leveraging hierarchical representations,” 2022. [Online]. Available: https://arxiv.org/abs/2212.11770

    2022 arXiv

  9. [16]

    S-Graphs 2.0 – a hierarchical-semantic optimization and loop closure for SLAM,

    ——, “S-Graphs 2.0 – a hierarchical-semantic optimization and loop closure for SLAM,” 2025. [Online]. Available: https://arxiv.org/abs/2502.18044

    2025

  10. [17]

    Search3D: Hierarchical open-vocabulary 3D segmentation,

    A. Takmaz, A. Delitzas, R. W. Sumner, F. Engelmann, J. Wald, and F. Tombari, “Search3D: Hierarchical open-vocabulary 3D segmentation,” 2024. [Online]. Available: https://arxiv.org/abs/2409.18431

    2024 arXiv

  11. [18]

    Point2Graph: An end-to-end point cloud-based 3D open-vocabulary scene graph for robot navigation,

    Y . Xu, Z. Luo, Q. Wang, V . Kamat, and C. Menassa, “Point2Graph: An end-to-end point cloud-based 3D open-vocabulary scene graph for robot navigation,”

  12. [19]

    Clio: Real-time task-driven open-set 3D scene graphs,

    D. Maggio, Y . Chang, N. Hughes, M. Trang, D. Griffith, C. Dougherty, E. Cristofalo, L. Schmid, and L. Carlone, “Clio: Real-time task-driven open-set 3D scene graphs,” IEEE Robotics and Automation Letters, vol. 9, no. 10, pp. 8921–8928, 2024

    2024

  13. [20]

    Enhancing scene understanding for vision-and-language navigation by knowledge awareness,

    F. Gao, J. Tang, J. Wang, S. Li, and J. Yu, “Enhancing scene understanding for vision-and-language navigation by knowledge awareness,”IEEE Robotics and Automa- tion Letters, vol. 9, no. 12, pp. 10 874–10 881, 2024

    2024

  14. [21]

    TB- HSU: Hierarchical 3D scene understanding with contextual affordances,

    W. Xu, V . Ila, L. Zhou, and C. T. Jin, “TB- HSU: Hierarchical 3D scene understanding with contextual affordances,” 2025. [Online]. Available: https://arxiv.org/abs/2412.05596

    2025 arXiv

  15. [22]

    The bare necessities: Designing simple, effective open- vocabulary scene graphs,

    C. Kassab, M. Mattamala, S. Morin, M. B ¨uchner, A. Valada, L. Paull, and M. Fallon, “The bare necessities: Designing simple, effective open- vocabulary scene graphs,” 2024. [Online]. Available: https://arxiv.org/abs/2412.01539

    2024 arXiv

  16. [23]

    OpenIN: Open-vocabulary instance-oriented navigation in dynamic domestic environments,

    Y . Tang, M. Wang, Y . Deng, Z. Zheng, J. Deng, and Y . Yue, “OpenIN: Open-vocabulary instance-oriented navigation in dynamic domestic environments,” 2025. [Online]. Available: https://arxiv.org/abs/2501.04279

    2025 arXiv

  17. [24]

    TOPO-FIELD: Topometric mapping with brainin- spired hierarchical layout-object-position fields,

    “TOPO-FIELD: Topometric mapping with brainin- spired hierarchical layout-object-position fields,” in Proceedings of the International Conference on Learn- ing Representations (ICLR), 2025

    2025

  18. [25]

    Using language to generate state abstractions for long-range planning in outdoor environments,

    M. Berg, G. Konidaris, and S. Tellex, “Using language to generate state abstractions for long-range planning in outdoor environments,” inProceedings of the Interna- tional Conference on Robotics and Automation (ICRA), Presented at the 2025 IEEE ICRA Workshop on Field Robotics ...

    2025

  19. [27]

    Indoor and outdoor 3D scene graph gen- eration via language-enabled spatial ontologies,

    J. Strader, N. Hughes, W. Chen, A. Speranzon, and L. Carlone, “Indoor and outdoor 3D scene graph gen- eration via language-enabled spatial ontologies,”IEEE Robotics and Automation Letters, vol. 9, no. 6, pp. 4886–4893, 2024

    2024

  20. [28]

    Task and motion planning in hierarchical 3D scene graphs,

    A. Ray, C. Bradley, L. Carlone, and N. Roy, “Task and motion planning in hierarchical 3D scene graphs,” 2024. [Online]. Available: https: //arxiv.org/abs/2403.08094

    2024 arXiv

  21. [29]

    Visual genome: Con- necting language and vision using crowdsourced dense image annotations,

    R. Krishna, Y . Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y . Kalantidis, L. Li, D. A. Shamma, M. S. Bernstein, and L. Fei-Fei, “Visual genome: Con- necting language and vision using crowdsourced dense image annotations,”International Journal of Computer Vision,...

    2017

  22. [30]

    3D semantic parsing of large-scale indoor spaces,

    I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese, “3D semantic parsing of large-scale indoor spaces,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1534–1543

    2016

  23. [31]

    GPT-4o technical report,

    OpenAI, “GPT-4o technical report,” 2024. [On- line]. Available: https://platform.openai.com/docs/ models/gpt-4o

    2024

  24. [32]

    LLaMA: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “LLaMA: Open and efficient foundation language models,” 2023. [Online]. Available: https://arxiv.org/abs/2302.13971

    2023 arXiv

  25. [33]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in Proceedings of the International Conference on Ma- chine Lear...

    2021

  26. [34]

    ConceptFusion: Open-set multimodal 3D mapping,

    K. Jatavallabhula, A. Kuwajerwala, Q. Gu, M. Omama, T. Chen, S. Li, G. Iyer, S. Saryazdi, N. Keetha, A. Tewari, J. Tenenbaum, C. de Melo, M. Krishna, L. Paull, F. Shkurti, and A. Torralba, “ConceptFusion: Open-set multimodal 3D mapping,”Robotics: Science and Systems (RSS), 2023

    2023

  27. [35]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll´ar, and R. Girshick, “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 3992–4003

    2023

  28. [36]

    SAM 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Doll ´ar, and C. Feichtenhofer, “SAM 2: Segment anything in images and videos,” 2024. [Online]. Avail...

    2024 arXiv

  29. [37]

    Fast segment anything,

    X. Zhao, W. Ding, Y . An, Y . Du, T. Yu, M. Li, M. Tang, and J. Wang, “Fast segment anything,” 2023. [Online]. Available: https://arxiv.org/abs/2306.12156

    2023 arXiv

  30. [38]

    Faster segment anything: Towards lightweight SAM for mobile applications,

    C. Zhang, D. Han, Y . Qiao, J. U. Kim, S.-H. Bae, S. Lee, and C. S. Hong, “Faster segment anything: Towards lightweight SAM for mobile applications,”

  31. [39]

    YOLO-World: Real-time open-vocabulary object detection,

    T. Cheng, L. Song, Y . Ge, W. Liu, X. Wang, and Y . Shan, “YOLO-World: Real-time open-vocabulary object detection,” inProceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  32. [40]

    Deep multispectral semantic scene understand- ing of forested environments using multimodal fu- sion,

    A. Valada, G. L. Oliveira, T. Brox, and W. Bur- gard, “Deep multispectral semantic scene understand- ing of forested environments using multimodal fu- sion,” in2016 International Symposium on Experimen- tal Robotics, D. Kuli ´c, Y . Nakamura, O. Khatib, and G. Venture, Eds., v...

  33. [41]

    Real-time semantic mapping for au- tonomous off-road navigation,

    D. Maturana, P.-W. Chou, M. Uenoyama, and S. Scherer, “Real-time semantic mapping for au- tonomous off-road navigation,” inField and Service Robotics, M. Hutter and R. Siegwart, Eds. Springer International Publishing, vol. 5, pp. 335–350

  34. [42]

    A RUGD dataset for autonomous navigation and visual perception in unstructured outdoor environ- ments,

    M. Wigness, S. Eum, J. G. Rogers, D. Han, and H. Kwon, “A RUGD dataset for autonomous navigation and visual perception in unstructured outdoor environ- ments,” inProceedings of the IEEE Conference on Intelligent Robots and Systems (IROS), 2019, pp. 5000– 5007

    2019

  35. [43]

    RELLIS-3D dataset: Data, benchmarks and analysis,

    P. Jiang, P. Osteen, M. Wigness, and S. Saripalli, “RELLIS-3D dataset: Data, benchmarks and analysis,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 1110– 1116

    2021

  36. [44]

    A fine-grained dataset and its efficient semantic segmenta- tion for unstructured driving scenarios,

    K. A. Metzger, P. Mortimer, and H.-J. Wuensche, “A fine-grained dataset and its efficient semantic segmenta- tion for unstructured driving scenarios,” inProceedings of the International Conference on Pattern Recognition (ICPR), 2021, pp. 7892–7899

    2021

  37. [45]

    The GOOSE dataset for perception in unstructured environments,

    P. Mortimer, R. Hagmanns, M. Granero, T. Luettel, J. Petereit, and H.-J. Wuensche, “The GOOSE dataset for perception in unstructured environments,” inPro- ceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  38. [46]

    Excavating in the wild: The GOOSE-Ex dataset for semantic segmentation,

    R. Hagmanns, P. Mortimer, M. Granero, T. Luettel, and J. Petereit, “Excavating in the wild: The GOOSE-Ex dataset for semantic segmentation,” 2024. [Online]. Available: https://arxiv.org/abs/2409.18788

    2024

  39. [47]

    Wild- Scenes: A benchmark for 2D and 3D semantic seg- mentation in large-scale natural environments,

    K. Vidanapathirana, J. Knights, S. Hausler, M. Cox, M. Ramezani, J. Jooste, E. Griffiths, S. Mohamed, S. Sridharan, C. Fookes, and P. Moghadam, “Wild- Scenes: A benchmark for 2D and 3D semantic seg- mentation in large-scale natural environments,”The International Journal of Ro...

    2024

  40. [48]

    LOAM: Lidar odometry and Presented at the 2025 IEEE ICRA Workshop on Field Robotics mapping in real-time,

    J. Zhang and S. Singh, “LOAM: Lidar odometry and Presented at the 2025 IEEE ICRA Workshop on Field Robotics mapping in real-time,”In Robotics: Science and Sys- tems, 2014

    2025

  41. [49]

    YOLOv8: A novel object detection algorithm with enhanced performance and robustness,

    R. Varghese and S. M., “YOLOv8: A novel object detection algorithm with enhanced performance and robustness,” inProceedings of the International Confer- ence on Advances in Data Engineering and Intelligent Computing Systems (ADICS), 2024, pp. 1–6

    2024

  42. [50]

    Multidimensional binary search trees used for associative searching,

    J. L. Bentley, “Multidimensional binary search trees used for associative searching,”Communications of the ACM, vol. 18, no. 9, p. 509–517, 9 1975

    1975

  43. [51]

    Efficient grid- based spatial representations for robot navigation in dynamic environments,

    B. Lau, C. Sprunk, and W. Burgard, “Efficient grid- based spatial representations for robot navigation in dynamic environments,”Robotics and Autonomous Sys- tems, vol. 61, no. 10, pp. 1116–1130, 2013. Presented at the 2025 IEEE ICRA Workshop on Field Robotics

    2013

  44. [2023]

    Available: https://arxiv.org/abs/2306

    [Online]. Available: https://arxiv.org/abs/2306. 14289

  45. [2024]

    Available: https://arxiv.org/abs/2409

    [Online]. Available: https://arxiv.org/abs/2409. 10350

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.