Pith. sign in

REVIEW 5 major objections 5 minor 24 references

A compact 2.5D map with instance captions and expert-fusion retrieval can match or beat heavy 3D scene graphs on language-guided navigation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:22 UTC pith:F4EIG4UN

load-bearing objection A serious systems paper with a real representational idea—the 2.5D multi-occupancy instance map—but the headline numbers compare full pipelines, not isolated representations, so they overstate what is shown. the 5 major comments →

arxiv 2607.12630 v2 pith:F4EIG4UN submitted 2026-07-14 cs.RO cs.CV

Instance-Enriched Semantic Maps for Visual Language Navigation

Jiho Hong , Eunae Kang , Sanghyun Kim , Young-Sik Shin This is my paper
classification cs.RO cs.CV
keywords visual language navigation2.5D semantic mappinginstance-level mappingopen-vocabulary segmentationlarge language modelszero-shot navigationstorage-efficient scene representationexpert fusion retrieval
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper sets out to show that an embodied robot can follow natural-language navigation queries using a compact 2.5D semantic map that keeps individual object instances, their vertical stacking, and rich text captions, instead of a heavy 3D reconstruction. It claims its instance-enriched map improves object-level mapping accuracy by over 27% in prediction-normalized AUC over a 3D scene-graph baseline, while cutting storage by about 96% (roughly 10 MB versus 248 MB). For navigation, it reports over 17% higher object retrieval and 23% higher navigation success across diverse query formulations. The central bet is that storing instance captions plus a query-adaptive expert-fusion retrieval module matters more than geometric fidelity for grounding language in indoor spaces.

Core claim

The paper argues that the limiting factor in language-guided navigation is not geometric fidelity but the richness of per-instance semantics and the way queries are matched to them. It builds a 2.5D map in which each grid cell can hold multiple vertically stacked instance IDs, so objects above one another are not blended. Each instance stores an open-vocabulary feature, a room label, structured attributes such as color and material, and a compressed natural-language caption distilled from multiple views. On top of that, the Multi-Type Expert Fusion Retrieval (MTEFR) module routes each user query through a gate and four type-specialized experts—object category, room context, description, and

What carries the argument

The central object is the 2.5D instance-enriched map: a top-down grid where each cell can list several vertically stacked instance indices, paired with an instance dictionary where each entry is an open-vocabulary embedding, room label, structured tags, and a compact caption. The query-side mechanism is MTEFR: a gate language model assigns weights to four expert rankers (object, room, description, abstract), each produces a top-κ list, and score fusion combines a weighted sum with a peak score to pick the goal instance. The map does the storage work; MTEFR does the interpretation work.

Load-bearing premise

The whole map rests on the assumption of accurate, globally consistent camera poses, and every experiment gets those poses from the simulator rather than from a real SLAM estimate.

What would settle it

Run the same mapping and retrieval pipeline on the same scenes but inject increasingly large pose drift (for instance, 0.1–0.5 m errors) into the camera poses before fusion, and track prediction-normalized AUC and instance-count gap; a sharp collapse at modest drift would show the compact 2.5D map's advantage depends on perfect localization.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A robot can carry a semantically useful map of an indoor floor in about 10 MB, making long-term and embedded deployment feasible without losing instance-level grounding.
  • Abstract queries that name no category, such as 'the most comfortable object to rest on for three people,' become addressable because captions and room context are searchable, not just labels.
  • Vertical stacking is resolved without a voxel or mesh reconstruction, so multi-shelf and cluttered scenes avoid semantic blending in the grid.
  • Providing richer captions alone does not close the gap; the query-adaptive fusion of expert rankings is what delivers the reported gains.
  • Room segmentation from structural boundaries works best in partitioned spaces; open-plan areas remain a recognized failure point that would need a different boundary source.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the paper's core bet would be to replace simulator poses with noisy SLAM trajectories and re-measure the same AUC and success metrics; if fragmentation appears at drift below the stated association thresholds, the 10 MB claim may only hold under perfect localization.
  • Since captions are the main semantic carrier, the approach invites a fully asynchronous or distilled captioning pipeline; caption latency, not map size, would likely bound real-time use.
  • The abstract-query success rate (about 14%) suggests the next bottleneck is functional knowledge rather than perception; a hierarchy of affordances or functional categories could tighten the abstract expert's ranking.
  • The single-floor evaluation leaves multi-floor maps untested; extending the 2.5D representation to floors connected by stairs or elevators is a natural, concrete next experiment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Instance-Enriched Semantic Maps (IESM), a zero-shot VLN framework that builds a compact 2.5D instance-level map from RGB-D observations using SEEM panoptic segmentation, enriches instances with room context and LLM-generated captions, and performs goal selection via a Multi-Type Expert Fusion Retrieval (MTEFR) module. The mapping, room segmentation, captioning, and retrieval components are evaluated on MP3D, Replica, and HM3DSEM against HOV-SG and VLMaps. The paper claims that the 2.5D representation outperforms a 3D scene-graph baseline by over 27% in prediction-normalized AUC, that MTEFR yields over 17% object-retrieval and 23% navigation-success improvements, and that the representation reduces storage by ~96%.

Significance. If the claims are substantiated, the paper would demonstrate a practical alternative to 3D scene graphs: a 10 MB instance-enriched map with vertical stacking support and LLM-based retrieval can outperform a 248 MB 3D map on mapping and navigation. The manuscript has notable strengths: hyperparameters and prompts are listed, the system is modular and clearly described, and the authors explicitly acknowledge failure modes such as over-segmentation (Fig. 5), open-plan room failures (Fig. 6), and low absolute success on abstract queries (Sec. 4.3.2). The matched-backbone comparison (VLMaps+SEEM vs. Ours in Table 3) and the Full+SPR ablation are the right controls, and the storage comparison is concrete. However, the headline comparisons are confounded across multiple components and lack statistical support, so the central attribution claims are not yet tightly established.

major comments (5)
  1. [Sec. 4.1.1, Table 1; Abstract] The headline claim that the 2.5D representation outperforms the 3D baseline by >27% in prediction-normalized AUC is confounded: 'Ours' uses SEEM (Focal-L) while HOV-SG uses SAM+CLIP (ViT-H-14). Table 3 shows that replacing LSeg with SEEM alone improves VLMaps on both MP3D and Replica (mIoU 0.198→0.226 and 0.175→0.200), so the perception backbone materially affects instance quality. The large AUC_pred gaps (0.356 vs 0.093; 0.371 vs 0.085) and Instance Diff gaps (2.6 vs 86.6; 67.4 vs 417.6) are consistent with HOV-SG's over-segmentation, not necessarily with the 2.5D representation. The matched-backbone comparison (VLMaps+SEEM vs Ours, Table 3) is not the comparison cited in the abstract/conclusion. To support the headline, please report HOV-SG with SEEM or otherwise isolate representation from perception backbone.
  2. [Sec. 4.3.2, Table 5] The claimed 17%/23% navigation improvements are full-system comparisons against HOV-SG: the two systems differ in perception backbone, map representation, captioning, retrieval, and navigation. HOV-SG (w/ desc.) adds only captions and shows marginal gains, but it does not isolate MTEFR because HOV-SG's SAM+CLIP over-segmentation may prevent caption assignment at the same object granularity. The internal ablation Full+MTEFR vs Full+SPR is the cleaner isolation, yet the gains are uneven: on (o) O-SR improves 70 vs 58, on (a,r,d) 14 vs 6, while conditional N-SR is lower for MTEFR on (o,d), (o,r,d), and (a,r,d). With 250 trials per query type, no confidence intervals or significance tests are reported. Please provide per-query-type errors/significance and state explicitly that the headline is a full-system comparison.
  3. [Sec. 4.2, Table 4] Room-level semantic segmentation is evaluated on only two MP3D scenes, with no per-scene breakdown or variance. The claimed improvements (mAcc 0.72→0.86, mIoU 0.53→0.75) rest on very thin evidence, especially since the paper itself identifies a failure on open-plan scenes (Fig. 6b). Please evaluate on the same scene sets as the mapping experiments (at least the full Replica/MP3D sets) and report per-scene results with standard errors, or substantially temper the claim.
  4. [Sec. 3.1.3 and Sec. 5] The pipeline assumes accurate and globally consistent camera poses (Sec. 3.1), and all experiments use simulator poses. The conclusion asserts that the secondary fusion pass 'already tolerates moderate pose drift,' but no experiment with realistic SLAM noise is provided. Given the association thresholds δgeo=0.4 and δsem=0.85 (Sec. 4), pose drift beyond these thresholds would fragment or merge instances. If the robustness claim is retained, it must be tested; otherwise the statement should be removed or clearly marked as untested speculation.
  5. [Sec. 3.4 and Sec. B] The expert taxonomy (obj, room, desc, abs) exactly matches the query types used to construct the test set (object, room, caption, mixed, abstract). This alignment is favorable to MTEFR by design and may overstate its advantage on 'diverse' query formulations. It is not circular in the strict sense because the baselines see the same queries, but it should be acknowledged as a favorable protocol, and ideally supplemented with queries that do not fit neatly into a single expert type.
minor comments (5)
  1. [Abstract/Conclusion] The 'over 27%' improvement is actually an average absolute difference in percentage points (0.356−0.093=0.263; 0.371−0.085=0.286). Please reword to '27 percentage points' or report relative improvement to avoid misinterpretation.
  2. [Tables 1/2] 'topend' is not defined. Specify whether it denotes the final point before saturation or the full ranking endpoint.
  3. [Sec. 4.1.1] The definition of Instance Diff is given as 'absolute difference between ground-truth and predicted object counts,' but it is unclear whether this is per scene, per category, or aggregated. Please clarify and report per-scene values.
  4. [Sec. 3.3] Representative keyframe selection is described as based on 'confidence of the predicted masks and frequency of observations,' but no algorithm or criterion is given. Please include the selection procedure for reproducibility.
  5. [Sec. 4.1.3] The qualitative statement that the red-boxed 'table' is 'indeed a kitchen countertop' is a useful open-vocabulary discussion, but it also highlights that the ground-truth label and prediction can be both valid. Consider adding a quantitative analysis of such label ambiguity.

Circularity Check

0 steps flagged

No significant circularity: the mapping, retrieval, and navigation claims are evaluated against external baselines with fixed hyperparameters; no prediction reduces to a fitted input or self-citation.

full rationale

The derivation chain is self-contained. Instance association (Eqs. 5-10) uses geometric/semantic similarities with fixed thresholds (Sec. 4: δgeo=0.4, δsem=0.85, ωgeo=0.7, ωsem=0.3), and instance fusion (Eqs. 9-10) is a standard weighted update; none of these equations embed the evaluation metrics (AUC variants, O-SR, N-SR). The headline AUC_pred improvement is computed with an independently defined denominator |P| (Eq. 25) on external baselines HOV-SG and VLMaps; although the comparison is not fully matched in segmentation backbone (SEEM vs. SAM+CLIP), that is an attribution/correctness confound, not a circular reduction. MTEFR's expert taxonomy (obj/room/desc/abs, Sec. 3.4.2) aligns with the query types in App. B, so the evaluation is method-aligned, but the queries themselves are generated from target object/room/description triples and the method is compared against HOV-SG, HOV-SG (w/ desc.), and a single-pass ablation (Table 5); no parameter is fitted to force those outcomes. The storage-reduction claim (Sec. 4.4) is a direct measurement of map sizes. The paper does not rely on any load-bearing self-citation or imported uniqueness theorem; its references to HOV-SG, VLMaps, and SEEM are external systems. Therefore the central claims—2.5D mapping accuracy, MTEFR retrieval gains, and storage reduction—are not circular.

Axiom & Free-Parameter Ledger

10 free parameters · 5 axioms · 0 invented entities

All central claims rest on simulator-grade camera poses, the quality of off-the-shelf VLMs (SEEM/CLIP/LLaMA/GPT-4), and hand-set hyperparameters; the paper provides values for most hyperparameters but omits the obstacle height range and category lists.

free parameters (10)
  • delta_geo = 0.4
    Threshold for geometric overlap in instance association (Sec. 3.1.2), chosen by hand and fixed across datasets.
  • delta_sem = 0.85
    Threshold for semantic (cosine) similarity in instance association; hand-set.
  • omega_geo = 0.7
    Weight on geometric similarity in association score; paired with omega_sem=0.3.
  • omega_sem = 0.3
    Weight on semantic similarity in the association convex combination.
  • kappa = 2
    Top-k rank window for expert LLM scores in MTEFR (Eq. 21).
  • gamma = 0.75
    Base reliance on peak evidence in fusion (Eq. 23).
  • mu = 16
    Controls growth of peak score with kappa (Eq. 23).
  • obstacle height range = not specified
    h_min/h_max for obstacle map construction (Sec. 3.4.1); values not reported in the paper.
  • obstacle category list = natural-language list
    Open-vocabulary obstacle categories Ocat; hand-derived, not listed in the paper.
  • instance/room category lists = predefined lists
    Icat and Rcat needed for open-vocabulary labeling; not included in the paper text.
axioms (5)
  • domain assumption Camera poses are accurate and globally consistent
    Instance association and fusion in Sec. 3.1 rely on globally consistent camera poses; in experiments these come from the simulator. Real SLAM drift could break long-range fusion.
  • domain assumption SEEM panoptic masks are temporally consistent enough for cross-frame association
    The whole pipeline depends on SEEM providing stable instance masks; the paper argues this but does not prove it.
  • domain assumption CLIP pixel features are informative for room semantics in top-down view
    Room labeling (Sec. 3.2.2) relies on CLIP dense embedding alignment with room text prompts.
  • domain assumption LLM captions reflect observable visual properties accurately
    MTEFR desc/abstract experts trust captions from LLaMA/GPT-4; the paper notes viewpoint limits.
  • domain assumption GPT-generated queries with ground-truth labels approximate real user queries
    Navigation evaluation uses automatically generated queries conditioned on ground-truth category/room (Sec. B).

pith-pipeline@v1.3.0-alltime-deepseek · 25841 in / 14192 out tokens · 121233 ms · 2026-08-02T06:22:29.702071+00:00 · methodology

0 comments
read the original abstract

Visual Language Navigation (VLN) aims to enable an embodied agent to navigate complex environments by following natural language instructions. Recent approaches build semantic spatial maps and leverage Large Language Models (LLMs) for reasoning and decision making. Despite these advances, existing systems lack instance-level object detail and robustness to diverse user queries, limiting reliable navigation in complex indoor environments. To address these limitations, we propose Instance-Enriched Semantic Maps, a unified framework with three key contributions: (1) Instance-level two-and-a-half-dimensional (2.5D) rich information mapping that constructs maps from color and depth observations via open-vocabulary panoptic segmentation, preserving vertical distinctions and capturing small objects, while storing diverse semantic attributes and natural language captions enriched with room-level context. (2) Robust query processing via LLM-based target selection, which dynamically routes queries across type-specialized experts and integrates their outputs through score-level fusion, enabling consistent goal selection across diverse query formulations. (3) Storage-efficient semantic representation that achieves approximately 96% reduction compared to three-dimensional (3D) scene-graph approaches while preserving sufficient spatial information for navigation. The proposed 2.5D representation outperforms the 3D baseline by over 27% in prediction-normalized Area Under the Curve (AUC). In navigation experiments, our method achieves over 17% improvement in object retrieval and over 23% in navigation success compared to the baseline across diverse query types. The project page is available at https://rcilab.github.io/iesm_vln.

Figures

Figures reproduced from arXiv: 2607.12630 by Eunae Kang, Jiho Hong, Sanghyun Kim, Young-Sik Shin.

Figure 1
Figure 1. Figure 1: Instance-Enriched Semantic Map is an open-vocabulary 2.5D map representation built upon Instance-Level 2.5D Open-Vocabulary Mapping, Room-Level Semantic Segmentation, and Instance Captioning. During navigation, the Multi￾Type Expert Fusion Retrieval module interprets diverse natural language queries by routing inference across type-specialized experts and fusing their outputs, enabling robust zero-shot goa… view at source ↗
Figure 2
Figure 2. Figure 2: System overview. The proposed framework consists of four main components. The upper section covers Instance￾Enriched Semantic Mapping, comprising three stages. First, Instance-Level 2.5D Open-Vocabulary Mapping (left, Sec. 3.1) extracts per-frame instance masks and embeddings using SEEM and incrementally fuses them into a global 2.5D semantic map in which each grid cell stores indices of multiple verticall… view at source ↗
Figure 3
Figure 3. Figure 3: Example of instance-level caption annotations. Each detected object instance is associated with semantic attributes including category, room type, color, and material. representation enables the system to interpret varied open-vocabulary navigation commands, such as "find the small red cup in the kitchen" or "go to the wooden chair in the living room". A detailed description of the prompts and captioning p… view at source ↗
Figure 4
Figure 4. Figure 4: Instance-level 2.5D open-vocabulary mapping results on two scenes from the Replica dataset. For each scene, the left side shows a 3D rendered view of the environment and the right side shows the corresponding top-down 2.5D semantic map, where each instance is color-coded according to the category legend at the top. The colored bounding boxes highlight small objects (e.g., bottles, laptops, cups) that are r… view at source ↗
Figure 5
Figure 5. Figure 5: 2D mapping results of each method. The red boxes highlight that our method preserves objects as coherent instances without fragmentation. This demonstrates that the 2.5D mapping, when projected into a top-down view, mitigates the semantic ambiguity caused by vertically overlapping object features in conventional 2D projections. The yellow boxes indicate cases where a single object is split into multiple fr… view at source ↗
Figure 6
Figure 6. Figure 6: Room segmentation results on two scenes from the MP3D dataset. The top row shows the colorized top￾down map, predicted instance map, and region segmentation. The bottom row compares the GT, CLIP-based semantic segmentation, and our final room-level segmentation output. The red box highlights a representative failure region caused by this structural absence. 4.1.3. Additional Validation: Effectiveness of 2.… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative navigation result for the abstract query, "Navigate to the red seating object in the office," on scene 00824 of the HM3DSEM dataset. (Left) The 2.5D semantic map with room segmentation and instances associated with the abstract concept of "seating." (Center) Full instance-level information stored in the map for each candidate, including category, room type, color, material, and natural language… view at source ↗
Figure 8
Figure 8. Figure 8: Example of the prompt used for instance-level captioning with LLaMA 3.2 Vision. : Preprint submitted to Elsevier Page 21 of 24 [PITH_FULL_IMAGE:figures/full_fig_p021_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Example of the result with LLaMA 3.2 Vision. System: Identify and describe instances. Input and output must be in JSON format. The input field 'captions' contains a list of image captions aiming to identify the instance. Output 'color' as a visual color of the identified instance. Output 'material’ making up the identified instance. Output 'caption' as a concise description of the identified instance [PIT… view at source ↗
Figure 10
Figure 10. Figure 10: Example of the system prompt with GPT-4. LLaMA 3.2 Vision was guided to produce descriptive captions for segmented instances, conditioned on represen￾tative keyframe images and the corresponding predicted category label. The employed prompt is shown in [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Example of a navigation query. References Achiam, J., Adler, S., Agarwal, S., et al., 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 . An, D., Qi, Y., Li, Y., Huang, Y., Wang, L., Tan, T., Shao, J., 2023. Bevbert: Multimodal map pre-training for language-guided navigation . Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sünderhauf, N., Reid, I., Gould, S., van den Hengel, A., 2018… view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 3 canonical work pages

  1. [5]

    11509–11522

    Open-vocabulary queryable scene representations for real world planning, in: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 11509–11522. doi:10.1109/ICRA48891.2023.10161534. Dai, G., Zhao, J., Chen, Y., Qin, Y., Zhao, H., Xie, G., Yao, Y., Shu, X., Li, X.,

    arXiv 2023

  2. [7]

    arXiv preprint arXiv:2407.21783

    The llama 3 herd of models. arXiv preprint arXiv:2407.21783 . Fried,D.,Hu,R.,Cirik,V.,Rohrbach,A.,Andreas,J.,Morency,L.P.,Berg-Kirkpatrick,T.,Saenko,K.,Klein,D.,Darrell,T.,2018. Speaker-follower models for vision-and-language navigation. Advances in neural information processing systems

    Pith/arXiv arXiv 2018

  3. [8]

    23171–23181

    Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 23171–23181. Gao,J.,Liu,R.,Wang,W.,2025. 3dgaussianmapwithopen-setsemanticgroupingforvision-languagenavigation,in:ProceedingsoftheIEEE/CVF International Conference on C...

    2025

  4. [10]

    5021–5028

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning, in: 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 5021–5028. doi:10.1109/ICRA57147.2024.10610243. Huang,C.,Mees,O.,Zeng,A.,Burgard,W.,2023. Visuallanguagemapsforrobotnavigation,in:2023IEEEInternationalConferenceonRobotics and Automation (ICRA), pp. 1060...

    arXiv 2024

  5. [12]

    Advanced Robotics 38, 1232–1254

    Real-world robot applications of foundation models: a review. Advanced Robotics 38, 1232–1254. URL:https://doi.org/10.1080/01691864.2024.2408593, doi:10.1080/01691864. 2024.2408593, arXiv:https://doi.org/10.1080/01691864.2024.2408593. Kerbl, B., Kopanas, G., Leimkuehler, T., Drettakis, G.,

    arXiv 2024

  6. [13]

    Yin, H., Xu, X., Wu, Z., Zhou, J., Lu, J.,

    URL:https://www.mdpi.com/2227-7390/13/5/810, doi:10.3390/math13050810. Yin, H., Xu, X., Wu, Z., Zhou, J., Lu, J.,

    work page doi:10.3390/math13050810

  7. [14]

    arXiv preprint arXiv:1712.05474

    Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474 . Labbé, M., Michaud, F.,

    Pith/arXiv arXiv

  8. [17]

    10968–10980

    Bird’s-eye-view scene graph for vision-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10968–10980. Nanwani, L., Agarwal, A., Jain, K., Prabhakar, R., Monis, A., Mathur, A., Jatavallabhula, K.M., Abdul Hafez, A.H., Gandhi, V., Krishna, K.M., 2023.Instance-levelsemanticmapsforvisionlanguagenaviga...

    arXiv 2023

  9. [18]

    (Eds.), Proceedings of the 38th International Conference on Machine Learning, PMLR

    Learning transferable visual models from natural language supervision, in: Meila, M., Zhang, T. (Eds.), Proceedings of the 38th International Conference on Machine Learning, PMLR. pp. 8748–8763. URL:https://proceedings.mlr. press/v139/radford21a.html. Radosavovic,I., Xiao,T.,Zhang, B.,Darrell, T.,Malik,J., Sreenath,K.,2024. Real-worldhumanoid locomotionwi...

    doi:10.1126/scirobotics.adi9579 2024

  10. [19]

    arXiv preprint arXiv:1906.05797

    The Replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797 . Sumers, T., Marino, K., Ahuja, A., Fergus, R., Dasgupta, I.,

    Pith/arXiv arXiv 1906

  11. [20]

    Engineering Applications of Artificial Intelligence 128, 107487

    Pasts: Progress-aware spatio-temporal transformer speaker for vision-and- language navigation. Engineering Applications of Artificial Intelligence 128, 107487. URL:https://www.sciencedirect.com/science/ article/pii/S0952197623016718, doi:https://doi.org/10.1016/j.engappai.2023.107487. Wang,Z.,Lee,G.H.,2025. g3d-lf:Generalizable3d-languagefeaturefieldsfore...

    arXiv 2023

  12. [22]

    Advances in neural information processing systems 37, 5285–5307

    Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation. Advances in neural information processing systems 37, 5285–5307. Yu,B.,Liu,Y.,Han,L.,Kasaei,H.,Li,T.,Cao,M.,2024. Vln-game:Vision-languageequilibriumsearchforzero-shotsemanticnavigation. arXiv preprint arXiv:2411.11609 . Yuan, M., Wang, L., Waslander, S.L.,

    Pith/arXiv arXiv 2024

  13. [23]

    arXiv preprint arXiv:2507.18033

    Opennav: Open-world navigation with multimodal large language models. arXiv preprint arXiv:2507.18033 . Zhang, T., Hu, X., Xiao, J., Zhang, G.,

    arXiv

  14. [24]

    A survey of visual navigation: From geometry to embodied ai. Engineering Applications of ArtificialIntelligence114,105036.URL: https://www.sciencedirect.com/science/article/pii/S095219762200207X,doi: https: //doi.org/10.1016/j.engappai.2022.105036. Zou,X.,Yang,J.,Zhang,H.,Li,F.,Li,L.,Wang,J.,Wang,L.,Gao,J.,Lee,Y.J.,2023. Segmenteverythingeverywhereallaton...

    arXiv 2022

  15. [42]

    Kirillov, A., Mintun, E., Ravi, N., et al.,

    URL: https://doi.org/10.1145/3592433, doi:10.1145/3592433. Kirillov, A., Mintun, E., Ravi, N., et al.,

    doi:10.1145/3592433

  16. [2012]

    Image Processing On Line 2, 35–55

    LSD: a Line Segment Detector. Image Processing On Line 2, 35–55. https://doi.org/10.5201/ipol.2012.gjmr-lsd. Gu, Q., Kuwajerwala, A., Morin, S., et al.,

    work page doi:10.5201/ipol.2012.gjmr-lsd 2012

  17. [2017]

    Matterport3d: Learning from rgb-d data in indoor environments, in: 2017 International Conference on 3D Vision (3DV), pp. 667–676. doi:10.1109/3DV.2017.00081. Chen, B., Xia, F., Ichter, B., Rao, K., Gopalakrishnan, K., Ryoo, M.S., Stone, A., Kappler, D.,

    arXiv 2017

  18. [2018]

    Arumugam,D.,Karamcheti,S.,Gopalan,N.,Wong,L.,Tellex,S.,2017

    Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Arumugam,D.,Karamcheti,S.,Gopalan,N.,Wong,L.,Tellex,S.,2017. Accuratelyandefficientlyinterpretinghuman-robotinstructionsofvarying granularities, in: Proceedin...

    work page doi:10.15607/rss.2017.xiii.056 2017

  19. [2019]

    Journal of Field Robotics 36, 416–446

    Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. Journal of Field Robotics 36, 416–446. URL:https://onlinelibrary-wiley-com-ssl.webgate.khu.ac. kr/doi/abs/10.1002/rob.21831, doi:https://doi-org-ssl.webgate.khu.ac.kr/10.1002/rob.21831. Lei,X.,Wang,M.,Zhou,W.,Li,H.,2025...

    doi:10.1002/rob.21831 2025

  20. [2021]

    IEEE Transactions on Robotics 37, 1874–1890

    Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics 37, 1874–1890. doi:10.1109/TRO.2021.3075644. Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., Zhang, Y.,

    arXiv 2021

  21. [2022]

    URL:https://openreview.net/forum?id=RriDjddCLN

    Language-driven semantic segmentation, in: International Conference on Learning Representations. URL:https://openreview.net/forum?id=RriDjddCLN. Li,D.,Yang,Z.,Qi,G.,Pang,S.,Shang,G.,Ma,Q.,Yang,Z.,2025.Openmap:Instructiongroundingviaopen-vocabularyvisual-languagemapping, in: Proceedings of the 33rd ACM International Conference on Multimedia, pp. 7444–7452....

    2025

  22. [2023]

    arXiv preprint arXiv:2303.08774

    Gpt-4 technical report. arXiv preprint arXiv:2303.08774 . An, D., Qi, Y., Li, Y., Huang, Y., Wang, L., Tan, T., Shao, J.,

    Pith/arXiv arXiv

  23. [2024]

    arXiv preprint arXiv:2411.16053

    Unitedvln: Generalizable gaussian splatting for continuous vision-language navigation. arXiv preprint arXiv:2411.16053 . Driess, D., Xia, F., Sajjadi, M.S.M., et al.,

    Pith/arXiv arXiv

  24. [2025]

    doi:10.1109/TCE.2025.3601582

    Instance-aware visual language grounding for consumer robot navigation, in: IEEE Transactions on Consumer Electronics. doi:10.1109/TCE.2025.3601582. Ichter,B.,Brohan,A.,Chebotar,Y.,etal.,2022. Doasican,notasisay:Groundinglanguageinroboticaffordances,in:6thAnnualConferenceon Robot Learning. URL:https://openreview.net/forum?id=bdHkMjBJG_w. Jatavallabhula,K....

    arXiv 2025