Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

A new benchmark shows agents that ask an oracle questions beat silent instance-goal navigation, lifting success from 14.8% to 20.2%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:52 UTC pith:UMOUNLLH

load-bearing objection A genuinely useful house-scale benchmark for dialog-enabled instance navigation, with a real but fixable oracle-validation weakness in the headline empirical claims. the 3 major comments →

arxiv 2512.22342 v5 pith:UMOUNLLH submitted 2025-12-26 cs.RO

VL-LN Bench: Towards Long-horizon Goal-oriented Navigation with Active Dialogs

Wensi Huang , Shaohao Zhu , Meng Wei , Siqi Zhang , Jinming Xu , Xihui Liu , Hanqing Wang , Tai Wang
show 2 more authors
Feng Zhao Jiangmiao Pang
This is my paper
classification cs.RO
keywords interactive instance goal navigationembodied navigationvision-and-language navigationdialog-enabled agentbenchmark datasetoraclelong-horizon explorationinstance grounding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most embodied navigation benchmarks give the agent a complete, unambiguous instruction. This paper argues that real instructions are often underspecified, so an agent should be able to ask questions while moving. It proposes Interactive Instance Goal Navigation (IIGN), extending instance-goal navigation with free-form natural-language queries to an oracle, and builds VL-LN, a benchmark with 41,891 automatically generated long-horizon trajectories, an evaluation oracle, and a new metric, Mean Success Progress (MSP), that measures how much each extra dialog turn buys in navigation success. Training a dialog-enabled agent on this data improves IIGN success from 14.8% (no dialog) to 20.2%, and the paper attributes the remaining gap to exploration difficulty and, above all, to image–attribute alignment: 73% of failures involve missing or misidentifying the target under detailed attributes. The contribution is therefore not only a dataset but evidence that proactive querying partially addresses exploration and disambiguation, while grounding remains open.

Core claim

On the paper's own terms, the central discovery is that IIGN is a well-posed, learnable task: a single multimodal policy can be trained to interleave navigation actions with natural-language questions, and doing so improves both the interactive setting (IIGN) and the non-interactive instance-goal setting (IGN) to the best results among evaluated baselines (SR 20.2 and 25.0, respectively). The same experiments identify the task's two core bottlenecks — long-horizon exploration and fine-grained instance grounding — and show that dialog partially relieves both: exploration failures drop from 159 to 145 in IIGN and from 143 to 124 in IGN when dialog is enabled, and the agent can follow short rou

What carries the argument

The load-bearing object is the agent–oracle interaction loop defined by three question types — attribute, route, and disambiguation — together with the automated oracle used for data collection and evaluation. The oracle answers attribute queries from privileged instance metadata, converts shortest paths to four-meter natural-language route instructions via a rule-based waypoint procedure, and confirms the target only when it is centered and within three meters. This oracle makes the task fully automated: the same mechanism generates 41,891 training trajectories (collected by a frontier-based explorer that asks questions at scripted triggers) and evaluates any agent online. The new MSP metri

Load-bearing premise

The experiments stand or fall on whether the scripted oracle—which answers from privileged true shortest paths and metadata—behaves like a genuinely helpful human; only a 100-episode cross-role comparison supports this, and it shows very different dialog costs.

What would settle it

Run the same 500 test episodes with a human oracle answering the agent's questions instead of the scripted oracle and compare SR, SPL, and MSP. If human-answered runs show materially different success rates or markedly different dialog-turn counts — the 100-episode study already shows 9.72 vs 2.04 turns — then the oracle is not a faithful proxy and the 'dialog helps' conclusion may not transfer to real users.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Training on VL-LN yields a dialog-enabled agent that achieves the best measured results on both IIGN (20.2 SR) and IGN (25.0 SR), improving on the strongest dialog-free baseline by about six points.
  • Active dialog reduces exploration failures and ambiguity failures, but the dominant failure mode becomes image–attribute alignment; 73% of residual failures are detection errors, so better grounding is the most promising next step.
  • The benchmark is practical as an automated testbed: human performance is high (93 SR), the oracle approximates human support in terms of success (91 vs 93), and the agent's questioning is economical (fewer than two turns on average even with high budgets).
  • The new MSP metric measures dialog utility by averaging success improvements across increasing dialog-turn budgets, capturing both the effectiveness and the economy of queries.
  • Co-training with vision-and-language navigation data is what lets the agent translate short route guidance into executable actions; dialog data alone would not suffice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the oracle's answers are more informative or more forgiving than a real human's, the MSP gains reported for the dialog-enabled agent could overstate the benefit of learning to ask questions; the cross-role study's large turn-count gap (9.72 vs 2.04) is a warning sign.
  • A testable extension: train with hard-negative images — same-category objects with different attributes — as the paper itself suggests; if grounding errors drop substantially, image–attribute alignment, not question selection, is the binding constraint.
  • The three-type question taxonomy (attribute, route, disambiguation) could be adopted as a general interface for other embodied tasks such as rearrangement or social navigation, but its success depends on the oracle answering from privileged ground truth rather than partial observability.
  • The human–human two-turn average suggests a ceiling: well-chosen questions are worth more than many questions, so a promising research direction is selecting maximally informative questions based on observation history, which the paper leaves implicit.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Interactive Instance Goal Navigation (IIGN), which extends Instance Goal Navigation by allowing an agent to query an oracle in free-form natural language during long-horizon navigation. To support IIGN, the authors build the VL-LN Bench, an automated pipeline that produces about 41k dialog-augmented trajectories from MMScan-annotated MP3D scenes, together with an online evaluation protocol and a scripted oracle. They train three VLLN baselines initialized from Qwen2.5-VL-7B and report that the dialog-enabled model VLLN-D achieves the best results on both IIGN and IGN (SR 20.2 and 25.0 respectively). The paper also provides failure analysis, a cross-role human study, and a sensitivity analysis of dialog-turn budgets, concluding that active dialog partially alleviates exploration and grounding bottlenecks.

Significance. If the benchmark is reliable, it fills a real gap: existing interactive navigation datasets are either small, room-level, or lack online evaluation, whereas VL-LN provides a house-scale, long-horizon dataset with oracle-based evaluation and a concrete training recipe. The automated collection pipeline, the large trajectory count, and the quantitative failure analysis are useful assets for the community. The central empirical claim—that dialog improves instance-goal navigation—is plausible and supported by several ablations, but the paper currently overstates the strength of the evidence: the oracle's validity as a human proxy rests on only 100 episodes, and no uncertainty quantification is provided for any headline number. These issues are fixable within the scope of a revision, but they are load-bearing for the paper's main conclusions.

major comments (3)
  1. [Sec. III-B, Sec. IV-C, Table IV] The oracle's privileged access is not sufficiently validated as a human proxy. Route queries are answered from a true shortest path and attribute queries from exact metadata; disambiguation is a deterministic centering/distance rule. The only human comparison (Table IV) has 100 episodes: Human-GPT achieves 91 SR vs 93 for Human-Human, but uses 9.72 vs 2.04 turns on average. That large turn-cost difference indicates qualitatively different interaction, and the 2-point SR gap is not statistically informative at this sample size. Moreover, VLLN-D with Human vs GPT oracle gives 16 vs 17 SR, too low and insensitive to establish equivalence. Since the abstract and Section IV-C claim the oracle is 'comparable to human support' and the measured dialog benefit rests on this, the authors should add a substantially larger human study with confidence intervals, and/or ablate with a degraded oracle (
  2. [Tables II, III, and V] No error bars, confidence intervals, or significance tests are reported anywhere. All headline numbers are single-run point estimates. For example, IGN VLLN-D 25.0 vs VLLN-D* 21.8 is a 3.2-point gap; with n=500 test episodes and no repeated seeds, this is within normal sampling noise. The MSP gains (2.76 and 2.16) and the failure-count differences in Table III also lack intervals. The claims of 'significant improvements', 'state-of-the-art results', and 'dialog helps' need at least bootstrap confidence intervals over test episodes, and ideally multiple training seeds for the learned baselines.
  3. [Sec. IV-B, Table II] For IIGN, the dialog benefit is inferred by comparing VLLN-I (trained without dialog) to VLLN-D (trained with dialog). Unlike the IGN block, there is no no-response control (VLLN-D*) for IIGN. The IIGN improvement could therefore come from the added dialog supervision or from the model's increased capacity to generate language, rather than from the oracle's answers. The authors should add the VLLN-D* condition for IIGN (agent may ask, oracle does not respond), or explicitly state that the causal attribution of the IIGN gain to oracle responses is not established by the current design.
minor comments (4)
  1. [Eq. (1)] Notation is inconsistent: the text defines s_0, s_1, ..., s_n, but Eq. (1) uses S_i and S_0. Please unify the symbols and clarify that S_i = s_i.
  2. [Sec. IV-A] The phrase 'filtered IGN data (11,661 trajectories)' is never defined. What filtering was applied to the 41,891 collected trajectories to obtain the IGN subset? A one-sentence description would help reproducibility.
  3. [Abstract / Sec. IV-B] The paper uses 'state-of-the-art results' but compares only five baselines, three of which are ablated versions of the same architecture. Please qualify the claim (e.g., 'state-of-the-art among evaluated baselines') or add comparisons to more recent IGN methods.
  4. [Sec. III-C and Fig. 2] The route-question trigger is 'when the best frontier is chosen', but the exploration policy selects the best frontier with only 10% probability. It would be clearer to state explicitly whether route questions in the training set are therefore conditioned on a privileged signal that is not available at test time, and how this affects the learned question-asking policy.

Circularity Check

0 steps flagged

No circularity: VL-LN's derivation chain is a self-contained benchmark construction and empirical evaluation; dependencies on MMScan/InternVLA-N1 are inputs, not conclusions defined by outputs.

full rationale

The paper's central chain is: define IIGN -> build VL-LN via the pipeline -> train VLLN-D on generated trajectories -> evaluate against baselines and compare with/without dialog. Each link is empirical or definitional but not self-referential: the model's success is measured against an oracle fixed by the task, and dialog utility is established by comparing VLLN-D with dialog to no-dialog conditions (Table II, Table V), not by construction from the training distribution. The MSP metric (Eq. 1) is a descriptive evaluation statistic, not a fitted parameter used to produce the reported SR numbers. The only same-team dependencies are MMScan [6] as the annotation source and InternVLA-N1 [36] as the training recipe/data; both are external inputs and do not carry the paper's conclusions. The GPT-4o+rule oracle is part of the environment, and its comparability to humans is tested in a cross-role study (Table IV); whether that validation is sufficient is a correctness/validity concern, not circularity. No step was found where a predicted quantity reduces by definition to an input.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claims rest on the reliability of the MMScan annotations that seed the pipeline, the GPT-4o oracle used for evaluation, and hand-set thresholds defining success and dialog budgets. These are all stated in the paper but none are independently verified beyond a 100-episode human comparison.

free parameters (5)
  • viewpoint expansion distance = 0.6 m
    Expands each instance 3D bbox to define success viewpoints; hand-set in §III-C2, affects episode difficulty and the success criterion.
  • stop threshold = 0.25 m
    Agent must stop within 0.25 m of a viewpoint; standard but hand-set, affects success rate.
  • FBE best-frontier probability = 0.1 (10%)
    During data collection the policy selects the frontier closest to the target 10% of the time (§III-C3); shapes the training distribution and how often route questions appear.
  • route-guidance horizon = 4 m
    Oracle converts only the first 4 m of shortest path into language (§III-B); limits the amount of privileged help and affects dialog utility.
  • MSP dialog allowance = n=5
    Mean Success Progress averages success improvements over up to 5 dialog budgets (§III-E).
axioms (5)
  • domain assumption MMScan/MP3D instance and region annotations are accurate, sufficiently complete, and aligned with visual observations.
    House-level instance and region dictionaries are built from MMScan hierarchical labels (§III-C1); if annotations are wrong, generated instructions and oracle answers are wrong.
  • domain assumption The GPT-4o oracle with deterministic rules produces correct answers to arbitrary agent questions; in particular its disambiguation test (target centered in view and within 3 m) is a valid proxy for recognizing the target.
    Section III-B defines the oracle; human-GPT validation is only on 100 episodes and is imperfect (human-GPT needs 9.72 turns vs 2.04 human-human).
  • domain assumption Trajectories generated by frontier-based exploration with a 90/10 oracle-biased frontier choice and a ground-truth detector form a useful training distribution for learned dialog policies.
    The data-collection pipeline (§III-C3) uses privileged target knowledge; the paper assumes this distribution transfers to agents that must explore without it.
  • domain assumption The 500 test episodes and 100 cross-role episodes are representative samples of the benchmark distribution.
    No sampling procedure or random seed is given; statistical noise is not reported.
  • domain assumption GPT-4o-generated full descriptions uniquely identify the target instance.
    Full instructions are created by prompting GPT-4o with discriminative attributes and relations (§III-C2); if they are ambiguous, IGN evaluation is compromised.

pith-pipeline@v1.3.0-alltime-deepseek · 11588 in / 15190 out tokens · 152046 ms · 2026-08-03T13:52:08.911533+00:00 · methodology

0 comments
read the original abstract

In most existing embodied navigation tasks, instructions are well-defined and unambiguous, such as instruction following and object searching. Under this idealized setting, agents are required solely to produce effective navigation outputs conditioned on vision and language (VL) inputs. Real-world instructions, however, are often underspecified and require interaction to resolve ambiguity and infer user intent. To bridge this gap, we propose Interactive Instance Goal Navigation (IIGN), which extends Instance Goal Navigation (IGN) by allowing agents to freely consult an oracle in natural language while searching for a specific instance. IIGN requires agents to produce both Language and Navigation (LN) outputs, enabling interaction while moving in the environment. To support this task, we introduce VL-LN Bench, a benchmark with an automated data collection pipeline and over 41k collected long-horizon dialog-augmented trajectories for training, alongside an automatic evaluation protocol paired with a dedicated oracle for answering agent queries. Experiments reveal two core bottlenecks of IIGN: long-horizon exploration and fine-grained grounding of textual information to the correct instance among same-category distractors. Although active dialog partially alleviates these challenges, current models still lag far behind human performance. Further ablations validate the value of the data generated by our pipeline and show that the proposed oracle provides scalable assistance comparable to human support, proving VL-LN Bench as a practical testbed for dialog-enabled embodied navigation.

Figures

Figures reproduced from arXiv: 2512.22342 by Feng Zhao, Hanqing Wang, Jiangmiao Pang, Jinming Xu, Meng Wei, Shaohao Zhu, Siqi Zhang, Tai Wang, Wensi Huang, Xihui Liu.

Figure 1
Figure 1. Figure 1: A case for the IIGN task. The oracle (top left) first gives a simple goal-oriented navigation instruction (“Search for the chair.”). The agent has to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Automatic pipeline for collecting dialog-augmented trajectories. We first aggregate room-level instance attributes into unified house-level annotations. We then pair each target instance with a start point to generate episodes. Finally, we collect dialog-augmented trajectories using a frontier￾based exploration (FBE) agent that, with 90% probability, selects the frontier nearest to the previously chosen fr… view at source ↗
Figure 3
Figure 3. Figure 3: Trajectory statistics of the VL–LN dataset. (a–b) Frequency histograms of per-episode path steps, and dialog turns; (c) frequency histogram of per-turn dialog length (tokens). Black lines denote smoothed density fits. (d) Nested donut of dialog data. Outer ring: target-instance category proportions; inner disk: question-type proportions (Attribute, Dis￾ambiguation, and Route question). IGN and 23,774 Objec… view at source ↗
Figure 4
Figure 4. Figure 4: Failure cases. Green curves denote the geodesic shortest paths; blue curves are the navigator’s exploration trajectories; red shaded regions indicate the success zone around the target. (a) Referential ambiguity: within the same view, the navigator and the NPC refer to different instances, causing the navigator to stop at a wrong instance. (b) Partial observability: the navigator only observes a single can… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation

    cs.CV 2026-07 conditional novelty 6.0

    REAL, a benchmark and trained vision-language agent for oracle-free mobile manipulation with user interaction, achieves 78.3% end-to-end success on 60 physical-robot episodes after simulation-only high-level training.

  2. Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation

    cs.CV 2026-06 unverdicted novelty 4.0

    Proposes cost-aware question selection for ambiguous object navigation via information-gain analysis on corpora, a cost-penalizing benchmark, and a zero-shot MLLM agent.

Reference graph

Works this paper leans on

37 extracted references · 10 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Ion: Instance- level object navigation,

    W. Li, X. Song, Y . Bai, S. Zhang, and S. Jiang, “Ion: Instance- level object navigation,” inProceedings of the 29th ACM international conference on multimedia, 2021, pp. 4343–4352

    2021

  2. [2]

    Objectnav revisited: On evaluation of embodied agents navigating to objects,

    D. Batra, A. Gokaslan, A. Kembhavi, O. Maksymets, R. Mottaghi, M. Savva, A. Toshev, and E. Wijmans, “Objectnav revisited: On evaluation of embodied agents navigating to objects,” 2020. [Online]. Available: https://arxiv.org/abs/2006.13171

    Pith/arXiv arXiv 2020

  3. [3]

    Vision- and-dialog navigation,

    J. Thomason, M. Murray, M. Cakmak, and L. Zettlemoyer, “Vision- and-dialog navigation,” inConference on Robot Learning. PMLR, 2020, pp. 394–406

    2020

  4. [4]

    Dialfred: Dialogue-enabled agents for embodied instruction follow- ing,

    X. Gao, Q. Gao, R. Gong, K. Lin, G. Thattai, and G. S. Sukhatme, “Dialfred: Dialogue-enabled agents for embodied instruction follow- ing,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 049– 10 056, 2022

    2022

  5. [5]

    Collaborative instance object navigation: Leveraging uncertainty-awareness to minimize human-agent dialogues,

    F. Taioli, E. Zorzi, G. Franchi, A. Castellini, A. Farinelli, M. Cristani, and Y . Wang, “Collaborative instance object navigation: Leveraging uncertainty-awareness to minimize human-agent dialogues,”arXiv preprint arXiv:2412.01250, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Mmscan: A multi-modal 3d scene dataset with hierarchical grounded language annotations,

    R. Lyu, J. Lin, T. Wang, S. Yang, X. Mao, Y . Chen, R. Xu, H. Huang, C. Zhu, D. Lin, and J. Pang, “Mmscan: A multi-modal 3d scene dataset with hierarchical grounded language annotations,”

  7. [7]

    Multimodal perception for goal-oriented navigation: A survey,

    I.-T. Ieong and H. Tang, “Multimodal perception for goal-oriented navigation: A survey,”arXiv preprint arXiv:2504.15643, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    On evaluation of embodied navigation agents,

    P. Anderson, A. Chang, D. S. Chaplot, A. Dosovitskiy, S. Gupta, V . Koltun, J. Kosecka, J. Malik, R. Mottaghi, M. Savvaet al., “On evaluation of embodied navigation agents,”arXiv preprint arXiv:1807.06757, 2018

    Pith/arXiv arXiv 2018

  9. [9]

    Exploiting scene- specific features for object goal navigation,

    T. Campari, P. Eccher, L. Serafini, and L. Ballan, “Exploiting scene- specific features for object goal navigation,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 406–421

    2020

  10. [10]

    Offline visual representation learning for embodied navigation,

    K. Yadav, R. Ramrakhya, A. Majumdar, V .-P. Berges, S. Kuhar, D. Ba- tra, A. Baevski, and O. Maksymets, “Offline visual representation learning for embodied navigation,” inWorkshop on Reincarnating Reinforcement Learning at ICLR 2023, 2023

    2023

  11. [11]

    Auxiliary tasks and explo- ration enable objectgoal navigation,

    J. Ye, D. Batra, A. Das, and E. Wijmans, “Auxiliary tasks and explo- ration enable objectgoal navigation,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16 117–16 126

    2021

  12. [12]

    Implicit obstacle map-driven in- door navigation model for robust obstacle avoidance,

    W. Xie, H. Jiang, S. Gu, and J. Xie, “Implicit obstacle map-driven in- door navigation model for robust obstacle avoidance,” inProceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 6785–6793

    2023

  13. [13]

    Unigoal: Towards universal zero-shot goal-oriented navigation,

    H. Yin, X. Xu, L. Zhao, Z. Wang, J. Zhou, and J. Lu, “Unigoal: Towards universal zero-shot goal-oriented navigation,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 19 057–19 066

    2025

  14. [14]

    Vlfm: Vision- language frontier maps for zero-shot semantic navigation,

    N. Yokoyama, S. Ha, D. Batra, J. Wang, and B. Bucher, “Vlfm: Vision- language frontier maps for zero-shot semantic navigation,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 42–48

    2024

  15. [15]

    A survey of imitation learning: Algorithms, recent developments, and challenges,

    M. Zare, P. M. Kebria, A. Khosravi, and S. Nahavandi, “A survey of imitation learning: Algorithms, recent developments, and challenges,” IEEE Transactions on Cybernetics, 2024

    2024

  16. [16]

    Reinforcement learning: A survey,

    L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,”Journal of artificial intelligence research, vol. 4, pp. 237–285, 1996

    1996

  17. [17]

    How to not train your dragon: Training-free embodied object goal navigation with semantic frontiers,

    J. Chen, G. Li, S. Kumar, B. Ghanem, and F. Yu, “How to not train your dragon: Training-free embodied object goal navigation with semantic frontiers,”arXiv preprint arXiv:2305.16925, 2023

    Pith/arXiv arXiv 2023

  18. [18]

    Mo- ddn: A coarse-to-fine attribute-based exploration agent for multi- object demand-driven navigation,

    W. Cai, H. Wang, P. Liu, M. Wu, Z. Qian, and H. Dong, “Mo- ddn: A coarse-to-fine attribute-based exploration agent for multi- object demand-driven navigation,”Advances in Neural Information Processing Systems, vol. 37, pp. 64 176–64 214, 2024

    2024

  19. [19]

    A frontier-based approach for autonomous exploration,

    B. Yamauchi, “A frontier-based approach for autonomous exploration,” inProceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97. ’Towards New Com- putational Principles for Robotics and Automation’. IEEE, 1997, pp. 146–151

    1997

  20. [20]

    L3mvn: Leveraging large language models for visual target navigation,

    B. Yu, H. Kasaei, and M. Cao, “L3mvn: Leveraging large language models for visual target navigation,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 3554–3560

    2023

  21. [21]

    Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill,

    W. Cai, S. Huang, G. Cheng, Y . Long, P. Gao, C. Sun, and H. Dong, “Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 5228–5234

    2024

  22. [22]

    Zson: Zero-shot object-goal navigation using multimodal goal embed- dings,

    A. Majumdar, G. Aggarwal, B. Devnani, J. Hoffman, and D. Batra, “Zson: Zero-shot object-goal navigation using multimodal goal embed- dings,”Advances in Neural Information Processing Systems, vol. 35, pp. 32 340–32 352, 2022

    2022

  23. [23]

    Personalized instance-based navigation toward user-specific objects in realistic environments,

    L. Barsellotti, R. Bigazzi, M. Cornia, L. Baraldi, and R. Cucchiara, “Personalized instance-based navigation toward user-specific objects in realistic environments,”Advances in Neural Information Processing Systems, vol. 37, pp. 11 228–11 250, 2024

    2024

  24. [24]

    Prioritized semantic learning for zero-shot instance navigation,

    X. Sun, L. Liu, H. Zhi, R. Qiu, and J. Liang, “Prioritized semantic learning for zero-shot instance navigation,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 161–178

    2024

  25. [25]

    Vision-based nav- igation with language-based assistance via imitation learning with indirect intervention,

    K. Nguyen, D. Dey, C. Brockett, and B. Dolan, “Vision-based nav- igation with language-based assistance via imitation learning with indirect intervention,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 12 527–12 537

    2019

  26. [26]

    Just ask: An interactive learning framework for vision and language navigation,

    T.-C. Chi, M. Shen, M. Eric, S. Kim, and D. Hakkani-Tur, “Just ask: An interactive learning framework for vision and language navigation,” inProceedings of the AAAI conference on artificial intelligence, vol. 34, no. 03, 2020, pp. 2459–2466

    2020

  27. [27]

    Findthis: Language- driven object disambiguation in indoor environments,

    A. Majumdar, F. Xia, D. Batra, L. Guibaset al., “Findthis: Language- driven object disambiguation in indoor environments,” in7th Annual Conference on Robot Learning, 2023

    2023

  28. [28]

    Alexa arena: A user-centric interactive platform for embodied ai,

    Q. Gao, G. Thattai, S. Shakiah, X. Gao, S. Pansare, V . Sharma, G. Sukhatme, H. Shi, B. Yang, D. Zhanget al., “Alexa arena: A user-centric interactive platform for embodied ai,”Advances in Neural Information Processing Systems, vol. 36, pp. 19 170–19 194, 2023

    2023

  29. [29]

    Rmm: A recursive mental model for dialog navigation,

    H. R. Roman, Y . Bisk, J. Thomason, A. Celikyilmaz, and J. Gao, “Rmm: A recursive mental model for dialog navigation,”arXiv preprint arXiv:2005.00728, 2020

    Pith/arXiv arXiv 2005

  30. [30]

    Robots that ask for help: Uncertainty alignment for large language model planners,

    A. Z. Ren, A. Dixit, A. Bodrova, S. Singh, S. Tu, N. Brown, P. Xu, L. Takayama, F. Xia, J. Varleyet al., “Robots that ask for help: Uncertainty alignment for large language model planners,”arXiv preprint arXiv:2307.01928, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Habitat 2.0: Training home assistants to rearrange their habitat,

    A. Szot, A. Clegg, E. Undersander, E. Wijmans, Y . Zhao, J. Turner, N. Maestre, M. Mukadam, D. S. Chaplot, O. Maksymetset al., “Habitat 2.0: Training home assistants to rearrange their habitat,” Advances in neural information processing systems, vol. 34, pp. 251– 266, 2021

    2021

  32. [32]

    Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes,

    P. Achlioptas, A. Abdelreheem, F. Xia, M. Elhoseiny, and L. Guibas, “Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes,” inEuropean conference on computer vision. Springer, 2020, pp. 422–440

    2020

  33. [33]

    Beyond the nav-graph: Vision-and-language navigation in continuous environ- ments,

    J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee, “Beyond the nav-graph: Vision-and-language navigation in continuous environ- ments,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 104–120

    2020

  34. [34]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radfordet al., “Gpt-4o system card,”arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Doll ´ar, and C. Feichtenhofer, “Sam 2: Segment anything in images and videos,” 2024. [Online]. Available: https://arxiv.org/abs/2408.00714

    Pith/arXiv arXiv 2024

  36. [36]

    InternVLA-N1: An open dual-system navigation founda- tion model with learned latent plans,

    I.-N. Team, “InternVLA-N1: An open dual-system navigation founda- tion model with learned latent plans,” 2025

    2025

  37. [2025]

    Available: https://arxiv.org/abs/2406.09401

    [Online]. Available: https://arxiv.org/abs/2406.09401

    Pith/arXiv arXiv