Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

MAG-Nav: Language-Driven Object Navigation Leveraging Memory-Reserved Active Grounding

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MAG-Nav, a training-free language-driven navigation framework, achieves state-of-the-art zero-shot object navigation by combining viewpoint optimization and memory replay with an off-the-shelf VLM.

desk verdict A credible SOTA claim for zero-shot language-driven object navigation, but the active-viewpoint ablation is confounded and there are no error bars. read the letter →

arxiv 2508.05021 v1 pith:7DTUKM5E submitted 2025-08-07 cs.RO

Weifan Zhang , Tingguang Li , Yuzhen Liu This is my paper
classification cs.RO
keywords language-drivenobjectnavigationzero-shotvisualgroundingactiveperceptionviewpointoptimizationmemoryreplayvision-languagemodelsL-ZSON
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a robot can navigate to objects described in free-form natural language, in unseen environments and without any training, by actively improving the visual input it feeds to a vision-language model rather than passively accepting whatever the camera sees. It claims this is done with two human-inspired mechanisms: perspective-based active grounding, which plans a better viewpoint before confirming a target, and historical memory backtracking, which stores uncertain observations and re-evaluates them later. On the GOAT-Bench validation set, the full system reaches a 0.408 Success rate versus 0.367 for the strongest baseline, and an ablation shows that removing either the active-grounding or memory-replay module drops success to 0.307 or 0.313. The authors argue that this makes off-the-shelf VLMs practical for zero-shot language-driven object navigation without labeled data or fine-tuning.

What carries the argument

Memory-Reserved Active Grounding: a three-stage grounding loop carried by the robot's ability to move. Stage one is VLM-based initial grounding, where GPT-4o selects a numbered bounding box from an annotated keyframe. Stage two is active grounding, where a viewpoint optimization with objective f(v) = -R_visible(v) - R_fov(v) + P_distance(v) + P_feasibility(v) selects a better pose in SE(2) and the VLM is re-queried; only a double-confirmed target is accepted. Stage three is reserved grounding, which replays uniformly sampled keyframes from visual memory and asks the VLM to identify the target across images. The mechanism converts a static image-interpretation problem into an embodied percept

What would settle it

On the same GOAT-Bench validation split, run the full MAG-Nav pipeline but replace the optimized viewpoint with a random reachable viewpoint at a similar distance from the target. If the Success rate does not drop below the reported 0.408 (or if the ablation gap between w/o A-G and full MAG-Nav disappears), the viewpoint objective is not the carrier of the claimed active-grounding benefit.

Watch

Extended reading notes

Core claim

The central claim is that a zero-shot language-driven object navigation (L-ZSON) agent can outperform prior work by treating its own mobility as a way to improve VLM grounding quality. The system first builds 3D object memory and annotated keyframe memory from egocentric RGB-D observations. When a VLM (GPT-4o) tentatively grounds the target from a keyframe, the robot computes an improved observation pose by optimizing a discrete objective over a 2D grid, balancing target visibility, field-of-view coverage, distance, and traversability, then re-queries the VLM from that pose. If the target is still uncertain after exploration, the robot samples up to 13 stored keyframes and asks the VLM to lo

Load-bearing premise

The central risk is that the hand-designed viewpoint scoring function f(v) is assumed to rank viewpoints by how accurately the VLM will ground the target, even though that relationship is never directly measured; if the score picks views the VLM handles worse, the active grounding gain may not transfer to other scenes.

Editorial extensions

If this is right

  • If correct, zero-shot language navigation can be improved without any model training by adding an explicit viewpoint-planning step before committing to a VLM grounding decision.
  • The 0.408 Success versus 0.367 baseline suggests that active perception and memory replay are worth more than a stronger grounding model alone on this benchmark.
  • The memory replay design implies that a robot can answer queries about objects it passed earlier, enabling cross-task recall without retraining.
  • The framework's reliance on off-the-shelf detectors (DINO), segmenters (SAM), and a generative VLM means future improvements in any of these components should transfer directly to navigation performance.
  • The reported 500-step episode budget and 0.3-meter success radius mean the metric captures both accuracy and path efficiency, so gains in Success and SPL indicate practical navigation improvements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the same viewpoint objective could be applied to other VLM-based embodied tasks, such as open-vocabulary detection or 3D scene understanding, wherever the robot can choose where to look before querying the model.
  • The paper's framing implies that the principal bottleneck in language-grounded navigation is not the VLM's reasoning ability but the quality and completeness of the egocentric frames it is asked to interpret; if so, perception-driven viewpoint planning should matter more as language descriptions become more ambiguous.
  • An unstated consequence is that the memory buffer size (m_max = 13) and uniform sampling strategy could be tuned per environment: query-driven retrieval based on language relevance might outperform timestamp-based uniform sampling when many objects look alike.
  • The real-world active-grouping example suggests that false-positive rejections from a better viewpoint are a key failure mode to quantify; a controlled experiment measuring how often a second viewpoint overturns an initial correct grounding would sharpen the claimed benefit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. MAG-Nav is a zero-shot, training-free framework for language-driven object navigation. It combines off-the-shelf components: DINO as the open-vocabulary detector, SAM for segmentation, CLIP for object features, and GPT-4o for visual grounding. The two claimed contributions are (1) perspective-based active grounding, which optimizes a viewpoint using a hand-designed objective f(v) and performs a second VLM grounding pass from that viewpoint, and (2) memory-reserved grounding, which replays sampled keyframes from visual memory when the target is not found during exploration. In simulation on GOAT-Bench/HM3D, the paper reports SR 0.408 vs 0.367 for the VLN-Game+GPT-4o baseline, and an ablation in which removing both modules drops SR to 0.285. Real-world experiments on a quadruped robot are described qualitatively.

Significance. If the result holds, the paper makes a useful empirical contribution: it shows that a modular, training-free pipeline can improve zero-shot language-driven object navigation on a standard benchmark, and that active viewpoint adjustment and memory replay can help off-the-shelf VLMs in egocentric settings. The paper also promises code release, which is valuable for reproducibility. The simulated comparison includes a strong GPT-4o baseline, and the ablation pattern is internally consistent. However, the central claim is not yet statistically robust, and the active-grounding ablation currently conflates viewpoint selection with the effect of an extra VLM call. The work is therefore suggestive rather than conclusive in its present form.

major comments (3)
  1. [§V-B5, Table II] The ablation labeled "w/o A-G" removes both the viewpoint-optimization step and the second VLM grounding call. The improvement from 0.307 to 0.405 is therefore compatible with the simpler explanation that the agent benefits merely from an additional independent GPT-4o evaluation, and it does not isolate the contribution of the viewpoint selected by f(v). Add control conditions in which the second grounding is performed at the same initial viewpoint or at a random feasible viewpoint while keeping the extra VLM call. This control is essential because the Rvisible/Rfov terms in Eq. (1) are heuristic and are not directly validated.
  2. [§V-B4, Table I and §V-B5, Table II] The main SOTA comparison reports no variance, confidence intervals, or significance tests for the 0.408 vs 0.367 SR gap, so the claim of outperforming all baselines is not yet statistically supported. In addition, the same full system is reported as SR=0.408 in Table I and SR=0.405 in Table II; these numbers must be reconciled (e.g., different subsets/seeds) or corrected. Please report multiple seeds or bootstrap confidence intervals and state the evaluation protocol explicitly.
  3. [§IV-C2, §V-A] The viewpoint objective f(v) relies on hand-set weights wvisible=15, wfov=7, wdistance=1, Cinfeasible=1000, plus additional parameters (ddesired, δsim, Nsamples, genetic-algorithm settings) that the text says appear in the Appendix but are not present in the manuscript. No sensitivity analysis is provided for any of these values. Since the claimed contribution is the heuristic viewpoint objective, the absence of such analysis leaves open the possibility that the gain in Table II depends on constants tuned to the benchmark. Please add a sensitivity study over at least the three weights, or compare against a simple viewpoint heuristic.
minor comments (5)
  1. [§IV-C2] Typo: "physically traversable voint" should read "physically traversable point."
  2. [§V-A] The manuscript states that implementation details are in the Appendix, but the submitted text contains no Appendix. Please include the missing parameters (δsim, ddesired, Nsamples, population size and iterations for DEAP, GPT-4o version/date) for reproducibility.
  3. [§V-B5] The row "w/o A-M-G and w/o raw" is ambiguous regarding whether the raw RGB image is removed in addition to both modules, and whether this is the intended no-module baseline. Clarify in the table caption or text.
  4. [§IV-C3] When the VLM is asked to ground among Nsamples keyframes, the paper does not define the output when no keyframe contains the target. A "no match" option is needed to make the reserved-grounding stage well specified.
  5. [§V-C] The real-world evaluation is entirely qualitative. A compact table with per-task success/failure and time or step counts would strengthen the claim of practical effectiveness. The code-release statement also lacks a repository URL.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity: empirical comparisons dominate; only non-load-bearing self-citation of the authors' prior VLN-Game baseline/component.

full rationale

The manuscript was checked for all circularity patterns. The central claim (Section V-B4, Table I) is an empirical comparison against external baselines (CLIP, CoW, LLM2CLIP) plus the authors' own VLN-Game [40]; outperforming one's own prior system is a normal, non-circular comparison. The ablation (Section V-B5, Table II) compares the full system with variants lacking active grounding and/or memory replay; the measured Success Rate is the task metric, not a quantity derived from the method's equations, so there is no fitted-input-called-prediction. The viewpoint objective f(v) is a hand-designed heuristic with fixed weights (wvisible=15, wfov=7, wdistance=1), not a parameter fitted to the evaluation labels, so it is not a fitted quantity renamed as a prediction. The only reuse of the authors' prior work is the frontier-exploration and semantic-map component cited to [40] in Section IV-D; this is a reused module, and [40] is also used as a baseline, not as a justification of methodological correctness or uniqueness. That self-citation is therefore non-load-bearing. The 'w/o A-G' ablation does conflate the viewpoint optimization with the extra VLM grounding call, and the hand-set weights and memory cap (mmax=13) may be overfit to the evaluation benchmark; these are correctness/generalization risks, not circularity. No self-definitional equations, imported uniqueness theorems, or ansatz-smuggling via citation were found. The paper's stated limitation about GPT-4o latency does not bear on circularity.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

No new physical entities are postulated; visual and object memories are software data structures, not entities requiring independent evidence. The main extras the paper relies on are hand-set weights and borrowed thresholds, which are listed above.

free parameters (8)
  • wvisible = 15.0
    Weight of visibility reward in viewpoint optimization; chosen by hand, no derivation or sensitivity analysis.
  • wfov = 7.0
    Weight of field-of-view reward; hand-set.
  • wdistance = 1.0
    Weight of distance penalty; hand-set.
  • Cinfeasible = 1000.0
    Large penalty for infeasible viewpoints; hand-set.
  • mmax = 13
    Memory buffer cap; chosen for context limits; affects memory replay.
  • delta_sim = unspecified
    Similarity threshold for object association across frames, borrowed from [38]; not given a value, yet determines when a new object is instantiated and keyframes saved.
  • Nsamples = unspecified
    Number of keyframes sampled for reserved grounding; not specified.
  • ddesired = unspecified
    Desired viewing distance in the distance penalty; not specified.
assumptions (4)
  • domain assumption Viewpoints that maximize f(v) = -Rvisible - Rfov + Pdistance + Pfeasibility are viewpoints from which GPT-4o grounding accuracy is higher.
    This is the core premise of active grounding; validated only indirectly by the ablation, not by any independent measurement of VLM accuracy vs viewpoint.
  • domain assumption DINO open-vocabulary detection can detect the target and landmark classes from the instruction at some point during exploration, and SAM segmentation plus CLIP features provide a reliable object memory.
    The entire pipeline fails if the target or landmark is never detected; the paper provides no failure analysis on detection misses.
  • domain assumption GPT-4o with the described visual prompts (annotated bounding-box image + raw RGB image) returns the correct bounding box identifier reliably enough for navigation decisions.
    The method's correctness hinges on the generative VLM's grounded reasoning; only a few qualitative examples are shown, and there is no per-step accuracy measurement.
  • domain assumption The evaluation protocol on the GOAT-Bench validation subset (36 scenes, 316 instructions) is representative of 'diverse and open-ended language descriptions'.
    The claim of strong generalization is based on a single benchmark subset; no other environments or instruction distributions are tested in simulation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MAG-Nav: Language-Driven Object Navigation Leveraging Memory-Reserved Active Grounding." pith.science (2026). https://pith.science/paper/7DTUKM5E

@misc{pith2026250805021,
  author       = {Pith},
  title        = {Pith review of: MAG-Nav: Language-Driven Object Navigation Leveraging Memory-Reserved Active Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7DTUKM5E}},
  note         = {Machine review of arXiv:2508.05021}
}
read the original abstract

Visual navigation in unknown environments based solely on natural language descriptions is a key capability for intelligent robots. In this work, we propose a navigation framework built upon off-the-shelf Visual Language Models (VLMs), enhanced with two human-inspired mechanisms: perspective-based active grounding, which dynamically adjusts the robot's viewpoint for improved visual inspection, and historical memory backtracking, which enables the system to retain and re-evaluate uncertain observations over time. Unlike existing approaches that passively rely on incidental visual inputs, our method actively optimizes perception and leverages memory to resolve ambiguity, significantly improving vision-language grounding in complex, unseen environments. Our framework operates in a zero-shot manner, achieving strong generalization to diverse and open-ended language descriptions without requiring labeled data or model fine-tuning. Experimental results on Habitat-Matterport 3D (HM3D) show that our method outperforms state-of-the-art approaches in language-driven object navigation. We further demonstrate its practicality through real-world deployment on a quadruped robot, achieving robust and effective navigation performance.

Figures

Figures reproduced from arXiv: 2508.05021 by the authors.

Figure 1
Figure 1. Example of active grounding. Given the instruction ”Please find my black bag on the red stool next to the trash [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed MAG-Nav system. Given a natural language instruction, the system extracts the target [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of VLM grounding results. Each task includes an instruction and annotated images used for grounding in both the initial grounding phase and the active grounding phase, along with their corresponding VLM text outputs. The target object in the instruction is highlighted in red font. The grounding results provided by the VLM are highlighted in blue font, where obj i indicates that the i-th bounding box in th… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. QuASH: Using Natural-Language Heuristics to Query Visual-Language Robotic Maps

    cs.RO 2025-10 conditional novelty 6.0 of 10

    Querying VLM robot maps with an SVM trained on LLM-generated synonym/antonym embeddings outperforms cosine-threshold and single-antonym baselines on images and OpenSeg maps, but not consistently on LSeg maps.

Reference graph

Works this paper leans on

47 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Talk2nav: Long- range vision-and-language navigation with dual attention and spatial memory,

    A. B. Vasudevan, D. Dai, and L. Van Gool, “Talk2nav: Long- range vision-and-language navigation with dual attention and spatial memory,”International Journal of Computer Vision, vol. 129, pp. 246– 266, 2021

    work page 2021

  2. [2]

    Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action,

    D. Shah, B. Osi ´nski, S. Levine et al. , “Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action,” in Conference on robot learning . PMLR, 2023, pp. 492–504

    2023

  3. [3]

    Visual language maps for robot navigation,

    C. Huang, O. Mees, A. Zeng, and W. Burgard, “Visual language maps for robot navigation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 10 608–10 615

    work page 2023

  4. [4]

    Object goal navigation using goal-oriented semantic exploration,

    D. Chaplot, D. Gandhi, A. Gupta, and R. Salakhutdinov, “Object goal navigation using goal-oriented semantic exploration,” Cornell University - arXiv,Cornell University - arXiv , Jul 2020

    work page 2020

  5. [5]

    L3mvn: Leveraging large language models for visual target navigation,

    B. Yu, H. Kasaei, and M. Cao, “L3mvn: Leveraging large language models for visual target navigation,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 3554–3560

    2023

  6. [6]

    Vlfm: Vision- language frontier maps for zero-shot semantic navigation,

    N. Yokoyama, S. Ha, D. Batra, J. Wang, and B. Bucher, “Vlfm: Vision- language frontier maps for zero-shot semantic navigation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 42–48

    2024

  7. [7]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

  8. [8]

    Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent,

    J. Yang, X. Chen, S. Qian, N. Madaan, M. Iyengar, D. F. Fouhey, and J. Chai, “Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 7694–7701

    work page 2024

Show all 47 references
  1. [9]

    Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation,

    S. Y . Gadre, M. Wortsman, G. Ilharco, L. Schmidt, and S. Song, “Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 23 171–23 181

    2023

  2. [10]

    When and why vision-language models behave like bags-of-words, and what to do about it?

    M. Yuksekgonul, F. Bianchi, P. Kalluri, D. Jurafsky, and J. Zou, “When and why vision-language models behave like bags-of-words, and what to do about it?” arXiv preprint arXiv:2210.01936 , 2022

    2022 arXiv

  3. [11]

    Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames,

    E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra, “Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames,” arXiv: Computer Vision and Pattern Recognition,arXiv: Computer Vision and Pattern Recognition , Nov 2019

    2019

  4. [12]

    Learning to explore using active neural slam,

    D. Chaplot, D. Gandhi, S. Gupta, A. Gupta, and R. Salakhutdi- nov, “Learning to explore using active neural slam,” International Conference on Learning Representations,International Conference on Learning Representations, Apr 2020

    2020

  5. [13]

    Ro- bustnav: Towards benchmarking robustness in embodied navigation,

    P. Chattopadhyay, J. Hoffman, R. Mottaghi, and A. Kembhavi, “Ro- bustnav: Towards benchmarking robustness in embodied navigation,” arXiv: Computer Vision and Pattern Recognition,arXiv: Computer Vision and Pattern Recognition , Jun 2021

    2021

  6. [14]

    Splitnet: Sim2sim and task2task transfer for embodied visual navigation,

    D. Gordon, A. Kadian, D. Parikh, J. Hoffman, and D. Batra, “Splitnet: Sim2sim and task2task transfer for embodied visual navigation,” Cornell University - arXiv,Cornell University - arXiv , May 2019

    2019

  7. [15]

    No rl, no simulation: Learning to navigate without nav- igating,

    M. Hahn, D. Chaplot, S. Tulsiani, M. Mukadam, J. Rehg, and A. Gupta, “No rl, no simulation: Learning to navigate without nav- igating,” Neural Information Processing Systems,Neural Information Processing Systems, Dec 2021

    2021

  8. [16]

    Minos: Multimodal indoor simulator for navigation in complex environments,

    M. Savva, A. Chang, A. Dosovitskiy, T. Funkhouser, and V . Koltun, “Minos: Multimodal indoor simulator for navigation in complex environments,” arXiv: Learning,arXiv: Learning , Dec 2017

    2017

  9. [17]

    Gibson env: Real-world perception for embodied agents,

    F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese, “Gibson env: Real-world perception for embodied agents,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , Jun 2018. [Online]. Available: http://dx.doi.org/10.1109/cvpr.2018.00945

    2018

  10. [18]

    Chalet: Cornell house agent learning environment

    C. Yan, D. Misra, A. Bennett, A. Walsman, Y . Bisk, and Y . Artzi, “Chalet: Cornell house agent learning environment.” arXiv: Artificial Intelligence,arXiv: Artificial Intelligence , Jan 2018

    2018

  11. [19]

    Memory- augmented reinforcement learning for image-goal navigation,

    L. Mezghan, S. Sukhbaatar, T. Lavril, O. Maksymets, D. Batra, P. Bojanowski, and K. Alahari, “Memory- augmented reinforcement learning for image-goal navigation,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , Oct 2022. [Online]. Available...

    2022

  12. [20]

    Poni: Potential functions for objectgoal navigation with interaction-free learning,

    S. K. Ramakrishnan, D. S. Chaplot, Z. Al-Halah, J. Malik, and K. Grauman, “Poni: Potential functions for objectgoal navigation with interaction-free learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 890–18 900

    2022

  13. [21]

    Target-driven visual navigation in indoor scenes using deep reinforcement learning,

    Y . Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-Fei, and A. Farhadi, “Target-driven visual navigation in indoor scenes using deep reinforcement learning,” in 2017 IEEE International Conference on Robotics and Automation (ICRA) , May 2017. [Online]. Available: http:...

    2017

  14. [22]

    Zero experience required: Plug & play modular transfer learning for semantic visual navigation,

    Z. Al-Halah, S. K. Ramakrishnan, and K. Grauman, “Zero experience required: Plug & play modular transfer learning for semantic visual navigation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 17 031–17 041

    2022

  15. [23]

    Semantic visual navigation by watching youtube videos,

    M. Chang, A. Gupta, and S. Gupta, “Semantic visual navigation by watching youtube videos,” arXiv: Computer Vision and Pattern Recognition,arXiv: Computer Vision and Pattern Recognition , Jun 2020

    2020

  16. [24]

    Sscnav: Confidence- aware semantic scene completion for visual semantic navigation,

    Y . Liang, B. Chen, and S. Song, “Sscnav: Confidence- aware semantic scene completion for visual semantic navigation,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) , May 2021. [Online]. Available: http://dx.doi.org/10.1109/icra48506.2021.9560925

    2021

  17. [25]

    Multion: Benchmarking semantic map memory using multi-object navigation,

    S. Wani, S. Patel, U. Jain, A. Chang, and M. Savva, “Multion: Benchmarking semantic map memory using multi-object navigation,” Neural Information Processing Systems,Neural Information Process- ing Systems, Jan 2020

    2020

  18. [26]

    Learning to learn how to learn: Self-adaptive visual navigation using meta-learning,

    M. Wortsman, K. Ehsani, M. Rastegari, A. Farhadi, and R. Mottaghi, “Learning to learn how to learn: Self-adaptive visual navigation using meta-learning,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , Jun 2019. [Online]. Available: http://dx.do...

    2019

  19. [27]

    Visual semantic navigation using scene priors,

    W. Yang, X. Wang, A. Farhadi, A. Gupta, and R. Mottaghi, “Visual semantic navigation using scene priors,” arXiv: Computer Vision and Pattern Recognition,arXiv: Computer Vision and Pattern Recognition , Oct 2018

    2018

  20. [28]

    Simple but effective: Clip embeddings for embodied ai,

    A. Khandelwal, L. Weihs, R. Mottaghi, and A. Kembhavi, “Simple but effective: Clip embeddings for embodied ai,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 14 829–14 838

    2022

  21. [29]

    Zson: Zero-shot object-goal navigation using multimodal goal embed- dings,

    A. Majumdar, G. Aggarwal, B. Devnani, J. Hoffman, and D. Batra, “Zson: Zero-shot object-goal navigation using multimodal goal embed- dings,” Advances in Neural Information Processing Systems , vol. 35, pp. 32 340–32 352, 2022

    2022

  22. [30]

    Esc: Exploration with soft commonsense constraints for zero-shot object navigation,

    K. Zhou, K. Zheng, C. Pryor, Y . Shen, H. Jin, L. Getoor, and X. E. Wang, “Esc: Exploration with soft commonsense constraints for zero-shot object navigation,” in International Conference on Machine Learning. PMLR, 2023, pp. 42 829–42 842

    2023

  23. [31]

    V oronav: V oronoi-based zero-shot object navigation with large language model,

    P. Wu, Y . Mu, B. Wu, Y . Hou, J. Ma, S. Zhang, and C. Liu, “V oronav: V oronoi-based zero-shot object navigation with large language model,” arXiv preprint arXiv:2401.02695 , 2024

    2024 arXiv

  24. [32]

    Navigation with large language models: Semantic guesswork as a heuristic for planning,

    D. Shah, M. R. Equi, B. Osi ´nski, F. Xia, B. Ichter, and S. Levine, “Navigation with large language models: Semantic guesswork as a heuristic for planning,” in Conference on Robot Learning . PMLR, 2023, pp. 2683–2699

    2023

  25. [33]

    Cogvlm: Visual expert for pretrained language models,

    W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y . Wang, J. Ji, Z. Yang, L. Zhao, X. Song et al. , “Cogvlm: Visual expert for pretrained language models,” arXiv preprint arXiv:2311.03079 , 2023

    2023 arXiv

  26. [34]

    Llava-grounding: Grounded visual chat with large multimodal models,

    H. Zhang, H. Li, F. Li, T. Ren, X. Zou, S. Liu, S. Huang, J. Gao, L. Zhang, C. Li et al., “Llava-grounding: Grounded visual chat with large multimodal models,” arXiv preprint arXiv:2312.02949 , 2023

    2023 arXiv

  27. [35]

    Semantic abstraction: Open-world 3d scene understanding from 2d vision-language models,

    H. Ha and S. Song, “Semantic abstraction: Open-world 3d scene understanding from 2d vision-language models,” arXiv preprint arXiv:2207.11514, 2022

    2022 arXiv

  28. [36]

    3d concept learning and reasoning from multi-view images,

    Y . Hong, C. Lin, Y . Du, Z. Chen, J. B. Tenenbaum, and C. Gan, “3d concept learning and reasoning from multi-view images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9202–9212

    2023

  29. [37]

    Objectnav revisited: On evaluation of embodied agents navigating to objects,

    D. Batra, A. Gokaslan, A. Kembhavi, O. Maksymets, R. Mottaghi, M. Savva, A. Toshev, and E. Wijmans, “Objectnav revisited: On evaluation of embodied agents navigating to objects,” arXiv preprint arXiv:2006.13171, 2020

    2006 arXiv

  30. [38]

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning,

    Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa et al. , “Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning,” in 2024 IEEE International Conference on Robotics and Automation (ICRA)....

    2024

  31. [39]

    DEAP: Evolutionary algorithms made easy,

    F.-A. Fortin, F.-M. De Rainville, M.-A. Gardner, M. Parizeau, and C. Gagn ´e, “DEAP: Evolutionary algorithms made easy,” Journal of Machine Learning Research , vol. 13, pp. 2171–2175, jul 2012

    2012

  32. [40]

    Vln-game: Vision-language equilibrium search for zero-shot semantic navigation,

    B. Yu, Y . Liu, L. Han, H. Kasaei, T. Li, and M. Cao, “Vln-game: Vision-language equilibrium search for zero-shot semantic navigation,” arXiv preprint arXiv:2411.11609 , 2024

    2024 arXiv

  33. [41]

    A fast marching level set method for monotonically advancing fronts

    J. A. Sethian, “A fast marching level set method for monotonically advancing fronts.” proceedings of the National Academy of Sciences , vol. 93, no. 4, pp. 1591–1595, 1996

    1996

  34. [42]

    Habitat 2.0: Training home assistants to rearrange their habitat,

    A. Szot, A. Clegg, E. Undersander, E. Wijmans, Y . Zhao, J. Turner, N. Maestre, M. Mukadam, D. Chaplot, O. Maksymets, A. Gokaslan, V . V ondrus, S. Dharur, F. Meier, W. Galuba, A. Chang, Z. Kira, V . Koltun, J. Malik, M. Savva, and D. Batra, “Habitat 2.0: Training home assista...

    2021

  35. [43]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu et al. , “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  36. [44]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Gir- shick, “Segment anything,” arXiv:2304.02643, 2023

    2023 arXiv

  37. [45]

    Goat-bench: A benchmark for multi-modal lifelong navigation,

    M. Khanna, R. Ramrakhya, G. Chhablani, S. Yenamandra, T. Gervet, M. Chang, Z. Kira, D. S. Chaplot, D. Batra, and R. Mottaghi, “Goat-bench: A benchmark for multi-modal lifelong navigation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CV...

    2024

  38. [46]

    Llm2clip: Powerful language model unlock richer visual representation,

    W. Huang, A. Wu, Y . Yang, X. Luo, Y . Yang, L. Hu, Q. Dai, X. Dai, D. Chen, C. Luo, and L. Qiu, “Llm2clip: Powerful language model unlock richer visual representation,” 2024. [Online]. Available: https://arxiv.org/abs/2411.04997

    2024

  39. [47]

    On evaluation of embodied navigation agents,

    P. Anderson, A. Chang, D. S. Chaplot, A. Dosovitskiy, S. Gupta, V . Koltun, J. Kosecka, J. Malik, R. Mottaghi, M. Savva et al. , “On evaluation of embodied navigation agents,” arXiv preprint arXiv:1807.06757, 2018

    2018 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.