Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

Proactive scene-graph imagination raises zero-shot object-goal navigation to 65.4% success on HM3D and 66.8% on HSSD, exceeding prior zero-shot and training-required methods.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

SGImagineNav uses an imagined hierarchical scene graph, filled in by an LLM, that guides a robot to unseen objects and achieves 65.4% and 66.8% success on HM3D and HSSD.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A useful, honest systems result: LLM imagination over hierarchical scene graphs helps zero-shot object navigation and adds multi-floor planning, but the headline gains need error bars and a prior-mismatch stress test. the 3 major comments →

arxiv 2508.06990 v1 pith:AGAE4VJS submitted 2025-08-09 cs.RO

Imaginative World Modeling with Scene Graphs for Embodied Agent Navigation

Yue Hu , Junzhe Wu , Ruihan Xu , Hang Liu , Avery Xi , Henry X. Liu , Ram Vasudevan , Maani Ghaffari This is my paper
classification cs.RO
keywords object-goal navigationzero-shot navigationscene graphsworld modelingvision-language modelssemantic explorationmulti-floor navigationembodied agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that proactive symbolic imagination—using a vision-language model to complete an agent's scene graph of unseen regions before acting—improves object-goal navigation in unseen indoor environments. It introduces SGImagineNav, which builds a hierarchical scene graph of objects, regions, and floors, then asks a vision-language model to label unknown areas using commonsense layout priors. This imagined graph guides frontier selection by balancing semantic exploitation with geometric exploration. The authors report success rates of 65.4% on HM3D and 66.8% on HSSD, above prior zero-shot and training-required methods, plus real-world cross-room and cross-floor demonstrations. If correct, cheap symbolic prediction can substitute for heavy image or map imagination in embodied search.

Core claim

The central claim is that proactive symbolic imagination—not better perception, planning, or more training—is the missing ingredient in zero-shot object-goal navigation. SGImagineNav builds an evolving hierarchical scene graph from RGB-D observations, then uses a vision-language model to label unobserved regions on a bird's-eye view before acting. The completed graph supplies semantic shortcuts: seeing stairs and a kitchen on the main floor suggests a bedroom upstairs, so the agent heads for the staircase. The authors report 65.4% success on HM3D and 66.8% on HSSD, surpassing zero-shot and training-required imaginative-navigation baselines; an oracle variant reaches 72.25%, and ablations att

What carries the argument

The central object is the completed hierarchical scene graph $\hat{g}_t = \psi(g_t, I_t)$: a tree over object nodes, region nodes, and floor nodes, where $g_t$ is the observed graph and $I_t$ is a bird's-eye projection of known nodes and unknown region centers. A vision-language model fills in the unknown regions by applying indoor-layout priors, turning the graph from a record of what has been seen into a hypothesis about what exists. The second load-bearing mechanism is the frontier-selection rule $l^*_t = \arg\max_k (S_t^{s,k} \text{ if } S_t^{s,k} > \lambda \text{ else } S_t^{g,k})$: the exploitation score $S_t^{s,k}$ measures semantic relevance of nearby scene-graph nodes to the target,

Load-bearing premise

The load-bearing premise is that indoor spaces are laid out according to the commonsense patterns a vision-language model knows—bedrooms near bathrooms, kitchens near dining rooms, bedrooms on upper floors—so imagined regions are trustworthy guides for choosing where to search; in environments that violate these priors, imagined labels could mislead the agent and reverse the reported gains.

What would settle it

Compare SGImagineNav against its own observation-only variant on episodes where the environment violates layout priors, such as single-floor layouts or mixed-use buildings. The paper's oracle variant (ground-truth scene graph) reaches 72.25% on HM3D; if VLM-predicted graphs do not beat observation-only exploration on such episodes, or if replacing VLM labels with random region labels yields the same success rate, the claim that proactive imagination drives the gain is falsified. A direct check is to compare predicted scene-graph labels against ground-truth labels for unexplored regions: the cl

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If SGImagineNav's results hold, zero-shot navigation can beat supervised methods on multi-floor benchmarks because the hierarchical graph captures cross-floor semantics that single-floor imitation or reinforcement-learning policies miss.
  • The 6.75-point ablation gain from scene-graph prediction supports a design principle: completing the world model with cheap symbolic guesses can contribute more than refining planning or perception on observed data.
  • The floor node is load-bearing: removing it hurts success more than removing object nodes, so multi-floor planning—stair detection plus a 2.5D traversable map—is the right unit of generality for indoor search.
  • Semantic relevance scoring by a language model beats distance-based frontier selection, so target search should prioritize semantically plausible regions even when they are not adjacent to the current position.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same scene-graph imagination could support open-vocabulary instruction following beyond object search, since the graph already labels rooms and objects; a 'bring the mug from the kitchen' task could reuse the same completed graph without retraining.
  • The fixed fallback threshold suggests a natural scalability test: letting the exploration/exploitation balance depend on scene size or graph uncertainty would likely reduce the redundant back-and-forth failures the error analysis attributes to greedy planning.
  • The reliance on a small set of region captions implies a testable ceiling: expanding the caption list beyond residential room types, or letting the vision-language model propose novel labels, would show whether the method generalizes to offices, labs, or warehouses where the priors are weaker.
  • The paper's own error analysis puts an upper bound near 88.84% on HM3D due to annotation and mesh noise, so part of the remaining gap to perfect navigation is benchmark imperfection rather than a reasoning failure; a cleaner synthetic benchmark with programmatic labels would isolate the imagination module's true headroom.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents SGImagineNav, a zero-shot object-goal navigation system that constructs a hierarchical 3D scene graph (objects, regions, floors) online and uses a VLM/LLM to hallucinate labels for unknown regions from semantic-layout priors. An adaptive frontier-selection module scores frontiers with a semantic exploitation gain and a raycast-based exploration gain, switching via Eq. (3), and a 2.5D FMM planner handles multi-floor traversal. Experiments report SR 65.4/66.8 on HM3D/HSSD, ablations on 400 HM3D episodes, scene-graph prediction precision/recall, and real-robot cross-room/cross-floor demonstrations.

Significance. If the results hold, the paper makes a useful contribution by showing that symbolic imagination can be zero-shot and global, in contrast to learned pixel/map imagination, and the hierarchical scene graph plus 3D planner extends prior single-floor symbolic methods. The strongest features are the modular system, the independent evaluation of scene-graph prediction against ground truth, the oracle ablation, and the real-robot deployment. However, the central claim rests on VLM priors and single-run point estimates, and the paper's own error analysis acknowledges that those priors are imperfect, so the robustness of the claimed improvement is not yet established.

major comments (3)
  1. [Sec. 5.1 / Table 1] The success distance threshold is reported as 0.1 m, whereas many prior ObjectNav results cited in Table 1 use the standard threshold of 1.0 m. Because the baseline numbers are taken from external papers, a different threshold would make the headline SR/SPL comparisons incommensurable. The authors must state the threshold used for every listed baseline or rerun/caveat the comparisons. This is load-bearing for the central claim of outperforming prior methods.
  2. [Sec. 5.3 / Table 2] The ablations are single runs on 400 episodes. With 400 Bernoulli trials, the standard error of a success rate is about 2.4 percentage points, so the imagination gain (62.50 vs 57.25, +5.25) is only about two standard errors and the goal-verification gain (67.00 vs 65.25, +1.75) is within one standard error. No seeds, confidence intervals, or significance tests are provided. This is not enough to establish the module-level contributions that the paper's claims rest on. Please report multi-seed variance or a paired significance test.
  3. [Sec. 4.1 / Eq. (3) / Sec. 7.7] The exploitation gain uses the same VLM commonsense priors both to label unknown regions and to score semantic relevance. If a scene violates those priors, a confidently wrong predicted label can produce S_s > lambda and actively mislead frontier selection, since the fallback to exploration only triggers when S_s <= lambda. Section 7.7 explicitly concedes that 'semantic priors do not perfectly align with real-world object arrangements,' yet no experiment varies or corrupts the prior. The authors should include a stress test (e.g., scrambled/anti-prior prompts, or scene subsets grouped by layout-prior agreement) to show that the reported gains are not an artifact of the HM3D/HSSD scenes incidentally matching the prompt's adjacency lists.
minor comments (4)
  1. [Sec. 5.2] The sentence 'pixel-level map prediction [21] ... and image-based imagination [19]' appears to swap the references: [21] is the image-based method (ImagineNav) and [19] is the map-based method (SGM). Please correct.
  2. [Sec. 7.4 / Sec. 7.6] Typos: '2 × 2 meteres local map window', 'the it relative height', and 'it's behavior' should read 'meters', 'its relative height', and 'its behavior'.
  3. [Sec. 4.3 / Eq. (3)] The fallback threshold lambda and the discount factor gamma in Eq. (4) are free parameters but their values are not reported, and no sensitivity analysis is given. Since the balance between exploitation and exploration is central to the method, please state the values and test robustness to at least one of these parameters.
  4. [Sec. 7.2] The scene-graph prediction prompt and output format contain Python-style boilerplate ('dict(...)', 'REMOVE spaces in the JSON string') that are implementation details; consider moving the exact prompt to source code and keeping a cleaned version in the appendix.

Circularity Check

0 steps flagged

No significant circularity: the framework is evaluated against external benchmarks and ground-truth scene graphs, with no fitted parameters or self-citation chain.

full rationale

The derivation chain does not reduce to its inputs. The claimed result (navigation success) is measured against external benchmarks (HM3D/HSSD) and real-world deployment; no parameter is fitted to these outcomes. The world-modeling and imagination module is evaluated separately against ground-truth scene graphs (Tab. 3a), which is an independent check of prediction quality. The exploitation gain in Eq. (3) uses semantic relevance scores over the observed and imagined scene graph; although the imagination prompt in Sec. 7.2 is target-conditioned ("can help find **bed**"), this does not make the prediction equivalent to the navigation outcome—it is a design choice that biases predictions toward target-relevant room types, which is precisely the commonsense prior the paper claims. The paper's own error analysis concedes that these priors do not perfectly align with real-world arrangements (Sec. 7.7), which is a robustness limitation, not circularity. There are no self-citations that are load-bearing: the cited prior work (SG-Nav, VLFM, etc.) is external. No uniqueness theorem or ansatz is imported from the authors' own prior work. The central ablation (c vs b in Tab. 2) shows a real empirical difference, and the oracle variant (f) provides an upper bound; these are not forced by construction. Thus the derivation is self-contained and the circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The central claim depends on semantic-prior assumptions and many hand-set engineering constants, but not on parameters fitted to the success metric. The scene graph is a standard representation and the LLM is a pretrained tool, so no new physical entities or forces are introduced.

free parameters (5)
  • fallback threshold lambda (Eq. 3)
    Hand-set threshold deciding when semantic exploitation gain overrides geometric exploration gain; no sensitivity analysis or ablation is reported.
  • exploration discount gamma (Eq. 4)
    Discount factor weighting unknown cells by distance along the path; value is not reported or swept.
  • stair gradient threshold = 0.3 m
    Gradient magnitude below which cells are classified as stair traversable; chosen heuristically from Habitat climb limits.
  • region grouping parameters (k, dmax, wmax, nmin) = 5, 2.5 m, 15, 3
    Neighbor count, distance, wall-crossing, and minimum-object thresholds for clustering objects into regions; stated without ablation.
  • raycasting parameters (20 rays, waypoint cap 12) = 20 rays, n < 12
    Computational approximations for exploration gain; chosen for efficiency, not validated.
axioms (4)
  • domain assumption Indoor floor plans follow semantic priors such as bedroom near bathroom, kitchen near dining room, bedrooms upstairs.
    Invoked throughout Sec. 4.1 and in the prompt template in Sec. 7.2; both imagination and exploitation scoring rely on these priors being mostly true in new scenes.
  • domain assumption The VLM/LLM prior used for imagination is applicable to unseen regions when given a BEV image and nearby scene graph context.
    Sec. 4.1 assumes psi(gt, It) yields labels closer to ground truth; Tab. 3a measures the average effect but does not characterize scenes where the prior mismatches reality.
  • domain assumption Clustering geometrically close objects not separated by walls yields meaningful room-level regions.
    Region grouping in Sec. 7.1 uses hand-set k, dmax, wmax, and nmin; if walls or open layouts violate this, region nodes mislead navigation.
  • domain assumption Stair regions can be made traversable by thresholding a gradient map and by the simulator's climb limit.
    Sec. 7.4 treats detected stairs as navigable via a 2.5D map; planner failures in Fig. 19 show this assumption is brittle in practice.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Imaginative World Modeling with Scene Graphs for Embodied Agent Navigation." pith.science (2026). https://pith.science/paper/AGAE4VJS

@misc{pith2026250806990,
  author       = {Pith},
  title        = {Pith review of: Imaginative World Modeling with Scene Graphs for Embodied Agent Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AGAE4VJS}},
  note         = {Machine review of arXiv:2508.06990}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Semantic navigation requires an agent to navigate toward a specified target in an unseen environment. Employing an imaginative navigation strategy that predicts future scenes before taking action, can empower the agent to find target faster. Inspired by this idea, we propose SGImagineNav, a novel imaginative navigation framework that leverages symbolic world modeling to proactively build a global environmental representation. SGImagineNav maintains an evolving hierarchical scene graphs and uses large language models to predict and explore unseen parts of the environment. While existing methods solely relying on past observations, this imaginative scene graph provides richer semantic context, enabling the agent to proactively estimate target locations. Building upon this, SGImagineNav adopts an adaptive navigation strategy that exploits semantic shortcuts when promising and explores unknown areas otherwise to gather additional context. This strategy continuously expands the known environment and accumulates valuable semantic contexts, ultimately guiding the agent toward the target. SGImagineNav is evaluated in both real-world scenarios and simulation benchmarks. SGImagineNav consistently outperforms previous methods, improving success rate to 65.4 and 66.8 on HM3D and HSSD, and demonstrating cross-floor and cross-room navigation in real-world environments, underscoring its effectiveness and generalizability.

Figures

Figures reproduced from arXiv: 2508.06990 by Avery Xi, Hang Liu, Henry X. Liu, Junzhe Wu, Maani Ghaffari, Ram Vasudevan, Ruihan Xu, Yue Hu.

Figure 1
Figure 1. Figure 1: By harnessing the imaginative capabilities of LLMs to anticipate the future in symbolic [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SGImagineNav. The SGImagineNav enhances frontier-based navigation strategy with two key designs: i) world imagination that looks ahead before acting, enabling proactive and informed exploration; ii) target information gain that balances exploitation and exploration gains to guide frontier selection, enabling target-focused search. imaginative scene graph, which extracts rich semantic context fr… view at source ↗
Figure 3
Figure 3. Figure 3: World modeling and imagination module builds a global environmental representation with [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Target information gain estimation module considers both exploration and exploitation [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: SGImagineNav successfully finds targets in real-world cross-floor and cross-room settings. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: During navigation, the agent explores the first floor and builds a hierarchical scene graph. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of region groups with four sample scenes. Objects are marked by red circles, [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of region captions. We see that CLIP can effectively select the correct region [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of scene graphs projected onto BEV image space, where objects are shown in [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The exploration gain is calculated as the accumulated visible unknown space that the [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visualization of internal maps used to construct a specialized traversable map that [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Our real-world setup on GO1 legged robot. (1) Intel D435 stereo camera; (2) Intel T265 [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Successful real-world deployments demonstrate the effectiveness and generalizability of [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Visualization of the error distribution. Currently, the five most common errors are from i) [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Visualization of episodes which has missing labels. In the above episodes, the agent [PITH_FULL_IMAGE:figures/full_fig_p017_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Visualization of imperfect meshes. Scenes might include artifacts, missing meshes, large [PITH_FULL_IMAGE:figures/full_fig_p017_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Visualization of false positive cases even with LLM verification. [PITH_FULL_IMAGE:figures/full_fig_p018_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Visualization of mapping error. Incorrect mapping produces false occupancy, leading to [PITH_FULL_IMAGE:figures/full_fig_p018_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Visualization of planner failure when climbing stairs. These cases often cause the agent to [PITH_FULL_IMAGE:figures/full_fig_p019_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Visualization of exploration error. Redundant navigation strategies result in timeouts. [PITH_FULL_IMAGE:figures/full_fig_p019_20.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. NavVerse: Benchmarking Indoor-to-Outdoor Embodied Navigation in Continuous Robot Simulation

    cs.RO 2026-07 conditional novelty 7.0

    A new physics-enabled benchmark with 10,000 indoor, outdoor, and indoor-to-outdoor navigation episodes shows current zero-shot agents fail most when crossing the indoor-outdoor boundary, especially on the new PlaceNav task.

  2. 3D Scene Graphs: Open Challenges and Future Directions

    cs.RO 2026-06 unverdicted novelty 2.0

    A survey that formalizes 3D Scene Graphs under a common definition, analyzes modeling choices, reviews construction from sensory data, examines applications and evaluations, and highlights open challenges with a suppo...

Reference graph

Works this paper leans on

46 extracted references · 27 canonical work pages · cited by 2 Pith papers · 1 internal anchor

  1. [1]

    Semantic visual navigation by watching youtube videos

    Matthew Chang, Arjun Gupta, and Saurabh Gupta. Semantic visual navigation by watching youtube videos. Advances in Neural Information Processing Systems , 33:4283–4294, 2020

    work page 2020

  2. [2]

    Object goal navigation using data regularized q-learning

    Nandiraju Gireesh, DA Sasi Kiran, Snehasis Banerjee, Mohan Sridharan, Brojeshwar Bhowmick, and Mad- hava Krishna. Object goal navigation using data regularized q-learning. In 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) , pages 1092–1097. IEEE, 2022

    work page 2022

  3. [3]

    Procthor: Large-scale embodied ai using procedural generation, 2022

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation, 2022

    2022

  4. [4]

    Habitat-web: Learning embodied object-search strategies from human demonstrations at scale

    Ram Ramrakhya, Eric Undersander, Dhruv Batra, and Abhishek Das. Habitat-web: Learning embodied object-search strategies from human demonstrations at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5173–5183, June 2022

    work page 2022

  5. [5]

    Pirlnav: Pretraining with imitation and rl finetuning for objectnav

    Ram Ramrakhya, Dhruv Batra, Erik Wijmans, and Abhishek Das. Pirlnav: Pretraining with imitation and rl finetuning for objectnav. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17896–17906, 2023

    2023

  6. [6]

    Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation

    Hang Yin, Xiuwei Xu, Zhenyu Wu, Jie Zhou, and Jiwen Lu. Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation. arXiv preprint arXiv:2410.08189, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Vlfm: Vision- language frontier maps for zero-shot semantic navigation

    Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher. Vlfm: Vision- language frontier maps for zero-shot semantic navigation. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 42–48. IEEE, 2024

    work page 2024

  8. [8]

    Instructnav: Zero-shot system for generic instruction navigation in unexplored environment

    Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. arXiv preprint arXiv:2406.04882, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Esc: Exploration with soft commonsense constraints for zero-shot object navigation

    Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, and Xin Eric Wang. Esc: Exploration with soft commonsense constraints for zero-shot object navigation. In International Conference on Machine Learning, pages 42829–42842. PMLR, 2023

    work page 2023

  10. [10]

    Object goal navigation using goal-oriented semantic exploration

    Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhutdinov. Object goal navigation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems, 33:4247–4258, 2020

    work page 2020

  11. [11]

    Poni: Potential functions for objectgoal navigation with interaction-free learning

    Santhosh Kumar Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, Jitendra Malik, and Kristen Grauman. Poni: Potential functions for objectgoal navigation with interaction-free learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18890–18900, 2022

    work page 2022

  12. [12]

    Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation

    Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song. Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23171–23181, 2023

    work page 2023

  13. [13]

    Dubey et al

    A. Dubey et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023

    2023

  15. [15]

    Uncertainty-driven planner for exploration and navigation

    Georgios Georgakis, Bernadette Bucher, Anton Arapin, Karl Schmeckpeper, Nikolai Matni, and Kostas Daniilidis. Uncertainty-driven planner for exploration and navigation. In 2022 International Conference on Robotics and Automation (ICRA) , pages 11295–11302. IEEE, 2022

    work page 2022

  16. [16]

    Gamap: Zero-shot object goal navigation with multi-scale geometric-affordance guidance

    Hao Huang, Yu Hao, Congcong Wen, Anthony Tzes, Yi Fang, et al. Gamap: Zero-shot object goal navigation with multi-scale geometric-affordance guidance. Advances in Neural Information Processing Systems, 37:39386–39408, 2024

    work page 2024

  17. [17]

    Open scene graphs for open world object-goal navigation

    Joel Loo, Zhanxin Wu, and David Hsu. Open scene graphs for open world object-goal navigation. arXiv preprint arXiv:2407.02473, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Clio: Real-time task-driven open-set 3d scene graphs

    Dominic Maggio, Yun Chang, Nathan Hughes, Matthew Trang, Dan Griffith, Carlyn Dougherty, Eric Cristofalo, Lukas Schmid, and Luca Carlone. Clio: Real-time task-driven open-set 3d scene graphs. IEEE Robotics and Automation Letters, 2024

    work page 2024

  19. [19]

    Imagine before go: Self- supervised generative map for object goal navigation

    Sixian Zhang, Xinyao Yu, Xinhang Song, Xiaohan Wang, and Shuqiang Jiang. Imagine before go: Self- supervised generative map for object goal navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16414–16425, 2024. 21

    work page 2024

  20. [20]

    ForesightNav: Learning Scene Imagination for Efficient Exploration

    Hardik Shah, Jiaxu Xing, Nico Messikommer, Boyang Sun, Marc Pollefeys, and Davide Scaramuzza. Foresightnav: Learning scene imagination for efficient exploration. arXiv preprint arXiv:2504.16062 , 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  21. [21]

    Imaginenav: Prompting vision-language models as embodied navigator through scene imagination

    Xinxin Zhao, Wenzhe Cai, Likun Tang, and Teng Wang. Imaginenav: Prompting vision-language models as embodied navigator through scene imagination. arXiv preprint arXiv:2410.09874, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Mapex: Indoor structure exploration with probabilistic information gain from global map predictions

    Cherie Ho, Seungchan Kim, Brady Moon, Aditya Parandekar, Narek Harutyunyan, Chen Wang, Katia Sycara, Graeme Best, and Sebastian Scherer. Mapex: Indoor structure exploration with probabilistic information gain from global map predictions. arXiv preprint arXiv:2409.15590, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv preprint arXiv:2109.08238, 2021

    Pith/arXiv arXiv 2021

  24. [24]

    Chang, and Manolis Savva

    Mukul Khanna*, Yongsen Mao*, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexan- der Clegg, Eric Undersander, Angel X. Chang, and Manolis Savva. Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation. arXiv preprint, 2023

    work page 2023

  25. [25]

    Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames

    Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. arXiv preprint arXiv:1911.00357, 2019

    Pith/arXiv arXiv 1911

  26. [26]

    Visual representations for semantic target driven navigation

    Arsalan Mousavian, Alexander Toshev, Marek Fišer, Jana Košecká, Ayzaan Wahid, and James Davidson. Visual representations for semantic target driven navigation. In 2019 International Conference on Robotics and Automation (ICRA), pages 8846–8852. IEEE, 2019

    work page 2019

  27. [27]

    Visual semantic navigation using scene priors

    Wei Yang, Xiaolong Wang, Ali Farhadi, Abhinav Gupta, and Roozbeh Mottaghi. Visual semantic navigation using scene priors. arXiv preprint arXiv:1810.06543, 2018

    Pith/arXiv arXiv 2018

  28. [28]

    Zson: Zero-shot object-goal navigation using multimodal goal embeddings

    Arjun Majumdar, Gunjan Aggarwal, Bhavika Devnani, Judy Hoffman, and Dhruv Batra. Zson: Zero-shot object-goal navigation using multimodal goal embeddings. Advances in Neural Information Processing Systems, 35:32340–32352, 2022

    work page 2022

  29. [29]

    Thda: Treasure hunt data augmentation for semantic navigation

    Oleksandr Maksymets, Vincent Cartillier, Aaron Gokaslan, Erik Wijmans, Wojciech Galuba, Stefan Lee, and Dhruv Batra. Thda: Treasure hunt data augmentation for semantic navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15374–15383, 2021

    work page 2021

  30. [30]

    Simple but effective: Clip embeddings for embodied ai

    Apoorv Khandelwal, Luca Weihs, Roozbeh Mottaghi, and Aniruddha Kembhavi. Simple but effective: Clip embeddings for embodied ai. In CVPR, pages 14829–14838, 2022

    work page 2022

  31. [31]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  32. [32]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Claude ai

    Anthropic. Claude ai. https://claude.ai, 2023. Accessed: 2025-05-05

    work page 2023

  34. [34]

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y . K. Li, Wenfeng Liang, Fangyun Lin, A. X. Liu, Bo Liu,...

    Pith/arXiv arXiv 2024

  35. [35]

    L3mvn: Leveraging large language models for visual target navigation

    Bangguo Yu, Hamidreza Kasaei, and Ming Cao. L3mvn: Leveraging large language models for visual target navigation. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 3554–3560. IEEE, 2023. 22

    work page 2023

  36. [36]

    Lay, Andreas Dömel, Martin J

    Ryo Sakagami, Florian S. Lay, Andreas Dömel, Martin J. Schuster, Alin Albu-Schäffer, and Freek Stulp. Robotic world models—conceptualization, review, and engineering best practices. Frontiers in Robotics and AI, V olume 10 - 2023, 2023

    work page 2023

  37. [37]

    Recurrent world models facilitate policy evolution

    David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018

    work page 2018

  38. [38]

    World models and predictive coding for cognitive and developmental robotics: frontiers and challenges.Advanced Robotics, 37(13):780–806, 2023

    Tadahiro Taniguchi, Shingo Murata, Masahiro Suzuki, Dimitri Ognibene, Pablo Lanillos, Emre Ugur, Lorenzo Jamone, Tomoaki Nakamura, Alejandra Ciria, Bruno Lara, and Giovanni Pezzulo and. World models and predictive coding for cognitive and developmental robotics: frontiers and challenges.Advanced Robotics, 37(13):780–806, 2023

    work page 2023

  39. [39]

    A frontier-based approach for autonomous exploration

    Brian Yamauchi. A frontier-based approach for autonomous exploration. In Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97. ’Towards New Computational Principles for Robotics and Automation’ , pages 146–151. IEEE, 1997

    work page 1997

  40. [40]

    Fast marching methods

    James A Sethian. Fast marching methods. SIAM review, 41(2):199–235, 1999

    1999

  41. [41]

    Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X

    Santhosh K. Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X. Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai, 2021

    work page 2021

  42. [42]

    V oronav: V oronoi-based zero-shot object navigation with large language model.arXiv preprint arXiv:2401.02695, 2024

    Pengying Wu, Yao Mu, Bingxian Wu, Yi Hou, Ji Ma, Shanghang Zhang, and Chang Liu. V oronav: V oronoi-based zero-shot object navigation with large language model.arXiv preprint arXiv:2401.02695, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Openfmnav: Towards open-set zero-shot object navigation via vision-language foundation models

    Yuxuan Kuang, Hai Lin, and Meng Jiang. Openfmnav: Towards open-set zero-shot object navigation via vision-language foundation models. arXiv preprint arXiv:2402.10670, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Multi-floor zero-shot object navigation policy

    Lingfeng Zhang, Hao Wang, Erjia Xiao, Xinyao Zhang, Qiang Zhang, Zixuan Jiang, and Renjing Xu. Multi-floor zero-shot object navigation policy. arXiv preprint arXiv:2409.10906, 2024

    Pith/arXiv arXiv 2024

  45. [45]

    Integrat- ing egocentric localization for more realistic point-goal navigation agents

    Samyak Datta, Oleksandr Maksymets, Judy Hoffman, Stefan Lee, Dhruv Batra, and Devi Parikh. Integrat- ing egocentric localization for more realistic point-goal navigation agents. CoRL, 2020

    work page 2020

  46. [46]

    Grounded sam: Assembling open-world models for diverse visual tasks, 2024

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024. 23

    work page 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.