Pith. sign in

REVIEW 3 major objections 3 minor 1 cited by

Open Scene Graphs for Open-World Object-Goal Navigation

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that a modular, foundation-model-based navigator can search novel indoor environments for arbitrary natural-language object goals in zero-shot, using a hierarchically organized Open Scene Graph as its spatial memory.

desk verdict Plausible schema-based memory for zero-shot ObjectNav, but the corrupted full text means the headline claims are unverifiable from this copy. read the letter →

arxiv 2508.04678 v1 pith:ZV4VYYMR submitted 2025-08-06 cs.RO

Joel Loo , Zhanxin Wu , David Hsu This is my paper
classification cs.RO
keywords OpenSceneGraphObject-GoalNavigationzero-shotgeneralizationfoundationmodelsspatialmemoryschemasmobilerobotssemantic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a robot can enter a building it has never seen, be told a target object in natural language, and find that object without any per-environment training, by organising its spatial memory as an Open Scene Graph. The memory's top level is an OSG schema, a template for a class of environments that is generated automatically from a single semantic label such as 'home' or 'supermarket'; lower levels are the robot's live observations. The paper reports that this modular system, built entirely from foundation models, achieves top benchmark results on Object-Goal Navigation and transfers zero-shot to new goals, environments, and robot bodies. A sympathetic reader would take the central claim to be that class-level scene templates are a sufficient spatial-memory mechanism for open-world semantic search, replacing environment-specific maps or training.

What carries the argument

Open Scene Graph (OSG) representation, used as spatial memory. It is a hierarchy: each OSG schema is a class-level template—generated automatically from a simple semantic label such as 'home' or 'supermarket'—that describes the common spatial structure of a class of environments (typical rooms, objects, and relations); below it sit the robot's observations of the specific instance. This structure carries the argument by giving the navigator a prior over where to look for a target, while remaining grounded in what the robot actually sees.

What would settle it

Run the navigator in unseen buildings whose layouts deliberately violate the label's typical template—a place called 'home' with no bedrooms, or a 'supermarket' without aisles—and compare success rate and time against a flat, map-only navigator with no schema prior. If the schema-guided agent is not faster or more successful in environments where the template is wrong, the zero-shot benefit is not real.

Watch

Extended reading notes

Core claim

The central discovery, on the paper's own terms, is that hierarchical scene schemas can serve as spatial memory for open-world Object-Goal Navigation. OSG Navigator combines foundation models for perception and planning and uses the Open Scene Graph to organise what it knows. The schema layer encodes a prior over where rooms and objects tend to be in a class of environments; the robot instantiates and revises that prior as it senses the actual environment. In experiments with Fetch and Spot robots, in simulation and the real world, the authors report state-of-the-art performance on Object-Goal Navigation benchmarks and zero-shot generalisation over diverse goals, environments, and robot embo

Load-bearing premise

The load-bearing premise is that a template generated from one label like 'home' correctly predicts the layout of the particular unseen building the robot is dropped into; if the template misfires, the hierarchical memory can actively steer exploration the wrong way and fall below a simpler map-only baseline.

Editorial extensions

If this is right

  • A single semantic label ('home', 'supermarket') is enough to instantiate a usable spatial prior for a never-seen building, removing the need for per-environment training or map building.
  • Because the modules interact through generic observations and actions, the same navigation stack transfers across robot embodiments, e.g., Fetch and Spot.
  • Targets can be specified in natural language from an open vocabulary rather than a fixed object list, so the system is not limited to pre-enumerated goals.
  • Adding a new environment type only requires a new label and schema, making the system's coverage of environment classes expand without retraining.
  • On the reported benchmarks, the combination of foundation-model semantics with hierarchical scene memory outperforms prior ObjectNav methods, suggesting the two capabilities are complementary rather than competing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cleanest way to stress-test the paper's claim is to adversarially mismatch labels and layouts: the method's zero-shot edge should disappear or reverse when the label's template does not fit the actual building, revealing whether the prior helps or merely biases exploration.
  • Because the schemas come from foundation-model knowledge, the approach inherits those models' blind spots: environment types that are common outside the training distribution may yield poor templates, so open-world generalisation is bounded by the breadth of pretraining.
  • A natural extension the paper leaves implicit is composing schemas from partial observations: after the robot sees a few rooms, it could refine or replace the label-derived template online, which would soften the single-label assumption without giving up the hierarchical prior.
  • If the hierarchical representation is what matters, one testable prediction is that adding more levels of abstraction (e.g., neighbourhood to building to room to object) should further improve sample efficiency in environments where the extra levels align with real structure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper introduces OSG Navigator, a modular ObjectNav system that combines foundation models with a new Open Scene Graph (OSG) representation used as hierarchical spatial memory. OSG schemas, which are templates describing common structures of environment classes, are auto-generated from semantic labels such as 'home' or 'supermarket' and are intended to enable zero-shot adaptation to novel environment types. The abstract reports state-of-the-art performance on ObjectNav benchmarks and zero-shot generalization across goals, environments, and robot embodiments, with experiments using Fetch and Spot robots in simulation and the real world. However, the supplied full text is almost entirely corrupted (mojibake), so only the abstract and a few table fragments can be read; the architecture details, schema-generation procedure, experimental setup, ablations, and statistics behind the empirical claims are not verifiable in the submitted material.

Significance. If the claims hold, the paper would make a useful contribution: a modular, foundation-model-only navigation system with an automatically generated hierarchical scene-graph memory, evaluated on two robot platforms and in the real world, without per-environment training. The idea of using OSG schemas as zero-shot spatial priors is timely and plausible. I also see no visible circularity in the abstract: the schemas encode common-sense knowledge rather than benchmark answers, and the system is claimed to be empirically evaluated. The main unresolved correctness risk is the schema-prior premise: if an auto-generated OSG schema mismatches a particular unseen environment, the hierarchical memory could actively misdirect exploration. The readable portion of the submitted text does not address this failure mode. Because the full text is unreadable, the significance is necessarily conditional.

major comments (3)
  1. [Full text (all post-abstract content)] The submitted manuscript is corrupted: after the abstract, almost all text is unreadable mojibake. This includes the architecture description, the schema-generation method, the experimental setup, the comparative tables, ablations, and real-robot statistics. As a result, the claimed 'state-of-the-art performance' and 'zero-shot generalization' cannot be checked from the submitted material. This is a review-blocking issue independent of the scientific merit.
  2. [Abstract / OSG schema premise] The zero-shot claim rests on the premise that an OSG schema generated from a single label such as 'home' or 'supermarket' predicts the spatial structure of the particular unseen test environment closely enough to improve exploration. If the schema misfires, the hierarchical memory can bias the robot toward nonexistent locations and away from the target. The abstract and the legible text provide no mismatched-schema analysis, no ablation with incorrect or coarse labels, and no fallback mechanism. A concrete test would be to run the system with deliberately wrong or noisy environment labels and compare against a map-only baseline to show graceful degradation.
  3. [Final section (near final tables)] A readable fragment near the end appears to state that system performance depends on the environment type T, and two small tables of numeric values follow. However, the captions, metric definitions, and error bars are illegible in the supplied file. Without these details, the table fragments cannot support the state-of-the-art claim, and the apparent type-dependence limitation only reinforces the need for a mismatched-schema analysis.
minor comments (3)
  1. [Front matter / header] The line 'arXiv:2508.04682v2 [cs.CV] 7 Aug 2025' appears inside the text; it does not match the paper's arXiv ID and should be removed or corrected.
  2. [Abstract / notation] The acronym OSG is introduced, but the relationship between 'Open Scene Graph' and 'OSG schemas' could be stated more explicitly; for example, clarify that a schema is a template instantiated over the OSG hierarchy.
  3. [Tables (corrupted)] If the final tables are intended to report results, they need legible captions, explicit metric names (e.g., Success, SPL, episode length), and error bars or trial counts; in the current file they appear as unlabeled numeric rows.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: OSG schemas are priors derived from semantic labels, not fitted to benchmark outcomes; the zero-shot claim is empirically evaluated against external benchmarks.

full rationale

The available text is heavily corrupted, so the assessment is based on the abstract and legible fragments. The derivation chain is: foundation models provide semantic knowledge; OSG schemas are templates generated from simple semantic labels (e.g., 'home', 'supermarket'); these schemas organize spatial memory for navigation; the system is then evaluated on ObjectNav benchmarks with Fetch and Spot robots in simulation and the real world. Nothing in this chain reduces by construction to the benchmark result. The schema is a common-sense prior, not a parameter fitted to the target environments or to the evaluation metrics. The abstract's zero-shot claim is an empirical outcome contingent on schema-environment match, but a mismatch risk is a validity concern, not definitional circularity. No load-bearing self-citation, no renamed known result, and no equation equating prediction with input were identifiable in the available text. Therefore the paper shows no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 1 invented entities

Audit performed on abstract text only because the supplied full text was corrupted and unreadable. No free parameters could be identified or excluded; three domain assumptions are load-bearing; one invented construct, the OSG schema, carries the system's novelty and has no evidence base outside the paper's own experiments.

assumptions (3)
  • domain assumption Foundation-model components (VLM/LLM) provide reliable open-vocabulary perception and semantic reasoning in novel environments.
    Abstract states OSG Navigator is 'composed of foundation models' and relies on their 'enormous semantic knowledge'; no evidence in the abstract that perception errors or hallucinated semantics were quantified.
  • ad hoc to paper OSG schemas auto-generated from a semantic label (e.g., 'home', 'supermarket') correctly predict the spatial structure of the specific unseen environment.
    The zero-shot generality claim depends on label-to-schema templates transferring to novel layouts; if the template is inaccurate, hierarchical spatial memory misleads the exploration.
  • domain assumption The hierarchical graph maintains spatial information effectively at scale in real time on robot hardware.
    Abstract asserts foundation models 'struggle to organise and maintain spatial information effectively at scale' and that OSG fixes this; no runtime or memory evidence is given in the abstract.
invented entities (1)
  • Open Scene Graph (OSG) schema
    purpose: Template describing the common structure of an environment class, auto-generated from a semantic label, used to organize the spatial memory hierarchy and guide goal-directed exploration.
    The schema is the paper's core construct; its only validation is the paper's own benchmark and robot experiments. No external or cross-lab validation, and no released artifact, is described in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Open Scene Graphs for Open-World Object-Goal Navigation." pith.science (2026). https://pith.science/paper/ZV4VYYMR

@misc{pith2026250804678,
  author       = {Pith},
  title        = {Pith review of: Open Scene Graphs for Open-World Object-Goal Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZV4VYYMR}},
  note         = {Machine review of arXiv:2508.04678}
}
read the original abstract

How can we build general-purpose robot systems for open-world semantic navigation, e.g., searching a novel environment for a target object specified in natural language? To tackle this challenge, we introduce OSG Navigator, a modular system composed of foundation models, for open-world Object-Goal Navigation (ObjectNav). Foundation models provide enormous semantic knowledge about the world, but struggle to organise and maintain spatial information effectively at scale. Key to OSG Navigator is the Open Scene Graph representation, which acts as spatial memory for OSG Navigator. It organises spatial information hierarchically using OSG schemas, which are templates, each describing the common structure of a class of environments. OSG schemas can be automatically generated from simple semantic labels of a given environment, e.g., "home" or "supermarket". They enable OSG Navigator to adapt zero-shot to new environment types. We conducted experiments using both Fetch and Spot robots in simulation and in the real world, showing that OSG Navigator achieves state-of-the-art performance on ObjectNav benchmarks and generalises zero-shot over diverse goals, environments, and robot embodiments.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VTM-Nav: Harnessing Cross-Episode Experience for Object-Goal Navigation with Hierarchical Visual-Topological Memory

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A hierarchical room-and-object memory that persists across independent ObjectNav episodes yields small success-rate gains, but most of the gain comes from within-episode memory rather than the cross-episode component.

Reference graph

Works this paper leans on

92 extracted references · 72 canonical work pages · cited by 1 Pith paper

  1. [1]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Al-Halah Z, Ramakrishnan SK and Grauman K (2022) Zero experience required: Plug & play modular transfer learning for semantic visual navigation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 17031--17041

    2022

  2. [2]

    IEEE Robotics and Automation Letters

    Amiri S, Chandan K and Zhang S (2022) Reasoning with scene graphs for robot planning under partial observability. IEEE Robotics and Automation Letters

    2022

  3. [3]

    (2018) On evaluation of embodied navigation agents

    Anderson P, Chang A, Chaplot DS, Dosovitskiy A, Gupta S et al. (2018) On evaluation of embodied navigation agents

    2018

  4. [4]

    Anwar A, Welsh J, Biswas J, Pouya S and Chang Y (2024) Remembr: Building and reasoning over long-horizon spatio-temporal memory for robot navigation

    2024

  5. [5]

    (2019) 3d scene graph: A structure for unified semantics, 3d space, and camera

    Armeni I, He ZY, Gwak J, Zamir A, Fischer M et al. (2019) 3d scene graph: A structure for unified semantics, 3d space, and camera. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) : 5663--5672

    2019

  6. [6]

    (2020) Language models are few-shot learners

    Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD et al. (2020) Language models are few-shot learners. In: Larochelle H, Ranzato M, Hadsell R, Balcan M and Lin H (eds.) Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., pp. 1877--1901

    2020

  7. [7]

    (2023) Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill

    Cai W, Huang S, Cheng G, Long Y, Gao P et al. (2023) Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill

    2023

  8. [8]

    (2023) Goat: Go to any thing

    Chang M, Gervet T, Khanna M, Yenamandra S, Shah D et al. (2023) Goat: Go to any thing

    2023

Show all 92 references
  1. [9]

    In: Larochelle H, Ranzato M, Hadsell R, Balcan M and Lin H (eds.) Advances in Neural Information Processing Systems, volume 33

    Chaplot DS, Gandhi DP, Gupta A and Salakhutdinov RR (2020) Object goal navigation using goal-oriented semantic exploration. In: Larochelle H, Ranzato M, Hadsell R, Balcan M and Lin H (eds.) Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., ...

    2020

  2. [10]

    In: Proceedings of Robotics: Science and Systems

    Chen J, Li G, Kumar S, Ghanem B and Yu F (2023) How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers . In: Proceedings of Robotics: Science and Systems. Daegu, Republic of Korea

    2023

  3. [11]

    Chen J, Lin B, Xu R, Chai Z, Liang X and Wong KYK (2024) Mapgpt: Map-guided prompting for unified vision-and-language navigation

    2024

  4. [12]

    (2024) Optimal scene graph planning with large language model guidance

    Dai Z, Asgharivaskasi A, Duong T, Lin S, Tzes ME et al. (2024) Optimal scene graph planning with large language model guidance. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 14062--14069. doi:10.1109/ICRA57147.2024.10610599

    2024

  5. [13]

    (2024) Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    Deitke M, Clark C, Lee S, Tripathi R, Yang Y et al. (2024) Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    2024

  6. [14]

    cat-shaped mug

    Dorbala VS, Mullen JF and Manocha D (2024) Can an embodied agent find your “cat-shaped mug”? llm-based zero-shot object navigation. IEEE Robotics and Automation Letters 9(5): 4083--4090

    2024

  7. [15]

    (2024) The llama 3 herd of models

    Dubey A, Jauhri A, Pandey A, Kadian A, Al-Dahle A et al. (2024) The llama 3 herd of models

    2024

  8. [16]

    Gadre SY, Wortsman M, Ilharco G, Schmidt L and Song S (2023) Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation. CVPR

    2023

  9. [17]

    (2019) Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery

    Grinvald M, Furrer F, Novkovic T, Chung JJ, Cadena C et al. (2019) Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery . IEEE Robotics and Automation Letters

    2019

  10. [18]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Gu J, Stefani E, Wu Q, Thomason J and Wang X (2022) Vision-and-language navigation: A survey of tasks, methods, and future directions. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 7606--7623

    2022

  11. [19]

    (2023) Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning

    Gu Q, Kuwajerwala A, Morin S, Jatavallabhula K, Sen B et al. (2023) Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. arXiv

    2023

  12. [20]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Gupta A, Dollar P and Girshick R (2019) LVIS : A dataset for large vocabulary instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . pp. 5351--5359

    2019

  13. [21]

    arXiv preprint arXiv:2403.08605

    Honerkamp D, Büchner M, Despinoy F, Welschehold T and Valada A (2024) Language-grounded dynamic scene graphs for interactive object search with mobile manipulation. arXiv preprint arXiv:2403.08605

    2024 arXiv

  14. [22]

    (2023) Toward general-purpose robots via foundation models: A survey and meta-analysis

    Hu Y, Xie Q, Jain V, Francis J, Patrikar J et al. (2023) Toward general-purpose robots via foundation models: A survey and meta-analysis

    2023

  15. [23]

    Huang C and Zhang L (2025) On the limit of language models as planning formalizers

    2025

  16. [24]

    arXiv preprint arXiv:2212.10403

    Huang J and Chang KCC (2022) Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403

    2022 arXiv

  17. [25]

    Huang W, Abbeel P, Pathak D and Mordatch I (2022) Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In: Chaudhuri K, Jegelka S, Song L, Szepesvari C, Niu G and Sabato S (eds.) Proceedings of the 39th International Conference on Machine...

    2022

  18. [26]

    In: Proceedings of Robotics: Science and Systems

    Hughes N, Chang Y and Carlone L (2022) Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization . In: Proceedings of Robotics: Science and Systems. New York City, NY, USA

    2022

  19. [27]

    Robotics and Automation, IEEE Transactions on 12: 651 -- 670

    Hutchinson S, Hager G and Corke P (1996) A tutorial on visual servo control. Robotics and Automation, IEEE Transactions on 12: 651 -- 670

    1996

  20. [28]

    (2023) Do as i can, not as i say: Grounding language in robotic affordances

    Ichter B, Brohan A, Chebotar Y, Finn C, Hausman K et al. (2023) Do as i can, not as i say: Grounding language in robotic affordances. In: Liu K, Kulic D and Ichnowski J (eds.) Proceedings of The 6th Conference on Robot Learning, Proceedings of Machine Learning Research, volume...

    2023

  21. [29]

    (2023) Conceptfusion: Open-set multimodal 3d mapping

    Jatavallabhula K, Kuwajerwala A, Gu Q, Omama M, Chen T et al. (2023) Conceptfusion: Open-set multimodal 3d mapping. Robotics: Science and Systems (RSS)

    2023

  22. [30]

    In: 2024 IEEE International Conference on Robotics and Automation (ICRA)

    Kassab C, Mattamala M, Zhang L and Fallon M (2024) Language-extended indoor slam (lexis): A versatile system for real-time visual scene understanding. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 15988--15994

    2024

  23. [31]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Khandelwal A, Weihs L, Mottaghi R and Kembhavi A (2022) Simple but effective: Clip embeddings for embodied ai. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 14829--14838

    2022

  24. [32]

    (2023) Segment anything

    Kirillov A, Mintun E, Ravi N, Mao H, Rolland C et al. (2023) Segment anything. arXiv:2304.02643

    2023 arXiv

  25. [33]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Krantz J, Banerjee S, Zhu W, Corso J, Anderson P, Lee S and Thomason J (2023) Iterative vision-and-language navigation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 14921--14930

    2023

  26. [34]

    Krantz J, Lee S, Malik J, Batra D and Chaplot DS (2022) Instance-specific image goal navigation: Training embodied agents to find object instances

    2022

  27. [35]

    u ttler H, Lewis M, Yih Wt, Rockt\

    Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, K\" u ttler H, Lewis M, Yih Wt, Rockt\" a schel T, Riedel S and Kiela D (2020) Retrieval-augmented generation for knowledge-intensive nlp tasks. In: Larochelle H, Ranzato M, Hadsell R, Balcan M and Lin H (eds.) Advan...

    2020

  28. [36]

    Li J, Li D, Savarese S and Hoi S (2023) BLIP -2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In: Krause A, Brunskill E, Cho K, Engelhardt B, Sabato S and Scarlett J (eds.) Proceedings of the 40th International Conference on M...

    2023

  29. [37]

    Liu H, Li C, Wu Q and Lee YJ (2023 a ) Visual instruction tuning

    2023

  30. [38]

    Dublin, Ireland and Online: Association for Computational Linguistics, pp

    Liu J, Shen D, Zhang Y, Dolan B, Carin L and Chen W (2022) What makes good in-context examples for GPT -3? In: Agirre E, Apidianaki M and Vuli \'c I (eds.) Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep ...

    2022

  31. [39]

    (2023 b ) Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Liu S, Zeng Z, Ren T, Li F, Zhang H et al. (2023 b ) Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499

    2023 arXiv

  32. [40]

    In: Agrawal P, Kroemer O and Burgard W (eds.) Proceedings of The 8th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 270

    Long Y, Cai W, Wang H, Zhan G and Dong H (2025) Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. In: Agrawal P, Kroemer O and Burgard W (eds.) Proceedings of The 8th Conference on Robot Learning, Proceedings of Machine Learning Resear...

    2025

  33. [41]

    (2024) Clio: Real-time task-driven open-set 3d scene graphs

    Maggio D, Chang Y, Hughes N, Trang M, Griffith D et al. (2024) Clio: Real-time task-driven open-set 3d scene graphs. IEEE Robotics and Automation Letters 9(10): 8921--8928. doi:10.1109/LRA.2024.3451395

    2024

  34. [42]

    In: Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K and Oh A (eds.) Advances in Neural Information Processing Systems, volume 35

    Majumdar A, Aggarwal G, Devnani B, Hoffman J and Batra D (2022) Zson: Zero-shot object-goal navigation using multimodal goal embeddings. In: Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K and Oh A (eds.) Advances in Neural Information Processing Systems, volume 35. Curran A...

    2022

  35. [43]

    In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 229

    Majumdar A, Xia F, Ichter B, Batra D and Guibas L (2023) Findthis: Language-driven object disambiguation in indoor environments. In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 22...

    2023

  36. [44]

    (2021) Thda: Treasure hunt data augmentation for semantic navigation

    Maksymets O, Cartillier V, Gokaslan A, Wijmans E, Galuba W et al. (2021) Thda: Treasure hunt data augmentation for semantic navigation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 15374--15383

    2021

  37. [45]

    Annual Review of Control, Robotics, and Autonomous Systems

    Mascaro R and Chli M (2024) Scene representations for robotic spatial perception. Annual Review of Control, Robotics, and Autonomous Systems

    2024

  38. [46]

    (2022) Simple open-vocabulary object detection

    Minderer M, Gritsenko A, Stone A, Neumann M, Weissenborn D et al. (2022) Simple open-vocabulary object detection. In: Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part X. Berlin, Heidelberg: Springer-Verlag, p. 728–755

    2022

  39. [47]

    In: 2019 International Conference on Robotics and Automation (ICRA)

    Mousavian A, Toshev A, Fi s er M, Ko s eck\' a J, Wahid A and Davidson J (2019) Visual representations for semantic target driven navigation. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE Press, p. 8846–8852

    2019

  40. [48]

    (2024) Gpt-4 technical report

    OpenAI, Achiam J, Adler S, Agarwal S, Ahmad L et al. (2024) Gpt-4 technical report

    2024

  41. [49]

    (2023) Dinov2: Learning robust visual features without supervision

    Oquab M, Darcet T, Moutakanni T, Vo HV, Szafraniec M et al. (2023) Dinov2: Learning robust visual features without supervision

    2023

  42. [50]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Peng S, Genova K, Jiang C , Tagliasacchi A, Pollefeys M and Funkhouser T (2023) Openscene: 3d scene understanding with open vocabularies. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 815--824

    2023

  43. [51]

    In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Qin M, Li W, Zhou J, Wang H and Pfister H (2024) LangSplat: 3D Language Gaussian Splatting . In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, pp. 20051--20060

    2024

  44. [52]

    (2021) Learning transferable visual models from natural language supervision

    Radford A, Kim JW, Hallacy C, Ramesh A, Goh G et al. (2021) Learning transferable visual models from natural language supervision. In: Meila M and Zhang T (eds.) Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol...

    2021

  45. [53]

    Rajvanshi A, Sikka K, Lin X, Lee B, Chiu HP and Velasquez A (2023) Saynav: Grounding large language models for dynamic planning to navigation in new environments

    2023

  46. [54]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Ramakrishnan SK, Chaplot DS, Al-Halah Z, Malik J and Grauman K (2022) Poni: Potential functions for objectgoal navigation with interaction-free learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 18890--18900

    2022

  47. [55]

    (2021) Habitat-matterport 3d dataset ( HM 3d): 1000 large-scale 3d environments for embodied AI

    Ramakrishnan SK, Gokaslan A, Wijmans E, Maksymets O, Clegg A et al. (2021) Habitat-matterport 3d dataset ( HM 3d): 1000 large-scale 3d environments for embodied AI . In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2021

  48. [56]

    In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 229

    Rana K, Haviland J, Garg S, Abou-Chakra J, Reid I and Suenderhauf N (2023) Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning. In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedi...

    2023

  49. [57]

    The International Journal of Robotics Research 30(6): 755--771

    Ranganathan A and Dellaert F (2011) Online probabilistic topological mapping. The International Journal of Robotics Research 30(6): 755--771

    2011

  50. [58]

    International Conference on Robotics and Automation (ICRA)

    Ravichandran Z, Murali V, Tzes M, Pappas GJ and Kumar V (2025) Spine: Online semantic planning for missions with incomplete natural language specifications in unstructured environments. International Conference on Robotics and Automation (ICRA)

    2025

  51. [59]

    In: 2022 International Conference on Robotics and Automation (ICRA)

    Ravichandran Z, Peng L, Hughes N, Griffith JD and Carlone L (2022) Hierarchical representations and explicit memory: Learning effective navigation policies on 3d scene graphs using graph neural networks. In: 2022 International Conference on Robotics and Automation (ICRA). pp. ...

    2022

  52. [60]

    Ray A, Bradley C, Carlone L and Roy N (2024) Task and motion planning in hierarchical 3d scene graphs

    2024

  53. [61]

    Ren AZ, Clark J, Dixit A, Itkina M, Majumdar A and Sadigh D (2024) Explore until confident: Efficient exploration for embodied question answering

    2024

  54. [62]

    (2021) Kimera: From slam to spatial perception with 3d dynamic scene graphs

    Rosinol A, Violette A, Abate M, Hughes N, Chang Y et al. (2021) Kimera: From slam to spatial perception with 3d dynamic scene graphs. The International Journal of Robotics Research 40(12-14): 1510--1546

    2021

  55. [63]

    Proceedings of the National Academy of Sciences 93(4): 1591--1595

    Sethian JA (1996) A fast marching level set method for monotonically advancing fronts. Proceedings of the National Academy of Sciences 93(4): 1591--1595

    1996

  56. [64]

    In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 229

    Shah D, Equi MR, Osi\' n ski B, Xia F, Ichter B and Levine S (2023 a ) Navigation with large language models: Semantic guesswork as a heuristic for planning. In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine...

    2023

  57. [65]

    (2023 b ) Vint: A foundation model for visual navigation

    Shah D, Sridhar A, Dashora N, Stachowicz K, Black K et al. (2023 b ) Vint: A foundation model for visual navigation. In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 229. PMLR, pp....

    2023

  58. [66]

    In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 229

    Shen W, Yang G, Yu A, Wong J, Kaelbling LP and Isola P (2023) Distilled feature fields enable few-shot language-guided manipulation. In: Tan J, Toussaint M and Darvish K (eds.) Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research, volum...

    2023

  59. [67]

    (2023) Progprompt: Generating situated robot task plans using large language models

    Singh I, Blukis V, Mousavian A, Goyal A, Xu D et al. (2023) Progprompt: Generating situated robot task plans using large language models. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 11523--11530

    2023

  60. [68]

    arXiv pre-print

    Sridhar A, Shah D, Glossop C and Levine S (2023) NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration . arXiv pre-print

    2023

  61. [69]

    IEEE Robotics and Automation Letters

    Strader J, Hughes N, Chen W, Speranzon A and Carlone L (2024) Indoor and outdoor 3d scene graph generation via language-enabled spatial ontologies. IEEE Robotics and Automation Letters

    2024

  62. [70]

    (2021) Habitat 2.0: Training home assistants to rearrange their habitat

    Szot A, Clegg A, Undersander E, Wijmans E, Zhao Y et al. (2021) Habitat 2.0: Training home assistants to rearrange their habitat. In: Ranzato M, Beygelzimer A, Dauphin Y, Liang P and Vaughan JW (eds.) Advances in Neural Information Processing Systems, volume 34. Curran Associa...

    2021

  63. [71]

    (2023) Llama 2: Open foundation and fine-tuned chat models

    Touvron H, Martin L, Stone K, Albert P, Almahairi A et al. (2023) Llama 2: Open foundation and fine-tuned chat models

    2023

  64. [72]

    In: Kober J, Ramos F and Tomlin C (eds.) Proceedings of the 2020 Conference on Robot Learning, Proceedings of Machine Learning Research, volume 155

    Wahid A, Stone A, Chen K, Ichter B and Toshev A (2021) Learning object-conditioned exploration using distributed soft actor critic. In: Kober J, Ramos F and Tomlin C (eds.) Proceedings of the 2020 Conference on Robot Learning, Proceedings of Machine Learning Research, volume 1...

    2021

  65. [73]

    In: 2019 International Conference on Robotics and Automation (ICRA)

    Wandzel A, Oh Y, Fishman M, Kumar N, Wong LL and Tellex S (2019) Multi-object search using object-oriented pomdps. In: 2019 International Conference on Robotics and Automation (ICRA). pp. 7194--7200

    2019

  66. [74]

    IEEE Robotics and Automation Letters 10(1): 160--167

    Wang J, He G and Kantaros Y (2025) Probabilistically correct language-based multi-robot planning using conformal prediction. IEEE Robotics and Automation Letters 10(1): 160--167

    2025

  67. [75]

    In: Larochelle H, Ranzato M, Hadsell R, Balcan M and Lin H (eds.) Advances in Neural Information Processing Systems, volume 33

    Wani S, Patel S, Jain U, Chang A and Savva M (2020) Multion: Benchmarking semantic map memory using multi-object navigation. In: Larochelle H, Ranzato M, Hadsell R, Balcan M and Lin H (eds.) Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc.,...

    2020

  68. [76]

    Robotics: Science and Systems

    Werby A, Huang C, Büchner M, Valada A and Burgard W (2024) Hierarchical open-vocabulary 3d scene graphs for language-grounded robot navigation. Robotics: Science and Systems

    2024

  69. [77]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Wu SC, Wald J, Tateno K, Navab N and Tombari F (2021) Scenegraphfusion: Incremental 3d scene graph prediction from rgb-d sequences. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 7515--7525

    2021

  70. [78]

    In: Computer Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on

    Xia F, R Zamir A, He ZY, Sax A, Malik J and Savarese S (2018) Gibson env: real-world perception for embodied agents. In: Computer Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on. IEEE, pp. 9068--9079

    2018

  71. [79]

    Xie Q, Min SY, Ji P, Yang Y, Zhang T, Bajaj A, Salakhutdinov R, Johnson-Roberson M and Bisk Y (2024) Embodied-rag: General non-parametric embodied memory for retrieval and generation

    2024

  72. [80]

    (2022 a ) Habitat challenge 2022

    Yadav K, Ramakrishnan SK, Turner J, Gokaslan A, Maksymets O et al. (2022 a ) Habitat challenge 2022. https://aihabitat.org/challenge/2022/

    2022

  73. [81]

    (2022 b ) Habitat-matterport 3d semantics dataset

    Yadav K, Ramrakhya R, Ramakrishnan SK, Gervet T, Turner J, Gokaslan A, Maestre N, Chang AX, Batra D, Savva M et al. (2022 b ) Habitat-matterport 3d semantics dataset. arXiv preprint arXiv:2210.05633

    2022 arXiv

  74. [82]

    In: Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97

    Yamauchi B (1997) A frontier-based approach for autonomous exploration. In: Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'. pp. 146--151

    1997

  75. [83]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Yang L, Kang B, Huang Z, Xu X, Feng J and Zhao H (2024) Depth anything: Unleashing the power of large-scale unlabeled data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10371--10381

    2024

  76. [84]

    In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)

    Ye J, Batra D, Das A and Wijmans E (2021) Auxiliary tasks and exploration enable objectgoal navigation. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 16097--16106

    2021

  77. [85]

    In: 2024 IEEE International Conference on Robotics and Automation (ICRA)

    Yokoyama N, Ha S, Batra D, Wang J and Bucher B (2024) Vlfm: Vision-language frontier maps for zero-shot semantic navigation. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 42--48

    2024

  78. [86]

    In: IEEE/RSJ International Conference on Intelligent Robots and Systems

    Yu B, Kasaei H and Cao M (2023) L3mvn: Leveraging large language models for visual target navigation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 3554--3560

    2023

  79. [87]

    arXiv preprint arXiv:2404.03868

    Zhang B and Soh H (2024) Extract, define, canonicalize: An llm-based framework for knowledge graph construction. arXiv preprint arXiv:2404.03868

    2024 arXiv

  80. [88]

    In: Oh A, Naumann T, Globerson A, Saenko K, Hardt M and Levine S (eds.) Advances in Neural Information Processing Systems, volume 36

    Zhao Z, Lee WS and Hsu D (2023) Large language models as commonsense knowledge for large-scale task planning. In: Oh A, Naumann T, Globerson A, Saenko K, Hardt M and Levine S (eds.) Advances in Neural Information Processing Systems, volume 36. Curran Associates, Inc., pp. 31967--31987

    2023

  81. [89]

    In: Kim B, Yue Y, Chaudhuri S, Fragkiadaki K, Khan M and Sun Y (eds.) International Conference on Representation Learning, volume 2024

    Zheng HS, Mishra S, Chen X, Cheng HT, Chi EH, Le QV and Zhou D (2024) Take a step back: Evoking reasoning via abstraction in large language models. In: Kim B, Yue Y, Chaudhuri S, Fragkiadaki K, Khan M and Sun Y (eds.) International Conference on Representation Learning, volume...

    2024

  82. [90]

    In: 2023 IEEE International Conference on Robotics and Automation (ICRA)

    Zheng K, Paul A and Tellex S (2023) A system for generalized 3d multi-object search. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 1638--1644

    2023

  83. [91]

    2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

    Zheng K, Sung Y, Konidaris GD and Tellex S (2020) Multi-resolution pomdp planning for multi-object search in 3d. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

    2020

  84. [92]

    Zhou G, Hong Y and Wu Q (2023) Navgpt: Explicit reasoning in vision-and-language navigation with large language models

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.