Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

ObjectReact: Learning Object-Relative Control for Visual Navigation

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Object-level costmaps, not image pairs, let a robot navigate unseen routes and change heights without retraining.

desk verdict ObjectReact's WayObject Costmap is a real contribution, but the headline SPL gains all come with a ground-truth localization oracle, so they are controller-only numbers. read the letter →

arxiv 2509.09594 v1 pith:HBRFVJKB submitted 2025-09-11 cs.RO cs.AIcs.CVcs.LGcs.SYeess.SY

Sourav Garg , Dustin Craggs , Vineeth Bhat , Lachlan Mares , Stefan Podgorski , Madhava Krishna , Feras Dayoub , Ian Reid This is my paper
classification cs.ROcs.AIcs.CVcs.LGcs.SYeess.SY
keywords object-relativecontrolvisualtopologicalnavigation3DscenegraphWayObjectCostmapcross-embodimentgeneralizationmonoculardepthsegmentation-basedmappingpolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ObjectReact is a navigation controller that learns from a "WayObject Costmap"—a per-object map of path lengths to the goal overlaid on segmentation masks—instead of from an image pair. The paper's central claim is that this object-relative representation decouples control from image matching, so the controller can handle tasks that require spatial reasoning beyond imitating a prior trajectory: reaching previously unvisited goals, cutting shortcuts, and travelling routes in reverse. On these tasks the object-relative controller scores roughly ten to twenty percentage points higher in SPL than an image-relative baseline trained on the same data, and it is almost unaffected by changing the execution camera height from 1.3m to 0.4m (SPL drop of 3 vs 48 points). The paper also reports that a policy trained only in simulation transfers to real indoor environments, and that the remaining gap on hard tasks is mostly due to imperfect perception rather than the learned controller itself.

What carries the argument

WayObject Costmap: a multi-channel image in which each pixel of an object's segmentation mask carries a sine-cosine encoded path length from that object to the goal, with unmatched or undetected segments set to a fixed outlier cost. This representation converts a variable number of object subgoals into a fixed-size learned input that retains both spatial layout and goal-directed cost, so a controller can predict waypoints without any RGB input.

What would settle it

In the same four simulator tasks, replace oracle localization with matcher-based localization or add random submap offsets; if SPL on Alt Goal, Shortcut, and Reverse drops toward the image-relative baseline as offset grows, the invariance claim is localization-dependent. A real-world run without any ground-truth localization would settle it.

Watch

Extended reading notes

Core claim

The central discovery is that the control problem in topological visual navigation can be re-expressed as reacting to a dense cost image built from objects. The map is a relative 3D scene graph: nodes are segmentation masks, intra-image edges are 3D Euclidean distances estimated from monocular depth, and inter-image edges link corresponding objects across frames. During execution the agent matches visible objects to map nodes, computes Dijkstra path lengths from each matched object to the long-horizon goal, normalizes those lengths per image, and encodes them with sine-cosine positional embeddings; the resulting multi-channel WayObject Costmap is the only input to a convolutional controller

Load-bearing premise

Every simulator run hands the agent the correct map image index as localization, so the costmap is always built from the right map context; if localization is noisy, the object-relative advantage could shrink or vanish.

Editorial extensions

If this is right

  • A map captured at one camera height can be executed at another height with little loss: SPL drops 3.04 points for ObjectReact versus 48.49 for the image-relative baseline.
  • Controller performance no longer depends on retrieving a subgoal image, so previously unvisited goals, shortcuts, and reversed routes become navigable.
  • Training data requirements drop from quadratic (all embodiment pairs of current and goal images) to linear, since the controller input is embodiment-invariant.
  • A policy trained only in simulation can generalize to real indoor environments, as demonstrated on a quadruped robot, including around obstacles that appeared after mapping.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the WayObject Costmap is a fixed-size grid encoding of goal-directed object costs, it could likely be fed to other policy classes (discrete action, diffusion) without changing the pipeline; the paper only tests its own convolutional predictor.
  • The closest testable extension is to add localization noise in simulation; the paper's use of oracle localization means the robustness of the object-relative advantage to real perception remains an open question.
  • The costmap might also be generated from language instructions or exploration objectives, extending it beyond the paper's map-based planning source and toward a more general goal-conditioning interface.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces ObjectReact, a visual navigation controller that is conditioned on a 'WayObject Costmap' rather than on a pair of current and subgoal images. The map is a relative 3D scene graph built from object segments, monocular depth, and pairwise feature matching; during execution, matched objects receive global path lengths to the goal, and these are encoded as a multi-channel costmap that serves as the sole controller input. The controller is trained in HM3D and compared against an image-relative GNM baseline on four tasks: Imitate, Alt Goal, Shortcut, and Reverse. The reported results show large SPL improvements for the object-relative controller on the non-imitation tasks and much smaller sensitivity to a change in camera height between mapping and execution. Real-world deployment on a quadruped is presented qualitatively. The paper's central claim is that object-relative control decouples control prediction from image matching and prior-trajectory imitation, enabling new routes and cross-embodiment transfer.

Significance. If the reported results hold up, the paper makes a useful conceptual and empirical contribution: conditioning a local controller on object-level path-length costmaps is an interesting alternative to image-pair conditioning, and the height-invariance result in Table 2 is a compelling illustration of the potential benefit of object-level representations. The tasks themselves (Alt Goal, Shortcut, Reverse) are also a valuable addition to the topological navigation evaluation toolbox. The availability of code via the project page is a positive reproducibility signal. However, the strength of the evidence is currently limited by the use of a ground-truth localization oracle in all simulator experiments, by the large number of manually excluded episodes, and by the absence of uncertainty quantification. These issues directly bear on the paper's claim to demonstrate a visual navigation paradigm rather than a control policy evaluated under idealized localization.

major comments (4)
  1. [Section 4 and Section A.4] All simulator experiments use a ground-truth localization oracle: 'the agent is given access to ground truth localization in the form of map image index which is closest to the agent's current 2D position.' The WayObject Costmap is then built by matching the current FastSAM segments only against the submap centered at that oracle index. This means Tables 1, 2, and 3 evaluate the controller conditional on perfect map context, not the full visual navigation pipeline. The real-world active localization described in Section A.4 is only demonstrated qualitatively in videos. Since the abstract and introduction claim a new 'visual navigation' paradigm and say the control prediction problem is 'decoupled from solving the image matching problem,' this protocol gap is load-bearing. Please add simulator experiments that use the active localization procedure or a controlled localization-noise model,
  2. [Section B.2 and Tables 1-3] The evaluation excludes a large fraction of episodes based on manual inspection: 6/36 Imitate, 16/36 Alt-Goal, 13/36 Shortcut, and 9/36 Reverse episodes are removed. The reported Alt-Goal SPL of 21.74 vs. 2.17 and Shortcut SPL of 23.08 vs. 7.69 are therefore computed on 20 and 23 episodes, respectively. Manual post-hoc exclusion without pre-specified criteria, and without a sensitivity analysis, can bias the comparison. Please report results on all 36 episodes (for example, treating invalid episodes as failures for all methods) and provide per-episode results. If some episodes are truly invalid, the exclusion criteria should be applied identically to every method and justified in advance.
  3. [Tables 1-3] No error bars, confidence intervals, or significance tests are reported. With 20-30 episodes per task and 72 runs total for the height experiment, the large point differences could be driven by a small number of episodes. Please report per-episode distributions, standard errors, and paired tests (e.g., Wilcoxon signed-rank) where appropriate. In addition, Table 3 is generated using ground-truth object instances from the simulator, while Table 1 uses inferred segmentation and matching; the protocol shift should be clearly flagged whenever Table 3 results are cited.
  4. [Section 5.1 and Section 4] The GNM baseline's subgoal-image selection procedure is not specified in the evaluation. Section 5.1 attributes GNM's poor performance on Alt Goal, Shortcut, and Reverse to 'temporal distance prediction' for subgoal selection, but it is unclear whether the GNM system used its learned distance head or the ground-truth localization oracle to choose the subgoal image. This distinction matters for interpretation: an oracle-selected subgoal would give GNM an advantage on subgoal selection, while a learned distance head could introduce a separate failure source. Please specify exactly how the GNM subgoal image was selected in Tables 1-2, and ideally ablate GNM with oracle-selected subgoals versus learned-distance subgoals.
minor comments (4)
  1. [Section A.1] Equation (2) and the surrounding text contain a typo: 'such that l∈[1,L] Z.l=L' is garbled; presumably 'l=L represents the shortest path length per image and l=0 represents outliers.' Please clarify the encoding of the cost values.
  2. [Section A.2 and Section A.6] The costmap resolution is given as W=85, H=64 in Section A.2 but as 85×65 in Section A.6. Please correct the inconsistency.
  3. [Reference [61]] Reference [61] spells the author as 'Dijksta'; it should be Dijkstra.
  4. [Appendix D.2] There are unmatched quotation marks in '“floor” or “‘ceiling”' in the text discussing CLIP-based removal of floor/ceiling nodes.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: WayObject Costmaps are planning-conditioning inputs, not fitted targets, and the central claim is tested against external baselines (GNM, PixNav).

full rationale

The paper's derivation chain is empirical, not definitional. ObjectReact is trained to predict trajectory rollouts from WayObject Costmaps (Sec. 3.3, Eq. 1-2), where the costmap is a conditioning input computed from object masks and Dijkstra path lengths; it is not a supervised target derived from the evaluation metric. The headline comparisons (Tables 1-2) pit this controller against GNM and PixNav, external baselines, so the central claim does not reduce to a fit. The method reuses the authors' prior RoboHop for object-level localization and planning (Sec. 3), but this is a modular component that is also included as a baseline (Sec. 5.3); no uniqueness theorem or ansatz is imported from self-citations to force the result. The remaining concerns are experimental limitations, not circularity: Sec. 4 grants ground-truth localization ('the agent is given access to ground truth localization in the form of map image index which is closest to the agent's current 2D position'), and Sec. B.2 excludes 6-16 episodes per task; these affect external validity but do not mean any 'prediction' is equivalent to its inputs by construction. Real-world demos (Sec. C.2.2) are qualitative, again a validity caveat rather than a circularity.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

No new physical entities are proposed. The new representations, the relative 3D scene graph and the WayObject Costmap, are the paper's contribution and are tested empirically. The free parameters are mostly encoding and augmentation scales; the axiomatic weight sits on perception quality, on the graph path-cost proxy, and on the oracle localization used in simulation.

free parameters (7)
  • WayObject encoding dimension D = 8
    Appendix A.1 sets D=8; sine-cosine embedding width chosen by hand, not fitted to data.
  • Path length scale L = 100
    Normalized costs are rescaled to [1, L] with L=100 (Appendix A.1); hand-set scale for encoding.
  • Encoding base Z = 10000
    Standard transformer positional-encoding base, Appendix A.1.
  • Outlier fraction in training augmentation = 30%
    30% of segments randomly receive outlier cost during training to bridge ground-truth and inferred costmaps (Section 3.3).
  • Outlier cost handling = l=0 in encoding; described as fixed high cost in text
    Undetected, unlocalized, and unmatched segments are outliers; the exact encoding value is a design choice (Appendices A.1, A.4).
  • Monocular projection focal length = arbitrary but fixed
    Section 3.1 projects object pixels into 3D with an arbitrary fixed focal length; this affects intra-image edge weights, though per-image normalization may reduce its influence.
  • Controller velocity limits and history smoothing = linear 0-0.05 m/s sim, angular -0.1..0.1 rad/s, 5-frame average
    Control execution parameters in Appendix A.5; influence collision behavior and path quality, not the conceptual representation.
assumptions (6)
  • domain assumption Monocular depth (Depth Anything) plus a fixed focal length gives accurate enough relative 3D object positions for edge weights.
    Invoked in Section 3.1 to build the relative 3D scene graph; inaccurate depth changes shortest paths and therefore the WayObject Costmap.
  • domain assumption SuperPoint-LightGlue correspondences reliably associate object segments between frames and with map images.
    Used for inter-image edges (Section 3.1) and execution localization (Appendix A.4); the paper's own limitations section says perception remains an open challenge.
  • domain assumption Dijkstra path lengths over the object-level graph are a valid proxy for navigational distance to the goal.
    The WayObject Costmap is built from these path lengths (Sections 3.2 and A.4); the floor/ceiling shortcut failure mode in Section D.2 shows this proxy can mislead.
  • domain assumption Behavioral cloning from shortest-path demonstrations with ground-truth costmaps plus 30% outlier augmentation transfers to inferred perception and real-world conditions.
    Training protocol in Section 3.3; the training-deployment gap is acknowledged in Section D.6.
  • domain assumption All simulator evaluations may assume ground-truth localization of the agent in the map.
    Stated in Section 4: the agent is given the map image index closest to its current 2D position. The central quantitative results depend on this oracle.
  • domain assumption Object-level representations are inherently embodiment- and trajectory-invariant.
    Conceptual framing in Sections 1 and 2; partially supported by the height-invariance result in Table 2, but assumed in the design of the method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ObjectReact: Learning Object-Relative Control for Visual Navigation." pith.science (2026). https://pith.science/paper/HBRFVJKB

@misc{pith2026250909594,
  author       = {Pith},
  title        = {Pith review of: ObjectReact: Learning Object-Relative Control for Visual Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HBRFVJKB}},
  note         = {Machine review of arXiv:2509.09594}
}
read the original abstract

Visual navigation using only a single camera and a topological map has recently become an appealing alternative to methods that require additional sensors and 3D maps. This is typically achieved through an "image-relative" approach to estimating control from a given pair of current observation and subgoal image. However, image-level representations of the world have limitations because images are strictly tied to the agent's pose and embodiment. In contrast, objects, being a property of the map, offer an embodiment- and trajectory-invariant world representation. In this work, we present a new paradigm of learning "object-relative" control that exhibits several desirable characteristics: a) new routes can be traversed without strictly requiring to imitate prior experience, b) the control prediction problem can be decoupled from solving the image matching problem, and c) high invariance can be achieved in cross-embodiment deployment for variations across both training-testing and mapping-execution settings. We propose a topometric map representation in the form of a "relative" 3D scene graph, which is used to obtain more informative object-level global path planning costs. We train a local controller, dubbed "ObjectReact", conditioned directly on a high-level "WayObject Costmap" representation that eliminates the need for an explicit RGB input. We demonstrate the advantages of learning object-relative control over its image-relative counterpart across sensor height variations and multiple navigation tasks that challenge the underlying spatial understanding capability, e.g., navigating a map trajectory in the reverse direction. We further show that our sim-only policy is able to generalize well to real-world indoor environments. Code and supplementary material are accessible via project page: https://object-react.github.io/

Figures

Figures reproduced from arXiv: 2509.09594 by the authors.

Figure 1
Figure 1. Tasks: Each column shows a topdown view with the prior experience (map) trajectory displayed as a purple path from the purple circle (start) to green point (goal). The tasks are referred to as following: Imitate which is akin to teach-and-repeat; Alt-Goal, where the goal object is previously seen but unvisited; Shortcut, where the prior trajectory is made longer for agent to take a shortcut during inference; and Rev… view at source ↗
Figure 2
Figure 2. Object-Relative Navigation Pipeline. a) Mapping: We construct a topometric map as a relative 3D scene graph, where image segments are used as object nodes, which are connected intra-image using 3D Euclidean distances and inter-image using object association. b) Execution: Given the map, we localize each of the query objects and compute its path to the goal node; we assign these path lengths to the object’s segmentat… view at source ↗
Figure 3
Figure 3. Examples of demonstration videos. Real-world demonstration video example (left) and simulator video example (right). * The localized image is the closest match found in the map (see Section A.4 for details). 12 [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Real-world Experiments. We deploy our approach on the Unitree Go1 robot dog [86]. Here, we show egocentric RGB images, their corresponding WayObject Costmaps, and the predicted trajectory rollout at several timesteps during autonomous navigation to the goal object. At …
Figure 5
Figure 5. Figure 5: An illustration of the effect of dynamic objects and occlusions on the predicted control [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Following

    cs.AI 2026-07 conditional novelty 6.0 of 10

    UESF-Bench is a 1.43M-sample simulated benchmark for embodied agents that must first find a language-described person and then follow them; SeekFollow-VLA with task-driven routing outperforms the paper's internal baselines.

Reference graph

Works this paper leans on

94 extracted references · 3 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Savinov, A

    N. Savinov, A. Dosovitskiy, and V . Koltun. Semi-parametric topological memory for naviga- tion. InInternational Conference on Learning Representations, 2018

    2018

  2. [2]

    D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine. Vint: A large-scale, multi-task visual navigation backbone with cross-robot generalization. In7th Annual Conference on Robot Learning, 2023

    2023

  3. [3]

    D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine. Gnm: A general navigation model to drive any robot. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7226–7233. IEEE, 2023

    2023

  4. [4]

    D. S. Chaplot, R. Salakhutdinov, A. Gupta, and S. Gupta. Neural topological slam for visual navigation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12875–12884, 2020

    2020

  5. [5]

    Q. Gu, A. Kuwajerwala, S. Morin, K. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa, C. Gan, C. de Melo, J. Tenenbaum, A. Torralba, F. Shkurti, and L. Paull. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. InarXiv, 2023

    2023

  6. [6]

    R. F. Salas-Moreno, R. A. Newcombe, H. Strasdat, P. H. Kelly, and A. J. Davison. Slam++: Simultaneous localisation and mapping at the level of objects. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1352–1359, 2013

    2013

  7. [7]

    Georgakis, B

    G. Georgakis, B. Bucher, K. Schmeckpeper, S. Singh, and K. Daniilidis. Learning to map for active semantic goal navigation. InThe Tenth International Conference on Learning Repre- sentations (ICLR 2022), 2022

    2022

  8. [8]

    P. Paul, A. Garg, T. Choudhary, A. K. Singh, and K. M. Krishna. Lego-drive: Language- enhanced goal-oriented closed-loop end-to-end autonomous driving. In2024 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS), pages 10020–10026. IEEE, 2024

    2024

Show all 94 references
  1. [9]

    Huang, O

    C. Huang, O. Mees, A. Zeng, and W. Burgard. Visual language maps for robot navigation. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), London, UK, 2023

    2023

  2. [10]

    Weiss, D

    S. Weiss, D. Scaramuzza, and R. Siegwart. Monocular-slam–based navigation for autonomous micro helicopters in gps-denied environments.Journal of Field Robotics, 28(6):854–874, 2011

    2011

  3. [11]

    Zhang, L

    J. Zhang, L. Dai, F. Meng, Q. Fan, X. Chen, K. Xu, and H. Wang. 3d-aware object goal navigation via simultaneous exploration and identification. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6672–6682, 2023

    2023

  4. [12]

    Q. Zhao, L. Zhang, B. He, H. Qiao, and Z. Liu. Zero-shot object goal visual navigation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 2025–2031. IEEE, 2023

    2023

  5. [13]

    cat-shaped mug

    V . S. Dorbala, J. F. Mullen Jr, and D. Manocha. Can an embodied agent find your “cat-shaped mug”? llm-based zero-shot object navigation.IEEE Robotics and Automation Letters, 2023

    2023

  6. [14]

    D. S. Chaplot, D. P. Gandhi, A. Gupta, and R. R. Salakhutdinov. Object goal navigation using goal-oriented semantic exploration.Advances in Neural Information Processing Systems, 33: 4247–4258, 2020

    2020

  7. [15]

    N. Kim, O. Kwon, H. Yoo, Y . Choi, J. Park, and S. Oh. Topological Semantic Graph Memory for Image Goal Navigation. InCoRL, 2022. 17

    2022

  8. [16]

    O. Kwon, N. Kim, Y . Choi, H. Yoo, J. Park, and S. Oh. Visual graph memory with unsupervised representation for visual navigation. InProceedings of the IEEE/CVF international conference on computer vision, pages 15890–15899, 2021

    2021

  9. [17]

    Armeni, Z.-Y

    I. Armeni, Z.-Y . He, J. Gwak, A. R. Zamir, M. Fischer, J. Malik, and S. Savarese. 3d scene graph: A structure for unified semantics, 3d space, and camera. InProceedings of the IEEE/CVF international conference on computer vision, pages 5664–5673, 2019

    2019

  10. [18]

    Rosinol, A

    A. Rosinol, A. Violette, M. Abate, N. Hughes, Y . Chang, J. Shi, A. Gupta, and L. Carlone. Kimera: From slam to spatial perception with 3d dynamic scene graphs.The International Journal of Robotics Research, 40(12-14):1510–1546, 2021

    2021

  11. [19]

    K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suenderhauf. Sayplan: Ground- ing large language models using 3d scene graphs for scalable task planning. In7th Annual Conference on Robot Learning, 2023. URLhttps://openreview.net/forum?id= wMpOMO0Ss7a

    2023

  12. [20]

    Werby, C

    A. Werby, C. Huang, M. B¨uchner, A. Valada, and W. Burgard. Hierarchical open-vocabulary 3d scene graphs for language-grounded robot navigation. InFirst Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024, 2024

    2024

  13. [21]

    Ravichandran, L

    Z. Ravichandran, L. Peng, N. Hughes, J. D. Griffith, and L. Carlone. Hierarchical represen- tations and explicit memory: Learning effective navigation policies on 3d scene graphs using graph neural networks. In2022 International Conference on Robotics and Automation (ICRA), pa...

    2022

  14. [22]

    Seymour, N

    Z. Seymour, N. C. Mithun, H.-P. Chiu, S. Samarasekera, and R. Kumar. Graphmapper: Effi- cient visual navigation by scene graph generation. In2022 26th International Conference on Pattern Recognition (ICPR), pages 4146–4153. IEEE, 2022

    2022

  15. [23]

    K. P. Singh, J. Salvador, L. Weihs, and A. Kembhavi. Scene graph contrastive learning for embodied navigation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10884–10894, 2023

    2023

  16. [24]

    H. Yin, X. Xu, Z. Wu, J. Zhou, and J. Lu. Sg-nav: Online 3d scene graph prompting for llm- based zero-shot object navigation.Advances in neural information processing systems, 37: 5285–5307, 2024

    2024

  17. [25]

    R. Liu, X. Wang, W. Wang, and Y . Yang. Bird’s-eye-view scene graph for vision-language navigation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10968–10980, 2023

    2023

  18. [26]

    Shah and S

    D. Shah and S. Levine. ViKiNG: Vision-Based Kilometer-Scale Navigation with Geo- graphic Hints. InProceedings of Robotics: Science and Systems, 2022. URLhttp: //www.roboticsproceedings.org/rss18/p019.html

    2022

  19. [27]

    D. Shah, B. Osinski, B. Ichter, and S. Levine. LM-nav: Robotic navigation with large pre- trained models of language, vision, and action. In6th Annual Conference on Robot Learning,

  20. [28]

    Sridhar, D

    A. Sridhar, D. Shah, C. Glossop, and S. Levine. Nomad: Goal masked diffusion policies for navigation and exploration. In2024 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 63–70. IEEE, 2024

    2024

  21. [29]

    D. Shah, B. Eysenbach, G. Kahn, N. Rhinehart, and S. Levine. Ving: Learning open-world navigation with visual goals. In2021 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 13215–13222. IEEE, 2021. 18

    2021

  22. [30]

    D. Shah, B. Eysenbach, N. Rhinehart, and S. Levine. Rapid exploration for open-world nav- igation with latent goal models. InConference on Robot Learning, pages 674–684. PMLR, 2022

    2022

  23. [31]

    R. S. Sutton. Learning to predict by the methods of temporal differences.Machine learning, 3 (1):9–44, 1988

    1988

  24. [32]

    Horswill

    I. Horswill. Polly: A vision-based artificial agent. InAAAI, pages 824–829, 1993

    1993

  25. [33]

    Matsumoto, M

    Y . Matsumoto, M. Inaba, and H. Inoue. Visual navigation using view-sequenced route rep- resentation. InProceedings of IEEE International conference on Robotics and Automation, volume 1, pages 83–88. IEEE, 1996

    1996

  26. [34]

    R. F. Vassallo, H. J. Schneebeli, and J. Santos-Victor. Visual servoing and appearance for navigation.Robotics and autonomous systems, 31(1-2):87–97, 2000

    2000

  27. [35]

    S. Thrun. An approach to learning mobile robot navigation.Robotics and Autonomous systems, 15(4):301–319, 1995

    1995

  28. [36]

    Saxena, H

    A. Saxena, H. Pandya, G. Kumar, A. Gaud, and K. M. Krishna. Exploring convolutional networks for end-to-end visual servoing. In2017 IEEE International Conference on Robotics and Automation (ICRA), pages 3817–3823. IEEE, 2017

    2017

  29. [37]

    Pathak, P

    D. Pathak, P. Mahmoudieh, G. Luo, P. Agrawal, D. Chen, Y . Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell. Zero-shot visual imitation. InProceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 2050–2053, 2018

  30. [38]

    Li and J

    Y . Li and J. Ko ˇsecka. Learning view and target invariant visual servoing for navigation. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 658–664. IEEE, 2020

    2020

  31. [39]

    X. Meng, N. Ratliff, Y . Xiang, and D. Fox. Scaling local control to large-scale topological navigation. In2020 IEEE International Conference on Robotics and Automation (ICRA), pages 672–678. IEEE, 2020

    2020

  32. [40]

    Katara, Y

    P. Katara, Y . Harish, H. Pandya, A. Gupta, A. Sanchawala, G. Kumar, B. Bhowmick, and M. Krishna. Deepmpcvs: Deep model predictive control for visual servoing. InConference on Robot Learning, pages 2006–2015. PMLR, 2021

    2006

  33. [41]

    Pathre, G

    P. Pathre, G. Gupta, M. N. Qureshi, M. Brunda, S. Brahmbhatt, and K. M. Krishna. Imag- ine2servo: Intelligent visual servoing with diffusion-driven goal generation for robotic tasks. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 13466...

    2024

  34. [42]

    Ehsani, T

    K. Ehsani, T. Gupta, R. Hendrix, J. Salvador, L. Weihs, K.-H. Zeng, K. P. Singh, Y . Kim, W. Han, A. Herrasti, et al. Spoc: Imitating shortest paths in simulation enables effective navigation and manipulation in the real world. InProceedings of the IEEE/CVF Conference on Compu...

    2024

  35. [43]

    Hutchinson, G

    S. Hutchinson, G. D. Hager, and P. I. Corke. A tutorial on visual servo control.IEEE transac- tions on robotics and automation, 12(5):651–670, 1996

    1996

  36. [44]

    S. D. Jones, C. Andresen, and J. L. Crowley. Appearance based process for visual navigation. InProceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Sys- tems. Innovative Robotics for Real-World Applications. IROS’97, volume 2, pages 551–557. IEEE, 1997

    1997

  37. [45]

    Mezouar and F

    Y . Mezouar and F. Chaumette. Path planning for robust image-based control.IEEE transac- tions on robotics and automation, 18(4):534–549, 2002. 19

    2002

  38. [46]

    Blanc, Y

    G. Blanc, Y . Mezouar, and P. Martinet. Indoor navigation of a wheeled mobile robot along visual routes. InProceedings of the 2005 IEEE international conference on robotics and automation, pages 3354–3359. IEEE, 2005

    2005

  39. [47]

    Remazeilles, F

    A. Remazeilles, F. Chaumette, and P. Gros. 3d navigation based on a visual memory. In Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., pages 2719–2725. IEEE, 2006

    2006

  40. [48]

    Cherubini, F

    A. Cherubini, F. Chaumette, and G. Oriolo. Visual servoing for path reaching with nonholo- nomic robots.Robotica, 29(7):1037–1048, 2011

    2011

  41. [49]

    Diosi, S

    A. Diosi, S. Segvic, A. Remazeilles, and F. Chaumette. Experimental evaluation of au- tonomous driving based on visual memory and image-based visual servoing.IEEE Trans- actions on Intelligent Transportation Systems, 12(3):870–883, 2011

    2011

  42. [50]

    S. R. Bista, P. R. Giordano, and F. Chaumette. Appearance-based indoor navigation by ibvs using line segments.IEEE robotics and automation letters, 1(1):423–430, 2016

    2016

  43. [51]

    Ahmadi, L

    A. Ahmadi, L. Nardi, N. Chebrolu, and C. Stachniss. Visual servoing-based navigation for monitoring row-crop fields. In2020 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 4920–4926. IEEE, 2020

    2020

  44. [52]

    S. Feng, Z. Wu, Y . Zhao, and P. A. Vela. Trajectory servoing: Image-based trajectory tracking using slam.CoRR, 2021

    2021

  45. [53]

    S. Garg, K. Rana, M. Hosseinzadeh, L. Mares, N. Suenderhauf, F. Dayoub, and I. Reid. Robo- hop: Segment-based topological map representation for open-world visual navigation. In2024 International Conference on Robotics and Automation (ICRA). IEEE, 2024

    2024

  46. [54]

    W. Cai, S. Huang, G. Cheng, Y . Long, P. Gao, C. Sun, and H. Dong. Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5228–5234. IEEE, 2024

    2024

  47. [55]

    Podgorski, S

    S. Podgorski, S. Garg, M. Hosseinzadeh, L. Mares, F. Dayoub, and I. Reid. Tango: Traversablility-aware navigation with local metric control for topological goals. In2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025

    2025

  48. [56]

    Y . Wu, Y . Wu, A. Tamar, S. Russell, G. Gkioxari, and Y . Tian. Bayesian relational memory for semantic visual navigation. InProceedings of the IEEE/CVF international conference on computer vision, pages 2769–2779, 2019

    2019

  49. [57]

    W. Yang, X. Wang, A. Farhadi, A. Gupta, and R. Mottaghi. Visual semantic navigation using scene priors. InInternational Conference on Learning Representations, 2019

    2019

  50. [58]

    H. Du, X. Yu, and L. Zheng. Learning object relation graph and tentative policy for visual navigation. InEuropean Conference on Computer Vision, pages 19–34. Springer, 2020

    2020

  51. [59]

    H. Yoo, Y . Choi, J. Park, and S. Oh. Commonsense-aware object value graph for object goal navigation.IEEE Robotics and Automation Letters, 9(5):4423–4430, 2024

    2024

  52. [60]

    M. Hahn, D. S. Chaplot, S. Tulsiani, M. Mukadam, J. M. Rehg, and A. Gupta. No rl, no sim- ulation: Learning to navigate without navigating.Advances in Neural Information Processing Systems, 34:26661–26673, 2021

    2021

  53. [61]

    E. W. Dijksta. A note on two problems in connexion with graphs.Numerische mathematik, 1 (1):269–271, 1959

    1959

  54. [62]

    Furgale and T

    P. Furgale and T. D. Barfoot. Visual teach and repeat for long-range rover autonomy.Journal of field robotics, 27(5):534–560, 2010. 20

    2010

  55. [63]

    ˇSegvi´c, A

    S. ˇSegvi´c, A. Remazeilles, A. Diosi, and F. Chaumette. A mapping and localization framework for scalable appearance-based navigation.Computer Vision and Image Understanding, 113(2): 172–187, 2009

    2009

  56. [64]

    A. M. Zhang and L. Kleeman. Robust appearance based visual route following for navigation in large-scale outdoor environments.The International Journal of Robotics Research, 28(3): 331–356, 2009

    2009

  57. [65]

    Dall’Osto, T

    D. Dall’Osto, T. Fischer, and M. Milford. Fast and robust bio-inspired teach and repeat navi- gation. In2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 500–507. IEEE, 2021

    2021

  58. [66]

    Mattamala, N

    M. Mattamala, N. Chebrolu, and M. Fallon. An efficient locally reactive controller for safe navigation in visual teach and repeat missions.IEEE Robotics and Automation Letters, 7(2): 2353–2360, 2022

    2022

  59. [67]

    Krajn ´ık, F

    T. Krajn ´ık, F. Majer, L. Halodov´a, and T. Vintr. Navigation without localisation: reliable teach and repeat based on the convergence theorem. In2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1657–1664. IEEE, 2018

    2018

  60. [68]

    Halodov ´a, E

    L. Halodov ´a, E. Dvo ˇrr´akov´a, F. Majer, T. Vintr, O. M. Mozos, F. Dayoub, and T. Krajn ´ık. Predictive and adaptive maps for long-term visual navigation in changing environments. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7033...

    2019

  61. [69]

    T. Do, L. C. Carrillo-Arce, and S. I. Roumeliotis. High-speed autonomous quadrotor navigation through visual and inertial paths.The International Journal of Robotics Research, 38(4):486– 504, 2019

    2019

  62. [70]

    Krajn ´ık, P

    T. Krajn ´ık, P. Crist´oforis, K. Kusumam, P. Neubert, and T. Duckett. Image features for visual teach-and-repeat navigation in changing environments.Robotics and Autonomous Systems, 88: 127–141, 2017

    2017

  63. [71]

    Kumar, S

    A. Kumar, S. Gupta, D. Fouhey, S. Levine, and J. Malik. Visual memory for robust path following.Advances in neural information processing systems, 31, 2018

    2018

  64. [72]

    Levine and D

    S. Levine and D. Shah. Learning robotic navigation from experience: principles, methods and recent results.Philosophical Transactions of the Royal Society B, 378(1869):20210447, 2023

    2023

  65. [73]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  66. [74]

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, et al. Sam 2: Segment anything in images and videos. InThe Thirteenth In- ternational Conference on Learning Representations, 2025

    2025

  67. [75]

    X. Zhao, W. Ding, Y . An, Y . Du, T. Yu, M. Li, M. Tang, and J. Wang. Fast segment anything, 2023

    2023

  68. [76]

    Lindenberger, P.-E

    P. Lindenberger, P.-E. Sarlin, and M. Pollefeys. LightGlue: Local Feature Matching at Light Speed. InICCV, 2023

    2023

  69. [77]

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InCVPR, 2024

    2024

  70. [78]

    DeTone, T

    D. DeTone, T. Malisiewicz, and A. Rabinovich. Superpoint: Self-supervised interest point detection and description. InProceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 224–236, 2018. 21

    2018

  71. [79]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polo- sukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  72. [80]

    S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. M. Turner, E. Un- dersander, W. Galuba, A. Westbury, A. X. Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. InThirty-fifth Conference on Neural Infor- m...

    2021

  73. [81]

    Krantz, S

    J. Krantz, S. Lee, J. Malik, D. Batra, and D. S. Chaplot. Instance-specific image goal navi- gation: Training embodied agents to find object instances.arXiv preprint arXiv:2211.15876, 2022

    2022 arXiv

  74. [82]

    Yadav, J

    K. Yadav, J. Krantz, R. Ramrakhya, S. K. Ramakrishnan, J. Yang, A. Wang, J. Turner, A. Gokaslan, V .-P. Berges, R. Mootaghi, O. Maksymets, A. X. Chang, M. Savva, A. Clegg, D. S. Chaplot, and D. Batra. Habitat challenge 2023.https://aihabitat.org/ challenge/2023/, 2023

    2023

  75. [83]

    Anderson, A

    P. Anderson, A. Chang, D. S. Chaplot, A. Dosovitskiy, S. Gupta, V . Koltun, J. Kosecka, J. Ma- lik, R. Mottaghi, M. Savva, et al. On evaluation of embodied navigation agents.arXiv preprint arXiv:1807.06757, 2018

    2018 arXiv

  76. [84]

    Datta, O

    S. Datta, O. Maksymets, J. Hoffman, S. Lee, D. Batra, and D. Parikh. Integrating egocen- tric localization for more realistic point-goal navigation agents. In J. Kober, F. Ramos, and C. Tomlin, editors,Proceedings of the 2020 Conference on Robot Learning, volume 155 of Proceed...

    2020

  77. [85]

    C. C. Kemp, A. Edsinger, H. M. Clever, and B. Matulevich. The design of stretch: A com- pact, lightweight mobile manipulator for indoor human environments, 2022. URLhttps: //arxiv.org/abs/2109.10892

    2022 arXiv

  78. [86]

    Unitree go1 product page.https://www.unitree.com/go1, 2025

    Unitree. Unitree go1 product page.https://www.unitree.com/go1, 2025. Accessed: 2025-05-01

    2025

  79. [87]

    S. Garg, N. Suenderhauf, and M. Milford. Lost? appearance-invariant place recognition for opposite viewpoints using visual semantics.Robotics: Science and Systems XIV, pages 1–10, 2018

    2018

  80. [88]

    Keetha, A

    N. Keetha, A. Mishra, J. Karhade, K. M. Jatavallabhula, S. Scherer, M. Krishna, and S. Garg. Anyloc: Towards universal visual place recognition.IEEE Robotics and Automation Letters, 9 (2):1286–1293, 2023

    2023

  81. [89]

    K. Garg, S. S. Puligilla, S. Kolathaya, M. Krishna, and S. Garg. Revisit anything: Visual place recognition via image segment retrieval. InEuropean Conference on Computer Vision (ECCV), September 2024

    2024

  82. [90]

    P. Foo, W. H. Warren, A. Duchon, and M. J. Tarr. Do humans integrate routes into a cognitive map? map-versus landmark-based navigation of novel shortcuts.Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(2):195, 2005

    2005

  83. [91]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PMLR, 2021

    2021

  84. [92]

    A. A. Hagberg, D. A. Schult, and P. J. Swart. Exploring network structure, dynamics, and function using networkx. InProceedings of the 7th Python in Science Conference, pages 11 – 15, Pasadena, CA USA, 2008. URLhttps://networkx.org/documentation/ stable/index.html. 22

    2008

  85. [93]

    Edstedt, Q

    J. Edstedt, Q. Sun, G. B ¨okman, M. Wadenb¨ack, and M. Felsberg. Roma: Robust dense fea- ture matching. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19790–19800, 2024. 23

    2024

  86. [2022]

    URLhttps://openreview.net/forum?id=UW5A3SweAH

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.