Pith. sign in

REVIEW 3 major objections 5 minor 82 references

A single RGB camera, no 3D map or trained controller, can drive a robot to object goals by fusing topological plans with terrain-aware metric control.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A navigation pipeline that bridges object-level global planning with traversability-aware local control, using only RGB images and pretrained models, improves success over prior zero-shot and learned baselines in simulation.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Novel integration with a plausible but unverified metric-depth assumption; worth a careful referee, not a desk reject. the 3 major comments →

arxiv 2509.08699 v1 pith:QRPP3WWQ submitted 2025-09-10 cs.RO cs.AIcs.CVcs.LGcs.SYeess.SY

TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goals

Stefan Podgorski , Sourav Garg , Mehdi Hosseinzadeh , Lachlan Mares , Feras Dayoub , Ian Reid This is my paper
classification cs.RO cs.AIcs.CVcs.LGcs.SYeess.SY
keywords topometric navigationobject-level topological maptraversability-aware controlBEV cost mapmonocular depthzero-shot navigationCLIP traversabilityinstance image goal navigation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a robot can perform long-horizon, open-set navigation toward object goals using a single RGB camera, with no 3D map, no learned controller, and no environment-specific training. The system couples an object-level topological global planner with a local metric planner: the global planner decides which visible object to approach based on graph connectivity to an (often distant) goal, and the local planner projects traversable image segments into a bird's-eye-view cost map using monocular depth, then plans a short collision-free path with Dijkstra's algorithm. In the zero-shot, no-ground-truth setting on the HM3D benchmark, the integrated system outperforms both a learned pixel-navigation controller and the topological controller on which it builds, across short, medium, and long trajectories, and the same modular pipeline is demonstrated on a real robot. The authors argue the method's practical benefit is deployability: it relies on foundation models that can be replaced as better ones appear, rather than on task-specific training.

Core claim

On the paper's own terms, the key result is that adding a traversability-aware metric controller on top of an object-level topological planner materially improves navigation success. In the no-ground-truth setting, success on easy (1-3m), hard (3-5m), and full (8-10m) trajectories is 61.76%, 43.14%, and 21.57%, versus 43.56%, 34.56%, and 13.73% for the topological-only baseline and 51.96%, 39.22%, and 14.0% for the learned pixel controller. The ablations attribute the larger remaining gap to perception—segment matching from the current view to the map—not to the metric controller: swapping in simulator ground-truth depth and traversability costs only about 5% on hard trajectories, while grou

What carries the argument

The central mechanism is the topometric controller: at every timestep it (1) segments the current RGB image into object masks, (2) labels each segment traversable or not with CLIP text queries (floor, ground, rug), (3) projects the traversable masks and the selected sub-goal segment into a metric bird's-eye-view (BEV) cost map using monocular depth from Depth-Anything, (4) selects the farthest projected point inside the lowest-cost sub-goal segment as the local target, and (5) plans a shortest path in the BEV cost map with Dijkstra's algorithm, driving a proportional yaw controller. This loop converts the topological planner's coarse 'head toward this object' signal into a continuous, obstac

Load-bearing premise

The system treats monocular depth from Depth-Anything as metric-scale depth without calibration, scale factor, or uncertainty model; if the network's relative depth cannot be converted to geometrically consistent metric coordinates across views, the BEV cost map and the selected sub-goal point will be wrong, breaking path planning and collision avoidance.

What would settle it

Run TANGO on the same HM3D episodes with simulator-provided depth in place of Depth-Anything while keeping every other component fixed. If the success rate rises by more than a few points (or if the Depth-Anything version exhibits collisions that the ground-truth-depth version avoids), then the metric controller's correctness depends on the monocular model's scale calibration. A complementary measurement: in real-world deployment, compare the planned BEV trajectory against a known obstacle layout and check whether planned paths ever clip walls or objects by more than the robot's radius.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Robots could be deployed in new indoor environments using only a single teach drive (or any object-level map) and a monocular camera, with no simulation training or domain-specific fine-tuning.
  • Because the stack is modular, improving any component—segmentation, depth, text-based traversability, or segment matching—should directly improve navigation without redesigning the controller.
  • The 'seen but unvisited' result suggests object-level topological maps encode more than a route: the same graph can support goals the robot has never approached, which would extend teach-and-repeat systems into open-vocabulary tasks.
  • The auto-switch design offers a general pattern: letting a myopic but reliable controller take over when the learned traversability signal is missing is a cheap way to improve robustness in tight spaces.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the monocular depth model's scale is inconsistent across viewpoints, the BEV cost map will be geometrically wrong; a direct test would be to substitute an onboard depth sensor or a SLAM-based scale correction while keeping everything else fixed and measure the resulting success and collision rates.
  • The ablation pattern (perception, not depth, drives the remaining 18% gap) suggests that future work should concentrate on open-set segment association—e.g., stronger visual-language matching—rather than on improving depth or traversability estimates.
  • The auto-switch mechanism hints at a broader principle: explicit uncertainty signals ('no traversable segment visible') can be used to trigger a fallback behavior, and this could be exported to other hybrid planners that blend learned and reactive control.
  • An immediate testable extension is to run the same pipeline on outdoor or off-road scenes with adapted text prompts (dirt, grass, gravel) to see whether the CLIP-based traversability filter keeps its open-set promise beyond indoor floors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents TANGO, a zero-shot RGB-only navigation pipeline that couples RoboHop's object-level topological graph planning with a local BEV traversability controller. At each step, SAM segments are assigned topological costs; CLIP filters traversable segments; Depth-Anything projects them into a BEV map; Dijkstra plans a path to the farthest point of the selected sub-goal segment; and an auto-switch falls back to RoboHop's segment servoing when no traversable region is detected. Experiments on 108 HM3D InstanceImageNav episodes across easy/hard/full trajectory lengths, with GT-Metric and GT-Topological ablations, compare TANGO against RoboHop and PixNav, plus a seen-but-unvisited goal evaluation and real-world demonstrations.

Significance. If the reported results are reproducible, TANGO is a useful modular alternative to end-to-end learned navigation: it is open-set, interpretable, and zero-shot, and the code release facilitates verification. The strongest evidence is the consistent improvement over RoboHop in the No-GT setting (Table I), and the seen-but-unvisited goal results (Table IV) extend teach-and-repeat beyond visited poses. However, the central 'metric local control' claim rests on an uncalibrated monocular depth source, and the performance differences are reported without uncertainty, so the current evidence is suggestive rather than conclusive.

major comments (3)
  1. [III-B] The paper refers to 'monocular metric depth' and a 'metric BEV traversability map' (Section III-B) and the central claim in Section I is that the robot 'metrically plans its motion'. However, the cited Depth-Anything (CVPR 2024) outputs affine-invariant, not metric, depth, and no calibration, scale factor, or uncertainty model is given. Since the controller selects the farthest projected point of the sub-goal segment and runs Dijkstra on the BEV cost map, unknown per-frame scale/shift can distort distances, obstacle extents, and the selected sub-goal point. Table II confounds changes in depth and traversability and uses a binary success metric, so it does not establish metric consistency. Please either add a scale calibration (e.g., camera height against a known floor plane) or explicitly reframe the controller as using relative depth and analyze what 'metric' then means.
  2. [Table I] All success rates in Table I are single point estimates from 108 episodes with no confidence intervals, error bars, or significance tests. In the No-GT setting, the hard-trajectory gap between TANGO (43.14) and RoboHop (34.56) is about 8.6 percentage points; with 108 episodes, the approximate binomial standard error is around 5 points, so this difference may not be significant. The full-trajectory gap (21.57 vs 13.73) is even less certain. Please report the number of successful episodes per cell, confidence intervals, or per-episode results, and perform a statistical comparison to support the headline outperformance claim.
  3. [IV-B and V-A] The PixNav baseline is given a non-standard protocol: it receives topologically planned sub-goals and updates them only when the model outputs 'Done' or its memory buffer is full, while TANGO and RoboHop use a continuous, fixed-camera controller. The paper acknowledges this in Section IV-B, but the abstract claims outperformance against 'existing state-of-the-art methods' without this qualification. The comparison against RoboHop is unaffected, but the PixNav result should be either made more faithful to PixNav's intended usage or explicitly framed as a protocol-specific comparison.
minor comments (5)
  1. [Abstract and III-B] Replace 'monocular metric depth' with 'monocular depth' or describe the calibration procedure; the current phrasing overstates what Depth-Anything provides.
  2. [Table III] The caption should state the setting (GT-metric, 250 steps) explicitly; the 'Improvement' column gives 11.64 but the table lists 62.14 and 73.78, so the meaning should be clarified.
  3. [V-D] The real-world tests are only mentioned as supplementary video. Add at least the robot platform, number of runs, and quantitative outcomes (success/failure and any failure modes).
  4. [Title page] Typos: 'Go als' in the title and 'contirbution' in the author footnote.
  5. [VI] The limitations paragraph lists perception, planning, and traversability errors, but not the monocular depth scale assumption. Please add it to the limitation list, or justify why it is not a limitation.

Circularity Check

0 steps flagged

No significant circularity: the paper reports empirical system performance; its only self-citation (RoboHop) is a published, externally consumable component, not an unverified premise that reduces the derivation to its inputs.

full rationale

The paper's load-bearing claims are empirical success rates (Table I) and ablations (Tables II–IV) rather than derivations from the method's own equations. The topological global planner is inherited from RoboHop, a prior peer-reviewed ICRA publication by overlapping authors, but TANGO consumes RoboHop's sub-goal masks as an input, and RoboHop is independently published and code-reproducible; this is dependency, not circularity. The local metric controller projects traversable segments and sub-goal points using Depth-Anything monocular depth (Section III-B); the lack of explicit depth scale calibration is a correctness/robustness limitation, not a circular step, because the depth values are not fitted to the success metric and the planning pipeline is evaluated against external goal-reaching criteria. Eq. 1 is the RoboHop fallback controller with fixed constants (τ=5, G=0.4); it is not fitted from TANGO's own results or used to predict the reported success. No equation in the paper defines the target claim in terms of its inputs, no fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is smuggled in via self-citation. The paper is self-contained as an empirical system integration paper, and the central comparison isolates control by providing all methods with the same goal information.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The system inherits the entire RoboHop topological mapper and combines it with several pretrained models. The genuinely new pieces are the traversability-aware BEV local planner and the auto-switch rule, both of which rest on the assumptions above. No new physical entities, forces, or conserved quantities are introduced.

free parameters (5)
  • CLIP traversability text prompts = 'floor', 'ground', 'rug'
    Hand-chosen open-vocabulary queries; no ablation across prompt sets; directly determines the binary traversability mask.
  • Controller gain G and softmax temperature tau = G=0.4, tau=5
    Inherited from RoboHop (Eq. 1), set by hand in prior work, not re-tuned here.
  • Auto-switch trigger threshold
    Defined only as 'absence of visible traversable regions'; no numerical criterion is given, making behavior sensitive to implementation.
  • BEV cost map smoothing parameters
    Distance transform and box filter are described, but kernel sizes and cost scales are not reported.
  • Sub-goal point selection heuristic = farthest projected point
    Choosing the farthest projected point in the sub-goal segment is a hand-selected rule; it may drive the robot behind obstacles.
axioms (4)
  • domain assumption Topological graph edge weights (intra-image 1, inter-image 0) adequately encode navigational proximity.
    Section III-A: Dijkstra uses these weights for sub-goal costs; objects co-visible in the same image are treated as one step apart, which may not hold geometrically.
  • ad hoc to paper Depth Anything monocular depth yields a metric-scale BEV projection without calibration.
    Section III-B projects segments into 3D using monocular depth and calls the result a 'metric BEV traversability map'; no scale factor or uncertainty model is given.
  • domain assumption CLIP text queries on SAM segments reliably identify traversable surfaces.
    Section III-B: traversability is binary, based on CLIP similarity to floor/ground/rug; the paper acknowledges errors in Section VI (limitations).
  • domain assumption A single teaching-run trajectory provides a topological graph sufficient for long-horizon goal reaching.
    Section IV-A: a map from one simulated shortest-path teach run is used for all inference; dynamic changes are not modeled except in a qualitative real-world demo.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goals." pith.science (2026). https://pith.science/paper/QRPP3WWQ

@misc{pith2026250908699,
  author       = {Pith},
  title        = {Pith review of: TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goals},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QRPP3WWQ}},
  note         = {Machine review of arXiv:2509.08699}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Visual navigation in robotics traditionally relies on globally-consistent 3D maps or learned controllers, which can be computationally expensive and difficult to generalize across diverse environments. In this work, we present a novel RGB-only, object-level topometric navigation pipeline that enables zero-shot, long-horizon robot navigation without requiring 3D maps or pre-trained controllers. Our approach integrates global topological path planning with local metric trajectory control, allowing the robot to navigate towards object-level sub-goals while avoiding obstacles. We address key limitations of previous methods by continuously predicting local trajectory using monocular depth and traversability estimation, and incorporating an auto-switching mechanism that falls back to a baseline controller when necessary. The system operates using foundational models, ensuring open-set applicability without the need for domain-specific fine-tuning. We demonstrate the effectiveness of our method in both simulated environments and real-world tests, highlighting its robustness and deployability. Our approach outperforms existing state-of-the-art methods, offering a more adaptable and effective solution for visual navigation in open-set environments. The source code is made publicly available: https://github.com/podgorki/TANGO.

Figures

Figures reproduced from arXiv: 2509.08699 by Feras Dayoub, Ian Reid, Lachlan Mares, Mehdi Hosseinzadeh, Sourav Garg, Stefan Podgorski.

Figure 1
Figure 1. Figure 1: We present a topometric navigation pipeline that uniquely bridges [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: TANGO’s Navigation Pipeline. Perception: The robot’s current view is segmented using a foundational segmentation model (SAM), the segments are localised within an object-level topological map using local feature matching (LightGlue). Each segment is assigned a cost based on its topological proximity to the final goal segment, the segment closest to the final goal is selected to drive the controller. Contro… view at source ↗
Figure 3
Figure 3. Figure 3: Seen-but-Unvisited Goals: An example episode with agent’s starting [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Unsuccessful sequence showing the controller navigating around [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 14 linked inside Pith

  1. [1]

    Orb-slam: a versatile and accurate monocular slam system,

    R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos, “Orb-slam: a versatile and accurate monocular slam system,”IEEE transactions on robotics, vol. 31, no. 5, pp. 1147–1163, 2015

    2015

  2. [2]

    Slam++: Simultaneous localisation and mapping at the level of objects,

    R. F. Salas-Moreno, R. A. Newcombe, H. Strasdat, P. H. Kelly, and A. J. Davison, “Slam++: Simultaneous localisation and mapping at the level of objects,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 1352–1359

    2013

  3. [3]

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning,

    Q. Gu, A. Kuwajerwala, S. Morin, K. Jatavallabhula, B. Sen, A. Agar- wal, C. Rivera, W. Paul, K. Ellis, R. Chellappa, C. Gan, C. de Melo, J. Tenenbaum, A. Torralba, F. Shkurti, and L. Paull, “Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning,” in arXiv, 2023

    2023

  4. [4]

    Neu- ral topological slam for visual navigation,

    D. S. Chaplot, R. Salakhutdinov, A. Gupta, and S. Gupta, “Neu- ral topological slam for visual navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12 875–12 884

    2020

  5. [5]

    Learning navigational visual representations with semantic map supervision,

    Y . Hong, Y . Zhou, R. Zhang, F. Dernoncourt, T. Bui, S. Gould, and H. Tan, “Learning navigational visual representations with semantic map supervision,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3055–3067

    2023

  6. [6]

    Semi-parametric topolog- ical memory for navigation,

    N. Savinov, A. Dosovitskiy, and V . Koltun, “Semi-parametric topolog- ical memory for navigation,”arXiv preprint arXiv:1803.00653, 2018

    Pith/arXiv arXiv 2018

  7. [7]

    Gnm: A general navigation model to drive any robot,

    D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine, “Gnm: A general navigation model to drive any robot,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 7226–7233

    2023

  8. [8]

    Robohop: Segment-based topological map representation for open-world visual navigation,

    S. Garg, K. Rana, M. Hosseinzadeh, L. Mares, N. Suenderhauf, F. Dayoub, and I. Reid, “Robohop: Segment-based topological map representation for open-world visual navigation,” in2024 International Conference on Robotics and Automation (ICRA). IEEE, 2024

    2024

  9. [9]

    A hierarchical dual model of environment-and place-specific utility for visual place recognition,

    N. V . Keetha, M. Milford, and S. Garg, “A hierarchical dual model of environment-and place-specific utility for visual place recognition,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 6969–6976, 2021

    2021

  10. [10]

    Re- visit anything: Visual place recognition via image segment retrieval,

    K. Garg, S. S. Puligilla, S. Kolathaya, M. Krishna, and S. Garg, “Re- visit anything: Visual place recognition via image segment retrieval,” inEuropean Conference on Computer Vision (ECCV), September 2024

    2024

  11. [11]

    Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill,

    W. Cai, S. Huang, G. Cheng, Y . Long, P. Gao, C. Sun, and H. Dong, “Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 5228–5234

    2024

  12. [12]

    Fast segment anything,

    X. Zhao, W. Ding, Y . An, Y . Du, T. Yu, M. Li, M. Tang, and J. Wang, “Fast segment anything,” 2023

    2023

  13. [13]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Gir- shick, “Segment anything,” 2023

    2023

  14. [14]

    Depth anything: Unleashing the power of large-scale unlabeled data,

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” in CVPR, 2024

    2024

  15. [15]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  16. [16]

    Visual language maps for robot navigation,

    C. Huang, O. Mees, A. Zeng, and W. Burgard, “Visual language maps for robot navigation,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), London, UK, 2023

    2023

  17. [17]

    Kimera: From slam to spatial perception with 3d dynamic scene graphs,

    A. Rosinol, A. Violette, M. Abate, N. Hughes, Y . Chang, J. Shi, A. Gupta, and L. Carlone, “Kimera: From slam to spatial perception with 3d dynamic scene graphs,”The International Journal of Robotics Research, vol. 40, no. 12-14, pp. 1510–1546, 2021

    2021

  18. [18]

    Learning to map for active semantic goal navigation,

    G. Georgakis, B. Bucher, K. Schmeckpeper, S. Singh, and K. Dani- ilidis, “Learning to map for active semantic goal navigation,”arXiv preprint arXiv:2106.15648, 2021

    Pith/arXiv arXiv 2021

  19. [19]

    3d-aware object goal navigation via simultaneous exploration and identification,

    J. Zhang, L. Dai, F. Meng, Q. Fan, X. Chen, K. Xu, and H. Wang, “3d-aware object goal navigation via simultaneous exploration and identification,” inProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2023, pp. 6672–6682

    2023

  20. [20]

    Zero-shot object goal visual navigation,

    Q. Zhao, L. Zhang, B. He, H. Qiao, and Z. Liu, “Zero-shot object goal visual navigation,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 2025–2031

    2023

  21. [21]

    Can an embodied agent find your “cat-shaped mug

    V . S. Dorbala, J. F. Mullen Jr, and D. Manocha, “Can an embodied agent find your “cat-shaped mug”? llm-based zero-shot object naviga- tion,”IEEE Robotics and Automation Letters, 2023

    2023

  22. [22]

    Object goal navigation using goal-oriented semantic exploration,

    D. S. Chaplot, D. P. Gandhi, A. Gupta, and R. R. Salakhutdinov, “Object goal navigation using goal-oriented semantic exploration,” Advances in Neural Information Processing Systems, vol. 33, pp. 4247–4258, 2020

    2020

  23. [23]

    Monocular-slam–based navigation for autonomous micro helicopters in gps-denied environ- ments,

    S. Weiss, D. Scaramuzza, and R. Siegwart, “Monocular-slam–based navigation for autonomous micro helicopters in gps-denied environ- ments,”Journal of Field Robotics, vol. 28, no. 6, pp. 854–874, 2011

    2011

  24. [24]

    Topological Semantic Graph Memory for Image Goal Navigation,

    N. Kim, O. Kwon, H. Yoo, Y . Choi, J. Park, and S. Oh, “Topological Semantic Graph Memory for Image Goal Navigation,” inCoRL, 2022

    2022

  25. [25]

    Viking: Vision-based kilometer-scale naviga- tion with geographic hints,

    D. Shah and S. Levine, “Viking: Vision-based kilometer-scale naviga- tion with geographic hints,”arXiv preprint arXiv:2202.11271, 2022

    Pith/arXiv arXiv 2022

  26. [26]

    LM-nav: Robotic navigation with large pre-trained models of language, vision, and action,

    D. Shah, B. Osinski, B. Ichter, and S. Levine, “LM-nav: Robotic navigation with large pre-trained models of language, vision, and action,” in6th Annual Conference on Robot Learning, 2022. [Online]. Available: https://openreview.net/forum?id=UW5A3SweAH

    2022

  27. [27]

    Nomad: Goal masked diffusion policies for navigation and exploration,

    A. Sridhar, D. Shah, C. Glossop, and S. Levine, “Nomad: Goal masked diffusion policies for navigation and exploration,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 63–70

    2024

  28. [28]

    Learning view and target invariant visual servoing for navigation,

    Y . Li and J. Ko ˇsecka, “Learning view and target invariant visual servoing for navigation,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 658–664

    2020

  29. [29]

    Scaling local control to large-scale topological navigation,

    X. Meng, N. Ratliff, Y . Xiang, and D. Fox, “Scaling local control to large-scale topological navigation,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 672–678

    2020

  30. [30]

    Spoc: Imitating short- est paths in simulation enables effective navigation and manipulation in the real world,

    K. Ehsani, T. Gupta, R. Hendrix, J. Salvador, L. Weihs, K.-H. Zeng, K. P. Singh, Y . Kim, W. Han, A. Herrasti,et al., “Spoc: Imitating short- est paths in simulation enables effective navigation and manipulation in the real world,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 238–16 250

    2024

  31. [31]

    Trajectory servoing: Image- based trajectory tracking using slam

    S. Feng, Z. Wu, Y . Zhao, and P. A. Vela, “Trajectory servoing: Image- based trajectory tracking using slam.”CoRR, 2021

    2021

  32. [32]

    Appearance-based indoor navigation by ibvs using line segments,

    S. R. Bista, P. R. Giordano, and F. Chaumette, “Appearance-based indoor navigation by ibvs using line segments,”IEEE robotics and automation letters, vol. 1, no. 1, pp. 423–430, 2016

    2016

  33. [33]

    Path planning for robust image-based control,

    Y . Mezouar and F. Chaumette, “Path planning for robust image-based control,”IEEE transactions on robotics and automation, vol. 18, no. 4, pp. 534–549, 2002

    2002

  34. [34]

    A tutorial on visual servo control,

    S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo control,”IEEE transactions on robotics and automation, vol. 12, no. 5, pp. 651–670, 1996

    1996

  35. [35]

    Visual servoing for path reaching with nonholonomic robots,

    A. Cherubini, F. Chaumette, and G. Oriolo, “Visual servoing for path reaching with nonholonomic robots,”Robotica, vol. 29, no. 7, pp. 1037–1048, 2011

    2011

  36. [36]

    Visual servoing- based navigation for monitoring row-crop fields,

    A. Ahmadi, L. Nardi, N. Chebrolu, and C. Stachniss, “Visual servoing- based navigation for monitoring row-crop fields,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 4920–4926

    2020

  37. [37]

    3d navigation based on a visual memory,

    A. Remazeilles, F. Chaumette, and P. Gros, “3d navigation based on a visual memory,” inProceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006.IEEE, 2006, pp. 2719–2725

    2006

  38. [38]

    Experimental evaluation of autonomous driving based on visual memory and image- based visual servoing,

    A. Diosi, S. Segvic, A. Remazeilles, and F. Chaumette, “Experimental evaluation of autonomous driving based on visual memory and image- based visual servoing,”IEEE Transactions on Intelligent Transporta- tion Systems, vol. 12, no. 3, pp. 870–883, 2011

    2011

  39. [39]

    Indoor navigation of a wheeled mobile robot along visual routes,

    G. Blanc, Y . Mezouar, and P. Martinet, “Indoor navigation of a wheeled mobile robot along visual routes,” inProceedings of the 2005 IEEE international conference on robotics and automation. IEEE, 2005, pp. 3354–3359

    2005

  40. [40]

    Learning exploration policies for navigation,

    T. Chen, S. Gupta, and A. Gupta, “Learning exploration policies for navigation,” inInternational Conference on Learning Representations,

  41. [41]

    Learning object-conditioned exploration using distributed soft actor critic,

    A. Wahid, A. Stone, K. Chen, B. Ichter, and A. Toshev, “Learning object-conditioned exploration using distributed soft actor critic,” in Conference on Robot Learning. PMLR, 2021, pp. 1684–1695

    2021

  42. [42]

    Bruce, N

    J. Bruce, N. S ¨underhauf, P. Deepmind, London, R. Deepmind, and M. Milford,Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal. [Online]. Available: http://proceedings.mlr.press/v87/bruce18a/bruce18a.pdf

  43. [43]

    Generalizable imitation learning from observation via inferring goal proximity,

    Y . Lee, A. Szot, S.-H. Sun, and J. J. Lim, “Generalizable imitation learning from observation via inferring goal proximity,” in Advances in Neural Information Processing Systems, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., 2021. [Online]. Available: https://openreview.net/forum?id=lp9foO8AFoD

    2021

  44. [44]

    Pirlnav: Pretraining with imitation and rl finetuning for objectnav,

    R. Ramrakhya, D. Batra, E. Wijmans, and A. Das, “Pirlnav: Pretraining with imitation and rl finetuning for objectnav,”2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2023. [Online]. Available: http://dx.doi.org/10.1109/ CVPR52729.2023.01716

    arXiv 2023

  45. [45]

    Rethinking closed-loop training for autonomous driving,

    C. Zhang, R. Guo, W. Zeng, Y . Xiong, B. Dai, R. Hu, M. Ren, and R. Urtasun, “Rethinking closed-loop training for autonomous driving,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 264–282

    2022

  46. [46]

    Planning-oriented autonomous driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang,et al., “Planning-oriented autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 853–17 862

    2023

  47. [47]

    V-strong: Visual self-supervised traversability learning for off-road navigation,

    S. Jung, J. Lee, X. Meng, B. Boots, and A. Lambert, “V-strong: Visual self-supervised traversability learning for off-road navigation,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 1766–1773

    2024

  48. [48]

    Special issue on future challenges and opportunities in vision-based drone navigation

    G. Loianno and D. Scaramuzza, “Special issue on future challenges and opportunities in vision-based drone navigation.”Journal of Field Robotics, vol. 37, no. 4, 2020

    2020

  49. [49]

    Air learning: a deep reinforcement learning gym for autonomous aerial robot visual navigation,

    S. Krishnan, B. Boroujerdian, W. Fu, A. Faust, and V . J. Reddi, “Air learning: a deep reinforcement learning gym for autonomous aerial robot visual navigation,”Machine Learning, vol. 110, no. 9, pp. 2501– 2540, 2021

    2021

  50. [50]

    Learn- ing visuomotor policies for aerial navigation using cross-modal repre- sentations,

    R. Bonatti, R. Madaan, V . Vineet, S. Scherer, and A. Kapoor, “Learn- ing visuomotor policies for aerial navigation using cross-modal repre- sentations,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 1637–1644

    2020

  51. [51]

    Zson: Zero-shot object-goal navigation using multimodal goal embed- dings,

    A. Majumdar, G. Aggarwal, B. Devnani, J. Hoffman, and D. Batra, “Zson: Zero-shot object-goal navigation using multimodal goal embed- dings,”Advances in Neural Information Processing Systems, vol. 35, pp. 32 340–32 352, 2022

    2022

  52. [52]

    Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation,

    S. Y . Gadre, M. Wortsman, G. Ilharco, L. Schmidt, and S. Song, “Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23 171–23 181

    2023

  53. [53]

    Goat: Go to any thing,

    M. Chang, T. Gervet, M. Khanna, S. Yenamandra, D. Shah, S. Y . Min, K. Shah, C. Paxton, S. Gupta, D. Batra,et al., “Goat: Go to any thing,” arXiv preprint arXiv:2311.06430, 2023

    Pith/arXiv arXiv 2023

  54. [54]

    Goat-bench: A benchmark for multi-modal lifelong navigation,

    M. Khanna, R. Ramrakhya, G. Chhablani, S. Yenamandra, T. Gervet, M. Chang, Z. Kira, D. S. Chaplot, D. Batra, and R. Mottaghi, “Goat-bench: A benchmark for multi-modal lifelong navigation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 373–16 383

    2024

  55. [55]

    Navgpt: Explicit reasoning in vision- and-language navigation with large language models,

    G. Zhou, Y . Hong, and Q. Wu, “Navgpt: Explicit reasoning in vision- and-language navigation with large language models,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 7, 2024, pp. 7641–7649

    2024

  56. [56]

    Navgpt-2: Unleashing navigational reasoning capability for large vision-language models,

    G. Zhou, Y . Hong, Z. Wang, X. E. Wang, and Q. Wu, “Navgpt-2: Unleashing navigational reasoning capability for large vision-language models,”arXiv preprint arXiv:2407.12366, 2024

    Pith/arXiv arXiv 2024

  57. [57]

    Mapgpt: Map-guided prompting for unified vision-and-language nav- igation,

    J. Chen, B. Lin, R. Xu, Z. Chai, X. Liang, and K.-Y . K. Wong, “Mapgpt: Map-guided prompting for unified vision-and-language nav- igation,”arXiv preprint arXiv:2401.07314, 2024

    Pith/arXiv arXiv 2024

  58. [58]

    Visiongpt: Llm-assisted real-time anomaly detection for safe visual navigation,

    H. Wang, J. Qin, A. Bastola, X. Chen, J. Suchanek, Z. Gong, and A. Razi, “Visiongpt: Llm-assisted real-time anomaly detection for safe visual navigation,”arXiv preprint arXiv:2403.12415, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Navid: Video-based vlm plans the next step for vision-and-language navigation,

    J. Zhang, K. Wang, R. Xu, G. Zhou, Y . Hong, X. Fang, Q. Wu, Z. Zhang, and W. He, “Navid: Video-based vlm plans the next step for vision-and-language navigation,”arXiv preprint arXiv:2402.15852, 2024

    Pith/arXiv arXiv 2024

  60. [60]

    Mobility vla: Multimodal instruction navigation with long-context vlms and topological graphs,

    H.-T. L. Chiang, Z. Xu, Z. Fu, M. G. Jacob, T. Zhang, T.-W. E. Lee, W. Yu, C. Schenck, D. Rendleman, D. Shah,et al., “Mobility vla: Multimodal instruction navigation with long-context vlms and topological graphs,”arXiv preprint arXiv:2407.07775, 2024

    Pith/arXiv arXiv 2024

  61. [61]

    Pivot: Iterative visual prompting elicits actionable knowledge for vlms,

    S. Nasiriany, F. Xia, W. Yu, T. Xiao, J. Liang, I. Dasgupta, A. Xie, D. Driess, A. Wahid, Z. Xu,et al., “Pivot: Iterative visual prompting elicits actionable knowledge for vlms,”arXiv preprint arXiv:2402.07872, 2024

    Pith/arXiv arXiv 2024

  62. [62]

    Convoi: Context-aware navigation using vision language models in outdoor and indoor en- vironments,

    A. J. Sathyamoorthy, K. Weerakoon, M. Elnoor, A. Zore, B. Ichter, F. Xia, J. Tan, W. Yu, and D. Manocha, “Convoi: Context-aware navigation using vision language models in outdoor and indoor en- vironments,”arXiv preprint arXiv:2403.15637, 2024

    Pith/arXiv arXiv 2024

  63. [63]

    Visual teach and repeat for long-range rover autonomy,

    P. Furgale and T. D. Barfoot, “Visual teach and repeat for long-range rover autonomy,”Journal of field robotics, vol. 27, no. 5, pp. 534–560, 2010

    2010

  64. [64]

    A mapping and localization framework for scalable appearance-based navigation,

    S. ˇSegvi´c, A. Remazeilles, A. Diosi, and F. Chaumette, “A mapping and localization framework for scalable appearance-based navigation,” Computer Vision and Image Understanding, vol. 113, no. 2, pp. 172– 187, 2009

    2009

  65. [65]

    Robust appearance based visual route following for navigation in large-scale outdoor environments,

    A. M. Zhang and L. Kleeman, “Robust appearance based visual route following for navigation in large-scale outdoor environments,”The International Journal of Robotics Research, vol. 28, no. 3, pp. 331– 356, 2009

    2009

  66. [66]

    Fast and robust bio- inspired teach and repeat navigation,

    D. Dall’Osto, T. Fischer, and M. Milford, “Fast and robust bio- inspired teach and repeat navigation,” in2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021, pp. 500–507

    2021

  67. [67]

    An efficient locally reactive controller for safe navigation in visual teach and repeat missions,

    M. Mattamala, N. Chebrolu, and M. Fallon, “An efficient locally reactive controller for safe navigation in visual teach and repeat missions,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2353–2360, 2022

    2022

  68. [68]

    Navigation without localisation: reliable teach and repeat based on the convergence theorem,

    T. Krajn ´ık, F. Majer, L. Halodov ´a, and T. Vintr, “Navigation without localisation: reliable teach and repeat based on the convergence theorem,” in2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 1657–1664

    2018

  69. [69]

    Predictive and adaptive maps for long- term visual navigation in changing environments,

    L. Halodov ´a, E. Dvo ˇrr´akov´a, F. Majer, T. Vintr, O. M. Mozos, F. Dayoub, and T. Krajn ´ık, “Predictive and adaptive maps for long- term visual navigation in changing environments,” in2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2019, pp. 7033–7039

    2019

  70. [70]

    High-speed autonomous quadrotor navigation through visual and inertial paths,

    T. Do, L. C. Carrillo-Arce, and S. I. Roumeliotis, “High-speed autonomous quadrotor navigation through visual and inertial paths,” The International Journal of Robotics Research, vol. 38, no. 4, pp. 486–504, 2019

    2019

  71. [71]

    Image features for visual teach-and-repeat navigation in changing environments,

    T. Krajn ´ık, P. Crist ´oforis, K. Kusumam, P. Neubert, and T. Duckett, “Image features for visual teach-and-repeat navigation in changing environments,”Robotics and Autonomous Systems, vol. 88, pp. 127– 141, 2017

    2017

  72. [72]

    Image-based indoor topological navigation with collision avoidance for resource-constrained mobile robots,

    S. R. Bista, B. Ward, and P. Corke, “Image-based indoor topological navigation with collision avoidance for resource-constrained mobile robots,”Journal of Intelligent & Robotic Systems, vol. 102, no. 3, p. 55, 2021

    2021

  73. [73]

    Looking high and low: Learning place- dependent gaussian mixture height models for terrain assessment,

    L.-P. Berczi and T. D. Barfoot, “Looking high and low: Learning place- dependent gaussian mixture height models for terrain assessment,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 3918–3925

    2017

  74. [74]

    Learning robotic navigation from experience: principles, methods and recent results,

    S. Levine and D. Shah, “Learning robotic navigation from experience: principles, methods and recent results,”Philosophical Transactions of the Royal Society B, vol. 378, no. 1869, p. 20210447, 2023

    2023

  75. [75]

    Vint: A large-scale, multi-task visual navigation backbone with cross-robot generalization,

    D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine, “Vint: A large-scale, multi-task visual navigation backbone with cross-robot generalization,” in7th Annual Conference on Robot Learning, 2023

    2023

  76. [76]

    Superpoint: Self- supervised interest point detection and description,

    D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self- supervised interest point detection and description,” inProceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, pp. 224–236

    2018

  77. [77]

    LightGlue: Local Feature Matching at Light Speed,

    P. Lindenberger, P.-E. Sarlin, and M. Pollefeys, “LightGlue: Local Feature Matching at Light Speed,” inICCV, 2023

    2023

  78. [78]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” 2021. [Online]. Available: https: //arxiv.org/abs/2103.00020

    Pith/arXiv arXiv 2021

  79. [79]

    Habitat- matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai,

    S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. Turner, E. Undersander, W. Galuba, A. Westbury, A. X. Chang, M. Savva, Y . Zhao, and D. Batra, “Habitat- matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai,”ArXiv, vol. abs/2109.08238, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:237563216

    Pith/arXiv arXiv 2021

  80. [80]

    Instance- specific image goal navigation: Training embodied agents to find object instances,

    J. Krantz, S. Lee, J. Malik, D. Batra, and D. S. Chaplot, “Instance- specific image goal navigation: Training embodied agents to find object instances,”arXiv preprint arXiv:2211.15876, 2022

    Pith/arXiv arXiv 2022

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.