Pith. sign in

REVIEW 4 major objections 5 minor 37 references

Narrate2Nav: Real-Time Visual Navigation with Implicit Language Reasoning in Human-Centric Environments

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Narrate2Nav claims that a 29M-parameter RGB-only model can carry the social reasoning of a large vision-language model by distilling narrations into a visual encoder during training.

desk verdict Plausible new distillation recipe for language-informed navigation, but the evidence that language reasoning transfers through Barlow Twins is not yet solid; the paper deserves serious peer review with major revisions. read the letter →

arxiv 2506.14233 v1 pith:5PI5U3A4 submitted 2025-06-17 cs.RO

Amirreza Payandeh , Anuj Pokhrel , Daeun Song , Marcos Zampieri , Xuesu Xiao This is my paper
classification cs.RO
keywords VisualnavigationVision-language-actionmodelBarlowTwinsSelf-supervisedrepresentationlearningSocialrobotChain-of-thoughtreasoningRGB-onlyHuman-centricenvironments
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the social and language reasoning normally supplied by a large vision-language model at inference time can be moved into a small visual encoder during training, so a robot can navigate crowded human environments from RGB images alone in real time. Narrate2Nav trains a roughly 330M-parameter teacher on images, future waypoints, and natural-language narrations of scene context, human intentions, and chain-of-thought reasoning, then distills the teacher's future-state representation into a roughly 29M-parameter ResNet-50 student using the Barlow Twins redundancy-reduction loss. On an unseen offline dataset the paper reports 52.94% lower average displacement error than the next best baseline, and in real-world trials the model reaches the goal in more trials with fewer collisions than GNM, ViNT, and NoMaD. The significance, if the claim holds, is that language-grounded social navigation need not run a large language model on the robot; the capability can be compressed into the weights of a vision encoder.

What carries the argument

The load-bearing mechanism is the Barlow Twins loss applied to projected context tokens rather than to augmented image pairs. With $C_{ij}$ the cross-correlation between normalized past-pixel embeddings $Z_{\text{past}}$ from the student and future multi-modal embeddings $Z_{\text{future}}$ from the teacher, the loss is $\mathcal{L}_{BT} = \sum_i (1 - C_{ii})^2 + \lambda \sum_i \sum_{j \neq i} C_{ij}^2$, forcing the student's context token to match the teacher's token on the diagonal while decorrelating off-diagonal components. Since the teacher's token already mixes vision, action, and language, this alignment is the conduit through which language-derived social cues enter the student's latent space. Supporting machinery includes early fusion of action and observation embeddings into shared tokens, learned positional encodings, stochastic goal masking for half of the training instances, and a two-stage training schedule of hundreds of epochs.

What would settle it

Train an identical student against a teacher whose narrations are removed or scrambled while future images and actions are kept the same; if offline errors and attention maps stay within noise of the text-conditioned teacher, the language signal is not what drives the improvement. A complementary check is to train linear probes on the student's frozen context tokens to predict the presence and direction of nearby pedestrians, where chance-level accuracy would indicate that no social information survived distillation.

Watch

Extended reading notes

Core claim

The central claim is that textual narrations, used only as a training-time modality, can reshape a visual encoder's attention so that RGB-only inference carries human-like reasoning about who will move, where it is safe to pass, and when to yield. Each state is first encoded into a unified multi-modal token: RGB features from a frozen DINOv2, projected future waypoints, and a CLIP-encoded narration describing perception, prediction, and chain-of-thought justifications. A cross-modal transformer teacher is trained end-to-end to predict the next trajectory from these tokens, and then the teacher is frozen and the student's context token is aligned to the teacher's future multi-modal context token through the Barlow Twins loss. At deployment the student sees five past RGB frames and a 2D goal point and outputs five future waypoints, which a Pure Pursuit controller tracks. The paper attributes the resulting attention on people and obstacles blocking the path to the textual signal embedded during pretraining.

Load-bearing premise

The load-bearing premise is that aligning the student's context token to the teacher's future multi-modal context token transfers language-derived reasoning specifically, and not just generic correlations between past pixels and future actions or appearances.

Editorial extensions

If this is right

  • If the claim holds, a robot can navigate socially without running a large vision-language model at inference, because the teacher's reasoning is embedded in a small student that only consumes RGB images and a goal point.
  • The same two-stage distillation could transfer any auxiliary modality available during training, such as depth, LiDAR, or scene graphs, into an RGB-only policy.
  • Attention maps that concentrate on people and obstacles could make failure analysis and human-robot trust easier, since the model's focus becomes interpretable.
  • Natural-language distance estimates act as weak 3D information, reducing reliance on LiDAR while keeping spatial awareness in RGB-only deployment.
  • Comparisons with large-VLM navigation systems should be made on real-time RGB-only terms, because the paper's contribution is precisely to avoid token-space reasoning at run time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ablation pattern, ADE worsens by about 16 percent while FDE is unchanged, suggests the narration mainly improves mid-trajectory social avoidance rather than final goal accuracy; the paper does not draw this distinction.
  • A sharper test of whether the transferred signal is reasoning rather than future-action correlation would be to train the teacher on narrations that contradict the observed future, such as captions saying a pedestrian will stop when the person actually walks, and check whether the student follows the text or the trajectory.
  • The recipe generalizes in principle to other embodied tasks, such as manipulation or inspection, wherever offline demonstrations can be annotated with text and the deployed policy must act from sensors alone in real time.
  • The paper's Limitations section leaves open cross-embodiment and robustness questions, so the practical reach of the method is not yet established beyond the reported proof of concept.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Narrate2Nav, a two-stage vision-action model for social robot navigation. In Stage 1, a large teacher model is trained end-to-end on RGB observations, waypoint trajectories, and VLM-generated textual narrations (including chain-of-thought reasoning) to predict future trajectories. In Stage 2, a lightweight student model is trained with a Barlow Twins redundancy-reduction loss to align its past-only context token with the teacher's future multimodal context token, with the aim of distilling language-based reasoning into the student's visual encoder. At inference, the student takes only RGB history and a 2D goal and outputs low-level waypoints. The authors report a 52.94% lower average displacement error offline and a 41.67% higher goal-reaching success in real-world trials relative to the next best baseline, together with qualitative attention-map evidence.

Significance. If the central claim holds, the contribution is significant: it would demonstrate that a small (~29M parameter) RGB-only navigation policy can carry implicit language-based social reasoning through self-supervised latent alignment, avoiding the inference-time cost of large VLMs. The paper ships a fairly extensive evaluation on the SCAND dataset and a real-robot deployment across four scenarios, and it explicitly targets a timely problem in vision-language-action models. The offline results, if reproducible, are strong. However, the load-bearing causal claim—that the Barlow Twins alignment transfers language-derived reasoning rather than only future-action statistics—is not established by the current experiments, and the real-world evaluation has selective reporting that undercuts the headline improvement. The paper is honest about some of these gaps in its Limitations section, which is a credit, but the central interpretation currently outruns the evidence.

major comments (4)
  1. [§3.2.3, Eq. (Z_final)] The student input formula is written as Z_final = [reg, z_{t−τP}, ..., z_t, \hat{t}, ctx] + P, and \hat{t} is defined in §3.2.2 as the CLIP embedding of the text narration. This directly contradicts the stated RGB-only student input in the same subsection and in Figure 2. If the text token is actually fed to the student during pretraining, the inference-time RGB-only claim is misleading. If it is a typo, the formula must be corrected and the text clearly excluded from the student input. As written, the paper does not support the claim that the student's visual encoder alone receives only visual and goal information.
  2. [§3.2.2 and §4.4] The central claim that language reasoning is embedded in the student's visual encoder is not supported by the provided evidence. In §3.2.2, the teacher is trained with the same trajectory as both input and output ('the input trajectory and the output trajectory are the same'), so its context token can encode the future action sequence almost by copying, and the Barlow Twins loss in §3.2.3 could align the student with a generic past-to-future action predictor. The text ablation in Table 3 removes the entire text encoder, changing ADE by 0.03 (0.19 vs 0.16) and leaving FDE unchanged at 0.24; it does not vary text content by shuffling, corrupting, or replacing narrations, nor does it probe the student's context token for language-derived information. Without such controls, the improvement over the no-text ablation can be attributed to the mere presence of an extra input token rather than to semantic chain-of-thought reasoning.
  3. [§4.3, Table 2] The real-world evaluation is selectively reported in a way that weakens the headline 41.67% improvement. The Navigation in Crowds scenario is omitted because 'all models, including Narrate2Nav, exhibited degraded performance,' and in the Narrow Passageway scenario Narrate2Nav has 5/10 collisions and 4/10 success, which is worse than GNM (5/10 collisions, 9/10 success) and ViNT (7/10 collisions, 10/10 success) on success rate. The aggregate 'All' column therefore reflects only scenarios chosen after seeing the results, not a pre-registered protocol. The paper should report all attempted scenarios, including the failures, and the aggregate claim should be recomputed or qualified accordingly.
  4. [§4.4, Table 3] The ablation claim of a '14.8% drop' is not clearly defined, and it is not supported by the numbers shown. Table 3 reports AOE 0.06 vs 0.04, ADE 0.19 vs 0.16, and FDE 0.24 vs 0.24; the change in ADE is 18.75% and in AOE 33%, while FDE is unchanged. The paper should state exactly which metric the 14.8% refers to, report variance or significance tests across multiple seeds or evaluation splits, and acknowledge that the FDE result is unchanged, which is not consistent with the strong text-effectiveness interpretation.
minor comments (5)
  1. [§4.1, Table 1] The 'All' column is described as a broader set of unlabeled samples rather than an average of scenario columns, but the number of such samples and their relationship to the scenario-specific test sets is not specified; this should be clarified for reproducibility.
  2. [§3.2.3] The notation τP and τf is introduced but not precisely defined: τP denotes past frames and τf future frames, yet the text uses both 'n' and 'τ' inconsistently in §3.2.1–§3.2.3. The paper should unify the notation and state the exact ranges (e.g., τP∈[−5,0], τf∈[1,5]).
  3. [§4.2, Figure 4] The qualitative attention maps are presented for two scenarios only, and the paper states that 'all SoTA models generate trajectories that collide with them,' but Table 2 shows that baselines achieve high success in some real-world trials. The qualitative claim would be stronger with more examples and with a quantitative attention-grounding metric.
  4. [§6, Limitations] The Limitations section is candid, but it also states that 'further testing is required to fully verify emergent reasoning behaviors.' This sentence should be elevated to a prominently placed caveat in the abstract or introduction, because the headline claim of implicit language reasoning is currently presented as established in the abstract.
  5. [General] The paper does not mention whether code, trained models, or the newly collected dataset will be released. Given the reported sensitivity to prompt design and scenario choice, a release (or at least a detailed implementation checklist) would materially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are held-out evaluations, and the distillation design is not an input-to-prediction identity.

full rationale

The paper's headline metrics (52.94% offline ADE improvement and 41.67% real-world success improvement) are measured on an unseen offline dataset and in new physical trials against external baselines (GNM, ViNT, NoMaD, CityWalker), so no fitted parameter is being relabeled as a prediction. The teacher's use of the same future trajectory as input and output (Section 3.2.2: 'the input trajectory and the output trajectory are the same') is a representation-learning and distillation device, not a circular result: the student is evaluated at inference without the teacher, without future frames, and without text, so the held-out prediction is not forced by construction. The text ablation in Section 4.4 removes the text encoder and reports a performance drop, providing some independent grounding for the language-modality claim, although it does not isolate semantic content from token presence; the Limitations section explicitly disclaims verified reasoning ('Further testing is required to fully verify emergent reasoning behaviors'), which further shows the reasoning claim is an interpretation rather than a derivation. Self-citations (Social-LLaVA, VAMP, CAHSOR, SCAND) are background, dataset, or methodology references and are not load-bearing for the main result. One consistency issue—the Stage 2 student input formula in Section 3.2.3 lists a text token \hat{t} despite the RGB-only student description—is a correctness or typo concern, not circularity.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claims rely on several unverified assumptions: the quality of VLM-generated narrations, the transfer of reasoning through Barlow Twins, the teacher context token containing useful future information, and the fairness of private baselines and datasets. These are domain assumptions rather than physical entities, and no new objects are invented.

free parameters (4)
  • Barlow Twins weighting factor lambda
    Balances the invariance and redundancy-reduction terms in the pretraining loss. No value is reported, and the central distillation claim depends on this balance.
  • Context/future frame horizon tau = 5
    The paper sets tau_P = tau_f = 5 frames (2.5 seconds) for both the student history and teacher future context. This horizon determines what future state information is distilled.
  • Goal masking probability = 0.5
    The goal is replaced with a learned embedding for 50% of training instances to support both goal-directed and exploration behaviors. This ratio is chosen by hand and affects the policy.
  • Learning rate and training epochs = 2e-4, 267 and 747 epochs
    AdamW learning rate and stage durations are chosen without reported sensitivity analysis. They are not fit to the evaluation metric in the paper, but they influence the final model.
assumptions (6)
  • domain assumption Text narrations generated by Qwen2-VL-72B are accurate and informative enough to act as a useful training signal.
    Section 3.2.1 selects Qwen2-VL for narrations, but the Limitations section admits that the generated dataset contains inaccurate descriptions. The central method depends on the overall quality of these text labels.
  • ad hoc to paper Barlow Twins alignment between the student's past context token and the teacher's future multimodal context token transfers language-derived reasoning rather than only action or appearance statistics.
    Section 3.2.3 applies Barlow Twins loss to context tokens. No experiment directly verifies what information is transferred, and the Limitations state that further testing is required to verify emergent reasoning behaviors.
  • domain assumption The teacher's context token contains a useful future multimodal state representation that can be distilled into the student.
    Section 3.2.2 describes the teacher as learning an informative embedding because input and output trajectories are the same while inputs are augmented with vision and text. This design assumes the context token carries more than a copy of the action input.
  • domain assumption Frozen DINOv2 and CLIP encoders provide sufficient perceptual and textual grounding for this navigation task.
    Sections 3.2.2 and 3.2.3 use frozen pretrained encoders. The paper does not test whether these specific representations limit performance on human-centric scenes.
  • domain assumption The selected SCAND subset and the authors' self-collected dataset are representative of the deployment scenarios.
    Section 3.3 says a selected subset of SCAND is used and a new dataset is collected, but no statistics or distribution plots are provided. The real-world evaluation also drops one scenario, so coverage of the claimed scenario space is uncertain.
  • domain assumption The four baselines are configured and evaluated fairly in their intended use.
    Section 4.1 compares against GNM, ViNT, NoMaD, and CityWalker, but no code or fine-tuning details are given for how each baseline was set up. The closest VLM-based method, OLiVia-Nav, is not compared because its code is unavailable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Narrate2Nav: Real-Time Visual Navigation with Implicit Language Reasoning in Human-Centric Environments." pith.science (2026). https://pith.science/paper/5PI5U3A4

@misc{pith2026250614233,
  author       = {Pith},
  title        = {Pith review of: Narrate2Nav: Real-Time Visual Navigation with Implicit Language Reasoning in Human-Centric Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5PI5U3A4}},
  note         = {Machine review of arXiv:2506.14233}
}
read the original abstract

Large Vision-Language Models (VLMs) have demonstrated potential in enhancing mobile robot navigation in human-centric environments by understanding contextual cues, human intentions, and social dynamics while exhibiting reasoning capabilities. However, their computational complexity and limited sensitivity to continuous numerical data impede real-time performance and precise motion control. To this end, we propose Narrate2Nav, a novel real-time vision-action model that leverages a novel self-supervised learning framework based on the Barlow Twins redundancy reduction loss to embed implicit natural language reasoning, social cues, and human intentions within a visual encoder-enabling reasoning in the model's latent space rather than token space. The model combines RGB inputs, motion commands, and textual signals of scene context during training to bridge from robot observations to low-level motion commands for short-horizon point-goal navigation during deployment. Extensive evaluation of Narrate2Nav across various challenging scenarios in both offline unseen dataset and real-world experiments demonstrates an overall improvement of 52.94 percent and 41.67 percent, respectively, over the next best baseline. Additionally, qualitative comparative analysis of Narrate2Nav's visual encoder attention map against four other baselines demonstrates enhanced attention to navigation-critical scene elements, underscoring its effectiveness in human-centric navigation tasks.

Figures

Figures reproduced from arXiv: 2506.14233 by the authors.

Figure 1
Figure 1. NARRATE2NAV: Implicit natural language reasoning to navigate in human-centered en￾vironments while understanding the social cues and contextual information. 9th Conference on Robot Learning (CoRL 2025), Seoul, Korea. arXiv:2506.14233v1 [cs.RO] 17 Jun 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. NARRATE2NAV In Stage 1, a large Teacher Model (∼330M parameters) is pre-trained to predict trajectories by leveraging multi-modal inputs—observations, low-level actions, and textual signals—to embed a comprehensive understanding for each. In Stage 2, a lightweight Student Model (∼29M parameters) is trained using a redundancy-reducing Barlow Twins loss (BT Loss) to embed future state representations and implicitly in… view at source ↗
Figure 3
Figure 3. Early fusion of action and observation embeddings, resulting in an effective embedding of each state. Unified Multi-Modal Latent Representation: We employ early fusion of modalities (action, vi￾sion) for each state, subsequently passing them as unified tokens to the cross-modal trans￾former. Specifically, at each time step, given encoded RGB observations ot = Seq(It+τf → It) and corresponding encoded waypoint tra￾je… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative comparative analysis of activation maps and predicted trajectories (projected [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 14 canonical work pages

  1. [1]

    Francis, C

    A. Francis, C. P ´erez-d’Arpino, C. Li, F. Xia, A. Alahi, R. Alami, A. Bera, A. Biswas, J. Biswas, R. Chandra, et al. Principles and guidelines for evaluating social robot navigation algorithms. ACM Transactions on Human-Robot Interaction, 14(2):1–65, 2025

    work page 2025

  2. [2]

    Mirsky, X

    R. Mirsky, X. Xiao, J. Hart, and P. Stone. Conflict avoidance in social navigation—a survey. ACM Transactions on Human-Robot Interaction, 13(1):1–36, 2024

    work page 2024

  3. [3]

    Mavrogiannis, F

    C. Mavrogiannis, F. Baldini, A. Wang, D. Zhao, P. Trautman, A. Steinfeld, and J. Oh. Core challenges of social robot navigation: A survey. ACM Transactions on Human-Robot Interac- tion, 12(3):1–39, 2023

    work page 2023

  4. [4]

    X. Xiao, B. Liu, G. Warnell, and P. Stone. Motion planning and control for mobile robot navigation using machine learning: a survey. Autonomous Robots, 46(5):569–597, 2022

    work page 2022

  5. [5]

    Payandeh, K

    A. Payandeh, K. T. Baghaei, P. Fayyazsanavi, S. B. Ramezani, Z. Chen, and S. Rahimi. Deep representation learning: Fundamentals, technologies, applications, and open challenges. IEEE Access, 11:137621–137659, 2023. doi:10.1109/ACCESS.2023.3335196

    arXiv 2023

  6. [6]

    M. H. Nazeri and M. Bohlouli. Exploring reflective limitation of behavior cloning in au- tonomous vehicles. In 2021 IEEE International Conference on Data Mining (ICDM) , pages 1252–1257, 2021. doi:10.1109/ICDM51629.2021.00153

    arXiv 2021

  7. [7]

    D. M. Nguyen, M. Nazeri, A. Payandeh, A. Datar, and X. Xiao. Toward human-like so- cial robot navigation: A large-scale, multi-modal, social human navigation dataset. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 7442– 7447, 2023. doi:10.1109/IROS55552.2023.10342447

    arXiv 2023

  8. [8]

    Liang, A

    J. Liang, A. Payandeh, D. Song, X. Xiao, and D. Manocha. Dtg : Diffusion-based trajectory generation for mapless global navigation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 5340–5347, 2024. doi:10.1109/IROS58592. 2024.10802055

    arXiv 2024

Show all 37 references
  1. [9]

    A. H. Raj, Z. Hu, H. Karnan, R. Chandra, A. Payandeh, L. Mao, P. Stone, J. Biswas, and X. Xiao. Rethinking social robot navigation: Leveraging the best of two worlds. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 16330–16337, 2024. doi: 10.1109...

    2024

  2. [10]

    Kretzschmar, M

    H. Kretzschmar, M. Spies, C. Sprunk, and W. Burgard. Socially compliant mobile robot nav- igation via inverse reinforcement learning. The International Journal of Robotics Research ,

  3. [11]

    D. Song, J. Liang, A. Payandeh, A. H. Raj, X. Xiao, and D. Manocha. Vlm-social-nav: Socially aware robot navigation through scoring using vision-language models. IEEE Robotics and Automation Letters, 10(1):508–515, 2025. doi:10.1109/LRA.2024.3511409

    2025

  4. [12]

    Payandeh, D

    A. Payandeh, D. Song, M. Nazeri, J. Liang, P. Mukherjee, A. H. Raj, Y . Kong, D. Manocha, and X. Xiao. Social-llava: Enhancing robot navigation through human-language reasoning in social spaces, 2024. URL https://arxiv.org/abs/2501.09024

    2024 arXiv

  5. [13]

    Narasimhan, A

    S. Narasimhan, A. H. Tan, D. Choi, and G. Nejat. Olivia-nav: An online lifelong vision language approach for mobile robot social navigation, 2025. URL https://arxiv.org/ abs/2409.13675

    2025 arXiv

  6. [14]

    Payandeh, D

    A. Payandeh, D. Pluth, J. Hosier, X. Xiao, and V . K. Gurbani. How susceptible are LLMs to logical fallacies? In N. Calzolari, M.-Y . Kan, V . Hoste, A. Lenci, S. Sakti, and N. Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, L...

    2024

  7. [15]

    D. Song, J. Liang, X. Xiao, and D. Manocha. Vl-tgs: Trajectory generation and selection using vision language models in mapless outdoor environments. IEEE Robotics and Automation Letters, 10(6):5791–5798, 2025. doi:10.1109/LRA.2025.3559822

    2025

  8. [16]

    Zbontar, L

    J. Zbontar, L. Jing, I. Misra, Y . LeCun, and S. Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International conference on machine learning , pages 12310–12320. PMLR, 2021

    2021

  9. [17]

    D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine. Gnm: A general navigation model to drive any robot, 2023. URL https://arxiv.org/abs/2210.03370

    2023 arXiv

  10. [18]

    D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine. Vint: A foundation model for visual navigation, 2023. URLhttps://arxiv.org/abs/2306.14846

    2023 arXiv

  11. [19]

    Sridhar, D

    A. Sridhar, D. Shah, C. Glossop, and S. Levine. Nomad: Goal masked diffusion policies for navigation and exploration, 2023. URL https://arxiv.org/abs/2310.07896

    2023 arXiv

  12. [20]

    X. Liu, J. Li, Y . Jiang, N. Sujay, Z. Yang, J. Zhang, J. Abanes, J. Zhang, and C. Feng. Citywalker: Learning embodied urban navigation from web-scale videos. arXiv preprint arXiv:2411.17820, 2024

    2024 arXiv

  13. [21]

    Eftekhar, K.-H

    A. Eftekhar, K.-H. Zeng, J. Duan, A. Farhadi, A. Kembhavi, and R. Krishna. Selective visual representations improve convergence and generalization for embodied ai, 2024. URL https: //arxiv.org/abs/2311.04193

    2024 arXiv

  14. [22]

    Nazeri, J

    M. Nazeri, J. Wang, A. Payandeh, and X. Xiao. Vanp: Learning where to see for navigation with self-supervised vision-action pre-training. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 2741–2746. IEEE, 2024

    2024

  15. [23]

    Elnoor, K

    M. Elnoor, K. Weerakoon, G. Seneviratne, J. Liang, V . Rajagopal, and D. Manocha. Vi-lad: Vision-language attention distillation for socially-aware robot navigation in dynamic environ- ments, 2025. URL https://arxiv.org/abs/2503.09820

    2025

  16. [24]

    Pokhrel, M

    A. Pokhrel, M. Nazeri, A. Datar, and X. Xiao. Cahsor: Competence-aware high-speed off-road ground navigation in SE(3). IEEE Robotics and Automation Letters , 9(11):9653–9660, 2024. doi:10.1109/LRA.2024.3457369

    2024

  17. [25]

    P. Roth, J. Nubert, F. Yang, M. Mittal, and M. Hutter. Viplanner: Visual semantic imperative learning for local navigation, 2024. URL https://arxiv.org/abs/2310.00982

    2024 arXiv

  18. [26]

    H. Wang, A. H. Tan, and G. Nejat. NavFormer: A Transformer Architecture for Robot Target- Driven Navigation in Unknown and Dynamic Environments.arXiv preprint arXiv:2402.06838, 2024

    2024 arXiv

  19. [27]

    Hirose, C

    N. Hirose, C. Glossop, A. Sridhar, D. Shah, O. Mees, and S. Levine. Lelan: Learning a language-conditioned navigation policy from in-the-wild video. In Conference on Robot Learning, 2024

    2024

  20. [28]

    Eftekhar, L

    A. Eftekhar, L. Weihs, R. Hendrix, E. Caglar, J. Salvador, A. Herrasti, W. Han, E. VanderBil, A. Kembhavi, A. Farhadi, R. Krishna, K. Ehsani, and K.-H. Zeng. The one ring: a robotic indoor navigation generalist, 2024. URL https://arxiv.org/abs/2412.14401

    2024 arXiv

  21. [29]

    Cheng, H

    A.-C. Cheng, H. Yin, Y . Fu, Q. Guo, R. Yang, J. Kautz, X. Wang, and S. Liu. Spatialrgpt: Grounded spatial reasoning in vision language models, 2024. URL https://arxiv.org/ abs/2406.01584

    2024 arXiv

  22. [30]

    A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang, J. Wang, J. Yang, J. Tu, J. Zhang, J. Ma, J. Yang, J. Xu, J. Zhou, J. Bai, J. He, J. Lin, K. Dang, K. Lu, K. Chen, K. Yang, M. Li, M. Xue, N. Ni, P. Zhang, P. ...

    2024 arXiv

  23. [31]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A...

    2024 arXiv

  24. [32]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning , pages 8748–8763. PmLR, 2021

    2021

  25. [33]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition, 2015. URL https://arxiv.org/abs/1512.03385

    2015 arXiv

  26. [34]

    Nazeri, A

    M. Nazeri, A. Pokhrel, A. Card, A. Datar, G. Warnell, and X. Xiao. Vertiformer: A data- efficient multi-task transformer for off-road robot mobility, 2025. URL https://arxiv. org/abs/2502.00543

    2025 arXiv

  27. [35]

    Karnan, A

    H. Karnan, A. Nair, X. Xiao, G. Warnell, S. Pirk, A. Toshev, J. Hart, J. Biswas, and P. Stone. Socially compliant navigation dataset (scand): A large-scale dataset of demonstrations for social navigation. IEEE Robotics and Automation Letters , 2022

    2022

  28. [36]

    S. Pirk, E. Lee, X. Xiao, L. Takayama, A. Francis, and A. Toshev. A protocol for validating social navigation policies. arXiv preprint arXiv:2204.05443, 2022. 12

    2022 arXiv

  29. [2016]

    doi:10.1177/0278364915619772

    doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.