pith. sign in

arxiv: 2109.15207 · v1 · pith:3NIWC5MBnew · submitted 2021-09-30 · 💻 cs.CV · cs.CL· cs.RO

Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments

Sonia Raychaudhuri , Saim Wani , Shivansh Patel , Unnat Jain , Angel X. Chang This is my paper
classification 💻 cs.CV cs.CLcs.RO
keywords agentnavigationpathsupervisionworkinstructionlanguagelanguage-aligned
0
0 comments X
read the original abstract

In the Vision-and-Language Navigation (VLN) task an embodied agent navigates a 3D environment, following natural language instructions. A challenge in this task is how to handle 'off the path' scenarios where an agent veers from a reference path. Prior work supervises the agent with actions based on the shortest path from the agent's location to the goal, but such goal-oriented supervision is often not in alignment with the instruction. Furthermore, the evaluation metrics employed by prior work do not measure how much of a language instruction the agent is able to follow. In this work, we propose a simple and effective language-aligned supervision scheme, and a new metric that measures the number of sub-instructions the agent has completed during navigation.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LookasideVLN: Direction-Aware Aerial Vision-and-Language Navigation

    cs.CV 2026-04 unverdicted novelty 7.0

    LookasideVLN improves aerial vision-and-language navigation by encoding directional cues from instructions into an egocentric graph and lightweight knowledge base, outperforming prior methods like CityNavAgent even wi...

  2. AstraNav-World: World Model for Foresight Control and Consistency

    cs.CV 2025-12 unverdicted novelty 6.0

    AstraNav-World unifies diffusion video generation and vision-language action planning in a single bidirectional model that improves trajectory accuracy, success rates, and zero-shot real-world adaptation in embodied n...

  3. Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks

    cs.RO 2024-12 unverdicted novelty 6.0

    Uni-NaVid unifies diverse embodied navigation tasks into one video-based vision-language-action model trained on 3.6 million samples from four sub-tasks, achieving state-of-the-art performance on benchmarks and real-w...

  4. NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation

    cs.CV 2024-02 unverdicted novelty 6.0

    NaVid, a video-based VLM trained on 510k navigation and 763k web samples, achieves SOTA VLN performance using only monocular RGB video for next-step action planning in sim and real environments.