Pith. sign in

REVIEW 3 major objections 5 minor 56 references

Recursive Visual Imagination and Adaptive Linguistic Grounding for Vision Language Navigation

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A navigation agent that imagines scenes and aligns instructions component-by-component reports the best results yet on two continuous-environment benchmarks.

desk verdict Coherent new combination of scene-memory and grounding losses, but the reported SOTA margins are fragile and the tables disagree on the full model's SR. read the letter →

arxiv 2507.21450 v1 pith:3CXEUDAI submitted 2025-07-29 cs.CV cs.RO

Bolei Chen , Jiaxu Kang , Yifei Wang , Ping Zhong , Qi Wu , Jianxin Wang This is my paper
classification cs.CVcs.RO
keywords VisionLanguageNavigationImplicitSceneRepresentationRecursiveVisualImaginationAdaptiveLinguisticGroundingVLN-CEObjectNavBehaviorCloningInstructionDecoupling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a vision-language navigation agent performs better when its memory of the route is a compact, fixed-size set of learned neural grids rather than a detailed geometric map, and when the navigation instruction is split into components (landmarks, scenes, actions, orientations, and other words) that are each aligned with parts of that memory. The authors propose Recursive Visual Imagination (RVI) to train the memory through three self-supervised tasks: imagining views at queried poses, predicting the local semantic layout, and predicting visible object categories. They then use Adaptive Linguistic Grounding (ALG) to match instruction components to memory grids positionally and semantically, together with a progress-tracking module. On the R2R-CE benchmark the policy reaches 59% success rate on unseen validation scenes and 57% on the test split, and on MP3D ObjectNav it reaches 40.9% success rate, both reported as the best numbers in the paper's comparison tables. If correct, the work shows that deliberately discarding geometric detail and forcing fine-grained vision-language matching is a viable route to instruction-following in continuous environments.

What carries the argument

The load-bearing object is the Implicit Scene Representation (ISR): a fixed h-by-w grid (10 x 10 in the main experiments) of d-dimensional feature vectors, initialized from position embeddings and updated at each step by a transformer that fuses the grid with the current observation token. Because the grid count is a hyperparameter, memory cost does not grow with trajectory length. RVI supplies the learning signal that makes the grid navigation-friendly: view imagination retrieves visual features at queried past or future poses with a contrastive loss and a KL divergence between prior and posterior future-frame distributions; scene layout imagination predicts an egocentric semantic map; visual semantic prediction estimates object presence and occupancy ratios. ALG then uses the transformer's own cross-attention matrix as an affinity map, row-wise max-pools it to find the grid most attended by each language token, and supervises position alignment (predicting which word positions are landmarks or scenes) and semantic alignment (contrastive matching of averaged grid features to landmark and scene text features), together with a progress-tracking module that reweights instruction words as the agent advances.

What would settle it

Train the full pipeline on R2R-CE Val-Unseen with the ALG component labels randomly permuted across the five categories while keeping every other loss unchanged; if success rate stays near the reported 59 rather than dropping substantially, the gains attributed to component-level alignment do not come from correct instruction decoupling. A cheaper check is to rerun the GPT-4-decoupling experiment, where the paper reports a 60 SR with GPT-4 labels versus 58 without manual checking, a gap that an independent replication should reproduce.

Watch

Extended reading notes

Core claim

The central claim is that navigation performance in continuous indoor environments improves when the agent's historical observations are stored as an Implicit Scene Representation (ISR), a small grid of learned feature vectors whose size is fixed regardless of trajectory length, and when language grounding is done at the level of instruction components rather than whole sentences. The paper argues that explicit maps and feature fields retain too much geometric and textural detail, which distracts the transformer from navigation-relevant scene priors and makes vision-language alignment ambiguous. RVI trains the ISR by contrasting pose-query visual features, predicting future frame distributions, forecasting egocentric semantic maps, and predicting object presence, so that the memory encodes regularities of visual change and landmark layout. ALG then parses the instruction into landmarks, scenes, actions, orientations, and others, tracks navigation progress with learned word weights, and aligns the decoded components with the neural grids through position-label prediction and contrastive semantic matching. The result is a single behavior-cloned policy that the paper reports as the strongest on R2R-CE Val-Unseen (SR 59, SPL 50) and MP3D ObjectNav (SR 40.9, SPL 17.1).

Load-bearing premise

The method depends on the assumption that navigation instructions can be reliably split into five semantic components by automatic parsers: the paper reports that skipping the manual check of parser output costs about one point of success rate, and that manual checking becomes impractical as instruction sets grow, so noisy parsing would feed wrong pairings into the alignment losses.

Editorial extensions

If this is right

  • Memory footprint and token count stay constant as episodes lengthen, because the ISR grid size does not depend on trajectory length or scene scale.
  • Each RVI task contributes: adding scene-layout prediction, pose-query contrast, and future-imagination KL raises Val-Unseen SR from 49 to 53 in the paper's ablation, before ALG is added.
  • Progress tracking and the two alignment losses work together: the ablation shows that removing progress tracking while keeping the alignments lowers performance, consistent with the claim that alignment must be progress-aware.
  • Using GPT-4 for instruction decoupling outperforms off-the-shelf parsing with manual correction, suggesting the component split itself, not the specific parser, is what matters.
  • Scene- and landmark-focused alignment beats action-focused alignment on R2R-CE, indicating the benchmark rewards attention to landmark entities over action words.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the ISR capacity is fixed, very large scenes or very long episodes could saturate the grid; a testable extension is to grow grid dimensions with scene area and check whether the reported gains persist or plateau.
  • The component-level grounding idea transfers naturally to other instruction-following embodied tasks, such as robotic manipulation, where commands can be split into object, action, and spatial-relation components and aligned to separate memory slots.
  • The attention-matrix-as-affinity trick avoids a separate matching network; one could extend it to weakly supervised settings where component labels come from free-form LLM reasoning rather than a fixed five-way taxonomy.
  • The progress-tracking word weights could serve as an interpretability output: a human could read which instruction words the agent believes it has completed and use that signal to debug failures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a vision-language navigation policy for continuous environments and object navigation. Historical observations, poses, and actions are encoded into a fixed h×w neural-grid Implicit Scene Representation (ISR) that is updated by a transformer. Recursive Visual Imagination (RVI) adds pre-training objectives: a contrastive view-imagination loss with a KL regularizer over future-frame distributions, egocentric semantic-map prediction, and view-level semantic prediction. Adaptive Linguistic Grounding (ALG) parses instructions into landmarks, scenes, actions, orientations, and others; uses the last cross-attention map to select relevant ISR grids; and aligns them via progress-aware positional and semantic losses. Experiments on R2R-CE and MP3D-ObjectNav report state-of-the-art success rates of 59.0 and 40.9, together with ablations, diagnostics, and sensitivity analyses over the imagination range and grid size.

Significance. If the reported numbers are stable, the paper makes a useful contribution: a fixed-size learned scene memory that avoids unbounded growth, imagination-style auxiliary objectives that are anchored to external signals (future views, semantic maps, parsed components), and component-wise linguistic grounding. The method is not circular: all auxiliary losses are supervised by externally derived targets, and the use of the model's own attention map for grid selection is a self-referential but not a self-supervising design. However, the margins over prior state of the art are only 1–2 SR points, no variance estimates are reported, one central loss equation is inconsistent with its textual description, and the main configuration relies on manual correction of parser output. The evidence as written is therefore not yet sufficient to establish the claimed superiority, although the approach is plausible and the issues appear fixable.

major comments (3)
  1. [Tables 1, 3, and Supplementary Table 5] The paper does not report a single reproducible configuration for the full model. Table 1 lists Ours on R2R-CE Val-Unseen as 67/59/50; Table 3's last row, described as the ablation of the full model with all losses enabled, is 67/58/50; Supplementary Table 5 reports w/o manual check as 66/58/49, w/ manual check as 67/59/50, and w/ GPT-4 as 67/60/51. The Table 3 row therefore combines the OSR of the manual/GPT-4 rows with the SR of the w/o-manual row and the SPL of the manual/GPT-4 rows. No footnote states which instruction-decoupling protocol was used in Table 3, and no error bars or multiple seeds are reported anywhere. Since the headline advantage over Zhang et al. on Val-Unseen is exactly 1 SR point (58 vs. 59) and the Test-Unseen margins over ETPNav and Zhang et al. are also 1–2 points, these inconsistencies and the missing variance estimates leave the SOTA claim unsupported as written.
  2. [Equation (5)] The semantic alignment loss in Eq. (5) does not implement the contrast described in the text. The denominator is sum_{j=1}^l with l = |X+| defined as the number of landmark/scene tokens, so the normalization runs only over the positive set. The prose states that the loss should 'keep away the dissimilar ones' using actions, orientations, and others as negatives, but those components never enter Eq. (5). As written, the loss ranks each positive token against other positive tokens, and the alpha- weighting is applied to positive examples rather than to negatives, so the reported gains from LSA cannot be attributed to contrastive separation of instruction components. The equation should be corrected (for example, by summing over all five components in the denominator) and the ablation re-run.
  3. [Instruction Decoupling and Supplementary Table 5] The main results depend on a subjective annotation step. The paper states that off-the-shelf tools are adjusted by manual checking and that this is impractical as the instruction set grows; Supplementary Table 5 shows that the w/ manual-check row equals Table 1's Ours, while w/o manual check drops by 1 point in SR and SPL, and GPT-4 decoupling gives 67/60/51, which is actually higher than the headline result. The authors need to state which protocol produced Tables 1–3, how many instructions were manually corrected and with what guidelines, and preferably adopt the automatic GPT-4 protocol as the primary configuration. Without this, the SOTA result is not reproducible without access to the authors' manual corrections.
minor comments (5)
  1. [Equation (6)] The inflection-weighted action loss is written with the multiplier inside the logarithm: -(1 + gamma * indicator * log p(a*_t)). It should be -(1 + gamma * indicator) * log p(a*_t), with the indicator multiplying the loss weight rather than the log-probability.
  2. [Implementation Details] The text sets 'beta and lambda in Eq. 9' to 0.3 and 0.5, but Eq. (9) appears in the supplementary material and the main-text total loss is Eq. (7). Please renumber or cross-reference consistently.
  3. [References] The reference list contains duplicate entries that are cited as different works: Chen et al. 2021a and 2021b are the same paper, Hong et al. 2023a and 2023b are the same, and Yin et al. 2024a and 2024b are the same. These should be merged and cited by their original unique identifiers.
  4. [Table 4] The abbreviation 'DIA' in the table title is not defined; it appears to refer to the alignment-priority variant of ALG, but this should be stated explicitly.
  5. [Figure 5] The caption says that a redder color indicates the agent is more attentive to the corresponding words, but the scale is not quantified. Please specify the color range or provide a legend so the reader can interpret the magnitudes.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RVI and ALG are supervised by external visual, map, pose, and parsed-language signals, so the SOTA claim is not derived from its own inputs.

full rationale

Score 0. No load-bearing step in the claimed derivation reduces to its own inputs. RVI's view-imagination loss (Eq. 1) contrasts a query-pose readout against the frozen CLIP feature of the actual panoramic frame at that pose, and the KL term in Eq. 2 regularizes a latent future distribution; neither target is produced by the model itself. SLI (Eq. 3) regresses egocentric semantic maps pre-generated from MP3D, and VSP uses semantic-segmentation labels, both external. ALG's positional and semantic losses use component position labels obtained by syntactic parsing (with manual-check or GPT-4 variants compared in supplementary Table 5), and progress tracking uses the externally measured normalized distance to the goal. The one self-referential mechanism (using the last cross-modal attention as an affinity to select which grids to align) is still supervised by external component labels, so the target is not defined by the prediction. Self-citations (Chen et al. 2023, Chen et al. 2024) appear only as related-work context or as a baseline, not as the proof of any prediction, and are therefore not load-bearing. Reproducibility weaknesses are real but distinct: Table 1's Ours 67/59/50 does not match the full-model row 67/58/50 in Table 3 nor any single variant in Table 5, and no error bars are reported; the supplementary also admits manual instruction checking is impractical as the instruction set grows. These are correctness and robustness concerns, not circularity.

Assumptions & free parameters 8 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a set of hyperparameters, the representation assumption of the ISR, the instruction decoupling assumption, and the transferability of self-supervised imagination losses. The paper provides no external validation for the ISR beyond its own benchmark results, and the manual checking of parsing is a human-in-the-loop component that is hard to quantify.

free parameters (8)
  • k (range of visual imagination) = 20
    Number of future time steps imagined; chosen by validation sensitivity analysis shown in Figure 6.
  • ISR grid size h and w = 10x10
    Dimensions of the neural grid; chosen by sensitivity analysis in Figure 6.
  • Loss weight beta = 0.3
    Scales the auxiliary losses (LVF, LMap, LSem); set by hand.
  • Loss weight lambda = 0.5
    Scales the ALG losses (LPro, LPA, LSA); set by hand.
  • Contrastive weights alpha+ and alpha- = 1.0 and 2.0
    Weights for positive and negative terms in the semantic alignment loss LSA.
  • Temperature tau = 0.07
    Temperature in the contrastive losses for view imagination and semantic alignment.
  • Instruction length cap L = 160
    Maximum instruction length used in ALG; set empirically to cover R2R-CE instructions.
  • Embedding dimension d = 512
    Dimension of each neural grid vector; model capacity hyperparameter.
assumptions (5)
  • ad hoc to paper A fixed-size h x w neural grid can summarize arbitrarily long navigation trajectories without losing the information needed for high-level scene priors.
    This is the core representation assumption of the ISR; it is not derived from theory and is only tested empirically on specific benchmarks.
  • domain assumption Navigation instructions can be decoupled into five semantic components (landmark, scene, action, orientation, others) by syntactic parsing, and manual correction of parsing errors is acceptable for training.
    Invoked in the ALG section; the paper shows sensitivity to this assumption via the w/o manual check ablation, where SR drops from 59 to 58.
  • domain assumption Future view prediction and egocentric semantic map prediction are effective self-supervised objectives for learning navigation-friendly representations.
    This motivates RVI; it is an empirical grounding assumption with no independent proof of transfer to navigation.
  • domain assumption The frozen CLIP ResNet50 and PointNav ResNet18 visual encoders provide features adequate for VLN and ObjectNav.
    Used for observation encoding; no fine-tuning is performed on these encoders.
  • standard math Standard transformer attention, contrastive learning, and KL divergence are valid building blocks for the proposed losses.
    Background ML machinery assumed by the paper.
invented entities (1)
  • Implicit Scene Representation (ISR) neural grids
    purpose: Compress historical trajectory observations into a fixed-size set of learnable feature vectors aligned with instruction components.
    The ISR is a new learned representation introduced by the paper; its effectiveness is only demonstrated through the paper's own benchmarks, with no external validation or formal guarantees.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recursive Visual Imagination and Adaptive Linguistic Grounding for Vision Language Navigation." pith.science (2026). https://pith.science/paper/3CXEUDAI

@misc{pith2026250721450,
  author       = {Pith},
  title        = {Pith review of: Recursive Visual Imagination and Adaptive Linguistic Grounding for Vision Language Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3CXEUDAI}},
  note         = {Machine review of arXiv:2507.21450}
}
read the original abstract

Vision Language Navigation (VLN) typically requires agents to navigate to specified objects or remote regions in unknown scenes by obeying linguistic commands. Such tasks require organizing historical visual observations for linguistic grounding, which is critical for long-sequence navigational decisions. However, current agents suffer from overly detailed scene representation and ambiguous vision-language alignment, which weaken their comprehension of navigation-friendly high-level scene priors and easily lead to behaviors that violate linguistic commands. To tackle these issues, we propose a navigation policy by recursively summarizing along-the-way visual perceptions, which are adaptively aligned with commands to enhance linguistic grounding. In particular, by structurally modeling historical trajectories as compact neural grids, several Recursive Visual Imagination (RVI) techniques are proposed to motivate agents to focus on the regularity of visual transitions and semantic scene layouts, instead of dealing with misleading geometric details. Then, an Adaptive Linguistic Grounding (ALG) technique is proposed to align the learned situational memories with different linguistic components purposefully. Such fine-grained semantic matching facilitates the accurate anticipation of navigation actions and progress. Our navigation policy outperforms the state-of-the-art methods on the challenging VLN-CE and ObjectNav tasks, showing the superiority of our RVI and ALG techniques for VLN.

Figures

Figures reproduced from arXiv: 2507.21450 by the authors.

Figure 1
Figure 1. The VLN agent decouples an instruction into dif [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An illustration of our VLN policy with RVI (Fig. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustration of RVI, including view imagination, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The process of position label prediction is as follows: [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: A visualization of how the instruction weights [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Illustrations of parametric studies. the RVI techniques (LMap, LCon, and LKL) can improve the VLN performance. In addition, the involvements of posi￾tional alignment LP A and semantic alignment LSA promote ALG, which further leads to substantial OSR, SR, and SPL boosts…
Figure 7
Figure 7. Figure 7: An example of scene and trajectory used for data collection for pre-training. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 9
Figure 9. Figure 9: An illustration of semantic component division [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: (a)-(f) illustrate the navigation views and pro [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 43 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    An, D.; Qi, Y.; Li, Y.; Huang, Y.; Wang, L.; Tan, T.; and Shao, J. 2023. Bevbert: Multimodal map pre-training for language-guided navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2737--2748

    work page 2023

  5. [5]

    An, D.; Wang, H.; Wang, W.; Wang, Z.; Huang, Y.; He, K.; and Wang, L. 2024. Etpnav: Evolving topological planning for vision-language navigation in continuous environments. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  6. [6]

    Anderson, P.; Wu, Q.; Teney, D.; Bruce, J.; Johnson, M.; S \"u nderhauf, N.; Reid, I.; Gould, S.; and Van Den Hengel, A. 2018. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3674--3683

    2018

  7. [7]

    Chang, A.; Dai, A.; Funkhouser, T.; Halber, M.; Niessner, M.; Savva, M.; Song, S.; Zeng, A.; and Zhang, Y. 2017. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158

    arXiv 2017

  8. [8]

    S.; Gandhi, D

    Chaplot, D. S.; Gandhi, D. P.; Gupta, A.; and Salakhutdinov, R. R. 2020. Object goal navigation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems, 33: 4247--4258

    work page 2020

Show all 56 references
  1. [9]

    Chen, B.; Kang, J.; Zhong, P.; Cui, Y.; Lu, S.; Liang, Y.; and Wang, J. 2023. Think holistically, act down-to-earth: A semantic navigation strategy with continuous environmental representation and multi-step forward planning. IEEE Transactions on Circuits and Systems for Video...

    2023

  2. [10]

    Chen, B.; Kang, J.; Zhong, P.; Liang, Y.; Sheng, Y.; and Wang, J. 2024. Embodied Contrastive Learning with Geometric Consistency and Behavioral Awareness for Object Navigation. In Proceedings of the 32nd ACM International Conference on Multimedia, 4776--4785

    2024

  3. [11]

    Chen, P.; Ji, D.; Lin, K.; Zeng, R.; Li, T.; Tan, M.; and Gan, C. 2022 a . Weakly-supervised multi-granularity map learning for vision-and-language navigation. Advances in Neural Information Processing Systems, 35: 38149--38161

    2022

  4. [12]

    Chen, S.; Guhur, P.-L.; Schmid, C.; and Laptev, I. 2021 a . History aware multimodal transformer for vision-and-language navigation. Advances in neural information processing systems, 34: 5834--5847

    2021

  5. [13]

    Chen, S.; Guhur, P.-L.; Schmid, C.; and Laptev, I. 2021 b . History aware multimodal transformer for vision-and-language navigation. Advances in neural information processing systems, 34: 5834--5847

    2021

  6. [14]

    Chen, S.; Guhur, P.-L.; Tapaswi, M.; Schmid, C.; and Laptev, I. 2022 b . Think global, act local: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16537--16547

    2022

  7. [15]

    Cheng, W.; Dong, X.; Khan, S.; and Shen, J. 2022. Learning disentanglement with decoupled labels for vision-language navigation. In European Conference on Computer Vision, 309--329. Springer

    2022

  8. [16]

    Cui, Y.; Xie, L.; Zhang, Y.; Zhang, M.; Yan, Y.; and Yin, E. 2023. Grounded entity-landmark adaptive pre-training for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 12043--12053

    2023

  9. [17]

    Dang, R.; Shi, Z.; Wang, L.; He, Z.; Liu, C.; and Chen, Q. 2022. Unbiased directed object attention graph for object navigation. In Proceedings of the 30th ACM International Conference on Multimedia, 3617--3627

    2022

  10. [18]

    P.; Kim, Y.; Han, W.; Herrasti, A.; et al

    Ehsani, K.; Gupta, T.; Hendrix, R.; Salvador, J.; Weihs, L.; Zeng, K.-H.; Singh, K. P.; Kim, Y.; Han, W.; Herrasti, A.; et al. 2024. SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World. In Proceedings of the IEEE/CVF Con...

    2024

  11. [19]

    Georgakis, G.; Schmeckpeper, K.; Wanchoo, K.; Dan, S.; Miltsakaki, E.; Roth, D.; and Daniilidis, K. 2022. Cross-modal map learning for vision and language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15460--15470

    2022

  12. [20]

    Gervet, T.; Chintala, S.; Batra, D.; Malik, J.; and Chaplot, D. S. 2022. Navigating to objects in the real world. Science Robotics, 8

    2022

  13. [21]

    Hong, Y.; Wang, Z.; Wu, Q.; and Gould, S. 2022. Bridging the gap between learning in discrete and continuous environments for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15439--15449

    2022

  14. [22]

    Hong, Y.; Wu, Q.; Qi, Y.; Rodriguez-Opazo, C.; and Gould, S. 2021. Vln bert: A recurrent vision-and-language bert for navigation. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, 1643--1653

    2021

  15. [23]

    Hong, Y.; Zhou, Y.; Zhang, R.; Dernoncourt, F.; Bui, T.; Gould, S.; and Tan, H. 2023 a . Learning navigational visual representations with semantic map supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3055--3067

    2023

  16. [24]

    Hong, Y.; Zhou, Y.; Zhang, R.; Dernoncourt, F.; Bui, T.; Gould, S.; and Tan, H. 2023 b . Learning navigational visual representations with semantic map supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3055--3067

    2023

  17. [25]

    Hu, S.; Shen, L.; Zhang, Y.; Chen, Y.; and Tao, D. 2024. On Transforming Reinforcement Learning With Transformers: The Development Trajectory. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  18. [26]

    Krantz, J.; Wijmans, E.; Majumdar, A.; Batra, D.; and Lee, S. 2020. Beyond the nav-graph: Vision-and-language navigation in continuous environments. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXVIII 16, 104--120...

    2020

  19. [27]

    Ku, A.; Anderson, P.; Patel, R.; Ie, E.; and Baldridge, J. 2020. Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding. arXiv preprint arXiv:2010.07954

    2020 arXiv

  20. [28]

    Lin, C.; Jiang, Y.; Cai, J.; Qu, L.; Haffari, G.; and Yuan, Z. 2022. Multimodal transformer with variable-length memory for vision-and-language navigation. In European Conference on Computer Vision, 380--397. Springer

    2022

  21. [29]

    O'Keefe, J.; and Burgess, N. 1996. Geometric determinants of the place fields of hippocampal neurons. Nature, 381(6581): 425--428

    1996

  22. [30]

    Pardyl, A.; Rype\' s \' c , G.; Kurzejamski, G.; Zieli\' n ski, B.; and Trzci\' n ski, T. 2023. Active visual exploration based on attention-map entropy. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI '23. ISBN 978-1-956792-03-4

    2023

  23. [31]

    Y.; Shen, C.; and Hengel, A

    Qi, Y.; Wu, Q.; Anderson, P.; Wang, X.; Wang, W. Y.; Shen, C.; and Hengel, A. v. d. 2020. Reverie: Remote embodied visual referring expression in real indoor environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9982--9991

    2020

  24. [32]

    Qiao, Y.; Qi, Y.; Hong, Y.; Yu, Z.; Wang, P.; and Wu, Q. 2023. Hop+: History-enhanced and order-aware pre-training for vision-and-language navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7): 8524--8537

    2023

  25. [33]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  26. [34]

    K.; Gokaslan, A.; Wijmans, E.; Maksymets, O.; Clegg, A.; Turner, J.; Undersander, E.; Galuba, W.; Westbury, A.; Chang, A

    Ramakrishnan, S. K.; Gokaslan, A.; Wijmans, E.; Maksymets, O.; Clegg, A.; Turner, J.; Undersander, E.; Galuba, W.; Westbury, A.; Chang, A. X.; et al. 2021. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv preprint arXiv:2109.08238

    2021 arXiv

  27. [35]

    Ross, S.; Gordon, G.; and Bagnell, D. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, 627--635. JMLR Workshop and Conference Proceedings

    2011

  28. [36]

    Schuster, S.; Krishna, R.; Chang, A.; Fei-Fei, L.; and Manning, C. D. 2015. Generating semantically precise scene graphs from textual descriptions for improved image retrieval. In Proceedings of the fourth workshop on vision and language, 70--80

    2015

  29. [37]

    A.; Miall, R

    Sokolov, A. A.; Miall, R. C.; and Ivry, R. B. 2017. The cerebellum: adaptive prediction for movement and cognition. Trends in cognitive sciences, 21(5): 313--332

    2017

  30. [38]

    Tan, S.; Sima, K.; Wang, D.; Ge, M.; Guo, D.; and Liu, H. 2024. Self-Supervised 3-D Semantic Representation Learning for Vision-and-Language Navigation. IEEE Transactions on Neural Networks and Learning Systems

    2024

  31. [39]

    Tolman, E. C. 1948. Cognitive maps in rats and men. Psychological review, 55(4): 189

    1948

  32. [40]

    G.; Watkins, K

    Vargha-Khadem, F.; Gadian, D. G.; Watkins, K. E.; Connelly, A.; Van Paesschen, W.; and Mishkin, M. 1997. Differential effects of early hippocampal pathology on episodic and semantic memory. Science, 277(5324): 376--380

    1997

  33. [41]

    Wang, H.; Liang, W.; Van Gool, L.; and Wang, W. 2023 a . Dreamwalker: Mental planning for continuous vision-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10873--10883

    2023

  34. [42]

    Wang, L.; He, Z.; Tang, J.; Dang, R.; Wang, N.; Liu, C.; and Chen, Q. 2023 b . A dual semantic-aware recurrent global-adaptive network for vision-and-language navigation. arXiv preprint arXiv:2305.03602

    2023 arXiv

  35. [43]

    Wang, Z.; Li, X.; Yang, J.; Liu, Y.; and Jiang, S. 2023 c . Gridmm: Grid memory map for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 15625--15636

    2023

  36. [44]

    Wijmans, E.; Datta, S.; Maksymets, O.; Das, A.; Gkioxari, G.; Lee, S.; Essa, I.; Parikh, D.; and Batra, D. 2019 a . Embodied question answering in photorealistic environments with point cloud perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2019

  37. [45]

    Wijmans, E.; Kadian, A.; Morcos, A.; Lee, S.; Essa, I.; Parikh, D.; Savva, M.; and Batra, D. 2019 b . Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. arXiv preprint arXiv:1911.00357

    2019 arXiv

  38. [46]

    Wu, H.; Mao, J.; Zhang, Y.; Jiang, Y.; Li, L.; Sun, W.; and Ma, W.-Y. 2019. Unified visual-semantic embeddings: Bridging vision and language with structured meaning representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6609--6618

    2019

  39. [47]

    Wu, S.; Fu, X.; Wu, F.; and Zha, Z.-J. 2024. Vision-and-Language Navigation via Latent Semantic Alignment Learning. IEEE Transactions on Multimedia

    2024

  40. [48]

    Xie, W.; Jiang, H.; Zhu, Y.; Qian, J.; and Xie, J. 2025. NaviFormer: A Spatio-Temporal Context-Aware Transformer for Object Navigation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 14708--14716

    2025

  41. [49]

    Yadav, K.; Ramrakhya, R.; Majumdar, A.; Berges, V.-P.; Kuhar, S.; Batra, D.; Baevski, A.; and Maksymets, O. 2023. Offline visual representation learning for embodied navigation. In Workshop on Reincarnating Reinforcement Learning at ICLR 2023

    2023

  42. [51]

    Yin, H.; Xu, X.; Wu, Z.; Zhou, J.; and Lu, J. 2024 b . SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation. arXiv preprint arXiv:2410.08189

    2024 arXiv

  43. [52]

    Yokoyama, N.; Ha, S.; Batra, D.; Wang, J.; and Bucher, B. 2024. Vlfm: Vision-language frontier maps for zero-shot semantic navigation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 42--48. IEEE

    2024

  44. [53]

    ???? Trajectory Diffusion for ObjectGoal Navigation

    Yu, X.; Zhang, S.; Song, X.; Qin, X.; and Jiang, S. ???? Trajectory Diffusion for ObjectGoal Navigation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

  45. [54]

    Zhang, J.; Dai, L.; Meng, F.; Fan, Q.; Chen, X.; Xu, K.; and Wang, H. 2023. 3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6672--6682

    2023

  46. [55]

    Zhang, S.; Song, X.; Yu, X.; Bai, Y.; Guo, X.; Li, W.; and Jiang, S. 2025. HOZ++: Versatile Hierarchical Object-to-Zone Graph for Object Navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2025

  47. [56]

    Zhang, S.; Yu, X.; Song, X.; Wang, X.; and Jiang, S. 2024. Imagine Before Go: Self-Supervised Generative Map for Object Goal Navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16414--16425

    2024

  48. [57]

    Zhang, Y.; and Kordjamshidi, P. 2024. Narrowing the gap between vision and action in navigation. In Proceedings of the 32nd ACM International Conference on Multimedia, 856--865

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.