Pith. sign in

REVIEW 4 major objections 7 minor 49 references

Planning from Imagination: Episodic Simulation and Episodic Memory for Vision-and-Language Navigation

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A navigation agent that stores imagined future views alongside real ones tops R2R and REVERIE benchmarks.

desk verdict The architecture idea is genuine and the ablations are suggestive, but the headline R2R SPL=78 contradicts the paper's own ablation tables (70/71), so the SoTA claim is unsupported as written. read the letter →

arxiv 2412.01857 v2 pith:2Q6JKOE6 submitted 2024-11-30 cs.CV cs.LGcs.RO

Yiyuan Pan , Yunzhe Xu , Zhe Liu , Hesheng Wang This is my paper · ORCID
classification cs.CVcs.LGcs.RO
keywords vision-and-languagenavigationepisodicmemorysimulationtopologicalmapgenerativeworldmodelSPADER2RREVERIE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Navigation agents often lose their way in new buildings because they only act on what they currently see. This paper argues that an agent can do better by maintaining an episodic memory that includes imagined future scenes, not just observed ones. It introduces SALI, which builds a topological map whose nodes are either real observations or images the agent generates of places it has not yet visited. On the R2R and REVERIE benchmarks, that hybrid memory raises success-weighted path length by 8% and 4% in unseen environments over prior state-of-the-art systems. The claim is that mentally simulating the next views—not just perceiving the current one—is a practical navigation resource.

What carries the argument

The load-bearing mechanism is the recurrent imagination tree, which generates future scenes one step at a time and writes them into the topological memory. Each expansion takes the current semantic, depth, and RGB context plus a candidate position, and produces the next semantic/depth images with an inpaint model, then the next RGB image with a SPADE generator; room-type and waypoint models add priors and propose successor positions. These imagined nodes are pruned against existing nodes by feature cosine similarity and position distance, and their navigation scores are added to the nearest real node's score through a sigmoid fusion factor. The mechanism matters because it converts transient image predictions into durable, reusable memory that the agent's graph-aware transformer can reason over jointly with real observations.

What would settle it

Rerun the R2R validation-unseen evaluation replacing the imagined images with ground-truth future panoramas from the simulator; if SPL does not rise above the imagined-image version, the benefit is not coming from the fidelity of what the agent imagines.

Watch

Extended reading notes

Core claim

SALI's central discovery is that a vision-and-language navigation agent can use a 'reality-imagination hybrid memory' to plan more effectively in unseen indoor environments. The agent keeps a topological map of the building, where visited nodes store real RGB-D and semantic images and unvisited waypoints can store images the agent generates by a recurrent imagination tree: an inpaint model produces depth and semantic maps, a SPADE-based model renders the RGB view, a room-type model injects commonsense object priors, and a waypoint model proposes the next positions. The imagined nodes are fused into the map and scored together with real nodes by a graph-aware transformer, with a learned weighting that fades imagination over time. The paper reports state-of-the-art SPL on R2R and REVERIE unseen splits, with the imagination component contributing most on instructions that combine room and object terms. In effect, the agent plans from what it can picture, not only from what it has seen.

Load-bearing premise

The imagined scenes—semantic, depth, and RGB images of unvisited places—are accurate enough that adding them to the memory map improves action selection.

Editorial extensions

If this is right

  • Imagination should be stored in long-term memory, not discarded after each step; durable imagined nodes are what lift SPL on unseen environments.
  • A moderate imagination horizon of two steps and at most four imagined nodes is optimal; too many imagined nodes makes different locations compete and lowers SPL.
  • The learned dynamic weight for imagined nodes, which shrinks as navigation proceeds, beats a fixed 0.5 blend.
  • Room-type and waypoint auxiliary models each add measurable SPL, confirming that commonsense priors and position prediction contribute beyond raw image generation.
  • The biggest gains occur on instructions mixing room and object terms, indicating that imagined scenes help the agent ground relational language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If imagined views act like cheap exploration, the same hybrid memory could reduce collisions and dead-ends in continuous navigation, where the waypoint model is replaced by a real motion planner.
  • The framework suggests a principled test of generative world models for embodied AI: replace the imagined images with ground-truth future frames and measure how much of the SPL gain survives, which would isolate image fidelity from graph expansion effects.
  • The room-type dictionary is a hand-built commonsense prior; learning this mapping jointly with the navigation policy could remove the manual weights and adapt to novel room-object statistics.
  • The decreasing fusion weight resembles a memory-forgetting curve, so one could make it a function of navigation confidence rather than step index.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes SALI, a vision-and-language navigation (VLN) agent that combines a topological memory of real observations with a recurrently generated set of imagined future observations. The imagination module is composed of an inpaint model, a SPADE-based RGB generator, a room-type model, and a waypoint model, producing depth, semantic, and RGB images for potential future nodes. A multimodal transformer with graph-aware self-attention scores real and imagined nodes, and a learned dynamic fusion factor combines these scores for action selection. The authors report state-of-the-art results on R2R and REVERIE, with an 8% SPL improvement on R2R val-unseen and a 4% improvement on REVERIE unseen, and they present ablations showing consistent gains from the imagination and memory components. The central claim is that episodic simulation and episodic memory improve navigation generalization in unseen environments.

Significance. If the reported results are reliable, SALI would be a meaningful advance: it is among the first VLN agents to integrate recurrently generated future images into a persistent topological memory, and the ablation study suggests that both the imagination module and the memory mechanism contribute to navigation performance. The paper is also commendable for specifying its pre-training losses and for attempting quantitative evaluation of imagined images (PSNR, Pearson correlation). However, the central state-of-the-art claim currently rests on irreconcilable SPL numbers across tables, and no variance or seed information is provided. The significance of the contribution is therefore contingent on the authors resolving these reproducibility issues with exact evaluation protocols and per-seed results.

major comments (4)
  1. [§4.2, Table 1 vs. §4.4, Tables 2–5] The headline R2R val-unseen SPL of 78 for SALI in Table 1 is inconsistent with the ablation tables for what appears to be the same configuration. Table 2 row 3 ('Reality + Imagination') reports NE=1.92, OSR=86, SR=82, and TL=10.34 but SPL=70. Table 3 row 3 (M=2, Nbar=4) reports SR=82 and SPL=71; Table 4 row 4 and Table 5 row 1 report SR=80 and SPL=71. All of these rows correspond to the full SALI model with imagination and dynamic fusion. Identical values of NE, OSR, and SR cannot concurrently yield SPL values of 78, 70, and 71 unless different checkpoints, seeds, or evaluation protocols are being used without disclosure. Since the abstract and introduction claim an 8% SPL improvement over ScaleVLN (SPL=70) and the ablation row implies a SPL of 70, the state-of-the-art claim is not reproducible from the paper as written. Please reconcile the numbers, state the exact configuration used for Table 1, and report per-seed results.
  2. [§4.2, Tables 1–5] No error bars or multiple-seed results are reported anywhere in the experimental section. All tables present single deterministic numbers, with no standard deviation, number of runs, or statement that the benchmark evaluation is deterministic. Given the observed 8-point SPL discrepancy between Table 1 and the ablation tables, the statistical significance of the claimed improvement over prior work cannot be assessed. Please report mean and standard deviation over at least three seeds, or justify determinism of the evaluation protocol.
  3. [§3.3.2, Imagination Model Pre-training] The paper never states which environments are used to train the inpaint, room-type, SPADE, and waypoint models. The central claim is that SALI generalizes to unseen environments, but if these pre-trained imagination models were trained on the same Matterport3D scenes that appear in the R2R or REVERIE val-unseen splits, the 'unseen environment' generalization claim would be compromised. Please specify the exact train/validation/test scene split for each pre-trained component, and clarify whether any of the val-unseen environments are seen during any stage of imagination-model training.
  4. [Abstract and §4.2, Table 1] The claimed 4% improvement on REVERIE unseen is not verifiable from Table 1. The ScaleVLN baseline row has missing values ('-') for RGS and RGSPL, so no comparison is possible on the reported REVERIE metrics; moreover the abstract refers to an 'SPL' improvement on REVERIE, but REVERIE results in Table 1 do not include an SPL column. Please define the exact metric and baseline used for the 4% claim, and report the corresponding numbers for that baseline.
minor comments (7)
  1. [§2, Related Works] There is a typo in the sentence 'xisting imagination mechanisms operate in isolation'—the word 'existing' is missing its leading 'e'.
  2. [§3.3.2, Eqs. (7)–(10)] The weight parameter λ appears in Eqs. (7) and (8) and the loss weights λG, λF, λP appear in Eqs. (9) and (10), but their values or ranges are never specified. Please provide the hyperparameter values used in training.
  3. [§3.2.2, Eq. (6)] Equation (6) writes σCS(X) + (1 − σ)CF(Xt) with σ = CG(X), but the notation is inconsistent: σ appears to be both a scalar and a gating map, and X vs. Xt is used interchangeably. Please clarify the tensor shapes and the roles of CG, CS, and CF.
  4. [§4.3, Figure 6] The caption states that PSNR and Pearson correlation coefficients were calculated for imagined images and that the shaded curve represents mean and variance of pixel errors, but no numerical values are reported. Please include the actual PSNR and correlation values, or remove the claim that they were computed.
  5. [§4.1, Table 1] In the REVERIE rows of Table 1, the ScaleVLN baseline has dashes for RGS and RGSPL, making it impossible to assess the relative improvement on those metrics. Please fill in the values or explain why they are unavailable.
  6. [§3.1.1, Memory Map Representation] The text refers to 'visited nodes , current nodes , navigable nodes , and imagination nodes' but the node-type icons appear as blank spaces in the manuscript. Please ensure the figure symbols are rendered correctly.
  7. [§4.1, Implementation Details] The implementation details section reports training iterations and batch sizes but omits learning rates, optimizer settings, and any hyperparameter search details. Please provide these to facilitate reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported gains come from an independently trained imagination module and external benchmark evaluation; the Table 1 vs ablation SPL discrepancy is a consistency issue, not a circularity.

full rationale

The paper's central claim is that an imagination module, trained with auxiliary losses on ground-truth depth, semantic, RGB, and room-type labels, enriches a topological memory and improves SPL on R2R and REVERIE. The evaluation is against external benchmark splits and prior published methods. The fusion factor gamma_t is learned by an FFN from node embeddings (Eq. 3), not hand-set or fitted to the reported SPL. The imagination models are pre-trained with explicit losses (Eqs. 7-11) and then used to generate inputs for the navigation policy, so the navigation result is not definitionally equal to a training target. No load-bearing argument invokes a self-citation: the cited DUET, BEVBert, Pathdreamer, and Hong et al. waypoint model are external prior works, and none is relied on as an unverified 'uniqueness theorem' or as a substitute for the present empirical evaluation. The ablations in Tables 2-5 compare configurations and report improvements from the imagination mechanism, but these comparisons use the same benchmark metrics rather than redefining them in terms of the model's own outputs. The only notable defect is the internal inconsistency between Table 1 (SPL=78) and Tables 2-4 (SPL=70/71 for the same full configuration), which threatens reproducibility of the state-of-the-art claim; however, that is a correctness or reporting issue, not a circular derivation. No step in the paper reduces a prediction to its input by construction, and no fitted parameter is renamed as a prediction. The derivation chain is therefore self-contained for circularity purposes, and the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the fidelity of four pre-trained generative models and the hand-crafted commonsense weights; the paper does not release these models or their training data, making the assumption of accuracy load-bearing.

free parameters (4)
  • Imagination step upper bound M = 2 (best in ablation)
    Section 4.4, Table 3: M is tuned on validation; larger M (2) improves SPL but M=2, Nbar=8 degrades it.
  • Imagination node upper limit Nbar = 4 (best in ablation)
    Table 3: Nbar=4 selected; Nbar=8 reduces SPL.
  • History length K = 2
    Section 4.1: 'we set the imagination history length K to 2', a fixed design choice.
  • Object weight dictionary w = not specified
    Section 3.2.2: predefined weight dictionary for each room refines one-hot semantic codes; values are not given and are manual.
assumptions (4)
  • domain assumption The visual classifier yields reliable semantic images st for all viewpoints (following Irshad et al. 2022).
    Section 3, problem formulation and method overview assume access to a pre-trained visual classifier that produces semantic segmentation for navigation observations.
  • domain assumption The Matterport3D simulator provides ground-truth room-type labels and object annotations for pre-training the room-type model.
    Section 3.3.2 assumes access to simulator-provided one-hot room-type codes for each viewpoint.
  • standard math The graph-aware self-attention (GASA) layer, inherited from Chen et al. 2022, captures environment layout as claimed.
    Section 3.1.2 adopts GASA unchanged and does not validate it independently.
  • ad hoc to paper The predefined object weight dictionary w encodes valid commonsense associations between rooms and objects.
    Section 3.2.2 introduces w as a manually defined weight vector to refine semantic codes; no learning or external validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Planning from Imagination: Episodic Simulation and Episodic Memory for Vision-and-Language Navigation." pith.science (2026). https://pith.science/paper/2Q6JKOE6

@misc{pith2026241201857,
  author       = {Pith},
  title        = {Pith review of: Planning from Imagination: Episodic Simulation and Episodic Memory for Vision-and-Language Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2Q6JKOE6}},
  note         = {Machine review of arXiv:2412.01857}
}
read the original abstract

Humans navigate unfamiliar environments using episodic simulation and episodic memory, which facilitate a deeper understanding of the complex relationships between environments and objects. Developing an imaginative memory system inspired by human mechanisms can enhance the navigation performance of embodied agents in unseen environments. However, existing Vision-and-Language Navigation (VLN) agents lack a memory mechanism of this kind. To address this, we propose a novel architecture that equips agents with a reality-imagination hybrid memory system. This system enables agents to maintain and expand their memory through both imaginative mechanisms and navigation actions. Additionally, we design tailored pre-training tasks to develop the agent's imaginative capabilities. Our agent can imagine high-fidelity RGB images for future scenes, achieving state-of-the-art result in Success rate weighted by Path Length (SPL).

Figures

Figures reproduced from arXiv: 2412.01857 by the authors.

Figure 1
Figure 1. Humans utilize episodic memory and episodic [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We propose building a hybrid imagination-reality memory for long-term navigation decisions. Based on the navigation [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The imagination model includes four pre-trained models (inpaint model, spade model, room-type model, and way [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Before and after adding room-type model. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Comparison of SR and SPL between SALI and BEVBert models on different sub-datasets. Implementation Details. We utilize ViT-B/16 (Dosovitskiy et al. 2020) and ResNet-50 (He et al. 2016b) to extract node features ft, and adopt LXMERT (Tan and Bansal 2019) as our cross-mo…
Figure 6
Figure 6. Figure 6: Imagined picture display and waypoint prediction schematic. The imagination error is visualized by the curve. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 31 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    An, D.; Qi, Y.; Li, Y.; Huang, Y.; Wang, L.; Tan, T.; and Shao, J. 2023. Bevbert: Multimodal map pre-training for language-guided navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2737--2748

    2023

  4. [4]

    An, D.; Wang, H.; Wang, W.; Wang, Z.; Huang, Y.; He, K.; and Wang, L. 2024. Etpnav: Evolving topological planning for vision-language navigation in continuous environments. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  5. [5]

    S.; Dosovitskiy, A.; Gupta, S.; Koltun, V.; Kosecka, J.; Malik, J.; Mottaghi, R.; Savva, M.; et al

    Anderson, P.; Chang, A.; Chaplot, D. S.; Dosovitskiy, A.; Gupta, S.; Koltun, V.; Kosecka, J.; Malik, J.; Mottaghi, R.; Savva, M.; et al. 2018 a . On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757

    arXiv 2018

  6. [6]

    Anderson, P.; Wu, Q.; Teney, D.; Bruce, J.; Johnson, M.; S \"u nderhauf, N.; Reid, I.; Gould, S.; and Van Den Hengel, A. 2018 b . Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3674--3683

    work page 2018

  7. [7]

    S.; Salakhutdinov, R.; Gupta, A.; and Gupta, S

    Chaplot, D. S.; Salakhutdinov, R.; Gupta, A.; and Gupta, S. 2020. Neural topological slam for visual navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12875--12884

    work page 2020

  8. [8]

    Chen, S.; Guhur, P.-L.; Schmid, C.; and Laptev, I. 2021. History aware multimodal transformer for vision-and-language navigation. Advances in Neural Information Processing Systems, 34

    work page 2021

Show all 49 references
  1. [9]

    Chen, S.; Guhur, P.-L.; Tapaswi, M.; Schmid, C.; and Laptev, I. 2022. Think global, act local: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16537--16547

    2022

  2. [10]

    Cohen, I.; Huang, Y.; Chen, J.; Benesty, J.; Benesty, J.; Chen, J.; Huang, Y.; and Cohen, I. 2009. Pearson correlation coefficient. Noise Reduction in Speech Processing, 1--4

    2009

  3. [11]

    Cui, H.; Radosavljevic, V.; Chou, F.-C.; Lin, T.-H.; Nguyen, T.; Huang, T.-K.; Schneider, J.; and Djuric, N. 2019. Multimodal trajectory predictions for autonomous driving using deep convolutional networks. In 2019 International Conference on Robotics and Automation (ICRA), 20...

    2019

  4. [12]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    2020 arXiv

  5. [13]

    Georgakis, G.; Schmeckpeper, K.; Wanchoo, K.; Dan, S.; Miltsakaki, E.; Roth, D.; and Daniilidis, K. 2022. Cross-modal map learning for vision and language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15460--15470

    2022

  6. [14]

    Gomez, A.; Rousset, S.; and Baciu, M. 2009. Egocentric-updating during navigation facilitates episodic memory retrieval. Acta Psychologica, 132(3): 221--227

    2009

  7. [15]

    Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y. 2020. Generative adversarial networks. Communications of the ACM, 63(11): 139--144

    2020

  8. [16]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016 a . Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770--778

    2016

  9. [17]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016 b . Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778

    2016

  10. [18]

    Hong, Y.; Wang, Z.; Wu, Q.; and Gould, S. 2022. Bridging the gap between learning in discrete and continuous environments for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15439--15449

    2022

  11. [19]

    Hong, Y.; Wu, Q.; Qi, Y.; Rodriguez-Opazo, C.; and Gould, S. 2021. Vln bert: A recurrent vision-and-language bert for navigation. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, 1643--1653

    2021

  12. [20]

    Hosang, J.; Benenson, R.; and Schiele, B. 2017. Learning non-maximum suppression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4507--4515

    2017

  13. [21]

    Z.; Mithun, N

    Irshad, M. Z.; Mithun, N. C.; Seymour, Z.; Chiu, H.-P.; Samarasekera, S.; and Kumar, R. 2022. Semantically-aware spatio-temporal reasoning agent for vision-and-language navigation in continuous environments. In 2022 26th International Conference on Pattern Recognition (ICPR), ...

    2022

  14. [22]

    Kaneko, T.; and Harada, T. 2021. Blur, noise, and compression robust generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13579--13589

    2021

  15. [23]

    Kenton, J. D. M.-W. C.; and Toutanova, L. K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, 2. Minneapolis, Minnesota

    2019

  16. [24]

    Y.; Lee, H.; Yang, Y.; Baldridge, J.; and Anderson, P

    Koh, J. Y.; Lee, H.; Yang, Y.; Baldridge, J.; and Anderson, P. 2021. Pathdreamer: A world model for indoor navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 14738--14748

    2021

  17. [25]

    Korhonen, J.; and You, J. 2012. Peak signal-to-noise ratio revisited: Is simple beautiful? In 2012 Fourth International Workshop on Quality of Multimedia Experience, 37--38. IEEE

    2012

  18. [26]

    K \"u hn, S.; and Gallinat, J. 2014. Segregating cognitive functions within hippocampal formation: A quantitative meta-analysis on spatial navigation and episodic memory. Human Brain Mapping, 35(4): 1129--1142

    2014

  19. [27]

    Li, M.; Wang, Z.; Tuytelaars, T.; and Moens, M.-F. 2023. Layout-aware dreamer for embodied visual referring expression grounding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 1386--1395

    2023

  20. [28]

    Lin, C.; Jiang, Y.; Cai, J.; Qu, L.; Haffari, G.; and Yuan, Z. 2022. Multimodal transformer with variable-length memory for vision-and-language navigation. In European Conference on Computer Vision, 380--397. Springer

    2022

  21. [29]

    H.; Tan, M.; and Gan, C

    Lin, K.; Chen, P.; Huang, D.; Li, T. H.; Tan, M.; and Gan, C. 2023. Learning vision-and-language navigation from youtube videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 8317--8326

    2023

  22. [30]

    Lin, X.; Li, G.; and Yu, Y. 2021. Scene-intuitive agent for remote embodied visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7036--7045

    2021

  23. [31]

    Liu, R.; Wang, X.; Wang, W.; and Yang, Y. 2023. Bird's-eye-view scene graph for vision-language Navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10968--10980

    2023

  24. [32]

    Lu, J.; Batra, D.; Parikh, D.; and Lee, S. 2019. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in Neural Information Processing Systems, 32

    2019

  25. [33]

    Mallya, A.; Wang, T.-C.; Sapra, K.; and Liu, M.-Y. 2020. World-consistent video-to-video synthesis. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part VIII 16, 359--378. Springer

    2020

  26. [34]

    Moudgil, A.; Majumdar, A.; Agrawal, H.; Lee, S.; and Batra, D. 2021. Soat: A scene-and object-aware transformer for vision-and-language navigation. Advances in Neural Information Processing Systems, 34

    2021

  27. [35]

    Park, T.; Liu, M.-Y.; Wang, T.-C.; and Zhu, J.-Y. 2019. Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2337--2346

    2019

  28. [36]

    Phillip, I.; Jun-Yan, Z.; Tinghui, Z.; Alexei, A.; et al. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume 3

    2017

  29. [37]

    Qi, Y.; Pan, Z.; Hong, Y.; Yang, M.-H.; Van Den Hengel, A.; and Wu, Q. 2021. The road to know-where: An object-and-room informed sequential bert for indoor vision-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1655--1664

    2021

  30. [38]

    Y.; Shen, C.; and Hengel, A

    Qi, Y.; Wu, Q.; Anderson, P.; Wang, X.; Wang, W. Y.; Shen, C.; and Hengel, A. v. d. 2020. Reverie: Remote embodied visual referring expression in real indoor environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9982--9991

    2020

  31. [39]

    Qiao, Y.; Qi, Y.; Hong, Y.; Yu, Z.; Wang, P.; and Wu, Q. 2023. Hop+: History-enhanced and order-aware pre-training for vision-and-language navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7): 8524--8537

    2023

  32. [40]

    Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115: 211--252

    2015

  33. [41]

    Shah, D.; Osi \'n ski, B.; Levine, S.; et al. 2023. Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. In Conference on Robot Learning, 492--504. PMLR

    2023

  34. [42]

    Tan, H.; and Bansal, M. 2019. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490

    2019 arXiv

  35. [43]

    Tang, H.; Yan, R.; and Tan, K. C. 2017. Cognitive navigation by neuro-inspired localization, mapping, and episodic memory. IEEE Transactions on Cognitive and Developmental Systems, 10(3): 751--761

    2017

  36. [44]

    Temeltas, H.; and Kayak, D. 2008. SLAM for robot navigation. IEEE Aerospace and Electronic Systems Magazine, 23(12): 16--19

    2008

  37. [45]

    Wang, H.; Liang, W.; Van Gool, L.; and Wang, W. 2023 a . Dreamwalker: Mental planning for continuous vision-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10873--10883

    2023

  38. [46]

    Wang, T.-C.; Liu, M.-Y.; Zhu, J.-Y.; Tao, A.; Kautz, J.; and Catanzaro, B. 2018 a . High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8798--8807

    2018

  39. [47]

    Wang, X.; Xiong, W.; Wang, H.; and Wang, W. Y. 2018 b . Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation. In Proceedings of the European Conference on Computer Vision (ECCV), 37--53

    2018

  40. [48]

    Wang, Z.; Li, J.; Hong, Y.; Wang, Y.; Wu, Q.; Bansal, M.; Gould, S.; Tan, H.; and Qiao, Y. 2023 b . Scaling data generation in vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 12009--12020

    2023

  41. [49]

    Wang, Z.; Li, X.; Yang, J.; Liu, Y.; and Jiang, S. 2023 c . Gridmm: Grid memory map for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 15625--15636

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.