Pith. sign in

REVIEW 3 major objections 6 minor 9 cited by

A two-level memory system—short-term visual localization and long-term goal grounding—can nearly double open-source zero-shot navigation success.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:07 UTC pith:TR74Z6DC

load-bearing objection A sensible memory-augmented navigation system, but the headline 2x gain over OpenNav is confounded: the model changes along with the memory, and no same-MLLM ablation is shown. the 3 major comments →

arxiv 2603.14807 v3 pith:TR74Z6DC submitted 2026-03-16 cs.CV cs.RO

HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory System

Kailin Lyu , Kangyi Wu , Pengna Li , Xiuyu Hu , Qingyi Si , Cui Miao , Ning Yang , Zihang Wang
show 4 more authors
Long Xiao Lianyu Hu Jingyuan Sun Ce Hao
This is my paper
classification cs.CV cs.RO
keywords vision-and-language navigationzero-shot navigationopen-source language modelshierarchical memorynavigation amnesiavisual graph memoryembodied AIcontinuous environments
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that open-source vision-language models fail at zero-shot navigation mainly because of a failure mode it names Navigation Amnesia: at short range the agent forgets where it has just been and circles back, and over long horizons it drifts away from the instruction's global intent. To fix this, it proposes HiMemVLN, a two-level memory system wrapped around a multimodal large model. A Short-Term Localer maintains an online visual graph of visited locations and uses image-embedding similarity to detect revisits and suppress redundant exploration. A Long-Term Globaler extracts a global navigation schema—primary direction, target landmark, movement pattern—from the instruction and re-anchors decisions to that schema at every step. On the standard simulated benchmark the method raises success rate from 14% to 30% and path-weighted success from 12.11 to 26.85; on real-world trials the success rate rises from 18 to 32.

Core claim

The paper's central claim is that the gap between open-source and closed-source zero-shot navigators is not mostly a reasoning-capacity gap but a memory gap. It identifies two concrete amnesia symptoms—short-term visual forgetting (local loops and redundant exploration) and long-term goal drift (decisions losing alignment with the instruction)—and argues that structured memory can compensate for the weaker implicit memory of open-source models. The proposed HiMemVLN couples a visually dominant Short-Term Localer with a semantically dominant Long-Term Globaler. The Localer stores appearance embeddings of visited viewpoints in a graph, matches the current view against that graph to localize th

What carries the argument

The load-bearing mechanism is the hierarchical memory prompt. The Short-Term Localer builds a Visual Graph Memory: each visited viewpoint becomes a node holding an appearance embedding (computed by a frozen vision encoder), the latest visit step, and a visit count, with edges connecting consecutive steps. At each step, the current panoramic observation is aggregated into one location embedding and matched by cosine similarity to existing nodes; a piecewise adaptive threshold decides whether the agent is revisiting or exploring new space. Revisited nodes get a momentum update, and candidate directions are scored for novelty so the model can be softly steered away from already-seen areas. The

Load-bearing premise

The paper's doubled success rate is attributed to the memory system, but the assumption that the same open-source multimodal model without memory would not already achieve similar results is never tested, so the causal role of memory is not isolated.

What would settle it

Run the paper's best open-source multimodal model as navigator on the same 100 simulated episodes with the same waypoint predictor, once with both memory modules, once with the Localer disabled, and once with the Globaler disabled. If the success rate without memory is already close to 30%—or if removing either module changes the result by only a point or two—the central claim that memory causes the improvement would not hold.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Open-source zero-shot navigation can reach success rates in the same range as several closed-source systems, reducing dependence on paid cloud APIs and their associated latency.
  • Because the memory is stored as graph embeddings and a compact schema rather than raw video or text history, the context provided to the language model stays roughly constant as episodes lengthen.
  • The framework is training-free: the memory modules require no task-specific fine-tuning, so new open-source multimodal models can be plugged in and compared directly.
  • The same system transfers from simulation to a physical wheeled robot without retraining, suggesting the mechanism generalizes beyond benchmark environments.
  • The ablation indicates both memory levels contribute: removing the Localer causes looping, removing the Globaler causes directional drift, and combining them gives the best results.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A same-model control—the identical base multimodal model without the memory modules—would isolate how much of the gain comes from memory versus from upgrading to a larger or visual model; the paper's comparison with a text-only older baseline leaves this open.
  • The hierarchical-memory pattern could transfer to other long-horizon embodied tasks such as object search or mobile manipulation, where re-localization and goal persistence are the same failure modes.
  • The Globaler's compressed global schema could be reused across episodes as a lightweight spatial-semantic cache, potentially lowering token costs for repeated tasks in the same environment.
  • The Localer's visual graph could be extended to persist across episodes, building a topological map that gives the agent prior knowledge on the second visit to a space.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes HiMemVLN, a zero-shot vision-and-language navigation (VLN-CE) framework built on open-source multimodal LLMs. Its main innovation is a hierarchical memory system: (1) a Short-Term Localer that maintains a visual graph memory of CLIP embeddings from visited viewpoints, detects revisits via cosine similarity, and injects soft exploration constraints into the MLLM prompt, and (2) a Long-Term Globaler that extracts a global navigation schema (primary direction, target landmark, movement pattern), tracks a CameFrom variable, and periodically injects this long-term context. The authors claim this mitigates 'Navigation Amnesia' and report SR 30 / SPL 26.85 on R2R-CE with Qwen2-VL-72B, versus SR 14 / SPL 12.11 for Open-Nav with Qwen2-72B, plus real-world deployments on a Unitree Go2W robot.

Significance. If the effect is real, HiMemVLN is a useful step toward privacy-preserving, deployment-oriented zero-shot VLN with open-source models. The paper releases code, tests four MLLMs, uses a shared waypoint predictor in simulation, and includes real-robot experiments. However, the central causal claim—that the memory system, not the base model, is responsible for the roughly 2x improvement—is currently not isolated. The headline comparison changes both the model (multimodal Qwen2-VL-72B vs text-only Qwen2-72B) and the memory modules simultaneously, and the ablation does not provide a same-backbone memory-on/off control. The contribution is defensible, but the evidence needs to be strengthened before the stated claims are supported.

major comments (3)
  1. [§V-B, Table I] The headline comparison is confounded. HiMemVLN-Qwen2-VL-72B (SR 30, SPL 26.85) is compared against Open-Nav-Qwen2-72B (SR 14, SPL 12.11), but these differ in both the base model (multimodal Qwen2-VL-72B vs text-only Qwen2-72B) and the memory system. The claimed 2x gain is attributed to hierarchical memory, yet no control runs the same Qwen2-VL-72B navigator through the OpenNav pipeline or the HiMemVLN pipeline without Localer/Globaler. This is load-bearing: model-capability alone could explain the gain. Please add a same-MLLM baseline (e.g., Qwen2-VL-72B with OpenNav-style textual prompting, no memory modules) and, if feasible, HiMemVLN on Qwen2-72B.
  2. [§V-D, Fig. 6] The ablation is presented only as a qualitative bar chart with no numeric values, and the condition 'without the multimodal LLM' removes the perception backbone rather than isolating the memory modules. To support the causal claim, report SR/SPL/OSR for: full HiMemVLN, w/o Localer, w/o Globaler, and the same Qwen2-VL-72B MLLM with neither memory module. This is necessary to quantify the marginal contribution of each memory component under a fixed base model.
  3. [§V-C] The real-world evaluation compares OpenNav SR 18/NE 4.27 against HiMemVLN SR 32/NE 3.54 but does not state which models were used, how many episodes per environment, or whether the same base model was used. Given the sim-to-real claim, please report the full protocol and a same-backbone comparison; otherwise the improvement cannot be attributed to the memory system.
minor comments (6)
  1. [Fig. 6] The bar chart lacks numeric axis labels or values. Provide a table with SR/SPL/OSR for all ablation conditions, or add value labels to the bars.
  2. [Table I] Some table entries appear misaligned or have stray spacing (e.g., '45 37', '22.00 2', '57 48'). Please reformat for readability and double-check column alignment.
  3. [§III-B / §V-A] The waypoint predictor is described as 'similar to OpenNav' and 'pretrained in [18]' (Matterport3D). Clarify the exact training data and whether this component is considered part of the zero-shot setting; if it is trained on R2R-CE data, this should be acknowledged.
  4. [§IV-B] The notation for the long-term summary window k=5 is ambiguous: does it summarize the last k actions or the last k steps including failed actions? Please define clearly.
  5. [§V-C] Only aggregate SR and NE are given for real-world experiments. Report per-environment results, number of episodes per scenario, and any stopping criterion details.
  6. [Title / Fig. 1] Several typos: 'Hi erarchical' in the title, 'Perfermance' in Fig. 1, and 'behaviors' vs 'behaviour' inconsistencies. A careful proofread is needed.

Circularity Check

0 steps flagged

No significant circularity: HiMemVLN's reported gains are empirical comparisons and not derived from fitted targets or self-citation chains.

full rationale

I walked the paper's derivation chain looking for steps where a claimed prediction or first-principles result reduces by construction to its own inputs. The central claim is that HiMemVLN achieves nearly twice the success rate of Open-Nav (SR 30 vs 14, SPL 26.85 vs 12.11, Table I). The memory system is built from external components: CLIP visual embeddings, RAM/SpatialBot scene descriptions, and prompt-based MLLM reasoning. The thresholds (θ_t initialized 0.85, α=0.15, K=5, k=5) are manually set hyperparameters, not fitted values that later reappear as predictions. No equation in the paper defines SR or SPL as a function of these hyperparameters, and no reported metric is a fitted constant. The ablation (Fig. 6) is qualitative, but this is a weakness in experimental evidence, not circularity. There is no self-citation chain that supplies a load-bearing 'uniqueness' or 'ansatz' result: the cited cognitive theory (Hippocampal Memory Indexing Theory) is used as inspiration, not as a mathematical constraint. The main confound—that Table I compares Qwen2-VL-72B with memory against Qwen2-72B without memory, changing both the model and the memory system—undermines the causal attribution of the gain to the memory components, but it does not make the reported result equivalent to an input or fitted by construction. A missing control is an experimental design concern, not a circularity. Therefore the appropriate score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

The central claim rests on hand-set thresholds and window sizes rather than a derivation, and on two domain assumptions: CLIP similarity indicates spatial revisits, and a pretrained waypoint predictor transfers. No new physical entities are introduced.

free parameters (5)
  • revisit similarity threshold theta_t = 0.85 initial, piecewise adaptive
    Sec IV-A, Eq (3): controls whether the current view is matched to an existing graph node; hand-set and central to revisit detection.
  • momentum alpha = 0.15
    Sec IV-A, Eq (4): node embedding update rate; hand-set.
  • forward-biased view weights w_d = not specified
    Sec IV-A, Eq (2): multi-view aggregation weights; 'higher weight' to forward view but exact values are not given.
  • waypoint count K = 5
    Sec III-B: top-K candidate waypoints after NMS; defines the action space.
  • long-term summary window k = 5
    Sec IV-B: number of recent steps summarized for the Globaler; hand-set.
axioms (3)
  • domain assumption CLIP embedding cosine similarity over forward-biased panorama aggregates is a reliable place-recognition signal in indoor continuous environments.
    Sec IV-A, Eq (2)-(3): the entire Short-Term Localer depends on this; visually similar but distinct locations could be conflated or revisits missed.
  • domain assumption The pretrained waypoint predictor transfers to the test environments and to the real robot without adaptation.
    Sec III-B and V-A: all decisions are made over predicted waypoints; if the predictor fails, memory cannot compensate.
  • domain assumption Open-source MLLMs can act on textualized scene descriptions plus memory prompts to make correct navigation decisions.
    Sec IV and V: the navigator is never fine-tuned; success depends on zero-shot instruction following from RAM/SpatialBot text and inserted memory notes.

pith-pipeline@v1.3.0-alltime-deepseek · 11463 in / 13521 out tokens · 131953 ms · 2026-08-02T18:07:09.781121+00:00 · methodology

0 comments
read the original abstract

LLM-based agents have demonstrated impressive zero-shot performance in vision-language navigation (VLN) tasks. However, most zero-shot methods primarily rely on closed-source LLMs as navigators, which face challenges related to high token costs and potential data leakage risks. Recent efforts have attempted to address this by using open-source LLMs combined with a spatiotemporal CoT framework, but they still fall far short compared to closed-source models. In this work, we identify a critical issue, Navigation Amnesia, through a detailed analysis of the navigation process. This issue leads to navigation failures and amplifies the gap between open-source and closed-source methods. To address this, we propose HiMemVLN, which incorporates a Hierarchical Memory System into a multimodal large model to enhance visual perception recall and long-term localization, mitigating the amnesia issue and improving the agent's navigation performance. Extensive experiments in both simulated and real-world environments demonstrate that HiMemVLN achieves nearly twice the performance of the open-source state-of-the-art method. The code is available at https://github.com/lvkailin0118/HiMemVLN.

Figures

Figures reproduced from arXiv: 2603.14807 by Ce Hao, Cui Miao, Jingyuan Sun, Kailin Lyu, Kangyi Wu, Lianyu Hu, Long Xiao, Ning Yang, Pengna Li, Qingyi Si, Xiuyu Hu, Zihang Wang.

Figure 1
Figure 1. Figure 1: (a) Comparison between GPT-based Navigator and open-source LLM-based Navigator. (b) The Phenomenon of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of HiMemVLN. Built upon a MLLM, HiMemVLN integrates short-term and long-term memory systems to accomplish task execution through a closed-loop memory-reasoning-execution process. A. Short-Term Localer System As discussed in Sec. 1, short-term Amnesia will lead to state drift and local looping issues. To address this challenge, we introduce the visually dominant Short-Term Localer system, as shown … view at source ↗
Figure 3
Figure 3. Figure 3: Workflow of the hierarchical memory system. The visually driven Short-Term Localer mimics human spatial reasoning to detect revisits and reduce redundant exploration. The semantically driven Long-Term Globaler mirrors human global reflection to preserve origin awareness and directional alignment, ensuring long-horizon consistency. where PrimaryDir represents the general directional bias, FinalTarget is the… view at source ↗
Figure 4
Figure 4. Figure 4: The Go2W robot and the arrangement of the real [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results of OpenNav and HiMemVLN in simulation environments. In contrast to the state-of-the￾art open-source method , OpenNav, which exhibits short-term and long-term amnesia reflected by looping and deviating behaviors in the red dashed boxes, our method eliminates navigation amnesia and accurately follows the given instructions. TABLE I: Comparison on simulated environment R2R￾CE. Relevant rep… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study of different components. compared to supervised learning approaches. Among them, closed-source methods benefit from stronger reasoning and contextual modeling capabilities, resulting in relatively better performance than open-source models. However, they rely on costly API calls and pose security risks. Under a strict open￾source zero-shot setting, HiMemVLN significantly outper￾forms OpenNav… view at source ↗
Figure 7
Figure 7. Figure 7: Real-world deployment of HiMemVLN in three representative indoor environments. The robot successfully executes long-horizon task sequences and demonstrates robust navigation in complex settings [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 unverdicted novelty 7.0

    Dual-Anchoring Framework mitigates progress drift via structured instruction tokens and memory drift via landmark-centric retrospective prediction, yielding 15.2% success rate gain and 24.7% on long trajectories.

  2. Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation

    cs.RO 2026-05 unverdicted novelty 6.0

    A zero-shot unified agent for VLN-CE, ObjectNav, EQA and Aerial-VLN on wheeled, quadruped, humanoid and UAV platforms that translates language and vision inputs into actions via MLLMs plus TDM and SCB mechanisms, matc...

  3. SEDualVLN: A Spatially-Enhanced Dual-System for Vision-Language Navigation

    cs.RO 2026-05 unverdicted novelty 6.0

    SEDualVLN proposes a spatially-enhanced dual-system VLN framework that pairs a fast VLM action generator with a slow MLLM waypoint planner and reports state-of-the-art results on VLN-CE benchmarks.

  4. SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigation

    cs.CV 2026-04 unverdicted novelty 6.0

    SpaAct activates spatial awareness in VLMs using action retrospection, future frame prediction, and progressive curriculum learning to reach SOTA on VLN-CE benchmarks.

  5. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 conditional novelty 6.0

    Privatar partitions VR avatar reconstruction via frequency-domain decomposition, keeping sensitive components local and offloading the rest with distribution-aware minimal perturbation noise, achieving 2.37x throughpu...

  6. HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation

    cs.RO 2026-07 conditional novelty 5.0

    A decision-coupled, hierarchical agentic memory, stored in a depth-grounded world graph, lets a zero-shot multimodal LLM navigator reach higher success rates with fewer context tokens.

  7. SEDualVLN: A Spatially-Enhanced Dual-System for Vision-Language Navigation

    cs.RO 2026-05 unverdicted novelty 5.0

    SEDualVLN introduces a spatially-enhanced dual-system VLN architecture that achieves state-of-the-art results on VLN-CE benchmarks through coordinated VLM action generation and MLLM waypoint planning.

  8. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 unverdicted novelty 5.0

    Dual-Anchoring adds explicit progress tokens and retrospective landmark verification to VLN agents, cutting state drift and lifting success rate 15.2% overall with 24.7% gains on long trajectories.

  9. Think before Go: Hierarchical Reasoning for Image-goal Navigation

    cs.RO 2026-04 unverdicted novelty 5.0

    HRNav decomposes image-goal navigation into VLM-based short-horizon planning and RL-based execution with a wandering suppression penalty to improve performance in complex unseen settings.

Reference graph

Works this paper leans on

37 extracted references · 12 linked inside Pith · cited by 6 Pith papers

  1. [1]

    Vision-and-language nav- igation: Interpreting visually-grounded navigation instructions in real environments,

    P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. S ¨underhauf, I. Reid, S. Gould, and A. Van Den Hengel, “Vision-and-language nav- igation: Interpreting visually-grounded navigation instructions in real environments,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3674–3683

    2018

  2. [2]

    Etpnav: Evolving topological planning for vision-language navigation in continuous environments,

    D. An, H. Wang, W. Wang, Z. Wang, Y . Huang, K. He, and L. Wang, “Etpnav: Evolving topological planning for vision-language navigation in continuous environments,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  3. [3]

    Bridging the gap between learning in discrete and continuous environments for vision-and- language navigation,

    Y . Hong, Z. Wang, Q. Wu, and S. Gould, “Bridging the gap between learning in discrete and continuous environments for vision-and- language navigation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 15 439–15 449

    2022

  4. [4]

    Bevbert: Multimodal map pre-training for language-guided navigation,

    D. An, Y . Qi, Y . Li, Y . Huang, L. Wang, T. Tan, and J. Shao, “Bevbert: Multimodal map pre-training for language-guided navigation,”arXiv preprint arXiv:2212.04385, 2022

    Pith/arXiv arXiv 2022

  5. [5]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  6. [6]

    Constraint-aware zero-shot vision-language navigation in continuous environments,

    K. Chen, D. An, Y . Huang, R. Xu, Y . Su, Y . Ling, I. Reid, and L. Wang, “Constraint-aware zero-shot vision-language navigation in continuous environments,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  7. [7]

    Smartway: Enhanced waypoint prediction and backtracking for zero- shot vision-and-language navigation,

    X. Shi, Z. Li, W. Lyu, J. Xia, F. Dayoub, Y . Qiao, and Q. Wu, “Smartway: Enhanced waypoint prediction and backtracking for zero- shot vision-and-language navigation,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 16 923–16 930

    2025

  8. [8]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radfordet al., “Gpt-4o system card,”arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Lawyer llama technical report,

    Q. Huang, M. Tao, C. Zhang, Z. An, C. Jiang, Z. Chen, Z. Wu, and Y . Feng, “Lawyer llama technical report,”arXiv preprint arXiv:2305.15062, 2023

    Pith/arXiv arXiv 2023

  10. [10]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Geet al., “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,”arXiv preprint arXiv:2409.12191, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Open-nav: Exploring zero-shot vision-and-language naviga- tion in continuous environment with open-source llms,

    Y . Qiao, W. Lyu, H. Wang, Z. Wang, Z. Li, Y . Zhang, M. Tan, and Q. Wu, “Open-nav: Exploring zero-shot vision-and-language naviga- tion in continuous environment with open-source llms,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 6710–6717

    2025

  12. [12]

    Navgpt: Explicit reasoning in vision- and-language navigation with large language models,

    G. Zhou, Y . Hong, and Q. Wu, “Navgpt: Explicit reasoning in vision- and-language navigation with large language models,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 7, 2024, pp. 7641–7649

    2024

  13. [13]

    Discuss before moving: Visual language navigation via multi-expert discussions,

    Y . Long, X. Li, W. Cai, and H. Dong, “Discuss before moving: Visual language navigation via multi-expert discussions,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 17 380–17 387

    2024

  14. [14]

    Deepseek llm: Scaling open-source language models with longtermism,

    X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fuet al., “Deepseek llm: Scaling open-source language models with longtermism,”arXiv preprint arXiv:2401.02954, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    A liter- ature survey on open source large language models,

    S. Kukreja, T. Kumar, A. Purohit, A. Dasgupta, and D. Guha, “A liter- ature survey on open source large language models,” inProceedings of the 2024 7th International Conference on Computers in Management and Business, 2024, pp. 133–143

    2024

  16. [16]

    Short-term memory and long-term memory are still different

    D. Norris, “Short-term memory and long-term memory are still different.”Psychological bulletin, vol. 143, no. 9, p. 992, 2017

    2017

  17. [17]

    The hippocampal memory indexing theory

    T. J. Teyler and P. DiScenna, “The hippocampal memory indexing theory.”Behavioral neuroscience, vol. 100, no. 2, p. 147, 1986

    1986

  18. [18]

    Matterport3d: Learning from rgb-d data in indoor environments,

    A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y . Zhang, “Matterport3d: Learning from rgb-d data in indoor environments,”arXiv preprint arXiv:1709.06158, 2017

    Pith/arXiv arXiv 2017

  19. [19]

    Beyond the nav-graph: Vision-and-language navigation in continuous environ- ments,

    J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee, “Beyond the nav-graph: Vision-and-language navigation in continuous environ- ments,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 104–120

    2020

  20. [20]

    Neighbor-view enhanced model for vision and language navigation,

    D. An, Y . Qi, Y . Huang, Q. Wu, L. Wang, and T. Tan, “Neighbor-view enhanced model for vision and language navigation,” inProceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 5101–5109

    2021

  21. [21]

    Language and visual entity relationship graph for agent navigation,

    Y . Hong, C. Rodriguez, Y . Qi, Q. Wu, and S. Gould, “Language and visual entity relationship graph for agent navigation,”Advances in Neural Information Processing Systems, vol. 33, pp. 7685–7696, 2020

    2020

  22. [22]

    Towards learning a generic agent for vision-and-language navigation via pre-training,

    W. Hao, C. Li, X. Li, L. Carin, and J. Gao, “Towards learning a generic agent for vision-and-language navigation via pre-training,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 13 137–13 146

    2020

  23. [23]

    Hop: History- and-order aware pre-training for vision-and-language navigation,

    Y . Qiao, Y . Qi, Y . Hong, Z. Yu, P. Wang, and Q. Wu, “Hop: History- and-order aware pre-training for vision-and-language navigation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15 418–15 427

    2022

  24. [24]

    Regnav: Room expert guided image- goal navigation,

    P. Li, K. Wu, J. Fu, and S. Zhou, “Regnav: Room expert guided image- goal navigation,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 5, 2025, pp. 4860–4868

    2025

  25. [25]

    Instructnav: Zero-shot system for generic instruction navigation in unexplored environment,

    Y . Long, W. Cai, H. Wang, G. Zhan, and H. Dong, “Instructnav: Zero-shot system for generic instruction navigation in unexplored environment,”arXiv preprint arXiv:2406.04882, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Llm-planner: Few-shot grounded planning for embodied agents with large language models,

    C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, and Y . Su, “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 2998–3009

    2023

  27. [27]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  28. [28]

    Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames,

    E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra, “Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames,”arXiv preprint arXiv:1911.00357, 2019

    Pith/arXiv arXiv 1911

  29. [29]

    Recognize anything: A strong image tagging model,

    Y . Zhang, X. Huang, J. Ma, Z. Li, Z. Luo, Y . Xie, Y . Qin, T. Luo, Y . Li, S. Liuet al., “Recognize anything: A strong image tagging model,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1724–1732

    2024

  30. [30]

    Spatialbot: Precise spatial understanding with vision lan- guage models,

    W. Cai, I. Ponomarenko, J. Yuan, X. Li, W. Yang, H. Dong, and B. Zhao, “Spatialbot: Precise spatial understanding with vision lan- guage models,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 9490–9498

    2025

  31. [31]

    Sigmoid loss for language image pre-training,

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmoid loss for language image pre-training,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 11 975–11 986

    2023

  32. [32]

    Habitat: A platform for embodied ai research,

    M. Savva, A. Kadian, O. Maksymets, Y . Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V . Koltun, J. Maliket al., “Habitat: A platform for embodied ai research,” inProceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9339–9347

    2019

  33. [33]

    Navid: Video-based vlm plans the next step for vision-and-language navigation,

    J. Zhang, K. Wang, R. Xu, G. Zhou, Y . Hong, X. Fang, Q. Wu, Z. Zhang, and H. Wang, “Navid: Video-based vlm plans the next step for vision-and-language navigation,”arXiv preprint arXiv:2402.15852, 2024

    Pith/arXiv arXiv 2024

  34. [34]

    Llava-onevision-1.5: Fully open framework for democratized multimodal training,

    X. An, Y . Xie, K. Yang, W. Zhang, X. Zhao, Z. Cheng, Y . Wang, S. Xu, C. Chen, D. Zhuet al., “Llava-onevision-1.5: Fully open framework for democratized multimodal training,”arXiv preprint arXiv:2509.23661, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Internvl3. 5: Advancing open-source multi- modal models in versatility, reasoning, and efficiency,

    W. Wang, Z. Gao, L. Gu, H. Pu, L. Cui, X. Wei, Z. Liu, L. Jing, S. Ye, J. Shaoet al., “Internvl3. 5: Advancing open-source multi- modal models in versatility, reasoning, and efficiency,”arXiv preprint arXiv:2508.18265, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    Fine-tuning the qwen2. 5-vl model for intelligent applications in the electrical domain,

    S. Yao, C. Lv, K. Zhu, and X. Qiu, “Fine-tuning the qwen2. 5-vl model for intelligent applications in the electrical domain,”EAI Endorsed Transactions on Energy Web, vol. 12, 2024

    2024

  37. [37]

    Lxmert: Learning cross-modality encoder representations from transformers,

    H. Tan and M. Bansal, “Lxmert: Learning cross-modality encoder representations from transformers,” inProceedings of the 2019 con- ference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, pp. 5100–5111. Coffee Shop Instruction: Walk straight ahead until you s...

    2019