Pith. sign in

REVIEW 3 major objections 5 minor 66 references

Efficient and Generalizable Environmental Understanding for Visual Navigation

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that visual navigation is governed by single-step causal links and that adding a one-step next-observation prediction loss to an EmbCLIP-style agent nearly doubles its success metrics.

desk verdict A cheap next-state prediction loss consistently improves EmbCLIP-style navigation, but the causal derivation does not connect to the loss and the EmbCLIP baseline is used inconsistently across the paper. read the letter →

arxiv 2506.15377 v1 pith:KSAYMEQM submitted 2025-06-18 cs.AI

Ruoyu Wang , Xinshu Li , Chen Wang , Lina Yao This is my paper · ORCID
classification cs.AI
keywords visualnavigationcausalunderstandingnext-statepredictionauxiliarylossembodiedAIreinforcementlearningvision-languageobject
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual navigation agents typically model their whole observation history with RNNs or Transformers, but this paper argues that in navigation the next observation is caused almost entirely by the current observation and the current action, so long-range dependencies are indirect and weak. The paper's central claim is that adding a Causal Understanding Module to an EmbCLIP-style agent, a simple CLIP-embedding policy, enforces this one-step causal structure and yields large gains. The module is a linear layer plus an auxiliary loss that predicts the next observation's features from the current observation and action. With it, success-weighted path length rises from 0.18 to 0.31 on RoboTHOR object navigation and from 0.07 to 0.16 on Habitat object navigation, while training time drops by over an order of magnitude. The same module also improves three supervised visual-language navigation baselines on R2R.

What carries the argument

The Causal Understanding Module is the load-bearing mechanism: a small linear network that takes the current observation feature and action embedding and outputs a prediction of the next observation feature, trained with the loss $L_{\text{causal}}(\theta) = \mathbb{E}_t[(\mu_\theta - h_{t+1})^2]$. This loss is derived from an estimate of the conditional mutual information $I(O_t; a_{t-1} \mid O_{t-1})$ using lower and upper bounds on the KL divergence between a Gaussian and a mixture of Gaussians. The total objective is $L_{\text{total}} = L_{\text{PPO}} - \alpha L_{\text{causal}}$, which forces the agent to be accurate about one-step state transitions, the direct causal edge the paper identifies as the only strong dependency in navigation.

What would settle it

Run EmbCLIP from its official implementation for the same 100 million training steps under identical seeds and hyperparameters and check whether the baseline reproduces the paper's numbers; or train the Transformer-only ablation (CAN without the causal loss) under identical conditions and verify that it matches EmbCLIP. If the baseline turns out stronger than reported, or if the ablation matches CAN, then the causal module's contribution to the results is not established.

Watch

Extended reading notes

Core claim

The paper's discovery is that navigation data has a causal structure in which all direct causal edges are single-step: the transition (observation, action) leads to the next observation, and (observation, objective) leads to the action. Consequently there are no direct causal links between observations separated by more than one step, so long-term associations are mediated and comparatively weak. CAN exploits this by estimating the conditional mutual information between the previous action and the next observation given the current observation, approximated by a Gaussian-mixture KL objective and realized as a mean-squared-error loss between a predicted next-observation feature and the actual next-observation feature. This causal loss, subtracted from the PPO objective or appended in supervised training, is shown by ablations to be the component responsible for most of the observed performance gains.

Load-bearing premise

The reported gains assume that the EmbCLIP baseline was faithfully re-run under identical training conditions, with the same number of steps, seeds, and hyperparameters, and that the only difference between CAN and the Transformer ablation is the Causal Understanding Module; if the baseline is weaker than the original, the headline improvements are inflated.

Editorial extensions

If this is right

  • Adding the Causal Understanding Module to EmbCLIP nearly doubles success-weighted path length on RoboTHOR object navigation (SPL 0.18 to 0.31) and Habitat object navigation (SPL 0.07 to 0.16), with success rate rising from 0.42 to 0.73 on RoboTHOR.
  • The module reduces training time by more than an order of magnitude: a Causal-RNN reaches 0.48 success rate in 15 million steps, whereas EmbCLIP needs 200 million steps to reach 0.47.
  • The same module improves three supervised visual-language navigation methods on R2R, raising EnvDrop's success rate from 52.2% to 68.7% and cutting navigation error by roughly 1.5 points.
  • Because the module is a single linear layer plus a regression loss, it adds no computational overhead at inference time and can be dropped into existing sequential navigation policies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the single-step causality prior is the real driver, the module's benefit should shrink in environments with moving obstacles or other agents, where the next observation depends on more than the current observation and action; testing CAN in dynamic scenes would sharpen the boundary of the claim.
  • The causal loss is effectively a one-step forward model, so the results suggest that any auxiliary next-state prediction objective might capture a large share of the benefit, independent of the causal derivation.
  • The success on R2R, a language-guided task, hints that the one-step causal prior may transfer to other embodied tasks with dense temporal structure, such as active perception or manipulation, but the paper does not test those settings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Causality-Aware Navigation (CAN), a visual navigation agent built on EmbCLIP-style frozen CLIP features, a multimodal transformer, and a Causal Understanding Module. The module adds an auxiliary loss L_causal(θ)=E_t[(μθ−h_{t+1})^2] intended to predict the next observation feature from the current observation and action. The paper motivates this with a causal framework (Assumptions 1–2 and Propositions II.1–II.3) and claims that the loss estimates the conditional mutual information between the action and the next observation. It reports large gains over EmbCLIP in RoboTHOR ObjNav, Habitat ObjNav, and Habitat PointNav, and over Seq2Seq, Speaker Follower, and EnvDrop on R2R in the supervised setting.

Significance. If the empirical results are reproducible under controlled comparisons, the paper makes a useful and simple contribution: a single auxiliary next-state prediction loss that improves both RL and supervised navigation agents without architectural overhead. The breadth of evaluation (three simulator tasks plus R2R), the use of external benchmarks and error bars in Table I, and the module-level ablation are strengths. However, the theoretical derivation connecting conditional mutual information to the implemented MSE loss is incomplete, and the headline comparison to EmbCLIP is not fully controlled as reported. These issues must be resolved before the paper's central claims can be accepted.

major comments (3)
  1. [Section III, Eq. (6)] The paper does not derive L_causal from the conditional mutual information that it introduces. After stating that the KL divergence between a Gaussian and a mixture of Gaussians is intractable and that the midpoint of bounds from [10] is used as the objective, Eq. (6) simply defines a mean squared error between μθ and h_{t+1}. No algebraic step or approximation is given that converts the bound estimator into this MSE, and Eq. (4) is not a valid maximum-likelihood objective because the product over N is missing and the expression is not normalized over the data. Additionally, the Monte-Carlo estimate in Eq. (5) samples actions from the dataset D without importance weighting for P(a|O_{t-1}), and gθ is written as a function of O_t rather than O_{t-1}, making the estimator inconsistent. The theoretical justification and the implemented loss are therefore disconnected; the authors should either supply the missing derivation or present the module as a next-state prediction auxiliary task and not claim it estimates the proposed causal quantity.
  2. [Section IV-A, Section IV-C, Fig. 6] The paper uses EmbCLIP in two incompatible ways. Section IV-A states that both methods are trained for the same number of steps under identical settings, and the Table I caption says EmbCLIP results are obtained as described in Section IV-A for a fair comparison. Yet Section IV-C states that EmbCLIP, as reported in [4], reaches 0.47 only after 200M steps, and uses this to claim a 10x training-time reduction for Causal-RNN at 15M steps. If the Table I EmbCLIP row were a controlled rerun under identical settings, the 200M-step published number would be irrelevant; if it is the published number, then the Table I comparison is not a controlled rerun. No code, seeds, checkpoints, or training curves are provided to disambiguate. Because the headline gains (e.g., RoboTHOR SPL 0.18 to 0.31, Habitat ObjNav SPL 0.07 to 0.16) depend on the EmbCLIP baseline, this must be fixed with a fully specified controlled comparison.
  3. [Section IV-D, Table III] The supervised learning comparison is not adequately controlled. The table reports large improvements over Seq2Seq, Speaker Follower, and EnvDrop, but the text gives no training protocol, number of steps, or hyperparameters for the '+Ours' variants and no error bars. If the baseline numbers are taken from prior papers rather than rerun under identical settings, the improvement could reflect different training budgets rather than the module itself. Please specify the protocol or provide reruns under identical conditions.
minor comments (5)
  1. [Section III] There are typos in the manuscript, including 'datasaetD' in the paragraph before Eq. (5) and 'benefitial' in the Table III caption.
  2. [Eq. (4)] The maximum-likelihood expression is missing the product over the N samples; as written it is not a proper likelihood for a dataset.
  3. [Fig. 6] The caption says the values are averages over 10 random runs, but no error bars or per-run variance are shown, so the reader cannot assess the variability of the training curves.
  4. [Table II] No error bars are reported for the ablation rows, whereas Table I includes ± values; this makes it hard to judge whether the ablation differences are significant.
  5. [Eq. (6)] The notation h_{t+1} is introduced without a definition; please clarify that it refers to the next visual feature from the frozen CLIP encoder and not the raw observation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the auxiliary next-state loss is tested on held-out metrics, and no load-bearing step reduces to its inputs.

full rationale

The paper's central claim is that adding the Causal Understanding Module (Eq. 6, an MSE next-state prediction loss) to PPO or supervised baselines improves navigation performance. This is evaluated on held-out episodes in RoboTHOR, Habitat, and R2R; the auxiliary loss is computed on training transitions and is not fitted to the evaluation metrics. The causal-framework discussion (Section II-C) motivates the loss via conditional mutual information, but Eq. 6 is presented as an implementation choice ('In practice, we implement a causal loss as defined in Equation 6'), so the theoretical-to-practical step is a heuristic approximation, not an identity that makes the outcome equivalent to the input. The paper relies on external baselines (EmbCLIP [4]) and an external KL-bound reference [10]; there are no load-bearing self-citations by the present authors. The only flagged issue is methodological: Table I says EmbCLIP results are obtained under identical settings, while Fig. 6 uses EmbCLIP's published 200M-step number for a training-time comparison. That is a fairness/reproducibility concern, not a circular derivation where a prediction reduces to its inputs. Accordingly, no circular step is exhibited and the circularity score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The method is essentially an auxiliary next-state prediction loss on top of EmbCLIP. The causal graph assumptions are not needed to define L_causal, which reduces to an MSE; the only tuned free parameter is the loss weight α.

free parameters (1)
  • α (causal loss weight) = 1
    Set by hand in Section IV-A with no sensitivity analysis. The reported gains depend on this auxiliary loss being weighted by 1 in Eq. (7).
assumptions (4)
  • domain assumption Assumption 1: At any time step t, observation O_t and action a_t are the sole direct causal parents of O_{t+1}.
    Introduced in Section II-C as the basis for the causal graph and for prioritizing one-step associations. It is not proven and sits in tension with Proposition II.1, which claims non-Markov dependence on the full history.
  • domain assumption Assumption 2: At any time step t, observation O_t and the Objective are the only direct causes of action a_t.
    Introduced in Section II-C and used to justify the Causal Feature Integration Module. It is asserted without evidence.
  • domain assumption Normality assumption: P(O_t|O_{t-1}, a_{t-1}) is Gaussian with mean and variance output by a neural network fθ.
    Stated in Section III without justification. The final loss is an MSE that ignores the variance term, so the Gaussian form is not actually used.
  • domain assumption Monte-Carlo approximation of P(O_t|O_{t-1}) by a K-sample mixture from the dataset.
    Section III, Eq. (5). Assumes samples are representative and ignores dependence structure among samples.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient and Generalizable Environmental Understanding for Visual Navigation." pith.science (2026). https://pith.science/paper/KSAYMEQM

@misc{pith2026250615377,
  author       = {Pith},
  title        = {Pith review of: Efficient and Generalizable Environmental Understanding for Visual Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KSAYMEQM}},
  note         = {Machine review of arXiv:2506.15377}
}
read the original abstract

Visual Navigation is a core task in Embodied AI, enabling agents to navigate complex environments toward given objectives. Across diverse settings within Navigation tasks, many necessitate the modelling of sequential data accumulated from preceding time steps. While existing methods perform well, they typically process all historical observations simultaneously, overlooking the internal association structure within the data, which may limit the potential for further improvements in task performance. We address this by examining the unique characteristics of Navigation tasks through the lens of causality, introducing a causal framework to highlight the limitations of conventional sequential methods. Leveraging this insight, we propose Causality-Aware Navigation (CAN), which incorporates a Causal Understanding Module to enhance the agent's environmental understanding capability. Empirical evaluations show that our approach consistently outperforms baselines across various tasks and simulation environments. Extensive ablations studies attribute these gains to the Causal Understanding Module, which generalizes effectively in both Reinforcement and Supervised Learning settings without computational overhead.

Figures

Figures reproduced from arXiv: 2506.15377 by the authors.

Figure 1
Figure 1. A key characteristic of Navigation is that [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Capturing long-term dependencies, such as [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The workflow of our proposed method. First, we process the observations by the frozen CLIP vision model and [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison on RoboTHOR ObjNav Find an Alarm￾Clock task. (a) EmbCLIP stops far away from the goal object, making it fail to complete the task. (b) Our method stops at a closer spot, thus benefiting the performance. real-world indoor scenes from MatterPort3D [16]. (3) Po…
Figure 6
Figure 6. Figure 6: Average Success Rate for EmbCLIP and Causal-RNN [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 24 canonical work pages

  1. [10]

    Lower and upper bounds for approximation of the kullback-leibler divergence between gaussian mixture models,

    J.-L. Durrieu, J.-P. Thiran, and F. Kelly, “Lower and upper bounds for approximation of the kullback-leibler divergence between gaussian mixture models,” in2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Ieee, 2012, pp. 4833–4836

    work page 2012

  2. [4]

    Simple but effective: Clip embeddings for embodied ai,

    A. Khandelwal, L. Weihs, R. Mottaghi, and A. Kembhavi, “Simple but effective: Clip embeddings for embodied ai,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 14 829–14 838

    2022

  3. [1]

    Core challenges in embodied vision-language planning,

    J. Francis, N. Kitamura, F. Labelle, X. Lu, I. Navarro, and J. Oh, “Core challenges in embodied vision-language planning,”Journal of Artificial Intelligence Research, vol. 74, pp. 459–515, 2022

    2022

  4. [2]

    Long short-term memory,

    S. Hochreiter, “Long short-term memory,”Neural Computation MIT- Press, 1997

    1997

  5. [3]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  6. [5]

    Optimal control of markov processes with incomplete state information i,

    K. J. ˚Astr¨om, “Optimal control of markov processes with incomplete state information i,”Journal of mathematical analysis and applications, vol. 10, pp. 174–205, 1965

    work page 1965

  7. [6]

    Markov decision processes,

    M. L. Puterman, “Markov decision processes,”Handbooks in operations research and management science, vol. 2, pp. 331–434, 1990

    work page 1990

  8. [7]

    T. M. Cover,Elements of information theory. John Wiley & Sons, 1999

    1999

Show all 66 references
  1. [8]

    Causal influence detection for improving efficiency in reinforcement learning,

    M. Seitzer, B. Sch ¨olkopf, and G. Martius, “Causal influence detection for improving efficiency in reinforcement learning,”Advances in Neural Information Processing Systems, vol. 34, pp. 22 905–22 918, 2021

    2021

  2. [9]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  3. [11]

    Causal curiosity: Rl agents discovering self-supervised experiments for causal representation learning,

    S. A. Sontakke, A. Mehrjou, L. Itti, and B. Sch ¨olkopf, “Causal curiosity: Rl agents discovering self-supervised experiments for causal representation learning,” inInternational conference on machine learning. PMLR, 2021, pp. 9848–9858

    2021

  4. [12]

    Curiosity-driven exploration by self-supervised prediction,

    D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervised prediction,” inInternational conference on machine learning. PMLR, 2017, pp. 2778–2787

    2017

  5. [13]

    Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation,

    X. Wang, W. Xiong, H. Wang, and W. Y . Wang, “Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation,” inProceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 37–53

    2018

  6. [14]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  7. [15]

    Robothor: An open simulation-to-real embodied ai platform,

    M. Deitke, W. Han, A. Herrasti, A. Kembhavi, E. Kolve, R. Mottaghi, J. Salvador, D. Schwenk, E. VanderBilt, M. Wallingford,et al., “Robothor: An open simulation-to-real embodied ai platform,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,...

    2020

  8. [16]

    Matterport3d: Learning from rgb-d data in indoor environments,

    A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y . Zhang, “Matterport3d: Learning from rgb-d data in indoor environments,”arXiv preprint arXiv:1709.06158, 2017

    2017 arXiv

  9. [17]

    Gibson env: Real-world perception for embodied agents,

    F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese, “Gibson env: Real-world perception for embodied agents,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9068–9079

    2018

  10. [18]

    Procthor: Large- scale embodied ai using procedural generation,

    M. Deitke, E. VanderBilt, A. Herrasti, L. Weihs, K. Ehsani, J. Salvador, W. Han, E. Kolve, A. Kembhavi, and R. Mottaghi, “Procthor: Large- scale embodied ai using procedural generation,”Advances in Neural Information Processing Systems, vol. 35, pp. 5982–5994, 2022

    2022

  11. [19]

    Stubborn: A strong baseline for indoor object navigation,

    H. Luo, A. Yue, Z.-W. Hong, and P. Agrawal, “Stubborn: A strong baseline for indoor object navigation,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 3287–3293

    2022

  12. [20]

    Thda: Treasure hunt data augmentation for semantic navigation,

    O. Maksymets, V . Cartillier, A. Gokaslan, E. Wijmans, W. Galuba, S. Lee, and D. Batra, “Thda: Treasure hunt data augmentation for semantic navigation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15 374–15 383

    2021

  13. [21]

    Habitat-web: Learning embodied object-search strategies from human demonstrations at scale,

    R. Ramrakhya, E. Undersander, D. Batra, and A. Das, “Habitat-web: Learning embodied object-search strategies from human demonstrations at scale,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5173–5183

    2022

  14. [22]

    Auxiliary tasks and exploration enable objectgoal navigation,

    J. Ye, D. Batra, A. Das, and E. Wijmans, “Auxiliary tasks and exploration enable objectgoal navigation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 16 117–16 126

    2021

  15. [23]

    Pirlnav: Pretraining with imitation and rl finetuning for objectnav,

    R. Ramrakhya, D. Batra, E. Wijmans, and A. Das, “Pirlnav: Pretraining with imitation and rl finetuning for objectnav,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 896–17 906

    2023

  16. [24]

    Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames,

    E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra, “Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames,”arXiv preprint arXiv:1911.00357, 2019

    1911 arXiv

  17. [25]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  18. [26]

    Allenact: A framework for embodied ai research,

    L. Weihs, J. Salvador, K. Kotar, U. Jain, K.-H. Zeng, R. Mottaghi, and A. Kembhavi, “Allenact: A framework for embodied ai research,” arXiv preprint arXiv:2008.12760, 2020

    2008 arXiv

  19. [27]

    Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments,

    P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. S ¨underhauf, I. Reid, S. Gould, and A. Van Den Hengel, “Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments,” inProceedings of the IEEE conference on computer vision...

    2018

  20. [28]

    Speaker- follower models for vision-and-language navigation,

    D. Fried, R. Hu, V . Cirik, A. Rohrbach, J. Andreas, L.-P. Morency, T. Berg-Kirkpatrick, K. Saenko, D. Klein, and T. Darrell, “Speaker- follower models for vision-and-language navigation,”Advances in neural information processing systems, vol. 31, 2018

    2018

  21. [29]

    Learning to navigate unseen environ- ments: Back translation with environmental dropout,

    H. Tan, L. Yu, and M. Bansal, “Learning to navigate unseen environ- ments: Back translation with environmental dropout,”arXiv preprint arXiv:1904.04195, 2019

    1904 arXiv

  22. [30]

    Room- across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding,

    A. Ku, P. Anderson, R. Patel, E. Ie, and J. Baldridge, “Room- across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding,”arXiv preprint arXiv:2010.07954, 2020

    2010 arXiv

  23. [31]

    Reverie: Remote embodied visual referring expression in real indoor environments,

    Y . Qi, Q. Wu, P. Anderson, X. Wang, W. Y . Wang, C. Shen, and A. v. d. Hengel, “Reverie: Remote embodied visual referring expression in real indoor environments,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9982–9991

    2020

  24. [32]

    History aware multimodal transformer for vision-and-language navigation,

    S. Chen, P.-L. Guhur, C. Schmid, and I. Laptev, “History aware multimodal transformer for vision-and-language navigation,”Advances in neural information processing systems, vol. 34, pp. 5834–5847, 2021

    2021

  25. [33]

    Gpf-bg: A hierarchical vision-based planning framework for safe quadrupedal navigation,

    S. Feng, Z. Zhou, J. S. Smith, M. Asselmeier, Y . Zhao, and P. A. Vela, “Gpf-bg: A hierarchical vision-based planning framework for safe quadrupedal navigation,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 1968–1975

    2023

  26. [34]

    Topo- logical planning with transformers for vision-and-language navigation,

    K. Chen, J. K. Chen, J. Chuang, M. V ´azquez, and S. Savarese, “Topo- logical planning with transformers for vision-and-language navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 11 276–11 286

    2021

  27. [35]

    Think global, act local: Dual-scale graph transformer for vision-and-language navigation,

    S. Chen, P.-L. Guhur, M. Tapaswi, C. Schmid, and I. Laptev, “Think global, act local: Dual-scale graph transformer for vision-and-language navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16 537–16 547

    2022

  28. [36]

    Gridmm: Grid memory map for vision-and-language navigation,

    Z. Wang, X. Li, J. Yang, Y . Liu, and S. Jiang, “Gridmm: Grid memory map for vision-and-language navigation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 15 625–15 636

    2023

  29. [37]

    Target-driven structured transformer planner for vision- language navigation,

    Y . Zhao, J. Chen, C. Gao, W. Wang, L. Yang, H. Ren, H. Xia, and S. Liu, “Target-driven structured transformer planner for vision- language navigation,” inProceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 4194–4203

    2022

  30. [38]

    Kerm: Knowledge enhanced reasoning for vision-and-language navigation,

    X. Li, Z. Wang, J. Yang, Y . Wang, and S. Jiang, “Kerm: Knowledge enhanced reasoning for vision-and-language navigation,” inProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2583–2592

    2023

  31. [39]

    Zero-shot object goal visual navigation,

    Q. Zhao, L. Zhang, B. He, H. Qiao, and Z. Liu, “Zero-shot object goal visual navigation,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 2025–2031

    2023

  32. [40]

    Loc-zson: Language-driven object-centric zero-shot object retrieval and navigation,

    T. Guan, Y . Yang, H. Cheng, M. Lin, R. Kim, R. Madhivanan, A. Sen, and D. Manocha, “Loc-zson: Language-driven object-centric zero-shot object retrieval and navigation,”arXiv preprint arXiv:2405.05363, 2024

    2024 arXiv

  33. [41]

    Aligning knowledge graph with visual perception for object-goal navigation,

    N. Xu, W. Wang, R. Yang, M. Qin, Z. Lin, W. Song, C. Zhang, J. Gu, and C. Li, “Aligning knowledge graph with visual perception for object-goal navigation,”arXiv preprint arXiv:2402.18892, 2024

    2024 arXiv

  34. [42]

    Prompter: Utilizing large language model prompting for a data efficient embodied instruction following,

    Y . Inoue and H. Ohashi, “Prompter: Utilizing large language model prompting for a data efficient embodied instruction following,”arXiv preprint arXiv:2211.03267, 2022

    2022 arXiv

  35. [43]

    A persistent spatial semantic representation for high-level natural language instruction execution,

    V . Blukis, C. Paxton, D. Fox, A. Garg, and Y . Artzi, “A persistent spatial semantic representation for high-level natural language instruction execution,” inConference on Robot Learning. PMLR, 2022, pp. 706–717

    2022

  36. [44]

    A survey of embodied ai: From simulators to research tasks,

    J. Duan, S. Yu, H. L. Tan, H. Zhu, and C. Tan, “A survey of embodied ai: From simulators to research tasks,”IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 6, no. 2, pp. 230–244, 2022

    2022

  37. [45]

    Navgpt: Explicit reasoning in vision- and-language navigation with large language models,

    G. Zhou, Y . Hong, and Q. Wu, “Navgpt: Explicit reasoning in vision- and-language navigation with large language models,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 7, 2024, pp. 7641–7649

    2024

  38. [46]

    Discuss before moving: Visual language navigation via multi-expert discussions,

    Y . Long, X. Li, W. Cai, and H. Dong, “Discuss before moving: Visual language navigation via multi-expert discussions,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 17 380–17 387

    2024

  39. [47]

    Mapgpt: Map-guided prompting for unified vision-and-language navigation,

    J. Chen, B. Lin, R. Xu, Z. Chai, X. Liang, and K.-Y . K. Wong, “Mapgpt: Map-guided prompting for unified vision-and-language navigation,” arXiv preprint arXiv:2401.07314, 2024

    2024 arXiv

  40. [48]

    Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill,

    W. Cai, S. Huang, G. Cheng, Y . Long, P. Gao, C. Sun, and H. Dong, “Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 5228–5234

    2024

  41. [49]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat,et al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  42. [50]

    Langnav: Language as a perceptual representation for navigation,

    B. Pan, R. Panda, S. Jin, R. Feris, A. Oliva, P. Isola, and Y . Kim, “Langnav: Language as a perceptual representation for navigation,” arXiv preprint arXiv:2310.07889, 2023

    2023 arXiv

  43. [51]

    Navcot: Boosting llm-based vision-and-language navigation via learning disentangled reasoning,

    B. Lin, Y . Nie, Z. Wei, J. Chen, S. Ma, J. Han, H. Xu, X. Chang, and X. Liang, “Navcot: Boosting llm-based vision-and-language navigation via learning disentangled reasoning,”arXiv preprint arXiv:2403.07376, 2024

    2024 arXiv

  44. [52]

    Towards learning a generalist model for embodied navigation,

    D. Zheng, S. Huang, L. Zhao, Y . Zhong, and L. Wang, “Towards learning a generalist model for embodied navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 13 624–13 634

    2024

  45. [53]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar,et al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  46. [54]

    Interventional few-shot learning,

    Z. Yue, H. Zhang, Q. Sun, and X.-S. Hua, “Interventional few-shot learning,”Advances in neural information processing systems, vol. 33, pp. 2734–2746, 2020

    2020

  47. [55]

    Causalgan: Learning causal im- plicit generative models with adversarial training,

    M. Kocaoglu, C. Snyder,et al., “Causalgan: Learning causal im- plicit generative models with adversarial training,”arXiv preprint arXiv:1709.02023, 2017

    2017 arXiv

  48. [56]

    Causal reinforcement learning: A survey,

    Z. Deng, J. Jiang, G. Long, and C. Zhang, “Causal reinforcement learning: A survey,”arXiv preprint arXiv:2307.01452, 2023

    2023 arXiv

  49. [57]

    Causal confusion in imitation learning,

    P. De Haan, D. Jayaraman, and S. Levine, “Causal confusion in imitation learning,”Advances in neural information processing systems, vol. 32, 2019

    2019

  50. [58]

    The essential role of causality in foundation world models for embodied ai,

    T. Gupta, W. Gong, C. Ma, N. Pawlowski, A. Hilmkil, M. Scetbon, A. Famoti, A. J. Llorens, J. Gao, S. Bauer,et al., “The essential role of causality in foundation world models for embodied ai,”arXiv preprint arXiv:2402.06665, 2024

    2024 arXiv

  51. [59]

    Learning invariant representations for reinforcement learning without reconstruction,

    A. Zhang, R. McAllister, R. Calandra, Y . Gal, and S. Levine, “Learning invariant representations for reinforcement learning without reconstruction,”arXiv preprint arXiv:2006.10742, 2020

    2006 arXiv

  52. [60]

    Causal world models by unsupervised deconfounding of physical dynamics,

    M. Li, M. Yang, F. Liu, X. Chen, Z. Chen, and J. Wang, “Causal world models by unsupervised deconfounding of physical dynamics,”arXiv preprint arXiv:2012.14228, 2020

    2012 arXiv

  53. [61]

    Intrinsically motivated learning of causal world models,

    L. Annabi, “Intrinsically motivated learning of causal world models,” arXiv preprint arXiv:2208.04892, 2022

    2022 arXiv

  54. [62]

    Resolving spurious correlations in causal models of environments via interventions,

    S. V olodin, N. Wichers, and J. Nixon, “Resolving spurious correlations in causal models of environments via interventions,”arXiv preprint arXiv:2002.05217, 2020

    2002 arXiv

  55. [63]

    Offline reinforcement learning with causal structured world models,

    Z.-M. Zhu, X.-H. Chen, H.-L. Tian, K. Zhang, and Y . Yu, “Offline reinforcement learning with causal structured world models,”arXiv preprint arXiv:2206.01474, 2022

    2022 arXiv

  56. [64]

    Causal dynam- ics learning for task-independent state abstraction,

    Z. Wang, X. Xiao, Z. Xu, Y . Zhu, and P. Stone, “Causal dynam- ics learning for task-independent state abstraction,”arXiv preprint arXiv:2206.13452, 2022

    2022 arXiv

  57. [65]

    Model- invariant state abstractions for model-based reinforcement learning,

    M. Tomar, A. Zhang, R. Calandra, M. E. Taylor, and J. Pineau, “Model- invariant state abstractions for model-based reinforcement learning,” arXiv preprint arXiv:2102.09850, 2021

    2021 arXiv

  58. [66]

    Robust agents learn causal world models,

    J. Richens and T. Everitt, “Robust agents learn causal world models,” arXiv preprint arXiv:2402.10877, 2024

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.