Pith. sign in

REVIEW 3 major objections 4 minor 46 references

Fine-Grained Alignment in Vision-and-Language Navigation through Bayesian Optimization

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that replacing only the frames a Bayesian optimizer flags in a positive navigation trajectory produces fine-grained vision negatives that improve vision-language alignment.

desk verdict A plausible adversarial hard-negative mining method for VLN whose 'fine-grained' framing outruns the evidence; worth refereeing, but the authors should be pushed on mechanism and selection bias. read the letter →

arxiv 2411.14811 v2 pith:IILAWC5D submitted 2024-11-22 cs.CV cs.CLcs.LG

Yuhang Song , Mario Gianni , Chenguang Yang , Kunyang Lin , Te-Chuan Chiu , Anh Nguyen , Chun-Yi Lee This is my paper · ORCID
classification cs.CVcs.CLcs.LG
keywords Vision-and-LanguageNavigationContrastiveLearningBayesianOptimizationFine-grainedNegativeSamplingAdversarialTrainingR2RBenchmarkREVERIECross-modalEmbeddings
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that the quality of negative path-instruction pairs, not just their quantity, is what limits how well vision-and-language navigation models align visual trajectories with language. It introduces FGVLN, a Bayesian-optimization-based adversarial training loop that searches a positive path for the one or two frames whose replacement most increases the contrastive loss, then uses those replacements to build fine-grained vision negatives. On R2R in the discriminative setting, the method reaches 67.69% success on unseen scenes versus 66.70% for the Lily baseline, with a higher SPL, and it also improves SPL and grounding metrics on REVERIE. The authors argue that the resulting embeddings separate fine-grained negatives from positives far better than the baseline encoder, and that this separation is what drives the navigation gains.

What carries the argument

The load-bearing object is a binary frame mask $M$ over the $K$ frames of a positive trajectory, together with a replacement generator $F(v^+, M, x_r)$ that swaps the flagged frames for a replacement frame $x_r$, normally sampled from a different room. The inner maximization uses a TPE-based Bayesian optimizer to find the set of masks that maximize the target model's path-ranking loss; the outer minimization trains the online model against a batch augmented with the resulting fine-grained negatives. The delayed-update rule, which refreshes the target parameters every $J$ steps, keeps the inner search from chasing a moving target, and the ablation credits it with about 3% success on the unseen validation split.

What would settle it

A concrete test would be to run the full-scale R2R discriminative experiment with a random mask selector in place of the Bayesian optimizer, using identical out-domain replacement frames and the same number of additional negatives; if the random selector matches FGVLN's 67.69% unseen success within noise, the BO search is not the operative ingredient. A second test would measure the semantic distance between original and replaced paths with a vision-language embedding model or human ratings; if out-domain replacements are far from the positives, the fine-grained mechanism is not what the data show.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a training procedure that turns a positive trajectory into a set of hard, fine-grained vision negatives. A Tree-structured Parzen Estimator Bayesian optimizer scores candidate binary masks over the frames of a positive trajectory, selecting masks that maximize the path-ranking loss of a frozen target model. The masked frames are replaced with a frame drawn from a different room, producing negative paths that share most of their visual content with the positive path. These negatives are then fed to an online model in a minimax-style loop, and the target model's weights are refreshed only every J steps to stabilise the search. The paper reports that this procedure yields a vision encoder whose embeddings separate fine-grained negatives from positives more cleanly than the Lily encoder does, and that the encoder transfers to both the discriminative path-ranking setting and a recurrent generative navigation policy.

Load-bearing premise

The whole method depends on the assumption that replacing one or two frames of a positive trajectory with frames from a different room produces a negative that is still semantically close to the original path; if those replacements actually create a coarse, obviously different trajectory, then the reported gains are not caused by fine-grained alignment.

Editorial extensions

If this is right

  • In the discriminative R2R setting, FGVLN reaches 67.69% success and 0.64 SPL on unseen scenes, against 66.70% and 0.62 for Lily, while keeping trajectory length comparable.
  • In the generative R2R setting, FGVLN reports the best success rate and SPL among the compared models that are fine-tuned without the extra augmented data used by EnvDrop.
  • On REVERIE unseen scenes, FGVLN improves navigation SPL to 22.09% and remote grounding RGSPL to 19.10%, indicating gains beyond path selection.
  • Ablations show that out-domain replacement frames outperform in-domain ones, and that more Bayesian-optimizer iterations improve unseen-scene success, whereas a random mask selector with the same replacement frames does not.
  • The learned embeddings keep small L2 distances between positives and fine-grained negatives yet still separate them, implying that the encoder captures subtle visual differences that the baseline misses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: An implicit claim worth testing is that the hardness of the negative, rather than the specific identity of the replaced frame, is what drives the gain; a direct test would compare BO-selected masks against a random mask selector on the full dataset with identical replacement frames and the same number of negatives.
  • Editorial inference: Because out-domain replacement frames come from a different room, the label 'fine-grained' depends on the negative staying semantically close to the original path; measuring path-level semantic distance with a separate vision-language model would show whether the mechanism is truly fine-grained or partly a domain-mixing regularizer.
  • Editorial inference: The same mask-search-plus-replacement recipe could transfer to other contrastive sequence-alignment tasks, such as video-text retrieval or embodied instruction following, wherever a sequence can be decomposed into frames that can be masked and swapped.
  • Editorial inference: The paper's observation of more uniform attention across sequences predicts that FGVLN encoders should be more robust to single-frame perturbations, such as occlusions or noise, than the Lily encoder; this could be tested directly on the R2R validation set.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FGVLN, a Bayesian-optimization-based adversarial training framework for Vision-and-Language Navigation (VLN). The key idea is to use a TPE-based Bayesian optimizer in an inner maximization loop to select frames in a positive trajectory whose replacement produces contrastive 'fine-grained' vision negatives, then to train the model on these negatives via an outer minimization loop. The authors evaluate the learned embeddings on R2R and REVERIE in both discriminative and generative settings, reporting modest gains over the Lily baseline (e.g., R2R validation unseen SR 67.69 vs. 66.70) and providing an ablation study of BO configurations. The paper includes source code and trained models.

Significance. If the central claim is supported, the contribution is potentially useful: it introduces a principled, BO-driven mechanism for generating vision negatives in VLN contrastive learning, with a clear separation of inner and outer optimization and a delayed-update strategy. The empirical study covers two benchmarks and both discriminative and generative settings, and the release of code and models is a strength. However, the current evidence does not fully establish that the improvement is due to 'fine-grained' negatives: the paper's own best configuration uses out-domain replacement frames, and the statistical support for the headline improvements is weak, with large variances and no significance tests. The significance of the work is therefore contingent on additional validation that the proposed negatives are indeed semantically fine-grained and that the gains are not attributable to generic hard-negative augmentation or selection on the validation split.

major comments (3)
  1. [Section V-D, Table V, Eq. (4)] The central mechanism is not established by the reported experiments. Table V identifies index 7 (FGVLN-add-iter) as the best configuration, and this configuration uses out-domain replacement frames, i.e., frames sampled from a different room. The generation function in Eq. (4) replaces masked frames with such a frame, producing a trajectory that contains one or more observations from a different scene context. The paper does not provide any measure of semantic distance between the original and replaced paths, so it is unclear why these negatives should be called 'fine-grained' rather than 'hard' or 'coarse' negatives. The L2 distances in Table I have enormous standard deviations (e.g., 4.72 ± 95.79 and 7.64 ± 47.35), so they do not resolve this. To support the central claim, the authors should either define and measure fine-grainedness directly (e.g., through per-frame semantic similarity or a perceptual distance metric) or compare against an alternative hard-negative generator (e.g., random out-domain replacement without BO) to show that the BO-selected masks, rather than the out-domain operation, drive the gains.
  2. [Table I and Tables II–III] The headline navigation improvements are small and are reported without statistical confidence. On R2R validation unseen, FGVLN improves SR by 0.99 absolute (67.69 vs. 66.70) and SPL by 0.02 absolute (0.64 vs. 0.62); the paper does not report the number of seeds, standard deviations, or significance tests for any navigation metric. Table I, which is used to argue that the learned encoder better distinguishes fine-grained negatives, reports L2 means with standard deviations that exceed the mean differences (e.g., 4.72 ± 95.79 for Lily and 7.64 ± 47.35 for FGVLN). With such variance, the claim of better distinguishability is not statistically supported. Please provide multiple runs, confidence intervals, and a statistical test (e.g., bootstrap or paired test) for both the embedding distances and the navigation metrics.
  3. [Section V-D and experimental protocol] The configuration is selected on the validation unseen split and the same split is then used to report the final performance in Tables II and III. The text states: 'we select the model setting with the best performance in the unseen dataset for all our experiments.' This is a form of selection on the evaluation split, which can inflate the reported gains. The authors should either report results on a held-out test unseen split (as used in the R2R/REVERIE benchmarks) or justify why using validation unseen for both model selection and final reporting does not bias the comparison.
minor comments (4)
  1. [Section V-D] The ablation text refers to 'Table 4 of the main manuscript' when presenting Table V, and the reference to 'row 5' is inconsistent with the numbering: index 7 has a higher validation-seen SR (61.98) than index 5 (61.25), contrary to the statement that 'the model with three iterations (row 5) performed better.' Please correct the cross-references and the data interpretation.
  2. [Section IV-A, Eq. (4)] The generation function F(v+, M, xr) and the mask M are not fully specified: the paper does not state the number of frames that can be replaced (i.e., the cardinality of M) nor how xr is sampled in the in-domain versus out-domain cases. Please define the sampling procedure precisely, as this is central to the definition of a 'fine-grained' negative.
  3. [Section V-B, Figure 3] The embedding visualization in Figure 3 is qualitative and lacks axes, a dimensionality reduction method (e.g., t-SNE or PCA), and a color legend that maps dots to the three negative types. Please specify the visualization procedure so the reader can interpret the claimed overlap.
  4. [Abstract] The abstract contains grammatical errors and awkward phrasings (e.g., 'experiments on the them demonstrate'), which should be corrected. The paper would also benefit from a final proofread throughout (e.g., 'Airbert' vs. 'AirBert', 'Baysian' in the title of Section V-D).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the BO adversarial training and empirical benchmarks give the framework independent content.

full rationale

The paper proposes a min-max adversarial training framework where an inner BO loop selects frames whose replacement maximizes the contrastive loss, and an outer loop minimizes that loss on the resulting negatives. This is a standard adversarial/hard-negative mining procedure, not a derivation that reduces to its inputs. The navigation improvements are measured on R2R/REVERIE with standard metrics and compared against published baselines; no fitted parameter is inserted into a formula to produce the reported numbers. The L2 embedding-distance analysis is an evaluation of the generation mechanism, and although it is somewhat self-referential (the negatives are selected to be hard, which correlates with closeness in embedding space), it is not presented as a derivation of the navigation results. The only self-citation is the use of Lily [8] as a pretrained backbone and baseline; Kunyang Lin is a co-author of both works, but Lily is an externally published model with code, so this citation is independent support rather than a circular justification. Therefore no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The main design choices are BO iterations, number of fine-grained negatives, delayed update period, and replacement frame source. These are selected via ablation on a validation subset, not fitted to the test set. The method introduces no new simulated entities, forces, or conserved quantities.

free parameters (4)
  • Number of BO iterations R = 5 (selected via ablation)
    Ablation index 7 uses 5 iterations and gives the best val unseen SR of 56.45; 3 iterations is worse.
  • Number of fine-grained negatives b = 2
    Ablation index 6 uses 2 FGNs and improves over 1 FGN; this configuration is adopted.
  • Delayed update period J = not specified numerically
    Delayed updates improve performance by 3% on unseen validation in ablation index 4, but the exact J is not reported.
  • Replacement frame sampling strategy = out-domain (different room)
    Ablation index 5 shows out-domain replacement beats in-domain, and this choice is adopted for all experiments.
assumptions (4)
  • domain assumption Hard negative examples improve contrastive learning embeddings
    Invoked in the introduction and Section II based on refs [9]-[11].
  • domain assumption The path ranking loss L_PR is a valid measure of alignment and negative difficulty
    Eq. (2) and the inner maximization rely on L_PR as the objective; this follows from VLN-BERT formulation [6].
  • ad hoc to paper The target model's L_PR is a reliable difficulty measure for selecting frames to replace
    The inner maximization uses the target model's loss to rank masks; the authors provide no separate validation that this loss correlates with downstream navigation performance.
  • domain assumption ViLBERT and the Lily pretrained backbone are appropriate encoders
    The authors adapt the trained encoders into [41] and use Lily [8] as the pretrained model; the method depends on this backbone.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fine-Grained Alignment in Vision-and-Language Navigation through Bayesian Optimization." pith.science (2026). https://pith.science/paper/IILAWC5D

@misc{pith2026241114811,
  author       = {Pith},
  title        = {Pith review of: Fine-Grained Alignment in Vision-and-Language Navigation through Bayesian Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IILAWC5D}},
  note         = {Machine review of arXiv:2411.14811}
}
read the original abstract

This paper addresses the challenge of fine-grained alignment in Vision-and-Language Navigation (VLN) tasks, where robots navigate realistic 3D environments based on natural language instructions. Current approaches use contrastive learning to align language with visual trajectory sequences. Nevertheless, they encounter difficulties with fine-grained vision negatives. To enhance cross-modal embeddings, we introduce a novel Bayesian Optimization-based adversarial optimization framework for creating fine-grained contrastive vision samples. To validate the proposed methodology, we conduct a series of experiments to assess the effectiveness of the enriched embeddings on fine-grained vision negatives. We conduct experiments on two common VLN benchmarks R2R and REVERIE, experiments on the them demonstrate that these embeddings benefit navigation, and can lead to a promising performance enhancement. Our source code and trained models are available at: https://anonymous.4open.science/r/FGVLN.

Figures

Figures reproduced from arXiv: 2411.14811 by the authors.

Figure 1
Figure 1. An illustration of existing strategies for generat [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed Fine-grained VLN (FGVLN). In the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. A comparison of the embeddings from the vision encoder trained by different methods. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: An illustration of an example trajectory determined by our framework for a given instruction compared to that determined by Lily. Each robot starts at position 0 (marked in blue). Our framework selects a path (marked in green) that stops at the top of the stairs, while…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 37 canonical work pages

  1. [1]

    Attention is all you need,

    A. Vaswani et al. , “Attention is all you need,” Advances in neural information processing systems (NIPS) , 2017. 1, 2

    work page 2017

  2. [2]

    Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments,

    P. Anderson et al. , “Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments,” in In Proc. IEEE Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2018. 1, 2, 5, 6

    work page 2018

  3. [3]

    A survey on the application trends of home service robotics,

    G. A. Zachiotis et al. , “A survey on the application trends of home service robotics,” in In Proc. IEEE Int. Conf. on Robotics and Biomimetics (ROBIO), 2018. 1

    work page 2018

  4. [4]

    Sim-to-real transfer for vision-and-language navigation,

    P. Anderson et al. , “Sim-to-real transfer for vision-and-language navigation,” in In Proc. IEEE/CVF Int. Conf. on Comp. Vision (ICCV),

    work page

  5. [5]

    Vision-and-language navigation in the real-world,

    C. Zhao, “Vision-and-language navigation in the real-world,” Ph.D. dissertation, 2023. 1

    work page 2023

  6. [6]

    Improving vision-and-language navigation with image-text pairs from the web,

    A. Majumdar, A. Shrivastava, S. Lee, P. Anderson, D. Parikh, and D. Batra, “Improving vision-and-language navigation with image-text pairs from the web,” in In Proc. Eur. Conf. on Comp. Vision (ECCV) . Springer, 2020. 1, 2, 5

    work page 2020

  7. [7]

    Airbert: In-domain pretraining for vision-and-language navigation,

    P.-L. Guhur, M. Tapaswi, S. Chen, I. Laptev, and C. Schmid, “Airbert: In-domain pretraining for vision-and-language navigation,” in In Proc. IEEE/CVF Int. Conf. on Comp. Vision (ICCV) , 2021. 1, 2, 5, 6

    work page 2021

  8. [8]

    Learning vision-and-language navigation from youtube videos,

    K. Lin, P. Chen, D. Huang, T. H. Li, M. Tan, and C. Gan, “Learning vision-and-language navigation from youtube videos,” in In Proc. IEEE/CVF Int. Conf. on Comp. Vision (ICCV) , 2023. 1, 2, 3, 5, 7

    work page 2023

Show all 46 references
  1. [9]

    Contrastive learning with hard negative samples,

    J. Robinson, C.-Y . Chuang, S. Sra, and S. Jegelka, “Contrastive learning with hard negative samples,” arXiv:2010.04592, 2020. 1

    2010 arXiv

  2. [10]

    Ultimate negative sampling for contrastive learning,

    H. Guo and L. Shi, “Ultimate negative sampling for contrastive learning,” in In Proc. ICASSP 2023-2023 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023. 1

    2023

  3. [11]

    Multimodal contrastive learn- ing with hard negative sampling for human activity recognition,

    H. Choi, A. Beedu, and I. Essa, “Multimodal contrastive learn- ing with hard negative sampling for human activity recognition,” arXiv:2309.01262, 2023. 1

    2023 arXiv

  4. [12]

    Contrastive learning for weakly supervised phrase grounding,

    T. Gupta et al. , “Contrastive learning for weakly supervised phrase grounding,” in In Proc. Eur. Conf. on Computer Vision (ECCV) . Springer, 2020. 2

    2020

  5. [13]

    Learning to navigate un- seen environments: Back translation with environmental dropout,

    H. Tan, L. Yu, and M. Bansal, “Learning to navigate un- seen environments: Back translation with environmental dropout,” arXiv:1904.04195, 2019. 2, 6

    1904 arXiv

  6. [14]

    Sparse adversarial video attacks with spatial transformations,

    R. Mu, W. Ruan, L. S. Marcolino, and Q. Ni, “Sparse adversarial video attacks with spatial transformations,” arXiv:2111.05468, 2021. 2

    2021 arXiv

  7. [15]

    Reverie: Remote embodied visual referring expression in real indoor environments,

    Y . Qi et al. , “Reverie: Remote embodied visual referring expression in real indoor environments,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2020. 2

    2020

  8. [16]

    On evaluation of embodied navigation agents,

    P. Anderson et al. , “On evaluation of embodied navigation agents,” arXiv:1807.06757, 2018. 2

    2018 arXiv

  9. [17]

    Touchdown: Natural language navigation and spatial reasoning in visual street environments,

    H. Chen et al., “Touchdown: Natural language navigation and spatial reasoning in visual street environments,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2019. 2

    2019

  10. [18]

    Beyond the nav-graph: Vision-and-language navigation in continuous environ- ments,

    J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee, “Beyond the nav-graph: Vision-and-language navigation in continuous environ- ments,” in In Proc. Eur. Conf. on Comp. Vision (ECCV) . Springer,

  11. [19]

    Help, anna! visual navigation with natural multimodal assistance via retrospective curiosity-encouraging imitation learning,

    K. Nguyen and H. Daum ´e III, “Help, anna! visual navigation with natural multimodal assistance via retrospective curiosity-encouraging imitation learning,” in In Proc. Empirical Methods in Nat. Lang. Proc. and 9th Int. Joint Conf. on Nat. Lang. Proc. (EMNLP-IJCNLP) , K. Inui,...

    2019

  12. [20]

    Vision-based nav- igation with language-based assistance via imitation learning with indirect intervention,

    K. Nguyen, D. Dey, C. Brockett, and B. Dolan, “Vision-based nav- igation with language-based assistance via imitation learning with indirect intervention,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2019. 2

    2019

  13. [21]

    Alfred: A benchmark for interpreting grounded instructions for everyday tasks,

    M. Shridhar et al. , “Alfred: A benchmark for interpreting grounded instructions for everyday tasks,” inIn Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2020. 2

    2020

  14. [22]

    Vision- and-dialog navigation,

    J. Thomason, M. Murray, M. Cakmak, and L. Zettlemoyer, “Vision- and-dialog navigation,” in In Proc. Conf. on Robot Learning (CoRL) . PMLR, 2020. 2

    2020

  15. [23]

    Embodied concept learner: Self-supervised learning of concepts and mapping through instruction following,

    M. Ding et al. , “Embodied concept learner: Self-supervised learning of concepts and mapping through instruction following,” in In Proc. Conf. on Robot Learning (CoRL) . PMLR, 2023. 2

    2023

  16. [24]

    One step at a time: Long-horizon vision-and- language navigation with milestones,

    C. H. Song et al. , “One step at a time: Long-horizon vision-and- language navigation with milestones,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2022. 2

    2022

  17. [25]

    Chasing ghosts: Instruction following as bayesian state tracking,

    P. Anderson, A. Shrivastava, D. Parikh, D. Batra, and S. Lee, “Chasing ghosts: Instruction following as bayesian state tracking,” Advances in neural information processing systems (NIPS) , 2019. 2

    2019

  18. [26]

    Speaker-follower models for vision-and-language nav- igation,

    D. Fried et al., “Speaker-follower models for vision-and-language nav- igation,” Advances in neural information processing systems (NIPS) ,

  19. [27]

    Tactical rewind: Self-correction via backtracking in vision-and-language navigation,

    L. Ke et al. , “Tactical rewind: Self-correction via backtracking in vision-and-language navigation,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2019. 2

    2019

  20. [28]

    Self-monitoring navigation agent via auxiliary progress estimation,

    C.-Y . Ma, J. Lu, Z. Wu, G. AlRegib, Z. Kira, R. Socher, and C. Xiong, “Self-monitoring navigation agent via auxiliary progress estimation,” arXiv:1901.03035, 2019. 2

    1901 arXiv

  21. [29]

    The regretful agent: Heuristic-aided navigation through progress estimation,

    C.-Y . Ma, Z. Wu, G. AlRegib, C. Xiong, and Z. Kira, “The regretful agent: Heuristic-aided navigation through progress estimation,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) ,

  22. [30]

    Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation,

    X. Wang et al., “Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2019. 2

    2019

  23. [31]

    Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation,

    X. Wang, W. Xiong, H. Wang, and W. Y . Wang, “Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation,” in In Proc. Eur. Conf. on Comp. Vision (ECCV) , 2018. 2

    2018

  24. [32]

    Visual-language nav- igation pretraining via prompt-based environmental self-exploration,

    X. Liang, F. Zhu, L. Li, H. Xu, and X. Liang, “Visual-language nav- igation pretraining via prompt-based environmental self-exploration,” arXiv:2203.04006, 2022. 2

    2022 arXiv

  25. [33]

    Vilbert: Pretraining task- agnostic visiolinguistic representations for vision-and-language tasks,

    J. Lu, D. Batra, D. Parikh, and S. Lee, “Vilbert: Pretraining task- agnostic visiolinguistic representations for vision-and-language tasks,” Advances in neural information processing systems (NIPS) , 2019. 2, 3, 6

    2019

  26. [34]

    Object-and- action aware model for visual language navigation,

    Y . Qi, Z. Pan, S. Zhang, A. van den Hengel, and Q. Wu, “Object-and- action aware model for visual language navigation,” in In Proc. Eur. Conf. on Comp. Vision (ECCV) . Springer, 2020. 2

    2020

  27. [35]

    Vision-language navigation with self-supervised auxiliary reasoning tasks,

    F. Zhu, Y . Zhu, X. Chang, and X. Liang, “Vision-language navigation with self-supervised auxiliary reasoning tasks,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2020. 2

    2020

  28. [36]

    Hop: History- and-order aware pre-training for vision-and-language navigation,

    Y . Qiao, Y . Qi, Y . Hong, Z. Yu, P. Wang, and Q. Wu, “Hop: History- and-order aware pre-training for vision-and-language navigation,” inIn Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) ,

  29. [37]

    History aware multimodal transformer for vision-and-language navigation,

    S. Chen, P.-L. Guhur, C. Schmid, and I. Laptev, “History aware multimodal transformer for vision-and-language navigation,”Advances in neural information processing systems (NIPS) , 2021. 2

    2021

  30. [38]

    Soat: A scene-and object-aware transformer for vision-and-language navi- gation,

    A. Moudgil, A. Majumdar, H. Agrawal, S. Lee, and D. Batra, “Soat: A scene-and object-aware transformer for vision-and-language navi- gation,” Advances in Neural Information Processing Systems (NIPS) ,

  31. [39]

    Structure-encoding auxiliary tasks for improved visual representation in vision-and- language navigation,

    C.-W. Kuo, C.-Y . Ma, J. Hoffman, and Z. Kira, “Structure-encoding auxiliary tasks for improved visual representation in vision-and- language navigation,” in In Proc. IEEE/CVF Winter Conf. on Appl. of Comp. Vision (WACV) , 2023. 2

    2023

  32. [40]

    Bert: Pre- training of deep bidirectional transformers for language understand- ing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language understand- ing,” arXiv:1810.04805, 2018. 2, 6

    2018 arXiv

  33. [41]

    Vln-bert: A recurrent vision-and-language bert for navigation,

    Y . Hong et al. , “Vln-bert: A recurrent vision-and-language bert for navigation,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR), 06 2021. 2, 4, 5, 6

    2021

  34. [42]

    Tree-structured parzen estimator: Understanding its al- gorithm components and their roles for better empirical performance,

    S. Watanabe, “Tree-structured parzen estimator: Understanding its al- gorithm components and their roles for better empirical performance,” arXiv:2304.11127, 2023. 3

    2023 arXiv

  35. [43]

    Matterport3d: Learning from rgb-d data in indoor environments,

    A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y . Zhang, “Matterport3d: Learning from rgb-d data in indoor environments,” arXiv:1709.06158, 2017. 5

    2017 arXiv

  36. [44]

    Asynchronous methods for deep reinforcement learning,

    V . Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in In Proc. Int. Conf. on Machine Learning (ICML). PMLR, 2016, pp. 1928–1937. 5

    2016

  37. [45]

    Robust navigation with language pretraining and stochastic sampling,

    X. Li, C. Li, Q. Xia, Y . Bisk, A. Celikyilmaz, J. Gao, N. Smith, and Y . Choi, “Robust navigation with language pretraining and stochastic sampling,” Proc. of the EMNLP-IJCNLP , 2019. 6

    2019

  38. [46]

    Towards learning a generic agent for vision-and-language navigation via pre-training,

    W. Hao, C. Li, X. Li, L. Carin, and J. Gao, “Towards learning a generic agent for vision-and-language navigation via pre-training,” in In Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR) , 2020. 6

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.