REVIEW 4 cited by
A Recurrent Vision-and-Language BERT for Navigation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Accuracy of many visiolinguistic tasks has benefited significantly from the application of vision-and-language(V&L) BERT. However, its application for the task of vision-and-language navigation (VLN) remains limited. One reason for this is the difficulty adapting the BERT architecture to the partially observable Markov decision process present in VLN, requiring history-dependent attention and decision making. In this paper we propose a recurrent BERT model that is time-aware for use in VLN. Specifically, we equip the BERT model with a recurrent function that maintains cross-modal state information for the agent. Through extensive experiments on R2R and REVERIE we demonstrate that our model can replace more complex encoder-decoder models to achieve state-of-the-art results. Moreover, our approach can be generalised to other transformer-based architectures, supports pre-training, and is capable of solving navigation and referring expression tasks simultaneously.
Forward citations
Cited by 4 Pith papers
-
CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models
Counterfactual language-action relabeling raises instruction-following success from about 26% to 53% in real-world navigation tests.
-
Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigation
A VLM-guided partial contrastive loss, which treats views sharing an object as positive pairs, improves visual language navigation success on R2R, REVERIE and SOON without VLM fine-tuning.
-
World-Consistent Data Generation for Vision-and-Language Navigation
A 3D-guided data augmentation method generates world-consistent panoramic training data that improves a VLN agent's performance on unseen environments over prior augmentation baselines.
-
Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation
MFRA combines a hierarchical DIRformer-style fusion backbone with instruction-guided attention and a GRU history encoder, reporting improved VLN benchmark scores.
Discussion (0). Continue with ORCID to comment.