Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Harnessing Input-Adaptive Inference for Efficient VLN

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that input-adaptive inference—masking non-navigable panorama views, early-exiting distant views with rank-based thresholds, and reusing cached embeddings—reduces computation in vision-and-language navigation (VLN) by over

desk verdict Useful empirical contribution to efficient VLN, but the headline 2x claim is not uniformly supported across the seven benchmarks. read the letter →

arxiv 2508.09262 v1 pith:UG5FT27Z submitted 2025-08-12 cs.CV cs.LG

Dongwoo Kang , Akhil Perincherry , Zachary Coalson , Aiden Gabriel , Stefan Lee , Sanghyun Hong This is my paper
classification cs.CVcs.LG
keywords input-adaptiveinferencevision-and-languagenavigationearlyexitspatiallocalitytemporallocality-sensitivehashingpanoramicviewmaskingcontinuousVLN
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the heavy computation in vision-and-language navigation (VLN) is mostly wasted on redundant visual processing, and shows how to exploit spatial and temporal structure to skip much of it. It claims that across three off-the-shelf VLN agents and seven benchmarks, an input-adaptive pipeline—masking non-navigable panorama views, early-exiting distant views with rank-based thresholds, and reusing cached embeddings of similar views—cuts computation by more than half (up to ~86% in continuous settings) while dropping success rate by an average of about 8–12%. If true, this means existing VLN models can run on resource-constrained robots at roughly half the cost without retraining, just by being smarter about which pixels to process and which computations to reuse.

What carries the argument

The k-extension masking: at each panorama, the agent fully processes only navigable views plus k neighbors on each side, zeroes out the rest, and early-exits the extended views with thresholds that decay exponentially with rank (distance from the navigable view). A SimHash cache reuses embeddings of views already seen; for continuous VLN, a scan-only subgoal generation module predicts navigable views from laser scans before any image encoding.

What would settle it

Take a VLN benchmark and add instructions that explicitly refer to objects or landmarks visible only in non-navigable, non-neighboring panorama views (e.g., 'turn toward the red tower visible outside the window'). If the agent's success rate on these instruction-conditioned episodes drops far more than the paper's average ~11.7% for the same computational savings, the spatial locality hypothesis fails. More directly, one could ablate the k-extension: if increasing k from 4 to 36 does not meaningfully improve success on such episodes, the hypothesis is supported; if it does, the reported saving

Watch

Extended reading notes

Core claim

The central discovery is that the overthinking problem in VLN is concentrated in the visual encoder, which spends ~99.5% of computation processing all 36 panoramic views at every step, and that existing input-adaptive methods (like MuE) fail because they treat views independently without respecting the spatial and temporal dependencies of navigation. The paper proposes to leverage spatial locality: only navigable views and a small k-neighborhood around them carry information needed for action selection; temporal locality: consecutive steps repeat similar views, so their embeddings can be cached and reused via locality-sensitive hashing; and a rank-based adaptive early-exit threshold that let

Load-bearing premise

The load-bearing premise is that, at each navigation step, the information needed to pick the correct action is contained in the navigable views and a small number of neighboring views, so all other panorama views can be safely masked; this has only been tested in Matterport3D/R2R-style indoor scenes, not arbitrary layouts.

Editorial extensions

If this is right

  • If the claims hold, practitioners can deploy existing VLN agents on low-power robots with 2–7.5x less compute at a success-rate cost of ~10%.
  • The method is orthogonal to model compression (quantization/pruning) and can stack with those approaches.
  • The k-extension parameter offers a tunable accuracy/efficiency knob, with k=1 yielding 74% savings at an 18% SR drop, and k=4–6 giving the reported near-2x savings.
  • The scan-only SGM extends the savings to continuous environments where navigability is not given a priori, achieving ~87% GFLOPs reduction at only 8% SR drop.
  • Robustness to visual corruption remains the weak point: under corruption, both compute and SR loss grow, suggesting the masking decisions themselves are sensitive to input quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims, one testable extension is to measure whether non-navigable views carry task-critical distant landmarks: the paper's spatial locality hypothesis would be falsified if an instruction referencing a landmark only visible in a far view leads to a large success drop.
  • The paper's robustness results imply that cache hit rates and early-exit decisions degrade under visual corruption; a simple fix like denoising (which they show recovers 17.9% SR on speckle) could be combined with their method.
  • Because the method reuses embeddings rather than recomputing them, it introduces stateful memory; the reported storage overheads (up to 2.9GB worst-case in continuous VLN) suggest that memory-limited deployments need a budget-aware eviction policy that the paper does not specify.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes an input-adaptive inference method for vision-and-language navigation (VLN), combining three mechanisms: (1) spatial masking of non-navigable panoramic views with a k-extension around navigable views, (2) importance-based adaptive early-exit thresholds applied to a batch-compatible version of the MuE early-exit method, and (3) a SimHash-based caching mechanism to reuse embeddings of previously seen similar views. The method is evaluated on seven VLN benchmarks using three off-the-shelf agents (HAMT, DUET, and VLN-CE BERT), reporting roughly a 2x reduction in GFLOPs with moderate success-rate drops, plus sensitivity analyses and robustness to visual corruptions. The central claim is that exploiting spatial and temporal locality in panoramic VLN observations can substantially reduce computation without catastrophic performance loss.

Significance. If the reported savings hold, the paper addresses a real practical bottleneck: the visual encoder accounts for 99.5% of inference FLOPs in the studied agents, and the proposed view-masking, adaptive early-exit, and caching mechanisms are portable to off-the-shelf models. The paper is also thorough in scope: it covers six discrete benchmarks and one continuous benchmark, includes per-mechanism ablations, analyzes sensitivity to key hyperparameters, and provides robustness experiments under visual corruptions. The code release is a concrete asset for reproducibility. However, the headline quantitative claim ('over 2x reduction across seven VLN benchmarks') is not uniformly supported by the reported tables, and the evaluation protocol relies on hyperparameters selected on the same validation splits used for reporting. These issues materially affect the strength of the central claim, so the paper needs revision before the results can be taken at face value.

major comments (4)
  1. [Abstract; Sec. 4, Tables 3 and 9] The headline claim 'over a 2x reduction in computation across seven VLN benchmarks' is not supported by the reported numbers. In Table 9, HAMT on R2R-Last reduces GFLOPs from 4982.68 to 2589.44, a 48.0% saving. In Table 3, DUET on REVERIE reduces from 6185.15+C to 3145.92+C, i.e., 49.1% saving even with C=0; with any positive object-feature cost C the saving is below 50%. The conclusion's '2-7.5x reduction' is therefore not uniformly true. Moreover, for REVERIE and SOON the GFLOPs exclude object-feature extraction cost C, while Sec. 4 states that the strategy is applied only to image feature extraction; the end-to-end computation reduction is not quantified. Please either restrict the claim to the benchmarks/conditions where 2x is achieved, report a measured or bounded C, or provide per-benchmark full-pipeline costs.
  2. [Sec. 4, Sec. 4.3, Tables 3, 4, 5, 6] The main hyperparameters (k, early-exit aggressiveness A, LSH similarity threshold, SimHash length) are selected on the same validation (unseen) splits used to report the headline results: 'For REVERIE, we set k=6 ... For all other benchmarks ... k=4' (Sec. 4); A=0.0009 is chosen after inspecting Table 6; LSH thresholds 0.85/0.95 are stated in Sec. 3.2.3. No held-out test split or multiple-seed evaluation is reported. This makes the efficiency/accuracy trade-off numbers optimistically biased and weakens the claim in Sec. 4.3 that the strategy is 'transferable to new settings.' I request either a separate validation set for hyperparameter selection, a leave-one-benchmark-out analysis, or a clear statement that the reported numbers are tuning results rather than predictions.
  3. [Sec. 3.2.2] The description of budgeted-batch inference is ambiguous: 'We assign a sufficiently large budget so that the mechanism can handle the worst-case complexity, where none of the samples utilize early stopping' appears to preclude any FLOP savings from early exits, yet Table 12 reports an 18% GFLOPs reduction from thresholds alone. It is unclear how per-view early exits translate into FLOP reductions in the batched ViT forward pass (do later transformer layers skip masked/early-exited views in the batch?). Please clarify the implementation; this is needed to reproduce the reported savings and to interpret the decomposition into the three mechanisms.
  4. [Sec. 3.2.1, Sec. 4, Appendix J] The spatial-locality hypothesis (only navigable views and their k-neighborhood are needed) is validated only on Matterport3D-derived benchmarks (R2R, R2R-Back, R2R-Last, REVERIE, CVDN, SOON, R2R-CE). All are indoor, panoramic, and graph- or mesh-based Matterport3D scenes. Appendix J claims generalizability to humanoid robots and autonomous driving without evidence. This is a limitation, not an error, but the abstract and conclusion should temper the claim of an 'input-adaptive navigation method' to the evaluated scene distribution, or include at least one non-Matterport3D setting (e.g., a photorealistic outdoor simulator) to support the transferability claim.
minor comments (6)
  1. [Abstract; Sec. 5] The abstract says 'over a 2x reduction' while the conclusion says '2-7.5x reduction'; these should be aligned, and given the data, 'up to 2.6x' or 'average 56%' would be more accurate than 'over 2x across seven benchmarks.'
  2. [Sec. 3.2.1, Eq. (1)] The notation for the k-extension is confusing: V_i^k and v_j^i are not clearly defined; the superscript/subscript structure should be simplified and the union over navigable views should be spelled out, including how navigable views themselves are treated.
  3. [Algorithm 1] Line 8: 'if ei does not exist' is ambiguous; it should be 'if the view vi is not in the hash table h' to distinguish between a missing hash entry and a zero embedding.
  4. [Table 6] The threshold rows are misformatted (e.g., '0.0071.0' and '0.0091.0'), making the sweep hard to read. Please add separators and clarify that A is the aggressiveness factor.
  5. [Sec. 4, Tables 3 and 9] The fact that REVERIE and SOON GFLOPs exclude object-feature extraction cost C appears only in the text; the table captions should explicitly state that the reported GFLOPs for these two benchmarks are image-feature-only, so a careless reader does not misinterpret the headline.
  6. [Appendix D, Table 11] The wall-clock savings (~40%) are considerably lower than the GFLOPs savings (~60%); a sentence in the main text would help readers understand that FLOPs are the primary metric and wall-clock is affected by implementation and memory bandwidth.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported savings are measured outcomes of transparent hyperparameter choices, with an acknowledged reporting caveat (excluded object-feature cost C) that affects the headline but is not circular.

full rationale

The paper's derivation chain is empirical rather than formal. Efficiency gains come from three mechanisms (k-extension masking, adaptive MuE thresholds, and SimHash caching) whose hyperparameters (k, A, similarity thresholds) are selected on the validation splits used for reporting; this is tuning, not a fitted parameter disguised as a prediction. The paper is explicit in Sec 4.3: 'We choose k such that an agent processes approximately half of the total views in each panorama; this results in k=4–6 for the benchmarks we consider.' It also provides per-mechanism ablations in Appendix E showing the individual contributions. The continuous-VLN component reuses the U-Net SGM from Krantz et al. [35], which shares an author (Lee), but the paper retrains it with the original procedure and reports equal validation loss ('Our scan-only SGM achieves a validation loss of 0.63 on Matterport3D scene data, the same as the original work'), making it independently reproducible rather than a load-bearing self-citation. The abstract's 'over a 2x reduction' claim is weakened by the paper's own caveat in Sec 4 that REVERIE and SOON GFLOPs exclude object-feature extraction cost C ('We then report the GFLOPs for image feature processing and treat the cost of object feature extraction as a constant (C)'), so the headline is not uniformly established across all seven benchmarks; however, this is a measurement/reporting limitation, not circularity. No equation reduces to its own input, and no self-citation is invoked to forbid alternatives or force the method choice.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central efficiency claims rest on a small number of tuned scalar parameters (k, A, T0, LSH thresholds, hash length) and on three domain assumptions about spatial redundancy, temporal redundancy, and the reliability of early-exit similarity criteria. The paper does not introduce new physical entities; its scan-only subgoal module is a reconfiguration of an existing SGM with image inputs removed.

free parameters (6)
  • k: number of extended views around each navigable view = 4 for most benchmarks, 6 for REVERIE
    Chosen per benchmark to process roughly half the views per panorama while keeping SR drop near 10%; Sec 3.2.1 and Table 5.
  • A: aggressiveness for adaptive early-exit threshold = 9e-4 (0.0009)
    Set from Table 6 sweep; text says larger values cause >10% SR drop. Text value is inconsistent with table entries (0.007-0.022).
  • T0: base early-exit threshold = 1.0
    Initial cosine-similarity threshold in Ti,j = T0 * exp(-A * Ri,j), Sec 3.2.2.
  • LSH similarity threshold = 0.85 standard, 0.95 continuous
    Hand-set to balance performance and efficiency, Sec 3.2.3.
  • SimHash length n = 10
    Hand-set hash length for binary encoding, Sec 3.2.3.
  • MuE threshold for baseline comparison = 0.998
    Optimized for best performance-efficiency trade-off in Sec 3.1/Appendix B; used to argue existing methods fail.
assumptions (5)
  • domain assumption Navigable views and their k-neighborhood contain the information needed for correct navigation decisions.
    Core of k-extension (Sec 3.2.1); empirically demonstrated on R2R but not guaranteed across environments.
  • domain assumption Views from consecutive navigation steps are similar enough that cached embeddings can be reused without changing decisions.
    Motivates LSH caching (Sec 3.2.3); relies on cosine-similarity threshold.
  • domain assumption MuE's cosine-similarity based early-exit criterion is reliable when thresholds are close to 1.0.
    Adopted from [51] and used in Sec 3.2.2/Appendix B; Fig 7 shows ViT activations do not saturate in VLN, so this assumption is fragile.
  • domain assumption GFLOPs is a valid proxy for computational cost and efficiency.
    Used throughout; wall-time shows only ~40% savings because of overheads (Appendix D), so the proxy does not fully capture real-world latency.
  • domain assumption The scan-only SGM predicts navigable views from laser scans with accuracy comparable to the image-based SGM.
    Required for continuous VLN (Sec 3.2.4); paper reports validation loss 0.63, same as original, but no navigation-level comparison with an image-based SGM is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Harnessing Input-Adaptive Inference for Efficient VLN." pith.science (2026). https://pith.science/paper/UG5FT27Z

@misc{pith2026250809262,
  author       = {Pith},
  title        = {Pith review of: Harnessing Input-Adaptive Inference for Efficient VLN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UG5FT27Z}},
  note         = {Machine review of arXiv:2508.09262}
}
abstract

An emerging paradigm in vision-and-language navigation (VLN) is the use of history-aware multi-modal transformer models. Given a language instruction, these models process observation and navigation history to predict the most appropriate action for an agent. While they have significantly improved performance, the scale of these models can be a bottleneck in practical settings with limited computational resources. In this work, we propose a novel input-adaptive navigation method to enhance VLN model efficiency. We first show that existing input-adaptive mechanisms fail to reduce computations without substantial performance degradation. To address this, we introduce three adaptive algorithms, each deployed at a different level: (1) To improve spatial efficiency, we selectively process panoramic views at each observation of an agent. (2) To improve intra-model efficiency, we propose importance-based adaptive thresholding for the early-exit methods. (3) To improve temporal efficiency, we implement a caching mechanism that prevents reprocessing of views previously seen by the agent. In evaluations on seven VLN benchmarks, we demonstrate over a 2$\times$ reduction in computation across three off-the-shelf agents in both standard and continuous environments. Our code is publicly available at https://github.com/secure-ai-systems-group/adaptive-vision-and-language-navigation.

Figures

Figures reproduced from arXiv: 2508.09262 by the authors.

Figure 1
Figure 1. Our input-adaptive, efficient navigation method. We show on the left an agent navigating a visual environment upon a natural language instruction. On the right, we provide a high-level overview of the three input-adaptive mechanisms we propose at different levels. The shaded rectangles (embeddings) and squares (views) correspond to components that our method skips or zeroes out to improve efficiency. igable views fr… view at source ↗
Figure 2
Figure 2. Problems in employing existing input-adaptive methods in VLN. We show that employing existing strategies leads to performance loss and an increase in computations. (Left) The increase in computations stems from inappropriate navigation actions, and (Right) such decisions come from the inaccurate understanding of the visual world, e.g., the agent confuses where to navigate. so the information they contain should suff… view at source ↗
Figure 3
Figure 3. Our masking and thresholding. The top figure shows how we mask non-navigable views, and the bottom figure shows how we adaptively assign the exit thresholds of MuE. To understand this issue, we analyze cases where masking non-navigable views prevents the agent from reaching the target. In [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Comparison of baseline and our agent trajectories under Spatter corruption. We demonstrate that our agent fails to stop at the target location, resulting in incorrect navigation (Right), whereas the baseline agent successfully stops as instructed (Left). Clean Motion B…
Figure 5
Figure 5. Figure 5: Examples of the visual corruptions we consider. Agent Corruption Performance TL(Ó) OSR(Ò) SR(Ò) SPL(Ò) GFLOPs(Ó) HAMT None 11.53 74.29 66.16 61.49 4763.24 Spatter 13.30 69.82 58.71 52.91 5227.36 Defocus Blur 13.87 66.50 55.21 49.32 5383.35 Speckle Noise 13.60 62.88 51.…
Figure 6
Figure 6. Figure 6: Comparison of perfor￾mance (in SR) and GFLOPs in MuE across different thresholds [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 8
Figure 8. Figure 8: Attention visualization across different exit thresholds on HAMT. Lower thresholds use fewer ViT layers. 0 4 8 12 16 20 24 k 0 20 40 60 80 L2 Distance Visual Language History [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 10
Figure 10. Figure 10: Cosine similarity of different views from R2R. Com￾parisons are made between the upper and lower images. language embeddings, which encode the navigation instruc￾tions, are also notably impacted. This further explains the performance degradation: if the agent does not…
Figure 9
Figure 9. Figure 9: L2 distance of cross-modal embeddings from the base￾line and our efficient HAMT agent for different k values. Embed￾dings are computed on 100 navigation instructions from R2R. understanding. However, this change is minimal, and the overall navigation outcome is unaffec…
Figure 11
Figure 11. Figure 11: Trade-off between Performance (SR) and GFLOPs. plied to the HAMT agent achieves significant computational savings. For reference, the baseline HAMT model achieves a SR of 66.16 with a computational cost of 4763.24 GFLOPs [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness

    cs.RO 2026-03 conditional novelty 7.0 of 10

    VLN-Cache delivers up to 1.52x faster inference in VLN models by using view-aligned remapping for geometric consistency and a task-relevance saliency filter to manage semantic changes during navigation.

Reference graph

Works this paper leans on

68 extracted references · 56 canonical work pages · cited by 1 Pith paper

  1. [1]

    Vision-and-language navigation: In- terpreting visually-grounded navigation instructions in real environments

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko S ¨underhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-and-language navigation: In- terpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 2, 5, 12, 14

    work page 2018

  2. [2]

    Sim- to-real transfer for vision-and-language navigation

    Peter Anderson, Ayush Shrivastava, Joanne Truong, Arjun Majumdar, Devi Parikh, Dhruv Batra, and Stefan Lee. Sim- to-real transfer for vision-and-language navigation. In Con- ference on Robot Learning , pages 671–681. PMLR, 2021. 18

    work page 2021

  3. [3]

    Near-optimal hashing algo- rithms for approximate nearest neighbor in high dimensions

    Alexandr Andoni and Piotr Indyk. Near-optimal hashing algo- rithms for approximate nearest neighbor in high dimensions. Communications of the ACM, 51(1):117–122, 2008. 5, 13

    work page 2008

  4. [4]

    Post train- ing 4-bit quantization of convolutional networks for rapid- deployment

    Ron Banner, Yury Nahshan, and Daniel Soudry. Post train- ing 4-bit quantization of convolutional networks for rapid- deployment. Advances in Neural Information Processing Systems, 32, 2019. 17

    work page 2019

  5. [5]

    Surf: Speeded up robust features

    Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. In Computer Vision–ECCV 2006: 9th European Conference on Computer Vision, Graz, Aus- tria, May 7-13, 2006. Proceedings, Part I 9, pages 404–417. Springer, 2006. 7, 16, 17

    work page 2006

  6. [6]

    Lsq+: Improving low-bit quantization through learnable offsets and better initialization

    Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak. Lsq+: Improving low-bit quantization through learnable offsets and better initialization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , pages 696–697,

    work page

  7. [7]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Hal- ber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. International Conference on 3D Vision (3DV), 2017. 2, 12

    work page 2017

  8. [8]

    Similarity estimation techniques from rounding algorithms

    Moses S Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of the thiry-fourth an- nual ACM symposium on Theory of computing, pages 380– 388, 2002. 5, 13

    work page 2002

Show all 68 references
  1. [9]

    Robustnav: Towards benchmark- ing robustness in embodied navigation

    Prithvijit Chattopadhyay, Judy Hoffman, Roozbeh Mottaghi, and Aniruddha Kembhavi. Robustnav: Towards benchmark- ing robustness in embodied navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15691–15700, 2021. 7, 8

    2021

  2. [10]

    History aware multimodal transformer for vision- and-language navigation

    Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. History aware multimodal transformer for vision- and-language navigation. Advances in neural information processing systems, 34:5834–5847, 2021. 1, 2, 5, 6, 12, 14

    2021

  3. [11]

    Think global, act lo- cal: Dual-scale graph transformer for vision-and-language navigation

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Think global, act lo- cal: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16537– 1...

    2022

  4. [12]

    Pact: Parameterized clipping activation for quantized neural networks

    Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085,

    arXiv

  5. [13]

    Low-bit quantization of neural networks for efficient infer- ence

    Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural networks for efficient infer- ence. In 2019 IEEE/CVF International Conference on Com- puter Vision Workshop (ICCVW), pages 3009–3018. IEEE,

    2019

  6. [14]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013. 5

    2013

  7. [15]

    BERT: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human L...

    2019

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  9. [17]

    Reducing transformer depth on demand with structured dropout

    Angela Fan, Edouard Grave, and Armand Joulin. Reducing transformer depth on demand with structured dropout. arXiv preprint arXiv:1909.11556, 2019. 17

    1909 arXiv

  10. [18]

    Depgraph: Towards any structural pruning

    Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, and Xinchao Wang. Depgraph: Towards any structural pruning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16091–16101, 2023. 17

    2023

  11. [19]

    Spatially adaptive computation time for residual networks

    Michael Figurnov, Maxwell D Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, and Ruslan Salakhutdinov. Spatially adaptive computation time for residual networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1039–1048, 2017. 2

    2017

  12. [20]

    Speaker-follower models for vision-and-language navigation

    Daniel Fried, Ronghang Hu, V olkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg- Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. Speaker-follower models for vision-and-language navigation. Advances in neural information processing systems, ...

    2018

  13. [21]

    Airbert: In-domain pretraining for vision-and-language navigation

    Pierre-Louis Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, and Cordelia Schmid. Airbert: In-domain pretraining for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1634–1643, 2021. 1

    2021

  14. [22]

    Improving robust- ness of vision transformers by reducing sensitivity to patch corruptions

    Yong Guo, David Stutz, and Bernt Schiele. Improving robust- ness of vision transformers by reducing sensitivity to patch corruptions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4108–4118,

  15. [23]

    Deep com- pression: Compressing deep neural networks with pruning, trained quantization and huffman coding

    Song Han, Huizi Mao, and William J Dally. Deep com- pression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015. 17

    2015 arXiv

  16. [24]

    Learning both weights and connections for efficient neural network

    Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015. 17

    2015

  17. [25]

    Towards learning a generic agent for vision- and-language navigation via pre-training

    Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. Towards learning a generic agent for vision- and-language navigation via pre-training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13137–13146, 2020. 1

    2020

  18. [26]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6, 12

    2016

  19. [27]

    Benchmarking neural network robustness to common corruptions and perturbations

    Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations ,

  20. [28]

    Revisiting pruning at ini- tialization through the lens of ramanujan graph

    Duc NM Hoang and Shiwei Liu. Revisiting pruning at ini- tialization through the lens of ramanujan graph. ICLR 2023,

    2023

  21. [29]

    A recurrent vision-and-language bert for navigation

    Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazo, and Stephen Gould. A recurrent vision-and-language bert for navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1643–1653, 2021. 1, 2

    2021

  22. [30]

    Bridg- ing the gap between learning in discrete and continuous envi- ronments for vision-and-language navigation

    Yicong Hong, Zun Wang, Qi Wu, and Stephen Gould. Bridg- ing the gap between learning in discrete and continuous envi- ronments for vision-and-language navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15439–15449, 2022. 5

    2022

  23. [31]

    Multi-scale dense networks for resource efficient image classification

    Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Weinberger. Multi-scale dense networks for resource efficient image classification. In Inter- national Conference on Learning Representations, 2018. 1, 2, 4, 6

    2018

  24. [32]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceed- ings of the IEEE conference on computer vision ...

    2018

  25. [33]

    A new path: Scaling vision-and-language navigation with synthetic instructions and imitation learning,

    Aishwarya Kamath, Peter Anderson, Su Wang, Jing Yu Koh, Alexander Ku, Austin Waters, Yinfei Yang, Jason Baldridge, and Zarana Parekh. A new path: Scaling vision-and-language navigation with synthetic instructions and imitation learning,

  26. [34]

    Shallow-deep networks: Understanding and mitigating net- work overthinking

    Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. Shallow-deep networks: Understanding and mitigating net- work overthinking. In International conference on machine learning, pages 3301–3310. PMLR, 2019. 1, 2, 6

    2019

  27. [35]

    Sim-2-sim transfer for vision- and-language navigation in continuous environments

    Jacob Krantz and Stefan Lee. Sim-2-sim transfer for vision- and-language navigation in continuous environments. In European Conference on Computer Vision, pages 588–603. Springer, 2022. 1, 5, 6, 7, 12

    2022

  28. [36]

    Beyond the nav-graph: Vision and language navigation in continuous environments

    Jacob Krantz, Erik Wijmans, Arjun Majundar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision and language navigation in continuous environments. In European Confer- ence on Computer Vision (ECCV), 2020. 2, 5, 6, 12

    2020

  29. [37]

    Improving vision-and-language navigation by generating future-view image semantics

    Jialu Li and Mohit Bansal. Improving vision-and-language navigation by generating future-view image semantics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10803–10812, 2023. 2

    2023

  30. [38]

    Brecq: Pushing the limit of post-training quantization by block reconstruction

    Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Brecq: Pushing the limit of post-training quantization by block reconstruction. arXiv preprint arXiv:2102.05426, 2021. 17

    2021 arXiv

  31. [39]

    Fastbert: a self-distilling bert with adaptive inference time

    Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Haotang Deng, and Qi Ju. Fastbert: a self-distilling bert with adaptive inference time. arXiv preprint arXiv:2004.02178, 2020. 1, 6

    2004 arXiv

  32. [40]

    Relaxed quanti- zation for discretized neural networks

    Christos Louizos, Matthias Reisser, Tijmen Blankevoort, Efstratios Gavves, and Max Welling. Relaxed quanti- zation for discretized neural networks. arXiv preprint arXiv:1810.01875, 2018. 17

    2018 arXiv

  33. [41]

    Distinctive image features from scale- invariant keypoints

    David G Lowe. Distinctive image features from scale- invariant keypoints. International journal of computer vision, 60:91–110, 2004. 7, 16, 17

    2004

  34. [42]

    Pruning convolutional neural networks for resource efficient inference

    Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440,

    arXiv

  35. [43]

    Soat: A scene-and object-aware transformer for vision-and-language navigation

    Abhinav Moudgil, Arjun Majumdar, Harsh Agrawal, Ste- fan Lee, and Dhruv Batra. Soat: A scene-and object-aware transformer for vision-and-language navigation. Advances in Neural Information Processing Systems, 34:7357–7367, 2021. 1

    2021

  36. [44]

    Up or down? adap- tive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Chris- tos Louizos, and Tijmen Blankevoort. Up or down? adap- tive rounding for post-training quantization. In International Conference on Machine Learning, pages 7197–7206. PMLR,

  37. [45]

    Gradient- free structured pruning with unlabeled data

    Azade Nova, Hanjun Dai, and Dale Schuurmans. Gradient- free structured pruning with unlabeled data. In Interna- tional Conference on Machine Learning, pages 26326–26341. PMLR, 2023. 17

    2023

  38. [46]

    Do vi- sual imaginations improve vision-and-language navigation agents? In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3846–3855, 2025

    Akhil Perincherry, Jacob Krantz, and Stefan Lee. Do vi- sual imaginations improve vision-and-language navigation agents? In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3846–3855, 2025. 2

    2025

  39. [47]

    Reverie: Remote embodied visual referring expression in real indoor environments

    Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. Reverie: Remote embodied visual referring expression in real indoor environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2020

  40. [48]

    Orb: An efficient alternative to sift or surf

    Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. Orb: An efficient alternative to sift or surf. In 2011 International conference on computer vision , pages 2564–

    2011

  41. [49]

    Habitat: A platform for embodied ai research

    Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF international conference on computer vision ,...

    2019

  42. [50]

    P4q: Learning to prompt for quantization in visual-language models

    Huixin Sun, Runqi Wang, Yanjing Li, Xianbin Cao, Xiaolong Jiang, Yao Hu, and Baochang Zhang. P4q: Learning to prompt for quantization in visual-language models. arXiv preprint arXiv:2409.17634, 2024. 17

    2024 arXiv

  43. [51]

    S. Tang, Y . Wang, Z. Kong, T. Zhang, Y . Li, C. Ding, Y . Wang, Y . Liang, and D. Xu. You need multiple exiting: Dynamic early exiting for accelerating unified vision language model. In 2023 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 10781–...

    2023

  44. [52]

    Branchynet: Fast inference via early exiting from deep neural networks

    Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR), pages 2464–2469. IEEE, 2016. 2

    2016

  45. [53]

    Vision-and-dialog navigation

    Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. Vision-and-dialog navigation. In Conference on Robot Learning, pages 394–406. PMLR, 2020. 5, 12, 14

    2020

  46. [54]

    Mixed precision dnns: All you need is a good parametrization

    Stefan Uhlich, Lukas Mauch, Fabien Cardinaux, Kazuki Yoshiyama, Javier Alonso Garcia, Stephen Tiedemann, Thomas Kemp, and Akira Nakamura. Mixed precision dnns: All you need is a good parametrization. arXiv preprint arXiv:1905.11452, 2019. 17

    1905 arXiv

  47. [55]

    Efficientvlm: Fast and accurate vision-language mod- els via knowledge distillation and modal-adaptive pruning

    Tiannan Wang, Wangchunshu Zhou, Yan Zeng, and Xinsong Zhang. Efficientvlm: Fast and accurate vision-language mod- els via knowledge distillation and modal-adaptive pruning. arXiv preprint arXiv:2210.07795, 2022. 17

    2022 arXiv

  48. [56]

    Skipnet: Learning dynamic routing in convolu- tional networks

    Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. Skipnet: Learning dynamic routing in convolu- tional networks. In Proceedings of the European conference on computer vision (ECCV), pages 409–424, 2018. 2

    2018

  49. [57]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 7, 17

    2004

  50. [58]

    Scaling data generation in vision-and-language navigation

    Zun Wang, Jialu Li, Yicong Hong, Yi Wang, Qi Wu, Mohit Bansal, Stephen Gould, Hao Tan, and Yu Qiao. Scaling data generation in vision-and-language navigation. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 12009–12020, 2023. 2

    2023

  51. [59]

    Wang et al

    Z. Wang et al. Sim-to-real transfer via 3d feature fields for vision-and-language navigation. CoRL, 2024. 18

    2024

  52. [60]

    Wasserman et al

    J. Wasserman et al. Last-mile embodied visual navigation. CoRL, 2023. 18

    2023

  53. [61]

    Deebert: Dynamic early exiting for accelerating bert inference

    Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, and Jimmy Lin. Deebert: Dynamic early exiting for accelerating bert inference. arXiv preprint arXiv:2004.12993, 2020. 1, 2, 6

    2004 arXiv

  54. [62]

    Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution

    Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han, Shenzhi Wang, Shiji Song, Jiashi Feng, and Gao Huang. Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution. Advances in Neural Information Processing Systems, 37:56619–56643, 2024. 2

    2024

  55. [63]

    Fsim: A feature similarity index for image quality assessment.IEEE transactions on Image Processing, 20(8):2378–2386, 2011

    Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. Fsim: A feature similarity index for image quality assessment.IEEE transactions on Image Processing, 20(8):2378–2386, 2011. 7, 17

    2011

  56. [64]

    Zhang et al

    Q. Zhang et al. Humanoidpano: Hybrid spherical panoramic- lidar cross-modal perception for humanoid robots. arXiv,

  57. [65]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. 7, 17

    2018

  58. [66]

    Soon: Scenario oriented object nav- igation with graph-based exploration

    Fengda Zhu, Xiwen Liang, Yi Zhu, Qizhi Yu, Xiaojun Chang, and Xiaodan Liang. Soon: Scenario oriented object nav- igation with graph-based exploration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12689–12699, 2021. 5, 12, 14

    2021

  59. [67]

    Learning unforeseen robustness from out-of-distribution data using equivariant domain translator

    Sicheng Zhu, Bang An, Furong Huang, and Sanghyun Hong. Learning unforeseen robustness from out-of-distribution data using equivariant domain translator. In Proceedings of the 40th International Conference on Machine Learning, pages 42915–42937. PMLR, 2023. 8

    2023

  60. [68]

    Zhu et al

    S. Zhu et al. Vigor: Cross-view image geo-localization be- yond one-to-one retrieval. CVPR, 2021. 18 A. Experimental Setup in Detail We describe the experimental setup used to evaluate our input-adaptive inference mechanism in detail. We imple- mented our strategy on top of th...

    2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.