{"id":"645432c3-1efe-4b15-95c2-a66ff1a0e679","arxiv_id":"2506.10756","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VLFly integrates LLaMA, CLIP, and a pretrained ViNT waypoint planner to guide a drone by matching language instructions to goal images, reporting 83% success on direct and 70% on indirect real-world instructions.","lead":"A drone navigation system called VLFly uses a large language model to translate instructions like \"fly where a student can keep textbooks\" into a target image, then steers a camera-only drone toward that image. The authors report strong success rates in simulated and real indoor and outdoor flights, but the comparison misses key baselines and the goal list is predefined.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No full-VLN baseline comparison: Table I tests navigation only, while real-world tests of language understanding have no baselines; 'outperforms all baselines' is unsupported for the claimed task.","rationale":"VLFly is a coherent modular system and the real flights with 83%/70% success are a genuine feasibility demonstration; I would not dismiss it. But the claim 'consistently outperforms all baselines' is a quantitative comparison claim, and the paper's only quantitative comparison (Table I) is, by the paper's own description, a navigation-only test in which baselines receive a goal position and language grounding is excluded. If full VLFly was evaluated with language in Table I, the comparison is unfair; if it was evaluated without language, Table I does not test VLFly as proposed. Either way the result cannot support the central claim. The real-world experiments show the system can follow instructions, but without baselines or error bars they cannot establish relative superiority. This is why the reader's ViNT-isolation concern matters: ViNT already provides goal-image navigation, so the novel modules must be tested against a version of VLFly with oracle goal selection, and the full pipeline must be compared against language-capable baselines on identical instructions. Those experiments are feasible and would settle the claim, which is why I keep the conditional verdict rather than rejecting: the contribution is presented clearly and the missing evidence is obtainable, not conceptually impossible. The self-admitted predefined-image-pool limitation (Section V) should also be brought into the abstract's 'open-vocabulary' language.","tokens_in":12896,"tokens_out":9169,"duration_ms":103038,"concrete_test":"Run a simulation benchmark in the furniture and barrier environments where every method receives the same task input: a natural-language instruction and the same candidate goal-image pool, for both direct and indirect instructions (e.g., Section IV-C's instruction types). Compare full VLFly (LLM+CLIP+ViNT) against (i) Seq2Seq and CMA fed the same language instructions, (ii) Hybrid-APF given the ground-truth goal image, and (iii) VLFly with an oracle goal image (bypassing LLM+CLIP). Report SR and SPL over 200 episodes per condition. If full VLFly does not beat the language-capable baselines, or if oracle-goal VLFly performs as well as full VLFly, the superiority attributed to the VLFly integration is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of the paper is that VLFly, as the LLM+CLIP+ViNT integration, 'consistently outperforms all baselines' on open-vocabulary VLN. The strongest experimental support, Table I, does not test that claim. Section IV-B explicitly says 'all baselines are evaluated solely on their navigation performance in simulation, without involving any language grounding components' and that the UAV is given a specified goal position; the language-driven open-vocabulary capability is deferred to the real-world section. So Table I compares VLFly (or, if language is disabled there, its waypoint planner) against navigation-only baselines on an image-goal/navigation task, not on instruction-following. Conversely, Section IV-C, which is the only evaluation of natural-language goal grounding, contains no baselines: only VLFly is flown, with 20 trials per condition. Therefore no experiment compares the full system with the baselines on the same VLN task, and the headline 'consistently outperforms all baselines' is unsupported for the claimed task. The conclusion's own admission that goal retrieval 'depends on a predefined image pool, limiting adaptability in open-world environments' further qualifies the open-vocabulary claim, but the missing head-to-head VLN comparison is the load-bearing gap: it prevents attributing any observed navigation competence to the proposed modular integration rather than to the pretrained ViNT planner.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VLFly, a modular zero-shot vision-and-language navigation (VLN) framework for UAVs. VLFly combines an LLM-based instruction encoder (LLaMA) that reformulates natural language into structured prompts, a CLIP-based goal retriever that matches the prompt to a predefined pool of goal images, and a ViNT-based waypoint planner that outputs continuous velocity commands from egocentric monocular RGB observations. The authors claim that VLFly, without any fine-tuning, outperforms all baselines in simulation and achieves robust open-vocabulary goal understanding in real-world indoor and outdoor flight tests. The manuscript presents simulation results in three Unity environments, real-world experiments with a Tello Edu drone, and ablation studies of the three modules.","tokens_in":13169,"tokens_out":4472,"duration_ms":50463,"significance":"If the central claim were fully supported, VLFly would be a valuable demonstration that off-the-shelf foundation models (LLM, CLIP, ViNT) can be integrated into a zero-shot, continuous-control UAV navigation pipeline. The real-world flights with egocentric monocular input and no external localization are a positive contribution, as is the use of continuous velocity commands rather than discrete actions. The paper also gives credit for releasing materials via a project page. However, the current evidence does not establish the headline claim of consistently outperforming all baselines on VLN, because the simulation comparison omits language grounding for the baselines and the real-world language experiments have no baselines. The lack of a ViNT-only ablation makes the specific contribution of the proposed modules unclear.","major_comments":[{"comment":"The central claim that VLFly 'consistently outperforms all baselines' is not supported by the experiments as designed. Section IV-B explicitly states that 'all baselines are evaluated solely on their navigation performance in simulation, without involving any language grounding components' and that the UAV is given a specified goal position. Thus Table I compares navigation-only performance, not vision-and-language navigation. The only instruction-driven evaluation, in Section IV-C, reports no baselines. Consequently, there is no experiment that compares the full VLFly system with the baselines on the same VLN task. This is load-bearing because the paper's main contribution is the language-to-navigation integration, so the headline claim should either be supported by a full VLN comparison or qualified accordingly.","section":"Section IV-B, Table I, and Abstract"},{"comment":"The waypoint planner is ViNT, a pretrained visual navigation model described in [34], used without any fine-tuning. ViNT itself is never evaluated as a standalone baseline or as an ablation of VLFly. Without comparing VLFly to ViNT given the same retrieved goal image (or a ground-truth goal image), the observed navigation performance cannot be attributed to the proposed instruction encoding and goal retrieval modules rather than to ViNT's pretrained navigation competence. The ablation 'w/ RL Policy' in Table II replaces the waypoint planner, but it does not isolate ViNT's contribution to the integrated system, leaving a significant attribution gap.","section":"Section III-C3 and Table II"},{"comment":"The box environment is excluded from the trajectory-based evaluation because 'it is primarily used for training certain baselines.' Table I shows that PPO and NPE-DRL outperform VLFly in that environment (SR 90.4 and 92.2 vs. 86.4). While the text qualifies the statement to 'unseen environments,' the abstract and conclusion state that VLFly 'consistently outperforms all baselines' without this qualification. The authors should either include a fully fair comparison in the training environment and discuss the outcome, or consistently restrict the claim to unseen environments throughout the paper.","section":"Section IV-B and Figure 3"},{"comment":"The 'open-vocabulary goal understanding' claim is constrained by the design of the evaluation. The LLM prompt in Figure 2 restricts selection to a list of 100 common daily-life objects plus user-specified custom items, and the goal retriever selects from a predefined image pool. The conclusion acknowledges this limitation ('the goal retrieval module depends on a predefined image pool, limiting adaptability in open-world environments'). Given this explicit limitation, the abstract's unqualified 'open-vocabulary' wording should be tempered, or the paper should demonstrate retrieval from a dynamically generated candidate set to substantiate the stronger claim.","section":"Section IV-C, Figure 2, and Conclusion"}],"minor_comments":[{"comment":"The real-world success threshold is set to 0.8 m while the simulation threshold is 0.5 m; please justify this difference or report the sensitivity of the success rate to the threshold choice.","section":"Section IV-C"},{"comment":"The phrase 'under indirect instreuctions' contains a typo; it should read 'instructions'.","section":"Section IV-D"},{"comment":"The caption notes that the box environment is excluded from trajectory evaluation, but this information appears only in the caption; consider adding a corresponding statement in the main text of Section IV-B where the environments are introduced.","section":"Figure 3 caption"},{"comment":"The text contains a formatting issue: 'UA V' appears with a space; also in Section IV-C 'VLFLy' should be 'VLFly'.","section":"Section III-C3"},{"comment":"Since ViNT [34] is a core component, the paper should describe its pretraining data and domain more explicitly when discussing the transfer from ground robots to UAVs, to support the claim that the architecture is robust without adaptation.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a promising integration of LLM, CLIP, and ViNT, and the real-world flights are a useful existence proof. The major issue is experimental: the claimed superiority over baselines is not demonstrated for the full VLN task, and the contribution of the ViNT planner is not isolated. These are fixable by adding a ViNT-only baseline, a full VLN comparison in simulation, and a clear separation of the navigation-only and language-grounded evaluations. The predefined image pool and custom item list also need to be framed as a limitation in the abstract rather than only in the conclusion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this as a system integration paper, not a validated new method. The genuinely new thing is the specific pipeline: LLaMA prompting reformulates instructions into a fixed 'Goal Image: a photo of X' format, CLIP retrieves from a candidate image pool, and a frozen ViNT waypoint predictor on ego RGB produces continuous velocity commands. That composition, with no fine-tuning and no localization, flying a Tello in indoor and outdoor settings under indirect instructions, is a legitimate contribution, and the real-world footage is the strongest evidence. The ablations are also reasonably informative: removing the prompt reformulation or swapping in BLIP degrades performance, which supports the claim that the LLM formatting step matters.\n\nThe soft spot is exactly where the stress-test note lands. Table I is a navigation-only comparison: baselines are given goal positions and never see language, so it does not support 'consistently outperforms all baselines' for VLN. The only language-grounded evaluation is the real-world section, and there are no baselines there, just 20 trials per condition reported as percentages without confidence intervals. Moreover, ViNT itself is never tested as a standalone baseline or with a ground-truth goal image, so you cannot tell how much of the navigation competence comes from VLFly's integration versus from the pretrained planner. The paper also excludes the box environment from trajectory evaluation because it was used to train some baselines; that is disclosed and defensible, but it is another reason the headline claim is softer than advertised. The paper itself concedes in the conclusion that goal retrieval depends on a predefined image pool, which further qualifies 'open-vocabulary.'\n\nI do not think there is a load-bearing technical error. The POMDP formulation is standard, the equations are simple and correct, and the implementation details are clear enough to reproduce. The weaknesses are experimental framing, not math. The related work is honest about prior UAV VLN work, and the self-citations are to relevant prior navigation papers, not padding. For a robotics venue that values physical demonstrations, this deserves a serious referee with a request for a head-to-head VLN baseline and a ViNT-only control. Without those, I would treat the results as a promising existence proof rather than a comparative evaluation. I would cite it as a system demonstration, not as evidence of state-of-the-art performance.","headline":"A clean modular zero-shot UAV VLN system with real flight demos, but the 'outperforms all baselines' claim is not actually tested end-to-end.","tokens_in":13701,"tokens_out":2698,"would_cite":true,"duration_ms":31242,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VLFly enables a UAV to follow open-vocabulary natural language instructions in unseen environments, without fine-tuning and using only monocular RGB observations.","keywords":["vision-language navigation","unmanned aerial vehicles","open-vocabulary goal understanding","zero-shot transfer","continuous velocity control","modular learning","large language models","CLIP"],"falsifier":"Take the VLFly pipeline in the furniture simulation, replace the retrieved goal image with an unrelated image from the same candidate pool, and keep everything else identical; if success rate stays comparable to the full system rather than collapsing, the language-grounding modules are not doing the work attributed to them. A complementary check is running ViNT alone with the true goal image and no language input against VLFly.","tokens_in":12708,"feed_emoji":"🛸","tokens_out":5732,"duration_ms":61081,"temperature":0.7,"pith_summary":"This paper proposes VLFly, a modular framework that lets a UAV follow natural-language navigation instructions in unseen indoor and outdoor places without any task-specific fine-tuning. The intended contribution is an open-vocabulary vision-language navigation system that maps egocentric monocular RGB frames and a sentence like \"fly where a student can keep textbooks\" directly to continuous velocity commands, avoiding the discrete action sets and localization or ranging requirements of earlier VLN systems. A sympathetic reading of the results is that VLFly outperforms the compared baselines in unseen simulation environments and succeeds on real-world direct and indirect instructions, supporting the claim that language-to-image grounding plus waypoint planning transfers zero-shot.","feed_headline":"Off-the-shelf AI models let a drone follow open-vocabulary commands","feed_subtitle":"A language-to-image-to-waypoints chain steers a UAV using only its onboard camera.","key_machinery":"The load-bearing chain is a cascade of three modules. The instruction encoder uses an LLM (LLaMA) with a constrained prompt template to rewrite any free-form instruction into the standardized line \"Goal Image: a photo of [selected item]\", restricting answers to a supplied list plus common objects. The goal retriever uses CLIP to embed that prompt and every candidate image in a shared space, selecting the image with the highest scaled cosine similarity after a softmax. The waypoint planner is a ViNT-style decoder-only Transformer: past egocentric frames are encoded separately, concatenated with a goal token produced from the current frame and goal image, and decoded into a temporal distance estimate plus a sequence of future relative waypoints; a PID controller then converts those waypoints to linear and angular velocities.","core_discovery":"The paper's central claim is that decomposing UAV vision-language navigation into three pre-trained components—an LLM that reformulates instructions into fixed-format goal prompts, a VLM that retrieves the best-matching goal image from a candidate pool, and a waypoint Transformer that predicts future egocentric positions from stacked frames plus the goal image—enables a drone to navigate by language with no additional training. VLFly outputs continuous velocity commands at about 7–15 Hz in simulation and on a real Tello Edu, using only monocular RGB observations. The authors report that it outperforms all baselines across medium and hard simulated environments and achieves 83% success on direct and 70% on indirect real-world instructions, with the indirect gap traced to semantic ambiguity in goal grounding.","pith_inferences":["Editorial inference: the simulation baselines are evaluated without language grounding, so the simulation table chiefly tests waypoint planning; the language-grounding claim rests on the real-world trials and ablations, and a cleaner test would run VLFly in simulation with the same language instructions used in the real-world tasks.","Editorial inference: because the goal retriever requires a pre-collected candidate image pool, the framework's open-vocabulary power is bounded by pool coverage; pairing the retriever with an open-set detector could extend it to goals never imaged beforehand, a direction the conclusion names for future work.","Editorial inference: the reported 70–83% real-world success rates over 20 trials per condition suggest that instruction complexity, not environmental clutter, is the main failure driver; tracking goal-retrieval accuracy separately from navigation success would quantify how much of the indirect-instruction gap comes from ambiguous grounding versus trajectory error."],"forward_implications":["If VLFly's reported performance holds, UAV language navigation can be assembled from off-the-shelf pretrained components, avoiding per-environment data collection and fine-tuning.","Continuous velocity output removes the discrete-action assumption that most VLN methods inherit, making the framework directly deployable on real quadrotors.","Indirect instructions become tractable because the LLM step converts abstract semantics into a concrete goal phrase before visual grounding.","The system works without localization, depth sensors, or mapping, so deployment cost reduces to a monocular camera and a laptop or onboard GPU.","Performance degrades gracefully under semantic ambiguity, pointing to goal grounding rather than flight control as the limiting stage."],"supporting_citations":[{"why":"LLaMA serves as the instruction encoder that reformulates natural-language instructions into standardized goal prompts.","marker":"[18]"},{"why":"CLIP supplies the shared vision-language embedding space used to retrieve the goal image by cosine similarity.","marker":"[19]"},{"why":"ViNT provides the Transformer waypoint-planning architecture that predicts future waypoints from egocentric observations and a goal image.","marker":"[34]"},{"why":"Defines the VLN evaluation metrics (SR, OS, SPL, NE) and supplies the Seq2Seq and CMA baselines the paper compares against.","marker":"[35]"},{"why":"BLIP is the unified VLM used in the ablation replacing instruction encoding plus goal retrieval, demonstrating the modular design's advantage.","marker":"[38]"},{"why":"PPO is a trained RL baseline evaluated in simulation to show the generalization gap against zero-shot VLFly.","marker":"[36]"}],"fun_headline_variants":["Zero-shot drone navigation: language straight to flight commands","VLFly: open-vocabulary drone control from a single camera","Pre-trained models turn language into drone waypoints","Camera-only drone navigation from open-vocabulary language"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the ViNT waypoint planner, pretrained on ground robots, transfers to UAV egocentric video without any adaptation, a premise the experiments never isolate by comparing VLFly against ViNT alone or with a ground-truth goal image.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot drone navigation: language straight to flight commands","VLFly: open-vocabulary drone control from a single camera","Pre-trained models turn language into drone waypoints","Camera-only drone navigation from open-vocabulary language"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0017,"raw_usage":{"total_tokens":6726,"prompt_tokens":932,"completion_tokens":5794,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":5729}},"tokens_in":548,"tokens_out":5794,"duration_ms":45074,"temperature":1.0,"reasoning_tokens":5729,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:18:43.773918+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the VLFly pipeline in the furniture simulation, replace the retrieved goal image with an unrelated image from the same candidate pool, and keep everything else identical; if success rate stays comparable to the full system rather than collapsing, the language-grounding modules are not doing the work attributed to them. A complementary check is running ViNT alone with the true goal image and no language input against VLFly.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"CLIP supplies the shared vision-language embedding space used to retrieve the goal image by cosine similarity."},{"cited_title":"Vint: A foundation model for visual navigation,","cited_arxiv_id":null,"evidence_quote":"ViNT provides the Transformer waypoint-planning architecture that predicts future waypoints from egocentric observations and a goal image."},{"cited_title":"Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments,","cited_arxiv_id":null,"evidence_quote":"Defines the VLN evaluation metrics (SR, OS, SPL, NE) and supplies the Seq2Seq and CMA baselines the paper compares against."},{"cited_title":"Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,","cited_arxiv_id":null,"evidence_quote":"BLIP is the unified VLM used in the ablation replacing instruction encoding plus goal retrieval, demonstrating the modular design's advantage."}],"review_version":1}