{"id":"e08a5208-aedb-40cf-9c67-498afb68a065","arxiv_id":"2501.16300","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An LLM and a VQA model converse to steer a simulated drone through a scene, improving descriptions and hazard detection over a static baseline.","lead":"This paper pairs a large language model with a visual question-answering model that talk to each other while controlling a simulated drone. The system moves the drone to explore a scene, asks questions, validates what it sees, and reports hazards such as fires or crashes more reliably than a static camera.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quantitative support for the central claim rests on a self-referential matching score: the same PnP-VQA model that generates captions also scores them, so the reported gains may reflect model self-consistency rather than improved perception.","rationale":"The reader's weakest assumption is exactly the concern I identify: the caption-image matching score computed by PnP-VQA is not an independent, unbiased measure because the same model generates and scores the captions. This is the most load-bearing concern because the central claim — that the dialogue-based system 'increases perception and anomaly detection accuracy' — rests on Tables 1 and 2. If the metric is circular, the quantitative support collapses; if the anomaly-detection criterion is subjective, the qualitative claim lacks a rigorous basis. The concern is not fatal in principle: the pipeline is coherent, the qualitative examples in Tables 3 and 4 suggest the method can produce richer descriptions, and the authors provide code and acknowledge remaining hallucinations. A conditional acceptance is appropriate, pending independent metric validation. I considered a secondary concern about the lack of an ablation isolating the dialogue from the drone's movement, but the central claim as stated compares the full system to a static baseline, so the movement is part of the proposed approach; the self-referential metric is the more direct threat to correctness. The proposed concrete test (independent scoring model or human raters) would settle whether the reported improvements reflect genuine perceptual gains.","tokens_in":9474,"tokens_out":6131,"duration_ms":62575,"concrete_test":"Regenerate the baseline and proposed captions for the four simulated environments (or reuse those from the provided repository) and score them with an independent image-caption metric, e.g., OpenAI CLIP (ViT-L/14) similarity, or human raters blind to condition rating factual accuracy against the ground-truth scene state in AirSim. Additionally, for the anomaly-detection results, have independent annotators judge whether each final caption actually identifies the placed hazard, given the known ground-truth anomaly positions. If the proposed method no longer consistently beats the baseline under the independent metric, the reported gains are an artifact of PnP-VQA self-scoring.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's primary quantitative evidence (Table 1) uses the PnP-VQA model's image-caption matching score as both the internal caption-selection signal and the external evaluation metric. Captions are generated by BLIP's captioning module (Section 3, Fig. 3) and scored by BLIP's image-text matching module; the evaluation in Section 4 averages this same score at the spawn position and every revisited position. Because the scoring model is identical to the captioning model, the metric measures the model's own likelihood, not an independent ground truth. If the dialogue process shifts captions toward the model's stylistic preferences, or if the ensemble caption merely re-uses language the model already treats as self-consistent, the reported improvements (e.g., 0.384 to 0.585 for Mountain Landscape) do not establish that the captions are more accurate or that perception improved. The paper even concedes hallucinations still occur after validation (Section 4, Table 3 note), yet those hallucinated captions can still receive high matching scores from the same model. Table 2's anomaly detection success criterion — 'appears in captions in a coherent and grammatically logical manner' — is not operationally defined; without a stated protocol for judging coherence and logicality, the score could reflect fluent but hallucinated anomaly mentions. No independent metric, human evaluation, or external benchmark is provided. This circularity is load-bearing because it undermines the only quantitative evidence that the proposed dialogue-based active perception outperforms the static baseline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an active-perception and anomaly-detection framework for a simulated drone. A GPT-3.5 controller issues natural-language movement commands and exploratory questions; a PnP-VQA/BLIP module answers and captions images; positions are saved, revisited with noise during validation, and captions/descriptions are ensembled. Experiments in four AirSim environments report caption-image matching scores and anomaly detection accuracies for baseline versus proposed method, along with qualitative captions, descriptions, and attention maps.","tokens_in":9776,"tokens_out":3260,"duration_ms":30013,"significance":"If the reported gains were measured against an independent standard, the contribution would be useful: zero-shot integration of an LLM controller with a VQA sensor, active perception via dialogue, and explainable attention maps. The authors also release code and prompts, which supports reproducibility. However, the current evaluation does not establish the central claim because the headline metric is generated and scored by the same model, and the anomaly criterion is subjective. The qualitative examples and attention maps provide partial support but do not compensate for the lack of an external benchmark.","major_comments":[{"comment":"The caption-image matching score is computed by the PnP-VQA model, which is the same model whose BLIP captioning module generates the captions being scored. The score is also used inside the pipeline to select and validate captions (Section 3). Therefore the improvement from baseline to proposed (e.g., 0.384 to 0.585 for Mountain Landscape) may reflect the model's stylistic self-consistency rather than improved perception. The authors should add an independent evaluation, such as human ratings, an external vision-language scorer, or standard captioning metrics against ground-truth descriptions, and report per-run variance.","section":"Section 4, Table 1"},{"comment":"Anomaly detection success is defined as the anomaly \"appears in captions in a coherent and grammatically logical manner,\" but no operational protocol is given for judging coherence and logicality. Without a rubric, ground-truth anomaly labels, or inter-annotator agreement, the reported accuracies are not reproducible and cannot be compared across methods. Please define detection as a specific mention of the injected hazard type (fire, car crash, etc.), or use human evaluation with multiple annotators, and report false positives.","section":"Section 4, Table 2"},{"comment":"All quantitative results are averages over ten runs with no standard deviations, confidence intervals, or statistical tests. Given the small number of runs and the stochasticity of LLM sampling, navigation noise, and caption sampling, the reader cannot assess whether the differences are meaningful. Please report per-run results, variance, and significance tests.","section":"Section 4, Tables 1 and 2"},{"comment":"The paper concedes that \"hallucinations can still occur despite the validation process,\" and the final descriptions contain fabricated elements (e.g., \"computer artwork\" in Mountain Landscape with fire, \"digital object\" in Public Square, \"bush tucker\" in Lake with fire). These examples show that the validation module does not reliably filter hallucinated content, which weakens the claim that the proposed method outputs a validated, detailed scene description. The authors should quantify hallucination rates and show, under an external metric, that final descriptions are more accurate than baseline captions.","section":"Section 4, Table 4 and accompanying text"}],"minor_comments":[{"comment":"The notation f(A,C) and g(Q,I) is confusing because A is used both for the answer sequence and for the LLM input sequence; please introduce distinct symbols for the LLM input context and the VQA answer.","section":"Section 3"},{"comment":"Report the exact prompt templates and the random seed handling in the paper or supplementary material; the repository link is helpful but the paper should state which parts of the experiments are deterministic.","section":"Section 4"},{"comment":"The claim that \"there is currently no other established way to implement and evaluate active perception in unstructured open-world setups\" is too strong; please qualify it to the authors' knowledge and cite existing benchmarks for embodied question answering and active perception.","section":"Section 5"},{"comment":"The green/red/orange highlighting may not be visible in grayscale printing; please add textual markers or a legend.","section":"Tables 3 and 4"},{"comment":"The four environment images are not labeled in the text; consider labeling the subfigures to make references clear.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The evaluation is the main weakness. If the authors can add an external, independent evaluation, even a small human study or simple ground-truth-based metrics, the paper could become acceptable. Without that, the headline quantitative claims are not supported. The paper is also quite short for the breadth of claims; the editor may ask for more details on the dialogue protocol and the anomaly detection rubric."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a legitimate, modest extension of the existing ChatGPT-asks/BLIP-2-answers line of work. The new piece is putting that dialogue inside an active-perception loop where the LLM actually moves the drone, saves positions, and later validates captions by revisiting. That is a real combination, and the authors are clear that it builds on the closest cited works rather than pretending those don't exist. The code link, detailed prompts, and qualitative examples are useful; the attention maps are a nice touch. Credit where due.\n\nThe soft spot is the one the stress-test flags, and it is load-bearing. Table 1's caption-image matching score comes from PnP-VQA, the same model that generates the captions and that the LLM uses to select good captions during the dialogue. So the reported gains, e.g., 0.384 to 0.585, show the dialogue converges toward what its own scorer likes; they don't independently establish better perception. The paper needs an external caption-quality metric or human evaluation, or at least a different pre-trained scorer for evaluation. Table 2's anomaly criterion — 'appears in captions in a coherent and grammatically logical manner' — is not operationally defined, and the scoring appears to be manual without a stated protocol. Ten runs with no variance also makes Tables 1 and 2 weaker than they look.\n\nThe good news is the qualitative material partly carries the paper. The captions in Tables 3 and 4 are visibly more specific after active perception, and the anomaly examples show the loop finding things from a distance or around occlusion. The authors also explicitly admit hallucinations persist after validation; that is honest, but it cuts against using the self-referential score as evidence.\n\nThe citation pattern is appropriate; related work includes the closest baselines, and the novelty claim is scoped correctly. No invented entities or free-parameter abuse beyond reasonable prompt choices.\n\nVerdict: this deserves a serious referee. The framework is coherent, reproducible, and timely, but the reported numbers should not be taken at face value until the evaluation is strengthened. I'd send it out rather than desk-reject, with reviewers asked to focus on the circularity. Reading group: maybe. I'd cite it if I worked on LLM/robot dialogue; otherwise it's a small building block.","headline":"A coherent extension of LLM-VQA dialogue to drone active perception, with a load-bearing but fixable evaluation flaw: the caption-quality metric is computed by the same model that generates the captions.","tokens_in":10284,"tokens_out":2638,"would_cite":true,"duration_ms":26696,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that engaging an LLM and a VQA model in a dialogue that controls a drone raises caption quality and anomaly detection accuracy in four simulated aerial monitoring scenes.","keywords":["Active perception","Anomaly detection","Large language models","Visual question answering","Aerial monitoring","Drone navigation","Zero-shot learning"],"falsifier":"Run the same dialogue pipeline in the same four scenes, but have the final validated captions scored by a different vision-language model or by human annotators who do not see the PnP-VQA scores; if the proposed method no longer beats the static baseline, the reported gains are an artifact of self-scoring.","tokens_in":9304,"feed_emoji":"🚁","tokens_out":8468,"duration_ms":70034,"temperature":0.7,"pith_summary":"This paper tries to establish that active perception in aerial monitoring can be built entirely from two off-the-shelf models conversing: a large language model (LLM) that issues drone movement commands and exploratory questions, and a visual question-answering (VQA) model that answers and captions each new image. The authors argue that this dialogue lets the system inspect a scene from multiple viewpoints and produce a richer, validated description, and that the same loop can detect anomalies such as fires and car crashes without any model fine-tuning. In four simulated environments, the proposed method is reported to raise the average caption-image matching score over a static baseline and to boost anomaly detection accuracy from around 0.20-0.53 to 0.73-0.90. The significance, if true, is a practical recipe for zero-shot active perception: pretrained language and vision components can be wired together with prompts and simple movement commands to guide exploration.","feed_headline":"Two AI models chat to fly a drone and spot anomalies","feed_subtitle":"An LLM and a vision model chat to steer a drone, boosting detection scores in four simulated scenes.","key_machinery":"The mechanism that carries the argument is the model dialogue loop, written as Q = f(A, C) for the LLM and (A, C) = g(Q, I) for the VQA network: the LLM maps the previous answer and caption to a new question and movement command, and the VQA maps that question and the current image to a new answer and caption. The named components are the Plug-and-Play VQA (PnP-VQA) framework, which uses BLIP for captioning and question answering and GradCAM to focus on relevant image patches, and GPT-3.5 as the LLM controller. What this loop does is convert one static image into a sequence of deliberately chosen viewpoints, then re-check claimed facts from new poses during validation before composing the final scene description, safety rules, and attention maps.","core_discovery":"The central claim is that two deep learning models can perform active perception and anomaly detection by taking turns in a dialogue. The LLM acts as the controller: it receives the VQA model's previous answer and caption, then outputs a new exploratory question and a movement command chosen from a fixed set (move closer, move back, move left/right, save position, ask a question, I know enough). The VQA model acts as the sensor: given the question and the new drone image, it returns an answer, a caption, and a caption-image matching score. The process runs in three phases: active perception, validation (revisiting saved positions with added pose noise and ensembling the answers), and explanation (GradCAM attention maps that show which image regions support the conclusions). The authors report that this pipeline improves caption-image matching scores in all four environments and raises anomaly detection accuracy from a 0.20-0.53 baseline to 0.73-0.90, while an anomaly-triggered early-stop rule cuts the average experiment time from roughly 12 minutes to under 5 minutes.","pith_inferences":["Because the caption-image matching scores come from the same PnP-VQA model that writes the captions, the numerical gains may partly reflect the model agreeing with itself; scoring the final captions with an independent vision-language model or human raters would test this.","The prompt trick of framing the task as a game to suppress hallucination is a reusable design pattern: explicitly constraining an LLM's role and world model can improve reliability in other LLM-controlled systems.","The validation phase's noisy repositioning and answer ensembling effectively create a small, automatically labeled dataset of question-image-answer triples, which could be used to fine-tune a more specialized perception model.","A direct next experiment is to run the same dialogue loop on real drone footage and compare the validated captions against human-written descriptions of the same scenes."],"forward_implications":["If the reported gains hold, drones can monitor unfamiliar scenes in a zero-shot fashion, without retraining, by reusing general-purpose language and vision models.","The same template of the LLM as controller and the VQA model as sensor, with saved poses revisited for validation, should transfer to other robots, including ground vehicles and pan-tilt cameras.","The anomaly-triggered early-stop rule makes the system practical for time-critical hazard response, reducing average exploration time from about 12 minutes to under 5 minutes.","Attention maps generated during validation give a human operator a concrete account of which image regions led to a caption or hazard alert.","The paper's evaluation suggests that dialogue-based active perception can outperform static perception even without any domain-specific training data."],"supporting_citations":[{"why":"Supplies the PnP-VQA framework that performs captioning, question answering, and matching-score computation.","marker":"[30]"},{"why":"Provides the BLIP vision-language model used inside PnP-VQA for captioning and matching.","marker":"[31]"},{"why":"Provides GradCAM, the attribution method used to select image patches and produce explanation attention maps.","marker":"[32]"},{"why":"Supplies GPT-3.5, the LLM controller that issues commands and questions.","marker":"[33]"},{"why":"Provides the AirSim simulation environment used for all experiments.","marker":"[35]"},{"why":"Introduces the dialogue pattern of an LLM asking and a VQA model answering for enriched image descriptions, which the paper extends to active perception.","marker":"[23]"},{"why":"Documents hallucination in LLMs, motivating the game-scenario prompt and validation phase.","marker":"[34]"}],"fun_headline_variants":["LLM steers drone via chat with vision model to find anomalies","LLM-VQA dialogue lets drone spot anomalies actively","Two AI models converse to drive drone anomaly patrol","Dialogue between LLM and vision model boosts drone anomaly spotting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's quantitative evidence assumes that the image-caption matching score produced by the PnP-VQA model is an independent, unbiased measure of caption quality and anomaly detection success, even though the same model generates the captions being scored.","fun_headline_variants_meta":{"raw":{"variants":["LLM steers drone via chat with vision model to find anomalies","LLM-VQA dialogue lets drone spot anomalies actively","Two AI models converse to drive drone anomaly patrol","Dialogue between LLM and vision model boosts drone anomaly spotting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000944,"raw_usage":{"total_tokens":4058,"prompt_tokens":998,"completion_tokens":3060,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":2993}},"tokens_in":614,"tokens_out":3060,"duration_ms":20140,"temperature":1.0,"reasoning_tokens":2993,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:32:37.118019+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same dialogue pipeline in the same four scenes, but have the final validated captions scored by a different vision-language model or by human annotators who do not see the PnP-VQA scores; if the proposed method no longer beats the static baseline, the reported gains are an artifact of self-scoring.","supporting_citations":[{"cited_title":"Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion,","cited_arxiv_id":null,"evidence_quote":"Provides GradCAM, the attribution method used to select image patches and produce explanation attention maps."},{"cited_title":"Language models are few-shot learners,","cited_arxiv_id":null,"evidence_quote":"Supplies GPT-3.5, the LLM controller that issues commands and questions."}],"review_version":1}