REVIEW 5 major objections 5 minor 2 cited by
UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoning
T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A multi-agent system with a reactive reason-act loop and pixel-level vision grounding converts vague natural-language reports into UAV waypoint plans, succeeding on 93% of tested fire scenes.
desk verdict Plausible multi-agent VLM system for UAV mission planning, but the headline 93% success rate is not supported by the evaluation as reported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the coupling of a ReAct (Reason + Act) loop with a vision-grounded pixel-pointing step. In the loop, the manager agent observes the scene, asks a vision-language model for a detailed description, reasons about how the scene relates to the mission query, decides on waypoints or revisions, and acts by dispatching instructions to the UAV agent; after the simulated flight, frames flow back for another round. Pixel-pointing is what connects a semantic label (such as "the warehouse near the forest") to actual coordinates on a satellite image, using a 7B vision-language model fine-tuned on 9,000 annotated satellite images. The same mechanism lets the planner rank candidate targets by estimated fire probability and route the UAV to the highest-probability locations first.
What would settle it
Take a held-out set of satellite images of real fire incidents that differ from the 9,000 training images, run the same vague query through the system, and compare each generated waypoint plan against the known fire locations; if the system does not put waypoints near the fires on a comparable fraction of scenes—or if smoke and cloud cover in fresh imagery break the pixel grounding—the central claim fails.
Extended reading notes
Core claim
On its own terms, the paper claims that a modular multi-agent arrangement—an airspace manager that reasons over language and images, plus an embodied UAV agent that simulates flight and reports observations—can turn a vague, ambiguous instruction into a flyable mission, and that the two ingredients doing the heavy lifting are the reactive thinking loop and pixel-level grounding. The reported evidence is that, on 30 fire-scene images drawn from an augmented version of the UAV-VLA benchmark, the system plans 28 missions successfully (93%) with a mean generation time of 96.96 seconds at temperature 0.5, compared with 26 successes and 105.29 seconds at temperature 0.7. The paper also claims that fine-tuning a 7B vision-language model on 9,000 annotated satellite images gives it reliable spatial grounding across categories such as buildings, roads, ponds, parking lots, and farmland, with mean pixel distances and coverage values reported per category.
Load-bearing premise
The central assumption is that the 30 benchmark images, with flights simulated by straight-line interpolation over satellite imagery, represent the real fire-detection situations where the system would be used.
Editorial extensions
If this is right
- A natural-language query like "I've heard there are fires in our area" can seed a complete UAV waypoint mission, so operators do not need to place waypoints by hand.
- The reactive loop should let the same architecture recover from ambiguous or incomplete initial instructions by re-querying the visual scene and revising waypoints mid-mission.
- The pixel-pointing grounding makes semantic map categories (buildings, roads, ponds, farmland) directly addressable as flight targets, not just as caption text.
- Lower decoding temperature becomes a usable reliability knob for mission-critical planning: the authors find 0.5 outperforms 0.7 on both success rate and latency.
- Because agents are modular and communicate by messages, the system can scale to more UAVs and can mix heavy and light models across ground and onboard agents.
Reading between the lines
- Beyond the paper's fire-detection setting, the same pixel-pointing-plus-ReAct pipeline could be applied to flood mapping, crop monitoring, or post-disaster damage assessment, provided the grounding model is fine-tuned on those visual categories.
- Because the 93% figure comes from a 2D simulator that linearly interpolates between waypoints, real flights are likely to be harder; the most direct test is to fly the generated plans on a physical drone and measure whether the target is actually reached.
- An ablation that removes the reactive loop while keeping the same agents would isolate how much of the success comes from iterative re-planning rather than from the underlying vision-language grounding.
- The fine-tuned grounding model could be packaged as a standalone geospatial phrase-to-coordinate service, independent of the multi-agent mission planner.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes UAV-CodeAgents, a multi-agent framework built on LLMs/VLMs and the ReAct loop, intended to convert natural-language mission queries plus satellite imagery into executable UAV waypoint plans. The system combines an Airspace Manager Agent that interprets queries and grounds objects at the pixel level, a UAV Agent that simulates flights and performs VLM-based fire detection, and a reactive thinking loop for plan revision. The authors evaluate the system on 30 images from an augmented version of their own UAV-VLA benchmark using a single vague query, report a 93% success rate and 96.96 s average mission creation time at temperature 0.5, compare to temperature 0.7, and separately fine-tune Qwen2.5VL-7B on 9,000 annotated satellite images, reporting per-category mean distance and coverage. The paper claims that lower temperature yields higher planning reliability and reduced execution time, and that the fine-tuned model achieves strong spatial grounding.
Significance. If the reported results held, the framework would be a useful demonstration of VLM-driven UAV mission planning from high-level language and satellite imagery, with potential value for disaster-response scenarios. The paper also promises a release of the codebase and a new benchmark dataset, which are constructive reproducibility contributions, and the fine-tuning dataset for pixel-pointing is a plausible resource. However, the significance is currently not established empirically: the headline success rate is uninterpretable without a success rubric, the only quantitative metric is defined but never reported, no baselines are provided, the simulator is a linear-interpolation abstraction, and the grounding evaluation lacks a train/test split or pre-fine-tuning comparison. The architecture itself is a reasonable integration of existing components, but the evidence as presented does not yet demonstrate that the framework outperforms simpler alternatives or generalizes beyond the 30 selected images.
major comments (5)
- [Section V.B, Eq. (1), and Table II] The 93% success rate is not tied to any explicitly defined per-sample success criterion. Equation (1) defines Time-to-Detection but says N = 30 test cases 'excluding those with false positives or false negatives,' yet the paper never states what constitutes a success, a false positive, or a false negative on a sample. Table II reports 'Successful samples' (28 and 26) with no rubric, so a reader cannot verify whether success means merely that the agent chain terminated or that the UAV visited the true fire location. Please define the success criterion, apply it to all 30 samples, and report the number of samples excluded after each exclusion rule.
- [Section IV.A and V.C] The evaluation is conducted only on 30 images from the authors' own augmented UAV-VLA benchmark, and no baseline is compared. Without a baseline such as the original UAV-VLA system, a random-waypoint policy, a simple image-thresholding detector, or a direct single-agent prompt on the same benchmark, the observed 28/30 success cannot be attributed to the ReAct loop, the multi-agent design, or the pixel-pointing mechanism. The temperature comparison is also only between two settings of the same system. Please add at least one external baseline and report per-scene results to enable calibration of the 93% figure.
- [Section V.B and V.C] Time-to-Detection (TTD) is defined in Eq. (1) but never reported anywhere in the paper. The reported 'Avg. Elapsed Time' is a different quantity, and it is not accompanied by the standard deviation, the number of samples remaining after exclusions, or the operational exclusion criteria. The text also does not state whether the 96.96 s average includes all samples or only the 28 'successful' ones. Please report TTD explicitly for each temperature, with the sample count after exclusions, and explain how failures are handled (e.g., censored at max time) rather than silently discarded.
- [Section V.D and Table III] The fine-tuning results are reported only as per-category mean distance and mean coverage, without an evaluation split, a comparison to the zero-shot Qwen2.5VL-7B before fine-tuning, or any statement of the units or normalization of pixel distances. Table III also contains duplicated category names such as 'ponds' versus 'pond', 'roads' versus 'road', and 'farm' versus 'farms', which makes the table ambiguous. Without a held-out test set and a pre-fine-tuning baseline, the claim of 'strong spatial grounding' is not supported. Please add a train/test split, report the pre-fine-tuning numbers, and clarify the category naming.
- [Section IV.B and V.B] The simulation linearly interpolates waypoints over a static satellite image and generates crops at interpolated positions; there is no sensor model, altitude, heading, or temporal dynamics. In addition, the 'fire confirmed' signal comes from VLM inference on the same cropped frames produced by the same pipeline, so the only ground truth in the loop is the same model under evaluation. This creates a circularity that makes the 93% success rate hard to interpret as a real detection result. Please use independent ground-truth annotations of fire locations and a more realistic simulator, or clearly state the result as a planning-completion rate rather than a fire-detection success rate.
minor comments (5)
- [Section I (Contributions)] The bullet 'We design a UAV-oriented reactive thinking loop...' appears twice verbatim in the contribution list; please remove the duplicate.
- [Section III.D] The text states that pixel-pointing is 'not a novel method,' yet the abstract and contributions present it as a core component. Please clarify what is new here, for instance whether the novelty lies in the fine-tuning dataset or in the particular integration with ReAct.
- [Section V.C and Table II] With N = 30, the difference between 28 and 26 successful samples is small, and no confidence interval or statistical test is reported. A p-value or confidence interval would help assess whether the temperature effect is meaningful.
- [Section V.A] The 'Avg. Elapsed Time' is not precisely defined. Please state whether it includes API inference time, agent communication overhead, simulation time, or all of these, since the abstract refers to 'mission creation time.'
- [General] The paper promises release of the codebase and benchmark but does not state the license, access conditions, or expected release timeline. Including these details would strengthen the reproducibility claim.
Circularity Check
The 93% success rate is self-referential: the only defined metric excludes false positives/negatives, and the simulation labels are produced by the same VLM pipeline under evaluation.
-
self definitional
[Section V-B, Eq. (1); Table II; Section VI]
"Time-to-Detection (TTD) = 1/N ... where N = 30 test cases, excluding those with false positives or false negatives. ... with the system successfully processing 28 out of 30 images—achieving a 93% success rate."
The only quantitative metric defined is TTD, whose N is defined as 30 'excluding those with false positives or false negatives.' No independent success criterion is provided anywhere; Table II labels 28/30 as 'Successful samples' and the conclusion calls this a '93% success rate.' Thus 'successful' can only be read as 'not excluded from the TTD computation.' Because exclusions are based on FP/FN and no external ground-truth fire map is specified, the 93% success rate measures the fraction of samples that survive the metric's own exclusion rule, not a verified mission success. The success claim is therefore tied by construction to the metric that already discards the samples that would count as failures.
-
other
[Section IV-B Simulation; Section III-A.2 UAV Agent]
"Each waypoint consists of a 2D pixel coordinate and an associated semantic label (e.g., 'lake', 'building', 'road'). ... Each crop is then resized and annotated with the corresponding label. ... performs VLM inference to update task status (e.g., fire confirmed, object located)."
The simulation annotates each crop with the label of the waypoint generated by the Airspace Manager Agent's VLM-based pixel-pointing and planning. The UAV Agent then uses VLM inference to decide whether fire is 'confirmed.' False positives and false negatives in the TTD metric can therefore only be judged against labels produced by the same vision-language pipeline that is under test. No independent ground-truth fire locations are described. Consequently, the detection 'success' is self-referential: the system's own predicted labels serve as the reference for judging the system's detection outputs.
full rationale
The framework's design and fine-tuning are not circular: the multi-agent ReAct architecture, pixel-pointing mechanism, and supervised fine-tuning of Qwen2.5VL-7B on 9,000 annotated satellite images are independent engineering contributions with their own training data. The circularity is confined to the headline quantitative claim. The only defined metric, TTD, defines N as 'excluding those with false positives or false negatives,' yet the paper reports '28 out of 30' as a '93% success rate' without any separate success criterion. Moreover, the simulation's crop labels are the system's own predicted waypoint labels, and fire confirmation is performed by VLM inference, so there is no externally anchored ground truth for FP/FN determination. The 93% figure is therefore at least partially a self-consistency measure rather than an externally verified success rate. I assign 6 rather than higher because the core system contribution stands independently; only the central empirical claim reduces by construction to the metric's exclusion rule and the pipeline's self-generated labels.
Assumptions & free parameters
free parameters (2)
- Sampling temperature T =
0.5
- Test set size N =
30
assumptions (4)
- domain assumption The 2D linear-interpolation simulator is a faithful proxy for real UAV flight.
- domain assumption The annotations and fire scenarios in the augmented UAV-VLA benchmark are correct and representative.
- domain assumption The fine-tuned Qwen2.5VL-7B pixel-pointing outputs accurately localize semantic objects on satellite imagery.
- ad hoc to paper The success of a mission plan is equivalent to generating a coherent plan on the 30 benchmark images.
Cite this review
Pith. "Pith review of UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoning." pith.science (2026). https://pith.science/paper/TMVV2KDU
@misc{pith2026250507236,
author = {Pith},
title = {Pith review of: UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/TMVV2KDU}},
note = {Machine review of arXiv:2505.07236}
}
read the original abstract
We present UAV-CodeAgents, a scalable multi-agent framework for autonomous UAV mission generation, built on large language and vision-language models (LLMs/VLMs). The system leverages the ReAct (Reason + Act) paradigm to interpret satellite imagery, ground high-level natural language instructions, and collaboratively generate UAV trajectories with minimal human supervision. A core component is a vision-grounded, pixel-pointing mechanism that enables precise localization of semantic targets on aerial maps. To support real-time adaptability, we introduce a reactive thinking loop, allowing agents to iteratively reflect on observations, revise mission goals, and coordinate dynamically in evolving environments. UAV-CodeAgents is evaluated on large-scale mission scenarios involving industrial and environmental fire detection. Our results show that a lower decoding temperature (0.5) yields higher planning reliability and reduced execution time, with an average mission creation time of 96.96 seconds and a success rate of 93%. We further fine-tune Qwen2.5VL-7B on 9,000 annotated satellite images, achieving strong spatial grounding across diverse visual categories. To foster reproducibility and future research, we will release the full codebase and a novel benchmark dataset for vision-language-based UAV planning.
Figures
Forward citations
Cited by 2 Pith papers
-
UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs
A narrative survey defines 'agentic UAVs' as drones with perception, cognition, control, and communication layers and catalogs applications and challenges across eight domains.
-
From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications
This paper is a broad tutorial on applying LAMs and agentic AI to 6G, largely restating existing research rather than introducing new results.
Reference graph
Works this paper leans on
-
[1]
AI meets UA Vs: A survey on AI empowered UA V perception systems for precision agriculture,
J. Su, X. Zhu, S. Li, and W.-H. Chen, “AI meets UA Vs: A survey on AI empowered UA V perception systems for precision agriculture,” Neurocomputing, vol. 518, pp. 242–270, 2023
2023
-
[2]
Unmanned aerial vehicles for search and rescue: A survey,
M. Lyu, Y . Zhao, C. Huang, and H. Huang, “Unmanned aerial vehicles for search and rescue: A survey,” Remote Sensing , vol. 15, no. 13, 2023
2023
-
[3]
O. Sautenkov, S. Asfaw, Y . Yaqoot, M. A. Mustafa, A. Fedoseev, D. Trinitatova, and D. Tsetserukou, “FlightAR: AR flight assistance interface with multiple video streams and object detection aimed at immersive drone control,” in 2024 IEEE Int. Conf. on Robotics and Biomimetics (ROBIO), 2024, pp. 614–619
work page 2024
-
[4]
An image is worth 16x16 words: Transformers for image recognition at scale,
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021, arXiv:2010.11929
arXiv 2021
-
[5]
Learning transferable visual models from natural language supervi- sion,
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” in Int. Conf. on Machine Learning , 2021, p. 8748–8763
work page 2021
- [6]
-
[7]
AerialVLN: Vision-and-language navigation for UA Vs,
S. Liu, H. Zhang, Y . Qi, P. Wang, Y . Zhang, and Q. Wu, “AerialVLN: Vision-and-language navigation for UA Vs,” in 2023 IEEE/CVF Int. Conf. on Computer Vision (ICCV) , 2023, pp. 15 338–15 348
work page 2023
-
[8]
Aerial vision-and-dialog navigation,
Y . Fan, W. Chen, T. Jiang, C. Zhou, Y . Zhang, and X. Wang, “Aerial vision-and-dialog navigation,” in Findings of the Association for Computational Linguistics: ACL 2023 , 2023, pp. 3043–3061
work page 2023
Show all 22 references
-
[9]
Towards realistic UA V vision-language navigation: Plat- form, benchmark, and methodology,
X. Wang, D. Yang, Z. Wang, H. Kwan, J. Chen, W. Wu, H. Li, Y . Liao, and S. Liu, “Towards realistic UA V vision-language navigation: Plat- form, benchmark, and methodology,” 2024, arXiv:2410.07087
2024 arXiv
-
[10]
Aerial vision-and-language navigation via semantic-topo-metric representa- tion guided LLM reasoning,
Y . Gao, Z. Wang, L. Jing, D. Wang, X. Li, and B. Zhao, “Aerial vision-and-language navigation via semantic-topo-metric representa- tion guided LLM reasoning,” 2024, arXiv:2410.08500
2024 arXiv
-
[11]
EmbodiedCity: Embodied aerial agent for city-level visual language navigation using large language model,
W. Zhang, Y . Liu, X. Wang, X. Chen, C. Gao, and X. Chen, “EmbodiedCity: Embodied aerial agent for city-level visual language navigation using large language model,” in 2024 23rd ACM/IEEE Int. Conf. on Information Processing in Sensor Networks (IPSN) , 2024, pp. 265–266
2024
-
[12]
Morphonavi: Aerial-ground robot navigation with object oriented mapping in digital twin,
S. Karaf, M. Martynov, O. Sautenkov, Z. Darush, and D. Tsetserukou, “Morphonavi: Aerial-ground robot navigation with object oriented mapping in digital twin,” 2025. [Online]. Available: https://arxiv.org/ abs/2504.16914
2025 arXiv
-
[13]
CognitiveDrone: A VLA model and evaluation benchmark for real-time cognitive task solving and reasoning in UA Vs,
A. Lykov, V . Serpiva, M. H. Khan, O. Sautenkov, A. Myshlyaev, G. Tadevosyan, Y . Yaqoot, and D. Tsetserukou, “CognitiveDrone: A VLA model and evaluation benchmark for real-time cognitive task solving and reasoning in UA Vs,” 2025, arXiv:2503.01378
2025 arXiv
-
[14]
UA V-VLA: Vision-language-action system for large scale aerial mission generation,
O. Sautenkov, Y . Yaqoot, A. Lykov, M. A. Mustafa, G. Tadevosyan, A. Akhmetkazy, M. Altamirano Cabrera, M. Martynov, S. Karaf, and D. Tsetserukou, “UA V-VLA: Vision-language-action system for large scale aerial mission generation,” in Proc. of the 2025 ACM/IEEE Int. Conf. on H...
2025
-
[15]
UA V-VLPA*: A vision- language-path-action system for optimal route generation on a large scales,
O. Sautenkov, A. Akhmetkazy, Y . Yaqoot, M. A. Mustafa, G. Tade- vosyan, A. Lykov, and D. Tsetserukou, “UA V-VLPA*: A vision- language-path-action system for optimal route generation on a large scales,” 2025, arXiv:2503.02454
2025 arXiv
-
[16]
Uav-vlrr: Vision-language informed nmpc for rapid response in uav search and rescue,
Y . Yaqoot, M. A. Mustafa, O. Sautenkov, and D. Tsetserukou, “Uav-vlrr: Vision-language informed nmpc for rapid response in uav search and rescue,” 2025. [Online]. Available: https: //arxiv.org/abs/2503.02465
2025 arXiv
-
[17]
RaceVLA: VLA- based racing drone navigation with human-like behaviour,
V . Serpiva, A. Lykov, A. Myshlyaev, M. H. Khan, A. A. Ab- dulkarim, O. Sautenkov, and D. Tsetserukou, “RaceVLA: VLA- based racing drone navigation with human-like behaviour,” 2025, arXiv:2503.02572
2025 arXiv
-
[18]
OpenVLA: An open-source vision-language-action model,
M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair et al. , “OpenVLA: An open-source vision-language-action model,” 2024, arXiv:2406.09246
2024 arXiv
-
[19]
MiniVLA: A better VLA with a smaller footprint,
S. Belkhale and D. Sadigh, “MiniVLA: A better VLA with a smaller footprint,” 2024. [Online]. Available: https://github.com/ Stanford-ILIAD/openvla-mini
2024
-
[20]
Swarm-GPT: Combining large language models with safe motion planning for robot choreography design,
A. Jiao, T. P. Patel, S. Khurana, A.-M. Korol, L. Brunke, V . K. Adajania, U. Culha, S. Zhou, and A. P. Schoellig, “Swarm-GPT: Combining large language models with safe motion planning for robot choreography design,” 2023, arXiv:2312.01059
2023 arXiv
-
[21]
FlockGPT: Guiding UA V flocking with linguistic orchestration,
A. Lykov, S. Karaf, M. Martynov, V . Serpiva, A. Fedoseev, M. Ko- nenkov, and D. Tsetserukou, “FlockGPT: Guiding UA V flocking with linguistic orchestration,” in 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) , 2024, pp. 485–488
2024
-
[22]
WildfireGPT: Tailored large language model for wildfire analysis,
Y . Xie, B. Jiang, T. Mallick, J. D. Bergerson, J. K. Hutchison, D. R. Verner, J. Branham, M. R. Alexander, R. B. Ross, Y . Feng, L.-A. Levy, W. Su, and C. J. Taylor, “WildfireGPT: Tailored large language model for wildfire analysis,” 2025, arXiv:2402.07877
2025 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.