{"id":"88ca9e5f-0f8e-4c3e-8154-75efe3a88730","arxiv_id":"2501.14488","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"HGAM combines heterogeneous graph attention with continuous-action multi-agent actor-critic learning to coordinate mission and charging drones from local observations, and reports large simulated gains over Greedy, MADDPG, and MAAC.","lead":"A new reinforcement learning system lets a team of data-collecting drones and one charging drone adapt their paths in real time using only nearby information. In simulations it collects more data and charges more effectively than three baseline methods, though no code or uncertainty estimates are provided.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-seed results and no variance reporting leave the headline performance gains unsubstantiated; a multi-seed rerun is the decisive check.","rationale":"I agree with the reader's overall CONDITIONAL verdict, but I identify the lack of multi-seed statistics as the single most load-bearing concern. The paper's novelty is an empirical demonstration; if the results are not reproducible across seeds, the main contribution is void. The reader's weakest_assumption about partial observability is relevant to whether the method truly uses local observations, but the experiment section explicitly states a 4.0-unit local observation range, so the contradiction is likely a modeling leftover rather than an implementation flaw. In contrast, no amount of model consistency can validate the reported advantage without repeated trials. This is standard practice for MARL papers. I therefore recommend no change to the CONDITIONAL verdict, but the conditions should emphasize multi-seed reporting as a prerequisite for acceptance.","tokens_in":17438,"tokens_out":11813,"duration_ms":106774,"concrete_test":"Rerun the local-view evaluation in Table 1 with at least 10 random seeds for all four methods, holding environment and hyperparameters fixed, and report mean ± standard deviation for each metric (C, ω, υ, D, F). The claim stands only if HGAM's mean C and D exceed MADDPG's by the reported margins with non-overlapping confidence intervals (or an equivalent statistical test).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: HGAM 'substantially surpasses' Greedy, MADDPG, and MAAC, with claimed 30%/20% improvements. The evidence in Tables 1 and 2 and the training curves in Figures 6 and 7 is presented without any measure of variability: no standard deviations, no confidence intervals, no seed count. Appendix D.4 shows substantial fluctuations in reward and episode length over training (e.g., Figure 6(c) is described as 'fluctuates considerably'), so a single trajectory can be unrepresentative. If the margin in Table 1 (C=0.928 vs MADDPG 0.630) lies within run-to-run noise, the 'substantially surpasses' claim collapses. This is the load-bearing condition for the paper's contribution; the partial-observability contradiction (global link in §3.1 vs. 4.0 local range in §6.1) is a secondary inconsistency that should be fixed but does not directly test the reported superiority.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces HGAM, a multi-agent reinforcement learning framework for coordinating mission UAVs (MUAVs) and charging UAVs (CUAVs) in a continuous 2D workspace with obstacles and points of interest. The framework combines a heterogeneous graph attention encoder with an actor-critic architecture in a centralized-training/decentralized-execution setting: actors consume local graph observations, the critic uses a fully connected global graph, and training uses N-step returns, prioritized experience replay, and a rotation-dilemma penalty. The empirical section compares HGAM against Greedy, MADDPG, and MAAC under local-view and global-view training/evaluation in a simulated environment with two MUAVs, one CUAV, and 100 PoIs. The abstract claims a 30% improvement in data collection coverage and a 20% increase in charging efficiency relative to existing methods.","tokens_in":17745,"tokens_out":5457,"duration_ms":48569,"significance":"If the empirical claims are reproducible, the paper would provide a useful demonstration that a graph-attention actor-critic can coordinate heterogeneous UAVs with continuous actions under local observations, and the comparison to MADDPG and MAAC is relevant. The paper is also transparent in providing training curves, hyperparameters, and pseudo-code in the appendices. However, the significance is conditional: the headline results rest on single-run point estimates with no variance reporting, the partial-observability setup is internally inconsistent, and the reported metrics overlap heavily with the training rewards. The evaluation therefore does not yet establish a generalizable advantage over the baselines.","major_comments":[{"comment":"Every reported metric is a single point estimate with no seeds, standard deviations, or confidence intervals. Appendix D.4 documents considerable fluctuation in reward and episode length (e.g., Figure 6(c) 'fluctuates considerably'), so a single run can be unrepresentative. The central claim that HGAM 'substantially surpasses' Greedy, MADDPG, and MAAC is therefore not statistically supported. Please rerun each configuration with multiple seeds and report means with standard deviations or confidence intervals for all metrics, and state the number of seeds used.","section":"§6.3, Tables 1–2"},{"comment":"The system model states that 'A global communication link covering the entire workspace allows continuous information exchange among all UAVs,' which directly contradicts the 4.0-unit local observation range used in §6.1 and the paper's claim of decentralized execution based solely on local observations. If the global link is available during execution, then the local graph in Eq. (11) is an artificial mask rather than a genuine communication constraint, weakening the stated contribution. Please clarify whether the global communication link exists only during centralized training, or revise the §3.1 sentence to reflect the actual execution-time observation model.","section":"§3.1 vs §3.3.2/§6.1"},{"comment":"The reported performance metrics are near transcriptions of the training reward. The CUAV reward h_c^t = w_e f_t uses a fairness factor f_t built from the same Jain fairness formulas (Eqs. 16–18) that define the reported charging fairness F_T (Eq. 5), and the geographical fairness omega_T (Eq. 2) is also Jain-based. The data collection ratio C_T (Eq. 1) is the cumulative data volume directly maximized by the MUAV reward w_c c_m^t. Tables 1–2 therefore partly measure how well each method optimizes the HGAM reward rather than independent mission outcomes. Please add evaluation metrics that are not used in the reward, or explicitly discuss this overlap as a limitation.","section":"§3.3.3, §3.2, Appendix C"},{"comment":"HGAM's data collection ratio drops from 0.928 under local view to 0.582 under global view, and charging efficiency drops from 0.613 to 0.370, despite the global view providing strictly more information. This large reversal is not adequately described as 'moderate' and is not explained. It raises the possibility that the global-view runs did not converge or that the reported local-view advantage is an artifact. Please provide the training curves and variance for all global-view runs and explain why more information systematically degrades HGAM's performance.","section":"§6.3, Table 1 vs Table 2"}],"minor_comments":[{"comment":"The claimed '30% improvement in data collection coverage' and '20% increase in charging efficiency' are not well-defined against the tables: with MADDPG as baseline, local-view C improves by about 0.298 absolute (roughly 47% relative), while D improves by about 0.184 absolute (roughly 43% relative), and global-view D is lower than MADDPG's. Please state the exact baseline and calculation used for the abstract numbers.","section":"Abstract and §6.3"},{"comment":"The paragraph beginning 'The critic network ψ_u ...' is repeated almost verbatim a few lines later; please delete the duplicate.","section":"§4.3"},{"comment":"The algorithm's cross-references to 'Eq. 10', 'Eq. 11', and 'Eq. 12' are inconsistent with the equations in the main text (the actor update is Eq. 13, the attention mechanism is Eq. 9, and the PER weighting is Eq. 14). Please update the numbering.","section":"Algorithm 1"},{"comment":"There is an unfilled citation '[?]' for mobile ground charging vehicles; please replace it with the intended reference.","section":"§2.1"},{"comment":"The paper says UAVs operate at different horizontal altitudes to prevent collisions, but the state and action spaces are defined in 2D; please clarify whether altitude is a fixed per-agent parameter or an active collision-avoidance mechanism.","section":"§3.1 and §6.1"},{"comment":"The observation model is described inconsistently: §3.3.2 says MUAVs observe within sensing range and CUAVs within charging radius, while Appendix D.2 says every UAV detects agents/obstacles within a 4.0-unit range. Please define a single consistent observation model.","section":"§3.3.2 and Appendix D.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's main contribution is empirical, but the evaluation is not yet reproducible: no code, no seeds, no variance reporting, and the §3.1 global-communication statement contradicts the decentralized-execution claim. The metric-reward overlap is also a substantive evaluation concern. These issues are fixable, and I would not recommend rejection if multi-seed results confirm the relative ordering and the authors reconcile the observation model. The abstract's quantitative claims should also be re-derived from the reported tables."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent, incremental MARL paper whose headline gains over baselines rest on a single simulation run with no variance, and whose partial-observability setup is contradicted by the paper's own system model. It deserves a serious referee, but the empirical section needs a major rework before the 30%/20% claims can be taken at face value.\n\nWhat is actually new: the specific combination of heterogeneous GAT node embeddings, local/global graphs under CTDE, continuous actions, PER, and N-step returns for a mixed MUAV/CUAV coordination task. None of these ingredients are new—the authors cite GAT, MADDPG, PER, and the rotation-detection trick—but the integration into a single framework with tailored rewards is a reasonable engineering contribution. The problem is well motivated and the method description is detailed, including pseudocode and a limitations appendix.\n\nWhere it falls short: (1) The evidence is single-run point estimates. Appendix D.4 shows substantial fluctuations in reward and episode length, so without multiple seeds and standard deviations, the gap in Table 1 (0.928 vs 0.630) could easily be noise. (2) Section 3.1 says a global communication link covers the entire workspace, while Section 6.1 gives agents a 4.0-unit observation range. If the simulator allows all UAVs to exchange all state, then the \"local\" observations are just a mask, and the headline decentralized advantage is not tested. The authors should fix this contradiction and either enforce real partial observability or report the global-link variant. (3) The reward function embeds the same Jain fairness formulas used for the reported metrics, so the CUAV fairness numbers are partly constructed by design. That is not disqualifying, but it weakens the comparison to baselines that do not have this shaping. (4) No code or data are released, despite the abstract promising they \"will be released soon.\"\n\nOn the plus side, the authors are transparent about limitations, the hyperparameters are given in the appendix, and the comparison includes a reasonable baseline set. The drop in HGAM's data collection ratio under global view (0.582 vs 0.928) is odd but not necessarily a red flag, since the policy is trained for local observation.\n\nRecommendation: this is a paper a competent reviewer should see, but it needs a multi-seed rerun, code/data release, and clarification of the communication model. I would not cite it in its current form.","headline":"A plausible but empirically thin MARL combination; single-seed results and a partial-observability contradiction undercut the headline gains.","tokens_in":18234,"tokens_out":2390,"would_cite":false,"duration_ms":22079,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HGAM lets mission and charging UAVs coordinate in real time from local observations alone, with no pre-planned routes, and reports about 30 percent higher coverage and 20 percent higher charging efficiency than three baselines in…","keywords":["multi-agent reinforcement learning","heterogeneous graph attention","continuous action spaces","UAV coordination","partial observability","aerial recharging","decentralized execution","data collection coverage"],"falsifier":"Run the trained HGAM actors in a variant of the simulator where the workspace-wide communication link described in Section 3.1 is removed, so UAVs exchange messages only peer-to-peer within their 4.0-unit range and with simulated latency or dropout; if data collection coverage and charging efficiency fall back toward the MADDPG and MAAC baselines, the reported local-observation advantage is an artifact of that global link rather than a property of the method. The paper's own global-view experiment provides the comparison point: under full observability HGAM's coverage ratio drops from 0.928 (local view) to 0.582, directly implicating the local-graph construction of Eq. (11) in the claimed gain.","tokens_in":17253,"feed_emoji":"🚁","tokens_out":13494,"duration_ms":105454,"temperature":0.7,"pith_summary":"The paper tries to break what it calls the pre-planning barrier: it argues that a mixed team of data-collecting mission UAVs (MUAVs) and in-flight charging UAVs (CUAVs) can be coordinated in real time without any pre-planned routes, using only each agent's local, graph-structured observations and continuous actions. The proposed framework, HGAM (Heterogeneous Graph Attention Multi-agent Deep Deterministic Policy Gradient), combines a heterogeneous graph attention network with an actor-critic reinforcement learning architecture, trained centrally but executed by each UAV from its own local subgraph. If the claim holds, the payoff is practical: search-and-rescue and environmental-monitoring fleets could adapt mid-mission to new tasks, disturbances, and battery emergencies instead of following static schedules. The evidence is a simulation study with two MUAVs, one CUAV, and 100 points of interest, where HGAM reports roughly 30 percent higher data collection coverage and 20 percent higher charging efficiency than Greedy, MADDPG, and MAAC baselines. The paper positions HGAM as the first method to combine heterogeneous graph attention, continuous action spaces, and fully decentralized execution for mixed MUAV/CUAV teams.","feed_headline":"UAV teams coordinate on the fly, beating plan-then-fly by 30%","feed_subtitle":"Drones that share only local views still coordinate in real time, with coverage up 30% and charging efficiency up 20%.","key_machinery":"The load-bearing mechanism is the heterogeneous graph with node-type embeddings, processed by a graph attention layer inside an actor-critic. Agents form the node set of a graph $G = (V, E)$; each node carries a feature vector that includes its agent type (MUAV or CUAV) as an embedding or flag, and edges mark agents within communication range, so the graph topology changes as UAVs move. The GAT layer computes a neighbor-weighted embedding $g_u = \\sum_{v \\in \\mathcal{N}(u)} \\alpha_{vu} W h_v$, where the attention coefficients $\\alpha_{vu}$ are a softmax over LeakyReLU-scored concatenated pairs, letting each agent attend to whichever neighbors matter most for its role. The paper runs two graphs on top of this: a local graph for the actor containing only the closest neighbor of each type, and a global graph for the critic containing all agents with their actions concatenated into node features. That local/global split is what allows the actor to act on local observations while the critic supplies a centralized performance benchmark, and it is the component the paper credits for the coverage and charging gains.","core_discovery":"On its own terms, the paper's central claim is that precise, decentralized continuous-action coordination among heterogeneous UAVs is achievable from strictly local, heterogeneous graph-based observations, and that the HGAM framework demonstrates this. Each UAV's observation — laser obstacle distances, nearby agents and points of interest, battery states — is encoded by an MLP, then aggregated through a graph attention layer whose attention coefficients weight the relevance of each neighbor, and finally mapped to a continuous two-dimensional angular-velocity action. During training a centralized critic builds a global graph over all agents and actions to estimate joint Q-values, while at execution each actor sees only its local subgraph of nearest neighbors per type, following the centralized-training/decentralized-execution paradigm. The paper reports that the resulting policies make the CUAV dynamically switch charging targets according to battery urgency and make the MUAVs cover complementary regions with minimal route overlap: in the local-view benchmark HGAM reaches 0.928 data collection ratio and 0.929 geographical fairness against MADDPG's 0.630 and 0.633, with charging efficiency 0.613 versus 0.429. It concludes that HGAM resolves three previously open challenges at once — real-time adaptation without predefined routes, decision-making from local observations alone, and fine-grained control in continuous action spaces.","pith_inferences":["If genuine peering (removing the Section 3.1 global link, adding latency and dropout) preserves the margins, HGAM's design is a template for real fleets; if not, the framework's value lies in the centralized-training regime. The paper's own Section E.2 acknowledges that sensor noise and communication disruptions are not modeled, so this is the open experiment.","The actor's local graph keeps only the nearest neighbor of each type, so per-agent computation stays roughly constant as the fleet grows; that suggests a scaling prediction — HGAM's advantage over MADDPG should widen as more MUAVs and CUAVs join — that the paper does not test, since its scenarios use just two MUAVs and one CUAV.","The CUAV's learned behavior of following one MUAV and then switching to the other as battery urgency shifts looks like genuine coordination, but it may be imposed by the hierarchical penalty scheme in Appendix C; ablating that penalty would separate learned negotiation from reward-shaping.","Editorial note: Section 2.1 carries an unresolved citation marker ('[?]') for ground-vehicle mobile charging, and Algorithm 1's references to Eqs. 10-12 do not match the equations actually numbered 10-15 in the text; these are internal-consistency gaps in the comparison lineage and derivation chain, not statements about whether the reported gains hold."],"forward_implications":["UAV missions can run without any pre-planned routes: the trained policies react in real time to changing PoI data volumes, obstacle proximity, and battery states, with the CUAV switching charging targets by urgency.","Because agent type enters the graph as a node embedding rather than a hard-coded rule, the heterogeneous GAT mechanism transfers to other role-differentiated multi-agent teams.","The dual local/global graph design provides a bounded performance ladder: the global critic's Q-values serve as an upper benchmark that the local-graph actors can approach, and the Section 6.3 numbers show them doing so in the local-view regime.","The gains concentrate in the partial-observability regime the paper targets — under global observations HGAM's coverage edge over MADDPG narrows from 0.928 versus 0.630 to 0.582 versus 0.492 — which the paper reads as evidence the method is built for decentralized deployment."],"supporting_citations":[{"why":"Supplies the graph attention mechanism (Eqs. 8-9) that aggregates neighbor information into node embeddings; it is the representational core of HGAM.","marker":"[14]"},{"why":"The MADDPG baseline and the actor-critic ancestor whose centralized-training/decentralized-execution design HGAM extends with heterogeneous graphs.","marker":"[10]"},{"why":"The MAAC baseline with attention in the critic, the strongest attention-based comparison HGAM must beat in Section 6.3.","marker":"[5]"},{"why":"Provides prioritized experience replay (Eqs. 14-15), one of the three training stabilizers the paper credits for convergence.","marker":"[12]"},{"why":"Source of the rotation-dilemma detection idea and the N-step return usage in Section 5, which prevent local-loop stagnation.","marker":"[15]"},{"why":"Defines Jain's fairness index used in Eqs. (2) and (5) for the geographical and charging fairness metrics that structure the evaluation.","marker":"[6]"},{"why":"The aerial-refueling scheduling approach with pre-planned wireless charging that HGAM positions itself against as its main predecessor.","marker":"[21]"},{"why":"Option-based multi-agent hierarchical DRL for CUAV-powered charging, another pre-planned or scheduled baseline the paper claims to supersede.","marker":"[16]"}],"fun_headline_variants":["Drones ditch pre-plans, adapt on the fly with local views only","HGAM: drone teams adapt in real time, beating pre-planned routes","No pre-plans: heterogeneous UAVs self-coordinate from local views","30% more coverage: drones adapt in real time, no pre-plans needed","Mission and charging drones self-coordinate in real time, no pre-plans"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The simulation must genuinely withhold global information from each agent's decisions, but the system model in Section 3.1 states that a communication link spanning the whole workspace lets all UAVs exchange information continuously; if that link makes global state available to everyone, the 'local observations' are an artificial mask, and the reported decentralization advantage may not survive real deployment.","fun_headline_variants_meta":{"raw":{"variants":["Drones ditch pre-plans, adapt on the fly with local views only","HGAM: drone teams adapt in real time, beating pre-planned routes","No pre-plans: heterogeneous UAVs self-coordinate from local views","30% more coverage: drones adapt in real time, no pre-plans needed","Mission and charging drones self-coordinate in real time, no pre-plans"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000846,"raw_usage":{"total_tokens":3716,"prompt_tokens":1016,"completion_tokens":2700,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":2599}},"tokens_in":632,"tokens_out":2700,"duration_ms":18055,"temperature":1.0,"reasoning_tokens":2599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:06:44.264528+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained HGAM actors in a variant of the simulator where the workspace-wide communication link described in Section 3.1 is removed, so UAVs exchange messages only peer-to-peer within their 4.0-unit range and with simulated latency or dropout; if data collection coverage and charging efficiency fall back toward the MADDPG and MAAC baselines, the reported local-observation advantage is an artifact of that global link rather than a property of the method. The paper's own global-view experiment provides the comparison point: under full observability HGAM's coverage ratio drops from 0.928 (local view) to 0.582, directly implicating the local-graph construction of Eq. (11) in the claimed gain.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The MAAC baseline with attention in the critic, the strongest attention-based comparison HGAM must beat in Section 6.3."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the rotation-dilemma detection idea and the N-step return usage in Section 5, which prevent local-loop stagnation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Jain's fairness index used in Eqs. (2) and (5) for the geographical and charging fairness metrics that structure the evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The aerial-refueling scheduling approach with pre-planned wireless charging that HGAM positions itself against as its main predecessor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Option-based multi-agent hierarchical DRL for CUAV-powered charging, another pre-planned or scheduled baseline the paper claims to supersede."}],"review_version":1}