Breaking the Pre-Planning Barrier: Adaptive Real-Time Coordination of Heterogeneous UAVs

Wei Zhang; Xiaoyu Shen; Xinghao Chen; Yanjun Chen; Yirong Sun; Yuhan Hu

REVIEW 4 major objections 6 minor 23 references

Breaking the Pre-Planning Barrier: Adaptive Real-Time Coordination of Heterogeneous UAVs

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HGAM lets mission and charging UAVs coordinate in real time from local observations alone, with no pre-planned routes, and reports about 30 percent higher coverage and 20 percent higher charging efficiency than three baselines in…

desk verdict A plausible but empirically thin MARL combination; single-seed results and a partial-observability contradiction undercut the headline gains. read the letter →

arxiv 2501.14488 v2 pith:DMSCNGX7 submitted 2025-01-24 cs.MA

Yuhan Hu , Yirong Sun , Yanjun Chen , Xinghao Chen , Xiaoyu Shen , Wei Zhang This is my paper · ORCID

classification cs.MA

keywords multi-agentreinforcementlearningheterogeneousgraphattentioncontinuousactionspacesUAVcoordinationpartialobservabilityaerialrechargingdecentralizedexecutiondatacollectioncoverage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to break what it calls the pre-planning barrier: it argues that a mixed team of data-collecting mission UAVs (MUAVs) and in-flight charging UAVs (CUAVs) can be coordinated in real time without any pre-planned routes, using only each agent's local, graph-structured observations and continuous actions. The proposed framework, HGAM (Heterogeneous Graph Attention Multi-agent Deep Deterministic Policy Gradient), combines a heterogeneous graph attention network with an actor-critic reinforcement learning architecture, trained centrally but executed by each UAV from its own local subgraph. If the claim holds, the payoff is practical: search-and-rescue and environmental-monitoring fleets could adapt mid-mission to new tasks, disturbances, and battery emergencies instead of following static schedules. The evidence is a simulation study with two MUAVs, one CUAV, and 100 points of interest, where HGAM reports roughly 30 percent higher data collection coverage and 20 percent higher charging efficiency than Greedy, MADDPG, and MAAC baselines. The paper positions HGAM as the first method to combine heterogeneous graph attention, continuous action spaces, and fully decentralized execution for mixed MUAV/CUAV teams.

What carries the argument

The load-bearing mechanism is the heterogeneous graph with node-type embeddings, processed by a graph attention layer inside an actor-critic. Agents form the node set of a graph $G = (V, E)$; each node carries a feature vector that includes its agent type (MUAV or CUAV) as an embedding or flag, and edges mark agents within communication range, so the graph topology changes as UAVs move. The GAT layer computes a neighbor-weighted embedding $g_u = \sum_{v \in \mathcal{N}(u)} \alpha_{vu} W h_v$, where the attention coefficients $\alpha_{vu}$ are a softmax over LeakyReLU-scored concatenated pairs, letting each agent attend to whichever neighbors matter most for its role. The paper runs two graphs on top of this: a local graph for the actor containing only the closest neighbor of each type, and a global graph for the critic containing all agents with their actions concatenated into node features. That local/global split is what allows the actor to act on local observations while the critic supplies a centralized performance benchmark, and it is the component the paper credits for the coverage and charging gains.

What would settle it

Run the trained HGAM actors in a variant of the simulator where the workspace-wide communication link described in Section 3.1 is removed, so UAVs exchange messages only peer-to-peer within their 4.0-unit range and with simulated latency or dropout; if data collection coverage and charging efficiency fall back toward the MADDPG and MAAC baselines, the reported local-observation advantage is an artifact of that global link rather than a property of the method. The paper's own global-view experiment provides the comparison point: under full observability HGAM's coverage ratio drops from 0.928 (local view) to 0.582, directly implicating the local-graph construction of Eq. (11) in the claimed gain.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that precise, decentralized continuous-action coordination among heterogeneous UAVs is achievable from strictly local, heterogeneous graph-based observations, and that the HGAM framework demonstrates this. Each UAV's observation — laser obstacle distances, nearby agents and points of interest, battery states — is encoded by an MLP, then aggregated through a graph attention layer whose attention coefficients weight the relevance of each neighbor, and finally mapped to a continuous two-dimensional angular-velocity action. During training a centralized critic builds a global graph over all agents and actions to estimate joint Q-values, while at execution each actor sees only its local subgraph of nearest neighbors per type, following the centralized-training/decentralized-execution paradigm. The paper reports that the resulting policies make the CUAV dynamically switch charging targets according to battery urgency and make the MUAVs cover complementary regions with minimal route overlap: in the local-view benchmark HGAM reaches 0.928 data collection ratio and 0.929 geographical fairness against MADDPG's 0.630 and 0.633, with charging efficiency 0.613 versus 0.429. It concludes that HGAM resolves three previously open challenges at once — real-time adaptation without predefined routes, decision-making from local observations alone, and fine-grained control in continuous action spaces.

Load-bearing premise

The simulation must genuinely withhold global information from each agent's decisions, but the system model in Section 3.1 states that a communication link spanning the whole workspace lets all UAVs exchange information continuously; if that link makes global state available to everyone, the 'local observations' are an artificial mask, and the reported decentralization advantage may not survive real deployment.

Editorial extensions

If this is right

UAV missions can run without any pre-planned routes: the trained policies react in real time to changing PoI data volumes, obstacle proximity, and battery states, with the CUAV switching charging targets by urgency.
Because agent type enters the graph as a node embedding rather than a hard-coded rule, the heterogeneous GAT mechanism transfers to other role-differentiated multi-agent teams.
The dual local/global graph design provides a bounded performance ladder: the global critic's Q-values serve as an upper benchmark that the local-graph actors can approach, and the Section 6.3 numbers show them doing so in the local-view regime.
The gains concentrate in the partial-observability regime the paper targets — under global observations HGAM's coverage edge over MADDPG narrows from 0.928 versus 0.630 to 0.582 versus 0.492 — which the paper reads as evidence the method is built for decentralized deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If genuine peering (removing the Section 3.1 global link, adding latency and dropout) preserves the margins, HGAM's design is a template for real fleets; if not, the framework's value lies in the centralized-training regime. The paper's own Section E.2 acknowledges that sensor noise and communication disruptions are not modeled, so this is the open experiment.
The actor's local graph keeps only the nearest neighbor of each type, so per-agent computation stays roughly constant as the fleet grows; that suggests a scaling prediction — HGAM's advantage over MADDPG should widen as more MUAVs and CUAVs join — that the paper does not test, since its scenarios use just two MUAVs and one CUAV.
The CUAV's learned behavior of following one MUAV and then switching to the other as battery urgency shifts looks like genuine coordination, but it may be imposed by the hierarchical penalty scheme in Appendix C; ablating that penalty would separate learned negotiation from reward-shaping.
Editorial note: Section 2.1 carries an unresolved citation marker ('[?]') for ground-vehicle mobile charging, and Algorithm 1's references to Eqs. 10-12 do not match the equations actually numbered 10-15 in the text; these are internal-consistency gaps in the comparison lineage and derivation chain, not statements about whether the reported gains hold.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a competent, incremental MARL paper whose headline gains over baselines rest on a single simulation run with no variance, and whose partial-observability setup is contradicted by the paper's own system model. It deserves a serious referee, but the empirical section needs a major rework before the 30%/20% claims can be taken at face value.

What is actually new: the specific combination of heterogeneous GAT node embeddings, local/global graphs under CTDE, continuous actions, PER, and N-step returns for a mixed MUAV/CUAV coordination task. None of these ingredients are new—the authors cite GAT, MADDPG, PER, and the rotation-detection trick—but the integration into a single framework with tailored rewards is a reasonable engineering contribution. The problem is well motivated and the method description is detailed, including pseudocode and a limitations appendix.

Where it falls short: (1) The evidence is single-run point estimates. Appendix D.4 shows substantial fluctuations in reward and episode length, so without multiple seeds and standard deviations, the gap in Table 1 (0.928 vs 0.630) could easily be noise. (2) Section 3.1 says a global communication link covers the entire workspace, while Section 6.1 gives agents a 4.0-unit observation range. If the simulator allows all UAVs to exchange all state, then the "local" observations are just a mask, and the headline decentralized advantage is not tested. The authors should fix this contradiction and either enforce real partial observability or report the global-link variant. (3) The reward function embeds the same Jain fairness formulas used for the reported metrics, so the CUAV fairness numbers are partly constructed by design. That is not disqualifying, but it weakens the comparison to baselines that do not have this shaping. (4) No code or data are released, despite the abstract promising they "will be released soon."

On the plus side, the authors are transparent about limitations, the hyperparameters are given in the appendix, and the comparison includes a reasonable baseline set. The drop in HGAM's data collection ratio under global view (0.582 vs 0.928) is odd but not necessarily a red flag, since the policy is trained for local observation.

Recommendation: this is a paper a competent reviewer should see, but it needs a multi-seed rerun, code/data release, and clarification of the communication model. I would not cite it in its current form.

Referee Report

4 major / 6 minor

Summary. The paper introduces HGAM, a multi-agent reinforcement learning framework for coordinating mission UAVs (MUAVs) and charging UAVs (CUAVs) in a continuous 2D workspace with obstacles and points of interest. The framework combines a heterogeneous graph attention encoder with an actor-critic architecture in a centralized-training/decentralized-execution setting: actors consume local graph observations, the critic uses a fully connected global graph, and training uses N-step returns, prioritized experience replay, and a rotation-dilemma penalty. The empirical section compares HGAM against Greedy, MADDPG, and MAAC under local-view and global-view training/evaluation in a simulated environment with two MUAVs, one CUAV, and 100 PoIs. The abstract claims a 30% improvement in data collection coverage and a 20% increase in charging efficiency relative to existing methods.

Significance. If the empirical claims are reproducible, the paper would provide a useful demonstration that a graph-attention actor-critic can coordinate heterogeneous UAVs with continuous actions under local observations, and the comparison to MADDPG and MAAC is relevant. The paper is also transparent in providing training curves, hyperparameters, and pseudo-code in the appendices. However, the significance is conditional: the headline results rest on single-run point estimates with no variance reporting, the partial-observability setup is internally inconsistent, and the reported metrics overlap heavily with the training rewards. The evaluation therefore does not yet establish a generalizable advantage over the baselines.

major comments (4)

[§6.3, Tables 1–2] Every reported metric is a single point estimate with no seeds, standard deviations, or confidence intervals. Appendix D.4 documents considerable fluctuation in reward and episode length (e.g., Figure 6(c) 'fluctuates considerably'), so a single run can be unrepresentative. The central claim that HGAM 'substantially surpasses' Greedy, MADDPG, and MAAC is therefore not statistically supported. Please rerun each configuration with multiple seeds and report means with standard deviations or confidence intervals for all metrics, and state the number of seeds used.
[§3.1 vs §3.3.2/§6.1] The system model states that 'A global communication link covering the entire workspace allows continuous information exchange among all UAVs,' which directly contradicts the 4.0-unit local observation range used in §6.1 and the paper's claim of decentralized execution based solely on local observations. If the global link is available during execution, then the local graph in Eq. (11) is an artificial mask rather than a genuine communication constraint, weakening the stated contribution. Please clarify whether the global communication link exists only during centralized training, or revise the §3.1 sentence to reflect the actual execution-time observation model.
[§3.3.3, §3.2, Appendix C] The reported performance metrics are near transcriptions of the training reward. The CUAV reward h_c^t = w_e f_t uses a fairness factor f_t built from the same Jain fairness formulas (Eqs. 16–18) that define the reported charging fairness F_T (Eq. 5), and the geographical fairness omega_T (Eq. 2) is also Jain-based. The data collection ratio C_T (Eq. 1) is the cumulative data volume directly maximized by the MUAV reward w_c c_m^t. Tables 1–2 therefore partly measure how well each method optimizes the HGAM reward rather than independent mission outcomes. Please add evaluation metrics that are not used in the reward, or explicitly discuss this overlap as a limitation.
[§6.3, Table 1 vs Table 2] HGAM's data collection ratio drops from 0.928 under local view to 0.582 under global view, and charging efficiency drops from 0.613 to 0.370, despite the global view providing strictly more information. This large reversal is not adequately described as 'moderate' and is not explained. It raises the possibility that the global-view runs did not converge or that the reported local-view advantage is an artifact. Please provide the training curves and variance for all global-view runs and explain why more information systematically degrades HGAM's performance.

minor comments (6)

[Abstract and §6.3] The claimed '30% improvement in data collection coverage' and '20% increase in charging efficiency' are not well-defined against the tables: with MADDPG as baseline, local-view C improves by about 0.298 absolute (roughly 47% relative), while D improves by about 0.184 absolute (roughly 43% relative), and global-view D is lower than MADDPG's. Please state the exact baseline and calculation used for the abstract numbers.
[§4.3] The paragraph beginning 'The critic network ψ_u ...' is repeated almost verbatim a few lines later; please delete the duplicate.
[Algorithm 1] The algorithm's cross-references to 'Eq. 10', 'Eq. 11', and 'Eq. 12' are inconsistent with the equations in the main text (the actor update is Eq. 13, the attention mechanism is Eq. 9, and the PER weighting is Eq. 14). Please update the numbering.
[§2.1] There is an unfilled citation '[?]' for mobile ground charging vehicles; please replace it with the intended reference.
[§3.1 and §6.1] The paper says UAVs operate at different horizontal altitudes to prevent collisions, but the state and action spaces are defined in 2D; please clarify whether altitude is a fixed per-agent parameter or an active collision-avoidance mechanism.
[§3.3.2 and Appendix D.2] The observation model is described inconsistently: §3.3.2 says MUAVs observe within sensing range and CUAVs within charging radius, while Appendix D.2 says every UAV detects agents/obstacles within a 4.0-unit range. Please define a single consistent observation model.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HGAM's headline gains are relative empirical comparisons against external baselines, not derivations from self-citations or definitions.

full rationale

HGAM is an empirical reinforcement-learning paper. Its headline claim is a relative performance comparison (e.g., C=0.928 vs MADDPG=0.630 in Table 1) under the same simulated environment. The reward functions in Section 3.3.3 are intentionally aligned with the evaluation metrics in Section 3.2: the MUAV reward's main term is w_c times the collected data volume, and the CUAV fairness factor in Appendix C uses the same Jain fairness formula as Eq. (5). However, this is standard reward design rather than a hidden derivation. The paper does not present the absolute metric values as independent first-principles predictions, and all methods are evaluated against the same metric definitions, so the relative superiority claim is not forced by construction. There are no load-bearing self-citations: the GAT mechanism cites Velickovic et al., PER cites Schaul et al., and the dilemma detection cites Wei et al., none of which involve the present authors. The internal inconsistency between the global communication link in Section 3.1 and the 4.0-unit local observation range in Section 6.1 weakens the partial-observability framing, but it is a modeling inconsistency, not circularity. Minor reporting artifacts (the unresolved '[?]' citation in Section 2.1, the cross-reference to a non-existent Table 3 in Appendix D.4, and the '30%' improvement figure not matching Tables 1-2) are correctness issues outside the circularity scope. No specific equation-to-equation reduction can be quoted that makes a prediction equivalent to its inputs, so the circularity score is 0.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The central claim rests almost entirely on a hand-tuned RL pipeline: reward weights, N-step depth, PER exponent, and general hyperparameters are chosen without sensitivity analysis, and the simulator itself relies on simplified energy, altitude-separation, and global-link assumptions. No new physical entities are introduced. The ledger therefore emphasizes that the reported performance gains are conditional on these choices.

free parameters (9)

Data collection reward weight w_c = 0.5
Hand-tuned in Appendix D.2; it directly sets the MUAV incentive to collect data, which is the headline data-collection metric.
Movement reward weight w_l = 0.02
Hand-tuned in Appendix D.2; encourages MUAVs to keep moving and avoid idle-rotation penalties.
Charging reward weight w_e = 1.6
Hand-tuned in Appendix D.2; scales the CUAV incentive for effective charging and therefore shapes the charging-efficiency metric.
Fairness weight w_f = 0.5
Equation (16) balances charged-level fairness and remaining-level fairness; no sensitivity analysis is reported.
CUAV neglect penalty weights w_d and w_e in Eq. (19) = not specified
The penalty for ignoring the lowest-battery MUAV depends on these weights, but their numeric values are not given.
N-step return depth N = 3
Appendix D.3 states N=3 was chosen based on preliminary tests on the same benchmark, so part of the reported gain may reflect tuning.
PER priority exponent alpha = 0.6
Set in Appendix D.3; the paper provides no ablation showing how sensitive results are to this choice.
Hierarchical penalty baseline p_low^c = not specified
The baseline low-battery penalty in Eq. (20) is defined symbolically but never assigned a value, making the CUAV reward scheme incomplete.
General RL hyperparameters (learning rates, gamma, tau, hidden dims, batch, buffer) = lr_critic=0.001, lr_actor=0.0001, gamma=0.98, tau=0.01, hidden=128/64, batch=128, buffer=1e5
Hand-selected MARL hyperparameters from Appendix D.3; no grid search or sensitivity analysis is provided.

assumptions (6)

domain assumption A global communication link covers the entire workspace and allows continuous information exchange among all UAVs.
Stated in Section 3.1; it conflicts with the local-observation premise that the method's contributions hinge on, so the simulation may not enforce genuine partial observability.
domain assumption MUAVs operate at different horizontal altitudes to prevent mutual collisions; only collisions with obstacles or walls at the same altitude can occur.
Section 3.1; this simplification removes inter-UAV collision avoidance, which is a central difficulty in real coordinated flight.
domain assumption Energy consumption is linear in collected data and distance, charging is a constant amount per timestep, and only one MUAV can be charged at a time.
Section 3.1; beta, kappa, e0, Emax, and initial battery values are not specified in the appendices.
ad hoc to paper The joint objective is the product of data-collection ratio and fairness for MUAVs, and charging efficiency and fairness for CUAVs.
The product form and the separate fairness indices are introduced for this paper and are not derived from physical constraints or a game-theoretic model.
ad hoc to paper Hand-tuned reward weights and penalties are a faithful encoding of the desired mission behavior.
The reported metrics closely mirror these reward terms, so absolute performance partly reflects the chosen reward design.
domain assumption The actor's local graph, restricted to the closest neighbor of each type, represents the information available during decentralized execution.
The notation in Eq. (11) is ambiguous, and the existence of the global communication link from Section 3.1 makes it unclear why only nearest-neighbor information is used.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Breaking the Pre-Planning Barrier: Adaptive Real-Time Coordination of Heterogeneous UAVs." pith.science (2026). https://pith.science/paper/DMSCNGX7

@misc{pith2026250114488,
  author       = {Pith},
  title        = {Pith review of: Breaking the Pre-Planning Barrier: Adaptive Real-Time Coordination of Heterogeneous UAVs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DMSCNGX7}},
  note         = {Machine review of arXiv:2501.14488}
}

read the original abstract

Unmanned Aerial Vehicles (UAVs) offer significant potential in dynamic, perception-intensive tasks such as search and rescue and environmental monitoring; however, their effectiveness is severely restricted by conventional pre-planned routing methods, which lack the flexibility to respond in real-time to evolving task demands, unexpected disturbances, and localized view limitations in real-world scenarios. To address this fundamental limitation, we introduce a novel multi-agent reinforcement learning framework named \textbf{H}eterogeneous \textbf{G}raph \textbf{A}ttention \textbf{M}ulti-agent Deep Deterministic Policy Gradient (HGAM), uniquely designed to enable adaptive real-time coordination between mission UAVs (MUAVs) and charging UAVs (CUAVs). HGAM specifically addresses the previously unsolved challenge of enabling precise, decentralized continuous-action coordination solely based on local, heterogeneous graph-based observations. Extensive simulations demonstrate that HGAM substantially surpasses existing methods, achieving, for example, a 30\% improvement in data collection coverage and a 20\% increase in charging efficiency, providing crucial insights and foundations for the future deployment of intelligent, flexible UAV networks in complex, dynamic environments.

Figures

Figures reproduced from arXiv: 2501.14488 by the authors.

**Figure 2.** Illustration of MUAV sensing and CUAV charg [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Overview of the actor-critic architecture with het [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Overall HGAM pipeline under the CTDE paradigm. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Adaptive UAV trajectories generated by HGAM un [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Model training curve visualization comparison: UAVs with local view [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: Model training curve visualization comparison:UAVs with global view [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 19 canonical work pages

[1]

Yining Chen, Guanghua Song, Zhenhui Ye, and Xiaohong Jiang. 2022. Scalable and transferable reinforcement learning for multi-agent mixed cooperative– competitive environments based on hierarchical graph attention. Entropy 24, 4 (2022), 563

work page 2022
[2]

Anna Dai, Rongpeng Li, Zhifeng Zhao, and Honggang Zhang. 2020. Graph convolutional multi-agent reinforcement learning for UAV coverage control. In 2020 International Conference on Wireless Communications and Signal Processing (WCSP). IEEE, 1106–1111

work page 2020
[3]

Jizhe Dou, Haotian Zhang, and Guodong Sun. 2024. Scheduling Drone and Mobile Charger via Hybrid-Action Deep Reinforcement Learning. arXiv preprint arXiv:2403.10761 (2024)

work page Pith review arXiv 2024
[4]

Mingfeng Fan, Yaoxin Wu, Tianjun Liao, Zhiguang Cao, Hongliang Guo, Guil- laume Sartoretti, and Guohua Wu. 2022. Deep reinforcement learning for uav routing in the presence of multiple charging stations. IEEE Transactions on Vehicular Technology 72, 5 (2022), 5732–5746

work page 2022
[5]

Shariq Iqbal and Fei Sha. 2019. Actor-attention-critic for multi-agent reinforce- ment learning. InInternational conference on machine learning. PMLR, 2961–2970

work page 2019
[6]

Rajendra K Jain, Dah-Ming W Chiu, William R Hawe, et al. 1984. A quantitative measure of fairness and discrimination. Eastern Research Laboratory, Digital Equipment Corporation, Hudson, MA 21 (1984), 1

work page 1984
[7]

Chi Harold Liu, Zipeng Dai, Yinuo Zhao, Jon Crowcroft, Dapeng Wu, and Kin K Leung. 2019. Distributed and energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning. IEEE Transactions on Mobile Computing 20, 1 (2019), 130–146

work page 2019
[8]

Chi Harold Liu, Chengzhe Piao, and Jian Tang. 2020. Energy-efficient UAV crowdsensing with multiple charging stations by deep learning. InIEEE INFOCOm 2020-IEEE conference on computer communications . IEEE, 199–208

work page 2020

Show all 23 references

[9]

Ning Liu, Jian Zhang, Chuanwen Luo, Jia Cao, Yi Hong, Zhibo Chen, and Ting Chen. 2023. Dynamic Charging Strategy Optimization for UAV-Assisted Wireless Rechargeable Sensor Networks Based On Deep Q-network. IEEE Internet of Things Journal (2023)

2023
[10]

Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. 2020. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. arXiv:1706.02275 [cs.LG] https://arxiv.org/abs/1706.02275

2020 arXiv
[11]

Zhiyu Mou, Yu Zhang, Dian Fan, Jun Liu, and Feifei Gao. 2020. Research on the UAV-aided data collection and trajectory design based on the deep reinforcement learning. Chinese Journal on Internet of Things 4, 3 (2020), 42–51

2020
[12]

Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. 2015. Prioritized experience replay. arXiv preprint arXiv:1511.05952 (2015)

2015 arXiv
[13]

Gokul Swamy, Sanjiban Choudhury, Wen Sun, Zhiwei Steven Wu, and J Andrew Bagnell. 2025. All roads lead to likelihood: The value of reinforcement learning in fine-tuning. arXiv preprint arXiv:2503.01067 (2025)

2025
[14]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

2017 arXiv
[15]

Kaimin Wei, Kai Huang, Yongdong Wu, Zhetao Li, Hongliang He, Jilian Zhang, Jinpeng Chen, and Song Guo. 2022. High-performance UAV crowdsensing: A deep reinforcement learning approach. IEEE Internet of Things Journal 9, 19 (2022), 18487–18499

2022
[16]

Jingren Xu, Xin Kang, Ronghaixiang Zhang, Ying-Chang Liang, and Sumei Sun
[17]

Zhenhui Ye, Ke Wang, Yining Chen, Xiaohong Jiang, and Guanghua Song. 2022. Multi-UAV navigation for partially observable communication coverage by graph reinforcement learning. IEEE transactions on mobile computing 22, 7 (2022), 4056–4069

2022
[18]

Xiaochen Zhang, Haitao Zhao, Jibo Wei, Chao Yan, Jun Xiong, and Xiaoran Liu
[19]

Ying Zhang, Meng Yue, Jianhui Wang, and Shinjae Yoo. 2024. Multi-agent graph- attention deep reinforcement learning for post-contingency grid emergency voltage control. IEEE Transactions on Neural Networks and Learning Systems 35, 3 (2024), 3340–3350

2024
[20]

IEEE Transactions on Wireless Communications 22, 3 (2022), 1495–1509

Cooperative trajectory design of multiple UAV base stations with heteroge- neous graph neural networks. IEEE Transactions on Wireless Communications 22, 3 (2022), 1495–1509

2022
[21]

Kun Zhu, Jia Yang, Yang Zhang, Jiangtian Nie, Wei Yang Bryan Lim, Hongliang Zhang, and Zehui Xiong. 2022. Aerial refueling: Scheduling wireless energy charging for UAV enabled data collection. IEEE Transactions on Green Communi- cations and Networking 6, 3 (2022), 1494–1510. M...

2022
[22]

Yang Zhou, Jiuhong Xiao, Yue Zhou, and Giuseppe Loianno. 2022. Multi-robot col- laborative perception with graph neural networks. IEEE Robotics and Automation Letters 7, 2 (2022), 2289–2296

2022
[2022]

IEEE Internet of Things Journal 9, 22 (2022), 22887–22902

Optimization for master-UAV-powered auxiliary-aerial-IRS-assisted IoT networks: An option-based multi-agent hierarchical deep reinforcement learning approach. IEEE Internet of Things Journal 9, 22 (2022), 22887–22902

2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Yining Chen, Guanghua Song, Zhenhui Ye, and Xiaohong Jiang. 2022. Scalable and transferable reinforcement learning for multi-agent mixed cooperative– competitive environments based on hierarchical graph attention. Entropy 24, 4 (2022), 563

work page 2022

[2] [2]

Anna Dai, Rongpeng Li, Zhifeng Zhao, and Honggang Zhang. 2020. Graph convolutional multi-agent reinforcement learning for UAV coverage control. In 2020 International Conference on Wireless Communications and Signal Processing (WCSP). IEEE, 1106–1111

work page 2020

[3] [3]

Jizhe Dou, Haotian Zhang, and Guodong Sun. 2024. Scheduling Drone and Mobile Charger via Hybrid-Action Deep Reinforcement Learning. arXiv preprint arXiv:2403.10761 (2024)

work page Pith review arXiv 2024

[4] [4]

Mingfeng Fan, Yaoxin Wu, Tianjun Liao, Zhiguang Cao, Hongliang Guo, Guil- laume Sartoretti, and Guohua Wu. 2022. Deep reinforcement learning for uav routing in the presence of multiple charging stations. IEEE Transactions on Vehicular Technology 72, 5 (2022), 5732–5746

work page 2022

[5] [5]

Shariq Iqbal and Fei Sha. 2019. Actor-attention-critic for multi-agent reinforce- ment learning. InInternational conference on machine learning. PMLR, 2961–2970

work page 2019

[6] [6]

Rajendra K Jain, Dah-Ming W Chiu, William R Hawe, et al. 1984. A quantitative measure of fairness and discrimination. Eastern Research Laboratory, Digital Equipment Corporation, Hudson, MA 21 (1984), 1

work page 1984

[7] [7]

Chi Harold Liu, Zipeng Dai, Yinuo Zhao, Jon Crowcroft, Dapeng Wu, and Kin K Leung. 2019. Distributed and energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning. IEEE Transactions on Mobile Computing 20, 1 (2019), 130–146

work page 2019

[8] [8]

Chi Harold Liu, Chengzhe Piao, and Jian Tang. 2020. Energy-efficient UAV crowdsensing with multiple charging stations by deep learning. InIEEE INFOCOm 2020-IEEE conference on computer communications . IEEE, 199–208

work page 2020