Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Dynamic Trajectory and Power Control in Ultra-Dense UAV Networks: A Mean-Field Reinforcement Learning Approach

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a dense swarm of self-interested UAVs, each choosing where to fly and how strongly to transmit, has a unique mean-field equilibrium, and that a maximum-entropy deep reinforcement learning algorithm can find it…

desk verdict Useful new MFG formulation for dense UAV resource allocation, but the central equilibrium theorem is not proven—the appendix misses the contraction bound. read the letter →

arxiv 2411.14052 v1 pith:OSOT7RTK submitted 2024-11-21 eess.SY cs.SY

Fei Song , Zhe Wang , Jun Li , Long Shi , Wen Chen , Shi Jin This is my paper · ORCID
classification eess.SYcs.SY
keywords mean-fieldgameUAVnetworkstrajectorydesignpowercontroldeepreinforcementlearningmaximumentropyenergyefficiencyequilibrium
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the non-cooperative resource-allocation problem in ultra-dense UAV networks is tractable: each UAV selfishly chooses its trajectory, user association, and downlink transmit power, and the paper shows this can be formulated as a mean-field game with a unique equilibrium. The motivation is that as UAV densities grow, the standard stochastic-game formulation becomes computationally hopeless, while a mean-field game reduces the interactions to a two-player game between a representative UAV and an aggregate distribution. A model-free maximum-entropy deep Q-network algorithm, ME-MFDQN, is then proposed to approximate the equilibrium, and simulations show it improves energy efficiency over baselines. The practical stake is a scalable, decentralized way to control dense UAV fleets serving ground users with unknown, time-varying demand.

What carries the argument

The carrying object is the mean-field equilibrium, a policy-distribution pair $(\pi^*, L^*)$ in which $\pi^*$ is the representative UAV's best response to the fixed aggregate behavior $L^*$, and $L^*$ is the joint state-action distribution that arises when every UAV follows $\pi^*$. The machinery is the two-step iteration $\Upsilon(L) = \Upsilon_2(\Upsilon_1(L), L)$: fix the mean-field distribution, solve a single-agent reinforcement-learning problem for the best response, then update the distribution through the transition kernel. To make the first step model-free, the paper uses a maximum-entropy soft Q-function, so the policy is the Boltzmann softmax of the Q-values and exploration is controlled by a temperature parameter while the mean-field distribution is updated between training phases.

What would settle it

In the paper's own simulated network, run the two-step iteration from two very different initial mean-field distributions and check whether they converge to the same limit; also estimate the Lipschitz constants in Lemmas 1 and 2 for the reward in (20). If the product $s_1 s_2 + s_3$ is at least 1, or the two runs settle at different distributions, the uniqueness claim of Theorem 1 fails.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: with any initial mean-field distribution $L$, the fixed-point iteration $L' = \Upsilon_2(\Upsilon_1(L), L)$ converges to the unique stationary mean-field equilibrium. In the paper's two-step construction, $\Upsilon_1$ maps a fixed mean-field distribution to the representative UAV's optimal policy, and $\Upsilon_2$ maps that policy back to an updated joint state-action distribution. The proof reasons through the Banach fixed-point theorem, using two Lipschitz lemmas imported from prior work, and the algorithm ME-MFDQN approximates the same equilibrium by maximizing a soft Q-function with a policy-entropy bonus. The paper also extends the formulation to a partially observable scenario where each UAV records a compressed history trajectory, and reports that wider observation helps energy efficiency.

Load-bearing premise

The whole equilibrium proof rides on the assumption that the iteration updating the crowd's behavior always pulls any two possible crowd behaviors closer together; the paper takes this shrinking property from other works and does not verify it for its specific reward and movement model.

Editorial extensions

If this is right

  • If Theorem 1 is right, the two-step iteration converges to the same equilibrium no matter which initial mean-field distribution is chosen, so dense-UAV resource allocation has a well-defined single prediction.
  • Because ME-MFDQN is model-free, UAVs can learn the equilibrium from accumulated experience alone, without knowing the distribution of ground users' service demands.
  • The simulations indicate that as service demands become more temporally correlated, the learned policy flies less and transmits at lower power, increasing energy efficiency, a behavioral prediction of the mean-field model.
  • The algorithm degrades gracefully with partial observability: 75% observation nearly matches full observation, while 25% observation significantly reduces energy efficiency.
  • The robustness experiment shows that a policy trained with 361 UAVs continues to perform well with 351 UAVs, meaning the mean-field solution tolerates small population changes without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to compute the contraction coefficient numerically for the reward in (20) and the transition structure; if it is strictly below 1, the uniqueness proof would be confirmed for this model rather than inherited from other settings.
  • The mean-field assumption of homogeneous, exchangeable UAVs is the price of the simplification; heterogeneous fleets with different propulsion, battery, or cell sizes would likely need a multi-class mean-field formulation, which the paper does not address.
  • The observed robustness to a 10-UAV drop suggests mean-field RL policies could serve as fault-tolerant controllers for UAV networks, but the paper only demonstrates this for one small population change.
  • Comparing ME-MFDQN's learned flying probabilities to an analytic benchmark of optimal user tracking could reveal how close the model-free equilibrium is to the true best response.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies ultra-dense UAV downlink networks with time-varying user demands. Each UAV jointly optimizes its hovering location, user association, and transmit power to maximize its own cumulative energy efficiency under interference and energy constraints. The authors formulate this as a stochastic game, then approximate it as a mean-field game (MFG) in which a representative UAV interacts with a state-action distribution mean-field. They state Theorem 1, claiming that the fixed-point iteration L' = Y2(Y1(L), L) converges to a unique stationary mean-field equilibrium. They then propose a maximum-entropy mean-field deep Q-network (ME-MFDQN) for fully and partially observable scenarios, and present simulations showing energy-efficiency gains over several baselines, plus qualitative trends relating demand predictability and observation radius to flying probability and transmit power.

Significance. The application domain is timely, and the paper includes a fairly complete system model with an energy-harvesting queue, a fly-hover-communicate protocol, and a clear simulation comparison. If the mean-field equilibrium result were fully established, the paper would provide a scalable learning-theoretic foundation for decentralized UAV resource allocation under non-stationary demand. The empirical work is internally consistent, the parameters are listed in detail, and the qualitative conclusions (less flying and lower power when demand is more predictable or observation is wider) are plausible and clearly explained. However, the central theoretical contribution, Theorem 1, is not actually proved in the manuscript; the proof in Appendix A stops at a bound that is not shown to be a contraction. This is a load-bearing gap because the algorithm and the reported gains are justified as approximations to a well-defined equilibrium.

major comments (3)
  1. [Appendix A, Eq. (48)] The proof of Theorem 1 ends with dM(Y(L), Y(Lhat)) <= (c1*c2 + c3) dM(L, Lhat). The Banach fixed-point theorem is then invoked, but the authors never establish the strict contraction condition c1*c2 + c3 < 1. Without this inequality, neither existence nor uniqueness of the mean-field equilibrium follows from the argument presented. This is not a purely cosmetic omission: the constants c1, c2, c3 are introduced only as abstract existence constants in Lemmas 1 and 2, and no bound or parameter condition is given for the specific reward (20), transition structure, or state and action spaces of this UAV model.
  2. [Appendix A, Lemmas 1 and 2] Lemma 1 is imported from [46] and Lemma 2 from [35] with proofs omitted, but the hypotheses of those lemmas are not verified for the present model. In particular, the reward in Eq. (20) contains the energy-efficiency term built from the indicator-based rate expression in Eq. (9) and the min/max nonlinearity in the energy queue update (18), and the state includes the two-state Markov demand process. No argument shows that the reward and transition kernels satisfy the Lipschitz conditions in the measure required by the cited lemmas, so the existence of c1, c2, c3 is not established for this system.
  3. [Section III-B, Theorem 1 and Section IV-A] The theorem is advertised as proving existence and uniqueness of the mean-field equilibrium, and the ME-MFDQN algorithm is described as approximating that equilibrium by iterating between policy optimization (lines 5-15 of Algorithm 1) and mean-field update (line 16). Because the contraction proof is incomplete, the link between the algorithm's empirical convergence curves (Figs. 4-12) and a formal equilibrium guarantee is unsupported. The paper should either provide a complete proof of the contraction constant, or substantially reformulate the theoretical claims to reflect what is actually proven.
minor comments (5)
  1. [Section I] The last paragraph of the introduction says 'Finally, Section VII concludes the paper', but the conclusion is Section VI. Please correct the cross-reference.
  2. [Section II] The text says 'In Section V-B, we will extend the discussions to the partially observable scenario', but the partially observable extension appears in Section IV-B, not V-B.
  3. [Figures 4, 5, 10] The y-axis label is spelled 'Averge reward' in Figures 4(a), 5, and 10; it should be 'Average reward'.
  4. [Algorithm 1] Line 16 computes L_{k+1} from the current policy, but the stopping criterion is only described verbally as 'until the new mean-field L matches L'. A formal convergence criterion or tolerance would improve reproducibility.
  5. [Section IV-B] The observation notation x_o(u,t) in {0,1,2} is introduced, but the mapping from these values to the belief state in the history compression z(t) is not fully specified; please clarify how the history is constructed from the raw observations.

Circularity Check

0 steps flagged · score 2.0 of 10

No constructional circularity; the minor self-citations are not load-bearing, and the main proof gap (unverified contraction constant) is a correctness issue rather than a circular derivation.

full rationale

The central claim is Theorem 1, whose proof rests on Lemmas 1 and 2 imported from the external papers [46] (Huang, Malhamé, and Caines) and [35] (Guo, Hu, Xu, and Zhang), together with the Banach fixed-point theorem. These are not self-citations and they are not equivalent to the paper's target result: the lemmas are standard Lipschitz-continuity statements for mean-field best-response and distribution-update maps, and applying them to the specific reward in Eq. (20) and transition structure in Eq. (18) would require verifying hypotheses that the paper never checks. The missing verification that the composite constant ς1ς2+ς3 is strictly less than 1 in Eq. (48) is a proof gap or correctness risk, not a circularity: the paper does not define the mean-field equilibrium to be the limit of the iteration, nor does it fit any parameter to force the claimed convergence. The reward penalties σ and ξ are design and simulation choices rather than fitted parameters, and the simulation comparisons are against independent baseline algorithms. The self-citations [38] and [45] are used only to motivate the two-state demand Markov model and a robustness experiment; neither supplies the existence-uniqueness argument. Therefore no load-bearing step reduces by construction to its own input, and the paper is not circular beyond minor non-load-bearing self-citations.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central theoretical claim rests on standard mean-field game assumptions (homogeneity, exchangeability, large-population limit) and on two imported Lipschitz lemmas whose applicability is not checked. The highest-risk entry is the unverified contraction condition s1*s2 + s3 < 1, which is exactly what the Banach fixed point argument needs.

free parameters (3)
  • Interference penalty factor sigma = 240 (baseline; varied 160-1120)
    Hand-chosen weight in reward (20); no principled selection; simulation curves vary it to show trade-off.
  • Energy penalty factor xi = not specified
    Appears in reward (20) but no value is given in Section V, so simulations cannot be exactly reproduced.
  • Maximum entropy weight phi = not specified
    Controls entropy bonus in (32); no numerical value provided in Algorithm 1 or in the simulation settings.
assumptions (6)
  • domain assumption All UAVs are homogeneous, indistinguishable, and exchangeable with identical state and action spaces (Section III-B).
    Needed so a representative agent and a single mean-field distribution characterize the population.
  • domain assumption The network is large enough that the aggregate behavior of other UAVs is captured by the mean-field state-action distribution, with individual impact vanishing in the K to infinity limit.
    Used to define L(t) in (23) and to justify the MFG reduction; finite K=361 in simulations is treated as adequate.
  • domain assumption Lemma 1 from [46] applies: the best-response map Y1 is Lipschitz in the mean-field with constant s1.
    Stated in Appendix A as 'given in [46]' without proof or verification of its conditions for this specific reward and transition.
  • domain assumption Lemma 2 from [35] applies: the distribution update Y2 is Lipschitz in policy and mean-field with constants s2 and s3.
    Stated in Appendix A as 'given in [35]' without proof; applicability to the UAV reward structure is not shown.
  • ad hoc to paper The composite contraction constant satisfies s1*s2 + s3 < 1, so the Banach fixed point theorem applies.
    Appendix A derives d(Y(L), Y(Lhat)) <= (s1*s2 + s3) d(L, Lhat), but never bounds the product below 1; without this the claimed uniqueness does not follow.
  • domain assumption Service demands follow a two-state Markov chain with transition probabilities p and q, as described in Section II.
    The entire simulation environment and the partially observable inference task rely on this Markov model; only q=0.7 is given in Section V.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic Trajectory and Power Control in Ultra-Dense UAV Networks: A Mean-Field Reinforcement Learning Approach." pith.science (2026). https://pith.science/paper/OSOT7RTK

@misc{pith2026241114052,
  author       = {Pith},
  title        = {Pith review of: Dynamic Trajectory and Power Control in Ultra-Dense UAV Networks: A Mean-Field Reinforcement Learning Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OSOT7RTK}},
  note         = {Machine review of arXiv:2411.14052}
}
read the original abstract

In ultra-dense unmanned aerial vehicle (UAV) networks, it is challenging to coordinate the resource allocation and interference management among large-scale UAVs, for providing flexible and efficient service coverage to the ground users (GUs). In this paper, we propose a learning-based resource allocation scheme in an ultra-dense UAV communication network, where the GUs' service demands are time-varying with unknown distributions. We formulate the non-cooperative game among multiple co-channel UAVs as a stochastic game, where each UAV jointly optimizes its trajectory, user association, and downlink power control to maximize the expectation of its locally cumulative energy efficiency under the interference and energy constraints. To cope with the scalability issue in a large-scale network, we further formulate the problem as a mean-field game (MFG), which simplifies the interactions among the UAVs into a two-player game between a representative UAV and a mean-field. We prove the existence and uniqueness of the equilibrium for the MFG, and propose a model-free mean-field reinforcement learning algorithm named maximum entropy mean-field deep Q network (ME-MFDQN) to solve the mean-field equilibrium in both fully and partially observable scenarios. The simulation results reveal that the proposed algorithm improves the energy efficiency compared with the benchmark algorithms. Moreover, the performance can be further enhanced if the GUs' service demands exhibit higher temporal correlation or if the UAVs have wider observation capabilities over their nearby GUs.

Figures

Figures reproduced from arXiv: 2411.14052 by the authors.

Figure 1
Figure 1. An ultra-dense UAV network provides communica [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Division of a time slot. B. Communication Model The channel coefficient hk,u(t) between UAV k and GU u for time slot t can be expressed as hk,u(t) = q lk,u(t)gk,u(t), (1) where gk,u(t) is the small-scale fading, lk,u(t) is the path loss characterizing the large-scale fading. The large-scale fading [39] is modeled as a random variable depending on the LoS probability P rLoS k,u (t) and non-LoS (NLoS) probability 1 − … view at source ↗
Figure 3
Figure 3. The two-step iterative solution for MFG. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Average reward, energy efficiency and interference p [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Average reward per episode for the active UAVs. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: The impact of self-transition probability [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 8
Figure 8. Figure 8: The impact of penalty factor σ on the optimal policies: (a) the flying probability versus σ; (b) the average transmit power versus σ. 160 320 480 640 800 9601120 Penalty factor 0 0.5 1 1.5 2 2.5 Energy efficiency 103 (d) = -4 dB 160 320 480 640 800 9601120 Penalty fact…
Figure 9
Figure 9. Figure 9: Energy efficiency versus various penalty factor [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 11
Figure 11. Figure 11: The impact of self-transition probability [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Energy efficiency versus various self-transition [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages

  1. [46]

    Large popula tion stochastic dynamic games: Closed-loop McKean-Vlasov syst ems and the Nash certainty equivalence principle,

    M. Huang, R. P . Malham´ e, and P . E. Caines, “Large popula tion stochastic dynamic games: Closed-loop McKean-Vlasov syst ems and the Nash certainty equivalence principle,” Commun. Inf. Syst. , vol. 6, no. 3, pp. 221–252, Mar. 2006

    work page 2006

  2. [35]

    Learning mean-field ga mes,

    X. Guo, A. Hu, R. Xu, and J. Zhang, “Learning mean-field ga mes,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , V ancouver, Canada, Dec. 2019, pp. 4966–4976

    work page 2019

  3. [1]

    Energy-effi cient UA V control for effective and fair communication coverage: A deep reinforcement learning approach,

    C. H. Liu, Z. Chen, J. Tang, J. Xu, and C. Piao, “Energy-effi cient UA V control for effective and fair communication coverage: A deep reinforcement learning approach,” IEEE J. Sel. Areas Commun. , vol. 36, no. 9, pp. 2059–2070, Sep. 2018

    work page 2018

  4. [2]

    UA V communications based on non-orthogonal multiple access,

    Y . Liu, Z. Qin, Y . Cai, Y . Gao, G. Y . Li, and A. Nallanathan, “UA V communications based on non-orthogonal multiple access,” IEEE Wireless Commun., vol. 26, no. 1, pp. 52–57, Feb. 2019

    work page 2019

  5. [3]

    A survey of channel modeling for UA V communications,

    A. A. Khuwaja, Y . Chen, N. Zhao, M. S. Alouini, and P . Dobbi ns, “A survey of channel modeling for UA V communications,” IEEE Commun. Surveys Tuts. , vol. 20, no. 4, pp. 2804–2821, 4th Quart. 2018

    work page 2018

  6. [4]

    Optimal 3D-trajectory design and resource allocation for solar-po wered UA V communication systems,

    Y . Sun, D. Xu, D. W. K. Ng, L. Dai, and R. Schober, “Optimal 3D-trajectory design and resource allocation for solar-po wered UA V communication systems,” IEEE Trans. Commun. , vol. 67, no. 6, pp. 4281–4298, Jun. 2019

    work page 2019

  7. [5]

    Energy efficient 3 - D UA V control for persistent communication service and fair ness: A deep reinforcement learning approach,

    H. Qi, Z. Hu, H. Huang, X. Wen, and Z. Lu, “Energy efficient 3 - D UA V control for persistent communication service and fair ness: A deep reinforcement learning approach,” IEEE Access , vol. 8, pp. 53 172–53 184, Mar. 2020

    work page 2020

  8. [6]

    Wireles s communication using unmanned aerial vehicles (UA Vs): Opti mal transport theory for hover time optimization,

    M. Mozaffari, W. Saad, M. Bennis, and M. Debbah, “Wireles s communication using unmanned aerial vehicles (UA Vs): Opti mal transport theory for hover time optimization,” IEEE Trans. Wireless Commun., vol. 16, no. 12, pp. 8052–8066, Dec. 2017

    work page 2017

Show all 46 references
  1. [7]

    UA V communications for 5G and beyond: Recent advances and future trends,

    B. Li, Z. Fei, and Y . Zhang, “UA V communications for 5G and beyond: Recent advances and future trends,” IEEE Internet Things J., vol. 6, no. 2, pp. 2241–2263, Apr. 2019

    2019

  2. [8]

    Toward advanc ed UA V communications: Properties, research challenges, and future potential,

    B. P . S. Sahoo, D. Puthal, and P . K. Sharma, “Toward advanc ed UA V communications: Properties, research challenges, and future potential,” IEEE Internet Things Mag. , vol. 5, no. 1, pp. 154–159, Mar. 2022

    2022

  3. [9]

    High-performance UA V crowdsensing: A deep reinfor ce- ment learning approach,

    K. Wei, K. Huang, Y . Wu, Z. Li, H. He, J. Zhang, J. Chen, and S. Guo, “High-performance UA V crowdsensing: A deep reinfor ce- ment learning approach,” IEEE Internet Things J. , vol. 9, no. 19, pp. 18 487–18 499, Oct. 2022

    2022

  4. [10]

    Delay-sensitive energy-efficient UA V crowdsensing by dee p rein- forcement learning,

    Z. Dai, C. H. Liu, R. Han, G. Wang, K. K. Leung, and J. Tang, “Delay-sensitive energy-efficient UA V crowdsensing by dee p rein- forcement learning,” IEEE Trans. Mobile Comput. , vol. 22, no. 4, pp. 2038–2052, Apr. 2023

    2023

  5. [11]

    A UA V-ass isted multi-task allocation method for mobile crowd sensing,

    H. Gao, J. Feng, Y . Xiao, B. Zhang, and W. Wang, “A UA V-ass isted multi-task allocation method for mobile crowd sensing,” IEEE Trans. Mobile Comput. , pp. 1–16, Early Access, 2022

    2022

  6. [12]

    Secure communications for UA V-enabled mobile ed ge computing systems,

    Y . Zhou, C. Pan, P . L. Y eoh, K. Wang, M. Elkashlan, B. Vuce tic, and U. Li, “Secure communications for UA V-enabled mobile ed ge computing systems,” IEEE Trans. Commun. , vol. 68, no. 1, pp. 376– 388, Jan. 2020

    2020

  7. [13]

    Multi-UA V trajectory and power optimization for cached UA V wireless networks with energy a nd content recharging-demand driven deep learning approach,

    S. Chai and V . K. N. Lau, “Multi-UA V trajectory and power optimization for cached UA V wireless networks with energy a nd content recharging-demand driven deep learning approach, ” IEEE J. Sel. Areas Commun. , vol. 39, no. 10, pp. 3208–3224, Oct. 2021

    2021

  8. [14]

    UA V trajectory optimization for data offloading at t he edge of multiple cells,

    F. Cheng, S. Zhang, Z. Li, Y . Chen, N. Zhao, F. R. Y u, and V . C. M. Leung, “UA V trajectory optimization for data offloading at t he edge of multiple cells,” IEEE Trans. V eh. Technol. , vol. 67, no. 7, pp. 6732–6736, Jul. 2018

    2018

  9. [15]

    Distributed energy -efficient multi-UA V navigation for long-term communication coverag e by deep reinforcement learning,

    C. H. Liu, X. Ma, X. Gao, and J. Tang, “Distributed energy -efficient multi-UA V navigation for long-term communication coverag e by deep reinforcement learning,” IEEE Trans. Mobile Comput. , vol. 19, no. 6, pp. 1274–1285, Jun. 2020

    2020

  10. [16]

    Mu lti- UA V trajectory planning for energy-efficient content cover age: A decentralized learning-based approach,

    C. Zhao, J. Liu, M. Sheng, W. Teng, Y . Zheng, and J. Li, “Mu lti- UA V trajectory planning for energy-efficient content cover age: A decentralized learning-based approach,” IEEE J. Sel. Areas Commun., vol. 39, no. 10, pp. 3193–3207, Oct. 2021

    2021

  11. [17]

    Collaborative computation offloading and resou rce allocation in multi-UA V-assisted IoT networks: A deep reinforcement learning approach,

    A. M. Seid, G. O. Boateng, S. Anokye, T. Kwantwi, G. Sun, and G. Liu, “Collaborative computation offloading and resou rce allocation in multi-UA V-assisted IoT networks: A deep reinforcement learning approach,” IEEE Internet Things J. , vol. 8, no. 15, pp. 12 203–12 218, Aug. 2021

    2021

  12. [18]

    Path pla nning for UA V-mounted mobile edge computing with deep reinforcem ent learning,

    Q. Liu, L. Shi, L. Sun, J. Li, M. Ding, and F. Shu, “Path pla nning for UA V-mounted mobile edge computing with deep reinforcem ent learning,” IEEE Trans. V eh. Technol., vol. 69, no. 5, pp. 5723–5728, May 2020

    2020

  13. [19]

    3D UA V trajectory design and frequency band allocation for energy-efficient and fair com munica- tion: A deep reinforcement learning approach,

    R. Ding, F. Gao, and X. S. Shen, “3D UA V trajectory design and frequency band allocation for energy-efficient and fair com munica- tion: A deep reinforcement learning approach,” IEEE Trans. Wireless Commun., vol. 19, no. 12, pp. 7796–7809, Dec. 2020

    2020

  14. [20]

    Joint 3D deployment and powe r allocation for UA V -BS: A deep reinforcement learning appro ach,

    M. Zhang, S. Fu, and Q. Fan, “Joint 3D deployment and powe r allocation for UA V -BS: A deep reinforcement learning appro ach,” IEEE Wireless Commun. Lett. , vol. 10, no. 10, pp. 2309–2312, Oct. 2021

    2021

  15. [21]

    Multi-agent deep reinforcement learning-based trajecto ry planning for multi-UA V assisted mobile edge computing,

    L. Wang, K. Wang, C. Pan, W. Xu, N. Aslam, and L. Hanzo, “Multi-agent deep reinforcement learning-based trajecto ry planning for multi-UA V assisted mobile edge computing,” IEEE Trans. Cognit. Commun. Netw., vol. 7, no. 1, pp. 73–84, Mar. 2021

    2021

  16. [22]

    Multi- agent DRL for task offloading and resource allocation in mult i- UA V enabled IoT edge network,

    A. M. Seid, G. O. Boateng, B. Mareri, G. Sun, and W. Jiang, “Multi- agent DRL for task offloading and resource allocation in mult i- UA V enabled IoT edge network,” IEEE Trans. Netw. Service Manag., vol. 18, no. 4, pp. 4531–4547, Dec. 2021

    2021

  17. [23]

    Multi-agent reinforcement learni ng based re- source management in MEC- and UA V-assisted vehicular netwo rks,

    H. Peng and X. Shen, “Multi-agent reinforcement learni ng based re- source management in MEC- and UA V-assisted vehicular netwo rks,” IEEE J. Sel. Areas .Commun. , vol. 39, no. 1, pp. 131–141, Jan. 2021

    2021

  18. [24]

    Multi-agent reinfo rcement learning-based resource allocation for UA V networks,

    J. Cui, Y . Liu, and A. Nallanathan, “Multi-agent reinfo rcement learning-based resource allocation for UA V networks,” IEEE Trans. Wireless Commun., vol. 19, no. 2, pp. 729–743, Feb. 2020

    2020

  19. [25]

    Multiagent collaborative learning for UA V enabl ed wire- less networks,

    W. Xia, Y . Zhu, L. D. Simone, T. Dagiuklas, K. K. Wong, and G. Zheng, “Multiagent collaborative learning for UA V enabl ed wire- less networks,” IEEE J. Sel. Areas Commun. , vol. 40, no. 9, pp. 2630–2642, Sep. 2022

    2022

  20. [26]

    Interferenc e management for cellular-connected UA Vs: A deep reinforcement learnin g ap- proach,

    U. Challita, W. Saad, and C. Bettstetter, “Interferenc e management for cellular-connected UA Vs: A deep reinforcement learnin g ap- proach,” IEEE Trans. Wireless Commun. , vol. 18, no. 4, pp. 2125– 2140, Apr. 2019

    2019

  21. [27]

    Downlink power control i n self- organizing dense small cells underlaying macrocells: A mea n field game,

    P . Semasinghe and E. Hossain, “Downlink power control i n self- organizing dense small cells underlaying macrocells: A mea n field game,” IEEE Trans. Mobile Comput. , vol. 15, no. 2, pp. 350–363, Feb. 2016

    2016

  22. [28]

    Delay optimization in multi-UA V edge caching netwo rks: A robust mean field game,

    L. Li, M. Wang, K. Xue, Q. Cheng, D. Wang, W. Chen, M. Pan, a nd Z. Han, “Delay optimization in multi-UA V edge caching netwo rks: A robust mean field game,” IEEE Trans. V eh. Technol., vol. 70, no. 1, pp. 808–819, Jan. 2021

    2021

  23. [29]

    Multi- UA V delay optimization in edge caching networks: A mean field gam e approach,

    K. Xue, L. Li, F. Y ang, H. Zhang, X. Li, and Z. Han, “Multi- UA V delay optimization in edge caching networks: A mean field gam e approach,” in Proc. Wireless Opt. Commun. Conf. (WOCC) , Beijing, China, May 2019, pp. 1–5

    2019

  24. [30]

    Mean-field game theory based altitude control strategy for massive UA V relay- assisted mobile edge computing,

    W. Cai, D. Zhang, Z. Chen, W. Luo, and Y . Tang, “Mean-field game theory based altitude control strategy for massive UA V relay- assisted mobile edge computing,” in Proc. IEEE Wireless Commun. Networking Conf. (WCNC), Dubai, United Arab Emirates, Apr. 2024, pp. 1–6

    2024

  25. [31]

    Massive UA V-to - ground communication and its stable movement control: A mea n- field approach,

    H. Kim, J. Park, M. Bennis, and S. L. Kim, “Massive UA V-to - ground communication and its stable movement control: A mea n- field approach,” in Proc. IEEE Int. W orkshop Signal Process. Adv. Wireless Commun. (SPAWC), Kalamata, Greece, Jun. 2018, pp. 1–5

    2018

  26. [32]

    Joint power control and scheduling for high-dyn amic multi-hop UA V communication: A robust mean field game,

    T. Li, C. Y ang, L. Chang, L. Y ang, P . Gong, H. Dai, J. Shen, M. Wen, and Z. Han, “Joint power control and scheduling for high-dyn amic multi-hop UA V communication: A robust mean field game,” IEEE Access, vol. 9, pp. 130 649–130 664, 2021

    2021

  27. [33]

    Mea n field multi-agent reinforcement learning,

    Y . Y ang, R. Luo, M. Li, M. Zhou, W. Zhang, and J. Wang, “Mea n field multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML) , Stockholm, Sweden, Jul. 2018, pp. 5567–5576

    2018

  28. [34]

    Approximately solving mean field g ames via entropy-regularized deep reinforcement learning,

    K. Cui and H. Koeppl, “Approximately solving mean field g ames via entropy-regularized deep reinforcement learning,” in Proc. Int. Conf. Artif. Intell. Statist. (AISTATS) , San Diego, USA, Apr. 2021, pp. 1–10

    2021

  29. [36]

    Downlink tra nsmit power control in ultra-dense UA V network based on mean field g ame and deep reinforcement learning,

    L. Li, Q. Cheng, K. Xue, C. Y ang, and Z. Han, “Downlink tra nsmit power control in ultra-dense UA V network based on mean field g ame and deep reinforcement learning,” IEEE Trans. V eh. Technol., vol. 69, no. 12, pp. 15 594–15 605, Dec. 2020

    2020

  30. [37]

    Me an field deep reinforcement learning for fair and efficient UA V c ontrol,

    D. Chen, Q. Qi, Z. Zhuang, J. Wang, J. Liao, and Z. Han, “Me an field deep reinforcement learning for fair and efficient UA V c ontrol,” IEEE Internet Things J. , vol. 8, no. 2, pp. 813–828, Jan. 2021

    2021

  31. [38]

    Chase or wait: Dynamic UA V deployme nt to learn and catch time-varying user activities,

    Z. Wang and L. Duan, “Chase or wait: Dynamic UA V deployme nt to learn and catch time-varying user activities,” IEEE Trans. Mobile Comput., vol. 22, no. 3, pp. 1369–1383, Mar. 2023

    2023

  32. [39]

    Energy minimization for wi reless communication with rotary-wing UA V,

    Y . Zeng, J. Xu, and R. Zhang, “Energy minimization for wi reless communication with rotary-wing UA V,” IEEE Trans. Wireless Com- mun., vol. 18, no. 4, pp. 2329–2345, Apr. 2019

    2019

  33. [40]

    Decen tralized federated reinforcement learning for user-centric dynami c TFDD control,

    Z. Yin, Z. Wang, J. Li, M. Ding, W. Chen, and S. Jin, “Decen tralized federated reinforcement learning for user-centric dynami c TFDD control,” IEEE J. Sel. Top. Signal Process. , vol. 17, no. 1, pp. 40–53, Jan. 2023

    2023

  34. [41]

    Downlink coverage and rate analysis of an aerial user in ver tical heterogeneous networks (VHetNets),

    N. Cherif, M. Alzenad, H. Y anikomeroglu, and A. Y ongaco glu, “Downlink coverage and rate analysis of an aerial user in ver tical heterogeneous networks (VHetNets),” IEEE Trans. Wireless Com- mun., vol. 20, no. 3, pp. 1501–1516, Mar. 2021

    2021

  35. [42]

    Reinfor cement learning with deep energy-based policies,

    T. Haarnoja, H. Tang, P . Abbeel, and S. Levine, “Reinfor cement learning with deep energy-based policies,” in Proc. Int. Conf. Mach. Learn. (ICML) , Sydney, Australia, Aug. 2017, pp. 1352–1361. 14

    2017

  36. [43]

    Adaptive deployment for UA V- aided communication networks,

    Z. Wang, L. Duan, and R. Zhang, “Adaptive deployment for UA V- aided communication networks,” IEEE Trans. Wireless Commun. , vol. 18, no. 9, pp. 4531–4543, Sep. 2019

    2019

  37. [44]

    Probabilistic caching for small-cell networks with terrestrial and aerial users,

    F. Song, J. Li, M. Ding, L. Shi, F. Shu, M. Tao, W. Chen, and H. V . Poor, “Probabilistic caching for small-cell networks with terrestrial and aerial users,” IEEE Trans. V eh. Technol., vol. 68, no. 9, pp. 9162– 9177, Sep. 2019

    2019

  38. [45]

    Decision transformers for wireless communications : A new paradigm of resource management,

    J. Zhang, J. Li, Z. Wang, L. Shi, S. Jin, W. Chen, and H. V . Poor, “Decision transformers for wireless communications : A new paradigm of resource management,” IEEE Wireless Commun. , to appear

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.