{"id":"acadd3fa-5244-47b0-b912-cb34c369a8bb","arxiv_id":"2502.05824","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A new reinforcement learning algorithm, EMOPPO-VLH, jointly optimizes UAV positions and beamforming weights to serve a moving ground user while minimizing flight energy.","lead":"This paper trains a swarm of drones to act as one virtual antenna, deciding where to fly and how to beam signals to a moving phone user while conserving battery. It combines evolutionary search with deep reinforcement learning to produce a family of trade-off policies. A generalist may care because it addresses the practical problem of keeping mobile users connected with limited airborne power.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The superiority claim rests on single-run comparisons without variance estimates; PPO-family methods are high-variance, so the reported margins may be within run-to-run noise. A multi-seed rerun is needed before 'smallest IGD / largest HV' can be taken as established.","rationale":"The paper is a serious engineering contribution: it formulates a nontrivial MOMDP with realistic mobility and energy models, and the algorithmic machinery (LSTM-augmented PPO, vectorized value function, performance buffers, hyper-sphere selection) is described in enough detail to be reimplemented. The reader's verdict of CONDITIONAL is appropriate, and this stress-test does not move it. The most load-bearing concern is internal validity of the empirical comparison, not the sim-to-real transferability highlighted as the reader's weakest assumption: the paper's own abstract and Section 6.4 stake the contribution on the simulation results, so if those results are statistically ungrounded, the central claim fails regardless of transferability. The reader did note the absence of error bars and code in their rationale, but framed the weakest assumption differently. A multi-seed rerun is the decisive check: if confidence intervals overlap between EMOPPO-VLH and the strongest RL baselines, the headline claim should be weakened; if the gaps persist, the conditional concerns are resolved. No ad hominem is intended; the issue is purely the missing statistical support for a universal superiority claim.","tokens_in":28563,"tokens_out":6258,"duration_ms":67150,"concrete_test":"Obtain the code (or rerun from the paper's hyperparameters) and repeat the small-scale and large-scale experiments with at least 10 independent random seeds for EMOPPO-VLH, EPPO-GRU, ETD3, MOEA/D, and MOPSO, fixing identical environment seeds within each trial. Report mean and 95% confidence intervals (or per-seed points) for final f1, f2, IGD, and HV at the last generation, and explicitly state how the IGD reference front is constructed. If the best-algorithm label changes across seeds or the confidence intervals overlap, the claim of universal superiority is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 6.4 asserts that EMOPPO-VLH 'can achieve the smallest IGD values and the largest HV values across all scenarios' and 'outperforms all other optimization approaches across both scales.' The supporting evidence in Figs. 5–8 consists of single convergence curves and bar charts with no error bars, and the paper reports no number of random seeds. EMOPPO-VLH is an on-policy PPO variant with evolutionary task selection (n=15 tasks, G=100 generations, stochastic LSTM policy and value networks); PPO is known to exhibit high variance across seeds. Without run-to-run statistics, the observed gaps over EPPO-GRU and ETD3 cannot be distinguished from seed noise. Additionally, IGD and HV require a reference Pareto front, and the full text does not state how the reference set is constructed; if the reference is the union of all obtained fronts, the diversity-oriented hyper-sphere selection could trivially inflate the reported IGD/HV metrics. The load-bearing condition for the headline claim is therefore the statistical and metric validity of the comparison, not the real-world transferability asserted in Sec. 5.5.1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies a UAV swarm that forms a virtual antenna array (UVAA) via collaborative beamforming to serve a terrestrial mobile user, under interference from a non-associated base station and time-varying channels. It formulates a long-term multi-objective optimization problem (MOP) with two conflicting objectives: maximizing the total achievable rate and minimizing UAV flight energy consumption. After arguing that the problem is NP-hard, the authors convert it into a multi-objective Markov decision process and propose EMOPPO-VLH, an evolutionary multi-objective PPO variant combining a vectorized value function, LSTM networks, and a hyper-sphere-based task selection mechanism. The paper reports simulations for small- and large-scale UAV swarms and claims that EMOPPO-VLH achieves the smallest IGD and largest HV values and outperforms six baseline algorithms, with additional robustness experiments summarized in appendices. The central empirical claim is that EMOPPO-VLH outperforms existing methods in both objective values and Pareto-set quality.","tokens_in":28856,"tokens_out":4109,"duration_ms":44180,"significance":"If the claimed performance is sustained, the work would be a useful engineering contribution to UAV-enabled collaborative beamforming with mobile users: the system model is reasonably detailed, the use of LSTM to capture user mobility is sensible, and the evolutionary task-selection idea is a plausible way to improve Pareto diversity in multi-objective reinforcement learning. The paper supplies algorithmic pseudo-code, complexity expressions, and a substantial set of scenarios, which makes the method reproducible in principle. However, the manuscript does not release code or data, and the headline quantitative claims rest entirely on simulation curves without variance information. The comparison also omits the authors' own closest prior evolutionary MORL algorithm [29], which weakens the support for the 'outperforms all other approaches' statement. The lack of seed statistics and an unspecified reference Pareto front are load-bearing for the main claims, so the paper needs a revision that strengthens the empirical evidence rather than a purely editorial fix.","major_comments":[{"comment":"The central claim that EMOPPO-VLH 'can achieve the smallest IGD values and the largest HV values across all scenarios' and 'outperforms all other optimization approaches across both scales' is supported only by single convergence curves and bar charts without error bars, and the number of random seeds is not reported. EMOPPO-VLH is an on-policy PPO variant, a class known to have high run-to-run variance; without multiple seeds and variance or significance statistics, the reported margins over EPPO-GRU and ETD3 cannot be distinguished from seed noise. Please rerun all compared algorithms with multiple seeds and report mean and standard deviation for f1, f2, IGD, and HV, and state the seed count explicitly.","section":"Sec. 6.4, Figs. 5-8"},{"comment":"The IGD and HV metrics require a reference Pareto front and a reference point, but the manuscript does not specify how the reference set is constructed or how the reference point is chosen. If the reference front is the union of all obtained fronts of all algorithms, then the hyper-sphere-based diversity mechanism could inflate IGD/HV values by construction, because a more spread-out set of solutions would appear better even if it does not dominate the true Pareto front. Please state the reference-front construction, the normalization procedure, and the reference-point selection, and verify that the reported IGD/HV improvements are not an artifact of the metric setup.","section":"Sec. 6.2 and Sec. 6.4"},{"comment":"The NP-hardness argument is not rigorous. The authors reduce to minimizing -R_UM over excitation current weights with 0 ≤ I_i ≤ 1 and a single constraint g(X') < N, and then assert this is a nonlinear knapsack problem that is NP-hard. This is not a valid reduction: no known NP-hard problem is mapped to this instance, and a continuous linear knapsack with one constraint is solvable in polynomial time. The statement that the original problem is 'more complex than f1'' does not establish NP-hardness. Please replace this subsection with a proper complexity argument or soften the claim to 'computationally challenging' with supporting justification.","section":"Sec. 4.2, Eq. (16)"},{"comment":"The experimental comparison does not include the closest prior algorithm, namely the authors' own evolutionary multi-objective DRL method [29], which the paper explicitly describes as the basis for EMOPPO-VLH. Since the contribution is framed as an improvement over [29] for a different scenario, omitting it from the baselines makes it impossible to verify the key novelty claim. Please add [29] as a baseline (adapted to the current setting with a clear description of the adaptation) or justify its exclusion in the text.","section":"Sec. 6.3 and Sec. 6.4"},{"comment":"The statement that 'the policy trained in this environment can be effectively applied to real-world scenarios' is asserted without validation. The simulation environment uses Rician fading, a Gauss-Markov user mobility model, a specific UAV energy model, and beamforming-gain formulas; no real-world measurements, channel traces, or hardware tests are provided. If simulation-to-real transferability is part of the contribution, it needs evidence or should be explicitly demoted to an assumption and limitation, rather than presented as a conclusion.","section":"Sec. 5.5.1"}],"minor_comments":[{"comment":"The contribution bullet refers to the proposed algorithm as 'MOPPO-PLE', but the algorithm is otherwise named EMOPPO-VLH throughout the paper; please correct the typo and keep the name consistent.","section":"Sec. 1, contribution bullet"},{"comment":"The text compares 'EMOPPO-VLH with LSTM' with 'EMOPPO-GRU', but the baseline introduced in Sec. 6.3 is called 'EPPO with GRU'; the naming is inconsistent and should be unified.","section":"Sec. 6.4"},{"comment":"Algorithm 4 uses the parameter name 'k' in its input list while the surrounding text defines 'kcan'; please align these notations.","section":"Sec. 5.5.2 and Algorithm 4"},{"comment":"The reference point Zref is used in Algorithm 3 and in the text, but its value or construction is never specified in the main text; please define it or point to a precise appendix section.","section":"Algorithms 3 and 4"},{"comment":"The sentence stating that EMOPPO-VLH is better than 'the other five algorithms' is inaccurate because six baselines are listed (MOEA/D, MOPSO, EDDPG, ETD3, EPPO, and EPPO with GRU).","section":"Sec. 6.4"},{"comment":"In the Rician fading PDF, the symbol Ω_UM is used both as the random variable and as its mean normalization constant, which is confusing; please use a separate notation for the mean, such as E[Ω].","section":"Eq. (4)"},{"comment":"The paper repeatedly refers to Appendix B, Appendix C, and Appendix D, but the provided arXiv version does not include these appendices; please ensure that the supplementary material is available to reviewers or summarize the key results in the main text.","section":"Secs. 6.2, 6.5, and 7"}],"recommendation":"major_revision","confidential_remarks":"The paper's main contribution claim is an improvement over the authors' previous algorithm [29], yet [29] is absent from the experimental comparison; this is important for judging the novelty and should be checked carefully. The IGD/HV reference-set issue is the single most impactful technical ambiguity: if the reference front is built from the union of all obtained fronts, the diversity-oriented selection can trivially improve the reported metrics. The missing seed statistics are routine to fix and should be required."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a plausible but incremental extension of the authors' own evolutionary MORL line, and the headline superiority claim is not yet established because all comparisons come from single-run simulations with no error bars and no described reference front for IGD/HV.\n\nWhat is actually new: applying LSTM-augmented multi-objective PPO with hyper-sphere task selection to UAV collaborative beamforming serving a mobile ground user, under Rician fading, Gauss-Markov mobility, and interference from a non-associated BS. The system model is reasonably detailed, the action space is sensible, and the experiments include six baselines plus an EPPO-GRU ablation and a complexity analysis. That is a solid, if narrow, engineering contribution.\n\nThe soft spots are real and mostly statistical. Section 6.4 claims the smallest IGD and largest HV across all scenarios, but Figures 5-8 show no error bars, and no number of seeds is reported. PPO-family methods are high-variance; the margin over EPPO-GRU in particular could easily be seed noise. The paper also never states how the reference Pareto front for IGD/HV is constructed. If it is the union of all obtained fronts, the hyper-sphere diversity mechanism can inflate the metrics. This is load-bearing and fixable: run multiple seeds, report mean and spread, and specify the reference set.\n\nSecond, the closest predecessor, the authors' own [29] (evolutionary MORL for LEO satellite CB), is not benchmarked. Since the claimed improvement is relative to that line, its absence leaves the increment partially unmeasured. Third, the NP-hard argument in Section 4.2 is a one-line reduction to 'nonlinear knapsack' with no proof; likely true, but not rigorous. Minor. Fourth, Section 5.5.1 asserts the trained policy transfers to real scenarios without any validation. That is common in this literature, but it should be presented as an assumption, not a conclusion.\n\nThe reader's stress-test concern is valid on the evidence; I do not see a fatal methodology flaw, just an unproven statistical claim. The work deserves peer review, but the review should require multi-seed experiments, error bars, a stated IGD/HV reference, and ideally a comparison against [29] before the superiority claim is accepted.","headline":"A plausible but incremental MORL-for-UAV-CB paper whose headline claim rests on single-run simulations with no variance and an unspecified IGD/HV reference; it deserves review after statistical fixes.","tokens_in":29379,"tokens_out":3005,"would_cite":false,"duration_ms":30807,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A UAV swarm can serve a moving ground user by acting as one virtual antenna array, and an evolutionary multi-objective RL algorithm charts the rate-versus-energy trade-off.","keywords":["UAV communications","collaborative beamforming","virtual antenna array","multi-objective reinforcement learning","proximal policy optimization","LSTM","user mobility","Pareto front"],"falsifier":"Run a field trial with 8 rotary-wing UAVs and a GPS-tracked mobile user under the paper's Rician-and-interference parameters, then compare measured total achievable rate and total energy consumption to the simulated Pareto front; if the realized points are dominated by a baseline policy or lie far outside the front, the central claim fails.","tokens_in":28388,"feed_emoji":"📡","tokens_out":8015,"duration_ms":76976,"temperature":0.7,"pith_summary":"Small UAVs can pool their antennas into a virtual array and, by moving and adjusting transmit phases in real time, keep a high-rate link to a ground user who moves unpredictably. This paper tries to establish that this joint control task — choosing each UAV's trajectory and excitation current weight at every time slot — should be treated as an NP-hard, long-term multi-objective problem that trades total achievable rate against flight energy, and that an evolutionary deep reinforcement-learning algorithm can solve it online. The proposed method, EMOPPO-VLH, extends PPO with a vectorized value function, LSTM memory for the user's Gauss–Markov motion and time-varying Rician channels, and a hyper-sphere-based task-selection rule that keeps the Pareto set diverse. If the reported simulations are right, an operator can pick from this Pareto set a policy matching the current preference for throughput versus battery life, and update the policy set when the environment changes.","feed_headline":"UAV swarms fly as one antenna to serve moving users","feed_subtitle":"A multi-objective RL algorithm returns a rate-versus-energy menu and beats six baselines in simulation.","key_machinery":"The central object is the UAV-enabled virtual antenna array (UVAA), whose array factor $AF[t](\\theta,\\varphi)=\\sum_{i=1}^N I_i[t]e^{jk_c(x_i^U[t]\\sin\\theta\\cos\\varphi+y_i^U[t]\\sin\\theta\\sin\\varphi+z_i^U[t]\\cos\\theta)}$ determines the beamforming gain toward the mobile user; every rate calculation flows through this quantity. The algorithmic machinery is the MOMDP formulation with vectorized reward $r[t]=(R_{UM}[t], -\\epsilon_1 E[t])$ and a vectorized value function, extended with LSTM layers in the actor and critic to capture temporal dependencies in user movement and channel fading, and with a hyper-sphere-based task-selection step that biases training toward sparse regions of the objective space so the final Pareto set is diverse.","core_discovery":"The paper claims that a swarm of rotary-wing UAVs can maintain a reliable high-rate downlink to a terrestrial user who moves randomly by treating the whole swarm as a UAV-enabled virtual antenna array (UVAA) and optimizing, at every time slot, each UAV's horizontal direction, horizontal and vertical flight distances, and excitation current weight. It argues that the resulting long-term problem—maximize total achievable rate $\\sum_t R_{UM}[t]$ while minimizing total flight energy $\\sum_t \\sum_i E_i[t]$—is NP-hard and too dynamic for offline methods, so it converts it into a multi-objective Markov decision process and solves it with EMOPPO-VLH. EMOPPO-VLH evolves a population of PPO policies whose value functions are vectorized to handle two rewards, whose actor and critic networks use LSTM layers to remember user and channel history, and whose hyper-sphere-based task selection keeps the approximate Pareto set well spread. In simulations with 8 and 16 UAVs, the paper reports that EMOPPO-VLH attains the smallest inverted generational distance and the largest hypervolume among the compared algorithms, together with the best total achievable rate and energy consumption.","pith_inferences":["Not tested in the paper is the step from simulation to hardware; if that transfer holds, the same training-and-policy-set workflow could replace offline weighted-sum planning in field UAV deployments, but a field trial is what would establish it.","The MOMDP and task-selection machinery are built for a single mobile user; an extension the paper leaves implicit is several users, where the state and reward vectors would grow and the UVAA would need to split its mainlobe or time-share users.","A testable prediction follows from the Gauss–Markov assumption: the performance gap between EMOPPO-VLH and a memoryless version should widen as the memory level $\\alpha_g$ increases, which would isolate how much of the gain comes from the LSTM.","The same evolutionary multi-objective RL structure could transfer to other aerial trade-offs, such as sensing-versus-communication or coverage-versus-jamming, because the vectorized-value-function and population-evolution design is not beamforming-specific."],"forward_implications":["Operators get a Pareto set of control policies from one training run, so the rate-versus-energy preference can be changed at deployment time without retraining.","The same formulation covers users whose motion has memory; the LSTM layer is what lets the policy exploit the Gauss–Markov correlation in user movement.","The approach scales from a small 8-UAV swarm to a large 16-UAV swarm while keeping the best IGD and hypervolume values among the compared baselines.","Because actions are expressed as directions, distances, and excitation current weights, the trained policies map directly to UAV control commands and can run on a central controller using only position information."],"supporting_citations":[{"why":"Supplies the closed-form 3D flight-energy model $E_{fly}$ used in the energy objective and frames the UAV communication-energy constraints.","marker":"[4]"},{"why":"Establishes collaborative beamforming as the technique that lets many UAVs act as one virtual antenna array.","marker":"[5]"},{"why":"Provides the collaborative-beamforming relay model and the array-gain expression used to compute $G_{UM}[t]$ and the achievable rate.","marker":"[11]"},{"why":"Supplies the drone-based antenna-array model and the antenna efficiency parameter $\\eta$ used in the gain equation.","marker":"[19]"},{"why":"Establishes the authors' earlier evolutionary multi-objective DRL design that this paper adapts from LEO satellites to UAV swarms with mobile users.","marker":"[29]"},{"why":"Gives the rotary-wing propulsion power model $P(v_i)$ used in the UAV energy consumption objective.","marker":"[35]"},{"why":"Introduces the Gauss–Markov random-walk mobility model used to simulate the mobile user's memory-dependent motion.","marker":"[36]"},{"why":"Defines PPO, the base reinforcement-learning algorithm that EMOPPO-VLH extends with a vectorized value function.","marker":"[51]"},{"why":"Provides the MOEA/D baseline whose IGD and hypervolume are compared against EMOPPO-VLH.","marker":"[64]"},{"why":"Provides the MOPSO baseline whose IGD and hypervolume are compared against EMOPPO-VLH.","marker":"[65]"}],"fun_headline_variants":["UAV swarm becomes one antenna for mobile users via RL","Swarm beamforming RL serves moving users with Pareto-optimal tradeoffs","Evolutionary RL tunes UAV swarm as aerial antenna for mobile users","Multi-objective RL gives UAV swarm a rate-vs-energy menu for user tracking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the mathematical simulation — Rician fading, Gauss–Markov user movement, the UAV propulsion model, and the array-factor equations — is faithful enough that a policy trained in it transfers to real hardware; if it does not, the claimed advantages are unverified.","fun_headline_variants_meta":{"raw":{"variants":["UAV swarm becomes one antenna for mobile users via RL","Swarm beamforming RL serves moving users with Pareto-optimal tradeoffs","Evolutionary RL tunes UAV swarm as aerial antenna for mobile users","Multi-objective RL gives UAV swarm a rate-vs-energy menu for user tracking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000653,"raw_usage":{"total_tokens":3043,"prompt_tokens":1044,"completion_tokens":1999,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":1935}},"tokens_in":660,"tokens_out":1999,"duration_ms":13553,"temperature":1.0,"reasoning_tokens":1935,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:50:02.195398+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a field trial with 8 rotary-wing UAVs and a GPS-tracked mobile user under the paper's Rician-and-interference parameters, then compare measured total achievable rate and total energy consumption to the simulated Pareto front; if the realized points are dominated by a baseline policy or lie far outside the front, the central claim fails.","supporting_citations":[{"cited_title":"Accessing from the sky: A tutorial on UAV communications for 5g and beyond,","cited_arxiv_id":null,"evidence_quote":"Supplies the closed-form 3D flight-energy model $E_{fly}$ used in the energy objective and frames the UAV communication-energy constraints."},{"cited_title":"Distributed and collaborative beamforming in wireless sensor networks: Clas- sifications, trends, and research directions,","cited_arxiv_id":null,"evidence_quote":"Establishes collaborative beamforming as the technique that lets many UAVs act as one virtual antenna array."},{"cited_title":"Secure and energy-efficient UAV relay communications exploiting collabora- tive beamforming,","cited_arxiv_id":null,"evidence_quote":"Provides the collaborative-beamforming relay model and the array-gain expression used to compute $G_{UM}[t]$ and the achievable rate."},{"cited_title":"Communi- cations and control for wireless drone-based antenna array,","cited_arxiv_id":null,"evidence_quote":"Supplies the drone-based antenna-array model and the antenna efficiency parameter $\\eta$ used in the gain equation."},{"cited_title":"Energy minimization for wireless communication with rotary-wing UAV,","cited_arxiv_id":null,"evidence_quote":"Gives the rotary-wing propulsion power model $P(v_i)$ used in the UAV energy consumption objective."},{"cited_title":"Fundamentals of mobility-aware performance characterization of cellular networks: A tutorial,","cited_arxiv_id":null,"evidence_quote":"Introduces the Gauss–Markov random-walk mobility model used to simulate the mobile user's memory-dependent motion."},{"cited_title":"MOEA/D: A multiobjective evolutionary algorithm based on decomposition,","cited_arxiv_id":null,"evidence_quote":"Provides the MOEA/D baseline whose IGD and hypervolume are compared against EMOPPO-VLH."},{"cited_title":"MOPSO: a proposal for multiple objective particle swarm optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the MOPSO baseline whose IGD and hypervolume are compared against EMOPPO-VLH."}],"review_version":1}