{"id":"b5aa5968-bfd6-403d-bef4-6d2d59854603","arxiv_id":"2501.02787","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"An enhanced PPO algorithm that jointly plans a UAV's trajectory and an IRS's phase shifts is shown in simulation to outperform DDPG, TD3, SAC, and PPO for urban low-altitude mmWave links.","lead":"This paper simulates a drone carrying a smart reflector (an intelligent reflecting surface) to bounce millimeter-wave signals around city buildings to mobile users, and trains a reinforcement learning agent to steer the drone and set the reflector for higher data rates and lower energy use. It reports that its enhanced PPO algorithm beats four standard reinforcement learning baselines on its own simulated urban scenario.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The rate expression in Eq. (5) is invalid as written: it uses the complex inner product rather than its squared modulus and treats dB path loss as a linear gain, so every simulated rate and reward is uninterpretable.","rationale":"The reader's verdict is REJECT, and I agree with that outcome. The reader's weakest_assumption focuses on the unproven optimality of the IRS phase-shift formula in Eq. (15), which is a legitimate concern about comparison fairness because EPPO's action space is reduced by a closed-form phase controller while benchmarks must learn the phases. However, I find an even more load-bearing problem one step earlier: Eq. (5), the rate expression on which the objective, reward, and all simulations depend, is invalid as reproduced. The missing squared modulus makes the log argument non-real or non-positive for permissible phase choices, and the dB-valued path loss is used as a linear channel gain, corrupting the channel magnitudes. Either issue by itself would invalidate the numerical evidence for the central claim; together they make the simulations uninterpretable without a correction. The concrete test I propose would settle the matter directly by evaluating Eq. (5) with the paper's own parameter set. If the source turns out to contain the proper |g^T Θ h|^2 and a linear path-loss conversion, my concern would be withdrawn, and the phase-fairness issue would remain the primary obstacle. Since the reader already rejected the paper and my concern supports that rejection, the verdict should remain unchanged. I marked agreement as partial because the reader's rationale mentions the invalid rate expression but their weakest_assumption highlights a different aspect.","tokens_in":30045,"tokens_out":10311,"duration_ms":105791,"concrete_test":"Evaluate Eq. (5) exactly as written using the simulation parameters in Table 3 with one reflection element (M=1), d_SI=100 m, d_IE=50 m, P=15 W, B=2 MHz, σ^2=-174 dBm/Hz, and an arbitrary phase shift. Compute Ψ^SI_t=30+22 log10(100)=74 and Ψ^IE_{u,t}=30+22 log10(50)≈67.4, then form g^T Θ h. Show that choosing the reflection phase to make g^T Θ h real and negative yields a log argument ≤0, so R_{u,t} is undefined; this proves the formula lacks the required |·|^2. If the source LaTeX instead contains |g^T Θ h|^2, recompute Fig. 5(b) with the corrected expression to see whether the reported rates change by more than a few percent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that EPPO improves transmission rate and energy efficiency, supported entirely by simulations in Figs. 5, 6, 8, 11, and 12. Those simulations evaluate rewards and rates using Eq. (5), reproduced as R_{u,t} = B log2(1 + P g_t^T Θ_t h_{u,t}/(B σ^2)). As written, this is not a valid achievable rate. Standard communication theory requires the squared modulus of the composite channel: SNR = P |g_t^T Θ_t h_{u,t}|^2 / (B σ^2). Without the |·|^2, the log argument is a complex quantity, and for phase choices that make the inner product negative real, the argument can be zero or negative, making log2 undefined. Additionally, Eqs. (1)-(2) define the path-loss factors Ψ^SI_t and Ψ^IE_{u,t} in dB (β(D)=30 dB, n=2.2), yet these dB values are multiplied directly into the channel vectors as if they were linear gains. For example, at d_SI=100 m, Ψ^SI_t=74 dB, so the 'channel gain' would be 74, not 10^{-74/20}≈2×10^{-4}. Either error alone would invalidate every computed rate, reward, and fairness metric, and therefore the empirical support for the headline claim. Even if these expressions are corrected, the reader's concern about the unproven optimality of the phase-shift formula in Eq. (15) would remain a separate fairness issue for the benchmark comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper considers a UAV-carried intelligent reflecting surface (IRS) assisting an urban low-altitude mmWave communication link between a source user and mobile users whose direct path is blocked. It formulates a joint optimization problem over the IRS phase shifts and the UAV trajectory, with an objective that combines the sum rate, Jain's fairness index, and UAV propulsion energy consumption. To solve this non-convex, dynamic problem, the authors propose an enhanced PPO algorithm (EPPO) that integrates neural episodic control with state abstraction, a Mogrifier LSTM actor network, and a closed-form IRS phase-shift control strategy said to reduce the action space to the three UAV displacement components. The empirical evaluation compares EPPO with DDPG, TD3, SAC, and PPO in single-user and multi-user simulations and reports higher cumulative rewards, higher average rates, and lower energy consumption for EPPO.","tokens_in":30454,"tokens_out":2538,"duration_ms":26553,"significance":"If the results were valid, the paper would offer a practical DRL pipeline for UAV-carried IRS deployments, with useful engineering heuristics such as the phase-shift control strategy and the episodic control acceleration. The authors also provide a reasonably thorough ablation study (Fig. 6) isolating the contributions of the Mogrifier LSTM, NECSA, and the phase-shift control, and they report a complexity comparison in Table 2. However, the central quantitative claim rests on a rate expression that is missing the squared modulus of the composite channel and that treats dB path loss as a linear gain, so every numerical rate, reward, and energy-efficiency comparison in the paper is uninterpretable as presented. The asserted optimality of the closed-form phase-shift formula is also unproven for the Rician channel with NLoS components, which makes the benchmark comparison potentially unfair regardless of the rate-model error. The significance of the work therefore cannot be assessed until these load-bearing issues are resolved.","major_comments":[{"comment":"The achievable rate expression in Eq. (5) is invalid as written. The standard SNR term is P |g_t^T Θ_t h_{u,t}|^2 / (B σ^2), but the paper uses P g_t^T Θ_t h_{u,t} / (B σ^2) without the squared magnitude. The inner product is a complex quantity; for phase configurations that make it negative real, the log2 argument becomes non-positive and the rate is undefined. Moreover, Eqs. (1)–(2) define the path-loss factors Ψ^SI_t and Ψ^IE_{u,t} in dB (β(D) = 30 dB, exponent n = 2.2), yet these dB values are multiplied directly into the channel vectors as linear gains. At a distance of 100 m, Ψ = 74 dB is treated as a gain of 74 instead of approximately 2×10^{-4} (linear). Either error alone invalidates every simulated rate and reward; together they completely undermine the empirical support for the headline claim in Figs. 5, 6, 8, 11, and 12.","section":"§3.2, Eq. (5)"},{"comment":"The IRS phase-shift control strategy is called 'optimal' in the text, but Eq. (15) is asserted rather than derived, and it aligns only the LoS geometric phases. The channel model in Eqs. (1)–(2) explicitly includes Rician NLoS components with independent Gaussian phases, whose contribution is absent from Eq. (15). Without a proof that this closed-form phase vector maximizes the achievable rate (or a statement of the conditions under which it is optimal), the claimed 'optimality' is unsupported. This matters for the evaluation: EPPO uses Eq. (15) to remove the IRS phase shifts from the learned action space, while the benchmark algorithms (DDPG, TD3, SAC, PPO) must learn these phases. If the closed-form phase is only a heuristic, the reward advantage in Figs. 5 and 8 may simply reflect that EPPO is given a better-informed action space, not a superior policy.","section":"§4.3.3, Eq. (15)"},{"comment":"The optimization objective and the reward function both inherit the invalid rate expression from Eq. (5). In particular, Eq. (10) sets r_t = ξ R_{u,t}/E_t for LoS links and r_t = 0 for NLoS links; since R_{u,t} is not a proper rate, the reward does not correspond to any well-defined communication quantity. Consequently, the cumulative-reward curves, convergence comparisons, and the conclusions about 'higher transmission rates and lower energy consumption' are not grounded in a valid system model. The authors should re-derive the rate with |g_t^T Θ_t h_{u,t}|^2 and consistent linear path-loss gains, then rerun the simulations before the comparative claims can be evaluated.","section":"§4.1.3 and §3.4, reward/objective"}],"minor_comments":[{"comment":"There are numerous typographical errors that should be corrected: 'alforithm' (§4.2), 'the the' (§4.2), 'Muti-user' (Fig. 8 caption), 'Enhanced PPO' in the title of §4.3, and inconsistent notation between cSU and cUAV in §3.2 and §4.1.1.","section":"Global"},{"comment":"The captions contain font-encoding artifacts that make them unreadable in the provided manuscript; these should be regenerated with the correct text.","section":"Fig. 6 and Fig. 8 captions"},{"comment":"The LoS channel expressions omit the normalization by the array size; the authors should clarify whether the steering vectors are normalized, since this affects the scaling in Eq. (5).","section":"§3.2, Eq. (3)–(4)"},{"comment":"The paper cites [20] for the phase-shift formula, but [20] considers a different system (IRS-assisted UAV OFDMA) and does not establish optimality for the Rician channel used here; the authors should provide a self-contained derivation or clearly state the heuristic nature of Eq. (15).","section":"§4.3.3"}],"recommendation":"reject","confidential_remarks":"This manuscript is a journal extension of a previously published ICC 2024 paper, and the overlap is acknowledged. The main technical problem is not a matter of presentation: the rate expression in Eq. (5) is mathematically invalid, and the simulation results are therefore not interpretable. Even if the authors fix the rate model, the unproven optimality of Eq. (15) would still need to be addressed to ensure a fair benchmark comparison. Given that these are load-bearing errors in the central claim, I recommend rejection in its current form. The authors may, of course, resubmit a substantially revised manuscript with a corrected channel model and re-run experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper combines UAV-carried IRS, PPO, NECSA, Mogrifier LSTM, and a closed-form IRS phase rule for urban mmWave obstacle environments. That specific package is new, and the simulation effort is extensive: single- and multi-user scenarios, ablations, complexity analysis, and parameter sweeps. The related work is thorough and the writing is clear.\n\nThe problem is the rate equation. Eq. (5) is B log2(1 + P g^T Θ h / (B σ^2)) — the squared modulus of the composite channel is missing. The SNR argument is a complex number, and can go negative, making the log undefined. This alone invalidates every rate, reward, and fairness number in the paper. Additionally, Eqs. (1)-(2) define path loss in dB and then multiply that dB value directly into the channel vector, effectively using 74 dB as a linear gain. So the 'channel gains' are off by orders of magnitude. The stress-test note is correct, and the reader's take lands.\n\nA separate, softer issue: Eq. (15) calls the phase rule 'optimal' but it only aligns LoS phases and ignores the Rician NLoS components in the channel. That gives EPPO an action-space advantage over benchmarks that must learn phases, so the comparison is not fair. The rule might still be a good heuristic, but it needs a derivation or a caveat.\n\nThe core errors are fixable: correct Eq. (5), convert dB to linear properly, rerun with multiple seeds and report variance, and either prove or relabel Eq. (15). The system model and algorithmic framing are otherwise sensible. But as it stands the empirical claims are unsupported.\n\nI would not cite this in its current form. If it comes to you as an editor, I'd send it to review so the issues get formally flagged, but the expected outcome is rejection unless the math is fixed and the simulations redone.","headline":"A plausible DRL package for UAV-mounted IRS, undone by a rate formula missing the squared modulus and a dB/linear path-loss mixup that makes every simulation number uninterpretable.","tokens_in":30950,"tokens_out":4096,"would_cite":false,"duration_ms":39878,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that its EPPO algorithm jointly optimizes UAV trajectory and IRS phase shifts to deliver higher urban mmWave rates and lower UAV energy use than DDPG, TD3, SAC, and PPO in simulation.","keywords":["UAV-carried IRS","joint trajectory and phase shift optimization","deep reinforcement learning","proximal policy optimization","mmWave communications","Jain's fairness index","Rician fading","low altitude economy"],"falsifier":"Numerically maximize the achievable rate over IRS phase shifts for the full Rician channel model at a fixed UAV location with non-zero $k$ and the Gaussian NLoS terms, then compare that optimum to Eq. (15). If the geometric formula falls short, EPPO's advantage comes from a shortcut the baselines are not given, and the comparison would need to be redone with the same phase law applied to all algorithms.","tokens_in":29893,"feed_emoji":"📡","tokens_out":8986,"duration_ms":74628,"temperature":0.7,"pith_summary":"This paper tries to establish that a UAV carrying an intelligent reflecting surface (IRS) can restore millimeter-wave connectivity in dense urban low-altitude environments where buildings block the direct path between a source user and mobile users. The authors formulate a joint optimization of the UAV trajectory and the IRS phase shifts, with an objective that balances transmission rate, UAV energy consumption, and fairness across users, and they solve it with an enhanced proximal policy optimization (EPPO) algorithm. EPPO combines neural episodic control with state abstraction, a Mogrifier LSTM actor, and a closed-form IRS phase shift computation that removes the phase variables from the action space. The simulation evidence shows EPPO reaching higher cumulative rewards, higher average rates, and lower energy consumption than DDPG, TD3, SAC, and PPO in both single-user and multi-user urban settings.","feed_headline":"Learned flight path lets a drone IRS beat RL baselines in mmWave tests","feed_subtitle":"Paper argues joint optimization of trajectory and phase shifts lifts data rates and cuts energy; the geometric phase law carries the gain.","key_machinery":"The key machinery is the EPPO algorithm, built on three mechanisms: neural episodic control with state abstraction (NECSA), which griddizes the continuous state space and provides intrinsic rewards to speed convergence; a Mogrifier LSTM that augments the actor network to learn long-range dependencies between states and actions; and an IRS phase shift control strategy that evaluates Eq. (15) for each reflecting element, a closed-form phase based only on the geometric angles, reducing the action space from $M_rM_c+3$ to 3. The optimization objective is the fairness rate energy consumption ratio $F_t=(\\sum_{i=1}^n \\xi R_{i,t})/E_t$, which combines Jain's fairness index, the achievable rate, and the UAV propulsion energy.","core_discovery":"The central claim is that joint optimization of trajectory and phase shifts is tractable in real time with the proposed EPPO algorithm, despite the problem being non-convex and NP-hard in its phase-shift subproblem. EPPO treats the UAV displacement as the only learned action, because the IRS phase shifts are computed in closed form from the azimuth and elevation angles of arrival and departure, shrinking the action space from $M_rM_c+3$ to 3. The reward is a fairness-rate-energy ratio, so the policy is driven to positions that serve all users equitably while keeping propulsion energy low. The paper reports that this design yields higher average data rates and lower cumulative energy consumption than DDPG, TD3, SAC, and PPO in urban simulations, and that the learned trajectories tend to hover at energy-efficient positions.","pith_inferences":["A testable extension is to replace the closed-form phase law with a learned phase policy in a fair comparison, keeping all other EPPO components; if the closed-form law is truly optimal under the Rician model, the learned phase policy should converge to the same rates but take longer, and if not, the reported EPPO advantage would shrink.","The paper's comparison with DDPG, TD3, and SAC does not give those baselines the same phase-control shortcut; a fairer benchmark would impose the geometric phase law on all algorithms, isolating the contribution of trajectory learning.","The phase law itself assumes the LoS geometric phases dominate the channel; under strong NLoS or scattering, an adaptive phase policy that accounts for the random Gaussian components could outperform the closed-form formula, which would weaken the paper's claim that geometric alignment is optimal.","The closed-form phase makes the IRS beamforming a deterministic function of geometry, so the UAV trajectory and IRS beamforming are effectively optimized by a single policy; this coupling suggests the method could be extended to multiple IRS elements or inter-cell interference scenarios by augmenting the state space."],"forward_implications":["EPPO's phase law makes the IRS beamforming independent of the learning process, so the drone policy only has to learn three continuous actions per time slot regardless of the number of reflecting elements.","The fairness-weighted reward means the learned trajectory consciously balances per-user rates, not just the sum rate, in multi-user urban grids.","The learned trajectories in simulation converge to low-energy hovering and cruise patterns near the minimum-energy speed, indicating the policy internalizes the propulsion model.","Because phase shifts are computed from the current coordinates of source, IRS, and user, the algorithm can react to user mobility at each time slot without re-optimizing beamforming."],"supporting_citations":[{"why":"Basis for the IRS channel model (Rician fading) and for the NP-hardness reduction of the phase-shift subproblem.","marker":"[8]"},{"why":"Supplies the closed-form IRS phase shift expression (Eq. 15) that reduces the action space.","marker":"[20]"},{"why":"Provides the UAV propulsion energy model and a DRL design for IRS-aided UAV communications used as a benchmark.","marker":"[27]"},{"why":"The base proximal policy optimization algorithm that EPPO enhances.","marker":"[41]"},{"why":"Introduces neural episodic control with state abstraction, used to accelerate learning.","marker":"[42]"},{"why":"Introduces the Mogrifier LSTM variant used in the actor network.","marker":"[43]"},{"why":"Baseline DDPG algorithm compared in the simulations.","marker":"[47]"},{"why":"Baseline TD3 algorithm compared in the simulations.","marker":"[48]"},{"why":"Baseline SAC algorithm compared in the simulations.","marker":"[49]"}],"fun_headline_variants":["Drone IRS learns to hover and reflect, outdoing RL rivals in mmWave","Closed-form phase law lets UAV IRS beat RL baselines in urban mmWave","EPPO shrinks action space to 3, wins on rates and energy for drone IRS","Joint path and phase optimization: drone IRS tops RL in mmWave tests","UAV-carried IRS: one learned action, closed-form phases, better than RL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The closed-form phase shift law assumes that aligning the geometric line-of-sight phases between source, IRS, and user maximizes the data rate, even though the channel model also contains random scattered (non-line-of-sight) components with random phases.","fun_headline_variants_meta":{"raw":{"variants":["Drone IRS learns to hover and reflect, outdoing RL rivals in mmWave","Closed-form phase law lets UAV IRS beat RL baselines in urban mmWave","EPPO shrinks action space to 3, wins on rates and energy for drone IRS","Joint path and phase optimization: drone IRS tops RL in mmWave tests","UAV-carried IRS: one learned action, closed-form phases, better than RL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1416,"prompt_tokens":987,"completion_tokens":429,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":323}},"tokens_in":603,"tokens_out":429,"duration_ms":4539,"temperature":1.0,"reasoning_tokens":323,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:04:21.861267+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Numerically maximize the achievable rate over IRS phase shifts for the full Rician channel model at a fixed UAV location with non-zero $k$ and the Gaussian NLoS terms, then compare that optimum to Eq. (15). If the geometric formula falls short, EPPO's advantage comes from a shortcut the baselines are not given, and the comparison would need to be redone with the same phase law applied to all algorithms.","supporting_citations":[{"cited_title":"Intelligent reflecting surface- aided wireless communications: A tutorial,","cited_arxiv_id":null,"evidence_quote":"Basis for the IRS channel model (Rician fading) and for the NP-hardness reduction of the phase-shift subproblem."},{"cited_title":"Sum-rate maximization for IRS- assisted UAV OFDMA communication systems,","cited_arxiv_id":null,"evidence_quote":"Supplies the closed-form IRS phase shift expression (Eq. 15) that reduces the action space."},{"cited_title":"Joint trajectory and passive beamforming design for intelligent reflecting surface-aided UAV communications: A deep reinforcement learning approach,","cited_arxiv_id":null,"evidence_quote":"Provides the UAV propulsion energy model and a DRL design for IRS-aided UAV communications used as a benchmark."},{"cited_title":"Neural episodic control with state abstraction,","cited_arxiv_id":null,"evidence_quote":"Introduces neural episodic control with state abstraction, used to accelerate learning."},{"cited_title":"Mogrifier LSTM,","cited_arxiv_id":null,"evidence_quote":"Introduces the Mogrifier LSTM variant used in the actor network."},{"cited_title":"Continuous control with deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Baseline DDPG algorithm compared in the simulations."},{"cited_title":"Addressing function ap- proximation error in actor-critic methods,","cited_arxiv_id":null,"evidence_quote":"Baseline TD3 algorithm compared in the simulations."},{"cited_title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,","cited_arxiv_id":null,"evidence_quote":"Baseline SAC algorithm compared in the simulations."}],"review_version":1}