Pith. sign in

REVIEW 4 major objections 4 minor 18 references

Reflections in the Sky: Millimeter Wave Communication with UAV-Carried Intelligent Reflectors

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read A UAV-carried intelligent reflector, positioned by deep Q-learning, can hold a millimeter-wave downlink above 90% line of sight in a simulated urban blockage scenario.

desk verdict Nice architectural idea and a correct phase-alignment proposition, but the RL algorithm in the pseudocode does not match the paper's own Bellman update, so the headline RL gains are not supported as written. read the letter →

arxiv 1908.03271 v2 pith:PXGMPNTU submitted 2019-08-08 cs.IT cs.NIeess.SPmath.IT

classification cs.ITcs.NIeess.SPmath.IT
keywords millimeterwavecommunicationsintelligentreflectingsurfaceunmannedaerialvehiclereinforcementlearningQ-learningLSTMRFenergyharvestingline-of-sightprobability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that a drone carrying an intelligent reflecting surface can keep a millimeter-wave (mmW) downlink working for a moving outdoor user even when the direct path is blocked by buildings, trees, or the user's own body. The central claim is that a reinforcement-learning agent, using Q-learning with a neural-network value function, can learn where to position the drone and how to set the reflector's phases so that the two reflected hops stay in line of sight, and that the reflector can harvest enough radio-frequency energy from the same signal to power itself. In the paper's urban simulation, the learned drone reflector holds the line-of-sight probability above 90% and raises the average data rate well above both a static rooftop reflector, which stays in line of sight under 5% of the time, and a drone reflector that moves without learning, which stays in line of sight above 70% of the time. The importance, if the claim holds, is that a small mobile reflector with no extra transmit power could rescue mmW links that otherwise fail at the first obstacle, and could do so without drawing energy from the drone.

What carries the argument

The central object is the intelligent reflector on the UAV, a passive array whose reflection coefficient $\Theta = \mathrm{diag}(a e^{j\theta_1},\dots,a e^{j\theta_N})$ phase-aligns the incident beamforming signal before sending it toward the user; the paper proves that aligning each phase to $-\operatorname{Arg}(h_n r_n)$ maximizes the downlink capacity for a known channel. The learning machinery is a deep Q-network built on LSTM cells, with the value function $Q_t(h,x,y,\omega)$ storing the expected discounted reward from deploying the UAV at location $x$; the agent updates this value function via Q-learning with reward $r(t)=c(t)\Delta T$ for each hovering slot, and chooses the next deployment by maximizing $Q$. The user's movement is predicted through a Gaussian model $f(y,\omega|Y_t)=\mathcal{N}(\mu,\Lambda|Y_t)$, which turns the nonstationary channel into a Markov decision process. The energy sub-system harvests $p_e(\Theta)=\kappa\|(I-\Theta)r\|^2$ from the portion of the incident signal the reflector does not send onward, and the paper uses this power, together with the hovering and mobility cost model, to argue the reflector can self-power.

What would settle it

Re-run the authors' simulation, or a small-scale drone test, with a user trajectory that includes sudden turns and long straight stretches, so that the Gaussian prediction model is visibly wrong, and check whether the learned UAV-IR keeps the LOS probability above 90% and still beats the no-learning drone. If the LOS probability drops toward the non-learning level, the result depends on the motion model rather than on the learning mechanism itself.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is a blueprint for turning a blocked non-line-of-sight mmW path into two connected line-of-sight paths using a reflector carried by a UAV. The reflector is modeled as a diagonal phase-shift matrix $\Theta = \mathrm{diag}(a e^{j\theta_1},\dots,a e^{j\theta_N})$, and the paper proves that for known channel state $h$ the capacity-maximizing phase on element $n$ is $\theta_n^* = -\operatorname{Arg}(h_n r_n)$, where $r_n$ is the incident signal component. Around that phase law, an LSTM-based Q-network learns a value function $Q_t(h,x,y,\omega)$ giving the expected cumulative downlink data a UAV at location $x$ can deliver to a user at $(y,\omega)$, with future user positions predicted by a Gaussian model $f(y,\omega|Y_t)=\mathcal{N}(\mu,\Lambda|Y_t)$. In simulation, this learned deployment achieves a downlink LOS probability above 90% and average data rates higher than a static IR (below 5% LOS) and a non-learning UAV-IR (above 70% LOS), while the RF-energy-harvesting model $p_e(\Theta)=\kappa\|(I-\Theta)r\|^2$ produces around 1 mW, enough to power the reflector.

Load-bearing premise

The load-bearing premise is that the learning network converges to a good-enough positioning policy and that the user's future positions follow the Gaussian model used in the simulations; if real motion deviates from that model or the network fails, the claimed line-of-sight and data-rate advantages may not persist outside the simulator.

Editorial extensions

If this is right

  • If the RL deployment works outside the simulator, an operator could dispatch a UAV-IR to a blocked user on demand and restore a usable mmW link without building new infrastructure.
  • A LOS probability above 90%, versus under 5% for a fixed rooftop reflector, means mobility is the decisive factor and that a learning-based positioning policy may be necessary to get the full gain.
  • At transmit powers above the SNR threshold, the UAV-reflected rate grows faster than the static case, so the scheme is most valuable in moderate-to-high-power mmW cells.
  • Because harvested RF power scales with the transmitted signal, the same mechanism can power the reflector during active service, avoiding reliance on the drone's onboard battery.
  • Performance degrades as the drone's altitude increases, so the deployment must balance LOS likelihood against path loss; the paper's constant-altitude simulation leaves altitude as a tunable system parameter.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the Gaussian-motion assumption is likely the first place to stress-test: real pedestrians and vehicles make abrupt turns, so a practical system would need online model updates or a richer motion prior, and the 90%-LOS result would probably weaken without them.
  • The paper's simulation compares one learned drone against one static reflector and one non-learning drone; a natural extension is multiple UAV-IRs coordinating to serve several users on the same frequency, where the reflection phases must be co-designed rather than set independently.
  • If the Q-network generalizes across urban layouts, the same reward structure could be trained in one simulated city and transferred to another, turning deployment into a policy lookup rather than per-environment training.
  • The gap between the non-learning drone (above 70% LOS) and the learned drone (above 90% LOS) suggests a large part of the gain comes from anticipating user motion; measuring that gap under different user speeds would isolate the value of prediction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes to mount an intelligent reflector (IR) on a UAV (UAV-IR) to serve a mobile outdoor user in a millimeter-wave downlink, reflecting a beam from the BS to the UE while harvesting RF energy to power the IR. The core analytic result (Proposition 1) gives the optimal reflection phases for fixed CSI. To maintain a LOS link as the UE moves, the paper introduces a deep Q-learning scheme with an LSTM network that selects the UAV-IR's deployment location; the location and reflection coefficient are chosen to maximize long-term discounted downlink capacity. The problem is formulated in Section II, the RL method in Section III, and simulations in Section IV compare a static IR, a non-learning UAV-IR, and the proposed learning-based UAV-IR in terms of average data rate and LOS probability.

Significance. The paper addresses a timely problem and the system concept of a mobile, self-powered IRS/UAV-IR is interesting. Proposition 1 is correct and standard, and the energy-harvesting model is clearly set out. The claimed empirical gains (LOS probability >90% and higher rates for the RL-based UAV-IR) are potentially important. However, the RL algorithm as written is inconsistent with its own Bellman equation (Algorithm 1 vs. Eq. (10)), the network and action-selection details are missing, and the simulation results lack confidence intervals. These issues prevent the reader from verifying the central claim; if the algorithm were corrected and the details provided, the contribution could be of interest.

major comments (4)
  1. [III.B, Eq. (10), Algorithm 1] Algorithm 1 does not implement the Q-learning update defined by Eq. (10). In Eq. (10), the bootstrap target is r(t+K)+γ max_x Q_t(h(t+K),x,y(t+K),ω(t+K)), i.e., the value of the next state reached after the UAV moves. In Algorithm 1, step 4 instead computes q(t)=r(t)+γ max_x \tilde Q(h(t),x,y(t),ω(t)|\phi(t)) when a move is made, using the same h(t), y(t), ω(t) as the state at time t, and step 5 fits \tilde Q(h(t),\tilde x,y(t),ω(t)|\phi) to that target. Consequently, the trained network is not an estimator of the value function in Eq. (8), and the cited convergence argument after Eq. (10) does not apply. The LOS/data-rate improvements in Figs. 3–4 are therefore not supported by the learning rule as written; either Algorithm 1 should be corrected to use the next state and r(t+K), or the formulation in Eq. (10) must be revised to match the implemented update.
  2. [III.B, after Eq. (10)] The paper asserts that Q-learning 'can converge to a local optimal value function' if T is long enough and the UE movement model is properly modeled, citing [13]. This is not a proof, and standard convergence results for Q-learning do not apply to nonstationary environments with function approximation. The Gaussian movement model f(y,ω|Y_t)=N(µ,Λ|Y_t) is introduced without empirical justification, and no sensitivity analysis is provided for deviations from this model. Since the simulated UE motion in Section IV is described as an MDP with a random destination, it is not evident that the assumed Gaussian model matches the simulation environment; the claimed performance gains may depend on a model that the algorithm itself assumes.
  3. [III.B, Eqs. (8)-(9), Algorithm 1] The action space in Eq. (9) is continuous (x∈R^3, or R^2 at constant altitude), but the paper does not specify how the arg max over x is computed during training or inference. The description of the function approximator is also limited to 'a deep neural network, based on the LSTM architecture' after Eq. (10); no layer sizes, input/output representation, optimizer, replay buffer, or exploration scheme are given. These omissions make the proposed deep RL algorithm irreproducible and leave open the possibility that the reported results rely on an unspecified discretization or an ad hoc maximization procedure.
  4. [IV, Figs. 3–4] The simulation results are presented without error bars, confidence intervals, or the number of independent runs, and the learning rate β is not reported. Since deep RL training is stochastic and the performance differences between RL and no-RL cases (LOS probability 90% vs. 70%) appear modest, the reader cannot assess whether these differences are statistically significant. Without this information, the central claim of an advantage from RL-based deployment is not fully supported by the evidence shown.
minor comments (4)
  1. [II.A, Proposition 1 proof] The proof of Proposition 1 states 'Without loss of generality, let h_n r_n e^{jθ_n}=0' for all n, which is a trivial case, not the condition for equality. The correct equality condition is that the complex numbers h_n r_n e^{jθ_n} share a common argument; the resulting optimal phase θ*_n=-Arg(h_n r_n) is correct, but the proof's wording should be fixed.
  2. [II.B, Eq. (4)] The notation '1_{v_r=0}' is missing its subscript formatting; the indicator function should be typeset clearly, for example as \mathbb{1}_{\{v_r=0\}}.
  3. [IV, first paragraph] The sentence 'The planar antenna array with 4×4 IR components and 8×8 antennas are equipped on the BS and the UAV-IR, respectively' is ambiguous; clarify that the BS has 64 antennas and the IR has 16 reconfigurable components.
  4. [IV, Fig. 4] The figure label 'RF energy harvesting power' should specify units (e.g., mW or dBm) for clarity, and the ordinate should be explicitly identified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Proposition 1 is proved directly and the RL claims rest on standard, externally cited Q-learning.

full rationale

I examined the claimed derivation chain. Proposition 1 is a direct phase-alignment proof using the triangle inequality on |hΘr|; it does not depend on the paper's own conclusions or on any fitted quantity. Equation (10) is the standard Q-learning Bellman backup, cited to the external textbook by Sutton and Barto [13], and the convergence statement is explicitly conditional on a long service time and proper movement modeling. The Gaussian movement model f(y,ω|Yt)=N(μ,Λ|Yt) and the UMi-Street Canyon path loss are stated as modeling assumptions, not derived from the target LOS/rate results. The self-citations ([1], [2], [10]-[12], [16]) are background on 6G, UAV communications, and neural-network tutorials; none is load-bearing for the core RL deployment claim, and no uniqueness theorem from the authors' prior work is invoked to forbid alternatives. The simulation results are generated and evaluated in the same simulated environment, so there is no independent empirical benchmark, but that is a validation limitation rather than circularity: no fitted parameter is renamed as a prediction, and no equation reduces to its own input by construction. The discrepancy between Algorithm 1's target q(t)=r(t)+γ max_x Q~(h(t),x,y(t),ω(t)|φ) and the Bellman backup in Eq. (10) is a correctness/implementation inconsistency, not a circular definition. Therefore no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard channel and antenna assumptions plus two model-specific choices: a Gaussian movement prediction model and an unproven convergence assumption for deep Q-learning in a nonstationary environment. The hand-chosen hyperparameters gamma, beta, a, and kappa affect the quantitative results but are not fitted to data, so they are listed as free parameters rather than evidence of circular fitting.

free parameters (4)
  • Discount factor gamma = 0.1
    Chosen by hand for the Q-learning value function in Eqs. (8)-(10); no sensitivity analysis is provided, and a low gamma undervalues future rewards, which directly affects the learned deployment policy and the reported RL gains.
  • Learning rate beta = unspecified in simulations
    Q-learning update in Eq. (10) uses beta in (0,1] but the paper never states the value used in training, making the learning dynamics non-reproducible.
  • Reflection amplitude a = 0.8
    Sets the amplitude reflection coefficient of each IR element in Theta = diag(a e^{j theta_n}); this is a design choice that affects SNR and the energy harvesting trade-off (Section II.A).
  • Energy conversion efficiency kappa = 60%
    Average RF-to-electrical conversion efficiency in Eq. (3), chosen for the energy harvesting claim; the conclusion that harvested power is sufficient for self-powering depends on this value.
assumptions (4)
  • domain assumption The mmW channel follows the UMi-Street Canyon path loss model and the massive MIMO channel model of [18].
    Used in all simulations; if the real channel deviates, the quantified gains change. See Section IV.
  • domain assumption The BS-IR channel H is perfectly known to both BS and UAV-IR, and the IR can implement arbitrary continuous phase shifts with constant amplitude a.
    Section II.A assumes full CSI H and Proposition 1 requires exact phase alignment; real IRs have discrete phase shifts and imperfect CSI.
  • ad hoc to paper The UE's future position follows a Gaussian distribution with mean y(t)+v_e Delta T and variance Var(Y_t).
    Section III.B defines f(y,omega|Y_t) = N(mu,Lambda|Y_t); no empirical validation is provided, and the RL value function depends on this model for prediction.
  • ad hoc to paper Deep Q-learning with an LSTM network converges to a local optimal value function in the nonstationary environment given long enough T.
    Section III.B asserts convergence citing [13], but [13] covers tabular or simpler settings; the continuous, nonstationary function-approximation case is not proven here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reflections in the Sky: Millimeter Wave Communication with UAV-Carried Intelligent Reflectors." pith.science (2026). https://pith.science/paper/PXGMPNTU

@misc{pith2026190803271,
  author       = {Pith},
  title        = {Pith review of: Reflections in the Sky: Millimeter Wave Communication with UAV-Carried Intelligent Reflectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXGMPNTU}},
  note         = {Machine review of arXiv:1908.03271}
}
read the original abstract

In this paper, a novel approach that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed to enhance the performance of millimeter wave (mmW) networks. In particular, the UAV-IR is used to intelligently reflect mmW beamforming signals from a base station towards a mobile outdoor user, while harvesting energy from mmW signals to power the IR. To maintain a line-of-sight (LOS) channel, a reinforcement learning (RL) approach, based on Q-learning and neural networks, is proposed to model the propagation environment, such that the location and reflection coefficient of the UAV-IR can be optimized to maximize the downlink transmission capacity. Simulation results show a significant advantage for using a UAV-IR over a static IR, in terms of the average data rate and the achievable downlink LOS probability. The results also show that the RL-based deployment of the UAV-IR further improves the network performance, relative to a scheme without learning.

Figures

Figures reproduced from arXiv: 1908.03271 by the authors.

Figure 1
Figure 1. If the downlink channel over mmW between the BS and a UE is blocked, a UAV-IR will be deployed to serve the NLOS UE. body on mmW signals, the real-time value of CSI is difficult to obtain. Thus, to enable an efficient deployment of a UAV￾IR for mmW transmissions, the challenge of CSI estimation must be properly addressed. The main contribution of this paper is, thus, a novel frame￾work for effective deployment of a … view at source ↗
Figure 2
Figure 2. Illustration of the considered deployment environment for our simulations [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Average data rate per time slot of the BS-IR-UE link and the average LOS probability of the IR-UE link. multi-input-multi-output channel is based on [18]. The carrier frequency is set to be 30 GHz, the BS transmit power is 40 dBm, with b = 0.1 GHz, N = 16, M = 64, V max r = 20 m/s, Ve = 1 m/s, a = 0.8, κ = 60%, ∆T = 0.1 s, τ = 5 dB, and γ = 0.1. The planar antenna array with 4 × 4 IR components and 8 × 8 antennas ar… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Average data rate of the BS-IR-UE link and the RF energy harvesting power, with a fixed UAV height at 40 m. to the static case. Compared with the RL-based deployment, the lower data rate of the UAV-IR with no RL is caused by the frequent movement, due to its short-sigh…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 17 canonical work pages

  1. [13]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT press, 2018

  2. [1]

    A vision of 6G wireless systems: Applications, trends, technologies, and open research problems,

    W. Saad, M. Bennis, and M. Chen, “A vision of 6G wireless systems: Applications, trends, technologies, and open research problems,” IEEE Network, to appear, 2019

  3. [2]

    Ultrareliable and low-latency wireless communication: Tail, risk, and scale,

    M. Bennis, M. Debbah, and H. V . Poor, “Ultrareliable and low-latency wireless communication: Tail, risk, and scale,” Proceedings of the IEEE, vol. 106, no. 10, pp. 1834–1853, Sep 2018

  4. [3]

    An effective coverage scheme with passive-reflectors for urban millimeter- wave communication,

    Z. Peng, L. Li, M. Wang, Z. Zhang, Q. Liu, Y . Liu, and R. Liu, “An effective coverage scheme with passive-reflectors for urban millimeter- wave communication,” IEEE Antennas and Wireless Propagation Let- ters, vol. 15, pp. 398–401, June 2015

  5. [4]

    Enabling indoor mobile millimeter-wave networks based on smart reflect-arrays,

    X. Tan, Z. Sun, D. Koutsonikolas, and J. M. Jornet, “Enabling indoor mobile millimeter-wave networks based on smart reflect-arrays,” inIEEE International Conference on Computer Communications , Honolulu, HI, USA, April 2018, pp. 270–278

  6. [5]

    Backscatter communication and RFID: Coding, energy, and MIMO analysis,

    C. Boyer and S. Roy, “Backscatter communication and RFID: Coding, energy, and MIMO analysis,” IEEE Transactions on Communications , vol. 62, no. 3, pp. 770–785, Dec 2014

  7. [6]

    Mmwave measurement of RF reflectors for 5G green communications,

    T. Hong, J. Yao, C. Liu, and F. Qi, “Mmwave measurement of RF reflectors for 5G green communications,” Wireless Communications and Mobile Computing, vol. 2018, May 2018

  8. [7]

    Intelligent reflecting surface enhanced wireless network: Joint active and passive beamforming design,

    Q. Wu and R. Zhang, “Intelligent reflecting surface enhanced wireless network: Joint active and passive beamforming design,” in 2018 IEEE Global Communications Conference, Abu Dhabi, United Arab Emirates, Dec 2018, pp. 1–6

Show all 18 references
  1. [8]

    Integrated tera- hertz communication with reflectors for 5G small-cell networks,

    M. T. Barros, R. Mullins, and S. Balasubramaniam, “Integrated tera- hertz communication with reflectors for 5G small-cell networks,” IEEE Transactions on Vehicular Technology , vol. 66, no. 7, pp. 5647–5657, Dec 2016

  2. [9]

    Energy efficient multi-user MISO communication using low resolution large intelligent surfaces,

    C. Huang, G. C. Alexandropoulos, A. Zappone, M. Debbah, and C. Yuen, “Energy efficient multi-user MISO communication using low resolution large intelligent surfaces,” arXiv preprint arXiv:1809.05397 , 2018

  3. [10]

    A tutorial on UA Vs for wireless networks: Applications, challenges, and open problems,

    M. Mozaffari, W. Saad, M. Bennis, Y .-H. Nam, and M. Debbah, “A tutorial on UA Vs for wireless networks: Applications, challenges, and open problems,”IEEE Communications Surveys and Tutorials, to appear, 2019

  4. [11]

    Caching in the sky: Proactive deployment of cache-enabled unmanned aerial vehicles for optimized quality-of-experience,

    M. Chen, M. Mozaffari, W. Saad, C. Yin, M. Debbah, and C. S. Hong, “Caching in the sky: Proactive deployment of cache-enabled unmanned aerial vehicles for optimized quality-of-experience,” IEEE Journal on Selected Areas in Communications , vol. 35, no. 5, pp. 1046–1061, Mar 2017

  5. [12]

    Mobile unmanned aerial vehicles (UA Vs) for energy-efficient internet of things communica- tions,

    M. Mozaffari, W. Saad, M. Bennis, and M. Debbah, “Mobile unmanned aerial vehicles (UA Vs) for energy-efficient internet of things communica- tions,” IEEE Transactions on Wireless Communications, vol. 16, no. 11, pp. 7574–7589, Sep 2017

  6. [14]

    Millimeter-wave massive mimo: The next wireless revolution?

    A. L. Swindlehurst, E. Ayanoglu, P. Heydari, and F. Capolino, “Millimeter-wave massive mimo: The next wireless revolution?” IEEE Communications Magazine, vol. 52, no. 9, pp. 56–62, Sep 2014

  7. [15]

    Basic relationship between channel coherence time and beamwidth in vehicular channels,

    V . Va and R. W. Heath, “Basic relationship between channel coherence time and beamwidth in vehicular channels,” in Proc. of IEEE Vehicular Technology Conference, Boston, MA, USA, Sept 2015, pp. 1–5

  8. [16]

    Artificial neural networks-based machine learning for wireless networks: A tutorial,

    M. Chen, U. Challita, W. Saad, C. Yin, and M. Debbah, “Artificial neural networks-based machine learning for wireless networks: A tutorial,” IEEE Communications Surveys & Tutorials ( Early Access ) , July 2019

  9. [17]

    5G channel model for bands up to 100 GHz,

    “5G channel model for bands up to 100 GHz,” Oct. 2016, [Online]. Available: http://www.5gworkshops.com/5GCM.html

  10. [18]

    Continuous analog channel esti- mation aided beamforming for massive mimo systems,

    V . V . Ratnam and A. F. Molisch, “Continuous analog channel esti- mation aided beamforming for massive mimo systems,” arXiv preprint arXiv:1901.08763, Jan 2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.