Scheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning

Mahadesh Panju; Ramkumar Raghu; Vaneet Aggarwal; Vinod Sharma

REVIEW 5 major objections 6 minor 42 references

Scheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-27 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a deep reinforcement learning variant, AC-DQN, can learn a multicast power-control policy that matches the global optimum of a non-scalable direct-search method while respecting an average power constraint and…

desk verdict Useful engineering paper on deep RL for multicast power control that overclaims exact optimality, but the algorithms, honest caveats, and released code make it worth refereeing. read the letter →

arxiv 2011.14799 v1 pith:QMFBZT27 submitted 2020-09-27 cs.NI cs.LGstat.ML

Ramkumar Raghu , Mahadesh Panju , Vaneet Aggarwal , Vinod Sharma This is my paper · ORCID

classification cs.NIcs.LGstat.ML

keywords multicastpowercontroldeepreinforcementlearningconstrainedMarkovdecisionprocessmulti-timescalestochasticapproximationqueueingschedulingDQN

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that deep reinforcement learning can replace a computationally prohibitive model-based optimization for wireless multicast downlinks. It proposes AC-DQN, a constrained Deep Q-Network that learns a transmit-power policy online, and reports that in a small four-user network the learned policy matches the global optimum found by the MADS direct-search algorithm while keeping average power at the constraint. It adds DSGD, a deep-network-assisted stochastic gradient method that selects the best queueing strategy from among retransmit, loopback, and defer, and then IDA, which runs both learners on separate timescales to optimize queueing and power jointly. Simulations with up to 20 users and time-varying arrivals show roughly 50 percent delay improvement over constant-power transmission and successful tracking of changing traffic. If the claim holds, scalable online power control and scheduling for multicast systems do not require knowing traffic or fading statistics.

What carries the argument

The load-bearing mechanism is multi-timescale stochastic gradient descent applied to a constrained Markov decision process. AC-DQN approximates the action-value function with a deep neural network and enforces the average power constraint with a Lagrange multiplier $\beta$; the Q-network is updated on the fast timescale ($\eta_1$) while $\beta$ is updated on a slower timescale ($\eta_2$), so the learner balances reward against power cost. DSGD uses a deep network $f_\theta$ trained on replay-memory samples of noisy sojourn-time observations, then takes a projected gradient step over the probability simplex of queueing strategies, which lets it pick among retransmit, loopback, and defer without a closed-form delay expression. IDA adds the queueing-strategy update as a third, slowest timescale, so the power-control MDP appears nearly stationary while both the Q-function and the Lagrange multiplier are being learned.

What would settle it

Run the small four-user scenario with $\gamma = 0.9$, $\gamma = 0.99$, and a direct average-cost Q-learning variant, and compare each resulting policy against the MADS solution on both mean sojourn time and average power. If any discounted policy misses the MADS-optimal delay at the same average power, or if the average power drifts from the constraint when arrival rates change, the central optimality claim is not supported.

Watch

Extended reading notes

Core claim

The central claim is that the optimal power-control policy for a multicast queue under fading can be learned by a deep Q-network that enforces an average-power constraint, without knowing arrival rates or channel statistics, and that the same learning machinery can be extended to choose the queueing strategy. In the small-user regime, AC-DQN is reported to achieve the same mean sojourn time as the MADS power-control policy, which is treated as the global optimum; in a 20-user system with Zipf popularity and Rayleigh fading it improves mean sojourn time by roughly half over constant-power transmission. DSGD learns the mapping from queueing parameters to mean sojourn time with a neural network and follows its gradient to select among retransmit, loopback, and defer. IDA combines the two and is reported to reach the optimal point obtained by DSGD and AC-DQN together, meaning queueing strategy and power control can be optimized simultaneously. The authors also claim that keeping the learning rates constant rather than decaying them lets AC-DQN track time-varying system statistics, while decaying rates leave the Lagrange multiplier frozen at the wrong value.

Load-bearing premise

The load-bearing premise is that the discounted-MDP formulation with discount factor $\gamma = 0.9$ closely approximates the true average-cost objective of minimizing stationary mean sojourn time, but the paper does not test how sensitive the conclusions are to this choice.

Editorial extensions

If this is right

A multicast base station can learn its transmit-power policy online from its own transmissions, with no prior model of arrival rates or fading.
The Lagrange-multiplier update generalizes to multiple constraints: one extra slow SGD step per constraint, with the same timescale separation.
Constant step sizes are the practical choice for non-stationary traffic, because decaying step sizes prevent the multiplier from adapting after a regime change.
Queueing and power control can be co-optimized, and the joint learner selects whichever pure or mixed queueing strategy is best for the current load.
The approach scales to at least 20 users and 20 power levels, a regime where the MADS comparison baseline is computationally infeasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same three-timescale template could be applied to other cross-layer wireless problems, such as joint caching, scheduling, and power control, since the algorithm only requires a Q-network, one Lagrange multiplier per constraint, and a parametrized policy to optimize.
Editorial inference: the mixed-policy convergence at some arrival rates suggests there are flat regions in the objective, so the exact queueing probabilities may matter less than which pure strategies they interpolate between.
Editorial inference: a natural and decisive extension would be to test AC-DQN on the small network with $\gamma = 0.99$ and with a genuinely average-cost learning rule; if either deviates from the MADS optimum, the discounted approximation rather than the learning itself would be the limiting factor.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Raghu et al. tackle a real problem: adaptive queueing and power control for a multicast downlink, where the prior exact approach (MADS) is computationally prohibitive beyond a handful of users. The paper's contribution is a set of deep RL algorithms - DSGD for queueing, AC-DQN for constrained power control, and IDA for the joint problem - plus simulations and released code. Credit where due: the problem is well-motivated, the algorithms are clearly specified, the 20-user results show roughly 50% delay reduction over constant power, and the authors are upfront about what is not proved. They explicitly state that AC-DQN convergence is not established and that the discounted-to-average-cost link needs gamma close to 1.

The soft spots are around the word 'achieves' in Section VII-C. AC-DQN uses a discrete 20-level power grid, while MADS optimizes continuous powers; the paper never checks whether the MADS-optimal powers fall on that grid, so exact equality is structurally questionable. Second, all simulations use gamma=0.9 with no sensitivity analysis, even though Section IV-C admits the approximation is only valid near gamma=1. Third, the average power constraint is implemented with a finite window T_W=200 rather than the infinite-horizon average in the theory, so the two-timescale convergence cited from Borkar doesn't directly apply. Finally, the MADS comparison appears to be a single run; no error bars or seeds are reported. These are not fatal - the empirical gains may be real - but they mean the global-optimality claim is stronger than the evidence supports. The queueing-side DSGD also lacks a proof, though the simulations show it picking sensible policies.

This is an engineering paper, and a useful one. The code is public, the experiments are extensive, and the limitations are mostly acknowledged. The citation pattern leans on the authors' own prior work, which is appropriate given the problem is theirs. I think a serious referee should engage with it, mainly to push for softer claims and statistical rigor in the comparisons. I would not cite it in my own work in the next year, but it would be a reasonable reading-group pick for anyone interested in constrained deep RL for wireless.

Referee Report

5 major / 6 minor

Summary. This paper addresses power control and queueing-strategy selection for a single-server multicast downlink with fading. It proposes three algorithms: DSGD, which fits a neural network to the mean sojourn time as a function of a queueing-strategy vector and performs projected gradient descent; AC-DQN, a DQN variant that adds a learned Lagrange multiplier to respect an average power constraint; and IDA, which combines DSGD and AC-DQN in a multi-timescale scheme. The paper reports simulations for systems of 4, 10, and 20 users, claims that AC-DQN reproduces the MADS-optimal power policy and tracks non-stationary arrivals, and claims that IDA achieves the optimal queueing strategy and power control jointly.

Significance. The contribution is potentially useful: if the empirical claims hold, the paper offers an online, scalable alternative to the MADS optimization of [8], with the attractive extra properties of tracking non-stationary statistics and joint queueing/power optimization. The multi-timescale Lagrange-DQN construction is a reasonable way to handle average-power constraints, and the authors state that the code is available. The paper is also honest about some limitations, notably that convergence of the modified DQN is not proved. However, the central optimality claims currently outrun the evidence, so the manuscript needs revision rather than acceptance.

major comments (5)

[Section VII-C and Introduction] The claim that AC-DQN 'achieves the global optimum obtained by MADS' is not supported, because the two algorithms optimize over different action sets. MADS solves Eq. (7) with continuous P_k >= 0, whereas AC-DQN uses exactly 20 discrete transmit power levels (Section VII-A). The paper never checks whether the MADS-optimal powers for the L=4 case lie on that discrete grid, and if any optimum is off-grid, exact equality is structurally impossible. Please either verify grid confinement, report the quantization gap, or soften the claim to 'matches the MADS value at the resolution of the available action grid.'
[Section IV-C and Section VII-A] The discounted MDP with discount factor gamma=0.9 is used as a proxy for the average-cost objective in Eqs. (10)-(11), although Section IV-C itself states that the proxy is valid only when gamma is close to 1. No sensitivity analysis over gamma is provided, so it is unclear whether AC-DQN and IDA optimize the intended mean-sojourn-time objective; please add experiments with gamma, say, 0.99 and 0.999, or use an average-cost RL formulation, and report average power and delay under each setting.
[Section V-B and Eq. (16)] The Lagrange multiplier update uses CP(St), a finite-window empirical average over TW=200 transmissions, as a stand-in for the limiting average in Eq. (11). The two-timescale convergence invoked from [9] is not directly applicable to this biased, finite-horizon estimator, and the text immediately admits that convergence of the modified algorithm is not proved. Please either provide a formal statement for the finite-window estimator under appropriate mixing conditions, or present empirical evidence that the estimator bias is negligible for the chosen TW.
[Sections VII-B, VII-C, VII-E, VII-F] The quantitative claims rest on single simulation runs without error bars, multiple seeds, or confidence intervals; this applies to Figure 4a (MADS vs. AC-DQN), Figure 5a, and Figure 7a. Without a statistical statement, a visual match between two learning curves cannot carry the weight of a global-optimality claim. Please run several independent seeds and report means with confidence intervals or variance bands.
[Section VI and Section VII-F] No convergence or optimality proof is given for IDA. Eq. (18) merely states step-size conditions 'required for convergence' of a generic multi-timescale stochastic approximation, but IDA includes DNN training, replay memory, exploration noise, and finite-difference gradient approximations; it is not established that the coupled iterates converge, let alone to the joint optimum of the queueing and power-control objectives. The conclusion in Section VII-F that IDA 'achieves the optimal point obtained by both DSGD and AC-DQN' is therefore a simulation observation, not an established result.

minor comments (6)

[Figures 3 and 8 captions] The word 'Porbability' in the captions should be 'Probability'.
[Section VII-A] The specification 'Power Transmit Levels = 20 (1 to 50)' is ambiguous; please state whether the 20 levels are uniformly spaced over [1,50] and how the integer scale maps to physical transmit power.
[Eq. (4)] The projection onto the probability simplex is garbled in the typeset text; the normalization and the positive-part operator should be written unambiguously.
[Algorithm 3] The line 'D<-append(sojourtime d'is)' should be cleaned up to 'append the sojourn times d_i of the current service window to D'.
[Introduction, DQN discussion] The statement that value-iteration deep RL methods like DQN 'provide global minimum' is stronger than the cited convergence results in [16], which require specific smoothness and network assumptions; please qualify the claim.
[Section II and Section VII-A] The notation for transmission rate and file size is inconsistent: Section II defines R bits/sec and T=F/R, while Section VII-A lists both C=10 MB/s and R=10 MB/s; unify the notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: AC-DQN/DSGD/IDA are validated against an independent simulator and prior MADS baseline; self-citations are background, and the admitted gamma approximation is a correctness gap, not a circularity.

full rationale

The derivation chain is not circular in the sense defined here. DSGD optimizes f(p)=E_p[D] using a DNN surrogate trained on noisy observations; its 'optimal' claim is an empirical match to baseline strategies run on the same simulator, not an equation that reduces to the fitted surrogate. AC-DQN is compared against MADS, which is an external derivative-free optimizer (Audet & Dennis) applied to Eq. (7); the paper's constrained-MDP Lagrangian reward r_t = R(S_t,P_t) - beta*P_t is the standard relaxation of that same average-reward objective, so the comparison is a genuine validation rather than a renaming. The self-citations to [7] and [8] supply the queueing model, the stationarity proposition, and the MADS baseline, but they do not by themselves imply AC-DQN's convergence or its performance; those are established by simulation of the full system. IDA literally appends the DSGD update to AC-DQN, so its 'achieving' the separately obtained optima is an engineering integration check, not a definitional identity. The paper even flags the main non-circular weaknesses: gamma=0.9 is only an approximation to the average-cost objective, the finite 20-level action space may not contain the continuous MADS optimum, and convergence of the tracking modification is 'not proved yet'. These are correctness and evidence concerns, not circularity. Hence score 0.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The paper's central claims rest on the multicast queue model and optimality benchmarks inherited from the authors' prior works, on the discounted-to-average-cost proxy, and on a set of hand-chosen hyperparameters. No new physical entity is introduced; the algorithms are the contribution.

free parameters (6)

Discount factor gamma = 0.9
Chosen in Section VII-A4; the average-cost objective requires gamma close to 1, and 0.9 gives an effective horizon of about 10 steps, which may not approximate the stationary sojourn time well.
AC-DQN learning rates eta1, eta2 = eta1=0.001, eta2=0.0001 (decaying); eta1=0.001, eta2=0.00003 (constant tracking)
Section V-B and VII-E; the two-timescale ratio eta2/eta1 to 0 is required, and for tracking the constant step sizes are chosen manually without a prescribed procedure.
Average power window T_W = 200
Used in (16) to estimate C_P(S_t); a finite window introduces bias and variance in the Lagrange multiplier update.
DQN hyperparameters (memory, minibatch, target period, epsilon schedule) = |M|=30000, n=64, T_target=100, epsilon=0.98^t
Section VII-A4; no sensitivity analysis; these standard choices affect the empirical convergence and the claimed optimality.
DSGD approximation window S_approx and training time T_train = S_approx=100, T_train=100
Section VII-A4; these control the noise of the observations (1) and the initial exploration; no analysis of their effect on the final policy p.
Discrete transmit power grid = 20 levels in [1, 50] W
Section VII-A4; the average power constraint is 7 W, so the action space is coarse relative to the constraint; the DQN can only choose among these 20 levels.

assumptions (6)

domain assumption The multicast queue has a unique stationary distribution for any randomized policy p, and E_p[D] exists (Theorem 1 and Proposition 1 of [7]).
Invoked in Section III to define the objective f(p); this is a theorem from the authors' prior work, not reproved here.
domain assumption Channel gains H_j(t) are i.i.d. across time, take finitely many values, and are independent across users (Section II).
The MDP formulation and the state space O(2^L G^L) depend on this finite-state fading model; the simulations use continuous Rayleigh fading, so the formulation is an approximation.
domain assumption The required transmit power for a successful transmission is given by the ideal Shannon formula (5).
Equation (5) neglects practical coding and finite blocklength effects; the reward function (6) depends entirely on this model.
ad hoc to paper The optimal queueing strategy is contained in the probability simplex over retransmit, loopback, and defer (Section II-B).
The parameterization p in the simplex restricts DSGD to a fixed family of strategies; there is no proof that the true optimal policy lies in this family.
domain assumption A discounted MDP with gamma sufficiently close to 1 is a good proxy for the average-cost MDP (Section IV-C).
This is the standard but unproved bridging assumption used to justify DQN; with gamma=0.9 in the simulations it may not hold.
standard math The coupled iterates in (14)-(16) and (2)-(3) converge under the learning-rate conditions (17)-(18), or track when step sizes are constant (Sections V-B, VI).
The multi-timescale stochastic approximation theory of Borkar [9] is cited, but the tracking simulations violate the decreasing-step-size conditions, so convergence and tracking are asserted empirically.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Scheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/QMFBZT27

@misc{pith2026201114799,
  author       = {Pith},
  title        = {Pith review of: Scheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QMFBZT27}},
  note         = {Machine review of arXiv:2011.14799}
}

read the original abstract

Multicasting in wireless systems is a natural way to exploit the redundancy in user requests in a Content Centric Network. Power control and optimal scheduling can significantly improve the wireless multicast network's performance under fading. However, the model based approaches for power control and scheduling studied earlier are not scalable to large state space or changing system dynamics. In this paper, we use deep reinforcement learning where we use function approximation of the Q-function via a deep neural network to obtain a power control policy that matches the optimal policy for a small network. We show that power control policy can be learnt for reasonably large systems via this approach. Further we use multi-timescale stochastic optimization to maintain the average power constraint. We demonstrate that a slight modification of the learning algorithm allows tracking of time varying system statistics. Finally, we extend the multi-timescale approach to simultaneously learn the optimal queueing strategy along with power control. We demonstrate scalability, tracking and cross layer optimization capabilities of our algorithms via simulations. The proposed multi-timescale approach can be used in general large state space dynamical systems with multiple objectives and constraints, and may be of independent interest.

Figures

Figures reproduced from arXiv: 2011.14799 by the authors.

**Figure 1.** System model More details of the system are described in the following sections as follows. Section II-A describes the basic Multicast queue proposed in [7]. The queueing schemes to mitigate the effects of fading studied in [8] are also presented. Section II-B parametrizes the queuing schemes. Section III provides an online learning scheme to obtain the optimal policy for a given setup. In Sections IV-A and IV-B, we… view at source ↗

**Figure 2.** DSGD Performance in parametrized multicast system [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 4.** AC-DQN Performance in 1-LB system with L = 4, P = 7, Uniform Popularity, Uniform fading. size maintains the average power constraint of P = 5, the average power achieved by the decaying step-size AC-DQN drops to 4. Hence, the decaying step-size AC-DQN suffers suboptimal utilization of available power. Thus in practical systems, only constant step-size AC-DQN will be capable of adapting to the changing system statist… view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: AC-DQN Performance in 1-LB system with L = 20, P = 7, Zipf(1) Popularity, Rayleigh fading. plots showing convergence of probabilities for rates 0.8, 2.0 and 3.0 are shown in Figures 8a, 8b and 8c respectively. We see, in 8a, for arrival rate 0.8, that the probability c…

**Figure 8.** Figure 8: IDA convergence of queueing strategies for different [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

**Figure 7.** Figure 7: IDA Performance in parametrized multicast system [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 39 canonical work pages

[8]

Queueing theoretic models for multicasting under fading,

M. Panju, R. Raghu, V . Agarwal, V . Sharma, and R. Ramachandran, “Queueing theoretic models for multicasting under fading,” IEEE Wire- less Communications and Networking Conference (WCNC), Marrakech, Morocco, 2019

work page 2019
[9]

Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint

V . Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint . Cambridge University Press, 2008

2008
[1]

Cisco visual networking index: global mobile data trafﬁc fore- cast update 2016-2021 white paper,

Cisco, “Cisco visual networking index: global mobile data trafﬁc fore- cast update 2016-2021 white paper,” 2016

work page 2016
[2]

I tube, you tube, everybody tubes: Analyzing the world’s largest user generated content video system,

M. Cha, H. Kwak, P. Rodriguez, Y .-Y . Ahn, and S. Moon, “I tube, you tube, everybody tubes: Analyzing the world’s largest user generated content video system,” in Proc. 7th ACM SIGCOMM Conf. on Internet Measure., 2007, pp. 1–14

work page 2007
[3]

Improving queue stability in wireless mul- ticast with network coding,

N. Moghadam and H. Li, “Improving queue stability in wireless mul- ticast with network coding,” IEEE Inter. Conf. on Commun. (ICC) , pp. 3382–3387, 2015

work page 2015
[4]

Queue length analysis for multicast: Limits of performance and achievable queue length with random linear coding,

R. Cogill and B. Shrader, “Queue length analysis for multicast: Limits of performance and achievable queue length with random linear coding,” in 47th Allerton Conf on Commun., Control, and Comp. , 2009, pp. 462– 468

work page 2009
[5]

Fundamental limits of caching,

M. A. Maddah-Ali and U. Niesen, “Fundamental limits of caching,” IEEE Trans. Inf. Theory , vol. 60, pp. 2856–2867, 2014

work page 2014
[6]

Stability, rate, and delay analysis of single bottleneck caching networks,

F. Rezaei and B. H. Khalaj, “Stability, rate, and delay analysis of single bottleneck caching networks,” IEEE Trans. Commun., vol. 64, no. 1, pp. 300–313, 2016

work page 2016

Show all 42 references

[7]

Queuing theoretic models for multicast and coded-caching in downlink wireless systems,

M. Panju, R. Raghu, V . Sharma, and R. Ramachandran, “Queuing theoretic models for multicast and coded-caching in downlink wireless systems,” arXiv:1804.10590, 2018

2018 arXiv
[10]

Deep reinforcement learning,

Y . Li, “Deep reinforcement learning,” CoRR, 2018. [Online]. Available: http://arxiv.org/abs/1810.06339

2018 arXiv
[11]

Human-level control through deep reinforcement learn- ing,

V . Mnih et al., “Human-level control through deep reinforcement learn- ing,” Nature, vol. 518, Feb 2015

2015
[12]

Trust region policy optimization,

J. Schulman et al. , “Trust region policy optimization,” in Inter. Conf. Machine Learning, 2015, pp. 1889–1897

2015
[13]

Proximal policy optimization algorithms,

——, “Proximal policy optimization algorithms,” arXiv: 1707.06347 , 2017

2017 arXiv
[14]

Playing atari with deep reinforcement learning,

V . Mnih et al., “Playing atari with deep reinforcement learning,” NIPS Deep Learning Workshop, 2013

2013
[15]

A general reinforcement learning algorithm that masters chess, shogi, and go through self-play,

D. Silver et al., “A general reinforcement learning algorithm that masters chess, shogi, and go through self-play,” Science, vol. 362, no. 6419, pp. 1140–1144, 2018

2018
[16]

A theoretical analysis of deep q- learning,

Z. Yang, Y . Xie, and Z. Wang, “A theoretical analysis of deep q- learning,” arXiv: 1901.00137, 2019

1901 arXiv
[17]

Deep reinforcement learning with double q-learning,

H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Thirtieth AAAI Conference on Artiﬁcial Intelligence, 2016

2016
[18]

Lyapunov scheduling and optimization in network coded wireless multicast network,

N. Moghadam, H. Li, H. Zeng, and L. Liu, “Lyapunov scheduling and optimization in network coded wireless multicast network,” IEEE Transactions on Vehicular Technology , vol. 67, no. 6, pp. 5135–5145, 2018

2018
[19]

Simpliﬁed optimal scheduling (sos) for network coded wireless multicast,

N. Moghadam, G. Zhang, and H. Li, “Simpliﬁed optimal scheduling (sos) for network coded wireless multicast,” in2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), 2018, pp. 1–5

2018
[20]

Multicast scheduling for delay-energy trade-off under bursty request arrivals in cellular networks,

Y . Zhou et al. , “Multicast scheduling for delay-energy trade-off under bursty request arrivals in cellular networks,” IET Comm. , vol. 13, pp. 1696–1701(5), 2019

2019
[21]

Capacity and optimal power allocation for fading broadcast channels with minimum rates,

N. Jindal and A. Goldsmith, “Capacity and optimal power allocation for fading broadcast channels with minimum rates,” IEEE Transactions on Information Theory, vol. 49, no. 11, pp. 2895–2909, Nov 2003

2003
[22]

A distributed joint scheduling and power control algorithm for multicasting in wireless ad hoc networks,

K. Wang, C. F. Chiasserini, R. R. Rao, and J. G. Proakis, “A distributed joint scheduling and power control algorithm for multicasting in wireless ad hoc networks,” in IEEE Intern. Conf. on Comm. , vol. 1, 2003, pp. 725–731

2003
[23]

Deep reinforcement learning based resource allocation for v2v communications,

H. Ye, G. Y . Li, and B. F. Juang, “Deep reinforcement learning based resource allocation for v2v communications,” IEEE Transactions on Vehicular Technology, vol. 68, no. 4, pp. 3163–3173, April 2019

2019
[24]

Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,

Y . S. Nasir and D. Guo, “Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,” arXiv, 2018. [Online]. Available: http://arxiv.org/abs/1808.00490v3

2018 arXiv
[25]

Double coded caching in ultra dense networks: Caching and multicast scheduling via deep reinforcement learning,

Z. Zhang et al., “Double coded caching in ultra dense networks: Caching and multicast scheduling via deep reinforcement learning,” IEEE Trans. Comm., vol. 68, no. 2, pp. 1071–1086, 2020

2020
[26]

Deep learning for intelligent wireless networks: A comprehensive survey,

Q. Mao, F. Hu, and Q. Hao, “Deep learning for intelligent wireless networks: A comprehensive survey,” IEEE Communications Surveys Tutorials, vol. 20, no. 4, pp. 2595–2621, 2018

2018
[27]

Altman, Constrained Markov Decision Processes

E. Altman, Constrained Markov Decision Processes. CRC Press, 1999

1999
[28]

Constrained policy optimization,

J. Achiam et al. , “Constrained policy optimization,” in Proc. Intern. Conf. on Machine Learning , 2017, pp. 22–31

2017
[29]

Reward constrained policy optimization,

C. Tessler, D. J. Mankowitz, and S. Mannor, “Reward constrained policy optimization,” arXiv: 1805.11074, 2018

2018 arXiv
[30]

Convergent policy optimization for safe reinforcement learning,

M. Yu et al. , “Convergent policy optimization for safe reinforcement learning,” in Advances in NIPS , 2019, pp. 3127–3139

2019
[31]

Finite- state markov modeling of fading channels - a survey of principles and applications,

P. Sadeghi, R. A. Kennedy, P. B. Rapajic, and R. Shams, “Finite- state markov modeling of fading channels - a survey of principles and applications,” IEEE Signal Processing Magazine, vol. 25, no. 5, pp. 57– 80, September 2008

2008
[32]

Bhatnagar, H

S. Bhatnagar, H. Prasad, and L. Prashanth, Stochastic Recursive Algo- rithms for Optimization: Simultaneous Perturbation Methods . Springer London, 2013

2013
[33]

Universal function approximation by deep neural nets with bounded width and relu activations,

B. Hanin, “Universal function approximation by deep neural nets with bounded width and relu activations,” Open Access Journals MDPI, Mathematics , vol. 7, no. 10, 2019. [Online]. Available: https://www.mdpi.com/2227-7390/7/10/992

2019
[34]

Deep learning is robust to massive label noise,

D. Rolnick et al., “Deep learning is robust to massive label noise,”arXiv: 1705.10694, 2017

2017 arXiv
[35]

Approximation of functions and their derivatives: A neural network implementation with applications,

T. Nguyen-Thien and T. Tran-Cong, “Approximation of functions and their derivatives: A neural network implementation with applications,” Applied Mathematical Modelling , vol. 23, pp. 687 – 704, 1999

1999
[36]

Self-improving reactive agents based on reinforcement learning, planning and teaching,

L.-J. Lin, “Self-improving reactive agents based on reinforcement learning, planning and teaching,” Machine Learning , vol. 8, no. 3, pp. 293–321, May 1992. [Online]. Available: https://doi.org/10.1007/ BF00992699 15

1992
[38]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv: 1412.6980, 2014

2014 arXiv
[39]

Proakis and M

J. Proakis and M. Salehi, Digital Communications, 5th ed. McGraw- Hill, 2008

2008
[40]

Mesh adaptive direct search algorithms for constrained optimization,

C. Audet and J. E. Dennis Jr, “Mesh adaptive direct search algorithms for constrained optimization,” SIAM Journal on optimization , vol. 17, no. 1, pp. 188–217, 2006

2006
[41]

M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dy- namic Programming, 1st ed. J. Wiley & Sons, Inc., 1994

1994
[42]

Continuous control with deep reinforcement learning,

T. P. Lillicrap et al. , “Continuous control with deep reinforcement learning,” arXiv: 1509.02971v5, 2016

2016 arXiv
[2018]

Available: http://arxiv.org/abs/1803.00942

[Online]. Available: http://arxiv.org/abs/1803.00942

arXiv

Pith tools

Reviewed August 27, 2026 · model on record in the stance chip above.

[1] [8]

Queueing theoretic models for multicasting under fading,

M. Panju, R. Raghu, V . Agarwal, V . Sharma, and R. Ramachandran, “Queueing theoretic models for multicasting under fading,” IEEE Wire- less Communications and Networking Conference (WCNC), Marrakech, Morocco, 2019

work page 2019

[2] [9]

Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint

V . Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint . Cambridge University Press, 2008

2008

[3] [1]

Cisco visual networking index: global mobile data trafﬁc fore- cast update 2016-2021 white paper,

Cisco, “Cisco visual networking index: global mobile data trafﬁc fore- cast update 2016-2021 white paper,” 2016

work page 2016

[4] [2]

I tube, you tube, everybody tubes: Analyzing the world’s largest user generated content video system,

M. Cha, H. Kwak, P. Rodriguez, Y .-Y . Ahn, and S. Moon, “I tube, you tube, everybody tubes: Analyzing the world’s largest user generated content video system,” in Proc. 7th ACM SIGCOMM Conf. on Internet Measure., 2007, pp. 1–14

work page 2007

[5] [3]

Improving queue stability in wireless mul- ticast with network coding,

N. Moghadam and H. Li, “Improving queue stability in wireless mul- ticast with network coding,” IEEE Inter. Conf. on Commun. (ICC) , pp. 3382–3387, 2015

work page 2015

[6] [4]

Queue length analysis for multicast: Limits of performance and achievable queue length with random linear coding,

R. Cogill and B. Shrader, “Queue length analysis for multicast: Limits of performance and achievable queue length with random linear coding,” in 47th Allerton Conf on Commun., Control, and Comp. , 2009, pp. 462– 468

work page 2009

[7] [5]

Fundamental limits of caching,

M. A. Maddah-Ali and U. Niesen, “Fundamental limits of caching,” IEEE Trans. Inf. Theory , vol. 60, pp. 2856–2867, 2014

work page 2014

[8] [6]

Stability, rate, and delay analysis of single bottleneck caching networks,

F. Rezaei and B. H. Khalaj, “Stability, rate, and delay analysis of single bottleneck caching networks,” IEEE Trans. Commun., vol. 64, no. 1, pp. 300–313, 2016

work page 2016