Pith. sign in

REVIEW 3 major objections 5 minor 35 references

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that an actor-critic deep reinforcement learning framework outperforms a deep Q-network in dynamic multichannel access when the number of channels is 32 or 64, while staying competitive at 16 channels.

desk verdict Competent applied RL paper whose scaling claim over DQN is plausible but not yet evidenced, because the DQN baseline is poorly configured and no variance is reported; the runtime advantage is the most solid result. read the letter →

arxiv 1908.08401 v1 pith:LT35SZVO submitted 2019-08-20 cs.LG cs.ITmath.ITstat.ML

Chen Zhong , Ziyang Lu , M. Cenk Gursoy , Senem Velipasalar This is my paper · ORCID
classification cs.LGcs.ITmath.ITstat.ML
keywords dynamicmultichannelaccessdeepreinforcementlearningactor-criticPOMDPQ-networkmulti-agentwirelessspectrumtemporal-difference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an actor-critic deep reinforcement learning agent is a more scalable channel-access learner than a deep Q-network: it stays competitive at 16 channels and outperforms DQN at 32 and 64 channels, with roughly 87 to 93 percent less runtime per decision. The authors model each user's problem as a partially observable Markov decision process in which the user sees only the channels it selects, and they give single-agent and decentralized multi-agent algorithms. A sympathetic reader should care because spectrum access decisions must be made quickly and at scale, and the paper points to a standard RL architecture that avoids the replay-buffer bottleneck of DQN while learning both channel switching patterns and, in the multi-user case, collision avoidance.

What carries the argument

The load-bearing mechanism is a two-network actor-critic agent whose input is the last $\Omega$ observations, stored as a sparse sliding window. The actor network ends in a softmax layer and assigns a score to every valid $k$-channel action; the critic estimates the state value and produces the temporal-difference error $\delta_t = R_t + \gamma V_\mu(O_{t+1}) - V_\mu(O_t)$. The actor is updated by the policy gradient $\alpha \nabla_\theta \log \pi_\theta(O_t,a_t)\delta_t$, and the critic by minimizing $(\delta_t)^2$. The absence of experience replay is what makes the argument work: the authors argue that the DQN baseline's minibatch replay is the dominant computational cost, so an actor-critic update with no replay buffer explains both the runtime savings and the ability to keep learning as the action space grows.

What would settle it

Rerun the 32- and 64-channel single-good-channel experiments with DQN hyperparameters retuned (for instance, sweep minibatch size and network width, and average over multiple random seeds with error bars). If the tuned DQN earns average rewards at or above the actor-critic agent's reported values, the paper's central scalability claim is refuted; if the tuned DQN still collapses at 64 channels, the claim survives.

Watch

Extended reading notes

Core claim

The central claim is that replacing DQN's value-function-plus-replay design with an actor-critic design removes the main scalability bottleneck for learned multichannel access. In the authors' experiments, DQN is slightly better when there are 16 channels, but at 32 channels DQN earns negative average reward for switching probabilities at or below 0.85, and at 64 channels DQN cannot learn the task at all; the actor-critic agent produces positive, stable average rewards in both regimes. The same pattern appears when a user can access several channels at once: DQN's reward diminishes as the combinatorial action space grows, while the actor-critic reward keeps increasing. In the decentralized multi-user setting, actor-critic agents also outperform DQN agents in average reward and are better at avoiding collisions on excellent and good channels. The runtime claim is that per-decision computation is roughly $3/M$ times DQN's, with $M$ the DQN minibatch size, because the actor-critic agent updates from the critic's temporal-difference error without replaying stored transitions.

Load-bearing premise

The paper's comparison claims rest on the assumption that the DQN baseline, configured with fixed replay memory and minibatch sizes from the prior work and not retuned for 32 or 64 channels, is the right benchmark; if DQN were properly tuned for the large-channel cases, the reported gap could shrink or disappear.

Editorial extensions

If this is right

  • Actor-critic learning remains usable when the number of channels grows to 64 with a single good channel and switching probability as low as 0.75, where DQN fails or earns negative reward.
  • When a user can select $k$ channels per slot, the actor-critic agent's average sum reward continues to rise as $k$ grows, whereas DQN's reward drops, so the framework tolerates combinatorial action spaces better.
  • In a decentralized multi-user network with no information exchange, independent actor-critic agents learn to avoid each other's channels, producing higher average sum reward and fewer collisions than decentralized DQN agents.
  • Per-decision runtime is reduced by about 87 to 93 percent because the agent omits minibatch replay, which matters for real-time spectrum access.
  • After an unknown change point in the channel switching pattern, actor-critic agents recover to their previous reward level, while DQN recovers to a lower level.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the scalability claim holds, the actor-critic architecture may be the safer default for any spectrum-access task with action spaces large enough that replay becomes the bottleneck, even outside the exact correlated-Markov model tested here.
  • The paper does not tune the DQN baseline for large $N$, so the boundary at which DQN genuinely breaks is unknown; a fair benchmark might place it between 32 and 64 channels rather than at 32 or 64.
  • A natural extension is to vary the observation window $\Omega$ and the actor/critic learning-rate ratio: the re-training experiments suggest the learning-rate decay schedule controls the speed-versus-stability tradeoff in time-varying environments.
  • The multi-agent results suggest a testable prediction: if user priorities are set by asymmetric rewards, independent actor-critic agents should converge to a division of excellent channels, which could be verified by longer runs with more than three users.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a deep actor-critic reinforcement learning framework for dynamic multichannel access, covering both a single-user scenario and a decentralized multi-user scenario. The system model treats channels as correlated Markovian sources with good/bad states, gives users only observations of the channels they select, and defines average-reward objectives for both the single-user and multi-user problems. The authors develop actor-critic algorithms (Algorithms 1 and 2) with standard TD-error-based critic updates and policy-gradient actor updates. The experimental section compares average reward and runtime against DQN, random access, the Whittle index heuristic, and an optimal policy with known dynamics, across 16, 32, and 64 channels. The central claim is that the proposed actor-critic framework achieves competitive performance at 16 channels and better performance than DQN at 32 and 64 channels, and that it handles large action spaces and time-varying environments effectively.

Significance. If the central claim holds, the paper provides a useful scalability result: an actor-critic approach that remains effective when the number of channels and the action space grow, in a setting where DQN degrades sharply. The paper's strengths include a clear problem formulation, standard and correctly stated actor-critic update equations, a decentralized multi-agent extension that requires no information exchange, and a comparison against several independent baselines including an optimal policy. The time-efficiency argument in Section VI-E is also reasonable and follows from avoiding minibatch replay. However, the comparative empirical claims currently rest on single-run curves and a DQN baseline that is not retuned for larger channel counts, so the significance is conditional on additional statistical evidence.

major comments (3)
  1. [Section VI-A, Fig. 4] The DQN baseline is taken from [7] with a replay memory of 1,000,000 and minibatch of 32, with no retuning for N=32 and N=64, and the paper does not state whether a target network is used. The headline claim that actor-critic outperforms DQN at 32 and 64 channels depends entirely on the DQN curves in Fig. 4(b)-(c). Without either retuning the DQN for each problem size or explicitly justifying the fixed configuration, the comparison may reflect a poorly configured baseline rather than a scalability advantage of actor-critic. Please retune the DQN baseline for each N, state clearly whether a target network is employed, and report the corresponding DQN learning curves.
  2. [Sections VI-B, VI-C, VI-D; Figs. 4, 8, 9, 14; Tables I-IV] All reported learning curves and distribution tables appear to come from a single run, or a single 500-time-slot window for Tables I-IV, with no error bars, seed counts, or variance information. Because deep RL training is stochastic, a single trajectory does not support strong comparative statements such as "the DQN framework cannot handle 64 channels" in Section VI-B1. Please run multiple independent seeds for each configuration and report means with confidence intervals or standard deviations, and where feasible a pairwise significance test. Without this, the empirical advantage of the proposed framework is not statistically established.
  3. [Sections V-A and VI-A, Fig. 8] The action space is defined as D = C(N,k) discrete actions in Section III-B3, but Section VI-A states that the actor's output layer has N neurons with softmax. The algorithm text also says the agent scores all actions in the action space and selects the highest score, which requires D output units. If in fact the actor outputs per-channel scores and the k largest are selected, that decision rule should be stated explicitly and the policy-gradient expressions in (25)-(26) should be reconciled with it. This matters because the multiple-good-channel experiments in Fig. 8 are used to support the claim that the framework handles large action spaces.
minor comments (5)
  1. [Algorithm 2] In the actor update line, the quantity "δπ θj,t" is undefined; it should presumably be the TD error δj,t.
  2. [Section VI-B1] There is a typo: "when the nunber of channels is increased" should read "number."
  3. [Fig. 4] The axis labels in the submitted version are garbled (e.g., "P obability" and "Ave age Rewa d"); these should be replaced with clean labels.
  4. [Equations (11), (15), (19), (20)] The symbol R is used both for the long-term average reward and for the per-slot reward; please distinguish these, for instance by using R_t for the per-slot reward.
  5. [Section VI-A, Whittle index heuristic] The Whittle index baseline is allowed to observe channels separately for 10,000 iterations to estimate transition probabilities; this gives it privileged information relative to the online learners, and the comparison would benefit from stating this explicitly when interpreting its low performance.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's contribution is an empirical actor-critic framework tested against independent baselines in synthetic environments, with no fitted quantity renamed as a prediction.

full rationale

The paper makes no claims that resemble circular derivation. The actor-critic algorithm is a standard RL method applied to a POMDP formulation; the objective and rewards in Section IV are defined from the environment, not from the method's outputs. The central comparative claims are empirical: simulation results in Section VI compare actor-critic against DQN from reference [7], random access, Whittle index heuristic, and an optimal policy from reference [7, Theorem 1]. These baselines are independent external work by other authors, not self-citations. No parameter is fitted to a subset of results and then presented as a prediction; all curves are measured simulation outcomes in synthetic channel environments. The runtime advantage in Section VI-E is argued from the algorithm's minibatch-free update, which is a structural property rather than a circular restatement of the measured result. The only self-reference is the note that the material was presented in part at GlobalSIP 2018, which is not load-bearing for any derivation. Potential concerns about DQN hyperparameter tuning or missing error bars are correctness or reproducibility issues, not circularity. Therefore the derivation chain is self-contained and no circular steps are present.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claims depend on a synthetic Markov channel model, perfect feedback, and hand-set hyperparameters. The performance comparisons are simulations, so the environment model and hyperparameter choices are the main unproved inputs that a reader must accept.

free parameters (7)
  • Actor learning rate alpha = 0.0001
    Hand-set in Section VI-A; controls the policy update step in Eq. (26) and affects convergence and final reward.
  • Critic learning rate = 0.0005
    Hand-set in Section VI-A; set larger than the actor learning rate so the critic converges faster.
  • Learning rate decay rate = 0.95 per 250,000 slots
    Exponential decay schedule in Section VI-A chosen for stability; the decay time constant is arbitrary.
  • Hidden layer width = 200 neurons
    Both actor and critic first layers use 200 ReLU neurons (Section VI-A); no ablation or justification is given.
  • Observation window Omega = not stated (16 in Fig. 1)
    The state fed to the networks is the last Omega observations (Section V-A); the paper does not report the value used in each experiment.
  • Discount factor gamma = not stated
    Defined in Eq. (23) as in (0,1) but no numerical value is given; it affects all TD-error updates.
  • DQN minibatch size and replay memory = 32 and 1,000,000
    Baseline hyperparameters taken from [7] (Section VI-B); they are not retuned for N=32/64, which affects the fairness of the comparison.
assumptions (4)
  • domain assumption Channel states evolve as a Markov chain where from any state the probability of leaving to a different state is p and remaining is 1-p
    Assumed in Section III-A to generate the correlated channel switching patterns; the optimal policy baseline in Section VI-B also relies on this structure.
  • domain assumption A user that selects a channel observes its state exactly, with no sensing error or delay
    Equations (2) and (7) define the observation as the true state if selected and zero otherwise; this perfect-feedback assumption is used throughout the simulations.
  • domain assumption Users in the multi-user scenario cannot exchange information about selections or observations
    Stated in Section III-B2; this makes the multi-agent framework decentralized and is necessary for the collision-avoidance learning claim.
  • ad hoc to paper The discount factor d_i,t for a collided good channel is proportional to 1/m, where m is the number of users selecting that channel
    Introduced in Eq. (4) with an SINR-based justification in Section III-B2; it is a modeling choice specific to this paper that shapes the multi-user reward and the agents' collision-avoidance behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access." pith.science (2026). https://pith.science/paper/LT35SZVO

@misc{pith2026190808401,
  author       = {Pith},
  title        = {Pith review of: A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LT35SZVO}},
  note         = {Machine review of arXiv:1908.08401}
}
read the original abstract

To make efficient use of limited spectral resources, we in this work propose a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user case and a scenario in which multiple users attempt to access channels simultaneously. We employ the proposed framework as a single agent in the single-user case, and extend it to a decentralized multi-agent framework in the multi-user scenario. In both cases, we develop algorithms for the actor-critic deep reinforcement learning and evaluate the proposed learning policies via experiments and numerical results. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework's tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. We also address a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons (in terms of both the average reward and time efficiency) between the proposed actor-critic deep reinforcement learning framework, Deep-Q network (DQN) based approach, random access, and the optimal policy when the channel dynamics are known.

Figures

Figures reproduced from arXiv: 1908.08401 by the authors.

Figure 1
Figure 1. In reinforcement learning, the agent constantly obs [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. Structure of the actor-critic deep reinforcement le [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Round-robin switching pattern when only one of the 32 [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Average reward vs. switching probability. We consid [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: A switching pattern when only one of the 32 channels is [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: The average reward for different arbitrary switchin [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: A switching pattern when each four channels of the 32 c [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Average sum of reward vs. number of channels that can b [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Average reward vs. number of users decentralized multi-user channel selection scenario. 2) Multi-User Scenario with Priorities: Now, we address the multi-user case where there are 3 users and 16 channels, and assume that one of these three users has higher priority tha…
Figure 10
Figure 10. Figure 10: The channel selection results based on the decentra [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: The channel selection results based on the decentra [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]
Figure 12
Figure 12. Figure 12: Channel selection results based on decentralized a [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]
Figure 13
Figure 13. Figure 13: Channel selection results based on decentralized D [PITH_FULL_IMAGE:figures/full_fig_p026_13.png]
Figure 14
Figure 14. Figure 14: The re-training process in a time-varying environm [PITH_FULL_IMAGE:figures/full_fig_p027_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 33 canonical work pages

  1. [7]

    Deep reinforcement learning for dynamic multichannel access in wireless networks,

    S. Wang, H. Liu, P . H. Gomes, and B. Krishnamachari, “Deep reinforcement learning for dynamic multichannel access in wireless networks,” IEEE Transactions on Cognitive Communications and Network ing, vol. 4, pp. 257–265, June 2018

    work page 2018

  2. [1]

    Heuristic search value iterati on for POMDPs,

    T. Smith and R. Simmons, “Heuristic search value iterati on for POMDPs,” in Proceedings of the 20th conference on Uncertainty in artificial intelligence , pp. 520–527, AUAI Press, 2004

    work page 2004

  3. [2]

    MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPs

    D. Szer, F. Charpillet, and S. Zilberstein, “Maa*: A heur istic search algorithm for solving decentralized POMDPs,” arXiv preprint arXiv:1207.1359, 2012

    work page Pith review arXiv 2012

  4. [3]

    Monte-carlo planning in large P OMDPs,

    D. Silver and J. V eness, “Monte-carlo planning in large P OMDPs,” in Advances in neural information processing systems , pp. 2164– 2172, 2010

    work page 2010

  5. [4]

    Monte carlo POMDPs,

    S. Thrun, “Monte carlo POMDPs,” in Advances in neural information processing systems , pp. 1064–1070, 2000

    work page 2000

  6. [5]

    Natural actor -critic,

    J. Peters, S. Vijayakumar, and S. Schaal, “Natural actor -critic,” Neurocomputing, vol. 71, pp. 1180–1190, 2005

    work page 2005

  7. [6]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction , vol. 1. MIT press Cambridge, 1998

    work page 1998

  8. [8]

    Full spectrum sharing in cogni tive radio networks toward 5G: A survey,

    F. Hu, B. Chen, and K. Zhu, “Full spectrum sharing in cogni tive radio networks toward 5G: A survey,” IEEE Access , vol. 6, pp. 15754–15776, 2018

    work page 2018

Show all 35 references
  1. [9]

    Decentralized co gnitive MAC for opportunistic spectrum access in ad hoc netw orks: A POMDP framework,

    Q. Zhao, L. Tong, A. Swami, and Y . Chen, “Decentralized co gnitive MAC for opportunistic spectrum access in ad hoc netw orks: A POMDP framework,” IEEE Journal on selected areas in communications , vol. 25, no. 3, 2007

    2007

  2. [10]

    A restless bandit formulation of opp ortunistic access: Indexablity and index policy,

    K. Liu and Q. Zhao, “A restless bandit formulation of opp ortunistic access: Indexablity and index policy,” in Sensor , Mesh and Ad Hoc Communications and Networks W orkshops, 2008. SECON W orkshops’ 08. 5th IEEE Annual Communications Society Confere nce on, pp. 1–5, IEEE, 200...

    2008

  3. [11]

    On myopic sensi ng for multi-channel opportunistic access: structure, opt imality, and performance,

    Q. Zhao, B. Krishnamachari, and K. Liu, “On myopic sensi ng for multi-channel opportunistic access: structure, opt imality, and performance,” IEEE Transactions on Wireless Communications , vol. 7, no. 12, 2008

    2008

  4. [12]

    Optimality of myopic sensing in multichannel oppo rtunistic access,

    S. H. A. Ahmad, M. Liu, T. Javidi, Q. Zhao, and B. Krishnam achari, “Optimality of myopic sensing in multichannel oppo rtunistic access,” IEEE Transactions on Information Theory , vol. 55, no. 9, pp. 4040–4050, 2009

    2009

  5. [13]

    Oppor tunistic spectrum access in unknown dynamic environment: A game-theoretic stochastic learning solution,

    Y . Xu, J. Wang, Q. Wu, A. Anpalagan, and Y .-D. Yao, “Oppor tunistic spectrum access in unknown dynamic environment: A game-theoretic stochastic learning solution,” IEEE transactions on wireless communications , vol. 11, no. 4, pp. 1380–1391, 2012

    2012

  6. [14]

    Stochastic ga me-theoretic spectrum access in distributed and dynamic en vironment,

    J. Zheng, Y . Cai, N. Lu, Y . Xu, and X. Shen, “Stochastic ga me-theoretic spectrum access in distributed and dynamic en vironment,” IEEE transactions on vehicular technology , vol. 64, no. 10, pp. 4807–4820, 2015

    2015

  7. [15]

    Optimally probing cha nnel in opportunistic spectrum access,

    K. Wang, Q. Liu, Q. Fan, and Q. Ai, “Optimally probing cha nnel in opportunistic spectrum access,” IEEE Communications Letters , 2017

    2017

  8. [16]

    Human-level control through deep reinforcement learnin g,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. V eness , M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. , “Human-level control through deep reinforcement learnin g,” Nature, vol. 518, no. 7540, p. 529, 2015

    2015

  9. [17]

    Mastering the game of go without human knowledge,

    D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglo u, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al., “Mastering the game of go without human knowledge,” Nature, vol. 550, no. 7676, p. 354, 2017

    2017

  10. [18]

    Applications of deep reinforcement l earning in communications and networking: A survey,

    N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P . Wang, Y .-C . Liang, and D. I. Kim, “Applications of deep reinforcement l earning in communications and networking: A survey,” arXiv preprint arXiv:1810.07862 , 2018

    2018 arXiv

  11. [19]

    Deep learning in mo bile and wireless networking: A survey,

    C. Zhang, P . Patras, and H. Haddadi, “Deep learning in mo bile and wireless networking: A survey,” arXiv preprint arXiv:1803.04311, 2018

    2018 arXiv

  12. [20]

    Reinforcem ent learning-based NOMA power allocation in the presence of smart jamming,

    L. Xiao, Y . Li, C. Dai, H. Dai, and H. V . Poor, “Reinforcem ent learning-based NOMA power allocation in the presence of smart jamming,” IEEE Transactions on V ehicular Technology, vol. 67, no. 4, pp. 3377–3389, 2018

    2018

  13. [21]

    Re inforcement learning for energy harvesting decode-and-fo rward two-hop communications,

    A. Ortiz, H. Al-Shatri, X. Li, T. Weber, and A. Klein, “Re inforcement learning for energy harvesting decode-and-fo rward two-hop communications,” IEEE Transactions on Green Communications and Networking , vol. 1, no. 3, pp. 309–319, 2017

    2017

  14. [22]

    Deep q-learning based dynamic resource allocation for self-powe red ultra-dense networks,

    H. Li, H. Gao, T. Lv, and Y . Lu, “Deep q-learning based dynamic resource allocation for self-powe red ultra-dense networks,” in 2018 IEEE International Conference on Communications W ork shops (ICC W orkshops), pp. 1–6, IEEE, 2018

    2018

  15. [23]

    A reinforcement learning- based resource allocation scheme for cloud robotics,

    H. Liu, S. Liu, and K. Zheng, “A reinforcement learning- based resource allocation scheme for cloud robotics,” IEEE Access , vol. 6, pp. 17215–17222, 2018

    2018

  16. [24]

    Deep reinforcement learning for reso urce allocation in v2v communications,

    H. Ye and G. Y . Li, “Deep reinforcement learning for reso urce allocation in v2v communications,” in 2018 IEEE International Conference on Communications (ICC) , pp. 1–6, IEEE, 2018

    2018

  17. [25]

    User scheduling and resource allocation in hetnets with hybrid energy supply: A n actor-critic reinforcement learning approach,

    Y . Wei, F. R. Y u, M. Song, and Z. Han, “User scheduling and resource allocation in hetnets with hybrid energy supply: A n actor-critic reinforcement learning approach,” IEEE Transactions on Wireless Communications , vol. 17, no. 1, pp. 680–692, 2018

    2018

  18. [26]

    Reinforcement-learning-b ased resource allocation in fog radio access networks for va rious iot environments,

    A. T. Nassar and Y . Yilmaz, “Reinforcement-learning-b ased resource allocation in fog radio access networks for va rious iot environments,” arXiv preprint arXiv:1806.04582 , 2018

    2018 arXiv

  19. [27]

    Deep-reinforcement lear ning multiple access for heterogeneous wireless networks,

    Y . Y u, T. Wang, and S. C. Liew, “Deep-reinforcement lear ning multiple access for heterogeneous wireless networks, ” in 2018 IEEE International Conference on Communications (ICC) , pp. 1–7, IEEE, 2018

    2018

  20. [28]

    Online learning for multi-channel opportunistic access over unknown Marko vian channels,

    W. Dai, Y . Gai, and B. Krishnamachari, “Online learning for multi-channel opportunistic access over unknown Marko vian channels,” in Sensing, Communication, and Networking (SECON), 2014 Elev enth Annual IEEE International Conference on , pp. 64–71, IEEE, 2014

    2014

  21. [29]

    Model free dynam ic sensing order selection for imperfect sensing multichan nel cognitive radio networks: A Q-learning approach,

    Y . Zhang, Q. Zhang, B. Cao, and P . Chen, “Model free dynam ic sensing order selection for imperfect sensing multichan nel cognitive radio networks: A Q-learning approach,” in Communication Systems (ICCS), 2014 IEEE International Con ference on, pp. 364–368, IEEE, 2014

    2014

  22. [30]

    Deep multi-user reinforcem ent learning for distributed dynamic spectrum access,

    O. Naparstek and K. Cohen, “Deep multi-user reinforcem ent learning for distributed dynamic spectrum access,” IEEE Transactions on Wireless Communications , vol. 18, no. 1, pp. 310–323, 2019

    2019

  23. [31]

    Deep reinforcement learning based dynamic channel allocation algorithm in multibeam sa tellite systems,

    S. Liu, X. Hu, and W. Wang, “Deep reinforcement learning based dynamic channel allocation algorithm in multibeam sa tellite systems,” IEEE ACCESS , vol. 6, pp. 15733–15742, 2018

    2018

  24. [32]

    Multiagent Q-learning for aloha-like spectrum access in cognitive radio systems,

    H. Li, “Multiagent Q-learning for aloha-like spectrum access in cognitive radio systems,” EURASIP Journal on Wireless Communications and Networking , vol. 2010, p. 56, 2010

    2010

  25. [33]

    Distribu ted reinforcement learning based MAC protocols for autonom ous cognitive secondary users,

    M. Bkassiny, S. K. Jayaweera, and K. A. Avery, “Distribu ted reinforcement learning based MAC protocols for autonom ous cognitive secondary users,” in Wireless and Optical Communications Conference (WOCC), 20 11 20th Annual , pp. 1–6, IEEE, 2011

    2011

  26. [34]

    Enhancing network performance in distributed cognitive radio networ ks using single- agent and multi-agent reinforcement learning,

    K.-L. A. Yau, P . Komisarczuk, and D. T. Paul, “Enhancing network performance in distributed cognitive radio networ ks using single- agent and multi-agent reinforcement learning,” in Local Computer Networks (LCN), 2010 IEEE 35th Conference on , pp. 152–159, IEEE, 2010

    2010

  27. [35]

    Indexability of restless bandit pro blems and optimality of Whittle index for dynamic multichan nel access,

    K. Liu and Q. Zhao, “Indexability of restless bandit pro blems and optimality of Whittle index for dynamic multichan nel access,” IEEE Transactions on Information Theory , vol. 56, no. 11, pp. 5547–5567, 2010. DRAFT

    2010

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.