Pith. sign in

REVIEW 5 major objections 4 minor 3 cited by

An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management

T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Offline multi-agent reinforcement learning can learn wireless scheduling policies from a static dataset and beat classic baselines by over 15%.

desk verdict Useful and re-implementable offline MARL for RRM, but the headline gains rest on an unvalidated PF surrogate and under-reported statistics. read the letter →

arxiv 2501.12991 v1 pith:DSGSV2A7 submitted 2025-01-22 cs.MA cs.LG

Eslam Eldeeb , Hirley Alves This is my paper · ORCID
classification cs.MAcs.LG
keywords offlinemulti-agentreinforcementlearningconservativeQ-learningradioresourcemanagementsoftactor-criticcentralizedtrainingdecentralizedexecutionproportionalfairnesstailrateoptimizationwirelessscheduling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that offline multi-agent reinforcement learning, trained entirely from a fixed dataset with no live interaction with the network, can learn good multi-access-point scheduling policies for radio resource management. Using a conservative Q-learning regularizer built on a soft actor-critic base, it trains centralized, independent, and CTDE variants and reports that all three outperform random-walk, greedy, time-division multiplexing, and ITLinQ baselines in a simulated cellular downlink. The central promise is that the safety, signaling overhead, and data-collection costs of online MARL can be sidestepped without sacrificing much performance, with the CTDE variant claimed as the best complexity-performance trade-off. If true, this makes offline MARL a practical tool for 6G resource management, where online exploration is expensive or unsafe.

What carries the argument

The central object is the offline MARL framework that adds a conservative Q-learning (CQL) regularization term to the soft actor-critic (SAC) loss, applied in three training paradigms: centralized (C-MARL-CQL), independent (I-MARL-CQL), and centralized-training-decentralized-execution (CTDE-MARL-CQL). The CQL term penalizes out-of-distribution actions and prevents overestimation from offline data, while SAC provides stable entropy-regularized policy optimization. The reward uses the proportional-fairness surrogate, r = sum_j w_j^$\lambda$ C_j, with running-average weights w_j, making the tail-rate objective tractable, and in CTDE the global Q-function is decomposed as the sum of per-agent Q-functions to enable coordination during training with decentralized execution.

What would settle it

Evaluate a CTDE-MARL-CQL policy learned with the proportional-fairness surrogate directly under the true objective Cscore = mu1 * Csum + mu2 * C5% as defined in equation (11) on the same simulated network; if the resulting Cscore is not at least 15% above the ITLinQ baseline, the central performance claim fails.

Watch

Extended reading notes

Core claim

The paper proposes offline MARL algorithms, C-MARL-CQL, I-MARL-CQL, and CTDE-MARL-CQL, built on soft actor-critic and conservative Q-learning for a partially observable MDP formulation of downlink scheduling with joint sum-rate and 5-percentile-rate objectives. Simulation results show that these offline schemes outperform random-walk, greedy, TDM, and ITLinQ by over 15% in Rscore, the weighted combination of sum and 5-percentile rates, and that the CTDE variant approaches the centralized performance at lower execution complexity. The paper also shows that dataset quality and size directly control offline performance, with high-quality behavioral datasets and sufficiently large sizes yielding the best policies.

Load-bearing premise

The proportional-fairness surrogate reward, with running-average weights and lambda = 0.8, is assumed to faithfully approximate the true goal of maximizing the weighted sum of sum rate and 5-percentile tail rate; the paper does not prove or test this equivalence.

Editorial extensions

If this is right

  • Static datasets collected by an online SAC agent suffice to train scheduling policies that outperform classic baselines, so deployment could avoid unsafe or costly live exploration.
  • CTDE-MARL-CQL achieves near-centralized performance while requiring only per-agent execution, suggesting that value-sharing during offline training recovers coordination without a central controller at runtime.
  • Offline performance depends strongly on dataset quality and size; good-quality behavioral datasets produce high Rscore, while small datasets collapse toward TDM-level performance.
  • The framework extends prior single-agent offline RL results for radio resource management to a genuinely multi-agent setting with multiple access points.
  • Mixing SAC and DQN datasets still yields high scores, indicating tolerance to diverse data sources in offline training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the reported Rscore is the proportional-fairness surrogate rather than the true weighted sum of sum rate and 5-percentile rate, a direct evaluation of the learned policies under Cscore as defined in equation (11) would be needed to confirm that the tail-rate objective truly improves.
  • The surrogate's validity may be sensitive to the choice of lambda = 0.8 and the running-average parameter eta; the paper gives no sensitivity analysis, so transfer to different networks or fairness weights remains untested.
  • A natural testable extension is to deploy the offline-learned policies in a standard system-level simulator (e.g., a 3GPP-compliant model) where the true 5-percentile rate is directly measurable, which would separate the contribution of the surrogate from the contribution of the offline learning machinery.
  • The same offline MARL recipe could plausibly be applied to other radio resource management objectives, such as age-of-information or energy efficiency, provided the reward can be rewritten as a decomposable surrogate, but the paper does not demonstrate this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes an offline MARL framework for radio resource management, built on SAC with a CQL conservative penalty, and evaluates three training paradigms (centralized, independent, and CTDE). The setting is a multi-AP downlink system with moving UEs, and the stated goal is to maximize a weighted combination of sum rate and 5-percentile tail rate. The authors compare their offline variants against random-walk, greedy, TDM, and ITLinQ baselines, and report Rsum, Rperc, and Rscore learning curves. The abstract claims over 15% improvement in a weighted combination of sum and tail rates, while the contributions section claims more than 50% gain.

Significance. If the central claim is validated, the paper would make a useful contribution by showing that a static offline dataset can train multi-AP scheduling policies that beat standard heuristics, and by comparing centralized, independent, and CTDE training under an offline constraint. The paper also adapts CQL to three MARL paradigms in a clean way, which is a reasonable technical contribution. However, the empirical evidence as presented is not yet load-bearing: the main reported metric is never defined, the relationship between the optimized surrogate reward and the stated objective is asserted from a prior reference rather than demonstrated, dataset sizes are inconsistent, and no statistical error bars are provided. These issues prevent the current version from substantiating the headline performance claims.

major comments (5)
  1. [Section II-A and Section V-A] The central reported metric Rscore, plotted in Figs. 4-7, is never defined, and it is not established that Rscore equals the Cscore of Eq. (11). The reward actually optimized in Eq. (16) is the proportional-fairness surrogate sum_j w_j^lambda C_j, not the linear combination mu1*Csum + mu2*C5% of Eq. (11). The claim in Section II-A that optimizing the PF surrogate 'directly influences' Cscore is imported from [26] without derivation, and no sensitivity analysis is provided for lambda (set to 0.8 in Table III). Since C5% in Eq. (10) is a nonlinear quantile functional, the equivalence is not self-evident, and the reported gains may be for a different objective. Please define Rsum, Rperc, and Rscore explicitly, report Csum and C5% directly, and add a sensitivity analysis over lambda and the mu weights.
  2. [Section V-C and Section V-E] The dataset size used for the main offline result is inconsistent with the dataset-size experiment. Section V-C states that the centralized offline training uses a dataset of 16000 data points, while Section V-E and Fig. 7b list the smallest dataset as 20000 points. Moreover, it is unclear whether these counts refer to environment transitions or to episodes; with T=200 time steps per episode, 16000 transitions would correspond to only 80 episodes. This matters for interpreting the offline RL claim and the claimed dependence on dataset size. Please reconcile these numbers and state the unit (transitions or episodes) explicitly.
  3. [Section V (Figs. 4-7)] All reported comparisons are single learning curves with no random seeds, error bars, or confidence intervals. MARL training, especially with SAC and CQL, is stochastic in initialization, exploration, and minibatch sampling, so the observed qualitative ordering among C-MARL-CQL, CTDE-MARL-CQL, and I-MARL-CQL may not be significant. The claim that CTDE 'approaches' the performance of C-MARL (Fig. 6c) and the claim that offline variants outperform online C-MARL (Fig. 5c) need to be supported by multiple seeds and a measure of variability, or by a table of mean and standard deviation over seeds.
  4. [Section V-B and Section V-C] The offline dataset used for the main comparison is collected from the online C-MARL (SAC) agent, and the same online C-MARL (SAC) agent is then used as a benchmark in Figs. 5 and 6. This creates a favorable information asymmetry: the offline method is trained on the experience of the online agent, while the online benchmark is evaluated from scratch. This does not invalidate the comparison against RW, greedy, TDM, and ITLinQ, but it weakens the claim that offline MARL 'outperforms' online MARL. The paper should either use datasets from multiple behavior policies in the main comparison, or explicitly frame the online comparison as a dataset-quality study rather than a head-to-head performance claim.
  5. [Abstract and Section I-C] The two headline improvement numbers are inconsistent: the abstract claims 'over a 15% improvement', while Section I-C claims 'more than 50% gain regarding the linear combination of sum and tail rates'. The percentage gain also depends on which baseline and which operating point are used, but no such details are given. Please state the exact metric, the baseline, the dataset size, and the evaluation point for any percentage improvement claim, and reconcile the 15% and 50% figures.
minor comments (4)
  1. [Section II, paragraph after Eq. (5)] The sentence 'Then, each time t, each user selects one of its associated UEs to serve' appears to have a typo: it should be 'each AP selects one of its associated UEs to serve.'
  2. [Section III-B, Eqs. (17)-(18)] The expectation notation in the SAC losses is imprecise: Eq. (18) writes E_{a~pi_k(s|a)} but should involve sampling states from the replay buffer and actions from the policy, and the argmax in Eq. (25) would benefit from a clarifying comment about tie-breaking. These are presentation issues, but they make the algorithmic description harder to reproduce.
  3. [Table III] The parameter 'P Lo' (listed as 10 dB) is not defined in the text or in the symbol table; it appears to be related to shadowing or path-loss offset but should be named explicitly. Also, the CQL penalty alpha is fixed at 1 without any ablation, although alpha is a key hyperparameter for offline RL performance.
  4. [References] References [36] and [37] are the same paper (Haarnoja et al., SAC); one of them should be removed or replaced with a different citation.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular steps: the central comparisons are against external baselines and external CQL/SAC components; self-citations are not load-bearing.

full rationale

The paper defines Cscore in Eq. 11 and then replaces optimization of Cscore by the PF surrogate in Eq. 15, imported from reference [26], and uses the per-step form of Eq. 15 as the MARL reward in Eq. 16. This is a proxy substitution with an unvalidated link to the tail-rate objective, which is a correctness/sensitivity concern, not a circular reduction: the reward is not defined in terms of the reported Rscore, and the reported Rsum, Rperc, and Rscore curves are not shown to equal Csum, C5%, and Cscore. The offline dataset is collected from the same online C-MARL (SAC) agent, and later comparisons include that online agent as a benchmark, which is a favorable but not forced setup because the CQL conservative penalty can move the learned policy away from the behavioral policy. The main baselines (random-walk, greedy, TDM, ITLinQ) are external and independent of the proposed method. The load-bearing algorithmic components, CQL [19], SAC [36], value decomposition [38], and the PF surrogate [26], are all external citations rather than self-citations. Self-citations [30] and [32] are related-work descriptions and do not justify the central claim. Therefore no specific circular step can be exhibited; the derivation is self-contained apart from the unproven surrogate-objective equivalence, which should be treated as correctness risk rather than circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

All quantities in the ledger are standard RL components or hand-chosen hyperparameters; no new physical entities are introduced. The central empirical claim depends mainly on the PF surrogate, the CQL penalty strength, and the quality of the offline dataset.

free parameters (5)
  • lambda (fairness exponent) = 0.8
    Chosen by hand in Table III; controls the trade-off between sum rate and 5-percentile rate in objective (15). No sensitivity analysis is reported.
  • alpha (CQL penalty) = 1
    Chosen by hand in Table III; controls strength of conservative regularization in (26)-(28). No ablation is reported.
  • eta (running-average parameter) = not reported
    Required to compute the long-term average rate in (13), but its value is missing from Table III and the text.
  • mu1, mu2 (score weights) = 1/M and 3
    User-chosen weights in the evaluation metric Cscore (11); M is not defined. These weights define what the 15% or 50% gain means.
  • N (top users per AP) = 3
    Action space is truncated to the top N PF users per AP in Section II-A; chosen by hand, affects state and action dimensions and fairness.
assumptions (4)
  • domain assumption The RRM scheduling problem can be modeled as a PO-MDP where each AP is an agent with local observations of SINR and PF weights of its top N UEs and a shared reward (16).
    Invoked in Section III-A; standard MARL formulation but assumes the local observation contains enough information for near-optimal scheduling.
  • domain assumption The PF weighted sum objective (15) with recursively smoothed rates (13) is a valid surrogate for maximizing the 5-percentile tail rate C5% in (10).
    Adopted from [26] in Section II-A without proof or sensitivity analysis; the whole training reward depends on this proxy.
  • domain assumption CQL's log-sum-exp penalty transfers from single-agent offline RL to multi-agent settings without modification.
    Equations (26)-(28) assume conservative regularization on the joint or individual Q-functions controls OOD actions in MARL; no multi-agent theory is provided.
  • domain assumption The static dataset collected from an online C-MARL (SAC) policy is a faithful behavioral dataset for the deployment environment.
    Section V-B; no coverage or distribution-shift analysis is given beyond dataset-size experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management." pith.science (2026). https://pith.science/paper/DSGSV2A7

@misc{pith2026250112991,
  author       = {Pith},
  title        = {Pith review of: An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DSGSV2A7}},
  note         = {Machine review of arXiv:2501.12991}
}
read the original abstract

Offline multi-agent reinforcement learning (MARL) addresses key limitations of online MARL, such as safety concerns, expensive data collection, extended training intervals, and high signaling overhead caused by online interactions with the environment. In this work, we propose an offline MARL algorithm for radio resource management (RRM), focusing on optimizing scheduling policies for multiple access points (APs) to jointly maximize the sum and tail rates of user equipment (UEs). We evaluate three training paradigms: centralized, independent, and centralized training with decentralized execution (CTDE). Our simulation results demonstrate that the proposed offline MARL framework outperforms conventional baseline approaches, achieving over a 15\% improvement in a weighted combination of sum and tail rates. Additionally, the CTDE framework strikes an effective balance, reducing the computational complexity of centralized methods while addressing the inefficiencies of independent training. These results underscore the potential of offline MARL to deliver scalable, robust, and efficient solutions for resource management in dynamic wireless networks.

Figures

Figures reproduced from arXiv: 2501.12991 by the authors.

Figure 1
Figure 1. A wireless environment consists of I APs and J UEs. Each UE is associated with only one AP, which chooses one of its associated UEs to serve at a time. Then, each time t, each user selects one of its associated UEs to serve. To this end, the instantaneous rate and SINR of UE j that is associated with AP i are, respectively, Cj (t) = log2 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. An illustrative comparison between centralized MARL, independent MARL, and centralized training decentralized execution MARL. As shown, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. An illustrative comparison between online MARL and offline MARL. Online MARL utilizes online interaction with the environment to optimize the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The sum rate, 5-percentile rate, and Rscore reported for the proposed C-MARL-CQL algorithm built on top of SAC and DQN compared to C￾MARL and other benchmark schemes. compare the developed C-MARL (SAC) scheme to a famous online RL algorithm, C-MARL (DQN) scheme, as a l…
Figure 6
Figure 6. Figure 6: The sum rate, 5-percentile rate, and Rscore reported for the proposed C-MARL-CQL, I-MARL-CQL and CTDE-MARL-CQL built on top of SAC architecture. E. Dataset Quality Finally, we show the effect of the quality of the dataset and its size on the Rscore performance of the p…
Figure 7
Figure 7. Figure 7: The effect of the dataset on the overall performance of the proposed [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DRL-Based Spectrum Sharing for RIS-Aided Local High-Quality Wireless Networks

    eess.SY 2026-03 conditional novelty 4.0 of 10

    SAC-based deep reinforcement learning jointly allocates subchannels, power, and RIS phases in a multi-operator spectrum-sharing setting, outperforming DDPG and approaching an exhaustive-search benchmark in simulation.

  2. Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning

    cs.RO 2025-02 conditional novelty 4.0 of 10

    A hybrid meta-offline reinforcement learning algorithm trains a UAV to minimize data age and transmission power from static datasets and adapts to new tasks in under 40 epochs.

  3. Federated Multi-Agent Reinforcement Learning for Privacy-Preserving and Energy-Aware Resource Management in 6G Edge Networks

    cs.LG 2025-09 reject novelty 3.0 of 10

    FERMI-6G, a federated multi-agent DRQN framework with secure aggregation, reportedly improves latency, energy, reliability, and fairness over centralized and heuristic baselines in a simulated 6G edge network.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages · cited by 3 Pith papers

  1. [26]

    Resource management in wireless networks via multi-agent deep reinforcement learning,

    N. Naderializadeh, J. J. Sydir, M. Simsek, and H. Nikopour, “Resource management in wireless networks via multi-agent deep reinforcement learning,” IEEE Transactions on Wireless Communications , vol. 20, no. 6, pp. 3507–3523, 2021

    work page 2021

  2. [31]

    Offline reinforcement learning for wireless network optimization with mixture datasets,

    K. Yang, C. Shi, C. Shen, J. Yang, S.-p. Yeh, and J. J. Sydir, “Offline reinforcement learning for wireless network optimization with mixture datasets,” IEEE Transactions on Wireless Communications , pp. 1–1, 2024

    work page 2024

  3. [32]

    Offline and distributional reinforcement learning for radio resource management,

    E. Eldeeb and H. Alves, “Offline and distributional reinforcement learning for radio resource management,” 2024. [Online]. Available: https://arxiv.org/abs/2409.16764

    arXiv 2024

  4. [1]

    Applications of deep reinforcement learning in communications and networking: A survey,

    N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y .-C. Liang, and D. I. Kim, “Applications of deep reinforcement learning in communications and networking: A survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133–3174, 2019

    2019

  5. [2]

    Machine type communications: key drivers and enablers towards the 6G era,

    N. H. Mahmood, S. B ¨ocker, I. Moerman, O. A. L ´opez, A. Munari, K. Mikhaylov, F. Clazzer, H. Bartz, O.-S. Park, E. Mercier et al. , “Machine type communications: key drivers and enablers towards the 6G era,” EURASIP Journal on Wireless Communications and Networking , vol. 2021, no. 1, p. 134, 2021

    work page 2021

  6. [3]

    Machine learning for large-scale optimization in 6G wireless networks,

    Y . Shi, L. Lian, Y . Shi, Z. Wang, Y . Zhou, L. Fu, L. Bai, J. Zhang, and W. Zhang, “Machine learning for large-scale optimization in 6G wireless networks,” IEEE Communications Surveys & Tutorials , vol. 25, no. 4, pp. 2088–2132, 2023

    2023

  7. [4]

    Traffic prediction and fast uplink for hidden markov IoT models,

    E. Eldeeb, M. Shehab, A. E. Kalø r, P. Popovski, and H. Alves, “Traffic prediction and fast uplink for hidden markov IoT models,” IEEE Internet of Things Journal , vol. 9, no. 18, pp. 17 172–17 184, 2022

    2022

  8. [5]

    ITLinQ+: An improved spectrum sharing mech- anism for device-to-device communications,

    X. Yi and G. Caire, “ITLinQ+: An improved spectrum sharing mech- anism for device-to-device communications,” in 2015 49th Asilomar Conference on Signals, Systems and Computers , 2015, pp. 1310–1314

    work page 2015

Show all 40 references
  1. [6]

    Binary power control for sum rate maximization over multiple interfering links,

    A. Gjendemsjo, D. Gesbert, G. E. Oien, and S. G. Kiani, “Binary power control for sum rate maximization over multiple interfering links,” IEEE Transactions on Wireless Communications, vol. 7, no. 8, pp. 3164–3173, 2008

    2008

  2. [7]

    Game-theoretic resource allocation methods for device-to-device communication,

    L. Song, D. Niyato, Z. Han, and E. Hossain, “Game-theoretic resource allocation methods for device-to-device communication,” IEEE Wireless Communications, vol. 21, no. 3, pp. 136–144, 2014

    2014

  3. [8]

    Reinforcement learning for radio resource management in RAN slicing: A survey,

    M. Zangooei, N. Saha, M. Golkarifard, and R. Boutaba, “Reinforcement learning for radio resource management in RAN slicing: A survey,” IEEE Communications Magazine , vol. 61, no. 2, pp. 118–124, 2023

    2023

  4. [9]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al. , “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015. 12

    2015

  5. [10]

    Distributed learning methodologies for massive machine type commu- nication,

    M. V . Da Silva, E. Eldeeb, M. Shehab, H. Alves, and R. D. Souza, “Distributed learning methodologies for massive machine type commu- nication,” Authorea Preprints, 2024

    2024

  6. [11]

    A review of cooperative multi-agent deep reinforcement learning,

    A. Oroojlooy and D. Hajinezhad, “A review of cooperative multi-agent deep reinforcement learning,” Applied Intelligence, vol. 53, no. 11, pp. 13 677–13 722, 2023

    2023

  7. [12]

    Multiagent cooperation and competition with deep reinforcement learning,

    A. Tampuu, T. Matiisen, D. Kodelja, I. Kuzovkin, K. Korjus, J. Aru, J. Aru, and R. Vicente, “Multiagent cooperation and competition with deep reinforcement learning,” PloS one , vol. 12, no. 4, p. e0172395, 2017

    2017

  8. [13]

    Traffic learning and proactive UA V trajectory planning for data uplink in markovian IoT models,

    E. Eldeeb, M. Shehab, and H. Alves, “Traffic learning and proactive UA V trajectory planning for data uplink in markovian IoT models,”IEEE Internet of Things Journal , vol. 11, no. 8, pp. 13 496–13 508, 2024

    2024

  9. [14]

    S. V . Albrecht, F. Christianos, and L. Sch¨afer, Multi-Agent Reinforcement Learning: Foundations and Modern Approaches . MIT Press, 2024. [Online]. Available: https://www.marl-book.com

    2024

  10. [15]

    Value-decomposition networks for cooperative multi-agent learning,

    P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V . Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, and T. Graepel, “Value-decomposition networks for cooperative multi-agent learning,” 2017. [Online]. Available: https://arxiv.org/abs/1706.05296

    2017 arXiv

  11. [16]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems,

    S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” arXiv preprint arXiv:2005.01643, 2020

    2005 arXiv

  12. [17]

    Offline reinforcement learning with implicit Q-learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit Q-learning,” 2021. [Online]. Available: https: //arxiv.org/abs/2110.06169

    2021 arXiv

  13. [18]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International conference on machine learning. PMLR, 2015, pp. 1889–1897

    2015

  14. [19]

    Conservative Q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative Q-learning for offline reinforcement learning,” in Advances in Neural Information Processing Systems , vol. 33. Curran Associates, Inc., 2020, pp. 1179–1191. [Online]. Available: https://proceedings.neurips.cc/paper fi...

    2020

  15. [20]

    Self- organization in small cell networks: A reinforcement learning approach,

    M. Bennis, S. M. Perlaza, P. Blasco, Z. Han, and H. V . Poor, “Self- organization in small cell networks: A reinforcement learning approach,” IEEE Transactions on Wireless Communications , vol. 12, no. 7, pp. 3202–3212, 2013

    2013

  16. [21]

    Intelligent power control for spectrum sharing in cognitive radios: A deep rein- forcement learning approach,

    X. Li, J. Fang, W. Cheng, H. Duan, Z. Chen, and H. Li, “Intelligent power control for spectrum sharing in cognitive radios: A deep rein- forcement learning approach,” IEEE Access, vol. 6, pp. 25 463–25 473, 2018

    2018

  17. [22]

    Multi-UA V path learning for age and power optimization in IoT with UA V battery recharge,

    E. Eldeeb, J. M. de Souza Sant’Ana, D. E. P ´erez, M. Shehab, N. H. Mahmood, and H. Alves, “Multi-UA V path learning for age and power optimization in IoT with UA V battery recharge,” IEEE Transactions on Vehicular Technology, vol. 72, no. 4, pp. 5356–5360, 2022

    2022

  18. [23]

    GAN-Powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing,

    Y . Hua, R. Li, Z. Zhao, X. Chen, and H. Zhang, “GAN-Powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing,” IEEE Journal on Selected Areas in Communications , vol. 38, no. 2, pp. 334–349, 2020

    2020

  19. [24]

    Learning resilient radio resource management policies with graph neural networks,

    N. NaderiAlizadeh, M. Eisen, and A. Ribeiro, “Learning resilient radio resource management policies with graph neural networks,” IEEE Transactions on Signal Processing , vol. 71, pp. 995–1009, 2023

    2023

  20. [25]

    Age minimization in massive IoT via UA V swarm: A multi-agent reinforcement learning approach,

    E. Eldeeb, M. Shehab, and H. Alves, “Age minimization in massive IoT via UA V swarm: A multi-agent reinforcement learning approach,” in 2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) , 2023, pp. 1–6

    2023

  21. [27]

    Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,

    Y . S. Nasir and D. Guo, “Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,” IEEE Journal on Selected Areas in Communications , vol. 37, no. 10, pp. 2239–2250, 2019

    2019

  22. [28]

    Multi-agent deep reinforcement learning for distributed resource management in wirelessly powered communication networks,

    S. Hwang, H. Kim, H. Lee, and I. Lee, “Multi-agent deep reinforcement learning for distributed resource management in wirelessly powered communication networks,” IEEE Transactions on Vehicular Technology, vol. 69, no. 11, pp. 14 055–14 060, 2020

    2020

  23. [29]

    Multi-agent reinforcement learning for dynamic resource management in 6G in-X subnetworks,

    X. Du, T. Wang, Q. Feng, C. Ye, T. Tao, L. Wang, Y . Shi, and M. Chen, “Multi-agent reinforcement learning for dynamic resource management in 6G in-X subnetworks,” IEEE Transactions on Wireless Communications, vol. 22, no. 3, pp. 1900–1914, 2023

    1900

  24. [30]

    Con- servative and risk-aware offline multi-agent reinforcement learning for digital twins,

    E. Eldeeb, H. Sifaou, O. Simeone, M. Shehab, and H. Alves, “Con- servative and risk-aware offline multi-agent reinforcement learning for digital twins,” arXiv preprint arXiv:2402.08421 , 2024

    2024 arXiv

  25. [33]

    Offline pre-trained multi-agent decision transformer,

    L. Meng, M. Wen, C. Le, X. Li, D. Xing, W. Zhang, Y . Wen, H. Zhang, J. Wang, Y . Yang et al. , “Offline pre-trained multi-agent decision transformer,” Machine Intelligence Research , vol. 20, no. 2, pp. 233– 248, 2023

    2023

  26. [34]

    Simulation assumptions and parameters for FDD HeNB RF requirements,

    3GPP, “Simulation assumptions and parameters for FDD HeNB RF requirements,” Tech. Rep. R4-092042

  27. [35]

    NR; physical layer measurements,

    ——, “NR; physical layer measurements,” Technical specification (TS) 8.215 V18.4.0, 2024-12

    2024

  28. [36]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proceedings of the 35th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, vol. 80...

    2018

  29. [37]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    ——, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proceedings of the 35th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, vol. 80. PMLR, 10–15 Jul 2018, pp. 1861–1870. [Onli...

    2018

  30. [38]

    Value-decomposition multi-agent actor- critics,

    J. Su, S. Adams, and P. Beling, “Value-decomposition multi-agent actor- critics,” in Proceedings of the AAAI conference on artificial intelligence , vol. 35, no. 13, 2021, pp. 11 352–11 360

    2021

  31. [39]

    Off-policy deep reinforcement learning without exploration,

    S. Fujimoto, D. Meger, and D. Precup, “Off-policy deep reinforcement learning without exploration,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, vol. 97. PMLR, 09–15 Jun 2019, pp. 2052–2062. [Online]. ...

    2019

  32. [40]

    ITLinQ: A new approach for spectrum sharing in device-to-device communication systems,

    N. Naderializadeh and A. S. Avestimehr, “ITLinQ: A new approach for spectrum sharing in device-to-device communication systems,” in 2014 IEEE International Symposium on Information Theory , 2014, pp. 1573– 1577

    2014

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.