Pith. sign in

REVIEW 4 major objections 5 minor 32 references

Average Reward Reinforcement Learning for Wireless Radio Resource Management

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that the standard discounted-reward RL formulation is misaligned with wireless RRM objectives and shows that an average-reward version of SAC closes the gap with a 15% gain.

desk verdict The mismatch story is real, but the proposed ARO-SAC is not SAC, so the 15% gain is confounded. read the letter →

arxiv 2501.06700 v1 pith:SSYCKOKQ submitted 2025-01-12 cs.IT cs.LGcs.NIeess.SPmath.IT

Kun Yang , Jing Yang , Cong Shen This is my paper · ORCID
classification cs.ITcs.LGcs.NIeess.SPmath.IT
keywords averagerewardreinforcementlearningradioresourcemanagementRANslicingSoftActor-Criticdiscountedmismatchwirelessnetworkoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that standard reinforcement learning for wireless radio resource management maximizes discounted cumulative reward, while the actual network goal is undiscounted long-term average quality of service, and this mismatch costs performance. It shows empirically that increasing the discount factor or the episode horizon helps, but setting the discount factor to one causes unstable training. To close the gap, the paper introduces ARO-SAC, an average-reward version of Soft Actor-Critic that replaces the discount factor with a learned estimate of the average reward rate. In a RAN slicing simulator, ARO-SAC improves system performance by 15% over the best tuned discounted SAC with a discount factor of 0.99, while avoiding the instability of a discount factor of one. The paper's point is that average-reward RL is a better objective match for wireless network optimization and is practically implementable by modifying an off-policy actor-critic algorithm.

What carries the argument

The central object is the average-reward TD error and the trainable average reward estimator $\rho$. In discounted SAC the TD target is $r + \gamma \min_i Q_{\phi_{targ,i}}(s', \tilde a')$; in ARO-SAC it becomes $r - \rho + \min_i Q_{\phi_{targ,i}}(s', \tilde a')$. The estimator $\rho$ is trained by gradient descent on $\varepsilon_t = r(s_t,a_t) - \rho_t - \min(Q_1(s_t,a_t), Q_2(s_t,a_t))$, following the off-policy estimation strategy from ARO-DDPG. This machinery replaces geometric discounting of future rewards with a constant subtraction of the estimated average reward, making the value function measure deviations from the long-term average rather than a shrinking tail. It is what lets the policy optimize the undiscounted objective without setting the discount factor to one.

What would settle it

Train ARO-SAC and discounted SAC on a synthetic MDP whose optimal average reward per step is known analytically; if ARO-SAC's learned $\rho$ does not track the true average reward or its policy converges to a lower average reward than the optimal policy, the claim that it solves the average-reward objective fails. A simpler check is to run ARO-SAC on the RAN slicing task with the $\rho$ update disabled and a fixed $\rho=0$; if performance keeps the 15% gain, the gain would be attributable to the algorithm change rather than to the average reward estimate.

Watch

Extended reading notes

Core claim

The paper's central claim is that the widely used discounted reward RL formulation is misaligned with wireless RRM objectives, which are undiscounted long-term average QoS, and that switching to average reward RL closes the gap. The discovery is an algorithm, ARO-SAC: take SAC's actor-critic updates, drop the discount factor, and instead maintain a trainable average reward estimate, updated by gradient descent on the temporal-difference residual. The TD target becomes the reward minus the estimated average reward plus the target value of the next state, so the critic learns a differential value function rather than a discounted one. In the RAN slicing testbed with fixed horizon, ARO-SAC achieves a 15% gain over the best discounted SAC and avoids the divergence that appears with a discount factor of one. The author's intended conclusion is that average reward RL should replace discounted reward RL in RRM because it matches the true network objective.

Load-bearing premise

The load-bearing premise is that the modified Bellman updates in Algorithm 1 define a valid and convergent average-reward SAC: the paper assumes the trainable average reward estimate $\rho$ converges and that the altered TD target remains a sound learning signal, but it provides no proof.

Editorial extensions

If this is right

  • In the tested RAN slicing setup, a larger discount factor with fixed horizon increases cumulative reward, showing that discounted agents leave performance on the table.
  • Extending the environment horizon at a fixed discount factor improves average reward per step, confirming that horizon truncation compounds the objective mismatch.
  • Setting the discount factor to one is not a workable fix because training becomes unstable, motivating the average-reward reformulation.
  • ARO-SAC with a properly tuned learning rate for the average reward estimate delivers a 15% gain over vanilla SAC with a discount factor of 0.99 in the simulator.
  • The average reward RL framework adds one trainable parameter and one extra hyperparameter to SAC, so the gain comes with a modest tuning cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims: the same TD-target swap should transfer to other off-policy actor-critic algorithms, suggesting the 15% gain reflects objective alignment rather than a SAC-specific quirk.
  • Because the mismatch is about the objective, the benefit should grow in tasks with long horizons and non-stationary traffic; a testable prediction is that ARO-SAC's margin over discounted SAC increases as the episode period grows beyond the tested value.
  • The paper's update for the average reward estimate resembles a differential baseline, so convergence may depend on the ratio of learning rates between the estimator and the critics; the reported choice of a smaller learning rate for the estimator is a plausible practical rule.
  • Under the average reward objective, reward shaping choices such as the delay-violation penalty weight should be re-tuned, so the reported 15% gain may shift with different QoS trade-offs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that the standard discounted-reward RL objective is misaligned with the long-term average QoS goals of radio resource management (RRM), and uses a RAN slicing task to illustrate this. It proposes ARO-SAC, an adaptation of Soft Actor-Critic to the average-reward setting, and reports a 15% performance gain over vanilla SAC with gamma=0.99. The paper also studies the effect of the discount factor and horizon on the discounted formulation, showing that larger gamma and longer horizons help but gamma=1 is unstable.

Significance. If the central claim were supported, the paper would address a real and under-appreciated mismatch between the RL objective and wireless network optimization goals. The use of an industry-grade simulator (NetworkGym/NS-3) and the systematic study of discount factor and horizon effects are useful and credit-worthy. However, the proposed ARO-SAC algorithm is not a valid average-reward SAC: it removes SAC's entropy regularization and uses an unjustified update for the average-reward estimate. The empirical headline result is therefore confounded by multiple algorithmic changes and lacks reported uncertainty. As presented, the paper does not support its main contribution.

major comments (4)
  1. [Section V-B, Eq. (8), Algorithm 1 line 19] The average-reward estimator is updated by minimizing eps_t = r - rho - min Q(s,a), whose fixed point is rho = r - Q(s,a), a state-action-dependent residual rather than the average reward. This omits the successor value that appears in the differential TD error of Eq. (6), delta_t = R_{t+1} - rho + V(S_{t+1}) - V(S_t). No convergence or consistency argument is provided; the paper itself defers theoretical guarantees to Section VI. This invalidates the claim that Algorithm 1 implements an average-reward RL algorithm.
  2. [Algorithm 1, lines 12 and 17] Algorithm 1 is not Soft Actor-Critic. The policy update in line 17 is the plain policy gradient sum log pi(a|s) Q(s,a) without SAC's entropy regularization term -alpha log pi(a|s), and the target in line 12 is y = r - rho + min Q_targ(s', a') rather than SAC's r + gamma(min Q - alpha log pi). ARO-SAC therefore differs from the vanilla SAC baseline in both the return criterion and the policy-update rule. Consequently, the 15% gain in Fig. 3 cannot be attributed to alignment with the average-reward objective; it may stem from removing entropy regularization or from the altered update altogether.
  3. [Fig. 3] The headline comparison lacks error bars or confidence intervals, even though Fig. 2 reports them. The text states the result is averaged over 5 independent runs and 5 user-number combinations, but no variance is reported. Given that the central claim is an empirical performance gain, the absence of uncertainty quantification makes the 15% figure difficult to assess.
  4. [Section III-B] The motivating example states that with gamma = 0.95, rewards beyond 50 time steps contribute only about 0.01 of their original value. This is quantitatively wrong: 0.95^50 is approximately 0.077, not 0.01. While illustrative, the error weakens the numerical motivation for the paper's central mismatch argument.
minor comments (5)
  1. [Abstract] There is a grammatical error: 'called the Average Reward Off policy Soft Actor Critic (ARO SAC) is an adaptation' should be 'called the Average Reward Off-policy Soft Actor-Critic (ARO-SAC), which is an adaptation.'
  2. [Section III-B] There are missing citation placeholders '[?]' in the paragraph on previous DRL studies of RAN slicing; these should be filled before publication.
  3. [Section V-A, Eq. (3)] Equation (3) uses r(pi) for the average reward, while later equations use rho. The notation should be unified.
  4. [Algorithm 1, line 19] The line 'Update average reward estimator rho: nabla_rho 1/|B| sum (eps_t)^2' is not a complete update rule; it should specify the loss and the learning rate or optimizer used.
  5. [Section V-C] The paper states that the learning rate for rho needs careful selection and was set to 1e-5, but it does not report how this value was chosen or how sensitive the result is to it, which is relevant given the new hyperparameter.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the 15% gain is an empirical comparison, though the SAC naming and convergence assumptions are correctness risks.

full rationale

ARO-SAC's central claim is an empirical comparison (Fig. 3), not a derivation from its own assumptions. The average-reward estimator rho is fitted online, but the reported gain is the measured policy performance, not the fitted rho, and no equation forces the outcome by construction. The alignment in Sec. V-A is definitional (setting r(t) to the QoS function makes the average-reward objective Eq. (2) equal the wireless objective Eq. (1)), but it is used as motivation rather than as a predicted result, so it is not load-bearing circularity. The algorithm design is explicitly adapted from the external ARO-DDPG work [3] and standard average-reward RL [1]; the self-citations to prior wireless RL work [13, 18, 21, 22] are background and do not support the average-reward extension. The more substantive concerns—that Algorithm 1 omits SAC's entropy regularization and that the rho update has no convergence proof—are validity/correctness issues, not circularity; the paper itself acknowledges the missing theory in Section VI ('Interesting future works include providing theoretical guarantees of ARO-SAC'). No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors, and no self-citation chain forces the method. Verdict: no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No physical entities are invented. The load is carried by algorithmic assumptions: that the average reward MDP formulation, the chosen reward, and the modified SAC updates are correct. These are plausible but unverified, and the missing entropy term is a notable ad hoc deviation.

free parameters (4)
  • Reward penalty weight alpha = 4
    Set by hand in Section III.B to weight delay violation penalty against throughput; affects what policy is optimal, so the 15% gain is relative to this reward design.
  • Average reward estimator learning rate = 1e-5
    Chosen in Section V.C as a hyperparameter for ARO-SAC; the paper notes it needs careful selection and reports no search or sensitivity analysis.
  • Baseline discount factor gamma = 0.99
    Used for vanilla SAC comparison in Fig. 3; the gain is measured against this value, not against a tuned range.
  • Episode/reset horizon T = 200
    Fixed in the main comparison; Tables II and III show performance depends on T and gamma, so the 15% claim is tied to T=200 and gamma=0.99.
assumptions (5)
  • domain assumption The RAN slicing MDP and its netgymenv/NS-3 simulation faithfully represent real wireless network behavior.
    Section III.C uses netgymenv with LTE NS-3 traffic; no real network validation is given.
  • standard math The average reward Bellman equation for differential returns is applicable to this MDP.
    Section V.A derives V from the average reward objective following [1]; requires standard ergodicity or stationarity assumptions not stated.
  • ad hoc to paper Removing SAC's entropy regularization still yields a valid average reward Soft Actor Critic algorithm.
    Algorithm 1 lines 12 and 17 omit the entropy bonus present in SAC; no proof or ablation shows this preserves SAC's properties.
  • domain assumption The scalar reward R(t)=sum_i(Trx,i - 4 Dvio,i) captures QoS objectives for RAN slicing.
    Reward design in Section III.B is chosen by the authors; the optimal policy depends on it.
  • domain assumption The softmax normalized action space over N-1 proportions is sufficient to represent optimal RBG allocation.
    Actions defined in Section III.B as proportions summing to at most 1; assumes no benefit to discrete RBG allocations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Average Reward Reinforcement Learning for Wireless Radio Resource Management." pith.science (2026). https://pith.science/paper/SSYCKOKQ

@misc{pith2026250106700,
  author       = {Pith},
  title        = {Pith review of: Average Reward Reinforcement Learning for Wireless Radio Resource Management},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SSYCKOKQ}},
  note         = {Machine review of arXiv:2501.06700}
}
read the original abstract

In this paper, we address a crucial but often overlooked issue in applying reinforcement learning (RL) to radio resource management (RRM) in wireless communications: the mismatch between the discounted reward RL formulation and the undiscounted goal of wireless network optimization. To the best of our knowledge, we are the first to systematically investigate this discrepancy, starting with a discussion of the problem formulation followed by simulations that quantify the extent of the gap. To bridge this gap, we introduce the use of average reward RL, a method that aligns more closely with the long-term objectives of RRM. We propose a new method called the Average Reward Off policy Soft Actor Critic (ARO SAC) is an adaptation of the well known Soft Actor Critic algorithm in the average reward framework. This new method achieves significant performance improvement our simulation results demonstrate a 15% gain in the system performance over the traditional discounted reward RL approach, underscoring the potential of average reward RL in enhancing the efficiency and effectiveness of wireless network optimization.

Figures

Figures reproduced from arXiv: 2501.06700 by the authors.

Figure 1
Figure 1. Illustration of a RAN slicing system Assume the system has M RBGs in total, and the QoS function at time t is ft(M(t)), where M(t) = [m1(t), · · · , mN (t)], where M(t) is the resource allocation vector which stands for the resource blocks allocated to different slices and mi(t) stands for the resource allocated to slice i. Then, we can formulate the optimization problem as: maximize M(t) lim T→∞ 1 T X T t=1 ft(M(t)… view at source ↗
Figure 2
Figure 2. Experimental results with γ = 1. Shadowed areas indicate the confidence intervals. V. AVERAGE REWARD SOFT ACTOR-CRITIC The concept of average reward RL, as the name suggests, is to maximize the long-term average reward for sequential decision-making problems [1]. In this section, we discuss the difference between the average reward RL and the discounted reward RL, and show how to change a discounted reward formulati… view at source ↗
Figure 3
Figure 3. Experimental result using ARO-SAC, where the experiment is [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 28 canonical work pages

  1. [30]

    A Deeper Look at Discounting Mismatch in Actor-Critic Algorithms

    S. Zhang, R. Laroche, H. van Seijen, S. Whiteson, and R. T. d. Combes, “A deeper look at discounting mismatch in actor-critic algorithms,”arXiv preprint arXiv:2010.01069, 2020

    work page Pith review arXiv 2010

  2. [1]

    Reinforcement learning: An introduction by richards’ sutton,

    A. G. Barto, “Reinforcement learning: An introduction by richards’ sutton,” SIAM Rev, vol. 6, no. 2, p. 423, 2021

    work page 2021

  3. [2]

    Average-reward off- policy policy evaluation with function approximation,

    S. Zhang, Y . Wan, R. S. Sutton, and S. Whiteson, “Average-reward off- policy policy evaluation with function approximation,” in international conference on machine learning . PMLR, 2021, pp. 12 578–12 588

    work page 2021

  4. [3]

    Off-policy average reward actor-critic with deterministic policy search,

    N. Saxena, S. Khastagir, S. Kolathaya, and S. Bhatnagar, “Off-policy average reward actor-critic with deterministic policy search,” in Inter- national Conference on Machine Learning . PMLR, 2023, pp. 30 130– 30 203

    work page 2023

  5. [4]

    Power control for a network of access points,

    S. Nagaraja, F. Meshkati, M. Yavuz, S. Mitra, V . Khaitan, V . P. S. Makh, C. S. Patel, Y . Tokgoz, and C. Shen, “Power control for a network of access points,” Nov. 2016, US Patent 9,497,714

    work page 2016

  6. [5]

    Base station employing shared resources among antenna units,

    N. Valliappan, C. Chevallier, A. D. Radulescu, and C. Shen, “Base station employing shared resources among antenna units,” Sept. 2016, US Patent 9,451,466

    work page 2016

  7. [6]

    Methods and apparatus for power management in a wireless communication system,

    Y . Huang, C. S. Patel, T. A. Kadous, M. Yavuz, L. Zhang, R. Prakash, V . Chande, C. Chevallier, S. Nagaraja, F. Meshkatiet al., “Methods and apparatus for power management in a wireless communication system,” 2016, uS Patent 9,451,480

    work page 2016

  8. [7]

    Generalized global bandit and its application in cellular coverage optimization,

    C. Shen, R. Zhou, C. Tekin, and M. van der Schaar, “Generalized global bandit and its application in cellular coverage optimization,” IEEE J. Sel. Topics Signal Process., vol. 12, no. 1, pp. 218–232, Feb. 2018

    work page 2018

Show all 32 references
  1. [8]

    A non-stationary online learning approach to mobility management,

    Y . Zhou, C. Shen, and M. van der Schaar, “A non-stationary online learning approach to mobility management,” IEEE Trans. Wireless Commun., vol. 18, no. 2, pp. 1434–1446, Feb. 2019

    2019

  2. [9]

    A deep Q-learning method for downlink power allocation in multi-cell networks,

    K. I. Ahmed and E. Hossain, “A deep Q-learning method for downlink power allocation in multi-cell networks,” arXiv preprint arXiv:1904.13032, 2019

    1904 arXiv

  3. [10]

    Power allocation in multi-user cellular networks with deep Q learning approach,

    F. Meng, P. Chen, and L. Wu, “Power allocation in multi-user cellular networks with deep Q learning approach,” in IEEE International Con- ference on Communications (ICC) . IEEE, 2019, pp. 1–6

    2019

  4. [11]

    Joint power control and channel allocation for interference mitigation based on reinforcement learning,

    G. Zhao, Y . Li, C. Xu, Z. Han, Y . Xing, and S. Yu, “Joint power control and channel allocation for interference mitigation based on reinforcement learning,” IEEE Access , vol. 7, pp. 177 254–177 265, 2019

    2019

  5. [12]

    Deep actor-critic learning for distributed power control in wireless mobile networks,

    Y . S. Nasir and D. Guo, “Deep actor-critic learning for distributed power control in wireless mobile networks,” in 2020 54th Asilomar Conference on Signals, Systems, and Computers . IEEE, 2020, pp. 398–402

    2020

  6. [13]

    Deep reinforcement learning based wire- less network optimization: A comparative study,

    K. Yang, C. Shen, and T. Liu, “Deep reinforcement learning based wire- less network optimization: A comparative study,” in IEEE INFOCOM Workshop on Data Driven Intelligence for Networks , Toronto, Canada, Jul. 2020, pp. 1248–1253

    2020

  7. [14]

    ColO- RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms,

    M. Polese, L. Bonati, S. D’Oro, S. Basagni, and T. Melodia, “ColO- RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms,” IEEE Transactions on Mobile Computing , pp. 1–14, July 2022

    2022

  8. [15]

    FlexRAN: A flexible and programmable platform for software- defined radio access networks,

    X. Foukas, N. Nikaein, M. M. Kassem, M. K. Marina, and K. Konto- vasilis, “FlexRAN: A flexible and programmable platform for software- defined radio access networks,” in Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies , 2...

    2016

  9. [16]

    Deep reinforcement learning for joint spectrum and power allocation in cellular networks,

    Y . S. Nasir and D. Guo, “Deep reinforcement learning for joint spectrum and power allocation in cellular networks,” in 2021 IEEE Globecom Workshops (GC Wkshps). IEEE, 2021, pp. 1–6

    2021

  10. [17]

    Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,

    ——, “Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 10, pp. 2239–2250, 2019

    2019

  11. [18]

    Multi- agent reinforcement learning for wireless user scheduling: Performance, scalablility, and generalization,

    K. Yang, D. Li, C. Shen, J. Yang, S.-p. Yeh, and J. Sydir, “Multi- agent reinforcement learning for wireless user scheduling: Performance, scalablility, and generalization,” in 2022 56th Asilomar Conference on Signals, Systems, and Computers . IEEE, 2022, pp. 1169–1174

    2022

  12. [19]

    Resource management in wireless networks via multi-agent deep reinforcement learning,

    N. Naderializadeh, J. J. Sydir, M. Simsek, and H. Nikopour, “Resource management in wireless networks via multi-agent deep reinforcement learning,” IEEE Transactions on Wireless Communications , vol. 20, no. 6, pp. 3507–3523, 2021

    2021

  13. [20]

    Distributed MARL for scheduling in conflict graphs,

    Y . Zhang and D. Guo, “Distributed MARL for scheduling in conflict graphs,” in 2023 59th Annual Allerton Conference on Communication, Control, and Computing (Allerton) . IEEE, 2023, pp. 1–8

    2023

  14. [21]

    Offline reinforcement learning for wireless network optimization with mixture datasets,

    K. Yang, C. Shi, C. Shen, J. Yang, S. Yeh, and J. Sydir, “Offline reinforcement learning for wireless network optimization with mixture datasets,” IEEE Transactions on Wireless Communications , vol. 23, no. 10, pp. 12 703–12 716, Oct. 2024

    2024

  15. [22]

    Advancing RAN slicing with offline reinforcement learning,

    K. Yang, S.-P. Yeh, M. Zhang, J. Sydir, J. Yang, and C. Shen, “Advancing RAN slicing with offline reinforcement learning,” in 2024 IEEE Inter- national Symposium on Dynamic Spectrum Access Networks (DySPAN) . IEEE, 2024, pp. 331–338

    2024

  16. [23]

    Mean-variance policy iteration for risk-averse reinforcement learning,

    S. Zhang, B. Liu, and S. Whiteson, “Mean-variance policy iteration for risk-averse reinforcement learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 12, 2021, pp. 10 905– 10 913

    2021

  17. [24]

    Averaged-dqn: Variance reduc- tion and stabilization for deep reinforcement learning,

    O. Anschel, N. Baram, and N. Shimkin, “Averaged-dqn: Variance reduc- tion and stabilization for deep reinforcement learning,” in International conference on machine learning . PMLR, 2017, pp. 176–185

    2017

  18. [25]

    Average-reward reinforcement learning with trust region methods,

    X. Ma, X. Tang, L. Xia, J. Yang, and Q. Zhao, “Average-reward reinforcement learning with trust region methods,” arXiv preprint arXiv:2106.03442, 2021

    2021 arXiv

  19. [26]

    The NS-3 network simulator,

    G. F. Riley and T. R. Henderson, “The NS-3 network simulator,” in Modeling and tools for network simulation . Springer, 2010, pp. 15–34

    2010

  20. [27]

    Network slicing architecture,

    L. Geng, J. Dong, S. Bryant, K. Makhijani, A. Galis, X. de Foy, and S. Kuklinsk, “Network slicing architecture,” Internet Engineering Task Force, Internet-Draft draft-geng-netslices-architecture- 02, 2017, available online: https://datatracker.ietf.org/doc/html/ draft-geng-net...

    2017

  21. [28]

    NetworkGym: Democratizing Network AI via Sim-aaS,

    M. Zhang and J. Zhu, “NetworkGym: Democratizing Network AI via Sim-aaS,” https://intellabs.github.io/networkgym/, 2023

    2023

  22. [29]

    Soft actor-critic algorithms and applications,

    T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V . Ku- mar, H. Zhu, A. Gupta, P. Abbeel et al. , “Soft actor-critic algorithms and applications,” arXiv preprint arXiv:1812.05905 , 2018

    2018 arXiv

  23. [31]

    Revisiting the minimalist approach to offline reinforcement learning,

    D. Tarasov, V . Kurenkov, A. Nikulin, and S. Kolesnikov, “Revisiting the minimalist approach to offline reinforcement learning,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  24. [32]

    Supported policy optimization for offline reinforcement learning,

    J. Wu, H. Wu, Z. Qiu, J. Wang, and M. Long, “Supported policy optimization for offline reinforcement learning,” Advances in Neural Information Processing Systems , vol. 35, pp. 31 278–31 291, 2022

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.