Pith. sign in

REVIEW 3 major objections 6 minor 42 references

Cyrus+: A DRL-based Puncturing Solution to URLLC/eMBB Multiplexing in O-RAN

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims a decoder-feedback DRL policy can schedule URLLC punctures so that eMBB goodput beats fixed heuristics while staying inside O-RAN's real-time budget.

desk verdict Cyrus+ is a solid systems paper with a plausible DRL design and honest simulator work, but the headline goodput gains rest on an idealized receiver model and the numerology-3 timing claim is under-supported. read the letter →

arxiv 2506.00167 v1 pith:ARYG4HYI submitted 2025-05-30 cs.NI

Ehsan Ghoreishi , Bahman Abolhassani , Yan Huang , Shiva Acharya , Wenjing Lou , Y. Thomas Hou This is my paper
classification cs.NI
keywords 5GNRURLLCeMBBpuncturingdeepreinforcementlearningsoftactor-criticO-RANgoodput
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the standard way of measuring the harm of URLLC puncturing on eMBB traffic—raw throughput—is the wrong target, because a punctured packet may still decode at its receiver. It presents Cyrus+, a deep reinforcement learning scheduler whose reward is goodput: the fraction of eMBB subcarriers whose packets actually survive LDPC decoding and CRC at the UE. The design is placed inside O-RAN's three control loops, with a curriculum-pretrained policy in the Non-RT RIC, critic refinement in the Near-RT RIC, and a parallelized RT O-DU that precomputes a puncturing codebook for every possible URLLC arrival count. In a standard-compliant 5G NR link-level simulator, Cyrus+ reports eMBB goodput of 49.852 Mb/s at URLLC packet probability p=0.08, against 16.430 Mb/s for resource-proportional and 27.384 Mb/s for smallest-eMBB-first puncturing, while the RT O-DU averages 94.4 microseconds per TTI—inside the 125 microsecond numerology-3 budget. If the result holds, it shows that a model-free, receiver-aware policy can exploit LDPC decoding resilience in a way that fixed heuristics and threshold damage models cannot.

What carries the argument

The load-bearing object is the goodput reward $$ r(t) = \frac{1}{N}\sum_{e\in E}(d_e(t)-1)n_e(t) $$ the normalized fraction of eMBB subcarriers whose packets fail decoding in TTI t, where d_e(t) is the CRC outcome at eMBB receiver e. This reward trains a Soft Actor-Critic (SAC) agent split across O-RAN: the actor in the RT O-DU, the two critics in the Near-RT RIC. The actor outputs, for each possible URLLC arrival count j, per-user Gaussian distributions over puncture amounts; the feasibility enforcer projects these samples into a feasible integer puncturing vector (KL-divergence minimization followed by water-filling and Huntington-Hill rounding), and the vectors for all j are packed into the puncturing codebook B*(t). The codebook turns real-time scheduling into a lookup: when k(t,τ) URLLC packets arrive in mini-slot τ, the O-RU selects column k(t,τ) instantly, decoupling the learning loop from the mini-slot deadline.

What would settle it

Run the same trained Cyrus+ policy in a simulator whose receiver must infer puncture locations from control signaling instead of being handed the puncturing matrix, or whose LDPC decoder treats punctured bits as random noise rather than erasures, and compare eMBB goodput against RP and SeF under identical traffic and channel conditions.

Watch

Extended reading notes

Core claim

Cyrus+'s central claim is that a deep reinforcement learning agent trained on decoder feedback can choose which eMBB codewords to puncture so that the total goodput—the subcarriers whose packets decode successfully—consistently exceeds that of resource-proportional (RP) and smallest-eMBB-first (SeF) puncturing, while the decision engine still meets O-RAN real-time constraints. Each TTI, the RT O-DU runs the current policy in parallel branches, one for each possible number of URLLC packets in a mini-slot, sampling Gaussian per-user puncturing amounts and projecting them through a feasibility enforcer into a valid integer puncturing codebook. When URLLC packets arrive in a mini-slot, the O-RU simply looks up the corresponding codebook column. The reward used for learning is the normalized fraction of eMBB subcarriers that fail decoding, computed from per-receiver CRC feedback, so the agent learns to spread punctures across codewords in a way that protects decodability rather than raw bit rate. The paper reports this scheme beats both benchmarks across URLLC arrival probability, packet size, bandwidth, and two channel models, and that the O-DU computation stays under the 125 µs numerology-3 timing requirement with an average of 94.4 µs over 1,000 TTIs.

Load-bearing premise

The whole advantage rests on the assumption that eMBB receivers are told exactly which resource elements were punctured and can hand those bit positions to the LDPC decoder as erasures (NULL confidence), so the decoder can recover around them; if real receivers must infer puncture locations from control signaling, or if erasure handling differs from the simulated decoder, the learned policy's edge over fixed heuristics may not transfer.

Editorial extensions

If this is right

  • Reward design matters more than the puncturing rule: training on decoder goodput rather than raw throughput is what lets the learned policy outperform both RP and SeF.
  • Precomputing a codebook for every possible URLLC arrival count moves the real-time decision to a lookup at the O-RU, so the expensive part of the policy can run once per TTI, in parallel.
  • The same trained actor can be reused across changing eMBB scheduling vectors because the critic keeps refining it from an input buffer of recent (scheduling, arrivals, actions, rewards) tuples.
  • Without the curriculum-based pre-training in the Non-RT RIC, the shallow network needed for RT execution fails to converge, so the three-loop split is not just architectural but necessary for the approach to work.
  • The reported timing of 94.4 µs average O-DU computation indicates the approach is compatible with the 125 µs numerology-3 slot budget on COTS GPU hardware.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested extension is applying the same goodput-reward codebook design to other preemption multiplexing schemes (superposition coding or rate splitting), where decodability rather than raw rate is again the true cost.
  • The receiver model's reliance on perfect puncture-location knowledge suggests a stress test: retrain with the receiver inferring punctured positions from signaling, or with punctured bits treated as random noise instead of LLR erasures, and measure how much of the reported gain survives.
  • The curriculum schedule is hand-tuned in the paper; a natural follow-up is to make the Near-RT RIC adjust the curriculum automatically, which would remove a deployment-specific tuning burden.
  • Because the simulator assumes ideal channel estimation and a single codeword per eMBB user, the reported gaps are an upper-bound estimate; a testable extension is to reproduce the comparison under channel-estimation error and HARQ retransmissions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Cyrus+, a soft actor-critic (SAC) deep reinforcement learning scheduler that decides how URLLC traffic should puncture ongoing eMBB transmissions in a 5G NR O-RAN deployment. The key design choices are (i) a decoder-goodput-based reward rather than a raw-throughput reward, (ii) separation of the actor and critics across the O-RAN Non-RT RIC, Near-RT RIC, and RT O-DU control loops, (iii) pre-computation of a puncturing codebook for all possible URLLC arrival counts in each mini-slot, and (iv) a feasibility enforcer that maps continuous policy outputs to valid integer puncturing vectors. The authors implement a link-level simulator using MATLAB 5G Toolbox with a CDL channel, and compare Cyrus+ against resource-proportional (RP) and smallest-eMBB-first (SeF) puncturing. They report higher eMBB goodput for Cyrus+ across URLLC arrival rate, packet size, bandwidth, and channel model, and an average RT O-DU computation time of 94.4 microseconds, which they interpret as meeting the 125 microsecond numerology 3 timing requirement.

Significance. If the reported gains are robust, the paper makes a useful contribution: it replaces raw-throughput objectives with actual decoder-goodput feedback, which is the right metric for puncturing, and it explicitly maps the learning architecture onto the O-RAN three-loop structure. The GPU-parallel codebook generation and the feasibility enforcer are concrete engineering contributions, and the evaluation against two natural heuristics across several system parameters is broader than in much prior work. The use of a standard 5G NR PHY simulator with LDPC decoding is a strength, as is the explicit measurement of per-TTI O-DU latency. However, two load-bearing issues prevent acceptance in the current form: the timing claim is made for numerology 3 while the simulator is configured for numerology 2, and the receiver model assumes perfect, zero-cost knowledge of the exact puncturing map, which is not a 3GPP baseline receive assumption and may materially inflate the reported goodput advantage.

major comments (3)
  1. [Section VII-B, Table IV, Fig. 8] The claim that the measured O-DU runtime meets the numerology 3 timing requirement is not supported by the stated experimental configuration. Table IV sets the simulator to "NR numerology 2" with 60 kHz subcarrier spacing, which corresponds to a 250 microsecond slot, not the 125 microsecond slot of numerology 3 (120 kHz SCS). The abstract and Section VII-B claim that the 94.4 microsecond average runtime 'meets the timing requirement for 5G NR numerology 3', but the experiment never exercises numerology 3 physical-layer parameters. The authors should either reconfigure the simulator (and re-measure the codebook generation time) with 120 kHz SCS and 125 microsecond slots, or explicitly restrict the timing claim to numerology 2.
  2. [Section VI-D (UE, Rx Signal Processing)] The eMBB receivers are assumed to know the exact puncturing matrix Y(t) and to treat punctured bit LLRs as NULL erasures during LDPC decoding. In 5G NR, puncturing is designed to be transparent to eMBB UEs; a compliant receiver does not generally know which resource elements were preempted, and the preemption indication (PI) that could provide such knowledge is coarse in time/frequency and consumes control resources that are not modeled here. Since the DRL reward and the reported goodput numbers depend directly on decoder outcomes under this erasure model, the headline advantage of Cyrus+ over RP and SeF (49.852 Mb/s vs. 27.384 Mb/s at p=0.08) is an upper-bound result for this receiver assumption. The comparison among the three schemes is internally fair because all use the same oracle, but the general claim that Cyrus+ 'consistently achieves the best performance' is not demonstrated for a 3GPP-compliant receiver chain. The authors should either add a preemption-indication overhead and coarse puncturing-awareness model, or rerun the evaluation without exact puncturing knowledge at the receiver.
  3. [Section VII-B and Figs. 9-12] The performance comparison is based on what appears to be a single simulation run per configuration, with no report of variance. The DRL policy is stochastic at sampling time, the URLLC arrivals are random, and the channel is fading, so the 5,000-TTI time averages could vary substantially across seeds. The claim that Cyrus+ 'consistently outperforms' RP and SeF would be considerably stronger if the authors reported mean and standard deviation (or confidence intervals) over multiple independent runs, especially for the parameter points where the margins appear small, such as p=0.03 in Fig. 9.
minor comments (6)
  1. [Section V-A, Eq. (4)] There is an indexing inconsistency in the definition of the target Q-value: the text says the actual reward r(t) is used at the final step (τ=M), but Eq. (4) assigns r(t) to the case τ=M-1 and leaves τ=M undefined. Please align the equation with the seven-step description.
  2. [Section V-B, Eq. (6)] The notation in Eq. (6) uses "m_e(t, .)" without defining the omitted argument; the placeholder is confusing in a formal definition. Also, the expression is called a KL divergence even though the vectors are not normalized probability distributions; a brief justification or a different divergence name would help.
  3. [Section VII-B] Typos: 'Cyru+' should be 'Cyrus+' in the goodput paragraph, 'behvior' in the Fig. 6 caption, and 'incresing' in the Fig. 7 caption. Minor proofreading would improve the presentation.
  4. [Section V-C, footnote 5] The maximum number of URLLC arrivals used as the training ceiling is determined empirically by observing where the average reward drops to -1. This is a reasonable practical choice, but the paper should state explicitly that behavior for arrival counts above this empirically found ceiling is outside the trained regime and may require re-training in deployment.
  5. [Section I and III-A] The contribution bullet states that Cyrus+ 'does not require any knowledge of assumptions on packet arrival, channel conditions, and decoding behavior', but Section III-A models URLLC arrivals as Bernoulli with probability p. It would be clearer to state that these assumptions are used only for generating training data and for evaluation, not at inference time.
  6. [Section VII-C, Fig. 9] The text says that at low p (e.g., p=0.03) RP and SeF perform 'similar' to Cyrus+, but the figure appears to show Cyrus+ still ahead; please quantify the differences at low arrival rates rather than relying on the figure alone.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DRL policy is trained on a goodput-derived reward and evaluated on the same simulator, but the goodput numbers are empirical outcomes against external benchmark heuristics, not consequences of the reward definition by construction.

full rationale

The paper's derivation chain is an empirical machine-learning pipeline, not a closed-form derivation. The reward in Eq. (2), r(t) = (1/N) sum_e (d_e(t)-1) n_e(t), is the negative normalized count of undecodable SCs, and the objective in Eq. (1) is the long-term average goodput sum_e d_e(t) n_e(t); maximizing one is equivalent to maximizing the other because N is constant. This alignment is the intended RL training signal, not a hidden premise smuggled in as a prediction. The reported goodput values (e.g., 49.852 Mb/s for Cyrus+ versus 27.384 Mb/s for SeF at p=0.08) are produced by running the trained policy, RP, and SeF through the same 5G NR link-level simulator with decoder feedback, so they are independent simulation outcomes rather than consequences of the reward equation. The benchmarks RP and SeF are defined as external heuristics in Section VII-A, not fitted from the data or derived from the reward function. No parameter is fitted to a subset of the data and then renamed as a prediction. The self-citations [1] (the abridged ICCCN version) and [12] (prior work by overlapping authors) are contextual related-work references; neither is invoked as a load-bearing justification for Cyrus+ performance, and no uniqueness theorem or forced-choice argument is imported from those works. The skeptical reviewer's concern about the receiver being given the exact puncturing map Y(t) (Section VI-D) is a modeling-assumption or external-validity issue that affects whether the advantage over RP and SeF transfers to 3GPP-compliant receivers; it is not circularity, because the same receiver oracle is applied fairly to all schemes and the performance numbers are not equivalent to the reward by construction. Overall, the paper is self-contained against external benchmarks, and no step in the claimed derivation reduces to its own inputs.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The central claim rests mainly on simulation-domain assumptions (channel coherence, Bernoulli arrivals, erasure-style LDPC decoding) and on empirically chosen training hyperparameters. There are no fitted physical constants or invented entities; the main risk is that the decoder model and the omitted URLLC service constraints define the results.

free parameters (6)
  • Entropy regularization coefficient zeta
    Introduced in Eq. (4) and Eq. (5) to balance entropy versus reward; no value is reported, yet it strongly affects exploration and the final policy.
  • Discount factor lambda = 0.95
    Table IV; sets how far the critic looks ahead in the discounted cumulative reward.
  • Adam learning rate = 3e-4
    Table IV; used for both actor and critics, and it is a hand-set hyperparameter for the SAC agent.
  • Soft target update rate = 0.005
    Section VII-B; controls how slowly target Q-networks track the main critics and affects stability.
  • ACL window and episode schedule = (50,1e4), (40,1e4), (30,1e4), (20,1.4e4), (10,1.4e4), (5,1.4e4), (2,1.4e4), (1,1.4e4), (1,1.4e4)
    Section V-C; the curriculum schedule is hand-picked, and the stored policy is taken after 1e5 episodes, so this schedule shapes all reported results.
  • Maximum URLLC arrivals used as training ceiling = 9 packets per TTI
    Footnote 5 and Section VII-B: determined empirically as the point where average reward drops close to -1; used to set the curriculum and to stop training.
assumptions (7)
  • domain assumption The eMBB scheduling vector s(t) is given a priori by an external scheduler and is not optimized by Cyrus+.
    Section III-A states that s(t) is decided by the eMBB scheduler before each TTI and assumed given; the paper solves only the puncturing subproblem.
  • domain assumption The channel remains coherent across all mini-slots within a TTI.
    Section IV-A: 'assuming all channel remains coherent for all mini-slots in a TTI'; this justifies computing one codebook per TTI.
  • domain assumption Each URLLC packet has fixed size L and arrivals follow a Bernoulli distribution with probability p per mini-slot per user.
    Section III-A; this defines the j=1..floor(N/L) branches and the traffic generator used in evaluation.
  • domain assumption Punctured eMBB bits can be modeled as erasures by setting LLR confidence to NULL, and LDPC decoding with those erasures is accurately captured by the 5G Toolbox.
    Section VI-D receiver processing; all goodput results depend on this decoder model, including the assumption that receivers know exact puncturing positions.
  • domain assumption URLLC reliability and latency constraints can be omitted from the objective and the evaluation.
    Objective (1) and reward (2) consider only eMBB decoding success, and Section VI-A says URLLC bits are all zeros with no URLLC decoder simulated.
  • ad hoc to paper The empirically found maximum URLLC arrival count and ACL schedule are valid proxies for real operating conditions.
    Section V-C and footnote 5: the 9-packet ceiling and the window sizes are found by trial and error, not derived from channel or traffic models.
  • standard math KL-divergence minimization over the simplex with linear constraints is convex, and the water-filling plus Huntington-Hill rounding produces the nearest feasible integer puncturing vector.
    Section V-B relies on [28], [29], and [30] for feasibility enforcement; this is standard convex projection with a rounding step.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cyrus+: A DRL-based Puncturing Solution to URLLC/eMBB Multiplexing in O-RAN." pith.science (2026). https://pith.science/paper/ARYG4HYI

@misc{pith2026250600167,
  author       = {Pith},
  title        = {Pith review of: Cyrus+: A DRL-based Puncturing Solution to URLLC/eMBB Multiplexing in O-RAN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ARYG4HYI}},
  note         = {Machine review of arXiv:2506.00167}
}
read the original abstract

Puncturing is a promising technique in 3GPP to multiplex Enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low Latency Communications (URLLC) traffic on the same 5G New Radio (NR) air interface. The essence of puncturing is to transmit URLLC packets on demand upon their arrival, by preempting the radio resources (or subcarriers) that are already allocated to eMBB traffic. Although it is considered most bandwidth efficient, puncturing URLLC data on eMBB can lead to degradation of eMBB's performance. Most of the state-of-the-art research addressing this problem employ raw eMBB data throughput as performance metric. This is inadequate as, after puncturing, eMBB data may or may not be successfully decoded at its receiver. This paper presents Cyrus+, a deep reinforcement learning (DRL)-based puncturing solution that employs goodput (through feedback from a receiver's decoder), rather than estimated raw throughput, in its design of reward function. Further, Cyrus+ is tailored specifically for the Open RAN (O-RAN) architecture and fully leverages O-RAN's three control loops at different time scales in its design of DRL. In the Non-Real-Time (Non-RT) RAN Intelligent Controller (RIC), Cyrus+ initializes the policy network that will be used in the RT Open Distributed Unit (O-DU). In the Near-RT RIC, Cyrus+ refines the policy based on dynamic network conditions and feedback from the receivers. In the RT O-DU, Cyrus+ generates a puncturing codebook by considering all possible URLLC arrivals. We build a standard-compliant link-level 5G NR simulator to demonstrate the efficacy of Cyrus+. Experimental results show that Cyrus+ outperforms benchmark puncturing algorithms and meets the stringent timing requirement in 5G NR (numerology 3).

Figures

Figures reproduced from arXiv: 2506.00167 by the authors.

Figure 1
Figure 1. An illustration of multiplexing eMBB and URLLC on [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of Cyrus+ in the O-RAN architecture. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Architecture diagram of the NR simulator. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Concurrent forward propagation for all possible [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: A network topology with 10 eMBB users and 12 URLLC users. User 1 User 3 User 2 User 5 User 4 User 6 User 7 User 9 User 8 User 10 Mini-slot Mini-slot 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 Cyrus+ RP Mini-slot 0 1 2 3 4 5 6 7 SeF [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: URLLC puncturing behviaor at TTI 300 under Cyrus+, [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 8
Figure 8. Figure 8: Computation time of Cyrus+ in each TTI for the RT [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: eMBB goodput under varying URLLC packet arrival [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: eMBB goodput under varying URLLC packet size. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: eMBB goodput under varying channel bandwidth. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: eMBB goodput in a CDL-D channel under varying [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 35 canonical work pages

  1. [1]

    Cyrus: A DRL-based Puncturing Solution to URLLC/eMBB Multiplexing in O-RAN,

    E. Ghoreishi, B. Abolhassani, Y . Huang, S. Acharya, W. Lou, and Y . T. Hou, “Cyrus: A DRL-based Puncturing Solution to URLLC/eMBB Multiplexing in O-RAN,” inProc. 33rd International Conference on Computer Communications and Networks (ICCCN), pp. 1–9, July 29– 31, 2024, Hawaii, USA

    work page 2024

  2. [2]

    aspx?specificationId=3107 (Last accessed: April 2024)

    3GPP,Service Requirements for the 5G System (3GPP TS 22.261, Version 19.6.0, Release 19, April 2024), Available: https: //portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails. aspx?specificationId=3107 (Last accessed: April 2024)

    work page 2024

  3. [3]

    3GPP,Study on Physical Layer Enhancements for NR Ultra-Reliable and Low Latency Case (URLLC) (3GPP TR 38.824, Version 16.0.0, Release 16, March 2019), Available: https://portal.3gpp.org/desktopmodules/ Specifications/SpecificationDetails.aspx?specificationId=3498 (Last ac- cessed: April 5, 2024)

    work page 2019

  4. [4]

    3GPP,Final Report of 3GPP TSG RAN WG1 Meeting #88 (Version 1.0.0, February 2017), Available: https://www.3gpp.org/ftp/TSG RAN/ WG1 RL1/TSGR1 88/Report/ (Last accessed: April 3, 2024)

    work page 2017

  5. [5]

    3GPP,NR; Physical channels and modulation (3GPP TS 38.211, Version 18.4.0, Release 18, September 2024), Available: https://portal.3gpp.org/desktopmodules/Specifications/ SpecificationDetails.aspx?specificationId=3214 (Last accessed: October 1, 2024)

    work page 2024

  6. [6]

    Joint Scheduling of URLLC and eMBB Traffic in 5G Wireless Networks,

    A. Anand, G. De Veciana, and S. Shakkottai, “Joint Scheduling of URLLC and eMBB Traffic in 5G Wireless Networks,” inProc. IEEE INFOCOM, pp. 1970–1978, April 16–19, 2018, Honolulu, HI, USA

    work page 1970

  7. [7]

    O-RAN Alliance,O-RAN Architecture Description, Available: https:// orandownloadsweb.azurewebsites.net/specifications (Last accessed: Jan- uary 5, 2024)

    work page 2024

  8. [8]

    Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor,” inProc. ICML, pp. 1861–1870, July 10–15, 2018, Stockholm Sweden

    work page 2018

Show all 42 references
  1. [9]

    Intelli- gent Dynamic Resource Allocation and Puncturing for Next-Generation Wireless Networks,

    W. Alqwider, A. S. Abdalla, T. F. Rahman, and V . Marojevic, “Intelli- gent Dynamic Resource Allocation and Puncturing for Next-Generation Wireless Networks,”IEEE Internet of Things Journal, vol. 11, no. 19, pp. 31438–31452, 2024

    2024

  2. [10]

    Intelligent Resource Management for eMBB and URLLC in 5G and Beyond Wireless Networks,

    R. M. Sohaib, O. Onireti, Y . Sambo, R. Swash, S. Ansari, and M. A. Imran, “Intelligent Resource Management for eMBB and URLLC in 5G and Beyond Wireless Networks,”IEEE Access, vol. 11, pp. 65205- 65221, 2023

    2023

  3. [11]

    Deep Rein- forcement Learning for URLLC data management on top of scheduled eMBB traffic,

    F. Saggese, L. Pasqualini, M. Moretti, and A. Abrardo, “Deep Rein- forcement Learning for URLLC data management on top of scheduled eMBB traffic,” inProc. IEEE GLOBECOM, pp. 1–6, December 7–11, 2021, Madrid, Spain

    2021

  4. [12]

    A Deep-Reinforcement- Learning-Based Approach to Dynamic eMBB/URLLC Multiplexing in 5G NR,

    Y . Huang, S. Li, C. Li, Y . T. Hou, and W. Lou, “A Deep-Reinforcement- Learning-Based Approach to Dynamic eMBB/URLLC Multiplexing in 5G NR,”IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6439–6456, 2020

    2020

  5. [13]

    Coexistence Mechanism Between eMBB and uRLLC in 5G Wireless Networks,

    A. K. Bairagi, M. S. Munir, M. Alsenwi, N. H. Tran, S. S. Alshamrani, M. Masud, Z. Han, and C. S. Hong, “Coexistence Mechanism Between eMBB and uRLLC in 5G Wireless Networks,”IEEE Transactions on Communications, vol. 69, no. 3, pp. 1736–1749, 2021

    2021

  6. [14]

    eMBB-URLLC Resource Slicing: A Risk-Sensitive Approach,

    M. Alsenwi, N. H. Tran, M. Bennis, A. Kumar Bairagi, and C. S. Hong, “eMBB-URLLC Resource Slicing: A Risk-Sensitive Approach,”IEEE Communications Letters, vol. 23, no. 4, pp. 740–743, 2019

    2019

  7. [15]

    Joint Resource and Power Allocation for URLLC-eMBB Traffics Multiplexing in 6G Wireless Networks,

    M. Almekhlafi, M. A. Arfaoui, C. Assi, and A. Ghrayeb, “Joint Resource and Power Allocation for URLLC-eMBB Traffics Multiplexing in 6G Wireless Networks,” inProc. IEEE ICC, pp. 1-6, June 14–23, 2021, Montreal Canada

    2021

  8. [16]

    A Downlink Puncturing Scheme for Simultaneous Trans- mission of URLLC and eMBB Traffic by Exploiting Data Similarity,

    M. Almekhlafi, M. Chraiti, M. A. Arfaoui, C. Assi, A. Ghrayeb, and A. Alloum, “A Downlink Puncturing Scheme for Simultaneous Trans- mission of URLLC and eMBB Traffic by Exploiting Data Similarity,” IEEE Transactions on Vehicular Technology, vol. 70, no. 12, pp. 13087- 13100, 2021

    2021

  9. [17]

    R. G. Gallager,Low-Density Parity-Check Codes, ser. Monograph No. RLE-TR-156. Cambridge, MA: MIT Press, 1963

    1963

  10. [18]

    2023), Available: https: //portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails

    3GPP,NR; Physical layer; General description (3GPP TS 38.201, Version 18.0.0, Release 18, Sep. 2023), Available: https: //portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails. aspx?specificationId=3211 (Last accessed: Apr. 8, 2024)

    2023

  11. [19]

    On the error-correcting capabilities of low-complexity de- coded irregular LDPC codes,

    P. Rybin, “On the error-correcting capabilities of low-complexity de- coded irregular LDPC codes,” inProc. IEEE International Symposium on Information Theory, pp. 3165–3169, Jun 30– July 5, 2014, Honolulu, Hawai, USA

    2014

  12. [20]

    Design of Low-Density Parity Check Codes for 5G New Radio,

    T. Richardson and S. Kudekar, “Design of Low-Density Parity Check Codes for 5G New Radio,”IEEE Communications Magazine, vol. 56, no. 3, pp. 28–34, 2018

    2018

  13. [21]

    Proximal Policy Optimization Algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” arXiv:1707.06347, 2017

    2017 arXiv

  14. [22]

    Power Control Based on Deep Reinforcement Learning for Spectrum Sharing,

    H. Zhang, N. Yang, W. Huangfu, K. Long, and V . C. M. Leung, “Power Control Based on Deep Reinforcement Learning for Spectrum Sharing,” IEEE Transactions on Wireless Communications, vol. 19, no. 6, pp. 4209–4219, 2020

    2020

  15. [23]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv:1509.02971, 2019

    2019 arXiv

  16. [24]

    Deep Reinforcement Learning Based Resource Management for Multi-Access Edge Computing in Vehicular Networks,

    H. Peng and X. Shen, “Deep Reinforcement Learning Based Resource Management for Multi-Access Edge Computing in Vehicular Networks,” IEEE Transactions on Network Science and Engineering, vol. 7, no. 4, pp. 2416–2428, 2020

    2020

  17. [25]

    Addressing Function Approx- imation Error in Actor-Critic Methods,

    S. Fujimoto, H. van Hoof, and D. Meger, “Addressing Function Approx- imation Error in Actor-Critic Methods,” inProc. ICML, pp. 1582–1591, July 10–15, 2018, Stockholm Sweden

    2018

  18. [26]

    R. S. Sutton and A. G. Barto,Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018

    2018

  19. [27]

    On Information and Sufficiency,

    S. Kullback and R. A. Leibler, “On Information and Sufficiency,”The Annals of Mathematical Statistics, vol. 22, no. 1, pp. 79–86, 1951

    1951

  20. [28]

    Efficient projections onto the l 1-ball for learning in high dimensions,

    J. Duchi, S. Shalev-Shwartz, Y . Singer, and T. Chandra, “Efficient projections onto the l 1-ball for learning in high dimensions,” inProc. ICML, pp. 272–279, July 5–9, 2008, Helsinki, Finland

    2008

  21. [29]

    Generalized Iterative Scaling for Log- Linear Models,

    J. N. Darroch and D. Ratcliff, “Generalized Iterative Scaling for Log- Linear Models,”The Annals of Mathematical Statistics, vol. 43, no. 5, pp. 1470–1480, 1972

    1972

  22. [30]

    M. L. Balinski and H. P. Young,Fair Representation: Meeting the Ideal of One Man, One Vote. Washington, DC, USA: Brookings Institution Press, 2001

    2001

  23. [31]

    Curriculum learning,

    Y . Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” inProc. ICML, pp. 41–48, June 14–18, 2009, Quebec, Canada

    2009

  24. [32]

    Automatic Curriculum Learning For Deep RL: A Short Survey,

    R. Portelas, C. Colas, L. Weng, K. Hofmann, and P.-Y . Oudeyer, “Automatic Curriculum Learning For Deep RL: A Short Survey,” in Proc. 29th International Joint Conference on Artificial Intelligence, pp. 1–7, January 7–15, 2021, Yokohama, Japan

    2021

  25. [33]

    Reverse Curricu- lum Generation for Reinforcement Learning,

    C. Florensa, D. Held, M. Wulfmeier, and P. Abbeel, “Reverse Curricu- lum Generation for Reinforcement Learning,” inProc. Conference on Robot Learning, pp. 482–495, November 13–15, 2017, California, USA

    2017

  26. [34]

    5G Toolbox Documentation,

    MathWorks, “5G Toolbox Documentation,” 2023, Available: https:// www.mathworks.com/help/5g/ (Last accessed: April 5, 2024)

    2023

  27. [35]

    TensorFlow: Large-Scale Machine Learning on Het- erogeneous Distributed Systems,

    M. Abadi et al., “TensorFlow: Large-Scale Machine Learning on Het- erogeneous Distributed Systems,” 2016, arXiv:1603.04467

    2016 arXiv

  28. [36]

    Rate Control for Communication Networks: Shadow Prices, Proportional Fairness and Stability,

    F. Kelly, A. Maulloo, and D. Tan, “Rate Control for Communication Networks: Shadow Prices, Proportional Fairness and Stability,”Journal of the Operational Research Society, vol. 49, no. 3, pp. 237–252, 1998

    1998

  29. [37]

    cuDNN Developer Guide V9.1.0,

    NVIDIA, “cuDNN Developer Guide V9.1.0,” Available: https: //docs.nvidia.com/deeplearning/cudnn/latest/developer/overview.html, 2024, (Last accessed: April 12, 2024)

    2024

  30. [38]

    Auto-Encoding Variational Bayes,

    D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” arXiv:1312.6114, 2022

    2022 arXiv

  31. [39]

    Low-Density Parity-Check Codes with Erasures and Puncturing,

    J. Ha, “Low-Density Parity-Check Codes with Erasures and Puncturing,” Ph.D. dissertation, Georgia Institute of Technology, November 2003, Atlanta, USA. 17

    2003

  32. [40]

    Rectified Linear Units Improve Restricted Boltzmann Machines,

    V . Nair and G. E. Hinton, “Rectified Linear Units Improve Restricted Boltzmann Machines,” inProc. ICML, pp. 807–814, June 21–14, 2010, Haifa, Israel

    2010

  33. [41]

    Adam: A Method for Stochastic Optimization,

    D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980, 2016

    2016 arXiv

  34. [42]

    Accelerated Methods for Deep Reinforcement Learning,

    A. Stooke and P. Abbeel, “Accelerated Methods for Deep Reinforcement Learning,” arXiv:1803.02811, 2019

    2019 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.