pith. sign in

arxiv: 2604.24369 · v1 · submitted 2026-04-27 · 📡 eess.SP · cs.NI

Beam Scheduling for Cross-Layer ISAC: A Deep Reinforcement Learning Approach

Xiyu Wang , Gilberto Berardinelli , Hei Victor Cheng , Petar Popovski , Ramoni Adeogun This is my paper

Pith reviewed 2026-05-08 02:12 UTC · model grok-4.3

classification 📡 eess.SP cs.NI
keywords beam schedulingintegrated sensing and communicationdeep reinforcement learningresource allocationcross-layer optimizationmulti-user systemssensing performancelow-latency communication
0
0 comments X

The pith

Deep reinforcement learning for beam scheduling in ISAC systems reaches performance near a genie-aided benchmark that assumes perfect angle knowledge.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops a deep reinforcement learning method to allocate beams in integrated sensing and communication setups. The goal is to support low-latency communication while reducing sensing estimation errors in environments with changing user data traffic and wireless conditions. Rather than requiring full channel state information, the approach uses sensing observations to cut feedback overhead. The resulting multi-beam allocation improves total throughput and adapts to buffer status, with only small added delays. Overall, the learned policy delivers communication and sensing results close to an ideal case with complete prior knowledge of angles of departure.

Core claim

The DRL-assisted beam allocation reduces feedback overhead by leveraging sensing observations. The proposed multi-beam scheme improves overall throughput with only modest delay increases. The DRL framework effectively takes buffer status into account and adapts to the wireless environment while allocating resources. The DRL-assisted beam management achieves both communication and sensing performance close to that of the genie-aided benchmark with perfect angle-of-departure knowledge.

What carries the argument

A deep reinforcement learning agent that learns beam allocation decisions directly from sensing observations and buffer queue states to jointly optimize communication latency and sensing accuracy.

If this is right

  • Resource allocation accounts for cross-layer data buffer dynamics and queue status in addition to physical-layer channels.
  • The method handles the coupling between practical buffer states and time-varying wireless conditions without separate channel estimation.
  • Overall system throughput rises while communication delay grows only modestly.
  • Both communication and sensing metrics approach the levels obtained with perfect angle-of-departure information.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Sensing data may substitute for channel feedback in other ISAC resource allocation tasks, lowering control signaling.
  • The same learning structure could be tested in scenarios with higher user counts or burstier traffic to check robustness.
  • Cross-layer DRL policies might allow joint design of sensing and communication waveforms without explicit separation of the two objectives.

Load-bearing premise

Sensing observations can stand in for explicit channel state information when learning beam policies that work across varying traffic loads and multi-user channels.

What would settle it

A set of simulations in which the DRL policy's throughput or sensing error deviates substantially from the genie-aided benchmark once traffic arrival rates or channel coherence times change faster than the training distribution.

Figures

Figures reproduced from arXiv: 2604.24369 by Gilberto Berardinelli, Hei Victor Cheng, Petar Popovski, Ramoni Adeogun, Xiyu Wang.

Figure 1
Figure 1. Figure 1: A multi-user mono-static ISAC system and |A|. The n-th column of matrix A is represented as [A]n. CN (a, A) denotes circularly symmetric complex Gaussian vector with mean a and covariance matrix A, and N (·) denotes the real Gaussian distribution. The indicator function is 1. The statistical expectation is E{·}. The imaginary unit is j = √ −1. The remainder of this paper is organized as follows. The system… view at source ↗
Figure 2
Figure 2. Figure 2: An illustration of DRL-assisted cross-layer beam allocation for ISAC view at source ↗
Figure 3
Figure 3. Figure 3: Comparison between the training reward of RL algorithms and the view at source ↗
Figure 4
Figure 4. Figure 4: Performance of the proposed DRL-assisted method. The comparison between PPO-1b (one beam for a user) and PPO-mb (one or multiple beams for a view at source ↗
Figure 5
Figure 5. Figure 5: Communication performance results. In (a), comparison of the throughput CDF of different methods in the two channel conditions. In (b), a comparison view at source ↗
Figure 6
Figure 6. Figure 6: Variation in the probabilities of estimation error less than a predefined threshold when different methods are adopted in the two channel conditions. view at source ↗
read the original abstract

Resource allocation in integrated sensing and communication (ISAC) systems needs to be optimized to balance the requirements of the communication and sensing modules considering complicated cross-layer data traffic and queue status in dynamic multi-user environments. This paper studies the beam allocation for cross-layer ISAC that achieves low-latency communication and minimizes sensing parameters estimation error. To handle the complex coupling between practical data buffer dynamics and varying wireless channels, we propose a deep reinforcement learning (DRL)-assisted approach. Rather than relying on explicit channel state information, the DRL-assisted beam allocation reduces feedback overhead by leveraging sensing observations. Simulation results verify that the DRL framework effectively takes buffer status into account and adapts to the wireless environment while allocating resources. The proposed multi-beam scheme improves overall throughput with only modest delay increases. Finally, the DRL-assisted beam management achieves both communication and sensing performance close to that of the genie-aided benchmark with perfect angle-of-departure (AoD) knowledge. These contributions advance the state-of-the-art intelligent resource management for ISAC systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a deep reinforcement learning (DRL) framework for beam scheduling in cross-layer integrated sensing and communication (ISAC) systems. It optimizes multi-beam allocation to jointly minimize communication latency/throughput degradation and sensing parameter estimation error in dynamic multi-user environments with time-varying data buffers and wireless channels. The approach relies solely on sensing observations (rather than explicit CSI) to reduce feedback overhead and claims, via simulations, to approach the performance of a genie-aided benchmark that has perfect angle-of-departure knowledge.

Significance. If the simulation results hold under rigorous validation, the work would contribute to practical ISAC resource management by showing that DRL policies can incorporate cross-layer buffer dynamics while using sensing returns to substitute for CSI, thereby lowering overhead in multi-user scenarios. The emphasis on joint communication-sensing trade-offs and adaptation to traffic variations is a constructive direction for the field.

major comments (2)
  1. [Abstract] Abstract and simulation section: the central claim that the DRL policy achieves communication and sensing performance 'close to' the genie-aided benchmark with perfect AoD knowledge is load-bearing but unsupported by presented evidence. No details are given on the DRL state/action/reward formulation, neural architecture, training algorithm, baseline comparisons (e.g., myopic or CSI-based schedulers), number of Monte Carlo runs, or statistical significance of the reported gaps in throughput, delay, and sensing error.
  2. [Method and Simulation Results] The assumption that sensing observations (range-Doppler-angle maps or equivalent) encode sufficient instantaneous multi-user channel structure to enable near-genie beam allocation under rapidly varying buffers and user-specific arrivals is untested. No ablation on partial observability, information-theoretic gap analysis, or sensitivity to sensing resolution is provided; this directly affects whether the DRL can systematically close the performance gap to the perfect-AoD benchmark.
minor comments (2)
  1. Notation for the DRL reward function weights and the precise definition of 'sensing parameters estimation error' should be clarified with explicit equations.
  2. Figure captions and axis labels in the simulation results should explicitly state the number of users, traffic arrival model, and sensing SNR regime to allow reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed review. We address each major comment below and outline the revisions planned for the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract and simulation section: the central claim that the DRL policy achieves communication and sensing performance 'close to' the genie-aided benchmark with perfect AoD knowledge is load-bearing but unsupported by presented evidence. No details are given on the DRL state/action/reward formulation, neural architecture, training algorithm, baseline comparisons (e.g., myopic or CSI-based schedulers), number of Monte Carlo runs, or statistical significance of the reported gaps in throughput, delay, and sensing error.

    Authors: We agree that the abstract and simulation section would benefit from greater explicitness to support the performance claims. The DRL formulation (state consisting of sensing observations and buffer status, action as multi-beam allocation, reward as joint latency-sensing error metric), neural architecture, and training algorithm are described in Section III of the manuscript, but we will revise the abstract to include a concise summary of these elements. In the simulation results, we will add explicit baseline comparisons (including myopic and CSI-based schedulers), state the number of Monte Carlo runs performed, and report statistical significance or confidence intervals for the gaps in throughput, delay, and sensing error. These changes will make the evidence for approaching the genie-aided benchmark more transparent. revision: yes

  2. Referee: [Method and Simulation Results] The assumption that sensing observations (range-Doppler-angle maps or equivalent) encode sufficient instantaneous multi-user channel structure to enable near-genie beam allocation under rapidly varying buffers and user-specific arrivals is untested. No ablation on partial observability, information-theoretic gap analysis, or sensitivity to sensing resolution is provided; this directly affects whether the DRL can systematically close the performance gap to the perfect-AoD benchmark.

    Authors: The referee correctly identifies the absence of targeted validation for the sufficiency of sensing observations. While the presented simulations demonstrate that the DRL policy approaches genie-aided performance, we did not include dedicated ablations. In the revision we will add an ablation study varying sensing resolution (e.g., angle bin size in the range-Doppler-angle maps) and its effect on beam allocation quality and overall performance. We will also explicitly discuss how the maps provide partial channel structure information via AoD estimation. A full information-theoretic gap analysis lies beyond the scope of this work and would require new theoretical development; we will acknowledge this limitation while retaining the empirical demonstration that sensing observations enable near-benchmark operation under the tested dynamic buffer and channel conditions. revision: partial

Circularity Check

0 steps flagged

No circularity: DRL beam allocation is simulation-validated without self-referential derivations

full rationale

The paper defines a DRL policy whose state uses sensing observations (range-Doppler-angle maps) to allocate beams, with reward incorporating buffer status, throughput, latency, and sensing error. Training occurs via standard RL interaction with a simulated environment; performance is then compared to a genie benchmark with perfect AoD. No equations reduce a claimed prediction to a fitted input by construction, no uniqueness theorem is imported via self-citation, and no ansatz is smuggled. The approach is an empirical heuristic whose validity rests on external simulation benchmarks rather than internal redefinition of its own outputs.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The paper relies on standard DRL assumptions and simulation-based validation; no explicit free parameters or new entities are detailed in the abstract.

free parameters (2)
  • DRL reward function weights
    Likely tuned to balance communication latency and sensing error, but not specified in abstract.
  • Neural network architecture parameters
    Standard in DRL but chosen for this problem.
axioms (1)
  • domain assumption The wireless channel and buffer dynamics can be modeled as a Markov decision process suitable for DRL.
    Implicit in proposing DRL for this dynamic environment.

pith-pipeline@v0.9.0 · 5493 in / 1443 out tokens · 38308 ms · 2026-05-08T02:12:01.214298+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    Integrated Sensing and Communications: Toward Dual- Functional Wireless Networks for 6G and Beyond,

    F. Liu, Y . Cui, C. Masouros, J. Xu, T. X. Han, Y . C. Eldar, and S. Buzzi, “Integrated Sensing and Communications: Toward Dual- Functional Wireless Networks for 6G and Beyond,”IEEE J. Sel. Areas Commun, vol. 40, no. 6, pp. 1728–1767, Jun. 2022

    2022

  2. [2]

    A Survey of Beam Management for mmWave and THz Communications Towards 6G,

    Q. Xue, C. Ji, S. Ma, J. Guo, Y . Xu, Q. Chen, and W. Zhang, “A Survey of Beam Management for mmWave and THz Communications Towards 6G,”IEEE Commun. Surv. Tutor, vol. 26, no. 3, p. 1520–1559, Feb. 2024

    2024

  3. [3]

    Multi- beam integrated sensing and communication: State-of-the-art, challenges and opportunities,

    Y . Zhuo, T. Mao, H. Li, C. Sun, Z. Wang, Z. Han, and S. Chen, “Multi- beam integrated sensing and communication: State-of-the-art, challenges and opportunities,”IEEE Commun. Mag., vol. 62, no. 9, pp. 90–96, 2024

    2024

  4. [4]

    Beam Drift in Millimeter Wave Links: Beamwidth Tradeoffs and Learning Based Optimization,

    J. Zhang and C. Masouros, “Beam Drift in Millimeter Wave Links: Beamwidth Tradeoffs and Learning Based Optimization,”IEEE Trans. Commun., vol. 69, no. 10, pp. 6661–6674, 2021

    2021

  5. [5]

    Vehic- ular Connectivity on Complex Trajectories: Roadway-Geometry Aware ISAC Beam-Tracking,

    X. Meng, F. Liu, C. Masouros, W. Yuan, Q. Zhang, and Z. Feng, “Vehic- ular Connectivity on Complex Trajectories: Roadway-Geometry Aware ISAC Beam-Tracking,”IEEE Transactions on Wireless Communications, vol. 22, no. 11, pp. 7408–7423, Nov. 2023

    2023

  6. [6]

    Joint Beam Alignment and Resource Allocation for Multi-User Mmwave Integrated Sensing and Communica- tion Systems,

    J. Zhang, S. Yan, and M. Peng, “Joint Beam Alignment and Resource Allocation for Multi-User Mmwave Integrated Sensing and Communica- tion Systems,”IEEE Trans. Veh. Technol., vol. 73, no. 4, pp. 5288–5303, April 2024

    2024

  7. [7]

    Multiuser Beam Tracking and Target Detection in Integrated Sensing and Communication,

    K. Chen, C. Qi, and O. A. Dobre, “Multiuser Beam Tracking and Target Detection in Integrated Sensing and Communication,” inIEEE ICC2023. Rome, Italy: IEEE, May 2023, pp. 5743–5748

    2023

  8. [8]

    Radar-Assisted Predictive Beamforming for Vehicular Links: Communication Served by Sensing,

    F. Liu, W. Yuan, C. Masouros, and J. Yuan, “Radar-Assisted Predictive Beamforming for Vehicular Links: Communication Served by Sensing,” IEEE Trans. Wireless Commun., vol. 19, no. 11, pp. 7704–7719, Nov. 2020

    2020

  9. [9]

    Deep Reinforcement Learning-based Beamforming Design in ISAC-assisted Vehicular Networks,

    Y . Liu, S. Zhang, X. Li, Y . Huang, Y . Fang, and H. Cao, “Deep Reinforcement Learning-based Beamforming Design in ISAC-assisted Vehicular Networks,” in2024 IEEE WCNC, 2024

    2024

  10. [10]

    DRL Based Beam Management for Joint Sensing and Communications in HSR mmWave Wireless Networks,

    L. Yan, X. Fang, S. Li, Y . Li, and Q. Xue, “DRL Based Beam Management for Joint Sensing and Communications in HSR mmWave Wireless Networks,” in2022 IEEE 95th VTC2022-Spring, 2022

    2022

  11. [11]

    Beam Selection and Power Allocation: Using Deep Learning for Sensing-Assisted Communication,

    L. Chen, K. Liu, Z. Zhang, and B. Li, “Beam Selection and Power Allocation: Using Deep Learning for Sensing-Assisted Communication,” IEEE Wireless Commun. Lett., vol. 13, no. 2, pp. 323–327, 2024

    2024

  12. [12]

    In- Band Full-Duplex Multiple-Input Multiple-Output Systems for Simulta- neous Communications and Sensing: Challenges, methods, and future perspectives,

    B. Smida, G. C. Alexandropoulos, T. Riihonen, and M. A. Islam, “In- Band Full-Duplex Multiple-Input Multiple-Output Systems for Simulta- neous Communications and Sensing: Challenges, methods, and future perspectives,”IEEE Signal Processing Magazine, vol. 41, no. 5, pp. 8–16, Sep. 2024

    2024

  13. [13]

    Cooperative ISAC Networks: Opportunities and Challenges,

    K. Meng, C. Masouros, A. P. Petropulu, and L. Hanzo, “Cooperative ISAC Networks: Opportunities and Challenges,”IEEE Wireless Commu- nications, vol. 32, no. 3, pp. 212–219, Jun. 2025. 12

    2025

  14. [14]

    Optimal Scheduling Policy for Time-Division Joint Radar and Communication Systems: Cross-Layer Design and Sensing for Free,

    Z. Xie, R. Li, Z. Jiang, J. Zhu, X. She, and P. Chen, “Optimal Scheduling Policy for Time-Division Joint Radar and Communication Systems: Cross-Layer Design and Sensing for Free,”IEEE Internet of Things Journal, vol. 10, no. 23, pp. 20 746–20 760, Dec. 2023

    2023

  15. [15]

    Joint Beamforming and Power Allocation Strategy for NOMA Empowered ISAC Systems,

    Z. Xie, R. Li, Y . Gu, Z. Jiang, J. Zhu, P. Chen, and S. Song, “Joint Beamforming and Power Allocation Strategy for NOMA Empowered ISAC Systems,”IEEE Transactions on Vehicular Technology, vol. 74, no. 2, pp. 3445–3450, 2025

    2025

  16. [16]

    Joint precoding for MIMO radar and URLLC in ISAC systems,

    C. Ding, C. Zeng, C. Chang, J.-B. Wang, and M. Lin, “Joint precoding for MIMO radar and URLLC in ISAC systems,” inProceedings of the 1st ACM MobiCom Workshop on Integrated Sensing and Communications Systems. Sydney NSW Australia: ACM, Oct. 2022, pp. 12–18

    2022

  17. [17]

    Multi-User Beamforming with Deep Reinforcement Learning in Sensing- Aided Communication,

    X. Wang, G. Berardinelli, H. V . Cheng, P. Popovski, and R. Adeogun, “Multi-User Beamforming with Deep Reinforcement Learning in Sensing- Aided Communication,” in2025 European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), 2025, pp. 61–66

    2025

  18. [18]

    DFT-Based Beamforming Weight- Vector Codebook Design for Spatially Correlated Channels in the Unitary Precoding Aided Multiuser Downlink,

    D. Yang, L.-L. Yang, and L. Hanzo, “DFT-Based Beamforming Weight- Vector Codebook Design for Spatially Correlated Channels in the Unitary Precoding Aided Multiuser Downlink,” in2010 IEEE International Conference on Communications. Cape Town, South Africa: IEEE, May 2010, pp. 1–5

    2010

  19. [19]

    A Novel Joint Angle-Range-Velocity Estimation Method for MIMO-OFDM ISAC Systems,

    Z. Xiao, R. Liu, M. Li, Q. Liu, and A. L. Swindlehurst, “A Novel Joint Angle-Range-Velocity Estimation Method for MIMO-OFDM ISAC Systems,”IEEE Transactions on Signal Processing, vol. 72, pp. 3805– 3818, 2024

    2024

  20. [20]

    MIMO-OFDM ISAC Waveform Design for Range-Doppler Sidelobe Suppression,

    P. Li, M. Li, R. Liu, Q. Liu, and A. Lee Swindlehurst, “MIMO-OFDM ISAC Waveform Design for Range-Doppler Sidelobe Suppression,”IEEE Transactions on Wireless Communications, vol. 24, no. 2, pp. 1001–1015, 2025

    2025

  21. [21]

    The impact of beamwidth on temporal channel variation in vehicular channels and its implications,

    V . Va, J. Choi, and R. W. Heath, “The impact of beamwidth on temporal channel variation in vehicular channels and its implications,”IEEE Trans. Veh. Technol., vol. 66, no. 6, pp. 5014–5029, 2017

    2017

  22. [22]

    Cohen,Signals, Systems, and Transforms: Concise Coverage from Theory to Application

    F. Cohen,Signals, Systems, and Transforms: Concise Coverage from Theory to Application. John Wiley & Sons, Ltd, 2025

    2025

  23. [23]

    Limited Feedforward Waveform Design for OFDM Dual-Functional Radar-Communications,

    M. F. Keskin, V . Koivunen, and H. Wymeersch, “Limited Feedforward Waveform Design for OFDM Dual-Functional Radar-Communications,” IEEE Transactions on Signal Processing, vol. 69, pp. 2955–2970, 2021

    2021

  24. [24]

    Multibeam for Joint Communication and Radar Sensing Using Steerable Analog Antenna Arrays,

    J. A. Zhang, X. Huang, Y . J. Guo, J. Yuan, and R. W. Heath, “Multibeam for Joint Communication and Radar Sensing Using Steerable Analog Antenna Arrays,”IEEE Trans. Veh. Technol., vol. 68, no. 1, pp. 671–685, Jan. 2019

    2019

  25. [25]

    Codebook Designs for Millimeter-Wave Communication Systems in Both Lowand High- Mobility: Achievements and Challenges,

    S. Mabrouki, I. Dayoub, Q. Li, and M. Berbineau, “Codebook Designs for Millimeter-Wave Communication Systems in Both Lowand High- Mobility: Achievements and Challenges,”IEEE Access, vol. 10, pp. 25 786 – 25 810, 2022

    2022

  26. [26]

    For sale: state-action representation learning for deep reinforcement learning,

    S. Fujimoto, W.-D. Chang, E. J. Smith, S. S. Gu, D. Precup, and D. Meger, “For sale: state-action representation learning for deep reinforcement learning,” inProceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS ’23, 2023

    2023

  27. [27]

    Proximal Policy Optimization Algorithms

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,”CoRR, vol. abs/1707.06347, 2017

    work page internal anchor Pith review arXiv 2017

  28. [28]

    6G Use Cases and Requirements,

    —, “6G Use Cases and Requirements,”HEXA-X-II, 2023. [Online]. Available: https://hexa-x-ii.eu/wp-content/uploads/2024/01/Hexa-X-II D1.2.pdf 13

    2023