Pith. sign in

REVIEW 2 major objections 6 minor 41 references

This paper claims that for C-V2X radio resource allocation, the dominant multi-agent RL challenge is robustness to diverse and unseen vehicular topologies, not non-stationarity, coordination, action-space size, or partial observability.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 22:30 UTC pith:SO4TAI6X

load-bearing objection Useful benchmark for C-V2X MARL; central robustness claim holds up, but the partial-observability conclusion rests on a flawed comparison. the 2 major comments →

arxiv 2603.06607 v2 pith:SO4TAI6X submitted 2026-02-18 cs.MA cs.AIcs.LG

Multi-Agent Reinforcement Learning for V2X Resource Allocation: Disentangling MARL Challenges Through Benchmarking

Siyuan Wang , Lei Lei , Pranav Maheshwari , Sam Bellefeuille , Kan Zheng This is my paper
classification cs.MA cs.AIcs.LG
keywords multi-agent reinforcement learningC-V2Xradio resource allocationbenchmarkinggeneralizationrobustnessinterference gamesvehicular networks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds a ladder of multi-agent interference games — from a single-snapshot normal-form game to a partially observable stochastic game — so that each rung adds one MARL challenge. Benchmarking eight representative algorithms on these tasks, it finds that non-stationarity, coordination, large action spaces, and partial observability cause only mild degradation, whereas training on many diverse vehicular topologies cuts average normalized returns by tens of percentage points and is the hardest hurdle. Actor-critic methods, especially PPO-style independent learning, hold up best as the number of vehicles grows, while value-based methods degrade sharply or turn negative at 16 agents. The paper concludes that future C-V2X MARL research should target zero-shot policy transfer across topologies and recommends independent PPO as a practical baseline.

Core claim

On its own terms, the paper's discovery is that topology robustness and generalization, rather than the four textbook MARL challenges, are the binding constraint for learning radio resource allocation policies in C-V2X networks. The evidence is a controlled benchmark: performance stays near-optimal when agents face a single fixed topology, fast fading, multi-step decisions, and up to 16 agents, but collapses when the training set spans many vehicle arrangements — dropping up to 59 percentage points in normalized return — and it stays low even on training topologies, showing the difficulty is robustness rather than extrapolation alone. The paper further finds that partial observability is not

What carries the argument

A hierarchy of multi-agent interference games — normal-form, stochastic, and partially observable stochastic — with progressively added challenges, along with a dataset of highway topologies spanning sparse to dense traffic. Each game is a controlled rung on a ladder, so comparing adjacent rungs isolates a single MARL challenge and lets the authors attribute performance differences to that challenge.

Load-bearing premise

The conclusion rests on the benchmark ladder changing only one intended challenge at a time; the step from single-location to multi-location training alters dataset size and diversity simultaneously, and the step from global-state to local-observation tasks alters state dimension along with observability, so the relative ranking of challenges could shift if these confounds were removed.

What would settle it

Evaluate the best actor-critic policy trained on a single fixed topology directly on the nine test topologies of the multi-location task. If its normalized return stays close to the multi-topology-trained policy instead of collapsing, the paper's central attribution — that diverse topologies are the dominant challenge — would be falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, C-V2X MARL research should prioritize methods that improve zero-shot transfer across topologies over further coordination or observability fixes.
  • Actor-critic algorithms, particularly PPO variants, are the safer baseline for vehicular resource allocation at scale; value-decomposition methods may help at small agent counts but degrade as agents grow.
  • The benchmark suite and open datasets allow reproducible comparison, so future algorithms can be evaluated against a known difficulty ladder.
  • High-dimensional global states can hurt learning in diverse-topology settings; compact local observations are often better, which is a concrete guide for state design.
  • The topology-determined coordination difficulty score predicts per-topology learning hardness and can serve as a cheap diagnostic before running full MARL training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: an algorithm whose state representation is explicitly topology-invariant (for example, relative positions or graph embeddings instead of raw channel gains) should recover most of the multi-topology performance gap, confirming that raw channel-gain features are the bottleneck.
  • Because the global-state-to-local-observation comparison changes state dimension along with observability, the paper's 'partial observability is not critical' conclusion is partly confounded; a cleaner test would hide information while keeping state dimension fixed.
  • The same interference-game ladder could be reused to study topology robustness in millimeter-wave or massive MIMO vehicular systems, where interference patterns change even more sharply with geometry.
  • A natural benchmark extension is to allow topologies to change within an episode (vehicles moving during the decision horizon), which the current fixed-within-episode design does not exercise and which may further stress robustness.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This paper constructs a hierarchy of multi-agent interference games—NFIG, SIG SL (with and without fast fading), SIG ML, and POSIG—intended to isolate individual MARL challenges in C-V2X radio resource allocation, and benchmarks eight algorithms (IDQN, Hys-IDQN, VDN, QMIX, IA2C, MAA2C, IPPO, MAPPO) on SUMO-generated highway topologies with 4-, 8-, and 16-agent configurations. Returns are normalized against exhaustive-search or greedy-iterative approximate optima. The main empirical finding is a large performance drop in SIG ML relative to SIG SL, which the paper attributes to robustness and generalization across diverse vehicular topologies, with the largest drops reaching negative normalized returns for some value-based methods at 16 agents. The paper further reports that actor-critic methods, especially PPO variants, outperform value-based methods on the diverse-topology tasks, and concludes that robustness/generalization is the dominant challenge, while non-stationarity, coordination, large action space, and partial observability are not critical. The paper also recommends IPPO as a practical baseline and releases code, datasets, and benchmark tasks.

Significance. If the central conclusion is correct, the paper provides a useful reorientation for C-V2X MARL research: algorithm development should focus on topology robustness and zero-shot transfer rather than only on coordination or partial-observability mechanisms. The benchmark suite, SUMO-based datasets, and open-sourced code are concrete community assets, and the paper includes a thoughtful internal robustness-vs-generalization ablation for IDQN and IPPO. The consistent performance ordering across algorithms and scales is a valuable empirical contribution. However, the strength of the headline claim is weakened by a confounded partial-observability comparison and by the fact that the main SIG SL-to-SIG ML contrast changes several factors simultaneously. These issues do not undermine the robustness/generalization drop itself, but they do undermine the claim that partial observability is not critical, which is load-bearing for the challenge ranking.

major comments (2)
  1. [§VI.B.3 and §VII.D.1] The SIG ML vs. POSIG comparison does not isolate partial observability. SIG ML gives each agent a global state containing O(L^2) pairwise channel-gain and interference terms, while POSIG supplies the fixed-size local observation in Eq. (11). The comparison therefore changes observability and state dimensionality simultaneously. The paper's own explanation in §VII.D.1—that the POSIG advantage is due to the reduced state dimension—demonstrates the confound: the result cannot distinguish 'partial observability is harmless' from 'the high-dimensional global state is redundant/harmful.' The GRU/FC comparison in §VII.D.1 tests recurrence, not observability. A dimension-matched control (e.g., SIG ML with a compact global feature, or POSIG with the full global state appended) is needed before concluding that partial observability is 'not a critical limitation.' This conclusion is load-bearing fo
  2. [§VI.B.2, §VII.C.1, and Tables VII–X] The SIG SL FF vs. SIG ML comparison changes more than topology diversity. It also changes the number of training positional samples (one vs. 15,000/60,000), the training episode budgets (30,000 vs. 100,000 for actor-critic; 3,000 vs. 30,000 for value-based), and the sampling procedure. Thus the headline drop of up to 59 percentage points conflates the intended challenge with dataset size and computational budget. The internal ablation in §VII.C.1—comparing performance on random training samples vs. held-out topologies at 16 agents—is a good first step and supports a robustness effect, but it does not by itself quantify how much of the SIG SL-to-SIG ML drop is due to topology diversity versus increased dataset scale. An additional control that matches dataset size per topology or otherwise varies only the number/distribution of topologies would substantially strengthen the central claim.
minor comments (6)
  1. [§VI.A] Typo: 'with with three lanes' should be 'with three lanes.'
  2. [§VI.D] For SIG ML and POSIG, the normalization reuses approximate optimal returns from SIG SL NFF without recomputing baselines under the diverse-topology regime. Since the approximate per-step exhaustive/greedy baseline ignores exact queue dynamics and the early-completion bonus Z, the sensitivity of the normalized rankings to this baseline should be reported or at least discussed.
  3. [§VI.B.2] The statement that 15,000/60,000 training samples were 'experimentally determined' to yield the best performance is ambiguous. Was this selected using a validation set or the final test set? Using test performance for dataset-size selection could inflate reported results; please clarify and, if necessary, report sensitivity to dataset size.
  4. [§VII.A.2] The coordination difficulty score in Eq. (12) uses G_max^NE, G_min^NE, and G_mean^NE, but these are not defined precisely. Please specify how equilibrium returns are computed and what 'normalized equilibrium returns' means in this context.
  5. [Table V] Some normalized returns are below zero (e.g., IDQN/Hys-IDQN at 16 agents in SIG ML). This is not inherently problematic, but the text should comment on what it means for an algorithm to perform worse than the random policy used for normalization, especially since the random baseline is not variance-bounded.
  6. [§V.B.1 and §VII.D.2] The discussion of IGM violations in value decomposition would benefit from a concrete connection to the POSIG results: the assertion that 'errors compound through the mixing network' is plausible, but no diagnostic evidence is provided. This is a secondary point.

Circularity Check

0 steps flagged

No significant circularity: benchmark is empirical and normalization constants are environment-computed, not fitted predictions.

full rationale

This is an experimental benchmarking study rather than a derivation, and the paper's quantitative results are generated by simulator rollouts of eight algorithms, not derived from the assumptions. Normalized returns use G_min (random-policy return) and G_max (per-step exhaustive or greedy optimum) computed directly from the environment model (Section VI.D); neither is a parameter fitted to the evaluated policies, so the 'approximate optimal returns' are not relabeled predictions. The claim that robustness/generalization is the dominant challenge follows from observed SIG SL vs SIG ML return differences (Table V, Section VII.C), an ablation, not an identity. Self-citations ([1], [13], [26], [40]) supply prior channel/queue models and the earlier ICC version; they are externally published building blocks, are not used as a uniqueness theorem, and do not force the benchmark conclusions. One caveat belongs to correctness rather than circularity: Section VI.B.3 states 'Comparing SIG ML and POSIG isolates the effect of partial observability,' but SIG ML uses a global state scaling quadratically with agent count while POSIG uses fixed-size local observations (Eq. 11), so the comparison changes observability and state dimensionality simultaneously. Consequently Section VII.D.1's inference that 'partial observability is not a critical limitation' is confounded; however, this is an experimental-validity issue, not a circular reduction of outputs to inputs, and does not raise the circularity score.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new physical entities, forces, or conserved quantities are introduced; the NFIG/SIG/POSIG games are formal abstractions of existing C-V2X models. The paper's load-bearing choices are reward weights, the early-completion bonus, the horizon reduction, training budgets, and tuning procedure, plus the standard-model and representativeness assumptions.

free parameters (5)
  • Reward weights lambda_1/lambda_2 = NFIG: 0.1/0.9; SIG/POSIG: 0.2/1.8
    Hand-set to prioritize V2V CAM delivery over V2I throughput. Changing these weights changes the task objective and likely the difficulty ranking; they are not derived.
  • Early-completion bonus Z = 0.5
    Tuned hyper-parameter in the reward (Eq. 9) to encourage early CAM completion; directly shapes the return and what policies are optimal.
  • Time horizon reduction to 50 ms = 50 ms instead of 100 ms
    The authors assert queues are typically exhausted within 20 steps and reduce the horizon to accelerate training. If this is not accurate, the SIG results do not match the full control interval.
  • Training dataset size and episode budgets = 15k/60k samples; 30k/100k episodes depending on task and algorithm
    Dataset sizes and training episodes were determined experimentally and differ across tasks and algorithms. These differences confound cross-task comparisons of challenge difficulty.
  • Per-algorithm hyperparameters = Appendix A, Tables VII-X
    Learning rates, batch sizes, discount factors, entropy coefficients, etc. were tuned per task category by selecting the best over five seeds. Tuning effort affects the algorithm ranking and introduces selection bias.
axioms (4)
  • domain assumption 3GPP TR 36.885 and ETSI TR 103 766 channel/traffic models are appropriate for C-V2X RRA benchmarking
    Used to generate channel gains, path loss, vehicle speed, and density; all SINR and queue results rest on these standard models.
  • domain assumption The SUMO-generated topologies and the nine test samples are representative of the diversity of highway C-V2X interference conditions
    The robustness/generalization claim is about all diverse topologies, but only three density levels and three distance levels are tested. If these are not representative, the dominant-challenge conclusion may not generalize.
  • ad hoc to paper Per-step exhaustive search (4 agents) and greedy iterative assignment (8/16 agents) provide valid approximate optimal returns for normalization
    Normalized returns in SIG ML/POSIG reuse these approximations. If the greedy optimum is loose or mismatched to the fast-fading ML task, normalized scores are biased.
  • ad hoc to paper Adding or removing factors across NFIG/SIG/POSIG isolates the named MARL challenge
    The task hierarchy assumes each task differs from the previous only by the intended challenge. POSIG vs SIG ML also changes the state-space dimension and representation, so this axiom is violated in that comparison.

pith-pipeline@v1.3.0-alltime-deepseek · 21287 in / 12358 out tokens · 112810 ms · 2026-08-02T22:30:00.788986+00:00 · methodology

0 comments
read the original abstract

Radio resource allocation (RRA) is a critical function in cellular vehicle-to-everything (C-V2X) networks, where vehicles must share limited wireless resources to support safety-critical communications. Multi-agent reinforcement learning (MARL) has emerged as a promising approach for this problem. However, key MARL challenges, including non-stationarity, coordination difficulty, large action space, partial observability, and limited robustness and generalization, are often intertwined, making it difficult to assess their individual impact on performance in vehicular environments. Moreover, existing studies primarily focus on developing new algorithms, while systematic benchmarking and comparative analyses remain limited. To address this gap, we formulate C-V2X RRA as a hierarchy of multi-agent interference games that progressively introduce key MARL challenges. Based on this framework, we develop a suite of benchmark learning tasks and construct training and testing datasets from SUMO-generated highway traces with diverse vehicular topologies and interference conditions. Using the proposed benchmark, we evaluate representative MARL algorithms spanning value-based, actor-critic, Independent Learning (IL), and Centralized Training with Decentralized Execution (CTDE) paradigms. The results identify robustness and generalization across diverse vehicular topologies as the dominant challenge among those considered in this work, reducing average normalized return by up to 59 percentage points, and show that, on the most challenging task, the best actor-critic method outperforms the best value-based method by 42\%. By revealing the relative strengths and limitations of different MARL paradigms and open-sourcing the code, datasets, and benchmark suite, this work provides a systematic and reproducible foundation for evaluating and advancing MARL algorithms in vehicular networks.

Figures

Figures reproduced from arXiv: 2603.06607 by Kan Zheng, Lei Lei, Pranav Maheshwari, Sam Bellefeuille, Siyuan Wang.

Figure 1
Figure 1. Figure 1: Normalized returns in selected SIG SL NFF tasks. Shaded areas represent the 95% CI over five seeds. and IDQN (0.95), highlighting the advantage of value￾decomposition methods in capturing the relationship between individual and joint action values. In contrast, actor-critic CTDE algorithms show little advantage over their IL counter￾parts: MAPPO (0.98) and MAA2C (0.99) perform similarly to IPPO (0.97) and … view at source ↗
Figure 2
Figure 2. Figure 2: Normalized returns in selected SIG ML tasks. Shaded areas represent the 95% CI over five seeds. of robustness and generalization. As the number of agents increases, the diversity of interference patterns grows rapidly, making it increasingly challenging for learned policies to generalize to unseen topologies. 2) Comparison between different algorithms: a) Value-based vs. Actor-Critic: As shown in Table V, … view at source ↗
Figure 3
Figure 3. Figure 3: Normalized returns in selected POSIG tasks. Shaded areas represent the 95% CI over five seeds. The convergence curves of all algorithms for SIG SL NFF, SIG ML, and POSIG tasks are shown in [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 5 linked inside Pith

  1. [1]

    Multi-agent drl for resource allocation in vehicular networks: A comparative study,

    S. Wang, P. Maheshwari, L. Lei, J. Mei, and K. Zheng, “Multi-agent drl for resource allocation in vehicular networks: A comparative study,” in ICC 2025 - IEEE International Conference on Communications, 2025, pp. 1936–1941

    2025

  2. [2]

    Recent advances and future trends in vehicular communication systems: A comprehensive survey,

    Y . Liu, J. Wu, and J. Zhang, “Recent advances and future trends in vehicular communication systems: A comprehensive survey,”IEEE Communications Surveys & Tutorials, vol. 23, no. 1, pp. 38–84, 2021

    2021

  3. [3]

    Multiuser resource control with deep reinforcement learning in iot edge comput- ing,

    L. Lei, H. Xu, X. Xiong, K. Zheng, W. Xiang, and X. Wang, “Multiuser resource control with deep reinforcement learning in iot edge comput- ing,”IEEE Internet of Things Journal, vol. 6, no. 6, pp. 10 119–10 133, 2019

    2019

  4. [4]

    Deep reinforcement learning based resource allocation for v2v communications,

    H. Ye and G. Y . Li, “Deep reinforcement learning based resource allocation for v2v communications,”IEEE Transactions on Vehicular Technology, vol. 68, no. 4, pp. 3163–3173, 2019

    2019

  5. [5]

    S. V . Albrecht, F. Christianos, and L. Sch¨afer,Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024. [Online]. Available: https://www.marl-book.com

    2024

  6. [6]

    Spectrum sharing in vehicular networks based on multi-agent reinforcement learning,

    L. Liang, H. Ye, and G. Y . Li, “Spectrum sharing in vehicular networks based on multi-agent reinforcement learning,”IEEE Journal on Selected Areas in Communications, vol. 37, no. 10, pp. 2282–2292, 2019

    2019

  7. [7]

    Multi-agent RL enables decentralized spectrum access in vehicular networks,

    P. Xiang, H. Shan, M. Wang, Z. Xiang, and Z. Zhu, “Multi-agent RL enables decentralized spectrum access in vehicular networks,”IEEE Transactions on Vehicular Technology, vol. 70, no. 10, pp. 10 750– 10 762, 2021

    2021

  8. [8]

    Multi-agent reinforcement learning-based decentralized spectrum access in vehic- ular networks with emergent communication,

    P. Xiang, H. Shan, Z. Su, Z. Zhang, C. Chen, and E.-P. Li, “Multi-agent reinforcement learning-based decentralized spectrum access in vehic- ular networks with emergent communication,”IEEE Communications Letters, vol. 27, no. 1, pp. 195–199, 2023

    2023

  9. [9]

    Resource allocation in V2X communications based on multi-agent reinforcement learning with attention mechanism,

    Y . Ding, Y . Huang, L. Tang, X. Qin, and Z. Jia, “Resource allocation in V2X communications based on multi-agent reinforcement learning with attention mechanism,”Mathematics, vol. 10, no. 19, p. Art. no. 3415, 2022. [Online]. Available: https://www.mdpi.com/2227-7390/10/ 19/3415

    2022

  10. [10]

    Mean-field aided multi-agent reinforcement learning for resource allocation in vehicular networks,

    H. Zhang, C. Lu, H. Tang, X. Wei, L. Liang, L. Cheng, W. Ding, and Z. Han, “Mean-field aided multi-agent reinforcement learning for resource allocation in vehicular networks,”IEEE Internet of Things Journal, 2022

    2022

  11. [11]

    Graph neural networks and deep reinforcement learning based resource allocation for v2x communications,

    M. Ji, Q. Wu, P. Fan, N. Cheng, W. Chen, J. Wang, and K. B. Letaief, “Graph neural networks and deep reinforcement learning based resource allocation for v2x communications,” 2025. [Online]. Available: https://arxiv.org/abs/2407.06518

    Pith/arXiv arXiv 2025

  12. [12]

    Meta-reinforcement learning based resource allocation for dynamic v2x communications,

    Y . Yuan, G. Zheng, K.-K. Wong, and K. B. Letaief, “Meta-reinforcement learning based resource allocation for dynamic v2x communications,” IEEE Transactions on Vehicular Technology, vol. 70, no. 9, pp. 8964– 8977, 2021

    2021

  13. [13]

    Multitimescale control and communications with deep reinforcement learning—part ii: Control- aware radio resource allocation,

    L. Lei, T. Liu, K. Zheng, and X. Shen, “Multitimescale control and communications with deep reinforcement learning—part ii: Control- aware radio resource allocation,”IEEE Internet of Things Journal, vol. 11, no. 9, pp. 15 475–15 489, 2023

    2023

  14. [14]

    A review of cooperative multi-agent deep reinforcement learning,

    A. Oroojlooy and D. Hajinezhad, “A review of cooperative multi-agent deep reinforcement learning,”Applied Intelligence, vol. 53, no. 11, pp. 13 677–13 722, 2023

    2023

  15. [15]

    Bench- marking multi-agent deep reinforcement learning algorithms in cooper- ative tasks,

    G. Papoudakis, F. Christianos, L. Sch ¨afer, and S. V . Albrecht, “Bench- marking multi-agent deep reinforcement learning algorithms in cooper- ative tasks,”arXiv preprint arXiv:2006.07869, 2020

    Pith/arXiv arXiv 2006

  16. [16]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017, [Online]. Available: https://arxiv.org/abs/1707.06347

    Pith/arXiv arXiv 2017

  17. [17]

    Deep reinforcement learning based resource allocation with heterogeneous qos for cellular v2x,

    J. Tian, Y . Shi, X. Tong, S. Chen, and R. Zhao, “Deep reinforcement learning based resource allocation with heterogeneous qos for cellular v2x,” in2023 IEEE Wireless Communications and Networking Confer- ence (WCNC), 2023, pp. 1–6

    2023

  18. [18]

    Spectrum-energy-efficient mode selection and resource allocation for heterogeneous v2x networks: A fed- erated multi-agent deep reinforcement learning approach,

    J. Gui, L. Lin, X. Deng, and L. Cai, “Spectrum-energy-efficient mode selection and resource allocation for heterogeneous v2x networks: A fed- erated multi-agent deep reinforcement learning approach,”IEEE/ACM Transactions on Networking, vol. 32, no. 3, pp. 2689–2704, 2024

    2024

  19. [19]

    A hybrid multi-agent reinforcement learning approach for spectrum sharing in vehicular networks,

    M. Jamal, Z. Ullah, M. Naeem, M. Abbas, and A. Coronato, “A hybrid multi-agent reinforcement learning approach for spectrum sharing in vehicular networks,”Future Internet, vol. 16, no. 5, p. 152, 2024

    2024

  20. [20]

    Federated reinforcement learning for resource allocation in v2x networks,

    K. Xu, S. Zhou, and G. Y . Li, “Federated reinforcement learning for resource allocation in v2x networks,”IEEE Journal of Selected Topics in Signal Processing, 2024

    2024

  21. [21]

    Semantic- aware resource allocation based on deep reinforcement learning for 5g- v2x hetnets,

    Z. Shao, Q. Wu, P. Fan, N. Cheng, Q. Fan, and J. Wang, “Semantic- aware resource allocation based on deep reinforcement learning for 5g- v2x hetnets,”IEEE Communications Letters, 2024

    2024

  22. [22]

    Deep reinforcement learning for multi- objective resource allocation in multi-platoon cooperative vehicular networks,

    Y . Xu, K. Zhu, H. Xu, and J. Ji, “Deep reinforcement learning for multi- objective resource allocation in multi-platoon cooperative vehicular networks,”IEEE Transactions on Wireless Communications, 2023

    2023

  23. [23]

    Enabling adaptive optimization of energy efficiency and quality of service in nr-v2x communications via multiagent deep reinforcement learning,

    Y . Song, Y . Xiao, and J. Liu, “Enabling adaptive optimization of energy efficiency and quality of service in nr-v2x communications via multiagent deep reinforcement learning,”IEEE Internet of Things Journal, vol. 12, no. 4, pp. 4022–4037, 2025

    2025

  24. [24]

    Aoi-aware resource allocation for platoon-based c-v2x networks via multi-agent multi-task reinforcement learning,

    M. Parvini, M. R. Javan, N. Mokari, B. Abbasi, and E. A. Jorswieck, “Aoi-aware resource allocation for platoon-based c-v2x networks via multi-agent multi-task reinforcement learning,”IEEE Transactions on Vehicular Technology, vol. 72, no. 8, pp. 9880–9896, 2023

    2023

  25. [25]

    Semantic-aware resource management for c-v2x platooning via multi-agent reinforcement learning,

    W. Zhang, Q. Wu, P. Fan, K. Wang, N. Cheng, W. Chen, and K. B. Letaief, “Semantic-aware resource management for c-v2x platooning via multi-agent reinforcement learning,” 2025. [Online]. Available: https://arxiv.org/abs/2411.04672

    Pith/arXiv arXiv 2025

  26. [26]

    Deep reinforcement learning for autonomous internet of things: Model, ap- plications and challenges,

    L. Lei, Y . Tan, K. Zheng, S. Liu, K. Zhang, and X. Shen, “Deep reinforcement learning for autonomous internet of things: Model, ap- plications and challenges,”IEEE Communications Surveys & Tutorials, vol. 22, no. 3, pp. 1722–1760, 2020

    2020

  27. [27]

    Multi-agent actor-critic for mixed cooperative-competitive environ- ments,

    R. Lowe, Y . I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environ- ments,”Advances in neural information processing systems, vol. 30, 2017

    2017

  28. [28]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” inInternational Conference on Learning Representations, 2016

    2016

  29. [29]

    Technical Specification Group Radio Access Network; Study LTE- Based V2X Services; (Release 14),

    “Technical Specification Group Radio Access Network; Study LTE- Based V2X Services; (Release 14),” 3rd Generation Partnership Project (3GPP), Tech. Rep. 3GPP TR 36.885 V14.0.0, Jun. 2016, release 14. [Online]. Available: https://www.3gpp.org/ftp/Specs/archive/36 series/ 36.885/36885-f00.zip

    2016

  30. [30]

    TR 103 766, no

    “Intelligent transport systems (its); pre-standardization study on co- channel co-existence between ieee- and 3gpp- based its technologies in the 5 855 mhz - 5 925 mhz frequency band,”European Telecommu- nications Standards Institute (ETSI) Technical Report, vol. TR 103 766, no. V1.1.1, 2021

    2021

  31. [31]

    Delay- optimal dynamic mode selection and resource allocation in device-to- device communications—part ii: Practical algorithm,

    L. Lei, Y . Kuang, N. Cheng, X. Shen, Z. Zhong, and C. Lin, “Delay- optimal dynamic mode selection and resource allocation in device-to- device communications—part ii: Practical algorithm,”IEEE Transac- tions on Vehicular Technology, vol. 65, no. 5, pp. 3491–3505, 2015

    2015

  32. [32]

    Lenient learning in independent-learner stochastic cooperative games,

    E. Wei and S. Luke, “Lenient learning in independent-learner stochastic cooperative games,”Journal of Machine Learning Research, vol. 17, no. 84, pp. 1–42, 2016. [Online]. Available: http://jmlr.org/papers/v17/ 15-417.html

    2016

  33. [33]

    Leveraging procedural generation to benchmark reinforcement learning,

    K. Cobbe, C. Hesse, J. Hilton, and J. Schulman, “Leveraging procedural generation to benchmark reinforcement learning,” inProceedings of the 37th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 119. PMLR, 2020, pp. 2048–2056

    2020

  34. [34]

    Human-level control through deep reinforcement learning,

    V . M. et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, 2015

    2015

  35. [35]

    Hysteretic q-learning: an algorithm for decentralized reinforcement learning in cooperative multi-agent teams,

    L. Matignon, G. J. Laurent, and N. Le Fort-Piat, “Hysteretic q-learning: an algorithm for decentralized reinforcement learning in cooperative multi-agent teams,” in2007 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2007, pp. 64–69

    2007

  36. [36]

    Asynchronous methods for deep reinforcement learning,

    V . Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” inProceedings of The 33rd International Con- ference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 48. PMLR, 2016, pp. 1928–1937

    2016

  37. [37]

    Value-decomposition networks for cooperative multi-agent learning,

    P. Sunehag, G. Lever, C. De Witt, T. Lillicrap, D. Balduzzi, D. Reichert, V . Zambaldi, K. Tuyls, and T. Graepel, “Value-decomposition networks for cooperative multi-agent learning,” inProceedings of the 16th Inter- national Conference on Autonomous Agents and MultiAgent Systems, 2017, pp. 2085–2087

    2017

  38. [38]

    Qmix: Monotonic value function factorization for deep multi-agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. De Witt, G. Farquhar, N. Nardelli, T. Rud- ner, and S. Whiteson, “Qmix: Monotonic value function factorization for deep multi-agent reinforcement learning,” inProceedings of the 35th International Conference on Machine Learning, 2018, pp. 4295–4304

    2018

  39. [39]

    The surprising effectiveness of ppo in cooperative, multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of ppo in cooperative, multi-agent games,”

  40. [40]

    Performance analysis of device-to-device communications with dynamic interference using stochastic petri nets,

    L. Lei, Y . Zhang, X. S. Shen, C. Lin, and Z. Zhong, “Performance analysis of device-to-device communications with dynamic interference using stochastic petri nets,”IEEE transactions on wireless communica- tions, vol. 12, no. 12, pp. 6121–6141, 2013

    2013

  41. [2022]

    Available: https://arxiv.org/abs/2103.01955

    [Online]. Available: https://arxiv.org/abs/2103.01955

    Pith/arXiv arXiv