Pith. sign in

REVIEW 4 major objections 5 minor 49 references

Explainable Multi-Agent Reinforcement Learning for Extended Reality Codec Adaptation

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that decomposing rewards in value-function-factorization MARL reveals the packet delivery ratio as the primary driver of optimal XR codec adaptation, and that an adaptive multi-headed QMIX variant exploiting this insight…

desk verdict Useful engineering extension of reward decomposition to MARL, but the headline 'PDR is primary' claim is scale-dependent and not yet supported. read the letter →

arxiv 2411.14264 v1 pith:OLIZEFOC submitted 2024-11-21 cs.NI

Pedro Enrique Iturria-Rivera , Raimundas Gaigalas , Medhat Elsayed , Majid Bavand , Yigit Ozcan , Melike Erol-Kantarci This is my paper · ORCID
classification cs.NI
keywords Explainablemulti-agentreinforcementlearningRewarddecompositionValuefunctionfactorizationXRcodecadaptation5G/6GnetworksQualityofexperienceRDFX
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that reward decomposition can make multi-agent reinforcement learning explainable in a wireless setting, and that the resulting explanations are not just descriptive but useful for improving policy performance. Focusing on extended reality (XR) codec adaptation over 5G, the authors split the joint reward into delay, packet delivery ratio (PDR), and data-rate ratio components and modify three value-function-factorization algorithms, VDN, QMIX, and QTRAN, to output separate Q-values for each component. They claim that the PDR reward is the primary contributor to optimal performance compared with the initial composite reward that also included delay and data-rate terms. They also introduce a multi-headed architecture to cut computation, a metric called RDFX for explaining codec parameter choices, and an adaptive weighting scheme that uses reward means and variances during training. If these claims hold, network operators can use the decomposition both to explain why a codec agent chooses an action and to prune or reweight reward components accordingly.

What carries the argument

The machinery is reward decomposition carried through value function factorization: each agent outputs C separate action-value heads, one per reward component, and these are combined per component into joint Q-values via additive summation for VDN or per-component mixing networks for QMIX and QTRAN. This rests on Theorem 1, which states that the joint action-value function factorizes as a weighted sum of per-agent, per-component Q-functions. Multi-headed shared layers reduce the number of networks, while RDFX defines the difference between the chosen action's Q-value and the expected Q-value of increasing or decreasing the codec parameter, and the adaptive loss uses weights computed from the mean and variance of each episode's reward sequence, rebalancing gradient training rates.

What would settle it

Retrain the same agents after swapping the reward scales, for example making the delay reward finely graded and the PDR reward coarse; if the relative Q-value ranking flips while network KPIs stay the same, the 'PDR is primary' conclusion is an artifact of reward scaling rather than an intrinsic property of the task.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a composite reward in value-function-factorization MARL can be decomposed into per-component Q-functions, and the resulting per-component values carry actionable information about reward design. Across agents, distances, and algorithms, the PDR component Q2 has the highest Q-value, with delay second and data-rate ratio lowest; MSX analysis often requires only the PDR component to explain the chosen action. The authors leverage this observation online with adaptive weights, giving the best algorithm, MHA-QMIX, average gains over the Adjust Packet Size baseline of up to 10.7% in XR index, 41.4% in jitter, 33.3% in delay, and 67.9% in packet loss ratio.

Load-bearing premise

The load-bearing premise is that the three decomposed Q-value heads are directly comparable in magnitude, so the largest one marks the truly most important reward rather than merely the reward with the most favorable hand-chosen scale, and that the multi-agent transition function splits additively across agents as assumed in the theorem's proof.

Editorial extensions

If this is right

  • Reward designers for XR codec adaptation should treat PDR as the dominant objective, with delay secondary and data-rate ratio least important, if the decomposition reflects true contribution.
  • The adaptive mechanism can replace post-hoc explainability: reward importance weights computed during training can be fed back into the loss without a separate explanation phase.
  • Multi-headed architectures give comparable or better KPIs than vanilla decomposed versions while reducing the number of networks, so explainability need not cost extra inference structures.
  • The decomposition framework and RDFX metric carry over to other wireless parameter-selection problems where actions are ordered values rather than discrete semantic actions.
  • Benchmarking against the Adjust Packet Size baseline at 400 m shows the largest gains, suggesting the biggest benefit appears in path-loss-limited conditions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The paper's conclusion that PDR is primary rests on comparing Q-values whose reward scales are hand-chosen, so a normalized-reward ablation would determine whether the conclusion is about the environment or about the reward encoding.
  • Inference: A direct test of the explanation is to train a PDR-only variant and compare it against the composite reward; the decomposition predicts comparable KPI performance with fewer reward components.
  • Inference: The adaptive weighting via mean-plus-sinh-variance may be the more general contribution, since it is a parameter-free way to balance multiple reward terms online and could apply beyond MARL to any multi-objective reinforcement learning loss.
  • Inference: If deployed online, per-episode reward-importance weights could interact with off-policy replay because the loss surface changes as the weights move; the paper does not analyze this stability question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes explainable multi-agent reinforcement learning (MARL) for XR codec adaptation in 5G networks. It extends reward decomposition to value-function-factorization MARL algorithms (VDN, QMIX, QTRAN), introduces multi-headed and adaptive variants, defines a new explainability metric RDFX, and evaluates the methods in an ns-3 New Radio simulator. The central claims are that, for XR codec adaptation, the Packet Delivery Ratio (PDR) reward component is the primary contributor to performance, and that the best proposed algorithm (MHA-QMIX) achieves notable KPI gains over an Adjust Packet Size baseline.

Significance. If the PDR-importance finding were robust, it would provide concrete guidance for reward design in wireless XR applications and would give practitioners a principled reason to emphasize reliability terms over throughput and delay terms. The paper also makes several constructive algorithmic contributions: a multi-headed architecture that reduces the parameter overhead of decomposed value-function-factorization methods, an online adaptive weighting scheme inspired by multi-task learning, and a new metric (RDFX) for explaining action preferences over ordered action sets. The ns-3 evaluation covers a realistic 5G XR scenario with three user types and three distances. However, these strengths are partly offset by the fragility of the headline interpretability conclusion, which rests on cross-component comparisons of Q-values whose reward scales are not commensurable. The empirical KPI gains are plausible but lack statistical support, and no comparison with non-decomposed versions of the same MARL algorithms is reported, leaving the effect of reward decomposition itself unclear.

major comments (4)
  1. [Section VI-C, Fig. 7, Eqs. (15)-(17)] The claim that PDR is the primary contributor to optimal performance, based on Q2 > Q1 > Q3 in Fig. 7, is not established because the decomposed Q-values are trained against reward components with different hand-chosen scales and saturation patterns. The delay reward in Eq. (15) is a staircase taking values in {1, 0.75, 0.5, 0.25, 0}, the PDR reward in Eq. (16) takes {1, 0.5, 0}, and the data-rate reward in Eq. (17) is a continuous ratio in [0,1]. Since a decomposed Q-function is approximately proportional to the scale of its target reward, the observed ordering may simply reflect that the PDR reward is near 1 more often in the simulated favorable regime, rather than a causal dominance of PDR in the optimal policy. To support the headline conclusion, the authors should either normalize the reward components to comparable scales before comparing Q-values, run counterfactual ablations (e.g., train without the PDR component, or re-weight components so that all have similar ranges), or both. Without such evidence, the abstract and Section VII statements about PDR being "the primary contributor" are scale artifacts.
  2. [Appendix A, Definition A.1 and Theorem 1] The theoretical foundation of the paper depends on an additive decomposability assumption that is not satisfied by the system model used in the simulations. Definition A.1 (Eq. (38)-(39)) assumes both the team reward and the transition function decompose additively over agents, i.e., P(s'|s,a) = Σ_i F_i(s'|s_i,a_i). In the simulated 5G scenario, agents share a common wireless channel, so the next state cannot be written as a sum of per-agent transition functions. In addition, the team rewards used in the experiments are minima over agents, as defined in Eqs. (18)-(20), not the sum required by Eq. (38). Therefore Theorem 1, as proved in Appendix A, does not apply to the actual problem setting. The authors should either relax the theorem to cover the min-based and shared-channel case, clearly state that Theorem 1 holds only for an idealized additive model and is not the basis for the simulation results, or provide a separate argument for the simulation setting.
  3. [Section V-H, Eq. (35), Figs. 12-13] The adaptive importance weights in Eq. (35) are computed as a softmax over mean plus sinh(std) of the raw reward components. Because the raw rewards have different scales, as described above, this weight update inherently upweights components with larger mean or variance. The observation in Figs. 12-13 that w2 (PDR) dominates is therefore partly a consequence of the reward construction and does not provide independent confirmation of PDR's causal importance. To make the adaptive mechanism a valid source of evidence about reward importance, the authors should demonstrate that the weight dynamics are stable under rescaling of the reward components, or they should compute weights on normalized reward signals. At minimum, the text should acknowledge this scale dependence and avoid presenting the adaptive weights as corroboration of the PDR-primary conclusion.
  4. [Section VI-E, Table IV, Fig. 14] The performance results in Table IV are reported as single percentage gains without error bars, confidence intervals, or information about the number of random seeds used. The abstract and Section VI-E describe these gains as "significant average gains," but no statistical test or variance measure supports that wording. Additionally, the paper does not compare the decomposed algorithms against their non-decomposed counterparts (VDN, QMIX, QTRAN), so it is unclear whether reward decomposition itself degrades, preserves, or improves learning. The authors should report mean and standard deviation over multiple independent training runs and, ideally, include the non-decomposed baselines so that the effect of the proposed architectural and adaptive changes is isolated.
minor comments (5)
  1. [Abstract] The phrase "compared to the initial composite reward" is ambiguous; the intended contrast is between the PDR component and a composite reward that also includes delay and data-rate components, but the sentence as written suggests a comparison between the PDR reward and the initial composite reward. Please rephrase for clarity.
  2. [Algorithm 1, line 10] The condition "if ϵ ≥ N(0, 1)" uses the normal-distribution notation in what appears to be an epsilon-greedy exploration check. This is likely a typographical error; the intended condition is probably a uniform random draw compared against epsilon. Please correct the pseudocode.
  3. [Section III-F] The definitions of MSX+ and MSX- would benefit from a worked example or a small illustrative figure. The set notation in Eqs. (10)-(12) is dense, and the accompanying text does not fully explain how the sets X+ and X- are constructed in practice.
  4. [Section V-C, Eq. (23)] The potential-shaping function ϕ(x, y, z) appears to have three arguments in its definition but is later evaluated as ϕ(x, y) with two arguments, and the text states "we assume y = z_norm." Please align the notation to avoid confusion.
  5. [Figures 2-4] The architecture diagrams are very dense and the small font makes them difficult to read, especially the multi-headed variants in Fig. 4. Consider enlarging the panels or separating them into multiple figures for the camera-ready version.

Circularity Check

3 steps flagged · score 6.0 of 10

The 'PDR is primary contributor' claim is substantially an artifact of hand-set reward scales and of an adaptive weight formula that up-weights numerically large reward components; Appendix A's decomposition theorem is a definitional restatement.

  1. self definitional [Section V-C (Eqs. 15-17) and Section VI-C (Fig. 7, text after Fig. 7)]
    "Upon initial examination of Fig. 7(a), corresponding to the AR agent, it is evident that Q2 (PDR) exhibits the highest Q-value, serving as the primary contributor in decision-making, followed by Q1 (Delay). ... Despite this exception, we can conclude that, in general, PDR holds the highest importance."

    The paper reads the primacy of PDR directly off the relative magnitudes of decomposed Q-values Q1, Q2, Q3. But by the paper's own definitions, Q_c is the expected discounted sum of reward component r_c (Eqs. 8 and 43), and the r_c are hand-set on different scales: delay is a staircase taking {1, 0.75, 0.5, 0.25, 0}, PDR is {1, 0.5, 0}, and DRR is a continuous throughput ratio in [0,1]. Thus the statement 'Q2 > Q1 > Q3' is, to first order, a comparison of chosen reward units, not a measured causal contribution to optimal performance. No normalization, counterfactual ablation, or reward-scale invariance test is provided, so the headline conclusion is partly constructed by the reward design rather than discovered.

  2. fitted input called prediction [Section V-H (Eqs. 31, 35) and Section VI-D (text after Fig. 12)]
    "Le = 1/C sum_c wc lc,e Lc_ι (31) ... lc,e = exp(¯rc,e + sinh(σc,e)) / sum_c exp(¯rc,e + sinh(σc,e)) (35). ... The weights computed per episode tend to have a preference for the best reward in terms of magnitude, as observed in Fig. 12."

    The adaptive importance weights l_c,e are computed directly from the mean and variance of the same hand-scaled reward components r_c (Eqs. 33-35), and then those weights multiply the per-component TD losses in Eq. 31. Therefore a reward component that was assigned numerically larger values, such as PDR's frequent 1.0, is up-weighted by construction. The subsequent claim that the adaptive algorithm 'leverages reward decomposition' to determine reward importance is circular: the importance measure is a deterministic function of the reward scale that the authors themselves chose, not an independent explanation of why PDR matters.

1 more flagged steps
  1. self definitional [Appendix A, Definition A.1 / Eq. (39), and Theorem 1]
    "Definition A.1: A multi-agent Markov Game MG(S, A, P, r, N, γ, d0) is considered a decomposable game if its team objective function r : S × A takes the form, r(s, a) = r1(s1, a1) + ... + rN (sN , aN ), (38) ... This definition allows the decomposition of the transition function P as, P (s′|s, a) = F1(s′|s1, a1) + ... + FN (s′|sN , aN ), (39)"

    Theorem 1 is presented as a theoretical guarantee that joint Q-values can be decomposed in MARL, but its proof (Appendix A) assumes exactly the conclusion: the team reward is defined to be a sum of independent agent rewards (Eq. 38), and the transition function is assumed to be a sum of independent per-agent transitions (Eq. 39). Substituting these assumed additive forms into the Bellman equation recovers an additive Q (Eq. 47), so the theorem is a restatement of the definition rather than an independent result. Moreover, Eq. 39 does not hold for the shared 5G wireless channel used in the paper, so this definitional assumption cannot lend support to the architecture.

full rationale

The paper's network-performance comparison against the external APS baseline is a genuine benchmark and is not circular: the measured KPI gains of MHA-QMIX (Table IV) are empirical results against an external algorithm. The circularity is concentrated in the explainability claim. The decomposed Q-values are defined as expected discounted sums of the hand-set reward components in Eqs. (15)-(17), so the relative magnitudes used to declare PDR 'the primary contributor' inherit the arbitrary reward scales; no normalization or counterfactual ablation is performed. The adaptive weighting in Eq. (35) is explicitly a function of the mean and variance of those same reward components, and Eq. (31) multiplies the loss by those weights, so PDR dominance is partly enforced by construction. Appendix A's Theorem 1 is a self-definitional restatement: it assumes additive reward and transition decompositions (Definition A.1) and derives additive Q-values. These issues do not make every result in the paper circular, but they do mean the central explainability conclusion is substantially an artifact of reward design rather than an independently established finding.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rely on three main types of input: (1) the decomposable game assumption in the proof, which is introduced ad hoc and likely false for wireless channels; (2) the existing VFF theory from VDN/QMIX/QTRAN, which the paper takes as given; (3) the hand-chosen reward scales that make the PDR component inherently larger. The free parameters are the reward thresholds and shaping factors that are not fitted to data but are selected by the authors and inherited from the XR index.

free parameters (4)
  • Delay reward thresholds = 7, 10, 13, 20 s
    Staircase intervals in Eq. (15) chosen from Table I, [45]; affect relative magnitudes of Q1 and hence the PDR-dominance conclusion.
  • PDR reward thresholds = 99%, 95%
    Eq. (16) from Table I; the mostly-1 reward for PDR inflates Q2 relative to other components.
  • Reward shaping discount factor gamma_p = 1
    Set in Table II; controls the potential-based shaping term in Eq. (22).
  • QTRAN loss weights lambda_opt and lambda_nopt = 1, 1
    Used in Eq. (27), set without tuning; affect the QTRAN loss landscape and thus the conclusions about QTRAN variants.
assumptions (4)
  • ad hoc to paper Definition A.1: decomposable game, where team reward and transition function decompose additively over agents (Eqs. 38-39).
    Assumed in the proof of Theorem 1 (Appendix A); not justified for a shared wireless channel where user transmissions interfere and transition probabilities do not split into per-agent sums.
  • domain assumption The joint Q-function is factorizable as per VDN/QMIX/QTRAN (IGM and monotonicity assumptions).
    The paper takes the established VFF theory from [30]-[32] as given and builds its decomposed architectures on top of it.
  • ad hoc to paper The reward components' magnitudes are comparable when interpreting Q-values.
    The paper treats Q1, Q2, Q3 magnitudes as directly indicative of importance, but the reward functions in Section V.C have different scales and shapes, making this comparison nontrivial.
  • domain assumption The XR index mapping from Table I / [45] is a valid proxy for QoE.
    The reward thresholds inherit from the XR Quality Index literature; the paper does not validate this mapping for its specific traffic mix.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Explainable Multi-Agent Reinforcement Learning for Extended Reality Codec Adaptation." pith.science (2026). https://pith.science/paper/OLIZEFOC

@misc{pith2026241114264,
  author       = {Pith},
  title        = {Pith review of: Explainable Multi-Agent Reinforcement Learning for Extended Reality Codec Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OLIZEFOC}},
  note         = {Machine review of arXiv:2411.14264}
}
read the original abstract

Extended Reality (XR) services are set to transform applications over 5th and 6th generation wireless networks, delivering immersive experiences. Concurrently, Artificial Intelligence (AI) advancements have expanded their role in wireless networks, however, trust and transparency in AI remain to be strengthened. Thus, providing explanations for AI-enabled systems can enhance trust. We introduce Value Function Factorization (VFF)-based Explainable (X) Multi-Agent Reinforcement Learning (MARL) algorithms, explaining reward design in XR codec adaptation through reward decomposition. We contribute four enhancements to XMARL algorithms. Firstly, we detail architectural modifications to enable reward decomposition in VFF-based MARL algorithms: Value Decomposition Networks (VDN), Mixture of Q-Values (QMIX), and Q-Transformation (Q-TRAN). Secondly, inspired by multi-task learning, we reduce the overhead of vanilla XMARL algorithms. Thirdly, we propose a new explainability metric, Reward Difference Fluctuation Explanation (RDFX), suitable for problems with adjustable parameters. Lastly, we propose adaptive XMARL, leveraging network gradients and reward decomposition for improved action selection. Simulation results indicate that, in XR codec adaptation, the Packet Delivery Ratio reward is the primary contributor to optimal performance compared to the initial composite reward, which included delay and Data Rate Ratio components. Modifications to VFF-based XMARL algorithms, incorporating multi-headed structures and adaptive loss functions, enable the best-performing algorithm, Multi-Headed Adaptive (MHA)-QMIX, to achieve significant average gains over the Adjust Packet Size baseline up to 10.7%, 41.4%, 33.3%, and 67.9% in XR index, jitter, delay, and Packet Loss Ratio (PLR), respectively.

Figures

Figures reproduced from arXiv: 2411.14264 by the authors.

Figure 1
Figure 1. High-level taxonomy of proposed algorithms. The red colored-dashed boxes depict [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Internal structure of the adapted explainable MARL algorithms: [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Internal structure of the Multi-Headed Explainable MARL algorithms: [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Internal structure and loss calculation of the Multi-Headed Decomposed QTRAN [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Summary of the importance gradient mechanism. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 7
Figure 7. Figure 7: Average Q-Value functions decompositions for DQTRAN, DQMIX, and DVDN for three distances [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Average RDFX ∆c(·, Qc, “inc”) for DQTRAN, DQMIX, and DVDN for three distances 200, 300, 400 m: (a) AR agent, (b) VR agent, and (c) CG agent. 200 300 400 Distance (m) (a) 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 RDFX 200 300 400 Distance (m) (b) 0 1 2 3 4 RDFX 200 300 400 Distan…
Figure 9
Figure 9. Figure 9: Average RDFX ∆c(·, Qc, “dec”) for DQTRAN, DQMIX, and DVDN for three distances 200, 300, 400 m: (a) AR agent, (b) VR agent, and (c) CG agent. 200 300 400 Distance (m) (a) 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 MSX+ 200 300 400 Distance (m) (b) 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4…
Figure 10
Figure 10. Figure 10: Average MSX+(“inc”) for DQTRAN, DQMIX, and DVDN for three distances 200, 300, 400 m: (a) AR agent, (b) VR agent, and (c) CG agent. D. Importance reward metric In this subsection, we present the results that led us to the design of the importance reward metric utilized…
Figure 11
Figure 11. Figure 11: Average MSX+(“dec”) for DQTRAN, DQMIX, and DVDN for three distances 200, 300, 400 m: (a) AR agent, (b) VR agent, and (c) CG agent. 0 2000 4000 6000 Episodes 0.0 0.2 0.4 0.6 0.8 1.0 Weight MHA-DVDN (w1) MHA-DVDN (w2) MHA-DVDN (w3) [PITH_FULL_IMAGE:figures/full_fig_p01…
Figure 12
Figure 12. Figure 12: Behavior of importance weights in Eq. (36) when considering no variance. w1, w2, w3 corresponds to the weights of the reward components, delay, PDR, and DRR, respectively. The figure presents the behavior of the weights for the Multi-Headed Adaptive DVDN algorithm whe…
Figure 13
Figure 13. Figure 13: Behavior of importance weights in Eq. (35) when considering variance. w1, w2, w3 corresponds to the weights of the reward components, delay, PDR, and DRR, respectively. The figure presents the behavior of the weights for the Multi-Headed Adaptive DVDN algorithm when u…
Figure 14
Figure 14. Figure 14: Performance comparison between the baseline APS and the proposed decomposed reward MARL algorithms DQTRAN, DQMIX, and DVDN for three distances [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 39 canonical work pages

  1. [1]

    Peeking inside the black-box: a survey on explainable artificial intelligence (XAI),

    A. Adadi and M. Berrada, “Peeking inside the black-box: a survey on explainable artificial intelligence (XAI),” IEEE access , vol. 6, pp. 52 138–52 160, 2018

    work page 2018

  2. [2]

    The Artificial Intelligence and Data Act (AIDA) – Companion document,

    G. of Canada, “The Artificial Intelligence and Data Act (AIDA) – Companion document,” https: //ised-isde.canada.ca/site/innovation-better-canada/en/ artificial-intelligence-and-data-act-aida-companion-document, June 2022, (Accessed on 04/26/2024)

    work page 2022

  3. [3]

    A survey on the explainability of su- pervised machine learning,

    N. Burkart and M. F. Huber, “A survey on the explainability of su- pervised machine learning,” Journal of Artificial Intelligence Research , vol. 70, pp. 245–317, 2021

    work page 2021

  4. [4]

    Explainable deep reinforcement learning: state of the art and challenges,

    G. A. V ouros, “Explainable deep reinforcement learning: state of the art and challenges,” ACM Computing Surveys, vol. 55, no. 5, pp. 1–39, 2022

    work page 2022

  5. [5]

    Explainability in deep reinforcement learning: A review into current methods and applications,

    T. Hickling, A. Zenati, N. Aouf, and P. Spencer, “Explainability in deep reinforcement learning: A review into current methods and applications,” ACM Computing Surveys , vol. 56, no. 5, pp. 1–35, 2023

    work page 2023

  6. [6]

    Explainable reinforcement learning: A survey and comparative review,

    S. Milani, N. Topin, M. Veloso, and F. Fang, “Explainable reinforcement learning: A survey and comparative review,” ACM Computing Surveys , vol. 56, no. 7, pp. 1–36, 2024

    2024

  7. [7]

    Explainable multi-agent reinforce- ment learning for temporal queries,

    K. Boggess, S. Kraus, and L. Feng, “Explainable multi-agent reinforce- ment learning for temporal queries,” arXiv preprint arXiv:2305.10378 , 2023

    arXiv 2023

  8. [8]

    Digital-twin- enabled 6G: Vision, architectural trends, and future directions,

    L. U. Khan, W. Saad, D. Niyato, Z. Han, and C. S. Hong, “Digital-twin- enabled 6G: Vision, architectural trends, and future directions,” IEEE Commun. Mag., vol. 60, no. 1, pp. 74–80, 2022

    work page 2022

Show all 49 references
  1. [9]

    5G; QoE parameters and metrics relevant to the Virtual Reality (VR) user experience ,

    3GPP, “5G; QoE parameters and metrics relevant to the Virtual Reality (VR) user experience ,” 3rd Generation Partnership Project (3GPP), Technical Report (TR) 26.919, Nov. 2020, version 16.1.0. [Online]. Available: https://www.etsi.org/deliver/etsi tr/126900 126999/ 126929/16....

    2020

  2. [10]

    5G; Extended Reality (XR) in 5G,

    3GPP, “5G; Extended Reality (XR) in 5G,” 3rd Generation Partnership Project (3GPP), Technical Report (TR) 26.928, Nov. 2020, version 16.0.0. [Online]. Available: https://www.etsi.org/deliver/etsi tr/126900 126999/126928/16.00.00 60/tr 126928v160000p.pdf

    2020

  3. [11]

    6G: The next Giant leap for AI and ML,

    A. Bang, K. K. Kamal, P. Joshi, and K. Bhatia, “6G: The next Giant leap for AI and ML,” Procedia Computer Science , vol. 218, pp. 310–317, 2023

    2023

  4. [12]

    Five research insights on 6G and XR,

    Ericsson, “Five research insights on 6G and XR,” https://www.ericsson. com/en/blog/2023/5/five-research-insights-on-6g-xr, May 2023, (Ac- cessed on 04/26/2024)

    2023

  5. [13]

    Explainable reinforcement learning via reward decomposition,

    Z. Juozapaitis, A. Koul, A. Fern, M. Erwig, and F. Doshi-Velez, “Explainable reinforcement learning via reward decomposition,” in IJCAI/ECAI Workshop on explainable artificial intelligence , 2019

    2019

  6. [14]

    G.1035: Influencing factors on quality of experience for vir- tual reality services,

    ITU-T, “G.1035: Influencing factors on quality of experience for vir- tual reality services,” https://www.itu.int/rec/T-REC-G.1035-202111-I, 2021

    2021

  7. [15]

    Enhancing 5G QoS Management for XR Traffic Through XR Loopback Mechanism,

    B. Bojovic, S. Lagen, K. Koutlia, X. Zhang, P. Wang, and L. Yu, “Enhancing 5G QoS Management for XR Traffic Through XR Loopback Mechanism,” IEEE Journal on Selected Areas in Communications, 2023

    2023

  8. [16]

    QoS Management for XR Traffic in 5G NR: A Multi-Layer System View & End-to-End Evaluation,

    S. Lagen, B. Bojovic, K. Koutlia, X. Zhang, P. Wang, and Q. Qu, “QoS Management for XR Traffic in 5G NR: A Multi-Layer System View & End-to-End Evaluation,” IEEE Communications Magazine , 2023

    2023

  9. [17]

    Extended Reality (XR) Codec Adaptation in 5G using Multi-Agent Reinforcement Learning with Attention Action Selection,

    P. E. Iturria-Rivera, R. Gaigalas, M. Elsayed, M. Bavand, O. Yigit, and M. Erol-Kantarci, “Extended Reality (XR) Codec Adaptation in 5G using Multi-Agent Reinforcement Learning with Attention Action Selection,” in 2024 IEEE 34th Annual International Symposium on Personal, Indo...

    2024

  10. [18]

    EXPLORA: AI/ML EXPLainability for the Open RAN,

    C. Fiandrino, L. Bonati, S. D’Oro, M. Polese, T. Melodia, and J. Widmer, “EXPLORA: AI/ML EXPLainability for the Open RAN,” Proceedings of the ACM on Networking , vol. 1, no. CoNEXT3, pp. 1–26, 2023

    2023

  11. [19]

    Explainable Deep Reinforcement Learning for UA V autonomous path planning,

    L. He, N. Aouf, and B. Song, “Explainable Deep Reinforcement Learning for UA V autonomous path planning,” Aerospace science and technology, vol. 118, p. 107052, 2021

    2021

  12. [20]

    Explainable artificial intelligence for 6G: Improving trust be- tween human and machine,

    W. Guo, “Explainable artificial intelligence for 6G: Improving trust be- tween human and machine,” IEEE Communications Magazine , vol. 58, no. 6, pp. 39–45, 2020

    2020

  13. [21]

    Toward native explainable and robust AI in 6G networks: Current state, challenges and road ahead,

    C. Fiandrino, G. Attanasio, M. Fiore, and J. Widmer, “Toward native explainable and robust AI in 6G networks: Current state, challenges and road ahead,” Computer Communications, vol. 193, pp. 47–52, 2022

    2022

  14. [22]

    A survey on explainable AI for 6G O-RAN: Architecture, use cases, challenges and research directions,

    B. Brik, H. Chergui, L. Zanzi, F. Devoti, A. Ksentini, M. S. Siddiqui, X. Costa-P´erez, and C. Verikoukis, “A survey on explainable AI for 6G O-RAN: Architecture, use cases, challenges and research directions,” arXiv preprint arXiv:2307.00319 , 2023

    2023 arXiv

  15. [23]

    Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environ- ments,

    R. Lowe, Y . I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environ- ments,” Advances in Neural Information Processing Systems , vol. 30, 2017

    2017

  16. [24]

    Counterfactual multi-agent policy gradients,

    J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” in Proceedings of the AAAI conference on artificial intelligence , vol. 32, no. 1, 2018

    2018

  17. [25]

    The surprising effectiveness of ppo in cooperative multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of ppo in cooperative multi-agent games,” Ad- vances in Neural Information Processing Systems , vol. 35, pp. 24 611– 24 624, 2022

    2022

  18. [26]

    Multiagent trust region policy optimization,

    H. Li and H. He, “Multiagent trust region policy optimization,” IEEE Transactions on Neural Networks and Learning Systems , 2023

    2023

  19. [27]

    Decomposed soft actor- critic method for cooperative multi-agent reinforcement learning,

    Y . Pu, S. Wang, R. Yang, X. Yao, and B. Li, “Decomposed soft actor- critic method for cooperative multi-agent reinforcement learning,” arXiv preprint arXiv:2104.06655, 2021

    2021 arXiv

  20. [28]

    Actor-attention-critic for multi-agent reinforcement learning,

    S. Iqbal and F. Sha, “Actor-attention-critic for multi-agent reinforcement learning,” in International conference on machine learning . PMLR, 2019, pp. 2961–2970

    2019

  21. [29]

    F. A. Oliehoek, C. Amato et al., A concise introduction to decentralized POMDPs. Springer, 2016, vol. 1

    2016

  22. [30]

    Value-Decomposition Networks for Cooperative Multi-Agent Learn- ing,

    P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V . Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls et al. , “Value-Decomposition Networks for Cooperative Multi-Agent Learn- ing,” arXiv preprint arXiv:1706.05296 , 2017

    2017 arXiv

  23. [31]

    QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning,

    K. Son, D. Kim, W. J. Kang, D. E. Hostallero, and Y . Yi, “QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning,” in International conference on machine learn- ing. PMLR, 2019, pp. 5887–5896

    2019

  24. [32]

    Monotonic Value Function Factorisation for Deep Multi- Agent Reinforcement Learning,

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Monotonic Value Function Factorisation for Deep Multi- Agent Reinforcement Learning,” Journal of Machine Learning Research, vol. 21, no. 178, pp. 1–51, 2020

    2020

  25. [33]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT press, 2018

    2018

  26. [34]

    A survey on explainable reinforcement learning: Concepts, algorithms, challenges,

    Y . Qing, S. Liu, J. Song, H. Wang, and M. Song, “A survey on explainable reinforcement learning: Concepts, algorithms, challenges,” arXiv preprint arXiv:2211.06665 , 2022

    2022 arXiv

  27. [35]

    Markov Games as a Framework for Multi-agent Rein- forcement Learning,

    M. L. Littman, “Markov Games as a Framework for Multi-agent Rein- forcement Learning,” in Machine learning proceedings 1994. Elsevier, 1994, pp. 157–163

    1994

  28. [36]

    Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learning,

    Z. Dou, J. G. Kuba, and Y . Yang, “Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learning,” arXiv preprint arXiv:2202.04868 , 2022

    2022 arXiv

  29. [37]

    Value Function Decomposition for Iterative Design of Reinforcement Learning Agents,

    J. MacGlashan, E. Archer, A. Devlic, T. Seno, C. Sherstan, P. Wurman, and P. Stone, “Value Function Decomposition for Iterative Design of Reinforcement Learning Agents,” Advances in Neural Information Processing Systems, vol. 35, pp. 12 001–12 013, 2022

    2022

  30. [38]

    Towards Understanding Cooperative Multi-Agent Q-learning with Value Factorization,

    J. Wang, Z. Ren, B. Han, J. Ye, and C. Zhang, “Towards Understanding Cooperative Multi-Agent Q-learning with Value Factorization,” Ad- vances in Neural Information Processing Systems , vol. 34, pp. 29 142– 29 155, 2021

    2021

  31. [39]

    Influence of Video Delay on Quality, Presence, and Sickness in Viewport Adaptive Immer- sive Streaming,

    C. Cort ´es, P. P ´erez, J. Guti ´errez, and N. Garc ´ıa, “Influence of Video Delay on Quality, Presence, and Sickness in Viewport Adaptive Immer- sive Streaming,” in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX) , 2020, pp. 1–4

    2020

  32. [40]

    PQoE Evaluation Methods for 360-Degree VR Video Transmission,

    Z. Fei, F. Wang, J. Wang, and X. Xie, “PQoE Evaluation Methods for 360-Degree VR Video Transmission,” IEEE Journal of Selected Topics in Signal Processing , vol. 14, no. 1, pp. 78–88, 2020

    2020

  33. [41]

    Proposed QoE Models Associated with Delay and Jitter Using Subjective Approach and Applications for 4G and 5G Networks,

    T. Daengsi, P. Sirawongphatsara, and P. Wuttidittachotti, “Proposed QoE Models Associated with Delay and Jitter Using Subjective Approach and Applications for 4G and 5G Networks,” in 2021 4th International Conference on Advanced Communication Technologies and Networking (CommN...

    2021

  34. [42]

    Wireless communication research challenges for extended reality (XR),

    I. F. Akyildiz and H. Guo, “Wireless communication research challenges for extended reality (XR),” ITU Journal on Future and Evolving Tech- nologies, vol. 3, no. 1, pp. 1–15, 2022

    2022

  35. [43]

    A comprehensive survey of the tactile internet: State-of-the-art and research directions,

    N. Promwongsa, A. Ebrahimzadeh, D. Naboulsi, S. Kianpisheh, F. Belqasmi, R. Glitho, N. Crespi, and O. Alfandi, “A comprehensive survey of the tactile internet: State-of-the-art and research directions,” IEEE Communications Surveys & Tutorials, vol. 23, no. 1, pp. 472–523, 2020

    2020

  36. [44]

    Study on XR (Extended Reality) Evaluations for NR,

    3GPP, “Study on XR (Extended Reality) Evaluations for NR,” 3rd Generation Partnership Project (3GPP), Technical Report (TR) 38.838, Dec. 2021, version 17.0.0. [Online]. Available: https: //www.3gpp.org/ftp/Specs/archive/38 series/38.838/38838-h00.zip

    2021

  37. [45]

    XR Quality Index: Evaluating RAN Transmission Quality for XR services over 5G and beyond,

    S. Dou, S. Liao, J. Wu, K. Wu, E. Chen, W. Chen, H. Shen, and N. Li, “XR Quality Index: Evaluating RAN Transmission Quality for XR services over 5G and beyond,” in IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, PIMRC , 2021

    2021

  38. [46]

    Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping,

    A. Y . Ng, D. Harada, and S. Russell, “Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping,” in Icml, vol. 99, 1999, pp. 278–287

    1999

  39. [47]

    Multitask Learning,

    R. Caruana, “Multitask Learning,” Machine learning, vol. 28, pp. 41–75, 1997

    1997

  40. [48]

    Gradnorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks,

    Z. Chen, V . Badrinarayanan, C.-Y . Lee, and A. Rabinovich, “Gradnorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks,” in International conference on machine learning . PMLR, 2018, pp. 794–803

    2018

  41. [49]

    ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research,

    P. Gawłowicz and A. Zubow, “ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research,” in ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM) , November 2019. [Online]. Available: http://www.tkn.tu-b...

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.