Pith. sign in

REVIEW 4 major objections 6 minor 27 references

Heterogeneous Value Decomposition Policy Fusion for Multi-Agent Cooperation

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an adaptively sampled composite of a network-constrained value-decomposition policy and a surrogate-target policy, regularized by a KL constraint, improves cooperative MARL performance and still satisfies the IGM…

desk verdict HPF's adaptive policy-fusion idea is new and the experiments are convincing, but the IGM guarantee doesn't cover the actual implementation, so the paper needs a fix to its theory before I'd trust it. read the letter →

arxiv 2502.02875 v1 pith:QP23YJXV submitted 2025-02-05 cs.MA

Siying Wang , Yang Zhou , Zhitong Zhao , Ruoning Zhang , Jinliang Shao , Wenyu Chen , Yuhua Cheng This is my paper · ORCID
classification cs.MA
keywords heterogeneouspolicyfusionvaluedecompositionmulti-agentreinforcementlearningcooperativeagentsindividual-global-maxcentralizedtrainingdecentralizedexecutionKLdivergencecomposite
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Value-decomposition (VD) methods for cooperative multi-agent reinforcement learning fall into two families: those that restrict the network so the decomposed local utilities satisfy the Individual-Global-Max (IGM) criterion (fast but representationally limited), and those that build a surrogate target to keep full representational freedom (expressive but harder and slower to train). The paper claims that these families can be merged at the policy level instead of designing yet another factorization. Heterogeneous Policy Fusion (HPF) puts one policy from each family into a composite policy set, samples which policy interacts with the environment using a Boltzmann distribution over their value estimates, and applies a KL-divergence constraint between their per-agent utility functions so the two policies do not drift into conflicting updates. The authors report that the resulting fused policies learn optimal joint actions in a relative-overgeneralization matrix game and reach higher win rates with better sample efficiency than the individual baselines on StarCraft and predator-prey tasks, and they argue the fused policy set still satisfies the IGM criterion.

What carries the argument

The carrying object is the composite policy set $\Pi = [\pi_\alpha, \pi_\beta]$ together with its adaptive selection rule. HPF estimates a value $Q^{\pi_k}$ for each member policy---either the sum of per-agent utilities (the additive rule) or the surrogate-target joint value at the joint greedy action (the optimistic rule)---turns these estimates into a Boltzmann categorical distribution $P_w[k] \propto \exp(Q^{\pi_k}/\eta)$, and draws which policy will take the next environment step. The second load-bearing mechanism is the instructive constraint $L_I$, a KL divergence between the per-agent Boltzmann policies derived from the two utility functions; it keeps the expressive policy's correct local optima from being contradicted by the constrained policy's limited estimates. Training then jointly updates both VD policies on the mixed replay data.

What would settle it

Train HPF on the one-step matrix game of Table 1a and inspect the learned $Q_{jt}$ and each utility network at the end of training. If the joint action maximizing $Q_{jt}$ is not the optimal $(u_1,u_1)$, or if any agent's greedy action under its own utility differs from that joint maximizer, the central claims of optimal action selection and of IGM preservation fail for the implemented HPF. Alternatively, run the paper's own ablation with random policy selection: if the value-based selection procedure produces no measurable gain over random selection, the adaptive-composition claim is falsified.

Watch

Extended reading notes

Core claim

HPF's central claim is that the weakness of a VD method lies less in its factorization scheme than in the single policy it uses to generate experience, so combining two heterogeneous VD policies can improve both representation and training efficiency without inventing a new decomposition. The composite policy set is $\Pi = [\pi_\alpha, \pi_\beta]$, where $\pi_\alpha$ is a surrogate-target policy such as WQMIX or QPLEX and $\pi_\beta$ is a network-constrained policy such as QMIX or VDN. At each step HPF samples the interacting policy from a categorical distribution over $\Pi$ with Boltzmann weights computed from the value estimates of the two policies, using either the additive sum of per-agent utilities or the optimistic joint optimal-action value. Both policies train on the resulting mixed trajectories, with the composite loss $L = L_{\mathrm{tot}}^{\mathrm{TD}} + L_{\mathrm{jt}}^{\mathrm{TD}} + L_I$, where $L_I$ is the KL constraint between the per-agent Boltzmann policies induced by the two sets of utilities. Proposition 1 asserts that any extended policy set formed from heterogeneous VD policies still satisfies the IGM criterion, so local greedy actions remain consistent with global optimality. Experiments with HPF-WQ (WQMIX plus QMIX) and HPF-QV (QPLEX plus VDN) show higher performance than the individual baselines, and the policy-ratio curves show the composite gradually favoring whichever policy is learning more reliably.

Load-bearing premise

The formal guarantee that the fused policy set still satisfies IGM assumes both candidate VD policies share the same per-agent utility decomposition $[Q_i]$, whereas the implemented HPF gives the two policies separate utility networks and a KL constraint between their differing utilities; if those utilities are not shared, the proof as written does not cover the implemented system.

Editorial extensions

If this is right

  • HPF can be layered onto any existing pair of value-decomposition methods without changing their internal factorization or mixing networks, making it a portable performance add-on rather than a competing algorithm.
  • The value-based selection acts as a soft curriculum: early in training the cheap constrained policy may dominate data collection, while the surrogate-target policy takes over as its value estimates become reliable, as the recorded selection ratios indicate.
  • On tasks that punish one of the two families, such as relative overgeneralization, super-hard StarCraft maps, and predator-prey, the fused policy should recover behavior that neither baseline alone reaches.
  • Because both policies learn from the same mixed trajectories, the update signal from one policy's exploration can correct the other's misleading estimates, which is the mechanism behind the reported faster convergence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit stronger hypothesis is that the data distribution a factorization produces, not the factorization itself, is the main bottleneck; if so, policy fusion should also help two policies from the same family that differ in architecture, a variant the paper does not test.
  • The Boltzmann selection over value estimates is one small exploration design; sweeping the temperature $\eta$ or replacing it with a learned gating network is a natural testable extension that could make the balance between exploration and exploitation adaptive.
  • The gap between the shared-utility proof and the separate-utility implementation suggests a concrete fix: keep one utility network and attach two mixing heads (one monotone, one unrestricted); a variant like this would make the IGM guarantee hold exactly as proven.
  • The KL constraint can be read as a trust-region regularizer between policies; separating its effect from the mixed-data effect, for example by varying its weight while keeping the policy set fixed, would identify which ingredient drives HPF's gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Heterogeneous Policy Fusion (HPF), a meta-algorithm that combines two existing value-decomposition (VD) methods: a network-constrained method such as QMIX or VDN and a surrogate-target method such as WQMIX or QPLEX. HPF maintains a policy set, selects which candidate policy interacts with the environment at each timestep via a Boltzmann distribution over value estimates, and adds a KL-divergence constraint between the per-agent Boltzmann policies of the two candidates. The authors claim that the extended policy set preserves the Individual-Global-Max (IGM) criterion (Proposition 1), and they report experiments on a matrix game, five SMAC maps, and a predator-prey task showing that HPF variants outperform the baselines. The paper is written as an orthogonal alternative to designing new factorization schemes, with the theoretical guarantee and the empirical results as its two pillars.

Significance. If the empirical pattern holds, the idea of adaptively fusing heterogeneous VD policies is a simple, useful, and orthogonal contribution: it does not introduce a new factorization but leverages existing ones. The paper evaluates the method on three benchmarks, uses standard baselines, and reports consistent advantages across most scenarios. However, the advertised formal guarantee (Proposition 1) is the main advertised support for the method, and it does not cover the implemented architecture; the proof is also not valid for the two surrogate-target methods actually used (WQMIX and QPLEX). Because the theoretical claim is load-bearing and currently unsupported, the paper is not ready for publication in its present form. The core algorithmic idea is plausible and worth re-examining after the theoretical statement and the algorithm description are corrected.

major comments (4)
  1. [Appendix A, Proposition 1] The proof of Proposition 1 assumes that both candidate VD policies share a single per-agent utility decomposition [Q_i], writing Qjt in Eq. (17) as a convex combination of Q(1)_tot and Q(2)_tot that are both monotone in the same [Q_i]. The implemented HPF, described in Section 4.3 and Algorithm 1, maintains separate utility networks {Q^{πα}_i} and {Q^{πβ}_i}, and the KL constraint in Eq. (8) only makes sense when the two utility sets differ. With distinct utilities, the monotonicity argument in Eqs. (18a)-(18d) does not apply, so the proof does not establish IGM for the composite system described in the paper. Moreover, the composite policy in Eqs. (5)-(6) is a stochastic switch between two independent policies, not a convex combination of joint value functions; the proposition as formalized in Eq. (17) does not correspond to the algorithm. The proposition and proof need to be reformulated to match the actual architecture, or the claim must be weakened substantially.
  2. [Algorithm 1, line 25] The TD error for the QMIX branch is written as ∆Qtot = Qπβ(−)_tot − Qπα_tot, and line 26 then updates θβ by the gradient of (∆Qtot)^2. This mixes the target network of the β policy with the current Q of the α policy. The correct error should be Qπβ(−)_tot − Qπβ_tot. As written, the algorithm is not the one described by Eqs. (10)-(12) or the one evaluated in Section 5. This is a load-bearing implementation error, not a mere typo, because it changes the update rule for one of the two candidate policies.
  3. [Appendix A, Eqs. (13)-(14)] The proof claims that all surrogate-target VD methods can be written as QT ype1_jt = Q(1)_tot + w_r Q_r with w_r ∈ {0,1} and Q_r ≤ 0, citing ResQ. This representation is not general: WQMIX uses a weighted projection loss, and QPLEX uses an advantage-based dueling decomposition with consistency constraints; neither reduces to a masked residual of this form. Since the experiments use HPF-WQ (WQMIX) and HPF-QV (QPLEX), the proof, even if corrected for shared utilities, would not cover the two instantiations that the paper evaluates. The theoretical claim must either be restricted to the specific residual form or proved for the actual mechanisms used by WQMIX and QPLEX.
  4. [Section 5, experiments] All performance claims are based on 5 random seeds, reported as medians with 25%-75% quartiles, and no statistical significance tests are provided. The matrix game results in Table 1 are presented without error bars, so it is unclear whether the differences between HPF and WQMIX/ResQ are robust. Given that the central claim of the paper is empirical superiority, the evaluation needs at least additional seeds or a statistical test, and the matrix-game table should include variance or a multiple-seed aggregation.
minor comments (6)
  1. [Eq. (8)] The loss LI is written as a 'min' without specifying the optimization variable; it should be presented as a standard minimization objective over the network parameters, rather than an operator applied at each update.
  2. [Eq. (6)] The notation δ_{u∼πα} is nonstandard and ambiguous: δ typically denotes a Dirac measure, but here it is meant to indicate an action sampled from πα. This should be written more clearly, e.g., as a mixture distribution over the two policies.
  3. [Section 4.3] The paper should state explicitly that the KL constraint is between Boltzmann policies induced by each agent's utility functions, and that the constraint is applied per agent; currently this is only implicit in Eqs. (8)-(9).
  4. [Appendix B] The appendix states 'Since QMIX assumes the task can be directly decomposed, which also means Qjt = Qtot is this setting.' This identity is used in the Optimistic sampling variant, but it is not explained in the main text; please clarify it in Section 4.2 where Eq. (7) is introduced.
  5. [Appendix D and Figure 3] The captions of Figures 3 and 5 contain garbled characters (e.g., '一一－' and invalid unicode escape sequences), making part of the supplementary text unreadable. These should be fixed before any resubmission.
  6. [Appendix A, proposition numbering] Proposition 1 in the main text is restated as Proposition 2 in Appendix A; the numbering should be consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; Appendix A proof gap is a soundness issue, not a circular derivation.

full rationale

HPF's central claims—adaptive composite-policy selection via Eqs. (5)–(7), the KL instructive constraint in Eq. (8), and the overall objective in Eq. (10)—are not constructed from the experimental results or from the paper's own fitted values. The method is compared against independently published baselines (WQMIX, QPLEX, QMIX, VDN, ResQ) on a matrix game, SMAC, and predator-prey, with hyperparameters inherited from the baseline codebases or stated openly; no parameter is fit to the benchmark outcomes and then renamed as a prediction. There is no load-bearing self-citation: Proposition 1's proof appeals to an external theorem from ResQ (Shen et al., 2022), not to prior work by the present authors, and the other references are standard MARL literature. The reader-identified weakness in Appendix A is real but is not circularity: the proof of Proposition 1 writes both heterogeneous policies in terms of a common per-agent utility decomposition [Qi] in Eqs. (17)–(18), whereas Algorithm 1 and Section 4.3 maintain separate utility networks for the two policies and the KL constraint presupposes distinct utilities. This means the proven IGM guarantee may not cover the implemented architecture, which is an assumption-mismatch or proof-gap concern, not a reduction of the claimed result to its own inputs by construction. Because the core contribution is tested against external benchmarks and the theoretical gap does not amount to deriving the conclusion from the conclusion, the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces a small number of hand-chosen hyperparameters (temperature η, implicit KL weight, inherited WQMIX weight) and rests on an unstated assumption in Proposition 1 that the two policies share utility functions. No new physical or conceptual entities are postulated.

free parameters (3)
  • Boltzmann temperature η = 1
    Hand-chosen in Appendix C.2; controls how strongly Q-value estimates determine which VD policy is selected for interaction (Equation 5). No sensitivity analysis is reported.
  • KL constraint weight = implicitly 1
    The overall loss in Equation (10) sums Ltot_TD, Ljt_TD, and LI without a tuned weighting coefficient; the implicit weight of 1 on LI is a design choice not studied experimentally.
  • WQMIX weight w = 0.1 in matrix game, 0.75 in SMAC
    Inherited from the WQMIX baseline and chosen from the range (0.5, 0.75) in SMAC; it changes the surrogate-target policy used inside HPF-WQ and is not optimized by HPF itself.
assumptions (3)
  • standard math The IGM principle and value-decomposition framework are valid criteria for cooperative MARL
    Used throughout Section 3 and Proposition 1 as the correctness criterion for factorized joint value functions.
  • domain assumption Surrogate-target VD policies can learn the correct optimal joint action, so their local action preferences are safe to imitate
    The instructive constraint in Equation (8) pushes the network-constrained policy toward the surrogate-target policy, which assumes the surrogate policy is the better guide in all tasks; the paper's own ablation notes cases where WQMIX fails to learn good behavior.
  • ad hoc to paper All candidate VD policies in the extended set share a common factorized utility decomposition Qi
    The proof of Proposition 1 (Appendix A, Equations 17-18) expresses Qjt as a convex combination using the same [Qi], but the implementation uses separate utility networks for πα and πβ and links them only through the KL penalty, so the proof does not cover the implemented architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Heterogeneous Value Decomposition Policy Fusion for Multi-Agent Cooperation." pith.science (2026). https://pith.science/paper/QP23YJXV

@misc{pith2026250202875,
  author       = {Pith},
  title        = {Pith review of: Heterogeneous Value Decomposition Policy Fusion for Multi-Agent Cooperation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QP23YJXV}},
  note         = {Machine review of arXiv:2502.02875}
}
read the original abstract

Value decomposition (VD) has become one of the most prominent solutions in cooperative multi-agent reinforcement learning. Most existing methods generally explore how to factorize the joint value and minimize the discrepancies between agent observations and characteristics of environmental states. However, direct decomposition may result in limited representation or difficulty in optimization. Orthogonal to designing a new factorization scheme, in this paper, we propose Heterogeneous Policy Fusion (HPF) to integrate the strengths of various VD methods. We construct a composite policy set to select policies for interaction adaptively. Specifically, this adaptive mechanism allows agents' trajectories to benefit from diverse policy transitions while incorporating the advantages of each factorization method. Additionally, HPF introduces a constraint between these heterogeneous policies to rectify the misleading update caused by the unexpected exploratory or suboptimal non-cooperation. Experimental results on cooperative tasks show HPF's superior performance over multiple baselines, proving its effectiveness and ease of implementation.

Figures

Figures reproduced from arXiv: 2502.02875 by the authors.

Figure 1
Figure 1. The illustration of the distinction between HPF and traditional VD methods. The traditional scheme directly aligns the optimal joint action and optimizes the central value function with the presupposed VD policy itself. The proposed HPF integrates the benefits of different types of VD policies, and expands them into a policy set to sample the experiences for capturing further performance improvement. These VD polici… view at source ↗
Figure 2
Figure 2. The architecture of HPF. (a) The VD method with surrogate target. (b) The VD method with network parameters constraint. (c) The instructive constraint between heterogeneous utility functions. The policies of both VD methods constitute a composite policy set and interact with the environment after sampling. constructs a Boltzmann policy-based categorical distribution for selecting the final interactive policy: Pw[k] … view at source ↗
Figure 3
Figure 3. Comparison results on the selected scenarios in the StarCraft Multi-Agent Challenge. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison results in the predator and prey. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Ablation studies of the random candidate VD policy sam [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: The selection probabilities of different VD policies in HPF [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 21 canonical work pages

  1. [1]

    In Add-HPF-WQ, WQMIX[Rashid et al., 2020] is chosen as πα in Eq.(4) of the main text

    Here we take Add- HPF-WQ as an example to illustrate the HPF algorithm pro- cess, with other HPF variants following a similar training pro- cedure. In Add-HPF-WQ, WQMIX[Rashid et al., 2020] is chosen as πα in Eq.(4) of the main text. While the πβ with limited representation is QMIX[Rashid et al., 2018]. Since QMIX as- sumes the task can be directly decomp...

    work page 2020

  2. [2]

    If Π = [ πα, πβ] is an extended policy set formed by existing heterogeneous VD policies, then Π still satisfies the IGM criterion. Proof. The Theorem 1 and Section 4 of ResQ [Shen et al., 2022] show that for any hard-to-factorize state-action value function Qjt, It can generally always be regarded as satisfy- ing the following formula: Qjt(τ , u) = wtot(τ...

    work page 2022

  3. [4]

    Mac-po: Multi-agent experience replay via collective priority optimization

    [Mei et al., 2023] Yongsheng Mei, Hanhan Zhou, Tian Lan, Guru Venkataramani, and Peng Wei. Mac-po: Multi-agent experience replay via collective priority optimization. In Proceedings of the 2023 International Conference on Au- tonomous Agents and Multiagent Systems, pages 466–475,

    work page 2023

  4. [5]

    Rusu, Joel Veness, Marc G

    [Mnih et al., 2015] V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Belle- mare, Alex Graves, Martin Riedmiller, Andreas K. Fidje- land, and Georg Ostrovski. Human-level control through deep reinforcement learning. nature, 518(7540):529–533,

    work page 2015

  5. [10]

    Foerster, and Shimon Whiteson

    [Rashid et al., 2018] Tabish Rashid, Mikayel Samvelyan, Christian Schr ¨oder de Witt, Gregory Farquhar, Jakob N. Foerster, and Shimon Whiteson. QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforce- ment Learning. In Proceedings of the 35th Interna- tional Conference on Machine Learning , pages 4292–

    work page 2018

  6. [12]

    [Samvelyan et al., 2019] Mikayel Samvelyan, Tabish Rashid, Christian Schr ¨oder de Witt, Gregory Farquhar, Nantas Nardelli, Tim G. J. Rudner, Chia-Man Hung, Philip H. S. Torr, Jakob N. Foerster, and Shimon Whiteson. The StarCraft Multi-Agent Challenge. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS, ...

    work page 2019

  7. [13]

    Resq: A residual q function-based approach for multi-agent reinforcement learning value factoriza- tion

    [Shen et al., 2022] Siqi Shen, Mengwei Qiu, Jun Liu, Wei- quan Liu, Yongquan Fu, Xinwang Liu, and Cheng Wang. Resq: A residual q function-based approach for multi-agent reinforcement learning value factoriza- tion. Advances in Neural Information Processing Systems, 35:5471–5483,

    2022

  8. [14]

    Qtran: Learn- ing to factorize with transformation for cooperative multi- agent reinforcement learning

    [Son et al., 2019] Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. Qtran: Learn- ing to factorize with transformation for cooperative multi- agent reinforcement learning. In International conference on machine learning, pages 5887–5896. PMLR,

    2019

Show all 27 references
  1. [15]

    Qtran++: improved value transformation for cooperative multi-agent reinforcement learning

    [Son et al., 2020] Kyunghwan Son, Sungsoo Ahn, Roben Delos Reyes, Jinwoo Shin, and Yung Yi. Qtran++: improved value transformation for cooperative multi-agent reinforcement learning. arXiv preprint arXiv:2006.12010,

    2020 arXiv

  2. [16]

    Value-decomposition networks for cooperative multi-agent learning

    [Sunehag et al., 2017] Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, et al. Value-decomposition networks for cooperative multi-agent learning. arXiv preprint ar...

    2017 arXiv

  3. [18]

    Multi-agent reinforcement learning: Independent vs

    [Tan, 1993] Ming Tan. Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proceedings of the 10th international conference on machine learning, pages 330–337. PMLR,

    1993

  4. [20]

    Vgn: Value decomposition with graph atten- tion networks for multiagent reinforcement learning.IEEE Transactions on Neural Networks and Learning Systems ,

    [Wei et al., 2022] Qinglai Wei, Yugu Li, Jie Zhang, and Fei- Yue Wang. Vgn: Value decomposition with graph atten- tion networks for multiagent reinforcement learning.IEEE Transactions on Neural Networks and Learning Systems ,

    2022

  5. [21]

    Haven: Hierarchical coop- erative multi-agent reinforcement learning with dual coor- dination mechanism

    [Xu et al., 2023] Zhiwei Xu, Yunpeng Bai, Bin Zhang, Dapeng Li, and Guoliang Fan. Haven: Hierarchical coop- erative multi-agent reinforcement learning with dual coor- dination mechanism. In Proceedings of the AAAI Confer- ence on Artificial Intelligence , volume 37, pages 1173...

    2023

  6. [22]

    Qatten: A general framework for coopera- tive multiagent reinforcement learning

    [Yang et al., 2020] Yaodong Yang, Jianye Hao, Ben Liao, Kun Shao, Guangyong Chen, Wulong Liu, and Hongyao Tang. Qatten: A general framework for coopera- tive multiagent reinforcement learning. arXiv preprint arXiv:2002.03939,

    2020 arXiv

  7. [23]

    Causal inference multi-agent rein- forcement learning for traffic signal control

    [Yang et al., 2023] Shantian Yang, Bo Yang, Zheng Zeng, and Zhongfeng Kang. Causal inference multi-agent rein- forcement learning for traffic signal control. Information Fusion, 94:243–256,

    2023

  8. [24]

    Game of drones: Multi-uav pursuit-evasion game with online motion plan- ning by deep reinforcement learning

    [Zhang et al., 2023] Ruilong Zhang, Qun Zong, Xiuyun Zhang, Liqian Dou, and Bailing Tian. Game of drones: Multi-uav pursuit-evasion game with online motion plan- ning by deep reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 34(10):7900– 7909,

    2023

  9. [25]

    Locality matters: A scalable value decompo- sition approach for cooperative multi-agent reinforcement learning

    [Zohar et al., 2022] Roy Zohar, Shie Mannor, and Guy Ten- nenholtz. Locality matters: A scalable value decompo- sition approach for cooperative multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, volume 36, pages 9278–9285,

    2022

  10. [1993]

    {QPLEX}: Duplex duel- ing multi-agent q-learning

    [Wang et al., 2021] Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. {QPLEX}: Duplex duel- ing multi-agent q-learning. In International Conference on Learning Representations,

    2021

  11. [2015]

    Graph convo- lutional value decomposition in multi-agent reinforcement learning

    [Naderializadeh et al., 2020] Navid Naderializadeh, Fan H Hung, Sean Soleyman, and Deepak Khosla. Graph convo- lutional value decomposition in multi-agent reinforcement learning. arXiv preprint arXiv:2010.04740,

    2020 arXiv

  12. [2016]

    Multi-agent reinforcement learning for redun- dant robot control in task-space

    [Perrusqu´ıa et al., 2021] Adolfo Perrusqu ´ıa, Wen Yu, and Xiaoou Li. Multi-agent reinforcement learning for redun- dant robot control in task-space. International Journal of Machine Learning and Cybernetics, 12:231–241,

    2021

  13. [2017]

    Sutton and Andrew G

    [Sutton and Barto, 2018] Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction . MIT press,

    2018

  14. [2018]

    Weighted qmix: Expand- ing monotonic value function factorisation for deep multi- agent reinforcement learning

    [Rashid et al., 2020] Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. Weighted qmix: Expand- ing monotonic value function factorisation for deep multi- agent reinforcement learning. Advances in neural infor- mation processing systems, 33:10199–10210,

    2020

  15. [2019]

    Transformer-based value function decomposition for cooperative multi-agent rein- forcement learning in starcraft

    [Khan et al., 2022] Muhammad Junaid Khan, Syed Hammad Ahmed, and Gita Sukthankar. Transformer-based value function decomposition for cooperative multi-agent rein- forcement learning in starcraft. InProceedings of the AAAI Conference on Artificial Intelligence and Interactive D...

    2022

  16. [2020]

    Oliehoek and Christo- pher Amato

    [Oliehoek and Amato, 2016] Frans A. Oliehoek and Christo- pher Amato. A Concise Introduction to Decentral- ized POMDPs . Springer Briefs in Intelligent Systems. Springer,

    2016

  17. [2021]

    Residual q-networks for value function factorizing in multiagent reinforcement learning

    [Pina et al., 2022] Rafael Pina, Varuna De Silva, Joosep Hook, and Ahmet Kondoz. Residual q-networks for value function factorizing in multiagent reinforcement learning. IEEE Transactions on Neural Networks and Learning Sys- tems,

    2022

  18. [2022]

    Contrastive identity-aware learning for multi-agent value decomposition

    [Liu et al., 2023] Shunyu Liu, Yihe Zhou, Jie Song, Tongya Zheng, Kaixuan Chen, Tongtian Zhu, Zunlei Feng, and Mingli Song. Contrastive identity-aware learning for multi-agent value decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pa...

    2023

  19. [2023]

    Social influence as intrinsic motivation for multi-agent deep reinforcement learning

    [Jaques et al., 2019] Natasha Jaques, Angeliki Lazari- dou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, DJ Strouse, Joel Z Leibo, and Nando De Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In International conference on machin...

    2019

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.