Pith. sign in

REVIEW 4 major objections 5 minor 29 references

Improving monotonic optimization in heterogeneous multi-agent reinforcement learning with optimal marginal deterministic policy gradient

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Heterogeneous MARL can keep monotonic improvement while sharing policy weights.

desk verdict The baseline-drift observation is real and the empirical recipe may be useful, but Eq. (6) invalidates the claimed monotonic-improvement guarantee. read the letter →

arxiv 2507.09989 v1 pith:FA6JY4XP submitted 2025-07-14 cs.AI cs.MA

Xiaoyang Yu , Youfang Lin , Shuo Wang , Sheng Han This is my paper
classification cs.AIcs.MA
keywords heterogeneousmulti-agentreinforcementlearningmonotonicimprovementpartialparametersharingoptimalmarginalQgeneralizedcriticpolicyupdatingbaselinedriftSMACMAMuJoCo
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that cooperative heterogeneous multi-agent reinforcement learning does not have to choose between monotonic improvement and partial parameter sharing. Monotonic improvement guarantees, such as HAPPO's MAAD theorem, have required each agent to update its own policy independently, while practical heterogeneous teams perform better when similar agents share network weights. The paper identifies a concrete failure mode, policy updating baseline drift, where sharing parameters rewrites earlier agents' policies before their scheduled advantage ratios are computed, and proposes the OMDPG algorithm to remove it. If correct, heterogeneous teams can group agents, share parameters within groups, and still inherit a monotonic improvement guarantee, which would make cooperative MARL training both more stable and more sample-efficient.

What carries the argument

The load-bearing object is the Optimal Marginal Q (OMQ) function, $\phi^*_i(s,a_{1:i}) = Q^c_i(s,a_{1:i-1},a_i,a^*_{i+1:n}) - Q^c_i(s,a_{1:i-1},0,a^*_{i+1:n})$, which measures agent $i$'s marginal contribution under a joint action whose later entries are the optimal or greedy actions of subsequent agents. It replaces the sequential policy-ratio factor $F_i$ from HAPPO, eliminating the shared-weight baseline drift; however, it also asks the critic to evaluate out-of-distribution joint actions, which is handled by the Generalized Q Critic (GQC): multiple Q-networks trained with a pessimistic uncertainty loss that subtracts a standard-deviation penalty for such inputs. Together with the Centralized Critic Grouped Actor (CCGA) architecture, the machinery supplies parallel, Q-based per-agent update targets while keeping parameter sharing within agent groups.

What would settle it

A concrete test is to verify Eq. (6) numerically on a trained critic and then run OMDPG in a controlled environment where agents' action sets contain no no-op action; if the per-update joint return ever decreases when no zero action exists, or if $Q^s_i(s,a_{1:i}) - Q^s_{i-1}(s,a_{1:i-1})$ departs from $Q^s_i(s,a_{1:i-1},a_i)$, the monotonic improvement guarantee does not survive.

Watch

Extended reading notes

Core claim

The central claim is that OMDPG maintains MAAD's monotonic improvement while permitting grouped parameter sharing, by replacing the sequentially computed policy-ratio approximation of per-agent advantage with an Optimal Marginal Q (OMQ) function computed from Q-functions under optimal joint action sequences. The paper argues that Eq. (6) makes the sequential Q-function equal to the sequential advantage, so an update can optimize Q-values directly, and Eq. (7) gives each agent's marginal contribution as the difference between its Q-value under the optimal joint action and its Q-value when that agent's action is replaced by a zero baseline. On top of this, a Generalized Q Critic (GQC) with pessimistic uncertainty loss estimates the true, optimal-action, and zero-baseline Q-values, and a Centralized Critic Grouped Actor (CCGA) architecture allows partial parameter sharing among homogeneous groups of actors while keeping a centralized critic. The paper reports that this design outperforms HAPPO, HASAC, MAPPO, MATD3, HetGPPO, MADDPG, and Kaleidoscope in SMAC and MAMuJoCo heterogeneous benchmarks, with the strongest gains in the largest maps.

Load-bearing premise

The load-bearing premise is that the sequential Q advantage equals the sequential Q difference in Eq. (6) and that every agent has a zero action whose value acts as a valid no-contribution baseline.

Editorial extensions

If this is right

  • Grouped parameter sharing, not full independence, becomes compatible with MAAD-style monotonic improvement; HAPPO's NoPS requirement is an artifact of its sequential ratio approximation.
  • Per-agent update targets become computable from Q-functions along optimal joint action sequences, which removes sequential dependence and makes updates parallelizable.
  • Pessimistic uncertainty control over out-of-distribution Q-values becomes a standard component when value-based MARL optimizes against optimal-action targets; the paper's ablation shows OMQ, not GQC alone, carries most of the gain.
  • Benchmark results indicate that in large heterogeneous SMAC maps such as 12m4m_30m, only OMDPG reaches a usable win rate while baselines stay below 0.4, and a similar pattern appears in the modified MAMuJoCo environments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The zero-action baseline in Eq. (7) is a hidden normalization: in many real robotic tasks agents cannot perform a semantically neutral no-op, so a learned reference action or state-value baseline would be needed before OMQ transfers beyond the paper's benchmarks.
  • Because OMQ is defined through Q-differences rather than policy ratios, the same construction could be lifted to stochastic-policy and value-decomposition MARL rather than only deterministic policy gradient; this is an extension the paper does not test.
  • A natural falsification is to plot monotonicity itself, per-update joint return never decreasing, rather than final performance; the paper reports convergence curves, not the monotonicity certificate it claims.
  • The paper's own cost table shows OMDPG taking 18.9-19.6 hours compared with 14.2 for HAPPO and 10.5 for MATD3, so the theoretical gains come with additional wall-clock cost; whether that cost is justified outside complex heterogeneous maps is left open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses heterogeneous multi-agent reinforcement learning under partial parameter sharing. It identifies a 'policy updating baseline drift' problem when HAPPO's sequential update scheme is combined with grouped parameter sharing, and proposes OMDPG, which replaces sequential policy ratios with an Optimal Marginal Q (OMQ) function, uses a Generalized Q Critic with pessimistic uncertainty loss, and employs a Centralized Critic Grouped Actor architecture. The authors report experiments in SMAC and MAMuJoCo environments and claim that OMDPG preserves the monotonic improvement guarantee of MAAD while outperforming several baselines.

Significance. If the central theoretical claim were established, the paper would make a useful contribution: it targets a real practical issue in heterogeneous MARL, namely the interaction between sequential updates and parameter sharing, and it provides a concrete algorithmic recipe with extensive experiments. The empirical results, especially the ablations separating OMQ and GQC, are informative and suggest that the proposed components can help in practice. However, the paper's headline claim is a formal guarantee of monotonic improvement, and that guarantee is not proven; the derivation in Eq. (6) is algebraically inconsistent, Eq. (7) relies on an unjustified 'zero action' assumption, and no theorem or lemma is stated. As a result, the significance of the paper is currently limited to an empirical study of a heuristic algorithm.

major comments (4)
  1. [§4.1, Eq. (6)] The central derivation is algebraically invalid. The text writes Adv^s_i(s, a_{1:i-1}, a_i) = Adv^s_i(s, a_{1:i}) - Adv^s_{i-1}(s, a_{1:i-1}) and then computes (Q^s_i(s, a_{1:i}) - V(s)) - (Q^s_{i-1}(s, a_{1:i-1}) - V(s)) = Q^s_i(s, a_{1:i}) - Q^s_{i-1}(s, a_{1:i-1}). This equals Q^s_i(s, a_{1:i-1}, a_i) only if Q^s_{i-1}(s, a_{1:i-1}) is zero or is otherwise identified with the i-th sequential Q-function, which is not the case. No definition in the paper makes the final equality true, and the same notation is used differently in Eq. (7) and Eq. (15), where OMQ is a difference of two Q-values with an inserted zero action. This is not a minor notation issue; it is the only derivation offered for the monotonic improvement claim.
  2. [§4.1, Eq. (7)] The substitution of V^c_i(s, a_{1:i-1}, a*_{i+1:n}) by Q^c_i(s, a_{1:i-1}, 0, a*_{i+1:n}) requires the existence of a 'zero' action that represents a no-contribution baseline in every agent's action space. The paper does not justify this for SMAC's discrete action spaces or MAMuJoCo's continuous action spaces, where there is no canonical no-op action. Even if such a zero action were available, the missing Q_{i-1} term from Eq. (6) would not be recovered by this substitution. Thus the derivation of OMQ is not connected to the MAAD sequential advantage it is claimed to replace.
  3. [§4 and §6] The abstract and conclusion state that OMDPG 'maintains MAAD's monotonic improvement,' but the paper contains no theorem, lemma, or proof of this claim. Equations (6)-(7) and (15) do not constitute a proof, and the empirical learning curves in Section 5 are not evidence of a formal monotonicity guarantee. The phrase 'optimal monotonic improvement' is used as a description of convergence behavior, not a verified property. A formal claim of this kind requires either a proof under stated conditions or a clear statement that only an empirical analogue is provided.
  4. [§5.3, Tables 3 and 4] The hyperparameters C_k and lambda_PU are tuned per scenario, with the best values selected and used in the final comparisons, but no held-out validation or seed count is reported. Tables 1 and 2 report means and standard deviations without stating the number of seeds or whether the same hyperparameters were used across all scenarios. This makes it difficult to assess whether OMDPG's improvements are robust or a product of per-scenario tuning.
minor comments (5)
  1. [§3.2, Eq. (1)] The displayed formula for F_i appears to be missing the prime on the numerator; as printed, the ratio is identically 1 and cannot represent the old-to-new policy ratio described in the text.
  2. [§3.3] The text says 'The results are illustrated in Fig. 1' when referring to validation experiment curves that appear in Fig. 2; the figure reference should be corrected.
  3. [§5.2] In the MAMuJoCo experiments, the vertical axis is labeled 'Test WR', but 'win rate' is not the correct performance metric for MuJoCo locomotion tasks; the axis label should be the mean episode return.
  4. [Table 5] Table 5 reports computing times of 18.9 h and 19.6 h for OMDPG with C_k=2 and C_k=5, versus 14.2 h for HAPPO and 10.5 h for MATD3, but the text states that 'the computational time costs of OMDPG are less than HAPPO and MATD3.' This contradicts the table and should be corrected.
  5. [§4.2, Eq. (9)] The target in Eq. (9) uses the notation s', a', and epsilon without defining how the next action a' is sampled for the target Q evaluation; clearer notation would help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central monotonic-improvement guarantee is unsupported by an algebraic error, but it is not reduced to the paper's own fitted inputs or self-citations.

full rationale

OMDPG's OMQ objective is defined in Eq. (7) as a difference of learned critic Q-values, and the actor loss in Eq. (15) maximizes that difference. This is standard value-based actor-critic bootstrapping rather than a prediction that is statistically forced by a fitted input. The claimed MAAD-preserving monotonic guarantee is the paper's central theoretical assertion, and it is not established: Eq. (6) algebraically identifies a difference of sequential advantages with a single Q-value while dropping the Q_{i-1} baseline, and Eq. (7) further substitutes a learned critic with a zero action for a value baseline without proving that a no-contribution action exists. These are correctness/foundational gaps, not circularity, because the theorem is not defined into the inputs. No load-bearing self-citation was found: HAPPO's MAAD theorem is cited to external authors, and the greedy-marginal-contribution target is cited to Liu et al.; the self-citations (GHQ, GAPO) appear only in related work. The empirical claims are tested against external SMAC and MAMuJoCo benchmarks, so performance is not an artifact of the paper's own definitions. Accordingly, no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The central contribution relies on a Q-value difference with an artificial zero action and on an unproven equivalence to the sequential advantage. Hyperparameters are tuned per environment, and several key values are unreported.

free parameters (4)
  • lambda_PU = 0.01 to 0.1 (per map; best mostly 0.1)
    Balances true TD loss and PU loss in Eq. (14); tuned per scenario in Table 4.
  • C_k = 5 for most SMAC maps; not clearly stated for MAMuJoCo
    Number of parallel critics; ablated in Table 3 and affects performance.
  • sigma (target noise scale) = not reported
    Truncated Gaussian noise in Eq. (9) for target Q-value updates; standard deviation not given.
  • beta (PU uncertainty weight) = not reported
    Controls the penalty on out-of-distribution Q-values in Eq. (12); not ablated or specified.
assumptions (3)
  • domain assumption MAAD theorem from HAPPO
    The paper relies on the Multi-Agent Advantage Decomposition to justify monotonic improvement, but does not prove that the OMQ-modified objective still satisfies the theorem.
  • ad hoc to paper Equivalence in Eq. (6)
    The claim that sequential advantage equals the difference of Q-values, and that V^c_i can be replaced by Q^c_i(s,a_{1:i-1},0,a^*_{i+1:n}), is stated without proof and is the basis for OMQ.
  • ad hoc to paper Existence of a zero (no-op) action
    OMQ uses action 0 as a baseline in Eq. (7); no such action is described in SMAC or MAMuJoCo environments, so this assumption may be invalid.
invented entities (1)
  • Zero (no-op) action '0'
    purpose: Baseline for marginal contribution in OMQ (Eq. 7)
    No environment description includes a no-op action for all agents; if it does not exist, the marginal Q computation is undefined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving monotonic optimization in heterogeneous multi-agent reinforcement learning with optimal marginal deterministic policy gradient." pith.science (2026). https://pith.science/paper/FA6JY4XP

@misc{pith2026250709989,
  author       = {Pith},
  title        = {Pith review of: Improving monotonic optimization in heterogeneous multi-agent reinforcement learning with optimal marginal deterministic policy gradient},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FA6JY4XP}},
  note         = {Machine review of arXiv:2507.09989}
}
abstract

In heterogeneous multi-agent reinforcement learning (MARL), achieving monotonic improvement plays a pivotal role in enhancing performance. The HAPPO algorithm proposes a feasible solution by introducing a sequential update scheme, which requires independent learning with No Parameter-sharing (NoPS). However, heterogeneous MARL generally requires Partial Parameter-sharing (ParPS) based on agent grouping to achieve high cooperative performance. Our experiments prove that directly combining ParPS with the sequential update scheme leads to the policy updating baseline drift problem, thereby failing to achieve improvement. To solve the conflict between monotonic improvement and ParPS, we propose the Optimal Marginal Deterministic Policy Gradient (OMDPG) algorithm. First, we replace the sequentially computed $Q_{\psi}^s(s,a_{1:i})$ with the Optimal Marginal Q (OMQ) function $\phi_{\psi}^*(s,a_{1:i})$ derived from Q-functions. This maintains MAAD's monotonic improvement while eliminating the conflict through optimal joint action sequences instead of sequential policy ratio calculations. Second, we introduce the Generalized Q Critic (GQC) as the critic function, employing pessimistic uncertainty-constrained loss to optimize different Q-value estimations. This provides the required Q-values for OMQ computation and stable baselines for actor updates. Finally, we implement a Centralized Critic Grouped Actor (CCGA) architecture that simultaneously achieves ParPS in local policy networks and accurate global Q-function computation. Experimental results in SMAC and MAMuJoCo environments demonstrate that OMDPG outperforms various state-of-the-art MARL baselines.

Figures

Figures reproduced from arXiv: 2507.09989 by the authors.

Figure 1
Figure 1. Schematic diagram of parameter sharing and training paradigms in different [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Validation experiments using HAPPO as baseline in SMAC environments. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Architecture of the OMDPG algorithm. group-specific observation trajectories τGi to generate corresponding group actions aGi , forming the Centralized Critic Grouped Actor (CCGA) architecture. This enables effective balance between intra-group policy similarity and inter-group cooperation. The temporal difference (TD) loss function for actor networks aims to maximize the action values estimated by critic networks. B… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparative Experimental Curves in SMAC Environments [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparative Learning Curves in MAMuJoCo Environments [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Ablation Experiments in SMAC Environments [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 25 canonical work pages

  1. [1]

    Multi-Agent Reinforcement Learning for Power Control in Wireless Networks via Adaptive Graphs

    Amorosa, L.M., Skocaj, M., Verdone, R., Gündüz, D.: Multi-agent reinforcement learning for power control in wireless networks via adaptive graphs. arXiv preprint arXiv:2311.15858 (2023)

    work page Pith review arXiv 2023

  2. [2]

    Proceedings of the International Conference on Learning Representations (2022)

    Bai, C., Wang, L., Yang, Z., Deng, Z., Garg, A., Liu, P., Wang, Z.: Pessimistic bootstrap- ping for uncertainty-driven offline reinforcement learning. Proceedings of the International Conference on Learning Representations (2022)

    work page 2022

  3. [3]

    Pro- ceedings of the 2023 International Conference on Autonomous Agents and Multiagent Sys- tems pp

    Bettini, M., Shankar, A., Prorok, A.: Heterogeneous multi-robot reinforcement learning. Pro- ceedings of the 2023 International Conference on Autonomous Agents and Multiagent Sys- tems pp. 1485–1494 (2023)

    work page 2023

  4. [4]

    Joint European Conference on Machine Learning and Knowledge Discovery in Databases pp

    Bono, G., Dibangoye, J.S., Matignon, L., Pereyron, F., Simonin, O.: Cooperative multi-agent policy gradient. Joint European Conference on Machine Learning and Knowledge Discovery in Databases pp. 459–476 (2018)

    work page 2018

  5. [5]

    International Joint Conference on Artificial Intelligence (2024)

    Chen, R., Zhang, X., Du, Y ., Zhong, Y ., Tian, Z., Sun, F., Yang, Y .: Off-agent trust region policy optimization. International Joint Conference on Artificial Intelligence (2024)

    work page 2024

  6. [6]

    arXiv preprint arXiv:2109.11251 (2021)

    Kuba, J.G., Chen, R., Wen, M., Wen, Y ., Sun, F., Wang, J., Yang, Y .: Trust region policy optimisation in multi-agent reinforcement learning. arXiv preprint arXiv:2109.11251 (2021)

    arXiv 2021

  7. [7]

    the Thirty-Eighth Annual Conference on Neural Information Pro- cessing Systems (NeurIPS) (2024)

    Li, X., Pan, L., Zhang, J.: Kaleidoscope: Learnable masks for heterogeneous multi-agent reinforcement learning. the Thirty-Eighth Annual Conference on Neural Information Pro- cessing Systems (NeurIPS) (2024)

    work page 2024

  8. [8]

    CoRR (2023)

    Liu, J., Zhong, Y ., Hu, S., Fu, H., Fu, Q., Chang, X., Yang, Y .: Maximum entropy heterogeneous-agent mirror learning. CoRR (2023)

    work page 2023

Show all 29 references
  1. [9]

    The Twelfth International Conference on Learning Representations (2024)

    Liu, J., Zhong, Y ., Hu, S., Fu, H., Fu, Q., Chang, X., Yang, Y .: Maximum entropy heterogeneous-agent reinforcement learning. The Twelfth International Conference on Learning Representations (2024)

    2024

  2. [10]

    The Twelfth International Conference on Learning Representations (2024) Title Suppressed Due to Excessive Length 13

    Liu, S., Xing, D., Gu, P., Wang, X., An, B., Liu, Y .: Solving homogeneous and heterogeneous cooperative tasks with greedy sequential execution. The Twelfth International Conference on Learning Representations (2024) Title Suppressed Due to Excessive Length 13

    2024

  3. [11]

    Neural Information Processing Systems (NIPS) (2017)

    Lowe, R., Wu, Y ., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. Neural Information Processing Systems (NIPS) (2017)

    2017

  4. [12]

    Advances in Neural Information Processing Systems 32 (2019)

    Mahajan, A., Rashid, T., Samvelyan, M., Whiteson, S.: Maven: Multi-agent variational ex- ploration. Advances in Neural Information Processing Systems 32 (2019)

    2019

  5. [13]

    Springer (2016)

    Oliehoek, F.A., Amato, C.: A concise introduction to decentralized POMDPs. Springer (2016)

    2016

  6. [14]

    Applied Intelligence 53(4), 4483–4498 (2023)

    Qiao, Z., Ke, L., Wang, X.: Traffic signal control using a cooperative ewma-based multi- agent reinforcement learning. Applied Intelligence 53(4), 4483–4498 (2023)

    2023

  7. [15]

    The Journal of Machine Learning Research 21(1), 7234–7284 (2020)

    Rashid, T., Samvelyan, M., De Witt, C.S., Farquhar, G., Foerster, J., Whiteson, S.: Mono- tonic value function factorisation for deep multi-agent reinforcement learning. The Journal of Machine Learning Research 21(1), 7234–7284 (2020)

    2020

  8. [16]

    arXiv preprint arXiv:2407.05800 (2024)

    Sahoo, P., Tripathi, A., Saha, S., Mondal, S.: Fedmrl: Data heterogeneity aware fed- erated multi-agent deep reinforcement learning for medical imaging. arXiv preprint arXiv:2407.05800 (2024)

    2024 arXiv

  9. [17]

    Neural Computing and Applications 35(27), 19765–19781 (2023)

    Sun, Z., Wu, H., Shi, Y ., Yu, X., Gao, Y ., Pei, W., Yang, Z., Piao, H., Hou, Y .: Multi-agent air combat with two-stage graph-attention communication. Neural Computing and Applications 35(27), 19765–19781 (2023)

    2023

  10. [18]

    Advances in Neural Information Processing Systems 35, 16509–16521 (2022)

    Wen, M., Kuba, J., Lin, R., Zhang, W., Wen, Y ., Wang, J., Yang, Y .: Multi-agent reinforce- ment learning is a sequence modeling problem. Advances in Neural Information Processing Systems 35, 16509–16521 (2022)

    2022

  11. [19]

    Theses and Dissertations

    Wilson, R.J.: Performance of heterogeneous multi-agent systems with applications in com- bined arms. Theses and Dissertations. 5330 (2022)

    2022

  12. [20]

    The International FLAIRS Conference Proceedings, 35 (2022)

    Wilson, R.J., King, D.W., Peterson, G.L.: Evolution of combined arms tactics in heteroge- neous multi-agent teams. The International FLAIRS Conference Proceedings, 35 (2022)

    2022

  13. [21]

    IEEE Transactions on Vehicular Technology69(8), 8243–8256 (2020)

    Wu, T., Zhou, P., Liu, K., Yuan, Y ., Wang, X., Huang, H., Wu, D.O.: Multi-agent deep rein- forcement learning for urban traffic light control in vehicular networks. IEEE Transactions on Vehicular Technology69(8), 8243–8256 (2020)

    2020

  14. [22]

    arXiv preprint arXiv:2401.05799 (2024)

    Xing, F.: Designing heterogeneous llm agents for financial sentiment analysis. arXiv preprint arXiv:2401.05799 (2024)

    2024 arXiv

  15. [23]

    2021 IEEE International Confer- ence on Systems, Man, and Cybernetics (SMC) pp

    Yang, Q., Parasuraman, R.: How can robots trust each other for better cooperation? a relative needs entropy based robot-robot trust assessment model. 2021 IEEE International Confer- ence on Systems, Man, and Cybernetics (SMC) pp. 2656–2663 (2021)

    2021

  16. [24]

    AIAA Scitech 2019 Forum p

    Yoon, H.J., Chen, H., Long, K., Zhang, H., Gahlawat, A., Lee, D., Hovakimyan, N.: Learn- ing to communicate: A machine learning framework for heterogeneous multi-agent robotic systems. AIAA Scitech 2019 Forum p. 1456 (2019)

    2019

  17. [25]

    arXiv preprint arXiv:2103.01955 (2021)

    Yu, C., Velu, A., Vinitsky, E., Wang, Y ., Bayen, A., Wu, Y .: The surprising effectiveness of ppo in cooperative, multi-agent games. arXiv preprint arXiv:2103.01955 (2021)

    2021 arXiv

  18. [26]

    Applied Sciences 15(5), 2580 (2025)

    Yu, X., Lin, Y ., Wang, S., Han, S.: Solving action semantic conflict in physically heteroge- neous multi-agent reinforcement learning with generalized action-prediction optimization. Applied Sciences 15(5), 2580 (2025)

    2025

  19. [27]

    Complex & Intelligent Systems pp

    Yu, X., Lin, Y ., Wang, X., Han, S., Lv, K.: Ghq: grouped hybrid q-learning for cooperative heterogeneous multi-agent reinforcement learning. Complex & Intelligent Systems pp. 1–20 (2024)

    2024

  20. [28]

    Neurocomputing 411, 206–215 (2020)

    Zhang, F., Li, J., Li, Z.: A td3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment. Neurocomputing 411, 206–215 (2020)

    2020

  21. [29]

    Autonomous Agents and Multi-Agent Systems 38(1), 4 (2024)

    Zhu, C., Dastani, M., Wang, S.: A survey of multi-agent deep reinforcement learning with communication. Autonomous Agents and Multi-Agent Systems 38(1), 4 (2024)

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.