Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control

Jieyu Lin; Qi Zhang; Sai Qian Zhang

REVIEW 3 major objections 6 minor 25 references

Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that suppressing low-variance message traffic reduces communication overhead by 2-10x in cooperative multi-agent RL while improving coordination.

desk verdict VBC is a genuinely useful empirical method for cutting communication overhead in cooperative MARL, but the convergence proof doesn't analyze the actual algorithm and should be removed or rewritten. read the letter →

arxiv 1909.02682 v2 pith:5UWTTXYI submitted 2019-09-06 cs.LG stat.ML

Sai Qian Zhang , Qi Zhang , Jieyu Lin This is my paper · ORCID

classification cs.LGstat.ML

keywords multi-agentreinforcementlearningcommunicationefficiencyvariance-basedcontrolmessagevariancecentralizedtrainingdecentralizedexecutionStarCraftIIdeepQ-learningprotocol

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that in cooperative multi-agent reinforcement learning, the noisy, uninformative component of inter-agent messages can be identified by the variance of the message encoder's output and suppressed. It proposes Variance Based Control (VBC), a plug-in addition to multi-agent Q-learning that adds a variance penalty to the training loss and uses a run-time communication protocol: an agent asks for help only when its local action values are close, and replies only when its own message variance is high. On six StarCraft II micromanagement tasks, VBC built on VDN and QMIX reports 2-10x lower communication overhead than the SchedNet baseline while matching or improving winning rates. A tabular convergence result bounds the deviation from the optimal joint Q-function by a term proportional to the variance-penalty weight. The paper's bottom line is that sparser, variance-gated communication is not just cheaper but can be more effective than full communication.

What carries the argument

The machinery is the variance-based control loop: a training-time penalty $\lambda \sum_{i=1}^N \mathrm{Var}(f_{\mathrm{enc}}(c_i^t))$ on message-encoder outputs, paired with run-time thresholds $\delta_1$ and $\delta_2$ in the request/reply protocol. The message encoders are MLPs that take another agent's recurrent state $c_j^t$ and output a vector that is summed elementwise with the local action values, so the combiner needs no extra parameters. The same variance statistic is doing two jobs: as a regularizer it forces messages to be low-noise, and as a gate it decides whether a message can change the recipient's decision; the architecture deliberately avoids a learned communication-scheduling module.

What would settle it

Replace the reply gate with the opposite rule, replying only when $\mathrm{Var}(f_{\mathrm{enc}}(c_i^t))$ is below $\delta_2$, while keeping the request rule and total communication volume fixed; if a SMAC winning rate does not drop, variance is not the property doing the work.

Watch

Extended reading notes

Core claim

The central claim is that the variance of a message encoder's output, $\mathrm{Var}(f_{\mathrm{enc}}(c_i^t))$, is a usable online proxy for whether a message will change the recipient's action. VBC enforces this idea twice. During training, the joint Q-learning loss gains a term $\lambda \sum_i \mathrm{Var}(f_{\mathrm{enc}}(c_i^t))$, which squeezes the messages toward a low-variance, denoised representation. During execution, agent $i$ broadcasts a request only when the gap between its top two local action values $m_1 - m_2 < \delta_1$, and answers a request only when $\mathrm{Var}(f_{\mathrm{enc}}(c_i^t)) \ge \delta_2$. In the reported experiments, this pair of rules produces communication overhead of 4-36% of all agent pairs across the six SMAC maps, versus 33-67% for the fixed-schedule SchedNet baseline, while VBC's winning rates match or exceed those of full communication and clearly exceed those of the no-communication baselines.

Load-bearing premise

The load-bearing premise is that the variance of a message encoder's output tracks how informative the message is: high-variance messages are useful, low-variance messages are noise, and discarding the latter does not lose coordination-relevant information.

Editorial extensions

If this is right

VBC can be wrapped around either the VDN or QMIX mixing networks, and on the six SMAC maps it reports winning rates near or above those of full communication while transmitting far fewer messages.
Reported overhead drops to 5.25% and 4.33% on MMM and 2s3z, about 10x lower than SchedNet's 50% and 60%, and 2-6x lower on the other four maps.
The learned strategies in asymmetric battles are emergent rather than imposed: Hydralisks form a row at the map edge, Stalkers kite with a shield, and Zealots split into bait and kill groups, with communication concentrated at episode start and after a formation breaks.
The tabular convergence theorem gives a bound $\|Q^k_{\mathrm{tot}} - Q^*_{\mathrm{tot}}\| \le \lambda N G$ as $k \to \infty$, so the variance penalty degrades the asymptotic Q-error in a controlled, linear way.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Sweeping $\lambda$, $\delta_1$, and $\delta_2$ over a grid on a fixed SMAC map would map the communication-performance frontier implied by the method, and could show how far overhead can be reduced before winning rate falls.
The same 'low variance means removable noise' principle could be applied to other bandwidth-limited channels in MARL, such as gradient exchanges or quantized messages, where a cheap variance statistic could gate transmission.
The protocol assumes requests are nearly free compared with messages; in a channel with per-packet costs or latency, the $\delta_1$ threshold would need to be retuned, and the reported overhead advantage might not transfer directly.
A control experiment that sends messages at the same rate but selected by low variance instead of high variance would isolate whether variance, rather than communication volume, is what improves coordination.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: VBC is a worthwhile empirical paper with a clean, plug-in idea, and the authors were honest enough to include a full-communication ablation. But the theory section is not analyzing the algorithm, and the core assumption that message variance tracks decision-relevant information is left untested. Read it as an empirical method paper; treat the proof with skepticism.

What is new: the two-threshold protocol — a local confidence gap (δ1) plus a message-variance gate (δ2) — is a simple and elegant alternative to learned schedulers like SchedNet. It builds directly on VDN/QMIX, requires no extra decision module, and still cuts communication by 2–10× on the six SMAC tasks while matching or beating full-communication baselines on most of them. The MPE results and the strategy analysis add useful evidence. The FC baseline is a good control.

Soft spots: the convergence proof in Section 5 and Appendix 8.1 does not hold. Equation (2) differentiates Var(f_enc(c_i)) with respect to Q_tot, but in the architecture Q_tot is a function of the message encoder outputs, not the reverse. That derivative is undefined in the deep network and vacuous in the tabular setting where no f_enc exists. Theorem 1's bound is also proportional to λNG, the variance penalty term itself, so it partially restates the loss. I would cut the theorem or rewrite it as a stability argument for a simplified linear model. The variance-as-informativeness proxy is plausible but not validated: a high-variance message can fail to change the argmax, and a low-variance one can flip a near-tie. The per-task tuning of λ, δ1, δ2 is a minor concern; the overhead numbers are operating points, not a free lunch. None of this sinks the empirical claim.

Who it is for: researchers working on communication-efficient MARL will get real value from the method and the benchmark comparisons. It deserves serious referee time; the right decision is to accept after the theory is removed or fixed.

Referee Report

3 major / 6 minor

Summary. This paper proposes Variance Based Control (VBC), a technique for reducing communication overhead in cooperative multi-agent reinforcement learning. VBC adds a variance penalty on message-encoder outputs to the QMIX/VDN loss (Eq. 1) and, during execution, uses a two-threshold protocol (Algorithm 1): an agent requests messages from other agents only when the gap between its top two local Q-values is below δ1, and an agent replies only when the variance of its message vector exceeds δ2. The method is evaluated on six StarCraft Multi-Agent Challenge maps and two multi-particle environments, reporting winning rates comparable to or better than full-communication and SchedNet baselines while using substantially fewer message exchanges (Table 1). The paper also states a convergence theorem (Theorem 1) claiming that the joint Q-table converges to within λNG of the optimal Q-function.

Significance. If the empirical claims hold, VBC is an appealingly simple contribution: it avoids a learned communication-scheduling module, works with both VDN and QMIX mixing networks, and the authors provide code and a demo. The SMAC results show consistent improvements over communication-free baselines and, on several maps, over SchedNet at lower communication counts. However, the theoretical convergence result is not a valid analysis of the implemented algorithm, and the central proxy assumption (message variance tracks message informativeness) is asserted rather than tested. The significance of the paper therefore rests on the empirical validation, which is suggestive but not yet sufficient to establish the abstract's causal claim that reducing variance removes noise while preserving useful information.

major comments (3)

[Section 5, Eq. (2); Appendix Theorem 2] The convergence analysis differentiates Var(f_enc(c_i^t)) with respect to Q_tot^k(o_t,a_t), but in the proposed architecture Q_tot is a function of the message-encoder outputs, not the reverse; f_enc depends on the encoder parameters and on c_i^t, and in the tabular setting where Eq. (2) is supposed to apply there is no f_enc at all. This derivative is undefined, so Theorem 1 does not analyze the algorithm that is actually executed and the 'guaranteed stability' claim in Section 1 is unsupported. The theorem should be removed or replaced with a statement about a regularized update under a well-defined gradient.
[Section 4.2, Eq. (1); Algorithm 1, lines 11-13] The core mechanism rests on the claim that high-variance messages are informative and low-variance messages are noise. This is used twice: as a training penalty in Eq. (1) and as the reply gate in Algorithm 1. The paper provides no evidence that Var(f_enc) tracks the ability of a message to change the recipient's action. High variance is not sufficient (a high-variance vector can leave the argmax unchanged) and not necessary (a low-variance vector can flip a decision when the local margin m1-m2 is small, as in the example of Figure 2). The thresholds δ1 and δ2 are tuned per task and per variant (Figure 3 legends), so the reported overhead figures are operating points of this proxy, not independent evidence that variance selects informative messages. A direct validation (e.g., whether a message actually changes the recipient's action in test episodes) and a sensitivity analysis over thresholds are needed.
[Section 6.2, Table 1; Algorithm 1] The communication-overhead comparison counts only f_enc replies (pairs of agents that 'conduct communication'), but Algorithm 1 also sends a request to all other agents whenever m1-m2 < δ1. Requests are described as low bandwidth but are still messages and are not counted in β; moreover, a pair exchanging a small request plus a large f_enc vector is weighted the same as a pair exchanging only an f_enc reply. Since the abstract's headline is a 2-10x reduction in communication overhead, the metric should be reported in total bits or at least in total message counts including requests, and the 2-10x factor should be confirmed under that measure.

minor comments (6)

[Section 1 and Section 6.1] There are naming typos: 'StarCraft Multi-Agent Chanllenge' should be 'Challenge', the task called '2s5z' in the introduction should be '3s5z', and Section 6.1 refers to '2s_vs_3z' where the rest of the paper uses '2s3z'.
[Figure 3] The legends render hyperparameters as '( = 5.0, 1 = 0.04, 2 = 0.02)' with the symbols λ, δ1, and δ2 missing; this makes the per-task tuning hard to read and should be fixed.
[Algorithm 1] Line 11 stores f_enc(c_i^t) in a buffer when its variance exceeds δ2, but the buffer is never read in the algorithm; this appears to be dead code or a missing step where buffered messages are sent on subsequent requests.
[Section 4.1] The parameter-sharing statement for the message encoders is unclear: the text says θ_enc^{ij} is the same for all i and j, but Figure 1(a) shows separate 'Msg Enc' modules for each agent; please state explicitly whether all sender-receiver pairs share one global encoder or each sender has its own encoder with shared weights.
[Section 3 and Reference list] The background section cites 'VDN [14] and QMIX [18]', but the bibliography lists [14] as QMIX and [18] as VDN; the citations are swapped and should be corrected.
[Appendix, Eq. (4)] Equation (4) uses δ_t on the right-hand side where it should use δ_k; this typo makes the proof harder to follow.

Circularity Check

3 steps flagged · score 4.0 of 10

Partial circularity: 'informative' is defined as high message variance, and the reported overhead and convergence bounds are operating points of the tuned thresholds and the inserted penalty; the FC/SchedNet comparisons provide independent empirical grounding.

self definitional [Section 4, first paragraph (communication protocol); Algorithm 1, lines 11-13]
"the agent replies only if its feedback is informative, namely the variance of the feedback is high. ... Calculate variance of f_enc(c_i^t), if Var(f_enc(c_i^t))≥δ2, store f_enc(c_i^t) in the buffer."

The paper's central claim is that limiting message variance 'eliminate[s] the noisy component ... while the useful part can be preserved.' But the reply rule defines 'informative' as Var(f_enc) ≥ δ2. Therefore the assertion that variance-filtered messages are the useful ones is true by definition of the filter, not by an independent measure of informativeness. The follow-on inference that high-variance messages 'may change the current action decision' is also asserted from the same variance criterion, even though a low-variance message can flip a small action gap. The empirical wins over FC and SchedNet are independent evidence that the overall pipeline works, so this is partial rather than total circularity.
fitted input called prediction [Section 6.1 hyperparameter tuning; Section 6.2, Table 1]
"For hyperparameters used by VBC (i.e., λ used in equation (1), δ1, δ2 in Algorithm 1), we first search for a coarse parameter range based on random trial, experience and message statistics. We then perform a random search within a smaller hyperparameter space. ... To quantify the amount of communication, we run Algorithm 1 and count the total number of pairs of agents g_t that conduct communication for each timestep t, then divided by the total number of pairs of agents in the user group, R."

The headline 2-10x overhead reduction is an operating point of the tuned thresholds δ1 and δ2. Algorithm 1 suppresses communication whenever the local action gap m1-m2 ≥ δ1 and suppresses replies whenever Var(f_enc) < δ2, so the reported β = (1/T) Σ g_t / R is, by construction, the empirical frequency of the gating event (low local confidence and high message variance). Because δ1 and δ2 are per-task tuned to produce the reported behavior, the overhead figure is selected by the thresholds rather than independently predicted by the method. The comparison against SchedNet's fixed K remains a meaningful engineering benchmark, so this is partial, not full, circularity.

1 more flagged steps

other [Section 5, Equation (2) and Theorem 1; Appendix 8.1, proof]
"Q^{k+1}_tot(o_t,a_t) = Q^k_tot(o_t,a_t) + η_k [ r_t + γ max_a Q^k_tot(o_{t+1},a) − Q^k_tot(o_t,a_t) − λ Σ_{i=1}^N ∂Var(f_enc(c_i^t))/∂Q^k_tot(o_t,a_t) ] ... we have ||Q^k_tot(o_t,a_t) − Q^*_tot(o_t,a_t)|| ≤ λNG ... where G satisfies ||∂Var(f_enc(c_i^t))/∂Q^k_tot(o_t,a_t)|| ≤ G."

The convergence 'guarantee' is a bound on the magnitude of the very penalty term inserted into the update rule: the proof splits the error into a standard Q-learning term that converges to zero and a term driven by −η λ U_k, which by construction converges to a value no larger than λNG. Thus Theorem 1 states only that a bounded added penalty produces bounded bias; it does not establish a property specific to variance-based control. Moreover, in the actual architecture Q_tot is a function of the encoder outputs f_enc, not the reverse, so the derivative ∂Var(f_enc)/∂Q_tot in Eq. (2) is undefined for the deep network and vacuous in the tabular setting. The analyzed update is therefore not the gradient of the loss in Eq.

full rationale

The paper proposes VBC as a variance-penalty and variance-gating mechanism and evaluates it empirically against VDN, QMIX, SchedNet, and full communication. The performance comparisons are self-contained and do not reduce to the paper's own design choices: the winning-rate curves show VBC matching or beating FC and SchedNet on StarCraft II, cooperative navigation, and predator-prey, so the central performance claim has independent empirical content. However, two parts of the presentation are partially circular. First, the method's conceptual justification equates 'informative' messages with high variance by construction: the reply gate and the training penalty both use Var(f_enc) as the definition of usefulness, so the statement that variance filtering preserves useful information is a restatement of the filter rather than a verified property. Second, the reported 2-10x communication-overhead reduction is a direct consequence of the per-task tuned thresholds δ1 and δ2; since Algorithm 1's gating conditions define when communication occurs, the measured β is the selected operating point of those thresholds, not an independent prediction. The convergence theorem is also essentially self-referential: its error bound λNG is the size of the penalty term inserted into Eq. (2), and the derivative that defines G does not exist in the actual architecture. These issues are correctness and circularity concerns, but the empirical benchmarks against external baselines prevent the paper from being fully circular. Score 4 reflects partial circularity with substantial independent evaluation.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central mechanism rests on two ad hoc assumptions (variance equals informativeness, and a differentiable variance penalty) plus three per-task tuned hyperparameters. No new physical entities are introduced.

free parameters (3)

lambda (variance penalty weight) = Per task: 5.0 (MMM, 3s5z, 6z_vs_24zerg), 4.0 (2s3z), 2.0 (3s_vs_4z), 1.7 (6h_vs_8z)
Hyperparameter in Eq 1 controlling the strength of the variance penalty; chosen by random search per task.
delta1 (local confidence threshold) = Per task: 0.04 (MMM), 0.03 (2s3z), 0.1 (3s5z), 0.06/0.07 (3s_vs_4z), 0.11/0.23 (6h_vs_8z), 0.04 (6z_vs_24zerg)
Threshold in Algorithm 1 deciding whether an agent broadcasts a request; tuned per task.
delta2 (message variance reply threshold) = Per task: 0.02 (MMM), 0.015 (2s3z), 0.15 (3s5z), 0.002/0.004 (3s_vs_4z), 0.03/0.2 (6h_vs_8z), 0.023/0.019 (6z_vs_24zerg)
Threshold in Algorithm 1 deciding whether an agent replies with its message; tuned per task.

assumptions (3)

ad hoc to paper Variance of message encoder output is a valid measure of message informativeness.
The whole communication gating mechanism (Eq 1, Algorithm 1) presumes low-variance messages are noise and high-variance messages are useful; no independent evidence is provided.
ad hoc to paper The gradient dVar(fenc(ct_i))/dQk_tot(ot,at) exists and is bounded by G.
Eq 2 and Theorem 1 require this derivative, but fenc depends on the GRU hidden state ct_i, not on Q_tot, so this quantity is undefined in the stated architecture.
standard math Standard Q-learning convergence conditions (finite states/actions, Robbins-Monro learning rates) hold.
Theorem 1 relies on [21,22] and assumes 0 <= eta_k <= 1, sum eta_k = infinity, sum eta_k^2 < infinity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control." pith.science (2026). https://pith.science/paper/5UWTTXYI

@misc{pith2026190902682,
  author       = {Pith},
  title        = {Pith review of: Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5UWTTXYI}},
  note         = {Machine review of arXiv:1909.02682}
}

abstract

Multi-agent reinforcement learning (MARL) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in MARL. In this work, we propose Variance Based Control (VBC), a simple yet efficient technique to improve communication efficiency in MARL. By limiting the variance of the exchanged messages between agents during the training phase, the noisy component in the messages can be eliminated effectively, while the useful part can be preserved and utilized by the agents for better performance. Our evaluation using a challenging set of StarCraft II benchmarks indicates that our method achieves $2-10\times$ lower in communication overhead than state-of-the-art MARL algorithms, while allowing agents to better collaborate by developing sophisticated strategies.

Figures

Figures reproduced from arXiv: 1909.02682 by the authors.

**Figure 2.** An example on communication protocol of the agents during execution. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Winning rates for the six tasks, the shaded regions represent the 95% confidence intervals. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Strategies and communication pattern for different scenarios [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: (a) Results on PP with 3 predators and 3 prey. (b) shows results of CN. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 18 canonical work pages

[1]

Shield levels, health levels and cooldown levels of all the units at t

work page
[2]

The actions taken by all the units at t− 1

work page
[3]

For all the six battles, each allied or enemy agent has a sight range of 9 and shooting range of 6 for all types of agents

The x,y coordinates of all the units relative to the center of the map at t. For all the six battles, each allied or enemy agent has a sight range of 9 and shooting range of 6 for all types of agents. For additional information, please refer to [15]. 9.2 Hyperparameter For network of agenti, at timestept, raw observationot i is ﬁrst passed through a singl...

work page
[4]

Starcraft ofﬁcial game site: https://starcraft2.com/

work page
[5]

VBC Video demo: https://bit.ly/2VFkvCZ

work page
[6]

Foerster, I

J. Foerster, I. A. Assael, N. de Freitas, and S. Whiteson. Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 2137–2145, 2016

work page 2016
[7]

J. N. Foerster, C. A. S. de Witt, G. Farquhar, P. H. Torr, W. Boehmer, and S. Whiteson. Multi-agent common knowledge reinforcement learning. arXiv preprint arXiv:1810.11702, 2018

work page Pith review arXiv 2018
[8]

J. N. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson. Counterfactual multi-agent policy gradients. In Thirty-Second AAAI Conference on Artiﬁcial Intelligence, 2018

work page 2018

Show all 25 references

[9]

Hausknecht and P

M. Hausknecht and P. Stone. Deep recurrent q-learning for partially observable mdps. In 2015 AAAI Fall Symposium Series, 2015

2015
[10]

Jiang and Z

J. Jiang and Z. Lu. Learning attentional communication for multi-agent cooperation. In Advances in Neural Information Processing Systems, pages 7254–7264, 2018

2018
[11]

D. Kim, S. Moon, D. Hostallero, W. J. Kang, T. Lee, K. Son, and Y . Yi. Learning to schedule communication in multi-agent reinforcement learning. arXiv preprint arXiv:1902.01554, 2019

1902 arXiv
[12]

reinforcement learning in robotics: A survey

J. Kober, J. A. Bagnell, and J. Peters. "reinforcement learning in robotics: A survey.". The International Journal of Robotics Research, 2013

2013
[13]

R. Lowe, Y . Wu, A. Tamar, J. Harb, O. P. Abbeel, and I. Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems, pages 6379–6390, 2017

2017
[14]

a uniﬁed game-theoretic approach to multiagent reinforcement learning

L. Marc, V . Zambaldi, A. Gruslys, A. Lazaridou, K. Tuyls, J. Pérolat, D. Silver, and T. Graepel. "a uniﬁed game-theoretic approach to multiagent reinforcement learning.". In Advances in Neural Information Processing Systems, 2017

2017
[15]

playing atari with deep reinforcement learning

V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. "playing atari with deep reinforcement learning.". arXiv preprint arXiv:1312.5602, 2013

2013 arXiv
[16]

P. Peng, Y . Wen, Y . Yang, Q. Yuan, Z. Tang, H. Long, and J. Wang. Multiagent bidirectionally- coordinated nets: Emergence of human-level coordination in learning to play starcraft combat games. arXiv preprint arXiv:1703.10069, 2017

2017 arXiv
[17]

Rashid, M

T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. Foerster, and S. Whiteson. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. arXiv preprint arXiv:1803.11485, 2018

2018 arXiv
[18]

Samvelyan, T

M. Samvelyan, T. Rashid, C. S. de Witt, G. Farquhar, N. Nardelli, T. G. J. Rudner, C.-M. Hung, P. H. S. Torr, J. Foerster, and S. Whiteson. The StarCraft Multi-Agent Challenge. CoRR, abs/1902.04043, 2019

1902 arXiv
[19]

safe, multi-agent, reinforcement learning for autonomous driving

S.-S. Shai, S. Shammah, and A. Shashua. "safe, multi-agent, reinforcement learning for autonomous driving.". arXiv preprint arXiv:1610.03295, 2016

2016 arXiv
[20]

Sukhbaatar, R

S. Sukhbaatar, R. Fergus, et al. Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems, pages 2244–2252, 2016

2016
[21]

Sunehag, G

P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V . Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, et al. Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296, 2017

2017 arXiv
[22]

Tampuu, T

A. Tampuu, T. Matiisen, D. Kodelja, I. Kuzovkin, K. Korjus, J. Aru, J. Aru, and R. Vicente. Multiagent cooperation and competition with deep reinforcement learning. PloS one, 12(4):e0172395, 2017. 12

2017
[23]

multi-agent reinforcement learning: Independent vs. cooperative agents

M. Tan. "multi-agent reinforcement learning: Independent vs. cooperative agents.". In Proceedings of the tenth international conference on machine learning. IEEE, 1993

1993
[24]

Jaakkola, M

T. Jaakkola, M. I. Jordan, and S. P. Singh. Convergence of stochastic iterative dynamic programming algorithms. In Advances in neural information processing systems, pages 703–710, 1994

1994
[25]

F. S. Melo. Convergence of q-learning: A simple proof. Institute Of Systems and Robotics, Tech. Rep, pages 1–4, 2001. 13

2001

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Shield levels, health levels and cooldown levels of all the units at t

work page

[2] [2]

The actions taken by all the units at t− 1

work page

[3] [3]

For all the six battles, each allied or enemy agent has a sight range of 9 and shooting range of 6 for all types of agents

The x,y coordinates of all the units relative to the center of the map at t. For all the six battles, each allied or enemy agent has a sight range of 9 and shooting range of 6 for all types of agents. For additional information, please refer to [15]. 9.2 Hyperparameter For network of agenti, at timestept, raw observationot i is ﬁrst passed through a singl...

work page

[4] [4]

Starcraft ofﬁcial game site: https://starcraft2.com/

work page

[5] [5]

VBC Video demo: https://bit.ly/2VFkvCZ

work page

[6] [6]

Foerster, I

J. Foerster, I. A. Assael, N. de Freitas, and S. Whiteson. Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 2137–2145, 2016

work page 2016

[7] [7]

J. N. Foerster, C. A. S. de Witt, G. Farquhar, P. H. Torr, W. Boehmer, and S. Whiteson. Multi-agent common knowledge reinforcement learning. arXiv preprint arXiv:1810.11702, 2018

work page Pith review arXiv 2018

[8] [8]

J. N. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson. Counterfactual multi-agent policy gradients. In Thirty-Second AAAI Conference on Artiﬁcial Intelligence, 2018

work page 2018