Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning

Byoung-Tak Zhang; Kibeom Kim; Minsu Lee; Min Whoo Lee; Soo Wung Shin

REVIEW 4 major objections 4 minor 26 references

Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that cooperative agents can adapt to out-of-distribution changes by communicating a compressed version of their observation-prediction error, and demonstrates the benefit in a multi-robot warehouse task.

desk verdict A plausible, clearly written communication scheme for OOD MARL, but the current experiments do not yet isolate the 'unexpectedness' mechanism from channel capacity and hyperparameter confounds. read the letter →

arxiv 2501.01140 v1 pith:ZXSERH2O submitted 2025-01-02 cs.MA

Min Whoo Lee , Kibeom Kim , Soo Wung Shin , Minsu Lee , Byoung-Tak Zhang This is my paper · ORCID

classification cs.MA

keywords multi-agentreinforcementlearningout-of-distributiongeneralizationagentcommunicationdecentralizedtrainingforwarddynamicsmodelpredictionerrorfew-shotadaptationmulti-robotwarehouse

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Out-of-distribution situations are events an agent never saw in training, and a multi-agent team can adapt to them only if one agent's surprise can reach the others. This paper claims that surprise can be generated automatically: each agent learns a forward model that predicts its next local observation, measures the difference between the prediction and the observation it actually receives, compresses that difference with an autoencoder, and broadcasts the compressed vector as a message. When this 'unexpectedness encoding' is concatenated with a message trained by environmental reward, two decentralized agents in the Multi-Robot Warehouse environment deliver more shelves on the training distribution and after two designed distribution shifts than either message type alone or no communication. In the reported experiments, the combined scheme reaches 2.51 delivered shelves per episode in training, 2.29 after a goal-layout shift, and 1.45 after a shelf-layout shift. If the claim is right, communicating prediction error is a practical route to few-shot adaptation in partially observable cooperative tasks.

What carries the argument

The load-bearing mechanism is the Unexpectedness Encoding Module (UEM). At each step, agent $i$'s forward model $f$ predicts the next projected observation $g(\hat{o}^i_t)$ from the previous projected observation $g(o^i_{t-1})$, the previous action $a^i_{t-1}$, and the messages it received; the difference between this prediction and the actual projected observation $g(o^i_t)$ is the 'unexpectedness' vector. A reconstruction-trained autoencoder maps that vector into a short continuous message, which is concatenated with a reward-trained binary message to form the agent's broadcast. This concatenated message is the only channel through which one agent's unexpected experience can reach another, and the paper's ablations show that removing either half of it removes the benefit.

What would settle it

Compare UES+R with a control that concatenates the reward-driven message with an encoding of random noise of the same length and trains it under the same 10M-step protocol and 10-episode fine-tuning. If the control matches UES+R's delivered-shelf counts under Goal-Shift and Shelf-Shift, then the semantic content of the unexpectedness encoding is not what drives the reported improvement.

Watch

Extended reading notes

Core claim

The central claim is that in cooperative tasks with partial observability, the discrepancy between what an agent expects to observe and what it actually observes is a communicable signal that helps the whole team adapt to distribution shift. The proposed UES+R scheme computes this discrepancy on a linear projection of the observation, $\mathbf{x}^i_t = g(\hat{o}^i_t) - g(o^i_t)$, where the prediction $\hat{o}^i_t = f(o^i_{t-1}, m^{-i}_{t-2}, a^i_{t-1})$ comes from a forward dynamics module trained by an $\ell^2$ prediction loss. An autoencoder trained only on reconstruction loss compresses $\mathbf{x}^i_t$ into a fixed-length message, and this message is concatenated with a separate binary message trained through the same actor-critic objective as the actions. In the two-agent Multi-Robot Warehouse environment, the paper reports that this combination delivers more requested shelves on the training distribution and after both designed distribution shifts than either message type alone or no communication, and it attributes the gain to the complementarity of surprise-oriented and reward-oriented information.

Load-bearing premise

The load-bearing premise is that the difference between what a forward model predicts and what an agent actually observes, after compression by a reconstruction-trained autoencoder, still conveys information that helps other agents adapt to a distribution shift; the paper does not separately test whether the encoded content matters beyond supplying an extra communication channel.

Editorial extensions

If this is right

Decentralized agents can approach the robustness of centralized training in these warehouse shifts without sharing raw observations.
Reward-driven and surprise-driven messages are complementary: each alone performs about as well as silence, while the pair does better.
A short autoencoded surprise vector of fixed length is enough to carry the adaptation signal, so the scheme does not need to transmit full observations.
The same few-shot protocol can be applied to both beneficial shifts (extra goal tiles) and harmful shifts (shelves moved toward walls), giving a reusable evaluation template.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The method's signal depends on the forward model's prediction error remaining meaningful during adaptation; if the forward model is updated too aggressively while fine-tuning, the discrepancy shrinks and the message may stop flagging novelty. A direct test is to freeze the UEM weights during the 10-episode fine-tuning window and compare with updating them.
Because the surprise vector is computed from each agent's own $3\times3$ local observation, shifts that change only the reward function or another agent's hidden strategy would not be detected by this channel. Extending UES to predict rewards or joint states might cover those cases.
The paper assumes messages are broadcast to all agents; in larger teams the concatenated message length grows with team size. An attentional selection mechanism over received messages is the natural scaling fix, a direction the paper names as a limitation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take on the UES+R paper. The idea is simple and worth a look: agents run a forward dynamics model and communicate the discrepancy between predicted and actual observations, compressed by an autoencoder, alongside a reward-trained message. That is distinct from Abu et al.'s reward-based confusion, and the combination is nicely motivated. The paper writes things up clearly, and the learning curves and table are internally consistent: UES+R is better than either component alone and matches or beats the centralized MAPPO baseline in most settings. Credit where due: the method is easy to understand, the ablations separate reward-only and UES-only messages, and the authors admit the broadcast assumption and the lack of message interpretability.

The soft spots are method-level, not fatal. The biggest one is that the experiments do not isolate the 'unexpectedness' content. UES+R differs from +M(R) in three ways at once: an extra 5-bit channel carries UES output, the actor-critic learning rate and entropy coefficient are different (0.0005/0.05 vs 0.0005/0.01), and the message split changes capacity. So even if the numbers hold, the contribution of the surprise signal itself is confounded. A content-matched control—same number of bits carrying raw local observations, a random projection, or a fixed random signal—would settle that. The paper does not do this. The reader's conditional verdict is reasonable. There is also no code release, only five seeds, no significance tests, and one environment. These are real problems, but not disqualifying; they make the result suggestive rather than conclusive.

The claim itself is not circular: UES is defined by prediction error, not fitted to the performance metric, and the forward model is trained on the same distribution, so it is a measure of novelty relative to training experience. That is a sensible definition of unexpectedness. The math in Eq. 1–3 is straightforward, and the derivation is honest. The paper's own limitation paragraph about broadcasting is a fair acknowledgment, though it does not fix the confound.

Overall, this is a within-subfield paper that a serious referee should see. It is not a breakthrough, and the central attribution needs a cleaner experiment, but the idea is coherent and the direction is useful. If I were editing, I would send it to peer review with a request for the content-matched ablation and hyperparameter egalization before publication.

Referee Report

4 major / 4 minor

Summary. This paper proposes Unexpectedness Encoding Scheme with Reward (UES+R), a decentralized MARL communication method for out-of-distribution adaptation. Each agent maintains a forward dynamics model that predicts the next observation; the prediction error is compressed by an autoencoder trained only on reconstruction loss, and the resulting continuous vector is concatenated with a reward-trained discrete message. The combined message is fed into the agents' policies. Experiments in a two-agent Multi-Robot Warehouse environment compare IA2C, IA2C with reward-only messages, IA2C with UES-only messages, IA2C with UES+R, and centralized MAPPO, in the training distribution and after Goal-Shift and Shelf-Shift. The reported results show UES+R outperforms the decentralized baselines in all settings and is competitive with MAPPO in most settings. The paper interprets these gains as evidence that communicating observation-prediction discrepancy helps agents adapt to distribution shifts.

Significance. The proposed mechanism is original and, if substantiated, practically useful: it uses a reward-free surprise signal to share information about distribution shift in a decentralized setting, and the architecture is simple and additive to any MARL communication baseline. The paper also provides a clear statement of the broadcast limitation and points to compression as future work. However, the empirical support is currently too narrow to establish the central claim. The experiments are restricted to a single two-agent environment, use only five seeds, report no significance tests, and—most importantly—do not isolate the UES message content from channel capacity, hyperparameter differences, or the extra learned forward model. The contribution is credible but not yet convincingly demonstrated.

major comments (4)

[Appendix A.2] The ablations do not use matched hyperparameters. IA2C+M(UES) is trained with learning rate 0.001, while IA2C+M(R) and IA2C+M(UES+R) use 0.0005; IA2C+M(UES+R) uses an entropy coefficient of 0.05, whereas all other methods use 0.01. Because the proposed method differs from the baselines on these dimensions, the improvements in Table 1 cannot currently be attributed to the UES message content. Please rerun all methods with identical learning rates and entropy coefficients, or provide a sensitivity analysis showing the reported ranking is invariant to these choices.
[Section 3.3, Table 1] No control isolates the content of the UES message. IA2C+M(UES+R) sends a 5-bit UES component plus a 5-bit reward component, while IA2C+M(UES) and IA2C+M(R) each send 10 bits of a single type. The observed gains could therefore come from the extra observation-dependent channel, from the particular split of capacity, or from increased exploration (via the higher entropy coefficient), rather than from 'unexpectedness' as defined in Eq. (1). Add content-matched controls—for example, a 5-bit random projection of the local observation, or a fixed random signal, concatenated with the same 5-bit reward message—to test the specific claim that the prediction-error content is the driver.
[Section 3.2, Table 1] The statistical support is weak. The study uses only one environment (two-agent RWARE), five seeds, and no significance tests. Some comparisons are also close: in Shelf-Shift, IA2C+M(UES+R) (1.45±0.07) and IA2C+M(R) (1.33±0.12) overlap at one standard deviation. Please report confidence intervals or significance tests and either increase the number of seeds or add an additional environment/task before making general claims about robust OOD adaptation.
[Section 3.2] It is not specified whether the forward dynamics model f and the autoencoder are updated during the 10-batch fine-tuning phase. If they are updated, the 'unexpectedness' signal is no longer a fixed measure of distribution shift; if they are frozen, the stale predictor may be responsible for the observed adaptation. Please state this explicitly and, ideally, compare both variants.

minor comments (4)

[Eq. (1) vs Appendix A.2] Equation (1) defines x_t = f(o_{t-1}, m_{t-2}, a_{t-1}) - o_t, but Appendix A.2 defines x_t = g(hat o_t) - g(o_t) using a random linear projection g. Please unify these definitions.
[Section 2.3] The sentence 'we omitted the the superscript' contains a typo, and Eq. (4) does not show the entropy regularization mentioned in the text; please add the entropy term or clarify the notation.
[Section 3.4] The claim that 'merely using reward or UES individually led to performances on par with or even worse than IA2C' is too strong given that the error bars overlap for some of the comparisons in Table 1; please soften the wording or add statistical evidence.
[Section 4] The limitations paragraph mentions the broadcast assumption but not the absence of a content-matched ablation for the UES message; this omission should be acknowledged.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are empirical and the UES message is defined by prediction error, not by the outcome it is claimed to improve.

full rationale

The paper's derivation chain does not contain a load-bearing step that reduces to its own inputs. The unexpectedness message is defined in Eq. 1 as m_UES,t = Enc(f(o_{t-1}, m_{t-2}, a_{t-1}) - o_t), with f trained by the prediction loss of Eq. 2 and the autoencoder trained only by the reconstruction loss of Eq. 3; none of these quantities is fitted to delivered shelves or to the Table 1 performance values, so the reported improvements are not predictions forced by construction. The method is benchmarked on an external environment (RWARE, Papoudakis et al. 2020), the baselines are contemporaneous methods rather than the authors' prior work, and there is no self-citation chain or imported uniqueness theorem used to declare the design forced. The appended limitation that 'interpreting the message contents in relation to actual environmental shift is an important topic for future work' acknowledges that the information content of UES is not established by definition. The absence of a content-matched communication control and the unequal hyperparameters across ablations are experimental confounds relevant to correctness, but they are not circularity under the definitions used here.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced; the Unexpectedness Encoding Module is an architectural component, not an invented natural entity. The main unstated cost is carried by the domain assumptions about prediction error and autoencoder reconstruction, which are plausible but not independently validated.

free parameters (5)

Message length and split = 10 total; 5 reward-driven + 5 UES for UES+R, 10 for others
Chosen to match across baselines; the 5/5 split is a design decision not justified by analysis.
Entropy coefficient = 0.05 for UES+R, 0.01 for others
Different exploration regularization across methods may confound the comparison; no tuning protocol is reported.
Learning rate = 0.0005 for most, 0.001 for M(UES)
Per-method learning rates differ; the choice is not justified or controlled.
Autoencoder latent dimension = 10
Set equal to message length by hand; no sensitivity analysis.
Random linear projection output dimension = 64
Used to embed observations before computing prediction error; the size is arbitrary.

assumptions (4)

domain assumption Forward prediction error is a useful signal for out-of-distribution adaptation
Central design premise in Section 2.2; not proven theoretically or through dedicated ablations.
domain assumption Autoencoder reconstruction preserves task-relevant unexpectedness
The encoding is trained with L2 reconstruction only, not task reward; no evidence that the compressed message retains useful information.
domain assumption Broadcast messages are available to all agents
Stated in Appendix A.1 and acknowledged as a limitation in the Conclusion.
domain assumption Fine-tuning on 10 batches measures few-shot adaptation
Section 3.2 describes the protocol but does not specify batch granularity, number of updates, or whether the forward model is adapted.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning." pith.science (2026). https://pith.science/paper/ZXSERH2O

@misc{pith2026250101140,
  author       = {Pith},
  title        = {Pith review of: Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZXSERH2O}},
  note         = {Machine review of arXiv:2501.01140}
}

read the original abstract

Applying multi-agent reinforcement learning methods to realistic settings is challenging as it may require the agents to quickly adapt to unexpected situations that are rarely or never encountered in training. Recent methods for generalization to such out-of-distribution settings are limited to more specific, restricted instances of distribution shifts. To tackle adaptation to distribution shifts, we propose Unexpected Encoding Scheme, a novel decentralized multi-agent reinforcement learning algorithm where agents communicate "unexpectedness," the aspects of the environment that are surprising. In addition to a message yielded by the original reward-driven communication, each agent predicts the next observation based on previous experience, measures the discrepancy between the prediction and the actually encountered observation, and encodes this discrepancy as a message. Experiments on multi-robot warehouse environment support that our proposed method adapts robustly to dynamically changing training environments as well as out-of-distribution environment.

Figures

Figures reproduced from arXiv: 2501.01140 by the authors.

**Figure 2.** Overview of the Unexpectedness Encoding Scheme with Reward (UES+R). [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Learning curves on training distribution. Mean [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 11 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Abu, O.; Gerstgrasser, M.; Rosenschein, J.; and Keren, S. 2021. Promoting Resilience in Multi-Agent Reinforcement Learning via Confusion-Based Communication. arXiv preprint arXiv:2111.06614

work page Pith review arXiv 2021
[4]

Berner, C.; Brockman, G.; Chan, B.; Cheung, V.; D e biak, P.; Dennison, C.; Farhi, D.; Fischer, Q.; Hashme, S.; Hesse, C.; et al. 2019. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680

arXiv 2019
[5]

Brandizzi, N. 2023. Towards More Human-like AI Communication: A Review of Emergent Communication Research. arXiv preprint arXiv:2308.02541

work page Pith review arXiv 2023
[6]

Cho, K.; Van Merri \"e nboer, B.; Bahdanau, D.; and Bengio, Y. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259

arXiv 2014
[7]

Hernandez-Leal, P.; Kaisers, M.; Baarslag, T.; and de Cote, E. M. 2017. A survey of learning in multiagent environments: Dealing with non-stationarity. arXiv preprint arXiv:1707.09183

arXiv 2017
[8]

Z.; and De Freitas, N

Jaques, N.; Lazaridou, A.; Hughes, E.; Gulcehre, C.; Ortega, P.; Strouse, D.; Leibo, J. Z.; and De Freitas, N. 2019. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In International Conference on Machine Learning, 3040--3049. PMLR

work page 2019

Show all 26 references

[9]

Jiang, J.; and Lu, Z. 2018. Learning attentional communication for multi-agent cooperation. Advances in neural information processing systems, 31

2018
[10]

P.; and Ba, J

Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

2014 arXiv
[11]

Littman, M. L. 1994. Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings 1994, 157--163. Elsevier

1994
[12]

Liu, B.; Liu, Q.; Stone, P.; Garg, A.; Zhu, Y.; and Anandkumar, A. 2021. Coach-player multi-agent reinforcement learning for dynamic team composition. In International Conference on Machine Learning, 6860--6870. PMLR

2021
[13]

P.; Mirza, M.; Graves, A.; Lillicrap, T.; Harley, T.; Silver, D.; and Kavukcuoglu, K

Mnih, V.; Badia, A. P.; Mirza, M.; Graves, A.; Lillicrap, T.; Harley, T.; Silver, D.; and Kavukcuoglu, K. 2016. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, 1928--1937

2016
[14]

A.; Spaan, M

Oliehoek, F. A.; Spaan, M. T.; and Vlassis, N. 2008. Optimal and approximate Q-value functions for decentralized POMDPs. Journal of Artificial Intelligence Research, 32: 289--353

2008
[15]

Papoudakis, G.; Christianos, F.; Rahman, A.; and Albrecht, S. V. 2019. Dealing with non-stationarity in multi-agent deep reinforcement learning. arXiv preprint arXiv:1906.04737

2019 arXiv
[16]

Papoudakis, G.; Christianos, F.; Sch \"a fer, L.; and Albrecht, S. V. 2020. Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. arXiv preprint arXiv:2006.07869

2020 arXiv
[17]

Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T.; et al. 2020. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839): 604--609

2020
[18]

Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

2017 arXiv
[19]

Shao, J.; Lou, Z.; Zhang, H.; Jiang, Y.; He, S.; and Ji, X. 2022. Self-Organized Group for Cooperative Multi-agent Reinforcement Learning. Advances in Neural Information Processing Systems, 35: 5711--5723

2022
[20]

Shoham, Y.; and Leyton-Brown, K. 2008. Multiagent systems: Algorithmic, game-theoretic, and logical foundations. Cambridge University Press

2008
[21]

M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D

Vinyals, O.; Babuschkin, I.; Czarnecki, W. M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D. H.; Powell, R.; Ewalds, T.; Georgiev, P.; et al. 2019. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782): 350--354

2019
[22]

Yu, C.; Velu, A.; Vinitsky, E.; Wang, Y.; Bayen, A.; and Wu, Y. 2021. The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games. arXiv preprint arXiv:2103.01955

2021 arXiv
[23]

Zhang, K.; Yang, Z.; and Ba s ar, T. 2021. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, 321--384

2021
[24]

Zhang, Y.; Yang, Q.; An, D.; Li, D.; and Wu, Z. 2022. Multistep multiagent reinforcement learning for optimal energy schedule strategy of charging stations in smart grid. IEEE Transactions on Cybernetics

2022
[25]

Zhou, W.; Liu, Z.; Li, J.; Xu, X.; and Shen, L. 2021. Multi-target tracking for unmanned aerial vehicle swarms using deep reinforcement learning. Neurocomputing, 466: 285--297

2021
[26]

Zhu, C.; Dastani, M.; and Wang, S. 2022. A survey of multi-agent reinforcement learning with communication. arXiv preprint arXiv:2203.08975

2022 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Abu, O.; Gerstgrasser, M.; Rosenschein, J.; and Keren, S. 2021. Promoting Resilience in Multi-Agent Reinforcement Learning via Confusion-Based Communication. arXiv preprint arXiv:2111.06614

work page Pith review arXiv 2021

[4] [4]

Berner, C.; Brockman, G.; Chan, B.; Cheung, V.; D e biak, P.; Dennison, C.; Farhi, D.; Fischer, Q.; Hashme, S.; Hesse, C.; et al. 2019. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680

arXiv 2019

[5] [5]

Brandizzi, N. 2023. Towards More Human-like AI Communication: A Review of Emergent Communication Research. arXiv preprint arXiv:2308.02541

work page Pith review arXiv 2023

[6] [6]

Cho, K.; Van Merri \"e nboer, B.; Bahdanau, D.; and Bengio, Y. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259

arXiv 2014

[7] [7]

Hernandez-Leal, P.; Kaisers, M.; Baarslag, T.; and de Cote, E. M. 2017. A survey of learning in multiagent environments: Dealing with non-stationarity. arXiv preprint arXiv:1707.09183

arXiv 2017

[8] [8]

Z.; and De Freitas, N

Jaques, N.; Lazaridou, A.; Hughes, E.; Gulcehre, C.; Ortega, P.; Strouse, D.; Leibo, J. Z.; and De Freitas, N. 2019. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In International Conference on Machine Learning, 3040--3049. PMLR

work page 2019