REVIEW 4 major objections 4 minor 82 references
Training Small LLMs as Spatial Multi-Agent Policies
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Small frozen LLMs become competent cooperative agents when they choose from safe symbolic options instead of raw actions.
desk verdict Frozen small LLMs can learn spatial cooperative policies via a state-filtered option menu and per-agent LoRA; the auto-guard synthesis is the main uncertainty. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key machinery is the option menu interface: at each decision point the LLM sees a lettered menu of typed, state-feasible, short-horizon symbolic options (e.g., move_to(x,y), pick_up(onion)), each executed by a symbolic planner, with feasibility decided by code rather than by the model. Because options start and finish asynchronously across agents, the team process is formalized as a macro-action Dec-POMDP—a Dec-POMDP whose actions are temporally extended options. The menu is filtered by guards synthesized mechanically from burn-in executions, adopted only when they explain repeated failures and hide zero logged successes. On top of this interface, per-agent LoRA adapters on a shared froz
What would settle it
On a new game, corrupt the effect predicates in a known way (absolute-form or shared-state attribution, the two defects the paper fixed) and run guard synthesis; the zero-success certificate should veto correct guards and the pipeline should collapse. If a corrupted contract still yields a working menu and competent play, the load-bearing role of the contract-repair loop is refuted.
Extended reading notes
Core claim
Small frozen LLMs cannot act directly in spatial games: they hallucinate spatial preconditions and collapse onto a single 'safe' action, earning zero reward, and on-policy RL only sharpens the collapse. The discovery is a working recipe: let each agent choose from a state-filtered menu of symbolic options executed by a planner; mine the menu's feasibility guards mechanically from uniform-random rollouts; and train a private LoRA adapter per agent with PA-MAGRPO, a per-agent group-relative policy optimization. This lifts frozen bases to competent play on three games across four backbones. It also reveals a second finding: reward and cooperation decouple—Cleanup produces cleaner/eater speciali
Load-bearing premise
The draft contract (effect predicates plus read-set features) must faithfully capture what each option promises; if it mislabels executions—or the fixed option library simply lacks an option needed for success—the mechanically synthesized filters and the trained policies cannot recover.
Editorial extensions
If this is right
- Frozen 2-4B LLMs can serve as viable spatial cooperative policies if the decision space is the right one: option menus plus per-agent adapters produce non-trivial behavior on three games where raw-action prompting yields exactly zero.
- The menu filter is the load-bearing piece for harder games: with it, MAGRPO training on native actions or on compound-JSON without the filter fails (zero reward on Overcooked, high variance on Cleanup).
- Role specialization requires per-agent parameter capacity: a single shared LoRA adapter holds only symmetric policies and recovers roughly a third of the full method's Overcooked return.
- Reward-based evaluation alone is insufficient: behavioral audits are needed to tell whether high reward comes from coordination, one-sided work, or hidden restraint.
Reading between the lines
- The guard-synthesis criterion (adopt only if it hides zero successes and explains repeated failures) is a general recipe for building action-feasibility filters from execution logs; the paper demonstrates it for three games, but it should transfer to any environment with a simulator and typed effects—untested by the authors.
- If the frontier-model drafting step could be replaced by cheaper or deterministic generation, the entire pipeline would become reproducible and free of external model calls; the paper leaves this as an open possibility.
- The 'letter-A prior' effect suggests that menu position randomization should be a default design rule for any LLM acting over indexed menus, since models otherwise exploit position rather than content.
- The decoupling evidence implies that the community should standardize coordination metrics (hand-offs, role entropy, patch survival) alongside reward in LLM-MARL; the paper shows what such audits can reveal but does not propose a single metric.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a training recipe for small (2–4B) frozen LLMs acting as decentralized cooperative spatial policies. Instead of primitive actions, each agent selects from a state-filtered menu of symbolic options (move_to, pick_onion, clean, etc.), with a symbolic planner executing the chosen option. The option library is drafted once by a frontier coding model; the feasibility guards that filter the menu are then synthesized mechanically from random-policy burn-in rollouts under a purity certificate (a guard is adopted only if it covers repeated failures and zero logged successes). Each agent carries a private LoRA adapter on a shared frozen base, trained by a per-agent variant of MAGRPO (PA-MAGRPO). Experiments on Melting Pot Cleanup, Overcooked-AI (Asymmetric Advantages and Forced Coordination), and Commons Harvest report that the frozen bases earn zero reward while the trained policies achieve positive reward across four small backbones. Behavioral audits show high reward without coordination in Overcooked AA, emergent cleaner/eater roles in Cleanup, and patch conservation with nearly flat reward in Commons Harvest, leading to the central claim that team reward alone does not certify cooperation.
Significance. If the results hold, the paper makes a useful practical contribution: a concrete recipe for turning small frozen LLMs into competent cooperative spatial policies, with per-agent specialization and without hand-authored guards. The behavioral decoupling result is also valuable and is supported by independent, non-reward metrics (patch survival, per-agent macro histograms, solo-control tests). The paper is unusually explicit about its statistics, uses external environment rewards rather than fitted quantities as outcomes, and acknowledges several limitations. Its main risk is that the fully automatic guard-synthesis pipeline—a stated contribution—is not end-to-end validated with the same rigor as the hand-selected-guard ablation, and some central controls are underreported.
major comments (4)
- [§5.1, §6.1, Table 2] The 'no hand-authored guard' claim is not end-to-end validated. The paper states that the ablation suite, including the 'Full method (ours)' row and apparently the behavioral audits of §6.4, was run on 'a slightly earlier interface version whose guards were hand-selected rather than synthesized' (§6.1). Table 1 does run the synthesized pipeline, but reports only aggregate mean±std over five seeds, without guard files, burn-in logs, or certificate data, so the correctness of the synthesized menus cannot be independently checked. This is load-bearing: §5.1 itself says effect predicates are 'load-bearing' and documents Defects 1 and 2 in which vacuous successes vetoed correct guards. An undisclosed contract defect would corrupt the menu and directly undermine the claimed lift and the decoupling findings. Please provide (i) an experiment comparing hand-selected vs synthesized guards on the s
- [§6.1, Table 1] The 'robust across four backbones' claim is not supported by the reported statistics. With n=5 independent seeds, standard deviations are the same order as the means in several cells: Cleanup, Qwen3.5-2B: 105.69±50.71; gemma-4-E4B-it: 60.25±39.40; gemma-4-E2B-it: 68.62±41.59. A mean of 60.25 with SD 39.40 could be driven by one or two successful seeds, which would not support 'consistent non-trivial behavior.' The bootstrap CIs and Mann–Whitney tests described in §6.1 apply to the 20-episode fixed-backbone evaluation, not to the seed-level cross-backbone claim. Please report per-seed values, per-backbone minima/bootstrap CIs, or a seed-level statistical model.
- [§6.4] The Forced Coordination (FC) control is central to the conclusion that 'coordination appears when—and only when—the layout requires it,' but it is reported in one sentence: the trained checkpoint reaches 7.57±1.16 deliveries and the solo test flips. The text does not state whether FC uses the same full method (per-agent LoRA, PA-MAGRPO, synthesized guards), the number of training seeds, hyperparameters, or evaluation protocol. Since the decoupling argument depends on the contrast between AA (solo-solvable) and FC (not solo-solvable), FC must receive the same statistical and procedural detail as the other rows. Please add an FC row or table with seeds, CIs, and the full solo-test protocol.
- [§5.1, outer contract-repair loop] The defect-detection step is underspecified. The text says 'the mechanical layers detect a defect (it surfaces as an impossible pattern in the burn-in log or the certificate),' but no algorithm or criterion for recognizing an 'impossible pattern' is given. The three defect families are described with bug/symptom/fix, but it is not stated whether they were found by the mechanical layer or by the authors during development. If detection required human or frontier-model inspection, the claim that Stages 2–4 have 'no model and no human in the loop' is too strong. Please define the detection criterion or qualify the claim.
minor comments (4)
- [§6.2, Table 2] The sentence 'Every ours cell is statistically separated from its frozen and zero-shot cells' should clarify that this is evaluation-side only, because the ablation table uses a single training seed; the paper later acknowledges this, but the wording in §6.2 overstates it.
- [§5.1, code snippet] In the raw_expand snippet, the variable 'held' appears without being defined in the displayed code; this makes the executor snippet harder to follow. Minor clarity issue.
- [Figures 1–3] The figures are referenced in the text but the captions are minimal. In particular, Figure 1 should state what fraction of decisions is summarized (all training seeds or a single checkpoint) and how the 'wait' category is defined.
- [§7, Limitations] The paper does not state whether code or artifacts will be released. Given that the automatic guard-synthesis claim depends on inspectable guard files and burn-in logs, a release statement would materially help reproducibility.
Circularity Check
No significant circularity: reported outcomes are external environment rewards and independent behavioral metrics, and the guard-synthesis loop, while oracle-dependent, is not a fitted-parameter-as-prediction or self-citation chain.
full rationale
The paper's central claims are evaluated against external quantities that the method does not fit: apples eaten, soups delivered, patch survival, and per-agent macro-choice histograms come from environment rollouts and trajectory audits, not from any parameter the method optimizes or defines. The guard-synthesis procedure ('a guard is adopted only if it explains repeated execution failures while hiding no logged success') is a consistency filter over burn-in logs whose 'success' label is supplied by frontier-drafted effect predicates; this makes the certificate conditional on the correctness of those predicates, but it is not circular in the sense that the predicted outcome is defined by the fitted input. The paper explicitly treats the contract as potentially defective (Section 5.1, Defects 1-3) and documents how wrong predicates corrupted the certificate, which is an acknowledged robustness limitation rather than a hidden reduction. There is no fitted-input-called-prediction pattern: LoRA hyperparameters and MAGRPO settings are fixed for stability, not tuned to the reported metric. The only apparent self-citation overlap is the Melting Pot 2.0 environment reference (which includes an author named Mao), and it is used as a benchmark substrate, not as load-bearing evidence for the method's claims. Behavioral decoupling findings are computed from independent trajectory metrics and are not renamed versions of the reward signal. Overall, the derivation chain is self-contained against external benchmarks, and no step reduces by construction to its own inputs.
Assumptions & free parameters
free parameters (4)
- MIN_SUPPORT (guard adoption threshold) =
5 logged failures
- Training hyperparameters (per game) =
Cleanup/Harvest: LoRA rank 32, lr 5e-5, K=4 rollouts, temperature 0.7; Overcooked: rank 16, lr 2e-5, K=6, temperature 0.
- Burn-in rollout budget =
2,000 solo + 2,000 two-agent episodes per game
- Guard search shape =
1- and 2-literal conjunctions over the declared read set
assumptions (6)
- standard math Acting over a fixed option set turns each agent's decision problem into a semi-MDP, and asynchronous option termination across agents makes the team-level process a macro-action Dec-POMDP.
- domain assumption The frontier-drafted effect predicates, written in delta form and bound to private transitions of the acting agent, correctly identify whether an option execution did what its name promises.
- domain assumption Uniform-random burn-in over the guardless menu (2,000 solo + 2,000 two-agent episodes) provides enough coverage of valid and invalid executions for guard synthesis.
- domain assumption A candidate guard that covers at least five failures and zero logged successes is a valid feasibility filter and not a strategy artifact.
- domain assumption The base models (Qwen3-4B-Instruct-2507, Qwen3.5-2B, gemma-4-E4B-it, gemma-4-E2B-it) and environment simulators (Melting Pot 2.0, Overcooked-AI) behave as stated, including the documented failure modes of Section 4.
- domain assumption The Overcooked Forced Coordination layout is not solo-solvable, so zero solo-delivery counts certify that team deliveries require a partner.
Cite this review
Pith. "Pith review of Training Small LLMs as Spatial Multi-Agent Policies." pith.science (2026). https://pith.science/paper/SGSP55PQ
@misc{pith2026260801425,
author = {Pith},
title = {Pith review of: Training Small LLMs as Spatial Multi-Agent Policies},
year = {2026},
howpublished = {\url{https://pith.science/paper/SGSP55PQ}},
note = {Machine review of arXiv:2608.01425}
}
read the original abstract
Training LLM-based multi-agent systems with multi-agent reinforcement learning is rapidly gaining traction, and a parallel line of work argues that such systems should be judged by their behavior, not only their reward. We take up both threads in spatial cooperative games, where small frozen LLMs prompted with low-level actions fail outright, earning zero reward. Guided by the options/semi-MDP framework---and, because option execution is asynchronous across agents, its multi-agent extension in macro-action Dec-POMDPs---we equip each game with a library of symbolic \emph{options}: typed, state-feasible, short-horizon behaviors executed by a symbolic planner. Each library is drafted by a frontier coding model from the game's source code; the feasibility guards that filter each menu are then synthesized mechanically from cheap random-policy burn-in rollouts---a guard is adopted only if it explains repeated execution failures while hiding no logged success---so no guard is authored, selected, or reward-tuned by hand. Each agent's LLM acts as its policy over options, with a private per-agent LoRA adapter trained by a per-agent variant of multi-agent GRPO (PA-MAGRPO); this lifts frozen bases from zero reward to competent play across three games and four small backbones. Behavioral audits then reveal that reward and cooperation decouple: a rising reward curve may simply mean that one agent has learned to run the entire task alone while its partner idles---cooperation emerges only when the task makes it necessary. Reward alone is thus an unreliable readout of cooperation; behavioral evaluation must sit alongside it.
Figures
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2305.16291 , year=
Voyager: An open-ended embodied agent with large language models , author=. arXiv preprint arXiv:2305.16291 , year=
-
[2]
Yuan Zhuang and Yi Shen and Zhili Zhang and Yuxiao Chen and Fei Miao , booktitle=
-
[3]
arXiv preprint arXiv:2601.17152 , year=
Dynamic Role Assignment for Multi-Agent Debate , author=. arXiv preprint arXiv:2601.17152 , year=
-
[4]
OpenReview , year=
RL-GPT: Integrating Reinforcement Learning and Code-as-policy , author=. OpenReview , year=
-
[5]
Advances in Neural Information Processing Systems , volume=
Coevolving with the other you: Fine-tuning llm with sequential cooperative multi-agent reinforcement learning , author=. Advances in Neural Information Processing Systems , volume=
-
[6]
arXiv preprint arXiv:2510.01586 , year=
AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution , author=. arXiv preprint arXiv:2510.01586 , year=
-
[7]
arXiv preprint arXiv:2502.10148 , year=
Cooperative Multi-Agent Planning with Adaptive Skill Synthesis , author=. arXiv preprint arXiv:2502.10148 , year=
-
[8]
Agents of Change: Self-Evolving LLM Agents for Strategic Planning , author=. UC Santa Barbara , year=
Show all 82 references
-
[9]
arXiv preprint arXiv:2503.20783 , year=
Understanding r1-zero-like training: A critical perspective , author=. arXiv preprint arXiv:2503.20783 , year=
-
[10]
Advances in Neural Information Processing Systems , volume=
The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games , author=. Advances in Neural Information Processing Systems , volume=
-
[11]
arXiv preprint arXiv:1707.06347 , year=
Proximal Policy Optimization Algorithms , author=. arXiv preprint arXiv:1707.06347 , year=
-
[13]
International Conference on Machine Learning , pages=
Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot , author=. International Conference on Machine Learning , pages=. 2021 , organization=
2021
-
[15]
NeurIPS Workshop on Foundation Models for Decision Making , year=
Mitigating Generative Agent Social Dilemmas , author=. NeurIPS Workshop on Foundation Models for Decision Making , year=
-
[17]
Proceedings of the AAAI Conference on Artificial Intelligence , year=
LLM Collaboration with Multi-Agent Reinforcement Learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year=
-
[18]
Advances in Neural Information Processing Systems (NeurIPS) , year=
On the Planning Abilities of Large Language Models -- A Critical Investigation , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[19]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Who Is Helping Whom? Analyzing Inter-Dependencies to Evaluate Cooperation in Human-AI Teaming , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[20]
Advances in Neural Information Processing Systems (NeurIPS) , year=
A Multi-Agent Reinforcement Learning Model of Common-Pool Resource Appropriation , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[21]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Inequity Aversion Improves Cooperation in Intertemporal Social Dilemmas , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[22]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[23]
Springer , year=
A Concise Introduction to Decentralized POMDPs , author=. Springer , year=
-
[24]
arXiv preprint arXiv:2106.09685 , year=
LoRA: Low-Rank Adaptation of Large Language Models , author=. arXiv preprint arXiv:2106.09685 , year=
-
[25]
International Conference on Machine Learning , year=
Guiding Pretraining in Reinforcement Learning with Large Language Models , author=. International Conference on Machine Learning , year=
-
[26]
Advances in Neural Information Processing Systems , volume=
CAMEL: Communicative Agents for ``Mind'' Exploration of Large Language Model Society , author=. Advances in Neural Information Processing Systems , volume=
-
[27]
arXiv preprint arXiv:2308.00352 , year=
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework , author=. arXiv preprint arXiv:2308.00352 , year=
-
[28]
ACL , year=
ChatDev: Communicative Agents for Software Development , author=. ACL , year=
-
[29]
Advances in Neural Information Processing Systems , volume=
Direct Preference Optimization: Your Language Model is Secretly a Reward Model , author=. Advances in Neural Information Processing Systems , volume=
-
[30]
and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=
Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=
-
[31]
arXiv preprint arXiv:2305.17144 , year=
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory , author=. arXiv preprint arXiv:2305.17144 , year=
-
[32]
Wu, Yue and Prabhumoye, Shrimai and Min, So Yeon and Bisk, Yonatan and Salakhutdinov, Ruslan and Azaria, Amos and Mitchell, Tom and Li, Yuanzhi , booktitle=
-
[33]
Hu, Sihao and Huang, Tiansheng and Liu, Ling , journal=
-
[34]
International Conference on Learning Representations (ICLR) , year=
Benchmarking the Spectrum of Agent Capabilities , author=. International Conference on Learning Representations (ICLR) , year=
-
[35]
Conference on Robot Learning (CoRL) , year=
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances , author=. Conference on Robot Learning (CoRL) , year=
-
[36]
Conference on Robot Learning (CoRL) , year=
Inner Monologue: Embodied Reasoning through Planning with Language Models , author=. Conference on Robot Learning (CoRL) , year=
-
[37]
IEEE International Conference on Robotics and Automation (ICRA) , year=
Code as Policies: Language Model Programs for Embodied Control , author=. IEEE International Conference on Robotics and Automation (ICRA) , year=
-
[38]
Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle=
-
[39]
arXiv preprint arXiv:2307.09702 , year=
Efficient Guided Generation for Large Language Models , author=. arXiv preprint arXiv:2307.09702 , year=
-
[40]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Training Language Models to Follow Instructions with Human Feedback , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[41]
2025 , note=
Nature , volume=. 2025 , note=
2025
-
[42]
Wang, Zihan and Wang, Kangrui and Wang, Qineng and Zhang, Pingyue and Li, Linjie and Yang, Zhengyuan and Jin, Xing and Yu, Kefan and Nguyen, Minh Nhat and others , journal=
-
[43]
Zhou, Yifei and Zanette, Andrea and Pan, Jiayi and Levine, Sergey and Kumar, Aviral , booktitle=
-
[44]
Ma, Yecheng Jason and Liang, William and Wang, Guanzhi and Huang, De-An and Bastani, Osbert and Jayaraman, Dinesh and Zhu, Yuke and Fan, Linxi and Anandkumar, Anima , booktitle=
-
[45]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[46]
Rashid, Tabish and Samvelyan, Mikayel and de Witt, Christian Schroeder and Farquhar, Gregory and Foerster, Jakob and Whiteson, Shimon , booktitle=
-
[47]
Wang, Tonghan and Dong, Heng and Lesser, Victor and Zhang, Chongjie , booktitle=
-
[48]
Samvelyan, Mikayel and Rashid, Tabish and de Witt, Christian Schroeder and Farquhar, Gregory and Nardelli, Nantas and Rudner, Tim G. J. and Hung, Chia-Man and Torr, Philip H. S. and Foerster, Jakob and Whiteson, Shimon , booktitle=. The
-
[49]
and Chandar, Sarath and Burch, Neil and Lanctot, Marc and Song, H
Bard, Nolan and Foerster, Jakob N. and Chandar, Sarath and Burch, Neil and Lanctot, Marc and Song, H. Francis and Parisotto, Emilio and Dumoulin, Vincent and Moitra, Subhodeep and Hughes, Edward and Dunning, Iain and Mourad, Shibl and Larochelle, Hugo and Bellemare, Marc G. an...
-
[50]
Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , year=
Multi-Agent Reinforcement Learning in Sequential Social Dilemmas , author=. Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , year=
-
[51]
and Griffiths, Thomas L
Carroll, Micah and Shah, Rohin and Ho, Mark K. and Griffiths, Thomas L. and Seshia, Sanjit A. and Abbeel, Pieter and Dragan, Anca , booktitle=. On the Utility of Learning about Humans for Human-
-
[52]
Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST) , year=
Generative Agents: Interactive Simulacra of Human Behavior , author=. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST) , year=
-
[53]
Wu, Qingyun and Bansal, Gagan and Zhang, Jieyu and Wu, Yiran and Zhang, Shaokun and Zhu, Erkang and Li, Beibin and Jiang, Li and Zhang, Xiaoyun and Wang, Chi , journal=
-
[54]
and Precup, Doina and Singh, Satinder , journal=
Sutton, Richard S. and Precup, Doina and Singh, Satinder , journal=. Between
-
[55]
Vezhnevets, Alexander Sasha and Osindero, Simon and Schaul, Tom and Heess, Nicolas and Jaderberg, Max and Silver, David and Kavukcuoglu, Koray , booktitle=
-
[56]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
The Option-Critic Architecture , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[57]
International Conference on Learning Representations (ICLR) , year=
Diversity is All You Need: Learning Skills without a Reward Function , author=. International Conference on Learning Representations (ICLR) , year=
-
[58]
Mosquera, Manuel and Pinzon, Juan Sebastian and Rios, Manuel and Fonseca, Yesid and Giraldo, Luis Felipe and Quijano, Nicanor and Manrique, Ruben , journal=. Can
-
[62]
Findings of the Association for Computational Linguistics: EACL , year=
MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM , author=. Findings of the Association for Computational Linguistics: EACL , year=
-
[63]
Planning with Macro-Actions in Decentralized
Amato, Christopher and Konidaris, George and Kaelbling, Leslie Pack , booktitle =. Planning with Macro-Actions in Decentralized
-
[64]
Proceedings of the 3rd Conference on Robot Learning (CoRL) , series =
Macro-Action-Based Deep Multi-Agent Reinforcement Learning , author =. Proceedings of the 3rd Conference on Robot Learning (CoRL) , series =
-
[65]
P.; Vezhnevets, A
Agapiou, J. P.; Vezhnevets, A. S.; Du \'e \ n ez-Guzm \'a n, E. A.; Matyas, J.; Mao, Y.; Sunehag, P.; K \"o ster, R.; et al. 2022. Melting Pot 2.0. arXiv preprint arXiv:2211.13746
2022 arXiv
-
[66]
Ahn, M.; Brohan, A.; Brown, N.; et al. 2022. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. In Conference on Robot Learning (CoRL)
2022
-
[67]
Amato, C.; Konidaris, G.; and Kaelbling, L. P. 2014. Planning with Macro-Actions in Decentralized POMDPs . In Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
2014
-
[68]
Biswas, U.; Palod, V.; Bhambri, S.; and Kambhampati, S. 2026. Who Is Helping Whom? Analyzing Inter-Dependencies to Evaluate Cooperation in Human-AI Teaming. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 17347--17356
2026
-
[69]
K.; Griffiths, T
Carroll, M.; Shah, R.; Ho, M. K.; Griffiths, T. L.; Seshia, S. A.; Abbeel, P.; and Dragan, A. 2019. On the Utility of Learning about Humans for Human- AI Coordination. In Advances in Neural Information Processing Systems (NeurIPS)
2019
-
[70]
Gallego, V. 2026. Cooperation and Exploitation in LLM Policy Synthesis for Sequential Social Dilemmas. arXiv preprint arXiv:2603.19453
2026 arXiv
-
[71]
J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W
Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. LoRA : Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations (ICLR)
2022
-
[72]
Hua, Y.; Chen, H.; Wang, S.; Li, W.; Wang, X.; and Luo, J. 2025. Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents. In Advances in Neural Information Processing Systems (NeurIPS). ArXiv:2506.07388
2025 arXiv
-
[73]
Huang, W.; Xia, F.; Xiao, T.; et al. 2022. Inner Monologue: Embodied Reasoning through Planning with Language Models. In Conference on Robot Learning (CoRL)
2022
-
[74]
Z.; Phillips, M.; Tuyls, K.; Du \'e \ n ez-Guzm \'a n, E
Hughes, E.; Leibo, J. Z.; Phillips, M.; Tuyls, K.; Du \'e \ n ez-Guzm \'a n, E. A.; Casta \ n eda, A. G.; Dunning, I.; Zhu, T.; McKee, K. R.; Koster, R.; et al. 2018. Inequity Aversion Improves Cooperation in Intertemporal Social Dilemmas. In Advances in Neural Information Pro...
2018
-
[75]
Lee, W.; Cho, J.; and Choi, J. 2026. MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM. In Findings of the Association for Computational Linguistics: EACL. ArXiv:2509.17489
2026
-
[76]
Z.; Zambaldi, V.; Lanctot, M.; Marecki, J.; and Graepel, T
Leibo, J. Z.; Zambaldi, V.; Lanctot, M.; Marecki, J.; and Graepel, T. 2017. Multi-Agent Reinforcement Learning in Sequential Social Dilemmas. In Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
2017
-
[77]
Liang, J.; Huang, W.; Xia, F.; Xu, P.; Hausman, K.; Ichter, B.; Florence, P.; and Zeng, A. 2023. Code as Policies: Language Model Programs for Embodied Control. In IEEE International Conference on Robotics and Automation (ICRA)
2023
-
[78]
Liu, S.; Liang, Z.; Lyu, X.; and Amato, C. 2026. LLM Collaboration with Multi-Agent Reinforcement Learning. In Proceedings of the AAAI Conference on Artificial Intelligence. ArXiv:2508.04652
2026
-
[79]
Ma, H.; Hu, T.; Pu, Z.; Liu, B.; Ai, X.; Liang, Y.; and Chen, M. 2024. Coevolving with the other you: Fine-tuning llm with sequential cooperative multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 37: 15497--15525
2024
-
[80]
S.; Rios, M.; Fonseca, Y.; Giraldo, L
Mosquera, M.; Pinzon, J. S.; Rios, M.; Fonseca, Y.; Giraldo, L. F.; Quijano, N.; and Manrique, R. 2025. Can LLM-Augmented Autonomous Agents Cooperate? An Evaluation of Their Cooperative Capabilities through Melting Pot . IEEE Transactions on Artificial Intelligence
2025
-
[81]
Z.; Zambaldi, V.; Beattie, C.; Tuyls, K.; and Graepel, T
P \'e rolat, J.; Leibo, J. Z.; Zambaldi, V.; Beattie, C.; Tuyls, K.; and Graepel, T. 2017. A Multi-Agent Reinforcement Learning Model of Common-Pool Resource Appropriation. In Advances in Neural Information Processing Systems (NeurIPS)
2017
-
[82]
Piche, D.; Muqeeth, M.; Aghajohari, M.; Duque, J.; Noukhovitch, M.; and Courville, A. 2025. Learning Robust Social Strategies with Large Language Models. arXiv preprint arXiv:2511.19405
2025
-
[83]
Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y.; Wu, Y.; et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300
2024 arXiv
-
[84]
S.; Precup, D.; and Singh, S
Sutton, R. S.; Precup, D.; and Singh, S. 1999. Between MDPs and Semi- MDPs : A Framework for Temporal Abstraction in Reinforcement Learning. Artificial Intelligence, 112(1-2): 181--211
1999
-
[85]
Valmeekam, K.; Marquez, M.; Sreedharan, S.; and Kambhampati, S. 2023. On the Planning Abilities of Large Language Models -- A Critical Investigation. In Advances in Neural Information Processing Systems (NeurIPS). ArXiv:2305.15771
2023 arXiv
-
[86]
Xiao, Y.; Hoffman, J.; and Amato, C. 2020. Macro-Action-Based Deep Multi-Agent Reinforcement Learning. In Proceedings of the 3rd Conference on Robot Learning (CoRL), volume 100 of Proceedings of Machine Learning Research, 1146--1161
2020
-
[87]
Zhang, B.; Kapoor, A.; and Sun, M. 2025. Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learning. arXiv preprint arXiv:2502.05573
2025 arXiv
-
[88]
Zhuang, Y.; Shen, Y.; Zhang, Z.; Chen, Y.; and Miao, F. 2025. YOLO - MARL : You Only LLM Once for Multi-agent Reinforcement Learning. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
2025
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.