REVIEW 5 cited by
On the Utility of Learning about Humans for Human-AI Coordination
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
While we would like agents that can coordinate with humans, current algorithms such as self-play and population-based training create agents that can coordinate with themselves. Agents that assume their partner to be optimal or similar to them can converge to coordination protocols that fail to understand and be understood by humans. To demonstrate this, we introduce a simple environment that requires challenging coordination, based on the popular game Overcooked, and learn a simple model that mimics human play. We evaluate the performance of agents trained via self-play and population-based training. These agents perform very well when paired with themselves, but when paired with our human model, they are significantly worse than agents designed to play with the human model. An experiment with a planning algorithm yields the same conclusion, though only when the human-aware planner is given the exact human model that it is playing with. A user study with real humans shows this pattern as well, though less strongly. Qualitatively, we find that the gains come from having the agent adapt to the human's gameplay. Given this result, we suggest several approaches for designing agents that learn about humans in order to better coordinate with them. Code is available at https://github.com/HumanCompatibleAI/overcooked_ai.
Forward citations
Cited by 5 Pith papers
-
ColorGrid: A Multi-Agent Non-Stationary Environment for Goal Inference and Assistance
A new benchmark shows that IPPO, a standard MARL algorithm, cannot reliably infer a partner's changing goal from its actions alone.
-
Learning to Cooperate with Humans using Generative Agents
A generative model of partner strategies, sampled during training, improves human-AI cooperation in Overcooked over population-based and behavior-cloning baselines.
-
InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma
InvestESG is a MARL benchmark showing that ESG-conscious investors, not the disclosure mandate itself, drive corporate mitigation in long-run simulated markets.
-
NestRL: A Nested Training Regime for Mutual Adaptation in Human-AI Teaming
A nested training ladder—train adaptive simulated partners at one level, then train the agent against them—improves coordination with unseen adaptive partners in Overcooked, but its non-collapse theorem restates its o...
-
HIVEX: A High-Impact Environment Suite for Multi-Agent Research (extended version)
HIVEX is an open-source benchmark with five Unity-based multi-agent environments for ecological tasks, plus trained PPO baselines and a Hugging Face leaderboard.
Discussion (0). Continue with ORCID to comment.