RECORD
Co-Learning Empirical Games and World Models
Multiagent Systems · Submitted 23 May 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim When compared to PSRO -- a baseline empirical-game building algorithm, Dyna-PSRO is found to compute lower regret solutions on partially observable general-sum games.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Game-based decision-making involves reasoning over both world dynamics and strategic interactions among the agents. Typically, empirical models capturing these respective aspects are learned and used separately. We investigate the potential gain from co-learning these elements: a world model for dynamics and an empirical game for strategic interactions. Empirical games drive world models toward a broader consideration of possible game dynamics induced by a diversity of strategy profiles. Conversely, world models guide empirical games to efficiently discover new strategies through planning. We demonstrate these benefits first independently, then in combination as realized by a new algorithm, Dyna-PSRO, that co-learns an empirical game and a world model. When compared to PSRO -- a baseline empirical-game building algorithm, Dyna-PSRO is found to compute lower regret solutions on partially observable general-sum games. In our experiments, Dyna-PSRO also requires substantially fewer experiences than PSRO, a key algorithmic advantage for settings where collecting player-game interaction data is a cost-limiting factor.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Game-based decision-making involves reasoning over both world dynamics and strategic interactions among the agents. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Typically, empirical models capturing these respective aspects are learned and used separately. | background 0.74 | 0.05 | 0.02 | asserted 1.00 |
| s2 | We investigate the potential gain from co-learning these elements: a world model for dynamics and an empirical game for strategic interactions. | contribution 0.94 | 0.29 | 0.08 | hedged possibility 0.73 |
| s3 | Empirical games drive world models toward a broader consideration of possible game dynamics induced by a diversity of strategy profiles. | interpretation 0.39 | 0.35 | 0.10 | asserted 0.97 |
| s4 | Conversely, world models guide empirical games to efficiently discover new strategies through planning. | method 0.38 | 0.38 | 0.10 | asserted 0.99 |
| s5 | We demonstrate these benefits first independently, then in combination as realized by a new algorithm, Dyna-PSRO, that co-learns an empirical game and a world model. | contribution 0.91 | 0.94 | 0.32 | asserted 1.00 |
| s6 | When compared to PSRO -- a baseline empirical-game building algorithm, Dyna-PSRO is found to compute lower regret solutions on partially observable general-sum games. | result 1.00 | 0.95 | 0.47 | asserted 1.00 |
| s7 | In our experiments, Dyna-PSRO also requires substantially fewer experiences than PSRO, a key algorithmic advantage for settings where collecting player-game interaction data is a cost-limiting factor. | result 0.95 | 0.96 | 0.18 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Game-based decision-making involves reasoning over both world dynamics and strategic interactions among the agents. Typically, empirical models capturing these respective aspects are learned and used separately. We investigate the potential gain from co-learning these elements: a world model for dynamics and an empirical game for strategic interactions. Empirical games drive world models toward a broader consideration of possible game dynamics induced by a diversity of strategy profiles. Conversely, world models guide empirical games to efficiently discover new strategies through planning. We demonstrate these benefits first independently, then in combination as realized by a new algorithm, Dyna-PSRO, that co-learns an empirical game and a world model. When compared to PSRO -- a baseline empirical-game building algorithm, Dyna-PSRO is found to compute lower regret solutions on partially observable general-sum games. In our experiments, Dyna-PSRO also requires substantially fewer experiences than PSRO, a key algorithmic advantage for settings where collecting player-game interaction data is a cost-limiting factor.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites
Cites as background 2 papers
-
World models represent a game's transition dynamics and reward signal directly [ref] .
World Models · Introduction · 0.98
-
World models may either predict successor observations directly (e.g., at pixel level [ref] ), or in a learned latent space [ref] .
From Pixels to Torques: Policy Learning with Deep Dynamical Models · Related Work · 0.98
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.