Pith. sign in

REVIEW 4 major objections 6 minor 84 references

A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim: five dimensions — Observability, State Space, Action Space, Reward Sparsity, Multi-Agent Scale — estimate how hard a game is for a learning agent, and ordering genres by them maps the field from two-agent board ga

desk verdict A useful, well-organized survey of MARL in video games, but its claimed five-dimension complexity framework is applied with only four dimensions and contains internal numeric contradictions; worth revising rather than desk-rejecting. read the letter →

arxiv 2509.03682 v1 pith:5P2IBTY2 submitted 2025-09-03 cs.LG

Zhengyang Li , Qijin Ji , Xinghong Ling , Quan Liu This is my paper
classification cs.LG
keywords multi-agentreinforcementlearningvideogameAIcomplexityframeworkself-playcentralizedtrainingdecentralizedexecutiondeepRTSandMOBAgamessurveyreview
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This review of multi-agent reinforcement learning (MARL) in video games argues that the field has a recognizable arc — from TD-Gammon's self-play backgammon in the 1990s to AlphaStar in StarCraft II, OpenAI Five in Dota 2, and superhuman agents in Honor of Kings — and that this arc can be organized and compared systematically. Its own contribution is a five-dimension method for estimating how hard a game is for a learning agent: Observability, State Space, Action Space, Reward Sparsity, and Multi-Agent Scale. The paper applies those dimensions to order game genres from simplest to hardest — competitive/sports games, then first-person games, then RTS and MOBA — and reviews the landmark systems in each, based on a curated corpus of 40 core studies plus 44 supporting works. If the framework holds, developers and researchers gain a common, MDP-grounded language for comparing environments and judging where MARL techniques are likely to pay off. The paper also catalogs recurring obstacles — nonstationarity, partial observability, sparse rewards, credit assignment, scalability — and the gap between research systems and an industry that prizes predictable, budget-friendly AI.

What carries the argument

The central instrument is the paper's five-dimension game-complexity estimate, derived from the Markov Decision Process formalization that underlies RL: Observability (how much of the state agents perceive), State Space (size and format of observations), Action Space (number and type of actions per step), Reward Sparsity (frequency of feedback), and Multi-Agent Scale (number of interacting agents). The framework does the argument's organizing work: it replaces subjective or production-budget game classification (AAA/AA/indie) with a learning-oriented measure, and it supplies the review's ordering principle, so that each genre's placement in the difficulty ladder, and each cited system's sign

What would settle it

Re-run the review's search with documented per-stage screening counts and publish the full list of 40 core studies; if the claimed ordering — sports games easiest, FPS intermediate, RTS/MOBA hardest — or the 'CTDE dominates' claim changes when negative results and DTDE-based deployments are included, the synthesis fails. For the framework itself, train one fixed algorithm (e.g., PPO with identical hyperparameters) on two games the five dimensions rank as equal; if the learning curves diverge sharply, the dimensions are missing a load-bearing factor. A single documented commercial game that suc

Watch

Extended reading notes

Core claim

The paper asserts that the learning difficulty of a game for a MARL agent can be estimated along five MDP-derived dimensions — Observability, State Space, Action Space, Reward Sparsity, and Multi-Agent Scale — giving a reproducible complexity ordering in place of subjective or budget-based labels. Applied to the field, it orders games from two-agent turn-based play (TD-Gammon, AlphaGo) to real-time combat (Blade & Soul), then by genre from sports (3v3 Snake, Google Research Football, Roller Champions, Rocket League) to first-person games (ViZDoom, Minecraft, Quake III CTF with the FTW agent) to RTS/MOBA (StarCraft II's AlphaStar, Dota 2's OpenAI Five, Honor of Kings), summarizing each system

Load-bearing premise

The load-bearing premise is that the 40 hand-picked core studies fairly represent MARL in video games — if famous successes like AlphaStar and OpenAI Five crowd out failures and smaller-venue work (a bias the paper itself concedes in Section III.D), the genre difficulty ranking and the 'CTDE dominates' claim both inherit that skew.

Editorial extensions

If this is right

  • Game teams could classify a title's AI difficulty before building anything: the five dimensions replace budget- or subjectivity-based labels such as AAA/AA/indie with a learning-oriented measure (Section V.A).
  • The genre ordering gives the field a shared map — sports and competitive games as the easy end, first-person games in the middle, RTS and MOBA at the hard end — making cross-game comparisons of MARL results possible.
  • The claimed dominance of centralized training with decentralized execution (CTDE) tells practitioners that most solvable game settings need shared information at training time but local observations at run time; only simple fully-competitive settings like ViZDoom deathmatch use fully decentralized training.
  • The collected scale figures — tens of thousands of steps per Dota 2 match, roughly 500 actions per minute in StarCraft II, and the compute budgets of the landmark systems — indicate that scalability, not algorithm design alone, limits MARL in commercial games.
  • For the game industry, the review concludes that adoption hinges on designer-controllable AI and cost-efficient toolkits rather than on further superhuman benchmark wins.
  • The compute figures the review collects (about 180 years of gameplay data per day for OpenAI Five's training; 35,000 CPUs for Honor of Kings) imply that scalability is a binding constraint on full commercial adoption, alongside the need for predictable, designer-controlled behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The five dimensions are used as an ordering device, not a measured metric: the paper never specifies how the dimensions trade off against one another, so two games ranked equal could still differ sharply in practice. Calibrating the dimensions against measured learning curves would turn the framework into a predictive instrument.
  • The framework counts agents but not their diversity. Asymmetric roles, rich drafting phases (over 10^11 lineups in Honor of Kings before expanding to the full hero pool), or tight inter-agent coupling could make a game harder than its five-dimensional profile suggests, pointing to a possible sixth axis.
  • Because the corpus was curated around landmark successes, negative or partial results are likely underrepresented; a public registry of failed MARL game deployments would test whether the sports-to-MOBA difficulty ladder survives unbiased sampling.
  • The industry-gap discussion carries an untested prescription: designer-controllable reward shaping and pretrained generalist agents, if made cheap and accessible, would likely move MARL adoption in commercial games more than any single superhuman demonstration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper is a survey of multi-agent reinforcement learning (MARL) applied to video games, spanning two-agent games (Backgammon, Go, Blade & Soul) and multi-agent genres (sports, FPS, RTS/MOBA), with detailed case studies of AlphaStar, OpenAI Five, FTW, and Tencent's Honor of Kings AI. It also proposes a "novel method to estimate game complexity" based on five dimensions: Observability, State Space, Action Space, Reward Sparsity, and Multi-Agent Scale. The review is built on a curated corpus of 40 core studies plus 44 supporting works, and it discusses MARL challenges, training paradigms, industry gaps, and future directions.

Significance. If the factual summaries and the complexity framework are reliable, the paper would provide a useful organized reference for MARL in games and a practical taxonomy for comparing game environments. Strengths include the breadth of systems covered, the explicit attention to implementation details (architectures, reward structures, training setups), and the candid acknowledgment of methodology limitations in Section III.D. The paper ships no derivations or predictions, so the standard circularity concerns do not apply. However, the paper's own central analytical contribution is not consistently delivered: the five-dimension framework is applied as a four-dimension one, and key quantitative cells in the summary tables contradict the narrative text. These issues must be resolved before the survey can serve as a reliable reference.

major comments (4)
  1. [Section V.B vs. Table II and Fig. 4] The Introduction and Section V.B announce a five-dimension complexity framework (Observability, State Space, Action Space, Reward Sparsity, Multi-Agent Scale). Yet Table II, which Section VII.A introduces as the summary of this classification, has no Multi-Agent Scale column, and Fig. 4's caption lists only state space, action space, reward sparsity, and environment observability. Section VII.A-C use Table II and Fig. 4 as the operational output of the framework. As written, the claimed "novel method to estimate game complexity" is not demonstrated with five dimensions. Either the framework must be extended to include Multi-Agent Scale in the tables and figure, or the claim should be revised to a four-dimension framework. This is load-bearing because the framework is the paper's main analytical contribution.
  2. [Table II vs. Sections IV.B and VII.C.1] There are unresolved unit and numeric contradictions in the quantitative basis of the complexity classification. Table II lists Dota 2's action-space row as "8000–80000 options per step, ~81,000 steps"; Section IV.B states the OpenAI Five agent selects an action every fourth frame and that a 45-minute, 30 fps Dota 2 game sums to roughly 20,000 steps per episode. The 81,000 figure is the frame count, not the agent step count, and the table does not define which it uses. Similarly, Table II gives StarCraft II ~14,400 steps at 16 steps/s, which corresponds to a 15-minute match, but Section VII.C.1 says matches last "approximately ten minutes" (which would be ~9,600 steps). These are not one-off typos: they are the entries used for the genre-wise complexity ordering. Please define units consistently (agent steps vs. frames) and reconcile all table and text values.
  3. [Section III.B-D and the comprehensiveness claim] The "comprehensive review" claim rests on a curated 40-study core corpus, but the methodology is not auditable as reported. Section III.B describes an initial professional-experience-based selection, a database search, manual screening of the top 500 of over 17,000 Google Scholar results, and citation chaining, but no per-stage counts, no inter-rater screening procedure, and no enumeration of the 40 core studies are provided. Section III.D itself concedes that citation chaining can bias selection toward more visible or recent studies and that manual filtering introduces subjectivity. Without an audit trail, field-level conclusions such as "the CTDE paradigm dominates" (Section II.D) and the genre-wise complexity ordering (Section VII) may inherit selection bias. A PRISMA-style flow diagram, the list of core studies, and screening agreement details should be added.
  4. [Table II and Table IV: missing operationalization of dimensions] The framework's dimensions are presented without a scoring rubric. In Table II, several Reward Sparsity cells are blank (e.g., Google Research Football, Roller Champions, Rocket League, Dota 2, Honor of Kings), and Table IV has no Reward Sparsity column at all. The Observability labels (Full/High/Partial) are not defined, and no criterion is given for classifying a game as having "Frequent" vs. "Intermediate" vs. "Sparse" rewards. Section V.C promises a "practical and reproducible classification," but the missing definitions and missing cells prevent reproduction. Please provide explicit ordinal definitions for each dimension and fill or explicitly explain every cell in the summary tables.
minor comments (6)
  1. [Introduction] Typo: "a novel L eague Training mythology" should be "methodology." Also "noval" in Section VII.C.1 should be "novel."
  2. [Eq. (15)] Eq. (15) as printed defines R_i' in terms of itself: R_i' = (1 - tau) * R_i' + tau * R'_team - R'_opponent. This appears to be a typo for an update R_i' <- (1 - tau) * R_i + tau * (R'_team - R'_opponent), or a missing superscript on the right-hand side. Please correct.
  3. [Eq. (16)] The formal expression in Eq. (16) uses an emoji symbol for the winning comparator. This is not appropriate in a formal definition; use a standard relation symbol or define a named operator.
  4. [Table II and Table IV] Formatting issue: "1026 options per step" should be "10^26 options per step" in both tables, matching the text in Section VII.C.1. Also the Dota 2 row mixes action-space size with episode length in one cell; separate the two quantities.
  5. [Section VII.C.3] The sentence "Minimax algorithm used in OpenAI Five" is inaccurate. OpenAI Five did not use Minimax; the preceding discussion concerns MCTS for hero drafting in Honor of Kings. Please correct this reference.
  6. [References] Reference [62] lacks complete publication information (venue, date). Some other references (e.g., [78]) are blog posts; this is acceptable for context but should be consistently identified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey's complexity taxonomy is definitional and the review's claims rest on external, non-self-cited systems.

full rationale

Walked the paper's claimed derivation chain. The paper is a survey; its central analytical contribution is the five-dimension complexity classification (Section V.B), which is explicitly definitional: each dimension (observability, state space, action space, reward sparsity, multi-agent scale) is introduced as a category derived from the MDP formalism, not as a quantity computed from downstream results. The later genre ordering (Fig. 4, Table II) applies these definitions; even where Table II omits Multi-Agent Scale or contains step-count inconsistencies, those are failures of consistent application and factual accuracy, not cases where a prediction is equivalent to an input by construction. No parameter is fitted to a subset and then relabeled as a prediction. No uniqueness theorem is imported from the authors' prior work; the reference list contains no self-citations by the four authors, and the load-bearing summaries (AlphaStar, OpenAI Five, FTW, Honor of Kings) cite external published systems. The methodology section discloses selection bias and subjectivity (Section III.D), which is a limitation on representativeness, not circularity. Accordingly no circular step meets the quote-and-reduction standard.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The review's central contribution rests on three inputs the reader does not get to verify: (1) the representativeness of the manually curated 40-study corpus (Section III.B), which underlies every field-level summary; (2) the validity of the MDP-derived five dimensions as proxies for learning difficulty (Section V), asserted without validation and applied inconsistently (Table II omits the Multi-Agent Scale dimension); and (3) the fidelity of the summaries of cited works, with one likely misattribution (OpenAI Five drafting, Section VII.C.3). The complexity labels in Tables II and IV are hand-assigned categorical values, listed here as free parameters because no algorithm or rubric generates them. No invented entities are introduced.

free parameters (2)
  • Per-game complexity codings in Table II and Table IV (observability labels, reward sparsity labels) = categorical labels, e.g., 'Full', 'Partial', 'Frequent', 'Sparse'
    Assigned by the authors with no stated scoring rubric, no inter-rater check, and no aggregation formula; the framework's proposed fifth dimension, Multi-Agent Scale, does not appear in Table II.
  • Action-space and episode-length numbers in Table II (e.g., Dota 2 '~81,000 steps') = numeric strings, e.g., '~81,000 steps'
    Transcribed from source papers; at least one value (Dota 2 steps) contradicts the paper's own Section IV.B figure of roughly 20,000 steps per episode, so the numbers are not reliable as transcribed parameters.
assumptions (4)
  • domain assumption The five MDP-derived dimensions are sufficient and meaningful proxies for the complexity of learning a game.
    Invoked in Section V.B and used to order the entire review (Section VII). No validation against measured learning difficulty or inter-rater reliability is given.
  • domain assumption Formalizing a game as an MDP enables consistent reasoning about how complex a game is to learn, play and master.
    Section V.A asserts this without demonstration; Table II mixes categorical and numeric entries, so cross-game comparisons are not consistent.
  • domain assumption The cited primary sources are represented faithfully.
    Fidelity of summaries is the review's empirical base; Section VII.C.3's claim that OpenAI Five used a Minimax algorithm for drafting appears to misrepresent the OpenAI Five report [24].
  • standard math Standard RL/MARL background (MDPs, Bellman equations, policy gradients, Markov games) is accepted background.
    Sections II.B-II.D summarize standard textbook material from [6], [32], [33], [43]; equations (1)-(11) are standard.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games." pith.science (2026). https://pith.science/paper/5P2IBTY2

@misc{pith2026250903682,
  author       = {Pith},
  title        = {Pith review of: A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5P2IBTY2}},
  note         = {Machine review of arXiv:2509.03682}
}
read the original abstract

Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated its application potential in modern games. Beginning with foundational work and progressing to landmark achievements such as AlphaStar in StarCraft II and OpenAI Five in Dota 2, MARL has proven capable of achieving superhuman performance across diverse game environments through techniques like self-play, supervised learning, and deep reinforcement learning. With its growing impact, a comprehensive review has become increasingly important in this field. This paper aims to provide a thorough examination of MARL's application from turn-based two-agent games to real-time multi-agent video games including popular genres such as Sports games, First-Person Shooter (FPS) games, Real-Time Strategy (RTS) games and Multiplayer Online Battle Arena (MOBA) games. We further analyze critical challenges posed by MARL in video games, including nonstationary, partial observability, sparse rewards, team coordination, and scalability, and highlight successful implementations in games like Rocket League, Minecraft, Quake III Arena, StarCraft II, Dota 2, Honor of Kings, etc. This paper offers insights into MARL in video game AI systems, proposes a novel method to estimate game complexity, and suggests future research directions to advance MARL and its applications in game development, inspiring further innovation in this rapidly evolving field.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 65 canonical work pages

  1. [1]

    Video games remain America’s favorite pastime with more than 212 million Americans playing regularly,

    "Video games remain America’s favorite pastime with more than 212 million Americans playing regularly," Entertainment Software Association, Washington, DC, USA, Press Release, Jul. 10, 2023. [Online]. Available: https://www.theesa.com/video -games-remain- americas-favorite-pastime-with-more-than-212-million-americans- playing-regularly/

    2023

  2. [2]

    GVR-4-68038-527-4,

    Video Game Market Size, Share & Trends Analysis Report by Device (Console, Mobile, Computer), by Type (Online, Offline), by Region (Asia Pacific, North America, Europe), and Se gment Forecasts, 2023 –2030, Horizon Databook, San Francisco, CA, USA, Rep. GVR-4-68038-527-4,

    2023

  3. [3]

    2024 essential facts about the U.S. video game industry,

    “2024 essential facts about the U.S. video game industry,” the ESA, 2024. [Online]. Available: https://www.theesa.com/resources/essential -facts- about-the-us-video-game-industry/2024-data/

    2024

  4. [4]

    A machine for playing the game Nim,

    R. Redheffer, “A machine for playing the game Nim,” Am. Math. Mon., vol. 55, no. 6, pp. 343–349, Jun. 1948.,

    work page 1948

  5. [5]

    Understanding behavior trees,

    A. J. Champandard, "Understanding behavior trees," AiGameDev.com, no. 6, 2007

    work page 2007

  6. [6]

    R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 1998

    work page 1998

  7. [7]

    Using reinforcement learning to solve AI control problems,

    J. Manslow, “Using reinforcement learning to solve AI control problems,” in AI Game Programming Wisdom 2, S. Rabin, Ed. Hingham, MA, USA: Charles River Media, 2004

    work page 2004

  8. [8]

    Connectionist reinforcement learning for intelligent unit micromanagement in StarCraft,

    A. Shantia, E. Begue, and M. Wiering, “Connectionist reinforcement learning for intelligent unit micromanagement in StarCraft,” in Proc. Int. Joint Conf. Neural Netw., San Jose, CA, USA, 2011, pp. 1794–1801, doi: 10.1109/IJCNN.2011.6033442

    work page arXiv 2011

Show all 84 references
  1. [9]

    Go with the flow: Reinforcement learning in turn-based battle video games,

    E. Pagalyte, M. Mancini, and L. Climent, "Go with the flow: Reinforcement learning in turn-based battle video games," in Proc. 20th ACM Int. Conf. Intell. Virtual Agents (IVA) , Oct. 2020, pp. 1 –8, doi: 10.1145/3383652.3423868

    2020

  2. [10]

    A review of real -time strategy game AI,

    G. Robertson and I. Watson, “A review of real -time strategy game AI,” AI Mag., vol. 35, no. 4, pp. 75–104, Dec. 2014.,

    2014

  3. [11]

    Human -level control th rough deep reinforcement learning,

    V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, et al. , “Human -level control th rough deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, Feb. 26 2015.,

    2015

  4. [12]

    Playing FPS games with deep reinforcement learning,

    G. Lample and D. S. Chaplot, “Playing FPS games with deep reinforcement learning,” in Proc. AAAI Conf. Artif. Intell., vol. 31, no. 1, Feb. 2017, doi: 10.1609/aaai.v31i1.10827

    2017 doi

  5. [13]

    Deep reinforcement learning for navigation in AAA video games,

    E. Alonso, M. Peter, D. Goumard, and J. Romoff, “Deep reinforcement learning for navigation in AAA video games,” in Proc. 30th Int. Joint Conf. Artif. Intell. (IJCAI), Z.-H. Zhou, Ed., Montreal, QC, Canada, Aug. 2021, pp. 2133–2139. doi: 10.24963/ijcai.2021/294

    2021 doi

  6. [14]

    S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed. Boston, MA, USA: Pearson, 2018

    2018

  7. [15]

    Mastering Atari, Go, chess and shogi by planning with a learned model,

    J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, et al., “Mastering Atari, Go, chess and shogi by planning with a learned model,” Nature, vol. 588, no. 7839, pp. 604–609, Dec. 2020.,

    2020

  8. [16]

    Expert human - level driving in Gran Turismo Sport using deep reinforcement learning with image-based representation,

    R. Imamura, T. Seno, K. Kawamoto, and M. Spranger, “Expert human - level driving in Gran Turismo Sport using deep reinforcement learning with image-based representation,” arXiv:2111.06449, 2021

    2021 arXiv

  9. [17]

    Super - human performance in Gran Turismo Sport using deep reinforcement learning,

    F. Fuchs, Y. Song, E. Kaufmann, D. Scaramuzza, and P. Duerr, “Super - human performance in Gran Turismo Sport using deep reinforcement learning,” IEEE Robot. Autom. Lett., vol. 6, no. 2, pp. 1–8, Apr. 2021.,

    2021

  10. [18]

    Technical challenges of deploying reinforcement learning agents for game testing in AAA games,

    J. Gillberg, J. Bergdahl, A. Sestini, A. Eakins, and L. Gisslén, “Technical challenges of deploying reinforcement learning agents for game testing in AAA games,” in Proc. IEEE Conf. Games (CoG) , Boston, MA, USA, 2023, pp. 1–8, doi: 10.1109/CoG57401.2023.10333194

    2023

  11. [19]

    TD -Gammon, a self -teaching backgammon program, achieves master-level play,

    G. Tesau ro, “TD -Gammon, a self -teaching backgammon program, achieves master-level play,” Neural Comput., vol. 6, no. 2, pp. 215–219, Mar. 1994

    1994

  12. [20]

    Mastering the game of Go with dee p neural networks and tree search,

    D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, et al., “Mastering the game of Go with dee p neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489, Jan. 28 2016.,

    2016

  13. [21]

    Mastering the game of Go without human knowledge,

    D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, et al. , “Mastering the game of Go without human knowledge,” Nature, vol. 550, no. 7676, pp. 354–359, Oct. 18 2017.,

    2017

  14. [22]

    A general reinforcement learning algorithm that masters chess, shogi, and Go through self -play,

    D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, et al., “A general reinforcement learning algorithm that masters chess, shogi, and Go through self -play,” Science, vol. 362, no. 6419, pp. 1140–1144, Dec. 7 2018.,

    2018

  15. [23]

    Grandmaster level in StarCraft II using multi -agent reinforcement learning,

    O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, et al. , “Grandmaster level in StarCraft II using multi -agent reinforcement learning,” Nature, vol. 575, no. 7782, pp. 350–354, Nov. 2019.,

    2019

  16. [24]

    Dota 2 with larg e scale deep reinforcement learning,

    OpenAI et al. , “Dota 2 with larg e scale deep reinforcement learning,” arXiv:1912.06680 [cs, stat], Dec. 2019

    1912 arXiv

  17. [25]

    A comprehensive survey of multiagent reinforcement learning,

    L. Busoniu, R. Babuska, and B. De Schutter, "A comprehensive survey of multiagent reinforcement learning," IEEE Trans. Syst., Man, Cybern., Part C (Appl. Rev.) , vol. 38, no. 2, pp. 156 –172, Mar. 2008, doi: 10.1109/TSMCC.2007.913919

    2008

  18. [26]

    Multi-agent reinforcement learning: A selective overview of theories and algorithms,

    K. Zhang, Z. Yang, and T. Basar, “Multi-agent reinforcement learning: A selective overview of theories and algorithms,” in Handbook of Reinforcement Learning and Control , pp. 321 –384, 2021, doi: 10.1007/978-3-030-60990-0_12

    2021 doi

  19. [27]

    Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications,

    T. T. Nguyen, N. D. Nguyen, and S. Nahavandi, “Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications,” IEEE Trans. Cybern., vol. 50, no. 9, pp. 3826–3839, Sep. 2020.,

    2020

  20. [28]

    Multi-agent reinforcement learning: A review of challenges and applications,

    L. Canese, G. C. Cardarilli, L. Di Nunzio, R. Fazzolari, D. Giardino, M. Re, et al., “Multi-agent reinforcement learning: A review of challenges and applications,” Appl. Sci. (Basel), vol. 11, no. 11, p. 4948, May 2021.,

    2021

  21. [29]

    Deep learning for video game playing,

    N. Justesen, P. Bontrager, J. T ogelius, and S. Risi, “Deep learning for video game playing,” IEEE Trans. Games, vol. 12, no. 1, pp. 1–20, Mar. 2020.,

    2020

  22. [31]

    Reinforcement learning in game industry—Review, prospects and challenges,

    K. Souchleris, G. K. Sidiropoulos, and G. A. Papakostas, “Reinforcement learning in game industry—Review, prospects and challenges,” Appl. Sci. (Basel), vol. 13, no. 4, p. 2441, Feb. 2023.,

    2023

  23. [32]

    Q-learning,

    C. J. C. H. Watkins and P. Dayan, “Q-learning,” Mach. Learn., vol. 8, no. 3–4, pp. 279–292, 1992

    1992

  24. [33]

    Policy gradient methods for reinforcement learning with function approximation,

    R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” in Proc. 12th Int. Conf. Neural Inf. Process. Syst. (NIPS'99) . Cambridge, MA, USA: MIT Press, 1999, pp. 1057–1063

    1999

  25. [34]

    Wooldridge, An Introduction to MultiAgent Systems , 2nd ed

    M. Wooldridge, An Introduction to MultiAgent Systems , 2nd ed. Chichester, U.K.: Wiley, 2009

    2009

  26. [35]

    Modular AI,

    K. Dill and C. Dragert, “Modular AI,” in Game AI Pro 3: Collected Wisdom of Game AI Professionals, S. Rabin, Ed., Boca Raton, FL, USA: CRC Press, 2017

    2017

  27. [36]

    Long short -term memory,

    S. Hochreiter and J. Schmidhuber, “Long short -term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 15 1997

    1997

  28. [37]

    Deep recurrent Q -learning for partially observable MDPs,

    M. Hausknecht and P. Stone, “Deep recurrent Q -learning for partially observable MDPs,” in Proc. AAAI Fall Symp. Ser., 2015

    2015

  29. [38]

    Rainbow: Combining improvements in deep reinforcement learning,

    M. Hessel et al. , "Rainbow: Combining improvements in deep reinforcement learning," in Proc. 32nd AAAI Conf. Artif. Intell. (AAAI'18), New Orleans, LA, USA, 2018, pp. 3215–3222

    2018

  30. [39]

    Actor-critic algorithms,

    V. R. Konda and J. N. Tsitsiklis, “Actor-critic algorithms,” in Adv. Neural Inf. Process. Syst. (NIPS), 2000, pp. 1008–1014

    2000

  31. [40]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, "Continuous control with deep reinforcement learning," arXiv preprint arXiv:1509.02971, 2019. [Online]. Available: https://arxiv.org/abs/1509.02971

    2019 arXiv

  32. [41]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  33. [42]

    Shoham and K

    Y. Shoham and K. Leyton -Brown, Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations . New York, NY, USA: Cambridge Univ. Press, 2008. doi.org/10.1017/CBO9780511811654

    2008 doi

  34. [43]

    Markov games as a framework for multi -agent reinforcement learning,

    M. L. Littman, "Markov games as a framework for multi -agent reinforcement learning," in Proc. 11th Int. Conf. Mach. Learn. (ICML'94), San Francisco, CA, USA: Morgan Kaufmann, 1994, pp. 157–163

    1994

  35. [44]

    Multiagent reinforcement learning: Theoretical framework and an algorithm,

    J. Hu and M. P. Wellman, "Multiagent reinforcement learning: Theoretical framework and an algorithm," in Proc. 15th Int. Conf. Mach. Learn. (ICML '98), San Francisco, CA, USA: Morgan Kaufmann, 1998, pp. 242–250

    1998

  36. [45]

    StarCraft II: A new challenge for reinforcement learning,

    O. Vinyals et al. , “StarCraft II: A new challenge for reinforcement learning,” arXiv:1708.04782 [cs], Aug. 2017

    2017 arXiv

  37. [46]

    The StarCraft multi -agent challenge,

    M. Samvelyan et al. , "The StarCraft multi -agent challenge," arXiv preprint arXiv:1902.04043 , Dec. 2019. [Online]. Available: https://arxiv.org/abs/1902.04043 21 ToG-2024-0292.R1

    1902 arXiv

  38. [47]

    Strangers and friends,

    B. Nardi and J. Harris, “Strangers and friends,” in Proc. 2006 20th Anniversary Conf. Computer Supported Cooperative Work - CSCW '06, 2006, doi: 10.1145/1180875.1180898

    2006

  39. [48]

    Hierarchical macro strategy model for MOBA game AI,

    B. Wu, “Hierarchical macro strategy model for MOBA game AI,” in Proc. AAAI Conf. Artif. Intell. , vol. 33, pp. 1206 –1213, Jul. 2019, doi: 10.1609/aaai.v33i01.33011206

    2019 doi

  40. [49]

    Unifying temporal and structural credit assignment problems,

    A. K. Agogino and K. Tumer , "Unifying temporal and structural credit assignment problems," in Proc. 17th Int. Conf. Auton. Agents Multiagent Syst. (AAMAS), 2004

    2004

  41. [50]

    Multiagent systems: A survey from a machine learning perspective,

    P. Stone and M. Veloso, “Multiagent systems: A survey from a machine learning perspective,” Auton. Robots, vol. 8, no. 3, pp. 345–383, 2000.,

    2000

  42. [51]

    Temporal difference learning and TD-Gammon,

    G. Tesauro, “Temporal difference learning and TD-Gammon,” Commun. ACM, vol. 38, no. 3, pp. 58–68, Mar. 1995.,

    1995

  43. [52]

    Creating pro-level AI for a real -time fighting game using deep reinfo rcement learning,

    I. Oh, S. Rho, S. Moon, S. Son, H. Lee, and J. Chung, “Creating pro-level AI for a real -time fighting game using deep reinfo rcement learning,” IEEE Trans. Games, vol. 14, no. 2, pp. 212–220, Jun. 2022.,

    2022

  44. [53]

    On the utility of learning about humans for human-AI coordination,

    M. Carroll et al., “On the utility of learning about humans for human-AI coordination,” in Proc. 33rd Int. Conf. Neural Inf. Process. Syst. (NeurIPS), Dec. 2019, pp. 5174–5185

    2019

  45. [55]

    IMPALA: Scalable distributed deep -RL with importance weighted actor -learner architectures,

    L. Espeholt, et al. , “IMPALA: Scalable distributed deep -RL with importance weighted actor -learner architectures,” in Proc. 35th Int. Conf,” Mach. Learn., vol. 80, pp. 1407–1416, Jul. 10–15 2018

    2018

  46. [56]

    Distributed prioritized experience replay,

    D. Horgan et al., “Distributed prioritized experience replay,” arXiv, Mar. 2, 2018. [Online]. Available: https://arxiv.org/abs/1803.00933

    2018 arXiv

  47. [57]

    Make a more engaging game w/ ML -Agents | Machine learning bots for game development | Reinforcement learning | Unity

    Unity Technologies, “Make a more engaging game w/ ML -Agents | Machine learning bots for game development | Reinforcement learning | Unity.” [Online]. Available: https://unity.com/pr oducts/machine- learning-agents

  48. [58]

    Reinforcement learning agents for Ubisoft’s Roller Champions,

    N. Iskander, A. Simoni, E. Alonso, and M. Peter, “Reinforcement learning agents for Ubisoft’s Roller Champions,” arXiv, 2020. [Online]. Available: https://arxiv.org/abs/2012.06031

    2020 arXiv

  49. [59]

    MonoBehaviour.FixedUpd ate,

    Unity Technologies, "MonoBehaviour.FixedUpd ate," Unity Documentation. [Online]. Available: https://docs.unity3d.com/ScriptRef erence/MonoBehaviour.FixedUpdate.html. [Accessed: Nov. 22, 2024]

    2024

  50. [60]

    On the verge of solving Rocket League using deep reinforcement learning and sim -to-sim transfer,

    M. Pleines et al., “On the verge of solving Rocket League using deep reinforcement learning and sim -to-sim transfer,” in Proc. IEEE Conf. Games (CoG) , Beijing, China, 2022, pp. 253 –260, doi: 10.1109/CoG51982.2022.9893628

    2022

  51. [61]

    On the potential of Rocket League for driving team AI development,

    Y. Verhoeven and M. Preuss, “On the potential of Rocket League for driving team AI development,” in Proc. IEEE Symp. Series Comput. Intell. ( SSCI), Canberra, ACT, Australia, 2020, pp. 2335 –2342, doi: 10.1109/SSCI47803.2020.9308248

    2020

  52. [62]

    Lucy - SKG: Learning to play Rocket League efficiently using deep reinforcement learning,

    V. Moschopoulos, P. Kyriakidis, A. Lazaridis, and I. Vlahavas, “Lucy - SKG: Learning to play Rocket League efficiently using deep reinforcement learning,” arXiv

  53. [63]

    RLGym: Reinforcement learning in Rocket League,

    RLGym, "RLGym: Reinforcement learning in Rocket League," [Online]. Available: https://rlgym.org/. [Accessed: Nov. 22, 2024]

    2024

  54. [64]

    Attention is all you need,

    A. Vaswani et al., "Attention is all you need," in Proc. 31st Conf. Neural Inf. Process. Syst. (NeurIPS), 2017, vol. 30, pp. 6000–6010

    2017

  55. [65]

    ViZDoom: A Doom-based AI research platform for visual reinforcement learning,

    M. Kempka, M. Wydmuch, G. Runc, J. Toczek, and W. Jaśkowski, “ViZDoom: A Doom-based AI research platform for visual reinforcement learning,” in Proc. IEEE Conf. Comput. Intell. Games (CIG) , Santorini, Greece, 2016, pp. 1–8, doi: 10.1109/CIG.2016.7860433

    2016

  56. [66]

    ViZDoom competitions: Playing Doom from pixels,

    M. Wydmuch, M. Kempka, and W. Jaśkowski, “ViZDoom competitions: Playing Doom from pixels,” IEEE Trans. Games, vol. 11, no. 3, pp. 248– 259, Sep. 2019.,

    2019

  57. [67]

    Training agent for first -person shooter game with actor-critic curriculum learning,

    Y. Wu and Y. Tian, “Training agent for first -person shooter game with actor-critic curriculum learning,” in Proc. Int. Conf. Learn. Representations (ICLR), 2016

    2016

  58. [68]

    Asynchronous methods for deep reinforcement learning,

    V. Mnih et al., "Asynchronous methods for deep reinforcement learning," CoRR, vol. abs/1602.01783, 2016. [Online]. Available: http://arxiv.org/abs/1602.01783

    2016 arXiv

  59. [69]

    A Mi necraft-based simulated task environment for human-AI teaming,

    A. Amresh, N. Cooke, and A. Fouse, “A Mi necraft-based simulated task environment for human-AI teaming,” in Proc. 23rd ACM Int. Conf. Intell. Virtual Agents, Wü rzburg, Germany, 2023

    2023

  60. [70]

    Minecraft as an experimental world for AI in robotics,

    K. C. Aluru, S. Tellex, J. G. Oberlin, and J. MacGlashan , “Minecraft as an experimental world for AI in robotics,” in AAAI Fall Symp., 2015

    2015

  61. [71]

    MineRL: A large -scale dataset of Minecraft demonstrations,

    W. H. Guss et al. , “MineRL: A large -scale dataset of Minecraft demonstrations,” arXiv, Jul. 29, 2019. [Online]. Available: https://arxiv.org/abs/1907.13440

    2019 arXiv

  62. [72]

    The Malmo platform for artificial intelligence experimentation,

    M. Johnson, K. Hofmann, T. Hutton, and D. Bignell, “The Malmo platform for artificial intelligence experimentation,” in Proc. 25th Int. Joint Conf. Artif. Intell., 2016, pp. 4246–4247

    2016

  63. [73]

    The Malmo Collaborative AI Challenge - Microsoft Research,

    “The Malmo Collaborative AI Challenge - Microsoft Research,” Microsoft Research , Mar. 16, 2022. [Online]. Available: https://www.microsoft.com/en-us/research/academic-program/collaborat ive-ai-challenge/

    2022

  64. [74]

    HogRider: Champion agent of Microsoft Malmo collaborative AI challenge,

    Y. Xiong, H. Chen, M. Zhao, and B. An, “HogRider: Champion agent of Microsoft Malmo collaborative AI challenge,” in Proc. 32nd AAAI Conf. Artif. Intell., Innovative Appl. Artif. Intell. Conf., and 8th AAAI Symp. Educ. Adv. Artif. Intell., 2018

    2018

  65. [75]

    The Multi -Agent Reinforcement Learning in MalmÖ (MARLÖ) competition,

    D. Perez-Liebana et al. , “The Multi -Agent Reinforcement Learning in MalmÖ (MARLÖ) competition,” arXiv, 2019. [Online]. Available: https://arxiv.org/abs/1901.08129

    2019 arXiv

  66. [76]

    Human-level performance in 3D multiplayer games with population -based reinforcement learning,

    M. Jaderberg, W. M. Czarnecki, I. Dunning, L. Marris, G. Lever, A. G. Castañ eda, et al., “Human-level performance in 3D multiplayer games with population -based reinforcement learning,” Science, vol. 364, no. 6443, pp. 859–865, May 31 2019.,

    2019

  67. [77]

    The exploration - exploitation dilemma: A multidisciplinary framework,

    O. Berger -Tal, J. Nathan, E. Meron, and D. Saltz, “The exploration - exploitation dilemma: A multidisciplinary framework,” PLoS One, vol. 9, no. 4, p. e95693, Apr. 22 2014.,

    2014

  68. [78]

    AlphaStar: Mastering the real -time strategy game StarCraft II,

    The AlphaStar team, “AlphaStar: Mastering the real -time strategy game StarCraft II,” Google DeepMind, Jan. 24, 2019. [Online]. Available: https://deepmind.google/discover/blog/alphastar-mastering-the-real- time-strategy-game-starcraft-ii/

    2019

  69. [79]

    Grid-wise control for multi-agent reinforcement learning in video game AI,

    L. Han, et al., “Grid-wise control for multi-agent reinforcement learning in video game AI,” i n Proc. 36th Int. Conf ,” Mach. Learn. , vol. 97, pp. 2570–2585, 2019

    2019

  70. [80]

    Multiagent bidirectionally-coordinated nets: Emergence of human-level coordination in learning to play StarCraft combat games,

    P. Peng et al., “Multiagent bidirectionally-coordinated nets: Emergence of human-level coordination in learning to play StarCraft combat games,” arXiv, 2017. [Online]. Available: https://arxiv.org/abs/1703.10069

    2017 arXiv

  71. [81]

    Ye, et al., Towards playing full MOBA games with deep reinforcement learning, vol

    D. Ye, et al., Towards playing full MOBA games with deep reinforcement learning, vol. 33. 2020, Adv. Neural Inf. Process. Syst., pp. 621–632

    2020

  72. [82]

    A recommender system for hero line-ups in MOBA games,

    L. Hanke and L. Chaimowicz, “A recommender system for hero line-ups in MOBA games,” in Proc. 13th AAAI Conf. Artif. Intell. Interactive Digit. Entertain., 2017

    2017

  73. [83]

    The art of drafting: A team-oriented hero recommendation system for multiplayer online battle arena games,

    Z. Chen et al., “The art of drafting: A team-oriented hero recommendation system for multiplayer online battle arena games,” arXiv, 2018. [Online]. Available: https://arxiv.org/abs/1806.10130

    2018 arXiv

  74. [84]

    Acting with style: Towards designer-centred reinforcement learning for the video games industry,

    B. Aytemiz, M. Jacob, and S. Devlin, "Acting with style: Towards designer-centred reinforcement learning for the video games industry," in CHI Workshop on Reinforcement Learning for Humans, Computer, and Interaction (RL4HCI), May 2021, p. 16

    2021

  75. [2019]

    Available: https://arxiv.org/abs/1912.10944

    [Online]. Available: https://arxiv.org/abs/1912.10944

    1912 arXiv

  76. [2023]

    Available: https://www.grandviewresearch.com/ industry-analysis/video-game-market

    [Online]. Available: https://www.grandviewresearch.com/ industry-analysis/video-game-market

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.