Pith. sign in

REVIEW 3 major objections 6 minor 36 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Hidden information, not network size, caps lightweight game agents

2026-07-10 00:09 UTC pith:CCYJQLEH

load-bearing objection Solid empirical study with a clean methodology; the central interpretive claim is slightly oversold but the paper deserves a serious referee. the 3 major comments →

arxiv 2607.06854 v1 pith:CCYJQLEH submitted 2026-07-07 cs.LG cs.AIcs.GT

A Gold-Standard Study of What Makes a Lightweight Game-Playing Agent Strong

Nima Kelidari , Mohammadsaeed Haghi , Mahdi Salmani This is my paper
classification cs.LG cs.AIcs.GT
keywords agentexpertlightweightonlypercentstrongtheyagents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds a fixed, strong, rule-based expert for Gin Rummy and uses it purely as a measuring stick—never as a training partner—to run over a hundred controlled experiments that isolate which training choices actually make a lightweight reinforcement-learning agent stronger. The central finding is twofold. First, a specific recipe works: trust-region updates (TRPO over PPO), a reward that favors early knocking over chasing gin, a curriculum of progressively tougher opponents, warm-starting from a prior champion, and shipping the best checkpoint rather than the last one. Stacking these lifts a self-play agent from roughly 30 to about 34 percent against the expert. Second, and more fundamentally, the remaining performance ceiling is set by hidden information, not by model capacity or training technique. Every architecture tested—MLP, convolutional, permutation-invariant set encoders, attention, and recurrent networks—lands in a statistically indistinguishable band. A fair search that cannot see the opponent's cards tops out at 26 percent, below the trained agents, while the same search given oracle access to hidden cards reaches 85 percent. That 26-versus-85 gap is the paper's key piece of evidence: the bottleneck is what the agent does not know, not how big or clever its network is.

Core claim

The performance ceiling for a lightweight agent in an imperfect-information card game is information-bound rather than capacity-bound. The paper establishes this through three converging lines of evidence: (1) varying network architecture across MLP, convolutional, set-based, attention, and recurrent encoders does not meaningfully change win-rate against a fixed expert, with all confidence intervals overlapping; (2) a determinized search graded fairly—re-dealing hidden cards before each rollout—tops out at 26 percent, below the trained agents, while the same search given oracle access to the true hidden state reaches 85 percent, directly quantifying the value of the concealed information; (3

What carries the argument

The fixed rule-based expert serves as the central measuring instrument. It solves one subproblem exactly—finding the lowest-deadwood meld decomposition of a hand—and otherwise plays deterministic, principled endgame heuristics: draw only when it strictly lowers deadwood, discard the card that minimizes resulting deadwood, and knock as soon as legally possible. This expert never enters the training loop; it only grades agents. The training pipeline itself centers on a masked actor-critic (illegal actions get a large finite negative logit before softmax, so both PPO and TRPO can use the same network), a three-stage opponent curriculum (random, then a pool of past checkpoints, then self-play),

Load-bearing premise

The fixed rule-based expert is assumed to be a meaningful and stable yardstick for measuring agent strength. The paper acknowledges it is not a game-theoretic optimum, just a strong, cheap, reproducible heuristic. If the expert's specific endgame strategy is exploitable in a way that self-play agents systematically cannot discover but a different training method could, then the measured ceiling of roughly 34 percent would be an artifact of this particular yardstick rather a

What would settle it

Construct a training method that does not practice against the expert but achieves a win-rate against it significantly above 34 percent—say, above 40—while using a network of comparable size. Alternatively, show that a different fixed expert (still strong, cheap, and deterministic) produces a different ranking of the same training choices, which would undermine the claim that the yardstick cleanly isolates which ingredients matter.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • For any imperfect-information game where a strong, cheap, fixed reference opponent can be constructed, the controlled one-factor-at-a-time methodology demonstrated here can be applied to separate training choices that genuinely help from those that merely sound plausible.
  • The finding that reward shaping cannot bribe an agent into a losing habit—chasing gin—suggests that in adversarial settings, agents may discover strategic truths that override designer intentions, which has implications for how reward design is taught and practiced.
  • The 26-versus-85-percent gap between fair and oracle search provides a reusable diagnostic: for any hidden-information game, comparing fair-search performance to oracle-search performance quantifies how much of the remaining difficulty is informational versus computational.
  • If the information-bound ceiling is real, then the next gains in lightweight card-game agents should come from opponent-hand inference or belief-state reasoning, not from larger networks or more elaborate reward shaping.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The information-bound ceiling argument implicitly assumes the fixed expert is not itself exploitable in ways a different training method could discover. If the expert has systematic weaknesses that self-play cannot find but, say, curriculum-based exploitation could, then the 34-percent ceiling is partly an artifact of the yardstick rather than a fundamental information limit.
  • The failure of learned state embeddings to beat the raw 4×52 sparse observation may be specific to Gin Rummy's small observation space. Games with larger or more structured hidden-state spaces (e.g., Mahjong, DouDizhu) might still benefit from learned embeddings, so the negative result may not generalize as broadly as the paper's game-agnostic framing suggests.
  • The LLM opponent's competence but impractical speed (9–27 seconds per move) hints that offline distillation—generating expert games with an LLM and training on that dataset—could be a productive path the paper identifies but does not pursue, which if successful would complicate the clean information-bound narrative by injecting external strategic knowledge.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper presents a controlled study of training choices for lightweight reinforcement-learning agents in imperfect-information card games, using Gin Rummy as the primary testbed. The authors build a fixed, deterministic, rule-based expert (used only for evaluation, never for training) and run over one hundred controlled experiments, changing one factor at a time, to isolate which ingredients help: TRPO over PPO, a knock-first reward, a rising opponent curriculum, warm-starting, and keeping the best checkpoint. Stacking these lifts a self-play champion from ~30% to ~34.2% win-rate against the expert. Several popular approaches fail (learned embeddings, imitation/DAgger, dense step rewards, live LLM opponent), each with an identified mechanism. An architecture sweep (MLP, convolutional, Deep Sets, recurrent, attention) finds no encoder breaks the ceiling. A fair ISMCTS baseline reaches only 26% while an oracle variant reaches 85%, which the authors interpret as evidence that the ceiling is information-bound rather than capacity-bound. The method is replicated on Leduc Hold'em against a CFR-computed optimum.

Significance. The paper makes a solid methodological contribution by demonstrating the value of a fixed, strong, reproducible reference opponent for grading RL training choices in imperfect-information games—a setting where evaluation is notoriously difficult. The controlled, one-factor-at-a-time experimental design, the use of IQM with stratified bootstrap CIs for the architecture sweep, the honest reporting of negative results with mechanistic explanations, and the release of a reusable pipeline are all commendable. The finding that reward shaping cannot bribe an agent into a losing habit (chasing gin) is a clean and memorable result. The Leduc Hold'em replication against a computable optimum adds external validation. The code release is a genuine strength.

major comments (3)
  1. The central interpretive claim—that the ~34% ceiling is 'information-bound, not capacity-bound' (Abstract; Discussion, 'An information-bound ceiling'; Conclusion)—is supported by three lines of evidence, but the strongest of these rests on a comparison between two variants of a search method (ISMCTS), not of the trained agent. Specifically, the 26% (fair ISMCTS) vs. 85% (oracle ISMCTS) gap in Figure 5b measures the value of perfect information for search, not for the trained policy. The trained agents already exceed the fair search (34% vs. 26%), meaning they implicitly extract something the fair ISMCTS lacks, so the two systems are not directly comparable. The most direct test—retraining the same agent architecture with the opponent's hand added to the 4×52 observation—is not performed. The paper acknowledges opponent-hand inference as future work (Discussion, 'Future work'), yet states
  2. The headline result of 34.2±2.1% (Table 1) appears to be from a single training run evaluated over 2000 games, where the confidence interval reflects binomial evaluation noise rather than training-seed variability. In contrast, the architecture sweep (Figure 5a) properly uses IQM with stratified bootstrap CIs over multiple seeds, following Agarwal et al. (2021). The headline result should be reported with the same multi-seed protocol to establish that the stacked recipe reliably achieves ~34% rather than reflecting a single favorable run. The two PFSP sibling agents at 34.0% provide partial evidence of reproducibility, but they use a different opponent-sampling scheme, so they do not directly replicate the headline recipe.
  3. The Leduc Hold'em result (Table 3) creates a tension with the information-bound claim that is not addressed. The tabular self-play learner reaches near-parity with the CFR optimum (mean return −0.085), yet Leduc is also an imperfect-information game with hidden cards. If the ceiling is fundamentally set by hidden information, one would expect a gap in Leduc as well. The likely explanation is that Leduc is small enough for tabular methods to effectively reason over the hidden information, but this means the ceiling is a function of information complexity relative to model expressiveness and game size—not purely 'information-bound' as a general principle. The paper should either scope the claim to Gin Rummy specifically or discuss why the Leduc near-parity does not contradict the general claim.
minor comments (6)
  1. The abstract states the stacked recipe lifts the agent 'from about 30 to 36 percent against the expert,' but Table 1 reports 34.2±2.1%. The value 36 appears to correspond to the upper bound of the 95% CI (36.3), not the point estimate. The abstract should report the point estimate.
  2. The architecture sweep (Figure 5a) retrains from scratch without warm-starting, since a different network shape cannot warm-start from the MLP champion. The best IQM in the sweep is ~31% (convolutional), below the headline 34.2% which uses warm-start. The paper should explicitly note that the architecture sweep tests a different (and harder) configuration than the headline recipe, so the 'architecture does not break the ceiling' claim is relative to the from-scratch baseline, not the full stacked recipe.
  3. In the 'Reward: you cannot pay the agent into ginning' section, the text states 'paying three times more for a gin than a knock leaves the gin rate under one percent.' It would help to specify the exact reward coefficients used for each condition in the sweep, perhaps in a table or appendix, for reproducibility.
  4. The self-attention encoder result is reported as unable to train under TRPO due to double-differentiation cost, and under PPO it does not beat a plain MLP (17% vs. 21%). Since the attention encoder is the most natural fit for an unordered card set, a brief discussion of whether this is a fundamental limitation or an implementation artifact would strengthen the architecture conclusion.
  5. The paper mentions a 'web game (human vs. agent)' component in Figure 1 but does not report any human-play results or evaluation. If this is only a tool release, that should be clarified; if human data was collected, it should be reported or removed from the figure.
  6. Table 2 lists 'Pay three times more for gin' as 'no effect,' but the text also notes that against weak opponents, a gin-first reward does increase the gin rate (22% vs. 97% knocks). The verdict 'no effect' should be qualified as 'no effect against the expert' to avoid confusion.

Circularity Check

0 steps flagged

No circularity found: the expert yardstick is built independently of the agents it evaluates, and the information-bound ceiling claim is supported by independent measurements (architecture sweep, oracle vs. fair search gap, CFR optimum on Leduc).

full rationale

The paper's central claims are grounded in externally verifiable benchmarks and do not reduce to their inputs by construction. (1) The fixed rule-based expert is built from RLCard's meld enumeration and principled heuristics (exact meld decomposition, draw/discard/knock rules). It is never used in training—only for evaluation—so no agent is fitted to the yardstick it is measured against. (2) The 'information-bound ceiling' conclusion is supported by three independent lines of evidence: the architecture sweep (varying network capacity shows overlapping CIs), the oracle-vs-fair ISMCTS gap (26% vs 85%), and the lack of opponent-hand estimation in the reactive policy. The oracle search is an independent upper bound that does not share parameters with the trained agents. (3) The Leduc Hold'em validation uses a CFR-computed near-optimal expert (exploitability 0.026), which is an external game-theoretic benchmark, not a self-referential one. (4) The reward-shaping result ('you cannot pay the agent into ginning') is an empirical finding: the gin rate stays under 1% regardless of reward design, which is not forced by construction. The skeptic's concern that the oracle-vs-fair-search gap measures the value of information for search rather than for the trained policy is a validity/generalization concern, not a circularity concern—the paper does not define its conclusion in terms of its own inputs or fit a parameter to the data it then predicts. No self-definitional, fitted-input-as-prediction, or self-citation-chain circularity is present.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper introduces no new entities or particles. It uses standard RL algorithms, known game environments, and established theoretical frameworks. The free parameters are standard hyperparameters and design choices expected in an RL study.

free parameters (4)
  • Curriculum stage boundaries = fractions of training budget (not specified exactly)
    The stage boundaries for the opponent curriculum (random -> pool -> self-play) are set as fractions of the training budget, chosen by the authors.
  • Reward shaping coefficients = e.g., 3x for gin vs knock
    The gin-to-knock payoff ratios and deadwood-reduction bonus magnitudes are hand-set dials explored in the reward sweep.
  • Evaluation frequency for keep-the-best = every 1 million steps
    The frequency at which the agent is evaluated against the hardest reference during training for checkpoint saving is a chosen parameter.
  • Illegal action logit fill value = large finite negative (exact value not stated)
    Chosen to avoid NaNs in TRPO's conjugate gradient while masking illegal actions.
axioms (4)
  • domain assumption A fixed, strong, deterministic heuristic expert is a valid yardstick for measuring agent progress in an imperfect-information game, even if it is not a game-theoretic optimum.
    Invoked in 'The Gold-Standard Expert' section: 'what matters is that it is strong, reproducible, and cheap, so it makes a fair yardstick.'
  • domain assumption The single-agent reduction (wrapper steps opponent between learner's decisions, rotates seats) preserves the strategic structure of the two-player game sufficiently for learning.
    Stated in 'Problem Setup': 'This reduction is what lets us swap in any opponent... without changing the learner.'
  • standard math Potential-based reward shaping theory (Ng et al. 1999) correctly predicts which shaped rewards will help or hurt in this domain.
    Used in 'Training Methods Studied' to interpret reward shaping results.
  • domain assumption Causal confusion (de Haan et al. 2019) is the correct framework for explaining DAgger's failure here.
    Invoked in 'Discussion' to explain why imitation learning collapsed.

pith-pipeline@v1.1.0-glm · 17914 in / 2632 out tokens · 248488 ms · 2026-07-10T00:09:25.809070+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning agents for imperfect-information card games are only as strong as the opponents they train against, and they are hard to grade, since they beat a random opponent over 99 percent of the time and only tie copies of themselves. So we build a strong, fixed, rule-based expert for Gin Rummy and use it only as a yardstick, never for training. It beats every agent we trained 70 to 99 percent of the time. Across more than a hundred runs, we isolate what makes a lightweight agent stronger. Trust region updates, a well-aimed reward, a curriculum of tougher opponents, warm starting, and keeping the best checkpoint all help, and stacking them lifts a self-play champion from about 30 to 36 percent against the expert. Several ideas did not pay off. Short-term and longer-term reward shaping, learned state embeddings, imitation and DAgger, and a live large language model opponent were each unhelpful, too slow, or too heavy to train at scale. Comparing MLP, convolutional, set-based, attention, and recurrent encoders shows that extra capacity does little to break the ceiling, suggesting the limit is information rather than network size. We add standard baselines (neural fictitious self-play and information set Monte Carlo search) and confirm the approach carries over to Leduc Hold'em, where the optimum is computable. The result is a lightweight, game-agnostic recipe that trains competitive agents without training on the expert, for any game a small model can handle, reported with robust statistics and released as a reusable package.

Figures

Figures reproduced from arXiv: 2607.06854 by Mahdi Salmani, Mohammadsaeed Haghi, Nima Kelidari.

Figure 1
Figure 1. Figure 1: System overview. A masked PPO or TRPO learner trains against a rising curriculum of opponents (random play, a [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The fixed expert beats every agent we trained (left) yet gins in under two percent of games (right). It wins by knocking [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Left: TRPO beats PPO against every opponent. Right: the gin rate stays under one percent for every reward design [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Win-rate against the champion as training moves through curriculum stages. The climb is real, but the late dip is why [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (a) Win-rate against the fixed expert by network architecture (IQM with [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages · 6 internal anchors

  1. [1]

    Meta Fundamental AI Research Diplomacy Team (FAIR); Bakhtin, A.; Brown, N.; Dinan, E.; Farina, G.; Flaherty, C.; Fried, D.; Goff, A.; Gray, J.; Hu, H.; et al. 2022. Human-Level Play in the Game of Diplomacy by Combining Language Models with Strategic Reasoning. Science 378(6624): 1067-1074

    work page 2022

  2. [2]

    S.; Courville, A

    Agarwal, R.; Schwarzer, M.; Castro, P. S.; Courville, A. C.; and Bellemare, M. G. 2021. Deep Reinforcement Learning at the Edge of the Statistical Precipice. In Advances in Neural Information Processing Systems (NeurIPS) 34

    work page 2021

  3. [3]

    Bengio, Y.; Louradour, J.; Collobert, R.; and Weston, J. 2009. Curriculum Learning. In Proceedings of the 26th International Conference on Machine Learning (ICML), 41-48

    work page 2009

  4. [4]

    Berner, C.; Brockman, G.; Chan, B.; Cheung, V.; Debiak, P.; Dennison, C.; Farhi, D.; Fischer, Q.; Hashme, S.; Hesse, C.; et al. 2019. Dota 2 with Large Scale Deep Reinforcement Learning. arXiv:1912.06680

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  5. [5]

    Brown, N.; and Sandholm, T. 2018. Superhuman AI for Heads-up No-Limit Poker: Libratus Beats Top Professionals. Science 359(6374): 418-424

    work page 2018

  6. [6]

    Brown, N.; and Sandholm, T. 2019. Superhuman AI for Multiplayer Poker. Science 365(6456): 885-890

    work page 2019

  7. [7]

    Brown, N.; Lerer, A.; Gross, S.; and Sandholm, T. 2019. Deep Counterfactual Regret Minimization. In Proceedings of the 36th International Conference on Machine Learning (ICML), 793-802

    work page 2019

  8. [8]

    I.; Powley, E

    Cowling, P. I.; Powley, E. J.; and Whitehouse, D. 2012. Information Set Monte Carlo Tree Search. IEEE Transactions on Computational Intelligence and AI in Games 4(2): 120-143

    work page 2012

  9. [9]

    de Haan, P.; Jayaraman, D.; and Levine, S. 2019. Causal Confusion in Imitation Learning. In Advances in Neural Information Processing Systems (NeurIPS) 32

    work page 2019

  10. [10]

    Henderson, P.; Islam, R.; Bachman, P.; Pineau, J.; Precup, D.; and Meger, D. 2018. Deep Reinforcement Learning That Matters. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 3207-3214

    work page 2018

  11. [11]

    Hochreiter, S.; and Schmidhuber, J. 1997. Long Short-Term Memory. Neural Computation 9(8): 1735-1780

    work page 1997

  12. [12]

    Heinrich, J.; and Silver, D. 2016. Deep Reinforcement Learning from Self-Play in Imperfect-Information Games. arXiv:1603.01121

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  13. [13]

    Huang, S.; and Onta\ n \' o n, S. 2022. A Closer Look at Invalid Action Masking in Policy Gradient Algorithms. In Proceedings of the 35th International FLAIRS Conference

    work page 2022

  14. [14]

    Kotnik, C.; and Kalita, J. 2003. The Significance of Temporal-Difference Learning in Self-Play Training: TD-rummy versus EVO-rummy. In Proceedings of the 20th International Conference on Machine Learning (ICML), 369-375

    work page 2003

  15. [15]

    H.; Gonzalez, J

    Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP)

    work page 2023

  16. [16]

    Lanctot, M.; Lockhart, E.; Lespiau, J.-B.; Zambaldi, V.; Upadhyay, S.; P\' e rolat, J.; Srinivasan, S.; Timbers, F.; Tuyls, K.; Omidshafiei, S.; et al. 2019. OpenSpiel: A Framework for Reinforcement Learning in Games. arXiv:1908.09453

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  17. [17]

    Li, J.; Koyamada, S.; Ye, Q.; Liu, G.; Wang, C.; Yang, R.; Zhao, L.; Qin, T.; Liu, T.-Y.; and Hon, H.-W. 2020. Suphx: Mastering Mahjong with Deep Reinforcement Learning. arXiv:2003.13590

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  18. [18]

    R.; Sturtevant, N

    Long, J. R.; Sturtevant, N. R.; Buro, M.; and Furtak, T. 2010. Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI), 134-140

    work page 2010

  19. [19]

    Nagai, M.; Shrivastava, K.; Ta, K.; Bogaerts, S.; and Byers, C. 2021. A Highly-Parameterized Ensemble to Play Gin Rummy. Proceedings of the AAAI Conference on Artificial Intelligence 35(17): 15614-15621 (EAAI Symposium)

    work page 2021

  20. [20]

    E.; and Stone, P

    Narvekar, S.; Peng, B.; Leonetti, M.; Sinapov, J.; Taylor, M. E.; and Stone, P. 2020. Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey. Journal of Machine Learning Research 21(181): 1-50

    work page 2020

  21. [21]

    Y.; Harada, D.; and Russell, S

    Ng, A. Y.; Harada, D.; and Russell, S. 1999. Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping. In Proceedings of the 16th International Conference on Machine Learning (ICML), 278-287

    work page 1999

  22. [22]

    Qwen Team. 2024. Qwen2.5 Technical Report. arXiv:2412.15115

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  23. [23]

    Raffin, A.; Hill, A.; Gleave, A.; Kanervisto, A.; Ernestus, M.; and Dormann, N. 2021. Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research 22(268): 1-8

    work page 2021

  24. [24]

    J.; and Bagnell, J

    Ross, S.; Gordon, G. J.; and Bagnell, J. A. 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), 627-635

    work page 2011

  25. [25]

    Schulman, J.; Levine, S.; Abbeel, P.; Jordan, M.; and Moritz, P. 2015. Trust Region Policy Optimization. In Proceedings of the 32nd International Conference on Machine Learning (ICML), 1889-1897

    work page 2015

  26. [26]

    Schulman, J.; Moritz, P.; Levine, S.; Jordan, M.; and Abbeel, P. 2016. High-Dimensional Continuous Control Using Generalized Advantage Estimation. In International Conference on Learning Representations (ICLR)

    work page 2016

  27. [27]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  28. [28]

    Silver, D.; Schrittwieser, J.; Simonyan, K.; Antonoglou, I.; Huang, A.; Guez, A.; Hubert, T.; Baker, L.; Lai, M.; Bolton, A.; et al. 2017. Mastering the Game of Go without Human Knowledge. Nature 550(7676): 354-359

    work page 2017

  29. [29]

    Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; et al. 2018. A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play. Science 362(6419): 1140-1144

    work page 2018

  30. [30]

    K.; Black, B.; Grammel, N.; Jayakumar, M.; Hari, A.; Sullivan, R.; Santos, L.; Perez, R.; Horsch, C.; Dieffendahl, C.; et al

    Terry, J. K.; Black, B.; Grammel, N.; Jayakumar, M.; Hari, A.; Sullivan, R.; Santos, L.; Perez, R.; Horsch, C.; Dieffendahl, C.; et al. 2021. PettingZoo: Gym for Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems (NeurIPS) 34

    work page 2021

  31. [31]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems (NeurIPS) 30

    work page 2017

  32. [32]

    M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D

    Vinyals, O.; Babuschkin, I.; Czarnecki, W. M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D. H.; Powell, R.; Ewalds, T.; Georgiev, P.; et al. 2019. Grandmaster Level in StarCraft II using Multi-Agent Reinforcement Learning. Nature 575(7782): 350-354

    work page 2019

  33. [33]

    Zaheer, M.; Kottur, S.; Ravanbakhsh, S.; P\' o czos, B.; Salakhutdinov, R.; and Smola, A. J. 2017. Deep Sets. In Advances in Neural Information Processing Systems (NeurIPS) 30

    work page 2017

  34. [34]

    Zha, D.; Lai, K.-H.; Huang, S.; Cao, Y.; Reddy, K.; Vargas, J.; Nguyen, A.; Wei, R.; Guo, J.; and Hu, X. 2020. RLCard: A Platform for Reinforcement Learning in Card Games. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI), Demo Track

    work page 2020

  35. [35]

    Zha, D.; Xie, J.; Ma, W.; Zhang, S.; Lian, X.; Hu, X.; and Liu, J. 2021. DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning. In Proceedings of the 38th International Conference on Machine Learning (ICML), 12333-12344

    work page 2021

  36. [36]

    Zinkevich, M.; Johanson, M.; Bowling, M.; and Piccione, C. 2007. Regret Minimization in Games with Incomplete Information. In Advances in Neural Information Processing Systems (NeurIPS) 20

    work page 2007