Pith. sign in

REVIEW 6 minor 46 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

T0 review · glm-5.2

Fighting game stripped to rock-paper-scissors core becomes RL testbed

2026-07-08 03:05 UTC pith:36ENTQCS

load-bearing objection Solid benchmark paper with honest baselines and genuine research findings; deserves a serious referee.

arxiv 2607.06514 v1 pith:36ENTQCS submitted 2026-07-07 cs.AI cs.GT

FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games

classification cs.AI cs.GT
keywords gameenvironmentfightingfootsiesgymbenchmarkenablesimperfect-informationlearning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces FootsiesGym, an open-source reinforcement learning environment built on HiFight's minimalist 2D fighting game Footsies. The authors argue that fighting games occupy a useful middle ground between simple matrix-game benchmarks (which expose cyclic mixed-strategy structure but lack real-time dynamics) and large real-time environments like StarCraft II (which are rich but prohibitively expensive to train). By stripping out combos and keeping only the neutral game — the phase where neither player has an advantage and both maneuver for openings — FootsiesGym isolates the cyclic, non-transitive strategic interactions that make fighting games interesting while remaining trainable on a single workstation at roughly 52,500 environment steps per second. The authors support this claim with a vectorized headless simulator, throughput measurements, baseline experiments across four algorithms (two PPO variants, EMAgnet, and PFSP), and identification of concrete research challenges: standard algorithms fail to discover special attacks that require extended action sequences, and policies that win head-to-head can still be highly exploitable, revealing a disconnect between skill and robustness that simple win-rate metrics miss.

Core claim

The central object is the neutral game of Footsies — the spacing-and-timing phase of a fighting match where neither player holds an advantage and both must commit to mixed strategies over movement and attacks that counter each other in cycles, like rock-paper-scissors. The paper shows that this structure can be isolated from the rest of the fighting-game genre (combos, execution contests, multiple characters) and packaged into a tractable benchmark that is simultaneously real-time, spatial, imperfect-information, and non-transitive. The baseline experiments then surface two concrete pathologies: entropy regularization and standard training methods drive hard-to-discover but strategically重要s如

What carries the argument

The environment's architecture rests on several load-bearing design choices: (1) the neutral-game isolation that removes combos to preserve only cyclic strategic interactions, (2) an action-delay parameter that controls whether attacks are reactable or must be predicted, effectively tuning the game from pure reaction to pure prediction, (3) an 85-dimensional observation vector with a privileged self-block that creates information asymmetry mirroring two players at separate controllers, (4) a vectorized C# headless simulator with gRPC ports achieving ~52,500 steps/second, and (5) approximate exploitability measured by training PPO best responses against fixed checkpoints, which serves as a (n

Load-bearing premise

The paper assumes that Footsies' neutral game — with a single character, four attacks, no combos, and no health bar — is representative enough of imperfect-information game dynamics to serve as a meaningful benchmark for the broader field, despite the authors' own acknowledgment that a substantial complexity gap separates it from commercial fighting games.

What would settle it

Train multiple algorithms on FootsiesGym and on existing benchmarks (e.g., Leduc poker, Slime Volleyball); if the algorithms produce qualitatively identical failure modes, learning dynamics, and relative rankings in both, the environment adds no distinguishing research value beyond what simpler benchmarks already provide.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 6 minor

Summary. This paper introduces FootsiesGym, an open-source reinforcement learning environment built on HiFight's minimalist 2D fighting game Footsies. The environment is designed to capture the cyclic, non-transitive, imperfect-information dynamics of fighting-game neutral play while remaining tractable enough to train on a single workstation. The authors describe the game mechanics, observation space, action space, and reward structure; provide throughput measurements; and benchmark four baseline algorithms (two PPO variants, EMAgnet, and PFSP). They report approximate exploitability via trained best responses, identify a skill-exploitability disconnect, and highlight the difficulty of discovering special attacks through random exploration. The code is publicly available.

Significance. The paper fills a genuine gap between small matrix-game benchmarks and large real-time environments. The environment has verifiable imperfect-information structure (hidden opponent special-charge progress and dash availability; action-delay-induced prediction requirements), measured throughput (~52,500 steps/s at P=4, K=128), and publicly available code. The identification of concrete research challenges—special-attack discovery failure under standard regularization, and the skill-exploitability disconnect—adds scientific value beyond a pure software release. The authors are commendably transparent about limitations: baselines are explicitly labeled as naïvely tuned, and approximate exploitability is correctly described as a lower bound on true exploitability.

minor comments (6)
  1. §3, Observations: The text states the observation is 85-dimensional (88 with special-charge option), but Table 2 lists 1 (Common) + 2×18 (Shared) + 6 (Privileged, with |A|=6) = 43 dimensions, not 85. The discrepancy likely arises from additional features not listed or a different counting method; this should be clarified so users can correctly reproduce the observation vector.
  2. §4, Approximate Exploitability: The best-response PPO agents are trained for 2.5×10^7 steps, the same budget as the original policies. It would strengthen the approximate exploitability claims to briefly justify why this budget is sufficient for a meaningful best response, or to note that the lower bound may be loose if the best-response training is underpowered.
  3. Figure 5a: The head-to-head matrix uses expected return, but the reward scale depends on win_reward_scaling_coeff (default w=10). Stating the return scale or converting to win-rate would improve interpretability for readers unfamiliar with the reward structure.
  4. §C.3, Eq. (3): The PFSP priority weight uses exponent 1/2, while Vinyals et al. (2019) use this formulation in a league context. A brief note on how the single-policy reservoir affects the prioritization dynamics (e.g., reduced opponent diversity) would help readers assess the approach's fidelity to the original method.
  5. Table 6: The frame data lists four attacks, but the text in §3 mentions 'four attacks' without explicitly naming them in the main body. Adding a brief cross-reference to Table 6 in §3 would improve readability.
  6. §4, Discovering Special Attacks: The probability of B_SPECIAL under uniform random play is stated as 'roughly 0.001%'. A brief derivation (e.g., (1/6)^15 × conditional) would make this claim verifiable.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their careful and accurate summary of the paper, and for the recommendation of minor revision. The referee's assessment aligns with our intended contributions: filling the gap between small matrix-game benchmarks and large real-time environments, providing verifiable imperfect-information structure, and identifying concrete research challenges (special-attack discovery failure and the skill-exploitability disconnect). We note that the 'MAJOR COMMENTS' section of the report appears to be empty, so we address the points raised in the summary and significance sections.

read point-by-point responses
  1. Referee: The referee's summary and significance sections accurately characterize the paper's contributions, including the environment design, throughput measurements, baseline benchmarks, approximate exploitability analysis, and identification of research challenges. No specific major comments were listed.

    Authors: We appreciate the referee's thorough and accurate reading of the manuscript. We confirm that the referee's summary correctly captures the paper's structure and contributions. Since no specific major comments were provided, we have reviewed the manuscript for any clarifications that would strengthen it in light of the referee's positive assessment. We will make the following minor revisions: (1) We will add a brief note in the Discussion section explicitly connecting the skill-exploitability disconnect to the broader literature on evaluation in imperfect-information games, making the research challenge more actionable for future work. (2) We will clarify in the Limitations paragraph that the approximate exploitability lower bound applies both to the absolute exploitability values and to the pairwise comparisons between algorithms, so readers do not over-interpret the statistical tests. (3) We will ensure the special-attack discovery challenge is framed consistently with the Appendix E results showing that the special-charge action mode alleviates but does not fully solve the problem. These are clarifications only; no structural changes are needed. revision: yes

Circularity Check

0 steps flagged

No significant circularity. Benchmark paper with no derivation or prediction to be circular about.

full rationale

This is a benchmark/environment paper, not a derivation or prediction paper. The central claim — that FootsiesGym is a tractable, open-source fighting-game environment capturing cyclic, non-transitive, imperfect-information dynamics — is supported by: (1) the external Footsies game by HiFight (2018) as the design basis, (2) direct throughput measurements (Figure 2), (3) frame-data analysis showing attack counter-relationships (Table 6, Figure 8), and (4) baseline experiments across four algorithms with honest caveats about naive tuning and approximate exploitability. The EMAgnet citation (Maidment et al., 2026) shares co-authors with this paper, but EMAgnet is used as one of four baseline algorithms, not as a load-bearing premise for the environment's value or design. The environment's strategic structure is grounded in the external game's mechanics, not in a self-cited theoretical result. No equation, prediction, or derivation reduces to its own inputs by construction. The approximate exploitability methodology (training best responses) is a standard approach cited to Timbers et al. (2020) and others, not a self-cited method. The skill-exploitability disconnect is presented as an empirical finding, not a derived prediction. No circularity patterns (self-definitional, fitted-input-as-prediction, self-citation load-bearing, uniqueness imported, ansatz smuggled, or renaming) are present.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

No new entities invented.

free parameters (4)
  • win_reward_scaling_coeff (w) = 10.0
    Terminal win/loss reward magnitude; default design choice, not fitted to data.
  • action_delay = 8
    Number of frames between action selection and execution; chosen to make attacks partially unreactable. Not fitted but a design choice that shapes the strategic regime.
  • frame_skip = 4
    Game frames per environment step; controls temporal resolution. Design choice.
  • max_t = 4000
    Truncation horizon in environment steps. Design choice.
axioms (3)
  • domain assumption Footsies' neutral game (without combos) preserves the essential cyclic, non-transitive strategic structure of fighting games.
    Invoked in §1 and §3 to justify the environment's design: 'By foregoing the latter [combos], Footsies isolates the neutral game while preserving its cyclic strategic interactions.' This is the foundational premise for the benchmark's relevance.
  • domain assumption Approximate exploitability via PPO-trained best responses is a meaningful proxy for true exploitability.
    Used in §4 to compare algorithms. The authors acknowledge this is only a lower bound (citing Timbers et al., 2020), but the experimental conclusions (e.g., EMAgnet is most exploitable) depend on this proxy being informative.
  • domain assumption A 2-layer MLP with 256 hidden units is a sufficient function class for policies in Footsies.
    Used for all four baseline algorithms (Appendix C). If the architecture is too small to represent competent policies, baseline comparisons may be uninformative.

pith-pipeline@v1.1.0-glm · 12807 in / 3941 out tokens · 326661 ms · 2026-07-08T03:05:48.451768+00:00 · methodology

0 comments
read the original abstract

We present FootsiesGym, an open-source environment for learning in a non-trivial two-player, zero-sum, imperfect-information game. Built on HiFight's minimalist 2D fighting game Footsies, it isolates the cyclic, non-transitive strategic interactions of fighting game neutral play while remaining simple enough for efficient analysis. We provide a vectorized simulator that enables high-throughput training on standard hardware, making the environment accessible and reproducible. We describe the design of the environment, benchmark several reinforcement learning algorithms, and discuss open research directions it enables. The code is available at https://github.com/como-research/FootsiesGym.

Figures

Figures reproduced from arXiv: 2607.06514 by Chase McDonald, Nathan Tsang, Wesley N. Kerr.

Figure 1
Figure 1. Figure 1: A screenshot of the Footsies game, from HiFight (2018). Two characters fight one another using a combination of movement, quick attacks, and special attacks. A player wins when they K.O. the other. as Kuhn Poker, Goofspiel, Leduc Poker, Phantom Tic-Tac-Toe, and Dark Hex. While these environ￾ments are valuable for exact exploitability calculations and equilibrium analysis, they capture little of the complex… view at source ↗
Figure 2
Figure 2. Figure 2: Aggregate environment steps per second as a function of the number of in-process parallel [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: PPO, PPO (Sched.), EMAgnet, and PFSP baselines evaluated against two heuristic oppo [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Results of training approximate best responses for all ( [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (a) Head-to-head and (b) best response returns for all policies. PFSP has a positive return against all other algorithms. EMAgnet has a positive return against both PPO variants, but the best responses trained against it consistently attain the highest returns. In (a), each cell shows the expected return to the row policies when playing against the column policies (mean ± SEM). Each seed of the row algorit… view at source ↗
Figure 6
Figure 6. Figure 6: Usage of the directional special attack ( [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: PPO self-play policies trained with action_delay ∈ {0, 12}, evaluated against a ran￾dom opponent (a) and a no-op opponent (b). Curves are aggregated over 5 seeds; the line is the mean and the shaded band is the 95% confidence interval. The impact of action delay manipulations is inherently tied to the frame data in Footsies. Whether a move is reactable or must be predicted is determined by the length of it… view at source ↗
Figure 8
Figure 8. Figure 8: Reactability of each attack across action delays [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparison of PPO (Sched.) training with and without special charge actions enabled. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages · 15 internal anchors

  1. [1]

    Nature , volume=

    Outracing champion Gran Turismo drivers with deep reinforcement learning , author=. Nature , volume=. 2022 , publisher=

  2. [2]

    CoGrid & the Multi-User Gymnasium: A Framework for Multi-Agent Experimentation

    CoGrid & the Multi-User Gymnasium: A Framework for Multi-Agent Experimentation , author=. arXiv preprint arXiv:2604.15044 , year=

  3. [3]

    nature , volume=

    Grandmaster level in StarCraft II using multi-agent reinforcement learning , author=. nature , volume=. 2019 , publisher=

  4. [4]

    Science , volume=

    Mastering the game of Stratego with model-free multiagent reinforcement learning , author=. Science , volume=. 2022 , publisher=

  5. [5]

    A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games

    A unified approach to reinforcement learning, quantal response equilibria, and two-player zero-sum games , author=. arXiv preprint arXiv:2206.05825 , year=

  6. [6]

    Advances in neural information processing systems , volume=

    A unified game-theoretic approach to multiagent reinforcement learning , author=. Advances in neural information processing systems , volume=

  7. [7]

    Reevaluating Policy Gradient Methods for Imperfect-Information Games

    Reevaluating policy gradient methods for imperfect-information games , author=. arXiv preprint arXiv:2502.08938 , year=

  8. [8]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Mastering complex control in moba games with deep reinforcement learning , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  9. [9]

    International conference on machine learning , pages=

    RLlib: Abstractions for distributed reinforcement learning , author=. International conference on machine learning , pages=. 2018 , organization=

  10. [10]

    Advances in neural information processing systems , volume=

    Unifying count-based exploration and intrinsic motivation , author=. Advances in neural information processing systems , volume=

  11. [11]

    Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits

    Exponential moving average of weights in deep learning: Dynamics and benefits , author=. arXiv preprint arXiv:2411.18704 , year=

  12. [12]

    EMAgnet: Parameter-Space EMA Regularization for Policy Gradient Self-Play in Large Games

    EMAgnet: Parameter-Space EMA Regularization for Policy Gradient Self-Play in Large Games , author=. arXiv preprint arXiv:2606.23995 , year=

  13. [13]

    International conference on machine learning , pages=

    Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures , author=. International conference on machine learning , pages=. 2018 , organization=

  14. [14]

    Goal-Conditioned Reinforcement Learning: Problems and Solutions

    Goal-conditioned reinforcement learning: Problems and solutions , author=. arXiv preprint arXiv:2201.08299 , year=

  15. [15]

    Approximate exploitability: Learning a best response in large games

    Approximate exploitability: Learning a best response in large games , author=. arXiv preprint arXiv:2004.09677 , year=

  16. [16]

    Advances in neural information processing systems , volume=

    Collaborating with humans without human data , author=. Advances in neural information processing systems , volume=

  17. [17]

    Other-play

    “Other-play” for zero-shot coordination , author=. International conference on machine learning , pages=. 2020 , organization=

  18. [18]

    International Conference on Learning Representations , volume=

    Overcookedv2: Rethinking overcooked for zero-shot coordination , author=. International Conference on Learning Representations , volume=

  19. [19]

    Any-Play: An Intrinsic Augmentation for Zero-Shot Coordination

    Any-play: An intrinsic augmentation for zero-shot coordination , author=. arXiv preprint arXiv:2201.12436 , year=

  20. [20]

    Advances in neural information processing systems , volume=

    \# exploration: A study of count-based exploration for deep reinforcement learning , author=. Advances in neural information processing systems , volume=

  21. [21]

    Proximal Policy Optimization Algorithms

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

  22. [22]

    IMPACT: Importance Weighted Asynchronous Architectures with Clipped Target Networks

    Impact: Importance weighted asynchronous architectures with clipped target networks , author=. arXiv preprint arXiv:1912.00167 , year=

  23. [23]

    International conference on machine learning , pages=

    Count-based exploration with neural density models , author=. International conference on machine learning , pages=. 2017 , organization=

  24. [24]

    nature , volume=

    Mastering the game of Go with deep neural networks and tree search , author=. nature , volume=. 2016 , publisher=

  25. [25]

    Controllable Complementarity: Subjective Preferences in Human-AI Collaboration

    Controllable Complementarity: Subjective Preferences in Human-AI Collaboration , author=. arXiv preprint arXiv:2503.05455 , year=

  26. [26]

    Neural Computation MIT-Press , year=

    Long short-term memory , author=. Neural Computation MIT-Press , year=

  27. [27]

    Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

    Deep reinforcement learning from self-play in imperfect-information games , author=. arXiv preprint arXiv:1603.01121 , year=

  28. [28]

    Palmas, Alessandro , journal =

  29. [29]

    Unity: A General Platform for Intelligent Agents

    Unity: A general platform for intelligent agents , author =. arXiv preprint arXiv:1809.02627 , year =

  30. [30]

    Comparison of Reaction Time Between eSports Players of Different Genres and Sportsmen , journal =

    Bickmann, Peter and Wechsler, Konstantin and Rudolf, Kevin and Tholl, Chuck and Frob. Comparison of Reaction Time Between eSports Players of Different Genres and Sportsmen , journal =. 2021 , volume =. doi:10.4018/IJER.20210101.oa1 , url =

  31. [31]

    and Black, Benjamin and Grammel, Nathaniel and Jayakumar, Mario and Hari, Ananth and Sullivan, Ryan and Santos, Luis S

    Terry, J. and Black, Benjamin and Grammel, Nathaniel and Jayakumar, Mario and Hari, Ananth and Sullivan, Ryan and Santos, Luis S. and Dieffendahl, Clemens and Horsch, Caroline and Perez-Vicente, Rodrigo and Williams, Niall L. and Lokesh, Yashas and Ravi, Praveen , journal =

  32. [32]

    OpenSpiel: A Framework for Reinforcement Learning in Games

    Lanctot, Marc and Lockhart, Edward and Lespiau, Jean-Baptiste and Zambaldi, Vinicius and Upadhyay, Satyaki and P. arXiv preprint arXiv:1908.09453 , year =

  33. [33]

    2020 , howpublished =

    Slime Volleyball Gym Environment , author =. 2020 , howpublished =

  34. [34]

    2020 , howpublished =

    Derk's Gym , author =. 2020 , howpublished =

  35. [35]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Kurach, Karol and Raichuk, Anton and Sta. Proceedings of the AAAI Conference on Artificial Intelligence , volume =

  36. [36]

    EvoJAX: Hardware-Accelerated Neuroevolution

    EvoJAX: Hardware-Accelerated Neuroevolution , author=. arXiv preprint arXiv:2202.05008 , year=

  37. [37]

    Advances in neural information processing systems , volume=

    Pipeline psro: A scalable approach for finding approximate nash equilibria in large games , author=. Advances in neural information processing systems , volume=

  38. [38]

    Suarez, Joseph and Du, Yilun and Isola, Phillip and Mordatch, Igor , journal =. Neural

  39. [39]

    Gotta Learn Fast: A New Benchmark for Generalization in

    Nichol, Alex and Pfau, Vicki and Hesse, Christopher and Klimov, Oleg and Schulman, John , journal =. Gotta Learn Fast: A New Benchmark for Generalization in

  40. [40]

    2022 IEEE Conference on Games (CoG) , pages=

    Darefightingice competition: A fighting game sound design and ai competition , author=. 2022 IEEE Conference on Games (CoG) , pages=. 2022 , organization=

  41. [41]

    2013 IEEE 2nd Global Conference on Consumer Electronics (GCCE) , pages=

    Fighting game artificial intelligence competition platform , author=. 2013 IEEE 2nd Global Conference on Consumer Electronics (GCCE) , pages=. 2013 , organization=

  42. [42]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Li, Wenzhe and Ding, Zihan and Karten, Seth and Jin, Chi , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

  43. [43]

    Creating Pro-Level

    Oh, Inseok and Rho, Seungeun and Moon, Sangbin and Son, Seongho and Lee, Hyoil and Chung, Jinyun , journal =. Creating Pro-Level. 2022 , publisher =

  44. [44]

    and Tenenbaum, Joshua B

    Firoiu, Vlad and Whitney, William F. and Tenenbaum, Joshua B. , journal =. Beating the World's Best at

  45. [45]

    IEEE Transactions on Games , volume=

    Winning is not everything: Enhancing game development with intelligent agents , author=. IEEE Transactions on Games , volume=. 2020 , publisher=

  46. [46]

    2018 IEEE Conference on Computational Intelligence and Games (CIG) , pages =

    Human-Like Playtesting with Deep Learning , author =. 2018 IEEE Conference on Computational Intelligence and Games (CIG) , pages =. 2018 , organization =