Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

An LLM's strategic reasoning can be scored by a fitted cognitive-hierarchy level that stays stable across opponents, rises with memory, and falls when it chats before deciding.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

CHBench fits Level-K and Poisson cognitive hierarchy models to LLM game play and uses the fitted reasoning level as a benchmark score.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Fresh idea and real data, but the central metric is unidentifiable and the robustness claims are in-sample artifacts; worth a rigorous review, not publication as is. the 4 major comments →

arxiv 2508.11944 v1 pith:APFOEU6I submitted 2025-08-16 cs.AI cs.CLcs.HC

CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs

Hongtao Liu , Zhicheng Du , Zihe Wang , Weiran Shen This is my paper
classification cs.AI cs.CLcs.HC
keywords cognitive hierarchystrategic reasoningLLM evaluationnormal-form gamesbounded rationalitymaximum likelihood estimationlevel-k reasoningbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the right way to measure an LLM's strategic reasoning is to fit a cognitive hierarchy model — a behavioral-economics model where agents reason at different depths — to the actions the model chooses in two-player normal-form games, and then read off the mean of the fitted level distribution. Across six LLMs, fifteen games, and 30 rounds of play, it reports that this fitted level barely moves when the opponent changes, while average payoffs swing substantially; this is the evidence that CHBench measures capability rather than matchup luck. The paper also claims that feeding the LLM a memory of prior rounds raises its measured level, that fuller memory helps more than partial memory, and that allowing the LLM to chat with an advisor beforehand lowers its measured level. A sympathetic reader would care because utility-based game benchmarks cannot separate skill from opponent behavior and game structure, while a stable fitted reasoning level would be a game-independent score.

Core claim

CHBench's central claim is that the strategic reasoning capability of an LLM can be quantified as the fitted mean of a cognitive hierarchy (CH) level distribution. In the model, level-0 agents play uniformly at random; a level-$k$ agent best-responds to opponents it believes to be at lower levels, with the Level-K variant best-responding only to level $k-1$ and the Poisson variant best-responding to all lower levels weighted by a Poisson distribution. The parameters are estimated by maximum likelihood from the action choices of six LLMs over 30 rounds in fifteen normal-form games, and the paper sets the maximum level to $\hat{k}=4$ because moving to level 5 gives negligible likelihood gains.

What carries the argument

The load-bearing object is the cognitive hierarchy model with iterative best response. Level-0 play is uniform random choice; each higher level $k$ computes a best response to a belief about lower-level opponents, and the agent's overall action distribution is a weighted mixture of the per-level strategies. The benchmark's output is the mean and variance of the fitted level distribution, which the paper treats as the agent's average strategic reasoning depth and as the stability of that depth across games and opponents.

Load-bearing premise

The whole measure rests on the cognitive hierarchy model — level-0 uniform randomness plus iterative best response to lower-level beliefs — being an adequate generative model of LLM action choices, so that the fitted mean level $k$ really is strategic depth and not some other regularity in the data.

What would settle it

Fit the two CH models on half of the fifteen games and compare their predictive likelihood on the other half against a model that predicts actions from payoff features alone (for instance, a logit choice rule on payoff differences) and against a model with freely varying per-opponent levels. If the payoff-only model predicts held-out behavior just as well, or if per-opponent fitted levels vary as much as payoffs do, the claim that the level is a stable opponent-independent measure of reasoning capability would be refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Payoff-based rankings of LLM agents can misrank them: the same model's utility depends heavily on who it faces, while its fitted cognitive level stays nearly constant.
  • The mean fitted level $k$ can serve as a game-independent and opponent-independent score for comparing LLMs in normal-form settings.
  • Prompt design changes the measured level: injecting historical rounds raises the fitted level, with full memory outperforming partial memory.
  • Conversational pre-play with an advisor lowers the measured level, so adding chat-based scaffolding can hide or suppress strategic capability.
  • A maximum level of $\hat{k}=4$ suffices to describe LLM behavior; allowing a fifth level adds little and destabilizes the fit.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the consistency claim is conditional on the CH model being the right generative account — the same stable fit could in principle be produced by a simpler payoff-driven choice model, in which case the fitted level would be a behavioral label rather than a direct window into reasoning depth.
  • A natural next test the authors do not run: train the two CH models on a subset of the fifteen games and score them on the rest, comparing predictive likelihood against a payoff-only baseline; strong held-out prediction would turn the reported consistency into a true generalization result.
  • The memory effect suggests a practical use for the fitted level as a cheap screening probe: measure whether prompt changes, instruction tuning, or fine-tuning shift the fitted level before running expensive utility-based evaluations.
  • Because the chat mechanism's negative effect is attributed to cooperative models adopting advisors' suggestions, one could test whether the effect shrinks when the advisor is explicitly adversarial or when models are instructed to disregard advice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CHBench, a three-phase framework that fits Level-K and Poisson cognitive hierarchy models to repeated-game action data from six LLMs across fifteen normal-form games, then uses the fitted mean cognitive level as a measure of strategic reasoning. The authors report that maximum level k=4 is sufficient, that fitted mean cognitive levels are stable across opponents, and that the Memory Mechanism enhances while the Chat Mechanism degrades strategic reasoning. The central measurement, however, is a fitted mixture weight obtained by maximum likelihood on the same data used for all conclusions, and no out-of-sample or identifiability validation is provided.

Significance. The motivation is strong: utility-based evaluation is confounded by opponent behavior and game structure, and cognitive hierarchy models offer a principled alternative. The paper also contributes a substantial behavioral dataset and an ablation of chat/memory mechanisms. If the fitted mean cognitive level were identifiable and predictive, this would be a useful benchmark. However, the manuscript does not establish identifiability, out-of-sample predictive validity, or statistical reliability; the presented evidence actually suggests adjacent level strategies coincide, making the headline comparisons unreliable. The contribution at present is a proposal rather than a validated benchmark.

major comments (4)
  1. [§4.1 / §2.1] The paper reports D_KL(s_i^3, s_i^4) ≈ 0 across LLMs, but this same observation destroys identifiability of the mixture model s_i(a) = Σ_k α_i^k s_i^k(a). If levels 3 and 4 have identical predicted strategies, the log-likelihood in §3.2 is flat along directions that shift probability mass between α_i^3 and α_i^4. The MLE (SLSQP with ten restarts, lowest loss) then returns an arbitrary point on that flat surface, so the fitted mean k (Section 3.3) is arbitrary. Consequently, the low variance of mean k across opponents in Tables 3 and 4 may reflect optimizer noise, not stable strategic depth.
  2. [§3.3 / §4.2] The 'predictions' are in-sample fitted values. The likelihood L in §3.2 is optimized on the same action data D_G used to report level distributions, robustness, and mechanism effects. There is no held-out game, no held-out model/hyperparameter split, no cross-validation, and no comparison against alternative generative models (e.g., quantal response, Nash, level-0). The claim that CHBench 'robustly evaluates strategic reasoning capability while remaining resilient to variations in opponent strategies' is therefore an in-sample summary, not an evaluation of generalization.
  3. [§4.3 / Tables 3-5] Mechanism comparisons rest on point estimates from the lowest-loss MLE over ten restarts, with no confidence intervals, significance tests, or repeated-seed variability. Several differences are opposite to the stated conclusion: DeepSeek-chat's mean k increases from 0.6639 (baseline) to 0.9490 (Chat) in Table 3 and from 0.7139 to 1.0506 in Table 4, contradicting the claim that the Chat Mechanism 'exerts a negative impact.' The claim that Memory 'consistently yields the most substantial enhancement' is not supported by the reported statistics.
  4. [§2 / §3.2] The validity of mean k as a cardinal measure presupposes that the CH model with level-0 uniform randomization and iterative best response is the true generative model for LLM behavior. The paper provides no goodness-of-fit diagnostics, no posterior predictive checks, and no evidence that level-specific strategies are behaviorally distinguishable (indeed §4.1 shows the opposite for levels 3 and 4). Without an independent anchor for the level scale, the numbers in Tables 3-5 are uninterpretable as capability measures.
minor comments (6)
  1. [§4.1] The sentence 'Thus, we set k=4 across all experiments. whereas the k=5 already exhibits overfitting.' appears twice, and the first occurrence is a fragment.
  2. [Appendix A.2, Table 6] Games #11 and #12 appear to have identical payoff matrices. If so, the '15 carefully selected games' reduce to 14 unique instances, which should be acknowledged or corrected.
  3. [Abstract] The first sentence is a grammatical fragment: 'While most existing studies rely on utility performance metrics, which are not robust enough due to variations in opponent behavior and game structure.'
  4. [§3.1] 'Training Dataset' is a misnomer because there is no separate evaluation set; all reported results are on the same data used for fitting. Please clarify that the framework is not evaluated on unseen games.
  5. [Appendix A.4] The conclusion that reduced utility variance under Memory demonstrates improved strategic reasoning is asserted without a statistical test. Also, utility variance is not a direct measure of reasoning depth.
  6. [Tables 3-4] Bold/red/green formatting is not accompanied by effect sizes or statistical significance; the criterion for 'best LLM strategy reasoning capability' should be defined explicitly.

Circularity Check

2 steps flagged

Fitted level means are presented as predictions; near-identical level strategies make the reported mean level arbitrary.

specific steps
  1. fitted input called prediction [Section 3.2-3.3 (Optimization and Evaluation)]
    "With the optimized model parameters, for any given normal-form game with LLMs acting as the row player and column player respectively, we can employ cognitive hierarchy models to predict the potential level distributions of the two LLMs, as well as the specific strategy distributions at each level. These predicted results serve as the model’s assessment and prediction of the LLMs’ strategic reasoning capability."

    The parameters (including level weights α_i^k or Poisson λ_i) are estimated by maximum likelihood on the same dataset D_G (Section 3.2). The 'predicted level distributions' are therefore the fitted weights/functions of the fitted parameters, and the 'predicted strategies' are the likelihood terms themselves. The mean level k in Tables 3-5 is a fitted statistic, not an out-of-sample prediction. The robustness and Chat/Memory conclusions in Sections 4.2-4.3 are comparisons of these fitted statistics on the training data, so the claimed 'predictions' reduce to the fit by construction.

  2. other [Section 4.1 (Maximum Level k)]
    "The values of D_KL (s_i^3, s_i^4) are almost zero across all LLMs, demonstrating that the distributions s_i^3 and s_i^4 in LLMs are nearly identical."

    In the defining mixture s_i(a)=Σ_k α_i^k s_i^k(a), if s_i^3 = s_i^4, the likelihood is flat along directions that shift probability mass between levels 3 and 4. Consequently the MLE weights α_i^3 and α_i^4, and hence the reported mean level k, are not identified; SLSQP with random restarts returns an arbitrary point on a flat surface. The paper nevertheless uses these means to claim low variance across opponents (Sec. 4.2) and to compare mechanisms (Sec. 4.3). The reported stability is therefore an artifact of the optimizer's arbitrary selection rather than an identified property of the LLM, i.e., the central metric reduces to an arbitrary fit by construction.

full rationale

The CHBench evaluation is not validated against held-out data or alternative models; its core metric is the mean of the MLE-fitted level distribution, and the paper labels these fitted values as 'predictions.' All main conclusions (robustness, Chat degrades, Memory enhances) are comparisons of these fitted means on the same data, so there is a real in-sample circularity. The identifiability evidence (s3≈s4) further undermines the metric. However, the fitted means do depend on the data empirically; the paper does not simply assert its conclusions, and no self-citation chain is involved. Hence partial circularity: score 6.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The central claim rests on four fitted parameters (the per-level error rates and weights in Level-K, the Poisson rates, and the chosen maximum level) and on the untested adequacy of the cognitive hierarchy model as a description of LLM behavior. The mean level score is an invented latent quantity with no external anchor.

free parameters (4)
  • Level-K error rate epsilon_i^k = not reported
    Per-agent, per-level error rate in the Level-K CH model, fitted by MLE in Section 3.2.
  • Level-K level weight alpha_i^k = not reported
    Weight of each cognitive level in the agent's overall strategy, fitted by MLE and used to compute mean level.
  • Poisson CH rate lambda_i = not reported
    Per-agent Poisson rate determining the level distribution in the Poisson CH model, fitted by MLE.
  • Maximum level khat = 4
    Chosen by likelihood improvement and KL-divergence heuristics in Section 4.1, not independently grounded.
axioms (4)
  • domain assumption Agents' behavior is generated by a mixture of cognitive levels, each level best responding to lower levels; level 0 is uniform random.
    Core CH model assumption from behavioral economics, adopted in Section 2 and never tested against alternative generative models for LLMs.
  • domain assumption LLM action choices are exchangeable draws from a fixed mixture strategy over 30 rounds.
    The MLE objective in Section 3.2 treats each round as an independent draw from si, but chat and memory mechanisms make behavior history-dependent, violating stationarity.
  • ad hoc to paper The fitted mean cognitive level is a valid cardinal measure of strategic reasoning capability.
    Introduced in Section 3.3; no external validation ties the fitted level to actual strategic skill.
  • ad hoc to paper The chosen maximum level khat=4 is sufficient; higher levels add no explanatory power.
    Section 4.1 selects khat=4 with contradictory observations: Level-K shows no improvement from 3 to 4 while Poisson shows over 2% improvement, and no statistical test is provided.
invented entities (1)
  • Mean cognitive hierarchy level (mean k) of an LLM no independent evidence
    purpose: Serves as the benchmark score for strategic reasoning capability, replacing utility-based metrics.
    The latent level is inferred from the fitted mixture, not measured directly or validated against an independent task; its meaning depends entirely on model adequacy.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs." pith.science (2026). https://pith.science/paper/APFOEU6I

@misc{pith2026250811944,
  author       = {Pith},
  title        = {Pith review of: CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/APFOEU6I}},
  note         = {Machine review of arXiv:2508.11944}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Game-playing ability serves as an indicator for evaluating the strategic reasoning capability of large language models (LLMs). While most existing studies rely on utility performance metrics, which are not robust enough due to variations in opponent behavior and game structure. To address this limitation, we propose \textbf{Cognitive Hierarchy Benchmark (CHBench)}, a novel evaluation framework inspired by the cognitive hierarchy models from behavioral economics. We hypothesize that agents have bounded rationality -- different agents behave at varying reasoning depths/levels. We evaluate LLMs' strategic reasoning through a three-phase systematic framework, utilizing behavioral data from six state-of-the-art LLMs across fifteen carefully selected normal-form games. Experiments show that LLMs exhibit consistent strategic reasoning levels across diverse opponents, confirming the framework's robustness and generalization capability. We also analyze the effects of two key mechanisms (Chat Mechanism and Memory Mechanism) on strategic reasoning performance. Results indicate that the Chat Mechanism significantly degrades strategic reasoning, whereas the Memory Mechanism enhances it. These insights position CHBench as a promising tool for evaluating LLM capabilities, with significant potential for future research and practical applications.

Figures

Figures reproduced from arXiv: 2508.11944 by Hongtao Liu, Weiran Shen, Zhicheng Du, Zihe Wang.

Figure 1
Figure 1. Figure 1: The full architecture of CHBench framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Likelihood improvement ratio for maximum level [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Likelihood improvement ratio for maximum level [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Level distributions under different reasoning mechanisms in the Level-K CH model. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Level distributions under different reasoning mechanisms in the Poisson CH model. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Decision prompt template under the Baseline Mechanism. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Consultation prompt template under the Chat Mechanism. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Response prompt template under the Chat Mechanism. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Decision prompt template under the Chat Mechanism [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Decision prompt template under the Memory Mechanism. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Consultation prompt template under the Chat & Memory Mechanism. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Response prompt template under the Chat & Memory Mechanism. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Decision prompt template under the Chat & Memory Mechanism [PITH_FULL_IMAGE:figures/full_fig_p015_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks

    cs.CL 2026-06 unverdicted novelty 7.0

    BehaviorBench is a benchmark for foundation models on behavioral tasks that reveals fine-tuned behavioral models outperform general models on distributional alignment while general models lead on individual-level accuracy.

Reference graph

Works this paper leans on

38 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Akata, L

    E. Akata, L. Schulz, J. Coda-Forno, S. J. Oh, M. Bethge, and E. Schulz. Playing repeated games with large language models. Nature Human Behavior, 2025

    work page 2025

  3. [3]

    X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu, et al. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    K. Binmore. Modeling rational players: Part i. Economics & Philosophy, 3(2):179–214, 1987

    work page 1987

  5. [5]

    C. F. Camerer, T.-H. Ho, and J.-K. Chong. A cognitive hierarchy model of games. The Quarterly Journal of Economics, 119(3):861–898, 2004

    work page 2004

  6. [6]

    Chong, T.-H

    J.-K. Chong, T.-H. Ho, and C. Camerer. A generalized cognitive hierarchy model of games.Games and Economic Behavior, 99:257–274, 2016

    work page 2016

  7. [7]

    Costa-Gomes, V

    M. Costa-Gomes, V . P. Crawford, and B. Broseta. Cognition and behavior in normal-form games: An experimental study. Econometrica, 69(5):1193–1235, 2001

    work page 2001

  8. [8]

    Costarelli, M

    A. Costarelli, M. Allen, R. Hauksson, G. Sodunke, S. Hariharan, C. Cheng, W. Li, J. Clymer, and A. Yadav. Gamebench: Evaluating strategic reasoning abilities of llm agents. arXiv preprint arXiv:2406.06613, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    J. Duan, R. Zhang, J. Diffenderfer, B. Kailkhura, L. Sun, E. Stengel-Eskin, M. Bansal, T. Chen, and K. Xu. Gtbench: Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations. Advances in Neural Information Processing Systems, 37:28219–28253, 2024

    work page 2024

  10. [10]

    X. Feng, L. Dou, E. Li, Q. Wang, H. Wang, Y . Guo, C. Ma, and L. Kong. A survey on large language model-based social agents in game-theoretic scenarios. arXiv preprint arXiv:2412.03920, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    N. Fontana. Llm’s dilemma: analyzing the behaviors of large language models in the iterated prisoner’s dilemma. 2023

    work page 2023

  12. [12]

    Fontana, F

    N. Fontana, F. Pierri, and L. M. Aiello. Nicer than humans: How do large language models behave in the prisoner’s dilemma? arXiv preprint arXiv:2406.13605, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Fudenberg and J

    D. Fudenberg and J. Tirole. Game theory. MIT press, 1991

    work page 1991

  14. [14]

    J. K. Goeree and C. A. Holt. Ten little treasures of game theory and ten intuitive contradictions. American Economic Review, 91(5):1402–1422, 2001

    work page 2001

  15. [15]

    Grattafiori, A

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    S. Guo, H. Wang, H. Bu, Y . Ren, D. Sui, Y .-M. Shang, and S. E. Lu. Economics arena for large language models. In Language Gamification-NeurIPS 2024 Workshop, 2024

    work page 2024

  17. [17]

    N. Herr, F. Acero, R. Raileanu, M. Perez-Ortiz, and Z. Li. Large language models are bad game theoretic reasoners: Evaluating performance and bias in two-player non-zero-sum games. In ICML 2024 Workshop on LLMs and Cognition, 2024. 9

    work page 2024

  18. [18]

    S. Hu, T. Huang, F. Ilhan, S. Tekin, G. Liu, R. Kompella, and L. Liu. A survey on large language model-based game agents. arXiv preprint arXiv:2404.02039, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    D. Kraft. A software package for sequential quadratic programming. Forschungsbericht- Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raumfahrt, 1988

    1988

  20. [20]

    Lorè and B

    N. Lorè and B. Heydari. Strategic behavior of large language models and the role of game structure versus contextual framing. Scientific Reports, 14(1):18490, 2024

    work page 2024

  21. [21]

    S. Mao, Y . Cai, Y . Xia, W. Wu, X. Wang, F. Wang, Q. Guan, T. Ge, and F. Wei. Alympics: Llm agents meet game theory. In Proceedings of the 31st International Conference on Computational Linguistics, pages 2845–2866, 2025

    work page 2025

  22. [22]

    R. B. Myerson. Game theory. Harvard university press, 2013

    work page 2013

  23. [23]

    G. Owen. Game theory. Emerald Group Publishing, 2013

    work page 2013

  24. [24]

    J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023

    work page 2023

  25. [25]

    J. Ross, Y . Kim, and A. W. Lo. Llm economicus? mapping the behavioral biases of llms via utility theory.arXiv preprint arXiv:2408.02784, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    D. O. Stahl and P. W. Wilson. On players’ models of other players: Theory and experimental evidence. Games and Economic Behavior, 10(1):218–254, 1995

    work page 1995

  27. [27]

    W. C. Stirling. Satisficing Games and Decision Making: with applications to engineering and computer science. Cambridge University Press, 2003

    work page 2003

  28. [28]

    H. Sun, Y . Wu, Y . Cheng, and X. Chu. Game theory meets large language models: A systematic survey.arXiv preprint arXiv:2502.09053, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    S. M. Taylor and B. K. Bergen. Do large language models exhibit spontaneous rational deception? arXiv preprint arXiv:2504.00285, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    G. Team, P. Georgiev, V . I. Lei, R. Burnell, L. Bai, A. Gulati, G. Tanzer, D. Vincent, Z. Pan, S. Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    V on Neumann and O

    J. V on Neumann and O. Morgenstern. Theory of games and economic behavior, 2nd rev. 1947

    work page 1947

  32. [32]

    H. Wang, X. Feng, L. Li, Z. Qin, D. Sui, and L. Kong. Tmgbench: A systematic game benchmark for evaluating strategic reasoning abilities of llms. arXiv preprint arXiv:2410.10479, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Wright and K

    J. Wright and K. Leyton-Brown. Beyond equilibrium: Predicting human behavior in normal-form games. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 901–907, 2010

    work page 2010

  34. [34]

    J. R. Wright and K. Leyton-Brown. Level-0 meta-models for predicting human behavior in games. In Proceedings of the fifteenth ACM conference on Economics and computation, pages 857–874, 2014

    work page 2014

  35. [35]

    J. R. Wright and K. Leyton-Brown. Predicting human behavior in unrepeated, simultaneous-move games. Games and Economic Behavior, 106:16–37, 2017

    work page 2017

  36. [36]

    J. R. Wright and K. Leyton-Brown. A formal separation between strategic and nonstrategic behavior. In Proceedings of the 21st ACM Conference on Economics and Computation, pages 535–536, 2020

    work page 2020

  37. [37]

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Zhang, S

    Y . Zhang, S. Mao, T. Ge, X. Wang, A. de Wynter, Y . Xia, W. Wu, T. Song, M. Lan, and F. Wei. Llm as a mastermind: A survey of strategic reasoning with large language models. arXiv preprint arXiv:2404.01230, 2024. 10 A Appendix A.1 Prompt Templates In this section, we present the prompt templates we used for constructing the LLM-based agents and the game-...

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.