Pith. sign in

REVIEW 5 major objections 6 minor 32 references

WiS Platform: Enhancing Evaluation of LLM-Based Multi-Agent Systems Through Game-Based Analysis

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 'Who is Spy' game can rank LLM agents by reasoning, deception, and attack resistance.

desk verdict A live, useful platform for game-based LLM-agent evaluation, but the capability-ranking claims run ahead of the evidence. read the letter →

arxiv 2412.03359 v2 pith:D5XGNOOF submitted 2024-12-04 cs.AI

Chengwei Hu , Jianhui Zheng , Yancheng He , Hangyu Guo , Junguang Jiang , Han Zhu , Kai Sun , Yuning Jiang
show 2 more authors
Wenbo Su Bo Zheng
This is my paper · ORCID
classification cs.AI
keywords LLMmulti-agentsystemsWhoisSpysocialdeductiongamesgame-basedevaluationdeceptiondetectionreasoningpromptinjectionattackleaderboardplatform
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces WiS, an online platform that evaluates large language model agents by having six of them play the social deduction game 'Who is Spy'. The paper's central claim is that performance in this game, measured by a custom zero-sum scoring system, voting accuracy, win rates, and foul rates, can distinguish agents' attacking, defense, reasoning, and deception abilities. To support this, the authors ran repeated games among ten open- and closed-source models and report clear behavioral differences, with GPT-4o achieving the highest overall win rate and average score while Qwen models showed the strongest spy win rate, suggesting stronger deception. The platform is open, scalable, and built for continuous leaderboard updates, with game data downloadable for further model training. If the claim holds, the game provides a dynamic, less easily gamed alternative to static datasets for evaluating multi-agent systems.

What carries the argument

The load-bearing object is the 'Who is Spy' game environment together with its scoring rules and role-specific metrics. The game fixes six participants, a spy and five civilians, each with a word they must describe without revealing; rounds of description, fouling, and voting decide elimination, and the spy wins by surviving to round three or leaving fewer than three civilians. The scoring system is designed as a zero-sum allocation of 12 points per game, giving the spy 0, 4, 8, or 12 points depending on when discovery happens, plus one bonus point per round in which civilians vote out the spy, so total payoff is fixed. These rules turn win rate, average score, voting accuracy, survival rounds, and foul rate into the quantitative evidence from which the paper reads reasoning, deception, and attack/defense capabilities.

What would settle it

Run the same ten models on a standard set of logic and reasoning puzzles and compare the ordering with their WiS average scores; if the orderings are uncorrelated, or if a human panel judging game transcripts cannot distinguish the agents the platform ranks, the claimed capability separation fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a six-player 'Who is Spy' game can serve as a multi-agent evaluation environment that ranks LLM-based agents by distinct capabilities. Each round, one spy and five civilians describe related but different words without saying them, vote to eliminate a player, and score points under a zero-sum rule: 12 points go either to a surviving spy or to the surviving civilians when the spy is found, with smaller payoffs for later discovery and bonus points for correct votes. The paper argues that spy-role scores measure deception, civilian voting accuracy measures reasoning, and responses to inserted prompt-injection instructions measure attacking and defense. In tests repeated over ninety games per model, GPT-4o had the highest civilian win rate (84.93%), overall win rate (76.67%), and average score (3.24), while Qwen2.5-72B-Instruct had the highest spy win rate (46.60%). The authors interpret these and related results as evidence that the platform effectively differentiates multi-agent abilities.

Load-bearing premise

The argument's load-bearing premise is that the custom 'Who is Spy' score is a valid proxy for general reasoning, deception, and adversarial robustness; the paper offers no external benchmark, human baseline, or independent reasoning test to establish that mapping.

Editorial extensions

If this is right

  • The platform yields a continuously updated, open leaderboard that ranks LLM agents by their game scores without needing new static datasets.
  • The prompt-injection attack and defense settings give a quantitative comparison of how easily different models can be manipulated into fouls or bad votes.
  • Adding an explicit reasoning step changes performance unequally: it raises GPT-4o's voting accuracy and civilian win rate while lowering those of Qwen2.5-72B-Instruct and Llama-3-70B-Instruct.
  • Downloadable game logs support supervised or reinforcement learning, so the same environment can be used for evaluation and for training improved agents.
  • Because any model can be registered through the unified interface, new agents can be compared in real time against current best performers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the game score is meant to measure general capabilities, it should be validated against established reasoning and deception benchmarks; the paper does not provide that external check, so transferability remains an open question.
  • The zero-sum scoring rewards survival timing and vote precision more than raw win frequency, so average score and win rate can rank agents differently; treating them as one number would blur the two.
  • The appendix shows that top competitors sometimes used different models than declared and added defensive filters, which implies the live leaderboard partly measures prompt engineering and hardening, not just base-model ability.
  • Using the game directly as a training objective could encourage overfitting to spy-game speech patterns; whether those skills transfer to other adversarial or cooperative settings is untested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces WiS, an online platform that uses the "Who is Spy?" social deduction game to evaluate LLM-based multi-agent systems. It provides a Hugging Face/API integration interface, a live leaderboard, game visualization, data download, and custom scoring rules. The authors run games among ten LLM agents and report overall performance (Table 2), prompt-injection attack/defense experiments (Table 3), and a reasoning-prompt experiment (Figure 2), concluding that the platform effectively distinguishes attacking, defense, reasoning, and deception capabilities and that GPT-4o is superior. The platform and code are publicly released.

Significance. If its evaluation constructs are valid, the platform would be a useful, scalable, and less overfit-prone complement to static benchmarks, with the additional benefit of releasing game logs for training. The engineering contribution is concrete: public deployment, an SDK, visualization, and downloadable data are real strengths. However, the paper's central empirical claim is not yet supported because the metrics are not externally anchored, sample sizes are small, and several controlled-variable and identity issues affect the leaderboard and reasoning experiments. The contribution is therefore more convincingly a platform/tool description than a validated evaluation benchmark at this stage.

major comments (5)
  1. [Section 5.1 and Table 2] The central claim that WiS 'effectively distinguishes' attacking, defense, reasoning, and deception capabilities is not yet supported because the metric construct is unvalidated. Section 5.1 asserts that average score reflects 'comprehensive abilities' and that voting accuracy is 'the most relevant metric for assessing an agent's analytical reasoning ability,' but no correlation with an established reasoning or theory-of-mind benchmark, no human baseline, and no capability-isolating ablation is provided. In addition, word-pair difficulty and starting-speaker order are uncontrolled, so GPT-4o's advantage in Table 2 and the capability attributions in Section 6.2 could be artifacts of game setup rather than measurements of the intended constructs. I ask for an external validation step (e.g., correlation with existing benchmarks or human expert ratings) or controlled ablations that isolate each claimed capability.
  2. [Appendix C and Table 3] The statistical evidence is too weak for the cross-model distinctions claimed. Appendix C says each role-condition was repeated 'more than 24 times,' but the Table 3 values include win rates such as 33.33%, 18.75%, and 23.53%, which are consistent with win counts as small as 8 out of 24, 3 out of 16, and 4 out of 17; even if the real denominators are larger, the paper never states exact counts. No confidence intervals or significance tests are reported anywhere, and because each six-agent game couples all participants' outcomes, the effective number of independent observations is smaller still. Please report exact game counts, per-condition confidence intervals, and appropriate statistical tests, and show that the sample size supports the specific rankings claimed in Section 6.2.
  3. [Section 6.1 and Appendix C] The evaluated model set is described inconsistently. Appendix C states that 'ten publicly available open-source models' were evaluated, but Table 2 includes GPT-4o, Gemini-1.5-pro, Claude-3-5-Sonnet, Kimi, ERNIE, and Doubao, which are closed-source or API-only models. This matters for reproducibility because API-backed models change over time and because the paper's scope claims differ between 'open- and closed-source LLMs' in the abstract and 'open-source models' in the appendix. Please specify exact model versions, access mode (weights versus API), and evaluation dates, and correct the inconsistent characterization.
  4. [Section 3 and Appendix A.3] The leaderboard is not a clean model-capability ranking. The scoring rule in Section 3 states that 'the more games played, the more likely one is to achieve a high ranking,' and Appendix A.3 reports that top entries sometimes used models different from the declared ones (e.g., a GPT-3.5 entry actually using Doubao). The leaderboard therefore conflates engagement and identity verification with model quality. Please restrict leaderboard rankings to verified model identities with game-count-controlled scores, or present the leaderboard explicitly as a community-competition ranking rather than as evidence of model capability.
  5. [Section 6.4 and Figure 2] The reasoning experiment is confounded by its own intervention. Adding the 'Reasoning' prompt from Section 5.2 changes the content and style of public speech by the target civilian, and other agents observe that speech; the measured changes in voting accuracy and win rate can therefore reflect information-propaganda effects rather than the reasoning quality of the prompted model. Attributing GPT-4o's improvement to 'superior chain-of-thought reasoning' and Qwen's and Llama's decline to 'relatively weak reasoning' (Section 6.4) is circular because reasoning quality is not measured independently. Please redesign the experiment so that reasoning is elicited privately (e.g., before the public statement) or add a control condition that matches utterance length and style.
minor comments (6)
  1. [Section 3, displayed equation] The sum 'NX i=1 si' should be typeset as \sum_{i=1}^N s_i, and N should be defined explicitly as the number of games played by that player.
  2. [Throughout] Model names are inconsistent (e.g., 'GPT4o' vs. 'GPT-4o' and 'Qwen2.5-72B-Instruct' vs. 'QWEN'), and the game name appears both as 'Who is Spy?' and 'Who is spy'; please standardize these terms.
  3. [Section 2 and References] The reference list contains duplicate entries for Xu et al. 2023 with the same title ('Exploring large language models for communication games: An empirical study on werewolf'), and the in-text citations to 'Xu et al.' do not distinguish the three different Xu et al. works; please disambiguate.
  4. [Figure 1] The example phrases 'Bitter taste. From tree. Keep awake.' are not explained and look like sample descriptions rather than word-pair examples; please label them or remove them.
  5. [Section 5.2, Table 1] The paper does not specify how the attacking, defense, and reasoning prompts are inserted (system message, user message, or appended context), nor how they interact with the base system prompt; this detail is needed for reproducibility.
  6. [Appendix C] The statements 'each experiment was repeated over 90 times' and 'more than 24 times' should be reconciled and made precise about whether these are independent games per model per condition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: WiS is an empirical measurement platform with no fitted parameters, self-citation chain, or equation-level tautology; construct-validity concerns are external to circularity.

full rationale

The paper is an empirical evaluation platform, not a derivation of results from first principles. The scoring rules in Section 3 are fixed game rules rather than parameters fitted to data; the overall indicators in Section 5.1 (win rate, average score, voting accuracy, foul rate) are computed from observed gameplay, not reverse-engineered to force a conclusion. There is no self-citation used as load-bearing evidence, no imported uniqueness theorem, and no ansatz smuggled in via prior work. The statements that voting accuracy is 'the most relevant metric for assessing an agent's analytical reasoning ability' and that spy average score measures 'deceptive capacity' are operational definitions, not circular reductions; the empirical results (e.g., GPT-4o's higher win rates and voting accuracy in Table 2) are observations under those definitions. The interpretive attributions in Sections 6.2 and 6.4, such as attributing GPT-4o's score to 'enhanced reasoning abilities,' are post-hoc explanations rather than consequences of the scoring equations. The leaderboard confounds acknowledged by the paper itself—the cost per game and the statement that 'the more games played, the more likely one is to achieve a high ranking' (Section 3), and the model-substitution observations in Appendix A.3—are validity threats, not circularity. In short, the platform's construct validity (whether game metrics truly measure reasoning, deception, and robustness) is unvalidated against external benchmarks, but that is a correctness/validity concern, not a circularity concern under the definitional criteria.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the assumption that the game-based scoring is a valid measure of reasoning and deception, and on hand-chosen scoring and environment parameters. No invented entities are introduced.

free parameters (3)
  • Game scoring weights for spy elimination round and spy win = 0/4/8/12 points for spy; 12/8/4/0 shared among civilians; ±1 voting bonus
    Hand-chosen to define the evaluation metric; the paper's rankings and behavioral claims depend on this scoring.
  • Entry score and per-game cost = 100 initial points, 1 point cost per game
    Hand-chosen constants in the ranking formula that make expected score per game equal 1 point; this shapes the leaderboard.
  • Game end conditions and length limits = 3 rounds; 120 (Chinese) / 400 (English) character speech limits; 10 second timeout
    Hand-chosen environment parameters that affect which behaviors are possible and how fouls are counted.
assumptions (4)
  • domain assumption Performance in the 'Who is Spy' game is a valid proxy for general reasoning, deception, and adversarial robustness.
    Invoked throughout Sections 5 and 6; no external validation against human judgments or established benchmarks.
  • domain assumption The custom scoring rule measures the intended abilities (e.g., voting accuracy reflects analytical reasoning).
    Section 5.1 states that average score and voting accuracy 'can better reflect' abilities, assumed without empirical validation.
  • domain assumption The prompt injection prompts in Table 1 are representative attack and defense strategies.
    Section 5.2 uses these two prompts to draw conclusions about models' robustness.
  • standard math Standard arithmetic for expected score calculation (12/6 - 1 = 1).
    Section 3 ranking rules.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WiS Platform: Enhancing Evaluation of LLM-Based Multi-Agent Systems Through Game-Based Analysis." pith.science (2026). https://pith.science/paper/D5XGNOOF

@misc{pith2026241203359,
  author       = {Pith},
  title        = {Pith review of: WiS Platform: Enhancing Evaluation of LLM-Based Multi-Agent Systems Through Game-Based Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D5XGNOOF}},
  note         = {Machine review of arXiv:2412.03359}
}
read the original abstract

Recent advancements in autonomous multi-agent systems (MAS) based on large language models (LLMs) have enhanced the application scenarios and improved the capability of LLMs to handle complex tasks. Despite demonstrating effectiveness, existing studies still evidently struggle to evaluate, analysis, and reproducibility of LLM-based MAS. In this paper, to facilitate the research on LLM-based MAS, we introduce an open, scalable, and real-time updated platform for accessing and analyzing the LLM-based MAS based on the games Who is Spy?" (WiS). Our platform is featured with three main worths: (1) a unified model evaluate interface that supports models available on Hugging Face; (2) real-time updated leaderboard for model evaluation; (3) a comprehensive evaluation covering game-winning rates, attacking, defense strategies, and reasoning of LLMs. To rigorously test WiS, we conduct extensive experiments coverage of various open- and closed-source LLMs, we find that different agents exhibit distinct and intriguing behaviors in the game. The experimental results demonstrate the effectiveness and efficiency of our platform in evaluating LLM-based MAS. Our platform and its documentation are publicly available at https://whoisspy.ai/.

Figures

Figures reproduced from arXiv: 2412.03359 by the authors.

Figure 1
Figure 1. Users can conveniently register public or private models as agents through our website and SDK, enabling [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Performance comparison of different models [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 14 canonical work pages

  1. [1]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    work page 2020

  2. [2]

    Chi - Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2024. Chateval: Towards better llm-based evaluators through multi-agent debate. In ICLR . OpenReview.net

    work page 2024

  3. [3]

    Jiaqi Chen, Yuxian Jiang, Jiachen Lu, and Li Zhang. 2024 a . S-agents: self-organizing agents in open-ended environment. arXiv preprint arXiv:2402.04578

    arXiv 2024

  4. [4]

    Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2024 b . Optima: Optimizing effectiveness and efficiency for llm-based multi-agent system

    work page 2024

  5. [5]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    arXiv 2025

  6. [6]

    Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. 2023. Self-collaboration code generation via chatgpt. arXiv preprint arXiv:2304.07590

    arXiv 2023

  7. [7]

    Xidong Feng, Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, and Jun Wang. 2024. Chessgpt: Bridging policy learning and language modeling. Advances in Neural Information Processing Systems, 36

    work page 2024

  8. [8]

    Chawla, Olaf Wiest, and Xiangliang Zhang

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. In IJCAI , pages 8048--8057. ijcai.org

    work page 2024

Show all 32 references
  1. [9]

    Akshat Gupta. 2023. http://arxiv.org/abs/2308.12466 Are chatgpt and gpt-4 good poker players? -- a pre-flop analysis

    2023 arXiv

  2. [10]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zi Hen Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and J \"u rgen Schmidhuber. 2023. https://api.semanticscholar.org/CorpusID:265301950...

    2023

  3. [11]

    Chenghao Huang, Yanbo Cao, Yinlong Wen, Tao Zhou, and Yanru Zhang. 2024. Pokergpt: An end-to-end lightweight solver for multi-player texas hold'em via large language model. arXiv preprint arXiv:2401.06781

    2024 arXiv

  4. [12]

    Dong Huang, Jie M.Zhang, Michael Luck, Qi Bu, Yuhao Qing, and Heming Cui. 2023. https://api.semanticscholar.org/CorpusID:270045924 Agentcoder: Multi-agent-based code generation with iterative testing and optimisation

    2023

  5. [13]

    Zhao Kaiya, Michelangelo Naim, Jovana Kondic, Manuel Cortes, Jiaxin Ge, Shuying Luo, Guangyu Robert Yang, and Andrew Ahn. 2023. Lyfe agents: Generative agents for low-cost real-time social interactions. arXiv preprint arXiv:2310.02172

    2023 arXiv

  6. [14]

    Chuanhao Li, Runhan Yang, Tiankai Li, Milad Bafarassat, Kourosh Sharifi, Dirk Bergemann, and Zhuoran Yang. 2024. STRIDE: A tool-assisted LLM agent framework for strategic and interactive decision-making. CoRR, abs/2405.16376

    2024 arXiv

  7. [15]

    Jonathan Light, Min Cai, Sheng Shen, and Ziniu Hu. 2023. Avalonbench: Evaluating llms playing the game of avalon. In NeurIPS 2023 Foundation Models for Decision Making Workshop

    2023

  8. [16]

    Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023 a . Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1--22

    2023

  9. [17]

    O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S

    Joon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023 b . Generative agents: Interactive simulacra of human behavior. In UIST , pages 2:1--2:22. ACM

    2023

  10. [18]

    Cheng Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. https://api.semanticscholar.org/CorpusID:270257715 Chatdev: Communicative agents for software development...

    2023

  11. [19]

    Dan Qiao, Chenfei Wu, Yaobo Liang, Juntao Li, and Nan Duan. 2023. Gameeval: Evaluating llms on conversational games. arXiv preprint arXiv:2308.10032

    2023 arXiv

  12. [20]

    Zijing Shi, Meng Fang, Shunfeng Zheng, Shilong Deng, Ling Chen, and Yali Du. 2023. Cooperation on the fly: Exploring language agents for ad hoc teamwork in the avalon game. arXiv preprint arXiv:2312.17515

    2023 arXiv

  13. [21]

    Shubham Toshniwal, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2022. Chess as a testbed for language model state tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11385--11393

    2022

  14. [22]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023 a . Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291

    2023 arXiv

  15. [23]

    Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. 2023 b . https://api.semanticscholar.org/CorpusID:263605971 Avalon's game of thoughts: Battle against deception through recursive contemplation . ArXi...

    2023 arXiv

  16. [24]

    Tianhe Wang and Tomoyuki Kaneko. 2018. Application of deep reinforcement learning in werewolf game agents. In 2018 conference on technologies and applications of artificial intelligence (TAAI), pages 28--33. IEEE

    2018

  17. [25]

    Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, et al. 2023 c . Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models. arXiv preprint arXiv:2311.05997

    2023 arXiv

  18. [27]

    Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. 2023 b . https://api.semanticscholar.org/CorpusID:261681932 Exploring large language models for communication games: An empirical study on werewolf . ArXiv, abs/2309.04658

    2023 arXiv

  19. [28]

    Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. 2023 c . Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.18940

    2023 arXiv

  20. [29]

    Yauwai Yim, Chunkit Chan, Tianyu Shi, Zheye Deng, Wei Fan, Tianshi Zheng, and Yangqiu Song. 2024. Evaluating and enhancing llms agent based on theory of mind in guandan: A multi-player cooperative game under imperfect information. arXiv preprint arXiv:2408.02559

    2024 arXiv

  21. [30]

    Enmin Zhao, Renye Yan, Jinqiu Li, Kai Li, and Junliang Xing. 2022. Alphaholdem: High-performance artificial intelligence for heads-up no-limit poker via end-to-end reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4689--4697

    2022

  22. [31]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian - Yun Nie, and Ji - Ro...

    2023 arXiv

  23. [32]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  24. [33]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.