Evaluating Collective Behaviour of Hundreds of LLM Agents

Jianing Zhao; Joel Z. Leibo; Richard Willis; Yali Du

REVIEW 4 major objections 5 minor 5 cited by

Evaluating Collective Behaviour of Hundreds of LLM Agents

T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM agent populations evolve toward exploitative, low-welfare equilibria when users imitate successful payoffs.

desk verdict Valuable evaluation framework and a plausible race-to-the-bottom result, but the 'newer models are worse' claim rests on a pipeline filter that could bias everything. read the letter →

arxiv 2602.16662 v2 pith:XRPTGFZF submitted 2026-02-18 cs.MA

Richard Willis , Jianing Zhao , Yali Du , Joel Z. Leibo This is my paper

classification cs.MA

keywords culturalevolutionsocialdilemmasLLMagentsself-playpublicgoodsgamecollectiveriskdilemmacommonpoolresourceemergentbehaviour

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when LLM-based assistants choose strategies through payoff-biased imitation, populations of hundreds converge to exploitative, low-welfare equilibria in social dilemmas, despite some models being capable of cooperative outcomes. It introduces a pipeline in which each model writes a natural-language strategy and then codes it into an algorithm, enabling pre-deployment inspection and large-scale simulation. Across three games, self-play shows large cross-model differences: more recent reasoning models often perform worse in groups of exploitative agents, while older non-reasoning models can be more robust. Cultural evolution simulations show the Exploitative attitude dominates in most games and group sizes, with welfare efficiency dropping as group size increases. The authors caution that this implies a race to the bottom if autonomous agents are deployed.

What carries the argument

The strategy-to-code pipeline, in which each LLM writes a fixed natural-language strategy and then compiles it into an executable algorithm, is the central mechanism. It reduces each model's behaviour to a reusable strategy with an attitude label (Collective or Exploitative), enabling exhaustive fingerprinting, self-play mixtures, and cultural evolution over hundreds of agents. The cultural-evolution loop—top-64 selection, proportional imitation, 10% mutation, and a 75% dominance stopping rule—is the mechanism that yields the convergence result.

What would settle it

Re-run self-play and cultural evolution using human-verified reference implementations of the same natural-language strategies (or a code-execution check that does not discard failed implementations). If the cross-model welfare rankings and the dominance of exploitative strategies change materially, the reported convergence to low-welfare equilibria is an artifact of the strategy-to-code pipeline rather than of the models' strategic propensities.

Watch

Extended reading notes

Core claim

The central claim is that payoff-biased imitation among users of LLM agents drives convergence to exploitative equilibria rather than cooperative ones, in most of the tested social dilemmas. The paper develops a three-part evaluation: behavioural fingerprinting via PCA over opponent histories, self-play welfare across mixtures of Collective and Exploitative strategies, and cultural evolution with mutation and selection on model-plus-attitude genes. Results show substantial model differences, with one frontier model's aggressive Exploitative strategies dominating cultural evolution in nearly every game at group size 64, while the only Collective-dominant outcomes appear in the Common Pool Res

Load-bearing premise

The results assume that the strategy-description-to-code pipeline produces strategies that fairly represent each model's behaviour; for models that failed to implement many descriptions, only successful, likely simpler strategies were kept, so if implementation success correlates with strategic content, the cross-model differences and cultural-evolution winners could be artifacts of the pipeline rather than of strategic reasoning.

Editorial extensions

If this is right

If deployed LLM assistants are selected by users who imitate successful payoffs, populations will drift to defection-dominated equilibria in public-goods-style settings.
Model developers should test robustness against exploitative strategies, since some models collapse to minimum welfare even when a majority of users request collective behaviour.
More recent reasoning models may produce worse societal outcomes when users prioritise individual gain, consistent with a pattern of reasoning impairing cooperation.
Group size is decisive: larger populations converge faster and to lower welfare, while small groups can sustain collective equilibria through group selection.
Institutions or mechanisms to sustain cooperation will be needed before large-scale deployment of autonomous agents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The pipeline's bias toward simpler strategies for models that failed many implementations means their apparent robustness may reflect limited strategic repertoires, not pro-social reasoning; a human-verified implementation step could test this directly.
The exploitative-dominance result suggests a selection pressure that punishes cooperative agents; an omitted dynamic is whether visible aggregate outcomes or user preferences for fairness would change the equilibria.
A testable extension is to reintroduce communication or reputation mechanisms in these multi-player games and ask whether the exploitative convergence reverses.
The same evaluation pipeline could be applied to coordination games or other non-dilemma settings to see whether cultural evolution converges to cooperative outcomes when individual and collective incentives align.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The evaluation framework is a genuine step forward, and the cultural-evolution convergence to exploitation is a believable risk signal. But the headline "newer models are worse" is the least supported claim in the paper, and it is exactly the one most likely to be quoted.

What's new: they have LLMs write natural-language strategies and then translate them to code, which lets them evaluate hundreds of agents and actually inspect the strategy before deployment. The behavioural fingerprinting with PCA and the self-play welfare curves are thoughtful tools, and the cultural-evolution setup with payoff-based imitation is a reasonable first-order model of user selection. The code is on GitHub, which makes the parts that don't depend on a specific API reproducible.

The soft spot is the strategy-generation pipeline. Section 4.1 admits that Llama 3.1 70B and Mistral 7B couldn't implement all their descriptions, so only successful implementations are kept, and the authors say this "likely biases these models' strategy sets toward simpler strategies." That sentence is doing a lot of work. Because each model both authors and implements its own strategies, the filter does not just remove bad code—it can remove entire modes of behaviour. If aggressive or complex exploitative strategies are harder to implement, the older models will look more cooperative than they are. The stress-test note has it right: the cross-model comparisons and the recency trend should be re-run with an external coder or a human translator before being trusted. The authors do list this as a limitation, but the abstract still states the recency claim without the caveat.

The cultural-evolution race-to-the-bottom result is more robust. It appears across the games and is more pronounced in larger groups, and it aligns with the incentive structure of the games and with prior work. But the specific gene frequencies—Claude dominating almost everywhere—could also be altered by the same coding bias, and the hyperparameters are untuned, so I would treat the exact numbers as illustrative.

The paper would benefit from statistical tests on the welfare differences and from releasing the strategy sets, including the failed descriptions if possible. As it stands, the framework is worth publishing and the qualitative risk finding is worth discussing. The cross-model ranking and the recency claim need more evidence.

It deserves a serious referee, but the referee should push for the external-implementation control and a more careful calibration between the abstract and the actual evidence.

Referee Report

4 major / 5 minor

Summary. The paper proposes an evaluation framework for collective behaviour of LLM agents in three repeated social dilemmas (PGG, CRD, CPR). For each model, 512 natural-language strategies per attitude (Collective/Exploitative) are generated and translated into code by the same model; strategies are then fingerprinted via PCA, evaluated in self-play at group sizes 4–256, and evolved under a cultural-evolution algorithm with payoff-biased imitation. The authors report that more recent models tend to produce worse societal outcomes and that cultural evolution converges to low-welfare, Exploitative equilibria, especially at larger group sizes.

Significance. The framework addresses an important gap: pre-deployment evaluation of emergent multi-agent behaviour at scale. The multi-game design (linear, threshold, dynamic-state) and the combination of PCA fingerprinting, self-play, and cultural evolution are thoughtful, and the code release supports reproducibility. The qualitative 'race to the bottom' result is consistent with game-theoretic intuition and with previous LLM-agent studies. However, the central cross-model and cultural-evolution claims are not yet robust because the strategy-generation pipeline filters on each model's own implementation success, and the paper does not quantify how this filtering affects the reported rankings.

major comments (4)

[§4.1, Abstract, Table 2] Section 4.1's implementation-filter bias is load-bearing for the cross-model claims. For Llama 3.1 70B and Mistral 7B, only successful implementations from 600 descriptions are kept, 'likely biasing these models' strategy sets toward simpler strategies.' Since each model both writes and implements its own strategies, strategy sets are conditioned on coding ability. The abstract's claim that 'more recent models tend to produce worse societal outcomes' and the cultural-evolution dominance of Claude's Exploitative gene (Table 2) rely on these filtered sets. If failed Llama/Mistral descriptions were more aggressive, dropping them would make these models appear more cooperative, inflating the recency trend and Claude's dominance. The paper notes the bias without quantifying its impact, and strategy sets are not released for external audit. Please release the strategy corpora and re-analyze wi
[§4.1, §5.2] The self-referential pipeline (model writes strategy, translates to code, tests its own code, deletes failed outputs) conditions each strategy set on the model's implementation and self-consistency, not just its strategic reasoning. The paper does not test whether implementation success correlates with strategic content (e.g., aggressiveness or complexity). This is a correctness risk for the self-play comparisons in Section 5.2, where the 'newer models worse' pattern in PGG and CPR could reflect coding/implementation differences rather than strategic preferences. Concrete check: compare success rates by attitude and game, and use an external coder to generate implementations for a fixed set of descriptions across all models.
[§6.2, Table 2] The cultural-evolution result is almost entirely driven by Claude's Exploitative gene: for n=64, Claude Exploitative wins 100%, 100%, 98% across PGG, CRD, CPR; at n=4 it wins PGG 100% and CRD 50%. Thus the paper's headline 'race to the bottom' conclusion in larger groups rests on a single model's strategy set. Given the implementation-filter bias, the species-level conclusion is not yet established. The authors should show that the result persists when Claude's strategies are not the only highly effective exploitative options—e.g., by controlling for implementation quality or using a balanced set of strategies per model.
[§7, Abstract] The abstract's recency claim ('more recent models tend to produce worse societal outcomes') is conflated with reasoning vs non-reasoning models in Section 5.2, and with the filtering bias. The paper's own Section 7 acknowledges that attitudes were provided without definition and that alternative framings should be tested. Please state the recency claim only as a hypothesis, or provide a design that separates recency from reasoning capability and implementation success.

minor comments (5)

[§4.2] The 961-dimensional feature vector is not fully specified; please state explicitly how own prior actions are included in the history and how the dimensionality is obtained.
[Appendix A.3] Figure 5 caption mentions grey and red points, but the figure lacks a legend and axis labels are not defined; please make the color coding explicit and self-contained.
[§7] Typo: 'promps' should be 'prompts'.
[§4.1] Model names are inconsistent: 'Llama 3.1 70b' vs '70B', 'Mistral 7b' vs '7B'. Please standardize.
[Appendix B] The k=3 sensitivity analysis reports 'very similar results' but provides no quantitative comparison; a distance metric or effect size between the k=2 and k=3 welfare surfaces would be more informative.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; acknowledged implementation-filter bias is a confound, not a circular step.

full rationale

The paper's central claims—cross-model differences in self-play welfare and cultural-evolution convergence to exploitative equilibria—are empirical results computed from generated strategies under fixed game payoffs; they are not fitted to the target outcomes. Section 4.1's strategy-to-code pipeline is self-referential (the same model writes, translates, and tests strategies), and the paper explicitly concedes in Section 4.1 that for Llama 3.1 70B and Mistral 7b, including only successful implementations 'likely biases these models' strategy sets toward simpler strategies.' This is a validity threat to cross-model comparisons, not circularity: the measured strategies are not definitionally equal to the reported welfare or dominance rankings, and an external re-test with an independent coder could change the results without any tautology. The cultural-evolution dominance result is driven by game payoffs and imitation dynamics, not by the attitude labels; the labels 'Exploitative' and 'Collective' are prompt conditions, not fitted parameters. Self-citations (e.g., Willis et al. 2025, Du et al. 2023) describe prior frameworks or cooperation mechanisms and are not load-bearing for the present derivation. No equation-level reduction, fitted-input-as-prediction, imported uniqueness, or ansatz-by-citation was found.

Assumptions & free parameters 13 free parameters · 5 assumptions · 0 invented entities

The framework rests on hand-chosen simulation hyperparameters and on the premise that a model's self-generated, self-implemented strategies are representative of its strategic behavior. The latter is the most fragile input: the paper's own Section 4.1 reports deletion of failed implementations and a 600-description filter for Llama/Mistral, which biases the strategy sets.

free parameters (13)

cultural evolution population size = 512
Hand-chosen; equilibrium frequencies and termination may depend on population size.
top-k survivors per generation = 64
Hand-chosen; selection pressure strength affects which gene dominates.
mutation rate = 0.10
Hand-chosen; not tuned, affects exploration in cultural evolution.
dominance threshold = 0.75
Termination criterion for cultural evolution; not tuned.
max generations = 200
Termination bound; many runs reached it without dominance.
games per agent per generation = 4
Hand-chosen; sample size for payoff estimation.
self-play samples per proportion = 200
Number of strategy samples for mean welfare estimates.
PCA rollout count = 50
Number of rollouts per opponent history to estimate cooperation rates.
strategy descriptions per model/attitude = 512 (600 for Llama/Mistral)
Set size; larger sets for less capable implementers to compensate for failures.
game rounds = 20
Interaction length; may affect cooperation emergence.
PGG multiplication factor k = 2
Standard parameter from Isaac et al. 1984; sensitivity check with k=3 in Appendix B shows similar results.
CRD threshold m = n/2
Majority threshold from Milinski et al. 2008.
CPR carrying capacity K = 4n
Scales with group size; from Levhari & Mirman 1980.

assumptions (5)

domain assumption The three iterated games (PGG, CRD, CPR) are valid models of social dilemmas relevant to LLM agent deployment.
Section 3 defines payoffs; the paper's conclusions about 'race to the bottom' depend on these games capturing real-world incentive structures.
ad hoc to paper An LLM's natural-language strategy, once translated to code by the same model, faithfully represents the model's strategic reasoning.
Section 4.1: strategies are generated and implemented by the same model; failures are discarded. This is the central methodological premise and is acknowledged as imperfect.
domain assumption Payoff-biased imitation with mutation approximates how users select and prompt AI assistants.
Section 6.1; the paper explicitly notes it omits brand loyalty and switching costs (Section 7).
domain assumption PCA on 961-dimensional behavioral fingerprints captures meaningful strategic variation.
Section 4.2; first two components explain only 74.5% of variance.
standard math Standard game-theoretic assumptions (perfect information, no communication, common knowledge) hold in the games.
Appendix A.2 Task prompt; this is a modeling assumption inherited from game theory.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Evaluating Collective Behaviour of Hundreds of LLM Agents." pith.science (2026). https://pith.science/paper/XRPTGFZF

@misc{pith2026260216662,
  author       = {Pith},
  title        = {Pith review of: Evaluating Collective Behaviour of Hundreds of LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XRPTGFZF}},
  note         = {Machine review of arXiv:2602.16662}
}

read the original abstract

LLM-powered AI assistants acting on behalf of users can produce poor collective outcomes at scale. We introduce a framework for evaluating their emergent behaviour in social dilemmas, applied to three iterated games (Public Goods, Collective Risk, Common Pool Resource). We prompt each model to produce a natural-language strategy, then have the same model translate it into code. This aims to isolate strategic reasoning from input-parsing, enables pre-deployment inspection, and scales to populations of hundreds of agents. We propose three analyses: behavioural fingerprinting via exhaustive evaluation over opponent histories; self-play robustness across mixtures of a model's strategies with either a Selfish or Collective disposition; and cultural evolution under payoff-biased imitation. Applied to three state-of-the-art LLMs, we find substantial cross-model differences in self-play welfare, and that cultural evolution converges to low-welfare, Selfish-dominant equilibria in larger groups.

Figures

Figures reproduced from arXiv: 2602.16662 by the authors.

**Figure 2.** Social welfare of the self-play in Public Goods Game [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Social welfare of the self-play in Collective Risk Game [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Social welfare of self-play in Common Pool Resource [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Principal Component Analysis: Grey denotes new prompt and red denotes original prompt. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 6.** Figure 6: Social welfare of the self-play in Public Goods Game using [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: A sample cultural evolution run in Common Pool Resource, group size [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents
cs.AI 2026-04 unverdicted novelty 6.0 of 10

Large-scale experiments on two million agents reveal that collective intelligence does not emerge from scale alone due to sparse and shallow interactions.
Evolutionary Dynamics of Cooperation in Next-Generation LLM Agent Systems: A Cross-Provider Empirical Extension
cs.MA 2026-05 unverdicted novelty 5.0 of 10

Empirical tests on four new frontier LLMs show cooperative equilibria favored in most balanced conditions, with provider identity correlating more strongly with outcomes than model generation.
AlphaEval: Evaluating Agents in Production
cs.CL 2026-04 unverdicted novelty 5.0 of 10

AlphaEval is a benchmark of 94 production-sourced tasks from seven companies for evaluating full AI agent products across six domains using multiple judgment methods, plus a framework to build similar benchmarks.
Transition from Statistical to Hardware-Limited Scaling in Photonic Quantum State Reconstruction
quant-ph 2026-03 unverdicted novelty 5.0 of 10

Classical shadow tomography on integrated photonics shows a sharp transition from statistical O(M^{-1/2}) error scaling to a hardware-limited floor set by unitary spectral distortions.
Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Game
cs.MA 2026-06 unverdicted novelty 4.0 of 10

LLM agents exhibit emergent deception in a sustainability game even without lying permission, with neighbor info increasing attacks while aiding biosphere retention.

Reference graph

Works this paper leans on

36 extracted references · 3 canonical work pages · cited by 5 Pith papers

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

V., Arriaga, R

Aher, G. V., Arriaga, R. I., and Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, ...

2023
[3]

J., Bethge, M., and Schulz, E

Akata, E., Schulz, L., Coda-Forno , J., Oh, S. J., Bethge, M., and Schulz, E. Playing repeated games with large language models. Nature Human Behaviour, May 2025. ISSN 2397-3374. doi:10.1038/s41562-025-02172-y

doi:10.1038/s41562-025-02172-y 2025
[4]

S., Jenner, E., Casper, S., Sourbut, O., Edelman, B

Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., Edelman, B. L., Zhang, Z., G \"u nther, M., Korinek, A., Hern \'a ndez-Orallo , J., Hammond, L., Bigelow, E. J., Pan, A., Langosco, L., Korbak, T., Zhang, H. C., Zhong, R., \'O h \'E igeartaigh , S., Recchia, G., Corsi, G., Chan, A., An...

2024
[5]

G., Tewolde, E., Mihalcea, R., Sch \"o lkopf, B., and Jin, Z

Backmann, S., Piedrahita, D. G., Tewolde, E., Mihalcea, R., Sch \"o lkopf, B., and Jin, Z. When Ethics and Payoffs Diverge : LLM Agents in Morally Charged Social Dilemmas , May 2025

2025
[6]

Cooperate or Collapse : Emergence of Sustainable Cooperation in a Society of LLM Agents

Curvo, P. M. P., Dragomir, M., Torpes, S., and Rahimi, M. Reproducibility Study of " Cooperate or Collapse : Emergence of Sustainable Cooperation in a Society of LLM Agents ", May 2025

2025
[7]

Z., Islam, U., Willis, R., and Sunehag, P

Du, Y., Leibo, J. Z., Islam, U., Willis, R., and Sunehag, P. A Review of Cooperation in Multi-agent Learning , December 2023

2023
[8]

GTBench : Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations

Duan, J., Zhang, R., Diffenderfer, J., Kailkhura, B., Sun, L., Stengel-Eskin , E., Bansal, M., Chen, T., and Xu, K. GTBench : Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processi...

2024

Show all 36 references

[9]

Can Large Language Models Serve as Rational Players in Game Theory : A Systematic Analysis

Fan, C., Chen, J., Jin, Y., and He, H. Can Large Language Models Serve as Rational Players in Game Theory : A Systematic Analysis . Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (16): 0 17960--17967, March 2024. doi:10.1609/aaai.v38i16.29751

2024 doi
[10]

A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios

Feng, X., Dou, L., Li, M., Wang, Q., Guo, Y., Wang, H., Ma, C., and Kong, L. A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios . Transactions on Machine Learning Research, 2025, 2025

2025
[11]

Fontana, N., Pierri, F., and Aiello, L. M. Nicer than humans: How do large language models behave in the prisoner's dilemma? In An, J., Lin, Y.-R., Mejova, Y., Mustafaraj, E., Kulshrestha, J., and Weber, I. (eds.), Proceedings of the Nineteenth International AAAI Conference on...

2025 doi
[12]

Gandhi, K., Sadigh, D., and Goodman, N. D. Strategic Reasoning with Language Models . In Foundation Models for Decision Making Workshop at NeurIPS 2023 , 2023

2023
[13]

Gordon, H. S. The Economic Theory of a Common Property Resource : The Fishery . Journal of Political Economy, 62: 0 124--142, 1954

1954
[14]

V., Wiest, O., and Zhang, X

Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., and Zhang, X. Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI 2024, Jeju...

2024
[15]

A., Hughes, E., Kova r \'i k, V., Kulveit, J., Leibo, J

Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier , J., Khan, A., McLean, E., Smith, C., Barfuss, W., Foerster, J., Gaven c iak, T., Han, T. A., Hughes, E., Kova r \'i k, V., Kulveit, J., Leibo, J. Z., Oesterheld, C., de Witt, C. S., Shah, N., Wellman, M., Bova, P., Cimp...

2025
[16]

Large Language Models as Simulated Economic Agents : What Can We Learn from Homo Silicus ? Technical Report w31122, National Bureau of Economic Research, Cambridge, MA, April 2023

Horton, J. Large Language Models as Simulated Economic Agents : What Can We Learn from Homo Silicus ? Technical Report w31122, National Bureau of Economic Research, Cambridge, MA, April 2023

2023
[17]

M., Walker, J

Isaac, R. M., Walker, J. M., and Thomas, S. H. Divergent evidence on free riding: An experimental examination of possible explanations. Public Choice, 43 0 (2): 0 113--149, January 1984. ISSN 1573-7101. doi:10.1007/BF00140829

1984 doi
[18]

Game of thoughts: Iterative reasoning in game-theoretic domains with large language models

Kempinski, B., Gemp, I., Larson, K., Lanctot, M., Bachrach, Y., and Kachman, T. Game of thoughts: Iterative reasoning in game-theoretic domains with large language models. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, Aamas '2...

2025
[19]

King's Computational Research , Engineering and Technology Environment ( CREATE ), 2024

King's College London e-Research team . King's Computational Research , Engineering and Technology Environment ( CREATE ), 2024

2024
[20]

and Mirman, L

Levhari, D. and Mirman, L. J. The great fish war: An example using a dynamic Cournot-Nash solution. The Bell Journal of Economics, pp.\ 322--334, 1980

1980
[21]

AgentBench : Evaluating llms as agents

Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., Zhang, S., Deng, X., Zeng, A., Du, Z., Zhang, C., Shen, S., Zhang, T., Su, Y., Sun, H., Huang, M., Dong, Y., and Tang, J. AgentBench : Evaluating llms as agents. In The Twelfth Internati...

2024
[22]

and Heydari, B

Lor \`e , N. and Heydari, B. Strategic behavior of large language models and the role of game structure versus contextual framing. Scientific Reports, 14 0 (1): 0 18490, August 2024. ISSN 2045-2322. doi:10.1038/s41598-024-69032-z

2024 doi
[23]

ALYMPICS : LLM agents meet game theory

Mao, S., Cai, Y., Xia, Y., Wu, W., Wang, X., Wang, F., Guan, Q., Ge, T., and Wei, F. ALYMPICS : LLM agents meet game theory. In Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa , H., Eugenio, B. D., and Schockaert, S. (eds.), Proceedings of the 31st International Conference ...

2025
[24]

D., Krambeck, H.-J., Reed, F

Milinski, M., Sommerfeld, R. D., Krambeck, H.-J., Reed, F. A., and Marotzke, J. The collective-risk social dilemma and the prevention of simulated dangerous climate change. Proceedings of the National Academy of Sciences, 105 0 (7): 0 2291--2294, 2008. doi:10.1073/pnas.0709546105

2008 doi
[25]

Nowak, M. A. Five Rules for the Evolution of Cooperation . Science, 314 0 (5805): 0 1560--1563, December 2006. ISSN 0036-8075, 1095-9203. doi:10.1126/science.1133755

2006 doi
[26]

Rules, Games, and Common-Pool Resources

Ostrom, Gardner, R., and Walker, J. Rules, Games, and Common-Pool Resources. University of Michigan press, 1994

1994
[27]

Covenants with and without a Sword : Self-Governance Is Possible

Ostrom, E., Walker, J., and Gardner, R. Covenants with and without a Sword : Self-Governance Is Possible . American Political Science Review, 86 0 (2): 0 404--417, June 1992. ISSN 0003-0554, 1537-5943. doi:10.2307/1964229

1992 doi
[28]

S., Zou, A., Li, N., Basart, S., Woodside, T., Zhang, H., Emmons, S., and Hendrycks, D

Pan, A., Chan, J. S., Zou, A., Li, N., Basart, S., Woodside, T., Zhang, H., Emmons, S., and Hendrycks, D. Do the rewards justify the means? Measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark. In Krause, A., Brunskill, E., Cho, K., Engelhardt...

2023
[29]

Cooperate or collapse: Emergence of sustainable cooperation in a society of LLM agents

Piatti, G., Jin, Z., Kleiman-Weiner , M., Sch \"o lkopf, B., Sachan, M., and Mihalcea, R. Cooperate or collapse: Emergence of sustainable cooperation in a society of LLM agents. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (ed...

2024
[30]

G., Yang, Y., Sachan, M., Ramponi, G., Sch \"o lkopf, B., and Jin, Z

Piedrahita, D. G., Yang, Y., Sachan, M., Ramponi, G., Sch \"o lkopf, B., and Jin, Z. Corrupted by Reasoning : Reasoning Language Models Become Free-Riders in Public Goods Games . In COLM 2025 , Montreal, Canada, October 2025. doi:10.48550/arXiv.2506.23276

2025 doi arXiv
[31]

Game theory meets large language models: A systematic survey

Sun, H., Wu, Y., Cheng, Y., and Chu, X. Game theory meets large language models: A systematic survey. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2025, Montreal, Canada, August 16-22, 2025 , pp.\ 10669--10677. ijcai.org,...

2025 doi
[32]

and Hughes, E

Vallinder, A. and Hughes, E. Cultural Evolution of Cooperation among LLM Agents , December 2024

2024
[33]

Z., and Luck, M

Willis, R., Du, Y., Leibo, J. Z., and Luck, M. Will Systems of LLM Agents Cooperate : An Investigation into a Social Dilemma - Extended Abstract . In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems , Detroit, Michigan, USA, May 2025...

2025
[34]

I., Onizuka, M., Tang, S., and Xiao, C

Wu, Z., Peng, R., Zheng, S., Liu, Q., Han, X., Kwon, B. I., Onizuka, M., Tang, S., and Xiao, C. Shall we team up: Exploring spontaneous cooperation of competing LLM agents. In Al-Onaizan , Y., Bansal, M., and Chen, Y.-N. (eds.), Findings of the Association for Computational Li...

2024 doi
[35]

LLM as a Mastermind : A Survey of Strategic Reasoning with Large Language Models

Zhang, Y., Mao, S., Ge, T., Wang, X., Xia, Y., Wu, W., Song, T., Lan, M., and Wei, F. LLM as a Mastermind : A Survey of Strategic Reasoning with Large Language Models . In COLM 2024 , 2024

2024
[36]

MultiAgentBench : Evaluating the Collaboration and Competition of LLM agents, March 2025

Zhu, K., Du, H., Hong, Z., Yang, X., Guo, S., Wang, Z., Wang, Z., Qian, C., Tang, X., Ji, H., and You, J. MultiAgentBench : Evaluating the Collaboration and Competition of LLM agents, March 2025

2025

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

V., Arriaga, R

Aher, G. V., Arriaga, R. I., and Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, ...

2023

[3] [3]

J., Bethge, M., and Schulz, E

Akata, E., Schulz, L., Coda-Forno , J., Oh, S. J., Bethge, M., and Schulz, E. Playing repeated games with large language models. Nature Human Behaviour, May 2025. ISSN 2397-3374. doi:10.1038/s41562-025-02172-y

doi:10.1038/s41562-025-02172-y 2025

[4] [4]

S., Jenner, E., Casper, S., Sourbut, O., Edelman, B

Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., Edelman, B. L., Zhang, Z., G \"u nther, M., Korinek, A., Hern \'a ndez-Orallo , J., Hammond, L., Bigelow, E. J., Pan, A., Langosco, L., Korbak, T., Zhang, H. C., Zhong, R., \'O h \'E igeartaigh , S., Recchia, G., Corsi, G., Chan, A., An...

2024

[5] [5]

G., Tewolde, E., Mihalcea, R., Sch \"o lkopf, B., and Jin, Z

Backmann, S., Piedrahita, D. G., Tewolde, E., Mihalcea, R., Sch \"o lkopf, B., and Jin, Z. When Ethics and Payoffs Diverge : LLM Agents in Morally Charged Social Dilemmas , May 2025

2025

[6] [6]

Cooperate or Collapse : Emergence of Sustainable Cooperation in a Society of LLM Agents

Curvo, P. M. P., Dragomir, M., Torpes, S., and Rahimi, M. Reproducibility Study of " Cooperate or Collapse : Emergence of Sustainable Cooperation in a Society of LLM Agents ", May 2025

2025

[7] [7]

Z., Islam, U., Willis, R., and Sunehag, P

Du, Y., Leibo, J. Z., Islam, U., Willis, R., and Sunehag, P. A Review of Cooperation in Multi-agent Learning , December 2023

2023

[8] [8]

GTBench : Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations

Duan, J., Zhang, R., Diffenderfer, J., Kailkhura, B., Sun, L., Stengel-Eskin , E., Bansal, M., Chen, T., and Xu, K. GTBench : Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processi...

2024