REVIEW 7 cited by
Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The behavior of Large Language Models (LLMs) as artificial social agents is largely unexplored, and we still lack extensive evidence of how these agents react to simple social stimuli. Testing the behavior of AI agents in classic Game Theory experiments provides a promising theoretical framework for evaluating the norms and values of these agents in archetypal social situations. In this work, we investigate the cooperative behavior of three LLMs (Llama2, Llama3, and GPT3.5) when playing the Iterated Prisoner's Dilemma against random adversaries displaying various levels of hostility. We introduce a systematic methodology to evaluate an LLM's comprehension of the game rules and its capability to parse historical gameplay logs for decision-making. We conducted simulations of games lasting for 100 rounds and analyzed the LLMs' decisions in terms of dimensions defined in the behavioral economics literature. We find that all models tend not to initiate defection but act cautiously, favoring cooperation over defection only when the opponent's defection rate is low. Overall, LLMs behave at least as cooperatively as the typical human player, although our results indicate some substantial differences among models. In particular, Llama2 and GPT3.5 are more cooperative than humans, and especially forgiving and non-retaliatory for opponent defection rates below 30%. More similar to humans, Llama3 exhibits consistently uncooperative and exploitative behavior unless the opponent always cooperates. Our systematic approach to the study of LLMs in game theoretical scenarios is a step towards using these simulations to inform practices of LLM auditing and alignment.
Forward citations
Cited by 7 Pith papers
-
The Energy Society: A Simulation Environment for Studying Agent Cooperation under Survival Pressure
A new multi-agent simulation ties token-generation cost to survival, showing larger models are less efficient, shared objectives induce donations, and recommendation and memory mechanisms shape coordination.
-
How memory can affect collective and cooperative behaviors in an LLM-Based Social Particle Swarm
LLM agents in a spatial Prisoner's Dilemma exhibit model-specific effects of memory length on cooperation, with Gemini suppressing and Gemma promoting it as memory increases.
-
Super-additive Cooperation in Language Model Agents
Language model agents cooperate more in a prisoner's dilemma when repeated interactions and inter-team competition are combined, but only for some models.
-
CHBench: A Cognitive Hierarchy Benchmark for Evaluating Strategic Reasoning Capability of LLMs
CHBench fits Level-K and Poisson cognitive hierarchy models to LLM game play and uses the fitted reasoning level as a benchmark score.
-
How Large Language Models play humans in online conversations: a simulated study of the 2016 US politics on Reddit
GPT-4 impersonating Reddit users in 2016 election threads produces comments that lean toward consensus and are semantically separable from real human comments.
-
The Effect of State Representation on LLM Agent Behavior in Dynamic Routing Games
In a repeated Braess routing game, LLM agents given summarized, regret-based, and own-action-only state representations converge closer to Nash equilibrium and behave more stably than agents given full chat transcript...
-
Beyond Nash Equilibrium: Bounded Rationality of LLMs and humans in Strategic Decision-making
LLMs reproduce human heuristics like switching after a loss and cooperating when future rounds loom, but apply them more rigidly and adapt less than humans.
Discussion (0). Sign in to comment.