REVIEW 9 cited by
Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advancements in AI have reinvigorated Agent-Based Models (ABMs), as the integration of Large Language Models (LLMs) has led to the emergence of ``generative ABMs'' as a novel approach to simulating social systems. While ABMs offer means to bridge micro-level interactions with macro-level patterns, they have long faced criticisms from social scientists, pointing to e.g., lack of realism, computational complexity, and challenges of calibrating and validating against empirical data. This paper reviews the generative ABM literature to assess how this new approach adequately addresses these long-standing criticisms. Our findings show that studies show limited awareness of historical debates. Validation remains poorly addressed, with many studies relying solely on subjective assessments of model `believability', and even the most rigorous validation failing to adequately evidence operational validity. We argue that there are reasons to believe that LLMs will exacerbate rather than resolve the long-standing challenges of ABMs. The black-box nature of LLMs moreover limit their usefulness for disentangling complex emergent causal mechanisms. While generative ABMs are still in a stage of early experimentation, these findings question of whether and how the field can transition to the type of rigorous modeling needed to contribute to social scientific theory.
Forward citations
Cited by 9 Pith papers
-
Can Generative AI agents behave like humans? Evidence from laboratory market experiments
With at least three steps of memory and high randomness, large language model agents broadly replicate human forecasting patterns in positive and negative feedback markets, but are less heterogeneous than humans.
-
Empirical Grounding Improves the Realism of LLM Agents Simulating Human Behavior During Disruptions
Grounding LLM agents with ACS demographics and ATUS time-use routines substantially improved reproduction of observed activity profiles under normal and heatwave conditions in Philadelphia, though over half of the hea...
-
Large language models replicate and predict human cooperation across experiments in game theory
Llama-3.1-8B with a multi-step reasoning-and-filter prompt reproduces human cooperation rates across 121 dyadic games (MSD=0.031, r=0.89), outperforming Nash-equilibrium predictions (MSD=0.096, r=0.78).
-
CoBAD: Modeling Collective Behaviors for Human Mobility Anomaly Detection
CoBAD detects collective mobility anomalies (unexpected co-occurrence and absence) by pre-training a two-stage attention model over collective event sequences and event graphs.
-
Large Language Models for Agent-Based Modelling: Current and possible uses across the modelling cycle
A structured review showing that current LLM use in agent-based modelling is concentrated in implementation, with untapped but risky potential across problem formulation, conceptualization, verification, validation, i...
-
LLM-Based Social Simulations Require a Boundary
LLM-based social simulations are scientifically useful only within boundaries set by behavioral variance, and current validation practice under-checks variance.
-
Modeling Social Dynamics with an LLM-Enabled Agent Based Network-Dynamic (LAND) Model
A large LLM-powered simulation of a fictional song contest claims coordination and influence emerge from network-narrative feedback, but the supporting analysis contains internal contradictions and unreleased artifacts.
-
A Large Language Model-Driven Agent-Based Modeling Framework with Multi-Round Communication for Simulating Vaccine Opinion Dynamics
An LLM-driven agent-based model with multi-round dialogue reproduces non-linear social influence patterns in vaccination opinion dynamics, with memory increasing resistance and prompt diversity increasing adoption.
-
Large Language Models for History, Philosophy, and Sociology of Science: Interpretive Uses, Methodological Challenges, and Critical Perspectives
A position paper arguing that LLMs can enhance interpretive research in history, philosophy, and sociology of science, but only with model literacy, domain-specific benchmarks, and critical reflection.
Discussion (0). Continue with ORCID to comment.