Introduces measurable coupling gain gamma and randomized-initial-condition diagnostic to distinguish genuine social dynamics from LLM prior artifacts in agent societies.
The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
Large language models (LLMs) are increasingly deployed to simulate human collective behaviors, yet the methodological rigor of these "AI societies" remains under-explored. Through a systematic audit of 39 recent studies, we identify six pervasive flaws-spanning agent profiles, interaction, memory, control, unawareness, and realism (PIMMUR). Our analysis reveals that 89.7% of studies violate at least one principle, undermining simulation validity. We demonstrate that frontier LLMs correctly identify the underlying social experiment in 50.8% of cases, while 61.0% of prompts exert excessive control that pre-determines outcomes. By reproducing five representative experiments (e.g., telephone game), we show that reported collective phenomena often vanish or reverse when PIMMUR principles are enforced, suggesting that many "emergent" behaviors are methodological artifacts rather than genuine social dynamics. Our findings suggest that current AI simulations may capture model-specific biases rather than universal human social behaviors, raising critical concerns about the use of LLMs as scientific proxies for human society.
years
2026 4representative citing papers
Minor perturbations in persona format, instruction framing, and network structure shift cooperation by up to 76 percentage points and polarization metrics consistently, showing that LLM social simulations require per-claim robustness audits via the new TRAILS taxonomy.
An LLM-driven agent-based model with multi-round dialogue reproduces non-linear social influence patterns in vaccination opinion dynamics, with memory increasing resistance and prompt diversity increasing adoption.
Proposes AI-driven simulations for literary-historical experiments and reports preliminary text-generation results claiming the first limited in-distribution outputs matching human novels.
citing papers explorer
-
When Is Emergent Consensus Real? A Measured Coupling Gain and a Validity Diagnostic for LLM Agent Societies
Introduces measurable coupling gain gamma and randomized-initial-condition diagnostic to distinguish genuine social dynamics from LLM prior artifacts in agent societies.
-
Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits
Minor perturbations in persona format, instruction framing, and network structure shift cooperation by up to 76 percentage points and polarization metrics consistently, showing that LLM social simulations require per-claim robustness audits via the new TRAILS taxonomy.
-
A Large Language Model-Driven Agent-Based Modeling Framework with Multi-Round Communication for Simulating Vaccine Opinion Dynamics
An LLM-driven agent-based model with multi-round dialogue reproduces non-linear social influence patterns in vaccination opinion dynamics, with memory increasing resistance and prompt diversity increasing adoption.
-
AI as a Tool for Simulation-Based Experiments in Literary Studies
Proposes AI-driven simulations for literary-historical experiments and reports preliminary text-generation results claiming the first limited in-distribution outputs matching human novels.