REVIEW 3 major objections 5 minor 28 cited by
OASIS: Open Agent Social Interaction Simulations with One Million Agents
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper claims OASIS, a modular LLM-agent simulator, scales to one million users and reproduces information spreading, group polarization, and herd effects on X and Reddit, while larger populations yield more diverse and helpful…
desk verdict A genuinely useful open-source simulator for million-agent social media studies, but the headline scale findings are confounded and need control conditions before they can be stated as results. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing components are the five modules of OASIS. The Environment Server stores users, posts, comments, and relations in a database. The RecSys controls what each agent sees: on X it ranks in-network posts by likes and out-of-network posts by recency, follower impact, and TwHIN-BERT cosine similarity of interests; on Reddit it ranks by the platform's hot-score formula. The Agent Module gives each LLM-backed user a memory and 21 action types, with chain-of-thought reasoning. The Time Engine activates each agent according to a 24-dimensional hourly activity profile, and the Scalable Inferencer distributes inference requests across GPUs asynchronously. A large-scale user-generation algorithm builds networks that preserve the scale-free structure of real social graphs. Together these parts make the simulator platform-adaptable and population-scalable.
What would settle it
Run OASIS with a fixed agent count of 1,000 while sweeping the number of posts injected per time step (30, 300, 3000) and the recommendation cache size (50, 500, 5000). If the diversity and helpfulness of the 196 core users' opinions rise with content volume alone, the paper's 'larger group' conclusion is called into question. Alternatively, fix content volume and cache size while sweeping agent count from 196 to 10,196; if helpfulness and diversity do not rise, the population-size claim fails.
Extended reading notes
Core claim
On its own terms, the paper demonstrates that a single architecture with five components—an environment server, platform-specific recommendation systems, an LLM-based agent module, a time engine, and a scalable inference layer—can simulate up to one million agents across X and Reddit. Using real-world data from Twitter15/16, Reddit, and counterfactual posts, OASIS reproduces real propagation trends with a normalized RMSE around 30%, produces increasing group polarization over 80 time steps (more strongly with uncensored models), and replicates the human herd effect, while finding that agents are more conformist than humans on down-treated content. In scale experiments, the same 196 core users receive more diverse and more helpful opinions when the surrounding population grows from 196 to 10,196 and then to 100,196 agents, and herd effects on counterfactual posts appear only above a certain group size.
Load-bearing premise
The claim that a larger agent population improves opinion diversity and helpfulness assumes that the concurrent increases in the number of posts injected per time step and the size of the recommendation cache are not the actual causes of those improvements.
Editorial extensions
If this is right
- Social-science findings replicated in small groups could be re-tested at population scale, where the paper claims emergent behavior appears only with enough agents.
- Researchers can swap modules to compare platform designs, since OASIS supports both interest-based (X) and hot-score-based (Reddit) recommendation systems.
- The scale effect on helpfulness suggests that simulations with hundreds of agents may underestimate the quality and variety of opinions a core user receives.
- The million-agent misinformation experiment shows a tractable path for studying how false content spreads relative to official news, with misinformation generating more related posts.
Reading between the lines
- The paper's scale-ablation design increases post-injection volume and recommendation cache size together with agent count, so the reported gains in diversity and helpfulness may be driven by more content or broader exposure rather than by population size per se; an editorially proposed control experiment would hold content volume fixed while varying only the number of agents.
- The finding that agents herd on dislikes but not likes, unlike humans, could reflect the LLM's prior about negative signals, and would be worth testing across different base models to see whether the asymmetry is a property of the simulator or of the particular model.
- OASIS's architecture lends itself to studying interventions—modifying the recommendation algorithm or content visibility—in a way that would be hard to test on real platforms, a direction the paper notes as untapped potential.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces OASIS, an LLM-agent-based social media simulator with an environment server, recommendation systems, a 21-action agent module, a time engine, and a scalable inference backend. The authors claim generalizability across X and Reddit, demonstrate simulations with up to one million agents, and report replications of information spreading, group polarization, and herd effects. They also report two scale findings: that larger agent populations yield more helpful and diverse opinions (Finding 4) and that larger groups amplify dislike-driven herd effects (Finding 5).
Significance. If the claims hold, OASIS is a valuable open-source platform for studying social-media dynamics at a scale that prior LLM-agent simulators have not reached. The modular five-component architecture, the open-source release, and the 1M-agent demonstration on 24 A100s are genuine engineering contributions, and the repeated propagation runs in Appendix F.2.2 are a commendable reproducibility check. However, the empirical validation is weakened by confounded scale experiments and by a propagation fit with roughly 30% normalized RMSE and a systematically lower depth curve. The scale findings are the paper's headline novelty, so the confounds are load-bearing.
major comments (3)
- [§3.4.2, F.4.2] The scale comparison for the counterfactual herd effect varies the number of agents (100, 1k, 10k) together with the number of posts injected per time step (30, 300, 3k) and the recommendation cache size (50, 500, 5k). Because activated agents sample 5 posts from the cache (F.4.2), the probability of seeing a down-treated post increases with both injection rate and cache size, so the monotonic down-treated disagree-score gap in Figure 8 can be explained by exposure volume rather than population size. Finding 5 requires a control that varies N while holding post volume and cache size fixed, or varies volume at fixed N.
- [§3.4.1, Fig. 7] Finding 4 compares the same 196 core users under population sizes 196, 10,196, and 100,196, but no condition isolates N from the amount of content the core users receive. As the surrounding population grows, the number of comments, likes, and recommendations directed at the 196 core users grows as well; the observed increase in diversity and helpfulness could therefore reflect a larger response sample under unchanged per-agent behavior. A matched condition that fixes the post volume and recommendation exposure per core user is needed to support the causal claim that 'larger agent group scale leads to ... more diverse and helpful agents' opinions.'
- [§3.3.1, Fig. 4] Finding 1 states that OASIS replicates the information-spreading process, but the reported normalized RMSE is about 30% and the depth curve is systematically lower than the real data for most of the observation window. Since 'closely replicate' is a headline claim of the paper, this level of fit needs a quantitative threshold or a benchmark comparison (e.g., against a rule-based baseline) before the claim is supported. Reporting per-metric NRMSE and clearly separating the depth limitation from the scale/breadth fit would also help.
minor comments (5)
- [§3.5] The text contains an unresolved cross-reference, 'as described in Section ??'; please fix it.
- [Fig. 9 caption] The caption begins 'TThe figure shows...'; the extra 'T' should be removed.
- [§F.2.1, Eq. (7)] The sentence defining the variables repeats 'yi_simu, yi_simu' where the second variable should presumably be 'yi_real'.
- [§3.3.1 and Fig. 5] The protagonist's name is spelled inconsistently as both 'Halen' and 'Helen' across the text and figure.
- [§3.2 and F.3.3] There are minor typos: 'Appenix' instead of 'Appendix' and 'Helpfullness' instead of 'Helpfulness'.
Circularity Check
No significant circularity: core replications are benchmarked against external data; the scale findings are confounded but not reducible to their inputs by construction.
full rationale
OASIS is primarily an engineering and empirical-simulation paper. Its central replications—information propagation, group polarization, and herd effects—are compared against external real-world benchmarks (Vosoughi et al., Muchnik et al., real Reddit data), and no parameter is fitted to the target outcomes before the simulation is run. The agent responses are generated by an LLM rather than solved from the input equations, so the polarization and herd-effect observations are not definitionally equivalent to the prompts or to the RecSys formulas. Seeing like/dislike counts and hot scores is an environmental input, but the LLM's conformity, disagreement, or extremification behavior is not entailed by those inputs alone. The only notable self-citation is CAMEL (Li et al., 2023), used for codebase inheritance of the agent module; this is not load-bearing for the paper's scientific claims. The scale findings (Findings 4 and 5) are indeed confounded: Section F.4.2 varies agent count together with recommendation-cache size and the controlled user's injected post volume, so attributing the observed differences to population size alone is not established. However, confounding is an experimental validity concern rather than a circular derivation: there is no equation, fitted parameter, or definitional identity that reduces the claimed scale effect to its inputs by construction. The paper also candidly acknowledges limitations in RecSys fidelity, user-generation abstraction, and simulation cost, which further supports the conclusion that the main empirical claims are not circular.
Assumptions & free parameters
free parameters (5)
- Core-user follow probability =
0.2 (agent follows core user) and 0.1 (general following probability)
- Agent activation probability =
0.1 per time step
- TF-IDF relevance threshold =
0.2 cosine similarity
- X RecSys recency and fan-count constants =
271.8, 100, 1000 in Eq. (2)-(5)
- RecSys cache size and post injection rate =
50/500/5000 posts cached; 30/300/3000 posts per step for 100/1k/10k agents
assumptions (4)
- domain assumption LLM agents' actions are a valid proxy for human social media behavior
- domain assumption LLM judges (GPT-4o-mini, GPT-4o) give reliable ratings of extremeness, helpfulness, and agreement
- domain assumption The simplified recommendation systems capture the essential information-flow mechanisms of X and Reddit
- ad hoc to paper The scale experiments differ only in agent count, not in information volume
Cite this review
Pith. "Pith review of OASIS: Open Agent Social Interaction Simulations with One Million Agents." pith.science (2026). https://pith.science/paper/JIJZO3BI
@misc{pith2026241111581,
author = {Pith},
title = {Pith review of: OASIS: Open Agent Social Interaction Simulations with One Million Agents},
year = {2026},
howpublished = {\url{https://pith.science/paper/JIJZO3BI}},
note = {Machine review of arXiv:2411.11581}
}
read the original abstract
There has been a growing interest in enhancing rule-based agent-based models (ABMs) for social media platforms (i.e., X, Reddit) with more realistic large language model (LLM) agents, thereby allowing for a more nuanced study of complex systems. As a result, several LLM-based ABMs have been proposed in the past year. While they hold promise, each simulator is specifically designed to study a particular scenario, making it time-consuming and resource-intensive to explore other phenomena using the same ABM. Additionally, these models simulate only a limited number of agents, whereas real-world social media platforms involve millions of users. To this end, we propose OASIS, a generalizable and scalable social media simulator. OASIS is designed based on real-world social media platforms, incorporating dynamically updated environments (i.e., dynamic social networks and post information), diverse action spaces (i.e., following, commenting), and recommendation systems (i.e., interest-based and hot-score-based). Additionally, OASIS supports large-scale user simulations, capable of modeling up to one million users. With these features, OASIS can be easily extended to different social media platforms to study large-scale group phenomena and behaviors. We replicate various social phenomena, including information spreading, group polarization, and herd effects across X and Reddit platforms. Moreover, we provide observations of social phenomena at different agent group scales. We observe that the larger agent group scale leads to more enhanced group dynamics and more diverse and helpful agents' opinions. These findings demonstrate OASIS's potential as a powerful tool for studying complex systems in digital environments.
Figures
Figures from the paper (13 more)
Forward citations
Cited by 28 Pith papers
-
IO Factory: Simulating AI-Enabled Influence Campaigns at Scale
A framework that simulates AI influence campaigns end to end, recording exposure and belief shifts in a controlled platform with matched baselines.
-
CARD: Controlled Agentic Reddit Discussions for Credit Card Simulation
CARD plans and generates Reddit-style credit card discussions, then self-revises them until their lexical, semantic, and structural distributions match real threads more closely than existing baselines.
-
Eco3S: Complex Socio-Economic System Simulation via Agent-Based Models
Eco3S packages co-evolving environments, checkpoint counterfactuals, and auto-refinement into one LLM agent-based simulation platform, demonstrated on canal-rebellion, state-formation, and information-spread cases.
-
CoSimRec: Measuring Coordinated-Content Penetration in Recommender Feedback Loops
Under controlled simulation, coordinated content reaches non-bot recommendation slots when rankers reward popularity or feedback (APR-Lift up to 0.47 on LastFM), while random ranking shows none.
-
Can We Steer the Black-Box? Towards Controllability-Centric Evaluation of Recommender Systems with Collaborative Agents
CtrlBench-Rec uses LLM-driven agent probes, evolved through clustering and merging, to measure how well recommender systems can be steered toward target content, interest profiles, and long-tail items.
-
What is Left for Us? Second Scholarship Against the Degradation of Research by AI
AI degrades research by automating formative practices of judgment; second scholarship reclaims tacit knowledge, commitment, socialization, and deep reading as what communities must value.
-
AgentSociety 2: An Integrated Research Environment for Executable Social Science
An integrated LLM-agent environment runs social-science experiments from hypothesis to manuscript, reproducing several known human patterns while failing on others (implicit self-bias, free-riding decay, norm collapse).
-
Topology-Aware LLM-Driven Social Simulation: A Unified Framework for Efficient and Realistic Agent Dynamics
TopoSim reduces LLM-based social simulation costs by 50-90% by grouping structurally similar agents for coordinated updates and using topology-derived PageRank to model asymmetric social influence.
-
Think Before You Code: Dual Reasoning for the NLSafety-Utility Trade-Off in LLM Code Generation
Dual Reasoning with explicit safety audits improves the new SUDS metric by 1.32x to 3.42x over baselines on code generation benchmarks containing injected harmful keywords.
-
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
A five-aspect, 24-metric benchmark, a 26K human-annotated dataset, and an AI evaluator show that today's driving world models cannot simultaneously look real, respect geometry, and behave safely.
-
RoboTron-Sim: Improving Real-World Driving via Simulated Hard-Case
A simulation-to-real pipeline (HASS synthetic hard cases, scenario-aware prompts, and an image-to-ego geometry encoder) improves an MLLM's open-loop planning on nuScenes, especially in hard scenarios.
-
LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra
The LLM Economist framework couples persona-conditioned worker agents with an in-context RL planner to search US-bracket tax schedules, yet its Saez benchmark is derived from the planner's own solution and its headlin...
-
When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems
In a 1,000-agent social simulation, decentralized groups of malicious AI agents spread more misinformation and commit more fraud than centralized groups, and they adapt to evade content moderation.
-
TinyTroupe: An LLM-powered Multiagent Persona Simulation Toolkit
TinyTroupe provides a toolkit for fine-grained persona-based LLM multi-agent simulations with built-in support for population sampling, experimentation, and validation.
-
Modeling Earth-Scale Human-Like Societies with One Billion Agents
Light Society scales LLM-agent social simulations to one billion agents by substituting most LLM interactions with a distilled surrogate model.
-
From Blueprint to Reality: Modeling and Applying Putnam's Social Capital Theory with LLM-based Multi-agent Simulations
SocaSim uses LLM-based multi-agent simulations to model Putnam's Social Capital Theory, reproducing macro-level patterns and aligning with human group-level decisions, then applies the framework to smart elderly care ...
-
From Plausible to Causal: Counterfactual Semantics for Policy Evaluation in Simulated Online Communities
Policy claims from LLM community simulations should be stated as probabilities of necessary and sufficient causation, mapped to stakeholder needs and conditioned on simulator fidelity.
-
RecoWorld: Building Simulated Environments for Agentic Recommender Systems
A design proposal, not a tested system: a dual-view simulation loop in which an LLM-simulated user issues reflective instructions when about to disengage, and an instruction-following recommender adapts to maximize si...
-
LLMs are Introvert
A psychology-inspired prompting method (SIP-CoT with emotion-guided memory) makes LLM agents reproduce human-like attitudes and behaviors more closely in social simulations, but the evaluation lacks error bars, a name...
-
Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Values
An external 'superego' module that filters agentic AI plans against user-selected 'constitutions' plus a universal safety floor is reported to cut harmful outputs by up to 98% on safety benchmarks.
-
AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents
A new nine-task benchmark measures LLM agents' propensity for misalignment and finds more capable models misalign more on average, with persona effects sometimes exceeding model effects.
-
GGBond: Growing Graph-Based AI-Agent Society for Socially-Aware Recommender Simulation
GGBond is an agent-based simulator that couples a five-layer cognitive agent model with a dynamic multilayer social graph to evaluate recommender systems under long-term feedback.
-
Agentic Workflows for Conversational Human-AI Interaction Design
A four-cycle design study found that agentic workflows with contextualization, goal formulation, and prompt articulation help users and designers manage ambiguity and transience in conversational AI.
-
Toward LLM-Agent-Based Modeling of Transportation Systems: A Conceptual Framework
LLM-driven agents with profiles, memory, and feedback loops can generate plausible daily travel activities and learn to adjust commute timing in a small proof-of-concept, pointing toward a new direction for agent-base...
-
SimViews: An Interactive Multi-Agent System Simulating Visitor-to-Visitor Conversational Patterns to Present Diverse Perspectives of Artifacts in Virtual Museums
The body text belongs to a different paper (Chimera), so the abstract's SimViews claims and its 20-participant study are unsupported in the submitted text.
-
Public Discourse Sandbox: Facilitating Human and AI Digital Communication Research
The Public Discourse Sandbox is a Django-based research platform with human-AI and AI-AI accounts, IRB-based experiment management, and open-source deployment.
-
Recalibrating the Compass: Integrating Large Language Models into Classical Research Methods
LLMs extend, rather than replace, classical social science methods, with a proposed three-tier bias framework for LLM-augmented surveys.
-
From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents
A structured survey that categorizes LLM-based social simulation into individual, scenario, and society simulation, with associated methods, benchmarks, and observed trends.
Reference graph
Works this paper leans on
-
[1]
Economics: The study and management of production, distribution, and consumption of goods and services. Economics focuses on how individuals, businesses, governments, and nations make choices about allocating resources to satisfy their wants and needs, and tries to determine how these groups should organize and coordinate efforts to achieve maximum output
-
[2]
IT (Information Technology): The use of computers, networking, and other physical devices, infrastructure, and processes to create, process, store, secure, and exchange all forms of electronic data. IT is commonly used within the context of business operations as opposed to personal or entertainment technologies
-
[3]
Lev Muchnik, Sinan Aral, and Sean J Taylor
URL https://arxiv.org/abs/2402.16333. Lev Muchnik, Sinan Aral, and Sean J Taylor. Social influence bias: A randomized experiment. Science, 341(6146):647–651, 2013. Nature Reviews Psychology. Social media needs science-based guidelines. Nature Reviews Psy- chology, 3(6):367–367, Jun 2024. ISSN 2731-0574. doi: 10.1038/s44159-024-00327-8. URL https://doi.org...
arXiv 2013
-
[4]
General News: A broad category that includes current events, happenings, and trends across a wide range of areas such as politics, business, science, technology, and entertainment. General news provides a comprehensive overview of the latest developments affecting the world at large
-
[5]
Politics: The activities associated with the governance of a country or other area, especially the debate or conflict among individuals or parties having or hoping to achieve power. Politics is often a battle over control of resources, policy decisions, and the direction of societal norms
-
[6]
URL https://arxiv.org/abs/2403.19267. Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 12104–12113, 2022. An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang, and Tat-Seng Chua. On generative agents in recommendation. In Pro...
arXiv 2015
-
[7]
Fun: Activities or ideas that are light-hearted or amusing. This topic covers a wide range of entertainment choices and leisure activities that bring joy, laughter , and enjoyment to individuals and groups. Output: [list of topic numbers] Ensure your output could be parsed to **list**, don’t output anything else. # Prompt of Step-3 Please generate a socia...
work page 2013
-
[8]
"Reference to Reliable Sources": If the AI claims certain facts , it should be able to refer to recognized and trusted sources
Show all 21 references
-
[9]
This topic explores how cultural expressions and societal structures influence human behavior, relationships, and social norms
Culture & Society: The way of life for an entire society, including codes of manners, dress, language, religion, rituals, norms of behavior, and systems of belief. This topic explores how cultural expressions and societal structures influence human behavior, relationships, and...
-
[12]
This topic encompasses the entrepreneurial, managerial, and administrative processes involved in starting, managing, and growing a business entity
Business: The practice of making one’s living through commerce, trade, or services. This topic encompasses the entrepreneurial, managerial, and administrative processes involved in starting, managing, and growing a business entity
-
[14]
Accurate Information
"Accurate Information": Ensure the AI provides information that is factual and up to date. 21
-
[15]
Clarity and Comprehensibility
"Clarity and Comprehensibility": Check if the AI delivers information in a clear and easily understandable manner
-
[16]
Completeness of the Response
"Completeness of the Response": Ascertain that the AI answers all aspects of the user’s query
-
[17]
Contextual Understanding
"Contextual Understanding": The AI should demonstrate a clear understanding of the context of the user’s query
-
[18]
Creative Problem-Solving
"Creative Problem-Solving": If applicable, observe if the AI proposes creative solutions to the user’s problem
-
[19]
Depth of Explanation
"Depth of Explanation": Examine whether the AI provides detailed and in-depth responses when required
-
[20]
Politeness and Professionalism
"Politeness and Professionalism": The AI should deliver responses using respectful and professional language
-
[22]
User Engagement
"User Engagement": The AI should engage the user effectively and pleasantly, encouraging positive user interaction. 33 Preprint [User Question] {question} [Answer1] {answer1} [Answer2] {answer2} [Response Format] Reason: Choice: Answer1 or Answer2 F.4 H ERD EFFECT F.4.1 M ETRI...
-
[2015]
Thomas C Schelling
URL https://medium.com/hacking-and-gonzo/ how-reddit-ranking-algorithms-work-ef111e33d0d9 . Thomas C Schelling. Models of segregation. The American economic review, 59(2):488–493, 1969. Hyunjin Song and Hajo G Boomgaarden. Dynamic spirals put to test: An agent-based model of r...
1969 arXiv
-
[2023]
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al
URL https://arxiv.org/abs/2307.07924. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789, 2023. Ni...
2023 arXiv
-
[2024]
Xinyi Mou, Zhongyu Wei, and Xuanjing Huang
URL https://arxiv.org/abs/2403.11381. Xinyi Mou, Zhongyu Wei, and Xuanjing Huang. Unveiling the truth and facilitating change: To- wards agent-based large-scale social movement simulation. ArXiv preprint , abs/2402.16333,
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.