Pith. sign in

REVIEW 4 major objections 4 minor 113 references

AgentSociety 2 claims that coupling AI social scientists with simulated participants in one runtime makes social science executable end to end, reproducing over 95% of qualitative patterns from prior studies.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:38 UTC pith:MM5U34V7

load-bearing objection Real infrastructure with credible engineering numbers, but the 95% qualitative-pattern claim is unfalsifiable and several canonical effects come out null in the paper's own studies. the 4 major comments →

arxiv 2607.11895 v2 pith:MM5U34V7 submitted 2026-06-11 cs.CY cs.MA

AgentSociety 2: An Integrated Research Environment for Executable Social Science

Jinghua Piao , Jun Zhang , Haoyu Huang , Keming Zhang , Jing Yi Wang , Xinran Zhao , Songwei Li , Boyuan Sun
show 11 more authors
Jiayi Chang Fengli Xu Chunyan Wang Fang Zhang Ke Rong Jun Su Tianguang Meng Yi Liu Qingguo Meng Yu Wang Yong Li
This is my paper
classification cs.CY cs.MA
keywords executable social sciencelarge language model agentsAI social scientistsilicon participantssocial simulationhuman-in-the-loop researchagentic environmentscomputational social science
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AgentSociety 2 tries to close the gap between AI-assisted research workflows and simulated societies by giving LLM agents two roles in the same runtime: AI social scientists that plan and run studies, and silicon participants that generate the behavioral data. The platform's central claim is that this dual-role design turns social-science hypotheses into auditable agent behaviors, environment rules, interventions, and measurements, supporting an end-to-end research process. Seven illustrative studies across micro, meso, and macro scales are offered as evidence, with the paper reporting that the platform reproduces more than 95% of qualitative patterns from prior studies, scales to 10,000 agents and 100 million interactions, and matches two disaster-mobility scenarios with low RMSE values. A sympathetic reader would care because, if correct, the platform would let social scientists delegate procedural work while keeping conceptual control, turning questions about norms, cooperation, polarization, and urban response into runnable experiments.

Core claim

The central discovery is that coupling the two roles, researcher agents and participant agents, in one runtime makes social-science experiments executable end to end. Hypotheses are encoded not as free text but as structured packages: agent profiles, environment modules, interventions, measurement plans, and analysis and manuscript drafts all trace back to the same workspace. The paper reports that this architecture reproduced the qualitative direction of classic findings across laboratory games, psychological surveys, social-media dynamics, and urban and disaster mobility, with quantitative agreement in mobility (RMSE 0.0073 to 0.0188). The deviations it did find, such as no implicit-associ

What carries the argument

The load-bearing mechanism is the dual-role runtime built from a skill-based agent harness and agentic environment modules. AI social scientist agents use domain skills, including literature retrieval, hypothesis generation, experiment configuration, analysis, and drafting, inside a state-machine workflow with human approval gates. Silicon participants run in private workspaces with modular skills such as observation, cognition, plan, and memory. CodeGenRouter translates agents' natural-language instructions into validated environment calls, with pre-generation and semantic caching handling over 70% of requests and cutting LLM calls by about 66.5%. The norms and metanorms game, public-goods

Load-bearing premise

The load-bearing premise is that LLM 'silicon participants' produce behavior that validly stands in for human social behavior; if that proxy fails, the experiments still run but their output cannot be interpreted as social-science evidence.

What would settle it

Run a pre-registered battery of, say, 20 well-established human behavioral effects with known directions, require the system to label each result as reproduced or deviated before comparing with human data, and count the accuracy; if the labels are no better than chance, or if a majority of cognitive-process effects fail, the 95% reproduction claim falls.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the dual-role design works as claimed, social-science hypotheses can be registered as structured, versioned, executable packages rather than prose, making the entire chain from idea to manuscript auditable.
  • The same runtime can serve micro, meso, and macro questions, so a single platform could replace the fragmented toolchains that currently separate theory, data, simulation, and analysis.
  • Human researchers retain high-level agency through approval gates, which matters if such systems become part of real research workflows rather than fully autonomous pipelines.
  • Simulations at 10,000 agents and 100 million interactions are claimed feasible, making large-scale computational social experiments more practical.
  • Deviations from prior findings are interpreted as hypotheses about mechanism specification and boundary conditions, giving the platform a generative role rather than a purely replicative one.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The platform's evidentiary value depends on whether silicon participants validly proxy human behavior; the paper itself reports null results in the psychological tasks (no implicit-association latency difference, no self-reference memory advantage) that illustrate exactly where that proxy can fail.
  • An independent pre-registered battery of well-established human behavioral effects, with expected directions and effect sizes, would convert the paper's post-hoc classification of deviations into a testable claim about the 95% reproduction rate.
  • If LLM agents consistently show the cooperative stability seen in the public-goods simulation, the platform may be better suited to studying institutional rules and interventions than to reproducing motivational decay in repeated interactions.
  • A shared registry of deviation cases could let the community accumulate which behavioral signatures current LLMs can and cannot carry, turning the platform into an instrument for studying LLM cognition as well as for studying society.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces AgentSociety 2, an integrated research environment that couples LLM agents in two roles: AI social scientists that orchestrate the full research workflow (literature grounding, hypothesis generation, experiment design, simulation execution, result analysis, manuscript drafting) and silicon participants that generate behavioral responses in configurable social environments. The platform adds a skill-based agent architecture, a unified natural-language environment interface with CodeGenRouter, caching and pre-generation optimizations, an agentic data foundation, and a paper-generation pipeline. The authors report engineering evaluations (CodeGenRouter improves strict successful-call ratio over baselines by up to 31.6%; pre-generation handles 70.5–74.1% of interaction requests; caching reduces LLM calls by 66.5%) and seven illustrative social-science studies spanning micro-, meso-, and macro-level phenomena. The headline validation claims are that the platform supports simulations with up to 10,000 agents and 100 million interactions while reproducing more than 95% of qualitative patterns from prior studies and quantitatively matching disaster-mobility responses with RMSE 0.0073–0.0188.

Significance. If the validation claims are supportable, AgentSociety 2 would be a significant infrastructure contribution: it visibly ships as open source with an online platform, provides concrete engineering benchmarks with reproducible metrics, and addresses a real gap by coupling simulation of participants with agentic research orchestration. The engineering results are specific and largely checkable: the routing benchmark is well-defined, the efficiency numbers for pre-generation and caching are concrete, and the ablation against five baselines across six LLMs is informative. However, the central social-science validation claim — that the platform reproduces more than 95% of qualitative patterns from prior studies — is not operationalized in the manuscript, and several canonical effects reported in Section 7 are null or reversed. The value of the paper therefore rests more on the engineering system than on the claimed social-science validity, and that validity claim currently lacks a transparent, falsifiable protocol.

major comments (4)
  1. [Abstract; Sec. 1; Sec. 7] The claim that the platform 'reproduc[es] more than 95% of qualitative patterns reported in prior studies' is never operationalized. No pre-defined list of patterns, counting rules, scoring rubric, or independent classification is provided. The seven illustrative studies are presented as evidence, but several canonical patterns fail within them (IAT, SRE, free-riding decay, norm collapse; see Secs. 7.3.3, 7.2.3, 7.1.3). Without an enumerated pattern set and a pre-specified decision rule, the 95% figure is unfalsifiable. Please provide the complete pattern inventory, the scoring procedure, and the threshold used, and show how each of the seven studies contributes to the numerator and denominator. If deviations are included as 'informative,' the counting rule and who applies it must be explicit and independent of the system under test.
  2. [Sec. 6.1] The routing benchmark is constructed from 'records of agent calls to the environment interface during actual simulation runs' and manually labeled with ground-truth functions, parameters, and call order. This makes the benchmark circular for the platform's own claim: the test cases reflect the distribution of calls produced by the platform's own agents and environment implementations, so CodeGenRouter is being evaluated on the same patterns it is designed to serve. An independent evaluation should use a held-out, human-authored task set, or at least a cross-validation protocol that separates the agent-call distribution used for development from that used for testing. The current design also leaves open whether the ground-truth labels were produced with knowledge of the router's behavior, which could inflate reported SR.
  3. [Sec. 7.1.3; Sec. 7.2.3; Sec. 7.3.3] Null results for canonical effects are relabeled as 'informative deviations' by the system under test, with no independent criterion. The Norms Game does not show the expected norm collapse (boldness drops to 0.10 rather than remaining high), the public-goods agents show no sustained free-riding decay (contributions stabilize around 8, with 70.83% of agents contributing above preference-predicted levels in round 10), the IAT shows no latency difference (807.7 vs 809.3 ms, t=0.25, p=0.81), and the SRE shows no self-reference memory gradient. These are exactly the outcomes that the 95% reproduction claim ought to count as failures unless a pre-registered deviation taxonomy exists. The paper's assertion that these are 'informative' is made by the AI social scientist pipeline itself, including its internal reviewer agents (Sec. 4.7), which is self-referential. Please specify an external, sub
  4. [Sec. 7.5.2; Sec. 7.4.2; Sec. 3.1.4] Several load-bearing parameters that directly affect the reproduction outcomes are either fixed without sensitivity analysis or left underspecified. The opinion-polarization experiment sets algorithm_filter_strength = 0.4 (Sec. 7.5.2); the information-cocoon environment applies 'category-diversity constraints and deduplication' whose strengths are not quantified (Sec. 7.4.2); and the context-compression thresholds are not given (Sec. 3.1.4). The paper's qualitative-reproduction claims would be more convincing if these parameters were varied over plausible ranges and the main patterns shown to be robust. As written, it is possible that the reproduced patterns depend on hand-tuned settings, which would weaken the claim that the platform faithfully reproduces prior social-science findings rather than being fitted to them.
minor comments (4)
  1. [Sec. 4.7 / Fig. 14] Figure 14 labels the paper-generation release gate as 'No Human Gates', which appears to conflict with Design Principle 2 (Sec. 2) that emphasizes human approval gates. Please clarify whether the paper-generation stage operates without human intervention or whether the human gate is external to the figure.
  2. [Sec. 7.6/7.7] The quantitative RMSE values (0.0073–0.0188) are reported for disaster mobility but the evaluation protocol — which data are used for ground truth, how the mobility outflux index is computed, and how many simulation runs are averaged — is not fully specified. Please provide these details for reproducibility.
  3. [Sec. 3.2.4] The security restrictions in CodeGenRouter are described only qualitatively ('selectively disable certain Python advanced syntax features'). Please list the exact disabled features and the whitelist of allowed imports so the sandbox guarantee is assessable.
  4. [Sec. 6.3] The overhead analysis is based on 100 agents over 40 steps. Since the headline scale claim is 10,000 agents, it would be helpful to state whether the 70.5–74.1% pre-generation and 66.5% cache reduction figures were measured at that scale or extrapolated.

Circularity Check

2 steps flagged

Self-referential validation loop: the platform labels its own null results as 'informative deviations' and benchmarks its router on its own call traces; no equation-level circularity found.

specific steps
  1. self definitional [Abstract / Sec. 1; operationalized in Secs. 7.1.3, 7.2.3, 7.3.3]
    "Importantly, the few deviations from prior findings are not merely reproduction failures, but informative inconsistencies that suggest new hypotheses about behavioral assumptions, mechanism specification, and contextual boundary conditions."

    The same AI social-scientist pipeline that generates the simulation outputs also classifies its own null results as 'informative deviations'; no external or pre-registered criterion is provided. The paper's own results include no norm collapse in the Norms Game, no sustained free-riding decay, no IAT latency difference (807.7 vs 809.3 ms, p=0.81), and no SRE memory advantage. Because the label is assigned by the system under test, the claimed capacity to 'identify informative deviations' is true by construction, and the headline reproduction claim cannot be independently audited.

  2. other [Sec. 6.1, Benchmark Design]
    "We constructed the benchmark from records of agent calls to the environment interface during actual simulation runs, rather than from artificially authored instructions."

    The 'actual simulation runs' are executed inside AgentSociety2, whose agent–environment routing is performed by CodeGenRouter, the very component this benchmark evaluates. The test distribution is therefore generated by the system under test. While the ground-truth labels may have been annotated independently, the benchmark does not sample from an external task distribution; it measures the router on traces produced by the platform itself. This is a self-referential evaluation loop for the headline 31.6% reliability improvement, though it is not an equation-level reduction.

full rationale

The paper's engineering contributions (code generation, caching, scale) are concrete and measurable, and I found no place where a fitted parameter is renamed as a prediction or where a result is equivalent to its input by an explicit equation. The circularity that exists is in the validation loop: the platform's own AI scientist labels its reproduction failures as 'informative deviations,' and the routing benchmark is built from the platform's own call traces. Both are self-referential rather than equationally circular. The social-science studies do include external comparisons (Qi et al. human baselines, Levy's field experiment, SafeGraph mobility records), so the central empirical content is not wholly constructed from the platform's own outputs. Score 4 reflects partial self-referential validation, not a derivation that reduces to its inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

The platform's central claims rest on three classes of inputs the reader did not pay for: hand-set experimental parameters that shape the demonstrated outcomes (algorithm_filter_strength, norms-game punishment structure, recommendation constraints); domain assumptions about LLM-agent behavior being human-like and about protocol fidelity; and the entities of the platform itself (silicon participants, AI social scientist, CodeGenRouter), none of which has independent external validation. The 95% pattern-reproduction figure is an ad hoc claim with no defined counting method.

free parameters (4)
  • algorithm_filter_strength = 0.4
    Hand-set ranking penalty in NewsPolarizationEnv (Sec. 7.5.2); the exposure/reading polarization results depend on this value and no sensitivity analysis is reported.
  • norms-game effective punishment structure = Axelrod parameters T=3, H=-1, P=-9, E=-2, MP=-9, ME=-2, plus implementation-specific multi-observer punishment
    Sec. 7.1.3 admits the implementation's punishment strength suppresses violations and prevents the expected norm collapse; effective enforcement strength is an implementation-dependent choice that changes the outcome.
  • recommendation-environment category-diversity constraints and deduplication = not specified
    Sec. 7.4.2: added to the DIN recommender in the information-cocoon experiment; directly shapes the entropy/cocoon metrics.
  • context-compression thresholds = not specified
    Sec. 3.1.4: 'predefined thresholds' for light/medium/high pressure compression are never quantified, though they control agent behavior in long-horizon runs.
axioms (5)
  • domain assumption LLM agent behavior is a valid proxy for human behavior in the studied tasks
    Central premise of all seven studies; contradicted by the paper's own null IAT/SRE results (Sec. 7.3.3).
  • domain assumption The AI social scientist's extraction of prior protocols (Axelrod, Fischbacher-Gächter, Qi, Levy) faithfully implements the original experimental conditions
    Sec. 7.1.1/7.2.1/7.5.1: design steps are delegated to the agent; fidelity is asserted rather than independently audited.
  • ad hoc to paper 'Qualitative patterns' are countable and the 95% reproduction figure is meaningful
    Abstract and Sec. 7: no definition of a pattern, no scoring rubric, no inter-rater check.
  • domain assumption The external datasets (ACS, SafeGraph, short-video logs) are accurate and representative
    Table 4 and Sec. 5.4: agent profiles and disaster benchmarks rest on these datasets without error analysis.
  • standard math Statistical machinery (paired t-tests, RMSE) is applied appropriately to agent-generated samples
    Sec. 7.3.3 uses t(133) tests on agents initialized from participant profiles; the i.i.d. assumption for LLM-agent responses is not discussed.
invented entities (3)
  • Silicon participants (LLM agents as human subjects) no independent evidence
    purpose: Stand in for human respondents, players, and residents in all seven studies
    No external benchmark validates agent behavior as human-like; the paper's own IAT (RT difference 807.7 vs 809.3 ms, p=0.81) and SRE results show the proxy fails on core tasks (Sec. 7.3.3).
  • AI social scientist harness no independent evidence
    purpose: Orchestrates literature grounding, hypothesis generation, experiment design/execution, analysis, and manuscript drafting
    Outputs are gated by humans and internal skeptic agents, but there is no external evaluation comparing its hypotheses or manuscripts to human researcher quality (Sec. 4).
  • Agentic environments + CodeGenRouter no independent evidence
    purpose: Translate agent intents into validated environment function calls
    Evaluated only on a benchmark built from the platform's own simulation call records (Sec. 6.1); no external function-calling benchmark is used, despite BFCLv4 being cited for model selection.

pith-pipeline@v1.3.0-alltime-deepseek · 42416 in / 21309 out tokens · 213060 ms · 2026-08-02T11:38:49.942420+00:00 · methodology

0 comments
read the original abstract

AI scientist systems are beginning to automate parts of scientific research, but social science poses a distinct challenge: its objects of inquiry are not merely datasets or laboratory protocols, but integrated social processes involving situated participants, interaction contexts, interventions, and outcomes. Yet a critical link is missing: existing systems either assist isolated research tasks or simulate agents as experimental subjects, leaving the research workflow and simulated society decoupled. Here we introduce AgentSociety 2, an Integrated Research Environment for executable social science. It couples two roles of LLM agents in the same runtime: AI social scientists that coordinate literature grounding, hypothesis generation, experiment design, simulation execution, result interpretation, and manuscript drafting; and silicon participants that generate behavioral responses within configurable social environments. This dual-role design turns hypotheses into auditable agent behaviors, environment rules, interventions, and measurements, thereby supporting an end-to-end workflow. Across seven illustrative studies spanning micro-level social-science laboratory experiments, meso-level dynamics in social media, and macro-level urban scenarios, we demonstrate its capacity to support diverse disciplinary questions, reproduce major qualitative patterns from prior studies, identify informative deviations, and enable large-scale simulations through optimized agent-environment interactions. By preserving human researchers' high-level agency while delegating procedural orchestration to agentic systems, it provides a human-in-the-loop and controllable infrastructure for next-generation computational social science, with broader applications in scalable computational social experimentation and AI-enabled social governance platforms.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

113 extracted references · 19 linked inside Pith

  1. [1]

    Microsoft research Redmond, WA, 2009

    Anthony JG Hey, Stewart Tansley, Kristin Michele Tolle, et al.The fourth paradigm: data-intensive scientific discovery, volume 1. Microsoft research Redmond, WA, 2009

    2009

  2. [2]

    University of Chicago press Chicago, 1970

    Thomas S Kuhn and Ian Hacking.The structure of scientific revolutions, volume 2. University of Chicago press Chicago, 1970

    1970

  3. [3]

    Cambridge university press, 1983

    Ian Hacking.Representing and intervening: Introductory topics in the philosophy of natural science. Cambridge university press, 1983

    1983

  4. [4]

    University of Chicago Press, 2019

    Eric Winsberg.Science in the age of computer simulation. University of Chicago Press, 2019

    2019

  5. [5]

    Computational social science

    David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-László Barabási, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, et al. Computational social science. Science, 323(5915):721–723, 2009

    2009

  6. [6]

    Princeton University Press, 2019

    Matthew J Salganik.Bit by bit: Social research in the digital age. Princeton University Press, 2019

    2019

  7. [7]

    From factors to actors: Computational sociology and agent-based modeling.Annual review of sociology, 28(1):143–166, 2002

    Michael W Macy and Robert Willer. From factors to actors: Computational sociology and agent-based modeling.Annual review of sociology, 28(1):143–166, 2002

    2002

  8. [8]

    Princeton University Press, 2012

    Joshua M Epstein.Generative social science: Studies in agent-based computational modeling. Princeton University Press, 2012

    2012

  9. [9]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  10. [10]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 73 AgentSociety 2: An Integrated Research Environment for Executable Social Science

    Pith/arXiv arXiv 2023

  11. [11]

    Mind2web: Towards a generalist agent for the web.Advances in Neural Information Processing Systems, 36:28091–28114, 2023

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web.Advances in Neural Information Processing Systems, 36:28091–28114, 2023

    2023

  12. [12]

    Natural language to code translation with execution

    Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I Wang. Natural language to code translation with execution. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3533–3546, 2022

    2022

  13. [13]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Pith/arXiv arXiv 2021

  14. [14]

    quantifying github copilot’s impact on developer productivity and happiness

    GitHub. quantifying github copilot’s impact on developer productivity and happiness. 2022

    2022

  15. [15]

    OpenClaw: Personal ai assistant.https://github

    Peter Steinberger and OpenClaw Contributors. OpenClaw: Personal ai assistant.https://github. com/openclaw/openclaw, 2026. GitHub repository

    2026

  16. [16]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Maiden, Li Fei-Fei, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022

    2022

  17. [17]

    Plan- and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models

    Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. Plan- and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...

    doi:10.18653/v1/2023.acl-long.147 2023

  18. [18]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations, 2022

    2022

  19. [19]

    The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models

    Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. InForty-second International Conference on Machine Learning

  20. [20]

    Autonomous chemical research with large language models.Nature, 624(7992):570–578, 2023

    Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models.Nature, 624(7992):570–578, 2023

    2023

  21. [21]

    The virtual lab of ai agents designs new sars-cov-2 nanobodies.Nature, 646(8085):716–723, 2025

    Kyle Swanson, Wesley Wu, Nash L Bulaong, John E Pak, and James Zou. The virtual lab of ai agents designs new sars-cov-2 nanobodies.Nature, 646(8085):716–723, 2025

    2025

  22. [22]

    Towards end-to-end automation of ai research.Nature, 651(8107):914–919, 2026

    Chris Lu, Cong Lu, Robert Tjarko Lange, Yutaro Yamada, Shengran Hu, Jakob Foerster, David Ha, and Jeff Clune. Towards end-to-end automation of ai research.Nature, 651(8107):914–919, 2026

    2026

  23. [23]

    Accelerating scientific discovery with co-scientist.Nature, pages 1–3, 2026

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Petar Sirkovic, Artiom Myaskovsky, Grzegorz Glowaty, Felix Weissenberger, Alessio Orlandi, Dan Popovici, et al. Accelerating scientific discovery with co-scientist.Nature, pages 1–3, 2026

    2026

  24. [24]

    A multi-agent system for automating scientific discovery.Nature, pages 1–3, 2026

    Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J Szostkiewicz, Dmytro Shved, Gavin J Gyimesi, Jon M Laurent, Samantha M Wright, Muhammed T Razzak, et al. A multi-agent system for automating scientific discovery.Nature, pages 1–3, 2026

    2026

  25. [25]

    Omniscientist: Toward a co-evolving ecosystem of human and ai scientists.arXiv preprint arXiv:2511.16931, 2025

    Chenyang Shao, Dehao Huang, Yu Li, Keyu Zhao, Weiquan Lin, Yining Zhang, Qingbin Zeng, Zhiyu Chen, Tianxing Li, Yifei Huang, et al. Omniscientist: Toward a co-evolving ecosystem of human and ai scientists.arXiv preprint arXiv:2511.16931, 2025

    arXiv 2025

  26. [26]

    Autosota: An end-to-end automated research system for state-of-the-art ai model discovery.arXiv preprint arXiv:2604.05550, 2026

    Yu Li, Chenyang Shao, Xinyang Liu, Ruotong Zhao, Peijie Liu, Hongyuan Su, Zhibin Chen, Qinglong Yang, Anjie Xu, Yi Fang, et al. Autosota: An end-to-end automated research system for state-of-the-art ai model discovery.arXiv preprint arXiv:2604.05550, 2026. 74 AgentSociety 2: An Integrated Research Environment for Executable Social Science

    Pith/arXiv arXiv 2026

  27. [27]

    Internagent-1.5: A unified agentic framework for long-horizon autonomous scientific discovery.arXiv preprint arXiv:2602.08990, 2026

    Shiyang Feng, Runmin Ma, Xiangchao Yan, Yue Fan, Yusong Hu, Songtao Huang, Shuaiyu Zhang, Zongsheng Cao, Tianshuo Peng, Jiakang Yuan, et al. Internagent-1.5: A unified agentic framework for long-horizon autonomous scientific discovery.arXiv preprint arXiv:2602.08990, 2026

    arXiv 2026

  28. [28]

    An ai system to help scientists write expert-level empirical software.Nature, pages 1–3, 2026

    Eser Aygün, Anastasiya Belyaeva, Gheorghe Comanici, Marc Coram, Hao Cui, Jake Garrison, Renee Johnston, Anton Kast, Cory Y McLean, Peter Norgaard, et al. An ai system to help scientists write expert-level empirical software.Nature, pages 1–3, 2026

    2026

  29. [29]

    Using large language models to simulate multiple humans and replicate human subject studies

    Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. Using large language models to simulate multiple humans and replicate human subject studies. InInternational conference on machine learning, pages 337–371. PMLR, 2023

    2023

  30. [30]

    Out of one, many: Using language models to simulate human samples.Political Analysis, 31(3):337–351, 2023

    Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples.Political Analysis, 31(3):337–351, 2023

    2023

  31. [31]

    Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023

    John J Horton, Apostolos Filippas, and Benjamin S Manning. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023

    2023

  32. [32]

    Can ai language models replace human participants?Trends in Cognitive Sciences, 27(7):597–600, 2023

    Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. Can ai language models replace human participants?Trends in Cognitive Sciences, 27(7):597–600, 2023

    2023

  33. [33]

    Using large language models in psychology.Nature Reviews Psychology, 2(11):688–701, 2023

    Dorottya Demszky, Diyi Yang, David S Yeager, Christopher J Bryan, Margarett Clapper, Susannah Chandhok, Johannes C Eichstaedt, Cameron Hecht, Jeremy Jamieson, Meghann Johnson, et al. Using large language models in psychology.Nature Reviews Psychology, 2(11):688–701, 2023

    2023

  34. [34]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023

    2023

  35. [35]

    Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors and society.arXiv preprint arXiv:2502.08691, 2025

    Jinghua Piao, Yuwei Yan, Jun Zhang, Nian Li, Junbo Yan, Xiaochong Lan, Zhihong Lu, Zhiheng Zheng, Jing Yi Wang, Di Zhou, et al. Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors and society.arXiv preprint arXiv:2502.08691, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    From individual to society: A survey on social simulation driven by large language model-based agents.ACM Computing Surveys, 58(11):1–41, 2026

    Xinyi Mou, Xuanwen Ding, Qi He, Liang Wang, Jingcong Liang, Xinnong Zhang, Libo Sun, Jiayu Lin, Jie Zhou, Huang Xuanjing, et al. From individual to society: A survey on social simulation driven by large language model-based agents.ACM Computing Surveys, 58(11):1–41, 2026

    2026

  37. [37]

    Oasis: Open agent social interaction simulations with one million agents

    Ziyi Yang, Zaibin Zhang, Zirui Zheng, Yuxian Jiang, Ziyue Gan, Zhiyu Wang, Zijian Ling, Jinsong Chen, Martz Ma, Bowen Dong, et al. Oasis: Open agent social interaction simulations with one million agents. arXiv preprint arXiv:2411.11581, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Yulan-onesim: Towards the next generation of social simulator with large language models

    Lei Wang, Heyang Gao, Xiaohe Bo, Xu Chen, and Ji-Rong Wen. Yulan-onesim: Towards the next generation of social simulator with large language models. InWorkshop on Scaling Environments for Agents, 2025

    2025

  39. [39]

    Large language models empowered agent-based modeling and simulation: A survey and perspectives

    Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1):1–24, 2024

    2024

  40. [40]

    Long-context llms meet rag: Overcoming challenges for long inputs in rag

    Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan Arik. Long-context llms meet rag: Overcoming challenges for long inputs in rag. InInternational Conference on Learning Representations, volume 2025, pages 37784–37822, 2025

    2025

  41. [41]

    Longgenbench: Long-context generation benchmark

    Xiang Liu, Peijie Dong, Xuming Hu, and Xiaowen Chu. Longgenbench: Long-context generation benchmark. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 865–883, 2024

    2024

  42. [42]

    Lost in the middle: How language models use long contexts.Transactions of the association for computational linguistics, 12:157–173, 2024

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the association for computational linguistics, 12:157–173, 2024. 75 AgentSociety 2: An Integrated Research Environment for Executable Social Science

    2024

  43. [43]

    Agent skills

    Anthropic. Agent skills. https://platform.claude.com/docs/en/agents-and-tools/ agent-skills/overview, 2025

    2025

  44. [44]

    Agent skills.https://developers.openai.com/codex/skills, 2026

    OpenAI. Agent skills.https://developers.openai.com/codex/skills, 2026

    2026

  45. [45]

    Skill creator.https://claude.com/plugins/skill-creator, 2026

    Anthropic. Skill creator.https://claude.com/plugins/skill-creator, 2026. Accessed: 2026- 05-28

    2026

  46. [46]

    The ai scientist: Towards fully automated open-ended scientific discovery.arXiv preprint arXiv:2408.06292, 2024

    Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery.arXiv preprint arXiv:2408.06292, 2024

    Pith/arXiv arXiv 2024

  47. [47]

    Researchagent: Iterative research idea generation over scientific literature with large language models

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Researchagent: Iterative research idea generation over scientific literature with large language models. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pag...

    2025

  48. [48]

    Autonomous agents for scientific discovery: Orchestrating scientists, language, code, and physics.arXiv preprint arXiv:2510.09901, 2025

    Lianhao Zhou, Hongyi Ling, Cong Fu, Yepeng Huang, Michael Sun, Wendi Yu, Xiaoxuan Wang, Xiner Li, Xingyu Su, Junkai Zhang, et al. Autonomous agents for scientific discovery: Orchestrating scientists, language, code, and physics.arXiv preprint arXiv:2510.09901, 2025

    Pith/arXiv arXiv 2025

  49. [49]

    Equipping agents for the real world with agent skills, october 2025.URL https://www

    Barry Zhang, Keith Lazuka, and Mahesh Murag. Equipping agents for the real world with agent skills, october 2025.URL https://www. anthropic. com/engineering/equipping-agents-for-the-real-world-with- agent-skills. Accessed, pages 01–28, 2026

    2025

  50. [50]

    Scientific workflows for computational reproducibility in the life sciences: Status, challenges and opportunities.Future Generation Computer Systems, 75:284–298, 2017

    Sarah Cohen-Boulakia, Khalid Belhajjame, Olivier Collin, Jérôme Chopard, Christine Froidevaux, Alban Gaignard, Konrad Hinsen, Pierre Larmande, Yvan Le Bras, Frédéric Lemoine, et al. Scientific workflows for computational reproducibility in the life sciences: Status, challenges and opportunities.Future Generation Computer Systems, 75:284–298, 2017

    2017

  51. [51]

    FS Chirigati and D Shasha. és j. freire,„reprozip: Using provenance to support computational repro- ducibility.”, 2013

    2013

  52. [52]

    The complexities of agent-based modeling output analysis.Journal of Artificial Societies and Social Simulation, 18(4), 2015

    Ju-Sung Lee, Tatiana Filatova, Arika Ligmann-Zielinska, Behrooz Hassani-Mahmooei, Forrest Stonedahl, Iris Lorscheid, Alexey Voinov, J Gareth Polhill, Zhanli Sun, and Dawn C Parker. The complexities of agent-based modeling output analysis.Journal of Artificial Societies and Social Simulation, 18(4), 2015

    2015

  53. [53]

    Census Bureau

    U.S. Census Bureau. 2023 American Community Survey: 1-Year Estimates, Public Use Microdata Sample. Public Use Microdata Sample (PUMS), 2023. URLhttps://catalog.data.gov/dataset/ 2023-american-community-survey-1-year-estimates-public-use-microdata-sample . Accessed: 2026-05-26

    2023

  54. [54]

    Bureau of Labor Statistics

    U.S. Bureau of Labor Statistics. Consumer Expenditure Surveys Public Use Microdata, 2023. Consumer Expenditure Surveys Public Use Microdata, 2023. URLhttps://www.bls.gov/cex/pumd_data. htm. Accessed: 2026-05-26

    2023

  55. [55]

    Panel Study of Income Dynamics, 2023 Family Survey: Public-Use Dataset

    Panel Study of Income Dynamics. Panel Study of Income Dynamics, 2023 Family Survey: Public-Use Dataset. Survey Research Center, Institute for Social Research, University of Michigan, 2023. URL https://psidonline.isr.umich.edu/. Accessed: 2026-05-26

    2023

  56. [56]

    2023 Survey of Household Economics and De- cisionmaking: Survey Data and Codebook

    Board of Governors of the Federal Reserve System. 2023 Survey of Household Economics and De- cisionmaking: Survey Data and Codebook. Public-use survey data, 2023. URL https://www. federalreserve.gov/consumerscommunities/shed_data.htm. Accessed: 2026-05-26

    2023

  57. [57]

    Census Bureau

    U.S. Census Bureau. 2023 Survey of Income and Program Participation. Public- use microdata and API dataset, 2023. URL https://catalog.data.gov/dataset/ 2023-survey-of-income-and-program-participation-sipp. Accessed: 2026-05-26

    2023

  58. [58]

    Census Bureau

    U.S. Census Bureau. American Community Survey 5-Year Data, 2019. ACS 5-year estimates, includ- ing block-group-level detailed tables, 2019. URLhttps://www.census.gov/data/developers/ data-sets/acs-5year.html. Used as source data for census block-group features. Accessed: 2026- 05-26. 76 AgentSociety 2: An Integrated Research Environment for Executable Soc...

    2019

  59. [59]

    FactSet Ownership

    FactSet. FactSet Ownership. Institutional ownership data product, 2026. URLhttps://www.factset. com/marketplace/catalog/product/factset-ownership. Accessed: 2026-05-26

    2026

  60. [60]

    Company News API

    Finnhub. Company News API. Finnhub API Documentation, 2026. URLhttps://finnhub.io/docs/ api/company-news. Accessed: 2026-05-26

    2026

  61. [61]

    Stock Candles API

    Finnhub. Stock Candles API. Finnhub API Documentation, 2026. URLhttps://finnhub.io/docs/ api/stock-candles. Accessed: 2026-05-26

    2026

  62. [62]

    Toolreagt: tool retrieval for llm-based complex task solution via retrieval augmented generation

    Norbert Braunschweiler, Rama Doddipatla, and Tudor-Catalin Zorila. Toolreagt: tool retrieval for llm-based complex task solution via retrieval augmented generation. InProceedings of the 3rd Workshop on Towards Knowledgeable Foundation Models (KnowFM), pages 75–83, 2025

    2025

  63. [63]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  66. [66]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  67. [67]

    An evolutionary approach to norms.American Political Science Review, 80(4):1095–1111, 1986

    Robert Axelrod. An evolutionary approach to norms.American Political Science Review, 80(4):1095–1111, 1986

    1986

  68. [68]

    Social preferences, beliefs, and the dynamics of free riding in public goods experiments.American economic review, 100(1):541–556, 2010

    Urs Fischbacher and Simon Gächter. Social preferences, beliefs, and the dynamics of free riding in public goods experiments.American economic review, 100(1):541–556, 2010

    2010

  69. [69]

    A comprehensive dataset for investigating the structure of self-bias.Scientific Data, 12(1):1755, 2025

    Yuxuan Qi, Fengjie Zou, Xi Ying Chau, Michelle Zhou, Fei Wang, and Jie Sui. A comprehensive dataset for investigating the structure of self-bias.Scientific Data, 12(1):1755, 2025

    2025

  70. [70]

    A large-scale dataset with behavior, attributes, and content of mobile short-video platform

    Yu Shang, Chen Gao, Nian Li, and Yong Li. A large-scale dataset with behavior, attributes, and content of mobile short-video platform. InCompanion Proceedings of the ACM on Web Conference 2025, pages 793–796, 2025

    2025

  71. [71]

    Social media, news consumption, and polarization: Evidence from a field experiment

    Ro’ee Levy. Social media, news consumption, and polarization: Evidence from a field experiment. American Economic Review, 111(3):831–870, March 2021. doi: 10.1257/aer.20191777. URLhttps: //www.aeaweb.org/articles?id=10.1257/aer.20191777

    doi:10.1257/aer.20191777 2021

  72. [72]

    Census Bureau

    U.S. Census Bureau. American Community Survey 5-Year Estimates, 2015–2019. Data set, 2020. URL https://data.census.gov/

    2015

  73. [73]

    SafeGraph Core Places and Patterns Data

    SafeGraph. SafeGraph Core Places and Patterns Data. Data set, 2019. URLhttps://www.safegraph. com/

    2019

  74. [74]

    SafeGraph Core Places and Patterns Data

    SafeGraph. SafeGraph Core Places and Patterns Data. Data set, 2021. URLhttps://www.safegraph. com/

    2021

  75. [75]

    Chain-of-planned-behaviour workflow elicits few-shot mobility generation in llms.arXiv preprint arXiv:2402.09836, 2024

    Chenyang Shao, Fengli Xu, Bingbing Fan, Jingtao Ding, Yuan Yuan, Meng Wang, and Yong Li. Chain-of-planned-behaviour workflow elicits few-shot mobility generation in llms.arXiv preprint arXiv:2402.09836, 2024

    Pith/arXiv arXiv 2024

  76. [76]

    Experimental tests of the endowment effect and the coase theorem.Journal of political Economy, 98(6):1325–1348, 1990

    Daniel Kahneman, Jack L Knetsch, and Richard H Thaler. Experimental tests of the endowment effect and the coase theorem.Journal of political Economy, 98(6):1325–1348, 1990

    1990

  77. [77]

    Illusion and well-being: a social psychological perspective on mental health.Psychological bulletin, 103(2):193, 1988

    Shelley E Taylor and Jonathon D Brown. Illusion and well-being: a social psychological perspective on mental health.Psychological bulletin, 103(2):193, 1988. 77 AgentSociety 2: An Integrated Research Environment for Executable Social Science

    1988

  78. [78]

    Is there a universal need for positive self-regard?Psychological review, 106(4):766, 1999

    Steven J Heine, Darrin R Lehman, Hazel Rose Markus, and Shinobu Kitayama. Is there a universal need for positive self-regard?Psychological review, 106(4):766, 1999

    1999

  79. [79]

    Using the implicit association test to measure self-esteem and self-concept.Journal of personality and social psychology, 79(6):1022, 2000

    Anthony G Greenwald and Shelly D Farnham. Using the implicit association test to measure self-esteem and self-concept.Journal of personality and social psychology, 79(6):1022, 2000

    2000

  80. [80]

    Self-reference and the encoding of personal information.Journal of personality and social psychology, 35(9):677, 1977

    Timothy B Rogers, Nicholas A Kuiper, and William S Kirker. Self-reference and the encoding of personal information.Journal of personality and social psychology, 35(9):677, 1977

    1977

Showing first 80 references.