Pith. sign in

REVIEW 5 major objections 5 minor 29 references

PolicySimEval: A Benchmark for Evaluating Policy Outcomes through Agent-Based Simulation

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PolicySimEval, the first benchmark for evaluating agent-based simulations in policy assessment, reports that current LLM-driven agent frameworks achieve at most 24.5% coverage on expert-annotated comprehensive scenarios.

desk verdict A real gap and a plausible task taxonomy, but the headline difficulty numbers are unvalidated BLEU scores and several metrics are read backward against their own formulas. read the letter →

arxiv 2502.07853 v1 pith:IO64IBBT submitted 2025-02-11 cs.MA cs.CY

Jiaju Kang , Puyu Han , Tian Zhang , Luqi Gong This is my paper · ORCID
classification cs.MAcs.CY
keywords agent-basedmodelingpolicyevaluationbenchmarksimulationlargelanguagemodelsmulti-agentsystemsBLEUcoverageassessment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces PolicySimEval, a benchmark with 285 policy-evaluation tasks for agent-based simulation systems. It claims this is the first benchmark aimed at scoring how well such systems can model complex social scenarios end-to-end, and it documents that current LLM-based agent frameworks fall far short: the best system reaches 24.5% coverage on comprehensive scenarios, 15.04% on sub-tasks, and 14.5% on auto-generated tasks. A sympathetic reader would take this as strong evidence that agent-based simulation for real-world policy evaluation is still at an early stage, and that progress needs a standardized test bed.

What carries the argument

The load-bearing object is the benchmark itself: three task families that require an agent to configure, run, and interpret an agent-based simulation of a policy question, plus a five-category metric suite (argument coverage, behavior calibration, language and ethics, outcome effectiveness, system performance). The headline metric is coverage rate, defined as $\mathrm{R_{cover}} = \mathrm{BLEU}(\text{Generated Text}, \text{Reference Text})$. Together these convert a qualitative judgment about policy expertise into a numerical score, and the paper uses that score to compare baseline agents.

What would settle it

Ask a panel of policy experts to score the same model outputs for coverage of the required policy points without seeing the reference texts, then correlate their scores with the BLEU-based coverage rates. If the correlation is weak or negative, the claim that state-of-the-art systems 'struggle' at 24.5% coverage would not be established by this benchmark.

Watch

Extended reading notes

Core claim

The central claim is that no existing agent-based simulation system can yet handle realistic policy-evaluation tasks at a level usable for decisions. The paper supports this by constructing 20 comprehensive scenarios with expert-annotated reference solutions, 65 targeted sub-tasks, and 200 auto-generated tasks, then running two agent architectures (ReAct and a retrieval-augmented ReAct variant) on a commercial and an open-source large language model. Its headline numbers are BLEU-based coverage rates between generated policy analysis and expert reference text, and the paper reports that the best configuration, the retrieval-augmented ReAct variant on the commercial model, achieves only 24.5% coverage on comprehensive scenarios.

Load-bearing premise

The headline difficulty numbers rest on the assumption that BLEU similarity between an agent's generated text and an expert reference text measures how much of the policy outcome the system actually got right, and that the expert references are fair gold standards.

Editorial extensions

If this is right

  • If PolicySimEval's scores are accepted, then any agent-based policy-simulation tool must clear a low bar: the current best system covers less than a quarter of expert-annotated outcomes on comprehensive tasks.
  • The 65 sub-tasks isolate specific bottlenecks (behavior calibration, data integration, hyperparameter tuning), so future work can target those weak points directly.
  • The 200 auto-generated tasks give a large-scale, low-cost way to benchmark and fine-tune new methods without hand-labeling.
  • The multi-metric framework means that raw text coverage alone is not enough; systems will also be judged on calibration error, ethics, outcome alignment, and response time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test of the benchmark's validity would be to have policy experts score the same model outputs for substantive completeness and compare those ratings to the BLEU-based coverage rates; if the two disagree, the headline gap numbers would need reinterpreting.
  • Because the expert reference texts are not shown in the paper, an immediate extension is to release a sample and let independent groups re-score the baselines, confirming whether the 24.5%, 15.04%, and 14.5% figures are stable.
  • One could extend the benchmark beyond text similarity to measure whether the agent's simulated policy outcomes match real historical or empirical outcomes, which is the yardstick that ultimately matters for policy decisions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. PolicySimEval introduces a benchmark for evaluating agent-based simulation systems on policy-assessment tasks, with 20 comprehensive scenarios, 65 targeted sub-tasks, and 200 auto-generated tasks, plus a taxonomy of metrics across argument coverage, behavior calibration, language and ethics, outcome effectiveness, and system performance. The paper reports experiments with ReAct and ReAct-RAG agents built on GPT-4o and Llama 3.1 70B, and concludes that current state-of-the-art frameworks struggle because the best system attains only 24.5% coverage on comprehensive scenarios, 15.04% on sub-tasks, and 14.5% on auto-generated tasks.

Significance. If the benchmark, its gold references, and its metric validations were sound, PolicySimEval could be a useful contribution to a largely unbenchmarked area: evaluating LLM-driven agent-based simulation for policy analysis. The task design is structured and the breadth of the metric taxonomy is a reasonable starting point. However, the central empirical claim depends on a lexical-overlap score called 'coverage rate' with no validation against human judgment, and several metric definitions are contradicted by the reported numbers and their interpretations. As presented, the paper does not establish its headline difficulty results, and the contribution is therefore not yet ready for the claims made in the abstract.

major comments (5)
  1. [Table I, Table II, Abstract] The headline results (24.5% on comprehensive scenarios, 15.04% on sub-tasks, 14.5% on auto-generated tasks) are explicitly Rcover values, defined in Table I as BLEU(Generated Text, Reference Text). BLEU is lexical n-gram overlap, not a coverage rate of policy outcomes: a system can phrase all required elements differently and score near zero, or echo the reference phrasing while covering few substantive points. The paper provides no human evaluation, inter-annotator agreement, or example gold references to support BLEU as a measure of policy-outcome coverage. Consequently, the abstract's claim that current frameworks 'struggle' is not established by the reported numbers.
  2. [Table I, Table V] The formula for Outcome Alignment, Ar = 1 - (1/M) sum_i ||Oi - Ohat_i||, cannot exceed 1 for any nonnegative norm, yet Table V reports Ar values up to 15.26 and 15.72. As written, the formula and the reported results are mutually inconsistent, so the outcome-alignment scores and the text saying that higher Ar indicates 'better alignment' are unsupported. The authors need either to correct the formula, report the units/normalization, or explain the discrepancy.
  3. [Table I, Table V, Table VI] The interpretations of Ta and Sv contradict their own definitions. Table I defines Ta = Time(Updated Result) - Time(Input Change), which is a latency where lower is better, but the Section V discussion states that ReAct-RAG with GPT-4o achieves the 'highest Ta (38.90), indicating ... faster updates.' Similarly, Sv = max(F) - min(F) is a fluctuation range, where lower means more stable, but the text says that a high Sv (16.96) 'reflect[s] greater stability.' These reversals mean the system-performance and dynamic-adjustment conclusions are not derivable from the specified metrics.
  4. [Section IV, Section V] The experiments cover only two agent architectures (ReAct and ReAct-RAG) built on two LLMs (GPT-4o and Llama 3.1 70B), with no repeated runs, variance estimates, or significance tests. The Related Work discusses classic ABM frameworks such as NetLogo, MASON, and Repast, but these are not evaluated. The paper's conclusion that 'current state-of-the-art frameworks struggle' therefore overgeneralizes beyond the tested configurations and does not support a robust statement about the state of the art.
  5. [Section III-A, Section III (overall)] The benchmark itself is the main contribution, yet no dataset, gold-standard answers, task-generation code, or evaluation scripts are released. Section III-A says gold-standard answers are 'manually curated by domain experts,' but the paper reports no inter-annotator agreement, no sample tasks, and no example outputs or references. Because the reference texts are not available and the scoring choices are all author-defined, an independent researcher cannot verify the reported scores or the difficulty claims; the benchmark is not reproducible as described.
minor comments (5)
  1. [Abstract vs. Section III] The paper interchangeably uses 'coverage rate' (abstract) and 'accuracy' (introduction) for the same reported figures; this should be made consistent.
  2. [Figure 1] The figure contains a misspelling: 'Evaluater' should be 'Evaluator.'
  3. [Section IV] The text uses 'automated generation tasks' in one paragraph and 'auto-generated tasks' elsewhere; unify the terminology.
  4. [References] Reference [18] (Grabrucker and Grimm, an agricultural-shocks study) appears unrelated to the supporting claim about ensemble modeling for ABM calibration; the citation should be checked.
  5. [Table I] The Ql formula sums 'Grammar Score' and 'Semantic Score' without stating their ranges or whether Ql is normalized; the table would be clearer with explicit value ranges for all metrics.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the benchmark's difficulty numbers are empirical scores from an external metric (BLEU) against fixed gold references, not fitted parameters or self-cited premises.

full rationale

The derivation chain in PolicySimEval is not circular. The headline claims ('highest-performing system achieving only 24.5% coverage rate on comprehensive scenarios, 15.04% on sub-tasks, and 14.5% on auto-generated tasks') are empirical results computed with the metric Rcover = BLEU(Generated Text, Reference Text), as defined in Table I. BLEU is an external, pre-existing similarity measure, not a quantity defined in terms of the paper's conclusion; the low scores are measurements against independently curated expert references, not consequences of the benchmark's construction. No parameter is fitted to a subset of data and then renamed as a prediction. No load-bearing self-citation appears: references [1]-[29] are external works, and Figure 1's use of an external multi-agent framework [6] is a design choice, not an argument whose conclusion depends on that citation. No uniqueness theorem from prior work by the same authors is invoked. The paper does contain serious metric-validity and internal-consistency problems that are outside circularity: calling BLEU 'coverage rate' conflates lexical overlap with substantive policy coverage; Table V reports Ar values up to 15.26 although the stated formula Ar = 1 - (1/M) * sum ||Oi - Ohat_i|| cannot exceed 1; and the text interprets higher Ta as 'faster updates' and higher Sv as 'greater stability' even though Ta is a time difference where lower is faster and Sv is a range where lower is more stable. These are correctness and interpretability concerns, but they do not make the reported results equivalent to the paper's inputs by construction. The benchmark's self-contained gold-standard evaluation gives the central claim independent empirical content, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no physical entities and no fitted constants. Its central evaluation rests on the assumptions listed above: treating BLEU as coverage, treating LLM wrappers as representative ABM frameworks, and trusting unvalidated expert gold answers plus unvalidated auto-generated tasks. The token and time limits are hand-chosen experimental constraints that influence completion but are not fitted to data.

free parameters (3)
  • per-task time limit = 40 minutes
    Section IV sets a 40-minute execution cap (excluding API time). This cap affects task completion and response-time metrics; the value is chosen by hand.
  • sub-task token limit = 400k tokens
    Section IV caps targeted sub-tasks at 400k tokens, which constrains ReAct and ReAct-RAG behavior. The value is chosen by the authors.
  • comprehensive and auto-generated task token limit = 600k tokens
    Section IV caps comprehensive scenarios and auto-generated tasks at 600k tokens. This is a hand-chosen constraint that influences the reported scores.
assumptions (4)
  • domain assumption BLEU/ROUGE similarity to a gold text is a valid measure of policy outcome coverage.
    Table I defines the Coverage Rate as BLEU(Generated Text, Reference Text), and the abstract's 24.5% headline is this BLEU score. BLEU measures lexical overlap, not substantive coverage of policy outcomes.
  • domain assumption The two LLM-agent wrappers (ReAct and ReAct-RAG) represent current state-of-the-art agent-based simulation frameworks for policy evaluation.
    The conclusion that 'current state-of-the-art frameworks struggle' depends on treating these wrappers as representative ABM systems, but no traditional ABM framework such as MASON, Repast, NetLogo, or OpenABM is actually run.
  • domain assumption Expert-curated gold answers are an objective ground truth.
    Section III-A says gold standards are 'manually curated by domain experts', but no expert count, rubric, or inter-annotator agreement is given, making the ground-truth objectivity an assumption.
  • domain assumption Automatically generated tasks are valid and comparable in difficulty to expert-written tasks.
    Section III introduces 200 'programmatically created' tasks but reports no validation, difficulty audit, or filtering against expert judgment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PolicySimEval: A Benchmark for Evaluating Policy Outcomes through Agent-Based Simulation." pith.science (2026). https://pith.science/paper/IO64IBBT

@misc{pith2026250207853,
  author       = {Pith},
  title        = {Pith review of: PolicySimEval: A Benchmark for Evaluating Policy Outcomes through Agent-Based Simulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IO64IBBT}},
  note         = {Machine review of arXiv:2502.07853}
}
read the original abstract

With the growing adoption of agent-based models in policy evaluation, a pressing question arises: Can such systems effectively simulate and analyze complex social scenarios to inform policy decisions? Addressing this challenge could significantly enhance the policy-making process, offering researchers and practitioners a systematic way to validate, explore, and refine policy outcomes. To advance this goal, we introduce PolicySimEval, the first benchmark designed to evaluate the capability of agent-based simulations in policy assessment tasks. PolicySimEval aims to reflect the real-world complexities faced by social scientists and policymakers. The benchmark is composed of three categories of evaluation tasks: (1) 20 comprehensive scenarios that replicate end-to-end policy modeling challenges, complete with annotated expert solutions; (2) 65 targeted sub-tasks that address specific aspects of agent-based simulation (e.g., agent behavior calibration); and (3) 200 auto-generated tasks to enable large-scale evaluation and method development. Experiments show that current state-of-the-art frameworks struggle to tackle these tasks effectively, with the highest-performing system achieving only 24.5\% coverage rate on comprehensive scenarios, 15.04\% on sub-tasks, and 14.5\% on auto-generated tasks. These results highlight the difficulty of the task and the gap between current capabilities and the requirements for real-world policy evaluation.

Figures

Figures reproduced from arXiv: 2502.07853 by the authors.

Figure 1
Figure 1. Inspired by [6], we adopted this open-source and efficient multi-agent framework as the foundation for multi-agent communication in task construction. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages

  1. [1]

    Agent based modeling for agricultural policy evaluation: A review,

    Dimitrios Kremmydas, “Agent based modeling for agricultural policy evaluation: A review,” 2012

    work page 2012

  2. [2]

    Agent-based modeling as organizational and public policy simulators,

    Robert Lempert, “Agent-based modeling as organizational and public policy simulators,” Proceedings of the National Academy of Sciences , vol. 99, no. suppl 3, pp. 7195–7196, 2002

    work page 2002

  3. [3]

    Considerations and best practices in agent-based modeling to inform policy,

    Ross A Hammond, “Considerations and best practices in agent-based modeling to inform policy,” in Assessing the Use of Agent-Based Models for Tobacco Regulation , Rodrick Wallace, Alan Geller, and Vivian A Ogawa, Eds., Committee on the Assessment of Agent-Based Models to Inform Tobacco Product Regulation; Board on Population Health and Public Health Practi...

    work page 2015

  4. [4]

    Agent-based models for policy analysis,

    Lawrence Blume, “Agent-based models for policy analysis,” in Assessing the Use of Agent-Based Models for Tobacco Regulation , Rodrick Wallace, Alan Geller, and Vivian A Ogawa, Eds., Committee on the Assessment of Agent-Based Models to Inform Tobacco Product Regulation; Board on Population Health and Public Health Practice; Institute of Medicine, chapter A...

    work page 2015

  5. [5]

    Hybrid agent-based simulation for policy evaluation of solar power generation systems,

    Jiayun Zhao, Esfandyar Mazhari, Nurcin Celik, and Young-Jun Son, “Hybrid agent-based simulation for policy evaluation of solar power generation systems,” Simulation Modelling Practice and Theory , vol. 19, no. 10, pp. 2189–2205, 2011

    work page 2011

  6. [6]

    Multi-agent large language models for conversational task-solving,

    Jonas Becker, “Multi-agent large language models for conversational task-solving,” 2024

    work page 2024

  7. [7]

    Mason: A new multi-agent simulation toolkit,

    Sean Luke, Claudio Cioffi, Liviu Panait, and Keith Sullivan, “Mason: A new multi-agent simulation toolkit,” Proceedings of the 2004 SwarmFest Workshop, 01 2004

    work page 2004

  8. [8]

    Repast: An extensible framework for agent simulation,

    Nick Collier, “Repast: An extensible framework for agent simulation,” Natural Resources and Environmental Issues , vol. 8, no. 1, pp. Article 4, 2001

    work page 2001

Show all 29 references
  1. [9]

    Netlogo: A simple environment for modeling complexity,

    Seth Tisue and Uri Wilensky, “Netlogo: A simple environment for modeling complexity,” in International conference on complex systems . Citeseer, 2004, vol. 21, pp. 16–21

    2004

  2. [10]

    Empirical validation of agent-based models: Alternatives and prospects,

    Paul Windrum, Giorgio Fagiolo, and Alessio Moneta, “Empirical validation of agent-based models: Alternatives and prospects,” J. Artif. Soc. Soc. Simul. , vol. 10, 2007

    2007

  3. [11]

    Openabm-covid19—an agent-based model for non-pharmaceutical in- terventions against covid-19 including contact tracing,

    R Hinch, WJM Probert, A Nurtay, M Kendall, C Wymant, M Hall, et al., “Openabm-covid19—an agent-based model for non-pharmaceutical in- terventions against covid-19 including contact tracing,” PLoS Compu- tational Biology, vol. 17, no. 7, pp. e1009146, Jul. 2021

    2021

  4. [12]

    Empirically based, agent-based models,

    Marco A. Janssen and Elinor Ostrom, “Empirically based, agent-based models,” Ecology and Society , vol. 11, no. 2, 2006, Accessed: 23 Dec. 2024

    2006

  5. [13]

    Agent-based modeling for urban development planning based on human needs. conceptual basis and model formulation,

    Mauricio Gonz ´alez-M´endez, Camilo Olaya, Isidoro Fasolino, Michele Grimaldi, and Nelson Obreg ´on, “Agent-based modeling for urban development planning based on human needs. conceptual basis and model formulation,” Land Use Policy, vol. 101, pp. 105110, 2021

    2021

  6. [14]

    Op- timization and control of agent-based models in biology: A perspective,

    G. An, B. G. Fitzpatrick, S. Christley, P. Federico, A. Kanarek, R. Miller Neilan, M. Oremland, R. Salinas, R. Laubenbacher, and S. Lenhart, “Op- timization and control of agent-based models in biology: A perspective,” Bulletin of Mathematical Biology , vol. 79, no. 1, pp. 63–...

    2017

  7. [15]

    Challenges, tasks, and opportunities in modeling agent-based complex systems,

    Li An, V olker Grimm, Abigail Sullivan, B.L. Turner II, Nicolas Malle- son, Alison Heppenstall, Christian Vincenot, Derek Robinson, Xinyue Ye, Jianguo Liu, Emilie Lindkvist, and Wenwu Tang, “Challenges, tasks, and opportunities in modeling agent-based complex systems,” Ecologi...

    2021

  8. [16]

    Agent-based modelling of social-ecological systems: Achievements, challenges, and a way forward,

    Jule Thober, Birgit M ¨uller, J ¨urgen Groeneveld, and V olker Grimm, “Agent-based modelling of social-ecological systems: Achievements, challenges, and a way forward,” Journal of Artificial Societies and Social Simulation, vol. 20, no. 2, pp. 8, 2017

    2017

  9. [17]

    Agent-based model calibration using machine learning surrogates,

    Francesco Lamperti, Andrea Roventini, and Amir Sani, “Agent-based model calibration using machine learning surrogates,” Journal of Economic Dynamics and Control , vol. 90, no. C, pp. 366–389, 2018

    2018

  10. [18]

    Is there a rainbow after the rain? how do agricultural shocks affect non-farm enterprises? evidence from thailand,

    Katharina Grabrucker and Michael Grimm, “Is there a rainbow after the rain? how do agricultural shocks affect non-farm enterprises? evidence from thailand,” American Journal of Agricultural Economics , vol. 103, no. 5, pp. 1612–1636, 2021

    2021

  11. [19]

    Representation of decision-making in european agricultural agent-based models,

    Robert Huber, Martha Bakker, Alfons Balmann, Thomas Berger, Mike Bithell, Calum Brown, Adrienne Gr ˆet-Regamey, Hang Xiong, Quang Bao Le, Gabriele Mack, Patrick Meyfroidt, James Millington, Birgit M ¨uller, J. Gareth Polhill, Zhanli Sun, Roman Seidl, Christian Troost, and Robe...

    2018

  12. [20]

    The complexities of agent-based modeling output analysis,

    Ju-Sung Lee, Tatiana Filatova, Arika Ligmann-Zielinska, Behrooz Hassani-Mahmooei, Forrest Stonedahl, Iris Lorscheid, Alexey V oinov, J. Gareth Polhill, Zhanli Sun, and Dawn C. Parker, “The complexities of agent-based modeling output analysis,” Journal of Artificial Societies a...

    2015

  13. [21]

    The development of an agent-based modeling framework for simulating engineering team work,

    Richard M. Crowder, Mark A. Robinson, Helen P. N. Hughes, and Yee- Wai Sim, “The development of an agent-based modeling framework for simulating engineering team work,” IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans , vol. 42, no. 6, pp. 1425–1...

    2012

  14. [22]

    Integrated framework for space- and energy-efficient retrofitting in multifunctional buildings: A synergy of agent-based mod- eling and performance-based modeling,

    Yuchi Shen, Xinyi Hu, Xiaotong Wang, Mengting Zhang, Lirui Deng, and Wei Wang, “Integrated framework for space- and energy-efficient retrofitting in multifunctional buildings: A synergy of agent-based mod- eling and performance-based modeling,” Building Simulation , vol. 17, n...

    2024

  15. [23]

    Empirically grounded agent-based models of innovation diffusion: a critical review,

    Haifeng Zhang and Yevgeniy V orobeychik, “Empirically grounded agent-based models of innovation diffusion: a critical review,” Artificial Intelligence Review, vol. 52, no. 1, pp. 707–741, Jun. 2019

    2019

  16. [24]

    Pacar: Covid-19 pandemic control decision making via large-scale agent-based modeling and deep rein- forcement learning,

    X Guo, P Chen, S Liang, et al., “Pacar: Covid-19 pandemic control decision making via large-scale agent-based modeling and deep rein- forcement learning,” Medical Decision Making , vol. 42, no. 8, pp. 1064–1077, 2022

    2022

  17. [25]

    Macroeconomic policy in dsge and agent-based models redux: New developments and challenges ahead,

    Giorgio Fagiolo and Andrea Roventini, “Macroeconomic policy in dsge and agent-based models redux: New developments and challenges ahead,” SSRN Scholarly Paper ID 2763735, SSRN, Rochester, NY , Nov. 2016

    2016

  18. [26]

    Sepsis reconsidered: Identifying novel metrics for behavioral landscape characterization with a high- performance computing implementation of an agent-based model,

    Chase Cockrell and Gary An, “Sepsis reconsidered: Identifying novel metrics for behavioral landscape characterization with a high- performance computing implementation of an agent-based model,” Jour- nal of Theoretical Biology , vol. 430, pp. 157–168, 2017

    2017

  19. [27]

    Large-scale agent- based simulations of online social networks,

    Goran Muri ´c, Alexey Tregubov, Jim Blythe, Andr ´es Abeliuk, Divya Choudhary, Kristina Lerman, and Emilio Ferrara, “Large-scale agent- based simulations of online social networks,” Autonomous Agents and Multi-Agent Systems, vol. 36, no. 2, pp. 38, Jun. 2022

    2022

  20. [28]

    Agent-based modelling as a foundation for big data,

    Shu-Heng Chen and Ragupathy Venkatachalam, “Agent-based modelling as a foundation for big data,” Journal of Economic Methodology , vol. 24, no. 4, pp. 362–383, 2017

    2017

  21. [29]

    React: Synergizing reasoning and acting in language models,

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao, “React: Synergizing reasoning and acting in language models,” 2023

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.