Pith. sign in

REVIEW 3 major objections 6 minor 44 references

Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes that LLM-based agent-based models of opinion dynamics are highly sensitive to small, semantically equivalent prompt changes, and that without a reference model there is no way to know what a prompt change will do.

desk verdict Useful evaluation framework for LLM-ABMs, but the headline sensitivity result conflates task changes with arbitrary prompt noise; still worth refereeing. read the letter →

arxiv 2412.05093 v1 pith:OUQUE6IL submitted 2024-12-06 cs.CY

Da Ju , Adina Williams , Brian Karrer , Maximilian Nickel This is my paper · ORCID
classification cs.CY
keywords agent-basedmodelslargelanguageopiniondynamicspromptsensitivityevaluationframeworksystemidentificationDeGrootmodelHegselmann-Krause
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that replacing classical agent-based models (ABMs) of social dynamics with large language model (LLM) agents is not yet reliable for scientific theory-building. It proposes an evaluation framework that treats the LLM as a black-box function and measures its simulation error against the known dynamics of the DeGroot and Hegselmann-Krause opinion models. Using this framework, the authors show that with careful prompt engineering an LLM-ABM can approximate the reference dynamics, with polarity agreement above 90 percent in the base configuration. But small, intended-to-be-semantically-identical prompt changes—rewording the decoding question, switching the output scale, even removing newlines and apostrophes—can quadruple the error rate or move the simulation off target. The paper concludes that if semantically equivalent changes alter outcomes this much, the micro-behavior encoded in prompts cannot be treated as the source of reliable insights.

What carries the argument

The machinery is a system-identification-style evaluation loop. The population state $\boldsymbol{x}(t)$ is encoded by $e$ into a text opinion, each agent updates its text opinion $g$ from a timeline of posts in its ego network within the confidence bound $\epsilon$ (for Hegselmann-Krause), and $d$ decodes the result back to a scalar. The comparison quantity is the expected simulation error $C(f,h_\theta)$ averaged over time steps, with encoding-decoding consistency checked separately and prompt sensitivity defined as the absolute difference in consistency between the base prompt set and a slightly varied one. This loop turns "how well does the LLM follow its instructions?" into a measurable input-output deviation against a known opinion-dynamics model.

What would settle it

Take the base Hegselmann-Krause prompt set and build, say, 50 variants that human annotators independently judge to ask exactly the same question with the same response options and the same scale, differing only in word choice and whitespace. If the polarity error and distance stay within the base configuration's confidence intervals across all variants, the paper's central sensitivity claim is falsified; if even these controlled variants scatter widely, it is confirmed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that LLM-based agent-based simulations do not have a stable relationship to the natural-language instructions that define them. Treating the LLM as a black box $h_\theta = d \circ g \circ e$, where $e$ encodes scalar opinions as text, $g$ updates opinions from neighbor posts, and $d$ decodes text back to scalars, the authors measure consistency with reference dynamics via the expected simulation error $C(f,h_\theta)$ and sensitivity as the absolute difference in consistency between base and slightly varied prompts. They first show that the base prompt set reproduces Hegselmann-Krause dynamics reasonably well, with distance near 1.0 to 1.4 on a [-5,5] scale and polarity mismatches mostly below 10 to 16 percent, then show that one rewording of the decoding prompt raises polarity mismatches from 11.8 percent to 44.0 percent, a logic-scale change raises them to 49.3 percent, and newline and apostrophe removal shifts distance and polarity across many configurations. The conclusion drawn is that without a reference model to ground the prompts, there is no way to know a priori what a seemingly meaningless prompt change will do to the simulation.

Load-bearing premise

The load-bearing premise is that the prompt variants used in the sensitivity experiments are truly semantically equivalent to the base prompt; one variant changes a yes/no support question into a forced choice between two economic systems, and another changes a two-step polarity-plus-magnitude output into a direct 11-point scale, so if those changes are not semantically neutral, some of the measured sensitivity is really sensitivity to a different task rather than to arbitrary prompt noise.

Editorial extensions

If this is right

  • If the central claim is right, LLM-ABM results should always be reported with a stability analysis across paraphrases, otherwise any emergent pattern could be a prompt artifact.
  • When a base prompt set fits a reference model, fitting alone is insufficient evidence that the simulation is meaningful, because near-equivalent variants diverge.
  • Reference-model grounding provides a cheap calibration signal that should be used before drawing social-science conclusions from LLM-ABMs.
  • The sensitivity result extends beyond opinion dynamics: any LLM-based ABM whose micro-behavior is specified in natural language inherits the same identification problem.
  • Neither larger nor newer LLMs resolve the issue, since the paper finds that one model is not consistently more robust than the other, suggesting the problem is structural to current LLMs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical extension would be to treat prompt sensitivity as a diagnostic: automated prompt search methods could be judged not only by average fit to a reference model but by the variance across paraphrases, with lower variance preferred.
  • If sensitivity persists under rigorous human-annotated semantic equivalence tests, where the response task and output scale are held identical and only word substitutions and whitespace change, the paper's negative conclusion would be strengthened to a claim about LLM brittleness rather than about evaluation design.
  • One could test whether sensitivity shrinks with model scale or training by measuring the sensitivity statistic across many model sizes on the same reference task, which would show whether the problem is likely to disappear with better models or is intrinsic to the generation process.
  • For applied work, these results imply that a single LLM-ABM run should never be read as a prediction; the variance across prompts is part of the model's actual output, so policy or theory claims should average over that variance.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a framework for evaluating LLM-based agent-based models (LLM-ABMs) of opinion dynamics by grounding them in classical reference models, specifically DeGroot and Hegselmann-Krause dynamics. The LLM is treated as a black-box transition function that maps a population state to its successor state, and the framework measures two quantities: consistency, defined as the simulation error relative to the reference model, and sensitivity, defined as the change in consistency when prompts are varied. The authors instantiate the framework with LLaMA-2 and LLaMA-3, PersonaChat-based personas, Erdős-Rényi networks, and hand-written encoding, dynamics, and decoding prompts. They report that a base prompt approximates the reference dynamics reasonably well, but that three 'semantically identical' prompt variants (wording change, logic/format change, and newline/apostrophe removal) produce substantial changes in distance and polarity metrics. They conclude that current LLM-ABMs are too sensitive to prompt choice to yield reliable insights for theory-building.

Significance. If the central claim survives scrutiny, this is an important cautionary result for the rapidly growing LLM-ABM literature. The framework itself is a useful contribution: treating the LLM as a black-box system-identification problem and comparing against established opinion-dynamics models is a clean, general evaluation protocol. The paper also provides meaningful baselines (zero and random agents), reports confidence intervals in all tables, and describes a scalable inference infrastructure with a stated plan to release code. However, the headline sensitivity result is currently overclaimed: two of the three prompt variants change the decoding task or response format, not merely the wording, and the only truly arbitrary variant shows smaller and less consistent effects. The paper also contains an internal inconsistency in the reported accuracy of the base model. With corrections to the claims and additional control/equivalence experiments, the work would be a solid contribution to the evaluation and validation of LLM-based simulations.

major comments (3)
  1. [Section 5 'Sensitivity to prompts' and Appendix A.1 (Tables 2-3)] The central claim that the prompt variants are 'semantically identical' to the base prompt fails on inspection of the actual prompts. The base decoding prompt asks a yes/no question ('do they support {topic}? Reply yes or no'), whereas the θwording prompt replaces this with a forced binary choice between 'planned economy' and 'free market economy'. A yes/no question about a single proposition and a two-alternative forced choice have different decision boundaries, especially for neutral, mixed, or weakly opposed opinions, so the 4x polarity increase in Table 1a for θwording is partly a task change rather than arbitrary wording noise. Similarly, θlogic replaces the two-stage polarity-plus-magnitude decoding with a direct 11-point scale, which changes the response format and can change the LLM's output distribution even under identical intended semantics. The only cleanly arbitrary variant, newline and apostrophe removal, shows smaller and less consistent effects in Tables 8-9, with several differences falling within the reported 95% confidence intervals (e.g., distance for LLaMA-2 at N=20, p=0.3, ε=0.5: 1.2 (0.4) vs 1.3 (0.3)). The paper should either restrict the 'arbitrary variation' claim to the clean text-format variant, or add equivalence controls (e.g., human/LLM ratings of semantic equivalence, and a forced-choice control that includes a neutral option) and re-run the analysis.
  2. [Section 5 'Model consistency' and Table 6] The statement that 'polarity is inaccurate in less than 10% cases' is not supported by the reported data. Table 6 lists many LLaMA-2 configurations with polarity error above 10%, including 14.4%, 13.5%, 15.7%, 15.5%, 11.6%, and 11.8% for N=20, and LLaMA-3 configurations including 14.5%, 11.8%, 11.2%, and 10.8%. Since Section 3 explicitly states that without consistency 'any observed prompt variations are not informative', the sensitivity interpretation is fragile for exactly the configurations where the base model is not consistent. The claim must be corrected, and consistency should be reported separately for the configurations used in the sensitivity analysis.
  3. [Section 5 and Section 7 (Discussion)] The paper's broad conclusion that 'no insights can be gained for theory-building and analysis by analyzing the microbehavior encoded in the prompts' depends on the premise that the observed sensitivity is due to arbitrary, semantically neutral prompt variations. Given the task/format confounds described above, what is currently demonstrated is sensitivity to response-format and task changes, plus a weaker and less consistent sensitivity to text-level formatting. The authors should either narrow the conclusion to 'sensitivity to prompt and response-format variations', or provide additional evidence that the wording and logic variants are truly arbitrary with respect to the intended semantics. Without this, the headline claim overreaches the presented evidence.
minor comments (6)
  1. [Section 5 'Prompt wording'] The text says that in θbase the agent is asked 'if the polarity of a post equals 1 and selecting -1 otherwise' and that θwording asks 'whether the polarity equals 1 or -1', but this description does not match the actual prompts in Appendix A.1 (Tables 2 and 3); the wording in the main text should be aligned with the appendix.
  2. [Section 4] The phrase 'large number of calls to an LMM' should read 'LLM'.
  3. [Table 1 caption] The word 'Negativitiy' is misspelled; it should be 'Negativity'.
  4. [Tables 5-7] Several LLaMA-2 entries for N=50 are missing without explanation in the table captions; a footnote should clarify that these configurations exceed the context length of LLaMA-2, as mentioned in Section 4.
  5. [Figure 4] The legend in Figure 4 is confusing, with duplicate labels such as 'LLaMA-2 LLaMA-2'; the figure should be relabeled so that each method appears once and the legend distinguishes the line style and color.
  6. [Experimental setup] The paper does not state how many independent runs or random seeds were used to compute the confidence intervals, nor whether temperature or other sampling parameters were fixed; reporting this information would make the comparisons in Tables 1, 5, 6, 8, and 9 more interpretable.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the consistency and sensitivity measurements are anchored to external DeGroot/Hegselmann-Krause benchmarks, and the prompt variants are empirical perturbations rather than fitted outputs; minor self-citations are not load-bearing.

full rationale

The paper's derivation chain is: (i) define reference models DeGroot and Hegselmann-Krause from the literature; (ii) define inconsistency as simulation error against those reference models (Eq. 6); (iii) define sensitivity as the absolute difference between inconsistency values for a base prompt and perturbed prompts (Eq. 9); (iv) measure these quantities using LLM agents with fixed persona, encoding, dynamics, and decoding prompts. No step fits a parameter to the measured quantity and then re-predicts it: the base prompt is manually engineered, not fitted by minimizing C(f,h_theta), and the sensitivity statistic is a comparison of two independently evaluated inconsistencies. The reference model output f(x) is computed from the classical update equations, not from the LLM, so the evaluation does not reduce to the LLM's own output. The only self-citations (Karrer and Newman 2011 for stochastic block models; Touvron et al. 2023 for LLAMA-2; Goodarzi et al. 2023 for prompt sensitivity) are background citations and are not load-bearing; the paper's sensitivity evidence comes from its own Table 1a and Tables 8-9 experiments. One caveat is that the claim that the wording and logic variants are semantically identical is an assumption that could be challenged because the forced-choice and 11-point-scale decoders alter the response format, but this is a validity threat to the interpretation, not a circular derivation: the sensitivity values are still computed from fresh LLM outputs against external reference dynamics, and the paper also includes the cleaner whitespace-only variant. Overall, the central claim is empirically grounded rather than equivalent to its inputs.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The framework is built on established reference models and a hand-designed prompt set. The only hand-chosen element that could affect results is the opinion encoding mapping. No new physical or conceptual entities are introduced.

free parameters (1)
  • Manual opinion encoding mapping = [-5,5] mapped to 11-point text scale
    Chosen by hand rather than derived; the encode-decode consistency experiment (Fig. 4c) shows quality depends on this mapping and deteriorates near zero.
assumptions (3)
  • domain assumption DeGroot and Hegselmann-Krause models are valid reference dynamics for opinion formation.
    The paper treats these classical models as ground truth for evaluating LLM-ABMs (Section 2).
  • domain assumption Scalar opinion values and natural-language encoding/decoding faithfully represent the quantities of interest.
    The entire framework depends on the encode-decode chain (eq. 5); the paper tests but cannot guarantee its fidelity.
  • ad hoc to paper The prompt variants are semantically equivalent to the base prompt.
    The sensitivity analysis assumes θwording, θlogic, and θ\n are semantically neutral changes; this is asserted rather than measured (Section 5, Table 3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models." pith.science (2026). https://pith.science/paper/OUQUE6IL

@misc{pith2026241205093,
  author       = {Pith},
  title        = {Pith review of: Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OUQUE6IL}},
  note         = {Machine review of arXiv:2412.05093}
}
read the original abstract

Large language models have increasingly been proposed as a powerful replacement for classical agent-based models (ABMs) to simulate social dynamics. By using LLMs as a proxy for human behavior, the hope of this new approach is to be able to simulate significantly more complex dynamics than with classical ABMs and gain new insights in fields such as social science, political science, and economics. However, due to the black box nature of LLMs, it is unclear whether LLM agents actually execute the intended semantics that are encoded in their natural language instructions and, if the resulting dynamics of interactions are meaningful. To study this question, we propose a new evaluation framework that grounds LLM simulations within the dynamics of established reference models of social science. By treating LLMs as a black-box function, we evaluate their input-output behavior relative to this reference model, which allows us to evaluate detailed aspects of their behavior. Our results show that, while it is possible to engineer prompts that approximate the intended dynamics, the quality of these simulations is highly sensitive to the particular choice of prompts. Importantly, simulations are even sensitive to arbitrary variations such as minor wording changes and whitespace. This puts into question the usefulness of current versions of LLMs for meaningful simulations, as without a reference model, it is impossible to determine a priori what impact seemingly meaningless changes in prompt will have on the simulation.

Figures

Figures reproduced from arXiv: 2412.05093 by the authors.

Figure 1
Figure 1. Opinion Dynamics Models. Scalar opinion 𝑥𝑖 (𝑡) per agent 𝑖 and time 𝑡 for the (a) DeGroot and (fig. 1b) Hegselmann-Krause models, simulated using eq. (4) with 𝜖 = 2 and 𝜖 = 0.3 respectively. While the DeGroot model leads to consensus and the main quantity of interest is the speed of convergence, the Hegselmann-Krause model can show a wider range of dynamics, including polarization as in fig. 1b. important roles in t… view at source ↗
Figure 2
Figure 2. Evaluation Framework. (a) Simulation error in black-box system identification (b) Schematic of the evaluation framework, showing encoding, decoding and dynamics operations. See fig. 1 for an illustration of the main differences in opinion dynamics between a DeGroot and a Hegselmann-Krause model, i.e., between consensus and polarization. Next, we will introduce our proposed framework to measure the consistency of an … view at source ↗
Figure 3
Figure 3. (a) Ego network and opinion aggregation. The ego network of node Blue is indicated by solid edges. Nodes in Blue’s ego network that are within an 𝜖 distance of Blue’s opinion are indicated in green. Edges which transmit opinions are indicated by a text icon. (b) System architecture. Number of vLLM instances depends on task load. Load balancer dispatches to the least utilized instance. Algorithm 1 Simulation protocol… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Consistency Evaluation content length, particularly when aggregating interactions from other agents. For LLMs with larger context length, these limitations disappear in our implementation. 5 Experiments In the following, we evaluate the ability of LLMs to reproduce the…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 39 canonical work pages

  1. [1]

    The political blogosphere and the 2004 U

    Lada A Adamic and Natalie Glance. The political blogosphere and the 2004 U . S . election. In Proceedings of the 3rd international workshop on Link discovery, New York, NY, USA, August 2005. ACM

    work page 2004

  2. [2]

    Challenges, tasks, and opportunities in modeling agent-based complex systems

    Li An, Volker Grimm, Abigail Sullivan, B L Turner, II, Nicolas Malleson, Alison Heppenstall, Christian Vincenot, Derek Robinson, Xinyue Ye, Jianguo Liu, Emilie Lindkvist, and Wenwu Tang. Challenges, tasks, and opportunities in modeling agent-based complex systems. Ecol. Modell., 457 0 (109685): 0 109685, October 2021

    work page 2021

  3. [3]

    A new tool for epidemiology: the usefulness of dynamic-agent models in understanding place effects on health

    Amy H Auchincloss and Ana V Diez Roux. A new tool for epidemiology: the usefulness of dynamic-agent models in understanding place effects on health. Am. J. Epidemiol., 168 0 (1): 0 1--8, 2008

    work page 2008

  4. [4]

    Group formation in large social networks

    Lars Backstrom, Dan Huttenlocher, Jon Kleinberg, and Xiangyang Lan. Group formation in large social networks. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, New York, NY, USA, August 2006. ACM

    work page 2006

  5. [5]

    Emergence of scaling in random networks

    Albert-László Barabási and Réka Albert. Emergence of scaling in random networks. Science, 286 0 (5439): 0 509--512, 1999

    work page 1999

  6. [6]

    Asset pricing under endogenous expectations in an artificial stock market

    W Brian Arthur, John H Holland, Blake LeBaron, Richard Palmer, and Paul Tayler. Asset pricing under endogenous expectations in an artificial stock market. pp.\ 15--44, May 2018

    work page 2018

  7. [7]

    Agent-based models of financial markets: A comparison with experimental markets

    Nicholas T Chan, Blake LeBaron, Andrew W Lo, and Tomaso Poggio. Agent-based models of financial markets: A comparison with experimental markets. Unpublished Working Paper, MIT Artificial Markets Project, MIT, MA, 1999

    work page 1999

  8. [8]

    Simulating opinion dynamics with networks of LLM -based agents

    Yun-Shiuan Chuang, Agam Goyal, Nikunj Harlalka, Siddharth Suresh, Robert Hawkins, Sijia Yang, Dhavan Shah, Junjie Hu, and Timothy T Rogers. Simulating opinion dynamics with networks of LLM -based agents. arXiv preprint arXiv:2311. 09618, 2023

    work page 2023

Show all 44 references
  1. [9]

    Why I no longer work with agents

    Helen Couclelis. Why I no longer work with agents. In Special Workshop on Agent-Based Models of Land-Use, Land-Cover Change, CSISS, Irvine, available on-line at www. csiss. org/events/other/agent-based/papers/couclelis. pdf, 2001

    2001

  2. [10]

    Reaching a consensus

    Morris H DeGroot. Reaching a consensus. J. Am. Stat. Assoc., 69 0 (345): 0 118–121, March 1974

    1974

  3. [11]

    Growing artificial societies

    Joshua M Epstein and Robert L Axtell. Growing artificial societies. Complex adaptive systems. MIT Press, London, England, October 1997

    1997

  4. [12]

    On the evolution of random graphs

    Paul Erdős and Alfréd Rényi. On the evolution of random graphs. Publ. math. inst. hung. acad. sci, 5 0 (1): 0 17--60, 1960

    1960

  5. [13]

    Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \"a schel

    Chrisantha Fernando, Dylan S. Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \"a schel. Promptbreeder: Self-referential self-improvement via prompt evolution. ArXiv, abs/2309.16797, 2023. URL https://api.semanticscholar.org/CorpusID:263310323

    2023 arXiv

  6. [14]

    A centrality measure for quantifying spread on weighted, directed networks

    Christian G Fink, Kelly Fullin, Guillermo Gutierrez, Nathan Omodt, Sydney Zinnecker, Gina Sprint, and Sean McCulloch. A centrality measure for quantifying spread on weighted, directed networks. Physica A, 2023

    2023

  7. [15]

    S 3 : Social-network simulation system with large language model-empowered agents

    Chen Gao, Xiaochong Lan, Zhihong Lu, Jinzhu Mao, Jinghua Piao, Huandong Wang, Depeng Jin, and Yong Li. S 3 : Social-network simulation system with large language model-empowered agents. arXiv preprint arXiv:2307. 14984, 2023

    2023

  8. [16]

    How to build and use agent-based models in social science

    Nigel Gilbert and Pietro Terna. How to build and use agent-based models in social science. Mind Soc., 1: 0 57--72, 2000

    2000

  9. [17]

    Robustness of named-entity replacements for in-context learning

    Saeed Goodarzi, Nikhil Kagita, Dennis Minn, Shufan Wang, Roberto Dessi, Shubham Toshniwal, Adina Williams, Jack Lanchantin, and Koustuv Sinha. Robustness of named-entity replacements for in-context learning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the ...

    2023 doi

  10. [18]

    Opinion dynamics and bounded confidence models, analysis, and simulation

    Rainer Hegselmann and Ulrich Krause. Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of artificial societies and social simulation, 5 0 (3), 2002

    2002

  11. [19]

    Stochastic blockmodels: First steps

    Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Soc. Networks, 5 0 (2): 0 109--137, 1983

    1983

  12. [20]

    Lyfe agents: Generative agents for low-cost real-time social interactions

    Zhao Kaiya, Michelangelo Naim, Jovana Kondic, Manuel Cortes, Jiaxin Ge, Shuying Luo, Guangyu Robert Yang, and Andrew Ahn. Lyfe agents: Generative agents for low-cost real-time social interactions. arXiv preprint arXiv:2310. 02172, 2023

  13. [21]

    Stochastic blockmodels and community structure in networks

    Brian Karrer and Mark E J Newman. Stochastic blockmodels and community structure in networks. Physical review E, 83 0 (1): 0 016107, 2011

    2011

  14. [22]

    Efficient memory management for large language model serving with PagedAttention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention . In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems...

    2023

  15. [23]

    Graphs over time

    Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. Graphs over time. In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, New York, NY, USA, August 2005. ACM

    2005

  16. [24]

    System identification

    Lennart Ljung. System identification. In Signal Analysis and Prediction, pp.\ 163--173. Birkhäuser Boston, Boston, MA, 1998

    1998

  17. [25]

    Keeping LLMs aligned after fine-tuning: The crucial role of prompt templates

    Kaifeng Lyu, Haoyu Zhao, Xinran Gu, Dingli Yu, Anirudh Goyal, and Sanjeev Arora. Keeping LLMs aligned after fine-tuning: The crucial role of prompt templates. arXiv preprint arXiv:2402.18540, 2024

    2024 arXiv

  18. [26]

    SimLM : Can language models infer parameters of physical systems? arXiv preprint arXiv:2312

    Sean Memery, Mirella Lapata, and Kartic Subr. SimLM : Can language models infer parameters of physical systems? arXiv preprint arXiv:2312. 14215, 2023

    2023

  19. [27]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference o...

    2022 doi

  20. [28]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, New York, NY, US...

    2023

  21. [29]

    Advprompter: Fast adaptive adversarial prompting for llms

    Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, and Yuandong Tian. Advprompter: Fast adaptive adversarial prompting for llms. 2024. URL https://api.semanticscholar.org/CorpusID:269430799

    2024

  22. [30]

    Prompt programming for large language models: Beyond the few-shot paradigm

    Laria Reynolds and Kyle McDonell. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp.\ 1--7, 2021

    2021

  23. [31]

    Individual based models in ecology: An evaluation, or how not to ruin a good thing

    Joan Roughgarden. Individual based models in ecology: An evaluation, or how not to ruin a good thing. 2012

    2012

  24. [32]

    Dynamic models of segregation

    Thomas C Schelling. Dynamic models of segregation. J. Math. Sociol., 1 0 (2): 0 143–186, July 1971

    1971

  25. [33]

    Deconstructing in-context learning: Understanding prompts via corruption

    Namrata Shivagunde, Vladislav Lialin, Sherin Muckatira, and Anna Rumshisky. Deconstructing in-context learning: Understanding prompts via corruption. In Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (eds.), Proceedings of ...

    2024

  26. [34]

    Systematic biases in LLM simulations of debates

    Amir Taubenfeld, Yaniv Dover, Roi Reichart, and Ariel Goldstein. Systematic biases in LLM simulations of debates. arXiv [cs.CL], February 2024

    2024

  27. [35]

    Agent-based computational economics: Overview and brief history

    Leigh Tesfatsion. Agent-based computational economics: Overview and brief history. Artificial Intelligence, Learning and Computation in Economics and Finance, pp.\ 41--58, 2023

    2023

  28. [36]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023

  29. [37]

    Agent-based modeling in public health: current applications and future directions

    Melissa Tracy, Magdalena Cerdá, and Katherine M Keyes. Agent-based modeling in public health: current applications and future directions. Annu. Rev. Public Health, 39: 0 77--94, 2018

    2018

  30. [38]

    Simulating social media using large language models to evaluate alternative news feed algorithms

    Petter Törnberg, Diliara Valeeva, Justus Uitermark, and Christopher Bail. Simulating social media using large language models to evaluate alternative news feed algorithms. arXiv [cs.SI], October 2023

    2023

  31. [39]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Adv. Neural Inf. Process. Syst., 30, 2017

    2017

  32. [40]

    Universal adversarial triggers for attacking and analyzing NLP

    Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing NLP . arXiv [cs.CL], August 2019

    2019

  33. [41]

    Collective dynamics of ‘small-world’networks

    Duncan J Watts and Steven H Strogatz. Collective dynamics of ‘small-world’networks. Nature, 393 0 (6684): 0 440--442, 1998

    1998

  34. [42]

    Albert Webson and Ellie Pavlick. Do prompt-based models really understand the meaning of their prompts? In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for C...

    2022 doi

  35. [43]

    Multi-agent reinforcement learning: A selective overview of theories and algorithms

    Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. In Handbook of Reinforcement Learning and Control, Studies in systems, decision and control, pp.\ 321--384. Springer International Publishing, Cham, 2021

    2021

  36. [44]

    Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801. 07243, 2018

    2018

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.