{"id":"5cde3161-e587-4804-907d-d2e17829b4db","arxiv_id":"2502.01450","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An LLM-agent simulation framework shows that network structure, initialization strategy, and persona settings strongly affect how widely four test rumors spread across synthetic and real Facebook networks.","lead":"This study uses ChatGPT-based agents connected in social network graphs to simulate how rumors spread. The framework runs on networks with over 100 agents and reports that network structure, agent personality settings, and who starts a rumor change how far it travels.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that network structure significantly shapes rumor spread rests on single unreplicated LLM runs; with stochastic agents and no variance estimates, the observed 0–83% range may be seed-level noise.","rationale":"The reader's weakest assumption identifies the lack of human validation and the single-run design. I agree with the single-run concern, which is the most load-bearing issue for the paper's quantitative claims. I do not rest the critique primarily on the absence of human-data validation, because the framework could still be a useful internal simulation tool even if its human fidelity is unproven; however, the paper's abstract explicitly claims that network structure, personas, and spreading schemes 'significantly influence' dissemination, and that claim requires statistical support that the single-run experiments do not provide. My proposed test directly targets that gap. The verdict remains CONDITIONAL because the concern is concrete and addressable but does not invalidate the framework as a demonstration; it only means the evaluative conclusions should be treated as provisional pending replication. The available code and clearly described prompt template are positive features, but they are not a substitute for variance-aware evaluation of a stochastic LLM-based system.","tokens_in":7312,"tokens_out":1860,"duration_ms":21361,"concrete_test":"Re-run the network-structure experiment (Figure 2) with identical prompts, networks, and rumor lists, but with K=20 independent runs per network, each with a different random seed and a recorded, fixed sampling temperature (e.g., the API default or 0.7). For each run, compute the maximum percentage of affected nodes and the full time series. Then test whether the between-network variance exceeds the within-network run variance using, for example, a Kruskal-Wallis test on the per-run maxima, and report the per-network mean and standard deviation. If the between-network differences are not larger than the within-network spread, the claim that network structure significantly influences rumor dissemination is not supported; the same replication test should be applied to the initialization/activation matrix in Figure 4 to verify the reported 0-83% range.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that LLM-driven agents produce differentiated, reproducible rumor dynamics that depend systematically on network structure, personas, and spreading schemes. The evidence in the Experiments section does not establish this because every reported condition is a single 500-iteration run: Algorithm 1 selects agents stochastically (fully random or degree-proportional activation), and ChatGPT-4o-mini responses are sampled without any stated temperature or seed; no replication, confidence interval, or statistical test is reported. Consequently, the 'maximum percentage of affected nodes' in Figures 2 and 4 is one sample from a highly stochastic process, and the abstract's headline range 'from no spread to affecting 83% of agents' has no error bar attached. The observed ordering across networks (e.g., Small-World > Erdős-Rényi) could be driven by run-to-run LLM variance or by the specific random activation sequence, not by network topology. This is not a claim about whether LLMs can ever approximate human rumor behavior; it is a more basic evidentiary problem: the reported quantitative comparisons are not demonstrated to be stable under resampling. The paper's own complexity discussion and reliance on a proprietary API reinforce, rather than resolve, this concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a framework in which each node of a social network is an LLM-driven agent (ChatGPT-4o-mini) with a persona, a post history, and a stated tendency to accept and forward rumors, and simulates rumor propagation over four network types: Erdős-Rényi, Scale-Free, Small-World, and a real Facebook network. Each simulation runs for 500 iterations, with agents selected randomly or by degree for posting; the agent's belief in each rumor is updated by the LLM. Three experiments investigate the effect of network structure, of initialization/activation strategy, and of agent personas on the maximum percentage of affected nodes. The paper claims that network structure, personas, and spreading schemes significantly influence rumor dissemination, with outcomes ranging from no spread to 83% of agents affected, and concludes that the framework provides a realistic simulation of rumor spread.","tokens_in":7589,"tokens_out":3143,"duration_ms":31621,"significance":"If the central claims were established, the framework would be a useful tool for generating hypotheses about rumor dynamics and for demonstrating LLM-based agent-based modeling at nontrivial network scale. Creditable strengths include that the code is publicly released, the framework is described precisely enough to reimplement, and no fitted parameters are used to produce the network-level differences. However, as presented, the quantitative conclusions are not supported by the evidence: every experimental condition is a single unreplicated stochastic run, no variance or significance information is reported, and the abstract's 83% figure is not reconciled with the 50% maximum reported for the network-structure experiment. The persona experiment largely confirms the manipulation built into the prompt rather than discovering an emergent property of personas. The 'realistic simulation' claim is also not validated against human rumor-spreading data or against classical rumor models. The contribution is therefore more a demonstration of feasibility than a validated finding about rumor dynamics.","major_comments":[{"comment":"The paper's central quantitative claim is not supported by the reported evidence because all reported conditions are single 500-iteration runs. Algorithm 1 selects agents stochastically (fully random or degree-proportional activation), and ChatGPT-4o-mini responses are sampled without any reported temperature, seed, or replication. Figures 2 and 4 each plot the maximum percentage of affected nodes from one run per condition, so the observed ordering across networks (e.g., Small-World above Erdős-Rényi) could be due to run-to-run LLM variance or the particular activation sequence rather than network topology. To support the abstract's statement that network structure, personas, and spreading schemes 'can significantly influence' rumor dissemination, the authors should provide multiple replications per condition, report means and confidence intervals, and perform significance tests for the reported orderings.","section":"Experiments / Effect of Network Structure"},{"comment":"The abstract's headline range 'from no spread to affecting 83% of agents' is not reconciled with Figure 2, where the Small-World network's maximum is stated as 50% of nodes affected. The 83% figure appears to come from the initialization/activation experiment in Figure 4, but the text does not explicitly connect the abstract's number to any specific condition or table value, and no variance or replication context is given. As written, a reader cannot determine whether the 83% outcome is a single run, a stable finding, or an outlier. The abstract and the experimental reporting should be aligned with explicit quantitative reporting, or the headline claim should be tempered.","section":"Abstract / Figure 2 and Figure 4"},{"comment":"The persona experiment is described as showing that agents' personality configurations 'significantly influence' rumor spread, but the manipulation is built directly into the prompt: the values 'agent rumors acc' and 'agent rumors spread' (1-4) are set by the experimenters and the prompt explicitly instructs the LLM whether to accept or forward rumors. The observed decline from 'high acceptance' to 'random' to 'low acceptance' is therefore a manipulation check, not an emergent or discovered property of LLM agents. Furthermore, Figure 8 apparently reports a single run per condition without error bars or statistical tests. The conclusion should be reframed as confirming that the prompt-based trait controls affect the simulation, and the limitations of this design should be acknowledged.","section":"Appendix / Effect of Agent's Personas"},{"comment":"The comparison across network types is confounded because the networks differ in several structural properties simultaneously. The Small-World network has only 200 edges and average degree 4.0, while the Erdős-Rényi and Scale-Free networks have roughly twice the edges (396 and 390) and average degrees 7.92 and 7.80; the Facebook network has 1656 edges and average degree 19.71. The paper attributes the higher spread in Small-World to 'sparse connectivity and moderate clustering,' but this is indistinguishable from a simple density effect, and the clustering-coefficient explanation is not tested independently. To support a claim about network structure per se, the authors should use matched networks that vary one property at a time, or at least perform a controlled ablation.","section":"Table 1 / Effect of Network Structure"},{"comment":"The claim that the framework offers 'a realistic simulation of rumor spread in social networks' is not established. The paper does not compare the simulated dynamics against human rumor-spreading data, existing empirical studies, or classical rumor models such as SIR-like or compartmental models. Without such a benchmark, the realism claim is unsupported. The authors should either add a validation experiment or substantially soften the realism language to 'illustrative' or 'qualitatively plausible' until such validation is provided.","section":"Abstract / Conclusion"}],"minor_comments":[{"comment":"Table 1 contains a stray '#686' entry in the header row that appears to be an artifact; it should be removed or explained.","section":"Table 1"},{"comment":"Algorithm 1 line 2 writes '{rj}L i=j' but the bound should likely be 'j=1'; please correct the subscript.","section":"Algorithm 1"},{"comment":"The first rumor is typeset as 'Nicolae Ceaus, escu is not dead!' with an apparent comma inside the name; the typo should be fixed.","section":"Rumor list"},{"comment":"Figure 3 is said to show 'an almost linear relationship' between rumor spread and time, but the axes are not described and no quantitative slope or confidence information is given; please add axis labels and units or clarify what is plotted.","section":"Figure 3"},{"comment":"The example prompts include rumors such as 'COVID-19 now named as COVID-114514' that are not in the actual rumor list; the authors should clarify whether these are illustrative only and avoid confusing the reader about the experimental rumors.","section":"Prompt Template"},{"comment":"Several references are incomplete (e.g., 'OpenAI. 2024. ChatGPT' gives only a URL, and some arXiv entries lack version or venue information); please complete them according to the journal style.","section":"References"},{"comment":"Figure 8 is described as showing a decline in rumor spread as receptivity decreases, but no numerical axis values or error bars are visible in the description; please provide a readable figure with explicit quantitative results.","section":"Figure 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short empirical demonstration of an LLM-based multi-agent rumor simulation. The framework is plausible and the code release is a useful asset, but the quantitative claims in the abstract and conclusions are not supported by the single-run experimental design. I believe the central claims are defensible in principle and can be strengthened by adding replications, reporting variance, reconciling the 83% figure with the figures, and reframing the persona experiment as a manipulation check. I recommend major revision rather than rejection, since the evidentiary gap is fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a working demonstration, not a validated finding. The framework runs, the code is out, and the differentiated outcomes across networks and strategies are plausible. But every quantitative claim in the paper comes from a single 500-iteration run with no error bars or significance tests, so the specific ordering (Small-World > Erdős-Rényi) and the 0–83% range should be read as anecdotal until replicated.\n\nWhat's genuinely new: applying LLM-driven agents with personas to rumor spreading, across Erdős-Rényi, Scale-Free, Small-World, and a real Facebook network, and testing centrality-based initialization and activation. Chuang et al. did opinion dynamics on networks with LLM agents; this extends that to rumor propagation and adds the strategy comparison. The authors also release code, which is real evidence of work. The paper is clearly written, the prompt design is simple and reproducible, and they are honest that the proprietary API prevents verification of some claims.\n\nThe soft spots are where the stress-test note lands. One run per condition is not enough when both the activation schedule and the LLM sampling are stochastic. The abstract's '83%' figure is confusing because Figure 2 tops out at 50% for Small-World; the 83% comes from the Scale-Free degree-targeted condition in Figure 4, but the abstract doesn't say that. The persona experiment is largely a manipulation check—if you tell the model to be gullible, it is—so the conclusion there is not a discovery. There is also no validation against human rumor data or classical rumor models, despite the abstract calling the simulation 'realistic.' These are not fatal to the demonstration, but they cap the paper's contribution.\n\nThe citation pattern looks fine. They cite the relevant LLM-agent and rumor-modeling work, and there's no self-citation or obvious omission that would change the verdict.\n\nWho gets value: anyone building or evaluating LLM-agent social simulations will want to know this exists; it's a useful baseline and a good sanity-check template. I would not bend my own conclusions around the specific numbers until they are replicated with seeds and intervals.\n\nRecommendation: send it to peer review. The issues are fixable: run multiple seeds, report means and variances, and calibrate the abstract to the evidence. A good reviewer will push them to do that.","headline":"A clean, runnable LLM-agent rumor simulator with released code, but the quantitative network comparisons rest on single unreplicated runs and need error bars before the claims hold.","tokens_in":8086,"tokens_out":2522,"would_cite":false,"duration_ms":23985,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that LLM-driven agents with personas and post feeds can simulate rumor spreading in social networks, with spread ranging from no agents affected to 83% of agents, depending on network structure, seeding, and personality.","keywords":["rumor spreading","LLM agents","agent-based modeling","social networks","misinformation","network structure","personas","simulation"],"falsifier":"Run one rumor through one network configuration 50 times with different random seeds and nonzero temperature; if the maximum percentage of affected agents swings from near zero to above 80% across runs, the reported network, persona, and seeding effects are not stable enough to support the paper's conclusions.","tokens_in":7152,"feed_emoji":"📣","tokens_out":9398,"duration_ms":80711,"temperature":0.7,"pith_summary":"This paper tries to establish that large language models can stand in for human users in agent-based simulations of rumor spreading, and that the simulated dynamics respond to the same factors that shape real misinformation cascades. The authors build social networks of more than one hundred ChatGPT-4o-mini agents, each with a persona, a post history, and a list of rumors, and run iterative rounds in which selected agents post and update their belief in each rumor. Across four network types and multiple seeding and activation schemes, the reported spread ranges from no agents affected to 83% of agents affected. If correct, this gives social-network researchers a way to study rumors with far more behavioral texture than equation-based models, and a testbed for interventions before they are tried in the wild.","feed_headline":"LLM agents spread a rumor to 83% of simulated users","feed_subtitle":"Network shape, agent personalities, and seeding strategy decide whether a rumor fizzles or reaches most users in 100-agent tests.","key_machinery":"The central object is the LLM-driven agent: one ChatGPT-4o-mini call per network node, with a prompt that bundles a persona (name, age, job, traits, and explicit willingness to accept and forward rumors), the posts visible to that agent, and the complete list of rumors under test. Each agent response contains a new post and an updated belief $\\langle b_{ij}\\rangle \\in [0,1]$ for each rumor $r_j$. The process runs on an iterative loop: at each time step one agent is selected randomly or by degree, it posts, the post is appended to its own history and to the histories of all neighbors, and the agent's beliefs are refreshed. This loop is what carries the argument because it couples network topology (who sees what) with individual psychology (who believes and forwards what) in a single simulation.","core_discovery":"On the paper's own terms, the central discovery is that LLM-driven agents reproduce differentiated rumor dynamics: network structure, agent personas, and spreading schemes each change whether and how far a rumor travels. The Small-World network, with sparse connectivity and moderate clustering, is the most susceptible, reaching up to 50% of nodes, while the denser and more clustered Facebook-derived network suppresses spread. Degree-based seeding and degree-based activation spread every tested rumor, whereas random strategies leave most rumors confined to a few agents. Agents also behave like flexible readers rather than fixed carriers: some who initially accept a rumor later reject it as they see new posts, and rumors that are easy to disprove from pretrained knowledge are dismissed while less familiar health and nature rumors propagate. The headline number is the range of outcomes, from no spread to 83% of agents affected, showing that the framework is sensitive to the variables it claims to model.","pith_inferences":["A direct comparison against classical threshold or compartmental rumor models on the same networks would show whether the LLM agents add predictive value beyond a parameterized curve.","Because the model's pretraining data is entangled with rumor content, running the framework on fictional, time-stamped rumors would isolate the effects of network structure and persona from the model's prior knowledge.","The same machinery is an intervention testbed the paper does not fully exploit: injecting fact-checker agents, deleting high-degree nodes, or rewiring the graph mid-simulation would show how the 83% ceiling moves under mitigation.","Re-running each configuration across many seeds and temperatures would reveal how much of the reported spread differences is stable signal versus sample noise."],"forward_implications":["The framework runs at practical scale: more than one hundred agents and thousands of edges, so LLM-agent rumor simulations are not limited to toy graphs.","Network topology alone can change a rumor's fate: the sparse Small-World network spreads rumors to more agents than the dense, highly clustered Facebook-derived network.","Seeding and activity rules are decisive: rumors that start at high-degree agents and are posted by high-degree agents spread across the whole network, while random seeding and activation leave most rumors localized.","Agent personality shifts the outcome monotonically: moving all agents from 'likely to accept' to 'unlikely to accept' produces a clear decline in the maximum percentage of nodes affected.","Rumor content matters: agents tend to dismiss rumors they can disprove from pretrained knowledge and accept unfamiliar health and nature rumors, and some agents reverse their belief over time."],"supporting_citations":[{"why":"Defines the Erdős–Rényi random graph used as one of the four network topologies.","marker":"Erdős and Rényi 1959"},{"why":"Supplies the scale-free network construction used in the experiments.","marker":"Barabási and Bonabeau 2003"},{"why":"Supplies the small-world network generation model, the topology reported as most susceptible to rumor spread.","marker":"Watts and Strogatz 1998"},{"why":"Provides the real-world Facebook network data used to test rumor spread on a dense, clustered topology.","marker":"Leskovec and Mcauley 2012"},{"why":"Provides the ChatGPT-4o-mini model that drives every agent's posting and belief decisions.","marker":"OpenAI 2024"},{"why":"Prior demonstration that networks of LLM agents can simulate opinion dynamics, motivating the agent-based design.","marker":"Chuang et al. 2024"},{"why":"Supplies the traditional agent-based model of competing rumor spread that the LLM framework extends.","marker":"Kaligotla, Yücesan, and Chick 2015"},{"why":"Supports the interpretation that LLM agents reject easily disprovable rumors using knowledge from pretrained models.","marker":"Liu et al. 2024"}],"fun_headline_variants":["LLM agents show network shape decides rumor spread","Degree-based seeding spreads all LLM rumors","LLM agents reveal small-world networks fuel rumor spread","LLM simulations: rumor spread ranges from zero to 83%","LLM agents sometimes reject accepted rumors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the premise that ChatGPT-4o-mini agents, given a persona and a post feed, accept and forward rumors the way real people would, a premise supported here by a single 500-iteration run with no replication and no comparison to human rumor-spreading data.","fun_headline_variants_meta":{"raw":{"variants":["LLM agents show network shape decides rumor spread","Degree-based seeding spreads all LLM rumors","LLM agents reveal small-world networks fuel rumor spread","LLM simulations: rumor spread ranges from zero to 83%","LLM agents sometimes reject accepted rumors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001951,"raw_usage":{"total_tokens":7590,"prompt_tokens":866,"completion_tokens":6724,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":6651}},"tokens_in":482,"tokens_out":6724,"duration_ms":43313,"temperature":1.0,"reasoning_tokens":6651,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:14:32.615975+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run one rumor through one network configuration 50 times with different random seeds and nonzero temperature; if the maximum percentage of affected agents swings from near zero to above 80% across runs, the reported network, persona, and seeding effects are not stable enough to support the paper's conclusions.","supporting_citations":[{"cited_title":"J.; and Strogatz, S","cited_arxiv_id":null,"evidence_quote":"Supplies the small-world network generation model, the topology reported as most susceptible to rumor spread."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the real-world Facebook network data used to test rumor spread on a dense, clustered topology."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ChatGPT-4o-mini model that drives every agent's posting and belief decisions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the traditional agent-based model of competing rumor spread that the LLM framework extends."}],"review_version":1}