{"id":"447a123a-f036-44c8-9bd4-8f72b8a2397b","arxiv_id":"2412.10781","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Crowd provides a configuration-driven, GUI-supported Python framework for social network agent-based simulations, demonstrated on epidemic, influence maximization, and trust game case studies.","lead":"Crowd is a new open-source Python framework that lets researchers run agent-based simulations on social networks by describing the setup in YAML config files instead of writing simulation code. It is meant to simplify modeling of phenomena like disease spread, influence, and trust in networked populations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative support for Crowd's 'fasten' claim rests on unpublished Mesa baselines and timing tables without variance; until the benchmark code is archived and error bars are reported, the speed comparison is unverifiable.","rationale":"The reader's weakest_assumption is on target: the most load-bearing issue is the fairness and reproducibility of the Mesa baseline used to support the speed part of Crowd's central claim. A software-tool paper's central claim is usability and performance; usability evidence here is largely qualitative, which is acceptable for a tool presentation, but the performance evidence is presented as precise numbers and is therefore testable. Without the baseline code and without variance information, those numbers cannot be independently verified, and the reader cannot distinguish a genuine design advantage from an under-optimized comparison point. I do not see an internal inconsistency in the framework design itself; notably, the first case study honestly acknowledges that its modified network setting makes direct comparison with the original GABM study not meaningful, which is a sign of good faith. The concern does not justify rejection because Crowd is open-source and the missing pieces—archived benchmark code, per-run values, and confidence intervals—are straightforward to supply. The existing CONDITIONAL verdict remains the right call, so my read does not change it.","tokens_in":18591,"tokens_out":4493,"duration_ms":44871,"concrete_test":"Make the benchmark package public: archive the exact Mesa and Crowd implementations for all three case studies at a fixed commit, then rerun Tables II-IV with at least 10 independent executions per condition on identical hardware, reporting full distributions and confidence intervals. If the Mesa code is not available, or if Crowd's advantage is within noise after re-running, downgrade 'fastens' and 'without any performance drawback' to 'comparable' and mark the timing comparison as preliminary.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing part of the paper is the quantitative claim that Crowd 'fastens' development relative to general-purpose ABMS tools, supported by Tables II-IV in Section VI-B-2. That comparison depends on two unverified conditions. First, the Mesa implementations are not archived with a fixed commit; no benchmark code is in the repository, so a reader cannot check whether the Mesa side was written faithfully and with comparable optimization, or whether the workloads were shaped around Crowd's abstractions. Second, each table reports only an average of 5 runs, with no variance, per-run values, or statistical test. For the LLM-driven scenario, run-to-run noise from GPU inference alone can easily exceed the roughly 10% gap reported; the paper itself quotes 10-10.5 seconds per query in Section V-A but about 6 seconds in Section VI-B-2, suggesting the timing environment is not tightly controlled. In Table III, Crowd is actually slower than Mesa on Facebook (5.19 s vs 4.62 s), so the advantage is not even consistent across conditions. The framework may well function as described, but the 'fasten' advantage is currently an unverified comparative claim rather than an established result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Crowd, a Python-based agent-based modeling and simulation framework specialized for social networks. The framework provides YAML-based configuration for simulation setup, no-code diffusion modeling via NDLib-style compartments, interactive network visualization, automatic data collection, and a Tauri-based desktop GUI. The authors demonstrate Crowd through three case studies: an LLM-driven epidemic simulation with generative agents, an influence-maximization scenario on real social networks, and a networked N-player trust game. They also compare Crowd with the general-purpose framework Mesa in terms of modeling effort, execution time, and visualization facilities, reporting execution-time tables for the three case studies. The central claim is that Crowd simplifies and accelerates the development of agent-based network simulations.","tokens_in":18742,"tokens_out":2415,"duration_ms":22646,"significance":"If the claims are substantiated, Crowd is a useful open-source contribution: it targets a genuine gap—general-purpose ABMS tools require repetitive boilerplate for network simulations—and it demonstrates concrete integrations with external libraries (NetworkX, igraph, Hugging Face Transformers) and with real datasets (Facebook, GitHub, Twitch). The case studies are nontrivial and show that the framework can express models from the literature with relatively little code. The paper is honest about its scope (single-threaded, small-to-medium scale) and identifies limitations and future work. The main unproven part is the quantitative 'fasten' claim: the execution-time comparison against Mesa lacks reproducibility artifacts, variance reporting, and statistical testing, and it contains at least one internal contradiction. The framework's qualitative advantages (configuration-driven setup, automatic data saving, interactive GUI) are plausible even without the timing results.","major_comments":[{"comment":"The execution-time comparison is the quantitative backbone of the 'fasten' claim, but it is reported only as averages of 5 runs with no variance, no per-run values, and no statistical test. Differences on the order of 10–20%, as in Table II (107.07 min vs 95.55 min) and Table IV (38.03 s vs 24.24 s), could plausibly fall within run-to-run noise, especially for the GPU-based LLM scenario. Moreover, Table III shows Crowd is slower than Mesa on Facebook (5.19 s vs 4.62 s), which contradicts the summary statement that 'Crowd takes less time in most simulations' only if that one case is treated as noise. Please report standard deviations or full distributions, state whether the differences are statistically significant, and reconcile the Facebook row with the claimed advantage.","section":"§VI-B-2, Tables II-IV"},{"comment":"There is an unexplained inconsistency in the reported LLM inference time: Section V-A states each query takes 'approximately 10–10.5 seconds' and a 50-day, 100-agent experiment with 5000 queries takes 857 minutes, while Section VI-B-2 states 'each query taking around 6 seconds on average' for the same scenario in Table II. These numbers cannot both describe the same setup unless the hardware or model settings changed, and the paper does not say so. This discrepancy undermines the reliability of the Table II timing comparison and needs to be resolved explicitly (e.g., different GPU, different quantization, different prompt length).","section":"§V-A vs §VI-B-2"},{"comment":"The Mesa baseline implementations are not archived or documented in enough detail for a reader to verify that they are faithful, fairly optimized, and equivalent in functionality to the Crowd implementations. Without the benchmark code (with a fixed commit or version) and a clear description of how the Mesa models were written, the comparison is unverifiable, and the workload may have been shaped around Crowd's abstractions. Please make the benchmark scripts publicly available and include a precise specification of the Mesa models and their configuration, so that the claim 'fasten ... development' can be independently checked.","section":"§VI-B (all), reproducibility"}],"minor_comments":[{"comment":"The caption for Figure 11 says 'n = 1000' while the text in Section V-C states the Scale-Free network has 1024 nodes; please correct the caption or the text to be consistent.","section":"§V-C, Figure 11 caption"},{"comment":"The text refers to 'William et al.' but the reference [34] is Williams et al.; the name is misspelled in the body text.","section":"§V-A"},{"comment":"YAML is rendered as 'Y AML' in multiple places (e.g., Sections I, III, and Table I). This appears to be a formatting artifact, but it should be fixed as a single token 'YAML'.","section":"Throughout"},{"comment":"The tool name 'MASON' is written as 'Mason' in the first column; use the official capitalization consistently.","section":"Table I"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is already accepted for publication in IEEE TCSS per the header, but as a referee I am reviewing the scientific content as presented. The central claim of 'fastening' development is not yet established because the performance comparison lacks reproducibility artifacts and variance reporting, and because the LLM timing numbers are internally inconsistent. The framework itself appears functional and the case studies are useful, so the path to revision is clear: provide the benchmark code, report full timing distributions, and reconcile the LLM timings. No concerns about novelty overlap or authorship conduct surfaced in my review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: Crowd is a real, functioning framework, not vaporware. The paper delivers a config-driven (YAML) network ABMS with a GUI, built on NetworkX/NDLib, and shows three case studies that run against external models. The trust-game case reproduces Chica et al.'s results, which is a solid external anchor. The writing is clear, and the authors are upfront that Crowd is a convenience layer over existing pieces rather than a new simulation paradigm.\n\nWhat is actually new: the specific abstraction set (DiffusionNetwork, CustomSimNetwork, EdgeSimNetwork), the YAML schema, the Tauri-based GUI, and the integration of LLM agents into that structure. None of these is individually novel, but the packaging as one tool fills a real usability gap for computational social scientists who want network simulations without writing Model/Agent boilerplate. The Mesa comparison is a good instinct, but it is the soft spot. The timing tables (II–IV) report only averages of five runs, with no variance, no per-run values, and no statistical test. The Mesa baseline code is not archived, so a reader cannot check whether it was written with comparable optimization. The paper even quotes ~10–10.5 s per LLM query in Section V-A but ~6 s in Section VI-B-2, which tells me the environment was not tightly controlled. And Table III shows Crowd slower than Mesa on Facebook (5.19 s vs 4.62 s), so the 'fasten' claim is not even consistent. That said, the performance gap is secondary: the paper's real pitch is lower modeling effort, which the case-study narrative supports qualitatively. The 'fastens' language should be toned down or backed by archived benchmarks with error bars.\n\nThe circularity burden is low—no derivation reduces to its own input, and the case studies are external implementations. Citation pattern is fair and includes the obvious comparators (Soil, Mesa, NDLib). The framework repository exists, but the benchmark code and a fixed commit hash are missing; that is an addressable deficiency.\n\nProportionate verdict: this is a solid incremental tool paper. The core artifact works, the case studies demonstrate real use, and the flaws are fixable. A serious referee should see it; the benchmarking section needs revision, but the paper does not deserve rejection. I'd send it out, with the request that the authors archive the Mesa code and report variance.\n\nFor your reading group: maybe, if anyone does network ABMS. I would not cite it this year unless I needed the tool itself.","headline":"Crowd is a genuinely usable framework with honest scoping; the speed-up claim is the weakest link and needs better evidence, but the tool itself deserves review.","tokens_in":19338,"tokens_out":1459,"would_cite":false,"duration_ms":16380,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Crowd is a Python framework that lets researchers build agent-based social network simulations from a YAML configuration file, and the paper claims this makes model development faster without slowing execution.","keywords":["agent-based modeling","social network simulation","YAML configuration","diffusion models","generative agents","influence maximization","trust games","simulation framework"],"falsifier":"Run the same three studies in Crowd and in a general-purpose Python agent-based framework, using independently written, comparably optimized code, with many repetitions and reported variance; if the general-purpose framework matches or beats Crowd on most workloads and the total setup time is not clearly lower, the paper's central promise fails.","tokens_in":18342,"feed_emoji":"🕸️","tokens_out":6501,"duration_ms":53568,"temperature":0.7,"pith_summary":"The paper presents Crowd, a Python framework for agent-based simulations on social networks, and claims it lowers the effort needed to build, run, and analyze such simulations. Instead of writing model and agent classes from scratch, a user describes the network, node types, parameters, and diffusion rules in a YAML configuration file, optionally adding custom Python methods for study-specific logic. Crowd also supplies interactive network visualization, automatic data collection, chart drawing, and merging of results across runs. The authors demonstrate the framework on three diverse studies—a generative-agent epidemic model, influence maximization on a large Facebook network, and an evolutionary trust game—and report execution times comparable to a general-purpose baseline while requiring less setup code. The point of the paper is that a network-focused, config-driven design can make social network simulation accessible without sacrificing performance.","feed_headline":"Crowd runs social-network simulations from a YAML file","feed_subtitle":"Fewer lines of code for epidemics, influence spread, and trust games on networks.","key_machinery":"The load-bearing mechanism is Crowd's class hierarchy of networks, anchored by a base Network class and a CustomSimNetwork subclass that stores node and edge parameters and executes user-supplied methods at designated points in each iteration. A DiffusionNetwork subclass integrates a compartment model so that state transitions such as Susceptible to Infected to Recovered can be declared as YAML rules instead of programmed. Configuration files, the GUI, and the Python library all drive the same simulation engine, with a standard graph data structure as the underlying representation and the frame's own data savers writing JSON snapshots at chosen intervals.","core_discovery":"The central claim is that Crowd makes agent-based modeling on networks simpler and faster to develop by moving simulation setup out of code and into declarative configuration. A researcher selects the network structure, node types, node and edge parameters, and—for diffusion tasks—compartment-based rules in YAML; the framework then handles iteration scheduling, snapshotting, data saving, and visualization. For cases the built-in diffusion engine cannot express, Crowd's custom simulation network lets the user attach Python methods that run before, during, or after each iteration, with return values automatically persisted. The paper's three case studies are offered as evidence that this design covers enough ground to reproduce published results from epidemic modeling, influence maximization, and trust games, and its timing tables are offered as evidence that the added conveniences do not cost runtime. In short, the authors claim to have separated the reusable mechanics of network simulation from the study-specific logic.","pith_inferences":["If the configuration approach catches on, one testable prediction is that published network-diffusion results will become easier to replicate, because the simulation logic is declared in a machine-readable file rather than embedded in bespoke code.","The generative-agent epidemic case hints at a broader pattern: Crowd could serve as a testbed for comparing LLM-driven and rule-based agents under identical network topologies, since only the decision method changes.","The paper's timing claims ignore variance; a natural extension would be a benchmark suite with confidence intervals across more datasets to see whether Crowd's edge persists.","Whether the no-code diffusion layer can express arbitrary compartment models, or only a fixed subset, limits how far the 'no code' promise generalizes; a community-contributed library of YAML examples would test that boundary."],"forward_implications":["A social scientist who can write a YAML file can run a diffusion simulation and inspect interactive charts without writing any simulation-loop code.","Research teams that currently reuse boilerplate model and agent classes will cut setup effort, since configuration files and automatic data saving replace repetitive scaffolding.","Because user methods run inside Python, existing data-analysis and machine-learning libraries can be called directly from simulations, including LLM-based generative agents.","The framework's built-in centrality-based seed selection and result merging make influence-maximization experiments faster to iterate.","The three reproduced case studies suggest the abstractions are general enough to cover epidemics, information diffusion, and evolutionary games, so new studies in those areas can start from a configuration rather than from scratch."],"supporting_citations":[{"why":"Serves as the baseline general-purpose Python agent-based framework against which Crowd's simplicity and speed claims are measured.","marker":"[16]"},{"why":"Provides the compartment structure that Crowd's no-code diffusion configuration wraps.","marker":"[17]"},{"why":"The closest prior network-focused Python simulator, used to position Crowd's extra GUI and configuration features.","marker":"[28]"},{"why":"Supplies the graph data structures and network algorithms on which Crowd's simulations are built.","marker":"[31]"},{"why":"The generative-agent epidemic study that Crowd's first case study re-implements on a network.","marker":"[34]"},{"why":"The real-world Facebook network dataset used in the influence maximization case study.","marker":"[35]"},{"why":"The networked trust game whose evolutionary dynamics Crowd's third case study reproduces.","marker":"[36]"},{"why":"The Independent Cascade diffusion model underlying the influence maximization case study.","marker":"[44]"}],"fun_headline_variants":["Crowd: simulate social networks via YAML setup","Declare YAML, Crowd runs your network simulation","Crowd framework: YAML-driven social network simulations","Skip the simulation code: Crowd reads YAML and runs","Crowd: network simulations defined in YAML"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed time savings rest on the comparison with a general-purpose baseline being fair, meaning the baseline implementations must be reasonably optimized and the chosen case studies representative, and on the reported averages not being noise, since no variance is given.","fun_headline_variants_meta":{"raw":{"variants":["Crowd: simulate social networks via YAML setup","Declare YAML, Crowd runs your network simulation","Crowd framework: YAML-driven social network simulations","Skip the simulation code: Crowd reads YAML and runs","Crowd: network simulations defined in YAML"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001071,"raw_usage":{"total_tokens":4461,"prompt_tokens":893,"completion_tokens":3568,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":3487}},"tokens_in":509,"tokens_out":3568,"duration_ms":25292,"temperature":1.0,"reasoning_tokens":3487,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:36:16.003349+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same three studies in Crowd and in a general-purpose Python agent-based framework, using independently written, comparably optimized code, with many repetitions and reported variance; if the general-purpose framework matches or beats Crowd on most workloads and the total setup time is not clearly lower, the paper's central promise fails.","supporting_citations":[{"cited_title":"Masad and J","cited_arxiv_id":null,"evidence_quote":"Serves as the baseline general-purpose Python agent-based framework against which Crowd's simplicity and speed claims are measured."},{"cited_title":"Rossetti, L","cited_arxiv_id":null,"evidence_quote":"Provides the compartment structure that Crowd's no-code diffusion configuration wraps."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The closest prior network-focused Python simulator, used to position Crowd's extra GUI and configuration features."},{"cited_title":"Exploring network structure, dynamics, and function using NetworkX,","cited_arxiv_id":null,"evidence_quote":"Supplies the graph data structures and network algorithms on which Crowd's simulations are built."},{"cited_title":"Learning to discover social circles in ego networks,","cited_arxiv_id":null,"evidence_quote":"The real-world Facebook network dataset used in the influence maximization case study."},{"cited_title":"Chica, R","cited_arxiv_id":null,"evidence_quote":"The networked trust game whose evolutionary dynamics Crowd's third case study reproduces."},{"cited_title":"Kempe, J","cited_arxiv_id":null,"evidence_quote":"The Independent Cascade diffusion model underlying the influence maximization case study."}],"review_version":1}