{"id":"ac325801-51df-4ab9-ba6d-6a138c20297e","arxiv_id":"2506.05698","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A bibliometric analysis of 2,077 Scopus papers shows discrete-event simulation research has grown steadily from 2010 to 2024 and expanded into many new application areas.","lead":"This paper counts and maps 2,077 academic papers on discrete-event simulation published between 2010 and 2024, using data from the Scopus database. It finds that research on this modeling method has continued to grow and spread into fields like healthcare, manufacturing, and logistics, despite past predictions that it would be replaced by newer techniques.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The undocumented filter that reduces 186,797 Scopus hits to 2,077 is the load-bearing step; without reproducible screening rules, the temporal and domain evidence for DES expansion could be a selection artifact.","rationale":"I read the paper in good faith: the qualitative claim that DES remains a relevant OR technique is plausible and supported by prior literature and by broad publication counts visible in Figure 2. The conditional verdict is appropriate because the study's own quantitative evidence is what would substantiate the 'expansion' part of the argument, and that evidence depends entirely on the reproducibility and validity of the sampling procedure. The reader's weakest assumption correctly identifies sample representativeness as the central risk. My reading reinforces that, rather than introducing a new objection: the filtering from 186,797 to 2,077 documents is not documented, the table's arithmetic is internally inconsistent, and the screening decisions were evidently made by the authors without reported reliability checks. Because the data are proprietary and unavailable, independent verification is especially important. This does not warrant rejection—the central narrative is not implausible, and the raw data, if properly screened, might well support it. But it does mean the paper should remain conditional until the screening protocol is made transparent and the reconstruction test is run. I therefore recommend no change to the reader's verdict.","tokens_in":17591,"tokens_out":4686,"duration_ms":52175,"concrete_test":"Use Scopus to rerun the exact query, export all records after the document-type filter, and have two independent coders apply a pre-specified inclusion rule—e.g., title/abstract must explicitly describe discrete-event simulation and an operations-research/application context—to a random sample of 300 excluded records and 200 included records. Compute Cohen's kappa and the proportion of excluded records that meet the rule. If kappa < 0.7 or if more than 5% of excluded records are actually in-scope DES-in-OR papers, the 2,077 sample and its derived trends are not trustworthy; if the rule reproduces the sample with high agreement, the concern is resolved. Also recompute Table 1 with the corrected arithmetic.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DES is 'alive, well, and kicking' rests on the 2,077-document sample. The search query ('discret*' AND ('simulat*' OR 'model*')) is deliberately broad, and the step that removes 184,666 documents as 'Non Discrete-Event Simulation in OR' is the pivotal filter. The paper never states the screening rule (title/abstract keywords? manual relevance judgment? exclusion of non-OR fields?), nor reports who applied it or whether decisions were reliable. Table 1 compounds the problem: it says 187,797 initial hits minus 930 document-type exclusions gives 186,797, then subtracts 184,666 and labels the result 2,131, and then says 'Screened out 9 Irrelevant Documents' while computing 2,131 - 54 = 2,077. These inconsistencies make the sample impossible to reproduce. If the filter was applied unevenly across years or fields, the annual publication trend (slope 2.232, R² = 0.40) and the domain percentages in Figure 3 could be artifacts. This is the weak link because the qualitative plausibility of DES's relevance is not enough to validate the quantitative expansion evidence; the sample construction is.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript reports a bibliometric study of discrete-event simulation (DES) research and practice from 2010 to 2024, based on 2,077 Scopus-indexed documents. It claims that DES has undergone continuous expansion despite earlier predictions of its demise, identifies application domains and thematic structures, and analyzes citation and collaboration patterns. The paper's central conclusion is that 'DES is alive, well, and kicking,' with evidence of broadened adoption, sustained relevance, and a role in Industry 4.0 and digital transformation.","tokens_in":17820,"tokens_out":2514,"duration_ms":24487,"significance":"If the empirical basis were solid, the study would provide a useful quantitative complement to the qualitative debate about DES's vitality and would update the community's understanding of DES's application landscape. The manuscript applies standard bibliometric tools (VOSviewer, Bibliometrix) and offers detailed descriptive statistics, which are valuable for replication. However, the central claim rests on a sample whose construction is not reproducible from the information provided, and the growth analysis is statistically weak. The paper's contribution is therefore conditional on fixing the sample-construction and trend-analysis issues.","major_comments":[{"comment":"The sample-construction steps are internally inconsistent and not reproducible. The search is stated to generate 187,727 documents, but Table 1 then lists 187,797 in the subtraction; the subtraction '187,797 - 184,666 = 2131' contains a typo (the initial value should be 186,797, if the prior filtering is retained) and yields 2,131, while the text simultaneously reports 'Screened out 9 Irrelevant Documents' and computes '2131 - 54 = 2077.' More fundamentally, the decisive filter that removes 184,666 documents as 'Non Discrete-Event Simulation in OR' is never defined: the manuscript does not state whether irrelevance was judged by title/abstract keywords, manual reading, subject-area exclusion, or some algorithmic rule, nor does it report who applied the judgment or how reliable it was. Because the annual publication trend and the domain percentages in Figure 3 are computed from the final 2,077 documents, the undocumented filter is load-bearing for the central claim.","section":"§3.1, Table 1"},{"comment":"The linear regression used to support the growth claim is not convincing as presented. The reported equation y = 2.232x + 120.61 has R² = 0.40, meaning time explains only 40% of the annual variation in publication counts; with only 15 data points, the high 2020 value (driven by COVID-19-related publications, as the text itself notes) could dominate the fitted slope. The text states 'The model is appropriate for this dataset because the trend of the SLP over time exhibits growth,' which is circular rather than a statistical justification. Additionally, the reported annual growth rate of 2.03% (Table 2) is not reconciled with the regression slope of 2.232 publications per year on a base of roughly 120-140, which would be closer to 1.6-1.9% per year; the manuscript offers no measure of uncertainty. These issues undermine the quantitative support for RO1.","section":"§4.2, Figure 2b"},{"comment":"The author and country counts are inconsistent across the manuscript, which casts doubt on the descriptive statistics. Section 4.1 reports 5,627 researchers, while Table 2 reports Authors 5,629; Section 4.6.1 then reports '1,968 unique authors from 89 countries' with '5,429 total frequencies.' These numbers cannot all be correct descriptions of the same sample. Similarly, the definitions of SCP and MCP in Table 6 are unconventional (SCP is usually single-country publications and MCP multi-country publications, whereas the table defines SCP as non-corresponding-author collaboration and MCP as corresponding-author contribution), and the country-assignment rule (all authors vs. corresponding author) is not specified. Since RO4 concerns social structures, these discrepancies matter for the paper's fourth objective.","section":"§4.6.1 and §4.1"}],"minor_comments":[{"comment":"The text has two tables numbered Table 5: 'Ten most cited documents' and 'The top 20 sources with most eminent sources and citation impacts.' Please renumber the second table as Table 6 and adjust subsequent table numbers.","section":"§4.5.2"},{"comment":"The text states 'Figure 4 shows the complete themes under this category' when describing the least-frequent themes, but the relevant visualization is Figure 5. Please correct the cross-reference.","section":"§4.4.3"},{"comment":"The manuscript contains several typos that should be corrected: 'coronavirus disease 2029' in the Introduction, 'InformaƟon' in the author affiliation line, 'the gift of live' in the Acknowledgments, and 'generative adversarial networks' in Section 4.4.3. These are presentation issues but should be fixed.","section":"Throughout"},{"comment":"Reference 65 (Zhu et al., AGU Fall Meeting, on sulfate simulation) appears irrelevant to the statement in Section 4.4.3 about machine learning and generative adversarial networks; please verify the intended citation.","section":"References"},{"comment":"The sentence 'The first step in the simulation and modeling process involves problem definition or formulation is the first step in the simulation and modeling process' is grammatically garbled and should be rewritten.","section":"§2.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a high proportion of self-citations by the first author (e.g., refs 6, 7, 12, 25, 31, 37, 41, 47, 56, 68, 71, 72). This is not in itself disqualifying, given the author's prior work on DES visualization and bibliometrics, but it is worth noting in case the editor wishes to monitor citation practice. The main concern is the undocumented and arithmetically inconsistent sample construction, which must be addressed before the empirical claims can be accepted. The qualitative claim that DES remains relevant is plausible and consistent with the broader literature, but the quantitative evidence needs to be made reproducible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked about arXiv:2506.05698. Short version: it's a legitimate but limited bibliometric update, and the central claim—DES is alive, well, and kicking—is almost certainly true, but it was already true in the papers they cite. The value here is the 2010–2024 window and the descriptive breakdowns of domains, themes, and citation impact. If you need a current snapshot of where DES shows up, this is a decent place to look.\n\nWhat the paper does well: the data collection is broad, the thematic clustering and keyword analysis give a reasonable feel for the field's diversity, and the authors are transparent about their tools (VOSviewer, Bibliometrix). The modeling-process section is clear, and the limitations of simulation are acknowledged. The regression is weak (R² = 0.40), but they report it without overclaiming; the growth trend is visible in the raw counts anyway.\n\nWhere it gets soft: the screening step is the load-bearing wall. Table 1 says the Scopus query produced 187,727 documents, then later 187,797; after type exclusions it's 186,797; then they subtract 184,666 as \"Non Discrete-Event Simulation in OR\" and get 2,131. Then they say \"Screened out 9 Irrelevant Documents\" but compute 2,131 − 54 = 2,077. These numbers do not reconcile, and the criterion \"Non Discrete-Event Simulation in OR\" is never defined. No title-abstract protocol, no inter-rater check, no explanation of how 184,666 documents were judged irrelevant. That matters because the yearly publication curve and domain percentages are built on this sample. If the filter was applied unevenly across years or fields, the expansion evidence could be an artifact. The data are proprietary, so outsiders cannot check.\n\nAlso, there are avoidable editorial slips: two tables numbered 5, inconsistent author counts (5,627 vs 5,629), and a stray reference [65] that seems unrelated to simulation. These are minor but suggest the manuscript needs another pass.\n\nFor a reader, this is a useful survey of DES research output over the last 15 years, but I would not rely on its specific numbers until the sample construction is documented. The central qualitative message is robust and consistent with Brailsford's \"alive and kicking\" and the authors' own 2024 study. It deserves a serious referee: the topic is relevant to the OR community, and the flaws are fixable. I'd send it to review with major revision, asking for a step-by-step screening protocol, a corrected table of numbers, and a clear statement of what exactly the 184,666 excluded documents were.","headline":"A useful updated bibliometric snapshot of DES that confirms the field's continued vitality, but the undocumented screening step from 186,797 to 2,077 papers makes the quantitative expansion claims shaky until the methodology is fully disclosed.","tokens_in":18376,"tokens_out":1732,"would_cite":false,"duration_ms":20012,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Discrete-event simulation has not died; a 15-year bibliometric analysis of 2,077 papers concludes it is alive, well, and still expanding as an operations research tool.","keywords":["discrete-event simulation","bibliometric analysis","operations research","simulation everywhere","digital transformation","Industry 4.0","healthcare simulation","COVID-19 modeling"],"falsifier":"Re-run the annual publication counts on the complete 186,797-record query output, or on a differently specified query that does not require 'discret*' in the title or keywords, and check whether the positive 2.03% growth trend and the application-domain spread still appear; if the trend flattens or reverses under reasonable alternative filters, the central claim of continuous expansion would collapse.","tokens_in":17356,"feed_emoji":"📊","tokens_out":6115,"duration_ms":55836,"temperature":0.7,"pith_summary":"This paper tries to settle a long-running dispute in operations research: whether discrete-event simulation (DES) has outlived its usefulness. Using bibliographic metadata from 2,077 peer-reviewed publications between 2010 and 2024, the authors argue that DES is 'alive, well, and kicking,' with a positive (if uneven) annual growth rate of about 2%, a widening spread of application areas, and a central role in modeling COVID-19, digital twins, and Industry 4.0. The claim matters because it reverses the narrative, prominent around 2010, that DES had been superseded by agent-based simulation. The evidence is bibliometric, so the conclusion depends on whether the filtered sample fairly represents the whole field.","feed_headline":"DES is alive and expanding, 15-year publication study finds","feed_subtitle":"A review of 2,077 papers shows the OR method spread across healthcare, logistics, and Industry 4.0.","key_machinery":"The load-bearing instrument is a scientometric pipeline: a bibliographic database query yields 186,797 candidate records, which are filtered down to 2,077 documents; bibliometric software then generates publication counts, citation counts, co-authorship networks, and keyword co-occurrence maps. The analytical heart is a stratification of author keywords into prominent, emerging, and least-frequent themes, plus a linear regression of annual publication counts. This machinery converts raw publication metadata into the study's evidence of growth, thematic breadth, and international collaboration (19.31% international co-authorship across 89 countries).","core_discovery":"The central claim is that discrete-event simulation has continuously expanded as an operations research technique from 2010 to 2024, both in volume and in the breadth of problems it tackles. The study reports 2,077 screened publications across 947 sources, with an average annual growth rate of 2.03%, a regression trend of $y = 2.232x + 120.61$ ($R^2 = 0.40$), and adoption across at least a dozen domains, led by computer science (26%) and engineering (21%). Thematic analysis of author keywords shows that DES remains anchored in classic OR topics—optimization, healthcare, logistics, scheduling—while absorbing newer ones such as Industry 4.0, digital twin, COVID-19, machine learning, and Internet of Things. The authors conclude that the 'simulation everywhere' label is now empirically warranted and that the next step is integrating generative AI into DES practice.","pith_inferences":["Editor's inference: the reported growth may be inflated by the COVID-19 publication spike; the raw yearly counts drop from 163 in 2020 to 147 in 2023, so the 'continuous expansion' claim would be better tested by comparing 2010-2019 and 2021-2024 separately.","Editor's inference: the paper's breadth result could reflect keyword relabeling (adding trendy terms like 'digital twin' to old simulation studies) rather than genuinely new application areas; a full-text topic model could check that.","Editor's inference: the unspecified filter that removed 184,666 records is the main uncontrolled variable; re-running the counts on the full query results would show whether the growth trend survives.","Editor's inference: a direct head-to-head bibliometric comparison of DES versus agent-based simulation over the same years would reveal whether DES's survival also meant it retained market share, or just that both grew."],"forward_implications":["DES remains an active OR backbone for optimizing healthcare, logistics, manufacturing, and service operations, not a legacy technique.","The method is absorbing the digital-transformation toolkit—digital twins, virtual reality, IoT, machine learning—suggesting continued demand for simulation skills in industry.","DES is now a standard emergency-response tool, as shown by the 2020 peak in publications driven by COVID-19 modeling.","Simulation education and practice should keep DES at the core of OR curricula alongside agent-based and system-dynamics approaches.","The next expansion frontier is generative AI integration into model building and experimentation, which the authors flag as an open research direction."],"supporting_citations":[{"why":"Establishes the 'last resort' origin of simulation that the paper argues DES has overcome.","marker":"[1]"},{"why":"Raises the 'reality check' debate about simulation's future that motivates the study.","marker":"[15]"},{"why":"Formulates the 'DES is dead, long live agent-based simulation' challenge that the paper's evidence rebuts.","marker":"[17]"},{"why":"Provides an earlier defense of DES's relevance, which the current bibliometric study extends.","marker":"[19]"},{"why":"Supplies the 'simulation everywhere' framing from the WSC 50th anniversary that gives the paper its title and thesis.","marker":"[24]"},{"why":"Prior bibliometric evaluation of DES during COVID-19 that this study builds on with a longer 2010-2024 window.","marker":"[25]"},{"why":"The most-cited document in the sample, used as evidence of DES's sustained impact in logistics and supply chain research.","marker":"[58]"}],"fun_headline_variants":["Discrete-event simulation keeps expanding, 15-year review shows","DES goes 'simulation everywhere': 2,077 papers affirm growth","From last resort to mainstream: DES expansion documented","DES adoption broadens: healthcare, logistics, Industry 4.0"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 2,077 selected papers are assumed to represent all discrete-event simulation research, even though the rules that excluded roughly 184,000 other records from the initial query are not fully disclosed.","fun_headline_variants_meta":{"raw":{"variants":["Discrete-event simulation keeps expanding, 15-year review shows","DES goes 'simulation everywhere': 2,077 papers affirm growth","From last resort to mainstream: DES expansion documented","DES adoption broadens: healthcare, logistics, Industry 4.0"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000597,"raw_usage":{"total_tokens":2802,"prompt_tokens":966,"completion_tokens":1836,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":1764}},"tokens_in":582,"tokens_out":1836,"duration_ms":15056,"temperature":1.0,"reasoning_tokens":1764,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:13:03.452961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the annual publication counts on the complete 186,797-record query output, or on a differently specified query that does not require 'discret*' in the title or keywords, and check whether the positive 2.03% growth trend and the application-domain spread still appear; if the trend flattens or reverses under reasonable alternative filters, the central claim of continuous expansion would collapse.","supporting_citations":[{"cited_title":"Simulation techniques in operations research - a review","cited_arxiv_id":null,"evidence_quote":"Establishes the 'last resort' origin of simulation that the paper argues DES has overcome."},{"cited_title":"Simulation modelling is 50! Do we need a reality check? Journal of the Operational Research Society 2009, S69-S82","cited_arxiv_id":null,"evidence_quote":"Raises the 'reality check' debate about simulation's future that motivates the study."},{"cited_title":"O., Macal, C","cited_arxiv_id":null,"evidence_quote":"Formulates the 'DES is dead, long live agent-based simulation' challenge that the paper's evidence rebuts."},{"cited_title":"Discrete-event simulation is alive and kicking! Journal of Simulation 2014, 8(1), 1-8, DOI: 10.1057/jos.2013.13","cited_arxiv_id":null,"evidence_quote":"Provides an earlier defense of DES's relevance, which the current bibliometric study extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 'simulation everywhere' framing from the WSC 50th anniversary that gives the paper its title and thesis."},{"cited_title":"Discrete-event simulation is still alive and strong: evidence from bibliometric performance evaluation of research during COVID-19 global health pandemic","cited_arxiv_id":null,"evidence_quote":"Prior bibliometric evaluation of DES during COVID-19 that this study builds on with a longer 2010-2024 window."},{"cited_title":"A, Robinson, S","cited_arxiv_id":null,"evidence_quote":"The most-cited document in the sample, used as evidence of DES's sustained impact in logistics and supply chain research."}],"review_version":1}