{"id":"7f266b5c-9e81-4480-b844-791d3bafb993","arxiv_id":"2505.09995","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey classifies 40+ open-source edge-computing simulators and emulators into packet-level, application-level, and emulators, and compares them across five dimensions.","lead":"This paper reviews more than 40 open-source simulators and emulators used to test edge computing systems. It groups them into three types and compares them across five dimensions, offering a tool-selection guide for researchers in computing and networking convergence.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Comprehensiveness is the load-bearing claim, and the 42-tool set may omit at least one general-purpose open-source edge simulator (e.g., EdgeSimPy) that meets the paper's own criteria; an inclusion audit would settle it.","rationale":"The reader's weakest assumption was that Tables I-V are accurate and complete, with selection criteria potentially biasing the comprehensive claim. I agree that completeness is a primary risk, but I push it further: there is a concrete, checkable omission candidate (EdgeSimPy) that appears to meet the paper's own inclusion criteria. This is more decisive than generic concern about manual table accuracy, because a single qualifying omission directly falsifies the word 'comprehensive' even if all included entries are correct. The GitHub repository's 80+ tracked tools also make the 42-tool gap a visible, unexplained discrepancy. I do not think this concern warrants moving away from the reader's CONDITIONAL verdict: the survey's core organization and five-dimension framework are useful, and the issue is resolvable by a verification appendix rather than by rewriting the paper. Hence UNCHANGED. I marked agreement as 'partial' because the reader emphasized cell-level accuracy and specialized-tool exclusions, whereas my concern centers on a specific potential omission of a general-purpose tool and a tool-level inclusion audit.","tokens_in":27136,"tokens_out":10168,"duration_ms":103328,"concrete_test":"Run a tool-level inclusion audit. From the authors' GitHub repository and recent edge-computing literature, compile a candidate list that includes EdgeSimPy, EdgeNetworkSim, and any other tracked tools excluded from Tables I-V. For each candidate, record whether it meets the Section I.B criteria: open-source code link, update within the six-year window, general-purpose (not UAV- or caching-specific), and relevance to computing and networking convergence. If the subset satisfying all criteria but absent from Tables I-V is non-empty, the 'comprehensive' claim fails and the paper needs either to add those tools or to give a per-tool exclusion reason.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that this is a 'comprehensive survey' of open-source edge computing simulators and emulators. That claim is load-bearing because the paper's practical contribution is a tool-selection guide: a reader who follows the tables should see all viable open-source options. Section I.B defines the inclusion criteria (open-source with a code link, updated within six years, general-purpose rather than specialized). The paper's own GitHub repository tracks 80+ tools, yet Tables I-V contain only 42. The text explains exclusions categorically but does not provide a tool-by-tool reconciliation. In particular, EdgeSimPy--a general-purpose, open-source, Python-based edge computing simulator that appears to satisfy the stated criteria--is absent from all five tables. If EdgeSimPy or comparable omitted tools meet the paper's own criteria, then the abstract's unqualified 'comprehensive' claim is false by the paper's own standard, and a researcher using the guide would miss a valid option. This is a correctness/scope risk, not a stylistic one: even if every cell in Tables I-V is accurate, omission of a qualifying tool undermines the central contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper surveys open-source edge-computing simulators and emulators, with a companion GitHub repository, classifying more than 40 tools into packet-level simulators, application-level simulators, and emulators. It compares the tools across five dimensions (computing paradigms, resource simulation, performance metrics, resource management and utilization, and usability), presents the results in five tables, draws lessons about paradigm support and resource representation, and proposes five future research directions. The intended contribution is a practical tool-selection guide for researchers working on the convergence of computing and networking (CNC).","tokens_in":27351,"tokens_out":5278,"duration_ms":55563,"significance":"If the tool inventory is complete and the table entries are accurate, the survey would be a useful and timely reference for researchers selecting simulation and emulation platforms for CNC experiments. The companion GitHub repository, the explicit tracing of core-engine inheritance (e.g., CloudSim-derived tools), and the inclusion of community metrics such as stars and forks are concrete strengths that go beyond a purely narrative survey. The paper does not offer formal proofs or quantitative benchmarks, but the central value is as a structured, multidimensional comparison; its correctness hinges on the completeness and accuracy of the manually compiled Tables I–V.","major_comments":[{"comment":"The paper's central claim of comprehensiveness is not auditable as written. The selection criteria in Section I.B exclude tools without code updates in more than six years and tools that are specialized rather than general-purpose, but the terms 'specialized' and 'general-purpose' are never defined, and the manuscript does not provide a tool-by-tool reconciliation of the 80+ tools in the companion repository with the 42 tools in Tables I–V. At least one actively maintained, open-source, general-purpose edge-computing simulator, EdgeSimPy, appears to satisfy the stated criteria yet is absent from all five tables. If EdgeSimPy or comparable qualifying tools are omitted, the abstract's unqualified 'comprehensive' claim is false by the paper's own standard, and the selection guide would miss valid options. Please add an explicit inclusion/exclusion audit, define the 'general-purpose' criterion operationally, and either add qualifying tools or restrict the claimed scope.","section":"Section I.B and Tables I–V"},{"comment":"The relationship between the literature-review process and the final tool list is unclear. Section I.B reports a Web of Science / Ei Compendex search yielding 192 results, refined to 41 unique papers, but the text never maps those 41 papers to the 42 tools in Tables I–V, nor does it explain how tools such as NS-3 and OMNeT++, which predate the five-year window, were selected. Without this mapping, the reader cannot distinguish tools that were deliberately excluded because they failed a criterion from tools that were simply not considered. This is load-bearing for the survey's completeness claim.","section":"Section I.B and Section II.A"},{"comment":"The three-way classification into packet-level simulators, application-level simulators, and emulators is central to the paper, but the operational rule for resolving overlap is not fully specified. The Remark in Section II.A states that classification is guided by the 'core purpose and most prominent use case,' yet Table V then assigns each tool a single 'Type' value (DES, Emu, or Hybrid) without showing how the rule was applied. For example, Simu5G is labeled Hybrid and PureEdgeSim is labeled DSE (almost certainly a typo for DES), but the criteria for 'Hybrid' are not defined in the text. The paper should make the classification rule precise enough that an independent reader could reproduce the column.","section":"Section II.A and Table V"}],"minor_comments":[{"comment":"The organization paragraph contains an unresolved cross-reference: 'explores the core engines and supported computing paradigms (Section ??)' should be replaced with the actual section number.","section":"Section I.C"},{"comment":"The paragraph beginning 'We believe these adjustments, along with the detailed metrics in Section V, will provide a clearer and more accurate representation...' is leftover reviewer-response text and should be removed from the published version.","section":"Section VI (paragraph before Section VII)"},{"comment":"The simulator YAFS is consistently rendered as 'Y AFS' in the text and in all four tables; this should be corrected to YAFS, matching reference [38].","section":"Section III.C and Tables I, II, IV, V"},{"comment":"The sentence 'For emulators, basic metrics can be directly obtained using well-developed tools such astopandiPerfonLinux' has missing spacing and should read 'such as top and iPerf on Linux.'","section":"Section IV.A"},{"comment":"Several grammatical and typographical errors should be fixed, including 'faas-sim and SimGrid belongs to this category, as it simulates basic components' (subject-verb agreement and pronoun agreement), 'having a uniformed optimizing target' (should be 'a uniform optimizing target'), and the double comma in 'due to diverse user requirements,,'.","section":"Section III.A and Section V.C"},{"comment":"The 'Statistics' column is labeled inconsistently: the table footnote says 'citations/licenses/project information (Stars, Forks, Watchers, Contributors)', but the values in the column are GitHub stars, forks, watchers, and contributors, not citations. The column header and footnote should be aligned, and the retrieval date for these metrics should be stated.","section":"Table V"}],"recommendation":"major_revision","confidential_remarks":"The self-citation of EasiEI and the companion repository is not circular in a technical sense, because the survey's conclusions are grounded in external tool documentation and prior literature rather than in the authors' own simulator. However, the authors should be asked to verify that EasiEI meets the same inclusion criteria as the other tools and to disclose the relationship between the first author's repository and the 42-tool list. The main risk is the unverified completeness claim; if the authors can provide a full inclusion/exclusion audit, including the EdgeSimPy question, the paper could become a valuable reference."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"A useful but uneven survey; the central claim has a hole. The tool tables are dense and mostly plausible, but the selection process is not transparent enough to support 'comprehensive,' and one obvious general-purpose edge simulator (EdgeSimPy) is missing.\n\nWhat's new: the CNC (computing-networking convergence) lens and the five-dimension comparison are a real service. The three-way split (packet-level, application-level, emulators) is not novel on its own, but the paper applies it consistently and shows inheritance relations in Fig. 2. It also covers recent tools (EasiEI, SimFaaS, StarryNet) that older surveys miss, and the tables include practical details like license, language, and scale numbers. The authors are honest about classification overlaps and about not being able to verify every cell from the docs.\n\nWhere it's soft: the comprehensiveness claim is load-bearing. Section I.B gives categorical filters but no tool-by-tool reconciliation. The GitHub repo has 80+ tools; the tables have 42. EdgeSimPy—a general-purpose, open-source, Python-based edge simulator—meets the stated criteria as far as I can tell and is absent. That is not a stylistic objection; a researcher using the tables would miss a legitimate option. The editorial state is also rough: an unresolved 'Section ??', a leftover reviewer-response sentence at the end of Section VI, and at least one typo ('Y AFS') in the tables. These are minor individually but signal an incomplete final pass.\n\nThe underlying survey content is defensible. The classification is thoughtful, and the future directions (packet processing in app-level simulators, edge environment support, user-defined metrics) are sensible. The math and data are not an issue here—it's a survey, so the burden is on selection and characterization.\n\nWho this is for: anyone choosing an edge simulator for research or teaching, and survey readers wanting a CNC-oriented map. With an inclusion audit and a cleanup pass, it would be a solid reference. As submitted, I'd send it to referees but expect major revision on the scope justification and verification of table entries.","headline":"Useful but uneven survey of edge simulators; the 'comprehensive' claim needs an inclusion audit before the guide can be trusted.","tokens_in":27860,"tokens_out":2400,"would_cite":false,"duration_ms":24306,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper maps more than 40 open-source edge simulators and emulators and compares them on five dimensions so researchers can pick the right tool for validating computing and networking convergence ideas.","keywords":["edge computing","simulator","emulator","open-source software","computing and networking convergence","computing paradigms","performance metrics","resource management"],"falsifier":"Take one row of Tables I–V, for instance the claim that EdgeCloudSim models WLAN/WAN links and device mobility, and run a minimal edge-cloud scenario in that tool. If the advertised paradigm, metric, or scheduling policy is missing or unusable in practice, that row's guidance is false; enough such mismatches across the tables would refute the survey's usefulness as a selection guide.","tokens_in":26964,"feed_emoji":"⚙️","tokens_out":5845,"duration_ms":53995,"temperature":0.7,"pith_summary":"The paper aims to establish a practical tool-selection guide for researchers working on the convergence of computing and networking. It surveys more than 40 open-source edge simulators and emulators, sorts them into packet-level simulators, application-level simulators, and emulators, and compares them along five dimensions: computing paradigms, resource simulation, performance metrics, resource management and utilization, and usability. A reader who trusts the survey gains a quick way to screen tools before reading code or documentation, which matters because edge-computing research often relies on self-written scripts and a mismatched tool can skew results or waste effort. The comparison also exposes weak spots in the ecosystem, such as sparse support for edge-environment conditions, custom metrics, and packet-level processing inside application-level simulators, and it uses those gaps to propose future directions for tool builders.","feed_headline":"Edge-computing simulators sorted into three classes and five axes","feed_subtitle":"Choose packet-level, application-level, or emulator tools by the features your edge study needs.","key_machinery":"The survey's load-bearing instrument is a five-dimension comparison grid — computing paradigms, resource simulation, performance metrics, resource management and utilization, and usability — applied to tools grouped into three implementation levels: packet-level simulation, application-level simulation, and emulation. The grid is what lets the paper hold tools as different as NS-3 and CloudSim side by side, by mapping each tool's core engine, its inheritance from a base engine such as CloudSim or OMNeT++, and the features it exposes in each dimension. The five tables produced by this grid are the mechanism that turns more than 40 separate tools into a comparable landscape, and they are also what makes the survey actionable for a reader choosing a tool.","core_discovery":"The authors claim that the open-source edge-computing ecosystem can be usefully organized by implementation level and by a common set of evaluation axes. They classify tools into packet-level simulators such as NS-3 and OMNeT++ and their extensions, application-level simulators such as CloudSim and its many descendants plus flow-level and serverless simulators, and emulators such as the Mininet family, CORE, Fogify, and StarryNet. For each tool they document which computing paradigms it supports, which hardware and communication resources it represents, which performance metrics it reports, how it allocates resources and schedules tasks, and how usable it is in terms of modular components, scripting, logging, and visualization. The payoff claim is that the five resulting tables let a researcher match a tool's strengths to the fidelity and scale their experiment needs, and they tell simulator builders where the ecosystem is still weak.","pith_inferences":["An implication the authors leave implicit is that their exclusion of specialized tools such as Emu5GNet, EmuEdge, and RaSim means the survey is best read as a general-purpose shortlist; a reader working on satellite, V2X, or constrained-device scenarios may still need a separate dedicated-tool pass.","Because the five tables record a snapshot of actively maintained repositories, the specific entries will decay quickly; the durable contribution is the five-dimension framework itself, which could stay useful as a living checklist that is re-run as repositories evolve.","The paper's observation that application-level simulators lack packet-level processing suggests an unexplored design: a hybrid that combines CloudSim-scale resource scheduling with NS-3-style packet event tracing could serve computing-while-transmitting scenarios better than either tool class alone."],"forward_implications":["A researcher facing an edge-computing experiment can use the three-level classification to decide upfront whether they need packet-level fidelity, application-level scalability, or real-code emulation.","The five tables provide a quick shortlisting pass: check which tools support the needed computing paradigm, resource granularity, metric, and scheduling policy before reading any code.","The survey identifies concrete weak spots in the ecosystem — edge-environment factors such as weather, user-defined metric interfaces, and scenario-script and visualization support are rare — giving tool builders a targeted agenda.","Future edge simulators should move toward integrating multiple computing paradigms and toward bringing packet-level processing into application-level tools, since those are the gaps the comparison exposes.","The accompanying repository extends the comparison beyond the 40 surveyed tools to an actively curated list of more than 80 open-source projects, keeping the guide live as new tools appear."],"supporting_citations":[{"why":"CloudSim is the foundational application-level simulator and the baseline engine for most cloud, fog, and edge derivatives in the survey.","marker":"[6]"},{"why":"NS-3 is the representative packet-level simulator and the base engine for extensions such as EasiEI that add in-network computing.","marker":"[68]"},{"why":"OMNeT++ is the core engine behind several packet-level simulators and emulator-adjacent tools, including Simu5G, Artery, and ECSNeT++.","marker":"[84]"},{"why":"iFogSim2 is a major fog and mobile-edge extension of CloudSim and a reference point for mobility, clustering, and microservice comparisons.","marker":"[44]"},{"why":"EdgeCloudSim is a prominent edge-specific simulator whose WLAN/WAN, mobility, and multi-tier features are compared in the tables.","marker":"[76]"},{"why":"ndnSIM is the information-centric networking simulator that supports the compute-first networking extension CFN.","marker":"[48]"},{"why":"EasiEI is the authors' own NS-3-based edge simulator and is cited as the origin of their tool-tracking effort.","marker":"[92]"},{"why":"Mininet and Mininet-WiFi are the representative emulators and the base for derived emulators such as MaxiNet, Mini-NDN, and EmuFog.","marker":"[37]"},{"why":"The authors' curated repository supplies the survey's tool list and the ongoing tracking that extends beyond the 40 surveyed tools.","marker":"[62]"}],"fun_headline_variants":["Edge simulators sorted: 3 classes, 5 axes, 40+ tools","Edge simulators: from NS-3 to Fogify in 5 axes","Pick edge simulators wisely: 3 types, 5 checks","40+ edge simulators ranked on five dimensions","Edge computing tools: classify, evaluate, choose"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey's usefulness rests on the accuracy of its five hand-compiled tables: if a table misstates what a tool actually supports, a researcher who follows the guide can pick the wrong tool.","fun_headline_variants_meta":{"raw":{"variants":["Edge simulators sorted: 3 classes, 5 axes, 40+ tools","Edge simulators: from NS-3 to Fogify in 5 axes","Pick edge simulators wisely: 3 types, 5 checks","40+ edge simulators ranked on five dimensions","Edge computing tools: classify, evaluate, choose"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000297,"raw_usage":{"total_tokens":1704,"prompt_tokens":912,"completion_tokens":792,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":703}},"tokens_in":528,"tokens_out":792,"duration_ms":6721,"temperature":1.0,"reasoning_tokens":703,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:18:39.351162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one row of Tables I–V, for instance the claim that EdgeCloudSim models WLAN/WAN links and device mobility, and run a minimal edge-cloud scenario in that tool. If the advertised paradigm, metric, or scheduling policy is missing or unusable in practice, that row's guidance is false; enough such mismatches across the tables would refute the survey's usefulness as a selection guide.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OMNeT++ is the core engine behind several packet-level simulators and emulator-adjacent tools, including Simu5G, Artery, and ECSNeT++."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"iFogSim2 is a major fog and mobile-edge extension of CloudSim and a reference point for mobility, clustering, and microservice comparisons."},{"cited_title":"EdgeCloudSim: An environment for performance evaluation of edge computing sys- tems.Transactions on Emerging Telecommunications Technologies, 29(11):e3493, 2018","cited_arxiv_id":null,"evidence_quote":"EdgeCloudSim is a prominent edge-specific simulator whose WLAN/WAN, mobility, and multi-tier features are compared in the tables."},{"cited_title":"On the evolution of ndnSIM: an open-source simulator for ndn experimenta- tion.ACM Computer Communication Review, 2017","cited_arxiv_id":null,"evidence_quote":"ndnSIM is the information-centric networking simulator that supports the compute-first networking extension CFN."},{"cited_title":"EasiEI: A simulator to flexibly modeling complex edge computing environments, 2023","cited_arxiv_id":null,"evidence_quote":"EasiEI is the authors' own NS-3-based edge simulator and is cited as the origin of their tool-tracking effort."},{"cited_title":"A network in a lap- top: rapid prototyping for software-defined networks","cited_arxiv_id":null,"evidence_quote":"Mininet and Mininet-WiFi are the representative emulators and the base for derived emulators such as MaxiNet, Mini-NDN, and EmuFog."},{"cited_title":"Awesome edge computing, 2023","cited_arxiv_id":null,"evidence_quote":"The authors' curated repository supplies the survey's tool list and the ongoing tracking that extends beyond the 40 surveyed tools."}],"review_version":1}