{"id":"57bfbb5b-deb8-4f74-a6ec-467d7ce3c5cb","arxiv_id":"2506.12290","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A review proposing that separating 'what is modeled' from 'how it is modeled' ontologies improves hybrid simulation interoperability and reusability.","lead":"This paper argues that formal ontologies, structured vocabularies of concepts and relationships, can make hybrid modeling and simulation clearer and more interoperable. It reviews existing methods and four case studies showing how ontologies help people and machines share a common understanding.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is asserted rather than demonstrated: no baseline comparison shows that ontology-enabled hybrid M&S improves interoperability or reliability.","rationale":"The paper provides a coherent conceptual framework and a useful synthesis of prior work, and the authors are transparent about the challenges in Section 5. The distinction between methodological and referential ontologies, and the mapping onto the three interoperability axes, is a meaningful contribution to the hybrid M&S literature. The MS-SDF integration in Section 3.3 gives practitioners a structured path to incorporate ontologies. However, the paper goes beyond a pure review: the abstract states that the authors 'demonstrate' how the complementary approaches address interoperability challenges, and the Discussion presents the central claim as a conclusion. The evidence for this demonstration consists of four cited case studies, several of which are the authors' own prior publications, and none of which includes a comparative baseline. The reader's weakest_assumption correctly identifies the unverified premise: that semantic alignment is the primary barrier and that ontologies substantially remove it. I agree with this assessment. My stress-test sharpens the concrete consequence: even if the framework is logically coherent, the claimed benefits of interpretability, reusability, and reliability are empirical claims that require comparative validation. The paper's own acknowledgements of tool integration gaps and the cost of ontology development suggest that the unconditional central claim is too strong. Therefore, the appropriate verdict remains CONDITIONAL: accept the framework as a promising approach, but require independent evidence before treating the central claim as established. No change from the reader's verdict is needed; the conditionality precisely reflects this gap.","tokens_in":11461,"tokens_out":4784,"duration_ms":55800,"concrete_test":"Run a controlled comparison on a representative hybrid M&S problem, e.g., a coupled system dynamics–agent-based evacuation model. Implement two versions: (A) full MS-SDF workflow with OWL2 referential and methodological ontologies, SWRL constraints, and SHACL validation, as described in Sections 3.2–3.3; (B) conventional conceptual modeling (e.g., UML) with manual integration and documentation. Use the same modeling question and stakeholders. Measure: (i) number of semantic inconsistencies surfaced before simulation, (ii) person-hours to achieve a valid, accepted simulation, (iii) reuse of the model components in a second, unforeseen scenario. If variant A does not significantly reduce inconsistencies or integration effort, or if the ontology development overhead outweighs the benefits, the central claim fails. If variant A shows clear gains, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the Discussion, is that coordinating referential and methodological ontologies provides the structure and semantic discipline needed to build simulations that are interpretable, reusable, and reliable. For this claim to hold, two things must be true: (1) semantic misalignment is a primary, non-marginal barrier to hybrid M&S interoperability, and (2) the proposed ontology-based workflow substantially removes that barrier in practice. The paper asserts (1) in Section 1 and defines interoperability through ontology-style semantic alignment in Section 2, but offers no empirical evidence that non-semantic barriers (tool coupling, execution semantics, data format conversion) are not equally or more decisive. For (2), the four application examples in Section 4 are summaries of prior, largely self-cited work; none includes a baseline or controlled comparison showing that the ontology-driven approach outperforms conventional integration methods on quantifiable outcomes like development time, number of semantic conflicts detected, or component reusability. The paper itself acknowledges in Section 5 that ontology development is resource intensive, that alignment in practice is non-trivial, and that tools are not natively integrated with simulation engines. These caveats are consistent with the claim that ontologies are a useful supplement, but they undercut the unconditional version of the central claim. Thus the load-bearing assumption — that the ontology framework is not merely expressive but actually delivers the claimed interoperability and reliability benefits — remains untested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that ontology engineering, organized around the distinction between methodological ontologies (how to model) and referential ontologies (what is modeled), should be a standard component of hybrid modeling and simulation (M&S) workflows. It maps these two ontology types onto three interoperability axes—Human–Human, Human–Machine, and Machine–Machine—and uses the M&S System Development Framework (MS-SDF) to show where ontological strategies enter the M&S lifecycle. Four application examples—sea-level rise analysis, Industry 4.0 modeling, artificial societies for policy support, and cyber threat evaluation—are presented as evidence of feasibility. The paper closes with challenges, including the resource intensity of ontology development, nontrivial semantic alignment in practice, and the lack of native integration between ontology tools and simulation engines.","tokens_in":11706,"tokens_out":4165,"duration_ms":50839,"significance":"As a conceptual synthesis, the paper has value: the methodological/referential distinction is clearly presented, the three interoperability axes provide a useful organizing scheme, and the inclusion of external examples by Jurasky et al. (2021) and Teng et al. (2025) shows that the approach is not exclusively an artifact of the authors' own prior work. The paper also names concrete mechanisms—competency questions, ontology design patterns, layered architectures, and SWRL rules—rather than remaining at the level of general promises. However, the manuscript's central conclusion is stronger than the evidence supports. No quantitative, controlled, or baseline evidence is provided that ontology-driven workflows improve reliability, reusability, or interpretability relative to conventional integration practice, and the two most fully described workflows come from the authors' own prior publications. The paper is best read as a position statement and research agenda, not as a demonstrated result.","major_comments":[{"comment":"The four application examples are summaries of previously published work, and none includes a baseline or comparison condition. For example, §4.1 reports that the ontology-aligned approach 'aided the identification' of areas to protect, but no measure of correctness, development cost, or comparison with a non-ontology integration method is reported. Similarly, §4.4 states that OntoCPS4PMS 'was able to simulate' the 2015 Ukrainian power grid attack, but no evaluation metrics, runtime behavior, or comparison with a hand-coded rule-based simulation is provided. Since the Discussion's central claim is that ontologies make simulations interpretable, reusable, and reliable, the evidence as presented is insufficient to support the causal/benefit formulation. I recommend explicitly reframing the paper as a perspective or roadmap, treating the four cases as existence proofs of feasibility, and adding a short discussion of what empirical evidence would be needed to test the claim that ontology-enabled workflows outperform conventional integration.","section":"Section 4, especially §4.1 and §4.4"},{"comment":"The paper's load-bearing premise is that semantic misalignment is a primary barrier to hybrid M&S interoperability, and this premise is asserted rather than argued. It enters through the 'WHAT is modeled' argument in §1 and is built into §2, where interoperability is effectively defined through ontology-style semantic alignment along the three axes. However, §5 itself acknowledges that ontology tools are not natively integrated with simulation engines and that semantic alignment in practice is non-trivial. The manuscript does not address the possibility that non-semantic barriers—tool coupling, execution semantics, data-format conversion, or performance—might dominate interoperability effort in hybrid M&S. A concrete strengthening would be to discuss or propose a controlled comparison of an ontology-enabled integration against a conventional mediator-based integration on a common task, isolating the contribution of the semantic layer.","section":"Section 1 and Section 2"},{"comment":"The MS-SDF framework in §3.3 and the sea-level rise use case in §4.1 both draw directly on Tolk et al. (2013), and §4.3 is based on Clemen et al. (2025), work in which the second author is involved. Using these self-cited cases to validate the framework creates a circularity: the framework is claimed to be useful, and the main evidence of its usefulness comes from studies that presuppose it. The external cases in §4.2 and §4.4 mitigate this concern, but the manuscript should explicitly classify the self-cited examples as the authors' prior work and state that they are existence proofs rather than independent validation. This would also make the degree of independent support for the central claim more transparent.","section":"Section 3.3 and §4.1/§4.3"}],"minor_comments":[{"comment":"Figure 1 is referenced ('as shown in Figure 1'), but no figure content appears in this preprint; the camera-ready version should include the diagram, since the seven steps are central to the argument being made.","section":"Section 3.3"},{"comment":"The passage 'Thehybridsimulationwasthendevelopedusingthealignedreferentialontologiestoderivetheconceptualmodel' is missing spaces and should read 'The hybrid simulation was then developed using the aligned referential ontologies to derive the conceptual model'.","section":"Section 4.1"},{"comment":"The reference 'Natasha Noy and Alan Rector 2006, April' is inconsistent with the citation style used elsewhere; in the text the authors are referred to by first names and the year, but the reference list entry should follow the same format as the other entries.","section":"References"},{"comment":"The sentence 'Despite the importance of interoperability, the term itself is highly ambiguous' is followed by a long list of citations without a synthesis; consider briefly stating which dimensions of ambiguity are most relevant to hybrid M&S, so the subsequent three-axis distinction is better motivated.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"This is a plausible Winter Simulation Conference position paper, and the external examples (Jurasky et al. 2021; Teng et al. 2025) indicate that the topic is of interest to the community. The main weakness is the gap between the modest evidence and the strong claims in the Discussion; a careful reframing and the addition of an evaluation-oriented discussion would make the contribution sound. No broader integrity concerns beyond the self-citation pattern noted in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this before reading: it is a position paper, not an experimental study. The useful part is the stepwise mapping of the MS-SDF development steps onto the three interoperability axes, which gives a concrete way to see where ontologies enter a hybrid M&S workflow. The rest is a well-organized restatement of ideas that Tolk and others have published before.\n\nWhat the paper does well: the methodological/referential distinction is explained cleanly, the layered ontology strategy (top, mid, domain) is presented with relevant examples, and the four case studies give a fair picture of what has been tried. The two external cases—Jurasky et al. and Teng et al.—provide independent grounding, which is more than many papers of this kind offer.\n\nSoft spots, in proportion: the gap between the claimed demonstration and what is actually shown. The central claim in the Discussion—that ontologies provide the semantic discipline needed for interpretable, reusable, reliable simulations—is asserted rather than demonstrated. None of the four use cases includes a baseline or controlled comparison showing that the ontology-driven approach beats conventional integration methods on development time, conflict detection, or reusability. The paper itself acknowledges that ontology development is resource intensive and that tool integration is lacking, which is honest but undercuts the unconditional version of the claim. That said, the stress-test concern is a fair reading but not a fatal one, because the paper reads better as an advocacy piece and framework proposal than as an empirical proof. If the authors had said \"we propose a framework and review supportive evidence\" instead of \"we demonstrate,\" the gap would be minor.\n\nCircularity: the MS-SDF and two of the four case studies come from the authors' own prior work. That is a real weakness, but the external cases corroborate the general point, so it is not damning. The original contribution is modest—the mapping of MS-SDF steps to the interoperability axes—but that mapping is a legitimate pedagogical and analytic device.\n\nWho this is for: readers in the M&S community who want a compact overview of how ontologies are used for hybrid modeling, and who want a structured way to choose where ontology support can help. It would be a good introduction for someone new to the area. A reader looking for hard evidence or new results will be disappointed.\n\nRecommendation: yes, send it to peer review. It is a coherent position paper from people who know the field, and the mapping deserves referee time. The referee should push for the authors to either soften the demonstration language or add at least one baseline comparison. I would accept it as a conference paper with minor revisions.","headline":"A clearly written synthesis of ontology approaches for hybrid M&S, but the central claim is asserted rather than shown; the new mapping of MS-SDF to interoperability axes is useful but modest.","tokens_in":12229,"tokens_out":2055,"would_cite":true,"duration_ms":25379,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that hybrid modeling and simulation becomes interpretable, reusable, and reliable when ontologies are separated into a methodological layer for how to model and a referential layer for what is modeled, coordinated across…","keywords":["hybrid modeling and simulation","ontologies","interoperability","methodological ontology","referential ontology","semantic alignment","layered ontology architecture","OWL2/SWRL"],"falsifier":"Take a hybrid simulation project coupling an agent-based and a system-dynamics model, and build it twice: once following the paper's ontology-aligned MS-SDF workflow and once with conventional interfaces and shared data schemas only. If the ontology-driven version shows no measurable reduction in integration errors, rework, or cross-team misunderstanding, the central claim that semantic discipline is the decisive factor is falsified.","tokens_in":11273,"feed_emoji":"🧩","tokens_out":6411,"duration_ms":69011,"temperature":0.7,"pith_summary":"This paper argues that hybrid modeling and simulation (M&S) suffers less from missing computational power than from missing semantic discipline: different experts and tools describe the same situation in ways that do not line up. The proposed fix is to build simulations with two coordinated kinds of ontology: methodological ontologies that prescribe how something is modeled, and referential ontologies that describe what is modeled, including all the partially conflicting viewpoints stakeholders bring. The paper shows this pairing addresses interoperability along three axes—human to human, human to machine, and machine to machine—and illustrates it with four application cases. A reader should care because, if the argument holds, ontology engineering becomes a standard part of the simulation workflow rather than an optional documentation layer.","feed_headline":"Two ontology layers make hybrid simulations interoperable","feed_subtitle":"Separating what is modeled from how it is modeled aligns humans and machines across all three interoperability axes.","key_machinery":"The central object is the pairing of methodological and referential ontologies. A methodological ontology is a normative blueprint for how to model something using a chosen paradigm, such as discrete-event, agent-based, or system dynamics. A referential ontology is a descriptive, logic-based representation of what is modeled—capable of holding multiple, even mutually inconsistent, stakeholder viewpoints. The argument is carried by coordinating these two layers with the three interoperability axes and with a layered ontology architecture (top-level, mid-level, domain) encoded in decidable languages such as OWL2 and extended by rule languages like SWRL and SHACL. The M&S System Development Framework (MS-SDF) then supplies seven steps in which these ontologies are used, from capturing reality and assumptions through reference and conceptual models to verification and validation.","core_discovery":"The central claim is that hybrid M&S becomes interpretable, reusable, and reliable when the referential question of what is modeled is separated from the methodological question of how it is modeled, and the two are explicitly coordinated through formal ontologies. Referential ontologies capture all viewpoints as true/false statements about a system, allowing inconsistent perspectives to coexist in a reference model. Methodological ontologies act as normative blueprints that specify how simulation constructs such as events, entities, and dependencies must be instantiated. Together, aligned with human and machine interoperability requirements, they turn ontologies into the semantic structure of a hybrid simulation: descriptive for the domain, prescriptive for the simulation, and contractual between independent components.","pith_inferences":["If semantic alignment is truly the bottleneck, then controlled experiments comparing ontology-driven integration with conventional interface-based integration in identical hybrid projects should show large differences in rework and ambiguity; the paper does not run such an experiment.","The same distinction could sharpen explainable AI: referential ontologies say what a model's terms mean, methodological ontologies explain how its outputs were derived, giving regulators precise handles for audit.","A testable extension is to generate hybrid simulation code directly from ontology-encoded constraints with LLM assistance, using the methodological ontology as a guardrail; the paper notes this direction but leaves it open."],"forward_implications":["Ontology engineering moves from optional documentation to a load-bearing component in hybrid M&S workflows.","A single referential ontology can hold inconsistent expert viewpoints, while each simulation implements a consistent subset, enabling ensembles of models from one knowledge base.","Methodological ontologies as normative artifacts make simulation code generation, constraint validation, and on-the-fly model composition possible.","Shared referential ontologies let independently built simulation components exchange and reason over information without human rewiring.","Verification and validation unify: validation checks the simulation against the referential ontology and modeling question, verification checks the relationships using the methodological ontology."],"supporting_citations":[{"why":"Introduces the distinction between methodological and referential ontologies that the whole argument rests on.","marker":"Hofmann et al. 2011"},{"why":"Supplies the working definition of ontology as controlled vocabularies plus formal relations among entities.","marker":"Arp et al. 2015"},{"why":"Defines the reference-modeling approach and the MS-SDF, and provides the sea-level rise application case.","marker":"Tolk et al. 2013"},{"why":"Early demonstration that meta-level ontology relations can measure conceptual alignment and interoperability of simulation models.","marker":"Yilmaz 2007"},{"why":"DeMO exemplifies a methodological ontology for discrete-event M&S.","marker":"Silver et al. 2011"},{"why":"Supplies the Industry 4.0 use case: a simulation ontology meta-model that transforms semantic knowledge into executable hybrid simulations.","marker":"Jurasky et al. 2021"},{"why":"Grounds hybrid simulation as a transdisciplinary enabler and links hybrid M&S to the three interoperability axes.","marker":"Tolk et al. 2021"},{"why":"Defines hybrid simulation's purpose and benefits, the problem the ontology approach is meant to serve.","marker":"Mustafee et al. 2017"}],"fun_headline_variants":["What vs how: ontology split boosts hybrid sim interoperability","Referential and methodological ontologies align hybrid sims","Ontology duo bridges human and machine in hybrid modeling","Separating what and how in modeling via ontologies","Ontologies as semantic backbone for hybrid simulation workflows"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole framework depends on semantic misalignment being the main obstacle to hybrid M&S interoperability; if non-semantic barriers such as tool coupling, execution-semantics mismatches, and data-format incompatibilities dominate, the proposed ontology discipline cannot deliver the promised benefits.","fun_headline_variants_meta":{"raw":{"variants":["What vs how: ontology split boosts hybrid sim interoperability","Referential and methodological ontologies align hybrid sims","Ontology duo bridges human and machine in hybrid modeling","Separating what and how in modeling via ontologies","Ontologies as semantic backbone for hybrid simulation workflows"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000695,"raw_usage":{"total_tokens":3081,"prompt_tokens":823,"completion_tokens":2258,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":439,"completion_tokens_details":{"reasoning_tokens":2183}},"tokens_in":439,"tokens_out":2258,"duration_ms":17978,"temperature":1.0,"reasoning_tokens":2183,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:53:19.161880+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a hybrid simulation project coupling an agent-based and a system-dynamics model, and build it twice: once following the paper's ontology-aligned MS-SDF workflow and once with conventional interfaces and shared data schemas only. If the ontology-driven version shows no measurable reduction in integration errors, rework, or cross-team misunderstanding, the central claim that semantic discipline is the decisive factor is falsified.","supporting_citations":[],"review_version":1}