pith. sign in

arxiv: 2604.19686 · v1 · submitted 2026-04-21 · 📡 eess.SY · cs.SY

Towards Reproducible Test Annotation for Cyber-Physical Energy Systems using Ontology-driven Dataspaces

Kai Heussen , Jawad Kazmi , Narges Mehran , Artjoms Obushevs , Terence O'Donnell , Thomas I. Strasser This is my paper

Pith reviewed 2026-05-10 01:45 UTC · model grok-4.3

classification 📡 eess.SY cs.SY
keywords reproducibilityontologydataspacecyber-physical energy systemstest annotationsemantic metadatacross-laboratory testing
0
0 comments X

The pith

Ontology-driven dataspaces provide machine-actionable descriptions that support reproducible testing of cyber-physical energy systems.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that standard test documentation in cyber-physical energy systems lacks formal semantics, which blocks reliable reproduction of experiments, data sharing, and advanced analysis. An ontology-based dataspace supplies structured, machine-readable metadata to close this gap. The authors demonstrate the approach with cross-laboratory use cases, surface remaining semantic and metadata shortcomings, and propose an open three-viewpoint ontology framework to direct future extensions.

Core claim

Reproducibility in cyber-physical energy system testing requires formal semantic descriptions that current practices do not supply. An ontology-driven dataspace supplies these descriptions in machine-actionable form. Application to representative cross-laboratory use cases shows the approach is feasible yet reveals persistent semantic and metadata gaps; an open three-viewpoint ontology framework is therefore offered to guide systematic extensions that address those gaps.

What carries the argument

The open three-viewpoint ontology framework, which organizes test descriptions into complementary semantic perspectives to enable machine-actionable annotation and traceability.

If this is right

  • Test data become directly usable by AI analysis tools without manual reinterpretation.
  • Cross-laboratory collaboration improves because experiment metadata carry explicit semantic context.
  • Traceability of results increases, supporting regulatory and scientific validation requirements.
  • Identified semantic gaps become explicit targets for community ontology development.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The framework could serve as a template for ontology design in other cyber-physical domains facing similar reproducibility challenges.
  • Adoption would require mapping existing test protocols onto the three viewpoints before full interoperability is achieved.
  • A measurable outcome would be the fraction of test variables that can be automatically validated for semantic consistency across sites.

Load-bearing premise

That the selected cross-laboratory use cases are representative enough to establish both feasibility of the ontology approach and the effectiveness of the proposed three-viewpoint framework in closing identified gaps.

What would settle it

Implementation of the three-viewpoint framework on a new set of cross-lab experiments followed by an independent attempt to reproduce the annotated tests; if reproduction success does not increase relative to current practices, or if semantic gaps remain unaddressed, the claim would be undermined.

Figures

Figures reproduced from arXiv: 2604.19686 by Artjoms Obushevs, Jawad Kazmi, Kai Heussen, Narges Mehran, Terence O'Donnell, Thomas I. Strasser.

Figure 2
Figure 2. Figure 2: Testing process example with provenance concepts: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Inverter test case study setup. B. Digital-twin-based Workflow As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Digital-twin-based workflow. should be well reproducible. The replicability of this process is a case for provenance as it requires linking predictions back to the original data and model. IV. METADATA ANNOTATION AND REPRODUCIBILITY Regarding the PV inverter test case study, this section describes how we applied an existing dataspace ontology to an￾notate metadata from a facilitated test of an OpenSVP soft… view at source ↗
Figure 6
Figure 6. Figure 6: Ontology graph of test annotation in the open-source [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Reproduction of EN 50549-10:2022 tests. lighting the need for consistent, complete setup documenta￾tion. Improvements are needed in impedance characterization, metadata capture, and more systematic recording to support reproducibility and automated post-processing. V. PROPOSED FRAMEWORK FOR ONTOLOGY-BASED ANNOTATION FOR EXPERIMENT REPRODUCIBILITY The conceptual framework for reproducible testing data entai… view at source ↗
Figure 8
Figure 8. Figure 8: Outline of System configuration model ontology (SCM [PITH_FULL_IMAGE:figures/full_fig_p006_8.png] view at source ↗
read the original abstract

Reproducibility, traceability, and transparency in testing cyber-physical energy systems are crucial for scientific advancement and cross-laboratory collaboration. Current experimentation and test documentation practices lack formal semantics, making it difficult to reproduce experiments, share data, and apply, for example, the artificial intelligence-driven analysis. A dataspace that relies on structured ontologies aims to address these gaps by providing machine-actionable descriptions. In this work, we outline an ontology-driven approach for reproducibility of cyber-physical energy systems testing and illustrate its applicability through representative cross-laboratory use cases, demonstrating feasibility while identifying remaining semantic and metadata gaps that limit reproducibility. Based on these observations, we propose an open three-viewpoint ontology framework to guide future ontology extensions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript claims that current test documentation practices for cyber-physical energy systems lack formal semantics, impeding reproducibility, data sharing, and AI-driven analysis. It outlines an ontology-driven dataspace approach to provide machine-actionable descriptions, illustrates applicability via representative cross-laboratory use cases, identifies remaining semantic and metadata gaps, and proposes an open three-viewpoint ontology framework to guide future extensions.

Significance. If validated, the work could meaningfully advance reproducibility and collaboration in cyber-physical energy systems testing by enabling structured, shareable metadata. The open framework proposal is a constructive element that could foster community-driven ontology development, though its impact depends on concrete implementation and evaluation.

major comments (2)
  1. [Use Cases] Use Cases section: The representative cross-laboratory use cases are presented as descriptive scenarios that identify gaps in current practices but supply no concrete ontology artifacts (e.g., OWL classes, SHACL shapes, or sample RDF triples for test metadata), no before/after reproducibility metrics, and no third-party replication exercise to show that machine-actionable descriptions actually improve reproduction over existing methods.
  2. [Proposed Framework] Proposed Framework section: The three-viewpoint ontology framework is introduced at a conceptual level based on the use-case observations, yet the manuscript provides neither a detailed mapping of the three viewpoints to the specific semantic and metadata gaps identified nor any validation plan or prototype to demonstrate how the framework will close those gaps.
minor comments (2)
  1. [Abstract] The abstract and introduction could more explicitly define the three viewpoints of the proposed framework to help readers understand its structure without needing to infer from later sections.
  2. [Approach] Notation for ontology elements (e.g., classes, properties) is introduced inconsistently; a dedicated table or figure summarizing the core vocabulary would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the scope and presentation of our work. We address each major comment below and indicate the revisions we will make to the manuscript.

read point-by-point responses

  1. Referee: [Use Cases] Use Cases section: The representative cross-laboratory use cases are presented as descriptive scenarios that identify gaps in current practices but supply no concrete ontology artifacts (e.g., OWL classes, SHACL shapes, or sample RDF triples for test metadata), no before/after reproducibility metrics, and no third-party replication exercise to show that machine-actionable descriptions actually improve reproduction over existing methods.

    Authors: We agree that the use cases are presented as descriptive scenarios to identify gaps rather than as a quantitative evaluation with concrete artifacts or replication studies. The manuscript is positioned as an outline of the approach and a proposal for the framework, demonstrating feasibility through cross-laboratory observations without claiming empirical validation of improved reproducibility. In the revised version, we will augment the Use Cases section with illustrative ontology artifacts, such as sample RDF triples for test metadata, to show how machine-actionable descriptions could be structured. We cannot add before/after metrics or third-party replication results, as these were outside the scope of the current work. revision: partial

  2. Referee: [Proposed Framework] Proposed Framework section: The three-viewpoint ontology framework is introduced at a conceptual level based on the use-case observations, yet the manuscript provides neither a detailed mapping of the three viewpoints to the specific semantic and metadata gaps identified nor any validation plan or prototype to demonstrate how the framework will close those gaps.

    Authors: The three-viewpoint framework is intentionally proposed at a conceptual level to serve as an open guide for future extensions, consistent with the manuscript's focus on identifying gaps and outlining a structure rather than delivering a fully implemented ontology. We acknowledge that a detailed mapping to the identified gaps and a validation plan are not provided in the current text. In the revision, we will add an explicit mapping of the three viewpoints to the semantic and metadata gaps from the use cases, along with a high-level validation plan describing steps for prototype development and community evaluation. revision: yes

Circularity Check

0 steps flagged

No circularity: conceptual proposal remains self-contained

full rationale

The paper outlines an ontology-driven dataspace approach, illustrates it via descriptive cross-laboratory use cases, identifies semantic gaps, and proposes an open three-viewpoint framework to guide extensions. No load-bearing step reduces by definition, fitted parameter, or self-citation chain to its own inputs; there are no equations, no parameter fitting presented as prediction, and no uniqueness theorems or ansatzes imported from prior author work that close the argument. The derivation is forward-looking and observational rather than self-referential.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption that structured ontologies can deliver machine-actionable test descriptions sufficient for reproducibility. It introduces one new entity (the three-viewpoint framework) without independent evidence of its effectiveness.

axioms (1)
  • domain assumption Structured ontologies can provide machine-actionable descriptions that address gaps in reproducibility, traceability, and transparency for cyber-physical energy systems testing.
    Invoked as the core mechanism in the abstract to solve current informal documentation practices.
invented entities (1)
  • three-viewpoint ontology framework no independent evidence
    purpose: To guide future ontology extensions for test annotation in energy systems.
    Proposed as an open structure based on observed gaps; no falsifiable independent evidence or validation is described.

pith-pipeline@v0.9.0 · 5440 in / 1299 out tokens · 54978 ms · 2026-05-10T01:45:05.232687+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    Understanding experiments and research practices for reproducibility: an exploratory study,

    S. Samuel and B. K ¨onig-Ries, “Understanding experiments and research practices for reproducibility: an exploratory study,” PeerJ, vol. 9, p. e11140, 2021

    work page 2021

  2. [2]

    The FAIR guiding principles for scientific data management and stewardship,

    M. D. Wilkinson, M. Dumontier, I. J. Aalbersberg, G. Appleton, M. Axton, A. Baak, N. Blomberg, J.-W. Boiten, L. B. da Silva Santos, P. E. Bourne et al. , “The FAIR guiding principles for scientific data management and stewardship,” Scientific data , vol. 3, no. 1, pp. 1–9, 2016

    work page 2016

  3. [4]

    IEC 61970 and 61968 common information model for power systems,

    M. Uslar, M. Specht, S. Rohjans, J. Trefke, and J. Gonz ´alez, “IEC 61970 and 61968 common information model for power systems,” IEEE Transactions on Power Systems , vol. 24, no. 1, pp. 134–142, 2009

    work page 2009

  4. [5]

    SAREF: An IoT ontology to bridge the interoperability gap,

    L. Daniele, F. den Hartog, and J. Roes, “SAREF: An IoT ontology to bridge the interoperability gap,” Semantic Web, vol. 8, no. 5, pp. 721– 752, 2017

    work page 2017

  5. [6]

    SAREF4ENER: An extension of saref for the energy domain,

    ETSI, “SAREF4ENER: An extension of saref for the energy domain,” European Telecommunications Standards Institute, Tech. Rep. ETSI TS 103 410-1, 2018

    work page 2018

  6. [7]

    The smart energy aware systems ontology,

    M. Lefranc ¸ois and A. Zimmermann, “The smart energy aware systems ontology,” Proceedings of the ISWC Workshop on Semantic Sensor Networks, 2015

    work page 2015

  7. [8]

    Brick: Towards a unified metadata schema for buildings,

    B. Balaji, A. Bhattacharya, G. Fierro, J. Gao, J. Gluck, D. Hong, A. Johansen, J. Koh, M. Pritoni, J. Seidman et al. , “Brick: Towards a unified metadata schema for buildings,” in Proceedings of the 3rd ACM International Conference on Systems for Energy-Efficient Built Environments, 2016, pp. 41–50

    work page 2016

  8. [9]

    QUDT: Quantities, units, dimensions and data types ontologies,

    R. Hodgson, P. Keller, J. Hodges, and J. Spivak, “QUDT: Quantities, units, dimensions and data types ontologies,” Journal of Web Semantics, vol. 21, pp. 1–13, 2014

    work page 2014

  9. [10]

    Graph-based impact analysis of cyber-attacks on behind-the-meter infrastructure,

    I. Hacker, ¨O. Sen, F. Klein-Helmkamp, and A. Ulbig, “Graph-based impact analysis of cyber-attacks on behind-the-meter infrastructure,” in 14th Mediterranean Conference on Power Generation Transmission, Distribution and Energy Conversion (MEDPOWER 2024) , vol. 2024. IET, 2024, pp. 529–534

    work page 2024

  10. [11]

    A model-based approach for converting CGMES power system models into operational networks,

    A. Memari and A. Aljamous, “A model-based approach for converting CGMES power system models into operational networks,” Energy Informatics, vol. 6, no. Suppl 1, p. 26, 2023

    work page 2023

  11. [12]

    A consistent view of the smart grid: bridging the gap between IEC CIM and IEC 61850,

    A. Schumilin, C. Duepmeier, K.-U. Stucky, and V . Hagenmeyer, “A consistent view of the smart grid: bridging the gap between IEC CIM and IEC 61850,” in 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 2018, pp. 321– 325

    work page 2018

  12. [13]

    OntoPowSys: A power system ontology for cross domain interactions in an eco industrial park,

    A. Devanand, G. Karmakar, N. Krdzavac, R. Rigo-Mariani, Y . F. Eddy, I. A. Karimi, and M. Kraft, “OntoPowSys: A power system ontology for cross domain interactions in an eco industrial park,” Energy and AI, vol. 1, p. 100008, 2020

    work page 2020

  13. [14]

    Developing interoperable ontologies for research data management in the energy domain,

    A. Wein, A. Steinert, J. S. Schwarz, L. Fuentes-Grau, Z. Pan, A. Monti, and A. Nieße, “Developing interoperable ontologies for research data management in the energy domain,” 2024

    work page 2024

  14. [15]

    Introducing the Open Energy Ontology: Enhancing data interpretation and interfac- ing in energy systems analysis,

    M. Booshehri, L. Emele, S. Fl ¨ugel, H. F ¨orster, J. Frey, U. Frey, M. Glauer, J. Hastings, C. Hofmann, C. Hoyer-Klick et al., “Introducing the Open Energy Ontology: Enhancing data interpretation and interfac- ing in energy systems analysis,” Energy and AI, vol. 5, p. 100074, 2021

    work page 2021

  15. [16]

    Ontologies for describing the con- text of scientific experiment processes,

    R. Mayer, T. Miksa, and A. Rauber, “Ontologies for describing the con- text of scientific experiment processes,” in2014 IEEE 10th International Conference on e-Science , vol. 1. IEEE, 2014, pp. 153–160

    work page 2014

  16. [17]

    Research data infrastructures and engineering metadata,

    M. T. Horsch, S. Chiacchiera, W. L. Cavalcanti, and B. Schembera, “Research data infrastructures and engineering metadata,” in Data Technology in Materials Modelling . Springer, 2021, pp. 13–30

    work page 2021

  17. [18]

    Towards the integration of data space technology in hydrogen research workflows,

    R. Ford, R. Beigait ˙e, and L. Daniel, “Towards the integration of data space technology in hydrogen research workflows,” in Nordic Energy Informatics Academy Conference . Springer, 2025, pp. 36–50

    work page 2025

  18. [19]

    An ontology of scientific experiments,

    L. N. Soldatova and R. D. King, “An ontology of scientific experiments,” Journal of the royal society interface, vol. 3, no. 11, pp. 795–803, 2006

    work page 2006

  19. [20]

    An ap- proach to experiment reproducibility through mlops and semantic web technologies,

    D. Seaman, D. Pe ˜nafiel, K. Palacio-Baus, and V . Saquicela, “An ap- proach to experiment reproducibility through mlops and semantic web technologies,” in 2023 XLIX Latin American Computer Conference (CLEI), 2023, pp. 1–5

    work page 2023

  20. [21]

    Deliverable D5.1 – D-NA5. 1 Smart Grid configuration validation scenario description method,

    K. Heussen, D. E. M. Bondy, M. Blank, T. Klingenberg, A. Kulmala, I. F. Abdulhadi, D. Pala, M. Rossi, C. Carlini, A. van der Meer et al., “Deliverable D5.1 – D-NA5. 1 Smart Grid configuration validation scenario description method,” 2017

    work page 2017

  21. [22]

    Erigrid holistic test description for validating cyber-physical energy systems,

    K. Heussen, C. Steinbrink, I. F. Abdulhadi, V . H. Nguyen, M. Z. Degefa, J. Merino, T. V . Jensen, H. Guo, O. Gehrke, D. E. M. Bondy, D. Babazadeh, F. Pr ¨ostl Andr ´en, and T. I. Strasser, “Erigrid holistic test description for validating cyber-physical energy systems,” Energies, vol. 12, no. 14, 2019. [Online]. Available: https://www.mdpi.com/1996-1073/...

    work page 2019

  22. [23]

    Description of optimization strategies: An overview of the PreCISE approach used for the definition of simulation and optimization workflows,

    E. Widl, “Description of optimization strategies: An overview of the PreCISE approach used for the definition of simulation and optimization workflows,” AIT Austrian Institute of Technology, ECRIA SmILES Project Deliverable D4.3, Apr. 2019. [Online]. Available: https://cordis.europa.eu/project/id/730936/results

    work page 2019

  23. [24]

    Uncertainty annotations for holis- tic test description of cyber-physical energy systems,

    K. Heussen, J. S. Schwarz, E. Schulte, Z. Feng, L. E. R. Perez, J. Nikoletatos, and F. P. Andren, “Uncertainty annotations for holis- tic test description of cyber-physical energy systems,” arXiv preprint arXiv:2506.23436, 2025

    work page arXiv 2025

  24. [25]

    A toolbox for design of experiments for energy systems in co- simulation and hardware tests,

    J. S. Schwarz, L. E. R. Perez, M. C. Pham, K. Heussen, and Q. T. Tran, “A toolbox for design of experiments for energy systems in co- simulation and hardware tests,” in 2024 Open Source Modelling and Simulation of Energy Systems (OSMSES) , 2024, pp. 1–7

    work page 2024

  25. [26]

    Design of experiments aided holistic testing of cyber-physical energy systems,

    A. A. van der Meer, C. Steinbrink, K. Heussen, D. E. M. Bondy, M. Z. Degefa, F. P. Andr ´en, T. I. Strasser, S. Lehnhoff, and P. Palensky, “Design of experiments aided holistic testing of cyber-physical energy systems,” in 2018 Workshop on Modeling and Simulation of Cyber- Physical Energy Systems (MSCPES) . IEEE, 2018, pp. 1–7

    work page 2018

  26. [27]

    Toward principles for the design of ontologies used for knowledge sharing,

    T. R. Gruber, “Toward principles for the design of ontologies used for knowledge sharing,” International Journal of Human-Computer Studies, vol. 43, no. 5–6, pp. 907–928, 1995

    work page 1995

  27. [28]

    Staab and R

    S. Staab and R. Studer, Handbook on Ontologies , 2nd ed. Springer, 2009

    work page 2009

  28. [29]

    Hitzler, M

    P. Hitzler, M. Kr ¨otzsch, B. Parsia, P. F. Patel-Schneider, and S. Rudolph, OWL 2 Web Ontology Language Primer . W3C Recommendation,

    work page

  29. [30]

    Available: https://www.w3.org/TR/owl2-primer/

    [Online]. Available: https://www.w3.org/TR/owl2-primer/

    work page

  30. [31]

    A review of semantic technolo- gies for energy management,

    F. Bergenti, S. Monica, and A. Poggi, “A review of semantic technolo- gies for energy management,” Applied Energy, vol. 211, pp. 124–146, 2018

    work page 2018

  31. [32]

    The PROV family of documents,

    L. Moreau and P. Missier, “The PROV family of documents,” Future Generation Computer Systems , vol. 52, pp. 1–2, 2015

    work page 2015

  32. [33]

    PROV-DM: The PROV data model,

    L. Moreau, K. Belhajjame, J. Cheney, S. Coppens, S. Cresswell et al., “PROV-DM: The PROV data model,” W3C, W3C Recommendation,

    work page

  33. [34]

    Available: https://www.w3.org/TR/prov-dm/

    [Online]. Available: https://www.w3.org/TR/prov-dm/

    work page

  34. [35]

    Provenance and data differencing for workflow re- producibility analysis,

    P. Missier, S. Dey, K. Belhajjame, V . Cuevas-Vicentt ´ın, and B. Lud ¨ascher, “Provenance and data differencing for workflow re- producibility analysis,” Concurrency and Computation: Practice and Experience, vol. 27, no. 16, pp. 1–19, 2013

    work page 2013

  35. [36]

    An ontology of workflow provenance,

    D. Garijo and Y . Gil, “An ontology of workflow provenance,” Journal of Web Semantics, vol. 25, pp. 1–23, 2014

    work page 2014

  36. [37]

    OPMW: Ontology for describing workflows and metadata based on the open provenance model,

    Y . Gil, V . Ratnakar, and D. Garijo, “OPMW: Ontology for describing workflows and metadata based on the open provenance model,” W3C Provenance Working Group Community, Tech. Rep., 2013. [Online]. Available: http://www.opmw.org/

    work page 2013

  37. [38]

    Accompanying dataset for paper

    J. Kazmi, N. Mehran, T. O’Donnell, A. Obusevs, and K. Heussen, “Accompanying dataset for paper ”towards reproducible test annotation for cyber-physical energy systems using ontology-driven dataspaces” (osmses 2026),” Mar. 2026. [Online]. Available: https://doi.org/10.5281/ zenodo.18868542

    work page 2026

  38. [39]

    O’Donnell, A

    T. O’Donnell, A. Malik, A. Obusevs, P. Schmitz, and N. Ninad, “Automated testing of distributed energy resources (DER): Use of an open source testing platform (OpenSVP) for testing smart inverters against European network connection standards EN50549 and VDE 4105,” in Research Infrastructure Automation Workshop . Roskilde, Denmark: Zenodo, September 24–25...

    work page doi:10.5281/zenodo.14699238 2024

  39. [40]

    International development of a distributed energy resource test platform for electrical and interoperability certification,

    J. Johnson, E. Apablaza-Arancibia, N. Ninad, D. Turcotte, A. Prieur, R. Ablinger, R. Br ¨ındlinger, T. Moore, R. Heidari, J. Hashimoto et al., “International development of a distributed energy resource test platform for electrical and interoperability certification,” in 2018 IEEE 7th World Conference on Photovoltaic Energy Conversion (WCPEC)(A Joint Con-...

    work page 2018

  40. [41]

    The prot ´eg´e project: a look back and a look forward,

    M. A. Musen, “The prot ´eg´e project: a look back and a look forward,” AI Matters, vol. 1, no. 4, pp. 4—-12, Jun. 2015

    work page 2015