Demonstrators for Industrial Cyber-Physical System Research: A Requirements Hierarchy Driven by Software-Intensive Design

Richard Loendersloot; Tiedo Tinga; Uraz Odyurt

arxiv: 2510.18534 · v3 · pith:6YFRHXRWnew · submitted 2025-10-21 · 💻 cs.SE

Demonstrators for Industrial Cyber-Physical System Research: A Requirements Hierarchy Driven by Software-Intensive Design

Uraz Odyurt , Richard Loendersloot , Tiedo Tinga This is my paper

Pith reviewed 2026-05-25 08:02 UTC · model grok-4.3

classification 💻 cs.SE

keywords demonstratorsrequirements engineeringcyber-physical systemsresearch project managementhierarchical levelsTRL scaleindustrial use-cases

0 comments

The pith

A five-level hierarchy of demonstration requirements reduces mismatches between targets and results in industrial cyber-physical systems projects.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper identifies the common practice of treating demonstrators as an afterthought in research proposals, which creates ongoing confusion and gaps between what projects aim to show and what they can actually deliver. It introduces a requirements elaboration framework built around five hierarchical demonstration levels that tie directly to work package interactions and specific industrial use-cases. The framework supports feasibility checks and realistic adjustments during planning, with the scope limited to software-intensive industrial cyber-physical systems. Application to two portfolio projects at different stages showed the approach can clarify expectations where the TRL scale alone falls short.

Core claim

The authors define five hierarchical levels of demonstration within a requirements elaboration framework. These levels link demonstration coverage to work package expectations and the project's industrial use-cases, enabling evaluation of targeted demonstration feasibility and adjustments to requirements. The framework addresses uncertainties that arise when demonstration details are left vague in project organization, particularly for software-intensive systems and industrial cyber-physical systems.

What carries the argument

The five-level demonstrator hierarchy inside the requirements elaboration framework, which connects demonstration expectations to work packages and industrial use-cases.

Load-bearing premise

That applying the five-level hierarchy to project demonstrators will reduce confusion and mismatches between targeted and achievable results.

What would settle it

Track multiple projects that apply the framework from the proposal stage onward and measure whether mismatches between planned and delivered demonstrations decrease compared to projects that do not use it.

Figures

Figures reproduced from arXiv: 2510.18534 by Richard Loendersloot, Tiedo Tinga, Uraz Odyurt.

**Figure 1.** Figure 1: WP dependency diagram for the ZORRO project, covering direct dependencies with solid and uncertain/undefined demonstration WP input with dashed arrows. similar to TRLs. Clear interfaces between WPs, well-defined data exchange protocols, and shared understanding of demonstration scope(s) are essential for smooth cross-WP collaboration. 3 DEMONSTRATOR TAXONOMY We define a demonstrator taxonomy for software-… view at source ↗

**Figure 2.** Figure 2: Visualising the coverage scope per different demon [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: Visualising the project-wide view, covering all in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: The demonstrator requirements elaboration frame [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

**Figure 5.** Figure 5: WP dependency diagram for the PrimaVera project, [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

read the original abstract

One of the challenges apparent in the organisation of research projects is the uncertainties around the subject of demonstrators. A precise and detailed elicitation of the coverage for project demonstrators is often an afterthought and not sufficiently detailed during proposal writing. This practice leads to continuous confusion and a mismatch between targeted and achievable demonstration of results, hindering progress. The reliance on the TRL scale as a loose descriptor does not help either. We propose a demonstrator requirements elaboration framework aiming to evaluate the feasibility of targeted demonstrations, making realistic adjustments, and assist in describing requirements. In doing so, we define 5 hierarchical levels of demonstration, clearly connected to expectations, e.g., work package interaction, and also connected to the project's industrial use-cases. The considered application scope in this paper is the domain of software-intensive systems and industrial cyber-physical systems. A complete validation is not accessible, as it would require application of our framework at the start of a project and observing the results at the end, taking 4-5 years. Nonetheless, we have applied it to two research projects from our portfolio, one at the early and another at the final stages, revealing its effectiveness.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper defines a five-level demonstrator hierarchy for CPS but backs its value with only two retrospective internal applications lacking metrics.

read the letter

The paper's main contribution is a five-level hierarchy for demonstrator requirements in software-intensive industrial cyber-physical systems. It aims to make demonstration planning more precise than TRL labels allow and to reduce mismatches between targeted and achievable results by linking levels to work packages and use cases. They do a decent job laying out the problem in project organization and then presenting the hierarchy as a practical tool. The connections to industrial use cases and project elements give it some grounding in real workflows. Applying it to two projects from their portfolio shows they have tried it out. The weakness is in how they support the effectiveness claim. The applications are retrospective, one early and one late stage, and they say it revealed effectiveness without providing any specific indicators of change or improvement. They are upfront that complete validation would take years of forward application, so the current evidence is limited to their own assessment. This leaves the central promise—that the hierarchy helps with feasibility evaluation and realistic adjustments—without strong backing data. The hierarchy itself might still be useful as a template, but readers will have to judge that for themselves. Readers who manage or participate in CPS research projects would get the most from this. It could serve as a reference for structuring demonstrator goals in similar applied settings. The work shows clear thinking about a practical issue and honest discussion of validation limits. It deserves peer review so that reviewers can examine the level definitions in detail and suggest ways to strengthen the evidence. Recommendation: Send it to referees rather than desk reject.

Referee Report

2 major / 1 minor

Summary. The paper proposes a demonstrator requirements elaboration framework for software-intensive industrial cyber-physical systems research. It defines a 5-level hierarchy of demonstrations explicitly linked to work-package expectations and industrial use-cases, intended to evaluate feasibility of targeted demonstrations, enable realistic adjustments during proposal writing, and reduce mismatches between targeted and achievable results. The TRL scale is critiqued as insufficiently precise. The framework is applied retrospectively to two portfolio projects (one early-stage, one final-stage), with the authors stating that this application reveals its effectiveness, while acknowledging that full validation would require multi-year observation from project inception.

Significance. If the hierarchy demonstrably improves demonstrator planning, the work could offer a practical tool for reducing common coordination failures in CPS research projects. The explicit linkage to work packages and use-cases, together with the acknowledgment of validation constraints, provides a transparent basis for further testing; however, the current evidence base does not yet establish measurable gains over existing practices.

major comments (2)

[Abstract / application to projects] Abstract and application section: the central claim that the five-level hierarchy reduces confusion and mismatch is supported only by the statement that retrospective application to two internal projects 'revealing its effectiveness.' No quantitative indicators (e.g., counts of requirement mismatches before vs. after, stakeholder agreement scores, or scope-change metrics) or comparison against non-framework projects are supplied, which directly undercuts the value proposition given that the framework's utility is defined in terms of producing such improvements.
[Framework definition] Framework definition section: while the five levels are asserted to be 'clearly connected to expectations, e.g., work package interaction, and also connected to the project's industrial use-cases,' the manuscript provides neither the explicit level definitions nor example mappings in sufficient detail for independent assessment or replication; this absence makes it impossible to verify the claimed improvement over the TRL scale.

minor comments (1)

[Abstract] The abstract notes that 'a complete validation is not accessible' yet still concludes effectiveness from the two applications; a brief explicit statement of the precise limitations this imposes on the reported results would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. We address each major comment below, indicating planned revisions where appropriate. Our responses focus on clarifying the manuscript's contributions while acknowledging its evidential limitations.

read point-by-point responses

Referee: [Abstract / application to projects] Abstract and application section: the central claim that the five-level hierarchy reduces confusion and mismatch is supported only by the statement that retrospective application to two internal projects 'revealing its effectiveness.' No quantitative indicators (e.g., counts of requirement mismatches before vs. after, stakeholder agreement scores, or scope-change metrics) or comparison against non-framework projects are supplied, which directly undercuts the value proposition given that the framework's utility is defined in terms of producing such improvements.

Authors: We agree that the current evidence is qualitative and retrospective, based on post-hoc application to two portfolio projects (one early-stage, one final-stage). A prospective study with quantitative metrics or controlled comparisons would require multi-year observation from project inception, which is outside the scope of this work as already noted in the manuscript. In revision we will expand the application section with concrete examples of mismatches identified and adjustments made in each project, including explicit before/after requirement descriptions. This will strengthen the illustration without claiming quantitative proof. revision: partial
Referee: [Framework definition] Framework definition section: while the five levels are asserted to be 'clearly connected to expectations, e.g., work package interaction, and also connected to the project's industrial use-cases,' the manuscript provides neither the explicit level definitions nor example mappings in sufficient detail for independent assessment or replication; this absence makes it impossible to verify the claimed improvement over the TRL scale.

Authors: We accept this point. The manuscript introduces the five hierarchical levels and their linkage to work packages and use-cases but does not present them with the granularity needed for replication. In the revised version we will add a dedicated table or subsection with precise definitions for each level, together with concrete mapping examples drawn from the two projects, to enable independent evaluation and comparison with the TRL scale. revision: yes

Circularity Check

0 steps flagged

No circularity; 5-level hierarchy defined from project challenges, not reduced to self-application or citations

full rationale

The paper identifies a challenge (mismatch in demonstrator expectations) and proposes a 5-level requirements hierarchy as a response, with retrospective application to two portfolio projects offered only as illustrative support. No equations, fitted parameters, self-citations, or uniqueness theorems are present. The central definition of the levels is not constructed from the validation cases by definition, nor does any step reduce to prior author work that itself assumes the result. This meets the default expectation of a non-circular proposal resting on observed issues.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no information on free parameters, axioms, or invented entities; the framework appears to introduce the five levels as a new structuring device without additional postulated entities.

pith-pipeline@v0.9.0 · 5742 in / 1082 out tokens · 26785 ms · 2026-05-25T08:02:30.013455+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Implementing CPSLint: A Data Validation and Sanitisation Tool for Industrial Cyber-Physical Systems
cs.PL 2026-04 unverdicted novelty 4.0

CPSLint is a publicly available DSL that lets users express data sanitization for CPS time-series collections in a few lines of code, improving readability and maintainability over custom scripts.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages · cited by 1 Pith paper

[1]

Engineering of Software-Intensive Systems: State of the Art and Research Challenges

2008. Engineering of Software-Intensive Systems: State of the Art and Research Challenges. Software-Intensive Systems and New Computing Paradigms: Chal- lenges and Visions. doi: 10.1007/978-3-540-89437-7_1

work page doi:10.1007/978-3-540-89437-7_1 2008
[2]

Olumuyiwa Ibidunmoye, Francisco Hernández-Rodriguez, and Erik Elmroth

work page
[3]

ACM Com- put

Performance anomaly detection and bottleneck identification. ACM Com- put. Surv. doi: 10.1145/2791120

work page doi:10.1145/2791120
[4]

John C. Mankins. 2004. Technology readiness levels: a white paper. Originally published in 1995, edited on 22 December 2004. (2004)

work page 2004
[5]

NWO – Nederlandse Organisatie voor Wetenschappelijk Onderzoek. 2023. ZORRO: Engineering for Zero Downtime in Cyber-Physical Systems via In- telligent Diagnostics. Project start date: September 1, 2023. NWO. Retrieved Sept. 8, 2025 from https://www.nwo.nl/en/projects/kich1st0221003

work page 2023
[6]

NWO – Nederlandse Organisatie voor Wetenschappelijk Onderzoek. 2019. Pri- maVera: Predictive maintenance for Very effective asset management. Project start date: November 1, 2019. NWO. Retrieved Sept. 8, 2025 from https://www .nwo.nl/en/projects/nwa116018238

work page 2019
[7]

Kimmel, Patricia M

William M. Kimmel, Patricia M. Beauchamp, Margaret A. Frerking, Teresa R. Kline, Kenny Koorosh Vassigh, Douglas E. Willard, Michael A. Johnson, and Timothy G. Trenkle. 2020. Technology Readiness Assessment Best Practices Guide. Tech. rep. Acquired June 17, 2020. NASA. https://ntrs.nasa.gov/citation s/20205003605

work page arXiv 2020
[8]

Ulli Vilsmaier, Moritz Engbers, Philip Luthardt, Rina Marie Maas-Deipenbrock, Sebastian Wunderlich, and Roland W. Scholz. 2015. Case-based Mutual Learning Sessions: knowledge integration and transfer in transdisciplinary processes. Sustainability Science. doi: 10.1007/s11625-015-0335-3

work page doi:10.1007/s11625-015-0335-3 2015
[9]

Berenike Feldhoff, Nils Stockmann, Nora Fanderl, Anne-Kathrin Gahle, Anto- nia Graf, Matthias Leger, and Marco Sonnberger. 2019. Bridging Theories and Practices: Boundary Objects and Constellation Analysis as Vehicles for Inter- disciplinary Knowledge Integration. Sustainability. doi: 10.3390/su11195357

work page doi:10.3390/su11195357 2019
[10]

Dominique Vinck. 2003. Everyday Engineering: An Ethnography of Design and Innovation. The MIT Press. doi: 10.7551/mitpress/2862.001.0001

work page doi:10.7551/mitpress/2862.001.0001 2003
[11]

Dominique Vinck. 2012. Accessing Material Culture by Following Intermediary Objects. In An Ethnography of Global Landscapes and Corridors . doi: 10.5772/34 719

work page doi:10.5772/34 2012
[12]

James Moultrie. 2015. Understanding and classifying the role of design demon- strators in scientific exploration. Technovation. doi: 10.1016/j.technovation.201 5.05.002

work page doi:10.1016/j.technovation.201 2015
[13]

Gerda Gemser and Mark A.A.M Leenders. 2001. How integrating industrial design in the product development process impacts on company performance. Journal of Product Innovation Management . doi: 10.1016/S0737-6782(00)00069- 2

work page doi:10.1016/s0737-6782(00)00069- 2001
[14]

Hertenstein, Marjorie B

Julie H. Hertenstein, Marjorie B. Platt, and David R. Brown. 2001. Valuing design: Enhancing corporate performance through design effectiveness.Design Management Journal (Former Series) . doi: 10.1111/j.1948-7169.2001.tb00548.x

work page doi:10.1111/j.1948-7169.2001.tb00548.x 2001
[15]

Varun Gopinath, Kerstin Johansen, and Micael Derelöv. 2018. Demonstrators to support research in Industrial safety – A Methodology.Procedia Manufacturing. doi: 10.1016/j.promfg.2018.10.043

work page doi:10.1016/j.promfg.2018.10.043 2018
[16]

Tina Bobbe, Lenard Opeskin, Lisa-Marie Lüneburg, Helge Wanta, Joshwa Pohlmann, and Jens Krzywinski. 2023. Design for communication: how do demonstrators demonstrate technology? Design Science. doi: 10.1017/dsj.2023.1

work page doi:10.1017/dsj.2023.1 2023

[1] [1]

Engineering of Software-Intensive Systems: State of the Art and Research Challenges

2008. Engineering of Software-Intensive Systems: State of the Art and Research Challenges. Software-Intensive Systems and New Computing Paradigms: Chal- lenges and Visions. doi: 10.1007/978-3-540-89437-7_1

work page doi:10.1007/978-3-540-89437-7_1 2008

[2] [2]

Olumuyiwa Ibidunmoye, Francisco Hernández-Rodriguez, and Erik Elmroth

work page

[3] [3]

ACM Com- put

Performance anomaly detection and bottleneck identification. ACM Com- put. Surv. doi: 10.1145/2791120

work page doi:10.1145/2791120

[4] [4]

John C. Mankins. 2004. Technology readiness levels: a white paper. Originally published in 1995, edited on 22 December 2004. (2004)

work page 2004

[5] [5]

NWO – Nederlandse Organisatie voor Wetenschappelijk Onderzoek. 2023. ZORRO: Engineering for Zero Downtime in Cyber-Physical Systems via In- telligent Diagnostics. Project start date: September 1, 2023. NWO. Retrieved Sept. 8, 2025 from https://www.nwo.nl/en/projects/kich1st0221003

work page 2023

[6] [6]

NWO – Nederlandse Organisatie voor Wetenschappelijk Onderzoek. 2019. Pri- maVera: Predictive maintenance for Very effective asset management. Project start date: November 1, 2019. NWO. Retrieved Sept. 8, 2025 from https://www .nwo.nl/en/projects/nwa116018238

work page 2019

[7] [7]

Kimmel, Patricia M

William M. Kimmel, Patricia M. Beauchamp, Margaret A. Frerking, Teresa R. Kline, Kenny Koorosh Vassigh, Douglas E. Willard, Michael A. Johnson, and Timothy G. Trenkle. 2020. Technology Readiness Assessment Best Practices Guide. Tech. rep. Acquired June 17, 2020. NASA. https://ntrs.nasa.gov/citation s/20205003605

work page arXiv 2020

[8] [8]

Ulli Vilsmaier, Moritz Engbers, Philip Luthardt, Rina Marie Maas-Deipenbrock, Sebastian Wunderlich, and Roland W. Scholz. 2015. Case-based Mutual Learning Sessions: knowledge integration and transfer in transdisciplinary processes. Sustainability Science. doi: 10.1007/s11625-015-0335-3

work page doi:10.1007/s11625-015-0335-3 2015

[9] [9]

Berenike Feldhoff, Nils Stockmann, Nora Fanderl, Anne-Kathrin Gahle, Anto- nia Graf, Matthias Leger, and Marco Sonnberger. 2019. Bridging Theories and Practices: Boundary Objects and Constellation Analysis as Vehicles for Inter- disciplinary Knowledge Integration. Sustainability. doi: 10.3390/su11195357

work page doi:10.3390/su11195357 2019

[10] [10]

Dominique Vinck. 2003. Everyday Engineering: An Ethnography of Design and Innovation. The MIT Press. doi: 10.7551/mitpress/2862.001.0001

work page doi:10.7551/mitpress/2862.001.0001 2003

[11] [11]

Dominique Vinck. 2012. Accessing Material Culture by Following Intermediary Objects. In An Ethnography of Global Landscapes and Corridors . doi: 10.5772/34 719

work page doi:10.5772/34 2012

[12] [12]

James Moultrie. 2015. Understanding and classifying the role of design demon- strators in scientific exploration. Technovation. doi: 10.1016/j.technovation.201 5.05.002

work page doi:10.1016/j.technovation.201 2015

[13] [13]

Gerda Gemser and Mark A.A.M Leenders. 2001. How integrating industrial design in the product development process impacts on company performance. Journal of Product Innovation Management . doi: 10.1016/S0737-6782(00)00069- 2

work page doi:10.1016/s0737-6782(00)00069- 2001

[14] [14]

Hertenstein, Marjorie B

Julie H. Hertenstein, Marjorie B. Platt, and David R. Brown. 2001. Valuing design: Enhancing corporate performance through design effectiveness.Design Management Journal (Former Series) . doi: 10.1111/j.1948-7169.2001.tb00548.x

work page doi:10.1111/j.1948-7169.2001.tb00548.x 2001

[15] [15]

Varun Gopinath, Kerstin Johansen, and Micael Derelöv. 2018. Demonstrators to support research in Industrial safety – A Methodology.Procedia Manufacturing. doi: 10.1016/j.promfg.2018.10.043

work page doi:10.1016/j.promfg.2018.10.043 2018

[16] [16]

Tina Bobbe, Lenard Opeskin, Lisa-Marie Lüneburg, Helge Wanta, Joshwa Pohlmann, and Jens Krzywinski. 2023. Design for communication: how do demonstrators demonstrate technology? Design Science. doi: 10.1017/dsj.2023.1

work page doi:10.1017/dsj.2023.1 2023