Towards a Framework for Operationalizing the Specification of Trustworthy AI Requirements

Daniel Mendez; Hugo Villamizar; Marcos Kalinowski

REVIEW 2 major objections 5 minor 16 references

Towards a Framework for Operationalizing the Specification of Trustworthy AI Requirements

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This vision paper claims that integrating AMDiRE's artefact-based structure with PerSpecML's concern-driven guidance gives a concrete pathway from high-level trustworthy AI principles to structured, traceable requirement artefacts.

desk verdict An honest, well-scoped vision paper: the AMDiRE-PerSpecML pairing is a plausible research direction, and the paper's main weakness is that its only illustrative mapping is both hand-crafted and contains a malformed example. read the letter →

arxiv 2507.10228 v1 pith:CM4ZMHGL submitted 2025-07-14 cs.SE

Hugo Villamizar , Daniel Mendez , Marcos Kalinowski This is my paper · ORCID

classification cs.SE

keywords trustworthyAIrequirementsengineeringAI-enabledsystemsartefact-basedperspective-basedelicitationmachinelearningEUActtraceability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is a vision proposal, not a finished method. It claims that two existing requirements engineering approaches can be combined to make trustworthy AI goals operational: AMDiRE supplies structured artefacts with defined content, roles, and milestones, while PerSpecML supplies a catalogue of 60 concerns across 28 ML tasks from five perspectives. The combination would let a team start from a high-level commitment like 'the system must be compliant, ethical, robust' and end with documented requirement artefacts that trace back to stakeholder concerns and regulatory sources. The authors illustrate the idea on an AI hiring platform, mapping concerns such as fairness, explainability, and regulatory compliance to AMDiRE artefacts. If the pathway works, it would give practitioners concrete guidance rather than principles alone, and would make AI trustworthiness auditable during development.

What carries the argument

The load-bearing mechanism is the Perspective-based ML Task and Concern Diagram, PerSpecML's central object that links 60 concerns to 28 ML tasks across five perspectives (system objectives, user experience, infrastructure, model, data). The paper proposes to extend that diagram with regulatory requirements and then attach each concern to one of AMDiRE's artefact content structures — context specification, requirements specification, system specification — plus the role model that assigns validation duties. That attachment is what turns an abstract concern such as explainability into a documented, traceable requirement entry. The illustrative mapping tables in the paper show the intended shape of the mechanism, not its validated behaviour.

What would settle it

A concrete test would be to take all 60 PerSpecML concerns plus the EU AI Act's seven AI-HLEG requirements and attempt to assign each to an AMDiRE artefact type with a well-formed example requirement. If even one concern admits no artefact type, or if the resulting example entries are malformed or lose traceability to the original stakeholder concern, the central bridging claim would fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that the gap between high-level trustworthiness principles and concrete RE artefacts can be closed by a layered integration. PerSpecML's perspective-based task and concern diagram is the elicitation front end; AMDiRE's artefact, role, and process models are the documentation back end; regulatory frameworks such as the EU AI Act enter as first-class stakeholders whose constraints extend the concern catalogue. The discovery is the bridging move itself: each trustworthiness concern is assigned to an artefact type — context, requirements, or system specification — whose predefined content structure then guides the writing of the requirement. The authors present the mapping as a research direction, scaffolded by an illustrative example rather than a validated method.

Load-bearing premise

The load-bearing premise is that PerSpecML's concern catalogue can be extended with regulatory requirements and mapped cleanly onto AMDiRE's artefact content structures at compatible granularity; the paper does not yet show that such a mapping can be systematic rather than hand-crafted.

Editorial extensions

If this is right

Practitioners would get a defined route from a trustworthiness principle to a documentable requirement artefact, with roles and approval milestones supplied by AMDiRE's process and role models.
Regulatory constraints could be treated as elicited concerns from first-class stakeholders, so compliance evidence would be traceable from EU AI Act clauses to system specifications.
The integration would let teams reason about trade-offs among concerns, such as explainability versus model complexity, during elicitation rather than after implementation.
Tool support could guide concern navigation and artefact generation by extending the existing AMDiRE tool environment.
If validated in practice, the framework would complement existing responsible-AI frameworks by adding a concrete mechanism for specifying trustworthiness requirements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Modeling regulators as first-class stakeholders could generalize to other normative sources, such as safety standards or data-protection authorities, making the pathway a generic compliance-to-specification channel.
A natural next test is to compare teams using the integrated templates against teams using free-form checklists for the same regulation, measuring completeness, consistency, and traceability of trustworthiness requirements.
If the concern-to-artefact mapping is made explicit and tool-supported, the framework could also support automated consistency checks between regulatory clauses and system specifications.
The authors scope the vision to machine learning; the same structure could extend to generative AI by adding new tasks and concerns for emergent behaviours such as hallucination or misuse.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a short vision paper, and it is honestly labeled as one. The useful idea is pairing AMDiRE's artefact-centric structure with PerSpecML's concern catalog to give requirements engineers a traceable route from trustworthiness principles to concrete specifications. That is a plausible and bounded contribution to the RE subfield, and the paper does not overclaim: it explicitly says validation is future work and calls for practical evaluation. What the paper does well is positioning. The background on trustworthy AI is compact, the related work is relevant, and the authors correctly identify that neither source method alone covers the gap. The illustrative hiring-platform example, despite being hand-crafted, at least makes the vision concrete. The self-citation is transparent and not by itself a problem, since both methods are published and the authors know them deeply. The soft spots are real but not disqualifying. The load-bearing assumption is that PerSpecML's concerns can be systematically mapped onto AMDiRE's artefact content structures, preserving granularity and covering non-functional concerns like fairness or human oversight. The paper does not analyze this compatibility; it only asserts the mapping in Table II. That table also contains a genuinely sloppy entry: 'The system architecture shall performance under input perturbations' is not a well-formed requirement. That is a credibility ding. But the paper itself lists 'bridging to artefact models' as a future step and concludes that meaningful results require empirical evaluation, so the stress-test's concern identifies the key condition rather than a contradiction internal to the paper. Who is this for? Researchers already working with AMDiRE or PerSpecML, and RE folks thinking about how to operationalize the EU AI Act. It is workshop-level material, not a complete method. As a referee, I would send it out, not desk-reject it, because the research direction is timely and the authors are the right people to pursue it. The main revision requests would be: fix the malformed example, and say something substantive about how the two metamodels meet, even if it is only a sketch of possible mapping rules.

Referee Report

2 major / 5 minor

Summary. This short vision paper proposes integrating two existing requirements engineering approaches: AMDiRE, an artefact-based RE method, and PerSpecML, a perspective-based method for ML-enabled systems. The motivation is that AMDiRE offers structured, traceable artefacts but was designed for deterministic systems, while PerSpecML provides stakeholder-driven concern elicitation for ML contexts including trustworthiness-related concerns. The authors envision a framework in which PerSpecML's concern catalog is extended with regulatory requirements (e.g., the EU AI Act), mapped onto AMDiRE's artefact model, and supported by a tool. The paper illustrates the idea with a hiring-platform example (Tables I and II), reviews related work, and concludes that practical evaluation in industry collaborations is required for the vision to produce meaningful results.

Significance. If the envisioned integration is realized, it would address a genuine gap: translating high-level trustworthiness principles into concrete, traceable RE artefacts. The paper builds transparently on the authors' prior methods (AMDiRE and PerSpecML), which is appropriate for a vision paper, and it explicitly acknowledges that the mappings require validation. The clearest strength is the clear articulation of complementary strengths and the structured set of research directions. The chief weakness is that the only concrete illustration of the central bridging mechanism is a hand-crafted table with a malformed entry, so the reader cannot yet assess whether the mapping is feasible at the required granularity.

major comments (2)

[Section III-C, Table II] The only concrete illustration of the concern-to-artefact mapping contains a malformed requirement entry: 'The system architecture shall performance under input perturbations' is missing the main verb and is not a well-formed requirement. This is a load-bearing defect because Table II is the paper's sole evidence that the envisioned operationalization can work. The authors must correct the example and ideally explain which specific content elements of the target artefact (e.g., the quality model of the System Specification) receive the concern-related information. Without a coherent example, the feasibility of the bridging step remains unsubstantiated.
[Section III-B, 'Bridging to artefact models'] The paper asserts that the extended concern model will be mapped to AMDiRE's artefacts, but it provides no method or criteria for this mapping and no analysis of whether the two metamodels are compatible at the required level of granularity. Non-functional, emergent concerns such as fairness or human oversight may not correspond naturally to the predefined content structures of AMDiRE's context, requirements, or system specification artefacts. Since this compatibility is the central enabling assumption of the proposed framework, the paper should either provide a proof-of-concept mapping for at least one concern-artefact pair (beyond the flawed example) or explicitly position the mapping as an open research question rather than as a planned step. As written, the claim that the approaches 'can and should' be integrated overstates what is currently shown.

minor comments (5)

[Section I] The phrase 'let along' should be 'let alone'.
[Section II-A] The sentence 'Governmental bodies and institutions at, such as the ones in the European Union' contains a misplaced 'at'; it should read 'institutions, such as those in the European Union'.
[Section II-C] The sentence 'since of them (e.g., decision trees, linear regression) are inherently more explainable' should read 'since some of them'.
[Section III-C, Table II] The first column of Table II is labeled 'Concern', but the entry 'Stakeholder Roles' is not a trustworthiness concern; it is a modeling element, which makes the mapping unclear. Please either remove it or rename the column to something broader such as 'Element'.
[Section V] There are two language issues in the concluding paragraph: 'an unique role' should be 'a unique role', and 'will hopefully contributing' should be 'will hopefully contribute'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: this is an explicitly scoped vision proposal that defers validation, so its central claim does not reduce to its inputs by construction.

full rationale

The paper does not claim a derived, validated result; it proposes a research direction that combines two previously published methods, AMDiRE and PerSpecML. Its central claim is an envisioned pathway, not a prediction forced by a fitted parameter, a definitional equivalence, or a self-citation chain. The only concrete illustration, Table II, is explicitly illustrative and hand-crafted; it is not presented as an empirical validation or as a result that follows from the methods by construction. The paper itself states in the conclusion that the development 'can only yield meaningful results if continuously evaluated based in practical settings,' acknowledging that the integration remains unvalidated. Self-citations to AMDiRE and PerSpecML are transparent and appropriate for a vision paper that builds on prior work; they are not used to import a uniqueness theorem, smuggle in an ansatz, or forbid alternatives. The weakness noted in the illustrative mapping, including the malformed entry 'The system architecture shall performance under input perturbations,' is a validity and completeness risk for future practical work, not a circularity defect: nothing is being passed off as a derived prediction when it is an input to the proposal. Consequently, no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper makes no quantitative claims, so there are no fitted parameters. The proposal rests instead on four domain assumptions about the adequacy and integrability of the two source methods, both developed by the present authors. No new entities (forces, particles, dimensions) are introduced; the 'envisioned framework' is a stated plan rather than a postulated mechanism.

assumptions (4)

domain assumption The seven AI-HLEG requirements give an adequate decomposition of AI trustworthiness for engineering purposes.
Section II-A adopts the AI-HLEG seven requirements as the normative foundation without arguing they are complete, mutually consistent, or operationalizable into requirements.
domain assumption PerSpecML's 60 concerns across 28 ML tasks provide a sufficient base catalog for trustworthy AI concerns.
Section III-B plans to extend PerSpecML's catalog, which presupposes its adequacy as a base; the paper concedes some trustworthiness aspects, such as societal and environmental well-being, are currently underrepresented.
domain assumption AMDiRE's artefact model can be extended to AI-enabled systems without breaking its deterministic-system orientation.
Section III-A notes AMDiRE was originally conceived for deterministic software-intensive systems, yet the extension is assumed to be unproblematic for AI components.
domain assumption Concern-to-artefact mappings can be made systematic and faithful enough to preserve traceability.
Section III-B's bridging step and Tables I and II assume clean one-to-many mappings between PerSpecML concerns and AMDiRE artefacts; the paper provides no method for deriving or validating these mappings.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards a Framework for Operationalizing the Specification of Trustworthy AI Requirements." pith.science (2026). https://pith.science/paper/CM4ZMHGL

@misc{pith2026250710228,
  author       = {Pith},
  title        = {Pith review of: Towards a Framework for Operationalizing the Specification of Trustworthy AI Requirements},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CM4ZMHGL}},
  note         = {Machine review of arXiv:2507.10228}
}

read the original abstract

Growing concerns around the trustworthiness of AI-enabled systems highlight the role of requirements engineering (RE) in addressing emergent, context-dependent properties that are difficult to specify without structured approaches. In this short vision paper, we propose the integration of two complementary approaches: AMDiRE, an artefact-based approach for RE, and PerSpecML, a perspective-based method designed to support the elicitation, analysis, and specification of machine learning (ML)-enabled systems. AMDiRE provides a structured, artefact-centric, process-agnostic methodology and templates that promote consistency and traceability in the results; however, it is primarily oriented toward deterministic systems. PerSpecML, in turn, introduces multi-perspective guidance to uncover concerns arising from the data-driven and non-deterministic behavior of ML-enabled systems. We envision a pathway to operationalize trustworthiness-related requirements, bridging stakeholder-driven concerns and structured artefact models. We conclude by outlining key research directions and open challenges to be discussed with the RE community.

Figures

Figures reproduced from arXiv: 2507.10228 by the authors.

**Figure 1.** Overview of AMDiRE. relevant to RE. The artefact model describes a family of artefacts and their interdependencies, specifying what content should be produced and how it is structured. The process model outlines a minimal set of milestones, indicating when artefacts should be created and approved and allows for criteria for their quality assurance. To support the practical application of these models, AMDiRE has bee… view at source ↗

**Figure 2.** Overview of PerSpecML. In addition to the ML task and concern diagram, PerSpecML provides a Perspective-based ML Specification Template to help document and organize requirements derived from the identified concerns. This template aids in translating abstract stakeholder needs into concrete specifications relevant to the ML project. III. A FRAMEWORK VISION FOR TRUSTWORTHY AI REQUIREMENTS A. Motivation: Complementary… view at source ↗

**Figure 3.** Overview of the envisioned framework. Concerns and AI Tasks Modeling. Using the extended version of PerSpecML, the team engages stakeholders, including HR personnel, data scientists, legal experts, and regulators, to identify relevant tasks, associated concerns, and potential trade-offs. Table I summarizes these insights. TABLE I SAMPLE TASKS AND TRUSTWORTHINESS CONCERNS Task Stakeholders Concerns Train NLP model D… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 15 canonical work pages

[1]

Regulation (eu) 2024/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence,

European Parliament and Council of the European Union, “Regulation (eu) 2024/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence,” Official Journal of the European Union, L Series, pp. 1–212, 2024, published on 12 July 2024. Entered into force on 1 August 2024. [Online]. Available: htt...

work page 2024
[2]

The global landscape of ai ethics guidelines,

A. Jobin, M. Ienca, and E. Vayena, “The global landscape of ai ethics guidelines,” Nature machine intelligence , vol. 1, no. 9, pp. 389–399, 2019

work page 2019
[3]

Ai4people—an ethical framework for a good ai society: opportunities, risks, principles, and recommendations,

L. Floridi, J. Cowls, M. Beltrametti, R. Chatila, P. Chazerand, V . Dignum, C. Luetge, R. Madelin, U. Pagallo, F. Rossi et al. , “Ai4people—an ethical framework for a good ai society: opportunities, risks, principles, and recommendations,” Minds and machines , vol. 28, pp. 689–707, 2018

work page 2018
[4]

Responsible ai principles from microsoft,

Microsoft, “Responsible ai principles from microsoft,” https://www. microsoft.com/en-us/ai/responsible-ai, 2020, accessed: 2025-06-07

work page 2020
[5]

Tailoring requirements engineering for responsible ai,

W. Maalej, Y . D. Pham, and L. Chazette, “Tailoring requirements engineering for responsible ai,” Computer, vol. 56, no. 4, pp. 18–27, 2023

work page 2023
[6]

Towards achieving trust through transparency and ethics,

D. Kwan, L. M. Cysneiros, and J. C. S. do Prado Leite, “Towards achieving trust through transparency and ethics,” in 2021 IEEE 29th International Requirements Engineering Conference (RE). IEEE, 2021, pp. 82–93

work page 2021
[7]

Ai-enabled regulatory change analysis of legal requirements,

S. Abualhaija, M. Ceci, N. Sannier, D. Bianculli, L. C. Briand, D. Zet- zsche, and M. Bodellini, “Ai-enabled regulatory change analysis of legal requirements,” in 2024 IEEE 32nd International Requirements Engineering Conference (RE) . IEEE, 2024, pp. 5–17

work page 2024
[8]

Regulatory requirements engineering in large enterprises: An interview study on the european accessibility act,

O. Kosenkov, M. Unterkalmsteiner, D. Mendez, and J. Fischbach, “Regulatory requirements engineering in large enterprises: An interview study on the european accessibility act,” in International Conference on Product-Focused Software Process Improvement . Springer, 2024, pp. 204–220

work page 2024

Show all 16 references

[9]

Status quo and problems of requirements engineering for machine learning: Results from an international survey,

A. P. S. Alves, M. Kalinowski, G. Giray, D. Mendez, N. Lavesson, K. Azevedo, H. Villamizar, T. Escovedo, H. Lopes, S. Bifflet al., “Status quo and problems of requirements engineering for machine learning: Results from an international survey,” in International Conference on P...

2023
[10]

Trustworthy ai in practice: an analysis of practitioners’ needs and challenges,

M. T. Baldassarre, D. Gigante, M. Kalinowski, A. Ragone, and S. Tibid `o, “Trustworthy ai in practice: an analysis of practitioners’ needs and challenges,” in Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering , 2024, pp. 293– 302

2024
[11]

Artefact-based require- ments engineering: the amdire approach,

D. M ´endez Fern ´andez and B. Penzenstadler, “Artefact-based require- ments engineering: the amdire approach,” Requirements Engineering , vol. 20, no. 4, pp. 405–434, 2015

2015
[12]

Identify- ing concerns when specifying machine learning-enabled systems: A perspective-based approach,

H. Villamizar, M. Kalinowski, H. Lopes, and D. Mendez, “Identify- ing concerns when specifying machine learning-enabled systems: A perspective-based approach,” Journal of Systems and Software, vol. 213, 2024

2024
[13]

Ethics guidelines for trustworthy ai,

High-Level Expert Group on Artificial Intelligence, “Ethics guidelines for trustworthy ai,” European Commission, Tech. Rep., 2019, accessed: 2025-06-07. [Online]. Available: https://digital-strategy.ec.europa.eu/en/ library/ethics-guidelines-trustworthy-ai

2019
[14]

A rapid review of responsible ai frameworks: How to guide the development of ethical ai,

V . S. Barletta, D. Caivano, D. Gigante, and A. Ragone, “A rapid review of responsible ai frameworks: How to guide the development of ethical ai,” in Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering , 2023, pp. 358–367

2023
[15]

Polaris: A framework to guide the development of trustworthy ai systems,

M. T. Baldassarre, D. Gigante, M. Kalinowski, and A. Ragone, “Polaris: A framework to guide the development of trustworthy ai systems,” in Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering-Software Engineering for AI , 2024, pp. 200–210

2024
[16]

Requirements engineering framework for human-centered artificial intelligence software systems,

K. Ahmad, M. Abdelrazek, C. Arora, A. A. Baniya, M. Bano, and J. Grundy, “Requirements engineering framework for human-centered artificial intelligence software systems,” Applied Soft Computing , vol. 143, p. 110455, 2023

2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Regulation (eu) 2024/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence,

European Parliament and Council of the European Union, “Regulation (eu) 2024/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence,” Official Journal of the European Union, L Series, pp. 1–212, 2024, published on 12 July 2024. Entered into force on 1 August 2024. [Online]. Available: htt...

work page 2024

[2] [2]

The global landscape of ai ethics guidelines,

A. Jobin, M. Ienca, and E. Vayena, “The global landscape of ai ethics guidelines,” Nature machine intelligence , vol. 1, no. 9, pp. 389–399, 2019

work page 2019

[3] [3]

Ai4people—an ethical framework for a good ai society: opportunities, risks, principles, and recommendations,

L. Floridi, J. Cowls, M. Beltrametti, R. Chatila, P. Chazerand, V . Dignum, C. Luetge, R. Madelin, U. Pagallo, F. Rossi et al. , “Ai4people—an ethical framework for a good ai society: opportunities, risks, principles, and recommendations,” Minds and machines , vol. 28, pp. 689–707, 2018

work page 2018

[4] [4]

Responsible ai principles from microsoft,

Microsoft, “Responsible ai principles from microsoft,” https://www. microsoft.com/en-us/ai/responsible-ai, 2020, accessed: 2025-06-07

work page 2020

[5] [5]

Tailoring requirements engineering for responsible ai,

W. Maalej, Y . D. Pham, and L. Chazette, “Tailoring requirements engineering for responsible ai,” Computer, vol. 56, no. 4, pp. 18–27, 2023

work page 2023

[6] [6]

Towards achieving trust through transparency and ethics,

D. Kwan, L. M. Cysneiros, and J. C. S. do Prado Leite, “Towards achieving trust through transparency and ethics,” in 2021 IEEE 29th International Requirements Engineering Conference (RE). IEEE, 2021, pp. 82–93

work page 2021

[7] [7]

Ai-enabled regulatory change analysis of legal requirements,

S. Abualhaija, M. Ceci, N. Sannier, D. Bianculli, L. C. Briand, D. Zet- zsche, and M. Bodellini, “Ai-enabled regulatory change analysis of legal requirements,” in 2024 IEEE 32nd International Requirements Engineering Conference (RE) . IEEE, 2024, pp. 5–17

work page 2024

[8] [8]

Regulatory requirements engineering in large enterprises: An interview study on the european accessibility act,

O. Kosenkov, M. Unterkalmsteiner, D. Mendez, and J. Fischbach, “Regulatory requirements engineering in large enterprises: An interview study on the european accessibility act,” in International Conference on Product-Focused Software Process Improvement . Springer, 2024, pp. 204–220

work page 2024