pith. sign in

arxiv: 2606.09414 · v1 · pith:3B3YZC4Hnew · submitted 2026-06-08 · 💻 cs.HC · cs.AI

AI Assurance in UK Defence: Challenges in Operationalising JSP 936

Callum Cockburn , Sam Farrow This is my paper

Pith reviewed 2026-06-27 15:10 UTC · model grok-4.3

classification 💻 cs.HC cs.AI
keywords AI assuranceUK DefenceJSP 936operationalisation challengessocio-technical systemshuman-AI interactionethical AIsafety and security
0
0 comments X

The pith

JSP 936 supplies a governance basis for AI assurance in UK Defence, but its use hinges on eight unresolved technical, organisational and assurance questions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper conducts a structured interpretive review of JSP 936 Part 1 and extracts eight thematic challenge areas that block practical application. It maintains that the directive supplies a sound governance foundation yet real implementation still requires solutions to problems of evidence adequacy, human-AI interaction management, operational-environment definition, systems-of-systems integration, performance assessment and maintenance, safety and security analysis, ethicality measurement, and AI complexity mitigation. These difficulties arise from the socio-technical character of AI systems, uncertain deployment settings, gaps in existing assurance techniques, and unavoidable trade-offs among performance, safety, oversight, security and ethics. The authors conclude that further methods, guidance and organisational capability are required to achieve ambitious yet safe and responsible AI adoption across Defence.

Core claim

JSP 936 provides a useful governance basis for AI assurance, but implementation depends on unresolved technical, organisational, and assurance questions that stem from the socio-technical nature of AI-enabled systems, uncertainty in real-world deployment contexts, limitations in current assurance methodologies, and tensions between performance, safety, human oversight, security, and ethical acceptability.

What carries the argument

A structured interpretive review of JSP 936 Part 1 that isolates eight thematic challenge areas as the primary barriers to operationalisation.

If this is right

  • Further methods and guidance must be developed to address the eight challenge areas for AI adoption in Defence.
  • Organisational capability building is required to manage the socio-technical aspects of AI assurance.
  • JSP 936 implementation must proceed iteratively to accommodate uncertainties in real-world contexts.
  • Explicit management of trade-offs among performance, safety, human oversight, security and ethics is necessary.
  • Current assurance methodologies require extension to handle AI-specific complexities.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar governance documents in civilian sectors may encounter parallel socio-technical barriers when applied to AI.
  • Targeted training for Defence personnel on ethical measurement and human-AI oversight could reduce several of the identified tensions.
  • Case studies that apply the eight areas to specific AI systems would show which challenges dominate in practice.
  • Linking JSP 936 requirements to established safety standards could ease some of the performance-safety conflicts.

Load-bearing premise

The structured interpretive review has correctly and comprehensively identified the eight thematic challenge areas as the primary barriers to operationalisation.

What would settle it

An empirical demonstration that all eight listed challenge areas can be fully resolved using only existing methods, guidance and organisational structures without further development would falsify the central claim.

read the original abstract

This report examines practical challenges in operationalising JSP 936 Part 1 for AI assurance in UK Defence. Using a structured interpretive review of the directive's requirements, the analysis identifies eight thematic challenge areas adequacy of evidence and argument, management of human interaction with AI, definition of the operational environment, integration of AI within systems of systems, assessment and maintenance of AI performance, analysis of safety and security, measurement of ethicality, and mitigation of the inherent complexities of AI. The report argues that JSP 936 provides a useful governance basis, but that implementation depends on unresolved technical, organisational, and assurance questions. These challenges stem from the socio-technical nature of AI-enabled systems, uncertainty in real-world deployment contexts, limitations in current assurance methodologies, and tensions between performance, safety, human oversight, security, and ethical acceptability. The report identifies areas where further methods, guidance, and organisational capability are needed for the ambitious, safe, and responsible adoption of AI across Defence. This is consistent with MOD's own framing of JSP 936 as requiring iterative implementation and supporting guidance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper examines practical challenges in operationalising JSP 936 Part 1 for AI assurance in UK Defence. Using a structured interpretive review of the directive's requirements, it identifies eight thematic challenge areas (adequacy of evidence and argument, management of human interaction with AI, definition of the operational environment, integration of AI within systems of systems, assessment and maintenance of AI performance, analysis of safety and security, measurement of ethicality, and mitigation of the inherent complexities of AI). It argues that JSP 936 provides a useful governance basis but that implementation depends on unresolved technical, organisational, and assurance questions stemming from the socio-technical nature of AI-enabled systems.

Significance. If the eight themes are shown to be exhaustive and correctly derived, the report would usefully map socio-technical and methodological gaps that must be addressed for responsible AI adoption in defence, aligning with MOD's own framing of iterative implementation.

major comments (1)
  1. [Abstract / methods description of the review] The description of the 'structured interpretive review' (Abstract and the section presenting the eight themes) provides no protocol details, requirement-by-requirement mapping to JSP 936 Part 1, exclusion criteria, or completeness checks. Without these, it is impossible to verify whether the eight areas exhaustively cover the directive or whether other barriers were overlooked, directly undermining the central claim that implementation depends on resolving exactly these gaps.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive feedback on the manuscript. The single major comment raises a valid methodological concern that we can address through revision.

read point-by-point responses

  1. Referee: [Abstract / methods description of the review] The description of the 'structured interpretive review' (Abstract and the section presenting the eight themes) provides no protocol details, requirement-by-requirement mapping to JSP 936 Part 1, exclusion criteria, or completeness checks. Without these, it is impossible to verify whether the eight areas exhaustively cover the directive or whether other barriers were overlooked, directly undermining the central claim that implementation depends on resolving exactly these gaps.

    Authors: We agree that the current description of the structured interpretive review lacks sufficient methodological transparency. The review was interpretive rather than a formal systematic review, drawing on close reading of JSP 936 Part 1 to surface socio-technical challenges, but no explicit protocol, mapping table, or completeness argument was provided. In the revised manuscript we will add a dedicated methods subsection that: (1) outlines the interpretive process (iterative thematic grouping of directive requirements), (2) provides a high-level mapping of key JSP 936 requirements to the eight themes, (3) states the scope and any implicit exclusion criteria (e.g., focus on Part 1 only), and (4) includes a short limitations paragraph acknowledging that exhaustiveness cannot be formally demonstrated. This will allow readers to assess coverage without altering the identified challenge areas or the overall argument. revision: yes

Circularity Check

0 steps flagged

No significant circularity: external policy review with no self-referential derivations

full rationale

The paper performs a structured interpretive review of the external JSP 936 Part 1 directive to extract eight thematic challenge areas, then concludes that operationalisation depends on resolving those challenges. No equations, fitted parameters, self-citations, uniqueness theorems, or ansatzes appear in the provided text. The derivation chain consists of mapping an external policy document to themes and does not reduce any claim to the paper's own inputs by construction. This is a standard interpretive analysis of an outside source and is self-contained against that benchmark.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

This is a policy analysis report with no mathematical derivations, data fitting, or new postulated entities. The only background assumption is that JSP 936 Part 1 constitutes the relevant governance document for the analysis.

axioms (1)
  • domain assumption JSP 936 Part 1 is the authoritative and relevant directive whose operationalisation challenges are under examination
    The entire analysis is scoped to this single document; the paper does not justify why other directives or standards are secondary.

pith-pipeline@v0.9.1-grok · 5711 in / 1244 out tokens · 24318 ms · 2026-06-27T15:10:08.555219+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

119 extracted references · 50 canonical work pages · 1 internal anchor

  1. [1]

    Ver- sion 1

    Ministry of Defence.JSP 936: Dependable Artificial Intelligence (AI) in defence (part 1: directive). Ver- sion 1. Nov. 13, 2024.URL: https : / / www . gov . uk / government / publications / jsp - 936 - dependable - artificial- intelligence- ai- in- defence- part- 1- directive (visited on 05/29/2026). 8TH JUNE 2025 14

    2024

  2. [2]

    Safety and As- surance Cases: Past, Present and Possible Future – an Adelard Perspective

    Robin Bloomfield and Peter Bishop. “Safety and As- surance Cases: Past, Present and Possible Future – an Adelard Perspective”. In:Making Systems Safer. Ed. by Chris Dale and Tom Anderson. London: Springer, 2010, pp. 51–67.ISBN: 978-1-84996-086-1.DOI: 10. 1007/978-1-84996-086-1 4

    2010

  3. [3]

    Software Assurance Using Structured Assurance Case Models

    Thomas Rhodes, Elizabeth Fong, and Michael Kass. “Software Assurance Using Structured Assurance Case Models”. In:Journal of Research of the National Institute of Standards and Technology115.3 (2010)

    2010

  4. [4]

    Version Number: 3

    Ibrahim Habli et al.The BIG Argument for AI Safety Cases. Version Number: 3. 2025.DOI: 10 . 48550 / ARXIV.2503.11705.URL: https://arxiv.org/abs/2503. 11705 (visited on 06/04/2026)

    arXiv 2025

  5. [5]

    Center for International Security and Cooperation: Freeman Spogli Institute for International Studies, Stanford University, Nov

    Isaac Gazendam and Philip Dawson.Mind the Gap: The Challenges of Assurance for Artificial Intelligence. Center for International Security and Cooperation: Freeman Spogli Institute for International Studies, Stanford University, Nov. 26, 2023.URL: https : / / cisac . fsi . stanford . edu / publication / mind - gap - challenges-assurance-artificial-intellig...

    2023

  6. [6]

    Introduction to AI assurance

    Department for Science, Innovation and Technology. Introduction to AI assurance. GOV.UK. Feb. 12, 2024. URL: https://www.gov.uk/government/publications/ introduction - to - ai - assurance / introduction - to - ai - assurance (visited on 06/04/2026)

    2024

  7. [7]

    A sociotechnical system perspective on AI

    Olya Kudina and Ibo van de Poel. “A sociotechnical system perspective on AI”. In:Minds and Machines 34.3 (June 12, 2024), p. 21.ISSN: 1572-8641.DOI: 10.1007/s11023-024-09680-2.URL: https://doi.org/ 10.1007/s11023-024-09680-2 (visited on 04/02/2026)

    work page doi:10.1007/s11023-024-09680-2.url: 2024

  8. [8]

    Facing & mitigating common chal- lenges when working with real-world data: The Data Learning Paradigm

    Jake Lever et al. “Facing & mitigating common chal- lenges when working with real-world data: The Data Learning Paradigm”. In:Journal of Computational Science85 (Feb. 1, 2025), p. 102523.ISSN: 1877- 7503.DOI: 10 . 1016 / j . jocs . 2024 . 102523.URL: https : / / www. sciencedirect . com / science / article / pii / S1877750324003168 (visited on 04/03/2026)

    2025

  9. [9]

    Data models, rep- resentation and adequacy-for-purpose

    Alisa Bokulich and Wendy Parker. “Data models, rep- resentation and adequacy-for-purpose”. In:European Journal for Philosophy of Science11.1 (Mar. 2021), p. 31.ISSN: 1879-4912, 1879-4920.DOI: 10 . 1007 / s13194-020-00345-2.URL: http://link.springer.com/ 10.1007/s13194-020-00345-2 (visited on 06/02/2026)

    work page doi:10.1007/s13194-020-00345-2 2021

  10. [10]

    Defence Committee.Developing AI capacity and ex- pertise in UK Defence. HC 590. UK Parliment, Jan. 10, 2025.URL: https : / / publications . parliament . uk / pa / cm5901/cmselect/cmdfence/812/report.html (visited on 06/04/2026)

    2025

  11. [11]

    Non-functional requirements for ma- chine learning: understanding current use and chal- lenges among practitioners

    Khan Mohammad Habibullah, Gregory Gay, and Jen- nifer Horkoff. “Non-functional requirements for ma- chine learning: understanding current use and chal- lenges among practitioners”. In:Requirements Engi- neering28.2 (June 1, 2023), pp. 283–316.ISSN: 1432- 010X.DOI: 10.1007/s00766-022-00395-3.URL: https: //doi.org/10.1007/s00766- 022- 00395- 3 (visited on 0...

    work page doi:10.1007/s00766-022-00395-3.url: 2023

  12. [12]

    Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges

    Rob Ashmore, Radu Calinescu, and Colin Paterson. “Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges”. In:ACM Computing Sur- veys54.5 (June 30, 2022), pp. 1–39.ISSN: 0360-0300, 1557-7341.DOI: 10.1145/3453444.URL: https://dl. acm.org/doi/10.1145/3453444 (visited on 05/25/2026)

    work page doi:10.1145/3453444.url: 2022

  13. [13]

    Agile Management for Machine Learning: A Systematic Mapping Study

    Lucas Romao et al. “Agile Management for Machine Learning: A Systematic Mapping Study”. In:Software Engineering and Advanced Applications. Ed. by Da- vide Taibi and Darja Smite. Cham: Springer Nature Switzerland, 2026, pp. 350–360.ISBN: 978-3-032- 04200-2.DOI: 10.1007/978-3-032-04200-2 24

    work page doi:10.1007/978-3-032-04200-2 2026

  14. [14]

    Software Engineering for Ma- chine Learning: A Case Study

    Saleema Amershi et al. “Software Engineering for Ma- chine Learning: A Case Study”. In:2019 IEEE/ACM 41st International Conference on Software Engineer- ing: Software Engineering in Practice (ICSE-SEIP). 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Prac- tice (ICSE-SEIP). May 2019, pp. 291–300.DOI: 10. 1109/...

    arXiv 2019

  15. [15]

    How mature is requirements engineering for AI-based systems? A systematic map- ping study on practices, challenges, and future re- search directions

    Umm-e- Habiba et al. “How mature is requirements engineering for AI-based systems? A systematic map- ping study on practices, challenges, and future re- search directions”. In:Requirements Engineering29.4 (Dec. 1, 2024), pp. 567–600.ISSN: 1432-010X.DOI: 10.1007/s00766-024-00432-3.URL: https://doi.org/ 10.1007/s00766-024-00432-3 (visited on 04/03/2026)

    work page doi:10.1007/s00766-024-00432-3.url: 2024

  16. [16]

    Requirements engineering for artificial intelligence systems: A systematic mapping study

    Khlood Ahmad et al. “Requirements engineering for artificial intelligence systems: A systematic mapping study”. In:Information and Software Technology158 (June 1, 2023), p. 107176.ISSN: 0950-5849.DOI: 10 . 1016 / j . infsof . 2023 . 107176.URL: https : / / www . sciencedirect . com / science / article / pii / S0950584923000307 (visited on 05/25/2026)

    2023

  17. [17]

    Modeling machine learning requirements from three perspectives: a case report from the healthcare domain

    Soroosh Nalchigar, Eric Yu, and Karim Keshav- jee. “Modeling machine learning requirements from three perspectives: a case report from the healthcare domain”. In:Requirements Engineering26.2 (June 2021), pp. 237–254.ISSN: 0947-3602, 1432-010X. DOI: 10 . 1007 / s00766 - 020 - 00343 - z.URL: https : / / link . springer. com / 10 . 1007 / s00766 - 020 - 0034...

    2021

  18. [18]

    Antonio Pedro Santos Alves et al.Status Quo and Problems of Requirements Engineering for Ma- chine Learning: Results from an International Survey. Oct. 10, 2023.DOI: 10 . 48550 / arXiv. 2310 . 06726. arXiv: 2310.06726[cs.SE].URL: http://arxiv.org/abs/ 2310.06726 (visited on 05/25/2026)

    arXiv 2023

  19. [19]

    Expectation management in AI: A framework for understanding stakeholder trust and acceptance of artificial intelligence systems

    Marjorie Kinney et al. “Expectation management in AI: A framework for understanding stakeholder trust and acceptance of artificial intelligence systems”. In: Heliyon10.7 (Apr. 2024), e28562.ISSN: 24058440. DOI: 10 . 1016 / j . heliyon . 2024 . e28562.URL: https : / / linkinghub . elsevier . com / retrieve / pii / S2405844024045936 (visited on 05/25/2026)....

    2024

  20. [20]

    Version 1

    Lloyd’s Register.LR Code for Unmanned Marine Systems. Version 1. Feb. 2017.URL: https://events. iala . int / content / uploads / 2021 / 07 / LR Code for Unmanned Marine Systems February 2017.pdf (visited on 05/25/2026)

    2017

  21. [21]

    Version 3

    Society of Automotive Engineers.Taxonomy and Def- initions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Version 3. Apr. 2024.URL: https://webstore.ansi.org/standards/sae/ sae30162021 (visited on 05/25/2026)

    2024

  22. [22]

    A model for types and levels of human interaction with automation

    R. Parasuraman, T.B. Sheridan, and C.D. Wickens. “A model for types and levels of human interaction with automation”. In:IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans30.3 (May 2000), pp. 286–297.ISSN: 10834427.DOI: 10. 1109/3468.844354.URL: http://ieeexplore.ieee.org/ document/844354/ (visited on 05/25/2026)

    arXiv 2000

  23. [23]

    Version 1

    International Standards Organisation.Ships and ma- rine technology — Vocabulary related to autonomous ship systems. Version 1. May 2022.URL: https://www. iso.org/standard/77186.html (visited on 05/25/2026)

    2022

  24. [24]

    Ironies of automation

    Lisanne Bainbridge. “Ironies of automation”. In:Auto- matica19.6 (Nov. 1, 1983), pp. 775–779.ISSN: 0005- 1098.DOI: 10 . 1016 / 0005 - 1098(83 ) 90046 - 8.URL: https : / / www. sciencedirect . com / science / article / pii / 0005109883900468 (visited on 01/05/2026)

    1983

  25. [25]

    The Out-of-the- Loop Performance Problem and Level of Control in Automation

    Mica R. Endsley and Esin O. Kiris. “The Out-of-the- Loop Performance Problem and Level of Control in Automation”. In:Human Factors37.2 (June 1, 1995). Publisher: SAGE Publications Inc, pp. 381–394.ISSN: 0018-7208.DOI: 10.1518/001872095779064555.URL: https://doi.org/10.1518/001872095779064555 (visited on 05/25/2026)

    work page doi:10.1518/001872095779064555.url: 1995

  26. [26]

    Com- placency and Bias in Human Use of Automation: An Attentional Integration

    Raja Parasuraman and Dietrich H. Manzey. “Com- placency and Bias in Human Use of Automation: An Attentional Integration”. In:Human Factors52.3 (June 1, 2010). Publisher: SAGE Publications Inc, pp. 381–410.ISSN: 0018-7208.DOI: 10 . 1177 / 0018720810376055.URL: https : / / doi . org / 10 . 1177 / 0018720810376055 (visited on 05/25/2026)

    2010

  27. [27]

    Trust in Automation: Designing for Appropriate Reliance

    John D. Lee and Katrina A. See. “Trust in Automation: Designing for Appropriate Reliance”. In:Human Fac- tors46.1 (Mar. 1, 2004). Publisher: SAGE Publications Inc, pp. 50–80.ISSN: 0018-7208.DOI: 10.1518/hfes. 46.1.50 30392.URL: https://journals.sagepub.com/ action/showAbstract (visited on 08/20/2025)

    work page doi:10.1518/hfes 2004

  28. [28]

    Richard Hawkins et al.Guidance on the Safety Assur- ance of Autonomous Systems in Complex Environments (SACE). Aug. 1, 2022.DOI: 10 . 48550 / arXiv. 2208 . 00853. arXiv: 2208.00853[cs.SE].URL: http://arxiv. org/abs/2208.00853 (visited on 05/26/2026)

    arXiv 2022

  29. [29]

    Dynamic Assurance Cases: A Pathway to Trusted Autonomy

    Erfan Asaadi et al. “Dynamic Assurance Cases: A Pathway to Trusted Autonomy”. In:Computer53.12 (Dec. 2020), pp. 35–46.ISSN: 0018-9162, 1558-0814. DOI: 10 . 1109 / MC . 2020 . 3022030.URL: https : / / ieeexplore . ieee . org / document / 9269875/ (visited on 05/26/2026)

    2020

  30. [30]

    Robin Bloomfield and John Rushby.Assurance 2.0: A Manifesto. Jan. 14, 2021.DOI: 10 . 48550 / arXiv. 2004 . 10474. arXiv: 2004 . 10474[cs . SE].URL: http : //arxiv.org/abs/2004.10474 (visited on 05/26/2026)

    arXiv 2021

  31. [31]

    A framework to identify failure scenarios in the control mode transition process for autonomous ships with dynamic autonomy

    X. Yang et al. “A framework to identify failure scenarios in the control mode transition process for autonomous ships with dynamic autonomy”. In:Ocean & Coastal Management249 (Mar. 1, 2024), p. 107003. ISSN: 0964-5691.DOI: 10.1016/j.ocecoaman.2023. 107003.URL: https : / / www . sciencedirect . com / science / article / pii / S0964569123005288 (visited on ...

    work page doi:10.1016/j.ocecoaman.2023 2024

  32. [32]

    A systematic review of ar- tificial intelligence impact assessments

    Bernd Carsten Stahl et al. “A systematic review of ar- tificial intelligence impact assessments”. In:Artificial Intelligence Review56.11 (Nov. 1, 2023), pp. 12799– 12831.ISSN: 1573-7462.DOI: 10.1007/s10462-023- 10420-8.URL: https://doi.org/10.1007/s10462-023- 10420-8 (visited on 04/03/2026)

    work page doi:10.1007/s10462-023- 2023

  33. [33]

    Version 1

    International Standards Organisation.ISO/IEC 42001:2023 Information technology — Artificial intelligence — Management system. Version 1. Dec. 2023.URL: https://www.iso.org/standard/81230.html

    2023

  34. [34]

    Topics — European Parliament

    European Parliment.EU AI Act: first regulation on artificial intelligence. Topics — European Parliament. June 8, 2023.URL: https : / / www. europarl . europa . eu/topics/en/article/20230601STO93804/eu- ai- act- first - regulation - on - artificial - intelligence (visited on 11/05/2024)

    arXiv 2023

  35. [35]

    Let Me Take Over: Variable Au- tonomy for Meaningful Human Control

    Leila Methnani et al. “Let Me Take Over: Variable Au- tonomy for Meaningful Human Control”. In:Frontiers in Artificial Intelligence4 (Sept. 14, 2021), p. 737072. ISSN: 2624-8212.DOI: 10 . 3389 / frai . 2021 . 737072. URL: https : / / pmc . ncbi . nlm . nih . gov / articles / PMC8477008/ (visited on 05/26/2026)

    2021

  36. [37]

    Cai, Y ., Goswami, M., Choudhry, A., Srinivasan, A., and Dubrawski, A

    Isaac Triguero et al. “General Purpose Artificial Intel- ligence Systems (GPAIS): Properties, definition, tax- onomy, societal implications and responsible gover- nance”. In:Information Fusion103 (Mar. 1, 2024), p. 102135.ISSN: 1566-2535.DOI: 10.1016/j.inffus. 2023.102135.URL: https://www.sciencedirect.com/ science / article / pii / S1566253523004517 (vis...

    work page doi:10.1016/j.inffus 2024

  37. [38]

    Trust in Artificial Intelli- gence: Meta-Analytic Findings

    Alexandra D. Kaplan et al. “Trust in Artificial Intelli- gence: Meta-Analytic Findings”. In:Human Factors 65.2 (Mar. 1, 2023). Publisher: SAGE Publications Inc, pp. 337–359.ISSN: 0018-7208.DOI: 10 . 1177 / 00187208211013988.URL: https://doi.org/10.1177/ 00187208211013988 (visited on 05/26/2026)

    2023

  38. [39]

    Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust

    Kevin Anthony Hoff and Masooda Bashir. “Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust”. In:Human Factors 57.3 (May 1, 2015). Publisher: SAGE Publications Inc, pp. 407–434.ISSN: 0018-7208.DOI: 10 . 1177 / 8TH JUNE 2025 16 0018720814547570.URL: https : / / doi . org / 10 . 1177 / 0018720814547570 (visited on 05/26/2026)

    2015

  39. [40]

    Trust in Automation

    Robert R. Hoffman et al. “Trust in Automation”. In: IEEE Intelligent Systems28.1 (Jan. 2013), pp. 84–88. ISSN: 1941-1294.DOI: 10.1109/MIS.2013.24.URL: https://ieeexplore.ieee.org/document/6468022 (visited on 08/20/2025)

    work page doi:10.1109/mis.2013.24.url: 2013

  40. [41]

    Measuring and Calibrating Trust in Artificial Intelligence

    Mathias Bollaert, Olivier Augereau, and Gilles Cop- pin. “Measuring and Calibrating Trust in Artificial Intelligence”. Mar. 2024.URL: https://hal.science/hal- 04493669 (visited on 05/26/2026)

    2024

  41. [42]

    Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions

    Magdalena Wischnewski, Nicole Kr ¨amer, and Em- manuel M ¨uller. “Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions”. In:Proceed- ings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23. New York, NY, USA: Association for Computing Machinery, Apr. 19, 2023, p...

    arXiv 2023

  42. [43]

    Adaptive trust calibration for human-AI collaboration

    Kazuo Okamura and Seiji Yamada. “Adaptive trust calibration for human-AI collaboration”. In:PLOS ONE15.2 (Feb. 21, 2020). Publisher: Public Library of Science, e0229132.ISSN: 1932-6203.DOI: 10.1371/ journal.pone.0229132.URL: https://journals.plos.org/ plosone / article ? id = 10 . 1371 / journal . pone . 0229132 (visited on 07/01/2025)

    2020

  43. [44]

    Adaptive Cognitive Mech- anisms to Maintain Calibrated Trust and Reliance in Automation

    Christian Lebiere et al. “Adaptive Cognitive Mech- anisms to Maintain Calibrated Trust and Reliance in Automation”. In:Frontiers in Robotics and AI8 (May 24, 2021), p. 652776.ISSN: 2296-9144.DOI: 10 . 3389 / frobt . 2021 . 652776.URL: https : / / pmc . ncbi.nlm.nih.gov/articles/PMC8181412/ (visited on 05/26/2026)

    2021

  44. [45]

    Trust Calibration for Joint Human/AI Decision-Making in Dynamic and Uncer- tain Contexts

    Laura R. Marusich et al. “Trust Calibration for Joint Human/AI Decision-Making in Dynamic and Uncer- tain Contexts”. In:Artificial Intelligence in HCI. Ed. by Helmut Degen and Stavroula Ntoa. Cham: Springer Nature Switzerland, 2025, pp. 106–120.ISBN: 978-3- 031-93412-4.DOI: 10.1007/978-3-031-93412-4 6

    work page doi:10.1007/978-3-031-93412-4 2025

  45. [46]

    Building trustworthy Artificial Intelligence through transparency explain- ability uncertainty and trust calibration

    Haochen Guo and Petr Polak. “Building trustworthy Artificial Intelligence through transparency explain- ability uncertainty and trust calibration”. In:Discover Artificial Intelligence(Apr. 13, 2026).ISSN: 2731- 0809.DOI: 10.1007/s44163-026-01219-x.URL: https: //doi.org/10.1007/s44163- 026- 01219- x (visited on 05/26/2026)

    work page doi:10.1007/s44163-026-01219-x.url: 2026

  46. [47]

    Supporting Human-AI Teams:Transparency, explainability, and situation awareness

    Mica R. Endsley. “Supporting Human-AI Teams:Transparency, explainability, and situation awareness”. In:Computers in Human Behavior 140 (Mar. 1, 2023), p. 107574.ISSN: 0747- 5632.DOI: 10 . 1016 / j . chb . 2022 . 107574.URL: https : / / www. sciencedirect . com / science / article / pii / S0747563222003946 (visited on 05/26/2026)

    2023

  47. [48]

    Situation awareness- based agent transparency and human-autonomy teaming effectiveness

    Jessie Y . C. Chen et al. “Situation awareness- based agent transparency and human-autonomy teaming effectiveness”. In:Theoretical Issues in Ergonomics Science19.3 (May 4, 2018). Publisher: Taylor & Francis eprint: https://doi.org/10.1080/1463922X.2017.1315750, pp. 259–282.ISSN: 1463-922X.DOI: 10 . 1080 / 1463922X . 2017 . 1315750.URL: https : / / doi . o...

    work page doi:10.1080/1463922x.2017.1315750 2018

  48. [49]

    https://doi.org/10.1145/3630106.3659051, fAccT ’24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Trans- parency

    Sarah Sterz et al. “On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives”. In:Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. FAccT ’24. New York, NY, USA: Association for Computing Machinery, June 5, 2024, pp. 2495–2507.ISBN: 979- 8-4007-0450-5.DOI: 10.1145/3630106.3659051.URL: https://do...

    work page doi:10.1145/3630106.3659051.url: 2024

  49. [50]

    AI Safety and Automation Bias

    Lauren Kahn, Emelia Probasco, and Ronnie Kinoshita. AI Safety and Automation Bias. Center for Secu- rity and Emerging Technology, Nov. 2024.DOI: 10. 51593/20230057.URL: https://cset.georgetown.edu/ publication/ai-safety-and-automation-bias/ (visited on 05/26/2026)

    2024

  50. [51]

    Human oversight of artificial intelligence: An operations management perspective

    Jes ´us Salgado-Criado. “Human oversight of artificial intelligence: An operations management perspective”. In:Journal of Industrial Engineering and Management 18.2 (June 16, 2025), p. 285.ISSN: 2013-0953, 2013- 8423.DOI: 10.3926/jiem.8567.URL: https://www. jiem.org/index.php/jiem/article/view/8567 (visited on 05/26/2026)

    work page doi:10.3926/jiem.8567.url: 2025

  51. [52]

    Vaishali Vinay.Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applications. Nov. 26, 2025.DOI: 10 . 48550 / arXiv. 2511 . 19933. arXiv: 2511.19933[cs.AI].URL: http://arxiv.org/abs/ 2511.19933 (visited on 05/26/2026)

    arXiv 2025

  52. [53]

    Evaluation of LLM-based chatbots for OSINT- based Cyber Threat Awareness

    Samaneh Shafee, Alysson Bessani, and Pedro M. Fer- reira. “Evaluation of LLM-based chatbots for OSINT- based Cyber Threat Awareness”. In:Expert Systems with Applications261 (Feb. 1, 2025), p. 125509.ISSN: 0957-4174.DOI: 10.1016/j.eswa.2024.125509.URL: https : / / www. sciencedirect . com / science / article / pii / S0957417424023765 (visited on 05/26/2026)

    work page doi:10.1016/j.eswa.2024.125509.url: 2025

  53. [54]

    Ruling the Operational Boundaries: A Survey on Operational Design Domains of Au- tonomous Driving Systems

    Marcel Aguirre Mehlhorn, Andreas Richter, and Yuri A. W. Shardt. “Ruling the Operational Boundaries: A Survey on Operational Design Domains of Au- tonomous Driving Systems”. In:IFAC-PapersOnLine. 22nd IFAC World Congress 56.2 (Jan. 1, 2023), pp. 2202–2213.ISSN: 2405-8963.DOI: 10 . 1016 / j . ifacol.2023.10.1128.URL: https://www.sciencedirect. com/science/...

    2023

  54. [55]

    Version Num- ber: 2

    Quanshi Zhang and Song-Chun Zhu.Visual Inter- pretability for Deep Learning: a Survey. Version Num- ber: 2. 2018.DOI: 10 . 48550 / ARXIV. 1802 . 00614. URL: https : / / arxiv. org / abs / 1802 . 00614 (visited on 05/27/2026)

    2018

  55. [56]

    Do Computational Models Differ Systematically from Human Object Perception?

    R. T. Pramod and S. P. Arun. “Do Computational Models Differ Systematically from Human Object Perception?” In:2016 IEEE Conference on Computer 8TH JUNE 2025 17 Vision and Pattern Recognition (CVPR). 2016 IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR). Las Vegas, NV, USA: IEEE, June 2016, pp. 1601–1609.ISBN: 978-1-4673-8851-1.DOI: 10 ....

    arXiv 2016

  56. [57]

    Informing Autonomous System Design Through the Lens of Skill-, Rule-, and Knowledge-Based Behaviors

    Mary (Missy) Cummings. “Informing Autonomous System Design Through the Lens of Skill-, Rule-, and Knowledge-Based Behaviors”. In:Journal of Cogni- tive Engineering and Decision Making12.1 (Mar. 1, 2018). Publisher: SAGE Publications, pp. 58–61.ISSN: 1555-3434.DOI: 10.1177/1555343417736461.URL: https://doi.org/10.1177/1555343417736461 (visited on 05/27/2026)

    work page doi:10.1177/1555343417736461.url: 2018

  57. [58]

    Beyond the Machine: An Integrative Framework of Anthropomor- phism in AI

    Petru Lucian Curs , eu and S, tefana Radu. “Beyond the Machine: An Integrative Framework of Anthropomor- phism in AI”. In:Behavioral Sciences16.3 (Mar. 2026). Publisher: Multidisciplinary Digital Publishing Institute, p. 358.ISSN: 2076-328X.DOI: 10 . 3390 / bs16030358.URL: https : / / www. mdpi . com / 2076 - 328X/16/3/358 (visited on 05/26/2026)

    2026

  58. [59]

    Anthropomorphism in AI

    Arleen Salles, Kathinka Evers, and Michele Farisco. “Anthropomorphism in AI”. In:AJOB Neuroscience 11.2 (Apr. 2, 2020), pp. 88–95.ISSN: 2150-7740, 2150-7759.DOI: 10.1080/21507740.2020.1740350. URL: https://www.tandfonline.com/doi/full/10.1080/ 21507740.2020.1740350 (visited on 05/26/2026)

    work page doi:10.1080/21507740.2020.1740350 2020

  59. [60]

    Approach for Argumenting Safety on Basis of an Operational Design Domain

    Gereon Weiss et al. “Approach for Argumenting Safety on Basis of an Operational Design Domain”. In:Pro- ceedings of the IEEE/ACM 3rd International Confer- ence on AI Engineering - Software Engineering for AI. CAIN 2024: IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI. Lisbon Portugal: ACM, Apr. 14, 2024, pp. 184–193. ...

    arXiv 2024

  60. [61]

    version: 1

    Ali Shakeri.Formalization of Operational Domain and Operational Design Domain for Automated Vehicles. version: 1. Aug. 16, 2024.DOI: 10.48550/arXiv.2408. 14481. arXiv: 2408.14481[cs.RO].URL: http://arxiv. org/abs/2408.14481 (visited on 05/26/2026)

    work page doi:10.48550/arxiv.2408 2024

  61. [62]

    Hyunin Lee et al.A Black Swan Hypothesis: The Role of Human Irrationality in AI Safety. Mar. 20, 2025. DOI: 10 . 48550 / arXiv . 2407 . 18422. arXiv: 2407 . 18422[cs.AI].URL: http://arxiv.org/abs/2407.18422 (visited on 05/27/2026)

    arXiv 2025

  62. [63]

    Safety at the edge: a safety framework to identify edge conditions in the future transportation system with highly automated vehicles

    Megan S Ryerson et al. “Safety at the edge: a safety framework to identify edge conditions in the future transportation system with highly automated vehicles”. In:Injury Prevention26.4 (Aug. 2020), pp. 386–390.ISSN: 1353-8047, 1475-5785.DOI: 10. 1136 / injuryprev - 2019 - 043134.URL: https : / / injuryprevention . bmj . com / lookup / doi / 10 . 1136 / in...

    2020

  63. [64]

    June 15, 2024.DOI: 10

    Philip Koopman.Lessons from the Cruise Robotaxi Pedestrian Dragging Mishap. June 15, 2024.DOI: 10. 48550/arXiv.2406.05281. arXiv: 2406.05281[cs.RO]. URL: http : / / arxiv. org / abs / 2406 . 05281 (visited on 05/26/2026)

    arXiv 2024

  64. [65]

    How Many Op- erational Design Domains, Objects, and Events?

    Philip Koopman and Frank Fratrik. “How Many Op- erational Design Domains, Objects, and Events?” In: SafeAI@AAAI. V ol. 2301. CEUR Workshop Proceed- ings. CEUR-WS.org, 2019.URL: https://ceur-ws.org/ V ol-2301/paper 6.pdf

    2019

  65. [66]

    Philip Koopman and William Widen.Redefining Safety for Autonomous Vehicles. Aug. 12, 2024.DOI: 10 . 48550/arXiv.2404.16768. arXiv: 2404.16768[cs.RO]. URL: http : / / arxiv. org / abs / 2404 . 16768 (visited on 05/27/2026)

    arXiv 2024

  66. [67]

    Digital tech- nologies: civilian vs. military trajectories

    Dario Guarascio and Mario Pianta. “Digital tech- nologies: civilian vs. military trajectories”. In:LEM Working Papers(2025). Ed. by Federico Tamagni. Publisher: Scuola Superiore Sant’Anna, pp. 2025/08. ISSN: 2284-0400.DOI: 10.57838/SSSA/DKB0-WB35. URL: https : / / ideas . repec . org / p / ssa / lemwps / 2025 - 08.html (visited on 05/27/2026)

    work page doi:10.57838/sssa/dkb0-wb35 2025

  67. [68]

    Unlocking the strategic power of dual-use technologies

    Tobias Aebi et al. “Unlocking the strategic power of dual-use technologies”. In:VIEWPOINT. ARTHUR D. LITTLE (2025).URL: https : / / www . adlittle . com / sites/default/files/viewpoints/ADL%20Dual%20use% 20technologies%202025.pdf

    2025

  68. [69]

    Public procurement and innovation: is defence different?

    Oishee Kundu. “Public procurement and innovation: is defence different?” PhD thesis. Alliance Manch- ester Business School: The University of Manchester, Dec. 6, 2021.URL: https://research.manchester.ac.uk/ en/studentTheses/public-procurement-and-innovation- is-defence-different/ (visited on 05/27/2026)

    2021

  69. [70]

    Drezner, and Elizabeth M

    Brodi Kotila, Jeffrey A. Drezner, and Elizabeth M. Bartels.Fostering Innovation in Military Technol- ogy: Strengthening DoD’s Commercial Technology Pipeline. RR-A1352-1. RAND Corporation, 2023. DOI: 10.7249/RBA1352- 1.URL: https://www.rand. org/pubs/research briefs/RBA1352-1.html (visited on 05/27/2026)

    work page doi:10.7249/rba1352- 2023

  70. [71]

    Navigating the landscape of operational design domains: A comprehensive map- ping study

    Daniel Hillen et al. “Navigating the landscape of operational design domains: A comprehensive map- ping study”. In:Next Research2.4 (Dec. 1, 2025), p. 101036.ISSN: 3050-4759.DOI: 10.1016/j.nexres. 2025.101036.URL: https://www.sciencedirect.com/ science / article / pii / S3050475925009030 (visited on 05/27/2026)

    work page doi:10.1016/j.nexres 2025

  71. [72]

    Unknown knowns: Tacit knowledge in requirements engineering

    Peter Sawyer, Vincenzo Gervasi, and Bashar Nuseibeh. “Unknown knowns: Tacit knowledge in requirements engineering”. In:RE ’11: Proceedings of the 2011 IEEE 19th International Requirements Engineering Conference(2011). Publisher: IEEE, p. 329.ISSN: 978-1-4577-0921-0.DOI: 10.1109/RE.2011.6051683. URL: http://ieeexplore.ieee.org/document/6051683/ (visited on...

    work page doi:10.1109/re.2011.6051683 2011

  72. [73]

    Operational Design Domain (ODD) framework for driver-automation integrated systems

    HongSeok Cho. “Operational Design Domain (ODD) framework for driver-automation integrated systems”. PhD thesis. Massachusetts Institute of Technology, 2020.URL: https : / / hdl . handle . net / 1721 . 1 / 129156 (visited on 05/27/2026). 8TH JUNE 2025 18

    2020

  73. [74]

    Hodge, Colin Paterson, and Ibrahim Habli

    Victoria J. Hodge, Colin Paterson, and Ibrahim Habli. Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systems. Oct. 24, 2025.DOI: 10. 48550/arXiv.2510.21254. arXiv: 2510.21254[cs.AI]. URL: http : / / arxiv. org / abs / 2510 . 21254 (visited on 05/27/2026)

    arXiv 2025

  74. [75]

    Humans and Automation: Use, Misuse, Disuse, Abuse

    Raja Parasuraman and Victor Riley. “Humans and Automation: Use, Misuse, Disuse, Abuse”. In:Human Factors39.2 (June 1, 1997). Publisher: SAGE Publi- cations Inc, pp. 230–253.ISSN: 0018-7208.DOI: 10. 1518/001872097778543886.URL: https://doi.org/10. 1518/001872097778543886 (visited on 05/27/2026)

    1997

  75. [76]

    2024 , isbn =

    Takuya Maeda and Anabel Quan-Haase. “When Human-AI Interactions Become Parasocial: Agency and Anthropomorphism in Affective Design”. In:Pro- ceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. FAccT ’24. New York, NY, USA: Association for Computing Machin- ery, June 5, 2024, pp. 1068–1077.ISBN: 979-8-4007- 0450-5.DOI: 10.114...

    work page doi:10.1145/3630106.3658956.url: 2024

  76. [77]

    Architecting princi- ples for systems-of-systems

    Mark W. Maier. “Architecting princi- ples for systems-of-systems”. In:Sys- tems Engineering1.4 (1998). eprint: https://incose.onlinelibrary.wiley.com/doi/pdf/10.1002/%28SICI%291520- 6858%281998%291%3A4%3C267%3A%3AAID- SYS3%3E3.0.CO%3B2-D, pp. 267–284.ISSN: 1520-6858.DOI: 10.1002/(SICI)1520- 6858(1998)1: 4⟨267 :: AID - SYS3⟩3 . 0 . CO ; 2 - D. (Visited on ...

    work page doi:10.1002/(sici)1520- 1998

  77. [79]

    Traceability for Trust- worthy AI: A Review of Models and Tools

    Marc ¸al Mora-Cantallops et al. “Traceability for Trust- worthy AI: A Review of Models and Tools”. In:Big Data and Cognitive Computing5.2 (June 2021). Pub- lisher: Multidisciplinary Digital Publishing Institute, p. 20.ISSN: 2504-2289.DOI: 10.3390/bdcc5020020. URL: https : / / www. mdpi . com / 2504 - 2289 / 5 / 2 / 20 (visited on 05/27/2026)

    work page doi:10.3390/bdcc5020020 2021

  78. [80]

    Towards Trustworthy and Understandable AI: Unraveling Explainability Strategies on Simplify- ing Algorithms, Appropriate Information Disclosure, and High-level Collaboration

    Shuren Yu. “Towards Trustworthy and Understandable AI: Unraveling Explainability Strategies on Simplify- ing Algorithms, Appropriate Information Disclosure, and High-level Collaboration”. In:26th International Academic Mindtrek Conference. Mindtrek ’23: 26th International Academic Mindtrek Conference. Tam- pere Finland: ACM, Oct. 3, 2023, pp. 133–143.ISBN...

    work page doi:10.1145/3616961.3616965 2023

  79. [81]

    Designing Emer- gence in Systems of Systems Using Information Streams

    Uriel Hochmann and Yoram Reich. “Designing Emer- gence in Systems of Systems Using Information Streams”. In:Proceedings of the Design Society3 (July 2023), pp. 1357–1366.ISSN: 2732-527X.DOI: 10.1017/pds.2023.136.URL: https://www.cambridge. org/core/product/identifier/S2732527X23001360/type/ journal article (visited on 05/27/2026)

    work page doi:10.1017/pds.2023.136.url: 2023

  80. [82]

    Learning from the Failure of Au- tonomous and Intelligent Systems: Accidents, Safety, and Sociotechnical Sources of Risk

    Carl Macrae. “Learning from the Failure of Au- tonomous and Intelligent Systems: Accidents, Safety, and Sociotechnical Sources of Risk”. In:Risk Analysis 42.9 (Sept. 2022), pp. 1999–2025.ISSN: 0272-4332, 1539-6924.DOI: 10 . 1111 / risa . 13850.URL: https : / / onlinelibrary. wiley. com / doi / 10 . 1111 / risa . 13850 (visited on 05/27/2026)

    2022

Showing first 80 references.