pith. sign in

arxiv: 2606.07211 · v1 · pith:IJ5UEUEKnew · submitted 2026-06-05 · 💻 cs.RO · cs.AI

An Abstract Architecture for Explainable Autonomy in Hazardous Environments

Matt Luckcuck , Hazel M Taylor , Marie Farrell This is my paper

Pith reviewed 2026-06-27 21:44 UTC · model grok-4.3

classification 💻 cs.RO cs.AI
keywords explainable autonomyautonomous systemshazardous environmentscivil nuclear industrysystem architecturetrustrobotics
0
0 comments X

The pith

An abstract architecture provides a template for autonomous systems to explain their behavior in hazardous environments.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes an abstract architecture to enable explainable autonomy in robotic systems operating in dangerous settings. This design incorporates explainability from the start, similar to safety features, to foster trust from human users and regulators. A worked example applies it to the civil nuclear industry, where decision-making transparency is crucial. The architecture acts as a reusable template for developing such systems.

Core claim

The authors present an abstract architecture that supports an autonomous system explaining its behaviour, providing a design template for implementing explainable autonomous systems, illustrated with an application in the civil nuclear industry.

What carries the argument

The abstract architecture for explainable autonomy, which integrates explanation capabilities into the system's design.

If this is right

  • Autonomous systems built with the architecture can generate explanations of their decisions during operation.
  • The template can be applied to specific hazardous domains such as civil nuclear work to support human oversight.
  • Explainability becomes a core design property rather than an after-the-fact addition.
  • Regulators and workers gain a structured way to understand the system's decision-making.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The architecture could inform certification standards that require built-in explanation mechanisms for autonomous systems in regulated industries.
  • Similar templates might extend to other high-risk autonomous applications such as search-and-rescue or space robotics.
  • Practical validation would involve implementing the architecture on a real robot platform and measuring explanation quality against operator feedback.

Load-bearing premise

That designing explainability into the system from the start will meaningfully increase trust among users and regulators in hazardous environments.

What would settle it

User studies or field deployments in the civil nuclear industry showing no measurable increase in trust or acceptance for systems built with the architecture compared to those without explicit explainability features.

Figures

Figures reproduced from arXiv: 2606.07211 by Hazel M Taylor, Marie Farrell, Matt Luckcuck.

Figure 1
Figure 1. Figure 1: High-Level view of our abstract architecture. The squares are system modules (which may [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
read the original abstract

Autonomous robotic systems are being proposed for use in hazardous environments, often to reduce the risks to human workers. In the immediate future, it is likely that human workers will continue to use and direct these autonomous robots, much like other computerised tools but with more sophisticated decision-making. Therefore, one important area on which to focus engineering effort is ensuring that these users trust the system. Recent literature suggests that explainability is closely related to how trustworthy a system is. Like safety and security properties, explainability should be designed into a system, instead of being added afterwards. This paper presents an abstract architecture that supports an autonomous system explaining its behaviour (explainable autonomy), providing a design template for implementing explainable autonomous systems. We present a worked example of how our architecture could be applied in the civil nuclear industry, where both workers and regulators need to trust the system's decision-making capabilities.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper presents an abstract architecture for explainable autonomy in autonomous robotic systems for hazardous environments. Motivated by the link between explainability and trust (drawn from prior literature), it positions explainability as a property to be designed in from the start and offers the architecture as a design template, illustrated via a worked example in the civil nuclear industry where both operators and regulators require trust in decision-making.

Significance. If the architecture is realized as described, the work supplies a high-level conceptual template that could inform the engineering of explainable autonomous systems in safety-critical domains. Its value lies in synthesizing existing ideas on trust and explainability into an architectural form rather than in new empirical results or formal proofs; the absence of component-level specifications or validation limits immediate applicability.

major comments (1)
  1. [Abstract] Abstract and introduction: the central claim that the proposed architecture 'supports an autonomous system explaining its behaviour' and provides a 'design template' rests entirely on assertion; no component definitions, interfaces, or mechanisms for generating explanations are supplied, so the reader cannot evaluate whether the template is usable or novel.
minor comments (1)
  1. The worked example in the civil nuclear industry is referenced but not expanded with concrete mappings from architecture elements to nuclear-specific requirements or regulatory standards.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and constructive comments. We address the major comment point-by-point below.

read point-by-point responses

  1. Referee: [Abstract] Abstract and introduction: the central claim that the proposed architecture 'supports an autonomous system explaining its behaviour' and provides a 'design template' rests entirely on assertion; no component definitions, interfaces, or mechanisms for generating explanations are supplied, so the reader cannot evaluate whether the template is usable or novel.

    Authors: The manuscript is explicitly positioned as presenting an abstract architecture (see title and abstract), intended as a high-level conceptual template rather than a detailed engineering specification. Component roles are defined conceptually in the architecture description (e.g., the separation of decision-making, monitoring, and explanation modules), with interfaces shown via the data-flow diagram and illustrated through the nuclear industry worked example. The mechanism for explanations is the dedicated explanation module that draws on internal state representations to produce human-readable outputs. This level of abstraction aligns with the paper's contribution of synthesising trust and explainability literature into an architectural form for safety-critical domains; more granular component specifications would be appropriate for a follow-on implementation paper. revision: no

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is a high-level design proposal for an abstract architecture supporting explainable autonomy. It contains no equations, fitted parameters, predictions, or derivations. Motivational claims about trust and explainability are attributed to external recent literature rather than derived internally or via self-citation chains. The central contribution is a descriptive template with a worked example; this does not reduce to any self-referential input by construction. No load-bearing steps match the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim depends on the domain assumption that explainability directly supports trustworthiness and that this property can be engineered into autonomous systems at the architectural level; no free parameters or invented entities are introduced.

axioms (1)
  • domain assumption Explainability is closely related to how trustworthy a system is.
    Invoked in the opening paragraph of the abstract as the basis for focusing engineering effort on explainability.

pith-pipeline@v0.9.1-grok · 5676 in / 1218 out tokens · 19090 ms · 2026-06-27T21:44:20.241421+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references

  1. [1]

    Design of a Robotic Software Manager for using Heterogeneous Components of Different Frameworks.J

    Ho Seok Ahn, Min Ho Lee, and Bruce MacDonald. Design of a Robotic Software Manager for using Heterogeneous Components of Different Frameworks.J. Softw. Eng. Robot., 8(1):45–64, 2017

    2017

  2. [2]

    Baddeley and Graham Hitch

    Alan D. Baddeley and Graham Hitch. Working memory. volume 8 ofPsychology of Learning and Motivation, pages 47–89. Academic Press, 1974

    1974

  3. [3]

    Transparency and Explainability of AI Systems: Ethical Guidelines in Practice

    Nagadivya Balasubramaniam, Marjo Kauppinen, Kari Hiekkanen, and Sari Kujala. Transparency and Explainability of AI Systems: Ethical Guidelines in Practice. In Vincenzo Gervasi and Andreas Vogelsang, editors,Requirements Engineering: Foundation for Software Quality, volume 13216, pages 3–18. Springer, 2022. Series Title: Lecture Notes in Computer Science

    2022

  4. [4]

    M. E. Bratman.Intentions, Plans, and Practical Reason. Harvard University Press, 1987

    1987

  5. [5]

    BS 8611:2016 - Robots and robotic devices

    British Standards Institution. BS 8611:2016 - Robots and robotic devices. Guide to the ethical design and application of robots and robotic systems, 2016. 6

    2016

  6. [6]

    Do you get it? user-evaluated explainable bdi agents

    Joost Broekens, Maaike Harbers, Koen Hindriks, Karel van den Bosch, Catholijn Jonker, and John- Jules Meyer. Do you get it? user-evaluated explainable bdi agents. In Jürgen Dix and Cees Witteveen, editors,Multiagent System Technologies, pages 28–39. Springer, 2010

    2010

  7. [7]

    Quo Vadis, Explainability? – A Research Roadmap for Explainability Engineering

    Wasja Brunotte, Larissa Chazette, Verena Klös, and Timo Speith. Quo Vadis, Explainability? – A Research Roadmap for Explainability Engineering. In Vincenzo Gervasi and Andreas Vogelsang, editors,Requirements Engineering: Foundation for Software Quality, volume 13216, pages 26–32. Springer, 2022. Series Title: Lecture Notes in Computer Science

    2022

  8. [8]

    Cardoso, Louise A

    Rafael C. Cardoso, Louise A. Dennis, Marie Farrell, Michael Fisher, and Matt Luckcuck. Towards Compositional Verification for Modular Robotic Systems. InProceedings 2nd Workshop on Formal Methods for Autonomous Systems (FMAS) [Virtual], volume 329 ofEPTCS, pages 15–22, 2020

    2020

  9. [9]

    Heteroge- neous verification of an autonomous curiosity rover

    Rafael C Cardoso, Marie Farrell, Matt Luckcuck, Angelo Ferrando, and Michael Fisher. Heteroge- neous verification of an autonomous curiosity rover. InNASA Formal Methods Symposium, pages 353–360. Springer, 2020

    2020

  10. [10]

    Towards Explainable Formal Methods: From LTL to Natural Language with Neural Machine Translation

    Himaja Cherukuri, Alessio Ferrari, and Paola Spoletini. Towards Explainable Formal Methods: From LTL to Natural Language with Neural Machine Translation. In Vincenzo Gervasi and Andreas Vogel- sang, editors,Requirements Engineering: Foundation for Software Quality, volume 13216, pages 79–86. Springer, 2022. Series Title: Lecture Notes in Computer Science

    2022

  11. [11]

    Scenario-Based Require- ments Elicitation for User-Centric Explainable AI

    Douglas Cirqueira, Dietmar Nedbal, Markus Helfert, and Marija Bezbradica. Scenario-Based Require- ments Elicitation for User-Centric Explainable AI. In Andreas Holzinger, Peter Kieseberg, A Min Tjoa, and Edgar Weippl, editors,Machine Learning and Knowledge Extraction, pages 321–341. Springer, 2020

    2020

  12. [12]

    The mcapl framework including the agent infrastructure layer and agent java pathfinder.The Journal of Open Source Software, 2018

    Louise A Dennis. The mcapl framework including the agent infrastructure layer and agent java pathfinder.The Journal of Open Source Software, 2018

    2018

  13. [13]

    Gwendolen: A bdi language for verifiable agents

    Louise A Dennis and Berndt Farwer. Gwendolen: A bdi language for verifiable agents. InProceedings of the AISB 2008 Symposium on Logic and the Simulation of Interaction and Reasoning, Society for the Study of Artificial Intelligence and Simulation of Behaviour, pages 16–23, 2008

    2008

  14. [14]

    Louise Abigail Dennis, Michael Fisher, and Alan F. T. Winfield. Towards verifiably ethical robot be- haviour. In Toby Walsh, editor,Artificial Intelligence and Ethics, Papers from the 2015 AAAI Workshop, Austin, Texas, USA, January 25, 2015, volume WS-15-02 ofAAAI Technical Report. AAAI Press, 2015

    2015

  15. [15]

    Cardoso, Emily C

    Michael Fisher, Rafael C. Cardoso, Emily C. Collins, Christopher Dadswell, Louise A. Dennis, Clare Dixon, Marie Farrell, Angelo Ferrando, Xiaowei Huang, Mike Jump, Georgios Kourtis, Alexei Lisitsa, Matt Luckcuck, Shan Luo, Vincent Page, Fabio Papacchini, and Matt Webster. An overview of veri- fication and validation challenges for inspection robots.Roboti...

    2021

  16. [16]

    Gilpin, David Bau, Ben Z

    Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explain- ing Explanations: An Overview of Interpretability of Machine Learning. In2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pages 80–89, Turin, Italy, October 2018. IEEE

    2018

  17. [17]

    A systematic method to understand requirements for explainable ai (xai) systems

    Mark Hall, Daniel Harborne, Richard Tomsett, Vedran Galetic, Santiago Quintana-Amate, Alistair Nottle, and Alun Preece. A systematic method to understand requirements for explainable ai (xai) systems. InProceedings of the IJCAI Workshop on eXplainable Artificial Intelligence (XAI 2019), Macau, China, volume 11, 2019

    2019

  18. [18]

    Hindriks

    Koen V. Hindriks. Debugging is explaining. In Iyad Rahwan, Wayne Wobcke, Sandip Sen, and Toshiharu Sugawara, editors,PRIMA 2012: Principles and Practice of Multi-Agent Systems, pages 31– 45, Berlin, Heidelberg, 2012. Springer

    2012

  19. [19]

    Why Did You Do That?

    Vincent J. Koeman, Louise A. Dennis, Matt Webster, Michael Fisher, and Koen Hindriks. The “Why Did You Do That?” Button: Answering Why-Questions for End Users of Robotic Systems. In Louise A. Dennis, Rafael H. Bordini, and Yves Lespérance, editors,Engineering Multi-Agent Systems, volume 12058, pages 152–172. Springer, 2020. Series Title: Lecture Notes in ...

    2020

  20. [20]

    Workshop Report: Scoping Certifiable Autonomous Nuclear Robotics

    Matt Luckcuck. Workshop Report: Scoping Certifiable Autonomous Nuclear Robotics. Technical Report ULCS-20-001, University of Liverpool, 2020

    2020

  21. [21]

    Matt Luckcuck. Using Formal Methods for Autonomous Systems: Five Recipes for Formal Verifi- cation.Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 2021

    2021

  22. [22]

    Formal Specification and Verification of Autonomous Robotic Systems: A Survey.ACM Computing Surveys, 52(5):1–41, 2019

    Matt Luckcuck, Marie Farrell, Louise A Dennis, Clare Dixon, and Michael Fisher. Formal Specification and Verification of Autonomous Robotic Systems: A Survey.ACM Computing Surveys, 52(5):1–41, 2019

    2019

  23. [24]

    Principles for the Development and Assurance of Autonomous Systems for Safe Use in Hazardous Environ- ments

    Luckcuck, Matt, Michael Fisher, Louise Dennis, Steve Frost, Andy White, and Doug Styles. Principles for the Development and Assurance of Autonomous Systems for Safe Use in Hazardous Environ- ments. Technical report, Zenodo, June 2021

    2021

  24. [25]

    Explanation in artificial intelligence: Insights from the social sciences.Artificial Intelli- gence, 267:1–38, 2019

    Tim Miller. Explanation in artificial intelligence: Insights from the social sciences.Artificial Intelli- gence, 267:1–38, 2019

    2019

  25. [26]

    A method to deal with social bias and desirability in ethical requirements

    Claudia Negri-Ribalta. A method to deal with social bias and desirability in ethical requirements. Joint Proceedings of REFSQ-2022 Workshops, Doctoral Symposium, and Poster & Tools Track, 2022

    2022

  26. [27]

    ROS: an open-source Robot Operating System

    Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Ng. ROS: an open-source Robot Operating System. InWorkshop on Open Source Software. IEEE, 2009

    2009

  27. [28]

    A. S. Rao and M. Georgeff. BDI Agents: From Theory to Practice. InProc. First International Conference on Multi-Agent Systems (ICMAS), pages 312–319, San Francisco, USA, 1995

    1995

  28. [29]

    Component models in robotics software

    Azamat Shakhimardanov, Nico Hochgeschwender, and Gerhard Kraetzschmar. Component models in robotics software. InPerform. Metrics Intell. Syst., page 82. ACM, 2010

    2010

  29. [30]

    Effects of a Social Robot’s Self-Explanations on How Humans Un- derstand and Evaluate Its Behavior

    Sonja Stange and Stefan Kopp. Effects of a Social Robot’s Self-Explanations on How Humans Un- derstand and Evaluate Its Behavior. InProceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, pages 619–627, Cambridge United Kingdom, March 2020. ACM

    2020

  30. [31]

    Should ai systems in nuclear facilities explain decisions the way humans do? an interview study

    Hazel M Taylor, Caroline Jay, Barry Lennox, Angelo Cangelosi, and Louise Dennis. Should ai systems in nuclear facilities explain decisions the way humans do? an interview study. InInternational Conference on Robot and Human Interactive Communication (RO-MAN 2022). IEEE, 2022 (In Press)

    2022

  31. [32]

    Interpretable to whom? a role-based model for analyzing interpretable machine learning systems

    Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, and Supriyo Chakraborty. Interpretable to whom? a role-based model for analyzing interpretable machine learning systems. InProceedings of the ICML Workshop on Human Interpretability in Machine Learning, 2018

    2018

  32. [33]

    Alan F. T. Winfield and Marina Jirotka. The case for an ethical black box. In Yang Gao, Saber Fallah, Yaochu Jin, and Constantina Lekakou, editors,Towards Autonomous Robotic Systems, pages 262–273. Springer, 2017

    2017

  33. [34]

    Debugging agent programs with why?: Questions

    Michael Winikoff. Debugging agent programs with why?: Questions. In Kate Larson, Michael Winikoff, Sanmay Das, and Edmund H. Durfee, editors,Proceedings of the 16th Conference on Au- tonomous Agents and MultiAgent Systems, AAMAS 2017, São Paulo, Brazil, May 8-12, 2017, pages 251–259. ACM, 2017

    2017

  34. [35]

    Turton, and Paul Bremner

    Katie Winkle, Praminda Caleb-Solly, Ailie J. Turton, and Paul Bremner. Mutual shaping in the design of socially assistive robots: A case study on social robots for therapy.Int. J. Soc. Robotics, 12(4):847– 866, 2020. 8

    2020

  35. [36]

    Christine T. Wolf. Explainability Scenarios: Towards Scenario-Based XAI Design. IUI ’19, New York, NY, USA, 2019. Association for Computing Machinery

    2019

  36. [37]

    Simulating ionising radiation in gazebo for robotic nuclear inspection challenges.Robotics, 10(3), 2021

    Thomas Wright, Andrew West, Mauro Licata, Nick Hawes, and Barry Lennox. Simulating ionising radiation in gazebo for robotic nuclear inspection challenges.Robotics, 10(3), 2021. 9

    2021