Designing for Accountable Agents: a Viewpoint
Pith reviewed 2026-05-10 17:01 UTC · model grok-4.3
The pith
Accountability concepts from multiple fields can be combined into a definition suitable for agents in multi-agent systems.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors survey accountability across disciplines and synthesize a coherent definition that enables agents in multi-agent systems, which may include humans, to be accountable to one another or to hold others accountable. Using a realistic MAS example, they illustrate the practical benefits of incorporating such accountability processes, and they outline research challenges for the community along with initial ideas for solutions to create accountable agents.
What carries the argument
The cross-disciplinary coherent definition of accountability applied to autonomous agents in open multi-agent systems.
If this is right
- Agents can participate in accountability processes within socio-technical systems.
- Benefits of accountability are shown in a realistic multi-agent application domain.
- Initial solutions are sketched for challenges in building accountable agents.
- Future research is guided by the outlined roadmap for enabling accountability in MAS.
Where Pith is reading between the lines
- Such a definition might be extended to evaluate accountability in single-agent AI systems as well.
- Practical implementations could link accountability to existing work on explainable AI and value alignment.
- Testing the definition in simulated open systems could uncover additional challenges not identified in the survey.
Load-bearing premise
That a single coherent definition of accountability can be extracted from disparate disciplines and applied effectively to artificial agents operating in open multi-agent systems.
What would settle it
If the realistic MAS example is implemented both with and without accountability processes and no measurable benefits in compliance, trust, or impact mitigation are observed, this would undermine the paper's illustration of benefits.
read the original abstract
AI systems are becoming increasingly complex, ubiquitous and autonomous, leading to increasing concerns about their impacts on individuals and society. In response, researchers have begun investigating how to ensure that the methods underlying AI decision-making are transparent and their decisions are explainable to people and conformant to human values and ethical principles. As part of this research thrust, the need for accountability within AI systems has been noted, but this notion has proven elusive to define; we aim to address this issue in the current paper. Unlike much recent work, we do not address accountability within the human organisational processes of developing and deploying AI; rather we consider what it would it mean for the agents within a multi-agent system (MAS), potentially including human agents, to be accountable to other agents or to have others accountable to them. In this work, we make the following contributions: we provide an in-depth survey of existing work on accountability in multiple disciplines, seeking to identify a coherent definition of the concept; we give a realistic example of a multi-agent system application domain that illustrates the benefits of enabling agents to follow accountability processes, and we identify a set of research challenges for the MAS community in building accountable agents, sketching out some initial solutions to these, thereby laying out a road-map for future research. Our focus is on laying the groundwork to enable autonomous elements within open socio-technical systems to take part in accountability processes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper surveys accountability concepts across multiple disciplines to derive a coherent definition applicable to agents (including human agents) in open multi-agent systems. It illustrates the benefits via a realistic MAS application example, identifies a set of research challenges for the MAS community, sketches initial solutions to them, and positions the work as laying groundwork for enabling accountability processes in autonomous socio-technical systems.
Significance. If the synthesized definition holds as coherent and the challenges are well-posed, the paper offers a useful interdisciplinary foundation and roadmap for accountability research in MAS and AI. Its focus on agent-level processes (rather than solely human organizational ones) and inclusion of an illustrative example plus initial directions provide concrete starting points that could guide subsequent technical work on transparent, value-conformant autonomous agents.
minor comments (2)
- Abstract: the summary of contributions would be strengthened by briefly indicating the core elements of the proposed coherent definition of accountability, rather than leaving it implicit until later sections.
- The example section: ensure the MAS scenario explicitly maps the accountability processes back to the surveyed definition to demonstrate applicability without selectivity.
Simulated Author's Rebuttal
We appreciate the referee's positive evaluation of our manuscript and the recommendation for minor revision. The summary accurately captures our goals in surveying accountability concepts from multiple disciplines to formulate a definition suitable for agents in open multi-agent systems, providing an application example, and outlining research challenges with initial solutions. We are pleased that the significance for the MAS and AI communities is recognized. Given that no specific major comments were listed, we will perform minor revisions to enhance the paper's presentation and clarity where appropriate.
Circularity Check
No significant circularity; survey-based viewpoint with external foundations
full rationale
The paper is explicitly positioned as a survey and roadmap contribution. It draws definitions and concepts from external literatures across multiple disciplines (law, philosophy, organizational science, etc.) to synthesize a coherent definition of accountability, illustrates it with a realistic MAS example, and sketches forward research challenges. No equations, formal derivations, fitted parameters, or predictions are present. No load-bearing steps reduce by construction to the paper's own inputs, self-citations, or prior author work. The central claim is integrative and conceptual rather than deductive, making the work self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Accountability concepts from multiple disciplines can be integrated into one coherent definition applicable to both human and artificial agents.
- domain assumption Autonomous agents in open socio-technical systems can and should participate in accountability processes.
Reference graph
Works this paper leans on
-
[1]
Melvin J. Dubnick. Accountability as a cultural keyword. In Bovens et al. [90]
-
[2]
Patricia Day and Rudolf Klein.Accountabilities: five public services, chapter 1: The career of a concept. Tavistock Publications, 1987
work page 1987
-
[3]
Jonathan Fox. The uncertain relationship between transparency and accountability.Development in Practice, 17(4-5):663–671, 2007
work page 2007
-
[4]
Marcel Kaba. NGO accountability: A conceptual review across the engaged disciplines.International Studies Review, 23(3):958–996, 2021
work page 2021
-
[5]
The global landscape of AI ethics guidelines.Nature Machine Intelligence, 1, 2019
Anna Jobin, Marcello Ienca, and Effy Vayena. The global landscape of AI ethics guidelines.Nature Machine Intelligence, 1, 2019
work page 2019
-
[6]
Towards formal definitions of blameworthiness, intention, and moral responsibility
Joseph Halpern and Max Kleiman-Weiner. Towards formal definitions of blameworthiness, intention, and moral responsibility. InProceedings of the AAAI Conference on Artificial Intelligence, 2018
work page 2018
-
[7]
Responsibility-aware strategic reasoning in probabilistic multi-agent systems
Chunyan Mu, Muhammad Najib, and Nir Oren. Responsibility-aware strategic reasoning in probabilistic multi-agent systems. In Toby Walsh, Julie Shah, and Zico Kolter, editors,Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI-25), pages 23258–23266. AAAI Press, 2025
work page 2025
-
[8]
Anticipating responsibility in multiagent plan- ning
Timothy Parker, Umberto Grandi, and Emiliano Lorini. Anticipating responsibility in multiagent plan- ning. InECAI 2023 - 26th European Conference on Artificial Intelligence, volume 372 ofFrontiers in Artificial Intelligence and Applications, pages 1859–1866. IOS Press, 2023
work page 2023
-
[9]
Stephen Cranefield, Nir Oren, and Wamberto W. Vasconcelos. Accountability for practical reasoning agents. In Marin Lujak, editor,Agreement Technologies, pages 33–48, Cham, 2019. Springer
work page 2019
-
[10]
‘Accountability’: An ever-expanding concept?Public Administration, 78(3):555–573, 2000
Mulgan Richard. ‘Accountability’: An ever-expanding concept?Public Administration, 78(3):555–573, 2000
work page 2000
-
[11]
Accountability: The core concept and its subtypes
Staffan I Lindberg. Accountability: The core concept and its subtypes. Working Paper 1, Africa Power and Politics Programme, Overseas Development Institute, 2009
work page 2009
-
[12]
Mark Bovens. Analysing and assessing accountability: A conceptual framework.European Law Journal, 13(4):447–468, 2007
work page 2007
-
[13]
James G. March and Johan P. Olsen.Democratic governance. Free Press, New York, N.Y ., 1995
work page 1995
-
[14]
Dorothea Greiling. Accountability and Trust. In Bovens et al. [90]
- [15]
-
[16]
Arie Hlachmi and Marc Holzer. Citizen participation and performance measurement: Operationalizing democracy through better accountability.Public Administration Quarterly, 34(3):378 – 399, 2010
work page 2010
-
[17]
Tim Miller. Explanation in artificial intelligence: Insights from the social sciences.Artificial Intelligence, 267:1–38, 2019
work page 2019
-
[18]
Conceptualizing accountability
Andreas Schedler. Conceptualizing accountability. In Andreas Schedler, Larry Diamond, and Marc F. Plattner, editors,The Self-Restraining State: Power and Accountability in New Democracies. Lynne Ri- enner Publishers, 1999
work page 1999
-
[19]
Mark Bovens, Thomas Schillemans, and Robert E. Goodin. Public accountability. In Bovens et al. [90]
-
[20]
Thomas Schillemans. The public accountability review: A meta-analysis of public accountability re- search in six academic disciplines. Working paper, Utrecht University School of Governance, https: //dspace.library.uu.nl/handle/1874/275784, 2013. 22 Stephen Cranefield, Nir Oren
work page 2013
-
[21]
Elizabeth D. Gibbons. Child rights and social accountability in the post-2015 world. Background paper, UNICEF Programme Division, 2014
work page 2015
-
[22]
E. J. Emanuel and L. L. Emanuel. What is accountability in health care?Annals of Internal Medicine, 124(2):229–239, 1996
work page 1996
-
[23]
Harvard University Press, 1981
Robert Nozick.Philosophical explanation. Harvard University Press, 1981
work page 1981
-
[24]
Melvin J. Dubnick. Accountability and ethics: Reconsidering the relationships.International Journal of Organization Theory & Behavior, 6(3):405–441, 2003
work page 2003
-
[25]
multiple account- abilities disorder
Jonathan G.S. Koppell. Pathologies of accountability: ICANN and the challenge of “multiple account- abilities disorder”.Public Administration Review, 65(1):94–108, 2005
work page 2005
-
[26]
Andrew Eshleman. Moral responsibility. In Edward N. Zalta, editor,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, winter 2016 edition, 2016
work page 2016
-
[27]
The relation between forward-looking and backward-looking responsibility
Ibo van de Poel. The relation between forward-looking and backward-looking responsibility. In Nicole A. Vincent, Ibo van de Poel, and Jeroen van den Hoven, editors,Moral Responsibility: Beyond Free Will and Determinism, pages 37–52. Springer Netherlands, 2011
work page 2011
-
[28]
Ain’t no one here but us social forces
Michael Davis. “Ain’t no one here but us social forces”: Constructing the professional responsibility of engineers.Science and Engineering Ethics, 18(1):13–34, 2012
work page 2012
-
[29]
Project Management Institute, 5th, 2013
PMI.Guide to the Project Management Body of Knowledge (PMBOK®Guide). Project Management Institute, 5th, 2013
work page 2013
-
[30]
J. Mike Jacka and Paulette J. Keller.Business Process Mapping: Improving Customer Satisfaction. Wiley, 2nd edition, 2009
work page 2009
-
[31]
Tiago De Lima, Lamb `er M. M. Royakkers, and Frank Dignum. Modeling the problem of many hands in organisations. InProceedings of the 19th European Conference on Artificial Intelligence, volume 215 of Frontiers in Artificial Intelligence and Applications, pages 79–84. IOS Press, 2010
work page 2010
-
[32]
Characterising responsibility in organisational structures: The problem of many hands
Rosja Mastop. Characterising responsibility in organisational structures: The problem of many hands. InDeontic Logic in Computer Science, 10th International Conference, DEON 2010, Fiesole, Italy, July 7-9, 2010. Proceedings, volume 6181 ofLecture Notes in Computer Science, pages 274–287. Springer, 2010
work page 2010
-
[33]
Organizational structure and responsibility
Davide Grossi, Lamb `er Royakkers, and Frank Dignum. Organizational structure and responsibility. Artificial Intelligence and Law, 15(3):223–249, 2007
work page 2007
-
[34]
Neelke Doorn and Ibo van de Poel. Editors’ overview: Moral responsibility in technology and engineer- ing.Science and Engineering Ethics, 18(1):1–11, 2012
work page 2012
-
[35]
Shaver.The Attribution of Blame: Causality, Responsibility, and Blameworthiness
Kelly G. Shaver.The Attribution of Blame: Causality, Responsibility, and Blameworthiness. Springer New York, 1985
work page 1985
- [36]
-
[37]
A. K. Chopra and M. P. Singh. The thing itself speaks: Accountability as a foundation for requirements in sociotechnical systems. In2014 IEEE 7th International Workshop on Requirements Engineering and Law, page 22. IEEE, 2014
work page 2014
-
[38]
Amit K. Chopra and Munindar P. Singh. From social machines to social protocols: Software engineering foundations for sociotechnical systems. InProceedings of the 25th International Conference on World Wide Web, WWW 2016, pages 903–914. ACM, 2016
work page 2016
-
[39]
Mehdi Dastani, Leendert van der Torre, and Neil Yorke-Smith. Commitments and interaction norms in organisations.Autonomous Agents and Multi-Agent Systems, 31(2):207–249, Mar 2017
work page 2017
-
[40]
Nicoletta Fornara and Marco Colombetti. Representation and monitoring of commitments and norms using OWL.AI Communications, 23(4):341–356, December 2010
work page 2010
-
[41]
May, Roberto Micalizio, and Stefano Tedeschi
Matteo Baldoni, Cristina Baroglio, Katherine M. May, Roberto Micalizio, and Stefano Tedeschi. Compu- tational accountability. InProceedings of the AI*IA Workshop on Deep Understanding and Reasoning: A Challenge for Next-generation Intelligent Agents, volume 1802 ofCEUR Workshop Proceedings, pages 56–62. CEUR-WS.org, 2017
work page 2017
-
[42]
May, Roberto Micalizio, and Stefano Tedeschi
Matteo Baldoni, Cristina Baroglio, Katherine M. May, Roberto Micalizio, and Stefano Tedeschi. ADOPT JaCaMo: Accountability-driven organization programming technique for JaCaMo. InPRIMA 2017: Principles and Practice of Multi-Agent Systems, pages 295–312. Springer, 2017
work page 2017
-
[43]
The AThOS project: First steps towards computational accountability
Matteo Baldoni, Cristina Baroglio, and Roberto Micalizio. The AThOS project: First steps towards computational accountability. InProceedings of the 1st Workshop on Computational Accountability and Responsibility in Multiagent Systems, volume 2051 ofCEUR Workshop Proceedings, pages 3–19. CEUR-WS.org, 2018
work page 2051
-
[44]
Matteo Baldoni, Cristina Baroglio, Roberto Micalizio, and Stefano Tedeschi. Accountability in multi- agent organizations: from conceptual design to agent programming.Autonomous Agents and Multi-Agent Systems, 37(1):7, 2022
work page 2022
-
[45]
Davide Grossi, Frank Dignum, Lamb `er M. M. Royakkers, and John-Jules Ch. Meyer. Collective obliga- tions and agents: Who gets the blame? In Alessio Lomuscio and Donald Nute, editors,Deontic Logic in Computer Science, pages 129–145. Springer, 2004. Designing for Accountable Agents: a Viewpoint 23
work page 2004
-
[46]
On-line monitoring and diagnosis of multi- agent systems: A model based approach
Roberto Micalizio, Pietro Torasso, and Gianluca Torta. On-line monitoring and diagnosis of multi- agent systems: A model based approach. InProceedings of the 16th European Conference on Artificial Intelligence, pages 848–852. IOS Press, 2004
work page 2004
-
[47]
Diagnosis of single and multi-agent plans
Cees Witteveen, Nico Roos, Roman van der Krogt, and Mathijs de Weerdt. Diagnosis of single and multi-agent plans. InProceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, pages 805–812. ACM, 2005
work page 2005
-
[48]
Femke de Jonge, Nico Roos, and Cees Witteveen. Primary and secondary diagnosis of multi-agent plan execution.Autonomous Agents and Multi-Agent Systems, 18(2):267–294, 2009
work page 2009
-
[49]
Coalitional responsibility in strategic settings
Nils Bulling and Mehdi Dastani. Coalitional responsibility in strategic settings. In Jo ˜ao Leite, Tran Cao Son, Paolo Torroni, Leon van der Torre, and Stefan Woltran, editors,Computational Logic in Multi-Agent Systems, pages 172–189. Springer, 2013
work page 2013
-
[50]
Roberto Micalizio and Pietro Torasso. Cooperative monitoring to diagnose multiagent plans.Journal of Artificial Intelligence Research, 51:1–70, 2014
work page 2014
-
[51]
Emiliano Lorini, Dominique Longin, and Eunate Mayor. A logical analysis of responsibility attribution: emotions, individuals and collectives.Journal of Logic and Computation, 24(6):1313–1339, 2014
work page 2014
-
[52]
Huib Aldewereld, Virginia Dignum, and Wamberto W. Vasconcelos. Group norms for multi-agent or- ganisations.ACM Transactions on Autonomous and Adaptive Systems, 11(2):15:1–15:31, 2016
work page 2016
-
[53]
Natasha Alechina, Joseph Y . Halpern, and Brian Logan. Causality, responsibility and blame in team plans. InProceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems, pages 1091–1099. IFAAMAS, 2017
work page 2017
-
[54]
Virginia Dignum. Responsible artificial intelligence: designing AI for human values.ITU journal: ICT Discoveries, 1(1):1–8, 2018
work page 2018
-
[55]
Towards trusting autonomous systems
Michael Winikoff. Towards trusting autonomous systems. InEngineering Multi-Agent Systems, number 10738 in Lecture Notes in Computer Science, pages 3–20. Springer, 2018
work page 2018
-
[56]
Robustness based on ac- countability in multiagent organizations
Matteo Baldoni, Cristina Baroglio, Roberto Micalizio, and Stefano Tedeschi. Robustness based on ac- countability in multiagent organizations. InProceedings of the 20th International Conference on Au- tonomous Agents and Multiagent Systems, pages 142–150. IFAAMAS, 2021
work page 2021
-
[57]
History and philosophy of Goal Programming
Dylan Jones and Mehrdad Tamiz. History and philosophy of Goal Programming. In Jones and Tamiz [74]
-
[58]
Herbert Alexander Simon.Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting. John Wiley & Sons, 1957
work page 1957
-
[59]
State University of New York Press, 1995
Douglas Walton and Erik CW Krabbe.Commitment in dialogue: Basic concepts of interpersonal rea- soning. State University of New York Press, 1995
work page 1995
-
[60]
Stefan Poslad. Specifying protocols for multi-agent systems interaction.ACM Transactions on Au- tonomous and Adaptive Systems, 2(4):15–es, November 2007
work page 2007
-
[61]
https://jade.io/ article/1013981, 2012
Roberts v Professional Conduct Committee of the Nursing Council of New Zealand. https://jade.io/ article/1013981, 2012
-
[62]
On social laws for artificial agent societies: off-line design
Yoav Shoham and Moshe Tennenholtz. On social laws for artificial agent societies: off-line design. Artificial Intelligence, 73(1):231–252, 1995. Computational Research on Interaction and Agency, Part 2
work page 1995
-
[63]
Rodriguez-Aguilar, Wamberto Vasconcelos, and Michael Wooldridge
Javier Morales, Maite L ´opez-S´anchez, Juan A. Rodriguez-Aguilar, Wamberto Vasconcelos, and Michael Wooldridge. Online automated synthesis of compact normative systems.ACM Transactions on Au- tonomous Adaptive Systems, 10(1), March 2015
work page 2015
-
[64]
NEMAS: norm entrepreneurship in multi-agent systems.Systems, 12(6):187, 2024
Amritha Menon Anavankot, Stephen Cranefield, and Bastin Tony Roy Savarimuthu. NEMAS: norm entrepreneurship in multi-agent systems.Systems, 12(6):187, 2024
work page 2024
-
[65]
Maintenance goals in intelligent agents.Computa- tional Intelligence, 30(1):71–114, February 2014
Simon Duff, John Thangarajah, and James Harland. Maintenance goals in intelligent agents.Computa- tional Intelligence, 30(1):71–114, February 2014
work page 2014
-
[66]
Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2004
Dana Nau, Malik Ghallab, and Paolo Traverso.Automated Planning: Theory & Practice. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2004
work page 2004
-
[67]
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Shixiang Gu, Ethan Holly, Timothy Lillicrap, and Sergey Levine. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In2017 IEEE International Conference on Robotics and Automation (ICRA), page 3389–3396. IEEE Press, 2017
work page 2017
-
[68]
Kanghyun Ryu, Qiayuan Liao, Zhongyu Li, Payam Delgosha, Koushil Sreenath, and Negar Mehr. Cur- ricuLLM: Automatic task curricula design for learning complex robot skills using large language models, 2025
work page 2025
-
[69]
TAC a trading agent competition.ACM SIGecom Exchanges, 1(1):26–28, June 2000
Betsy Strother. TAC a trading agent competition.ACM SIGecom Exchanges, 1(1):26–28, June 2000
work page 2000
-
[70]
The intention progression competition
Simon Castle-Green, Alexi Dewfall, and Brian Logan. The intention progression competition. In Cristina Baroglio, Jomi Fred H ¨ubner, and Michael Winikoff, editors,Engineering Multi-Agent Systems - 8th International Workshop, EMAS 2020, volume 12589 ofLecture Notes in Computer Science, pages 144–
work page 2020
-
[71]
The 2023 International Planning Competition.AI Magazine, 45(2):280–296, June 2024
Ayal Taitler, Ron Alford, Joan Espasa, Gregor Behnke, Daniel Fi ˇser, Michael Gimelfarb, Florian Pom- merening, Scott Sanner, Enrico Scala, Dominik Schreiber, Javier Segovia-Aguas, and Jendrik Seipp. The 2023 International Planning Competition.AI Magazine, 45(2):280–296, June 2024. 24 Stephen Cranefield, Nir Oren
work page 2023
-
[72]
Tobias Ahlbrecht, J ¨urgen Dix, Niklas Fiekas, and Tabajara Krausburg, editors.The Multi-Agent Pro- gramming Contest 2022, volume 13997 ofLecture Notes in Computer Science. Springer, 2023
work page 2022
-
[73]
Matti J ¨arvisalo, Tuomo Lehtonen, and Andreas Niskanen. ICCMA 2023: 5th international competition on computational models of argumentation.Artificial Intelligence, 342:104311, 2025
work page 2023
-
[74]
Dylan Jones and Mehrdad Tamiz, editors.Practical Goal Programming. Springer US, Boston, MA, 2010
work page 2010
-
[75]
Melvyn Sim, Qinshen Tang, Minglong Zhou, and Taozeng Zhu. The analytics of robust satisficing: Predict, optimize, satisfice, then fortify.Operations Research, 73(5), 2024
work page 2024
-
[76]
Senthil Rajasekaran, Suguman Bansal, and Moshe Y . Vardi. Multi-agent systems with quantitative sat- isficing goals. In Edith Elkind, editor,Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 280–288. International Joint Conferences on Artificial Intelli- gence Organization, 8 2023
work page 2023
-
[77]
Yoav Shoham and Moshe Tennenholtz. On the emergence of social conventions: Modeling, analysis, and simulations.Artificial Intelligence, 94(1-2):139–166, 1997
work page 1997
-
[78]
Rosaria Conte, Rino Falcone, and Giovanni Sartor. Introduction: Agents and norms: How to fill the gap? Artificial Intelligence and Law, 7(1):1–15, 1999
work page 1999
-
[79]
Kluwer Academic Publishers, 2001
Rosaria Conte and Chrysanthos Dellarocas, editors.Social order in multiagent systems, volume 2 of Multiagent Systems, Artificial Societies, and Simulated Organizations. Kluwer Academic Publishers, 2001
work page 2001
-
[80]
Guido Boella, Leendert W. N. van der Torre, and Harko Verhagen. Introduction to normative multiagent systems.Computational & Mathematical Organization Theory, 12(2-3):71–79, 2006
work page 2006
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.