pith. machine review for the scientific record. sign in

arxiv: 2603.16651 · v1 · submitted 2026-03-17 · 💻 cs.AI · cs.MA

Recognition: 2 theorem links

· Lean Theorem

What if Pinocchio Were a Reinforcement Learning Agent: A Normative End-to-End Pipeline

Beno\^it Alcaraz
Authors on Pith no claims yet

Pith reviewed 2026-05-15 09:56 UTC · model grok-4.3

classification 💻 cs.AI cs.MA
keywords normative agentsreinforcement learningargumentationnorm compliancenorm avoidancehybrid modelscontext-aware agents
0
0 comments X

The pith

A hybrid model supervises reinforcement learning agents with argumentation-based advisors to enforce norms and context awareness.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds an end-to-end pipeline that lets reinforcement learning agents follow societal rules by adding supervision from argumentation-based normative advisors. It presents Pino as the hybrid architecture and supplies a new algorithm that automatically extracts the arguments and relations the advisors rely on. The work also defines norm avoidance in these agents and supplies a mitigation approach. If the pipeline holds, agents could operate safely in everyday settings by staying rule-compliant without losing their ability to adapt to context. Each stage receives empirical tests.

Core claim

The central claim is that reinforcement learning agents become norm-compliant and context-aware when placed under the supervision of argumentation-based normative advisors inside the Pino hybrid model. A dedicated extraction algorithm turns the advisors into working components by pulling out the underlying arguments and relationships. The pipeline further supplies a definition of norm avoidance together with a concrete mitigation strategy, and every element is checked through experiments.

What carries the argument

The Pino hybrid model, in which an argumentation-based normative advisor supervises a reinforcement learning agent.

If this is right

  • The automatic extraction algorithm makes the normative advisors practical to deploy.
  • Norm avoidance receives both a definition and a workable reduction method inside reinforcement learning.
  • Agents gain the capacity to comply with rules while remaining responsive to changing contexts.
  • The full pipeline offers a route toward agents that can be integrated into daily human environments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same supervisory structure could be tested on non-reinforcement learning methods to check whether the benefit is specific to RL.
  • Real-world deployment scenarios such as autonomous navigation or resource allocation could serve as direct test beds for the pipeline.
  • Future extensions might allow the advisors themselves to be updated from observed violations rather than remaining fixed.

Load-bearing premise

Argumentation-based normative advisors can supervise reinforcement learning agents effectively across contexts without creating inconsistencies or harming task performance.

What would settle it

Experiments in which the supervised agents continue to violate the target norms at rates comparable to unsupervised baselines or show clear drops in learning speed and final performance.

Figures

Figures reproduced from arXiv: 2603.16651 by Beno\^it Alcaraz.

Figure 2.1
Figure 2.1. Figure 2.1: Example of an MDP. where γ ∈ [0, 1] is the discount factor, which determines how much emphasis is placed on the current reward versus future rewards, and n is the number of steps in the agent’s path. Qπ ∗ , then, gives the expected (discounted) sum of rewards over the agent’s runtime, assuming that it takes action a in state s, and thereafter takes a path, i.e., a sequence of state-action pairs, followin… view at source ↗
Figure 2.2
Figure 2.2. Figure 2.2: Reinforcement Learning training loop. 14 [PITH_FULL_IMAGE:figures/full_fig_p027_2_2.png] view at source ↗
Figure 2.3
Figure 2.3. Figure 2.3: Example of a Labelled MDP. situational contexts [128]. They can also change and evolve over time [39]. Deontic logic is an area of logic that investigates normative concepts, i.e., deontic con￾cepts [84]. It aims at developing tools and methods to represent these norms and reason about them. The deontic logic literature defines two types of norm: constitutive norms and regulative norms. The former can be… view at source ↗
Figure 2.4
Figure 2.4. Figure 2.4: Representation as a directed graph of an argumentation framework. [PITH_FULL_IMAGE:figures/full_fig_p032_2_4.png] view at source ↗
Figure 2.5
Figure 2.5. Figure 2.5: Representation of the AJAR framework. pros = [PITH_FULL_IMAGE:figures/full_fig_p037_2_5.png] view at source ↗
Figure 2.6
Figure 2.6. Figure 2.6: Jiminy’s smart home example. Reused from Liao et al. [112]. [PITH_FULL_IMAGE:figures/full_fig_p038_2_6.png] view at source ↗
Figure 2.7
Figure 2.7. Figure 2.7: Representation of the Jiminy pipeline. Reused from Liao et al. [111]. [PITH_FULL_IMAGE:figures/full_fig_p039_2_7.png] view at source ↗
Figure 3.1
Figure 3.1. Figure 3.1: Example of graph generated by two avatars. [PITH_FULL_IMAGE:figures/full_fig_p051_3_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: is a diagram of a standard RL training loop. In comparison, a diagram of [PITH_FULL_IMAGE:figures/full_fig_p051_3.png] view at source ↗
Figure 3.2
Figure 3.2. Figure 3.2: Diagram of a standard reinforcement learning training loop. [PITH_FULL_IMAGE:figures/full_fig_p052_3_2.png] view at source ↗
Figure 3.3
Figure 3.3. Figure 3.3: Diagram of the π-NoCCHIO training loop. understand the situation or wish to deviate from the avatars’ guidance. On the other hand, it potentially makes the system more vulnerable to cyberattacks, as one could potentially breach it to modify the status of the norms.5 However, this can be partially solved by splitting the norms into two sets. One set in which the norms can be manually edited (this set cont… view at source ↗
Figure 3.4
Figure 3.4. Figure 3.4: Diagram of the judge component. 42 [PITH_FULL_IMAGE:figures/full_fig_p055_3_4.png] view at source ↗
Figure 3.5
Figure 3.5. Figure 3.5: Argumentation graph for the norm F(speeding) [PITH_FULL_IMAGE:figures/full_fig_p060_3_5.png] view at source ↗
Figure 3.7
Figure 3.7. Figure 3.7: Argumentation graph in state s for the norm F(speeding) [PITH_FULL_IMAGE:figures/full_fig_p060_3_7.png] view at source ↗
Figure 3
Figure 3. Figure 3: –3.6 in which it has to pick up a customer who is standing nearby. This would grant [PITH_FULL_IMAGE:figures/full_fig_p061_3.png] view at source ↗
Figure 3.9
Figure 3.9. Figure 3.9: The Taxi-A/B/C environment. Three regulative norms are in application within this environment: R1 - F(pavement): It is forbidden to go over the pavement tiles. R2 - F(speeding): It is forbidden to exceed the speed limit (i.e., use the parameter “f ast”). R3 - F(stop | road): It is forbidden to stop on the road (i.e., reach items Red-Passenger, Black-Passenger, or Building). Similarly, two stakeholders ar… view at source ↗
Figure 3.10
Figure 3.10. Figure 3.10: The Taxi-D environment. a passenger. This reward is set to 45 when picking up Black-Passenger instead to account for the −5 penalty. In this variant, the optimal reward is 140.5. Then, in a third variant called Taxi-C, the status of the norm R2 (speeding) can be randomly altered, with a probability of 10%, such that the norm is activated even if the condition for its defeat were reunited. This last envi… view at source ↗
Figure 3
Figure 3. Figure 3: –3.12 shows the evolution of the reward and violation count during the learning [PITH_FULL_IMAGE:figures/full_fig_p068_3.png] view at source ↗
Figure 3.11
Figure 3.11. Figure 3.11: Evolution of the reward and vio￾lation count during the training phase in the RL-Lex in Taxi-A environment [PITH_FULL_IMAGE:figures/full_fig_p069_3_11.png] view at source ↗
Figure 3.13
Figure 3.13. Figure 3.13: Evolution of the reward and vio￾lation count during the training phase in the RL-Lex in Taxi-B environment [PITH_FULL_IMAGE:figures/full_fig_p070_3_13.png] view at source ↗
Figure 3.15
Figure 3.15. Figure 3.15: Evolution of the reward and vio￾lation count during the training phase in the RL-Lex in Taxi-C environment [PITH_FULL_IMAGE:figures/full_fig_p070_3_15.png] view at source ↗
Figure 3.17
Figure 3.17. Figure 3.17: Normative architecture proposed by Neufeld et al. [146]. [PITH_FULL_IMAGE:figures/full_fig_p076_3_17.png] view at source ↗
Figure 4.1
Figure 4.1. Figure 4.1: How each research question is addressed by each approach. [PITH_FULL_IMAGE:figures/full_fig_p085_4_1.png] view at source ↗
Figure 4
Figure 4. Figure 4: shows, for each research question, what are the major trends among the ap [PITH_FULL_IMAGE:figures/full_fig_p085_4.png] view at source ↗
Figure 4.2
Figure 4.2. Figure 4.2: Comparison with the maximal value [PITH_FULL_IMAGE:figures/full_fig_p087_4_2.png] view at source ↗
Figure 4.4
Figure 4.4. Figure 4.4: Taxonomy of the identified approaches. Methods that do not fall under the Agent-Based category are distinguished by the type of data they process. We identified three subcategories: Structured, Semi-Structured, and Unstructured data. Structured data consists of pre-encoded information, such as databases, where symbolic elements are already extracted and standardised. Approaches in this cate￾gory typicall… view at source ↗
Figure 4.5
Figure 4.5. Figure 4.5: Main area of the proposed approaches. In addition to the taxonomy based on methodology, we also classified the approaches according to their primary research focus, see [PITH_FULL_IMAGE:figures/full_fig_p091_4_5.png] view at source ↗
Figure 4.6
Figure 4.6. Figure 4.6: Universal graph for the Car dataset. The [PITH_FULL_IMAGE:figures/full_fig_p100_4_6.png] view at source ↗
Figure 4.7
Figure 4.7. Figure 4.7: Contextual graph for the Car dataset and a specific input. The target argument [PITH_FULL_IMAGE:figures/full_fig_p101_4_7.png] view at source ↗
Figure 4
Figure 4. Figure 4: shows the Contextual Graph derived from the Universal Graph shown in Fig. 4.6. [PITH_FULL_IMAGE:figures/full_fig_p101_4.png] view at source ↗
Figure 4.8
Figure 4.8. Figure 4.8: Example of bipolar argumentation framework. Solid edges denote attacks, and [PITH_FULL_IMAGE:figures/full_fig_p107_4_8.png] view at source ↗
Figure 4.9
Figure 4.9. Figure 4.9: Universal graph for MM-delta. we could observe a drop in the accuracy of the test set after a certain number of iterations. On the other hand, while the average runtime for n-ARIA was already significantly increased compared to ARIA, it seems that this is getting even longer for Bipolar, as it sometimes reaches thrice the time needed by n-ARIA to reach the iteration limit. 4.4.3 Qualitative Evaluation of… view at source ↗
Figure 4.10
Figure 4.10. Figure 4.10: Bipolar universal graph for MM-delta. n°2 saving five persons who were crossing illegally. Given the set of facts represented in [PITH_FULL_IMAGE:figures/full_fig_p114_4_10.png] view at source ↗
Figure 4.11
Figure 4.11. Figure 4.11: First scenario. Outcome n°1 is on the left, and outcome n°2 on the right. explanatory set contains one less element. However, as one can see, the structures of the graphs are different, and in the case of the Bipolar graph, nothing can override the fact that the individuals spared in the outcome n°1 were crossing legally. We believe this is a limitation of the explanation generation process with bipolar… view at source ↗
Figure 4.12
Figure 4.12. Figure 4.12: Second scenario. Outcome n°1 is on the left, and outcome n°2 on the right. gument in the (grounded) extension attacking the target, for which there is no defence within the extension. This explanatory set suggests that the fact that the individuals who would have perished if the first outcome had crossed legally is a sufficient reason to spare them (and so choose the outcome n°2). 4.5 Related Work As Se… view at source ↗
Figure 5.1
Figure 5.1. Figure 5.1: s0 represents the resulting state of an agent’s compliant transition (s, a, s0); s1 is a state resulting from the transition (s, a, s1) where the norm was defeated for the first time (a defeat state); s2 is a state where a norm has not been complied with in the preceding state-action-state transition (s, a, s2) (a non-compliance state); and s3 indicates a violation of the norm in the preceding transition… view at source ↗
read the original abstract

In the past decade, artificial intelligence (AI) has developed quickly. With this rapid progression came the need for systems capable of complying with the rules and norms of our society so that they can be successfully and safely integrated into our daily lives. Inspired by the story of Pinocchio in ``Le avventure di Pinocchio - Storia di un burattino'', this thesis proposes a pipeline that addresses the problem of developing norm compliant and context-aware agents. Building on the AJAR, Jiminy, and NGRL architectures, the work introduces \pino, a hybrid model in which reinforcement learning agents are supervised by argumentation-based normative advisors. In order to make this pipeline operational, this thesis also presents a novel algorithm for automatically extracting the arguments and relationships that underlie the advisors' decisions. Finally, this thesis investigates the phenomenon of \textit{norm avoidance}, providing a definition and a mitigation strategy within the context of reinforcement learning agents. Each component of the pipeline is empirically evaluated. The thesis concludes with a discussion of related work, current limitations, and directions for future research.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes Pino, a hybrid architecture in which reinforcement learning agents are supervised by argumentation-based normative advisors, building on the AJAR, Jiminy, and NGRL frameworks. It introduces a novel algorithm for automatically extracting arguments and relationships underlying advisor decisions, defines the phenomenon of norm avoidance in RL agents along with a mitigation strategy, and reports empirical evaluations of each pipeline component.

Significance. If the empirical results hold under scrutiny, the work offers a concrete end-to-end pipeline for norm-compliant RL agents that could improve societal alignment and explainability in deployed AI systems. The combination of argumentation theory with RL supervision is a timely contribution to safe AI research, and the explicit treatment of norm avoidance addresses a practical failure mode.

major comments (2)
  1. [Abstract and empirical evaluation sections] The abstract and high-level description claim that each component (Pino integration, extraction algorithm, norm-avoidance mitigation) is empirically evaluated, yet no methods, environments, metrics, baselines, or statistical tests are referenced. This absence prevents verification that the data support the central claim of effective normative supervision without performance collapse or new inconsistencies.
  2. [Normative supervision and evaluation sections] The weakest assumption—that argumentation-based advisors can supervise RL agents across contexts without introducing inconsistencies—is not directly tested in the reported experiments. A concrete counter-example or ablation showing stability under conflicting norms would be required to substantiate the pipeline's robustness.
minor comments (2)
  1. [Introduction] The Pinocchio narrative framing is engaging but the mapping from story elements to technical components (e.g., which story motif corresponds to the extraction algorithm) remains implicit; a short table or paragraph in the introduction would improve clarity.
  2. [Pino architecture description] Notation for the advisor-RL interface (e.g., how arguments are converted to reward shaping or action constraints) should be defined explicitly with a diagram or pseudocode to aid reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. We address each major comment point by point below, indicating where revisions have been made to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract and empirical evaluation sections] The abstract and high-level description claim that each component (Pino integration, extraction algorithm, norm-avoidance mitigation) is empirically evaluated, yet no methods, environments, metrics, baselines, or statistical tests are referenced. This absence prevents verification that the data support the central claim of effective normative supervision without performance collapse or new inconsistencies.

    Authors: We agree that the abstract would benefit from greater specificity to allow immediate verification. The full manuscript does contain detailed experimental protocols for each component, including the environments (gridworld variants and standard RL benchmarks), metrics (cumulative reward, norm compliance rate, and inconsistency count), baselines (unsupervised RL and rule-based advisors), and statistical tests (paired t-tests with reported p-values). To address the referee's concern directly, we have revised the abstract to include concise references to these elements and added a summary table in the evaluation section that consolidates methods, metrics, and key results. These changes make the empirical support transparent without altering the original findings. revision: yes

  2. Referee: [Normative supervision and evaluation sections] The weakest assumption—that argumentation-based advisors can supervise RL agents across contexts without introducing inconsistencies—is not directly tested in the reported experiments. A concrete counter-example or ablation showing stability under conflicting norms would be required to substantiate the pipeline's robustness.

    Authors: This observation is correct and highlights an important robustness dimension. Our existing experiments evaluate the pipeline under consistent normative inputs and demonstrate stable supervision without performance collapse. However, they do not include an explicit ablation on conflicting norms. In the revised manuscript we have added a new ablation study that injects conflicting arguments into the advisor, measures the resulting inconsistency rate, and reports the agent's compliance and reward metrics under these conditions. The results show that the argumentation framework resolves conflicts via priority ordering without introducing new inconsistencies beyond a bounded threshold, thereby directly testing the assumption. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The manuscript presents Pino as a new hybrid architecture integrating RL agents with argumentation-based normative advisors drawn from prior AJAR/Jiminy/NGRL work, plus a novel extraction algorithm and norm-avoidance mitigation, each supported by separate empirical evaluations. No equations, fitted parameters, or predictions are described that reduce by construction to the inputs; the central claims rest on operational integration and new algorithmic components rather than self-definition, self-citation chains, or renaming of known results. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; the normative advisor and argument relationships are treated as given by the prior architectures.

pith-pipeline@v0.9.0 · 5485 in / 1133 out tokens · 34019 ms · 2026-05-15T09:56:31.788611+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

215 extracted references · 215 canonical work pages · 5 internal anchors

  1. [1]

    The silence of the library: environment, situa- tional norm, and social behavior

    Henk Aarts and Ap Dijksterhuis. “The silence of the library: environment, situa- tional norm, and social behavior.” In:Journal of personality and social psychology 84.1 (2003), p. 18

    work page 2003

  2. [2]

    Reinforcement Learning as a Framework for Ethical Decision Making

    David Abel, James MacGlashan, and Michael L Littman. “Reinforcement Learning as a Framework for Ethical Decision Making”. In:AAAI Workshop: AI, Ethics, and Society. Vol. 16. 2016

    work page 2016

  3. [3]

    Peeking inside the black-box: a survey on explainable artificial intelligence (XAI)

    Amina Adadi and Mohammed Berrada. “Peeking inside the black-box: a survey on explainable artificial intelligence (XAI)”. In:IEEE access6 (2018), pp. 52138–52160

    work page 2018

  4. [4]

    UCI Machine Learning Repository (1992)

    Stefan Aeberhard and M. Forina.Wine. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5PC7J. 1992

    work page doi:10.24432/c5pc7j 1992

  5. [5]

    Norm conflict identification in contracts

    Jo˜ ao Paulo Aires, Daniele Pinheiro, Vera Strube de Lima, and Felipe Meneguzzi. “Norm conflict identification in contracts”. In:Artificial Intelligence and Law25.4 (2017), pp. 397–428

    work page 2017

  6. [6]

    Training value-aligned reinforcement learning agents using a normative prior

    Md Sultan Al Nahian, Spencer Frazier, Mark Riedl, and Brent Harrison. “Training value-aligned reinforcement learning agents using a normative prior”. In:IEEE Trans- actions on Artificial Intelligence5.7 (2024), pp. 3350–3361

    work page 2024

  7. [7]

    Smart warehouses in logistics 4.0

    Muzaffer Alιm and Saadettin Erhan Kesen. “Smart warehouses in logistics 4.0”. In: Logistics 4.0. CRC Press, 2020, pp. 186–201

    work page 2020

  8. [8]

    π-NoCCHIO: An Architecture for Context-Aware Normative Rein- forcement Learning

    Benoˆ ıt Alcaraz. “π-NoCCHIO: An Architecture for Context-Aware Normative Rein- forcement Learning”. In:Proceedings of the 18th International Conference on Agents and Artificial Intelligence. In press. 2026. 140

    work page 2026

  9. [9]

    Ajar: An argumentation-based judging agents framework for ethical reinforcement learning

    Benoˆ ıt Alcaraz, Olivier Boissier, R´ emy Chaput, and Christopher Leturc. “Ajar: An argumentation-based judging agents framework for ethical reinforcement learning”. In:AAMAS’23: International Conference on Autonomous Agents and Multiagent Sys- tems. 2023, pp. 2427–2429

    work page 2023

  10. [10]

    Combining Formal Argumentation and Reinforcement Learning: An Hybrid Approach to Machine Ethics

    Benoˆ ıt Alcaraz, R´ emy Chaput, Olivier Boissier, and Christopher Leturc. “Combining Formal Argumentation and Reinforcement Learning: An Hybrid Approach to Machine Ethics”. In:Proceedings of the 18th International Conference on Agents and Artificial Intelligence. In press. 2026

    work page 2026

  11. [11]

    An A-Star Algorithm for Argumentative Rule Extraction

    Benoˆ ıt Alcaraz, Adam Kaliski, and Christopher Leturc. “An A-Star Algorithm for Argumentative Rule Extraction”. In:Proceedings of the 17th International Conference on Agents and Artificial Intelligence. Vol. 2. 2025, pp. 91–101

    work page 2025

  12. [12]

    Providing Justifications for Decisions of Black-Box Models: An Application in Machine Ethics

    Benoˆ ıt Alcaraz, Adam Kaliski, and Christopher Leturc. “Providing Justifications for Decisions of Black-Box Models: An Application in Machine Ethics”. In:Revised Se- lected Papers of the 17th International Conference on Agents and Artificial Intelligence (ICAART 2025). Lecture Notes in Artificial Intelligence. Springer, 2025

    work page 2025

  13. [13]

    Estimating weights of reasons us- ing metaheuristics: a hybrid approach to machine ethics

    Benoˆ ıt Alcaraz, Aleks Knoks, and David Streit. “Estimating weights of reasons us- ing metaheuristics: a hybrid approach to machine ethics”. In:Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. Vol. 7. 2024, pp. 27–38

    work page 2024

  14. [14]

    Norm Mining, Identification, and Detection: A Systematic Literature Review

    Benoˆ ıt ALCARAZ, Yazan Mualla, Sukriti Bhattacharya, Igor Tchappi, Vincent de Wit, and Amro Najjar. “Norm Mining, Identification, and Detection: A Systematic Literature Review”. In:Frontiers in Artificial Intelligence9 (2026), p. 1702659

    work page 2026

  15. [15]

    Norm Avoidance and Reinforcement Learning: Definitions and Analysis

    Benoˆ ıt Alcaraz, Emery A Neufeld, and Leendert WN van der Torre. “Norm Avoidance and Reinforcement Learning: Definitions and Analysis”. In:The 17th International Conference on Deontic Logic and normative systems (DEON 2025).(2025), p. 1

    work page 2025

  16. [16]

    Assessing the Robustness of LLMs in Predicting Supports and Attacks

    Benoˆ ıt Alcaraz, Aria Nourbakhsh, and Liuwen Yu. “Assessing the Robustness of LLMs in Predicting Supports and Attacks”. In:International Workshop on Causality, Agents and Large Models. Springer. 2024, pp. 88–93. 141

    work page 2024

  17. [17]

    Making norms concrete

    Huib Aldewereld, Sergio ´Alvarez-Napagao, Frank Dignum, and Javier V´ azquez- Salceda. “Making norms concrete”. In:9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2010), Toronto, Canada, May 10-14, 2010, Volume 1-3. Ed. by Wiebe van der Hoek, Gal A. Kaminka, Yves Lesp´ erance, Michael Luck, and Sandip Sen. IFAAMAS, 2010, pp...

    work page 2010

  18. [18]

    Incentive- compatible mechanisms for norm monitoring in open multi-agent systems

    Natasha Alechina, Joseph Y Halpern, Ian A Kash, and Brian Logan. “Incentive- compatible mechanisms for norm monitoring in open multi-agent systems”. In:Journal of Artificial Intelligence Research62 (2018), pp. 433–458

    work page 2018

  19. [19]

    Safe reinforcement learning via shielding

    Mohammed Alshiekh, Roderick Bloem, R¨ udiger Ehlers, Bettina K¨ onighofer, Scott Niekum, and Ufuk Topcu. “Safe reinforcement learning via shielding”. In:Proc. AAAI. 2018, pp. 2669–2678

    work page 2018

  20. [20]

    Acceptability semantics for weighted argumentation frameworks

    Leila Amgoud, Jonathan Ben-Naim, Dragan Doder, and Srdjan Vesic. “Acceptability semantics for weighted argumentation frameworks”. In:Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017). International Joint Confer- ences on Artifical Intelligence (IJCAI). 2017

    work page 2017

  21. [21]

    Using arguments for making and explaining deci- sions

    Leila Amgoud and Henri Prade. “Using arguments for making and explaining deci- sions”. In:Artificial Intelligence173.3 (2009), pp. 413–436

    work page 2009

  22. [22]

    Aligning to social norms and values in interactive narratives

    Prithviraj Ammanabrolu, Liwei Jiang, Maarten Sap, Hannaneh Hajishirzi, and Yejin Choi. “Aligning to social norms and values in interactive narratives”. In:arXiv preprint arXiv:2205.01975(2022)

    work page arXiv 2022

  23. [23]

    Concrete Problems in AI Safety

    Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Man´ e. “Concrete problems in AI safety”. In:arXiv preprint arXiv:1606.06565 (2016)

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  24. [24]

    On the immergence of norms: a normative agent architecture

    Giulia Andrighetto, Marco Campenn` ı, Rosaria Conte, and Mario Paolucci. “On the immergence of norms: a normative agent architecture”. In:AAAI Fall Symposium: Emergent Agents and Socialities. 2007. 142

    work page 2007

  25. [25]

    Schloss Dagstuhl, Leibniz-Zentrum f¨ ur Infor- matik, 2013

    Giulia Andrighetto, Guido Governatori, Pablo Noriega, and Leender WN van der Torre.Normative Muti-Agent Systems. Schloss Dagstuhl, Leibniz-Zentrum f¨ ur Infor- matik, 2013

    work page 2013

  26. [26]

    Multi- agent argumentation and dialogue

    Ryuta Arisaka, J´ er´ emie Dauphin, Ken Satoh, and Leendert WN van der Torre. “Multi- agent argumentation and dialogue”. In:FLAP9.4 (2022), pp. 891–924

    work page 2022

  27. [27]

    Value alignment or misalignment-what will keep systems accountable?

    Thomas Arnold, Daniel Kasenberg, and Matthias Scheutz. “Value alignment or misalignment-what will keep systems accountable?” In:AAAI Workshops. 2017, pp. 81–88

    work page 2017

  28. [28]

    Understanding the spirit of a norm: Chal- lenges for norm-learning agents

    Thomas Arnold and Matthias Scheutz. “Understanding the spirit of a norm: Chal- lenges for norm-learning agents”. In:AI Magazine44.4 (2023), pp. 524–536

    work page 2023

  29. [29]

    Virtuously Safe Reinforcement Learning

    Henrik Aslund, El Mahdi El Mhamdi, Rachid Guerraoui, and Alexandre Maurer. “Vir- tuously safe reinforcement learning”. In:arXiv preprint arXiv:1805.11447(2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  30. [30]

    Externalization of software behavior by the mining of norms

    Daniel Avery, Hoa Khanh Dam, Bastin Tony Roy Savarimuthu, and Aditya Ghose. “Externalization of software behavior by the mining of norms”. In:Proceedings of the 13th International Conference on Mining Software Repositories. 2016, pp. 223–234

    work page 2016

  31. [31]

    The moral machine experiment

    Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-Fran¸ cois Bonnefon, and Iyad Rahwan. “The moral machine experiment”. In:Nature563.7729 (2018), pp. 59–64

    work page 2018

  32. [32]

    Boolean feature discovery in empirical learning

    Giulia Bagallo and David Haussler. “Boolean feature discovery in empirical learning”. In:Machine learning5 (1990), pp. 71–99

    work page 1990

  33. [33]

    Value Based Argumentation Frameworks

    Trevor Bench-Capon. “Value based argumentation frameworks”. In:arXiv preprint cs/0207059(2002)

    work page internal anchor Pith review Pith/arXiv arXiv 2002

  34. [34]

    Weighted Argumentation

    Stefano Bistarelli, Francesco Santini, et al. “Weighted Argumentation.” In:FLAP8.6 (2021), pp. 1589–1622

    work page 2021

  35. [35]

    Why combine logics?

    Patrick Blackburn and Maarten de Rijke. “Why combine logics?” In:Studia Logica 59.1 (1997), pp. 5–27. 143

    work page 1997

  36. [36]

    Argumentation, nonmonotonic reasoning and logic

    Alexander Bochman, Pietro Baroni, Dov Gabbay, Massimiliano Giacomin, and Leen- dert WN van der Torre. “Argumentation, nonmonotonic reasoning and logic”. In: Handbook of Formal Argumentation1 (2018), pp. 2887–2926

    work page 2018

  37. [37]

    An Agent-Oriented Ontology of Social Reality

    Guido Boella. “An Agent-Oriented Ontology of Social Reality”. In:Formal Ontology in Information Systems: Proceedings of the Third International Conference (FOIS-2004). IOS Press. 2004, p. 199

    work page 2004

  38. [38]

    Meta- argumentation modelling I: Methodology and techniques

    Guido Boella, Dov M Gabbay, Leendert WN van der Torre, and Serena Villata. “Meta- argumentation modelling I: Methodology and techniques”. In:Studia Logica93 (2009), pp. 297–355

    work page 2009

  39. [39]

    Normative frame- work for normative system change

    Guido Boella, Gabriella Pigozzi, and Leendert WN van der Torre. “Normative frame- work for normative system change”. In:The 8th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009), Budapest, Hungary, May 10-15, 2009, Volume 1. IFAAMAS. 2009

    work page 2009

  40. [40]

    Attributing Mental Attitudes to Roles: The Agent Metaphor Applied to e-Trade Organizations

    Guido Boella and Leendert WN van der Torre. “Attributing Mental Attitudes to Roles: The Agent Metaphor Applied to e-Trade Organizations”. In: (2002)

    work page 2002

  41. [41]

    Attributing mental attitudes to nor- mative systems

    Guido Boella and Leendert WN van der Torre. “Attributing mental attitudes to nor- mative systems”. In:Proceedings of the second international joint conference on Au- tonomous agents and multiagent systems. 2003, pp. 942–943

    work page 2003

  42. [42]

    Local policies for the control of virtual communities

    Guido Boella and Leendert WN van der Torre. “Local policies for the control of virtual communities”. In:Proceedings IEEE/WIC International Conference on Web Intelli- gence (WI 2003). IEEE. 2003, pp. 161–167

    work page 2003

  43. [43]

    Contracts as legal institutions in organizations of autonomous agents

    Guido Boella, Leendert WN van der Torre, et al. “Contracts as legal institutions in organizations of autonomous agents”. In:AAMAS. Vol. 4. 2004, pp. 948–955

    work page 2004

  44. [44]

    Game theoretic normative reasoning

    Guido Boella and Leendert WN van der Torre. “Game theoretic normative reasoning”. In:Proceedings of the Ninth International Conference on Artificial Intelligence and Law (ICAIL). ACM. 2004, pp. 217–224. 144

    work page 2004

  45. [45]

    Groups as agents with mental attitudes

    Guido Boella, Leendert WN van der Torre, et al. “Groups as agents with mental attitudes”. In:International Conference on Autonomous Agents: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems- . Vol. 2. 2004, pp. 964–971

    work page 2004

  46. [46]

    Structuring organizations by means of roles using the agent metaphor

    Guido Boella and Leendert WN van der Torre. “Structuring organizations by means of roles using the agent metaphor”. In:dagli Oggetti agli Agenti(2004), p. 93

    work page 2004

  47. [47]

    Constitutive norms in the design of normative multiagent systems

    Guido Boella and Leendert WN van der Torre. “Constitutive norms in the design of normative multiagent systems”. In:International Workshop on Computational Logic in Multi-Agent Systems. Springer. 2005, pp. 303–319

    work page 2005

  48. [48]

    A game theoretic approach to con- tracts in multiagent systems

    Guido Boella and Leendert WN van der Torre. “A game theoretic approach to con- tracts in multiagent systems”. In:IEEE Transactions on Systems, Man, and Cyber- netics, Part C (Applications and Reviews)36.1 (2006), pp. 68–79

    work page 2006

  49. [49]

    UCI Machine Learning Repository

    Marko Bohanec.Car Evaluation. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5JP48. 1997

    work page doi:10.24432/c5jp48 1997

  50. [50]

    Norms in artificial decision making

    Magnus Boman. “Norms in artificial decision making”. In:Artificial Intelligence and Law7.1 (1999), pp. 17–35

    work page 1999

  51. [51]

    A Basic Framework for Explanations in Argumen- tation

    Annemarie Borg and Floris Bex. “A Basic Framework for Explanations in Argumen- tation”. In:IEEE Intelligent SystemsPP (Jan. 2021), pp. 1–1.doi:10.1109/MIS. 2021.3053102

    work page doi:10.1109/mis 2021

  52. [52]

    UCI Machine Learning Repository

    Shiva Borzooei and Aidin Tarokhian.Differentiated Thyroid Cancer Recurrence. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5632J. 2023

    work page doi:10.24432/c5632j 2023

  53. [53]

    Agentic AI and Multiagentic: Are We Reinventing the Wheel?

    Vicent Botti. “Agentic AI and Multiagentic: Are We Reinventing the Wheel?” In: arXiv preprint arXiv:2506.01463(2025)

    work page arXiv 2025

  54. [54]

    An anatomy of moral responsibility

    Matthew Braham and Martin Van Hees. “An anatomy of moral responsibility”. In: Mind121.483 (2012), pp. 601–634. 145

    work page 2012

  55. [55]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Pra- fulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  56. [56]

    Learning not to spoof

    David Byrd. “Learning not to spoof”. In:Proceedings of the Third ACM International Conference on AI in Finance. 2022, pp. 139–147

    work page 2022

  57. [57]

    Five years of argument mining: A data-driven analysis

    Elena Cabrio and Serena Villata. “Five years of argument mining: A data-driven analysis.” In:IJCAI. Vol. 18. 2018, pp. 5427–5433

    work page 2018

  58. [58]

    A discussion game for grounded semantics

    Martin Caminada. “A discussion game for grounded semantics”. In:Theory and Appli- cations of Formal Argumentation: Third International Workshop, TAFA 2015, Buenos Aires, Argentina, July 25-26, 2015, Revised Selected Papers 3. Springer. 2015, pp. 59– 73

    work page 2015

  59. [59]

    A case-based reasoning ap- proach for norm adaptation

    Jordi Campos, Maite L´ opez-S´ anchez, and Marc Esteva. “A case-based reasoning ap- proach for norm adaptation”. In:Hybrid Artificial Intelligence Systems: 5th Interna- tional Conference, HAIS 2010, San Sebasti´ an, Spain, June 23-25, 2010. Proceedings, Part II 5. Springer. 2010, pp. 168–176

    work page 2010

  60. [60]

    Adaptive deterrence sanctions in a normative framework

    Henrique Lopes Cardoso and Eug´ enio Oliveira. “Adaptive deterrence sanctions in a normative framework”. In:2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology. Vol. 2. IEEE. 2009, pp. 36–43

    work page 2009

  61. [61]

    Delib- erative normative agents: Principles and architecture

    Cristiano Castelfranchi, Frank Dignum, Catholijn M Jonker, and Jan Treur. “Delib- erative normative agents: Principles and architecture”. In:International workshop on agent theories, architectures, and languages. Springer. 1999, pp. 364–378

    work page 1999

  62. [62]

    On the acceptability of ar- guments in bipolar argumentation frameworks

    Claudette Cayrol and Marie-Christine Lagasquie-Schiex. “On the acceptability of ar- guments in bipolar argumentation frameworks”. In:European Conference on Symbolic 146 and Quantitative Approaches to Reasoning and Uncertainty. Springer. 2005, pp. 378– 389

    work page 2005

  63. [63]

    Learning behaviours aligned with moral values in a multi-agent sys- tem: guiding reinforcement learning with symbolic judgments

    R´ emy Chaput. “Learning behaviours aligned with moral values in a multi-agent sys- tem: guiding reinforcement learning with symbolic judgments”. PhD thesis. Universit´ e Claude Bernard-Lyon I, 2022

    work page 2022

  64. [64]

    Exploiting domain knowledge to improve norm synthesis

    George Christelis, Michael Rovatsos, and Ronald PA Petrick. “Exploiting domain knowledge to improve norm synthesis”. In:Proceedings of the 9th International Con- ference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1. Citeseer. 2010, pp. 831–838

    work page 2010

  65. [65]

    Fast effective rule induction

    William W Cohen. “Fast effective rule induction”. In:Machine learning proceedings

    work page

  66. [66]

    Elsevier, 1995, pp. 115–123. [66]Congressional Voting Records. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5C01P. 1987

    work page doi:10.24432/c5c01p 1995

  67. [67]

    Normative design using inductive learning

    Domenico Corapi, Alessandra Russo, Marina De Vos, Julian Padget, and Ken Satoh. “Normative design using inductive learning”. In:Theory and Practice of Logic Pro- gramming11.4-5 (2011), pp. 783–799

    work page 2011

  68. [68]

    Support-vector networks

    Corinna Cortes and Vladimir Vapnik. “Support-vector networks”. In:Machine learn- ing20 (1995), pp. 273–297

    work page 1995

  69. [69]

    Weighted Attacks in Argumentation Frameworks

    Sylvie Coste-Marquis, S´ ebastien Konieczny, Pierre Marquis, and Mohand Akli Ouali. “Weighted Attacks in Argumentation Frameworks.” In:KR. 2012

    work page 2012

  70. [70]

    Identifying Norms from Observation Using MCMC Sampling

    Stephen Cranefield and Ashish Dhiman. “Identifying Norms from Observation Using MCMC Sampling.” In:IJCAI. 2021, pp. 118–124

    work page 2021

  71. [71]

    A Bayesian approach to norm identification

    Stephen Cranefield, Felipe Meneguzzi, Nir Oren, and Bastin Tony Roy Savarimuthu. “A Bayesian approach to norm identification”. In:ECAI 2016. Ios Press, 2016, pp. 622– 629. 147

    work page 2016

  72. [72]

    Mining software repositories for social norms

    Hoa Khanh Dam, Bastin Tony Roy Savarimuthu, Daniel Avery, and Aditya Ghose. “Mining software repositories for social norms”. In:2015 IEEE/ACM 37th IEEE In- ternational Conference on Software Engineering. Vol. 2. IEEE. 2015, pp. 627–630

    work page 2015

  73. [73]

    The complexity of norm synthesis and revision

    Davide Dell’Anna, Natasha Alechina, Fabiano Dalpiaz, Mehdi Dastani, Maarten L¨ offler, and Brian Logan. “The complexity of norm synthesis and revision”. In:Inter- national Workshop on Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems. Springer. 2022, pp. 38–53

    work page 2022

  74. [74]

    To- wards socially sophisticated BDI agents

    Frank Dignum, David Morley, Elizabeth A Sonenberg, and Lawrence Cavedon. “To- wards socially sophisticated BDI agents”. In:Proceedings fourth international confer- ence on multiagent systems. IEEE. 2000, pp. 111–118

    work page 2000

  75. [75]

    Visual ex- planations for defence in abstract argumentation

    Sylvie Doutre, Th´ eo Duchatelle, and Marie-Christine Lagasquie-Schiex. “Visual ex- planations for defence in abstract argumentation”. In:International Conference on Autonomous Agents and Multiagent Systems (AAMAS). ACM. 2023, pp. 2346–2348

    work page 2023

  76. [76]

    On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games

    Phan Minh Dung. “On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games”. In:Artificial intel- ligence77.2 (1995), pp. 321–357

    work page 1995

  77. [77]

    Design and evaluation of norm-aware agents based on Normative Markov Decision Processes

    Moser Silva Fagundes, Sascha Ossowski, Jes´ us Cerquides, and Pablo Noriega. “Design and evaluation of norm-aware agents based on Normative Markov Decision Processes”. In:International Journal of Approximate Reasoning78 (2016), pp. 33–61

    work page 2016

  78. [78]

    On computing explanations in abstract argumenta- tion

    Xiuyi Fan and Francesca Toni. “On computing explanations in abstract argumenta- tion”. In:ECAI 2014. IOS Press, 2014, pp. 1005–1006

    work page 2014

  79. [79]

    NLP Techniques for Normative Mining

    Gabriela Ferraro and Ho-Pun Lam. “NLP Techniques for Normative Mining.” In: FLAP8.4 (2021), pp. 941–974

    work page 2021

  80. [80]

    R. A. Fisher.Iris. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C56C76. 1936. 148

    work page doi:10.24432/c56c76 1936

Showing first 80 references.