Operationalising Rawlsian Ethics for Fairness in Norm-Learning Agents

Jessica Woodgate; Nirav Ajmeri; Paul Marshall

REVIEW 5 major objections 6 minor 55 references

Operationalising Rawlsian Ethics for Fairness in Norm-Learning Agents

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RAWL·E embeds Rawlsian maximin into each agent's reward, producing fairer norms in simulated societies.

desk verdict Maximin reward shaping works in a simple grid world, but the norm module is decorative and the paper's causal claims about norms are not supported by its own architecture. read the letter →

arxiv 2412.15163 v1 pith:XYIVZ6IO submitted 2024-12-19 cs.MA cs.AIcs.LG

Jessica Woodgate , Paul Marshall , Nirav Ajmeri This is my paper · ORCID

classification cs.MAcs.AIcs.LG

keywords normemergencemulti-agentreinforcementlearningRawlsianethicsmaximinfairnessrewardshapingsocialwelfareDQN

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a multi-agent reinforcement learning system can be made to learn ethical social norms by building a Rawlsian maximin principle directly into each agent's reward. The authors propose RAWL·E, in which an ethics module compares the lowest well-being in the society before and after an agent acts, and rewards or penalises the agent according to whether that minimum rose, stayed flat, or fell. They test the method in two simulated berry-harvesting worlds, comparing societies of RAWL·E agents to baseline deep-Q agents whose rewards are normalised to be lower. They report that RAWL·E societies produce more cooperative and more generalised norms, lower Gini inequality, higher minimum well-being, and longer survival. The significance is that it offers a way to derive 'ought' from a normative principle rather than from observed behaviour, which is what they argue lets it avoid perpetuating bad existing norms.

What carries the argument

The carrying mechanism is the ethics module that implements the maximin function $M_A(d) = \min_w u(d, \upsilon_i)$ adapted from Leben, comparing the minimum well-being before and after an action and generating a self-directed sanction $F_{t+1}$ via Equation (2). This sanction is combined with the environmental reward through reward shaping, $r'_{t+1} = r_{t+1} + F_{t+1}$, so every agent's DQN policy is trained on a signal that includes whether the least advantaged agent was made better off. The norms module stores behaviours as if-then rules over preconditions and actions, counts their usage, and decays their fitness, so that the 90%-converged norms that emerge are those the society actually uses under the shaped reward.

What would settle it

Run the same harvest scenarios with an oracle that attributes the minimum-well-being change to its true cause through counterfactual rewards computed by replaying the step without the agent's action, and compare to RAWL·E; if the fair-metrics advantage disappears or reverses, the observed fairness improvement is an artefact of misattributed sanction rather than learned ethics. A cheaper check is to log per-step cases where the acting agent's action could not have changed the minimum yet the sanction was nonzero, and test whether those cases alone explain the inequality reduction.

Watch

Extended reading notes

Core claim

The central claim is that operationalising Rawlsian maximin in individual decision-making changes which norms emerge in a multi-agent society, and that the emerged norms are fairer and more robust. In the paper's own terms, a RAWL·E agent takes the vector of all agents' well-being $U_t$ and $U_{t+1}$, finds $\upsilon_{\min}$ at each step, and produces a self-directed sanction $\xi = 0.4$ if the minimum improved, $-\xi$ if it worsened, and 0 otherwise; this sanction is added to the environmental reward to form the learning signal. Across 2000 episodes in both harvesting scenarios, societies of RAWL·E agents showed lower inequality (Gini), higher minimum experience, higher social welfare, and longer episodes than baseline DQN societies, and the emerged cooperative norms were more generalised (for example, IF <high health> THEN <throw>). The paper states this as evidence that normative ethics can be operationalised to promote ethical norm emergence without relying on descriptive accounts of existing behaviour.

Load-bearing premise

The learning signal assumes the change in the minimum well-being from one step to the next was caused by the acting agent's own action, so sanctioning that change teaches correct ethical behaviour; in the asynchronous setting other agents' actions also move the minimum, which can reward or punish an agent for effects it did not cause.

Editorial extensions

If this is right

RAWL·E societies learn cooperative norms that are more generalised and used more often: for example, IF <high health> THEN <throw> emerges in the RAWL·E society while baseline cooperative norms remain more specialised.
Inequality falls: the Gini index for well-being drops from 0.20 to 0.10 in the allotment scenario, with large effect sizes (d = 1.58).
Minimum individual well-being is higher in RAWL·E societies (10.82 vs 7.18 in allotment well-being, d = 3.09), meaning the least advantaged agent is better off.
Social welfare and robustness improve: cumulative well-being is higher and episodes last longer, though the robustness effect is negligible (d = 0.11).
Because the ethics module is decoupled from the environment, the method is compatible with other RL algorithms and scenarios, so the same reward-shaping pattern can be applied elsewhere.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the attribution assumption is the real driver, a counterfactual-credit variant (rewarding only when the acting agent's own action raised the minimum) would tell whether the improved metrics come from learning to help or from a noisy but benevolent signal; this is a testable extension the paper does not run.
The same reward-shaping template could be applied to other normative principles, such as egalitarian or prioritarian functions, and compared directly; the paper leaves this comparison to future work.
In environments with simultaneous moves or confounded effects, the per-step min-difference sanction may be too blunt, suggesting the method may need temporal credit assignment to scale beyond the asynchronous one-agent-at-a-time setting used here.
The generality of the emerged cooperative norms (IF high health THEN throw) suggests the shaped reward biases the society toward a division of labour, but the paper does not test whether this specialisation is robust to changes in agent capabilities or resource distribution; a follow-up with perturbed environments would clarify.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: the norms module does not feed back into action selection. Algorithm 3 writes to the norm base but nothing ever reads from it into the policy. So the differences between RAWL·E and baseline—where they exist—are wholly attributable to the shaped reward, not to 'norms emerging.' The hypotheses H1–H4 are phrased as norms causing outcomes, but the architecture cannot support that.

What's actually new and useful: applying maximin reward shaping to DQN agents in a harvesting game, with a transparent implementation and public code. That is a legitimate small contribution. The reward normalization between baseline and RAWL·E is a fair attempt to isolate the ethics signal, and the appendices are unusually thorough.

The soft spots are substantial. First, the metric M2 (minimum experience) is exactly the quantity the ethics module rewards. A positive sanction is given when the minimum increases, so RAWL·E societies having higher M2 is by construction, not a discovery. Second, Eq. 4 defines well-being as health + (berries × hgain)/hdecay; since hdecay is negative, more berries make well-being lower. That sign error is hard to reconcile with the text. Third, Section 4.3 describes a sanction for 'could have improved but did not,' but Algorithm 1 only checks whether the actual minimum changed; the implemented sanction is simpler than described. Fourth, for agresource, the minimum-experience and social-welfare differences are not significant, and robustness effects are negligible (d = 0.11–0.18), so the headline overreaches even on its own terms.

The paper is a decent engineering write-up of a reward-shaping baseline, with good reproducibility. It is not a demonstration of norm emergence influencing behavior; the norm module is an inert record-keeper. For a reader interested in fairness rewards in MARL, it's worth a skim. For a reader hoping to see norms do causal work, it will disappoint.

I would send it to peer review only with the expectation of major revision: rephrase the claims to say the shaped reward improves fairness, remove the norm-emergence causation language, fix Eq. 4, and reconcile the sanction description. With those changes it could be a solid workshop-level paper; as is, the central claim is unsupported.

Referee Report

5 major / 6 minor

Summary. The paper proposes RAWL·E, a method that augments DQN agents in a multi-agent harvesting domain with a Rawlsian maximin-based ethics module. The ethics module computes a self-directed sanction from the change in minimum well-being between time steps and adds it to the environmental reward. A separate norms module records behaviors and detects when 90% of agents share a behavior, treating that as an emerged norm. The authors evaluate RAWL·E against a baseline DQN society in two harvesting scenarios (capabilities and allotment), reporting that RAWL·E societies have lower inequality, higher minimum experience, higher social welfare, and higher robustness. They conclude that norms emerging in RAWL·E societies lead to fairer and more sustainable collective behavior.

Significance. If the causal claims about norm emergence were empirically supported, this paper would make a useful contribution by connecting normative ethics (Rawlsian maximin) to bottom-up norm emergence in multi-agent reinforcement learning. Strengths include a publicly released codebase, detailed parameter tables in appendices, and a clearly described modular architecture. However, the significance is substantially weakened by three issues: (1) a sign error in the well-being formula that likely corrupts the ethics signal and the main fairness metric; (2) a circularity between the reward function and the primary evaluation metric; and (3) an architecture in which the norms module is never consulted during action selection, so the observed differences are attributable to reward shaping rather than to norms. The statistical claims also overreach the reported data.

major comments (5)

[Eq. (4), Section 4.1] The well-being formula is agwell-being = aghealth + (agberries × hgain) / hdecay. Since hgain = 0.1 and hdecay = −0.01, the second term equals −10 × agberries, so well-being decreases with berry count. This sign error directly affects the ethics module's input (the minimum well-being in Eq. (1)) and the M2 metric (minimum experience) that is used to support H2. The formula should be corrected (e.g., hgain × agberries / |hdecay|, or hgain × agberries plus health) and the experiments re-run, as the reported numeric values and effect sizes may change substantially.
[Sections 3.2 and 4.4] The evaluation metric M2 (minimum experience) is the same quantity that the ethics module rewards: Eq. (2) gives a positive sanction when the minimum well-being increases, and M2 measures the minimum well-being across agents. Therefore the finding that RAWL·E societies have higher minimum experience is not an emergent property of norms; it is a direct consequence of the reward shaping. The claim in H2 and in the abstract that norms 'lead to' higher minimum experience is thus circular. The paper needs a metric that is not the optimization target of the sanction, or an explicit argument that the improvement exceeds what reward shaping alone would produce.
[Algorithm 3, Section 3.2] The norms module is causally inert with respect to behavior. In Algorithm 3, the policy is updated using the shaped reward (lines 5–7), and only afterward are νt, at, and r′t+1 passed to the norms module (lines 8–9). Nothing from the norms module or the norm base is fed back into action selection. Thus the norms module records behavior but never influences it. Consequently, any differences in M1–M4 between RAWL·E and baseline agents are attributable to the ethics-module reward Ft+1, not to emerged norms. The hypotheses H1–H4 are phrased as 'norms emerging ... lead to' outcomes, which is not supported by this architecture. An ablation separating reward shaping from norm recording, or a mechanism by which norms affect decisions, is needed to support the stated contribution.
[Section 4.1, Eq. (2)] In the asynchronous setting, agents act in random order within each step. The ethics module compares Ut and Ut+1, where the minimum well-being can change because of another agent's action in the same step. The sanction Ft+1 is then assigned to the acting agent, even if the change in the minimum was caused by a different agent. This attribution problem means the learning signal may reward or punish an agent for effects it did not cause, undermining the interpretation that RAWL·E agents 'learn ethical behavior' rather than merely responding to a noisy proxy. The paper should either restrict the comparison to changes caused by the acting agent or analyze the extent of misattribution.
[Section 5.2, Table 8] The headline claims overreach the reported statistics. For the M2 (minimum experience) metric on agresource, the differences are not significant in either scenario (d = 0.15 in capabilities, d = 0.27 in allotment; the text says p > 0.01). For M3 (social welfare) on agresource, differences are not significant (d = 0.04 and 0.14). For M4 (robustness), although p < 0.01, the effect sizes are negligible (d = 0.18 and 0.11). Yet the Summary of Findings states 'Our results support our hypotheses' and the abstract claims 'higher social welfare, fairness, and robustness' without these qualifications. The conclusions should be scaled back to the metrics that show significant and at least small-to-medium effects, or the hypotheses should be revised to match the evidence.

minor comments (6)

[Section 4.4] The hypothesis labels are inconsistent: Section 4.4 defines H1 as minimum experience and H2 as inequality, but Section 5.2 presents 'H1 (inequality)' and 'H2 (minimum experience)'. The numbering should be corrected throughout.
[Algorithm 2 and Table 1] Algorithm 2 uses 'clipNorm' (line 8) and updates emerged norms (line 10), but Table 1 lists 'tclip behaviours' and 'tclip norms' without explaining how these map to clipNorm and the clipping interval. The relation between the parameters and the algorithm should be made explicit.
[Tables 3 and 8] Tables 3 and 8 contain overlapping data for the allotment scenario; Table 8 in the appendix duplicates most of Table 3. Either merge them or clearly separate the summary table from the full appendix table to avoid confusion.
[Figures 3 and 4] The captions for Figures 3 and 4 say the quantities are 'summed for e, normalised by step frequency', but the y-axis labels are just 'Minimum agwell-being' and 'Cumulative agwell-being'. Please define the normalization and the aggregation in the caption or in the text so the reader can interpret the plots accurately.
[Notation] The paper uses 'RAWL·E' with a middle dot in most of the text, but the abstract, headings, and some equations sometimes use 'RAWL-E' or 'RAWL·E' inconsistently. Please standardize the notation.
[Section 4.3, Table 6] The reward normalization is described as giving RAWL·E agents 'lower raw rewards', but Table 6 shows that for 'Try to eat without berries' and 'Try to throw without berries' the RAWL·E penalty is −0.10 while baseline is −0.20. Please clarify the normalization procedure, since the stated rationale does not match all entries.

Circularity Check

2 steps flagged · score 7.0 of 10

The higher-minimum-experience result is directly shaped by the ethics reward, and the norm module is write-only, so the central 'norms enhance fairness' claim reduces to reward shaping.

self definitional [Section 3.2, Eq. 2; Section 4.4 metric M2]
"Ft+1(st, st+1) = ξ, if υmint < υmint+1; 0, if υmint = υmint+1; −ξ, if υmint > υmint+1 ... M2 (minimum experience) Lowest individual experience across the society. Higher is better."

The shaped reward Ft+1 trains the policy to increase the minimum well-being υmin, and M2 is defined as the lowest individual experience (agwell-being). Therefore RAWL·E's higher M2 (e.g., 10.82 vs 7.18 in the allotment harvest) is the optimization target itself, not a consequence of emerged norms. Reducing raw rewards to 0.8 for eating and foraging does not remove this built-in objective; it only makes the extra min-experience reward the differentiating term. Hypothesis H2, phrased as 'norms ... lead to higher minimum individual experience,' restates the reward shaping by construction.
renaming known result [Section 3.2, Algorithm 3; Section 1 and Section 5 hypotheses]
"1: at ← π(st) ... 6: r′t+1 ← rt+1 + Ft+1 /* Shape reward */ 7: Π(θ, a) ← update(Π, st, r′t+1, st+1) /* Update policy */ 8: νt ← getView(st) 9: NormsModule(νt, at, r′t+1) /* Update norms module */"

The policy is updated solely from the shaped reward before NormsModule is invoked, and nothing returned by NormsModule is fed back into action selection. The norm base is written but never read during decision making. Hence the headline claim that 'norms emerging in RAWL·E agent societies enhance social welfare, fairness, and robustness' renames the effect of ethics reward shaping as an emergent-norm effect. The norm module records behavior; it does not cause it, so the hypotheses that 'norms emerging ... lead to' outcomes are post-hoc attributions rather than a derived causal chain.

full rationale

The paper's most emphasized result—higher minimum experience under RAWL·E—is circular by construction: the ethics module's sanction is exactly a function of whether the minimum well-being increased, and the evaluation metric M2 is that same minimum well-being. The attempted reward normalization lowers unrelated raw rewards but leaves the direct min-experience reward in place, so the comparison still builds the target metric into the treated agents' objective. Separately, the norms module is causally inert: in Algorithm 3 the DQN is updated with the shaped reward before the norms module runs, and the norm base is never consulted during action selection. Thus 'norms emerging ... lead to' outcomes is a renaming of the reward-shaping effect, not a measured mechanism. There is some independent content: Gini, social welfare, and robustness are not the exact shaped quantity, and the reported effects on those metrics are not fully predetermined. The self-citations to the authors' prior ethics work are not load-bearing here, and the code is public, so this is not a pure self-citation chain. Overall, the central fairness claim is substantially forced by the reward definition, while secondary metrics retain partial independent evidentiary value, giving a partial-circularity score of 7.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hand-chosen quantities (sanction magnitude, reward normalization) and on domain assumptions about attribution and the meaning of norms. The well-being formula is internally inconsistent. No new physical entities are introduced.

free parameters (3)

Sanction magnitude ξ = 0.4 (and -0.4 for decrease)
The ethics reward strength is set to 0.4 without a reported sensitivity analysis; it directly scales the pressure on agents to improve the minimum well-being and may determine whether the effect appears.
Raw reward normalization for RAWL-E = Eat/forage 0.8 vs 1.0; survive 1.0; other penalties adjusted
The authors lower RAWL-E's raw rewards 'to avoid obvious results', but this hand-chosen normalization changes the reward landscape and is not independently justified.
Well-being formula coefficients = hgain/hdecay with hdecay sign inconsistent
Equation (4) uses hdecay=-0.01, which makes the berry term negative; the reported positive minima imply a different formula was used, so the metric calculation is ambiguous.

assumptions (4)

domain assumption Rawlsian maximin is an appropriate ethical principle for agent fairness
The paper adopts maximin from Rawls (1958) without justifying that it is the right ethical theory for MAS; this is a philosophical premise.
domain assumption A self-directed reward sanction can operationalise ethics in RL
The paper assumes that adding a numeric sanction based on the minimum well-being makes agents 'ethical' rather than merely optimizing a different objective. Section 3.2.
ad hoc to paper Norm emergence can be detected when 90% of the population adopts the same behaviour
The 90% threshold is taken from Kittock (1995) and is a convention, not a property of the system.
domain assumption The change in minimum well-being between t and t+1 is attributable to the acting agent's action
In asynchronous multi-agent settings this attribution is not guaranteed; Section 4.1 says agents act in random order, so other agents also affect the minimum.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Operationalising Rawlsian Ethics for Fairness in Norm-Learning Agents." pith.science (2026). https://pith.science/paper/XYIVZ6IO

@misc{pith2026241215163,
  author       = {Pith},
  title        = {Pith review of: Operationalising Rawlsian Ethics for Fairness in Norm-Learning Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYIVZ6IO}},
  note         = {Machine review of arXiv:2412.15163}
}

read the original abstract

Social norms are standards of behaviour common in a society. However, when agents make decisions without considering how others are impacted, norms can emerge that lead to the subjugation of certain agents. We present RAWL-E, a method to create ethical norm-learning agents. RAWL-E agents operationalise maximin, a fairness principle from Rawlsian ethics, in their decision-making processes to promote ethical norms by balancing societal well-being with individual goals. We evaluate RAWL-E agents in simulated harvesting scenarios. We find that norms emerging in RAWL-E agent societies enhance social welfare, fairness, and robustness, and yield higher minimum experience compared to those that emerge in agent societies that do not implement Rawlsian ethics.

Figures

Figures reproduced from arXiv: 2412.15163 by the authors.

**Figure 1.** Harvesting environment. (a) Capabilities harvest scenario explores how agents learn to identify and reach desired [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

**Figure 2.** Comparing Gini index of agwell-being and agresource for e. Lower Gini in RAWL·E indicates lower inequality. 0 10 20 30 40 50 6 7 8 9 Minimum agwell-being Baseline RAWL·E (a) Capabilities harvest. 0 10 20 30 40 50 8 10 12 Steps Minimum agwell-being Baseline RAWL·E (b) Allotment harvest [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Minimum agwell-being over tmax steps summed for e, normalised by step frequency. RAWL·E yields higher minimum well-being. H3 (social welfare) RAWL·E yields higher social welfare. For agwell-being, the allotment harvest yields x¯ = 59.80 for RAWL·E and x¯ = 20.60 for baseline which has a medium effect (0.64). We reject the null hypothesis corresponding to H3 for agwell-being (p < 0.01), the difference, however, for … view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Cumulative agwell-being and agresource of each society over tmax steps summed for e, normalised by step frequency. Societies of RAWL·E agents have higher well-being and cumulative resource consumption. hypothesis corresponding to H4 as p < 0.01; the effect is negligi…

**Figure 5.** Figure 5: Days survived for e. Societies of RAWL·E agents survive for longer, indicating higher robustness. Summary of Findings Our results support our hypotheses. Our main findings are: (1) in a society of RAWL·E agents, social welfare is improved, indicated by higher cumulat…

**Figure 6.** Figure 6: N for capabilities harvest over eepochs. blue highlights cooperative norms. In societies of RAWL·E agents, more generalised cooperative norms emerge than in baseline societies. For example, in Figure 6b the general norm of “IF high health THEN throw” emerges. 12 [PITH…

**Figure 7.** Figure 7: N for allotment harvest over eepochs. blue highlights cooperative norms. In societies of RAWL·E agents, cooperative norms which emerge are more generalised than in baseline societies. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 51 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Agrawal, R.; Ajmeri, N.; and Singh, M. P. 2022. Socially Intelligent Genetic Agents for the Emergence of Explicit Norms. In Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI), 10--14. Vienna: IJCAI

work page 2022
[4]

K.; and Singh, M

Ajmeri, N.; Guo, H.; Murukannaiah, P. K.; and Singh, M. P. 2020. Elessar: Ethics in Norm-Aware Agents. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 16--24. Auckland: IFAAMAS

work page 2020
[5]

M.; Cranefield, S.; and Savarimuthu, B

Anavankot, A. M.; Cranefield, S.; and Savarimuthu, B. T. R. 2023. Towards Norm Entrepreneurship in Agent Societies. In Advances in Practical Applications of Agents, Multi-Agent Systems, and Cognitive Mimetics. The PAAMS Collection, 188--199. Switzerland: Springer

work page 2023
[6]

L.; and Armen, C

Anderson, M.; Anderson, S. L.; and Armen, C. 2004. Towards Machine Ethics. In AAAI-04 Workshop on Agent Organizations: Theory and Practice, 1--7. San Jose: AAAI

work page 2004
[7]

Balakrishnan, S.; Bi, J.; and Soh, H. 2022. SCALES: From Fairness Principles to Constrained Decision-Making. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AIES), 46--55. Oxford: ACM

work page 2022
[8]

Bengio, Y. 2012. Practical recommendations for gradient-based training of deep architectures. CoRR, abs/1206.5533

arXiv 2012

Show all 55 references

[9]

Binns, R. 2018. Fairness in Machine Learning: Lessons from Political Philosophy. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAccT), volume 81, 149--159. New York: PMLR

2018
[10]

F.; Everett, M.; Liu, M.; and How, J

Chen, Y. F.; Everett, M.; Liu, M.; and How, J. P. 2017. Socially aware motion planning with deep reinforcement learning. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), volume 1, 1343--1350. Vancouver: IEEE

2017
[11]

Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 2nd edition

1988
[12]

Dell'Anna, D.; Dastani, M.; and Dalpiaz, F. 2020. Runtime Revision of Sanctions in Normative Multi-Agent Systems. Autonomous Agents and Multi-Agent Systems (JAAMAS), 34(2): 1--54

2020
[13]

Dignum, V. 2021. The Myth of Complete AI -Fairness. In Tucker, A.; Henriques Abreu, P.; Cardoso, J.; Pereira Rodrigues, P.; and Ria \ n o, D., eds., Artificial Intelligence in Medicine, 3--8. Online: Springer

2021
[14]

Dong, S.; Li, C.; Yang, S.; An, B.; Li, W.; and Gao, Y. 2024. Egoism, utilitarianism and egalitarianism in multi-agent reinforcement learning. Neural Networks, 178: 106544

2024
[15]

Endriss, U. 2013. Reduction of Economic Inequality in Combinatorial Domains. In Proceedings of the 12th International Conference on Autonomous Agents and Multi-Agent Systems ( AAMAS ) , 175--182. Minnesota: IFAAMAS

2013
[16]

A.; Selman, B.; and Lee, D

Grupen, N. A.; Selman, B.; and Lee, D. D. 2022. Cooperative Multi-Agent Fairness and Equivariant Policies. In Proceedings of the 36th AAAI Conference on Artificial Intelligence ( AAAI ) , volume 36, 9350--9359. Online

2022
[17]

Guinebert, S. 2020. How do moral theories stand to each other? Zeitschrift f \"u r Ethik und Moralphilosophie , 3(2): 279--299

2020
[18]

Guo, Y.; Wang, B.; Hughes, D.; Lewis, M.; and Sycara, K. 2020. Designing Context-Sensitive Norm Inverse Reinforcement Learning Framework for Norm-Compliant Autonomous Agents. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 618-...

2020
[19]

Huber, P. J. 1964. Robust Estimation of a Location Parameter. Annals of Mathematical Statistics, 35: 492--518

1964
[20]

Jing, S.; and Doorn, N. 2020. Engineers’ Moral Responsibility : A Confucian Perspective . Science and Engineering Ethics, 26(1): 233--253

2020
[21]

W.; Hooker, J.; and Donaldson, T

Kim, T. W.; Hooker, J.; and Donaldson, T. 2021. Taking Principles Seriously: A Hybrid Approach to Value Alignment in Artificial Intelligence. JAIR, 70: 871--890

2021
[22]

Kittock, J. E. 1995. Emergent Conventions and the Structure of Multi-Agent Systems. In Lectures in complex systems: The proceedings ofthe 1993 complex systems summer school, Santa Fe Institute Studies in the Sciences of Complexity Lecture Volume VI, 507--521. Santa Fe Institut...

1995
[23]

Leben, D. 2020. Normative Principles for Evaluating Fairness in Machine Learning. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), 86--92. New York: ACM

2020
[24]

Levy, P.; and Griffiths, N. 2021. Convention Emergence with Congested Resources. In Rosenfeld, A.; and Talmon, N., eds., Multi-Agent Systems, 126--143. Cham: Springer International Publishing

2021
[25]

B.; and Whitney, D

Mann, H. B.; and Whitney, D. R. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics, 18(1): 50 -- 60

1947
[26]

Maranh \ a o, J.; Casini, G.; Pigozzi, G.; and van Der Torre, L. 2022. Normative change: an AGM approach . Journal of Applied Logics - IfCoLoG Journal of Logics and their Applications , 9(4): 855--920

2022
[27]

F.; and Singh, M

Mashayekhi, M.; Ajmeri, N.; List, G. F.; and Singh, M. P. 2022. Prosocial Norm Emergence in Multiagent Systems. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 17(1--2): 3:1--3:24

2022
[28]

Moor, J. 2006. The Nature, Importance, and Difficulty of Machine Ethics. IEEE Intelligent Systems, 21: 18--21

2006
[29]

Morris-Martin, A.; De Vos, M.; and Padget, J. 2019. Norm Emergence in Multiagent Systems: A Viewpoint Paper. Autonomous Agents and Multi-Agent Systems (JAAMAS), 33(6): 706--749

2019
[30]

Mu \ n oz, A.; Billsberry, J.; and Ambrosini, V. 2022. Resilience, robustness, and antifragility: towards an appreciation of distinct organizational responses to adversity. International Journal of Management Reviews, 24: 181--187

2022
[31]

K.; Ajmeri, N.; Jonker, C

Murukannaiah, P. K.; Ajmeri, N.; Jonker, C. M.; and Singh, M. P. 2020. New Foundations of Ethical Multiagent Systems. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 1706--1710. Auckland: IFAAMAS. Blue Sky Ideas Track

2020
[32]

K.; and Singh, M

Murukannaiah, P. K.; and Singh, M. P. 2020. From Machine Ethics to Internet Ethics: Broadening the Horizon. IEEE Internet Computing, 24(3): 51--57

2020
[33]

G.; Balke-Visser, T.; Ajmeri, N.; Kalia, A

Nardin, L. G.; Balke-Visser, T.; Ajmeri, N.; Kalia, A. K.; Sichman, J. S.; and Singh, M. P. 2016. Classifying Sanctions and Designing a Conceptual Sanctioning Process Model for Socio-Technical Systems. The Knowledge Engineering Review (KER), 31: 142--166

2016
[34]

Nashed, S.; Svegliato, J.; and Zilberstein, S. 2021. Ethically Compliant Planning within Moral Communities. In Proceedings of the 4th AAAI/ACM Conference on AI, Ethics, and Society ( AIES ) , 188--198. Virtual Event: ACM

2021
[35]

B.; Svegliato, J.; and Blodgett, S

Nashed, S. B.; Svegliato, J.; and Blodgett, S. L. 2023. Fairness and Sequential Decision Making: Limits, Lessons, and Opportunities. ArXiv, abs/2301.05753: 1--15

2023 arXiv
[36]

A.; Bartocci, E.; Ciabattoni, A.; and Governatori, G

Neufeld, E. A.; Bartocci, E.; Ciabattoni, A.; and Governatori, G. 2022. Enforcing ethical goals over reinforcement-learning policies. Ethics and Information Technology, 24(4): 43

2022
[37]

Oldenburg, N.; and Zhi-Xuan, T. 2024. Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Games. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 1510--1520. Auckland: IFAAMAS

2024
[38]

C.; Protopapas, N.; Yazdanpanah, V.; Gerding, E

Ong, Y. C.; Protopapas, N.; Yazdanpanah, V.; Gerding, E. H.; and Stein, S. 2024. Fair and efficient ride-scheduling: a preference-driven approach. Journal of Simulation, 1(1): 1--17

2024
[39]

Rawls, J. 1958. Justice as Fairness. The Philosophical Review, 67(2): 164--194

1958
[40]

Rawls, J.; and Kelly, E. I. 2001. Justice As Fairness : A Restatement. Online Access with DDA: YBP Pick and Choose. Cambridge: Belknap Press

2001
[41]

Robinson, P. 2023. Moral disagreement and artificial intelligence. AI and Society, 38(3): 1--14

2023
[42]

Savarimuthu, B. T. R.; Cranefield, S.; Purvis, M. A.; and Purvis, M. K. 2013. Identifying prohibition norms in agent societies. Artificial Intelligence and Law, 21(1): 1--46

2013
[43]

Shoham, Y.; and Tennenholtz, M. 1997. On the emergence of social conventions: modeling, analysis, and simulations. Artificial Intelligence, 94(1): 139--166. Economic Principles of Multi-Agent Systems

1997
[44]

S.; and Barto, A

Sutton, R. S.; and Barto, A. G. 2018. Reinforcement learning : an introduction. Adaptive computation and machine learning. Cambridge, Massachusetts: The MIT Press, second edition edition

2018
[45]

B.; and Zilberstein, S

Svegliato, J.; Nashed, S. B.; and Zilberstein, S. 2021. Ethically Compliant Sequential Decision Making. Proceedings of the 35th AAAI Conference on Artificial Intelligence ( AAAI ) , 35(13): 11657--11665

2021
[46]

Tuomela, R. 1995. The Importance of Us: A Philosophical Study of Basic Social Notions. Stanford: Stanford University Press

1995
[47]

Tzeng, S.-T.; Ajmeri, N.; and Singh, M. P. 2022. Fleur: Social Values Orientation for Robust Norm Emergence. In Proceedings of the International Workshop on Coordination, Organizations, Institutions, Norms and Ethics for Governance of Multi-Agent Systems (COINE), 185--200. Vir...

2022
[48]

P.; Du \'e \ n ez-Guzm \'a n, E

Vinitsky, E.; K \"o ster, R.; Agapiou, J. P.; Du \'e \ n ez-Guzm \'a n, E. A.; Vezhnevets, A. S.; and Leibo, J. Z. 2023. A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings. Collective Intelligence, 2(2): 1--14

2023
[49]

Woodgate, J.; and Ajmeri, N. 2022. Macro Ethics for Governing Equitable Sociotechnical Systems. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 1824--1828. Online: IFAAMAS. Blue Sky Ideas Track

2022
[50]

Woodgate, J.; and Ajmeri, N. 2024. Macro Ethics Principles for Responsible AI Systems: Taxonomy and Directions. ACM Computing Surveys, 56(11): 1--37

2024
[51]

Woodgate, J.; Marshall, P.; and Ajmeri, N. 2024. Codebase for Operationalising Rawlsian Ethics for Fairness in Norm-Learning Agents. https://doi.org/10.5281/zenodo.14520386

2024 doi
[52]

Wright . 1963. Norm and Action : A Logical Enquiry . New York: Humanities

1963
[53]

Z.; Iacca, G.; and Wan Lee, S

Yaman, A.; Leibo, J. Z.; Iacca, G.; and Wan Lee, S. 2023. The emergence of division of labour through decentralized social sanctioning. Proceedings of the Royal Society B: Biological Sciences, 290(2009)

2009
[54]

Yu, C.; Zhang, M.; and Ren, F. 2014. Collective Learning for the Emergence of Social Norms in Networked Multiagent Systems. IEEE Transactions on Cybernetics, 44(12): 2342--2355

2014
[55]

Zimmer, M.; Glanois, C.; Siddique, U.; and Weng, P. 2021. Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning. In Meila, M.; and Zhang, T., eds., Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139, 12967--...

2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Agrawal, R.; Ajmeri, N.; and Singh, M. P. 2022. Socially Intelligent Genetic Agents for the Emergence of Explicit Norms. In Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI), 10--14. Vienna: IJCAI

work page 2022

[4] [4]

K.; and Singh, M

Ajmeri, N.; Guo, H.; Murukannaiah, P. K.; and Singh, M. P. 2020. Elessar: Ethics in Norm-Aware Agents. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 16--24. Auckland: IFAAMAS

work page 2020

[5] [5]

M.; Cranefield, S.; and Savarimuthu, B

Anavankot, A. M.; Cranefield, S.; and Savarimuthu, B. T. R. 2023. Towards Norm Entrepreneurship in Agent Societies. In Advances in Practical Applications of Agents, Multi-Agent Systems, and Cognitive Mimetics. The PAAMS Collection, 188--199. Switzerland: Springer

work page 2023

[6] [6]

L.; and Armen, C

Anderson, M.; Anderson, S. L.; and Armen, C. 2004. Towards Machine Ethics. In AAAI-04 Workshop on Agent Organizations: Theory and Practice, 1--7. San Jose: AAAI

work page 2004

[7] [7]

Balakrishnan, S.; Bi, J.; and Soh, H. 2022. SCALES: From Fairness Principles to Constrained Decision-Making. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AIES), 46--55. Oxford: ACM

work page 2022

[8] [8]

Bengio, Y. 2012. Practical recommendations for gradient-based training of deep architectures. CoRR, abs/1206.5533

arXiv 2012