pith. sign in

arxiv: 2604.15078 · v1 · submitted 2026-04-16 · 💻 cs.AI

Where are the Humans? A Scoping Review of Fairness in Multi-agent AI Systems

Simeon Allmendinger , Luca Deck , Lucas Mueller This is my paper

Pith reviewed 2026-05-10 11:38 UTC · model grok-4.3

classification 💻 cs.AI
keywords multi-agent AIfairnessscoping reviewAI ethicsagent autonomynormative foundationssystem interactionslifecycle integration
0
0 comments X

The pith

Fairness in multi-agent AI systems is typically treated superficially without normative foundations and must instead be integrated structurally across the full development lifecycle.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines how fairness is handled in multi-agent AI systems that combine multiple autonomous agents, often powered by generative models. Through a review of 23 studies it shows that existing work usually applies fairness ideas in limited ways after systems are built, without clear ethical grounding or attention to how agents interact and make decisions together. A reader would care because these systems are becoming more common and their decisions can affect people in interconnected ways that single-agent fairness fixes do not cover. The authors conclude that fairness needs to be planned into every stage of creation, supported by human oversight and explicit statements about who benefits and what fairness means.

Core claim

The central claim is that a qualitative analysis of 23 studies reveals five common patterns in how fairness is approached in multi-agent AI, yet these patterns consistently remain surface-level, rest on weak or absent ethical principles, and ignore the effects of agent autonomy and collective system behavior. The authors state that fairness cannot be added at the end but must be built into the entire lifecycle, which also requires clear human involvement to define objectives and identify the people or groups whose interests count.

What carries the argument

The scoping review and qualitative content analysis that groups prior work into five archetypal approaches, exposing gaps in normative depth and system-level consideration.

If this is right

  • Evaluation of fairness in these systems requires explicit human oversight to define goals and beneficiaries.
  • Normative clarity must be established before technical fairness methods are chosen.
  • Fairness objectives need precise statements about which agents or humans are affected and in what way.
  • Ignoring interactions among autonomous agents leads to incomplete fairness assessments at the system level.
  • Post-hoc fairness additions are insufficient once agent autonomy and collective dynamics are present.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the claim holds, standards and regulations for multi-agent systems would need to require fairness checks at every development phase rather than only at deployment.
  • Design teams may need new methods or checklists that treat agent interactions as a core fairness variable from the start.
  • The review's emphasis on human oversight could extend to questions of accountability when multiple agents jointly produce outcomes.
  • This pattern of superficial fairness treatment might appear in other emerging AI architectures that involve distributed decision-making.

Load-bearing premise

The 23 chosen studies give a fair picture of all research on fairness in multi-agent AI and the analysis correctly spots superficial treatment without missing important counterexamples or interpretive bias.

What would settle it

A follow-up review that locates and analyzes a substantial set of multi-agent AI papers containing detailed normative justifications, explicit fairness goals for multiple agents, and evidence of fairness measures applied from the earliest design stages would directly test the claim of widespread superficial treatment.

read the original abstract

Rapid advances in Generative AI are giving rise to increasingly sophisticated Multi-Agent AI (MAAI) systems. While AI fairness has been extensively studied in traditional predictive scenarios, its examination in MAAI remains nascent and fragmented. This scoping review critically synthesizes existing research on fairness in MAAI systems. Through a qualitative content analysis of 23 selected studies, we identify five archetypal approaches. Our findings reveal that fairness in MAAI systems is often addressed superficially, lacks robust normative foundations, and frequently overlooks the complex dynamics introduced by agent autonomy and system-level interactions. We argue that fairness must be embedded structurally throughout the development lifecycle of MAAI, rather than appended as a post-hoc consideration. Meaningful evaluation requires explicit human oversight, normative clarity, and a precise articulation of fairness objectives and beneficiaries. This review provides a foundation for advancing fairness research in MAAI systems by highlighting critical gaps, exposing prevailing limitations, and suggesting pathways.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper presents a scoping review of fairness in Multi-Agent AI (MAAI) systems. It performs qualitative content analysis on 23 selected studies, identifies five archetypal approaches to addressing fairness, and concludes that fairness is typically treated superficially, lacks robust normative foundations, and overlooks agent autonomy and system-level dynamics. The authors argue that fairness must be embedded structurally throughout the MAAI development lifecycle rather than as a post-hoc addition, with requirements for explicit human oversight, normative clarity, and precise fairness objectives.

Significance. If the synthesis holds, the review provides a useful map of an emerging subfield, identifies recurring limitations in current MAAI fairness work, and offers a high-level framework for integrating fairness earlier in system design. This could help orient researchers working at the intersection of multi-agent systems and AI ethics.

major comments (3)
  1. [Methods] The methods description provides no search strategy, databases queried, date ranges, keywords, or explicit inclusion/exclusion criteria for arriving at the final set of 23 studies. This information is load-bearing for the claim that the sample supports general statements about the field being 'nascent and fragmented' and for the reliability of the five-archetype synthesis.
  2. [Methods / Results] The qualitative content analysis lacks reported inter-rater reliability metrics, coding scheme details, or explicit decision rules for labeling studies as 'superficial,' 'lacking normative foundations,' or overlooking 'agent autonomy and system-level interactions.' Without these, the central interpretive claims rest on unstated criteria and are vulnerable to selection or confirmation bias.
  3. [Results] The five archetypal approaches are presented as the main empirical output, yet the paper does not supply a table or appendix mapping each of the 23 studies to archetypes with supporting excerpts or justification. This makes it impossible to evaluate whether the typology is exhaustive or whether the 'superficial' characterization is consistently applied.
minor comments (2)
  1. [Abstract] The abstract states the number of studies and the main findings but omits any mention of the search process or analysis method, which is conventional for scoping reviews and would help readers assess scope immediately.
  2. [Introduction] Notation and terminology for 'MAAI' and fairness concepts are introduced without a dedicated glossary or consistent definition across sections, which could confuse readers unfamiliar with multi-agent terminology.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback on our scoping review. The comments highlight important areas for improving methodological transparency, and we have revised the manuscript accordingly to address them while preserving the integrity of our synthesis.

read point-by-point responses

  1. Referee: [Methods] The methods description provides no search strategy, databases queried, date ranges, keywords, or explicit inclusion/exclusion criteria for arriving at the final set of 23 studies. This information is load-bearing for the claim that the sample supports general statements about the field being 'nascent and fragmented' and for the reliability of the five-archetype synthesis.

    Authors: We agree that detailed reporting of the search protocol is essential for reproducibility in a scoping review. The original submission summarized the selection process at a high level for brevity. In the revised manuscript, we have expanded the Methods section to fully document the search strategy, including the databases queried (ACM Digital Library, IEEE Xplore, Scopus, Web of Science, and arXiv), the date range (2015–2023), the complete keyword strings and Boolean operators used, and the explicit inclusion/exclusion criteria applied to arrive at the final set of 23 studies. These additions directly support the claims regarding the nascent state of the field. revision: yes

  2. Referee: [Methods / Results] The qualitative content analysis lacks reported inter-rater reliability metrics, coding scheme details, or explicit decision rules for labeling studies as 'superficial,' 'lacking normative foundations,' or overlooking 'agent autonomy and system-level interactions.' Without these, the central interpretive claims rest on unstated criteria and are vulnerable to selection or confirmation bias.

    Authors: We acknowledge that greater detail on the qualitative analysis process would strengthen the paper. The analysis followed an inductive approach with the lead author performing initial coding and the full team meeting to discuss and reach consensus on classifications. We have now added to the Methods section a description of the coding scheme, including how codes for fairness approaches, normative foundations, and autonomy considerations were developed, along with explicit decision rules (e.g., a study is labeled 'superficial' when fairness is mentioned without operationalization or metrics; 'lacking normative foundations' when no ethical principles or theories are referenced). We note the absence of formal inter-rater reliability statistics (such as Cohen’s kappa) as a limitation, given the small-team, consensus-based process, and provide illustrative coding examples. revision: partial

  3. Referee: [Results] The five archetypal approaches are presented as the main empirical output, yet the paper does not supply a table or appendix mapping each of the 23 studies to archetypes with supporting excerpts or justification. This makes it impossible to evaluate whether the typology is exhaustive or whether the 'superficial' characterization is consistently applied.

    Authors: We appreciate this recommendation for enhancing evaluability. We have added a new appendix (Appendix A) containing a table that maps each of the 23 studies to one of the five archetypes. The table includes, for every study, the assigned archetype, a concise justification, and verbatim excerpts from the source paper that informed the classification. This addition allows readers to assess the consistency and exhaustiveness of the typology directly. revision: yes

Circularity Check

0 steps flagged

No circularity: external synthesis from independent studies

full rationale

This scoping review derives its claims about superficial fairness treatment, missing normative foundations, and the need for structural embedding solely from qualitative content analysis of 23 external studies. No self-definitional loops, fitted parameters renamed as predictions, load-bearing self-citations, uniqueness theorems, or ansatzes appear in the derivation. The central synthesis is not equivalent to its inputs by construction; it reports patterns observed in cited literature. Representativeness and interpretive bias are validity issues, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper introduces no free parameters, new entities, or mathematical constructs. Its claims rest on the domain assumption that the reviewed studies adequately capture the state of MAAI fairness research.

axioms (1)
  • domain assumption The 23 studies identified through the scoping process are representative of existing research on fairness in multi-agent AI systems.
    Scoping reviews depend on the assumption that the search and selection yield a sufficient sample to support field-wide conclusions.

pith-pipeline@v0.9.0 · 5465 in / 1246 out tokens · 26644 ms · 2026-05-10T11:38:59.296347+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages

  1. [1]

    While AI fairness has been extensively studied in traditional predictive scenarios, its examination in MAAI remains nascent and fragmented

    Thirty-Fourth European Conference on Information Systems (ECIS 2026), Milan, Italy 1 WHERE ARE THE HUMANS? A SCOPING REVIEW OF FAIRNESS IN MULTI-AGENT AI SYSTEMS Completed Research Paper Simeon Allmendinger, University of Bayreuth & Fraunhofer FIT, Bayreuth, Germany, simeon.allmendinger@uni-bayreuth.de Luca Deck, University of Bayreuth & Fraunhofer FIT, B...

    work page 2026

  2. [2]

    and financial analysis, e.g., Moody’s employs MAAI systems in their day-to-day operations (Moody’s, 2024). While fairness issues such as unjustified discrimination are well-documented in analytical AI systems (De‐Arteaga et al., 2022), fairness research on generative AI, particularly Large Language Models (LLMs), is only starting to gain traction (Friedri...

    work page 2024

  3. [3]

    fairness desiderata

    that synthesizes existing research and maps out the normative foundations as well as methodological approaches for operationalization and evaluation of fairness in MAAI systems. On this basis, we discuss how existing research contributes to established fairness goals and needs (“fairness desiderata”) from interdisciplinary literature (Deck, Schoeffer, et ...

    work page 2024

  4. [4]

    perspective—a departure from prior work focused predominantly on single-agent or model-centric fairness. Second, it conceptually systematizes the field by introducing a framework, through which it identifies five recurring fairness archetypes—Normative Delegation, Fairness Facade, Fairness Schooling, Petri Dish Fairness, and Fairness Effectiveness. In thi...

    work page 2024

  5. [5]

    and AI legislation (Deck, Müller, et al., 2024; European Commission & High Level Expert Group on Artificial Intelligence, 2019). A noteworthy characteristic of fairness is that it subsumes a range of desiderata (i.e., stakeholder needs an AI system is expected to satisfy) from multiple interdisciplinary perspectives (Deck, Schomäcker, et al., 2024; Mullig...

    work page 2024

  6. [6]

    This has raised calls for effective mechanisms which require (i) concrete specification of fairness objective (Deck, Schomäcker, et al., 2024; Langer et al., 2024; Sterz et al.,

    as well as several conflicting fairness objectives (Friedler et al., 2021; Mulligan et al., 2019). This has raised calls for effective mechanisms which require (i) concrete specification of fairness objective (Deck, Schomäcker, et al., 2024; Langer et al., 2024; Sterz et al.,

    work page 2021

  7. [7]

    WHERE ARE THE HUMANS? Thirty-Fourth European Conference on Information Systems (ECIS 2026), Milan, Italy 3 Figure

    and (ii) rigorous evaluation of the claimed effects on fairness (Chen, 2023; Deck, Schoeffer, et al., 2024). WHERE ARE THE HUMANS? Thirty-Fourth European Conference on Information Systems (ECIS 2026), Milan, Italy 3 Figure

    work page 2023

  8. [8]

    This schema illustrates an AI-automated workflow with n tasks: A human initiator (A) submits an input request, which is processed by a Multi-Agent AI (B)

    A Human-Centric Perspective on MAAI systems. This schema illustrates an AI-automated workflow with n tasks: A human initiator (A) submits an input request, which is processed by a Multi-Agent AI (B). This can include AI-based agents and human agents who may or may not be the original initiator. It can also contain interfaces to external data and services....

    work page 2024

  9. [9]

    system architectures composed of multiple autonomous or semi-autonomous AI-based agents that collaboratively perceive, reason, and act […] within shared, dynamic environments

    interacting with a human, our work assumes a broader perspective. Fairness in MAAI emerges—or fails—within the interactions of multiple AI-based agents. An MAAI system may produce unfair outcomes even if the underlying foundation models used by its AI-based agents behave fairly in isolation. This is because AI-based agent coordination, negotiation, and em...

    work page 2025

  10. [10]

    Zhang et al., 2025)), and post-processing approaches to rectify undesired outputs after the model has been deployed (e.g., based on human instructions(Friedrich et al., 2025))

    or reinforcement learning (Y. Zhang et al., 2025)), and post-processing approaches to rectify undesired outputs after the model has been deployed (e.g., based on human instructions(Friedrich et al., 2025)). As previous research has primarily focused on singular models and formal aspects of fairness, such as feature distributions and metrics, this leaves a...

    work page 2025

  11. [11]

    An initial exploratory review of the literature was conducted prior to finalizing the search strategy, in order to gain insights into the WHERE ARE THE HUMANS? Thirty-Fourth European Conference on Information Systems (ECIS 2026), Milan, Italy 5 Figure

    work page 2026

  12. [12]

    AND” operator: fairness, multi agent, and Large Language Model based AI. For fairness and ethics, we used the terms “fair*

    PRISMA flowchart illustrating the literature review process. domain, assess keyword effectiveness, and identify relevant publishers. Our search string was designed to capture various components of MAAI systems while ensuring relevance to fairness-related contexts. The search string consisted of three parts, which are connected by the “AND” operator: fairn...

    work page 2025

  13. [13]

    Figure 2 visualizes how the initial set of 455 records (as of January

    and the PRISMA framework (Page et al., 2021), we ensured a transparent and reproducible selection process. Figure 2 visualizes how the initial set of 455 records (as of January

    work page 2021

  14. [14]

    Out of these, 3 met the inclusion criteria, resulting in a final set of 23 records

    After that, a backward and forward search was conducted, identifying 78 entries in total. Out of these, 3 met the inclusion criteria, resulting in a final set of 23 records. On the screening level, we manually reviewed each paper based on its abstract. To be included in this review, a paper had to address fairness norms in the context of an LLM-based MAAI...

    work page 2024

  15. [15]

    The high share of recently published and non-peer-reviewed papers highlights the novelty and growing interest in fairness in MAAI systems

    As of March 2026, eight of the 20 arXiv papers have since appeared in peer-reviewed venues, three as posters at NeurIPS, one as a poster at ICLR, and four in WHERE ARE THE HUMANS? Thirty-Fourth European Conference on Information Systems (ECIS 2026), Milan, Italy 6 peer-reviewed journals. The high share of recently published and non-peer-reviewed papers hi...

    work page 2026

  16. [16]

    simplistic

    Overview of existing literature classified using a morphological box along five dimensions. A darker color indicates more occurrences of certain characteristics. WHERE ARE THE HUMANS? Thirty-Fourth European Conference on Information Systems (ECIS 2026), Milan, Italy 7 5 Results: Archetypes of Fairness in MAAI As MAAI systems increasingly mediate decision-...

    work page 2026

  17. [17]

    Shared cells (multiple colors) indicate papers exhibiting characteristics of more than one value within dimension

    Archetypes categorized via the morphological box. Shared cells (multiple colors) indicate papers exhibiting characteristics of more than one value within dimension. Gray cells indicate papers that do not define a specific archetype, while white cells highlight research gaps. WHERE ARE THE HUMANS? Thirty-Fourth European Conference on Information Systems (E...

    work page 2026

  18. [18]

    utilize GPT-4o to compare the output of a single agent and an MAAI using thematic analysis. Combined with their hierarchical clustering and further analysis, they conclude that their MAAI system is more trustworthy and therefore more aligned with ethical principles, making it superior to the single-agent system. Central to the argument of Cerqueira et al....

    work page 2024

  19. [19]

    Simple Ethical Questions

    Extracted archetypes addressing the fairness desiderata of MAAI systems. unsupported. Q&A employs sets of ethical questions with predefined correct answers to evaluate the system based on accuracy. For example, Becker et al. (2024) uses the “Simple Ethical Questions” benchmark (Sitelew et al. 2021), in which agents must decide if historical personalities ...

    work page 2024

  20. [20]

    agreeable

    introduces a benchmark to assess fairness, revealing subtle patterns of discrimination—such as assigning technical or leadership roles based on perceived gender or ethnicity—that are absent in traditional single-agent systems. Although they describe the rationale behind their fairness benchmark, they fail to specify the concrete implementation or to provi...

    work page 2024

  21. [21]

    fairness desiderata

    normatively grounding its approach and explicitly aiming to teach fairness to agents with fine-tuning (normatively grounded → teach, 1 out of 5; all others → Teach, 1 out of 15). Despite this limited evidence, Fairness Schooling may present a nascent but promising shift—positioning fairness as a central design principle. 6 Discussion In this section, we r...

    work page 2024

  22. [22]

    explicitly incorporates a normatively grounded concept of fairness into the creation of an AI-based agent by finetuning a lightweight LLM. However, Fairness Schooling requires accessible LLMs, which is not the case for closed-sourced models, such as the market-leading products of OpenAI, Google, and Anthropic (Xiao et al., 2024). In contrast, the archetyp...

    work page 2024

  23. [23]

    [b]lindly applying fairness constraints [...] may greatly compromise the overall effectiveness of the solution method

    warn that “[b]lindly applying fairness constraints [...] may greatly compromise the overall effectiveness of the solution method.” In contrast, most studies of the archetype Fairness Effectiveness show that fairness measures can actually increase task performance, particularly in formal game settings. For example, Tennant et al. (2024) demonstrate that ag...

    work page 2024

  24. [24]

    References Allmendinger, S., Bonenberger, L., Endres, K., Fetzer, D., Gimpel, H., & Kühl, N

    and evolve from convenience-driven artifacts into truly responsible technologies. References Allmendinger, S., Bonenberger, L., Endres, K., Fetzer, D., Gimpel, H., & Kühl, N. (2025). Multi-Agent AI. https://doi.org/10.13140/RG.2.2.33104.21769 Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of S...

    work page doi:10.13140/rg.2.2.33104.21769 2025

  25. [25]

    https://doi.org/10.3389/frai.2020.00034 Cerqueira, J. A. S. de, Agbese, M., Rousi, R., Xi, N., Hamari, J., & Abrahamsson, P. (2024, October 25). Can We Trust AI Agents? An Experimental Study Towards Trustworthy LLM-Based Multi-Agent Systems for AI Ethics (Number arXiv:2411.08881). arXiv. Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krash...

    work page doi:10.3389/frai.2020.00034 2020

  26. [26]

    Danaher, J

    https://doi.org/10.1057/s41599-023-02079-x Cheng, R., Ma, H.-X., Cao, S., Li, J., Pei, A., Wang, Z., Ji, P., Wang, H., & Huo, J. (2024, November 15). Reinforcement Learning from Multi-role Debates as Feedback for Bias Mitigation in LLMs. Pluralistic Alignment Workshop at NeurIPS

    work page doi:10.1057/s41599-023-02079-x 2024

  27. [27]

    (2024, September 24)

    https://openreview.net/forum?id=aqLPNGxgTS Cisneros-Velarde, P. (2024, September 24). On the Principles behind Opinion Dynamics in Multi-Agent Systems of Large Language Models (Number arXiv:2406.15492). arXiv. Cohen, G. A. (1979). Karl Marx’s theory of history. Princeton University Press. De‐Arteaga, M., Feuerriegel, S., & Saar‐Tsechansky, M. (2022). Algo...

    work page doi:10.1111/poms.13839 2024

  28. [28]

    https://doi.org/10.1145/3670865.3673632 Li, J., & Li, G. (2025). Triangular Trade-off between Robustness, Accuracy, and Fairness in Deep Neural Networks: A Survey. ACM Comput. Surv., 57(6), 140:1-140:40. https://doi.org/10.1145/3645088 Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024). A survey on LLM-based multi-agent systems: Workflow, infrastructur...

    work page doi:10.1145/3670865.3673632 2025

  29. [29]

    https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Verma, S., & Rubin, J. (2018). Fairness definitions explained. Proceedings of the International Workshop on Software Fairness, FairWare ’18, 1–7. https://doi.org/10.1145/3194770.3194776 Wohlin, C., Kalinowski, M., Romero Felizardo, K., & Mendes, E. (2...

    work page doi:10.1145/3194770.3194776 2017