pith. sign in

arxiv: 2604.13705 · v1 · submitted 2026-04-15 · 💻 cs.CL · cs.AI· cs.GT· cs.MA

Beyond Arrow's Impossibility: Fairness as an Emergent Property of Multi-Agent Collaboration

Sayan Kumar Chaki , Antoine Gourru , Julien Velcin This is my paper

Pith reviewed 2026-05-10 13:06 UTC · model grok-4.3

classification 💻 cs.CL cs.AIcs.GTcs.MA
keywords fairnessmulti-agent collaborationlarge language modelsethical decision makinghospital triageArrow's impossibility theorememergent propertiesnegotiation
0
0 comments X

The pith

Fairness emerges when two AI agents negotiate ethical decisions that neither could achieve alone.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that fairness in language models is not best achieved by optimizing a single model but can arise as an emergent outcome of interactions between multiple agents. Using a hospital triage scenario, it shows two agents debating over patient allocations—one guided by an ethical framework via retrieval-augmented generation and the other unaligned or biased—can reach a final decision that meets fairness standards neither agent attains by itself. This matters because as large language models become agentic, single-model fairness optimization may be insufficient, and interaction can lead to better outcomes despite individual biases. The work links this to Arrow's Impossibility Theorem, suggesting multi-agent deliberation navigates inherent conflicts in fairness criteria instead of overcoming them.

Core claim

In a controlled hospital triage framework, two agents negotiate patient allocations across three structured debate rounds. One agent is aligned to an ethical framework using retrieval-augmented generation, while the other is unaligned or prompted to favor certain demographics. The resulting joint allocation satisfies fairness criteria that neither agent's independent allocation meets. Aligned agents moderate biases in their counterparts through contestation rather than full conversion, and the system as a whole serves as the proper unit for evaluating fairness.

What carries the argument

The three-round structured debate negotiation between an RAG-aligned agent and an unaligned or adversarial agent in the hospital triage framework, where contestation allows the joint outcome to exceed individual ethical adequacy.

Load-bearing premise

The specific hospital triage setup with three debate rounds and RAG-based alignment sufficiently captures the dynamics of real-world multi-agent ethical decision-making.

What would settle it

A demonstration that in repeated trials or alternative scenarios without the controlled debate structure, the joint allocations consistently fail to meet fairness criteria that the individual agents also fail to meet.

Figures

Figures reproduced from arXiv: 2604.13705 by Antoine Gourru, Julien Velcin, Sayan Kumar Chaki.

Figure 1
Figure 1. Figure 1: The deliberative arena. Two agents with preference profiles [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
read the original abstract

Fairness in language models is typically studied as a property of a single, centrally optimized model. As large language models become increasingly agentic, we propose that fairness emerges through interaction and exchange. We study this via a controlled hospital triage framework in which two agents negotiate over three structured debate rounds. One agent is aligned to a specific ethical framework via retrieval-augmented generation (RAG), while the other is either unaligned or adversarially prompted to favor demographic groups over clinical need. We find that alignment systematically shapes negotiation strategies and allocation patterns, and that neither agent's allocation is ethically adequate in isolation, yet their joint final allocation can satisfy fairness criteria that neither would have reached alone. Aligned agents partially moderate bias through contestation rather than override, acting as corrective patches that restore access for marginalized groups without fully converting a biased counterpart. We further observe that even explicitly aligned agents exhibit intrinsic biases toward certain frameworks, consistent with known left-leaning tendencies in LLMs. We connect these limits to Arrow's Impossibility Theorem: no aggregation mechanism can simultaneously satisfy all desiderata of collective rationality, and multi-agent deliberation navigates rather than resolves this constraint. Our results reposition fairness as an emergent, procedural property of decentralized agent interaction, and the system rather than the individual agent as the appropriate unit of evaluation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that fairness emerges as a property of multi-agent LLM collaboration rather than single-model optimization. In a controlled hospital triage negotiation with two agents over three structured debate rounds—one aligned via RAG to an ethical framework and the other unaligned or adversarially prompted—the joint final allocation satisfies fairness criteria (e.g., balancing demographic and clinical need) that neither agent achieves in isolation. Alignment moderates bias through contestation, and the results are connected to Arrow's Impossibility Theorem to argue that multi-agent deliberation navigates rather than resolves limits on collective rationality, repositioning the system as the unit of fairness evaluation.

Significance. If the empirical patterns hold under broader conditions, the work offers a valuable shift toward system-level evaluation of fairness in agentic LLMs, with the triage setup providing a concrete, falsifiable testbed for emergent properties. The observation that aligned agents act as 'corrective patches' without full conversion is a nuanced empirical contribution. However, the significance is tempered by the lack of demonstrated robustness, as the central emergence claim rests on a narrow experimental design without ablations or quantitative validation.

major comments (2)
  1. [Abstract and §5 (Discussion)] Abstract and §5 (Discussion): The invocation of Arrow's Impossibility Theorem to frame the limits of multi-agent deliberation is not load-bearing for the central claim. The negotiation consists of iterative debate and allocation adjustments rather than the aggregation of complete, transitive preference rankings over alternatives that Arrow's theorem addresses; this mismatch means the formal connection does not follow from the setup and weakens the 'beyond Arrow's' positioning.
  2. [§3 (Experimental Setup) and §4 (Results)] §3 (Experimental Setup) and §4 (Results): The key finding that joint allocations satisfy fairness criteria neither agent reaches alone depends on the three-round RAG debate producing genuine emergence rather than artifacts of the fixed structure or model priors. No ablation studies varying debate length, alignment method, or domain are reported, and the abstract supplies no sample sizes, statistical tests, or effect sizes; without these, it is unclear whether the fairness gain generalizes or is scenario-specific.
minor comments (2)
  1. [Abstract] Abstract: The statement that 'aligned agents partially moderate bias through contestation' would benefit from a brief concrete example of an allocation change to illustrate the mechanism.
  2. [Throughout] Throughout: Notation for fairness criteria (e.g., how 'ethically adequate' or 'fairness criteria' are operationalized in the triage allocations) should be defined explicitly in a dedicated subsection to aid reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major point below, indicating where revisions will be made.

read point-by-point responses

  1. Referee: [Abstract and §5 (Discussion)] The invocation of Arrow's Impossibility Theorem to frame the limits of multi-agent deliberation is not load-bearing for the central claim. The negotiation consists of iterative debate and allocation adjustments rather than the aggregation of complete, transitive preference rankings over alternatives that Arrow's theorem addresses; this mismatch means the formal connection does not follow from the setup and weakens the 'beyond Arrow's' positioning.

    Authors: We acknowledge the distinction the referee draws between our iterative negotiation protocol and the formal setting of Arrow's theorem, which concerns the aggregation of complete preference orderings. Our reference to the theorem is conceptual, intended to highlight that no single agent (or mechanism) can simultaneously satisfy all fairness desiderata, and that multi-agent deliberation provides a procedural means to navigate these trade-offs. We agree this is an analogy rather than a direct formal mapping. We will revise the abstract and §5 to clarify the interpretive nature of the connection and remove any implication of equivalence, while preserving the discussion of limits on individual optimization. This change does not affect the empirical results. revision: yes

  2. Referee: [§3 (Experimental Setup) and §4 (Results)] The key finding that joint allocations satisfy fairness criteria neither agent reaches alone depends on the three-round RAG debate producing genuine emergence rather than artifacts of the fixed structure or model priors. No ablation studies varying debate length, alignment method, or domain are reported, and the abstract supplies no sample sizes, statistical tests, or effect sizes; without these, it is unclear whether the fairness gain generalizes or is scenario-specific.

    Authors: The referee correctly notes limitations in the reported experimental validation. Our design uses a fixed three-round structure to control for the effects of alignment and contestation in the triage domain, with multiple runs performed to observe consistent patterns. We will revise the abstract to report the number of trials conducted and include basic statistical summaries and effect sizes in §4. Full ablations across debate lengths, alignment methods, and domains would require substantial additional computation; we will therefore add an explicit limitations subsection in §4 discussing this scope and outlining targeted ablation studies as future work. These updates address the concern about potential artifacts while remaining within the bounds of the current study. revision: partial

Circularity Check

0 steps flagged

No significant circularity; empirical observations from simulation stand independently of inputs

full rationale

The paper's central claims rest on post-hoc measurements of allocation outcomes after three-round negotiations in a controlled triage scenario, where one agent uses RAG alignment and the other is unaligned or adversarial. Fairness criteria satisfaction in the joint result is reported as an observed pattern rather than presupposed by the setup definition or by any fitted parameter renamed as a prediction. The Arrow's theorem linkage is presented as an interpretive connection to explain observed limits, not as a formal derivation that reduces the result to the theorem by construction. No self-citations are load-bearing for the emergence claim, and the methodology does not smuggle ansatzes or rename known results via prior author work. The derivation chain is self-contained as an empirical study of interaction effects.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The central claim depends on unverified assumptions about the simulation's fidelity to real ethical dynamics and the effectiveness of RAG alignment; no free parameters or invented entities are explicitly introduced in the abstract.

axioms (3)
  • domain assumption Retrieval-augmented generation can align an agent to a specific ethical framework without introducing its own biases.
    Invoked to establish the 'aligned' agent condition in the triage setup.
  • domain assumption Three rounds of structured debate enable observable bias moderation and emergent fairness through contestation.
    Central premise for observing that joint allocations exceed individual ones.
  • ad hoc to paper The hospital triage scenario with demographic vs. clinical need tradeoffs is representative of general fairness challenges.
    The specific framework is constructed for this study to test the emergence hypothesis.

pith-pipeline@v0.9.0 · 5546 in / 1631 out tokens · 59311 ms · 2026-05-10T13:06:57.398698+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages

  1. [1]

    Bergmann et al

    P. Bergmann et al. MVTec AD -- A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. CVPR, 2019

    work page 2019

  2. [2]

    Bergmann et al

    P. Bergmann et al. Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders. VISAPP, 2019

    work page 2019

  3. [3]

    Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE TIP, 2004

    work page 2004

  4. [4]

    Girshick, J

    R. Girshick, J. Donahue, T. Darrell, J. Malik. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. CVPR, 2014

    work page 2014

  5. [5]

    Girshick

    R. Girshick. Fast R-CNN. ICCV, 2015

    work page 2015

  6. [6]

    T.-Y. Lin, M. Maire, S. Belongie, et al. Microsoft COCO: Common Objects in Context. ECCV, 2014

    work page 2014

  7. [7]

    J. R. R. Uijlings et al. Selective Search for Object Recognition. IJCV, 2013

    work page 2013

  8. [8]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

    work page

  9. [9]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

    work page

  10. [10]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

    work page

  11. [11]

    fuF( 6MIU 5Gc(B 9 wS4h&jt TŪ

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page 2025