REVIEW 6 cited by
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
As Large Language Models (LLMs) continue to evolve, they are increasingly being employed in numerous studies to simulate societies and execute diverse social tasks. However, LLMs are susceptible to societal biases due to their exposure to human-generated data. Given that LLMs are being used to gain insights into various societal aspects, it is essential to mitigate these biases. To that end, our study investigates the presence of implicit gender biases in multi-agent LLM interactions and proposes two strategies to mitigate these biases. We begin by creating a dataset of scenarios where implicit gender biases might arise, and subsequently develop a metric to assess the presence of biases. Our empirical analysis reveals that LLMs generate outputs characterized by strong implicit bias associations (>= 50\% of the time). Furthermore, these biases tend to escalate following multi-agent interactions. To mitigate them, we propose two strategies: self-reflection with in-context examples (ICE); and supervised fine-tuning. Our research demonstrates that both methods effectively mitigate implicit biases, with the ensemble of fine-tuning and self-reflection proving to be the most successful.
Forward citations
Cited by 6 Pith papers
-
Breaking Down Bias: On The Limits of Generalizable Pruning Strategies
Pruning-based bias removal in Llama-3-8B reduces racial bias mainly in the context used to choose what to prune, and transfers poorly across contexts.
-
Are Human Interactions Replicable by Generative Agents? A Case Study on Pronoun Usage in Hierarchical Interactions
In a leader/subordinate discussion simulation, most LLM agents do not reproduce the human pronoun pattern, and knowing the pattern does not help them show it.
-
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
A large collaborative survey organizes LLM and LLM-agent safety issues into a full-stack lifecycle framework from data preparation to deployment.
-
Unmasking Conversational Bias in AI Multiagent Systems
In simulated echo-chamber chats, conservative-aligned LLM agents often shift to liberal-aligned messages, a drift that one-shot questionnaire tests do not detect.
-
Bias Mitigation Agent: Optimizing Source Selection for Fair and Balanced Knowledge Retrieval
A multi-agent retrieval system that filters sources by a bias classifier reports an 81.82% relative drop in bias rate, but the evaluation uses the same classifier as the filter.
-
Multi-agent Systems for Misinformation Lifecycle : Detection, Correction And Source Identification
A conceptual multi-agent architecture for classifying, detecting, correcting, and sourcing misinformation is proposed but not implemented or evaluated.
Discussion (0). Continue with ORCID to comment.