REVIEW 7 cited by
Do as We Do, Not as You Think: the Conformity of Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in large language models (LLMs) revolutionize the field of intelligent agents, enabling collaborative multi-agent systems capable of tackling complex problems across various domains. However, the potential of conformity within these systems, analogous to phenomena like conformity bias and groupthink in human group dynamics, remains largely unexplored, raising concerns about their collective problem-solving capabilities and possible ethical implications. This paper presents a comprehensive study on conformity in LLM-driven multi-agent systems, focusing on three aspects: the existence of conformity, the factors influencing conformity, and potential mitigation strategies. In particular, we introduce BenchForm, a new conformity-oriented benchmark, featuring reasoning-intensive tasks and five distinct interaction protocols designed to probe LLMs' behavior in collaborative scenarios. Several representative LLMs are evaluated on BenchForm, using metrics such as conformity rate and independence rate to quantify conformity's impact. Our analysis delves into factors influencing conformity, including interaction time and majority size, and examines how the subject agent rationalizes its conforming behavior. Furthermore, we explore two strategies to mitigate conformity effects, i.e., developing enhanced personas and implementing a reflection mechanism. Several interesting findings regarding LLMs' conformity are derived from empirical results and case studies. We hope that these insights can pave the way for more robust and ethically-aligned collaborative AI systems. Our benchmark and code are available at BenchForm.
Forward citations
Cited by 7 Pith papers
-
Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks
Across six open-weight LLMs and seven datasets, a speaker-free wrong-answer assertion alone flips 66.5% of initially correct answers, versus 10.3% for a plain re-ask; source labels mainly add a modest increment above ...
-
Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systems
Showing LLM agents precomputed rankings of their peers' sycophancy improves multi-agent discussion accuracy by ~10.5 absolute points and reduces agreement with incorrect user stances.
-
LLM Abstention Can Be a Prompt Artifact, in Addition to Genuine Uncertainty
Adding an extra 'Unknown' option to True/False prompts causes LLMs to abstain on questions they can answer, and random words reproduce the effect, indicating abstention is partly a prompt artifact.
-
Herd Behavior: Investigating Peer Influence in LLM-based Multi-Agent Systems
LLM agents flip their answers more when their own confidence is low and their peer seems confident, and the format and order of peer information can amplify or dampen this herd behavior.
-
Multi-Agent Synergy-Driven Iterative Visual Narrative Synthesis
A three-stage multi-agent system with reflective chain-of-thought, learned layout generation, and iterative visual critique outperforms prior document-to-slide methods on content, coherence, and design metrics, alongs...
-
Towards Simulating Social Influence Dynamics with LLM-based Multi-agents
In simulated BBS-style discussions, reasoning-focused LLM agents show lower conformity and more persistent dissent than standard generative models, though no human baseline validates the simulation.
- Social Pressure Breaks Majority Voting in LLM Safety Panels
Discussion (0). Sign in to comment.