Pith. sign in

REVIEW 2 major objections 7 minor 1 cited by

Most apparent LLM conformity survives after the peer is removed: the repeated wrong answer alone flips correct answers far more than speaker labels do.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 06:00 UTC pith:SLBBFRC2

load-bearing objection Clean, well-controlled result: most measured LLM “conformity” is a speaker-free floor from the asserted answer itself, and the field should report that floor before crediting social influence. the 2 major comments →

arxiv 2607.05545 v1 pith:SLBBFRC2 submitted 2026-07-06 cs.CL cs.AI

Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks

Yibo Hu , Jiaming Qu This is my paper
classification cs.CL cs.AI
keywords LLM conformityspeaker-free floorpeer-pressure benchmarksharmful revisionsource attributionprompt artifactsmulti-agent systemsillusory truth
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard LLM conformity benchmarks mix two cues at once: an explicit speaker and a repeated answer. This paper holds the asserted answer fixed and removes the speaker, creating a no-source control. Across six open-weight models and seven QA and reasoning datasets, that bare assertion alone flips initially correct answers in 66.5% of cases, versus 10.3% under a plain re-ask. Source framing mainly modulates this floor: expert panels raise it, while minimal person labels often do not. The reason a reader should care is methodological and practical: without first measuring what remains after the speaker is gone, benchmarks and multi-agent systems can mistake repeated text for social influence.

Core claim

Most of what looks like LLM conformity does not require an explicit speaker. Holding the wrong answer fixed and deleting the speaker produces a speaker-free floor of 66.5% harmful revision of initially correct answers, more than six times the plain re-ask rate. The floor persists under paraphrase and in open-ended settings with options hidden. Source framing mainly adds a modest increment above that floor: expert-panel framing raises it by about 12.9 percentage points, while bare person labels do not reliably raise it. When models flip they are usually confidently wrong, and simple recalibration does not restore the original answer. Conformity should therefore be reported as a source-attribu

What carries the argument

The no-source condition: the same asserted answer with the explicit speaker removed, compared against a framing ladder (people, rich peers, experts) in a deterministic two-read arbitration protocol under greedy decoding. Harmful revision rate (HRR) is the fraction of initially correct answers flipped after a single inserted block, so any change is attributable to that text.

Load-bearing premise

The load-bearing premise is that a bare line such as "The answer is X," with no named speaker, is free of implicit source or authority, so the large flip rate can be cleanly credited to the answer text rather than to a hidden speaker the model still reads into the prompt.

What would settle it

If the same two-read protocol showed no-source harmful revision near the plain re-ask baseline (around 10%) across models and datasets while expert-panel framing still produced large flips, the claim that most conformity needs no speaker would be false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Conformity benchmarks should report four quantities: plain re-ask stability, the no-source floor, labeled-source revision, and the source-attributed increment.
  • A count of agreeing sources is not by itself evidence of independent agreement, because repeated identical assertions can rival distinct speakers.
  • Retrieval and multi-agent pipelines that re-insert the same claim under different labels risk treating echoed text as corroboration.
  • When models flip under pressure they are typically confidently wrong, so confidence gates and simple temperature rescaling do not recover the original answer.
  • Minimal person labels are neither necessary for a large floor nor sufficient to raise it; evidential framing (expert panel, retrieved reference) is what amplifies revision.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the floor is mainly informational updating on repeated text, work that frames conformity or sycophancy as social deference may be targeting the wrong mechanism for much of the observed behavior.
  • A defensive pattern suggested by the results is to strip source labels from untrusted context and compare against a source-scrubbed paraphrase of the same content before treating multi-source agreement as independent evidence.
  • Because the authority increment shrinks when answer options are hidden while the floor stays large, constrained multiple-choice formats may inflate the measured social component relative to free-form use.
  • Within-family size increased susceptibility here; if that pattern generalizes, larger models may need stronger content-level controls rather than relying on scale alone.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper argues that standard LLM conformity benchmarks confound two cues—explicit speaker attribution and the repeated asserted answer—so total revision under peer framing cannot be read as social influence. Using a deterministic two-read log-probability arbitration protocol that holds the asserted answer fixed, the authors introduce a no-source control (bare assertions with no named person, group, status, or majority). Across six open-weight models (1.5–9B) and seven QA/reasoning datasets, no-source alone produces 66.5% harmful revision of initially correct answers, versus 10.3% under plain re-ask and 19.7% under a length-matched neutral control; expert-panel framing raises the rate only to 79.4% (+12.9 pp). The floor survives paraphrase, off-ceiling items, open-ended hidden-option evaluation, random wrong targets, and non-conversational containers, while an invalid-label placebo collapses near baseline. Minimal person labels do not reliably raise the floor; evidential framings (expert panel, retrieved reference) do. When models flip they are confidently wrong, and temperature rescaling does not restore the original ranking. The methodological recommendation is to report the speaker-free floor and the source-attributed increment separately before crediting revision to social influence.

Significance. If the result holds, this is a load-bearing methodological correction for the growing LLM conformity, multi-agent, and retrieval-influence literatures: those literatures have largely varied speaker and assertion together and therefore risk mistaking repeated answer text for peer pressure. The contribution is a clean measurement design rather than a new theory of social cognition. Strengths that should be credited explicitly include the fixed-answer source ladder, plain re-ask and length baselines, invalid-label placebo, paraphrase and open-ended robustness checks, token-matched source-noun minimal pairs, non-conversational containers, repeated-versus-distinct dose contrast, mixed-effects inference for the authority increment, and public code/data. The recommended four-quantity reporting practice (re-ask stability, no-source floor, labeled-source revision, source-attributed increment) plus a repeated-versus-distinct control is immediately actionable for subsequent benchmarks.

major comments (2)
  1. [§3.2 / Appendix A.2 / Table 1] §3.2 and Appendix A.2: the primary no-source template still includes the residual preamble “The following text appeared before your final answer,” which is not a named speaker but is still a system-level framing of the inserted block. Container conditions (retrieved reference / webpage / corrupted log) and the token-matched bare assertion in Table 3 adequately show that a large floor does not require that preamble, but the main-grid 66.5% figure is not itself the fully scrubbed condition. For the central claim that the floor is speaker-free, the paper should either (i) promote a no-preamble bare-assertion cell into the main Table 1 aggregate, or (ii) state more explicitly in §4.1 that the headline floor is an upper bound that may still include residual prompt-author framing, with containers as the cleaner lower-bound evidence.
  2. [Abstract / §4 / Limitations] §4.2–§5 and Limitations: the abstract and title generalize to “Most LLM Conformity,” while the measurement is restricted to open-weight 1.5–9B instruction-tuned models, greedy decoding (T=0), and a mostly multiple-choice single-turn protocol (with one open-ended anchor check). The Limitations section already flags frontier systems, stochastic decoding, and multi-turn pipelines as open. Because the paper’s methodological lesson is aimed at the whole conformity-benchmark literature, the abstract and conclusion should scope the empirical claim more carefully (e.g., “in the open-weight models and single-turn settings we test”) so that the strong measurement result is not read as already established for frontier multi-agent deployments.
minor comments (7)
  1. [Figure 1 / §3.1] Figure 1 and Table 1: the +12.9 pp expert increment is clear, but the figure caption’s “A→B” icon could briefly note that w is the model’s own top non-gold option under p0, so readers do not assume an arbitrary distractor.
  2. [Table 3 / §4.2] Table 3 vs Table 1: absolute HRR levels in the token-matched source-noun experiment (bare 97%) are far above the main-grid no-source rate (66.5%). The text correctly says levels are not comparable; a one-sentence note on the design difference (anchor models/datasets, six identical lines, different prefix packaging) would prevent misreading.
  3. [Figure 4 / §4.3] Figure 4 / Table 10: the decline of repeated-assertion HRR with N is attributed to “many identical lines start to look like a perturbation.” That reading is plausible but post hoc; flagging it as speculative in the caption would help.
  4. [§4.4] §4.4 / Appendix D.1: “simple recalibration does not undo the flip” is supported for temperature rescaling of Round-2 logits; avoid language that could be read as covering all recalibration or unlearning methods.
  5. [Appendix D.2 / §4.4] Appendix D.2 justification probe: Cohen’s κ=0.65 overall / 0.75 on the diagnostic split is acceptable for a supporting analysis; state sample size for the human re-coding (n=60) in the main text when the probe is first mentioned so readers can weight it.
  6. [§2] Related Work: the separation from sycophancy is clear; a short pointer that the floor may interact with known option-order and illusory-truth effects (already cited) would situate the result for readers outside the conformity subliterature.
  7. [§1 / global] Typos / polish: “associal confor-” line break in §1; ensure consistent hyphenation of “speaker-free” and “no-source” across abstract, figures, and tables.

Circularity Check

0 steps flagged

No significant circularity: empirical floor-vs-increment measurement with external baselines, not a derivation that redefines its target as its input.

full rationale

This is a controlled measurement paper, not a first-principles derivation. Harmful revision rate is defined as HRR = P(a1 ≠ t | a0 = t) under a two-read protocol with greedy decoding; the no-source condition is an operational insertion of bare asserted-answer text with explicit speaker labels removed. The central claim (66.5% no-source HRR vs 10.3% plain re-ask, with expert framing adding +12.9 pp) is an empirical dissociation against external baselines (plain re-ask, length control, invalid-label placebo) and stress tests (paraphrase, open-ended, containers, random-string prefix). Naming the measured no-source rate a “speaker-free floor” and recommending that source effects be reported as increments above it is definitional packaging of a measured quantity, not circular reduction of the result to its inputs. Citation of Qu et al. (2026) supplies the item pool and all-wrong/mixed/all-correct structures; that is ordinary setup reuse with author overlap, not a load-bearing uniqueness theorem or ansatz that forces the floor claim. No fitted parameter is re-labeled as a prediction; no self-definitional loop equates the floor to the social increment by construction. Score 0 is the honest finding.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claim rests on operational measurement choices rather than free physical constants or new ontological entities. The main commitments are: revision is attributable to a single inserted block under greedy decoding; no-source removes explicit source attribution; harmful revision on initially correct items is the primary quantity; and open-weight option-probability reads are adequate for the methodological conclusion. Invented constructs are operational labels (speaker-free floor, no-source), not postulated mechanisms with independent physical existence.

free parameters (4)
  • assertion_count_N = 6 (main grid)
    Main grid uses six asserted lines; dose experiment varies N in {1,2,3,6}. The headline 66.5% floor is tied to this design choice even though the qualitative floor appears at lower N.
  • off_ceiling_confidence_threshold = 0.9
    Off-ceiling analyses restrict to c0 < 0.9; the threshold is a hand-chosen cut that affects reported robustness rates though not the main floor claim.
  • wrong_target_selection_rule = argmax non-gold under Round-1 option distribution
    Primary all-wrong pressure pushes the model’s own top non-gold option under p0 rather than a fixed distractor; random-target control is secondary. This choice can raise absolute HRR while the paper argues ordering is preserved.
  • decoding_temperature = 0
    Greedy decoding (T=0) is required so shifts are attributed to inserted text; stochastic regimes are left open and could change absolute rates.
axioms (4)
  • domain assumption A change between Round-1 and Round-2 option distributions under greedy decoding is attributable to the single inserted assertion block rather than sampling noise or unmodeled context drift.
    Stated in Section 3.1 and Appendix A; load-bearing for causal attribution of HRR to the cue.
  • ad hoc to paper Removing named persons, groups, status, and majority language yields a speaker-free condition even if the model may still treat inserted text as evidence.
    Operational definition in Sections 2 and 3.2; the paper explicitly scopes 'speaker-free' this way and uses containers/random-string controls as support.
  • domain assumption Harmful revision rate on initially correct items, plus beneficial revision and Δp_target, are adequate primary outcomes for evaluating conformity-style pressure.
    Section 3.6; standard for this literature but still a modeling choice about what counts as conformity-relevant behavior.
  • domain assumption Open-weight instruction-tuned models with readable option-token probabilities are sufficient to support methodological claims about conformity benchmarks more broadly.
    Limitations section acknowledges frontier/hosted systems as future work; generalization beyond 1.5–9B open models is assumed for the broader lesson.
invented entities (2)
  • speaker-free floor (no-source harmful revision rate) independent evidence
    purpose: Quantifies residual revision after explicit source attribution is removed while the asserted answer is held fixed; becomes the baseline against which source increments are measured.
    Operational construct defined by the experimental contrast, not a latent cognitive module with independent existence. Independent evidence is the set of behavioral controls (placebo collapse, paraphrase, open-ended, containers).
  • source-attributed increment independent evidence
    purpose: Difference between labeled-source revision and the no-source floor; intended as the measurable social/source component of conformity prompts.
    Derived quantity from the framing ladder; useful reporting recommendation rather than a new physical entity.

pith-pipeline@v1.1.0-grok45 · 23113 in / 3701 out tokens · 36236 ms · 2026-07-11T06:00:58.679069+00:00 · methodology

0 comments
read the original abstract

LLM conformity is often used to describe cases where a model changes a correct answer toward a peer or group response. We show that most of this apparent conformity survives even after the peer is removed. The reason is a confound: standard conformity prompts mix two cues at once, the presence of a speaker and the repeated wrong answer itself. Existing benchmarks vary these cues together, so they cannot tell how much of the revision actually depends on the speaker. We introduce a no-source condition: the same asserted answer with the explicit speaker removed. Across six open-weight LLMs and seven QA and reasoning datasets, this condition alone causes harmful revision in $66.5\%$ of initially correct cases, compared with $10.3\%$ under a plain re-ask. The effect also remains when the repeated answer is paraphrased and when answer options are hidden in an open-ended setting. Source framing mainly modulates this floor: expert-panel framing raises it, while minimal person labels do not reliably raise it. When models flip, they are usually confidently wrong, and simple recalibration does not recover the original answer. Source attribution still matters, but it should be measured as an increment above this speaker-free floor. The methodological lesson is that conformity benchmarks should first measure what remains after the speaker is removed; without this step, benchmarks may mistake repeated text for social influence.

Figures

Figures reproduced from arXiv: 2607.05545 by Jiaming Qu, Yibo Hu.

Figure 1
Figure 1. Figure 1: Most apparent LLM “conformity” survives without an explicit speaker. Removing the speaker leaves a 66.5% harmful revision rate, compared with 79.4% under the strongest expert-panel framing and 10.3% under a plain re-ask with no inserted content. The strongest expert-panel framing adds only +12.9 pp above the no-source floor. Aggregated over six models and seven datasets; the A→B icon illustrates a multiple… view at source ↗
Figure 2
Figure 2. Figure 2: Experimental design. Step 1: four conditions assert the same answer (B); only the source wrapper varies, from no explicit speaker (no-source) up to a panel of experts. The no-source condition isolates what remains once the speaker is removed. Step 2: a deterministic two-read protocol reads the model’s answer before and after a single inserted block; under greedy decoding (T=0) any change between the two re… view at source ↗
Figure 3
Figure 3. Figure 3: The speaker-free floor appears in every model. Harmful revision rate (%) under all-wrong pres￾sure, by model and framing (NS = no-source). No￾source revision is substantial in all six models, and ex￾perts exceed no-source in each. in an open-ended setting, and a randomly chosen wrong target, holding in the mid-60s to high-70s throughout. The one change that collapses it is asserting an option that does not… view at source ↗
Figure 4
Figure 4. Figure 4: Repeated assertions can mimic majority pressure. Harmful revision vs. the number of wrong￾answer lines N, for a repeated identical assertion (one “voice”) vs. N distinct speakers. Anchor models and datasets; absolute levels not comparable to the main grid. because the bare assertion already sits near the top with no source clause at all, the no-source condi￾tion is not quietly importing a hidden speaker. W… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. When Local Monitors Miss Compositional Harm: Diagnosing Distributed Backdoors in Multi-Agent Systems

    cs.CR 2026-07 accept novelty 7.0

    Once attack fragments are locally benign and ε-indistinguishable from benign traffic, no local monitor can separate them (TPR−FPR≤ε); the signal reappears only in the right assembled representation.

Reference graph

Works this paper leans on

53 extracted references · 21 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Solomon E Asch. 1955. Opinions and social pressure. Scientific american, 193(5):31--35

    1955

  2. [2]

    Ariel Flint Ashery, Luca Maria Aiello, and Andrea Baronchelli. 2024. https://arxiv.org/abs/2410.08948 Emergent social conventions and collective bias in llm populations . Preprint, arXiv:2410.08948

    Pith/arXiv arXiv 2024

  3. [3]

    Anooshka Bajaj and Zoran Tiganj. 2026. Who do LLMs trust? human experts matter more than other LLMs . arXiv preprint arXiv:2602.13568

    arXiv 2026

  4. [4]

    Sushil Bikhchandani, David Hirshleifer, and Ivo Welch. 1992. A theory of fads, fashion, custom, and cultural change as informational cascades. Journal of Political Economy, 100(5):992--1026

    1992

  5. [5]

    Mikako Bito, Keita Nishimoto, Kimitaka Asatani, and Ichiro Sakata. 2026. Large language models exhibit normative conformity. arXiv preprint arXiv:2604.19301

    Pith/arXiv arXiv 2026

  6. [6]

    Brucks and Olivier Toubia

    Melanie S. Brucks and Olivier Toubia. 2025. https://doi.org/10.1371/journal.pone.0319159 Prompt architecture induces methodological artifacts in large language models . PLOS ONE, 20(4):e0319159

    doi:10.1371/journal.pone.0319159 2025

  7. [7]

    Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, and Dan Jurafsky. 2025. https://arxiv.org/abs/2505.13995 ELEPHANT : Measuring and understanding social sycophancy in LLMs . Preprint, arXiv:2505.13995

    Pith/arXiv arXiv 2025

  8. [8]

    Young-Min Cho, Sharath Chandra Guntuku, and Lyle Ungar. 2025. Herd behavior: Investigating peer influence in llm-based multi-agent systems. arXiv preprint arXiv:2505.21588

    Pith/arXiv arXiv 2025

  9. [9]

    Junhyuk Choi, Jeongyoun Kwon, Heeju Kim, Haeun Cho, Hayeong Jung, Sehee Min, and Bugeun Kim. 2026. Belief in authority: Impact of authority in multi-agent evaluation framework. arXiv preprint arXiv:2601.04790

    arXiv 2026

  10. [10]

    Min Choi, Keonwoo Kim, Sungwon Chae, and Sangyeop Baek. 2025. An empirical study of group conformity in multi-agent systems. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5123--5139

    2025

  11. [11]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    Pith/arXiv arXiv 2018

  12. [12]

    Giordano De Marzo, Alessandro Bellina, Claudio Castellano, Viola Priesemann, and David Garcia. 2026. https://arxiv.org/abs/2605.10721 Conformity generates collective misalignment in ai agents societies . Preprint, arXiv:2605.10721

    Pith/arXiv arXiv 2026

  13. [13]

    Morton Deutsch and Harold B Gerard. 1955. A study of normative and informational social influences upon individual judgment. The journal of abnormal and social psychology, 51(3):629

    1955

  14. [14]

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Forty-first international conference on machine learning

    2024

  15. [15]

    John R. P. French and Bertram H. Raven. 1959. The bases of social power. In Dorwin Cartwright, editor, Studies in Social Power, pages 150--167. University of Michigan, Institute for Social Research, Ann Arbor, MI

    1959

  16. [16]

    Federico Germani and Giovanni Spitale. 2025. https://arxiv.org/abs/2505.13488 Source framing triggers systematic evaluation bias in large language models . Preprint, arXiv:2505.13488

    Pith/arXiv arXiv 2025

  17. [17]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    Pith/arXiv arXiv 2024

  18. [18]

    Chen Han, Jin Tan, Bohan Yu, Wenzhen Zheng, and Xijin Tang. 2026. Conformity dynamics in llm multi-agent systems: The roles of topology and self-social weighting. arXiv preprint arXiv:2601.05606

    arXiv 2026

  19. [19]

    Lynn Hasher, David Goldstein, and Thomas Toppino. 1977. Frequency and the conference of referential validity. Journal of Verbal Learning and Verbal Behavior, 16(1):107--112

    1977

  20. [20]

    Yibo Hu, Yu Lin, Erick Skorupa Parolin, Latifur Khan, and Kevin Hamlen. 2022. Controllable fake document infilling for cyber deception. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6505--6519

    2022

  21. [21]

    Shomik Jain, Charlotte Park, Matt Viana, Ashia Wilson, and Dana Calacci. 2026. https://arxiv.org/abs/2509.12517 Interaction context often increases sycophancy in LLMs . In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026)

    arXiv 2026

  22. [22]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    Pith/arXiv arXiv 2023

  23. [23]

    Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, and Jindong Wang. 2024. Agentreview: Exploring peer review dynamics with llm agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1208--1226

    2024

  24. [24]

    Herbert C. Kelman. 1958. Compliance, identification, and internalization: Three processes of attitude change. Journal of Conflict Resolution, 2(1):51--60

    1958

  25. [25]

    Esben Kran, Hieu Minh Nguyen, Akash Kundu, Sami Jawhar, Jinsuk Park, and Mateusz Jurewicz. 2025. Darkbench: Benchmarking dark patterns in large language models. In International Conference on Learning Representations, volume 2025, pages 44988--45008

    2025

  26. [26]

    Philippe Laban, Lidiya Murakhovs'ka, Caiming Xiong, and Chien-Sheng Wu. 2023. https://arxiv.org/abs/2311.08596 Are you sure? challenging llms leads to performance drops in the flipflop experiment . Preprint, arXiv:2311.08596

    Pith/arXiv arXiv 2023

  27. [27]

    Bibb Latan \'e . 1981. The psychology of social impact. American psychologist, 36(4):343

    1981

  28. [28]

    Stephan Lewandowsky, Ullrich KH Ecker, Colleen M Seifert, Norbert Schwarz, and John Cook. 2012. Misinformation and its correction: Continued influence and successful debiasing. Psychological science in the public interest, 13(3):106--131

    2012

  29. [29]

    Yuxuan Li, Xinwei Guo, Jiashi Gao, Guanhua Chen, Xiangyu Zhao, Jiaxin Zhang, Quanying Liu, Haiyan Wu, Xin Yao, and Xuetao Wei. 2025. Llms trust humans more, that’s a problem! unveiling and mitigating the authority bias in retrieval-augmented generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

    2025

  30. [30]

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 17889--17904

    2024

  31. [31]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers), pages 3214--3252

    2022

  32. [32]

    Joshua Liu, Aarav Jain, Soham Takuri, Srihan Vege, Aslihan Akalin, Kevin Zhu, Sean O'Brien, and Vasu Sharma. 2025. https://arxiv.org/abs/2503.11656 Truth decay: Quantifying multi-turn sycophancy in language models . Preprint, arXiv:2503.11656

    Pith/arXiv arXiv 2025

  33. [33]

    Andreas Madsen, Sarath Chandar, and Siva Reddy. 2024. https://arxiv.org/abs/2401.07927 Are self-explanations from large language models faithful? In Findings of the Association for Computational Linguistics: ACL 2024

    Pith/arXiv arXiv 2024

  34. [34]

    Ali Mahmoodi, Hamed Nili, Dan Bang, Carsten Mehring, and Bahador Bahrami. 2022. Distinct neurocomputational mechanisms support informational and socially normative conformity. PLoS biology, 20(3):e3001565

    2022

  35. [35]

    Aliakbar Mehdizadeh and Martin Hilbert. 2025. When your ai agent succumbs to peer-pressure: Studying opinion-change dynamics of llms. arXiv preprint arXiv:2510.19107

    arXiv 2025

  36. [36]

    Stanley Milgram. 1963. Behavioral study of obedience. The Journal of abnormal and social psychology, 67(4):371

    1963

  37. [37]

    Jeremy Perez, Grgur Kovac, Corentin Leger, Cedric Colas, Gaia Molinaro, Maxime Derex, Pierre-Yves Oudeyer, and Clement Moulin-Frier. 2025. https://arxiv.org/abs/2407.04503 When LLMs play the telephone game: Cultural attractors as conceptual tools to evaluate LLMs in multi-turn settings . In Proceedings of the 13th International Conference on Learning Repr...

    arXiv 2025

  38. [38]

    Pouya Pezeshkpour and Estevam Hruschka. 2024. Large language models sensitivity to the order of options in multiple-choice questions. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2006--2017

    2024

  39. [39]

    Jiaming Qu, Lucheng Fu, and Yibo Hu. 2026. https://arxiv.org/abs/2606.01637 Easier to mislead than to correct: Harmful and beneficial revision in llm conformity . Preprint, arXiv:2606.01637. ARR May 2026 submission

    Pith/arXiv arXiv 2026

  40. [40]

    Jakob Schuster, Vagrant Gautam, and Katja Markert. 2026. Whose facts win? LLM source preferences under knowledge conflicts. arXiv preprint arXiv:2601.03746

    Pith/arXiv arXiv 2026

  41. [41]

    Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R

    Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. 2024. https://arxiv.org/abs/2310.13548 Towards understanding syc...

    Pith/arXiv arXiv 2024

  42. [42]

    Adi Simhi, Fazl Barez, Martin Tutek, Yonatan Belinkov, and Shay B. Cohen. 2026. https://arxiv.org/abs/2603.03308 Old habits die hard: How conversational history geometrically traps llms . Preprint, arXiv:2603.03308

    Pith/arXiv arXiv 2026

  43. [43]

    Maojia Song, Tej Deep Pala, Ruiwen Zhou, Weisheng Jin, Amir Zadeh, Chuan Li, Dorien Herremans, and Soujanya Poria. 2025. LLMs can't handle peer pressure: Crumbling under multi-agent social interactions. arXiv preprint arXiv:2508.18321. KAIROS benchmark

    arXiv 2025

  44. [44]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and 1 others. 2023. Challenging big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13003--13051

    2023

  45. [45]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    Pith/arXiv arXiv 2024

  46. [46]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. https://arxiv.org/abs/2305.04388 Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting . In Advances in Neural Information Processing Systems 36 (NeurIPS)

    Pith/arXiv arXiv 2023

  47. [47]

    Daniel Vennemeyer, Phan Anh Duong, Tiffany Zhan, and Tianyu Jiang. 2025. https://arxiv.org/abs/2509.21305 Sycophancy is not one thing: Causal separation of sycophantic behaviors in llms . Preprint, arXiv:2509.21305

    arXiv 2025

  48. [48]

    Keyu Wang, Jin Li, Shu Yang, Zhuoran Zhang, and Di Wang. 2025. https://arxiv.org/abs/2508.02087 When truth is overridden: Uncovering the internal origins of sycophancy in large language models . Preprint, arXiv:2508.02087

    arXiv 2025

  49. [49]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266--95290

    2024

  50. [50]

    Zhiyuan Weng, Guikun Chen, and Wenguan Wang. 2025. Do as we do, not as you think: the conformity of large language models. arXiv preprint arXiv:2501.13381

    Pith/arXiv arXiv 2025

  51. [51]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115

    Pith/arXiv arXiv 2024

  52. [52]

    Huixin Zhong, Yanan Liu, Qi Cao, Shijin Wang, Zijing Ye, Zimu Wang, and Shiyao Zhang. 2025. https://arxiv.org/abs/2508.14918 Disentangling the drivers of llm social conformity: An uncertainty-moderated dual-process mechanism . Preprint, arXiv:2508.14918

    Pith/arXiv arXiv 2025

  53. [53]

    Xiaochen Zhu, Caiqi Zhang, Tom Stafford, Nigel Collier, and Andreas Vlachos. 2025. Conformity in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3854--3872

    2025