Pith. sign in

REVIEW 3 major objections 4 minor 101 references

Social Pressure Breaks Majority Voting in LLM Safety Panels

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A shared wrong “unsafe” peer message makes a six-reviewer panel flag every benign item it evaluates.

desk verdict A clean empirical demonstration that shared wrong-label peer context can destroy majority-vote protection in LLM safety panels, provided the unreported parse-failure exclusions don't bias the 100% headline. read the letter →

arxiv 2608.04415 v1 pith:5TAX4LGE submitted 2026-08-05 cs.CL

Yibo Hu , Jiaming Qu This is my paper · ORCID
classification cs.CL
keywords socialconformitymajorityvotingLLMsafetypanelsfalsealarmssycophancycontentmoderationaggregationfailuremulti-agentreviewers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single shared message, in which six simulated peers assert the wrong “unsafe” label, can destroy the benefit of aggregating six LLM reviewers by majority vote. Across six open-weight models and six safety datasets, the cue raises the average reviewer false-alarm rate on benign content from 56.5% under silent peers to 87.5%, and the six-reviewer majority then flags every benign item included in the complete-panel analysis. The authors argue the failure begins before aggregation: every reviewer shifts in the same direction, so majority voting has no diverse errors left to correct. The shift is strongly asymmetric—reviewers adopt pushes toward “unsafe” about 75% of the time but pushes toward “safe” only about 17%—so the panel’s false alarms rise sharply while its rate of missing harmful content changes little. The practical point is that panels should be tested under the exact shared context their reviewers will receive, not just on solo accuracy and model diversity.

What carries the argument

The central object is a two-round reviewer-panel protocol: each reviewer judges an item alone, then judges the same item again after one inserted message in which six simulated peer lines either assert a label or abstain, and the final judgments are combined by strict majority vote (at least four of six). The paper isolates the effect of a complete shared cue by measuring shifts in verdict and severity relative to the SILENT-PEERS control, which matches the second ask and the presence of peer lines but asserts no label. The load-bearing identity is the Poisson–binomial majority tail $P_{\text{maj}}=\sum_{k=4}^{6}\binom{6}{k}p^{k}(1-p)^{6-k}$ for equal false-alarm rates, generalized to each reviewer’s own marginal rate; the observed panel rate matching this prediction shows that the shared message acts by moving the per-reviewer marginals, not by adding correlation at the voting stage.

What would settle it

Run the same two-round protocol while constraining all six reviewers to emit parseable verdicts on every benign item, so no trial is excluded, and recompute the wrong-peer panel false-alarm rate per dataset; if any dataset’s rate falls below 100%, the strongest version of the headline claim fails.

Watch

Extended reading notes

Core claim

The central discovery is a quantitative reversal in what majority voting does. Under a silent-peer control, the six-reviewer majority lowers benign false alarms relative to the average reviewer (43.0% panel rate versus 56.5% average reviewer rate), reproducing the usual aggregation benefit. After six simulated peers assert the wrong “unsafe” label, the average reviewer false-alarm rate rises to 87.5% and the panel rate reaches 100%, with every evaluated benign unit flagged in each of the six datasets. The observed panel rate matches the Poisson–binomial independence prediction computed from the shifted per-reviewer rates, showing that the failure is inherited from the reviewers’ changed individual judgments rather than created by the voting rule itself. The effect is directional: pooled adoption is 75.3% for flag-directed pushes and 16.8% for safe-directed pushes, so the panel failure appears as systematic over-flagging while harmful-miss rates change little (6.3% to 7.7% at the panel level). A verdict-level probe on four proprietary models shows wide variation, with some models flipping nearly every benign item they had judged safe.

Load-bearing premise

The headline panel rates are computed only on trials where all six reviewers returned parseable verdicts in every condition compared, and the paper does not report how many trials were dropped or whether dropping correlates with condition or item difficulty, so the 100% false-alarm number could be biased if excluded trials differ.

Editorial extensions

If this is right

  • Pre-deployment checks that measure only solo accuracy or model-family diversity will miss the failure mode, because the same panel that reduces false alarms under silent peers flags every benign complete-panel item after a shared wrong-label message.
  • Because flag-directed pushes are adopted about 75% of the time and safe-directed pushes only about 17%, shared context biased toward “unsafe” will appear as systematic over-flagging rather than a balanced redistribution of errors.
  • The panel result is predictable from per-reviewer false-alarm rates after the message, so reporting the shifted reviewer marginals and the resulting independence-predicted panel rate is a sufficient and simple diagnostic.
  • Follow-up prompt instructions are not a reliable repair: the best tested prompt recovered 44% of shifted verdicts, and the ordering of recovery across prompt types was not consistent across datasets and shift directions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test reviewer selection as a mitigation; the strong flag-versus-safe asymmetry implies that a panel composed of reviewers with low flag-push adoption might preserve most of the aggregation benefit, and that selection rule could be evaluated with the same two-round probe.
  • If parse-failures excluded more item-seeds under one condition than another, the headline 100% panel rate could be sensitive to that attrition; reporting parse-failure rates per condition would settle whether the excluded units behave differently.
  • Because the panel rate after the message matches the independence prediction from shifted marginals, one can screen candidate reviewers without running a full panel: measure each reviewer’s benign false-alarm rate under the shared context, compute the Poisson–binomial majority tail, and inspect whether it approaches 100% before deployment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper studies whether a shared misleading peer message can break majority voting in LLM safety panels. In a controlled two-round experiment, six open-weight LLMs judge items from six safety datasets alone and again after one of four inserted messages (wrong-label peers, right-label peers, a claimed senior authority, or a silent-peer control), and the paper measures how far each reviewer shifts toward the asserted label and how a strict six-reviewer majority vote behaves. The headline findings are that the wrong-label peer message raises the average reviewer false-alarm rate from 56.5% to 87.5% and the panel false-alarm rate to 100%, while harmful-miss rates change little; that flag-directed pushes are adopted far more often than safe-directed pushes (75.3% versus 16.8%); and that proprietary models show wide variation, with prompt-based recovery only partial. The authors argue that the panel failure is explained by shifted per-reviewer marginals before aggregation, using a Poisson-binomial independence prediction.

Significance. If the results withstand scrutiny, this is a practically important contribution: it identifies a concrete failure mode for LLM safety panels and proposes a simple pre-deployment screening diagnostic. The study has genuine strengths: a matched silent-peer control that separates message content from re-querying, per-reviewer marginals measured directly rather than fitted to the panel outcome, bootstrap confidence intervals, robustness across 20 three-member subpanels, multiple message wordings, and the inclusion of a proprietary-model probe. The paper also ships code and data and is candid about several limitations. The central caveat is that the headline quantitative claims are computed on complete-panel units whose parse-failure exclusions are not reported, so the 100% figure is not yet fully established from the published data.

major comments (3)
  1. [Appendix E, Table 3] The headline 100% panel false-alarm rate is computed only on complete-panel units, defined in Appendix E as item-seeds for which all six reviewers returned a parseable verdict under every condition compared, yet the manuscript never reports how many item-seeds were excluded per dataset, condition, or reviewer, nor whether exclusion correlates with condition or item difficulty. Table 3 reports only surviving denominators (273/273, 300/300, 318/318, etc.), and Table 4 and Appendix F use the same restricted units, so the independence prediction cannot correct for selection. If a reviewer who resists the shared cue is more likely to emit an unparseable response, such as a refusal outside the requested JSON schema, the denominator is censored and the observed 100% panel rate and the 56.5% to 87.5% reviewer shift could both be inflated. Please report the exclusion counts and re-run the headline analyses under worst-case assumptions about the excluded units, for example by treating unparseable outputs as non-flags.
  2. [Abstract, Conclusion, §3.3] The wording 'every evaluated benign unit' and 'the panel false-alarm rate to 100%' overstates the reported result, because the result is restricted to complete-panel item-seeds rather than to all benign items in the six datasets. Unless the exclusion analysis called for above shows the omissions are ignorable, the abstract and conclusion should either state '100% of complete-panel units' or report the fraction of all benign item-seeds covered by those units. This is load-bearing because readers will naturally take the 100% as a claim about all benign items in the evaluated datasets.
  3. [§3.2 and Limitations] The directional asymmetry (75.3% versus 16.8%) compares two different eligible pools: flag-directed pairs are reviewer–item pairs with a silent safe verdict, while safe-directed pairs are those with a silent unsafe verdict. The pools therefore differ in item mix and reviewer composition, so the +58.5 point gap is not a within-item treatment contrast, and the abstract's wording that reviewers follow pushes toward unsafe far more than pushes toward safe is stronger than this design alone supports. The Limitations paragraph acknowledges the point, but the abstract and §3.2 present the pooled gap without the caveat. Please provide a matched within-item comparison where both directions are tested on the same items, or explicitly restrict the directional claim to the stated eligibility rule.
minor comments (4)
  1. [Appendix D] The prompt reproduction contains a likely formatting error: the sentence 'after the full chat template we append the fixed prefix {"judgment":"( and score the next token' appears to have an unmatched brace or truncated string; please verify that the prompt text matches what was actually sent.
  2. [Appendix A] The phrase 'flagged rate' is used to describe the gold-label positive rate in the sampled subset; consider using 'positive rate' to avoid confusion with the panel's flagging outcome.
  3. [Figure 1 caption] The caption says percentages are computed using the full six-reviewer panels, but the 43% silent-peer value is the pooled panel false-alarm rate rather than a per-item proportion; clarify that the 100% refers to complete-panel units and report the total number of units behind the figure.
  4. [Appendix B and §3.1] The near-ceiling silent-peer severity for gemma-2-9B and OLMo-2-7B is correctly distinguished from resistance in the text, but Table 2 and the surrounding discussion could state this distinction more prominently so readers do not read the small severity rises as evidence that those models resist the message.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all headline quantities are directly measured and the panel check is an independent consistency test.

full rationale

The derivation chain is self-contained. The per-reviewer false-alarm rates (56.5% under silent peers, 87.5% under wrong peers) are observed verdicts, not fitted values; the panel false-alarm rates (43.0% and 100%) are separate majority-vote tallies of those same observed verdicts. The Poisson-binomial "independence prediction" in Table 4 is computed from the six measured reviewer-specific marginals and then compared with the observed panel rate; no parameter is fitted to the panel outcome, so the match is a genuine consistency check rather than a construction. The asymmetry finding (75.3% versus 16.8%) is also a direct conditional measurement whose eligibility rule is stated in Appendix E. No load-bearing self-citation, uniqueness theorem, or ansatz-smuggling citation appears: the cited Condorcet jury theorem and conformity literature are external and standard. The complete-panel-unit definition in Appendix E raises a possible selection-bias question because parse-failure exclusions are not enumerated, but that is a data-validity concern rather than circularity: it does not make any predicted quantity equal to an input by definition.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted in this empirical study; all quantities are measured reviewer verdicts, severities, and log-probabilities, plus standard combinatorial calculations. The central claim rests on three assumptions: the simulated peer message proxies real shared context, the complete-panel-unit restriction is unbiased, and the independence benchmark is an appropriate decomposition. No new entities are postulated.

assumptions (3)
  • domain assumption The six simulated peer lines are an adequate operationalization of shared social context in safety panels.
    The manipulation is a single-turn prompt with six named peers asserting a label. Real panel shared context (debate transcripts, shared summaries, prior votes) may differ in interactivity and authority structure. The central claim depends on this proxy; the paper discloses this in Limitations but does not test live interactions.
  • domain assumption Complete-panel units are representative of all evaluated items.
    Panel statistics use only item-seeds where all six reviewers produced a parseable verdict under every compared condition. If parseability correlates with item difficulty or condition, the 100% false-alarm result may be biased. The paper does not report parse-failure rates or excluded counts.
  • standard math Majority-vote aggregation can be benchmarked with reviewers' marginal error rates under an independence model (Poisson-binomial).
    The Poisson-binomial tail is computed from measured per-reviewer marginals as a null benchmark. This is a standard probabilistic calculation, not a fitted model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Social Pressure Breaks Majority Voting in LLM Safety Panels." pith.science (2026). https://pith.science/paper/5TAX4LGE

@misc{pith2026260804415,
  author       = {Pith},
  title        = {Pith review of: Social Pressure Breaks Majority Voting in LLM Safety Panels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5TAX4LGE}},
  note         = {Machine review of arXiv:2608.04415}
}
read the original abstract

Large language models (LLMs) are increasingly used to detect unsafe content. A common approach is to combine judgments from a panel of models to correct individual mistakes, but this benefit may disappear when every model sees the same misleading context before voting. We study this risk in a controlled two-round experiment. Each model first judges an item alone, then judges it again after six simulated peers either assert the wrong label or abstain. We combine the final judgments by majority vote. Across six open-weight LLMs and six datasets, we find that the wrong-label peer message raises the average reviewer false-alarm rate from 56.5% under silent peers to 87.5%, and majority voting raises the panel false-alarm rate to 100%. Without an asserted label, the same panel outperforms its average member. The effect is strongly asymmetric: reviewers follow pushes toward "unsafe" far more than pushes toward "safe" (about 75% versus 17%), so the panel's false-alarm rate rises sharply while its harmful-miss rate changes little. The proprietary-model probe shows substantial variation across models. These results identify susceptibility to shared social cues as a failure mode of safety panels and provide a simple pre-deployment diagnostic.

Figures

Figures reproduced from arXiv: 2608.04415 by the authors.

Figure 1
Figure 1. A shared wrong cue collapses majority￾vote protection. With no asserted peer label, reviewer errors differ and a six-reviewer majority falsely flags 43% of benign items. When every reviewer receives the same incorrect “it’s unsafe” cue, their judgments shift in the same direction and the panel falsely flags 100% of benign items in each of the six datasets. The three reviewer boxes are schematic; percentages are comp… view at source ↗
Figure 2
Figure 2. The experimental setup. A reviewer judges each input twice, alone (Round 1) and again after one inserted message (Round 2), and we measure the shift toward the asserted label. The message takes one of four forms: three assert a label (WRONG-PEERS, RIGHT-PEERS, AUTHORITY) and one is a matched control (SILENT-PEERS). A panel aggregates several such reviewers by majority vote. ing still reduce errors after every review… view at source ↗
Figure 3
Figure 3. One shared message flips every reviewer on a plainly benign item. Six reviewers judge a benign XSTest cooking question. Answering alone, all six call it safe; after one shared message in which six peers assert the unsafe label, all six flip to unsafe, three of them sharply (severity 8). Bars show 0–10 severity; a verdict can flip while severity barely moves (Llama-3.1- 8B). complete WRONG-PEERS message relative to t… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Reviewers adopt a flag push far more than a safe push. Rate at which reviewers switch to the asserted label, by push direction, among reviewer–item pairs where the silent verdict disagreed with it (pooled over six reviewers and BeaverTails, XSTest, Ethics). A flag push…
Figure 6
Figure 6. Figure 6: The same manipulation across four propri￾etary OpenAI models, ordered by release. Benign flip-to-unsafe rate under a wrong-peer message (verdict level, n ≈ 40 per cell; bars are Wilson 95% CIs; silent￾peer controls stay below 10%), on BeaverTails, XSTest, and Ethics. E…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

101 extracted references · 8 canonical work pages

  1. [1]

    Journal of Economic perspectives , volume=

    Cognitive reflection and decision making , author=. Journal of Economic perspectives , volume=. 2005 , publisher=

    2005

  2. [2]

    , author=

    Considering the opposite: a corrective strategy for social judgment. , author=. Journal of personality and social psychology , volume=. 1984 , publisher=

    1984

  3. [3]

    Scientific american , volume=

    Opinions and social pressure , author=. Scientific american , volume=. 1955 , publisher=

    1955

  4. [4]

    , author=

    A study of normative and informational social influences upon individual judgment. , author=. The journal of abnormal and social psychology , volume=. 1955 , publisher=

    1955

  5. [5]

    , author=

    Behavioral study of obedience. , author=. The Journal of abnormal and social psychology , volume=. 1963 , publisher=

    1963

  6. [6]

    , author=

    The psychology of social impact. , author=. American psychologist , volume=. 1981 , publisher=

    1981

  7. [7]

    , author=

    Sources of the continued influence effect: When misinformation in memory affects later inferences. , author=. Journal of experimental psychology: Learning, memory, and cognition , volume=. 1994 , publisher=

    1994

  8. [8]

    Psychological science in the public interest , volume=

    Misinformation and its correction: Continued influence and successful debiasing , author=. Psychological science in the public interest , volume=. 2012 , publisher=

    2012

Show all 101 references
  1. [9]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Conformity in large language models , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  2. [10]

    arXiv preprint arXiv:2501.13381 , year=

    Do as We Do, Not as You Think: the Conformity of Large Language Models , author=. arXiv preprint arXiv:2501.13381 , year=

    arXiv

  3. [11]

    Herd Behavior: Investigating Peer Influence in

    Cho, Young-Min and Guntuku, Sharath Chandra and Ungar, Lyle , journal=. Herd Behavior: Investigating Peer Influence in

  4. [12]

    When Your

    Mehdizadeh, Aliakbar and Hilbert, Martin , journal=. When Your

  5. [13]

    arXiv preprint arXiv:2601.04790 , year=

    Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework , author=. arXiv preprint arXiv:2601.04790 , year=

  6. [14]

    Conformity Dynamics in

    Han, Chen and Tan, Jin and Yu, Bohan and Zheng, Wenzhen and Tang, Xijin , journal=. Conformity Dynamics in

  7. [15]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    An Empirical Study of Group Conformity in Multi-Agent Systems , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  8. [16]

    Li, Yuxuan and Guo, Xinwei and Gao, Jiashi and Chen, Guanhua and Zhao, Xiangyu and Zhang, Jiaxin and Liu, Quanying and Wu, Haiyan and Yao, Xin and Wei, Xuetao , booktitle=

  9. [17]

    Justice or Prejudice? Quantifying Biases in

    Jiayi Ye and Yanbo Wang and Yue Huang and Dongping Chen and Qihui Zhang and Nuno Moniz and Tian Gao and Werner Geyer and Chao Huang and Pin-Yu Chen and Nitesh V Chawla and Xiangliang Zhang , booktitle=. Justice or Prejudice? Quantifying Biases in

  10. [18]

    arXiv preprint arXiv:2604.19301 , year=

    Large Language Models Exhibit Normative Conformity , author=. arXiv preprint arXiv:2604.19301 , year=

    arXiv

  11. [19]

    Studies in Social Power , editor=

    The bases of social power , author=. Studies in Social Power , editor=. 1959 , publisher=

    1959

  12. [20]

    Sociometry , volume=

    Influence of a consistent minority on the responses of a majority in a color perception task , author=. Sociometry , volume=. 1969 , publisher=

    1969

  13. [21]

    PLoS biology , volume=

    Distinct neurocomputational mechanisms support informational and socially normative conformity , author=. PLoS biology , volume=. 2022 , publisher=

    2022

  14. [22]

    Organizational behavior and human decision processes , volume=

    Advice taking in decision making: Egocentric discounting and reputation formation , author=. Organizational behavior and human decision processes , volume=. 2000 , publisher=

    2000

  15. [23]

    Organizational behavior and human decision processes , volume=

    Trust, confidence, and expertise in a judge-advisor system , author=. Organizational behavior and human decision processes , volume=. 2001 , publisher=

    2001

  16. [24]

    Xiong, Miao and Hu, Zhiyuan and Lu, Xinyang and Li, Yifei and Fu, Jie and He, Junxian and Hooi, Bryan , booktitle=. Can

  17. [25]

    International conference on learning representations , volume=

    Large language models cannot self-correct reasoning yet , author=. International conference on learning representations , volume=

  18. [26]

    International Conference on Learning Representations , volume=

    Critic: Large language models can self-correct with tool-interactive critiquing , author=. International Conference on Learning Representations , volume=

  19. [27]

    AutoGen: Enabling Next-Gen

    Qingyun Wu and Gagan Bansal and Jieyu Zhang and Yiran Wu and Beibin Li and Erkang Zhu and Li Jiang and Xiaoyun Zhang and Shaokun Zhang and Jiale Liu and Ahmed Hassan Awadallah and Ryen W White and Doug Burger and Chi Wang , booktitle=. AutoGen: Enabling Next-Gen

  20. [28]

    Forty-first international conference on machine learning , year=

    Improving Factuality and Reasoning in Language Models through Multiagent Debate , author=. Forty-first international conference on machine learning , year=

  21. [29]

    Chan, Chi-Min and Chen, Weize and Su, Yusheng and Yu, Jianxuan and Xue, Wei and Zhang, Shanghang and Fu, Jie and Liu, Zhiyuan , booktitle=

  22. [30]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration , author=. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno...

    2024

  23. [31]

    Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    Encouraging divergent thinking in large language models through multi-agent debate , author=. Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    2024

  24. [32]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

  25. [33]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    Challenging big-bench tasks and whether chain-of-thought can solve them , author=. Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    2023

  26. [34]

    Advances in Neural Information Processing Systems , volume=

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark , author=. Advances in Neural Information Processing Systems , volume=

  27. [35]

    arXiv preprint arXiv:1803.05457 , year=

    Think you have solved question answering? try arc, the ai2 reasoning challenge , author=. arXiv preprint arXiv:1803.05457 , year=

    arXiv

  28. [36]

    Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers) , pages=

    Truthfulqa: Measuring how models mimic human falsehoods , author=. Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers) , pages=

  29. [37]

    arXiv preprint arXiv:2412.15115 , year=

    Qwen2.5 Technical Report , author=. arXiv preprint arXiv:2412.15115 , year=

    arXiv

  30. [38]

    arXiv preprint arXiv:2310.06825 , year=

    Mistral 7B , author=. arXiv preprint arXiv:2310.06825 , year=

    arXiv

  31. [39]

    arXiv preprint arXiv:2408.00118 , year=

    Gemma 2: Improving Open Language Models at a Practical Size , author=. arXiv preprint arXiv:2408.00118 , year=

    arXiv

  32. [40]

    arXiv preprint arXiv:2407.21783 , year=

    The. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  33. [41]

    arXiv preprint arXiv:2501.00656 , year=

    2 OLMo 2 Furious , author=. arXiv preprint arXiv:2501.00656 , year=

    arXiv

  34. [42]

    International Conference on Learning Representations , volume=

    Darkbench: Benchmarking dark patterns in large language models , author=. International Conference on Learning Representations , volume=

  35. [43]

    arXiv preprint arXiv:2303.13988 , year=

    Machine psychology , author=. arXiv preprint arXiv:2303.13988 , year=

    arXiv

  36. [44]

    Advances in neural information processing systems , volume=

    Self-refine: Iterative refinement with self-feedback , author=. Advances in neural information processing systems , volume=

  37. [45]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  38. [46]

    arXiv preprint arXiv:2605.21318 , year=

    TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization , author=. arXiv preprint arXiv:2605.21318 , year=

    arXiv

  39. [47]

    Advances in neural information processing systems , volume=

    Large language models are zero-shot reasoners , author=. Advances in neural information processing systems , volume=

  40. [48]

    , author=

    Metacognition and cognitive monitoring: A new area of cognitive--developmental inquiry. , author=. American psychologist , volume=. 1979 , publisher=

    1979

  41. [49]

    Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages=

    Uncertainty-aware reliable text classification , author=. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages=

  42. [50]

    Proceedings of the ACM Web Conference 2024 , pages=

    Better to ask in english: Cross-lingual evaluation of large language models for healthcare queries , author=. Proceedings of the ACM Web Conference 2024 , pages=

    2024

  43. [51]

    Findings of the Association for Computational Linguistics: EMNLP 2022 , pages=

    Controllable fake document infilling for cyber deception , author=. Findings of the Association for Computational Linguistics: EMNLP 2022 , pages=

    2022

  44. [52]

    Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , pages=

    Understanding the Effects of Explaining Predictive but Unintuitive Features in Human-XAI Interaction , author=. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , pages=

    2025

  45. [53]

    Proceedings of the 2023 Conference on Human Information Interaction and Retrieval , pages=

    Understanding the cognitive influences of interpretability features on how users scrutinize machine-predicted categories , author=. Proceedings of the 2023 Conference on Human Information Interaction and Retrieval , pages=

    2023

  46. [54]

    Proceedings of the 30th ACM International Conference on Information & Knowledge Management , pages=

    A study of explainability features to scrutinize faceted filtering results , author=. Proceedings of the 30th ACM International Conference on Information & Knowledge Management , pages=

  47. [55]

    arXiv:2601.14230 , year=

    MASCOT: Towards Multi-Agent Socio-Collaborative Companion Systems , author=. arXiv:2601.14230 , year=

    arXiv

  48. [56]

    Jin, Yiqiao and Zhao, Qinlin and Wang, Yiyang and Chen, Hao and Zhu, Kaijie and Xiao, Yijia and Wang, Jindong , booktitle=

  49. [57]

    2026 , eprint =

    Qu, Jiaming and Fu, Lucheng and Hu, Yibo , title =. 2026 , eprint =

    2026

  50. [58]

    2026 , note =

    Hu, Yibo and Qu, Jiaming , title =. 2026 , note =

    2026

  51. [59]

    2024 , eprint =

    Xiaochen Zhu and Caiqi Zhang and Tom Stafford and Nigel Collier and Andreas Vlachos , title =. 2024 , eprint =

    2024

  52. [60]

    2025 , eprint =

    Huixin Zhong and Yanan Liu and Qi Cao and Shijin Wang and Zijing Ye and Zimu Wang and Shiyao Zhang , title =. 2025 , eprint =

    2025

  53. [61]

    2025 , eprint =

    Keyu Wang and Jin Li and Shu Yang and Zhuoran Zhang and Di Wang , title =. 2025 , eprint =

    2025

  54. [62]

    2025 , eprint =

    Daniel Vennemeyer and Phan Anh Duong and Tiffany Zhan and Tianyu Jiang , title =. 2025 , eprint =

    2025

  55. [63]

    , title =

    Turpin, Miles and Michael, Julian and Perez, Ethan and Bowman, Samuel R. , title =. Advances in Neural Information Processing Systems 36 (NeurIPS) , year =. 2305.04388 , archivePrefix =

    arXiv

  56. [64]

    and Toubia, Olivier , title =

    Brucks, Melanie S. and Toubia, Olivier , title =. PLOS ONE , volume =. 2025 , publisher =

    2025

  57. [65]

    Findings of the Association for Computational Linguistics: ACL 2024 , year =

    Madsen, Andreas and Chandar, Sarath and Reddy, Siva , title =. Findings of the Association for Computational Linguistics: ACL 2024 , year =. 2401.07927 , archivePrefix =

    2024 arXiv

  58. [66]

    Strachan, James W. A. and Albergo, Dalila and Borghini, Giulia and Pansardi, Oriana and Scaliti, Eugenio and Gupta, Saurabh and Saxena, Krati and Rufo, Alessandro and Panzeri, Stefano and Manzi, Guido and Graziano, Michael S. A. and Becchio, Cristina , title =. Nature Human Be...

    2024

  59. [67]

    and Cheng, Newton and Durmus, Esin and Hatfield-Dodds, Zac and Johnston, Scott R

    Sharma, Mrinank and Tong, Meg and Korbak, Tomasz and Duvenaud, David and Askell, Amanda and Bowman, Samuel R. and Cheng, Newton and Durmus, Esin and Hatfield-Dodds, Zac and Johnston, Scott R. and Kravec, Shauna and Maxwell, Timothy and McCandlish, Sam and Ndousse, Kamal and Ra...

    arXiv

  60. [68]

    Findings of the Association for Computational Linguistics: NAACL 2024 , pages=

    Large language models sensitivity to the order of options in multiple-choice questions , author=. Findings of the Association for Computational Linguistics: NAACL 2024 , pages=

    2024

  61. [69]

    2025 , eprint =

    Needham, Joe and Edkins, Giles and Pimpale, Govind and Bartsch, Henning and Hobbhahn, Marius , title =. 2025 , eprint =

    2025

  62. [70]

    Findings of the Association for Computational Linguistics: ACL 2023 , month = jul, year =

    Discovering Language Model Behaviors with Model-Written Evaluations , author =. Findings of the Association for Computational Linguistics: ACL 2023 , month = jul, year =. doi:10.18653/v1/2023.findings-acl.847 , pages =

    2023 doi

  63. [71]

    arXiv preprint arXiv:2601.03746 , year =

    Schuster, Jakob and Gautam, Vagrant and Markert, Katja , title =. arXiv preprint arXiv:2601.03746 , year =

    arXiv

  64. [72]

    arXiv preprint arXiv:2602.13568 , year =

    Bajaj, Anooshka and Tiganj, Zoran , title =. arXiv preprint arXiv:2602.13568 , year =

  65. [73]

    arXiv preprint arXiv:2601.11563 , year =

    Zhang, Long and Chen, Wei-neng , title =. arXiv preprint arXiv:2601.11563 , year =

  66. [74]

    Journal of Verbal Learning and Verbal Behavior , volume =

    Hasher, Lynn and Goldstein, David and Toppino, Thomas , title =. Journal of Verbal Learning and Verbal Behavior , volume =

  67. [75]

    arXiv preprint arXiv:2508.18321 , year =

    Song, Maojia and Pala, Tej Deep and Zhou, Ruiwen and Jin, Weisheng and Zadeh, Amir and Li, Chuan and Herremans, Dorien and Poria, Soujanya , title =. arXiv preprint arXiv:2508.18321 , year =

  68. [76]

    Mitigating

    Abbasi-Yadkori, Yasin and Kuzborskij, Ilja and Stutz, David and Gy. Mitigating. 2024 , eprint =

    2024

  69. [77]

    and Bates, Stephen , title =

    Angelopoulos, Anastasios N. and Bates, Stephen , title =. arXiv preprint arXiv:2107.07511 , year =

    arXiv

  70. [78]

    2025 , eprint =

    Federico Germani and Giovanni Spitale , title =. 2025 , eprint =

    2025

  71. [79]

    2023 , eprint =

    Philippe Laban and Lidiya Murakhovs'ka and Caiming Xiong and Chien-Sheng Wu , title =. 2023 , eprint =

    2023

  72. [80]

    Science , volume =

    Tversky, Amos and Kahneman, Daniel , title =. Science , volume =

  73. [81]

    Proceedings of the 13th International Conference on Learning Representations (ICLR 2025) , year =

    Jeremy Perez and Grgur Kovac and Corentin Leger and Cedric Colas and Gaia Molinaro and Maxime Derex and Pierre-Yves Oudeyer and Clement Moulin-Frier , title =. Proceedings of the 13th International Conference on Learning Representations (ICLR 2025) , year =. 2407.04503 , archi...

    2025

  74. [82]

    Cohen , title =

    Adi Simhi and Fazl Barez and Martin Tutek and Yonatan Belinkov and Shay B. Cohen , title =. 2026 , eprint =

    2026

  75. [83]

    Journal of Political Economy , volume =

    Bikhchandani, Sushil and Hirshleifer, David and Welch, Ivo , title =. Journal of Political Economy , volume =

  76. [84]

    , title =

    Kelman, Herbert C. , title =. Journal of Conflict Resolution , volume =

  77. [85]

    2025 , eprint =

    Myra Cheng and Sunny Yu and Cinoo Lee and Pranav Khadpe and Lujain Ibrahim and Dan Jurafsky , title =. 2025 , eprint =

    2025

  78. [86]

    2025 , eprint =

    Joshua Liu and Aarav Jain and Soham Takuri and Srihan Vege and Aslihan Akalin and Kevin Zhu and Sean O'Brien and Vasu Sharma , title =. 2025 , eprint =

    2025

  79. [87]

    Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026) , year =

    Shomik Jain and Charlotte Park and Matt Viana and Ashia Wilson and Dana Calacci , title =. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026) , year =. 2509.12517 , archivePrefix =

    2026

  80. [88]

    2026 , eprint =

    De Marzo, Giordano and Bellina, Alessandro and Castellano, Claudio and Priesemann, Viola and Garcia, David , title =. 2026 , eprint =

    2026

  81. [89]

    2024 , eprint =

    Ariel Flint Ashery and Luca Maria Aiello and Andrea Baronchelli , title =. 2024 , eprint =

    2024

  82. [90]

    2023 , eprint=

    Ji, Jiaming and Liu, Mickel and Dai, Josef and Pan, Xuehai and Zhang, Chi and Bian, Ce and Chen, Boyuan and Sun, Ruiyang and Wang, Yizhou and Yang, Yaodong , booktitle=. 2023 , eprint=

    2023

  83. [91]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    R. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=. 2024 , eprint=

    2024

  84. [92]

    2024 , eprint=

    Zhang, Zhexin and Lei, Leqi and Wu, Lindong and Sun, Rui and Huang, Yongkang and Long, Chong and Liu, Xiao and Lei, Xuanyu and Tang, Jie and Huang, Minlie , booktitle=. 2024 , eprint=

    2024

  85. [93]

    Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE) , year=

    Vulnerability Detection with Code Language Models: How Far Are We? , author=. Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE) , year=. 2403.18624 , archivePrefix=

    arXiv

  86. [94]

    Aligning

    Hendrycks, Dan and Burns, Collin and Basart, Steven and Critch, Andrew and Li, Jerry and Song, Dawn and Steinhardt, Jacob , booktitle=. Aligning. 2021 , eprint=

    2021

  87. [95]

    2024 , eprint=

    Han, Seungju and Rao, Kavel and Ettinger, Allyson and Jiang, Liwei and Lin, Bill Yuchen and Lambert, Nathan and Choi, Yejin and Dziri, Nouha , booktitle=. 2024 , eprint=

    2024

  88. [96]

    Ghosh, Shaona and Varshney, Prasoon and Galinkin, Erick and Parisien, Christopher , journal=

  89. [97]

    Inan, Hakan and Upasani, Kartikeya and Chi, Jianfeng and Rungta, Rashi and Iyer, Krithika and Mao, Yuning and Tontchev, Michael and Hu, Qing and Fuller, Brian and Testuggine, Davide and Khabsa, Madian , journal=

  90. [98]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    A Holistic Approach to Undesired Content Detection in the Real World , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=. doi:10.1609/aaai.v37i12.26752 , year=

    doi

  91. [99]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=. 2021 , eprint=

    2021

  92. [100]

    , title =

    Ladha, Krishna K. , title =. American Journal of Political Science , volume =

  93. [101]

    2024 , eprint =

    Verga, Pat and Hofstatter, Sebastian and Althammer, Sophia and Su, Yixuan and Piktus, Aleksandra and Arkhangorodsky, Arkady and Xu, Minjie and White, Naomi and Lewis, Patrick , title =. 2024 , eprint =

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.