Pith. sign in

REVIEW 5 major objections 5 minor 47 references

Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Zero-shot LLMs can reliably detect agreement in multi-agent debates, and inserting a dedicated agreement-detection judge into a simulated decision conference makes the simulated debate cover the same ground as a real expert conference.

desk verdict Useful new application with solid objective benchmarks, but the central reliability claim is currently carried by a self-referential LLM-as-a-judge setup and a single transcript. read the letter →

arxiv 2507.08440 v1 pith:GFGW36QF submitted 2025-07-11 cs.CL cs.AIcs.MA

Selina Heller , Mohamed Ibrahim , David Antony Selby , Sebastian Vollmer This is my paper
classification cs.CLcs.AIcs.MA
keywords LargeLanguageModelsAgreementDetectionMulti-AgentCollaborationDecisionConferenceStanceLLM-as-a-JudgeGroupDecision-Making
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper builds a multi-agent system that simulates a decision conference, a structured meeting where experts debate a complex issue and work toward consensus, and asks whether large language models can detect when the debating participants have reached agreement. The authors evaluate six LLMs zero-shot on stance detection and stance polarity detection benchmarks, then place the models in a 'judge agent' role that decides whether the agents should keep debating or move on. They report that the best models, ChatGPT 4, LLaMA 3 70B, and the smaller open-source Gemma 2 9B, detect agreement reliably even in nuanced, multi-round debate, matching or beating task-specific models trained for stance detection. The central evidence for the system is a comparison against a real drug-policy decision conference: without the judge agent, the simulated debate covered six of the seven thematic clusters of criteria from the real conference; with the judge agent, it covered all seven, matching the real outcome. If correct, this means zero-shot LLMs can serve as agreement detectors in deliberation systems, and a dedicated agreement-detection module makes simulated group decision-making behave more like the real thing.

What carries the argument

The load-bearing component is the judge agent: an LLM that, after each round of participant debate, decides whether agreement has been reached and signals the moderator, through a custom speaker-selection function, to either continue debating or advance to the next stage of the conference. The judge's task is operationalized by two NLP benchmarks: stance detection, which identifies whether a statement supports or opposes a proposition, and stance polarity detection, which classifies sentiment as positive, negative, or neutral; these serve as the objective proxy for agreement. The simulated conference itself is built on the AutoGen framework for multi-agent conversation, and its output is evaluated by a second LLM-as-a-judge layer plus a manual transcript comparison against a published real-world decision conference on drug policy. The architecture keeps the judge's verdict binary, agreement or continued debate, so the system's value rests entirely on whether that binary call is made at the right moment.

What would settle it

Have human experts rate the judge agent's agreement calls on the simulated transcripts and compare their ratings with ChatGPT 4's; if the two diverge on this task, the central claim loses its measurement ground. Simpler still, run the with- and without-judge simulations on a second real decision conference with a published outcome: if the without-judge debate also covers all the real conference's topic clusters, the single transcript comparison that supports the judge agent's benefit would no longer distinguish the two designs.

Watch

Extended reading notes

Core claim

The central claim is that LLMs can perform zero-shot agreement detection in dynamic, nuanced debates, and that a dedicated agreement-detection agent materially improves a simulated decision conference. On objective benchmarks, the top LLMs match or surpass task-specific stance-detection systems without any fine-tuning or prompt engineering; the three leaders from the benchmarks, LLaMA 3 70B, Gemma 2 9B, and ChatGPT 4, remain the top performers when placed inside the simulated conference and judged by an independent LLM-as-a-judge evaluation. The authors' most concrete evidence for the judge agent's value is a direct outcome comparison: a simulated debate about drug-policy criteria, run without the judge, produced six of the seven thematic clusters (health, social, political, public, crime, economic, cost) identified by real experts, omitting 'public'; the same debate with the judge detected that agreement was premature and continued until all seven clusters were covered, reproducing the real conference's outcome. The paper concludes that agreement detection is a critical component for LLM-based simulation of group decision-making, and that open-source models of moderate size are sufficient for the role.

Load-bearing premise

The evaluation assumes ChatGPT 4's scores are a trustworthy measure of whether the judge agent correctly detected agreement, but the evidence for that trustworthiness comes from general chatbot-quality assessment, not from agreement detection in decision conferences.

Editorial extensions

If this is right

  • Zero-shot LLMs can replace fine-tuned stance-detection models for agreement detection in debate settings, removing the need for task-specific training data.
  • Adding a judge agent that detects agreement prevents premature transitions between debate topics, so simulated discussions cover the full range of perspectives a real expert panel would raise.
  • Mid-sized open-source models perform at the level of ChatGPT 4 for this task, so agreement-detection systems can be run locally and at lower cost.
  • Debate-based multi-agent systems generally can use a dedicated agreement-detection module to know when to stop arguing and consolidate a decision, improving both efficiency and coverage.
  • The simulation approach could support real decision-making workflows, such as expert elicitation workshops, by revealing which perspectives a group is at risk of overlooking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The objective evidence measures stance, not agreement itself; the step from 'a statement supports or opposes a claim' to 'two debating agents have reached agreement' is an assumption the benchmarks never directly test, so a dedicated agreement-annotation dataset would be the natural next experiment.
  • The with-judge benefit rests on a single transcript; a statistical test over many simulated conferences, with varied topics, personas, and participant counts, is needed to confirm that the judge agent, rather than prompt randomness, produces the fuller coverage.
  • Because the judge only checks that agreement has been reached, not that the agreed content is accurate or grounded, retrieval-augmented grounding of participant claims could change both how fast agreement forms and whether it is well-founded.
  • The paper's result that a 9-billion-parameter model matches GPT-4 suggests agreement detection may hinge on instruction-following and output-format compliance more than raw reasoning scale, a prediction testable by varying prompt strictness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a multi-agent system for simulating decision conferences, built on AutoGen, in which a moderator guides participant agents through staged debates and a judge agent decides whether the participants have reached agreement. The judge agent is then evaluated in two ways: objectively, by testing six LLMs (Gemma 2 9B, Gemma 7B, Mixtral 8x7B, LLaMA 3 70B, ChatGPT 3.5 Turbo, ChatGPT 4) on stance detection and stance polarity detection using the VAST and Claim Stance Classification datasets; and subjectively, by using ChatGPT 4 as an LLM-as-a-judge to score the judge agent's decisions in a simulated drug-policy decision conference. The authors also compare a simulation with the judge agent against one without it, arguing that the judge agent leads to more complete topic coverage (specifically, covering the 'public' cluster). The paper concludes that LLMs can reliably detect agreement in dynamic debates and that incorporating an agreement-detection agent improves the quality and realism of simulated deliberations.

Significance. If the central claim holds, the paper would provide a practical blueprint for using zero-shot LLMs as agreement detectors in multi-agent deliberation, with a potentially useful application in expert elicitation and decision-support systems. The objective evaluation is a genuine strength: it compares six LLMs against established fine-tuned baselines on standard benchmarks, demonstrating that mid-sized open models such as Gemma 2 9B and LLaMA 3 70B can be competitive without task-specific training. The architectural description and the reproducible benchmark comparisons are also valuable. However, the central claim about 'reliable agreement detection in dynamic and nuanced debates' rests primarily on a subjective evaluation whose ground truth is itself an LLM judgment (ChatGPT 4), and the paper does not provide task-specific human validation for that judgment. The with/without-judge comparison is based on a single transcript. These gaps currently make the main conclusion stronger than the evidence supports.

major comments (5)
  1. [Section 5.2.1, Table 5] The subjective evaluation uses ChatGPT 4 as the LLM-as-a-judge to grade each model's judge-agent decisions, but no human annotations are provided for agreement detection in decision conferences. The paper justifies this choice by citing reference [11] (MT-Bench/Chatbot Arena), yet that work validates LLM judges on general chat-assistant quality, not on detecting agreement in multi-agent debates, and it explicitly documents judge biases such as self-preference and verbosity bias. The fact that ChatGPT 4 receives a perfect 10/10 in its own row of Table 5 is therefore not evidence of correctness, and the scores cannot currently support the abstract's claim that LLMs 'reliably detect agreement.' A task-specific validation set, or at least a random sample of judge decisions scored by human annotators, is needed before the central claim can be accepted.
  2. [Section 5.1.1, Table 2] In the VAST evaluation, the neutral class contains only 2 examples, and after reporting that all models essentially fail on this class (F1 scores of 0.0 to 0.028), the paper states that this 'can be considered less impactful' and that the primary focus should remain on pro and con. This is a post-hoc dismissal of a class that is directly relevant to the system's purpose: the judge agent must distinguish 'agreement,' 'disagreement,' and 'still debating' (a neutral state). Excluding or downweighting the neutral class changes the reported macro-F1 substantially, and the paper should either justify the exclusion a priori or report micro-averaged metrics and per-class results without the post-hoc reinterpretation.
  3. [Section 5.1.3, Table 4 and Figure 3] The stance polarity results show a systematic bias toward predicting negative polarity across all models, with the paper noting that 'some positive labels are predicted as negative.' The rationalization that this 'is not that bad' because missing negative polarity would cause premature termination is not supported by the task definition: if the judge agent relies on polarity to detect agreement, then misclassifying positive (supportive) statements as negative could cause the debate to continue unnecessarily or cause an agreement to be missed. The paper should present a confusion matrix or error analysis for the polarity task and discuss how this bias affects the judge agent's agreement decisions, rather than asserting that the failure mode is benign.
  4. [Section 5.2.2] The comparison of the system with and without the judge agent is based on a single manually inspected transcript for one topic. The claimed benefit is that the with-judge simulation covers all seven thematic clusters (health, social, political, public, crime, economic, cost), while the without-judge simulation misses 'public.' Because LLM simulations are stochastic (no random seed reporting, no repeated runs, no error bars), a single run cannot establish that this difference is due to the judge agent rather than to sampling variability. The paper should report multiple runs (e.g., 5–10 per condition) with a measure of coverage variability, or explicitly frame the result as an illustrative case study rather than evidence for the general claim that the judge agent 'prevents premature transitions between topics.'
  5. [Section 4.1 and Section 5.1] The objective evaluation operationalizes agreement detection as stance detection and stance polarity detection on isolated benchmark texts. However, the judge agent's actual function in the simulated decision conference is a relational, dialogue-level decision: given the exchange between two or more participants, determine whether they have reached agreement. Stance classification of individual claims is a necessary component but not sufficient evidence for the ability to perform this relational judgment, because agreement detection requires tracking whether a later utterance aligns with, responds to, and resolves prior statements. The paper should either provide a dialogue-level objective evaluation (e.g., on a conversational agreement or negotiation dataset) or explicitly narrow the central claim to stance-based agreement detection rather than 'agreement detection in dynamic and nuanced debates.'
minor comments (5)
  1. [Throughout] Several typographical and formatting issues should be corrected: 'V AST' appears with an inconsistent space (e.g., 'VAST' vs 'V AST'), '1.355' should be '1,355' in Section 4.1.1, and 'T able 1' in Section 5.1 has an erroneous space.
  2. [Section 3.2.1, Algorithm 1] The pseudocode labels an 'evaluation agent' that scores the debate, while the text mostly refers to a 'judge agent' for agreement detection. The relationship between these two agents should be clarified, since the evaluation agent appears to use LLM-as-a-judge during the simulation while the judge agent makes the agreement decision.
  3. [Section 5.1.2] The paper states that all models perform exceptionally well on the Claim Stance Classification dataset and that the top three surpass the 0.849 state-of-the-art baseline, but the comparison could be made more precise by reporting the variance or confidence intervals, especially since zero-shot prompting can be sensitive to prompt wording.
  4. [Section 7.1.7 'Code availability'] The paper states 'Not applicable' for code availability, but the custom speaker selection function and system prompts are central to the reproducibility of the simulations. Making at least the prompts and the speaker selection logic available would strengthen the paper.
  5. [Section 5.2.1] The use of 'five decisions from the judge agent' is not justified in the text; the paper should explain why five decision points were selected, how they were sampled, and whether the judge agent produced more than five decisions that were excluded.

Circularity Check

1 steps flagged · score 3.0 of 10

Objective benchmark results are self-contained, but the central subjective reliability claim is graded by ChatGPT 4 on its own judge-agent row, leaving that part of the evidence self-referential rather than independently validated.

  1. other [Section 5.2.1, Table 5, and Section 4.2]
    "Since a strong LLM is needed to fulfil the needs of the LLM-as-a-judge approach we decided to use ChatGPT 4 as the model evaluating the models on the task of agreement detection in decision conferences [11]. ... Using ChatGPT 4 as an evaluator helps to increase the consistency and reliability of the subjective evaluation ... and it also has the highest agreement with human evaluators on different tasks [11]."

    The paper's direct evidence for the abstract claim that 'LLMs can reliably detect agreement even in dynamic and nuanced debates' is the subjective evaluation in Table 5, where ChatGPT 4 itself receives 10/10. Those scores are produced by ChatGPT 4 as the LLM-as-a-judge grader, so the model whose agreement-detection ability is being certified is also the grader that certifies it. No human-annotated ground truth for agreement detection in decision conferences is collected, and the cited justification [11] validates LLM-as-a-judge on MT-Bench chat-assistant quality rather than on relational agreement detection across a debate.

full rationale

The core objective evaluation is self-contained: six models are scored zero-shot on the external VAST and Claim Stance Classification benchmarks using fixed prompts and standard metrics, and no fitted parameter defines the reported accuracies. The with/without-judge comparison rests on a single manually inspected transcript, which is weak evidence but not a circular derivation, and the paper itself acknowledges that more decision conferences are needed. There is no load-bearing author self-citation: the justification for LLM-as-a-judge points to the external MT-Bench paper [11], no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The only self-referential element is the subjective evaluation in Section 5.2.1: ChatGPT 4 is both the judge agent being scored, with a perfect 10/10 in Table 5, and the LLM-as-a-judge grader, with no task-specific human validation. That makes the abstract's 'reliably detect agreement in dynamic and nuanced debates' claim partially in-family, but it does not collapse the objective benchmark results. The appropriate finding is therefore a moderate, partial circularity rather than a wholesale reduction of the paper's claims to their inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numeric parameters are fitted; all LLM evaluations are zero-shot with basic prompts, so the free-parameter ledger is empty. The load-bearing assumptions are conceptual: that stance labels proxy agreement in dialogue, that ChatGPT 4's judgments are valid without task-specific human validation, and that one real drug-policy conference is a sufficient reference. These assumptions are flagged rather than demonstrated.

assumptions (4)
  • domain assumption Stance detection and stance polarity detection on the VAST and Claim Stance datasets are valid proxies for detecting agreement between debating agents.
    Invoked in Section 4.1 to justify the objective evaluation; argues that aligning stance implies agreement, even though stance toward a topic is not identical to agreement between dialogue participants in a multi-turn debate.
  • domain assumption ChatGPT 4's LLM-as-a-judge scores are a reliable measure of whether the judge agent correctly detects agreement in the simulated decision conferences.
    Stated in Section 5.2.1 and justified only by citing MT-Bench [11] for general LLM-as-a-judge quality, not by human validation on this task.
  • domain assumption The single real decision conference on drug policy [41] provides a valid reference for comparing the simulated conferences.
    Used in Section 4.2 and Section 5.2.2 as the ground truth for topic coverage; assumes the real conference's structure and outcome are representative and transferable to the simulated setting.
  • domain assumption LLM agents' utterances can be interpreted as genuine positions that can be classified as agreement or disagreement.
    Underlies the entire system design in Section 3.2; treats generated text as meaningful deliberative content rather than linguistic pattern completion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences." pith.science (2026). https://pith.science/paper/GFGW36QF

@misc{pith2026250708440,
  author       = {Pith},
  title        = {Pith review of: Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GFGW36QF}},
  note         = {Machine review of arXiv:2507.08440}
}
read the original abstract

Decision conferences are structured, collaborative meetings that bring together experts from various fields to address complex issues and reach a consensus on recommendations for future actions or policies. These conferences often rely on facilitated discussions to ensure productive dialogue and collective agreement. Recently, Large Language Models (LLMs) have shown significant promise in simulating real-world scenarios, particularly through collaborative multi-agent systems that mimic group interactions. In this work, we present a novel LLM-based multi-agent system designed to simulate decision conferences, specifically focusing on detecting agreement among the participant agents. To achieve this, we evaluate six distinct LLMs on two tasks: stance detection, which identifies the position an agent takes on a given issue, and stance polarity detection, which identifies the sentiment as positive, negative, or neutral. These models are further assessed within the multi-agent system to determine their effectiveness in complex simulations. Our results indicate that LLMs can reliably detect agreement even in dynamic and nuanced debates. Incorporating an agreement-detection agent within the system can also improve the efficiency of group debates and enhance the overall quality and coherence of deliberations, making them comparable to real-world decision conferences regarding outcome and decision-making. These findings demonstrate the potential for LLM-based multi-agent systems to simulate group decision-making processes. They also highlight that such systems could be instrumental in supporting decision-making with expert elicitation workshops across various domains.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 22 canonical work pages

  1. [11]

    https://doi.org/10.48550/ arXiv.2306.05685

    Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E.P., Zhang, H., Gonzalez, J.E., Stoica, I.: Judging LLM-as- a-Judge with MT-Bench and Chatbot Arena (2023). https://doi.org/10.48550/ arXiv.2306.05685

    arXiv doi:10.48550/arxiv.2306.05685 2023

  2. [1]

    arXiv preprint arXiv:2402.01680 (2024) https://doi.org/10.13140/RG.2.2.36311

    Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N.V., Wiest, O., Zhang, X.: Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680 (2024) https://doi.org/10.13140/RG.2.2.36311. 85928

    arXiv 2024

  3. [2]

    https://doi.org/10.48550/arXiv.2308.07201

    Chan, C.-M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., Fu, J., Liu, Z.: ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (2023). https://doi.org/10.48550/arXiv.2308.07201

    arXiv doi:10.48550/arxiv.2308.07201 2023

  4. [3]

    https: //doi.org/10.48550/arXiv.2305.14325

    Du, Y., Li, S., Torralba, A., Tenenbaum, J.B., Mordatch, I.: Improving Factuality and Reasoning in Language Models through Multiagent Debate (2023). https: //doi.org/10.48550/arXiv.2305.14325

    arXiv doi:10.48550/arxiv.2305.14325 2023

  5. [4]

    https://doi.org/10.48550/arXiv.2305.19118

    Liang, T., He, Z., Jiao, W., Wang, X., Wang, R., Yang, Y., Tu, Z., Shi, S.: Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate (2024). https://doi.org/10.48550/arXiv.2305.19118

    arXiv doi:10.48550/arxiv.2305.19118 2024

  6. [5]

    (eds.) SHELF: The Sheffield Elicitation Framework, pp

    Gosling, J.P.: In: Dias, L.C., Morton, A., Quigley, J. (eds.) SHELF: The Sheffield Elicitation Framework, pp. 61–93. Springer, Cham (2018). https://doi.org/10. 1007/978-3-319-65052-4 4

    work page 2018

  7. [6]

    Annals of Operations Research 154(1), 51–68 (2007) https://doi.org/10.1007/ s10479-007-0183-3

    Phillips, L.D., Costa, C.A.: Transparent prioritisation, budgeting and resource allocation with multi-criteria decision analysis and decision conferencing. Annals of Operations Research 154(1), 51–68 (2007) https://doi.org/10.1007/ s10479-007-0183-3

    work page 2007

  8. [7]

    In: IEE Colloquium on CSCW: Some Fundamental Issues, pp

    Phillips, L.D.: Decision conferencing. In: IEE Colloquium on CSCW: Some Fundamental Issues, pp. 6–1 (1991). IET

    work page 1991

Show all 47 references
  1. [8]

    Autonomous Agents and Multi-Agent Systems 27, 52–84 (2013) https://doi.org/10.1007/ s10458-012-9201-1

    Bosse, T., Hoogendoorn, M., Klein, M.C., Treur, J., Van Der Wal, C.N., Van Wis- sen, A.: Modelling collective decision making in groups and crowds: Integrating 27 social contagion and interacting emotions, beliefs and intentions. Autonomous Agents and Multi-Agent Systems 27, 5...

    2013

  2. [9]

    https://doi.org/10.48550/ arXiv.2311.09665

    Chuang, Y.-S., Suresh, S., Harlalka, N., Goyal, A., Hawkins, R., Yang, S., Shah, D., Hu, J., Rogers, T.T.: The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents (2024). https://doi.org/10.48550/ arXiv.2311.09665

    2024 doi arXiv

  3. [10]

    In: Proceed- ings of the Annual Meeting of the Cognitive Science Society, vol

    Du, Y., Rajivan, P., Gonzalez, C.: Large language models for collective problem-solving: Insights into group consensus decision-making. In: Proceed- ings of the Annual Meeting of the Cognitive Science Society, vol. 46 (2024). https://escholarship.org/uc/item/6s060914

    2024

  4. [12]

    Frontiers of Computer Science 18(6), 186345 (2024) https://doi.org/10

    Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al.: A survey on large language model based autonomous agents. Frontiers of Computer Science 18(6), 186345 (2024) https://doi.org/10. 1007/s11704-024-40231-1

    2024

  5. [13]

    In: Larson, K

    Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N.V., Wiest, O., Zhang, X.: Large language model based multi-agents: A survey of progress and challenges. In: Larson, K. (ed.) Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJ...

    2024 doi

  6. [14]

    https://doi.org/ 10.48550/arXiv.2309.13007

    Chen, J.C.-Y., Saha, S., Bansal, M.: ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs (2024). https://doi.org/ 10.48550/arXiv.2309.13007

    2024 doi arXiv

  7. [15]

    In: The 10th International Conf

    Tsao, W.-K., AILAB, T.: Multi-agent reasoning with large language models for effective corporate planning. In: The 10th International Conf. on Computational Science and Computational Intelligence (2023)

    2023

  8. [16]

    https://doi.org/10.48550/arXiv.2307.07924

    Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., Xu, J., Li, D., Liu, Z., Sun, M.: ChatDev: Communicative Agents for Software Development (2024). https://doi.org/10.48550/arXiv.2307.07924

    2024 doi arXiv

  9. [17]

    https://doi.org/10.48550/arXiv.2305.11595 28

    Xiong, K., Ding, X., Cao, Y., Liu, T., Qin, B.: Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate (2023). https://doi.org/10.48550/arXiv.2305.11595 28

    2023 doi arXiv

  10. [18]

    https://doi.org/10.48550/arXiv.2312.04854

    Wang, H., Du, X., Yu, W., Chen, Q., Zhu, K., Chu, Z., Yan, L., Guan, Y.: Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System (2024). https://doi.org/10.48550/arXiv.2312.04854

    2024 doi arXiv

  11. [19]

    https://doi.org/10.48550/arXiv.2208.04024

    Park, J.S., Popowski, L., Cai, C.J., Morris, M.R., Liang, P., Bernstein, M.S.: Social Simulacra: Creating Populated Prototypes for Social Computing Systems (2022). https://doi.org/10.48550/arXiv.2208.04024

    2022 doi arXiv

  12. [20]

    https://doi.org/10.48550/arXiv.2311.06957

    Xiao, B., Yin, Z., Shan, Z.: Simulating Public Administration Crisis: A Novel Gen- erative Agent-Based Simulation System to Lower Technology Barriers in Social Science Research (2023). https://doi.org/10.48550/arXiv.2311.06957

    2023 doi arXiv

  13. [21]

    https://doi.org/10.48550/arXiv.2311.17227

    Hua, W., Fan, L., Li, L., Mei, K., Ji, J., Ge, Y., Hemphill, L., Zhang, Y.: War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars (2024). https://doi.org/10.48550/arXiv.2311.17227

    2024 doi arXiv

  14. [22]

    https://doi.org/10.48550/arXiv.2402.17385

    Eigner, E., H¨ andler, T.: Determinants of LLM-assisted Decision-Making (2024). https://doi.org/10.48550/arXiv.2402.17385

    2024 doi arXiv

  15. [23]

    In: Fourth Workshop on Knowledge-infused Learning (2024)

    Ondula, E., Orner, D., Mumero, N., Rusti, C.: Sentimental agents: Explor- ing deliberation, cognitive biases, and decision-making in llm-based multiagent systems. In: Fourth Workshop on Knowledge-infused Learning (2024)

    2024

  16. [24]

    https://doi.org/10.48550/arXiv.2309.17234

    Abdelnabi, S., Gomaa, A., Sivaprasad, S., Sch¨ onherr, L., Fritz, M.: Coopera- tion, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation (2024). https://doi.org/10.48550/arXiv.2309.17234

    2024 doi arXiv

  17. [25]

    In: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, pp

    Nardi, O., Boixel, A., Endriss, U.: A graph-based algorithm for the automated justification of collective decisions. In: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, pp. 935–943 (2022)

    2022

  18. [26]

    https://doi.org/10.48550/arXiv.2310.20151

    Chen, H., Ji, W., Xu, L., Zhao, S.: Multi-Agent Consensus Seeking via Large Language Models (2023). https://doi.org/10.48550/arXiv.2310.20151

    2023 doi arXiv

  19. [27]

    https://doi.org/10.48550/arXiv.2402.06634

    Chang, E.Y.: SocraSynth: Multi-LLM Reasoning with Conditional Statistics (2024). https://doi.org/10.48550/arXiv.2402.06634

    2024 doi arXiv

  20. [28]

    https://doi.org/10.48550/arXiv.2303.08769

    Chang, E.Y.: Prompting Large Language Models With the Socratic Method (2023). https://doi.org/10.48550/arXiv.2303.08769

    2023 doi arXiv

  21. [29]

    Advances in Decision Analysis, 375–399 (2007) https://doi.org/10.1017/cbo9780511611308.020

    Phillips, L.D.: Decision conferencing. Advances in Decision Analysis, 375–399 (2007) https://doi.org/10.1017/cbo9780511611308.020

    2007 doi

  22. [30]

    In: Journal of Convention & Event Tourism, vol

    Comas, M., Moscardo, G.: Understanding associations and their conference decision-making processes. In: Journal of Convention & Event Tourism, vol. 7, pp. 117–138 (2005). https://doi.org/10.1300/J452v07n03 07 . Taylor & Francis

    2005 doi

  23. [31]

    Social Work in Health Care 27(3), 57–74 (1998) https://doi.org/10.1300/j010v27n03 04

    Hansen, P., Cornish, P., Kayser, K.: Family conferences as forums for decision 29 making in hospital settings. Social Work in Health Care 27(3), 57–74 (1998) https://doi.org/10.1300/j010v27n03 04

    1998 doi

  24. [32]

    (eds.) Decision-Making: Overview, pp

    Salinas, E.: In: Jaeger, D., Jung, R. (eds.) Decision-Making: Overview, pp. 35–37. Springer, New York, NY (2022). https://doi.org/10.1007/978-1-0716-1006-0 248

    2022 doi

  25. [33]

    https://doi.org/10.48550/arXiv.2308.08155

    Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A.H., White, R.W., Burger, D., Wang, C.: Auto- Gen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (2023). https://doi.org/10.48550/arXiv.2308.08155

    2023 doi arXiv

  26. [34]

    https://doi.org/10.48550/arXiv.2404.12171

    G¨ ul, Lebret, R., Aberer, K.: Stance Detection on Social Media with Fine-Tuned Large Language Models (2024). https://doi.org/10.48550/arXiv.2404.12171

    2024 doi arXiv

  27. [35]

    In: Bethard, S., Carpuat, M., Cer, D., Jurgens, D., Nakov, P., Zesch, T

    Mohammad, S., Kiritchenko, S., Sobhani, P., Zhu, X., Cherry, C.: Semeval-2016 task 6: Detecting stance in tweets. In: Bethard, S., Carpuat, M., Cer, D., Jurgens, D., Nakov, P., Zesch, T. (eds.) Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016...

    2016 doi

  28. [36]

    In: Zong, C., Xia, F., Li, W., Navigli, R

    Li, Y., Sosea, T., Sawant, A., Nair, A.J., Inkpen, D., Caragea, C.: P-stance: A large dataset for stance detection in political domain. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 2355–2365. ...

    2021 doi

  29. [37]

    In: Webber, B., Cohn, T., He, Y., Liu, Y

    Allaway, E., McKeown, K.: Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8913–8931. As...

    2020 doi

  30. [38]

    In: Lapata, M., Blunsom, P., Koller, A

    Bar-Haim, R., Bhattacharya, I., Dinuzzo, F., Saha, A., Slonim, N.: Stance classifi- cation of context-dependent claims. In: Lapata, M., Blunsom, P., Koller, A. (eds.) Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: V...

    2017

  31. [39]

    In: Ku, L.-W., Martins, A., Srikumar, V

    Chen, G., Cheng, L., Luu, A.T., Bing, L.: Exploring the potential of large language models in computational argumentation. In: Ku, L.-W., Martins, A., Srikumar, V. (eds.) Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long ...

    2024

  32. [40]

    https://doi.org/10.48550/arXiv.2409.00222 30

    Akash, A.U., Fahmy, A., Trabelsi, A.: Can Large Language Models Address Open- Target Stance Detection? (2024). https://doi.org/10.48550/arXiv.2409.00222 30

    2024 doi arXiv

  33. [41]

    International Journal of Drug Policy 56, 144–152 (2018)

    Rogeberg, O., Bergsvik, D., Phillips, L.D., Van Amsterdam, J., Eastwood, N., Henderson, G., Lynskey, M., Measham, F., Ponton, R., Rolles, S., et al.: A new approach to formulating and appraising drug policy: a multi-criterion decision analysis applied to alcohol and cannabis r...

    2018

  34. [42]

    https://doi.org/10.48550/ arXiv.2408.00118

    Team, G., Riviere, M., Pathak, S., Sessa, P.G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram´ e, A., et al.: Gemma 2: Improv- ing Open Language Models at a Practical Size (2024). https://doi.org/10.48550/ arXiv.2408.00118

    2024 doi arXiv

  35. [43]

    https://doi.org/10.48550/arXiv.2403

    Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivi` ere, M., Kale, M.S., Love, J., et al.: Gemma: Open Models Based on Gemini Research and Technology (2024). https://doi.org/10.48550/arXiv.2403. 08295

    2024 doi

  36. [44]

    https://doi.org/10.48550/arXiv.2401.04088

    Jiang, A.Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D.S., Casas, D.d.l., Hanna, E.B., Bressand, F., et al.: Mixtral of Experts (2024). https://doi.org/10.48550/arXiv.2401.04088

    2024 doi arXiv

  37. [45]

    https://doi.org/10.48550/arXiv.2407.21783

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al.: The Llama 3 Herd of Models (2024). https://doi.org/10.48550/arXiv.2407.21783

    2024 doi arXiv

  38. [46]

    https://platform.openai.com/docs/models/ gpt-3-5-turbo

    Documentation of GPT-3.5 Turbo. https://platform.openai.com/docs/models/ gpt-3-5-turbo. [Accessed 23-09-2024]

    2024

  39. [47]

    https://platform.openai.com/docs/ models/gpt-4-turbo-and-gpt-4

    Documentation of GPT-4 Turbo and GPT-4. https://platform.openai.com/docs/ models/gpt-4-turbo-and-gpt-4. [Accessed 23-09-2024] 31 Fig. A1 Output of the transcript of simulated decision conference for the first topic without the judge agent 32 Fig. A2 Part 1 of the transcript of...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.