The reviewed record of science sign in
Pith

arxiv: 2607.06157 · v1 · pith:IDJEKEGW · submitted 2026-07-07 · cs.CL · cs.AI

LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability

Chenxu Wang , Yongkun Yang , Boyuan Du , Shiwei Lin , Huaping Liu This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 14:45 UTCglm-5.2pith:IDJEKEGWrecord.jsonopen to challenge →

classification cs.CL cs.AI
keywords LLM agentsmulti-agent collaborationpartial observabilitydeliberationjoint decision makingbenchmarkcombinatorial optimizationtool-augmented reasoning
0
0 comments X

The pith

LLMs Fail at Collaborative Decision-Making Under Partial Observability

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper formalizes a specific problem: multiple LLM agents, each holding only partial information about a shared environment, must communicate through deliberation to reach a single joint decision that maximizes a common reward. The authors build a benchmark across three task settings—menu design with numerical observations, menu design with semantic observations, and task allocation with asymmetric roles—and evaluate seven LLMs ranging from frontier models to smaller open-weight models. The central finding is that even state-of-the-art LLMs remain unreliable at deliberative collaboration: they fail either at accurately exchanging and aggregating information from their partial views, or at the combinatorial reasoning needed to find optimal joint decisions. External mathematical tools help some models but hurt others, because models sometimes reject correct tool outputs in favor of their own flawed calculations—a phenomenon the authors call the 'skepticism trap.' A counterintuitive secondary finding is that the multi-round deliberation process itself can improve performance over a centralized baseline where a single agent receives all information at once, because deliberation creates opportunities for distributed verification and error correction that a single-pass decision lacks.

Core claim

The paper identifies a structural tension in LLM-based multi-agent collaboration: the deliberation process is simultaneously a source of error (through information loss, hallucination, and aggregation mistakes) and a source of correction (through distributed verification and reflection). The benchmark reveals that LLM failures in collaborative settings decompose into two distinct failure modes—information exchange failures and reasoning failures—that can be partially diagnosed using hallucination rate and resource aggregation error metrics. The 'skepticism trap,' where agents reject correct solver outputs because they misinterpret their local partial observations as global constraints, is a新

What carries the argument

The benchmark instantiates a formal tuple (s, O₁,...,Oₙ, D, R) where s is a hidden ground-truth state, Oᵢ are partial observations, D is a finite decision space, and R is a deterministic reward function. A deliberation protocol (Algorithm 1) structures multi-turn communication among agents built from four modules: observation (estimates global state from partial inputs and dialogue), planning (proposes a candidate decision), decision (accepts or rejects the current proposal), and conversation (generates messages to the partner). Two external tools are available: a Solver that formulates the decision as integer linear programming and a Calculator that checks feasibility. The benchmark spans 3

If this is right

  • LLM agents deployed in real-world multi-agent settings—such as multi-robot coordination, distributed resource allocation, or collaborative planning—cannot yet be trusted to reliably aggregate partial information and reach optimal joint decisions without additional safeguards.
  • The finding that deliberation can outperform centralization suggests that multi-round verification protocols may be a practical design pattern for improving LLM decision reliability, even in single-agent settings where self-verification across multiple passes could substitute for inter-agent deliberation.
  • The 'skepticism trap' implies that tool-augmented LLM systems need better calibration of trust: agents must learn when to defer to external tools and when their own local information genuinely contradicts a tool's output.
  • The decomposition of failures into information-exchange errors versus reasoning errors provides a diagnostic framework that could guide targeted improvements—different model capabilities or scaffolding improvements address different failure modes.
  • The benchmark's task-generation pipeline (database-driven, LLM-generated, human-refined) offers a reproducible template for constructing evaluation environments for other collaborative AI settings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The observed performance differences between models may reflect scaffold-prompt compatibility rather than pure model capability, as the authors acknowledge. A model that performs poorly under this scaffold might improve substantially with a different deliberation protocol or prompt structure, meaning the absolute performance numbers are less informative than the relative failure-mode patterns.
  • The six-round deliberation limit may suppress potential gains from longer deliberation chains; the optimal number of rounds likely varies by task complexity and model capability, and the fixed budget could systematically disadvantage models that need more conversational turns to converge.
  • If deliberation provides error-correction benefits, then an ensemble-like protocol—where multiple independent agents deliberate and a meta-agent selects among their proposals—might capture similar verification benefits without the communication overhead of multi-round dialogue.
  • The 'skepticism trap' suggests that the problem is not just about tool compliance but about epistemic calibration under partial observability: agents need explicit mechanisms to distinguish 'this contradicts my local knowledge' from 'this exceeds my local knowledge but may be feasible given information I don't have.'

Load-bearing premise

The paper evaluates all LLMs through a single fixed agent scaffold with specific prompt templates, so the observed failures may reflect limitations of this particular scaffolding rather than fundamental model limitations.

What would settle it

If a different agent scaffold or prompt design eliminated the information-exchange and tool-compliance failures observed here, the conclusion that LLMs are fundamentally insufficient for deliberative collaboration would need revision.

Figures

Figures reproduced from arXiv: 2607.06157 by Boyuan Du, Chenxu Wang, Huaping Liu, Shiwei Lin, Yongkun Yang.

Figure 1
Figure 1. Figure 1: An illustrative example of deliberative col [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of our agent framework for solv [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
read the original abstract

Deliberation plays a crucial role in collaboration; when humans work together, they naturally engage in communication to align information and reach an agreement. In this paper, we investigate deliberative large language model (LLM) agents under partially observable joint decision-making tasks. We formalize deliberative collaboration as a cooperative joint decision problem with partial and asymmetric observations, and introduce a scalable benchmark that instantiates this problem across multiple task settings and domains in which agents must exchange information through deliberation to reach a joint decision with a shared reward. We then instantiate a reference scaffold and evaluation protocol for deliberative agents and conduct a systematic evaluation of a range of representative LLMs. The results reveal that complex deliberative collaboration tasks continue to challenge state-of-the-art language models. Even with the aid of external mathematical tools, language models may fail in either the deliberation process for aligning information or the complex reasoning process for making the decision. On the other hand, diagnostic analysis reveals that the deliberation process may also provide opportunities for reflection and error correction, sometimes improving performance over centralized baselines. Altogether, our work establishes a foundation for evaluating and improving LLM agents in deliberative collaboration and provides insights into the strengths, limitations, and properties of current LLM-based multi-agent systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 6 minor

Summary. This paper formalizes deliberative collaboration among LLM agents as a partially observable cooperative joint decision-making problem and introduces a benchmark with three task settings (menu-numeric, menu-semantic, task allocation) spanning 180 instances. The benchmark uses combinatorial optimization formulations with solver-computed optimal solutions as upper bounds, enabling objective reward computation. The authors evaluate seven LLMs under deliberative, centralized, and oracle conditions, with and without external tools, reporting normalized reward, valid ratio, and additional diagnostic metrics (hallucination rate, NMAE). The main findings are that current LLMs remain insufficient for reliable deliberative collaboration, that tool use helps some models but can hurt others due to non-compliance, and that the deliberation process can sometimes improve performance over centralized single-shot baselines. The benchmark design with solver-grounded rewards, the systematic evaluation across multiple conditions, and the diagnostic analysis of failure modes are genuine contributions.

Significance. The paper makes a solid contribution to the growing literature on LLM-based multi-agent systems. Strengths include: (1) a benchmark with objective, solver-computed reward functions that are independent of LLM judgment, providing reliable ground truth; (2) a systematic evaluation matrix spanning 7 models, 3 domains, with/without tools, and centralized/oracle baselines; (3) reproducible code and detailed prompt templates (Appendix D); (4) diagnostic metrics (HR, NMAE, compliance rate) that go beyond aggregate accuracy to characterize specific failure modes. The finding that agents sometimes reject correct tool outputs (the 'skepticism trap' in §C.2) is a practically useful observation. The formalization in §2.1, while not novel in its mathematical structure, is well-matched to the benchmark design.

major comments (2)
  1. §4.3, Table 3: The claim that 'the deliberation process itself can provide opportunities for reflection and error correction' is confounded by an experimental design mismatch. The centralized baseline (Table 8 prompt) makes a single-shot decision, while the deliberative setting allows up to 6 rounds of interaction. The oracle baseline combines full information with deliberation and outperforms centralized for some models (e.g., GLM-4.7: +20.99 NR without tools). However, the observed benefit cannot be attributed to multi-agent deliberation specifically rather than to multi-round reasoning. A centralized agent given multiple rounds of self-reflection (re-examining its own proposal without a partner) would serve as the necessary control. Without it, the claim that deliberation per se adds value beyond information access is not supported by the current evidence. This is load-bearing for one
  2. §4.1: All experiments use self-play (same model for all agents). The paper does not test cross-model play, which is a common real-world scenario. While the authors acknowledge this in the Limitations section, the absence means that findings about deliberation failures may not generalize to heterogeneous agent teams where models have different reasoning styles, capabilities, or prompt sensitivities. At minimum, the paper should explicitly scope its claims to self-play settings throughout the paper, not only in the Limitations.
minor comments (6)
  1. Table 2: The NR values for some models show large gaps between with-tools and without-tools conditions (e.g., Qwen3-32B menu-numeric: 4.78 without tools vs. 18.77 with tools). The standard errors are reported, but it would help to state whether any of these differences are statistically significant, or at least note the number of trials per condition.
  2. §2.2.2: The menu-semantic setting notes that 'even an oracle baseline that has access to all natural language observations may not achieve the full reward.' It would strengthen the paper to report the oracle NR for menu-semantic to quantify this ceiling, as is done for task allocation via the solver.
  3. Table 4: The compliance rate of 0% for GPT-5.1 in task allocation is a striking result. The text attributes this to the model trying calculations itself, but a 0% compliance rate suggests the model never once adopted the tool output. Could the authors verify this is not a prompt or parsing issue? A brief note on how compliance was measured would help interpretation.
  4. §A.1.2: The dish value formula r_d(n_i + 0.2*n_i^1.5) is introduced to 'encourage the agents to cooperate and choose more complex dishes.' The coefficient 0.2 is a free parameter; a brief justification for this specific value or a note on sensitivity would be appropriate.
  5. The paper uses model names like 'GPT-5.1' and 'DeepSeek-V3.2' with footnote dates. For full reproducibility, consider including the exact API version strings or model identifiers in an appendix.
  6. Figure 2: The agent scaffold diagram is somewhat abstract. A more detailed description of how modules interact (e.g., what triggers re-entry into the observation module vs. planning module across rounds) would help readers understand the deliberation loop beyond Algorithm 1.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful and constructive review. The referee correctly identifies two issues that require attention: (1) the confound between multi-agent deliberation and multi-round reasoning in our centralization-vs-deliberation analysis, and (2) the need to scope our claims to self-play settings throughout the paper rather than only in the Limitations. We agree with both points and will revise the manuscript accordingly. On the first point, we will add a centralized multi-round self-reflection baseline to properly isolate the effect of deliberation from multi-round reasoning, and we will temper our claims to match what the evidence supports. On the second, we will add explicit self-play scoping language at key claim sites in the abstract, introduction, and results sections.

read point-by-point responses

  1. Referee: §4.3, Table 3: The claim that 'the deliberation process itself can provide opportunities for reflection and error correction' is confounded by an experimental design mismatch. The centralized baseline makes a single-shot decision, while the deliberative setting allows up to 6 rounds of interaction. The observed benefit cannot be attributed to multi-agent deliberation specifically rather than to multi-round reasoning. A centralized agent given multiple rounds of self-reflection would serve as the necessary control. Without it, the claim that deliberation per se adds value beyond information access is not supported.

    Authors: The referee is correct that our current experimental design does not cleanly separate the effect of multi-agent deliberation from the effect of multi-round reasoning. The centralized baseline is single-shot, while the deliberative setting allows up to 6 rounds, so the comparison conflates two variables. We agree that a centralized multi-round self-reflection baseline—in which a single agent with full information re-examines its own proposal across multiple rounds without a partner—is the necessary control to attribute any benefit to deliberation per se rather than to additional reasoning iterations. We will add this baseline in the revision and adjust our claims accordingly. Specifically, if the multi-round centralized baseline closes the gap with the deliberative setting, we will scope our claim to 'multi-round interaction' rather than 'multi-agent deliberation.' If the deliberative setting still outperforms, we will be able to make the stronger claim. Either way, the current wording in §4.3 and the abstract overstates what the existing evidence supports, and we will revise it to be precise about what is and is not demonstrated. We note that the oracle baseline (Table 3) does provide partial evidence that deliberation adds value beyond information access alone, since it combines full information with deliberation and still outperforms centralized for some models—but the referee's point stands that this does not isolate deliberation from multi-round reasoning, because the oracle also uses multiple rounds. We acknowledge this limitation honestly. revision: yes

  2. Referee: §4.1: All experiments use self-play (same model for all agents). The paper does not test cross-model play. While acknowledged in Limitations, the absence means findings about deliberation failures may not generalize to heterogeneous agent teams. The paper should explicitly scope its claims to self-play settings throughout the paper, not only in the Limitations.

    Authors: We agree. The Limitations section mentions this, but the main text does not consistently flag that all results are from self-play. This matters because deliberation dynamics—especially information exchange, proposal acceptance, and the 'skepticism trap'—could differ substantially in heterogeneous teams where models have different capabilities, prompt sensitivities, or reasoning styles. We will add explicit self-play scoping at the key claim sites: in the abstract ('we evaluate... in self-play settings'), in §4.1 (where the experimental setup is described), and in the results discussion where we draw conclusions about deliberation failures. We will not claim that findings generalize to cross-model play without evidence. We note that running cross-model experiments is feasible with our benchmark code and could be included, but given the combinatorial explosion of model pairs and the referee's suggestion that scoping is the minimum requirement, we will prioritize clear scoping in this revision and note cross-model evaluation as immediate future work. revision: yes

Circularity Check

0 steps flagged

No circularity found: benchmark rewards grounded in external integer programming solver, claims evaluated against solver-computed upper bounds

full rationale

The paper's central claims about LLM capabilities in deliberative collaboration are evaluated against rewards computed by an external integer programming solver (the `pulp` package), which provides independent grounding for the upper bounds and optimal solutions. The benchmark tasks are generated from databases initially created using LLMs and refined by authors, but the reward functions and optimal solutions are computed via external solvers, not by the same LLMs being evaluated. No prediction or first-principles result reduces to its inputs by construction. The deliberation-vs-centralization comparison (Section 4.3) has a confound (multi-round vs. single-shot reasoning), but this is an experimental design concern, not circularity. The oracle baseline combines full information with deliberation and is compared against both centralized and deliberative settings, but all reward computations remain externally grounded. No self-citation chain is load-bearing for the central claims. The derivation from problem formulation to evaluation metrics is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The paper introduces several hand-set parameters (temperature, round limits, observation probabilities, dish value formula) that shape the benchmark. No new entities or particles are postulated. The axioms are standard domain assumptions for LLM benchmark papers.

free parameters (4)
  • Temperature = 0.3
    Fixed across all models; not tuned per model.
  • Max deliberation rounds = 6
    Fixed turn limitation for the deliberation process.
  • Observed probability thresholds (menu-semantic) = various (e.g., 0.4-0.8)
    Hand-set probabilities for each persona field observed by each agent.
  • Dish value formula coefficient = 0.2
    The formula r_d(n_i + 0.2*n_i^1.5) uses 0.2 as a hand-tuned coefficient to encourage complex dishes.
axioms (3)
  • domain assumption LLM-generated databases refined by authors produce valid and unbiased task instances
    Section 2.3: databases are initially generated using LLMs and subsequently reviewed and refined by the authors.
  • domain assumption Self-play evaluation is representative of multi-agent collaboration capability
    Section 4.1: agents are evaluated in a self-play manner; cross-model play is not tested.
  • domain assumption The fixed reference scaffold (Figure 2) is a reasonable proxy for agent architecture
    Limitations section acknowledges results are representative under the evaluated scaffold.

pith-pipeline@v1.1.0-glm · 24700 in / 1905 out tokens · 438467 ms · 2026-07-08T14:45:47.597325+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

167 extracted references · 167 canonical work pages · 44 internal anchors

  1. [1]

    A Survey of Large Language Models

    A survey of large language models , author=. arXiv preprint arXiv:2303.18223 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [3]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Optima: Optimizing effectiveness and efficiency for llm-based multi-agent system , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    work page 2025

  3. [4]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    Mac-sql: A multi-agent collaborative framework for text-to-sql , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

    work page

  4. [6]

    2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=

    AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environments , author=. 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=. 2025 , organization=

    work page 2025

  5. [7]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Towards efficient llm grounding for embodied multi-agent collaboration , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    work page 2025

  6. [8]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track) , pages=

    To chat or task: a multi-turn dialogue generation framework for task-oriented dialogue systems , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track) , pages=

    work page

  7. [9]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Multi-agent collaboration via cross-team orchestration , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    work page 2025

  8. [10]

    International Conference on Principles and Practice of Multi-Agent Systems , pages=

    HMCF: A human-in-the-loop multi-robot collaboration framework based on large language models , author=. International Conference on Principles and Practice of Multi-Agent Systems , pages=. 2025 , organization=

    work page 2025

  9. [11]

    2025 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Coherent: Collaboration of heterogeneous multi-robot system with large language models , author=. 2025 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2025 , organization=

    work page 2025

  10. [13]

    ICML 2025 Workshop on Reliable and Responsible Foundation Models , year=

    The automated but risky game: Modeling agent-to-agent negotiations and transactions in consumer markets , author=. ICML 2025 Workshop on Reliable and Responsible Foundation Models , year=

    work page 2025

  11. [15]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Multiagentbench: Evaluating the collaboration and competition of llm agents , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    work page

  12. [17]

    ACM Transactions on Software Engineering and Methodology , volume=

    LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead , author=. ACM Transactions on Software Engineering and Methodology , volume=. 2025 , publisher=

    work page 2025

  13. [18]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Advancing collaborative debates with role differentiation through multi-agent reinforcement learning , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    work page

  14. [20]

    2025 , url =

    GLM-4.7: Advancing the Coding Capability , author =. 2025 , url =

    work page 2025

  15. [22]

    2025 , url =

    Qwen3-Next: A New Frontier in Large Language Models , author =. 2025 , url =

    work page 2025

  16. [23]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    Debate-to-write: A persona-driven multi-agent framework for diverse argument generation , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

    work page

  17. [25]

    A Survey on Large Language Model based Autonomous Agents

    A survey on large language model based autonomous agents , author=. arXiv preprint arXiv:2308.11432 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  18. [26]

    A Survey on Evaluation of Large Language Models

    A survey on evaluation of large language models , author=. arXiv preprint arXiv:2307.03109 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [27]

    and Cai, Carrie J

    Park, Joon Sung and O'Brien, Joseph C. and Cai, Carrie J. and Morris, Meredith Ringel and Liang, Percy and Bernstein, Michael S. , title =. In the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23) , keywords =. 2023 , publisher =

    work page 2023

  20. [28]

    Political Analysis , volume=

    Out of one, many: Using language models to simulate human samples , author=. Political Analysis , volume=. 2023 , publisher=

    work page 2023

  21. [29]

    Psychological reports , volume=

    Social intelligence: Its history and measurement , author=. Psychological reports , volume=. 1973 , publisher=

    work page 1973

  22. [30]

    SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents

    SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents , author=. arXiv preprint arXiv:2310.11667 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [31]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Alfred: A benchmark for interpreting grounded instructions for everyday tasks , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    work page

  24. [32]

    Revisiting the evaluation of theory of mind through question answering , author=. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages=

    work page 2019

  25. [33]

    Conference on Empirical Methods in Natural Language Processing , year=

    SocialIQA: Commonsense Reasoning about Social Interactions , author=. Conference on Empirical Methods in Natural Language Processing , year=

    work page

  26. [34]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Social-iq: A question answering benchmark for artificial social intelligence , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    work page

  27. [35]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    How Well Do Large Language Models Perform on Faux Pas Tests? , author=. Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    work page 2023

  28. [36]

    Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models

    Clever hans or neural theory of mind? stress testing social reasoning in large language models , author=. arXiv preprint arXiv:2305.14763 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  29. [37]

    Social-iq 2.0 challenge: Benchmarking multimodal social understanding , author=

    work page

  30. [38]

    Transactions on Machine Learning Research , issn=

    Evaluating Human-Language Model Interaction , author=. Transactions on Machine Learning Research , issn=. 2023 , url=

    work page 2023

  31. [39]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2022

  32. [40]

    Advances in Neural Information Processing Systems , volume=

    A simple language model for task-oriented dialogue , author=. Advances in Neural Information Processing Systems , volume=

    work page

  33. [41]

    Science , volume=

    Human-level play in the game of Diplomacy by combining language models with strategic reasoning , author=. Science , volume=. 2022 , publisher=

    work page 2022

  34. [42]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    work page

  35. [43]

    Expert Systems with Applications , volume=

    Towards personalized persuasive dialogue generation for adversarial task oriented dialogue setting , author=. Expert Systems with Applications , volume=. 2023 , publisher=

    work page 2023

  36. [44]

    Proceedings of the 4th Workshop on NLP for Conversational AI , pages=

    Open-domain dialogue generation: What we can do, cannot do, and should do next , author=. Proceedings of the 4th Workshop on NLP for Conversational AI , pages=

    work page

  37. [45]

    SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization

    Soda: Million-scale dialogue distillation with social commonsense contextualization , author=. arXiv preprint arXiv:2212.10465 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [46]

    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Personalizing Dialogue Agents: I have a dog, do you have pets too? , author=. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    work page

  39. [47]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

    You Impress Me: Dialogue Generation via Mutual Persona Perception , author=. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

    work page

  40. [48]

    SN Computer Science , volume=

    The power of personalization: A systematic review of personality-adaptive chatbots , author=. SN Computer Science , volume=. 2023 , publisher=

    work page 2023

  41. [49]

    The Open-domain Paradox for Chatbots: Common Ground as the Basis for Human-like Dialogue

    The Open-domain Paradox for Chatbots: Common Ground as the Basis for Human-like Dialogue , author=. arXiv preprint arXiv:2303.11708 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  42. [50]

    Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology , pages=

    Social simulacra: Creating populated prototypes for social computing systems , author=. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology , pages=

    work page

  43. [51]

    Findings of the Association for Computational Linguistics: EMNLP 2022 , pages=

    Language Models as Agent Models , author=. Findings of the Association for Computational Linguistics: EMNLP 2022 , pages=

    work page 2022

  44. [52]

    Voyager: An Open-Ended Embodied Agent with Large Language Models

    Voyager: An open-ended embodied agent with large language models , author=. arXiv preprint arXiv:2305.16291 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  45. [53]

    Can Large Language Models Transform Computational Social Science?

    Can Large Language Models Transform Computational Social Science? , author=. arXiv preprint arXiv:2305.03514 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  46. [54]

    NeurIPS 2023 AI for Science Workshop , year=

    Augmenting large language models with chemistry tools , author=. NeurIPS 2023 AI for Science Workshop , year=

    work page 2023

  47. [55]

    Conference on Robot Learning , pages=

    Do as i can, not as i say: Grounding language in robotic affordances , author=. Conference on Robot Learning , pages=. 2023 , organization=

    work page 2023

  48. [56]

    International Conference on Machine Learning , pages=

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents , author=. International Conference on Machine Learning , pages=. 2022 , organization=

    work page 2022

  49. [57]

    International Conference on Learning Representations , year=

    ALFWorld: Aligning Text and Embodied Environments for Interactive Learning , author=. International Conference on Learning Representations , year=

    work page

  50. [58]

    International Conference on Machine Learning , pages=

    Using large language models to simulate multiple humans and replicate human subject studies , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    work page 2023

  51. [59]

    AgentBench: Evaluating LLMs as Agents

    Agentbench: Evaluating llms as agents , author=. arXiv preprint arXiv:2308.03688 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  52. [60]

    MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming

    Metaagents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collaborative generative agents , author=. arXiv preprint arXiv:2310.06500 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  53. [61]

    Advances in Neural Information Processing Systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in Neural Information Processing Systems , volume=

    work page

  54. [62]

    2023 , eprint=

    GPT-4 Technical Report , author=. 2023 , eprint=

    work page 2023

  55. [63]

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  56. [64]

    Baichuan 2: Open Large-scale Language Models

    Baichuan 2: Open large-scale language models , author=. arXiv preprint arXiv:2309.10305 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  57. [65]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

    work page

  58. [66]

    2025 , url =

    GPT-5.1: A smarter, more conversational ChatGPT , author =. 2025 , url =

    work page 2025

  59. [67]

    2025 , url =

    Introducing GPT-4.1 in the API , author =. 2025 , url =

    work page 2025

  60. [68]

    S$^3$: Social-network Simulation System with Large Language Model-Empowered Agents

    S3: Social-network Simulation System with Large Language Model-Empowered Agents , author=. arXiv preprint arXiv:2307.14984 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  61. [69]

    NeurIPS 2023 Foundation Models for Decision Making Workshop , year=

    Building Cooperative Embodied Agents Modularly with Large Language Models , author=. NeurIPS 2023 Foundation Models for Decision Making Workshop , year=

    work page 2023

  62. [70]

    Advances in Neural Information Processing Systems , year=

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author=. Advances in Neural Information Processing Systems , year=

    work page

  63. [71]

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models

    Tree of thoughts: Deliberate problem solving with large language models , author=. arXiv preprint arXiv:2305.10601 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  64. [72]

    Graph of Thoughts: Solving Elaborate Problems with Large Language Models

    Graph of thoughts: Solving elaborate problems with large language models , author=. arXiv preprint arXiv:2308.09687 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  65. [73]

    Contrastive Chain-of-Thought Prompting

    Contrastive Chain-of-Thought Prompting , author=. arXiv preprint arXiv:2311.09277 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  66. [74]

    Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine

    Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine , author=. arXiv preprint arXiv:2311.16452 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  67. [75]

    ReAct: Synergizing Reasoning and Acting in Language Models

    React: Synergizing reasoning and acting in language models , author=. arXiv preprint arXiv:2210.03629 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  68. [76]

    Large Language Models Cannot Self-Correct Reasoning Yet

    Large language models cannot self-correct reasoning yet , author=. arXiv preprint arXiv:2310.01798 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  69. [77]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Reflexion: Language agents with verbal reinforcement learning , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

    work page

  70. [78]

    BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents

    Bolaa: Benchmarking and orchestrating llm-augmented autonomous agents , author=. arXiv preprint arXiv:2308.05960 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  71. [79]

    MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

    Metagpt: Meta programming for multi-agent collaborative framework , author=. arXiv preprint arXiv:2308.00352 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  72. [80]

    Humanoid Agents: Platform for Simulating Human-like Generative Agents

    Humanoid agents: Platform for simulating human-like generative agents , author=. arXiv preprint arXiv:2310.05418 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  73. [82]

    Forty-first International Conference on Machine Learning , year=

    How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis , author=. Forty-first International Conference on Machine Learning , year=

    work page

  74. [83]

    The Twelfth International Conference on Learning Representations , year=

    SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents , author=. The Twelfth International Conference on Learning Representations , year=

    work page

  75. [84]

    Findings of the Association for Computational Linguistics

    Towards Objectively Benchmarking Social Intelligence for Language Agents at Action Level , author=. Findings of the Association for Computational Linguistics

    work page

  76. [85]

    Agentsense: Benchmarking social intelligence of language agents through interactive scenarios , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    work page 2025

  77. [86]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2024

  78. [89]

    Findings of the Association for Computational Linguistics: EMNLP 2024

    Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues , author=. Findings of the Association for Computational Linguistics: EMNLP 2024. doi:10.18653/v1/2024.findings-emnlp.310

    work page doi:10.18653/v1/2024.findings-emnlp.310 2024

  79. [90]

    Advances in neural information processing systems , volume=

    On the utility of learning about humans for human-ai coordination , author=. Advances in neural information processing systems , volume=

    work page

  80. [91]

    The Twelfth International Conference on Learning Representations , year=

    AgentBench: Evaluating LLMs as Agents , author=. The Twelfth International Conference on Learning Representations , year=

    work page

Showing first 80 references.