Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that self-interested LLM agents can be made to cooperate in open-ended environments if they negotiate task-time transfer payments and redistribute rewards afterward using Shapley-value marginal contributions.

desk verdict The Shapley equation in the paper doesn't compute a Shapley value, so the central mechanism is mislabeled, but the negotiation workflow and empirical comparison are worth a second look. read the letter →

arxiv 2506.07388 v1 pith:PNHHBTNW submitted 2025-06-09 cs.MA cs.AI

Yun Hua , Haosheng Chen , Shiqin Wang , Wenhao Li , Xiangfeng Wang , Jun Luo This is my paper · ORCID
classification cs.MAcs.AI
keywords multi-agentLLMcooperationShapleyvaluecreditassignmentnegotiationprotocolemergentpricingmechanismsocialdilemmasrewardredistribution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Self-interested LLM agents, left to their own devices, fall into social dilemmas in open-ended tasks; the paper's claim is that they can be brought to cooperate by turning contribution assessment into a pricing problem. Shapley-Coop does this in two moves: a short-term Shapley Chain-of-Thought classifies each agent's action as creating a positive or negative externality, triggering an offer to pay or request payment during the task, and a long-term Shapley Chain-of-Thought estimates marginal contributions after the task and negotiates reward redistribution. The paper reports that this workflow achieves 100% cooperative success in a two-agent Escape Room where plain negotiation succeeds 25% of the time and no mechanism 0%, produces balanced roles and near-expected Shapley reward shares in a four-agent Raid Battle, and yields software-engineering reward splits within a few percent of a weighted earned-value benchmark. If the claim is right, it gives a practical way to align heterogeneous LLM agent goals while preserving their autonomy.

What carries the argument

The central machinery is the Shapley value paired with a two-stage Chain-of-Thought. The Shapley value $$\phi_i = \sum_{C\subseteq \{1,\dots,N\}\setminus\{i\}} \frac{|C|!(N-|C|-1)!}{N!}\bigl(R(C\cup\{i\}) - R(C)\bigr)$$ measures the average marginal contribution of agent $i$ over all coalitions. Shapley-Coop operationalizes this for LLM agents with (1) a structured negotiation protocol using machine-readable messages, (2) Short-Term Shapley CoT, which estimates whether an action creates positive or negative externalities and thus whether pricing is needed, and (3) Long-Term Shapley CoT, which computes the coalition reward from the trajectory, estimates the counterfactual marginal contribution $\Delta_i(N,\tau_N) = R(N,\tau_N) - R(N\setminus\{i\},\tau_N)$, and feeds the resulting Shapley value into negotiation. The negotiation loop turns these numbers into agreed transfers, closing the pricing cycle.

What would settle it

Take a recorded Raid Battle trajectory from a successful Shapley-Coop run and recompute, in the simulator, the global reward with each agent's actions replaced by inaction while keeping the other agents' actions fixed. If the resulting marginal contributions yield Shapley shares that differ considerably from the shares the agents negotiated, then the workflow's credit assignment is not actually tracking Shapley values; if they match, the central mechanism is supported.

Watch

Extended reading notes

Core claim

Shapley-Coop's central claim is that fair pricing based on marginal contributions is the mechanism that lets self-interested LLM agents cooperate without predefined roles, and that Shapley values supply the price. In the Escape Room, the Shapley value assigns each agent 4.5, so the agent who collects the +10 door payoff transfers 5.5 to the lever-puller; the paper argues this transfer converts a payoff matrix whose Nash equilibrium is mutual inaction into one where cooperation is individually rational. For multi-step tasks, exact Shapley computation is replaced by an LLM-guided approximation: agents qualitatively estimate the sign of their externality at task time, then after the task compute the observed coalition reward $R(N,\tau_N)$, estimate their marginal contribution $\Delta_i(N,\tau_N) = R(N,\tau_N) - R(N\setminus\{i\},\tau_N)$, and enter structured negotiation over the resulting Shapley value. The experimental sections take the high cooperation rates, balanced role distributions, and small gaps between allocated and expected Shapley shares as evidence that this pricing loop works.

Load-bearing premise

The load-bearing premise is the long-term Shapley step's requirement that each agent can obtain the counterfactual group reward $R(N\setminus\{i\},\tau_N)$ — what the team would have earned without agent $i$ — from a single completed trajectory; the paper invokes this value in Eq. (8) but gives no method for computing it in the test environments.

Editorial extensions

If this is right

  • In the two-agent Escape Room, the full Shapley-Coop workflow reaches 100% successful cooperation with the Shapley-fair split of 4.5 for each agent, compared with 25% success for plain negotiation and 0% for no mechanism.
  • In the four-agent Raid Battle, Shapley-Coop shifts agents into taunting and healing roles and lands reward allocations within about 0.2 to 2.9 percentage points of the expected Shapley shares, while short-term-only reasoning under-rewards support roles by 3.7 to 6.3 points.
  • Increasing the number of negotiation rounds moves the final reward allocation monotonically toward the expected Shapley allocation, indicating that bargaining can approximate exact Shapley arithmetic.
  • In the software-engineering simulation, reward shares stay within roughly 6 percentage points of the weighted earned-value benchmark, with hands-on roles matching almost exactly.
  • Together the experiments support the paper's claim that pricing based on marginal contributions, not fixed roles or rules, is what enables spontaneous cooperation among self-interested LLM agents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A working deployment would have to supply the counterfactual $R(N\setminus\{i\},\tau_N)$ in Eq. (8) somehow—through a simulator, a learned reward model, or an agreed hypothetical—since a single trajectory does not reveal what would have happened without agent $i$; the paper leaves this unspecified.
  • A direct way to test whether the Shapley arithmetic is doing the work is to replay recorded Raid Battle trajectories with each agent's actions removed and compare simulator-computed Shapley shares with the shares the agents negotiated; a large mismatch would mean the bargaining loop, not the Shapley calculation, drives the outcome.
  • The software-engineering validation uses weighted earned value, a project-management heuristic, rather than true counterfactual Shapley contributions; extending the workflow there would need a way to estimate how the final artifact changes when a role is absent.
  • If the short-term externality classification is what actually produces cooperation, a stripped-down mechanism that simply triggers transfers on action type might capture most of the benefit; comparing Shapley-Coop against such a baseline would isolate the contribution of the Shapley reasoning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes Shapley-Coop, a workflow for coordinating self-interested LLM agents in open-ended multi-agent tasks. The workflow combines a structured negotiation protocol with two reasoning modules: a Short-Term Shapley Chain-of-Thought that heuristically aligns agents' goals through task-time pricing, and a Long-Term Shapley Chain-of-Thought that retrospectively computes Shapley values for post-task reward redistribution. The authors evaluate the approach in an Escape Room social dilemma, a four-agent Raid Battle game, and a ChatDEV software-engineering simulation, reporting improved cooperation and fairer credit assignment compared to baselines without Shapley reasoning.

Significance. The motivating problem is timely: enabling self-interested LLM agents to cooperate in open-ended settings requires incentive alignment, and framing this as a credit-assignment problem with pricing is a reasonable design choice. The modular structure of the workflow and the use of structured negotiation are sensible. However, the central formal mechanism is not the Shapley value claimed in the title and abstract. Equation (9) reduces to a single grand-coalition marginal contribution because the term being averaged over coalitions is coalition-independent, and the counterfactual reward in Eq. (8) is not defined for a single observed trajectory. Since the paper's contribution is precisely Shapley-based credit assignment, these issues undermine the core claim. The empirical evaluation also lacks a definition of the 'Expected' fairness benchmark, error bars, and a stated number of runs, so the reported quantitative comparisons are difficult to assess. If the formal derivation were corrected and the evaluation made rigorous, the problem and workflow could be of interest to the multi-agent LLM community, but the current manuscript does not support its central claims.

major comments (4)
  1. [Section 3, Eq. (9)] Equation (9) does not compute a Shapley value. The term Δ_i(N, τ_N) defined in Eq. (8) is R(N, τ_N) - R(N\{i}, τ_N) and has no dependence on the coalition C. Factoring it out of the weighted sum gives φ_i(τ_N) = Δ_i(N, τ_N) because the Shapley weights sum to 1 over all coalitions C ⊆ N\{i}. The apparent averaging over coalitions is thus vacuous, and every agent is assigned its raw marginal contribution with respect to the grand coalition, not an average of marginal contributions over all coalitions. In the paper's own two-agent Escape Room example, this formula would assign each agent 9, rather than the 4.5 computed in Section 2, and the resulting allocation would not generally sum to the total reward. This is a load-bearing error because the title, abstract, and conclusion all claim that Shapley-Coop implements Shapley-based credit assignment.
  2. [Section 3, Eq. (8) and the Long-Term Shapley CoT prompts] The counterfactual quantity R(N\{i}, τ_N) is not well-defined for a single observed trajectory τ_N. Removing agent i from the trajectory requires specifying what the remaining agents would have done in the absence of i, but the paper provides no generative model, simulation procedure, or estimation method for this counterfactual. The prompt 'call external calculation function' does not identify such a function, and in the tested environments the counterfactual is not identifiable from the observed trajectory alone. Because every marginal contribution and every negotiated transfer in the Long-Term Shapley CoT depends on this quantity, the core mechanism rests on an unsupported assumption.
  3. [Section 4 and Appendix B, Raid Battle evaluation] The 'Expected' reward allocation in Figure 5 and Tables 4–6 is never defined. Without a precise definition of this benchmark, it is impossible to verify the claim that LLM+SC is more accurate than LLM+STS. Moreover, the reported reward-allocation percentages and contribution scores appear to be single values with no error bars, confidence intervals, or number of independent runs, so the statistical significance of the claimed improvements (e.g., the '+209%' and '+94%' improvements in Figure 4) cannot be assessed.
  4. [Section 4, Table 3 (ChatDEV)] The ChatDEV analysis does not substantiate the claim that the workflow achieves fair credit assignment. The WEV benchmark is reported as a range (e.g., 7.5–17.5 for the CEO), while the allocated reward is a single value, and the 'Adj.' column reports differences that are not tied to any hypothesis test or error analysis. In particular, the CTO adjustment of -13.3% in the BMI task is described inconsistently with the claim that gaps are 'minor' and below 6% for most roles. The table alone does not establish that the negotiated rewards reflect marginal contributions.
minor comments (5)
  1. [Abstract and Introduction] There are repeated typos such as 'incredit assignment' instead of 'in credit assignment' in the abstract and introduction.
  2. [Figure 2] The caption refers to the 'Shapely-Coop Workflow' with a misspelling; it should be 'Shapley-Coop'.
  3. [Section 3, Long-Term Shapley CoT, Step 1] The phrase 'first step in calculating the Shapley value shown in Equation. 1' has an unclear reference; the intended equation for the total reward appears to be Eq. (7), not Eq. (1).
  4. [Section 4 and Appendix B] The ChatDEV description and Table 3 are nearly identical in Section 4 and Appendix B.2; the duplicated text should be consolidated to avoid redundancy.
  5. [References] Reference [23] appears incomplete: it is listed as 'Model Definition Manual. Cocomo ii. 1995.' rather than as a standard citation for COCOMO II.

Circularity Check

1 steps flagged · score 6.0 of 10

Eq. 9's Shapley averaging is vacuous: the purported Shapley value reduces to the raw marginal contribution by construction, so the central Long-Term Shapley CoT mechanism collapses to its own input.

  1. self definitional [Section 3, Long-Term Shapley Chain-of-Thought, Eqs. (8)-(9)]
    "Formally, the marginal contribution is defined as: ∆i(N, τN ) = R(N, τN )−R(N/{i}, τN ).(8) ... ϕi(τN ) = X C⊆{1,...,N}\{i} |C|! (N− |C| −1)! / N! (∆i(N, τN ) ).(9)"

    Δ_i(N,τ_N) depends only on the grand coalition N and not on the summation variable C. Since the Shapley weights sum to 1 over all C⊆N\{i}, Eq. (9) simplifies to φ_i(τ_N)=Δ_i(N,τ_N). The purported averaging over all coalitions is therefore empty: the output of the Long-Term Shapley CoT is identical, by construction, to the single grand-coalition marginal contribution already defined in Eq. (8). The paper calls this a Shapley value, but it reduces to its own input rather than computing marginal contributions to every coalition R(C∪{i})−R(C). This also contradicts the paper's own Escape Room calculation, where the correct Shapley value is 4.5, whereas Eq. (9) would assign the raw marginal contribution.

full rationale

The only genuine reduction-by-construction I can exhibit is in Long-Term Shapley CoT: Eq. (9) sums coalition weights against a term Δ_i(N,τ_N) that is independent of the coalition, so the Shapley value collapses to the marginal contribution Δ_i(N,τ_N) and the central credit-assignment claim loses its Shapley content. This is load-bearing because the title and abstract present the mechanism as Shapley-based pricing. I do not count the ill-defined counterfactual R(N\{i},τ_N) in Eq. (8) as circularity; it is a missing-support/correctness problem rather than a reduction to inputs. The empirical comparisons are not themselves fitted to the target allocations: the Raid Battle 'Expected' values are computed from observed contribution data, and ChatDEV uses external WEV benchmarks, so those parts do not exhibit fitted-input circularity. There are no load-bearing self-citations. Overall score 6: one central 'prediction' (the Shapley value estimate) reduces by definition to its input, while the negotiation and short-term reasoning components retain independent content.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. Its central dependence is on the availability of counterfactual rewards and on LLM agents' ability to reason accurately about marginal contributions, both of which are assumed without direct support.

assumptions (4)
  • standard math The Shapley value formula (Eq. 3) is taken as the correct credit allocation scheme.
    Standard cooperative game theory result, not derived in the paper.
  • domain assumption Agents can compute the counterfactual global reward R(N\{i}, τ) from a single trajectory.
    Needed for Eq. 8; paper invokes 'call external calculation function' without specifying how the counterfactual is obtained.
  • domain assumption LLM agents' qualitative assessments of externalities and marginal contribution estimates are sufficiently accurate for negotiation to converge to fair outcomes.
    Central to the workflow; no validation of the reliability of these estimates is provided.
  • domain assumption The negotiation protocol converges to an agreement and agents are truthful about their valuations.
    The paper assumes proposals and counter-proposals lead to agreement; no mechanism enforces truthfulness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents." pith.science (2026). https://pith.science/paper/PNHHBTNW

@misc{pith2026250607388,
  author       = {Pith},
  title        = {Pith review of: Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PNHHBTNW}},
  note         = {Machine review of arXiv:2506.07388}
}
read the original abstract

Large Language Models (LLMs) show strong collaborative performance in multi-agent systems with predefined roles and workflows. However, in open-ended environments lacking coordination rules, agents tend to act in self-interested ways. The central challenge in achieving coordination lies in credit assignment -- fairly evaluating each agent's contribution and designing pricing mechanisms that align their heterogeneous goals. This problem is critical as LLMs increasingly participate in complex human-AI collaborations, where fair compensation and accountability rely on effective pricing mechanisms. Inspired by how human societies address similar coordination challenges (e.g., through temporary collaborations such as employment or subcontracting), we propose a cooperative workflow, Shapley-Coop. Shapley-Coop integrates Shapley Chain-of-Thought -- leveraging marginal contributions as a principled basis for pricing -- with structured negotiation protocols for effective price matching, enabling LLM agents to coordinate through rational task-time pricing and post-task reward redistribution. This approach aligns agent incentives, fosters cooperation, and maintains autonomy. We evaluate Shapley-Coop across two multi-agent games and a software engineering simulation, demonstrating that it consistently enhances LLM agent collaboration and facilitates equitable credit assignment. These results highlight the effectiveness of Shapley-Coop's pricing mechanisms in accurately reflecting individual contributions during task execution.

Figures

Figures reproduced from arXiv: 2506.07388 by the authors.

Figure 1
Figure 1. Escape room: One agent pulls a lever (−1) to let the other escape through a door (+10). Cooperation is necessary. misalignments often induce social dilemmas, where private optimization creates spillover effects on system-level outcomes. Consider the Escape Room scenario ( [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The Shapley-Coop Workflow for spontaneous cooperation among self-interested LLM [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Comparison of agent payoffs and negotiation [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of Contributions for Raid Battle [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparison of Reward Allocation/Credit Assignment for Raid Battle [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Game scene example of Raid Battle B.1 Raid Battle To further evaluate the effectiveness of the Shapley Coop framework in a more complex, multi-turn, and multi-agent environment, the Raid Battle scenario is introduced ( [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: The Example of ChatDEV (2) ArtCanvas: Create a virtual painting studio app providing canvas, brushes, and color palettes. We measured contributions using weighted earned value (WEV), a widely-adopted project manage￾ment metric [23], using four key artefacts already rou…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Training Small LLMs as Spatial Multi-Agent Policies

    cs.MA 2026-08 conditional novelty 7.0 of 10

    Small frozen LLMs trained over state-filtered symbolic option menus with per-agent LoRA adapters reach competent play in three cooperative spatial games, while behavioral audits show reward and cooperation decouple.

Reference graph

Works this paper leans on

45 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    Playing repeated games with large language models

    Elif Akata, Lion Schulz, Julian Coda-Forno, SeongJoon Oh, Matthias Bethge, and Eric Schulz. Playing repeated games with large language models. May 2023

    work page 2023

  2. [2]

    Welfare economics

    Mark Blaug. Welfare economics. InA Handbook of Cultural Economics, Second Edition. Edward Elgar Publishing, 2011

    work page 2011

  3. [3]

    Software engineering economics.IEEE transactions on Software Engineering, (1):4–21, 1984

    Barry W Boehm. Software engineering economics.IEEE transactions on Software Engineering, (1):4–21, 1984

    work page 1984

  4. [4]

    Multi-agent consensus seeking via large language models.arXiv preprint arXiv:2310.20151, 2023

    Huaben Chen, Wenkang Ji, Lufeng Xu, and Shiyu Zhao. Multi-agent consensus seeking via large language models.arXiv preprint arXiv:2310.20151, 2023

    arXiv 2023

  5. [5]

    Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. InICLR, 2024

    2024

  6. [6]

    Llm agents for education: Advances and applications

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, et al. Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733, 2025

    arXiv 2025

  7. [7]

    Cogbench: a large language model walks into a psychology lab.arXiv preprint arXiv:2402.18225, 2024

    Julian Coda-Forno, Marcel Binz, Jane X Wang, and Eric Schulz. Cogbench: a large language model walks into a psychology lab.arXiv preprint arXiv:2402.18225, 2024

    arXiv 2024

  8. [8]

    Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations.arXiv preprint arXiv:2402.12348, 2024

    Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, and Kaidi Xu. Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations.arXiv preprint arXiv:2402.12348, 2024

    arXiv 2024

Show all 45 references
  1. [9]

    Can large language models serve as rational players in game theory? a systematic analysis

    Caoyun Fan, Jindou Chen, Yaohui Jin, and Hao He. Can large language models serve as rational players in game theory? a systematic analysis. Dec 2023

    2023

  2. [10]

    Information and the coase theorem.Journal of Economic Perspectives, 1(2):113– 129, 1987

    Joseph Farrell. Information and the coase theorem.Journal of Economic Perspectives, 1(2):113– 129, 1987

    1987

  3. [11]

    MIT press, 1991

    Drew Fudenberg and Jean Tirole.Game theory. MIT press, 1991

    1991

  4. [12]

    Openagi: When llm meets domain experts, 2023

    Yingqiang Ge, Wenyue Hua, Kai Mei, Jianchao Ji, Juntao Tan, Shuyuan Xu, Zelong Li, and Yongfeng Zhang. Openagi: When llm meets domain experts, 2023

    2023

  5. [13]

    Gpt in game theory experiments

    Fulin Guo. Gpt in game theory experiments. Dec 2023

    2023

  6. [14]

    Metagpt: Meta programming for a multi-agent collaborative framework

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta programming for a multi-agent collaborative framework. InICLR, 2024

    2024

  7. [15]

    Game-theoretic llm: Agent workflow for negotiation games.arXiv preprint arXiv:2411.05990, 2024

    Wenyue Hua, Ollie Liu, Lingyao Li, Alfonso Amayuelas, Julie Chen, Lucas Jiang, Mingyu Jin, Lizhou Fan, Fei Sun, William Wang, et al. Game-theoretic llm: Agent workflow for negotiation games.arXiv preprint arXiv:2411.05990, 2024

    2024 arXiv

  8. [16]

    Jen-tse Huang, EricJohn Li, ManHo Lam, Tian Liang, Wenxuan Wang, Youliang Yuan, Wenxi- ang Jiao, Xing Wang, Zhaopeng Tu, and MichaelR. Lyu. How far are we on the decision-making of llms? evaluating llms’ gaming ability in multi-agent environments. Apr 2024

    2024

  9. [17]

    A comprehensive study on llm agent challenges

    Palash Ingle, Mithun Parab, Pranay Lendave, Amisha Bhanushali, and Pavan Kumar Bn. A comprehensive study on llm agent challenges

  10. [18]

    Camel: Communicative agents for "mind" exploration of large language model society, 2023

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large language model society, 2023

    2023

  11. [19]

    Theory of mind for multi-agent collaboration via large language models

    Huao Li, Yu Chong, Simon Stepputtis, Joseph P Campbell, Dana Hughes, Charles Lewis, and Katia Sycara. Theory of mind for multi-agent collaboration via large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 180–19...

    2023

  12. [20]

    Metaagents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collaborative generative agents, 2023

    Yuan Li, Yixuan Zhang, and Lichao Sun. Metaagents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collaborative generative agents, 2023

    2023

  13. [21]

    Strategic behavior of large language models: Game structure vs

    Nunzio Lorè and Babak Heydari. Strategic behavior of large language models: Game structure vs. contextual framing.arXiv preprint arXiv:2309.05898, 2023

    2023 arXiv

  14. [22]

    Fingpt: Large generative models for a small language

    Risto Luukkonen, Ville Komulainen, Jouni Luoma, Anni Eskelinen, Jenna Kanerva, Hanna- Mari Kupari, Filip Ginter, Veronika Laippala, Niklas Muennighoff, Aleksandra Piktus, et al. Fingpt: Large generative models for a small language. InProceedings of the 2023 Conference on Empir...

    2023

  15. [23]

    Model Definition Manual.Cocomo ii. 1995

    1995

  16. [24]

    Runtime verification of self-adaptive multi-agent system using probabilistic timed automata.Journal of Intelligent & Fuzzy Systems, 45(6):10305–10322, 2023

    Yongan Mu, Wei Liu, Tao Lu, Juan Li, Sheng Gao, and Zihao Wang. Runtime verification of self-adaptive multi-agent system using probabilistic timed automata.Journal of Intelligent & Fuzzy Systems, 45(6):10305–10322, 2023

    2023

  17. [25]

    Ai deception: A survey of examples, risks, and potential solutions.Patterns, 5(5), 2024

    Peter S Park, Simon Goldstein, Aidan O’Gara, Michael Chen, and Dan Hendrycks. Ai deception: A survey of examples, risks, and potential solutions.Patterns, 5(5), 2024

    2024

  18. [26]

    Cooperate or collapse: Emergence of sustainable cooperation in a society of llm agents.Advances in Neural Information Processing Systems, 37:111715–111759, 2024

    Giorgio Piatti, Zhijing Jin, Max Kleiman-Weiner, Bernhard Schölkopf, Mrinmaya Sachan, and Rada Mihalcea. Cooperate or collapse: Emergence of sustainable cooperation in a society of llm agents.Advances in Neural Information Processing Systems, 37:111715–111759, 2024

    2024

  19. [27]

    Chatdev: Communicative agents for software development

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. Chatdev: Communicative agents for software development. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1:...

    2024

  20. [28]

    Agent laboratory: Using llm agents as research assistants

    Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227, 2025

    2025 arXiv

  21. [29]

    Enhancing trust in llm-based ai automation agents: New considerations and future challenges.arXiv preprint arXiv:2308.05391, 2023

    Sivan Schwartz, Avi Yaeli, and Segev Shlomov. Enhancing trust in llm-based ai automation agents: New considerations and future challenges.arXiv preprint arXiv:2308.05391, 2023

    2023 arXiv

  22. [30]

    True knowledge comes from practice: Aligning llms with embodied environments via reinforcement learning.arXiv preprint arXiv:2401.14151, 2024

    Weihao Tan, Wentao Zhang, Shanqi Liu, Longtao Zheng, Xinrun Wang, and Bo An. True knowledge comes from practice: Aligning llms with embodied environments via reinforcement learning.arXiv preprint arXiv:2401.14151, 2024

    2024 arXiv

  23. [31]

    Creating large language model applications utilizing langchain: A primer on developing llm apps fast

    Oguzhan Topsakal and Tahir Cetin Akinci. Creating large language model applications utilizing langchain: A primer on developing llm apps fast. InInternational Conference on Applied Engineering and Natural Sciences, volume 1, pages 1050–1056, 2023

    2023

  24. [32]

    Avalon’s game of thoughts: Battle against deception through recursive contemplation, 2023

    Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. Avalon’s game of thoughts: Battle against deception through recursive contemplation, 2023

    2023

  25. [33]

    Will systems of llm agents cooperate: An investigation into a social dilemma.arXiv preprint arXiv:2501.16173, 2025

    Richard Willis, Yali Du, Joel Z Leibo, and Michael Luck. Will systems of llm agents cooperate: An investigation into a social dilemma.arXiv preprint arXiv:2501.16173, 2025

    2025 arXiv

  26. [34]

    Autogen: Enabling next-gen llm applications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023

    2023 arXiv

  27. [35]

    Stateflow: Enhancing llm task-solving through state-driven workflows, 2024

    Yiran Wu, Tianwei Yue, Shaokun Zhang, Chi Wang, and Qingyun Wu. Stateflow: Enhancing llm task-solving through state-driven workflows, 2024

    2024

  28. [36]

    The rise and potential of large language model based agents: A survey, 2023

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, W...

    2023

  29. [37]

    Agentless: Demystifying llm-based software engineering agents.arXiv preprint arXiv:2407.01489, 2024

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying llm-based software engineering agents.arXiv preprint arXiv:2407.01489, 2024

    2024 arXiv

  30. [38]

    Flowbench: Revisiting and benchmarking workflow-guided planning for llm-based agents.arXiv preprint arXiv:2406.14884, 2024

    Ruixuan Xiao, Wentao Ma, Ke Wang, Yuchuan Wu, Junbo Zhao, Haobo Wang, Fei Huang, and Yongbin Li. Flowbench: Revisiting and benchmarking workflow-guided planning for llm-based agents.arXiv preprint arXiv:2406.14884, 2024

    2024 arXiv

  31. [39]

    Travelplanner: A benchmark for real-world planning with language agents

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. Travelplanner: A benchmark for real-world planning with language agents. Feb 2024

    2024

  32. [40]

    Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration

    Lin Xu, Zhiyuan Hu, Daquan Zhou, Hongyu Ren, Zhen Dong, Kurt Keutzer, See Kiong Ng, and Jiashi Feng. Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration. InProceedings of the 2024 Conference on Empirical Me...

    2024

  33. [41]

    Phase distribution of software development effort

    Ye Yang, Mei He, Mingshu Li, Qing Wang, and Barry Boehm. Phase distribution of software development effort. InProceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, pages 61–69, 2008

    2008

  34. [42]

    Thought propagation: An analogical approach to complex reasoning with large language models, 2024

    Junchi Yu, Ran He, and Rex Ying. Thought propagation: An analogical approach to complex reasoning with large language models, 2024

    2024

  35. [43]

    Flowmind: automatic workflow generation with llms

    Zhen Zeng, William Watson, Nicole Cho, Saba Rahimi, Shayleen Reynolds, Tucker Balch, and Manuela Veloso. Flowmind: automatic workflow generation with llms. InProceedings of the Fourth ACM International Conference on AI in Finance, pages 73–81, 2023

    2023

  36. [44]

    Exploring collaboration mechanisms for llm agents: A social psychology view

    Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. Exploring collaboration mechanisms for llm agents: A social psychology view. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14...

    2024

  37. [45]

    Agent4 should Heal Agent2 (200 HP). Agent1 and Agent3 Fireball for maximum damage

    Benhui Zhuang, Chunhong Zhang, and Zheng Hu. Pose: Suppressing perceptual noise in embodied agents for enhanced semantic navigation.IEEE Robotics and Automation Letters, 9(2):963–970, 2023. 12 A Related Work LLMs in Multi-Agent Game EnvironmentsThe study of how large language ...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.