Pith. sign in

REVIEW 4 major objections 5 minor 69 references

LLM agents cast as SAFe roles can carry out Agile project-management work and improve measured outcomes, the paper argues.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

LLM agents acting as Agile roles produced plausible project artifacts in simulation, but the claimed improvements over human teams are unsupported because no comparison or validated metrics are provided.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A plausible LLM-agent framework for Agile/SAFe that overclaims measurable gains in the abstract, then walks them back in its own conclusions; the empirical section is not interpretable as evidence. the 4 major comments →

arxiv 2508.16678 v1 pith:BUB7GQK2 submitted 2025-08-21 cs.SE cs.MA

Cognitive Agents Powered by Large Language Models for Agile Software Project Management

Konrad Cinkusz , Jaros{\l}aw A. Chudziak , Ewa Niewiadomska-Szynkiewicz This is my paper
classification cs.SE cs.MA
keywords cognitive agentslarge language modelsAgile software project managementScaled Agile Frameworkmulti-agent systemsCogniSimsimulated software projects
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that LLM-powered cognitive agents, assigned to roles such as Product Owner, System Architect, and QA Engineer and coordinated through a multi-agent platform named CogniSim, can carry out core Scaled Agile Framework (SAFe) activities by talking to each other in natural language. It reports that in simulated Program Increment planning, iteration execution, and inspect-and-adapt workshops, the agents produced user stories, code snippets, architectural plans, and CI/CD scripts while reducing task completion times relative to human-managed processes and keeping communication coherent. A sympathetic reader would care because this is a concrete template for automating routine coordination work in software teams, freeing humans for strategic decisions; the authors frame the results as preliminary and call for real-project validation before the gains are treated as established. If the framework performs as claimed, LLM agents could become repeatable, configurable participants in Agile ceremonies rather than one-off chatbots.

Core claim

The central claim is that a multi-agent system whose agents are backed by large language models can emulate the role structure of SAFe and, within simulation, measurably improve Agile project outcomes. The authors build this claim on the CogniSim platform, where each agent has an LLM core, a prompt-defined role, memory of past conversations, and selected tools, and where interactions are governed by a speaker-selection mechanism. Across six simulation runs that varied model type, iteration count, temperature, and included roles, the agents generated artifacts associated with real software work and showed higher diversity, completion, and sentiment-stability scores in some configurations. The

What carries the argument

The central object is CogniSim, the paper's layered multi-agent architecture. At the bottom is a large-language-model foundation; above it sits a multi-agent layer that can connect to project tools; and at the top are role-specific cognitive agents grouped into Manager, Executor, Quality Checker, and Methodology Reviewer categories, mapped onto SAFe roles such as Product Owner, System Architect, and DevOps Engineer. The carrying mechanism is structured natural-language dialogue with memory and speaker selection, which lets agents negotiate priorities, surface dependencies, and produce project artifacts as deliverables.

Load-bearing premise

The entire result rests on the assumption that the simulation environment and its custom metrics—unique content percentage, diversity score, completion score, and sentiment stability—are faithful proxies for success in real Agile software projects.

What would settle it

Run a controlled experiment in which human teams and CogniSim agents receive the same backlog and SAFe phase objectives, then compare task completion time, defect rate, and stakeholder satisfaction; if human teams match or beat the agents on those measures, the claimed reduction in task completion time fails. A cheaper check: replace the LLM core with a scripted random responder while keeping the role prompts; if the reported metric patterns survive, the metrics are not measuring the LLM's contribution.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Routine SAFe ceremonies—PI Planning, Iteration Execution, and Inspect-and-Adapt—could be at least partially delegated to LLM agents, with humans concentrating on judgment calls and stakeholder relationships.
  • Agent configurations such as model type, temperature, number of iterations, and role set become tuning knobs that project managers can set per phase, much like adjusting team composition.
  • Because the framework logs every dialogue and decision, it could yield an automatic audit trail showing why a backlog item was prioritized or an architecture choice was made.
  • The same role-based templates could be reused across projects, lowering the start-up cost of a new project once the agent definitions exist.
  • Dedicated quality and methodology-reviewer agents could monitor adherence to Agile practices continuously, catching drift earlier than periodic retrospectives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the simulation results transfer to real settings, a natural next test is to replace the paper's ad hoc metrics with standard engineering measures such as defect density, cycle time, and stakeholder satisfaction, and run the same backlog with human and agent teams.
  • The framework's dialogue logs could be turned into training data for smaller, cheaper models, letting teams run the same role structure without relying on large commercial models.
  • A promising extension the paper leaves implicit is using the agent team as an interactive rehearsal environment where human project managers practice SAFe ceremonies and stress-test scenarios before committing resources.
  • The mixed performance in Inspect-and-Adapt suggests that open-ended reflection tasks may need a different mechanism—such as external memory of past iterations or explicit retrospective templates—rather than the same planning dialogue.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper describes CogniSim, a multi-agent framework in which LLM-based cognitive agents emulate Agile/SAFe roles (Product Owner, System Architect, etc.) to automate project management tasks in simulated software projects. The authors present a layered architecture, a Python/LangChain implementation, a case study of PI Planning and Iteration Execution, and a table of simulation results. The abstract and Section 6.3 claim measurable improvements, including significantly reduced task completion times relative to human-managed processes, but the experimental evidence consists of six simulation runs whose metrics are undefined, and Section 9.1 explicitly defers rigorous quantification and human-baseline comparisons to future work.

Significance. The framework addresses a timely topic—using LLM agents in Agile/SAFe project management—and the modular design with reproducible configurations is a useful starting point. If the claimed performance gains were properly validated, the work could be relevant; however, the current evidence is limited to illustrative simulations with no control condition, so the significance is mainly as a framework description rather than an empirical demonstration.

major comments (4)
  1. [Table 5 / Section 7.2] The quantitative results are uninterpretable: the metrics 'Unique Content', 'Diversity Score', 'Completion Score', and 'Sentiment Stability' are never defined, no formulas or measurement procedures are given, and Section 7.3 describes a different set of metrics (Task Completion Time, Communication Efficiency, etc.) without linking them to Table 5. Readers cannot verify or reproduce these numbers.
  2. [Section 6.3 vs. Section 9.1] The central claim of significantly reduced task completion times compared to human-managed processes is unsupported: no human baseline or control group is presented anywhere, and Section 9.1 states that 'rigorous quantification of these gains or direct comparisons against human-led baselines are deferred to future research.' The abstract's 'measurable improvements' is thus contradicted by the authors' own stated limitations.
  3. [Section 7.1 / Table 5] The experimental design varies model type, iterations, temperature, and agent roles, but the six runs in Table 5 are not replicated and no statistical analysis is provided. Section 7.4's conclusion that 'adjusting independent variables allowed us to identify settings that optimize dependent variables' is not justified by the data. Moreover, no connection is made between the manipulated parameters and the reported metrics.
  4. [Section 1.6] The evaluation framework promises comparison against accepted best practices or human-generated baselines, but no such comparison is reported. Without a control condition, the claimed improvements cannot be attributed to the CogniSim framework rather than to the underlying LLMs or the simulation environment itself.
minor comments (5)
  1. [Section 5.3] 'Boxs' should be 'Boxes'.
  2. [Section 6.3/6.4] The terms 'astute agents' and 'perceptive agents' are used; use 'cognitive agents' consistently.
  3. [Figure 12] The figure lacks axis labels and units; the radar chart's additional metric 'context retention' is not defined in the text.
  4. [Section 7.3] Quality measures such as cyclomatic complexity and code coverage are listed but no such measurements are reported; clarify whether these were computed.
  5. [Data Availability] Consider adding a link to the code repository, if released, to support reproducibility.

Circularity Check

0 steps flagged

No circularity: the framework paper's empirical claims are under-supported and partly retracted, but no result is derived from its own inputs or from a load-bearing self-citation chain.

full rationale

This paper is a framework-and-simulation study, not a mathematical derivation. There is no fitted parameter that is later relabeled as a prediction, no uniqueness theorem imported from prior work by the same authors, and no ansatz smuggled in via citation. The earlier self-citations (e.g., refs. [17] and [52]) are used to describe the authors' own prior components and figures, but they do not carry the empirical conclusion; the claimed 'measurable improvements' rests, insecurely, on Table 5 and qualitative observations rather than on a self-citation chain. The most serious problems are evidentiary, not circular. Section 7.2 (Table 5) introduces 'Unique Content', 'Diversity Score', 'Completion Score', and 'Sentiment Stability' without defining formulas or validation, and the headline comparison in Section 6.3 ('task completion times being significantly reduced compared to human-managed processes') has no human-managed control in the paper. The authors themselves concede this: Section 1.6 says 'future work involves... incorporating benchmarking against human-driven baselines'; Section 6.4 calls for 'more extensive quantitative analyses and real-world field studies'; and Section 9.1 explicitly retracts: 'Rather than making definitive claims... we now present these as preliminary high-level opportunities' and 'rigorous quantification... or direct comparisons against human-led baselines are deferred to future research.' Those concessions undermine the abstract's claim of 'measurable improvements', but undermining an unsupported empirical claim is not the same as a circular derivation. Since no specific reduction of a claimed result to its own inputs or to an unverified self-citation can be quoted, the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 1 invented entities

The central claim rests on the validity of the simulation as a proxy for real projects and on the author-defined metrics. No free parameters are fitted in the traditional sense; the varied parameters (temperature, iterations) are experimental inputs, not fitted constants.

axioms (3)
  • domain assumption SAFe phase definitions and role responsibilities are as described in the paper's references.
    The simulation's validity rests on the assumed mapping between agent roles and SAFe roles (Section 3.3).
  • ad hoc to paper LLM-generated outputs (code snippets, chat) can be scored meaningfully by the paper's ad hoc metrics.
    The 'completion score' and 'diversity score' are defined by the authors, not validated against any external ground truth (Section 7.2, Table 5).
  • domain assumption The simulation environment faithfully emulates real-world software project management.
    No validation of the simulator against real project data is provided (Section 5).
invented entities (1)
  • CogniSim platform no independent evidence
    purpose: Multi-agent simulation platform for LLM-powered Agile roles
    The platform is introduced and evaluated only within this paper and the authors' prior self-cited work; no external users or benchmarks validate it.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Cognitive Agents Powered by Large Language Models for Agile Software Project Management." pith.science (2026). https://pith.science/paper/BUB7GQK2

@misc{pith2026250816678,
  author       = {Pith},
  title        = {Pith review of: Cognitive Agents Powered by Large Language Models for Agile Software Project Management},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BUB7GQK2}},
  note         = {Machine review of arXiv:2508.16678}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper investigates the integration of cognitive agents powered by Large Language Models (LLMs) within the Scaled Agile Framework (SAFe) to reinforce software project management. By deploying virtual agents in simulated software environments, this study explores their potential to fulfill fundamental roles in IT project development, thereby optimizing project outcomes through intelligent automation. Particular emphasis is placed on the adaptability of these agents to Agile methodologies and their transformative impact on decision-making, problem-solving, and collaboration dynamics. The research leverages the CogniSim ecosystem, a platform designed to simulate real-world software engineering challenges, such as aligning technical capabilities with business objectives, managing interdependencies, and maintaining project agility. Through iterative simulations, cognitive agents demonstrate advanced capabilities in task delegation, inter-agent communication, and project lifecycle management. By employing natural language processing to facilitate meaningful dialogues, these agents emulate human roles and improve the efficiency and precision of Agile practices. Key findings from this investigation highlight the ability of LLM-powered cognitive agents to deliver measurable improvements in various metrics, including task completion times, quality of deliverables, and communication coherence. These agents exhibit scalability and adaptability, ensuring their applicability across diverse and complex project environments. This study underscores the potential of integrating LLM-powered agents into Agile project management frameworks as a means of advancing software engineering practices. This integration not only refines the execution of project management tasks but also sets the stage for a paradigm shift in how teams collaborate and address emerging challenges.

Figures

Figures reproduced from arXiv: 2508.16678 by Ewa Niewiadomska-Szynkiewicz, Jaros{\l}aw A. Chudziak, Konrad Cinkusz.

Figure 1
Figure 1. Figure 1: Scrum framework with key artifacts, meetings, and processes [7]. As detailed in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Software engineering layers [5]. Multi-Agent Systems, comprising networks of autonomous agents that interact to achieve defined objectives within their environment, offer a resilient solution to these challenges [8]. These agents collaborate seamlessly, emulating human problem-solving processes and contributing to overall system efficiency [9] [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A generic agile iteration cycle illustrating planning, development, review, stakeholder feedback, and continuous improvement. For expanding organizations dealing with advanced systems and multiple teams, the Scaled Agile Framework provides a coherent approach to scaling Agile practices [25]. SAFe [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: offers a framework-neutral conceptual representation of how multiple Agile teams can plan, deliver, and integrate increments to support complex product development. This alignment facilitates improved coordination, synchronization, and collaboration between teams, enhancing the ability to handle advanced projects and product development [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Single cognitive agent and its components [17]. In the context of software engineering, LLMs enhance cognitive agents by enabling them to automate code generation [36], assist in documentation [37], facilitate communi￾cation [23], and support decision-making [24]. By automating code generation, cognitive agents can produce code snippets or entire functions based on natural language descrip￾tions, streamlin… view at source ↗
Figure 6
Figure 6. Figure 6: b illustrates the architecture of cognitive agents, highlighting the four inter￾dependent components, perception, reasoning, learning, and action, arranged in a cyclic process. At the center of this architecture, Large Language Models act as the unifying core, enhancing each layer’s functionality. The perception layer utilizes LLMs to improve natural language understanding, enabling agents to process and i… view at source ↗
Figure 7
Figure 7. Figure 7: Applications of Multi-Agent Systems in software engineering. The application of MASs in software engineering offers several benefits. Scalability is achieved because MASs can handle expanding systems by distributing tasks among agents, allowing the system to grow without a significant drop in performance [47]. Flexibility is provided by agents’ ability to adapt to changes in the environment or requirements… view at source ↗
Figure 8
Figure 8. Figure 8: Layered ecosystem of CogniSim. This modular and layered architecture ensures scalability, flexibility, and enhanced collaboration across human teams and AI-driven agents, optimizing software engineer￾ing workflows. 3.2. Agent Categorization Within the CogniSim framework, agents are categorized based on their roles and responsibilities to optimize collaboration and task execution. The main categories includ… view at source ↗
Figure 9
Figure 9. Figure 9: Integration of CogniSim with SAFe framework [52]. By integrating with SAFe, CogniSim enhances coordination across multiple teams, improves communication efficiency, and ensures that development efforts are aligned with organizational goals. The cognitive agents automate routine coordination tasks, allowing human team members to focus on strategic decision-making and innovation. Cognitive agents dynamically… view at source ↗
Figure 10
Figure 10. Figure 10: Project structure. The simulations and simulators directories form the backbone of the experimental environment, enabling the simulation of various scenarios and tasks. These components replicate real-world dynamics and support iterative testing, allowing agents to participate in software project management tasks by adhering to Agile methodologies. The helpers and utils directories provide utility functio… view at source ↗
Figure 11
Figure 11. Figure 11: Simulation workflow in CogniSim, showing the iterative Agile process from setup through data analysis. 6.3. Results and Analysis The simulation results provide compelling evidence of the CogniSim framework’s effectiveness in enhancing Agile software development processes. One of the most signif￾icant findings is the enhancement in decision-making capabilities of the agents. Astute agents demonstrated the … view at source ↗
Figure 12
Figure 12. Figure 12: Simulation results. These visualizations offer an intuitive understanding of the simulation outcomes, enabling clear identification of strengths and areas for improvement. For instance, the unique content bar chart highlights simulations with high redundancy, while the radar chart underscores the overall balance among key performance indicators. 7.3. Quality Measures and Performance Metrics To evaluate th… view at source ↗
Figure 13
Figure 13. Figure 13: Conceptual enterprise-scale Agile layers with cognitive agents and MASs (inspired by principles in frameworks such as SAFe 6.0 [25,69]). By leveraging cognitive agents and Multi-Agent Systems at these higher conceptual layers, organizations can enhance decision-making, streamline operations, and foster continuous alignment of business strategies. For example, cognitive agents can automate data analysis to… view at source ↗
Figure 14
Figure 14. Figure 14: MAS concept diagram. 9.2. Implications for Practice The integration of these technologies presents significant implications for Agile soft￾ware project management. Organizations that adopt MASs and LLMs can realize strategic benefits, including enhanced decision-making capabilities, improved risk management, and better project insights. Automating routine tasks enables human team members to focus on strat… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 48 canonical work pages · 4 internal anchors

  1. [1]

    Agile Software Development Methods: Review and Analysis

    Abrahamsson, P .; Salo, O.; Ronkainen, J.; Warsta, J. Agile Software Development Methods: Review and Analysis.arXiv 2017, arXiv:1709.08439

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  2. [2]

    Intelligent software engineering in the context of agile software development: A systematic literature review

    Perkusich, M.; Chaves E Silva, L.; Costa, A.; Ramos, F.; Saraiva, R.; Freire, A.; Dilorenzo, E.; Dantas, E.; Santos, D.; Gorgônio, K.; et al. Intelligent software engineering in the context of agile software development: A systematic literature review. Inf. Softw. Technol. 2020, 119, 106241. [CrossRef]

    work page 2020

  3. [3]

    A decade of agile methodologies: Towards explaining agile software development

    Dingsøyr, T.; Nerur, S.; Balijepally, V .; Moe, N.B. A decade of agile methodologies: Towards explaining agile software development. J. Syst. Softw. 2012, 85, 1213–1221. [CrossRef]

    work page 2012

  4. [4]

    The role of the project manager in agile software development projects

    Shastri, Y.; Hoda, R.; Amor, R. The role of the project manager in agile software development projects. J. Syst. Softw. 2021, 173, 110871. [CrossRef]

    work page 2021

  5. [5]

    Software Engineering: A Practitioner’s Approach, 9th ed.; McGraw-Hill Education: New York, NY, USA, 2020

    Pressman, R.S.; Maxim, B.R. Software Engineering: A Practitioner’s Approach, 9th ed.; McGraw-Hill Education: New York, NY, USA, 2020

    work page 2020

  6. [6]

    Rubin, K.S.; Cohn, M.; Jeffries, R. Essential Scrum: A Practical Guide to the Most Popular Agile Process; The Addison-Wesley Signature Series; Addison-Wesley: Upper Saddle River, NJ, USA; Boston, MA, USA; Indianapolis, Indiana; San Francisco, CA, USA; New York, NY, USA; Toronto, ON, Canada; Montreal, QC, Canada; London, UK; Munich, Germany; Paris, France;...

    work page 2013

  7. [7]

    Scrum Framework

    Scrum.org. Scrum Framework. Scrum.org. 2020. Available online: https://www.scrum.org (accessed on 24 December 2024)

    work page 2020

  8. [8]

    Transforming Competition into Collaboration: The Revolutionary Role of Multi-Agent Systems and Language Models in Modern Organizations

    Cruz, C.J.X. Transforming Competition into Collaboration: The Revolutionary Role of Multi-Agent Systems and Language Models in Modern Organizations. arXiv 2024, arXiv:2403.07769. [CrossRef]

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  9. [9]

    Engineering Multi-agent Systems with Statecharts: Theory and Practice

    Spanoudakis, N.I. Engineering Multi-agent Systems with Statecharts: Theory and Practice. SN Comput. Sci. 2021, 2, 317. [CrossRef]

    work page 2021

  10. [10]

    Evaluating Large Language Models: A Comprehensive Survey

    Guo, Z.; Jin, R.; Liu, C.; Huang, Y.; Shi, D.; Supryadi.; Yu, L.; Liu, Y.; Li, J.; Xiong, B.; et al. Evaluating Large Language Models: A Comprehensive Survey. arXiv 2023, arXiv:2310.19736

    Pith/arXiv arXiv 2023

  11. [11]

    Exploring Autonomous Agents through the Lens of Large Language Models: A Review

    Barua, S. Exploring Autonomous Agents through the Lens of Large Language Models: A Review. arXiv 2024, arXiv:2404.04442. [CrossRef]

    Pith/arXiv arXiv 2024

  12. [12]

    A Comparative Analysis of Large Language Models for Code Documentation Generation

    Dvivedi, S.S.; Vijay, V .; Pujari, S.L.R.; Lodh, S.; Kumar, D. A Comparative Analysis of Large Language Models for Code Documentation Generation. arXiv 2024, arXiv:2312.10349. [CrossRef]

    Pith/arXiv arXiv 2024

  13. [13]

    DebugBench: Evaluating Debugging Capability of Large Language Models

    Tian, R.; Ye, Y.; Qin, Y.; Cong, X.; Lin, Y.; Pan, Y.; Wu, Y.; Hui, H.; Liu, W.; Liu, Z.; et al. DebugBench: Evaluating Debugging Capability of Large Language Models. arXiv 2024, arXiv:2401.04621. [CrossRef]

    Pith/arXiv arXiv 2024

  14. [14]

    Yuan, S.T.; Yokoo, M.; Goos, G.; Hartmanis, J.; Van Leeuwen, J.; Carbonell, J.G.; Siekmann, J. (Eds.) Intelligent Agents: Specification, Modeling, and Applications: 4th Pacific Rim International Workshop on Multi-Agents, PRIMA 2001 Taipei, Taiwan, 28–29 July 2001 Proceedings; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2001; V...

    work page 2001

  15. [15]

    Design process documentation template

    IEEE Foundation for Intelligent Physical Agents (FIPA). Design process documentation template. In Manual SC00097B, IEEE FIP A DPDF Working Group, IEEE FIP A; Status: Standard tex.changelog: (Initial); Cossentino, M., Molesini, A., Omicini, A., Hilaire, V ., Fuentes, R., DeLoach, S., Migeon, F., Bonjean, N., Gleizes, M.P ., Maurel, C., et al., Eds.; FIPA: ...

    work page 2012

  16. [16]

    Cognitive Architectures for Language Agents

    Sumers, T.; Yao, S.; Narasimhan, K.; Griffiths, T. Cognitive Architectures for Language Agents. Transactions on Machine Learning Research. In Review. 2024. Available online: https://openreview.net/forum?id=1i6ZCvflQJ (accessed on 24 December 2024)

    work page 2024

  17. [17]

    Towards LLM-augmented multiagent systems for agile software engineering

    Cinkusz, K.; Chudziak, J.A. Towards LLM-augmented multiagent systems for agile software engineering. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, Sacramento, CA, USA, 27 October–1 November 2024; pp. 2476–2477. [CrossRef]

    work page 2024

  18. [18]

    LLM-Based Multi-Agent Systems for Software Engineering: Vision and the Road Ahead

    He, J.; Treude, C.; Lo, D. LLM-Based Multi-Agent Systems for Software Engineering: Vision and the Road Ahead. arXiv 2024, arXiv:2404.04834. [CrossRef]

    Pith/arXiv arXiv 2024

  19. [19]

    AGILE: A Novel Reinforcement Learning Framework of LLM Agents

    Feng, P .; He, Y.; Huang, G.; Lin, Y.; Zhang, H.; Zhang, Y.; Li, H. AGILE: A Novel Reinforcement Learning Framework of LLM Agents. arXiv 2024, arXiv:2405.14751. [CrossRef]

    Pith/arXiv arXiv 2024

  20. [20]

    From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future

    Jin, H.; Huang, L.; Cai, H.; Yan, J.; Li, B.; Chen, H. From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future. arXiv 2024, arXiv:2408.02479. [CrossRef]

    Pith/arXiv arXiv 2024

  21. [21]

    CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology

    Rasheed, Z.; Sami, M.A.; Kemell, K.K.; Waseem, M.; Saari, M.; Systä, K.; Abrahamsson, P . CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology. arXiv 2024, arXiv:2402.01411. [CrossRef]

    Pith/arXiv arXiv 2024

  22. [22]

    Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents.arXiv 2023, arXiv:2306.03314

    Talebirad, Y.; Nadiri, A. Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents.arXiv 2023, arXiv:2306.03314. [CrossRef] Electronics 2025, 14, 87 32 of 33

    Pith/arXiv arXiv 2023

  23. [23]

    Theory of Mind for Multi-Agent Collaboration via Large Language Models

    Li, H.; Chong, Y.Q.; Stepputtis, S.; Campbell, J.; Hughes, D.; Lewis, M.; Sycara, K. Theory of Mind for Multi-Agent Collaboration via Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 6–10 December 2023; pp. 180–192. [CrossRef]

    work page 2023

  24. [24]

    Publisher Correction: Large language models encode clinical knowledge

    Singhal, K.; Azizi, S.; Tu, T.; Mahdavi, S.S.; Wei, J.; Chung, H.W.; Scales, N.; Tanwani, A.; Cole-Lewis, H.; Pfohl, S.; et al. Publisher Correction: Large language models encode clinical knowledge. Nature 2023, 620, E19. [CrossRef]

    work page 2023

  25. [25]

    SAFe 6.0 Framework; Scaled Agile, Inc.: Boulder, CO, USA, 2024

    Scaled Agile, Inc. SAFe 6.0 Framework; Scaled Agile, Inc.: Boulder, CO, USA, 2024

    work page 2024

  26. [26]

    Financial Statement Analysis with Large Language Models.arXiv 2024, arXiv:2407.17866

    Kim, A.G.; Muhn, M.; Nikolaev, V .V . Financial Statement Analysis with Large Language Models.arXiv 2024, arXiv:2407.17866. [CrossRef]

    Pith/arXiv arXiv 2024

  27. [27]

    Chiang, C.H.; Lee, H.y. Can Large Language Models Be an Alternative to Human Evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, ON, Canada, 9–14 July 2023; pp. 15607–15631. [CrossRef]

    work page 2023

  28. [28]

    Guidelines for Future Agile Methodologies and Architecture Reconciliation for Software- Intensive Systems

    Rocha, F.G.; Misra, S.; Soares, M.S. Guidelines for Future Agile Methodologies and Architecture Reconciliation for Software- Intensive Systems. Electronics 2023, 12, 1582. [CrossRef]

    work page 2023

  29. [29]

    Agile Project Management: Creating Innovative Products , 2nd ed.; The Agile Software Development Series; Addison-Wesley: Upper Saddle River, NJ, USA, 2010

    Highsmith, J.A. Agile Project Management: Creating Innovative Products , 2nd ed.; The Agile Software Development Series; Addison-Wesley: Upper Saddle River, NJ, USA, 2010

    work page 2010

  30. [30]

    The Scrum Guide: The Definitive Guide to Scrum: The Rules of the Game

    Schwaber, K.; Sutherland, J. The Scrum Guide: The Definitive Guide to Scrum: The Rules of the Game. Available online: https://scrumguides.org/scrum-guide.html (accessed on 24 December 2024)

    work page 2024

  31. [31]

    SAFe Distilled: SAFe 5.0: Achieving Business Agility with the Scaled Agile Framework; Addison-Wesley: Hoboken, NJ, USA, 2020

    Knaster, R.; Leffingwell, D. SAFe Distilled: SAFe 5.0: Achieving Business Agility with the Scaled Agile Framework; Addison-Wesley: Hoboken, NJ, USA, 2020

    work page 2020

  32. [32]

    Scaling agile

    Ebert, C.; Paasivaara, M. Scaling agile. IEEE Softw. 2017, 34, 98–103. [CrossRef]

    work page 2017

  33. [33]

    Artificial Intelligence: A Modern Approach , 4th ed.; Pearson Series in Artificial Intelligence; Pearson: Hoboken, NJ, USA, 2021

    Russell, S.J.; Norvig, P . Artificial Intelligence: A Modern Approach , 4th ed.; Pearson Series in Artificial Intelligence; Pearson: Hoboken, NJ, USA, 2021

    work page 2021

  34. [34]

    GPT-4 Technical Report

    OpenAI.; Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F.L.; Almeida, D.; Altenschmidt, J.; Altman, S.; et al. GPT-4 Technical Report. arXiv 2024, arXiv:2303.08774. [CrossRef]

    Pith/arXiv arXiv 2024

  35. [35]

    Language Models are Few-Shot Learners

    Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P .; Neelakantan, A.; Shyam, P .; Sastry, G.; Askell, A.; et al. Language Models are Few-Shot Learners. arXiv 2020, arXiv:2005.14165

    Pith/arXiv arXiv 2020

  36. [36]

    Evaluating Large Language Models Trained on Code

    Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; Pinto, H.P .d.O.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. Evaluating Large Language Models Trained on Code. arXiv 2021, arXiv:2107.03374

    Pith/arXiv arXiv 2021

  37. [37]

    IntelliCode compose: Code generation using transformer

    Svyatkovskiy, A.; Deng, S.K.; Fu, S.; Sundaresan, N. IntelliCode compose: Code generation using transformer. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Virtual, 17–18 November 2020; pp. 1433–1443. [CrossRef]

    work page 2020

  38. [38]

    LLM-Powered Autonomous Agents; Online Resource, 2023

    Weng, L. LLM-Powered Autonomous Agents; Online Resource, 2023. Available online: https://lilianweng.github.io/posts/2023 -06-23-agent/ (accessed on 24 December 2024)

    work page 2023

  39. [39]

    MCQGen: A large language model-driven MCQ generator for personalized learning.IEEE Access Pract

    Hang, C.N.; Wei Tan, C.; Yu, P .D. MCQGen: A large language model-driven MCQ generator for personalized learning.IEEE Access Pract. Innov. Open Solut. 2024, 12, 102261–102273. [CrossRef]

    work page 2024

  40. [40]

    An Introduction to Multiagent Systems, 2nd ed.; Wiley: Chichester, UK, 2012

    Wooldridge, M.J. An Introduction to Multiagent Systems, 2nd ed.; Wiley: Chichester, UK, 2012

    work page 2012

  41. [41]

    Synergizing logical reasoning, long-term memory, and collaborative intelligence in multi-agent LLM systems

    Kostka, A.; Chudziak, J.A. Synergizing logical reasoning, long-term memory, and collaborative intelligence in multi-agent LLM systems. In Proceedings of the Pacific Asia Conference on Language, Information and Computation (PACLIC 38), Tokyo, Japan, 7–9 December 2024

    work page 2024

  42. [42]

    ElliottAgents: A natural language-driven multi-agent system for stock market analysis and prediction

    Chudziak, J.A.; Wawer, M. ElliottAgents: A natural language-driven multi-agent system for stock market analysis and prediction. In Proceedings of the 38th Pacific Asia Conference on Language, Information and Computation, Tokyo, Japan, 7–9 December

    work page

  43. [43]

    Trends in cooperative distributed problem solving.IEEE Trans

    Durfee, E.; Lesser, V .; Corkill, D. Trends in cooperative distributed problem solving.IEEE Trans. Knowl. Data Eng. 1989, 1, 63–83. [CrossRef]

    work page 1989

  44. [44]

    Tutorial on agent-based modelling and simulation

    Macal, C.M.; North, M.J. Tutorial on agent-based modelling and simulation. J. Simul. 2010, 4, 151–162. [CrossRef]

    work page 2010

  45. [45]

    Efficient resource management for Cloud computing environments

    Younge, A.J.; von Laszewski, G.; Wang, L.; Lopez-Alarcon, S.; Carithers, W. Efficient resource management for Cloud computing environments. In Proceedings of the International Conference on Green Computing, Chicago, IL, USA, 15–18 August 2010; pp. 357–364. [CrossRef]

    work page 2010

  46. [46]

    AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology

    Nguyen, M.H.; Chau, T.P .; Nguyen, P .X.; Bui, N.D.Q. AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology. arXiv 2024, arXiv:2406.11912

    Pith/arXiv arXiv 2024

  47. [47]

    On agent-based software engineering

    Jennings, N.R. On agent-based software engineering. Artif. Intell. 2000, 117, 277–296. [CrossRef]

    work page 2000

  48. [48]

    (Ed.) Multiagent Systems, 2nd ed.; Intelligent Robotics and Autonomous Agents; The MIT Press: Cambridge, MA, USA; London, UK, 2013

    Weiss, G. (Ed.) Multiagent Systems, 2nd ed.; Intelligent Robotics and Autonomous Agents; The MIT Press: Cambridge, MA, USA; London, UK, 2013

    work page 2013

  49. [49]

    Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence, 1st ed.; Addison-Wesley: Boston, MA, USA, 1999

    Ferber, J.; Ferber, J. Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence, 1st ed.; Addison-Wesley: Boston, MA, USA, 1999. Electronics 2025, 14, 87 33 of 33

    work page 1999

  50. [50]

    ChatDev: Communicative Agents for Software Development

    Qian, C.; Liu, W.; Liu, H.; Chen, N.; Dang, Y.; Li, J.; Yang, C.; Chen, W.; Su, Y.; Cong, X.; et al. ChatDev: Communicative Agents for Software Development. arXiv 2024, arXiv:2307.07924

    Pith/arXiv arXiv 2024

  51. [51]

    Software Metrics: A Rigorous and Practical Approach, 3rd ed.; Innovations in Software Engineering and Software Development; CRC Press: Boca Raton, FL, USA, 2015

    Fenton, N.E.; Bieman, J. Software Metrics: A Rigorous and Practical Approach, 3rd ed.; Innovations in Software Engineering and Software Development; CRC Press: Boca Raton, FL, USA, 2015. [CrossRef]

    work page 2015

  52. [52]

    Communicative agents for software project management and system development

    Cinkusz, K.; Chudziak, J. Communicative agents for software project management and system development. In Proceedings of the 21th International Conference on Modeling Decisions for Artificial Intelligence MDAI 2024, Tokyo, Japan, 27–31 August 2024; Torra, V ., Narukawa, Y., Kikuchi, H., Eds.; ISBN 978-91-531-0238-0

    work page 2024

  53. [53]

    TaskWeaver: A Code-First Agent Framework

    Qiao, B.; Li, L.; Zhang, X.; He, S.; Kang, Y.; Zhang, C.; Yang, F.; Dong, H.; Zhang, J.; Wang, L.; et al. TaskWeaver: A Code-First Agent Framework. arXiv 2024, arXiv:2311.17541

    Pith/arXiv arXiv 2024

  54. [54]

    Reflexion: Language Agents with Verbal Reinforcement Learning

    Shinn, N.; Cassano, F.; Berman, E.; Gopinath, A.; Narasimhan, K.; Yao, S. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv 2023, arXiv:2303.11366

    Pith/arXiv arXiv 2023

  55. [55]

    Version 0.2

    LangChain. Version 0.2. Introduction to LangChain. Online Resource. 2023. Available online: https://python.langchain.com/v0 .2/docs/introduction/ (accessed on 24 December 2024)

    work page 2023

  56. [56]

    Version 0.2

    LangChain. Version 0.2. LangChain Core API Reference. Online Resource. 2023. Available online: https://python.langchain. com/v0.2/api_reference/core/index.html (accessed on 24 December 2024)

    work page 2023

  57. [57]

    Is Temperature the Creativity Parameter of Large Language Models? arXiv 2024, arXiv:2405.00492

    Peeperkorn, M.; Kouwenhoven, T.; Brown, D.; Jordanous, A. Is Temperature the Creativity Parameter of Large Language Models? arXiv 2024, arXiv:2405.00492

    Pith/arXiv arXiv 2024

  58. [58]

    DataPerf: Benchmarks for Data-Centric AI Development

    Mazumder, M.; Banbury, C.; Yao, X.; Karlaš, B.; Rojas, W.G.; Diamos, S.; Diamos, G.; He, L.; Parrish, A.; Kirk, H.R.; et al. DataPerf: Benchmarks for Data-Centric AI Development. arXiv 2023, arXiv:2207.10062

    Pith/arXiv arXiv 2023

  59. [59]

    Communication management and its impact on successful IT program

    Chudziak, J.; Cegielski, R.W.; Meyer, J. Communication management and its impact on successful IT program. IADIS Int. J. Comput. Sci. Inf. Syst. 2008, 1, 14–28

    work page 2008

  60. [60]

    A survey of multi-agent organizational paradigms

    Horling, B.; Lesser, V . A survey of multi-agent organizational paradigms. Knowl. Eng. Rev. 2004, 19, 281–316. [CrossRef]

    work page 2004

  61. [61]

    Ontological Learning Assistant for Knowledge Discovery and Data Mining

    Choinski, M.; Chudziak, J.A. Ontological Learning Assistant for Knowledge Discovery and Data Mining. In Proceedings of the 2009 International Multiconference on Computer Science and Information Technology, Mragowo, Poland, 12–14 October 2009; pp. 147–155. [CrossRef]

    work page 2009

  62. [62]

    AI for Agile development: a Meta-Analysis

    Cabrero-Daniel, B. AI for Agile development: A Meta-Analysis. arXiv 2023, arXiv:2305.08093

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  63. [63]

    Large Language Model based Multi-Agents: A Survey of Progress and Challenges

    Guo, T.; Chen, X.; Wang, Y.; Chang, R.; Pei, S.; Chawla, N.V .; Wiest, O.; Zhang, X. Large Language Model based Multi-Agents: A Survey of Progress and Challenges. arXiv 2024, arXiv:2402.01680

    Pith/arXiv arXiv 2024

  64. [64]

    Trust from Ethical Point of View: Exploring Dynamics Through Multiagent-Driven Cognitive Modeling

    Tariverdi, A. Trust from Ethical Point of View: Exploring Dynamics Through Multiagent-Driven Cognitive Modeling. arXiv 2024, arXiv:2401.07255

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  65. [65]

    SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents

    Lin, F.; Kim, D.J.; Chen, T.-H. SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents. arXiv 2024, arXiv:2403.15852

    Pith/arXiv arXiv 2024

  66. [66]

    Multi-Agent Systems: A Survey

    Dorri, A.; Kanhere, S.S.; Jurdak, R. Multi-Agent Systems: A Survey. IEEE Access 2018, 6, 28573–28593. [CrossRef]

    work page 2018

  67. [67]

    Consensus in multi-agent systems: A review

    Amirkhani, A.; Barshooi, A.H. Consensus in multi-agent systems: A review. Artif. Intell. Rev. 2022, 55, 3897–3935. [CrossRef]

    work page 2022

  68. [68]

    Cognitive bias in decision-making with LLMs

    Echterhoff, J.M.; Liu, Y.; Alessa, A.; McAuley, J.; He, Z. Cognitive bias in decision-making with LLMs. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, FL, USA, 12–16 November 2024; Al-Onaizan, Y., Bansal, M., Chen, Y.N., Eds.; Association for Computational Linguistics: Stroudsburg, PA, USA, 2024; pp. 12...

    work page 2024

  69. [69]

    SAFe Scrum

    Scaled Agile, Inc. SAFe Scrum. Online Resource. 2024. Available online: https://scaledagileframework.com/safe-scrum/ (accessed on 24 December 2024). Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or t...

    work page 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.