Knowledge-Guided Multi-Agent Framework for Automated Requirements Development: A Vision

Dongming Jin; Jiangping Huang; Weisong Sun; Yang Liu; Zhi Jin

REVIEW 4 major objections 5 minor 23 references

Knowledge-Guided Multi-Agent Framework for Automated Requirements Development: A Vision

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Six LLM agents turn a brief idea into a full requirements spec.

desk verdict Clear vision for multi-agent requirements engineering; the case study is anecdotal and doesn't yet substantiate the knowledge-guided, adaptive core. read the letter →

arxiv 2506.22656 v1 pith:4TYXFDDV submitted 2025-06-27 cs.SE cs.AI

Jiangping Huang , Dongming Jin , Weisong Sun , Yang Liu , Zhi Jin This is my paper

classification cs.SEcs.AI

keywords automatedrequirementsdevelopmentengineeringlargelanguagemodelsmulti-agentsystemsknowledgeguidancesoftwarespecificationartifactpool

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KGMAF is a proposed knowledge-guided multi-agent framework whose stated goal is to automate the entire requirements engineering phase, turning a one-line system idea into a finished software requirements specification. The paper argues that existing multi-agent software development systems concentrate on code and leave requirements activities under-served, and that the missing ingredient is a role-specific design with explicit knowledge injection. The framework assigns six LLM-based agents — Interviewer, End-User, Analyst, Deployer, Archivist, and Reviewer — each with predefined actions, a planning mechanism, and injected knowledge, coordinated through an artifact pool inspired by blackboard systems. A case study on an insurance management system generates six artifacts, including a 12-page SRS and four use-case diagrams, which the authors present as preliminary evidence of practicality and use to sketch a research agenda.

What carries the argument

The load-bearing mechanism is the pairing of (1) six agents that each combine a functionality, a set of predefined actions, a planning mechanism, and injected knowledge (domain knowledge, templates, and standards specific to their role), with (2) an artifact pool modeled on the blackboard mechanism, whose state triggers agents to plan and act as artifacts are added or updated. This design turns the requirements workflow from a fixed chat into a state-driven pipeline: the Interviewer produces analysis and interview records, the Deployer drafts the environment list, the Analyst classifies requirements and builds the model, the Archivist writes and revises the SRS, and the Reviewer consolidates feedback. The knowledge injection is what the paper claims separates KGMAF from prior multi-agent systems that simply assign roles to LLMs without defining the software-engineering knowledge they need.

What would settle it

Run KGMAF end-to-end on a diverse set of at least 20 software ideas spanning different domains; if the pipeline frequently stalls, produces internally inconsistent SRSs, or fails a requirements-quality checklist (completeness, consistency, testability) at rates far below human-written specifications, the claim that KGMAF autonomously performs RE would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that a knowledge-guided multi-agent framework, KGMAF, can autonomously perform the four requirements-engineering activities — elicitation, analysis, specification, and validation — by orchestrating six specialized LLM-based agents through a shared artifact pool. Given a brief description of the system to be developed, the agents take turns producing intermediate artifacts (product analysis report, interview records, user requirements list, operating environment list, system requirements list, requirements model) and the final SRS; the artifact pool's state drives which agent acts next. The authors position the framework as filling a gap left by code-centric multi-agent systems, and their case study on an insurance management system produces artifacts whose quality they describe as satisfactory, though the evaluation is qualitative and limited to a single project.

Load-bearing premise

The framework assumes that six LLM-based agents, given predefined actions, planning mechanisms, and injected knowledge, can reliably execute and collaborate on requirements tasks across diverse projects, but this is only demonstrated on a single insurance-management case with subjective quality judgment and no quantitative evaluation.

Editorial extensions

If this is right

If the framework works as claimed, a requirements engineer's job shifts from manually drafting an SRS to reviewing and steering an automated pipeline that produces structured, traceable artifacts from a brief idea.
The artifact-pool design offers a general pattern for multi-agent software engineering collaboration: shared state, rather than direct conversation, determines which agent acts next, potentially reducing coordination overhead in larger pipelines.
The explicit role-action-knowledge decomposition provides a template for retrofitting existing LLM-based development agents with the requirements capabilities they currently lack.
Automated validation and revision loops between the Reviewer and Archivist could make requirement changes propagate quickly through the SRS, supporting iterative development in practice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's single insurance-management case, judged only by subjective inspection, leaves the core claim of reliability across projects untested; a multi-domain benchmark with quantitative consistency checks would be the natural next evaluation.
Because the end-user agent is itself an LLM simulation of users, requirements quality may be bounded by the model's ability to imitate real stakeholders; grounding that agent in actual user personas, interview transcripts, or usage data could make the pipeline more credible.
The framework's design does not specify how agent disagreements (for example, a user request that conflicts with deployment constraints) are arbitrated; an explicit conflict-resolution or negotiation mechanism would be needed before trusting the pipeline with safety-critical requirements.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

This is a short vision paper proposing KGMAF, a six-agent LLM framework for requirements engineering, with an artifact pool and injected knowledge. The design is clearly laid out and addresses a real gap: most multi-agent SE frameworks (ChatDev, MetaGPT) are code-focused and treat RE superficially.

What's genuinely useful here is the explicit decomposition of RE into agent roles (interviewer, end-user, analyst, archivist, reviewer, deployer) with named actions and knowledge types, plus the artifact pool as a shared blackboard. The paper is honest that this is a vision: it says so in the abstract and conclusion, and the case study is labeled preliminary. That framing matters.

The soft spots are the ones you'd expect. The single insurance management case study is anecdotal: no baseline, no quantitative evaluation, and quality is asserted, not measured. More importantly, the stress-test note is right that the two distinctive mechanisms are not actually demonstrated. The 'ConductResearch' action claims to collect URLs and summarize webpages, but the Product Analysis Report in Figure 2 could plausibly come from the LLM's parametric memory—no retrieval logs, no external citations. The planning mechanism is described as 'think and select the next action,' but the collaboration narrative in Section 3 reads as a fixed pipeline: PAR, then interview, then URL/OEL/SRL/RM, then SRS, then review. No evidence of adaptive planning, alternative choices, or recovery. The observed artifacts are fully consistent with a hard-coded prompt chain.

That said, for a vision paper this is not fatal. The authors don't wildly overclaim the evidence, though 'validate the practicality' is a bit strong for what's shown. My main concern is that they don't explicitly state the limits of the case study. A short paragraph saying 'we do not yet demonstrate that knowledge injection or adaptive planning caused these outputs' would have made this a better paper.

The citation pattern is fine; the self-citation to their Mare framework is appropriate. No math or fitted parameters here, so the circularity burden is low.

Who is this for? People working on LLM-based requirements automation, and anyone designing multi-agent systems with a blackboard architecture. It's a reasonable design proposal worth reading, but not a source of empirical facts.

I'd send it to peer review for a vision track or workshop. The design is coherent and the gap it addresses is real. I'd ask the authors to temper the 'autonomously performs' phrasing and add an explicit limitations subsection. But desk-rejecting it would be wrong—this is exactly the kind of position paper a community benefits from seeing.

Best

Referee Report

4 major / 5 minor

Summary. This paper proposes KGMAF, a multi-agent framework for requirements development, consisting of six LLM-based agents (Interviewer, End-User, Deployer, Analyst, Archivist, Reviewer) and an artifact pool. Each agent is assigned a functionality, predefined actions, a planning mechanism, and requirements-related knowledge, and the framework is intended to take a brief system description and autonomously perform elicitation, analysis, specification, and validation to produce a Software Requirements Specification and intermediate artifacts. The paper illustrates the framework with a case study on an insurance management system using GPT-4-turbo, reports the generated artifacts, and outlines a future-work roadmap.

Significance. KGMAF targets a genuine gap: existing multi-agent software development frameworks such as ChatDev and MetaGPT concentrate on code development and do not provide dedicated agent designs for requirements engineering. The conceptual decomposition into specialist agents with explicit knowledge and artifacts is clearly presented, and the included artifact excerpts give a concrete picture of what the framework aims to produce. The authors also report their model settings and provide a public link to the full artifacts, which is helpful for reproducibility of the demonstration. If the central mechanisms, especially knowledge injection and artifact-state-driven planning, are later substantiated, the framework would be a useful step toward automated requirements development. However, the paper is explicitly a vision paper, and the evidence currently remains anecdotal and does not validate the distinctive mechanisms that the framework claims.

major comments (4)

[Section 3 (Collaboration Process) vs. Section 2.3] The case narrative says the Interviewer selects WriteProductAnalysisReport and PrepareInterviewList, but Section 2.3 lists the Interviewer's actions as ConductResearch, StartInterview, and WriteReqList. This internal mismatch makes the precise workflow irreproducible and indicates that the action design is not yet stable enough to support the paper's claims.
[Section 2.3, Interviewer ConductResearch] The description of ConductResearch states that the agent collects URLs, summarizes webpages, and writes the Product Analysis Report, yet no URLs, external references, or retrieval traces appear in the case study artifacts shown in Figure 2 or in the collaboration narrative. The PAR excerpts, such as Product Background and Comparative Analysis, could plausibly have been generated from the LLM's parametric memory alone. Consequently, the evidence does not demonstrate that external knowledge is actually retrieved and injected, which weakens the central 'knowledge-guided' claim.
[Section 3, Collaboration Process] The reported collaboration process follows a fixed linear sequence: PAR, interview questions, dialogue, User Requirements List, Operating Environment List, System Requirements List, requirements model, SRS, and review. No instance is shown in which the planning mechanism selects an alternative action, reacts to an unexpected artifact state, or recovers from a failure. The claimed adaptive, artifact-state-driven planning is therefore not demonstrated, and the statement that the case study validates KGMAF overstates what a single successful linear trace can show.
[Section 3, Generated Artifacts] The quality of the generated artifacts is assessed only subjectively, with the statement that 'the quality ... is satisfactory' based on visual inspection. There are no evaluation criteria, baseline comparison, or independent assessment. This is acceptable for illustrating potential in a vision paper, but it does not support the stronger validation phrasing used in the introduction and Section 3.

minor comments (5)

[Section 1] The phrase 'blackboard mechansim' should be corrected to 'blackboard mechanism'.
[Figure 1 and Section 2.3] Several action names in Figure 1 and in the case narrative differ from those in Section 2.3, such as WriteProductAnalysisReport and PrepareInterviewList versus ConductResearch and StartInterview. The figure, the conceptual design, and the case study should use one consistent set of action names.
[Section 3, Collaboration Process] The collaboration narrative mentions an 'interview question list' artifact, but this artifact is not described in Section 2.3 and does not appear in the artifact list in Figure 2. Please clarify whether it is a distinct artifact and how it relates to the Interview Records.
[Section 2.2] The sentence 'The performance of both modules requires an injection of requirements-related knowledge' is awkward and should be rephrased, for example as 'both modules require the injection of requirements-related knowledge'.
[Section 1 and Reference [4]] The relationship to the authors' earlier Mare framework [4], which is also a multi-agent framework for requirements engineering, is not discussed. The introduction should state explicitly what KGMAF adds beyond Mare so that the novelty claim is clear.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: KGMAF is a design vision, and the only same-author citation (Mare [4]) is not load-bearing.

full rationale

KGMAF is a vision paper that proposes an architecture rather than deriving a result; it contains no equations, fitted parameters, uniqueness theorems, or predicted quantities that could reduce to its inputs by construction. The central claim is the design itself, and the Section 3 case study is an illustrative demonstration, not a statistical or mathematical prediction. The only same-author citation identified is reference [4] (Mare), used in Section 1 to support the general statement that requirements engineering is a pivotal phase in the software development life-cycle; this is a standard background claim and does not carry the weight of KGMAF's agent design, so it is not load-bearing. The paper's assertions that knowledge injection and planning mechanisms are effective are not validated by controlled comparison, but that is an evaluation gap rather than circularity. Accordingly, no specific circular step is identified, and the score reflects only the presence of a minor, non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on assumptions about LLM agent capabilities and knowledge injection, with no free parameters used for fitting; the experimental settings (temperature, top_p, max tokens) are reportable but not fitted.

free parameters (4)

temperature = 0.3
Set for the GPT-4-turbo case study to control randomness; affects output artifacts, not a fitted model parameter.
top_p = 1.0
Default value used in the case study; impacts sampling.
max_tokens = 4096
Limits generation length; an experimental configuration.
specific knowledge content per agent = unspecified
The paper describes types of knowledge but not the actual content or prompts, so the system's performance depends on these undisclosed choices.

assumptions (3)

domain assumption LLM-based agents can perform elicitation, analysis, specification, and validation tasks with sufficient quality when given predefined actions and knowledge.
The entire framework rests on this capability; the case study provides only anecdotal evidence on one system.
domain assumption The artifact pool blackboard mechanism can coordinate agent workflows by state monitoring.
The design assumes that monitoring pool state and triggering agents will produce correct collaboration; not validated systematically.
domain assumption Knowledge can be effectively injected into prompts using the described strategies.
The paper references prompt strategies [19,20] without demonstrating the marginal effect of knowledge injection.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Knowledge-Guided Multi-Agent Framework for Automated Requirements Development: A Vision." pith.science (2026). https://pith.science/paper/4TYXFDDV

@misc{pith2026250622656,
  author       = {Pith},
  title        = {Pith review of: Knowledge-Guided Multi-Agent Framework for Automated Requirements Development: A Vision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4TYXFDDV}},
  note         = {Machine review of arXiv:2506.22656}
}

read the original abstract

This paper envisions a knowledge-guided multi-agent framework named KGMAF for automated requirements development. KGMAF aims to address gaps in current automation systems for SE, which prioritize code development and overlook the complexities of requirements tasks. KGMAF is composed of six specialized agents and an artifact pool to improve efficiency and accuracy. Specifically, KGMAF outlines the functionality, actions, and knowledge of each agent and provides the conceptual design of the artifact pool. Our case study highlights the potential of KGMAF in real-world scenarios. Finally, we outline several research opportunities for implementing and enhancing automated requirements development using multi-agent systems. We believe that KGMAF will play a pivotal role in shaping the future of automated requirements development in the era of LLMs.

Figures

Figures reproduced from arXiv: 2506.22656 by the authors.

**Figure 2.** The artifacts generated by our KGMAF on the insurance management system. their prompts, e.g., research templates, system-specific knowledge, and requirements templates. ❸ Actions. ConductResearch: The action is designed to search for related questions for the given brief description of the system to be developed, collecting a list of URLs, summarizing the content of the webpages associated with the list, and subseq… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 18 canonical work pages

[1]

Research directions in requirements engineering

Betty HC Cheng and Joanne M Atlee. Research directions in requirements engineering. Future of software engineering , pages 285–303, 2007

work page 2007
[2]

Requirements engineering: a roadmap

Bashar Nuseibeh and Steve Easterbrook. Requirements engineering: a roadmap. In Proceedings of the Conference on the Future of Software Engineering , pages 35–46, 2000

work page 2000
[3]

Software engineering programs are not computer science programs

David Lorge Parnas. Software engineering programs are not computer science programs. IEEE software, 16(6):19–30, 1999

work page 1999
[4]

Mare: Multi- agents collaboration framework for requirements engineering

Dongming Jin, Zhi Jin, Xiaohong Chen, and Chunhui Wang. Mare: Multi- agents collaboration framework for requirements engineering. arXiv preprint arXiv:2405.03256, 2024

arXiv 2024
[5]

Software Requirements Engineering Methods and Practices

Zhi Jin, Lin Liu, Xiaohong Chen, and Tong Li. Software Requirements Engineering Methods and Practices. Tsinghua University Press, 2023

work page 2023
[6]

An evaluation of requirements modeling for cyber- physical systems via llms

Dongming Jin, Shengxin Zhao, Zhi Jin, Xiaohong Chen, Chunhui Wang, Zheng Fang, and Hongbin Xiao. An evaluation of requirements modeling for cyber- physical systems via llms. arXiv preprint arXiv:2408.02450, 2024

arXiv 2024
[7]

Source code summarization in the era of large language models

Weisong Sun, Yun Miao, Yuekang Li, Hongyu Zhang, Chunrong Fang, Yi Liu, Gelei Deng, Yang Liu, and Zhenyu Chen. Source code summarization in the era of large language models. In Proceedings of the 47th International Conference on Software Engineering, pages 419–431, Ottawa, Ontario, Canada, 27 April-3 May, 2025 2025. IEEE Computer Society

work page 2025
[8]

Webarena: A realistic web environment for building autonomous agents

Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Srid- har, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023

arXiv 2023

Show all 23 references

[9]

Chatdev: Communicative agents for software development

Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1...

2024
[10]

Metagpt: Meta programming for multi-agent collaborative framework

Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 2023

2023 arXiv
[11]

Gpt-engineer, 2023

Gpt-engineer. Gpt-engineer, 2023. Accessed: 2025-01-10

2023
[12]

Autogpt, 2023

AutoGPT. Autogpt, 2023. Accessed: 2025-01-10

2023
[13]

Gpt-powered elicitation interview script generator for requirements engineering training

Binnur Görer and Fatma Basak Aydemir. Gpt-powered elicitation interview script generator for requirements engineering training. In 32nd IEEE International Requirements Engineering Conference, pages 372–379. IEEE, 2024

2024
[14]

Multi-label requirements classifica- tion with large taxonomies

Waleed Abdeen, Michael Unterkalmsteiner, Krzysztof Wnuk, Alexandros Chir- toglou, Christoph Schimanski, and Heja Goli. Multi-label requirements classifica- tion with large taxonomies. In 32nd IEEE International Requirements Engineering Conference, pages 264–274. IEEE, 2024

2024
[15]

Using llms in software requirements specifications: An empirical evaluation

Madhava Krishna, Bhagesh Gaur, Arsh Verma, and Pankaj Jalote. Using llms in software requirements specifications: An empirical evaluation. In 32nd IEEE International Requirements Engineering Conference , pages 475–483. IEEE, 2024

2024
[16]

Automated smell detection and recommendation in natural language requirements

Alvaro Veizaga, Seung Yeob Shin, and Lionel C Briand. Automated smell detection and recommendation in natural language requirements. IEEE Transactions on Software Engineering, 2024

2024
[17]

Blackboard systems

Iain D Craig. Blackboard systems. Artificial Intelligence Review, 2(2):103–118, 1988

1988
[18]

https://github.com/lenve/javadoc, 2020

The source for the selected case study. https://github.com/lenve/javadoc, 2020

2020
[19]

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824–24837, 2022

2022
[20]

Tree of thoughts: Deliberate problem solving with large language models

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems , 36, 2024

2024
[21]

Gpt-4 technical report

R OpenAI et al. Gpt-4 technical report. ArXiv, 2303:08774, 2023

2023
[22]

https://openai.com/

Openai. https://openai.com/
[23]

https://anonymous.4open

The generated requirements artifacts in our case study. https://anonymous.4open. science/r/KARE-Vision-4C63

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Research directions in requirements engineering

Betty HC Cheng and Joanne M Atlee. Research directions in requirements engineering. Future of software engineering , pages 285–303, 2007

work page 2007

[2] [2]

Requirements engineering: a roadmap

Bashar Nuseibeh and Steve Easterbrook. Requirements engineering: a roadmap. In Proceedings of the Conference on the Future of Software Engineering , pages 35–46, 2000

work page 2000

[3] [3]

Software engineering programs are not computer science programs

David Lorge Parnas. Software engineering programs are not computer science programs. IEEE software, 16(6):19–30, 1999

work page 1999

[4] [4]

Mare: Multi- agents collaboration framework for requirements engineering

Dongming Jin, Zhi Jin, Xiaohong Chen, and Chunhui Wang. Mare: Multi- agents collaboration framework for requirements engineering. arXiv preprint arXiv:2405.03256, 2024

arXiv 2024

[5] [5]

Software Requirements Engineering Methods and Practices

Zhi Jin, Lin Liu, Xiaohong Chen, and Tong Li. Software Requirements Engineering Methods and Practices. Tsinghua University Press, 2023

work page 2023

[6] [6]

An evaluation of requirements modeling for cyber- physical systems via llms

Dongming Jin, Shengxin Zhao, Zhi Jin, Xiaohong Chen, Chunhui Wang, Zheng Fang, and Hongbin Xiao. An evaluation of requirements modeling for cyber- physical systems via llms. arXiv preprint arXiv:2408.02450, 2024

arXiv 2024

[7] [7]

Source code summarization in the era of large language models

Weisong Sun, Yun Miao, Yuekang Li, Hongyu Zhang, Chunrong Fang, Yi Liu, Gelei Deng, Yang Liu, and Zhenyu Chen. Source code summarization in the era of large language models. In Proceedings of the 47th International Conference on Software Engineering, pages 419–431, Ottawa, Ontario, Canada, 27 April-3 May, 2025 2025. IEEE Computer Society

work page 2025

[8] [8]

Webarena: A realistic web environment for building autonomous agents

Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Srid- har, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023

arXiv 2023