Pith. sign in

REVIEW 5 major objections 5 minor 63 references

TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development

T0 review · 5 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TraceDev lifts AI software generation success to 53-57 percent by linking requirements, design, and code.

desk verdict The traceability-graph mechanism is worth taking seriously; the reported magnitudes are not yet trustworthy, but peer review should engage rather than desk-reject. read the letter →

arxiv 2607.18886 v1 pith:47A4JPY7 submitted 2026-07-21 cs.SE

Mingyu Chen , Yakun Zhang , Zihao Xie , Yixing Luo , Jinrui Xu , Cuiyun Gao , Kaiqi Zhao , Yunming Ye This is my paper
classification cs.SE
keywords requirementtraceabilitymulti-agentLLMframeworkrepository-levelcodegenerationuse-case-drivendevelopmentgraphsemanticcoveragesuccessrateLLM-as-a-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TraceDev's central claim is that an explicitly maintained traceability graph connecting requirements, design models, and code is what makes end-to-end, use-case-level code generation reliable. Prior multi-agent frameworks start from simplified single-sentence instructions and never track whether a requirement was actually implemented, so generated repositories silently omit entry conditions or flow steps. TraceDev adds a Validator agent that builds a heterogeneous graph over refined use cases, UML designs, and code files; missing links trigger targeted repair to the designer or developer. Evaluated on 125 use cases from two real-world systems, the framework attains success rates of 53.63% and 56.82%, more than double the two leading baselines, while human raters score its executability above 3.6 of 4. The implication is that traceability, not just more capable models or more agents, is a distinct control lever for automated software development.

What carries the argument

The traceability graph G=(V,E) is a heterogeneous directed graph linking three artifact layers: requirement nodes extracted by regular expressions from the refined use case, design nodes from PlantUML class and sequence diagrams, and code-file nodes. Edges are created in two phases: requirement-to-design via LLM semantic matching, and design-to-code via AST parsing. The graph does double work: it is a completeness checker, since missing edges pinpoint unimplemented requirements or design elements, and a compressed memory structure that lets agents share the essential project structure without flooding the context window.

What would settle it

Recompute success rates using test suites generated from the use case only, with no ground-truth code shown to the test-writing model (or written by human experts blind to the code); if TraceDev's lead over the baselines shrinks to near zero, the reported improvement is an artifact of oracle leakage rather than traceability.

Watch

Extended reading notes

Core claim

The paper's central discovery is that requirement traceability, exercised during generation rather than recovered after the fact, is what prevents functional omissions and semantic deviations in repository-level code generation. TraceDev's Validator Agent constructs a directed graph whose nodes are requirement elements (actors, entry/exit conditions, flow-of-event steps, quality requirements), design elements (classes, methods, relations, messages), and code files; edges are established by LLM-based semantic matching from requirements to design and by AST-based syntactic matching from design to code. Every iteration the Validator finds missing links, compiles a quality report, and feeds it t

Load-bearing premise

The reported success rates rest on an LLM-generated test oracle that sees both the use case and the ground-truth code, and keeps only tests that pass on that ground truth; if those tests leak implementation details or favor the reference solution, TraceDev's advantage over the baselines may be overstated.

Editorial extensions

If this is right

  • Repository-level generation from multi-point use cases becomes viable: TraceDev passes more than half of the LLM-generated test suite on two real systems, while prior frameworks pass under a quarter.
  • The Validator's traceability check is the main driver of semantic completeness: removing it drops semantic coverage by roughly 15 points, confirming that explicit requirement-to-code linking prevents omitted functionality.
  • The Tester's execution feedback is the main driver of functional correctness: without it, success rate collapses to 13.92% even though generated code volume stays nearly unchanged.
  • The Designer is what produces modular multi-file repositories: without it, generated code shrinks from about 15 files to about 5 files, suggesting that use-case-driven generation without architecture yields monolithic scripts.
  • The mechanism generalizes across three different LLM backbones, with a success-rate lead over baselines in every configuration, indicating that the traceability graph rather than a specific model is responsible for the improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the traceability graph is the active ingredient, reusing it beyond generation — for change-impact analysis, bug localization, or explaining generated code to maintainers — could deliver the same completeness gains on the maintenance phase, which dominates lifecycle cost.
  • Because tests are the main correctness signal but are absent from the graph, adding test-to-requirement and test-to-code edges is an obvious extension; that would let failures be traced to specific requirements and potentially sharpen the repair loop.
  • The results were measured with an oracle built from the ground-truth code; a stronger demonstration would re-run the evaluation with tests generated only from use cases, or with independently written tests, to rule out oracle bias.
  • The requirement-refinement step limits inputs to structured use cases; applying the same graph machinery to user stories or free-text requirements would require new parsing and schema layers, so the approach's reach into other requirement formats remains untested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. TraceDev proposes a five-agent, traceability-driven framework that converts Natural Language Requirements expressed as use cases into repository-level Java code. The Validator Agent constructs a heterogeneous traceability graph linking requirement nodes, design-model nodes, and code nodes, and uses missing links to generate quality reports that drive iterative refinement. The paper evaluates TraceDev on 125 use cases from ETOUR and SMOS against ChatDev and MetaGPT under Gemini-2.5-Flash, DeepSeek-V3.2, and a supplementary GPT-5-mini experiment. Reported automatic metrics are semantic-coverage rate and success rate, complemented by a small human executability study and code statistics. The main claim is that TraceDev improves success rate by roughly 2–3x over the baselines, with the traceability graph as the key enabling mechanism.

Significance. If the reported gains are valid, the paper would make a meaningful contribution to LLM-based repository-level code generation: it targets use-case-granularity requirements rather than single-sentence prompts, and it is among the first to make traceability an explicit, machine-maintained artifact during generation. The ablation study is well designed at the agent level, the authors release code and data, and the effect is consistent across two datasets, three LLM backbones, and human evaluation. These strengths make the central direction worth pursuing. However, the significance hinges on whether the automatic oracle measures actual requirement satisfaction rather than proximity to the reference implementation; as described, the oracle is not independent of the ground truth, and the adaptation step is not specified.

major comments (5)
  1. [§4.1, Figure 7] The test-generation prompt includes both the use case and the ground-truth Java code, and instructs the LLM to achieve statement and branch coverage of that Java class. Tests are then filtered by retention only if they pass on the ground truth. This makes the oracle an approximate measure of behavioral similarity to the reference implementation, not of requirement satisfaction alone. The reported success rates therefore conflate 'matches the reference solution' with 'satisfies the use case'. This is a load-bearing validity threat because the paper's central 2–3x improvement over ChatDev and MetaGPT is computed from this oracle. I ask the authors to (a) generate or validate test cases from the use case alone or from independently written specifications, (b) quantify how many retained tests reference class/method names or control-flow details that are implementation-specific, and (c) repor
  2. [§4.3.1] The 'adapt and execute' step for running generated-code tests is not specified. If adaptation is permissive, an LLM can rewrite test expectations to match whatever API the generated code exposes, inflating success; if it is rigid, valid alternative designs—which TraceDev systematically produces (12–15 files vs. 4–5 for baselines)—are penalized. Either way the reported absolute success rates are not interpretable without a precise protocol. The paper must state exactly what is adapted, who adapts it, what constraints are placed on the adaptation, and provide examples of both successful and failed adaptations from the experiments. Without this, the functional-correctness claim is not reproducible.
  3. [§5.1, Table 2] No variance or significance testing is reported. TraceDev uses temperature 0 while ChatDev uses 0.2 and MetaGPT uses a dynamic temperature, so the comparison may mix determinism with stochasticity; but even setting that aside, a single run per condition cannot establish that TraceDev's advantage is not within run-to-run noise, especially for the smaller DeepSeek-V3.2 and GPT-5-mini comparisons. I request multiple seeds or repeated runs with confidence intervals, or a paired significance test across the 125 use cases, for at least the two main LLM backbones.
  4. [§3.5, Algorithm 2] The traceability graph is the hypothesized mechanism, yet its own accuracy is never measured. The Validator constructs requirement-to-design links via LLM semantic matching and design-to-code links via AST matching, but the paper reports no precision/recall for these links, no human inspection of graph correctness, and no error analysis of the quality reports. The ablation in Figure 8 shows that removing the Validator reduces semantic coverage, but it does not isolate whether the gain comes from accurate graph structure, from the extra repair iterations, or from the added context length. I ask the authors to measure graph link quality on a sample of use cases and to compare the Validator's feedback against a control that receives the same number of repair iterations without graph-derived links.
  5. [§6, 'Generalizability Across LLMs'] The GPT-5-mini experiment is reported as 'As shown in Table 2', but the results appear in Table 4. Beyond the citation error, the experiment uses only a 20% random subset with no reported sample size per dataset, so the stability of the 51.84% success rate is unclear. This is a minor fix, but combined with the absence of variance reporting it limits the strength of the cross-LLM generalization claim.
minor comments (5)
  1. [§4.1 / Figure 7] The prompt in Figure 7 uses '3.' twice in the instructions and has an unmatched quote in 'expected_result'; also 'semantic_explanation' is not in the described JSON field order. These should be cleaned up for reproducibility.
  2. [§3.4, Algorithm 1] The comments 'Line 2 in Algorithm 1' and 'Line 2 & 6 in Algorithm 1' are confusing because the referenced lines are not uniquely identified; renumber or use named step references.
  3. [§4.3.2] Human evaluation uses only 20 sampled use cases and averages scores across three evaluators; the Krippendorff's alpha of 0.940 is reported overall but not per dataset. Please report per-dataset agreement and the rating distribution.
  4. [§5.3, Table 3] The 'Tim' column header is a typo for 'Time'; 'Time effect.' is also nonstandard. Use 'Time effectiveness' consistently.
  5. [§5.1, Table 2] The 'Files', 'LOC', and 'Tokens' columns are averages over use cases but the table does not state whether they refer to valid files only; the definition of 'valid code file' in §4.3.3 should be stated before the table.

Circularity Check

1 steps flagged · score 3.0 of 10

Mild evaluation circularity: success-rate oracle is generated from ground-truth code, so the reported gains partly measure proximity to the reference implementation rather than requirement satisfaction.

  1. fitted input called prediction [Section 4.1 (Figure 7) and Section 4.3.1 (Success Rate)]
    "we first inject the use cases along with the relevant ground-truth code into the prompt (see Figure 7), guiding the LLM in generating test cases. ... we retain the test cases that pass on ground-truth code. ... For test cases that pass semantic verification, we adapt and execute them against the generated code to evaluate functional correctness."

    The test oracle is built from the ground-truth Java code, with explicit instructions to achieve statement and branch coverage of that reference class. The success-rate metric is then defined as passing these tests after an unspecified 'adapt and execute' step. Thus the measured 'functional correctness' reduces to conformance with a test suite that embeds the reference implementation's class names, method boundaries, and control flow, rather than an independent check of the use-case requirements. Because the adaptation step is not specified, the comparison can either be tautological (tests rewritten to match whatever API the generated code exposes) or unfairly penalize valid alternative designs. This makes the reported 2-3x advantage partially an artifact of oracle construction, though the

full rationale

TraceDev is an empirical systems paper, not an analytical derivation, so there is no equation-level circularity. I found no load-bearing self-citations: the evaluation methodology follows prior works [37,61] by different author teams, and the few self-citations (e.g., [9,19]) are contextual, not used to justify the central claim. The framework's traceability graph is not validated independently, but the ablation study provides direct empirical evidence for the Validator's contribution. The only meaningful circularity concern is in the automatic evaluation: the test suites are generated from the ground-truth code and then used as the oracle for success rate. This embeds reference-implementation details into the metric and, combined with an unspecified test-adaptation step, makes the absolute success rates and some of the relative advantage over baselines depend on proximity to the reference solution. However, the human executability evaluation (Krippendorff's alpha 0.940) and the ablation results are independent of this oracle and partially support the paper's conclusions. The central contribution is therefore not equivalent to its inputs by construction; the circularity is partial and confined to the automatic evaluation pipeline. Score 3 reflects this mild, non-load-bearing circularity rather than a collapse of the derivation chain.

Assumptions & free parameters 4 free parameters · 6 assumptions · 1 invented entities

The paper's central claim rests on several domain assumptions about the validity of use-case-based evaluation, the reliability of LLM/AST trace-link construction, and the fairness of an oracle built from ground-truth code. There are no mathematical free parameters, but the retry limits and judge settings are tuned on a subset of the same evaluation data. The only invented entity is the traceability graph itself, whose independent accuracy is not established.

free parameters (4)
  • Maximum traceability repair iterations = 3
    Set empirically on ~20% of the same datasets used for final evaluation (Section 4.4); controls how many Validator feedback rounds run.
  • Maximum test remediation attempts = 5
    Set empirically on ~20% of the same datasets (Section 4.4); caps Developer-Tester self-correction loops.
  • LLM temperature = 0 for TraceDev and DeepSeek judge; 0.2 for ChatDev; dynamic for MetaGPT
    Different temperature settings across frameworks affect determinism and comparability (Section 4.4).
  • Semantic-coverage judge rounds = 3 rounds, majority vote
    Design choice for the LLM-as-judge protocol (Section 4.3.1); no evidence that three rounds is sufficient.
assumptions (6)
  • domain assumption Use cases following Cockburn's schema are a representative and prevalent form of real-world software requirements.
    Used to justify ETOUR/SMOS as evaluation grounds and to motivate the framework (Sections 1, 2.1, Figure 3).
  • domain assumption LLM semantic matching (requirement-to-design) and AST syntactic matching (design-to-code) recover sufficiently accurate traceability links.
    The Validator's missing-link detection and repair loop depends on these linkings (Algorithm 2, Section 3.5); the paper itself concedes links can be incorrect (Section 6).
  • domain assumption LLM-generated test suites that pass on ground-truth code are correct, unbiased oracles for functional correctness.
    Tests are generated with ground-truth code in the prompt and filtered by passing on that code (Section 4.1, Figure 7).
  • domain assumption LLM-as-a-judge semantic coverage with three rounds and majority voting is a valid measure of whether generated code implements the requirement.
    No direct validation of the judge's accuracy against human labels beyond the overall human-evaluation result (Section 4.3.1).
  • domain assumption The ETOUR and SMOS datasets and their ground-truth Java implementations are representative of real-world repository-level development.
    The generalization claim rests on two small academic datasets (Section 5.4).
  • domain assumption Deterministic decoding (temperature 0) and single runs are sufficient to compare frameworks.
    No repeated runs or statistical significance testing are reported (Section 4.4, Table 2).
invented entities (1)
  • Heterogeneous traceability graph
    purpose: Links requirement nodes, design nodes, and code nodes; used by the Validator to detect missing links and to compress artifacts as shared memory.
    The graph's accuracy is never directly measured; only downstream pipeline success is reported, and no ablation isolates graph correctness from the other agents.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development." pith.science (2026). https://pith.science/paper/47A4JPY7

@misc{pith2026260718886,
  author       = {Pith},
  title        = {Pith review of: TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/47A4JPY7}},
  note         = {Machine review of arXiv:2607.18886}
}
read the original abstract

In modern software development, the rapid advancement of Large Language Models (LLMs) has made the end-to-end transformation of Natural Language Requirements (NLRs) into executable repository-level code increasingly feasible. However, existing approaches typically rely on simplified instructions (e.g., single-sentence descriptions), failing to reflect complex software development scenarios. Moreover, they lack explicit requirement traceability mechanisms, making it difficult to precisely align and validate generated code against original requirements. To address these limitations, we propose TraceDev, a multi-agent framework for automated software development grounded in use cases that contain multiple functional points and complex semantics. TraceDev employs five role-specific agents, including a Requirement Refiner, Designer, Developer, Tester, and Validator. Notably, the Validator Agent constructs and maintains a heterogeneous traceability graph that links requirements, design models, and code artifacts for interacting with the preceding four agents. The traceability graph maintains consistency across various artifacts and serves as a structured context for efficient memory management, supporting reliable repository-level code generation. We evaluate TraceDev on two widely used datasets (including 125 use cases) compared with two state-of-the-art approaches. On the ETOUR dataset, TraceDev achieves a success rate of 53.63\%, outperforming baseline approaches by up to 186.63\%. A similar trend is observed on the SMOS dataset, where TraceDev attains a success rate of 56.82\%, exceeding baseline approaches by up to 340.80\%. These results demonstrate the effectiveness of TraceDev in repository-level code generation from requirements.

Figures

Figures reproduced from arXiv: 2607.18886 by the authors.

Figure 1
Figure 1. The motivation example. 3 TraceDev In this section, we propose TraceDev, a traceability-driven multi-agent framework that enables end-to-end repository-level code generation from Natural Language Requirements (NLRs) described in use cases. We first present the overview of TraceDev and then describe its details. 3.1 Overview As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. An overview of TraceDev. preprocessor that takes original use cases as input and produces refined use cases with unified terminology and explicit semantics. We analyze ETOUR and SMOS, two widely recognized datasets in the requirements domain, and identify the following challenges: • Challenge 1: Terminology Inconsistency. Although the use case specification proposed by Alistair Cockburn has been widely recognized as… view at source ↗
Figure 3
Figure 3. A comparison of ETour and SMOS use cases, along with the Requirement Refiner Agent’s prompt that [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: A class diagram generated by TraceDev using Gemini-2.5-Flash. Due to space limitations, only a subset of the diagram is shown. The dia￾gram is based on the use case from [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: A traceability graph generated by TraceDev using Gemini-2.5-Flash. Due to space limitations, only a [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: The prompt for LLM-guided test case generation using use cases and ground-truth code. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Ablation study. The “red bar” denotes the original TraceDev, while the “green bar”, “yellow bar”, “purple bar”, and “blue bar” denote TraceDev without the Refiner Agent, without the Designer Agent, without the Tester Agent, and without the Validator Agent, respectively…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 4 canonical work pages

  1. [1]

    2003.Empirical Studies of Construction and Application of Use Case Models

    Bente Anda. 2003.Empirical Studies of Construction and Application of Use Case Models. Ph. D. Dissertation. ISSN 1501-7710, Nr. 268

    2003

  2. [2]

    Giulio Antoniol, Gerardo Canfora, Gerardo Casazza, Andrea De Lucia, and Ettore Merlo. 2025. Recovering Traceability Links Between Code and Documentation: A Retrospective.IEEE Transactions on Software Engineering51, 3 (2025), 825–832. doi:10.1109/TSE.2025.3534027

    arXiv 2025

  3. [3]

    Schneider

    Avijit Bhattacharjee, Banani Roy, and Kevin A. Schneider. 2022. Supporting program comprehension by generating abstract code summary tree. InProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA080. Publication date: October 2026. TraceDev: A Traceability-Driven Multi-age...

    arXiv 2022

  4. [4]

    Barry W. Boehm. 1984. Software Engineering Economics.IEEE Transactions on Software EngineeringSE-10, 1 (1984), 4–21. doi:10.1109/TSE.1984.5010193

    arXiv 1984

  5. [5]

    Markus Borg, Per Runeson, and Anders Ardö. 2014. Recovering from a decade: a systematic mapping of information retrieval approaches to software traceability.Empirical Software Engineering19, 6 (2014), 1565–1616

    2014

  6. [6]

    Center of Excellence for Software Systems Traceability. 2026. Datasets. http://sarec.nd.edu/coest/datasets.html

    2026

  7. [7]

    Mark Chen. 2021. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374(2021)

    arXiv 2021

  8. [8]

    Xiangping Chen, Xing Hu, Yuan Huang, He Jiang, Weixing Ji, Yanjie Jiang, Yanyan Jiang, Bo Liu, Hui Liu, Xiaochen Li, et al. 2025. Deep learning-based software engineering: progress, challenges, and opportunities.Science China Information Sciences68, 1 (2025), 111102

    2025

Show all 63 references
  1. [9]

    Yujia Chen, Mingyu Chen, Cuiyun Gao, Zhihan Jiang, Zhongqi Li, and Yuchi Ma. 2025. Towards Mitigating API Hallucination in Code Generated by LLMs with Hierarchical Dependency Aware. InProceedings of the 33rd ACM International Conference on the Foundations of Software Engineeri...

    2025

  2. [10]

    Yujia Chen, Yang Ye, Zhongqi Li, Yuchi Ma, and Cuiyun Gao. 2025. Smaller but Better: Self-Paced Knowledge Distillation for Lightweight yet Effective LCMs.Proceedings of the ACM on Software Engineering2, FSE (2025), 3057–3080

    2025

  3. [11]

    Jane Cleland-Huang, Orlena C. Z. Gotel, Jane Huffman Hayes, Patrick Mäder, and Andrea Zisman. 2014. Software traceability: trends and future directions. InFuture of Software Engineering Proceedings(Hyderabad, India)(FOSE 2014). Association for Computing Machinery, New York, NY...

    2014

  4. [12]

    2008.Writing effective use cases

    Alistair Cockburn and Lord Cockburn. 2008.Writing effective use cases. Pearson Education India

    2008

  5. [13]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic c...

    2025 arXiv

  6. [14]

    Gustavo de Aquino e Aquino, Nádila da Silva de Azevedo, Leandro Youiti Silva Okimoto, Leonardo Yuto Suzuki Camelo, Hendrio Luis de Souza Bragança, Rubens Fernandes, Andre Printes, Fábio Cardoso, Raimundo Gomes, and Israel Gondres Torné. 2025. From RAG to Multi-Agent Systems: A...

    2025

  7. [15]

    Guo, and Jane Cleland-Huang

    Davide Falessi, Justin Roll, Jin L.C. Guo, and Jane Cleland-Huang. 2020. Leveraging Historical Associations between Requirements and Source Code to Identify Impacted Classes .IEEE Transactions on Software Engineering46, 04 (April 2020), 420–441. doi:10.1109/TSE.2018.2861735

    2020

  8. [16]

    Xavier Franch, Cristina Palomares, Carme Quer, Panagiota Chatzipetrou, and Tony Gorschek. 2023. The state-of- practice in requirements specification: an extended interview study at 12 companies.Requirements Engineering28, 3 (Sept. 2023), 377–409. doi:10.1007/s00766-023-00399-7

    2023 doi

  9. [17]

    Julian Frattini and Anja Frattini. 2025. Adopting Use Case Descriptions for Requirements Specification: an Industrial Case Study.arXiv preprint arXiv:2506.13303(2025)

    2025 arXiv

  10. [18]

    Dominik Fuchß, Tobias Hey, Jan Keim, Haoyu Liu, Niklas Ewald, Tobias Thirolf, and Anne Koziolek. 2025. LiSSA: toward generic traceability link recovery through retrieval-augmented generation. InProceedings of the IEEE/ACM 47th International Conference on Software Engineering. ...

    2025

  11. [19]

    Cuiyun Gao, Xing Hu, Shan Gao, Xin Xia, and Zhi Jin. 2025. The current challenges of software engineering in the era of large language models.ACM Transactions on Software Engineering and Methodology34, 5 (2025), 1–30

    2025

  12. [20]

    Gotel and C.W

    O.C.Z. Gotel and C.W. Finkelstein. 1994. An analysis of the requirements traceability problem. InProceedings of IEEE International Conference on Requirements Engineering. 94–101. doi:10.1109/ICRE.1994.292398

    1994

  13. [21]

    John Grundy, John Hosking, and Warwick B Mugridge. 2002. Inconsistency management for multiple-view software development environments.IEEE Transactions on Software Engineering24, 11 (2002), 960–981

    2002

  14. [22]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence. arXiv preprint arXiv:2401.14196(2024)

    2024 arXiv

  15. [23]

    Jin LC Guo, Jan-Philipp Steghöfer, Andreas Vogelsang, and Jane Cleland-Huang. 2025. Natural language processing for requirements traceability. InHandbook on Natural Language Processing for Requirements Engineering. Springer, 89–116

    2025

  16. [24]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. 2023. MetaGPT: Meta programming for a multi-agent collaborative framework. InThe Twelfth International Conference on Learning Repr...

    2023

  17. [25]

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review.ACM Transactions on Software Engineering and Methodology33, 8 (2024), 1–79

    2024

  18. [26]

    Nam Huynh and Beiyu Lin. 2025. Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applications.arXiv preprint arXiv:2503.01245(2025). Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA080. Publication date: October...

    2025 arXiv

  19. [27]

    1993.Object-oriented software engineering: a use case driven approach

    Ivar Jacobson. 1993.Object-oriented software engineering: a use case driven approach. Pearson Education India

    1993

  20. [28]

    Dongming Jin, Zhi Jin, Yiran Zhang, Zheng Fang, Linyu Li, Yuanpeng He, Xiaohong Chen, and Weisong Sun. 2025. UserTrace: User-Level Requirements Generation and Traceability Recovery from Software Project Repositories.arXiv preprint arXiv:2509.11238(2025)

    2025 arXiv

  21. [29]

    Javed Ali Khan, Shamaila Qayyum, and Hafsa Shareef Dar. 2025. Large Language Model for Requirements Engineering: A Systematic Literature Review. (2025)

    2025

  22. [30]

    Klaus Krippendorff. 2011. Computing Krippendorff’s alpha-reliability. (2011)

    2011

  23. [31]

    Yukyung Lee, Joonghoon Kim, Jaehee Kim, Hyowon Cho, Jaewook Kang, Pilsung Kang, and Najoung Kim. 2025. Checkeval: A reliable llm-as-a-judge framework for evaluating text generation using checklists. InProceedings of the 2025 Conference on Empirical Methods in Natural Language ...

    2025

  24. [32]

    Jierui Li, Hung Le, Yingbo Zhou, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. 2025. CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computa...

    2025 doi

  25. [33]

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode.Science378, 6624 (2022), 1092–1097

    2022

  26. [34]

    Jinfeng Lin, Yalin Liu, Qingkai Zeng, Meng Jiang, and Jane Cleland-Huang. 2021. Traceability transformed: Generat- ing more accurate links with pre-trained bert models. In2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 324–335

    2021

  27. [35]

    Junhong Lin, Xinyue Zeng, Jie Zhu, Song Wang, Julian Shun, Jun Wu, and Dawei Zhou. 2025. Plan and Budget: Effective and Efficient Test-Time Scaling on Large Language Model Reasoning.arXiv preprint arXiv:2505.16122(2025)

    2025

  28. [36]

    Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al . 2025. Deepseek-v3. 2: Pushing the frontier of open large language models.arXiv preprint arXiv:2512.02556(2025)

    2025 arXiv

  29. [37]

    Jane Luo, Xin Zhang, Steven Liu, Jie Wu, Jianfeng Liu, Yiming Huang, Yangyu Huang, Chengyu Yin, Ying Xin, Yuefeng Zhan, et al. 2025. RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation.arXiv preprint arXiv:2509.16198(2025)

    2025

  30. [38]

    Patrick Mäder and Alexander Egyed. 2015. Do developers benefit from requirements traceability when evolving and maintaining a software system?Empirical Softw. Engg.20, 2 (April 2015), 413–441. doi:10.1007/s10664-014-9314-z

    2015 doi

  31. [39]

    Pedro Neves Mata, José Moleiro Martins, and João Carlos Ferreira. 2025. New software product development: Biblio- metric analysis.Journal of the Knowledge Economy16, 1 (2025), 4161–4184

    2025

  32. [40]

    2004.Deriving behavior specifications from textual use cases

    Vladimir Mencl. 2004.Deriving behavior specifications from textual use cases. Citeseer

    2004

  33. [41]

    Patrick Mäder and Alexander Egyed. 2012. Assessing the effect of requirements traceability for software maintenance. In2012 28th IEEE International Conference on Software Maintenance (ICSM). 171–180. doi:10.1109/ICSM.2012.6405269

    2012

  34. [42]

    JJ Norheim, E Rebentisch, D Xiao, L Draeger, A Kerbrat, and OL De Weck. 2024. Challenges in applying large language models to requirements engineering tasks. Des. Sci. 10 (2024)

    2024

  35. [43]

    Augustus Odena, Charles Sutton, David Martin Dohan, Ellen Jiang, Henryk Michalewski, Jacob Austin, Maarten Paul Bosma, Maxwell Nye, Michael Terry, and Quoc V Le. 2021. Program synthesis with large language models.n/a, page n/a, n/a. N/a(2021)

    2021

  36. [44]

    2005.Software engineering: a practitioner’s approach

    Roger S Pressman. 2005.Software engineering: a practitioner’s approach. Palgrave macmillan

    2005

  37. [45]

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. 2024. Chatdev: Communicative agents for software development. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol...

    2024

  38. [46]

    Michael Rath, David Lo, and Patrick Mäder. 2018. Analyzing requirements and traceability information to improve bug localization. InProceedings of the 15th International Conference on Mining Software Repositories(Gothenburg, Sweden) (MSR ’18). Association for Computing Machine...

    2018

  39. [47]

    Alberto D Rodriguez, Katherine R Dearstyne, and Jane Cleland-Huang. 2023. Prompts matter: Insights and strategies for prompt engineering in automated software traceability. In2023 IEEE 31st International Requirements Engineering Conference Workshops (REW). IEEE, 455–464

    2023

  40. [48]

    Marcela Ruiz, Jin Yang Hu, and Fabiano Dalpiaz. 2023. Why don’t we trace? A study on the barriers to software traceability in practice.Requirements Engineering28, 4 (2023), 619–637

    2023

  41. [49]

    Ian Sommerville. 2011. Software engineering (ed.).America: Pearson Education Inc(2011)

    2011

  42. [50]

    Thomas Stober and Uwe Hansmann. 2009. Traditional software development. InAgile Software Development: Best Practices for Large Software Development Projects. Springer, 15–33. Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA080. Publication date: October 2026. TraceDev: ...

    2009

  43. [51]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023)

    2023 arXiv

  44. [52]

    Tree-sitter. 2026. https://github.com/tree-sitter/tree-sitter

    2026

  45. [53]

    Félix Témolé and Desislava Atanasova. 2025. An Integrated Approach to Managing Software Qual- ity in Complex Systems.American Journal of Software Engineering and Applications13, 1 (2025), 1–17. arXiv:https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajsea.20251301.11 d...

    2025 doi

  46. [54]

    Jonathan Ullrich, Matthias Koch, and Andreas Vogelsang. 2025. From requirements to code: Understanding developer practices in llm-assisted software engineering. In2025 IEEE 33rd International Requirements Engineering Conference (RE). IEEE, 257–266

    2025

  47. [55]

    Hongyan Wan, Xinyu He, Yang Deng, and Bangchao Wang. 2025. A systematic mapping study of information retrieval-based requirements traceability methods.Information Processing & Management62, 6 (2025), 104287

    2025

  48. [56]

    O’Reilly Media, Inc

    Titus Winters, Tom Manshreck, and Hyrum Wright. 2020.Software engineering at google: Lessons learned from programming over time. " O’Reilly Media, Inc. "

    2020

  49. [57]

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. 2024. Autogen: Enabling next-gen LLM applications via multi-agent conversations. InFirst Conference on Language Modeling

    2024

  50. [58]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2025. The rise and potential of large language model based agents: A survey.Science China Information Sciences68, 2 (2025), 121101

    2025

  51. [59]

    Mohammad Amin Zadenoori, Jacek Dąbrowski, Waad Alhoshan, Liping Zhao, and Alessio Ferrari. 2025. Large Language Models (LLMs) for Requirements Engineering (RE): A Systematic Literature Review.arXiv preprint arXiv:2509.11446 (2025)

    2025

  52. [60]

    Quanjun Zhang, Chunrong Fang, Yang Xie, Yaxin Zhang, Yun Yang, Weisong Sun, Shengcheng Yu, and Zhenyu Chen

  53. [61]

    Sai Zhang, Zhenchang Xing, Ronghui Guo, Fangzhou Xu, Lei Chen, Zhaoyuan Zhang, Xiaowang Zhang, Zhiyong Feng, and Zhiqiang Zhuang. 2025. Empowering agile-based generative software development through human-ai teamwork. ACM Transactions on Software Engineering and Methodology(2025)

    2025

  54. [62]

    Dongqi Zheng. 2025. ARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models.arXiv preprint arXiv:2510.00071(2025). Received 2026-01-30; accepted 2026-04-16 Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA080. Publication date: October 2026

    2025

  55. [2023]

    A survey on large language models for software engineering.arXiv preprint arXiv:2312.15223(2023)

    2023 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.