Pith. sign in

REVIEW 5 major objections 5 minor 45 references

Autonomous Legacy Web Application Upgrades Using a Multi-Agent System

T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A multi-agent LLM pipeline keeps context while upgrading legacy web files, yet standalone prompts often outperform it.

desk verdict An honest mostly-negative empirical study of a multi-agent legacy-upgrade pipeline; the claim of occasional superiority rests on a single small comparison with a protocol that excludes failed generations. read the letter →

arxiv 2501.19204 v1 pith:KNIPHBX2 submitted 2025-01-31 cs.SE

Valtteri Ala-Salmi , Zeeshan Rasheed , Abdul Malik Sami , Zheying Zhang , Kai-Kristian Kemell , Jussi Rasku , Shahbaz Siddeeq , Mika Saari
show 1 more author
Pekka Abrahamsson
This is my paper · ORCID
classification cs.SE
keywords largelanguagemodelsmulti-agentsystemlegacysoftwareupgradeCakePHPzero-shotlearningone-shotcoderefactoringwebapplicationmaintenance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a multi-agent LLM system can autonomously upgrade legacy web application files, and that it does so while preserving context across subtasks and agents. The authors build a four-role pipeline — manager, task pipeline, verifier, and finalizer — and test it on five CakePHP view files migrated from version 1.2 to 4.5. They compare it against standalone zero-shot and one-shot prompts on the same model, counting errors and satisfied requirements over repeated runs. Their own results show the pipeline usually produces more errors, not fewer, and takes much longer, though it beats the standalone prompt on one complex jQuery-refactor task. The broader point they argue for is that current LLMs can update small outdated files with high precision even with basic prompts, making the pipeline a workable foundation for future improvements.

What carries the argument

The load-bearing mechanism is the multi-agent pipeline, a chain of four units that processes an upgrade request in phases. A manager agent turns the user's requirements into an ordered list of abstract tasks; a task pipeline builds a one-shot prompt for each task and executes it on the code sequentially; a verification agent inspects each task's output and either accepts it or sends it to a finalizer agent; and the finalizer edits the code and loops it back for re-verification, with a two-iteration cap before returning to the task pipeline. The intended work of the mechanism is to split an upgrade into small, verifiable steps to avoid long-context failures, to give every step a self-feedback loop, and to let the user write instructions without detailed prompt engineering.

What would settle it

Re-run the same five-file CakePHP 1.2-to-4.5 upgrade with an automated PHP test suite and linter, comparing the pipeline against standalone zero-shot and one-shot prompts on the same model. If the pipeline's mean distinct-error count is never below the best prompt's mean across files, and its requirement-pass rate on the two complex tasks never exceeds the prompt's, then the claimed occasional superiority of the pipeline is not reproducible.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that a manager–verifier–finalizer pipeline, where each task is planned, executed, verified, and finalized before the next task begins, can carry the context of a legacy code update across multiple agents and tasks, and that this can in some test cases yield better solutions than a standalone LLM prompt. The evidence is the comparison across five view files from a real CakePHP 1.2 dictionary application updated to CakePHP 4.5 using the GPT-4o-mini model. The pipeline kept generation context well enough to update every file type it was given, including a jQuery refactor where it achieved an average requirement score of 0.9 versus 0.5 for the zero-shot prompt. However, on the error-counted files it averaged 0.46 more distinct errors than the best prompt, and on the hardest file splitting the task into two subtasks collapsed requirement fulfillment from 1.8 to 0.5. The authors read these mixed numbers as evidence that the multi-agent architecture is a viable foundation, not yet a consistent improvement.

Load-bearing premise

The evaluation assumes that hand-counted 'different errors' and binary requirement checks, done without automated tests or a second independent rater, are a valid and unbiased measure of upgrade quality.

Editorial extensions

If this is right

  • If correct, the result implies that breaking a legacy upgrade into small, verified subtasks can preserve context across steps, so future systems can build on the pipeline rather than starting from scratch.
  • The comparison shows that current LLMs can update small, single-file view templates with high precision using only a short zero-shot or one-shot prompt, which has direct relevance for automating maintenance work.
  • The pipeline's best performance on the jQuery/Ajax refactor suggests that architectural rewrites may benefit more from multi-agent verification than simple syntax migrations do.
  • The system's slower runtime of 53 to 71 seconds versus 2 to 11 seconds for prompts means any real adoption must weigh accuracy gains against cost and latency.
  • Since splitting the hardest file into two tasks degraded results, the paper implies that task decomposition granularity is a key design choice for such pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the paper's telephone-game explanation suggests the verifier agent is not strong enough to stop cumulative code drift; a stronger reasoner in the verifier role, or an external test oracle, might reverse the current gap between pipeline and simple prompts.
  • My inference: the mixed results imply the value of multi-agent decomposition may be task-dependent, with cross-file consistency refactors benefiting while local syntax updates are better done in one shot.
  • My inference: a cheap testable extension would be to run the same five files with task order reversed and with an automated PHP syntax checker, separating genuine context preservation from simple prompt adherence.
  • My inference: because manual error counting lacked automated tests and inter-rater checks, the size of the claimed advantage on the jQuery refactor is uncertain, and replicating with a test suite is the natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper proposes a multi-agent pipeline—manager, task pipeline, verification agent, and finalizer—that takes a legacy web application file and user requirements, decomposes the update into subtasks, and iteratively verifies and finalizes the output. The authors evaluate it on five view files from a CakePHP 1.2 electronic dictionary application, upgrading them to CakePHP 4.5, using gpt-4o-mini and comparing against zero-shot and one-shot prompts over ten runs per condition. Outcome measures are manually counted distinct errors for views A–C and manually scored binary requirements for views D–E. The paper reports that the system sometimes outperforms standalone prompts (View D), but mostly performs comparably or worse, and concludes that the pipeline is a working foundation for future multi-agent code-update systems.

Significance. The paper's strength is its concrete, reproducible setup: a real legacy application, a 15-year version gap, public source code, repeated stochastic runs, and an explicit threats-to-validity discussion that acknowledges the manual evaluation and limited file count. If the central claim 'better solutions in some test cases' were firmly supported, the result would be a modest but useful empirical data point for multi-agent LLM systems in legacy code maintenance. However, the contribution is primarily empirical, and the current evidence is fragile: the failed-generation exclusion, inconsistent baselines, contradictory reporting in Section 4.1, an arithmetic error in Table 3, and the absence of significance tests all bear directly on the abstract's claim. The work is a reasonable foundation, but the reported comparisons need reanalysis before the claim can be accepted.

major comments (5)
  1. [Section 3.4 and Section 4.1, Table 2] The 'failed generation' rule (more than 7 error types or no code) and the explicit exclusion of one failed View C run from the error averages create a systematic bias in favor of the system; the mean 1.22 in Table 2 is computed over only the non-failed runs, and if the failed run were included as a worst-case outcome, the comparison against the ZSL mean of 0.3 would be materially worse. For View D (Table 3), the text does not state whether any failed generations occurred or were excluded, so the reported total of 0.9 is not robust. Please report all ten runs per condition, state explicitly how failed generations are handled, and provide a sensitivity analysis.
  2. [Table 3, View D row] The requirement averages 0.5, 0.3, and 0.2 sum to 1.0, not the reported total of 0.9; this is the only comparison in which the system beats the baseline, and it is also the basis of the abstract's 'better solutions in some test cases.' With n=10 per method and manually scored binary requirements, the difference is not accompanied by any significance test, confidence interval, or effect size, and the per-run data are not shown. Please correct the arithmetic and report a test (e.g., a permutation test or Fisher's exact test on the per-requirement counts) plus the raw scores.
  3. [Section 3.4 and Tables 2–4] The baseline is not held constant across files: View A is compared with both ZSL and OSL, View B only with OSL, View C only with ZSL, View D only with ZSL, and View E with ZSL plus two system variants. Because the baseline selection is made per file after seeing ZSL performance, the reported cross-file patterns and the introduction's claim of '0.46 more errors on average' are not well-defined. Please either pre-specify a single baseline for all files or report all conditions for every file so that differences cannot be attributed to prompt choice.
  4. [Section 4.1 and Figure 2] The sentence 'Only fatal and runtime errors were found from the generated files. Interestingly the proposed system did generate lesser runtime errors in View A and C, however it did generate more runtime errors in every file' is contradictory, and the error-category definitions in Section 3.4 include content and missing/additional features as well. Please clarify whether any content or missing-feature errors occurred, correct the runtime-error statement, and ensure Figure 2 is consistent with the text.
  5. [Section 3.4 and Section 5.1] All outcome measurements are manual, unblinded, and without inter-rater reliability, and the authors acknowledge this as a limitation. Because the reported differences are small (e.g., 0.6 vs 0.4 errors in View A; 0.5 vs 0.9 requirements in View D), the manual scoring protocol is load-bearing for every comparison in Tables 2–4. Please provide the full raw per-run results, a detailed scoring guide, and, ideally, a second rater's scores or a reproducibility statement.
minor comments (5)
  1. [Throughout] Several grammatical errors should be corrected, including 'an GPT model' in the conclusion, 'the prompt were run' in Section 4.1, and 'lesser runtime errors' in Section 4.1.
  2. [References] The reference (University, 2024) does not contain a URL or repository identifier, so the claimed public evaluation dataset cannot be located; please provide the link or DOI.
  3. [Figure 2] Figure 2 is referenced in Section 4.1, but no figure image appears in the manuscript as provided; please include the figure and make the caption self-contained.
  4. [Tables 2–4] The row naming conventions are inconsistent ('Syst.', 'Syst. (2 tasks)', 'Syst. (1 task)'), and Table 4 introduces a one-task system variant without explaining in Section 3.4 why the number of tasks was varied; please clarify.
  5. [Sections 3.2 and 3.4] Section 3.2 refers to a feedback loop that 'exceeds a certain amount of interactions,' while Section 3.4 specifies a maximum of two iterations; these statements should be made consistent.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the central empirical comparison is self-contained and the only self-citation (CodePori) is disclosed and not load-bearing.

full rationale

The paper does not derive any result from fitted parameters or equations, and it makes no prediction that reduces by construction to its inputs. The central claim is an empirical comparison between a multi-agent pipeline and standalone ZSL/OSL prompts on five CakePHP view files, with errors and requirements counted manually against an existing legacy application. That evaluation is external to the authors' prior work and does not presuppose the conclusion. The proposed system is explicitly stated to be 'referenced from the CodePori multi-agent system described in (Rasheed et al., 2024a)' (Section 3.2), which is a self-citation with overlapping authors, but it is used only as the architectural starting point; the tested capability of updating legacy view files is measured independently against baseline prompts rather than against CodePori's own outputs or benchmarks. No uniqueness theorem, ansatz, or hidden definition is imported from the authors' prior work to force the outcome. The paper even reports results unfavorable to its own system (e.g., View C Syst. with 1.22 errors vs ZSL 0.3, and View E two-task system performing much worse), which further indicates the evaluation is not engineered to confirm the hypothesis. The reviewer-level concerns about manual, non-blinded error counting and the handling of failed generations are threats to validity and statistical robustness, not circularity: they concern whether the measurements are accurate, not whether the conclusion is equivalent to the inputs. Accordingly, the only observable self-reference is a minor disclosed reuse of prior architecture, which is not load-bearing, so the circularity score is 1.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No physical entities are introduced. The paper's free parameters are evaluation-design thresholds rather than fitted model constants. The central claims depend on the manual evaluation assumptions and on the chosen model and prompts.

free parameters (3)
  • Max verification-finalizer loop iterations = 2
    Chosen by hand; caps how many times the finalizer can fix code before returning to the task pipeline, so it directly bounds solution quality.
  • Failed generation error threshold = more than 7 different errors
    A generation with more than 7 error types or no code is counted as failed and excluded from averages; this threshold is arbitrary and affects reported means.
  • Baseline prompt selection per file = varies by file
    The authors picked the better-looking prompt type per file after initial runs, so the comparison baseline is not fixed in advance.
assumptions (3)
  • domain assumption Manual error classification and requirement checking are reliable and consistent metrics for upgrade quality.
    The whole evaluation rests on hand-counted error categories and binary requirement passes in Section 3.4; no inter-rater reliability or automated oracle is provided.
  • domain assumption The five view files and the CakePHP 1.2 to 4.5 update task are representative enough to draw conclusions about legacy web application upgrades.
    The paper acknowledges in Section 5.1 that results cannot be generalized, yet the central narrative generalizes to legacy upgrades.
  • domain assumption The verification agent's judgment can detect incomplete or incorrect task completion without executing a test suite.
    The verification and finalizer agents rely on LLM judgment rather than runtime tests; Section 3.2 describes this design.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Autonomous Legacy Web Application Upgrades Using a Multi-Agent System." pith.science (2026). https://pith.science/paper/KNIPHBX2

@misc{pith2026250119204,
  author       = {Pith},
  title        = {Pith review of: Autonomous Legacy Web Application Upgrades Using a Multi-Agent System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KNIPHBX2}},
  note         = {Machine review of arXiv:2501.19204}
}
read the original abstract

The use of Large Language Models (LLMs) for autonomous code generation is gaining attention in emerging technologies. As LLM capabilities expand, they offer new possibilities such as code refactoring, security enhancements, and legacy application upgrades. Many outdated web applications pose security and reliability challenges, yet companies continue using them due to the complexity and cost of upgrades. To address this, we propose an LLM-based multi-agent system that autonomously upgrades legacy web applications to the latest versions. The system distributes tasks across multiple phases, updating all relevant files. To evaluate its effectiveness, we employed Zero-Shot Learning (ZSL) and One-Shot Learning (OSL) prompts, applying identical instructions in both cases. The evaluation involved updating view files and measuring the number and types of errors in the output. For complex tasks, we counted the successfully met requirements. The experiments compared the proposed system with standalone LLM execution, repeated multiple times to account for stochastic behavior. Results indicate that our system maintains context across tasks and agents, improving solution quality over the base model in some cases. This study provides a foundation for future model implementations in legacy code updates. Additionally, findings highlight LLMs' ability to update small outdated files with high precision, even with basic prompts. The source code is publicly available on GitHub: https://github.com/alasalm1/Multi-agent-pipeline.

Figures

Figures reproduced from arXiv: 2501.19204 by the authors.

Figure 1
Figure 1. Proposed system: Multi-agent pipeline for updating existing code [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Found error types by method and file 5 Discussion During this study files belonging to an existing depre￾cated web application were updated using GAI. The results shows that multi agent systems are capable of updating small deprecated web application files with a high precision with low rates of error and a low mean deviation of errors between the generations. The multi agent system could provide completely working … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 28 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    Ali, M., Hussain, S., Ashraf, M., and Paracha, K. (2020). Addressing software related issues on legacy systems -a review. International Journal of Scientific & Technology Research , 9:3738--3742

    work page 2020

  3. [3]

    Antal, G., Havas, D., Siket, I., Besz \'e des, \'A ., Ferenc, R., and Mihalicza, J. (2016). Transforming c++ 11 code to c++ 03 to support legacy compilation environments. In 2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM) , pages 177--186. IEEE

    work page 2016

  4. [4]

    B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., andGretchen Krueger, A

    Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., andGretchen Krueger, A. H., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A.,...

    arXiv 2020

  5. [5]

    Introduction to CakePHP

    CakePHP (2022). Introduction to CakePHP . https://book.cakephp.org/1.1/en/introduction-to-cakephp.html. Accessed: Sep.19, 2024

    work page 2022

  6. [6]

    CakePHP 4.5.0 Released

    CakePHP (2023). CakePHP 4.5.0 Released . https://bakery.cakephp.org/2023/10/14/cakephp_450.html. Accessed: Oct.14, 2024

    work page 2023

  7. [7]

    3.0 migration guide

    CakePHP (2024a). 3.0 migration guide. https://book.cakephp.org/3/en/appendices/3-0-migration-guide.html. Accessed: Nov.30, 2024

    work page 2024

  8. [8]

    Installation

    CakePHP (2024b). Installation. https://book.cakephp.org/4/en/installation.html. Accessed: Nov.3, 2024

    work page 2024

Show all 45 references
  1. [9]

    Retrieving Data & Results Sets

    CakePHP (2024c). Retrieving Data & Results Sets . https://book.cakephp.org/3/en/orm/retrieving-data-and-resultsets.html. Accessed: Sep.27, 2024

    2024

  2. [10]

    J., Yao, Z., and Neamtiu, I

    Chong, C. J., Yao, Z., and Neamtiu, I. (2024). Artificial-intelligence generated code considered harmful: A road map for secure and high-quality code generation. arXiv preprint arXiv:2409.19182

    2024 arXiv

  3. [11]

    97 things every project manager should know: collective wisdom from the experts

    Davis, B., editor (2009). 97 things every project manager should know: collective wisdom from the experts . O'Reilly, 1. aufl edition

    2009

  4. [12]

    De Marco, A., Iancu, V., and Asinofsky, I. (2018). Cobol to java and newspapers still get delivered. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME) , pages 583--586. IEEE

    2018

  5. [13]

    Demir, N., Urban, T., Wittek, K., and Pohlmann, N. (2021). Our (in)secure web: Understanding update behavior of websites and its impact on security. In Hohlfeld, O., Lutu, A., and Levin, D., editors, Passive and Active Measurement , volume 12671, pages 76--92. Springer Interna...

    2021

  6. [14]

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Pre-training of deep bidirectional transformers for language understanding. arxiv. arXiv preprint arXiv:1810.04805

    2018 arXiv

  7. [15]

    Dou, S., Jia, H., Wu, S., Zheng, H., Zhou, W., Wu, M., Chai, M., Fan, J., Huang, C., Tao, Y., et al. (2024). What's wrong with your code generated by large language models? an extensive study. arXiv preprint arXiv:2407.06153

    2024

  8. [16]

    F., Zhang, H., David, C., Wang, M., Christakis, M., Paulsen, B., Dodds, J., and Kroening, D

    Eniser, H. F., Zhang, H., David, C., Wang, M., Christakis, M., Paulsen, B., Dodds, J., and Kroening, D. (2024). Towards translating real-world code with llms: A study of translating to rust. arXiv preprint arXiv:2405.11514

    2024 arXiv

  9. [17]

    Erlikh, L. (2000). Leveraging legacy system dollars for e-business. IT Professional , 2(3):17--23

    2000

  10. [18]

    Fritzsch, J., Bogner, J., Wagner, S., and Zimmermann, A. (2019). Microservices migration in industry: Intentions, strategies, and challenges. In 2019 IEEE International Conference on Software Maintenance and Evolution ( ICSME ) , pages 481--490

    2019

  11. [19]

    M., Luck, M., and Cui, H

    Huang, D., Bu, Q., Zhang, J. M., Luck, M., and Cui, H. (2023). Agentcoder: Multi-agent-based code generation with iterative testing and optimisation. arXiv preprint arXiv:2312.13010

    2023 arXiv

  12. [20]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    2020 arXiv

  13. [21]

    Koschuetzki, T. (2008). Extra hot: CakePHP 1.2 stable is finally released! http://debuggable.com/posts/extra-hot-cakephp-1.2-stable-is-finally-released!:4954151c-f87c-434b-abbd-4e404834cda3. Accessed: Oct.30, 2024

    2008

  14. [22]

    F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P

    Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. (2024a). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics , 12:157--173

    2024

  15. [23]

    D., and Lo, D

    Liu, Y., Le-Cong, T., Widyasari, R., Tantithamthavorn, C., Li, L., Le, X.-B. D., and Lo, D. (2024b). Refining chatgpt-generated code: Characterizing and mitigating code quality issues. ACM Transactions on Software Engineering and Methodology , 33(5):1--26

    2024

  16. [24]

    Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2024). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems , 36

    2024

  17. [25]

    Norri, J., Junkkari, M., and Poranen, T. (2020). Digitization of data for a historical medical dictionary. Language Resources and Evaluation , 54(3):615--643

    2020

  18. [26]

    GPT -4o mini: advancing cost-efficient intelligence

    OpenAI (2024a). GPT -4o mini: advancing cost-efficient intelligence. https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/. Accessed: Sep.3, 2024

    2024

  19. [27]

    Introducing OpenAI o1-preview

    OpenAI (2024b). Introducing OpenAI o1-preview. https://openai.com/index/introducing-openai-o1-preview/. Accessed: Sep.19, 2024

    2024

  20. [28]

    C., Kabor \'e , K., Tian, H., Song, Y., Koyuncu, A., Klein, J., Lo, D., and Bissyand \'e , T

    Ou \'e draogo, W. C., Kabor \'e , K., Tian, H., Song, Y., Koyuncu, A., Klein, J., Lo, D., and Bissyand \'e , T. F. (2024). Large-scale, independent and comprehensive study of the power of llms for test case generation. arXiv preprint arXiv:2407.00225

    2024

  21. [29]

    R., Krishna, R., Sankar, D., Wassi, L

    Pan, R., Ibrahimzada, A. R., Krishna, R., Sankar, D., Wassi, L. P., Merler, M., Sobolev, B., Pavuluri, R., Sinha, S., and Jabbarvand, R. (2024). Lost in translation: A study of bugs introduced by large language models while translating code. In Proceedings of the IEEE / ACM 46...

    2024

  22. [30]

    and Narasimhan, K

    Radford, A. and Narasimhan, K. (2018). Improving language understanding by generative pre-training

    2018

  23. [31]

    A., Kemell, K.-K., Waseem, M., Saari, M., Syst \"a , K., and Abrahamsson, P

    Rasheed, Z., Sami, M. A., Kemell, K.-K., Waseem, M., Saari, M., Syst \"a , K., and Abrahamsson, P. (2024a). Codepori: Large-scale system for autonomous software development using multi-agent technology. arXiv preprint arXiv:2402.01411

    2024 arXiv

  24. [32]

    A., Rasku, J., Kemell, K.-K., Zhang, Z., Harjamaki, J., Siddeeq, S., Lahti, S., Herda, T., Nurminen, M., et al

    Rasheed, Z., Sami, M. A., Rasku, J., Kemell, K.-K., Zhang, Z., Harjamaki, J., Siddeeq, S., Lahti, S., Herda, T., Nurminen, M., et al. (2024b). Timeless: A vision for the next generation of software development. arXiv preprint arXiv:2411.08507

    2024 arXiv

  25. [33]

    A., Waseem, M., Kemell, K.-K., Wang, X., Nguyen, A., Syst \"a , K., and Abrahamsson, P

    Rasheed, Z., Sami, M. A., Waseem, M., Kemell, K.-K., Wang, X., Nguyen, A., Syst \"a , K., and Abrahamsson, P. (2024c). Ai-powered code review with llms: Early results. arXiv preprint arXiv:2404.18496

    2024

  26. [34]

    N., Syst \"a , K., and Abrahamsson, P

    Rasheed, Z., Waseem, M., Kemell, K.-K., Xiaofeng, W., Duc, A. N., Syst \"a , K., and Abrahamsson, P. (2023). Autonomous agents in software development: A vision paper. arXiv preprint arXiv:2311.18440

    2023 arXiv

  27. [35]

    Rasheed, Z., Waseem, M., Syst \"a , K., and Abrahamsson, P. (2024d). Large language model evaluation via multi AI agents: Preliminary results. In ICLR 2024 Workshop on Large Language Model (LLM) Agents

    2024

  28. [36]

    A., Waseem, M., Zhang, Z., Rasheed, Z., Syst \"a , K., and Abrahamsson, P

    Sami, M. A., Waseem, M., Zhang, Z., Rasheed, Z., Syst \"a , K., and Abrahamsson, P. (2024). Early results of an ai multiagent system for requirements elicitation and analysis. In International Conference on Product-Focused Software Process Improvement , pages 307--316. Springer

    2024

  29. [37]

    a , K., and Abrahamsson, P. (2025). Early results of an ai multiagent system for requirements elicitation and analysis. In Pfahl, D., Gonzalez Huerta, J., Kl \

    Sami, M. A., Waseem, M., Zhang, Z., Rasheed, Z., Syst \"a , K., and Abrahamsson, P. (2025). Early results of an ai multiagent system for requirements elicitation and analysis. In Pfahl, D., Gonzalez Huerta, J., Kl \"u nder, J., and Anwar, H., editors, Product-Focused Software ...

    2025

  30. [38]

    Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. (2024). Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems , 36

    2024

  31. [39]

    Smyth, S. (2023). Penetration testing and legacy systems. arXiv preprint arXiv:2402.10217

    2023 arXiv

  32. [40]

    Sommerville, I. (2016). Software engineering . Always learning. Pearson, tenth edition edition

    2016

  33. [41]

    University, T. (2024). Autonomous legacy web application upgrades using a multi-agent system

    2024

  34. [42]

    Vaswani, A. (2017). Attention is all you need. Advances in Neural Information Processing Systems

    2017

  35. [43]

    and Laković, D

    Vesić, S. and Laković, D. (2023). A framework for evaluating legacy systems – a case study. Kultura polisa , 20(1):32--50

    2023

  36. [44]

    Y., Hartmann, B., and Yang, Q

    Zamfirescu-Pereira, J., Wong, R. Y., Hartmann, B., and Yang, Q. (2023). Why johnny can’t prompt: How non- AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , pages 1--21. ACM

    2023

  37. [45]

    Zhong, L., Wang, Z., and Shang, J. (2024). Ldb: A large language model debugger via verifying runtime execution step-by-step. arXiv preprint arXiv:2402.16906

    2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.