Leveraging LLMs for Automated Translation of Legacy Code: A Case Study on PL/SQL to Java Transformation

Alper Tuncay; Andrea Capiluppi; Eduardo Carneiro Oliveira; Lola Solovyeva; Shamil Gareev; Shiyu Fan

REVIEW 4 major objections 4 minor 26 references

Leveraging LLMs for Automated Translation of Legacy Code: A Case Study on PL/SQL to Java Transformation

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a chain-of-guidance, n-shot prompting strategy with cosine-similarity-selected examples steers LLMs toward syntactically and functionally correct PL/SQL-to-Java translations.

desk verdict An honest industrial feasibility study whose central functional-correctness claim is undercut by manual de-anonymization edits made before scoring; worth reviewing as a cautionary case study, but not citable for its numbers. read the letter →

arxiv 2508.19663 v1 pith:JKOEE6N4 submitted 2025-08-27 cs.SE

Lola Solovyeva , Eduardo Carneiro Oliveira , Shiyu Fan , Alper Tuncay , Shamil Gareev , Andrea Capiluppi This is my paper

classification cs.SE

keywords LLMcodetranslationPL/SQLJavalegacymodernizationfew-shotpromptingchain-of-guidancecosinesimilarity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that large language models, steered by the right prompt, can produce Java code from legacy PL/SQL that is both syntactically sound and functionally correct enough to serve as a foundation for manual modernization. It proposes a pipeline that combines chain-of-guidance reasoning, n-shot prompting, and a Java domain model, and it tests that pipeline on ten real PL/SQL files from a 2.5-million-line undocumented system. Two prompting regimes are compared: giving the model all nine available translation examples versus giving it only the examples whose cosine similarity to the target file is highest. The authors report that the similarity-selected subset raises the share of passing test cases even though it sometimes preserves less code, and they conclude that example similarity matters more than example count. If the result holds, LLM-assisted modernization of large legacy systems becomes a credible, scalable workflow rather than a manual multi-year effort.

What carries the argument

The central mechanism is a three-part prompt that combines chain-of-guidance reasoning (asking the model to work through a sequence of guidance steps before answering) with n-shot examples: first the relevant Java domain classes are shown, then PL/SQL-to-Java example pairs, then the query file. A cosine-similarity function selects which pairs to include, and the paper's key move is comparing all-samples prompting to similarity-selected subset prompting. This pipeline carries the argument because every result in the study is attributed to the interaction of these components.

What would settle it

Re-run the ten translations without the manual de-anonymization edits: take the raw LLM output, compile it against the company's Java environment, and execute the prewritten tests. If the raw files pass tests at materially lower rates than the reported TP values, the effectiveness claim describes human-assisted output rather than the LLM itself; if they pass at similar rates, the claim survives.

Watch

Extended reading notes

Core claim

At the center of the paper is the claim that a purpose-built prompting pipeline can make LLM-generated PL/SQL-to-Java translations usable as a starting point for modernizing a 2.5-million-line legacy system. The pipeline first presents the Java domain model, then a small set of PL/SQL-to-Java pairs, then the single file to translate; the pairs are chosen by cosine similarity to the query file rather than by maximizing their count. On ten real files from the VT system, the authors report that this similarity-selected prompting achieved higher functional correctness—measured by the percentage of company test cases passed—than feeding all available examples, while the all-examples condition ten

Load-bearing premise

The reported success rates assume that the code-preservation and test-pass percentages reflect the LLM's raw translation, even though the generated files were manually adjusted during de-anonymization before being measured.

Editorial extensions

If this is right

On the VT system, a practical rollout would use the similarity-selection prompt to generate first-pass Java for each file, then route only the lowest-similarity files to human translators.
The same non-language-specific pipeline can be adapted to other legacy pairs, such as COBOL-to-Java or Fortran-to-modern Fortran, by swapping the domain model and example pairs.
For functional correctness, the paper's evidence implies that prompt-engineering effort should go into example selection rather than into maximizing the number of few-shot examples.
For larger datasets, the paper recommends AST-based similarity and other structural metrics instead of text-based cosine similarity, which would likely sharpen the selection effect.
The evaluation metrics CP and TP, combined into SR, give practitioners a single number to track when comparing prompt variants.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The similarity-over-quantity principle likely extends beyond translation: few-shot example selection by input similarity may also improve LLM performance in code repair, test generation, and summarization, where the same 'more examples is better' assumption is common.
A controlled replication that holds the number of examples fixed—random three versus similar three—would separate the similarity effect from the effect of a shorter prompt, and would test whether the mechanism is really similarity or simply less context noise.
For companies tackling legacy modernization, the paper's pipeline implies a triage workflow: compute similarity scores for all files, batch the high-similarity ones for fully automated translation, and reserve human review for the low-similarity tail.
If AST-based similarity reproduces or improves the reported gains, it would confirm that the active ingredient is structural resemblance between code samples, not surface token overlap.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a sincere industrial case study with a sensible idea buried under a measurement problem. The subset-vs-all similarity comparison is the most interesting thing here, but the CP/TP scores were computed only after a de-anonymization step that required manual adjustments to the generated files, so the headline claim about 'functional correctness' is not actually isolated from human repair.

What's genuinely useful: the paper tackles a real pain point — 2.5M lines of undocumented PL/SQL — with an actual company dataset. The domain model as a prompt constraint and the use of cosine similarity to pick few-shot examples are pragmatic extensions of existing techniques rather than new methods, but they are applied in a context where there's little published evidence. The paper is also honest about its limitations: small dataset, LLM nondeterminism, text-based similarity.

The soft spots are real and load-bearing. The de-anonymization step is described in a single sentence, and there is no quantification of what those manual adjustments changed, no raw pre-adjustment scores, and no comparison of TP on raw vs adjusted files. If the adjustments included fixing references or completing stubs, the measured 'functional correctness' is a human-assisted product. The stress-test note is right. On top of that: ten files is too few for the similarity claim; the optimal subset was selected by scanning all 502 combinations on the same ten files used for evaluation, so the selection rule is not independent of the test set; and there are no repeated runs despite acknowledged nondeterminism. The result in Table 1 (4 improved, 2 worsened) could easily flip with different seeds.

That said, I don't think this is a dishonest paper. The limitations section acknowledges the sample size and nondeterminism; the authors just don't connect those caveats to the strength of the abstract claim. The missing piece is evaluation hygiene, not intent.

Who should read it: people working on LLM-based migration or on evaluation of code generation in industrial settings. It works better as a cautionary case study than as a validated result. I would not cite its quantitative numbers. I would send it to peer review, because the practical problem is important, the dataset is real, and the flaws are fixable with a larger study and transparent reporting of the manual edit step. Just don't let the current version through unchanged.

Referee Report

4 major / 4 minor

Summary. The paper reports a case study on using LLMs to translate PL/SQL code to Java in the VT legacy system (2.5M lines) for the VTF3 modernization. The authors propose a prompting methodology that combines chain-of-guidance reasoning with n-shot prompting, and they introduce a cosine-similarity-based selection of example pairs to provide to the LLM. Ten PL/SQL-to-Java pairs and a Java domain model are used. The evaluation reports two metrics: percentage of code preserved (CP) and percentage of test cases passed (TP), combined into a Success Rate (SR). The main claims are that the methodology effectively guides LLMs toward syntactically accurate and functionally correct translations, and that the similarity of the selected samples matters more than their number.

Significance. If the central claims were adequately supported, the contribution would be practically relevant: it would demonstrate a reproducible prompt-level methodology for an industrial legacy translation task and suggest a low-cost retrieval heuristic (cosine similarity) for example selection. The use of an external, company-owned test suite is a strength, and the paper identifies a realistic deployment context. However, the evidence base is small (10 files, mostly one run per condition), and several load-bearing evaluation steps are insufficiently controlled or insufficiently reported. The claims are therefore not yet established at the level the abstract and conclusion state.

major comments (4)

[Evaluation] The CP and TP metrics are computed after a manual de-anonymization step that "required manual adjustments to certain parts of the file." The paper does not quantify these adjustments, does not report raw-before-adjustment outputs, and does not compare TP on raw vs. adjusted files. Consequently, the reported functional correctness may reflect a human-assisted product rather than the LLM's raw translation, which directly undermines the central claim that the prompting methodology achieves functional correctness. This is load-bearing: the paper must either report separate metrics for raw and adjusted outputs, or provide a diff-level justification that the adjustments were purely cosmetic and did not affect semantics.
[Results / Table 1] The comparison between "all samples" and "most similar subset" is confounded: the two conditions differ both in the number of provided examples and in their similarity to the query, so a change in SR cannot be attributed to similarity alone. Moreover, only four of ten files improved in the subset condition, two worsened (α2, α6), and the rest were unchanged or showed no improvement; this is not strong statistical evidence for the claim that "effectiveness is determined not by the number of samples but by their similarity." Given acknowledged LLM nondeterminism, the absence of repeated runs or any statistical test (e.g., a paired test or effect size) makes the claim fragile. The paper should present per-file raw data, repeated runs, and a controlled experiment where subset size is varied independently of similarity.
[Methodology / Prompt refinement] The contribution is a composite methodology (domain model + chain-of-guidance + n-shot examples + similarity-based selection), but the design does not include ablations isolating the components. The claim that "incorporating a domain model ... and using a targeted set of translation examples leads to more effective outcomes" cannot be disentangled from the effect of the prompt structure or the specific LLM (DeepSeek-R1). At minimum, the authors should compare the proposed prompt against a simpler baseline (e.g., plain n-shot without domain model and without chain-of-guidance) and against random example selection of the same subset size, so that the effect of similarity is not confounded by other prompt variations.
[Evaluation / Metrics] The definitions of CP and TP are not precise enough to interpret the reported numbers. For CP, it is unclear whether preservation is measured at the character, token, or statement level, and how manual de-anonymization edits affect it. For TP, the number of test cases per file and the distribution of passed tests are not reported; SR = CP×TP/100 is an ad hoc combination whose behavioral meaning is unclear (e.g., a file with CP=100% and TP=0% gets the same SR as CP=0% and TP=0%). The paper should define these metrics operationally and report TP and CP separately for each file, together with the test counts.

minor comments (4)

[Methodology / File similarity] The text says "We systematically evaluated all 502 possible combinations of the 10 PL/SQL code examples," but the number of non-empty subsets of 9 other files is 511, and the paper does not explain how 502 is obtained or what exactly is excluded. This should be clarified.
[Figures 2 and 3] Figures 2 and 3 lack axis labels and legends in the text. In particular, it is not clear what the x- and y-axes represent in Figure 3, making it impossible to read the claimed tradeoff between code preservation and test pass rate.
[Table 1] The Result row appears to have fewer arrows than there are columns (the table shows symbols for only six of the ten files). This formatting issue should be fixed so that each column clearly indicates whether the subset condition improved, degraded, or tied.
[General] The abstract and conclusion state that the methodology "effectively guides LLMs" and "produced in majority cases both syntactically correct, and functionally aligned translations." The body reports only four improvements out of ten in the subset condition, so the wording in the abstract is stronger than the evidence. Please temper the claims to match the experimental scope, or expand the experiments.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: success is measured against external test cases and the similarity selector is a retrieval rule, not a fitted predictor; de-anonymization edits are a validity risk, not a circular derivation.

full rationale

The paper's only formal quantity is SR = CP × TP / 100 (Eq. 1). CP (code preserved) and TP (test cases passed) are defined relative to external referents — the VTF3 target code and prewritten company test cases — not relative to the few-shot examples, the domain model, or the cosine-similarity selector. The similarity-based selection chooses prompt examples by cosine similarity to the query PL/SQL source; this is a fixed retrieval rule, and the outcome (TP) is measured on independent test cases, so no fitted parameter is renamed as a prediction. The chain-of-guidance prompting strategy was proposed by DeepSeek-R1 and then evaluated on DeepSeek-R1, which is mildly self-referential, but the evaluation still uses external test cases, so the central claim is not forced by construction. There are no load-bearing self-citations; [19] and [23] are external prior work. The notable threat is validity, not circularity: the Evaluation section states that 'due to an initial data anonymization, the generated files underwent a de-anonymization process, which required manual adjustments to certain parts of the file' before CP and TP were computed. If those adjustments were substantive, CP/TP may measure a human-assisted artifact rather than raw LLM output. This is an experimental confound that should lower confidence in the central claim; it is not a definitional equivalence or a fitted-input-as-prediction scheme. The Limitations section separately acknowledges the small sample size and non-deterministic LLM behavior, which affect generalizability but are likewise not circularity. Accordingly, no circular step is identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No new entities are postulated and no numeric parameters are fitted. The central claim rests on assumptions about the validity of the similarity metric, the faithfulness of the evaluation metrics after manual edits, and the representativeness of the small dataset.

assumptions (4)

domain assumption Cosine similarity over raw PL/SQL text captures code similarity relevant to translation quality.
Used in the File similarity analysis section to select examples; no evidence is given that textual cosine similarity correlates with semantic or functional similarity for PL/SQL.
domain assumption CP and TP metrics, measured after manual de-anonymization adjustments, reflect the quality of the LLM-generated translation.
The Evaluation section states that generated files underwent manual adjustments during de-anonymization; this assumption is load-bearing but not justified.
domain assumption The 10 PL/SQL-to-Java pairs and the company's test cases are representative of the 2.5 million line codebase.
The paper generalizes from 10 files to a large legacy system; no sampling strategy or representativeness argument is provided.
domain assumption The chain-of-guidance technique, as described in cited work [19], behaves as expected when combined with n-shot prompting for code translation.
The paper adopts chain-of-guidance without independent validation of its effect in this setting.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Leveraging LLMs for Automated Translation of Legacy Code: A Case Study on PL/SQL to Java Transformation." pith.science (2026). https://pith.science/paper/JKOEE6N4

@misc{pith2026250819663,
  author       = {Pith},
  title        = {Pith review of: Leveraging LLMs for Automated Translation of Legacy Code: A Case Study on PL/SQL to Java Transformation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JKOEE6N4}},
  note         = {Machine review of arXiv:2508.19663}
}

abstract

The VT legacy system, comprising approximately 2.5 million lines of PL/SQL code, lacks consistent documentation and automated tests, posing significant challenges for refactoring and modernisation. This study investigates the feasibility of leveraging large language models (LLMs) to assist in translating PL/SQL code into Java for the modernised "VTF3" system. By leveraging a dataset comprising 10 PL/SQL-to-Java code pairs and 15 Java classes, which collectively established a domain model for the translated files, multiple LLMs were evaluated. Furthermore, we propose a customized prompting strategy that integrates chain-of-guidance reasoning with $n$-shot prompting. Our findings indicate that this methodology effectively guides LLMs in generating syntactically accurate translations while also achieving functional correctness. However, the findings are limited by the small sample size of available code files and the restricted access to test cases used for validating the correctness of the generated code. Nevertheless, these findings lay the groundwork for scalable, automated solutions in modernising large legacy systems.

Figures

Figures reproduced from arXiv: 2508.19663 by the authors.

**Figure 2.** Highest similarity values (red) vs. using all ex [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 24 canonical work pages

[1]

Hu Jonathan, Ould-Ahmed-Vall ElMoustapha, and Kalinkin A

Gel A., J. Hu Jonathan, Ould-Ahmed-Vall ElMoustapha, and Kalinkin A. Mod- ernization and optimization of a legacy open-source CFD code for high- performance computing architectures. 2017

work page 2017
[2]

Au- tomated transpilation of imperative to functional code using neural-guided program synthesis

Mariano Benjamin, Chen Yanju, Feng Yu, Durrett Greg, and Dillig Işıl. Au- tomated transpilation of imperative to functional code using neural-guided program synthesis. Proc. ACM Program. Lang., 2022

work page 2022
[3]

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

1901
[4]

Charette

Robert N. Charette. Dragging Government Legacy Systems Out of the Shad- ows. Computer, 49(9):114–119, 9 2016

work page 2016
[5]

Viseval: A Benchmark for Data Visualization in the Era of Large Language Models

Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. Viseval: A Benchmark for Data Visualization in the Era of Large Language Models. IEEE Transactions on Visualization and Computer Graphics , 31(1):1301–1311, 1 2025

work page 2025
[6]

Stephen L. France. Navigating software development in the ChatGPT and GitHub Copilot era. Business Horizons, 67(5):649–661, 9 2024

work page 2024
[7]

The Current Chal- lenges of Software Engineering in the Era of Large Language Models

Cuiyun Gao, Xing Hu, Shan Gao, Xin Xia, and Zhi Jin. The Current Chal- lenges of Software Engineering in the Era of Large Language Models. ACM Transactions on Software Engineering and Methodology , jan 13 2025

work page 2025
[8]

Llm-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead

Junda He, Christoph Treude, and David Lo. Llm-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead. ACM Transactions on Software Engineering and Methodology , jan 13 2025

work page 2025

Show all 26 references

[9]

Gareth Hogan, Patricija Shalkauskaite, Mengte Zhu, Martin Derwin, Murat Yilmaz, Andrew McCarren, and Paul M. Clarke. Investigating systems mod- ernisation: Approaches, challenges and risks. In Murat Yilmaz, Paul Clarke, Andreas Riel, Richard Messnarz, Christian Greiner, and Th...
[10]

Deploying and Evaluating LLMs to Program Service Mobile Robots

Zichao Hu, Francesca Lucchetti, Claire Schlesinger, Yash Saxena, Anders Freeman, Sadanand Modak, Arjun Guha, and Joydeep Biswas. Deploying and Evaluating LLMs to Program Service Mobile Robots. IEEE Robotics and Automation Letters, 9(3):2853–2860, 3 2024

2024
[11]

hmcodetrans: Human–Machine Interactive Code Translation

Liu Jiaqi, Zhang Fengming, Zhang Xin, Yu Zhiwen, Wang Liang, Zhang Yao, and Guo Bin. hmcodetrans: Human–Machine Interactive Code Translation. IEEE Transactions on Software Engineering , 2024

2024
[12]

Udo-Imeh, Bonan Kou, and Tianyi Zhang

Samia Kabir, David N. Udo-Imeh, Bonan Kou, and Tianyi Zhang. Is stack overflow obsolete? an empirical study of the characteristics of chatgpt answers to stack overflow questions. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems , CHI ’24, New Yor...

2024
[13]

Evaluating diverse large language models for automatic and general bug reproduction

Sungmin Kang, Juyeon Yoon, Nargiz Askarbekkyzy, and Shin Yoo. Evaluating diverse large language models for automatic and general bug reproduction. IEEE Transactions on Software Engineering , 2024

2024
[14]

Using Microservices for Legacy Software Modernization

Holger Knoche and Wilhelm Hasselbring. Using Microservices for Legacy Software Modernization. IEEE Software, 35(3):44–49, 5 2018

2018
[15]

Using ChatGPT in Software Requirements Engineering: A Comprehensive Review

Nuno Marques, Rodrigo Rocha Silva, and Jorge Bernardino. Using ChatGPT in Software Requirements Engineering: A Comprehensive Review. Future Internet, 16(6):180, may 21 2024

2024
[16]

State of What Art? A Call for Multi-Prompt LLM Eval- uation

Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. State of What Art? A Call for Multi-Prompt LLM Eval- uation. Trans. of the Association for Computational Linguistics , 12:933–949, 2024

2024
[17]

Government Accountability Office

U.S. Government Accountability Office. Information technology: Agencies need to develop modernization plans for critical legacy systems, Aug 2019

2019
[18]

Hassan, and Bram Adams

Harsh Patel, Dominique Boucher, Emad Fallahzadeh, Ahmed E. Hassan, and Bram Adams. A State-of-the-Practice Release-Readiness Checklist for Gener- ative AI-Based Software Products: A Gray Literature Survey. IEEE Software, 42(1):74–83, 1 2025

2025
[19]

Improv- ing consistency in large language models through chain of guidance

Harsh Raj, Vipul Gupta, Domenic Rosati, and Subhabrata Majumdar. Improv- ing consistency in large language models through chain of guidance. arXiv preprint arXiv:2502.15924, 2025

2025 arXiv
[20]

Methodology for Code Syn- thesis Evaluation of LLMs Presented by a Case Study of ChatGPT and Copilot

Zoltán Ságodi, István Siket, and Rudolf Ferenc. Methodology for Code Syn- thesis Evaluation of LLMs Presented by a Case Study of ChatGPT and Copilot. IEEE Access, 12:72303–72316, 2024

2024
[21]

Software Testing With Large Language Models: Survey, Landscape, and Vision

Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. Software Testing With Large Language Models: Survey, Landscape, and Vision. IEEE Transactions on Software Engineering , 50(4):911–936, 4 2024

2024
[22]

Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review

Man-Fai Wong, Shangxin Guo, Ching-Nam Hang, Siu-Wai Ho, and Chee- Wei Tan. Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review. Entropy, 25(6):888, jun 1 2023

2023
[23]

Learning similarity with cosine similarity ensemble

Peipei Xia, Li Zhang, and Fanzhang Li. Learning similarity with cosine similarity ensemble. Information Sciences, 307:39–52, 2015

2015
[24]

Framework for evaluating code generation ability of large language models

Sangyeop Yeo, YuSeung Ma, Sang Cheol Kim, Hyungkook Jun, and Taeho Kim. Framework for evaluating code generation ability of large language models. ETRI Journal, 46(1):106–117, 2 2024

2024
[25]

Zhang, F

T. Zhang, F. Ladhak, E. Durmus, P. Liang, K. McKeown, and T. B. Hashimoto. Benchmarking Large Language Models for News Summarization. Trans. of the Association for Computational Linguistics , 12:39–57, 2024

2024
[2024]

Springer Nature Switzerland

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Hu Jonathan, Ould-Ahmed-Vall ElMoustapha, and Kalinkin A

Gel A., J. Hu Jonathan, Ould-Ahmed-Vall ElMoustapha, and Kalinkin A. Mod- ernization and optimization of a legacy open-source CFD code for high- performance computing architectures. 2017

work page 2017

[2] [2]

Au- tomated transpilation of imperative to functional code using neural-guided program synthesis

Mariano Benjamin, Chen Yanju, Feng Yu, Durrett Greg, and Dillig Işıl. Au- tomated transpilation of imperative to functional code using neural-guided program synthesis. Proc. ACM Program. Lang., 2022

work page 2022

[3] [3]

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

1901

[4] [4]

Charette

Robert N. Charette. Dragging Government Legacy Systems Out of the Shad- ows. Computer, 49(9):114–119, 9 2016

work page 2016

[5] [5]

Viseval: A Benchmark for Data Visualization in the Era of Large Language Models

Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. Viseval: A Benchmark for Data Visualization in the Era of Large Language Models. IEEE Transactions on Visualization and Computer Graphics , 31(1):1301–1311, 1 2025

work page 2025

[6] [6]

Stephen L. France. Navigating software development in the ChatGPT and GitHub Copilot era. Business Horizons, 67(5):649–661, 9 2024

work page 2024

[7] [7]

The Current Chal- lenges of Software Engineering in the Era of Large Language Models

Cuiyun Gao, Xing Hu, Shan Gao, Xin Xia, and Zhi Jin. The Current Chal- lenges of Software Engineering in the Era of Large Language Models. ACM Transactions on Software Engineering and Methodology , jan 13 2025

work page 2025

[8] [8]

Llm-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead

Junda He, Christoph Treude, and David Lo. Llm-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead. ACM Transactions on Software Engineering and Methodology , jan 13 2025

work page 2025