Pith. sign in

REVIEW 4 major objections 7 minor 45 references

An Exploration of Agentic Information Fusion for Test Maintenance Prediction

T0 review · 4 major / 7 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A multi-agent system fuses static, lexical, and semantic analyses to predict which tests need maintenance after production code changes, beating a semantic-only baseline on industrial Java repositories.

desk verdict Solid industrial multi-agent fusion for test localization: real precision gains over a semantic baseline on Ericsson data, with the co-evolution ground-truth rule as the main soft spot. read the letter →

arxiv 2607.04786 v1 pith:6ED4MP52 submitted 2026-07-06 cs.SE cs.AIcs.MA

Jingxiong Liu , Nasser Mohammadiha , Gregory Gay This is my paper
classification cs.SEcs.AIcs.MA
keywords testmaintenancelocalizationmulti-agentsystemslargelanguagemodelsinformationfusionstaticanalysislexicalsemantic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

When production code changes, developers must find which tests to update or delete—a costly first step called test localization. This paper presents MAST, a multi-agent framework that takes a standard git diff and recommends the affected tests. It runs three analyses in parallel (call-graph static analysis, BM25 lexical similarity, and RAG-based semantic summaries of tests and the change), has an LLM fuse the candidate lists, then runs a post-check that compares each candidate’s code to the change to drop false positives. On 21 industrial Java repositories, MAST raises precision, accuracy, F1, and F2 over a semantic-only baseline while cutting false positives sharply on commits that needed no maintenance, at some cost in recall. An ablation shows each source and the post-check contribute. The work argues that agentic fusion of complementary code analyses can make realistic, repository-level test localization practical without assuming a pre-existing production–test map.

What carries the argument

MAST’s agentic pipeline: a maintenance-prediction agent, three parallel analyses (static call-graph, BM25 lexical, RAG semantic), an LLM fusion agent that merges candidates without predetermined source weights, and a post-check agent that re-examines each fused candidate against the git-diff change.

What would settle it

Re-label a substantial sample of the industrial commits by independently determining which tests truly required updates (including later commits and developer interviews); if MAST’s precision advantage and F1/F2 gains disappear under that re-labeling, the central performance claim fails.

Watch

Extended reading notes

Core claim

On 21 industrial Java repositories, MAST’s multi-source fusion plus post-check yields substantially higher precision (0.621 vs 0.367 overall on positive commits), accuracy, F1, and F2 than a semantic-only multi-agent baseline, with only partial loss of recall, and far fewer false positives on commits where no test maintenance occurred. Each analysis contributes distinct true positives; fusion raises recall; the post-check prunes false positives.

Load-bearing premise

The evaluation treats tests changed in the same commit as the production change as exactly the tests that needed maintenance, and assumes developers never delayed or missed needed updates.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents MAST, a multi-agent LLM framework that predicts which test cases require maintenance after production-code changes (test localization). MAST takes git-diff chunks as input and combines three parallel analyses—static call-graph reachability, BM25 lexical similarity, and RAG-based semantic similarity of natural-language test summaries—then fuses the candidate lists with an LLM agent and applies a post-check agent that re-examines each candidate against the diff. The authors evaluate MAST on 21 industrial Java repositories from Ericsson AB, separately on positive commits (tests were updated) and negative commits (no test updates), against a semantic-only baseline drawn from their prior work, and via an ablation of each analysis, fusion, and post-check. On positive commits, MAST reports higher precision (0.621 vs 0.367), accuracy, F1, and F2 than the baseline at some cost in recall; on negative commits it substantially reduces false positives. An additional comparison of LLM-based versus heuristic fusion is included.

Significance. If the reported gains hold under more robust labeling, the work is a solid, practically oriented contribution to automated test maintenance. Strengths that should be credited include: (i) a realistic industrial evaluation setting (repository-level git diffs, no assumed production–test mapping, both positive and negative commits); (ii) a transparent multi-run protocol with means and standard deviations; (iii) a full ablation of information sources, fusion, and post-check, plus an LLM-vs-heuristic fusion comparison (Tables 3–4); and (iv) public release of the MAST implementation and experiment snapshot. The multi-source agentic fusion idea is transferable beyond test localization. The main limit on significance is that headline precision/F1 improvements are only as trustworthy as the co-evolution ground truth, which the paper itself flags as imperfect.

major comments (4)
  1. §4.1 and §6.3.2: Ground-truth construction is load-bearing for the central precision claim. Positives are tests modified in the same commit as production changes (after manual filtering of renames/lint/support-code/directory moves); every non-updated test is treated as a true negative. The paper correctly notes that delayed updates or unrelated co-commits would re-label TPs as FPs (or FNs as TNs). Because MAST’s gains are driven by post-check pruning of the fused list (Table 3: precision rises from 0.355 after fusion to 0.621 after post-check), systematic over-count of FPs in the baseline or under-count of true maintenance needs could shrink or reverse the headline deltas (Table 2 overall: 0.621 vs 0.367 precision; 0.617 vs 0.483 F1). No independent check—later-commit linkage, multi-commit window, or developer confirmation on a sample—is provided to bound this bias. Please add at least o
  2. §3.3.1–3.3.2: Similarity thresholds (FAISS >0.50 then mean+0.25·std; BM25 score >50.00, top-5) and the nine-line git-diff context window are stated as “chosen through experimentation” but no sensitivity or stability analysis is reported. These free parameters directly control candidate-set size for the fusion and post-check stages that produce the precision gains. Please report how precision/recall/F1 change under reasonable threshold perturbations (or justify the chosen values with a small grid on a held-out subset of commits) so readers can assess robustness of Table 2–3 results.
  3. §4.2 and §6.3.3: The only quantitative baseline is the authors’ own prior semantic-only pipeline. Related localization/repair systems (Hu et al. [13], Chi et al. [6]) are discussed and correctly noted as non-equivalent under MAST’s assumptions, but the manuscript still claims “superior … to a state-of-the-art baseline” without any adapted comparison or shared-metric re-implementation on even a subset of commits. At minimum, either (i) re-run a simplified version of one external method under MAST’s input constraints on a subset of projects, or (ii) soften the “state-of-the-art” wording and present the comparison strictly as an ablation against the authors’ prior semantic agent. The current framing overstates the competitive claim relative to the evidence.
  4. §4.3–4.4 and Tables 2–3: Results are aggregated by summing TP/FP/TN/FN across commits and projects, with three temperature-0 runs reported as mean±std, but no statistical significance tests (e.g., paired tests over commits or bootstrap CIs on the precision/F1 deltas) are provided. Given project imbalance (Table 1: commit counts range from 2 to 123 positives) and non-trivial run-to-run variance on some repositories, please add a commit-level or project-level significance assessment for the main MAST-vs-baseline deltas so that the overall improvements cannot be driven by a few large repositories alone.
minor comments (7)
  1. §1 and Abstract: “state-of-the-art baseline” should be qualified once the baseline is defined as the authors’ prior semantic pipeline (§4.2), to avoid implying a broad external SOTA comparison.
  2. Table 1: Repository names are anonymized (good), but the description column is useful; consider adding a column for average number of production methods changed per commit or average tests updated, to help interpret the 6.5-tests-updated figure in §6.2.
  3. §3.2 Listing 2 and §3.5 Listing 6: Example outputs mix Python-dict style with prose; a consistent JSON schema for agent outputs would aid reproducibility.
  4. §5.1: Three projects where the baseline has higher F1 are named; a short qualitative note on why recall collapsed for MAST on those projects (e.g., *-service-dataexport-download) would strengthen the discussion of the precision–recall trade-off.
  5. §6.3.1: External validity is limited to Ericsson Java projects; the planned open-source and multi-language extensions are welcome—please state them as concrete next steps rather than only future work.
  6. Typographical / consistency: “libray” in Table 1 (*-datastream-dump-common description); “prevision” in the RQ3 summary box (§5.3); “availability” → “available” in §8. Align “git diff” hyphenation throughout.
  7. Figure 2: The “If Maintenance Not Needed” branch is clear, but the figure caption could briefly note that the three analyses run only when the prediction agent returns needs_update=True.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: empirical multi-agent evaluation on industrial commits; baseline is prior semantic pipeline but metrics are not forced by definition or self-citation.

full rationale

MAST is an empirical software-engineering system paper. Its central claims are measured precision/recall/F1/F2/accuracy of a multi-agent fusion pipeline versus a semantic-only baseline on 21 Ericsson Java repositories (positive and negative commits). The baseline is extracted from the authors’ prior work [21] by taking the Maintenance Prediction + Semantic Analysis sub-workflow of MAST; this is ordinary self-comparison, not a load-bearing uniqueness theorem or a definition that forces the reported deltas. Thresholds (semantic similarity >0.50 then mean+0.25·std, BM25 >50, top-5) are chosen by experimentation and applied uniformly; they do not redefine the ground-truth labels or turn a fitted constant into a ‘prediction’ of the same quantity. Ground-truth construction (same-commit co-evolution after manual filtering of renames/lint/support-code moves) is an acknowledged internal-validity assumption (§6.3.2), not a circular reduction of the prediction to its inputs. Ablation and LLM-vs-heuristic fusion tables further show independent contribution of each analysis rather than tautological identity. No equation equates a claimed first-principles result to a fitted input by construction. Score 1 reflects only the minor, non-load-bearing self-citation of the baseline source.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

Load-bearing content is empirical: industrial commit labels, hand-chosen retrieval thresholds, and domain assumptions about co-evolution and analysis relevance. No new physical entities; free parameters are the experimental cutoffs that control candidate lists; axioms are standard SE/LLM practice plus the co-evolution ground-truth rule.

free parameters (4)
  • semantic FAISS similarity threshold and secondary filter = 0.50; mean+0.25*std
    Retrieve tests with similarity > 0.50 then keep those > mean + 0.25·std; values ‘chosen through experimentation’ and directly control semantic candidates.
  • BM25 lexical threshold and top-k = score>50.00, k≤5
    Retrieve up to five tests with BM25 score above 50.00; chosen experimentally and gates lexical candidates.
  • git-diff context window = 9 lines
    Nine unchanged lines above and below each change are included as context; fixed design choice affecting all agents.
  • LLM decoding temperature = 0.00
    Temperature set to 0.00 for stability; still three runs reported because residual variance remains.
assumptions (4)
  • domain assumption Tests modified in the same commit as a production change are the ground-truth set of tests that required maintenance; developers did not systematically miss needed updates.
    Stated as Internal Validity threat (§6.3.2); all TP/FP/FN counts rest on this labeling rule.
  • domain assumption Call-graph reachability, BM25 token similarity, and embedding similarity of NL summaries are each informative (if incomplete) signals of test–production relevance.
    Motivates the three parallel analyses (§3.3); without it fusion has nothing to merge.
  • domain assumption An LLM can usefully merge heterogeneous candidate lists and perform a code-level post-check without source-specific trust weights in the prompt.
    Core of fusion and post-check agents (§3.4–3.5); performance claims depend on this capability of the chosen model.
  • standard math Standard program-analysis and IR tools (tree-sitter call graphs, BM25, FAISS/bge-m3 embeddings) correctly extract the intended structures from Java/JUnit projects.
    Background tooling assumptions for static and retrieval components.
invented entities (1)
  • MAST multi-agent workflow (maintenance prediction, three analyses, fusion, post-check)
    purpose: End-to-end test localization from git diff without a pre-existing production–test map.
    The named system and agent graph are the paper’s primary artifact; independent evidence is the industrial evaluation, not an external physical measurement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Exploration of Agentic Information Fusion for Test Maintenance Prediction." pith.science (2026). https://pith.science/paper/6ED4MP52

@misc{pith2026260704786,
  author       = {Pith},
  title        = {Pith review of: An Exploration of Agentic Information Fusion for Test Maintenance Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ED4MP52}},
  note         = {Machine review of arXiv:2607.04786}
}
read the original abstract

Test maintenance is a critical, yet costly, activity - particularly as codebases rapidly evolve. To assist, we present MAST, a multi-agent framework that predicts which test cases require maintenance following changes to the production code. This identification task is necessary as a precondition to any subsequent maintenance activities, but remains challenging due to the complex relationships between production and test code. MAST advances the state-of-the-art by integrating multiple analyses -- including static, lexical, and semantic analyses - through an intelligent fusion and post-check procedure and by focusing on a realistic use and evaluation setting - i.e., standardized input formats, repository-level analyses, and the ability to infer relations between test and production artifacts rather than assuming a pre-existing mapping. We evaluated MAST on 21 industrial Java repositories from Ericsson AB, considering situations where test maintenance both was and was not required in the ground truth. MAST yielded superior precision to a state-of-the-art baseline - resulting in a higher accuracy, F1, and F2 score - with only some loss in recall. Our ablation study demonstrates the value of each analysis in producing the final recommendations. MAST illustrates the potential of multi-agent systems that can fuse multiple information sources when performing software testing tasks.

Figures

Figures reproduced from arXiv: 2607.04786 by the authors.

Figure 1
Figure 1. Input and output of the MAST framework. Regardless of the purpose of a test maintenance activity, one of the first steps in the process is to identify which existing test cases need to be modified or deleted—a task we refer to as “test localization”. In this study, we propose MAST (Multi-Agent Multi-Source Test Maintenance), a multi-agent framework that integrates multiple complementary analysis techniques to perfor… view at source ↗
Figure 2
Figure 2. Overview of MAST. Boxes colored green represent agents, while the ones colored blue are scripted components. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 2 canonical work pages

  1. [13]

    Junda He, Christoph Treude, and David Lo. 2025. LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead.ACM Trans. Softw. Eng. Methodol.(Jan. 2025). doi:10.1145/3712003

    doi:10.1145/3712003 2025

  2. [6]

    arXiv:2402.03216 [cs.CL]

    BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. arXiv:2402.03216 [cs.CL]

    arXiv

  3. [1]

    Emil Alégroth, Robert Feldt, and Pirjo Kolström. 2016. Maintenance of automated test suites in industry: An empirical study on Visual GUI Testing.Information and Software Technology73 (2016), 66–80

    2016

  4. [2]

    2022.Effective Software Testing: A developer’s guide

    Maurício Aniche. 2022.Effective Software Testing: A developer’s guide. Simon and Schuster

    2022

  5. [3]

    Max Brunsfeld. 2018. Tree-sitter: A parser generator tool and incremental parsing library. https://github.com/tree-sitter/tree-sitter. Accessed: 2026-04-10

    2018

  6. [4]

    Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S Yu, and Lichao Sun. 2023. A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt.arXiv preprint arXiv:2303.04226(2023)

    arXiv 2023

  7. [5]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

  8. [7]

    Jianlei Chi, Xiaotian Wang, Yuhan Huang, Lechen Yu, Di Cui, Jianguo Sun, and Jun Sun. 2025. REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models.Proc. ACM Softw. Eng.2, ISSTA, Article ISSTA055 (June 2025), 23 pages. doi:10.1145/3728930

    doi:10.1145/3728930 2025

Show all 45 references
  1. [8]

    Robert Feldt, Sungmin Kang, Juyeon Yoon, and Shin Yoo. 2023. Towards au- tonomous testing agents via conversational large language models. In2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1688–1693

    2023

  2. [9]

    Chaoyang Gao, Xiang Chen, and Guangbei Zhang. 2025. SVA-ICL: Improv- ing LLM-based software vulnerability assessment via in-context learning and information fusion.Information and Software Technology186 (2025), 107803. doi:10.1016/j.infsof.2025.107803

    2025 doi

  3. [10]

    Raji Ghawi and Jürgen Pfeffer. 2019. Efficient hyperparameter tuning with grid search for text categorization using kNN approach with BM25 similarity.Open Computer Science9, 1 (2019), 160–180

    2019

  4. [11]

    Danielle Gonzalez, Joanna CS Santos, Andrew Popovich, Mehdi Mirakhorli, and Mei Nagappan. 2017. A large-scale study on the usage of testing patterns that address maintainability attributes: patterns for ease of modification, diagnoses, and comprehension. In2017 IEEE/ACM 14th I...

    2017

  5. [12]

    Piyush Gupta, Sangjae Bae, and David Isele. 2025. Graph-Grounded LLMs: Leveraging Graphical Function Calling to Minimize LLM Hallucinations. arXiv:2503.10941 [cs.AI] https://arxiv.org/abs/2503.10941

    2025 arXiv

  6. [14]

    Xing Hu, Zhuang Liu, Xin Xia, Zhongxin Liu, Tongtong Xu, and Xiaohu Yang

  7. [15]

    In2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)

    Identify and Update Test Cases When Production Code Changes: A Transformer-Based Approach. In2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1111–1122

  8. [16]

    Javaria Imtiaz, Salman Sherin, Muhammad Uzair Khan, and Muhammad Zohaib Iqbal. 2019. A systematic literature review of test breakage prevention and repair techniques.Information and Software Technology113 (2019), 1–19

    2019

  9. [17]

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs.IEEE transactions on big data7, 3 (2019), 535–547

    2019

  10. [18]

    Doris, Amir Mo- hammad Vahedi, Daniele Grandi, Faez Ahmed, and Hongyi Xu

    Kiarash Naghavi Khanghah, Hoang Anh Nguyen, Anna C. Doris, Amir Mo- hammad Vahedi, Daniele Grandi, Faez Ahmed, and Hongyi Xu. 2026. MCERF: Advancing Multimodal LLM Evaluation of Engineering Documentation with Enhanced Retrieval. arXiv:2604.09552 [cs.IR] https://arxiv.org/abs/2...

    2026 arXiv

  11. [19]

    Pavneet Singh Kochhar, Xin Xia, and David Lo. 2019. Practitioners’ views on good software testing practices. In2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 61–70

    2019

  12. [20]

    LangChain

    Inc. LangChain. 2024. LangGraph: Building Stateful, Multi-Actor Applications with LLMs. https://github.com/langchain-ai/langgraph. Accessed: 2026-04-10

    2024

  13. [21]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing S...

    2020

  14. [22]

    Ying Li, Ye Zhong, Lijuan Yang, Yanbo Wang, and Penghua Zhu. 2025. LLM- Guided Crowdsourced Test Report Clustering.IEEE Access13 (2025), 24894–24904. doi:10.1109/ACCESS.2025.3530960

    2025 doi

  15. [23]

    Jingxiong Liu, Ludvig Lemner, Linnea Wahlgren, Gregory Gay, Nasser Moham- madiha, and Joakim Wennerberg. 2025. Exploring the Integration of Large Lan- guage Models in Industrial Test Maintenance Processes. arXiv:2409.06416 [cs.SE] https://arxiv.org/abs/2409.06416

    2025 arXiv

  16. [24]

    Jun Liu, Jiwei Yan, Yuanyuan Xie, Jun Yan, and Jian Zhang. 2024. Fix the Tests: Augmenting LLMs to Repair Test Cases with Static Collector and Neural Reranker. In2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 367–378

    2024

  17. [25]

    Panagiotis Louridas. 2006. Static code analysis.Ieee Software23, 4 (2006), 58–61

    2006

  18. [26]

    Yang Luo. 2025. Can we translate code better with LLMs and call graph analysis?. InProceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. 7625–7633

    2025

  19. [27]

    2014.Git in practice

    Mike McQuaid. 2014.Git in practice. Simon and Schuster

    2014

  20. [28]

    Mehdi Mirzaaghaei. 2011. Automatic test suite evolution. InProceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering. 396–399

    2011

  21. [29]

    Mehdi Mirzaaghaei, Fabrizio Pastore, and Mauro Pezze. 2010. Automatically repairing test cases for evolving method declarations. In2010 ieee international conference on software maintenance. IEEE, 1–5

    2010

  22. [30]

    Mehdi Mirzaaghaei, Fabrizio Pastore, and Mauro Pezzè. 2014. Automatic test case evolution.Software Testing, Verification and Reliability24, 5 (2014), 386–411

    2014

  23. [31]

    2009.The probabilistic relevance frame- work: BM25 and beyond

    Stephen Robertson and Hugo Zaragoza. 2009.The probabilistic relevance frame- work: BM25 and beyond. Vol. 4. Now Publishers Inc

    2009

  24. [32]

    B.G. Ryder. 1979. Constructing the Call Graph of a Program.IEEE Transactions on Software EngineeringSE-5, 3 (1979), 216–226. doi:10.1109/TSE.1979.234183

    1979 doi

  25. [33]

    Mats Skoglund and Per Runeson. 2004. A case study on regression test suite main- tenance in system evolution. In20th IEEE International Conference on Software Maintenance, 2004. Proceedings.IEEE, 438–442

    2004

  26. [34]

    Harry M Sneed. 2004. A cost model for software maintenance & evolution. In 20th IEEE International Conference on Software Maintenance, 2004. Proceedings. IEEE, 264–273

    2004

  27. [35]

    Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https: //arxiv.org/abs/2505.09388

    2025 arXiv

  28. [36]

    Jingxiang Tu, Bo Lin, Yihao Qin, Shangwen Wang, Liqian Chen, and Xiaoguang Mao. 2025. Trace: Test Repair via Agent-based Context Extraction with LLMs. In 2025 32nd Asia-Pacific Software Engineering Conference (APSEC). 57–68. doi:10. 1109/APSEC66846.2025.00017

    2025

  29. [37]

    Bart Van Rompaey and Serge Demeyer. 2009. Establishing traceability links between unit test cases and units under test. In2009 13th European Conference on Software Maintenance and Reengineering. IEEE, 209–218

    2009

  30. [38]

    Chaozheng Wang, Zezhou Yang, Shuzheng Gao, Cuiyun Gao, Ting Peng, Hailiang Huang, Yuetang Deng, and Michael Lyu. 2025. Rag or fine-tuning? a comparative study on lcms-based code completion in industry. InProceedings of the 33rd ACM International Conference on the Foundations o...

    2025

  31. [39]

    Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. 2024. Software testing with large language models: Survey, landscape, and vision.IEEE Transactions on Software Engineering(2024)

    2024

  32. [40]

    Sinan Wang, Ming Wen, Yepang Liu, Ying Wang, and Rongxin Wu. 2021. Under- standing and facilitating the co-evolution of production and test code. In2021 IEEE International conference on software analysis, evolution and reengineering (SANER). IEEE, 272–283

    2021

  33. [41]

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021. Codet5: Identifier- aware unified pre-trained encoder-decoder models for code understanding and generation.arXiv preprint arXiv:2109.00859(2021)

    2021 arXiv

  34. [42]

    Tangzhi Xu, Jianhan Liu, Yuan Yao, Cong Li, Feng Xu, and Xiaoxing Ma. 2025. Comprehend, Imitate, and then Update: Unleashing the Power of LLMs in Test Suite Evolution. In2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). 469–481. doi:10.1109/A...

    2025 doi

  35. [43]

    Zezhou Yang, Ting Peng, Cuiyun Gao, Chaozheng Wang, Hailiang Huang, and Yuetang Deng. 2025. A deep dive into retrieval-augmented generation for code completion: Experience on wechat. In2025 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 608–619

    2025

  36. [44]

    Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. Au- tocoderover: Autonomous program improvement. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1592–1604

    2024

  37. [45]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models.arXiv preprint arXiv:2303.18223(2023). Received 23 April 2026; accepted 21 June 2026

    2023 arXiv

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.