Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

CIgrate: Automating CI Service Migration with Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes CIgrate, an LLM-based migration framework, and claims it beats the rule-based CIMig on accuracy and parseability.

desk verdict Abstract reports results the body doesn't contain; the plan is sound but the submission is internally inconsistent. read the letter →

arxiv 2507.20402 v2 pith:25K775LE submitted 2025-07-27 cs.SE

Md Nazmul Hossain , Taher A. Ghaleb This is my paper
classification cs.SE
keywords ContinuousIntegrationCImigrationLargeLanguageModelsGitHubActionsTravisDevelopersatisfactionRegisteredreportFine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CIgrate is an LLM-based framework for moving continuous integration configurations between services, and this paper sets out to establish that LLMs can automate that migration more accurately and more reliably than the existing rule-based tool CIMig. The headline is that a fine-tuned Gemma 3 12B model reaches 0.90 cosine similarity and 0.74 CrystalBLEU on Travis CI to GitHub Actions migrations, gains of 82.2% and 295.5% over CIMig, with 100% of generated YAML syntactically parseable versus 5.6% for CIMig. The manuscript body is a registered report rather than a completed experiment: it describes planned phases and says a replication package will accompany the finalized study, so those figures are announced expectations rather than presented results. A sympathetic reader understands the contribution as the framework, the four research questions, and an evaluation protocol capable of testing whether LLM-based migration can replace hand-built rules.

What carries the argument

The load-bearing object is CIgrate, a framework that turns one CI service's YAML configuration into another service's YAML configuration by prompting a large language model in zero-shot, few-shot, or fine-tuned modes; the fine-tuned variant is trained on paired migration examples from the CIMig dataset. CIMig, the baseline, works by mining transformation rules from aligned Travis and GitHub Actions pairs and applying those rules to new files. The evaluation machinery is metric-based: cosine similarity measures lexical overlap between generated and human-written reference YAML, CrystalBLEU measures code-aware n-gram overlap after identifier abstraction, and CI-specific linters check that the output parses in the target service; all comparisons use the top-1 output at temperature 0.

What would settle it

Run the fine-tuned Gemma 3 12B protocol on the held-out CIMig test pairs and measure cosine similarity, CrystalBLEU, and YAML parseability against the human references; if the results do not come close to 0.90, 0.74, and 100%, the central claim is false. A second check is to execute the generated workflows and see whether they produce the same build behavior as the original Travis configurations.

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that LLM-based CI migration outperforms rule-based migration on both accuracy and practical validity: the fine-tuned Gemma 3 12B is reported to achieve 0.90 cosine similarity and 0.74 CrystalBLEU for Travis-to-GitHub Actions migration, versus 0.49 cosine similarity for CIMig, with 100% immediately parseable YAML compared to 5.6%. The authors further claim that even zero-shot prompting of LLMs beats the rule-based baseline, meaning the approach does not depend on pre-mined migration rules or large collections of curated examples. The body, however, labels the manuscript a registered report: Section I says the authors plan to evaluate CIgrate compared to CIMig, and Section VI states that a replication package will accompany the finalized study; the abstract's numeric results do not appear in the body text. Taken as a proposal, the paper's aim is to prove the protocol's expected outcome, namely that LLMs provide a more practical and accurate migration path for evolving CI configurations.

Load-bearing premise

The load-bearing premise is that the quoted similarity and parseability scores come from actually running the planned evaluation on held-out migration pairs, and that those scores capture whether a migrated CI configuration is correct and usable.

Editorial extensions

If this is right

  • If the reported numbers hold, a fine-tuned open LLM would more than double the cosine similarity of automated Travis-to-GitHub Actions migration relative to the rule-based baseline.
  • A 100% parseability rate versus 5.6% means developers would receive ready-to-run YAML far more often, reducing manual repair effort.
  • Because even zero-shot LLMs are claimed to beat CIMig, automated migration would no longer require first collecting and mining a large corpus of migration pairs.
  • With the same framework handling both Travis-to-GitHub Actions and GitHub Actions-to-Travis directions, developers could keep both CI services in sync as projects transition.
  • Pull-request feedback from active projects would supply evidence about whether developers trust and adopt LLM-generated CI configurations, beyond similarity scores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The abstract's 0.90, 0.74, and 100% figures should be read as expectations until the planned study is completed, because the body does not report any executed experiments.
  • Similarity to a human-written reference does not guarantee that a migrated configuration builds the same software; two YAML files can score nearly identical on token overlap yet differ in triggers, environment handling, or deployment semantics, a gap the paper's own construct-validity section acknowledges.
  • If the fine-tuning recipe works, the same protocol could be tested on other YAML configuration domains the paper lists as out of scope, such as CircleCI, GitLab CI, Docker Compose, or Kubernetes manifests.
  • The promised public replication package will be the main route for independent verification; without it, the claimed performance gains cannot be separated from prompt and dataset choices.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CIgrate, an LLM-based framework for migrating CI configuration files between Travis CI and GitHub Actions, and compares it with the rule-based CIMig baseline. The abstract reports strong quantitative results: the fine-tuned Gemma 3 12B achieves 0.90 Cosine Similarity and 0.74 CrystalBLEU for Travis-to-GitHub Actions migration, outperforming CIMig by 82.2% and 295.5%, with 100% syntactically parseable YAML versus 5.6% for CIMig. The body, however, is written entirely in the future tense: Section I describes the paper as 'a registered report' and says 'We plan to evaluate', Section IV is an 'Execution Plan', and Section VI promises a replication package 'will accompany the finalized study'. No results, tables, or measurements appear anywhere in the manuscript.

Significance. If the reported results were present and reproducible, this would be a significant contribution: it would be the first empirical evidence that LLM-based approaches substantially outperform rule-based CI migration, and the design is carefully aligned with the CIMig evaluation protocol (same dataset, cosine similarity and CrystalBLEU metrics, deterministic decoding, held-out split). The plan for multi-model evaluation (Gemma, Llama, Mistral, GPT-4), zero-shot/few-shot/fine-tuning, and developer feedback via pull requests is methodologically sound. However, because the central empirical claim is unsupported by any in-manuscript evidence, the significance is currently prospective only.

major comments (3)
  1. [Abstract; Section I; Section IV; Section VI] The abstract reports concrete results (0.90 Cosine Similarity, 0.74 CrystalBLEU, +82.2%, +295.5%, 100% vs 5.6% parseable YAML) but none of these numbers appear in the body. Section I explicitly states 'This registered report presents a study in which we aim to assess whether CI migration can be improved using LLMs' and 'We plan to evaluate the performance of CIgrate compared to CIMig as a baseline.' Section IV is titled 'Execution Plan' and all approaches are described with 'we will'. Section VI says 'A publicly available replication package will accompany the finalized study.' The manuscript contains no tables, figures, or measurements of results. This is an internal inconsistency: the document is a future-tense registered report, while the abstract claims past-tense results. The central claim of the paper therefore lacks any in-manuscript support and cannot be checked.
  2. [Abstract; Section II.A] The relative improvements over CIMig cannot be verified arithmetically from the information provided. The abstract quotes 0.49 Cosine Similarity for CIMig, and the +82.2% figure is roughly consistent (0.49 × 1.822 ≈ 0.89). However, CIMig's CrystalBLEU baseline is never reported anywhere in the manuscript, so the +295.5% improvement for CrystalBLEU has no stated base value. Similarly, the 5.6% parseable-YAML baseline for CIMig is not derived in the text. Even if the experiments had been run, the reader could not reproduce or audit these percentages without the underlying baseline measurements and test-set details (number of migration pairs, split identifier, decoding parameters).
  3. [Section V.A; Section IV Phase 2] The paper's own threat-to-validity section acknowledges that Cosine Similarity and CrystalBLEU 'may not fully capture accuracy or practical viability' (Section V.A). The abstract nonetheless frames a syntactic-parsing metric (100% parseable YAML) as a central advantage. The planned evaluation does not include a semantic or behavioral equivalence check (e.g., comparing build/test outcomes of the original and migrated configurations) beyond linters and developer feedback. If the reported results were restored or reproduced, the claims would still be insufficient to establish that LLM-generated migrations behave correctly, since token-overlap and syntactic validity do not imply functional equivalence. This design limitation should be addressed before results are reported.
minor comments (5)
  1. [Throughout] The framework name is inconsistent: 'CIgrate' appears in the abstract and some sections, while 'CIGRATE' is used in Section III.A.3 and Figure 2, and 'CI GRATE' appears in the introduction. Please unify to a single spelling.
  2. [Section II.A; Section III] The text contains 'Y AML' with a spurious space in several places (e.g., 'Y AML-based configuration files', 'All Y AML files'). This should read 'YAML'.
  3. [Section III] The dataset description says 'this dataset includes 13,403 Travis-only projects, 15,888 GitHub Actions-only projects, from which we will use 1,252 dual-CI projects for testing.' The first two counts are irrelevant to the selection of dual-CI projects and may confuse the reader; please clarify whether the 1,252 projects are a subset of the dual-CI projects and how they were sampled.
  4. [Section IV Phase 1] The heading 'Data Preparation, and Model Selection:' has a stray colon after the heading. In addition, 'Phase 4: Derivation of Key Takeaways and Implications' is the only phase whose implementation is not described, which is fine for a plan but could be marked as analytic rather than procedural.
  5. [References] Some references are informal: [13] points to an Ollama model page rather than a technical report for Gemma, and [16] appears to be a workshop paper without a DOI. If the venue requires archival references, update the citations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; the abstract's reported results are unsupported by the body, which is a future-tense registered report — an evidence problem, not a circularity problem.

full rationale

I find no circular step in this manuscript because there is no completed derivation chain to examine: the body is uniformly a registered report in the future tense. Section I says 'This registered report presents a study in which we aim to assess whether CI migration can be improved using LLMs' and 'We plan to evaluate the performance of CI GRATE compared to CIMig as a baseline'; Section IV is titled 'Execution Plan' and describes experiments to be run; Section VI promises 'A publicly available replication package will accompany the finalized study.' The abstract, however, asserts completed results: 'Our results show that CIgrate substantially outperforms CIMig: the fine-tuned Gemma 3 12B achieves 0.90 Cosine Similarity and 0.74 CrystalBLEU for Travis CI (Travis)->GitHub Actions (GHA) migrations (+82.2% and +295.5% over CIMig), while producing syntactically valid, immediately parseable YAML in 100% of cases.' Those numbers appear nowhere in the body, so they are not derived from the reported method; they are either unsupported claims or results from an omitted experiment. That is a serious reporting and reproducibility defect, but it is not circular: no fitted parameter is renamed as a prediction, no self-citation is load-bearing for the central comparison, and no definition builds the target result into the inputs. The planned evaluation uses an external dataset (CIMig's, from Rzig et al.) with separated training and held-out test splits and the same similarity metrics as the baseline, so the design, as described, does not reduce to its own inputs. I also checked self-citations [10, 26, 27, 28]: they are used as background about CI build behavior, not to justify the migration outcome. The circularity score is therefore 0; the abstract/body inconsistency should be handled as a validity and evidence concern rather than a circularity finding.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim depends on an executed study that is never documented: the abstract reports results while the entire body is a future-tense plan. No fitted parameters or invented entities exist to audit because no experiments are reported. The measurable assumptions are the unvalidated similarity-metric proxy for migration quality, the uncritical adoption of the CIMig dataset as ground truth, and the implicit claim that the study was actually run.

assumptions (3)
  • domain assumption Cosine similarity and CrystalBLEU scores against human-written reference configurations measure CI migration quality.
    Invoked in Section IV Phase 2 as the primary automated evaluation. Both metrics measure token or n-gram overlap, not functional equivalence of build behavior.
  • domain assumption The CIMig paired dataset is correct and representative, and its train/test split is sound without re-validation.
    Sections III and IV Phase 1 adopt Rzig et al.'s dataset and splits as ground truth; errors in pairing or normalization would propagate into any comparison.
  • ad hoc to paper The experiments described in the abstract were actually performed and are the source of the quoted numbers.
    The abstract claims completed results while the body (Sections I, IV, VI) is future tense and explicitly describes a registered report whose replication package is still to come. The central claim depends on an executed study that the manuscript does not document.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CIgrate: Automating CI Service Migration with Large Language Models." pith.science (2026). https://pith.science/paper/25K775LE

@misc{pith2026250720402,
  author       = {Pith},
  title        = {Pith review of: CIgrate: Automating CI Service Migration with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/25K775LE}},
  note         = {Machine review of arXiv:2507.20402}
}
read the original abstract

Continuous Integration (CI) configurations often need to be migrated between services (e.g., Travis CI to GitHub Actions) as projects evolve due to changes in service capabilities, usage limits, or service deprecation. Previous studies reported that migration across CI services is a recurring need in open-source development. However, manual migration is time-consuming and error-prone. The state-of-the-art approach, CIMig, addresses this challenge by analyzing past migration examples to create service-specific rules and produce equivalent configurations across CI services. However, its relatively low accuracy (0.49 Cosine Similarity for Travis->GHA) raises concerns about the feasibility of rule-based CI migration alone. Meanwhile, Large Language Models (LLMs) have demonstrated strong capabilities in code generation and transformation tasks, suggesting potential to improve the automation, usability, and generalizability of CI configuration migration. This paper presents an empirical study assessing whether CI migration can be improved using LLMs. We propose CIgrate, an LLM-based framework for automatically migrating CI configurations, and compare it with CIMig using (a) zero-shot/few-shot prompting and (b) fine-tuning on a dataset of established CI migrations. We also evaluate practical deployment by submitting pull requests to active open-source projects. Our results show that CIgrate substantially outperforms CIMig: the fine-tuned Gemma 3 12B achieves 0.90 Cosine Similarity and 0.74 CrystalBLEU for Travis CI (Travis)->GitHub Actions (GHA) migrations (+82.2% and +295.5% over CIMig), while producing syntactically valid, immediately parseable YAML in 100% of cases, compared to 5.6% for CIMig. Even zero-shot LLMs outperform the rule-based baseline, demonstrating that LLM-based approaches provide a more practical and accurate solution for CI configuration migration.

Figures

Figures reproduced from arXiv: 2507.20402 by the authors.

Figure 1
Figure 1. CI build lifecycle Though both formats use YAML [22], their schemas and semantics differ significantly. For instance, Travis CI spec￾ifies jobs via language directives and matrix expansions, whereas GitHub Actions requires explicit job declarations with ‘runs-on’ keys and step sequences [9, 25]. Even minor differences, such as the handling of environment variables or conditional job execution, can cause functional i… view at source ↗
Figure 2
Figure 2. Overview of our study 4) RQ4: How do developers rate the quality and usability of CIGRATE’s output?: Motivation: While functional accuracy is essential, the prac￾tical adoption of automated CI migration ultimately depends on how developers perceive its outputs in terms of quality, readability, and usability. Developers must find the generated configurations understandable, trustworthy, and maintainable to adopt them… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Android Instrumentation Testing in Continuous Integration: Practices, Patterns, and Performance

    cs.SE 2026-04 conditional novelty 7.0 of 10

    Only 10.6% of Android repositories with CI run instrumentation tests, with community-based setups proving most reliable and efficient for daily checks while custom scripts offer flexibility at the cost of more reruns.

  2. How Compliant Are GitHub Actions Workflows? A Checklist-Based Study with LLM-Assisted Auditing

    cs.SE 2026-05 accept novelty 6.0 of 10

    GitHub Actions workflows achieve only 28% overall compliance with best practices, with LLMs enabling an 81% reduction in verification effort via hybrid adjudication but still requiring expert oversight for security judgments.

  3. A Vision for Context-Aware CI Adoption Decisions

    cs.SE 2026-04 unverdicted novelty 3.0 of 10

    An AI-enabled framework is proposed to assess CI suitability, recommend services, and guide configurations according to project characteristics.

Reference graph

Works this paper leans on

39 extracted references · 32 canonical work pages · cited by 3 Pith papers

  1. [1]

    Continuous Integration,

    M. Fowler and M. Foemmel, “Continuous Integration,” URL http://martinfowler. com/articles/continuousIntegration. html, pp. 10–17, 2006

    work page 2006

  2. [2]

    Usage, costs, and benefits of continuous integration in open-source projects,

    M. Hilton, T. Tunnell, K. Huang, D. Marinov, and D. Dig, “Usage, costs, and benefits of continuous integration in open-source projects,” in Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering . ACM, 2016, pp. 426–437

    work page 2016

  3. [3]

    A study on the interplay between pull request review and continuous integration builds,

    F. Zampetti, G. Bavota, G. Canfora, and M. D. Penta, “A study on the interplay between pull request review and continuous integration builds,” in Proceedings of the IEEE 26th International Conference on Software Analysis, Evolution and Reengineering . IEEE, 2019, pp. 38–48

    work page 2019

  4. [4]

    The im- pact of continuous integration on other software development practices: a large-scale empirical study,

    Y . Zhao, A. Serebrenik, Y . Zhou, V . Filkov, and B. Vasilescu, “The im- pact of continuous integration on other software development practices: a large-scale empirical study,” in Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering . IEEE, 2017, pp. 60–71

    work page 2017

  5. [5]

    Travis CI usage limits for open source projects,

    Travis-CI, “Travis CI usage limits for open source projects,” 2020, accessed: 2025-05-20. [Online]. Available: https://blog.travis-ci.com/ 2020-11-02-travis-ci-new-billing

    work page 2020

  6. [6]

    Characterizing the usage of CI tools in ML projects,

    D. E. Rzig, F. Hassan, C. Bansal, and N. Nagappan, “Characterizing the usage of CI tools in ML projects,” in Proceedings of the 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2022, pp. 69–79

    work page 2022

  7. [7]

    On the usage, co-usage and migration of CI/CD tools: A qualitative analysis,

    P. Rostami Mazrae, T. Mens, M. Golzadeh, and A. Decan, “On the usage, co-usage and migration of CI/CD tools: A qualitative analysis,” Empirical Software Engineering , vol. 28, no. 2, p. 52, Mar 2023

    work page 2023

  8. [8]

    Example-Based Automatic Migration of Continuous Integration Systems

    D. E. Rzig, A. Houerbi, C. Sung, and F. Hassan, “Example-Based Automatic Migration of Continuous Integration Systems,” arXiv preprint arXiv:2407.02644, 2024

    work page Pith review arXiv 2024

Show all 39 references
  1. [9]

    About GitHub Actions,

    GitHub, “About GitHub Actions,” 2019, accessed: 2025-07-10. [Online]. Available: https://docs.github.com/en/actions

    2019

  2. [10]

    CI/CD configuration practices in open-source android apps: An empirical study,

    T. A. Ghaleb, O. Abduljalil, and S. Hassan, “CI/CD configuration practices in open-source android apps: An empirical study,” ACM Transactions on Software Engineering and Methodology , 2025

    2025

  3. [11]

    A survey on large language models for code generation,

    J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim, “A survey on large language models for code generation,” arXiv preprint arXiv:2406.00515, 2024

    2024 arXiv

  4. [12]

    A survey on evaluating large language models in code generation tasks,

    L. Chen, Q. Guo, H. Jia, Z. Zeng, X. Wang, Y . Xu, J. Wu, Y . Wang, Q. Gao, J. Wang et al., “A survey on evaluating large language models in code generation tasks,” arXiv preprint arXiv:2408.16498 , 2024

    2024 arXiv

  5. [13]

    Gemma 3 12B,

    G. DeepMind, “Gemma 3 12B,” 2024, accessed: 2025-06-03. [Online]. Available: https://ollama.com/library/gemma3:12b

    2024

  6. [14]

    Llama 3.1 8B,

    M. Llama, “Llama 3.1 8B,” 2024, accessed: 2025-06-03. [Online]. Available: https://ollama.com/library/llama3.1:8b

    2024

  7. [15]

    Announcing Mistral 7B,

    M. AI, “Announcing Mistral 7B,” 2023, accessed: 2025-06-03. [Online]. Available: https://mistral.ai/news/announcing-mistral-7b

    2023

  8. [16]

    Large language model based code completion is an effective genetic improvement mutation,

    J. Wang, C. Hanna, and J. Petke, “Large language model based code completion is an effective genetic improvement mutation,” inIEEE/ACM International Workshop on Genetic Improvement (GI), 2025, pp. 11–18

    2025

  9. [17]

    GPT-4 technical report,

    OpenAI, “GPT-4 technical report,” 2023, accessed: 2025-07-10. [Online]. Available: https://openai.com/research/gpt-4

    2023

  10. [18]

    Towards an understanding of large language models in software engineering tasks,

    Z. Zheng, K. Ning, Q. Zhong, J. Chen, W. Chen, L. Guo, W. Wang, and Y . Wang, “Towards an understanding of large language models in software engineering tasks,” Empirical Software Engineering , vol. 30, no. 2, p. 50, 2025

    2025

  11. [19]

    P. M. Duvall, A. Glover, and S. Matyas, Continuous Integration: Improving Software Quality and Reducing Risk, 1st ed. Addison Wesley, 2007

    2007

  12. [20]

    Home – travis-ci 2022,

    TravisCI, “Home – travis-ci 2022,” 2022, accessed: 2025-05-27. [Online]. Available: https://www.travis-ci.com

    2022

  13. [21]

    GitHub Actions,

    GitHub, “GitHub Actions,” 2023, accessed: 2025-06-03. [Online]. Available: https://github.com/features/actions

    2023

  14. [22]

    Travis CI documentation - using Y AML as a build configuration language,

    Travis-CI, “Travis CI documentation - using Y AML as a build configuration language,” 2021, accessed: 2021-08-31. [Online]. Available: https://docs.travis-ci.com/user/build-config-yaml/

    2021

  15. [23]

    Belmont, Hands-on Continuous Integration and Delivery: Build and Release Quality Software at Scale with Jenkins, Travis CI, and CircleCI, 1st ed

    J.-M. Belmont, Hands-on Continuous Integration and Delivery: Build and Release Quality Software at Scale with Jenkins, Travis CI, and CircleCI, 1st ed. Birmingham, UK: Packt Publishing, 2018

    2018

  16. [24]

    Heller, Automating Workflows with GitHub Actions: Automate Soft- ware Development Workflows and Seamlessly Deploy Your Applications Using GitHub Actions

    P. Heller, Automating Workflows with GitHub Actions: Automate Soft- ware Development Workflows and Seamlessly Deploy Your Applications Using GitHub Actions . Birmingham: Packt Publishing, Limited, 2021

    2021

  17. [25]

    Travis CI Documentation,

    Travis CI, Inc., “Travis CI Documentation,” 2025, accessed: 2025-05-27. [Online]. Available: https://docs.travis-ci.com/

    2025

  18. [26]

    An empirical study of the long duration of continuous integration builds,

    T. A. Ghaleb, D. A. Da Costa, and Y . Zou, “An empirical study of the long duration of continuous integration builds,” Empirical Software Engineering, vol. 24, pp. 2102–2139, 2019

    2019

  19. [27]

    Studying the impact of noises in build breakage data,

    T. A. Ghaleb, D. A. Da Costa, Y . Zou, and A. E. Hassan, “Studying the impact of noises in build breakage data,” IEEE Transactions on Software Engineering, vol. 47, no. 9, pp. 1998–2011, 2019

    1998

  20. [28]

    Studying the interplay between the durations and breakages of continuous integration builds,

    T. A. Ghaleb, S. Hassan, and Y . Zou, “Studying the interplay between the durations and breakages of continuous integration builds,” IEEE Transactions on Software Engineering , vol. 49, no. 4, pp. 2476–2497, 2022

    2022

  21. [29]

    I’m leaving you, Travis: a continuous integration breakup story,

    D. G. Widder, M. Hilton, C. K ¨astner, and B. Vasilescu, “I’m leaving you, Travis: a continuous integration breakup story,” in Proceedings of the 15th International Conference on Mining Software Repositories , 2018, pp. 165–169

    2018

  22. [30]

    Large language models for software engi- neering: A systematic literature review,

    X. Hou, Y . Zhao, Y . Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engi- neering: A systematic literature review,” ACM Transactions on Software Engineering and Methodology , vol. 33, no. 8, pp. 1–79, 2024

    2024

  23. [31]

    Software testing with large language models: Survey, landscape, and vision,

    J. Wang, Y . Huang, C. Chen, Z. Liu, S. Wang, and Q. Wang, “Software testing with large language models: Survey, landscape, and vision,”IEEE Transactions on Software Engineering , 2024

    2024

  24. [32]

    Transagent: An llm-based multi-agent system for code translation,

    Z. Yuan, W. Chen, H. Wang, K. Yu, X. Peng, and Y . Lou, “Transagent: An llm-based multi-agent system for code translation,” arXiv preprint arXiv:2409.19894, 2024

    2024 arXiv

  25. [33]

    Exploring and unleashing the power of large language models in automated code translation,

    Z. Yang, F. Liu, Z. Yu, J. W. Keung, J. Li, S. Liu, Y . Hong, X. Ma, Z. Jin, and G. Li, “Exploring and unleashing the power of large language models in automated code translation,” Proceedings of the ACM on Software Engineering, vol. 1, no. FSE, pp. 1585–1608, 2024

    2024

  26. [34]

    The Likert scale revisited,

    G. Albaum, “The Likert scale revisited,” Market Research Society. Journal., vol. 39, no. 2, pp. 1–21, 1997

    1997

  27. [35]

    Language mod- els are few-shot learners,

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  28. [36]

    The imple- mentation of cosine similarity to calculate text relevance between two documents,

    D. Gunawan, C. A. Sembiring, and M. A. Budiman, “The imple- mentation of cosine similarity to calculate text relevance between two documents,” Journal of Physics: Conference Series , vol. 978, no. 1, pp. 12 120–, 2018

    2018

  29. [37]

    CrystalBLEU: Precisely and efficiently measuring the similarity of code,

    A. Eghbali and M. Pradel, “CrystalBLEU: Precisely and efficiently measuring the similarity of code,” in IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings . New York, NY , USA: ACM, 2022, pp. 341–342

    2022

  30. [38]

    Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment,

    L. Xu, H. Xie, S.-Z. J. Qin, X. Tao, and F. L. Wang, “Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment,” arXiv preprint arXiv:2312.12148 , 2023

    2023 arXiv

  31. [39]

    Summary of ChatGPT-related research and perspective towards the future of large language models,

    Y . Liu, T. Han, S. Ma, J. Zhang, Y . Yang, J. Tian, H. He, A. Li, M. He, Z. Liu et al. , “Summary of ChatGPT-related research and perspective towards the future of large language models,” Meta-radiology, vol. 1, no. 2, p. 100017, 2023

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.