Pith. sign in

REVIEW 3 major objections 5 minor 52 references

Exploring the Impact of Integrating UI Testing in CI/CD Workflows on GitHub

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that a GitHub-wide keyword search of GitHub Actions workflow files can reveal whether adding UI testing to CI/CD changes open-source development, and it proposes an empirical study with comparative and interrupted…

desk verdict A clear, honest study plan for an under-explored topic, but the proposed designs cannot back the causal wording in the research questions. read the letter →

arxiv 2504.19335 v1 pith:CH4DY7ZE submitted 2025-04-27 cs.SE

Xiaoxiao Gan , Chris Brown This is my paper · ORCID
classification cs.SE
keywords CI/CDUItestingGitHubActionsSeleniumCypressPlaywrightopen-sourcesoftwareempiricalstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes an empirical study to find out whether integrating UI testing into CI/CD pipelines changes how open-source projects develop. It argues that repositories using Selenium, Cypress, or Playwright in GitHub Actions workflow files can be detected by searching for those keywords in .github/workflows/*.yml and .yaml files, and that comparing UI-test workflows against non-UI workflows before and after the keyword first appears will reveal impacts on development activity and CI/CD health. The authors care because UI testing is widely considered difficult to automate and practitioners report the effort is not worth the benefit, yet no prior work has measured the impact of UI testing on CI/CD in open-source software. The paper is a study plan; it states no results, but it lays out testable research questions and a concrete pipeline for answering them.

What carries the argument

The load-bearing mechanism is keyword-based detection: the GitHub REST API code search looks for the strings 'selenium', 'cypress', or 'playwright' inside GitHub Actions workflow files, supplemented by Sourcegraph to bypass the API's 1,000-record limit. Repository filtering then keeps only active, licensed, non-forked projects with enough workflow runs and at least one UI-test and one non-UI workflow, and the analytic core is an interrupted time series model using the first commit that introduced a UI-testing keyword as the intervention marker. This machinery carries the argument because the validity of both research questions rests on whether a workflow file containing one of these strings is a genuine UI-testing workflow.

What would settle it

Manually inspect a random sample of the candidate workflow files the keyword search returns and check whether each one references an actual UI test file or command; if a substantial share of matches are comments, installs without tests, or scraping scripts, the repository sample and every downstream comparison would not measure what the study claims.

Watch

Extended reading notes

Core claim

In the paper's own terms, the central claim is empirical and still unverified: open-source repositories that run UI testing inside GitHub Actions workflows experience measurable differences in development activity and CI/CD workflow behavior compared with their own non-UI workflows. The authors' planned evidence is a dataset of repositories whose GitHub Actions configuration files mention Selenium, Cypress, or Playwright, filtered to active, licensed, non-forked projects with enough workflow runs, then analyzed with descriptive and inferential statistics (paired t-test or Wilcoxon signed-rank) and interrupted time series around the commit that first introduced a UI-testing keyword. If the study succeeds, it would provide the first large-scale evidence on the cost and benefit of UI testing in CI/CD for open-source software.

Load-bearing premise

The whole study depends on the assumption that a GitHub Actions workflow file containing the word 'selenium', 'cypress', or 'playwright' actually runs UI tests, rather than mentioning the tool in a comment or using it for unrelated web automation.

Editorial extensions

If this is right

  • If the detection pipeline works, the study will produce the first large-scale census of open-source repositories using Selenium, Cypress, or Playwright inside GitHub Actions.
  • If UI-testing workflows differ on failure rate and execution time, teams will have concrete evidence about the CI/CD cost of UI testing.
  • If development activity shifts after the introducing commit, maintainers will learn whether UI testing tends to accompany busier or quieter projects.
  • The interrupted time series design can separate a step-change at UI-test adoption from gradual project trends, giving a stronger basis for causal language.
  • A public dataset and scripts would let other researchers extend the same pipeline to other frameworks and CI platforms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step the paper does not pursue is to validate the keyword detector's precision and recall on a labeled random sample before running any impact analysis; that validation alone would be a publishable measurement result.
  • The same GitHub Actions corpus could also be used to ask whether UI tests are more likely to be removed or disabled after flaky runs, a maintenance pattern the paper's metrics would only indirectly capture.
  • Because the paper selects only projects that already have both UI and non-UI workflows, it will not say anything about projects that tried UI testing and deleted it; a separate survival-style analysis would be needed.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes an empirical study plan to investigate whether integrating UI testing frameworks (Selenium, Cypress, Playwright) into GitHub Actions CI/CD workflows affects open-source development activity and CI/CD processes. The authors describe a repository selection strategy based on keyword search of workflow files, a set of filtering criteria, plans to collect development and workflow metrics, and a two-part analysis plan (comparative statistics and interrupted time series). No data collection, analysis, or results are presented; the paper is explicitly a study protocol.

Significance. If executed with a valid design, a study of UI testing in CI/CD workflows on GitHub could provide useful empirical evidence for a topic with limited prior work. The manuscript's strengths include a clear research question, an explicit filter criteria list, a plan to publish data and scripts, and an acknowledgment of several validity threats. However, as submitted, the paper makes no empirical contribution, and the proposed design contains load-bearing flaws that likely prevent it from answering the stated research questions. The significance is therefore conditional on a substantially revised protocol, not on the current content.

major comments (3)
  1. [§III-C.1b] The attribution of development activity to UI-test-related workflow runs is not valid as described. The authors state that there is 'no direct way to fetch commits, pull requests or issues influenced by a specific workflow run' and propose to filter by timestamp and message text. This heuristic conflates temporal proximity with causation: a commit that triggers a workflow containing a UI-test keyword is not necessarily caused by UI testing, and issues/PRs mentioning UI-test terms may be unrelated to the workflow run. Consequently, the planned comparative analysis between 'UI test-related' and 'non-UI test-related' metrics cannot support the causal language of RQ1 ('affect development processes'). This is a central threat to the validity of the main research question.
  2. [§III-C.2] The interrupted time series design uses the first commit introducing a UI-test keyword as the intervention marker, but UI testing integration is an ongoing practice rather than a one-time event: repositories add, remove, and update UI tests over time, and workflow files change frequently. Moreover, the design lacks a control group or any adjustment for confounding factors such as project popularity, contributor turnover, or simultaneous changes in other CI/CD tooling. Without such controls, any observed before/after differences in development or workflow metrics cannot be attributed to UI testing integration. The authors do not mention difference-in-differences, matched controls, or other identification strategies that would address this issue.
  3. [§III-A and §IV-A] The keyword-based detector for identifying UI-testing repositories is foundational to the entire sample, and the acknowledged false-positive threat (comments, web scraping, non-testing automation) is only partially mitigated by the planned manual review. Criterion 7 requires 'corresponding UI test files explicitly specified for execution in the workflow files,' but the manuscript does not specify how this will be operationalized, how the manual review will be conducted to avoid researcher bias, or how inter-rater reliability will be measured. A preliminary manual check on a subset of the same repositories is not a validation protocol; the authors should pre-register a detection rule and a validation threshold before proceeding.
minor comments (5)
  1. [§II-C] The sentence 'to our knowledge their is no prior work' contains a typo ('their is' should be 'there is').
  2. [§III-A] The phrase 'In necessary' appears near the end of the repository selection section; it should be 'If necessary.'
  3. [§III-A] The path expression '.github.workflows/.*.yml' should use the correct syntax '.github/workflows/*.yml' to denote GitHub Actions workflow files.
  4. [§III-A] The list of keywords is introduced with inconsistent quotation marks ('selenium', 'cypress', or playwright"), which makes the text slightly confusing.
  5. [Table II] The metric types 'integer' and 'boolean' are used as data types, but some columns (e.g., 'Pull Request Duration datetime') mix units with types; this should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the proposal contains no fitted parameters, no self-citation chain, and no construction that equates inputs with outputs.

full rationale

This manuscript is a study proposal, not a derivation. The central pipeline—keyword search for Selenium/Cypress/Playwright in GitHub Actions workflow files, manual filtering, metric collection, and comparative/time-series analysis—does not fit any circularity pattern. The keyword detector is an operationalization of 'UI testing-related workflow,' and the planned metrics (commits, PRs, issues, workflow run times, failure rates) are measured independently of that definition. Comparing keyword-positive and keyword-negative workflow runs is an empirical comparison, not a tautology; any observed difference would be a consequence of the operationalization and would need construct-validity defense, which the authors explicitly acknowledge in Section IV-A. The first-commit intervention marker for the time-series analysis is introduced as a design choice, not as a fitted parameter, and no result is pre-supposed. There are no load-bearing self-citations: the reference list includes no prior works by the same authors that are invoked to justify the approach, and the 'no prior work' claim in Section II-C is a literature assertion, not a circular step. Under the stated rules, absence of a derivation or fitted model means absence of circularity, so the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities or fitted parameters. Its load-bearing elements are standard mining assumptions about keyword-based detection, platform representativeness, and causal inference from observational data.

assumptions (4)
  • domain assumption Keyword search for 'selenium', 'cypress', 'playwright' in GitHub Actions config files is a valid detector of UI testing integration.
    Stated in Section III-A and acknowledged as a construct validity threat in Section IV-A. This assumption is foundational to repository selection.
  • domain assumption GitHub Actions is a representative CI/CD platform for studying open-source development.
    The paper focuses only on GitHub Actions and acknowledges limited generalizability in Section IV-C.
  • domain assumption Commit, pull request, and issue activity can be attributed to specific workflow runs using timestamps and message inspection.
    Described in Section III-C-1. The authors note there is no direct API link between workflow runs and development activity, so they plan to infer relevance, which may be imprecise.
  • domain assumption Interrupted time series analysis can support causal claims about the impact of UI testing without a control group.
    The time series design compares before and after the introduction of UI testing within the same repositories. Threats such as confounding changes are not fully controlled.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Impact of Integrating UI Testing in CI/CD Workflows on GitHub." pith.science (2026). https://pith.science/paper/CH4DY7ZE

@misc{pith2026250419335,
  author       = {Pith},
  title        = {Pith review of: Exploring the Impact of Integrating UI Testing in CI/CD Workflows on GitHub},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CH4DY7ZE}},
  note         = {Machine review of arXiv:2504.19335}
}
read the original abstract

Background: User interface (UI) testing, which is used to verify the behavior of interactive elements in applications, plays an important role in software development and quality assurance. However, little is known about the adoption of UI testing frameworks in continuous integration and continuous delivery (CI/CD) workflows and their impact on open-source software development processes. Objective: We aim to investigate the current usage of popular UI testing frameworks-Selenium, Playwright and Cypress-in CI/CD pipelines among GitHub repositories. Our goal is to understand how UI testing tools are used in CI/CD processes and assess their potential impacts on open-source development activity and CI/CD workflows. Method: We propose an empirical study to examine GitHub repositories that incorporate UI testing in CI/CD workflows. Our exploratory evaluation will collect repositories that implement UI testing frameworks in configuration files for GitHub Actions workflows to inspect UI testing-related and non-UI testing-related workflows. Moreover, we further plan to collect metrics related to repository development activity and GitHub Actions workflows to conduct comparative and time series analyses exploring whether UI testing integration and usage within CI/CD processes has an impact on open-source development.

Figures

Figures reproduced from arXiv: 2504.19335 by the authors.

Figure 1
Figure 1. Study Design Overview III. STUDY DESIGN Our study aims to explore whether UI testing integration and usage has any impacts on open-source development activity and CI/CD pipelines, in order to identify areas to better support and streamline UI testing in modern software development. To address this goal, we present a detailed study plan to answer the proposed research questions: RQ1: How does UI testing integration a… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 50 canonical work pages

  1. [1]

    Continuous integration, delivery and deployment: A systematic review on approaches, tools, challenges and practices,

    M. Shahin, M. Ali Babar, and L. Zhu, “Continuous integration, delivery and deployment: A systematic review on approaches, tools, challenges and practices,” IEEE Access, vol. 5, pp. 3909–3943, 2017

    work page 2017

  2. [2]

    Usage, costs, and benefits of continuous integration in open-source projects,

    M. Hilton, T. Tunnell, K. Huang, D. Marinov, and D. Dig, “Usage, costs, and benefits of continuous integration in open-source projects,” in Proceedings of the 31st IEEE/ACM international conference on automated software engineering , pp. 426–437, 2016

    work page 2016

  3. [3]

    Synthesizing continuous deployment practices used in software development,

    A. A. U. Rahman, E. Helms, L. Williams, and C. Parnin, “Synthesizing continuous deployment practices used in software development,” in2015 Agile Conference, pp. 1–10, IEEE, 2015

    work page 2015

  4. [4]

    Continuous delivery

    J. Humble, “Continuous delivery.” Continuous Delivery. https://continuousdelivery.com/evidence-case-studies/

    work page

  5. [5]

    Quality and productivity outcomes relating to continuous integration in GitHub,

    B. Vasilescu, Y . Yu, H. Wang, P. Devanbu, and V . Filkov, “Quality and productivity outcomes relating to continuous integration in GitHub,” in Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , ESEC/FSE, pp. 805–816, IEEE, 2015

    work page 2015

  6. [6]

    Analyzing the effects of ci/cd on open source repositories in github and gitlab,

    J. Fairbanks, A. Tharigonda, and N. U. Eisty, “Analyzing the effects of ci/cd on open source repositories in github and gitlab,” in 2023 IEEE/ACIS 21st International Conference on Software Engineering Research, Management and Applications (SERA) , pp. 176–181, IEEE, 2023

    work page 2023

  7. [7]

    How to attain business success with ci/cd pipeline automation testing,

    R. Ramesh, “How to attain business success with ci/cd pipeline automation testing,” Codefresh, 2023. https://www.headspin.io/blog/ why-you-should-consider-ci-cd-pipeline-automation-testing

    work page 2023

  8. [8]

    How to deliver faster with ci/cd integrated testing services?,

    A. Poth, M. Werner, and X. Lei, “How to deliver faster with ci/cd integrated testing services?,” in Systems, Software and Services Process Improvement: 25th European Conference, EuroSPI 2018, Bilbao, Spain, September 5-7, 2018, Proceedings 25 , pp. 401–409, Springer, 2018

    work page 2018

Show all 52 references
  1. [9]

    https://www.interaction-design.org/literature/topics/ graphical-user-interfaces

    What are Graphical User Interfaces (GUI)? Interaction Design Foun- dation - IxDF, 2016. https://www.interaction-design.org/literature/topics/ graphical-user-interfaces

    2016

  2. [10]

    What is gui testing? (types & best practices),

    S. Ganguly, “What is gui testing? (types & best practices),” Browser- Stack, 2023. https://www.browserstack.com/guide/gui-testing

    2023

  3. [11]

    How user interface testing can fit into the ci/cd pipeline

    R. Millan, “How user interface testing can fit into the ci/cd pipeline.” The New Stack, 2022. https://thenewstack.io/how-user-interface-testing- can-fit-into-the-ci-cd-pipeline/

    2022

  4. [12]

    5 software testing strategies to build into your ci/cd pipeline

    B. Cox, “5 software testing strategies to build into your ci/cd pipeline.” Red Hat, 2022. https://www.redhat.com/architect/test-applications-cicd- pipeline

    2022

  5. [13]

    The top 10 adages in continuous deployment,

    C. Parnin, E. Helms, C. Atlee, H. Boughton, M. Ghattas, A. Glover, J. Holman, J. Micco, B. Murphy, T. Savor, et al. , “The top 10 adages in continuous deployment,” IEEE Software, vol. 34, no. 3, pp. 86–95, 2017

    2017

  6. [14]

    Gui testing: Pitfalls and process,

    A. M. Memon, “Gui testing: Pitfalls and process,” Computer, vol. 35, no. 08, pp. 87–88, 2002

    2002

  7. [15]

    A conceptual replication of continuous integration pain points in the context of Travis CI,

    D. Widder, M. Hilton, C. K ¨astner, and B. Vasilescu, “A conceptual replication of continuous integration pain points in the context of Travis CI,” in Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , ESEC/FSE, pp. 647–65...

    2019

  8. [16]

    Experiences and practices in gui functional testing: A software practitioners’ view,

    N. Junior, H. Costa, L. Karita, I. Machado, and L. Soares, “Experiences and practices in gui functional testing: A software practitioners’ view,” in Proceedings of the XXXV Brazilian Symposium on Software Engi- neering, SBES ’21, (New York, NY , USA), p. 195–204, Association f...

    2021

  9. [17]

    Uncovering the benefits and challenges of continuous integra- tion practices,

    O. Elazhary, C. Werner, Z. S. Li, D. Lowlind, N. A. Ernst, and M.-A. Storey, “Uncovering the benefits and challenges of continuous integra- tion practices,” IEEE Transactions on Software Engineering , vol. 48, no. 7, pp. 2570–2583, 2021

    2021

  10. [18]

    Graphical user interface (gui) testing: Systematic mapping and repository,

    I. Banerjee, B. Nguyen, V . Garousi, and A. Memon, “Graphical user interface (gui) testing: Systematic mapping and repository,” Information and Software Technology, vol. 55, no. 10, pp. 1679–1694, 2013

    2013

  11. [19]

    Hindsight: Understanding the evolution of ui vulnerabilities in mobile browsers,

    M. Luo, O. Starov, N. Honarmand, and N. Nikiforakis, “Hindsight: Understanding the evolution of ui vulnerabilities in mobile browsers,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 149–162, 2017

    2017

  12. [20]

    A survey of the selenium ecosystem,

    B. Garc ´ıa, M. Gallego, F. Gort ´azar, and M. Munoz-Organero, “A survey of the selenium ecosystem,” Electronics, vol. 9, no. 7, p. 1067, 2020

    2020

  13. [21]

    Octoverse 2022: The state of open source software,

    GitHub, “Octoverse 2022: The state of open source software,” 2022

    2022

  14. [22]

    Open source security and risk analysis report,

    Synopsys, “Open source security and risk analysis report,” 2023

    2023

  15. [23]

    Chronicles of ci/cd: A deep dive into its usage over time,

    H. Da Gi ˜ao, A. Flores, R. Pereira, and J. Cunha, “Chronicles of ci/cd: A deep dive into its usage over time,” arXiv preprint arXiv:2402.17588, 2024

    2024 arXiv

  16. [24]

    How do software developers use github actions to automate their workflows?,

    T. Kinsman, M. Wessel, M. A. Gerosa, and C. Treude, “How do software developers use github actions to automate their workflows?,” in 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pp. 420–431, IEEE, 2021

    2021

  17. [25]

    Which web testing tool should i use?,

    A. Knight, “Which web testing tool should i use?,” Automation Panda: A blog for software development and testing, 2023. https://automationpanda.com/2023/04/24/ which-web-testing-tool-should-i-use/

    2023

  18. [26]

    Cypress vs. selenium vs. playwright: Choosing the right automation testing tool,

    Kosalai, “Cypress vs. selenium vs. playwright: Choosing the right automation testing tool,” Axelerant, 2023. https://www.axelerant.com/ blog/cypress-selenium-playwright

    2023

  19. [27]

    Automation testing frame- work for web applications with selenium webdriver: Opportunities and threats,

    E. Vila, G. Novakova, and D. Todorova, “Automation testing frame- work for web applications with selenium webdriver: Opportunities and threats,” in Proceedings of the International Conference on Advances in Image Processing, pp. 144–150, 2017

    2017

  20. [28]

    How to use selenium as life-saver when dealing with boring tasks,

    S. K. R, “How to use selenium as life-saver when dealing with boring tasks,” Towards Data Science , 2019. https://towardsdatascience.com/how-to-use-selenium-as-life-saver- when-dealing-with-boring-tasks-ca264d1ce88

    2019

  21. [29]

    Improving the testing efficiency of selenium-based load tests,

    S. M. Shariff, H. Li, C.-P. Bezemer, A. E. Hassan, T. H. Nguyen, and P. Flora, “Improving the testing efficiency of selenium-based load tests,” in 2019 IEEE/ACM 14th International Workshop on Automation of Software Test (AST) , pp. 14–20, IEEE, 2019

    2019

  22. [30]

    Selenium-jupiter: A junit 5 extension for selenium webdriver,

    B. Garc ´ıa, C. D. Kloos, C. Alario-Hoyos, and M. Munoz-Organero, “Selenium-jupiter: A junit 5 extension for selenium webdriver,” Journal of Systems and Software , vol. 189, p. 111298, 2022

    2022

  23. [31]

    Wait, wait. no, tell me. analyzing selenium configuration effects on test flakiness,

    K. Presler-Marshall, E. Horton, S. Heckman, and K. Stolee, “Wait, wait. no, tell me. analyzing selenium configuration effects on test flakiness,” in 2019 IEEE/ACM 14th International Workshop on Automation of Software Test (AST), pp. 7–13, IEEE, 2019

    2019

  24. [32]

    Challenges of end-to-end testing with selenium webdriver and how to face them: A survey,

    M. Leotta, B. Garc ´ıa, F. Ricca, et al., “Challenges of end-to-end testing with selenium webdriver and how to face them: A survey,” in2023 IEEE Conference on Software Testing, Verification and Validation (ICST) , pp. 339–350, IEEE, 2023

    2023

  25. [33]

    Automating functional tests using sele- nium,

    A. Holmes and M. Kellogg, “Automating functional tests using sele- nium,” in AGILE 2006 (AGILE’06) , pp. 6–pp, IEEE, 2006

    2006

  26. [34]

    M. T. Taky, Automated Testing With Cypress. PhD thesis, 2021

    2021

  27. [35]

    Using test automation frameworks to ensure the stability of graphical user interface application,

    A. Cao, “Using test automation frameworks to ensure the stability of graphical user interface application,” 2022

    2022

  28. [36]

    A comparative study of ui testing framework,

    E. Pelivani, A. Besimi, and B. Cico, “A comparative study of ui testing framework,” in 2022 11th Mediterranean Conference on Embedded Computing (MECO), pp. 1–5, IEEE, 2022

    2022

  29. [37]

    A comparative analysis of playwright adoption vs cypress and selenium,

    L. Gagan, “A comparative analysis of playwright adoption vs cypress and selenium,” Rayrun, 2023. https://ray.run/blog/ the-rapid-adoption-of-playwright-test-in-software-qa

    2023

  30. [38]

    Exploring browser automation: A comparative study of selenium, cypress, puppeteer, and playwright,

    B. Garc ´ıa, J. M. del Alamo, M. Leotta, and F. Ricca, “Exploring browser automation: A comparative study of selenium, cypress, puppeteer, and playwright,” in Quality of Information and Communications Technology (A. Bertolino, J. Pascoal Faria, P. Lago, and L. Semini, eds.), (...

    2024

  31. [39]

    Test automation with playwright: Definition and benefits of this testing framework,

    O. Sheremeta, “Test automation with playwright: Definition and benefits of this testing framework,” Testomat.io, 2023. https://testomat.io/blog/test-automation-with-playwright-definition- and-benefits-of-this-testing-framework/

    2023

  32. [40]

    Comparison of different ci/cd tools integrated with cloud platform,

    C. Singh, N. S. Gaba, M. Kaur, and B. Kaur, “Comparison of different ci/cd tools integrated with cloud platform,” in 2019 9th International Conference on Cloud Computing, Data Science & Engineering (Conflu- ence), pp. 7–12, 2019

    2019

  33. [41]

    On the rise and fall of ci services in github,

    M. Golzadeh, A. Decan, and T. Mens, “On the rise and fall of ci services in github,” in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) , pp. 662–672, IEEE, 2022

    2022

  34. [42]

    Github actions: The impact on the pull request process,

    M. Wessel, J. Vargovich, M. A. Gerosa, and C. Treude, “Github actions: The impact on the pull request process,” arXiv preprint arXiv:2206.14118, 2022

    2022 arXiv

  35. [43]

    Oops, my tests broke the build: An explorative analysis of travis ci with github,

    M. Beller, G. Gousios, and A. Zaidman, “Oops, my tests broke the build: An explorative analysis of travis ci with github,” in 2017 IEEE/ACM 14th International conference on mining software repositories (MSR) , pp. 356–367, IEEE, 2017

    2017

  36. [44]

    Implementation of continuous integration and continuous delivery (ci/cd) on automatic performance testing,

    M. R. Pratama and D. S. Kusumo, “Implementation of continuous integration and continuous delivery (ci/cd) on automatic performance testing,” in 2021 9th International Conference on Information and Communication Technology (ICoICT), pp. 230–235, IEEE, 2021

    2021

  37. [45]

    Implementation of continuous integration and automated testing in software development of smart grid scheduling support system,

    J. Lu, Z. Yang, and J. Qian, “Implementation of continuous integration and automated testing in software development of smart grid scheduling support system,” in 2014 International Conference on Power System Technology, pp. 2441–2446, 2014

    2014

  38. [46]

    On rapid releases and software testing: a case study and a semi-systematic literature review,

    M. V . M¨antyl¨a, B. Adams, F. Khomh, E. Engstr¨om, and K. Petersen, “On rapid releases and software testing: a case study and a semi-systematic literature review,” Empirical Software Engineering , vol. 20, pp. 1384– 1425, 2015

    2015

  39. [47]

    Sourcegraph: Universal code search and intelligence,

    B. Liu, “Sourcegraph: Universal code search and intelligence,” 2020. https://www.infoworld.com/article/3514213/sourcegraph-universal- code-search-and-intelligence.html

    2020

  40. [48]

    The promises and perils of mining github,

    E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German, and D. Damian, “The promises and perils of mining github,” in Proceedings of the 11th working conference on mining software repositories , pp. 92– 101, 2014

    2014

  41. [49]

    McDowall, R

    D. McDowall, R. McCleary, and B. J. Bartos, Interrupted time series analysis. Oxford University Press, 2019

    2019

  42. [50]

    Time series methods in reliability,

    L. Walls and A. Bendell, “Time series methods in reliability,” Reliability Engineering, vol. 18, no. 4, pp. 239–265, 1987

    1987

  43. [51]

    R. M. Warner, Spectral analysis of time-series data . Guilford Press, 1998

    1998

  44. [52]

    I’m leaving you, travis: a continuous integration breakup story,

    D. G. Widder, M. Hilton, C. K ¨astner, and B. Vasilescu, “I’m leaving you, travis: a continuous integration breakup story,” in Proceedings of the 15th International Conference on Mining Software Repositories , pp. 165–169, 2018

    2018

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.