Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

An Empirical Study of Complexity, Heterogeneity, and Compliance of GitHub Actions Workflows

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that most GitHub Actions workflows are small, shallow pipelines shaped by platform defaults and ecosystem conventions rather than by best practices, with a 39.5% common prefix and widespread compliance gaps.

desk verdict A results-bearing abstract sits on top of a future-tense research plan; the empirical findings are asserted but not present in the body. read the letter →

arxiv 2507.18062 v2 pith:YZ7QJQ4N submitted 2025-07-24 cs.SE

Edward Abrokwah , Taher A. Ghaleb This is my paper
classification cs.SE
keywords ContinuousIntegrationGitHubActionsCIworkflowspipelinesbestpracticesworkflowcomplexityheterogeneitycompliance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper analyzes 27,863 GitHub Actions workflow files from Java, Python, and C++ repositories to test whether real-world CI pipelines live up to the simplicity and best-practice goals of continuous integration. The authors claim that workflows are generally small and shallow, lean heavily on external actions, and show little sequence-level standardization even though recurring intent-level patterns exist. They report that a common pipeline prefix appears in 39.5% of workflows, while only one full step sequence exceeds the 5% global frequency threshold, indicating high heterogeneity. They also find that Java has the lowest explicit test adoption, Python follows canonical templates but with weaker security practices, and C++ workflows are larger and more structurally diverse; compliance gaps are widespread, especially in permissions, timeout configuration, and SHA pinning. Taking these results together, the paper argues that workflow design is driven more by ecosystem conventions and platform defaults than by documented best practices, which points to better defaults and tooling as the main levers for improvement.

What carries the argument

The load-bearing machinery is a two-stage labeling procedure. The authors manually code a statistically significant, stratified sample of 382 workflows (95% confidence, ±5% margin of error) to define keyword- and regular-expression-based rules, then apply those rules to all 27,863 workflow files to label step types, extract step sequences, and flag best-practice violations. A pattern counts as common only if it appears in at least 5% of the dataset, and the canonical pipeline 'checkout → setup → test → deploy' from GitHub's documentation serves as the reference for measuring deviations. This procedure produces the headline percentages for the common prefix, sequence heterogeneity, language differences, and compliance gaps.

What would settle it

Take a fresh stratified sample of workflows not used to build the rules, manually label the intent of every step, and compare with the automated labels; if per-step agreement is much lower than the reported figures imply, or if re-estimating the 39.5% common-prefix share on the cleaned labels moves it outside the claimed margin, the quantitative claims fail. As a second check, inspect Java workflows flagged as lacking explicit test steps to see whether tests are invoked through build-phase commands rather than a step explicitly named 'test'.

Watch

Extended reading notes

Core claim

The central claim, on the paper's own terms, is that GitHub Actions workflows in open-source Java, Python, and C++ projects are conservative in structure: most are small, shallow pipelines assembled from external marketplace actions, with limited standardization at the exact-step-sequence level. The paper reports that 39.5% of workflows share a common pipeline prefix yet almost no full sequence is common at the 5% threshold. Language-specific patterns emerge: Java shows the lowest explicit test adoption, Python workflows follow canonical templates but with weaker security practices, and C++ workflows are larger and more structurally diverse. Compliance gaps appear widespread, particularly missing or overly broad permissions, absent timeout configuration, and unpinned action versions, while reusable workflows remain rare. The authors interpret the prevalence of build-without-test pipelines as evidence that workflow design is driven more by platform defaults and ecosystem conventions than by best practices.

Load-bearing premise

The headline numbers depend on keyword- and regex-based rules derived from 382 manually reviewed workflows being accurate enough when applied to all 27,863 workflows; the paper does not provide the rule definitions, validation results, or error analysis needed to confirm that those rules generalize.

Editorial extensions

If this is right

  • If workflow design is driven by platform defaults and ecosystem conventions, then changing GHA starter templates and default settings would directly reshape a large fraction of real-world pipelines.
  • The 39.5% common-prefix figure means a single shared opening sequence reaches tens of thousands of workflows, so improvements to those early steps would propagate widely.
  • Java's low explicit test adoption indicates that language-specific build culture shapes CI structure, so compliance guidance needs to be tuned per ecosystem rather than issued once.
  • The cross-language differences imply that researchers analyzing CI systems should account for language- and repository-level factors.
  • Widespread permission, timeout, and SHA-pinning gaps imply most workflows inherit permissive or insecure defaults rather than make deliberate choices; stricter defaults would close most of the gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not trace where the common pipeline prefix comes from; a natural follow-up would test whether it originates in GitHub's starter templates, copy-paste between repositories, or documentation examples.
  • Measuring 'explicit test adoption' by the presence of a step labeled as testing may miss tests that run inside language-specific build phases, so Java's low ranking could partly be a labeling artifact rather than a true absence of testing; this is an editorial caution, not a claim the paper makes.
  • A testable extension of the compliance result: if GHA made stricter permissions and required timeouts the default, most workflow authors would likely accept them, because the observed gaps appear to come from inattention rather than deliberate configuration.
  • Cross-language comparisons in this paper could be confounded by project age and size, since Java, Python, and C++ repositories differ in those characteristics; researchers building CI datasets should control for them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript describes a planned empirical study of GitHub Actions (GHA) workflow complexity, heterogeneity, and compliance in 7,668 Java, Python, and C++ repositories containing 27,863 workflow files (Table I). It defines three research questions and a six-phase execution plan in Section IV. The abstract supplied with the paper reports completed quantitative findings: a common pipeline prefix in 39.5% of workflows, only one sequence above the 5% frequency threshold, Java having the lowest explicit test adoption, C++ workflows being larger and more structurally diverse, and widespread compliance gaps in permissions, timeout configuration, and SHA pinning. None of these results appear in Sections IV–VI; the abstract inside the full text is written in the future tense ("This study will investigate"), Section IV is an execution plan, Section V describes threats to validity that "we will address", and Section VI says findings "are expected to reveal". The only concrete empirical artifact is the dataset summary in Table I. The classification rules that would generate the headline numbers are not defined, and no statistical tests, confidence intervals, or effect sizes are reported. Thus, the paper's central empirical claims are unsupported by the manuscript's content.

Significance. If the planned analyses were carried out and validated, this study could provide a useful empirical baseline for GHA workflow design, combining complexity metrics, pattern mining, and compliance checks across three language ecosystems. The dataset-construction criteria (non-archived, non-fork, at least 10 stars, at least 50 GHA builds) are reasonable, and the planned use of Mann-Whitney U tests and Cliff's delta for non-parametric comparison is methodologically sound. However, as submitted, the manuscript delivers none of the claimed findings and lacks the rule definitions and validation needed to audit them. The result is that the contribution cannot be assessed, replicated, or falsified. The strengths of the paper are strengths of a study plan, not of a completed empirical study.

major comments (4)
  1. [Abstract / Sections IV–VI] The abstract's empirical claims—39.5% common pipeline prefix, only one sequence exceeding 5%, Java's lowest test adoption, C++ workflows larger and more diverse, and compliance gaps in permissions/timeouts/SHA pinning—are not reported anywhere in the body. Section IV is an "Execution Plan" written in the future tense, Section V says threats "we will address", and Section VI concludes that findings "are expected to reveal" results. Table I is the only result-bearing artifact. Because the central contribution of an empirical paper is its findings, the absence of all findings makes the abstract's claims unverifiable and unjustified.
  2. [Section III-B, RQ2 approach; Section IV Phase 3] The rule-based classification procedure is underspecified. The authors state that after manually analyzing 382 workflows (95% confidence, +/-5% margin) they will define regular-expression and keyword rules to automatically label components across the full dataset, but the rules are never given, and there is no report of coverage, precision, recall, inter-rater agreement, or error analysis for the automated labeling. The 39.5% prefix figure and all compliance percentages would depend on this classifier, so the reader cannot determine whether those numbers are valid or robust to small changes in rule definitions.
  3. [Section III-B, RQ3 approach; Section IV Phase 4] The compliance analysis equates any deviation from GHA documentation with a "bad practice" and assigns severity on a 1–5 scale, but the operational definitions of "deviation", the rubric for severity ratings, and the thresholds for classifying permissions, timeouts, and SHA pinning are not specified. The 5% threshold for calling a pattern "common" is also stated without justification or sensitivity analysis. Without these definitions, the claimed "widespread compliance gaps" cannot be independently computed, and the statement that only one sequence exceeds the threshold is definitionally tied to an arbitrary cutoff.
  4. [Section VI, Conclusion] The conclusion confirms that the study has not been completed: it states that findings "are expected to reveal" both strong adherence and gaps, and it defers temporal evolution and developer-survey questions to future work. A manuscript whose conclusion is in the future tense cannot support the past-tense quantitative claims of the supplied abstract. This is a load-bearing inconsistency, not a stylistic issue.
minor comments (5)
  1. [Table I] Table I reports only raw counts and averages; presenting per-language distributions (e.g., median and interquartile range of YAML files per project) would help readers evaluate the comparability of the three language groups.
  2. [References] Reference [12] is cited in Section II-B as an Android CI empirical study, but the title mentions "AI-driven predictive models for continuous integration and deployment pipelines"; please verify that this reference supports the claim.
  3. [Throughout] The text contains formatting issues such as "Y AML" instead of "YAML" and "I NTRODUCTION" with an extra space; these should be cleaned up.
  4. [Section III-B RQ2] The phrase "statistically significant sample" is imprecise; the authors likely mean a sample chosen to achieve a target confidence level and margin of error, and the justification for the 382-file sample size is not shown.
  5. [Section IV Phase 1] The paper does not state whether the dataset and analysis scripts will be made available; a data-availability statement would support the "reproducible empirical baseline" promised in the supplied abstract.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild definitional circularity in the compliance measure; otherwise the paper is a plan whose abstract claims are unsupported rather than circular.

  1. self definitional [Section III-B (RQ3 Approach); Abstract, paragraph 3]
    ""We will treat any deviation from the GHA documentation as a bad practice (e.g. missing version pinning or improper secret handling)." ... "Compliance gaps are widespread, especially in permissions, timeout configuration, and SHA pinning.""

    The compliance-gap result is a restatement of the coding rule: the paper defines bad practice as any deviation from GHA documentation and then reports the frequency of those deviations as 'compliance gaps.' The conclusion that workflows are noncompliant is forced by the operationalization (deviation = bad) rather than by an independent standard. The counts themselves are empirical, so this is a mild, label-level circularity, not a fitted-parameter reduction.

full rationale

The manuscript is an execution plan: Section IV is written in the future tense ('We will analyze', 'we will develop rules') and Section III-B describes planned procedures (manual open coding of 382 workflows, keyword rules, 5% common-pattern threshold). The abstract, however, reports finished statistics (39.5% prefix, Java lowest test adoption, C++ larger/diverse, widespread compliance gaps) that are nowhere derived in the body. That is a serious unsupported-claim / internal-completeness problem, but it is not a circular derivation. No quantity is fitted and then renamed as a prediction, and the self-citations (e.g., [5], [13]-[15]) are related-work context, not load-bearing justifications. The only mild circular element is the compliance operationalization: 'any deviation from the GHA documentation' is defined as a bad practice, so the 'compliance gaps are widespread' conclusion is a definitional consequence of that coding rule. The 5% 'common' threshold is similarly a hand-selected convention; reporting how many patterns exceed a self-chosen cutoff is threshold-dependent, but not a circular prediction. Under the hard rules, the score is 2 rather than 0 only because of this definitional labeling in the compliance measure.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new entities are introduced. The central claim depends on a hand-chosen 5% pattern-frequency threshold and on assumptions that the repository sample is representative and that static YAML parsing plus keyword rules capture workflow semantics and compliance. The paper itself acknowledges the selection-bias threat in Section V-B.

free parameters (1)
  • Common pattern frequency threshold = 5%
    The paper defines a step sequence as 'common' only if it occurs in at least 5% of the dataset (Section III-B, RQ2). The abstract's statement that only one sequence exceeds the 5% threshold is a direct consequence of this hand-chosen cutoff rather than an independent property of the data.
assumptions (3)
  • domain assumption GitHub REST API dominant-language labeling plus the filters (non-archive, non-fork, >=10 stars, >=50 build runs) produces a sample representative of real-world GHA workflows
    Sections III-A and V-B. The paper itself acknowledges this selection may bias complexity upward.
  • domain assumption Static YAML parsing and keyword-based rules can faithfully classify workflow steps and best-practice compliance at scale
    Section III-B, RQ2 and Phase 3: rules are derived from a manual sample of 382 workflows and then applied to all 27,863 files. No validation plan is given.
  • ad hoc to paper Deviations from GHA documentation constitute bad practice
    Section III-B, RQ3: 'We will treat any deviation from the GHA documentation as a bad practice.' This makes the compliance measure definitional, not an external benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Empirical Study of Complexity, Heterogeneity, and Compliance of GitHub Actions Workflows." pith.science (2026). https://pith.science/paper/YZ7QJQ4N

@misc{pith2026250718062,
  author       = {Pith},
  title        = {Pith review of: An Empirical Study of Complexity, Heterogeneity, and Compliance of GitHub Actions Workflows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YZ7QJQ4N}},
  note         = {Machine review of arXiv:2507.18062}
}
read the original abstract

Continuous Integration (CI) has become a core practice in modern software engineering, enabling rapid and collaborative software delivery. GitHub Actions (GHA) has become a leading CI platform due to its tight GitHub integration and growing ecosystem of reusable workflows. Despite extensive documentation and best practices, there is limited empirical understanding of how real-world GHA workflows align with recommended guidelines. This study analyzes the structure, complexity, heterogeneity, and compliance of GHA workflows across Java, Python, and C++ repositories. We (a) quantify workflow complexity, (b) identify recurring and diverse structural patterns, (c) evaluate compliance with best practices, and (d) compare workflow design across languages. GHA workflows are generally small, shallow, and heavily dependent on external actions, with limited sequence-level standardization despite recurring intent-level patterns. A common pipeline prefix appears in 39.5% of workflows, but workflow sequences are highly heterogeneous, with only one exceeding the 5% global frequency threshold. We observe that Java has the lowest explicit test adoption, Python follows canonical templates but shows weaker security practices, and C++ workflows are larger and more structurally diverse. Compliance gaps are widespread, especially in permissions, timeout configuration, and SHA pinning, while reusable workflows remain rare. Build-without-test patterns suggest that workflow design is driven more by ecosystem conventions and platform defaults than by best practices. This indicates that better defaults and tooling could improve workflow security, modularity, and maintainability, while researchers should account for language- and repository-level factors when analyzing CI systems. Overall, this work provides a reproducible empirical baseline for studying and improving GHA workflow design in open-source ecosystems.

Figures

Figures reproduced from arXiv: 2507.18062 by the authors.

Figure 1
Figure 1. Overview of our study B. Research Questions The primary goal of this study is to investigate the com￾plexity, degree of heterogeneity, and compliance with core CI principles within GHA workflows. To achieve this, we propose the following research questions, each of which will incorporate comparative analyses of Java, Python, and C++ projects to explore language-specific practices and variations. 1) RQ1: How complex … view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Studying Developer Perceptions on the Potential of CI Recommendation Systems

    cs.SE 2026-08 conditional novelty 4.0 of 10

    The paper presents a study design, not results: a survey protocol targeting 250-500 GitHub developers to measure CI adoption motives, barriers, and perceptions of CI recommendation systems.

Reference graph

Works this paper leans on

33 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Continuous integration,

    M. Fowler, “Continuous integration,” https://martinfowler.com/articles/ originalContinuousIntegration.html, accessed: 2025-06-02

    work page 2025

  2. [2]

    The hidden costs of automation: An empirical study on GitHub Actions workflow maintenance,

    P. Valenzuela-Toledo, A. Bergel, T. Kehrer, and O. Nierstrasz, “The hidden costs of automation: An empirical study on GitHub Actions workflow maintenance,” in 2024 IEEE International Conference on Source Code Analysis and Manipulation (SCAM) . IEEE, 2024, pp. 213–223

    work page 2024

  3. [3]

    On the rise and fall of CI services in GitHub,

    M. Golzadeh, A. Decan, and T. Mens, “On the rise and fall of CI services in GitHub,” in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2022, pp. 662– 672

    work page 2022

  4. [4]

    The GitHub de- velopment workflow automation ecosystems,

    M. Wessel, T. Mens, A. Decan, and P. R. Mazrae, “The GitHub de- velopment workflow automation ecosystems,” in Software Ecosystems: Tooling and Analytics. Springer, 2023, pp. 183–214

    work page 2023

  5. [5]

    CI/CD Configuration Practices in Open-Source Android Apps: An Empirical Study

    T. A. Ghaleb, O. Abduljalil, and S. Hassan, “CI/CD configuration practices in open-source android apps: An empirical study,” arXiv preprint arXiv:2411.06077, 2025

    work page Pith review arXiv 2025

  6. [6]

    actions/starter-workflows: Accelerating new GitHub Ac- tions workflows,

    GitHub, “actions/starter-workflows: Accelerating new GitHub Ac- tions workflows,” https://docs.github.com/en/actions/concepts/overview/ about-continuous-integration-with-github-actions, 2024, gitHub Repos- itory

    work page 2024

  7. [7]

    An explorative analysis of managed CI/CD usage among open- source C/C++ projects,

    ´A. Kiss, “An explorative analysis of managed CI/CD usage among open- source C/C++ projects,” Production Systems and Information Engineer- ing, vol. 10, no. 3, pp. 19–30, 2022

    work page 2022

  8. [8]

    Use and Misuse of Continuous Integration Features: An Empirical Study of Projects that (mis)use Travis CI,

    K. Gallaba and S. McIntosh, “Use and Misuse of Continuous Integration Features: An Empirical Study of Projects that (mis)use Travis CI,” IEEE Transactions on Software Engineering , vol. 46, no. 1, p. 33–50, 2020

    work page 2020

Show all 33 references
  1. [9]

    Developers’ perception of GitHub Actions: A survey analysis,

    S. G. Saroar and M. Nayebi, “Developers’ perception of GitHub Actions: A survey analysis,” in Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering , 2023, pp. 121– 130

    2023

  2. [10]

    Empirical analysis of CI/CD tools usage in GitHub Actions workflows,

    A. R. Faqih, A. Taufiqurrahman, J. H. Husen, and M. K. Sabariah, “Empirical analysis of CI/CD tools usage in GitHub Actions workflows,” Journal of Informatics and Web Engineering, vol. 3, no. 2, pp. 251–261, 2024

    2024

  3. [11]

    Catching smells in the act: A GitHub Actions workflow investigation,

    A. Khatami, C. Willekens, and A. Zaidman, “Catching smells in the act: A GitHub Actions workflow investigation,” in 2024 IEEE International Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 2024, pp. 47–58

    2024

  4. [12]

    Enhancing DevOps efficiency through AI-driven pre- dictive models for continuous integration and deployment pipelines,

    A. Enemosah, “Enhancing DevOps efficiency through AI-driven pre- dictive models for continuous integration and deployment pipelines,” International Journal of Research Publication and Reviews, vol. 6, no. 1, pp. 871–887, 2025

    2025

  5. [13]

    An empirical study of the long duration of continuous integration builds,

    T. A. Ghaleb, D. A. Da Costa, and Y . Zou, “An empirical study of the long duration of continuous integration builds,” Empirical Software Engineering, vol. 24, no. 4, pp. 2102–2139, 2019

    2019

  6. [14]

    Studying the interplay between the durations and breakages of continuous integration builds,

    T. A. Ghaleb, S. Hassan, and Y . Zou, “Studying the interplay between the durations and breakages of continuous integration builds,” IEEE Transactions on Software Engineering , vol. 49, no. 4, pp. 2476–2497, 2022

    2022

  7. [15]

    Studying the impact of noises in build breakage data,

    T. A. Ghaleb, D. A. Da Costa, Y . Zou, and A. E. Hassan, “Studying the impact of noises in build breakage data,” IEEE Transactions on Software Engineering, vol. 47, no. 9, pp. 1998–2011, 2019

    1998

  8. [16]

    An empirical characterization of bad practices in contin- uous integration,

    F. Zampetti, C. Vassallo, S. Panichella, G. Canfora, H. Gall, and M. Di Penta, “An empirical characterization of bad practices in contin- uous integration,” Empirical Software Engineering , vol. 25, pp. 1095– 1135, 2020

    2020

  9. [17]

    Resource usage and optimization oppor- tunities in workflows of Github Actions,

    I. Bouzenia and M. Pradel, “Resource usage and optimization oppor- tunities in workflows of Github Actions,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering , 2024, pp. 1–12

    2024

  10. [18]

    Empirical analysis on CI/CD pipeline evolution in machine learning projects,

    D. E. Rzig, A. Houerbi, R. G. Chavan, and F. Hassan, “Empirical analysis on CI/CD pipeline evolution in machine learning projects,” arXiv preprint arXiv:2403.12199 , 2024

    2024 arXiv

  11. [19]

    Travistorrent: Synthesizing Travis CI and GitHub for full-stack research on continuous integration,

    M. Beller, G. Gousios, and A. Zaidman, “Travistorrent: Synthesizing Travis CI and GitHub for full-stack research on continuous integration,” in 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 2017, pp. 447–450

    2017

  12. [20]

    Using software metrics for predicting vulnerable code-components: A study on Java and Python open source projects,

    T.-Y . Chong, V . Anu, and K. Z. Sultana, “Using software metrics for predicting vulnerable code-components: A study on Java and Python open source projects,” in 2019 ieee international conference on compu- tational science and engineering (cse) and ieee international conferen...

    2019

  13. [21]

    Getting started with GitHub Actions workflows,

    C. Chandrasekara, P. Herath, C. Chandrasekara, and P. Herath, “Getting started with GitHub Actions workflows,” Hands-on GitHub Actions: Implement CI/CD with GitHub Action Workflows for Your Applications , pp. 9–28, 2021

    2021

  14. [22]

    Grounded theory research: Procedures, canons, and evaluative criteria,

    J. M. Corbin and A. Strauss, “Grounded theory research: Procedures, canons, and evaluative criteria,” Qualitative sociology , vol. 13, no. 1, pp. 3–21, 1990

    1990

  15. [23]

    Learning-based identification of coding best practices from software documentation,

    N. Sawant and S. H. Sengamedu, “Learning-based identification of coding best practices from software documentation,” in2022 IEEE Inter- national Conference on Software Maintenance and Evolution (ICSME) . IEEE, 2022, pp. 533–542

    2022

  16. [24]

    Security misconfigurations in open source Kubernetes manifests: An empirical study,

    A. Rahman, S. I. Shamim, D. B. Bose, and R. Pandita, “Security misconfigurations in open source Kubernetes manifests: An empirical study,” ACM Transactions on Software Engineering and Methodology , vol. 32, no. 4, pp. 1–36, 2023

    2023

  17. [25]

    Automatic security assess- ment of GitHub Actions workflows,

    G. Benedetti, L. Verderame, and A. Merlo, “Automatic security assess- ment of GitHub Actions workflows,” in Proceedings of the 2022 ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses, 2022, pp. 37–45

    2022

  18. [26]

    Developer mistakes in writing Android manifests: An empirical study of configuration errors,

    A. K. Jha, S. Lee, and W. J. Lee, “Developer mistakes in writing Android manifests: An empirical study of configuration errors,” in 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 2017, pp. 25–36

    2017

  19. [27]

    Exploring complexity issues in junior developer code using static analysis and FCA,

    A.-J. Molnar, S. Motogna, D. Cristea, and D.-F. Sotropa, “Exploring complexity issues in junior developer code using static analysis and FCA,” in 2024 50th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) . IEEE, 2024, pp. 407–414

    2024

  20. [28]

    A metrics suite for static structure of large-scale software based on complex networks,

    H. Zhang, H. Zhao, W. Cai, M. Zhao, and G. Luo, “A metrics suite for static structure of large-scale software based on complex networks,” in 2008 International Conference on Intelligent Information Hiding and Multimedia Signal Processing . IEEE, 2008, pp. 512–515

    2008

  21. [29]

    Pattern matching in python: Expanding the horizons of engineering applications,

    M. Sibiya, “Pattern matching in python: Expanding the horizons of engineering applications,” in International Conference on Artificial Intelligence and its Applications , 2023, pp. 80–86

    2023

  22. [30]

    On a test of whether one of two random variables is stochastically larger than the other,

    H. B. Mann and D. R. Whitney, “On a test of whether one of two random variables is stochastically larger than the other,” The annals of mathematical statistics, pp. 50–60, 1947

    1947

  23. [31]

    Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys,

    J. Romano, J. D. Kromrey, J. Coraggio, and J. Skowronek, “Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys,” in annual meeting of the Florida Association of Institutiona...

    2006

  24. [32]

    R. J. Grissom and J. J. Kim, Effect Sizes for Research: A Broad Practical Approach. Mahwah, NJ: Lawrence Erlbaum Associates, 2005

    2005

  25. [33]

    On the use of GitHub Actions in software development repositories,

    A. Decan, T. Mens, P. R. Mazrae, and M. Golzadeh, “On the use of GitHub Actions in software development repositories,” in 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2022, pp. 235–245

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.