pith. sign in

arxiv: 2504.08678 · v4 · submitted 2025-04-11 · 💻 cs.SE · cs.HC

The Ultimate Configuration Management Tool? Lessons from a Mixed Methods Study of Ansible's Challenges

Carolina Carreira , Nuno Saavedra , Alexandra Mendes , Jo\~ao F. Ferreira This is my paper

Pith reviewed 2026-05-22 20:14 UTC · model grok-4.3

classification 💻 cs.SE cs.HC
keywords AnsibleInfrastructure as CodeIaCconfiguration managementuser challengesmixed methodsdebuggingperformance
0
0 comments X

The pith

A study of over 59,000 Ansible forum posts and 20 interviews identifies four concrete directions to fix common user struggles with debugging, language clarity, documentation, and speed.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to map the real difficulties that practitioners encounter when using Ansible to automate server and infrastructure setup. It does so by combining quantitative review of tens of thousands of public posts with qualitative interviews across different experience levels. The authors conclude that four targeted changes would ease the most frequent pain points. If those changes prove effective, both Ansible and similar automation tools could become easier to use and maintain in production environments. The work matters because configuration management tools now underpin much of modern IT operations.

Core claim

Based on analysis of 59,157 posts from Stack Overflow, Reddit, and the Ansible Forum plus 20 semi-structured interviews, the paper highlights key directions for improving Ansible including stronger failure locality to support debugging, clearer separation of language and templating boundaries, targeted documentation, and improved execution backends to address performance issues.

What carries the argument

Mixed-methods study that extracts recurring issues from large-scale forum data and validates them through practitioner interviews to derive specific improvement directions.

If this is right

  • Stronger failure locality would let users locate and fix errors more quickly during playbook runs.
  • Clearer boundaries between the core language and templating features would reduce confusion about what syntax is available where.
  • More targeted documentation would help practitioners find relevant guidance without sifting through unrelated material.
  • Faster execution backends would shorten run times for large inventories and complex playbooks.
  • The same four directions could guide improvements in other Infrastructure as Code tools.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Developers could prototype one of the suggested changes, such as enhanced error locality, and measure whether it reduces time spent on troubleshooting in real deployments.
  • Similar mixed-methods studies on other popular IaC tools might reveal whether the identified challenge categories are shared or tool-specific.
  • If documentation improvements are implemented, tracking changes in the volume and type of forum questions over time could test their impact.

Load-bearing premise

The chosen forum posts and the interviewed practitioners give a representative picture of challenges across the full range of Ansible users.

What would settle it

A follow-up study drawing from different data sources or a broader set of users that ranks substantially different problems as primary would show the original directions do not generalize.

Figures

Figures reproduced from arXiv: 2504.08678 by Alexandra Mendes, Carolina Carreira, Jo\~ao F. Ferreira, Nuno Saavedra.

Figure 1
Figure 1. Figure 1: Excerpt from a web server Ansible playbook. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the research methodology combining automated topic extraction and thematic analysis. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Topic distribution (Feb 2020–Feb 2025). Sparklines show yearly percentage trends across the full Ansible post history. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
read the original abstract

Infrastructure as Code (IaC) tools have transformed the way IT infrastructure is automated and managed, but their growing adoption has also exposed numerous challenges for practitioners. In this paper, we investigate these challenges through the lens of Ansible, a popular IaC tool. Using a mixed methods approach, we investigate challenges faced by practitioners. We analyze 59,157 posts from Stack Overflow, Reddit, and the Ansible Forum to identify common pain points, complemented by 20 semi-structured interviews with practitioners of varying expertise levels. Based on our findings, we highlight key directions for improving Ansible, with implications for other IaC technologies, including stronger failure locality to support debugging, clearer separation of language and templating boundaries, targeted documentation, and improved execution backends to address performance issues. By grounding these insights in the real-world struggles of Ansible users, this study provides actionable guidance for tool designers and for the broader IaC community, and contributes to a deeper understanding of the trade-offs inherent in IaC tools.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that a mixed-methods study of Ansible challenges—analyzing 59,157 posts from Stack Overflow, Reddit, and the Ansible Forum plus 20 semi-structured interviews—identifies recurring practitioner pain points and derives actionable improvement directions for Ansible (and by extension other IaC tools), specifically stronger failure locality to aid debugging, clearer separation between language and templating boundaries, targeted documentation, and improved execution backends to mitigate performance issues.

Significance. If the central claims hold, the work supplies empirically grounded, practitioner-derived guidance that could inform tool evolution in the IaC space and deepen understanding of inherent trade-offs in declarative configuration languages. The scale of the forum corpus combined with interview triangulation is a clear strength, offering broader coverage than purely qualitative studies while remaining tied to observable user struggles.

major comments (2)
  1. [§3] §3 (Methodology), Data Sources and Sampling subsection: the paper does not report any comparison of the 59,157 forum posts or the 20 interviewees against Ansible usage demographics, adoption statistics, or non-public support channels; without such checks or explicit bias-mitigation steps, the claim that the extracted challenges and recommended directions reflect central rather than self-selected pain points remains under-supported.
  2. [§4] §4 (Findings) and §5 (Implications): the mapping from the most frequent forum themes to the four headline improvement directions is presented without quantitative weighting or cross-validation against usage volume or satisfaction metrics, so it is unclear whether the proposed changes address load-bearing problems for the majority of users or primarily the subset who post publicly.
minor comments (2)
  1. [§3.3] §3.3 (Interview Analysis): the description of the qualitative coding process and inter-rater reliability measures is brief; expanding it would allow readers to assess consistency of theme extraction.
  2. [Figure 2] Figure 2 and Table 1: axis labels and category definitions could be clarified to make the distribution of challenge types immediately legible without cross-referencing the text.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which highlight important considerations for strengthening the generalizability of our mixed-methods findings. We respond to each major comment below and indicate planned revisions.

read point-by-point responses

  1. Referee: [§3] §3 (Methodology), Data Sources and Sampling subsection: the paper does not report any comparison of the 59,157 forum posts or the 20 interviewees against Ansible usage demographics, adoption statistics, or non-public support channels; without such checks or explicit bias-mitigation steps, the claim that the extracted challenges and recommended directions reflect central rather than self-selected pain points remains under-supported.

    Authors: We acknowledge this limitation. Comprehensive public statistics on Ansible user demographics, adoption rates, or volumes in non-public support channels are not available from Red Hat or other sources. Forum posts are self-selected by definition, as users typically post when encountering difficulties. In the revised version we will add a dedicated Limitations subsection to §3 that explicitly discusses selection bias in public forums, notes the absence of demographic benchmarks, and explains how the 20 interviews provide triangulation to surface challenges beyond the most vocal posters. We cannot retroactively obtain proprietary usage data, so this will be framed as an inherent constraint of the study design. revision: partial

  2. Referee: [§4] §4 (Findings) and §5 (Implications): the mapping from the most frequent forum themes to the four headline improvement directions is presented without quantitative weighting or cross-validation against usage volume or satisfaction metrics, so it is unclear whether the proposed changes address load-bearing problems for the majority of users or primarily the subset who post publicly.

    Authors: The four directions were derived from the highest-frequency themes identified through inductive thematic analysis of the forum corpus (detailed with counts in §4) and cross-checked against interview data. We do not possess internal usage-volume or satisfaction metrics that would allow claims about the 'majority' of all Ansible users. In the revision we will update §5 to state the selection rationale more precisely (top-frequency themes in public discussions plus interview confirmation) and to qualify that the recommendations target commonly reported pain points among practitioners who seek help in public venues, rather than asserting impact on the broader user base. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical findings drawn directly from external data sources

full rationale

The paper performs a mixed-methods empirical study by collecting and analyzing 59,157 external forum posts plus 20 interviews to surface practitioner challenges. No derivations, equations, fitted parameters, or first-principles claims exist that could reduce to self-definition or self-citation. The central outputs (identified pain points and improvement directions) are direct summaries of the collected data rather than constructs that loop back to the paper's own inputs or prior self-citations. This is standard non-circular empirical work.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The study depends on standard assumptions in empirical software engineering about the validity of qualitative and mixed-methods data for identifying real user challenges.

axioms (1)
  • domain assumption Analysis of public forum posts and semi-structured interviews can reliably surface representative practitioner challenges in IaC tools.
    Invoked to justify generalizing from the collected data to broader Ansible usage issues.

pith-pipeline@v0.9.0 · 5714 in / 1190 out tokens · 89453 ms · 2026-05-22T20:14:54.308804+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Understanding Bugs in Template Engine-Based Applications: Symptoms, Root Causes, and Fix Patterns

    cs.SE 2026-04 unverdicted novelty 7.0

    An empirical study of 1,004 bugs in template engine-based applications finds abnormal rendering results as the most common symptom (48.61%) and documents 17 root causes with fix patterns that often involve host-side l...

  2. Nix: A Solution With Problems

    cs.SE 2026-04 unverdicted novelty 2.0

    A literature review of Nix's functional package management solutions to software deployment problems alongside the new and unsolved issues it introduces.

Reference graph

Works this paper leans on

80 extracted references · 80 canonical work pages · cited by 2 Pith papers

  1. [1]

    Emad Aghajani, Csaba Nagy, Mario Linares-Vásquez, Laura Moreno, Gabriele Bavota, Michele Lanza, and David C Shepherd. 2020. Software documentation: the practitioners’ perspective. InProceedings of the acm/ieee 42nd international conference on software engineering. 590–601

    work page 2020

  2. [2]

    Emad Aghajani, Csaba Nagy, Olga Lucero Vega-Márquez, Mario Linares-Vásquez, Laura Moreno, Gabriele Bavota, and Michele Lanza. 2019. Software documenta- tion issues unveiled. In2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 1199–1210

    work page 2019

  3. [3]

    Amazon Web Services, Inc. 2011. AWS CloudFormation: Infrastructure as Code & AWS Resource Provisioning. https://aws.amazon.com/cloudformation. Accessed: 2025-03-10

    work page 2011

  4. [4]

    Anonymous Author(s). 2025. Replication Package. https://doi.org/10.5281/ zenodo.15031088 Accessed 20-02-2025

    work page 2025

  5. [5]

    Ansible Community. 2016. Issue #18305. https://github.com/ansible/ansible/ issues/18305. GitHub issue

    work page 2016

  6. [6]

    Ansible Community. 2017. Issue #30411. https://github.com/ansible/ansible/ issues/30411. GitHub issue

    work page 2017

  7. [7]

    Ansible Community. 2024. Doubt regarding loops and conditionals. https: //forum.ansible.com/t/doubt-regarding-loops-and-conditionals/32257 Accessed 21-01-2026

    work page 2024

  8. [8]

    Ansible, Inc. 2026. Ansible.Builtin Collection Documentation. https://docs. ansible.com/projects/ansible/13/collections/ansible/builtin/ Accessed 21-01-2026

    work page 2026

  9. [9]

    Ansible Project. 2026. Ansible Commit: Deprecation Warning for Tem- plating Delimiters in Conditionals. https://github.com/ansible/ansible/ blob/35750ed3218e7bce68b21f473cecb0a3b9d60321/lib/ansible/_internal/ _templating/_engine.py#L477 Accessed 21-01-2026

    work page 2026

  10. [10]

    Ansible project contributors. 2025. Ansible Community Documentation. https: //docs.ansible.com/ansible/latest Accessed 03-03-2025

    work page 2025

  11. [11]

    Claus Boye Asmussen and Charles Møller. 2019. Smart literature review: a practical topic modelling approach to exploratory literature review.Journal of Big Data6, 1 (2019), 1–18

    work page 2019

  12. [12]

    Sebastian Baltes and Stephan Diehl. 2016. Worse than spam: Issues in sampling software developers. InProceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement. 1–6

    work page 2016

  13. [13]

    Mahi Begoug, Narjes Bessghaier, Ali Ouni, Eman Abdullah AlOmar, and Mo- hamed Wiem Mkaouer. 2023. What do infrastructure-as-code practitioners discuss: An empirical study on stack overflow. In2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, 1–12

    work page 2023

  14. [14]

    Mahi Begoug, Moataz Chouchen, Ali Ouni, Eman Abdullah Alomar, and Mo- hamed Wiem Mkaouer. 2024. Fine-grained just-in-time defect prediction at the block level in Infrastructure-as-Code (IaC). InProceedings of the 21st International Conference on Mining Software Repositories. 100–112

    work page 2024

  15. [15]

    Narjes Bessghaier, Mahi Begoug, Chemseddine Mebarki, Ali Ouni, Mohammed Sayagh, and Mohamed Wiem Mkaouer. 2024. On the prevalence, co-occurrence, and impact of infrastructure-as-code smells. In2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 23–34

    work page 2024

  16. [16]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology3, 2 (2006), 77–101

    work page 2006

  17. [17]

    Gordon Burtch, Dokyun Lee, and Zhichen Chen. 2024. The consequences of generative AI for online knowledge communities.Scientific Reports14, 1 (2024), 10413

    work page 2024

  18. [18]

    Barbara DiCicco-Bloom and Benjamin F Crabtree. 2006. The qualitative research interview.Medical education40, 4 (2006), 314–321

    work page 2006

  19. [19]

    James Fryman. 2014. DNS outage post mortem. https://github.blog/2014-01-18- dns-outage-post-mortem/ Accessed: 3 May 2022

    work page 2014

  20. [20]

    GitLab. 2014. Gitlab.com Downtime 2014-07-07 Postmortem. https://docs.google. com/document/d/1ScqXAdb6BjhsDzCo3qdPYbt1uULzgZqPO8zHeHHarS0 [Ac- cessed 04-09-2024]

    work page 2014

  21. [21]

    gooeyblob. 2016. Why Reddit was down on Aug 11. https://web.archive.org/ web/20221029203405/https://www.reddit.com/r/announcements/comments/ 4y0m56/why_reddit_was_down_on_aug_11/ [Accessed 04-09-2024]

    work page arXiv 2016

  22. [22]

    Michele Guerriero, Martin Garriga, Damian A Tamburri, and Fabio Palomba

    work page

  23. [23]

    In2019 IEEE International conference on software maintenance and evolution (ICSME)

    Adoption, support, and challenges of infrastructure-as-code: Insights from industry. In2019 IEEE International conference on software maintenance and evolution (ICSME). IEEE, 580–589

    work page

  24. [24]

    Hacker News User gizdan. 2021. Ansible Techniques I Wish I’d Known Ear- lier. https://web.archive.org/web/20220611223833/https://news.ycombinator. com/item?id=28332684#expand Accessed: 2025-03-14

    work page arXiv 2021

  25. [25]

    HashiCorp. 2024. Terraform by HashiCorp. https://www.terraform.io/ Accessed 13-09-2024

    work page 2024

  26. [26]

    Md Mahadi Hassan, John Salvador, Shubhra Kanti Karmaker Santu, and Akond Rahman. 2024. State Reconciliation Defects in Infrastructure as Code.Proceedings of the ACM on Software Engineering1, FSE (2024), 1865–1888

    work page 2024

  27. [27]

    Rebecca Hersher. 2017. Amazon and the $150 Million typo. https: //www.npr.org/sections/thetwo-way/2017/03/03/518322734/amazon-and-the- 150-million-typo?t=1651588365675 Accessed: 3 May 2022

    work page 2017

  28. [28]

    Honeycomb. 2021. Incident Resolution: Do You Remember, the Twenty Fires of September? https://www.honeycomb.io/blog/incident-resolution-september- retrospective [Accessed 04-09-2024]

    work page 2021

  29. [29]

    Victoria Jackson, André Van Der Hoek, and Rafael Prikladnicki. 2024. Co- Creation in Fully Remote Software Teams. In2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). 604–615. doi:10.1145/3597503.3623297

    work page doi:10.1145/3597503.3623297 2024

  30. [30]

    Carina Jacobi, Wouter Van Atteveldt, and Kasper Welbers. 2018. Quantitative analysis of large amounts of journalistic texts using topic modelling. InRethinking research methods in an age of digital journalism. Routledge, 89–106

    work page 2018

  31. [31]

    Jan David Nose. 2023. DNS Outage on 2023-01-25. https://blog.rust-lang.org/ inside-rust/2023/02/08/dns-outage-portmortem.html [Accessed 04-09-2024]

    work page 2023

  32. [32]

    Lena Karlsson, Åsa G Dahlstedt, Björn Regnell, Johan Natt och Dag, and Anne Persson. 2007. Requirements engineering challenges in market-driven software development–An interview study with practitioners.Information and Software technology49, 6 (2007), 588–604

    work page 2007

  33. [33]

    Thomas D LaToza and Brad A Myers. 2010. Developers ask reachability ques- tions. InProceedings of the 32Nd ACM/IEEE International Conference on Software Engineering-Volume 1. 185–194. Carreira et al

    work page 2010

  34. [34]

    2017.Research methods in human-computer interaction

    Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017.Research methods in human-computer interaction. Morgan Kaufmann

    work page 2017

  35. [35]

    Julien Lepiller, Ruzica Piskac, Martin Schäf, and Mark Santolucito. 2021. Ana- lyzing infrastructure as code to prevent intra-update sniping vulnerabilities. In Tools and Algorithms for the Construction and Analysis of Systems: 27th Interna- tional Conference, TACAS 2021, Held as Part of the European Joint Conferences on Theory and Practice of Software, E...

    work page 2021

  36. [36]

    Khalid Mahmood. 2016. Do people overestimate their information literacy skills? A systematic review of empirical evidence on the Dunning-Kruger effect. Communications in Information Literacy10, 2 (2016), 3

    work page 2016

  37. [37]

    Mitogen Contributors. 2025. Mitogen for Ansible. https://mitogen. networkgenomics.com/ansible_detailed.html Accessed 15-03-2025

    work page 2025

  38. [38]

    NASA Goddard Space Flight Center. 2023. SIGNAL: Summer 2023.SIGNALSum- mer 2023 (2023). https://tempo.gsfc.nasa.gov/static-files/SIGNAL%20Summer% 202023.pdf Accessed: 2025-03-10

    work page 2023

  39. [39]

    2024.Static Analysis for Quality Assurance of Ansible Infrastructure-as-Code Artefacts

    Ruben Opdebeeck. 2024.Static Analysis for Quality Assurance of Ansible Infrastructure-as-Code Artefacts. Ph. D. Dissertation. Vrije Universiteit Brussel

    work page 2024

  40. [40]

    Ruben Opdebeeck, Ahmed Zerouali, and Coen De Roover. 2022. Smelly variables in ansible infrastructure code: Detection, prevalence, and lifetime. InProceedings of the 19th International Conference on Mining Software Repositories. 61–72

    work page 2022

  41. [41]

    Ruben Opdebeeck, Ahmed Zerouali, and Coen De Roover. 2023. Control and data flow in security smell detection for infrastructure as code: Is it worth the effort?. In2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR). IEEE, 534–545

    work page 2023

  42. [42]

    Perforce. 2024. Puppet Infrastructure & IT Automation at Scale. https://www. puppet.com/ Accessed 13-09-2024

    work page 2024

  43. [43]

    Chau Pham, Alexander Hoyle, Simeng Sun, Philip Resnik, and Mohit Iyyer. 2024. TopicGPT: A Prompt-based Topic Modeling Framework. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2956–2984

    work page 2024

  44. [44]

    PhilEv and Ansible Community. 2017. New Warning in Ansible 2.3. https: //forum.ansible.com/t/new-warning-in-ansible-2-3/24330 Published April 26–27 2017; Accessed 21-01-2026

    work page 2017

  45. [45]

    Progress. 2024. Chef Software DevOps Automation Solutions. https://www.chef. io/ Accessed 13-09-2024

    work page 2024

  46. [46]

    PullPush team. 2025. PullPush Reddit API. https://pullpush.io/ Accessed 20-02-2025

    work page 2025

  47. [47]

    Akond Rahman, Effat Farhana, Chris Parnin, and Laurie Williams. 2020. Gang of eight: A defect taxonomy for infrastructure as code scripts. In2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 752–764

    work page 2020

  48. [48]

    Akond Rahman, Chris Parnin, and Laurie Williams. 2019. The seven sins: Security smells in infrastructure as code scripts. In2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 164–175

    work page 2019

  49. [49]

    Akond Rahman, Asif Partho, Patrick Morrison, and Laurie Williams. 2018. What questions do programmers ask about configuration as code?. InProceedings of the 4th International Workshop on Rapid Continuous Software Engineering. 16–22

    work page 2018

  50. [50]

    Akond Rahman, Md Rayhanur Rahman, Chris Parnin, and Laurie Williams. 2021. Security smells in ansible and chef scripts: A replication study.ACM Transactions on Software Engineering and Methodology (TOSEM)30, 1 (2021), 1–31

    work page 2021

  51. [51]

    Akond Rahman, Shazibul Islam Shamim, Dibyendu Brinto Bose, and Rahul Pandita. 2023. Security misconfigurations in open source kubernetes manifests: An empirical study.ACM Transactions on Software Engineering and Methodology 32, 4 (2023), 1–36

    work page 2023

  52. [52]

    Akond Rahman and Laurie Williams. 2018. Characterizing defective configu- ration scripts used for continuous deployment. In2018 IEEE 11th International conference on software testing, verification and validation (ICST). IEEE, 34–45

    work page 2018

  53. [53]

    Red Hat. 2024. Ansible. https://www.ansible.com/ Accessed 13-09-2024

    work page 2024

  54. [54]

    Red Hat. 2025. Ansible Forum. https://forum.ansible.com/ Accessed 20-02-2025

    work page 2025

  55. [55]

    Red Hat. 2025. Ansible Forum. https://forum.ansible.com/t/important-ansible- google-group-moving-to-ansible-forum/38853 Accessed 21-02-2025

    work page 2025

  56. [56]

    Red Hat. 2026. Ansible - Iterating over a Dictionary. https://docs.ansible.com/ projects/ansible/13/playbook_guide/playbooks_loops.html#iterating-over-a- dictionary Accessed 21-01-2026

    work page 2026

  57. [57]

    Red Hat. 2026. Ansible - Loops and Nested Loops. https://docs.ansible.com/ projects/ansible/13/playbook_guide/playbooks_loops.html#nested-loops Ac- cessed 21-01-2026

    work page 2026

  58. [58]

    2023.Siemens standardizes on Red Hat Ansible Automation Platform for DevOps

    Red Hat, Inc. 2023.Siemens standardizes on Red Hat Ansible Automation Platform for DevOps. Case Study. Red Hat, Inc. https://www.redhat.com/en/resources/ siemens-case-study Accessed: 2025-03-10

    work page 2023

  59. [59]

    Reddit. 2025. Reddit. https://www.reddit.com Accessed 20-02-2025

    work page 2025

  60. [60]

    Reddit Ansible Community. 2023. What’s the logic behind builtin failed/success- ful? https://www.reddit.com/r/ansible/comments/10rma2g/whats_the_logic_ behind_builtin_failedsuccessful/ Accessed 21-01-2026

    work page 2023

  61. [61]

    Sofia Reis, Rui Abreu, Marcelo d’Amorim, and Daniel Fortunato. 2022. Leveraging practitioners’ feedback to improve a security linter. InProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. 1–12

    work page 2022

  62. [62]

    Martin P Robillard. 2009. What makes APIs hard to learn? Answers from devel- opers.IEEE software26, 6 (2009), 27–34

    work page 2009

  63. [63]

    Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej. 2012. How do professional developers comprehend software?. In2012 34th International Conference on Software Engineering (ICSE). IEEE, 255–265

    work page 2012

  64. [64]

    Nuno Saavedra and João F Ferreira. 2022. GLITCH: Automated Polyglot Security Smell Detection in Infrastructure as Code. InProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

    work page 2022

  65. [65]

    Nuno Saavedra, João F Ferreira, and Alexandra Mendes. 2025. InfraFix: Technology-Agnostic Repair of Infrastructure as Code.arXiv preprint arXiv:2503.17220(2025)

    work page arXiv 2025

  66. [66]

    Nuno Saavedra, João Gonçalves, Miguel Henriques, João F Ferreira, and Alexan- dra Mendes. 2023. Polyglot Code Smell Detection for Infrastructure as Code with GLITCH. InProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering

    work page 2023

  67. [67]

    Julian Schwarz, Andreas Steffens, and Horst Lichter. 2018. Code smells in infras- tructure as code. In2018 11Th international conference on the quality of information and communications technology (QUATIC). IEEE, 220–228

    work page 2018

  68. [68]

    Tushar Sharma, Marios Fragkoulis, and Diomidis Spinellis. 2016. Does your configuration code smell?. InProceedings of the 13th international conference on mining software repositories. 189–200

    work page 2016

  69. [69]

    Thodoris Sotiropoulos, Dimitris Mitropoulos, and Diomidis Spinellis. 2020. Prac- tical fault detection in puppet programs. InProceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 26–37

    work page 2020

  70. [70]

    Stack Exchange. 2025. Stack Exchange API. https://api.stackexchange.com Accessed 20-02-2025

    work page 2025

  71. [71]

    Stack Exchange. 2025. Stack Overflow. https://stackoverflow.com Accessed 20-02-2025

    work page 2025

  72. [72]

    Stack Exchange Network. 2014. Outage Post-Mortem: August 25th,

    work page 2014

  73. [73]

    https://web.archive.org/web/20201020103424/https://stackstatus.net/post/ 96025967369/outage-post-mortem-august-25th-2014 [Accessed 04-09-2024]

    work page arXiv 2014

  74. [74]

    Stack Overflow. 2024. Stack Overflow Developer Survey 2024. https://survey. stackoverflow.co/2024/ [Accessed 03-09-2024]

    work page 2024

  75. [75]

    Stack Overflow Community. 2019. How to retrieve and use a tar.gz.asc from GitHub in an Ansible playbook with Ansible Vault? https://stackoverflow.com/questions/57834711/how-to-retrieve-and-use- a-tar-gz-asc-from-github-in-an-ansible-playbook-with-a Accessed 21-01-2026

    work page arXiv 2019

  76. [76]

    Trevor Stalnaker, Nathan Wintersgill, Oscar Chaparro, Massimiliano Di Penta, Daniel M German, and Denys Poshyvanyk. 2024. BOMs Away! Inside the Minds of Stakeholders: A Comprehensive Study of Bills of Materials for Software Systems. In2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). 517–529. doi:10.1145/3597503.3623347

    work page doi:10.1145/3597503.3623347 2024

  77. [77]

    Minaoar Hossain Tanzil, Masud Sarker, Gias Uddin, and Anindya Iqbal. 2023. A mixed method study of DevOps challenges.Information and Software Technology 161 (2023), 107244

    work page 2023

  78. [78]

    Wikimedia. 2017. Incidents/2017-01-18 Labs. https://wikitech.wikimedia.org/ wiki/Incidents/2017-01-18_Labs [Accessed 04-09-2024]

    work page 2017

  79. [79]

    Shuang-Hong Yang, Alek Kolcz, Andy Schlaikjer, and Pankaj Gupta. 2014. Large- scale high-precision topic modeling on twitter. InProceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 1907– 1916

    work page 2014

  80. [80]

    Ahmed Zerouali, Ruben Opdebeeck, and Coen De Roover. 2023. Helm charts for Kubernetes applications: Evolution, outdatedness and security risks. In2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR). IEEE, 523–533

    work page 2023