pith. sign in

arxiv: 2605.00352 · v1 · submitted 2026-05-01 · 💻 cs.SE · cs.CR

Integrating Log-Based Security Analytics in Agile Workflows: A Real-World Experience Report

Arpit Thool , Chris Brown This is my paper

Pith reviewed 2026-05-09 19:35 UTC · model grok-4.3

classification 💻 cs.SE cs.CR
keywords Agile workflowslog-based security analyticsfraud detectiondeveloper perceptionsexperience reportsecurity integrationbest practices
0
0 comments X

The pith

An eight-person Agile team built and tested a log-based fraud detection system, then reported lessons for fitting security analytics into fast workflows.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines a cross-functional team of eight members that spent weekly iterations building a proof-of-concept system to scan logs for suspicious account activity. Semi-structured interviews captured how developers viewed adopting the system, the practical obstacles they encountered, and any shifts in their daily work or security awareness. The resulting lessons, mitigation steps, and recommended practices aim to show how organizations can add security analytics to Agile processes without sacrificing delivery speed.

Core claim

In the Red Flag Project the team delivered a weekly-iterated log-based alert system that notifies stakeholders of suspicious patterns, and interviews revealed both developer willingness to adopt it and concrete hurdles in routine development work, from which the authors distill targeted lessons and best practices for embedding such analytics in Agile settings.

What carries the argument

The Red Flag Project, a proof-of-concept log-analysis system that generates alerts on suspicious account patterns, together with semi-structured interviews that gathered developer perceptions of its integration.

If this is right

  • Weekly iteration cycles allow teams to test and refine log-based security tools inside existing Agile sprints.
  • Targeted mitigation techniques can address specific barriers to developer adoption of new analytics.
  • Security practices can be added while preserving the pace of software delivery.
  • The identified best practices supply a starting template for other teams pursuing similar integrations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same weekly-PoC-plus-interview method could be applied to integrate other forms of monitoring or analytics beyond fraud detection.
  • Larger or geographically distributed teams may encounter amplified versions of the reported hurdles.
  • The findings point toward a reusable structure for experience reports on security practice adoption in Agile environments.

Load-bearing premise

The experiences and views reported by this single eight-member team in one organization reflect the challenges and opportunities faced by other Agile teams.

What would settle it

A separate Agile team that integrates comparable log-based security analytics and reports substantially different perceptions, challenges, or no measurable impact on development speed would indicate the reported lessons do not generalize.

read the original abstract

Modern organizations increasingly rely on log data and monitoring signals to protect products against account takeovers and abuse, yet integrating security analytics into fast-moving Agile workflows remains challenging. While it is important to understand how security practices are developed and sustained within Agile, real-world case studies of such integrations remain scarce. This experience report provides insights on developer perceptions of an effort to integrate log-based fraud detection within an organization, known as the "Red Flag Project". A cross-functional team of eight members (including one author) iterated weekly to implement a proof-of-concept log-based system that alerts stakeholders when accounts exhibit suspicious activity patterns. Through semi-structured interviews, we investigate developer perceptions of log-based fraud detection integration-exploring their willingness to adopt the system, challenges encountered, and the overall impact on day-to-day development activities and security perceptions. Our findings highlight key lessons, mitigation techniques, and best practices for embedding security analytics into Agile workflows. We provide insights for practitioners and researchers seeking to incorporate security practices into modern development processes while maintaining both speed and resilience in software delivery.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript is an experience report on the 'Red Flag Project,' in which an eight-member cross-functional team (including one author) developed and iterated weekly on a proof-of-concept log-based fraud detection system integrated into Agile workflows. Semi-structured interviews with team members are used to examine perceptions of adoption willingness, encountered challenges, and impacts on daily development activities and security awareness; the authors distill these into key lessons, mitigation techniques, and best practices for embedding security analytics in Agile processes.

Significance. If the reported perceptions and derived practices are robust, the work supplies scarce real-world evidence on integrating log-based security tools into fast-moving Agile settings. It can directly inform practitioners on practical mitigations while maintaining delivery speed and offer researchers grounded examples of adoption dynamics in security analytics.

major comments (2)
  1. [Methodology] The methodology description provides no details on the semi-structured interview protocol (e.g., guide, number of questions, duration), sampling approach, or analysis method (e.g., thematic analysis steps or coding process). This information is required to evaluate the reliability of the challenges, perceptions, and best practices that constitute the paper's central contribution.
  2. [Findings and Discussion] The claim that the findings yield transferable 'key lessons, mitigation techniques, and best practices' for other Agile teams rests on data from a single eight-person team in one organization. No triangulation, member checking, or explicit discussion of generalizability limits is described, which is load-bearing for the transferability asserted in the abstract and conclusions.
minor comments (2)
  1. [Abstract] The abstract could more explicitly qualify the single-organization, small-team scope to align reader expectations with the experience-report nature of the work.
  2. [Throughout] Ensure consistent terminology between 'log-based security analytics,' 'log-based fraud detection,' and 'Red Flag Project' throughout to prevent minor reader confusion.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our experience report. We address the two major comments point by point below and outline the revisions we will make.

read point-by-point responses

  1. Referee: [Methodology] The methodology description provides no details on the semi-structured interview protocol (e.g., guide, number of questions, duration), sampling approach, or analysis method (e.g., thematic analysis steps or coding process). This information is required to evaluate the reliability of the challenges, perceptions, and best practices that constitute the paper's central contribution.

    Authors: We agree that these details should have been included for transparency. In the revised manuscript we will expand the methodology section to specify: the interview guide structure (five core open-ended questions on willingness to adopt, encountered challenges, workflow impact, and security awareness, with follow-up probes); approximate interview duration (30-45 minutes); sampling (all eight team members were interviewed via purposive sampling as the complete cross-functional team); and the analysis process (thematic analysis following Braun and Clarke's six-phase approach, with initial open coding by one author, theme development through iterative discussion among authors, and no use of software tools). revision: yes

  2. Referee: [Findings and Discussion] The claim that the findings yield transferable 'key lessons, mitigation techniques, and best practices' for other Agile teams rests on data from a single eight-person team in one organization. No triangulation, member checking, or explicit discussion of generalizability limits is described, which is load-bearing for the transferability asserted in the abstract and conclusions.

    Authors: We acknowledge the inherent limitation of a single-case experience report. We cannot add triangulation or member checking without new data collection. In revision we will insert an explicit limitations paragraph that states the single-team, single-organization scope, the absence of triangulation or member checking, and the exploratory character of the work. We will also revise wording in the abstract and conclusions to present the lessons as context-specific insights derived from this project that may inform rather than directly transfer to other Agile settings. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical experience report with no derivations or self-referential steps

full rationale

This paper is a straightforward qualitative experience report describing interviews with an 8-person team on a log-based fraud detection project. It contains no equations, no fitted parameters presented as predictions, no uniqueness theorems, and no derivation chain that reduces to its own inputs. The findings are presented as lessons drawn from the described case rather than as outputs forced by self-citation or definitional loops. The central premise rests on the reported perceptions and challenges, which are independent of any circular reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a qualitative experience report with no formal axioms, free parameters, or invented entities; all content derives from project execution and interview data.

pith-pipeline@v0.9.0 · 5477 in / 991 out tokens · 32362 ms · 2026-05-09T19:35:26.849331+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 51 canonical work pages · 1 internal anchor

  1. [1]

    New Cybersecurity Advisory Warns About Web Application Vulnera- bilities

    2023. New Cybersecurity Advisory Warns About Web Application Vulnera- bilities. National Security Agency. https://www.nsa.gov/Press-Room/Press- Releases-Statements/Press-Release-View/Article/3473830/new-cybersecurity- advisory-warns-about-web-application-vulnerabilities/

    work page arXiv 2023

  2. [2]

    Abdulrahman Alzahrani, Ali Alqazzaz, Ye Zhu, Huirong Fu, and Nabil Almashfi

    work page

  3. [3]

    Web application security tools analysis. In2017 ieee 3rd international con- ference on big data security on cloud (bigdatasecurity), ieee international conference on high performance and smart computing (hpsc), and ieee international conference on intelligent data and security (ids). IEEE, 237–242

    work page

  4. [4]

    Marc Andreessen. 2011. Why Software is Eating the World.The Wall Street Journal8 (2011), 20

    work page 2011

  5. [5]

    Pranshu Bajpai and Adam Lewis. 2022. Secure Development Workflows in CI/CD Pipelines. In2022 IEEE Secure Development Conference (SecDev). 65–66. https://doi.org/10.1109/SecDev53368.2022.00024

    work page doi:10.1109/secdev53368.2022.00024 2022

  6. [6]

    Steffen Bartsch. 2011. Practitioners’ perspectives on security in agile development. In2011 Sixth International Conference on A vailability, Reliability and Security. IEEE, 479–484

    work page 2011

  7. [7]

    Jeff Beckman. 2024. Agile Statistics: How Many Companies Use Agile in 2023? Tech Report(2024). https://techreport.com/statistics/business-workplace/how- many-companies-use-agile/

    work page 2024

  8. [8]

    Erik Blair. 2015. A reflexive exploration of two qualitative data coding techniques. Journal of Methods and Measurement in the Social Sciences6, 1 (2015), 14–29

    work page 2015

  9. [9]

    Claude Bolduc. 2016. Lessons learned: Using a static analysis tool within a continuous integration system. In2016 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 37–40

    work page 2016

  10. [10]

    Sébastien Dupont, Guillaume Ginis, Mirko Malacario, Claudio Porretti, Nicolò Maunero, Christophe Ponsard, and Philippe Massonet. 2021. Incremental common criteria certification processes using DeVSecOps practices. In2021 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, 12–23

    work page 2021

  11. [11]

    Jacki Fitzpatrick and Erin Kostina-Ritchey. 2013. US families’ adoption of Chinese daughters: A narrative analysis of family themes in children’s books.Family Relations62, 1 (2013), 58–71

    work page 2013

  12. [12]

    Xiaocheng Ge, Richard F Paige, Fiona AC Polack, Howard Chivers, and Phillip J Brooke. 2006. Agile development of secure web applications. InProceedings of the 6th international conference on Web engineering. 305–312

    work page 2006

  13. [13]

    Matthew Green and Matthew Smith. 2016. Developers are not the enemy!: The need for usable security apis.IEEE Security & Privacy14, 5 (2016), 40–46

    work page 2016

  14. [14]

    Rashina Hoda, Norsaremah Salleh, and John Grundy. 2018. The rise and evolution of agile software development.IEEE software35, 5 (2018), 58–63

    work page 2018

  15. [15]

    Constantin Hofmann, Sebastian Lauber, Benjamin Haefner, and Gisela Lanza

    work page

  16. [16]

    Development of an agile development method based on Kanban for dis- tributed part-time teams and an introduction framework.Procedia Manufacturing 23 (2018), 45–50

    work page 2018

  17. [17]

    Ran Hu, Zhuo Wang, Jun Hu, Jianfeng Xu, and Jun Xie. 2008. Agile web devel- opment with web framework. In2008 4th International Conference on Wireless Communications, Networking and Mobile Computing. IEEE, 1–4

    work page 2008

  18. [18]

    David Hylender, Philippe Langlois, Alex Pinto, and Suzanne Widup

    C. David Hylender, Philippe Langlois, Alex Pinto, and Suzanne Widup. 2024. 2024 Data Breach Investigations Report. Technical Report. Verizon Busi- ness. https://www.verizon.com/business/resources/Tac2/reports/2024-dbir-data- breach-investigations-report.pdf

    work page 2024

  19. [19]

    Mehdi Jazayeri. 2007. Some trends in web application development. InFuture of Software Engineering (FOSE’07). IEEE, 199–213

    work page 2007

  20. [20]

    Anna Koskinen. 2019. DevSecOps: building security into the core of DevOps. (2019)

    work page 2019

  21. [21]

    Hopefully We Are Mostly Secure

    Tamara Lopez, Helen Sharp, Thein Tun, Arosha Bandara, Mark Levine, and Bashar Nuseibeh. 2019. " Hopefully We Are Mostly Secure": Views on Secure Code in Professional Practice. In2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE). IEEE, 61–68

    work page 2019

  22. [22]

    2010.Employee fraud detection under real world conditions

    Jonas Luell. 2010.Employee fraud detection under real world conditions. Ph. D. Dissertation. University of Zurich

    work page 2010

  23. [23]

    Lucy Ellen Lwakatare, Ellinor Rånge, Ivica Crnkovic, and Jan Bosch. 2021. On the experiences of adopting automated data validation in an industrial machine learning project. In2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 248–257

    work page 2021

  24. [24]

    Patrik Maier, Zhendong Ma, and Roderick Bloem. 2017. Towards a secure scrum process for agile web application development. InProceedings of the 12th Interna- tional Conference on A vailability, Reliability and Security. 1–8

    work page 2017

  25. [25]

    Boubakr Nour, Makan Pourzandi, and Mourad Debbabi. 2023. A survey on threat hunting in enterprise networks.IEEE communications surveys & tutorials25, 4 (2023), 2299–2324

    work page 2023

  26. [26]

    Kai Petersen, Cigdem Gencel, Negin Asghari, Dejan Baca, and Stefanie Betz

    work page

  27. [27]

    InProceedings of the 2014 international workshop on Long-term industrial collaboration on software engineering

    Action research as a model for industry-academia collaboration in the software engineering context. InProceedings of the 2014 international workshop on Long-term industrial collaboration on software engineering. 55–62

    work page 2014

  28. [28]

    Andreas Poller, Laura Kocksch, Sven Türpe, Felix Anand Epp, and Katharina Kinder-Kurlanda. 2017. Can security become a routine? A study of organizational change in an agile software development group. InProceedings of the 2017 ACM conference on computer supported cooperative work and social computing. 2489– 2503

    work page 2017

  29. [29]

    Akond Ashfaque Ur Rahman, Eric Helms, Laurie Williams, and Chris Parnin. 2015. Synthesizing continuous deployment practices used in software development. In 2015 Agile Conference. IEEE, 1–10

    work page 2015

  30. [30]

    Roshan Namal Rajapakse, Mansooreh Zahedi, and Muhammad Ali Babar. 2021. An empirical analysis of practitioners’ perspectives on security tool integration into devops. InProceedings of the 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–12

    work page 2021

  31. [31]

    Thorsten Rangnau, Remco v Buijtenen, Frank Fransen, and Fatih Turkmen. 2020. Continuous security testing: A case study on integrating dynamic security testing tools in ci/cd pipelines. In2020 IEEE 24th International Enterprise Distributed Object Computing Conference (EDOC). IEEE, 145–154

    work page 2020

  32. [32]

    Klaus Reche Riisom, Martin Slusarczyk Hubel, Hasan Mousa Alradhi, Niels Bonde Nielsen, Kati Kuusinen, and Ronald Jabangwe. 2018. Software security in ag- ile software development: A literature review of challenges and solutions. In Proceedings of the 19th International Conference on Agile Software Development: Companion. 1–5

    work page 2018

  33. [33]

    Kalle Rindell, Sami Hyrynsalmi, and Ville Leppänen. 2017. Busting a myth: Review of agile security engineering methods. InProceedings of the 12th International Conference on A vailability, Reliability and Security. 1–10

    work page 2017

  34. [34]

    Kalle Rindell, Sami Hyrynsalmi, and Ville Leppänen. 2018. Aligning security ob- jectives with agile software development. InProceedings of the 19th International Conference on Agile Software Development: Companion. 1–9

    work page 2018

  35. [35]

    Per Runeson and Martin Höst. 2009. Guidelines for conducting and reporting case study research in software engineering.Empirical software engineering14 (2009), 131–164

    work page 2009

  36. [36]

    Palvi Shelke and Tapio Frantti. 2025. Exploring the Possibilities of Splunk Enter- prise Security in Advanced Cyber Threat Detection. InThe Proceedings of the... International Conference on Cyber Warfare and Security. Academic Conferences International Ltd

    work page 2025

  37. [37]

    Justin Smith, Lisa Nguyen Quang Do, and Emerson Murphy-Hill. 2020. Why can’t johnny fix vulnerabilities: A usability evaluation of static analysis tools for security. InSixteenth Symposium on Usable Privacy and Security (SOUPS 2020). 221–238

    work page 2020

  38. [38]

    Miroslaw Staron and Miroslaw Staron. 2020. Action research as research method- ology in software engineering.Action Research in Software Engineering: Theory and Applications(2020), 15–36

    work page 2020

  39. [39]

    Michael Stausberg. 2011. Structured observation.The Routledge Handbook of Research Methods in the Study of Religion(2011), 382

    work page 2011

  40. [40]

    Karun Subramanian. 2020. Introducing the Splunk platform. InPractical Splunk Search Processing Language: A Guide for Mastering SPL Commands for Maximum Efficiency and Outcome. Springer, 1–38

    work page 2020

  41. [41]

    Arpit Thool and Chris Brown. 2024. Harnessing the Power of LLMs: LLM Sum- marization for Human-Centric DAST Reports. In2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE

    work page 2024

  42. [42]

    Arpit Thool and Chris Brown. 2024. Securing Agile: Assessing the Impact of Security Activities on Agile Development. InInternational Workshop on Software Security: Challenges, Opportunities, and Lessons Learned

    work page 2024

  43. [43]

    Arpit Thool and Chris Brown. 2025. Integrating DAST in Kanban and CI/CD: A Real World Security Case Study.arXiv preprint arXiv:2503.21947(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  44. [44]

    Integrating Log-Based Security Analytics in Agile Workflows: A Real-World Experience Report

    Arpit Thool and Dwayne Brown. 2026. Supplemental Material for "Integrating Log-Based Security Analytics in Agile Workflows: A Real-World Experience Report". (3 2026). https://doi.org/10.6084/m9.figshare.31441105.v1

    work page doi:10.6084/m9.figshare.31441105.v1 2026

  45. [45]

    Arpit Uday Thool. 2026. Bridging Security and Agility: A Comprehensive Ap- proach to Integrating Security Practices in Agile Development through DAST, LLMs, and Automation. (2026)

    work page 2026

  46. [46]

    Serafeim Triantafyllou and Christos Georgiadis. 2022. Gamification of MOOCs and security awareness in corporate training. (2022)

    work page 2022

  47. [47]

    Akond Ashfaque Ur Rahman and Laurie Williams. 2016. Software security in DevOps: synthesizing practitioners’ perceptions and practices. InProceedings of the international workshop on continuous software evolution and delivery. 70–76

    work page 2016

  48. [48]

    Charles Weir, Ingolf Becker, and Lynne Blair. 2023. Incorporating software security: using developer workshops to engage product managers.Empirical Software Engineering28, 2 (2023), 21

    work page 2023

  49. [49]

    Laurie Williams. 2019. Secure software lifecycle.The Cyber Security Body of Knowledge(2019)

    work page 2019

  50. [50]

    Ewelina Wińska and Włodzimierz Dąbrowski. 2020. Software development artifacts in large agile organizations: a comparison of scaling agile methods. Data-Centric Business and Applications: Towards Software Development (Volume 4) (2020), 101–116

    work page 2020

  51. [51]

    Mounia Zaydi, Yassine Maleh, Hayat Zaydi, Youness Khourdifi, Bouchaib Nassereddine, and Zohra Bakouri. 2024. Agile security and compliance inte- gration.Agile Security in the Digital Era: Challenges and Cybersecurity Trends (2024), 68

    work page 2024