Pith. sign in

REVIEW 4 major objections 6 minor 23 references

Leveraging XP and CRISP-DM for Agile Data Science Projects

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that XP's engineering practices can be embedded inside CRISP-DM's six phases for data science work, based on a 13-person case study at the e-commerce company Elo7.

desk verdict A modest, honest case study with a useful practice-to-phase mapping, but the headline percentages are response-level self-reports and the feasibility claim outruns the evidence. read the letter →

arxiv 2505.21603 v1 pith:OIWSFNVF submitted 2025-05-27 cs.SE cs.AIcs.LG

Andre Massahiro Shimaoka , Renato Cordeiro Ferreira , Alfredo Goldman This is my paper
classification cs.SEcs.AIcs.LG
keywords eXtremeProgrammingAgileCRISP-DMDataScienceMachineLearningXPpracticescasestudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the agile engineering practices of eXtreme Programming (XP) can be combined with the six-phase data-mining process CRISP-DM in real data science work, not only in software development. It reports a case study at Elo7, a Brazilian e-commerce company, where 13 data scientists, machine learning engineers, and product managers answered a frequency questionnaire and one lead data scientist was interviewed. The team reported high adherence to both methods: 86% of responses indicated frequent or always CRISP-DM use, and 71% indicated frequent or always XP use. On that basis the authors conclude that the combination is viable and offers a structured yet collaborative workflow, while noting that test-first development and continuous integration were the hardest practices to apply outside the deployment phase. If true, this gives data science teams a concrete way to pair a widely used process model with team-level agile practices.

What carries the argument

The load-bearing object is the phase-by-practice crosswalk (Figure 4) built from the questionnaire, in which respondents linked each XP practice to one or more CRISP-DM phases and the authors aggregated those links into a map. CRISP-DM is the six-phase process model of business understanding, data understanding, data preparation, modeling, evaluation, and deployment; XP is the set of software-engineering practices built around short iterations and direct collaboration. The map does the argument's work because it turns the abstract question of whether the two methods can be combined into a concrete assignment of practices to process stages, and the Likert-frequency questions supply the supporting adoption numbers (86% for CRISP-DM, 71% for XP).

What would settle it

Shadow a data science team through a full project and tally, for each CRISP-DM phase, which XP practices actually occur; if a team that reports roughly 71% XP adoption shows almost no XP practices in the early analytical phases, the claim that XP can be integrated across CRISP-DM would be contradicted.

Watch

Extended reading notes

Core claim

The paper's central discovery is that XP does not have to replace CRISP-DM in data science; it can be embedded inside it. At Elo7, participants connected each of the six CRISP-DM phases to XP practices: user stories could be created or refined in any phase, iterations structured work across all phases, spikes supported exploration in Business Understanding, Data Understanding, and Modeling, pair programming was used beyond coding, and sustainable pace held across phases. The two practices that did not travel well were test-first development and continuous integration, which the team applied mainly during Deployment, where model-serving software is built. From this the authors conclude that combining CRISP-DM with XP is possible and produces a structured and collaborative approach, and that agile practices can be adopted without weakening the data science process.

Load-bearing premise

The whole feasibility conclusion depends on the 13 respondents' self-reports being accurate descriptions of their actual work, because no documents, code, or direct observation were used to verify what the team really does.

Editorial extensions

If this is right

  • A data science team that already follows CRISP-DM can adopt XP without discarding its process model: each phase can host a defined set of practices, so combining them becomes a matter of assignment rather than invention.
  • Teams that want to strengthen testing can start in the Deployment phase, because that is where test-first development and continuous integration were reported as feasible; moving them earlier will require training and automation infrastructure.
  • The map gives practitioners a diagnostic checklist: for every CRISP-DM phase, a team can ask which XP practices are active and which are missing, and use the gaps to plan improvements.
  • The reported adoption levels (86% CRISP-DM, 71% XP) suggest that a structured process and agile collaboration can coexist in a real e-commerce data science team rather than being competing alternatives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the survey was anonymous, unsupervised, and self-reported, so the percentages are best read as perceived practice rather than audited practice; direct observation would likely show more variation than the aggregate numbers suggest.
  • Inference: the Figure 4 mapping is reusable as a lightweight team self-assessment tool; any team could score each CRISP-DM phase against the XP practices and use the gaps to plan training.
  • Inference: the concentration of test-first development and continuous integration in Deployment suggests that the software-engineering side of XP attaches where code is operationalized, not where analysis happens; a testable extension is whether automation infrastructure moves those practices earlier in the pipeline.
  • Inference: the recommendation to write stories in business language and involve data scientists in Business Understanding hints that the main barrier to combining the two methods is cross-role communication rather than process mechanics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper reports a single-company case study at Elo7, an e-commerce company, investigating whether eXtreme Programming (XP) practices can be integrated with the CRISP-DM process model in data science projects. Data come from an open interview with the lead data scientist and a Likert-scale questionnaire completed by 13 professionals (data scientists, ML engineers, and data product managers). The authors report that 86% of responses indicate frequent or always use of CRISP-DM, that 71% of responses indicate frequent or always use of XP practices, and that participants associated each XP practice with one or more CRISP-DM phases. Based on this, the paper claims that combining CRISP-DM and XP is possible and provides a structured and collaborative approach, and it offers recommendations for the company.

Significance. The paper addresses a worthwhile and under-explored question: how a well-established data-mining process model (CRISP-DM) and an agile software method (XP) can coexist in a real data-science team. Its strengths are the use of a real company context, the explicit mapping of XP practices to CRISP-DM phases, and practical recommendations grounded in observed difficulties. If the evidence were stronger, the results would be useful for practitioners and for future multi-case studies. However, the empirical foundation is a perception-based, single-company, self-report survey of 13 professionals, with no documentary verification; the paper itself acknowledges these limitations in Section 3.5. Consequently, the contribution is better characterized as an exploratory case study that suggests feasibility rather than a demonstration.

major comments (4)
  1. [Abstract and Sections 3.1, 3.2] The headline percentages are computed over responses, not over participants, yet the abstract states that '86% of the team frequently or always applies CRISP-DM' and '71% adopt XP practices.' Section 3.1 says '86% of responses,' and Section 3.2 says '71% of responses.' Since each of the 13 respondents answered multiple Likert items (20 CRISP-DM questions and 13 XP questions), response-level percentages can be driven by a few participants who answer 'frequently/always' on many items. The paper should report per-participant aggregates (e.g., the proportion of respondents whose average or majority of answers was frequently/always), and should adjust the abstract and conclusions accordingly. As written, the claim that most of the team applies these methods is not directly supported by the reported analysis.
  2. [Section 3.3 and Figure 4] The mapping in Figure 4 is based on participants linking each XP practice to one or more CRISP-DM phases (Section 2.2), which records perceived associations rather than observed or documented practice. The conclusion in Section 4 states that Frequent Builds, Continuous Integration, and TDD were 'not applicable' in Business Understanding and Data Understanding. This is stronger than what the data can support: the survey can only show that these practices were rarely used in those phases by self-report. 'Not applicable' is an interpretive inference that needs either a documented rationale or a softer formulation such as 'rarely used.' The same issue applies to the phrase 'the study demonstrated' in the abstract.
  3. [Section 2.2] The full questionnaire is referenced only by the placeholder URL 'https://.....' and is not included in the manuscript. The survey items, response scales, and the exact task used to produce Figure 4 are therefore not available for independent checking or replication. The authors should provide the questionnaire as supplementary material or in an appendix, or state where it can be obtained.
  4. [Section 3.5 and Conclusion] The paper already acknowledges in Section 3.5 that the study relies on team perceptions and that no documentary evidence was analyzed. This is a significant limitation that should be reflected in the strength of the wording. Terms such as 'demonstrates' (Abstract) and 'it can be concluded that combining CRISP-DM and XP can be a viable approach' (Conclusion) overstate the evidential support. A formulation such as 'suggests' or 'provides initial evidence for' would be more proportionate, and would not reduce the value of the recommendations.
minor comments (6)
  1. [Section 2.2] The sentence 'we decided to conduct a qualitative, open-ended, and unstructured' is missing the noun 'interview' and the punctuation; it should read 'a qualitative, open-ended, and unstructured interview.'
  2. [Section 3.3] The subsection heading 'PairProgrammingPacewithCRISP-DMPhases' lacks spaces and should read 'Pair Programming with CRISP-DM Phases.'
  3. [Section 3.4] 'sing a common language' should be 'using a common language.'
  4. [Sections 1.2 and 2.2] The paper uses inconsistent names for the test-first practice: 'Test-First Development' in Section 1.2, 'Test-Driven Development' in Section 2.2, and 'Test-First Programming' in Sections 3.2 and the conclusion; choose one term and use it consistently.
  5. [Figures 2, 3, and 4] The figures are referred to but not described in the body text with respect to axes, legend, or how percentages were computed; adding a brief description of how to read them would improve accessibility, especially because Figure 4 is central to the integration claim.
  6. [Section 2.2] 'which provided input for the Survey' uses an inconsistent capital 'S' for 'Survey'; use lowercase 'survey' for consistency with the rest of the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the feasibility claim rests on an independent, self-reported case study with no fitted parameters or derivation-by-construction.

full rationale

The paper's central claim — that CRISP-DM and XP can be combined in Data Science projects — is supported by an empirical case study (interviews and a 13-person survey at Elo7). There is no mathematical derivation, no fitted parameter, and no equation in which the conclusion is built into the input. The 86% and 71% figures are response-level self-reported Likert aggregates, and Figure 4 is a participant-generated mapping of XP practices onto CRISP-DM phases. These are descriptive results, not predictions derived from assumptions. The only self-citation is reference [7], used to justify that no relevant prior studies address the combined use of CRISP-DM and XP and to motivate the research gap; this does not do load-bearing work for the survey findings. The paper is honest about its limitations in Section 3.5, noting that the study relied on perceptions and that no documentary evidence was analyzed. A limitation in validity, however, is not circularity: reporting what respondents said and then summarizing those responses is not reducing the conclusion to the method by construction. The conclusion goes slightly beyond the data in phrasing (e.g., calling practices 'not applicable' where the data showed rarity of use), but that is a correctness/overreach concern, not a circular-self-reference concern. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted; the paper is empirical. The load-bearing assumptions are about validity of self-report data and the mapping of questionnaire items to the two frameworks.

assumptions (3)
  • domain assumption Self-reported Likert frequency responses accurately reflect actual practice adoption.
    The entire quantitative claim rests on questionnaire responses without observational or documentary verification; acknowledged as a limitation in Section 3.5.
  • domain assumption Aggregating responses across questions and participants yields a meaningful team-level adoption percentage.
    Section 3 interprets '86% of responses' as team-level adherence in the abstract; this assumes equal weighting and no response bias.
  • domain assumption The operationalized CRISP-DM phases and XP practices in the questionnaire correctly represent the frameworks.
    Section 2.2 describes questions mapped to the six CRISP-DM phases and 12 XP practices; any misrepresentation would bias all answers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging XP and CRISP-DM for Agile Data Science Projects." pith.science (2026). https://pith.science/paper/OIWSFNVF

@misc{pith2026250521603,
  author       = {Pith},
  title        = {Pith review of: Leveraging XP and CRISP-DM for Agile Data Science Projects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OIWSFNVF}},
  note         = {Machine review of arXiv:2505.21603}
}
read the original abstract

This study explores the integration of eXtreme Programming (XP) and the Cross-Industry Standard Process for Data Mining (CRISP-DM) in agile Data Science projects. We conducted a case study at the e-commerce company Elo7 to answer the research question: How can the agility of the XP method be integrated with CRISP-DM in Data Science projects? Data was collected through interviews and questionnaires with a Data Science team consisting of data scientists, ML engineers, and data product managers. The results show that 86% of the team frequently or always applies CRISP-DM, while 71% adopt XP practices in their projects. Furthermore, the study demonstrates that it is possible to combine CRISP-DM with XP in Data Science projects, providing a structured and collaborative approach. Finally, the study generated improvement recommendations for the company.

Figures

Figures reproduced from arXiv: 2505.21603 by the authors.

Figure 1
Figure 1. Case study methodology [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Survey results on the application of CRISP-DM stages by the agile data science team Business Understanding: Part of the team reported a lack of alignment with business objectives. Additionally, the focus was generally on data exploration rather than on business goals. This phase was often led by product managers, with limited involvement from data scientists. Data Understanding: The company had well-established and … view at source ↗
Figure 3
Figure 3. Survey results on XP Practices adoption by the agile data science team User stories: In the company, user stories were not always clear or understand￾able to product managers. The focus was often on technical terms, sometimes overlooking business rules. There was also confusion about the responsibilities for writing user stories and tasks, with many stories being written at a technical task level. Releases: Some ML … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Survey results on the integration of CRISP-DM and XP Practices by the Agile Data Science Team Slacks with CRISP-DM Phases: Slacks practices were seldom adopted in the phases of CRISP-DM. Extraproject activities and free hours, such as time for training and skill develo…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 21 canonical work pages

  1. [7]

    M., Ferreira, R

    Shimaoka, A. M., Ferreira, R. C., Goldman, A.: The evolution of CRISP-DM for Data Science: Methods, Processes and Frameworks. SBC Reviews on Computer Science 4(1), 28–43 (2024).https://doi.org/10.5753/reviews.2024.3757

    work page arXiv 2024

  2. [1]

    In: 2019 IEEE International Conference on Big Data (Big Data), pp

    Saltz, J., Sutherland, A.: SKI: An Agile Framework for Data Science. In: 2019 IEEE International Conference on Big Data (Big Data), pp. 3468–3476 (2019). https://doi.org/10.1109/BigData47090.2019.9005591

    arXiv 2019

  3. [2]

    Martínez-Plumed, F., Contreras-Ochando, L., Ferri, C., Hernández-Orallo, J., Kull, M., Lachiche, N., Ramírez-Quintana, M. J., Flach, P.: CRISP-DM Twenty Years Later:FromDataMiningProcessestoDataScienceTrajectories.IEEETransactions on Knowledge and Data Engineering 33(8), 3048–3061 (2021).https://doi.org/ 10.1109/TKDE.2019.2962680

    arXiv 2021

  4. [3]

    Wirth, R., Hipp, J.: CRISP-DM: Towards a Standard Process Model for Data Mining. In: Practical Application Company (eds.) International Conference and Exhibition on the Practical Application of Knowledge Discovery and Data Min- ing, 4., 2000, Manchester, Reino Unido, pp. 29–40 (2000). Available at: https: //api.semanticscholar.org/CorpusID:1211505

    work page 2000

  5. [4]

    The Knowledge Engineering Review 25(2), 137–166 (2010).https://doi.org/10.1017/S0269888910000032

    Mariscal, G., Marbán, Ó., Fernández, C.: A survey of data mining and knowledge discovery process models and methodologies. The Knowledge Engineering Review 25(2), 137–166 (2010).https://doi.org/10.1017/S0269888910000032

    work page doi:10.1017/s0269888910000032 2010

  6. [5]

    In: 2020 IEEE 22nd Conference on Business Informatics (CBI), Antwerp, Belgium, pp

    Baijens, J., Helms, R., Iren, D.: Applying Scrum in Data Science Projects. In: 2020 IEEE 22nd Conference on Business Informatics (CBI), Antwerp, Belgium, pp. 30–38 (2020). https://doi.org/10.1109/CBI49978.2020.00011 14 Shimaoka et al

    work page arXiv 2020

  7. [6]

    P.: When Data Science Becomes Software Engineering

    Cruz, L. P.: When Data Science Becomes Software Engineering. In: 9th Interna- tional Conference on Knowledge Engineering and Ontology Development, 9., 2017, Funchal, Madeira, Portugal. Proceedings [...]. Setúbal, Portugal: Scitepress - Sci- ence and Technology Publications, pp. 226–233 (2017).https://doi.org/10.5220/ 0006508502260232

    work page 2017

  8. [8]

    Chapman, P.: CRISP-DM 1.0: Step-by-step data mining guide. (2000). Available at: https://api.semanticscholar.org/CorpusID:59777418

    work page 2000

Show all 23 references
  1. [9]

    BECK, Kent, FOWLER, Martin: Planning Extreme Programming. 1st ed. Addison- Wesley, Boston, EUA (2000)

    2000

  2. [10]

    Addison- Wesley, 2004

    COHN, Mike: User stories applied: For agile software development. Addison- Wesley, 2004

    2004

  3. [11]

    Addison-Wesley Professional, (2004)

    Beck, K., Andres, C.: Extreme Programming Explained: Embrace Change (2nd edn.). Addison-Wesley Professional, (2004)

    2004

  4. [12]

    WELLS, Don: What is Extreme Programming. 2011. Disponível em:http://www. extremeprogramming.org. Acesso em: 05 nov. 2024

    2011

  5. [13]

    FOWLER, Martin: Continuous Integration. 2006. Disponível em:https://www. martinfowler.com/articles/continuousIntegration.html. Acesso em: 02 nov. 2024

    2006

  6. [14]

    FOWLER, Martin: Is it worth the effort to design software well?. 2007. Disponível em: https://martinfowler.com/bliki/DesignStaminaHypothesis.html. Acesso em: 02 nov. 2024

    2007

  7. [15]

    K.: Case Study Research and Applications: Design and Methods

    Yin, R. K.: Case Study Research and Applications: Design and Methods. 6th edn. Sage, Thousand Oaks, EUA (2018)

    2018

  8. [16]

    M.: O Estudo de Caso como Modalidade de Pesquisa: The Case Study as a Research Mode

    Ventura, M. M.: O Estudo de Caso como Modalidade de Pesquisa: The Case Study as a Research Mode. Revista da Sociedade de Cardiologia do Estado do Rio de Janeiro, Rio de Janeiro, v. 20, n. 5, pp. 383–386 (out. 2007)

    2007

  9. [17]

    Bryman, A.: Social Research Methods. 5th edn. Oxford University Press, Oxford (2016)

    2016

  10. [18]

    In: Sammut, C., Webb, G.I

    Kohavi, R., Longbotham, R.: Online Controlled Experiments and A/B Testing. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA (2017).https://doi.org/10.1007/978-1-4899-7687-1_891

    2017 doi

  11. [19]

    In: 2019 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), pp

    Münch, J., Trieflinger, S., Lang, D.: Product Roadmap – From Vision to Re- ality: A Systematic Literature Review. In: 2019 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), pp. 1–8. IEEE, 2019. https://doi.org/10.1109/ICE.2019.8792654

    2019

  12. [20]

    Addison-Wesley Long- man Publishing Co., Inc., USA (2002)

    Williams, L., Kessler, R.: Pair Programming Illuminated. Addison-Wesley Long- man Publishing Co., Inc., USA (2002)

    2002

  13. [21]

    Lazar, N.: Ockham’s Razor.WIREs Computational Statistics 2(5), 243–246 (2010), https://doi.org/10.1002/wics.75

    2010 doi

  14. [22]

    IEEE Access10, 63606–63618 (2022)

    Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., Vessio, G.: MLOps: A Taxonomy and a Methodology. IEEE Access10, 63606–63618 (2022). https://doi.org/10.1109/ACCESS.2022.3181730

    2022

  15. [23]

    Manning Publications, New York, EUA (2014)

    Smart,J.F.:BDDinAction:Behavior-DrivenDevelopmentfortheWholeSoftware Lifecycle. Manning Publications, New York, EUA (2014). 353 p

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.