Model Development Process

Przemyslaw Biecek

arxiv: 1907.04461 · v1 · pith:VBCTTL5Tnew · submitted 2019-07-09 · 💻 cs.LG · stat.AP

Model Development Process

Przemyslaw Biecek This is my paper

Pith reviewed 2026-05-25 00:12 UTC · model grok-4.3

classification 💻 cs.LG stat.AP

keywords Model Development Processpredictive modelingmodel life-cycleRational Unified ProcessCRISP-DMautomation toolsdata science workflow

0 comments

The pith

A generic Model Development Process modeled on Rational Unified Process describes the predictive model life-cycle to guide automation tools.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a Model Development Process (MDP) as a high-level generic description of the steps involved in creating and maintaining predictive models. It draws directly from the structure of the Rational Unified Process used in software engineering and contrasts it with data-mining standards such as CRISP-DM and ASUM-DM. The central purpose is to make explicit what activities occur across the model life-cycle so that software tools can be built to automate training, testing, and ongoing maintenance. A sympathetic reader would see this as a step toward treating model development more like an engineering discipline with reusable processes rather than ad-hoc work. The claim rests on the idea that an open, shared description of the life-cycle will lower the barrier to building supporting automation.

Core claim

The paper claims that a generic Model Development Process (MDP) inspired by Rational Unified Process supplies an open standard for the predictive-model life-cycle; once stated, this standard can be used to create tools that automate model training, testing, and maintenance.

What carries the argument

The Model Development Process (MDP): a high-level, generic description of the model life-cycle that mirrors the phased structure of Rational Unified Process.

If this is right

Automation tools can target specific phases of model training, testing, and maintenance once the life-cycle stages are named.
The MDP can serve as a common reference point when comparing or combining existing methodologies such as CRISP-DM.
Data scientists gain a shared vocabulary for discussing what parts of their work are candidates for automation.
Open publication of the MDP lowers the cost for third parties to build supporting software.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Teams that adopt the MDP may converge on more consistent documentation practices even without new tools.
Future extensions could map the MDP phases onto specific machine-learning libraries to produce executable workflows.
Industry standards bodies might later formalize the MDP into a certification or compliance checklist.

Load-bearing premise

That stating a high-level generic description of the model life-cycle is enough by itself to enable the creation of practical automation tools.

What would settle it

Publication of several widely adopted open-source tools whose design documents explicitly reference the proposed MDP phases within three years.

read the original abstract

Predictive modeling has an increasing number of applications in various fields. High demand for predictive models drives creation of tools that automate and support work of data scientist on the model development. To better understand what can be automated we need first a description of the model life-cycle. In this paper we propose a generic Model Development Process (MDP). This process is inspired by Rational Unified Process (RUP) which was designed for software development. There are other approached to process description, like CRISP DM or ASUM DM, in this paper we discuss similarities and differences between these methodologies. We believe that the proposed open standard for model development will facilitate creation of tools for automation of model training, testing and maintaining.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

A thin position paper that restates existing data-mining lifecycles in RUP language and asserts without evidence that the description will enable automation tools.

read the letter

The paper's main move is to sketch a generic Model Development Process modeled on the Rational Unified Process and to note overlaps with CRISP-DM and ASUM-DM. That is the extent of the contribution. It correctly identifies that predictive modeling needs clearer phase descriptions if anyone wants to build supporting tools, but it stops at the high-level outline and the claim that an open standard will facilitate automation. No phase-to-tool mapping, no pseudocode, no worked example of an automation feature that existing descriptions block, and no validation appear in the text. The facilitation claim therefore remains an unsupported assertion rather than a derived consequence. The discussion of similarities and differences with prior methodologies is accurate but adds little beyond what is already in the CRISP-DM literature. Because the manuscript supplies only a descriptive framework and no new empirical result, derivation, or reproducible artifact, it does not generate a result that later work could cite for its technical content. Readers already working on workflow standardization or AutoML platforms will see nothing they can directly use or refute. The paper is best treated as an internal note rather than a candidate for peer review; an editor would be justified in desk-rejecting it on the grounds that the central belief is not argued or evidenced. If the authors later supply concrete mappings or a prototype that demonstrates the claimed facilitation, the work could be reconsidered, but the current version does not reach that threshold.

Referee Report

1 major / 0 minor

Summary. The manuscript proposes a generic Model Development Process (MDP) inspired by the Rational Unified Process (RUP) for software engineering. It compares this process to CRISP-DM and ASUM-DM, and asserts that the resulting open standard will facilitate the creation of automation tools for model training, testing, and maintenance.

Significance. If substantiated, a standardized MDP could aid in organizing model lifecycles and tool-building in applied ML. However, the manuscript supplies only a high-level descriptive outline with no validation, examples, or mappings, so its significance is limited to restating existing process ideas without advancing them.

major comments (1)

[Abstract] Abstract: the central claim that the proposed MDP 'will facilitate creation of tools for automation of model training, testing and maintaining' is presented as a belief with no mechanism, pseudocode, phase-to-API mapping, architecture sketch, or worked example showing how the MDP supplies actionable distinctions usable by tool builders beyond those already present in CRISP-DM or ASUM-DM.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address the single major comment below, focusing on the abstract claim. The paper is positioned as a high-level conceptual proposal for an MDP standard, with the comparison to existing methods intended to highlight distinctions.

read point-by-point responses

Referee: [Abstract] Abstract: the central claim that the proposed MDP 'will facilitate creation of tools for automation of model training, testing and maintaining' is presented as a belief with no mechanism, pseudocode, phase-to-API mapping, architecture sketch, or worked example showing how the MDP supplies actionable distinctions usable by tool builders beyond those already present in CRISP-DM or ASUM-DM.

Authors: We acknowledge that the abstract presents the facilitation claim as a belief without supplying mechanisms, pseudocode, mappings, sketches, or examples. The manuscript's contribution is a high-level process description inspired by RUP, including a discussion of similarities and differences with CRISP-DM and ASUM-DM (e.g., RUP's iterative disciplines and risk-driven approach adapted to modeling). These differences are argued to supply actionable distinctions for standardization. However, the paper does not include implementation details or validation, as its scope is limited to defining the open standard process itself. We will revise the abstract to rephrase the claim from a definitive 'will facilitate' to 'is intended to support' and to explicitly reference the comparison section as the source of distinctions. revision: yes

Circularity Check

0 steps flagged

No circularity; purely descriptive proposal with no derivations or self-referential reductions

full rationale

The paper offers a high-level generic description of a Model Development Process inspired by RUP and compared to CRISP-DM/ASUM-DM. It contains no equations, no fitted parameters, no predictions derived from data, and no load-bearing self-citations. The belief that the description 'will facilitate creation of tools' is stated as an assertion rather than a consequence derived from any chain that reduces to the inputs. No patterns from the enumerated list apply; the work is self-contained as a process outline.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that predictive modeling requires a standardized process description to enable automation, drawn from existing software engineering concepts without new parameters or entities.

axioms (1)

domain assumption Predictive modeling has an increasing number of applications in various fields. High demand for predictive models drives creation of tools that automate and support work of data scientist on the model development.
Opening motivation stated in the abstract that justifies the need for the proposed process.

pith-pipeline@v0.9.0 · 5628 in / 1223 out tokens · 30400 ms · 2026-05-25T00:12:59.516846+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

From XAI to MLOps: Explainable Concept Drift Detection with Profile Drift Detection
stat.ML 2024-12 unverdicted novelty 5.0

Profile Drift Detection measures changes in partial dependence profiles with new metrics to detect concept drift while providing explanations and supporting efficient MLOps monitoring.