Pith. sign in

REVIEW 3 major objections 5 minor 2 references

Consensus statement on the credibility assessment of ML predictors

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that the credibility of ML predictors in medicine can be estimated with the same seven-step process used for biophysical predictors, with robustness to bias as the key divergence.

desk verdict A useful, honest synthesis of credibility assessment for ML predictors in medicine, with a load-bearing ground-truth precision requirement that will need scoping or revision. read the letter →

arxiv 2501.18415 v1 pith:Y7MHX4C7 submitted 2025-01-30 q-bio.QM cs.LG

Alessandra Aldieri , Thiranja Prasad Babarenda Gamage , Antonino Amedeo La Mattina , Yi Li , Axel Loewe , Francesco Pappalardo , Marco Viceconti Italy This is my paper · ORCID
classification q-bio.QMcs.LG
keywords machinelearningcredibilityassessmentinsilicomedicineuncertaintyquantificationrobustnesstobiascontextofusequantityinterestbiophysicalpredictors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the credibility of a machine-learning medical predictor—its guaranteed accuracy across all allowed inputs—can be assessed with the same seven-step process already used for biophysical models. The process runs from defining the clinical context and error threshold, through measuring true values, quantifying and decomposing prediction error, to checking error distributions and robustness to bias. The authors claim the two model classes differ in one decisive respect: a biophysical model's input set is assumed necessary and sufficient, while an ML predictor's inputs are sufficient but not necessarily necessary. That is why ML robustness cannot be reduced to applicability alone and needs extra strategies such as lifecycle monitoring and a safety layer. The payoff, if correct, is a unified regulatory standard for evaluating ML predictors in high-stakes healthcare.

What carries the argument

The carrying mechanism is the seven-step credibility process (S1–S7), from context-of-use error thresholds through error decomposition to robustness-to-bias analysis, together with the sufficient-but-not-necessary distinction between ML and biophysical input sets. The distinction does the load-bearing work: it explains why ML predictors can hide a necessary observable quantity that never varied in the training set, and why robustness therefore needs dedicated strategies—total product lifecycle monitoring or an input-safety layer—rather than an assumption of smooth applicability.

What would settle it

Find any clinically deployed ML predictor whose credible assessment was performed without a measurement chain ten times more accurate than the error threshold, or run a controlled experiment in which a variable that was silent in training is deliberately varied; if the predictor's error remains within the threshold in the latter case, the central robustness concern identified in Statement #011 would be refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that credibility can be estimated using the same general process for both biophysical and ML predictors (Statement #010): define the context of use and maximum acceptable error; obtain true values through measurement chains at least an order of magnitude more accurate than that error; quantify and decompose prediction error; check that each component behaves as expected; and finally assess robustness to biases and applicability. Statement #011 then identifies the one structural difference: the input set of a biophysical predictor is necessary and sufficient, whereas an ML predictor's input set is sufficient but not necessarily necessary. That difference makes ML robustness assessment irreducible to applicability and directs attention to variables that were silent in training, which is the most critical risk the paper identifies.

Load-bearing premise

The framework assumes that true values for the quantity of interest and its correlated quantities can always be obtained through a measurement chain at least ten times more accurate than the acceptable error; for many clinically important quantities, no such ground-truth measurement exists.

Editorial extensions

If this is right

  • A single seven-step scaffold can guide credibility assessment for both ML and biophysical predictors, which would let regulators reuse familiar procedures when reviewing ML submissions.
  • For ML predictors, passing accuracy tests is not enough; developers must additionally show robustness to bias, either by expanding the certified context as diverse test sets accumulate or by adding a safety layer that detects out-of-population inputs.
  • The safety-layer strategy requires collecting and storing every observable quantity in training and test cohorts, even variables the predictor itself does not use, so that future inputs can be checked against the original population.
  • The framework treats error decomposition for ML predictors as achievable in principle, so developing an ML analogue of verification, validation, and uncertainty quantification becomes a concrete research target.
  • A credibility claim is always conditional on the defined context of use and limits of validity; changing the clinical use changes the error threshold and invalidates the previous assessment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the sufficient-but-not-necessary claim predicts a specific failure mode that can be tested experimentally—train an ML predictor on a cohort where a known causal variable is silent, then vary that variable in a holdout set and measure whether error stays within threshold.
  • Editorial inference: the order-of-magnitude measurement requirement implies that QIs without a high-accuracy reference measurement, such as pain or long-term outcomes, cannot be credentialed under this framework unless surrogate ground truths, like validated biophysical simulators, are accepted as measurement chains.
  • Editorial inference: the safety-layer proposal generalizes beyond medicine—any high-stakes ML deployment could adopt context-specific error thresholds plus distribution-shift checks instead of reporting a single accuracy number.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This position paper from the In Silico World Community of Practice proposes a consensus framework for assessing the credibility of machine learning (ML) predictors in in silico medicine. It introduces twelve statements defining quantities of interest (QIs), implicit versus explicit causal knowledge, and a seven-step credibility assessment process (S1–S7) intended to apply to both biophysical and ML predictors. The central claim is that credibility can be estimated by the same general process for both predictor classes (Statement #010), with ML predictors differing mainly in that their input sets are sufficient but not necessarily necessary, so robustness to bias cannot be reduced to applicability alone (Statement #011). The paper advocates two mitigation strategies: a total product life cycle approach and a safety layer that checks new inputs against training/test distributions.

Significance. If the proposed framework is accepted as a basis for regulatory thinking, it would extend established credibility concepts (ASME V&V40, FDA guidance) to ML predictors and provide a structured vocabulary for discussing error sources, robustness, and applicability. The paper's strengths include its explicit step-by-step process, its concrete engagement with current FDA draft guidance, and its identification of a real gap: the distinction between applicability and robustness for ML predictors whose input sets are not known to be necessary. The strategies of TPLC and safety layers are actionable and testable in principle. However, the paper is a position statement rather than a validated methodology, and its central claim relies on premises that are asserted rather than demonstrated, particularly the availability of high-accuracy ground truth and the adequacy of the proposed error taxonomy.

major comments (3)
  1. [Statement #010, step S2] The hard precondition that true values must come from a measurement chain with accuracy at least one order of magnitude smaller than the context-of-use error threshold is not satisfied for many QIs that the paper explicitly motivates (Statement #001, Introduction, Discussion). For patient-reported pain, individual-level disease risk, or long-term outcomes, no independent measurement chain with the required accuracy exists; the available labels are fallible clinical assessments or delayed outcomes with their own errors. In such cases steps S3–S6 cannot be executed, so the 'same general process' cannot yield a credibility estimate. The paper should either scope the framework to QIs with qualifying measurement chains or propose alternative anchors (e.g., reference-standard uncertainty propagation, validation against clinically relevant endpoints). This is load-bearing because it determines whether the central claim applies to the motivating cases.
  2. [Statement #006 and step S6] The assertion that the aleatoric component of the prediction error 'should be distributed normally' is used as a check in step S6, but no justification is provided. For ML predictors, errors are often heteroscedastic, heavy-tailed, or multimodal, and the decomposition into aleatoric, numerical, and epistemic components is not unique. If the expected distribution is not derived from a measurement model or error-generation mechanism, step S6 becomes unfalsifiable in practice. The authors should either justify the normality assumption for the specific predictor classes they consider or replace it with distributional expectations derived from the identified error sources.
  3. [Statement #011, step S1 (Identification of sources of error)] The claim that both biophysical and ML predictors are affected by 'similar sources of error' (numerical, aleatoric, epistemic) is asserted rather than demonstrated. For ML predictors, the relevant error sources include label noise, sampling and optimization stochasticity, distribution shift, and model misspecification; mapping these onto the three V&V categories is not automatic. Because steps S4–S5 depend on this taxonomy, the paper should provide a worked example (e.g., the cardiac ECG example from the Discussion) showing how the decomposition is performed in practice, or it should revise the taxonomy to reflect ML-specific error sources. Without this, the 'same general process' claim remains an analogy rather than a demonstrated equivalence.
minor comments (5)
  1. [Statements list] The statement numbering skips #005 and #009; the authors should renumber or explicitly state that these are intentionally absent.
  2. [Throughout] The manuscript inconsistently uses 'QI' and 'QoI' (e.g., Statement #003 uses 'QoI' in S3 and S6, elsewhere 'QI'); the notation should be unified.
  3. [Discussion, 'Position the consensus statement'] There is a duplicated phrase: 'for a detailed review, see (see, for example, [18][19])'; this should be corrected.
  4. [Statement #010, step S1] The expression '𝜀!<𝜀' is unclear; it should be typeset as ε < ε_max with a definition of ε_max, since the maximum error threshold is central to the process.
  5. [Consensus process] The manuscript states that all 35 experts supported the consensus statement, but it does not describe how the consensus was reached (e.g., voting, Delphi rounds, or discussion); a brief methodological note would support the 'consensus' claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified: the paper is a normative consensus proposal whose seven-step credibility framework is asserted as a postulate, not derived from fitted data or from a self-citation chain.

full rationale

This is a position/consensus paper rather than a derivation, and the circularity tests do not apply to its central claim. Statement #010 introduces the seven-step process (S1–S7) as a postulate ('credibility itself can be estimated using the same general process'), not as a result derived from data, fitted parameters, or an imported uniqueness theorem. Statement #011 extends the same process to ML predictors by arguing which steps differ; the difference claim rests on the sufficiency-but-not-necessity of ML input sets, which is an independent conceptual argument rather than a restatement of the conclusion. The paper does cite prior regulatory standards (ASME V&V40, FDA guidance) and prior work by the authors (reference [18] and [19]) to define the notion of credibility for biophysical predictors, but those citations are used as context for the consensus framework, not as a load-bearing proof that forces the paper's conclusion. The most significant weakness is Statement #010's S2 precondition that true values require a measurement chain at least one order of magnitude more accurate than the context-of-use error threshold; for QIs such as pain, long-term outcomes, or invasive cardiac measures, such a chain may not exist. That is an applicability gap, not a circularity, because the framework is openly conditional on S2 and does not pretend to derive the needed ground truth from the predictor itself. No step in the paper reduces by construction to its inputs, and no fitted parameter is relabeled as a prediction. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no free parameters and no invented entities. It rests on domain assumptions about causation, measurement accuracy, and error distributions, each of which is asserted rather than derived.

assumptions (4)
  • domain assumption We must have some causal knowledge about the SI to predict the value the QI will assume in SI at a specific time and space.
    Asserted in Statement #004; the entire distinction between explicit and implicit causal knowledge depends on this premise, which is not proven.
  • domain assumption True values can be obtained only through measurement, using a measurement chain that ensures for the QI and the correlated quantities C a class of accuracy at least one order of magnitude smaller than the maximum error.
    S2 of Statement #010; if such measurement chains do not exist for a QI, the proposed credibility assessment process cannot be applied.
  • domain assumption The input set of a biophysical predictor is necessary and sufficient to predict the QI, while for ML predictors it is sufficient but not necessarily necessary.
    Statement #011 S3; the central difference between ML and biophysical predictors in this framework rests on this assertion.
  • domain assumption Aleatoric component of the prediction error should be distributed normally.
    Statement #006; used as an expectation for error distribution checks, without justification for all ML prediction settings.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Consensus statement on the credibility assessment of ML predictors." pith.science (2026). https://pith.science/paper/Y7MHX4C7

@misc{pith2026250118415,
  author       = {Pith},
  title        = {Pith review of: Consensus statement on the credibility assessment of ML predictors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7MHX4C7}},
  note         = {Machine review of arXiv:2501.18415}
}
read the original abstract

The rapid integration of machine learning (ML) predictors into in silico medicine has revolutionized the estimation of quantities of interest (QIs) that are otherwise challenging to measure directly. However, the credibility of these predictors is critical, especially when they inform high-stakes healthcare decisions. This position paper presents a consensus statement developed by experts within the In Silico World Community of Practice. We outline twelve key statements forming the theoretical foundation for evaluating the credibility of ML predictors, emphasizing the necessity of causal knowledge, rigorous error quantification, and robustness to biases. By comparing ML predictors with biophysical models, we highlight unique challenges associated with implicit causal knowledge and propose strategies to ensure reliability and applicability. Our recommendations aim to guide researchers, developers, and regulators in the rigorous assessment and deployment of ML predictors in clinical and biomedical contexts.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [6]

    In Silico World: Lowering barriers to ubiquitous adoption of In Silico Trials

    Education and Training: Implement educational initiatives to enhance understanding of ML predictor development, validation, and credibility assessment among researchers, clinicians, and stakeholders. 7. Interdisciplinary Collaboration: Foster collaboration among data scientists, domain experts, clinicians, and regulators to address the multifaceted challe...

    work page

  2. [11]

    The wisdom hierarchy: representations of the DIKW hierarchy

    Rowley J. The wisdom hierarchy: representations of the DIKW hierarchy. Journal of Information Science 2007; 33:163–180 12. Grote T, Genin K, Sullivan E. Reliability in Machine Learning. Philosophy Compass 2024; 19:e12974 13. Nicora G, Rios M, Abu-Hanna A, et al. Evaluating pointwise reliability of machine learning prediction. Journal of Biomedical Informa...

    work page 2007

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.