REVIEW 2 major objections 4 minor
A Systematic Benchmark of Intensity Normalisation Methods for 3D Knee MRI Segmentation and Cross-Domain Generalisability
T0 review · 2 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read For knee MRI meniscus segmentation, the choice of intensity normalisation has a statistically significant but tiny effect on cross-domain performance; the dominant factor is the domain shift itself.
desk verdict A solid, well-run benchmark of seven normalisation methods for knee MRI segmentation; the main caveat is the missing no-normalisation control, which makes the 'intensity normalisation matters' framing slightly oversold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The evaluative engine is a controlled benchmark: seven normalisation methods - Z-score, min-max scaling, robust min-max, histogram equalisation, CLAHE, Nyul histogram matching, and a four-component Gaussian mixture model variant inspired by white-stripe normalisation - are applied identically to training and test images before training a 3D U-Net with five-fold cross-validation using shared splits. The statistical load-bearer is a linear mixed-effects model of the form Metric ~ Method + (1|Fold) + (1|TestImage), which separates the tiny fixed effect of normalisation from the large random effect of test image and the negligible fold effect. This design allows small cross-method differences to
What would settle it
Retrain the same 3D U-Net on IWOAI 2019 with GMM normalisation using each of the four fitted components (or a tissue-validated peak selection) and evaluate on SKM-TEA; if the ranking of GMM moves from the bottom group into the top group, the paper's method-specific conclusion about GMM would not hold. More broadly, if the same seven-method benchmark applied to an external dataset with milder protocol differences showed normalisation explaining more than a few percent of Dice variance, the claim that domain shift dwarfs normalisation would need qualification.
Extended reading notes
Core claim
On its own terms, the paper's central finding is that a systematic comparison of seven standard intensity normalisation schemes for knee MRI segmentation finds statistically significant differences only on external data, where Nyul histogram matching, Z-score, and CLAHE group ahead of the other four methods. Yet these differences are an order of magnitude smaller than the gap between internal and external datasets: estimated external Dice ranges from about 78.3% for histogram equalisation to 79.1% for Nyul, while all methods lose roughly 10 points relative to the about 89% internal baseline. A linear mixed-effects model attributes nearly all Dice variance to differences between test images (
Load-bearing premise
The GMM normalisation assumes that the fitted component with the second-largest mean represents the same tissue type in both the DESS and qDESS protocols, and the paper offers no evidence or sensitivity analysis for that mapping.
Editorial extensions
If this is right
- For practitioners deploying knee MRI segmentation models to unseen scanners or protocols, choosing Nyul, Z-score, or CLAHE is a cheap and safe default, but expecting normalisation to close the external-data gap is unrealistic.
- The roughly 10-point Dice drop between datasets means domain shift - protocol differences, patient population, annotation style - dominates performance; effort should shift toward harmonisation, domain adaptation, or targeted fine-tuning.
- Nyul's better alignment of intensity distributions across datasets did not translate into a meaningful segmentation gain over Z-score, suggesting histogram matching is not a substitute for true domain adaptation.
- The significant but tiny method effect (marginal R-squared 0.005) implies that studies evaluating normalisation on internal data alone cannot detect differences; external validation is necessary.
- The absence of any HD95 differences across methods indicates that segmentation shape is insensitive to normalisation choice, even when volume-based scores differ slightly.
Reading between the lines
- Because the two datasets differ in many simultaneous ways (sequence variant, resolution, population, annotators), the estimated 0.5% variance share for normalisation may be specific to this domain pair; on a pair with more similar protocols, normalisation's relative contribution could be larger.
- The GMM method's weak external performance could be an artifact of its component-selection heuristic rather than a failure of tissue-peak normalisation in general; a sensitivity analysis over which Gaussian component is used is a direct next experiment.
- A testable extension is to combine the top normalisation methods with intensity augmentation or a harmonisation step to see whether their small gains are additive, redundant, or even interfering.
- Z-score's reported about 2% systematic under-estimation of meniscal volume, concentrated at boundaries, could matter for downstream morphometric measurements even when Dice is unaffected - worth direct measurement.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a systematic comparison of seven intensity normalisation methods (Z-score, min-max, robust min-max, histogram equalisation, CLAHE, Nyúl histogram matching, and a GMM-based method) for 3D U-Net meniscus segmentation in knee MRI. All models are trained on the IWOAI 2019 DESS dataset with identical 5-fold splits and evaluated on an internal test split and on the external SKM-TEA qDESS dataset. The main empirical findings are that internal performance differs little across methods, while externally small but statistically significant differences emerge: Nyúl, Z-score, and CLAHE are most robust, with differences of roughly 0.1–0.6% DSC among methods and a marginal R² of 0.005, compared with an overall ~10 percentage-point drop from internal to external data. The authors conclude that intensity normalisation has a measurable but limited effect on generalisability relative to domain shift.
Significance. If the results hold, the paper is a useful empirical benchmark for an underexplored area: intensity normalisation for musculoskeletal MRI segmentation under domain shift. It has several concrete strengths: use of public datasets, identical cross-validation splits across all methods, a strong nnU-Net baseline, appropriate linear mixed-effects models with fold and test-image random intercepts, Tukey-adjusted pairwise comparisons, and public code. The headline quantitative result — normalisation method explains 0.5% of variance while domain shift costs ~10 DSC points — is a clear and actionable message for deployment decisions. The main weakness is that all arms receive a normalisation transform, so the design supports ranking the seven methods but not an absolute statement about the effect of normalisation versus none; this condition on the headline claim needs to be addressed.
major comments (2)
- [Abstract; §3.2, Eq. (1), Table 2] The claimed effect of 'intensity normalisation' is not directly supported because there is no raw/no-normalisation control arm. Every arm applies one of seven transforms, so Eq. (1) and the marginal R² = 0.005 measure differences among those transforms, not the effect of normalisation versus none. The abstract's sentence 'intensity normalisation had a measurable effect on model generalisability' is therefore stronger than the design permits. Please add a no-normalisation arm (or at least an nnU-Net-default-preprocessing arm) or, if that is infeasible, restate the causal claims as being about choice among the seven tested methods.
- [§2.4] The text states that experiments were planned using default nnU-Net settings. nnU-Net applies its own intensity normalisation in its preprocessing pipeline; the manuscript does not state whether this internal normalisation was disabled or overridden. If it remained active, linear methods such as Z-score and min-max would be conflated by a subsequent z-score transform. The numerical differences in Table 1 suggest the internal normalisation was adapted, but this must be stated explicitly for the benchmark to be interpretable and reproducible. Please specify the exact nnU-Net data-preprocessing configuration used.
minor comments (4)
- [§2.3] The GMM normalisation is under-specified: no equation is given for how the selected component mean is used to standardise the image, no argument is given that the 'second largest mean' component corresponds to a stable tissue class across DESS and qDESS, and no sensitivity analysis is reported for the number of components or component-selection rule. Because GMM changes rank between internal and external data and the Discussion interprets this, the implementation and its stability should be documented or the interpretation limited.
- [Fig. 2 caption] The caption refers to 'the five methods' but seven methods are compared; the figure actually shows seven estimated marginal means.
- [§3.2, Table 2] The unadjusted 95% CI for Nyúl is [0.05, 0.24] while the Tukey-adjusted pairwise p-value vs. Z-score is 0.053. These are not formally inconsistent because the CI is unadjusted, but the text should say so explicitly to avoid confusion.
- [Throughout] Minor typographical issues: 'has been been shown' (§1.1), 'worse that all methods' (§3.1), and '1.5×IQR' should read '1.5·IQR' or '1.5 IQR'.
Circularity Check
Empirical benchmark; no circular derivation. Central conclusions are measured from held-out external data and are not forced by construction.
full rationale
This is a systematic empirical comparison of seven intensity normalisation methods for knee MRI segmentation, not a derivation. Each method is an independent preprocessing transform; models are trained on IWOAI 2019 and evaluated on a held-out internal split and on the external SKM-TEA dataset. The statistical analysis (LMM, Type III ANOVA, marginal R²) is applied to measured Dice scores, and none of the reported quantities is defined in terms of the conclusions. No fitted parameter is renamed as a prediction: the Nyúl template is learned from training data and applied to test data, which is standard model fitting rather than circularity. The single self-citation (ref. [29], the authors' prior SAM paper) is used only to justify the choice of nnU-Net over foundation models and is not load-bearing for the claimed normalisation results. The absence of a raw/normalisation-free arm is a limitation in interpreting the phrase 'intensity normalisation had a measurable effect', but that is an experimental-design gap, not a claim that reduces to its own inputs by construction. The GMM component-selection concern is a robustness limitation, not a circular step. No uniqueness theorem, imported ansatz, or renaming of a known result is involved. The paper is self-contained as an empirical benchmark against public datasets, so no significant circularity is found.
Assumptions & free parameters
free parameters (3)
- GMM number of components and selected component =
4 components; second largest mean
- CLAHE kernel size and clip limit =
1/10th of image dimensions; clip limit 0.01
- Robust min-max percentile clip =
1st and 99th percentiles
assumptions (4)
- domain assumption Root-sum-of-squares combination of the two qDESS echos is a valid way to create a single image for segmentation
- ad hoc to paper The second largest mean GMM component corresponds to a meaningful, stable intensity peak in both DESS and qDESS
- standard math Linear mixed-effects model assumptions (normality of residuals, random intercepts) hold for DSC and HD95
- domain assumption The manual segmentations in both datasets are reliable enough to serve as ground truth for cross-dataset comparison
Cite this review
Pith. "Pith review of A Systematic Benchmark of Intensity Normalisation Methods for 3D Knee MRI Segmentation and Cross-Domain Generalisability." pith.science (2026). https://pith.science/paper/QNYOA3WM
@misc{pith2026260720028,
author = {Pith},
title = {Pith review of: A Systematic Benchmark of Intensity Normalisation Methods for 3D Knee MRI Segmentation and Cross-Domain Generalisability},
year = {2026},
howpublished = {\url{https://pith.science/paper/QNYOA3WM}},
note = {Machine review of arXiv:2607.20028}
}
read the original abstract
Robust out-of-the-box performance is essential for the clinical deployment of deep learning models in medical imaging. An important but underexplored factor affecting model generalisability is intensity normalisation, particularly for magnetic resonance imaging (MRI), where image intensities vary across scanners and protocols. In this study, we systematically compared seven normalisation methods and their impact on the performance of a 3D U-Net model for meniscus segmentation from knee MRI. The methods included standard scaling approaches, histogram-based techniques, and a Gaussian Mixture Model (GMM)-based method. Models were trained on the IWOAI 2019 dataset and evaluated on both internal and external test sets (SKM-TEA) to assess generalisability. Performance was similar internally but differences were significant on external data, with Z-score, Ny\'ul histogram matching, and CLAHE showing greater robustness than other methods. However, these differences were small compared to the significant performance drop observed between datasets. Overall, while intensity normalisation had a measurable effect on model generalisability, its impact was limited relative to the effects of domain shift, highlighting the need for complementary strategies for robust deployment.
Figures
Figures from the paper (2 more)
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.