REVIEW 3 major objections 5 minor 16 references
Reproducible White Matter Tract Segmentation Using 3D U-Net on a Large-scale DTI Dataset
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A 3D U-Net can segment white-matter tracts directly from low-resolution DTI with better scan-rescan reproducibility than the tractography reference.
desk verdict A careful empirical study of 3D U-Net tract segmentation on 9k low-res DTI scans with strong scan-rescan reproducibility evidence, but the accuracy claims rest on the same reference used to create labels. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a 3D U-Net, a convolutional encoder-decoder network that maps a 3D input volume to a per-voxel segmentation. The input is the voxel-wise diffusion tensor elements restricted to a tract-specific bounding-box region of interest; the network uses batch normalization, PReLU activation, and a weighted inner-product loss with a tract weight of 3. During optimization the authors compared input combinations, loss functions and tract weights, the V-Net variant, and Adam versus Nadam optimizers, and selected the configuration with the best test Dice. The model outputs a binary segmentation of a specific tract in about 0.5 seconds, avoiding tractography reconstruction and storage.
What would settle it
Take an independent cohort with manually annotated forceps minor and corticospinal tract segmentations and compare the trained 3D U-Net output against those annotations; if the Dice coefficients are substantially below the reported 0.66 and 0.77, the claim of comparable performance to a valid reference is not established.
Extended reading notes
Core claim
The central claim is that direct segmentation of white-matter tracts with a 3D U-Net trained on voxel-wise diffusion tensor elements is reproducible and performs comparably to the tractography-based reference standard. In the evaluation, the method segments a tract in about 0.5 seconds, achieves a mean Dice coefficient (segmentation overlap) of 0.66 for the forceps minor and 0.77 for the corticospinal tract against the reference, and shows higher scan-rescan agreement than the reference (kappa 0.74 vs 0.64 for FMI, 0.80 vs 0.72 for CST). Tract-specific fractional anisotropy and mean diffusivity values from the two scans show no significant paired differences, and the Bland-Altman plots show no consistent bias. The authors conclude that the method is applicable to clinical practice and to longitudinal analysis of white-matter microstructure.
Load-bearing premise
The tractography-based reference standard is assumed to be a correct enough map of each tract's true location, since it serves both as the training label and as the accuracy yardstick.
Editorial extensions
If this is right
- A single 3D U-Net can replace tractography-based pipelines for segmentation of the forceps minor and corticospinal tract in large cohorts, cutting per-subject runtime to about 0.5 seconds.
- Tract-specific FA and MD derived from the learned segmentations are stable across repeated scans, so the method can be used to track white-matter changes in longitudinal studies.
- Scan-rescan reproducibility reaches 'substantial' to 'almost perfect' agreement (kappa 0.74 and 0.80), a level commonly considered acceptable for clinical use.
- The method works on low-resolution 1.5T clinical images, making it applicable to existing large population datasets.
Reading between the lines
- A natural test is to retrain the same architecture on a small set of manually or independently validated tract annotations; if Dice against that ground truth is much lower than the reported 0.66 and 0.77, the method would be shown to reproduce the tractography pipeline rather than anatomical truth.
- Because the input is only the diffusion tensor, the same trained model might transfer to other DTI acquisitions, but cross-scanner and cross-protocol generalization is not tested and needs a dedicated dataset.
- The reported Dice scores compare the method with the reference standard that also generated the training labels, so accuracy and reproducibility cannot be fully separated from the reference's own quality.
- The speed and reproducibility make it plausible to pool tract-specific measures across multiple cohorts, but harmonization of preprocessing and ROI definitions would be required first.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a 3D U-Net based method for direct segmentation of the forceps minor (FMI) and corticospinal tract (CST) from low-resolution 1.5T DTI data, trained on a large dataset of 9149 scans from the Rotterdam Study. The authors optimize input representation, loss function, and network architecture, and then evaluate segmentation accuracy (Dice coefficient) against a probabilistic tractography reference standard and reproducibility on a separate 80-subject test-retest cohort using Cohen's kappa, R2 of scan-rescan diffusion measures, and Bland-Altman analysis. They report Dice values of 0.66 (FMI) and 0.77 (CST), and higher scan-rescan reproducibility than the tractography reference (e.g., kappa 0.74 vs 0.64 for FMI, 0.80 vs 0.72 for CST), with tract-specific FA and MD consistent between scans. The authors conclude that the method is applicable to clinical practice and longitudinal studies of white matter microstructure.
Significance. The scale of the training set (9149 scans) and the explicit scan-rescan reproducibility evaluation on 80 subjects are clear strengths, as is the use of a train-independent test cohort of 1036 subjects. The reported speed (0.5 seconds per tract) relative to tractography is practically relevant. If the reproducibility and accuracy claims hold, the method could provide a fast and automated alternative for tract-specific diffusion measures in large epidemiological and clinical studies. The optimization experiments (input, loss, architecture) are useful for practitioners. However, the accuracy evidence is partly circular because the Dice reference is the same tractography output used as training labels, which limits the strength of the 'comparable performance' conclusion.
major comments (3)
- [Section 4.2 and Section 5] The central claim of 'comparable performance to the reference standard' is supported by Dice coefficients computed against the de Groot et al. (2015) probabilistic tractography reference (Section 2.4), which is simultaneously the training target. This evaluation is circular: the DC values (0.66 FMI, 0.77 CST) measure how well the network imitates the reference, not agreement with independent anatomical truth. The authors acknowledge the reference is of 'low quality' (Section 5), but do not discuss that any systematic error in the reference is inherited by the learned segmentation and is invisible in the reported accuracy metrics. To support the clinical applicability claim, the authors should either (a) explicitly reframe the accuracy claim as 'agreement with the reference standard' and avoid implying independent accuracy, or (b) provide an independent validation on at least a subsample, e.g., manual delineation on tractography-derived ROIs or comparison to a public high-quality dataset.
- [Section 4.2 and Table 1] The higher scan-rescan reproducibility of the proposed method relative to the reference (e.g., kappa 0.74 vs 0.64 for FMI, 0.80 vs 0.72 for CST) is interpreted as a strength. However, a CNN trained on noisy labels may regress toward a mean tract location, thereby reducing scan-to-scan variability at the cost of sensitivity to true anatomical variation. The paper does not address this bias-variance trade-off. Without an independent accuracy anchor, the reproducibility improvement cannot be unambiguously interpreted as a net clinical benefit. Please discuss this possibility, ideally with an analysis of the segmentation boundaries on the test-retest cohort (e.g., comparing disagreement patterns with the reference) to determine whether the higher kappa reflects true stability or smoothing toward the mean.
- [Section 2.5 and Section 3.1, Eq. (1)] The loss Lwip in Eq. (1) defines pi as 'the binarized prediction.' If pi is indeed a binary thresholded value, the loss is non-differentiable and no gradient flows to the network, which is incompatible with gradient-based training of a neural network. If pi is intended to be a continuous probability, the definition is incorrect and the equation should be updated. This is a reproducibility issue: readers cannot reproduce the training procedure without knowing the exact differentiable form of the loss and how gradients are computed.
minor comments (5)
- [Section 4.2 and Table 1] The text states 'The R2 and K show that our method has better reproducibility than reference,' but the asterisk footnote indicates that only the kappa values and the CST volume difference reach statistical significance (p < .01). The R2 differences (e.g., FA R2 .91 vs .89 for FMI; .84 vs .83 for CST) are small and not tested. Please qualify the statement accordingly.
- [Section 4.1 and Fig. 1] It is stated that all input combinations gave similar performance, and the text reports the choice of tensor-only input. However, the actual numerical DC values and standard deviations for the input comparison are not given in the text or the figure. Please report them so that the similarity claim is quantifiable.
- [Section 2.2] The dataset decomposition (D1a, D1b, D2, D3) is difficult to follow in prose. A small table or schematic summarizing the sizes, roles, and subject independence of each set would improve clarity.
- [Section 2.3] The sentence 'with FLIRT we registered the MNI152 template and T1 weighted image to DTI space' is ambiguous about which image serves as the fixed and moving image. Please clarify the registration targets and the purpose of each registration.
- [References] Reference [8] is cited as the arXiv preprint (Wasserthal et al., 2017). A peer-reviewed version of this work exists (Medical Image Analysis, 2019); citing the updated version would be more helpful to readers.
Circularity Check
The 'comparable performance' accuracy claim is benchmarked against the same tractography-derived reference standard used as the training labels, so that part of the validation is partly self-referential; the scan-rescan reproducibility claim is independent.
-
other
[Section 2.4, Section 2.5, Section 3.1 (Eq. 1), Section 5]
"where ri → {0,1} is the reference standard ... Segmentation accuracy was quantified by the Dice coefficient (DC). ... Another limitation is our low quality reference standard."
The reference standard of Section 2.4 (de Groot et al. probabilistic tractography with tract-specific thresholds) is both the target r_i in the training loss (Eq. 1) and the reference against which Dice coefficients are computed in Section 2.5. The reported DC values therefore measure how well the network reproduces the same label generator used to train it, not agreement with an independent anatomical gold standard. The conclusion 'comparable performance to the reference standard' is consequently an internal consistency check with the training target, and it cannot reveal systematic biases of the reference. The paper itself acknowledges the reference is of low quality, which is consistent with this limitation.
full rationale
The paper's strongest independent contribution is the scan-rescan reproducibility evaluation: kappa 0.74 vs 0.64 for FMI and 0.80 vs 0.72 for CST, together with R2 values and paired differences for FA, MD, and volume, are computed across repeated scans without reference to the tractography labels. That claim is not circular. The circularity burden is confined to the accuracy/comparability claim: the network is trained to match the de Groot reference standard in Eq. (1), and the same reference standard is then used to compute Dice coefficients and to support the statement of 'comparable performance to the reference standard'. This is a self-referential evaluation of segmentation accuracy rather than an independent validation of anatomical truth, and the paper's own stated limitation about the 'low quality reference standard' makes the concern explicit. No other circularity patterns are present: there is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The authors also compare their CST Dice to the independent method of Wasserthal et al., providing some external context. Overall the central reproducibility claim stands independently, while the accuracy evidence is only partially weakened by the shared reference, warranting a low-to-moderate circularity score.
Assumptions & free parameters
free parameters (2)
- tract weight W in Lwip loss =
3
- binarization threshold for probabilistic output =
0.5
assumptions (3)
- domain assumption The tractography-based reference standard is a valid surrogate for true white matter tract anatomy.
- domain assumption Tract segmentation and diffusion measures are stable over the 19.3-day scan-rescan interval.
- domain assumption Low-resolution 1.5T DTI with the tensor model contains enough information for tract segmentation.
Cite this review
Pith. "Pith review of Reproducible White Matter Tract Segmentation Using 3D U-Net on a Large-scale DTI Dataset." pith.science (2026). https://pith.science/paper/7DDLOWLQ
@misc{pith2026190810219,
author = {Pith},
title = {Pith review of: Reproducible White Matter Tract Segmentation Using 3D U-Net on a Large-scale DTI Dataset},
year = {2026},
howpublished = {\url{https://pith.science/paper/7DDLOWLQ}},
note = {Machine review of arXiv:1908.10219}
}
read the original abstract
Tract-specific diffusion measures, as derived from brain diffusion MRI, have been linked to white matter tract structural integrity and neurodegeneration. As a consequence, there is a large interest in the automatic segmentation of white matter tract in diffusion tensor MRI data. Methods based on the tractography are popular for white matter tract segmentation. However, because of the limited consistency and long processing time, such methods may not be suitable for clinical practice. We therefore developed a novel convolutional neural network based method to directly segment white matter tract trained on a low-resolution dataset of 9149 DTI images. The method is optimized on input, loss function and network architecture selections. We evaluated both segmentation accuracy and reproducibility, and reproducibility of determining tract-specific diffusion measures. The reproducibility of the method is higher than that of the reference standard and the determined diffusion measures are consistent. Therefore, we expect our method to be applicable in clinical practice and in longitudinal analysis of white matter microstructure.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Neurology 57(4), 632–638 (2001)
OSullivan, M., Jones, D.K., Summers, P., Morris, R., Will iams, S., Markus, H.: Evidence for cortical disconnection as a mechanism of age-r elated cognitive decline. Neurology 57(4), 632–638 (2001)
work page 2001
-
[2]
Alzheimer’s & Dementia 11(3), 321–330 (2015)
de Groot, M., Ikram, M.A., Akoudad, S., Krestin, G.P., Hof man, A., van der Lugt, A., Niessen, W.J., Vernooij, M.W.: Tract-specific white mat ter degeneration in aging: the rotterdam study. Alzheimer’s & Dementia 11(3), 321–330 (2015)
work page 2015
-
[3]
Lawes, I.N.C., Barrick, T.R., Murugam, V., Spierings, N. , Evans, D.R., Song, M., Clark, C.A.: Atlas-based segmentation of white matter trac ts of the human brain using diffusion tensor tractography and comparison with cla ssical dissection. Neu- roimage 39(1), 62–79 (2008)
work page 2008
-
[4]
IEEE transactions on medic al imaging 26(11), 1562–1575 (2007)
O’Donnell, L.J., Westin, C.F.: Automatic tractography s egmentation using a high- dimensional white matter atlas. IEEE transactions on medic al imaging 26(11), 1562–1575 (2007)
work page 2007
-
[5]
Neuroimage 127, 277–286 (2016)
Yendiki, A., Reuter, M., Wilkens, P., Rosas, H.D., Fischl , B.: Joint reconstruction of white-matter pathways from longitudinal diffusion mri da ta with anatomical priors. Neuroimage 127, 277–286 (2016)
work page 2016
-
[6]
In: International Conference on Me dical image computing and computer-assisted intervention
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convoluti onal networks for biomedi- cal image segmentation. In: International Conference on Me dical image computing and computer-assisted intervention. pp. 234–241. Springe r (2015)
2015
-
[7]
In: 3D Vision (3 DV), 2016 Fourth International Conference on
Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully conv olutional neural networks for volumetric medical image segmentation. In: 3D Vision (3 DV), 2016 Fourth International Conference on. pp. 565–571. IEEE (2016)
work page 2016
-
[8]
Direct White Matter Bundle Segmentation using Stacked U-Nets
Wasserthal, J., Neher, P.F., Isensee, F., Maier-Hein, K. H.: Direct white matter bundle segmentation using stacked u-nets. arXiv preprint a rXiv:1703.02036 (2017)
work page Pith review arXiv 2017
Show all 16 references
-
[9]
, Franco, O.H., Goedegebure, A., Ikram, M.A., Klaver, C.C., Nijsten, T.E., Peeters, R.P., et al.: The rotterdam study: 2016 objectives and design update
Hofman, A., Brusselle, G.G., Murad, S.D., van Duijn, C.M. , Franco, O.H., Goedegebure, A., Ikram, M.A., Klaver, C.C., Nijsten, T.E., Peeters, R.P., et al.: The rotterdam study: 2016 objectives and design update. Eur opean journal of epidemiology 30(8), 661–708 (2015)
2015
-
[10]
IEEE t ransactions on medical imaging 29(1), 196–205 (2010) Direct and Reproducible White Matter Tract Segmentation 9
Klein, S., Staring, M., Murphy, K., Viergever, M.A., Plu im, J.P.: Elastix: a tool- box for intensity-based medical image registration. IEEE t ransactions on medical imaging 29(1), 196–205 (2010) Direct and Reproducible White Matter Tract Segmentation 9
2010
-
[11]
In: 17th annual meeting of intl soc mag reson med
Leemans, A., Jeurissen, B., Sijbers, J., Jones, D.: Expl oredti: a graphical toolbox for processing, analyzing, and visualizing diffusion mr dat a. In: 17th annual meeting of intl soc mag reson med. vol. 209, p. 3537. International So ciety for Magnetic Resonance in Medicine Be...
2009
-
[12]
Neuroimage 17(2), 825–841 (2002)
Jenkinson, M., Bannister, P., Brady, M., Smith, S.: Impr oved optimization for the robust and accurate linear registration and motion corr ection of brain images. Neuroimage 17(2), 825–841 (2002)
2002
-
[13]
Dozat, T.: Incorporating nesterov momentum into adam (2 016)
-
[14]
Journal of Systemics, Cybernetics and Informati cs 8(1), 43–48 (2010)
Choi, S.S., Cha, S.H., Tappert, C.C.: A survey of binary s imilarity and distance measures. Journal of Systemics, Cybernetics and Informati cs 8(1), 43–48 (2010)
2010
-
[15]
arXiv preprint arXiv:1412.6980 (2014)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optim ization. arXiv preprint arXiv:1412.6980 (2014)
2014 arXiv
-
[16]
biometrics pp
Landis, J.R., Koch, G.G.: The measurement of observer ag reement for categorical data. biometrics pp. 159–174 (1977)
1977
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.