Pith. sign in

REVIEW 3 major objections 5 minor 16 references

Reproducible White Matter Tract Segmentation Using 3D U-Net on a Large-scale DTI Dataset

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read A 3D U-Net can segment white-matter tracts directly from low-resolution DTI with better scan-rescan reproducibility than the tractography reference.

desk verdict A careful empirical study of 3D U-Net tract segmentation on 9k low-res DTI scans with strong scan-rescan reproducibility evidence, but the accuracy claims rest on the same reference used to create labels. read the letter →

arxiv 1908.10219 v1 pith:7DDLOWLQ submitted 2019-08-26 eess.IV cs.CVcs.LGstat.ML

classification eess.IVcs.CVcs.LGstat.ML
keywords whitemattertractsegmentationdiffusiontensorimaging3DU-Netconvolutionalneuralnetworkscan-rescanreproducibilitylow-resolutionDTIfractionalanisotropymeandiffusivity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that a 3D U-Net can segment white-matter tracts directly from low-resolution diffusion tensor images, replacing a slower tractography pipeline. The method is trained and tested on a large clinical-scale dataset of 9,149 scans, and the authors evaluate both segmentation accuracy and scan-rescan reproducibility. They report that the learned segmentations agree with the tractography-based reference standard (Dice 0.66 for the forceps minor, 0.77 for the corticospinal tract) while achieving higher scan-rescan agreement than that reference, with Cohen's kappa rising from 0.64 to 0.74 for FMI and from 0.72 to 0.80 for CST. The diffusion measures computed inside the segmented tracts are consistent between repeated scans, and the authors conclude that the method can be used in clinical practice and in longitudinal analysis of white-matter microstructure.

What carries the argument

The load-bearing mechanism is a 3D U-Net, a convolutional encoder-decoder network that maps a 3D input volume to a per-voxel segmentation. The input is the voxel-wise diffusion tensor elements restricted to a tract-specific bounding-box region of interest; the network uses batch normalization, PReLU activation, and a weighted inner-product loss with a tract weight of 3. During optimization the authors compared input combinations, loss functions and tract weights, the V-Net variant, and Adam versus Nadam optimizers, and selected the configuration with the best test Dice. The model outputs a binary segmentation of a specific tract in about 0.5 seconds, avoiding tractography reconstruction and storage.

What would settle it

Take an independent cohort with manually annotated forceps minor and corticospinal tract segmentations and compare the trained 3D U-Net output against those annotations; if the Dice coefficients are substantially below the reported 0.66 and 0.77, the claim of comparable performance to a valid reference is not established.

Watch

Extended reading notes

Core claim

The central claim is that direct segmentation of white-matter tracts with a 3D U-Net trained on voxel-wise diffusion tensor elements is reproducible and performs comparably to the tractography-based reference standard. In the evaluation, the method segments a tract in about 0.5 seconds, achieves a mean Dice coefficient (segmentation overlap) of 0.66 for the forceps minor and 0.77 for the corticospinal tract against the reference, and shows higher scan-rescan agreement than the reference (kappa 0.74 vs 0.64 for FMI, 0.80 vs 0.72 for CST). Tract-specific fractional anisotropy and mean diffusivity values from the two scans show no significant paired differences, and the Bland-Altman plots show no consistent bias. The authors conclude that the method is applicable to clinical practice and to longitudinal analysis of white-matter microstructure.

Load-bearing premise

The tractography-based reference standard is assumed to be a correct enough map of each tract's true location, since it serves both as the training label and as the accuracy yardstick.

Editorial extensions

If this is right

  • A single 3D U-Net can replace tractography-based pipelines for segmentation of the forceps minor and corticospinal tract in large cohorts, cutting per-subject runtime to about 0.5 seconds.
  • Tract-specific FA and MD derived from the learned segmentations are stable across repeated scans, so the method can be used to track white-matter changes in longitudinal studies.
  • Scan-rescan reproducibility reaches 'substantial' to 'almost perfect' agreement (kappa 0.74 and 0.80), a level commonly considered acceptable for clinical use.
  • The method works on low-resolution 1.5T clinical images, making it applicable to existing large population datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test is to retrain the same architecture on a small set of manually or independently validated tract annotations; if Dice against that ground truth is much lower than the reported 0.66 and 0.77, the method would be shown to reproduce the tractography pipeline rather than anatomical truth.
  • Because the input is only the diffusion tensor, the same trained model might transfer to other DTI acquisitions, but cross-scanner and cross-protocol generalization is not tested and needs a dedicated dataset.
  • The reported Dice scores compare the method with the reference standard that also generated the training labels, so accuracy and reproducibility cannot be fully separated from the reference's own quality.
  • The speed and reproducibility make it plausible to pool tract-specific measures across multiple cohorts, but harmonization of preprocessing and ROI definitions would be required first.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents a 3D U-Net based method for direct segmentation of the forceps minor (FMI) and corticospinal tract (CST) from low-resolution 1.5T DTI data, trained on a large dataset of 9149 scans from the Rotterdam Study. The authors optimize input representation, loss function, and network architecture, and then evaluate segmentation accuracy (Dice coefficient) against a probabilistic tractography reference standard and reproducibility on a separate 80-subject test-retest cohort using Cohen's kappa, R2 of scan-rescan diffusion measures, and Bland-Altman analysis. They report Dice values of 0.66 (FMI) and 0.77 (CST), and higher scan-rescan reproducibility than the tractography reference (e.g., kappa 0.74 vs 0.64 for FMI, 0.80 vs 0.72 for CST), with tract-specific FA and MD consistent between scans. The authors conclude that the method is applicable to clinical practice and longitudinal studies of white matter microstructure.

Significance. The scale of the training set (9149 scans) and the explicit scan-rescan reproducibility evaluation on 80 subjects are clear strengths, as is the use of a train-independent test cohort of 1036 subjects. The reported speed (0.5 seconds per tract) relative to tractography is practically relevant. If the reproducibility and accuracy claims hold, the method could provide a fast and automated alternative for tract-specific diffusion measures in large epidemiological and clinical studies. The optimization experiments (input, loss, architecture) are useful for practitioners. However, the accuracy evidence is partly circular because the Dice reference is the same tractography output used as training labels, which limits the strength of the 'comparable performance' conclusion.

major comments (3)
  1. [Section 4.2 and Section 5] The central claim of 'comparable performance to the reference standard' is supported by Dice coefficients computed against the de Groot et al. (2015) probabilistic tractography reference (Section 2.4), which is simultaneously the training target. This evaluation is circular: the DC values (0.66 FMI, 0.77 CST) measure how well the network imitates the reference, not agreement with independent anatomical truth. The authors acknowledge the reference is of 'low quality' (Section 5), but do not discuss that any systematic error in the reference is inherited by the learned segmentation and is invisible in the reported accuracy metrics. To support the clinical applicability claim, the authors should either (a) explicitly reframe the accuracy claim as 'agreement with the reference standard' and avoid implying independent accuracy, or (b) provide an independent validation on at least a subsample, e.g., manual delineation on tractography-derived ROIs or comparison to a public high-quality dataset.
  2. [Section 4.2 and Table 1] The higher scan-rescan reproducibility of the proposed method relative to the reference (e.g., kappa 0.74 vs 0.64 for FMI, 0.80 vs 0.72 for CST) is interpreted as a strength. However, a CNN trained on noisy labels may regress toward a mean tract location, thereby reducing scan-to-scan variability at the cost of sensitivity to true anatomical variation. The paper does not address this bias-variance trade-off. Without an independent accuracy anchor, the reproducibility improvement cannot be unambiguously interpreted as a net clinical benefit. Please discuss this possibility, ideally with an analysis of the segmentation boundaries on the test-retest cohort (e.g., comparing disagreement patterns with the reference) to determine whether the higher kappa reflects true stability or smoothing toward the mean.
  3. [Section 2.5 and Section 3.1, Eq. (1)] The loss Lwip in Eq. (1) defines pi as 'the binarized prediction.' If pi is indeed a binary thresholded value, the loss is non-differentiable and no gradient flows to the network, which is incompatible with gradient-based training of a neural network. If pi is intended to be a continuous probability, the definition is incorrect and the equation should be updated. This is a reproducibility issue: readers cannot reproduce the training procedure without knowing the exact differentiable form of the loss and how gradients are computed.
minor comments (5)
  1. [Section 4.2 and Table 1] The text states 'The R2 and K show that our method has better reproducibility than reference,' but the asterisk footnote indicates that only the kappa values and the CST volume difference reach statistical significance (p < .01). The R2 differences (e.g., FA R2 .91 vs .89 for FMI; .84 vs .83 for CST) are small and not tested. Please qualify the statement accordingly.
  2. [Section 4.1 and Fig. 1] It is stated that all input combinations gave similar performance, and the text reports the choice of tensor-only input. However, the actual numerical DC values and standard deviations for the input comparison are not given in the text or the figure. Please report them so that the similarity claim is quantifiable.
  3. [Section 2.2] The dataset decomposition (D1a, D1b, D2, D3) is difficult to follow in prose. A small table or schematic summarizing the sizes, roles, and subject independence of each set would improve clarity.
  4. [Section 2.3] The sentence 'with FLIRT we registered the MNI152 template and T1 weighted image to DTI space' is ambiguous about which image serves as the fixed and moving image. Please clarify the registration targets and the purpose of each registration.
  5. [References] Reference [8] is cited as the arXiv preprint (Wasserthal et al., 2017). A peer-reviewed version of this work exists (Medical Image Analysis, 2019); citing the updated version would be more helpful to readers.

Circularity Check

1 steps flagged · score 3.0 of 10

The 'comparable performance' accuracy claim is benchmarked against the same tractography-derived reference standard used as the training labels, so that part of the validation is partly self-referential; the scan-rescan reproducibility claim is independent.

  1. other [Section 2.4, Section 2.5, Section 3.1 (Eq. 1), Section 5]
    "where ri → {0,1} is the reference standard ... Segmentation accuracy was quantified by the Dice coefficient (DC). ... Another limitation is our low quality reference standard."

    The reference standard of Section 2.4 (de Groot et al. probabilistic tractography with tract-specific thresholds) is both the target r_i in the training loss (Eq. 1) and the reference against which Dice coefficients are computed in Section 2.5. The reported DC values therefore measure how well the network reproduces the same label generator used to train it, not agreement with an independent anatomical gold standard. The conclusion 'comparable performance to the reference standard' is consequently an internal consistency check with the training target, and it cannot reveal systematic biases of the reference. The paper itself acknowledges the reference is of low quality, which is consistent with this limitation.

full rationale

The paper's strongest independent contribution is the scan-rescan reproducibility evaluation: kappa 0.74 vs 0.64 for FMI and 0.80 vs 0.72 for CST, together with R2 values and paired differences for FA, MD, and volume, are computed across repeated scans without reference to the tractography labels. That claim is not circular. The circularity burden is confined to the accuracy/comparability claim: the network is trained to match the de Groot reference standard in Eq. (1), and the same reference standard is then used to compute Dice coefficients and to support the statement of 'comparable performance to the reference standard'. This is a self-referential evaluation of segmentation accuracy rather than an independent validation of anatomical truth, and the paper's own stated limitation about the 'low quality reference standard' makes the concern explicit. No other circularity patterns are present: there is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The authors also compare their CST Dice to the independent method of Wasserthal et al., providing some external context. Overall the central reproducibility claim stands independently, while the accuracy evidence is only partially weakened by the shared reference, warranting a low-to-moderate circularity score.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper contributes an empirical method evaluation rather than a derivation. It introduces no new physical entities. Its results rest on the validity of the tractography reference, the assumption of biological stability over the rescan interval, and two tuned hyperparameters.

free parameters (2)
  • tract weight W in Lwip loss = 3
    Selected by comparing W=1,3,5,10,100 on FMI test Dice (Fig. 2); affects the balance between tract and non-tract voxels in training.
  • binarization threshold for probabilistic output = 0.5
    Used to convert probabilistic network outputs to binary segmentations before computing Dice; chosen by convention, not tuned.
assumptions (3)
  • domain assumption The tractography-based reference standard is a valid surrogate for true white matter tract anatomy.
    Used to generate training labels and to compute Dice accuracy; the paper itself describes the reference as low quality in the Discussion.
  • domain assumption Tract segmentation and diffusion measures are stable over the 19.3-day scan-rescan interval.
    The D3test reproducibility design treats any scan-rescan difference as method noise rather than biological change.
  • domain assumption Low-resolution 1.5T DTI with the tensor model contains enough information for tract segmentation.
    Motivates the use of DTI and tensor elements as network input on the Rotterdam Study scans.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reproducible White Matter Tract Segmentation Using 3D U-Net on a Large-scale DTI Dataset." pith.science (2026). https://pith.science/paper/7DDLOWLQ

@misc{pith2026190810219,
  author       = {Pith},
  title        = {Pith review of: Reproducible White Matter Tract Segmentation Using 3D U-Net on a Large-scale DTI Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7DDLOWLQ}},
  note         = {Machine review of arXiv:1908.10219}
}
read the original abstract

Tract-specific diffusion measures, as derived from brain diffusion MRI, have been linked to white matter tract structural integrity and neurodegeneration. As a consequence, there is a large interest in the automatic segmentation of white matter tract in diffusion tensor MRI data. Methods based on the tractography are popular for white matter tract segmentation. However, because of the limited consistency and long processing time, such methods may not be suitable for clinical practice. We therefore developed a novel convolutional neural network based method to directly segment white matter tract trained on a low-resolution dataset of 9149 DTI images. The method is optimized on input, loss function and network architecture selections. We evaluated both segmentation accuracy and reproducibility, and reproducibility of determining tract-specific diffusion measures. The reproducibility of the method is higher than that of the reference standard and the determined diffusion measures are consistent. Therefore, we expect our method to be applicable in clinical practice and in longitudinal analysis of white matter microstructure.

Figures

Figures reproduced from arXiv: 1908.10219 by the authors.

Figure 1
Figure 1. (left) presents the test DC of FMI for different combinations of input im￾ages. The figure shows that all combinations gave similar performances. There￾fore, we used the simplest and most computation-efficient input, i.e. tensor only. The performance when varying the loss function (Lwip and Lwce) and tract weight is provided in [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Test dice coefficient of FMI using Lwip and Lwce loss function. W indicates the weight of tract [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. provides a visualization of our segmentation result. It overlaps with the reference standard in (a) and (c) for FMI and right CST, respectively. The mean test DC of FMI is 0.66 (SD 0.06), that of CST is 0.77 (SD 0.03) [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: provides subject-wise reproducibility in determining diffusion mea￾sures. The Bland-Altman plots show that almost all differences are within the 95% limits of agreement and the mean of which is close to zero, indicating no consistent bias in longitudinal-measures. Addi…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 14 canonical work pages

  1. [1]

    Neurology 57(4), 632–638 (2001)

    OSullivan, M., Jones, D.K., Summers, P., Morris, R., Will iams, S., Markus, H.: Evidence for cortical disconnection as a mechanism of age-r elated cognitive decline. Neurology 57(4), 632–638 (2001)

  2. [2]

    Alzheimer’s & Dementia 11(3), 321–330 (2015)

    de Groot, M., Ikram, M.A., Akoudad, S., Krestin, G.P., Hof man, A., van der Lugt, A., Niessen, W.J., Vernooij, M.W.: Tract-specific white mat ter degeneration in aging: the rotterdam study. Alzheimer’s & Dementia 11(3), 321–330 (2015)

  3. [3]

    Lawes, I.N.C., Barrick, T.R., Murugam, V., Spierings, N. , Evans, D.R., Song, M., Clark, C.A.: Atlas-based segmentation of white matter trac ts of the human brain using diffusion tensor tractography and comparison with cla ssical dissection. Neu- roimage 39(1), 62–79 (2008)

  4. [4]

    IEEE transactions on medic al imaging 26(11), 1562–1575 (2007)

    O’Donnell, L.J., Westin, C.F.: Automatic tractography s egmentation using a high- dimensional white matter atlas. IEEE transactions on medic al imaging 26(11), 1562–1575 (2007)

  5. [5]

    Neuroimage 127, 277–286 (2016)

    Yendiki, A., Reuter, M., Wilkens, P., Rosas, H.D., Fischl , B.: Joint reconstruction of white-matter pathways from longitudinal diffusion mri da ta with anatomical priors. Neuroimage 127, 277–286 (2016)

  6. [6]

    In: International Conference on Me dical image computing and computer-assisted intervention

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convoluti onal networks for biomedi- cal image segmentation. In: International Conference on Me dical image computing and computer-assisted intervention. pp. 234–241. Springe r (2015)

  7. [7]

    In: 3D Vision (3 DV), 2016 Fourth International Conference on

    Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully conv olutional neural networks for volumetric medical image segmentation. In: 3D Vision (3 DV), 2016 Fourth International Conference on. pp. 565–571. IEEE (2016)

  8. [8]

    Direct White Matter Bundle Segmentation using Stacked U-Nets

    Wasserthal, J., Neher, P.F., Isensee, F., Maier-Hein, K. H.: Direct white matter bundle segmentation using stacked u-nets. arXiv preprint a rXiv:1703.02036 (2017)

Show all 16 references
  1. [9]

    , Franco, O.H., Goedegebure, A., Ikram, M.A., Klaver, C.C., Nijsten, T.E., Peeters, R.P., et al.: The rotterdam study: 2016 objectives and design update

    Hofman, A., Brusselle, G.G., Murad, S.D., van Duijn, C.M. , Franco, O.H., Goedegebure, A., Ikram, M.A., Klaver, C.C., Nijsten, T.E., Peeters, R.P., et al.: The rotterdam study: 2016 objectives and design update. Eur opean journal of epidemiology 30(8), 661–708 (2015)

  2. [10]

    IEEE t ransactions on medical imaging 29(1), 196–205 (2010) Direct and Reproducible White Matter Tract Segmentation 9

    Klein, S., Staring, M., Murphy, K., Viergever, M.A., Plu im, J.P.: Elastix: a tool- box for intensity-based medical image registration. IEEE t ransactions on medical imaging 29(1), 196–205 (2010) Direct and Reproducible White Matter Tract Segmentation 9

  3. [11]

    In: 17th annual meeting of intl soc mag reson med

    Leemans, A., Jeurissen, B., Sijbers, J., Jones, D.: Expl oredti: a graphical toolbox for processing, analyzing, and visualizing diffusion mr dat a. In: 17th annual meeting of intl soc mag reson med. vol. 209, p. 3537. International So ciety for Magnetic Resonance in Medicine Be...

  4. [12]

    Neuroimage 17(2), 825–841 (2002)

    Jenkinson, M., Bannister, P., Brady, M., Smith, S.: Impr oved optimization for the robust and accurate linear registration and motion corr ection of brain images. Neuroimage 17(2), 825–841 (2002)

  5. [13]

    Dozat, T.: Incorporating nesterov momentum into adam (2 016)

  6. [14]

    Journal of Systemics, Cybernetics and Informati cs 8(1), 43–48 (2010)

    Choi, S.S., Cha, S.H., Tappert, C.C.: A survey of binary s imilarity and distance measures. Journal of Systemics, Cybernetics and Informati cs 8(1), 43–48 (2010)

  7. [15]

    arXiv preprint arXiv:1412.6980 (2014)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optim ization. arXiv preprint arXiv:1412.6980 (2014)

  8. [16]

    biometrics pp

    Landis, J.R., Koch, G.G.: The measurement of observer ag reement for categorical data. biometrics pp. 159–174 (1977)

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.