Pith. sign in

REVIEW 3 major objections 4 minor 11 references

A Comprehensive Framework for Automated Segmentation of Perivascular Spaces in Brain MRI with the nnU-Net

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims a voxel-spacing-agnostic nnU-Net can segment brain perivascular spaces on T1w MRI at DSC 85.6±1.4% by combining sparse annotation, iterative label cleaning, and pseudo-labelling, while conceding the gain comes mostly…

desk verdict Useful, honest engineering work, but the headline DSC is measured against model-affected labels and needs independent validation before the robustness claims can be trusted. read the letter →

arxiv 2411.19564 v2 pith:GSIMKQTG submitted 2024-11-29 eess.IV cs.CVcs.LG

William Pham , Alexander Jarema , Donggyu Rim , Zhibin Chen , Mohamed S. H. Khlif , Vaughan G. Macefield , Luke A. Henderson , Amy Brodtmann This is my paper · ORCID
classification eess.IVcs.CVcs.LG
keywords perivascularspacesVirchow-RobinnnU-NetbrainMRIsegmentationT1-weightedsparseannotationpseudo-labellingvoxelspacing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish an automated, publicly released pipeline for segmenting perivascular spaces (PVS), the fluid-filled sleeves around brain blood vessels whose enlargement is a candidate biomarker for small vessel disease, Alzheimer's disease, and Parkinson's disease, from T1-weighted MRI. Working with 30 healthy participants scanned on three different 3T and 7T protocols, the authors train a self-configuring deep-learning segmentation network (nnU-Net) using sparse manual labels on ten slices per scan. Their core finding is that a voxel-spacing agnostic configuration, which does not resample scans to a common resolution, clearly outperforms resampled variants (DSC 64.3% vs. 40.5–55%). After two rounds of iterative label cleaning and retraining with 12,740 pseudo-labelled scans, the model reaches a five-fold cross-validated Dice score of about 85.6%. A sympathetic reader would care because the work supplies a practical tool and a set of design choices, sparse annotation, no resampling, label cleaning, and pseudo-labelling, that other small-structure segmentation projects could reuse.

What carries the argument

The load-bearing machinery is the nnU-Net Residual Encoder, a 3D convolutional segmentation network with self-configuring preprocessing, configured in a voxel-spacing agnostic mode where voxel spacing metadata is ignored and no resampling to a common grid is performed, so small PVS structures are not blurred by interpolation. Around this sits a sparse annotation strategy in which only ten axial slices per scan are manually labelled and the remaining slices are masked with an ignore label trained under a partial loss, an iterative label cleaning loop in which non-local-means denoising plus adaptive histogram equalisation make PVS boundaries clearer for manual revision, and a pseudo-labelling stage where the enhanced model labels 12,740 unannotated T1w images that are added as training data. The combination of these mechanisms, not any single one, is what the paper credits for the final performance.

What would settle it

Have independent experts annotate a fresh set of T1w scans without model assistance and compare the released model's predictions to those labels; if the Dice score lands near the 66.2% the paper reports against its un-revised ground truth rather than near 85.6%, the framework's generalisation claim on raw data would be contradicted. Alternatively, re-run the five-fold cross-validation with the first-round pre-cleaning labels as ground truth; the model's DSC should then be substantially below 80% if the improvement is label realignment.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that its final T1-weighted nnU-Net pipeline is a practical, openly released framework for automated PVS quantification in brain MRI. Specifically, a voxel-spacing agnostic nnU-Net Residual Encoder trained from sparse ten-slice manual labels and refined by iterative label cleaning plus semi-supervised pseudo-labelling segments white matter and basal ganglia PVS across 3T and 7T T1w data with a mean five-fold cross-validated DSC of 85.6±1.4% (white matter 88.6±0.9%, basal ganglia 82.6±2.4%) and Lin's concordance correlation coefficient of 0.89 for predicted versus manual PVS cluster counts. The paper also reports pilot models for midbrain and hippocampal PVS (DSC 64.3±6.5% and 67.8±5.0%) and extended models using T1w+FLAIR and T2w inputs. Importantly, the paper's own comparison against the original pre-cleaning ground truth yields a mean DSC of 66.2%, and it explicitly states that the improvement to 85.6% likely arises from removing poor-quality or noisy labels rather than increasing the model's sensitivity to PVS.

Load-bearing premise

The load-bearing premise is that the iteratively cleaned manual segmentations, revised using the model's own predictions, are an unbiased gold standard, yet the paper concedes the DSC gain comes mainly from deleting noisy labels, so the headline metric partly reflects agreement with those revised labels rather than raw detection accuracy.

Editorial extensions

If this is right

  • Other researchers can apply the released nnU-Net to heterogeneous T1w scans acquired at 3T or 7T without resampling to a common voxel grid, since the spacing-agnostic configuration was the top performer.
  • Sparse annotation with a partial loss can generate full-brain PVS training labels from ten manually labelled slices per scan, reducing the annotation burden that has limited PVS segmentation datasets.
  • Adding pseudo-labels from large open-access repositories does not raise the Dice score but improves agreement of predicted versus manual PVS cluster counts (CCC 0.89), so unlabelled data can be used to make quantification more reliable.
  • The T1w+FLAIR model separates white matter hyperintensities from PVS in one pass, letting studies of cerebral small vessel disease measure both markers without separate lesion-masking tools.
  • Pilot midbrain and hippocampal PVS models give voxel-wise quantification in regions previously measured mainly by coarse rating scales, enabling finer-grained study of PVS there.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the finding that label cleaning explains the DSC gain implies that users of the released model should expect raw-image accuracy closer to the 66% measured against the pre-cleaning labels, and should validate on their own unrevised ground truth.
  • Beyond the paper, the voxel-spacing agnostic result carries a general lesson for other small-structure segmentation tasks such as microbleeds or cortical lesions: resampling to a common grid may be silently reducing accuracy, a hypothesis testable by re-running nnU-Net on raw-resolution data.
  • Beyond the paper, the divergence between DSC and cluster-count concordance in the pseudo-label experiment suggests count-based metrics may be more clinically meaningful than voxel overlap for PVS quantification, and they could be adopted as primary endpoints in future evaluations.
  • Beyond the paper, the low voxel-count concordance in the midbrain and hippocampal models (CCC 0.11–0.29) points to applying the paper's own label-cleaning and enhancement steps to those pilot models as the immediate next test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a comprehensive pipeline for automated segmentation of perivascular spaces (PVS) in brain MRI using the nnU-Net ResEnc architecture. The authors train and evaluate T1w models for white-matter and basal-ganglia PVS using a sparse annotation strategy on 30 healthy participants from three datasets, optimize target voxel spacing and preprocessing (NLMF, AHE), perform iterative label cleaning with model assistance, and add semi-supervised learning with 12,740 pseudo-labels from 18 public datasets. They report a 5-fold cross-validation DSC of 85.6±1.4% for the final T1w model and develop pilot models for midbrain and hippocampal PVS, a T1w+FLAIR model for concurrent WMH segmentation, and a T2w model. The authors claim the resulting framework is robust and holistic for automated PVS quantification, with models made publicly available.

Significance. If the reported performance is accurate, the paper would provide a practically useful, open-source tool for PVS segmentation on heterogeneous T1w MRI, a task of growing interest in cerebral small vessel disease and neurodegenerative research. The paper has notable strengths: it is unusually candid in reporting the comparison against original un-cleaned labels (Section 4.1.5), it makes models publicly available, it evaluates on multi-scanner, multi-field-strength data, and it documents the voxel-spacing-agnostic finding, which is a practically relevant methodological contribution. However, the main numerical claim of 85.6% DSC is measured against labels that were revised with model assistance, and the paper's own analysis shows a 19-point drop (to 66.2%) against the original manual labels. This does not invalidate the engineering contributions, but it means the headline metric cannot be read as an independent accuracy estimate. The pilot models for other regions and sequences are evaluated on very small test sets, further limiting the strength of the 'comprehensive framework' claim.

major comments (3)
  1. [§2.4.4 and §4.1.5] The central claim of robust segmentation at DSC=85.6±1.4% rests on ground-truth labels that were iteratively cleaned with the assistance of the model's own predictions (Section 2.4.4). The paper's own Section 4.1.5 reports that the same final model achieves mean DSC=66.2±7.0% against the original manual labels, and attributes the improvement to 'removing poor-quality or noisy labels rather than increasing the model's sensitivity to PVS.' This is a load-bearing issue: the headline metric partly measures agreement with model-influenced labels, not agreement with an independent gold standard. I recommend that the abstract and conclusion either report both numbers with equal emphasis, or explicitly frame the 85.6% figure as a reproducibility metric within the cleaning loop, and provide an external validation set with independent manual segmentations to support the 'robust' claim.
  2. [§2.6, §2.7, and Table 2] The T1w+FLAIR and T2w models are evaluated on only four and three held-out images respectively, with a train/test split rather than cross-validation. The paper acknowledges this in the Limitations section, but the abstract and conclusion still describe the overall framework as 'comprehensive' and 'robust' without qualification. Given the small test sizes, the reported DSCs (81.1±4.3% and 54.4±4.9%) are not sufficient to support strong claims about these models. Please temper the claims for these modalities or add additional validation data; at minimum, state the test-sample sizes in the abstract.
  3. [§3.1 and §2.9.2] The comparisons among image handling and preprocessing strategies are based on mean DSC values and standard deviations, without any statistical testing. For example, the pseudo-label model (85.6±1.4%) is not statistically different from the iteration-2 label-cleaning model (85.7±1.2%), yet the paper claims that semi-supervised learning improved the model on the basis of concordance metrics alone. I recommend reporting per-fold paired comparisons, confidence intervals, or a repeated-measures test for the main T1w model comparisons, so that readers can assess whether observed differences are within fold-to-fold variability.
minor comments (4)
  1. [§2.4.2 and §4.1.2] There is an inconsistency in the target spacing values: Section 2.4.2 lists 0.75, 0.80, 0.87, and 1.00 mm, while Section 4.1.2 lists 0.7, 0.8, 0.87, and 1.0 mm. Please correct the discussion text to match the methods.
  2. [§2.9] Please clarify whether the pseudo-labels (n=12,740) were kept entirely out of the validation folds in the 5FCV procedure. The text says all pseudo-labels were 'included alongside the training data in each training fold,' which is the expected design, but it should be stated explicitly to avoid ambiguity about possible data leakage.
  3. [§2.8, Eq. (2)] The multi-class dice loss in Eq. (2) uses u and v without defining the superscript k or the summation over i and k clearly in the text; a short sentence defining u_i^k and v_i^k as probability and one-hot label for class k at voxel i would improve readability.
  4. [Figure 8] The caption for Figure 8 (top row) refers to 'middle-left, middle, and middle-right' panels, which is confusing; please use explicit panel labels (a), (b), (c), etc. throughout the figure.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline 5FCV DSC is measured against labels that were revised with the same nnU-Net's predictions; the model scores only 66.2% against the original pre-cleaning labels, making the headline metric partly self-referential.

  1. self definitional [Section 2.4.4 (Iterative label cleaning), Section 3.1.5 and Section 4.1.5 (final evaluation against initial ground truth).]
    "Manual segmentation was conducted by a single rater (WP) then reviewed by an experienced radiologist (AJ). This comprised two rounds of model training and manually quality checking of PVS segmentations. ... when evaluating the final model against the 30 initial ground truth PVS segmentations, the mean DSC score was 66.2%. ... The observed improvement in model performance through iterative label cleaning likely arises by removing poor-quality or noisy labels rather than increasing the model’s sensitivity to PVS."

    The 5FCV ground truth is not independent of the model: the 'manual' segmentations were completed by an nnU-Net and then corrected in two rounds of model-assisted quality checking. The headline DSC (85.6-85.7%) is therefore agreement with labels that have been partially aligned to the model's own predictions. The paper's own control comparison against the original raw-T1w segmentations gives DSC 66.2%, and the paper concedes the improvement reflects label cleaning rather than increased sensitivity. Thus the main performance claim is partly defined by the cleaning procedure rather than by independent truth, though human review and the 66.2% comparison provide some external anchor.

full rationale

The paper is transparent and the nnU-Net pipeline itself is not circular: target spacing, preprocessing, pseudo-labels, and model training are all well-specified and benchmarked against manual labels. The only significant circularity is in the evaluation gold standard. Section 2.4.4 describes two rounds of model training and manual quality checking; Section 2.4.1 states that a nnU-Net was used to complete the sparse annotations, so the final labels are partly model-generated. The 5FCV DSC is then computed against those model-influenced labels. The paper's own Section 3.1.5 reports DSC 66.2% against the original pre-cleaning segmentations, and Section 4.1.5 explicitly attributes the improvement to removing noisy labels rather than improved detection. This makes the headline 85.6% partially self-referential and in need of independent validation. No self-citation chain, uniqueness theorem, or ansatz-smuggling is load-bearing; the issue is the non-independence of the ground truth, not the derivation from equations. Score 6 reflects partial, acknowledged circularity in the central performance claim.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework rests on human-defined PVS labels as ground truth, on model-assisted label cleaning preserving label validity, on the accuracy of self-generated pseudo-labels, and on co-registration accuracy for multi-sequence models. No new physical or anatomical entities are introduced.

free parameters (3)
  • Number of axial slices annotated per image = 10 slices (2 brainstem, 4 basal ganglia, 4 centrum semiovale)
    Hand-chosen annotation protocol; the paper does not justify this count or test its sensitivity.
  • Pseudo-label inclusion criteria = 12,740 images, field strength ≥3T, slice thickness ≤1.2 mm
    Thresholds chosen without ablation; they determine the semi-supervised training set and therefore the reported cluster-count concordance.
  • Iterative label cleaning rounds = 2 rounds
    Stopping criterion not specified; performance would likely change with additional rounds, and the headline DSC depends on this choice.
assumptions (4)
  • domain assumption Manual segmentations of PVS in T1w MRI are a valid gold standard.
    The paper treats the manually corrected labels as ground truth for DSC and concordance, but the cleaning process was model-assisted.
  • domain assumption PVS visible on T1w MRI at 3T and 7T can be reliably distinguished from mimics such as white matter hyperintensities.
    The entire labelling strategy assumes T1w contrast is sufficient for PVS identification; the paper relies on visual inspection and radiological review.
  • ad hoc to paper Pseudo-labels generated by the enhanced nnU-Net are accurate enough to improve generalization.
    The semi-supervised step uses 12,740 model-generated labels as training data; the paper does not quantify pseudo-label noise.
  • domain assumption Co-registration of FLAIR and T2w images to T1w is sufficiently accurate for label transfer.
    The T1w+FLAIR and T2w models rely on rigid/affine registration; the paper notes co-registration errors eroded T2w model performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Comprehensive Framework for Automated Segmentation of Perivascular Spaces in Brain MRI with the nnU-Net." pith.science (2026). https://pith.science/paper/GSIMKQTG

@misc{pith2026241119564,
  author       = {Pith},
  title        = {Pith review of: A Comprehensive Framework for Automated Segmentation of Perivascular Spaces in Brain MRI with the nnU-Net},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GSIMKQTG}},
  note         = {Machine review of arXiv:2411.19564}
}
abstract

Background: Enlargement of perivascular spaces (PVS) is common in neurodegenerative disorders including cerebral small vessel disease, Alzheimer's disease, and Parkinson's disease. PVS enlargement may indicate impaired clearance pathways and there is a need for reliable PVS detection methods which are currently lacking. Aim: To optimise a widely used deep learning model, the no-new-UNet (nnU-Net), for PVS segmentation. Methods: In 30 healthy participants (mean$\pm$SD age: 50$\pm$18.9 years; 13 females), T1-weighted MRI images were acquired using three different protocols on three MRI scanners (3T Siemens Tim Trio, 3T Philips Achieva, and 7T Siemens Magnetom). PVS were manually segmented across ten axial slices in each participant. Segmentations were completed using a sparse annotation strategy. In total, 11 models were compared using various strategies for image handling, preprocessing and semi-supervised learning with pseudo-labels. Model performance was evaluated using 5-fold cross validation (5FCV). The main performance metric was the Dice Similarity Coefficient (DSC). Results: The voxel-spacing agnostic model (mean$\pm$SD DSC=64.3$\pm$3.3%) outperformed models which resampled images to a common resolution (DSC=40.5-55%). Model performance improved substantially following iterative label cleaning (DSC=85.7$\pm$1.2%). Semi-supervised learning with pseudo-labels (n=12,740) from 18 additional datasets improved the agreement between raw and predicted PVS cluster counts (Lin's concordance correlation coefficient=0.89, 95%CI=0.82-0.94). We extended the model to enable PVS segmentation in the midbrain (DSC=64.3$\pm$6.5%) and hippocampus (DSC=67.8$\pm$5%). Conclusions: Our deep learning models provide a robust and holistic framework for the automated quantification of PVS in brain MRI.

Figures

Figures reproduced from arXiv: 2411.19564 by the authors.

Figure 1
Figure 1. Overall workflow of the study. Panel 1: development of the nnU-Net PVS segmentation in the white matter (WM) and basal ganglia (BG) using T1-weighted (T1w) MRI. Sub-panels show the acquisition of initial labelled data with a sparse annotation strategy, followed by model optimisation by comparing image handling parameters, preprocessing techniques, iterative label cleaning, and semi-supervised learning with pseudo-la… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 4 canonical work pages

  1. [1]

    Adams, H. H. H., Cavalieri, M., Verhaaren, B. F. J., Bos, D., Van Der Lugt, A., Enzinger, C., Vernooij, M. W., Schmidt, R., & Ikram, M. A. (2013). Rating Method for Dilated Virchow–Robin Spaces on Magnetic Resonance Imaging. Stroke, 44(6), 1732–1735. https://doi.org/10.1161/STROKEAHA.111.000620 Adams, H. H. H., Hilal, S., Schwingenschuh, P., Wittfeld, K.,...

    work page doi:10.1161/strokeaha.111.000620 2013

  2. [15]

    https://doi.org/10.3389/fninf.2021.641600 Brodtmann, A., Werden, E., Pardoe, H., Li, Q., Jackson, G., Donnan, G., Cowie, T., Bradshaw, J., Darby, D., & Cumming, T. (2014). Charting Cognitive and Volumetric Trajectories after Stroke: Protocol for the Cognition and Neocortical Volume after Stroke (CANVAS) Study. International Journal of Stroke, 9(6), 824–82...

    work page arXiv 2014

  3. [23]

    P., Choupan, J., Law, M., & Toga, A

    https://doi.org/10.1016/j.nicl.2019.101849 Sepehrband, F., Barisano, G., Sheikh-Bahaei, N., Cabeen, R. P., Choupan, J., Law, M., & Toga, A. W. (2019). Image processing approaches to enhance perivascular space visibility and quantification using MRI. Scientific Reports, 9(1). https://doi.org/10.1038/s41598-019-48910-x Shen, T., Yue, Y., Zhao, S., Xie, J., ...

    work page arXiv 2019

  4. [30]

    F., Ziegler, S., Krämer, L., Denner, S., Xiao, S., Disch, N., Maier-Hein, K

    https://doi.org/10.3389/FNINF.2014.00030/BIBTEX Gotkowski, K., Lüth, C., Jäger, P. F., Ziegler, S., Krämer, L., Denner, S., Xiao, S., Disch, N., Maier-Hein, K. H., & Isensee, F. (2024). Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation (arXiv:2403.12834). arXiv. http://arxiv.org/abs/2403.12834 Henschel, L., Conjeti, S., Estrada, S., D...

    work page arXiv 2024

  5. [152]

    H., Zong, X., Gao, Y., Lin, W., & Shen, D

    https://doi.org/10.3389/FNINS.2012.00152/BIBTEX Park, S. H., Zong, X., Gao, Y., Lin, W., & Shen, D. (2016). Segmentation of perivascular spaces in 7 T MR image using auto-context model with orientation-normalized features. NeuroImage, 134, 223–235. https://doi.org/10.1016/j.neuroimage.2016.03.076 Petersen, R. C., Aisen, P. S., Beckett, L. A., Donohue, M. ...

    arXiv 2016

  6. [219]

    J., Lee, H., Yu, M., Feng, T., Logan, J., Nedergaard, M., & Benveniste, H

    https://doi.org/10.1016/j.neuroimage.2020.117012 Iliff, J. J., Lee, H., Yu, M., Feng, T., Logan, J., Nedergaard, M., & Benveniste, H. (2013). Brain-wide pathway for waste clearance captured by contrast-enhanced MRI. Journal of Clinical Investigation, 123(3), 1299–1309. https://doi.org/10.1172/JCI67677 Iliff, J. J., Wang, M., Liao, Y., Plogg, B. A., Peng, ...

    arXiv 2013

  7. [298]

    B., Epstein, C

    https://doi.org/10.1038/s41597- 022-01426-y Avants, B. B., Epstein, C. L., Grossman, M., & Gee, J. C. (2008). Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis, 12(1), 26–41. https://doi.org/10.1016/j.media.2007.06.004 Avants, B. B., Tustison, N. ...

    doi:10.1038/s41597- 2008

  8. [558]

    https://doi.org/10.1049/iet-ipr.2011.0161 Coupe, P., Yger, P., Prima, S., Hellier, P., Kervrann, C., & Barillot, C. (2008). An Optimized Blockwise Nonlocal Means Denoising Filter for 3-D Magnetic Resonance Images. IEEE Transactions on Medical Imaging, 27(4), 425–441. https://doi.org/10.1109/TMI.2007.906087 Crawford, L. S., Mills, E. P., Hanson, T., Macey,...

    arXiv 2008

Show all 11 references
  1. [656]

    K., & Gholipour, A

    https://doi.org/10.1136/jnnp-2017-316724 Karimi, D., Dou, H., Warfield, S. K., & Gholipour, A. (2020). Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis. Medical Image Analysis, 65, 101759. https://doi.org/10.1016/j.media.2020.101759 ...

    2020

  2. [2013]

    https://doi.org/10.1016/S1474-4422(23)00131-X Eide, P

    The Lancet Neurology, 22(7), 602–618. https://doi.org/10.1016/S1474-4422(23)00131-X Eide, P. K., Vinje, V., Pripp, A. H., Mardal, K. A., & Ringstad, G. (2021). Sleep deprivation impairs molecular clearance from the human brain. Brain, 144(3), 863–874. https://doi.org/10.1093/b...

    2021

  3. [2022]

    https://doi.org/10.1002/alz.13449 Walt, S

    Alzheimer’s and Dementia, August, 1–43. https://doi.org/10.1002/alz.13449 Walt, S. van der, Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., Gouillart, E., & Yu, T. (2014). scikit-image: Image processing in Python. PeerJ, 2, e453. https://doi.org...

    2014

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.