Pith. sign in

REVIEW 4 major objections 6 minor 13 references

Cross-modality Knowledge Transfer for Prostate Segmentation from CT Scans

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Synthetic CT generated from unpaired MRI can train a prostate segmenter that performs on real CT as well as on the synthetic images.

desk verdict A plausible MR-to-CT transfer pipeline with an honest incremental result, but the headline equivalence claim rests on an unreported statistical test. read the letter →

arxiv 1908.10208 v2 pith:TZX2XUVD submitted 2019-08-26 eess.IV cs.LGphysics.med-phstat.ML

classification eess.IVcs.LGphysics.med-phstat.ML
keywords prostatesegmentationsyntheticCTCycleGANdomainadaptationMR-to-CTsynthesis2.5DresidualU-NetSSIMlossmedicalimage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that prostate segmentation on CT scans can be learned without a single CT ground-truth annotation. The method first uses unpaired CycleGAN to translate MR images into realistic synthetic CT (SynCT), with a structural similarity (SSIM) based cycle loss intended to keep the anatomy of the MR image intact. The MR delineations are then used as supervision for SynCT, training a 2.5D residual U-Net that is finally tested on real CT scans. The paper reports a Dice score of $0.73 \pm 0.09$ on real CT, statistically indistinguishable ($p>0.05$) from its performance on SynCT, suggesting that the synthetic images carry the label information across modalities. A sympathetic reader would care because it offers a way to segment a modality that lacks annotations by borrowing labels from a richly annotated one.

What carries the argument

The load-bearing mechanism is a CycleGAN whose cycle-consistency loss is replaced by a structural similarity index (SSIM) based cost, which compares local patches of the reconstructed MR with the original MR instead of pixel-level mean squared error. CycleGAN itself is an unpaired image-to-image translation framework with two generator-discriminator pairs and two inverse generators enforcing cycle consistency; here it maps MR to synthetic CT and back. The SSIM cycle loss is what the paper credits for improving anatomical similarity between real and synthetic CT. On the segmentation side, a 2.5D Residual U-Net takes three adjacent slices as input (context number 1) and adds short residual skip connections to the standard U-Net long skips; it is trained on SynCT and evaluated on real CT.

What would settle it

Take any patient with both a prostate MR and a CT scan that are co-registered, generate SynCT from the MR, and compute the Dice or Hausdorff distance between the MR-derived prostate mask projected onto SynCT and a radiologist's CT contour. If that distance is large, the SSIM cycle loss does not preserve boundary geometry, and the reported real-CT segmentation success must be attributed to something other than label transfer.

Watch

Extended reading notes

Core claim

The central claim is that unpaired image-to-image translation can transfer segmentation labels from MRI to CT without any paired data or CT annotations. Specifically, the CycleGAN generator maps MR into SynCT while the SSIM-based cycle-consistency term is asserted to guarantee that SynCT shares the identical manually drawn masks originally delineated on MR. A 2.5D Res-U-Net trained only on SynCT with those transferred masks achieves a Dice score of $0.73 \pm 0.09$ on real CT, and the paper reports no statistical difference between segmenting SynCT and segmenting real CT ($p>0.05$), interpreting this as evidence that the synthetic domain has become indistinguishable from the target domain from the segmenter's perspective.

Load-bearing premise

The cycle-consistency term is assumed to preserve the prostate boundary geometry when translating MR to synthetic CT, so that MR masks remain valid supervision for SynCT, but the paper provides no direct anatomical consistency check to confirm this.

Editorial extensions

If this is right

  • CT prostate segmentation becomes feasible in settings where no CT annotations exist, as long as unpaired MR images with prostate masks are available.
  • The reported parity between SynCT and real CT ($p>0.05$) implies the segmenter has effectively adapted to the target modality without ever seeing a labeled CT.
  • The SSIM cycle loss, by raising real-CT Dice from 0.68 to 0.73 while lowering SynCT Dice slightly, appears to trade off pure source-domain accuracy for better domain alignment.
  • The 2.5D input with three slices is the empirically optimal context depth; more slices (5 or 7) do not help and can degrade Dice significantly on CT.
  • The approach extends naturally to other organs or modality pairs where one modality has abundant annotations and the other does not.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the label-preservation claim would be to have radiologists contour SynCT and compare those contours with the original MR masks; the paper does not perform this check, so the equivalence remains an inference from segmentation scores.
  • Because cycle consistency is an image-level reconstruction constraint, it likely preserves coarse anatomy but not necessarily fine boundaries; adding an explicit boundary or shape loss to CycleGAN could close the remaining gap to the MRI upper bound (0.90 Dice).
  • The no-statistical-difference result suggests that, for training deep segmenters, synthetic CT may be interchangeable with real CT; this could be leveraged in radiotherapy planning pipelines where only CT is available for dosimetry but MR defines target volumes.
  • The method's reliance on unpaired data means its applicability does not require co-registered MR-CT pairs, making it attractive for retrospective multi-center datasets.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a two-step pipeline for prostate segmentation from CT scans without CT ground-truth labels. First, a CycleGAN with an SSIM-based cycle-consistency loss synthesizes 'SynCT' images from T2-weighted prostate MR images. Second, a 2.5D Residual U-Net is trained on the SynCT images paired with the original MR segmentations and tested on real CT scans. Using 19 manually segmented real CT patients, the authors report a mean Dice score of 0.73 ± 0.09, which they state is not statistically different from the 0.80 ± 0.12 obtained on SynCT test images. The paper also examines the effect of context size in the 2.5D input and ablates intensity clipping, data augmentation, and the cycle loss.

Significance. If the result were rigorously established, the paper would address a clinically relevant problem: exploiting abundant MR segmentations to segment CT in the absence of CT labels. The work uses public datasets and provides a systematic ablation of preprocessing, augmentation, and the cycle loss, which is useful for reproducibility. The SSIM-based cycle loss is a plausible improvement over plain MSE reconstruction. However, the central empirical claim is not yet established because the statistical comparison is under-reported and several components appear to be tuned on the test set. The paper therefore currently serves as a promising proof-of-concept rather than a definitive demonstration.

major comments (4)
  1. [Section 3, Table 1] The claim of 'no statistical difference (p>0.05)' between SynCT and CT testing sets is not supported by any reported test. The manuscript does not state which test was used, whether the SynCT and CT test sets are paired, or how multiple slices per patient were handled; the SynCT test set is derived from 60 MRI cases while the CT test set has 19 patients, so the comparison is not obviously paired. Please report the exact test, sample sizes, test statistic, and a confidence interval for the difference. Note also that p>0.05 is not evidence of equivalence; an equivalence test or a pre-specified non-inferiority margin would be needed to support the paper's conclusion that the SynCT-trained network performs comparably on real CT.
  2. [Section 3, Table 1 and Figure 3] Several design choices—CT intensity clipping, the data augmentation schedule, the SSIM cycle loss, and the context number for 2.5D input—were selected after observing performance on the 19-patient CT test set. This selection process makes the reported 0.73 ± 0.09 an optimistic estimate of generalization to unseen CT data. Please use a separate validation set or nested cross-validation for model selection, and report the final test-set results only after all choices are fixed. As written, the paper overstates the reliability of the reported Dice scores.
  3. [Abstract and Section 2.2] The claim that the cycle-consistency term 'guarantees that SynCT shares the identical manually-drawn masks' is stronger than what the loss actually enforces. The SSIM/MSE cycle loss operates on image appearance on a per-pixel or local-patch basis; it does not by itself guarantee semantic label preservation. No direct measurement of anatomical overlap between the generated SynCT and the original MR masks is provided. Please add a direct check, such as the Dice overlap between the original MR prostate mask and the corresponding region in the reconstructed MR after the cyclic translation, or a reader-based anatomical consistency assessment on a sample of cases.
  4. [Section 3 and Discussion] The paper does not include a baseline that trains the same 2.5D Res-U-Net directly on real MR images and tests on real CT, nor a CycleGAN baseline under otherwise identical settings that differs only in the cycle loss (the MSE-loss row changes the augmentation schedule and input cropping simultaneously). Without such baselines, the contribution of the synthetic CT pipeline to the CT Dice score is not quantified, and the reader cannot assess whether the observed performance is attributable to the proposed knowledge-transfer mechanism.
minor comments (6)
  1. [Section 2.2, Eq. (3)] In the definition of SSIM, the text refers to 'C1 and C1' but the formula uses C1 and C2; please correct the typo.
  2. [Section 2.3] The phrase 'in order to optimized the segmentation results' should be 'in order to optimize the segmentation results'.
  3. [Table 1] The label 'Data augmentated' should be 'Data augmented'.
  4. [Section 4] There are several typographical errors: 'compatibe' should be 'comparable', and 'DCS' should be 'DSC' in the sentence beginning 'In some cases DCS is low'.
  5. [Figure 3] The boxplots are not annotated with the significance tests that are referenced in the text; please add statistical annotations or describe the comparisons in the caption.
  6. [Figure 2 caption] The caption states that the DSC of CT and ST-CT is 0.57 and 0.80 for the shown case; please clarify that this is a single example and not the average performance, to avoid confusion with Table 1.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CT Dice is an external benchmark; the CycleGAN-to-SynCT pipeline is not definitionally tied to the measured CT segmentation outcome.

full rationale

The paper's central result is an empirically measured Dice score on real CT scans against radiologist annotations. This is an external benchmark: the 2.5D Res-U-Net is trained on SynCT (generated from MR with MR masks) and evaluated on real CT with manual CT masks. The CycleGAN/SSIM cycle-loss is an image-level reconstruction objective, and the segmentation network's CT DSC is not algebraically or statistically forced by that objective; it depends on the learned features and is measured against independent labels. No parameter is fitted to the CT test set and then renamed as a prediction: the configuration selection in Table 1 is a methodological leakage concern, not a constructional equivalence. There are also no load-bearing self-citations: CycleGAN, U-Net, and Res-U-Net citations are to external prior work, not to the authors' own uniqueness theorems or ansatzes. The abstract's claim that cycle consistency 'guarantees' SynCT shares MR masks is an unverified assumption about label preservation, but it is not used as an equation equating the prediction with the input; the CT result would stand or fall independently of that claim. Therefore no circular step can be exhibited under the required standard of quoting the paper and showing a specific reduction of the result to its inputs by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the unverified assumption that cycle-consistent synthetic CT preserves prostate boundaries so MR labels remain valid; on the representativeness of a 19-scan CT test set; and on a set of hyperparameters (clip window, augmentation, SSIM loss, context size) tuned on that same test set. No new physical or conceptual entities are introduced; SynCT is a standard generative model output.

free parameters (5)
  • Context number for 2.5D input = 1 (3-slice input)
    Selected by comparing SynCT and CT Dice scores across 1, 3, 5, and 7 slices on the same test set (Section 4, Figure 3). Tuning on the test data adds optimism to the reported 0.73.
  • CT intensity clip window = -500 to 500 HU
    Introduced as 'soft-tissue SynCT' and shown to raise CT DSC from 0.45 to 0.62 (Table 1). The window was chosen on the test set.
  • Data augmentation schedule = Random crop ratio 1 to 0.5, rotation, flipping
    Added after observing low CT DSC (0.62 to 0.68 in Table 1). The schedule was selected to maximize test-set DSC.
  • Cycle loss function = SSIM instead of MSE
    Replacing the MSE cycle loss with SSIM raised CT DSC from 0.68 to 0.73 (Table 1). This choice was made based on the same 19 CT test cases.
  • Adam learning rate = 0.0002
    Used for both CycleGAN and 2.5D Res-U-Net. Standard for these architectures, though it remains a tuning choice.
assumptions (4)
  • domain assumption Unpaired MRI and CT slices from different scanners and patients can be mapped by CycleGAN while preserving anatomical structure.
    The entire synthetic CT pipeline depends on this property. Section 2.2 introduces the CycleGAN model with no verification that anatomy is preserved beyond the image-level cycle loss.
  • domain assumption Cycle-consistency loss transfers MR segmentation labels to SynCT.
    Abstract states cycle consistency 'guarantees' SynCT shares identical masks. This is assumed, not shown. The segmentation network is trained on SynCT with MR labels (Section 2.3).
  • domain assumption The 19 manually segmented CTs are representative of the 120-patient cohort.
    Selection criteria are not described in Section 2.1, so the evaluation may not generalize to the broader population.
  • domain assumption Dice scores against a single radiologist's contours are an adequate gold standard.
    No inter-observer variability or consensus reading is reported in Section 3, and the paper notes some low Dice scores stem from noise in hand-drawn contours.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-modality Knowledge Transfer for Prostate Segmentation from CT Scans." pith.science (2026). https://pith.science/paper/TZX2XUVD

@misc{pith2026190810208,
  author       = {Pith},
  title        = {Pith review of: Cross-modality Knowledge Transfer for Prostate Segmentation from CT Scans},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TZX2XUVD}},
  note         = {Machine review of arXiv:1908.10208}
}
read the original abstract

Creating large scale high-quality annotations is a known challenge in medical imaging. In this work, based on the CycleGAN algorithm, we propose leveraging annotations from one modality to be useful in other modalities. More specifically, the proposed algorithm creates highly realistic synthetic CT images (SynCT) from prostate MR images using unpaired data sets. By using SynCT images (without segmentation labels) and MR images (with segmentation labels available), we have trained a deep segmentation network for precise delineation of prostate from real CT scans. For the generator in our CycleGAN, the cycle consistency term is used to guarantee that SynCT shares the identical manually-drawn, high-quality masks originally delineated on MR images. Further, we introduce a cost function based on structural similarity index (SSIM) to improve the anatomical similarity between real and synthetic images. For segmentation followed by the SynCT generation from CycleGAN, automatic delineation is achieved through a 2.5D Residual U-Net. Quantitative evaluation demonstrates comparable segmentation results between our SynCT and radiologist drawn masks for real CT images, solving an important problem in medical image segmentation field when ground truth annotations are not available for the modality of interest.

Figures

Figures reproduced from arXiv: 1908.10208 by the authors.

Figure 1
Figure 1. Workflow of CT image synthesis and automatic segmentation. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Example slices of segmentation results on true CT. (A) Under-segmented prostate by expert radiologists. 2.5D Res-U-Net can generate better segmentation (C) since it adapted the segmentation from MRI, however, resulting a misleadingly lower DSC, 0.74. CT with normal intensity can vary from -1000 HU (air) to 1000 (bone), therefore soft tissues consists of similar HU numbers may not be seen clearly on the images, as de… view at source ↗
Figure 3
Figure 3. Boxplots are showing the Dice scores for prostate segmentation from MRI, SynCT, and CT, respectively. 2. Smith, W.L. et al.: Prostate volume contouring: A 3D analysis of segmenta￾tion using 3DTRUS, CT, and MR. International Journal of Radiation Oncol￾ogy*Biology*Physics 67(4), 12381247 (2007) 3. Rasch, C. et al.: Definition of the prostate in CT and MRI: a multi-observer study. International Journal of Radiation Onc… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages

  1. [1]

    Prostate Cancer and Prostatic Diseases 21(1), 57-63 (2017) Automatic Prostate CT Segmentation using Synthetic CT 7 Fig

    Nordstrm, T., et al.: Prostate-specific antigen (PSA) density in the diagnostic algorithm of prostate cancer. Prostate Cancer and Prostatic Diseases 21(1), 57-63 (2017) Automatic Prostate CT Segmentation using Synthetic CT 7 Fig. 2. Example slices of segmentation results on true CT. (A) Under-segmented prostate by expert radiologists. 2.5D Res-U-Net can ge...

  2. [2]

    et al.: Prostate volume contouring: A 3D analysis of segmenta- tion using 3DTRUS, CT, and MR

    Smith, W.L. et al.: Prostate volume contouring: A 3D analysis of segmenta- tion using 3DTRUS, CT, and MR. International Journal of Radiation Oncol- ogy*Biology*Physics 67(4), 12381247 (2007)

  3. [3]

    et al.: Definition of the prostate in CT and MRI: a multi-observer study

    Rasch, C. et al.: Definition of the prostate in CT and MRI: a multi-observer study. International Journal of Radiation Oncology*Biology*Physics 43(1) 5766 (1999)

  4. [4]

    et al.: Concurrent segmentation of the prostate on MRI and CT via linked statistical shape models for radiotherapy planning

    Chowdhury, N. et al.: Concurrent segmentation of the prostate on MRI and CT via linked statistical shape models for radiotherapy planning. Medical Physics 39(4) 22142228 (2012)

  5. [5]

    In: International Conference on Com- puter Vision (2017)

    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: International Conference on Com- puter Vision (2017)

  6. [6]

    In: Workshop on Simu- lation and Synthesis in Medical Imaging (2017)

    Wolterink, J.M., Dinkla, A.M., Savenije, M.H., Seevinck, P.R., van den Berg, C.A., Isgum, I.: Deep MR to CT synthesis using unpaired data. In: Workshop on Simu- lation and Synthesis in Medical Imaging (2017)

  7. [7]

    In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomed- ical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234241. Springer, Cham (2015)

  8. [8]

    In: Carneiro, G., et al

    Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.: The importance of skip connections in biomedical image segmentation. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 179187. Springer, Cham (2016)

Show all 13 references
  1. [9]

    doi:10.7937/K9TCIA.2017.MURS5CL

    Litjens, G., Debats, O., Barentsz, J., Karssemeijer, N., Huisman, H.: SPIE-AAPM PROSTATEx Challenge Data. doi:10.7937/K9TCIA.2017.MURS5CL

  2. [10]

    et al.: NCI-ISBI 2013 Challenge: Automated Segmen- tation of Prostate Structures

    Bloch N. et al.: NCI-ISBI 2013 Challenge: Automated Segmen- tation of Prostate Structures. The Cancer Imaging Archive . http://doi.org/10.7937/K9/TCIA.2015.zF0vlOPv (2015)

  3. [11]

    et al.: Loss Functions for Image Restoration With Neural Networks

    Zhao, H. et al.: Loss Functions for Image Restoration With Neural Networks. IEEE Transactions on Computational Imaging. 3(1) 4757 (2017)

  4. [12]

    et al.: Automatic Segmentation of the Prostate on CT Images Us- ing Deep Neural Networks (DNN)

    Liu, C. et al.: Automatic Segmentation of the Prostate on CT Images Us- ing Deep Neural Networks (DNN). International Journal of Radiation Oncol- ogy*Biology*Physics. 104(4) 924932 (2019) Automatic Prostate CT Segmentation using Synthetic CT 9

  5. [13]

    Physics in Medicine and Biology

    Burgos, N., et al.: Iterative framework for the joint segmentation and CT synthesis of MR images: application to MRI-only radiotherapy treatment planning. Physics in Medicine and Biology. 62 42374253 (2017)

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.