Pith. sign in

REVIEW 3 major objections 5 minor 14 references

Enhancing Privacy: The Utility of Stand-Alone Synthetic CT and MRI for Tumor and Bone Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Synthetic CT and MRI can replace real data for some segmentation tasks — brain-glioma MRI and high-contrast bone — but not head-and-neck CT tumors, where the best synthetic-trained model scores Dice 0.064 versus 0.553.

desk verdict A careful empirical study of synthetic data for segmentation, but the MRI success claim is not yet separated from a possible label-conditioning border artifact. read the letter →

arxiv 2506.12106 v1 pith:U5GPYTKT submitted 2025-06-13 eess.IV cs.CV

André Ferreira , Kunpeng Xie , Caroline Wilpert , Gustavo Correia , Felix Barajas Ordonez , Tiago Gil Oliveira , Maike Bode , Robert Siepmann
show 7 more authors
Frank Hölzle Rainer Röhrig Jens Kleesiek Daniel Truhn Jan Egger Victor Alves Behrus Puladi
This is my paper
classification eess.IVcs.CV
keywords syntheticmedicalimagingCTMRIGANsDDPMsSegmentationVisualTuringTestRadiomics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether synthetic CT and MRI scans — created by conditional GANs and conditional diffusion models from real segmentation labels — can substitute for real patient data when training tumor and bone segmentation networks. The question matters because head-and-neck images cannot be anonymized by defacing without destroying the very region a clinician needs to examine. The answer the authors establish is that substitution works only up to the complexity of the target structure: a model trained on synthetic MRI alone scores a mean Dice of 0.834 on real brain-glioma scans against 0.890 for a real-trained model, while the best synthetic-CT-trained model scores 0.064 against 0.553 on head-and-neck tumor segmentation, and bone segmentation on synthetic CT reaches about 0.92 for the simpler annotation schemes. The paper also argues for a link between the concordance of radiomic features (CCC) between real and synthetic images and downstream tumor-segmentation performance, and identifies the mechanism behind the CT failure: synthetic tumors have unrealistically sharp borders, which the segmentation network learns, and then erases real tumors in its early layers.

What carries the argument

The machinery is the coupling of a conditional generative model with a segmentation probe. The generators are conditional GANs and conditional denoising diffusion probabilistic models (cDDPMs) — including wavelet-based 3D WDM variants and a RePaint-style diffusion inpainting model — conditioned on region of interest, contrast or intensity range, and the segmentation mask; the probe is a full-resolution nnU-Net whose Dice score on real test scans measures utility. Two analytic devices carry the argument: the radiomics Concordance Correlation Coefficient (CCC), computed over 1,065 features per region, which the paper shows correlates with downstream tumor-segmentation performance while MAE and MS-SSIM do not; and a layer-wise inspection of the nnU-Net's early convolutional feature maps, which shows that synthetic CT tumors have unnaturally well-defined borders, that networks trained on them amplify those borders, and that the same networks progressively delete real tumors in the first encoder layers when presented with genuine scans.

What would settle it

Train the same nnU-Net on synthetic CT tumors whose borders have been blurred, eroded, or set to the boundary gradient statistics of real head-and-neck tumors, and measure Dice on the real test set; if the score rises substantially from about 0.06 toward the real-trained 0.553, the well-defined-border shortcut is the operative mechanism. A directly observable check is to compare the distribution of gradient magnitudes at tumor boundaries between real and synthetic cases, which the paper's account predicts to be systematically sharper for synthetic tumors.

Watch

Extended reading notes

Core claim

The paper's central claim is that stand-alone synthetic data is a viable substitute for real data in segmentation tasks exactly to the extent that the generator reproduces the texture and border statistics of the target structure. Using 1,258 head-and-neck CT scans and 1,251 brain-glioma MRI scans, conditional GANs and conditional diffusion models (including wavelet-based 3D WDM variants and a RePaint-style inpainting model) generated synthetic scans conditioned on the region of interest, the intensity range, and the ground-truth segmentation masks. Realism was scored with MAE, MS-SSIM, 1,065 radiomic features per region (summarized by PCA and concordance correlation, CCC), and a Visual Turing Test by five radiologists; usefulness was scored by training full-resolution nnU-Net segmenters on synthetic-only data and testing them on held-out real scans. The results show that radiomics CCC tracks tumor-segmentation Dice — 0.8784 and 0.834 for MRI, 0.5461 and 0.064 for CT — while bone segmentation reaches Dice 0.92 on the simpler ground-truth label sets despite poor CCC, because bone has high intrinsic contrast. The paper concludes that diffusion-generated synthetic data can be created locally and shared with assured anonymity, but only for downstream tasks whose target structures have well-defined, high-contrast boundaries.

Load-bearing premise

The conclusions assume that testing a synthetic-trained model on real scans is a fair measure of whether synthetic data can replace real data, even though the synthetic tumors were generated from the very same segmentation maps used to train the downstream networks, which gives them unnaturally clean edges the network can latch onto.

Editorial extensions

If this is right

  • Brain-glioma MRI segmentation can be trained on synthetic-only scans and reach a mean Dice of 0.834 on real test data, within about 0.06 of a real-trained model, so fully synthetic sharing is feasible for this task.
  • Head-and-neck CT tumor segmentation cannot yet rely on synthetic-only training: the best synthetic Dice of 0.064 is far below the real-trained 0.553.
  • Diffusion-based generators outperform GANs across the tested tasks, so future synthetic-data pipelines should favor them.
  • Radiomics CCC between real and synthetic data is a more trustworthy predictor of downstream segmentation utility than MAE or MS-SSIM, and can screen synthetic datasets before expensive segmentation experiments.
  • For high-contrast targets such as bone, synthetic CT can substitute for real data even when radiomics correlation is poor, because segmentation then hinges on intensity boundaries the generator reproduces well.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A concrete fix the paper does not test: perturbing synthetic tumor borders — blurring, eroding, or matching their gradient statistics to real head-and-neck tumors — could close part of the 0.064-versus-0.553 gap, since the network would lose the unrealistically clean edge cue it currently relies on.
  • The CCC–Dice link suggests radiomics could serve as a cheap first-pass filter for synthetic data, reserving costly radiologist Visual Turing Tests for datasets that pass the feature-correlation threshold.
  • The label-conditioning shortcut may generalize: any conditional generative pipeline whose conditioning mask doubles as the downstream training label risks teaching the network a clean-boundary signal, so success claims should state whether generation conditions and downstream labels came from the same source.
  • The VTT finding that experts separate synthetic from real CT far better from whole volumes than from single slices implies that slice-level realism metrics overestimate perceived realism, and that volume-consistent evaluation is the more meaningful bar.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates whether synthetic CT and MRI, generated by conditional GANs and conditional diffusion models, can stand alone as training data for tumor and bone segmentation. Using head-and-neck CT (TCIA) and BraTS T1c MRI datasets, the authors generate synthetic volumes conditioned on segmentation labels, evaluate realism with MAE, MS-SSIM, radiomics CCC, and a radiologist Visual Turing Test, and evaluate utility by training nnU-Net models on synthetic data and testing on real data. The main quantitative findings are that synthetic MRI supports tumor segmentation with mean DSC 0.834 versus 0.890 for real-trained models, while synthetic CT supports only DSC 0.064 for tumor segmentation versus 0.553 for real data; bone segmentation on synthetic CT reaches DSC up to about 0.92 for simple structures. The authors attribute the CT tumor failure to unnaturally well-defined mask-aligned tumor borders learned in early encoder layers. The paper concludes that synthetic data can be used independently for segmentation, limited by task complexity.

Significance. If the central claim holds, the paper would be a useful systematic evaluation: it combines multiple realism metrics, a five-radiologist VTT, a downstream segmentation benchmark, and an open-source implementation. It also provides a plausible mechanistic explanation, supported by layer-wise feature maps, for why synthetic CT tumors fail to transfer to real data. The honest reporting of the CT tumor failure is a strength. However, the headline claim that synthetic MRI 'works independently' is currently under-supported because the same label-conditioning mechanism identified for the CT failure is not tested for MRI, and several load-bearing numerical comparisons lack uncertainty quantification. With additional targeted analyses, the study could make a solid contribution to the synthetic medical imaging literature.

major comments (3)
  1. [§3.2, Fig. 7; §2.2.2, Eq. (4)] The mechanism used to explain the CT tumor failure—synthetic images carry unnaturally mask-aligned borders that segmentation models learn in early encoder layers—is never applied to the MRI experiments. All MRI generative models (cGAN_MRI_seg and WDM_MRI_seg variants) are conditioned on the exact tumor masks used as downstream training labels, and the loss in Eq. (4) includes an explicit tumor-region MSE term that can reward sharp, mask-aligned transitions. If synthetic MRIs carry the same border cue, the reported DSC of 0.834 may reflect a match between the generator's border stereotype and the BraTS test set rather than a generalizable image-label relationship. The authors should apply the layer-wise diagnostic of Fig. 7, or an equivalent border-alignment analysis, to synthetic MRI before claiming that MRI synthetic data work independently.
  2. [§2.3, §3.1] The radiomics CCC used to support MRI fidelity is computed within the same ROIs that serve as conditioning labels for the generative models and as training labels for the downstream segmentation networks. Because the generator is forced to place tumors exactly at these masks, the high CCC between real and synthetic tumors may be inflated by construction and is not independent evidence of fidelity. The authors should report radiomics agreement on regions outside the conditioning masks, or at least discuss how the shared-label setup affects the CCC interpretation.
  3. [Tables A.3, A.5; §3.2] CT tumor DSC results are reported as point estimates without confidence intervals, error bars, or statistical tests, while the MRI table includes standard deviations. The difference between the best CT model (DSC=0.0640) and the runner-up (DSC=0.0559), and the many near-zero values, are within the range of plausible run-to-run variation. Consequently, the model rankings and the claimed 'association between CCC and DSC' in §3.2 are not yet statistically supported. The authors should provide repeated experiments or bootstrapped intervals, and a formal test (or at least a correlation with uncertainty) for the CCC-DSC relationship.
minor comments (5)
  1. [§3.3] The statistical reporting is internally inconsistent: the text states that the difference between experienced and less-experienced raters was highly significant (p<0.001) and then says that applying a more stringent threshold of α=0.04 left no significant difference; if p<0.001, the result would remain significant at α=0.04.
  2. [Ethics approval] The ethics statement says the research did not involve human participants, yet the VTT was performed by five radiologists; this should be clarified or reconciled.
  3. [Equation (1)] Equation (1) contains a typographical mismatch in parentheses: '∥x− G(z|c))∥' has an extra closing parenthesis.
  4. [Code availability] The GitHub repository URL contains a space ('generative networks') and should be made a valid URL.
  5. [§2.1.1] The numbers in the data-preparation section are stated as '1.355 cases' and '1.258 cases' with decimal points; these should be thousands separators (1,355 and 1,258).

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; the label-conditioned generation setup is transparent and held-out real test data break any by-construction reduction.

full rationale

The claim that synthetic data can substitute for real training data is tested by training nnU-Net on synthetic scans with labels and measuring DSC on held-out real scans; this is an external utility measurement, not a quantity fitted to the test set. The label-conditioning shortcut is disclosed (Sec. 2.4.1: 'these labels are used as condition of the generative models'; Appendix A: 'These experiments use the label as condition for the generation of the synthetic tumor'), and the paper's own analysis (Sec. 3.2, Fig. 7) explains the CT tumor failure as the segmenter learning mask-aligned border artifacts. That is an acknowledged confound for CT, not a reduction by construction; the MRI DSC of 0.834 is not forced by Eq. 4 or by radiomics CCC. Radiomics CCC is an independent fidelity measure computed on both real and synthetic images within the same ROIs; a high CCC does not logically produce a high DSC. Bone labels come from external automated tools (TotalSegmentator, TotalSpineSeg, AMASSS-CBCT). The self-citations (e.g., Ferreira et al. 2024 for GAN training details and a prior BraTS augmentation approach) are methodological and not load-bearing. The main weakness is that the MRI border-artifact diagnostic of Sec. 3.2 was not repeated for MRI, leaving a missing control rather than a circular step. Overall: no significant circularity, though the stand-alone MRI claim would be strengthened by ruling out the same border cue.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The main free parameters are preprocessing choices and arbitrary thresholds. The evaluation also assumes that public datasets and automatic labelers are trustworthy. No new theoretical entities are introduced, but the synthetic generation pipeline with its conditioning on segmentation labels creates a potential circular evaluation.

free parameters (3)
  • MAE threshold for synthetic realism = <0.4 HU
    Used in Section 3.1 to claim synthetic CT is realistic, but the threshold is arbitrary and not tied to downstream performance; the paper itself shows this threshold fails to predict utility.
  • VTT rating threshold = 5
    Raters classify as 'real' if rating <=5, 'synthetic' if >5. The threshold is arbitrary and has no justification.
  • HU clipping ranges = [-200,200] for tumor, [-1000,1000] for bone
    Chosen in Section 2.1.1 for tumor/bone tasks; the ranges are task-specific and hand-chosen, and they affect the synthetic data generation and segmentation.
assumptions (4)
  • domain assumption Public datasets (TCGA-HNSC, BraTS 2023) are representative of clinical distribution for head and neck tumors and brain gliomas.
    The evaluation assumes that these datasets are realistic and that the generative models trained on them will produce useful synthetic data for the intended clinical tasks.
  • domain assumption Radiologists' VTT ratings are a valid proxy for clinical realism.
    The paper uses VTT as evidence for limited educational utility (Section 3.3), assuming that expert visual assessment reflects what students or clinicians would need.
  • domain assumption The intensity ranges and normalization choices preserve relevant diagnostic information.
    The paper clips CT to [-200,200] HU for tumor and [-1000,1000] for bone, which discards information outside these ranges (e.g., bone density above 1000 HU), potentially affecting bone segmentation quality.
  • domain assumption The automatic tools (TotalSegmentator, TotalSpineSeg, AMASSS-CBCT) produce ground-truth labels that are sufficiently accurate for training and evaluation.
    These tools are used both to create synthetic data conditions and to evaluate the downstream segmentation models, as described in Section 2.4.1.
invented entities (1)
  • Synthetic tumor inpainting model with edge/full blur masks
    purpose: Generates synthetic tumors inside CT scans while avoiding unrealistic intensity boundaries.
    The mask blurring strategies are described in Section 2.2.2 to smooth the transition between tumor and surrounding tissue; the paper provides no external validation that this improves clinical realism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Privacy: The Utility of Stand-Alone Synthetic CT and MRI for Tumor and Bone Segmentation." pith.science (2026). https://pith.science/paper/U5GPYTKT

@misc{pith2026250612106,
  author       = {Pith},
  title        = {Pith review of: Enhancing Privacy: The Utility of Stand-Alone Synthetic CT and MRI for Tumor and Bone Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U5GPYTKT}},
  note         = {Machine review of arXiv:2506.12106}
}
read the original abstract

AI requires extensive datasets, while medical data is subject to high data protection. Anonymization is essential, but poses a challenge for some regions, such as the head, as identifying structures overlap with regions of clinical interest. Synthetic data offers a potential solution, but studies often lack rigorous evaluation of realism and utility. Therefore, we investigate to what extent synthetic data can replace real data in segmentation tasks. We employed head and neck cancer CT scans and brain glioma MRI scans from two large datasets. Synthetic data were generated using generative adversarial networks and diffusion models. We evaluated the quality of the synthetic data using MAE, MS-SSIM, Radiomics and a Visual Turing Test (VTT) performed by 5 radiologists and their usefulness in segmentation tasks using DSC. Radiomics indicates high fidelity of synthetic MRIs, but fall short in producing highly realistic CT tissue, with correlation coefficient of 0.8784 and 0.5461 for MRI and CT tumors, respectively. DSC results indicate limited utility of synthetic data: tumor segmentation achieved DSC=0.064 on CT and 0.834 on MRI, while bone segmentation a mean DSC=0.841. Relation between DSC and correlation is observed, but is limited by the complexity of the task. VTT results show synthetic CTs' utility, but with limited educational applications. Synthetic data can be used independently for the segmentation task, although limited by the complexity of the structures to segment. Advancing generative models to better tolerate heterogeneous inputs and learn subtle details is essential for enhancing their realism and expanding their application potential.

Figures

Figures reproduced from arXiv: 2506.12106 by the authors.

Figure 1
Figure 1. Study Workflow: The process began with the collection and organization of MRI and CT scans, which were used to train generative models, specifically [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visual comparison between real and synthetic samples for both tumor [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. PCA plots of Radiomic features. Each plot compares each pair of real [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Concordance Correlation Coefficient (CCC) analysis of Radiomic features extracted from CT and MRI scans across different regions. The heatmap visualizes the correlation results, with stronger correlations indicated in red and weaker correlations in black. Only the data…
Figure 6
Figure 6. Figure 6: Comparison of Dice scores for bone structure segmentation on the [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Comparison of Dice scores for tumor segmentation on the test set. A) [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 8
Figure 8. Figure 8: Visual Turing Test performance by each rater (R), ordered by total [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 7
Figure 7. Figure 7: Mean feature maps extracted from the first to fourth convolutional [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 9 canonical work pages

  1. [1]

    S. Y . Wang, S. Pershing, A. Y . Lee, A. T. on AI, A. M. I. T. Committee, et al., Big data requirements for artificial intelligence, Current opinion in ophthal- mology 31 (2020) 318–323. M. I. Ahmed, B. Spooner, J. Isherwood, M. Lane, E. Orrock, A. Dennison, A systematic review of the barriers to the implementation of artificial intelli- gence in healthca...

    work page 2020

  2. [24]

    Blanco-Justicia, J

    A. Blanco-Justicia, J. Domingo-Ferrer, S. Mart ´ınez, D. S ´anchez, A. Flanagan, K. E. Tan, Achieving security and privacy in federated learning systems: Survey, research challenges and future directions, Engineering Applications of Artificial Intelligence 106 (2021) 104468. J. M. Mendes, A. Barbar, M. Refaie, Synthetic data generation: a privacy- preserv...

    arXiv 2021

  3. [61]

    R. A. Kadam, Informed consent process: a step further towards making it meaningful!, Perspectives in clinical research 8 (2017) 107–112. E. G. Howe III, F. Elenberg, Ethical challenges posed by big data, Innovations in clinical neuroscience 17 (2020)

    work page 2017

  4. [182]

    J. Chen, C. Zhang, A. Traverso, I. Zhovannik, A. Dekker, L. Wee, I. Bermejo, Generative models improve radiomics reproducibility in low dose cts: a sim- ulation study, Physics in Medicine & Biology 66 (2021) 165002. A. Rozenshtein, L. K. Findeiss, M. J. Wood, G. Shih, J. R. Parikh, The us radiologist workforce: Ajr expert panel narrative review, American ...

    work page 2021

  5. [1283]

    The Table A.6 presents the MAE and MS-SSIM metrics for the CT datasets used for the bone segmentation task

    For the training of the nnUNet, each scan was considered a full patch. The Table A.6 presents the MAE and MS-SSIM metrics for the CT datasets used for the bone segmentation task. Tables A.7 and A.8 show the DSC for the segmentation of bone structures on the CT 195 real. In Appendix G (Figures G.17, G.18, G.19 and G.20) and Appendix H (Figures H.21, H.22 a...

    work page 1901

  6. [1364]

    Vallieres, E

    M. Vallieres, E. Kay-Rivest, L. J. Perrin, X. Liem, C. Furstoss, H. J. Aerts, N. Khaouam, P. F. Nguyen-Tan, C.-S. Wang, K. Sultanem, et al., Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer, Scientific reports 7 (2017) 10117. M. L. Zuley, R. Jarosz, S. Kirk, Y . Lee, R. Colen, K. Garcia, D. Delbeke, M. Pham, P. Nagy, G. S...

    work page 2017

  7. [2016]

    LXKQ47MS, data set

    doi: 10.7937/K9/TCIA.2016. LXKQ47MS, data set. National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CP- TAC), The clinical proteomic tumor analysis consortium head and neck squamous cell carcinoma collection (cptac-hnscc) (version 15), https:// doi.org/10.7937/K9/TCIA.2018.UW45NH81,

    doi:10.7937/k9/tcia.2016 2016

  8. [2018]

    doi: 10.7937/K9/ TCIA.2018.UW45NH81, data set. H. J. Aerts, E. R. Velazquez, R. T. Leijenaar, C. Parmar, P. Grossmann, S. Car- valho, J. Bussink, R. Monshouwer, B. Haibe-Kains, D. Rietveld, et al., De- coding tumour phenotype by noninvasive imaging using a quantitative ra- diomics approach, Nature communications 5 (2014)

    doi:10.7937/k9/ 2014

Show all 14 references
  1. [2019]

    8kap372n, data set

    doi: 10.7937/tcia.2019. 8kap372n, data set. K. Clark, B. Vendt, K. Smith, J. Freymann, J. Kirby, P. Koppel, S. Moore, S. Phillips, D. Maffitt, M. Pringle, et al., The cancer imaging archive (tcia): maintaining and operating a public information repository, Journal of digital i...

    2013 arXiv

  2. [2024]

    Gillot, B

    M. Gillot, B. Baquero, C. Le, R. Deleat-Besson, J. Bianchi, A. Ruellas, M. Gurgel, M. Yatabe, N. Al Turkestani, K. Najarian, et al., Automatic multi-anatomical skull structure segmentation of cone-beam computed to- mography scans using 3d unetr, PLoS One 17 (2022) e0275033. F....

    2022

  3. [2356]

    Khader, G

    F. Khader, G. M ¨uller-Franzes, S. Tayebi Arasteh, T. Han, C. Haarburger, M. Schulze-Hagen, P. Schad, S. Engelhardt, B. Baeßler, S. Foersch, et al., Denoising diffusion probabilistic models for 3d medical image generation, Scientific Reports 13 (2023)

    2023

  4. [3873]

    C. H. Blow, L. Qian, C. Gibson, P. Obiomon, X. Dong, Data augmentation via diffusion model to enhance ai fairness, Frontiers in Artificial Intelligence 8 (2025) 1530397. A. Ferreira, N. Solak, J. Li, P. Dammann, J. Kleesiek, V . Alves, J. Egger, How we won brats 2023 adult gli...

    2025 arXiv

  5. [4006]

    L. Wee, A. Dekker, Data from head-neck-radiomics-hn1, https://doi. org/10.7937/tcia.2019.8kap372n,

    2019 doi

  6. [7303]

    L. R. Koetzier, J. Wu, D. Mastrodicasa, A. Lutz, M. Chung, W. A. Koszek, J. Pratap, A. S. Chaudhari, P. Rajpurkar, M. P. Lungren, et al., Generating synthetic data for medical imaging, Radiology 312 (2024) e232471. T. Kokosi, K. Harron, Synthetic data in medical research, BMJ ...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.