REVIEW 4 major objections 6 minor 14 references
Style-invariant Cardiac Image Segmentation with Test-time Augmentation
T0 review · 4 major / 6 minor · reviewed 2026-08-27 · deepseek-v4-flash
Pith's one-line read Zero-shot style transfer removes MRI vendor appearance shifts so a source-trained U-Net segments unseen vendors with roughly 5 percent higher Dice.
desk verdict A useful but under-controlled test-time style transfer + TTA pipeline for cardiac MRI; the missing SegO+TTA control means the ~5-point gain is not yet attributable to the style transfer. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the improved WaveCT zero-shot style transfer network, a feed-forward network that restyles any content image to any style image in one pass without training on predefined style classes. It replaces max pooling with Haar wavelet pooling and unpooling, so low-frequency content passes through the main encoder-decoder while high-frequency detail bypasses via skip connections, preserving anatomical structure; a multi-scale feature fusion layer after the encoder and adaptive instance normalization layers render the target appearance. A small style library built from the training data supplies reference slices, chosen for each test slice by matching mean and standard deviation. The second mechanism is test-time augmentation: the stylized image is flipped and rotated, each version is segmented by the same U-Net, the predictions are inverse-transformed, and a majority vote yields the final label map.
What would settle it
Apply the same STSegO-TTA pipeline to a vendor whose images differ not only in intensity and texture but in anatomy, pathology, or cardiac phase coverage; if the Dice gain over SegO drops below the reported roughly 5 points or turns negative, the claim that appearance shift is the dominant obstacle is falsified. A more direct check is to have radiologists compare original and stylized slices for the same case: visible distortion of the myocardium or cavity boundaries would invalidate the structure-preservation assumption.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that appearance shift between MRI vendors can be treated as a style difference and removed by a zero-shot style transfer network, after which a segmentation model trained on source data generalizes to unseen vendors. The improved WaveCT network—built on VGG-16 with Haar wavelet pooling to preserve structure and adaptive instance normalization to render style—transfers the appearance of a reference slice from the source domain onto the test slice. The stylized image is then segmented by a U-Net, and test-time augmentation (flips and rotations followed by inverse transformation and majority voting) merges four predictions into one more stable output. In the main experiment this configuration, STSegO-TTA, improves average Dice by about 5 percent and Jaccard by about 6 percent per vendor compared with the plain U-Net, with the largest absolute gains on the myocardium. The authors also train a second model on style-unified data generated by the same network; it improves less, and both experiments show weak results on vendor D, which the authors attribute to a large distribution and anatomical difference from the source data.
Load-bearing premise
The method assumes that the style transfer network can strip away vendor-specific appearance while leaving the heart's anatomical boundaries intact, and that a single reference style from the training set adequately represents each unseen vendor's appearance; if either fails, the reported gains shrink or reverse.
Editorial extensions
If this is right
- If the central claim holds, any source-trained cardiac segmentation model can be applied to a new scanner without retraining, fine-tuning, or target-domain labels; the pipeline only needs a style reference from the source data.
- Test-time augmentation contributes measurable gains on top of style transfer, suggesting that part of the cross-vendor error is not appearance but geometric or anatomical variability that flips and rotations smooth out.
- Training on a style-unified dataset produced by the same network yields better baseline performance than training on original data, indicating the style transfer can also serve as a preprocessing step for building the training set.
- The poor vendor D results imply the method does not fully solve large anatomical or distributional shifts; the paper itself flags style-reference selection as critical and as future work.
- The framework is fully test-time, so it can be attached to an already deployed segmentation model without retraining the segmentation weights.
Reading between the lines
- The recipe is not organ-specific: zero-shot style transfer plus test-time augmentation could be lifted to any segmentation task where domain shift is mostly intensity and texture, since neither component uses cardiac priors.
- The per-slice style selection by mean and standard deviation is a weak link; a testable extension is to cluster test slices and choose one style reference per cluster, or synthesize a reference style, which might recover the vendor D failures.
- If the appearance-shift model is correct, the ST network should improve other source-trained segmentation architectures unchanged, because it sits outside the segmentation network; swapping the U-Net for another model without retraining the ST network would test this directly.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a two-stage test-time method for cross-vendor cardiac MRI segmentation: first, a zero-shot style transfer network based on WaveCT with AdaIN normalizes the appearance of a test image to a reference style; second, test-time augmentation applies flips and rotations to the stylized image and merges predictions by majority vote. The segmentation model is a U-Net trained on source vendor data. Two experiments are reported: Exp.1 trains on the original data (SegO) and Exp.2 trains on a style-unified dataset (SegST), each with and without test-time augmentation, evaluated on M&Ms 2020 vendors A-D. The headline claim in Section 3.2 is that the full method improves the Dice index by about 5 percent and the Jaccard index by about 6 percent on average per vendor.
Significance. If the reported gains are actually attributable to the style transfer module, the method would provide a training-free, target-data-free way to reduce cross-vendor performance drops in cardiac MRI segmentation, which is clinically valuable. The paper uses a public benchmark, reports four metrics, and includes two experiments that partially ablate the components. The zero-shot formulation is a strength: no target-domain labels are used. However, as presented, the evidence does not isolate the contribution of style transfer from test-time augmentation, and several key design choices are underspecified. The significance is therefore conditional: the idea is promising, but the current experiments do not yet establish that the style transfer network, rather than TTA or unspecified style-library selection, is responsible for the improvements.
major comments (4)
- [§3.2, Table 1] The headline comparison is SegO vs. STSegO-TTA, but there is no SegO-TTA control row. This bundles two interventions: style transfer (ST) and test-time augmentation (TTA). From Table 1, STSegO alone adds 2.20, 2.90, 4.07, and 2.64 DiceAVG points over SegO on VenA-D, while TTA on top of STSegO adds 1.78, -0.15, 0.29, and 3.14 points. On VenD, TTA contributes more than half of the total improvement, and VenD is the vendor the paper singles out as difficult. Without a SegO-TTA condition, the reader cannot determine whether the reported ~5-point Dice gain is a TTA artifact rather than an effect of the style transfer network, which is the paper's main claimed novelty. Please add the SegO-TTA control or re-frame the claims accordingly.
- [§3.2, style image library] The style library construction is underspecified and potentially circular: the paper states it contains '221 slices from the top-20 images via computing the Dice index' without saying which Dice index is computed, whether the segmentation model's own predictions or ground-truth labels are used, or how the top-20 selection is made. The subsequent per-slice style selection 'as close as possible to the mean and standard deviation of the testing image' also lacks a concrete distance metric. If the selection uses information from the segmentation model being evaluated, the zero-shot claim is compromised. Please specify the selection procedure exactly and report sensitivity to the library size and to the selection criterion.
- [§3.2, Table 1 (VenD)] The claim of 'consistent improvements' is not supported across all metrics and vendors. On VenD, STSegO decreases DiceRV from 61.96 to 55.66, increases ASSDAVG from 12.44 to 13.17, and increases ASSDRV from 19.34 to 31.08 relative to SegO. STSegO-TTA recovers the DiceAVG on VenD, but in Exp.2 (Table 2) SegST-TTA substantially worsens boundary metrics on VenD: HDB AVG goes from 35.56 to 47.86 and ASSD AVG from 14.55 to 17.52. The paper should present a per-vendor, per-metric analysis, explain the deterioration on VenD, and avoid over-generalizing the consistency claim.
- [Tables 1 and 2] All results are point estimates from what appears to be a single run, with no standard deviations, confidence intervals, or significance tests. Because the main gains are on the order of 2-5 Dice points and the VenD behavior is heterogeneous, the reliability of the differences is unclear. Please report means and standard deviations over multiple runs or bootstrap confidence intervals, and where appropriate paired significance tests for the key comparisons (SegO vs. STSegO vs. STSegO-TTA).
minor comments (6)
- [§3.2] The sentence 'It almost improves the Dice index by 5 percent and the Jaccard index by 6 percent on average for each vendor' does not match the table: the average DiceAVG improvement is 4.22 points and the average JacAVG improvement is 5.32 points, with per-vendor variation. Please report the exact mean values.
- [§3.2 (Exp.2)] The text refers to 'STSeg-TTA' while the tables use 'SegST-TTA'; please unify the notation. Also, the claim that 'STSeg-TTA shows improvements over STSeg' is not true for all metrics, since VenC DiceAVG decreases from 85.74 to 85.23 and VenD HDB and ASSD worsen.
- [§2.2] The ST network fine-tuning procedure is not specified: the paper mentions using 3000 cardiac slices but does not report the reconstruction loss, style loss, learning rate, or number of iterations. These details are needed for reproducibility.
- [§2 and §3] The phrase 'fully test-time adaptation' is overstated for Exp.2, in which the segmentation model is trained on a style-unified dataset generated by the ST network. Please clarify that only the ST network operates at test time in a zero-shot manner.
- [References] Reference [2] is listed as 'In preparation'; if the M&Ms 2020 challenge paper has been published by the time of revision, please update the citation. Also, a direct quantitative comparison with the prior style-transfer-based methods of Ma et al. [9] and Liu et al. [8] would help position the contribution.
- [Fig. 3] The caption says 'our better 3D segmentation results' without defining what 'better' means; please clarify whether these are qualitative examples and, if so, state the selection criterion.
Circularity Check
No circular derivation: style-transfer and test-time augmentation gains are empirical results on an external benchmark, with only a non-load-bearing self-citation.
full rationale
The paper makes no analytic derivation that could be circular. The reported gains are empirical comparisons on the M&Ms benchmark. The style-transfer network is a learned module fine-tuned on source cardiac slices; the segmentation models are trained on source labels; test-time augmentation is a fixed set of flips, rotations, and majority voting. No target-domain labels or test predictions are used as fitting inputs. The only in-house component reused from prior work is the WaveCT-AIN backbone (Section 2.2: "The ST network proposed in this paper is a significant extension of our prior conference paper proposed WaveCT-AIN [8]"), but this is a provenance statement and architectural starting point, not a load-bearing justification for the empirical outcome. The style-image library in Exp.1 is selected from training data "via computing the Dice index" (Section 3.2), which is a training-data hyperparameter choice; it does not use test labels and is therefore not a fitted prediction. The paper's own caveat that Ven D results are poor (Section 3.2) is an acknowledged limitation, not evidence of circularity. The absence of a SegO-TTA control is an experimental-design weakness, but it does not make the central claim equivalent to its inputs by construction.
Assumptions & free parameters
free parameters (3)
- lambda (loss weight) =
0.5
- TTA transformation set =
not applicable
- Style library size =
221 slices from top-20 images
assumptions (4)
- domain assumption Pre-trained VGG-16 features are a suitable basis for cardiac MR style transfer.
- domain assumption Haar wavelet pooling in WaveCT preserves anatomical structure during style transfer.
- domain assumption The M&Ms dataset vendors A-D are representative of the clinical appearance shift.
- domain assumption Majority voting over TTA predictions improves accuracy.
Cite this review
Pith. "Pith review of Style-invariant Cardiac Image Segmentation with Test-time Augmentation." pith.science (2026). https://pith.science/paper/X3MG72OD
@misc{pith2026200912193,
author = {Pith},
title = {Pith review of: Style-invariant Cardiac Image Segmentation with Test-time Augmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/X3MG72OD}},
note = {Machine review of arXiv:2009.12193}
}
read the original abstract
Deep models often suffer from severe performance drop due to the appearance shift in the real clinical setting. Most of the existing learning-based methods rely on images from multiple sites/vendors or even corresponding labels. However, collecting enough unknown data to robustly model segmentation cannot always hold since the complex appearance shift caused by imaging factors in daily application. In this paper, we propose a novel style-invariant method for cardiac image segmentation. Based on the zero-shot style transfer to remove appearance shift and test-time augmentation to explore diverse underlying anatomy, our proposed method is effective in combating the appearance shift. Our contribution is three-fold. First, inspired by the spirit of universal style transfer, we develop a zero-shot stylization for content images to generate stylized images that appearance similarity to the style images. Second, we build up a robust cardiac segmentation model based on the U-Net structure. Our framework mainly consists of two networks during testing: the ST network for removing appearance shift and the segmentation network. Third, we investigate test-time augmentation to explore transformed versions of the stylized image for prediction and the results are merged. Notably, our proposed framework is fully test-time adaptation. Experiment results demonstrate that our methods are promising and generic for generalizing deep segmentation models.
Figures
Reference graph
Works this paper leans on
-
[1]
NPJ digital medicine 1(1), 1–8 (2018)
Abr` amoff, M.D., Lavin, P.T., Birch, M., Shah, N., Folk, J.C.: Pivotal trial of an autonomous ai-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ digital medicine 1(1), 1–8 (2018)
work page 2018
-
[2]
Campello, V.M., et al.: Multi-centre, multi-vendor & multi-disease cardiac image segmentation. In preparation
-
[3]
In: Interna- tional Workshop on Statistical Atlases and Computational Models of the Heart
Chen, C., Ouyang, C., Tarroni, G., Schlemper, J., Qiu, H., Bai, W., Rueckert, D.: Unsupervised multi-modal style transfer for cardiac mr segmentation. In: Interna- tional Workshop on Statistical Atlases and Computational Models of the Heart. pp. 209–219. Springer (2019)
work page 2019
-
[4]
IEEE Transactions on Medical Imaging (2020)
Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.A.: Unsupervised bidirectional cross- modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Transactions on Medical Imaging (2020)
work page 2020
-
[5]
In: Advances in Neural Information Processing Systems
Dou, Q., de Castro, D.C., Kamnitsas, K., Glocker, B.: Domain generalization via model-agnostic learning of semantic features. In: Advances in Neural Information Processing Systems. pp. 6450–6461 (2019)
work page 2019
-
[6]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2414–2423 (2016) 10 Huang et al
work page 2016
-
[7]
In: Proceedings of the IEEE International Conference on Computer Vision
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1501–1510 (2017)
2017
-
[8]
In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)
Liu, Z., Yang, X., Gao, R., Liu, S., Dou, H., He, S., Huang, Y., Huang, Y., Luo, H., Zhang, Y., et al.: Remove appearance shift for ultrasound image segmentation via fast and universal style transfer. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). pp. 1824–1828. IEEE (2020)
work page 2020
Show all 14 references
-
[9]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Ma, C., Ji, Z., Gao, M.: Neural style transfer improves 3d cardiovascular mr image segmentation on inconsistent data. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 128–136. Springer (2019)
2019
-
[10]
Scientific reports 10(1), 1–7 (2020)
Moshkov, N., Mathe, B., Kertesz-Farkas, A., Hollandi, R., Horvath, P.: Test-time augmentation for deep learning-based cell segmentation on microscopy images. Scientific reports 10(1), 1–7 (2020)
2020
-
[11]
arXiv preprint arXiv:1712.04621 (2017)
Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 (2017)
2017 arXiv
-
[12]
In: International Conference on Medical image computing and computer-assisted intervention
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
2015
-
[13]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Yang, X., Dou, H., Li, R., Wang, X., Bian, C., Li, S., Ni, D., Heng, P.A.: General- izing deep models for ultrasound image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 497–505. Springer (2018)
2018
-
[14]
In: Proceedings of the IEEE International Conference on Com- puter Vision
Yoo, J., Uh, Y., Chun, S., Kang, B., Ha, J.W.: Photorealistic style transfer via wavelet transforms. In: Proceedings of the IEEE International Conference on Com- puter Vision. pp. 9036–9045 (2019)
2019
Reviewed August 27, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.