Pith. sign in

REVIEW 3 major objections 3 minor 12 references

PCA for Enhanced Cross-Dataset Generalizability in Breast Ultrasound Tumor Segmentation

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PCA preprocessing of breast ultrasound images cuts cross-dataset recall decline by 33% in the hardest cases.

desk verdict A cheap, plausible preprocessing idea for cross-dataset ultrasound segmentation, but the headline gains likely come from PCA fitted on whole datasets (test leakage) plus a post hoc worst-10 selection; the central claim is not established. read the letter →

arxiv 2505.23587 v1 pith:JQUNBLY6 submitted 2025-05-29 eess.IV cs.CVcs.LG

Christian Schmidt , Heinrich Martin Overhoff This is my paper
classification eess.IVcs.CVcs.LG
keywords breasttumorsegmentationdomaingeneralizationultrasoundimagingneuralnetworksprincipalcomponentanalysiscross-datasetvalidationU-Netexternalvalidity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that reconstructing breast ultrasound images with principal component analysis before training and testing keeps segmentation models from degrading as much when they move from one dataset's ultrasound machine to another's. The authors train the same U-Net architecture on six original public datasets and on six PCA-reconstructed versions, then evaluate each model on the other five datasets. They find the largest benefits exactly where baseline generalization is worst: for the ten hardest model-dataset pairs, recall rises from $0.57 \pm 0.07$ to $0.70 \pm 0.05$ ($p = 0.0004$) and Dice from $0.50 \pm 0.06$ to $0.58 \pm 0.06$ ($p = 0.03$), and the average recall decline from external validation falls from $0.12$ to $0.08$, a $33\%$ reduction. A sympathetic reader should care because this is a nearly free preprocessing step, unlike domain adaptation or GAN style transfer, and it could make ultrasound-trained models more trustworthy when deployed on unseen devices.

What carries the argument

The carrying object is PCA reconstruction, defined by $X_{\text{reconstructed}} = \bar{X} + \sum_{i=1}^{n} z_i w_i^{\top}$, where $\bar{X}$ is the dataset mean, $z_i$ is the score of the $i$-th principal component, $w_i$ is its vector, and $n$ is the number of kept components. The number of components is chosen per dataset from a scree plot using the eigenvalue-greater-than-one rule and checked against a cumulative explained variance of about 90%. This step is applied before the U-Net is trained, so the mechanism is a fixed linear denoising of each image that suppresses machine-specific artifacts and background texture while keeping the low-dimensional structure associated with tumors. It is the only difference between the baseline and treatment arms; architecture, loss (Dice plus binary cross-entropy with weight $\beta = 0.5$), optimizer, and the 70/10/20 train/validation/test split are held fixed.

What would settle it

Recompute the whole experiment with the PCA transform fit strictly on each dataset's 70% training split, then apply that fitted transform to the validation and test splits; if the worst-pair recall gain and the 33% decline reduction disappear or fall below significance, the reported safeguard is an artifact of using full-dataset statistics.

Watch

Extended reading notes

Core claim

Working across 3,983 B-mode images from six public breast ultrasound datasets, the paper's central claim is that PCA-reconstructed images are a better training and testing representation than original images for cross-dataset tumor segmentation. For each dataset the authors keep the principal components with eigenvalue greater than one, which retain about 90% of the variance, reconstruct the images from those components, and train one U-Net per dataset on the original and one on the reconstructed data. Comparing each model on the five out-of-domain datasets, they report statistically significant gains for the worst 10 of 30 pairs in both recall and Dice, and a 33% reduction in the average recall decline due to external validation. The authors interpret this as PCA removing device-specific noise while preserving the shared tumor structure, so that PCA models behave as if trained on a more uniform population of images.

Load-bearing premise

Section 3.2 describes creating each PCA-dataset before describing the 70/10/20 split and never states that the PCA transform was fitted on training images only; if it was fitted on the whole dataset, the test images would contribute their own statistics to the denoising and the reported gains would be inflated.

Editorial extensions

If this is right

  • PCA preprocessing can be dropped into an existing segmentation pipeline without architectural changes, additional training data, or target-domain access, so it is a low-cost safeguard for deployment on unseen ultrasound devices.
  • The benefit is concentrated where it matters most: the ten pairs with the largest baseline recall decline show statistically significant gains, while the remaining twenty pairs show essentially no change ($p = 0.94$ for recall, $p = 0.62$ for Dice).
  • In the worst cases PCA recovered tumors that the original model missed entirely, implying the safeguard is not just polishing outputs but restoring detections.
  • The mean external-validity recall decline drops from $0.12$ to $0.08$, meaning the method removes roughly a third of the performance loss incurred when leaving the training domain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A critical open question the paper does not settle is whether the PCA transform was fit on training images only or on the full dataset before splitting; if the latter, part of the reported gain could come from test-domain statistics leaking into the denoising, and the 33% figure would shrink under a strictly training-only fit.
  • The asymmetric improvement in recall over Dice suggests PCA's smoothing may trade boundary precision for detectability; measuring boundary distance or AUROC would show whether the net clinical benefit survives.
  • Datasets that need more principal components to reach 90% variance might encode more complex or informative structure; ablating the number of components per dataset could turn the current fixed criterion into a tunable knob for external validity.
  • An extension the paper only gestures toward is combining PCA with augmentation or style transfer, since PCA acts on within-dataset noise while those methods act on appearance shifts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes PCA-based image reconstruction as a preprocessing step to improve cross-dataset generalizability of U-Net breast ultrasound tumor segmentation. The authors construct PCA-reduced versions of six public datasets, train separate U-Nets on original and PCA-reconstructed images, and evaluate each model on the remaining out-of-domain datasets. They report that PCA improves recall and Dice, with statistically significant gains for the ten worst model-dataset pairs (recall 0.57±0.07 vs. 0.70±0.05, p=0.0004; Dice 0.50±0.06 vs. 0.58±0.06, p=0.03), and claim a 33% reduction in recall decline during external validation.

Significance. If the reported effect is real, PCA reconstruction would be a strikingly simple and computationally cheap domain-generalization safeguard for medical image segmentation, requiring no target-domain access or adversarial training. The study draws on six public breast ultrasound datasets with 3,983 images, which is a useful scale for this problem, and the experimental design is transparent in many respects: a fixed U-Net architecture, consistent training protocol, and a fixed random-split seed. However, the significance is currently undermined by two load-bearing problems: the PCA fitting protocol is not specified relative to the train/test split, creating a potential test-set leakage, and the headline significant results are computed on a post hoc selected subset while the all-pairs comparisons are not significant. The 33% decline-reduction claim is also not substantiated by the reported tables. These issues must be resolved before the paper's conclusion can be accepted.

major comments (3)
  1. [Section 3.2] The PCA fitting protocol is not described with respect to the train/validation/test split. The text says 'PCA was employed to decompose each dataset into its principal components, followed by a reconstruction of the original dataset' and then states that the same 70/10/20 split was used, but it never states that PCA was fit on the training split only. If PCA is fit on the full dataset before splitting, then each test image contributes to the estimation of the mean and eigenvectors used to reconstruct that same test image. This makes the PCA arm a transductive method that receives target-domain statistics at test time, so the comparison with the original-image arm is not a fair test of a fixed preprocessing rule. The observed worst-10 improvements could be fully or partly produced by this test-time adaptation. The authors must clarify exactly when PCA was fit, and if it was fit on the full datasets, all experiments must be rerun with PCA fit only on the training split, with the same split applied to compute PCA parameters.
  2. [Table 3] The central statistically significant claims are computed on the 'Worst 10' model-dataset pairs, which were selected post hoc based on the largest recall decline when using original images. The all-pairs comparison is not statistically significant (recall p=0.08, Dice p=0.16), and the 'Other 20' pairs show no effect (recall p=0.94, Dice p=0.62). Selecting the most favorable subset without correcting for multiple comparisons or pre-registering the analysis inflates the significance and makes the headline numbers unreliable. The abstract and Section 4 present the worst-10 results as the main finding, which is misleading. The primary analysis should be the all-pairs comparison, with the worst-10 analysis clearly labeled as exploratory and accompanied by an appropriate multiple-comparison correction or a pre-specified hypothesis.
  3. [Section 4] The claim that 'the absolute mean decline in recall values during external validation of initially 0.12 was reduced by 33% to 0.08' is not supported by the reported data. Neither the internal test recall values nor the per-pair external decline values are shown, so the reader cannot verify how the 0.12 decline was computed or how the 33% reduction is derived from the results in Tables 2 and 3. This is a headline quantitative claim and must be substantiated with the underlying per-dataset internal and external recall values, or removed.
minor comments (3)
  1. [Abstract and Section 3.2] The phrase 'twelve datasets' in the abstract is confusing: the study uses six original datasets and six PCA-reconstructed versions, so it would be clearer to say 'six original and six PCA-reconstructed datasets' rather than 'twelve datasets.'
  2. [Table 3] Several entries have spaced decimal points ('0 .72', '0 .64', '0 .58') that appear to be typographical artifacts; these should be corrected to standard decimal notation.
  3. [Section 4 and Figure 4] The word 'orignal' appears in the Figure 4 caption, and 'inferenced' is used in the abstract and Section 3.2; both should be corrected ('original' and 'evaluated' or 'inferred').

Circularity Check

1 steps flagged · score 5.0 of 10

PCA preprocessing appears to be fit on each full dataset before the split, so the PCA test arm is reconstructed with target-domain statistics; the headline worst-10 gains may reflect test-set leakage rather than a generalizable preprocessing effect.

  1. fitted input called prediction [Section 3.2, PCA reconstruction equation and train/validation/test split description]
    "PCA was employed to decompose each dataset into its principal components, followed by a reconstruction of the original dataset through a linear combination of these principal components. This is given by Xreconstructed = Xbar + sum... Consistency was a key focus throughout the experiments, as we maintained the same neural network architecture and training procedure, using the same 70/10/20 training/validation/test split to avoid data leakage (scikit-learn random state = 42)."

    The methods describe decomposing each dataset and reconstructing it before stating that the same split is used, and never say that PCA was fitted on the training split only. As written, the mean and principal components for each dataset are computed from all images in that dataset, including the images later assigned to the test set. Each out-of-domain PCA test image is therefore reconstructed with a basis estimated from that same image and its dataset, i.e., target-domain statistics are injected at test time.

full rationale

The paper contains no derivation chain in the usual sense: it trains U-Nets on original and PCA-reconstructed datasets and compares external validation. No self-citations, uniqueness theorems, or ansatz-imported claims are load-bearing. The central circularity risk is the evaluation protocol. Section 3.2 says 'PCA was employed to decompose each dataset' and then only later mentions using the same 70/10/20 split; it never states that PCA was fit on the training split alone. On the face of the text, the PCA test images are reconstructed using the test set's own mean and eigenvectors, so the 'unseen' PCA test domain is partially known to the preprocessing stage. That is a fitted input being presented as a fixed preprocessing prediction: the significant gains in the worst 10 pairs are statistically contaminated. The worst-10 subset is also selected by the baseline decline and then used for the headline p-values, which adds a regression-to-the-mean concern, though that is a statistical-selection issue rather than a definitional circularity. If the authors confirm PCA was fit only on training folds and the transform was applied to test images as a frozen mapping, the comparison would be legitimate and the score would drop to 0-1; as written, the evidence does not establish the external-validity claim.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that PCA per dataset does not leak test-set information, on the post hoc selection of the worst 10 pairs, and on the domain assumption that six datasets represent distinct domains. No new entities are introduced, but the PCA component counts are dataset-tuned hyperparameters that are not reported.

free parameters (3)
  • Number of principal components per dataset = not reported (chosen by Kaiser-Guttman criterion, ~90% variance)
    This per-dataset choice controls how much noise is removed and is tuned on each dataset's eigenvalue spectrum. The paper does not report the actual component counts, making the preprocessing hard to reproduce.
  • Loss weight beta = 0.5
    The combined Dice and binary cross-entropy loss uses a fixed weight of 0.5. It is not ablated, so its influence on the reported improvements is unknown.
  • Early stopping patience = 10 epochs
    A fixed training hyperparameter that affects model selection. It is standard but not varied, and its interaction with PCA preprocessing is not examined.
assumptions (4)
  • domain assumption Each dataset is a sample from the overall population of breast tumor ultrasound images, and the six datasets differ enough to represent meaningful domain shift.
    Stated in Section 3.2; the external-validity comparison depends on treating the six datasets as distinct domains with device, operator, and population variability.
  • domain assumption Keeping approximately 90% of variance via PCA removes noise while preserving tumor-relevant structure.
    Stated in Sections 3.2 and 4; the proposed mechanism for improved generalization relies on this variance-retention choice being neither too lossy nor too noisy.
  • ad hoc to paper The PCA transformation for each dataset is computed without leaking test-set information into the evaluation.
    The paper never states whether PCA is fit on training splits or whole datasets; the external-validity claim requires the former without test leakage.
  • standard math Standard PCA reconstruction formula X_reconstructed = mean + sum z_i w_i^T is correct.
    Section 3.2 presents the standard linear reconstruction used throughout; this is a textbook result and not a source of concern.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PCA for Enhanced Cross-Dataset Generalizability in Breast Ultrasound Tumor Segmentation." pith.science (2026). https://pith.science/paper/JQUNBLY6

@misc{pith2026250523587,
  author       = {Pith},
  title        = {Pith review of: PCA for Enhanced Cross-Dataset Generalizability in Breast Ultrasound Tumor Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JQUNBLY6}},
  note         = {Machine review of arXiv:2505.23587}
}
abstract

In medical image segmentation, limited external validity remains a critical obstacle when models are deployed across unseen datasets, an issue particularly pronounced in the ultrasound image domain. Existing solutions-such as domain adaptation and GAN-based style transfer-while promising, often fall short in the medical domain where datasets are typically small and diverse. This paper presents a novel application of principal component analysis (PCA) to address this limitation. PCA preprocessing reduces noise and emphasizes essential features by retaining approximately 90\% of the dataset variance. We evaluate our approach across six diverse breast tumor ultrasound datasets comprising 3,983 B-mode images and corresponding expert tumor segmentation masks. For each dataset, a corresponding dimensionality reduced PCA-dataset is created and U-Net-based segmentation models are trained on each of the twelve datasets. Each model trained on an original dataset was inferenced on the remaining five out-of-domain original datasets (baseline results), while each model trained on a PCA dataset was inferenced on five out-of-domain PCA datasets. Our experimental results indicate that using PCA reconstructed datasets, instead of original images, improves the model's recall and Dice scores, particularly for model-dataset pairs where baseline performance was lowest, achieving statistically significant gains in recall (0.57 $\pm$ 0.07 vs. 0.70 $\pm$ 0.05, $p = 0.0004$) and Dice scores (0.50 $\pm$ 0.06 vs. 0.58 $\pm$ 0.06, $p = 0.03$). Our method reduced the decline in recall values due to external validation by $33\%$. These findings underscore the potential of PCA reconstruction as a safeguard to mitigate declines in segmentation performance, especially in challenging cases, with implications for enhancing external validity in real-world medical applications.

Figures

Figures reproduced from arXiv: 2505.23587 by the authors.

Figure 1
Figure 1. Comparison of breast tumor image pair before (left) and after (right) PCA reconstruction, showing noise [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Exemplary scree plot of the BUSBRA dataset [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Cumulative explained variance graph for the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of exemplary visual segmentation results for the model-dataset-pair BUSI-BUS_UC. First [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

  1. [1]

    Dataset of breast ultrasound images

    Al-Dhabyani, W., Gomaa, M., Khaled, H., and Fahmy, A. Dataset of breast ultrasound images. Data in Brief, 28, pp.104863, 2020. DOI: https://doi.org/10.1016/j.dib.2019.104863

    arXiv 2020

  2. [3]

    On the Optimal Combination of Cross-Entropy and Soft Dice Losses for Lesion Segmentation with Out-of-Distribution Robustness

    Galdran, A., Carneiro, G., and Gonzalez Ballester, M. On the Optimal Combination of Cross-Entropy and Soft Dice Losses for Lesion Segmentation with Out-of-Distribution Robustness. In: Yap, M.H., Kendrick, C., Cassidy, B. (eds) Diabetic Foot Ulcers Grand Challenge. DFUC 2022. Lecture Notes in Computer Science, vol 13797. Springer, Cham. DOI: https://doi.or...

    work page doi:10.1007/978-3-031-26354-5 2022

  3. [4]

    BUS-BRA: A breast ultrasound dataset for assessing computer-aided diagnosis systems

    Gomez-Flores, W., Gregorio-Calas, M., and Pereira, W. BUS-BRA: A breast ultrasound dataset for assessing computer-aided diagnosis systems. Medical physics, 51, pp.3110-3123, 2023. DOI: https://doi.org/10.1002/mp.16812

    work page doi:10.1002/mp.16812 2023

  4. [5]

    Domain Adaptation for Medical Image Analysis: A Survey

    Guan, H., and Liu, M. Domain Adaptation for Medical Image Analysis: A Survey. IEEE Trans Biomed Eng, 69, pp.1173-1185, 2022. DOI: 10.1109/TBME.2021.3117407

    arXiv 2022

  5. [6]

    BUSI\_WHU: Breast Cancer Ultrasound Image Dataset

    Huang, J., and Zhang, J. BUSI\_WHU: Breast Cancer Ultrasound Image Dataset. Mendeley Data, 2023. DOI: 10.17632/k6cpmwybk3.1

    doi:10.17632/k6cpmwybk3.1 2023

  6. [7]

    Memory-efficient transformer network with feature fusion for breast segmentation and classification task

    Iqbal, A., and Sharif, M. Memory-efficient transformer network with feature fusion for breast segmentation and classification task. EAAI, 2023. DOI: 10.17632/3ksd7w7jkx.1

    doi:10.17632/3ksd7w7jkx.1 2023

  7. [8]

    Principal component analysis: A review and recent developments

    Joliffe, I., Cadima, J. Principal component analysis: A review and recent developments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 374 2016

    work page 2016

  8. [9]

    Contemporary Analysis of Reexcision and Conversion to Mastectomy Rates and Associated Healthcare Costs for Women Undergoing Breast-Conserving Surgery

    Kim, Y., and Ganduglia-Cabazan, C. Contemporary Analysis of Reexcision and Conversion to Mastectomy Rates and Associated Healthcare Costs for Women Undergoing Breast-Conserving Surgery. Ann Surg Oncol, 31, pp.3649-3660, 2024. DOI: https://doi.org/10.1245/s10434-024-14902-z

    work page doi:10.1245/s10434-024-14902-z 2024

Show all 12 references
  1. [10]

    Style transfer generative adversarial networks to harmonize multisite MRI to a single reference image to avoid overcorrection

    Liu, M., and Zhu, A.H. Style transfer generative adversarial networks to harmonize multisite MRI to a single reference image to avoid overcorrection. Human brain mapping, 44, pp.4875-4892, 2023. DOI: https://doi.org/10.1002/hbm.26422

    2023 doi

  2. [11]

    Curated benchmark dataset for ultrasound based breast lesion analysis

    Pawlowska, A., Cwierz-Pienkowska, A., Domalik, A., Jagus, D., Kasprzak, P., and Matkowski, R. Curated benchmark dataset for ultrasound based breast lesion analysis. Scientific Data, 11, 2024. DOI: 10.1038/s41597-024-02984-z

    2024 doi

  3. [12]

    Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation

    Stojanovski, D., and Hermida, U. Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation. In: Kainz, B., Noble, A., Schnabel, J., Khanal, B., Müller, J.P., Day, T. (eds) Simplifying Medical Ultrasound. ASMUS 2023. Lecture Notes...

    2023 doi

  4. [13]

    External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review

    Yu, A.C., and Mohajer, B. External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review. Radiology Artificial Intelligence, 2022. DOI: 10.1148/ryai.210064

    2022 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.