Semi-supervised Learning of Fetal Anatomy from Ultrasound

Anselm Au; Bernhard Kainz; Jeremy Tan; Qingjie Meng

REVIEW 3 major objections 4 minor 14 references

Semi-supervised Learning of Fetal Anatomy from Ultrasound

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Semi-supervised learning mostly benefits distinct classes in fetal ultrasound and can be detrimental when a background class is included, especially for similar cardiac views.

desk verdict Useful cautionary study on SSL in fetal ultrasound, but the headline claim about background classes is confounded by test-set hyperparameter tuning and missing variance estimates. read the letter →

arxiv 1908.11624 v1 pith:G2EQD4YJ submitted 2019-08-30 cs.CV

Jeremy Tan , Anselm Au , Qingjie Meng , Bernhard Kainz This is my paper · ORCID

classification cs.CV

keywords semi-supervisedlearningfetalultrasoundstandardplaneclassificationconsistencytrainingUDAcardiacviewconfusionbackgroundclassSonoNet

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish whether semi-supervised learning (SSL), which has worked on clean benchmark image sets, still helps when the data are messy fetal ultrasound frames. The authors find that a supervised Sononet baseline already classifies standard planes surprisingly well from as few as 20 labelled images per class. Adding a consistency-based SSL loss helps classes that are already visually distinct, such as brain, femur, kidney, and lips, but can increase confusion among the four very similar cardiac views. A large, heterogeneous background class in the unlabeled pool tends to make SSL worse than the supervised baseline. If these results are right, SSL cannot be applied uniformly in fetal ultrasound; it needs per-class treatment and careful handling of out-of-distribution frames.

What carries the argument

The central mechanism is a consistency loss $L_{KL}(x_u,w)=KL(f(x_u;w)\|f(x_u';w))$ that uses the softmax prediction of an unlabelled image as the target for the same image under random augmentation, weighted at $\lambda=0.5$ and combined with training signal annealing, confidence-based masking (default $\eta_{cbm}=0.75$), entropy minimization, and softmax temperature 0.8, a combination the paper calls UDA. The supervised backbone is Sononet, a 15-layer CNN with global average pooling. The decisive control in the argument is the lowered cardiac confidence-mask threshold of $\eta_{cbm}/3=0.25$, since this single change decides whether SSL helps or hurts the cardiac classes.

What would settle it

Run the 5/20/50-label protocol again but choose the cardiac threshold and TSA schedule on a held-out validation split for each label count, or fix all hyperparameters before seeing test labels; if SSL then fails to beat the supervised baseline on most label counts, or if the background class no longer causes a drop, the central claim would be weakened.

Watch

Extended reading notes

Core claim

The central claim is that SSL's benefit in fetal 2D ultrasound standard-plane classification is class-dependent and can be negative. Without the background class and with a cardiac-specific confidence-mask threshold of 0.25, the best UDA configuration improves both overall accuracy (0.720 to 0.754) and grouped cardiac accuracy (0.868 to 0.936) at 20 labels per class; inclusion of the background class makes SSL accuracy fall below the supervised baseline for most label counts. Confusion matrices show that SSL drives distinct classes toward fully-supervised performance while increasing mislabelling among the four cardiac views, implying that unlabeled data helps only when classes are already separable.

Load-bearing premise

The load-bearing premise is that the hyperparameters chosen on the test set for the 20-labelled-per-class case, especially the cardiac confidence threshold of 0.25 and the log-TSA-with-Adam configuration, transfer to the 5- and 50-label runs; if they do not, the size or direction of the claimed SSL benefit is not established.

Editorial extensions

If this is right

With 20 labelled images per class and no background class, the best UDA configuration raises overall accuracy from 0.720 to 0.754 and grouped cardiac accuracy from 0.868 to 0.936, but only when the cardiac confidence mask threshold is lowered to 0.25.
Including the background class in the unlabeled set makes SSL fall below the supervised baseline in most of the 5/20/50-label settings, so unlabeled frames that resemble standard planes but are not standard planes are not harmless.
In the 20-label case, SSL lifts distinct classes such as brain, femur, kidney, and lips from mid-0.80 to mid-0.90 accuracy while increasing confusion among the four cardiac views.
A supervised baseline already reaches roughly 70% overall and 80% grouped cardiac accuracy from 20 labels per class, and accuracy plateaus quickly, so the realistic margin SSL can add is small and concentrated in specific classes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Not shown in the paper: the test-set-selected hyperparameters, especially the cardiac threshold of 0.25 and the log-TSA-with-Adam configuration, likely inflate the reported SSL margin; a validation-split protocol would probably show a smaller benefit.
Not shown in the paper: a natural testable extension is to make the confidence threshold class-dependent, set near one over the number of classes in each group, and check whether SSL then helps similar classes.
Not shown in the paper: the background-class result suggests that explicit out-of-distribution detection or filtering of confusable background frames could turn detrimental unlabeled data into useful signal.
Not shown in the paper: the per-class pattern predicts that SSL will also hurt other high-similarity medical-image groupings, such as different views of the same organ, when unlabeled data come from the same heterogeneous distribution.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth knowing about: it tests consistency-based SSL (Pi model and UDA) on a realistic fetal ultrasound classification task and reports two things. First, SSL mostly helps classes that are already distinct and can increase confusion among very similar cardiac views. Second, adding a challenging background class to the unlabeled pool can make SSL worse than a supervised baseline. The first result is credible and nicely visualized in the confusion matrices. The second is the paper's headline, and it is not actually established by the reported experiments.

The problem is that the SSL configuration is tuned on the 20-labels-per-class no-background case. Table 1 picks a cardiac confidence threshold of 0.25; Table 2 picks log TSA with Adam. Then the paper says it uses that configuration for all label counts, with and without background. Adding the background class changes the output distribution from 13 to 14 classes and introduces a large, diverse category. There is no reason to expect the tuned threshold or TSA schedule to transfer well. The comparison therefore conflates "background hurts SSL" with "background-inappropriate hyperparameters hurt SSL." The authors' own justification that 0.25 is reasonable for four similar cardiac classes makes sense without background, but it does not address how background changes the confidence distribution. This is a fixable flaw: rerun the tuning on the with-background case, or at least do a small sensitivity sweep, and report whether the conclusion holds.

The paper also gives no standard deviations or multiple seeds. All results appear to be single runs. For a paper whose main quantitative claim is about a 5–10 point accuracy difference, that is a real gap. The qualitative pattern in the confusion matrices is less affected by this, but the accuracy bars in Figure 3 should be treated as indicative, not definitive.

What is genuinely new is the application domain and the observation that SSL's benefit is asymmetric across class similarity. That is a useful contribution to the medical-imaging SSL literature, and it aligns with Oliver et al.'s call for realistic evaluation. The paper does not ship code or data, which limits independent verification, and the citation pattern is fine—no obvious self-citation inflation.

I would accept this for peer review because the question is important and the empirical setup is relevant, even though the main claim needs a cleaner evaluation protocol before publication. If the authors fix the tuning issue and add variance estimates, this becomes a solid subfield-level measurement. For now, it is a cautionary result with a confound at its core.

Referee Report

3 major / 4 minor

Summary. The paper studies semi-supervised learning (consistency training / UDA) for standard-plane classification in fetal 2D ultrasound, using the SonoNet architecture and a dataset of 13 anatomical classes plus a background class. With 5, 20, and 50 labelled images per class, the authors report that supervised baselines are surprisingly strong, that SSL improves accuracy on distinct classes but can increase confusion among similar cardiac classes, and that including a challenging background class can make SSL detrimental. The SSL configuration is selected on the 20-labelled-per-class, no-background setting and then applied to all other conditions. The paper is positioned as a response to calls for realistic evaluation of SSL.

Significance. If the findings hold, the paper provides a valuable, realistic counterpoint to benchmark-driven SSL results, showing that class imbalance, a diverse background class, and high inter-class similarity can change the direction of SSL benefits. The use of a real clinical dataset, comparison with a strong supervised baseline, and confusion-matrix analysis are genuine strengths. The central claims, however, rest on test-set hyperparameter selection and single-run results, so the quantitative and qualitative conclusions are not yet established. With additional experiments and proper validation, the paper could make a useful contribution to the medical-imaging SSL literature.

major comments (3)

[Section 4, Tables 1 and 2, Figure 3] The central claim that including a background class makes SSL detrimental is not established because the SSL hyperparameters are selected on test accuracy from the 20-labelled-per-class, no-background setting and then applied to all other conditions. Table 1 selects the cardiac confidence threshold 1/3*eta_cbm = 0.25 from that setting, and Table 2 selects the log TSA schedule with Adam from the same setting; the text then states that the best found configuration is used for 5, 20, and 50 labelled images per class, with and without the background class. Adding the background class changes the output distribution from 13 to 14 classes and introduces a large, diverse, class-imbalanced category, so the optimal masking threshold and training schedule may not transfer. The comparison in Figure 3 therefore conflates 'background hurts SSL' with 'background-incorrect hyperparameters hurt SSL'. A proper validation split or per-condition hyperparameter selection is needed.
[Section 4, Tables 1 and 2, Figure 3] No variance estimates, multiple seeds, or significance tests are reported anywhere. Every accuracy number is the result of a single training run, and several differences used to support the conclusions are small (for example, overall 0.754 vs 0.720 in Table 2, and the blue-versus-red gaps in Figure 3). These differences could be within run-to-run variation, so the direction and magnitude of the claimed SSL benefit is not established. The authors should rerun experiments with several seeds and report mean and standard deviation.
[Section 4 and Figure 4] The qualitative claim that SSL helps distinct classes while increasing confusion among similar cardiac views is read from a single pair of confusion matrices and one threshold choice. The cardiac threshold of 0.25 was itself chosen to avoid harming cardiac classes in Table 1, so the confusion-matrix pattern is conditional on that test-selected hyperparameter. The authors should show how the confusion pattern varies across thresholds and seeds before drawing this conclusion.

minor comments (4)

[Figure 1] The class label 'SAGITAL' should be 'SAGITTAL', and the caption contains the same misspelling in 'SPINE-SAGITAL'.
[Figure 3 caption] There is a typo: 'baslines' should be 'baselines'.
[Table 1 and Section 4] The threshold notation is inconsistent between the table and the text: the table lists '1/3*eta_cbm', while the text says 'A cardiac threshold of 0.25' and '1/3*eta_cbm (0.25)'. State explicitly that eta_cbm = 0.75 in all experiments to make the arithmetic clear.
[Section 3.2] Clarify how the reported 'overall' accuracy is computed when the background class is excluded: is it mean per-class accuracy or overall accuracy over the non-background test subset? This affects interpretation of the numbers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical comparisons, not derivations from fitted inputs or self-cited results.

full rationale

This paper makes no formal derivation; its central findings—that a background class can make SSL detrimental, that SSL helps distinct classes while increasing cardiac confusion, and that supervised baselines are surprisingly strong—are supported by direct empirical comparisons reported in Figures 2, 3, and 4. No result is obtained by defining a quantity in terms of another, and no fitted parameter is renamed as a prediction. The architecture Sononet is taken from prior work that includes a co-author, but it is used as a fixed baseline and is not the source of the claimed SSL conclusions. The SSL method (UDA-style consistency training) is also external to the authors. The cardiac confidence-mask threshold and TSA schedule are selected using test accuracy in the no-background, 20-labels-per-class setting and then transferred to other label counts and to the background-inclusive setting; this is a legitimate methodological weakness and a possible source of confounding, but it is parameter selection on a test set, not circular reasoning. The observed effects do not reduce by construction to the chosen hyperparameters. No circular steps were identified, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

The paper introduces no new entities and makes no derivation. Its central claims rest on two domain assumptions about the background class and the evaluation metric, plus several hyperparameters, of which the cardiac threshold and UDA configuration are tuned on the test set.

free parameters (3)

cardiac confidence mask threshold = 0.25 (1/3 of base eta_cbm=0.75)
Selected based on test accuracy on 20 labelled images per class without background (Table 1), then fixed for all other experiments.
UDA configuration (TSA schedule, optimizer, learning rate) = log TSA schedule with Adam LR 1e-3
Selected based on test accuracy in Table 2 for the 20-labelled case, then used for 5, 20, and 50 labelled cases. Other settings (linear TSA, momentum, SGD cyclic) were tested and discarded.
consistency loss weight lambda = 0.5
Fixed by the authors for all experiments. It is a hand-chosen hyperparameter that affects the balance of supervised and consistency gradients.

assumptions (2)

domain assumption The background class, sampled by a minimum frame-distance filter, is representative of challenging out-of-distribution data for SSL.
Section 3.1 states that background frames are sampled during rapid probe movement, excluding frames near standard planes, but some background images still resemble standard planes. The paper's conclusion that background inclusion is harmful to SSL assumes this sampling yields realistic confounding data.
domain assumption The accuracy metric that excludes the background class and the test set are appropriate for measuring SSL benefit on anatomical classes.
Section 3.2 reports accuracy only for anatomical classes, not background. The paper assumes this metric reflects clinical utility and that the test set is representative of real exams.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Semi-supervised Learning of Fetal Anatomy from Ultrasound." pith.science (2026). https://pith.science/paper/G2EQD4YJ

@misc{pith2026190811624,
  author       = {Pith},
  title        = {Pith review of: Semi-supervised Learning of Fetal Anatomy from Ultrasound},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G2EQD4YJ}},
  note         = {Machine review of arXiv:1908.11624}
}

read the original abstract

Semi-supervised learning methods have achieved excellent performance on standard benchmark datasets using very few labelled images. Anatomy classification in fetal 2D ultrasound is an ideal problem setting to test whether these results translate to non-ideal data. Our results indicate that inclusion of a challenging background class can be detrimental and that semi-supervised learning mostly benefits classes that are already distinct, sometimes at the expense of more similar classes.

Figures

Figures reproduced from arXiv: 1908.11624 by the authors.

**Figure 2.** Supervised baselines with varying number of labelled images per class. Inclusion of the background class increases the difficulty of the classification task. Accuracy is reported as either overall or grouping all cardiac classes into a single class. When applying the SSL consistency loss, we find the performance is sensitive to the consistency loss masking threshold, ηcbm. This is particularly true for the cardiac c… view at source ↗

**Figure 3.** Overall and grouped cardiac accuracies with and without the background class for 5, 20, and 50 labelled images per class. Red bars indicate supervised baseline performance. Without the background class (green bars), SSL accuracy is almost always above supervised baslines (red). Inclusion of the background class can cause SSL performance to drop below supervised baselines. Confusion matrices are displayed in [PITH_F… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Confusion matrices for the case of 20 labelled images per class without background. The supervised baseline (a) performs surprisingly well given the limited data. SSL (b) is able to make considerable improvement for distinct classes, but can increase confusion of card…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 13 canonical work pages

[1]

ICLR (2019)

Athiwaratkun, B., Finzi, M., Izmailov, P., Wilson, A.G.: There are many consistent explanations of unlabeled data: Why you should average. ICLR (2019)

work page 2019
[2]

IEEE Transactions on Medical Imaging 36(11), 2204–2215 (2017)

Baumgartner, C.F., Kamnitsas, K., Matthew, J., Fletcher, T.P., Smith, S., Koch, L.M., Kainz, B., Rueckert, D.: SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound. IEEE Transactions on Medical Imaging 36(11), 2204–2215 (2017)

work page 2017
[3]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Cai, Y., Sharma, H., Chatelain, P., Noble, J.A.: Multi-task sonoeyenet: Detection of fetal standardized planes assisted by generated sonographer attention maps. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 871–879 (2018)

work page 2018
[4]

IEEE Journal of Biomedical and Health Informatics 19 (2015)

Chen, H., Ni, D., Qin, J., Li, S., Yang, X., Wang, T., Heng, P.A.: Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks. IEEE Journal of Biomedical and Health Informatics 19 (2015)

work page 2015
[5]

Medical Image Analysis 54, 280 – 296 (2019)

Cheplygina, V., de Bruijne, M., Pluim, J.P.: Not-so-supervised: A survey of semi- supervised, multi-instance, and transfer learning in medical image analysis. Medical Image Analysis 54, 280 – 296 (2019)

work page 2019
[6]

CVPR (2019)

Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learn- ing augmentation policies from data. CVPR (2019)

work page 2019
[7]

NeurIPS (2005)

Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. NeurIPS (2005)

work page 2005
[8]

Data Driven Treatment Response Assessment and Preterm, Perinatal, and Paedi- atric Image Analysis, LNCS 11076 (2018)

Kong, P., Ni, D., Chen, S., Wang, T., Lei, B.: Automatic and Eﬃcient Standard Plane Recognition in Fetal Ultrasound Images via Multi-scale Dense Networks. Data Driven Treatment Response Assessment and Preterm, Perinatal, and Paedi- atric Image Analysis, LNCS 11076 (2018)

work page 2018

Show all 14 references

[9]

ICLR (2017)

Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. ICLR (2017)

2017
[10]

NeurIPS (2018)

Oliver, A., Odena, A., Raﬀel, C., Cubuk, E.D., Goodfellow, I.J.: Realistic evalua- tion of deep semi-supervised learning algorithms. NeurIPS (2018)

2018
[11]

NeurIPS (2008)

Singh, A., Nowak, R., Zhu, J.: Unlabeled data: Now it helps, now it doesn’t. NeurIPS (2008)

2008
[12]

IJCAI (2019)

Verma, V., Lamb, A., Kannala, J., Bengio, Y.: Interpolation consistency training for semi-supervised learning. IJCAI (2019)

2019
[13]

World Health Organization (2016)

World Health Organization, et al.: WHO recommendations on antenatal care for a positive pregnancy experience. World Health Organization (2016)

2016
[14]

arXiv preprint arXiv:1904.12848 (2019)

Xie, Q., Dai, Z., Hovy, E., Luong, M.T., Le, Q.V.: Unsupervised data augmentation for consistency training. arXiv preprint arXiv:1904.12848 (2019)

2019 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

ICLR (2019)

Athiwaratkun, B., Finzi, M., Izmailov, P., Wilson, A.G.: There are many consistent explanations of unlabeled data: Why you should average. ICLR (2019)

work page 2019

[2] [2]

IEEE Transactions on Medical Imaging 36(11), 2204–2215 (2017)

Baumgartner, C.F., Kamnitsas, K., Matthew, J., Fletcher, T.P., Smith, S., Koch, L.M., Kainz, B., Rueckert, D.: SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound. IEEE Transactions on Medical Imaging 36(11), 2204–2215 (2017)

work page 2017

[3] [3]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Cai, Y., Sharma, H., Chatelain, P., Noble, J.A.: Multi-task sonoeyenet: Detection of fetal standardized planes assisted by generated sonographer attention maps. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 871–879 (2018)

work page 2018

[4] [4]

IEEE Journal of Biomedical and Health Informatics 19 (2015)

Chen, H., Ni, D., Qin, J., Li, S., Yang, X., Wang, T., Heng, P.A.: Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks. IEEE Journal of Biomedical and Health Informatics 19 (2015)

work page 2015

[5] [5]

Medical Image Analysis 54, 280 – 296 (2019)

Cheplygina, V., de Bruijne, M., Pluim, J.P.: Not-so-supervised: A survey of semi- supervised, multi-instance, and transfer learning in medical image analysis. Medical Image Analysis 54, 280 – 296 (2019)

work page 2019

[6] [6]

CVPR (2019)

Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learn- ing augmentation policies from data. CVPR (2019)

work page 2019

[7] [7]

NeurIPS (2005)

Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. NeurIPS (2005)

work page 2005

[8] [8]

Data Driven Treatment Response Assessment and Preterm, Perinatal, and Paedi- atric Image Analysis, LNCS 11076 (2018)

Kong, P., Ni, D., Chen, S., Wang, T., Lei, B.: Automatic and Eﬃcient Standard Plane Recognition in Fetal Ultrasound Images via Multi-scale Dense Networks. Data Driven Treatment Response Assessment and Preterm, Perinatal, and Paedi- atric Image Analysis, LNCS 11076 (2018)

work page 2018