REVIEW 4 major objections 6 minor 24 references
Automated Fetal Biometry Assessment with Deep Ensembles using Sparse-Sampling of 2D Intrapartum Ultrasound Images
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A fully automated pipeline can turn intrapartum ultrasound videos into fetal angle-of-progression and head-symphysis distance measurements, the paper claims, reporting a classification accuracy of 0.9452 and a segmentation Dice score of…
desk verdict A workmanlike challenge pipeline whose reported biometry numbers are contradicted by its own tables; the segmentation part is credible, the measurement section is not. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is sparse sampling combined with uniformly weighted deep ensembles for classification and segmentation, and a hand-crafted geometric post-processing stage for measurement. Sparse sampling (5 frames per positive video, 8 per negative video) breaks the spurious correlation between video label and patient identity that caused overfitting when entire videos were used. Ensembles average softmax probabilities across models; the best classifier ensemble is CLF-5 (EfficientNetB0, ConvNeXt, ResNet-18 from scratch, VGG-11, DenseNet-196) and the best segmenter ensemble is SEG-3 (DeepLabV3+, MA-Net, UNet++). The measurement stage enforces structural fidelity: largest connected components are kept, holes are closed, edges are detected with Canny, an AMS ellipse is fitted, protrusions are iteratively pruned, and a decision rule chooses between ellipse and raw mask; HSD is computed from the hole-closed mask alone.
What would settle it
Have several independent sonographers annotate the fetal head and pubic symphysis on the same 224 held-out frames and compare their AoP and HSD values with each other and with the pipeline's outputs; if the inter-observer spread is similar to or larger than the pipeline's average errors, the automated measurements are within observer noise, while a much smaller spread would show the pipeline has a real bias. Also, running the full pipeline on a larger hold-out set, for example 40 patients, would show whether the 0.983 AUC and the reported measurement errors are stable or shrink toward the single-model numbers.
Extended reading notes
Core claim
The central discovery is that a three-stage pipeline built from existing deep classification and segmentation architectures, with two targeted modifications, reaches strong accuracy on intrapartum ultrasound biometry. The first modification is sparse sampling of frames during training, which prevents the classifier from memorising patients rather than learning standard planes. The second is ensemble averaging of classifier and segmenter predictions. The pipeline then converts segmentations into AoP and HSD estimates with a geometric post-processing chain: morphological closing, Canny edge detection, AMS ellipse fitting with iterative pruning of protrusions, and a decision rule that keeps the fitted ellipse only when it tightly covers the mask; HSD deliberately does not use ellipse fitting. The reported end-to-end measurement errors are 8.90 degrees for AoP and 14.35 pixels for HSD, and the paper argues these results demonstrate the potential of automated methods for supporting prenatal care.
Load-bearing premise
The clinical value of the reported AoP and HSD errors rests on the assumption that measurements computed from the challenge ground-truth segmentation masks match what a physician would measure on the same ultrasound images.
Editorial extensions
If this is right
- A clinician or midwife could run the pipeline on a labour-ward ultrasound video and receive AoP and HSD values automatically, removing the manual step that currently introduces observer variability.
- The sparse-sampling finding suggests that other video-classification tasks with video-level labels should sample a fixed number of frames per video to stop models from memorising patients instead of learning the visual pattern.
- The measurement post-processing approach offers a template for computing distances from segmentations in roughly elliptical anatomy: keep the largest connected component, close holes, fit an ellipse only when it tightly covers the mask, and skip ellipse fitting when the shape is complex.
- If the measurement errors stay within the range of inter-observer variability, the pipeline could be used to screen for labour arrest and to build risk-stratification tools for delivery planning.
Reading between the lines
- A direct extension the authors do not run is a clinician-led repeatability study: having several sonographers measure AoP and HSD twice on the same frames would give an inter-observer spread against which the 8.90-degree and 14.35-pixel errors could be judged as acceptable or not.
- The sparse-sampling ratio (5 positive frames, 8 negative frames) is presented as a heuristic; sweeping this ratio on a validation split could turn it into a principled rule and might change the reported headline metrics.
- Because the hold-out set contains only 4 patients, the pipeline's clinical claims would be better tested on a multi-centre set; the ensemble design is explicitly aimed at acquisition diversity, so a cross-machine evaluation would exercise that claimed robustness.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript describes a three-stage automatic pipeline for intrapartum ultrasound analysis: (1) classification of standard planes from video frames using sparse sampling and deep ensembles, (2) segmentation of the fetal head and pubic symphysis using ensemble segmentation models followed by connected-component and elliptic post-processing, and (3) computation of the angle of progression (AoP) and head-symphysis distance (HSD) from the resulting masks. The authors report results on a hold-out set of 4 patients and 224 frames, with high classification and segmentation scores (ACC 0.9452, F1 0.9225, AUC 0.983, MCC 0.8361; DSC 0.918, HD 19.73, ASD 5.71) and measurement errors Delta_AoP = 8.90 degrees and Delta_HSD = 14.35 pixels. The paper is written as a challenge-participation report, with extensive architecture comparisons and ensemble averaging.
Significance. If the reported numbers are accurate, the pipeline would be a practically relevant contribution to automated intrapartum fetal biometry, potentially reducing observer variability in AoP and HSD measurement. The paper deserves credit for systematically comparing many classification and segmentation architectures, for clearly explaining the sparse-sampling rationale to prevent subject-label leakage, and for evaluating both raw and ellipse-fitted measurement variants. However, the central biometry claim is currently not verifiable: the abstract and Section 4.3 values do not match Table 7, Tables 6 and 7 contain incompatible model-wise errors, the protrusion-pruning inequality in Section 2.3 is trivially true as written, and the HSD post-processing description is self-contradictory. These issues are load-bearing because the paper's contribution is specifically the measurement accuracy. The lack of code or per-patient results further limits reproducibility, and the 4-patient test set makes the headline metrics fragile.
major comments (4)
- [Section 4.3, Abstract, Tables 6-7] The abstract and Section 4.3 report average Delta_AoP = 8.906 degrees and Delta_HSD = 14.356 pixels, but Table 7, which is the only phase-3 result table, gives for the model described as best (SEG-3): DeltaPred_AoP = 10.571, DeltaEllips_AoP = 12.653, DeltaPred_HSD = 13.331, DeltaEllips_HSD = 12.053. No combination of these entries reproduces 8.906 or 14.356. Moreover, Table 6 reports U-Net Delta_AoP = 8.801 and Delta_HSD = 8.077, while Table 7 reports for U-Net DeltaPred_AoP = 10.571, DeltaEllips_AoP = 11.659, DeltaPred_HSD = 13.331, DeltaEllips_HSD = 11.516. These cannot be average errors for the same model on the same test set. The authors must state exactly which prediction variant (raw, ellipse, or decision-rule selected) and which model (single or ensemble) produced the headline numbers, and reconcile Tables 6 and 7. As written, the main biometry claim is unsupported.
- [Section 2.3, Step 4 and Figure 3] The protrusion condition is written as E union S over E union S < 1. This ratio is identically equal to 1, so the condition can never be true and the pruning loop would never execute. The surrounding prose indicates the intended comparison is between the number of pixels inside the ellipse but outside the segmentation and the number of segmentation pixels outside the ellipse, i.e., |E \ S| versus |S \ E|. The same notation confuses Step 5, where E union S is called the red region in Figure 3 even though the red region appears to be E \ S. The formula and set notation must be corrected, and the authors should confirm that the implemented pruning matches the intended inequality; otherwise the described post-processing cannot have produced the reported measurements.
- [Section 2.3, Step 6 versus Section 4.3] Section 2.3, Step 6 states that for HSD 'we found it more accurate to avoid assuming elliptical shapes' and to use only hole-closed segmentations, whereas Section 4.3 states that 'the choice of only using ellipse fitting for HSD is optimal'. These statements are directly contradictory. Table 7 provides both raw and ellipse-fitted error columns, but the text does not say which variant underlies the reported Delta_HSD = 14.356. The exact post-processing chain used for HSD (and for AoP) must be specified unambiguously, and the two sections must be made consistent.
- [Section 3 Testing and Section 4] The evaluation uses only 4 patients and 224 frames, and Section 3 states that model selection, hyperparameter tuning, ensemble composition, and post-processing choices were made on the same validation data from which these test videos were drawn. No confidence intervals or per-patient error distributions are reported. Given that Table 7 shows substantial variation across models and that the abstract makes generalizability claims about 'unseen' data, the authors should provide per-patient results and interval estimates, or explicitly limit the conclusions to a preliminary challenge-style evaluation.
minor comments (6)
- [Table 7 caption] The caption uses 'HDS' instead of 'HSD'.
- [Section 2.3, Step 6] The text contains 'predicated' where 'predicted' is intended.
- [Section 2.1] The activation function is referred to as 'GERU'; the standard name is GELU.
- [Section 2.2] DeepLabV3+ is cited as [24], but reference [24] is the UNet++ paper; the citation should be to [2].
- [Section 4.2 and Table 8] The text says SEG-3 achieved the highest average Dice score of 0.919, but Table 8 lists DSC = 0.918 for SEG-3; this small discrepancy should be reconciled or explained.
- [Section 2.3, Step 2] The term 'Approximate Mean Squared (AMS) ellipse' is not standard; please define it or use the conventional name such as least-squares ellipse.
Circularity Check
No significant circularity: the pipeline evaluations are empirical and self-contained; reported inconsistencies affect reproducibility, not circularity.
full rationale
This is an empirical machine-learning challenge paper rather than a derivation. The classification and segmentation models are trained on labelled training data and evaluated on a held-out set of 4 patients and 224 frames, so the reported ACC, F1, AUC, MCC, DSC, HD and ASD are standard supervised-evaluation numbers and do not reduce to their training inputs by construction. The Phase 3 AoP and HSD errors are computed by comparing measurements derived from predicted segmentation masks with measurements derived from the provided ground-truth masks; this is a conventional evaluation protocol and not a definitional identity with the model outputs. No load-bearing self-citation chain, imported uniqueness theorem, or ansatz-smuggled-via-citation step is present. The paper does contain internal inconsistencies that are relevant to correctness but not to circularity: Section 2.3 Step 6 states that HSD should avoid ellipse fitting ('we found it more accurate to avoid assuming elliptical shapes'), while Section 4.3 states 'the choice of only using ellipse fitting for HSD is optimal'; the abstract's Delta_AoP of 8.90 and Delta_HSD of 14.35 are not traceable from Table 7; and the protrusion condition written as E union S over E union S < 1 is identically 1. These are reproducibility or reporting defects, not cases where a prediction is equivalent to its input. Likewise, using part of the validation data for training and selecting hyperparameters on a small validation set is a possible selection-on-test-set concern, but it does not make the reported results tautological. Overall, no circular step meeting the quoted-evidence standard was found.
Assumptions & free parameters
free parameters (6)
- Sparse sampling counts =
5 frames per positive video, 8 per negative video
- Augmentation ranges =
Gaussian noise sigma 0.0118-0.0588, gamma 0.4-1, contrast 0.8-1.2, affine rotation +/-20 deg, translation -0.1 to 0.1…
- Canny edge thresholds =
minVal=2, maxVal=5
- Morphological closing kernel =
10x10 elliptical kernel
- Protrusion pruning parameters =
15 iterations, 'user-specified threshold', 20% decision threshold in Step 5
- Ensemble composition =
CLF-5 (5 classifiers), SEG-3 (3 segmenters)
assumptions (4)
- domain assumption Sparse sampling decorrelates the video-level label from patient identity
- domain assumption Canny edge detection and AMS ellipse fitting correctly represent the anatomical boundaries
- domain assumption The 90/10 split of the challenge validation data used for training and model selection does not leak information into the final test set
- ad hoc to paper The ill-defined protrusion condition 'E union S over E union S < 1' is intended to be a meaningful geometric inequality
Cite this review
Pith. "Pith review of Automated Fetal Biometry Assessment with Deep Ensembles using Sparse-Sampling of 2D Intrapartum Ultrasound Images." pith.science (2026). https://pith.science/paper/COETXSB4
@misc{pith2026250514572,
author = {Pith},
title = {Pith review of: Automated Fetal Biometry Assessment with Deep Ensembles using Sparse-Sampling of 2D Intrapartum Ultrasound Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/COETXSB4}},
note = {Machine review of arXiv:2505.14572}
}
abstract
The International Society of Ultrasound advocates Intrapartum Ultrasound (US) Imaging in Obstetrics and Gynecology (ISUOG) to monitor labour progression through changes in fetal head position. Two reliable ultrasound-derived parameters that are used to predict outcomes of instrumental vaginal delivery are the angle of progression (AoP) and head-symphysis distance (HSD). In this work, as part of the Intrapartum Ultrasounds Grand Challenge (IUGC) 2024, we propose an automated fetal biometry measurement pipeline to reduce intra- and inter-observer variability and improve measurement reliability. Our pipeline consists of three key tasks: (i) classification of standard planes (SP) from US videos, (ii) segmentation of fetal head and pubic symphysis from the detected SPs, and (iii) computation of the AoP and HSD from the segmented regions. We perform sparse sampling to mitigate class imbalances and reduce spurious correlations in task (i), and utilize ensemble-based deep learning methods for task (i) and (ii) to enhance generalizability under different US acquisition settings. Finally, to promote robustness in task iii) with respect to the structural fidelity of measurements, we retain the largest connected components and apply ellipse fitting to the segmentations. Our solution achieved ACC: 0.9452, F1: 0.9225, AUC: 0.983, MCC: 0.8361, DSC: 0.918, HD: 19.73, ASD: 5.71, $\Delta_{AoP}$: 8.90 and $\Delta_{HSD}$: 14.35 across an unseen hold-out set of 4 patients and 224 US frames. The results from the proposed automated pipeline can improve the understanding of labour arrest causes and guide the development of clinical risk stratification tools for efficient and effective prenatal care.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Frontiers in physiology13, 940150 (2022) Title Suppressed Due to Excessive Length 15
Bai, J., Sun, Z., Yu, S., Lu, Y., Long, S., Wang, H., Qiu, R., Ou, Z., Zhou, M., Zhi, D., et al.: A framework for computing angle of progression from transperineal ultrasound images for evaluating fetal head descent using a novel double branch network. Frontiers in physiology13, 940150 (2022) Title Suppressed Due to Excessive Length 15
work page 2022
-
[2]
In: Proceedings of the European conference on computer vision (ECCV)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV). pp. 801–818 (2018)
2018
-
[3]
Advances in neural information processing systems30(2017)
Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S., Feng, J.: Dual path networks. Advances in neural information processing systems30(2017)
work page 2017
-
[4]
Expert Systems with Applications245, 123096 (2024)
Chen, Z., Ou, Z., Lu, Y., Bai, J.: Direction-guided and multi-scale feature screening for fetal head–pubic symphysis segmentation and angle of progression calculation. Expert Systems with Applications245, 123096 (2024)
2024
-
[5]
In: Jawahar, C.V., Li, H., Mori, G., Schindler, K
Dawson, M., Zisserman, A., Nellåker, C.: From same photo: Cheating on visual kinship challenges. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) Computer Vision – ACCV 2018. pp. 654–668. Springer International Publishing, Cham (2019)
work page 2019
-
[6]
IEEE Access8, 179656–179665 (2020)
Fan, T., Wang, G., Li, Y., Wang, H.: Ma-net: A multi-scale attention network for liver and tumor segmentation. IEEE Access8, 179656–179665 (2020)
work page 2020
-
[7]
arXiv preprint arXiv:1912.02757 (2019)
Fort, S., Hu, H., Lakshminarayanan, B.: Deep ensembles: A loss landscape perspective. arXiv preprint arXiv:1912.02757 (2019)
arXiv 2019
-
[8]
Ultrasound in Obstetrics & Gynecology52(1), 128–139 (2018)
Ghi, T., Eggebø, T., Lees, C., Kalache, K., Rozenberg, P., Youssef, A., Salomon, L., Tutschek, B.: Isuog practice guidelines: intrapartum ultrasound. Ultrasound in Obstetrics & Gynecology52(1), 128–139 (2018)
2018
Show all 24 references
-
[9]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015),https://arxiv.org/abs/1512.03385
2015 arXiv
-
[10]
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks (2018),https://arxiv.org/abs/1608.06993
2018 arXiv
-
[11]
Iakubovskii, P.: Segmentation models pytorch.https://github.com/qubvel/ segmentation_models.pytorch(2019)
2019
-
[12]
Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems30(2017)
2017
-
[13]
Advances in neural information processing systems30(2017)
Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems30(2017)
2017
-
[14]
Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s (2022),https://arxiv.org/abs/2201.03545
2022 arXiv
-
[15]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3431–3440 (2015)
2015
-
[16]
In: Medical image computing and computer- assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer- assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. pp...
2015
-
[17]
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015),https://arxiv.org/abs/1409.1556
2015 arXiv
-
[18]
Tan, M., Le, Q.V.: Efficientnet: Rethinking model scaling for convolutional neural networks (2020),https://arxiv.org/abs/1905.11946
2020 arXiv
-
[19]
Tan, M., Le, Q.V.: Efficientnetv2: Smaller models and faster training (2021), https://arxiv.org/abs/2104.00298
2021 arXiv
-
[20]
In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Valdenegro-Toro, M., Mori, D.S.: A deeper look into aleatoric and epistemic uncertainty disentanglement. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). pp. 1508–1516. IEEE (2022)
2022
-
[21]
Ramesh et al
Wightman, R.: Pytorch image models.https://github.com/rwightman/ pytorch-image-models(2019).https://doi.org/10.5281/zenodo.4414861 16 J. Ramesh et al
2019 doi
-
[22]
Pattern Recognition135, 109131 (2023)
Yu, X., Wang, J., Zhao, Y., Gao, Y.: Mix-vit: Mixing attentive vision transformer for ultra-fine-grained visual categorization. Pattern Recognition135, 109131 (2023)
2023
-
[23]
org/abs/1605.07146
Zagoruyko, S., Komodakis, N.: Wide residual networks (2017),https://arxiv. org/abs/1605.07146
2017 arXiv
-
[24]
IEEE transactions on medical imaging39(6), 1856–1867 (2019)
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE transactions on medical imaging39(6), 1856–1867 (2019)
2019
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.