REVIEW 4 major objections 6 minor 44 references
Bayesian Feature Pyramid Networks for Automatic Multi-Label Segmentation of Chest X-rays and Assessment of Cardio-Thoratic Ratio
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that a Bayesian Feature Pyramid Network with Monte-Carlo dropout can segment heart and lungs on chest X-rays and estimate the cardiothoracic ratio with uncertainty bounds, generalizing across four test sets from…
desk verdict A competent, honest applied paper whose central novelty—CTR uncertainty—is not yet substantiated because the MC-dropout spread is never calibrated or validated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the Feature Pyramid Network with a ResNet-50 backbone, modified in three ways: dropout layers placed before the second, third, and fourth residual blocks and inside the decoder; instance normalization replacing batch normalization in every decoder block; and spatial dropout before the final two-channel output. At test time the network is run multiple times with dropout enabled, producing Monte-Carlo samples of the heart and lung masks; the mean of these samples is the segmentation and their spread gives uncertainty. CTR is then the quotient of the widest heart chord to the widest lung chord, computed directly from the sampled masks. The variational interpretation of dropout, where training with dropout and cross-entropy minimizes the KL divergence to the posterior, is what licenses treating the sample spread as a Bayesian uncertainty estimate.
What would settle it
Segment a shared set of, say, 50 images with both radiologist A and radiologist B masks and compute their agreement; if their mutual IoU is no better than the model's IoU against radiologist B, the reported segmentation results mostly reflect annotation style rather than anatomical accuracy. Separately, collect many X-rays with CT-confirmed heart widths and check whether the model's 95% CTR intervals contain the true ratio at the claimed rate; coverage far below 95% would falsify the uncertainty claim.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that inserting dropout into both a ResNet-50 encoder and a Feature Pyramid Network decoder, and replacing batch normalization with instance normalization, turns a standard segmentation network into a Bayesian one that produces multi-label heart and lung masks together with per-pixel aleatoric (data noise) and epistemic (model uncertainty) uncertainty maps. Averaging twenty stochastic forward passes gives the segmentation, and the ratio of the widest heart diameter to the widest lung diameter computed from these masks reproduces the ground-truth CTR with Pearson correlations between 0.87 and 0.97 across four test sets. The paper claims this is the first CTR-assessment method that reports uncertainty bounds rather than point estimates.
Load-bearing premise
The external test sets were annotated by a different radiologist than the training data, and the paper assumes the two annotation styles are similar enough that IoU against the second radiologist's masks measures segmentation quality; this is stated but not verified by any inter-rater agreement score.
Editorial extensions
If this is right
- CTR measurements for a single X-ray can be reported as a range, not a point estimate, so a clinician sees when the model is unsure.
- A model trained on one annotation style with diverse radiological findings transfers to external scanners without domain adaptation, provided the test masks are annotated the same way.
- Twenty Monte-Carlo samples are enough for stable segmentation and CTR correlation; more samples add little.
- Instance normalization in the decoder improves heart and lung segmentation over batch and group normalization across all tested decoders.
- The released multi-label annotations with true organ boundaries create a more challenging benchmark than existing tuberculosis-oriented datasets.
Reading between the lines
- The reported IoU numbers likely mix segmentation accuracy with annotation-style agreement, because a different radiologist labeled the external test sets; computing inter-rater agreement on an overlapping subset would separate the two effects.
- If the uncertainty estimates are well calibrated, the epistemic (mutual information) maps could serve as an automatic flag for images that need human review, a use the paper mentions but does not validate.
- The same dropout-and-instance-normalization recipe could be applied to other ratio-based measurements in radiography, such as vertebral or cardiomediastinal ratios, where a confidence interval matters clinically.
- A direct calibration test, checking that the 95% predictive interval contains the true CTR roughly 95% of the time, would turn the proposed uncertainty bounds into a clinically usable quantity; the paper does not report this.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Bayesian Feature Pyramid Network (FPN) with a ResNet-50 backbone, instance normalization in the decoder, and Monte-Carlo dropout for multi-label segmentation of heart and lung fields in chest X-rays. The segmentation outputs are used to compute the cardiothoracic ratio (CTR), and the paper claims, for the first time, to estimate CTR with uncertainty bounds derived from MC dropout. The method is trained on a newly annotated subset of ChestXray14 and evaluated on ChestXray14, JSRT, Montgomery, and Shenzhen datasets, reporting IoU/Dice for segmentation and Pearson correlation for CTR. The authors also release annotations and report an extensive ablation study over encoders, decoders, and normalization layers.
Significance. If the uncertainty-quantification claim holds, the paper would make a useful clinical contribution by providing confidence bounds for an automated CTR measurement, and the released annotations and multi-dataset evaluation are valuable assets to the community. The segmentation results appear solid, and the ablation study is thorough. However, the central uncertainty contribution is compromised by the use of categorical entropy formulas on multi-label sigmoid outputs and by the absence of any quantitative validation of the uncertainty bounds, so the significance of the paper in its current form is limited to the segmentation and point-estimate CTR results.
major comments (4)
- [Bayesian Segmentation Framework: Aleotoric and Epistemic Uncertainties (Eqs. 2-3)] Equations (2) and (3) are the standard MC-dropout predictive entropy and mutual information formulas for a categorical softmax distribution over mutually exclusive classes. The network has two independent sigmoid outputs for heart and lungs, so at any pixel the joint predictive distribution is a product of two Bernoulli distributions. The correct predictive entropy is H[p_h] + H[p_l] = -p_h log p_h - (1-p_h) log(1-p_h) - p_l log p_l - (1-p_l) log(1-p_l). As written, Eq. (2) sums only over the label dimension and omits the (1-p) terms, systematically underestimating the aleatoric uncertainty; Eq. (3) inherits this error. The authors should either correct the formulas to the joint multi-label entropy or explicitly justify why the categorical approximation is appropriate for two independent sigmoid outputs.
- [Section 4.4, Fig. 5] The number of MC dropout samples T=20 is selected by inspecting IoU and Pearson correlation computed on the test sets (Fig. 5 states 'optimal number of iterations on all datasets' from test curves). Selecting hyperparameters on the test data makes the reported numbers in Table 3 a selected optimum rather than a clean hold-out evaluation. The choice of T should be made on the validation set, or the reported results should be accompanied by a sensitivity analysis that explicitly accounts for the selection.
- [Table 3 and Section 4.4] The central claim of the paper is estimating CTR with uncertainty bounds, yet no quantitative validation of these bounds is provided. The experiments report IoU, Dice, and Pearson correlation of point estimates only; there is no assessment of interval coverage, calibration curves, or correlation between the MC spread and the absolute error |CTR_pred - CTR_true|. Without such validation, the uncertainty bounds are unsupported. Please add a calibration or coverage analysis to substantiate the uncertainty claim.
- [Section 5 (Limitations)] The manuscript states that radiologist A annotated the training and ChestXray14 test data while radiologist B annotated JSRT, Montgomery, and Shenzhen, and that inter-rater agreement was not computed. The authors assert that this limitation has insignificant impact on the results, but provide no supporting evidence. Since cross-dataset generalization is a main contribution, the lack of inter-rater agreement assessment is load-bearing. A small double-annotated subset with IoU/Dice between the two radiologists would make the evaluation more compelling.
minor comments (6)
- [Title and Abstract] The title and abstract contain the typo 'Cardio-Thoratic'; the correct spelling is 'Cardiothoracic' or 'Cardio-Thoracic'.
- [Equation (5)] The loss in Eq. (5) is written as BCE - J, but J is not defined explicitly. If J is the soft Jaccard index (higher is better), the minimization should be BCE + (1 - J) or an equivalent form; please clarify the definition of J.
- [Related Work / References] The citation to Mukhoti and Gal [25] is misspelled as 'Mukohti' in the text.
- [Section 2] The phrase 'detection of plural effusion' should read 'pleural effusion'.
- [Data Availability] The URL given for the released dataset and code is the placeholder 'http://will.be.placed.after.review.' and should be replaced with an actual repository location.
- [Table 3] Table 3 reports IoU/Dice and Pearson correlations without any confidence intervals, although Fig. 5 shows bootstrapped 95% intervals for similar metrics; adding such intervals to Table 3 would improve the comparability and statistical transparency of the results.
Circularity Check
No significant circularity: the derivation is an empirical segmentation-to-CTR pipeline with external methodological citations and external test sets.
full rationale
The paper's derivation chain is empirical rather than definitional: the network outputs pixel-wise heart/lung masks, CTR is computed from those masks by the standard formula CTR=(A+B)/(C+D) following Dong et al. [9], and the uncertainty estimates are obtained by applying MC-dropout formulas from Kendall et al. [21] and Mukhoti and Gal [25], which are external sources. No quantity in the method is defined in terms of the target outcome, and no fitted parameter is renamed as a prediction. The few self-citations (e.g., [26], [36], [37], [40]) are peripheral and do not carry the central argument. Two concerns raised by the manuscript or by a critical reading are limitations, not circularity: (i) the number of MC samples T=20 is selected on the test sets using Fig. 5, which weakens the clean hold-out interpretation but does not make the reported IoU or CTR correlations equal to the selection criterion by construction; and (ii) Equations (2)-(3) are softmax-style entropy/mutual-information expressions applied to a multi-label two-sigmoid output, and no calibration of the resulting CTR uncertainty intervals is reported, which is a correctness/validation gap rather than a circular step. The annotation-protocol difference between radiologist A and B is acknowledged in the conclusion as a limitation and likewise does not reduce the evaluation to its inputs. Overall, the central claims rest on external benchmarks and independent data, so no circularity is found.
Assumptions & free parameters
free parameters (4)
- Number of MC dropout samples T =
20
- Dropout probability p in backbone and decoder =
0.5
- Spatial dropout rate before final output =
0.1
- Learning rate and batch size =
1e-4, 8
assumptions (4)
- standard math Dropout during training minimizes the KL divergence between the variational distribution and the posterior (Gal and Ghahramani, 2016)
- domain assumption The annotation protocols of radiologist A and radiologist B are consistent enough to compare segmentation masks across datasets
- domain assumption CTR computed from the widest diameters of the predicted heart and lung masks follows Dong et al. [9]
- domain assumption The re-annotated external test sets delineate the same anatomical boundaries as the training annotations
Cite this review
Pith. "Pith review of Bayesian Feature Pyramid Networks for Automatic Multi-Label Segmentation of Chest X-rays and Assessment of Cardio-Thoratic Ratio." pith.science (2026). https://pith.science/paper/JYSMMRN6
@misc{pith2026190802924,
author = {Pith},
title = {Pith review of: Bayesian Feature Pyramid Networks for Automatic Multi-Label Segmentation of Chest X-rays and Assessment of Cardio-Thoratic Ratio},
year = {2026},
howpublished = {\url{https://pith.science/paper/JYSMMRN6}},
note = {Machine review of arXiv:1908.02924}
}
read the original abstract
Cardiothoratic ratio (CTR) estimated from chest radiographs is a marker indicative of cardiomegaly, the presence of which is in the criteria for heart failure diagnosis. Existing methods for automatic assessment of CTR are driven by Deep Learning-based segmentation. However, these techniques produce only point estimates of CTR but clinical decision making typically assumes the uncertainty. In this paper, we propose a novel method for chest X-ray segmentation and CTR assessment in an automatic manner. In contrast to the previous art, we, for the first time, propose to estimate CTR with uncertainty bounds. Our method is based on Deep Convolutional Neural Network with Feature Pyramid Network (FPN) decoder. We propose two modifications of FPN: replace the batch normalization with instance normalization and inject the dropout which allows to obtain the Monte-Carlo estimates of the segmentation maps at test time. Finally, using the predicted segmentation mask samples, we estimate CTR with uncertainty. In our experiments we demonstrate that the proposed method generalizes well to three different test sets. Finally, we make the annotations produced by two radiologists for all our datasets publicly available.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
M. R. Arbabshirani, A. H. Dallal, C. Agarwal, A. Patel, and G. Moore. Accurate segmentation of lung fields on chest radiographs using deep convolutional networks. In Medi- cal Imaging 2017: Image Processing , volume 10133, page 1013305. International Society for Optics and Photonics,
work page 2017
-
[2]
A. Buslaev, A. Parinov, E. Khvedchenya, V . I. Iglovikov, and A. A. Kalinin. Albumentations: fast and flexible image aug- mentations. arXiv preprint arXiv:1809.06839, 2018. 4
arXiv 2018
-
[3]
A. Chaurasia and E. Culurciello. LinkNet: Exploiting En- coder Representations for Efficient Semantic Segmentation. CoRR, abs/1707.03718, 2017. 5, 6
arXiv 2017
-
[4]
C. Chen, Q. Dou, H. Chen, and P.-A. Heng. Semantic-aware generative adversarial nets for unsupervised domain adapta- tion in chest x-ray segmentation. In International Workshop on Machine Learning in Medical Imaging , pages 143–151. Springer, 2018. 2, 7
work page 2018
-
[5]
L.-C. Chen, Y . Zhu, G. Papandreou, F. Schroff, and H. Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In Proceedings of the Euro- pean conference on computer vision (ECCV) , pages 801– 818, 2018. 2
work page 2018
- [6]
-
[7]
W. Dai, N. Dong, Z. Wang, X. Liang, H. Zhang, and E. P. Xing. Scan: Structure correcting adversarial network for or- gan segmentation in chest x-rays. In Deep Learning in Med- ical Image Analysis and Multimodal Learning for Clinical Decision Support, pages 263–273. Springer, 2018. 1, 2
work page 2018
-
[8]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei- Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 2, 6 (a) ChestXray14 (b) JSRT (c) Montgomery (d) Schenzhen Figure 6: Examples of segmentation and uncertainty estimates for each of the test datas...
work page 2009
Show all 44 references
-
[9]
N. Dong, M. Kampffmeyer, X. Liang, Z. Wang, W. Dai, and E. Xing. Unsupervised domain adaptation for automatic es- timation of cardiothoracic ratio. In International Conference on Medical Image Computing and Computer-Assisted Inter- vention, pages 544–552. Springer, 2018. 2, 4, 7
2018
-
[10]
S. M. Dunlay, V . L. Roger, and M. M. Redfield. Epidemiol- ogy of heart failure with preserved ejection fraction. Nature reviews cardiology, 14(10):591, 2017. 1
2017
-
[11]
Eslami, S
M. Eslami, S. Tabarestani, S. Albarqouni, E. Adeli, N. Navab, and M. Adjouadi. Image to images translation for multi-task organ segmentation and bone suppression in chest x-ray radiography. arXiv preprint arXiv:1906.10089,
1906 arXiv
-
[12]
Gal and Z
Y . Gal and Z. Ghahramani. Dropout as a bayesian approxi- mation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050– 1059, 2016. 3
2016
-
[13]
X. Gao, X. Sun, Y . Zhang, M. Yan, G. Xu, H. Sun, J. Jiao, and K. Fu. An end-to-end neural network for road extraction from remote sensing imagery by multiple feature pyramid network. IEEE Access, 6:39401–39414, 2018. 2
2018
-
[14]
K. He, G. Gkioxari, P. Dollr, and R. Girshick. Mask r-cnn. In 2017 IEEE International Conference on Computer Vision (ICCV), 2017. 2
2017
-
[15]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- ing for image recognition. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 770–778, 2016. 2, 5
2016
-
[16]
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. Mobilenets: Effi- cient convolutional neural networks for mobile vision appli- cations. CoRR, abs/1704.04861, 2017. 5
2017 arXiv
-
[17]
J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation net- works. 2018. 5
2018
-
[18]
Huang, Z
G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 5
2017
-
[19]
Ioffe and C
S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32Nd International Conference on In- ternational Conference on Machine Learning - Volume 37 , ICML’15, pages 448–456, 2015. 5
2015
-
[20]
Jaeger, S
S. Jaeger, S. Candemir, S. Antani, Y .-X. J. W ´ang, P.-X. Lu, and G. Thoma. Two public chest x-ray datasets for computer- aided screening of pulmonary diseases. Quantitative imag- ing in medicine and surgery, 4(6):475, 2014. 6, 7
2014
-
[21]
Kendall, V
A. Kendall, V . Badrinarayanan, and R. Cipolla. Bayesian segnet: Model uncertainty in deep convolutional encoder- decoder architectures for scene understanding. arXiv preprint arXiv:1511.02680, 2015. 2, 3
2015 arXiv
-
[22]
Kirillov, R
A. Kirillov, R. Girshick, K. He, and P. Doll ´ar. Panoptic fea- ture pyramid networks. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 6399–6408, 2019. 2, 6
2019
-
[23]
Z. Li, Z. Hou, C. Chen, Z. Hao, Y . An, S. Liang, and B. Lu. Automatic cardiothoracic ratio calculation with deep learn- ing. IEEE Access, 7:37749–37756, 2019. 2
2019
-
[24]
T.-Y . Lin, P. Doll´ar, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017. 2, 3, 5, 6
2017
-
[25]
Mukhoti and Y
J. Mukhoti and Y . Gal. Evaluating bayesian deep learn- ing methods for semantic segmentation. arXiv preprint arXiv:1811.12709, 2018. 2, 3, 4
2018 arXiv
-
[26]
Rakhlin, A
A. Rakhlin, A. A. Shvets, A. A. Kalinin, A. Tiulpin, V . I. Iglovikov, and S. Nikolenko. Breast tumor cellular- ity assessment using deep neural networks. arXiv preprint arXiv:1905.01743, 2019. 2
1905 arXiv
-
[27]
Ronneberger, P
O. Ronneberger, P. Fischer, and T. Brox. U-net: Convo- lutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention , pages 234–241. Springer,
-
[28]
Ronneberger, P.Fischer, and T
O. Ronneberger, P.Fischer, and T. Brox. U-net: Convolu- tional networks for biomedical image segmentation. InMed- ical Image Computing and Computer-Assisted Intervention (MICCAI), volume 9351 of LNCS, pages 234–241, 2015. 5
2015
-
[29]
L. Saba, M. Biswas, V . Kuppili, E. C. Godia, H. S. Suri, D. R. Edla, T. Omerzu, J. R. Laird, N. N. Khanna, S. Mavrogeni, et al. The present and future of deep learning in radiology. European journal of radiology, 2019. 1
2019
-
[30]
M. B. Sandler, A. G. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. Mobilenetv2: Inverted residuals and linear bot- tlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4510–4520, 2018. 5
2018
-
[31]
Schmidhuber
J. Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61:85–117, 2015. 1
2015
-
[32]
S. S. Seferbekov, V . Iglovikov, A. Buslaev, and A. Shvets. Feature pyramid network for multi-class land segmentation. In CVPR Workshops, pages 272–275, 2018. 2, 3, 4
2018
-
[33]
Shiraishi, S
J. Shiraishi, S. Katsuragawa, J. Ikezoe, T. Matsumoto, T. Kobayashi, K.-i. Komatsu, M. Matsui, H. Fujita, Y . Kodera, and K. Doi. Development of a digital image database for chest radiographs with and without a lung nod- ule: receiver operating characteristic analysis of radio...
2000
-
[34]
Simonyan and A
K. Simonyan and A. Zisserman. Very deep convolu- tional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014. 5
2014 arXiv
-
[35]
J. C. Souza, J. O. B. Diniz, J. L. Ferreira, G. L. F. da Silva, A. C. Silva, and A. C. de Paiva. An automatic method for lung segmentation and reconstruction in chest x-ray using deep neural networks. Computer Methods and Programs in Biomedicine, 2019. 2
2019
-
[36]
Tiulpin, S
A. Tiulpin, S. Klein, S. Bierma-Zeinstra, J. Thevenot, E. Rahtu, J. van Meurs, E. H. Oei, and S. Saarakkala. Multi- modal machine learning-based knee osteoarthritis progres- sion prediction from plain radiographs and clinical data. arXiv preprint arXiv:1904.06236, 2019. 1
1904 arXiv
-
[37]
Tiulpin, J
A. Tiulpin, J. Thevenot, E. Rahtu, P. Lehenkari, and S. Saarakkala. Automatic knee osteoarthritis diagnosis from plain radiographs: A deep learning-based approach. Scien- tific reports, 8(1):1727, 2018. 1
2018
-
[38]
Ulyanov, A
D. Ulyanov, A. Vedaldi, and V . Lempitsky. Instance normal- ization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016. 2, 5
2016 arXiv
-
[39]
Van Ginneken, S
B. Van Ginneken, S. Katsuragawa, B. M. ter Haar Romeny, K. Doi, and M. A. Viergever. Automatic detection of ab- normalities in chest radiographs using local texture analy- sis. IEEE transactions on medical imaging, 21(2):139–149,
-
[40]
J. Wang, M. Knol, A. Tiulpin, F. Dubost, M. De Bruijne, M. Vernooij, H. Adams, M. A. Ikram, W. Niessen, and G. Roshchupkin. Grey matter age prediction as a biomarker for risk of dementia: A population-based study. BioRxiv, page 518506, 2019. 1
2019
-
[41]
X. Wang, Y . Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Sum- mers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and local- ization of common thorax diseases. In The IEEE Conference on Computer Vision and Pattern Recognition ...
-
[42]
Wessel, M
J. Wessel, M. P. Heinrich, J. von Berg, A. Franz, and A. Saal- bach. Sequential rib labeling and segmentation in chest x-ray using mask r-{cnn}. In International Conference on Medi- cal Imaging with Deep Learning – Extended Abstract Track, London, United Kingdom, 08–10 Jul 2019. 2
2019
- [43]
-
[44]
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia. Pyramid scene parsing network. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 5, 6
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.