Sequential Rib Labeling and Segmentation in Chest X-Ray using Mask R-CNN

Astrid Franz; Axel Saalbach; Jens von Berg; J\"oran Wessel; Mattias P. Heinrich

REVIEW 3 major objections 5 minor 8 references

Sequential Rib Labeling and Segmentation in Chest X-Ray using Mask R-CNN

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports the first approach to simultaneously segment and anatomically label ribs in chest X-rays, with mean Dice of 0.846 for bounding boxes and 0.733 for masks.

desk verdict First genuine combination of rib segmentation and labeling in chest X-ray, but the reported numbers are compromised by anchor boxes fitted to the whole dataset before cross-validation. read the letter →

arxiv 1908.08329 v1 pith:T2SV2Z7Z submitted 2019-08-22 eess.IV

Jöran Wessel , Mattias P. Heinrich , Jens von Berg , Astrid Franz , Axel Saalbach This is my paper · ORCID

classification eess.IV

keywords ribsegmentationlabelingchestX-rayMaskR-CNNinstanceanatomicalsequentialprocessinganchorboxes

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that one neural network can do what previously required separate tools: cut out individual rib shadows from a chest X-ray and simultaneously say which anatomical rib each shadow is. The authors modify an instance-segmentation network so that each rib gets its own branch and the branch for a rib also receives the segmentation of the rib above it as a hint; anchor boxes are computed from the data by clustering actual rib locations. In five-fold cross-validation on 174 images, the method achieves mean overlap scores of 0.846 for rib-box detection and 0.733 for rib-mask segmentation, improving on the unmodified network by more than 5% and 23%, respectively. If this holds, radiologists and automated pipelines could obtain labeled rib anatomy in a single fast pass, relevant for rib counting, quality assessment, and localizing findings.

What carries the argument

The load-bearing idea is sequential, rib-specific instance segmentation. For each label 1 through 9, a Mask R-CNN branch is trained with its own set of anchor boxes; these boxes come from Mean Shift clustering of all ground-truth rib boxes normalized to image size, producing 30 prototypical locations. For ribs 2 to 9, the predicted mask of the rib above is inserted as the third input channel, with the grayscale X-ray replicated in the other two channels, so the network can use the upper neighbor's shape and position to disambiguate the current rib. That sequential context is what gives the labels anatomical consistency rather than just separating arbitrary instances.

What would settle it

On a hold-out set of posterior-anterior chest X-rays with rib masks produced from CT bone segmentations, retrain and evaluate per-label Dice; if the sequential channel ceases to improve labeling accuracy over the non-sequential baseline, or if ribs 7–9 labels are frequently confused, the central claim is not supported.

Watch

Extended reading notes

Core claim

The central claim is that a Mask R-CNN, split into rib-specific networks and fed the segmentation of the upper neighboring rib as contextual input, is the first method to both segment ribs and assign anatomical labels in chest radiographs. The reported results—mean Dice 0.846 for bounding-box detection and 0.733 for segmentation on a five-fold cross-validation of 174 posterior-anterior chest X-rays—are said to be comparable to or better than published methods that only segment, and an improvement of over 5% and 23% over the base Mask R-CNN implementation. The authors also report higher accuracy, sensitivity, and specificity than an atlas-based rib-detection baseline. The claim matters because rib labeling and rib segmentation have previously been separate problems, and solving them together removes a preprocessing step and enables automatic rib counting for inhalation assessment.

Load-bearing premise

The accuracy of the hand-contoured rib masks in the 174 training images, and the consistent visibility of ribs 1–9, are assumed; errors there propagate into every anchor, segmentation target, and reported Dice value.

Editorial extensions

If this is right

A single forward pass through this network yields both rib masks and rib labels, so chest X-ray pipelines could replace separate bone-suppression and rib-labeling stages.
The large jump from the base architecture suggests that domain-specific anchors and sequential context are the main levers for instance segmentation of repetitive anatomy, not just more training data.
The reported speed advantage over atlas-based registration opens the possibility of real-time rib counting for assessing inspiratory effort in bedside or screening settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The upper-neighbor input channel is a transferable design: any elongated, serially ordered anatomy—vertebrae, teeth, bowel segments—could be labeled by feeding the previous segment's mask back into the next detection head.
Because the reference masks cover only visible rib shadows, the reported Dice measures fidelity to visible anatomy rather than bone recovery; a CT-derived ground truth would likely be a sterner test and might reveal systematic under-segmentation.
The normalization of anchor boxes by image size implies the same architecture should carry over to X-rays of different resolutions without retuning the proposal geometry, as long as the anatomy is similarly positioned in the frame.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is that they do rib segmentation and anatomical labeling in one pass, which nobody else seems to have done for chest X-rays. The per-rib sequential networks using the upper rib's mask as an extra input channel is a sensible idea, and they credit Lessmann et al. for the sequential concept. That part is worth paying attention to.

The evaluation is the problem. The paper computes 30 anchor boxes by Mean Shift clustering from all ground truth bounding boxes in the entire 174-image dataset, then runs five-fold cross-validation. The test fold's rib locations and sizes therefore enter the proposal geometry of the trained model. That is exactly the kind of circularity reviewers are supposed to catch. It does not invalidate the method, but it does mean the reported Dice values (0.846 detection, 0.733 segmentation) and the claimed 5%/23% gains over the original Mask R-CNN are likely optimistic, perhaps substantially so. The comparison to Candemir et al. also uses accuracy/sensitivity/specificity on what may be a different dataset, and there is no comparison to Mader et al., which is the relevant labeling baseline. Those are soft spots that a full paper could address with a properly split anchor calculation and head-to-head evaluation.

The paper is an extended abstract, so missing details like training hyperparameters and exact fold splits are forgivable, but the anchor leak is not a detail. That said, the central architectural contribution is coherent and the first-to-combine claim holds up. The hand-contoured annotations from von Berg et al. are a reasonable ground truth for visible rib shadows. The work is useful for bone suppression, quality assessment, and report localization, even if it is a niche task.

My take: the method deserves serious referee attention, but not on the strength of these numbers. A revised version should recompute anchors per training fold only and report per-fold variability. If the improvements shrink, the paper still has a publishable core—just a weaker quantitative claim. I would not cite the current numbers in my own work, but I would point to the method idea.

Recommendation: send it to peer review, with the evaluation protocol as the primary required revision.

Referee Report

3 major / 5 minor

Summary. The paper presents an adaptation of Mask R-CNN for simultaneous detection, anatomical labeling, and segmentation of ribs in PA chest X-ray images. The method introduces three extensions: rib-specific networks, dedicated anchor boxes computed by Mean Shift clustering from ground-truth bounding boxes, and a sequential processing scheme in which the segmentation of the upper neighbor rib is provided as an additional input channel for ribs 2-9. The authors report a five-fold cross-validation on 174 images, achieving mean Dice values of 0.846 for bounding-box detection and 0.733 for segmentation, and they claim improvements of more than 5% and 23% over the original Mask R-CNN implementation, as well as better performance than an atlas-based rib detection method.

Significance. If the reported results hold, this would be the first method to provide both rib segmentation and anatomical labeling in chest radiographs, which has practical value for rib counting, report generation, and follow-up correspondence. The paper builds on a publicly available Mask R-CNN implementation, uses a clearly described dataset, and reports quantitative results from cross-validation. The strengths are the explicit architectural extensions (rib-specific anchors and sequential neighbor-mask input) and the direct comparison to a baseline implementation. However, the central empirical claims depend on the soundness of the evaluation protocol, and the current manuscript leaves two methodological points unresolved: anchor-box information leakage across cross-validation folds, and the training/inference protocol for the sequential network input. These issues must be addressed before the reported gains can be considered reliable.

major comments (3)

[Section 2 (Data and Method) and Section 3 (Results)] The anchor boxes are computed by Mean Shift clustering from all ground-truth bounding boxes of the entire dataset, as stated in Section 2. In the five-fold cross-validation of Section 3, the ground-truth boxes of the test fold therefore contribute to the anchor geometry used by the model evaluated on that fold. Because the modified RPN determines shifts directly from these anchors without non-maximum suppression, the anchors are not a neutral training hyperparameter; they encode test-fold rib locations and sizes. This introduces a favorable bias of unknown magnitude into the reported Dice values (0.846 detection, 0.733 segmentation) and into the claimed >5% and >23% improvements over the original implementation. The authors should recompute the anchor boxes using only each training fold's ground truth, or otherwise demonstrate that the evaluation is insensitive to this leakage.
[Section 2, paragraph on sequential processing] The text states that for ribs 2 to 9, 'the output of the segmentation of the above rib is used for the third channel,' but it does not specify whether this output is the predicted mask from the network or the ground-truth mask during training. If ground-truth masks are used in training and predicted masks in testing, the network sees perfect upper-rib segmentations during training, creating a train-test mismatch that can inflate the reported segmentation accuracy. If predicted masks are used, the propagation order and the handling of errors in upstream ribs need to be described. This ambiguity is load-bearing because the sequential input is a central novelty of the method.
[Section 3, comparison with Candemir et al.] The comparison with the atlas-based method of Candemir et al. reports accuracy, sensitivity, and specificity values, but the text does not define the units, the reference standard, or the spatial extent over which these metrics are computed (e.g., pixel-level, rib-level, or image-level). Without a precise definition, and without confidence intervals or a statistical significance test, the statement that the proposed algorithm 'achieves better results in all categories' is not supported. The authors should specify the evaluation protocol used for this comparison and report variability across the cross-validation folds.

minor comments (5)

[Abstract and Section 2] The abstract describes the images as 'anterior-posterior' while Section 2 states 'posterior-anterior'; the terminology should be made consistent.
[Section 3, Table 1] The table heading 'Detection' should clarify that the reported value is the Dice coefficient for bounding boxes, and the spacing in '0 .841' should be corrected.
[Section 3, baseline comparison] The comparison to the 'original implementation' of Mask R-CNN would be more informative if the anchor configuration and training details of that baseline were described, since the claimed improvement depends on the baseline setup.
[Section 4, Conclusion] The claim of 'very fast run-times that enable realtime analysis' is not supported by any timing measurement in the paper; a statement of inference time per image would be needed.
[References] The reference for He et al. contains a formatting error: 'InComputer Vision (ICCV)' should be 'In Computer Vision (ICCV)'.

Circularity Check

1 steps flagged · score 6.0 of 10

Anchor boxes computed from all ground-truth boxes leak test-fold rib locations into five-fold cross-validation, inflating the reported Dice improvements.

fitted input called prediction [Section 2 (Data and Method, dedicated anchor boxes) and Section 3 (Results, five-fold cross-validation)]
"In this experiment, the anchor boxes have been estimated using the Mean Shift algorithm (Comaniciu and Meer, 2002). Therefore, 30 cluster boxes were computed from all ground truth bounding boxes of the entire dataset for all labels. ... For an evaluation of the proposed architecture, a five-fold cross-validation was performed."

The anchor boxes are a fitted parameter, and they are fitted to all ground-truth bounding boxes, including those of every fold's validation images, before the five-fold cross-validation is run. Because the modified RPN 'determines shifts on the basis of anchor boxes' without non-maximum suppression, the proposal locations and scales for a test image are derived from that image's own ground-truth boxes via the cluster centers. The reported detection Dice (0.846) and segmentation Dice (0.733), and the claimed more-than-5% and 23% improvements over the original implementation, therefore are not independent predictions: a load-bearing part of the bounding-box result is built from the very labels used for evaluation.

full rationale

The paper's contributions are empirical; there is no equation-level derivation to reduce. The sequential rib-labeling scheme is a legitimate architectural contribution, and the comparison with Candemir et al. uses external metrics. The central quantitative claim, however, rests on a five-fold cross-validation in which the anchor boxes, an active component of the modified RPN, are computed from all ground-truth bounding boxes of the entire dataset, including each validation fold. This is a concrete case of a fitted input being used to 'predict' closely related quantities (bounding boxes and, indirectly, masks), and it is load-bearing because the claimed improvements over the original Mask R-CNN are measured against this setup. The hand-contoured dataset is not itself circular: it is external annotation from prior work and is used as ground truth, not as the source of the derived claim. No self-citation chain forces the central result. The score is 6 rather than higher because the mask predictions still require the network to produce segmentation output and because the anchors are only one component of the architecture; the reported numbers are partially, not entirely, determined by construction.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The method and its evaluation rely on a private hand-annotated dataset, a transfer assumption for Mask R-CNN, and a sequential processing design whose error accumulation is not analyzed. The only fitted numeric component reported is the set of anchor boxes derived from ground truth boxes; this derivation uses the entire dataset and is the main source of leakage and circularity.

free parameters (1)

Anchor box clusters (mean shift) = 30
The number and positions of proposal anchor boxes are fitted to ground truth bounding boxes from the entire dataset, which introduces test-fold information into the detection prior.

assumptions (5)

domain assumption Hand-contoured rib masks in the von Berg et al. (2016) dataset are accurate and reflect all visible rib shadows.
These masks serve as ground truth for training, anchor derivation, and Dice evaluation; annotation errors would propagate to every reported number.
domain assumption Ribs 1 through 9 are consistently visible in all 174 PA chest X-rays and can be assigned stable anatomical labels.
The analysis and evaluation are restricted to these ribs; inconsistent visibility would break the labeling task and the cross-validation results.
domain assumption Mask R-CNN with ResNet50 and FPN, pretrained on natural images, transfers to grey-value chest X-ray anatomy.
The paper relies on the public Mask R-CNN implementation and standard feature extractors without discussing domain adaptation; transferability is assumed.
domain assumption Feeding the segmentation of the upper neighbor rib as an additional input channel does not accumulate errors across the rib sequence.
The sequential design assumes that upstream segmentation errors will not systematically degrade downstream rib predictions, but no error-propagation analysis is provided.
domain assumption Mean shift clustering of ground truth boxes yields representative anchor priors for all rib locations and image sizes.
Anchor design is based on cluster centers of normalized boxes; if clusters are unrepresentative, region proposals and final detections degrade.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Sequential Rib Labeling and Segmentation in Chest X-Ray using Mask R-CNN." pith.science (2026). https://pith.science/paper/T2SV2Z7Z

@misc{pith2026190808329,
  author       = {Pith},
  title        = {Pith review of: Sequential Rib Labeling and Segmentation in Chest X-Ray using Mask R-CNN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T2SV2Z7Z}},
  note         = {Machine review of arXiv:1908.08329}
}

read the original abstract

Mask R-CNN is a state-of-the-art network architecture for the detection and segmentation of object instances in the computer vision domain. In this contribution, it is used to localize, label and segment individual ribs in anterior-posterior chest X-ray images. For this purpose, several extensions have been made to the original architecture, in order to address the specific challenges of this application. This includes the use of rib specific networks, facilitating dedicated anchor boxes sampled from a training set, as well as a sequential processing of all ribs. Here, the segmentation result of the upper neighbor rib is used as additional input to the network. This approach is the first addressing both rib segmentation and anatomical labeling in chest radiographs. The results are comparable or even better than existing methods aiming only at segmentation.

Figures

Figures reproduced from arXiv: 1908.08329 by the authors.

**Figure 1.** Application of the adapted Mask R-CNN algorithm to a chest X-ray image. The ground truth is shown on the left and the prediction on the right. For illustration purposes, next to the segmentation, also the bounding boxes and the rib labels are given [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 6 canonical work pages

[1]

Atlas-based rib-bone detection in chest X -rays

Sema Candemir, Stefan Jaeger, Sameer Antani, Ulas Bagci, Les R Folio, Ziyue Xu, and George Thoma. Atlas-based rib-bone detection in chest X -rays. Computerized Medical Imaging and Graphics, 51: 0 32--39, 2016

work page 2016
[2]

Mean shift: A robust approach toward feature space analysis

Dorin Comaniciu and Peter Meer. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24 0 (5): 0 603--619, 2002

2002
[3]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770--778, 2016

2016
[4]

Mask R-CNN

Kaiming He, Georgia Gkioxari, Piotr Doll \'a r, and Ross Girshick. Mask R-CNN . In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 2980--2988. IEEE, 2017

work page 2017
[5]

Iterative fully convolutional neural networks for automatic vertebra segmentation and identification

Nikolas Lessmann, Bram van Ginneken, Pim A de Jong, and Ivana I s gum. Iterative fully convolutional neural networks for automatic vertebra segmentation and identification. Medical Image Analysis, 53: 0 142--155, 2019

work page 2019
[6]

Feature pyramid networks for object detection

Tsung-Yi Lin, Piotr Doll \'a r, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge J Belongie. Feature pyramid networks for object detection. In CVPR, volume 1, page 4, 2017

work page 2017
[7]

Localization and labeling of posterior ribs in chest radiographs using a CRF -regularized FCN with local refinement

Alexander Oliver Mader, Jens von Berg, Alexander Fabritz, Cristian Lorenz, and Carsten Meyer. Localization and labeling of posterior ribs in chest radiographs using a CRF -regularized FCN with local refinement. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 562--570. Springer, 2018

work page 2018
[8]

A novel bone suppression method that improves lung nodule detection

Jens von Berg, Stewart Young, Heike Carolus, Robin Wolz, Axel Saalbach, Alberto Hidalgo, Ana Gim \'e nez, and Tom \'a s Franquet. A novel bone suppression method that improves lung nodule detection. International Journal of Computer Assisted Radiology and Surgery, 11 0 (4): 0 641--655, 2016

work page 2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Atlas-based rib-bone detection in chest X -rays

Sema Candemir, Stefan Jaeger, Sameer Antani, Ulas Bagci, Les R Folio, Ziyue Xu, and George Thoma. Atlas-based rib-bone detection in chest X -rays. Computerized Medical Imaging and Graphics, 51: 0 32--39, 2016

work page 2016

[2] [2]

Mean shift: A robust approach toward feature space analysis

Dorin Comaniciu and Peter Meer. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24 0 (5): 0 603--619, 2002

2002

[3] [3]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770--778, 2016

2016

[4] [4]

Mask R-CNN

Kaiming He, Georgia Gkioxari, Piotr Doll \'a r, and Ross Girshick. Mask R-CNN . In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 2980--2988. IEEE, 2017

work page 2017

[5] [5]

Iterative fully convolutional neural networks for automatic vertebra segmentation and identification

Nikolas Lessmann, Bram van Ginneken, Pim A de Jong, and Ivana I s gum. Iterative fully convolutional neural networks for automatic vertebra segmentation and identification. Medical Image Analysis, 53: 0 142--155, 2019

work page 2019

[6] [6]

Feature pyramid networks for object detection

Tsung-Yi Lin, Piotr Doll \'a r, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge J Belongie. Feature pyramid networks for object detection. In CVPR, volume 1, page 4, 2017

work page 2017

[7] [7]

Localization and labeling of posterior ribs in chest radiographs using a CRF -regularized FCN with local refinement

Alexander Oliver Mader, Jens von Berg, Alexander Fabritz, Cristian Lorenz, and Carsten Meyer. Localization and labeling of posterior ribs in chest radiographs using a CRF -regularized FCN with local refinement. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 562--570. Springer, 2018

work page 2018

[8] [8]

A novel bone suppression method that improves lung nodule detection

Jens von Berg, Stewart Young, Heike Carolus, Robin Wolz, Axel Saalbach, Alberto Hidalgo, Ana Gim \'e nez, and Tom \'a s Franquet. A novel bone suppression method that improves lung nodule detection. International Journal of Computer Assisted Radiology and Surgery, 11 0 (4): 0 641--655, 2016

work page 2016