Transformer-Based Wireless Capsule Endoscopy Bleeding Tissue Detection and Classification

Adarsh Ghimire; Basit Alawode; Divya Velayudhan; Shibani Hamza

REVIEW 3 major objections 5 minor 11 references

Transformer-Based Wireless Capsule Endoscopy Bleeding Tissue Detection and Classification

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single end-to-end transformer model detects and classifies bleeding in wireless capsule endoscopy frames, reporting 98.28% validation accuracy.

desk verdict A DETR challenge entry for WCE bleeding detection whose real numbers rest on a self-created validation split, with the official test-set scores omitted; the code and setup are solid, but the central generalization claim needs those missing numbers. read the letter →

arxiv 2412.19218 v1 pith:HWZSIH6H submitted 2024-12-26 cs.CV

Basit Alawode , Shibani Hamza , Adarsh Ghimire , Divya Velayudhan This is my paper · ORCID

classification cs.CV

keywords wirelesscapsuleendoscopybleedingdetectionDETRtransformerobjectimageclassificationAuto-WCEBleedGendeeplearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to show that a single end-to-end transformer model can handle both tasks needed to automate wireless capsule endoscopy review: classifying each frame as bleeding or non-bleeding, and drawing bounding boxes around bleeding regions. Building on the DETR architecture with a ResNet50 backbone, the authors fine-tune pretrained weights on the Auto-WCEBleedGen challenge training set. On their own 80:20 validation split they report classification accuracy of 98.28%, recall of 96.79%, and F1 of 98.37%, along with detection AP@0.5 of 0.7447 and mAP of 0.7328. The claim is that the same weights perform both detection and classification, removing the need for separate pipelines and hand-designed post-processing such as non-maximum suppression. If those validation numbers transfer to the challenge's independent test sets, the model could be a practical aid for clinicians reviewing thousands of frames per procedure.

What carries the argument

The load-bearing mechanism is DETR's set-based transformer architecture: a ResNet50 CNN produces two-dimensional features, a transformer encoder attends over them with positional encodings, and a decoder with a fixed set of learned object queries outputs a fixed-size set of predictions. A bipartite matching loss, combining the Hungarian algorithm with cross-entropy and IoU terms, assigns each prediction to a ground-truth box one-to-one, which removes hand-designed components such as anchor boxes and non-maximum suppression. Transfer learning from published DETR/ResNet50 weights makes fine-tuning feasible on only 2618 frames.

What would settle it

Run the released model on the official Auto-WCEBleedGen Test Sets 1 and 2 and compute classification accuracy and detection mAP; if these fall materially below 98.28% accuracy or 0.7328 mAP, the claimed transfer to unseen WCE data is refuted.

Watch

Extended reading notes

Core claim

The central claim is that a fine-tuned DETR (Detection Transformer), with ResNet50 feature extraction and a three-neuron feedforward head for bleed, non-bleed, and background classes, detects and classifies bleeding in capsule endoscopy frames in one forward pass. Using the Hungarian-based bipartite matching loss with weights of 1, 5, and 2 for the Hungarian matching, cross-entropy, and IoU terms, the model is trained end-to-end without anchor boxes or non-maximum suppression. On the Auto-WCEBleedGen v1 validation split it attains 98.28% classification accuracy, 96.79% recall, 98.37% F1, and detection metrics of AP@0.5 = 0.7447, mAP = 0.7328, and recall at 0.5:0.95 = 0.7706. A frame is called bleeding if any detected region has a bleeding probability above 0.5. This placed the entry third in the challenge.

Load-bearing premise

The reported metrics come from an 80:20 split of the challenge training set, and the paper assumes these numbers carry over to the independent test sets, for which it shows only qualitative sample images.

Editorial extensions

If this is right

A single model can perform both frame-level classification and region-level detection in one forward pass, simplifying the WCE analysis pipeline.
Fine-tuning a pretrained DETR on a few thousand medical images yields strong classification metrics, suggesting transfer learning is a viable route for small medical datasets.
The frame-level rule (bleeding if any region exceeds 0.5 probability) provides a simple, clinically interpretable decision boundary.
The reported detection mAP of 0.7328 indicates that while frame triage is accurate, precise localisation of bleeding regions is the harder sub-problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper reports no quantitative metrics on the official Test Sets 1 and 2, so the strongest claim currently rests on the self-created validation split; generalisation to the independent test sets remains an open question until those numbers are published.
The gap between 98% classification accuracy and 0.73 detection mAP suggests the model may be better suited to triage (flagging bleeding frames) than to precise boundary delineation, pointing to detection-specific heads or more annotated boxes as a likely next improvement.
Because the DETR framework is label-agnostic, the same fine-tuning recipe could be extended to other capsule-endoscopy lesions such as angiodysplasia or erosions by changing the output classes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Read the Auto-WCEBleedGen challenge report from Alawode et al. The bottom line: this is a clean application of DETR to wireless capsule endoscopy bleeding detection, with a three-neuron classifier appended to (or replacing) the head, and the only real numbers come from an 80:20 split of the challenge training set. That split is the problem. The official Test Sets 1 and 2, independently collected from more than 30 patients, appear only as qualitative figures. The abstract says they placed third in the challenge, so those test-set scores exist. Leaving them out makes the headline accuracy/F1/mAP unverifiable for new-patient generalization.

What the paper does well: the code is public, the training setup is described in enough detail to reproduce (optimizer, per-module learning rates, loss weights, 500 epochs), and the validation split is balanced. For a challenge report it is reasonably honest, and there is no circular reasoning—the metrics are not defined in terms of fitted constants from the same evaluation. The clinical motivation is legitimate and the figures are easy to follow.

Soft spots, in order. First, the missing official test-set numbers. Section 6 explicitly acknowledges use of the test data, so the authors had access. Without those numbers, the stated clinical efficacy—automating analysis of unseen WCE frames—is not supported by the evidence in this manuscript. Second, there are no baselines or ablations. It is impossible to know whether the DETR adaptation beats a simple ResNet classifier or an earlier CNN detector on this benchmark. Third, the classification head description is ambiguous: DETR outputs 80 classes; the authors say they "append" a three-neuron FFN, but they likely replace the classification head. A diagram or one sentence would fix it. Fourth, no confidence intervals or repeated runs. Minor, given the fixed dataset. Fifth, typos ("append to, to the output") and the table's recall@0.5:0.95 column is never explained in the text.

The stress-test note is exactly right: the self-created validation split measures intra-dataset fit, not cross-patient generalization. That is the load-bearing gap.

This paper is for the WCE challenge community and for anyone who wants a baseline DETR result on Auto-WCEBleedGen. It is not a methodological contribution. Still, it deserves a serious referee: the code is real, the problem is relevant, and the missing test-set metrics are a straightforward, mandatory revision. I would not desk reject it. Ask the authors for official test-set scores, a comparison with at least one non-transformer baseline, and a clarified head description, then accept or reject on those numbers.

Recommend reading group? Maybe, if someone in the group cares about capsule endoscopy AI. I wouldn't cite it in my own work yet.

Referee Report

3 major / 5 minor

Summary. The paper describes a DETR-based end-to-end model for bleeding and non-bleeding frame classification and bleeding-region detection in wireless capsule endoscopy (WCE) images, developed for the Auto-WCEBleedGen Version 1 challenge. The model uses a ResNet50 backbone, a transformer encoder-decoder for detection, and an appended feedforward layer for three-class classification (bleed, non-bleed, background). It is fine-tuned for 500 epochs on an 80:20 split of the challenge's 2618-frame training set. The authors report classification accuracy 98.28%, recall 96.79%, F1 98.37%, and detection AP@50 0.7447 / mAP 0.7328 on their self-created validation split, and state that this earned third place in the challenge. They also show qualitative detection and Grad-CAM examples from validation set and from the two official test sets, but no quantitative test-set metrics are reported.

Significance. If the reported validation-set numbers were confirmed on the independent official test sets, the paper would provide a useful demonstration that a single DETR-based model can jointly perform bleeding-region detection and frame-level classification for WCE, a task with genuine clinical workflow implications. The paper ships its code publicly, which is a concrete asset for reproducibility, and the end-to-end training without hand-designed components (anchors, NMS) is a clean design choice. However, the evidence currently presented does not establish generalization to unseen WCE data: the headline numbers are computed on a split of the challenge training set, while the independently collected test sets appear only as visual examples. The clinical-motivation claim therefore remains unquantified, and the third-place statement is not verifiable from the manuscript. The gap is fixable and the work is a reasonable challenge report, but as a journal submission the missing test-set evaluation is load-bearing.

major comments (3)

[Section 2.1 and Section 3 (Table 1, Figures 4–5)] The central quantitative claims are computed on an 80:20 split of the challenge training set, not on the independently collected test sets. Section 2.1 states that the test dataset contains frames from more than 30 patients, but Section 3 reports only sample detection images and Grad-CAM visualizations from Test Set 1 and Test Set 2, with no quantitative evaluation. The abstract's claim of '3rd place position' implies that official test-set metrics exist, but they are not reported or compared with Table 1. As written, the paper supports only intra-dataset fit, not the stated clinical claim of automating analysis of unseen WCE frames. The authors should add the official test-set metrics (classification accuracy/recall/F1 and detection AP/mAP and recall) and, if available, the challenge leaderboard scores.
[Section 3 (Table 1)] The reported metrics have no confidence intervals, no number of repeated runs, and no statistical significance assessment. Given that the validation split is balanced and relatively small (roughly 524 frames), a single run with high metrics could be unstable. The paper should report variance across at least a few seeds or provide a bootstrap confidence interval, especially for the detection mAP, which (at 0.7328) is materially lower than the classification F1 and may be sensitive to the exact split.
[Section 2.2.2 and Table 1] The detection results are reported without comparison to any baseline, the challenge's own evaluation protocol, or an ablation of the loss-weight choices (Hungarian 1, cross-entropy 5, IoU 2). Since the loss weights and the frame-level bleeding threshold of 0.5 are free parameters, the paper should at least show that the reported metrics are not strongly dependent on these choices, or justify them by reference to the official evaluation settings.

minor comments (5)

[Section 2, paragraph 4] The sentence 'we append to, to the output of DETR' contains a duplicated comma and should be rewritten, for example: 'we append a single-layer FFN with 3 neurons to the output of DETR.'
[Section 4, Discussion] The phrase 'our model yield itself to parallel processing' is ungrammatical; it should be 'lends itself to parallel processing'.
[Section 2.1] The sentence describing the test dataset — 'an independently collected WCE data containing bleeding and non-bleeding frames of more than 30 patients suffering from acute, chronic, and occult GI bleeding' — would be clearer as 'an independently collected WCE dataset containing' to match the singular 'data set' usage elsewhere.
[Section 3] The evaluation-metrics paragraph defines Recall and F1 only implicitly; for completeness, the authors should specify whether classification recall is per-class macro-averaged or balanced, and whether detection recall is computed at a fixed IoU threshold or averaged over 0.5:0.95 as implied by 'Recall (@0.5:0.95)'.
[References] Reference [8] is an Authorea preprint; since the challenge also has an official website, the authors should cite the final published challenge description if one exists, and ensure the dataset references [9] and [10] include the version and access date consistent with the manuscript.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the model is trained on an 80:20 split and evaluated on the held-out 20%; no reported quantity reduces to a fitted constant from the same evaluation, and no load-bearing self-citation chain exists.

full rationale

The paper's derivation chain is an empirical one: a DETR-based model is initialized with pretrained weights from the external DETR repository [6], fine-tuned on the 80% training split of the Auto-WCEBleedGen challenge set, and evaluated on the 20% held-out validation split. This is a standard train/validation evaluation, not a circular construction. The reported accuracy, F1, AP, and mAP numbers in Table 1 are computed on that held-out split; none of these quantities is defined in terms of a parameter fitted to the same validation set. The rule that a frame is classified as bleeding if at least one region exceeds probability 0.5 is an inference threshold, not a fitted constant used to manufacture the reported metric. The test-set results are shown only as visual samples (Figures 4 and 5) with no quantitative comparison to Table 1, which is a legitimate concern about external generalization, but a missing external benchmark is not circular reasoning. The references include no prior work by the present authors that is invoked as a load-bearing premise; DETR, ResNet, Hungarian matching, cross-entropy, GIoU, and AdamW are all external, and the challenge datasets are cited to the challenge organizers. There is no uniqueness theorem imported from the authors, no ansatz smuggled in via self-citation, and no renaming of a known result as a new derivation. The central claim is therefore a standard held-out evaluation of a fine-tuned public architecture, and no step in the claimed derivation reduces to its own input by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The report adds no invented entities. The free parameters are standard training hyperparameters and a decision threshold; the load-bearing assumptions are transferability of COCO-pretrained weights, annotation quality, and generalizability of the self-made validation split.

free parameters (4)

Loss weights (Hungarian, cross-entropy, IoU) = 1, 5, 2
Hand-selected in Section 2.2.2; they balance classification and box regression and shape the reported metrics.
Learning rates and weight decay = 1e-6, 1e-5, 5e-5, weight decay 1e-4
Chosen for fine-tuning in Section 2.2.3 without stated justification or search procedure.
Fine-tuning epochs = 500
Fixed in Section 2.2.3; no early stopping or schedule is reported.
Frame-level bleeding threshold = 0.5
Used in Section 2 to convert region detections into a frame classification; affects reported accuracy and recall.

assumptions (4)

domain assumption Pretrained DETR weights trained on COCO provide a valid initialization for WCE imagery.
Invoked in Section 2.2.3; if the pretrained representation does not transfer, the fine-tuning numbers would not follow.
domain assumption The challenge dataset's bleeding/non-bleeding labels and bounding boxes are medically valid.
Used as ground truth in Section 2.1; the metrics inherit any annotation errors.
standard math The Hungarian matching, cross-entropy, and GIoU losses are correctly implemented and appropriate.
Adopted without re-derivation from DETR and cited works in Section 2.2.2.
domain assumption The self-created 80:20 split is representative of the independent test distribution.
Section 2.1; the paper's efficacy claim assumes validation performance transfers to the official test sets, but no quantitative test metrics are reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Transformer-Based Wireless Capsule Endoscopy Bleeding Tissue Detection and Classification." pith.science (2026). https://pith.science/paper/HWZSIH6H

@misc{pith2026241219218,
  author       = {Pith},
  title        = {Pith review of: Transformer-Based Wireless Capsule Endoscopy Bleeding Tissue Detection and Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HWZSIH6H}},
  note         = {Machine review of arXiv:2412.19218}
}

read the original abstract

Informed by the success of the transformer model in various computer vision tasks, we design an end-to-end trainable model for the automatic detection and classification of bleeding and non-bleeding frames extracted from Wireless Capsule Endoscopy (WCE) videos. Based on the DETR model, our model uses the Resnet50 for feature extraction, the transformer encoder-decoder for bleeding and non-bleeding region detection, and a feedforward neural network for classification. Trained in an end-to-end approach on the Auto-WCEBleedGen Version 1 challenge training set, our model performs both detection and classification tasks as a single unit. Our model achieves an accuracy, recall, and F1-score classification percentage score of 98.28, 96.79, and 98.37 respectively, on the Auto-WCEBleedGen version 1 validation set. Further, we record an average precision (AP @ 0.5), mean-average precision (mAP) of 0.7447 and 0.7328 detection results. This earned us a 3rd place position in the challenge. Our code is publicly available via https://github.com/BasitAlawode/WCEBleedGen.

Figures

Figures reproduced from arXiv: 2412.19218 by the authors.

**Figure 2.** Sample detection images from the validation set. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Corresponding Grad-CAM visualization on the Sample validation set of Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Sample detection images and corresponding Grad-CAM visualization from the test [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Sample detection images and corresponding Grad-CAM visualization from the test [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 7 canonical work pages

[1]

End-to-end object detection with transformers

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kir- illov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision, pages 213–229. Springer, 2020

work page 2020
[2]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

2016
[3]

Harold W. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83–97, 1955

work page 1955
[4]

Deep Learning

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press,
[5]

Generalized intersection over union: A metric and a loss for bounding box regression

Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 658–666, 2019

work page 2019
[6]

Detr: End-to-end object detection with transformers

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kir- illov, and Sergey Zagoruyko. Detr: End-to-end object detection with transformers. https://github.com/facebookresearch/detr, 2020. Accessed: 2024-12-24

work page 2020
[7]

Decoupled weight decay regularization

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Inter- national Conference on Learning Representations (ICLR), 2018

work page 2018
[8]

Auto- wcebleedgen version v1 and v2: Challenge, datasets and evaluation

Palak Handa, Divyansh Nautiyal, Deepti Chhabra, Manas Dhir, Anushka Saini, Shreshtha Jha, Harshita Mangotra, Nishu Pandey, Advika Thakur, et al. Auto- wcebleedgen version v1 and v2: Challenge, datasets and evaluation. Authorea Preprints, 2024. doi: 10.22541/essoar.171007121.19572474/v1

work page arXiv 2024

Show all 11 references

[9]

Wcebleedgen: A wireless capsule endoscopy dataset containing bleeding and non-bleeding frames, November 2023

Palak, Harshita Mangotra, Divyansh Nautiyal, Jyoti Dhatarwal, and Nidhi Gooel. Wcebleedgen: A wireless capsule endoscopy dataset containing bleeding and non-bleeding frames, November 2023. URL https://doi.org/10.5281/zenodo. 10156571

2023 doi
[10]

Autowcebleedgen-test dataset (improved), February 2024

Palak Handa, Nishu Pandey, Divyansh Nautiyal, Nidhi Goel, and Deepak Gunjan. Autowcebleedgen-test dataset (improved), February 2024. URL https://doi.org/ 10.5281/zenodo.10642779

2024 doi
[2016]

http://www.deeplearningbook.org

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

End-to-end object detection with transformers

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kir- illov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision, pages 213–229. Springer, 2020

work page 2020

[2] [2]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

2016

[3] [3]

Harold W. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83–97, 1955

work page 1955

[4] [4]

Deep Learning

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press,

[5] [5]

Generalized intersection over union: A metric and a loss for bounding box regression

Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 658–666, 2019

work page 2019

[6] [6]

Detr: End-to-end object detection with transformers

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kir- illov, and Sergey Zagoruyko. Detr: End-to-end object detection with transformers. https://github.com/facebookresearch/detr, 2020. Accessed: 2024-12-24

work page 2020

[7] [7]

Decoupled weight decay regularization

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Inter- national Conference on Learning Representations (ICLR), 2018

work page 2018

[8] [8]

Auto- wcebleedgen version v1 and v2: Challenge, datasets and evaluation

Palak Handa, Divyansh Nautiyal, Deepti Chhabra, Manas Dhir, Anushka Saini, Shreshtha Jha, Harshita Mangotra, Nishu Pandey, Advika Thakur, et al. Auto- wcebleedgen version v1 and v2: Challenge, datasets and evaluation. Authorea Preprints, 2024. doi: 10.22541/essoar.171007121.19572474/v1

work page arXiv 2024