MARIO: A Mixed Annotation Framework For Polyp Segmentation

Haoyang Li; Jun Wei; Yiwen Hu; Zhen Li

REVIEW 3 major objections 5 minor 15 references

MARIO: A Mixed Annotation Framework For Polyp Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MARIO mixes pixel, box, polygon, scribble, and point annotations in one transformer, reporting 85.8% weighted Dice on five polyp benchmarks, beating the best fully supervised rival by 1.0%.

desk verdict Mixed-supervision idea is real and the benchmark table is decent, but the headline comparison is confounded by training-data volume and the paper needs a matched-data baseline before the central claim is credible. read the letter →

arxiv 2501.10957 v2 pith:W5GYFTY5 submitted 2025-01-19 cs.CV cs.AI

Haoyang Li , Yiwen Hu , Jun Wei , Zhen Li This is my paper · ORCID

classification cs.CVcs.AI

keywords polypsegmentationmixedsupervisionweaklysupervisedlearningcolonoscopytransformerannotationefficiencymedicalimage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Polyp segmentation normally requires pixel-level masks, which are expensive to produce and limit dataset size. MARIO is a single transformer model that trains on five annotation types at once — pixel, box, polygon, scribble, and point — each paired with a loss designed to handle its noise. The paper reports that this mixed-supervision approach reaches the highest weighted-average performance on five benchmarks, with 85.8% Dice and 78.3% IoU, beating the best fully supervised model by 1.0% Dice. The intended payoff is that clinics can label data with cheaper annotation styles and still obtain competitive segmentation.

What carries the argument

The engine of MARIO is the combined loss $L_{total}=L_{pixel}+L_{polygon}+L_{box}+L_{scribble}+L_{points}$, with a distinct mechanism per annotation type. Pixel- and polygon-level annotations use the standard BCE plus Dice loss. Box annotations are supervised through a mask-to-box ($M2B$) transform, which projects a predicted mask into row and column vectors and then back-projects a box-shaped mask, letting the model learn location without being misled by box shape. Scribble annotations use an uncertainty loss $\min(-\log P_s, -\log(1-P_s))$ that keeps the model from becoming overconfident in unlabeled pixels. Point annotations use a consistency loss computed as the MSE between predictions on an image and on its 90-degree rotated version. The backbone is a Pyramid Vision Transformer (PVTv2-B2).

What would settle it

Run a fully supervised baseline (for example, CAFE-Net or Polyp-Pvt) with the same backbone and the same training protocol as MARIO, but give it both the 1,451 pixel-level images and the weakly labeled images MARIO uses. If its weighted-average Dice on the five test sets reaches or exceeds 85.8%, the claim that mixed-supervision design is the cause is falsified.

Watch

Extended reading notes

Core claim

MARIO claims that a transformer-based segmentation model can learn effectively from a mix of five annotation types, and that this mixed supervision outperforms full pixel-level supervision. On the five test datasets (ColonDB, Kvasir, ClinicDB, EndoScene, ETIS), its weighted-average Dice is 85.8% and IoU is 78.3%, exceeding the best fully supervised competitor CAFE-Net by 1.0% and 0.6%, respectively. The paper attributes this to the complementary strengths of the annotation types and to specially designed losses: dense supervision for pixel and polygon labels, a mask-to-box transformation for box labels, an uncertainty loss for scribble labels, and a rotation-consistency loss for point labels. The conclusion is that mixed-annotation training can simultaneously improve accuracy, expand usable data, and reduce labeling cost.

Load-bearing premise

The paper's central premise is that mixed supervision itself drives the improvement, but it is never isolated from the much larger volume of weakly labeled training data used only by MARIO; without a same-data baseline, the gain could be entirely due to more training images.

Editorial extensions

If this is right

If the results hold, large weakly labeled polyp datasets that are currently discarded become usable training material.
Clinicians could annotate with boxes, scribbles, or points instead of pixel masks and still expect segmentation accuracy on par with fully supervised models.
The per-annotation loss design offers a recipe for other medical segmentation problems where annotation types are heterogeneous.
Low-cost annotation could expand polyp screening programs in settings where expert pixel annotation is unavailable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not isolate the mixed-supervision mechanism from the larger training set: MARIO adds more than 80,000 weakly labeled images on top of the 1,451 pixel-level images the baselines use, so the reported gain could stem from data volume alone.
A direct test would train CAFE-Net or another fully supervised baseline with the same pool of weakly labeled images; if the gap disappears, the claim that mixed supervision causes the improvement would be falsified.
The rotation-consistency loss for point labels suggests a general equivariance principle; using several random rotations and flips might further stabilize point-supervised training.
The five-benchmark weighted average treats the five test sets by image count; a per-dataset analysis would show whether the headline gain is concentrated in one dataset (e.g., ETIS) rather than uniform.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper has a genuinely plausible idea: instead of forcing every dataset into pixel-level masks, train one polyp segmenter on whatever annotation type is available (box, polygon, scribble, point, and pixel). That is new for this specific task, and the engineering is sensible. Each loss is borrowed from prior work—mask-to-box, uncertainty for scribbles, rotation consistency—but combining them in one framework with five data sources is a reasonable contribution. The benchmark numbers are also in the right ballpark; the model does beat fully supervised methods on average, and the gain on ETIS is notable. The problem is that the central claim is not actually tested. Section 3.1 says the fully supervised baselines train on 1,451 pixel-level images. MARIO additionally trains on 33,884 box-annotated frames, 49,136 SUN-SEG frames split into polygon and scribble, and 1,412 point-annotated images. That is roughly 85,000 extra weakly labeled samples. With that much additional data, a 1.0% Dice improvement could come from sheer volume, not from the mixed-supervision mechanism. The paper never compares MARIO against a fully supervised model trained on the same weak data, nor does it run MARIO with only the 1,451 pixel images. The ablation in Table 2 only toggles loss terms while keeping all the weak data present, so it cannot separate data volume from the method. That is a load-bearing gap, not a minor omission. The small effect size also matters: 1.0% Dice on a weighted average, with no error bars, is a thin margin to hang a causal story on. There are also smaller issues: the paper names the second-best method "CAFA-Net" in the text but "CAFE-Net" in Table 1, and the notation for the total loss is a simple sum without weighting, which is fine but under-discussed. What is good: the writing is clear, the experimental setup is mostly described with enough detail to reproduce (datasets, splits, optimizer, iterations), and the idea of expanding usable annotation types is practically motivated. If the authors add a matched-data baseline—say, training MARIO on the pixel-only set and training a strong fully supervised baseline on the same weak data—the result would be much more convincing. The missing comparison is fixable, which makes this a solid borderline paper rather than a dead one. For a reading group, it is a useful case study in confounding in medical imaging benchmarks. I would not cite the headline claim until the data-volume issue is resolved. But I would send it to peer review: the core idea is relevant, the evaluation is extensive for the claim it makes, and the key weakness is something reviewers can reasonably demand be fixed. A serious referee should ask for the controlled experiment and probably some error bars, but the paper deserves the round.

Referee Report

3 major / 5 minor

Summary. The paper proposes MARIO, a polyp segmentation framework that trains a PVTv2-based model with five annotation types: pixel, polygon, box, scribble, and point. Each annotation type is paired with a tailored loss; the total loss is a sum of the individual losses. The authors report that MARIO achieves a weighted-average Dice of 85.8% and IoU of 78.3% across five polyp benchmarks, surpassing fully supervised baselines such as CAFE-Net. A small ablation in Table 2 shows that adding the uncertainty and consistency losses improves performance. The paper concludes that mixed supervision reduces labeling cost and improves segmentation accuracy.

Significance. If the central claim is supported, MARIO would be a useful practical contribution: it would show that a single model can exploit heterogeneous, cheap annotations from multiple public datasets and outperform models trained only on pixel-level labels. The paper also proposes a simple uncertainty loss for scribble supervision and a rotation-based consistency loss for point supervision. However, the paper's significance is contingent on demonstrating that the improvement comes from the mixed-supervision mechanism rather than from the much larger training set. The evaluation is on standard public benchmarks and the implementation details are mostly reproducible, but the missing controlled comparison is a serious weakness that must be addressed before the claim can be accepted.

major comments (3)

[§3.2, Table 1] The central claim that MARIO's mixed-supervision design outperforms fully supervised models is confounded by training-data volume. According to §3.1, the fully supervised baselines are trained on 1,451 pixel-level images, while MARIO additionally trains on 33,884 box-annotated LDPolypVideo frames, 49,136 SUN-SEG frames (polygon and scribble), and 1,412 PolypGen point-annotated images, totaling over 80,000 extra weakly labeled samples. No baseline is trained with the same weak data, and no experiment isolates the effect of the annotation-mixing mechanism from the effect of simply having more training data. Without a controlled comparison—for example, (a) MARIO trained only on the 1,451 pixel-level images, (b) a fully supervised baseline trained with the same additional weak data through pseudo-labels or auxiliary losses, or (c) subsampling the weak data to match the baseline's data volume—the reported 1.0% Dice improvement over CAFE-Net cannot be attributed to mixed supervision. This is a load-bearing gap.
[§2.2, Point Supervision] The paper states that PolypGen provides 1,412 images with point annotations (5 foreground and 5 background points), but the proposed point-supervision loss is a rotation-consistency MSE loss that does not use the point labels at all. The described loss constrains predictions on an image to be consistent with predictions on its 90-degree-rotated version, which is a generic consistency regularizer that can be applied to any image, labeled or not. As written, the point-level labels are never used to supervise the model; thus the claim that MARIO learns from point annotations is unsupported. Either the point labels are used in a way not described in the paper (e.g., a cross-entropy term on the annotated pixels), or the point-annotation data contributes nothing beyond unlabeled images. The authors must clarify or correct this.
[§3.3, Table 2] The ablation study only varies loss components while keeping the full mixed dataset fixed. It never runs MARIO on the pixel-only training set, nor does it remove individual annotation types (box, polygon, scribble, point) one at a time. Consequently, the ablation cannot distinguish the contribution of the mixed-supervision mechanism from the contribution of the additional data volume. A proper ablation should include at least a row with only dense supervision on the 1,451 pixel-level images, and ideally rows with each weak annotation type added while holding the total number of training images constant or controlling for data volume.

minor comments (5)

[Table 1 and §3.2] The second-best model is referred to as 'CAFA-Net' in the text but 'CAFE-Net' in the table caption and in reference [9]; the naming should be made consistent.
[§3.1] The statement that SUN-SEG's 49,136 samples were 'evenly split for polygon and scribble annotations' is ambiguous; please specify the exact numbers for each annotation type.
[§3.3, Table 2] The ablation table does not state which test set or aggregation the Dice and IoU values correspond to; presumably they are the weighted-average metrics over the five test sets, but this should be stated explicitly.
[§2.2, Eq. (2)] The total loss is a simple sum of the five losses with equal weights; the paper does not discuss whether the loss scales are comparable or whether any weight balancing was considered.
[§3.1] The datasets SUN-SEG, LDPolypVideo, and PolypGen are used in training but no references are provided for them; citations should be added.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MARIO's losses are standard, its evaluation is on external benchmarks, and no fitted parameter or self-citation chain makes the central result true by construction.

full rationale

The paper contains no derivation that reduces to its own inputs. The five supervision losses (LBCE/LDice, the mask-to-box box loss, the scribble uncertainty loss, and the rotation-based point consistency loss) are conventional losses applied to the given labels; none is fitted to the test benchmarks or defined in terms of the reported Dice/IoU numbers. The comparison in Table 1 is against fully supervised baselines trained on the 1,451 pixel-level images, while MARIO additionally trains on 33,884 box-annotated LDPolypVideo frames, 49,136 SUN-SEG frames, and 1,412 PolypGen point images. This is a genuine experimental confound: the ablation in Table 2 varies only which loss terms are active, not which training data are used, so the paper never isolates the mixed-supervision mechanism from the larger data volume. That weakness affects the causal claim, but it is a validity gap, not circularity, because the superior numbers are not entailed by the method's definition or by any fitted parameter. The self-citations to SANet [3] and WeakPolyp [5] are used for prior art and standard loss design, and the paper's own method is evaluated externally on five public benchmarks with held-out test splits; no load-bearing premise depends on an unverified result from those self-citations. Accordingly, there is no specific equation or fitted constant that makes the headline result true by construction.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the comparability of baselines and the validity of weakly annotated data. No new entities are introduced. The equal loss weights and training schedule are hand-set without reported tuning.

free parameters (3)

Loss weights (all equal to 1) = 1
Ltotal sums the five losses without learned or tuned weights; no ablation on weights is reported.
Training iterations = 60,000
Stopping criterion used in implementation; no validation curve or early stopping described.
Initial learning rate = 0.05
Chosen for SGD; no tuning study reported.

assumptions (3)

domain assumption Baseline numbers from published papers are comparable to the authors' own implementation.
The comparison in Table 1 relies on reusing reported metrics for U-Net, PraNet, SANet, Polyp-Pvt, LDNet, HSNet, UCFA-Net, and CAFE-Net, assuming identical evaluation protocols and splits.
domain assumption The weakly annotated datasets (LDPolypVideo, SUN-SEG, PolypGen) contain labels of sufficient quality for training.
The paper does not clean or validate these labels; it assumes they are usable.
standard math Standard loss functions (BCE, Dice) provide effective optimization.
The method builds on existing practice; no proof is provided beyond prior work.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MARIO: A Mixed Annotation Framework For Polyp Segmentation." pith.science (2026). https://pith.science/paper/W5GYFTY5

@misc{pith2026250110957,
  author       = {Pith},
  title        = {Pith review of: MARIO: A Mixed Annotation Framework For Polyp Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W5GYFTY5}},
  note         = {Machine review of arXiv:2501.10957}
}

read the original abstract

Existing polyp segmentation models are limited by high labeling costs and the small size of datasets. Additionally, vast polyp datasets remain underutilized because these models typically rely on a single type of annotation. To address this dilemma, we introduce MARIO, a mixed supervision model designed to accommodate various annotation types, significantly expanding the range of usable data. MARIO learns from underutilized datasets by incorporating five forms of supervision: pixel-level, box-level, polygon-level, scribblelevel, and point-level. Each form of supervision is associated with a tailored loss that effectively leverages the supervision labels while minimizing the noise. This allows MARIO to move beyond the constraints of relying on a single annotation type. Furthermore, MARIO primarily utilizes dataset with weak and cheap annotations, reducing the dependence on large-scale, fully annotated ones. Experimental results across five benchmark datasets demonstrate that MARIO consistently outperforms existing methods, highlighting its efficacy in balancing trade-offs between different forms of supervision and maximizing polyp segmentation performance

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

[1]

Recent advances in deep learn- ing have significantly improved polyp segmentation, with models such as U-Net and its variants [1, 2, 3] demonstrating strong performance

INTRODUCTION Colorectal cancer remains a significant global health chal- lenge, with early detection and treatment of polyps critical for improving patient outcomes. Recent advances in deep learn- ing have significantly improved polyp segmentation, with models such as U-Net and its variants [1, 2, 3] demonstrating strong performance. Transformer-based arc...

work page
[2]

MARIO: A Mixed Annotation Framework For Polyp Segmentation

METHODOLOGY 2.1. Transformer-based Polyp Segmentation Model MARIO is a transformer-based weakly supervised segmenta- tion model that uses PVTv2-B2 [10] as its backbone. It ini- arXiv:2501.10957v2 [cs.CV] 12 Feb 2025 Fig. 1. Illustration of our MARIO framework. tially processes input images with dimensions height H and width W , extracting feature maps at ...

work page Pith review arXiv 2025
[3]

Dataset and Implementation Detail This study leverages eight datasets: Kvasir, CVC-ClinicDB, CVC-ColonDB, EndoScene, ETIS, SUN-SEG, LDPolypVideo, and PolypGen

EXPERIMENT 3.1. Dataset and Implementation Detail This study leverages eight datasets: Kvasir, CVC-ClinicDB, CVC-ColonDB, EndoScene, ETIS, SUN-SEG, LDPolypVideo, and PolypGen. A total of 1,451 pixel-level annotated images from the first five datasets were used for training, with the remainder for testing. LDPolypVideo provided 33,884 box- annotated sample...

work page
[4]

MARIO unifies five annotation types: pixel-level, polygon-level, box-level, scribble-level, and point-level

CONCLUSION To address the challenge of data scarcity due to high label- ing costs, we present MARIO, a mixed-supervised model for polyp segmentation. MARIO unifies five annotation types: pixel-level, polygon-level, box-level, scribble-level, and point-level. This integration maximizes the utilization of existing annotated data, caters to medical professio...

work page
[5]

COMPLIANCE WITH ETHICAL STANDARDS This is a retrospective analysis relying exclusively on publicly available, fully anonymized datasets, and the need for ethical approval was waived

work page
[6]

U-net: Convolutional networks for biomedical image segmentation,

Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net: Convolutional networks for biomedical image segmentation,” in MICCAI, 2015, pp. 234–241

work page 2015
[7]

Pranet: Par- allel reverse attention network for polyp segmentation,

Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, and Ling Shao, “Pranet: Par- allel reverse attention network for polyp segmentation,” in MICCAI, 2020, pp. 263–273

work page 2020
[8]

Shallow attention network for polyp segmentation,

Jun Wei, Yiwen Hu, Ruimao Zhang, Zhen Li, S Kevin Zhou, and Shuguang Cui, “Shallow attention network for polyp segmentation,” in MICCAI, 2021

work page 2021

Show all 15 references

[9]

Polyp-pvt: Polyp segmen- tation with pyramid vision transformers,

Bo Dong, Wenhai Wang, Deng-Ping Fan, Jinpeng Li, Huazhu Fu, and Ling Shao, “Polyp-pvt: Polyp segmen- tation with pyramid vision transformers,” CAAI Artifi- cial Intelligence Research, p. 9150015, Dec. 2023

2023
[10]

Weakpolyp: You only look bounding box for polyp segmentation,

Jun Wei, Yiwen Hu, Shuguang Cui, S Kevin Zhou, and Zhen Li, “Weakpolyp: You only look bounding box for polyp segmentation,” in International Conference on Medical Image Computing and Computer-Assisted In- tervention. Springer, 2023, pp. 757–766

2023
[11]

Lesion- aware dynamic kernel for polyp segmentation,

Ruifei Zhang, Peiwen Lai, Xiang Wan, De-Jun Fan, Feng Gao, Xiao-Jian Wu, and Guanbin Li, “Lesion- aware dynamic kernel for polyp segmentation,” inMIC- CAI, 2022

2022
[12]

Hsnet: A hybrid semantic network for polyp segmentation,

Wenchao Zhang, Chong Fu, Yu Zheng, Fangyuan Zhang, Yanli Zhao, and Chiu-Wing Sham, “Hsnet: A hybrid semantic network for polyp segmentation,”Com- puters in biology and medicine , vol. 150, pp. 106173, 2022

2022
[13]

Ucfa-net: A u-shaped cross-fusion network with attention mechanism for enhanced polyp segmen- tation,

Shuai Wang, Tiejun Zhao, Guocun Wang, Ye Han, and Fan Wu, “Ucfa-net: A u-shaped cross-fusion network with attention mechanism for enhanced polyp segmen- tation,” IET Image Processing, vol. 19, no. 1, 2025

2025
[14]

Cafe- net: Cross-attention and feature exploration network for polyp segmentation,

Guoqi Liu, Sheng Yao, Dong Liu, Baofang Chang, Zongyu Chen, Jiajia Wang, and Jiangqi Wei, “Cafe- net: Cross-attention and feature exploration network for polyp segmentation,” Expert Systems with Applications, vol. 238, pp. 121754, 2024

2024
[15]

Pvtv2: Improved baselines with pyramid vision transformer,

Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao, “Pvtv2: Improved baselines with pyramid vision transformer,” CVMJ, vol. 8, no. 3, pp. 1–10, 2022

2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Recent advances in deep learn- ing have significantly improved polyp segmentation, with models such as U-Net and its variants [1, 2, 3] demonstrating strong performance

INTRODUCTION Colorectal cancer remains a significant global health chal- lenge, with early detection and treatment of polyps critical for improving patient outcomes. Recent advances in deep learn- ing have significantly improved polyp segmentation, with models such as U-Net and its variants [1, 2, 3] demonstrating strong performance. Transformer-based arc...

work page

[2] [2]

MARIO: A Mixed Annotation Framework For Polyp Segmentation

METHODOLOGY 2.1. Transformer-based Polyp Segmentation Model MARIO is a transformer-based weakly supervised segmenta- tion model that uses PVTv2-B2 [10] as its backbone. It ini- arXiv:2501.10957v2 [cs.CV] 12 Feb 2025 Fig. 1. Illustration of our MARIO framework. tially processes input images with dimensions height H and width W , extracting feature maps at ...

work page Pith review arXiv 2025

[3] [3]

Dataset and Implementation Detail This study leverages eight datasets: Kvasir, CVC-ClinicDB, CVC-ColonDB, EndoScene, ETIS, SUN-SEG, LDPolypVideo, and PolypGen

EXPERIMENT 3.1. Dataset and Implementation Detail This study leverages eight datasets: Kvasir, CVC-ClinicDB, CVC-ColonDB, EndoScene, ETIS, SUN-SEG, LDPolypVideo, and PolypGen. A total of 1,451 pixel-level annotated images from the first five datasets were used for training, with the remainder for testing. LDPolypVideo provided 33,884 box- annotated sample...

work page

[4] [4]

MARIO unifies five annotation types: pixel-level, polygon-level, box-level, scribble-level, and point-level

CONCLUSION To address the challenge of data scarcity due to high label- ing costs, we present MARIO, a mixed-supervised model for polyp segmentation. MARIO unifies five annotation types: pixel-level, polygon-level, box-level, scribble-level, and point-level. This integration maximizes the utilization of existing annotated data, caters to medical professio...

work page

[5] [5]

COMPLIANCE WITH ETHICAL STANDARDS This is a retrospective analysis relying exclusively on publicly available, fully anonymized datasets, and the need for ethical approval was waived

work page

[6] [6]

U-net: Convolutional networks for biomedical image segmentation,

Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net: Convolutional networks for biomedical image segmentation,” in MICCAI, 2015, pp. 234–241

work page 2015

[7] [7]

Pranet: Par- allel reverse attention network for polyp segmentation,

Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, and Ling Shao, “Pranet: Par- allel reverse attention network for polyp segmentation,” in MICCAI, 2020, pp. 263–273

work page 2020

[8] [8]

Shallow attention network for polyp segmentation,

Jun Wei, Yiwen Hu, Ruimao Zhang, Zhen Li, S Kevin Zhou, and Shuguang Cui, “Shallow attention network for polyp segmentation,” in MICCAI, 2021

work page 2021