Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

A multi-stage AI-plus-clinician pipeline densely labels full colonoscopy videos so modern multimodal models can be tested on real lesions, not just single-class polyps.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 18:04 UTC pith:G5QUGK3M

load-bearing objection Useful multi-task colonoscopy video benchmark with a practical agentic labeling recipe; multi-class quality is only partially grounded, but the release and evaluations still move the field. the 4 major comments →

arxiv 2603.25645 v3 pith:G5QUGK3M submitted 2026-03-26 eess.IV cs.CVcs.HC

Colon-Bench: An Agentic Workflow for Scalable Dense Lesion Annotation in Full-Procedure Colonoscopy Videos

Abdullah Hamdi , Changchun Yang , Xin Gao This is my paper
classification eess.IV cs.CVcs.HC
keywords colonoscopylesion detectionpolypsMLLMsvideo object segmentationagentic annotationopen-vocabulary segmentationvideo VQA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Colonoscopy screening can prevent many colon-cancer deaths, but dense annotation of long, noisy procedure videos has been too costly and inconsistent for modern multimodal models. This paper claims that a staged agentic workflow—vision-language proposals, verification, tracker-based boxes and masks, cued AI confirmation, and a final physician gate—can scalably produce high-quality multi-class annotations from existing full-procedure recordings. The resulting Colon-Bench contains 528 verified windows spanning 14 lesion types, more than 300,000 boxes, 213,000 masks, and 133,000 words of clinical text. With this resource the authors show that frontier multimodal models already localize lesions far better than a general segmenter when paired with a tracker, and that a short distilled “colon-skill” prompt, built from shared model errors, lifts zero-shot video question-answering accuracy by as much as 9.7 percent without any training.

Core claim

A multi-stage agentic annotation pipeline that successively filters VLM proposals through verification, EdgeTAM tracking, cued AI confirmation, and clinician review yields a verified multi-task colonoscopy benchmark of 528 windows, 14 lesion categories, over 300k boxes, 213k masks and 133k clinical words; on that benchmark modern multimodal models achieve strong localization (up to 48.3 mIoU) and a distilled colon-skill prompt improves zero-shot VQA by up to 9.7 percent.

What carries the argument

The multi-stage agentic workflow (temporal VLM proposals → verification agent → EdgeTAM box/mask tracking → cued AI confirmation → human-in-the-loop clinician review) that both densifies labels and progressively raises precision on the retained windows.

Load-bearing premise

That successive Gemini-based filters plus an 11.6 percent final human rejection rate produce high-quality labels for all fourteen lesion types, even though the quantitative precision/recall gains are measured only against the surrogate polyp boxes of the source dataset.

What would settle it

An independent multi-class re-annotation of a random subset of the 528 windows by clinicians who never saw the AI proposals; if their agreement with the released boxes, masks and clinical text falls well below the reported 88.4 percent acceptance rate, the quality claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Colon-Bench, a multi-task colonoscopy video benchmark produced by a multi-stage agentic annotation pipeline applied to REAL-COLON full-procedure videos. The pipeline combines VLM temporal proposals, verification filtering, EdgeTAM tracking, cued AI confirmation, and clinician review to yield 528 curated windows spanning 14 lesion categories, ~300k bounding boxes, ~213k masks, and ~133k words of clinical text. The authors use this resource to evaluate modern MLLMs on binary lesion classification, open-vocabulary video object segmentation (via three MLLM boxes prompting EdgeTAM), and prompted/unprompted video VQA, reporting strong Gemini-class localization (up to 48.3 mIoU) relative to SAM-3 and a distilled “colon-skill” prompt that improves zero-shot VQA by up to 9.7%. Stage-wise ablations against REAL-COLON polyp boxes, human acceptance rates, VQA debiasing, and frame-count/temporal-context ablations are provided.

Significance. If the multi-class dense labels are as reliable as claimed, Colon-Bench would fill a genuine gap: existing public colonoscopy resources are largely single-class polyp sets or anatomy-focused, and lack the joint spatial–temporal–linguistic supervision needed for modern MLLM evaluation. The agentic pipeline, public code/dataset link, multi-task suite (classification, OV-VOS, two-tier VQA), careful polyp-surrogate stage ablations (Tables 2 and 7), VQA debiasing protocol, and training-free colon-skill gains are concrete contributions that would be useful to the medical video and MLLM communities. The work is therefore significant as a resource paper, provided multi-class label quality and circularity risks are more rigorously bounded.

major comments (4)
  1. [§2.1, Tables 2 and 7] §2.1 and Tables 2/7: Stage-wise precision/recall/F1/specificity gains are computed exclusively against REAL-COLON polyp bounding boxes. Non-polyp classes that define the claimed 14-category taxonomy (ulcers, bleeding, SSL, angioectasia, diverticulum, etc.; Fig. 2) have no independent frame-level spatial or temporal ground truth. Because the central claim is a verified multi-class dense benchmark, the manuscript needs either (i) a clinician-annotated multi-class subsample with per-class spatial metrics or (ii) an explicit, quantified limitation that only polyp localization is externally validated.
  2. [§2.1, Table 3, §3.1] §2.1 and Table 3: Annotation proposals, verification, confirmation, and VQA item generation all rely on Gemini-family models that later dominate the leaderboard (Gemini 3 Pro/Flash top VQA and segmentation). Human review rejects only 11.6% of the final 597 windows and is described as binary accept/reject of pre-rendered overlay clips, not exhaustive multi-label spatial/temporal re-annotation. Residual systematic Gemini biases on non-polyp morphology can therefore propagate into both the benchmark and the reported rankings (including 48.3 mIoU and +9.7% colon-skill). The paper should report inter-rater or independent-clinician agreement on a stratified multi-class subset and, where possible, re-run key leaderboard entries with non-Gemini annotation sources or held-out human labels.
  3. [§2.2, Fig. 4, Table 3] §2.2 Baselines and Fig. 4: The OV-VOS protocol evaluates MLLMs by eliciting three boxes that seed EdgeTAM, then reporting tracker IoU/Dice. This is a fair localization-plus-propagation proxy, but it is not pure open-vocabulary video segmentation by the MLLM itself; SAM-3 is compared under a different regime. The manuscript should state this protocol limitation more prominently in the main results and avoid language that equates the numbers with end-to-end MLLM video segmentation.
  4. [§2.2 Debiasing Colon-Bench] §2.2 Debiasing: Blind text-only accuracy remains 44.6% (prompted) and 37.1% (unprompted) vs. 20% chance after two-stage adversarial distractor regeneration. Residual margins are attributed to dataset priors, but the paper does not quantify how much of the MLLM gains (especially colon-skill) could still exploit those priors rather than visual evidence. A short control that reports accuracy under scrambled or frozen video (or text-only with skill) would strengthen the claim that improvements reflect colonoscopy visual reasoning.
minor comments (5)
  1. [Global] Throughout: Multiple typos and inconsistencies (e.g., “Colon-becnh”, “segemtnation”, “evluated”, “Modles”, “Visual Quesion Answering”, “Title Suppressed Due to Excessive Length” headers). A full copy-edit pass is needed.
  2. [Table 1, §2.2] Table 1 vs. abstract/body: Frame and video counts should be cross-checked for consistency (464,035 frames / 528 videos in the curated set vs. the broader 1,597-clip evaluation suite in §2.2).
  3. [Fig. 3, Figs. 9–10] Fig. 3 and qualitative figures: Some rows mix detection and segmentation; captions should state which task and whether overlays are model boxes or EdgeTAM masks.
  4. [License / Acknowledgments] License/ethics: The CC BY note and REAL-COLON copyright retention are good; briefly state IRB/consent status of the source videos and any de-identification steps for clinical text.
  5. [§3.1, Fig. 16] §3.1 colon-skill: The skill text (Fig. 16) is useful; clarify whether it was frozen before the final re-evaluation or tuned on the same error set used for reporting, to avoid train–test leakage of the skill itself.

Circularity Check

1 steps flagged

Minor evaluation contamination in colon-skill (distilled from full-benchmark errors, then re-scored on the same set); the core annotation and MLLM claims are not circular by construction.

specific steps
  1. fitted input called prediction [§3.1 Colon-Skill for MLLMs; also Appendix C.2 / Table 18 / Fig. 5]
    "First, we collect per-model VQA predictions on the full benchmark and stratify errors by lesion category (Fig. 2), retaining only questions that a majority of models answer incorrectly. These shared failure cases, along with their question/answer context and category metadata, are fed to a frontier large language model which synthesises a concise, natural-language Colon-Skill... Second, we prepend this Colon-Skill to every VQA prompt and re-evaluate all models under identical conditions, yielding consistent improvements up to +9.7% (Fig. 5)."

    The skill text is fitted to majority-error items from the full prompted/unprompted VQA sets, then the same sets are re-scored with that skill prepended. Reported gains (up to +9.7 pp) are therefore measured on the data used to construct the intervention, not on held-out questions. This is evaluation contamination of a secondary claim, not a forced mathematical identity; some models still drop (e.g. Molmo), so the result is not fully by-construction, but the improvement is not an independent out-of-sample prediction.

full rationale

Colon-Bench is an empirical dataset/benchmark paper, not a first-principles derivation. The agentic annotation chain (VLM proposals → verification → EdgeTAM tracking → cued AI confirmation → clinician review) is validated against external REAL-COLON polyp boxes (Table 2, Table 7) and a final human accept/reject gate (69/597 rejected). Those steps do not reduce to self-definition or a self-citation uniqueness theorem. MLLM leaderboard results are zero-shot evaluations on held-out model weights, not fitted parameters renamed as predictions. The only clear circularity-adjacent step is colon-skill: error patterns are collected on the full VQA benchmark, synthesized into a skill prompt, and re-evaluated on that same benchmark, so the reported +9.7% is not a held-out generalization claim. That is mild fitted-input contamination of one secondary result, not a load-bearing reduction of the paper’s main contribution. No self-citation uniqueness chain, ansatz smuggling, or self-definitional equations appear. Score 2 reflects one minor non-load-bearing evaluation circularity; the dataset construction and primary MLLM comparisons remain independently grounded.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 2 invented entities

The work is empirical and engineering-heavy. It inherits the REAL-COLON videos and polyp boxes, assumes commercial VLMs (Gemini family) are sufficiently reliable for proposal and filtering stages, treats EdgeTAM as a fixed tracker, and relies on one clinician’s review as the final quality gate. No free parameters are fitted to produce the central performance numbers; the colon-skill is a post-hoc textual summary of observed errors rather than a new physical entity.

free parameters (2)
  • number of detection frames per window for segmentation = 3
    Chosen as 3 equally-spaced frames after an ablation (1–7 frames); the reported mIoU numbers depend on this design choice.
  • retention thresholds of successive AI filters
    Implicit decision thresholds of the verification and confirmation agents determine which windows reach human review; only aggregate retention rates are reported.
axioms (4)
  • domain assumption REAL-COLON polyp bounding boxes constitute a valid surrogate for measuring temporal precision/recall of the multi-lesion pipeline.
    Tables 2 and 7 report all stage-wise F1 gains exclusively against these polyp labels even though Colon-Bench claims 14 lesion classes.
  • domain assumption A single experienced surgeon’s binary accept/reject on short overlaid clips is sufficient final quality control for free-text clinical descriptions and multi-label categories.
    Section 2.1 and acknowledgments; 11.6% rejection rate is taken as evidence of high agreement.
  • ad hoc to paper EdgeTAM tracker seeded by three MLLM boxes fairly measures open-vocabulary video object segmentation performance of the MLLM.
    All non-SAM-3 segmentation numbers in Table 3 and Fig. 4 use this fixed protocol.
  • ad hoc to paper Two-stage adversarial distractor regeneration plus blind text-only stress test adequately removes surface-form shortcuts from Gemini-generated VQA items.
    Section 2.2; residual blind accuracy remains well above chance (44.6%/37.1%).
invented entities (2)
  • Colon-Bench multi-task suite (prompted/unprompted VQA, binary classification, OV-VOS) independent evidence
    purpose: Provide the first dense multi-lesion, language-rich, long-sequence evaluation resource for colonoscopy MLLMs.
    Constructed by the authors; no prior public set matches the combination of attributes claimed in Table 1.
  • colon-skill prompting strategy no independent evidence
    purpose: Distill cross-model VQA error patterns into a reusable textual checklist that improves zero-shot accuracy without training.
    Synthesized by a frontier LLM from majority-error questions; gains reported in Fig. 5 and Table 18.

pith-pipeline@v1.1.0-grok45 · 26028 in / 3101 out tokens · 39008 ms · 2026-07-13T18:04:57.440597+00:00 · methodology

0 comments
read the original abstract

Early screening via colonoscopy is critical for colon cancer prevention, yet developing robust AI systems for this domain is hindered by the lack of densely annotated, long-sequence video datasets. Existing datasets predominantly focus on single-class polyp detection and lack the rich spatial, temporal, and linguistic annotations required to evaluate modern Multimodal Large Language Models (MLLMs). To address this critical gap, we introduce Colon-Bench, generated via a novel multi-stage agentic workflow. Our pipeline seamlessly integrates temporal proposals, bounding-box tracking, AI-driven visual confirmation, and human-in-the-loop review to scalably annotate full-procedure videos. The resulting verified benchmark is unprecedented in scope, encompassing 528 videos, 14 distinct lesion categories (including polyps, ulcers, and bleeding), over 300,000 bounding boxes, 213,000 segmentation masks, and 133,000 words of clinical descriptions. We utilize Colon-Bench to rigorously evaluate state-of-the-art MLLMs across lesion classification, Open-Vocabulary Video Object Segmentation (OV-VOS), and video Visual Question Answering (VQA). The MLLM results demonstrate surprisingly high localization performance in medical domains compared to SAM-3. Finally, we analyze common VQA errors from MLLMs to introduce a novel "colon-skill" prompting strategy, improving zero-shot MLLM performance by up to 9.7% across most MLLMs. The dataset and the code are available at https://abdullahamdi.com/colon-bench .

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks

    cs.CV 2026-05 unverdicted novelty 7.0

    MAVEN pipeline generates multi-scale spatio-temporal event descriptions from videos using agentic adaptation and refinement, then produces training data that lets a fine-tuned 8B model outperform Gemini baselines on p...

  2. MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks

    cs.CV 2026-05 conditional novelty 6.0

    An agentic three-stage video annotation pipeline with an MSTED intermediate and top-down domain adaptation produces CoT training data that lifts Cosmos-Reason2 past Gemini on traffic event reasoning.

Reference graph

Works this paper leans on

33 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Scientific Data10(1), 75 (2023) 2

    Ali, S., Jha, D., Ghatwary, N., Realdon, S., Cannizzaro, R., Salem, O.E., Lamarâo, D.V., Da Roza, C., Riegler, M.A., Halvorsen, P.: A multi-centre polyp detection and segmentation dataset for generalisability assessment. Scientific Data10(1), 75 (2023) 2

    2023

  2. [2]

    arXiv preprint arXiv:2511.21631 (2025) 3, 7, 8

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025) 3, 7, 8

    Pith/arXiv arXiv 2025

  3. [3]

    Scientific Data11(1), 539 (2024)

    Biffi, C., Antonelli, G., Bernhofer, S., Hassan, C., Hirata, D., Iwatate, M., Maieron, A., Salvagnini, P., Cherubini, A.: Real-colon: A dataset for developing real-world ai applications in colonoscopy. Scientific Data11(1), 539 (2024). https://doi.org/10. 1038/s41597-024-03359-0 2, 4, 10, 13, 16

    2024

  4. [4]

    arXiv preprint arXiv:2601.03232 (2026) 5

    Bose, K., Kumar, A., Soundararajan, R., Mudgil, P., Ralmilay, S., Dutta, N., Singhal, M., Kumar, A., Sen, S., Patra, A., et al.: Multi-rads synthetic radiology report dataset and head-to-head benchmarking of 41 open-weight and proprietary language models. arXiv preprint arXiv:2601.03232 (2026) 5

    arXiv 2026

  5. [5]

    arXiv preprint arXiv:2511.16719 (2025) 6, 7

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala, K.V., Khedr, H., Huang, A., et al.: Sam 3: Segment anything with concepts. arXiv preprint arXiv:2511.16719 (2025) 6, 7

    Pith/arXiv arXiv 2025

  6. [6]

    arXiv preprint arXiv:2405.19538 (2024) 3

    Chambon, P., Delbrouck, J.B., Sounack, T., Huang, S.C., Chen, Z., Varma, M., Truong, S.Q., Chuong, C.T., Langlotz, C.P.: Chexpert plus: Augmenting a large chest x-ray dataset with text radiology reports, patient demographics and additional image formats. arXiv preprint arXiv:2405.19538 (2024) 3

    Pith/arXiv arXiv 2024

  7. [7]

    In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2025

    Choudhuri, A., Gao, Z., Zheng, M., Planche, B., Chen, T., Wu, Z.: Polypseg- track: Unified foundation model for colonoscopy video analysis. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2025. Springer (2025) 2

    2025

  8. [8]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025) 3, 7, 8 Title Suppressed Due to Excessive Length 11

    Deitke, M., Clark, C., et al.: Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025) 3, 7, 8 Title Suppressed Due to Excessive Length 11

    2025

  9. [9]

    In: European Conference on Computer Vision

    Fan, Y., Ma, X., Wu, R., Du, Y., Li, J., Gao, Z., Li, Q.: Videoagent: A memory- augmented multimodal agent for video understanding. In: European Conference on Computer Vision. pp. 75–92. Springer (2024) 3

    2024

  10. [10]

    arXiv preprint arXiv:2307.08041 (2023) 3, 7, 8

    Ge, Y., Zhao, Y., Zhu, Z., et al.: Planting a seed of vision in large language model. arXiv preprint arXiv:2307.08041 (2023) 3, 7, 8

    Pith/arXiv arXiv 2023

  11. [11]

    arXiv preprint arXiv:2403.05530 (2024) 2, 3, 7, 8

    Gemini Team: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024) 2, 3, 7, 8

    Pith/arXiv arXiv 2024

  12. [12]

    com/THUDM/GLM-4 3, 7, 8

    GLM Team: Glm-4: Open multilingual multimodal chat lms (2024), https://github. com/THUDM/GLM-4 3, 7, 8

    2024

  13. [13]

    In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2025

    He, Y., Zhu, Y., Fu, P., Yang, R., Chen, T., Wang, Z., Li, Q., Zhou, P., Yang, X., Wang, S.: Endo-clip: Progressive self-supervised pre-training on raw colonoscopy records. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2025. Springer (2025), early accepted. Available at: https://arxiv.org/abs/ 2505.09435 2, 6, 7

    Pith/arXiv arXiv 2025

  14. [14]

    In: MultiMedia Modeling

    Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., Johansen, H.D.: Kvasir-seg: A segmented polyp dataset. In: MultiMedia Modeling. Lecture Notes in Computer Science, vol. 11962, pp. 451–462. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37734-2_37 2

    work page doi:10.1007/978-3-030-37734-2_37 2020

  15. [15]

    Gastroenterology170(1), 174–187 (2026)

    Jong, M.R., Boers, T.G.W., Kusters, K.H.J., Jaspers, T.J.M., van der Putten, J.A., de With, P.H.N., van der Sommen, F., de Groof, A.J., Bergman, J.J.G.H.M., et al.: Gastronet-5m: A multicenter dataset for developing foundation models in gastrointestinal endoscopy. Gastroenterology170(1), 174–187 (2026). https: //doi.org/10.1053/j.gastro.2025.07.030 2

    doi:10.1053/j.gastro.2025.07.030 2026

  16. [16]

    PLoS ONE16(8), e0255809 (2021) 2

    Li, K., Fathan, M.I., Patel, K., Zhang, T., Zhong, C., Bansal, A., Rastogi, A., Wang, Z., Wang, G.: Colonoscopy polyp detection and classification: Dataset creation and comparative evaluations. PLoS ONE16(8), e0255809 (2021) 2

    2021

  17. [17]

    Neurocomputing (2024) 2

    Li, S., Ren, Y., Yu, Y., Li, H.: A survey of deep learning algorithms for colorectal polyp segmentation. Neurocomputing (2024) 2

    2024

  18. [18]

    In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference

    Ma, Y., Chen, X., Cheng, K., Li, Y., Sun, B.: Ldpolypvideo benchmark: A large- scale colonoscopy video dataset of diverse polyps. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference. pp. 387–396. Springer (2021) 2

    2021

  19. [19]

    Gastrointestinal Endoscopy93(4), 960–967 (2021) 2

    Misawa, M., Kudo, S.e., Mori, Y., Maeda, Y., Kataoka, S., Ogata, N., Ichimasa, K., Kudo, T., Mori, K., Ishigaki, T.: Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video). Gastrointestinal Endoscopy93(4), 960–967 (2021) 2

    2021

  20. [20]

    OpenAI: Gpt-4 technical report (2023) 2, 7, 8

    2023

  21. [21]

    In: ICML

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763. PMLR (2021) 2, 6, 7

    2021

  22. [22]

    Scientific Data12(1), 1382 (2025)

    Song, Y., Zhang, Z., Wang, R., Zhong, L., Cai, C., Chen, J., Zhou, Y., Wang, X., Li, Z., Yang, L., Li, Z., Yan, H., Zhang, Q., Qian, D., Li, X.: Cas-colon: A comprehensive colonoscopy anatomical segmentation dataset for artificial intelligence development. Scientific Data12(1), 1382 (2025). https://doi.org/10.1038/s41597-025-05588-3 2

    work page doi:10.1038/s41597-025-05588-3 2025

  23. [23]

    CA: A Cancer Journal for Clinicians 71(3), 209–249 (2021)

    Sung, H., Ferlay, J., Siegel, R.L., Laversanne, M., Soerjomataram, I., Jemal, A., Bray, F.: Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians 71(3), 209–249 (2021). https://doi.org/10.3322/caac.21660 1

    doi:10.3322/caac.21660 2020

  24. [24]

    In: 12 H

    Tian, Q., Liao, H., Huang, X., Yang, B., Lei, D., Ourselin, S., Liu, H.: Endomamba: An efficient foundation model for endoscopic videos via hierarchical pre-training. In: 12 H. Abdullah et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2025. Springer (2025) 2

    2025

  25. [25]

    In: The Twelfth International Conference on Learning Representations (ICLR)

    Wang, Y., He, Y., Li, Y., Li, K., Yu, J., Ma, X., Li, X., Chen, G., Chen, X., Wang, Y., Luo, P., Liu, Z., Wang, Y., Wang, L., Qiao, Y.: Internvid: A large-scale video-text dataset for multimodal understanding and generation. In: The Twelfth International Conference on Learning Representations (ICLR). OpenReview.net (2024), https://openreview.net/forum?id=...

    2024

  26. [26]

    IEEE Journal of Biomedical and Health Informatics pp

    Wang, Z., Liu, C., Dou, Q., Zhang, S., et al.: Improving foundation model for endoscopy video analysis via representation learning on long sequences. IEEE Journal of Biomedical and Health Informatics pp. 3526–3536 (2025). https://doi. org/10.1109/JBHI.2025.3349925 2

    work page doi:10.1109/jbhi.2025.3349925 2025

  27. [27]

    IEEE Transactions on Medical Imaging44(7), 2948–2959 (2025)

    Zeng, Q., Xie, Y., Lu, Z., Lu, M., Wu, Y., Xia, Y.: Segment together: A versatile paradigm for semi-supervised medical image segmentation. IEEE Transactions on Medical Imaging44(7), 2948–2959 (2025). https://doi.org/10.1109/TMI.2025. 3360447 2

    work page doi:10.1109/tmi.2025 2025

  28. [28]

    With bbox

    Zhou, C., Zhu, C., Xiong, Y., Suri, S., Xiao, F., Wu, L., Krishnamoorthi, R., Dai, B., Loy, C.C., Chandra, V., et al.: Edgetam: On-device track anything model. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 13832–13842 (2025) 4, 6, 7, 8, 22 Title Suppressed Due to Excessive Length 13 A Additional Dataset Details A.1 Annotat...

    2025

  29. [29]

    **Assess the geometry:** Is it a mass (protruding), a defect (depressed/ulcerated), a hole (diverticulum), or a flat vascular anomaly?

  30. [30]

    **Evaluate the attachment:** If it’s a polyp, is it draped over/broadly attached to a fold (sessile) or hanging by a tether (pedunculated)?

  31. [31]

    **Check the lighting mode:** Is the image under white light (pink/red hues) or NBI/BLI (green/brown/cyan hues)? Adjust color descriptions accordingly

  32. [32]

    **Examine the surface & margins:** Is the surface smooth, cloud-like, granular, or lobulated? Are the margins sharp, rolled, or indistinct?

  33. [33]

    Colon-Skill Prompt Context.The colon-skillSKILL.mdfile is used as context augmentation for MLLMs in the VQA benchmarks

    **Identify active tools or residue:** Are there white/yellow strings (mucus/stool), active red oozing (bleeding), a wire loop (snare), or blue fluid (submucosal injection)? Fig.16. Colon-Skill Prompt Context.The colon-skillSKILL.mdfile is used as context augmentation for MLLMs in the VQA benchmarks. The skill is extracted by analysing error patterns acros...