Pith. sign in

REVIEW 5 major objections 5 minor 47 references

CLIP-RL: Surgical Scene Segmentation Using Contrastive Language-Vision Pretraining & Reinforcement Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CLIP-RL combines a frozen CLIP encoder with an RL-based residual-correction step and reports 81% mean IoU on EndoVis 2018 surgical scenes.

desk verdict Promising architecture, but the 'outperforms SOTA' claim rests on comparison rows with broken provenance; worth a careful major revision, not a reject. read the letter →

arxiv 2507.04317 v1 pith:7CBBTXXO submitted 2025-07-06 eess.IV cs.AIcs.CVcs.LG

Fatmaelzahraa Ali Ahmed , Muhammad Arsalan , Abdulaziz Al-Ali , Khalid Al-Jalham , Shidin Balakrishnan This is my paper
classification eess.IVcs.AIcs.CVcs.LG
keywords CLIP-RLsurgicalscenesegmentationsemanticreinforcementlearningcurriculumEndoVis20182017contrastivelanguage-imagepretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CLIP-RL is an attempt to make surgical scene segmentation cheap and fully automatic by replacing prompt-dependent vision-language models with a frozen CLIP vision encoder, a lightweight decoder, an RL-based residual-refinement module, and a curriculum loss schedule. The paper claims that on EndoVis 2018 this combination reaches 81% mean IoU and 0.88 Dice, outperforming prompt-based and specialist baselines, and 74.12% mean IoU on EndoVis 2017. If that is right, a text-free segmentation pipeline built on pretrained contrastive features can beat methods that need manual prompt engineering, which would be directly useful for analyzing long minimally invasive surgery videos. The central problem the paper tries to solve is labor-intensive annotation and the impracticality of prompting every frame; its proposed fix is to let a learned residual correction refine the decoder output without any text input.

What carries the argument

The load-bearing object is the RL-based residual refinement defined by $O = s_L + \alpha r$: a lightweight agent samples a scalar residual scaling factor $\alpha$ after the decoder has produced its softmax map, and an auxiliary module derives a residual term $r$ from the same logits. This makes the whole pipeline act as a self-correcting segmentation network in which even small boundary corrections are learned rather than prompted. The second mechanism is the curriculum weighting $f_{\text{epoch}} = (1 - \text{epoch}/\text{epoch}_{\text{total}})^2$, which stabilizes training by focusing on segmentation loss early and on RL loss later, and the frozen CLIP vision transformer supplies multi-resolution semantic features after the CLS token is discarded.

What would settle it

Rerun U-Net and SegFormer on the official EndoVis 2018 test set using the same $224 \times 224$ resolution, the same 20% validation split, and the same evaluation code as CLIP-RL, and check the source of the S3Net row in Table I. If either baseline reproduces above 0.81 mIoU, or if the cited S3Net reference is not the model in the table, the 'outperforms state-of-the-art' claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the combination of a frozen CLIP vision transformer, a multi-stage upsampling decoder, and a reinforcement-learning-inspired refinement module produces state-of-the-art surgical scene segmentation. On EndoVis 2018 the reported mean IoU is $0.81$ and Dice is $0.88$, with 8 of 11 classes best among the compared models; on EndoVis 2017 the mean IoU is $74.12\%$, best in 5 of 7 instrument classes. The RL agent is a small fully connected network that maps a globally pooled encoder feature to a distribution over discrete residual scaling factors $\{-0.1, 0.0, +0.1\}$, and the final prediction is $O = s_L + \alpha r$, where $s_L$ is the softmax output of the decoder and $r$ is a residual correction computed from those logits. A curriculum schedule $f_{\text{epoch}} = (1 - \text{epoch}/\text{total})^2$ weights the segmentation loss heavily at first and gradually shifts weight to the policy-gradient RL loss, which the ablation study attributes to the jump from 72.4% to 81.0% mIoU.

Load-bearing premise

The load-bearing assumption is that the benchmark comparison is valid: the 'S3Net' row in Table I must be the surgical segmentation S3Net the text describes, not the image-relighting S3Net that reference [13] points to, and the starred U-Net and SegFormer results must come from the same training protocol, image resolution, validation split, and evaluation code used for CLIP-RL.

Editorial extensions

If this is right

  • If the reported numbers reproduce, CLIP-RL would be the first prompt-free CLIP-based surgical segmentation pipeline to beat prompt-dependent SAM variants on EndoVis 2018.
  • The ablation table (72.4 to 76.8 to 81.0 mIoU) indicates that curriculum learning and RL refinement each contribute independently, so both components are worth keeping when the method is transferred.
  • Because the model works without text prompts during inference, it could be applied to long surgical videos without per-frame interaction.
  • The per-class results suggest that thin, low-contrast structures (thread and covered kidney on EndoVis 2018) are the residual failure mode, pointing to where the next version of the refinement module should concentrate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The language side of CLIP is never used at inference; only the frozen image encoder is present. This suggests the same architecture could work with any contrastively trained vision backbone, and a fair test would swap in a pure-vision encoder to see how much of the gain is actually from language alignment.
  • The RL module is effectively a learned global residual calibration of the segmentation logits rather than a full sequential decision process; treating it as such makes it directly comparable to lightweight post-processing layers and opens a cheaper way to test the contribution.
  • A natural testable extension is to enlarge the action space from three discrete scalars to spatially varying or multi-scale residual maps, and to add temporal consistency across video frames; this could specifically target the thread and covered-kidney errors the paper reports.
  • Because validation was a random 20% split of the training set and checkpoints were selected by validation mIoU, the reported numbers should be re-checked against the official EndoVis test protocols before relying on the clinical claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript proposes CLIP-RL, a surgical scene segmentation framework that combines a frozen CLIP encoder with a lightweight decoder, a reinforcement-learning-based residual refinement module, and curriculum learning. The method is evaluated on the EndoVis 2017 and EndoVis 2018 benchmarks, reporting a mean IoU of 74.12% and 81%, respectively, and the authors claim that these results outperform existing state-of-the-art models. The paper also includes an ablation study showing incremental gains from curriculum learning and the RL module. The central methodological ingredients are described in Section III, with the architecture and loss defined in Equations (1)-(4), and results are presented in Tables I-IV.

Significance. If the reported results are reproducible, the idea of combining a frozen CLIP backbone with RL-based residual refinement and curriculum learning is a potentially useful contribution to surgical scene segmentation, and the per-class improvements on instruments and soft tissue are noteworthy. However, the current manuscript does not provide sufficient evidence for the central 'outperforms SOTA' claim. The comparison rows in Tables I and II lack provenance or protocol details, the RL module is under-specified, and all results are single-run point estimates without error bars. The paper would be substantially strengthened by releasing code and data, specifying the evaluation protocol, correcting the S3Net citation, and reporting variance over multiple runs.

major comments (5)
  1. [Table I / Reference [13]] The S3Net row in Table I (mIoU 71.99) is cited to reference [13], arXiv:2105.00681, which is 'S3Net: A Single Stream Structure for Depth Guided Image Relighting,' not a surgical segmentation method. The provenance of the 71.99 value is therefore untraceable, and because this row is one of the strongest specialist baselines on EndoVis 2017, the comparison claim is not verifiable. Please either correct the citation to the actual surgical segmentation S3Net paper and report the source of the number, or re-run the baseline under the same protocol and report that protocol.
  2. [Table II / Section IV.A] The asterisked rows in Table II (U-Net*, SegFormer*, CLIP-RL*) are described only as 'reported by us.' The text gives no training or evaluation protocol for these baselines: no image resolution, optimizer, number of epochs, loss function, validation split details, or evaluation code. Since the headline result (0.81 vs 0.75 for SegFormer, 0.65 for AdaptiveSAM, 0.61 for nn-UNet) depends on these rows being produced under identical train/validation/test splits and evaluation metrics, the comparison is currently unsupported. Please specify the full protocol for all self-reported rows and release the evaluation script.
  3. [Section III.B-C / Equations (2)-(4)] The RL component is not fully specified. Equation (2) defines the refined output as O = sL + α·r, but neither the reward function nor the baseline used in the policy-gradient loss is defined anywhere; the text only says that the advantage is 'the reward minus a running baseline.' Moreover, the action space is only given as an example ({−0.1, 0.0, +0.1}), and it is unclear whether α is sampled from a categorical distribution or a continuous one. Without these definitions, the claim that the RL module 'dynamically refines predictions' is not reproducible. Please provide the exact reward, baseline, action distribution, and how gradients flow through the sampling step.
  4. [Section III.B vs III.C] There are internal inconsistencies in the architectural description. Section III.B states the input resolution is 244×244 and that upsampling starts from a 14×14 feature map, while Section III.C states images are resized to 224×224 and upsampling starts from 16×16. The introduction (Section I) calls the encoder a 'ResNet-based CLIP model,' whereas Section III.B/C describes a CLIP vision transformer. Also, Section III.C says the softmax yields 'voxel-wise probability distributions' for what are 2D images. Please reconcile these statements so the architecture is unambiguous.
  5. [Tables I-IV / Section IV] All quantitative results are reported as single-run point estimates with no error bars or significance tests. Given that the reported gap to SegFormer on EndoVis 2018 is 6 mIoU points and the per-class gaps are often smaller, run-to-run variance could affect the conclusions. Please report results over multiple seeds (mean ± std) and, if possible, statistical significance. In addition, Table IV does not state which dataset the ablation was performed on, and Table III's claim of 'highest mIoU in 8 out of 11 classes' is inconsistent with the table, which shows 9 out of 11 classes.
minor comments (5)
  1. [Equation (1)] The notation UPL in Equation (1) is not defined and conflicts with the later use of UPl; please standardize the notation for the upsampling operators.
  2. [Section II.D] The comparison between CLIP and DINOv2 is attributed to references [7] and [9], which appear to be self-citations or unrelated to the specific benchmark claim; please cite primary sources and verify the numbers.
  3. [Section IV.A] Figure 2 is described qualitatively but no quantitative details are given for the selected examples; please specify which frames are shown and whether they are representative or adversarial cases.
  4. [References] The reference list contains formatting inconsistencies, including incomplete entries (e.g., [22]) and potentially incorrect author names (e.g., [27]); please check all references against the original sources.
  5. [Section IV] The phrase 'This two-task setting is critical' is unclear because the paper evaluates on two datasets rather than two joint tasks; please clarify what the two tasks are and how they relate.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the paper's claims are empirical benchmark results, not a derivation chain that reduces to its inputs.

full rationale

CLIP-RL does not derive a theoretical prediction from fitted parameters; it reports measured mIoU/Dice on public EndoVis 2017/2018 splits after training. The loss functions (cross-entropy, Dice, policy-gradient) and the residual-refinement equation O = s_L + alpha*r are explicit design choices, not hidden restatements of the reported metrics. The curriculum weighting f_epoch is given by a closed-form schedule, and the ablation study compares incremental module additions, so the reported gains are not forced by construction. Self-citations [7], [9], and [12] appear in background or related-work claims about CLIP's transfer performance and surgical deep learning, but the central segmentation result is supported by the paper's own experiments against public benchmark splits, not by those citations. The broken S3Net citation (reference [13] points to a depth-guided relighting paper while Table I lists a segmentation S3Net) and the asterisked baseline rows lacking protocol details are real checkability and provenance problems for the SOTA comparison, but they are correctness/verifiability issues, not circularity: they do not make any reported quantity equal to an input by definition. Under the provided rubric, where the paper is an empirical system paper with external benchmark comparisons and no derivation chain, the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The central claim rests on a frozen CLIP encoder, an unspecified RL reward, and a comparable evaluation setup. The hand-chosen action space and curriculum exponent are free parameters. No new physical entities are introduced; the RL agent is an invented model component without independent evidence.

free parameters (2)
  • RL action space scaling set = {-0.1, 0.0, +0.1}
    Hand-chosen discrete actions for the RL residual refinement; no sensitivity analysis or justification is provided (Section III.B).
  • Curriculum exponent 2 in f_epoch = 2
    The exponent in f_epoch = (1 - epoch/epoch_total)^2 is chosen without explanation and controls the loss weighting schedule (Eq. 4).
assumptions (3)
  • domain assumption Frozen CLIP vision transformer features transfer to surgical instrument and tissue segmentation without fine-tuning.
    Section III.C states the CLIP encoder is kept frozen to preserve semantic representations, but no experiment verifies transferability in the surgical domain.
  • ad hoc to paper A well-defined RL reward and advantage baseline exist and are computable from the segmentation output and ground truth.
    The paper relies on an unspecified reward signal for the policy gradient loss (Eq. 3), but never defines it, making the RL mechanism unfalsifiable as described.
  • domain assumption Evaluation protocol (MICCAI training/test split, 20% random validation) yields directly comparable conditions across all baselines.
    Section IV says splits follow MICCAI, but random 20% validation and the '*' baselines re-run by the authors are not documented per method, so comparability is assumed.
invented entities (1)
  • RL-based residual refinement agent
    purpose: Samples a scalar residual scaling factor alpha to adjust initial segmentation logits (O = sL + alpha * r), claimed to refine masks.
    The paper provides no external validation or falsifiable prediction for this component; the reward signal driving it is not defined, so its benefit cannot be independently assessed beyond the ablation table.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CLIP-RL: Surgical Scene Segmentation Using Contrastive Language-Vision Pretraining & Reinforcement Learning." pith.science (2026). https://pith.science/paper/7CBBTXXO

@misc{pith2026250704317,
  author       = {Pith},
  title        = {Pith review of: CLIP-RL: Surgical Scene Segmentation Using Contrastive Language-Vision Pretraining & Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7CBBTXXO}},
  note         = {Machine review of arXiv:2507.04317}
}
read the original abstract

Understanding surgical scenes can provide better healthcare quality for patients, especially with the vast amount of video data that is generated during MIS. Processing these videos generates valuable assets for training sophisticated models. In this paper, we introduce CLIP-RL, a novel contrastive language-image pre-training model tailored for semantic segmentation for surgical scenes. CLIP-RL presents a new segmentation approach which involves reinforcement learning and curriculum learning, enabling continuous refinement of the segmentation masks during the full training pipeline. Our model has shown robust performance in different optical settings, such as occlusions, texture variations, and dynamic lighting, presenting significant challenges. CLIP model serves as a powerful feature extractor, capturing rich semantic context that enhances the distinction between instruments and tissues. The RL module plays a pivotal role in dynamically refining predictions through iterative action-space adjustments. We evaluated CLIP-RL on the EndoVis 2018 and EndoVis 2017 datasets. CLIP-RL achieved a mean IoU of 81%, outperforming state-of-the-art models, and a mean IoU of 74.12% on EndoVis 2017. This superior performance was achieved due to the combination of contrastive learning with reinforcement learning and curriculum learning.

Figures

Figures reproduced from arXiv: 2507.04317 by the authors.

Figure 1
Figure 1. CLIP-RL Model Architecture boundaries can have significant clinical implications, par￾ticularly in distinguishing between surgical instruments and adjacent tissues. Curriculum Learning for Hybrid Loss To mitigate the potential instability of integrating the RL component, a learn￾ing strategy is employed in the curriculumlearning. The total training loss is defined as a weighted combination of the segmentation loss L… view at source ↗
Figure 3
Figure 3. Qualitative segmentation results on EndoVis 2018 showing CLIP-RL [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 31 canonical work pages

  1. [13]

    S3Net: A Single Stream Structure for Depth Guided Image Relighting

    H.-H. Yang, W.-T. Chen, and S.-Y . Kuo, “S3Net: A Single Stream Structure for Depth Guided Image Relighting,” 2021, arXiv preprint arXiv:2105.00681. [Online]. Available: https://arxiv.org/abs/2105.00681

    work page Pith review arXiv 2021

  2. [1]

    The Rise of Minimally Invasive Surgery: 16 Year Analysis of the Progres- sive Replacement of Open Surgery with Laparoscopy

    St John A, Caturegli I, Kubicki NS, Kavic SM. The Rise of Minimally Invasive Surgery: 16 Year Analysis of the Progres- sive Replacement of Open Surgery with Laparoscopy. JSLS. 2020 Oct-Dec;24(4):e2020.00076. doi: 10.4293/JSLS.2020.00076. PMID: 33510568; PMCID: PMC7810432

    arXiv 2020

  3. [2]

    Minimally Invasive Versus Open Lumbar Fusion: A Comparison of Blood Loss, Surgical Complications, and Hospital Course

    Patel AA, Zfass-Mendez M, Lebwohl NH, Wang MY , Green BA, Levi AD, Vanni S, Williams SK. Minimally Invasive Versus Open Lumbar Fusion: A Comparison of Blood Loss, Surgical Complications, and Hospital Course. Iowa Orthop J . 2015;35:130-4. PMID: 26361455; PMCID: PMC4492142

    work page 2015

  4. [3]

    Minimally invasive spine surgery decreases postoperative pain and inflammation for patients with lumbar spinal stenosis

    Peng H, Tang G, Zhuang X, Lu S, Bai Y , Xu L. Minimally invasive spine surgery decreases postoperative pain and inflammation for patients with lumbar spinal stenosis. Exp Ther Med . 2019 Oct;18(4):3032-3036. doi: 10.3892/etm.2019.7917. Epub 2019 Aug 20. PMID: 31555386; PMCID: PMC6755410

    work page arXiv 2019

  5. [4]

    Impact of minimally invasive surgery on immune function and stress response in gastric cancer patients,

    R. H. Zhu, P. C. Li, J. Zhang, and H. H. Song, “Impact of minimally invasive surgery on immune function and stress response in gastric cancer patients,” World Journal of Gastrointestinal Surgery , vol. 16, no. 8, pp. 2484–2493, 2024. DOI: 10.4240/wjgs.v16.i8.2484. [Online]. Available: https://doi.org/10.4240/wjgs.v16.i8.2484

    work page doi:10.4240/wjgs.v16.i8.2484 2024

  6. [5]

    Does Minimally Invasive Mitral Valve Repair Mean Less Postoperative Pain? The Annals of Thoracic Surgery

    Jahanian S, Arghami A, Wittwer ED, King KS, Daly RC, Dearani JA, Rowse PG, Crestanello JA, Schaff HV . Does Minimally Invasive Mitral Valve Repair Mean Less Postoperative Pain? The Annals of Thoracic Surgery . 2023;115(5):1172-

    work page 2023

  7. [6]

    Artificial intelligence- based computer vision in surgery: Recent advances and future perspectives

    Kitaguchi D, Takeshita N, Hasegawa H, Ito M. Artificial intelligence- based computer vision in surgery: Recent advances and future perspectives. Ann Gastroenterol Surg . 2021 Oct 8;6(1):29-36. doi: 10.1002/ags3.12513. PMID: 35106412; PMCID: PMC8786689

    work page doi:10.1002/ags3.12513 2021

  8. [7]

    Digital tools and innovative healthcare solutions: Serious games and gamification in surgical training and patient care,

    S. Kharbech, J. Abinahed, O. Aboumarzouk, W. El Ansari, A. Al Ansari, and S. Balakrishnan, “Digital tools and innovative healthcare solutions: Serious games and gamification in surgical training and patient care,” in *Artificial Intelligence, Big Data, Blockchain and 5G for the Digital Transformation of the Healthcare Industry*, vol. 3, P. O. De Pablos an...

    work page 2024

Show all 47 references
  1. [8]

    Computer Vi- sion in the Operating Room: Opportunities and Caveats,

    L. R. Kennedy-Metz, P. Mascagni, A. Torralba, R. D. Dias, P. Perona, J. A. Shah, N. Padoy, and M. A. Zenati, “Computer Vi- sion in the Operating Room: Opportunities and Caveats,” IEEE Transactions on Medical Robotics and Bionics , vol. 3, no. 1, pp. 2–10, 2021. DOI: 10.1109/tm...

    2021

  2. [9]

    Novel applications of deep learning in surgical training,

    S. Balakrishnan, S. P. Dakua, W. El Ansari, O. Aboumarzouk, and A. Al Ansari, “Novel applications of deep learning in surgical training,” in *Artificial Intelligence, Big Data, Blockchain and 5G for the Digital Transformation of the Healthcare Industry*, vol. 3, P. O. De Pablo...

    2024

  3. [10]

    Segmentation of surgical instruments in laparoscopic videos: training dataset generation and deep-learning- based framework

    Lee, Eung-Joo, Plishker, William, Liu, Xinyang, Kane, Timothy, Bhat- tacharyya, Shuvra, Shekhar, Raj. Segmentation of surgical instruments in laparoscopic videos: training dataset generation and deep-learning- based framework. Proc. SPIE Medical Imaging . 2019 Mar; 10951:65. d...

    2019 doi

  4. [11]

    Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning

    Goodman ED, Patel KK, Zhang Y , Locke W, Kennedy CJ, Mehrotra R, Ren S, Guan M, Zohar O, Downing M, Chen HW, Clark JZ, Berrigan MT, Brat GA, Yeung-Levy S. Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning. JAMA Surg. 2024 Feb 1;159(2)...

    2024

  5. [12]

    Deep learning for surgical instrument recognition and segmentation in robotic- assisted surgeries: a systematic review,

    F. A. Ahmed, M. Yousef, M. A. Ahmed, H. O. Ali, A. Mahboob, H. Ali, Z. Shah, O. Aboumarzouk, A. Al Ansari, and S. Balakrishnan, “Deep learning for surgical instrument recognition and segmentation in robotic- assisted surgeries: a systematic review,” Artificial Intelligence Rev...

    2024 doi

  6. [14]

    MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation,

    N. Ayobi, A. P ´erez-Rond´on, S. Rodr ´ıguez, and P. Arbel ´aez, “MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation,” in 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI) , pp. 1–5, IEEE, 2023. DOI: 10.1109/ISBI53787.2023.10230819. [On...

    2023

  7. [15]

    U-Net: Convolutional Net- works for Biomedical Image Segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Net- works for Biomedical Image Segmentation,” 2015, arXiv preprint arXiv:1505.04597. [Online]. Available: https://arxiv.org/abs/1505.04597

    2015 arXiv

  8. [16]

    Segment Anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Girshick, “Segment Anything,” 2023, arXiv preprint arXiv:2304.02643 . [Online]. Available: https://arxiv.org/abs/2304.02643

    2023 arXiv

  9. [17]

    Track Anything: Segment Anything Meets Videos,

    J. Yang, M. Gao, Z. Li, S. Gao, F. Wang, and F. Zheng, “Track Anything: Segment Anything Meets Videos,” 2023, arXiv preprint arXiv:2304.11968. [Online]. Available: https://arxiv.org/abs/2304.11968

    2023 arXiv

  10. [18]

    Personalize Segment Anything Model with One Shot,

    R. Zhang, Z. Jiang, Z. Guo, S. Yan, J. Pan, X. Ma, H. Dong, P. Gao, and H. Li, “Personalize Segment Anything Model with One Shot,” 2023, arXiv preprint arXiv:2305.03048 . [Online]. Available: https://arxiv.org/abs/2305.03048

    2023 arXiv

  11. [19]

    Sur- gicalSAM: Efficient Class Promptable Surgical Instrument Segmen- tation,

    W. Yue, J. Zhang, K. Hu, Y . Xia, J. Luo, and Z. Wang, “Sur- gicalSAM: Efficient Class Promptable Surgical Instrument Segmen- tation,” 2023, arXiv preprint arXiv:2308.08746 . [Online]. Available: https://arxiv.org/abs/2308.08746

    2023 arXiv

  12. [20]

    DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmen- tation,

    Y . Gao, W. Xia, D. Hu, W. Wang, and X. Gao, “DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmen- tation,” arXiv preprint arXiv:2306.00499, 2024. [Online]. Available: https://arxiv.org/abs/2306.00499

    2024 arXiv

  13. [21]

    Segmentation of Brain Tumor in MRI Scans Using Deep Learning,

    B. J. Frey et al., “Segmentation of Brain Tumor in MRI Scans Using Deep Learning,” *Medical Image Analysis*, vol. 50, pp. 98-110, 2019

    2019

  14. [22]

    Object Segmentation for Robotic Grasping in Cluttered Environments,

    T. Lu et al., “Object Segmentation for Robotic Grasping in Cluttered Environments,” in *IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*, 2020

    2020

  15. [23]

    A. K. Jain, *Fundamentals of Digital Image Processing*, Prentice-Hall, 1989

    1989

  16. [24]

    U-Net: Convolutional Net- works for Biomedical Image Segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Net- works for Biomedical Image Segmentation,” in *MICCAI*, 2015

    2015

  17. [25]

    Dermatologist-level Classification of Skin Cancer with Deep Neural Networks,

    M. Esteva et al., “Dermatologist-level Classification of Skin Cancer with Deep Neural Networks,” *Nature*, vol. 542, pp. 115-118, 2017

    2017

  18. [26]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,

    A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in *ICLR*, 2021

    2021

  19. [27]

    UNETR: Transformers for 3D Medical Image Segmentation,

    M. Hatamizadeh et al., “UNETR: Transformers for 3D Medical Image Segmentation,” *IEEE CVPR*, 2022

    2022

  20. [28]

    Segmenter: Transformer for Semantic Segmentation,

    R. Strudel et al., “Segmenter: Transformer for Semantic Segmentation,” in *ICCV*, 2021

    2021

  21. [29]

    SETR: Vision Transformers for Semantic Segmenta- tion,

    J. Zheng et al., “SETR: Vision Transformers for Semantic Segmenta- tion,” in *CVPR*, 2021

    2021

  22. [30]

    TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation,

    J. Chen et al., “TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation,” *arXiv preprint arXiv:2102.04306*, 2021

    2021 arXiv

  23. [31]

    Prostate Segmentation Using Hybrid Transformer-CNN Models,

    P. Hatamizadeh et al., “Prostate Segmentation Using Hybrid Transformer-CNN Models,” in *MICCAI*, 2022

    2022

  24. [32]

    SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers,

    E. Xie et al., “SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers,” in *NeurIPS*, 2021

    2021

  25. [33]

    Learning Transferable Visual Models From Natural Language Supervision,

    A. Radford et al., “Learning Transferable Visual Models From Natural Language Supervision,” in *ICML*, 2021

    2021

  26. [34]

    Zero-Shot Medical Image Segmentation Using CLIP,

    J. Lin et al., “Zero-Shot Medical Image Segmentation Using CLIP,” in *Medical Image Computing and Computer-Assisted Intervention (MICCAI)*, 2022

    2022

  27. [35]

    BLIP: Bootstrapped Language-Image Pre-training for Uni- fied Vision-Language Understanding and Generation,

    J. Li et al., “BLIP: Bootstrapped Language-Image Pre-training for Uni- fied Vision-Language Understanding and Generation,” *arXiv preprint arXiv:2201.12086*, 2022

    2022 arXiv

  28. [36]

    Vision-Language Models for Surgical Assistance: A Case Study in Laparoscopic Surgery,

    A. G. Howard et al., “Vision-Language Models for Surgical Assistance: A Case Study in Laparoscopic Surgery,” in *IEEE Transactions on Medical Imaging*, vol. 42, no. 3, pp. 759-772, 2023

    2023

  29. [37]

    A Sim- ple Framework for Contrastive Learning of Visual Representa- tions,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A Sim- ple Framework for Contrastive Learning of Visual Representa- tions,” arXiv preprint arXiv:2002.05709, 2020. [Online]. Available: https://arxiv.org/abs/2002.05709

    2002 arXiv

  30. [38]

    R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018

    2018

  31. [39]

    Bengio, J

    Y . Bengio, J. Louradour, R. Collobert, and J. Weston, ”Curriculum Learning,” Proc. 26th Int. Conf. Mach. Learn. (ICML), 2009, pp. 41–48

    2009

  32. [40]

    C. E. Shannon, ”A Mathematical Theory of Communication,” Bell Syst. Tech. J., vol. 27, no. 3, pp. 379–423, 1948

    1948

  33. [41]

    T. Sørensen, ”A Method of Establishing Groups of Equal Amplitude in Plant Sociology Based on Similarity of Species Content and Its Appli- cation to Analyses of the Vegetation on Danish Commons,” Kongelige Danske Videnskabernes Selskab, vol. 5, no. 4, pp. 1–34, 1948

    1948

  34. [42]

    L. R. Dice, ”Measures of the Amount of Ecologic Association Between Species,” Ecology, vol. 26, no. 3, pp. 297–302, 1945

    1945

  35. [43]

    D. P. Kingma and J. Ba, ”Adam: A Method for Stochastic Optimization,” arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  36. [44]

    2017 Robotic Instrument Segmentation Challenge,

    M. Allan, A. Shvets, T. Kurmann, Z. Zhang, R. Duggal, Y .-H. Su, N. Rieke, I. Laina, N. Kalavakonda, S. Bodenstedt, L. Herrera, W. Li, V . Iglovikov, H. Luo, J. Yang, D. Stoyanov, L. Maier-Hein, S. Speidel, and M. Azizian, “2017 Robotic Instrument Segmentation Challenge,” arXi...

    2017 arXiv

  37. [45]

    2018 Robotic Scene Segmentation Challenge,

    M. Allan, S. Kondo, S. Bodenstedt, S. Leger, R. Kadkhodamohammadi, I. Luengo, F. Fuentes, E. Flouty, A. Mohammed, M. Pedersen, A. Kori, V . Alex, G. Krishnamurthi, D. Rauber, R. Mendel, C. Palm, S. Bano, G. Saibro, C.-S. Shih, H.-A. Chiang, J. Zhuang, J. Yang, V . Iglovikov, A...

    2018 arXiv

  38. [46]

    ISINet: An Instance-Based Approach for Surgical Instrument Segmentation,

    C. Gonz ´alez, L. Bravo-S ´anchez, and P. Arbelaez, “ISINet: An Instance-Based Approach for Surgical Instrument Segmentation,” arXiv preprint arXiv:2007.05533, 2020. [Online]. Available: https://arxiv.org/abs/2007.05533

    2007 arXiv

  39. [1178]

    Available from: https://www.sciencedirect.com/science/article/pii/S0003497522014308

    doi: 10.1016/j.athoracsur.2022.11.009. Available from: https://www.sciencedirect.com/science/article/pii/S0003497522014308

    2022 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.