Pith. sign in

REVIEW 3 major objections 5 minor 53 references

Phase-Informed Tool Segmentation for Manual Small-Incision Cataract Surgery

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Phase information as a prior improves cataract tool segmentation by 38 percent.

desk verdict Sankara-MSICS is a genuine and useful dataset contribution; the ToolSeg gains are plausible but the pseudo-label protocol needs fold-wise clarification before the headline numbers carry weight. read the letter →

arxiv 2411.16794 v2 pith:OZBNBM3P submitted 2024-11-25 cs.CV

Bhuvan Sachdeva , Naren Akash , Tajamul Ashraf , Simon Mueller , Thomas Schultz , Maximilian W. M. Wintergerst , Niharika Singri Prasad , Kaushik Murali
show 1 more author
Mohit Jain
This is my paper · ORCID
classification cs.CV
keywords surgicaltoolsegmentationcataractsurgeryMSICSphase-conditionedsemi-supervisedlearningSAM2pseudo-labelsphaserecognitiondataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Sankara-MSICS, the first dataset of Manual Small-Incision Cataract Surgery (MSICS) videos, with 18 surgical phases and 13 pixel-level tool classes across 53 videos. It argues that knowing the surgical phase improves tool segmentation because phases and tools are strongly correlated, and proposes ToolSeg, a U-Net with a phase-conditional decoder, plus a semi-supervised step that uses SAM 2 to propagate expert masks to unlabeled frames. On Sankara-MSICS, ToolSeg raises mean Dice from 50.66 (U-Net baseline) to 69.96, a 38.10% relative gain, and also improves on the CaDIS dataset. The authors claim the gains are largest for small and less frequent tools.

What carries the argument

The central object is the Phase-informed Conditional Decoder (PCD) layer, inserted at each decoder level of a U-Net. It has three components: Phase-aware Affine Feature Transform (PAFT) applies per-phase channel-wise shift and scale ($f' = \gamma_p \odot f + \beta_p$); Dynamic Feature Blending Factor (DFBF) computes a spatial blending factor $\alpha = (H(f) + \eta_p) / 2$; Context-Aware Adaptive Gating (CGate) fuses phase-modulated and original features as $f'' = f' \cdot \alpha + f \cdot (1 - \alpha)$. The other load-bearing mechanism is the semi-supervised pipeline: SAM 2, prompted with points sampled from expert masks, propagates masks to neighboring frames (every 30th frame up to 90 frames away), generating 20,878 pseudo-labeled frames from 3,527 manually labeled ones.

What would settle it

Evaluate ToolSeg v3 (predicted phases) and v6 (ground-truth phases) on a large, realistically distributed MSICS test set and compare their DSC to a plain U-Net; if the predicted-phase model does not beat the U-Net by a substantial margin, the practical benefit of phase conditioning under realistic phase errors is not established.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that surgical phase information is a usable prior for tool segmentation in cataract surgery: the same-looking tool can appear in different phases, and the network can disambiguate it when told which phase it is in. The evidence is a set of ablation variants on the new Sankara-MSICS dataset: conditioning the decoder on ground-truth phases (v6) lifts mean IoU from 40.90 to 56.13 and DSC from 50.66 to 64.76; adding SAM 2 pseudo-labels (v7) lifts IoU to 61.62 and DSC to 69.96. The paper further claims the method transfers to a different cataract setting (CaDIS), where ToolSeg v6 reaches 60.73 IoU / 68.63 DSC, outperforming the same baselines. The authors are careful to state that with predicted (not ground-truth) phases the gain is smaller, with v3 reaching 57.52 DSC, and that the phase predictor itself is only 50.10% accurate.

Load-bearing premise

The method's headline gains assume that each frame's surgical phase is known correctly; with the paper's own phase predictor (which is only 50.10% accurate), the improvement over a plain U-Net shrinks from 19.3 to 6.9 DSC points.

Editorial extensions

If this is right

  • ToolSeg's phase-conditioned decoder plus SAM 2 pseudo-labels yields a 38.10% relative gain in mean DSC over a U-Net baseline on Sankara-MSICS, with the largest gains for rare and small tools, e.g., Rhexis Needle DSC jumps from 13.98 to 47.22.
  • Ablations show that both components contribute independently: phase conditioning with ground-truth phases raises DSC to 64.76, and pseudo-labels alone raise it to 58.29; together (v7) they reach 69.96.
  • Using only 50% of the manually annotated frames plus pseudo-labeled frames, the model reaches 65.92 DSC, exceeding the fully supervised baseline (64.76), indicating that the pseudo-labeling pipeline can halve annotation effort.
  • ToolSeg transfers to the CaDIS dataset (Phaco cataract surgery), where v6 achieves 68.63 DSC, outperforming U-Net (62.84) and other baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the practical gain with MS-TCN++-predicted phases (v3) is only 57.52 DSC versus 64.76 with ground-truth phases, the real-world benefit of phase conditioning hinges on phase-prediction accuracy; an end-to-end model that jointly predicts phases and segments tools could close part of this gap.
  • The phase–tool co-occurrence prior is likely to be weaker in complex or anomalous surgeries where tools are used out of their typical phase; testing ToolSeg on videos of complications would show whether phase conditioning degrades gracefully.
  • The SAM 2 pseudo-label pipeline with point prompting could be reused for other sparsely annotated surgical video datasets, but its quality depends on temporal coherence and the absence of specular reflections; the paper's own error examples (e.g., reflections from the surgeon lamp) suggest the propagation quality varies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Sankara-MSICS, a new dataset of 53 manual small-incision cataract surgery videos with phase labels for 18 surgical phases and pixel-level annotations of 13 surgical tools in 3,527 frames. It proposes ToolSeg, a U-Net-like segmentation network with a phase-conditioned decoder using PAFT, DFBF, and CGate modules, and a semi-supervised extension that propagates ground-truth masks with SAM 2 to create 20,878 pseudo-labeled frames. On Sankara-MSICS, ToolSeg v7 (ground-truth phases + pseudo-labels) reaches 69.96 DSC versus 50.66 DSC for the U-Net baseline, and the method also improves over baselines on CaDIS. The authors benchmark several baselines, report ablations in Table 1, and provide per-tool results in Table 4.

Significance. If the reported results hold, this is a useful contribution to surgical video analysis. Sankara-MSICS appears to be the first pixel-level tool-segmentation dataset for MSICS, a procedure with high relevance in low-resource settings, and the phase-conditioned decoder is a sensible inductive bias given the strong phase-tool co-occurrence shown in Figure 2. The paper also provides a fairly complete ablation study, five-fold cross-validation at the video level, and a cross-dataset test on CaDIS. The SAM 2 pseudo-labeling pipeline is a reasonable way to use unlabeled frames without manual annotation. However, the central quantitative claims rest on a semi-supervised protocol whose evaluation details are not fully specified, and the practical benefit of phase conditioning depends on the relatively weak phase predictor (50.10% accuracy), which the paper reports but does not foreground. These issues need to be addressed before the reported margins can be taken at face value.

major comments (3)
  1. [§4.4 and Supplementary §3.3, Table 7] The pseudo-labeling procedure as described may leak test-video content into the training set. Section 4.4 generates 20,878 pseudo-labeled frames 'from our base dataset of 3,527 annotated frames' by propagating each expert mask forward and backward 90 frames and sampling every 30th frame. The five-fold cross-validation is video-level (Supplementary Table 7), but the text never states that pseudo-labels generated from annotated frames belonging to a fold's test videos are excluded from that fold's training set. Since the base 3,527 frames include the test frames, a SAM 2 propagation seeded by a test-video annotation can place frames from that same test video into training, even if the exact test frames are not duplicated. This would inflate the pseudo-label contributions in v1, v4, and v7. The manuscript must report fold-wise pseudo-label provenance, and ideally rerun the ablations with pseudo-labels restricted to training/validation videos, before the semi-supervised results can be interpreted.
  2. [Abstract and Table 1] The headline '38.10% increase in mean Dice scores' refers to ToolSeg v7, which uses ground-truth phases as input. In a real deployment the phase source is MS-TCN++ predictions, and the corresponding variant with pseudo-data is v4, which reaches 62.70 DSC, a 23.77% relative gain over the v0 baseline. The paper does report v4 and v3 in Table 1, and the Limitations section acknowledges a gap between predicted and ground-truth phases, but the abstract and introduction emphasize the ground-truth-phase result without stating that it is an upper bound. I recommend making the predicted-phase variant the primary reported result, or at least clearly labeling the GT-phase variants as an oracle upper bound in the abstract and contribution list.
  3. [§5.2, Table 3] The comparison to ISINet and MATIS-Frame may be confounded by model capacity and training protocol. ISINet and MATIS are instance-segmentation methods trained for different settings, and MATIS-Frame in particular obtains only 18.24 DSC, which is far below U-Net. The paper attributes this to the ocular domain, but it is not clear whether these baselines were given the same training budget, learning rate, or data augmentation as ToolSeg, or whether they were fine-tuned from their original checkpoints. The text should state the exact training setup for each baseline so the comparison is fair and reproducible.
minor comments (5)
  1. [Table 3] The standard deviation for ToolSeg v4 is printed as '± 34.4' in the IoU column; this is almost certainly a typo for '± 4.4', matching Table 1.
  2. [§5.2, text after Table 3] The sentence 'followed by U-Net with an IoU of 40.90 and DSC of 52.03' gives U-Net a DSC of 52.03, but Table 3 reports U-Net DSC as 50.66. The value 52.03 belongs to TernausNet.
  3. [§5.2, CaDIS paragraph] The CaDIS experiment uses '13 tools across 18 surgical phases' without specifying how the 29 CaDIS classes were reduced to 13 or how the 18 phases were obtained for CaDIS. This selection should be described explicitly so the cross-dataset comparison is reproducible.
  4. [Supplementary §3.3] The pseudo-label quality check reports an average DSC of 66.06 and IoU of 55.79 against expert labels on 654 common frames. This is a useful sanity check, but it would be helpful to report whether these 654 frames overlap with the training folds and whether low-quality propagations (e.g., the reflection artifact in Figure 8) are filtered before training.
  5. [Throughout] There are several typos: 'complimentary' should be 'complementary', 'gasterointestinal' should be 'gastrointestinal', and the phrase 'with a notable boost for tools that are less prevalent and small' would read better as 'small and less prevalent tools'.

Circularity Check

1 steps flagged · score 6.0 of 10

Pseudo-label evaluation may leak test-video content into training: SAM2 propagates from all 3,527 annotated frames with no fold-wise restriction, so the reported DSC gains for v1/v7 are not independent predictions.

  1. fitted input called prediction [Section 4.4 and Supplementary Sections 3.2/3.3]
    "Using this method, we generated 20,878 pseudo-labeled frames from our base dataset of 3,527 annotated frames. ... From the 53 MSICS videos in our dataset, we generated five stratified data splits, each containing 7-9 test videos. ... The dataset is split into five equal test sets at the video level, with the remaining data randomly divided into train and validation sets at an 80:20 ratio."

    The base dataset of 3,527 annotated frames includes the test videos of every fold, yet the pseudo-label generation step is described without any fold-wise restriction. SAM2 propagates each expert mask forward and backward across 90 frames of the same video, so pseudo-labels derived from test-video ground-truth frames can enter the training set for that fold. The model is then evaluated on video content it has already seen through propagated pseudo-labels, making the reported semi-supervised gains (v0->v1, v6->v7) reflect test-label leakage rather than generalization. The paper never specifies that pseudo-labels were generated only from training-split frames, so the central 'prediction' claim is not shown to be independent of its test inputs.

full rationale

The phase-conditioning contribution is not circular: conditioning a decoder on phase is an inductive bias derived from an observed dataset correlation, and the phase/tool co-occurrence is measured on the same dataset only as a motivation, not as an evaluation target. The CaDIS generalization experiment provides external evidence and is not self-referential. The main circularity risk is in the semi-supervised evaluation: the pseudo-label pipeline starts from all 3,527 expert-annotated frames and propagates masks within each video, but the paper's video-level five-fold split is described only for the labeled evaluation frames. As written, there is no guarantee that pseudo-labels from a fold's test videos are excluded from training. If they are not excluded, the reported DSC improvement from pseudo-labels is partly a self-fulfilling result, because test-video appearance has been injected into the training set. This is an evaluation-protocol circularity rather than a mathematical derivation collapse, but it directly affects the paper's headline claim that SAM2 pseudo-labels improve tool segmentation. No other circular steps were found: the architecture equations are novel combinations, the SAM2 labels come from an external foundation model, and the phase-recognition model was trained on an independent internal video set.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the stability of phase-tool correlations, the completeness of the MSICS label schema, the reliability of SAM2 pseudo-labels, and the split design. The method's parameters are standard learned weights; no new physical entities or postulates are introduced.

free parameters (7)
  • PAFT per-phase embeddings gamma_p, beta_p = Learned during training
    Core conditioning parameters introduced in Eq. 1 of Section 4.3; their values are fit to the Sankara-MSICS training data.
  • DFBF per-phase embedding eta_p = Learned during training
    Used in the blending factor computation alpha = (H(f) + eta_p)/2; fitted to the training set.
  • DFBF convolution H = Learned during training
    The convolution that extracts spatial features for the blending factor; trained on the dataset.
  • U-Net encoder-decoder weights = Learned during training
    The base segmentation network is trained from scratch on Sankara-MSICS; no pretrained weights are mentioned.
  • MS-TCN++ phase recognition weights = Trained on internal 90-video dataset
    The phase predictor that produces predicted phases for variants v2, v3, v4; its 50.10% accuracy bounds the benefit.
  • Denoising thresholds = 100 px, 10% mask area, Gaussian kernel 25
    Hand-chosen in the annotation pipeline to clean SAM-generated masks; affects label quality.
  • Pseudo-label sampling scheme = 30-frame stride, 90-frame window
    Chosen by hand; determines the size and temporal coverage of pseudo-labeled data.
assumptions (4)
  • domain assumption The phase-tool co-occurrence pattern observed in the 53 Sankara-MSICS videos is a stable and generalizable prior for MSICS tool segmentation.
    ToolSeg conditions on phase; if phase-tool correlation is weaker in other hospitals or surgical settings, the reported gains may not transfer. See Figure 2 and Section 4.1.
  • domain assumption The 18-phase taxonomy and 13-tool set defined by two resident ophthalmologists is a complete and clinically valid representation of MSICS.
    The dataset and all experiments depend on this label schema, but no external expert panel validated it. See Section 3.
  • domain assumption SAM2-generated pseudo-labels are reliable enough to provide a useful training signal.
    The paper's own check on 654 common frames reports only 66.06% DSC, so roughly a third of pseudo-labeled pixels may be wrong; training still helps, but the assumption is not fully validated. See Supplementary Section 3.3.
  • domain assumption The video-level data split prevents information leakage between train and test.
    Splits are at video level, but because all videos come from the same hospital and microscope setup, some domain similarity is expected. See Section 5.1 and Table 7.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Phase-Informed Tool Segmentation for Manual Small-Incision Cataract Surgery." pith.science (2026). https://pith.science/paper/OZBNBM3P

@misc{pith2026241116794,
  author       = {Pith},
  title        = {Pith review of: Phase-Informed Tool Segmentation for Manual Small-Incision Cataract Surgery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZBNBM3P}},
  note         = {Machine review of arXiv:2411.16794}
}
abstract

Cataract surgery is the most common surgical procedure globally, with a disproportionately higher burden in developing countries. While automated surgical video analysis has been explored in general surgery, its application to ophthalmic procedures remains limited. Existing works primarily focus on Phaco cataract surgery, an expensive technique not accessible in regions where cataract treatment is most needed. In contrast, Manual Small-Incision Cataract Surgery (MSICS) is the preferred low-cost, faster alternative in high-volume settings and for challenging cases. However, no dataset exists for MSICS. To address this gap, we introduce Sankara-MSICS, the first comprehensive dataset containing 53 surgical videos annotated for 18 surgical phases and 3,527 frames with 13 surgical tools at the pixel level. We benchmark this dataset on state-of-the-art models and present ToolSeg, a novel framework that enhances tool segmentation by introducing a phase-conditional decoder and a simple yet effective semi-supervised setup leveraging pseudo-labels from foundation models. Our approach significantly improves segmentation performance, achieving a $23.77\%$ to $38.10\%$ increase in mean Dice scores, with a notable boost for tools that are less prevalent and small. Furthermore, we demonstrate that ToolSeg generalizes to other surgical settings, showcasing its effectiveness on the CaDIS dataset.

Figures

Figures reproduced from arXiv: 2411.16794 by the authors.

Figure 1
Figure 1. Although the surgical tools in Images A and B appear [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Surgical phase and surgical tool co-occurence. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of the ToolSeg network architecture. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Qualitative results of our proposed ToolSeg with SOTA methods. (A) Hydrodissection Cannula, (B) Hoskins Forceps and Rhexis Needle, (C) Hoskins Forceps and Keratome, (D) Hoskins Forceps and Blade, and (E) Vectis and Dialer [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: SAM based annotation web tool. 1.2. MSICS Details The surgical workflow of a standard MSICS procedure is outlined in [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 6
Figure 6. Figure 6: Comparison of surgical scenes: a laparoscopic pro [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 8
Figure 8. Figure 8: Surgical tool label propagation using SAM 2 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Confusion matrix illustrating the performance of surgical phase prediction on the Sankara-MSICS dataset using MSTCN++ [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Visualization of feature transformation in the proposed PCD-Gated layer. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 44 canonical work pages

  1. [1]

    Cataracts: Challenge on auto- matic tool annotation for cataract surgery

    Hassan Al Hajj, Mathieu Lamard, Pierre-Henri Conze, Soumali Roychowdhury, Xiaowei Hu, Gabija Mar ˇsalkait˙e, Odysseas Zisimopoulos, Muneer Ahmad Dedmari, Fenqiang Zhao, Jonas Prellberg, et al. Cataracts: Challenge on auto- matic tool annotation for cataract surgery. Medical image analysis, 52:24–41, 2019. 2

    work page 2019

  2. [2]

    Cataract surgery

    Jorge L Ali ´o, H Burkhard Dick, and Robert H Osher, editors. Cataract surgery. Springer Nature, Cham, Switzerland, 1 edition, 2022. 2, 3

    work page 2022

  3. [3]

    Matis: Masked-attention transformers for surgical instrument segmentation

    Nicol ´as Ayobi, Alejandra P ´erez-Rond´on, Santiago Rodr´ıguez, and Pablo Arbel ´aez. Matis: Masked-attention transformers for surgical instrument segmentation. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pages 1–5. IEEE, 2023. 3, 7, 8

    work page 2023

  4. [4]

    American college of surgeons and surgical infection society: surgical site infection guidelines, 2016 update

    Kristen A Ban, Joseph P Minei, Christine Laronga, Brian G Harbrecht, Eric H Jensen, Donald E Fry, Kamal MF Itani, Patchen E Dellinger, Clifford Y Ko, and Therese M Duane. American college of surgeons and surgical infection society: surgical site infection guidelines, 2016 update. Journal of the American College of Surgeons, 224(1):59–74, 2017. 2

    work page 2016

  5. [5]

    Comparative evaluation of instru- ment segmentation and tracking methods in minimally inva- sive surgery

    Sebastian Bodenstedt, Max Allan, Anthony Agustinos, Xi- aofei Du, Luis Garcia-Peraza-Herrera, Hannes Kenngott, Thomas Kurmann, Beat M ¨uller-Stich, Sebastien Ourselin, Daniil Pakhomov, et al. Comparative evaluation of instru- ment segmentation and tracking methods in minimally inva- sive surgery. arXiv preprint arXiv:1805.02475, 2018. 2

    arXiv 2018

  6. [6]

    Retinal burns occurring at cataract extraction

    Edwin E Boldrey, Ben T Ho, and Roger D Griffith. Retinal burns occurring at cataract extraction. Ophthalmology, 91 (11):1297–1302, 1984. 1

    work page 1984

  7. [7]

    The lancet global health commission on global eye health: vision beyond 2020

    Matthew J Burton, Jacqueline Ramke, Ana Patricia Marques, Rupert RA Bourne, Nathan Congdon, Iain Jones, Brandon AM Ah Tong, Simon Arunga, Damodar Bachani, Covadonga Bascaran, et al. The lancet global health commission on global eye health: vision beyond 2020. The Lancet Global Health, 9(4):e489–e551, 2021. 1

    work page 2020

  8. [8]

    Per- pixel classification is not all you need for semantic segmen- tation

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per- pixel classification is not all you need for semantic segmen- tation. Advances in neural information processing systems , 34:17864–17875, 2021. 3

    2021

Show all 53 references
  1. [9]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 3

    2022

  2. [10]

    Cataracts

    Maria Vittoria Cicinelli, John C Buchan, Maneck Nicholson, Varshini Varadaraj, and Rohit C Khanna. Cataracts. The Lancet, 401(10374):377–389, 2023. 1

    2023

  3. [11]

    Current state of sur- gical lighting

    Jahnavi Curlin and Charles K Herman. Current state of sur- gical lighting. The Surgery Journal , 6(02):e87–e97, 2020. 1

    2020

  4. [12]

    Trauma of major surgery: a global prob- lem that is not going away, 2020

    Geoffrey P Dobson. Trauma of major surgery: a global prob- lem that is not going away, 2020. 1

    2020

  5. [13]

    Global, regional, national burden and gender disparity of cataract: findings from the global burden of disease study 2019

    Rui Fang, Yang-Fan Yu, En-Jie Li, Ning-Xin Lv, Zhao- Chuan Liu, Hong-Gang Zhou, and Xu-Dong Song. Global, regional, national burden and gender disparity of cataract: findings from the global burden of disease study 2019. BMC Public Health, 22(1):2068, 2022. 2

    2019

  6. [14]

    Pixel-based tool segmentation in cataract surgery videos with mask r-cnn

    Markus Fox, Mario Taschwer, and Klaus Schoeffmann. Pixel-based tool segmentation in cataract surgery videos with mask r-cnn. In 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS), pages 565–568. IEEE, 2020. 2

    2020

  7. [15]

    Toolnet: holistically-nested real-time segmentation of robotic surgical tools

    Luis C Garcia-Peraza-Herrera, Wenqi Li, Lucas Fidon, Cas- par Gruijthuijsen, Alain Devreker, George Attilakos, Jan De- prest, Emmanuel Vander Poorten, Danail Stoyanov, Tom Vercauteren, et al. Toolnet: holistically-nested real-time segmentation of robotic surgical tools. In 201...

    2017

  8. [16]

    Cataract-1k dataset for deep-learning-assisted analysis of cataract surgery videos

    Negin Ghamsarian, Yosuf El-Shabrawi, Sahar Nasiri- haghighi, Doris Putzgruber-Adamitsch, Martin Zinkernagel, Sebastian Wolf, Klaus Schoeffmann, and Raphael Sznit- man. Cataract-1k dataset for deep-learning-assisted analysis of cataract surgery videos. Scientific data, 11(1):37...

    2024

  9. [17]

    Isinet: an instance-based approach for surgical instru- ment segmentation

    Cristina Gonz ´alez, Laura Bravo-S ´anchez, and Pablo Arbe- laez. Isinet: an instance-based approach for surgical instru- ment segmentation. In International Conference on Medi- cal Image Computing and Computer-Assisted Intervention , pages 595–605. Springer, 2020. 3, 7, 8

    2020

  10. [18]

    Cadis: Cataract dataset for surgical rgb-image segmentation

    Maria Grammatikopoulou, Evangello Flouty, Abdolrahim Kadkhodamohammadi, Gwenol ´e Quellec, Andre Chow, Jean Nehme, Imanol Luengo, and Danail Stoyanov. Cadis: Cataract dataset for surgical rgb-image segmentation. Medi- cal Image Analysis, 71:102053, 2021. 2, 3, 7

    2021

  11. [19]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 3

    2017

  12. [20]

    Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image seg- mentation

    Vladimir Iglovikov and Alexey Shvets. Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image seg- mentation. arXiv preprint arXiv:1801.05746, 2018. 3, 7, 8

    2018 arXiv

  13. [21]

    Learning where to look while tracking instruments in robot-assisted surgery

    Mobarakol Islam, Yueyuan Li, and Hongliang Ren. Learning where to look while tracking instruments in robot-assisted surgery. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 412–

  14. [22]

    Ap- mtl: Attention pruned multi-task learning model for real- time instrument detection and segmentation in robot-assisted surgery

    Mobarakol Islam, VS Vibashan, and Hongliang Ren. Ap- mtl: Attention pruned multi-task learning model for real- time instrument detection and segmentation in robot-assisted surgery. In 2020 IEEE international conference on robotics and automation (ICRA), pages 8433–8439. IEEE, 2020. 3

    2020

  15. [23]

    St-mtl: Spatio-temporal multitask learn- ing model to predict scanpath while tracking instruments in robotic surgery

    Mobarakol Islam, VS Vibashan, Chwee Ming Lim, and Hongliang Ren. St-mtl: Spatio-temporal multitask learn- ing model to predict scanpath while tracking instruments in robotic surgery. Medical Image Analysis, 67:101837, 2021. 3

    2021

  16. [24]

    Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video

    Yueming Jin, Keyun Cheng, Qi Dou, and Pheng-Ann Heng. Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzh...

    2019

  17. [25]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 4, 1

    2023

  18. [26]

    Accurate instance segmentation of surgical instruments in robotic surgery: model refinement and cross-dataset evalua- tion

    Xiaowen Kong, Yueming Jin, Qi Dou, Ziyi Wang, Zerui Wang, Bo Lu, Erbao Dong, Yun-Hui Liu, and Dong Sun. Accurate instance segmentation of surgical instruments in robotic surgery: model refinement and cross-dataset evalua- tion. International Journal of Computer Assisted Radiol...

    2021

  19. [27]

    Ms-tcn++: Multi-stage temporal convolu- tional network for action segmentation

    Shijie Li, Yazan Abu Farha, Yun Liu, Ming-Ming Cheng, and Juergen Gall. Ms-tcn++: Multi-stage temporal convolu- tional network for action segmentation. IEEE transactions on pattern analysis and machine intelligence , 45(6):6647– 6658, 2020. 4, 3

    2020

  20. [28]

    A proposed minimum standard set of outcome measures for cataract surgery

    Imran Mahmud, Thomas Kelley, Caleb Stowell, Aravind Haripriya, Anders Boman, Ingrid Kossler, Nigel Morlet, Suzann Pershing, Konrad Pesudovs, Pik Pin Goh, et al. A proposed minimum standard set of outcome measures for cataract surgery. JAMA ophthalmology, 133(11):1247–1252,

  21. [29]

    m2caiseg: Semantic segmentation of laparoscopic images using convolutional neural networks

    Salman Maqbool, Aqsa Riaz, Hasan Sajid, and Osman Hasan. m2caiseg: Semantic segmentation of laparoscopic images using convolutional neural networks. arXiv preprint arXiv:2008.10134, 2020. 2

    2008 arXiv

  22. [30]

    Corneal structure and transparency

    Keith M Meek and Carlo Knupp. Corneal structure and transparency. Progress in retinal and eye research, 49:1–16,

  23. [31]

    Artificial intelli- gence in cataract surgery: A systematic review.Translational Vision Science & Technology, 13(4):20–20, 2024

    Simon M ¨uller, Mohit Jain, Bhuvan Sachdeva, Payal N Shah, Frank G Holz, Robert P Finger, Kaushik Murali, Maximil- ian WM Wintergerst, and Thomas Schultz. Artificial intelli- gence in cataract surgery: A systematic review.Translational Vision Science & Technology, 13(4):20–20,...

    2024

  24. [32]

    Rasnet: Segmenta- tion for tracking surgical instruments in surgical videos us- ing refined attention segmentation network

    Zhen-Liang Ni, Gui-Bin Bian, Xiao-Liang Xie, Zeng-Guang Hou, Xiao-Hu Zhou, and Yan-Jie Zhou. Rasnet: Segmenta- tion for tracking surgical instruments in surgical videos us- ing refined attention segmentation network. In 2019 41st annual international conference of the IEEE eng...

    2019

  25. [33]

    Raunet: Residual attention u-net for semantic segmentation of cataract surgical instruments

    Zhen-Liang Ni, Gui-Bin Bian, Xiao-Hu Zhou, Zeng-Guang Hou, Xiao-Liang Xie, Chen Wang, Yan-Jie Zhou, Rui-Qi Li, and Zhen Li. Raunet: Residual attention u-net for semantic segmentation of cataract surgical instruments. In Interna- tional Conference on Neural Information Processi...

    2019

  26. [34]

    Pyramid attention aggregation network for semantic segmentation of surgical instruments

    Zhen-Liang Ni, Gui-Bin Bian, Guan-An Wang, Xiao-Hu Zhou, Zeng-Guang Hou, Hua-Bin Chen, and Xiao-Liang Xie. Pyramid attention aggregation network for semantic segmentation of surgical instruments. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 11782– 1...

    2020

  27. [35]

    Surginet: Pyramid attention aggregation and class-wise self- distillation for surgical instrument segmentation

    Zhen-Liang Ni, Xiao-Hu Zhou, Guan-An Wang, Wen- Qian Yue, Zhen Li, Gui-Bin Bian, and Zeng-Guang Hou. Surginet: Pyramid attention aggregation and class-wise self- distillation for surgical instrument segmentation. Medical Image Analysis, 76:102310, 2022. 3

    2022

  28. [36]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 2, 5

    2024 arXiv

  29. [37]

    Surgical tool datasets for machine learning research: a survey

    Mark Rodrigues, Michael Mayo, and Panos Patros. Surgical tool datasets for machine learning research: a survey. In- ternational Journal of Computer Vision, 130(9):2222–2248,

  30. [38]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  31. [39]

    Robust medical instrument segmentation challenge

    Tobias Ross, Annika Reinke, Peter M Full, Martin Wag- ner, Hannes Kenngott, Martin Apitz, Hellena Hempe, Di- ana Mindroc Filimon, Patrick Scholz, Thuy Nuong Tran, et al. Robust medical instrument segmentation challenge

  32. [40]

    Cataract-101: video dataset of 101 cataract surgeries

    Klaus Schoeffmann, Mario Taschwer, Stephanie Sarny, Bernd M ¨unzer, Manfred J ¨urgen Primus, and Doris Putzgru- ber. Cataract-101: video dataset of 101 cataract surgeries. In Proceedings of the 9th ACM multimedia systems conference, pages 421–425, 2018. 2

    2018

  33. [41]

    Automatic instrument segmenta- tion in robot-assisted surgery using deep learning

    Alexey A Shvets, Alexander Rakhlin, Alexandr A Kalinin, and Vladimir I Iglovikov. Automatic instrument segmenta- tion in robot-assisted surgery using deep learning. In 2018 17th IEEE international conference on machine learning and applications (ICMLA), pages 624–628. IEEE, 2018. 3

    2018

  34. [42]

    Review of manual small-incision cataract surgery

    Kamaljeet Singh, Arshi Misbah, Pranav Saluja, and Arun Kumar Singh. Review of manual small-incision cataract surgery. Indian journal of ophthalmology , 65(12): 1281–1288, 2017. 3

    2017

  35. [43]

    Phaco vs

    Martin Spencer. Phaco vs. small-incision. Ophthalmology, 113(2):353, 2006. 2, 3

    2006

  36. [44]

    Defining the standard for surgical video deidentification

    Matthew K Tollefson and Christopher J Ross. Defining the standard for surgical video deidentification. JAMA surgery, 159(1):104–105, 2024. 1

    2024

  37. [45]

    En- donet: a deep architecture for recognition tasks on laparo- scopic videos

    Andru P Twinanda, Sherif Shehata, Didier Mutter, Jacques Marescaux, Michel De Mathelin, and Nicolas Padoy. En- donet: a deep architecture for recognition tasks on laparo- scopic videos. IEEE transactions on medical imaging , 36 (1):86–97, 2016. 2

    2016

  38. [46]

    Artificial intelligence in surgery

    Chris Varghese, Ewen M Harrison, Greg O’Grady, and Eric J Topol. Artificial intelligence in surgery. Nature Medicine, pages 1–12, 2024. 1

    2024

  39. [47]

    High-volume surgery in developing countries

    D Yorston. High-volume surgery in developing countries. Eye, 19(10):1083–1089, 2005. 2

    2005

  40. [48]

    Trasetr: track-to-segment transformer with contrastive query for instance-level instrument segmentation in robotic surgery

    Zixu Zhao, Yueming Jin, and Pheng-Ann Heng. Trasetr: track-to-segment transformer with contrastive query for instance-level instrument segmentation in robotic surgery. In 2022 International conference on robotics and automation (ICRA), pages 11186–11193. IEEE, 2022. 3 Phase-In...

    2022

  41. [50]

    Background 1.1. Ocular vs Laparoscopic Surgery Most computer vision datasets in surgery focus on laparo- scopic procedures, particularly in gastroenterology, leaving other domains like ocular surgeries comparatively under- represented. Ocular surgeries pose unique challenges d...

    2017

  42. [51]

    Dataset 2.1. Annotation Web Tool and Data Processing The annotation tool, built using React.js and Node.js and leveraging Meta’s Segment Anything model (SAM) [25], was designed for labeling cataract surgery instruments. An- notators created masks using positive and negative po...

    1920

  43. [52]

    All results are averaged across five folds

    Implementation Details For all our experiments, the model achieving the highest DSC on the validation set is selected for evaluation on the test set. All results are averaged across five folds. -3s -2s -1s 0s +1s +2s +3s -3s -2s -1s 0s +1s +2s +3s Figure 8. Surgical tool label...

  44. [53]

    Feature Transformation in PCD-Gated Layer In this section, we provide visualizations demonstrating how the PCD-Gated layer interacts with features in the la- tent space

    Results 4.1. Feature Transformation in PCD-Gated Layer In this section, we provide visualizations demonstrating how the PCD-Gated layer interacts with features in the la- tent space. Figure 10 shows feature maps from the third and the fourth decoder stage, illustrating transfo...

  45. [2019]

    arXiv preprint arXiv:2003.10299, 2020. 2

    2003 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.