Pith. sign in

REVIEW 5 major objections 5 minor 33 references

Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes an automated, multi-modal system that combines transcribed trainer speech and surgical video to predict whether real-time feedback leads to a trainee behavior change, reporting AUROC 0.70±0.02 and up to 6.6% improvement…

desk verdict The new task is real, but the input window leaks the outcome, so the reported AUROC is a classification score, not a prediction. read the letter →

arxiv 2411.10919 v1 pith:DMRE7SSK submitted 2024-11-17 cs.LG cs.AIcs.CV

Arushi Gupta , Rafal Kocielnik , Jiayun Wang , Firdavs Nasriddinov , Cherine Yang , Elyssa Wong , Anima Anandkumar , Andrew Hung This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords roboticsurgerysurgicalfeedbackeffectivenesstraineebehaviorchangeself-supervisedlearningvideounderstandingmultimodalAUROC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that feedback effectiveness in live robotic surgery—defined as whether a trainer's verbal feedback produces an immediate change in trainee behavior—can be predicted automatically from two modalities: the transcribed words of the feedback and the endoscopic video around the moment it is delivered. The authors argue that such a system matters because human rating of feedback effectiveness is slow, prone to bias, and expert-intensive, and that an objective, scalable predictor could help trainers improve their feedback and eventually support real-time feedback systems. They report that text alone reaches an area under the ROC curve (AUROC) of 0.66 and video alone 0.61, while fusing both after self-supervised fine-tuning of the video encoder reaches AUROC 0.70±0.02, a relative improvement of up to 6.6% over the text-only baseline. They also show that self-supervised fine-tuning on unlabeled surgical video improves prediction, with fine-tuning on feedback-specific clips (only 14.8% of the data) performing comparably to fine-tuning on all surgical video. The main stated caveat is that feedback end times are unknown, so a fixed 10-second window around the annotated feedback onset may include unrelated speech or miss delayed trainee responses.

What carries the argument

The load-bearing mechanism is a two-stream fusion model. On the text side, Whisper ASR transcribes 10 seconds of audio per feedback instance, and Sentence-BERT (SBERT) encodes the transcription into a 384-dimensional vector. On the video side, VideoMAE—an 86-million-parameter masked autoencoder pre-trained on Kinetics-400—is first self-supervised fine-tuned on unlabeled surgical video, either feedback-only clips (task-relevant) or all surgical footage (domain-relevant), then supervised fine-tuned on labeled feedback videos, and finally average-pooled to a 768-dimensional vector. The two streams are compressed through funnel MLPs, concatenated into a 320-dimensional vector, and classified into behavior change versus no change. The self-supervised fine-tuning step is what adapts generic human-action video features to the surgical domain without needing additional labels.

What would settle it

A concrete check would be to re-run the same pipeline on a held-out set where feedback end times are manually annotated, so the video and text window is aligned to the true feedback boundaries rather than a fixed 5-seconds-before/5-seconds-after window; if AUROC collapses to near 0.5 or the multi-modal gain over text disappears, the reported 0.70 depends on window alignment rather than on feedback content. A simpler variant would be to shift each feedback instance's window by several seconds relative to the annotated onset and check whether prediction stays at chance.

Watch

Extended reading notes

Core claim

The central discovery is that trainee behavior change after surgical feedback is predictable from a combination of automatically transcribed speech and surgical video, and that the visual modality contributes beyond text once the video encoder has been adapted to surgery by self-supervised fine-tuning. On 4,204 feedback instances from 33 live da Vinci robot-assisted surgeries, the authors find the best multi-modal model reaches AUROC 0.70±0.02, versus 0.66 for text alone. They interpret this as evidence that the verbal content of feedback and the visual context of the surgical scene carry complementary, partially independent signals about whether the trainee will change behavior. They further report that self-supervised fine-tuning of the VideoMAE encoder—whether on feedback-specific clips or the full surgical video corpus—boosts both video-only and multi-modal AUROC, and that two existing surgery-specific video foundation models fine-tuned for the same task showed AUROC around 0.5, i.e., no predictive signal. The authors frame the contribution as the first demonstration that automated, objective feedback-effectiveness assessment is feasible from multi-modal features.

Load-bearing premise

The result rests on the assumption that a fixed 10-second window—5 seconds before and 5 seconds after the annotated feedback onset—captures both the full delivery of the feedback and the trainee's immediate response; the paper itself notes that feedback end times are unknown, so the window may include unrelated post-feedback speech or miss delayed behavior changes.

Editorial extensions

If this is right

  • Automated, objective, and scalable assessment of feedback effectiveness in robotic surgery is feasible, reaching AUROC 0.70±0.02 from speech and video.
  • Both transcribed feedback text and surgical video carry independent predictive signal: text alone reaches 0.66 and the best video-only model reaches 0.61.
  • Self-supervised fine-tuning on unlabeled surgical data improves performance, by up to 5.5% for video-only and up to 6.6% for the multi-modal model.
  • Task-relevant SSL fine-tuning on just 14.8% of the data matches domain-relevant fine-tuning on all surgical video, suggesting feedback-specific clips are especially informative.
  • The multi-modal gain is largest when text comes from automated transcription (up to 6.6%); with manually segmented transcriptions the gain shrinks to about 1.4%, indicating that visual context mainly helps when the text stream is noisy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the fixed-window alignment is the main bottleneck, re-labeling feedback with precise end timestamps or learning the alignment could push AUROC well beyond 0.70—a testable extension the authors did not run.
  • The comparable performance of task-relevant and domain-relevant SSL fine-tuning suggests data-collection efforts should prioritize more feedback-specific clips over additional generic surgical footage.
  • The confidence-score analysis in Appendix E implies a practical triage tool: surgeons could review only high-confidence feedback segments, where accuracy reaches 87% at the top 2.46% of instances.
  • A contrastive pre-/post-feedback learning objective, which the authors propose for future work, could be evaluated directly against this paper's 0.70 AUROC baseline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a multimodal model combining ASR-derived text embeddings (SBERT) and surgical video features (VideoMAE) to predict whether surgical feedback leads to an immediate trainee behavior change. On a dataset of 4,204 feedback instances from 33 robotic surgeries, the text-only model achieves AUROC 0.66, the video-only model 0.61, and the best fusion with self-supervised fine-tuning reaches AUROC 0.70 ± 0.02, an improvement the authors report as up to 6.6% over text-only. The authors claim both modalities are individually predictive and that self-supervised fine-tuning on surgical video improves performance. The paper includes analyses of video-only baselines (SurgVLP, GSViT), inspection of true/false positives, and a supplementary study using manual transcriptions.

Significance. If the prediction task is taken at face value, this is a practical and timely contribution to surgical education: it is, to my knowledge, the first automated system aimed at predicting feedback effectiveness from audio and video, and the release of code and the use of clinically annotated real-surgery data are strengths. However, the core result is compromised by an input-label temporal overlap that likely inflates the reported AUROC, and the claimed improvements are not supported by significance testing. The significance is therefore conditional: the manuscript demonstrates a feasible pipeline, but it does not yet establish that the model predicts feedback effectiveness rather than detecting the trainee's already-visible response.

major comments (5)
  1. [§3.3, §3.7, §3.1, §4.4] The input window contains the outcome being predicted. The label is defined as an immediate trainee adjustment made in direct response to the feedback (§3.1). The ASR text is transcribed for 10 seconds starting at the feedback onset timestamp (§3.3), and the video includes 5 seconds after onset (§3.7). Thus the text input can contain the trainee's verbal acknowledgment, which constitutes 46.2% of instances (Table 1), and the video can contain the behavioral change itself. Indeed, the authors' own visual inspection (§4.4) identifies true positives by a noticeable change at around 9 seconds, which is inside the input window. The Limitations paragraph explicitly acknowledges that post-feedback speech may be included, but it only discounts trainer commendations (14.8% of instances) and does not address trainee verbal acknowledgments. Consequently, the AUROC values in Tables 2 and 3 may reflect detection of the label rather than prediction of effectiveness from feedback content. This undermines the abstract's claim that 'both transcribed feedback and surgical video are individually predictive of trainee behavior changes.'
  2. [§4.3, Table 2] The headline improvement of up to 6.6% is a relative AUROC change from 0.66 to 0.70 (absolute 0.04), computed over only three random splits. The reported standard deviations (0.004 for text, 0.01 for text+video, 0.02 for the SSL-fine-tuned variants) overlap substantially, and no significance test is reported. The paper states in the Findings that self-supervised fine-tuning 'significantly helps' and in the Discussion that task-relevant and domain-relevant fine-tuning 'performed comparably,' but these claims are not supported by any statistical test. Please report paired comparisons (e.g., DeLong tests or bootstrap CIs) and per-split AUROC values so the reader can judge whether the improvements are real.
  3. [§4.2] Domain-relevant SSL fine-tuning may use test-set clips. The text says that 'domain-relevant SSL fine-tuning, all available surgical video data, split into non-overlapping 10-second clips, was used.' If the test-set feedback clips are part of 'all available surgical video data,' then the self-supervised model sees the test videos during pre-training, which can inflate the video-only and multimodal results. Please clarify whether test clips were excluded from the domain-relevant SSL fine-tuning, and rerun the condition with strict exclusion if they were not.
  4. [§4.2] The dataset split appears to be at the instance level rather than the surgery level. With only 33 surgeries, an 80/20 random split will likely place clips from the same surgery in both training and test, creating correlated data and optimistic AUROC estimates. A group-wise split by surgery (or patient) is needed to assess generalization to unseen procedures, especially since the paper reports performance stability by surgery type and trainer ID in Appendix D.
  5. [§3.3, §5 Limitations] No leakage-control experiment is reported. To establish that the model is not merely exploiting the post-onset window, the authors should provide baselines using (a) text restricted to the trainer's speech segments as identified by diarization, (b) video restricted to the 5 seconds before feedback onset, and (c) simple features such as trainee word count, presence of acknowledgment tokens, or post-onset frame-level motion. Without such controls, the individual predictiveness of text and video and the value of fusion remain ambiguous.
minor comments (5)
  1. [Appendix F, Table 8] Table 8 prints results as '0.790.04' and '0.720.03' with missing '±' separators; please fix the formatting.
  2. [§3.7] The phrase '16 randomly uniformly sampled frames' is ambiguous; please clarify whether frames are sampled independently at random or with a uniform stride.
  3. [§3.3] The paper states that speaker diarization is used to 'add context,' but it does not describe excluding the trainee's speech from the transcript. This is directly relevant to the leakage issue and should be explicitly acknowledged in the methods.
  4. [§5 Discussion] The Discussion uses 'significant difference' and 'significantly helps' without statistical tests; please either add tests or use weaker language such as 'substantial' to avoid implying formal significance.
  5. [Table 1] The annotation categories are described as resolved by consensus among three raters, but no inter-rater agreement statistic (e.g., Cohen's kappa or Fleiss' kappa) is reported; such a statistic would help assess label reliability.

Circularity Check

3 steps flagged · score 7.0 of 10

Input window includes post-feedback trainee response, so the reported AUROC may measure outcome detection rather than prediction; the manual-transcription baseline is circular by construction.

  1. self definitional [Section 3.7 (Data Processing), Section 3.1 (Data and Annotations), Section 4.4, Table 1]
    "Following Kocielnik et al. (2023), we trimmed a 10-second video when human-annotated feedback appears. This includes 5 seconds before (to capture context) and 5 seconds after (to capture delivery) the feedback onset. ... Behavioral change was defined as an immediate adjustment made by the trainee in direct response to the feedback."

    The label is the trainee's immediate response to feedback; Table 1 shows 46.2% of instances are verbal acknowledgments and 44.4% are behavioral changes. The 10-second input includes 5 seconds after the feedback-onset timestamp, and Section 4.4 states that true positives show 'a noticeable change in the video during feedback delivery (at around 5 seconds) and after feedback delivery (at around 9 seconds)'. Thus the input window can contain the labeled response itself. The model is not predicting a future outcome from pre-feedback cues; it is classifying a clip that already contains the outcome. The reported AUROC therefore measures detection of the outcome inside the window, not prediction of feedback effectiveness, by construction of the window.

  2. self definitional [Section 3.3 (Automated Speech Recognition)]
    "We employed Automated Speech Recognition (ASR) to generate automated transcriptions based on the starting timestamps of feedback instances (for a 10s duration). ... Speaker diarization segments the audio to identify different speakers, adding context to the dialogue around feedback delivery."

    The 10s ASR window starts at the feedback timestamp, and the diarization is used only to 'add context'; the trainee's speech is not excluded. Because verbal acknowledgment is itself a labeled outcome (46.2% of instances), a trainee saying 'okay' or 'got it' inside the window is part of the target. The text-only AUROC of 0.66 may therefore be driven by the ASR model transcribing the trainee's acknowledgment, not by semantic understanding of the trainer's feedback. This is the same input-label overlap as the video window, applied to the text modality.

1 more flagged steps
  1. self definitional [Appendix F (Segmented/Selective Manual Transcriptions) and Table 8]
    "human annotators split the feedback into smaller segments. Each segment corresponds to a part of the feedback that individually resulted in a trainee response such as verbal acknowledgment (which is inferred through audio) or behavior change (which is inferred visually)."

    The manual 'text-only' AUROC of 0.79 is produced from transcriptions that were segmented and selectively chosen according to the observed trainee response, so the text encodes the label. The paper acknowledges this: 'the segmentation was partly informed by visual cues, meaning some video context is implicitly reflected in the manual transcriptions.' Table 8 is therefore not an independent text modality; it is a circular predictor by construction, though it is a secondary result.

full rationale

The main auto-transcription and video results are not circular in the mathematical sense, and the SSL fine-tuning comparisons against Kinetics-400, SurgVLP, and GSViT are externally grounded. However, the central claim that 'both transcribed feedback and surgical video are individually predictive of trainee behavior changes' is undermined by a by-construction overlap: the 10s input includes the 5s after feedback onset, and the label is the trainee's immediate adjustment, which for 46.2% of instances is a verbal acknowledgment in the audio and for many true positives is visible instrument motion at approximately 9 seconds. The Limitations section acknowledges post-feedback speech but only discounts trainer commendations (14.8%, cited from overlapping-author prior work), not trainee verbal acknowledgments, which are the dominant label category. Appendix F's manual-transcription result is explicitly constructed using the trainee response, making its 'text-only' AUROC 0.79 circular by definition. Because these issues affect the paper's headline predictive-accuracy claims, the score reflects substantial, but not total, circularity; the SSL and baseline results retain independent content.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the transferability of pre-trained video and text models, the accuracy of ASR, the validity of the 10-second clip window, and the reliability of the behavior-change annotations. The paper introduces no new physical or methodological entities; the free parameters are standard ML hyperparameters tuned on a validation set.

free parameters (6)
  • Multi-modal training learning rate = 1e-3
    Tuned via grid search on a validation set (Section 4.2).
  • VideoMAE supervised fine-tuning learning rate = 1e-4
    Tuned via grid search on validation set (Section 4.2).
  • Number of supervised fine-tuning epochs = 4
    Set by grid search; early stopping not used.
  • Weight decay = 0.01
    Chosen for supervised and multi-modal training; standard regularization.
  • 10-second clip length = 5s before + 5s after feedback onset
    Chosen to approximate feedback delivery; acknowledged in Limitations as not knowing feedback end.
  • MLP layer dimensions = 768-512-256-2 for video; 384-128-64 for text; 320-256-128-2 for fusion
    Architecture choice, tuned via grid search.
assumptions (6)
  • domain assumption VideoMAE pre-trained on Kinetics-400 provides a useful general video representation that transfers to surgical video.
    The entire video feature extraction rests on this transfer (Section 3.4).
  • domain assumption SBERT embeddings capture the semantic meaning of transcribed feedback well enough for the prediction task.
    Text features are extracted with a frozen SBERT model (Section 3.3).
  • domain assumption Whisper medium ASR gives accurate enough transcriptions of surgeon-trainee dialogue.
    The main results use auto-transcriptions (Section 3.3).
  • domain assumption The 10-second clip around feedback onset contains the relevant visual and verbal context for assessing behavior change.
    This window defines the input features (Section 3.7) and is the weakest assumption.
  • domain assumption The consensus annotations of behavioral change are reliable ground truth for feedback effectiveness.
    Labels come from Wong et al. (2023) with 3-rater consensus (Section 3.1).
  • domain assumption The 80/20 split of feedback instances does not leak highly predictive surgery-specific information from training to test.
    The paper does not clarify whether instances from the same surgery appear in both splits (Section 4.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment." pith.science (2026). https://pith.science/paper/DMRE7SSK

@misc{pith2026241110919,
  author       = {Pith},
  title        = {Pith review of: Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DMRE7SSK}},
  note         = {Machine review of arXiv:2411.10919}
}
read the original abstract

During surgical training, real-time feedback from trainers to trainees is important for preventing errors and enhancing long-term skill acquisition. Accurately predicting the effectiveness of this feedback, specifically whether it leads to a change in trainee behavior, is crucial for developing methods for improving surgical training and education. However, relying on human annotations to assess feedback effectiveness is laborious and prone to biases, underscoring the need for an automated, scalable, and objective method. Creating such an automated system poses challenges, as it requires an understanding of both the verbal feedback delivered by the trainer and the visual context of the real-time surgical scene. To address this, we propose a method that integrates information from transcribed verbal feedback and corresponding surgical video to predict feedback effectiveness. Our findings show that both transcribed feedback and surgical video are individually predictive of trainee behavior changes, and their combination achieves an AUROC of 0.70+/-0.02, improving prediction accuracy by up to 6.6%. Additionally, we introduce self-supervised fine-tuning as a strategy for enhancing surgical video representation learning, which is scalable and further enhances prediction performance. Our results demonstrate the potential of multi-modal learning to advance the automated assessment of surgical feedback.

Figures

Figures reproduced from arXiv: 2411.10919 by the authors.

Figure 1
Figure 1. Motivation for automating the prediction of feedback effectiveness in surgical training. Manual annotation is time-consuming, prone to human cognitive biases (e.g., limited attention), and requires expertise, while automated prediction is quick, objective, and scalable. Trainers can use these assessments to improve their feedback delivery in the operating room, enhancing the quality of surgical training [PITH_FULL_… view at source ↗
Figure 2
Figure 2. Examples of feedback instances that are effective (result in trainee behavior change). The left example depicts feedback that describes addressing bleeding which causes the trainee to perform coagulation. The right example depicts feedback about pulling and retraction which causes the trainee to pull down the lower mass. (Haglund et al., 2021). The quality of communication has also been directly linked to operative … view at source ↗
Figure 3
Figure 3. The model pipeline for predicting trainee behavior change from feedback audio and video, composed of four steps: (A) Self-Supervised Pre-training of VideoMAE using Kinetics-400 (human action data); (B) Self-Supervised Fine-tuning of VideoMAE using either Task-Relevant fine-tuning on feedback-specific videos or Domain-Relevant fine-tuning on all surgical video; (C) Supervised Fine￾tuning of VideoMAE for predicting tr… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Examples of true positives and false pos [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Examples of true negatives and false neg [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 21 canonical work pages

  1. [1]

    The role of non-technical skills in surgery

    Riaz A Agha, Alexander J Fowler, and Nick Sevdalis. The role of non-technical skills in surgery. Annals of medicine and surgery, 4 0 (4): 0 422--427, 2015

    work page 2015

  2. [2]

    2018 robotic scene segmentation challenge

    Max Allan, Satoshi Kondo, Sebastian Bodenstedt, Stefan Leger, Rahim Kadkhodamohammadi, Imanol Luengo, Felix Fuentes, Evangello Flouty, Ahmed Mohammed, Marius Pedersen, et al. 2018 robotic scene segmentation challenge. arXiv preprint arXiv:2001.11190, 2020

    arXiv 2018

  3. [3]

    Coding qualitative data

    Melissa Baralt. Coding qualitative data. Research methods in second language acquisition: A practical guide, pages 222--244, 2011

    work page 2011

  4. [4]

    Comprehensive surgical coaching enhances surgical skill in the operating room

    Esther M Bonrath, Nicolas J Dedy, Lauren E Gordon, and Teodor P Grantcharov. Comprehensive surgical coaching enhances surgical skill in the operating room. Annals of surgery, 262 0 (2): 0 205--212, 2015

    work page 2015

  5. [5]

    End-to-end speaker segmentation for overlap-aware resegmentation

    Herv \'e Bredin and Antoine Laurent . End-to-end speaker segmentation for overlap-aware resegmentation . In Proc. Interspeech 2021, Brno, Czech Republic, August 2021

    work page 2021

  6. [6]

    pyannote.audio: neural building blocks for speaker diarization

    Herv \'e Bredin , Ruiqing Yin , Juan Manuel Coria , Gregory Gelly , Pavel Korshunov , Marvin Lavechin , Diego Fustes , Hadrien Titeux , Wassim Bouaziz , and Marie-Philippe Gill . pyannote.audio: neural building blocks for speaker diarization . In ICASSP 2020, IEEE International Conference on Acoustics, Speech, and Signal Processing, Barcelona, Spain, May 2020

    work page 2020

  7. [7]

    Unsupervised learning of visual features by contrasting cluster assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33: 0 9912--9924, 2020

    2020

  8. [8]

    Integrating qualitative and quantitative methods to develop a comprehensive coding manual: Measuring attention to mathematics in play contexts

    Jenny Yun-Chen Chan and Mich \`e le MM Mazzocco. Integrating qualitative and quantitative methods to develop a comprehensive coding manual: Measuring attention to mathematics in play contexts. Methods in Psychology, 4: 0 100044, 2021

    work page 2021

Show all 33 references
  1. [9]

    Ways of qualitative coding: A case study of four strategies for resolving disagreements

    Bonnie Chinh, Himanshu Zade, Abbas Ganji, and Cecilia Aragon. Ways of qualitative coding: A case study of four strategies for resolving disagreements. In Extended abstracts of the 2019 CHI conference on human factors in computing systems, pages 1--6, 2019

    2019

  2. [10]

    Evaluating how residents talk and what it means for surgical performance in the simulation lab

    Anne-Lise D D’Angelo, Andrew R Ruis, Wesley Collier, David Williamson Shaffer, and Carla M Pugh. Evaluating how residents talk and what it means for surgical performance in the simulation lab. The American Journal of Surgery, 220 0 (1): 0 37--43, 2020

    2020

  3. [11]

    Technical review of the da vinci surgical telemanipulator

    Cinzia Freschi, Vincenzo Ferrari, Franca Melfi, Mauro Ferrari, Franco Mosca, and Alfred Cuschieri. Technical review of the da vinci surgical telemanipulator. The International Journal of Medical Robotics and Computer Assisted Surgery, 9 0 (4): 0 396--406, 2013

    2013

  4. [12]

    The surgical autonomy program: a pilot study of social learning theory applied to competency-based neurosurgical education

    Michael M Haglund, Andrew B Cutler, Alexander Suarez, Rajeev Dharmapurikar, Shivanand P Lad, and Katherine E McDaniel. The surgical autonomy program: a pilot study of social learning theory applied to competency-based neurosurgical education. Neurosurgery, 88 0 (4): 0 E345--E350, 2021

    2021

  5. [13]

    Multi-task recurrent convolutional network with correlation loss for surgical video analysis

    Yueming Jin, Huaxia Li, Qi Dou, Hao Chen, Jing Qin, Chi-Wing Fu, and Pheng-Ann Heng. Multi-task recurrent convolutional network with correlation loss for surgical video analysis. Medical image analysis, 59: 0 101572, 2020

    2020

  6. [14]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  7. [15]

    Deep multimodal fusion for surgical feedback classification

    Rafal Kocielnik, Elyssa Y Wong, Timothy N Chu, Lydia Lin, De-An Huang, Jiayun Wang, Anima Anandkumar, and Andrew J Hung. Deep multimodal fusion for surgical feedback classification. In Machine Learning for Health (ML4H), pages 256--267. PMLR, 2023

    2023

  8. [16]

    The ava-kinetics localized human actions video dataset

    Ang Li, Meghana Thotakuri, David A Ross, Jo \ a o Carreira, Alexander Vostrikov, and Andrew Zisserman. The ava-kinetics localized human actions video dataset. arXiv preprint arXiv:2005.00214, 2020

    2005 arXiv

  9. [17]

    Surgical sam 2: Real-time segment anything in surgical video by efficient frame pruning

    Haofeng Liu, Erli Zhang, Junde Wu, Mingxuan Hong, and Yueming Jin. Surgical sam 2: Real-time segment anything in surgical video by efficient frame pruning. arXiv preprint arXiv:2408.07931, 2024

    2024 arXiv

  10. [18]

    Tailored feedback based on clinically relevant performance metrics expedites the acquisition of robotic suturing skills—an unblinded pilot randomized controlled trial

    Runzhuo Ma, Ryan S Lee, Jessica H Nguyen, Andrew Cowan, Taseen F Haque, Jonathan You, Sidney I Roberts, Steven Cen, Anthony Jarc, Inderbir S Gill, et al. Tailored feedback based on clinically relevant performance metrics expedites the acquisition of robotic suturing skills—an ...

    2022

  11. [19]

    Artificial intelligence-based video feedback to improve novice performance on robotic suturing skills: a pilot study

    Runzhuo Ma, Dani Kiyasseh, Jasper A Laca, Rafal Kocielnik, Elyssa Y Wong, Timothy N Chu, Steven Cen, Cherine H Yang, Istabraq S Dalieh, Taseen F Haque, et al. Artificial intelligence-based video feedback to improve novice performance on robotic suturing skills: a pilot study. ...

    2024

  12. [20]

    Thoracic surgery video analysis for surgical phase recognition

    Syed Abdul Mateen, Niharika Malvia, Syed Abdul Khader, Danny Wang, Deepti Srinivasan, Chi-Fu Jeffrey Yang, Lana Schumacher, and Sandeep Manjanna. Thoracic surgery video analysis for surgical phase recognition. arXiv preprint arXiv:2406.09185, 2024

    2024 arXiv

  13. [21]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  14. [22]

    Focus on the positives: Self-supervised learning for biodiversity monitoring

    Omiros Pantazis, Gabriel J Brostow, Kate E Jones, and Oisin Mac Aodha. Focus on the positives: Self-supervised learning for biodiversity monitoring. In Proceedings of the IEEE/CVF International conference on computer vision, pages 10583--10592, 2021

    2021

  15. [23]

    Robust speech recognition via large-scale weak supervision, 2022

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision, 2022. URL https://arxiv.org/abs/2212.04356

    2022 arXiv

  16. [24]

    Dissecting self-supervised learning methods for surgical computer vision

    Sanat Ramesh, Vinkle Srivastav, Deepak Alapatt, Tong Yu, Aditya Murali, Luca Sestini, Chinedu Innocent Nwoye, Idris Hamoud, Saurav Sharma, Antoine Fleurentin, et al. Dissecting self-supervised learning methods for surgical computer vision. Medical Image Analysis, 88: 0 102844, 2023

    2023

  17. [25]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. CoRR, abs/1908.10084, 2019. URL http://arxiv.org/abs/1908.10084

    1908 arXiv

  18. [26]

    Self-supervised learning for videos: A survey

    Madeline C Schiappa, Yogesh S Rawat, and Mubarak Shah. Self-supervised learning for videos: A survey. ACM Computing Surveys, 55 0 (13s): 0 1--37, 2023

    2023

  19. [27]

    General surgery vision transformer: A video pre-trained foundation model for general surgery

    Samuel Schmidgall, Ji Woong Kim, Jeffery Jopling, and Axel Krieger. General surgery vision transformer: A video pre-trained foundation model for general surgery. arXiv preprint arXiv:2403.05949, 2024

    2024 arXiv

  20. [28]

    Beyond neural scaling laws: beating power law scaling via data pruning

    Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35: 0 19523--19536, 2022

    2022

  21. [29]

    Video MAE : Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Video MAE : Masked autoencoders are data-efficient learners for self-supervised video pre-training. In Advances in Neural Information Processing Systems, 2022

    2022

  22. [30]

    Endonet: a deep architecture for recognition tasks on laparoscopic videos

    Andru P Twinanda, Sherif Shehata, Didier Mutter, Jacques Marescaux, Michel De Mathelin, and Nicolas Padoy. Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE transactions on medical imaging, 36 0 (1): 0 86--97, 2016

    2016

  23. [31]

    Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery

    Yuehao Wang, Yonghao Long, Siu Hin Fan, and Qi Dou. Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery. In International conference on medical image computing and computer-assisted intervention, pages 431--441. Springer, 2022

    2022

  24. [32]

    Development of a classification system for live surgical feedback

    Elyssa Y Wong, Timothy N Chu, Runzhuo Ma, Istabraq S Dalieh, Cherine H Yang, Ashwin Ramaswamy, Luis G Medina, Rafal Kocielnik, Seyedeh-Sanam Ladi-Seyedian, Andrew Shtulman, et al. Development of a classification system for live surgical feedback. JAMA Network Open, 6 0 (6): 0 ...

    2023

  25. [33]

    Learning multi-modal representations by watching hundreds of surgical video lectures

    Kun Yuan, Vinkle Srivastav, Tong Yu, Joel Lavanchy, Pietro Mascagni, Nassir Navab, and Nicolas Padoy. Learning multi-modal representations by watching hundreds of surgical video lectures. arXiv preprint arXiv:2307.15220, 2023

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.