Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes Surg-FTDA, a two-stage text-driven adaptation method that transfers pre-trained surgical vision-language foundation models to downstream tasks using only about 500 annotated image-label pairs, by training the decoder on…

desk verdict Plausible text-only adaptation recipe for surgical tasks, but the empirical case is under-built: broken low-data baseline, no error bars, and the anchor-alignment generalization is asserted rather than tested. read the letter →

arxiv 2501.09555 v3 pith:7BE7IPN6 submitted 2025-01-16 cs.CV cs.AI

Tingxuan Chen , Kun Yuan , Vinkle Srivastav , Nassir Navab , Nicolas Padoy This is my paper · ORCID
classification cs.CVcs.AI
keywords FoundationmodelSurgicaldatascienceMulti-modalitylearningworkflowanalysisFew-shotText-drivenadaptationImagecaptioningPhaserecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Surg-FTDA, a two-stage adaptation method that lets a pre-trained surgical vision-language foundation model handle downstream tasks using very few paired image-label examples. The first stage selects a small, diverse set of 'anchor' images, clusters or samples them, and trains a small MLP to push their embeddings into the text-embedding space. The second stage trains a text decoder (GPT-2) using only text labels or captions, never images, so the decoder learns to produce the right output from an embedding. At inference, the aligned image embeddings are fed into that text-trained decoder. If the claim is right, surgical workflow analysis—phase recognition, action triplet recognition, and captioning—can be adapted with about 500 annotated pairs instead of thousands, with performance close to fully supervised training.

What carries the argument

The load-bearing mechanism is the two-stage pipeline: (1) few-shot anchor selection via KMeans clustering or Farthest Point Sampling on frozen image embeddings, followed by an MLP trained with mean-squared error (Eq. 1) to map selected image embeddings to their corresponding text embeddings; (2) text-driven adaptation, where a GPT-2 decoder is fine-tuned to reconstruct downstream label texts or captions from embeddings produced by the frozen text encoder, with a reconstruction cross-entropy loss (Eq. 2). At inference the text encoder is swapped for the image encoder, and the MLP-aligned image embeddings are passed to the decoder. The key insight is that after alignment, the decoder cannot distinguish whether an embedding came from text or from a properly aligned image, so text-only training transfers to visual inputs.

What would settle it

Take the same pretrained SurgVLP model and the same 500-pair budget, but replace KMeans or FPS anchor selection with random sampling of 500 image-label pairs, and retrain the MLP and decoder from scratch. If performance on phase and triplet recognition drops to near chance, the anchor-selection strategy is the load-bearing component. Alternatively, train a fully supervised classifier on the same 500 randomly selected pairs with per-task hyperparameter tuning; if it matches or beats Surg-FTDA's F1 scores, the text-driven component is not the source of the reported data efficiency.

Watch

Extended reading notes

Core claim

The central claim is that a decoder trained exclusively on text can act as a visual classifier and caption generator for surgical data, provided the visual embeddings are first aligned into the text embedding space by a simple MLP trained on a few selected anchors. The paper shows this by replacing the text encoder with the image encoder at inference time after alignment, and reports that Surg-FTDA approaches fully-supervised performance on Cholec80 phase recognition and CholecT50 triplet recognition, and is competitive on SVL-Caption, using only 500 image-label pairs. It further claims that a single decoder trained jointly on phase and triplet text outperforms task-specific decoders. The method is presented as a generalizable alternative to full fine-tuning when annotation is scarce.

Load-bearing premise

The load-bearing assumption is that the few hundred anchor images selected by KMeans or FPS are representative enough of all downstream images that the MLP trained on them will map every test image into the text embedding space closely enough for the text-only decoder to still output correct labels or captions.

Editorial extensions

If this is right

  • Fully supervised models trained on 10% to 50% of the surgical datasets are outperformed by Surg-FTDA using only 500 pairs, while Surg-FTDA lands only slightly below the 100%-data model.
  • Increasing the number of selected anchors from 100 to 500 consistently improves performance, and KMeans anchor selection generally beats Farthest Point Sampling on the reported tasks.
  • A single text decoder trained on mixed task text, encoding both phase and triplet labels, yields better decision boundaries than separate task-specific decoders.
  • The same text-driven pipeline works for both discriminative tasks, such as phase and triplet recognition, and generative tasks, such as image captioning, with the surgical-domain SurgVLP initialization giving better results than the general CLIP initialization.
  • Because the decoder is trained on text only, the vocabulary of possible outputs is not limited to the fixed set of image-label pairs used during adaptation, which the paper identifies as enabling generative open-vocabulary outputs like captioning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The anchor-selection strategy is likely the main bottleneck of the pipeline: if the KMeans or FPS chosen anchors do not cover the test image distribution, the MLP will misalign out-of-distribution images, so we expect the choice of anchors to be at least as important as the alignment network's capacity.
  • A natural extension would be to apply the same text-driven adaptation to other medical imaging domains, such as radiology or pathology reports, where a vision-language foundation model exists and text annotations are cheaper than dense image labels.
  • The reported boost from a multi-task text decoder suggests that text-only training could scale toward a unified surgical language interface, with one decoder shared across phase recognition, triplet recognition, and captioning, and new tasks added by simply writing their label vocabularies.
  • A stronger test of the data-efficiency claim would pit Surg-FTDA against a fully supervised model whose hyperparameters are tuned specifically for the 500-pair regime; the paper's low-data fully supervised baselines were trained with the full-data hyperparameters, so they may understate the baseline's few-shot capability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Surg-FTDA, a two-stage adaptation method for surgical multimodal foundation models. In the first stage, KMeans or Farthest Point Sampling selects K image embeddings from a downstream dataset; the corresponding text embeddings are retrieved, and an MLP is trained to minimize the L2 distance between image and text embeddings on those K anchor pairs. In the second stage, a GPT-2 decoder is trained with a frozen text encoder to reconstruct label texts or captions using only text strings. At inference, test image embeddings are passed through the frozen image encoder, then through the alignment MLP, and then through the decoder to generate labels or captions. The method is evaluated on phase recognition (Cholec80), triplet recognition (CholecT50), and image captioning (SVL-Caption) using both SurgVLP and CLIP, and compared with CapDec and fully supervised image-text finetuning. The central claim is that a text-trained decoder can generalize to visual tasks with only about 500 image-label pairs, approaching fully supervised performance.

Significance. If the central claim holds, the method offers a practical way to adapt surgical vision-language models to label-scarce tasks with a single transferable text decoder and a small alignment MLP. The paper's strengths include evaluation on two foundation models, three task types, comparison with CapDec, and a commitment to release code. It also connects to a relevant line of work on text-only training (CapDec, CLOSE). However, the empirical support has gaps that directly affect the central claim: no quantitative validation of the alignment MLP's off-anchor generalization, ambiguous anchor-split provenance, a broken 10% supervised baseline, and no error bars. The significance is therefore conditional on additional evidence that the alignment learned on a few anchors actually transfers to unseen test images.

major comments (5)
  1. [Section 2.2, Eq. (1)] The central inference step assumes that the MLP trained on K anchors generalizes to all test image embeddings, but no quantitative evidence is provided for this off-anchor behavior. The paper reports no held-out alignment error, no nearest-neighbor accuracy of aligned test embeddings to their corresponding label text embeddings, and no variation over anchor selections or training seeds; Fig. 3 is a qualitative 2D projection. Because a two-hidden-layer MLP with 128 units can fit 500 points while behaving arbitrarily in between them, the reported downstream gains are not yet attributable to genuine modality alignment. Please add quantitative alignment or retrieval metrics on held-out images and evaluate across seeds and anchor sets.
  2. [Section 2.1] It is not stated whether the KMeans/FPS anchor selection is performed only on the training split. The text refers to a 'downstream dataset with large-scale unlabeled images' and does not explicitly restrict anchors to the training set; if anchor images can come from the test split, then Eq. (1) is trained with test labels and the few-shot comparison is invalid. Please state the split explicitly, and if anchors were selected from the test split, rerun the experiments with training-only selection.
  3. [Section 3.3, Tables 7 and 8] The fully supervised model trained on 10% of the data yields all-zero metrics because it 'fails to generate phase outputs in the correct format under the same experimental settings.' This makes the comparison invalid as a measure of task performance: a formatting failure is not equivalent to low accuracy. Please tune the baseline (e.g., epochs, learning rate, decoding strategy) and report its best valid performance, or exclude it from the comparison with an explicit explanation.
  4. [Section 2.3 and Section 3.2.1] The decoding protocol for discriminative tasks is underspecified. The paper does not describe how free-form decoder outputs are mapped to fixed phase or triplet labels, nor whether constrained decoding is used. This is essential for reproducing the results and for diagnosing the baseline format failure mentioned in Section 3.3; please provide the exact decoding and parsing procedure.
  5. [Tables 1-10] All experiments appear to be single runs with no error bars or seed variance. Given the stochasticity in the alignment MLP, GPT-2 fine-tuning, and anchor selection, comparative claims such as 'outperforms' and 'performance improves with more anchors' need confidence intervals or multiple seeds. Please add seeded repetitions and report mean and standard deviation for the main results.
minor comments (6)
  1. [Eq. (1) and surrounding text] The notation for the aligned embedding is inconsistent (\hat{v}^i_{image'} vs. \hat{V}_{image'} and 'image’' with an apostrophe); please standardize the notation.
  2. [Section 3.3, first sentence] The sentence 'As shown in Tab. 4, Tab. 5, and Tab. 6, demonstrate that...' is ungrammatical; remove 'demonstrate that' or restructure the sentence.
  3. [Section 3.2.1, Tables 1 and 2] The manuscript does not explain how CapDec is adapted to phase and triplet recognition, since CapDec is originally a captioning method; please clarify the setup so the comparison is meaningful.
  4. [Figure 1] Figure 1(b) is referenced only in the caption and not explicitly cited in the main text; please add a callout.
  5. [Section 3, model hyperparameters] The default batch size is stated as 34; if this is not a typo for 32, please comment on the choice.
  6. [Table 5] For SurgVLP, KMeans 100 reports accuracy 3.01 while KMeans 500 reports 27.97 and FPS 100 reports 10.28; the non-monotonic behavior and large jump deserve a brief comment.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity found: the few-shot alignment and text-decoder training are separate stages; the central risk is empirical off-anchor generalization, not a self-referential derivation.

full rationale

The derivation chain is not circular. Stage 1 trains the alignment MLP on K selected image-label anchors by minimizing MSE between aligned image embeddings and text embeddings (Eq. 1); Stage 2 trains the text decoder on text embeddings alone to reconstruct labels/captions (Eq. 2); inference combines the frozen image encoder, the trained MLP, and the text decoder. The target labels enter only as anchor supervision and as the text-training corpus, which is standard supervised training rather than a self-definitional equivalence. The decoder never sees images during training, and the MLP never sees test labels, so no equation reduces to its own input. The paper's main vulnerability is empirical: a K-point regression in a high-dimensional embedding space may not generalize off-anchor, and the paper provides no held-out alignment error or variance analysis. That is a generalization risk, not a circular construction. The admitted collapse of the fully supervised 10% baseline (Section 3.3, Tables 7 and 8) is a benchmark-execution limitation that weakens the comparative claim, but it does not make the method circular. Self-citations to SurgVLP [25] and other surgical video-language pretraining works are present and SurgVLP is used as a main backbone, but all key experiments are repeated with CLIP, an external model, and the text-only training idea is explicitly credited to external works (CapDec and CLOSE), so the self-citation is not load-bearing. Overall, the claimed adaptation is self-contained relative to its stated assumptions.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. The MLP alignment module and GPT-2 decoder are standard trainable components, and the frozen encoders come from prior work. The free parameters are mainly the anchor count and training hyperparameters, all chosen by hand without sensitivity analysis.

free parameters (3)
  • Number of anchors K = 500 (100 and 500 in ablation)
    K, the count of image-label pairs used for modality alignment, is set by the authors. The default is 500 and ablations use 100 and 500. Results depend strongly on K, so it is a hand-chosen tuning parameter.
  • MLP alignment hyperparameters = lr=0.001, 15 epochs, batch 16, hidden 128
    Training settings for the modality alignment MLP are given in Section 3 without sensitivity analysis. These are chosen by hand.
  • Text decoder hyperparameters = lr=2e-5, 10 epochs, batch 34
    Training settings for the GPT-2 decoder in Section 3 are chosen by hand, with no sensitivity analysis or justification.
assumptions (4)
  • domain assumption CLIP/SurgVLP frozen image and text encoders produce embeddings that are semantically comparable within each modality and that an MLP can map between.
    The alignment step in Section 2.1 assumes a shared semantic space and a trainable mapping between image and text embeddings, which is not proven for surgical data.
  • domain assumption Selected anchors are representative of the full downstream image distribution.
    Section 2.1 states that anchors must be 'evenly distributed and representative of the unlabeled dataset', but no formal guarantee is given that KMeans or FPS with K=500 achieves this for the test distribution.
  • domain assumption Text embeddings of downstream labels and captions form a manifold from which a decoder can reconstruct text, and aligned image embeddings lie on this manifold.
    Section 2.2 trains the decoder on text embeddings only and then applies it to aligned image embeddings at inference. The transfer implicitly assumes aligned image embeddings are close to text embeddings, a claim supported only by visualizations.
  • standard math Standard ML and optimization assumptions hold for MSE and cross-entropy training.
    Equations 1 and 2 use standard loss functions and optimizers; these are standard mathematical tools and do not require justification beyond the usual practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis." pith.science (2026). https://pith.science/paper/7BE7IPN6

@misc{pith2026250109555,
  author       = {Pith},
  title        = {Pith review of: Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7BE7IPN6}},
  note         = {Machine review of arXiv:2501.09555}
}
read the original abstract

Purpose: Surgical workflow analysis is crucial for improving surgical efficiency and safety. However, previous studies rely heavily on large-scale annotated datasets, posing challenges in cost, scalability, and reliance on expert annotations. To address this, we propose Surg-FTDA (Few-shot Text-driven Adaptation), designed to handle various surgical workflow analysis tasks with minimal paired image-label data. Methods: Our approach has two key components. First, Few-shot selection-based modality alignment selects a small subset of images and aligns their embeddings with text embeddings from the downstream task, bridging the modality gap. Second, Text-driven adaptation leverages only text data to train a decoder, eliminating the need for paired image-text data. This decoder is then applied to aligned image embeddings, enabling image-related tasks without explicit image-text pairs. Results: We evaluate our approach to generative tasks (image captioning) and discriminative tasks (triplet recognition and phase recognition). Results show that Surg-FTDA outperforms baselines and generalizes well across downstream tasks. Conclusion: We propose a text-driven adaptation approach that mitigates the modality gap and handles multiple downstream tasks in surgical workflow analysis, with minimal reliance on large annotated datasets. The code and dataset will be released in https://github.com/CAMMA-public/Surg-FTDA

Figures

Figures reproduced from arXiv: 2501.09555 by the authors.

Figure 1
Figure 1. (a) Conventional adaptation of multi-modal foundation model requires paired image-label data for training; (b) Our text-driven adaptation of the founda [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Few-shot data anchor selection based on the visual embedding space using KMeans or FPS; (b) The text-driven training and inference process of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visualization of modality gap across foundation models using few-shot data anchor selection and modality alignment. Yellow points represent the image [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CPKD: Clinical Prior Knowledge-Constrained Diffusion Models for Surgical Phase Recognition in Endoscopic Submucosal Dissection

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A diffusion-based generative model with training-time masking and clinical logic constraints achieves state-of-the-art surgical phase recognition on ESD videos and a small gain on cholecystectomy videos.

Reference graph

Works this paper leans on

27 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Banerjee, S., Lavie, A., 2005. Meteor: An automatic metric for mt eval- uation with improved correlation with human judgments, in: Proceed- ings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65–72

    work page 2005

  2. [2]

    Czempiel, T., Paschali, M., Keicher, M., Simson, W., Feussner, H., Kim, S.T., Navab, N., 2020. Tecno: Surgical phase recognition with multi- stage temporal convolutional networks, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, Springer. pp. 343–352

    work page 2020

  3. [3]

    Gao, X., Jin, Y ., Long, Y ., Dou, Q., Heng, P.A., 2021. Trans-svnet: Accurate phase recognition from surgical videos via hybrid embedding aggregation transformer, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, Springer. p...

    work page 2021

  4. [4]

    I can’t believe there’s no im- ages! learning visual tasks using only language supervision, in: Pro- ceedings of the IEEE /CVF International Conference on Computer Vi- sion, pp

    Gu, S., Clark, C., Kembhavi, A., 2023. I can’t believe there’s no im- ages! learning visual tasks using only language supervision, in: Pro- ceedings of the IEEE /CVF International Conference on Computer Vi- sion, pp. 2672–2683

    work page 2023

  5. [5]

    A k-means clustering algorithm

    Hartigan, J.A., Wong, M.A., 1979. A k-means clustering algorithm. Applied Statistics 28, 100–108

    work page 1979

  6. [6]

    Maier-Hein, L., Eisenmann, M., Sarikaya, D., M ¨arz, K., Collins, T., Malpani, A., Fallert, J., Feussner, H., Giannarou, S., Mascagni, P., et al.,

    work page

  7. [7]

    Latent graph representations for critical view of safety assessment

    Murali, A., Alapatt, D., Mascagni, P., Vardazaryan, A., Garcia, A., Okamoto, N., Mutter, D., Padoy, N., 2023. Latent graph representations for critical view of safety assessment. IEEE Transactions on Medical Imaging

    work page 2023

  8. [8]

    Text-only training for image captioning using noise-injected CLIP, in: Goldberg, Y ., Kozareva, Z., Zhang, Y

    Nukrai, D., Mokady, R., Globerson, A., 2022. Text-only training for image captioning using noise-injected CLIP, in: Goldberg, Y ., Kozareva, Z., Zhang, Y . (Eds.), Findings of the Association for Com- putational Linguistics: EMNLP 2022, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates. pp. 4055–4063. URL: https://aclanthology.org...

    doi:10.18653/v1/2022.findings-emnlp.299 2022

Show all 27 references
  1. [9]

    Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos

    Nwoye, C.I., Yu, T., Gonzalez, C., Seeliger, B., Mascagni, P., Mutter, D., Marescaux, J., Padoy, N., 2022. Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Me- dIA 78, 102433

    2022

  2. [10]

    Machine and deep learning for workflow recognition during surgery

    Padoy, N., 2019. Machine and deep learning for workflow recognition during surgery. Minimally Invasive Therapy & Allied Technologies 28, 82–90

    2019

  3. [11]

    Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J., 2002. Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318

    2002

  4. [12]

    Pointnet ++: Deep hierar- chical feature learning on point sets in a metric space

    Qi, C.R., Yi, L., Su, H., Guibas, L.J., 2017. Pointnet ++: Deep hierar- chical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413

    2017 arXiv

  5. [13]

    Learning transferable visual models from natural language supervision, in: Inter- national conference on machine learning, PMLR

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al., 2021. Learning transferable visual models from natural language supervision, in: Inter- national conference on machine learning, PMLR. pp. 8748–8763

    2021

  6. [14]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al., 2019. Language models are unsupervised multitask learners. Ope- nAI blog 1, 9

    2019

  7. [15]

    Ima- genet large scale visual recognition challenge

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al., 2015. Ima- genet large scale visual recognition challenge. International journal of computer vision 115, 211–252

    2015

  8. [16]

    Surgical- vqa: Visual question answering in surgical scenes using transformer, in: MICCAI, pp

    Seenivasan, L., Islam, M., Krishna, A.K., Ren, H., 2022. Surgical- vqa: Visual question answering in surgical scenes using transformer, in: MICCAI, pp. 33–43

    2022

  9. [17]

    Endonet: a deep architecture for recognition tasks on laparoscopic videos

    Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., De Mathelin, M., Padoy, N., 2016. Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE transactions on medical imaging 36, 86– 97

    2016

  10. [18]

    Cider: Consensus-based image description evaluation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp

    Vedantam, R., Lawrence Zitnick, C., Parikh, D., 2015. Cider: Consensus-based image description evaluation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4566– 4575

    2015

  11. [19]

    Actionclip: A new paradigm for video action recognition

    Wang, M., Xing, J., Liu, Y ., 2021. Actionclip: A new paradigm for video action recognition. arXiv preprint arXiv:2109.08472

    2021 arXiv

  12. [20]

    Visual-language prompt tuning with knowledge-guided context optimization, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp

    Yao, H., Zhang, R., Xu, C., 2023. Visual-language prompt tuning with knowledge-guided context optimization, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6757–6767

    2023

  13. [21]

    Anticipation for sur- gical workflow through instrument interaction and recognized signals

    Yuan, K., Holden, M., Gao, S., Lee, W., 2022. Anticipation for sur- gical workflow through instrument interaction and recognized signals. Medical Image Analysis 82, 102611

    2022

  14. [22]

    Advancing surgical vqa with scene graph knowledge

    Yuan, K., Kattel, M., Lavanchy, J.L., Navab, N., Srivastav, V ., Padoy, N., 2024a. Advancing surgical vqa with scene graph knowledge. Inter- national Journal of Computer Assisted Radiology and Surgery , 1–9

  15. [23]

    Yuan, K., Srivastav, V ., Navab, N., Padoy, N., 2024b. Hecvl: Hierar- chical video-language pretraining for zero-shot surgical phase recogni- tion, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 306–316

  16. [24]

    Procedure-aware surgical video-language pretraining with hierarchical knowledge aug- mentation

    Yuan, K., Srivastav, V ., Navab, N., Padoy, N., 2024c. Procedure-aware surgical video-language pretraining with hierarchical knowledge aug- mentation. arXiv preprint arXiv:2410.00263

    arXiv

  17. [25]

    Learning multi-modal representations by watching hundreds of surgical video lectures

    Yuan, K., Srivastav, V ., Yu, T., Lavanchy, J., Mascagni, P., Navab, N., Padoy, N., 2023. Learning multi-modal representations by watching hundreds of surgical video lectures. arXiv preprint arXiv:2307.15220

    2023 arXiv

  18. [26]

    Conditional prompt learn- ing for vision-language models, in: Proceedings of the IEEE /CVF con- ference on computer vision and pattern recognition, pp

    Zhou, K., Yang, J., Loy, C.C., Liu, Z., 2022. Conditional prompt learn- ing for vision-language models, in: Proceedings of the IEEE /CVF con- ference on computer vision and pattern recognition, pp. 16816–16825

    2022

  19. [2022]

    MedIA 76, 102306

    Surgical data science–from concepts toward clinical translation. MedIA 76, 102306

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.