REVIEW 5 major objections 5 minor 28 references
SurgX: Neuron-Concept Association for Explainable Surgical Phase Recognition
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read SurgX claims surgical phase recognition models can be explained by linking each neuron to surgeon-readable concepts, then reading off the concepts of the neurons that most influenced a prediction.
desk verdict A competent, clearly written adaptation of the group's own neuron-concept annotation to surgical phases, but the self-referential SurgVLP evaluation leaves the faithfulness claim unproven. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is neuron-concept annotation via cosine similarity in SurgVLP's shared text-visual embedding space: for each neuron, a representative example set is built from frames that highly activate it (selected video-wise by an adaptive threshold, then extended with preceding frames at a dilation interval), and the concept whose text embedding is closest to the average visual embedding of that set is assigned to the neuron (Eq. 1). To explain a prediction, the contribution of each penultimate-layer neuron is estimated with the approximation |a ∇a fθ(x)| from Eq. 2, the highest-contribution neurons are picked, and their assigned concepts are presented as the explanation. SurgVLP is doing double duty here: it supplies the text and visual features for annotation and also the embedding space in which both evaluation scores are computed.
What would settle it
Re-run the SurgX annotation using random text embeddings in place of SurgVLP's concept embeddings; if Concept Alignment and Prediction Interpretability scores stay high, the scores are not measuring neuron-concept fidelity.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that neurons in surgical phase recognition models carry meaning that can be captured by a curated concept set and a vision-language embedding space, and that the concepts of the most influential neurons align with the predicted phase. The authors demonstrate that a video-wise adaptive selection of highly activated frames, extended into dilated sequences to capture temporal context, yields the most faithful concept assignments, and that larger, domain-specific concept sets such as ChoLec-270 outperform word- and sentence-level sets derived from action triplets. They also show that the same neuron-concept apparatus explains mispredictions, such as clips visible during gallbladder dissection triggering the "clipping and cutting" phase, which suggests the explanations track the features the model actually uses.
Load-bearing premise
The evaluation assumes cosine similarity in SurgVLP's embedding space is a valid proxy for whether a concept genuinely explains a model's prediction.
Editorial extensions
If this is right
- The same SurgX pipeline can be applied to other temporal action segmentation or phase recognition models, not just TeCNO and Causal ASFormer, as long as neuron activations and SurgVLP embeddings are available.
- The concept sets built from CholecT45 action triplets and lecture videos can be reused as a standardized vocabulary for explaining cholecystectomy video models.
- Because the method identifies which neurons mislead predictions (e.g., "cystic artery is isolated between clips" during gallbladder dissection), it offers a target for debugging models: regularizing or suppressing those neurons could reduce systematic mispredictions.
- Quantitative interpretability scores can be reported for any surgical phase model without human annotations, since both metrics are computed automatically via SurgVLP.
Reading between the lines
- The evaluation's reliance on SurgVLP for both annotation and scoring means the reported scores could be inflated by embedding-space artifacts; a fairer test would compare SurgX's neuron explanations against human-annotated concept maps or against a different vision-language encoder.
- SurgX is implicitly a claim about causal feature use: if a neuron's assigned concept is faithfully its role, then removing that neuron should change predictions in the direction the concept suggests. The paper does not run that perturbation test, but it is a direct, testable consequence.
- Extending SurgX beyond cholecystectomy would require constructing new concept sets for each procedure; the ChoLec-270 result suggests concept coverage, not model architecture, is the binding constraint on explanation quality.
- The misprediction analysis (88.22% of errors involving a "cystic artery is isolated between clips" neuron) hints at a diagnostic application: concept-based monitoring could flag when a model's decision hinges on a feature a surgeon would judge as non-causal.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. SurgX proposes a concept-based explanation framework for surgical phase recognition models. The method constructs surgical-domain concept sets (CholecT45-W, CholecT45-S, ChoLec-270), selects representative sequences for each neuron from highly activated frames, annotates neurons with concepts via cosine similarity in the SurgVLP embedding space (Eq. 1), and identifies important neurons using a first-order Taylor approximation of their contribution to the prediction (Eq. 2). The authors validate SurgX on TeCNO and Causal ASFormer with Cholec80, reporting two quantitative scores (Concept Alignment and Prediction Interpretability) computed as cosine similarities in the same SurgVLP space, plus qualitative analyses of correct and incorrect predictions.
Significance. If the quantitative validation were trustworthy, SurgX would be a useful step toward interpretability in surgical phase recognition, an area where concept-based explanations are indeed underexplored. The paper includes a released code repository, a domain-specific concept construction procedure, and a nice qualitative analysis of mispredictions (e.g., the 88.22% statistic linking a specific concept to a common error). However, the central evaluation is self-referential: SurgVLP is used both to assign concepts to neurons and to compute both reported scores, so the numbers may certify internal consistency of the embedding space rather than faithfulness of the explanations to the target model. The lack of external anchors (human evaluation, intervention tests, control concepts) means the main claim is not yet supported.
major comments (5)
- [Section 3.1, Eq. (1) and evaluation metrics] The two evaluation metrics are computed as cosine similarities in the SurgVLP embedding space, which is the same space used to assign concepts to neurons in Eq. (1). This makes the validation largely self-referential: a high Concept Alignment or Prediction Interpretability score can arise even if the annotated concepts do not reflect what the phase recognition model computes, as long as SurgVLP's text-visual alignment is consistent with surgical phase semantics. The manuscript should add an external anchor: at minimum, an intervention test (e.g., zeroing or masking the top-contributing neurons and checking that the predicted phase changes in the direction implied by the concept), a human evaluation of concept relevance, or a control experiment with random or mismatched concept sets and a different embedding space.
- [Tables 1-3] All quantitative results are single numbers with no error bars, no repeated runs, and no significance tests. Given the small margins between configurations (e.g., Concept Alignment 0.4475 vs. 0.4485 in Table 2), the reported rankings are not interpretable without variance estimates. The authors should report standard deviations across multiple training seeds or across videos, and perform a statistical test where appropriate.
- [Section 2.4, Eq. (2)] The equality |fθ(x) − fθ(x; a_{p,i} ← 0)| = |a_{p,i} ∇_{a_{p,i}} fθ(x)| is a first-order Taylor approximation, not an exact equality. For highly nonlinear Transformer models such as Causal ASFormer, this approximation can have large error, and the manuscript does not assess it. The paper should acknowledge the approximation and validate the importance scores by actually ablating the identified neurons and measuring the change in prediction, rather than relying solely on the gradient approximation.
- [Section 2.3] The concept annotation threshold θ_concept is never reported, yet it directly determines which concepts are attached to each neuron and therefore influences every quantitative result in Tables 1-3. Without this value, the experiments are not reproducible, and the sensitivity of the results to this threshold is unknown. The authors should report θ_concept for each setting and include a sensitivity analysis.
- [Section 3.2] The experiments contain no comparison to any baseline interpretability method, such as CLIP-Dissect adapted to video, gradient-based attribution, or a simple random concept-assignment control. Since the contribution claims that SurgX enhances interpretability, the reader cannot judge whether the proposed concept-set construction and sequence selection add value beyond established methods. Adding at least one such baseline and a random-control condition would substantially strengthen the evidence.
minor comments (5)
- [Section 3.1] The sentence 'Using the visual encoder of SurgVLP to extract visual features from representative frames avoids the gap between the embedding spaces of the phase recognition model and SurgVLP' is confusing, since the method does not operate in the phase model's feature space at all. The authors likely mean that using SurgVLP for both text and visual features keeps them in a shared space; please clarify.
- [Table 1] The column 'Unique Concept' is not defined. Please state whether it is the number of distinct concepts assigned to any neuron, and explain why CholecT45-W (30 words) yields only 7 unique concepts while ChoLec-270 yields 34.
- [Figure 1 and Section 2.2] The parameter n for the number of previous frames in the representative sequence is never specified in the methodology. It appears only through the ablation experiments (10 frames). Please state the default value and how it was chosen.
- [Section 3.3] The percentages '88.22%' and '92.88%' are reported without denominators or a clear definition of 'involve'. Please specify how these numbers were computed and over how many predictions.
- [Keywords] The keywords are typeset without separators ('SurgicalPhaseRecognition ·EndoscopicVideo ·Interpretability ...'); they should be separated by spaces for readability.
Circularity Check
Quantitative validation is self-referential: concept assignment and the headline Concept Alignment / Prediction Interpretability scores both use SurgVLP cosine similarity, so high scores certify embedding-space consistency rather than faithful explanation of the phase-recognition models.
-
self definitional
[Section 2.3, Eq. (1) and Section 3.1, Evaluation Metrics]
"First, we define Concept Alignment Score to evaluate how well concepts are annotated to neurons by measuring the cosine similarity between the concepts assigned to neurons in the final layer and the ground-truth class ... For both aspects, we compute the cosine similarity between the concept and the phase expressed in word form, as well as between the concept and the phase expressed in sentence form provided in [28], and then average the two scores, since all the concepts and frames are encoded by SurgVLP [28]"
Eq. (1) defines the annotation: a concept is attached to a neuron when the SurgVLP cosine similarity between the neuron's representative frames and the concept text exceeds the threshold. The two headline metrics then recompute cosine similarity, again in SurgVLP, between the very concepts produced by that rule and word/sentence forms of the phase label. The evaluation therefore does not measure whether TeCNO or Causal ASFormer actually computes with the attached concept; it measures internal consistency of SurgVLP's text-visual geometry. Any neuron whose SurgVLP features happen to land near phase-like text will score high even if the neuron is causally irrelevant to the phase prediction.
full rationale
SurgX's central contribution (neuron-concept association for surgical phase recognition) has independent content in its qualitative analyses and concept-set ablations. However, the quantitative validation is partially circular: the same SurgVLP embedding space both assigns concepts to neurons (Eq. 1) and defines the two reported fidelity scores (Section 3.1). Thus a high score can be an artifact of SurgVLP's phase-like text-visual alignment rather than evidence about the phase-recognition model's internal computation. The self-citations to the authors' prior works [1,12] propagate the same cosine-similarity evaluation paradigm, but the core issue is the shared measurement model, not the self-citation itself. The paper would need an external anchor (human judgment, causal intervention, or control concept set) to break the self-reference; absent that, a score of 6 reflects partial circularity in the evaluation rather than a fully forced derivation.
Assumptions & free parameters
free parameters (5)
- concept annotation threshold theta_concept =
not reported
- adaptive threshold alpha =
0.95
- top-K values =
K=40 (global), K=1 (video-wise)
- dilation size and sequence length =
dilation m=5, length 10 frames (best)
- concept set choice =
ChoLec-270 (270 concepts)
assumptions (5)
- domain assumption SurgVLP embedding space reliably represents semantic similarity between surgical concepts and video frames
- domain assumption High post-ReLU activations indicate that a neuron encodes the selected visual content
- standard math First-order Taylor expansion exactly captures the effect of removing a neuron
- domain assumption Training videos used as the probing set are representative of the model's learned features
- domain assumption Penultimate-layer neurons are the appropriate explanatory unit for predictions
Cite this review
Pith. "Pith review of SurgX: Neuron-Concept Association for Explainable Surgical Phase Recognition." pith.science (2026). https://pith.science/paper/YK6DIB6T
@misc{pith2026250715418,
author = {Pith},
title = {Pith review of: SurgX: Neuron-Concept Association for Explainable Surgical Phase Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/YK6DIB6T}},
note = {Machine review of arXiv:2507.15418}
}
read the original abstract
Surgical phase recognition plays a crucial role in surgical workflow analysis, enabling various applications such as surgical monitoring, skill assessment, and workflow optimization. Despite significant advancements in deep learning-based surgical phase recognition, these models remain inherently opaque, making it difficult to understand how they make decisions. This lack of interpretability hinders trust and makes it challenging to debug the model. To address this challenge, we propose SurgX, a novel concept-based explanation framework that enhances the interpretability of surgical phase recognition models by associating neurons with relevant concepts. In this paper, we introduce the process of selecting representative example sequences for neurons, constructing a concept set tailored to the surgical video dataset, associating neurons with concepts and identifying neurons crucial for predictions. Through extensive experiments on two surgical phase recognition models, we validate our method and analyze the explanation for prediction. This highlights the potential of our method in explaining surgical phase recognition. The code is available at https://github.com/ailab-kyunghee/SurgX
Figures
Reference graph
Works this paper leans on
- [1]
-
[2]
Ahn, Y.H., Park, G.M., Kim, S.T.: Line: Out-of-distribution detection by leverag- ing important neurons (2023),https://arxiv.org/abs/2303.13995
arXiv 2023
-
[3]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: Quan- tifying interpretability of deep visual representations. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 6541–6549 (2017)
2017
-
[4]
In: International Conference on Medical Image Computing and Computer Assisted Intervention
Czempiel, T., Paschali, M., Keicher, M., Simson, W., Feussner, H., Kim, S.T., Navab, N.: Tecno: Surgical phase recognition with multi-stage temporal convolu- tional networks. In: International Conference on Medical Image Computing and Computer Assisted Intervention. pp. 343–352 (2020)
work page 2020
-
[5]
In: International Conference on Medical Image Computing and Computer Assisted Intervention
Czempiel, T., Paschali, M., Ostler, D., Kim, S.T., Busam, B., Navab, N.: Opera: Attention-regularized transformers for surgical phase recognition. In: International Conference on Medical Image Computing and Computer Assisted Intervention. pp. 604–614 (2021)
work page 2021
-
[6]
In: International Conference on Medical Image Computing and Computer Assisted Intervention
Gao, X., Jin, Y., Long, Y., Dou, Q., Heng, P.A.: Trans-svnet: Accurate phase recognition from surgical videos via hybrid embedding aggregation transformer. In: International Conference on Medical Image Computing and Computer Assisted Intervention. pp. 593–603 (2021)
work page 2021
-
[7]
Stat- Pearls Publishing (2025)
Hassler, K., Collins, J., Philip, K., Jones, M.: Laparoscopic Cholecystectomy. Stat- Pearls Publishing (2025)
work page 2025
-
[8]
IEEE transactions on medical imaging37(5), 1114–1126 (2017) 10 Kim et al
Jin, Y., Dou, Q., Chen, H., Yu, L., Qin, J., Fu, C.W., Heng, P.A.: Sv-rcnet: work- flow recognition from surgical videos using recurrent convolutional network. IEEE transactions on medical imaging37(5), 1114–1126 (2017) 10 Kim et al
work page 2017
Show all 28 references
-
[9]
In: International Conference on Machine Learning
Kalibhat, N., Bhardwaj, S., Bruss, C.B., Firooz, H., Sanjabi, M., Feizi, S.: Identify- ing interpretable subspaces in image representations. In: International Conference on Machine Learning. pp. 15623–15638 (2023)
2023
-
[10]
In: CVPR
Khakzar, A., Baselizadeh, S., Khanduja, S., Rupprecht, C., Kim, S.T., Navab, N.: Neural response interpretation through the lens of critical pathways. In: CVPR. pp. 13528–13538 (2021)
2021
-
[11]
In: International Conference on Medical Image Computing and Computer Assisted Intervention
Khakzar, A., Musatian, S., Buchberger, J., Valeriano Quiroz, I., Pinger, N., Baselizadeh, S., Kim, S.T., Navab, N.: Towards semantic interpretation of thoracic disease and covid-19 diagnosis models. In: International Conference on Medical Image Computing and Computer Assisted ...
2021
-
[12]
In: International Conference on Med- ical Image Computing and Computer-Assisted Intervention
Kim, H.B., Ahn, Y.H., Kim, S.T.: Mask-free neuron concept annotation for inter- preting neural networks in medical domain. In: International Conference on Med- ical Image Computing and Computer-Assisted Intervention. pp. 524–533 (2024)
2024
-
[13]
JAMA319(17), 1834 (2018)
Kim, S., Donahue, T.: Laparoscopic cholecystectomy. JAMA319(17), 1834 (2018)
2018
-
[14]
Applied Sciences 12(17), 8746 (2022)
Kirtac, K., Aydin, N., Lavanchy, J.L., Beldi, G., Smit, M., Woods, M.S., Aspart, F.: Surgical phase recognition: From public datasets to real-world data. Applied Sciences 12(17), 8746 (2022)
2022
-
[15]
Medical Image Analysis99, 103366 (2025)
Liu, Y., Boels, M., Garcia-Peraza-Herrera, L.C., Vercauteren, T., Dasgupta, P., Granados, A., Ourselin, S.: Lovit: Long video transformer for surgical phase recog- nition. Medical Image Analysis99, 103366 (2025)
2025
-
[16]
IEEE/CVF International Conference on Computer Vision pp
Liu, Y., Huo, J., Peng, J., Sparks, R., Dasgupta, P., Granados, A., Ourselin, S.: Skit: a fast key information video transformer for online surgical phase recognition. IEEE/CVF International Conference on Computer Vision pp. 21074–21084 (2023)
2023
-
[17]
arXiv preprint arXiv:1806.03184 (2018)
Maier-Hein, L., Eisenmann, M., Feldmann, C., Feussner, H., Forestier, G., Gian- narou, S., Gibaud, B., Hager, G.D., Hashizume, M., Katic, D., et al.: Surgical data science: A consensus perspective. arXiv preprint arXiv:1806.03184 (2018)
2018 arXiv
-
[18]
Annals of Laparoscopic and Endoscopic Surgery5 (2020)
Majumder, A., Altieri, M.S., Brunt, L.M.: How do i do it: laparoscopic cholecys- tectomy. Annals of Laparoscopic and Endoscopic Surgery5 (2020)
2020
-
[19]
Medical Image Analysis78, 102433 (2022)
Nwoye, C.I., Yu, T., Gonzalez, C., Seeliger, B., Mascagni, P., Mutter, D., Marescaux, J., Padoy, N.: Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Medical Image Analysis78, 102433 (2022)
2022
-
[20]
ICLR (2023)
Oikarinen, T., Weng, T.W.: Clip-dissect: Automatic description of neuron repre- sentations in deep vision networks. ICLR (2023)
2023
-
[21]
Medical image analysis16(3), 632–641 (2012)
Padoy, N., Blum, T., Ahmadi, S.A., Feussner, H., Berger, M.O., Navab, N.: Statis- tical modeling and recognition of surgical workflow. Medical image analysis16(3), 632–641 (2012)
2012
-
[22]
In: International conference on machine learning
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763 (2021)
2021
-
[23]
Computers in biology and medicine140, 105111 (2022)
Salahuddin, Z., Woodruff, H.C., Chatterjee, A., Lambin, P.: Transparency of deep neural networks for medical image analysis: A review of interpretability methods. Computers in biology and medicine140, 105111 (2022)
2022
-
[24]
J Hepatobiliary Pancreat Sci 26(4), 123–127 (2019)
Strasberg, S.M.: A three-step conceptual roadmap for avoiding bile duct injury in laparoscopic cholecystectomy: an invited perspective review. J Hepatobiliary Pancreat Sci 26(4), 123–127 (2019)
2019
-
[25]
Temme, M.: Algorithms and transparency in view of the new general data protec- tion regulation. Eur. Data Prot. L. Rev.3, 473 (2017) SurgX 11
2017
-
[26]
IEEE transactions on medical imaging36(1), 86–97 (2016)
Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., De Mathelin, M., Padoy, N.: Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE transactions on medical imaging36(1), 86–97 (2016)
2016
-
[27]
British Machine Vision Conference (BMVC) (2021)
Yi, F., Wen, H., Jiang, T.: Asformer: Transformer for action segmentation. British Machine Vision Conference (BMVC) (2021)
2021
-
[28]
arXiv preprint arXiv:2307.15220 (2023)
Yuan, K., Srivastav, V., Yu, T., Lavanchy, J.L., Mascagni, P., Navab, N., Padoy, N.: Learning multi-modal representations by watching hundreds of surgical video lectures. arXiv preprint arXiv:2307.15220 (2023)
2023 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.