Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

Germain Forestier; Hassan Ismail Fawaz; Jonathan Weber; Lhassane Idoumghar; Pierre-Alain Muller

REVIEW 4 major objections 6 minor 24 references

Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A fully convolutional network can classify surgical skill from raw kinematics at 100% accuracy on two of three tasks, and can point to the movements behind each decision.

desk verdict Competent application of known deep-learning components to a small benchmark, but the reported 100% accuracies are very likely inflated by a leave-one-trial-out protocol that keeps each test subject's other trials in training. read the letter →

arxiv 1908.07319 v1 pith:S7T4YBI4 submitted 2019-08-20 cs.LG cs.AIstat.ML

Hassan Ismail Fawaz , Germain Forestier , Jonathan Weber , Lhassane Idoumghar , Pierre-Alain Muller This is my paper · ORCID

classification cs.LGcs.AIstat.ML

keywords surgicalskillassessmentkinematicdatafullyconvolutionalnetworksglobalaveragepoolingclassactivationmapinterpretablemachinelearningtimeseriesclassificationJIGSAWS

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Manual assessment of surgical trainees by expert observers is slow, expensive, and subjective. This paper argues that a fully convolutional network trained on raw kinematic recordings of robotic surgery can evaluate skill automatically and, unlike earlier black-box classifiers, can explain its decisions. On the JIGSAWS dataset the reported accuracy is 100% micro and macro for suturing and needle passing, and 92.1% micro and 93.2% macro for knot tying, with OSATS score predictions comparable to a published entropy-based baseline. Because the network processes an entire trial without manual gesture boundaries, it points toward routine, objective, interpretable feedback for trainees.

What carries the argument

The load-bearing component is the fully convolutional network whose last convolutional feature map is passed through global average pooling instead of a dense layer. Because the pooled feature for filter $k$, $\sum_t \mathcal{A}_k(t)$, is weighted by $w^c_k$, the class activation map $\mathcal{M}_c(t)=\sum_k w^c_k \mathcal{A}_k(t)$ decomposes the output score into per-time-step contributions. This identity is what turns the classifier into an explainer: after training, plotting $\mathcal{M}_c(t)$ over the trial's trajectory shows which kinematic subsequences were responsible for classifying a surgeon as novice, intermediate, or expert, or for predicting an OSATS sub-score. The convolutions are also structured by domain knowledge, with separate filters for each manipulator and for kinematic subgroups (position, linear and rotational velocity, rotation matrix, gripper speed), and the use of global pooling lets the network accept trials of variable length.

What would settle it

Train the same network to say which of the eight surgeons is moving, using the same one-surgeon-out testing scheme; if it identifies held-out surgeons from their motion about as well as or better than it grades their skill, the skill results could be driven by identity rather than expertise.

Watch

Extended reading notes

Core claim

The paper's central claim is that an end-to-end fully convolutional network with global average pooling can both grade and explain surgical skill from the 76 kinematic channels recorded during robot-assisted surgery. For classification, the network outputs novice, intermediate, or expert; for regression, it predicts each of the six modified OSATS sub-scores. Under leave-one-super-trial-out validation on JIGSAWS, the authors report 100% micro and macro accuracy on suturing and needle passing, 92.1% micro and 93.2% macro on knot tying, and mean rank correlations of 0.60, 0.57, and 0.65 across the three tasks. The explanation comes from a class activation map, derived from the global average pooling weights, which assigns a contribution value to each time step and highlights which motion segments drove the prediction.

Load-bearing premise

The evaluation assumes that testing on each surgeon one at a time measures surgical skill rather than which surgeon is moving; with only two experts in the dataset, the model might recognize an individual's motion patterns instead of genuine expertise.

Editorial extensions

If this is right

Skill evaluation could run automatically after each training trial, without an expert surgeon in the loop and without manual gesture segmentation.
The heatmaps give trainees concrete, segment-level feedback, such as which 'pulling suture' motion marked them as novice, rather than just a single numeric score.
Because the architecture accepts variable-length trials, no resampling or window-based preprocessing is needed to compare performances across trials.
On the JIGSAWS benchmark, the method matches or beats the published baselines: 100% on suturing and needle passing, 92.1%/93.2% on knot tying, and rank correlations comparable to or higher than the entropy-based approach on all tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One implication left implicit is that the same global-pooling/class-activation machinery applies to other multivariate time-series tasks where practitioners need to know why a prediction was made, such as rehabilitation monitoring; the per-time-step attribution formula carries over unchanged.
A useful control the paper does not report is an identity-classification experiment: if an eight-way 'which surgeon is moving' classifier reaches similar one-surgeon-out accuracy, part of the skill signal could be subject identity rather than skill level.
The hand-built channel grouping is a design choice; ablating it by feeding all 76 channels through ungrouped convolutions would quantify how much of the accuracy comes from the kinematic-domain structure versus the fully convolutional network itself.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a clean, honest application of fully convolutional networks plus class activation maps to surgical skill evaluation on JIGSAWS. The architecture is sensible, with domain-informed channel grouping that is easy to follow. The authors include code, compare against several published baselines, and explicitly flag the dataset's limitations in their Section 5. What's genuinely new here is the regression extension that predicts six OSATS subscores, and the demonstration that CAM heatmaps can be produced for both classification and regression. The classification numbers themselves already appeared in their MICCAI 2018 paper, so the incremental novelty is modest.

The soft spot is load-bearing. They use leave-one-super-trial-out, which holds out one trial per subject while leaving that same subject's other four trials in the training set. As Section 5 admits, 'a surgeon should be present in the training set in order to make a prediction.' Under that protocol, a network can identify the subject's individual motor signature and classify held-out trials by identity rather than by skill. The 100% accuracy on suturing and needle passing is thus far less impressive than it looks—it may say nearly nothing about generalizing to an unseen surgeon. The stress-test note is correct, and the Section 5 concession effectively confirms it.

Other issues are real but secondary. The dataset has only eight subjects, two of them experts; Table 1 gives point estimates without variance despite averaging over 40 runs; and the regression evaluation uses only rank correlation, not calibration or error magnitude. The CAM analysis is illustrative, but the authors themselves say it needs validation with expert surgeons. On the positive side, the paper does not oversell: the limitations section is unusually candid, and the comparison with prior work is fair.

Who gets value from this? Someone working on surgical data science who wants a clear example of an interpretable deep model on JIGSAWS and a cautionary tale about evaluation protocols. It is not a paper whose headline accuracy should be cited as evidence of skill classification performance.

For peer review: yes, it deserves a serious referee—but the referee should push for a proper leave-one-user-out analysis (with the caveat that only two experts make that near-impossible on this dataset), error bars, and a direct test of identity versus skill. Without such a revision, the central claim doesn't hold. As it stands, it's a useful methodological prototype, not a validated result.

Referee Report

4 major / 6 minor

Summary. This paper proposes a fully convolutional neural network (FCN) for surgical skill evaluation from kinematic data in the JIGSAWS dataset. The architecture applies grouped convolutions over the 76 kinematic channels (organized by manipulator and kinematic type), global average pooling, and class activation maps to produce both skill classifications (novice/intermediate/expert) and OSATS regression scores. Three separate models are trained for suturing, needle passing, and knot tying. Results are reported under a leave-one-super-trial-out (LOSO) scheme and compared with published S-HMM, ApEn, SAX-VSM, and CNN baselines. The authors report 100% micro/macro accuracy for suturing and needle passing, 92.1%/93.2% for knot tying, and Spearman correlations of 0.60, 0.57, and 0.65 for OSATS regression, and they claim state-of-the-art performance as well as interpretable feedback via CAM heatmaps.

Significance. If the reported results were valid evidence of skill-level generalization, the paper would make a useful practical contribution: it offers an end-to-end architecture that avoids manual gesture segmentation, provides per-trial heatmaps via CAM, and ships code and comparisons to existing baselines. The grouping of kinematic channels by manipulator and physical meaning is a reasonable inductive bias. However, the current evidence does not support the broad claim of accurate and interpretable surgical-skill evaluation: the LOSO protocol leaves the same subject's other trials in the training set, and the interpretability component is not validated. The significance is therefore conditional on substantially stronger evaluation or on appropriately narrowed claims.

major comments (4)

[Section 3 (LOSO) and Table 1] The evaluation protocol cannot distinguish skill-level generalization from subject-identity recognition. Under leave-one-super-trial-out, the held-out trial comes from a subject whose other trials are in the training set, so the network can classify by matching an individual motor signature rather than by any skill-level concept. The paper's own Section 5 states that 'a surgeon should be present in the training set in order to make a prediction' and that this is 'a huge problem originating from the limited dataset size.' This is not a peripheral caveat: the 100% accuracies in Table 1 are the main evidence for the paper's central claim. The authors should report a leave-one-user-out experiment, a subject-identification control, or per-subject confusion matrices, and must temper the state-of-the-art claim according to what those results show.
[Section 4 and Table 1] The reported FCN accuracies are point estimates without dispersion. The text states that results are averaged over 40 runs, but no standard deviation, confidence interval, or significance test is given. With only 40 trials per task (8 subjects x 5 trials), one misclassification changes micro accuracy by 2.5 percentage points, so the 92.1% versus 97.4% difference for knot tying is within the sampling error of a handful of trials. The claim that the FCN is 'much more accurate' than S-HMM is therefore unsupported; per-fold or per-run results, or confidence intervals, are required.
[Section 4 (regression)] Spearman's rho is insufficient to support the regression claim. The paper reports only rank correlation averaged over the six OSATS targets, without absolute error, calibration, per-target breakdown, or uncertainty. The values (0.60, 0.57, 0.65) are modest, and the comparison is made against only one published method (ApEn). Because the same LOSO protocol applies, the same subject-identity confound affects the regression results. The authors should report mean absolute error or RMSE, per-target correlations with confidence intervals, and ideally subject-independent performance.
[Section 4 and Figures 2-3] The interpretability claim is not validated. A CAM heatmap is by construction a linear combination of the same weights that produced the prediction; it shows which parts of the input the model used, not that those parts correspond to meaningful surgical gestures or useful feedback. The text interprets the highlighted subsequence as the gesture 'pulling suture' without any gesture annotation or expert verification. Section 5 itself concedes that the feedback technique would benefit from real use-case validation. The conclusion should be limited to 'the model highlights discriminative segments' unless the authors provide evidence that the highlighted segments are clinically meaningful.

minor comments (6)

[Section 3] The term 'super trial' is used in the LOSO description but never defined in this paper; the reader should be told that a super trial contains one trial from each of the eight subjects.
[Section 3 and Section 4] Section 3 says the effects of the learning rate and l2 regularization are discussed in Section 4, but Section 4 contains no such sensitivity analysis; either add the analysis or remove the pointer.
[Section 4 (macro measure)] The macro metric is described as 'boiling down to computing the precision metric'; the exact formula should be given, since macro-averaged precision and macro-averaged recall can differ.
[Section 4 (knot tying)] The explanation for the lower knot-tying accuracy (mean OSATS 17.7 versus 17.1 for experts and intermediates) is asserted without supporting data; report the class distributions and score distributions for that task.
[Section 4 (interpretability comparison)] The statement that the proposed method does 'not sacrifice accuracy for interpretability' is a comparison to the SAX-VSM baseline only; no controlled comparison is made against an equally accurate but less interpretable model.
[Abstract and Conclusion] The abstract says 'new competitive state-of-the-art performance,' while the conclusion says 'new state-of-the-art performance'; Table 1 shows the proposed method is not the best on the knot-tying regression task, so the wording should be consistent and precise.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's accuracy and regression claims are benchmarked against external methods, and the CAM explanation is an algebraic decomposition of the model's own decision, not a fitted input disguised as a prediction.

full rationale

The paper's central claims—accurate surgical skill classification and OSATS-score regression from kinematic data—are evaluated against external published baselines (S-HMM, ApEn, SAX-VSM, and a prior CNN) on the JIGSAWS dataset, with results reported in Table 1. These comparisons are not derived from the model's own fitted values, and no parameter is fitted to the evaluation target and then re-reported as a prediction. The class activation map in Eq. (2) defines M_c(t) = sum_k w_c^k A_k(t), and the class score z_c = sum_t M_c(t); this is a mathematical decomposition of the network's pre-softmax output, so the heatmap explains the model by construction. That is an interpretability property, not a circular derivation of the accuracy claim. The paper's reference to its own prior work [11] for the original classification task is a normal continuity citation; the current paper reports the experimental setup, architecture, and results directly, and it does not rely on an unverified self-citation to justify its main conclusions. The limitation acknowledged in Section 5—that LOSO evaluation requires the surgeon to appear in training and that only two experts exist—is a dataset and external-validity concern about whether the task measures skill versus subject identity, not a circularity in the derivation chain. Therefore, no load-bearing step reduces to its own input by definition or by fitted-parameter renaming.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests mostly on assumptions about the dataset and the validity of the targets, not on new invented entities. The only hand-chosen elements are network hyperparameters and the channel grouping scheme. No new forces, particles, or conserved quantities are introduced.

free parameters (6)

convolution kernel length = 3
Chosen by hand without validation; the temporal receptive field of each convolution and hence the granularity of CAM explanations depend on it.
number of filters per convolutional layer = 8, 16, 32
Chosen by hand, doubling per layer; controls model capacity and CAM feature dimension K=32.
learning rate (Adam) = 0.001
Default chosen without validation; optimization trajectory and final checkpoint depend on it.
Adam optimizer moment estimates = 0.9 and 0.999
Default hyperparameters for the Adam optimizer used to train the network.
l2 regularization coefficient = 1e-5
Default chosen to avoid overfitting; Section 4 says its effect is discussed but no systematic results are reported.
maximum epochs with validation checkpoint = 1000 with best-validation checkpoint
Early stopping proxy on a random non-seen split; affects final model selection.

assumptions (5)

domain assumption The JIGSAWS self-proclaimed skill levels (N, I, E) and modified OSATS scores are valid ground truth for surgical skill.
Section 2 describes these labels as the prediction targets; if the labels are noisy or subjective, the reported accuracy measures label predictability, not surgical skill.
domain assumption Kinematic data alone contains sufficient discriminative information to evaluate surgical skill.
The entire pipeline uses only the 76 kinematic channels (Section 2, Methods); no video or patient context is used.
domain assumption LOSO evaluation on 8 subjects generalizes to unseen surgeons.
Table 1 reports LOSO results; Section 5 admits the dataset is too small and that only two experts exist, so this assumption is fragile.
ad hoc to paper The hand-designed grouping of 76 channels into four manipulators and five kinematic subgroups is a valid inductive bias.
Methods Section 3: 'We used domain knowledge when grouping the different channels'; the architecture's first two convolutional layers depend on this grouping.
domain assumption Class activation maps provide clinically meaningful explanations of skill evaluation.
Section 4 presents CAM heatmaps as feedback, but Section 5 notes they 'would benefit from an extended real use-case validation process'; semantic relevance is not established.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks." pith.science (2026). https://pith.science/paper/S7T4YBI4

@misc{pith2026190807319,
  author       = {Pith},
  title        = {Pith review of: Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S7T4YBI4}},
  note         = {Machine review of arXiv:1908.07319}
}

read the original abstract

Purpose: Manual feedback from senior surgeons observing less experienced trainees is a laborious task that is very expensive, time-consuming and prone to subjectivity. With the number of surgical procedures increasing annually, there is an unprecedented need to provide an accurate, objective and automatic evaluation of trainees' surgical skills in order to improve surgical practice. Methods: In this paper, we designed a convolutional neural network (CNN) to classify surgical skills by extracting latent patterns in the trainees' motions performed during robotic surgery. The method is validated on the JIGSAWS dataset for two surgical skills evaluation tasks: classification and regression. Results: Our results show that deep neural networks constitute robust machine learning models that are able to reach new competitive state-of-the-art performance on the JIGSAWS dataset. While we leveraged from CNNs' efficiency, we were able to minimize its black-box effect using the class activation map technique. Conclusions: This characteristic allowed our method to automatically pinpoint which parts of the surgery influenced the skill evaluation the most, thus allowing us to explain a surgical skill classification and provide surgeons with a novel personalized feedback technique. We believe this type of interpretable machine learning model could integrate within "Operation Room 2.0" and support novice surgeons in improving their skills to eventually become experts.

Figures

Figures reproduced from arXiv: 1908.07319 by the authors.

**Figure 2.** Using class activation map (CAM) to provide explainable classification [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Feedback using the CAM on subject E’s second knot-tying trial [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 23 canonical work pages

[1]

IEEE Transactions on Biomedical Engineering 64(9):2025–2041

Ahmidi N, Tao L, Sefati S, Gao Y, Lea C, Haro BB, Zappella L, Khudanpur S, Vidal R, Hager GD (2017) A dataset and benchmarks for segmentation and recogni- tion of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering 64(9):2025–2041

work page 2017
[2]

BMJ 327(7405):13–17

Bridgewater B, Grayson AD, Jackson M, Brooks N, Grotte GJ, Keenan DJ, Millner R, Fabri BM, Mark J (2003) Surgeon speciﬁc mortality in adult cardiac surgery: comparison between crude and risk stratiﬁed data. BMJ 327(7405):13–17

work page 2003
[3]

https://keras.io

Chollet Fea (2015) Keras. https://keras.io

work page 2015
[4]

In: Artiﬁcial Intelli- gence in Medicine, pp 136–145

Forestier G, Petitjean F, Senin P, Despinoy F, Jannin P (2017) Discovering discriminative and interpretable pat- terns for surgical motion analysis. In: Artiﬁcial Intelli- gence in Medicine, pp 136–145

work page 2017
[5]

Artiﬁcial Intelligence in Medicine 91:3 – 11

Forestier G, Petitjean F, Senin P, Despinoy F, Huaulm A, Ismail Fawaz H, Weber J, Idoumghar L, Muller PA, Jannin P (2018) Surgical motion analysis using discrim- inative interpretable patterns. Artiﬁcial Intelligence in Medicine 91:3 – 11

work page 2018
[6]

In: Modeling and Monitoring of Computer Assisted In- terventions MICCAI Workshop

Gao Y, Vedula SS, Reiley CE, Ahmidi N, Varadarajan B, Lin HC, Tao L, Zappella L, B´ ejar B, Yuh DD, Chen CCG, Vidal R, Khudanpur S, Hager GD (2014) The JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS): A surgical activity dataset for human motion modeling. In: Modeling and Monitoring of Computer Assisted In- terventions MICCAI Workshop

work page 2014
[7]

In: Interna- tional Conference on Artiﬁcial Intelligence and Statistics, vol 9, pp 249–256

Glorot X, Bengio Y (2010) Understanding the diﬃculty of training deep feedforward neural networks. In: Interna- tional Conference on Artiﬁcial Intelligence and Statistics, vol 9, pp 249–256

work page 2010
[8]

assessment of technical skills (OSATS): a systematic re- view of validity evidence

Hatala R, Cook DA, Brydges R, Hawkins R (2015) Con- structing a validity argument for the objective structured 8 Hassan Ismail Fawaz et al. assessment of technical skills (OSATS): a systematic re- view of validity evidence. Advances in Health Sciences Education 20(5):1149–1175

work page 2015

Show all 24 references

[9]

Intuitive Surgical Sunnyvale CA (2018) The Da Vinci Surgical System

2018
[10]

Journal of Biomedical Informatics 59:102 – 114

Islam G, Kahol K, Li B, Smith M, Patel VL (2016) Af- fordable, web-based surgical skill training and evaluation tool. Journal of Biomedical Informatics 59:102 – 114

2016
[11]

In: Inter- national Conference On Medical Image Computing and Computer Assisted Intervention, pp 214–221

Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller PA (2018) Evaluating surgical skills from kine- matic data using convolutional neural networks. In: Inter- national Conference On Medical Image Computing and Computer Assisted Intervention, pp 214–221

2018
[12]

In: IEEE International Conference on Big Data, pp 1367–1376

Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller PA (2018) Transfer learning for time series classi- ﬁcation. In: IEEE International Conference on Big Data, pp 1367–1376

2018
[13]

Data Mining and Knowledge Discovery

Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller PA (2019) Deep learning for time series classiﬁca- tion: a review. Data Mining and Knowledge Discovery

2019
[14]

International Journal of Computer Assisted Radiology and Surgery 11(4):553– 568

Kassahun Y, Yu B, Tibebu AT, Stoyanov D, Giannarou S, Metzen JH, Vander Poorten E (2016) Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. International Journal of...

2016
[15]

In: International Conference on Learning Representations

Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: International Conference on Learning Representations

2015
[16]

Nature Biomedical Engi- neering 1(9):691–696

Maier-Hein L, Vedula SS, Speidel S, Navab N, Kiki- nis R, Park A, Eisenmann M, Feussner H, Forestier G, Giannarou S, Hashizume M, Katic D, Kenngott H, Kranzfelder M, Malpani A, M¨ arz K, Neumuth T, Padoy N, Pugh C, Schoch N, Stoyanov D, Taylor R, Wagner M, Hager GD, Jannin P (...

2017
[17]

Surgery Today 43(3):271–275

Niitsu H, Hirabayashi N, Yoshimitsu M, Mimura T, Tao- moto J, Sugiyama Y, Murakami S, Saeki S, Mukaida H, Takiyama W (2013) Using the Objective Structured As- sessment of Technical Skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating ...

2013
[18]

Bulletin of the American College of Surgeons 98(7):22–29

Polavarapu HV, Kulaylat A, Sun S, Hamed O (2013) 100 years of surgical education: the past, present, and future. Bulletin of the American College of Surgeons 98(7):22–29

2013
[19]

In: Information Pro- cessing in Computer-Assisted Interventions, pp 167–177

Tao L, Elhamifar E, Khudanpur S, Hager GD, Vidal R (2012) Sparse hidden markov models for surgical gesture classiﬁcation and skill evaluation. In: Information Pro- cessing in Computer-Assisted Interventions, pp 167–177

2012
[20]

Public Library of Science One 11(3):1–14

Vedula SS, Malpani AO, Tao L, Chen G, Gao Y, Poddar P, Ahmidi N, Paxton C, Vidal R, Khudanpur S, Hager GD, Chen CCG (2016) Analysis of the structure of sur- gical activity for a suturing and knot-tying task. Public Library of Science One 11(3):1–14

2016
[21]

International Journal of Com- puter Assisted Radiology and Surgery 13(12):1959–1970

Wang Z, Majewicz Fey A (2018) Deep learning with con- volutional neural network for objective skill evaluation in robot-assisted surgery. International Journal of Com- puter Assisted Radiology and Surgery 13(12):1959–1970

2018
[22]

In: International Joint Conference on Neural Net- works, pp 1578–1585

Wang Z, Yan W, Oates T (2017) Time series classiﬁcation from scratch with deep neural networks: A strong base- line. In: International Joint Conference on Neural Net- works, pp 1578–1585

2017
[23]

In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2921–2929

Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative local- ization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2921–2929

2016
[24]

International Journal of Computer As- sisted Radiology and Surgery 13(5):731–739

Zia A, Essa I (2018) Automated surgical skill assessment in rmis training. International Journal of Computer As- sisted Radiology and Surgery 13(5):731–739

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

IEEE Transactions on Biomedical Engineering 64(9):2025–2041

Ahmidi N, Tao L, Sefati S, Gao Y, Lea C, Haro BB, Zappella L, Khudanpur S, Vidal R, Hager GD (2017) A dataset and benchmarks for segmentation and recogni- tion of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering 64(9):2025–2041

work page 2017

[2] [2]

BMJ 327(7405):13–17

Bridgewater B, Grayson AD, Jackson M, Brooks N, Grotte GJ, Keenan DJ, Millner R, Fabri BM, Mark J (2003) Surgeon speciﬁc mortality in adult cardiac surgery: comparison between crude and risk stratiﬁed data. BMJ 327(7405):13–17

work page 2003

[3] [3]

https://keras.io

Chollet Fea (2015) Keras. https://keras.io

work page 2015

[4] [4]

In: Artiﬁcial Intelli- gence in Medicine, pp 136–145

Forestier G, Petitjean F, Senin P, Despinoy F, Jannin P (2017) Discovering discriminative and interpretable pat- terns for surgical motion analysis. In: Artiﬁcial Intelli- gence in Medicine, pp 136–145

work page 2017

[5] [5]

Artiﬁcial Intelligence in Medicine 91:3 – 11

Forestier G, Petitjean F, Senin P, Despinoy F, Huaulm A, Ismail Fawaz H, Weber J, Idoumghar L, Muller PA, Jannin P (2018) Surgical motion analysis using discrim- inative interpretable patterns. Artiﬁcial Intelligence in Medicine 91:3 – 11

work page 2018

[6] [6]

In: Modeling and Monitoring of Computer Assisted In- terventions MICCAI Workshop

Gao Y, Vedula SS, Reiley CE, Ahmidi N, Varadarajan B, Lin HC, Tao L, Zappella L, B´ ejar B, Yuh DD, Chen CCG, Vidal R, Khudanpur S, Hager GD (2014) The JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS): A surgical activity dataset for human motion modeling. In: Modeling and Monitoring of Computer Assisted In- terventions MICCAI Workshop

work page 2014

[7] [7]

In: Interna- tional Conference on Artiﬁcial Intelligence and Statistics, vol 9, pp 249–256

Glorot X, Bengio Y (2010) Understanding the diﬃculty of training deep feedforward neural networks. In: Interna- tional Conference on Artiﬁcial Intelligence and Statistics, vol 9, pp 249–256

work page 2010

[8] [8]

assessment of technical skills (OSATS): a systematic re- view of validity evidence

Hatala R, Cook DA, Brydges R, Hawkins R (2015) Con- structing a validity argument for the objective structured 8 Hassan Ismail Fawaz et al. assessment of technical skills (OSATS): a systematic re- view of validity evidence. Advances in Health Sciences Education 20(5):1149–1175

work page 2015