CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operation

Kapotaksha Das; Mohamed Abouelenien; Naihao Deng; Rada Mihalcea; Vitaliy Popov

REVIEW 5 major objections 4 minor 12 references

CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operation

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces CliniDial, a multimodal dialogue dataset recorded from simulated surgical operations, and shows that current LLMs and fine-tuned encoders reach at best a macro F1 of 51.09 on teamwork behavior classification, arguing…

desk verdict Genuinely new clinical teamwork dataset with useful analyses, but the annotation reliability evidence and missing data release weaken the benchmark claims. read the letter →

arxiv 2506.12936 v1 pith:7MAWWDNK submitted 2025-06-15 cs.CL

Naihao Deng , Kapotaksha Das , Rada Mihalcea , Vitaliy Popov , Mohamed Abouelenien This is my paper

classification cs.CL

keywords CliniDialmultimodaldialoguedatasetteamreflectionbehavior-codeclassificationclinicalsimulationclassimbalancelargelanguagemodelspatientphysiologysignals

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CliniDial is a new multimodal dialogue dataset built from 22 simulated operating-room sessions, with more than 6.5 thousand utterances transcribed and each utterance labeled for teamwork behavior: seeking input, evaluating, planning, implementing, or none. The paper's central assertion is that this naturally unfolding, heavily imbalanced, conversational clinical data is difficult for today's models: the best evaluated system, GPT-4o with five demonstrations, reaches a macro F1 of only 51.09 on the five-class classification task, and adding video or physiological-signal inputs does not lift that score. If the assertion holds, automated analysis of team reflection in clinical operations is far from solved, and CliniDial offers a realistic stress test for models that must handle imbalanced, multimodal, domain-specific dialogue.

What carries the argument

The load-bearing object is the CliniDial corpus itself: 22 simulated operating-room sessions, 6.5k turns, 49.9k words, nine timestamp-aligned physiological signals, and two camera views per session. Its labels come from a five-class behavior-code scheme—'Seek,' 'Evaluate,' 'Plan,' 'Implement,' and 'None'—adapted from the TURBO team-reflection behavioral framework, and the paper's case studies are built around three properties of this corpus: label imbalance, long natural conversations averaging 311 turns per session, and multimodal inputs that can be aligned to each utterance by timestamp. The timestamps are what make the multimodal fusion experiments possible, and the imbalance in the label distribution is what makes macro F1 the decisive metric rather than micro F1.

What would settle it

Re-annotate the 16 independently coded sessions with a second annotator and compute per-session Cohen's kappa; if agreement on those sessions falls well below 0.73, the label distribution and the model F1 scores would change, and the claim that CliniDial reliably challenges current models would no longer be supported.

Watch

Extended reading notes

Core claim

CliniDial's central claim is that existing LLMs and fine-tuned encoders are not yet capable of reliably classifying teamwork behavior in naturally occurring clinical dialogue. Across ten-fold cross-validation, the strongest result is GPT-4o with five in-context demonstrations, which reaches a macro F1 of 51.09; fine-tuned BERTbase reaches a higher micro F1 of 66.6% but a macro F1 of only 48.6%, showing a bias toward the majority 'None' class. Experiments that add conversational context give GPT-4o a modest boost (macro F1 from 47.3% at 1-shot with no context to 49.8% with a three-turn context) while hurting Llama 70B, and experiments that add video frames or physiology screenshots either leave performance flat or lower it, with verbalized physiology signals dropping macro F1 to 42.8%. The authors interpret this as evidence that CliniDial poses significant challenges to existing models and invites methods designed for real-world clinical data.

Load-bearing premise

The benchmark results assume that the annotator agreement measured on 6 of the 22 sessions (Cohen's kappa 0.73) carries over to the remaining 16 sessions, which were coded independently by the two annotators without any reported reliability check.

Editorial extensions

If this is right

Automated behavior-code annotation of operating-room team communication is not yet reliable enough for clinical use, since the best model's macro F1 of 51.09 leaves roughly half of the average class's instances misclassified.
Fine-tuned encoders such as BERTbase favor the majority 'None' class: micro F1 of 66.6% hides a macro F1 of 48.6%, so imbalance-aware methods are a prerequisite for progress.
Adding conversational context helps GPT-4o modestly (macro F1 from 47.3% to 49.8% at 1-shot with context size 3) but hurts Llama 70B, so context handling is model-dependent and cannot be assumed beneficial.
Presenting raw video frames or physiology screenshots to GPT-4o does not improve over text alone, and verbalizing physiological signals lowers macro F1 to 42.8%, indicating multimodal fusion in clinical domains remains open.
With 6.9k annotated examples and timestamps across modalities, CliniDial can support supervised fine-tuning and continual pretraining for clinical-domain dialogue models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension the paper leaves open is chain-of-thought prompting or clinical-domain fine-tuning; if either closes much of the gap below GPT-4o's 51.09 macro F1, the bottleneck is reasoning or domain knowledge rather than the data itself.
Because every utterance carries a speaker role, role-conditioned classification—telling the model whether the speaker is trainee, surgeon, or support—is a natural next benchmark; the reported role-specific label distributions suggest it could disambiguate 'seek' from 'implement' utterances.
The timestamp alignment makes cross-modal prediction possible: hold out the text and ask the model to predict the speaker's next behavior code from physiology trends and video alone, which would test whether models can infer team state from patient state rather than from wording.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

CliniDial is a genuinely new resource. It combines unscripted operating-room dialogue from simulated malignant hyperthermia cases with two camera views and nine aligned physiological signals, labeled with a team-reflection behavior codebook. That combination does not exist in the cited literature, and the role-specific language analyses are useful (who says 'thank you' vs 'alright', dantrolene counts). The case study where GPT-4o misreads the EtCO2 trace as heart rate is a concrete, valuable failure example. The authors are also honest about the simulation setting and the limits of what they analyze.

The soft spots are concentrated in the benchmark. The reported numbers come from a single 10-fold cross-validation run with no variance, no significance tests, and no human-expert baseline, so the small differences between text-only and multimodal conditions could easily be noise. And 'only 51.09' macro-F1 cannot be properly calibrated without knowing what an expert annotator achieves on the same codebook. More fundamentally, the annotation reliability evidence is thin: Cohen's kappa 0.73 is computed on 6 of 22 sessions, after discussion and social moderation, and the remaining 16 sessions were coded independently by the two annotators with no reported agreement check or adjudication. The rare classes (Plan 4.5%, Implement 9.0%) drive macro-F1 and are the most likely places for annotator drift. If those sessions are noisier, the model scores are partly bounded by label noise. This is a genuine concern for a dataset paper and needs to be addressed, either by double-coding a subset of the remaining sessions or reporting pre-discussion agreement. The multimodal section also does not state which shot setting it uses; the text-only baseline of 48.2 looks like 0-shot, but it should be explicit.

The data access issue is also serious: this is a resource paper, yet the text data and video are only 'provided upon reasonable requests,' and the ethics statement frames the transcript release as still under consideration. Code without data is not a dataset. The authors should either release the data or describe a concrete access process, then the resource becomes much more useful.

None of this kills the paper. The dataset fills a real gap, and the experiments are a reasonable first pass, even if underpowered. With better reliability evidence, variance estimates, and a data-release plan, this becomes a solid contribution. As it stands, it deserves a serious referee, but the referee should ask for those revisions.

Referee Report

5 major / 4 minor

Summary. CliniDial is a multimodal dataset collected from simulated operating-room sessions: 22 sessions, 6.5k turns, about 49.9k words, with audio/transcripts, two camera views, and nine physiological signals. The utterances are annotated with five behavior codes (None, Seek, Evaluate, Plan, Implement), following Schmutz et al. (2021) plus a self-defined Implement label. The authors analyze three dataset characteristics — label imbalance, conversational richness, and multimodality — and benchmark fine-tuned BERT and several LLMs (GPT-4, GPT-4o, Llama 3 8B/70B) under different prompting, context, and input-modality settings. The best reported result is GPT-4o with 5-shot prompting at 51.09 macro F1, which the paper interprets as evidence that CliniDial poses significant challenges to existing models.

Significance. If the label quality and evaluation protocol hold up, CliniDial would fill a real gap as one of the first clinical team-communication datasets combining naturalistic spoken dialogue, two camera views, and time-aligned physiological signals. Its extreme imbalance (None 55.2%, Plan 4.5%) and long, role-structured conversations make it a useful stress test for imbalanced conversational clinical NLP. The paper is also commendable for using an established behavioral framework, providing a codebase, and including three targeted case studies with several model families. However, the benchmark's validity currently rests on thin annotation-reliability evidence and on point estimates without uncertainty quantification, so the dataset's advertised role as a reliable model benchmark is not yet fully established.

major comments (5)

[Appendix C.2] The annotation-reliability evidence is insufficient to support the gold labels used in every experiment. Cohen's kappa 0.73 was computed on 6 of 22 sessions after the two coders had discussed findings and resolved discrepancies through social moderation; no pre-discussion kappa is reported, no per-class kappa or raw agreement is given, and the remaining 16 sessions were coded independently with no reported agreement check or adjudication. Because the rare classes Plan (4.5%) and Implement (9.0%) dominate macro F1 and are plausibly the hardest to label, the reported 51.09 macro-F1 ceiling may be partly an artifact of label noise rather than a measure of model capability. The authors should report pre- and post-discussion kappa, per-class agreement, and either dual-code a subset of the remaining 16 sessions or provide a complete adjudication protocol.
[Sections 4.2, 5.2, 6.2] All reported macro and micro F1 scores are single averages over ten folds with no variance or significance testing. The qualitative conclusions that conversational context helps and that multimodal inputs hurt rest on differences of only one to three points (e.g., GPT-4o macro F1 48.2 text-only versus 46.8 with video and 44.9 with physiology in Figure 6; 47.3 at context size 1 versus 49.8 at context size 3 in Figure 5). For a dataset with this class imbalance, such differences are within plausible noise range. The authors should report per-fold results, standard deviations or confidence intervals, and paired significance tests (e.g., bootstrap or corrected t-tests) for the key comparisons.
[Abstract and Section 8] The abstract and conclusion interpret the 51.09 macro F1 as 'only' and as evidence of 'significant challenges,' but no human-expert baseline is reported. Without knowing the macro F1 that trained annotators or medical experts achieve on the same codebook and sampling scheme, the reader cannot calibrate whether 51.09 reflects a model limitation or an inherent difficulty of the labeling task. A human-expert evaluation on a held-out subset, or at least a per-class human upper-bound estimate, should be added before making the 'significant challenges' claim.
[Sections 3.1 and Appendix B] The title and abstract describe CliniDial as 'naturally occurring' dialogue, but Section 3.1 and Appendix B state that 'a confederate plays the role of the surgeon' and that the surgeon role 'served as a confederate along with the course instructors,' with the scenario scripted to progress through malignant hyperthermia. This means part of the dialogue is deliberately staged, so the term 'naturally occurring' is overstated. The paper should clarify the degree of scripting, distinguish confederate behavior from trainee behavior in the analysis, and adjust the wording accordingly.
[Sections 3.3 and 4.1] The evaluation protocol does not specify how the few-shot demonstrations were selected per fold. Section 4.1 states only that 'we provide corresponding examples along with the label definitions to the models,' without saying whether the demonstrations come from the training fold of each cross-validation split. If the demonstrations are drawn from the entire dataset, test-session utterances could appear in the prompt, making the reported scores uninterpretable as generalization estimates. The authors should specify the demonstration sampling procedure and ensure it is per-fold training-only.

minor comments (4)

[Appendix B] Appendix B cites 'Hopkins et al., 2020; Rosenberg et al., 2020' for MH treatment algorithms, but these references are not present in the reference list; please add them.
[Figure 6 caption] The caption contains typographical errors: 'standars' and 'standard' should be 'stands'; also, please define T, V, and P in the caption or in the figure legend.
[Section 7] The claim 'we are the first to propose a dataset that includes the physiological signals' is too broad, since physiological signals appear in existing medical datasets that combine text and vitals (e.g., MIMIC); please narrow the claim to the relevant scope, for example 'the first multimodal dialogue dataset with time-aligned physiological signals for team communication in clinical operations.'
[Appendix E] Per-class F1 scores are not reported, which would more directly substantiate the claim that models struggle on minority classes such as Plan and Implement; please include per-class precision, recall, and F1 for the main system configurations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CliniDial reports empirical benchmark measurements against externally sourced labels, with no fitted parameter or self-citation chain forcing the results.

full rationale

CliniDial is a dataset and benchmark paper, not a derivation. The label taxonomy comes from an external framework (Schmutz et al., 2021), plus a self-defined 'Implement' category defined in Section 3.2 and Appendix A before any model is run; the labels are therefore fixed inputs, and the paper never claims to predict them from first principles. The reported macro F1 of 51.09 (Section 8) is an empirical measurement of GPT-4o under a fixed 5-shot prompt, and the lower F1 on minority classes is a measured outcome, not a quantity constructed from the label definitions. The few-shot demonstrations and label descriptions are part of the evaluation protocol, not fitted parameters that predetermine the test scores. The self-citations present (e.g., Castro et al., 2022, in related work; the MichiganNLP codebase URL) are contextual and never used as evidence for the benchmark result. The Appendix C.2 statement that only six of twenty-two sessions were used for the reported Cohen's kappa of 0.73 is an explicit limitation; it bears on label reliability and the external validity of the 'significant challenges' interpretation, but it is not a circularity in the derivation. The same applies to the Limitations section's acknowledgments about simulation, scope, and missing audio analysis. No fitted-parameter call is relabeled as a prediction, no uniqueness theorem is imported, and no known result is renamed. Therefore there is no circular step.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces a new dataset and evaluation but no new theoretical entities or free parameters. Its central claims rest on the validity of the annotation framework, the generalizability of the reported inter-rater reliability, and the representativeness of the simulation. These are domain assumptions rather than standard mathematical axioms.

assumptions (3)

domain assumption The team reflection behavior framework of Schmutz et al. (2021) applies to this simulated clinical operation data, and the added 'Implement' label is a valid extension.
The annotation scheme in Section 3.2 and Appendix A adopts the framework and adds a label without validating the extension against an external criterion.
domain assumption The inter-rater reliability measured on 6 of 22 sessions (Cohen's kappa 0.73) is representative of the labeling quality on the remaining 16 sessions annotated independently.
Appendix C.2 reports agreement only on six randomly selected files; no agreement is reported for the rest of the dataset.
domain assumption The simulated operating-room scenario (malignant hyperthermia in a mannequin) yields communication patterns informative for real clinical operations.
The Limitations section acknowledges the simulation likely lacks the urgency of real medical settings, yet the dataset is framed as a step toward real-world clinical NLP.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operation." pith.science (2026). https://pith.science/paper/7MAWWDNK

@misc{pith2026250612936,
  author       = {Pith},
  title        = {Pith review of: CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MAWWDNK}},
  note         = {Machine review of arXiv:2506.12936}
}

read the original abstract

In clinical operations, teamwork can be the crucial factor that determines the final outcome. Prior studies have shown that sufficient collaboration is the key factor that determines the outcome of an operation. To understand how the team practices teamwork during the operation, we collected CliniDial from simulations of medical operations. CliniDial includes the audio data and its transcriptions, the simulated physiology signals of the patient manikins, and how the team operates from two camera angles. We annotate behavior codes following an existing framework to understand the teamwork process for CliniDial. We pinpoint three main characteristics of our dataset, including its label imbalances, rich and natural interactions, and multiple modalities, and conduct experiments to test existing LLMs' capabilities on handling data with these characteristics. Experimental results show that CliniDial poses significant challenges to the existing models, inviting future effort on developing methods that can deal with real-world clinical data. We open-source the codebase at https://github.com/MichiganNLP/CliniDial

Figures

Figures reproduced from arXiv: 2506.12936 by the authors.

**Figure 2.** Distribution of words uttered (a) and word [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Overall label distribution and role-specific [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Comparison of macro F1 scores (F1 scores [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 6.** Figure 6: Comparison of macro F1 scores (F1 scores [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Additional examples of the labeled dialogue in the simulated operation. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

**Figure 8.** Figure 8: Distributions of words uttered from different [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

**Figure 9.** Figure 9: Word cloud plots for frequent words uttered [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages

[1]

dantrolene

They see a great opportunity to apply these LLMs on behavioral evaluation in the medical domain. They point out that the current evalua- tion practices in medical domains have signifi- cant limitations (Kolbe and Boos, 2019; Klonek et al., 2019; Stevenson et al., 2022), which typ- ically are labor-intensive and prone to personal biases and errors. They ex...

work page 2019
[2]

We note that this is one of the characteristics for CliniDial

They expect a protocol that can take multimodal input into consideration including the team dia- logue, patient vitals, and procedure videos. We note that this is one of the characteristics for CliniDial. They also hope the NLP system could pinpoint specific teamwork deficiencies in the process

work page
[3]

They also point out the related NLP methods that they find useful in their domain. For instance, in- tent classification, dialogue summarization, and multimodal reasoning works from NLP can pro- vide quantifiable insights into teamwork dynam- ics and communication patterns in multimodal clinical data (Zhang et al., 2018; Allen et al., 2021; Lehmann-Willen...

work page 2018
[5]

arXiv preprint arXiv:2410.19168

Mmau: A massive multi-task audio under- standing and reasoning benchmark. arXiv preprint arXiv:2410.19168. Jan B Schmutz, Zhike Lei, and Walter J Eppich. 2021. Reflection on the fly: development of the team reflec- tion behavioral observation (turbo) system for acute care teams. Academic Medicine, 96(9):1337–1345. Jan B Schmutz, Laurenz L Meier, and Tanja Manser

arXiv 2021
[9]

Seek”, “Evalu- ate

Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. Yanxia Zhang, Jeffrey Olenick, Chu-Hsiang Chang, Steve WJ Kozlowski, and Hayley Hung. 2018. The i in team: Mining personal social interaction rou- tine with topic models from long-term team data. In 23rd International Conference on Intelligen...

work page 2021
[2008]

Annals of surgery, 247(4):699–706

Teamwork and error in the operating room: analysis of skills and roles. Annals of surgery, 247(4):699–706. Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. Smote: syn- thetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357. David Chen and William B Dolan. 2011. Collect- ing highly p...

work page 2002
[2016]

In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4631–4640

Movieqa: Understanding stories in movies through question-answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4631–4640. Sallie J Weaver, Michael A Rosen, Deborah DiazGrana- dos, Elizabeth H Lazzara, Rebecca Lyons, Eduardo Salas, Stephen A Knych, Margie McKeever, Lee Adler, Mary Barker, et al. 2010. Does team...

arXiv 2010
[2019]

BMJ open, 9(9):e028280

How effective is teamwork really? the relation- ship between teamwork and performance in health- care teams: a systematic review and meta-analysis. BMJ open, 9(9):e028280. Chris Seiffert, Taghi M Khoshgoftaar, Jason Van Hulse, and Amri Napolitano. 2009. Rusboost: A hy- brid approach to alleviating class imbalance. IEEE transactions on systems, man, and cy...

work page 2009

Show all 12 references

[2020]

In Proceedings of the 28th International Conference on Computational Linguistics, pages 717–729, Barcelona, Spain (On- line)

Summarizing medical conversations via iden- tifying important utterances. In Proceedings of the 28th International Conference on Computational Linguistics, pages 717–729, Barcelona, Spain (On- line). International Committee on Computational Lin- guistics. Irena Spasic, Goran N...

2020
[2021]

In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5062–5074, Online

DialogSum: A real-life scenario dialogue sum- marization dataset. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5062–5074, Online. Association for Computa- tional Linguistics. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutano...

2021
[2023]

Advances in Neural Information Processing Systems, 36:20482–20494

3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36:20482–20494. Hayley Hung, Litian Li, Jord Molhoek, and Jing Zhou
[2024]

In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1–9

The discontent with intent estimation in-the- wild: The case for unrealized intentions. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1–9. Robert Scott Isaak and Marjorie Podraza Stiegler. 2016. Review of crisis resource management (c...

2016 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

dantrolene

They see a great opportunity to apply these LLMs on behavioral evaluation in the medical domain. They point out that the current evalua- tion practices in medical domains have signifi- cant limitations (Kolbe and Boos, 2019; Klonek et al., 2019; Stevenson et al., 2022), which typ- ically are labor-intensive and prone to personal biases and errors. They ex...

work page 2019

[2] [2]

We note that this is one of the characteristics for CliniDial

They expect a protocol that can take multimodal input into consideration including the team dia- logue, patient vitals, and procedure videos. We note that this is one of the characteristics for CliniDial. They also hope the NLP system could pinpoint specific teamwork deficiencies in the process

work page

[3] [3]

They also point out the related NLP methods that they find useful in their domain. For instance, in- tent classification, dialogue summarization, and multimodal reasoning works from NLP can pro- vide quantifiable insights into teamwork dynam- ics and communication patterns in multimodal clinical data (Zhang et al., 2018; Allen et al., 2021; Lehmann-Willen...

work page 2018

[4] [5]

arXiv preprint arXiv:2410.19168

Mmau: A massive multi-task audio under- standing and reasoning benchmark. arXiv preprint arXiv:2410.19168. Jan B Schmutz, Zhike Lei, and Walter J Eppich. 2021. Reflection on the fly: development of the team reflec- tion behavioral observation (turbo) system for acute care teams. Academic Medicine, 96(9):1337–1345. Jan B Schmutz, Laurenz L Meier, and Tanja Manser

arXiv 2021

[5] [9]

Seek”, “Evalu- ate

Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. Yanxia Zhang, Jeffrey Olenick, Chu-Hsiang Chang, Steve WJ Kozlowski, and Hayley Hung. 2018. The i in team: Mining personal social interaction rou- tine with topic models from long-term team data. In 23rd International Conference on Intelligen...

work page 2021

[6] [2008]

Annals of surgery, 247(4):699–706

Teamwork and error in the operating room: analysis of skills and roles. Annals of surgery, 247(4):699–706. Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. Smote: syn- thetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357. David Chen and William B Dolan. 2011. Collect- ing highly p...

work page 2002

[7] [2016]

In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4631–4640

Movieqa: Understanding stories in movies through question-answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4631–4640. Sallie J Weaver, Michael A Rosen, Deborah DiazGrana- dos, Elizabeth H Lazzara, Rebecca Lyons, Eduardo Salas, Stephen A Knych, Margie McKeever, Lee Adler, Mary Barker, et al. 2010. Does team...

arXiv 2010

[8] [2019]

BMJ open, 9(9):e028280

How effective is teamwork really? the relation- ship between teamwork and performance in health- care teams: a systematic review and meta-analysis. BMJ open, 9(9):e028280. Chris Seiffert, Taghi M Khoshgoftaar, Jason Van Hulse, and Amri Napolitano. 2009. Rusboost: A hy- brid approach to alleviating class imbalance. IEEE transactions on systems, man, and cy...

work page 2009