{"id":"4422585f-bffe-430e-b27f-af4e20b7f4f5","arxiv_id":"2502.08299","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new Team-OR dataset with 55 annotated team briefings in 105 hours of real OR video, plus a detection model that outperforms baseline temporal action detectors.","lead":"Researchers built a new video dataset from 37 real laparoscopic surgeries, showing when operating room teams pause for structured safety briefings called Time-out and StOP?. The dataset and a detection model are a first step toward automatically spotting team communication in the operating room.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'outperforms' claim rests on 9 StOP? test positives with no confidence intervals; a single detection reordering could erase the reported margin.","rationale":"The dataset contribution (Team-OR) is credible and well-motivated: it is a new resource of real surgeries with temporal annotations of two communication protocols, and the annotation process is described. The primary weakness is in the empirical demonstration of the detection method. Table 2 reports AP on a test set of 18 videos with only 13 Time-out and 9 StOP? positive instances. With a binary detection task, AP@0.5 on 9 positives has a standard error that can easily exceed the 8-point gap between the proposed method and TemporalMaxer. The paper provides no confidence intervals, no multiple-seed experiments, and no statistical test; the conclusion 'outperforms' is therefore not justified by the reported evidence. The reader identified the single-camera-view assumption as the weakest assumption; while that is also a real reproducibility gap, the statistical fragility is more load-bearing because it directly undermines the comparative claim regardless of view selection. I recommend retaining the CONDITIONAL verdict: the dataset is valuable, but the detection claim requires additional statistical support and specification of the input view.","tokens_in":11227,"tokens_out":8201,"duration_ms":84021,"concrete_test":"Bootstrap-resample the 9 StOP? test positives 1000 times (or perform leave-one-video-out cross-validation over the test videos) and compute 95% confidence intervals for AP@0.5 for the proposed method and for TemporalMaxer; if the intervals overlap, the reported 8-point advantage is not significant.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the proposed detector 'outperforms existing temporal action detection approaches' (Abstract; Section 4.3) rests on AP scores in Table 2 computed from a test split with 13 Time-out and only 9 StOP? positive instances (Table 1). No confidence intervals, multiple-seed runs, or significance tests are reported. For StOP?, the margin over TemporalMaxer is 8.14 AP@0.5 (20.52 vs 12.38); with 9 positives, a single reordering of detections can shift AP by more than that amount. Additionally, the comparison is not feature-matched: the proposed method uses skeleton features, while the TAD baselines are evaluated with VideoMAEv2 features only (Section 4.4), so the improvement may stem from the extra modality rather than from the proposed pooling architecture. The claimed outperformance is therefore not empirically established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Team-OR, a multi-view dataset of 37 real laparoscopic surgeries (43 videos, 105 hours) with temporal annotations of 33 Time-out and 22 StOP? team communication briefings. It proposes a group activity detection approach that concatenates VideoMAEv2 scene features with STGCN++ skeleton features and feeds them into a lightweight 1D convolutional network with max- and average-pooling pyramids, followed by classification and regression heads. On a 60/40 video-level split, the method reports AP@0.5 of 99.45 for Time-out and 20.52 for StOP?, outperforming the TAD baselines in Table 2; the paper also reports ablations and a real-time FPS figure of 33.03.","tokens_in":11486,"tokens_out":5659,"duration_ms":57354,"significance":"The dataset contribution is valuable and timely: Team-OR appears to be the first OR dataset of real surgeries with temporal annotations of clinically validated team communication protocols, and the authors commit to releasing code. The multi-view setup, the connection to the StOP? II trial, and the light-weight pooling design are genuine strengths. However, the central empirical claim of outperforming state-of-the-art TAD approaches is not yet established: the test set has only 9 StOP? positives, no uncertainty quantification is reported, and the comparison is not feature-matched, since the proposed model uses an extra skeleton modality. The low StOP? AP also shows that the task remains far from solved, so the paper's main value at this stage is the dataset and the task formulation rather than the detection accuracy.","major_comments":[{"comment":"The central performance claim rests on a test split with 13 Time-out and only 9 StOP? positive instances, and the paper reports a single run with no confidence intervals, multiple seeds, or significance tests. For StOP?, the AP@0.5 margin over TemporalMaxer is 8.14 points (20.52 vs 12.38); with 9 positives, a small change in detection ranking can plausibly shift AP by more than this margin. Please report per-seed results, confidence intervals (e.g., bootstrap over videos), or a significance test, and temper the 'outperforms' claim accordingly.","section":"§4.1, Tables 1 and 2"},{"comment":"The comparison with TAD baselines is not feature-matched. The proposed method uses F = (Fg, Fl), i.e., VideoMAEv2 plus STGCN++ skeleton features, while the baselines in Table 2 are evaluated with VideoMAEv2 features only. Table 3 shows that adding skeleton features is responsible for a large part of the improvement: between rows 2 and 3, Time-out AP@0.5 rises from 79.06 to 99.45 and StOP? AP@0.5 from 12.77 to 20.52. Thus the reported advantage over TemporalMaxer and TriDet may reflect the extra skeleton modality rather than the proposed pooling architecture. Please evaluate the baselines with the same concatenated features, or ablate the proposed model without skeleton features, and compare all methods on equal footing.","section":"§4.4, Tables 2 and 3"},{"comment":"The dataset consists of 43 videos from 37 surgeries, and the train/test split is performed at the video level. Because the authors state that some operations had to be paused and therefore produced multiple videos, two videos from the same operation can in principle appear in both train and test, creating a risk of information leakage. Please clarify whether any operation contributes videos to both splits and, if so, re-split by operation or explicitly verify that no surgery spans the train/test boundary.","section":"§3.1, Table 1"},{"comment":"The method uses 'one camera view as the input', but the paper never specifies which of the three ceiling views or the laparoscopic view is used, how that view is selected, or whether that view always contains all team members participating in the briefing. Since both protocols require all team members to participate, a view that misses a participant cannot support the claimed detection of the group activity. Please specify the view-selection procedure and discuss its coverage relative to the team layout in the OR.","section":"§3.3"}],"minor_comments":[{"comment":"The FPR-95 experiment used to motivate the hierarchical analysis is described qualitatively, but no quantitative results are shown; please provide the actual FPR-95 values or state that they are omitted for space and available upon request.","section":"§3.2"},{"comment":"The annotation section mentions 'boundary errors of less than five seconds', but no inter-annotator agreement measure is reported; if available, please add one to support the reliability of the annotations.","section":"§3.1"},{"comment":"The implementation details include the feature stride and clip settings, but not the temporal resolution of the final proposals or the Soft-NMS threshold used at inference; please include these for reproducibility.","section":"§4.2"},{"comment":"The phrase 'our approach performs the best on both activities, which establishes a solid baseline' is stronger than the results justify, given the StOP? AP@0.5 of 20.52; please rephrase to acknowledge the limited test size and the remaining difficulty of the task.","section":"§4.3"},{"comment":"The ablation study would be more informative if each configuration were run with multiple seeds and reported with standard deviations, since the differences between some rows are small and may not be stable.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The dataset and task formulation are a good fit for IPCAI and likely useful to the community. My main concern is not scope but the strength of the empirical claim: with 9 StOP? test positives, no uncertainty estimates, a feature-mismatched baseline comparison, and a video-level split that may leak across videos of the same operation, the 'outperforms existing approaches' statement is not yet supported. These issues are fixable with additional experiments and reporting, so I would not recommend rejection. I did not identify citation or authorship concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: Team-OR is a genuine first — temporal annotations of Time-out and StOP? briefings in real multi-view OR video, 105 hours, 37 surgeries. That part of the paper is solid and is the reason to read it. The detection story is a lot weaker than the abstract suggests.\n\nWhat's actually new: no prior OR dataset has temporal labels for these team-communication protocols. The annotation protocol is described carefully (MOSaiC, audio+visual, <5s boundary error), and the dataset covers a sensible spread of abdominal procedures. The hierarchical analysis in Sec. 3.2, where they train a skeleton classifier and inspect hard negatives, is a nice, honest way to motivate the feature design. Credit where due: they state limitations — single site, small sample, hard StOP? detection — in Sec. 4.5.\n\nThe soft spots: the headline comparison in Table 2 is not feature-matched. Their method adds STGCN++ skeleton features; the TAD baselines get VideoMAEv2 features only. The ablation in Table 3 shows that adding the skeleton branch is what lifts StOP? AP@0.5 from 12.77 to 20.52; the pooling change alone is worth a few points. So 'outperforms existing approaches' is mostly 'adding an extra modality helps,' which is a weaker claim than the text implies. Also, the StOP? numbers come from 9 positive test instances, no error bars, no multiple seeds. A single detection reordering can swing AP by more than the reported margin over TemporalMaxer (8.14 AP@0.5). That's not a fatal flaw — the dataset is the product — but the method comparison should be re-run feature-matched, or the claim softened. The single-camera-view choice (Sec. 3.3) is never specified: which of the three ceiling views, and does it always contain the whole team? If a briefing happens partly off-camera in the chosen view, the group activity is undetectable. That needs a sentence of justification. Also, the dataset itself isn't released, only code, so the 'first dataset' is currently a claim with no public artifact.\n\nBottom line: read it for the dataset and the task formulation. The detection results need statistical support and a feature-matched baseline before I'd trust the modeling conclusions. Worth a serious referee — conditional accept, with the comparison and camera-view issues on the revision list.","headline":"Team-OR is a real dataset contribution; the detection 'outperforms' claim is not feature-matched and rests on 9 StOP? test positives.","tokens_in":11957,"tokens_out":1874,"would_cite":true,"duration_ms":19282,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces Team-OR, the first real-surgery dataset with temporal annotations of OR team briefings, and reports that its group activity detection approach outperforms existing temporal action detectors on both Time-out and StOP?","keywords":["temporal action detection","group activity detection","operating room video analysis","surgical team communication","Time-out protocol","StOP? protocol","skeleton-based action recognition","surgical video dataset"],"falsifier":"Run the released pipeline separately on each of the four synchronized camera views for the same operations and compare the predicted start and end times against the multi-view ground-truth annotations; if the detections, especially for StOP?, change materially with the chosen view, the reported scores depend on an unstated camera-selection choice rather than on group activity detection as such.","tokens_in":11001,"feed_emoji":"🏥","tokens_out":10258,"duration_ms":92651,"temperature":0.7,"pith_summary":"Good surgical outcomes depend on how the whole operating-room team communicates, not only on the surgeon's technical skill. This paper takes the first step toward measuring those briefings automatically by introducing Team-OR, a dataset of more than one hundred hours of real laparoscopic surgeries with start and end times marked for two standardized team communication protocols: the pre-incision Time-out and the intraoperative StOP? briefing. The paper also proposes a group activity detection method that combines scene-level video features with skeleton-based individual action features and a lightweight pooling network. On this dataset the method reports the best results among existing temporal action detectors, reaching Time-out AP@0.5 of 99.45 and StOP? AP@0.5 of 20.52. The practical motivation is patient safety: automatically knowing when and whether these briefings happen could support training, compliance monitoring, and context-aware OR assistance systems.","feed_headline":"Detector scores 99% on Time-out, 21% on StOP? in OR video","feed_subtitle":"A new multi-view dataset lets computers localize when the whole surgical team pauses for a Time-out or StOP? briefing.","key_machinery":"The central object is the group activity detection task: localizing the start and end times of a briefing that involves the whole OR team in an untrimmed surgery video. The pipeline extracts two complementary feature streams from a single camera view: global temporal scene features from the pretrained video model VideoMAEv2, and per-person skeleton action features obtained by detecting and tracking people, estimating their poses, and encoding the skeleton sequences with STGCN++. These are concatenated per clip and fed into a lightweight network whose two-branch max-pooling and average-pooling feature pyramid merges multi-level temporal information; a classification head predicts the activity and a regression head predicts its start and end, with Soft-NMS removing duplicate proposals and focal plus temporal DIoU losses used during training. The pooling-only design is what lets it generalize from very few positive samples.","core_discovery":"The central claim is that group communication protocols in the OR can be treated as a temporal action detection problem, provided the detector uses both global scene context and individual skeleton action features. The paper argues that existing OR datasets, which support pose estimation or scene graph generation and mostly come from simulated or short recordings, cannot support this task, and that on the new Team-OR dataset the proposed approach outperforms ActionFormer, TriDet, and TemporalMaxer on both activities. The StOP? protocol is markedly harder than Time-out, with 20.52 versus 99.45 AP@0.5, which the paper attributes to subtle visual differences between briefing and normal taskwork, imperfect protocol compliance, and very few positive examples.","pith_inferences":["Going beyond the paper: whether the single camera view contains every briefing participant is the first thing to test, since the paper does not specify which view is used or how it was selected.","Going beyond the paper: running the released system separately on each of the four synchronized views would show how much of the reported performance depends on an unstated recording-geometry choice.","Going beyond the paper: the deliberate exclusion of audio leaves a strong, privacy-sensitive cue unused; a multimodal version with speech activity detection might substantially improve the StOP? results.","Going beyond the paper: the same sparse-event, pooling-based detector could be applied to other long videos with rare and visually subtle group activities, such as emergency team training or classroom collaboration."],"forward_implications":["The same task definition, dataset split, and evaluation protocol can serve as a baseline for future work on group activity detection in the OR.","The reported end-to-end throughput of about 33 frames per second suggests that such detection could run in live intra-operative support systems.","The large gap between Time-out and StOP? results shows that subtle, poorly compliant group activities remain a hard target even with skeleton and scene features combined.","The average protocol durations reported in the dataset, about 90 seconds for Time-out and 63 seconds for StOP?, give concrete targets for detector granularity and for scheduling brief monitoring windows.","Because the two protocols follow standardized instructions, the annotations and detection approach could be transferred to other hospitals, although the paper only evaluates one clinical site."],"supporting_citations":[{"why":"Supplies the pretrained VideoMAEv2 scene-feature extractor that provides the global context half of the input representation.","marker":"[42]"},{"why":"Supplies the STGCN++ skeleton action extractor that provides the local individual-action half of the input representation.","marker":"[7]"},{"why":"Baseline that motivates the max-pooling temporal pyramid, which the paper extends with an average-pooling branch.","marker":"[38]"},{"why":"TriDet is the strongest baseline and the backbone used in the ablations, providing the comparison point for the pooling design.","marker":"[33]"},{"why":"ActionFormer is the full-self-attention baseline that the paper compares against to argue that lighter pooling-based designs avoid overfitting.","marker":"[44]"},{"why":"MVOR is the previous real-surgery multi-view OR dataset, limited to images and pose, and thus marks the gap Team-OR fills.","marker":"[36]"},{"why":"4D-OR is the prior OR scene-graph dataset whose simulated surgeries lack real team interactions, motivating the collection of Team-OR.","marker":"[26]"},{"why":"The videos are a subsample of operations from the StOP? II trial, which defines the StOP?-protocol setting and the data collection context.","marker":"[14]"},{"why":"The WHO surgical safety checklist establishes the Time-out as a standard safety briefing, motivating the detection task.","marker":"[12]"},{"why":"The structured intraoperative briefing study provides evidence that StOP?-style briefings improve patient outcomes, motivating the annotations.","marker":"[39]"}],"fun_headline_variants":["AI spots OR team briefings in multi-view video","New dataset + model find Time-out and StOP? in OR","First OR group-activity benchmark for team communication","Scene context + actions: detecting OR team protocols","Model tops action detectors on Team-OR protocols"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a single camera view is guaranteed to show all participating team members during every briefing, but the paper does not say which of the four recorded views is used or how it was chosen.","fun_headline_variants_meta":{"raw":{"variants":["AI spots OR team briefings in multi-view video","New dataset + model find Time-out and StOP? in OR","First OR group-activity benchmark for team communication","Scene context + actions: detecting OR team protocols","Model tops action detectors on Team-OR protocols"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000226,"raw_usage":{"total_tokens":1490,"prompt_tokens":986,"completion_tokens":504,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":428}},"tokens_in":602,"tokens_out":504,"duration_ms":5864,"temperature":1.0,"reasoning_tokens":428,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T05:38:43.825536+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released pipeline separately on each of the four synchronized camera views for the same operations and compare the predicted start and end times against the multi-view ground-truth annotations; if the detections, especially for StOP?, change materially with the chosen view, the reported scores depend on an unstated camera-selection choice rather than on group activity detection as such.","supporting_citations":[{"cited_title":"Videomae v2: Scaling video masked autoen- coders with dual masking, in: Proceedings of the IEEE /CVF Confer- ence on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained VideoMAEv2 scene-feature extractor that provides the global context half of the input representation."},{"cited_title":"Pyskl: Towards good practices for skeleton action recognition, in: Proceedings of the 30th ACM International Conference on Multimedia, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the STGCN++ skeleton action extractor that provides the local individual-action half of the input representation."},{"cited_title":"4d-or: Semantic scene graphs for or domain model- ing, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer","cited_arxiv_id":null,"evidence_quote":"4D-OR is the prior OR scene-graph dataset whose simulated surgeries lack real team interactions, motivating the collection of Team-OR."},{"cited_title":"Stop? ii trial: cluster randomized clinical trial to test the implementation of a toolbox for structured communication in the operating room—study protocol","cited_arxiv_id":null,"evidence_quote":"The videos are a subsample of operations from the StOP? II trial, which defines the StOP?-protocol setting and the data collection context."},{"cited_title":"A surgical safety checklist to reduce morbidity and mor- tality in a global population","cited_arxiv_id":null,"evidence_quote":"The WHO surgical safety checklist establishes the Time-out as a standard safety briefing, motivating the detection task."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The structured intraoperative briefing study provides evidence that StOP?-style briefings improve patient outcomes, motivating the annotations."}],"review_version":1}