{"id":"f0de25ab-d28a-4cf4-8d47-9afea8017ee1","arxiv_id":"2412.07072","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Stable Mean Teacher with an Error Recovery module and a Difference of Pixels constraint improves semi-supervised video action detection, reaching near fully-supervised accuracy with 10-20% labels.","lead":"This paper presents a semi-supervised video action detection method that uses a teacher model to generate pseudo-labels and two new modules to improve spatial boundaries and temporal consistency. It reports that with only 10-20% of labeled data the method approaches fully supervised performance on two action detection benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 10%/20%-versus-100% claim is not anchored: no fully supervised run on the same pipeline, and Table 1's 'Sup. baseline' row appears trained only on the labeled subset.","rationale":"The reader's weakest_assumption targeted pseudo-label accuracy under an unthresholded EMA teacher. That is a plausible SSL risk, but the manuscript's ablation table (Table 3: Lbase/LEoR/LDoP) and the comparison of EoR against a 2D replacement show internal consistency; the method's improvements over its own base are modest but coherent. The more load-bearing gap, in my reading, is that the central quantitative claim that 10%/20% labels match a 100%-supervised baseline lacks a same-pipeline 100%-supervised reference. The only '100%' numbers in Table 1 belong to other architectures with different backbones and modalities (e.g., BMViT ViT-B at 90.7; ACAR-Net at 84.3), not to the authors' own I3D/VideoCapsuleNet setup. Moreover, the 'Supervised baseline' rows are trained only on the labeled fraction (10%/20%), which makes the claimed 23.5% improvement over 'supervised baseline' an improvement over a much weaker reference than the 100%-supervised setting implied by the abstract. This is a standard evaluation-rigor issue in SSL papers, and it is exactly the kind of gap that a released codebase can settle quickly: rerun the same pipeline with 100% labels and compare. It does not invalidate the SSL gains over the same-fraction supervised baseline, but it does undercut the headline claim as worded, so I keep the conditional verdict rather than accepting outright.","tokens_in":19299,"tokens_out":1825,"duration_ms":16915,"concrete_test":"Train the authors' VideoCapsuleNet-based student model on 100% of per-frame annotations on UCF101-24 and JHMDB21 (same code release, same training schedule except annotation fraction), and report f-mAP@0.5/v-mAP@0.5. If the 100%-supervised run reaches values close to or above the 10%/20% semi-supervised numbers (73.9/76.3 and 69.8/70.7), the headline 'competitive with 100% supervision' claim changes to 'comparable at a specific threshold under a different training regime.' If the 100%-supervised run returns substantially lower numbers, the claim as stated is unsupported by the reported experiments.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's headline claim is 'Using merely 10% and 20% of data, it provides competitive performance compared to the supervised baseline trained on 100% annotations.' Yet no 100%-supervised number appears for the authors' own I3D/VideoCapsuleNet-based pipeline in Table 1 or the supplementary. The comparison points are TACNet, MOC, ACAR-Net, YOWO, TubeR, etc., which use different backbones, input modalities, and training recipes; this is not the same-base fully supervised baseline. The only same-pipeline numbers are the 'Supervised baseline' rows in Tables 1, 5, 6, 7 (e.g., UCF101-24 10%: f-mAP 53.5, v-mAP 49.7). That labeled-subset supervised run is used to claim 'outperforms the supervised baselines by 23.5%,' but it is a supervised model trained on only the 10% labeled subset while the semi-supervised model also sees (and is trained on) that same 10% plus unlabeled data. The full-supervision supervised baseline used to justify '10% vs 100%' is never run, so the 'competitive' wording is not quantitatively anchored. Because the semi-supervised method trains on the same 10% labeled subset plus the unlabeled pool, and the supervised 10% baseline is an order-of-magnitude weaker, the gap may partly reflect an unfair or weak same-label baseline rather than a genuine match to full supervision.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Stable Mean Teacher, an EMA student-teacher framework for semi-supervised video action detection. It introduces two components: an Error Recovery (EoR) module, a 3D U-Net that refines the teacher's localization outputs, and a Difference of Pixels (DoP) constraint that enforces temporal consistency on localization maps. The method is evaluated on UCF101-24, JHMDB21, AVA, and YouTube-VOS, with claims that 10% labels on UCF101-24 and 20% labels on JHMDB21 are competitive with fully supervised training, and that large gains are obtained over same-label supervised baselines.","tokens_in":19601,"tokens_out":7445,"duration_ms":73322,"significance":"If the claims hold, the paper would be a useful step toward label-efficient video action detection. The work has clear strengths: code and models are publicly available; the ablations in Table 3 and supplementary Table 9 isolate the contributions of EoR and DoP; and the evaluation spans four benchmarks with multiple thresholds and a generalization experiment to video object segmentation. The EoR idea of a class-agnostic correction module trained with a stopped gradient is interesting and the overall results are plausible. However, the headline comparison to full supervision is not anchored in the main tables, and the EoR training objective as written does not contain the ground-truth error signal that the text claims. These points must be resolved before the central claims are accepted.","major_comments":[{"comment":"The abstract's claim that 10% and 20% labels give performance competitive with a supervised baseline trained on 100% annotations is not anchored by a same-pipeline fully supervised run in the main text. The rows labeled 'Supervised baseline' in Tables 1, 5, 6, and 7 are trained on the labeled subset (10% or 20%), not on 100%. The supplementary Table 8a does report '100% sup.' numbers for a static/dynamic split, but no aggregate full-supervision number for the exact VideoCapsuleNet/STMT pipeline appears in the main comparison. Please add a same-model, same-recipe 100%-supervised row to the comparison tables, report the precise gaps that support the word 'competitive,' and adjust the abstract or figure accordingly if those gaps are larger than implied.","section":"Table 1; Abstract; Figure 1"},{"comment":"The EoR loss in Eq. (4) is written as LEoR = MSE(MEoR_t(tloc), sloc), which is a teacher-student consistency loss with no ground-truth term. The text states that EoR 'learns from students' mistakes on labeled samples,' but Eq. (4) contains no supervision from labeled annotations. If the supervised loss Ls is also applied to the EoR module's output on labeled samples, as suggested by the sentence 'and student's Error Recovery module predictions,' that term is not written in Eq. (7) or the surrounding text. Please specify the complete EoR objective on labeled samples, or explain how Eq. (4) alone can teach the module to recover true errors rather than simply imitate the teacher. Without this clarification, the 'error recovery' interpretation is not supported by the equations.","section":"Methodology, Error Recovery (EoR), Eq. (4)"},{"comment":"The statements 'outperforms the supervised baselines by 23.5%' and 'competitive performance compared to the supervised baseline trained on 100% annotations' conflate two different baselines. The 23.5% margin is computed against the same-label (10% or 20%) supervised baseline in Table 1, not against a fully supervised model. Please rename the rows (e.g., 'Supervised (same label fraction)') and state explicitly which baseline is used for each quantitative claim, so that the abstract's wording does not mislead readers into thinking the 23.5% gain is measured against full supervision.","section":"Abstract; Comparison with supervised and weakly-supervised"},{"comment":"No error bars or multiple-seed results are reported. Several of the claimed improvements are small, notably the DoP gains of 0.5-1% in Table 3 and the EoR gain of about 1% on UCF101-24 f-mAP@0.5 in the same table. Without variance estimates or a statement of the number of seeds, these differences could be run-to-run noise. Please report mean and standard deviation over at least three seeds for the key comparisons in Tables 1, 3, and 5, or explicitly state that the reported numbers are already averages over multiple runs.","section":"Tables 1-9"}],"minor_comments":[{"comment":"The notation 'M SE' in Eq. (4) should be typeset as a single operator, e.g., \\operatorname{MSE}.","section":"Eq. (4)"},{"comment":"'Jenson-Shennon Divergence' should be 'Jensen-Shannon Divergence.'","section":"Methodology, Learning objectives"},{"comment":"The word 'psuedo-labels' in the abstract should be 'pseudo-labels.'","section":"Abstract"},{"comment":"The column headers K, FPS, A, mAP, and GFLOPs are not all defined in the caption; please define K and A explicitly.","section":"Table 2"},{"comment":"The figure legend appears to include a 'Ours 100%' entry, but Table 1 contains no 100%-supervised row for the authors' pipeline; please align the figure with the new baseline row requested in the major comments.","section":"Figure 1"},{"comment":"The comparison with fully supervised methods that use different backbones and modalities (e.g., YOWO with ResNext-101, TACNet with optical flow) should be caveated more explicitly; the wording 'surpasses all the 2D-based approaches' could be misread as a same-architecture comparison.","section":"Comparison with supervised and weakly-supervised"},{"comment":"In the VOS results, the sentence 'outperforms the supervised baseline by an absolute margin of 31%' should identify which row is the 'supervised baseline' (it appears to be Xu 10%), and clarify that the comparison is not against the Xu 100% row.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a vision/AAAI-style venue, and the public code and model release is a clear plus. The main risks are that the headline full-supervision comparison is absent from the main tables and that the EoR loss is under-specified; both are fixable within the scope of a revision. I would be willing to accept a revised version that adds the missing same-pipeline 100%-supervised run, clarifies the EoR training signal, and reports variance for the key numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a legitimate extension of Mean Teacher to video action detection, and the ablations support that the two new modules help. The main problem is the headline claim. The paper says 10% labels on UCF101-24 and 20% on JHMDB21 give results competitive with a fully supervised baseline, but no same-base 100%-annotation run is reported. The fully supervised numbers come from other architectures with different training recipes, while the paper's own 'Supervised baseline' rows are trained on the labeled subset only. That is not an apples-to-apples comparison, and it weakens the central narrative. Credit where it is due: the DoP temporal-difference constraint is genuinely simple and well motivated, and Table 3 shows it helps, especially on JHMDB. The EoR module also gives a consistent boost. The VOS experiments are a nice extra. The writing is clear and the empirical work is mostly coherent. The soft spots are real but not fatal. First, EoR is described as 'learning from the student's mistakes,' but Eq. 4 is a self-consistency loss between the teacher-refined pseudo-label and the student prediction, with no ground-truth error signal. The module can still help, but the mechanism is not what the narrative claims. Second, there are no error bars, which is common in this literature but worth noting. Third, and most importantly, the abstract's 'competitive with 100%' claim needs a 100%-annotated run of their own I3D/VideoCapsuleNet pipeline. The comparison against other fully supervised methods is informative but does not support the specific statement in the abstract. The stress-test note about the missing 100% baseline is accurate. The reader should not be misled: the '23.5% over supervised baseline' margin is against the weak labeled-subset baseline, which is a reasonable way to show the value of unlabeled data, but it is not the same as matching full supervision. I would send this to review. The method is sensible, the experiments are broad, and the code and models are promised. The authors should be asked to add a same-base fully supervised comparison and to reframe the EoR explanation before publication. The paper is useful for the video understanding community and deserves referee time.","headline":"Solid empirical extension of Mean Teacher to spatio-temporal action detection with two useful modules, but the headline '10%/20% vs 100%' comparison is not anchored by a same-base fully supervised run.","tokens_in":704,"tokens_out":1714,"would_cite":true,"duration_ms":58566,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Stable Mean Teacher claims that semi-supervised video action detection reaches near fully-supervised quality with only 10–20% of per-frame labels.","keywords":["semi-supervised learning","video action detection","mean teacher","Error Recovery module","Difference of Pixels","temporal consistency","pseudo-labels","label efficiency"],"falsifier":"On a held-out subset of UCF101-24 at the 10% labeled setting, measure the IoU between teacher pseudo-labels and ground-truth boxes on unlabeled videos before and after EoR refinement; if the refined pseudo-labels are not systematically closer to ground truth, the claimed error-recovery mechanism is not doing the work attributed to it.","tokens_in":19056,"feed_emoji":"🎬","tokens_out":14106,"duration_ms":116840,"temperature":0.7,"pith_summary":"Stable Mean Teacher claims that semi-supervised video action detection can be made much more label-efficient with a student–teacher scheme that upgrades pseudo-labels in two concrete ways: a class-agnostic Error Recovery module that refines the teacher's spatial localization maps, and a Difference of Pixels constraint that keeps those maps temporally coherent. On UCF101-24, training with 10% of the frame-level annotations reaches 73.9 f-mAP@0.5 and 76.3 v-mAP@0.5, which the paper reports as competitive with fully supervised training at 100% labels and about 20 and 27 points above a supervised baseline trained on the same 10% subset. On JHMDB21, 20% labels give 69.8 f-mAP@0.5 and 70.7 v-mAP@0.5, also reported as competitive with full supervision. Reported margins over the same-percentage supervised baseline average 23.5% on UCF101-24 and 16% on JHMDB21. If these numbers hold, per-frame annotation cost for video action detection could drop by an order of magnitude while keeping most of the quality.","feed_headline":"10% of labels nearly matches full-supervision action detection","feed_subtitle":"Video action detection near full quality using only 10-20% of per-frame labels.","key_machinery":"Two pseudo-label refinements carry the argument. Error Recovery (EoR) is a class-agnostic 3D U-Net that sees only the teacher's localization map, not the input video or class labels, so it learns a boundary-correcting transformation that can generalize to unlabeled clips; the teacher-side EoR weights follow an EMA of the student-side EoR, and gradient stopping keeps the base model unchanged. Difference of Pixels (DoP) computes $\\phi(x^f_{loc}) = x^{f+1}_{loc} - x^f_{loc}$ for each pair of consecutive frames and enforces MSE consistency between teacher and student on these temporal differences, pushing predictions to move coherently across time. The full objective (Eq. 7) is $L = L_s + \\lambda (L_{base} + L_{EoR} + L_{DoP})$, with $\\lambda = 0.1$ and EMA decay $\\beta = 0.99$.","core_discovery":"The paper's central claim is that a Mean Teacher framework adapted to spatiotemporal detection, called Stable Mean Teacher, can generate pseudo-labels good enough that a small labeled subset yields near-fully-supervised action detection quality. The teacher is an exponential-moving-average of the student and sees weakly augmented clips, while the student sees strongly augmented clips; consistency between them is the unsupervised training signal. The Error Recovery (EoR) module, a class-agnostic 3D U-Net with roughly 1.1M parameters, takes the teacher's raw localization map and produces a refined map; its loss (Eq. 4) is the MSE between the refined teacher map and the student's localization output, with gradients intentionally stopped from updating the base model. The Difference of Pixels (DoP) constraint (Eqs. 5–6) enforces agreement on the temporal difference of consecutive localization maps, making pseudo-labels temporally coherent. The paper reports that on UCF101-24, JHMDB21, and AVA this framework outperforms the same-scale supervised baseline by average margins of 23.5%, 16%, and 3.3%, respectively, and that at 10–20% annotations it approaches fully supervised performance.","pith_inferences":["A natural next experiment the paper does not run is to add confidence-based pseudo-label selection on top of EoR/DoP; if teacher confidence correlates with correctness, filtering could push the low-label regime even higher.","The large gains on dynamic scenes (39.4% at v-mAP@0.5 on JHMDB21 dynamic classes) suggest the DoP constraint is the main driver where background and actor motion are strong; this could be tested by ablating DoP separately on static and dynamic class splits.","Because the EoR module is class-agnostic and the teacher-student recipe is detector-agnostic, the same pseudo-label refinement may transfer to other per-frame annotation-heavy video tasks, such as video instance segmentation or multi-person pose tracking; the YouTube-VOS result is preliminary but consistent with that.","The method's gains grow as labels get sparser (on UCF101-24 the relative gain at 5% labels is roughly double the gain at 10%), which suggests the approach is most valuable precisely when labels are hardest to obtain."],"forward_implications":["With 10% of UCF101-24 labels, Stable Mean Teacher reaches f-mAP@0.5 of 73.9 and v-mAP@0.5 of 76.3, outperforming the 10%-supervised baseline by about 20.4 and 26.6 points.","With 20% of JHMDB21 labels, it reaches f-mAP@0.5 of 69.8 and v-mAP@0.5 of 70.7, outperforming the 20%-supervised baseline by about 14.1 and 18.3 points.","It beats the prior semi-supervised video action detection baseline by 8.7 f-mAP@0.5 and 9.6 v-mAP@0.5 on UCF101-24, and by 5.4 and 7.2 on JHMDB21, the latter with 10% less labeled data.","On AVA, the method improves a real-time detector's mAP by 3.3 points using only 10% labels, indicating it scales to a large dataset.","On YouTube-VOS, 10% labels give an average Jaccard of 41.3, above the tested semi-supervised baseline and close to the fully supervised 47.9, evidence that the approach transfers to video object segmentation."],"supporting_citations":[{"why":"supplies the student-teacher EMA consistency scheme that Stable Mean Teacher adapts to spatio-temporal detection.","marker":"Mean Teacher (2017)"},{"why":"is the prior end-to-end semi-supervised video action detection approach used as the main comparison baseline.","marker":"Kumar and Rawat (2022)"},{"why":"is the base action detection architecture serving as both student and teacher.","marker":"VideoCapsuleNet (2018)"},{"why":"provides the U-Net architecture that the Error Recovery module extends to 3D.","marker":"UNet (2015)"},{"why":"is the primary benchmark supporting the 10%-labels result.","marker":"UCF101-24 (2012)"},{"why":"is the benchmark supporting the 20%-labels result.","marker":"JHMDB21 (2013)"},{"why":"is the large-scale benchmark used to test scalability.","marker":"AVA (2018)"},{"why":"is the benchmark used to show generalization to video object segmentation.","marker":"YouTube-VOS (2018c)"}],"fun_headline_variants":["10% labels, near-full action detection via stable teacher","Stable Mean Teacher: video action detection from sparse labels","Semi-supervised video detection: 10% labels rival 100%","Sparse-label video action detection gains 23% over supervised"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the teacher's pseudo-labels, and the EoR module's refinements of them, are accurate enough to serve as training targets: the method applies no confidence threshold or noise filtering, and the EoR objective in Eq. 4 is a self-consistency loss with no ground-truth error signal, so confident systematic teacher mistakes could be reinforced instead of corrected.","fun_headline_variants_meta":{"raw":{"variants":["10% labels, near-full action detection via stable teacher","Stable Mean Teacher: video action detection from sparse labels","Semi-supervised video detection: 10% labels rival 100%","Sparse-label video action detection gains 23% over supervised"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000552,"raw_usage":{"total_tokens":2696,"prompt_tokens":1076,"completion_tokens":1620,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":692,"completion_tokens_details":{"reasoning_tokens":1548}},"tokens_in":692,"tokens_out":1620,"duration_ms":11965,"temperature":1.0,"reasoning_tokens":1548,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:10:49.871086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out subset of UCF101-24 at the 10% labeled setting, measure the IoU between teacher pseudo-labels and ground-truth boxes on unlabeled videos before and after EoR refinement; if the refined pseudo-labels are not systematically closer to ground truth, the claimed error-recovery mechanism is not doing the work attributed to it.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"is the prior end-to-end semi-supervised video action detection approach used as the main comparison baseline."}],"review_version":1}