{"id":"005f5714-5580-4843-bf2e-79c0a9adf604","arxiv_id":"2506.13971","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Modality-fused co-training predicts low-fluidity and low-enjoyment moments in videoconference sessions nearly as well as a fully supervised model while using only 8% of the labels.","lead":"Semi-supervised learning on audio, face, and text features can predict awkward or unenjoyable moments in group video calls using only a fraction of the labels a fully supervised model needs. In tests, a model trained on 8% of the labeled clips matched about 96% of the supervised model's full-data performance, suggesting cheaper annotation for videoconference experience analysis.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4.2 describes Optuna tuning without a validation split; if the search objective used the same folds as the reported test results, the SSL label-efficiency gains may be inflated.","rationale":"The paper is an empirical, application-level contribution: SSL with fused features is proposed as a way to reduce annotation cost for detecting negative moments in videoconferences. The central assertion is comparative—SSL beats SL by a stated margin and reaches near-supervised performance with 8% labels. I read the evaluation description carefully. Section 4.1 describes the group-stratified folds and the 11,475 combinations; Section 4.2 describes Optuna tuning but gives no indication of how the search objective relates to these folds. Without an explicit statement that hyperparameters were selected on training folds only, the most parsimonious reading is that the protocol is underspecified, not necessarily that it is wrong. However, the underspecification is load-bearing because a test-set leak of even a modest number of Optuna trials can produce exactly the kind of 1–4% improvements reported. I considered alternative concerns—class imbalance making F1 unstable, annotation reliability, the targeted-clip sampling scope—but each would weaken confidence in the generalizability of the result without directly threatening the comparison that the headline depends on. The HP-evaluation issue is the one that, if confirmed, would eliminate the claimed advantage entirely. I therefore agree with the reader's weakest assumption. My recommended verdict is unchanged from the reader's CONDITIONAL: the paper is plausible and worth publishing after the authors clarify or rerun the nested protocol.","tokens_in":7855,"tokens_out":5160,"duration_ms":49919,"concrete_test":"Run a strictly nested evaluation: for each outer split (2 test folds, 8 training folds), use only the 8 training folds to select Optuna hyperparameters via an inner 2- or 3-fold split of the labeled data; freeze the selected hyperparameters and then evaluate once on the outer test folds. Compare SSL (modality-fused co-training) with its SL counterpart at 8%, 16%, and 32% labeled data. If the SSL-minus-SL AUC/F1 gap at any label budget falls below the reported 1–4%, or if the 96% matching figure from the abstract (95% in Figure 3) cannot be reproduced, the label-efficiency claim is not established. Also report the Optuna objective function to confirm it never accesses outer test folds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Abstract; Section 5) is that modality-fused co-training SSL outperforms SL by up to 4% with the same label budget and matches 96% of full-supervision performance with 8% labels. This claim stands or falls on whether the Optuna hyperparameter search in Section 4.2 is properly separated from test evaluation. The text lists the search space and the TPE sampler but never states the objective data, an inner validation split, or nested cross-validation. In a protocol with 11,475 train/test combinations, if the same holdout folds were used to select hyperparameters—or if tuning was done on pooled data containing held-out sessions—the 1–4% SSL-over-SL advantage could be selection bias rather than a genuine SSL benefit. This is the single most load-bearing concern because the contribution is specifically the magnitude of the label-efficiency gain; a clean re-run that shows no leak would support the claim, while a leak would invalidate it. The 96%/95% discrepancy and missing significance tests are secondary, but they reinforce the need to audit the evaluation protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semi-supervised learning (SSL) framework for predicting subjective moments of low fluidity or low enjoyment in videoconference sessions. The authors extract 7-second targeted clips around turn-taking disruptions (gaps and overlaps) and non-targeted clips from the remainder of each session, annotate the targeted clips with multiple crowd annotators, and binarize the ratings using a threshold from prior work. They then compare three SSL wrappers (self-training, modality-split co-training, and modality-fused co-training) against a supervised logistic regression baseline at varying labeled-data ratios, using audio (VGGish), facial action units (OpenFace), and text (Sentence-BERT) features. The core claim is that modality-fused co-training SSL outperforms supervised learning by up to 4% at the same label budget, reaches an ROC-AUC of 0.9 and F1 of 0.6, and with only 8% labeled data matches 96% of the supervised full-data performance, thereby providing an annotation-efficient method for modeling videoconference experience.","tokens_in":8052,"tokens_out":4655,"duration_ms":45283,"significance":"If the evaluation protocol is sound, this is a practically valuable contribution: it directly attacks the annotation bottleneck for a neglected but important social-computing problem, uses naturalistic multiparty videoconference data, and includes an informative ablation showing that audio and face features dominate over text. The paper also gives credit for comparing SSL against an SL counterpart at matched label budgets, which is the right way to isolate the benefit of unlabeled data. However, the central quantitative claims hinge on the integrity of the hyperparameter search and on the exact composition of the unlabeled pool; both are under-specified in the manuscript, so the current numerical findings are not fully supported as written.","major_comments":[{"comment":"The manuscript states that the Optuna framework was used for hyperparameter tuning with a tree-structured parzen estimator, but it never describes the objective data, the number of trials, or, critically, any validation split. If the search was evaluated on the same holdout folds that later produced the reported test scores, or on pooled data that includes held-out sessions, the claimed 1-4% SSL-over-SL advantage could be selection bias rather than a genuine SSL benefit. This is load-bearing for the paper's central label-efficiency claim, so please specify the exact protocol (e.g., nested cross-validation or a fixed validation split inside each training fold) and, if no such protocol was used, re-run the comparison under a clean evaluation scheme.","section":"Section 4.2"},{"comment":"Non-targeted clips are extracted from all sessions and are never assigned to the 10 folds; the text indicates they are always included as unlabeled data. As a result, for a test fold containing sessions S, the unlabeled pool includes non-targeted clips from those very same sessions S. This is a transductive setup in which the model sees the audiovisual and textual features of the target sessions (without labels) before predicting their targeted clips, which can inflate performance and materially weakens the claim that the model generalizes to new sessions with different participants. Please either exclude non-targeted clips from held-out sessions or report a purely inductive variant with unlabeled data restricted to training sessions, and clearly state which setting is shown in Figure 3.","section":"Section 4.1"},{"comment":"There is a numerical inconsistency: the Abstract reports that the best SSL model with 8% labeled data matched 96% of the SL model's full-data performance, while the Figure 3 caption says 95%, and the Results text specifies only that 'the ROC-AUC ... can achieve 96% performance ... on predicting Enjoyment.' This discrepancy must be reconciled, and the metric to which the percentage applies must be stated precisely. More importantly, the paper reports no significance tests or confidence intervals for the 'up to 4%' SSL-over-SL advantage; given the severe class imbalance (92 clips low on both scales versus 2,731 high on both) and the limited number of sessions per fold (about 3), the observed differences may be within sampling variability. Please provide paired statistical comparisons across folds, such as Wilcoxon signed-rank tests or bootstrap confidence intervals, for each labeled-data ratio and metric.","section":"Abstract, Section 5, and Figure 3"}],"minor_comments":[{"comment":"The phrase 'such as overlapping speech or unusually long gaps—instances' contains a stray em dash and an unnecessary word; it should read 'such as overlapping speech or unusually long gaps.'","section":"Section 3.1.2"},{"comment":"The hyperparameter description 'PCA explained variance: 0– or 20–100%' is likely a typo; please clarify whether the search space is 0-100%, or 0% and 20-100% as two distinct options.","section":"Section 4.2"},{"comment":"The error bars are described as standard error, but the effective sample size (number of folds or combinations) for each labeled-data percentage is not reported; please state n for each point so readers can interpret the error bars.","section":"Figures 3 and 4"},{"comment":"There is a typo in 'Y AMNet' (likely 'YAMNet').","section":"Section 3.3.1"},{"comment":"The phrase 'the phenomena is' should be 'the phenomena are' or 'this phenomenon is'; also, the reference to [11] for the binarization threshold is appropriate but should be accompanied by an explicit reproducibility note that the threshold is a free parameter from prior work.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The Optuna validation concern raised in the stress test is real and is compounded by a second, independent leakage risk: non-targeted clips from held-out sessions are present in the unlabeled pool. Both issues affect the central quantitative claim and require a clean rerun or a convincing protocol description. The paper is otherwise of interest to the speech/affective computing community, and the authors should be encouraged to resubmit after addressing these evaluation concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe interesting contribution here is empirical: the authors apply standard SSL wrappers (self-training, co-training) to a new and genuinely underexplored task—modeling subjective negative moments in videoconference group conversations—and they report a clean comparison of SL vs SSL across label budgets. The best result, modality-fused co-training slightly beating supervised baselines and reaching roughly 95–96% of full-supervision performance with 8% labels, would be useful if it holds. The ablation is also informative: audio and face features carry most of the signal, text less so, which matches the intuition that paralinguistic turn-taking dynamics matter more than lexical content in this setting.\n\nI agree with the reader's assessment that the central claim is plausible but not fully supported. The biggest soft spot is the evaluation protocol. Section 4.1 describes an exhaustive train/test scheme over 11,475 combinations, but Section 4.2 says only that Optuna was used for hyperparameter tuning, with no explicit validation split, nested CV, or statement of the objective data. With that many re-splits, selecting hyperparameters on the same folds used for evaluation could inflate the SSL-vs-SL gap. The reader's stress-test note lands; I don't see a paragraph anywhere that rules out leakage. This is the load-bearing issue because the paper's contribution is the magnitude of the label-efficiency gain.\n\nThe inconsistent numbers are a symptom, not the disease: abstract says 96%, Figure 3 caption says 95%. Minor in itself, but it adds uncertainty. The lack of significance tests is also worth flagging; with only 92 clips in the double-low class, macro-F1 differences of 1–4% could be noise. The authors also rely on their prior threshold of 2.5 for binarization; that is a reasonable continuity choice, not a flaw.\n\nOn the plus side, the paper is honest about SSL assumptions and reports that modality-split co-training performed worse than supervised, which is a useful negative result. The dataset construction is careful, with reliability filtering, targeted clip selection around gaps and overlaps, and holdout sessions.\n\nMy bottom line: this deserves a serious referee, not a desk reject. Ask the authors to document the hyperparameter-selection procedure, rerun with a proper inner validation loop if needed, add significance tests or confidence intervals, and correct the 96/95 inconsistency. For a reader working on affective computing or conversational UX, it is a worthwhile empirical datapoint; for a general ML audience, it is a straightforward application of existing methods.","headline":"A useful application-level result, but the label-efficiency numbers need an evaluation-protocol audit before they can be trusted.","tokens_in":8637,"tokens_out":2366,"would_cite":true,"duration_ms":23424,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A semi-supervised multimodal model can predict negative moments in videoconference conversations almost as well as a fully supervised one, using only a small fraction of the labels.","keywords":["semi-supervised learning","multimodal fusion","videoconference experience","co-training","annotation efficiency","audio embeddings","facial action units","conversation fluidity"],"falsifier":"Rerun the pipeline with nested cross-validation so the hyperparameter search sees only the training folds and the two holdout folds are evaluated exactly once after tuning; if the SSL advantage over supervised learning at 8% labeled data disappears or falls below the reported 4%, the central claim is refuted.","tokens_in":7622,"feed_emoji":"🎙️","tokens_out":10968,"duration_ms":94844,"temperature":0.7,"pith_summary":"This paper tries to establish that semi-supervised learning (SSL) can make the detection of negative moments in videoconference conversations dramatically cheaper to build. The authors target transient moments when a group conversation loses fluidity or enjoyment, which are rare in natural recordings and therefore expensive to label exhaustively. They combine three pretrained multimodal feature sets, audio, facial action units, and text, and wrap a simple logistic regression base classifier in a co-training procedure that assigns pseudo-labels to unlabeled clips. The resulting modality-fused co-training model reaches a ROC-AUC of 0.9 and an F1 score of 0.6, outperforming supervised baselines by up to 4% at the same label budget, and with only 8% of labeled clips it reaches 96% of the fully supervised model's performance. If correct, this means annotation quantity can be cut by an order of magnitude while keeping most of the predictive power for modeling conversational experience.","feed_headline":"8% of labels match 96% of full-supervision performance","feed_subtitle":"Fusing audio, face, and text cues, co-training spots non-fluid or unenjoyable moments with a fraction of the labels.","key_machinery":"The load-bearing mechanism is modality-fused co-training: two logistic-regression base classifiers are trained on two random halves of the principal-component-reduced concatenation of audio, facial-action, and text embeddings, and they alternately assign pseudo-labels to unlabeled clips, including non-targeted clips, across iterative rounds. Co-training is a self-training variant in which two classifiers with different views of the data mutually label unlabeled points; the fused version randomly splits the transformed multimodal feature space so that both classifiers see all modalities. The paper's comparison with a modality-split partition shows that the fused feature treatment is what provides the benefit.","core_discovery":"The central claim is that a semi-supervised classifier that fuses audio, facial-action, and text embeddings can predict low-fluidity and low-enjoyment moments in held-out videoconference sessions nearly as well as a fully supervised model while using a fraction of the labels. The best variant, modality-fused co-training, achieves a ROC-AUC of 0.9 and a macro F1 of 0.6, outperforming supervised logistic regression by up to 4% at matching label budgets, and with only 8% of the labeled targeted clips it reaches 96% of the supervised model's full-data ROC-AUC. The paper reports that audio and facial action are the critical modalities, that fused features are what make co-training work, and that splitting modalities into separate views degrades performance. This is offered as the first demonstration of SSL for videoconference experience modeling.","pith_inferences":["A natural extension would be to apply the same targeted-selection plus SSL recipe to other sparse subjective events in multiparty dialogue, such as confusion, conflict, or rapport breakdown, where exhaustive labeling is similarly prohibitive.","Because the base model is a linear classifier, the success of fused co-training implies the learned multimodal feature space already has a class-aligned cluster structure; testing the same wrapper on a nonlinear deep model would show whether the 8%-label regime advantage persists or is specific to linear separation.","The modality-split failure suggests that audio, face, and text views are not conditionally independent for this task, so applying co-training to multimodal conversation data should verify the independence assumption rather than assuming it."],"forward_implications":["At 8% labeled data, the best SSL model's ROC-AUC on Enjoyment reaches 96% of the fully supervised model's full-data performance.","At 24–32% labeled data, modality-fused co-training matches or exceeds the supervised full-data performance in both F1 and ROC-AUC on Enjoyment and Fluidity.","Non-targeted clips, representing ordinary uneventful periods, are useful unlabeled data for co-training.","Audio and facial-action features are the most informative; text contributes relatively little.","Modality-split co-training underperforms supervised learning, showing that the feature partition decisively affects SSL success."],"supporting_citations":[{"why":"Defines the videoconference fluidity and enjoyment prediction task and the binarization threshold used to split labels.","marker":"[11]"},{"why":"Supplies the SSL assumptions (cluster, smoothness, low-density) that motivate the approach and warn against naive use of unlabeled data.","marker":"[12]"},{"why":"Provides the self-training algorithm that the paper extends into co-training.","marker":"[13]"},{"why":"The RoomReader corpus supplies the videoconference recordings, participant metadata, and transcriptions.","marker":"[14]"},{"why":"Supplies the pretrained audio embedding features for each clip.","marker":"[16]"},{"why":"Supplies the facial action-unit intensity time series used as the visual modality.","marker":"[17]"},{"why":"Supplies the sentence-embedding text features used as the text modality.","marker":"[18]"},{"why":"Gives the theoretical analysis of co-training that the fused wrapper builds on.","marker":"[20]"},{"why":"Provides the co-training implementation used in the experiments.","marker":"[21]"}],"fun_headline_variants":["8% labels, 96% performance: SSL fusion for video calls","Semi-supervised fusion uses 8% labels to match 96% of full supervision","Co-training audio, face, text: 8% labels for near-full video chat quality","Fuse audio, face, text; use 8% labels to spot bad videoconferences"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes the automated hyperparameter search did not use the held-out test folds; the paper reports tuning but does not describe any validation split or nested cross-validation.","fun_headline_variants_meta":{"raw":{"variants":["8% labels, 96% performance: SSL fusion for video calls","Semi-supervised fusion uses 8% labels to match 96% of full supervision","Co-training audio, face, text: 8% labels for near-full video chat quality","Fuse audio, face, text; use 8% labels to spot bad videoconferences"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2565,"prompt_tokens":900,"completion_tokens":1665,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":1573}},"tokens_in":516,"tokens_out":1665,"duration_ms":12762,"temperature":1.0,"reasoning_tokens":1573,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:58:31.316535+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the pipeline with nested cross-validation so the hyperparameter search sees only the training folds and the two holdout folds are evaluated exactly once after tuning; if the SSL advantage over supervised learning at 8% labeled data disappears or falls below the reported 4%, the central claim is refuted.","supporting_citations":[{"cited_title":"The effect of video feedback delay on frustration and emo- tion communication accuracy,","cited_arxiv_id":null,"evidence_quote":"Defines the videoconference fluidity and enjoyment prediction task and the binarization threshold used to split labels."},{"cited_title":"A survey on the semi supervised learning paradigm in the context of speech emotion recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the SSL assumptions (cluster, smoothness, low-density) that motivate the approach and warn against naive use of unlabeled data."},{"cited_title":"Combining cross-modal knowledge transfer and semi-supervised learning for speech emotion recognition,","cited_arxiv_id":null,"evidence_quote":"Provides the self-training algorithm that the paper extends into co-training."},{"cited_title":"SMIN: Semi-supervised multi- modal interaction network for conversational emotion recogni- tion,","cited_arxiv_id":null,"evidence_quote":"The RoomReader corpus supplies the videoconference recordings, participant metadata, and transcriptions."},{"cited_title":"Focused or stuck together: multimodal patterns reveal triads’ performance in collaborative problem solving,","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained audio embedding features for each clip."},{"cited_title":"QoE estimation of webRTC-based audio-visual conversations from facial and speech features,","cited_arxiv_id":null,"evidence_quote":"Supplies the facial action-unit intensity time series used as the visual modality."},{"cited_title":"Multimodal machine learning can predict videoconference fluidity and enjoyment,","cited_arxiv_id":null,"evidence_quote":"Supplies the sentence-embedding text features used as the text modality."},{"cited_title":"Self-training: A survey,","cited_arxiv_id":null,"evidence_quote":"Gives the theoretical analysis of co-training that the fused wrapper builds on."},{"cited_title":"RoomReader: A multimodal corpus of online multiparty conversational interactions,","cited_arxiv_id":null,"evidence_quote":"Provides the co-training implementation used in the experiments."}],"review_version":1}