Pith. sign in

REVIEW 3 major objections 5 minor 35 references

Multimodal Fusion with Semi-Supervised Learning Minimizes Annotation Quantity for Modeling Videoconference Conversation Experience

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A semi-supervised multimodal model can predict negative moments in videoconference conversations almost as well as a fully supervised one, using only a small fraction of the labels.

desk verdict A useful application-level result, but the label-efficiency numbers need an evaluation-protocol audit before they can be trusted. read the letter →

arxiv 2506.13971 v1 pith:OQQ2Q7VF submitted 2025-06-01 eess.AS cs.CLcs.HCcs.LGcs.MM

Andrew Chang , Chenkai Hu , Ji Qi , Zhuojian Wei , Kexin Zhang , Viswadruth Akkaraju , David Poeppel , Dustin Freeman This is my paper
classification eess.AScs.CLcs.HCcs.LGcs.MM
keywords semi-supervisedlearningmultimodalfusionvideoconferenceexperienceco-trainingannotationefficiencyaudioembeddingsfacialactionunitsconversationfluidity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that semi-supervised learning (SSL) can make the detection of negative moments in videoconference conversations dramatically cheaper to build. The authors target transient moments when a group conversation loses fluidity or enjoyment, which are rare in natural recordings and therefore expensive to label exhaustively. They combine three pretrained multimodal feature sets, audio, facial action units, and text, and wrap a simple logistic regression base classifier in a co-training procedure that assigns pseudo-labels to unlabeled clips. The resulting modality-fused co-training model reaches a ROC-AUC of 0.9 and an F1 score of 0.6, outperforming supervised baselines by up to 4% at the same label budget, and with only 8% of labeled clips it reaches 96% of the fully supervised model's performance. If correct, this means annotation quantity can be cut by an order of magnitude while keeping most of the predictive power for modeling conversational experience.

What carries the argument

The load-bearing mechanism is modality-fused co-training: two logistic-regression base classifiers are trained on two random halves of the principal-component-reduced concatenation of audio, facial-action, and text embeddings, and they alternately assign pseudo-labels to unlabeled clips, including non-targeted clips, across iterative rounds. Co-training is a self-training variant in which two classifiers with different views of the data mutually label unlabeled points; the fused version randomly splits the transformed multimodal feature space so that both classifiers see all modalities. The paper's comparison with a modality-split partition shows that the fused feature treatment is what provides the benefit.

What would settle it

Rerun the pipeline with nested cross-validation so the hyperparameter search sees only the training folds and the two holdout folds are evaluated exactly once after tuning; if the SSL advantage over supervised learning at 8% labeled data disappears or falls below the reported 4%, the central claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that a semi-supervised classifier that fuses audio, facial-action, and text embeddings can predict low-fluidity and low-enjoyment moments in held-out videoconference sessions nearly as well as a fully supervised model while using a fraction of the labels. The best variant, modality-fused co-training, achieves a ROC-AUC of 0.9 and a macro F1 of 0.6, outperforming supervised logistic regression by up to 4% at matching label budgets, and with only 8% of the labeled targeted clips it reaches 96% of the supervised model's full-data ROC-AUC. The paper reports that audio and facial action are the critical modalities, that fused features are what make co-training work, and that splitting modalities into separate views degrades performance. This is offered as the first demonstration of SSL for videoconference experience modeling.

Load-bearing premise

The evaluation assumes the automated hyperparameter search did not use the held-out test folds; the paper reports tuning but does not describe any validation split or nested cross-validation.

Editorial extensions

If this is right

  • At 8% labeled data, the best SSL model's ROC-AUC on Enjoyment reaches 96% of the fully supervised model's full-data performance.
  • At 24–32% labeled data, modality-fused co-training matches or exceeds the supervised full-data performance in both F1 and ROC-AUC on Enjoyment and Fluidity.
  • Non-targeted clips, representing ordinary uneventful periods, are useful unlabeled data for co-training.
  • Audio and facial-action features are the most informative; text contributes relatively little.
  • Modality-split co-training underperforms supervised learning, showing that the feature partition decisively affects SSL success.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to apply the same targeted-selection plus SSL recipe to other sparse subjective events in multiparty dialogue, such as confusion, conflict, or rapport breakdown, where exhaustive labeling is similarly prohibitive.
  • Because the base model is a linear classifier, the success of fused co-training implies the learned multimodal feature space already has a class-aligned cluster structure; testing the same wrapper on a nonlinear deep model would show whether the 8%-label regime advantage persists or is specific to linear separation.
  • The modality-split failure suggests that audio, face, and text views are not conditionally independent for this task, so applying co-training to multimodal conversation data should verify the independence assumption rather than assuming it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a semi-supervised learning (SSL) framework for predicting subjective moments of low fluidity or low enjoyment in videoconference sessions. The authors extract 7-second targeted clips around turn-taking disruptions (gaps and overlaps) and non-targeted clips from the remainder of each session, annotate the targeted clips with multiple crowd annotators, and binarize the ratings using a threshold from prior work. They then compare three SSL wrappers (self-training, modality-split co-training, and modality-fused co-training) against a supervised logistic regression baseline at varying labeled-data ratios, using audio (VGGish), facial action units (OpenFace), and text (Sentence-BERT) features. The core claim is that modality-fused co-training SSL outperforms supervised learning by up to 4% at the same label budget, reaches an ROC-AUC of 0.9 and F1 of 0.6, and with only 8% labeled data matches 96% of the supervised full-data performance, thereby providing an annotation-efficient method for modeling videoconference experience.

Significance. If the evaluation protocol is sound, this is a practically valuable contribution: it directly attacks the annotation bottleneck for a neglected but important social-computing problem, uses naturalistic multiparty videoconference data, and includes an informative ablation showing that audio and face features dominate over text. The paper also gives credit for comparing SSL against an SL counterpart at matched label budgets, which is the right way to isolate the benefit of unlabeled data. However, the central quantitative claims hinge on the integrity of the hyperparameter search and on the exact composition of the unlabeled pool; both are under-specified in the manuscript, so the current numerical findings are not fully supported as written.

major comments (3)
  1. [Section 4.2] The manuscript states that the Optuna framework was used for hyperparameter tuning with a tree-structured parzen estimator, but it never describes the objective data, the number of trials, or, critically, any validation split. If the search was evaluated on the same holdout folds that later produced the reported test scores, or on pooled data that includes held-out sessions, the claimed 1-4% SSL-over-SL advantage could be selection bias rather than a genuine SSL benefit. This is load-bearing for the paper's central label-efficiency claim, so please specify the exact protocol (e.g., nested cross-validation or a fixed validation split inside each training fold) and, if no such protocol was used, re-run the comparison under a clean evaluation scheme.
  2. [Section 4.1] Non-targeted clips are extracted from all sessions and are never assigned to the 10 folds; the text indicates they are always included as unlabeled data. As a result, for a test fold containing sessions S, the unlabeled pool includes non-targeted clips from those very same sessions S. This is a transductive setup in which the model sees the audiovisual and textual features of the target sessions (without labels) before predicting their targeted clips, which can inflate performance and materially weakens the claim that the model generalizes to new sessions with different participants. Please either exclude non-targeted clips from held-out sessions or report a purely inductive variant with unlabeled data restricted to training sessions, and clearly state which setting is shown in Figure 3.
  3. [Abstract, Section 5, and Figure 3] There is a numerical inconsistency: the Abstract reports that the best SSL model with 8% labeled data matched 96% of the SL model's full-data performance, while the Figure 3 caption says 95%, and the Results text specifies only that 'the ROC-AUC ... can achieve 96% performance ... on predicting Enjoyment.' This discrepancy must be reconciled, and the metric to which the percentage applies must be stated precisely. More importantly, the paper reports no significance tests or confidence intervals for the 'up to 4%' SSL-over-SL advantage; given the severe class imbalance (92 clips low on both scales versus 2,731 high on both) and the limited number of sessions per fold (about 3), the observed differences may be within sampling variability. Please provide paired statistical comparisons across folds, such as Wilcoxon signed-rank tests or bootstrap confidence intervals, for each labeled-data ratio and metric.
minor comments (5)
  1. [Section 3.1.2] The phrase 'such as overlapping speech or unusually long gaps—instances' contains a stray em dash and an unnecessary word; it should read 'such as overlapping speech or unusually long gaps.'
  2. [Section 4.2] The hyperparameter description 'PCA explained variance: 0– or 20–100%' is likely a typo; please clarify whether the search space is 0-100%, or 0% and 20-100% as two distinct options.
  3. [Figures 3 and 4] The error bars are described as standard error, but the effective sample size (number of folds or combinations) for each labeled-data percentage is not reported; please state n for each point so readers can interpret the error bars.
  4. [Section 3.3.1] There is a typo in 'Y AMNet' (likely 'YAMNet').
  5. [Section 2] The phrase 'the phenomena is' should be 'the phenomena are' or 'this phenomenon is'; also, the reference to [11] for the binarization threshold is appropriate but should be accompanied by an explicit reproducibility note that the threshold is a free parameter from prior work.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims rest on an empirical benchmark with held-out sessions, not on a self-referential derivation.

full rationale

This is an empirical machine-learning benchmark paper, not a derivation. The central claims (SSL outperforming SL with the same label budget, and 8% labels matching 96% of full-supervision performance) are evaluated against held-out videoconference sessions via StratifiedGroupKFold, with sessions kept in the same fold to test generalization to new sessions. The SSL and SL models are trained on the same base classifier and compared under identical label budgets, so the reported SSL advantage is not forced by construction. The only self-referential element is the binary threshold of 2.5, justified by the authors' prior work [11], but that is a preprocessing choice inherited for continuity and does not define the prediction target; the labels themselves come from human annotators. The targeted clip selection creates a correlation between audio features and labels, but this is a deliberate sampling strategy, not a case of fitting a parameter and then predicting it back. A legitimate concern is that Section 4.2 describes Optuna hyperparameter tuning without specifying a validation split or nested cross-validation, which could inflate reported performance if the tuning objective used the same folds as the test evaluation; however, that is an evaluation-protocol risk and not a circularity, because no fitted quantity is being relabeled as a prediction. The paper is self-contained as an empirical comparison and does not rely on a uniqueness theorem, an ansatz smuggled in via citation, or a renaming of known results. Score 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The pipeline depends on several hand-chosen thresholds and a tuned hyperparameter set, plus assumptions about the validity of pretrained features and human ratings. The most consequential input is the set of Optuna hyperparameters, whose selection procedure is underdescribed.

free parameters (6)
  • RMS threshold for speaker activity = 0.05
    Chosen by visual inspection of audio waveforms; used to detect gaps and overlaps in targeted clip selection (Section 3.1.2).
  • Gap duration threshold = 0.75 seconds
    Defines a 'long gap' for targeted clips, exceeding typical turn-taking gaps (Section 3.1.2).
  • Binarization threshold for Likert ratings = 2.5
    Adopted from prior work [11] based on a gap in the data distribution (Section 3.2.4).
  • Annotator reliability cutoff = Pearson r > 0.2
    Filters annotators based on correlation with group average ratings (Section 3.2.3).
  • Optuna hyperparameters (PCA explained variance, loss, penalty, alpha, self-training criterion, pseudo-label threshold) = Ranges listed in Section 4.2; exact selected values not reported
    Tuned via Optuna; no separate validation set described, so tuning could affect test results.
  • Co-training feature split = Random split of PCA-transformed fused features
    Choice for modality-fused co-training; not based on a principled view-independence assumption (Section 4.2).
assumptions (4)
  • domain assumption SSL assumptions (cluster/smoothness and low-density) hold for the fused feature space.
    The success of self-training and co-training depends on these assumptions; the paper itself notes they may be violated (Section 2, 6).
  • domain assumption Pretrained feature extractors (VGGish, OpenFace, SBERT) provide features relevant to conversational experience.
    The model uses these embeddings without fine-tuning or task-specific adaptation (Section 3.3).
  • domain assumption Human Likert ratings, after reliability filtering and binarization, are valid ground truth for fluidity and enjoyment.
    The annotation pipeline (Section 3.2) assumes these ratings reflect the target construct and that a 2.5 threshold separates negative from non-negative moments.
  • standard math StratifiedGroupKFold produces independent session-level splits.
    The method keeps same-session clips in one fold (Section 4.1), but the paper does not verify that sessions are exchangeable or that fold sizes are balanced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Fusion with Semi-Supervised Learning Minimizes Annotation Quantity for Modeling Videoconference Conversation Experience." pith.science (2026). https://pith.science/paper/OQQ2Q7VF

@misc{pith2026250613971,
  author       = {Pith},
  title        = {Pith review of: Multimodal Fusion with Semi-Supervised Learning Minimizes Annotation Quantity for Modeling Videoconference Conversation Experience},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OQQ2Q7VF}},
  note         = {Machine review of arXiv:2506.13971}
}
read the original abstract

Group conversations over videoconferencing are a complex social behavior. However, the subjective moments of negative experience, where the conversation loses fluidity or enjoyment remain understudied. These moments are infrequent in naturalistic data, and thus training a supervised learning (SL) model requires costly manual data annotation. We applied semi-supervised learning (SSL) to leverage targeted labeled and unlabeled clips for training multimodal (audio, facial, text) deep features to predict non-fluid or unenjoyable moments in holdout videoconference sessions. The modality-fused co-training SSL achieved an ROC-AUC of 0.9 and an F1 score of 0.6, outperforming SL models by up to 4% with the same amount of labeled data. Remarkably, the best SSL model with just 8% labeled data matched 96% of the SL model's full-data performance. This shows an annotation-efficient framework for modeling videoconference experience.

Figures

Figures reproduced from arXiv: 2506.13971 by the authors.

Figure 1
Figure 1. An example of the audio of natural turn-taking in a 4-people group conversation. We marked the start of a gap or overlap and extracted a 7-second ”targeted clip” around each. These clips were then evaluated by multiple human annotators to identify those with low Fluidity or Enjoyment. The ”non￾targeted clips” were extracted from the remaining segments. 3.1. Video clips 3.1.1. Videoconference corpus We sourced the vi… view at source ↗
Figure 2
Figure 2. Multimodal feature extraction, modality fusion, and three SSL approaches. an increasingly popular initial choice for SSL. In the self-training algorithm, a base classifier iteratively assigns pseudo-labels to unlabeled datapoints whose confidence scores exceed a predefined threshold, thereby expanding the la￾beled dataset and retraining the base classifier [13, 19]. Co-training is an extension of self-training that … view at source ↗
Figure 4
Figure 4. Ablation study: the fusion of the audio and face fea￾tures is most critical for SSL modeling conversational experi￾ence in videoconferencing. (errorbar: standard error) importance of the paralinguistic speech features, which can bet￾ter be captured by an audio model than a BERT model, for the current tasks. 6. Discussion This study demonstrated the efficiency of selectively targeting a few moments in continuously re… view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Performance of SSL and SL models. The modality￾fused co-training SSL model outperforms SL models by up to 4% with the same amount of labeled data, and using just 8% of labeled data, it matches 95% of the SL model’s full-data perfor￾mance. (errorbar: standard error) Not…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 34 canonical work pages

  1. [1]

    Although it is an es- sential medium for communication, it has not been sufficiently studied

    Introduction Videoconferencing has become a common and irreplaceable channel for communication in both professional and casual con- texts, especially in the post-COVID-19 era. Although it is an es- sential medium for communication, it has not been sufficiently studied. One of the major differences is that, compared to in- person contexts, it is not yet as...

    work page

  2. [2]

    While audio-based SSL has been widely applied in speech emotion recognition, multimodal approaches have only recently emerged [5]

    Prior Works To the best of our knowledge, SSL has not been used to model videoconferencing experience. While audio-based SSL has been widely applied in speech emotion recognition, multimodal approaches have only recently emerged [5]. For instance, a study used SSL and a visual-to-audio knowledge transfer ap- proach to improve the robustness of the model [...

    work page

  3. [3]

    Multimodal Fusion with Semi-Supervised Learning Minimizes Annotation Quantity for Modeling Videoconference Conversation Experience

    Dataset This work built on a dataset curated in previous works [11, 14]. Specifically, the current study further extended the processed dataset by incorporating new clips and multimodal features. arXiv:2506.13971v1 [eess.AS] 1 Jun 2025 Figure 1: An example of the audio of natural turn-taking in a 4-people group conversation. We marked the start of a gap o...

    work page Pith review arXiv 2025

  4. [4]

    Data split To analyze the impact of the labeled data ratio on SSL model performance, we partitioned the targeted clips data into 10 folds

    Modeling 4.1. Data split To analyze the impact of the labeled data ratio on SSL model performance, we partitioned the targeted clips data into 10 folds. We employed StratifiedGroupKFold() from sklearn to preserve the approximate distribution of high- and low-rated clips while ensuring that data from the same videoconference session remained within the sam...

    work page

  5. [5]

    They both outperformed SL counterparts at nearly every levels of labeled data in both ROC-AUC and macro F1 score by 1-4% for predicting Enjoy- ment or Fluidity

    Results Overall, modality-fused co-training model performed the best, followed by self-training (Figure 3). They both outperformed SL counterparts at nearly every levels of labeled data in both ROC-AUC and macro F1 score by 1-4% for predicting Enjoy- ment or Fluidity. Notably, when only 8% of labeled data avail- able, the ROC-AUC of the modality-fused co-...

    work page

  6. [6]

    We also demonstrated this approach gener- alizes to new sessions with different participants

    Discussion This study demonstrated the efficiency of selectively targeting a few moments in continuously recorded data with minimal anno- tation to train a robust SSL classifier for detecting moments of negative experience. We also demonstrated this approach gener- alizes to new sessions with different participants. Even unevent- ful moments (i.e., non-ta...

    work page

  7. [7]

    are supported by NYU Discovery Research Fund for Human Health

    Acknowledgements A.C., D.P., and D.F. are supported by NYU Discovery Research Fund for Human Health. A.C. is supported by National Re- search Service Award, NIDCD/NIH (F32DC018205) and Leon Levy Scholarships in Neuroscience, Leon Levy Foundation and New York Academy of Sciences. The funders have no role in study design, data collection and analysis, decis...

    work page

  8. [8]

    Sepa- rable processes for live “in-person

    N. Zhao, X. Zhang, J. A. Noah, M. Tiede, and J. Hirsch, “Sepa- rable processes for live “in-person” and live “zoom-like” faces,” Imaging Neuroscience, vol. 1, pp. 1–17, 2023

    work page 2023

Show all 35 references
  1. [9]

    Perceiving others through a screen: Are first im- pressions of personality accurate and normative via videocon- ferencing?

    M.-C. Mignault, H. Tissera, J. Cecere, Z. Fargnoli-Brown, and L. Human, “Perceiving others through a screen: Are first im- pressions of personality accurate and normative via videocon- ferencing?” Personality and Social Psychology Bulletin , p. 01461672241263249, 2024

    2024

  2. [10]

    Virtual (zoom) interactions alter conversational behavior and in- terbrain coherence,

    S. Balters, J. G. Miller, R. Li, G. Hawthorne, and A. L. Reiss, “Virtual (zoom) interactions alter conversational behavior and in- terbrain coherence,” Journal of Neuroscience, vol. 43, no. 14, pp. 2568–2578, 2023

    2023

  3. [11]

    The effect of video feedback delay on frustration and emo- tion communication accuracy,

    S. R. Powers, C. Rauh, R. A. Henning, R. W. Buck, and T. V . West, “The effect of video feedback delay on frustration and emo- tion communication accuracy,” Computers in Human Behavior , vol. 27, no. 5, pp. 1651–1657, 2011

    2011

  4. [12]

    A survey on the semi supervised learning paradigm in the context of speech emotion recognition,

    G. Andrade, M. Rodrigues, and P. Novais, “A survey on the semi supervised learning paradigm in the context of speech emotion recognition,” in Intelligent Systems and Applications: Proceed- ings of the 2021 Intelligent Systems Conference (IntelliSys) Vol- ume 2. Springer, 2022,...

    2021

  5. [13]

    Combining cross-modal knowledge transfer and semi-supervised learning for speech emotion recognition,

    S. Zhang, M. Chen, J. Chen, Y .-F. Li, Y . Wu, M. Li, and C. Zhu, “Combining cross-modal knowledge transfer and semi-supervised learning for speech emotion recognition,” Knowledge-Based Sys- tems, vol. 229, p. 107340, 2021

    2021

  6. [14]

    SMIN: Semi-supervised multi- modal interaction network for conversational emotion recogni- tion,

    Z. Lian, B. Liu, and J. Tao, “SMIN: Semi-supervised multi- modal interaction network for conversational emotion recogni- tion,” IEEE Transactions on Affective Computing, vol. 14, no. 3, pp. 2415–2429, 2022

    2022

  7. [15]

    Multimodal emotion recognition with vision-language prompting and modality dropout,

    A. Qi, Z. Liu, X. Zhou, J. Xiao, F. Zhang, Q. Gan, M. Tao, G. Zhang, and L. Zhang, “Multimodal emotion recognition with vision-language prompting and modality dropout,” in Proceed- ings of the 2nd International Workshop on Multimodal and Re- sponsible Affective Computing, 2024...

    2024

  8. [16]

    Focused or stuck together: multimodal patterns reveal triads’ performance in collaborative problem solving,

    H. Vrzakova, M. J. Amon, A. Stewart, N. D. Duran, and S. K. D’Mello, “Focused or stuck together: multimodal patterns reveal triads’ performance in collaborative problem solving,” in Pro- ceedings of the Tenth International Conference on Learning Ana- lytics & Knowledge, 2020, ...

    2020

  9. [17]

    QoE estimation of webRTC-based audio-visual conversations from facial and speech features,

    G. Bing ¨ol, S. Porcu, A. Floris, and L. Atzori, “QoE estimation of webRTC-based audio-visual conversations from facial and speech features,” ACM Trans. Multimedia Comput. Commun. Appl. , vol. 20, no. 5, jan 2024. [Online]. Available: https://doi.org/10.1145/3638251

    2024 doi

  10. [18]

    Multimodal machine learning can predict videoconference fluidity and enjoyment,

    A. Chang, V . Akkaraju, R. M. Cogliano, D. Poeppel, and D. Free- man, “Multimodal machine learning can predict videoconference fluidity and enjoyment,” 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025

    2025

  11. [19]

    A survey on semi-supervised learning,

    J. E. Van Engelen and H. H. Hoos, “A survey on semi-supervised learning,” Machine Learning, vol. 109, no. 2, pp. 373–440, 2020

    2020

  12. [20]

    Self-training: A survey,

    M.-R. Amini, V . Feofanov, L. Pauletto, L. Hadjadj, E. Devijver, and Y . Maximov, “Self-training: A survey,”Neurocomputing, vol. 616, p. 128904, 2025

    2025

  13. [21]

    RoomReader: A multimodal corpus of online multiparty conversational interactions,

    J. Reverdy, S. O’Connor Russell, L. Duquenne, D. Garaialde, B. R. Cowan, and N. Harte, “RoomReader: A multimodal corpus of online multiparty conversational interactions,” in Pro- ceedings of the Thirteenth Language Resources and Evaluation Conference. Marseille, France: Europe...

    2022

  14. [22]

    Zoom disrupts the rhythm of conversation

    J. E. Boland, P. Fonseca, I. Mermelstein, and M. Williamson, “Zoom disrupts the rhythm of conversation.” Journal of Experi- mental Psychology: General, vol. 151, no. 6, p. 1272, 2022

    2022

  15. [23]

    CNN architectures for large-scale audio classification,

    S. Hershey, S. Chaudhuri, D. P. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seyboldet al., “CNN architectures for large-scale audio classification,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASS...

    2017

  16. [24]

    Openface 2.0: Facial behavior analysis toolkit,

    B. Tadas, Z. Amir, L. Y . Chong, and L.-M. Philippe, “Openface 2.0: Facial behavior analysis toolkit,” in 13th IEEE International Conference on Automatic Face & Gesture Recognition, 2018

    2018

  17. [25]

    Sentence-BERT: Sentence embeddings using siamese BERT-networks,

    N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using siamese BERT-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11

    2019

  18. [26]

    Dawn of the trans- former era in speech emotion recognition: closing the valence gap,

    J. Wagner, A. Triantafyllopoulos, H. Wierstorf, M. Schmitt, F. Burkhardt, F. Eyben, and B. W. Schuller, “Dawn of the trans- former era in speech emotion recognition: closing the valence gap,” IEEE Transactions on Pattern Analysis and Machine Intel- ligence, vol. 45, no. 9, pp....

    2023

  19. [27]

    Unsupervised word sense disambiguation rivaling supervised methods,

    D. Yarowsky, “Unsupervised word sense disambiguation rivaling supervised methods,” in 33rd Annual Meeting of the Association for Computational Linguistics, 1995, pp. 189–196

    1995

  20. [28]

    A new analysis of co-training

    W. Wang and Z.-H. Zhou, “A new analysis of co-training.” inPro- ceedings of the 27th International Conference on Machine Learn- ing (ICML), vol. 2, 2010, p. 3

    2010

  21. [29]

    SSLearn: A semi-supervised learning li- brary for python,

    J. L. Garrido-Labrador, J. M. Maudes-Raedo, J. J. Rodr ´ıguez, and C. I. Garc ´ıa-Osorio, “SSLearn: A semi-supervised learning li- brary for python,” SoftwareX, vol. 29, p. 102024, 2025

    2025

  22. [30]

    Optuna: A next-generation hyperparameter optimization framework,

    T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 2623–2631

    2019

  23. [31]

    3m- transformer: A multi-stage multi-stream multimodal transformer for embodied turn-taking prediction,

    M. Fatan, E. Mincato, D. Pintzou, and M. Dimiccoli, “3m- transformer: A multi-stage multi-stream multimodal transformer for embodied turn-taking prediction,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP). IEEE, 2024, pp....

    2024

  24. [32]

    Pre- dicting conversation outcomes using multimodal transformer,

    C. Li, W. Wang, B. Balducci, D. Marinova, and Y . Shang, “Pre- dicting conversation outcomes using multimodal transformer,” in 2021 International Joint Conference on Neural Networks (IJCNN). IEEE, 2021, pp. 1–6

    2021

  25. [33]

    Dyadformer: A multi-modal transformer for long-range model- ing of dyadic interactions,

    D. Curto, A. Clap ´es, J. Selva, S. Smeureanu, J. Junior, C. Jacques, D. Gallardo-Pujol, G. Guilera, D. Leiva, T. B. Moeslund et al., “Dyadformer: A multi-modal transformer for long-range model- ing of dyadic interactions,” inProceedings of the IEEE/CVF Inter- national Confere...

    2021

  26. [35]

    CTNet: Conversational transformer network for emotion recognition,

    Z. Lian, B. Liu, and J. Tao, “CTNet: Conversational transformer network for emotion recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 985–1000, 2021

    2021

  27. [2019]

    Available: https://arxiv.org/abs/1908.10084

    [Online]. Available: https://arxiv.org/abs/1908.10084

    1908 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.