Pith. sign in

REVIEW 3 major objections 3 minor 61 references

Rethinking the Potential of Multimodality in Collaborative Problem Solving Diagnosis with Large Language Models

T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Audio input gives BERT a small edge in diagnosing collaborative problem solving.

desk verdict A useful, honest applied comparison of multimodal vs text-only transformers for CPS diagnosis, but the AudiBERT gain over BERT rests on a single split with likely utterance-level leakage and needs grouped cross-validation before the 0.014 F1 difference is believed. read the letter →

arxiv 2504.15093 v1 pith:VJKJSA3C submitted 2025-04-21 cs.CL cs.AIcs.LG

K. Wong , B. Wu , S. Bulathwela , M. Cukurova This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords collaborativeproblemsolvingmultimodallearninganalyticstransformermodelsBERTacousticembeddingslargelanguageartificialintelligenceineducationCPSdiagnosis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether adding acoustic information to a transformer-based language model improves automated diagnosis of students' collaborative problem solving (CPS) skills from recorded discussions. On data from 78 secondary students working in triads, the multimodal AudiBERT model—text embeddings from BERT plus acoustic embeddings from Wav2Vec2.0—outperformed text-only BERT on weighted F1 for social-cognitive CPS classes (0.587 vs 0.573), but not consistently on affective classes. The authors' larger point is cautionary: multimodality and model choice should not be assumed universally beneficial; their value depends on the indicator type, label complexity, and class distribution. The paper also shows transformer models beat traditional TF-IDF baselines, but adding acoustic features to the traditional model hurt performance.

What carries the argument

The central object is AudiBERT, a dual self-attentive audio-textual framework in which BERT encodes transcribed utterances and Wav2Vec2.0 encodes the raw audio; each embedding stream passes through its own BiLSTM and a self-attention layer, and the two attended representations are concatenated to classify each utterance into CPS subskills or affective states. This fusion mechanism is what lets the model exploit acoustic-prosodic information beyond the spoken words. The comparison baselines are a random forest on TF-IDF, the same random forest with 11 openSMILE acoustic features appended, and a fine-tuned text-only BERT.

What would settle it

Re-run the BERT versus AudiBERT comparison using a split that keeps all instances of the same source utterance in the same fold (e.g., group by utterance before splitting, then stratify), and check whether AudiBERT's weighted F1 advantage on social-cognitive classes persists and whether the confusion matrices still differ on the sparse classes.

Watch

Extended reading notes

Core claim

The paper claims that a multimodal transformer that fuses textual and acoustic embeddings, AudiBERT, improves the automated diagnosis of social-cognitive collaborative problem-solving classes compared with a unimodal text-only BERT, while the same addition of acoustic features to a traditional random forest on TF-IDF does not help. The improvement is modest on weighted F1 and appears for specific subskills (e.g., sense-making, formulating a solution, reaching a solution, maintaining roles, using scripting) and for the sparse positive-affect class, while the affective dimension overall favours the unimodal text model on accuracy, precision, and recall. The authors therefore argue that the value of multimodality is conditional: it is tied to the choice of model architecture and to the alignment between the acoustic features and the human-coded labels.

Load-bearing premise

The evaluation splits the data after utterances with multiple coded indicators have been expanded into separate instances, so the 80:20 stratified split may place instances from the same source utterance in both training and testing; if so, the text encoder could memorize surface forms and inflate the reported performance difference.

Editorial extensions

If this is right

  • If the central claim holds, practitioners can expect transformer-based models (BERT and AudiBERT) to outperform traditional TF-IDF random forests for CPS diagnosis, especially for sparse classes.
  • Adding acoustic features helps a transformer model but not a traditional model, implying that the model architecture determines whether multimodal data is useful.
  • AudiBERT's gains concentrate in specific social-cognitive subskills and the sparse positive-affect class, suggesting that acoustic information matters where semantic content alone is insufficient.
  • Multimodality should not be assumed a universal improvement; its value depends on the indicator type, the complexity of the labels, and the class distribution of the dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the evaluation's split leakage is real, the reported F1 gains may shrink under a strict utterance-level split; the qualitative conclusion about conditionality may survive, but the class-level claims would need re-testing.
  • AudiBERT's advantage on sparse classes hints that acoustic features act as a regularizer or provide supplementary signal when text examples are few; one could test this by ablating the audio encoder and varying class size.
  • The authors' human-AI complementarity suggestion implies a practical deployment: use models only for classes that are semantically clear and route the rest to human coders; a concrete next step would be to measure total coding time and agreement under such a mixed workflow.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper investigates whether adding acoustic embeddings to transformer-based text models improves automated diagnosis of collaborative problem solving (CPS) subskills and affective states. Using transcripts and audio from 78 secondary students, the authors compare random forest with TF-IDF, a multimodal random forest, fine-tuned BERT, and an audio-assisted BERT variant (AudiBERT) on two coding dimensions with 80:20 stratified splits. They report that transformer models outperform traditional ones, and that AudiBERT achieves a small weighted F1 gain over BERT on the social-cognitive dimension (0.587 vs 0.573) while not improving over BERT on the affective dimension. The authors conclude that multimodality and model selection should not be assumed to help all CPS classes, and they advocate human-AI complementarity and ensemble approaches.

Significance. If the evaluation is valid, the paper provides useful empirical evidence on the value of multimodality in LLM-based CPS diagnosis, using a real classroom setting, a documented coding framework, high inter-rater reliability (Cohen's kappa 0.847), and publicly shared code and appendices. The findings are more nuanced than blanket claims about multimodal superiority and could inform practical choices in AIED systems. However, the central comparison is currently undermined by evaluation-design concerns: a single split, very small test classes, possible train/test leakage from instance-level splitting, and a confound between architecture and modality. Because these issues directly affect the main claim, the contribution is conditional rather than established.

major comments (3)
  1. [Section 3.4 and 3.5] The data processing pipeline creates a risk of train/test leakage. Section 3.4 states that utterances with multiple coded indicators were split into individual instances, each tagged to a single indicator, and Section 3.5 then describes the 80:20 stratified train-test split. If the split operates on these instances rather than on source utterances, the same original utterance can appear in both training and testing under different labels. Since BERT and AudiBERT are text-based, they could memorize surface forms of such near-duplicates, inflating reported performance and potentially changing the relative ordering of models, especially for rare classes. The paper should either confirm that all instances derived from one source utterance were kept on the same side of the split, or re-run the evaluation with an utterance-level grouping before splitting.
  2. [Section 3.5 and Table 2] The main comparison rests on a single fixed-seed 80:20 split with no confidence intervals or significance tests, and several test classes contain only 4, 6, or 8 instances (e.g., SS7, SS4, AS3). The reported AudiBERT advantage over BERT on the social-cognitive dimension is 0.014 weighted F1, which is within the range that could plausibly arise from sampling noise on such small classes. The authors acknowledge in Section 5.1 that multifold cross-validation and statistical analysis are needed before robust conclusions can be drawn; this acknowledgment should be reflected in the results by adding bootstrap confidence intervals, repeated stratified cross-validation, or significance tests, and by reporting per-class performance for the small classes.
  3. [Section 3.1] The BERT and AudiBERT comparison is confounded by architecture. The BERT baseline is fine-tuned directly, whereas AudiBERT feeds BERT text embeddings and Wav2Vec2.0 audio embeddings into separate BiLSTM layers followed by self-attention and concatenation. Thus AudiBERT differs from BERT not only by adding audio but also by adding recurrent and attention components over the text encoder. The observed improvement on social-cognitive classes could stem from these architectural additions rather than from multimodality. To support the claim that audio fusion is the cause, the authors should compare AudiBERT against a text-only model with the same BiLSTM/self-attention post-processing, or ablate the audio branch.
minor comments (3)
  1. [Section 5] The model name 'AudiBERT' is written as 'Audibert' in the first paragraph of the Discussion; please make the capitalization consistent.
  2. [Section 3.3] The reported average Word Error Rate of 0.553 is quite high and could affect the quality of the text modality. The paper should briefly discuss how ASR errors may interact with the comparison between BERT and AudiBERT, or provide a sensitivity check on a subset with lower WER.
  3. [Figure 1] The row-normalized confusion matrices are described only in general terms. Because several classes have very few test instances, the figure would be more informative if accompanied by the actual counts and per-class F1 scores, ideally in a supplementary table.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; AudiBERT comparison is an empirical evaluation on held-out human labels.

full rationale

The paper's central claim is an empirical comparison: AudiBERT (text plus audio embeddings) is evaluated against BERT (text only) and traditional TF-IDF models using human-coded CPS labels as ground truth. The model pipeline is described concretely in Sections 3.1 and 3.6: BERT is fine-tuned on an 80% training split, AudiBERT is initialized with the fine-tuned BERT text encoder and also trained on the same training split, and performance is measured on the held-out 20% test split using weighted precision, recall, and F1. No equation in the paper defines the test predictions in terms of the training labels or the evaluation metric; the reported differences (e.g., F1 0.587 vs 0.573 on the social-cognitive dimension) are computed from model predictions versus human labels on data not used for fitting. The AudiBERT architecture is explicitly attributed to Toto et al. [56], an external source, and the audio embeddings come from pre-trained Wav2Vec2.0, so no uniqueness theorem or ansatz is imported from the authors' own prior work to force the result. The transfer-learning initialization from BERT to AudiBERT is a modeling choice, not a circular step, because it does not encode the target labels or the evaluation outcome. Self-citations appear only in background statements about CPS, multimodal learning analytics, and hybrid intelligence; none of these citations is load-bearing for the central AudiBERT-versus-BERT comparison. The paper itself acknowledges in Section 5.1 that multifold cross-validation and statistical analysis are needed before robust conclusions can be drawn; that is an evaluation-validity limitation, not circular reasoning. A reviewer-raised concern about splitting multi-coded utterances into instances before the train-test split could affect test-set independence, but that would be a data-leakage issue rather than a case where the prediction is equivalent to its input by construction. Consequently, no significant circularity is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No new theoretical entities are introduced. The AudiBERT architecture, Wav2Vec2 embeddings, and GeMAPS features are borrowed from prior work.

assumptions (4)
  • domain assumption Human expert labels are valid ground truth for CPS subskills, indicators, and affect states.
    All models are trained and evaluated against labels assigned by two human coders. Inter-rater reliability is high (Cohen's kappa = 0.847), but label validity is assumed.
  • domain assumption The reduced 42-indicator and 10-subskill framework preserves the CPS constructs of interest.
    The authors merged indicators and realigned subskills from Taylor et al., which changes label semantics and class definitions.
  • ad hoc to paper The train/test split keeps each original utterance on one side of the split.
    Multi-coded utterances are split into separate instances; if the split is performed at instance level, the same source utterance can appear in both training and test sets, violating independence.
  • domain assumption ASR transcripts after manual correction are sufficiently accurate for the classification task.
    The average WER of the raw ASR output is 0.553, and the manual correction protocol is not fully specified, so transcript quality is uncertain.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking the Potential of Multimodality in Collaborative Problem Solving Diagnosis with Large Language Models." pith.science (2026). https://pith.science/paper/VJKJSA3C

@misc{pith2026250415093,
  author       = {Pith},
  title        = {Pith review of: Rethinking the Potential of Multimodality in Collaborative Problem Solving Diagnosis with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VJKJSA3C}},
  note         = {Machine review of arXiv:2504.15093}
}
read the original abstract

Detecting collaborative and problem-solving behaviours from digital traces to interpret students' collaborative problem solving (CPS) competency is a long-term goal in the Artificial Intelligence in Education (AIEd) field. Although multimodal data and advanced models are argued to have the potential to detect complex CPS behaviours, empirical evidence on their value remains limited with some contrasting evidence. In this study, we investigated the potential of multimodal data to improve model performance in diagnosing 78 secondary school students' CPS subskills and indicators in authentic educational settings. In particular, text embeddings from verbal data and acoustic embeddings from audio data were used in a multimodal classification model for CPS diagnosis. Both unimodal and multimodal transformer-based models outperformed traditional models in detecting CPS classes. Although the inclusion of multimodality did not improve the performance of traditional unimodal models, its integration into transformer-based models demonstrated improved performance for diagnosing social-cognitive CPS classes compared to unimodal transformer-based models. Based on the results, the paper argues that multimodality and the selection of a particular modelling technique should not be taken for granted to achieve the best performance in the automated detection of every CPS subskill and indicator. Rather, their value is limited to certain types of CPS indicators, affected by the complexity of the labels, and dependent on the composition of indicators in the dataset. We conclude the paper by discussing the required nuance when considering the value of LLMs and multimodality in automated CPS diagnosis, highlighting the need for human-AI complementarity, and proposing the exploration of relevant model architectures and techniques to improve CPS diagnosis in authentic educational contexts.

Figures

Figures reproduced from arXiv: 2504.15093 by the authors.

Figure 1
Figure 1. Row-normalized confusion matrix with values in each cell showing the propor￾tion of instances from the true class that were predicted as a particular class in the social-cognitive (rows 1 and 2) and in the affective dimension (row 3) [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 35 canonical work pages

  1. [1]

    In: Griffin, P., Care, E

    Adams, R., Vista, A., Scoular, C., Awwal, N., Griffin, P., Care, E.: Automatic coding procedures for collaborative problem solving. In: Griffin, P., Care, E. (eds.) Assessment and Teaching of 21st Century Skills: Methods and Approach. Springer, Dordrecht (2015)

    work page 2015

  2. [2]

    In- ternational Journal of Computer-Supported Collaborative Learning (2024)

    An, S., Zhang, S., Cai, Z., Pan, W., Li, M., Tong, M.: Revealing the in- terplay of cognitive, meta-cognitive, and social processes in university stu- dents’ collaborative problem solving: a three-stage analytical framework. In- ternational Journal of Computer-Supported Collaborative Learning (2024). https://doi.org/https://doi.org/10.1007/s11412-024-09429-0

    work page doi:10.1007/s11412-024-09429-0 2024

  3. [3]

    Artif Intell Rev43(2), 155–177 (2015)

    Anagnostopoulos, C.N., Iliou, T., Giannoukos, I.: Features and classifiers for emo- tion recognition from speech: A survey from 2000 to 2011. Artif Intell Rev43(2), 155–177 (2015). https://doi.org/10.1007/s10462-012-9368-5

    work page doi:10.1007/s10462-012-9368-5 2015

  4. [4]

    In: International Conference on Artificial Intelligence in Education

    Bulathwela,S.,Muse,H.,Yilmaz,E.:Scalableeducationalquestiongenerationwith pre-trained language models. In: International Conference on Artificial Intelligence in Education. pp. 327–339. Springer (2023)

    work page 2023

  5. [5]

    In: LAK23: 13th Interna- tional Learning Analytics and Knowledge Conference

    Chejara, P., Prieto, L.P., Rodriguez-Triana, M.J., Kasepalu, R., Ruiz- Calleja, A., Shankar, S.K.: How to build more generalizable models for collaboration quality? lessons learned from exploring multi-context audio- log datasets using multimodal learning analytics. In: LAK23: 13th Interna- tional Learning Analytics and Knowledge Conference. pp. 111–121 (...

    arXiv 2023

  6. [6]

    Telematics and Informatics 91, 102135 (2024)

    Chuan, C.H., Sun, R., Tian, S., Tsai, W.H.S.: Explainable artificial in- telligence (xai) for facilitating recognition of algorithmic bias: An ex- periment from imposed users’ perspectives. Telematics and Informatics 91, 102135 (2024). https://doi.org/https://doi.org/10.1016/j.tele.2024.102135, https://www.sciencedirect.com/science/article/pii/S073658532400039X

    arXiv 2024

  7. [7]

    British Journal of Educational Technology 00, 1–20 (2024)

    Cukurova, M.: The interplay of learning, analytics and artificial intelligence in edu- cation: A vision for hybrid intelligence. British Journal of Educational Technology 00, 1–20 (2024). https://doi.org/https://doi.org/10.1111/bjet.13514

    doi:10.1111/bjet.13514 2024

  8. [8]

    British Journal of Educational Technology 51(5), 1441–1449 (2020)

    Cukurova, M., Giannakos, M., Martinez-Maldonado, R.: The promise and chal- lenges of multimodal learning analytics. British Journal of Educational Technology 51(5), 1441–1449 (2020). https://doi.org/doi:10.1111/bject.13015

    work page doi:10.1111/bject.13015 2020

Show all 61 references
  1. [9]

    Computers & Education 116, 93–109 (2018)

    Cukurova, M., Luckin, R., Millán, E., Mavrikis, M.: The NISPI framework: Analysing collaborative problem-solving from students’ physical interactions. Computers & Education 116, 93–109 (2018). https://doi.org/10.1016/j.compedu.2017.08.007 14 K. Wong et al

    2018 doi

  2. [10]

    Cukurova, M., Zhou, Q., Spikol, D., Landolfi, L.: Modelling collaborative problem- solving competence with transparent learning analytics: Is video data enough? In: ACM Learning Analytics & Knowledge conference (LAK’20). pp. 270–275. ACM, New York, NY, USA (2020)

    2020

  3. [11]

    In: Burstein, J., Doran, C., Solorio, T

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computat...

    2019 doi

  4. [12]

    1–19 (1999)

    Dillenbourg, P.: What do you mean by collaborative learning? Collaborative- learning: Cognitive and computational approaches pp. 1–19 (1999)

    1999

  5. [13]

    User Modeling and User-Adapted Inter- action pp

    D’Mello, S.K., Duran, N., Michaels, A., Stewart, A.E.B.: Improving collabo- rative problem-solving skills via automated feedback and scaffolding: a quasi- experimental study with cpscoach 2.0. User Modeling and User-Adapted Inter- action pp. 1–39 (2024)

    2024

  6. [14]

    In: Proceedings of the 21st ACM International Conference on Mul- timedia

    Eyben, F., Weninger, F., Gross, F., Schuller, B.: Recent developments in opensmile, the munich open-source multimedia feature extractor. In: Proceedings of the 21st ACM International Conference on Mul- timedia. p. 835–838. MM ’13, Association for Computing Machinery, New York,...

    2013

  7. [15]

    Springer Science & Business Media6 (2007)

    Fischer, F., Ingo, K., Heinz, M., Jörg, M.H.: Scripting computer-supported collab- orative learning: Cognitive, computational and educational perspectives. Springer Science & Business Media6 (2007)

    2007

  8. [16]

    Engineering Applications of Artificial Intelligence115, 105151 (Oct 2022)

    Ganaie, M., Hu, M., Malik, A., Tanveer, M., Suganthan, P.: Ensemble deep learn- ing: A review. Engineering Applications of Artificial Intelligence115, 105151 (Oct 2022). https://doi.org/10.1016/j.engappai.2022.105151

    2022

  9. [17]

    International Journal of Information Management 48, 108–119 (2019)

    Giannakos, M.N., Sharma, K., Pappas, I.O., Kostakos, V., Velloso, E.: Multimodal data as a means to understand the learning experience. International Journal of Information Management 48, 108–119 (2019). https://doi.org/https://doi.org/10.1016/j.ijinfomgt.2019.02.003

    2019 doi

  10. [18]

    Psychological Science in the Public Interest 19(2), 59—-92 (2018)

    Graesser, A.C., Fiore, Stephen M.and Greiff, S., Andrews-Todd, J., Foltz, P.W., Hesse, F.W.: Advancing the science of collaborative problem solv- ing. Psychological Science in the Public Interest 19(2), 59—-92 (2018). https://doi.org/https://doi.org/10.1177/1529100618808244

    2018 doi

  11. [19]

    In: Von, Davier, A.A., Zhu, M., Kyllonen, P.C

    Griffin, P.: Assessing and teaching 21st century skills: Collaborative prob- lem solving as a case study. In: Von, Davier, A.A., Zhu, M., Kyllonen, P.C. (eds.) Innovative Assessment of Collaboration, Methodology of Educational Mea- surement and Assessment, pp. 113–134. Springe...

    2017 doi

  12. [20]

    Organizational Behavior and Human Decision Processes (1996)

    Gruenfeld, D.H., Mannix, E.A., Williams, K.Y., Neale, M.A.: Group composition and decision making: How member familiarity and information distribution affect process and performance. Organizational Behavior and Human Decision Processes (1996)

    1996

  13. [21]

    In: Griffin, P., Care, E

    Hesse, F., Care, E., Buder, J., Sassenberg, K., Griffin, P.: A Framework for Teach- able Collaborative Problem Solving Skills. In: Griffin, P., Care, E. (eds.) Assess- ment and Teaching of 21st Century Skills, pp. 37–56. Springer Netherlands (2015) Rethinking Multimodality wit...

    2015

  14. [22]

    IEEE Access8, 77060–77072 (2020)

    Jin, N., Wu, J., Ma, X., Yan, K., Mo, Y.: Multi-task learning model based on multi-scale cnn and lstm for sentiment classification. IEEE Access8, 77060–77072 (2020). https://doi.org/10.1109/ACCESS.2020.2989428

    2020

  15. [23]

    Cognition and emotion (1989)

    Johnson-Laird, P.N., Oatley, K.: The language of emotions: An analysis of a se- mantic field. Cognition and emotion (1989)

    1989

  16. [24]

    Sci- entific Reports 14(5515) (2024)

    Kappen, M., Vanhollebeke, G., Van Der Donckt, J., Van Hoecke, S., Vanderhas- selt, M.A.: Acoustic and prosodic speech features reflect physiological stress but not isolated negative affect: a multi-paradigm study on psychosocial stressors. Sci- entific Reports 14(5515) (2024)....

    2024 doi

  17. [25]

    In: von Davier, A.A., Zhu, M., Kyllonen, P.C

    Khan, S.M.: Multimodal behavioral analytics in intelligent learning and as- sessment systems. In: von Davier, A.A., Zhu, M., Kyllonen, P.C. (eds.) In- novative Assessment of Collaboration, pp. 173–184. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-33261-1_11

    2017 doi

  18. [26]

    IEEE Access 9, 7107–7126 (2021)

    Kumar, V., Recupero, D.R., Riboni, D., Helaoui, R.: Ensembling Clas- sical Machine Learning and Deep Learning Approaches for Morbidity Identification From Clinical Notes. IEEE Access 9, 7107–7126 (2021). https://doi.org/10.1109/ACCESS.2020.3043221

    2021

  19. [27]

    In: Proceedings of the 15th Learning Analytics and Knowledge Conference

    Li, Z., Cukurova, M., Bulathwela, S.: A novel approach to scalable and auto- matic topic-controlled question generation in education. In: Proceedings of the 15th Learning Analytics and Knowledge Conference. LAK ’25, Association for Computing Machinery (2025)

    2025

  20. [28]

    In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG)

    Littlewort,G.,Whitehill,J.,Wu,T.,Fasel,I.,Frank,M.,Movellan,J.,Bartlett,M.: The computer expression recognition toolbox (cert). In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG). pp. 298–305 (2011). https://doi.org/10.1109/FG.2011.5771414

    2011

  21. [29]

    In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Liu, C., Picheny, M., Sarı, L., Chitkara, P., Xiao, A., Zhang, X., Chou, M., Al- varado, A., Hazirbas, C., Saraf, Y.: Towards measuring fairness in speech recogni- tion: Casual conversations dataset transcriptions. In: ICASSP 2022 - 2022 IEEE International Conference on Acoust...

    2022

  22. [30]

    Luckin, R., Baines, E., Cukurova, M., Holmes, W., Mann, M.: Solved! Making the case for collaborative problem-solving (2017), https://www.nesta.org.uk/report/solved-making-the-case-for-collaborative- problem-solving/

    2017

  23. [31]

    https://doi.org/10.48550/arXiv.2412.16443

    Luo, C.: Has LLM Reached the Scaling Ceiling Yet? Unified Insights into LLM Regularities and Constraints (2024). https://doi.org/10.48550/arXiv.2412.16443

    2024 doi arXiv

  24. [32]

    Handbook of Educational Psychology pp

    Mayer, R.E., Wittrock, M.C.: Problem-solving transfer. Handbook of Educational Psychology pp. 47 – 62 (1996)

    1996

  25. [33]

    In: Zitouni, I

    Mohit, B.: Named Entity Recognition. In: Zitouni, I. (ed.) Natural Language Pro- cessing of Semitic Languages, pp. 221–245. Springer Berlin Heidelberg, Berlin, Heidelberg (2014). https://doi.org/10.1007/978-3-642-45358-8_7

    2014 doi

  26. [34]

    Sensors20(23), 6856 (2020)

    Mu, S., Cui, M., Huang, X.: Multimodal data fusion in learning analytics: A sys- tematic review. Sensors20(23), 6856 (2020)

    2020

  27. [35]

    In: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication

    Nasir, J., Abderrahim, M., Bruno, B., Dillenbourg, P.: An hmm-based real- time intervention methodology for a social robot supporting learning. In: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication. pp. 2204–2211. IEEE, New York City, United ...

    2023

  28. [36]

    OECD Pub- lishing, Paris (2017), https://doi.org/10.1787/9789264281820-en

    OECD: PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving. OECD Pub- lishing, Paris (2017), https://doi.org/10.1787/9789264281820-en

    2017 doi

  29. [37]

    PISA, OECD Publishing, Paris (2017)

    OECD: PISA 2015 Results (Volume V): Collaborative Problem Solving. PISA, OECD Publishing, Paris (2017). https://doi.org/10.1787/9789264285521-en

    2017 doi

  30. [38]

    In: Ifenthaler, D., Pirnay- Dummer, P., Seel, N.M

    O’Neil, H.F., Chuang, S.h.S., Baker, E.L.: Computer-Based Feedback for Computer-Based Collaborative Problem Solving. In: Ifenthaler, D., Pirnay- Dummer, P., Seel, N.M. (eds.) Computer-Based Diagnostics and Systematic Anal- ysis of Knowledge, pp. 261–279. Springer US (2010). ht...

    2010 doi

  31. [39]

    In: Patrick, G., Barry, M., Esther, C

    Patrick, G., Esther, C., McGaw, B.: The changing role of education and schools. In: Patrick, G., Barry, M., Esther, C. (eds.) Assessment and Teaching of 21st Century Skills, Educational Assessment in an Information Age, pp. 3–15. Springer, Dordrecht, Netherlands (2012)

    2012

  32. [40]

    Pugh, S.L., Rao, A., Stewart, A.E., D’Mello, S.K.: Do Speech-Based Collabora- tion Analytics Generalize Across Task Contexts? In: ACM International Confer- ence Proceeding Series. pp. 208–218. Association for Computing Machinery (2022). https://doi.org/10.1145/3506860.3506894

    2022

  33. [41]

    In: Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021

    Pugh, S.L., Subburaj, S.K., Rao, A.R., Stewart, A.E., Andrews-Todd, J., D’Mello, S.K.: Say what? automatic modeling of collaborative problem solving skills from student speech in the wild. In: Proceedings of the 14th International Conference on Educational Data Mining, EDM 202...

    2021

  34. [42]

    Scientific Reports 14(1), 6086 (Mar 2024)

    Rainio, O., Teuho, J., Klén, R.: Evaluation metrics and statistical tests for machine learning. Scientific Reports 14(1), 6086 (Mar 2024). https://doi.org/10.1038/s41598-024-56706-x

    2024 doi

  35. [43]

    Computers & Education175, 104313 (2021)

    Rojas, M., Nussbaum, M., Chiuminatto, P., Guerrero, O., Greiff, S., Krieger, F., Van Der Westhuizen, L.: Assessing collaborative problem-solving skills among el- ementary school students. Computers & Education175, 104313 (2021)

    2021

  36. [44]

    Speech Communication pp

    Scherer, K.R.: Vocal communication of emotion: A review of researchparadigms. Speech Communication pp. 227–256 (2003)

    2003

  37. [45]

    Journal of Big Data8 (2021)

    Shorten,C.,Khoshgoftaar,T.M.,Furht,B.:Textdataaugmentationfordeeplearn- ing. Journal of Big Data8 (2021)

    2021

  38. [46]

    IEEE Access 12, 143177–143192 (2024)

    Siddiqui, J.A., Yuhaniz, S.S., Shaikh, G.M., Soomro, S.A., Mahar, Z.A.: Fine-grained multilingual hate speech detection using explain- able ai and transformers. IEEE Access 12, 143177–143192 (2024). https://doi.org/10.1109/ACCESS.2024.3470901

    2024

  39. [47]

    Journal of Computer Assisted Learning 34(4), 366–377 (2018)

    Spikol, D., Ruffaldi, E., Dabisias, G., Cukurova, M.: Supervised machine learning in multimodal learning analytics for estimating success in project- based learning. Journal of Computer Assisted Learning 34(4), 366–377 (2018). https://doi.org/https://doi.org/10.1111/jcal.12263

    2018 doi

  40. [48]

    User Modeling and User-Adapted Interaction 31(4), 713–751 (2021)

    Stewart, A.E.B., Keirn, Z., D’Mello, S.K.: Multimodal modeling of collaborative problem-solving facets in triads. User Modeling and User-Adapted Interaction 31(4), 713–751 (2021). https://doi.org/10.1007/s11257-021-09290-y

    2021 doi

  41. [49]

    In: Proceedings of the ACM on Human- Computer Interaction

    Stewart, A.E., Vrzakova, H., Sun, C., Yonehiro, J., Stone, C.A., Du- ran, N.D., Shute, V., D’Mello, S.K.: I say, you say, we say: Using spo- ken language to model socio-cognitive processes during computer-supported collaborative problem solving. In: Proceedings of the ACM on H...

    2019 doi

  42. [50]

    Computers and Education 143 (2020)

    Sun, C., Shute, V.J., Stewart, A., Yonehiro, J., Duran, N., D’Mello, S.: Towards a generalized competency model of collaborative problem solving. Computers and Education 143 (2020)

    2020

  43. [52]

    Journal of Learning Analytics8(1), 75–94 (2021-04-09)

    Swiecki, Z.: Measuring the impact of interdependence on individuals during collab- orative problem-solving. Journal of Learning Analytics8(1), 75–94 (2021-04-09). https://doi.org/10.18608/jla.2021.7240

    2021

  44. [53]

    In: Hovy, D., Spruit, S., Mitchell, M., Bender, E.M., Strube, M., Wallach, H

    Tatman, R.: Gender and dialect bias in YouTube‘s automatic captions. In: Hovy, D., Spruit, S., Mitchell, M., Bender, E.M., Strube, M., Wallach, H. (eds.) Pro- ceedings of the First ACL Workshop on Ethics in Natural Language Process- ing. pp. 53–59. Association for Computationa...

    2017 doi

  45. [54]

    In: ACM Learning Analytics & Knowledge conference (LAK’24)

    Taylor, M., Barthakur, A., Azad, A., Joksimovic, S., Zhang, X., Siemens, G.: Quan- tifying collaborative complex problem solving in classrooms using learning ana- lytics. In: ACM Learning Analytics & Knowledge conference (LAK’24). pp. 551–

  46. [55]

    Radiology: Artificial Intelligence 4(4), e220007 (2022)

    Tejani, A.S., Ng, Y.S., Xi, Y., Fielding, J.R., Browning, T.G., Rayan, J.C.: Per- formance of Multiple Pretrained BERT Models to Automate and Accelerate Data Annotation for Large Datasets. Radiology: Artificial Intelligence 4(4), e220007 (2022). https://doi.org/10.1148/ryai.220007

    2022 doi

  47. [56]

    In: Proceedings of the 30th ACM International Conference on Information & Knowledge Manage- ment

    Toto, E., Tlachac, M., Rundensteiner, E.A.: AudiBERT: A Deep Transfer Learning Multimodal Classification Framework for Depression Screening. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Manage- ment. pp. 4145–4154. ACM (2021). https://doi...

    2021

  48. [57]

    Vaessen, N.: Word error rate for automatic speech recognition (2019), https: //pypi.org/project/jiwer/

    2019

  49. [58]

    In: ACM Learning Ana- lytics & Knowledge conference (LAK’20)

    Vrzakova, H., Amon, M.J., Stewart, A., Duran, N.D., D’Mello, S.K.: Focused or stuck together: multimodal patterns reveal triads’ per- formance in collaborative problem solving. In: ACM Learning Ana- lytics & Knowledge conference (LAK’20). pp. 295–304. ACM, New York, NY, USA (2...

    2020

  50. [59]

    In: 2008 IEEE International Conference on Sys- tems, Man and Cybernetics

    Zhang, W., Yoshida, T., Tang, X.: TFIDF, LSI and multi-word in information retrieval and text categorization. In: 2008 IEEE International Conference on Sys- tems, Man and Cybernetics. pp. 108–113. IEEE, Singapore, Singapore (2008). https://doi.org/10.1109/ICSMC.2008.4811259

    2008

  51. [60]

    British Journal of Educational Technology 55(4), 1673–1702 (2024)

    Zhao, L., Gašević, D., Swiecki, Z., Li, Y., Lin, J., Sha, L., Yan, L., Alfredo, R., Li, X., Martinez-Maldonado, R.: Towards automated transcribing and cod- ing of embodied teamwork communication through multimodal learning an- alytics. British Journal of Educational Technology...

    2024 doi

  52. [61]

    Education and Information Technologies29, 1071– 1098 (2024)

    Zhou, Q., Suraworachet, W., Cukurova, M.: Detecting non-verbal speech and gaze behaviours with multimodal data and computer vision to interpret effective collab- orative learning interactions. Education and Information Technologies29, 1071– 1098 (2024)

    2024

  53. [562]

    https://doi.org/10.1145/3636555.3636913, https://dl.acm.org/doi/10.1145/3636555.3636913

    ACM, New York, NY, USA (2024). https://doi.org/10.1145/3636555.3636913, https://dl.acm.org/doi/10.1145/3636555.3636913

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.