ECMF: Enhanced Cross-Modal Fusion for Multimodal Emotion Recognition in MER-SEMI Challenge

Jiulin Li; Juewen Hu; Pring Wong; Shuo Chen; Yexin Li

REVIEW 4 major objections 4 minor 25 references

ECMF: Enhanced Cross-Modal Fusion for Multimodal Emotion Recognition in MER-SEMI Challenge

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A fusion stack lifts emotion recognition from 78.63% to 87.49%.

desk verdict A plausible challenge-winning recipe with a load-bearing ambiguity about whether LLM emotion labels are fed to the model at test time. read the letter →

arxiv 2508.05991 v1 pith:V6ZLGJYG submitted 2025-08-08 cs.CV cs.AIcs.CY

Juewen Hu , Yexin Li , Jiulin Li , Shuo Chen , Pring Wong This is my paper

classification cs.CVcs.AIcs.CY

keywords multimodalemotionrecognitionMER2025-SEMIcross-modalfusionself-attentionlargelanguagemodelslabelrefinementensemblelearningpre-trainedencoders

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that on the MER2025-SEMI semi-supervised emotion recognition benchmark, a multimodal system built from frozen pre-trained encoders plus a lightweight self-attention fusion module beats the official baseline by a large margin, reaching 87.49% weighted F-score versus 78.63%. To get there, it enriches the text branch with LLM-generated emotional cues, adds a second visual stream that encodes facial patches alongside full frames, refines noisy training labels by majority voting across weak classifiers and an LLM, and ensembles several model variants. If correct, this supports the view that in data-scarce affective computing the decisive ingredients are large pre-trained feature extractors, a small trainable fusion head, and label cleaning, rather than training a large end-to-end model from scratch. The ablation table attributes incremental gains to each added component, with the final ensemble contributing roughly half a point to 1.3 points.

What carries the argument

The load-bearing mechanism is the Feature Fusion Module: each modality's standardized embedding first passes through a residual processing block, a learnable Modal_Token is prepended to encode which modality a sequence belongs to, and two self-attention layers produce a joint representation for classification. Two auxiliary mechanisms do the heavy lifting upstream: the dual-branch visual encoder, which runs both full frames and OpenFace facial patches through CLIP-ViT-Large, and the context-enriched text encoder, which appends GPT-4 and Qwen-Omni outputs to the original text before Chinese-RoBERTa encoding.

What would settle it

Take the official test split (or a held-out subset of the labeled training data) and rerun the pipeline with the text branch fed only the original transcripts, no GPT-4 keywords and no Qwen-Omni clues. If the weighted F-score drops from 87.49% toward the 78.63% baseline, the claimed improvement comes from LLM-supplied label information rather than from cross-modal fusion. A second check is to inspect enriched text for test samples and see whether any contains the ground-truth emotion word.

Watch

Extended reading notes

Core claim

On the MER2025-SEMI challenge, the proposed Enhanced Cross-Modal Fusion (ECMF) framework reports a test-set weighted F-score of 87.49%, against the official baseline's 78.63%. The framework freezes HuBERT-Large for audio, Chinese-RoBERTa-wwm-ext-large for text, and CLIP-ViT-Large for video; the video branch is dual-scale, encoding both whole frames and OpenFace-detected facial patches. Text is enriched by concatenating GPT-4-generated emotion keywords and Qwen-Omni-generated pseudo-labels, descriptions, and emotional clues with the original transcript. The standardized features from all modalities are fused by a module that prepends a learnable Modal_Token to each modality sequence and appli

Load-bearing premise

The LLM-generated emotional clues and pseudo-labels are used only as enriched input text at training time, and do not pass the predicted emotion category to the classifier at test time.

Editorial extensions

If this is right

Frozen large encoders plus a small trainable fusion head can beat a trained-from-scratch baseline by about 9 weighted-F-score points on MER2025-SEMI.
LLM-generated emotional cues added to the text input appear to compensate for the text modality's relative weakness in emotion recognition.
Cleaning noisy training labels with weak per-modality classifiers plus an LLM, then majority voting, improves generalization without a large curated dataset.
Ensembling model variants obtained by module removal or by changing random seeds yields a final gain of 0.5–1.3 points, suggesting the individual variants sit near a performance plateau.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not state whether GPT-4 and Qwen-Omni generation is run on test samples; if it is, the reported gain may partly come from external-model knowledge of the test content rather than from the fusion architecture itself.
A decisive ablative experiment would drop all LLM-enriched tokens from the text branch while keeping everything else fixed; if the F-score falls back toward 78.63%, the LLM is the active ingredient, not the fusion design.
The multi-source labeling procedure could be extended to self-training on the 20,000 unlabeled videos, effectively turning the semi-supervised setting into a weakly supervised one.
Because the pipeline depends on closed LLM APIs, reproducibility is conditional on those services; testing with open-weight models would separate the benefit of model scale from the benefit of this specific architecture.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the quick take. This is a MER2025-SEMI system paper that reports 87.49% WAF on the official test set, roughly 9 points above the official baseline. The gains come from a stack of known pieces: HuBERT audio features, CLIP visual features, RoBERTa text, an attention fusion layer with a modal token, multi-source label cleaning, and an ensemble. Nothing is conceptually new, but the combination is careful and the ablation table is mostly consistent with the story. The external test set is a real plus; the headline number is not self-reported on a private split.

What I'd want pinned down before believing the attribution. The text encoder is fed 'pseudo-labels and emotion-related keywords' from GPT-4 and Qwen-Omni. If those pseudo-labels are emotion categories for the same samples at test time, then the classifier is essentially reading another model's opinion, and the gain over the baseline (which sees no LLM output) can't be credited to the fusion architecture. The paper never says whether LLM enrichment is applied only to the labeled/unlabeled training data or also to the test split, and no code is released. That is a genuine soft spot, not a manufactured one. A second issue is that the final ensemble row in Table 1 has no validation number, so we can't check overfitting before the test evaluation. And there's the usual challenge-paper sloppiness: the baseline in the abstract (78.63%) differs from the test baseline in Table 1 (76.80%), and reference [22] is mis-mapped (the Ruan citation points to a salient-object paper). Minor, but it says something about polish.

The circularity concern is real but fixable: if the LLM outputs are used purely as feature extractors and never carry the predicted label into the classifier at inference, the main claim holds. The math is simple, no formal proofs to check; the empirical claim is the substance.

Who gets value: anyone working on the MER challenge or on low-label multimodal emotion recognition; the recipe is easy to rebuild and the components are standard. It deserves a serious referee, but I'd make the authors clearly state test-time LLM use and release code or at least the exact prompts and LLM outputs. If the test-time ambiguity resolves, I'd cite it as a solid systems result.

Referee Report

4 major / 4 minor

Summary. The paper describes ECMF, a multimodal emotion recognition system for the MER-SEMI track of the MER2025 challenge. The system extracts features with HuBERT-Large (audio), Chinese-RoBERTa-wwm-ext-large (text), and a dual CLIP-ViT-Large branch (video), enriches the text input with GPT-4 and Qwen-Omni outputs, fuses modalities through residual self-attention with learnable modal tokens, refines noisy training labels via a multi-source voting strategy, and finally ensembles variants by majority voting. The reported weighted F-score on the official test set is 87.49%, against an official baseline of 78.63%. The central claim is that this improvement validates the proposed fusion and label-refinement framework.

Significance. If the reported result is correctly attributed, the paper would be a useful challenge-system description and would provide evidence that the combination of pretrained encoders, attention-based fusion, and LLM-assisted text enrichment is competitive on the MER-SEMI benchmark. A strength is that the headline number is measured on the external official test set, so the main comparison is not circular. However, the significance is currently limited by unresolved questions about what exactly is fed into the text encoder at test time, by an inconsistency in the reported baseline numbers, and by the absence of repeated runs or variance estimates. The paper also does not release code, which makes it difficult to verify the load-bearing assumptions.

major comments (4)

[3.2.2] The text branch is described as augmenting the original text with GPT-4-generated 'pseudo-labels and emotion-related keywords' and Qwen-Omni-generated 'pseudo-labels, detailed video descriptions, and auxiliary emotional cues', which are then encoded together with the original text. The paper never states whether this LLM enrichment is applied to the official test set. If the same prompts are run on test samples and the pseudo-labels or emotion clues contain the predicted emotion category for the same video, the classifier is effectively reading an external model's emotion prediction. Since Table 1 separately credits 'GPT4-label' and 'GPT4-keywords' with gains, this is load-bearing: the reported 87.49% WAF could be largely an LLM-oracle effect rather than a property of the proposed fusion architecture. Please state explicitly whether LLM calls are made at test time; if they are, ensure th
[4.3] The central comparison is inconsistent. The Abstract and Introduction report an official baseline of 78.63%, but Table 1 lists the Baseline test WAF as 76.80%, and §4.3 acknowledges that this value 'is even lower than the 78.63% reported in the official benchmark paper'. The claimed improvement of about 8.9 points therefore depends on which baseline is used. Additionally, the final 'Ensemble learning' row reports only test WAF (87.49%) with no validation number, and no standard deviation or repeated-run statistics are reported for any row. Since successive ablation gains are sometimes as small as 0.05–0.14 percentage points, the current single-run experiment design does not establish that these increments are meaningful. Please report the official baseline consistently, add ensemble validation results, and, ideally, include multiple seeds or error bars.
[3.4.1] The paper motivates the work through the semi-supervised MER-SEMI setting and data scarcity, but the method description never uses the 20,000 unlabeled samples; all components consume the 7,369 labeled samples, plus external LLM knowledge. If the framework is not intended to exploit the unlabeled split, the semi-supervised framing in the Introduction and Abstract should be corrected, or the unlabeled data usage should be described. Otherwise, the comparison with the official semi-supervised baseline is not apples-to-apples and the novelty relative to a supervised system is unclear.
[3.4.1] The multi-source labeling strategy trains weak classifiers on the original training labels and then uses their predictions, along with Qwen-Omni labels, to re-label the same training set via majority voting, with manual correction of highly inconsistent cases. Because the weak classifiers are fit on the original annotations, their majority vote may reproduce or reinforce, rather than independently correct, labeling noise. The paper provides no analysis of how often the refined label differs from the original. Quantify the relabeling rate (e.g., percentage of changed labels) and show that the final model is not simply memorizing weak-classifier predictions. This concern is particularly important because the LLM that contributes to relabeling is also used in the text-feature branch.

minor comments (4)

[3.2.2] The description of GPT-4 and Qwen-Omni roles is inconsistent. §3.2.2 says GPT-4 processes text and Qwen-Omni processes audio/visual content, while §4.2 says 'GPT4-label leverages GPT-4 to analyze video content'. Please clarify which LLM produces which type of output and ensure the ablations correspond to those roles.
[4.2] The 'Fold-6' entry applies 6-fold cross-validation, whereas §4.1 states that the official baseline uses five-fold. Explain why 6 folds were chosen and how predictions from the six folds are aggregated for the test set. Also clarify whether all other ablations use the same fold configuration.
[Related Works] The text cites 'Ruan et al. [22]' for spatiotemporal video features, but reference [22] is Wang et al., 'Learning to Detect Salient Objects with Image-Level Supervision'. The citation appears mismatched and should be corrected.
[4.3] The sentence 'its test performance drops to 76.8%' uses a different number format than the table (76.80%). Minor, but please standardize numerical formatting throughout the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline WAF is measured on an external official test set, and no fitted parameter or predicted label is redefined as the target quantity by construction.

full rationale

The paper's central quantitative claim—87.49% WAF versus 78.63% on MER2025-SEMI—is a reported result on the official test/validation partitions, not a quantity defined in terms of the model's own outputs. The methodological components (pretrained encoders, attention fusion, LLM-enriched text, multi-source label refinement, ensembling) are feature-extraction and training-procedure choices. The multi-source labeling strategy in §3.4.1 uses weak classifiers and Qwen-Omni to produce refined training labels; this is a self-training/pseudo-labeling preprocessing step and does not enter the evaluation labels. The only potentially circular-looking passage is §3.2.2, where GPT-4 'generate[s] pseudo-labels and emotion-related keywords for each text sample' and Table 1 includes a '+ GPT4-label' row that uses GPT-4 emotion labels to enhance text features. If those LLM labels were applied to test samples, the classifier would be reading an external model's opinion; the paper never explicitly states whether this is done at test time. However, this is a test-time leakage/attribution ambiguity, not a circular reduction: the final WAF is still compared against fixed ground-truth labels, and no parameter fitted to the test set is renamed as a prediction. There are no load-bearing self-citations, no imported uniqueness theorems, and no equation that defines the predicted emotion in terms of itself. Under the stated circularity criteria, no step exhibits the required 'Eq. X = Eq. Y by construction' or 'fitted parameter renamed as prediction' structure, so no circularity is identified.

Assumptions & free parameters 8 free parameters · 7 assumptions · 0 invented entities

The framework rests on a stack of borrowed pretrained encoders and LLM services, plus hand-set hyperparameters and heuristic label cleaning. No new physical or mathematical entity is introduced. The central result is an empirical benchmark number, so the ledger is dominated by modeling assumptions about transferability and label reliability rather than by free parameters in a derivation.

free parameters (8)

hidden_dim = 128
Set by hand in Section 4.2 for stable training; no search or sensitivity analysis reported.
dropout_rate = 0.6
Set in Section 4.2; no tuning curve reported.
attention_heads = 2
Set in Section 4.2.
learning_rate = 5e-5
Set in Section 4.2.
training_epochs = 200
Set in Section 4.2.
gradient_clip = 1.0
Set in Section 4.2.
hubert_layers = 16-21
Selected in ablation 'Selective Hubert_Layer' (Table 1), based on validation performance.
fold_count = 6
Switched from 5-fold baseline to 6-fold in ablation 'Fold-6' (Table 1).

assumptions (7)

domain assumption Pretrained models (HuBERT-Large, Chinese-RoBERTa, CLIP-ViT-Large) transfer to MER2025 emotional video content.
Used as feature extractors in Section 3.2; no fine-tuning or domain adaptation is described.
domain assumption OpenFace reliably detects faces in all MER2025 frames.
The dual-branch visual encoder depends on facial patches (Section 3.2.3).
domain assumption HuBERT layers 16-21 carry emotion-relevant prosodic information.
Adopted from prior work [24] and validated only via one ablation row.
domain assumption Self-attention with residual connections can dynamically balance modalities and does not hurt when one modality is weak.
Fusion design in Section 3.3 is presented without a mathematical or empirical comparison to alternatives.
domain assumption LLM-generated pseudo-labels and keywords improve text features without injecting systematic errors.
The context-enriched text method (Section 3.2.2) relies on GPT-4 and Qwen-Omni outputs.
domain assumption Majority-vote label refinement produces better labels than the original annotations.
Label refinement in Section 3.4.1 uses weak classifiers and LLM votes; no independent label audit is reported.
domain assumption Official test labels and the weighted F-score metric are correct and comparable across runs.
The headline comparison assumes the official test set is fixed and correctly labeled.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ECMF: Enhanced Cross-Modal Fusion for Multimodal Emotion Recognition in MER-SEMI Challenge." pith.science (2026). https://pith.science/paper/V6ZLGJYG

@misc{pith2026250805991,
  author       = {Pith},
  title        = {Pith review of: ECMF: Enhanced Cross-Modal Fusion for Multimodal Emotion Recognition in MER-SEMI Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V6ZLGJYG}},
  note         = {Machine review of arXiv:2508.05991}
}

read the original abstract

Emotion recognition plays a vital role in enhancing human-computer interaction. In this study, we tackle the MER-SEMI challenge of the MER2025 competition by proposing a novel multimodal emotion recognition framework. To address the issue of data scarcity, we leverage large-scale pre-trained models to extract informative features from visual, audio, and textual modalities. Specifically, for the visual modality, we design a dual-branch visual encoder that captures both global frame-level features and localized facial representations. For the textual modality, we introduce a context-enriched method that employs large language models to enrich emotional cues within the input text. To effectively integrate these multimodal features, we propose a fusion strategy comprising two key components, i.e., self-attention mechanisms for dynamic modality weighting, and residual connections to preserve original representations. Beyond architectural design, we further refine noisy labels in the training set by a multi-source labeling strategy. Our approach achieves a substantial performance improvement over the official baseline on the MER2025-SEMI dataset, attaining a weighted F-score of 87.49% compared to 78.63%, thereby validating the effectiveness of the proposed framework.

Figures

Figures reproduced from arXiv: 2508.05991 by the authors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 11 canonical work pages

[22]

Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin, and Xiang Ruan. 2017. Learning to Detect Salient Objects with Image-Level Supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). IEEE, Honolulu, HI, 136–145. doi:10.1109/CVPR.2017.23

work page doi:10.1109/cvpr.2017.23 2017
[1]

Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representa- tions. In Proceedings of the 34th International Conference on Neural Information Pro- cessing Systems (NeurIPS’20). NeurIPS, Virtual. https://arxiv.org/abs/2006.11477

arXiv 2020
[2]

Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. 2016. OpenFace: An open source facial behavior analysis toolkit. In 2016 IEEE Winter Conference on Applications of Computer Vision . 1–10. doi:10.1109/WACV.2016.7477553

work page arXiv 2016
[3]

Joao Carreira and Andrew Zisserman. 2017. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17) . IEEE, Honolulu, HI, 4724–

work page 2017
[4]

MER2025 Challenge. 2025. MER2025: Multimodal Emotion Recognition Challenge . Retrieved July 22, 2025 from https://zeroqiaoba.github.io/MER2025-website/

work page 2025
[5]

Junghyun Cho and Hyungjoo Hwang. 2020. Spatio-Temporal Representation of an Electroencephalogram for Emotion Recognition Using a Three-Dimensional Convolutional Neural Network. Sensors 20, 12 (jun 2020), 3491. doi:10.3390/ s20123491

work page 2020
[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19) . Association for Computational Linguistic...

work page 2019
[7]

Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slow- Fast Networks for Video Recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’19) . IEEE, Seoul, South Korea, 6202–6211. doi:10.1109/ICCV.2019.00630

arXiv 2019

Show all 25 references

[8]

Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. HuBERT: Self- Supervised Speech Representation Learning by Masked Prediction of Hidden Units. IEEE/ACM Transactions on Audio, Speech, and Language Processi...

2021
[9]

Yu Huang, Junyang Lin, Chang Zhou, Hongxia Yang, and Longbo Huang. 2022. Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably). In Proceedings of the 39th International Conference on Machine Learning (ICML’22, Vol. 162) . PMLR...

2022 arXiv
[10]

Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. 2023. MaPLe: Multi-modal Prompt Learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’23). IEEE, Vancouver, Canada. https://arxiv.org/a...

2023 arXiv
[11]

Zheng Lian, Haoyu Chen, Lan Chen, Haiyang Sun, Licai Sun, Yong Ren, Zebang Cheng, Bin Liu, Rui Liu, Xiaojiang Peng, Jiangyan Yi, Jianhua Tao, et al . 2025. MER 2025: When Affective Computing Meets Large Language Models. arXiv preprint arXiv:2504.19423 (2025). https://arxiv.org...

2025 arXiv
[12]

Zheng Lian, Haiyang Sun, Licai Sun, Kang Chen, Mngyu Xu, Kexin Wang, Ke Xu, et al. 2023. Mer 2023: Multi-label learning, modality robustness, and semi- supervised learning. In Proceedings of the 31st ACM International Conference on Multimedia. ACM, 9610–9614

2023
[13]

Schuller, and Jianhua Tao

Zheng Lian, Haiyang Sun, Licai Sun, Zhuofan Wen, Siyuan Zhang, Shun Chen, Hao Gu, Jinming Zhao, Ziyang Ma, Xie Chen, Jiangyan Yi, Rui Liu, Kele Xu, Bin Liu, Erik Cambria, Guoying Zhao, Björn W. Schuller, and Jianhua Tao. 2024. MER 2024: Semi-Supervised Learning, Noise Robustne...

2024 arXiv
[14]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692 [cs.CL] https://arxiv.org/abs/1907.11692

2019 arXiv
[15]

Usman Malik, Simon Bernard, Alexandre Pauchet, Clément Chatelain, Romain Picot-Clémente, and Jérôme Cortinovis. 2024. Pseudo-Labeling With Large Language Models for Multi-Label Emotion Classification of French Tweets. IEEE Access 12 (2024), 15902–15916. doi:10.1109/ACCESS.2024.3354705

2024
[16]

OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774

2023 arXiv
[17]

Rosalind W. Picard. 1997. Affective Computing. MIT Press, Cambridge, MA

1997
[18]

Soujanya Poria, Erik Cambria, Rajiv Bajpai, and Amir Hussain. 2017. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion 37 (sep 2017), 98–125. doi:10.1016/j.inffus.2017.02.004

2017 doi
[19]

Anbin Qi. 2024. Multimodal Emotion Recognition with Vision-language Prompt- ing and Modality Dropout. arXiv:2409.07078 [cs.CV] https://arxiv.org/abs/2409. 07078

2024 arXiv
[20]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings...

2021 arXiv
[21]

Gomez, Lukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of the 31st International Conference on Neural Infor- mation Processing Systems (NeurIPS’17) . NeurIPS,...

2017 arXiv
[23]

Jin Xu et al. 2025. Qwen2.5-Omni Technical Report. arXiv:2503.20215 [cs.CL] https://arxiv.org/abs/2503.20215

2025 arXiv
[24]

Zhixian Zhao. 2024. Improving Multimodal Emotion Recognition by Leveraging Acoustic Adaptation and Visual Alignment. arXiv:2409.05015 [cs.CV] https: //arxiv.org/abs/2409.05015

2024 arXiv
[4733]

doi:10.1109/CVPR.2017.502

2017 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [22]

Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin, and Xiang Ruan. 2017. Learning to Detect Salient Objects with Image-Level Supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). IEEE, Honolulu, HI, 136–145. doi:10.1109/CVPR.2017.23

work page doi:10.1109/cvpr.2017.23 2017

[2] [1]

Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representa- tions. In Proceedings of the 34th International Conference on Neural Information Pro- cessing Systems (NeurIPS’20). NeurIPS, Virtual. https://arxiv.org/abs/2006.11477

arXiv 2020

[3] [2]

Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. 2016. OpenFace: An open source facial behavior analysis toolkit. In 2016 IEEE Winter Conference on Applications of Computer Vision . 1–10. doi:10.1109/WACV.2016.7477553

work page arXiv 2016

[4] [3]

Joao Carreira and Andrew Zisserman. 2017. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17) . IEEE, Honolulu, HI, 4724–

work page 2017

[5] [4]

MER2025 Challenge. 2025. MER2025: Multimodal Emotion Recognition Challenge . Retrieved July 22, 2025 from https://zeroqiaoba.github.io/MER2025-website/

work page 2025

[6] [5]

Junghyun Cho and Hyungjoo Hwang. 2020. Spatio-Temporal Representation of an Electroencephalogram for Emotion Recognition Using a Three-Dimensional Convolutional Neural Network. Sensors 20, 12 (jun 2020), 3491. doi:10.3390/ s20123491

work page 2020

[7] [6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19) . Association for Computational Linguistic...

work page 2019

[8] [7]

Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slow- Fast Networks for Video Recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’19) . IEEE, Seoul, South Korea, 6202–6211. doi:10.1109/ICCV.2019.00630

arXiv 2019