Rethinking Multimodal Sentiment Analysis: A High-Accuracy, Simplified Fusion Architecture

Nischal Mandal; Yang Li

REVIEW 4 major objections 5 minor 25 references

Rethinking Multimodal Sentiment Analysis: A High-Accuracy, Simplified Fusion Architecture

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A late-fusion dense model claims 92.55% accuracy on six-class IEMOCAP, beating attention-heavy baselines by about six points.

desk verdict The 92.55% IEMOCAP result is the whole paper, and the evaluation protocol as written does not come close to supporting it. read the letter →

arxiv 2505.04642 v1 pith:MZJADTAY submitted 2025-05-05 cs.CL cs.AI

Nischal Mandal , Yang Li This is my paper · ORCID

classification cs.CLcs.AI

keywords multimodalsentimentanalysislatefusiondenseencodersfeatureengineeringTF-IDFIEMOCAPemotionclassificationoversampling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's claim is that a deliberately simple architecture—three independent dense encoders, one for text, audio, and video, joined by plain concatenation and a single dense fusion layer—reaches 92.55% test accuracy and 92.34% weighted F1 on six-class IEMOCAP. The authors attribute the result to careful feature engineering per modality and oversampling of minority classes rather than to attention, tensor products, or pretrained transformers. A sympathetic reader would take this as evidence that 'better features, simpler fusion' can beat complex multimodal models by a wide margin on standard benchmarks and at a fraction of the compute. The paper reports a roughly six-point margin over the strongest listed baseline, and positions the architecture as suitable for resource-constrained deployment.

What carries the argument

The load-bearing mechanism is late fusion of independently encoded modality representations. Text is summarized as TF-IDF with regularization-based feature selection; audio is summarized with prosodic statistics enriched by gradient-boosted leaf embeddings; video is MoCap features concatenated with gradient-boosted classifier scores. Each modality encoder outputs a 128-dimensional vector, and concatenation followed by a 256-unit dense layer with dropout learns cross-modal interactions. Oversampling with replacement toward fixed class targets is the complementary component, and the paper credits it with balanced recall on fear and surprise.

What would settle it

Run the identical encoder under a leave-one-session-out speaker-independent protocol and check for exact feature-vector overlap between the training and test sets after oversampling. If accuracy falls near the 86.5% baseline rather than 92.55%, or duplicates are found, the reported margin is an artifact of the split.

Watch

Extended reading notes

Core claim

On IEMOCAP, after remapping the eight original emotion labels into six classes, the model encodes TF-IDF-selected text vectors, statistical audio features, and MoCap-derived video features through separate 128-unit dense encoders with dropout. The encoded streams are concatenated into $Z_{\text{fused}} = [z_{\text{audio}}; z_{\text{video}}; z_{\text{text}}]$, passed through a 256-unit ReLU layer with dropout, and classified by softmax. The reported test accuracy is 92.55%, weighted F1 is 92.34%, macro F1 is 93.05%, and macro ROC-AUC is 0.9881; these are the numbers that the paper argues show late fusion with engineered features can outperform or match attention-heavy and tensor-based alternatives.

Load-bearing premise

The result stands on the assumption that a stratified random split of IEMOCAP utterances, with oversampling applied cleanly before training, measures real generalization; the paper does not separate speakers or sessions and does not state whether oversampling happens before or after the split.

Editorial extensions

If this is right

If the reported numbers reproduce, a simple dense late-fusion baseline should become the default comparison for IEMOCAP experiments, not an upper bound that only attention-heavy models approach.
The recipe—per-modality feature engineering, class balancing, concatenation fusion—can be applied to other small multimodal emotion datasets with less GPU time than transformer training.
The six-point gap over the listed state-of-the-art transformer baseline implies that much of the reported gain in complex fusion models may come from evaluation protocol differences rather than architectural power.
The model's low parameter count and single-GPU training profile make real-time emotion-aware interfaces, such as dialogue agents or mental-health monitoring, more feasible.
The oversampling design suggests that data-balancing choices are as consequential for minority emotion classes as the fusion strategy itself.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the paper uses a stratified random utterance split rather than a speaker- or session-independent split, the honest interpretation is that 92.55% is an upper bound for a random-utterance setting; a session-split evaluation would likely lower the number and narrow the claimed margin.
A direct diagnostic for split leakage would be to compare the exact feature vectors of training and test rows after oversampling; if duplicates appear, the score is inflated by construction.
The gradient-boosted feature enrichment at the audio and video inputs is effectively a learned nonlinear preprocessing step; applying the same trick to text, or testing the architecture without it, would isolate how much of the gain is fusion versus feature engineering.
The same model on other multimodal sentiment datasets with speaker-independent splits would show whether the feature-engineering advantage transfers beyond IEMOCAP; the paper lists such datasets as future work but does not report numbers.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this one for the evaluation pitfalls, not for the result. The headline claim—92.55% test accuracy on six-class IEMOCAP with a dense late-fusion model—is not supported by the protocol as written.

The architecture is a straightforward assembly of known pieces: TF-IDF text with LASSO/RFE, handcrafted audio features boosted with XGBoost leaf embeddings, MoCap video passed through XGBoost softmax stacking, then concatenation and a few dense layers. Nothing here is new by itself, and the paper only claims novelty in the final number. To the authors' credit, they ran single-modality and early-fusion baselines, and the efficiency story is plausible. The writing is clear and the pipeline description is detailed enough to reproduce in principle. But no code or data are released.

The soft spots are structural. Section 3.6 says an 80/10/10 stratified split; Section 4.1 says an 80:20 train-test split with a further 10% validation. Those cannot both be true. More seriously, the split is utterance-level with no speaker or session separation. IEMOCAP's standard protocol is speaker-independent because same-speaker utterances are highly correlated; a random utterance split lets the model exploit speaker identity. The paper also never says whether the oversampling in Section 3.3 happens before or after the split. If before, exact duplicate rows can land in both train and test. Either issue alone would inflate the number; together they make 92.55% unreliable. The comparison against TFN, Multimodal Transformer, MAG-BERT, MMIM, and UniMSE uses published numbers from unknown protocols, so the six-point gap is not meaningful.

The reference list is also in disarray—citation [10] is called BERT but points to the Tensor Fusion Network paper, and the comparison table cites references that don't match the bibliography. That alone would need a rewrite.

The central message—simple, well-engineered fusion can compete with attention-heavy models—may be right, but this paper doesn't demonstrate it. The accuracy drop under a proper speaker-independent split is unknown, and there are no error bars or repeated runs.

I wouldn't send this to peer review in its current form. The load-bearing number isn't credible, and the fix is a new experimental campaign, not a minor revision. If the authors rerun under a proper protocol, release code and data, and fix the citations, the question is worth a second look. For now, it's a useful teaching example of why evaluation protocol matters.

Referee Report

4 major / 5 minor

Summary. The paper proposes a lightweight late-fusion model for six-class emotion classification on IEMOCAP, using TF-IDF text features, handcrafted audio features, MoCap-derived video features, per-class oversampling, XGBoost-based feature enrichment, and dense encoders followed by concatenation and a softmax layer. The central claim is that this simple architecture achieves 92.55% test accuracy and 92.34% weighted F1, outperforming TFN, Multimodal Transformer, MAG-BERT, MMIM, and UniMSE on IEMOCAP. The paper argues that strong feature engineering and modular design can outperform attention-heavy architectures at lower computational cost.

Significance. If the reported accuracy were obtained under a protocol comparable to the published baselines, the result would be noteworthy: a parameter-light, non-pretrained fusion model beating transformer-based multimodal systems by roughly six points on a standard benchmark would challenge the prevailing emphasis on architectural complexity. The manuscript is clearly written and transparent about its design choices, and it gives useful details on regularization and class balancing. However, no code or data is provided, and the central accuracy claim rests on an evaluation protocol that appears to allow speaker leakage and possibly duplicate oversampled rows across train and test. The reported comparison to prior work is therefore not yet credible, and the significance of the claimed improvement cannot be assessed from the manuscript as written.

major comments (4)

[3.6 and 4.1] The data-split description is internally contradictory. Section 3.6 states that a stratified sampling (80/10/10) split was used, while Section 4.1 states that the dataset was split into an 80:20 train-test configuration with an additional 10% of the training data set aside for validation. These protocols yield different test-set sizes and different training/validation proportions, so the reported 92.55% test accuracy cannot be reproduced or interpreted without clarification. Please specify the exact final train/validation/test sizes and the random seed used for stratification.
[3.6 and 4.1] The evaluation does not use a speaker-independent split. IEMOCAP's standard evaluation protocols are speaker-independent (e.g., leave-one-session-out or speaker-disjoint splits), because utterances from the same speaker are highly correlated in prosody, facial behavior, and lexical choices; a random utterance-level split allows the model to exploit speaker identity rather than learn generalizable multimodal sentiment. Since the paper reports only a stratified random split without any speaker or session separation, the measured accuracy is likely inflated relative to published baselines that respect speaker independence. Please rerun the experiments under a speaker-disjoint or leave-one-session-out protocol and report those results.
[3.3 and 4.3] The timing of the oversampling relative to the split is ambiguous and load-bearing. Section 3.3 says oversampling was performed per class using stratified sampling with replacement and that the final dataset exhibited a balanced distribution, while Section 4.3 refers to this balancing process without clarifying whether it is applied before or after the train/test split. If oversampling with replacement is applied before the split, exact duplicate rows can appear in both training and test sets, directly inflating test accuracy. The manuscript must state explicitly that oversampling is applied only to the training partition, and the experiments should be rerun if that was not the case.
[Table 1 and Section 5.5] The comparison with prior work is not valid because the baseline numbers are taken from papers that use different evaluation protocols, label sets, and possibly different IEMOCAP partitions. Moreover, the references in Table 1 are incorrect: entries labeled TFN, Multimodal Transformer, MAG-BERT, MMIM, and UniMSE cite reference numbers [12]–[16], which in the reference list correspond to MOSI, Wu and Xu, CoreNLP, spaCy, and the Audio Spectrogram Transformer, not to the stated models. Please provide a table in which all methods, including the proposed one, are evaluated on the same speaker-independent split and with the same label mapping, and correct the citations.

minor comments (5)

[Abstract and Section 1.4] The reported accuracy is inconsistent across the paper: the abstract says 92%, Section 1.4 says 92.5%, and Section 5.1 says 92.55%. Please unify these numbers.
[4.3] Section 4.3 refers to 'target counts defined in Section 3.2.4,' but the target counts actually appear in Section 3.3. Please fix the cross-reference.
[3.3] The emotion_mapped dictionary in Section 3.3 is not explained: the source label order and the meaning of the target class indices should be stated explicitly so that readers can map the six classes to the original IEMOCAP annotations.
[3.2] The feature engineering pipeline is not fully reproducible: the number of TF-IDF features after LASSO and RFE, the XGBoost hyperparameters, and the leaf-embedding dimensions are not reported. Please give these values or provide code.
[References] Several citations are mismatched: the BERT sentence in Section 2.1 cites [10], which is listed as the Tensor Fusion Network paper; [12] is listed as MOSI but is cited for TFN; and [23] is used for multiple distinct claims while also duplicating [5] and [13]. Please renumber and correct the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported accuracy is an empirical test-set measurement, not a quantity derived from its own inputs; remaining concerns (split protocol, oversampling timing) are validity issues, not circular reasoning.

full rationale

The paper's central claim is that a late-fusion dense model achieves 92.55% test accuracy and 92.34% weighted F1 on six-class IEMOCAP. This number is obtained by training on a train split and evaluating on a held-out test split (Sections 3.6, 4.1, 5.1); it is not derived from any equation whose terms include the target result, and no parameter is fitted to the test labels. The architecture is a plain feedforward stack with concatenation and softmax; there is no claim that any baseline number is derived from this model, and no uniqueness theorem or self-citation forces the design choice. The references are all external works, and the authors do not cite themselves. The main weakness identified by the reader — absence of speaker/session-independent splits and ambiguity about whether oversampling occurs before splitting (Sections 3.3, 3.6, 4.1, 4.3) — could inflate the measured accuracy through train/test overlap, but that is a benchmark-validity threat, not circular reasoning. Even if the evaluation protocol is flawed, the accuracy remains an empirical measurement rather than a construction that reduces to its inputs. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central accuracy claim rests on a small set of hand-chosen hyperparameters, a non-standard label mapping, an ambiguous oversampling procedure, and an unvalidated random-split evaluation protocol. None of these are derived from data or theory, and several are not fully specified in the text.

free parameters (5)

Dense encoder and fusion widths, dropout rates = text 128; audio 128; video 128-64-32; fusion 256; dropout 0.3 and 0.4
Chosen in Sections 3.4 and 3.5; no sensitivity analysis or ablation is reported, so the architectural capacity is a hand choice that affects the accuracy claim.
Oversampling target class counts = 0:2933, 1:2933, 2:5000, 3:2933, 4:2933, 5:4000
Set by hand in Section 3.3; these targets change the class distribution used for training and therefore directly determine the reported minority-class recall and overall accuracy.
Optimizer and training hyperparameters = Adam lr=0.001, batch size 64, epochs 50, early stopping patience 5, ReduceLROnPlateau factor 0.5 after 3 epochs
Given in Sections 3.6 and 4.1 without sensitivity analysis; standard values, but still hand-chosen inputs to the reported result.
TF-IDF dimensionality after LASSO and RFE
Section 3.2.1 says feature selection was applied via LASSO and RFE but never reports how many features remained; this unspecified choice affects the text encoder input.
XGBoost hyperparameters for audio leaf embeddings and video softmax stacking
Sections 3.2.2 and 3.2.3 rely on XGBoost for feature transformation but give no tree count, depth, or learning rate, making the pipeline under-specified.

assumptions (4)

domain assumption A stratified random split of utterances into train and test is a valid way to measure generalization on IEMOCAP.
Sections 3.6 and 4.1 describe random stratified splits without speaker or session separation. Standard IEMOCAP evaluation is speaker-independent; if the same actor appears in both splits, the accuracy number is inflated.
domain assumption Merging eight emotion labels into six preserves benchmark difficulty and comparability.
Section 3.1 remaps happiness/excitement and sadness/frustration into single classes. Prior results in Table 1 may use different label sets, so the comparison is not apples-to-apples.
domain assumption Oversampling with replacement is not applied in a way that leaks augmented samples into the test set.
Section 3.3 says the final dataset exhibited a balanced distribution, but it never states whether balancing was done before or after the split. If duplicates created by oversampling appear in both train and test, reported performance is biased.
domain assumption Handcrafted features and TF-IDF are sufficient for six-class emotion classification without large pretrained models.
This is the core design premise of Sections 1.3 and 3.2; the paper offers no feature-sufficiency analysis or upper-bound experiment, so the ceiling of the method is unknown.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Rethinking Multimodal Sentiment Analysis: A High-Accuracy, Simplified Fusion Architecture." pith.science (2026). https://pith.science/paper/MZJADTAY

@misc{pith2026250504642,
  author       = {Pith},
  title        = {Pith review of: Rethinking Multimodal Sentiment Analysis: A High-Accuracy, Simplified Fusion Architecture},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MZJADTAY}},
  note         = {Machine review of arXiv:2505.04642}
}

read the original abstract

Multimodal sentiment analysis, a pivotal task in affective computing, seeks to understand human emotions by integrating cues from language, audio, and visual signals. While many recent approaches leverage complex attention mechanisms and hierarchical architectures, we propose a lightweight, yet effective fusion-based deep learning model tailored for utterance-level emotion classification. Using the benchmark IEMOCAP dataset, which includes aligned text, audio-derived numeric features, and visual descriptors, we design a modality-specific encoder using fully connected layers followed by dropout regularization. The modality-specific representations are then fused using simple concatenation and passed through a dense fusion layer to capture cross-modal interactions. This streamlined architecture avoids computational overhead while preserving performance, achieving a classification accuracy of 92% across six emotion categories. Our approach demonstrates that with careful feature engineering and modular design, simpler fusion strategies can outperform or match more complex models, particularly in resource-constrained environments.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 23 canonical work pages

[10]

Tensor fusion network for multimodal sentiment analysis,

A. Zadeh, M. Chen, S. Poria, E. Cambria, and L. -P. Morency, “Tensor fusion network for multimodal sentiment analysis,” in Proc. EMNLP, 2017, pp. 1103–1114

work page 2017
[16]

Audio Spectrogram Transformer: Transformer architecture for audio data representation,

S. Vasudevan, S. Das, and R. Mittal, “Audio Spectrogram Transformer: Transformer architecture for audio data representation,” in Proc. ICASSP, 2021

work page 2021
[1]

A review of affective computing: From unimodal analysis to multimodal fusion,

S. Poria, E. Cambria, R. Bajpai, and A. Hussain, “A review of affective computing: From unimodal analysis to multimodal fusion,” IEEE Trans. Cogn. Dev. Syst., vol. 9, no. 4, pp. 453–465, Dec. 2017

work page 2017
[2]

IEMOCAP: Interactive emotional dyadic motion capture database,

C. Busso, M. Bulut, C. C. Lee, A. Kazemzadeh, E. Mower, S. Kim, et al., “IEMOCAP: Interactive emotional dyadic motion capture database,” Lang. Resour. Eval., vol. 42, no. 4, pp. 335–359, Dec. 2008

work page 2008
[3]

Memory fusion network for multi-view sequential learning,

A. Zadeh, R. Zellers, E. Pincus, and L. P. Morency, “Memory fusion network for multi-view sequential learning,” in Proc. AAAI, 2018, pp. 5634 – 5641

work page 2018
[4]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., “Attention is all you need,” in Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 5998–6008

work page 2017
[5]

On the efficiency of multimodal architectures: A case study in sentiment analysis,

Z. Wu and C. Xu, “On the efficiency of multimodal architectures: A case study in sentiment analysis,” in Proc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), 2020

work page 2020
[6]

Multimodal transformer for unaligned multimodal language sequences,

Y. -H. H. Tsai, S. Bai, P. P. Liang, J. Z. Kolter, L. -P. Morency, and R. Salakhutdinov, “Multimodal transformer for unaligned multimodal language sequences,” in Proc. ACL, 2019, pp. 6558–6569

work page 2019

Show all 25 references

[7]

Opinion mining and sentiment analysis,

B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Foundations and Trends in Information Retrieval, vol. 2, no. 1 –2, pp. 1 –135, 2008

2008
[8]

IEMOCAP: Interactive emotional dyadic motion capture database,

C. Busso et al., “IEMOCAP: Interactive emotional dyadic motion capture database,” Language Resources and Evaluation, vol. 42, no. 4, pp. 335 –359, 2008

2008
[9]

Domain adaptation for large -scale sentiment classification: A deep learning approach,

X. Glorot, A. Bordes, and Y. Bengio, “Domain adaptation for large -scale sentiment classification: A deep learning approach,” in Proc. ICML, 2011, pp. 513–520

2011
[11]

Multimodal transformer for unaligned multimodal language sequences,

Y.-H. H. Tsai et al., “Multimodal transformer for unaligned multimodal language sequences,” in Proc. ACL, 2019, pp. 6558–6569

2019
[14]

The Stanford CoreNLP natural language processing toolkit,

C. D. Manning et al., “The Stanford CoreNLP natural language processing toolkit,” in Proc. ACL System Demonstrations, 2014, pp. 55–60

2014
[15]

spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing,

M. Honnibal and I. Montani, “spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing,” To appear, 2017

2017
[17]

Speech emotion recognition combining acoustic features and linguistic information in a hybrid SVM - Bayesian network architecture,

B. Schuller, G. Rigoll, and M. Lang, “Speech emotion recognition combining acoustic features and linguistic information in a hybrid SVM - Bayesian network architecture,” in Proc. ICASSP, 2004

2004
[18]

Modality -to-modality translation: Adversarial representation learning and graph fusion network,

S. Mai, H. Hu, and S. Xing, “Modality -to-modality translation: Adversarial representation learning and graph fusion network,” in Proc. AAAI, 2020, pp. 164–172

2020
[19]

M -SENA: An integrated platform for multimodal sentiment analysis,

H. Mao et al., “M -SENA: An integrated platform for multimodal sentiment analysis,” in Proc. ACL: System Demonstrations, 2022, pp. 204 – 213

2022
[20]

MOSI: Multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos,

A. Zadeh et al., “MOSI: Multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos,” arXiv preprint arXiv:1606.06259, 2016

2016 arXiv
[21]

DialogueTRM: Exploring multimodal emotional dynamics in conversation,

Y. Mao et al., “DialogueTRM: Exploring multimodal emotional dynamics in conversation,” in Proc. EMNLP, 2021, pp. 2694–2704

2021
[22]

LXMERT: Learning cross -modality encoder representations,

H. Tan and M. Bansal, “LXMERT: Learning cross -modality encoder representations,” in Proc. EMNLP-IJCNLP, 2019, pp. 5103–5114

2019
[23]

On the efficiency of multimodal architectures: A case study in sentiment analysis,

Z. Wu and C. Xu, “On the efficiency of multimodal architectures: A case study in sentiment analysis,” in Proc. EMNLP, 2020

2020
[24]

ScaleVLAD: Improving multimodal sentiment analysis via multi-scale fusion of local descriptors,

H. Luo et al., “ScaleVLAD: Improving multimodal sentiment analysis via multi-scale fusion of local descriptors,” arXiv preprint arXiv:2112.01368, 2021

2021 arXiv
[25]

SentiXRL: An advanced large language model framework for fine -grained emotion classification,

J. Wang et al., “SentiXRL: An advanced large language model framework for fine -grained emotion classification,” arXiv preprint arXiv:2411.18162, 2024

2024 arXiv
[26]

Tensor Fusion Network for multimodal sentiment analysis,

A. Zadeh et al., “Tensor Fusion Network for multimodal sentiment analysis,” in Proc. EMNLP, 2017, pp. 1103–1114

2017
[27]

Multimodal deep learning,

J. Ngiam et al., “Multimodal deep learning,” in Proc. ICML, 2011, pp. 689–696

2011

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [10]

Tensor fusion network for multimodal sentiment analysis,

A. Zadeh, M. Chen, S. Poria, E. Cambria, and L. -P. Morency, “Tensor fusion network for multimodal sentiment analysis,” in Proc. EMNLP, 2017, pp. 1103–1114

work page 2017

[2] [16]

Audio Spectrogram Transformer: Transformer architecture for audio data representation,

S. Vasudevan, S. Das, and R. Mittal, “Audio Spectrogram Transformer: Transformer architecture for audio data representation,” in Proc. ICASSP, 2021

work page 2021

[3] [1]

A review of affective computing: From unimodal analysis to multimodal fusion,

S. Poria, E. Cambria, R. Bajpai, and A. Hussain, “A review of affective computing: From unimodal analysis to multimodal fusion,” IEEE Trans. Cogn. Dev. Syst., vol. 9, no. 4, pp. 453–465, Dec. 2017

work page 2017

[4] [2]

IEMOCAP: Interactive emotional dyadic motion capture database,

C. Busso, M. Bulut, C. C. Lee, A. Kazemzadeh, E. Mower, S. Kim, et al., “IEMOCAP: Interactive emotional dyadic motion capture database,” Lang. Resour. Eval., vol. 42, no. 4, pp. 335–359, Dec. 2008

work page 2008

[5] [3]

Memory fusion network for multi-view sequential learning,

A. Zadeh, R. Zellers, E. Pincus, and L. P. Morency, “Memory fusion network for multi-view sequential learning,” in Proc. AAAI, 2018, pp. 5634 – 5641

work page 2018

[6] [4]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., “Attention is all you need,” in Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 5998–6008

work page 2017

[7] [5]

On the efficiency of multimodal architectures: A case study in sentiment analysis,

Z. Wu and C. Xu, “On the efficiency of multimodal architectures: A case study in sentiment analysis,” in Proc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), 2020

work page 2020

[8] [6]

Multimodal transformer for unaligned multimodal language sequences,

Y. -H. H. Tsai, S. Bai, P. P. Liang, J. Z. Kolter, L. -P. Morency, and R. Salakhutdinov, “Multimodal transformer for unaligned multimodal language sequences,” in Proc. ACL, 2019, pp. 6558–6569

work page 2019