Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

RoHyDR: Robust Hybrid Diffusion Recovery for Incomplete Multimodal Emotion Recognition

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RoHyDR claims that missing audio, text, and visual modalities can be recovered by a diffusion generator at the unimodal level and an adversarial fusion refiner at the multimodal level, outperforming prior incomplete multimodal emotion…

desk verdict Solid empirical IMER paper that is worth refereeing despite missing representation-level validation and a data-table inconsistency. read the letter →

arxiv 2505.17501 v1 pith:LTW4SE6X submitted 2025-05-23 cs.CV cs.AIcs.LG

Yuehan Jin , Xiaoqing Liu , Yiyuan Yang , Zhiwen Yu , Tong Zhang , Kaixiang Yang This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords incompletemultimodalemotionrecognitionmissingmodalityrecoverydiffusionmodeladversariallearningfusionmulti-stageoptimizationconditionedgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that incomplete multimodal emotion recognition can be made robust by recovering missing information at two levels: unimodal feature representations and multimodal fused semantics. It proposes RoHyDR, a hybrid framework in which a diffusion generator conditioned on available modalities reconstructs missing audio, text, or vision representations, and an adversarial discriminator guides the fused representation toward the one a complete input would have produced. On the MOSI and MOSEI benchmarks, under both random missing rates up to 70% and modality-specific availability patterns, the method reports higher ACC2, ACC7, and F1 than the compared representation-learning and generative baselines, with the largest gains at high missing rates, up to about 6.5 points on MOSI and 6.8 points on MOSEI. A three-stage optimization is presented as the mechanism that keeps the generative, adversarial, and classification objectives stable enough to realize these gains.

What carries the argument

The load-bearing machinery is a hybrid two-module recovery stack. Module one is a high-dimensional diffusion model (HDDM) paired with a residual unimodal reconstructor: starting from Gaussian noise, the HDDM performs reverse diffusion with a conditional Vision Transformer denoiser, using available modalities as cross-attention keys and values to generate distribution-consistent, semantically aligned missing representations, which the reconstructor then refines toward the ground-truth representation. Module two is adversarial multimodal fusion recovery: a shared fusion network produces fused representations from complete and recovered inputs, a residual multimodal reconstructor refines the recovered fused representation, and a multi-layer-perceptron discriminator attempts to tell it apart from the complete-input fused representation, with the reconstructor and fusion network trained to fool the discriminator. A three-stage optimization updates the generator only, then the adversarial module, then the classifier within each epoch, which the paper credits with reducing gradient interference.

What would settle it

At a high missing rate, replace the generated missing representations with random Gaussian vectors of the same shape and scale while keeping the classifier and fusion network unchanged; if accuracy stays near the reported values, the gains are coming from the available modalities and the classifier rather than from semantically meaningful recovery.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that missing unimodal representations and the missing semantic content of a multimodal fusion can be recovered jointly, and that this joint recovery is what pushes incomplete-input emotion recognition closer to complete-input performance. A high-dimensional diffusion model, denoised by a conditional Vision Transformer that cross-attends to available modalities, generates each missing modality from Gaussian noise, and a residual unimodal reconstructor refines it; separately, the fusion network's recovered representation is refined by a multimodal reconstructor and judged by a discriminator that tries to distinguish it from the complete-input fused representation. The paper argues that the diffusion stage preserves modality-specific distributions while the adversarial stage preserves multimodal semantics, and that neither alone is sufficient. Experimental results on MOSI and MOSEI are offered as support, with the largest accuracy advantages appearing at missing rates of 0.6 and 0.7.

Load-bearing premise

The load-bearing assumption is that the generated missing-modality representations are semantically aligned with the real missing data and that the discriminator's realism signal is an emotion-relevant semantic signal, neither of which is directly measured.

Editorial extensions

If this is right

  • At a 70% missing rate on MOSI, RoHyDR's ACC2 declines by about 10.4 points, while the compared baselines decline by 13.8 to 15.3 points, so the method is claimed to tolerate severe sensor loss without collapsing.
  • On MOSEI, the reported accuracy advantage over the best generative baseline reaches 6.8 points at a 70% missing rate, indicating that the biggest relative gains occur exactly when recovery is hardest.
  • Under modality-specific availability, the largest gains appear when only audio or only vision is available, suggesting the diffusion stage recovers non-text modalities more effectively than prior single-level generators.
  • The three-stage optimization is reported to yield smoother training curves and lower classifier loss than one- and two-stage variants, implying the decomposition itself is a source of the performance gain, not just the generative components.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper evaluates only final classification metrics, so an implicit claim is that feature-level and semantic-level alignment are what produce those metrics; an editorial next step is to measure cosine similarity or probe-classifier accuracy on generated representations to separate recovery quality from classifier robustness.
  • If the recovery modules are responsible for the gains, the same hybrid recipe should transfer to other tasks with missing inputs, such as multimodal sentiment regression or speaker state estimation, which the paper does not test.
  • The three-stage curriculum suggests a general schedule for incomplete-input models: reconstruct at the signal level first, align fused representations adversarially, then train the task head; this ordering may matter more than the specific losses when modality incompleteness is severe.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. RoHyDR addresses incomplete multimodal emotion recognition by combining a diffusion-based unimodal generator conditioned on available modalities, an adversarial multimodal fusion recovery module, and a three-stage optimization procedure. The manuscript claims state-of-the-art performance on CMU-MOSI and CMU-MOSEI under both random missing-modality rates and modality-specific availability patterns, with particularly large gains at high missing rates. The architecture and ablations are described in detail, and the paper reports consistent directional improvements over the chosen baselines across nearly all settings.

Significance. If the reported results are reliable, the proposed hybrid recovery and multi-stage optimization are useful contributions to IMER. The paper provides a fairly complete architecture, ablates each major module, and demonstrates consistent directional improvements across two benchmarks and all tested missing rates. However, the strength of the evidence is currently limited by an internal baseline inconsistency, the absence of uncertainty quantification, and the lack of direct validation of the semantic-alignment mechanism claimed in the abstract. These issues need to be resolved before the central claim of state-of-the-art performance can be accepted.

major comments (4)
  1. [Tables 1 and 2, MOSEI MMIN row] The same experimental setting appears twice with different numbers: Table 1 reports MMIN at missing rate 0.0 as 84.71/84.11 on MOSEI, while Table 2 reports MMIN under the {a,t,v} available condition as 70.90/69.08. These two entries describe the same all-modalities-available condition and must coincide; the discrepancy suggests that the baseline numbers are unreliable. Since the paper's headline claim is superiority over these baselines, this inconsistency is load-bearing and must be resolved, for example by reproducing the baselines and reporting corrected numbers or by explaining a protocol difference that I do not see in the text.
  2. [Section 3.2.1, Tables 1 and 2] The text states that all experiments are repeated five times, but Tables 1 and 2 report only single ACC2/F1 values with no standard deviation, confidence interval, or significance test. Several claimed advantages are small at low missing rates, for example 1.29–2.00 points on MOSI at missing rate 0.0, so the reader cannot determine whether the differences are reproducible. Please report mean ± standard deviation over the five runs and, preferably, paired significance tests against the baselines.
  3. [Tables 3 and 4] The ablation study removes HDDM/UR and D/MR, but the '×' conditions are not defined: it is unclear whether the missing modality is zero-filled, replaced by Gaussian noise, handled by a simple linear projection, or treated in some other way. Without this specification, the ablation results are not reproducible and their interpretation is ambiguous. Please state exactly what input is used when each component is ablated, and consider adding a stronger baseline such as zero-filling or random-noise substitution for the missing modality in the same framework.
  4. [Abstract; Sections 2.2.1, 2.2.2, and 2.3 (Eqs. 9 and 12)] The paper claims that the diffusion generator produces semantically aligned representations and that adversarial learning recovers semantic content, but no representation-level evidence is provided. Final classification accuracy alone cannot distinguish a model that benefits from semantically informative recovered features from one that mainly exploits the Stage 3 loss LC2 on ground-truth fused representations fMgt, since the classifier is trained on complete fused representations at every step. I recommend adding direct evaluation of the recovered unimodal and fused representations against complete-input representations (e.g., feature similarity, downstream probing, or a variant that removes LC2), together with an analysis of what the discriminator learns. Without such evidence, the recovery modules are not shown to be load-bearing for the reported gains.
minor comments (5)
  1. [Section 3.1] The dataset description writes 'COV AREP'; this should be 'COVAREP' as in reference [3].
  2. [Equation (8) and following sentence] Equation (8) defines the weight as λal, but the sentence immediately after introduces λadv; the notation should be made consistent.
  3. [Section 3.3.2] This section is titled 'Qualitative Analysis' but contains hyperparameter sensitivity plots; consider renaming it to 'Hyperparameter Analysis' to match its content.
  4. [Figure 3(c-d)] The vertical axis for the classifier loss is not labeled in panels (c) and (d), and it is not clear which loss (LC1, LC2, or LS3) is being plotted; please add axis labels and specify the quantity.
  5. [Section 3.1 and Experimental Setup] The abstract states that code will be made publicly available upon acceptance; for reproducibility, the paper should also specify the exact feature extractor versions and hyperparameters used for the baselines, and state whether the baseline numbers are taken from prior papers or re-implemented.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: RoHyDR's claims are empirical, benchmarked on held-out test splits against external baselines, and no prediction reduces to a fitted parameter or self-citation chain.

full rationale

The paper's central claim is that RoHyDR outperforms state-of-the-art IMER methods under missing-modality scenarios, supported by ACC2/F1 on held-out test portions of CMU-MOSI and CMU-MOSEI. The model's training losses (L_HDDM, L_UR, L_rec, L_adv, L_C1, L_C2) supervise generated representations against ground-truth missing modalities and labels; no reported accuracy is defined in terms of a value used to fit a parameter, and no 'prediction' is a renamed fit. The diffusion objective is imported from DDPM [6], an external and independently established result, not an author self-citation, so no load-bearing step reduces to prior work by the same authors. The multi-stage optimization and ablations are empirical comparisons, and while the ablation tables do not specify what 'absent' means for a component, that is an experimental-reporting limitation, not circularity. The apparent internal inconsistency in Table 2, where MMIN under {a,t,v} is listed as 70.90/69.08 while Table 1 lists MMIN at missing rate 0.0 as 84.71/84.11 for the same setting, raises a concern about baseline reliability, but correctness of copied baseline numbers is outside the circularity axis. No evidence shows that any derivation is equivalent to its own inputs by construction. Therefore the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim relies on several unproved background assumptions from the deep learning literature, most importantly that conditional diffusion can produce useful missing-modality features in this feature space and that adversarial training in the fused representation space aligns emotion-relevant semantics. These are standard assumptions in this research program but are not verified with direct metrics here. The free parameters listed are the hand-chosen loss weights and unstated diffusion hyperparameters that the reported performance depends on.

free parameters (5)
  • lambda_g (generative loss weight) = 1.0 for both datasets
    Balances L_HDDM and L_UR in Stage 1 (Eq. 10); chosen by hyperparameter analysis (Figure 4a,d).
  • lambda_al (adversarial loss weight) = 0.5 in main results; 0.6 (MOSI) and 0.8 (MOSEI) found optimal in Figure 4b,e
    Balances L_adv and L_rec in Eq. 8; value affects the contribution of semantic alignment via the discriminator.
  • lambda_c (classifier loss weight) = 0.4 (MOSI), 0.6 (MOSEI)
    Balances L_C1 and L_C2 in Eq. 9; tuned per dataset via Figure 4c,f.
  • Diffusion schedule parameters gamma_t, delta_t = not reported
    Defines the forward and reverse diffusion process in Eq. 2; values are chosen by hand but not given, making reproduction difficult.
  • Number of reverse diffusion steps = not reported
    The reverse process iterates n steps from Gaussian noise; n is a hand-chosen architectural hyperparameter not specified in the paper.
assumptions (5)
  • domain assumption Conditional diffusion models can generate semantically aligned missing-modality representations from Gaussian noise
    Assumed in Section 2.2.1; the whole unimodal recovery module relies on this. It is not proven for this feature space and is not directly evaluated.
  • domain assumption Adversarial training in the fused representation space aligns recovered representations with complete ones in a way that improves emotion classification
    Assumed in Section 2.2.2; the discriminator's notion of realism is taken to correspond to emotion-relevant semantics, but no direct evaluation of representation alignment is provided.
  • domain assumption The multimodal transformer fusion network of [24] is suitable for fusing recovered and available representations
    Adopted from prior work without re-derivation in Section 2.2.2.
  • domain assumption COVAREP, Facet, and BERT features are sufficient fixed representations for the emotion recognition task
    Standard preprocessing used in Section 3.1; the paper does not question the feature extractors.
  • domain assumption The DDPM objective (Eq. 3), trained on ground-truth missing features, generalizes to test-time generation when only noise and available modalities are provided
    Training uses X_miss ground truth; at test time only available modalities and noise are given. The gap between these two settings is assumed to be small.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RoHyDR: Robust Hybrid Diffusion Recovery for Incomplete Multimodal Emotion Recognition." pith.science (2026). https://pith.science/paper/LTW4SE6X

@misc{pith2026250517501,
  author       = {Pith},
  title        = {Pith review of: RoHyDR: Robust Hybrid Diffusion Recovery for Incomplete Multimodal Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LTW4SE6X}},
  note         = {Machine review of arXiv:2505.17501}
}
read the original abstract

Multimodal emotion recognition analyzes emotions by combining data from multiple sources. However, real-world noise or sensor failures often cause missing or corrupted data, creating the Incomplete Multimodal Emotion Recognition (IMER) challenge. In this paper, we propose Robust Hybrid Diffusion Recovery (RoHyDR), a novel framework that performs missing-modality recovery at unimodal, multimodal, feature, and semantic levels. For unimodal representation recovery of missing modalities, RoHyDR exploits a diffusion-based generator to generate distribution-consistent and semantically aligned representations from Gaussian noise, using available modalities as conditioning. For multimodal fusion recovery, we introduce adversarial learning to produce a realistic fused multimodal representation and recover missing semantic content. We further propose a multi-stage optimization strategy that enhances training stability and efficiency. In contrast to previous work, the hybrid diffusion and adversarial learning-based recovery mechanism in RoHyDR allows recovery of missing information in both unimodal representation and multimodal fusion, at both feature and semantic levels, effectively mitigating performance degradation caused by suboptimal optimization. Comprehensive experiments conducted on two widely used multimodal emotion recognition benchmarks demonstrate that our proposed method outperforms state-of-the-art IMER methods, achieving robust recognition performance under various missing-modality scenarios. Our code will be made publicly available upon acceptance.

Figures

Figures reproduced from arXiv: 2505.17501 by the authors.

Figure 1
Figure 1. (a) Previous generative model-based methods typically employ a single-level autoencoder [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of the proposed RoHyDR. In the context of IMER, multimodal data [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visualization of training stability and efficiency. (a–d) illustrate the comparison between [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Hyper-parameter analysis. (a) and (d) analyze the effect of [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. General Incomplete Multimodal Learning via Dynamic Quality Perception

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A unified multimodal learning framework models modality degradation as a continuous variable and uses a noise-aware quality estimator to adaptively weight fused representations under both intra- and inter-modality mis...

Reference graph

Works this paper leans on

31 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    Affective computing and sentiment analysis

    Erik Cambria, Dipankar Das, Sivaji Bandyopadhyay, and Antonio Feraco. Affective computing and sentiment analysis. InA Practical Guide to Sentiment Analysis, pages 1–10. Springer, 2017

    work page 2017

  2. [2]

    Evolving semantic prototype improves generative zero-shot learning

    Shiming Chen, Wenjin Hou, Ziming Hong, Xiaohan Ding, Yibing Song, Xinge You, Tongliang Liu, and Kun Zhang. Evolving semantic prototype improves generative zero-shot learning. In International Conference on Machine Learning, pages 4611–4622. PMLR, 2023

    work page 2023

  3. [3]

    Covarep—a collaborative voice analysis repository for speech technologies

    Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. Covarep—a collaborative voice analysis repository for speech technologies. In2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 960–964. IEEE, 2014

    work page 2014

  4. [4]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, 2019

    work page 2019

  5. [5]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. InAdvances in Neural Information Processing Systems, volume 27, 2014

    2014

  6. [6]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840–6851, 2020

    2020

  7. [7]

    Facial expression analysis

    iMotions. Facial expression analysis. https://imotions.com/products/imotions-lab/ modules/fea-facial-expression-analysis/, 2017. Accessed: 2025-05-03

    work page 2017

  8. [8]

    Kirchner, Stephen H

    Elsa A. Kirchner, Stephen H. Fairclough, and Frank Kirchner. Embedded multimodal inter- faces in robotics: Applications, future trends, and societal implications. InThe Handbook of Multimodal-Multisensor Interfaces: Language Processing, Software, Commercialization, and Emerging Directions - Volume 3, pages 523–576. ACM, 2019. 10

    work page 2019

Show all 31 references
  1. [9]

    Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Jeannette Bohg

    Michelle A. Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Jeannette Bohg. Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks. In2019 International Conference on Roboti...

    2019

  2. [10]

    Gcnet: Graph completion network for incomplete multimodal learning in conversation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. Gcnet: Graph completion network for incomplete multimodal learning in conversation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  3. [11]

    Modality translation- based multimodal sentiment analysis under uncertain missing modalities.Information Fusion, 101:101973, 2024

    Zhizhong Liu, Bin Zhou, Dianhui Chu, Yuhang Sun, and Lingqiang Meng. Modality translation- based multimodal sentiment analysis under uncertain missing modalities.Information Fusion, 101:101973, 2024

    2024

  4. [12]

    Sijie Mai, Haifeng Hu, and Songlong Xing. Modality to modality translation: An adversarial representation learning and graph fusion network for multimodal fusion.Proceedings of the AAAI Conference on Artificial Intelligence, 34(01):164–172, 2020

    2020

  5. [13]

    Score-based generative models are provably robust: An uncertainty quantification perspective.Advances in Neural Information Processing Systems, 37:63154–63183, 2024

    Nikiforos Mimikos-Stamatopoulos, Benjamin Zhang, and Markos Katsoulakis. Score-based generative models are provably robust: An uncertainty quantification perspective.Advances in Neural Information Processing Systems, 37:63154–63183, 2024

    2024

  6. [14]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205. IEEE, 2023

    2023

  7. [15]

    A review of affective computing: From unimodal analysis to multimodal fusion.Information Fusion, 37:98–125, 2017

    Soujanya Poria, Erik Cambria, Rajiv Bajpai, and Amir Hussain. A review of affective computing: From unimodal analysis to multimodal fusion.Information Fusion, 37:98–125, 2017

    2017

  8. [16]

    Beneath the tip of the iceberg: Current challenges and new directions in sentiment analysis research.IEEE Transactions on Affective Computing, 2020

    Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, and Rada Mihalcea. Beneath the tip of the iceberg: Current challenges and new directions in sentiment analysis research.IEEE Transactions on Affective Computing, 2020

    2020

  9. [17]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022

    2022

  10. [18]

    Uncertainty-aware deep classifiers using generative models.Proceedings of the AAAI Conference on Artificial Intelli- gence, 34(04):5620–5627, 2020

    Murat Sensoy, Lance Kaplan, Federico Cerutti, and Maryam Saleki. Uncertainty-aware deep classifiers using generative models.Proceedings of the AAAI Conference on Artificial Intelli- gence, 34(04):5620–5627, 2020

    2020

  11. [19]

    A survey of multimodal sentiment analysis.Image and Vision Computing, 65:3–14, 2017

    Mohammad Soleymani, Daniele Garcia, Brendan Jou, Björn Schuller, Shih-Fu Chang, and Maja Pantic. A survey of multimodal sentiment analysis.Image and Vision Computing, 65:3–14, 2017

    2017

  12. [20]

    Martinez, Nithin Kumar, Hanchuan Adam, and Shrikanth Narayanan

    Kushal Somandepalli, Tanaya Guha, Victor R. Martinez, Nithin Kumar, Hanchuan Adam, and Shrikanth Narayanan. Computational media intelligence: Human-centered machine analysis of media.Proceedings of the IEEE, 109(5):891–910, 2021

    2021

  13. [21]

    The multimodal sentiment analysis in car reviews (muse-car) dataset: Collection, insights and improvements.IEEE Transactions on Affective Computing, 2021

    Leonhard Stappen, Alice Baird, Lukas Schumann, and Stefan Bjorn. The multimodal sentiment analysis in car reviews (muse-car) dataset: Collection, insights and improvements.IEEE Transactions on Affective Computing, 2021

    2021

  14. [22]

    Modality-invariant temporal repre- sentation learning for multimodal sentiment classification.Information Fusion, 91:504–514, 2023

    Hao Sun, Jiaqing Liu, Yen-Wei Chen, and Lanfen Lin. Modality-invariant temporal repre- sentation learning for multimodal sentiment classification.Information Fusion, 91:504–514, 2023

    2023

  15. [23]

    Sheng, Dianhui Chu, Jian Yu, and Hongxiang Sun

    Yuhang Sun, Zhizhong Liu, Quan Z. Sheng, Dianhui Chu, Jian Yu, and Hongxiang Sun. Similar modality completion-based multimodal sentiment analysis under uncertain missing modalities. Information Fusion, 110:102454, 2024

    2024

  16. [24]

    Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov

    Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. Multimodal transformer for unaligned multimodal language sequences. InProceedings of the 2019 Conference of the Association for Computational Linguistics (ACL), ...

    2019

  17. [25]

    Distribution-consistent modal recovering for incomplete multimodal learning

    Yuan Wang, Zhen Cui, and Yong Li. Distribution-consistent modal recovering for incomplete multimodal learning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 22025–22034, 2023

    2023

  18. [26]

    Incomplete multimodality-diffused emotion recognition

    Yuan Wang, Yong Li, and Zhen Cui. Incomplete multimodality-diffused emotion recognition. Advances in Neural Information Processing Systems, 36, 2024

    2024

  19. [27]

    Noise imitation based adversarial training for robust multimodal sentiment analysis.IEEE Transactions on Multimedia, 26:529–539, 2023

    Ziqi Yuan, Yihe Liu, Hua Xu, and Kai Gao. Noise imitation based adversarial training for robust multimodal sentiment analysis.IEEE Transactions on Multimedia, 26:529–539, 2023

    2023

  20. [28]

    Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages.IEEE Intelligent Systems, 31(6):82–88, 2016

    Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency. Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages.IEEE Intelligent Systems, 31(6):82–88, 2016

    2016

  21. [29]

    Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph

    AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. InProceedings of the 56th Annual Meeting of the Association for Computational Ling...

    2018

  22. [30]

    Tag-assisted multimodal sentiment analysis under uncertain missing modalities

    Jiandian Zeng, Tianyi Liu, and Jiantao Zhou. Tag-assisted multimodal sentiment analysis under uncertain missing modalities. InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1545–1554, 2022

    2022

  23. [31]

    Missing modality imagination network for emotion recognition with uncertain missing modalities

    Jinming Zhao, Ruichen Li, and Qin Jin. Missing modality imagination network for emotion recognition with uncertain missing modalities. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natur...

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.