Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that modeling each modality as a Student's $t$-distribution and distilling the fused uncertainty from a complete-modality teacher to a student makes multimodal sentiment analysis robust to missing modalities…

desk verdict Plausible core idea undermined by duplicated text from UMDF and an uncertainty score that can be undefined. read the letter →

arxiv 2506.01490 v1 pith:IJVSGTW4 submitted 2025-06-02 cs.LG

Yanxi Luo , Shijin Wang , Zhongxing Xu , Yulong Li , Feilong Tang , Jionglong Su This is my paper
classification cs.LG
keywords confidence-awareself-distillationmultimodalsentimentanalysisincompletemodalitiesStudent'st-distributionuncertaintyestimationknowledgedistillationmissingmodalityrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a training strategy, Confidence-Aware Self-Distillation (CASD), that keeps a multimodal sentiment model reliable when language, audio, or visual input is missing. Instead of forcing the available modalities into one fixed vector, CASD represents each modality as a Student's $t$-distribution, fuses those distributions using per-modality confidence weights, and estimates an uncertainty score for the fused representation. A teacher network trained on complete data supervises the student through both logits distillation and distillation of that uncertainty score, while a reparameterization module samples from the fused distribution during training. The paper claims this configuration achieves state-of-the-art results on MOSI, MOSEI, and IEMOCAP and that CASD works as a plug-in improving existing systems such as UMDF and CorrKD. The broader claim is that confidence-aware distributional embeddings, rather than reconstruction or common-subspace projection, are the right way to handle uncertain modality missingness.

What carries the argument

The machinery is a mixture-of-Student's-$t$ multimodal embedding with a closed-form fusion rule. Each modality outputs Normal-Inverse-Gamma prior parameters $(\gamma_m, \delta_m, \alpha_m, \beta_m)$, giving a predictive Student's $t$ distribution $\mathrm{St}(z_m; \gamma_m, \beta_m(1+\delta_m)/(\delta_m \alpha_m), 2\alpha_m)$. The fusion step sets the fused degrees of freedom to $v_F = \min(v_1, v_2, v_3)$, fuses the means with confidence weights built from the individual $v$'s, and averages the scale matrices with heavy-tail corrections. From the fused distribution, Eq.~9 computes the uncertainty score $U_F = \Sigma_F v_F/(v_F - 3)$, and Eq.~10 distills this score from teacher to student by mean squared error. A reparameterization trick samples the student's training embedding as $s = u_F + \sigma_F \cdot t$ with $t \sim \mathrm{St}(v_F)$, which makes sampling differentiable and weakens the requirement that inference representations of the same class point in the same direction.

What would settle it

Log $v_F$ on every training mini-batch: if any batch has fused degrees of freedom $v_F \leq 3$, then the uncertainty score $U_F = \Sigma_F v_F/(v_F - 3)$ is undefined or negative, so the distillation signal cannot have been computed as written. Alternatively, ablate Eq.~9 and replace it with a simple mean of per-modality aleatoric uncertainties: if accuracy is unchanged, the $v_F$-dependent uncertainty score is not the active ingredient.

Watch

Extended reading notes

Core claim

The central claim is that confidence-aware distributional embeddings, rather than deterministic concatenation, make knowledge distillation robust to missing modalities. Each modality is modeled through an evidential Normal-Inverse-Gamma prior that yields a Student's $t$ predictive distribution; the three modality distributions are fused into one Student's $t$ distribution by confidence weights, and the fused distribution's scale, scaled by $v_F/(v_F - 3)$, becomes the uncertainty score. The teacher network, trained on complete data, supervises the student through both logits distillation and a mean-squared-error distillation of this uncertainty. A reparameterization representation module samples embeddings from the fused distribution during training, which the paper argues relaxes the implicit directional constraint that forces same-class samples to share one representation direction and lets each modality combination keep its own geometry. The paper reports that this yields state-of-the-art accuracy under both intra-modality and inter-modality missingness on three benchmarks.

Load-bearing premise

The load-bearing premise is that the fused Student's $t$ distribution always has degrees of freedom $v_F > 3$ and that the borrowed fusion formula gives a meaningful confidence score, so the uncertainty distillation target $U_F = \Sigma_F v_F/(v_F - 3)$ is well-defined and informative.

Editorial extensions

If this is right

  • If the central claim is right, missing-modality sentiment systems do not need to reconstruct lost audio or video; instead, they can carry a calibrated uncertainty estimate about what is missing and distill that from a complete-modality teacher.
  • Because CASD is presented as a plug-in, the same uncertainty distillation term can be added to existing frameworks such as UMDF and CorrKD, and the reported tables show consistent average F1 gains across six missing-modality settings.
  • The ablation results imply each component contributes separately: confidence-aware fusion adds the largest average missing-modality gain, uncertainty distillation adds robustness under complete and incomplete inputs, and reparameterized sampling adds a further gain by relaxing representation constraints.
  • The directional-constraint argument predicts that the scale parameter $\sigma_F$ controls the trade-off: larger sampling variance should increase robustness to missingness, and setting $\sigma_F = 0$ should degenerate into the deterministic subspace-style methods.
  • The method is framed as dataset-agnostic, so the same teacher-student uncertainty distillation should transfer to other multimodal classification tasks beyond sentiment and emotion recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run is to replace the fusion rule in Eq.~7 with alternatives, such as a geometric mean of the degrees of freedom or data-weighted confidence weights, to isolate whether the min rule or the confidence weighting carries the gain.
  • The condition $v_F > 3$ suggests a practical guardrail: clamp or reparameterize the evidential head outputs so the fused degrees of freedom always exceed 3, since otherwise Eq.~9 is undefined during early training even if the final trained model avoids that regime.
  • The directional-constraint claim can be probed directly by measuring the cosine similarity of inference embeddings for same-class samples across different modality-drop conditions; CASD should reduce that similarity relative to deterministic baselines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Confidence-Aware Self-Distillation (CASD), a teacher--student framework for multimodal sentiment analysis under incomplete modalities. The method converts each modality embedding into a probabilistic representation via a Normal-Inverse-Gamma evidential prior, treats the predictive distribution of each modality as a Student's t distribution, fuses the three modality distributions using a closed-form approximation, and defines a scalar uncertainty score for the fused distribution. During training, the student is trained to match the teacher's fused uncertainty through a mean-squared-error loss, and a reparameterization module samples from the fused distribution to compute the task loss. Experiments on MOSI, MOSEI, and IEMOCAP report consistent improvements over several state-of-the-art baselines, including plug-in gains when CASD is added to UMDF and CorrKD.

Significance. If the proposed uncertainty-aware self-distillation mechanism were sound, the work would be a useful contribution to missing-modality multimodal learning: the plug-in experiments with UMDF and CorrKD, the comparison against seven baselines, and the component ablations are a reasonable evaluation design. However, the central uncertainty score is mathematically ill-defined unless a constraint on the degrees of freedom holds, and no such constraint is stated or enforced. The fusion rule also contains inconsistent weights and is attributed to a reference that does not appear to contain the rule. In addition, the submitted manuscript contains large verbatim blocks from a prior AAAI paper (UMDF, Li et al., AAAI 2024), including repeated framework descriptions and figure captions. Because the central training loss can be undefined or semantically meaningless during training, and because the empirical SOTA claim rests on unaccompanied point estimates, the paper as submitted is not acceptable.

major comments (4)
  1. [Section II-C, Eqs. (5), (7), (9), (10)] The uncertainty distillation loss L_UF in Eq. (10) depends on U_F = Sigma_F * v_F / (v_F - 3) from Eq. (9), which is undefined when v_F = 3 and negative when v_F < 3. Since v_m = 2*alpha_m by Eq. (5) and v_F = min(v1,v2,v3) by Eq. (7), any modality with alpha_m <= 1.5 makes the distillation target invalid. The paper never constrains the evidential head to guarantee alpha_m > 1.5; the standard softplus-plus-one construction only guarantees alpha_m > 1. Thus the central training signal can be undefined or semantically meaningless at any training step, and the reported results cannot be reproduced from the description as written.
  2. [Section II-C, Eqs. (7), (8)] The fusion rule is internally inconsistent: C1 and C2 use denominator (v1+v2) while C3 uses denominator (v1+v2+v3), so C1+C2+C3 = 1 + v3/(v1+v2+v3) > 1, meaning the fused mean u_F is not a convex combination of the modality means. The paper states that the update follows reference [18], but [18] is a Student's t filter paper and no demonstration is given that it contains this rule. Since the fused distribution parameters feed directly into the uncertainty score and the reparameterization sampling, this unverified and inconsistent fusion step is load-bearing.
  3. [Section III-B, Tables I and II] The state-of-the-art claim, including the highlighted +2.45% average F1 over GCNet on MOSI, is supported only by single point estimates. No error bars, repeated-seed standard deviations, or statistical significance tests are reported, and several differences between baselines are small relative to typical run-to-run variation in this setting. The SOTA claim is therefore not adequately supported as presented.
  4. [Entire manuscript (notably Section II and the figure captions)] The submitted text contains large verbatim blocks from the UMDF paper (Li et al., AAAI 2024), including the repeated framework description beginning with 'Given a multimodal video segment with three modalities', the repeated Figure 1 caption describing MCIM, coarse-grained/fine-grained CA, and DFIM, and the corresponding related-work paragraphs. This material is not presented as quoted or cited, and it makes it impossible to determine the boundary between the authors' original contribution and copied prior work. This is a serious integrity issue regardless of the technical content.
minor comments (5)
  1. [Table II] The column order appears inconsistent with the header: for example, the CubeMLP 'Happy' row shows 89.0 before 62.9, but the six missing-modality entries average to 62.85, not 89.0, indicating that the 'Avg.' and '{l,a,v}' columns are likely swapped relative to the caption.
  2. [Section III-C, Table III] The text and table use 'PPM' for the reparameterization module, while the rest of the paper calls it 'RRM'; the same abbreviation inconsistency appears in the ablations discussion.
  3. [Section II-E, Eq. (14)] Equation (14) has an unbalanced parenthesis in the definition of the Jensen-Shannon divergence, and the notation M for the average distribution is undefined until after the equation.
  4. [Section II-E, Eq. (15)] The hyperparameters alpha and beta in the total loss are never given values, nor is a sensitivity analysis provided, so the empirical results cannot be reproduced without additional information.
  5. [Section II-C, Eqs. (6) and (9)] The notation for Sigma_F is inconsistent: Eq. (6) defines it as a vector of Sigma1, Sigma2, Sigma3, but Eq. (9) treats it as a scalar in the uncertainty score; the distinction between u_F and U_F is also easy to confuse and should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular step: the SOTA claim is grounded in held-out benchmark metrics and the uncertainty distillation target is an internal training objective, not a fitted prediction.

full rationale

The paper's central empirical claim is stated in the abstract and supported by Tables I and II, which report F1/MAE on held-out MOSI, MOSEI, and IEMOCAP test conditions. These external benchmarks do not use U_F as a fitted predictor, so the performance claim is not derived from the uncertainty score by construction. Equation 9 defines U_F = Σ_F · v_F/(v_F − 3) as an internal statistic of the fused Student's t distribution, and Eq. 10 uses it only as a self-distillation target that matches teacher and student values; this is a training objective, not a prediction of an external quantity. The fusion rule in Eq. 7 is attributed to the external citation [18], not to the authors' own prior work, and no load-bearing argument reduces to a self-citation. The v_F > 3 requirement for U_F to be finite and positive is a technical validity concern, but it does not make the derivation circular: even if the target is undefined for some training steps, the reported benchmark numbers are not constructed from U_F. I therefore find no circular step that would warrant a score above 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The central claim rests on two fitted loss weights that are not reported, an unverified fusion approximation, an ad hoc uncertainty formula that requires an unconstrained v_F > 3, and two model-derived quantities (confidence weights and uncertainty score) that have no external validation. These are the main places where the reader is asked to accept a choice without independent evidence.

free parameters (2)
  • alpha (weight of L_logits) = not reported
    Eq. 15 defines the total loss with hand-set weights alpha and beta; no values or tuning procedure are given, so the reported results cannot be reproduced from the text.
  • beta (weight of L_UF) = not reported
    Table III shows that adding the uncertainty distillation term changes the averaged F1 by +0.65 points, but the actual weight value is not disclosed.
assumptions (4)
  • standard math NIG prior with Gaussian likelihood yields a Student's t predictive distribution (Eq. 5).
    Standard Bayesian conjugacy result; acceptable as background.
  • domain assumption The fused distribution of three per-modality Student's t marginals is again approximately a Student's t distribution with parameters updated by Eq. 7.
    The paper cites [18] but gives no derivation; the confidence weights in Eq. 8 are inconsistent (C3 uses a different denominator), so the approximation is not transparent.
  • ad hoc to paper U_F = Sigma_F * v_F / (v_F - 3) is a valid scalar uncertainty score for the fused distribution.
    This formula requires v_F > 3, which the network does not constrain; standard Student's t covariance is v/(v-2) Sigma for v > 2, not v/(v-3), so the measure is introduced ad hoc.
  • standard math Reparameterized sampling s = u_F + sigma_F * t with t ~ St(v_F) is differentiable and preserves gradients.
    Standard reparameterization trick, though the paper cites a non-canonical source; acceptable.
invented entities (2)
  • Modality confidence weight C_m
    purpose: Used to fuse modality means and represent each modality's confidence.
    It is a definition from the network's own degrees of freedom; no calibration or external evidence shows that higher v means higher confidence, and Eq. 8 is internally inconsistent.
  • Fused uncertainty score U_F
    purpose: Serves as the distillation target L_UF to transfer uncertainty from teacher to student.
    Computed entirely from parameters output by the same networks being trained; there is no independent uncertainty benchmark, so the score is self-referential.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities." pith.science (2026). https://pith.science/paper/IJVSGTW4

@misc{pith2026250601490,
  author       = {Pith},
  title        = {Pith review of: Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IJVSGTW4}},
  note         = {Machine review of arXiv:2506.01490}
}
abstract

Multimodal sentiment analysis (MSA) aims to understand human sentiment through multimodal data. In real-world scenarios, practical factors often lead to uncertain modality missingness. Existing methods for handling modality missingness are based on data reconstruction or common subspace projections. However, these methods neglect the confidence in multimodal combinations and impose constraints on intra-class representation, hindering the capture of modality-specific information and resulting in suboptimal performance. To address these challenges, we propose a Confidence-Aware Self-Distillation (CASD) strategy that effectively incorporates multimodal probabilistic embeddings via a mixture of Student's $t$-distributions, enhancing its robustness by incorporating confidence and accommodating heavy-tailed properties. This strategy estimates joint distributions with uncertainty scores and reduces uncertainty in the student network by consistency distillation. Furthermore, we introduce a reparameterization representation module that facilitates CASD in robust multimodal learning by sampling embeddings from the joint distribution for the prediction module to calculate the task loss. As a result, the directional constraint from the loss minimization is alleviated by the sampled representation. Experimental results on three benchmark datasets demonstrate that our method achieves state-of-the-art performance.

Figures

Figures reproduced from arXiv: 2506.01490 by the authors.

Figure 1
Figure 1. Under the missing-modality setting, traditional methods [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Comparison of MAE and F1 scores at various missing [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dynamic Interaction-Aware and Causality-Disentangled Framework for Multimodal Sentiment Analysis

    cs.MM 2026-05 unverdicted novelty 5.0 of 10

    MCAF reports state-of-the-art Acc-2/F1 on CMU-MOSI (86.52/86.51) and CMU-MOSEI (86.72/86.65), but its diffusion-denoising module is never specified in the methods.

Reference graph

Works this paper leans on

30 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [18]

    A student’s t filter for heavy tailed process and measurement noise,

    Michael Roth, Emre ¨Ozkan, and Fredrik Gustafsson, “A student’s t filter for heavy tailed process and measurement noise,” in ICASSP, 2013

    work page 2013

  2. [1]

    Sentiment analysis algorithms and applications: A survey,

    Walaa Medhat, Ahmed Hassan, and Hoda Korashy, “Sentiment analysis algorithms and applications: A survey,” Ain Shams engineering journal , vol. 5, no. 4, pp. 1093–1113, 2014

    work page 2014

  3. [2]

    Decoding the Flow: CauseMotion for Emotional Causality Analysis in Long-form Conversations

    Yuxuan Zhang, Yulong Li, Zichen Yu, Feilong Tang, Zhixiang Lu, Chong Li, Kang Dang, and Jionglong Su, “Decoding the flow: Causemo- tion for emotional causality analysis in long-form conversations,” arXiv preprint arXiv:2501.00778, 2025

    work page Pith review arXiv 2025

  4. [3]

    Duat: Dual-aggregation trans- former network for medical image segmentation,

    Feilong Tang, Zhongxing Xu, Qiming Huang, Jinfeng Wang, Xianxu Hou, Jionglong Su, and Jingxin Liu, “Duat: Dual-aggregation trans- former network for medical image segmentation,” in PRCV, 2023

    work page 2023

  5. [4]

    Fedmmr: Multi-modal federated learning via missing modality reconstruction,

    Shu Wang, Zhe Qu, Yuan Liu, Shichao Kan, Yixiong Liang, and Jianxin Wang, “Fedmmr: Multi-modal federated learning via missing modality reconstruction,” in ICME, 2024

    work page 2024

  6. [5]

    Joint training strategy of unimodal and multimodal for multimodal sentiment analysis,

    Meng Li, Zhenfang Zhu, Kefeng Li, Lihua Zhou, Zhen Zhao, and Hongli Pei, “Joint training strategy of unimodal and multimodal for multimodal sentiment analysis,” Image and Vision Computing , p. 105172, 2024

    work page 2024

  7. [6]

    Beyond words: Auralllm and signmst-c for precise sign language production and bidirectional accessibility,

    Yulong Li, Yuxuan Zhang, Feilong Tang, Mian Zhou, Zhixiang Lu, Haochen Xue, Yifang Wang, Kang Dang, and Jionglong Su, “Beyond words: Auralllm and signmst-c for precise sign language production and bidirectional accessibility,” arXiv preprint arXiv:2501.00765 , 2025

    arXiv 2025

  8. [7]

    A unified self-distillation framework for multimodal sentiment analysis with uncertain missing modalities,

    Mingcheng Li, Dingkang Yang, Yuxuan Lei, Shunli Wang, Shuaibing Wang, Liuzhen Su, Kun Yang, Yuzheng Wang, Mingyang Sun, and Lihua Zhang, “A unified self-distillation framework for multimodal sentiment analysis with uncertain missing modalities,” in AAAI, 2024

    work page 2024

Show all 30 references
  1. [8]

    Correlation-decoupled knowledge distillation for multimodal sentiment analysis with incomplete modalities,

    Mingcheng Li, Dingkang Yang, Xiao Zhao, Shuaibing Wang, Yan Wang, Kun Yang, Mingyang Sun, Dongliang Kou, Ziyun Qian, and Lihua Zhang, “Correlation-decoupled knowledge distillation for multimodal sentiment analysis with incomplete modalities,” in CVPR, 2024

    2024

  2. [9]

    KD-MSLRT: Lightweight sign language recognition model based on mediapipe and 3d to 1d knowledge dis- tillation,

    Yulong Li, Bolin Ren, Ke Hu, Changyuan Liu, Zhengyong Jiang, Kang Dang, and Jionglong Su, “KD-MSLRT: Lightweight sign language recognition model based on mediapipe and 3d to 1d knowledge dis- tillation,” AAAI, 2025

    2025

  3. [10]

    Modality translation-based multimodal sentiment analysis under uncertain missing modalities,

    Zhizhong Liu, Bin Zhou, Dianhui Chu, Yuhang Sun, and Lingqiang Meng, “Modality translation-based multimodal sentiment analysis under uncertain missing modalities,” Information Fusion, 2024

    2024

  4. [11]

    LEEGNet: Lightweight eeg sleep stage classification network with knowledge distillation,

    Boqian Wang, Yulong Li, Chao Zhang, Angelos Stefanidis, Mian Zhou, and Jionglong Su, “LEEGNet: Lightweight eeg sleep stage classification network with knowledge distillation,” in PRML. IEEE, 2024

    2024

  5. [12]

    Polyp-mamba: Polyp segmentation with visual mamba,

    Zhongxing Xu, Feilong Tang, Zhe Chen, Zheng Zhou, Weishan Wu, Yuyao Yang, Yu Liang, Jiyu Jiang, Xuyue Cai, and Jionglong Su, “Polyp-mamba: Polyp segmentation with visual mamba,” in MICCAI 2024, 2024

    2024

  6. [13]

    Stepwise feature fusion: Local guides global,

    Jinfeng Wang, Qiming Huang, Feilong Tang, Jia Meng, Jionglong Su, and Sifan Song, “Stepwise feature fusion: Local guides global,” in MICCAI. Springer, 2022

    2022

  7. [14]

    Sfc: Shared feature calibration in weakly supervised semantic segmentation,

    Xinqiao Zhao, Feilong Tang, Xiaoyang Wang, and Jimin Xiao, “Sfc: Shared feature calibration in weakly supervised semantic segmentation,” in AAAI, 2024

    2024

  8. [15]

    Deep evidential regression,

    Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus, “Deep evidential regression,” NeurIPS, 2020

    2020

  9. [16]

    Data uncertainty learning in face recognition,

    Jie Chang, Zhonghao Lan, Changmao Cheng, and Yichen Wei, “Data uncertainty learning in face recognition,” in CVPR, 2020

    2020

  10. [17]

    Probabilistic face embeddings,

    Yichun Shi and Anil K Jain, “Probabilistic face embeddings,” in ICCV, 2019

    2019

  11. [19]

    Smoothing the geometry of probabilistic box embeddings,

    Xiang Li, Luke Vilnis, Dongxu Zhang, Michael Boratko, and Andrew McCallum, “Smoothing the geometry of probabilistic box embeddings,” in ICLR, 2018

    2018

  12. [20]

    Mosi: multimodal corpus of sentiment intensity and subjectivity anal- ysis in online opinion videos,

    Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency, “Mosi: multimodal corpus of sentiment intensity and subjectivity anal- ysis in online opinion videos,” arXiv preprint arXiv:1606.06259 , 2016

    2016 arXiv

  13. [21]

    Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph,

    AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency, “Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph,” in ACL, 2018

    2018

  14. [22]

    Iemocap: Interactive emotional dyadic motion capture database,

    Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan, “Iemocap: Interactive emotional dyadic motion capture database,” Language resources and evaluation , 2008

    2008

  15. [23]

    Words can shift: Dynamically adjusting word representations using nonverbal behaviors,

    Yansen Wang, Ying Shen, Zhun Liu, Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency, “Words can shift: Dynamically adjusting word representations using nonverbal behaviors,” in AAAI, 2019

    2019

  16. [24]

    Learning modality- specific representations with self-supervised multi-task learning for multimodal sentiment analysis,

    Wenmeng Yu, Hua Xu, Ziqi Yuan, and Jiele Wu, “Learning modality- specific representations with self-supervised multi-task learning for multimodal sentiment analysis,” in AAAI, 2021

    2021

  17. [25]

    Cubemlp: An mlp-based model for multimodal sentiment analysis and depression estimation,

    Hao Sun, Hongyi Wang, Jiaqing Liu, Yen-Wei Chen, and Lanfen Lin, “Cubemlp: An mlp-based model for multimodal sentiment analysis and depression estimation,” in ACMMM, 2022

    2022

  18. [26]

    Decoupled multimodal distilling for emotion recognition,

    Yong Li, Yuanzhi Wang, and Zhen Cui, “Decoupled multimodal distilling for emotion recognition,” in CVPR, 2023

    2023

  19. [27]

    Towards multimodal sentiment analysis debiasing via bias purification,

    Dingkang Yang, Mingcheng Li, Dongling Xiao, Yang Liu, Kun Yang, Zhaoyu Chen, Yuzheng Wang, Peng Zhai, Ke Li, and Lihua Zhang, “Towards multimodal sentiment analysis debiasing via bias purification,” ECCV, 2024

    2024

  20. [28]

    Smil: Multimodal learning with severely missing modality,

    Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng, “Smil: Multimodal learning with severely missing modality,” in AAAI, 2021

    2021

  21. [29]

    Gc- net: Graph completion network for incomplete multimodal learning in conversation,

    Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao, “Gc- net: Graph completion network for incomplete multimodal learning in conversation,” TPAMI, 2023

    2023

  22. [30]

    Probabilistic embeddings for cross-modal retrieval,

    Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio De Rezende, Yannis Kalantidis, and Diane Larlus, “Probabilistic embeddings for cross-modal retrieval,” in CVPR, 2021

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.