Pith. sign in

REVIEW 6 major objections 6 minor 31 references

Hybrid CNN-Mamba Enhancement Network for Robust Multimodal Sentiment Analysis

T0 review · 6 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that HCMEN, a hybrid CNN-Mamba network, outperforms every compared method on all reported metrics on the MOSI and MOSEI datasets under missing-modality conditions.

desk verdict The architecture is coherent and worth a look, but the paper's central 'outperforms all metrics' claim is contradicted by its own Table I and needs correction before the experiments can be taken at face value. read the letter →

arxiv 2507.23444 v1 pith:CFUVMCPZ submitted 2025-07-31 cs.MM

Xiang Li , Xianfu Cheng , Xiaoming Zhang , Zhoujun Li This is my paper
classification cs.MM
keywords multimodalsentimentanalysismissingmodalitiesCNN-Mambahybridnetworkstatespacemodelscross-modalalignmentmutualinformationmaximizationmix-upfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal sentiment analysis aims to read emotion from text, video, and audio, and real inputs often lose one of those channels. HCMEN is a hybrid CNN-Mamba network built so that sentiment predictions stay accurate when a modality is missing or corrupted. The paper's central claim is that HCMEN beats all compared methods on every metric on both CMU-MOSI and CMU-MOSEI, with the best average binary F1 scores (74.78 and 78.11) and the best MAE and correlation on MOSEI. The authors attribute this to three components: hierarchical local-to-global unimodal modeling, mutual-information-based cross-modal enhancement and token-level alignment, and interleaved multimodal mix-up fusion. They also report roughly 60 percent lower parameter overhead than a Transformer baseline.

What carries the argument

The load-bearing mechanism is the interleaved multimodal token sequence. After a CTC-based temporal alignment block maps text, visual, and acoustic features into a common token grid of length $L$, each visual and acoustic token is replaced by its corresponding text token with probability $p > p^*$ (Eq. 5), then projected through a modality-specific MLP to form a proxy representation. An InfoNCE contrastive loss maximizes token-level mutual information between each proxy and the text embedding, while average cosine similarity (Eq. 7) measures alignment. The enhanced tokens are interleaved as $[E_v^1, U_t^1, E_a^1, \ldots, E_v^L, U_t^L, E_a^L]$ (Eq. 9) and fed through stacked fusion blocks, each applying LayerNorm, a 1D convolution for local patterns, and a Bi-Mamba layer for long-range dependencies. Mamba, a selective state space model, gives linear-time sequence modeling, which keeps the whole fusion process efficient.

What would settle it

Run the released HCMEN code on the MOSI test split under the exact random-masking protocol and hyperparameters reported in the paper, and compare Acc-2 and F1 against LNLN; if the reported leads of roughly 2.9 points in Acc-2 and 3.1 points in F1 do not reproduce, the central performance claim would be contradicted.

Watch

Extended reading notes

Core claim

The central discovery is that a hybrid CNN-Mamba design with token-level cross-modal alignment can handle missing-modality sentiment analysis in a single network. HCMEN combines one-dimensional convolutions with bidirectional Mamba state-space layers and a cross-modal enhancement module that generates proxy visual and acoustic tokens from mixed text tokens, then aligns them to text with an InfoNCE loss. The paper reports that this design reaches 34.37 Acc-7, 38.12 Acc-5, 74.79/73.50 Acc-2, and 74.78/73.41 F1 on CMU-MOSI with MAE 1.034 and correlation 0.546, and 46.17 Acc-7, 46.92 Acc-5, 78.14/78.30 Acc-2, and 78.11/76.93 F1 on CMU-MOSEI with MAE 0.662 and correlation 0.599, beating every compared baseline on every metric. An ablation on MOSI shows that removing the cross-modal enhancement module produces the largest F1 drop and removing Mamba produces the largest accuracy drop, so both global state-space modeling and token-level alignment are load-bearing.

Load-bearing premise

The CTC-based temporal alignment is assumed to put corresponding text, visual, and acoustic tokens at the same grid positions, because the interleaved fusion and the token-level contrastive loss have no built-in way to recover when tokens are misaligned.

Editorial extensions

If this is right

  • If the reported results hold, HCMEN becomes the current best-published method for multimodal sentiment analysis under missing-modality conditions, including on binary accuracy and F1.
  • The roughly 60 percent parameter reduction relative to a Transformer baseline means the same fusion design can scale to longer utterances at lower compute cost.
  • The ablation pattern implies that future robust fusion systems should prioritize token-level cross-modal alignment and global sequence modeling over larger unimodal encoders.
  • The CNN-Mamba fusion block is a reusable backbone for other multimodal tasks currently built on attention, with linear instead of quadratic sequence-time complexity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Alignment quality is not directly measured in the paper; if the CTC token correspondences are imperfect, replacing CTC with a learned monotonic aligner is a natural extension that could raise the reported gains further.
  • The text-to-visual and text-to-audio token substitution acts as a text-guided data augmentation, so the probability threshold $p^*$ and the masking distribution are testable knobs that could tune the robustness-performance trade-off.
  • The same interleaving recipe may transfer to other sequence-aligned multimodal problems, such as audio-visual speech recognition or emotion recognition with missing channels, where token-level alignment is similarly the bottleneck.
  • A stress test with structured missingness, such as deleting an entire modality per utterance rather than the paper's random token masking, would show how well the robustness claim holds under realistic partial inputs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The manuscript proposes HCMEN, a hybrid CNN-Mamba network for multimodal sentiment analysis under missing-modality conditions. Three components are introduced: hierarchical unimodal modeling with Conv1D and Bi-Mamba, a cross-modal enhancement and alignment module (CMEA) that mixes text tokens into visual and acoustic features and applies an InfoNCE loss, and a multimodal mix-up fusion stage that interleaves tokens into a 3L-length sequence processed by stacked CNN-Mamba blocks. The method is evaluated on CMU-MOSI and CMU-MOSEI using pre-extracted features, with results reported in Table I and an ablation study in Table II. The abstract, introduction, and Section III-B claim that HCMEN consistently outperforms state-of-the-art methods on all metrics on both datasets.

Significance. If the empirical claims were fully supported, HCMEN would be a meaningful contribution to robust multimodal sentiment analysis: it addresses a practical problem (missing modalities), combines local and global modeling in a linear-complexity backbone, and proposes a contrastive token-level alignment objective. The architectural description is reasonably complete at the equation level, and the ablation design directly tests the three claimed components. However, the paper's central empirical claim is contradicted by its own Table I, and the experimental section omits crucial protocol details (missing-modality simulation, meaning of paired metrics, hyperparameters, standard deviations). As it stands, the evidence is insufficient to establish the claimed superiority, and the manuscript needs substantial revision before the contribution can be assessed.

major comments (6)
  1. [Section III-B and Table I] The claim that HCMEN "consistently outperforms state-of-the-art methods across all metrics on both the MOSI and MOSEI datasets" is false as stated on the paper's own numbers. On MOSEI, CENET reports Acc-7 = 47.18 and Acc-5 = 47.83, while HCMEN reports 46.17 and 46.92; in the F1 column, LNLN has a second value of 79.95 while HCMEN has 76.93. The text should be revised to report exact wins/losses, and the table should be corrected or annotated so that the reader can verify the comparison.
  2. [Section II-C and Section III-A] The missing-modality simulation is not specified. The paper states only that random masking or substitution is applied "following the approach in LNLN [24]", but it does not state the masking ratios, which modality subsets are tested, whether substitution uses learned or random replacements, or how many random seeds are averaged. Since the entire evaluation is under missing-modality conditions, this omission makes Table I irreproducible and prevents interpretation of the reported gains.
  3. [Table I] The Acc-2 and F1 columns contain slash-separated pairs (e.g., 74.79 / 73.50 for HCMEN on MOSI), but the meaning of the two values is never defined. The text should state whether these correspond to different binary-threshold conventions, different test protocols, or something else, and should specify which value is used in the main comparisons.
  4. [Sections II-E, II-G, and III-A] Key hyperparameters are not reported: alpha in Eq. (14), tau in Eq. (8), the threshold p* in Eq. (5), the CNN kernel sizes and channel counts, the Mamba hidden size, the number of fusion blocks, and the batch size. In addition, no standard deviations, confidence intervals, or significance tests are given for any result in Table I or Table II. Without these, the observed differences cannot be distinguished from noise.
  5. [Section III-D] The efficiency claim of "approximately 60%" parameter reduction relative to a Transformer baseline is not supported by any measurement. No parameter counts, FLOPs, or runtime numbers are reported, and the comparison configuration is not described. Since efficiency is advertised as a contribution, this claim needs a concrete experimental backing.
  6. [Section II-F and Section III] The token-level interleaving in Eq. (9) depends on the correctness of the CTC-based alignment, but the paper does not evaluate alignment quality or its sensitivity to missing inputs. If the alignment is inaccurate for corrupted modalities, the interleaved sequence will combine mismatched tokens, which could explain performance differences for reasons unrelated to the proposed fusion mechanism. An alignment-quality analysis or a sensitivity experiment would strengthen the claim.
minor comments (6)
  1. [Equation (5)] The condition "with probability p > p*" is unclear because p* is never defined; it is presumably a threshold hyperparameter, but the text should state its value or how it is chosen.
  2. [Equation (8)] The displayed InfoNCE loss contains a typographical artifact in the denominator ("Í"), and the alignment of the fraction is hard to read; the formula should be typeset cleanly.
  3. [Table II] The table header has a typo: "MOdel" should be "Model".
  4. [Section III-A] Acc-2 is usually reported with both "zero" and "non-zero" thresholds in the MSA literature; the paper should state explicitly which convention is used, since this affects comparability with prior work.
  5. [Abstract and Section I] The phrase "the first hybrid CNN-Mamba architecture" is a strong novelty claim; given the existence of other Mamba-based multimodal fusion works cited in the paper, the claim should be qualified (e.g., "to our knowledge") and supported by a discussion of related work.
  6. [Section III-C] The ablation study is only performed on MOSI and only reports three metrics; adding MOSEI results and standard deviations would make the component analysis more convincing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HCMEN is evaluated on external benchmarks and its components are trained objectives, not fitted inputs renamed as predictions.

full rationale

The paper's derivation chain is self-contained with respect to circularity concerns. The proposed model (HCMEN) is trained on standard external benchmark datasets (CMU-MOSI and CMU-MOSEI) and evaluated against published baselines; no parameter is fitted to the test set used to report the headline results. The cross-modal enhancement uses an InfoNCE contrastive loss and the sentiment head uses MSE, both of which are training objectives rather than constants derived from the reported metrics, so the reported outcomes are not forced by construction. The CTC-based alignment is taken from prior external work [9], and the Bi-Mamba baseline is similarly cited as an external architectural choice; neither citation is invoked to justify the empirical superiority claim. The self-citation to TF-Mamba [22] appears only as contextual related-work background and is not load-bearing in the derivation of HCMEN's reported performance. A separate correctness concern exists: the paper's claim in Section III-B that HCMEN 'consistently outperforms state-of-the-art methods across all metrics' is contradicted by its own Table I on MOSEI (e.g., CENET reports higher Acc-7 and Acc-5 than HCMEN, and LNLN reports a higher second F1 value), but this is an internal-evidence inconsistency and a correctness risk, not a circularity of the kind where a prediction reduces to its input by definition or self-citation. Therefore, the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard experimental assumptions (pre-extracted features, CTC alignment, missing-modality simulation). No new physical entities are introduced. The free parameters are common training hyperparameters, but their absence from the paper makes the results hard to reproduce.

free parameters (4)
  • alpha (trade-off weight)
    Controls the balance between sentiment MSE and InfoNCE alignment loss in Eq. (14); value not reported.
  • tau (InfoNCE temperature)
    Temperature in the contrastive loss in Eq. (8); value not reported.
  • p (token mix-up probability)
    Probability in Eq. (5) that a visual/acoustic token is replaced by a text token; threshold p* and value not specified.
  • model dimensions and layer counts (CNN filters, Mamba hidden size, number of fusion blocks)
    Architectural hyperparameters not reported in the paper.
assumptions (5)
  • domain assumption Pre-extracted unimodal features from CMU-MOSI and CMU-MOSEI are reliable and sufficient for sentiment prediction.
    The method consumes pre-extracted features without end-to-end finetuning; the paper cites prior work for this setup.
  • domain assumption CTC-based alignment creates valid token-level correspondences across text, visual, and acoustic modalities.
    Used in Section II-C to align sequences before interleaving in Eq. (9); poor alignment would break the token-level fusion.
  • domain assumption Random masking or substitution of inputs adequately simulates real-world missing modalities.
    Section II-C states it follows LNLN [24], but no specific missing ratio or distribution is given in this paper.
  • standard math InfoNCE loss is a valid surrogate for mutual information maximization in this setting.
    Standard contrastive learning assumption used in Eq. (8).
  • domain assumption Bidirectional Mamba effectively models long-range dependencies with linear complexity.
    Relies on the selective state space model from [18,19] and Bi-Mamba [23] as the backbone.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hybrid CNN-Mamba Enhancement Network for Robust Multimodal Sentiment Analysis." pith.science (2026). https://pith.science/paper/CFUVMCPZ

@misc{pith2026250723444,
  author       = {Pith},
  title        = {Pith review of: Hybrid CNN-Mamba Enhancement Network for Robust Multimodal Sentiment Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CFUVMCPZ}},
  note         = {Machine review of arXiv:2507.23444}
}
read the original abstract

Multimodal Sentiment Analysis (MSA) with missing modalities has recently attracted increasing attention. Although existing research mainly focuses on designing complex model architectures to handle incomplete data, it still faces significant challenges in effectively aligning and fusing multimodal information. In this paper, we propose a novel framework called the Hybrid CNN-Mamba Enhancement Network (HCMEN) for robust multimodal sentiment analysis under missing modality conditions. HCMEN is designed around three key components: (1) hierarchical unimodal modeling, (2) cross-modal enhancement and alignment, and (3) multimodal mix-up fusion. First, HCMEN integrates the strengths of Convolutional Neural Network (CNN) for capturing local details and the Mamba architecture for modeling global contextual dependencies across different modalities. Furthermore, grounded in the principle of Mutual Information Maximization, we introduce a cross-modal enhancement mechanism that generates proxy modalities from mixed token-level representations and learns fine-grained token-level correspondences between modalities. The enhanced unimodal features are then fused and passed through the CNN-Mamba backbone, enabling local-to-global cross-modal interaction and comprehensive multimodal integration. Extensive experiments on two benchmark MSA datasets demonstrate that HCMEN consistently outperforms existing state-of-the-art methods, achieving superior performance across various missing modality scenarios. The code will be released publicly in the near future.

Figures

Figures reproduced from arXiv: 2507.23444 by the authors.

Figure 1
Figure 1. Architecture of the Hybrid CNN-Mamba Enhancement Network. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 29 canonical work pages

  1. [24]

    Towards robust multimodal sentiment analysis with incomplete data,

    Haoyu Zhang, Wenbin Wang, and Tianshu Yu, “Towards robust multimodal sentiment analysis with incomplete data,” Advances in Neural Information Processing Systems, vol. 37, pp. 55943–55974, 2024

    work page 2024

  2. [1]

    Disentangled representation learning for multimodal emotion recognition,

    Dingkang Yang, Shuai Huang, Haopeng Kuang, Yangtao Du, and Lihua Zhang, “Disentangled representation learning for multimodal emotion recognition,” in Proceedings of the 30th ACM international conference on multimedia, 2022, pp. 1642–1651

    work page 2022

  3. [2]

    Hybrid contrastive learning of tri-modal representation for multimodal sentiment analysis,

    Sijie Mai, Ying Zeng, Shuangjia Zheng, and Haifeng Hu, “Hybrid contrastive learning of tri-modal representation for multimodal sentiment analysis,” IEEE Transactions on Affective Computing, vol. 14, no. 3, pp. 2276–2289, 2022

    work page 2022

  4. [3]

    Learning language-guided adaptive hyper-modality representation for multimodal sentiment analysis,

    Haoyu Zhang, Yu Wang, Guanghao Yin, Kejun Liu, Yuanyuan Liu, and Tianshu Yu, “Learning language-guided adaptive hyper-modality representation for multimodal sentiment analysis,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 756–767

    work page 2023

  5. [4]

    Learning fine-grained representation with token- level alignment for multimodal sentiment analysis,

    Xiang Li, Haijun Zhang, Zhiqiang Dong, Xianfu Cheng, Yun Liu, and Xiaoming Zhang, “Learning fine-grained representation with token- level alignment for multimodal sentiment analysis,” Expert Systems with Applications, vol. 269, pp. 126274, 2025

    work page 2025

  6. [5]

    Recognizing emotions in video using multimodal dnn feature fusion,

    Jennifer Williams, Steven Kleinegesse, Ramona Comanescu, and Oana Radu, “Recognizing emotions in video using multimodal dnn feature fusion,” in Grand Challenge and Workshop on Human Multimodal Language. Association for Computational Linguistics, 2018, pp. 11–19

    work page 2018

  7. [6]

    Feature-level fusion approaches based on multimodal eeg data for depression recognition,

    Hanshu Cai, Zhidiao Qu, Zhe Li, Yi Zhang, Xiping Hu, and Bin Hu, “Feature-level fusion approaches based on multimodal eeg data for depression recognition,” Information Fusion, vol. 59, pp. 127–138, 2020

    work page 2020

  8. [7]

    Emotion recognition of affective speech based on multiple classifiers using acoustic-prosodic information and semantic labels,

    Chung-Hsien Wu and Wei-Bin Liang, “Emotion recognition of affective speech based on multiple classifiers using acoustic-prosodic information and semantic labels,” IEEE Transactions on Affective Computing , vol. 2, no. 1, pp. 10–21, 2010

    work page 2010

Show all 31 references
  1. [8]

    Adamow: Mul- timodal sentiment analysis based on adaptive modality-specific weight fusion network,

    Junling Zhang, Xuemei Wu, and Changqin Huang, “Adamow: Mul- timodal sentiment analysis based on adaptive modality-specific weight fusion network,” IEEE Access, vol. 11, pp. 48410–48420, 2023

    2023

  2. [9]

    Multimodal transformer for unaligned multimodal language sequences,

    Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis- Philippe Morency, and Ruslan Salakhutdinov, “Multimodal transformer for unaligned multimodal language sequences,” in Proceedings of the conference. Association for computational linguistics. Meeting , 2019...

    2019

  3. [10]

    Transformer-based fea- ture reconstruction network for robust multimodal sentiment analysis,

    Ziqi Yuan, Wei Li, Hua Xu, and Wenmeng Yu, “Transformer-based fea- ture reconstruction network for robust multimodal sentiment analysis,” in Proceedings of the 29th ACM international conference on multimedia, 2021, pp. 4400–4407

    2021

  4. [11]

    Cubemlp: An mlp-based model for multimodal sentiment analysis and depression estimation,

    Hao Sun, Hongyi Wang, Jiaqing Liu, Yen-Wei Chen, and Lanfen Lin, “Cubemlp: An mlp-based model for multimodal sentiment analysis and depression estimation,” in Proceedings of the 30th ACM international conference on multimedia , 2022, pp. 3722–3729

    2022

  5. [12]

    Efficient multimodal transformer with dual-level feature restoration for robust multimodal sentiment analysis,

    Licai Sun, Zheng Lian, Bin Liu, and Jianhua Tao, “Efficient multimodal transformer with dual-level feature restoration for robust multimodal sentiment analysis,” IEEE Transactions on Affective Computing , vol. 15, no. 1, pp. 309–325, 2023

    2023

  6. [13]

    Tetfn: A text enhanced transformer fusion network for multimodal sentiment analysis,

    Di Wang, Xutong Guo, Yumin Tian, Jinhui Liu, LiHuo He, and Xuemei Luo, “Tetfn: A text enhanced transformer fusion network for multimodal sentiment analysis,” Pattern Recognition, vol. 136, pp. 109259, 2023

    2023

  7. [14]

    Decoupled multimodal distilling for emotion recognition,

    Yong Li, Yuanzhi Wang, and Zhen Cui, “Decoupled multimodal distilling for emotion recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 6631– 6640

    2023

  8. [15]

    Toward robust incomplete multimodal sentiment analysis via hierarchical representation learning,

    Mingcheng Li, Dingkang Yang, Yang Liu, Shunli Wang, Jiawei Chen, Shuaibing Wang, Jinjie Wei, Yue Jiang, Qingyao Xu, Xiaolu Hou, et al., “Toward robust incomplete multimodal sentiment analysis via hierarchical representation learning,” arXiv preprint arXiv:2411.02793 , 2024

    2024 arXiv

  9. [16]

    Adaptive token selection and fusion network for multimodal sentiment analysis,

    Xiang Li, Ming Lu, Ziming Guo, and Xiaoming Zhang, “Adaptive token selection and fusion network for multimodal sentiment analysis,” in International Conference on Multimedia Modeling . Springer, 2024, pp. 228–241

    2024

  10. [17]

    Disentanglement translation network for multimodal sentiment analysis,

    Ying Zeng, Wenjun Yan, Sijie Mai, and Haifeng Hu, “Disentanglement translation network for multimodal sentiment analysis,” Information Fusion, vol. 102, pp. 102031, 2024

    2024

  11. [18]

    On the parameterization and initialization of diagonal state space models,

    Albert Gu, Karan Goel, Ankit Gupta, and Christopher R ´e, “On the parameterization and initialization of diagonal state space models,” Advances in Neural Information Processing Systems, vol. 35, pp. 35971– 35983, 2022

    2022

  12. [19]

    Mamba: Linear-time sequence modeling with selective state spaces,

    Albert Gu and Tri Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

    2023 arXiv

  13. [20]

    Coupled mamba: Enhanced multi-modal fusion with coupled state space model,

    Wenbing Li, Hang Zhou, Junqing Yu, Zikai Song, and Wei Yang, “Coupled mamba: Enhanced multi-modal fusion with coupled state space model,” arXiv preprint arXiv:2405.18014 , 2024

    2024 arXiv

  14. [21]

    Depmamba: Progres- sive fusion mamba for multimodal depression detection,

    Jiaxin Ye, Junping Zhang, and Hongming Shan, “Depmamba: Progres- sive fusion mamba for multimodal depression detection,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2025, pp. 1–5

    2025

  15. [22]

    Tf-mamba: Text-enhanced fusion mamba with missing modalities for robust multimodal sentiment analysis,

    Xiang Li, Xianfu Cheng, Dezhuang Miao, Xiaoming Zhang, and Zhoujun Li, “Tf-mamba: Text-enhanced fusion mamba with missing modalities for robust multimodal sentiment analysis,” arXiv preprint arXiv:2505.14329, 2025

    2025

  16. [23]

    Vmamba: Visual state space model,

    Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, Jianbin Jiao, and Yunfan Liu, “Vmamba: Visual state space model,” Advances in neural information processing systems , vol. 37, pp. 103031–103063, 2024

    2024

  17. [25]

    Misa: Modality-invariant and-specific representations for multimodal sentiment analysis,

    Devamanyu Hazarika, Roger Zimmermann, and Soujanya Poria, “Misa: Modality-invariant and-specific representations for multimodal sentiment analysis,” in Proceedings of the 28th ACM international conference on multimedia, 2020, pp. 1122–1131

    2020

  18. [26]

    Learning modality- specific representations with self-supervised multi-task learning for multimodal sentiment analysis,

    Wenmeng Yu, Hua Xu, Ziqi Yuan, and Jiele Wu, “Learning modality- specific representations with self-supervised multi-task learning for multimodal sentiment analysis,” in Proceedings of the AAAI conference on artificial intelligence , 2021, vol. 35, pp. 10790–10797

    2021

  19. [27]

    Improving multimodal fusion with hierarchical mutual information maximization for multimodal sen- timent analysis,

    Wei Han, Hui Chen, and Soujanya Poria, “Improving multimodal fusion with hierarchical mutual information maximization for multimodal sen- timent analysis,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021, pp. 9180–9192

    2021

  20. [28]

    Cross-modal enhancement network for multimodal sentiment analysis,

    Di Wang, Shuai Liu, Quan Wang, Yumin Tian, Lihuo He, and Xinbo Gao, “Cross-modal enhancement network for multimodal sentiment analysis,” IEEE Transactions on Multimedia , vol. 25, pp. 4909–4921, 2022

    2022

  21. [29]

    Cardiovascular disease detection from multi-view chest x-rays with bi-mamba,

    Zefan Yang, Jiajin Zhang, Ge Wang, Mannudeep K Kalra, and Pingkun Yan, “Cardiovascular disease detection from multi-view chest x-rays with bi-mamba,” in International Conference on Medical Image Com- puting and Computer-Assisted Intervention . Springer, 2024, pp. 134– 144

    2024

  22. [30]

    Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages,

    Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency, “Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages,” IEEE Intelligent Systems , vol. 31, no. 6, pp. 82–88, 2016

    2016

  23. [31]

    Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph,

    AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency, “Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph,” in Proceedings of the 56th Annual Meeting of the Association for Computational L...

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.