REVIEW 5 cited by
Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition
read the original abstract
The development of multimodal models has significantly advanced multimodal sentiment analysis and emotion recognition. However, in real-world applications, the presence of various missing modality cases often leads to a degradation in the model's performance. In this work, we propose a novel multimodal Transformer framework using prompt learning to address the issue of missing modalities. Our method introduces three types of prompts: generative prompts, missing-signal prompts, and missing-type prompts. These prompts enable the generation of missing modality features and facilitate the learning of intra- and inter-modality information. Through prompt learning, we achieve a substantial reduction in the number of trainable parameters. Our proposed method outperforms other methods significantly across all evaluation metrics. Extensive experiments and ablation studies are conducted to demonstrate the effectiveness and robustness of our method, showcasing its ability to effectively handle missing modalities.
Forward citations
Cited by 5 Pith papers
-
Deep Multimodal Learning with Missing Modality: A Survey
This survey provides the first comprehensive overview of deep multimodal learning methods designed to remain robust when some input modalities are absent.
-
Evaluation Before Generation: A Paradigm for Robust Multimodal Sentiment Analysis with Missing Modalities
The ProMMA framework evaluates missing modalities at input using a dedicated evaluator, then applies modality-invariant prompt disentanglement, mutual-information dynamic weighting, and multi-level residual prompt con...
-
Context-driven Missing-Modality Learning for Robust Medical Diagnosis with Image-Tabular Data
CMML uses a Cascade Residual Transformer Autoencoder with learnable context tokens and memory banks to synthesize missing modalities, followed by semantic alignment and contrastive refinement, achieving 1-1.3% AUC gai...
-
Controlling Decision Drift in Multimodal Sentiment Analysis with Missing Modalities
A two-level reference alignment framework uses complete-modality samples and prototype voting to reduce decision drift and improve robustness in multimodal sentiment analysis under missing modalities.
-
ModalImmune: Immunity Driven Unlearning via Self Destructive Training
ModalImmune enforces modality immunity in multimodal models by controlled collapse of input channels during training using adaptive regularizers and meta-optimization.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.