REVIEW 6 cited by
Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysis
read the original abstract
Though Multimodal Sentiment Analysis (MSA) proves effective by utilizing rich information from multiple sources (e.g., language, video, and audio), the potential sentiment-irrelevant and conflicting information across modalities may hinder the performance from being further improved. To alleviate this, we present Adaptive Language-guided Multimodal Transformer (ALMT), which incorporates an Adaptive Hyper-modality Learning (AHL) module to learn an irrelevance/conflict-suppressing representation from visual and audio features under the guidance of language features at different scales. With the obtained hyper-modality representation, the model can obtain a complementary and joint representation through multimodal fusion for effective MSA. In practice, ALMT achieves state-of-the-art performance on several popular datasets (e.g., MOSI, MOSEI and CH-SIMS) and an abundance of ablation demonstrates the validity and necessity of our irrelevance/conflict suppression mechanism.
Forward citations
Cited by 6 Pith papers
-
Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration
GCL uses a two-stage protocol with Routing, Auditing, Public-Factor, and Aggregation Agents to mitigate modality dominance and spurious coupling in multimodal learning, achieving state-of-the-art results on CMU-MOSI, ...
-
Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis
EBMC framework enhances weaker modalities via semantic disentanglement and cross-modal boosting, then balances them with energy-guided coordination and instance-aware trust distillation for improved MSA performance an...
-
Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis
MMCI uses multi-relational graph modeling and attention-based disentanglement of causal versus shortcut features, combined with backdoor adjustment, to reduce bias and improve generalization in multimodal sentiment analysis.
-
Controlling Decision Drift in Multimodal Sentiment Analysis with Missing Modalities
A two-level reference alignment framework uses complete-modality samples and prototype voting to reduce decision drift and improve robustness in multimodal sentiment analysis under missing modalities.
-
Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration
Group Cognition Learning uses governed two-stage agents after separate modality encoding to mitigate dominance and spurious coupling, reporting state-of-the-art results on CMU-MOSI, CMU-MOSEI, and MIntRec for regressi...
-
Recent Advances in Multimodal Affective Computing: An NLP Perspective
Survey organizing multimodal affective computing research around four NLP tasks, method paradigms, datasets, evaluation protocols, and future directions while releasing a resource repository.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.