Multimodal Fusion on Low-quality Data: A Comprehensive Survey
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:GXOYTINIrecord.jsonopen to challenge →
read the original abstract
Multimodal fusion focuses on integrating information from multiple modalities with the goal of more accurate prediction, which has achieved remarkable progress in a wide range of scenarios, including autonomous driving and medical diagnosis. However, the reliability of multimodal fusion remains largely unexplored especially under low-quality data settings. This paper surveys the common challenges and recent advances of multimodal fusion in the wild and presents them in a comprehensive taxonomy. From a data-centric view, we identify four main challenges that are faced by multimodal fusion on low-quality data, namely (1) noisy multimodal data that are contaminated with heterogeneous noises, (2) incomplete multimodal data that some modalities are missing, (3) imbalanced multimodal data that the qualities or properties of different modalities are significantly different and (4) quality-varying multimodal data that the quality of each modality dynamically changes with respect to different samples. This new taxonomy will enable researchers to understand the state of the field and identify several potential directions. We also provide discussion for the open problems in this field together with interesting future research directions.
This paper has not been read by Pith yet.
Forward citations
Cited by 14 Pith papers
-
BLEG: LLM Functions as Powerful fMRI Graph-Enhancer for Brain Network Analysis
BLEG enhances GNNs for fMRI brain network analysis by prompting LLMs for text augmentation, using cost-effective instruction tuning, and applying alignment losses during joint training.
-
Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classification
DyMo dynamically selects reliable recovered modalities at inference by using task loss as a proxy for task-relevant information, outperforming prior discard-or-impute methods on image datasets.
-
Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration
CPSC uses conformal prediction to decompose and fuse robust unimodal features and recalibrate gradients based on instance reliability, outperforming prior methods on imbalanced and noisy multimodal benchmarks.
-
VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representations
Fusing chart visualizations with raw time series improves or maintains classification accuracy on UCR datasets when the visuals add non-redundant information.
-
FASH-iCNN: Making Editorial Fashion Identity Inspectable Through Multimodal CNN Probing
A multimodal CNN on 87,547 Vogue images classifies fashion houses at 78.2% top-1 accuracy, decades at 88.6%, and years at 58.3% with 2.2-year mean error, and shows texture and luminance carry most of the house-identit...
-
Missing-Token Prompted Reliability-Aware Fusion for Robust Polyglot Speaker Identification
MRAF framework uses missing-token prompting and reliability-aware cross-attention fusion to achieve 100% accuracy on some POLY-SIM 2026 tasks and competitive results on missing-face cases.
-
Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition
A self-paced curriculum learning module with dual-level difficulty scoring improves weighted F1 scores by 1.2-10.4% when added to existing multimodal emotion recognition models on IEMOCAP and MELD.
-
Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linking
MSR-MEL synthesizes instance-centric, group-level, lexical, and statistical evidence with LLMs and asymmetric teacher-student GNNs to outperform prior unsupervised methods on multimodal entity linking benchmarks.
-
Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective
CmIR uses causal inference to separate invariant causal representations from spurious ones in multimodal data, improving generalization under distribution shifts and noise via invariance, mutual information, and recon...
-
MULTIBENCH++: A Unified and Comprehensive Multimodal Fusion Benchmarking Across Specialized Domains
MULTIBENCH++ is a new large-scale benchmark integrating over 30 datasets across 15 modalities and 20 tasks, accompanied by an open-source automated evaluation pipeline that establishes new performance baselines for mu...
-
RSEA-MVGNN: Multi-View Graph Neural Network with Reliable Structural Enhancement and Aggregation
RSEA-MVGNN estimates view uncertainty with subjective logic to enable diverse feature enhancement through de-correlation and quality-aware aggregation in GNNs, outperforming prior methods on five datasets.
-
Beyond Semantics: An Evidential Reasoning-Aware Multi-View Learning Framework for Trustworthy Mental Health Prediction
A multi-view evidential framework combines semantic and reasoning information to improve accuracy and provide trustworthy uncertainty estimates for mental health prediction on text data.
-
ModalImmune: Immunity Driven Unlearning via Self Destructive Training
ModalImmune enforces modality immunity in multimodal models by controlled collapse of input channels during training using adaptive regularizers and meta-optimization.
-
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning
Position paper claims multimodal LLMs can significantly advance scientific reasoning and proposes a four-stage roadmap plus challenges and suggestions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.