The reviewed record of science sign in
Pith

arxiv: 2505.15417 · v1 · pith:BLJI2TYT · submitted 2025-05-21 · stat.ML · cs.LG

Robust Multimodal Learning via Entropy-Gated Contrastive Fusion

Reviewed by Pithpith:BLJI2TYTopen to challenge →

classification stat.ML cs.LG
keywords aecffusionmultimodalcalibrationcontrastiveentropyentropy-gatedinference
0
0 comments X
read the original abstract

Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standard fusion layers either preserve robustness or calibration but never both. We introduce Adaptive Entropy-Gated Contrastive Fusion (AECF), a single light-weight layer that (i) adapts its entropy coefficient per instance, (ii) enforces monotone calibration across all modality subsets, and (iii) drives a curriculum mask directly from training-time entropy. On AV-MNIST and MS-COCO, AECF improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, yet adds 1% run-time. All back-bones remain frozen, making AECF an easy drop-in layer for robust, calibrated multimodal inference.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MSNN-LINet: Cross-Modal Learning via Continuous Linear Integration

    cs.CV 2026-06 unverdicted novelty 6.0

    LINet achieves 45.2% mean class accuracy on SUN RGB-D 19-class scene classification from scratch using continuous linear integration across dedicated RGB, depth, and integration streams.