Pith. sign in

REVIEW 4 major objections 5 minor 46 references

SPICE reorders multimodal samples by PID-inspired redundancy, uniqueness, and synergy scores, and the best variant beats static and balanced curricula on four benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 02:04 UTC pith:2F2TZVGN

load-bearing objection SPICE is a sensible dynamic-curriculum heuristic with a PID framing that overclaims; the gains over BSS-L are often within noise, and the evaluation needs seeds and re-run baselines. the 4 major comments →

arxiv 2606.16639 v2 pith:2F2TZVGN submitted 2026-06-15 cs.LG

SPICE: Synergy and Partial Information Based Curriculum Evolution

Ankush Pratap Singh , Houwei Cao , Yong Liu This is my paper
classification cs.LG
keywords multimodal learningcurriculum learningpartial information decompositiondynamic sample orderingredundancysynergyinteraction learningself-paced learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the difficulty of a multimodal sample is not fixed: it depends on the model's current state and on how the modalities interact. It proposes SPICE, a curriculum that estimates each sample's redundant, unique, and synergistic information from the model's own unimodal and multimodal predictions, then reorders samples every few epochs from easy shared cues to hard synergistic ones. Across four benchmarks (emotion, action, gesture, large-scale audio-visual), the full-dataset variant SPICE-E reports the best accuracy and mAP, surpassing conventional fusion, modality rebalancing, and a recent balance-aware curriculum. The paper's central message is that sample ordering should evolve with the model, and that PID provides an interpretable lens for that evolution.

Core claim

SPICE is the first dynamic multimodal curriculum driven by PID-inspired estimates: for each sample, confidence in the ground-truth label and KL divergence between unimodal and multimodal prediction distributions define three scores for redundancy, uniqueness, and synergy. These scores are recomputed every k epochs and used either to bin samples (SPICE-S) or to order all samples in each stage (SPICE-E). On CREMA-D, Kinetics-Sounds, NVGesture, and VGGSound, SPICE-E reports the highest ACC and mAP/F1 among all compared methods, with the largest gains on the large-scale VGGSound benchmark. The authors conclude that model-driven, dynamically updated sample ordering is more effective than static d

What carries the argument

The core machinery is the triple of scalar scores R_i, U_i, S_i defined from the model's current prediction probabilities. R_i rewards high, mutually agreeing confidences across modalities; U_i rewards one confident modality while others are unsure and its prediction is distinct; S_i rewards the fused prediction exceeding the strongest unimodal one while diverging from the unimodal average. These scores operationalize the PID categories redundant, unique, and synergistic without computing a real information decomposition, and their periodic recomputation every k epochs implements the curriculum evolution.

Load-bearing premise

The whole method rests on the assumption that confidence in the ground-truth label and KL divergence between softmax outputs faithfully estimate the true redundant, unique, and synergistic information content of a sample.

What would settle it

On a held-out subset of any of the four datasets, compute a canonical PID decomposition (for instance, using a histogram or Gaussian estimator on the encoder features and labels) and rank samples by the resulting redundancy/synergy; if the rank correlation with SPICE's R_i and S_i scores is near zero, the claimed PID basis is unsupported. A simpler ablation: replace the PID scores with randomly shuffled or confidence-only ordering while keeping the refresh schedule; if performance does not drop, the specific decomposition is not what drives the gains.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A curriculum that refreshes sample ordering every k epochs based on model predictions can improve final accuracy over any fixed ranking.
  • Progressing from redundant to unique to synergistic samples is a viable easy-to-hard schedule for multimodal training.
  • Using all samples with probabilistic ordering (SPICE-E) beats stage-wise binning (SPICE-S), at the cost of more gradient updates in early epochs.
  • The method requires no per-modality labels, only the model's own prediction probabilities.
  • The gains persist on a trimodal dataset (NVGesture) and a large-scale dataset (VGGSound), suggesting scalability beyond bimodal, medium-scale settings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The PID interpretation is an analogy: the scores are heuristic combinations of confidence and divergence, not quantities from a rigorous PID estimator. If they are later shown not to correlate with true redundant/unique/synergy information, the curriculum's theoretical story would reduce to 'self-paced ordering by prediction agreement'.
  • The ablation showing that hybrid annotator/model curricula underperform the fully model-driven one hints that interpolating between two difficulty signals can hurt; a natural test is whether treating annotator disagreement as a regularizer, as the paper's future work suggests, preserves the benefits without the instability.
  • The refresh interval k=5 is chosen without sensitivity analysis; sweeping k would reveal how sensitive the gains are to the dynamics, and whether a continuously updated curriculum is necessary or a few refreshes suffice.
  • Because the scores rely on softmax confidences, they may inherit miscalibration; temperature scaling or ensembling could alter the ordering, which is a testable extension.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SPICE, a curriculum learning framework for multimodal classification. It orders training samples by three 'PID-inspired' scores R_i, U_i, S_i computed from the current model's unimodal and multimodal prediction confidences and KL divergences. The curriculum progresses from redundant to unique to synergistic samples, with scores refreshed every k=5 epochs. Two variants are presented: SPICE-S bins samples and adds bins stagewise, while SPICE-E orders all samples by stage-specific sampling probabilities. Experiments on CREMA-D, Kinetics-Sounds, NVGesture, and VGGSound claim that SPICE-E outperforms existing baselines including BSS. An annotator-based ablation on CREMA-D suggests that model-driven dynamic ordering beats static human-derived curricula.

Significance. If the claimed empirical gains are reproducible and the scores truly reflect PID components, the paper would be a useful contribution: it offers an interpretable dynamic curriculum for multimodal learning and a concrete way to stage training by interaction type. The authors should be credited for the overall idea, for comparing with BSS, and for including the annotator-based ablation. However, the central theoretical premise—that Eqs. (10)–(14) estimate redundancy, unique information, and synergy—is not validated, and the empirical protocol (adopted baselines, no repeated-seed results) does not currently support the strongest claims. The significance is therefore conditional on substantial additional evidence.

major comments (4)
  1. [§3.3, Eqs. (10), (12), (14)] The scores R_i, U_i, S_i are introduced as 'PID-inspired,' but the abstract, introduction, and §4.4.6 repeatedly treat them as a decomposition of multimodal information into redundant, unique, and synergistic components. No argument or experiment shows that these formulas track the corresponding PID quantities. They are not shown to satisfy any PID axiom, they are not a partition of I(X_1,...,X_M;Y), and the paper never compares them with a known PID estimator or with the annotator-derived categories used in §5. As a result, the theoretical motivation for SPICE as an information-theoretic curriculum is unsupported; if the scores are merely heuristics, the method reduces to a self-paced reordering scheme. A concrete test would be to use CREMA-D's annotator responses as a ground-truth proxy (as defined in §3.2) and report rank correlations of R/U/S, or to evaluate on synthetic data with kn
  2. [§4.4.1, Tables 2–3] Baseline numbers are 'adopted from the BSS paper [13],' not re-run in the same training protocol, and no multiple-seed or confidence-interval results are reported for SPICE. This is problematic because the reported gains over BSS-L are small in some cases (CREMA-D ACC +0.26, VGGSound mAP +0.02). Differences of this size can arise from optimization noise or evaluation details. The 'consistently achieves the best performance' claim is therefore not supported by the current evidence. The authors should re-run baselines under identical conditions or report mean±std over several seeds and avoid making significance claims without statistical support.
  3. [§3.4.2, §4.4.5] No ablation isolates the contribution of the PID-inspired scores from the dynamic refresh mechanism. The comparison with BSS changes both the ordering criterion and the refresh schedule; the annotator ablation in §5 changes the signal source but also introduces a different (categorical) notion of difficulty. To support the attribution that the PID-inspired scores themselves drive the gains, the authors should fix the dynamic ordering and compare R/U/S scores against random ordering, confidence-only ordering, and agreement-only ordering. Without such an ablation, the experiments show only that some form of adaptive self-paced reordering can help, not that the PID-inspired component is the active ingredient.
  4. [§5, Table 4] The annotator experiments are presented as supporting the dynamic, model-dependent difficulty thesis, but they do not validate Eqs. (10)–(14). The 'Complete Static Curriculum' uses a discrete rule based on whether annotator unimodal/multimodal labels match the ground truth, rather than the continuous model-based scores. The hybrid variants mix this rule with model scores via Eq. (20), which is an additional design choice. Thus the comparison is between static human labels and dynamic model predictions, not between valid and invalid estimates of the same PID quantity. The conclusion that human-derived difficulty alone is insufficient may be correct, but it does not license the specific PID interpretation claimed in the paper.
minor comments (5)
  1. [§3.2, sentence before Eq. (7)] Typo: 'decomposed into informationredundantcross all modalities' should read 'redundant information across all modalities.'
  2. [Eq. (8)] Directional KL divergence is asymmetric, and the paper does not justify the choice of direction. Also, if unimodal predictions assign zero probability to classes present in the average prediction, the KL divergence is undefined; a smoothing strategy should be stated.
  3. [Eq. (15)] The min-max normalization in Eq. (15) does not specify the reference set over which C_min and C_max are computed. This matters because scores are refreshed every k=5 epochs; normalization over a batch versus the whole training set would produce different orderings.
  4. [§5.2, Eq. (20)] The notation D^{(c)} is used for scores, but D was previously used for KL divergence, which is confusing. Consider renaming the score variables (e.g., S^{(c)} or V^{(c)}).
  5. [Figure 3] The figure shows a single curve per method, with no indication of variance or number of runs. The claim that SPICE-S requires 'nearly one-third fewer gradient updates' would be more credible with repeated runs.

Circularity Check

1 steps flagged

The 'PID-guided' curriculum is partly a renaming of agreement/confidence heuristics: Eqs. (10)-(14) define redundancy/unique/synergy directly from model confidence and KL divergence, so the PID interpretation is supplied by definition rather than by a validated information decomposition. The empirical comparison against external baselines remains non-circular.

specific steps
  1. renaming known result [Section 3.3.1-3.3.3, Eqs. (10), (12), (14); cf. Abstract]
    "To develop a dynamic curriculum for any multimodal datasets, we propose to estimate PID information distribution for sample i using endogenous model outputs... We consider two key factors: agreement/disagreement across unimodal and multimodal predictions, and their alignment with the ground-truth label. ... The redundancy score for sample i is defined as R_i ≜ (∏_{m=1}^M c_i^(m)) · exp(−D̄_i) ... The synergy score is computed as S_i ≜ (c_i^{multi} − max_m c_i^(m)) · D_i^syn."

    Equations (10), (12), and (14) are not derived from the PID identity I(X1,X2;Y)=R+U1+U2+S, and no independent PID estimator (e.g., BROJA) or the annotator-derived PID categories of Section 5 is used to validate them. Instead, R is high exactly when all unimodal predictions are confident on the ground-truth label and agree; U is high when one modality is confident while others are not; S is high when the fused prediction beats the best unimodal confidence. These are standard agreement/confidence heuristics renamed 'redundant/unique/synergistic.' The paper's own text states the scores are based on 'agreement/disagreement' and 'alignment with the ground-truth label,' so the claim that SPICE is 'guided by PID' holds by renaming rather than by an information-theoretic decomposition. This is loa

full rationale

The central empirical result—SPICE-E outperforming BSS and other baselines on CREMA-D, Kinetics-Sounds, NVGesture, and VGGSound—is not entailed by the score definitions; it is an independent experimental finding, so this is not a fully circular derivation. However, the paper's framing that the method 'decomposes multimodal interactions into redundant, unique, and synergistic information components' is stronger than what the equations establish: the scores are defined directly from model confidence and KL divergence, and the paper never checks whether these quantities correspond to a true PID decomposition. The self-citation [30] is minor and not load-bearing. Thus the circularity is limited to the PID-interpretation claim, while the practical curriculum comparison stands on its own.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 1 invented entities

The central claim rests on hand-chosen hyperparameters, a heuristic mapping from model outputs to PID components, and a pedagogical assumption about easy-to-hard curriculum ordering. The paper does not fit parameters to the test set, but the score formulas and schedule are design choices that are not independently validated.

free parameters (5)
  • stage_epochs = 50 per stage
    Each of the redundant/unique/synergy stages lasts 50 epochs; this hand-chosen schedule is not derived and affects curriculum pacing.
  • warmup_epochs = 30
    An initial 30-epoch warm-up is introduced to stabilize representations before PID-based curriculum begins (Section 4.2).
  • pid_refresh_interval_k = 5 epochs
    PID scores are recalculated every k=5 epochs; this refresh rate is chosen by hand and controls how quickly the curriculum adapts.
  • score_formulas = Eqs. 10, 11, 14
    The redundancy/unique/synergy scores are hand-designed functions of prediction confidences and KL divergences, not derived from a PID estimator; their functional form is a modeling choice.
  • sampling_probabilities = P_R, 1-P_U, 1-P_S
    SPICE-E uses these stage-specific sampling probabilities; the choice to prioritize easy samples within each stage is a design decision, not an inference.
axioms (4)
  • standard math PID identity I(X1,X2;Y)=R+U1+U2+S (Section 2.3)
    The paper invokes the standard PID decomposition to motivate the curriculum, though it never computes these quantities.
  • ad hoc to paper Model prediction confidence and KL divergence between unimodal/multimodal predictions approximate the PID components
    Equations 10-14 assume that high confidence/agreement corresponds to redundancy, confident single-modality disagreement to uniqueness, and multimodal improvement over unimodal to synergy. This mapping is asserted without validation.
  • domain assumption Redundant-to-unique-to-synergistic ordering matches easy-to-hard learning
    The method assumes samples dominated by redundant cues are easiest and synergy samples hardest; this is a pedagogical assumption about multimodal learning.
  • domain assumption Dynamic reordering based on current model predictions is beneficial
    The paper's motivation assumes sample difficulty is model-dependent and that real-time reordering helps; supported only by comparison to static annotator curricula, not by an isolated ablation.
invented entities (1)
  • PID-inspired scores R_i, U_i, S_i no independent evidence
    purpose: To quantify sample-level redundant, unique, and synergistic information for curriculum ordering.
    These scores are defined purely from the model's own softmax outputs and KL divergences; they are not shown to correspond to any actual PID measure or to external information-theoretic quantities, so they have no falsifiable handle outside the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 13134 in / 12598 out tokens · 129150 ms · 2026-08-03T02:04:12.499336+00:00 · methodology

0 comments
read the original abstract

Multimodal learning exploits complementary information across heterogeneous modalities. The informativeness of each modality can vary widely across samples and training stages. Existing multimodal curriculum learning strategies often assume that the relative complexity of samples remains unchanged throughout training and therefore cannot adapt to model evolution. We propose SPICE (Synergy and Partial Information based Curriculum Evolution), a novel progressive curriculum framework for multimodal interaction learning. Guided by Partial Information Decomposition (PID) theory, our approach decomposes multimodal interactions into redundant, unique, and synergistic information components, enabling an interpretable and dynamic characterization of sample complexity. Building on this decomposition, we design a progressive curriculum that evolves throughout training, allowing the model to transition from learning shared cross-modal cues to modality-specific patterns and, finally, to complex synergistic interactions. Adapting to model evolution, sample ordering is refined in real-time using PID information estimates derived from unimodal and multimodal predictions. Experiments across multiple multimodal benchmarks demonstrate consistent improvements over conventional training and state-of-the-art baselines, highlighting the effectiveness of PID information decomposition and adaptive sample ordering for multimodal curriculum learning.

Figures

Figures reproduced from arXiv: 2606.16639 by Ankush Pratap Singh, Houwei Cao, Yong Liu.

Figure 1
Figure 1. Figure 1: SPICE Workflow: PID Information Estimation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of SPICE: (a). Multi-modal learning framework, (b). PID-Based Sample Ordering, (c). Dynamic Curriculum [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cost comparison in terms of gradient updates for [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 5 canonical work pages

  1. [1]

    Jean-Baptiste Alayrac, Adrià Recasens, Rosalia Schneider, Relja Arandjelović, Jason Ramapuram, Jeffrey De Fauw, Lucas Smaira, Sander Dieleman, and Andrew Zisserman. 2020. Self-Supervised MultiModal Versatile Networks. InNeurIPS

    2020

  2. [2]

    Jose Daniel Azofeifa, Julieta Noguez, Sergio Ruiz, José Martín Molina-Espinosa, Alejandra J Magana, and Bedrich Benes. 2022. Systematic review of multimodal human–computer interaction. InInformatics, Vol. 9. MDPI, 13

    2022

  3. [3]

    Tadas Baltrusaitis, Chaitanya Ahuja, and Louis-Philippe Morency. 2019. Multi- modal Machine Learning: A Survey and Taxonomy.IEEE Trans. Pattern Anal. Mach. Intell.41, 2 (Feb. 2019), 423–443. doi:10.1109/TPAMI.2018.2798607

    arXiv 2019

  4. [4]

    Adam B. Barrett. 2015. Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems.Phys. Rev. E91 (May 2015), 052802. Issue 5. doi:10.1103/PhysRevE.91.052802

    doi:10.1103/physreve.91.052802 2015

  5. [5]

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. InProceedings of the 26th annual international conference on machine learning. 41–48

    2009

  6. [6]

    Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost, and Nihat Ay

  7. [7]

    Houwei Cao, David Cooper, Michael Keutmann, Ruben Gur, Ani Nenkova, and Ragini Verma. 2014. CREMA-D: Crowd-sourced emotional multimodal actors dataset.IEEE transactions on affective computing5 (10 2014), 377–390. doi:10. 1109/TAFFC.2014.2336244

    arXiv 2014

  8. [8]

    Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. 2020. Vg- gsound: A Large-Scale Audio-Visual Dataset.ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)(2020), 721–725

    2020

  9. [9]

    Yunfeng Fan, Wenchao Xu, Haozhao Wang, Junxiao Wang, and Song Guo

  10. [10]

    Naotsuna Fujimori, Rei Endo, Yoshihiko Kawai, and Takahiro Mochizuki. 2019. Modality-Specific Learning Rate Control for Multimodal Classification. InAsian Conference on Pattern Recognition

    2019

  11. [11]

    Bellemare, Jacob Menick, Rémi Munos, and Koray Kavukcuoglu

    Alex Graves, Marc G. Bellemare, Jacob Menick, Rémi Munos, and Koray Kavukcuoglu. 2017. Automated curriculum learning for neural networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia)(ICML’17). JMLR.org, 1311–1320

    2017

  12. [12]

    Virgil Griffith and Christof Koch. 2014. Quantifying synergistic mutual informa- tion. InGuided self-organization: inception. Springer, 159–190

    2014

  13. [13]

    Zhi-Hao Guan, Qing-Yuan Jiang, and Yang Yang. 2025. Balance-aware Sequence Sampling Makes Multimodal Learning Better. InIJCAI

    2025

  14. [14]

    Guy Hacohen and Daphna Weinshall. 2019. On the power of curriculum learning in training deep networks. InInternational conference on machine learning. PMLR, 2535–2544

    2019

  15. [15]

    Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. 2023. Trusted Multi-View Classification With Dynamic Evidential Fusion .IEEE Transactions on Pattern Analysis & Machine Intelligence45, 02 (Feb. 2023), 2551–2566. doi:10. 1109/TPAMI.2022.3171983

    arXiv 2023

  16. [16]

    Cong Hua, Qianqian Xu, Shilong Bao, Zhiyong Yang, and Qingming Huang. 2024. ReconBoost: boosting can achieve modality reconcilement. InProceedings of the 41st International Conference on Machine Learning(Vienna, Austria)(ICML’24). JMLR.org, Article 789, 25 pages

    2024

  17. [17]

    Haupt- mann

    Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander G. Haupt- mann. 2015. Self-paced curriculum learning. InProceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence(Austin, Texas)(AAAI’15). AAAI Press, 2694–2700

    2015

  18. [18]

    Will Kay, João Carreira, Karen Simonyan, Brian Hu Zhang, Chloe Hillier, Sud- heendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Apostol Natsev, Mustafa Suleyman, and Andrew Zisserman. 2017. The Kinetics Human Action Video Dataset.ArXivabs/1705.06950 (2017)

    Pith/arXiv arXiv 2017

  19. [19]

    Hong Li, Xingyu Li, Pengbo Hu, Yinuo Lei, Chunxiao Li, and Yi Zhou. 2023. Boosting Multi-modal Model Performance with Adaptive Gradient Modulation. InProceedings of the IEEE/CVF International Conference on Computer Vision. 22214– 22224

    2023

  20. [20]

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multi- modal reasoning via thought chains for science question answering.Advances in neural information processing systems35 (2022), 2507–2521

    2022

  21. [21]

    Zhaozhao Ma and Shujian Yu. 2025. Explainable Multimodal Regression via Information Decomposition.arXiv preprint arXiv:2512.22102(2025)

    arXiv 2025

  22. [22]

    Pavlo Molchanov, Xiaodong Yang, Shalini Gupta, Kihwan Kim, Stephen Tyree, and Jan Kautz. 2016. Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks. In2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4207–4215

    2016

  23. [23]

    Weizhi Nie, Yan Yan, Dan Song, and Kun Wang. 2021. Multi-modal feature fusion based on multi-layers LSTM for video emotion recognition.Multimedia Tools and Applications80 (05 2021). doi:10.1007/s11042-020-08796-8

    work page doi:10.1007/s11042-020-08796-8 2021

  24. [24]

    Guanxiong Pei, Haiying Li, Yandi Lu, Yanlei Wang, Shizhen Hua, and Taihao Li. 2024. Affective Computing: Recent Advances, Challenges, and Future Trends.Intelligent Computing3 (2024),

    2024

  25. [25]

    Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu. 2022. Bal- anced Multimodal Learning via On-the-fly Gradient Modulation.2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2022), 8228–8237

    2022

  26. [26]

    Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu. 2022. Balanced Multimodal Learning via On-the-fly Gradient Modulation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2022

  27. [27]

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. 2018. FiLM: visual reasoning with a general conditioning layer. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Art...

    2018

  28. [28]

    R.W. Picard. 2000. Toward computers that recognize and respond to user emotion. IBM Systems Journal39, 3.4 (2000), 705–719. doi:10.1147/sj.393.0705

    work page doi:10.1147/sj.393.0705 2000

  29. [29]

    Sharma, V.I

    R. Sharma, V.I. Pavlovic, and T.S. Huang. 1998. Toward multimodal human- computer interface.Proc. IEEE86, 5 (1998), 853–869. doi:10.1109/5.664275

    work page doi:10.1109/5.664275 1998

  30. [30]

    Ankush Pratap Singh, Houwei Cao, and Yong Liu. 2026. CHUCKLE – When Humans Teach AI To Learn Emotions The Easy Way. arXiv:2510.09382 [cs.LG] https://arxiv.org/abs/2510.09382

    Pith/arXiv arXiv 2026

  31. [31]

    Petru Soviany, Radu Tudor Ionescu, Paolo Rota, and Nicu Sebe. 2022. Curriculum learning: A survey.International Journal of Computer Vision130, 6 (2022), 1526– 1565

    2022

  32. [33]

    Weiyao Wang, Du Tran, and Matt Feiszli. 2019. What Makes Training Multi- Modal Classification Networks Hard?2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2019), 12692–12702

    2019

  33. [34]

    Yiqi Wang, Wentao Chen, Xiaotian Han, Xudong Lin, Haiteng Zhao, Yongfei Liu, Bohan Zhai, Jianbo Yuan, Quanzeng You, and Hongxia Yang. 2024. Exploring the reasoning abilities of multimodal large language models (mllms): A com- prehensive survey on emerging trends in multimodal reasoning.arXiv preprint arXiv:2401.06805(2024)

    Pith/arXiv arXiv 2024

  34. [35]

    Yiru Wang, Weihao Gan, Jie Yang, Wei Wu, and Junjie Yan. 2019. Dynamic Curriculum Learning for Imbalanced Data Classification. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2019

  35. [36]

    Yake Wei, Ruoxuan Feng, Zihe Wang, and Di Hu. 2024. Enhancing multimodal Cooperation via Sample-level Modality Valuation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2024

  36. [37]

    Yake Wei and Di Hu. 2024. MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistance. InProceedings of the 41st International Confer- ence on Machine Learning (Proceedings of Machine Learning Research, Vol. 235), Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (Eds.). ...

    2024

  37. [38]

    Yake Wei, Di Hu, Yapeng Tian, and Xuelong Li. 2022. Learning in audio-visual context: A review, analysis, and new perspective.arXiv preprint arXiv:2208.09579 (2022)

    Pith/arXiv arXiv 2022

  38. [39]

    Williams and Randall D

    Paul L. Williams and Randall D. Beer. 2010. Nonnegative Decomposition of Multivariate Information.ArXivabs/1004.2515 (2010)

    Pith/arXiv arXiv 2010

  39. [40]

    Yang Yang, Hongpeng Pan, Qing-Yuan Jiang, Yi Xu, and Jinhui Tang. 2025. Learn- ing to Rebalance Multi-Modal Optimization by Adaptively Masking Subnet- works.IEEE Trans. Pattern Anal. Mach. Intell.47, 6 (June 2025), 4553–4566. doi:10.1109/TPAMI.2025.3547417

    arXiv 2025

  40. [41]

    Yang Yang, Jingshuai Zhang, Fan Gao, Xiaoru Gao, and Hengshu Zhu. 2022. DOMFN: A Divergence-Orientated Multi-Modal Fusion Network for Resume Assessment.Proceedings of the 30th ACM International Conference on Multimedia (2022)

    2022

  41. [42]

    Yiqun Yao and Rada Mihalcea. 2022. Modality-specific Learning Rates for Effective Multimodal Additive Late-fusion. InFindings of the Association for Computational Linguistics: ACL 2022, Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ireland, 1824–1834. doi:10.18653/v1/2022.findings-acl.143

    work page doi:10.18653/v1/2022.findings-acl.143 2022

  42. [43]

    Yuan Yuan, Zhaojian Li, and Bin Zhao. 2025. A Survey of Multimodal Learning: Methods, Applications, and Future.ACM Comput. Surv.57, 7, Article 167 (Feb. 2025), 34 pages. doi:10.1145/3713070

    work page doi:10.1145/3713070 2025

  43. [44]

    Xiaohui Zhang, Jaehong Yoon, Mohit Bansal, and Huaxiu Yao. 2023. Multimodal Representation Learning by Alternating Unimodal Adaptation.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2023), 27446– 27456

    2023

  44. [76]

    arXiv:https://spj.science.org/doi/pdf/10.34133/icomputing.0076 doi:10.34133/icomputing.0076

    doi:10.34133/icomputing.0076

  45. [2013]

    Quantifying unique information.Entropy16 (2013), 2161–2183

    2013

  46. [2023]

    PMR: Prototypical Modal Rebalance for Multimodal Learning. InPro- ceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, CVPR 2023 (Proceedings of the IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition). IEEE Computer Society, 20029–20038. doi:10.1109/CVPR52729.2023.01918 Publisher Copyright:©2023 IEEE...

    arXiv 2023