ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning

Fengbei Liu; Mert R. Sabuncu; Minh Nguyen; Nusrat Binta Nizam; Ruining Deng; Sunwoo Kwak

REVIEW 4 major objections 5 minor 28 references

Adaptive knockout rates driven by Shapley values of validation utility reduce modality dominance and improve clinical multimodal models under missing inputs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 14:50 UTC pith:VQRT7L3K

load-bearing objection Solid architecture-agnostic training loop that often beats Fixed KO on partial-modality clinical tasks; Shapley may not be the proven causal driver, but the paper is still worth a referee. the 4 major comments →

arxiv 2607.09884 v1 pith:VQRT7L3K submitted 2026-07-10 cs.CV cs.LG

ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning

Nusrat Binta Nizam , Fengbei Liu , Sunwoo Kwak , Minh Nguyen , Ruining Deng , Mert R. Sabuncu This is my paper

classification cs.CV cs.LG

keywords robust multimodal learningmissing modalitiesShapley valuemodality knockoutmodality dominanceclinical predictionadaptive masking

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Clinical multimodal models often fail when some inputs are missing at deployment, in part because training over-relies on whichever modality is currently strongest and undertrains the others. Existing training-time masking (knockout) helps but uses fixed rates that cannot track changing modality utility. This paper introduces ShapKO: every few epochs it scores the current model on every modality subset of a validation set, turns those scores into per-modality Shapley importance values, and raises the knockout probability of strong modalities while lowering it for weak ones. The result is an architecture-agnostic training loop that promotes complementary representations. Across prostate MRI cancer detection, multimodal survival prediction, and multitask clinical classification, ShapKO improves accuracy under partial inputs relative to standard training, fixed-rate knockout, and a gradient-modulation baseline, while leaving full-modality performance intact and producing readable trajectories of how each modality’s knockout rate evolves.

Core claim

A training strategy that periodically recomputes Shapley values of validation utility over modality subsets, then sets higher knockout probabilities for modalities with above-average importance (a drop-strong-more rule), yields more robust multimodal predictors under missing inputs than static knockout or gradient balancing, without any change to the model architecture.

What carries the argument

ShapKO’s Phase-2 update: validation utility v(S) over all modality coalitions is converted via the classical Shapley formula into importance weights, which are then mapped by a clipped power rule into per-modality Bernoulli knockout rates that deliberately mask dominant modalities more often.

Load-bearing premise

That periodically recomputed Shapley values of validation utility remain a stable, task-aligned signal for setting knockout rates, even while the model, the utilities, and the rates all change together and the conversion still depends on hand-chosen sensitivity and rate bounds.

What would settle it

On any of the three clinical benchmarks, re-run the identical architecture and training schedule with ShapKO’s adaptive rates replaced by fixed rates (or by random rates inside the same bounds); if the adaptive version no longer improves partial-modality AUC or C-index relative to those controls, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Clinical multimodal systems can be made more reliable under realistic missingness by adding only a periodic validation-subset loop, with no encoder or fusion redesign.
Knockout-rate trajectories themselves become a diagnostic of modality dominance and utility drift during training.
The same drop-strong-more rule can be ported to other multimodal tasks that already use masking or dropout-style regularization.
Approximate Shapley estimators can later cut the validation cost while preserving the same adaptive logic.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Because the method never alters architecture, any existing production multimodal pipeline that already supports placeholder embeddings can adopt it as a pure training-time upgrade.
The same utility-driven reweighting idea could be applied to feature groups or sensors outside medicine, wherever one channel systematically dominates optimization.
If worst-case rather than average-subset utility were substituted into the Shapley step, the resulting rates might better protect rare but clinically critical missingness patterns.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid architecture-agnostic training loop that often beats Fixed KO on partial-modality clinical tasks; Shapley may not be the proven causal driver, but the paper is still worth a referee.

read the letter

The useful takeaway is simple: they close a training loop that turns validation subset utilities into per-modality knockout rates via Shapley and a drop-strong-more rule, and on three clinical multimodal setups that usually helps under missing inputs without changing the architecture.

What is actually new is not knockout or Shapley alone—both are cited—but the packaged adaptive schedule: every K epochs freeze θ, score all modality subsets, convert ϕ to rm with the clip/power rule, then keep training. They run this on prostate MRI (T2w/ADC/DWI), MMD survival (path/rad/genomics/demo), and FlexCare multitask (EHR/CXR/notes), report multi-fold means±std across nearly all subsets, and ship code. Fixed KO already helps a lot; ShapKO is best or tied on most rows/panels, keeps full-modality performance roughly intact, and the rate trajectories are a readable diagnostic of which modalities get suppressed. That is real engineering value for people who ship multimodal medical models under incomplete acquisition.

Soft spots, in proportion: gains over Fixed KO are often modest and uneven (T2w-only is not a win; survival C-index lifts are small). Hyperparameters and placeholders are task-tuned (Table 1). Exhaustive 2^d Shapley only works for tiny d. The stress-test concern is fair: without freezing rates after warm-up, randomizing the ϕ ranking, or matching mean rates with a non-Shapley adaptive schedule, you cannot cleanly claim that Shapley attribution—not any refreshed non-uniform knockout—is the causal piece. That is a missing ablation, not a collapse of the empirical claim that the full method works.

Math is standard cooperative-game attribution plus a simple rate map; no circular proof. Citations cover Fixed KO, OGM-GE, ModDrop, and missing-modality medical work without obvious gaps. This is for multimodal medical ML practitioners who need robustness under partial availability, not for foundational theory. I would send it to peer review; ask for the causal ablations, sensitivity on δ/r bounds, and a note on approximate Shapley for larger d. Engage if you care about training-time missing-modality methods.

Referee Report

4 major / 5 minor

Summary. The paper proposes ShapKO, an architecture-agnostic training strategy for multimodal models that periodically recomputes per-modality knockout probabilities from Shapley values of validation utility over modality subsets, then applies a drop-strong-more rule so dominant modalities are masked more often. The method is evaluated on three clinical settings—prostate MRI cancer detection (AUC over all T2w/ADC/DWI subsets), MMD survival prediction (C-index over demographics/genomics/pathology/radiology subsets), and FlexCare multitask classification (AUC over EHR/CXR/notes subsets)—against standard training, fixed-rate knockout, and OGM-GE. The authors report generally improved or comparable partial-modality performance, full-modality capacity that is not clearly harmed, and interpretable knockout-rate trajectories, with code released.

Significance. If the adaptive Shapley schedule is genuinely responsible for the residual gains over fixed knockout, the work is a useful, practical contribution to missing-modality robustness in clinical multimodal learning: it requires no architectural change, applies across classification and survival tasks, and yields diagnostic rate trajectories. Strengths include multi-fold evaluation on three heterogeneous benchmarks, explicit comparison to Fixed KO and OGM-GE, an honest limitations section (approximate Shapley, hyperparameter dependence, average-utility objective), and public code. The absolute gains over Fixed KO are often modest, so significance hinges on whether Shapley attribution—not merely non-uniform or refreshed knockout—is the causal driver.

major comments (4)

Central claim vs. Fixed KO: Table 2 and Figs. 2 and 4 show that Fixed KO already improves most partial-modality subsets over Baseline/OGM-GE; ShapKO’s additional gains are often small, uneven (e.g., T2w-only AUC 0.663 vs Fixed KO 0.685), and obtained under task-tuned δ, rmin, rmax, base r, K, and placeholders (Table 1). Without ablations that (i) freeze rates after warm-up, (ii) randomize or reverse the ranking of ϕ_m, or (iii) replace Shapley with uniform/random adaptive rates of matched mean, it remains unclear that the Shapley→drop-strong-more loop (Eqs. 1–3, Phase 2) is the causal driver rather than any non-uniform or periodically refreshed knockout schedule. This is load-bearing for the paper’s main novelty claim over Fixed KO [15].
§2.1–2.2, Eqs. (2)–(3) and Table 1: The conversion from ϕ_m to r_m depends on several free choices (δ, rmin, rmax, base r from (1−r)^d=0.5, structural vs synthetic placeholders) that differ substantially across the three tasks. The manuscript states these were set empirically via cross-validation, but does not report sensitivity analyses or a shared default. Given that the weakest assumption of the method is that co-evolving validation Shapley values remain a stable, task-aligned signal under these knobs, a sensitivity study (or at least reporting performance under a single hyperparameter recipe) is needed to support generalizability of the adaptive rule.
Eq. (5) and training description: The loss is written as an average over modalities of L(f_θ(ê^i_m), y_i), which suggests a per-modality forward path rather than a single fused prediction under the joint mask. This is inconsistent with the fusion narrative in §3 (concatenated embeddings / Transformer fusion) and with the knockout definition in Eq. (4). Please clarify whether training uses one fused forward pass under the sampled mask or a modality-wise loss average, and align the equation with the implemented objective; the current form is load-bearing for reproducibility of the reported gains.
Statistical support for “consistently improves”: Results are mean±std over folds, but there are no paired significance tests or confidence intervals on the ShapKO–Fixed KO differences. On survival (Fig. 2) absolute C-index gains are modest and overlapping variability bands are visible; on prostate MRI, ShapKO is not best on every subset. For a claim of consistent improvement under modality absence, report at least paired tests (or bootstrap CIs) on the primary partial-modality aggregates, or temper the abstract/conclusion language to match the uneven pattern.

minor comments (5)

Typos/notation: “Shapely” appears in Fig. 1 caption; “Implmentation” in §3.1; “stabledrop-strong-more” / “stableknockout-strong-more” spacing issues in the introduction and Fig. 1; “structurally missingdue” missing space in §2.
Fig. 3: Trajectories are informative but only shown for the survival task; a brief corresponding plot or summary for prostate MRI and FlexCare would strengthen the “interpretable trajectories” claim.
§5 Limitations correctly flags 2^d validation cost; a short wall-clock or relative-overhead number for d=3 and d=4 would help practitioners assess practicality.
Related work: briefly position against other adaptive masking / modality-balancing schedules beyond OGM-GE so the Shapley-specific contribution is clearer to non-medical multimodal readers.
Placeholder design (Table 1): [MISS] vs [KO] for FlexCare is a reasonable distinction; one sentence on whether sharing a single missing token degrades results would reduce design ambiguity.

Circularity Check

0 steps flagged

No significant circularity: empirical adaptive training method evaluated on held-out folds, not a derivation that reduces to its inputs by construction.

full rationale

ShapKO is a training-time procedure that periodically recomputes validation utilities v(S) over modality subsets, obtains Shapley values φ_m (Eq. 1), converts them to knockout rates r_m via the drop-strong-more rule (Eqs. 2–3), and continues optimization under the resulting Bernoulli masks. The central claims are empirical performance gains under partial-modality evaluation on three clinical benchmarks versus Baseline, Fixed KO, and OGM-GE (Table 2, Figs. 2 and 4). These gains are not forced by construction: the rates co-evolve with the model but are never algebraically identical to the reported test metrics, and the method is architecture-agnostic with no uniqueness theorem or self-definitional identity. Self-citation of the Fixed-KO baseline [15] (overlapping authors) is ordinary comparator use, not a load-bearing premise that the present results reduce to. Hyperparameters (δ, r_min, r_max, base r, placeholders) are tuned on training folds and listed in Table 1; they do not make the subset AUCs/C-indices tautological. No fitted parameter is renamed a prediction, no ansatz is smuggled via citation, and no known empirical pattern is merely re-labeled. The derivation chain is therefore self-contained against external benchmarks; any residual concern about whether Shapley specifically (versus any non-uniform schedule) drives the gains is a causal/ablation question, not circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The central claim rests on standard cooperative-game Shapley attribution, the domain premise that validation subset utility should drive training-time masking, and several hand-chosen schedule parameters. No new physical entity is postulated; the invented piece is the specific adaptive knockout policy and its utility-to-rate map.

free parameters (5)

update period K
How often Shapley rates are recomputed (Table 1: 10/15/20 epochs); chosen empirically and controls adaptation vs. overhead.
sensitivity δ
Exponent mapping relative Shapley weight to knockout rate (Eq. 3; 0.5 or 1.0 in Table 1); hand-set lower under higher structural missingness.
knockout bounds rmin, rmax and base r
Clip range and base rate from (1−r)^d=0.5 (Eq. 3; Table 1); directly determine how aggressively dominant modalities are suppressed.
structural vs synthetic placeholders
Task-specific fixed zeros, −1, or learned [MISS]/[KO] tokens (Table 1, §3); change the training signal under missingness.
optimizer and training hyperparameters
Learning rates, weight decay, epochs refined by cross-validation on training folds (§3 Hyperparameters); affect absolute metrics used to claim gains.

axioms (5)

domain assumption Shapley values of a scalar validation utility v(S) correctly quantify modality dominance for the purpose of setting training knockout rates.
Invoked in §2.1 Eqs. (1)–(3); treats average marginal validation gain as the right training regularizer.
ad hoc to paper A drop-strong-more policy (higher knockout for above-average wm) improves complementary learning without harming full-modality capacity.
Core design rule in §2.1; not derived from a theorem, justified by empirical outcomes.
domain assumption Structural missingness and synthetic knockout can be handled by fixed/learned placeholders without destructive distribution shift.
Problem setup and §3 placeholder choices; standard in masking literature but load-bearing for all results.
domain assumption Exhaustive evaluation over all 2^|O| modality subsets on the validation split is a valid utility oracle during training.
Phase 2 in §2.2; feasible only because |O| is 3–4 in the experiments.
standard math Shapley value formula as weighted average of marginal contributions over coalitions.
Eq. (1) cites standard cooperative-game definition [22].

invented entities (1)

ShapKO adaptive knockout schedule (utility→Shapley→rm update loop) no independent evidence
purpose: Convert evolving modality importance into per-modality Bernoulli knockout probabilities during training.
The paper’s main proposed object; independent evidence is only the three empirical benchmarks, not an external falsifiable prediction outside this training setup.

pith-pipeline@v1.1.0-grok45 · 13236 in / 3336 out tokens · 39877 ms · 2026-07-14T14:50:10.731194+00:00 · methodology

0 comments

read the original abstract

Multimodal medical models often degrade when inputs are missing, a common scenario in real-world clinical workflows. Separately, even when all modalities are present, modality dominance is observed during training, where optimization over-relies on a highly predictive modality and undertrains complementary sources, resulting in poor robustness under partial availability. While training-time modality knockout improves missing-modality robustness, existing approaches use static masking rates that cannot adapt to evolving modality utility during training. We introduce ShapKO (Shapley-Adaptive Modality Knockout), a dynamic training strategy that learns modality-specific knockout probabilities based on validation utility. ShapKO periodically evaluates performance across modality subsets, estimates modality importance via Shapley values, and updates masking probabilities to suppress dominant modalities more frequently. This adaptive process promotes complementary representations, while requiring no architectural modifications. We evaluate ShapKO on three datasets covering multitask clinical classification, survival prediction, and cancer detection. ShapKO consistently improves performance under modality absence and yields interpretable trajectories of learned masking behavior. Code is available at: https://github.com/sumona00/ShapKO

Figures

Figures reproduced from arXiv: 2607.09884 by Fengbei Liu, Mert R. Sabuncu, Minh Nguyen, Nusrat Binta Nizam, Ruining Deng, Sunwoo Kwak.

**Figure 2.** Figure 2: C-index performance across modality subsets under different strategies (across 15 cross-validation folds). Grouped bar plots compare Baseline, OGM-GE, Fixed KO, and ShapKO for single-modality + full-modality, two-modality, and three-modality input subsets. Vertical gray ranges indicate the variability (± standard deviation) around each estimate. D = Demographics, G = Genomics, P = Pathology, R = Radiology… view at source ↗

**Figure 3.** Figure 3: ShapKO rates over training (warm up epoch, K =20). Mean (solid line) and standard deviation (shaded band) of the learned adaptive knockout rate for each modality across 15 cross-validation folds, plotted at the KO update epochs. Lower KO indicates the modality is retained more often, while higher KO indicates more frequent knockout. E+C+N E C N E+C E+N C+N 0.60 0.75 0.90 AUC In hospital mortality E+C+N E… view at source ↗

**Figure 4.** Figure 4: Impact of modality knockout on MIMIC-IV multitask classification. Grouped bar plots report AUC for six clinical prediction tasks (In-hospital mortality, Decompensation, Phenotyping, Length of stay, Readmission, and Diagnosis) under different input modality subsets. Modality shorthand: E = EHR, C = chest X-ray, N = clinical notes (e.g., E+C+N uses all modalities). We compare Baseline training, OGM-GE, Fixe… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

28 extracted references · 4 linked inside Pith

[1]

In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention

Cui, C., Liu, H., Liu, Q., Deng, R., Asad, Z., Wang, Y., Zhao, S., Yang, H., Land- man, B.A., Huo, Y.: Survival prediction of brain cancer with incomplete radiology, pathology, genomic, and demographic data. In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention. pp. 626–635. Springer (2022)

2022
[2]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Dorent, R., Joutard, S., Modat, M., Ourselin, S., Vercauteren, T.: Hetero-modal variational encoder-decoder for joint modality completion and segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 74–82. Springer (2019)

2019
[3]

The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development pp

Enders, C.K., Baraldi, A.N.: Missing data handling methods. The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development pp. 139–185 (2018)

2018
[4]

Advances in Neural Information Processing Systems 37, 133328–133344 (2024)

Guo, Z., Jin, T., Chen, J., Zhao, Z.: Classifier-guided gradient modulation for en- hanced multimodal learning. Advances in Neural Information Processing Systems 37, 133328–133344 (2024)

2024
[5]

In: International conference on medical image computing and computer-assisted intervention

Havaei, M., Guizard, N., Chapados, N., Bengio, Y.: Hemis: Hetero-modal im- age segmentation. In: International conference on medical image computing and computer-assisted intervention. pp. 469–477. Springer (2016)

2016
[6]

In: International Conference on Machine Learning

Javaloy, A., Meghdadi, M., Valera, I.: Mitigating modality collapse in multimodal vaes via impartial optimization. In: International Conference on Machine Learning. pp. 9938–9964. PMLR (2022)

2022
[7]

arXiv preprint arXiv:2405.07930 (2024)

Kontras, K., Chatzichristos, C., Blaschko, M., De Vos, M.: Improving multimodal learning with multi-loss gradient modulation. arXiv preprint arXiv:2405.07930 (2024)

Pith/arXiv arXiv 2024
[8]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Li, H., Li, X., Hu, P., Lei, Y., Li, C., Zhou, Y.: Boosting multi-modal model performance with adaptive gradient modulation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22214–22224 (2023)

2023
[9]

Li, S., Du, C., Zhao, Y., Huang, Y., Zhao, H.: What makes for robust multi-modal models in the face of missing modalities? arXiv preprint arXiv:2310.06383 (2023)

Pith/arXiv arXiv 2023
[10]

In: Conference on robot learning

Liu, G.H., Siravuru, A., Prabhakar, S., Veloso, M., Kantor, G.: Learning end-to- end multimodal sensor policies for autonomous navigation. In: Conference on robot learning. pp. 249–261. PMLR (2017)

2017
[11]

In: Proceedings of the 22nd international conference on Machine learning

Mannor, S., Peleg, D., Rubinstein, R.: The cross entropy method for classification. In: Proceedings of the 22nd international conference on Machine learning. pp. 561– 568 (2005) 10 NB. Nizam et al

2005
[12]

Journal of Machine Learning Research23(43), 1–46 (2022)

Mitchell, R., Cooper, J., Frank, E., Holmes, G.: Sampling permutations for shapley value estimation. Journal of Machine Learning Research23(43), 1–46 (2022)

2022
[13]

arXiv preprint arXiv:2101.03279 (2021)

Mohta, A., Chou, F.C., Becker, B.C., Vallespi-Gonzalez, C., Djuric, N.: Investi- gating the effect of sensor modalities in multi-sensor detection-prediction models. arXiv preprint arXiv:2101.03279 (2021)

Pith/arXiv arXiv 2021
[14]

IEEE Transactions on Pattern Analysis and Machine Intelli- gence38(8), 1692–1706 (2015)

Neverova, N., Wolf, C., Taylor, G., Nebout, F.: Moddrop: adaptive multi-modal gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelli- gence38(8), 1692–1706 (2015)

2015
[15]

Transactions on machine learning research2025, 4468 (2025)

Nguyen,M.,Karaman,B.K.,Kim,H.,Wang,A.Q.,Liu,F.,Sabuncu,M.R.:Knock- out: A simple way to handle missing inputs. Transactions on machine learning research2025, 4468 (2025)

2025
[16]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Novosad, P., Carano, R.A., Krishnan, A.P.: A task-conditional mixture-of-experts model for missing modality segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 34–43. Springer (2024)

2024
[17]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Peng, X., Wei, Y., Deng, A., Wang, D., Hu, D.: Balanced multimodal learning via on-the-fly gradient modulation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8238–8247 (2022)

2022
[18]

The Lancet Oncology25(7), 879–887 (2024)

Saha, A., Bosma, J.S., Twilt, J.J., Van Ginneken, B., Bjartell, A., Padhani, A.R., Bonekamp, D., Villeirs, G., Salomon, G., Giannarini, G., et al.: Artificial intelli- gence and radiologists in prostate cancer detection on mri (pi-cai): an international, paired, non-inferiority, confirmatory study. The Lancet Oncology25(7), 879–887 (2024)

2024
[19]

Journal of the American Statistical Association88(421), 144–152 (1993)

Sasieni, P.: Maximum weighted partial likelihood estimators for the cox model. Journal of the American Statistical Association88(421), 144–152 (1993)

1993
[20]

IEEE journal of biomedical and health informatics28(1), 379–390 (2023)

Shi, J., Yu, L., Cheng, Q., Yang, X., Cheng, K.T., Yan, Z.: Mftrans: Modality- masked fusion transformer for incomplete multi-modality brain tumor segmenta- tion. IEEE journal of biomedical and health informatics28(1), 379–390 (2023)

2023
[21]

Scientific Reports 15(1), 29057 (2025)

Uddin, M.S., Ahmed, A., Aktarujjaman, M., Moniruzzaman, M., Ahmed, M., Mridha, M., Hossen, M.J.: A hybrid reinforcement learning and knowledge graph framework for financial risk optimization in healthcare systems. Scientific Reports 15(1), 29057 (2025)

2025
[22]

Handbook of game theory with economic applica- tions3, 2025–2054 (2002)

Winter, E.: The shapley value. Handbook of game theory with economic applica- tions3, 2025–2054 (2002)

2025
[23]

arXiv preprint arXiv:2506.11849 (2025)

Witter, R.T., Liu, Y., Musco, C.: Regression-adjusted monte carlo estimators for shapley values and probabilistic values. arXiv preprint arXiv:2506.11849 (2025)

arXiv 2025
[24]

Advances in neural information processing systems31(2018)

Wu, M., Goodman, N.: Multimodal generative models for scalable weakly- supervised learning. Advances in neural information processing systems31(2018)

2018
[25]

arXiv preprint arXiv:2409.07825 (2024)

Wu, R., Wang, H., Chen, H.T., Carneiro, G.: Deep multimodal learning with miss- ing modality: A survey. arXiv preprint arXiv:2409.07825 (2024)

Pith/arXiv arXiv 2024
[26]

In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Xu, M., Zhu, Z., Li, Y., Zheng, S., Zhao, Y., He, K., Zhao, Y.: Flexcare: Leveraging cross-task synergy for flexible multimodal healthcare prediction. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 3610–3620 (2024)

2024
[27]

Advances in Neural Information Processing Systems37, 62108–62122 (2024)

Yang, Y., Wan, F., Jiang, Q.Y., Xu, Y.: Facilitating multimodal classification via dynamically learning modality gap. Advances in Neural Information Processing Systems37, 62108–62122 (2024)

2024
[28]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Zhang, X., Yoon, J., Bansal, M., Yao, H.: Multimodal representation learning by alternating unimodal adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 27456–27466 (2024)

2024

[1] [1]

In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention

Cui, C., Liu, H., Liu, Q., Deng, R., Asad, Z., Wang, Y., Zhao, S., Yang, H., Land- man, B.A., Huo, Y.: Survival prediction of brain cancer with incomplete radiology, pathology, genomic, and demographic data. In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention. pp. 626–635. Springer (2022)

2022

[2] [2]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Dorent, R., Joutard, S., Modat, M., Ourselin, S., Vercauteren, T.: Hetero-modal variational encoder-decoder for joint modality completion and segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 74–82. Springer (2019)

2019

[3] [3]

The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development pp

Enders, C.K., Baraldi, A.N.: Missing data handling methods. The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development pp. 139–185 (2018)

2018

[4] [4]

Advances in Neural Information Processing Systems 37, 133328–133344 (2024)

Guo, Z., Jin, T., Chen, J., Zhao, Z.: Classifier-guided gradient modulation for en- hanced multimodal learning. Advances in Neural Information Processing Systems 37, 133328–133344 (2024)

2024

[5] [5]

In: International conference on medical image computing and computer-assisted intervention

Havaei, M., Guizard, N., Chapados, N., Bengio, Y.: Hemis: Hetero-modal im- age segmentation. In: International conference on medical image computing and computer-assisted intervention. pp. 469–477. Springer (2016)

2016

[6] [6]

In: International Conference on Machine Learning

Javaloy, A., Meghdadi, M., Valera, I.: Mitigating modality collapse in multimodal vaes via impartial optimization. In: International Conference on Machine Learning. pp. 9938–9964. PMLR (2022)

2022

[7] [7]

arXiv preprint arXiv:2405.07930 (2024)

Kontras, K., Chatzichristos, C., Blaschko, M., De Vos, M.: Improving multimodal learning with multi-loss gradient modulation. arXiv preprint arXiv:2405.07930 (2024)

Pith/arXiv arXiv 2024

[8] [8]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Li, H., Li, X., Hu, P., Lei, Y., Li, C., Zhou, Y.: Boosting multi-modal model performance with adaptive gradient modulation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22214–22224 (2023)

2023

[9] [9]

Li, S., Du, C., Zhao, Y., Huang, Y., Zhao, H.: What makes for robust multi-modal models in the face of missing modalities? arXiv preprint arXiv:2310.06383 (2023)

Pith/arXiv arXiv 2023

[10] [10]

In: Conference on robot learning

Liu, G.H., Siravuru, A., Prabhakar, S., Veloso, M., Kantor, G.: Learning end-to- end multimodal sensor policies for autonomous navigation. In: Conference on robot learning. pp. 249–261. PMLR (2017)

2017

[11] [11]

In: Proceedings of the 22nd international conference on Machine learning

Mannor, S., Peleg, D., Rubinstein, R.: The cross entropy method for classification. In: Proceedings of the 22nd international conference on Machine learning. pp. 561– 568 (2005) 10 NB. Nizam et al

2005

[12] [12]

Journal of Machine Learning Research23(43), 1–46 (2022)

Mitchell, R., Cooper, J., Frank, E., Holmes, G.: Sampling permutations for shapley value estimation. Journal of Machine Learning Research23(43), 1–46 (2022)

2022

[13] [13]

arXiv preprint arXiv:2101.03279 (2021)

Mohta, A., Chou, F.C., Becker, B.C., Vallespi-Gonzalez, C., Djuric, N.: Investi- gating the effect of sensor modalities in multi-sensor detection-prediction models. arXiv preprint arXiv:2101.03279 (2021)

Pith/arXiv arXiv 2021

[14] [14]

IEEE Transactions on Pattern Analysis and Machine Intelli- gence38(8), 1692–1706 (2015)

Neverova, N., Wolf, C., Taylor, G., Nebout, F.: Moddrop: adaptive multi-modal gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelli- gence38(8), 1692–1706 (2015)

2015

[15] [15]

Transactions on machine learning research2025, 4468 (2025)

Nguyen,M.,Karaman,B.K.,Kim,H.,Wang,A.Q.,Liu,F.,Sabuncu,M.R.:Knock- out: A simple way to handle missing inputs. Transactions on machine learning research2025, 4468 (2025)

2025

[16] [16]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Novosad, P., Carano, R.A., Krishnan, A.P.: A task-conditional mixture-of-experts model for missing modality segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 34–43. Springer (2024)

2024

[17] [17]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Peng, X., Wei, Y., Deng, A., Wang, D., Hu, D.: Balanced multimodal learning via on-the-fly gradient modulation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8238–8247 (2022)

2022

[18] [18]

The Lancet Oncology25(7), 879–887 (2024)

Saha, A., Bosma, J.S., Twilt, J.J., Van Ginneken, B., Bjartell, A., Padhani, A.R., Bonekamp, D., Villeirs, G., Salomon, G., Giannarini, G., et al.: Artificial intelli- gence and radiologists in prostate cancer detection on mri (pi-cai): an international, paired, non-inferiority, confirmatory study. The Lancet Oncology25(7), 879–887 (2024)

2024

[19] [19]

Journal of the American Statistical Association88(421), 144–152 (1993)

Sasieni, P.: Maximum weighted partial likelihood estimators for the cox model. Journal of the American Statistical Association88(421), 144–152 (1993)

1993

[20] [20]

IEEE journal of biomedical and health informatics28(1), 379–390 (2023)

Shi, J., Yu, L., Cheng, Q., Yang, X., Cheng, K.T., Yan, Z.: Mftrans: Modality- masked fusion transformer for incomplete multi-modality brain tumor segmenta- tion. IEEE journal of biomedical and health informatics28(1), 379–390 (2023)

2023

[21] [21]

Scientific Reports 15(1), 29057 (2025)

Uddin, M.S., Ahmed, A., Aktarujjaman, M., Moniruzzaman, M., Ahmed, M., Mridha, M., Hossen, M.J.: A hybrid reinforcement learning and knowledge graph framework for financial risk optimization in healthcare systems. Scientific Reports 15(1), 29057 (2025)

2025

[22] [22]

Handbook of game theory with economic applica- tions3, 2025–2054 (2002)

Winter, E.: The shapley value. Handbook of game theory with economic applica- tions3, 2025–2054 (2002)

2025

[23] [23]

arXiv preprint arXiv:2506.11849 (2025)

Witter, R.T., Liu, Y., Musco, C.: Regression-adjusted monte carlo estimators for shapley values and probabilistic values. arXiv preprint arXiv:2506.11849 (2025)

arXiv 2025

[24] [24]

Advances in neural information processing systems31(2018)

Wu, M., Goodman, N.: Multimodal generative models for scalable weakly- supervised learning. Advances in neural information processing systems31(2018)

2018

[25] [25]

arXiv preprint arXiv:2409.07825 (2024)

Wu, R., Wang, H., Chen, H.T., Carneiro, G.: Deep multimodal learning with miss- ing modality: A survey. arXiv preprint arXiv:2409.07825 (2024)

Pith/arXiv arXiv 2024

[26] [26]

In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Xu, M., Zhu, Z., Li, Y., Zheng, S., Zhao, Y., He, K., Zhao, Y.: Flexcare: Leveraging cross-task synergy for flexible multimodal healthcare prediction. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 3610–3620 (2024)

2024

[27] [27]

Advances in Neural Information Processing Systems37, 62108–62122 (2024)

Yang, Y., Wan, F., Jiang, Q.Y., Xu, Y.: Facilitating multimodal classification via dynamically learning modality gap. Advances in Neural Information Processing Systems37, 62108–62122 (2024)

2024

[28] [28]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Zhang, X., Yoon, J., Bansal, M., Yao, H.: Multimodal representation learning by alternating unimodal adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 27456–27466 (2024)

2024