REVIEW 4 major objections 6 minor 30 references
A Step towards Interpretable Multimodal AI Models with MultiFIX
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read MultiFIX replaces the opaque parts of a multimodal deep-learning model with readable symbolic expressions, and on synthetic problems the resulting explainable model matches or beats the black box's balanced accuracy.
desk verdict The training-strategy comparison is solid, but the central claim that MultiFIX explains the DL model's features is under-supported by the missing fidelity checks. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is sparse feature engineering plus symbolic replacement. Each modality is reduced to a bottleneck of at most three real-valued features; the tabular features and the fusion block are then approximated by tiny symbolic expressions (trees of arithmetic, Boolean, and if-then-else operators) evolved by GP-GOMEA, while Grad-CAM visualizes which image regions drive each image feature. The narrow bottleneck is what makes fusion-level symbolic regression feasible and readable.
What would settle it
Run MultiFIX on a synthetic multimodal problem with a known hidden rule, give the user only the recovered expressions, and check whether they can correctly answer a question the intended rule answers. The AND and XOR results show the model can be right while describing a different, inverted logic; if a non-expert reader cannot tell whether the intended rule was actually learned, the claim that MultiFIX accurately explains the model's integration of modalities is not established.
Extended reading notes
Core claim
The paper claims that multimodal predictions can be made interpretable by design without sacrificing accuracy, provided each modality is forced through a sparse bottleneck of at most three engineered features. On problems built from synthetic images of shapes and ten tabular variables, the DL model's tabular features and its fusion were replaced with compact symbolic expressions (using GP-GOMEA), while image features were explained post-hoc with Grad-CAM. For the AND, XOR, Multifeature, and Multiclass problems, the resulting interpretable model matched or improved on the DL model's balanced accuracy, by up to 0.075. The authors also show that the recovered expressions may encode equivalent-but-inverted logic: e.g., a learned tabular feature was the complement of the intended $x_1 > x_2$, yet the model remained correct. They interpret this as evidence that component-wise explanation lets a user see what the model actually learned, not just that it predicts well.
Load-bearing premise
The claim depends on the assumption that a handful of symbolic expressions can faithfully stand in for the deep-learning feature extractors; if the extracted features are complex, the paper itself shows the recovering expressions can become hard to interpret, and the explanation then no longer explains much.
Editorial extensions
If this is right
- Interpretable models can be obtained from DL-trained multimodal pipelines without a performance penalty, at least on problems whose true feature structure is simple and known.
- Component-wise inspection can reveal that the model uses a different but equally valid encoding than the intended one, turning equivalent-model discovery from a surprise into a feature of the audit trail.
- Training strategy matters less than problem structure; end-to-end, hybrid, and sequential variants were mostly not significantly different, so practitioners can pick based on available pretrained blocks.
- The hard case is when a modality needs more than one complex feature: the Multifeature problem produced expressions that are transparent but not easy to interpret, motivating complexity penalties in future work.
Reading between the lines
- The same replacement strategy could be applied to other modality blocks—text or time series encoders—as long as the bottleneck stays small enough for symbolic regression to fit; on current evidence there is nothing image- or table-specific about the mechanism.
- A controlled ablation that keeps the DL fusion but uses symbolic features would separate two effects the paper conflates: the gains on XOR and Multiclass (0.035 and 0.075 BAcc) may come from GP's regularization rather than from interpretability itself.
- The observed equivalence phenomenon suggests a concrete diagnostic test for trust: if the symbolic expressions and a known ground-truth rule are equivalent-but-different, the model's reasoning is correct but non-unique, and a deployment decision should be reviewed by a domain expert.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents MultiFIX, a multimodal fusion pipeline that first trains a deep learning model with small bottlenecks for image and tabular features, then replaces the tabular feature-extraction block and the fusion block with symbolic expressions evolved by GP-GOMEA, while explaining image features with Grad-CAM. Experiments on four synthetic problems (AND, XOR, Multifeature, Multiclass) compare balanced accuracy across six training strategies and report interpretable models for the best-performing strategy per problem. The paper claims that MultiFIX can generate multimodal models that accurately explain both the extracted features and their integration without compromising predictive performance.
Significance. If the fidelity of the GP surrogates to the DL blocks were established, the combination of a sparse feature bottleneck with GP symbolic expressions would be a genuinely useful step toward interpretable multimodal modeling, particularly because the synthetic problems have known ground truth and the paper examines several training strategies. The paper is honest about several limitations, including the difficulty of interpreting complex image features and the post-hoc nature of Grad-CAM. However, the central interpretability claim is not yet supported by the reported evidence, since no fidelity metric is provided between the GP expressions and the DL blocks they replace, and the interpretable-model results are reported for only the best training strategy without variance.
major comments (4)
- [3.5, 4.1.3, 4.2.3, 4.3.3, 4.4.3] The central interpretability claim is unsupported because the GP symbolic expressions are never checked for fidelity to the DL blocks they replace. The evaluation only compares the final explainable model's balanced accuracy with the DL model's balanced accuracy (e.g., AND: 0.967 vs 0.973; XOR: 0.975 vs 0.940; Multifeature: 0.884 vs 0.864; Multiclass: 0.995 vs 0.920). A high-BAcc explainable model could be explaining the data-generating rule rather than the original DL model, especially since Sections 4.1.3 and 4.2.3 report equivalent models with inverted intermediate features. The authors should quantify the agreement between each GP expression and the corresponding DL block output on held-out data (e.g., R^2 or mean absolute error for tabular features, and accuracy or BAcc for the fusion expression), and report how often the final explainable model's predictions match the DL model's predictions.
- [4.1-4.4] The interpretable-model results are reported only for the single best-performing training strategy per problem, with point estimates and no variance. For the AND problem, the explainable model's BAcc is reported as 0.967 vs the DL model's 0.973, but the DL results in Table 3 have standard deviations around 0.02-0.04, and no standard deviation is given for the explainable model. This selection and the lack of variance make it impossible to assess the robustness of the 'no performance compromise' claim. The authors should report the explainable model's performance across all folds and, ideally, for all training strategies, not only the best one.
- [4.3.3 and 5] The manuscript itself acknowledges that in the Multifeature problem the intermediate image features are 'not easy to interpret' and that the fusion expression with depth three is 'not easy to interpret' (Section 4.3.3), and that Grad-CAM's post-hoc nature limits symbolic meaning (Section 5). This directly undercuts the conclusion that MultiFIX produces interpretable models for that problem. The authors should provide a concrete interpretability metric or a clear protocol for deciding when an obtained expression counts as interpretable, and discuss how the Multifeature result is compatible with the abstract's claim.
- [3.6, Tables 3-7] The statistical comparison of training strategies relies on paired t-tests over 5 folds. With n=5, the tests have very low power, and the paper interprets 'not significant' as evidence of similar performance, which is not justified. The authors should either report effect sizes with confidence intervals, use a resampling-based procedure such as the bootstrap or the corrected repeated k-fold CV test, or clearly state the power limitations of the comparison.
minor comments (6)
- [Table 1] The operator list 'numeric[+,−,∗,/,.2,.3,]]' is malformed; the intended numeric operators and constants (e.g., 0.2 and 0.3) are unclear and should be stated without stray brackets.
- [3.5] Clarify whether the GP symbolic expressions for tabular features are fitted to the DL-extracted feature values using the raw tabular inputs as independent variables, and whether the fusion GP uses the DL-extracted or the GP-reconstructed features.
- [3.6] The text states that HPO chooses the learning rate, weight decay, and 'number of extracted features (up to three) for the image and tabular inputs,' but Table 2 lists Image Bottleneck and Tabular Bottleneck; specify whether these are the same quantities.
- [5] The word 'MutiFIX' appears as a typo in the first paragraph of Section 5; it should be 'MultiFIX.'
- [Figures 3 and 4] The symbolic expressions in the figures are too small to read in the manuscript; consider displaying them in a larger font or presenting them in the text as well.
- [3.7] Clarify how the 1,000 samples are reused across the four problems and whether the same image/tabular pairs are used in each problem, since this affects the comparability of the reported results.
Circularity Check
No significant circularity: the BAcc results are externally benchmarked against held-out labels, and the GP surrogate fidelity gap is a correctness risk rather than a definitional reduction.
full rationale
Walking the claimed derivation chain, MultiFIX trains DL feature blocks, then fits GP-GOMEA expressions to the tabular feature outputs and to the final prediction, replacing those blocks. The final explainable model is evaluated by comparing its balanced accuracy on held-out validation folds to the DL model's balanced accuracy. This evaluation is benchmarked against external ground-truth labels, not against quantities defined by the paper's own construction. The abstract's interpretability claim is not backed by a reported fidelity metric between the GP expressions and the DL blocks they replace, and Sections 4.1.3/4.2.3 explicitly acknowledge that the learned intermediate features can be inverted complements of the intended features, so the explanation may describe an equivalent model rather than the exact DL internals. These are genuine correctness and validity concerns, but they are not circularity: neither the GP expression nor the BAcc score is defined in terms of the target claim, and no equation in the paper reduces a prediction to a fitted input. The self-citations ([14] for preliminary MultiFIX, [18] for the GP-GOMEA adaptation, [24]/[25] for background) are routine method and preliminary citations and are not load-bearing; the paper's experimental results are self-contained. Therefore no significant circularity is identified.
Assumptions & free parameters
free parameters (5)
- Image bottleneck size =
0-3 (selected by HPO)
- Tabular bottleneck size =
0-3 (selected by HPO)
- GP maximum tree depth =
2 or 3
- GP population size =
64 initially (IMS)
- Adam learning rate and weight decay =
grid searched over {1e-3,1e-4,1e-5} and {1e-3,1e-4,0}
assumptions (4)
- domain assumption GP-GOMEA can find compact symbolic expressions that faithfully approximate the DL feature blocks and fusion block.
- domain assumption The synthetic dataset with shapes and tabular features is representative of real-world multimodal dependencies.
- domain assumption Grad-CAM heatmaps correctly indicate which image regions drive each extracted feature.
- standard math A paired t-test on the 5 folds, with Bonferroni correction, is a valid statistical comparison.
Cite this review
Pith. "Pith review of A Step towards Interpretable Multimodal AI Models with MultiFIX." pith.science (2026). https://pith.science/paper/XJDHT6EW
@misc{pith2026250511262,
author = {Pith},
title = {Pith review of: A Step towards Interpretable Multimodal AI Models with MultiFIX},
year = {2026},
howpublished = {\url{https://pith.science/paper/XJDHT6EW}},
note = {Machine review of arXiv:2505.11262}
}
read the original abstract
Real-world problems are often dependent on multiple data modalities, making multimodal fusion essential for leveraging diverse information sources. In high-stakes domains, such as in healthcare, understanding how each modality contributes to the prediction is critical to ensure trustworthy and interpretable AI models. We present MultiFIX, an interpretability-driven multimodal data fusion pipeline that explicitly engineers distinct features from different modalities and combines them to make the final prediction. Initially, only deep learning components are used to train a model from data. The black-box (deep learning) components are subsequently either explained using post-hoc methods such as Grad-CAM for images or fully replaced by interpretable blocks, namely symbolic expressions for tabular data, resulting in an explainable model. We study the use of MultiFIX using several training strategies for feature extraction and predictive modeling. Besides highlighting strengths and weaknesses of MultiFIX, experiments on a variety of synthetic datasets with varying degrees of interaction between modalities demonstrate that MultiFIX can generate multimodal models that can be used to accurately explain both the extracted features and their integration without compromising predictive performance.
Figures
Reference graph
Works this paper leans on
-
[14]
Mafalda Malafaia, Thalea Schlender, Peter AN Bosman, and Tanja Alderliesten
-
[1]
Johannes Allgaier, Lena Mulansky, Rachel Draelos, and Rüdiger Pryss. 2023. How does the model make predictions? A systematic literature review on the explainability power of machine learning in healthcare.Artificial Intelligence in Medicine143 (09 2023), 102616. https://doi.org/10.1016/j.artmed.2023.102616
arXiv 2023
-
[2]
Nitin Arora, Anupam Singh, Vivek Shahare, and Goutam Datta. 2023. Introduction to Big Data Analytics. InTowards the Integration of IoT, Cloud and Big Data: Services, Applications and Standards. Springer, 1–18
work page 2023
-
[3]
Jaume Bacardit, Alexander EI Brownlee, Stefano Cagnoni, Giovanni Iacca, John McCall, and David Walker. 2022. The intersection of evolutionary computation and explainable AI. InProceedings of the Genetic and Evolutionary Computation conference companion. 1757–1762
work page 2022
-
[4]
Richard J Chen, Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Jana Lipkova, Zahra Noor, Muhammad Shaban, Maha Shady, Mane Williams, Bumjin Joo, et al
-
[5]
Benjamin P Evans, Bing Xue, and Mengjie Zhang. 2019. What’s inside the black- box? a genetic programming method for interpreting complex machine learning models. InProceedings of the genetic and evolutionary computation conference. 1012–1020
work page 2019
-
[6]
Jacob Gildenblat and contributors. 2021. PyTorch library for CAM methods. https://github.com/jacobgil/pytorch-grad-cam
work page 2021
-
[7]
Valerio Guarrasi, Fatih Aksu, Camillo Maria Caruso, Francesco Di Feola, Aurora Rofena, Filippo Ruffini, and Paolo Soda. 2024. A Systematic Review of Interme- diate Fusion in Multimodal Deep Learning for Biomedical Applications.arXiv preprint arXiv:2408.02686(2024)
arXiv 2024
Show all 30 references
-
[8]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770–778. https://doi.org/10.1109/CVPR.2016.90
2016 doi
-
[9]
Shih-Cheng Huang, Anuj Pareek, Saeed Seyyedi, Imon Banerjee, and Matthew P Lungren. 2020. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines.NPJ digital medicine3, 1 (2020), 136
2020
-
[10]
Gargi Joshi, Rahee Walambe, and Ketan Kotecha. 2021. A review on explainability in multimodal deep neural nets.IEEE Access9 (2021), 59800–59821
2021
-
[11]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Opti- mization.International Conference on Learning Representations (ICLR)(2015)
2015
-
[12]
Adrienne Kline, Hanyin Wang, Yikuan Li, Saya Dennis, Meghan Hutch, Zhenxing Xu, Fei Wang, Feixiong Cheng, and Yuan Luo. 2022. Multimodal machine learning in precision health: A scoping review.npj Digital Medicine5, 1 (2022), 171
2022
-
[13]
William La Cava, Patryk Orzechowski, Bogdan Burlacu, Fabricio de Franca, Marco Virgolin, Ying Jin, Michael Kommenda, and Jason Moore. 2021. Contemporary Symbolic Regression Methods and their Relative Performance. InProceedings of the Neural Information Processing Systems Track...
2021
-
[15]
Anil Rahate, Rahee Walambe, Sheela Ramanna, and Ketan Kotecha. 2022. Multi- modal co-learning: Challenges, applications with datasets, recent advances and future directions.Information Fusion81 (2022), 203–239
2022
-
[16]
Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature machine intelligence1, 5 (2019), 206–215
2019
-
[17]
Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. 2022. Interpretable machine learning: Fundamental principles and 10 grand challenges.Statistic Surveys16 (2022), 1–85
2022
-
[18]
Thalea Schlender, Mafalda Malafaia, Tanja Alderliesten, and Peter Bosman. 2024. Improving the efficiency of GP-GOMEA for higher-arity operators. InProceedings of the Genetic and Evolutionary Computation Conference. 971–979
2024
-
[19]
Daan Schouten, Giulia Nicoletti, Bas Dille, Catherine Chia, Pierpaolo Vendittelli, Megan Schuurmans, Geert Litjens, and Nadieh Khalili. 2024. Navigating the landscape of multimodal AI in medicine: a scoping review on technical challenges and clinical applications.arXiv preprin...
2024 arXiv
-
[20]
Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2019. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization.International Journal of Com- puter Vision128, 2 (2019), 336–359. https://doi....
2019 doi
-
[21]
William C Sleeman, Rishabh Kapoor, and Preetam Ghosh. 2022. Multimodal classification: Current landscape, taxonomy and future directions.Comput. Surveys55, 7 (2022), 1–31
2022
-
[22]
Sören Richard Stahlschmidt, Benjamin Ulfenborg, and Jane Synnergren. 2022. Multimodal deep learning for biomedical data fusion: a review.Briefings in Bioinformatics23, 2 (2022), bbab569
2022
-
[23]
Vinitra Swamy, Malika Satayeva, Jibril Frej, Thierry Bossy, Thijs Vogels, Martin Jaggi, Tanja Käser, and Mary-Anne Hartley. 2024. Multimodn—multimodal, multi- task, interpretable modular networks.Advances in Neural Information Processing Systems36 (2024)
2024
-
[24]
Marco Virgolin, Tanja Alderliesten, and Peter AN Bosman. 2020. On explaining machine learning models by evolving crucial and compact features.Swarm and Evolutionary Computation53 (2020), 100640
2020
-
[25]
Marco Virgolin, Tanja Alderliesten, Cees Witteveen, and Peter A. N. Bosman
-
[26]
Fei Zhao, Chengcui Zhang, and Baocheng Geng. 2024. Deep Multimodal Data Fusion.Comput. Surveys56, 9 (2024), 1–36
2024
-
[27]
Ryan Zhou and Ting Hu. 2023. Evolutionary approaches to explainable machine learning. InHandbook of Evolutionary Machine Learning. Springer, 487–506. 9
2023
-
[2021]
Improving model-based genetic programming for symbolic regression of small expressions.Evolutionary Computation29, 2 (2021), 211–237
2021
-
[2022]
Pan-cancer integrative histology-genomic analysis via multimodal deep learning.Cancer Cell40, 8 (2022), 865–878
2022
-
[2024]
MultiFIX: An XAI-friendly feature inducing approach to building models from multimodal data.arXiv preprint arXiv:2402.12183(2024)
2024 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.