REVIEW 5 major objections 5 minor 39 references
MUDI: A Multimodal Biomedical Dataset for Understanding Pharmacodynamic Drug-Drug Interactions
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper introduces MUDI, a 310,532-pair dataset that labels drug combinations as Synergism, Antagonism, or New Effect using pharmacodynamic evidence across multiple modalities, with a test split built to measure generalization to…
desk verdict MUDI is a valuable, open multimodal dataset whose pharmacodynamic claim is undercut by its own PK-labeled templates; this is a fixable major-revision, not a rejection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the set of 241 masked sentence templates and the labels assigned to them. Free-text interaction descriptions are aligned to drug mentions, masked as [DRUG1] and [DRUG2], clustered by recurring wording into templates, classified by lexical heuristics into Synergism, Antagonism, or New Effect, and finally reviewed by two physicians. New Effect is distinctive: it survives only if the effect term cannot be found in either drug's profile. Every one of the 310,532 labeled pairs is a drug-specific instance of one of these 241 templates, so the quality of the whole dataset rests on the correctness of the template labels.
What would settle it
Audit the 241 masked templates for pharmacokinetic wording such as increased absorption, decreased excretion, faster or slower metabolism, and elevated or reduced serum concentration, and count how many Synergism and Antagonism labels are produced by those templates. If a substantial share of the 310,532 labels trace back to templates that the paper's Appendix C.3 excludes from pharmacodynamics, the dataset's pharmacodynamic-scope claim is falsified; if instead almost all template labels are strictly effect-level, the claim holds.
Extended reading notes
Core claim
The central claim is that MUDI is a valid benchmark for pharmacodynamic drug-drug interactions. To build it, the authors take a curated list of approved human drugs, extract their interaction descriptions, align drug mentions, mask them into placeholder templates, and collapse 244,921 raw descriptions into 241 distinct sentence templates. Lexical rules assign each template to Synergism, Antagonism, or New Effect; New Effect candidates are checked against each drug's own side-effect profile and reassigned to Synergism when the effect is already known for one drug; two physicians then review and correct all 241 template labels. The resulting 310,532 instances have a class distribution approximating reported clinical frequency (82.85% Synergism, 12.63% Antagonism, 4.52% New Effect), and 95.42% of test pairs contain at least one drug never seen in training. The authors state that no existing DDI dataset combines this multimodal coverage with directed pharmacodynamic labels and a generalization-oriented split.
Load-bearing premise
The load-bearing assumption is that each of the 241 masked sentence templates, classified by lexical rules and reviewed by two physicians, transfers a correct pharmacodynamic label to every drug pair that matches it; if a template is wrong, or describes a pharmacokinetic mechanism outside the stated scope, the thousands of pairs inheriting that label inherit the error.
Editorial extensions
If this is right
- Off-the-shelf multimodal models can use MUDI as a benchmark, and the reported baselines give future work a target: intermediate fusion reaches 66.69% micro-F1 under direction-agnostic matching versus 64.67% for late fusion.
- Direction-aware evaluation is substantially harder than direction-agnostic evaluation, with micro-F1 falling from 66.69% to 52.74% for intermediate fusion, so models that capture interaction asymmetry will be needed for clinically faithful prediction.
- New Effect is the least predictable class, with best intermediate-fusion F1 of 49.00% in the direction-agnostic setting, exposing detection of genuinely novel combination effects as an open problem.
- Molecular structure graphs are the strongest single modality at 65.44% micro-F1 direction-agnostic, indicating that topological molecular information carries much of the pharmacodynamic signal.
- Because most test pairs contain at least one drug absent from training, MUDI enables at least partial evaluation of zero-shot generalization for unseen drugs.
Reading between the lines
- The paper leaves implicit that its Synergism template set includes pharmacokinetic wording such as increased absorption, decreased excretion, and elevated serum concentration, even though an appendix declares the dataset pharmacodynamic-only; an audit of the 241 templates would quantify how much of the 82.85% Synergism share rests on such wording.
- With 88.15% of test pairs having exactly one training-seen drug and only 7.27% having neither, the reported numbers blend a semi-supervised regime with a true zero-shot regime; separating these two test subsets would clarify what 'generalization' means in this benchmark.
- Because labels are mass-produced from templates, a model might learn template artifacts rather than pharmacology; conditioning on template identity during evaluation or ablating by template cardinality would test whether fusion gains are confounded by label source.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents MUDI, a large-scale multimodal dataset of 310,532 drug-drug interaction (DDI) instances over 1,295 drugs, with three labels (Synergism, Antagonism, New Effect) and five drug modalities (text, SMILES, chemical formula, molecular graph, and structure image). The construction pipeline starts from DrugBank interaction sentences, masks drug mentions to obtain 241 sentence templates, applies lexical rules to assign labels, and then has two physicians review the template labels. The authors also provide unimodal and multimodal (late-fusion and intermediate-fusion) baselines, evaluate under direction-aware and direction-agnostic settings, and release the data, code, and evaluation scripts. The central claim is that MUDI is the first large-scale pharmacodynamic DDI benchmark with directed labels and multimodal inputs.
Significance. If the labels were valid, MUDI would be a useful resource: it is large, openly released with preprocessing code and evaluation scripts, includes a test split with a substantial fraction of unseen drug pairs, and covers multiple modalities with standardized generation protocols. The direction-aware labeling and the explicit unseen-drug evaluation are also valuable design choices for benchmarking generalization in biomedical machine learning. The reproducibility-oriented appendices and public code release are genuine strengths. However, the dataset's foundational claim—that the labels encode pharmacodynamic interactions—is directly contradicted by the paper's own labeling rules, which include pharmacokinetic ADME mechanisms. Because the labels are mechanically transferred from a small number of templates to all matching instances, this contradiction affects the validity of the entire label set and cannot be treated as a minor scope clarification.
major comments (5)
- [Section 3.2; Appendix A.2; Appendix C.3] The labeling rules in Appendix A.2 explicitly assign Synergism and Antagonism to sentences about increased/decreased absorption, decreased/increased excretion, decreased/increased metabolism, increased/decreased bioavailability, decreased protein binding, and increased/decreased serum concentration. These are pharmacokinetic (ADME) parameter changes, not pharmacodynamic outcomes. This directly contradicts Appendix C.3, which states that MUDI 'does not cover pharmacokinetic DDIs such as those involving absorption, distribution, metabolism, or excretion (ADME) pathways.' Since Section 3.2 transfers template labels to all matching drug pairs, the central claim that MUDI is a pharmacodynamic DDI dataset is not supported as written. The authors must either reannotate all instances derived from ADME templates or revise the dataset's scope, title, and contribution claims accordingly.
- [Section 3.2; Appendix A.2] Per-template instance counts are never reported. The 241 masked sentence templates are the de facto labeling rules: every one of the 310,532 instances receives its label by matching one of these templates. Without reporting how many instances fall into each template, and in particular how many instances come from the pharmacokinetic templates listed in Appendix A.2, the magnitude of the scope violation cannot be quantified, and a reader cannot judge whether MUDI is even predominantly pharmacodynamic. The paper should report template frequencies and a stratified expert re-validation of instances from PK-related templates.
- [Section 4.1; Appendix F.3.4] The late-fusion tie-breaking priority order (graph→name→image→SMILES→formula→description) is selected based on the unimodal models' average F1 on the development set. This is a form of model selection on the benchmark's test-development pipeline. As written, the reported late-fusion results may be optimistically biased because the fusion rule was tuned to the same development data used to guide training. The paper should either fix the priority order a priori, use a nested or cross-validated selection procedure, or explicitly characterize this as a hyperparameter with its own selection variance.
- [Section 4.3; Appendix G.1, Table 6] The unimodal results in Table 6 show that the Description and SMILES modalities achieve zero F1 for Antagonism and New Effect in both evaluation settings, meaning these modalities never correctly predict the minority classes. Nevertheless, Section 4.3 states that 'textual fields (e.g., name and description) and images also show competitive results.' This is misleading: for Description, the macro-averaged F1 is 17.01% (direction-aware) and 22.90% (direction-agnostic), and the modality is effectively degenerate for two of the three positive classes. The main text should explicitly acknowledge these failure modes, since the fusion baselines are interpreted against these unimodal results.
- [Section 4.1; Appendix B.2; Appendix E] The benchmark protocol introduces 200,000 additional No Interaction pairs as negative examples during training, but the paper does not specify how these negatives are sampled, whether they are included in the released train/test CSV files, how duplicates or directionality are handled, or how they affect the direction-aware and direction-agnostic evaluation. Without this information, the reported benchmark numbers cannot be reproduced by other researchers. A precise description of the negative-sampling procedure and its interaction with the evaluation protocol is required for reproducibility.
minor comments (5)
- [Appendix D.1.2] The text says 'We apply global max pooling across nodes to produce a graph-level embedding,' but the displayed equation is z_graph = (1/n) Σ h_i, which is mean pooling, not max pooling. This inconsistency should be corrected.
- [Section 4.2; Appendix F.1] Section 4.2 states that 'All models are implemented in PyTorch 3.10,' but Appendix F.1 lists Python 3.10 and PyTorch 2.0.1. This is likely a typo and should be fixed.
- [Section 3.2; Appendix A.1] The paper reports 'near-perfect inter-annotator agreement' for the masking review but provides no numeric measure (e.g., Cohen's kappa or percentage agreement). A concrete number would strengthen the curation claim.
- [Figure 4] The heatmap in Figure 4 has modality names on both axes but no axis labels or colorbar explanation, making it harder to interpret the reported agreement rates. Adding explicit axis labels and a colorbar would improve clarity.
- [Section 5; Limitations] The limitations paragraph appropriately notes the absence of gold-standard clinical validation, but it does not mention the explicit pharmacokinetic templates in Appendix A.2. Given the dataset's pharmacodynamic scope claim, this specific limitation should be acknowledged in the main text.
Circularity Check
No significant circularity: MUDI's labels come from external DrugBank text and benchmarks are empirical, though the PD-only scope claim conflicts with PK-style templates in Appendix A.2.
full rationale
The paper's central construction is not circular. Labels are transferred from external DrugBank DDI sentences via masked templates and lexical rules (Section 3.2), and the model inputs exclude the DDI sentence text (Section 3.1), so predictions cannot reconstruct the labels from the training modalities by construction. The benchmark numbers are empirical outcomes of training on the fixed dataset, not quantities derived from the labels by a fitted equation. The only mild model-selection choice is the late-fusion tie-breaking priority (Appendix D.2.1), selected on development-set performance and then applied to the test set; this is standard, disclosed hyperparameter selection and does not make the test results a fitted artifact of the labels. Self-citation is not load-bearing: the pharmacological grounding cites textbooks [8, 23] and external resources [13, 30], and no uniqueness theorem or ansatz is imported from the authors' own prior work. One caveat is a scope/validity tension rather than circularity: the Synergism and Antagonism templates in Appendix A.2 include ADME-type expressions ("increased absorption", "decreased excretion rate", "serum concentration" changes, "metabolism" changes), while Appendix C.3 states that MUDI "does not cover pharmacokinetic DDIs such as those involving absorption, distribution, metabolism, or excretion (ADME) pathways." This is an internal consistency issue for the pharmacodynamic-scope claim, but the label-assignment procedure is still an external-data transformation, not a self-referential derivation, so it does not raise the circularity score.
Assumptions & free parameters
assumptions (4)
- domain assumption DrugBank DDI descriptions are a reliable source for pharmacodynamic interaction labeling.
- ad hoc to paper The 241 masked sentence templates preserve all label-relevant information and can be safely generalized to all 310,532 pairs.
- ad hoc to paper Lexical rules mapping concentration, absorption, and excretion changes to Synergism and Antagonism are pharmacodynamic effects.
- domain assumption The train/test split, with most test pairs sharing one drug with training, provides a valid generalization evaluation.
Cite this review
Pith. "Pith review of MUDI: A Multimodal Biomedical Dataset for Understanding Pharmacodynamic Drug-Drug Interactions." pith.science (2026). https://pith.science/paper/Z7ZQE6TR
@misc{pith2026250601478,
author = {Pith},
title = {Pith review of: MUDI: A Multimodal Biomedical Dataset for Understanding Pharmacodynamic Drug-Drug Interactions},
year = {2026},
howpublished = {\url{https://pith.science/paper/Z7ZQE6TR}},
note = {Machine review of arXiv:2506.01478}
}
read the original abstract
Understanding the interaction between different drugs (drug-drug interaction or DDI) is critical for ensuring patient safety and optimizing therapeutic outcomes. Existing DDI datasets primarily focus on textual information, overlooking multimodal data that reflect complex drug mechanisms. In this paper, we (1) introduce MUDI, a large-scale Multimodal biomedical dataset for Understanding pharmacodynamic Drug-drug Interactions, and (2) benchmark learning methods to study it. In brief, MUDI provides a comprehensive multimodal representation of drugs by combining pharmacological text, chemical formulas, molecular structure graphs, and images across 310,532 annotated drug pairs labeled as Synergism, Antagonism, or New Effect. Crucially, to effectively evaluate machine-learning based generalization, MUDI consists of unseen drug pairs in the test set. We evaluate benchmark models using both late fusion voting and intermediate fusion strategies. All data, annotations, evaluation scripts, and baselines are released under an open research license.
Figures
Reference graph
Works this paper leans on
-
[1]
Masaki Asada, Makoto Miwa, and Yutaka Sasaki. 2021. Using drug descriptions and molecular structures for drug–drug interaction extraction from literature. Bioinformatics 37, 12 (2021), 1739–1746
work page 2021
-
[2]
Masaki Asada, Makoto Miwa, and Yutaka Sasaki. 2023. Integrating heterogeneous knowledge graphs into drug–drug interaction extraction from the literature. Bioinformatics 39, 1 (2023), btac754
work page 2023
-
[3]
Juan M Banda, Tobias Kuhn, Nigam H Shah, and Michel Dumontier. 2015. Provenance-centered dataset of drug-drug interactions. In The Semantic Web- ISWC 2015: 14th International Semantic Web Conference, Bethlehem, PA, USA, October 11-15, 2015, Proceedings, Part II 14 . Springer, 293–300
work page 2015
-
[4]
Jari Björne, Suwisa Kaewphan, and Tapio Salakoski. 2013. UTurku: Drug named entity recognition and drug-drug interaction extraction using SVM classification and domain knowledge. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013) . 651–659
work page 2013
-
[5]
Olivier Bodenreider. 2004. The Unified Medical Language System (UMLS): in- tegrating biomedical terminology. Nucleic Acids Research 32, suppl_1 (2004), D267–D270
work page 2004
-
[6]
Yifan Deng, Xinran Xu, Yang Qiu, Jingbo Xia, Wen Zhang, and Shichao Liu. 2020. A multimodal deep learning framework for predicting drug–drug interaction events. Bioinformatics 36, 15 (2020), 4316–4322
work page 2020
-
[7]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al . 2021. An image is worth 16x16 words: Trans- formers for image recognition at scale. In International Conference on Learning Representations (ICLR)
work page 2021
-
[8]
Norig Ellison. 2002. Goodman & Gilman’s the pharmacological basis of thera- peutics. Anesthesia & Analgesia 94, 5 (2002), 1377
work page 2002
Show all 39 references
-
[9]
Yanglan Gan, Wenxiao Liu, Guangwei Xu, Cairong Yan, and Guobing Zou. 2023. DMFDDI: deep multimodal fusion for drug–drug interaction prediction.Briefings in Bioinformatics 24, 6 (2023), bbad397
2023
-
[10]
Yu Gu, Robert Tinn, Hao Cheng, Matthew Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH) 3, 1 (...
2021
-
[11]
Rave Harpaz, William DuMouchel, Nigam H Shah, David Madigan, Patrick Ryan, and Carol Friedman. 2012. Novel data-mining methodologies for adverse drug event discovery and analysis. Clinical Pharmacology & Therapeutics 91, 6 (2012), 1010–1021
2012
-
[12]
Haohuai He, Guanxing Chen, and Calvin Yu-Chian Chen. 2022. 3DGT-DDI: 3D graph and text based neural network for drug–drug interaction prediction. Briefings in bioinformatics 23, 3 (2022), bbac134
2022
-
[13]
Daniel Scott Himmelstein, Antoine Lizee, Christine Hessler, Leo Brueggeman, Sabrina L Chen, Dexter Hadley, Ari Green, Pouya Khankhanian, and Sergio E Baranzini. 2017. Systematic integration of biomedical knowledge prioritizes drugs for repurposing. elife 6 (2017), e26726
2017
-
[14]
Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. 2021. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. arXiv preprint arXiv:2102.09548 (2021)
2021 arXiv
-
[15]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic op- timization. International Conference on Learning Representations (ICLR) (2015). https://arxiv.org/abs/1412.6980
2015 arXiv
-
[16]
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR) (2017)
2017
-
[17]
Gregory Landrum. 2013. RDKit: Open-source cheminformatics. http://www. rdkit.org. Accessed: 2025-04-19
2013
-
[18]
Michael Lesk. 1986. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. Proceedings of the 5th annual international conference on Systems documentation (1986), 24–26
1986
-
[19]
Bradley M McQuade and Andrea Campbell. 2021. Drug prescribing: drug-drug interactions. FP essentials 508 (2021), 25–32
2021
-
[20]
Ishani Mondal. 2020. BERTChem-DDI: Improved Drug-Drug Interaction Predic- tion from text using Chemical Structure Information. In Proceedings of Knowl- edgeable NLP: the First Workshop on Integrating Structured Knowledge and Neural Networks for NLP. 27–32
2020
-
[21]
Yifan Peng, Anthony Rios, Ramakanth Kavuluru, and Zhiyong Lu. 2018. Extract- ing chemical-protein relations with ensembles of SVM and deep learning models. In Proceedings of the BioCreative VI Workshop . 155–157
2018
-
[22]
Martin F Porter. 1980. An algorithm for suffix stripping. Program 14, 3 (1980), 130–137
1980
-
[23]
Humphrey P Rang, Maureen M Dale, James M Ritter, Rod J Flower, and Graeme Henderson. 2011. Rang & Dale’s pharmacology . Elsevier Health Sciences
2011
-
[24]
Sunil Kumar Sahu and Ashish Anand. 2018. Drug-drug interaction extraction from biomedical texts using long short-term memory network.Journal of biomed- ical informatics 86 (2018), 15–24
2018
-
[25]
Yiyang Shi, Mingxiu He, Junheng Chen, Fangfang Han, and Yongming Cai
-
[26]
Nicholas P Tatonetti, Phyllis P Ye, Roxana Daneshjou, and Russ B Altman. 2012. Data-driven prediction of drug effects and interactions. Science translational medicine 4, 125 (2012), 125ra31–125ra31
2012
-
[27]
Food and Drug Administration
U.S. Food and Drug Administration. 2024. FDA Adverse Event Reporting System (FAERS) Database. https://www.fda.gov/drugs/drug-approvals-and-databases/ fda-adverse-event-reporting-system-faers-database. Accessed: 2025-05-29
2024
-
[28]
National Library of Medicine
U.S. National Library of Medicine. 2024. MEDLINE Database. https://www.nlm. nih.gov/medline/medline_home.html. Accessed: 2025-05-29
2024
-
[29]
Zhaoying Wang, Yingdan Shi, Xiang Liu, Can Chen, Jun Wen, and Ren Wang
-
[30]
David S Wishart, Yannick D Feunang, An C Guo, Elaine J Lo, Ana Marcu, Jason R Grant, Timothy Sajed, Daniel Johnson, Cecilia Li, Naina Sayeeda, et al . 2018. DrugBank 5.0: a major update to the DrugBank database for 2018. Nucleic acids research 46, D1 (2018), D1074–D1082
2018
-
[31]
Guoli Xiong, Zhijiang Yang, Jiacai Yi, Ningning Wang, Lei Wang, Huimin Zhu, Chengkun Wu, Aiping Lu, Xiang Chen, Shao Liu, et al. 2022. DDInter: an online drug–drug interaction database towards improving clinical decision-making and patient safety. Nucleic acids research 50, D1...
2022
-
[32]
Hui Yu. 2020. Data of multiple-type drug-drug interactions. doi:10.17632/ md5czfsfnd.1
2020
-
[33]
Tianlin Zhang, Jiaxu Leng, and Ying Liu. 2020. Deep learning for drug–drug interaction extraction from the literature: a review. Briefings in bioinformatics 21, 5 (2020), 1609–1627
2020
-
[34]
Zhehuan Zhao, Zhihao Yang, Lijing Luo, Hongfei Lin, and Jian Wang. 2016. Drug drug interaction extraction from biomedical literature using syntax convolutional neural network. Bioinformatics 32, 22 (2016), 3444–3453
2016
-
[35]
Jie Zhou, Cong Fu, Jianzhong Zhao, Weilin Lv, Sheng Zha, Qing Li, and Jiawei Han. 2021. A comprehensive survey on knowledge graphs: Representation, acquisition and applications. IEEE Transactions on Neural Networks and Learning Systems 32, 5 (2021), 2241–2263
2021
-
[36]
Yuchen Zhu, Lei Li, Hang Lu, Aoying Zhou, and Xiaoyong Qin. 2020. Extracting drug-drug interactions from texts with BioBERT and multiple entity-aware attentions. Journal of biomedical informatics 106 (2020), 103451
2020
-
[37]
Promazine
Marinka Zitnik, Rok Sosič, Sagar Maheshwari, and Jure Leskovec. 2018. BioSNAP Datasets: Stanford Biomedical Network Dataset Collection. http://snap.stanford. edu/biodata. Ngo et al. A Annotation Guidelines This section describes the manual annotation protocol used to label dru...
2018
-
[2024]
PLOS Computational Biology 20, 4 (2024), e1011989
SubGE-DDI: A new prediction model for drug-drug interaction established through biomedical texts and drug-pairs knowledge subgraph enhancement. PLOS Computational Biology 20, 4 (2024), e1011989
2024
-
[2025]
arXiv preprint arXiv:2502.06274 (2025)
HODDI: A Dataset of High-Order Drug-Drug Interactions for Computa- tional Pharmacovigilance. arXiv preprint arXiv:2502.06274 (2025)
2025 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.