REVIEW 3 major objections 5 minor 7 references
Exploring Methods for Integrating and Augmenting Multimodal Data to Improve Prognostic Accuracy in Imbalanced Datasets for Intraoperative Aneurysm Occlusion
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read An intermediate-fusion neural network combining angiographic imaging, biomarkers, and morphology, trained with SMOTE/SMOTENC oversampling, raises aneurysm occlusion prediction AUROC from 0.68±0.1 to 0.85±0.06.
desk verdict A clinically relevant multimodal fusion study whose headline AUROC gain to 0.85 depends on an unspecified augmentation/split ordering, so the result is not yet citable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the intermediate fusion deep network: separate dense branches process continuous imaging features, continuous numerical features, and one-hot encoded categorical features, and their pre-decision layer outputs are concatenated before passing through additional dense layers to a sigmoid output. The companion mechanism is SMOTE/SMOTENC, a synthetic minority oversampling technique that creates new training examples for the under-represented occlusion-failure class by interpolating between nearest neighbors, with the SMOTENC variant explicitly handling mixed nominal and continuous data. Together they allow the model to learn from a balanced representation of both outcome classes instead of being dominated by the majority successful-occlusion class.
What would settle it
Re-run the 20-split Monte Carlo cross-validation while instrumenting the pipeline to record where each synthetic SMOTE/SMOTENC sample is created relative to the train/validation/test boundaries; if the intermediate fusion AUROC drops from 0.85±0.06 to the un-augmented 0.68±0.1 (or below the 0.74 unimodal baseline) when synthetic samples are confined to training folds, the paper's central claim is refuted.
Extended reading notes
Core claim
The paper's central finding is that the way multimodal information is merged and the way class imbalance is handled jointly determine whether fusion helps. Intermediate fusion — where each data type passes through its own short neural network and the learned representations are concatenated before the final prediction layers — paired with SMOTE/SMOTENC synthetic oversampling achieves AUROC 0.85±0.06, the highest performance observed, compared with 0.68±0.1 for the same fusion without augmentation. Late fusion with the same augmentation reaches 0.73±0.04, while random oversampling and ADASYN produce widely varying results, with AUROC between 0.52±0.07 and 0.79±0.1 depending on the architecture. The authors interpret this as evidence that multimodal integration can improve prognostic accuracy provided the augmentation strategy is matched to the architecture and to the mixed categorical-and-continuous feature space.
Load-bearing premise
In Section 2.3 the augmentation step is described, but the paper never states whether SMOTE/SMOTENC synthetic samples are generated inside each training fold or before the data split, and the 0.85 AUROC claim depends on synthetic examples not leaking into validation or test folds.
Editorial extensions
If this is right
- Intermediate fusion plus SMOTE/SMOTENC is the strongest configuration studied, reaching AUROC 0.85±0.06 versus 0.68±0.1 without augmentation.
- Late fusion with the same augmentation reaches AUROC 0.73±0.04, so the gain is not generic to any fusion architecture.
- Unimodal API prediction (0.74±0.07) still beats un-augmented multimodal fusion, implying that naive fusion of extra variables can degrade accuracy.
- Augmentation choice matters: random oversampling and ADASYN produced AUROC from 0.52±0.07 to 0.79±0.1 depending on the architecture.
- If the result generalizes, the framework turns routine angiography, demographic, and morphological data into an occlusion-probability estimate that could inform endovascular treatment planning.
Reading between the lines
- Because the paper does not state whether SMOTE is applied before or after the train/test split, the strongest check is to rerun the same 20-split Monte Carlo protocol while generating synthetic samples only inside each training fold; if validation AUROC falls back toward the un-augmented 0.68–0.74 range, the reported gain is an artifact of leakage.
- The two-step normalization of API values for injection variability and foreshortening likely carries part of the benefit, so ablating that preprocessing step would reveal how much of the 0.85 AUROC depends on it.
- The model's clinical utility hinges on sensitivity for the minority class, the non-occluded aneurysms; reporting precision and recall for that class would be a more decision-relevant endpoint than AUROC alone.
- The same intermediate-fusion-plus-SMOTENC recipe is portable to other imbalanced medical prediction tasks where continuous measurements and categorical risk factors coexist.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper evaluates a multimodal machine learning framework for predicting intracranial aneurysm occlusion outcomes after endovascular treatment. Data from 340 patients include angiographic parametric imaging (API) features, demographic/clinical variables, and aneurysm morphology. Separate deep neural networks are trained on quantitative and categorical data, with late and intermediate fusion schemes, and several class-imbalance augmentation methods (SMOTE/SMOTENC, RandomOverSampler, ADASYN) are compared. Performance is reported as AUROC means and standard deviations over 20-split Monte Carlo cross-validation. The central claim is that SMOTE/SMOTENC augmentation raises intermediate-fusion AUROC from 0.68±0.1 to 0.85±0.06, the highest observed value.
Significance. If the reported gain is genuine, the paper would provide a useful, clinically relevant demonstration that a specific oversampling strategy can substantially improve multimodal deep learning predictions on a small, imbalanced neurovascular dataset. The work compares multiple augmentation techniques on a real 340-patient cohort, which is a practical strength. However, the central result rests on two currently unsupported pillars: the correct temporal placement of augmentation relative to cross-validation splits, and statistical evidence that the AUROC improvement is not due to chance or leakage. Because these pillars are not established, the significance of the finding cannot yet be assessed.
major comments (3)
- [Section 2.3 and Figure 1] The manuscript does not state whether SMOTE/SMOTENC, RandomOverSampler, and ADASYN are applied before or after the 20-split Monte Carlo cross-validation splits. This is load-bearing: if augmentation is applied to the entire dataset before splitting, synthetic samples can appear in both training and validation/test folds, and the reported AUROC of 0.85±0.06 for intermediate fusion could be artificially inflated by memorization of synthetic data. Please specify explicitly that the oversampling procedure is fitted on each training fold only and applied to that fold, with no synthetic example entering the corresponding validation or test fold. Also report the minority/majority class counts in the original dataset and the number of synthetic samples generated per method.
- [Section 3, Results] The claim that SMOTE/SMOTENC "significantly enhanced" the intermediate-fusion model is based on a comparison of 0.68±0.1 versus 0.85±0.06. The paper reports only means and standard deviations over 20 splits, with no confidence intervals, no paired significance test, and no statement about whether the splits are shared across compared methods or whether random seeds are fixed. Given the small cohort (n=340) and the large reported standard deviations, the observed gain may not be statistically reliable. Provide confidence intervals for the AUROC of each configuration and a paired test (e.g., DeLong test or bootstrap) comparing the best augmentation against no augmentation, with the same split indices used across all methods.
- [Section 2.2, Data fusion] The neural network architecture descriptions are too underspecified to reproduce the experiments or to attribute performance differences to fusion type rather than to arbitrary hyperparameter choices. The number of layers, layer widths, L1/L2 regularization coefficients, learning rate values and schedules, early-stopping criteria, and the exact "advanced pipeline setup" for the intermediate fusion model are not given. Please provide a complete specification of both architectures, or a pointer to released code, so that the comparison between late and intermediate fusion is interpretable and reproducible.
minor comments (5)
- [Abstract] The abbreviation "SMOTE NC" appears with a space in the abstract, while the main text uses "SMOTENC"; please use one consistent form throughout.
- [Section 2.2] The phrase "pre decision layers" should be hyphenated as "pre-decision layers" (it also appears in the abstract and the summary).
- [Section 3] The results are reported only in narrative form; a table summarizing AUROC (mean±std) for each fusion type and each augmentation method would greatly improve readability and allow direct comparison.
- [Section 2.1] The two-step normalization method attributed to Ionita et al. is described only by a reference to [7]; the two steps themselves are not described. Please provide a brief description or cite the original methodological source explicitly.
- [General] There is no data or code availability statement. Given the reproducibility concerns, please add a sentence indicating whether the de-identified dataset and implementation can be shared.
Circularity Check
No circularity: the augmentation and fusion comparisons are empirical evaluations whose central AUROC claims are not forced by definition or by self-citation.
full rationale
The paper's derivation chain is an empirical evaluation, not a derivation. It trains deep networks on 340 patient records, compares unimodal, late-fusion, and intermediate-fusion models, and then reports AUROC changes under SMOTE/SMOTENC, RandomOverSampler, and ADASYN (Section 3). No result is defined in terms of another result: the AUROC values are measured on held-out Monte Carlo cross-validation folds, and no equation or fitted parameter reduces to the claimed outcome. The only self-citations are background and methodology: reference [7] (same group) is cited for the prior finding that API features reach ~0.8 accuracy, and Ionita et al. is cited for a two-step normalization. Both are context or method, not load-bearing premises for the new augmentation comparison. The manuscript does not specify whether SMOTE/SMOTENC is applied per training fold, which is a possible leakage/methodological concern (and thus a correctness risk), but leakage would be an experimental flaw, not circularity by definition or by self-citation. Consequently, no circular step can be quoted or exhibited, and the appropriate score is 0.
Assumptions & free parameters
free parameters (4)
- learning rate and optimizer settings
- neural network architecture (layer count, units per layer)
- L1L2 regularization strength
- SMOTE/SMOTENC sampling parameters (sampling_strategy, number of neighbors)
assumptions (4)
- domain assumption The 340-patient retrospective cohort is representative of the broader population of intracranial aneurysm treatments.
- domain assumption The Monte Carlo cross-validation with 20 splits yields unbiased estimates of model performance.
- ad hoc to paper Synthetic samples generated by SMOTE/SMOTENC preserve the underlying distribution of the minority class without introducing unrealistic feature combinations.
- domain assumption The two-step normalization proposed by Ionita et al. adequately corrects for injection variability and foreshortening in the API data.
Cite this review
Pith. "Pith review of Exploring Methods for Integrating and Augmenting Multimodal Data to Improve Prognostic Accuracy in Imbalanced Datasets for Intraoperative Aneurysm Occlusion." pith.science (2026). https://pith.science/paper/ZYYGKDPK
@misc{pith2026241114407,
author = {Pith},
title = {Pith review of: Exploring Methods for Integrating and Augmenting Multimodal Data to Improve Prognostic Accuracy in Imbalanced Datasets for Intraoperative Aneurysm Occlusion},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZYYGKDPK}},
note = {Machine review of arXiv:2411.14407}
}
read the original abstract
This study evaluates a multimodal machine learning framework for predicting treatment outcomes in intracranial aneurysms (IAs). Combining angiographic parametric imaging (API), patient biomarkers, and disease morphology, the framework aims to enhance prognostic accuracy. Data from 340 patients were analyzed, with separate deep neural networks processing quantitative and categorical data. These networks' pre decision layers were concatenated and inputted into a final predictive network. Various data augmentation strategies, including Synthetic Minority Oversampling Technique for Nominal and Continuous data (SMOTE NC), addressed dataset imbalances. Performance metrics, evaluated through Monte Carlo cross validation, showed significant improvements with augmentation, particularly in intermediate fusion models. This study validates the framework's efficacy in accurately predicting IA treatment outcomes, demonstrating that data augmentation techniques can substantially enhance model performance.
Figures
Reference graph
Works this paper leans on
-
[7]
M. M. Shiraz Bhurwani, M. Waqas, A. R. Podgorsak, K. A. Williams, J. M. Davies, K. Snyder, E. Levy, A. Siddiqui, and C. N. Ionita, “Feasibility study for use of angiographic parametric imaging and deep neural networks for intracranial aneurysm occlusion prediction,” J Neurointerv Surg , 12(7), 714-719 (2020)
work page 2020
-
[1]
G. Boulouis, C. Rodriguez-Regent, E. C. Rasolonjatovo, W. Ben Hassen, D. Trystram, M. Edjlali-Goujon, J. F. Meder, C. Oppenheim, and O. Naggara, “Unruptured intracranial aneurysms: An updated review of current concepts for risk factors, detection and management,” Rev Neurol (Paris) , 173(9), 542-551 (2017)
work page 2017
-
[2]
The detection and management of unruptured intracranial aneurysms,
J. M. Wardlaw, and P. M. White, “The detection and management of unruptured intracranial aneurysms,” Brain, 123 ( Pt 2), 205-21 (2000)
2000
-
[3]
Digital subtraction angiography (DSA): a neurosurgical perspective,
P. Butler, “Digital subtraction angiography (DSA): a neurosurgical perspective,” Br J Neurosurg, 1(3), 32333 (1987)
work page 1987
-
[4]
Digital angiography: a perspective,
C. A. Mistretta, A. B. Crummy, and C. M. Strother, “Digital angiography: a perspective,” Radiology, 139(2), 273-6 (1981)
work page 1981
-
[5]
T. C. Huang, T. H. Wu, C. J. Lin, G. S. Mok, and W. Y . Guo, “Peritherapeutic quantitative flow analysis of arteriovenous malformation on digital subtraction angiography,” J Vasc Surg, 56(3), 812-5 (2012)
work page 2012
-
[6]
H. Tenjin, F. Asakura, Y . Nakahara, K. Matsumoto, T. Matsuo, F. Urano, and S. Ueda, “Evaluation of intraaneurysmal blood velocity by time -density curve analysis and digital subtraction angiography,” AJNR Am J Neuroradiol, 19(7), 1303-7 (1998)
work page 1998
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.