REVIEW 4 major objections 4 minor 27 references
CSF-Net: Cross-Modal Spatiotemporal Fusion Network for Pulmonary Nodule Malignancy Predicting
T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Combining follow-up CT scans with clinical text via learned cross-modal attention predicts nodule malignancy more accurately than single-image or naive-fusion baselines, with 0.8974 accuracy and 0.9389 AUC on a curated NLST cohort.
desk verdict The dataset is real, but the clinical feature list may include the NLST screening result, which could turn the reported gains into label leakage. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The architecture has three load-bearing components. The spatial feature extractor, built from 3D ResNet with CBAM, produces a feature map for each of the two CT timepoints ($t_0$ and $t_1$). The temporal residual fusion module concatenates these maps, upsamples and globally average-pools them, applies a 3D convolution, and then fuses the result with the raw $t_1$ features using learnable sigmoid-gated weights $\lambda_0$ and $\lambda_1$ ($\mathit{STF}_{\mathrm{feat}} = \sigma(\lambda_0) F_{\mathrm{feat}} + \sigma(\lambda_1) t_1$). The cross-modal attention fusion module takes the spatiotemporal image features $x_i$ and clinical text features $y_i$, computes matching scores $\beta_{j,i}$ and $\rho_{j,i}$ via query-key products ($s_{ij} = q_1(x_i)^T k_2(y_j)$ and $t_{ij} = q_2(y_i)^T k_1(x_j)$), and uses these scores to reweight the image features before classification. Together these modules let the model attend to spatial regions that changed between scans and that align with clinical attributes.
What would settle it
Open the released code and list the clinical variables passed to the text encoder; if the NLST screening-result field (or any variable derived from the follow-up cancer diagnosis) is present, retrain CSF-Net without it and check whether the accuracy and AUC remain near 0.8974 and 0.9389.
Extended reading notes
Core claim
The central discovery is that a three-module architecture, consisting of a 3D ResNet+CBAM spatial extractor, a temporal residual fusion (TRF) module, and a cross-modal attention fusion (CMAF) module, yields a clear predictive gain when applied to two-timepoint CT data plus clinical text. The model's reported metrics on NLST-cmst exceed every baseline, and the ablation study shows each design choice contributes: the follow-up timepoint $t_1$ alone reaches 0.8462 accuracy versus 0.7079 for the baseline $t_0$, adding clinical features to $t_1$ raises accuracy to 0.8590, and removing either the TRF or the CMAF module drops the full model's accuracy from 0.8974 to 0.8718. The paper presents these results as evidence that spatiotemporal and cross-modal fusion together simulate the clinician's practice of reading follow-up scans in light of patient history.
Load-bearing premise
The clinical features fed to the text encoder must not include the NLST screening-result variable, which is defined by the same follow-up cancer diagnoses used to label malignancy; the paper does not specify which clinical fields are encoded.
Editorial extensions
If this is right
- The reported performance implies that two-timepoint follow-up imaging plus routinely available clinical attributes is a higher-signal input for malignancy risk stratification than a single screening CT.
- The ablation results imply that the follow-up timepoint $t_1$ dominates the image signal, so screening archives missing the follow-up scan would forgo most of the predictive benefit.
- The model's gains over DeepCAD, the strongest multimodal temporal baseline, indicate that learned temporal residual fusion and cross-modal attention add value beyond simple multimodal temporal concatenation.
- The new NLST-cmst dataset provides a publicly available, pathologically labeled two-timepoint benchmark on which future two-timepoint and multimodal nodule models can be compared.
Reading between the lines
- The paper does not specify which clinical fields were fed to the text encoder; if the NLST screening-result variable (derived from the same follow-up cancer diagnoses that set the malignancy label) is among them, the cross-modal attention could be reading the label, and a retraining without that variable is the decisive test.
- The dominance of $t_1$ over $t_0$ may reflect recency as much as temporal dynamics; testing the same architecture with more than two timepoints or on a time-to-event outcome would separate 'learning growth' from 'learning recency'.
- The cross-modal attention maps could be visualized to ask whether the model grounds its predictions in clinically meaningful image-text correspondences (e.g., nodule regions matching smoking history), which the paper does not report.
- The NLST-cmst cohort is drawn from a U.S. screening trial; generalizability to other screening programs, to non-screen-detected nodules, or to differing CT acquisition protocols is untested.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CSF-Net, a deep learning model for predicting pulmonary nodule malignancy from two timepoint CT scans and clinical information. The architecture consists of a 3D ResNet with CBAM for spatial feature extraction, a Temporal Residual Fusion module that combines features from the two timepoints (Eqs. 1-2), and a Cross-Modal Attention Fusion module (Eqs. 3-4) that integrates image features with clinical text features. The authors construct a new dataset, NLST-cmst, of 443 NLST subjects with physician-annotated nodule ROIs and pathologically confirmed malignancy labels. On this dataset, CSF-Net reports accuracy 0.8974, precision 0.8235, F1 0.8750, AUC 0.9389, and recall 0.9333, outperforming six comparison methods across most metrics. The code is publicly available.
Significance. The potential contribution is twofold: a publicly documented dataset for longitudinal nodule malignancy prediction, and a modular architecture that explicitly models spatiotemporal and cross-modal fusion. If the reported gains are robust, the paper would support the clinical practice of combining follow-up imaging with structured clinical data. The strengths include public code, an external pathological gold standard for labels, and an ablation study that isolates several architectural choices. The main weakness is empirical: no confidence intervals or significance tests are reported, and the clinical feature set is not fully specified. In particular, the possible inclusion of NLST screening results - a variable derived from the same CT images and strongly associated with the outcome - creates a risk of label leakage that could invalidate the headline comparison. This concern is addressable and should be resolved before publication.
major comments (4)
- [Section 3.1] The clinical information fed to the text encoder is never enumerated. Section 3.1 lists 'age, gender, smoking status, and screening results as defined by the NLST' as available clinical information, but the manuscript does not state which of these fields are actually used as input features. The NLST screening result is a radiologist-generated categorization of the CT screen and is strongly predictive of an eventual cancer diagnosis; if this field is among the text-encoder inputs, the cross-modal attention module can effectively read the label from the input. Because none of the six baselines uses this field, the performance advantage in Table 1 would not demonstrate the value of spatiotemporal or cross-modal fusion. Please enumerate the exact clinical variables used and provide an ablation that removes the screening-result field, or explicitly confirm that this field was never included.
- [Section 3.3, Table 1] The empirical comparison lacks confidence intervals, significance tests, and a description of the data split. The abstract claims 'significant performance improvements,' but no statistical measure supports this wording. The manuscript should state how the 443 subjects were split into train/validation/test sets, for example the split ratio, whether the split was at the patient level, and the random seed, and it should report bootstrap confidence intervals or a paired permutation test over the test set. It should also state explicitly whether all baselines were evaluated on exactly the same partitions.
- [Section 3.3, comparative experiments] Several comparison methods, including SCANs, NAS-Lung, T-LSTM, RadFusion, DeepCAD, and MFCN, were reimplemented by the authors because public code was not available. The reported numbers may not reflect the official implementations, and the sentence 'The results presented are the actual results from our experiments' does not address how hyperparameters were tuned or whether the reimplementations were validated against known results. Please provide the exact training settings for each baseline and, where possible, report results from official code or from prior publications on the same data.
- [Section 3.4, Table 2] The ablation study does not isolate the contribution of clinical data from the contribution of the screening-result field. The row 'without CMAF+clinical' removes both the cross-modal attention module and all clinical data together, so any drop in performance cannot be attributed separately to the fusion mechanism or to the clinical variables. Given the concerns in the first major comment, an ablation that removes only the screening-result feature while keeping the remaining clinical variables is essential to support the claim that clinical information improves prediction.
minor comments (4)
- [Title/header] The PDF header line breaks the word as 'Spa tiotemporal'; it should read 'Spatiotemporal'.
- [Section 3.2] The Adam optimizer is reported with beta1=0.5, which is unusual (the standard value is 0.9); please clarify the choice or correct the value, and specify what is updated every 20 epochs.
- [Section 3.3] The method name is inconsistent: it appears as both 'MCFN [10]' in the introduction and 'MFCN [10]' in the comparative experiments and Table 1; please make it consistent.
- [Table 1] The row for CSF-Net contains formatting artifacts in the delimiters; the numbers should be displayed cleanly so that the improvements over the best baseline are immediately readable.
Circularity Check
No circularity: the reported gains are empirical measurements against a pathological gold standard; no prediction reduces to its input by construction.
full rationale
CSF-Net's central claim is an empirical architecture evaluation, not a derivation chain. The malignancy labels come from the NLST pathological gold standard, as stated in Section 3.1, and the reported metrics in Tables 1 and 2 are measured on the NLST-cmst test set. The method equations in Sections 2.2 and 2.3 define feature fusion operations, not a definition of the prediction target in terms of the input features or of a fitted parameter. The CMAF module is imported from an external citation [14] and used as a component; it does not by itself force the reported numerical superiority. The paper's self-citations [8] and [9] appear only in the related-work survey and do not carry the load of the experimental conclusion. The baseline implementations by the authors are a reproducibility concern, not a circularity issue. The possible inclusion of 'screening results as defined by the NLST' among clinical features is a label-leakage or data-contamination risk, but the paper never states that this field is fed to the text encoder, and there is no exhibited mathematical reduction of the result to that field. Under the hard rules requiring a quoted step that is equivalent to its input by construction, no circular step is present.
Assumptions & free parameters
free parameters (2)
- lambda_0 and lambda_1 in Temporal Residual Fusion (Eq. 2) =
learned during training; not reported
- Attention projection weights q1, k2, q2, k1 in CMAF (Eqs. 3-4) =
learned during training; not reported
assumptions (4)
- domain assumption The NLST-cmst nodule locations and malignancy labels, based on pathological confirmation, are accurate.
- domain assumption The clinical variables used as input do not contain a near-deterministic proxy for the outcome, such as the NLST screening result.
- domain assumption The train/test split is random, non-overlapping, and never used for hyperparameter selection in a way that inflates test metrics.
- domain assumption The six comparison models were reimplemented faithfully from their papers.
Cite this review
Pith. "Pith review of CSF-Net: Cross-Modal Spatiotemporal Fusion Network for Pulmonary Nodule Malignancy Predicting." pith.science (2026). https://pith.science/paper/GZFNHWRM
@misc{pith2026250116400,
author = {Pith},
title = {Pith review of: CSF-Net: Cross-Modal Spatiotemporal Fusion Network for Pulmonary Nodule Malignancy Predicting},
year = {2026},
howpublished = {\url{https://pith.science/paper/GZFNHWRM}},
note = {Machine review of arXiv:2501.16400}
}
read the original abstract
Pulmonary nodules are an early sign of lung cancer, and detecting them early is vital for improving patient survival rates. Most current methods use only single Computed Tomography (CT) images to assess nodule malignancy. However, doctors typically make a comprehensive assessment in clinical practice by integrating follow-up CT scans with clinical data. To enhance this process, our study introduces a Cross-Modal Spatiotemporal Fusion Network, named CSF-Net, designed to predict the malignancy of pulmonary nodules using follow-up CT scans. This approach simulates the decision-making process of clinicians who combine follow-up imaging with clinical information. CSF-Net comprises three key components: spatial feature extraction module, temporal residual fusion module, and cross-modal attention fusion module. Together, these modules enable precise predictions of nodule malignancy. Additionally, we utilized the publicly available NLST dataset to screen and annotate the specific locations of pulmonary nodules and created a new dataset named NLST-cmst. Our experimental results on the NLST-cmst dataset demonstrate significant performance improvements, with an accuracy of 0.8974, a precision of 0.8235, an F1 score of 0.8750, an AUC of 0.9389, and a recall of 0.9333. These findings indicate that our multimodal spatiotemporal fusion approach, which combines follow-up data with clinical information, surpasses existing methods, underscoring its effectiveness in predicting nodule malignancy.
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION Lung cancer, originating from lung cells, is a rapidly growing malignancy that poses a serious global public health threat. According to the World Health Organization and the Amer- ican Cancer Society, there were approximately 2.5 million ∗Correspondings : gespring@hdu.edu.cn, cmwangalbert@gmail.com new cases of lung cancer worldwide in 2022,...
work page 2022
-
[2]
CSF-Net: Cross-Modal Spatiotemporal Fusion Network for Pulmonary Nodule Malignancy Predicting
in the United States has shown that regular imaging and lung function assessments can track nodule changes, predict- ing malignancy risks. Follow-ups help in early detection, in- creasing survival rates. However, current assessments largely rely on single CT scans, limiting insights into nodule pro- gression and potentially leading to late diagnoses. Regu...
work page Pith review arXiv 2025
-
[3]
METHODS In this work, we aim to predict the malignancy of pulmonary nodules by exploring the relationships between follow-up imaging data and clinical information, framing it as a binary classification task. Specifically, we introduce the CSF-Net, which consists of three key modules: the spatial feature ex- traction module, the temporal residual fusion mo...
-
[4]
Data Preparation and Preprocess The NLST dataset is a pivotal research initiative by the U.S
EXPERIMENTAL RESULTS 3.1. Data Preparation and Preprocess The NLST dataset is a pivotal research initiative by the U.S. National Cancer Institute (NCI) aimed at reducing lung can- cer mortality in high-risk populations through early imaging screenings. It includes longitudinal CT scans of participants’ lungs and relevant clinical information, such as age,...
-
[5]
Impact of Using t0 vs. t1 Images Alone: When only t0 images were used, model performance dropped signifi- cantly, with Acc falling from 0.8974 to 0.7079, along with decreases in Prec and Rec. However, using only t1 images resulted in considerable performance improvements, with in- creased Acc, Prec, Rec, and F1 by 0.1666, 0.1683, 0.3000, and 0.2309, respe...
-
[6]
Impact of Clinical Data to t0 and t1 Images: In- troducing clinical data significantly enhanced model perfor- mance, especially when combined with t1 images, where Acc, Prec, Rec, and F1 further increased by 0.1283, 0.2313, 0.3167, and 0.1756, respectively. This highlights the critical role of clinical data in complementing image information, providing th...
-
[7]
Impact of Cross-Attention and Clinical Data: With- out cross-attention and clinical data, the model’s ability to integrate multimodal information is restricted, resulting in an Acc of 0.8718 and a Prec of 0.7941. The absence of cross-attention significantly limits the model’s capacity to capture complex relationships between data sources, leading to dimin...
-
[8]
Impact of Temporal Fusion: Eliminating the tempo- ral fusion strategy resulted in a model Acc of 0.8718 and a Prec of 0.7778. This outcome highlights the critical role of temporal fusion in optimizing the integration of temporal in- formation. Without this component, the model’s capability to reconstruct temporal features is impaired, reducing its sensi- ...
Show all 27 references
-
[9]
To address this, we pro- pose CSF-Net, a cross-modal spatiotemporal fusion network model for malignancy prediction
CONCLUSIONS Inefficient temporal feature integration and lack of clinical data consideration can reduce the accuracy of pulmonary nodule malignancy predictions. To address this, we pro- pose CSF-Net, a cross-modal spatiotemporal fusion network model for malignancy prediction. ...
-
[10]
The license accompanying this open-access data confirms that no ethical approval is required
COMPLIANCE WITH ETHICAL STANDARDS This study retrospectively utilized human subject data ob- tained from the publicly accessible National Lung Screening Trial dataset. The license accompanying this open-access data confirms that no ethical approval is required
-
[11]
LGK- CYLWS2023018), and Shenzhen Science and Technology Program (No.KCXFZ20201221173008022)
ACKNOWLEDGMENTS This work was supported by the Open Project Program of the State Key Laboratory of CAD&CG, Zhejiang Uni- versity (No.A2410), Zhejiang Provincial Natural Science Foundation of China (No.LY21F020017), National Natu- ral Science Foundation of China (No.61702146, 6...
-
[12]
Global cancer statis- tics 2022: Globocan estimates of incidence and mortal- ity worldwide for 36 cancers in 185 countries,
Freddie Bray, Mathieu Laversanne, Hyuna Sung, Jacques Ferlay, Rebecca L Siegel, Isabelle Soerjo- mataram, and Ahmedin Jemal, “Global cancer statis- tics 2022: Globocan estimates of incidence and mortal- ity worldwide for 36 cancers in 185 countries,” CA: A Cancer Journal for C...
2022
-
[13]
The na- tional lung screening trial: overview and study design,
National Lung Screening Trial Research Team, “The na- tional lung screening trial: overview and study design,” Radiology, vol. 258, no. 1, pp. 243–253, 2011
2011
-
[14]
Learning efficient, explainable and discrimina- tive representations for pulmonary nodules classifica- tion,
Hanliang Jiang, Fuhao Shen, Fei Gao, and Weidong Han, “Learning efficient, explainable and discrimina- tive representations for pulmonary nodules classifica- tion,” Pattern Recognition, vol. 113, pp. 107825, 2021
2021
-
[15]
Evaluate the malignancy of pulmonary nod- ules using the 3-d deep leaky noisy-or network,
Fangzhou Liao, Ming Liang, Zhe Li, Xiaolin Hu, and Sen Song, “Evaluate the malignancy of pulmonary nod- ules using the 3-d deep leaky noisy-or network,” IEEE Transactions on Neural Networks and Learning Sys- tems, vol. 30, no. 11, pp. 3484–3495, 2019
2019
-
[16]
Study on the prediction method of long-term benign and malignant pulmonary lesions based on lstm,
Xindong Liu, Mengnan Wang, and Rukhma Aftab, “Study on the prediction method of long-term benign and malignant pulmonary lesions based on lstm,” Fron- tiers in Bioengineering and Biotechnology , vol. 10, pp. 791424, 2022
2022
-
[17]
Lung nodule malignancy prediction from longitudinal ct scans with siamese convolutional attention networks,
Benjamin P. Veasey, Justin Broadhead, Michael Dahle, Albert Seow, and Amir A. Amini, “Lung nodule malignancy prediction from longitudinal ct scans with siamese convolutional attention networks,” IEEE Open Journal of Engineering in Medicine and Biology, vol. 1, pp. 257–264, 2020
2020
-
[18]
Spatio-temporal hybrid fusion of cae and swin trans- formers for lung cancer malignancy prediction,
Sadaf Khademi, Shahin Heidarian, Parnian Afshar, Farnoosh Naderkhani, Anastasia Oikonomou, Kon- stantinos N Plataniotis, and Arash Mohammadi, “Spatio-temporal hybrid fusion of cae and swin trans- formers for lung cancer malignancy prediction,” in ICASSP 2023-2023 IEEE Internat...
2023
-
[19]
Pe-mvcnet: Multi-view and cross-modal fusion network for pulmonary embolism prediction,
Zhaoxin Guo, Zhipeng Wang, Ruiquan Ge, Jianxun Yu, Feiwei Qin, Yuan Tian, Yuqing Peng, Yonghong Li, and Changmiao Wang, “Pe-mvcnet: Multi-view and cross-modal fusion network for pulmonary embolism prediction,” in 2024 IEEE International Symposium on Biomedical Imaging (ISBI), ...
2024
-
[20]
Ich-prnet: A cross- modal intracerebral haemorrhage prognostic prediction method using joint-attention interaction mechanism,
Xinlei Yu, Ahmed Elazab, Ruiquan Ge, Jichao Zhu, Lingyan Zhang, Gangyong Jia, Qing Wu, Xiang Wan, Lihua Li, and Changmiao Wang, “Ich-prnet: A cross- modal intracerebral haemorrhage prognostic prediction method using joint-attention interaction mechanism,” Neural Networks, p. 1...
2025
-
[21]
Multimodal co-attention fusion network with online data augmentation for cancer subtype classifica- tion,
Saisai Ding, Juncheng Li, Jun Wang, Shihui Ying, and Jun Shi, “Multimodal co-attention fusion network with online data augmentation for cancer subtype classifica- tion,” IEEE Transactions on Medical Imaging, pp. 1–1, 2024
2024
-
[22]
Multimodal fusion of imaging and genomics for lung cancer recurrence prediction,
Vaishnavi Subramanian, Minh N Do, and Tanveer Syeda-Mahmood, “Multimodal fusion of imaging and genomics for lung cancer recurrence prediction,” in 2020 IEEE 17th International Symposium on Biomed- ical Imaging (ISBI). IEEE, 2020, pp. 804–808
2020
-
[23]
Autoencoder-based multimodal prediction of non-small cell lung cancer survival,
Jacob G Ellen, Etai Jacob, Nikos Nikolaou, and Natasha Markuzon, “Autoencoder-based multimodal prediction of non-small cell lung cancer survival,” Scientific Re- ports, vol. 13, no. 1, pp. 15761, 2023
2023
-
[24]
Enhancing cancer prediction in challenging screen-detected incident lung nodules us- ing time-series deep learning,
Shahab Aslani, Pavan Alluri, Eyjolfur Gudmundsson, Edward Chandy, John McCabe, Anand Devaraj, Car- olyn Horst, Sam M Janes, Rahul Chakkara, Daniel C Alexander, et al., “Enhancing cancer prediction in challenging screen-detected incident lung nodules us- ing time-series deep le...
2024
-
[25]
Cmafgan: A cross-modal atten- tion fusion based generative adversarial network for at- tribute word-to-face synthesis,
Xiaodong Luo, Xiang Chen, Xiaohai He, Linbo Qing, and Xinyue Tan, “Cmafgan: A cross-modal atten- tion fusion based generative adversarial network for at- tribute word-to-face synthesis,” Knowledge-Based Sys- tems, vol. 255, pp. 109750, 2022
2022
-
[26]
Cbam: Convolutional block attention module,
Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon, “Cbam: Convolutional block attention module,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19
2018
-
[27]
Radfusion: Benchmarking performance and fairness for multimodal pulmonary embolism detection from ct and ehr,
Yuyin Zhou, Shih-Cheng Huang, Jason Alan Fries, Alaa Youssef, Timothy J Amrhein, Marcello Chang, Imon Banerjee, Daniel Rubin, Lei Xing, Nigam Shah, et al., “Radfusion: Benchmarking performance and fairness for multimodal pulmonary embolism detection from ct and ehr,” arXiv pre...
2021 arXiv
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.