REVIEW 5 major objections 6 minor 80 references
Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosis
T0 review · 5 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read A frozen CT-only segmentation transformer can be upgraded to use PET scans with just 8% of the trainable parameters.
desk verdict The efficiency story is plausible and worth referee time, but the paper's headline no-catastrophic-forgetting claim is never actually tested, and the +28% PET-only improvement is not traceable to any reported number. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is a parallel PET pathway grafted onto a frozen transformer. A new PET patch-embedding layer produces PET tokens, a PET-specific skip connection $\theta^P_{SK}$ carries low-level PET features directly into the decoder with a learned weight $\beta$, and LoRA or DoRA low-rank matrices placed in the query/value attention projections are the only updated encoder weights. For Swin UNETR, an adapter layer reshapes single-modality inputs into the expected two-channel token format so that missing-modality inference remains possible. The conceptual load-bearer is the claim, borrowed from visual prompt tuning, that PET tokens can be ignored by the decoder yet still influence it because self-attention distills their information into the CT tokens.
What would settle it
Set the PET skip-connection weight $\beta$ to zero, keep the PET tokens in the attention stream, and measure PET-only Dice on the adaptation test set; if PET Dice collapses to the CT-only baseline, then the claim that PET knowledge is distilled into CT tokens is false and the skip connection is doing the work. Conversely, remove the PET tokens from attention while keeping the skip connection to see whether the skip path alone accounts for the PET gain.
Extended reading notes
Core claim
The central claim is that the modularity of transformer encoders lets a CT-only model become a CT+PET model through a parameter-efficient adaptation that never touches the frozen base weights. PEMMA adds PET tokens via a new patch embedding, adds a direct PET skip connection to the decoder, and trains only LoRA or DoRA matrices in the attention blocks. The decoder continues to receive only CT tokens; the paper asserts that self-attention distills the PET information into those CT tokens, so the PET stream acts as visual context rather than as a second input channel. Experiments on HECKTOR with UNETR and Swin UNETR backbones report segmentation performance comparable to early fusion at a fraction of the trainable parameters, robust single-modality inference, and continual-learning updates on new centers that preserve earlier capabilities. In the prognosis extension, the same frozen-encoder recipe with LoRA/DoRA raises the time-dependent concordance index from 0.61-0.66 on CT alone to 0.67-0.68 with CT+PET and 0.75-0.80 with CT+PET+EHR.
Load-bearing premise
The paper's results stand on the assertion that self-attention transfers enough PET information into the CT tokens for the decoder to use; if that transfer is weak, the PET skip connection alone must carry the entire PET signal, and the paper never measures either contribution directly.
Editorial extensions
If this is right
- A clinical deployment can start with a CT-only model and later add PET capability by training roughly 0.5-8% of the parameters, avoiding a full retrain on paired data.
- The adapted model remains usable when only one modality is available at inference, which matches real clinical workflow where PET is comparatively scarce.
- New imaging centers can be incorporated by updating only the low-rank adapters, and CT-only updates do not erase previously learned PET segmentation.
- The same frozen-encoder recipe transfers across tasks: a segmentation encoder can be repurposed for survival prognosis by adding a small head, and adding PET and EHR modalities raises the concordance index.
- DoRA consistently outperforms LoRA under PEMMA in both segmentation and prognosis, so the magnitude-direction decomposition is the recommended PEFT choice.
Reading between the lines
- If the distillation-through-attention premise holds, even lighter adapters—lower rank, fewer layers, or a shared PET projector—should retain most of the gain, since the PET stream does not need its own representational capacity.
- The same recipe could likely adapt a pretrained model to a third imaging modality or a new segmentation target without growing a new decoder, as long as the modality can be tokenized and attend to the CT tokens.
- A direct check of the distillation assumption is missing from the paper: masking PET tokens from attention in selected blocks and measuring PET-only Dice would isolate how much information travels through attention versus through the skip connection.
- Because PEMMA assumes pre-registered CT/PET pairs, the next natural stress test is unregistered or misaligned data, where the PET skip connection would need an alignment module before tokenization.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PEMMA, a parameter-efficient framework for adapting a CT-only pre-trained transformer-based segmentation model (UNETR or Swin UNETR) to also use PET images, with an extension to prognosis. The approach freezes the base model and adds PET-specific patch embeddings, a PET-specific skip connection, LoRA/DoRA adapters on attention weights, and, for Swin UNETR, an adapter module that reshapes single-modality inputs to the expected two-channel format and supports modality dropout. Experiments on the HECKTOR dataset cover multi-modal adaptation on the MDA center, continual fine-tuning on the HGJ and HMR centers (using CT-only or CT+PET data), and prognosis with the Swin UNETR encoder. The authors report that PEMMA matches or exceeds early fusion with 0.5%-8% of the trainable parameters, shows large PET-only Dice improvements after CT-only fine-tuning, and improves the concordance index as PET and EHR modalities are added.
Significance. If the claims are validated, PEMMA addresses a practically important deployment problem: upgrading a widely deployable CT-only model to exploit PET scans when they become available, with a parameter overhead an order of magnitude smaller than early fusion. The Swin UNETR adapter with modality dropout is a concrete architectural contribution, and the prognosis extension tests task-level transfer. The point estimates in Table 3 on the public HECKTOR data make the parameter-efficiency claim plausible, and the DoRA comparison is a useful addition. However, the paper's most distinctive claims—no catastrophic forgetting and cross-modal knowledge distillation through self-attention—are not actually measured, and the absence of uncertainty quantification weakens the quantitative conclusions. The requested experiments would make the contribution substantially stronger.
major comments (5)
- [Section 3.2.3, Section 4.3.2, Table 3] The abstract and Section 3.2.3 claim that PEMMA 'allows updates using only one modality without causing catastrophic forgetting in the other,' but the continual-learning experiments in Section 4.3.2 do not evaluate backward transfer. Every row in Table 3 after the MDA columns reports performance on the new dataset (HGJ or HMR) only; no result is reported on the MDA distribution after fine-tuning on HGJ or HMR. Catastrophic forgetting is by definition a loss of performance on previously learned data, so the current evidence supports only forward transfer to new centers, not retention of the originally adapted MDA knowledge. I request a post-fine-tuning evaluation on the MDA test set for all continual-learning variants, with comparison to the pre-fine-tuning MDA baseline, before the no-forgetting claim is made.
- [Section 3.2.1, Figure 1, Table 4] The architecture sends only the N CT tokens to the decoder and relies on the assertion that 'the self-attention architecture ensures that the knowledge from the PET tokens gets distilled into the CT tokens.' This mechanism is never measured. Table 4 compares three token-selection strategies for reducing the encoder output to the decoder's expected size, but it does not isolate the contribution of the PET tokens through self-attention from the contribution of the PET-specific skip connection (z_P scaled by beta). Without ablations that remove the PET patch embedding while keeping the skip connection, and vice versa, the paper does not establish the claimed distillation mechanism, which also underlies the Swin UNETR adapter design in Section 3.2.2.
- [Tables 3 and 5, Abstract] All results in Tables 3 and 5 are single-run point estimates with no uncertainty quantification. The abstract describes the results as 'significant' (for example, '+28% Dice score improvement'), and Section 5 ranks methods on differences as small as 0.01-0.02 average Dice (e.g., DoRA versus LoRA on Swin UNETR in the MDA stage), but no standard deviations, confidence intervals, number of seeds, or significance tests are reported. This makes it impossible to assess whether the claimed improvements over early fusion and the DoRA-versus-LoRA ranking are reliable. Please provide results across multiple seeds with variance and appropriate statistical comparisons, and remove the word 'significant' unless it is supported by a test.
- [Section 4.4, Table 5] The prognosis experiments report C-index improvements of +10% (CT to CP) and +23% (CT to CPT), but these are within-method comparisons relative to the same method's CT-only result. No alternative fusion baseline (early fusion, late fusion, or full fine-tuning of the encoder) is evaluated for prognosis, so the paper does not establish that parameter-efficient adaptation is beneficial for prognosis relative to standard approaches. The absence of baseline comparisons also makes the abstract's +10% and +23% claims difficult to interpret. Additionally, the LoRA/DoRA parameter counts for the text encoder and the full EHR preprocessing pipeline are not described in enough detail to reproduce the CPT experiments.
- [Abstract, Section 3.1, Section 4.3.1] The headline '+28% Dice score improvement on PET scans when trained with a single modality' is not tied to a specific baseline or table cell. The Discussion states 'minimum Average Dice scores of approximately 19% and 28% increases for two new datasets' without specifying which baseline, which inference modality, or which rows in Table 3 underlie these numbers. In addition, Section 3.1 defines three PET initialization strategies for the early fusion baseline (random, zero, cross-modal), but Section 4.3 does not state which one was used in Table 3; this choice can materially affect early fusion performance and is essential for a fair comparison with PEMMA.
minor comments (6)
- [Abstract] There is a typo: 'segementation' should be 'segmentation'.
- [Section 3.2.1] The word 'allowimg' should be 'allowing'.
- [Table 3] The table layout is very dense and the 'Train Modalities' row is ambiguous; clearer grouping of the MDA, HGJ, and HMR blocks and of the inference-modality columns would improve readability. The caption also contains an incomplete parenthetical: 'relative toΦ orΨ)' should be 'relative to Φ or Ψ'.
- [Section 4.2.3] The modality-sampling probabilities are written as '('ct' = 0.2 , 'pet' = 0.2 , or 'ctpet' 0.6)'; the missing '=' after 'ctpet' should be fixed, and the sentence should clarify whether these probabilities apply to the multi-modal adaptation stage, the continual-learning stage, or both.
- [Section 5] The capitalization of 'DoRA' is inconsistent: the text sometimes uses 'DoRa' (e.g., 'DoRa consistently outperforms LoRA' in Section 5). Please standardize.
- [Section 4.2.2 / Section 4.4] The text encoder ClipMD is cited but not described; a sentence on its architecture, output dimension, and whether it is frozen or fine-tuned would aid reproducibility.
Circularity Check
No significant circularity; the paper's new Swin UNETR, DoRA, and prognosis results are measured on public HECKTOR data and do not reduce to fitted inputs or to a self-citation chain.
full rationale
I traced the PEMMA derivation chain. The framework is explicitly built on the authors' prior PEMMA [58] (Section 2.6), but the paper's stated new contributions are the Swin UNETR adapter (Section 3.2.2), the DoRA comparison, and the prognosis extension (Section 4.4). These are evaluated on the public HECKTOR benchmark against early and late fusion in Tables 3 and 5, and the reported Dice and C-index values are not used as training inputs or fitted parameters; they are measured on held-out center splits. No equation defines a predicted quantity in terms of the fitted quantity, and no uniqueness claim is imported from the authors' prior work. The abstract's no-catastrophic-forgetting claim (Abstract; Section 3.2.1; Section 5) is asserted rather than demonstrated: the continual-learning rows in Table 3 evaluate only HGJ and HMR Dice after fine-tuning, with no MDA backward-transfer measurement taken before and after the continual-learning steps. That is a missing-evidence correctness concern, not a circular reduction, because the claim is not derived from the same numbers that would establish it. The Section 3.2.1 assertion that self-attention distills PET knowledge into CT tokens is an architectural assumption, not a circular step. The acknowledged self-citation of prior PEMMA is real but is not load-bearing for the independent new Swin UNETR and prognosis results; hence the low score of 1.
Assumptions & free parameters
free parameters (5)
- LoRA/DoRA rank r for segmentation attention layers =
not reported
- LoRA/DoRA scaling factor alpha =
not reported for segmentation; 128 for text encoder only
- PET skip connection weight beta =
not reported
- Modality dropout probabilities =
ct=0.2, pet=0.2, ctpet=0.6
- Early fusion PET initialization strategy =
cross-modal initialization
assumptions (4)
- domain assumption Pre-trained CT-only weights can be reused for a second modality via prompt-style PET tokens and low-rank adapters
- domain assumption CT and PET volumes are registered and can be concatenated channel-wise
- ad hoc to paper The Swin UNETR adapter can emulate the presence of a missing modality by reshaping single-modality embeddings
- ad hoc to paper Self-attention distills PET token information into CT tokens
invented entities (3)
-
PET-specific patch embedding layer
-
PET-specific skip connection
-
Swin UNETR missing-modality adapter
Cite this review
Pith. "Pith review of Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosis." pith.science (2026). https://pith.science/paper/KA3PSSL2
@misc{pith2026250413645,
author = {Pith},
title = {Pith review of: Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosis},
year = {2026},
howpublished = {\url{https://pith.science/paper/KA3PSSL2}},
note = {Machine review of arXiv:2504.13645}
}
read the original abstract
Cancer detection and prognosis relies heavily on medical imaging, particularly CT and PET scans. Deep Neural Networks (DNNs) have shown promise in tumor segmentation by fusing information from these modalities. However, a critical bottleneck exists: the dependency on CT-PET data concurrently for training and inference, posing a challenge due to the limited availability of PET scans. Hence, there is a clear need for a flexible and efficient framework that can be trained with the widely available CT scans and can be still adapted for PET scans when they become available. In this work, we propose a parameter-efficient multi-modal adaptation (PEMMA) framework for lightweight upgrading of a transformer-based segmentation model trained only on CT scans such that it can be efficiently adapted for use with PET scans when they become available. This framework is further extended to perform prognosis task maintaining the same efficient cross-modal fine-tuning approach. The proposed approach is tested with two well-known segementation backbones, namely UNETR and Swin UNETR. Our approach offers two main advantages. Firstly, we leverage the inherent modularity of the transformer architecture and perform low-rank adaptation (LoRA) as well as decomposed low-rank adaptation (DoRA) of the attention weights to achieve parameter-efficient adaptation. Secondly, by minimizing cross-modal entanglement, PEMMA allows updates using only one modality without causing catastrophic forgetting in the other. Our method achieves comparable performance to early fusion, but with only 8% of the trainable parameters, and demonstrates a significant +28% Dice score improvement on PET scans when trained with a single modality. Furthermore, in prognosis, our method improves the concordance index by +10% when adapting a CT-pretrained model to include PET scans, and by +23% when adapting for both PET and EHR data.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[58]
PEMMA: Parameter-Efficient Multi-Modal Adaptation for Medical Image Segmentation
Saadi, N., Saeed, N., Yaqub, M., Nandakumar, K., 2024. Pemma: Parameter-efficient multi-modal adaptation for medical image segmentation. URL: https://arxiv.org/abs/2404.13704, arXiv:2404.13704
work page Pith review arXiv 2024
-
[1]
Parameter- efficient fine-tuning of large pretrained models for instance segmen- tation tasks
AbouBaker,N.,Rohrschneider,D.,Handmann,U.,2024. Parameter- efficient fine-tuning of large pretrained models for instance segmen- tation tasks. Machine Learning and Knowledge Extraction 6, 2783–
work page 2024
-
[2]
Acosta, J.N., Falcone, G.J., Rajpurkar, P., Topol, E.J., 2022. Multi- modal biomedical ai. Nature Medicine 28, 1773–1784. doi:10.1038/ s41591-022-01981-2
work page 2022
-
[3]
Aghili, M., Tabarestani, S., Adjouadi, M., 2022. Addressing the missing data challenge in multi-modal datasets for the diagnosis of alzheimer’s disease. Journal of Neuroscience Methods 375, 109582. URL: https://www.sciencedirect.com/science/article/pii/ S0165027022001091, doi: https://doi.org/10.1016/j.jneumeth.2022. 109582
-
[4]
The challenges of diagnostic imaging in the era of big data
Aiello, M., Cavaliere, C., D’Albore, A., Salvatore, M., 2019. The challenges of diagnostic imaging in the era of big data. Journal of Clinical Medicine 8, 316. doi:10.3390/jcm8030316
-
[5]
A time-dependent discrimination index for survival data
Antolini, L., Boracchi, P., Biganzoli, E., 2005. A time-dependent discrimination index for survival data. Stat. Med. 24, 3927–3944
work page 2005
-
[6]
Medical image segmentation on mri images with missing modalities: A review
Azad, R., Khosravi, N., Dehghanmanshadi, M., Cohen-Adad, J., Merhof, D., 2022. Medical image segmentation on mri images with missing modalities: A review. URL: https://arxiv.org/abs/2203. 06217, arXiv:2203.06217
arXiv 2022
-
[7]
Bassi, P.R., Li, W., Tang, Y., Isensee, F., Wang, Z., Chen, J., Chou, Y.C.,Kirchhoff,Y.,Rokuss,M.R.,Huang,Z.,etal.,2024. Touchstone benchmark: Are we on the right way for evaluating ai algorithms for medical segmentation? Advances in Neural Information Processing Systems 37, 15184–15201
work page 2024
Show all 80 references
-
[8]
Bilic, P., Christ, P., Li, H.B., Vorontsov, E., Ben-Cohen, A., Kaissis, G., Szeskin, A., Jacobs, C., Mamani, G.E.H., Chartrand, G., et al.,
-
[9]
Monai: An open-source framework for deeplearninginhealthcare
Cardoso, M.J., Li, W., Brown, R., Ma, N., Kerfoot, E., Wang, Y., Murrey,B.,Myronenko,A.,Zhao,C.,Yang,D.,Nath,V.,He,Y.,Xu, Z., Hatamizadeh, A., Myronenko, A., Zhu, W., Liu, Y., Zheng, M., Tang, Y., Yang, I., Zephyr, M., Hashemian, B., Alle, S., Darestani, M.Z., Budd, C., Modat,...
2022 arXiv
-
[10]
Low-count whole-body pet with deep learning in a multicenter and externally validated study
Chaudhari, A.S., Mittra, E., Davidzon, G., Gulaka, P., Gandhi, H., Brown, A., Zhang, T., Srinivas, S., Gong, E., Zaharchuk, G., Jad- var, H., 2021. Low-count whole-body pet with deep learning in a multicenter and externally validated study. npj Digital Medicine 4, 127. URL:htt...
2021 doi
-
[11]
Ma-sam:Modality- agnostic sam adaptation for 3d medical image segmentation
Chen,C.,Miao,J.,Wu,D.,Yan,Z.,Kim,S.,Hu,J.,Zhong,A.,Liu,Z., Sun,L.,Li,X.,Liu,T.,Heng,P.A.,Li,Q.,2023a. Ma-sam:Modality- agnostic sam adaptation for 3d medical image segmentation. URL: https://arxiv.org/abs/2309.08842, arXiv:2309.08842
-
[12]
Chen, S., Li, A., Chen, J., Zhang, X., Jiang, C., Xu, J., 2022. Hybrid attention fusion segmentation network for diffuse large b- cell lymphoma in pet-ct, in: 2022 14th International Conference on WirelessCommunicationsandSignalProcessing(WCSP),pp.72–76. doi:10.1109/WCSP55476....
2022
-
[13]
Parameterizing con- text: Unleashing the power of parameter-efficient fine-tuning and in- context tuning for continual table semantic parsing
Chen, Y., Zhang, S., Qi, G., Guo, X., 2023b. Parameterizing con- text: Unleashing the power of parameter-efficient fine-tuning and in- context tuning for continual table semantic parsing. URL:https: //arxiv.org/abs/2310.04801, arXiv:2310.04801
-
[14]
Advanced mri and pet imaging for assessment of treatment response in patients with gliomas
Dhermain, F., Hau, P., Lanfermann, H., Jacobs, A., Bent, M., 2010. Advanced mri and pet imaging for assessment of treatment response in patients with gliomas. The Lancet Neurology 9, 906–920. doi:10. 1016/S1474-4422(10)70181-2
2010
-
[15]
Segvol: Universal and interactive volumetric medical image segmentation
Du, Y., Bai, F., Huang, T., Zhao, B., 2024. Segvol: Universal and interactive volumetric medical image segmentation. Advances in Neural Information Processing Systems 37, 110746–110783
2024
-
[16]
Dutt, R., Ericsson, L., Sanchez, P., Tsaftaris, S.A., Hospedales, T.,
-
[17]
Deep learning techniques in pet/ct imaging: A comprehensive review from sinogram to im- age space
Fallahpoor, M., Chakraborty, S., Pradhan, B., Faust, O., Barua, P.D., Chegeni, H., Acharya, R., 2024. Deep learning techniques in pet/ct imaging: A comprehensive review from sinogram to im- age space. Computer Methods and Programs in Biomedicine 243, 107880. URL: https://www.s...
2024
-
[18]
Early response evaluation using 18f-fdg-pet/ct does notinfluencemanagementofpatientswithmetastaticgastrointestinal stromal tumors (gist) treated with palliative intent
Farag, S., IJzerman, N.S., Houdijk, M.P., Reyners, A.K., Arens, A.I., Grünhagen, D.J., Desar, I.M., Gelderblom, H., Steeghs, N., de Geus- Oei, L.F., 2021. Early response evaluation using 18f-fdg-pet/ct does notinfluencemanagementofpatientswithmetastaticgastrointestinal stromal...
2021
-
[19]
Prompttuningforparameter- efficient medical image segmentation
Fischer,M.,Bartler,A.,Yang,B.,2022. Prompttuningforparameter- efficient medical image segmentation. URL:https://arxiv.org/abs/ 2211.09233, arXiv:2211.09233
2022 arXiv
-
[20]
A unified continual learning framework with general parameter-efficient tuning
Gao, Q., Zhao, C., Sun, Y., Xi, T., Zhang, G., Ghanem, B., Zhang, J., 2023. A unified continual learning framework with general parameter-efficient tuning. URL:https://arxiv.org/abs/2303.10070, arXiv:2303.10070
2023 arXiv
-
[21]
Increasing textual context size boosts medical image-text matching
Glassberg, I., Hope, T., 2023. Increasing textual context size boosts medical image-text matching. URL: https://arxiv.org/abs/2303. 13340, arXiv:2303.13340
2023 arXiv
-
[22]
Gollub, M.J., Hong, R., Sarasohn, D.M., Akhurst, T.,
-
[23]
Parameter- efficientfine-tuningforlargemodels:Acomprehensivesurvey
Han, Z., Gao, C., Liu, J., Zhang, J., Zhang, S.Q., 2024. Parameter- efficientfine-tuningforlargemodels:Acomprehensivesurvey. URL: https://arxiv.org/abs/2403.14608, arXiv:2403.14608
2024 arXiv
-
[24]
Harvard Health Publishing URL: https://www.health.harvard.edu/ cancer/radiation-risk-from-medical-imaging
HarvardHealthPublishing,n.d.Radiationriskfrommedicalimaging. Harvard Health Publishing URL: https://www.health.harvard.edu/ cancer/radiation-risk-from-medical-imaging
-
[25]
Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H., Xu, D.,
-
[26]
Unetr: Transformers for 3d medical image segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H., Xu, D., 2021. Unetr: Transformers for 3d medical image segmentation. URL: https://arxiv.org/abs/2103. 10504, arXiv:2103.10504
2021 arXiv
-
[27]
Swinunetr-v2: Stronger swin transformers with stagewise convolu- tions for 3d medical image segmentation, in: MICCAI (4), pp
He, Y., Nath, V., Yang, D., Tang, Y., Myronenko, A., Xu, D., 2023. Swinunetr-v2: Stronger swin transformers with stagewise convolu- tions for 3d medical image segmentation, in: MICCAI (4), pp. 416–
2023
-
[28]
The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes
Heller, N., Sathianathen, N., Kalapara, A., Walczak, E., Moore, K., Kaluzniak, H., Rosenberg, J., Blake, P., Rengel, Z., Oestreich, M., et al., 2019. The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes. arXi...
2019 arXiv
-
[29]
Parameter-efficient transfer learning for nlp
Houlsby,N.,Giurgiu,A.,Jastrzebski,S.,Morrone,B.,deLaroussilhe, Q.,Gesmundo,A.,Attariyan,M.,Gelly,S.,2019. Parameter-efficient transfer learning for nlp. URL: https://arxiv.org/abs/1902.00751, arXiv:1902.00751
2019 arXiv
-
[30]
Lora: Low-rank adaptation of large language models
Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., 2021. Lora: Low-rank adaptation of large language models. URL: https://arxiv.org/abs/2106.09685, arXiv:2106.09685
2021 arXiv
-
[31]
Vision transformer-based multimodal feature fusion network for lymphoma segmentation on pet/ct images
Huang, H., Qiu, L., Yang, S., Li, L., Nan, J., Li, Y., Han, C., Zhu, F., Zhao, C., Zhou, W., 2024. Vision transformer-based multimodal feature fusion network for lymphoma segmentation on pet/ct images. arXiv e-prints , arXiv–2402
2024
-
[32]
Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines
Huang,S.C.,Pareek,A.,Seyyedi,S.,Banerjee,I.,Lungren,M.,2020. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. npj Digital Medicine 3. doi:10.1038/s41746-020-00341-z
2020 doi
-
[33]
Recent breakthroughs in pet-ct multimodality imaging: Innovations and clinical impact
Hussain, D., Abbas, N., Khan, J., 2024. Recent breakthroughs in pet-ct multimodality imaging: Innovations and clinical impact. Bio- engineering 11. URL: https://www.mdpi.com/2306-5354/11/12/1213, doi:10.3390/bioengineering11121213
2024 doi
-
[34]
Modern diagnostic imaging technique applications and risk factors in the medical field: A review
Hussain,S.,Mubeen,I.,Ullah,N.,Shah,S.S.U.D.,Khan,B.,Zahoor, M., Ullah, R., Khan, F., Sultan, M.A., 2022. Modern diagnostic imaging technique applications and risk factors in the medical field: A review. BioMed Research International 2022. doi:10.1155/2022/ 5164970
2022 doi
-
[35]
Miccai multi-atlas labeling beyond the cranial vault– workshop and challenge, in: Proc
Igelsias, J., Styner, M., Langerak, T., Landman, B., Xu, Z., Klein, A., 2015. Miccai multi-atlas labeling beyond the cranial vault– workshop and challenge, in: Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge
2015
-
[36]
Amos: Alarge-scale abdominal multi- organbenchmarkforversatilemedicalimagesegmentation.Advances in neural information processing systems 35, 36722–36732
Ji,Y.,Bai,H.,Ge,C.,Yang,J.,Zhu,Y.,Zhang,R.,Li,Z.,Zhanng,L., Ma, W.,Wan, X., et al.,2022. Amos: Alarge-scale abdominal multi- organbenchmarkforversatilemedicalimagesegmentation.Advances in neural information processing systems 35, 36722–36732
2022
-
[37]
Ji, Z., Guo, D., Wang, P., Yan, K., Lu, L., Xu, M., Wang, Q., Ge, J., Gao, M., Ye, X., et al., 2023. Continual segment: Towards a single, unified and non-forgetting continual segmentation model of 143whole-bodyorgansinctscans,in:ProceedingsoftheIEEE/CVF International Conferenc...
2023
-
[38]
Visual prompt tuning, in: European conference on computer vision, Springer
Jia,M.,Tang,L.,Chen,B.C.,Cardie,C.,Belongie,S.,Hariharan,B., Lim, S.N., 2022. Visual prompt tuning, in: European conference on computer vision, Springer. pp. 709–727
2022
-
[39]
Accuracy assessment of an automaticimage-basedpet,in:ProceedingsofSPIE,theInternational SocietyforOpticalEngineering,SocietyofPhoto-OpticalInstrumen- tation Engineers
KADOURY, S., WOOD, B.J., VENKATESAN, A.M., DALAL, S., SHENG, X., KRUECKER, J., 2011. Accuracy assessment of an automaticimage-basedpet,in:ProceedingsofSPIE,theInternational SocietyforOpticalEngineering,SocietyofPhoto-OpticalInstrumen- tation Engineers
2011
-
[40]
Learning feature fusion via an interpretation method for tu- mor segmentation on pet/ct
Kang, S., Chen, Z., Li, L., Lu, W., Qi, X.S., Tan, S., 2023. Learning feature fusion via an interpretation method for tu- mor segmentation on pet/ct. Applied Soft Computing 148, 110825. URL: https://www.sciencedirect.com/science/article/pii/ S1568494623008438, doi:https://doi....
2023
-
[41]
Kirillov,A.,Mintun,E.,Ravi,N.,Mao,H.,Rolland,C.,Gustafson,L., Xiao,T.,Whitehead,S.,Berg,A.C.,Lo,W.Y.,Dollár,P.,Girshick,R.,
-
[42]
Advances in neuro-oncology imaging
Langen, K., Galldiks, N., Hattingen, E., Shah, N., 2017. Advances in neuro-oncology imaging. Nature Reviews Neurology 13, 279–289. doi:10.1038/nrneurol.2017.44
2017 doi
-
[43]
doi:10.1186/1472-6963-10-283
Langer,A.,2010.Asystematicreviewofpetandpet/ctinoncology:A waytopersonalizecancertreatmentinacost-effectivemanner? BMC Health Services Research 10. doi:10.1186/1472-6963-10-283
2010 doi
-
[44]
DeepHit: A deep learning approach to survival analysis with competing risks
Lee, C., Zame, W., Yoon, J., Van der Schaar, M., 2018. DeepHit: A deep learning approach to survival analysis with competing risks. Proc. Conf. AAAI Artif. Intell. 32
2018
-
[45]
Swincross: Cross-modalswintransformerforhead-and-necktumorsegmentation in pet/ct images
Li, G.Y., Chen, J., Jang, S., Gong, K., Li, Q., 2023. Swincross: Cross-modalswintransformerforhead-and-necktumorsegmentation in pet/ct images. Medical Physics 51, 2096–2107. URL: http: //dx.doi.org/10.1002/mp.16703, doi:10.1002/mp.16703
2023 doi
-
[46]
Howwelldosupervised3dmodels transfer to medical imaging tasks? arXiv preprint arXiv:2501.11253
Li,W.,Yuille,A.,Zhou,Z.,2025. Howwelldosupervised3dmodels transfer to medical imaging tasks? arXiv preprint arXiv:2501.11253
2025 arXiv
-
[47]
Parameter-efficient fine-tuning for continual learning: A neural tan- gent kernel perspective
Liu, J., Ji, Z., Yu, Y., Cao, J., Pang, Y., Han, J., Li, X., 2024a. Parameter-efficient fine-tuning for continual learning: A neural tan- gent kernel perspective. URL: https://arxiv.org/abs/2407.17120, arXiv:2407.17120
-
[48]
URL: https://arxiv.org/abs/2304.02643, arXiv:2304.02643
Segment anything. URL: https://arxiv.org/abs/2304.02643, arXiv:2304.02643
-
[49]
doi:10.1109/TPAMI.2021.3100536
Ma, J., Zhang, Y., Gu, S., Zhang, Y., Zhu, C., Wang, Q., Liu, X., An, X.,Ge,C.,Cao,S.,Zhang,Q.,Liu,S.,Wang,Y.,Li,Y.,Wang,C.,He, J.,Yang,X.,2020.Abdomenct-1k:Isabdominalorgansegmentationa solvedproblem? IEEETransactionsonPatternAnalysisandMachine Intelligence 44, 6695–6714. doi...
2020
-
[50]
La radiologia medica 125, 296–305
Masjedi, H., Zare, M., Siahpoush, N.K., Razavi-Ratki, S., Alavi, F., Shabani,M.,2019.Europeantrendsinradiology:investigatingfactors affecting the number of examinations and the effective dose. La radiologia medica 125, 296–305. doi:10.1007/s11547-019-01109-6
2019 doi
-
[51]
Multi-modal modality-masked diffusion network for brain mri synthesis with ran- dom modality missing
Meng, X., Sun, K., Xu, J., He, X., Shen, D., 2024. Multi-modal modality-masked diffusion network for brain mri synthesis with ran- dom modality missing. IEEE Transactions on Medical Imaging PP. doi:10.1109/tmi.2024.3368664
2024
-
[52]
grand-challenge.org/Data/
Organizers,H.C.,.Hecktorchallengedataset.URL: https://hecktor. grand-challenge.org/Data/. accessed: March 30 2025
2025
-
[53]
Pytorch:Animperativestyle,high-performancedeeplearninglibrary
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al., 2019. Pytorch:Animperativestyle,high-performancedeeplearninglibrary. Advances in neural information processing systems 32
2019
-
[54]
Patients undergoing recurrent ct scans: assessing the magnitude
Rehani,M.M.,Yang,K.,Melick,E.R.,Heil,J.,Šalát,D.,Sensakovic, W., Liu, B., 2019. Patients undergoing recurrent ct scans: assessing the magnitude. European Radiology 30, 1828–1836. doi:10.1007/ s00330-019-06523-y
2019
-
[55]
Dora:Weight-decomposedlow-rankadap- tation
Liu, S.Y., Wang, C.Y., Yin, H., Molchanov, P., Wang, Y.C.F., Cheng, K.T.,Chen,M.H.,2024b. Dora:Weight-decomposedlow-rankadap- tation. URL: https://arxiv.org/abs/2402.09353, arXiv:2402.09353
-
[56]
Radiomics: from qualitativetoquantitativeimaging
Rogers, W., Seetha, S.T., Refaee, T., Lieverse, R., Granzier, R.W.Y., Ibrahim, A., Keek, S., Sanduleanu, S., Primakov, S., Beuque, M., Marcus, D., van der Wiel, A.M., Zerka, F., Oberije, C., van Tim- meren, J.V., Woodruff, H.C., Lambin, P., 2020. Radiomics: from qualitativetoq...
2020 doi
-
[57]
U-net: Convolutional networks for biomedical image segmentation
Ronneberger, O., Fischer, P., Brox, T., 2015. U-net: Convolutional networks for biomedical image segmentation. URL:https://arxiv. N. Saeed et al.:Preprint submitted to Elsevier Page 16 of 17 Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosi...
2015 arXiv
-
[59]
An ensemble approach for patient prognosis of head and neck tumor using multimodal data, in: 3D Head and Neck Tumor Segmentation in PET/CT Challenge
Saeed, N., Al Majzoub, R., Sobirov, I., Yaqub, M., 2021. An ensemble approach for patient prognosis of head and neck tumor using multimodal data, in: 3D Head and Neck Tumor Segmentation in PET/CT Challenge. Springer, pp. 278–286
2021
-
[60]
Saeed, N., Sobirov, I., Al Majzoub, R., Yaqub, M., 2022. Tmss: An end-to-end transformer-based multimodal network for segmentation and survival prediction, in: International Conference on Medical ImageComputingandComputer-AssistedIntervention,Springer.pp. 319–329
2022
-
[61]
Thefutureofmultimodalartificialintelligencemodels forintegratingimagingandclinicalmetadata:Anarrativereview
Simon, B.D., Ozyoruk, K.B., Gelikman, D.G., Harmon, S.A., Türk- bey,B.,2024. Thefutureofmultimodalartificialintelligencemodels forintegratingimagingandclinicalmetadata:Anarrativereview. Di- agnostic and Interventional Radiology doi:10.4274/dir.2024.242631
2024
-
[62]
Comparing different ct, pet and mri multi-modality image combinations for deep learning-based head and neck tumor segmentation
Ren,J.,Eriksen,J.G.,Nijkamp,J.,Korreman,S.S.,2021. Comparing different ct, pet and mri multi-modality image combinations for deep learning-based head and neck tumor segmentation. Acta Oncologica 60, 1399–1406. URL: http://dx.doi.org/10.1080/0284186x.2021. 1949034, doi:10.1080/...
2021
-
[63]
Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries
Sung, H., Ferlay, J., Siegel, R., Laversanne, M., Soerjomataram, I., Jemal, A., Bray, F., 2021. Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians 71, 209 – 249. doi:10.332...
2021 doi
-
[64]
Positron emission tomogra- phy: Current challenges and opportunities for technological ad- vances in clinical and preclinical imaging systems
Vaquero, J.J., Kinahan, P., 2015. Positron emission tomogra- phy: Current challenges and opportunities for technological ad- vances in clinical and preclinical imaging systems. Annual Review of Biomedical Engineering 17, 385–414. doi: 10.1146/ annurev-bioeng-071114-040723
2015
-
[65]
Continual learninginmedicaldevices:Fda’sactionplanandbeyond.TheLancet Digital Health 3
Vokinger, K.N., Feuerriegel, S., Kesselheim, A.S., 2021. Continual learninginmedicaldevices:Fda’sactionplanandbeyond.TheLancet Digital Health 3. doi:10.1016/s2589-7500(21)00076-5
2021 doi
-
[66]
A comprehensive survey of continual learning: Theory, method and application
Wang, L., Zhang, X., Su, H., Zhu, J., 2024. A comprehensive survey of continual learning: Theory, method and application. URL:https: //arxiv.org/abs/2302.00487, arXiv:2302.00487
2024 arXiv
-
[67]
Missing-modalityenabled multi-modal fusion architecture for medical data
Wang,M.,Fan,S.,Li,Y.,Chen,H.,2023. Missing-modalityenabled multi-modal fusion architecture for medical data. URL: https:// arxiv.org/abs/2309.15529, arXiv:2309.15529
2023 arXiv
-
[68]
Deep learning based time-to-event analysis with pet, ct and joint pet/ct for head and neck cancer prognosis
Wang, Y., Lombardo, E., Avanzo, M., Zschaek, S., Weingärtner, J., Holzgreve, A., Albert, N.L., Marschner, S., Fanetti, G., Franchin, G., et al., 2022. Deep learning based time-to-event analysis with pet, ct and joint pet/ct for head and neck cancer prognosis. Computer Methods ...
2022
-
[69]
A large annotated medical image dataset for the development and evaluation of segmentation algorithms
Simpson, A.L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., et al., 2019. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint ...
2019 arXiv
-
[70]
Parameter- efficientfine-tuningmethodsforpretrainedlanguagemodels:Acriti- cal review and assessment
Xu, L., Xie, H., Qin, S.Z.J., Tao, X., Wang, F.L., 2023. Parameter- efficientfine-tuningmethodsforpretrainedlanguagemodels:Acriti- cal review and assessment. URL:https://arxiv.org/abs/2312.12148, arXiv:2312.12148
2023 arXiv
-
[71]
Yousefirizi, F., Amiri, S., Ibragimov, B., Gowdy, C., Uribe, C., Rahmim, A., 2023. Test-time augmentation towards improved cross- centerpet/ctsegmentationsbytheswinunetr,in:2023IEEENuclear Science Symposium, Medical Imaging Conference and International Symposium on Room-Temper...
2023
-
[72]
C-lora:Continuallow- rankadaptationforpre-trainedmodels
Zhang,X.,Bai,L.,Yang,X.,Liang,J.,2025a. C-lora:Continuallow- rankadaptationforpre-trainedmodels. URL: https://arxiv.org/abs/ 2502.17920, arXiv:2502.17920
-
[73]
Unified multi-modal image synthesis for missing modal- ity imputation
Zhang, Y., Peng, C., Wang, Q., Song, D., Li, K., Kevin Zhou, S., 2025b. Unified multi-modal image synthesis for missing modal- ity imputation. IEEE Transactions on Medical Imaging 44, 4–18. URL: http://dx.doi.org/10.1109/TMI.2024.3424785,doi: 10.1109/tmi. 2024.3424785. N. Saee...
-
[76]
De- sign and validate a dual-modality characteristic information fu- sion system based on probabilistic graphical models
Xia, X., Zhang, R., Yao, X., Huang, G., Tang, T., 2023. De- sign and validate a dual-modality characteristic information fu- sion system based on probabilistic graphical models. Research Square URL: https://doi.org/10.21203/rs.3.rs-2565336/v1, doi:10. 21203/rs.3.rs-2565336/v1
2023 doi
-
[426]
URL: https://doi.org/10.1007/978-3-031-43901-8_40
-
[2007]
Journal of Nuclear Medicine 48, 1583–1591
Limitations of ct during pet/ct. Journal of Nuclear Medicine 48, 1583–1591. URL: https://jnm.snmjournals. org/content/48/10/1583, doi: 10.2967/jnumed.107.043109, arXiv:https://jnm.snmjournals.org/content/48/10/1583.full.pdf. N. Saeed et al.:Preprint submitted to Elsevier Page ...
-
[2022]
URL: https://arxiv.org/abs/2201.01266, arXiv:2201.01266
Swin unetr: Swin transformers for semantic segmentation of braintumorsinmriimages. URL: https://arxiv.org/abs/2201.01266, arXiv:2201.01266
-
[2023]
Medicalimage analysis 84, 102680
Thelivertumorsegmentationbenchmark(lits). Medicalimage analysis 84, 102680
-
[2024]
(Eds.), Proceedings of The 7nd International Conference onMedicalImagingwithDeepLearning,PMLR.pp.406–425
Parameter-efficient fine-tuning for medical image analysis: The missed opportunity, in: Burgos, N., Petitjean, C., Vakalopoulou, M., Christodoulidis, S., Coupe, P., Delingette, H., Lartizien, C., Mateus, D. (Eds.), Proceedings of The 7nd International Conference onMedicalImagi...
-
[2807]
URL: https://www.mdpi.com/2504-4990/6/4/133, doi:10.3390/ make6040133
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.