REVIEW 2 major objections 3 minor 32 references
Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities
T0 review · 2 major / 3 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read This paper claims that a low-rank convolutional basis learned on source medical imaging modalities transfers to a never-seen modality, allowing onboarding at a fraction of full fine-tuning's cost.
desk verdict A careful PEFT study with a clean leave-one-domain-out protocol; the central transfer claim is plausible but hinges on a random-basis control that is underspecified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the factorization of each convolutional adapter into a down-projection basis A_i (a 3x3 convolution mapping input channels to r=16 channels) and an up-projection B_i (a 1x1 convolution mapping back to output channels), with the update added before batch normalization. During source training, A is learned jointly across CT and MRI with interleaved batches; at onboarding, A is frozen and only B and a fresh classification head are trained. The random-basis control—an otherwise identical adapter with A drawn at random and frozen—isolates whether the learned basis carries information beyond the low-rank bottleneck, and is the comparison that defines the paper's claim.
What would settle it
Take the learned convolutional basis A* from the source modalities, randomly permute or re-initialize its filter weights while preserving the same norms, freeze, and retrain only the up-projections B on chest X-ray. If accuracy stays near 87.29%, the 'learned' content is not causal; if it drops toward the random-control's 81.18%, the learned directions are. Alternatively, test the reverse direction (pretrain on X-ray+MRI, onboard CT): if transfer fails symmetrically, the finding is specific to this source-target pair, not a general property.
Extended reading notes
Core claim
The central claim is that low-rank convolutional adaptation directions learned on one set of medical imaging modalities transfer to a modality never seen during pre-training. Under a strict leave-one-domain-out protocol (backbone pre-trained on Kidney CT and Brain MRI, frozen forever; Chest X-ray withheld until onboarding), freezing the learned 3x3 basis A and training only the 1x1 up-projections B attains 87.29% accuracy with 14,340 trainable parameters, exceeding a matched random-basis control by 6.11 percentage points. The same experiment at the decision layer shows no reliable transfer, localizing the transferable structure to the convolutional feature stage. Adapter-based onboarding kee
Load-bearing premise
That the random-basis control is otherwise identical to the learned-basis variant, so the entire 6.11pp gap is attributable to the basis having been learned on the source modalities rather than to differences in initialization scale, conditioning, or batch-normalization statistics.
Editorial extensions
If this is right
- A new imaging modality can be added to a deployed system by training about 14K parameters, rather than a full model, with no degradation of existing modalities.
- The 6.11pp gap over the random-basis control means the learned basis itself encodes transferable structure—it is not merely a bottleneck effect.
- Adaptation must reach convolutional features; decision-layer PEFT alone leaves a 4.6pp gap on the unseen modality, so the site of adaptation matters more than the parameter count.
- A Mahalanobis score on frozen features can trigger onboarding automatically, catching 100% of unseen inputs at 95% source retention, making the procedure deployable end-to-end.
- The marginal cost of adding modalities does not grow with the number already supported, since backbone and basis are stored once.
Reading between the lines
- If convolutional basis transfer generalizes beyond this CT/MRI→X-ray direction, a hospital system could maintain one shared basis and add modalities almost for free; the paper's own limitation section leaves the reverse direction untested, so a full leave-one-out matrix would be the direct test.
- The random-basis control's strength depends on the random basis having the same scale and conditioning as the learned one; re-running the control with basis filters reinitialized to the same norm would tighten the causal reading.
- The result suggests a principled way to think about PEFT in medical imaging: rather than per-task low-rank subspaces, there may be a common 'imaging grammar' of low-level filters that survives across acquisition modalities.
- A mechanistic follow-up—comparing principal angles or filter spectra of learned vs random bases—could turn the empirical transfer into a predictive account of when transfer will succeed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies onboarding of an unseen medical imaging modality onto a frozen convolutional backbone under a leave-one-domain-out protocol. The backbone is pretrained on Kidney CT and Brain MRI, then frozen; Chest X-ray is withheld and later onboarded by low-rank convolutional adapters. The central claim is that the low-rank basis A learned on source modalities transfers: freezing A and training only up-projections B achieves 87.29% accuracy with 14,340 trainable parameters (0.78% of full fine-tuning), 6.11 pp above an identically sized random basis. Secondary claims: decision-layer adaptation is insufficient; adapter onboarding causes exactly zero forgetting; a Mahalanobis score on frozen features detects the unseen modality. Results are reported over three seeds with paired bootstrap CIs.
Significance. If the central result holds, it is a valuable empirical contribution: it demonstrates a parameter-efficient way to add a never-seen modality without touching a deployed representation, with a realistic protocol that avoids the all-domains pretraining pitfall. The design is mostly clean: same frozen backbone, identical fresh heads for all variants, matched trainable budgets, seed-level standard deviations, and honest reporting of the weak decision-layer transfer. The random-basis control is conceptually the right counterfactual, and the zero-forgetting measurement against full fine-tuning is a strength. However, the load-bearing 6.11pp gap rests on the random-basis control being truly matched, and the paper does not yet establish that. The current evidence is also limited to one source-to-target direction on a small custom backbone, as the authors acknowledge.
major comments (2)
- [Section III.C, Table 2, Eq. (2)] The random-basis control is the load-bearing counterfactual for the central 6.11pp transfer claim. The paper states the control is 'otherwise identical' but does not specify the initialization distribution or scale of the randomly initialized, frozen A, nor report its conditioning. Since the gradient for B scales with A(x) in Eq. (2), a random A with smaller Frobenius norm would slow B training, and a poorly conditioned A would span a less useful subspace. The 6.11pp gap could thus reflect optimization dynamics or subspace quality rather than transferable, modality-agnostic structure. Please report the norms, singular-value spectra, or effective rank of learned vs random A; match the scale (e.g., normalize both to equal norm); and verify that the random-basis control has converged (e.g., train longer or report loss curves). Without this, the central claim is not cleanly established.
- [Section III.B, Eq. (3), Section V.C] The paper injects the low-rank update before BatchNorm but does not state whether BN running statistics are updated during adapter training. If they are updated, then the backbone is not strictly frozen and the claim of exactly zero forgetting (Δ=0.00pp) needs qualification, since BN statistics are shared parameters. If they are frozen, the distribution shift induced by the adapter may render the frozen statistics miscalibrated, potentially affecting both the adapter's convergence and the validity of the pre-BN injection. Please state the BN handling and, if frozen, provide evidence that the pre-activation distribution remains compatible (e.g., report BN statistics before/after or compare with a variant that updates BN statistics).
minor comments (3)
- [Section III.D vs Section IV] The Mahalanobis covariance is said to be estimated on 'source training data only' in Section III.D, but Section IV says the validation split is used for estimating the covariance. Please reconcile this inconsistency.
- [References [6] and [17]] References [6] and [17] are identical (Veasey and Amini). Also, [6] is a medical-imaging LoRA application rather than the original LoRA paper (Hu et al.); please cite the original work and remove the duplication.
- [Throughout] The abbreviation 'pp' for percentage points is used without definition. Consider defining it at first use for readers outside the PEFT community.
Circularity Check
No significant circularity: the transfer claim is an empirical comparison against an external random-basis control, with the target modality withheld.
full rationale
The paper's central claim is an empirical transfer comparison, not a derivation. The target Chest X-ray is withheld from backbone pre-training and from shared-basis learning ('it is withheld from backbone pre-training and from shared-basis learning, and is introduced only at onboarding'), and the learned basis A* is obtained by optimizing Eq. 4 on source modalities only. Onboarding freezes A* and trains only B^(tgt) and a fresh head. The random-basis control is an independent counterfactual of identical parameter count; the learned-vs-random contrast (+6.11 pp) is measured on a held-out target test split and therefore cannot reduce to a fitted input. The paper explicitly disclaims novelty for routed LoRA or shared low-rank bases (Table 1), and the 'zero forgetting' result is explicitly acknowledged as by construction and then converted into a measurement against full fine-tuning (Section V-C), so no self-definitional step is concealed. No load-bearing self-citations or imported uniqueness theorems are used; the only noted concerns (unspecified random-basis initialization scale, single-seed capacity sweep, patient-level splitting) are soundness or generality limitations, not circularity. The result is self-contained against an external benchmark.
Assumptions & free parameters
free parameters (5)
- rank r =
16 (default; swept 8-64)
- scaling factor alpha =
32
- adapted block set L =
{3,4}
- class cap =
1500 images/class
- training epochs / learning rate / batch size =
12 epochs, LR 1e-3, batch 32
assumptions (4)
- domain assumption Random-basis control is a valid counterfactual: A_random is drawn from the same distribution/scale as A_learned and differs only in being unlearned.
- domain assumption Frozen batch-normalization statistics remain valid when adapter output is added before BN.
- domain assumption The public datasets, capped at 1,500 images/class and re-split by filename, provide a valid proxy for modality transfer.
- standard math Three seeds and paired bootstrap over per-sample correctness adequately capture uncertainty.
Cite this review
Pith. "Pith review of Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities." pith.science (2026). https://pith.science/paper/NDWSPDB5
@misc{pith2026260716888,
author = {Pith},
title = {Pith review of: Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities},
year = {2026},
howpublished = {\url{https://pith.science/paper/NDWSPDB5}},
note = {Machine review of arXiv:2607.16888}
}
abstract
Deploying a medical imaging model that must later accommodate a modality it has never seen is a recurring practical problem: retraining the shared representation is expensive and destroys performance on the modalities already in service. We study this \emph{onboarding} problem under a strict leave-one-domain-out protocol, in which a convolutional backbone is pre-trained on source modalities (Kidney CT and Brain MRI), frozen permanently, and then required to accommodate an unseen modality (Chest X-ray). Under this protocol we establish three findings. First, decision-layer parameter-efficient fine-tuning is insufficient when the backbone has never observed the target modality: a linear probe and fully-connected LoRA both fall well short, whereas convolutional LoRA recovers most of the achievable accuracy, showing that adaptation must reach the convolutional features. Second, and centrally, the low-rank convolutional \emph{basis} learned on the source modalities \emph{transfers}: freezing that basis and training only its up-projections onboards the unseen modality using just $0.78\%$ of full fine-tuning's parameters, at an accuracy $6.11$ percentage points above a random basis of identical size, while an equivalent decision-layer basis exhibits no reliable transfer. Third, adapter-based onboarding leaves source-modality accuracy exactly unchanged ($\Delta = 0.00$ pp), whereas full fine-tuning reaches the highest target accuracy only by catastrophically degrading the source modalities. A Mahalanobis score on frozen backbone features detects the unseen modality with high sensitivity at a strict source-retention threshold, providing a practical trigger for when onboarding is required. All results are reported over three seeds with paired bootstrap confidence intervals.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Continual lifelong learning with neural networks: A review,
G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, “Continual lifelong learning with neural networks: A review,”Neural Networks, vol. 113, pp. 54–71, 2019
2019
-
[2]
Domain adaptation for medical image analysis: A survey,
H. Guan and M. Liu, “Domain adaptation for medical image analysis: A survey,”IEEE Transactions on Biomedical Engineering, vol. 69, no. 3, pp. 1173–1185, 2022
2022
-
[3]
Overcoming catastrophic forgetting in neural networks,
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell, “Overcoming catastrophic forgetting in neural networks,”Proceedings of the Na- tional Academy of Sciences, vol. 114, no. 13, pp. 3521–3526, 2017
2017
-
[4]
Continual learning through synap- tic intelligence,
F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synap- tic intelligence,” inInternational Conference on Machine Learning, vol. 70, 2017, pp. 3987–3995
2017
-
[5]
Catastrophic forgetting in connectionist networks,
R. M. French, “Catastrophic forgetting in connectionist networks,” Trends in Cognitive Sciences, vol. 3, no. 4, pp. 128–135, 1999
1999
-
[7]
Parameter-efficient fine-tuning for medical image analysis: The missed opportunity,
R. Dutt, L. Ericsson, P. Sanchez, S. A. Tsaftaris, and T. Hospedales, “Parameter-efficient fine-tuning for medical image analysis: The missed opportunity,” inProceedings of the 7th International Confer- ence on Medical Imaging with Deep Learning, ser. Proceedings of Machine Learning Research, vol. 250, 2024, pp. 406–425
2024
-
[8]
Parameter-efficient transfer learning for nlp,
N. Houlsbyet al., “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning, 2019, pp. 2790–2799
2019
-
[9]
Adapterfusion: Non-destructive task composi- tion for transfer learning,
J. Pfeiffer and et al., “Adapterfusion: Non-destructive task composi- tion for transfer learning,”Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), 2020
2020
Show all 32 references
-
[10]
When MOE meets LLMs: Parameter efficient fine-tuning for multi- task medical applications,
Q. Liu, X. Wu, X. Zhao, Y . Zhu, D. Xu, F. Tian, and Y . Zheng, “When MOE meets LLMs: Parameter efficient fine-tuning for multi- task medical applications,” inProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024,...
2024
-
[11]
TT-LoRA MoE: Unifying parameter-efficient fine-tuning and sparse mixture-of-experts,
P. Kunwar, M. N. Vu, M. Gupta, M. Abdelsalam, and M. Bhattarai, “TT-LoRA MoE: Unifying parameter-efficient fine-tuning and sparse mixture-of-experts,”arXiv preprint arXiv:2504.21190, 2025
2025
-
[12]
VeRA: Vector- based random matrix adaptation,
D. J. Kopiczko, T. Blankevoort, and Y . M. Asano, “VeRA: Vector- based random matrix adaptation,” inInternational Conference on Learning Representations (ICLR), 2024
2024
-
[13]
Tied-LoRA: Enhanc- ing parameter efficiency of LoRA with weight tying,
A. Renduchintala, T. Konuk, and O. Kuchaiev, “Tied-LoRA: Enhanc- ing parameter efficiency of LoRA with weight tying,” inProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024
2024
-
[14]
Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,
Q. Zhang, M. Chen, A. Bukharin, N. Karampatziakis, P. He, Y . Cheng, W. Chen, and T. Zhao, “Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,” inInternational Conference on Learn- ing Representations, 2023
2023
-
[15]
Qlora: Efficient finetuning of quantized llms,
T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,” inAdvances in Neural Infor- mation Processing Systems, vol. 36, 2023, pp. 10 088–10 115
2023
-
[16]
Layer-wise low-rank adaptation for neural networks,
X. Liu, F. Zhuang, and Y . Wu, “Layer-wise low-rank adaptation for neural networks,”Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), 2021
2021
-
[17]
Low-rank adaptation of pre-trained large vision models for improved lung nodule malignancy classifica- tion,
B. P. Veasey and A. A. Amini, “Low-rank adaptation of pre-trained large vision models for improved lung nodule malignancy classifica- tion,”IEEE Open Journal of Engineering in Medicine and Biology, vol. 6, pp. 296–304, 2025
2025
-
[18]
Mixture of lora experts,
X. Wu, S. Huang, and F. Wei, “Mixture of lora experts,”arXiv preprint arXiv:2404.13628, 2024
2024 arXiv
-
[19]
Learning without forgetting,
Z. Li and D. Hoiem, “Learning without forgetting,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 12, pp. 2935–2947, 2017
2017
-
[20]
Memory aware synapses,
R. Aljundiet al., “Memory aware synapses,” inEuropean Conference on Computer Vision, 2017, pp. 139–154
2017
-
[21]
Progressive neural networks,
A. A. Rusu and et al., “Progressive neural networks,”arXiv, 2016. VOLUME , 11 Author et al.:
2016
-
[22]
Packnet: Adding multiple tasks to a single network by iterative pruning,
A. Mallya and S. Lazebnik, “Packnet: Adding multiple tasks to a single network by iterative pruning,”Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), 2018
2018
-
[23]
Simple, fast, and accurate neural machine translation,
A. Bapna and O. Firat, “Simple, fast, and accurate neural machine translation,”Proceedings of the 57th Annual Meeting of the Associa- tion for Computational Linguistics (ACL 2019), 2019
2019
-
[24]
Universal adapters: Efficient multi-task transfer learning,
S. Wang and et al., “Universal adapters: Efficient multi-task transfer learning,”Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), 2020
2020
-
[25]
Adapterfusion: Non-destructive task composition for transfer learning,
J. Pfeifferet al., “Adapterfusion: Non-destructive task composition for transfer learning,” inEMNLP, 2021
2021
-
[26]
Boosting continual learning of vision-language models via mixture- of-experts adapters,
J. Yu, Y . Zhuge, L. Zhang, P. Hu, D. Wang, H. Lu, and Y . He, “Boosting continual learning of vision-language models via mixture- of-experts adapters,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024
2024
-
[27]
LoraHub: Efficient cross-task generalization via dynamic LoRA composition,
C. Huang, Q. Liu, B. Y . Lin, T. Pang, C. Du, and M. Lin, “LoraHub: Efficient cross-task generalization via dynamic LoRA composition,” inConference on Language Modeling (COLM), 2024
2024
-
[28]
Ct kidney dataset: Normal-cyst-tumor and stone,
nazmul0087, “Ct kidney dataset: Normal-cyst-tumor and stone,” Kaggle dataset, n.d., available: https://www.kaggle.com/datasets/ nazmul0087/ct-kidney-dataset-normal-cyst-tumor-and-stone. Accessed: Mar. 26, 2026
2026
-
[29]
Brain tumor mri dataset,
Masoud Nickparvar, “Brain tumor mri dataset,” n.d., dataset. Available: https://www.kaggle.com/datasets/masoudnickparvar/ brain-tumor-mri-dataset. Accessed: Mar. 26, 2026
2026
-
[30]
Chest x-ray (Pneumonia, COVID-19, Tuberculosis),
J. Tiwari, “Chest x-ray (Pneumonia, COVID-19, Tuberculosis),” Kag- gle dataset, 2022, available: https://www.kaggle.com/datasets/jtiptj/ chest-xray-pneumoniacovid19tuberculosis. Accessed: Jan. 2026
2022
-
[31]
A baseline for detecting misclassified and out-of-distribution examples in neural networks,
D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” inInternational Conference on Learning Representations (ICLR), 2017
2017
-
[32]
Energy-based out-of- distribution detection,
W. Liu, X. Wang, J. D. Owens, and Y . Li, “Energy-based out-of- distribution detection,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 21 464–21 475
2020
-
[33]
A simple unified framework for detecting out-of-distribution samples and adversarial attacks,
K. Lee, K. Lee, H. Lee, and J. Shin, “A simple unified framework for detecting out-of-distribution samples and adversarial attacks,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 31, 2018. Ranat Das Prangonis an undergraduate stu- dent in Chemical Enginee...
2018
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.