REVIEW 5 major objections 5 minor 27 references
Prototype-Guided and Lightweight Adapters for Inherent Interpretation and Generalisation in Federated Learning
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A federated setup communicating only lightweight adapters and class prototypes achieves 86.82% average accuracy on diabetic retinopathy while producing inherent visual explanations.
desk verdict A real but narrow idea—communicating adapters and prototypes in FL—undercut by an unvalidated interpretability claim and a table that contradicts the abstract's accuracy claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing objects are the adapter module and the prototype layer. The adapter is a bottleneck pair of convolutional layers, W_down ∈ $R^{{H'×W'×D×r}}$ and W_up ∈ $R^{{H'×W'×r×D}}$, applied residually to each of the four ResNet-50 blocks as h' = h + σ(h W_down) W_up, so only these small weights are learned and shared. The prototype layer holds m learnable vectors p_j ∈ $R^{{h×w×D}}$ that are matched to the feature map by squared ℓ2 distance at every spatial position; the negated distances form similarity scores used both to produce the class logits and, when upsampled, to render heatmaps that show where each prototype fires. A proximal penalty with coefficients µ1 and µ2 aligns each client's adapter and prototype parameters to the global reference, and the communicated global model is the average of these small parameter sets across clients.
What would settle it
Keep the federated protocol identical but replace the frozen ImageNet-pretrained ResNet-50 with a randomly initialised ResNet-50 of the same architecture (no pretraining). If average accuracy on EyePACS stays near the reported 86.82% and the prototype heatmaps still locate lesions, the claim that pretrained frozen features are sufficient is false, since untrained features would work as well. If accuracy collapses, the method's success depends on the pretrained backbone — exactly the transferability the paper leaves untested.
Extended reading notes
Core claim
The central claim is that communicating only adapter parameters α and prototype parameters ϕ — instead of the full backbone — is enough to handle non-IID client data and to make the federated model inherently interpretable. Locally, each client minimises a loss that combines cross-entropy with prototype clustering and separation penalties, adapter ℓ2 regularisation, and a proximal term pulling local θ_i toward the global reference θ_g; only θ_i = (α_i, ϕ_i) is sent to the server for averaging. With a frozen ResNet-50 backbone this procedure achieved 86.82% average accuracy across four EyePACS clinical sites, and the learned prototypes produced activation maps that consistently located disease-relevant retinal features (lesions on blood-vessel background). The same prototypes transferred to the external APTOS dataset with accuracies of 93.88–96.39% across the four clients, which the paper reads as evidence that the method learns core disease features rather than site-specific artefacts.
Load-bearing premise
The whole method relies on the ImageNet-pretrained ResNet-50 backbone staying frozen, assuming without testing that its features capture the lesions and vessels that define diabetic retinopathy in fundus images, since if they do not the adapters and prototypes cannot compensate and both accuracy and interpretability would collapse.
Editorial extensions
If this is right
- Each communication round transmits only the adapter and prototype parameters, not the full ResNet-50 weights, so communication load scales with the small bottleneck size rather than the backbone.
- The proximal penalty that pulls local θ_i toward the global reference is central to the accuracy gain: removing it dropped the adapter-only baseline from 87.58% to 72.02%.
- Prototypes learned at different clinical sites activate the same retinal disease markers, showing that a shared interpretable structure can emerge without sharing data.
- The external APTOS evaluation (93.88–96.39% accuracy across clients) supports the claim that the learned prototypes capture transferable disease features rather than site-specific patterns.
Reading between the lines
- The 0.76 percentage-point gap between the prototype-based method (86.82%) and the adapter-only baseline (87.58%) prices the interpretability guarantee: if a deployment does not need heatmaps, the adapter-only variant is the better accuracy choice.
- Because the backbone is frozen on natural-image features, a strong stress test would be applying the same protocol to a domain far from ImageNet (e.g., histopathology tiles); failure there would locate the limits of the frozen-feature premise.
- The consistency of prototype heatmaps across non-IID sites could be turned into a privacy-preserving audit tool: site disagreement about which regions fire would flag distribution shift, label noise, or new disease phenotypes.
- Since only adapters and prototypes are exchanged, the protocol is compatible in principle with clients running different backbone architectures, a form of model heterogeneity the paper lists as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a federated learning framework that combines lightweight adapter modules with a prototype-based classification layer. A frozen ImageNet-pretrained ResNet-50 backbone is augmented with four adapters, and a global set of adapter and prototype parameters is communicated between clients and server. Clients optimize a local loss consisting of cross-entropy, adapter regularization, prototype clustering/separation losses, and a proximal penalty. Experiments are conducted on a retinal fundus image dataset with four training clients defined by clinical sites and one test site, and an external APTOS evaluation is included. The paper claims that the method provides inherent interpretations via prototypes and improves accuracy over baseline algorithms.
Significance. If substantiated, the proposed framework would offer a useful combination of communication-efficient federated fine-tuning and prototype-based interpretability for medical imaging. The manuscript's strengths include the use of a real-world dataset with natural client splits from clinical sites, an external held-out evaluation on APTOS, and released code. However, the evidence currently presented does not establish the two main claims: the reported accuracy is lower than the closest baseline, and the interpretability evaluation is qualitative only. The absence of quantitative validation, unreported hyperparameters, and lack of error bars mean that the contribution is not yet convincingly benchmarked or reproducible.
major comments (5)
- [Abstract; Table 1] The abstract states that the method 'shows improvements in accuracy over baseline algorithms', but Table 1 reports an average accuracy of 86.82% for the proposed method versus 87.58% for FedAdapter; the proposed method only exceeds FedAdapter on Client 1 by 0.23% and is lower on Clients 2, 3, and 4. This directly contradicts the central performance claim. Either the claim should be revised to 'comparable accuracy' or the authors need to provide repeated-run statistics and a significance test to establish that the small difference is meaningful.
- [Section 4.2; Sections 2.1 and 2.3] The paper's distinct contribution is the prototype branch, but Section 4.2 contains only a few selected heatmaps and no quantitative interpretability evaluation. No prototype purity, no agreement with expert lesion annotations, and no comparison with existing explanation methods are provided. Moreover, the method borrows ProtoPNet [4] but omits the projection/push step that in ProtoPNet replaces each prototype with the nearest training patch of the same class. Without that step, the prototypes p_j are free feature-space vectors in a frozen ResNet-50 feature space, so the 'prototypical parts' highlighted in Figure 3 are not guaranteed to correspond to actual, class-grounded image patches. The 'inherent interpretation' claim is therefore unestablished.
- [Section 2.3; Section 3] Section 2.3 defines six regularization coefficients (β, λ_clst, λ_sep, γ, μ_1, μ_2), and the prototype layer involves the number of prototypes m and the bottleneck dimension r, but Section 3 reports no values for any of these. Because the local loss combines these terms and the method's behavior is likely sensitive to them, the experiments are not reproducible and the reader cannot assess whether the reported results are selective. The authors should report all hyperparameters and the validation procedure used to choose them.
- [Table 1; Section 4.3] No error bars, confidence intervals, or multiple-seed results are reported for any experiment. The average accuracy gap between the proposed method and FedAdapter is 0.76 percentage points, which is likely within run-to-run variation for deep models, especially given class-balanced sampling and relatively small client sizes. Without repeated runs, the ranking in Table 1 is not statistically meaningful. The same issue applies to the APTOS accuracy numbers in Section 4.3, which are point estimates with no measure of uncertainty.
- [Section 2.3; Section 3] A central claim is that communicating only adapters and prototypes reduces communication overhead, but the paper never quantifies the number of communicated parameters or bytes per round, nor compares this with communicating full ResNet-50 weights. The reader cannot verify the communication-efficiency advantage. The authors should report parameter counts for adapters and prototypes (including the dependence on m, h, w, and r) and, ideally, the total communication cost over the 100 rounds.
minor comments (5)
- [Section 2.1] There are grammatical errors such as 'the backbone weight are always frozen' and 'prevent vanish gradient'; these should be corrected to 'the backbone weights are always frozen' and 'prevent vanishing gradients'.
- [Figure 1 caption] The caption contains a duplicated article: 'A depiction of the the model' should be 'A depiction of the model'.
- [Section 4.2] The phrase 'disease makers' should be 'disease markers', and the sentence in Section 4.1 beginning 'FedAdapter only outperforming it by 0.23% for Client 1' is ambiguous; it should clarify that the proposed method outperforms FedAdapter on Client 1 by 0.23%.
- [Section 2.3] The prototype regularizer is written as ℓproto = λ_clst ℓclst(d) − λ_sep ℓsep(d) + γ∥ϕi∥1; the sign of the separation term should be clarified, since standard formulations typically add a penalty for small distances to wrong-class prototypes.
- [Section 4.3] The APTOS dataset is mentioned without a reference; a citation should be added, and the external evaluation would be strengthened by reporting the accuracy of the other baseline algorithms on APTOS as well.
Circularity Check
No circularity: the paper's claims are empirical comparisons; no prediction reduces by construction to fitted inputs or self-citation.
full rationale
The paper proposes an FL framework and evaluates it empirically, rather than deriving a prediction from an assumption that already contains the result. The central accuracy claim is an experimental comparison against baselines on EyePACS and a held-out APTOS test set, with no fitted parameter being renamed as a prediction. The interpretability claim rests on qualitative prototype visualizations and on adopting the ProtoPNet prototype definition; while the absence of ProtoPNet's projection step and of quantitative interpretability metrics is a real validation gap, it is a correctness/evidence concern, not a circular derivation. The only self-citation is the fundus circle cropping preprocessing tool [19], which is not load-bearing for the paper's main claims. No equation in Section 2 is equivalent to the reported results by construction, and no uniqueness theorem or prior-work-derived ansatz is used to force the method's form. Therefore, no significant circularity is present.
Assumptions & free parameters
free parameters (7)
- beta (adapter L2 coefficient)
- lambda_clst (prototype clustering weight)
- lambda_sep (prototype separation weight)
- gamma (prototype L1 weight)
- mu1 and mu2 (proximal coefficients)
- number of prototypes m
- adapter bottleneck dimension r
assumptions (3)
- domain assumption Frozen ImageNet-pretrained ResNet-50 features transfer to retinal fundus images.
- domain assumption Simple weighted averaging of prototypes and adapters across clients yields a meaningful global model.
- domain assumption The four clinical sites with different class balances approximate the non-IID heterogeneity FL must handle.
Cite this review
Pith. "Pith review of Prototype-Guided and Lightweight Adapters for Inherent Interpretation and Generalisation in Federated Learning." pith.science (2026). https://pith.science/paper/CPMMQ2WG
@misc{pith2026250705852,
author = {Pith},
title = {Pith review of: Prototype-Guided and Lightweight Adapters for Inherent Interpretation and Generalisation in Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/CPMMQ2WG}},
note = {Machine review of arXiv:2507.05852}
}
read the original abstract
Federated learning (FL) provides a promising paradigm for collaboratively training machine learning models across distributed data sources while maintaining privacy. Nevertheless, real-world FL often faces major challenges including communication overhead during the transfer of large model parameters and statistical heterogeneity, arising from non-identical independent data distributions across clients. In this work, we propose an FL framework that 1) provides inherent interpretations using prototypes, and 2) tackles statistical heterogeneity by utilising lightweight adapter modules to act as compressed surrogates of local models and guide clients to achieve generalisation despite varying client distribution. Each client locally refines its model by aligning class embeddings toward prototype representations and simultaneously adjust the lightweight adapter. Our approach replaces the need to communicate entire model weights with prototypes and lightweight adapters. This design ensures that each client's model aligns with a globally shared structure while minimising communication load and providing inherent interpretations. Moreover, we conducted our experiments on a real-world retinal fundus image dataset, which provides clinical-site information. We demonstrate inherent interpretable capabilities and perform a classification task, which shows improvements in accuracy over baseline algorithms.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[4]
Advances in neural information pro- cessing systems (2019)
Chen C., Li O., Tao D., Barnett A., Rudin C., Su J.K.: This looks like that: deep learning for interpretable image recognition. Advances in neural information pro- cessing systems (2019)
work page 2019
-
[1]
InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 383–393
Alhamoud K., Ghunaim Y., Alfarra M., Hartvigsen T., Torr P., Ghanem B., Bibi A., Ghassemi M.: FedmedICL: Towards holistic evaluation of distribution shifts in fed- erated medical imaging. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 383–393. Cham: Springer Nature Switzerland (2024)
work page 2024
-
[2]
International journal of medical informatics, 59–67 (2018)
Brisimi T.S., Chen R., Mela T., Olshevsky A., Paschalidis I.C., Shi W.: Federated learning of predictive models from federated electronic health records. International journal of medical informatics, 59–67 (2018)
work page 2018
-
[3]
arXiv preprint arXiv:2205.10162 (2022)
Cai D., Wu Y., Wang S., Lin F.X., Xu M.: FedAdapter: Efficient federated learning for modern NLP. arXiv preprint arXiv:2205.10162 (2022)
arXiv 2022
-
[5]
In- Proceedings of the AAAI Conference on Artificial Intelligence, 11285-11293 (2024)
Chen H., Zhang Y., Krompass D., Gu J., Tresp V.: FedDAT: An approach for foundation model finetuning in multi-modal heterogeneous federated learning. In- Proceedings of the AAAI Conference on Artificial Intelligence, 11285-11293 (2024)
work page 2024
-
[6]
IEEE conference on computer vision and pattern recog- nition, 248–255 (2009)
Deng J., Dong W., Socher R., Li L.J., Li K., Fei-Fei L.: Imagenet: A large-scale hi- erarchical image database. IEEE conference on computer vision and pattern recog- nition, 248–255 (2009)
work page 2009
-
[7]
Journal of diabetes science and technology (2009)
Cuadros J., Bresnick G.: EyePACS: an adaptable telemedicine system for diabetic retinopathy screening. Journal of diabetes science and technology (2009)
work page 2009
-
[8]
In Proceed- ings of the IEEE/CVF international conference on computer vision, 15076–15086 (2021)
Gong X., Sharma A., Karanam S., Wu Z., Chen T., Doermann D., Innanje A.: En- semble attention distillation for privacy-preserving federated learning. In Proceed- ings of the IEEE/CVF international conference on computer vision, 15076–15086 (2021)
work page 2021
Show all 27 references
-
[9]
Pattern Recognition (2024)
Guan H., Yap P.T., Bozoki A., Liu M.: Federated learning for medical image anal- ysis: A survey. Pattern Recognition (2024)
2024
-
[10]
In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016)
He K., Zhang X., Ren S., Sun J.: Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016)
2016
-
[11]
In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 16312-16322 (2023)
Huang W., Ye M., Shi Z., Li H., Du B.: Rethinking federated learning with domain shift: A prototype view. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 16312-16322 (2023)
2023
-
[12]
arXiv preprint arXiv:1412.6980 (2014)
Kingma D.P., Ba J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
2014 arXiv
-
[13]
IEEE signal processing magazine, 50–60 (2020)
LiT.,SahuA.K.,TalwalkarA.,SmithV.:Federatedlearning:Challenges,methods, and future directions. IEEE signal processing magazine, 50–60 (2020)
2020
-
[14]
Proceedings of Machine learning and systems, 429-450 (2020)
Li T., Sahu A.K., Zaheer M., Sanjabi M., Talwalkar A., Smith V.: Federated opti- mization in heterogeneous networks. Proceedings of Machine learning and systems, 429-450 (2020)
2020
-
[15]
arXiv preprint arXiv:2001.01523 (2020)
Liang P.P., Liu T., Ziyin L., Allen N.B., Auerbach R.P., Brent D., Salakhutdinov R., Morency L.P.: Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01523 (2020)
2020 arXiv
-
[16]
Mensah et al
Loftus T.J., Ruppert M.M., Shickel B., Ozrazgat-Baslanti T., Balch J.A., Efron P.A., Upchurch Jr G.R., Rashidi P., Tignanelli C., Bian J., Bihorac A.: Federated 10 S.O. Mensah et al. learning for preserving data privacy in collaborative healthcare research. Digital Health (2022)
2022
-
[17]
In Artificial intelligence and statistics, 1273–1282 (2017)
McMahan B., Moore E., Ramage D., Hampson S., y Arcas B.A.: Communication- efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, 1273–1282 (2017)
2017
-
[18]
NPJ digital medicine (2020)
Rieke N., Hancox J., Li W., Milletari F., Roth H.R., Albarqouni S., Bakas S., Galtier M.N., Landman B.A., Maier-Hein K., Ourselin S.: The future of digital health with federated learning. NPJ digital medicine (2020)
2020
-
[19]
URL: https://github.com/berenslab/fundus_circle_cropping(2023)
Müller S., Heidrich H., Koch L.M., Berens P.: Fundus Circle Cropping. URL: https://github.com/berenslab/fundus_circle_cropping(2023)
2023
-
[20]
NPJ digital medicine (2021)
Sadilek A., Liu L., Nguyen D., Kamruzzaman M., Serghiou S., Rader B., Ingerman A., Mellem S., Kairouz P., Nsoesie E.O., MacFarlane J.: Privacy-first health research with federated learning. NPJ digital medicine (2021)
2021
-
[21]
Ad- vances in neural information processing systems (2017)
Snell J., Swersky K., Zemel R.: Prototypical networks for few-shot learning. Ad- vances in neural information processing systems (2017)
2017
-
[22]
InProceedings of the AAAI conference on artificial intelligence, 8432–8440 (2022)
Tan Y., Long G., Liu L., Zhou T., Lu Q., Jiang J., Zhang C.: Fedproto: Feder- ated prototype learning across heterogeneous clients. InProceedings of the AAAI conference on artificial intelligence, 8432–8440 (2022)
2022
-
[23]
Advances in neural infor- mation processing systems, 7611-7623 (2020)
Wang J., Liu Q., Liang H., Joshi G., Poor H.V.: Tackling the objective inconsis- tency problem in heterogeneous federated optimization. Advances in neural infor- mation processing systems, 7611-7623 (2020)
2020
-
[24]
Nature communications (2022)
Wu C., Wu F., Lyu L., Huang Y., Xie X.: Communication-efficient federated learn- ing via knowledge distillation. Nature communications (2022)
2022
-
[25]
InProceedings of the 32nd ACM International Conference on Multimedia, 7172-7181 (2024)
Wu X., Liu X., Niu J., Wang H., Tang S., Zhu G., Su H.: Decoupling general and personalized knowledge in federated learning via additive and low-rank decompo- sition. InProceedings of the 32nd ACM International Conference on Multimedia, 7172-7181 (2024)
2024
-
[26]
APSIPA Transactions on Signal and Information Processing (2024)
Yang T., Yu X., McKeown M.J., Wang Z.J.: When federated learning meets med- ical image analysis: A systematic review with challenges and solutions. APSIPA Transactions on Signal and Information Processing (2024)
2024
-
[27]
International Journal of Web Information Systems, 77–99 (2022)
Yoo J.H., Jeong H., Lee J., Chung T.M.: Open problems in medical federated learning. International Journal of Web Information Systems, 77–99 (2022)
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.