REVIEW 3 major objections 5 minor 21 references
Evidential Federated Learning for Skin Lesion Image Classification
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A federated skin-lesion model that shares only attention maps outperforms parameter-sharing baselines.
desk verdict A novel uncertainty-selected attention-buffer mechanism for federated learning with a modest gain; the empirical claims need more runs and a corrected 'all clients' statement. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the uncertainty-aware attention buffer, a federation-wide collection $A = \bigcup_{c,k,m} a_{c,k,m}$ of attention rollout maps, one map per class per client per selected sample, computed with the attention-rollout procedure of Abnar and Zuidema. Each round, each client selects the $M$ samples per class with lowest evidential vacuity $u = K/S$, where $S$ is total Dirichlet strength, and replaces its contribution to the buffer. During local prompt learning, the knowledge-distillation loss $L_{KD} = \frac{1}{M}\sum_{i,m} \|a_{c,k,\cdot} - a_{i,k,m}\|^2$ pulls the current sample's attention map toward those in the buffer. The evidential loss uses a class-frequency-weighted Dirichlet prior $W_k = \frac{K}{K-1}(1 - N_k/N)$ so that the skewed class distribution in federated skin-lesion data does not dominate training.
What would settle it
Measure the per-client distributions of attention buffer maps (means, variances, and pairwise distances) before and after training; if cross-client map distances are no smaller than within-client distances, and removing the buffer does not change balanced accuracy, then the distillation mechanism is not doing what the accuracy comparison claims. A simpler decisive check is to replace the buffer with noise maps of the same statistics and observe whether FedEvPrompt's advantage over FedAvg vanishes.
Extended reading notes
Core claim
The paper's central claim is that evidential learning plus prompt tuning plus attention-map distillation yields a federated learner that is both more private and more accurate than parameter-sharing alternatives on distributed skin-lesion classification. On the Fed-ISIC2019 test bed, with Barcelona, Vienna, Queensland, and MSK4 sources organized as six clients, FedEvPrompt reaches 77.26 ± 4.65 average balanced accuracy, beating FedAvgPers' 76.50 and FedProx's 75.97, and it does so without any model parameter sharing. The ablation shows uncertainty-based buffer selection (77.26) outperforms random selection (75.17), and keeping b-prompts and t-prompts private while sharing attention maps outperforms sharing the prompts themselves. The authors interpret this as evidence that the informative content of a model's decision process lives in its attention maps, and that evidential uncertainty supplies the right criterion for choosing which maps to share.
Load-bearing premise
The load-bearing premise is that attention rollout maps from different clients are directly comparable through pixel-wise squared Euclidean distance, with no normalization, alignment, or check that the maps share a common scale and focus; if domain shift or prompt initialization biases the maps, the distillation loss could pull local models toward spurious averages.
Editorial extensions
If this is right
- Federated clients could share interpretable attention maps instead of parameters or images, shrinking the surface for gradient-inversion or sample-reconstruction attacks while keeping a knowledge-sharing channel.
- Non-parameter-sharing federated methods could reach parity with or exceed parameter-sharing ones, since on this task the gap over FedAvg is about +4 percentage points of balanced accuracy, with lower inter-client variance than FedDistill.
- Uncertainty-based selection of what to share is itself worth accuracy, because the ablation shows 77.26 versus 75.17 for random buffer selection, so the evidential prior is doing real work rather than serving as bookkeeping.
- Separating prompts by layer (slow b-prompts for low-level features, fast t-prompts for task-specific ones) appears to matter, because the single-set g-prompts variants drop to 60 to 75 percent accuracy.
- If attention maps carry the distilled knowledge, the same recipe could transfer to other frozen-backbone federated tasks where data is heterogeneous and privacy-sensitive.
Reading between the lines
- The paper leaves open whether raw attention rollout maps are commensurable across clients, because Equation 7 compares maps with squared Euclidean distance and no normalization, alignment, or scale check; a testable extension would be to whiten or align maps before the distance computation.
- Attention maps are not pixel-level images, but they are spatial maps over the input, so the privacy claim rests on how much patient-identifying structure survives in a low-resolution attention map; a quantitative membership-inference or reconstruction study would sharpen that claim.
- The buffer holds only the most confident samples per class, which could systematically under-represent hard or ambiguous lesions; an alternative buffer that also includes high-uncertainty samples, or that mixes vacuity and dissonance as the conclusion gestures at, is a natural extension.
- Because the method only needs a frozen backbone and prompts, it could be adapted to settings with hundreds of clients or to tasks beyond dermoscopy, but the current evidence is a single binary task at five rounds, so scaling behavior is unknown.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FedEvPrompt, a federated learning method that trains per-client prompt vectors on a frozen Vision Transformer with an evidential learning objective, and shares knowledge across clients only by exchanging attention rollout maps of selected low-uncertainty samples. The method is evaluated on a six-client partition of ISIC2019 for binary skin lesion classification, reporting an average balanced accuracy of 77.26 ± 4.65, which is higher than the compared FedAvg, FedProx, FedProto, and FedDistill baselines. The central claim is that the uncertainty-aware attention buffer provides a privacy-preserving and effective knowledge-sharing mechanism.
Significance. If the empirical result is robust, the method offers a communication-efficient and privacy-oriented alternative to parameter sharing in federated learning for medical imaging, with a modest but consistent accuracy gain over standard baselines. The paper has several strengths: it uses a realistic multi-source skin lesion dataset, includes a clear ablation separating the contributions of the evidential prior, prompt design, and uncertainty-based buffer selection, and formalizes the knowledge-sharing mechanism precisely. However, the headline performance gain is not statistically validated, and the central assumption that raw attention maps from heterogeneous clients are directly comparable is not tested; these issues currently prevent the claims from being fully load-bearing.
major comments (3)
- [Section 4, Table 1] The claim that FedEvPrompt is superior to the baselines is not supported by statistical evidence: only a single run is reported for each method, and the reported ± values are standard deviations across the six clients, not across independent random seeds. The average gap over FedAvgPers is only 0.76 points, well within the client-level variability (e.g., FedEvPrompt's std is 4.65), and the text's assertion that FedEvPrompt outperforms FedDistill 'across all clients' is contradicted by C5 (68.18 vs 70.45) and C6 (79.35 vs 80.33) in Table 1. Please report multiple seeds with significance tests (e.g., paired bootstrap or Wilcoxon) and correct the 'across all clients' wording.
- [Section 3.2, Eq. (7)] The knowledge-sharing mechanism assumes that raw attention rollout maps from different clients are directly comparable in pixel-wise squared Euclidean distance. The paper provides neither a normalization step nor any analysis of the buffer's distribution, and the prompts on different clients are trained independently with different learning rates, so the maps may carry client-specific scale and focus biases. Because the entire distillation objective rests on this comparability, please add (a) a quantitative analysis of the buffer maps (e.g., mean/variance per client, distribution of mass), (b) a comparison of Eq. (7) against alignment-invariant alternatives (e.g., cosine distance, normalizing each map to unit norm or zero mean), or (c) a theoretical justification for why raw rollout maps are commensurable.
- [Section 1, Abstract] The abstract and introduction claim that sharing attention maps 'ensures enhanced privacy preservation' compared to sharing parameters or synthetic images, but no privacy evaluation or threat model is provided. Since privacy preservation is a stated contribution, the paper should either report a concrete privacy analysis (e.g., reconstruction attack resilience) or explicitly scope the claim as an architectural argument, discussing known inversion risks of attention maps.
minor comments (5)
- [Section 4, first paragraph] There is a typo: 'as well ass in' should be 'as well as in'.
- [Section 3.2, Eq. (7)] The subscript notation 'ac,k,_' is undefined; use a placeholder such as 'ac,k,·' to denote the local attention map for the current training sample.
- [Section 3, prompt definitions] The definition of g-prompts as '[b-prompts, t-prompts]' with both learning rates set to µ1 is introduced after the methodology; please clarify the distinction earlier and explain why the single-prompt variant is expected to underperform.
- [Section 2, Eq. (5) reference] The citation 'Proof:' appearing in the equation text is a blog-style reference (Soch, 2021); please format it as a standard citation and move the 'Proof:' label out of the equation.
- [Section 4, dataset description] The dataset is described as a 'real distributed setting,' but the six clients are constructed by partitioning a central ISIC2019 collection; please clarify that this is a simulated federation and discuss any limitations of this setup.
Circularity Check
No significant circularity: FedEvPrompt's accuracy gain is an empirical comparison, and the only overlapping-author citation is background context, not load-bearing.
full rationale
The central result in Table 1 is a measured balanced accuracy obtained by training and evaluating on held-out test splits, not a quantity derived from the method's assumptions; no parameter is fitted to the reported accuracy and then renamed a prediction. The uncertainty-aware buffer selection is validated by an ablation contrasting KDrandom and KDuncertainty, which is a standard empirical comparison of two selection policies on the same federation, not a construction that forces the outcome. Equation (7), which uses squared Euclidean distance between raw attention rollout maps from different clients, rests on an assumption of cross-client comparability; that is a correctness and robustness risk, but not circularity, because the loss does not presuppose the reported ranking or accuracy. The evidential loss and KL terms are attributed to external work by Sensoy et al., and the Dirichlet KL identity is referenced to an external statproofbook derivation, so no load-bearing result depends on an unverified self-citation. The only self-citation is Pennisi et al. (2024) in the introduction, used to describe an alternative privacy-preserving strategy; it is not used to justify FedEvPrompt's design or results. Accordingly, no derivation step reduces to its own input. The score of 2 reflects the minor, non-load-bearing self-citation, not genuine circular reasoning.
Assumptions & free parameters
free parameters (9)
- mu1 (b-prompt learning rate) =
2.5e-4
- mu2 (t-prompt learning rate) =
5e-4
- lambda_KD =
1e-6
- lambda_KL annealing =
min(1, t/10)
- M (attention maps per class per client) =
5
- l (b-prompt layer count) =
3 of 12
- prompt sequence length =
50
- weight decay =
1e-2
- Wk weighted evidential prior =
K/(K-1)*(1 - Nk/N)
assumptions (6)
- standard math Dirichlet distribution is a conjugate prior for categorical distributions and the EDL evidence formulation.
- standard math The KL divergence formula for Dirichlet distributions (Eq. 5) is correct for the weighted prior sum w = K.
- domain assumption Attention rollout maps reflect the model's decision-relevant focus and can be compared across clients with L2 distance.
- domain assumption A frozen ViT pretrained on ImageNet provides transferable features across the six heterogeneous dermoscopic sources.
- ad hoc to paper Samples with lowest evidential uncertainty are the most informative for knowledge distillation.
- ad hoc to paper Sharing attention maps preserves privacy better than sharing parameters or synthetic images.
invented entities (1)
-
Uncertainty-aware attention buffer
Cite this review
Pith. "Pith review of Evidential Federated Learning for Skin Lesion Image Classification." pith.science (2026). https://pith.science/paper/DFNMU543
@misc{pith2026241110071,
author = {Pith},
title = {Pith review of: Evidential Federated Learning for Skin Lesion Image Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/DFNMU543}},
note = {Machine review of arXiv:2411.10071}
}
read the original abstract
We introduce FedEvPrompt, a federated learning approach that integrates principles of evidential deep learning, prompt tuning, and knowledge distillation for distributed skin lesion classification. FedEvPrompt leverages two sets of prompts: b-prompts (for low-level basic visual knowledge) and t-prompts (for task-specific knowledge) prepended to frozen pre-trained Vision Transformer (ViT) models trained in an evidential learning framework to maximize class evidences. Crucially, knowledge sharing across federation clients is achieved only through knowledge distillation on attention maps generated by the local ViT models, ensuring enhanced privacy preservation compared to traditional parameter or synthetic image sharing methodologies. FedEvPrompt is optimized within a round-based learning paradigm, where each round involves training local models followed by attention maps sharing with all federation clients. Experimental validation conducted in a real distributed setting, on the ISIC2019 dataset, demonstrates the superior performance of FedEvPrompt against baseline federated learning algorithms and knowledge distillation methods, without sharing model parameters. In conclusion, FedEvPrompt offers a promising approach for federated learning, effectively addressing challenges such as data heterogeneity, imbalance, privacy preservation, and knowledge sharing.
Figures
Reference graph
Works this paper leans on
-
[1]
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan et al. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273--1282. PMLR, 2017
work page 2017
-
[2]
Federated optimization in heterogeneous networks
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2: 0 429--450, 2020
2020
-
[3]
Evidential deep learning to quantify classification uncertainty
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems, 31, 2018 a
work page 2018
-
[4]
Federated Uncertainty-Aware Aggregation for Fundus Diabetic Retinopathy Staging
Meng Wang, Lianyu Wang, Xinxing Xu, Ke Zou, Yiming Qian, Rick Siow Mong Goh, Yong Liu, and Huazhu Fu. Federated uncertainty-aware aggregation for fundus diabetic retinopathy staging. arXiv preprint arXiv:2303.13033, 2023
work page Pith review arXiv 2023
-
[5]
Prefix-tuning: Optimizing continuous prompts for generation
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021
arXiv 2021
-
[6]
Visual prompt based personalized federated learning
Guanghao Li, Wansen Wu, Yan Sun, Li Shen, Baoyuan Wu, and Dacheng Tao. Visual prompt based personalized federated learning. arXiv preprint arXiv:2303.08678, 2023
arXiv 2023
-
[7]
3D-U-SAM Network For Few-shot Tooth Segmentation in CBCT Images
Yifu Zhang, Zuozhu Liu, Yang Feng, and Renjing Xu. 3d-u-sam network for few-shot tooth segmentation in cbct images. arXiv preprint arXiv:2309.11015, 2023
work page Pith review arXiv 2023
-
[8]
Inverting gradients-how easy is it to break privacy in federated learning? Advances in Neural Information Processing Systems, 33: 0 16937--16947, 2020
Jonas Geiping, Hartmut Bauermeister, Hannah Dr \"o ge, and Michael Moeller. Inverting gradients-how easy is it to break privacy in federated learning? Advances in Neural Information Processing Systems, 33: 0 16937--16947, 2020
2020
Show all 21 references
-
[9]
Deep leakage from gradients
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in neural information processing systems, 32, 2019
2019
-
[10]
Feder: Federated learning through experience replay and privacy-preserving data synthesis
Matteo Pennisi, Federica Proietto Salanitri, Giovanni Bellitto, Bruno Casella, Marco Aldinucci, Simone Palazzo, and Concetto Spampinato. Feder: Federated learning through experience replay and privacy-preserving data synthesis. Computer Vision and Image Understanding, 238: 0 1...
2024
-
[11]
Federated knowledge distillation
Hyowoon Seo, Jihong Park, Seungeun Oh, Mehdi Bennis, and Seong-Lyun Kim. Federated knowledge distillation. Machine Learning and Wireless Communications, page 457, 2022
2022
-
[12]
Fedproto: Federated prototype learning across heterogeneous clients
Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8432--8440, 2022
2022
-
[13]
Evaluation of uncertainty quantification in deep learning
Niclas St hl, G \"o ran Falkman, Alexander Karlsson, and Gunnar Mathiason. Evaluation of uncertainty quantification in deep learning. In International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, pages 556--568. Springer, 2020
2020
-
[14]
Evidential deep learning to quantify classification uncertainty
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran ...
2018
-
[15]
Quantifying attention flow in transformers
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928, 2020
2005 arXiv
-
[16]
JoramSoch(2021). Proof:. Kullback-leibler divergence for the dirichlet distribution. https://statproofbook.github.io/P/dir-kl.html ; DOI:10.5281/zenodo.4305949
2021 doi
-
[17]
Marc Combalia, Noel C. F. Codella, Veronica Rotemberg, Brian Helba, Veronica Vilaplana, Ofer Reiter, Cristina Carrera, Alicia Barreiro, Allan C. Halpern, Susana Puig, and Josep Malvehy. Bcn20000: Dermoscopic lesions in the wild, 2019
2019
-
[18]
Noel CF Codella, David Gutman, M Emre Celebi, Brian Helba, Michael A Marchetti, Stephen W Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, et al. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedic...
2017
-
[19]
The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5 0 (1): 0 1--9, 2018
2018
-
[20]
Uncertainty characteristics of subjective opinions
Audun Josang, Jin-Hee Cho, and Feng Chen. Uncertainty characteristics of subjective opinions. In 2018 21st International Conference on Information Fusion (FUSION), pages 1998--2005. IEEE, 2018
2018
-
[21]
A survey on uncertainty reasoning and quantification for decision making: Belief theory meets deep learning
Zhen Guo, Zelin Wan, Qisheng Zhang, Xujiang Zhao, Feng Chen, Jin-Hee Cho, Qi Zhang, Lance M Kaplan, Dong H Jeong, and Audun J sang. A survey on uncertainty reasoning and quantification for decision making: Belief theory meets deep learning. arXiv preprint arXiv:2206.05675, 2022
2022 arXiv
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.