{"id":"321ddd90-d5fb-456c-a141-bf6e7a7ffb7e","arxiv_id":"2411.10071","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"FedEvPrompt reports 77.26% average balanced accuracy on a 6-client federated ISIC2019 binary skin-lesion task by sharing uncertainty-selected attention maps, outperforming FedAvg and FedProx without sharing model parameters.","lead":"This paper combines evidential deep learning, prompt tuning, and attention-map distillation into a federated learning system for skin lesion classification. It reports higher average accuracy than standard federated baselines on a 6-client ISIC2019 split while sharing only attention maps, not model parameters.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (7) assumes raw attention-rollout maps from heterogeneous clients are directly comparable, but the paper provides no normalization or distributional check, so the reported gain could rest on an unvalidated distance metric.","rationale":"The paper's strongest result is empirical and its mechanism is novel, but the mechanism is the weakest link. For the central claim to hold, attention maps shared across Barcelona, Vienna, Queensland, and MSK4 must encode comparable spatial evidence after independent prompt training. Nothing in Section 3.2 establishes this, and the squared L2 loss in Eq. (7) is sensitive to scale and offset. The reader's verdict already flags this assumption, and the concern is reinforced by Table 1's 'higher performance across all clients' being contradicted by FedDistill on C1, C5, and C6. A normalization or distribution-shift check would settle whether the attention buffer is a meaningful distillation target or just a regularizer. I therefore do not move the reader's conditional verdict; the concern is the same one, and the proposed test should be a condition for accepting the mechanism claim.","tokens_in":8685,"tokens_out":5633,"duration_ms":58519,"concrete_test":"Compute the Wasserstein distance between the round-5 attention-map sets contributed by each pair of clients, normalized by the average within-client pairwise distance. If the normalized between-client distance is materially above 1, the buffer maps are not aligned. Then retrain FedEvPrompt with each attention map normalized to unit L1 norm before evaluating Eq. (7), keeping all other hyperparameters fixed. If the average balanced accuracy shifts by more than about 1 point, or if the ranking against FedAvgPers/FedDistill changes, the headline result depends on the unvalidated comparability assumption rather than on a robust knowledge-sharing mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Table 1 is that FedEvPrompt's 77.26 average balanced accuracy comes from sharing uncertainty-selected attention maps. The mechanism that must be true for this claim is Eq. (7): raw attention-rollout maps from six heterogeneous clients are directly comparable, so pixel-wise squared Euclidean distance is a meaningful distillation objective. That assumption is not demonstrated anywhere in the paper. Attention rollout outputs are probability flows over tokens; the amount of mass on the spatial grid can differ across clients, and prompts are trained independently with different learning rates, so maps will carry client-specific scale and focus biases. The paper reports no normalization of the buffer, no analysis of the distribution of maps, and no comparison of Eq. (7) against an alignment-invariant loss. Table 2 only contrasts uncertainty-based selection with random selection; it does not test whether cross-client maps are commensurable. If Eq. (7) is averaging incomparable distributions, the reported gain over FedAvg/FedProx could be a side effect of regularization rather than evidence that attention maps are a sound knowledge-sharing medium. Since the privacy and 'richer information' arguments both rest on this step, the comparability premise is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedEvPrompt, a federated learning method that trains per-client prompt vectors on a frozen Vision Transformer with an evidential learning objective, and shares knowledge across clients only by exchanging attention rollout maps of selected low-uncertainty samples. The method is evaluated on a six-client partition of ISIC2019 for binary skin lesion classification, reporting an average balanced accuracy of 77.26 ± 4.65, which is higher than the compared FedAvg, FedProx, FedProto, and FedDistill baselines. The central claim is that the uncertainty-aware attention buffer provides a privacy-preserving and effective knowledge-sharing mechanism.","tokens_in":8903,"tokens_out":5190,"duration_ms":48913,"significance":"If the empirical result is robust, the method offers a communication-efficient and privacy-oriented alternative to parameter sharing in federated learning for medical imaging, with a modest but consistent accuracy gain over standard baselines. The paper has several strengths: it uses a realistic multi-source skin lesion dataset, includes a clear ablation separating the contributions of the evidential prior, prompt design, and uncertainty-based buffer selection, and formalizes the knowledge-sharing mechanism precisely. However, the headline performance gain is not statistically validated, and the central assumption that raw attention maps from heterogeneous clients are directly comparable is not tested; these issues currently prevent the claims from being fully load-bearing.","major_comments":[{"comment":"The claim that FedEvPrompt is superior to the baselines is not supported by statistical evidence: only a single run is reported for each method, and the reported ± values are standard deviations across the six clients, not across independent random seeds. The average gap over FedAvgPers is only 0.76 points, well within the client-level variability (e.g., FedEvPrompt's std is 4.65), and the text's assertion that FedEvPrompt outperforms FedDistill 'across all clients' is contradicted by C5 (68.18 vs 70.45) and C6 (79.35 vs 80.33) in Table 1. Please report multiple seeds with significance tests (e.g., paired bootstrap or Wilcoxon) and correct the 'across all clients' wording.","section":"Section 4, Table 1"},{"comment":"The knowledge-sharing mechanism assumes that raw attention rollout maps from different clients are directly comparable in pixel-wise squared Euclidean distance. The paper provides neither a normalization step nor any analysis of the buffer's distribution, and the prompts on different clients are trained independently with different learning rates, so the maps may carry client-specific scale and focus biases. Because the entire distillation objective rests on this comparability, please add (a) a quantitative analysis of the buffer maps (e.g., mean/variance per client, distribution of mass), (b) a comparison of Eq. (7) against alignment-invariant alternatives (e.g., cosine distance, normalizing each map to unit norm or zero mean), or (c) a theoretical justification for why raw rollout maps are commensurable.","section":"Section 3.2, Eq. (7)"},{"comment":"The abstract and introduction claim that sharing attention maps 'ensures enhanced privacy preservation' compared to sharing parameters or synthetic images, but no privacy evaluation or threat model is provided. Since privacy preservation is a stated contribution, the paper should either report a concrete privacy analysis (e.g., reconstruction attack resilience) or explicitly scope the claim as an architectural argument, discussing known inversion risks of attention maps.","section":"Section 1, Abstract"}],"minor_comments":[{"comment":"There is a typo: 'as well ass in' should be 'as well as in'.","section":"Section 4, first paragraph"},{"comment":"The subscript notation 'ac,k,_' is undefined; use a placeholder such as 'ac,k,·' to denote the local attention map for the current training sample.","section":"Section 3.2, Eq. (7)"},{"comment":"The definition of g-prompts as '[b-prompts, t-prompts]' with both learning rates set to µ1 is introduced after the methodology; please clarify the distinction earlier and explain why the single-prompt variant is expected to underperform.","section":"Section 3, prompt definitions"},{"comment":"The citation 'Proof:' appearing in the equation text is a blog-style reference (Soch, 2021); please format it as a standard citation and move the 'Proof:' label out of the equation.","section":"Section 2, Eq. (5) reference"},{"comment":"The dataset is described as a 'real distributed setting,' but the six clients are constructed by partitioning a central ISIC2019 collection; please clarify that this is a simulated federation and discuss any limitations of this setup.","section":"Section 4, dataset description"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be a conference paper (ICPR 2024) submitted in extended form. The two main technical issues—lack of statistical validation and unexamined comparability of attention maps—are addressable with additional experiments, but they are central to the paper's claims. I would encourage the editor to request the revisions and also ask the authors to provide a reproducibility statement or code, since none is currently available."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FedEvPrompt is a reasonable next step in the line of communication-efficient FL: instead of sharing parameters, prototypes, or logits, each client shares a few attention-rollout maps from its lowest-uncertainty samples, and the others distill from that buffer. I've seen worse ideas. The combination of evidential deep learning, dual prompt sets, and attention-map KD is genuinely new as far as I know, and the paper is clearly written.\n\nThe empirical setup is a real strength: six geographically distinct ISIC2019 subsets, two-class problem, and a comparison against FedAvg, FedProx, FedProto, and FedDistill. The core result—77.26 average balanced accuracy vs 73.31 for FedAvg and 72.90 for FedDistill—is a real but modest gain, and the ablation showing uncertainty-selected maps beat random selection (77.26 vs 75.17) supports the mechanism.\n\nThe soft spots are mostly about evidence, not about the idea. First, there is one split, no multi-seed runs, no significance tests, and the reported std is across clients, not runs. The 0.76-point gap over FedAvgPers is well within noise. Second, the text claims FedEvPrompt shows \"higher performance across all clients\" versus FedProto and FedDistill, but Table 1 shows FedDistill wins on C1, C5, and C6; that overstatement should be fixed. Third, privacy is asserted, not measured—sharing attention maps may be safer than sharing gradients, but there is no attack or reconstruction experiment.\n\nOn the stress-test worry about Eq. (7): I think it is overstated. Attention rollout maps are typically normalized to a probability distribution over input tokens, so squared Euclidean distance between them is a defensible (if not invariant) divergence. The paper would be stronger with an analysis of the buffer's distribution, but I do not see this as a load-bearing flaw.\n\nWho is this for? Anyone working on privacy-preserving FL for medical imaging, especially with ViTs. It deserves a serious referee: the idea is original, the writing is clean, and the experiments are a good start. For publication, I would want seeds, significance tests, a corrected \"all clients\" claim, and at least a qualitative privacy discussion. That is revision-level work, not desk-reject material.\n\nRecommendation: send to peer review.","headline":"A novel uncertainty-selected attention-buffer mechanism for federated learning with a modest gain; the empirical claims need more runs and a corrected 'all clients' statement.","tokens_in":9518,"tokens_out":3118,"would_cite":false,"duration_ms":33224,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A federated skin-lesion model that shares only attention maps outperforms parameter-sharing baselines.","keywords":["federated learning","evidential deep learning","prompt tuning","knowledge distillation","attention rollout","skin lesion classification","privacy preservation","medical imaging"],"falsifier":"Measure the per-client distributions of attention buffer maps (means, variances, and pairwise distances) before and after training; if cross-client map distances are no smaller than within-client distances, and removing the buffer does not change balanced accuracy, then the distillation mechanism is not doing what the accuracy comparison claims. A simpler decisive check is to replace the buffer with noise maps of the same statistics and observe whether FedEvPrompt's advantage over FedAvg vanishes.","tokens_in":8391,"feed_emoji":"🩺","tokens_out":5245,"duration_ms":51440,"temperature":0.7,"pith_summary":"FedEvPrompt is a federated learning method that trains separate prompt sets on frozen vision transformers at each hospital or clinic and shares none of the model parameters or images. The only thing exchanged across clients is a small buffer of per-class attention maps, chosen by evidential uncertainty scores, and each client distills its own attention maps toward that buffer. On a six-client, real-world split of the ISIC2019 skin-lesion data, the paper reports an average balanced accuracy of 77.26 ± 4.65, above FedAvg (73.31), FedProx (75.97), FedProto (68.18), and FedDistill (72.90). The paper argues this shows that knowledge about decision focus can be shared more privately and more effectively than parameters or synthetic images.","feed_headline":"Federated skin-lesion model shares attention maps, beats FedAvg","feed_subtitle":"No parameters or images change hands: six clinical sites reach 77.26% balanced accuracy through attention-map distillation.","key_machinery":"The load-bearing mechanism is the uncertainty-aware attention buffer, a federation-wide collection $A = \\bigcup_{c,k,m} a_{c,k,m}$ of attention rollout maps, one map per class per client per selected sample, computed with the attention-rollout procedure of Abnar and Zuidema. Each round, each client selects the $M$ samples per class with lowest evidential vacuity $u = K/S$, where $S$ is total Dirichlet strength, and replaces its contribution to the buffer. During local prompt learning, the knowledge-distillation loss $L_{KD} = \\frac{1}{M}\\sum_{i,m} \\|a_{c,k,\\cdot} - a_{i,k,m}\\|^2$ pulls the current sample's attention map toward those in the buffer. The evidential loss uses a class-frequency-weighted Dirichlet prior $W_k = \\frac{K}{K-1}(1 - N_k/N)$ so that the skewed class distribution in federated skin-lesion data does not dominate training.","core_discovery":"The paper's central claim is that evidential learning plus prompt tuning plus attention-map distillation yields a federated learner that is both more private and more accurate than parameter-sharing alternatives on distributed skin-lesion classification. On the Fed-ISIC2019 test bed, with Barcelona, Vienna, Queensland, and MSK4 sources organized as six clients, FedEvPrompt reaches 77.26 ± 4.65 average balanced accuracy, beating FedAvgPers' 76.50 and FedProx's 75.97, and it does so without any model parameter sharing. The ablation shows uncertainty-based buffer selection (77.26) outperforms random selection (75.17), and keeping b-prompts and t-prompts private while sharing attention maps outperforms sharing the prompts themselves. The authors interpret this as evidence that the informative content of a model's decision process lives in its attention maps, and that evidential uncertainty supplies the right criterion for choosing which maps to share.","pith_inferences":["The paper leaves open whether raw attention rollout maps are commensurable across clients, because Equation 7 compares maps with squared Euclidean distance and no normalization, alignment, or scale check; a testable extension would be to whiten or align maps before the distance computation.","Attention maps are not pixel-level images, but they are spatial maps over the input, so the privacy claim rests on how much patient-identifying structure survives in a low-resolution attention map; a quantitative membership-inference or reconstruction study would sharpen that claim.","The buffer holds only the most confident samples per class, which could systematically under-represent hard or ambiguous lesions; an alternative buffer that also includes high-uncertainty samples, or that mixes vacuity and dissonance as the conclusion gestures at, is a natural extension.","Because the method only needs a frozen backbone and prompts, it could be adapted to settings with hundreds of clients or to tasks beyond dermoscopy, but the current evidence is a single binary task at five rounds, so scaling behavior is unknown."],"forward_implications":["Federated clients could share interpretable attention maps instead of parameters or images, shrinking the surface for gradient-inversion or sample-reconstruction attacks while keeping a knowledge-sharing channel.","Non-parameter-sharing federated methods could reach parity with or exceed parameter-sharing ones, since on this task the gap over FedAvg is about +4 percentage points of balanced accuracy, with lower inter-client variance than FedDistill.","Uncertainty-based selection of what to share is itself worth accuracy, because the ablation shows 77.26 versus 75.17 for random buffer selection, so the evidential prior is doing real work rather than serving as bookkeeping.","Separating prompts by layer (slow b-prompts for low-level features, fast t-prompts for task-specific ones) appears to matter, because the single-set g-prompts variants drop to 60 to 75 percent accuracy.","If attention maps carry the distilled knowledge, the same recipe could transfer to other frozen-backbone federated tasks where data is heterogeneous and privacy-sensitive."],"supporting_citations":[{"why":"Supplies the FedAvg parameter-averaging baseline that FedEvPrompt must beat.","marker":"McMahan et al. [2017]"},{"why":"Provides the FedProx baseline targeting non-i.i.d. data, compared in Table 1.","marker":"Li et al. [2020]"},{"why":"Defines the evidential MSE and KL losses that make up the evidential loss $L_\\epsilon$ used here.","marker":"Sensoy et al. [2018b]"},{"why":"Defines the attention rollout mechanism that produces the attention maps shared across clients.","marker":"Abnar and Zuidema [2020]"},{"why":"Supplies the FedProto prototype-sharing baseline that does not share parameters.","marker":"Tan et al. [2022]"},{"why":"Supplies the FedDistill logit-distillation baseline that does not share parameters.","marker":"Seo et al. [2022]"},{"why":"Provides the BCN20000 images that form the largest client in the Fed-ISIC2019 split.","marker":"Combalia et al. [2019]"},{"why":"Provides the HAM10000 images used for several clients in the federation.","marker":"Tschandl et al. [2018]"},{"why":"Provides the MSK4 images that form one client and part of the dataset benchmark.","marker":"Codella et al. [2018]"}],"fun_headline_variants":["Attention-map sharing beats parameter sharing in federated skin-lesion learning","Evidential prompts and attention distillation win federated skin-lesion","Federated skin-lesion model: share attention maps, not parameters","Attention maps, not weights: federated skin-lesion learning wins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that attention rollout maps from different clients are directly comparable through pixel-wise squared Euclidean distance, with no normalization, alignment, or check that the maps share a common scale and focus; if domain shift or prompt initialization biases the maps, the distillation loss could pull local models toward spurious averages.","fun_headline_variants_meta":{"raw":{"variants":["Attention-map sharing beats parameter sharing in federated skin-lesion learning","Evidential prompts and attention distillation win federated skin-lesion","Federated skin-lesion model: share attention maps, not parameters","Attention maps, not weights: federated skin-lesion learning wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001107,"raw_usage":{"total_tokens":4611,"prompt_tokens":935,"completion_tokens":3676,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":3601}},"tokens_in":551,"tokens_out":3676,"duration_ms":26008,"temperature":1.0,"reasoning_tokens":3601,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:00:43.131560+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the per-client distributions of attention buffer maps (means, variances, and pairwise distances) before and after training; if cross-client map distances are no smaller than within-client distances, and removing the buffer does not change balanced accuracy, then the distillation mechanism is not doing what the accuracy comparison claims. A simpler decisive check is to replace the buffer with noise maps of the same statistics and observe whether FedEvPrompt's advantage over FedAvg vanishes.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data","cited_arxiv_id":null,"evidence_quote":"Supplies the FedAvg parameter-averaging baseline that FedEvPrompt must beat."},{"cited_title":"Fedproto: Federated prototype learning across heterogeneous clients","cited_arxiv_id":null,"evidence_quote":"Supplies the FedProto prototype-sharing baseline that does not share parameters."},{"cited_title":"Federated knowledge distillation","cited_arxiv_id":null,"evidence_quote":"Supplies the FedDistill logit-distillation baseline that does not share parameters."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the BCN20000 images that form the largest client in the Fed-ISIC2019 split."},{"cited_title":"The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions","cited_arxiv_id":null,"evidence_quote":"Provides the HAM10000 images used for several clients in the federation."}],"review_version":1}