{"id":"883ba830-bb30-4da1-8875-ed65075fa4dd","arxiv_id":"2506.21144","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"pFedDC combines global and local text and vision prompts with cross-attention fusion to personalize federated CLIP models under label and domain shift.","lead":"This paper introduces pFedDC, a federated learning method that gives each client both shared and personal prompts for text and image encoders, fused with cross-attention. It reports accuracy gains over existing federated prompt-learning methods on nine benchmarks with label and domain shifts.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unexplained 'pseudo label regeneration' in Sec. 4.1 is a potential hidden self-training confound; without an ablation showing it is applied uniformly or has no effect, the claimed consistent superiority of pFedDC over FedOTP and pFedPrompt is not established.","rationale":"After reading the manuscript in full, I concur with the reader's conditional verdict. The most load-bearing concern is the unexplained pseudo-label regeneration step in Sec. 4.1. The method section and Algorithm 1 use only true-label cross-entropy; the existence of a pseudo-label regeneration procedure that is not defined, not included in the algorithm, and not ablated means the reported results may not correspond to the proposed method as described. This is a direct confound for the central claim of consistent SOTA performance, because self-training is a known performance booster in label-skewed settings. I considered other potential weaknesses -- the anomalously low zero-shot CLIP numbers on domain-shift datasets (Table 3) and the omission of recently cited baselines such as pFedMoAP -- but those would affect all methods similarly or are completeness issues rather than threats to the method's internal validity. The pseudo-label step, if applied asymmetrically, would invalidate the comparison. The concrete test I propose (ablate the step) would settle whether the concern lands. If the margin survives, the paper's empirical claims are credible, and the remaining issues (error bars, baseline coverage) would be minor. Therefore, the verdict stays CONDITIONAL until the authors clarify or ablate this step.","tokens_in":13961,"tokens_out":8744,"duration_ms":100352,"concrete_test":"Obtain the code (or re-run) and compare pFedDC with and without the pseudo-label regeneration step on the Table 1 label-shift benchmark: train pFedDC using only the true-label cross-entropy loss as defined by Eq. (6) and Algorithm 1, with all other hyperparameters fixed. Also run FedOTP and pFedPrompt with the same step disabled. If the pFedDC-vs-FedOTP margin disappears or shrinks substantially, the reported superiority is caused by pseudo-labeling; if the margin holds, the central claim survives conditional on documenting the step.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 (Implementation details) states: 'We conduct 20 communication rounds for all experimental datasets, regenerating pseudo labels using the updated local model every 5 communication rounds.' This is the only occurrence of pseudo-labeling in the paper. The method section (Sec. 3) and Algorithm 1 define a fully supervised objective (Eq. (6)) with true labels y and cross-entropy loss; no unlabeled data, pseudo-label generation rule, or use of pseudo-labels in local training is described. No ablation isolates this step. If this regeneration is applied only to pFedDC and not to the baselines, it is an additional self-training mechanism. Self-training with pseudo-labels is known to improve accuracy under label shift, so the reported gains (e.g., Table 1 avg 94.43 vs FedOTP 93.06; Table 4 avg 47.80 vs FedOTP 46.49) could derive from this step rather than from dual-prompt cross-fusion. If it is applied to all methods, the paper must specify the pseudo-label generation protocol for each baseline to ensure a fair comparison and reproducibility. The central claim is 'consistently outperforms state-of-the-art'; a hidden confound in the training procedure directly threatens this claim. The absence of any ablation or even a definition makes the discrepancy between the described method and the evaluated method a load-bearing issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes pFedDC, a personalized federated learning method for CLIP-based prompt tuning. Each client learns global and local text and vision prompts; global prompts are aggregated with FedAvg while local prompts and per-client cross-attention modules remain local. The cross-attention modules fuse global and local prompts before feeding the frozen text and image encoders. Experiments on seven label-shift datasets and on DomainNet and Office-Caltech10 under domain shift and joint domain+label shift compare pFedDC with Zero-shot CLIP, PromptFL, Promptprox, pFedPrompt, and FedOTP, reporting mean accuracies over three trials, robustness analyses over client numbers, participation rates and communication rounds, an RN50 backbone study, and an ablation of the cross-attention modules. The paper claims that pFedDC consistently outperforms state-of-the-art methods.","tokens_in":14214,"tokens_out":6860,"duration_ms":73327,"significance":"If the empirical claims are reliable, pFedDC is a useful contribution to federated prompt learning: it extends prior text-only or single-prompt personalization to simultaneous vision and text prompt personalization and introduces a plausible mechanism for combining global and local knowledge through cross-attention. The paper is original relative to FedOTP and pFedPrompt, and the benchmark suite covers meaningful heterogeneity types. Strengths include the ablation isolating the two cross-attention modules, the robustness experiments under varying client counts, participation rates, and communication rounds, and the additional RN50 backbone check. However, the central empirical claim is currently undermined by an unexplained pseudo-label regeneration step in the implementation, missing variance reporting, and a demonstrable error in the Table 5 averages; these issues must be resolved before the stated conclusion can be accepted.","major_comments":[{"comment":"The implementation details state: 'We conduct 20 communication rounds for all experimental datasets, regenerating pseudo labels using the updated local model every 5 communication rounds.' This is the only occurrence of pseudo-labeling in the paper. Section 3 and Algorithm 1 define a fully supervised objective with true labels y and cross-entropy loss (Eq. (6)), and no unlabeled data, pseudo-label generation rule, or use of pseudo-labels in local training is described. Since self-training with pseudo-labels is known to improve accuracy under label shift, the reported gains (e.g., Table 1 avg 94.43 vs. FedOTP 93.06; Table 4 avg 47.80 vs. FedOTP 46.49) could be due to this unexplained mechanism rather than to the dual-prompt cross-fusion design. The paper must either remove this sentence, specify precisely how pseudo-labels are generated and used, and show that the same procedure is applied to all baselines, or provide an ablation that isolates its effect.","section":"Sec. 4.1"},{"comment":"The average column in Table 5 reports exactly the same values as Table 1 (e.g., pFedDC 94.43, FedOTP 93.06) even though the per-dataset entries differ substantially. Averaging the seven listed RN50 entries gives approximately 91.58 for pFedDC and 90.07 for FedOTP, not the reported 94.43 and 93.06. This suggests the averages were copied from the ViT-B/16 table and invalidates the claim that pFedDC 'consistently outperforms' with the RN50 backbone. Please recompute the averages, or explain if the average is computed over a different set of runs.","section":"Table 5"},{"comment":"All tables report only the mean of three trials, with no standard deviations, and the curves in Fig. 3 show no error bars. Several reported advantages are small (e.g., Table 4 DomainNet average 47.80 vs. 46.49; Office-Caltech10 Amazon 27.36 vs. 25.54), so without variance or significance testing the 'consistently outperforms' claim is not statistically supported. Please report per-trial results or standard deviations/error bars and, where appropriate, significance tests.","section":"Tables 1-4 and Fig. 3"},{"comment":"The baseline set is too narrow for the 'state-of-the-art' claim. The Related Work cites recent personalized federated prompt-learning methods that are not compared, including pFedMoAP [57], FedAPT [32], FedCLIP [30], FedPR [53], pFedPG [10], and DiPrompt [18]. At least the methods specifically designed for joint label and domain shift (e.g., pFedMoAP, DiPrompt) should be included; without them the experiments establish only an advantage over PromptFL, Promptprox, pFedPrompt, and FedOTP.","section":"Sec. 2.2 and Sec. 4.1"},{"comment":"On DTD under Dirichlet-based label shift with beta=0.1, pFedDC (91.42) is below FedOTP (92.77), so 'consistently outperforms state-of-the-art methods' is not literally accurate even within the reported baselines. Please either revise the claim or provide an explanation for this exception.","section":"Table 1"}],"minor_comments":[{"comment":"The DTD entry '91.9587.03' appears to be missing a separator and should read '91.95 87.03'.","section":"Table 2"},{"comment":"The legend uses 'FedOPT' while the text and tables use 'FedOTP'; please unify the notation.","section":"Fig. 3"},{"comment":"Equation (1) has an unmatched parenthesis: 'exp(sim(f^v(x),f^t_c))/τ)' should be 'exp(sim(f^v(x),f^t_c)/τ)'.","section":"Eq. (1)"},{"comment":"Algorithm 1 line 8 initializes P^r_k <- P^{r-1}_k, but Section 3.2 says global prompts are replaced by the server aggregate while local prompts are retained; please clarify the exact local state update to avoid ambiguity.","section":"Algorithm 1"},{"comment":"Several table captions contain typographical artifacts, such as 'T able', 'EuroSATA vg.', and 'A vg.'; please clean these up.","section":"Table captions"},{"comment":"If the pseudo-label sentence is retained, the phrase 'using the updated local model' is unclear because only prompts and attention modules are trained while the CLIP encoders are frozen; please specify what 'local model' means.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The pseudo-label confound and the Table 5 average-column inconsistency are more than presentation issues; the authors should be asked for raw per-trial results and a clear statement of the exact training protocol applied to every baseline before this paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"pFedDC is a plausible, incremental but useful method for personalized federated prompt learning. The training loop has an unexplained pseudo-label regeneration step in Sec. 4.1 that is not in the method or Algorithm 1, and that is the one thing I would want resolved before trusting the benchmark claims.\n\nWhat is actually new: the cross-attention fusion of global and local prompts across both text and vision modalities. The paper is honest that PromptFL and Promptprox tune both text and vision prompts, and it cites DiPrompt and Dual Prompt Tuning for domain-aware FL. So the first-contribution claim (\"first to investigate dual-mode prompt tuning in FL\") is too strong. The distinctive bit is the client-side cross-attention fusing global and local prompts in each modality. That is a reasonable and testable idea. The ablation in Table 6 shows both attention modules help, and the gains over FedOTP are consistent across most settings.\n\nSoft spots: first, the pseudo-label regeneration sentence. It appears once in Implementation Details, with no definition, no ablation, and no mention in the method. If it is a self-training step used only for pFedDC, the reported margins could come from that rather than the fusion design. This is a fair concern, and I do not think the paper answers it. Second, only mean accuracies over three trials are reported, with no variance or significance. Given the margins are sometimes small (e.g., Table 4 avg 47.80 vs 46.49), that matters. Third, several cited recent methods like pFedMoAP and DiPrompt are not compared. Fourth, the novelty claim conflicts with the paper's own citation list.\n\nOverall: the central mechanism is coherent, no mathematical error is apparent, and the empirical direction is credible. But the missing description of pseudo-labeling is a real gap in the method-to-experiment chain, not a nitpick. This paper is for researchers working on federated prompt learning, and a serious referee could give useful guidance. I would send it to review with a request for clarification and an ablation of the pseudo-label step.","headline":"Useful incremental FL prompt method, but the unexplained pseudo-label regeneration step in Sec. 4.1 is a real confound that should be resolved before the SOTA claims are taken at face value.","tokens_in":14764,"tokens_out":1766,"would_cite":false,"duration_ms":18993,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"pFedDC claims that fusing global and local text and vision prompts through per-client cross-attention consistently outperforms prior federated prompt-learning methods on nine heterogeneous datasets.","keywords":["federated learning","personalized federated learning","prompt learning","vision-language models","dual-prompt optimization","cross-attention fusion","label shift","domain shift"],"falsifier":"Run the same nine-dataset comparison with the pseudo-label regeneration step either removed from pFedDC or added to every baseline; if the accuracy gap shrinks to near zero, the dual-prompt cross-fusion design is not what carries the result.","tokens_in":13727,"feed_emoji":"🔀","tokens_out":5429,"duration_ms":59791,"temperature":0.7,"pith_summary":"This paper proposes pFedDC, a personalized federated learning method for pretrained vision-language models in which each client maintains four sets of tunable prompts—global and local text prompts plus global and local vision prompts—and uses client-owned cross-attention modules to fuse the global and local versions before feeding them into the frozen CLIP encoders. The claim is that this dual-prompt design handles data heterogeneity better than existing federated prompt-learning methods, which tune text prompts alone or share prompts without personalization. The paper backs the claim with experiments on nine datasets covering label shift, domain shift, and both at once, where pFedDC reports the highest average accuracy in nearly every configuration. If true, the method offers a communication-light way to personalize foundation models under privacy constraints when clients have different labels and different visual domains.","feed_headline":"Dual-prompt cross-fusion beats prior federated prompt learning","feed_subtitle":"Global and local text and vision prompts, fused by cross-attention, lift accuracy under label and domain shift.","key_machinery":"The load-bearing object is the four-prompt tuple plus two per-client cross-attention modules. The prompts are initialized from the aggregated global state each round: $P = [P^t_g, P^t_l, P^v_g, P^v_l]$, with global text and vision prompts sent to the server and locally retained local prompts. The cross-attention modules compute keys, queries, and values from the global and local prompt pairs, producing fused text and vision prompts that are fed into the frozen CLIP encoders; they are updated locally and never communicated. This design lets the same shared global prompt coexist with a client-specific interpretation of it, addressing label shift through the text side and domain shift through the vision side.","core_discovery":"On its own terms, pFedDC's central discovery is that the failure mode of federated prompt learning under simultaneous label and domain shift is the mismatch between globally shared text prompts and client-specific visual features, and that the fix is to personalize both modalities. Each client optimizes a global text prompt, a local text prompt, a global vision prompt, and a local vision prompt; only the global prompts are sent to the server for FedAvg aggregation, while the local prompts and the cross-attention modules stay on the client. A text cross-attention module fuses global and local text prompts, and a vision cross-attention module fuses global and local vision prompts, so each client's encoder input is a personalized blend of federation-wide consensus and client-specific semantics. Across the reported settings the fused dual-prompt model outperforms the prior single-modality or non-personalized baselines, and the ablation shows that either attention module alone helps while both together give the best accuracy.","pith_inferences":["The pseudo-label regeneration described in Section 4.1 is not part of the method section's formulation; if it behaves like self-training, a natural test is to apply the same regeneration to all baselines or remove it from pFedDC to isolate the dual-prompt contribution.","Because the method is agnostic to the server aggregator, the same global/local dual-prompt structure could be combined with weighted or robust aggregation rules, which might matter when clients are unreliable.","The cross-attention fusion is prompt-specific but not architecture-specific; the design could transfer to other promptable encoders beyond CLIP, for example text-only or audio models, or to tasks beyond image classification."],"forward_implications":["If the reported accuracy holds, federated clients can personalize a frozen vision-language model by exchanging only prompt vectors, keeping communication cost small while still capturing client-specific semantics.","The vision-prompt branch gives a direct handle on domain shift: clients in different visual domains no longer rely on a single frozen image-encoder representation.","Personalized cross-attention modules add no communication overhead because they never leave the client, so the design scales to larger client populations without extra upload cost.","The ablation results indicate that each cross-attention module alone improves accuracy and that both together give the best result, making the fusion step the active ingredient in the reported gains."],"supporting_citations":[{"why":"Supplies the frozen CLIP dual-branch backbone whose text and vision encoders receive the learned prompts.","marker":"[1]"},{"why":"FedAvg is the aggregation rule used to combine global text and vision prompts across clients.","marker":"[5]"},{"why":"FedOTP is the main text-prompt-only personalized baseline that pFedDC extends and compares against.","marker":"[4]"},{"why":"pFedPrompt is the personalized prompt-learning baseline that personalizes visual features while sharing text prompts.","marker":"[16]"},{"why":"PromptFL is a federated prompt-learning baseline that directly applies FedAvg to prompt parameters.","marker":"[2]"},{"why":"Promptprox combines PromptFL with a proximal term and serves as a comparison for the label-shift and domain-shift settings.","marker":"[3]"},{"why":"Supplies the heterogeneous federated prompt-learning benchmark protocol and dataset selection used in the experiments.","marker":"[9]"}],"fun_headline_variants":["Personalize both vision and text prompts in federated learning","Cross-fusing global and local prompts personalizes FL","Dual-prompt cross-attention outperforms in federated settings","Fusing text and vision prompts tames client heterogeneity","Global and local prompt fusion beats prior federated methods"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume the pseudo-label regeneration step described in Section 4.1 either is applied equally to all baselines or has no material effect on accuracy, since the method section itself never mentions it.","fun_headline_variants_meta":{"raw":{"variants":["Personalize both vision and text prompts in federated learning","Cross-fusing global and local prompts personalizes FL","Dual-prompt cross-attention outperforms in federated settings","Fusing text and vision prompts tames client heterogeneity","Global and local prompt fusion beats prior federated methods"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1580,"prompt_tokens":905,"completion_tokens":675,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":595}},"tokens_in":521,"tokens_out":675,"duration_ms":7857,"temperature":1.0,"reasoning_tokens":595,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:32:07.378421+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same nine-dataset comparison with the pseudo-label regeneration step either removed from pFedDC or added to every baseline; if the accuracy gap shrinks to near zero, the dual-prompt cross-fusion design is not what carries the result.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen CLIP dual-branch backbone whose text and vision encoders receive the learned prompts."},{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"FedAvg is the aggregation rule used to combine global text and vision prompts across clients."},{"cited_title":"Global and local prompts coop- eration via optimal transport for federated learning,","cited_arxiv_id":null,"evidence_quote":"FedOTP is the main text-prompt-only personalized baseline that pFedDC extends and compares against."},{"cited_title":"Pfedprompt: Learning personalized prompt for vision-language models in federated learning,","cited_arxiv_id":null,"evidence_quote":"pFedPrompt is the personalized prompt-learning baseline that personalizes visual features while sharing text prompts."},{"cited_title":"Promptfl: Let feder- ated participants cooperatively learn prompts instead of models-federated learning in age of foundation model,","cited_arxiv_id":null,"evidence_quote":"PromptFL is a federated prompt-learning baseline that directly applies FedAvg to prompt parameters."},{"cited_title":"Federated optimization in heterogeneous networks,","cited_arxiv_id":null,"evidence_quote":"Promptprox combines PromptFL with a proximal term and serves as a comparison for the label-shift and domain-shift settings."},{"cited_title":"Harmonizing generalization and personalization in federated prompt learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the heterogeneous federated prompt-learning benchmark protocol and dataset selection used in the experiments."}],"review_version":1}