{"id":"bb7413a5-d97d-454a-9bd5-84e0cefc6393","arxiv_id":"2508.19621","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Instance-wise Bayesian prompt tuning with an implicit posterior gives consistent ~1% average accuracy gains over federated prompt baselines on DomainNet and CIFAR-100.","lead":"This paper describes pFedBayesPT, a federated learning method that creates a separate visual prompt for each image by treating prompts as random variables and learning their distribution with semi-implicit Bayesian inference. The method beat several prompt-based federated baselines by about 1% average accuracy on DomainNet and CIFAR-100, but the prior is unspecified and no error bars are reported.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Prior p(p|x) in Eq. 14 is never defined; without it, the training objective J in Eq. 26 is not fully specified and the reported results cannot be reproduced or independently verified.","rationale":"The reader's weakest-assumption analysis identifies exactly the issue I find most load-bearing: the prior p(p|x) in Eq. 14 (and used implicitly in Eq. 24) is never defined. The paper's central claim is an empirical one of consistent superiority, and verifying that claim requires a fully specified training objective. Without the prior, the loss is indeterminate. This concern is more fundamental than, say, the missing FedIns baseline or the absence of error bars, because even if those were addressed, the method still could not be reproduced without knowing the prior. I therefore agree with the reader's conditional stance: the paper should be accepted only after the prior is specified and the experiments are re-verified. My recommendation is UNCHANGED relative to the reader's CONDITIONAL verdict, because the concern is substantial but does not, by itself, disprove the central claim; it makes the current evidence insufficiently reproducible.","tokens_in":17034,"tokens_out":12616,"duration_ms":132534,"concrete_test":"Obtain the authors' code or an explicit statement of p(p|x). Then re-run the two key experimental settings, DomainNet m=6 and CIFAR-100 s=50, using (i) p(p|x) = N(0,I) and (ii) the authors' intended prior if it differs from standard normal. Compare the resulting accuracies to Table 1 and Table 2, focusing on whether the gap to SGPT (about 1%) is preserved. If the accuracy changes by more than that margin, the central claim is sensitive to the unspecified prior; if results are unchanged, the concern is a documentation gap rather than a substantive flaw.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that pFedBayesPT consistently outperforms strong baselines under feature and label heterogeneity. That claim rests on the exact training objective being well-defined. In Eq. 14, the evidence lower bound is expanded as the expected log-likelihood minus E_psi KL(q(p|psi) || p(p|x)). The prior p(p|x) appears again in the numerator of the final importance-weighted objective in Eq. 24, because p(y,p_j|x) = p(y|p_j,x) p(p_j|x). Nowhere in the paper is p(p|x) defined. If the authors intend a standard normal prior, the notation p(p|x) is misleading and the conditional dependence on x is unexplained; if they intend a data-dependent prior, its functional form is undisclosed. This is not a cosmetic issue: the KL term is explicitly credited with mitigating overfitting, and the exact form of p(p|x) changes the gradient of the entire objective. Consequently, the described algorithm is not fully determined by the text, and the claimed ~1% consistent gains over SGPT cannot be reproduced or checked. This is the most load-bearing concern because it blocks independent verification of the empirical contribution, even though the experimental tables may be accurate.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes pFedBayesPT, an instance-wise personalized federated learning framework based on visual prompt tuning. Prompts are treated as latent variables with a semi-implicit variational posterior: a prompt distribution q(p|ψ) is mixed over stochastic encoder outputs ψ, which are produced by randomly masking image features. The authors derive a variational lower bound under the SIVI framework, add a regularization term to avoid posterior collapse, and use an importance-weighted objective as the training loss. They evaluate on DomainNet (feature shift) and CIFAR-100 (label shift) against several pFL baselines, reporting consistent improvements of roughly 1% over the strongest baseline, SGPT, as well as experiments on unseen clients, the number of inference-time prompt samples, and ablations comparing deterministic, Gaussian, and implicit posterior variants.","tokens_in":17382,"tokens_out":8614,"duration_ms":101956,"significance":"If the training objective were fully specified, the paper would be a useful contribution to personalized federated learning: it targets intra-client heterogeneity at the instance level, keeps communication costs low by tuning prompts only, and imports SIVI to obtain an expressive posterior without introducing many new parameters. The experimental design is broad and the ablations are informative; the pseudo-code and hyperparameter details are also helpful. The principal weakness is that the prior p(p|x) appearing in the central variational objective is never defined, so the exact algorithm is not determined by the text. This blocks independent verification of the reported results, which is the key obstacle to accepting the empirical claims.","major_comments":[{"comment":"The training objective is not fully specified. Eq. (14) defines the KL regularizer as KL(q(p|ψ) || p(p|x)), and Eq. (24) computes importance weights using p(y,p_j|x), which factorizes as p(y|p_j,x) p(p_j|x). Nowhere in Sec. 4 or Sec. 5.1.3 is the functional form of p(p|x) given. If the prior is meant to be a standard normal independent of x, the conditioning on x is unexplained and the Bayesian interpretation is unclear; if it is data-dependent, its exact form and parameters are undisclosed. Because this term enters every gradient update of J in Eq. (26), the algorithm cannot be reconstructed or reproduced. Please specify p(p|x), justify the conditioning on x, and update the derivation and implementation description accordingly.","section":"Eq. (14), Eq. (24), Sec. 4.1"},{"comment":"The central empirical claim is a consistent ~1% advantage over SGPT, but only the mean over three random seeds is reported. No standard deviations, confidence intervals, or per-seed results are given. With a margin this small, the reader cannot assess whether the advantage is statistically reliable or within run-to-run variability. Please report variance information for the main tables, or explain why it is not applicable.","section":"Tables 1-3, Sec. 5.2"},{"comment":"The transition from L_S (Eq. 14 with A_S, Eq. 22) to the final importance-weighted objective L_S^J (Eq. 24) is not derived. In particular, the denominator Ω_j mixes q(p_j|ψ_j) with S additional samples q(p_j|ψ̃_s), and it is not obvious that the resulting expression is a valid lower bound for general J. Since the implementation fixes S=J=1, this may reduce to L+A_S, but the text should make that connection explicit and state the conditions under which Eq. (24) is a valid surrogate ELBO.","section":"Eq. (24), Sec. 4.3"}],"minor_comments":[{"comment":"Typo: 'whihc' should be 'which' in the description of the Worst Local metric.","section":"Sec. 5.1.3"},{"comment":"The symbol p is used both for the per-layer prompt and, after concatenation, for [p̄, p]. The text says 'For notational simplicity...' but this switch should be flagged more prominently to avoid confusion with the prior p(p|x).","section":"Sec. 4.2"},{"comment":"The derivation of Eq. (24) should include a sentence connecting A_S to the denominator Ω_j, since the current text jumps from Eq. (22) to the final objective.","section":"Sec. 4.3"},{"comment":"The effect of the number of prompt samples V is shown without error bars; adding them would make the plateau behavior more convincing.","section":"Fig. 1"}],"recommendation":"major_revision","confidential_remarks":"The missing prior p(p|x) is the key issue. It is possible that the authors use a standard normal prior in their implementation but failed to state it; in that case the fix is local and experiments need not be rerun. However, because the paper does not disclose the prior, I cannot rule out that the reported numbers correspond to a different and unspecified objective. I would ask the authors to state the prior explicitly, ideally with a short derivation of the resulting KL term, and to release code to confirm the objective."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what's actually here: pFedBayesPT is the first attempt, as far as I know, to treat instance-wise visual prompts as a semi-implicit Bayesian posterior in federated learning. The SIVI-based derivation in Section 4.1 is standard and the algebra checks out; the idea of generating per-instance prompts from a masked-feature encoder and adding a global prompt is clean. Empirically the paper does a decent job: two benchmarks, feature and label heterogeneity, worst-local accuracy, unseen-client generalization, and ablations against a Gaussian-posterior variant and a deterministic variant. The gains over SGPT are consistent, around 1%, which is small but believable.\n\nThe soft spots are real. The biggest one is the one you flagged: p(p|x) in Eq. 14 and Eq. 24 is never defined. The text says the KL term 'constrains the prompt distribution to remain close to the prior distribution p(p|x)' but no functional form is given anywhere. That means Eq. 26, the actual training objective, is not fully specified by the paper. You can't reproduce the algorithm without guessing whether the prior is a standard normal, a learned conditional, or something else. That's load-bearing, because the gradient of the KL term and the importance weights in Eq. 24 both depend on it. I don't think it's fatal to the central empirical claim—if the prior is a standard normal, the method is a reasonable VAE-style regularization and the results could well hold—but as written, the paper does not determine its own algorithm.\n\nSecond, there are no error bars or significance tests despite three seeds, and the margins over SGPT are ~1%. The consistency across settings is encouraging, but we can't rule out noise from the table alone. Third, the closest instance-wise baseline, FedIns, is discussed in the related work but not included in any experiment. Given that the paper's whole pitch is instance-wise personalization, that's a head-scratcher and should be addressed. Fourth, the hyperparameters, especially the encoder learning rate and prompt insertion depth, are tuned on the benchmark outcomes; that's common in this literature, but it does temper the 'consistently outperforms' claim.\n\nWho is this for? People working in federated prompt tuning or personalized FL. It's a useful combination and the negative result on the Gaussian posterior in the ablation actually strengthens the SIVI story. It deserves a proper peer review—an editor should send it out rather than desk-reject—but a referee should ask the authors to define the prior, add error bars, and run FedIns before the claims are fully supported.","headline":"A plausible instance-wise Bayesian prompt-tuning method for pFL, but the training objective is underspecified as written because the prior p(p|x) in Eq. 14 is never defined.","tokens_in":17836,"tokens_out":2531,"would_cite":true,"duration_ms":25634,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By generating a personalized Bayesian prompt for every input image, pFedBayesPT claims to consistently beat existing personalized federated learning methods on both feature-shift and label-shift benchmarks.","keywords":["federated learning","personalized federated learning","instance-wise personalization","visual prompt tuning","semi-implicit variational inference","Bayesian deep learning","data heterogeneity"],"falsifier":"Retrain pFedBayesPT with the KL term in Eq. 24 replaced by a fixed L2 penalty on prompt magnitudes (or removed entirely) on DomainNet (m=6) and CIFAR-100 (s=50). If accuracy does not drop materially, the Bayesian regularizer is not carrying the reported gains. Also compare pFedBayesPT against pFedBayesPT-G using identical encoder capacity and mask sampling: if the gap vanishes, the benefit attributed to the implicit posterior could be due to stochastic feature masking alone.","tokens_in":16973,"feed_emoji":"🖼️","tokens_out":7616,"duration_ms":79586,"temperature":0.7,"pith_summary":"Most personalized federated learning methods give each client one model, but a client often holds images from several domains, so a single model cannot fit them all. pFedBayesPT tries to fix this by personalizing at the level of individual images: a frozen vision transformer is paired with a small encoder that reads the image's features and outputs the parameters of a prompt distribution, and a prompt sampled from that distribution is inserted into the transformer for that image alone. The prompts are treated as Bayesian latent variables with an implicit posterior, trained through a semi-implicit variational inference objective. The paper reports that this instance-wise scheme consistently outperforms existing personalized federated learning baselines on DomainNet (feature shift) and CIFAR-100 (label shift), including on clients never seen in training. If the claim holds, fine-grained instance adaptation can be added to federated learning with only prompts, a classifier head, and a small encoder being communicated.","feed_headline":"Per-image Bayesian prompts win in personalized federated learning","feed_subtitle":"A prompt tuned to each input handles multi-domain clients and unseen clients in one frozen-ViT framework.","key_machinery":"The central object is the semi-implicit variational prompt posterior: p ~ q(p|ψ) with ψ ~ q_φ(ψ|x), so the marginal over prompts is an implicit distribution rather than a Gaussian. The randomness in ψ comes from Bernoulli-masked image features passed through layer-wise MLP encoders; the prompt is sampled with the reparameterization trick. This hierarchical setup is what lets a single model produce diverse instance-specific prompts without extra variational parameters. The training objective is the SIVI lower bound of Eq. 24, with the KL term regularizing the prompt distribution and the A_S term preventing the mixing distribution from collapsing to a point mass.","core_discovery":"pFedBayesPT's central claim is that personalization in federated learning should happen per instance, not per client, and that Bayesian visual prompt tuning is a workable way to do it. For each input, the model extracts layer-wise token features from a frozen ViT, applies random binary masks to those features, and runs the masked features through per-layer MLPs that output the mean and variance of a Gaussian prompt distribution. Sampling a prompt via reparameterization and concatenating it with a shared global prompt gives the prompt used by the transformer. The paper derives a semi-implicit variational lower bound—Eq. 24—whose first term is the classification log-likelihood and whose second","pith_inferences":["The unspecified prior p(p|x) leaves room for the method to actually be a conditional VAE-style regularizer; varying or removing the KL term would reveal how much of the gain is Bayesian regularization versus the stochastic prompt generator itself.","Because the prompt encoder is purely input-conditioned and the backbone is frozen, the same mechanism could be lifted out of federated learning and used as a test-time adaptation module for a pretrained vision transformer on a new domain; the paper does not explore this.","Bernoulli feature masking is a form of stochastic regularization; a head-to-head against ordinary dropout with a Gaussian posterior would isolate whether the implicit posterior or simply feature noise drives the improvement.","The roughly 1% margin over SGPT is measured on two benchmarks; whether it survives stricter privacy constraints, larger client populations, or heterogeneous device compute budgets is left open."],"forward_implications":["Instance-wise prompt generation lets a client whose data spans several domains be served by one model, without storing multiple client-level models.","Because prompts are generated from input features, the trained prompt encoder can personalize for clients that never participated in federated training.","The frozen backbone plus prompt/head/encoder exchange keeps communication and trainable parameters close to lightweight prompt-tuning baselines.","Sampling several prompts at inference gives a tunable accuracy/compute trade-off, with more samples helping until diminishing returns.","The ablation results indicate that both Bayesian uncertainty and the implicit (non-Gaussian) posterior contribute to the gains over deterministic prompt tuning."],"supporting_citations":[{"why":"Supplies the semi-implicit variational inference framework, including the degeneracy-avoiding A_S regularizer used in Eq. 22.","marker":"[49]"},{"why":"Defines visual prompt tuning and the VPT-Deep prompt injection scheme the method builds on.","marker":"[22]"},{"why":"Provides the pFedPG baseline, the domain-partitioned DomainNet setup, and the frozen-ViT prompt-generation paradigm.","marker":"[47]"},{"why":"Provides the SGPT baseline, the strongest comparator, and the shared-plus-selective prompt design.","marker":"[9]"},{"why":"Motivates instance-wise personalization by intra-client heterogeneity and supplies the FedIns baseline context.","marker":"[14]"},{"why":"Defines the ViT architecture and the layer-wise feature representations used to generate prompts.","marker":"[11]"},{"why":"Supplies the FedAvg aggregation protocol for synchronizing prompts, head, and encoder.","marker":"[34]"},{"why":"Gives the reparameterization trick used to sample prompts from the Gaussian conditional.","marker":"[24]"},{"why":"Supplies the importance-weighting technique used to tighten the variational lower bound in Eq. 24.","marker":"[4]"}],"fun_headline_variants":["Instance-wise Bayesian prompts beat client-level personalization","Per-instance prompts solve multi-domain federated learning","Bayesian visual prompts for fine-grained personalized FL","Instance-wise pFL via semi-implicit Bayesian prompts","Frozen ViT prompts personalize per input in FL"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The training objective contains a term that compares the prompt distribution with a prior distribution over prompts that the paper never defines, so the regularization meant to prevent overfitting—and to justify the Bayesian interpretation—is underspecified.","fun_headline_variants_meta":{"raw":{"variants":["Instance-wise Bayesian prompts beat client-level personalization","Per-instance prompts solve multi-domain federated learning","Bayesian visual prompts for fine-grained personalized FL","Instance-wise pFL via semi-implicit Bayesian prompts","Frozen ViT prompts personalize per input in FL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00146,"raw_usage":{"total_tokens":5690,"prompt_tokens":704,"completion_tokens":4986,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":448,"completion_tokens_details":{"reasoning_tokens":4912}},"tokens_in":448,"tokens_out":4986,"duration_ms":37998,"temperature":1.0,"reasoning_tokens":4912,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:37:40.025889+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain pFedBayesPT with the KL term in Eq. 24 replaced by a fixed L2 penalty on prompt magnitudes (or removed entirely) on DomainNet (m=6) and CIFAR-100 (s=50). If accuracy does not drop materially, the Bayesian regularizer is not carrying the reported gains. Also compare pFedBayesPT against pFedBayesPT-G using identical encoder capacity and mask sampling: if the gap vanishes, the benefit attributed to the implicit posterior could be due to stochastic feature masking alone.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the semi-implicit variational inference framework, including the degeneracy-avoiding A_S regularizer used in Eq. 22."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pFedPG baseline, the domain-partitioned DomainNet setup, and the frozen-ViT prompt-generation paradigm."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SGPT baseline, the strongest comparator, and the shared-plus-selective prompt design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates instance-wise personalization by intra-client heterogeneity and supplies the FedIns baseline context."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the ViT architecture and the layer-wise feature representations used to generate prompts."},{"cited_title":"Kingma and Max Welling","cited_arxiv_id":null,"evidence_quote":"Gives the reparameterization trick used to sample prompts from the Gaussian conditional."}],"review_version":1}