{"id":"e4eb9fc7-f59d-427e-848a-fee05e31ce59","arxiv_id":"2607.21417","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FedSEPT uses low-rank shared expert prompts with on-device instance-aware fusion to improve the privacy–utility trade-off in federated prompt tuning.","lead":"FedSEPT is a new method for federated prompt tuning that replaces one shared prompt with several low-rank 'expert' prompts and fuses them per image, while sending only a small noisy factor to the server. If its results hold, it offers a better privacy–utility–communication trade-off for adapting vision-language models on heterogeneous edge clients.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (9)'s logit-level fusion is an unvalidated linearization of a nonlinear text encoder; IEF's claimed prompt-level composition may not occur, and Table 5 never reports prompt-level fusion accuracy.","rationale":"The reader's weakest-assumption analysis correctly identifies the logit-level fusion approximation in Eq. (9) as the most load-bearing unvalidated step. The paper's claimed mechanism — instance-aware expert composition at the prompt level — is exactly what Eq. (9) bypasses. Since CLIP's text encoder is a nonlinear transformer, there is no a priori reason that a weighted sum of per-expert text-feature logits equals the logit of the weighted prompt. The paper does not provide an error analysis, and the one prompt-level fusion variant (FedSEPT-PF) is used only for efficiency comparison in Table 5, not for accuracy. This is a genuine correctness risk for the central claim: \n\n(1) If the approximation is poor, IEF's reported gains may come from a logit ensemble rather than from the subspace-decomposed expert composition the paper advertises. The central claim's wording — 'Instance-aware Expert Fusion ... combining ... expert prompts' — would be unsupported, and the mechanism story in §1 and §4.2 would need substantial revision. \n\n(2) The concern is concrete and testable: directly compare Eq. (9) against prompt-level fusion under identical conditions. This is feasible with the authors' released code and the existing hyperparameters in §5.1. A null result would strengthen the paper; a significant gap would weaken it. \n\nOther aspects of the paper are not the primary soft spot. The DP guarantee in Theorem 4.1 is standard DP-SGD composition applied to the shared factors; the local-DP threat model is coherent. The low-rank parameterization and fixed public basis address the coordinate-matching issue for aggregation. Ablations in Table 4 support the contribution of multiple experts, private residuals, and soft routing relative to ablated variants, though those ablations do not isolate the logit-level approximation from prompt-level fusion. The broad 11-benchmark evaluation and the availability of code are real strengths. \n\nBecause the reader already issued a CONDITIONAL verdict and this concern is exactly the condition that should be settled before full acceptance, no verdict adjustment is needed. The recommendation is to keep the conditional status and require the prompt-level-vs-logit-level fusion accuracy comparison as a condition for acceptance.","tokens_in":20398,"tokens_out":4493,"duration_ms":47645,"concrete_test":"Run FedSEPT with prompt-level fusion on the same three diagnostic benchmarks (OxfordPets, CIFAR-10 β=0.1, OfficeHome β=0.1) at ε=1, replacing Eq. (9) with P_k(x) = (Σ_m π_m A_m) B0 + R_k followed by full text encoding per batch. Compare In/Cross/HM against logit-level FedSEPT with at least 3 seeds and report mean±std. If prompt-level fusion outperforms by more than the seed noise (say >2 HM points), Eq. (9) materially changes the model and the IEF mechanism is not the one claimed; if the two are statistically indistinguishable, the late-fusion approximation is validated for these regimes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires IEF to compose semantically complementary expert prompts into an input-dependent prompt. But §4.2.2 explicitly avoids composing prompts: Eq. (9) sums logits from cached per-expert text features, i.e., z = Σ_m π_m ⟨v, t_m⟩/τ = ⟨v, Σ_m π_m t_m⟩/τ. This equals the true logit for the weighted prompt P_k(x) = (Σ_m π_m A_m) B0 + R_k only if the text encoder is linear in prompt space, which is false for a transformer. No error bound or empirical validation is provided. The only comparison to prompt-level fusion (FedSEPT-PF) in Table 5 reports latency, FLOPs, and memory — not accuracy. If the linearity approximation is poor, IEF is not performing the advertised instance-aware prompt composition; it is a logit ensemble, and the mechanism behind the reported HM gains differs from the paper's claim. The 'robust expert composition' motivation and the conclusion's 'multi-expert collaboration' claim therefore rest on an untested assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FedSEPT proposes a federated prompt-tuning method for heterogeneous clients under local differential privacy. It parameterizes each of M prompt experts as A_m B0 + R_k, with a fixed public basis B0 and a client-private residual R_k; only the low-rank factors A_m are communicated and aggregated, reducing communication cost and the dimension of DP perturbation. An instance-aware router computes soft expert weights from image features; rather than constructing an input-dependent prompt, the method caches per-expert text features and fuses logits (Eq. 9). Experiments on 11 benchmarks report In-Client, Cross-Client, and harmonic-mean accuracy against six baselines, together with ablations, efficiency measurements, and membership-inference/gradient-inversion attack evaluations. The central claim is that under the same privacy budget, FedSEPT achieves a better local-adaptation/global-generalization trade-off than existing baselines.","tokens_in":20703,"tokens_out":5998,"duration_ms":65904,"significance":"If the empirical claims hold, FedSEPT is a useful contribution to privacy-preserving federated prompt tuning. The subspace-decomposed expert parameterization is a practical way to scale to multiple experts under local DP, and the reported communication reduction (64 to 8 KiB per round) is substantial. The paper ships code, uses a standard DP-SGD privacy proof, and evaluates both utility and privacy attacks, which is a strength. Notable reported gains include OxfordPets HM 90.24 vs 80.67 for DP-FPL at ε=1, and consistent HM improvements on several other datasets. However, the central mechanism of 'instance-aware expert fusion' rests on an unvalidated logit-level linearity approximation, the empirical headline differences are not accompanied by uncertainty estimates or significance tests, and the 'privacy-preserving' label is stronger than the reported MIA results support. These issues are load-bearing for the paper's main claims and require revision.","major_comments":[{"comment":"The paper motivates IEF as composing experts into an input-dependent prompt P_k(x) = (Σ_m π_m A_m) B0 + R_k, but Eq. (9) actually fuses cached per-expert logits: z_{k,c}(x) = Σ_m π_{k,m}(x) ⟨v(x), t^m_{k,c}⟩/τ. This is exactly equivalent to the logit of the weighted prompt only if the CLIP text encoder is linear in the prompt parameters, which is false for a transformer. No error bound or empirical validation of this approximation is given. Table 5 compares FedSEPT-PF (prompt-level fusion) only on latency, FLOPs, and memory, not on accuracy. As written, the method is a logit-level ensemble, not the advertised instance-wise prompt composition, and the 'multi-expert collaboration' claim in the conclusion goes beyond what is demonstrated. Please add an accuracy comparison between Eq. (9) and true prompt-level fusion (or an approximation-error measurement) on at least one representative benc","section":null},{"comment":"The main empirical results are reported as point estimates only: the text states 'mean client performance over the last 5 rounds', but no standard deviations, confidence intervals, or number of seeds are given. Several headline differences are small, e.g., Food101 HM 80.75 vs 80.67 for DP-FPL in Table 1, and DomainNet HM 67.07 vs 66.79 at β=0.1 in Table 3. Without multiple seeds and significance testing, the central claim of a better trade-off is not robustly supported. Please report mean±std over at least three seeds for the main tables, and state whether the reported gains are statistically significant for the close comparisons.","section":null},{"comment":"The update-and-query MIA reaches final AUC 0.5521–0.5790 for ε∈{0.25,0.5,0.75,1.0}, down from 0.9471 without DP but still substantially above random guessing. Local DP protects only the uploaded shared factors; it does not protect the outputs of the final personalized model, which the query-based attacker exploits. The unqualified 'privacy-preserving' phrasing in the title, abstract, and conclusion is therefore too strong. Please add an explicit statement of the threat model (local DP covers server-visible updates only), report the query-attack residual leakage as a known limitation, and qualify the privacy claim accordingly.","section":null}],"minor_comments":[{"comment":"The 'Single' ablation is described as removing both SEM and IEF, but the exact parameterization is not fully specified. State whether it is a single full-dimensional prompt of length L, and clarify how it is aggregated under DP.","section":null},{"comment":"Algorithm 1 says to compute fused predictions using Eq. (9), but the caching of per-expert text features is not explicit. Clarify the timing: per-expert text features must be recomputed after each prompt update and then cached for reuse across the minibatch.","section":null},{"comment":"The DP-SGD formulation in Eq. (4) writes the noise as added to the minibatch average. Standard DP-SGD often clips per-example gradients and adds noise to the sum before dividing by batch size; the two are equivalent up to a rescaling of the noise multiplier, but this should be stated to avoid confusion.","section":null},{"comment":"The line plots for seven methods are dense and small at the current size. Consider larger panels, markers for individual methods, or shaded error regions if multiple seeds are added.","section":null},{"comment":"The DP-FPL baseline applies QR-based re-factorization after each local epoch. Please state explicitly how this operation interacts with the privacy accounting (e.g., as a deterministic post-processing step) so that the comparison is not open to a privacy-accounting objection.","section":null}],"recommendation":"major_revision","confidential_remarks":"I agree with the stress-test concern: the Eq. (9) approximation is the weakest link in the mechanism story, and the paper should either validate it empirically or soften the claimed mechanism. The other load-bearing issue is statistical: with no seeds or error bars, the impressive-looking tables cannot be fully assessed. The paper is likely salvageable with a focused revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: FedSEPT is a credible empirical contribution. The SEM mechanism—tying multiple prompt experts to a fixed public low-rank basis and private residuals, so only compact factors are communicated and DP-perturbed—is genuinely new in this combination and delivers consistent harmonic-mean gains over strong baselines on 11 benchmarks. The IEF router plus cached logit-level fusion also seems to work. Ablations are sensible: removing the low-rank decomposition, the private residual, or the soft routing all hurt, and the efficiency numbers are real. The code is available. That is more than most papers in this area deliver.\n\nBut I think the stress-test note is right, and it matters. Eq. (9) sums per-expert logits: z = Σ π_m z_m. For a nonlinear text encoder, that is not the same as the logit of the weighted prompt P_k(x) = (Σ π_m A_m)B0 + R_k. The paper explicitly says it 'overcomes' the prompt-level computation by shifting to logit-level fusion, but it never checks how far the two diverge, either with an error bound or an accuracy comparison against FedSEPT-PF. Table 5 compares efficiency only. So the mechanism the abstract and conclusion credit—'instance-aware expert fusion' that 'composes semantically complementary experts'—is actually a logit ensemble, not a prompt composition. That may be fine for the empirical story, but the framing overstates what is happening. This needs either a validation study or a rewritten claim.\n\nThe privacy section is more careful, but the abstract still says 'privacy-preserving' without qualifying that the update-and-query MIA AUC reaches 0.579 at ε=1. The DP guarantee covers the shared factors; the final personalized model necessarily leaks something. They acknowledge it in §5.5, but the title and abstract should carry the caveat.\n\nMissing standard deviations and seeds across all tables is a real weakness—several of the HM differences in Tables 1–3 could be noise. I would not call the central claim false; the pattern of gains is consistent, and the ablations support the design. But I would want error bars before trusting the exact margins.\n\nOverall: this deserves a serious referee, not a desk reject. The SEM idea is a useful step for private federated prompt tuning, and the empirical evaluation is broad. The fusion approximation is the main conceptual soft spot; a revision that either validates the approximation or re-frames the contribution as a logit-level ensemble would be convincing. Send it to review.","headline":"Solid empirical paper on federated prompt tuning with real novelty in subspace-decomposed experts, but the logit-level fusion is an unexamined approximation and the privacy claim outruns the DP guarantee.","tokens_in":21132,"tokens_out":1732,"would_cite":true,"duration_ms":21279,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A federated prompt-tuning method keeps multi-expert prompts privacy-efficient and achieves a better personalization-generalization trade-off under local differential privacy.","keywords":["federated prompt tuning","local differential privacy","multi-expert prompts","low-rank decomposition","instance-aware routing","vision-language models","data heterogeneity","personalization-generalization trade-off"],"falsifier":"Compare the fused logits from Eq. (9) against the logits obtained by actually encoding the weighted prompt P_k(x) = (sum_m pi_k,m(x) A^m_k) B0 + R_k on a held-out set, using the same trained experts and router. If the cosine similarity between the two logit vectors drops below a high threshold on any benchmark, the late-fusion approximation is the point of failure; the paper gives no error bound for this step.","tokens_in":20355,"feed_emoji":"🔐","tokens_out":5472,"duration_ms":53665,"temperature":0.7,"pith_summary":"The paper introduces FedSEPT, a federated prompt-tuning method that keeps the benefits of multiple prompt experts while staying cheap under local differential privacy. It claims that a single shared prompt over-smooths the diverse transferable knowledge held by heterogeneous clients, so it replaces it with several expert prompts, each expressed as a low-rank factor times a fixed public basis plus a private residual. Only the small shared factors are privatized and uploaded, so DP noise and communication scale with the factor dimension rather than with the number of full experts. An on-device router then combines the experts per image, using cached text features to fuse at the logit level instead of re-encoding prompts. Under the same privacy budget, the method reports better harmonic-mean accuracy across 11 heterogeneous benchmarks than existing baselines, cutting per-round communication from 64 KiB to 8 KiB.","feed_headline":"Four low-rank prompt experts beat single prompts under local DP","feed_subtitle":"Subspace-decomposed experts lift the personalization-generalization trade-off across 11 benchmarks at the same privacy budget.","key_machinery":"Subspace-Decomposed Expert Modeling (SEM) with a fixed public basis: each of M experts is parameterized as P^m_k = A^m_k B0 + R_k, where A^m_k is the shared low-rank factor, B0 is a fixed public basis aligning all clients in a common coordinate system, and R_k is a private residual that never leaves the device. This is paired with Instance-Aware Expert Fusion (IEF), which routes each image through learned weights pi_k(x) and fuses cached per-expert text features at the logit level: z_k,c(x) = sum_m pi_k,m(x) z^m_k,c(x). The machinery confines DP noise and communication to the M L r factor space, enables direct server aggregation, and avoids repeated text-encoder passes.","core_discovery":"The central discovery is that multi-expert prompt tuning can be made privacy-efficient by decomposing each expert prompt into a shared low-rank factor A^m_k, a fixed public basis B0, and a private residual R_k. Because the basis is shared and fixed, expert factors live in one coordinate system and can be averaged directly on the server; because only factors are transmitted, DP-SGD perturbs a compact space, avoiding the noise blow-up that would come from privatizing full-dimensional expert prompts. The second piece is instance-aware expert fusion: a client-specific router predicts per-image weights, and the experts' cached class-level text features are combined by weighted logits. The paper r","pith_inferences":["Because the public basis is fixed and data-independent, the method implicitly assumes the transferable directions of every task lie in its column space; the paper does not analyze this, so a natural check is to vary the basis initialization and rank and measure performance.","The same decomposition-plus-cached-logit recipe is not tied to text prompts: it could apply to other frozen encoders and other parameter-efficient modules (e.g., adapters), wherever per-module outputs can be cached and fused per instance.","Only the shared factors are protected by the stated local-DP guarantee; the private residual and router are never uploaded. In deployments where attackers can query the final personalized model, the paper's own query-based MIA results imply these unshared states need extra protection.","The diversity regularizer and routing weights may double as a diagnostic: inspecting per-class or per-domain routing distributions could reveal which experts capture which latent semantic or style clusters, giving a data-driven view of client heterogeneity."],"forward_implications":["DP noise and communication cost no longer grow with the number of expert prompts or the full prompt dimension; they scale with the M L r factor space, making multi-expert collaboration practical on bandwidth-limited clients.","The same privacy budget buys a better personalization-generalization trade-off: harmonic-mean accuracy improves on fine-grained, label-skew, and domain-shift benchmarks, e.g., OxfordPets HM 90.24 vs 80.67 at epsilon=1.","Instance-aware logit-level fusion with cached text features keeps inference cheap: 2.35 ms latency and 35.1 GFLOPs, lower than prompt-level fusion, while retaining comparable memory during training.","Local DP suppresses both membership inference and gradient inversion: final MIA AUC drops to 0.45-0.50 under update-only attacks and to 0.55-0.58 under update-and-query attacks, and CLIP cosine similarity for gradient inversion becomes negative at epsilon=1.","A single framework covers pathological label skew, practical Dirichlet label skew, and compound domain+label skew, suggesting the expert decomposition is a general remedy for over-smoothed shared prompts."],"fun_headline_variants":["Subspace-decomposed experts lift privacy trade-off across 11 benchmarks","Multi-expert prompts with shared factors cut DP noise in federated tuning","Instance-aware fusion of prompt experts improves private federated learning","Shared basis, private residuals: a better way to tune prompts under DP"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that fusing separately encoded expert logits with per-image weights closely reproduces the logits of the single weighted prompt, so that the text encoder's output is nearly linear in the prompt mixture; if that approximation fails, the instance-aware fusion gains could disappear.","fun_headline_variants_meta":{"raw":{"variants":["Subspace-decomposed experts lift privacy trade-off across 11 benchmarks","Multi-expert prompts with shared factors cut DP noise in federated tuning","Instance-aware fusion of prompt experts improves private federated learning","Shared basis, private residuals: a better way to tune prompts under DP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1299,"prompt_tokens":779,"completion_tokens":520,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":523,"completion_tokens_details":{"reasoning_tokens":444}},"tokens_in":523,"tokens_out":520,"duration_ms":6021,"temperature":1.0,"reasoning_tokens":444,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T07:29:15.461340+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the fused logits from Eq. (9) against the logits obtained by actually encoding the weighted prompt P_k(x) = (sum_m pi_k,m(x) A^m_k) B0 + R_k on a held-out set, using the same trained experts and router. If the cosine similarity between the two logit vectors drops below a high threshold on any benchmark, the late-fusion approximation is the point of failure; the paper gives no error bound for this step.","supporting_citations":[],"review_version":1}