{"id":"64719f8f-e958-4827-aed5-5add93b8cc4a","arxiv_id":"2501.01653","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"pFedSeq trains a server-side Mamba sequence model on clients' historical adapter updates to generate personalized adapter calibrations, improving personalized federated adapter tuning.","lead":"A new federated learning method called pFedSeq uses a Mamba sequence model at the server to process each client's past adapter updates and output personalized adapter calibrations for fine-tuning a shared foundation model. In experiments across four image benchmarks, it reports higher accuracy than several existing personalized federated learning methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The strongest claim is under-supported because the main comparison omits the most relevant adapter-specific PFL baselines (FedDPA, pFedLoRA, FedL2P, pFedPG) discussed in the paper; the small margins in Table 1 could be overturned by one omitted method.","rationale":"I read the paper in good faith: it gives a coherent algorithm, an architecture justification, ablations for each design component, architecture variants, efficiency measurements, and 3-seed results. Those are real evidence. The reader's formal weakest assumption—that Delta_i^t proxies the gradient of the local loss—is a standard hypernetwork approximation and is mild in the reported setting, which uses one local SGD epoch with batch size 32; thus I do not elevate it to the decisive concern. Instead, the decisive gap is the baseline set. A claim of superiority over 'state-of-the-art PFL methods' cannot be settled when the most closely related adapter-specific methods are omitted from the experiments. Since the paper nonetheless provides substantial evidence and no internal contradiction, the reader's CONDITIONAL verdict is the right one; I would keep it, with the condition that the missing baselines be evaluated or the claim be scoped to the ten selected methods.","tokens_in":18630,"tokens_out":18677,"duration_ms":191876,"concrete_test":"Using the paper's exact protocol (frozen ViT-B/16, LoRA, Dir(0.1) label skew for CIFAR-100 and Tiny-ImageNet, domain-as-client for DomainNet, writer-as-client for Omniglot, 80 rounds, 3 seeds, same validation-based hyperparameter grid), add FedDPA, pFedLoRA, FedL2P, and pFedPG to Table 1 under the same federated-adapter-tuning adaptation used for the other baselines. The concern lands if any added method's mean accuracy exceeds pFedSeq's on any dataset, or if the margin over the strongest included baseline falls below the pooled standard error; otherwise the headline claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: pFedSeq 'outperforms state-of-the-art PFL methods' on four benchmarks, as asserted in the Abstract and Conclusion and backed by Table 1. For that claim to be meaningful, the comparison set must include the strongest methods designed for the same federated-adapter-tuning setting. The paper discusses FedDPA, pFedLoRA, FedL2P, and pFedPG in Related Work, but none is in the main comparison; pFedPG appears only in Appendix Table 5 under its original prompt-based setup. Because all other baselines are adapted to a frozen ViT-B/16 with LoRA, the omitted adapter-specific methods are exactly the ones that could falsify the headline. The reported margins over the second-best method are only 0.84, 0.60, 2.21, and 2.27 percentage points, and the best included baselines are pFedHN and PeFLL rather than the most recent adapter-specific PFL work. The ablation study shows that global aggregation and architecture choice also contribute to pFedSeq's score, so the gains specifically attributable to 'looking back' are even smaller than the headline margin. This is a missing-support problem for the central claim, not an internal inconsistency.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes pFedSeq, a personalized federated adapter-tuning method in which the server maintains a sequential learner (implemented as a two-layer Mamba Selective SSM) that ingests clients' past adapter updates and outputs per-client calibrations, which are added to a Federated-Averaging-style global adapter. The server trains the sequential learner using the received adapter updates as proxies for local gradients. Experiments are reported on CIFAR-100, Tiny-ImageNet, DomainNet, and Omniglot under label-skew, feature-skew, and real-world heterogeneity, with ten PFL baselines adapted to a frozen ViT-B/16 plus LoRA setup. The paper also reports ablations of the global-aggregation, cross-step, and cross-client components, an ablation of the learner architecture (MLP, LSTM, Selective SSM), sensitivity to sequence length L and warm-up length W, and a per-round runtime comparison.","tokens_in":18899,"tokens_out":6394,"duration_ms":65838,"significance":"If the empirical claims hold, pFedSeq is a practical and communication-efficient contribution to personalized federated adapter tuning: it keeps the hypernetwork size independent of the adapter dimension, leverages a modern sequence model for cross-client and cross-step relations, and reports the strongest accuracy on all four benchmarks in its comparison. The paper also contains useful ablations and a computational-efficiency table. However, the headline empirical claim is weakened by the absence of the most directly relevant adapter-specific PFL baselines from the controlled comparison, and the component ablations show that global aggregation contributes substantially to the reported gains. These issues, rather than any internal mathematical inconsistency, are the main barriers to acceptance.","major_comments":[{"comment":"The main comparison omits the four adapter-specific PFL methods that the paper itself identifies in Related Work: FedDPA, pFedLoRA, FedL2P, and pFedPG. These are precisely the methods designed for the same federated-adapter-tuning setting, and all other baselines are adapted to the same frozen ViT-B/16 plus LoRA protocol. pFedPG appears only in Appendix Table 5 under its original prompt-tuning setup, which does not allow an apples-to-apples comparison. Given that the margins over the best included baselines are only 0.84, 0.60, 2.21, and 2.27 percentage points on CIFAR-100, Tiny-ImageNet, DomainNet, and Omniglot, the abstract's claim that pFedSeq 'outperforms state-of-the-art PFL methods' is under-supported. The authors should add these methods to the controlled comparison, or explicitly scope the claim to the methods evaluated and discuss the likely relationship to the omitted work.","section":"Related Work; Experiments, Baseline Comparison (Table 1); Appendix Table 5"},{"comment":"The ablation shows that the contribution specifically attributable to historical sequence modeling is modest on the label-skew datasets: removing cross-step modeling (variant B) costs 0.64 percentage points on CIFAR-100 and 1.53 on DomainNet, whereas removing global aggregation (variant A) costs 0.89 and 1.96 points. On CIFAR-100, the cross-step contribution is smaller than the 0.84-point margin over the second-best baseline pFedHN. The paper's central narrative—that 'looking back' at previous updates yields the improvement—would be substantially strengthened by reporting the component ablation on Tiny-ImageNet and Omniglot, and by including significance tests or confidence intervals; as presented, the label-skew results do not demonstrate that historical updates are the dominant driver of the method's advantage.","section":"Analysis of pFedSeq, Table 2"}],"minor_comments":[{"comment":"The notation in Eq. (5) writes ξ^{t-1}_i = (SeqLearner(∆^{1:t-1}; ψ))_{:,i} for all t, but for t > L the implementation described in Eq. (6) feeds only the last L updates; the formula should use the capped sequence (e.g., ∆^{t-L:t-1}) to be consistent.","section":"Method, Eq. (5) and Eq. (6)"},{"comment":"The claim of improvement 'up to 5.39%' is not directly traceable to the numbers in Table 1; the largest absolute gain over Local is 6.89 points on Omniglot, and the largest gain over the second-best method is 2.27 points. Please state how the 5.39% figure is computed.","section":"Abstract; Introduction; Experiments, Table 1"},{"comment":"The caption states that both the MLP-based and LSTM-based variants are shown with an 'orange line'; the two curves are presumably different colors, so the caption should be corrected.","section":"Appendix C, Figure 9b"},{"comment":"The paper reports 3 seeds and no significance testing; given that several headline margins are below one percentage point, a paired test across seeds or a repeated-measures analysis would make the comparisons more convincing.","section":"Experiments, Implementation Details"},{"comment":"No code is released; given the number of tuned baselines and the use of a relatively recent Mamba implementation, releasing code would materially improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The omission of FedDPA, pFedLoRA, FedL2P, and pFedPG from the controlled comparison will be a visible point of attack for any expert reviewer, since the paper itself cites these as related adapter-specific PFL methods. I would ask for these comparisons (or a well-justified exclusion) before considering the paper further. The manuscript is otherwise coherent and the empirical workload is substantial for the included methods."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague—\n\nHere's the short version: pFedSeq is a genuinely new mechanism in the hypernetwork-PFL line—the server trains a Mamba/SSM sequence model on clients' past adapter updates and uses it to generate additive calibrations to the global adapter. The paper is worth engaging, but the headline claim that it 'outperforms state-of-the-art PFL methods' is not fully supported by the comparison set. The margins over the second-best methods are small (0.6–2.3 pp), and the four closest adapter-specific competitors named in Related Work—FedDPA, pFedLoRA, FedL2P, pFedPG—are absent from the main table. pFedPG appears only in the appendix under its original prompt-tuning setup, which is not a like-for-like comparison. If any one of those omitted methods lands within the margin when run in the same frozen-ViT/LoRA setup, the central empirical claim weakens substantially. This is a missing-support issue, not an internal inconsistency.\n\nWhat is good: the modeling idea is clean. Using a selective SSM to capture both cross-client and cross-step relations in update trajectories is a reasonable, well-motivated choice, and the ablations back it up. The authors include LSTM and MLP learner comparisons, a sequence-length sweep, and a warm-up analysis, and they show the computational cost is low. The experiments span four datasets and three heterogeneity scenarios. The gradient-proxy assumption (treating the adapter update as a gradient at the previous adapter) is inherited from earlier hypernetwork work; it is a heuristic when clients run multiple epochs or use momentum, and the paper does not discuss how sensitive the method is to that. That is a moderate concern, not a fatal one.\n\nThe ablation table is the most informative part. Removing global aggregation costs 0.89–1.96 pp, and removing cross-step modeling costs 0.64–1.53 pp. So the 'look back' component contributes, but it is not the whole story. The sequence-length sweep shows a 1.17 pp gain from L=1 to L=15 on Omniglot, which is a fair direct test of the mechanism.\n\nBottom line: this is a solid, publishable-after-major-revision paper. The novelty is real, the evaluation is above average in breadth, and the critical missing piece is a fair baseline set and released code. If the authors add the omitted adapter-specific methods (FedDPA, pFedLoRA, FedL2P, and pFedPG in the LoRA setup) and make code available, the claims become much more convincing. I would send it to a serious referee—it deserves expert time—but I would flag the baseline gap and the gradient-proxy sensitivity as must-address points.\n\nMy advice: engage with it, ask for the baselines and code, and see whether the margins survive.","headline":"Genuinely new mechanism for personalized federated adapter tuning, but the SOTA claim is under-supported by missing adapter-specific baselines and small margins.","tokens_in":19437,"tokens_out":4042,"would_cite":true,"duration_ms":32738,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"pFedSeq improves personalized federated adapter tuning by having the server learn from clients' past adapter updates, reporting the top accuracy on four benchmarks.","keywords":["personalized federated learning","adapter tuning","LoRA","sequential learner","selective state space model","Mamba","hypernetwork","data heterogeneity"],"falsifier":"Run pFedSeq with clients performing several local epochs or using momentum before communicating, holding all else fixed; if its advantage over the latest-update-only ablation ($L=1$) and over PeFLL shrinks or disappears, the gradient-proxy assumption is the component carrying the result.","tokens_in":18443,"feed_emoji":"🎯","tokens_out":9604,"duration_ms":82460,"temperature":0.7,"pith_summary":"pFedSeq addresses a blind spot in personalized federated learning: existing methods form a client's personalized model from only the latest round of updates and discard the trajectory that led there. The paper proposes a server-side sequential learner, a hypernetwork built on Selective SSM (Mamba), that consumes the sequence of all clients' past adapter updates and emits per-client calibrations added to the globally aggregated adapter. The learner is trained at the server using the received updates as gradient proxies, so no local data leaves the clients. On CIFAR-100, Tiny-ImageNet, DomainNet, and Omniglot, pFedSeq reports the highest accuracy among the twelve compared methods, with the largest margins on the domain-shifted and real-world datasets. The intended upshot is that the updates already sent to the server are a cheap and informative resource for building more stable personalized adapters.","feed_headline":"Past adapter updates, not just the latest, drive personalized FL gains","feed_subtitle":"A server-side sequence model turns clients' accumulated adapter updates into per-client calibrations","key_machinery":"The load-bearing object is the sequential learner: a hypernetwork instantiated as a two-layer Selective SSM (Mamba) block, with the adapter's parameter dimension treated as the batch dimension so that the learner's size depends on the number of clients and the sequence length, not on adapter size. At each step the input is the stacked matrix of all clients' adapter updates for that round; input-dependent parameters capture cross-client interactions, while the recurrent hidden state carries cross-step dependencies. The training rule is the proxy-gradient update, where the server uses each received adapter update as a stand-in for the gradient of that client's local loss, giving $\\Delta\\psi = \\sum_i (\\nabla_\\psi \\xi_i^{t-1})^\\top \\Delta_i^t$.","core_discovery":"The paper's central claim is that personalized federated adapter tuning improves when the server models the recent history of adapter updates rather than only the latest round. The discovery is a concrete mechanism: a server-side sequential learner parameterized by Selective SSM maps the sequence of stacked client updates to per-client calibrations, and each personalized adapter is the global FedAvg adapter plus that client's calibration. The learner is optimized by a proxy-gradient rule that replaces each client's true local-loss gradient with the received adapter update, so training happens entirely at the server without local data. Empirically, the paper reports that pFedSeq reaches 95.30% on CIFAR-100, 94.30% on Tiny-ImageNet, 84.63% on DomainNet, and 45.25% on Omniglot, the highest numbers in Table 1, with the largest gains when clients have distinct feature distributions.","pith_inferences":["Beyond the paper, the same server-side sequential learner should transfer to other parameter-efficient modules such as prompts or adapters beyond LoRA, since the machinery operates on the update sequence rather than on adapter-specific structure.","The proxy-gradient premise implies the gains are most secure when clients run one local epoch with plain SGD; multi-epoch training or momentum would make the update a less faithful gradient estimate and could shrink the gap over latest-update-only baselines.","The reported learning curves suggest a positive feedback loop between better personalized adapters and better local updates, which implies the method may deliver its largest speedups in communication-limited regimes with few rounds."],"forward_implications":["If pFedSeq's claim is correct, personalized federated tuning can be improved without any extra client communication, because the server reuses adapter updates it already receives.","Modeling a longer history helps up to a point: performance rises with sequence length $L$ and then plateaus, giving a tunable trade-off between memory and accuracy.","The sequential learner's size is independent of adapter dimension, scaling with client count and sequence length, which keeps server-side cost modest for large foundation models.","Ablations attribute the gain to all three components — global aggregation, cross-step modeling, and cross-client modeling — with removal of global aggregation hurting most.","The architecture choice matters: replacing Selective SSM with an MLP or LSTM lowers accuracy, so the input-dependent recurrence is doing real work."],"supporting_citations":[{"why":"supplies the FedAvg aggregation rule used to form the global adapter","marker":"McMahan et al. 2017"},{"why":"supplies the Selective SSM (Mamba) architecture used as the sequential learner","marker":"Gu and Dao 2023"},{"why":"supplies the hypernetwork-based PFL approach and the proxy-gradient trick for server-side training","marker":"Shamsian et al. 2021"},{"why":"supplies the LoRA adapters that the framework personalizes","marker":"Hu et al. 2021"},{"why":"supplies the frozen ViT-B/16 backbone used across all experiments","marker":"Dosovitskiy et al. 2020"},{"why":"supplies the practice of keeping the classification head local, which pFedSeq adopts","marker":"Collins et al. 2021"},{"why":"supplies the federated personalization setup and Dirichlet label-skew partitions used in the experiments","marker":"Yang, Wang, and Wang 2023"},{"why":"provides the PeFLL hypernetwork baseline compared in the experiments","marker":"Scott, Zakerinia, and Lampert 2024"}],"fun_headline_variants":["History-aware server modeling boosts personalized federated adapters","pFedSeq: personalizing adapters via client update sequences","Selective SSM turns past adapter updates into personalized clients","Modeling update history beats latest-round finetuning in federated learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each client's adapter update faithfully represents the gradient of that client's local loss, because the server uses those updates to train the sequential learner without ever seeing local data.","fun_headline_variants_meta":{"raw":{"variants":["History-aware server modeling boosts personalized federated adapters","pFedSeq: personalizing adapters via client update sequences","Selective SSM turns past adapter updates into personalized clients","Modeling update history beats latest-round finetuning in federated learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000456,"raw_usage":{"total_tokens":2263,"prompt_tokens":896,"completion_tokens":1367,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":1297}},"tokens_in":512,"tokens_out":1367,"duration_ms":9877,"temperature":1.0,"reasoning_tokens":1297,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:22:29.974116+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run pFedSeq with clients performing several local epochs or using momentum before communicating, holding all else fixed; if its advantage over the latest-update-only ablation ($L=1$) and over PeFLL shrinks or disappears, the gradient-proxy assumption is the component carrying the result.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the hypernetwork-based PFL approach and the proxy-gradient trick for server-side training"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the practice of keeping the classification head local, which pFedSeq adopts"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the federated personalization setup and Dirichlet label-skew partitions used in the experiments"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the PeFLL hypernetwork baseline compared in the experiments"}],"review_version":1}