{"id":"ac2d42d7-a9ec-4f2b-8787-90f981fd9a9e","arxiv_id":"2412.01295","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FedAH improves personalized federated learning by element-wise mixing each client's local head with the global head before local training, and it reports better accuracy than ten federated baselines on five benchmarks.","lead":"FedAH mixes each client's private classification head with the globally averaged head before every local training round, using per-element weights learned on the client's own data. The authors report higher test accuracy than ten existing federated learning methods on five datasets, including in settings where clients drop out.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed benefit of element-level head aggregation is not isolated: FedAH differs from FedRep by head sharing, extra communication, and learned weights simultaneously, with no ablation separating these, and the promised theoretical derivation is absent.","rationale":"The paper's experimental evidence is solid and repeatable: open code, standard benchmarks, multiple heterogeneity settings, and consistent ranking. The abstract's 2.87% figure is an overstatement because it refers to the Cifar100 practical setting margin over the second-best method, not an average across all benchmarks, and the MNIST practical setting shows FedAH slightly below FedALA. These are presentation issues, not correctness failures. The most load-bearing concern is that the method's unique component—element-level head aggregation with learned weights—is confounded with head sharing and additional communication. The reader's weakest_assumption focused on whether the global head is a meaningful carrier of transferable knowledge; our concern is closely related but more actionable: even if the global head is meaningful, the paper does not demonstrate that the learned weights add value over simply using the global head, nor does it provide the promised theoretical analysis of when the global head is reliable. We therefore agree with the conditional verdict: the empirical claim is acceptable but should be reframed, and an ablation isolating the aggregation mechanism is necessary before the central explanatory claim can be accepted. Our concrete test directly settles this attribution question.","tokens_in":13670,"tokens_out":6989,"duration_ms":65334,"concrete_test":"Run FedAH and three controlled variants on Cifar100 (β=0.1) and on a fully label-disjoint pathological split (e.g., 20 clients, each seeing only one distinct class): (a) W≡1 — replace the local head with the global head before local training; (b) W≡0 — FedRep baseline with head sharing disabled; (c) full FedAH with learned W; (d) FedAH with learned W but skipping the Eq. 5 head fine-tuning step. If (a) matches (c) within standard deviation, the learned element-level weights are not responsible for the gain; if (c) degrades sharply under fully disjoint labels, the noise-robustness claim requires explicit qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—FedAH outperforms ten state-of-the-art FL methods—is plausible and backed by extensive tables, but the paper's key explanatory claim, that element-level aggregation of local and global heads (Eq. 3) with learned weights W (Eq. 4) is what drives the gains, is not actually tested. FedAH differs from FedRep in at least three coupled ways: (i) the head is shared and globally aggregated, (ii) the server transmits the full model (2*Σ parameters) instead of only the feature extractor (2*α*Σ), and (iii) the local initialization is an adaptive interpolation between local and global heads. Without ablations that fix W=1 (pure global head replacement) and W=0 (FedRep), the observed improvement could be attributable to head sharing or to the extra communication/parameters rather than to the learned aggregation weights. Section III explicitly promises 'a theoretical derivation of FedAH' but none appears, so the claimed robustness of W in filtering noisy global-head entries is unverified. This is a load-bearing gap because the paper's stated novelty is the aggregation mechanism, not the overall pipeline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes FedAH, a personalized federated learning method that combines the head-only personalization of FedRep with an element-wise interpolation between each client's previous head and the server-aggregated global head. The interpolation weights W_i^t are learned by one gradient step per round under a [0,1] clipping constraint (Eqs. 3-4). The server aggregates both feature extractor and head; clients fine-tune the aggregated head and then the shared extractor (Eqs. 5-6). Experiments on MNIST, Cifar10, Cifar100, Tiny-ImageNet, and AG News compare FedAH with ten baselines under pathological and Dirichlet-heterogeneous settings, plus scalability, dropout, and local-epoch ablations. FedAH reports the best average test accuracy in most settings, with the largest margin on Cifar100 (2.87% over FedALA), but is slightly below FedALA on MNIST under the practical setting.","tokens_in":13914,"tokens_out":8317,"duration_ms":68775,"significance":"If the mechanism is as claimed, FedAH would be a practically useful, low-complexity extension of FedRep and FedALA, and the paper provides extensive, careful evaluation with standard deviations and open code. The contribution is, however, incremental relative to FedALA, and the central explanatory claim—that the learned element-wise head aggregation (Eq. 3) is responsible for the gains—is not isolated by ablations. The paper also promises but does not deliver a theoretical derivation. The experimental evidence is strong enough to justify a major revision rather than rejection.","major_comments":[{"comment":"Section III, first paragraph, states that the authors will 'finally perform a theoretical derivation of FedAH,' but no theoretical derivation appears anywhere in the section or the rest of the paper. The method description in Section III-C is algorithmic only. The paper should either supply a convergence or noise-robustness argument for the learned aggregation weights W (e.g., under heterogeneous label distributions), or explicitly retract the promise. This is not merely a wording issue: Section IV-F attributes FedAH's dropout stability to the learned W without any supporting analysis.","section":"Section III"},{"comment":"The central attribution claim is untested. FedAH differs from FedRep by (i) sharing and aggregating the head, (ii) transmitting the full model rather than only the feature extractor, and (iii) the learned interpolation W in Eq. (3). The experiments compare against FedRep, FedPer, FedALA, etc., but never fix W=1 or W=0 within FedAH. Without these ablations (e.g., on Cifar100 and TINY*), the observed improvement cannot be attributed specifically to the learned element-level aggregation weights. Add results for FedAH with W identically 1, with W identically 0, and (optionally) with a random fixed W.","section":"Section III-C, Eqs. (3)-(4)"},{"comment":"Algorithm 1 returns {r_i, \\hat h_i} for each client, but \\hat h_i is the aggregated head before local training (Eq. 3, step 7), while the trained head h_i^t is produced in step 8. This makes the output specification inconsistent with the evaluation, which uses the average test accuracy of the best local models. Clarify what the final personalized model is and correct the pseudocode accordingly.","section":"Algorithm 1, line 12"},{"comment":"Equation (4) is circular as written: the loss is evaluated at \\hat h_i^t, which by Eq. (3) depends on W_i^t, the same quantity being updated. Specify whether \\hat h_i^t in Eq. (4) is computed using W_i^{t-1} or the current W_i^t, and state the exact number of gradient steps used to train W_i^t per round. This is necessary for reproducibility of the method.","section":"Equation (4)"},{"comment":"The abstract and conclusion overstate the results. Table I shows FedAH is not the best on MNIST under the practical heterogeneous setting (99.66 vs. FedALA's 99.71), and the 2.87% margin is specific to Cifar100, not a general average across all tasks. The claim that FedAH 'outperforms ten SOTA FL methods in terms of test accuracy by 2.87%' should be qualified to match the reported results.","section":"Abstract and Table I"}],"minor_comments":[{"comment":"The notation for learning rates is inconsistent: Algorithm 1 lists 'weight learning rate α' while Eq. (4) uses η for the weight update. Section IV-B says the aggregation-weight learning rate is set to the local learning rate, but the symbols should be made consistent in the algorithm and equations.","section":"Algorithm 1 and Eq. (4)"},{"comment":"The text introduces the notation 'TINY' and 'TINY*' but Table I's caption does not define them; the caption should state that TINY uses a 4-layer CNN and TINY* uses ResNet-18.","section":"Table I caption"},{"comment":"The phrase 'we can theoretically compare the communication overhead' is vague; the comparison is a parameter-count calculation, not a theoretical analysis, and the wording should be adjusted.","section":"Section IV-E"},{"comment":"The statement that all tasks are run for 2000 iterations until all methods empirically converge conflicts with Table III's early-stopping iteration counts (e.g., FedAH 40 iterations, FedAvg 165); clarify the stopping rule and how the 2000-iteration budget is used.","section":"Section IV-A"},{"comment":"The sentence 'the learning rate of the aggregation weights in FedAH is set to be the same as the local learning rate' should give the actual numerical value rather than only a relationship, since Table I does not report hyperparameters for the methods.","section":"Section IV-B"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly positioned between FedALA and FedRep, and the near-identical aggregation formula (Eqs. 3-4 vs. FedALA) should be explicitly inventoried so that the novelty claim is precise. The missing promised theory is a red flag for reviewers; if the authors cannot provide such an analysis, they should remove the sentence in Section III rather than leave a dangling promise. The empirical work is otherwise solid enough to warrant a revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: FedAH is a modest but honest empirical increment for personalized-head FL. The core idea is a learned element-wise interpolation between each client's previous head and the current global head, followed by FedRep-style alternating training. That's a legitimate narrow combination of FedALA-style aggregation and FedRep's head/feature split. The experiments are the real value: five datasets, ten baselines, pathological and Dirichlet heterogeneity, dropout, scalability, five runs with std, and code released. I believe the main empirical claim—FedAH generally beats FedRep and FedALA on these benchmarks—is credible.\n\nThree soft spots, in decreasing order. First, Section III promises a theoretical derivation that never appears. There is no convergence or noise-robustness argument, and the stress-test note is right that the claimed benefit of W filtering noisy global heads is unverified. That broken promise should be fixed by either delivering a real argument or explicitly removing the claim. Second, the novelty framing overshoots. Eq. (3)-(4) are essentially FedALA's element-level aggregation restricted to the head. That is a small extension, not a new mechanism, and the 'first to consider' claim in the contributions is not backed by any serious prior-art search. Third, the abstract's 2.87% number is misleading: it's the Cifar100 practical-setting margin over FedALA, not an average or worst-case claim, and FedAH actually ties or slightly loses to FedALA on MNIST practical. That should be qualified.\n\nThe stress-test note wants ablations with W fixed to 1 and to 0. That's a fair ask, but I wouldn't call it a load-bearing flaw. FedAH differs from FedRep by head sharing and extra communication, so the improvement over FedRep alone doesn't isolate the learned weights. Still, the method as a whole is what's proposed; without ablations the attribution is weak, but the pipeline's empirical value stands. Also, the communication overhead comparison is honest: FedAH ships the full model per round versus FedRep's feature-only, but converges in fewer rounds, and they say so.\n\nWho this is for: researchers working on personalized-head FL who want a simple, well-tested baseline. It deserves a serious referee, but the paper needs revision: add ablations, deliver or delete the theory promise, qualify the abstract, and fix the attribution to FedALA.","headline":"Honest incremental empirical paper on personalized-head FL; solid experiments but the theory promise is empty, the novelty claim is oversold, and the abstract needs qualification.","tokens_in":14426,"tokens_out":2487,"would_cite":false,"duration_ms":22756,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedAH claims that blending each client's previous local head with the server's current global head, using per-element learned weights, improves personalized federated learning accuracy across vision and text benchmarks.","keywords":["Personalized federated learning","Statistical heterogeneity","Aggregated head","Element-wise aggregation","Feature extractor","Model head","Test accuracy","Federated learning"],"falsifier":"Run FedAH and FedRep on a synthetic federated task with a known linear classifier and fully disjoint label partitions across clients; if the global head's parameters for a client's unseen classes are random, the learned aggregation weights should drive those entries of $W_i^t$ to zero, and the accuracy gain over FedRep should vanish.","tokens_in":43,"feed_emoji":"🧠","tokens_out":5978,"duration_ms":110666,"temperature":0.7,"pith_summary":"The paper proposes FedAH, a personalized federated learning method that modifies how each client's classification head is initialized. Instead of keeping the locally trained head from the previous round or ignoring the head split entirely, FedAH computes an Aggregated Head as an element-wise blend of the client's previous local head and the server's current global head, with per-element weights learned by gradient descent. The claim is that this one change lets the personalized head retain global knowledge while still fitting local data, and the experiments report that it beats ten existing federated learning methods on five image and text benchmarks, with the largest gain being 2.87% over the second-best method on Cifar100. If true, it means a client-specific head does not have to be trained in isolation to achieve personalization; a small learned interpolation step can capture both global and local information.","feed_headline":"Blending local and global heads lifts personalized FL accuracy","feed_subtitle":"A per-client learned blend of the local and global classifier head beats ten federated baselines on vision and text tasks.","key_machinery":"The central object is the Aggregated Head, defined by the element-wise interpolation formula $\\hat{h}_i^t = h_i^{t-1} + (h^t - h_i^{t-1}) \\odot W_i^t$, with $W_i^t \\in [0,1]$ learned via a gradient step per round (Eq. 4). This single mechanism does three jobs: it injects the global head's information into each client's personalized head, it preserves the client-specific part through the $1 - W_i^t$ weighting, and it subsumes FedAvg ($W=1$) and FedPer/FedRep ($W=0$) as extreme cases. The rest of the method—freezing the extractor while training the head, then freezing the head while training the extractor—follows the FedRep training loop, so the only new machinery is the learned blend and its gradient-based weight update.","core_discovery":"FedAH's central claim is that introducing global information at the level of the model head, not just the feature extractor, improves personalized accuracy. At each round, client $i$ computes $\\hat{h}_i^t := h_i^{t-1} + (h^t - h_i^{t-1}) \\odot W_i^t$, where $h_i^{t-1}$ is the client's head from the previous round, $h^t$ is the server-aggregated global head, and $W_i^t$ is a learned per-element weight vector clipped to $[0,1]$. The weights are trained by gradient descent with all other parameters frozen. The client then trains the blended head while freezing the shared feature extractor, then trains the feature extractor with the head frozen, following the alternating protocol of FedRep. The paper reports that this Aggregated Head initialization outperforms ten state-of-the-art federated learning baselines across MNIST, Cifar10, Cifar100, Tiny-ImageNet, and AG News, and remains ahead under stronger heterogeneity, larger client counts, and client dropout.","pith_inferences":["If the blend's benefit comes from the global head carrying transferable class-boundary information, then the gain should shrink when client label distributions are completely disjoint: there the global head's parameters for a client's unseen classes are arbitrary, and the learned $W_i^t$ entries would need to be driven to zero for those classes to avoid noise. This is a testable stress case the pa","The per-element weight learning is a form of learned interpolation; one could extend it to per-layer or per-block weights in deeper heads, or add a momentum term that smooths $W_i^t$ across rounds, with no change to the communication pattern.","Because $W_i^t$ is trained with all other parameters frozen, the method is compatible with robust or privacy-preserving aggregation on the server: the global head could be replaced by any robust aggregate, and the local weight learning would adapt.","The reported accuracy gain over FedALA is largest on Cifar100; a natural follow-up is to measure whether the head blend helps most when the label space is large relative to the feature dimension, since that is where the head carries the most class-specific information."],"forward_implications":["Personalized-head federated learning does not have to forfeit global knowledge: blending the global head into the local head before local training yields higher test accuracy than keeping heads fully local.","The learned weight vector $W_i^t$ acts as a per-client, per-parameter gate; when a client's local head is informative it can suppress global influence, and when local data are scarce it can lean on the global head.","Because the Aggregated Head is computed from the previous local head and the current global head, the method requires no extra communication beyond the standard model upload/download, and it converges in fewer iterations than baselines in the reported settings.","Under client dropout, re-joining clients re-learn their aggregation weights at the start of each round, which the experiments show keeps accuracy stable relative to fixed-ratio baselines.","The method extends to text classification (AG News) and larger models (ResNet-18 on Tiny-ImageNet), indicating the benefit is not limited to small CNN image tasks."],"supporting_citations":[{"why":"Supplies the split-model alternating training protocol that FedAH inherits (freeze extractor, train head; freeze head, train extractor).","marker":"[19]"},{"why":"Provides the idea of adaptive element-level aggregation weights and the weight clipping operation that FedAH borrows for the head.","marker":"[17]"},{"why":"Baseline personalized-head method that keeps heads local and is improved by head aggregation.","marker":"[18]"},{"why":"Base federated averaging framework and global model aggregation rule used to compute the global head.","marker":"[8]"},{"why":"Regularization-based baseline compared; shows that proximal regularization alone does not match explicit head aggregation.","marker":"[5]"}],"fun_headline_variants":["Learned head blend boosts personalized federated learning","Global head info via weighted blend lifts PFL accuracy","FedAH: element-wise head fusion beats ten FL baselines","Per-client head aggregation improves personalization","Weighted global-local head mix advances PFL"],"cache_read_input_tokens":16640,"weakest_assumption_plain":"The load-bearing premise is that the global head—the weighted average of the selected clients' locally trained heads—is a meaningful source of transferable knowledge for every client, so that blending it into each local head with learned weights improves accuracy rather than injecting noise.","fun_headline_variants_meta":{"raw":{"variants":["Learned head blend boosts personalized federated learning","Global head info via weighted blend lifts PFL accuracy","FedAH: element-wise head fusion beats ten FL baselines","Per-client head aggregation improves personalization","Weighted global-local head mix advances PFL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00079,"raw_usage":{"total_tokens":3523,"prompt_tokens":1029,"completion_tokens":2494,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":2421}},"tokens_in":645,"tokens_out":2494,"duration_ms":15575,"temperature":1.0,"reasoning_tokens":2421,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:30:26.995086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FedAH and FedRep on a synthetic federated task with a known linear classifier and fully disjoint label partitions across clients; if the global head's parameters for a client's unseen classes are random, the learned aggregation weights should drive those entries of $W_i^t$ to zero, and the accuracy gain over FedRep should vanish.","supporting_citations":[],"review_version":1}