{"id":"a8a01dfe-6525-4202-afad-9470c61b7211","arxiv_id":"2508.04470","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A personalized federated learning method replaces gradient updates with closed-form ridge regression, achieving a data-partition invariance property and large reported gains.","lead":"FedHiP trains personalized federated learning models using closed-form linear algebra instead of gradient updates, with a frozen foundation model for feature extraction. Its main claim is that personalized models are invariant to how non-IID data is split across other clients, reporting large accuracy gains over gradient-based baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Heterogeneity-invariance theorem only covers permutations of a fixed data pool; the abstract's claim of invariance 'regardless of how non-IID' is broader than what is proven.","rationale":"The reader's weakest_assumption correctly identifies that Theorem 3 assumes the alternative configuration is a permutation of the original data pool. My analysis agrees with this and further notes that the abstract's phrasing extends the claim beyond this assumption. Theorems 1 and 2 are arithmetically sound; the closed-form solutions are correctly derived. The only significant issue is that the heterogeneity-invariance property is conditional on a fixed overall data pool, and this condition is not stated in the paper's advertised claims or in the limitations section. This does not invalidate the mathematical results but does require a clarification of the scope. The reader's conditional acceptance already accounts for this and other concerns (experimental rigor, AFL positioning), so my stress-test does not change the verdict. I recommend UNCHANGED, meaning the reader's CONDITIONAL verdict remains appropriate, and the authors should be asked to explicitly state the permutation/fixed-pool assumption in the abstract and elsewhere and to temper the 'regardless' language accordingly.","tokens_in":939,"tokens_out":1476,"duration_ms":105821,"concrete_test":"Construct two federated configurations with the same target client k and the same frozen backbone, alpha, beta, but with other clients' datasets drawn as disjoint samples from different distributions (e.g., different class subsets or different domains) rather than as permutations of a fixed pool. Train FedHiP in both configurations and compare the resulting personalized models P_k (or their predictions on client k's test set). If the models differ, the invariance claim fails for arbitrary non-IID distributions. For a simpler analytical check, choose a non-permutation row transformation of F_{1:K} (e.g., replace one client's rows with new samples) and verify that F'^T F' != F^T F, directly contradicting the unqualified statement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central novelty is the heterogeneity-invariance property, stated in the abstract as 'each personalized model remains identical regardless of how non-IID the data are distributed across all other clients' and formalized in Theorem 3. The proof, however, explicitly assumes (Eq. 31) that the alternative configuration D'_{1:K} is a row permutation of the original D_{1:K} with the same overall data pool. Under this assumption, the identities F'^T F' = F^T F and F'^T Y' = F^T Y follow from the orthogonality of the permutation matrix, and the invariance is correct. But the informal claim does not include this fixed-pool condition. In standard non-IID federated settings, clients may have entirely different samples drawn from different distributions, so the stacked feature matrix F'_{1:K} is not a row permutation of F_{1:K}; then F'^T F' != F^T F in general, and the personalized model P_k changes. The paper does not flag this limitation: Section VI lists only the frozen-backbone and linear-classifier limitations, omitting the fixed-pool assumption. The experiments use Dirichlet partitions of a fixed dataset, so they validate only the narrow permutation-invariance, not the broader claim. Since heterogeneity invariance is the headline contribution, this gap between the theorem and the advertised property is load-bearing for the paper's central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedHiP, a personalized federated learning method that replaces gradient-based updates with closed-form ridge-regression solutions on features extracted by a frozen pretrained backbone. The method has three phases: analytic local training on each client, recursive analytic global aggregation on the server, and a final personalized analytic model for each client. The main theoretical contributions are Theorem 1 and Theorem 2, which show that the server-side global model and the client-side personalized model exactly solve global and personalized regularized least-squares objectives, respectively. Theorem 3 claims a 'heterogeneity-invariance' property, stating that each personalized model is identical regardless of how non-IID the other clients' data are. Experiments on CIFAR-100 and ImageNet-R under Dirichlet non-IID partitions report accuracy improvements over several PFL baselines.","tokens_in":17574,"tokens_out":5883,"duration_ms":67389,"significance":"If the claims hold, the paper makes a useful contribution to PFL: the closed-form formulation removes iterative gradient updates, reduces communication to a single round, and yields exact, transparent optimization guarantees. Theorems 1 and 2 are clean and verify the equivalence of the recursive aggregation to the stated objectives, and they are straightforward to check. The heterogeneity-invariance property is interesting but, as proven, is substantially narrower than the advertised claim, and the experimental section needs stronger statistical reporting. The paper also cites a prior analytic-FL work [26] while claiming to be the first to introduce analytic learning into PFL, which requires clarification.","major_comments":[{"comment":"The advertised heterogeneity-invariance property is broader than what is proven. The abstract and contribution 3 claim that each personalized model remains identical 'regardless of how non-IID the data are distributed across all other clients.' However, the proof of Theorem 3 explicitly assumes in Eq. (31) that D'_{1:K} is a permutation of the original D_{1:K} with the same overall data pool. Under that assumption, F'^T F' = F^T F and F'^T Y' = F^T Y follow from the permutation matrix's orthogonality, and the theorem is correct. But in standard non-IID FL settings, other clients can have entirely different samples drawn from different distributions; then D' is not a row permutation of D, and F'^T F' != F^T F in general, so the personalized model P_k changes. Section VI's discussion of limitations does not mention this fixed-pool condition. The experiments in Section V-A use Dirichlet par","section":"Abstract and Section IV-E, Theorem 3"},{"comment":"The empirical superiority claim is not supported with statistical detail. Table I reports single accuracy values without standard deviations, number of seeds, or significance tests, and Figures 4-5 appear to show single runs. The claim of outperforming baselines by 5.79%-20.97% would be more credible if the results were averaged over multiple independent runs with reported variance, and if the hyperparameter-tuning protocol for the baselines was described. Because FedHiP requires only one round while baselines use 200 rounds, the comparison should also state whether the reported baseline results are the best achievable under the chosen tuning budget.","section":"Section V-B, Table I and Figures 4-5"},{"comment":"The novelty claim 'to the best of our knowledge, we are the first to introduce analytic learning into PFL' is contradicted by the paper's own reference list. Reference [26], 'AFL: A single-round analytic approach for federated learning with pre-trained models,' is cited in Section II-B among analytic-learning applications, yet the text immediately says there is still a significant gap in introducing analytic learning into PFL. If [26] is indeed an analytic FL method, the paper must explicitly compare with it and correct the novelty claim; if it is not, the citation should be moved or clarified. This is load-bearing for the positioning of the contribution.","section":"Section II-B and Section VI.A"}],"minor_comments":[{"comment":"The derivation contains a typo: the second line 'S_k M_k = S_k S_k^{-1} F^T_{1:K}F_{1:K} = F^T_{1:K}F_{1:K}' should refer to F^T_{1:K}Y_{1:K}, not F^T_{1:K}F_{1:K}. The subsequent conclusion uses the correct quantity.","section":"Theorem 1 proof, Eq. (26)"},{"comment":"The matrix P in 'P^T P' is undefined; it should be the permutation matrix A introduced in Eq. (31).","section":"Theorem 3 proof, Eq. (33)"},{"comment":"Typo: 'Nevertherless' should be 'Nevertheless.'","section":"Section VI.B"},{"comment":"The phrase 'the target client k’s dataset D1:k' appears to be a typo for D_k. Also, reference [26] contains 'federated learing' in the title.","section":"Section II.B"},{"comment":"It is unclear how the reported 'improvement' is computed when the best baseline is sometimes not the second-best row in Table I. Please state the rule explicitly, e.g., improvement over the best-performing baseline in each column.","section":"Section V.B"}],"recommendation":"major_revision","confidential_remarks":"The central algebraic derivations are correct, but the heterogeneity-invariance claim needs to be substantially rescoped, and the relationship to reference [26] must be resolved before the paper can be accepted. Please ask the authors to explicitly acknowledge the fixed-pool permutation assumption in Theorem 3 and in the abstract, and to add multi-seed experimental results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is worth reading for the clean closed-form treatment, but the headline 'heterogeneity invariance' needs a qualifier—it holds for any repartitioning of the same global data pool, not for changes in what the other clients actually have. If you read the abstract as promising the latter, you'll be disappointed; if you read it as the former, the theorem is correct and the experiments support it.\n\nWhat's new: FedHiP uses local ridge-regression sufficient statistics (Gram matrices) to train a global classifier and then personalize per client, all without gradients. Theorems 1 and 2 correctly show the closed-form solutions match the stated objectives. The recursive aggregation on the server is a neat way to reconstruct the pooled-data solution from per-client statistics. The invariance property (Theorem 3) is a genuine algebraic observation: since the personalized model only depends on the client's own Gram matrix and the total pooled Gram matrices, any reassignment of samples among other clients with the same union leaves it unchanged. That's a useful property for FL with pre-trained features, and the Dirichlet experiments directly test that scenario.\n\nSoft spots, in order of importance:\n\n1. The abstract overstates the invariance. 'Regardless of how non-IID the data are distributed across all other clients' sounds like it covers other clients having entirely different data. The proof assumes the same overall data pool. That's the natural reading of 'distributed,' but it should be stated explicitly in the abstract and intro. The stress-test concern is fair as a wording issue, but I don't think it's a load-bearing flaw—the property they prove is the one they use in the experiments.\n\n2. The experimental reporting is thin. No error bars, no seeds, no details on hyperparameter tuning for baselines beyond a single sentence. On ImageNet-R, baselines are below 10% accuracy, which suggests they may be poorly tuned; the claimed 20%+ improvements over a nearly zero baseline aren't very convincing. At minimum, they need multiple runs and a table of baseline hyperparameters.\n\n3. Positioning against AFL [26] (same group, CVPR 2025) is missing. AFL already introduced single-round analytic FL with pre-trained models. FedHiP's addition is the personalization phase and the invariance claim, but the paper doesn't clearly delineate this. An honest comparison of the two methods would help.\n\n4. The method restricts to a frozen backbone and linear classifier. The paper acknowledges this, but it means the practical scope is narrow: it's a head-tuning method for foundation model features, not a general PFL algorithm.\n\nBottom line: the math is sound, the invariance is a real and interesting property (with the fixed-pool caveat), and the efficiency claim—one round, no gradients—is attractive. For a reader doing FL with pre-trained models, this is a useful reference. It deserves peer review, but the authors need to fix the overstatement, add statistical rigor, and contextualize against AFL. I'd suggest major revision.","headline":"Clean closed-form PFL with a real invariance property, but the abstract oversells it and the experiments need more rigor.","tokens_in":17936,"tokens_out":6234,"would_cite":false,"duration_ms":56120,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedHiP claims that replacing gradient-based updates with closed-form least-squares solutions makes each client's personalized model independent of how non-IID the other clients' data are.","keywords":["personalized federated learning","non-IID data","closed-form solution","analytic learning","ridge regression","heterogeneity invariance","frozen backbone","gradient-free training"],"falsifier":"Take a fixed client k's dataset and two configurations of the other clients: (i) the same pooled samples with rows permuted among clients, and (ii) a different pool with partially or wholly different samples. Compute $\\hat P_k$ using formulas (13)–(14) in both cases. The theorem predicts equality for (i); if the two matrices differ for (i), the theorem is false. For (ii), if $\\hat P_k$ differs, it does not falsify the theorem as stated, but it would bound the practical meaning of 'regardless of how non-IID.'","tokens_in":17133,"feed_emoji":"⚙️","tokens_out":7248,"duration_ms":71621,"temperature":0.7,"pith_summary":"The paper asserts that the persistent failure of personalized federated learning under data heterogeneity is caused by gradient-based updates: local gradients point in conflicting directions and pull the global model away from a generalizable solution. To remove this cause rather than patch its symptoms, FedHiP performs all training with closed-form least-squares solutions on features extracted by a frozen self-supervised backbone. The scheme has three analytic phases—local ridge regression, a recursive global aggregation that provably equals pooled-data ridge regression, and a local personalization step that provably solves a regularized objective mixing global and local terms. The central theoretical result is heterogeneity invariance: for a fixed client, the personalized model is unchanged no matter how the other clients' data are redistributed non-IID. This matters because it promises stable personalized accuracy without iterative client-server rounds or convergence tuning.","feed_headline":"Closed-form updates make federated models immune to data skew","feed_subtitle":"Frozen features plus ridge regression: each client's personalized model ignores how the other clients split their data","key_machinery":"The mechanism is the regularized Gram matrix $C_k=F_k^\\top F_k+\\beta I$ and its cumulative sum $S_K=\\sum_{k=1}^K C_k$, combined with a recursively updated Knowledge Fusion Matrix $M_k$. The recursion uses the telescoping identity $\\prod_{i=j+1}^K S_i^{-1}S_{i-1}=S_K^{-1}S_j$ to show that $M_K=(F_{1:K}^\\top F_{1:K}+K\\beta I)^{-1}F_{1:K}^\\top Y_{1:K}$. This identity lets distributed local models be aggregated into exactly the pooled-data least-squares solution, and the orthogonality of permutation matrices makes $F^\\top F$ and $F^\\top Y$ invariant under reordering of other clients' samples.","core_discovery":"FedHiP's core claim is that the personalized model of each client, computed by the formula $\\hat P_k=(S_K+\\tilde C_k-K\\beta I)^{-1}(S_K M_K+\\alpha F_k^\\top Y_k)$, is exactly the minimizer of the combined objective $\\|Y_{1:K}-F_{1:K}P_k\\|^2+\\alpha\\|Y_k-F_kP_k\\|^2+\\beta\\|P_k\\|^2$, and that the global model $\\hat G_K$ formed recursively at the server is exactly the pooled-data ridge solution $(F_{1:K}^\\top F_{1:K}+\\beta I)^{-1}F_{1:K}^\\top Y_{1:K}$. Because the personalized solution depends only on the four matrices $F_k^\\top F_k$, $F_k^\\top Y_k$, $F_{1:K}^\\top F_{1:K}$, and $F_{1:K}^\\top Y_{1:K}$, and these are invariant under row permutations of the pooled feature and label matrices, the pape","pith_inferences":["The invariance theorem as stated covers only reorderings, i.e., permutations, of the same pooled dataset; if other clients contribute entirely new samples, the claim that the personalized model is invariant is not proved and would need a different argument.","The same three-phase scheme should work with any frozen feature extractor, not only ViT-MAE, since the proof treats the feature matrices generically; testing backbone choice would be a cheap way to probe the invariance empirically.","Because the personalized model depends only on second-order feature moments, adding kernel or nonlinear feature maps should preserve the invariance while improving the linear classifier's capacity.","The privacy argument that raw data cannot be recovered from $C_k$ and $\\hat L_k$ relies on infinitely many semi-orthogonal factorizations; this rules out exact reconstruction but does not by itself establish formal privacy guarantees against inference attacks."],"forward_implications":["Only one round of client-server communication is needed: each client uploads $C_k$ and $\\hat L_k$, then downloads $S_K$ and $M_K$.","No convergence analysis or gradient steps are required, so non-IID data cannot cause client drift or conflicting gradient directions.","Performance is stable as the number of clients and the heterogeneity level change; the paper reports accuracy gains of 5.79–20.97% over gradient-based baselines.","Because the classifier is a single linear analytic layer, final accuracy depends heavily on the quality of the frozen backbone's features, a limitation the paper acknowledges.","Computational and communication overhead are sharply reduced because iterative back-propagation and multi-round aggregation are replaced by a few matrix products and inverses."],"supporting_citations":[{"why":"Supplies the frozen ViT-MAE foundation backbone used for gradient-free feature extraction, from which every FedHiP matrix is computed.","marker":"[28]"},{"why":"Provides the blockwise recursive Moore-Penrose inverse technique that motivates the recursive global aggregation and its closed-form knowledge fusion.","marker":"[24]"},{"why":"Prior analytic class-incremental learning with closed-form solutions and privacy protection; FedHiP positions itself as transferring this analytic approach to federated learning.","marker":"[16]"},{"why":"FedAvg is the canonical gradient-based federated baseline whose pooled-data objective the global aggregation is shown to reproduce exactly.","marker":"[2]"},{"why":"Ditto supplies the meta-learning personalization baseline used in the comparative accuracy and efficiency experiments.","marker":"[5]"},{"why":"FedALA supplies the model-splitting personalized baseline used for comparison in the experiments.","marker":"[6]"},{"why":"Provides the CIFAR-100 benchmark dataset on which the main accuracy, efficiency, and sensitivity comparisons are run.","marker":"[34]"},{"why":"Provides the ImageNet-R benchmark dataset used to show that gradient-based baselines struggle while FedHiP remains accurate.","marker":"[35]"}],"fun_headline_variants":["Closed-Form Federated Learning: Immune to Data Skew","No Gradients, No Skew: FedHiP's Analytic Personalization","Federated Models That Ignore Data Imbalance","Ridge Regression Beats Gradients for Personalized Federated Learning","Analytic Fed: Personalization Without Gradient Headaches"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The heterogeneity-invariance proof assumes that the alternative non-IID configurations are permutations of the same overall data pool with the same samples, so if other clients' datasets can contain entirely different samples, the claim that the personalized model is invariant is not established.","fun_headline_variants_meta":{"raw":{"variants":["Closed-Form Federated Learning: Immune to Data Skew","No Gradients, No Skew: FedHiP's Analytic Personalization","Federated Models That Ignore Data Imbalance","Ridge Regression Beats Gradients for Personalized Federated Learning","Analytic Fed: Personalization Without Gradient Headaches"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000497,"raw_usage":{"total_tokens":2340,"prompt_tokens":880,"completion_tokens":1460,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":1375}},"tokens_in":624,"tokens_out":1460,"duration_ms":12869,"temperature":1.0,"reasoning_tokens":1375,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:56:59.305229+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed client k's dataset and two configurations of the other clients: (i) the same pooled samples with rows permuted among clients, and (ii) a different pool with partially or wholly different samples. Compute $\\hat P_k$ using formulas (13)–(14) in both cases. The theorem predicts equality for (i); if the two matrices differ for (i), the theorem is false. For (ii), if $\\hat P_k$ differs, it does not falsify the theorem as stated, but it would bound the practical meaning of 'regardless of how non-IID.'","supporting_citations":[{"cited_title":"Masked au- toencoders are scalable vision learners,","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen ViT-MAE foundation backbone used for gradient-free feature extraction, from which every FedHiP matrix is computed."},{"cited_title":"Blockwise recursive moore–penrose inverse for network learning,","cited_arxiv_id":null,"evidence_quote":"Provides the blockwise recursive Moore-Penrose inverse technique that motivates the recursive global aggregation and its closed-form knowledge fusion."},{"cited_title":"ACIL: Analytic class-incremental learning with absolute memorization and privacy protection,","cited_arxiv_id":null,"evidence_quote":"Prior analytic class-incremental learning with closed-form solutions and privacy protection; FedHiP positions itself as transferring this analytic approach to federated learning."},{"cited_title":"Ditto: Fair and robust federated learning through personalization,","cited_arxiv_id":null,"evidence_quote":"Ditto supplies the meta-learning personalization baseline used in the comparative accuracy and efficiency experiments."},{"cited_title":"FedALA: Adaptive local aggregation for personalized federated learn- ing,","cited_arxiv_id":null,"evidence_quote":"FedALA supplies the model-splitting personalized baseline used for comparison in the experiments."},{"cited_title":"Learning multiple layers of features from tiny images,","cited_arxiv_id":null,"evidence_quote":"Provides the CIFAR-100 benchmark dataset on which the main accuracy, efficiency, and sensitivity comparisons are run."}],"review_version":1}