Pith. sign in

REVIEW 3 major objections 5 minor 40 references

FedHiP: Heterogeneity-Invariant Personalized Federated Learning Through Closed-Form Solutions

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FedHiP claims that replacing gradient-based updates with closed-form least-squares solutions makes each client's personalized model independent of how non-IID the other clients' data are.

desk verdict Clean closed-form PFL with a real invariance property, but the abstract oversells it and the experiments need more rigor. read the letter →

arxiv 2508.04470 v1 pith:FNWWH24V submitted 2025-08-06 cs.LG

Jianheng Tang , Zhirui Yang , Jingchao Wang , Kejia Fan , Jinfeng Xu , Huiping Zhuang , Anfeng Liu , Houbing Herbert Song
show 2 more authors
Leye Wang Yunhuai Liu
This is my paper · ORCID
classification cs.LG
keywords personalizedfederatedlearningnon-IIDdataclosed-formsolutionanalyticridgeregressionheterogeneityinvariancefrozenbackbonegradient-freetraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asserts that the persistent failure of personalized federated learning under data heterogeneity is caused by gradient-based updates: local gradients point in conflicting directions and pull the global model away from a generalizable solution. To remove this cause rather than patch its symptoms, FedHiP performs all training with closed-form least-squares solutions on features extracted by a frozen self-supervised backbone. The scheme has three analytic phases—local ridge regression, a recursive global aggregation that provably equals pooled-data ridge regression, and a local personalization step that provably solves a regularized objective mixing global and local terms. The central theoretical result is heterogeneity invariance: for a fixed client, the personalized model is unchanged no matter how the other clients' data are redistributed non-IID. This matters because it promises stable personalized accuracy without iterative client-server rounds or convergence tuning.

What carries the argument

The mechanism is the regularized Gram matrix $C_k=F_k^\top F_k+\beta I$ and its cumulative sum $S_K=\sum_{k=1}^K C_k$, combined with a recursively updated Knowledge Fusion Matrix $M_k$. The recursion uses the telescoping identity $\prod_{i=j+1}^K S_i^{-1}S_{i-1}=S_K^{-1}S_j$ to show that $M_K=(F_{1:K}^\top F_{1:K}+K\beta I)^{-1}F_{1:K}^\top Y_{1:K}$. This identity lets distributed local models be aggregated into exactly the pooled-data least-squares solution, and the orthogonality of permutation matrices makes $F^\top F$ and $F^\top Y$ invariant under reordering of other clients' samples.

What would settle it

Take a fixed client k's dataset and two configurations of the other clients: (i) the same pooled samples with rows permuted among clients, and (ii) a different pool with partially or wholly different samples. Compute $\hat P_k$ using formulas (13)–(14) in both cases. The theorem predicts equality for (i); if the two matrices differ for (i), the theorem is false. For (ii), if $\hat P_k$ differs, it does not falsify the theorem as stated, but it would bound the practical meaning of 'regardless of how non-IID.'

Watch

Extended reading notes

Core claim

FedHiP's core claim is that the personalized model of each client, computed by the formula $\hat P_k=(S_K+\tilde C_k-K\beta I)^{-1}(S_K M_K+\alpha F_k^\top Y_k)$, is exactly the minimizer of the combined objective $\|Y_{1:K}-F_{1:K}P_k\|^2+\alpha\|Y_k-F_kP_k\|^2+\beta\|P_k\|^2$, and that the global model $\hat G_K$ formed recursively at the server is exactly the pooled-data ridge solution $(F_{1:K}^\top F_{1:K}+\beta I)^{-1}F_{1:K}^\top Y_{1:K}$. Because the personalized solution depends only on the four matrices $F_k^\top F_k$, $F_k^\top Y_k$, $F_{1:K}^\top F_{1:K}$, and $F_{1:K}^\top Y_{1:K}$, and these are invariant under row permutations of the pooled feature and label matrices, the pape

Load-bearing premise

The heterogeneity-invariance proof assumes that the alternative non-IID configurations are permutations of the same overall data pool with the same samples, so if other clients' datasets can contain entirely different samples, the claim that the personalized model is invariant is not established.

Editorial extensions

If this is right

  • Only one round of client-server communication is needed: each client uploads $C_k$ and $\hat L_k$, then downloads $S_K$ and $M_K$.
  • No convergence analysis or gradient steps are required, so non-IID data cannot cause client drift or conflicting gradient directions.
  • Performance is stable as the number of clients and the heterogeneity level change; the paper reports accuracy gains of 5.79–20.97% over gradient-based baselines.
  • Because the classifier is a single linear analytic layer, final accuracy depends heavily on the quality of the frozen backbone's features, a limitation the paper acknowledges.
  • Computational and communication overhead are sharply reduced because iterative back-propagation and multi-round aggregation are replaced by a few matrix products and inverses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The invariance theorem as stated covers only reorderings, i.e., permutations, of the same pooled dataset; if other clients contribute entirely new samples, the claim that the personalized model is invariant is not proved and would need a different argument.
  • The same three-phase scheme should work with any frozen feature extractor, not only ViT-MAE, since the proof treats the feature matrices generically; testing backbone choice would be a cheap way to probe the invariance empirically.
  • Because the personalized model depends only on second-order feature moments, adding kernel or nonlinear feature maps should preserve the invariance while improving the linear classifier's capacity.
  • The privacy argument that raw data cannot be recovered from $C_k$ and $\hat L_k$ relies on infinitely many semi-orthogonal factorizations; this rules out exact reconstruction but does not by itself establish formal privacy guarantees against inference attacks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FedHiP, a personalized federated learning method that replaces gradient-based updates with closed-form ridge-regression solutions on features extracted by a frozen pretrained backbone. The method has three phases: analytic local training on each client, recursive analytic global aggregation on the server, and a final personalized analytic model for each client. The main theoretical contributions are Theorem 1 and Theorem 2, which show that the server-side global model and the client-side personalized model exactly solve global and personalized regularized least-squares objectives, respectively. Theorem 3 claims a 'heterogeneity-invariance' property, stating that each personalized model is identical regardless of how non-IID the other clients' data are. Experiments on CIFAR-100 and ImageNet-R under Dirichlet non-IID partitions report accuracy improvements over several PFL baselines.

Significance. If the claims hold, the paper makes a useful contribution to PFL: the closed-form formulation removes iterative gradient updates, reduces communication to a single round, and yields exact, transparent optimization guarantees. Theorems 1 and 2 are clean and verify the equivalence of the recursive aggregation to the stated objectives, and they are straightforward to check. The heterogeneity-invariance property is interesting but, as proven, is substantially narrower than the advertised claim, and the experimental section needs stronger statistical reporting. The paper also cites a prior analytic-FL work [26] while claiming to be the first to introduce analytic learning into PFL, which requires clarification.

major comments (3)
  1. [Abstract and Section IV-E, Theorem 3] The advertised heterogeneity-invariance property is broader than what is proven. The abstract and contribution 3 claim that each personalized model remains identical 'regardless of how non-IID the data are distributed across all other clients.' However, the proof of Theorem 3 explicitly assumes in Eq. (31) that D'_{1:K} is a permutation of the original D_{1:K} with the same overall data pool. Under that assumption, F'^T F' = F^T F and F'^T Y' = F^T Y follow from the permutation matrix's orthogonality, and the theorem is correct. But in standard non-IID FL settings, other clients can have entirely different samples drawn from different distributions; then D' is not a row permutation of D, and F'^T F' != F^T F in general, so the personalized model P_k changes. Section VI's discussion of limitations does not mention this fixed-pool condition. The experiments in Section V-A use Dirichlet par
  2. [Section V-B, Table I and Figures 4-5] The empirical superiority claim is not supported with statistical detail. Table I reports single accuracy values without standard deviations, number of seeds, or significance tests, and Figures 4-5 appear to show single runs. The claim of outperforming baselines by 5.79%-20.97% would be more credible if the results were averaged over multiple independent runs with reported variance, and if the hyperparameter-tuning protocol for the baselines was described. Because FedHiP requires only one round while baselines use 200 rounds, the comparison should also state whether the reported baseline results are the best achievable under the chosen tuning budget.
  3. [Section II-B and Section VI.A] The novelty claim 'to the best of our knowledge, we are the first to introduce analytic learning into PFL' is contradicted by the paper's own reference list. Reference [26], 'AFL: A single-round analytic approach for federated learning with pre-trained models,' is cited in Section II-B among analytic-learning applications, yet the text immediately says there is still a significant gap in introducing analytic learning into PFL. If [26] is indeed an analytic FL method, the paper must explicitly compare with it and correct the novelty claim; if it is not, the citation should be moved or clarified. This is load-bearing for the positioning of the contribution.
minor comments (5)
  1. [Theorem 1 proof, Eq. (26)] The derivation contains a typo: the second line 'S_k M_k = S_k S_k^{-1} F^T_{1:K}F_{1:K} = F^T_{1:K}F_{1:K}' should refer to F^T_{1:K}Y_{1:K}, not F^T_{1:K}F_{1:K}. The subsequent conclusion uses the correct quantity.
  2. [Theorem 3 proof, Eq. (33)] The matrix P in 'P^T P' is undefined; it should be the permutation matrix A introduced in Eq. (31).
  3. [Section VI.B] Typo: 'Nevertherless' should be 'Nevertheless.'
  4. [Section II.B] The phrase 'the target client k’s dataset D1:k' appears to be a typo for D_k. Also, reference [26] contains 'federated learing' in the title.
  5. [Section V.B] It is unclear how the reported 'improvement' is computed when the best baseline is sometimes not the second-best row in Table I. Please state the rule explicitly, e.g., improvement over the best-performing baseline in each column.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: closed-form derivations are self-contained and the invariance property is a proven algebraic consequence, not an input assumed as output.

full rationale

The paper's derivation chain is not circular. The method is defined by closed-form equations (5)-(14), and Theorems 1 and 2 directly verify that the proposed global and personalized models equal the least-squares minimizers of objectives (3) and (1); the proofs expand the recursive aggregation (Lemma 1) and use exact identities such as S_K + \tilde C_k - K\beta I = F^T_{1:K}F_{1:K} + \alpha F_k^T F_k + \beta I, and S_K M_K = F^T_{1:K}Y_{1:K}. No fitted parameter is relabeled as a prediction—\alpha and \beta are explicit hyperparameters, and the experiments compare accuracy against external baselines. Theorem 3's invariance is an algebraic consequence: under the stated row-permutation model, A^T A = I makes F'^T F' = F^T F and F'^T Y' = F^T Y, while the target client's local terms are fixed. The abstract's phrase 'regardless of how non-IID' is broader than the permutation-only assumption in Eq. (31), and Section VI omits this fixed-pool limitation; however, that is a scope/overclaim issue for a correctness review, not a circular derivation. The self-citation to [26] (a prior analytic-FL paper by a co-author) is a positioning/novelty concern and is not load-bearing for the mathematical claims. Hence no circular step is present.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no new entities. It relies on two hyperparameters (α and β), a frozen foundation model, and the assumption that data heterogeneity across clients can be modeled as permutations of a fixed dataset for the invariance property.

free parameters (2)
  • alpha (α) = tuned per setting; optimal around 10-30 (Table II)
    Controls the trade-off between global generalization and local personalization in objective (1). Set by hand via sensitivity analysis.
  • beta (β) = tuned per setting; minor impact (Table III)
    Ridge regularization coefficient in the closed-form solutions. Set by hand; experiments show low sensitivity.
assumptions (4)
  • ad hoc to paper The least-squares objective (1) correctly captures the trade-off between global generalization and local personalization.
    The whole scheme is derived from this objective; its adequacy is assumed, not proven from first principles.
  • domain assumption The frozen foundation model provides features sufficient for the image classification task.
    Empirical reliance on ViT-MAE; no fine-tuning of the backbone is allowed, so performance depends on the quality of its features.
  • ad hoc to paper In Theorem 3, the only way the other clients' data can differ is by a permutation of a fixed global pool.
    The invariance proof uses permutation matrices (Eq. 31); arbitrary changes in the other clients' data are not considered.
  • standard math Standard linear algebra facts (matrix inversion, transposition, associativity) hold.
    Used throughout the derivations of the closed-form solutions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedHiP: Heterogeneity-Invariant Personalized Federated Learning Through Closed-Form Solutions." pith.science (2026). https://pith.science/paper/FNWWH24V

@misc{pith2026250804470,
  author       = {Pith},
  title        = {Pith review of: FedHiP: Heterogeneity-Invariant Personalized Federated Learning Through Closed-Form Solutions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FNWWH24V}},
  note         = {Machine review of arXiv:2508.04470}
}
read the original abstract

Lately, Personalized Federated Learning (PFL) has emerged as a prevalent paradigm to deliver personalized models by collaboratively training while simultaneously adapting to each client's local applications. Existing PFL methods typically face a significant challenge due to the ubiquitous data heterogeneity (i.e., non-IID data) across clients, which severely hinders convergence and degrades performance. We identify that the root issue lies in the long-standing reliance on gradient-based updates, which are inherently sensitive to non-IID data. To fundamentally address this issue and bridge the research gap, in this paper, we propose a Heterogeneity-invariant Personalized Federated learning scheme, named FedHiP, through analytical (i.e., closed-form) solutions to avoid gradient-based updates. Specifically, we exploit the trend of self-supervised pre-training, leveraging a foundation model as a frozen backbone for gradient-free feature extraction. Following the feature extractor, we further develop an analytic classifier for gradient-free training. To support both collective generalization and individual personalization, our FedHiP scheme incorporates three phases: analytic local training, analytic global aggregation, and analytic local personalization. The closed-form solutions of our FedHiP scheme enable its ideal property of heterogeneity invariance, meaning that each personalized model remains identical regardless of how non-IID the data are distributed across all other clients. Extensive experiments on benchmark datasets validate the superiority of our FedHiP scheme, outperforming the state-of-the-art baselines by at least 5.79%-20.97% in accuracy.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 37 canonical work pages

  1. [26]

    AFL: A single-round analytic approach for federated learing with pre-trained models,

    R. He, K. Tong, D. Fang, H. Sun, Z. Zeng, H. Li, T. Chen, and H. Zhuang, “AFL: A single-round analytic approach for federated learing with pre-trained models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2025

    work page 2025

  2. [1]

    FairFed: Improving fairness and efficiency of contribution evaluation in federated learning via cooperative shapley value,

    Y . Liu, S. Chang, Y . Liu, B. Li, and C. Wang, “FairFed: Improving fairness and efficiency of contribution evaluation in federated learning via cooperative shapley value,” in IEEE INFOCOM 2024 - IEEE Conference on Computer Communications , 2024, pp. 621–630

    work page 2024

  3. [2]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics . PMLR, 2017, pp. 1273– 1282

    2017

  4. [3]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine Learning and Systems , vol. 2, pp. 429–450, 2020

    2020

  5. [4]

    Towards personalized federated learning,

    A. Z. Tan, H. Yu, L. Cui, and Q. Yang, “Towards personalized federated learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 12, pp. 9587–9603, 2022

    work page 2022

  6. [5]

    Ditto: Fair and robust federated learning through personalization,

    T. Li, S. Hu, A. Beirami, and V . Smith, “Ditto: Fair and robust federated learning through personalization,” in International conference on Machine Learning . PMLR, 2021, pp. 6357–6368

    work page 2021

  7. [6]

    FedALA: Adaptive local aggregation for personalized federated learn- ing,

    J. Zhang, Y . Hua, H. Wang, T. Song, Z. Xue, R. Ma, and H. Guan, “FedALA: Adaptive local aggregation for personalized federated learn- ing,” in Proceedings of the AAAI conference on Artificial Intelligence , vol. 37, no. 9, 2023, pp. 11 237–11 244

    work page 2023

  8. [7]

    FedLFP: Communication-efficient personalized federated learning on non-iid data in mobile edge computing environments,

    Y . Sun, S. Pan, A. Sun, Z. Fu, S. Long, and Z. Li, “FedLFP: Communication-efficient personalized federated learning on non-iid data in mobile edge computing environments,” IEEE Transactions on Mobile Computing, pp. 1–13, 2025

    work page 2025

Show all 40 references
  1. [8]

    Personalized federated learning with model-contrastive learning for multi-modal user modeling in human-centric metaverse,

    X. Zhou, Q. Yang, X. Zheng, W. Liang, K. I.-K. Wang, J. Ma, Y . Pan, and Q. Jin, “Personalized federated learning with model-contrastive learning for multi-modal user modeling in human-centric metaverse,” IEEE Journal on Selected Areas in Communications , vol. 42, no. 4, pp. 8...

    2024

  2. [9]

    Multi-level person- alized federated learning on heterogeneous and long-tailed data,

    R. Zhang, Y . Chen, C. Wu, F. Wang, and B. Li, “Multi-level person- alized federated learning on heterogeneous and long-tailed data,” IEEE Transactions on Mobile Computing , vol. 23, no. 12, pp. 12 396–12 409, 2024

    2024

  3. [10]

    Federated learning while providing model as a service: Joint training and inference optimization,

    P. Han, S. Wang, Y . Jiao, and J. Huang, “Federated learning while providing model as a service: Joint training and inference optimization,” in IEEE INFOCOM 2024 - IEEE Conference on Computer Communi- cations, 2024, pp. 631–640

    2024

  4. [11]

    CPDZ: A credibility-aware and privacy-preserving data collection scheme with zero-trust in next-generation crowdsensing networks,

    J. Tang, K. Fan, S. Yang, A. Liu, N. N. Xiong, H. H. Song, and V . C. M. Leung, “CPDZ: A credibility-aware and privacy-preserving data collection scheme with zero-trust in next-generation crowdsensing networks,” IEEE Journal on Selected Areas in Communications, vol. 43, no. 6,...

    2025

  5. [12]

    QLP-DCS: A quality-aware, low-cost, and privacy- preserving data collection service for mobile crowd sensing,

    Y . Huang, J. Guo, S. Yang, J. Liu, A. Liu, J. Tang, T. Wang, M. Dong, and H. Song, “QLP-DCS: A quality-aware, low-cost, and privacy- preserving data collection service for mobile crowd sensing,” IEEE Transactions on Services Computing , pp. 1–16, 2025

    2025

  6. [13]

    RMDF-CV: A reliable multi-source data fusion scheme with cross validation for quality service construction in mobile crowd sensing,

    K. Fan, J. Guo, R. Li, Y . Li, A. Liu, J. Tang, T. Wang, M. Dong, and H. Song, “RMDF-CV: A reliable multi-source data fusion scheme with cross validation for quality service construction in mobile crowd sensing,” IEEE Transactions on Services Computing , vol. 18, no. 1, pp. 39...

    2025

  7. [14]

    Pseudoinverse learning algo- rithm for feedforward neural networks,

    P. Guo, M. R. Lyu, and N. Mastorakis, “Pseudoinverse learning algo- rithm for feedforward neural networks,” Advances in Neural Networks and Applications, vol. 1, no. 321-326, 2001

    2001

  8. [15]

    GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task,

    H. Zhuang, Z. Weng, R. He, Z. Lin, and Z. Zeng, “GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7746–7755

    2023

  9. [16]

    ACIL: Analytic class-incremental learning with absolute memorization and privacy protection,

    H. Zhuang, Z. Weng, H. Wei, R. Xie, K.-A. Toh, and Z. Lin, “ACIL: Analytic class-incremental learning with absolute memorization and privacy protection,”Advances in Neural Information Processing Systems, vol. 35, pp. 11 602–11 614, 2022

    2022

  10. [17]

    A progressive stacking pseudoinverse learning framework via active learning in random sub- spaces,

    Z. Cai, S. Zhang, P. Guo, J. Zhang, and L. Hu, “A progressive stacking pseudoinverse learning framework via active learning in random sub- spaces,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 54, no. 5, pp. 2822–2832, 2024

    2024

  11. [18]

    Bayesian pseudoinverse learners: From uncertainty to deterministic learning,

    Q. Yin, B. Xu, K. Zhou, and P. Guo, “Bayesian pseudoinverse learners: From uncertainty to deterministic learning,” IEEE Transactions on Cybernetics, vol. 52, no. 11, pp. 12 205–12 216, 2022

    2022

  12. [19]

    Universal approximation using radial-basis- function networks,

    J. Park and I. W. Sandberg, “Universal approximation using radial-basis- function networks,” Neural Computation , vol. 3, no. 2, pp. 246–257, 1991

    1991

  13. [20]

    Learning from the kernel and the range space,

    K.-A. Toh, “Learning from the kernel and the range space,” in 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS). IEEE, 2018, pp. 1–6

    2018

  14. [21]

    Noniterative deep learning: Incorporating restricted boltzmann machine into multilayer random weight neural networks,

    X.-Z. Wang, T. Zhang, and R. Wang, “Noniterative deep learning: Incorporating restricted boltzmann machine into multilayer random weight neural networks,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 7, pp. 1299–1308, 2017

    2017

  15. [22]

    An analytic formulation of convolutional neural network learning for pattern recognition,

    H. Zhuang, Z. Lin, Y . Yang, and K.-A. Toh, “An analytic formulation of convolutional neural network learning for pattern recognition,” Infor- mation Sciences, vol. 686, p. 121317, 2025

    2025

  16. [23]

    Densepilae: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder,

    J. Wang, P. Guo, and Y . Li, “Densepilae: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder,” Complex & Intelligent Systems, pp. 1–11, 2022

    2022

  17. [24]

    Blockwise recursive moore–penrose inverse for network learning,

    H. Zhuang, Z. Lin, and K.-A. Toh, “Blockwise recursive moore–penrose inverse for network learning,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 52, no. 5, pp. 3237–3250, 2021

    2021

  18. [25]

    CALM: A ubiquitous crowdsourced analytic learning mechanism for continual service construction with data privacy preservation,

    K. Fan, Y . Huang, J. He, F. Han, J. Tang, H. Zhuang, A. Liu, T. Wang, M. Dong, H. H. Song, and Y . Liu, “CALM: A ubiquitous crowdsourced analytic learning mechanism for continual service construction with data privacy preservation,” Proceedings of the ACM on Interactive, Mobi...

    2025

  19. [27]

    Locality sensitive sparse encoding for learning world models online,

    Z. Liu, C. Du, W. S. Lee, and M. Lin, “Locality sensitive sparse encoding for learning world models online,” in The Twelfth International Conference on Learning Representations , 2024

    2024

  20. [28]

    Masked au- toencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll ´ar, and R. Girshick, “Masked au- toencoders are scalable vision learners,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 15 979–15 988

    2022

  21. [29]

    Where to begin? on the impact of pre-training and initialization in federated learning,

    J. Nguyen, J. Wang, K. Malik, M. Sanjabi, and M. Rabbat, “Where to begin? on the impact of pre-training and initialization in federated learning,” in The Eleventh International Conference on Learning Rep- resentations, 2023

    2023

  22. [30]

    On the importance and applicability of pre-training for federated learning,

    H.-Y . Chen, C.-H. Tu, Z. Li, H. W. Shen, and W.-L. Chao, “On the importance and applicability of pre-training for federated learning,” in The Eleventh International Conference on Learning Representations , 2023

    2023

  23. [31]

    FedBERT: When federated learning meets pre-training,

    Y . Tian, Y . Wan, L. Lyu, D. Yao, H. Jin, and L. Sun, “FedBERT: When federated learning meets pre-training,” ACM Transactions on Intelligent Systems and Technology, vol. 13, no. 4, Aug. 2022

    2022

  24. [32]

    AugFL: Aug- menting federated learning with pretrained models,

    S. Yue, Z. Qin, Y . Deng, J. Ren, Y . Zhang, and J. Zhang, “AugFL: Aug- menting federated learning with pretrained models,” IEEE Transactions on Networking, 2025

    2025

  25. [33]

    FedDAT: An approach for foundation model finetuning in multi-modal heterogeneous federated learning,

    H. Chen, Y . Zhang, D. Krompass, J. Gu, and V . Tresp, “FedDAT: An approach for foundation model finetuning in multi-modal heterogeneous federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 10, 2024, pp. 11 285–11 293

    2024

  26. [34]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Technical Report, 2009

    2009

  27. [35]

    The many faces of robustness: A critical analysis of out-of-distribution generalization,

    D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo et al. , “The many faces of robustness: A critical analysis of out-of-distribution generalization,” in Proceedings of the IEEE/CVF International Conference on Computer Visio...

    2021

  28. [36]

    Eliminating domain bias for federated learning in representa- tion space,

    J. Zhang, Y . Hua, J. Cao, H. Wang, T. Song, Z. XUE, R. Ma, and H. Guan, “Eliminating domain bias for federated learning in representa- tion space,” Advances in Neural Information Processing Systems, vol. 36, pp. 14 204–14 227, 2023

    2023

  29. [37]

    FedAS: Bridging inconsistency in personalized federated learning,

    X. Yang, W. Huang, and M. Ye, “FedAS: Bridging inconsistency in personalized federated learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 11 986–11 995

    2024

  30. [38]

    Federated learning from pre-trained models: A contrastive learning approach,

    Y . Tan, G. Long, J. Ma, L. Liu, T. Zhou, and J. Jiang, “Federated learning from pre-trained models: A contrastive learning approach,” Advances in neural information processing systems, vol. 35, pp. 19 332–19 344, 2022

    2022

  31. [39]

    FedSelect: Personalized federated learning with customized selection of parameters for fine-tuning,

    R. Tamirisa, C. Xie, W. Bao, A. Zhou, R. Arel, and A. Shamsian, “FedSelect: Personalized federated learning with customized selection of parameters for fine-tuning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 23 985–23 994....

    2024

  32. [2008]

    He is currently a Full Professor at the School of Computer Science, Peking University, China. He was a recipient of the National Talented Young Scholar Program (2013), and the National Distin- guished Young Scholar Program (2019), funded by the National Science Foundation Chin...

    2013

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.