Pith. sign in

REVIEW 4 major objections 7 minor 83 references

Hypernetworks for Model-Heterogeneous Personalized Federated Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A server-side hypernetwork can generate accurate personalized parameters for federated clients with heterogeneous, undisclosed model architectures, and the optional global-model distillation variant improves accuracy and generalization.

desk verdict Useful hypernetwork extension for heterogeneous pFL, but the architecture-privacy claim only holds in the easy regime where each architecture has its own parameter-count bucket; the direct shared-head test shows a consistent drop. read the letter →

arxiv 2507.22330 v1 pith:QVP226XK submitted 2025-07-30 cs.LG cs.DC

Chen Zhang , Husheng Li , Xiang Liu , Linshan Jiang , Danxin Wang This is my paper
classification cs.LGcs.DC
keywords federatedlearningpersonalizedmodelheterogeneityhypernetworkknowledgedistillationarchitectureprivacyparametergeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that model-heterogeneous personalized federated learning can be handled by a single server-side hypernetwork that generates a client's entire model parameters from a small client-specific embedding vector. The client uploads only the number of parameters its model needs, not the architecture, and the hypernetwork's shared feature extractor plus per-size heads produce a flat vector that the client reshapes into its own network. The paper further claims that adding a lightweight global model, trained briefly by all clients and used as a distillation teacher, improves accuracy and generalization. If this works, the main obstacles of earlier model-heterogeneous methods, such as external or synthetic data, model decoupling, and partial training, can be avoided, which would matter for privacy-sensitive and resource-heterogeneous deployments.

What carries the argument

The load-bearing object is the server-side hypernetwork $h(v; \phi) = (\phi_f, \{\phi_{H_l}\})$, where $\phi_f$ is a feature extractor shared by all clients and each head $\phi_{H_l}$ has output channels equal to the number of embedding vectors of the clients it serves. For client $i$ with $K_i$ parameters, the server computes $\tau_i = \lceil K_i/N \rceil$ embeddings, feeds each through the shared extractor and the client's head, concatenates the $\tau_i$ chunks of length $N$, and truncates to $K_i$: $\theta_i := \text{concat}(h(v_i^1; \phi_f, \phi_{H_l}), \ldots)[1:K_i]$. Because clients with equal parameter counts share a head, the hypernetwork generates several models in one forward pass; because only $K_i$ is revealed, the architecture itself is never transmitted. The mechanism's work is to replace direct aggregation of heterogeneous weights with aggregation in the hypernetwork's parameter space, and to let the global model in MH-pFedHNGD provide a second update signal plus a distillation target.

What would settle it

Construct a federation with two client groups that have the same total parameter count but incompatible layer decompositions, for instance one group using a single wide hidden layer and the other using several narrow hidden layers, and train MH-pFedHN with both groups sharing the same head. If one group converges to near-baseline accuracy while the other stays near chance, the claim that the hypernetwork can generate useful parameters for undisclosed architectures fails; if both succeed, the implicit alignment assumption is confirmed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a hypernetwork can act as a knowledge aggregator across clients with different, undisclosed model architectures. For client $i$ the server knows only $K_i$, the number of parameters; it chooses $\tau_i = \lceil K_i/N \rceil$ embedding vectors, passes them through a shared feature extractor and a head shared by clients with similar sizes, concatenates the outputs, and truncates to the first $K_i$ entries to obtain $\theta_i$. The client trains $\theta_i$ locally and returns only the update $\Delta\theta_i$, which the server uses to update the hypernetwork and the client's embedding. In MH-pFedHNGD, a global model with $K_g = \min_i K_i$ parameters is generated the same way, is trained by all clients, and then serves as a teacher in a KL-distillation term. The reported experiments across EMNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet, in both homogeneous and heterogeneous model settings, show these methods outperforming all compared baselines, and the ablation studies attribute the gain to the shared multi-head structure and to the global-model distillation.

Load-bearing premise

The method assumes that a flat parameter vector generated without ever seeing a client's tensor layout will, after the client reshapes its first $K_i$ entries, form a usable model even when clients with equal parameter counts have different layer boundaries.

Editorial extensions

If this is right

  • Clients can train with their own architectures while the server generates all parameters, so model heterogeneity no longer requires the server to know or store the client architectures.
  • The server can produce parameters for multiple clients with similar model sizes in one forward pass, cutting computation and communication compared with per-client generation.
  • Adding the lightweight global model improves accuracy over the hypernetwork alone in almost all reported settings, and the gain persists even when the global and personalized architectures differ.
  • The methods generalize to unseen clients and, with a newly trained head only, to entirely new architectures, which no compared baseline handles.
  • The paper itself notes that replacing homogeneous models with heterogeneous models lowers accuracy in the same settings, so heterogeneity remains an open cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the architecture-privacy guarantee rests on parameter count as the only leaked signal; a server that knows $K_i$ and the generated flat layout may still infer partial structural information, so 'no disclosure' should be understood as a practical feature, not a formal privacy bound.
  • Editorial inference: the method's success implies the hypernetwork must implicitly learn an ordering convention for parameters; testing clients with identical $K_i$ but permuted or interleaved layer layouts would reveal whether this ordering is required or merely convenient.
  • Editorial inference: the appendix's pruning experiment, where only the largest 30% of update magnitudes are uploaded with little accuracy loss, suggests the hypernetwork's gradient signal is sparse; combining this with the shared-head structure could make the approach communication-competitive with partial-training methods while keeping full personalization.
  • Editorial inference: the global model in MH-pFedHNGD could double as a cheap compatibility test for new devices, since a new client can first compare its local test accuracy against the global model before deciding whether to invest in a personalized architecture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes MH-pFedHN and MH-pFedHNGD, server-side hypernetwork frameworks for model-heterogeneous personalized federated learning. Clients upload only their required parameter counts; the server maps client-specific embeddings through a shared feature extractor and multiple heads to generate flat parameter vectors, truncated to the required sizes, which clients reshape into their local architectures. MH-pFedHNGD additionally generates a lightweight global model from the smallest client's head, uses it for an extra server-side hypernetwork update, and applies it as a teacher for local knowledge distillation. Experiments on EMNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet compare against pFedHN, pFedLA, FedGH, pFedLHN, PeFLL, FedAKT, FedAvg, and local training under homogeneous and heterogeneous model settings, with ablations on heads, global model, participation ratios, weight pruning, resource constraints, and gradient-inversion privacy.

Significance. If the method works as claimed, it would provide a simple, data-free approach to model-heterogeneous personalized FL that avoids external data, model decoupling, and architecture disclosure, which would be practically valuable. The paper's strengths are its extensive empirical coverage (multiple datasets, client counts, non-IID settings, ablations, overhead and privacy analyses), clear algorithm pseudocode, and explicit experiments on generalization to unseen clients and new architectures. However, the central theoretical assertion in Eq. (4) is not justified, and the key architecture-agnosticism claim is only tested in a regime where architectures have distinct parameter counts; the one ablation that tests the hard case shows a consistent performance drop. These issues, together with the absence of variance reporting, mean that the stated claim of exceeding state-of-the-art performance across all tasks is currently not fully supported.

major comments (4)
  1. [Section 3.2, Eq. (4)] The equality between the hypernetwork-constrained objective and the unconstrained per-client objective is incorrect without an expressivity assumption. For the equality to hold, for every tuple (theta_1,...,theta_n) there must exist phi and v_1,...,v_n such that h(v_i; phi)[1:K_i] = theta_i for all i. With a fixed MLP hypernetwork of output dimension N and shared phi, this is not established and is generally false: the shared feature extractor and heads couple all clients, and truncation discards coordinates. The same issue appears in Eq. (6). Please replace the equality with an approximate or surrogate objective, or provide a capacity/universality argument, and state the approximation explicitly.
  2. [Section 3.2, Eq. (3); Section 4.3; Appendix C.5, Table 10] The architecture-privacy claim is only supported in the easy regime. The server generates a flat vector from only K_i; if two clients share a head (same parameter-count bucket) but have different layer layouts, the identical generated numbers must be simultaneously useful under two different reshapings, and gradient updates from those clients will conflict at the same output coordinates. No alignment, permutation-invariance, or capacity argument is given. The main heterogeneous experiments (Table 2; architectures in Appendix E with parameter counts 0.915M, 1.048M, 1.347M, 2.018M, 2.179M) give each architecture a distinct parameter count, so each occupies its own head. The only direct test, Table 10, shows a consistent drop when LeNet, VGG, and MLP are forced to share a head at identical parameter counts (e.g., MH-pFedHN on CIFAR-100 non-IID_1, 50 clients: 55.92 vs 57.84). The claim that the method does not require disclosure of client model architectures should either be demonstrated in the same-parameter-count, different-architecture setting or substantially softened.
  3. [Section 4.1, Tables 1 and 2] The paper states that results are averaged over three runs but does not report standard deviations or confidence intervals in any main table. Many comparisons in Tables 1 and 2 are within a fraction of a point (e.g., EMNIST in Table 1: 97.56 vs 97.52), so without variance information it is impossible to judge whether the claimed improvements are significant. Please report standard deviations, confidence intervals, or per-run results for at least the main tables.
  4. [Section 4.1 and Appendices D.4-D.6] Hyperparameters such as hypernetwork output size (3072), distillation temperature (15 on CIFAR-100), and balancing factor (0.01 on CIFAR-100) are selected using test-set performance on the same benchmarks where final accuracies are reported. This creates a selection-circularity concern for the generalization claims in Section 4.4. Please use a validation split for hyperparameter selection, or show sensitivity results on held-out settings to demonstrate that the reported numbers are not inflated by test-set tuning.
minor comments (7)
  1. [Section 2.2] The phrase 'communication limitations, communication limitations' is duplicated in the first paragraph; please remove the repetition.
  2. [Equation (2)] The sentence beginning 'By allowing each client to optimize its model, ensuring...' is a fragment; also the subscript on x_j,y_j in Eq. (1) is inconsistent with the notation used elsewhere.
  3. [Section 3.2] The notation switches between K and K_i; please define K_i clearly as the number of parameters required by client i and use it consistently throughout.
  4. [Table 3] The green/red color coding is not visible in a black-and-white version and the rows are ambiguous; please add a legend or explicit labels to the table.
  5. [Algorithm 1] The server update phi = phi - alpha * grad_phi(theta_i^T) * Delta_theta_i is a first-order approximation of the hypernetwork gradient; please state this assumption explicitly rather than leaving it implicit.
  6. [Appendix C.5, Table 10] The statement that 'our method is sufficiently robust' is not supported by the numbers immediately preceding it, which show a consistent drop for shared heads; please rephrase to acknowledge the trade-off between privacy and accuracy.
  7. [Section 4.4, Figure 5] The claim that there is 'no significant difference' between MH-pFedHN and MH-pFedHNGD for new architectures is asserted without quantitative support; please provide the actual numbers or curves.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's equations define an optimization objective, and the reported results are empirical fits rather than predictions derived by construction from fitted inputs.

full rationale

The formal derivation chain is an optimization formulation, not a predictive theorem. Equation (3) defines the generated parameters as a truncated concatenation of hypernetwork outputs; Equation (4) substitutes this definition into the client-wise empirical loss, so the equality is notational. No fitted parameter is later renamed as a prediction, and no architectural assumption is imported through a self-citation. The only overlapping-author citation [74] supports a standard non-IID data partition choice and is not load-bearing. The generalization experiments in Section 4.4 explicitly allow test-client embedding vectors to be optimized and new heads to be added and trained, which makes the generalization wording adaptation-based rather than zero-shot; this is disclosed in the text and is a claims-quality issue, not a reduction of the method to its own inputs. Table 10's equal-parameter-count ablation shows a real limitation of the architecture-privacy claim, but that is an experimental weakness, not circularity under the quoted-reduction standard.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities: client embeddings, hypernetwork heads, and the global model are all standard machine-learning components. The main burden is carried by free hyperparameters tuned on the same benchmarks and by the unproven representational and alignment assumptions listed above.

free parameters (5)
  • Hypernetwork output dimension N = 3072
    Chosen by ablation on CIFAR-100 test accuracy (Appendix D.4); affects truncation length and number of embedding vectors per client.
  • Client embedding dimension = 64
    Chosen by ablation on CIFAR-100 (Appendix D.2).
  • Number of hypernetwork hidden layers = 3
    Chosen by ablation on CIFAR-100 (Appendix D.3).
  • Distillation temperature T = 15 (CIFAR-100), 24 (Tiny-ImageNet), 10 (EMNIST)
    Chosen per dataset via ablation (Section 4.1, Appendix D.5).
  • Distillation balancing factor lambda = 0.01 (CIFAR-100), 0.2 (Tiny-ImageNet), 0.1 (EMNIST)
    Chosen per dataset via ablation (Section 4.1, Appendix D.6).
assumptions (5)
  • ad hoc to paper The hypernetwork is expressive enough to make the constrained minimization in Eq. (4) equivalent to the unconstrained per-client objective.
    Eq. (4) asserts this equality; no surjectivity or capacity argument is given.
  • domain assumption The first-order pseudo-gradient update using Delta_theta_i as a constant target is a valid way to train phi and v_i.
    Algorithm 1 uses grad_phi theta_i^T Delta_theta_i; inherited from pFedHN without convergence analysis for heterogeneous architectures.
  • ad hoc to paper The server can generate usable model parameters for arbitrary client architectures from a flat vector of length K_i without knowing the tensor layout.
    Section 3.2 generates theta_i = h(v_i; phi)[1:K_i] and the client reshapes privately; this is the core architecture-privacy premise.
  • domain assumption Parameter count K_i is a sufficient statistic for grouping clients into heads without degrading accuracy.
    Multi-head sharing in Section 3.2 groups by ceil(K_i/N); Table 10 shows same-size different-architecture sharing slightly hurts, so the grouping is a design assumption.
  • domain assumption Standard benchmarks and Dirichlet or quantity-based non-IID partitions model realistic FL deployment.
    Section 4.1 defines evaluation; validity of empirical conclusions depends on this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hypernetworks for Model-Heterogeneous Personalized Federated Learning." pith.science (2026). https://pith.science/paper/QVP226XK

@misc{pith2026250722330,
  author       = {Pith},
  title        = {Pith review of: Hypernetworks for Model-Heterogeneous Personalized Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QVP226XK}},
  note         = {Machine review of arXiv:2507.22330}
}
read the original abstract

Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning strategies, which can limit their practicality and scalability. In this paper, we revisit hypernetwork-based methods and leverage their strong generalization capabilities to design a simple yet effective framework for heterogeneous personalized federated learning. Specifically, we propose MH-pFedHN, which leverages a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model. To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads. Furthermore, we further propose MH-pFedHNGD, which integrates an optional lightweight global model to improve generalization. Our framework does not rely on external datasets and does not require disclosure of client model architectures, thereby offering enhanced privacy and flexibility. Extensive experiments on multiple benchmarks and model settings demonstrate that our approach achieves competitive accuracy, strong generalization, and serves as a robust baseline for future research in model-heterogeneous personalized federated learning.

Figures

Figures reproduced from arXiv: 2507.22330 by the authors.

Figure 2
Figure 2. Framework of MH-pFedHNGD. The workflow contains 10 steps: ① input the em￾bedding matrix vg of the global model into the hypernetwork; ② generates the parameters wg of the global model; ③ each client receives the global model parameters wg from the server; ④ each client trains the global model using private data; ⑤ The server receives ∆wg,i from each client and updates the hypernetwork and global embedding vector par… view at source ↗
Figure 1
Figure 1. Framework of MH-pFedHN. The work￾flow contains 5 steps: ① input the embedding ma￾trix vi of client i into the hypernetwork; ② gener￾ates the parameters θ i of client i; ③ client i receives the parameters θ i from the server; ④ client i trains the personalized model θ i using the private data; ⑤ client i uploads the update of parameters ∆θ i to the server; server updates customized embedding vectors and hypernetworks… view at source ↗
Figure 4
Figure 4. Generalization experiments in the het￾erogeneous scenario, where solid lines denote test clients and dashed lines denote training clients. and Local Training do not allow model heterogeneity, we thus conduct repeat experiments using various models and take the average test accuracy as the final results. Experiment results in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Generalization experiments for new ar￾chitectures, where solid lines denote test clients and dashed lines denote training clients [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: iDLG Gradient Inversion Attack using MH-pFedHNGD [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: Experiment with the number of local epochs on the CIFAR-100 dataset using MH-pFedHN. [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: MH-pFedHN test results on CIFAR-100 showing the effect of (a) the dimension of client [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

83 extracted references · 58 canonical work pages

  1. [1]

    Adaptive segmentation enhanced asynchronous federated learning for sustainable intelligent transportation systems

    Xiaokang Zhou, Wei Liang, Akira Kawai, Kaoru Fueda, Jinhua She, I Kevin, and Kai Wang. Adaptive segmentation enhanced asynchronous federated learning for sustainable intelligent transportation systems. IEEE Transactions on Intelligent Transportation Systems, 2024

    2024

  2. [2]

    Ai- empowered trajectory anomaly detection for intelligent transportation systems: A hierarchi- cal federated learning approach

    Xiaoding Wang, Wenxin Liu, Hui Lin, Jia Hu, Kuljeet Kaur, and M Shamim Hossain. Ai- empowered trajectory anomaly detection for intelligent transportation systems: A hierarchi- cal federated learning approach. IEEE Transactions on Intelligent Transportation Systems , 24(4):4631–4640, 2022

    2022

  3. [3]

    Reliable federated learning with gan model for robust and resilient future healthcare system

    Anita Murmu, Piyush Kumar, Nageswara Rao Moparthi, Suyel Namasudra, and Pascal Lorenz. Reliable federated learning with gan model for robust and resilient future healthcare system. IEEE Transactions on Network and Service Management, 2024

    2024

  4. [4]

    Harmony: Heterogeneous multi-modal federated learning through disentangled model training

    Xiaomin Ouyang, Zhiyuan Xie, Heming Fu, Sitong Cheng, Li Pan, Neiwen Ling, Guoliang Xing, Jiayu Zhou, and Jianwei Huang. Harmony: Heterogeneous multi-modal federated learning through disentangled model training. In Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services, pages 530–543, 2023

    work page 2023

  5. [5]

    Federated learning for smart healthcare: A survey

    Dinh C Nguyen, Quoc-Viet Pham, Pubudu N Pathirana, Ming Ding, Aruna Seneviratne, Zihuai Lin, Octavia Dobre, and Won-Joo Hwang. Federated learning for smart healthcare: A survey. ACM Computing Surveys (Csur), 55(3):1–37, 2022

    2022

  6. [6]

    Robust privacy-preserving recommendation systems driven by multimodal federated learning

    Chenyuan Feng, Daquan Feng, Guanxin Huang, Zuozhu Liu, Zhenzhong Wang, and Xiang-Gen Xia. Robust privacy-preserving recommendation systems driven by multimodal federated learning. IEEE Transactions on Neural Networks and Learning Systems, 2024

    work page 2024

  7. [7]

    Federated unlearning for on-device recommendation

    Wei Yuan, Hongzhi Yin, Fangzhao Wu, Shijie Zhang, Tieke He, and Hao Wang. Federated unlearning for on-device recommendation. In Proceedings of the sixteenth ACM international conference on web search and data mining, pages 393–401, 2023

    work page 2023

  8. [8]

    Prefer: Point-of-interest recommendation with efficiency and privacy-preservation via federated edge learning

    Yeting Guo, Fang Liu, Zhiping Cai, Hui Zeng, Li Chen, Tongqing Zhou, and Nong Xiao. Prefer: Point-of-interest recommendation with efficiency and privacy-preservation via federated edge learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(1):1–25, 2021

    work page 2021

Show all 83 references
  1. [9]

    Federated multi-task learning

    Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. Federated multi-task learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17. Curran Associates Inc., 2017

    2017

  2. [10]

    Personalized federated learning with moreau envelopes

    Canh T Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. Advances in neural information processing systems, 33, 2020

    2020

  3. [11]

    Adaptive personalized federated learning

    Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461, 2020. 10

    2003 arXiv

  4. [12]

    H. B. McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics, 2016

    2016

  5. [13]

    Tifl: A tier-based federated learning system

    Zheng Chai, Ahsan Ali, Syed Zawad, Stacey Truex, Ali Anwar, Nathalie Baracaldo, Yi Zhou, Heiko Ludwig, Feng Yan, and Yue Cheng. Tifl: A tier-based federated learning system. In Proceedings of the 29th international symposium on high-performance parallel and distributed computi...

    2020

  6. [14]

    Effective heterogeneous federated learning via efficient hypernetwork-based weight generation

    Yujin Shin, Kichang Lee, Sungmin Lee, You Rim Choi, Hyung-Sin Kim, and JeongGil Ko. Effective heterogeneous federated learning via efficient hypernetwork-based weight generation. In Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems, pages 112–125, 2024

    2024

  7. [15]

    Sebastian Caldas, Jakub Konecný, H. B. McMahan, and Ameet Talwalkar. Expanding the reach of federated learning by reducing client resource requirements. ArXiv, abs/1812.07210, 2018

    2018 arXiv

  8. [16]

    Group knowledge transfer: federated learning of large cnns at the edge

    Chaoyang He, Murali Annavaram, and Salman Avestimehr. Group knowledge transfer: federated learning of large cnns at the edge. In Proceedings of the 34th International Conference on Neural Information Processing Systems, Red Hook, NY , USA, 2020. Curran Associates Inc

    2020

  9. [17]

    Suhail Mohmad Shah and Vincent K. N. Lau. Model compression for communication efficient federated learning. IEEE Transactions on Neural Networks and Learning Systems, 34:5937– 5951, 2021

    2021

  10. [18]

    Fedmd: Heterogenous federated learning via model distillation

    Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581, 2019

    1910 arXiv

  11. [19]

    Data-free knowledge distillation for heteroge- neous federated learning

    Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heteroge- neous federated learning. In International conference on machine learning, pages 12878–12889. PMLR, 2021

    2021

  12. [20]

    Exploring the distributed knowledge congruence in proxy-data-free federated distillation

    Zhiyuan Wu, Sheng Sun, Yuwei Wang, Min Liu, Quyang Pan, Junbo Zhang, Zeju Li, and Qingxiang Liu. Exploring the distributed knowledge congruence in proxy-data-free federated distillation. ACM Transactions on Intelligent Systems and Technology, 15(2):1–34, 2024

    2024

  13. [21]

    Efficient personalized federated learning via sparse model-adaptation

    Daoyuan Chen, Liuyi Yao, Dawei Gao, Bolin Ding, and Yaliang Li. Efficient personalized federated learning via sparse model-adaptation. ArXiv, abs/2305.02776, 2023

    2023 arXiv

  14. [22]

    Hetero{fl}: Computation and communication efficient federated learning for heterogeneous clients

    Enmao Diao, Jie Ding, and Vahid Tarokh. Hetero{fl}: Computation and communication efficient federated learning for heterogeneous clients. In International Conference on Learning Representations, 2021

    2021

  15. [23]

    Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction

    Samiul Alam, Luyang Liu, Ming Yan, and Mi Zhang. Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction. Advances in neural information processing systems, 35:29677–29690, 2022

    2022

  16. [24]

    Efficient split-mix federated learning for on-demand and in-situ customization, 2022

    Junyuan Hong, Haotao Wang, Zhangyang Wang, and Jiayu Zhou. Efficient split-mix federated learning for on-demand and in-situ customization, 2022

    2022

  17. [25]

    Fedconv: A learning-on-model paradigm for heterogeneous federated clients

    Leming Shen, Qiang Yang, Kaiyan Cui, Yuanqing Zheng, Xiao-Yong Wei, Jianwei Liu, and Jinsong Han. Fedconv: A learning-on-model paradigm for heterogeneous federated clients. In Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services,...

    2024

  18. [26]

    Zhang, Song Guo, Jingcai Guo, Deze Zeng, Jingren Zhou, and Albert Y

    J. Zhang, Song Guo, Jingcai Guo, Deze Zeng, Jingren Zhou, and Albert Y . Zomaya. To- wards data-independent knowledge transfer in model-heterogeneous federated learning. IEEE Transactions on Computers, 72:2888–2901, 2023

    2023

  19. [27]

    Dfrd: Data-free robustness distillation for heterogeneous federated learning

    Shuai Wang, Yexuan Fu, Xiang Li, Yunshi Lan, Ming Gao, et al. Dfrd: Data-free robustness distillation for heterogeneous federated learning. Advances in Neural Information Processing Systems, 36, 2024. 11

    2024

  20. [28]

    Adapter-guided knowledge transfer for heterogeneous federated learning

    Shichong Liu, Haozhe Jin, Zhiwei Tang, Rui Zhai, Ke Lu, Junyang Yu, and Chenxi Bai. Adapter-guided knowledge transfer for heterogeneous federated learning. Journal of Systems Architecture, page 103338, 2025

    2025

  21. [29]

    Heteroge- neous federated learning framework for iiot based on selective knowledge distillation

    Sheng Guo, Hui Chen, Yang Liu, Chengyi Yang, Zengxiang Li, and Cheng Hao Jin. Heteroge- neous federated learning framework for iiot based on selective knowledge distillation. IEEE Transactions on Industrial Informatics, 21(2):1078–1089, 2025

    2025

  22. [30]

    Personalized federated learning with feature alignment and classifier collaboration

    Jian Xu, Xinyi Tong, and Shao-Lun Huang. Personalized federated learning with feature alignment and classifier collaboration. arXiv preprint arXiv:2306.11867, 2023

    2023 arXiv

  23. [31]

    Exploiting shared representations for personalized federated learning

    Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared representations for personalized federated learning. In Marina Meila and Tong Zhang, edi- tors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings...

    2021

  24. [32]

    Federated learning with personalization layers

    Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818, 2019

    1912 arXiv

  25. [33]

    Allen, Randy P

    Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. Think locally, act globally: Federated learning with local and global representations, 2020

    2020

  26. [34]

    Fedclassavg: Local representation learning for personalized federated learning on heterogeneous neural networks

    Jaehee Jang, Heoneok Ha, Dahuin Jung, and Sungroh Yoon. Fedclassavg: Local representation learning for personalized federated learning on heterogeneous neural networks. In Proceedings of the 51st International Conference on Parallel Processing, ICPP ’22, New York, NY , USA,

  27. [35]

    Fedgh: Heterogeneous federated learning with generalized global header

    Liping Yi, Gang Wang, Xiaoguang Liu, Zhuan Shi, and Han Yu. Fedgh: Heterogeneous federated learning with generalized global header. InProceedings of the 31st ACM International Conference on Multimedia, MM ’23, page 8686¨C8696, New York, NY , USA, 2023. Association for Computin...

    2023

  28. [36]

    pfedes: Model heterogeneous personalized federated learning with feature extractor sharing

    Liping Yi, Han Yu, Gang Wang, and Xiaoguang Liu. pfedes: Model heterogeneous personalized federated learning with feature extractor sharing. arXiv preprint arXiv:2311.06879, 2023

    2023 arXiv

  29. [37]

    Fjord: Fair and accurate federated learning under heterogeneous targets with ordered dropout

    Samuel Horváth, Stefanos Laskaridis, Mario Almeida, Ilias Leontiadis, Stylianos Venieris, and Nicholas Lane. Fjord: Fair and accurate federated learning under heterogeneous targets with ordered dropout. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P.S. Liang, and J. Wortman Vau...

    2021

  30. [38]

    Scalefl: Resource-adaptive federated learning with heterogeneous clients

    Fatih Ilhan, Gong Su, and Ling Liu. Scalefl: Resource-adaptive federated learning with heterogeneous clients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24532–24541, June 2023

    2023

  31. [39]

    Royson Lee, Javier Fernandez-Marques, Shell Xu Hu, Da Li, Stefanos Laskaridis, Łukasz Dudziak, Timothy Hospedales, Ferenc Huszár, and Nicholas D. Lane. Recurrent early exits for federated learning with heterogeneous clients, 2024

    2024

  32. [40]

    Ensemble attention distillation for privacy-preserving federated learning

    Xuan Gong, Abhishek Sharma, Srikrishna Karanam, Ziyan Wu, Terrence Chen, David Do- ermann, and Arun Innanje. Ensemble attention distillation for privacy-preserving federated learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1507...

    2021

  33. [41]

    Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data

    Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura, and Koji Yamamoto. Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. IEEE Transactions on Mobile Computing, 22(1):191–205, 2021

    2021

  34. [42]

    Fed-dfa: Federated distillation for heterogeneous model fusion through the adversarial lens

    Zichen Wang, Feng Yan, Tianyi Wang, Cong Wang, Yuanchao Shu, Peng Cheng, and Jiming Chen. Fed-dfa: Federated distillation for heterogeneous model fusion through the adversarial lens. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20):21429–21437, Apr. 2025. 12

    2025

  35. [43]

    Dai, and Quoc V

    David Ha, Andrew M. Dai, and Quoc V . Le. Hypernetworks. InInternational Conference on Learning Representations, 2017

    2017

  36. [44]

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    Geoffrey Hinton. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  37. [45]

    Communication-efficient on-device machine learning: Federated distillation and augmen- tation under non-iid private data

    Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Communication-efficient on-device machine learning: Federated distillation and augmen- tation under non-iid private data. arXiv preprint arXiv:1811.11479, 2018

    2018 arXiv

  38. [46]

    Continual learning with hypernetworks

    Johannes von Oswald, Christian Henning, Benjamin F Grewe, and João Sacramento. Continual learning with hypernetworks. In 8th International Conference on Learning Representations (ICLR 2020)(virtual). International Conference on Learning Representations, 2020

    2020

  39. [47]

    Language modeling with recurrent highway hypernetworks

    Joseph Suarez. Language modeling with recurrent highway hypernetworks. Advances in neural information processing systems, 30, 2017

    2017

  40. [48]

    Hyperseg: Patch-wise hypernetwork for real-time semantic segmentation

    Yuval Nirkin, Lior Wolf, and Tal Hassner. Hyperseg: Patch-wise hypernetwork for real-time semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021

    2021

  41. [49]

    Recurrent hypernetworks are surprisingly strong in meta-rl

    Jacob Beck, Risto Vuorio, Zheng Xiong, and Shimon Whiteson. Recurrent hypernetworks are surprisingly strong in meta-rl. Advances in Neural Information Processing Systems, 36, 2024

    2024

  42. [50]

    Hyperdreambooth: Hypernetworks for fast personaliza- tion of text-to-image models

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. Hyperdreambooth: Hypernetworks for fast personaliza- tion of text-to-image models. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2024

  43. [51]

    Personalized federated learning using hypernetworks

    Aviv Shamsian, Aviv Navon, Ethan Fetaya, and Gal Chechik. Personalized federated learning using hypernetworks. In International Conference on Machine Learning, pages 9489–9502. PMLR, 2021

    2021

  44. [52]

    Layer-wise personalized federated learning with hypernetwork

    Suxia Zhu, Tianyu Liu, and Guanglu Sun. Layer-wise personalized federated learning with hypernetwork. Neural Processing Letters, 55(9):12273–12287, 2023

    2023

  45. [53]

    Pefll: Personalized federated learning by learning to learn

    Jonathan A Scott, Hossein Zakerinia, and Christoph Lampert. Pefll: Personalized federated learning by learning to learn. In 12th International Conference on Learning Representations, 2024

    2024

  46. [54]

    Layer-wised model aggregation for personalized federated learning

    Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu. Layer-wised model aggregation for personalized federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10092–10101, 2022

    2022

  47. [55]

    Hypershot: Few-shot learning by kernel hypernetworks

    Marcin Sendera, Marcin Przewiezlikowski, Konrad Karanowski, Maciej Zieba, Jacek Tabor, and Przemyslaw Spurek. Hypershot: Few-shot learning by kernel hypernetworks. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 2468–2477. IEEE Computer Soci...

    2023

  48. [56]

    Federated learning with heterogeneous architectures using graph hypernetworks

    Or Litany, Haggai Maron, David Acuna, Jan Kautz, Gal Chechik, and Sanja Fidler. Federated learning with heterogeneous architectures using graph hypernetworks. ArXiv, abs/2201.08459, 2022

    2022 arXiv

  49. [57]

    FedBN: Federated learning on non-IID features via local batch normalization

    Xiaoxiao Li, Meirui JIANG, Xiaofei Zhang, Michael Kamp, and Qi Dou. FedBN: Federated learning on non-IID features via local batch normalization. In International Conference on Learning Representations, 2021

    2021

  50. [58]

    How does a deep learning model architecture impact its privacy? a comprehensive study of privacy attacks on cnns and transformers

    Guangsheng Zhang, Bo Liu, Huan Tian, Tianqing Zhu, Ming Ding, and Wanlei Zhou. How does a deep learning model architecture impact its privacy? a comprehensive study of privacy attacks on cnns and transformers. In Proceedings of the 33rd USENIX Conference on Security Symposium,...

    2024

  51. [59]

    Depthfl: Depthwise federated learning for heterogeneous clients

    Minjae Kim, Sangyoon Yu, Suhyun Kim, and Soo-Mook Moon. Depthfl: Depthwise federated learning for heterogeneous clients. In The Eleventh International Conference on Learning Representations, 2023. 13

    2023

  52. [60]

    Communication-efficient federated distillation

    Felix Sattler, Arturo Marban, Roman Rischke, and Wojciech Samek. Communication-efficient federated distillation. arXiv preprint arXiv:2012.00632, 2020

    2012 arXiv

  53. [61]

    Fedkd: Communication efficient federated learning via knowledge distillation

    Chuhan Wu, Fangzhao Wu, Ruixuan Liu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. Fedkd: Communication efficient federated learning via knowledge distillation. CoRR, abs/2108.13323, 2021

    2021 arXiv

  54. [62]

    Ensemble distillation for robust model fusion in federated learning

    Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in neural information processing systems, 33:2351–2363, 2020

    2020

  55. [63]

    Parameterized knowledge transfer for personalized federated learning

    Jie Zhang, Song Guo, Xiaosong Ma, Haozhao Wang, Wenchao Xu, and Feijie Wu. Parameterized knowledge transfer for personalized federated learning. Advances in Neural Information Processing Systems, 34:10092–10104, 2021

    2021

  56. [64]

    Dense: Data-free one-shot federated learning

    Jie Zhang, Chen Chen, Bo Li, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chunhua Shen, and Chao Wu. Dense: Data-free one-shot federated learning. Advances in Neural Information Processing Systems, 35:21414–21428, 2022

    2022

  57. [65]

    Enhanc- ing one-shot federated learning through data and ensemble co-boosting

    Rong Dai, Yonggang Zhang, Ang Li, Tongliang Liu, Xun Yang, and Bo Han. Enhanc- ing one-shot federated learning through data and ensemble co-boosting. arXiv preprint arXiv:2402.15070, 2024

    2024 arXiv

  58. [66]

    Fedzkt: Zero-shot knowledge transfer towards resource-constrained federated learning with heterogeneous on-device models

    Lan Zhang, Dapeng Wu, and Xiaoyong Yuan. Fedzkt: Zero-shot knowledge transfer towards resource-constrained federated learning with heterogeneous on-device models. In 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS), pages 928–938. IEEE, 2022

    2022

  59. [67]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020

    2020

  60. [68]

    Dfrd: data- free robustness distillation for heterogeneous federated learning

    Kangyang Luo, Shuai Wang, Yexuan Fu, Xiang Li, Yunshi Lan, and Ming Gao. Dfrd: data- free robustness distillation for heterogeneous federated learning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA, 2...

    2024

  61. [69]

    $i$-divergence geometry of probability distributions and minimization problems

    Imre Csiszár. $i$-divergence geometry of probability distributions and minimization problems. Annals of Probability, 3:146–158, 1975

    1975

  62. [70]

    EMNIST: extending MNIST to handwritten letters

    Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. EMNIST: extending MNIST to handwritten letters. In 2017 International Joint Conference on Neural Networks, IJCNN 2017, Anchorage, AK, USA, May 14-19, 2017, pages 2921–2926. IEEE, 2017

    2017

  63. [71]

    Krizhevsky and G

    A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4), 2009

    2009

  64. [72]

    Tiny imagenet visual recognition challenge

    Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015

    2015

  65. [73]

    Federated learning on non-iid data silos: An experimental study

    Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In IEEE International Conference on Data Engineering, 2022

    2022

  66. [74]

    Fedlpa: One-shot federated learning with layer-wise posterior aggregation

    Xiang Liu, Liangxi Liu, Feiyang Ye, Yunheng Shen, Xia Li, Linshan Jiang, and Jialin Li. Fedlpa: One-shot federated learning with layer-wise posterior aggregation. Advances in Neural Information Processing Systems, 37:81510–81548, 2024

    2024

  67. [75]

    Gradient-based learning applied to document recognition

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998

    1998

  68. [76]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015

    2015

  69. [77]

    Deep residual learning for im- age recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 14

    2016

  70. [78]

    Wide residual networks

    Sergey Zagoruyko. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016

    2016 arXiv

  71. [79]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017

    2017

  72. [80]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014

    2014 arXiv

  73. [81]

    Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size

    Forrest N Iandola. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size. arXiv preprint arXiv:1602.07360, 2016

    2016 arXiv

  74. [82]

    idlg: Improved deep leakage from gradients

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. ArXiv, abs/2001.02610, 2020. 15 Part I Appendix Table of Contents A Algorithms 17 B Additional Experiment Settings 18 B.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....

    2001 arXiv

  75. [2023]

    Association for Computing Machinery

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.