REVIEW 23 cited by
FedMD: Heterogenous Federated Learning via Model Distillation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
FedMD: Heterogenous Federated Learning via Model Distillation
read the original abstract
Federated learning enables the creation of a powerful centralized model without compromising data privacy of multiple participants. While successful, it does not incorporate the case where each participant independently designs its own model. Due to intellectual property concerns and heterogeneous nature of tasks and data, this is a widespread requirement in applications of federated learning to areas such as health care and AI as a service. In this work, we use transfer learning and knowledge distillation to develop a universal framework that enables federated learning when each agent owns not only their private data, but also uniquely designed models. We test our framework on the MNIST/FEMNIST dataset and the CIFAR10/CIFAR100 dataset and observe fast improvement across all participating models. With 10 distinct participants, the final test accuracy of each model on average receives a 20% gain on top of what's possible without collaboration and is only a few percent lower than the performance each model would have obtained if all private datasets were pooled and made directly available for all participants.
Forward citations
Cited by 23 Pith papers
-
TallyTrain: Communication-Efficient Federated Distillation
TallyTrain is a hard-label distillation protocol for federated learning that uses argmax transmission and optional sparse merges to match soft-label performance at up to 1000x lower communication cost.
-
Quantifying and Defending against the Privacy Risk in Logit-based Federated Learning
Logit-based federated learning leaks private model information to a semi-honest server via shared logits even with unrelated public data, enabling an adaptive stealing attack with theoretical bounds and a logit-pertur...
-
FedQHD: Closed-Form Function-Space Federated Reinforcement Learning
FedQHD achieves closed-form federated Q-learning via hyperdimensional encoders with linear readouts, formalizes the federation gap under heterogeneous encoders, and reports competitive performance on continuous-state ...
-
From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learning
FedSAF shifts prototype alignment in heterogeneous federated learning from coordinate matching to inter-class structural relations and reports up to 3.52% gains over prior methods.
-
Learned Neighbor Trust for Collaborative Deployment in Model-Agnostic Decentralized Learning
LNTrust has nodes learn compact trust functions from validation evidence that both guide training distillation and define deployment ensembles, yielding higher accuracy with less communication than prior output-only b...
-
Tuning-Free Efficient Estimation for Multi-Source Data via Covariance-Aware Shrinkage
Proposes a covariance-aware tuning-free shrinkage framework and sequential algorithm for multi-source estimation that attains oracle risk asymptotically and improves on single-step methods.
-
Efficient Federated Estimation and Inference for High-Dimensional Tail Index Regression
Proposes a sparsity-plus-fusion-penalty federated estimator for tail index regression with non-asymptotic rates, oracle group recovery, ADMM solver, and debiased inference that improves on target-only methods.
-
Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning
Q-LocalAdam reduces optimizer memory by 3.37x via tailored 8-bit quantization for Adam states while maintaining or improving accuracy under data heterogeneity in edge federated learning.
-
On What We Can Learn from Low-Resolution Data
Low-resolution data improves high-resolution model performance when high-resolution samples are limited, via KL-divergence bounds and experiments on vision transformers and CNNs.
-
COSMOS: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communication
COSMOS performs model-agnostic personalized federated learning via server-side clustering on pseudo-label predictions and distillation of cluster models, claiming exponential personalization risk contraction.
-
COSMOS: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communication
COSMOS clusters clients via pseudo-label predictions on public data, trains cluster-specific server models, and distills them to clients, claiming exponential personalization risk contraction and outperforming model-a...
-
Function-Space ADMM for Decentralized Federated Learning: A Control Theoretic Perspective
FedF-ADMM uses function-space ADMM updates projected via knowledge distillation plus a PI-like stabilization term to deliver faster, more stable convergence and higher accuracy than prior decentralized FL methods unde...
-
HARMONY: Bridging the Personalization-Generalization Gap by Mitigating Representation Skew in Heterogeneous Split Federated Learning
HARMONY mitigates representation skew in heterogeneous hybrid split federated learning via meta-learning to simulate diverse extractors and server-side contrastive learning to align features, delivering up to 43% accu...
-
When To Adapt? Adapting the Model or Data in Federated Medical Imaging
Harmonization works better than personalization for appearance-based domain shifts in federated medical imaging while personalization is superior for structural shifts, with both performing similarly when shifts are small.
-
Collate: Collaborative Neural Network Learning for Latency-Critical Edge Systems
Collate jointly trains heterogeneous models under per-device latency constraints via dynamic zeroizing-recovering and proto-corrected aggregation, gaining ~2–3% accuracy over prior heterogeneous FL.
-
Asynchronous Decentralized Federated Learning over Lossy Wireless Links via Reception- and Age-Aware Aggregation
DFL-AA removes link-quality distortion in expectation from gossip aggregation in asynchronous decentralized federated learning by combining online IPW for selection bias correction with AoI weighting for staleness mitigation.
-
HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning
HASA computes client heterogeneity scores from local data and assigns wider subnets to less heterogeneous clients, raising mean client test accuracy from 13.82% to 14.32% and improving worst-client accuracy versus uni...
-
BiFedKD: Bidirectional Federated Knowledge Distillation Framework for Non-IID and Long-Tailed ECG Monitoring
BiFedKD improves ECG classification accuracy by 3.52% and Macro-F1 by 9.93% on MIT-BIH while cutting communication overhead 40% and computation cost 71.7% versus baseline federated methods.
-
FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion
FedProxy replaces weak adapters with a proxy SLM for federated LLM fine-tuning, outperforming prior methods and approaching centralized performance via compression, heterogeneity-aware aggregation, and training-free fusion.
-
Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning
FedBB addresses inter-case, inter-class, and inter-client imbalances in federated learning via Positive Negative Balanced loss and Client Balanced Reweighting, outperforming baselines on X-ray and natural image datase...
-
FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
FedMTFI clusters heterogeneous clients, trains cluster prototypes, and applies multi-teacher distillation with SHAP to improve accuracy over standard FL in non-IID settings.
-
Federated Knowledge Distillation for Multi-Model Architectures Lithography Hotspot Detection
FedKD-hybrid is a hybrid federated knowledge distillation framework for multi-model lithography hotspot detection that outperforms prior methods on ICCAD-2012 and real-world FAB datasets.
-
Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions
A survey organizing knowledge distillation techniques for addressing privacy, heterogeneity, communication, and personalization challenges in federated learning.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.