REVIEW 14 cited by
Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
read the original abstract
On-device machine learning (ML) enables the training process to exploit a massive amount of user-generated private data samples. To enjoy this benefit, inter-device communication overhead should be minimized. With this end, we propose federated distillation (FD), a distributed model training algorithm whose communication payload size is much smaller than a benchmark scheme, federated learning (FL), particularly when the model size is large. Moreover, user-generated data samples are likely to become non-IID across devices, which commonly degrades the performance compared to the case with an IID dataset. To cope with this, we propose federated augmentation (FAug), where each device collectively trains a generative model, and thereby augments its local data towards yielding an IID dataset. Empirical studies demonstrate that FD with FAug yields around 26x less communication overhead while achieving 95-98% test accuracy compared to FL.
Forward citations
Cited by 14 Pith papers
-
TallyTrain: Communication-Efficient Federated Distillation
TallyTrain is a hard-label distillation protocol for federated learning that uses argmax transmission and optional sparse merges to match soft-label performance at up to 1000x lower communication cost.
-
Quantifying and Defending against the Privacy Risk in Logit-based Federated Learning
Logit-based federated learning leaks private model information to a semi-honest server via shared logits even with unrelated public data, enabling an adaptive stealing attack with theoretical bounds and a logit-pertur...
-
From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learning
FedSAF shifts prototype alignment in heterogeneous federated learning from coordinate matching to inter-class structural relations and reports up to 3.52% gains over prior methods.
-
Enhancing Visual Representation with Textual Semantics: Textual Semantics-Powered Prototypes for Heterogeneous Federated Learning
FedTSP builds class prototypes from LLM-generated text descriptions via PLMs and trainable prompts to preserve semantic relationships and reduce heterogeneity effects in federated learning.
-
Function-Space ADMM for Decentralized Federated Learning: A Control Theoretic Perspective
FedF-ADMM uses function-space ADMM updates projected via knowledge distillation plus a PI-like stabilization term to deliver faster, more stable convergence and higher accuracy than prior decentralized FL methods unde...
-
Collate: Collaborative Neural Network Learning for Latency-Critical Edge Systems
Collate jointly trains heterogeneous models under per-device latency constraints via dynamic zeroizing-recovering and proto-corrected aggregation, gaining ~2–3% accuracy over prior heterogeneous FL.
-
Optimized Federated Knowledge Distillation with Distributed Neural Architecture Search
FedKDNAS combines client-side neural architecture search with knowledge distillation from aggregated server predictions to improve accuracy and efficiency in heterogeneous federated learning.
-
BiFedKD: Bidirectional Federated Knowledge Distillation Framework for Non-IID and Long-Tailed ECG Monitoring
BiFedKD improves ECG classification accuracy by 3.52% and Macro-F1 by 9.93% on MIT-BIH while cutting communication overhead 40% and computation cost 71.7% versus baseline federated methods.
-
PubSwap: Public-Data Off-Policy Coordination for Federated RLVR
PubSwap uses a small public dataset for selective off-policy response swapping in federated RLVR to improve coordination and performance over standard baselines on math and medical reasoning tasks.
-
Federated Distillation on Edge Devices: Efficient Client-Side Filtering for Non-IID Data
EdgeFD uses a KMeans-based client-side filter to improve federated distillation accuracy close to IID levels on non-IID data distributions for resource-constrained edge devices.
-
Multi-hop Federated Private Data Augmentation with Sample Compression
MultFAug combines multi-hop relaying and sample compression in federated settings to enhance privacy guarantees, cut transmission delay, and raise local training performance on non-IID data.
-
Internet of Everything in the 6G Era: Paradigms, Enablers, Potentials and Future Directions
The paper provides a structured overview of IoE concepts, components, architectures, enabling technologies, challenges, and open research directions for 6G-enabled systems.
-
Internet of Everything in the 6G Era: Paradigms, Enablers, Potentials and Future Directions
The paper provides a structured overview of IoE concepts, components, architectures, enabling technologies, challenges, and open research directions for 6G-enabled IoE systems.
-
Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions
A survey organizing knowledge distillation techniques for addressing privacy, heterogeneity, communication, and personalization challenges in federated learning.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.