Pith. sign in

REVIEW 16 cited by

Distilled One-Shot Federated Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2009.07999 v3 pith:LGMZ7T23 submitted 2020-09-17 cs.LG cs.AIstat.ML

Yanlin Zhou , George Pu , Xiyao Ma , Xiaolin Li , Dapeng Wu This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords modelcommunicationdatadistilleddosfllearningcostfederated
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Current federated learning algorithms take tens of communication rounds transmitting unwieldy model weights under ideal circumstances and hundreds when data is poorly distributed. Inspired by recent work on dataset distillation and distributed one-shot learning, we propose Distilled One-Shot Federated Learning (DOSFL) to significantly reduce the communication cost while achieving comparable performance. In just one round, each client distills their private dataset, sends the synthetic data (e.g. images or sentences) to the server, and collectively trains a global model. The distilled data look like noise and are only useful to the specific model weights, i.e., become useless after the model updates. With this weight-less and gradient-less design, the total communication cost of DOSFL is up to three orders of magnitude less than FedAvg while preserving between 93% to 99% performance of a centralized counterpart. Afterwards, clients could switch to traditional methods such as FedAvg to finetune the last few percent to fit personalized local models with local datasets. Through comprehensive experiments, we show the accuracy and communication performance of DOSFL on both vision and language tasks with different models including CNN, LSTM, Transformer, etc. We demonstrate that an eavesdropping attacker cannot properly train a good model using the leaked distilled data, without knowing the initial model weights. DOSFL serves as an inexpensive method to quickly converge on a performant pre-trained model with less than 0.1% communication cost of traditional methods.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dataset Distillation Efficiently Encodes Low-Dimensional Representations from Gradient-Based Learning of Non-Linear Tasks

    cs.LG 2026-03 accept novelty 7.0 of 10

    Gradient-based dataset distillation of two-layer ReLU nets on multi-index models encodes the r-dimensional principal subspace into synthetic data of memory complexity Θ̃(r²d+L) that recovers high generalization.

  2. CRIP: Channel Level Representation Injection for Personalized One-Shot Federated Learning

    cs.LG 2026-08 conditional novelty 6.0 of 10

    CRIP improves one-shot federated learning by matching and injecting only the most compatible feature channels from other clients' models, without extra training or public data.

  3. A New One-Shot Federated Learning Framework for Medical Imaging Classification with Feature-Guided Rectified Flow and Knowledge Distillation

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Feature-level rectified flow generation plus dual-layer knowledge distillation yields a one-shot federated learning method that beats several baselines on three non-IID medical imaging datasets.

  4. FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices

    cs.LG 2025-02 conditional novelty 6.0 of 10

    FedMHO is a hybrid one-shot federated learning framework where resource-sufficient clients contribute deep classifiers and resource-constrained clients contribute lightweight generative models, fused on the server int...

  5. One-shot Federated Learning via Synthetic Distiller-Distillate Communication

    cs.LG 2024-12 conditional novelty 6.0 of 10

    FedSD2C beats prior one-shot federated learning baselines on ImageNette, Tiny-ImageNet, and OpenImage by sending compact latent codes of selected, Fourier-perturbed images instead of local models.

  6. Tackling Data Heterogeneity in Federated Time Series Forecasting

    cs.LG 2024-11 conditional novelty 6.0 of 10

    Fed-TREND uses server-generated synthetic time series data, distilled from client and global model updates, to reduce the accuracy gap in heterogeneous federated forecasting.

  7. Personalized One-shot Federated Graph Learning for Heterogeneous Clients

    cs.LG 2024-11 conditional novelty 6.0 of 10

    O-pFGL achieves one-shot personalized federated graph learning by aggregating class-wise feature statistics into a surrogate graph and combining global distillation with local fine-tuning, outperforming baselines on 1...

  8. FedBiCross: Personalized One-Shot Federated Learning on Medical Images

    cs.LG 2026-01 unverdicted novelty 5.0 of 10

    FedBiCross clusters clients by model similarity, uses bi-level cross-cluster optimization for adaptive knowledge transfer, and applies personalized distillation to outperform baselines in non-IID data-free one-shot fe...

  9. Efficient Federated Learning with Timely Update Dissemination

    cs.DC 2025-07 conditional novelty 5.0 of 10

    FedASMU and FedSSMU improve federated learning accuracy and speed by dynamically disseminating fresh global models to devices during local training, using server-side and device-side adaptive weighting.

  10. On Learning Representations for Tabular Data Distillation

    cs.LG 2025-01 conditional novelty 5.0 of 10

    Distilling tabular data in a learned autoencoder latent space improves downstream model performance compared to distilling the raw features, especially for clustering-based distillation methods.

  11. Task Arithmetic Through The Lens Of One-Shot Federated Learning

    cs.LG 2024-11 conditional novelty 5.0 of 10

    Task arithmetic is exactly one-shot FedAvg with outer step size beta = lambda T, and FedNova, FedGMA, Median, and CCLIP can often improve merged model performance.

  12. Federated Gaussian Mixture Models

    cs.LG 2025-06 conditional novelty 4.0 of 10

    FedGenGMM aggregates client-trained Gaussian mixture models in a single communication round by sampling synthetic data from the reweighted client components and retraining a global GMM, matching non-federated and iter...

  13. Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks

    cs.LG 2025-04 conditional novelty 4.0 of 10

    The paper organizes large-small model collaboration into downward, upward, and inference-time transfer, and advocates multi-objective benchmarks for private-domain tasks.

  14. Regularizing Differentiable Architecture Search with Smooth Activation

    cs.NE 2025-04 conditional novelty 4.0 of 10

    SA-DARTS regularizes DARTS by penalizing the mean of architecture weights, reporting results on benchmarks that are similar to, but not clearly better than, existing methods.

  15. One-shot Federated Learning Methods: A Practical Guide

    cs.LG 2025-02 conditional novelty 3.0 of 10

    A survey that organizes one-shot federated learning methods into four categories (parameter learning, knowledge distillation, generative models, ensemble methods) and outlines future research directions.

  16. The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions

    cs.CV 2025-02 conditional novelty 3.0 of 10

    A 2023-2025 survey of dataset distillation that organizes matching, generative, decoupling, and selective methods and tabulates ImageNet-scale accuracy comparisons.

Pith tools