Pith. sign in

REVIEW 5 major objections 5 minor 70 references

Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Low-data federated models freeze their middle layers after early rounds; periodic transient sparsity restarts their updates and improves accuracy.

desk verdict A solid empirical paper that documents layer-wise inertia in low-data FL and offers a simple sparsity fix that works, but the causal story outruns the experiments. read the letter →

arxiv 2506.00932 v1 pith:JT5M72D5 submitted 2025-06-01 cs.LG

Qiao Xiao , Boqian Wu , Andrey Poddubnyy , Elena Mocanu , Phuong H. Nguyen , Mykola Pechenizkiy , Decebal Constantin Mocanu This is my paper
classification cs.LG
keywords federatedlearninglayer-wiseinertiatransientsparsitylow-dataregimenon-IIDdatasensitivity-basedpruningmodelaggregationsparsetraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning is supposed to cure the overfitting caused by tiny local datasets, but the paper claims it actually perpetuates it: in low-data non-IID settings, the middle layers of the global model freeze after the first few communication rounds. The paper names this the Layer-wise Inertia Phenomenon and shows it grows worse with deeper models and less data, while changing the number of clients or the degree of heterogeneity does not fix it. To counter it, the paper proposes LIPS, which every few rounds zeroes out low-sensitivity weights in the middle layers after aggregation, so local training restarts from a sparsified model. In experiments on CIFAR-10, CIFAR-100, and TinyImageNet, LIPS improves accuracy over the baselines, with larger gains precisely where data per client is smaller and the number of clients is larger. If the paper is right, periodically reawakening stale layers is a simple, communication-neutral way to make global aggregation more effective in resource-constrained federated learning.

What carries the argument

The diagnostic machinery is layer-wise cosine similarity of the global model relative to an early round, defined for each layer as $C_l^t = w_l^t \cdot w_l^{t_0} / (\|w_l^t\| \, \|w_l^{t_0}\|)$, where values close to 1 mark stagnation. The intervention machinery is a sensitivity score $s_{i,j}^t = |\Delta w_{i,j}^t \cdot w_{i,j}^t|$, a first-order Taylor approximation of the loss change when a parameter is zeroed out. After global aggregation, every $k$ communication rounds LIPS builds a binary mask that zeroes the lowest-scoring $\tau$ fraction of parameters in each middle layer; the sparsity ratio decays linearly as $\tau(t) = \tau_0(1 - t/T)$, and local training then proceeds densely. Sparsity is applied only to middle layers, excluding the first and last, targeting exactly the layers shown to be inert while preserving the model's full expressive capacity during local updates.

What would settle it

Run FedBN with dropout or weight decay tuned on a validation split of CIFAR-100 at Dir(α=0.1); if it reaches the 47.84 percent accuracy that LIPS reports, the gain is not specific to transient sparsity. Separately, retune $k$ and $\tau_0$ on held-out data and then report test accuracy; if the edge over FedBN disappears, the reported gains were partly fitted to the test sets.

Watch

Extended reading notes

Core claim

The paper's central claim is that in low-data federated learning, the global model's middle layers exhibit Layer-wise Inertia: measured against the round-2 state, their layer-wise cosine similarity stays above 0.95 for most of training, while only the first and last layers keep changing. This stagnation is not resolved by varying client counts or data heterogeneity, worsens in deeper models and with less data per client, and makes later middle-layer aggregation nearly useless: freezing those layers after round 50 changes test accuracy by less than a point. The paper's proposed method LIPS counters the inertia by periodically converting a fraction of middle-layer weights to zero using a sensitivity score, which lowers cosine similarity, raises layer-wise gradient norms, and improves accuracy (for example, CIFAR-100 at Dir(α=0.1) rises from 43.08 with FedBN to 47.84 with LIPS, and TinyImageNet at Dir(α=0.1) rises from 36.83 to 40.61). The paper concludes that transient sparsity is a simple, model-agnostic way to restore meaningful updates to stale layers and make the federated aggregation itself work better under data scarcity.

Load-bearing premise

The load-bearing premise is that LIPS's gains come specifically from reviving stagnant middle layers rather than from generic regularization; this is vulnerable because the paper's appendix argues against dropout but does not run a dropout or weight-decay baseline, and the sparsity hyperparameters are chosen from ablations without a reported held-out validation split.

Editorial extensions

If this is right

  • In low-data non-IID conditions, standard FedAvg leaves the capacity of middle layers largely unused after the first rounds; layer-wise cosine similarity can expose this stagnation.
  • Periodically zeroing the lowest-sensitivity weights in middle layers after aggregation, while keeping local training dense, is enough to reduce that stagnation and improve global accuracy.
  • The benefit of LIPS grows as the regime becomes harder—fewer samples per client and more clients—so it targets precisely where collaboration is weakest.
  • Because sparsification happens after aggregation and does not change what is communicated, LIPS adds no communication overhead and can be dropped into existing federated pipelines.
  • The near-tie between full aggregation and fixing middle layers after round 50 indicates that the later rounds of middle-layer aggregation contribute little, supporting the inertia diagnosis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test the paper leaves open is to compare LIPS against dropout and weight decay at matched regularization strength; if those match its gains, the inertia-specific mechanism would need revision.
  • The layer-wise cosine-similarity diagnostic could be used to schedule sparsity automatically, applying it only while inertia is detected, rather than at a fixed period $k$.
  • Combining transient sparsity with partial personalization methods that keep some layers client-specific might compound the gains, since LIPS already excludes the first and last layers.
  • The phenomenon may extend beyond vision: measuring layer-wise cosine similarity in low-data federated NLP or tabular tasks would show whether middle-layer freezing is a general federated failure mode.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper studies layer-wise update dynamics in low-data federated learning. It defines a Layer-wise Inertia Phenomenon through cosine similarity of global-model weights relative to an early checkpoint, showing that middle layers of VGG and ResNet models stagnate after early communication rounds. To address this, the authors propose LIPS, which periodically zeroes a decaying fraction of low-sensitivity weights in middle layers after aggregation and then trains densely. Experiments on CIFAR-10, CIFAR-100, and TinyImageNet with 100--300 clients report consistent accuracy gains over FedBN and other baselines, and cosine-similarity and gradient-norm plots indicate more active middle-layer updates under LIPS.

Significance. If the phenomenon and the proposed mechanism hold, LIPS is a simple, cheap, and model-agnostic intervention for low-data non-IID FL. The freezing experiment in Table 1 is a clean demonstration that middle-layer aggregation contributes little after the early rounds, and the paper ships code, reports multiple seeds, and covers several datasets, architectures, client counts, and data volumes. The main weakness is causal isolation: the accuracy gains are not yet separated from generic regularization, and the ablation table even shows random selection matching or beating sensitivity selection on one dataset. The significance is therefore conditional on additional control experiments and a validation-based hyperparameter protocol.

major comments (5)
  1. [Section 5.2, Table 4] The proposed sensitivity criterion is not necessary for the reported gains. On CIFAR-100 with alpha=0.1, LIPS-Random reaches 50.14 versus 47.84 for LIPS-Sensitivity, and on alpha=0.01 random selection is 80.77 versus 81.39 for sensitivity. Since Eq. (3) and Appendix B motivate the method as sensitivity-guided, this result directly undermines the claim that the selection criterion is what matters. The authors should report more random-seed repetitions with error bars for the Random and Magnitude variants, and if the finding persists, reframe the contribution as transient sparsity rather than sensitivity-guided selection.
  2. [Appendix I.1 and Section 5] The paper argues that LIPS differs from dropout in motivation and mechanism, but no dropout, weight-decay, or noise-injection baseline is run. Because the central claim is that LIPS repairs layer-wise inertia rather than acting as a generic regularizer, a FedBN + dropout or FedBN + weight-decay control with comparable compute is required. Without such a control, the accuracy gains in Tables 2 and 3 are also consistent with generic regularization.
  3. [Section 5.2, Appendix F, and Appendix A] The hyperparameters tau0 and k are selected from Figure 7(c)-(d), and the final numbers are reported on the same test sets; no validation split is described. Appendix A also states that test accuracy is evaluated in each communication round and the final accuracy is selected, which leaves unclear whether the reported number is the last-round or the best test accuracy. This makes the magnitude of the reported gains difficult to interpret. The authors should select hyperparameters on a validation split or report a sensitivity grid, and clarify whether the reported accuracy is the final-round or best-over-rounds test accuracy.
  4. [Section 3 and Appendix D] The paper attributes layer-wise inertia primarily to overfitting, but no direct overfitting measure is provided, such as local train/test gap, gradient variance, or effective update rank. The centralized-training comparison in Appendix D shows that the phenomenon exists there too, but it does not by itself establish the overfitting mechanism. Since the design of LIPS is motivated by this attribution, either provide a direct measurement of overfitting or soften the causal claim.
  5. [Section 4.3] The method sparsifies only the 'middle layers,' but no ablation compares middle-only sparsification with all-layer or first/last-layer sparsification. The claim that layer-wise targeting is important therefore rests on an untested design choice. Such an ablation is needed to support the proposed mechanism, especially because the paper already notes that the first and last layers are the main drivers of collaboration.
minor comments (5)
  1. [Table 2 caption] The caption says each client holds 100 samples, but Section 5 states that TinyImageNet clients hold 300 training samples; please correct the caption.
  2. [Section 4.3] The term 'middle layers' is not formally defined; please specify exactly which layers of VGG6, ResNet-8, and ResNet-10 are sparsified.
  3. [Algorithm 1] Please clarify when the mask is applied relative to aggregation: the pseudocode computes sparsity at the start of round t, while Section 4.3 says sparsification happens after global aggregation. The two are consistent if one treats the broadcast at the end of round t-1 as the aggregation outcome, but this should be stated explicitly.
  4. [Section 5.2] The text says the ablation uses Dir(alpha=0.1) and Dir(alpha=0.5) for CIFAR-10 and CIFAR-100, respectively, but Table 4 also reports alpha=0.01 for CIFAR-100; please align the text with the table.
  5. [Figure 7(d) caption] The caption reads 'CIFAR-10 (Dir(alpha=0.1) and CIFAR-10 (Dir(alpha=0.01))' but one curve is presumably for CIFAR-100; please check and correct.

Circularity Check

1 steps flagged · score 2.0 of 10

No definitional circularity: the inertia measurement, the sensitivity derivation, and the held-out accuracy gains are independently grounded; the only fitted-input concern is tuning k and tau0 on the same test sets used for the headline comparisons.

  1. fitted input called prediction [Section 5.2 (Effect of frequency k and initial sparsity ratio tau0); Appendix F; Appendix A]
    "Regarding τ0, we adopt τ0 = 0.5 for CIFAR-10 and τ0 = 0.7 for CIFAR-100 and TinyImageNet, as these configurations consistently yield better performance. In each run, we evaluate the uniform averaging test accuracy across all clients in each communication round and select the final accuracy as the final result."

    The headline claim 'LIPS improves overall performance' is supported by Tables 2-3 reporting accuracies for hyperparameters (k=5; tau0=0.5/0.7) selected in Section 5.2 and Appendix F to maximize the same test-accuracy metric on the same CIFAR-10/CIFAR-100/TinyImageNet test sets. Because no validation split is reported and Appendix A additionally takes the best test accuracy over rounds, the reported gain is partly a selection statistic: the configuration that scored best on these test sets is then reported as scoring best on the same test sets. The method itself is not defined in terms of test accuracy, so this is a partial, statistical circularity rather than a definitional one.

full rationale

The derivation chain is self-contained at the definitional level. Inertia is measured by layer-wise cosine similarity (Eq. 2), a metric grounded in external prior work, and the w/. fix experiment (Table 1) is an independent behavioral check showing that freezing middle layers costs nothing; the phenomenon is also reproduced in centralized low-data training (Appendix D). The sensitivity criterion s = |Delta_w · w| (Eq. 3) is derived in Appendix B from the loss-perturbation definition via a first-order Taylor expansion, with the gradient replaced by the local update, which is proportional to the gradient under SGD; no step fits the criterion to the reported test accuracies. The transient-sparsity mask (Eq. 4) and the linear decay of tau are construction choices, and the final accuracies are measured on held-out test data, so the central result is not equal to its inputs by construction. Self-citations (Mocanu et al. 2018, Xiao et al. 2022, Liu et al. 2023, Wu et al. 2025, Nowak et al. 2024) are contextual support for the sparse-training literature and the sensitivity concept; none is load-bearing, none is invoked as a uniqueness theorem, and the sensitivity formula itself is re-derived in-paper rather than imported. The one moderate circularity concern, reflected in the score, is hyperparameter selection: k=5 and tau0 are chosen via test-accuracy ablations on the same benchmarks used for the headline comparisons, with no validation split, so the reported margins are partly an artifact of selection on the target metric. Correctness risks that are not circularity and do not further raise the score: the inertia-repair mechanism is not isolated from generic regularization (no dropout or weight-decay baseline in Appendix I.1, and LIPS-Random beats LIPS-Sensitivity on CIFAR-100 alpha=0.1 in Table 4), and the reduced cosine similarity in Figure 5 is partly a mechanical consequence of zeroing weights.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two fitted hyperparameters (tau0 and k), three domain assumptions about the validity of the sensitivity metric, the interpretation of cosine similarity, and the representativeness of the low-data non-IID benchmarks. No new entities are introduced. The method is a heuristic; its success depends on these assumptions holding across real FL deployments.

free parameters (2)
  • initial sparsity ratio tau0 = 0.5 for CIFAR-10, 0.7 for CIFAR-100 and TinyImageNet
    Selected based on ablation performance across datasets; no separate validation split is reported.
  • sparsification frequency k = 5 communication rounds
    Selected from ablation (Figure 7c) as the best trade-off; no separate validation split is reported.
assumptions (4)
  • domain assumption The first-order Taylor approximation of loss difference, |grad_w L * w|, is a valid sensitivity measure, and the local parameter update Delta w is a valid proxy for the gradient in FL.
    Used in Eq. (3) and Appendix B to select parameters to zero out; the approximation ignores higher-order terms and the relationship between Delta w and grad L is not formally established.
  • domain assumption High layer-wise cosine similarity relative to an early round indicates harmful stagnation rather than benign early convergence.
    Central to diagnosing the Layer-wise Inertia Phenomenon in Section 3; no control shows that low similarity is necessary or sufficient for good FL performance.
  • domain assumption The low-data regime with 100 samples per client and Dirichlet non-IID partitions is representative of real federated low-data settings.
    All main results use this setup; generalization to other domains and scales is left to future work (Appendix J).
  • domain assumption Applying sparsity only to middle layers, excluding first and last, is sufficient to capture the inertia effect.
    Based on the observation that middle layers stagnate; the paper does not ablate applying sparsity to all layers (limitation noted in Appendix J).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity." pith.science (2026). https://pith.science/paper/JT5M72D5

@misc{pith2026250600932,
  author       = {Pith},
  title        = {Pith review of: Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JT5M72D5}},
  note         = {Machine review of arXiv:2506.00932}
}
read the original abstract

Federated learning (FL) enables collaborative model training across decentralized clients while preserving data privacy, leveraging aggregated updates to build robust global models. However, this training paradigm faces significant challenges due to data heterogeneity and limited local datasets, which often impede effective collaboration. In such scenarios, we identify the Layer-wise Inertia Phenomenon in FL, wherein the middle layers of global model undergo minimal updates after early communication rounds, ultimately limiting the effectiveness of global aggregation. We demonstrate the presence of this phenomenon across a wide range of federated settings, spanning diverse datasets and architectures. To address this issue, we propose LIPS (Layer-wise Inertia Phenomenon with Sparsity), a simple yet effective method that periodically introduces transient sparsity to stimulate meaningful updates and empower global aggregation. Experiments demonstrate that LIPS effectively mitigates layer-wise inertia, enhances aggregation effectiveness, and improves overall performance in various FL scenarios. This work not only deepens the understanding of layer-wise learning dynamics in FL but also paves the way for more effective collaboration strategies in resource-constrained environments. Our code is publicly available at: https://github.com/QiaoXiao7282/LIPS.

Figures

Figures reproduced from arXiv: 2506.00932 by the authors.

Figure 1
Figure 1. Layer-wise cosine similarity of global model throughout the training process. We track [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Layer-wise cosine similarity of the global model after aggregation. (a) and (b): Results on [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of the global model’s layer￾wise cosine similarity after training on CIFAR￾10. (a) Varying number of clients and (b) under different data distributions. indicating that the final states of the middle lay￾ers in the global model remain closer to their early states, regardless of the data distribution or the number of clients. These results suggest that increasing client heterogeneity, either by al￾tering t… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Performance comparison across varying numbers [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Layer-wise cosine similarity throughout the training process for LIPS on CIFAR-100 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Layer-wise cosine similarity during the training process. We track the global model’s layer [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Layer-wise cosine similarity during the training process. (a) and (b) show the global model’s [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Layer-wise cosine similarity of models under centralized training throughout the training [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Layer-wise cosine similarity and gradient norm analysis during the training process. (a) [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 49 canonical work pages

  1. [1]

    Federated residual learning

    Alekh Agarwal, John Langford, and Chen-Yu Wei. Federated residual learning. arXiv preprint arXiv:2003.12880, 2020

    arXiv 2003

  2. [2]

    Federated learning with personalization layers

    Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818, 2019

    arXiv 1912

  3. [3]

    Revisiting sparsity hunting in federated learning: Why does sparsity consensus matter? Transactions on Machine Learning Research, 2023

    Sara Babakniya, Souvik Kundu, Saurav Prakash, Yue Niu, and Salman Avestimehr. Revisiting sparsity hunting in federated learning: Why does sparsity consensus matter? Transactions on Machine Learning Research, 2023

    work page 2023

  4. [4]

    Federated dynamic sparse training: Computing less, communicating less, yet learning better

    Sameer Bibikar, Haris Vikalo, Zhangyang Wang, and Xiaohan Chen. Federated dynamic sparse training: Computing less, communicating less, yet learning better. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6080--6088, 2022

    work page 2022

  5. [5]

    Efficient personalized federated learning via sparse model-adaptation

    Daoyuan Chen, Liuyi Yao, Dawei Gao, Bolin Ding, and Yaliang Li. Efficient personalized federated learning via sparse model-adaptation. In International Conference on Machine Learning, pages 5234--5256. PMLR, 2023

    work page 2023

  6. [6]

    Sparsity winning twice: Better robust generalization from more efficient training

    Tianlong Chen, Zhenyu Zhang, pengjun wang, Santosh Balachandra, Haoyu Ma, Zehao Wang, and Zhangyang Wang. Sparsity winning twice: Better robust generalization from more efficient training. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=SYuJXrXq8tw

    work page 2022

  7. [7]

    Streamlining redundant layers to compress large language models

    Xiaodong Chen, Yuxuan Hu, Jing Zhang, Yanling Wang, Cuiping Li, and Hong Chen. Streamlining redundant layers to compress large language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=IC5RJvRoMp

    work page 2025

  8. [8]

    Exploiting shared representations for personalized federated learning

    Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared representations for personalized federated learning. In International conference on machine learning, pages 2089--2099. PMLR, 2021

    work page 2021

Show all 70 references
  1. [9]

    Dispfl: Towards communication-efficient personalized federated learning via decentralized sparse training

    Rong Dai, Li Shen, Fengxiang He, Xinmei Tian, and Dacheng Tao. Dispfl: Towards communication-efficient personalized federated learning via decentralized sparse training. In International Conference on Machine Learning, pages 4587--4604. PMLR, 2022

    2022

  2. [10]

    Flexible clustered federated learning for client-level data distribution shift

    Moming Duan, Duo Liu, Xinyuan Ji, Yu Wu, Liang Liang, Xianzhang Chen, Yujuan Tan, and Ao Ren. Flexible clustered federated learning for client-level data distribution shift. IEEE Transactions on Parallel and Distributed Systems, 33 0 (11): 0 2661--2674, 2021

    2021

  3. [11]

    Rigging the lottery: Making all tickets winners

    Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, and Erich Elsen. Rigging the lottery: Making all tickets winners. In International conference on machine learning, pages 2943--2952. PMLR, 2020

    2020

  4. [12]

    Gradient flow in sparse neural networks and how lottery tickets win

    Utku Evci, Yani Ioannou, Cem Keskin, and Yann Dauphin. Gradient flow in sparse neural networks and how lottery tickets win. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 6577--6586, 2022

    2022

  5. [13]

    An efficient framework for clustered federated learning

    Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, 33: 0 19586--19597, 2020

    2020

  6. [14]

    The unreasonable ineffectiveness of the deeper layers

    Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Dan Roberts. The unreasonable ineffectiveness of the deeper layers. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=ngmEcEer8a

    2025

  7. [15]

    Adaptive gradient sparsification for efficient federated learning: An online learning approach

    Pengchao Han, Shiqiang Wang, and Kin K Leung. Adaptive gradient sparsification for efficient federated learning: An online learning approach. In 2020 IEEE 40th international conference on distributed computing systems (ICDCS), pages 300--310. IEEE, 2020

    2020

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016

    2016

  9. [17]

    Measuring the effects of non-identical data distribution for federated visual classification

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335, 2019

    1909 arXiv

  10. [18]

    Achieving personalized federated learning with sparse local models

    Tiansheng Huang, Shiwei Liu, Li Shen, Fengxiang He, Weiwei Lin, and Dacheng Tao. Achieving personalized federated learning with sparse local models. arXiv preprint arXiv:2201.11380, 2022

    2022 arXiv

  11. [19]

    Federated learning via meta-variational dropout

    Insu Jeon, Minui Hong, Junhyeog Yun, and Gunhee Kim. Federated learning via meta-variational dropout. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=VNyKBipt91

    2023

  12. [20]

    Tracing representation progression: Analyzing and enhancing layer-wise similarity

    Jiachen Jiang, Jinxin Zhou, and Zhihui Zhu. Tracing representation progression: Analyzing and enhancing layer-wise similarity. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=vVxeFSR4fU

    2025

  13. [21]

    Complement sparsification: Low-overhead model pruning for federated learning

    Xiaopeng Jiang and Cristian Borcea. Complement sparsification: Low-overhead model pruning for federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8087--8095, 2023

    2023

  14. [22]

    Improving federated learning personalization via model agnostic meta learning

    Yihan Jiang, Jakub Kone c n \`y , Keith Rush, and Sreeram Kannan. Improving federated learning personalization via model agnostic meta learning. arXiv preprint arXiv:1909.12488, 2019

    1909 arXiv

  15. [23]

    Model pruning enables efficient federated learning on edge devices

    Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K Leung, and Leandros Tassiulas. Model pruning enables efficient federated learning on edge devices. IEEE Transactions on Neural Networks and Learning Systems, 34 0 (12): 0 10374--10386, 2022

    2022

  16. [24]

    Personalized edge intelligence via federated self-knowledge distillation

    Hai Jin, Dongshan Bai, Dezhong Yao, Yutong Dai, Lin Gu, Chen Yu, and Lichao Sun. Personalized edge intelligence via federated self-knowledge distillation. IEEE Transactions on Parallel and Distributed Systems, 34 0 (2): 0 567--580, 2022

    2022

  17. [25]

    Scaffold: Stochastic controlled averaging for federated learning

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pages 5132--5143. PMLR, 2020

    2020

  18. [26]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  19. [27]

    Cifar-10 (canadian institute for advanced research)

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). URL http://www. cs. toronto. edu/kriz/cifar. html, 5 0 (4): 0 1, 2010

    2010

  20. [28]

    Federated lora with sparse communication

    Kevin Kuo, Arian Raje, Kousik Rajesh, and Virginia Smith. Federated lora with sparse communication. arXiv preprint arXiv:2406.05233, 2024

    2024 arXiv

  21. [29]

    Tiny imagenet visual recognition challenge

    Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7 0 (7): 0 3, 2015

    2015

  22. [30]

    Preservation of the global knowledge by not-true distillation in federated learning

    Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae, and Se-Young Yun. Preservation of the global knowledge by not-true distillation in federated learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing S...

    2022

  23. [31]

    Snip: single-shot network pruning based on connection sensitivity

    N Lee, T Ajanthan, and P Torr. Snip: single-shot network pruning based on connection sensitivity. In International Conference on Learning Representations. Open Review, 2019

    2019

  24. [32]

    Lotteryfl: Empower edge intelligence with personalized and communication-efficient federated learning

    Ang Li, Jingwei Sun, Binghui Wang, Lin Duan, Sicheng Li, Yiran Chen, and Hai Li. Lotteryfl: Empower edge intelligence with personalized and communication-efficient federated learning. In 2021 IEEE/ACM Symposium on Edge Computing (SEC), pages 68--79. IEEE, 2021 a

    2021

  25. [33]

    Federated learning on non-iid data silos: An experimental study

    Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th international conference on data engineering (ICDE), pages 965--978. IEEE, 2022

    2022

  26. [34]

    Federated optimization in heterogeneous networks

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2: 0 429--450, 2020

    2020

  27. [35]

    Ditto: Fair and robust federated learning through personalization

    Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In International conference on machine learning, pages 6357--6368. PMLR, 2021 b

    2021

  28. [36]

    Fedbn: Federated learning on non-iid features via local batch normalization

    Xiaoxiao Li, Meirui JIANG, Xiaofei Zhang, Michael Kamp, and Qi Dou. Fedbn: Federated learning on non-iid features via local batch normalization. In International Conference on Learning Representations, 2021 c

    2021

  29. [37]

    Ensemble distillation for robust model fusion in federated learning

    Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in neural information processing systems, 33: 0 2351--2363, 2020

    2020

  30. [38]

    More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity

    Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Tommi Kärkkäinen, Mykola Pechenizkiy, Decebal Constantin Mocanu, and Zhangyang Wang. More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity. In ICLR, 2023. URL https://openreview.net/...

    2023

  31. [39]

    Federated learning for open banking

    Guodong Long, Yue Tan, Jing Jiang, and Chengqi Zhang. Federated learning for open banking. In Federated learning: privacy and incentive, pages 240--254. Springer, 2020

    2020

  32. [40]

    Data-aware gradient compression for fl in communication-constrained mobile computing

    Rongwei Lu, Yutong Jiang, Yinan Mao, Chen Tang, Bin Chen, Laizhong Cui, and Zhi Wang. Data-aware gradient compression for fl in communication-constrained mobile computing. IEEE Transactions on Mobile Computing, 2024

    2024

  33. [41]

    Layer-wised model aggregation for personalized federated learning

    Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu. Layer-wised model aggregation for personalized federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10092--10101, 2022

    2022

  34. [42]

    Three approaches for personalization with applications to federated learning

    Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh. Three approaches for personalization with applications to federated learning. arXiv preprint arXiv:2002.10619, 2020

    2002 arXiv

  35. [43]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273--1282. PMLR, 2017

    2017

  36. [44]

    Federated learning of deep networks using model averaging

    H Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Ag \"u era y Arcas. Federated learning of deep networks using model averaging. arXiv preprint arXiv:1602.05629, 2 0 (2), 2016

    2016 arXiv

  37. [45]

    DOCS : Quantifying weight similarity for deeper insights into large language models

    Zeping Min and Xinshang Wang. DOCS : Quantifying weight similarity for deeper insights into large language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=XBHoaHlGQM

    2025

  38. [46]

    Linear convergence in federated learning: Tackling client heterogeneity and sparse gradients

    Aritra Mitra, Rayana Jaafar, George J Pappas, and Hamed Hassani. Linear convergence in federated learning: Tackling client heterogeneity and sparse gradients. Advances in Neural Information Processing Systems, 34: 0 14606--14619, 2021

    2021

  39. [47]

    Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science

    Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H Nguyen, Madeleine Gibescu, and Antonio Liotta. Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nature communications, 9 0 (1): 0 2383, 2018

    2018

  40. [48]

    Federated learning for smart healthcare: A survey

    Dinh C Nguyen, Quoc-Viet Pham, Pubudu N Pathirana, Ming Ding, Aruna Seneviratne, Zihuai Lin, Octavia Dobre, and Won-Joo Hwang. Federated learning for smart healthcare: A survey. ACM Computing Surveys (Csur), 55 0 (3): 0 1--37, 2022

    2022

  41. [49]

    Fantastic weights and how to find them: Where to prune in dynamic sparse training

    Aleksandra Nowak, Bram Grooten, Decebal Constantin Mocanu, and Jacek Tabor. Fantastic weights and how to find them: Where to prune in dynamic sparse training. Advances in Neural Information Processing Systems, 36, 2024

    2024

  42. [50]

    The future of digital health with federated learning

    Nicola Rieke, Jonny Hancox, Wenqi Li, Fausto Milletari, Holger R Roth, Shadi Albarqouni, Spyridon Bakas, Mathieu N Galtier, Bennett A Landman, Klaus Maier-Hein, et al. The future of digital health with federated learning. NPJ digital medicine, 3 0 (1): 0 1--7, 2020

    2020

  43. [51]

    Privacy-first health research with federated learning

    Adam Sadilek, Luyang Liu, Dung Nguyen, Methun Kamruzzaman, Stylianos Serghiou, Benjamin Rader, Alex Ingerman, Stefan Mellem, Peter Kairouz, Elaine O Nsoesie, et al. Privacy-first health research with federated learning. NPJ digital medicine, 4 0 (1): 0 132, 2021

    2021

  44. [52]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR , 2015

    2015

  45. [53]

    Federated reconstruction: Partially local federated learning

    Karan Singhal, Hakim Sidahmed, Zachary Garrett, Shanshan Wu, John Rush, and Sushant Prakash. Federated reconstruction: Partially local federated learning. Advances in Neural Information Processing Systems, 34: 0 11220--11232, 2021

    2021

  46. [54]

    Federated multi-task learning

    Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. Advances in neural information processing systems, 30, 2017

    2017

  47. [55]

    Fedselect: Personalized federated learning with customized selection of parameters for fine-tuning

    Rishub Tamirisa, Chulin Xie, Wenxuan Bao, Andy Zhou, Ron Arel, and Aviv Shamsian. Fedselect: Personalized federated learning with customized selection of parameters for fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 239...

    2024

  48. [56]

    Towards personalized federated learning

    Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE transactions on neural networks and learning systems, 34 0 (12): 0 9587--9603, 2022

    2022

  49. [57]

    Fedgen: Generalizable federated learning for sequential data

    Praveen Venkateswaran, Vatche Isahagian, Vinod Muthusamy, and Nalini Venkatasubramanian. Fedgen: Generalizable federated learning for sequential data. In 2023 IEEE 16th International Conference on Cloud Computing (CLOUD), pages 308--318. IEEE, 2023

    2023

  50. [58]

    Gradient sparsification for communication-efficient distributed optimization

    Jianqiao Wangni, Jialei Wang, Ji Liu, and Tong Zhang. Gradient sparsification for communication-efficient distributed optimization. Advances in Neural Information Processing Systems, 31, 2018

    2018

  51. [59]

    Federated dropout—a simple approach for enabling federated learning on resource constrained devices

    Dingzhu Wen, Ki-Jun Jeon, and Kaibin Huang. Federated dropout—a simple approach for enabling federated learning on resource constrained devices. IEEE wireless communications letters, 11 0 (5): 0 923--927, 2022

    2022

  52. [60]

    A survey on federated learning: challenges and applications

    Jie Wen, Zhixia Zhang, Yang Lan, Zhihua Cui, Jianghui Cai, and Wensheng Zhang. A survey on federated learning: challenges and applications. International Journal of Machine Learning and Cybernetics, 14 0 (2): 0 513--535, 2023

    2023

  53. [61]

    Dynamic sparse training versus dense training: The unexpected winner in image corruption robustness

    Boqian Wu, Qiao Xiao, Shunxin Wang, Nicola Strisciuglio, Mykola Pechenizkiy, Maurice van Keulen, Decebal Constantin Mocanu, and Elena Mocanu. Dynamic sparse training versus dense training: The unexpected winner in image corruption robustness. In The Thirteenth International Co...

    2025

  54. [62]

    Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration

    Xinghao Wu, Xuefeng Liu, Jianwei Niu, Guogang Zhu, and Shaojie Tang. Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 193...

    2023

  55. [63]

    Dynamic sparse network for time series classification: Learning what to see

    Qiao Xiao, Boqian Wu, Yu Zhang, Shiwei Liu, Mykola Pechenizkiy, Elena Mocanu, and Decebal Constantin Mocanu. Dynamic sparse network for time series classification: Learning what to see . In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in ...

    2022

  56. [64]

    Federated machine learning: Concept and applications

    Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10 0 (2): 0 1--19, 2019

    2019

  57. [65]

    Fedmix: Approximation of mixup under mean augmented federated learning

    Tehrim Yoon, Sumin Shin, Sung Ju Hwang, and Eunho Yang. Fedmix: Approximation of mixup under mean augmented federated learning. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=Ogga20D2HO-

    2021

  58. [66]

    Salvaging federated learning by local adaptation

    Tao Yu, Eugene Bagdasaryan, and Vitaly Shmatikov. Salvaging federated learning by local adaptation. arXiv preprint arXiv:2002.04758, 2020

    2002 arXiv

  59. [67]

    What do we mean by generalization in federated learning? In International Conference on Learning Representations, 2022

    Honglin Yuan, Warren Richard Morningstar, Lin Ning, and Karan Singhal. What do we mean by generalization in federated learning? In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=VimqQq-i_Q

    2022

  60. [68]

    Fedala: Adaptive local aggregation for personalized federated learning

    Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11237--11244, 2023

    2023

  61. [69]

    Parameterized knowledge transfer for personalized federated learning

    Jie Zhang, Song Guo, Xiaosong Ma, Haozhao Wang, Wenchao Xu, and Feijie Wu. Parameterized knowledge transfer for personalized federated learning. Advances in Neural Information Processing Systems, 34: 0 10092--10104, 2021

    2021

  62. [70]

    Federated learning with non-iid data

    Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582, 2018

    2018 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.