Pith. sign in

REVIEW 4 major objections 6 minor 46 references

FedTLU: Federated Learning with Targeted Layer Updates

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FedTLU claims that a server-side score identifies the most critical blocks and that updating only those blocks improves federated fine-tuning of language models under non-IID data.

desk verdict Plausible new server-side layer-selection heuristic, but Section IV.C's proof is invalid and the empirical gains are too small and under-powered to support the central claim. read the letter →

arxiv 2412.17692 v2 pith:I2VRNJ2J submitted 2024-12-23 cs.LG cs.AIcs.DC

Jong-Ik Park , Carlee Joe-Wong This is my paper · ORCID
classification cs.LGcs.AIcs.DC
keywords federatedlearninglayer-wiseupdatetargetedfine-tuningnon-IIDdatalanguagemodelsserver-sidescoringnoisyclientsrobustaggregation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedTLU proposes that in federated fine-tuning of language models, the server should freeze most layers and update only a small set of blocks chosen by a per-layer score, rather than updating all layers or choosing layers at random. The paper's central claim is that this targeted update improves convergence and test perplexity, particularly when clients have non-IID data or when some clients send noisy or malicious updates. The method is attractive because the score is computed entirely on the server from the aggregated weight changes, so clients need no extra computation or information sharing. If the claim holds, federated fine-tuning becomes more robust near a stationary point, where noisy client updates otherwise dominate.

What carries the argument

The load-bearing object is the per-layer score $\text{Score}(W_i) = \frac{\|\Delta W_i\|}{\sqrt{n_i}\,\mathrm{std}(\Delta W_i)}$ together with the block-selection rule. The numerator rewards layers whose aggregated weights moved a lot, the standard deviation in the denominator penalizes layers whose parameter updates are scattered, and $\sqrt{n_i}$ normalizes across layers of different sizes. Blocks with identical parameter counts are grouped and the top $S$ blocks per group are updated, making the score a cheap client-agnostic mask over the model. The theoretical support rests on a smoothness bound and Assumption (b), that the selected subset's gradient retains all but $\delta$ of the full gradient's squared norm.

What would settle it

Run federated fine-tuning where the global model is known to have task-relevant layers that move little (low score) and noise-dominated layers that move a lot; if FedTLU consistently selects the noisy layers and underperforms random selection, the score's alignment assumption is false. More directly, compute the ratio of the selected subset's squared gradient norm to the full model's squared gradient norm over rounds: if it is often much smaller than $1-\delta$ for the claimed $\delta$, the bound in Section IV does not apply.

Watch

Extended reading notes

Core claim

The paper introduces a server-side layer score, $\text{Score}(W_i) = \frac{\|\Delta W_i\|}{\sqrt{n_i}\,\mathrm{std}(\Delta W_i)}$, where $\Delta W_i$ is the aggregated change in layer $i$'s weights in a round. Large aggregate movement with low parameter-level variance marks a layer as both influential and consistent, and layers are grouped into blocks with equal parameter counts so the top $S$ blocks per group can be compared fairly. The core discovery is that updating only those top-scored blocks after aggregation, leaving the rest frozen, yields lower global and local test perplexity than full updates, random block selection, or last-layer-only updates in experiments on Transformer and GPT-2 models under non-IID client splits. The paper also reports that this selectivity blocks noisy or label-shuffled clients from injecting bad updates, with gains up to 7.86% global and 8.27% local perplexity improvements over random selection in those settings. The theoretical analysis argues that when the loss is $L$-smooth and the learning rate satisfies $\eta L \ge 2$, updating a well-aligned subset can reduce loss more than a full update.

Load-bearing premise

The argument depends on the score selecting blocks whose gradient is nearly as effective at reducing global loss as the full gradient; the paper assumes this alignment rather than proving that its score produces such a subset.

Editorial extensions

If this is right

  • FedTLU is compatible with existing aggregation rules such as FedAvg and FedProx; the score is computed after aggregation and changes only which blocks overwrite the global model.
  • Under non-IID client splits, updating 75%, 50%, or 25% of blocks with FedTLU consistently reaches lower minimum perplexity than updating the same portion at random, in both from-scratch and fine-tuning settings.
  • With 10% of clients given shuffled labels, FedTLU keeps global perplexity below full and random update strategies on GPT-2/UDPOS, where random selection actually worsens perplexity.
  • Because the same number of layers is updated as in random or last-layer baselines, the gains are attributed to where updates are placed, not to a smaller update budget.
  • Near stationary points, the paper's bound implies that a well-chosen subset can dominate the full-model update whenever $\eta L \ge 2$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The score is a pure function of aggregated weight movements, so a natural extension is to use it as a drop-in mask for parameter-efficient fine-tuning methods such as adapters or LoRA in FL, where the server could select which modules to aggregate.
  • The $\eta L \ge 2$ condition suggests the benefit should be most visible in late-stage fine-tuning with relatively large learning rates or poorly conditioned losses; a direct test would vary the learning rate and measure the gap between FedTLU and full updates.
  • If the score truly captures gradient usefulness, similar rankings should correlate with per-layer Fisher information or gradient-norm contributions in centralized settings; checking that correlation is a concrete way to test the mechanism outside federated learning.
  • The noisy-client experiments use label shuffling; whether the method filters other poisoning patterns, such as targeted backdoors concentrated in low-scoring layers, is an open question that the paper does not address.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FedTLU, a server-side targeted layer update strategy for federated fine-tuning of language models. In each aggregation round, the server computes a per-layer score Score(W_i) = ||ΔW_i|| / (√n_i · std(ΔW_i)) from the difference between the pre-round global model and the aggregated client updates, sums these scores within blocks, groups blocks by parameter sequence, and updates only the top S blocks per group while freezing all other parameters. The authors claim this improves convergence and test perplexity compared with full, random, and last-layer updates in non-IID settings, and that it is robust to noisy or malicious clients. They provide a theoretical analysis in Section IV intended to show that subset updates can outperform full updates, and they report three-repetition experiments on a standard Transformer (Penn Treebank) and GPT-2 (UDPOS) with FedAvg and FedProx, covering training from scratch, fine-tuning, and noisy-client scenarios.

Significance. If the central claim were established, FedTLU would be a simple, client-agnostic, server-side mechanism for reducing the impact of noisy updates during federated fine-tuning, while also offering communication savings by updating only part of the model. The score is cheap to compute and requires no extra client-side information, which is an attractive practical property. The paper also contains a concrete experimental comparison across several update strategies and datasets. However, the current significance is limited by two serious problems: the theoretical argument in Section IV is invalid as written, and the empirical evidence is underpowered (three repetitions, no error bars) and even contains a direct counterexample to the 'consistently outperforms random' claim in Table III. The idea may still be viable as an empirical contribution, but the paper needs substantial revision of its theory and claims.

major comments (4)
  1. [Section IV.C] The comparison of lower bounds in Section IV.C is logically invalid. From L-smoothness the authors derive Δ_all ≥ ηA(1 − ηL/2) and Δ_S ≥ η(A − δ)(1 − ηL/2), where A = ||∇L(W(t))||², and then treat the fact that the subset lower bound is larger than the full lower bound as proof that the subset update yields greater loss reduction, concluding ηL ≥ 2. A larger lower bound on one quantity does not imply that the quantity itself is larger. Moreover, for ηL > 2 both lower bounds are negative, making the inequalities vacuous, and ηL ≥ 2 is precisely the regime in which gradient descent on an L-smooth nonconvex function is not guaranteed to decrease the loss. This section does not establish that subset updates improve convergence and should be removed or replaced with a valid argument.
  2. [Section IV.A, Assumption (b)] The theoretical analysis relies on Assumption (b), which states that the selected subset W_S satisfies ||∇_{W_S} L(W)||² ≥ ||∇L(W)||² − δ. This is essentially the property that FedTLU's score is supposed to provide, but the paper never proves that the Score defined in Section III selects layers whose gradient is aligned with the full gradient within a slack δ. The score is based on the L2 norm of parameter differences and their standard deviation, not on gradient alignment. Consequently, even if Section IV.C were corrected, it would apply to an oracle subset selector and not to FedTLU. The connection between the score and Assumption (b) needs to be established, or the theory should be reframed as an empirical heuristic with no formal convergence guarantee.
  3. [Table III and Section V] Table III (Transformer, FedAvg, noisy/malicious clients) reports FedTLU global perplexity 1.768 versus Random 1.764, and local perplexity 1.765 versus 1.763. That is, Random outperforms FedTLU in this configuration. This directly contradicts the abstract's claim that FedTLU 'consistently outperforms random and last-layer updates' and the contribution bullet in Section I. While Section V acknowledges the negative relative improvement in the text (e.g., −0.23% globally), the abstract and contribution statements remain unqualified. The claim of consistent improvement should be removed or replaced with a precise statement of where FedTLU helps and where it does not.
  4. [Section V.A and Tables I–III] All experiments are evaluated only three times and the paper reports only averaged minimum test perplexities, without standard deviations, confidence intervals, or per-seed values. Many reported differences are extremely small (for example, Table I, Transformer, FedAvg, portion 0.75: 2.107 versus 2.108 for FedTLU versus Random), so without variance information these differences cannot be distinguished from noise. The authors should provide error bars or standard deviations for the main comparisons, and ideally a paired statistical test over seeds, before claiming empirical superiority, especially for the from-scratch and fine-tuning experiments.
minor comments (6)
  1. [Section III] The score formula Score(W_i) = ||ΔW_i|| / (√n_i · std(ΔW_i)) is undefined when std(ΔW_i) = 0, which can occur for a frozen or unselected layer; the paper should specify how this case is handled.
  2. [Section III and Algorithm 1] The description of block grouping is ambiguous: the text says blocks are grouped 'by the sequence of parameters' and Algorithm 1 says 'Group blocks by the sequence of parameters,' but what this means operationally is not formally defined. Please clarify the grouping criterion and how it relates to equal parameter counts.
  3. [Section V.A] The paper reports 'averaged minimum global and local test perplexities' but does not define whether the minimum is taken over training rounds for each seed and then averaged, or computed on the averaged curve. This should be stated precisely because it affects the interpretation of the results.
  4. [Figures 1 and 2] The perplexity curves in Figures 1 and 2 show only single trajectories with no indication of variability across the three repetitions; adding error bars or shaded regions would make the figures more informative.
  5. [Abstract and Section I] The phrase 'extensive experiments' overstates the scope: the evaluation uses two model architectures and two datasets. Please temper the wording to match the actual experimental setup.
  6. [References] Several references are incomplete or inconsistently formatted (e.g., [8] gives a proceedings venue but no page numbers, and [3] is cited as an OpenAI blog post without a URL). Please check all references against a consistent style.

Circularity Check

2 steps flagged · score 6.0 of 10

Theoretical support for FedTLU presupposes the effectiveness of the score: Assumption (b) grants that the selected subset has near-full gradient norm, and Section IV.C then 'shows' subset updates can beat full updates without ever linking the Score to this assumption.

  1. self definitional [Section IV.A, Assumption (b); Section IV.C, Theoretical Comparison]
    "We assume that the subset of layers W(t)S contains layers where the gradient most effectively reduces the loss. Specifically, we assume that in each round t: ∥∇W(t)S L(W(t))∥2 ≥ ∥∇L(W(t))∥2 − δ, where δ is a positive constant."

    Assumption (b) is exactly the property FedTLU's Score is supposed to provide: that the selected blocks are the ones whose updates most reduce the loss. The paper never connects Score(W_i)=∥ΔW_i∥/(√n_i·std(ΔW_i)) to this assumption. Section IV.C then concludes that subset updates can yield greater loss reduction than full updates when ηL≥2, but this conclusion is derived from Assumption (b), i.e., from granting that the chosen subset already has near-full effective gradient. The theoretical 'prediction' is therefore conditional on the very effectiveness the scoring mechanism is meant to establish; as support for FedTLU's selection rule it is circular.

  2. other [Section IV.C, Theoretical Comparison: Full Model vs. Subset Updates]
    "Given the loss reduction inequalities for subset and full model updates, consider the scenario where the subset update yields a greater loss reduction than the full model update. This implies that the subset update provides a tighter boundary. The inequality describing the difference between the loss reduction boundaries for the subset update and the full model update is given by: −ηδ(1 − ηL/2) ≥ 0. For this inequality to hold, we require 1 − ηL/2 ≤ 0, which implies ηL ≥ 2."

    The derivation compares the lower bounds ΔS ≥ η(∥∇L∥²−δ)(1−ηL/2) and Δall ≥ η∥∇L∥²(1−ηL/2), then treats the subset bound being larger as proof that the subset update gives greater actual loss reduction. This is an equivocation: the 'difference between the loss reduction boundaries' is by construction the difference of the bounds, not the difference of the actual reductions. When ηL>2 both bounds are negative, so the comparison is vacuous; in any case a larger lower bound does not order the true quantities. The claimed condition ηL≥2 is thus manufactured by comparing the paper's own lower-bound expressions rather than derived from the loss dynamics.

full rationale

FedTLU's experimental comparison to random and last-layer updates is real empirical content, and no self-citation chain is load-bearing (Ref. [26] is only used to support the generic point that a well-optimized global model is a strong starting point). The main circularity is in Section IV. Assumption (b) simply asserts that the selected subset W_S contains the layers whose gradient most effectively reduces the loss—precisely the property that the Score in Section III is supposed to deliver but never proves. The Section IV.C conclusion that subset updates can beat full updates when ηL≥2 is derived from this assumption and from a comparison of lower bounds rather than actual loss reductions, so the theory's 'prediction' is effectively conditional on its own desired conclusion: it assumes the target subset is the effective one, then concludes that updating the effective subset is beneficial. Because the paper presents Section IV as theoretical support for FedTLU's design, the central claim's derivation chain is partly circular. The empirical results (Tables I–III) are independent of this derivation, though underpowered (three repetitions, no error bars) and inconsistent with the 'consistently outperforms' claim: Table III shows Random beating FedTLU by 0.004 in Transformer/FedAvg noisy global perplexity (1.764 vs. 1.768). Overall score 6 reflects partial circularity in the theory while acknowledging the independent empirical component.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

FedTLU adds one scoring statistic and a block-selection rule, but its stated theoretical support depends on an assumption that the selected subset contains the effective gradient, which is the property the score is supposed to deliver. The experiments introduce several hand-set quantities (update portion, plateau threshold, participation rates) that the analysis does not govern. No new entities are invented.

free parameters (3)
  • Number of selected blocks per group S (update portion) = 75%, 50%, 25% of blocks per group, with a sequential decrease during fine-tuning
    S controls how many blocks are updated and is central to the method; experiments vary it and use an adaptive schedule, but there is no principled rule for choosing it.
  • delta (subset gradient gap constant) = unspecified positive constant
    Introduced in Assumption (b) of Section IV.A; the theory's conclusion that subset updates can beat full updates depends on delta being large enough, but no value or estimator is given.
  • Fine-tuning reduction thresholds = loss decrease < 1% for 10 rounds triggers portion reduction
    This schedule in Section V.A is a hand-set rule that affects the fine-tuning perplexity results; it is not justified or analyzed.
assumptions (4)
  • standard math The global loss is L-smooth (gradient Lipschitz continuous).
    Used in Section IV.A(a) to write the descent inequality; standard in optimization analyses.
  • ad hoc to paper The selected subset W_S contains layers where the gradient most effectively reduces loss: ||grad_{W_S} L||^2 >= ||grad L||^2 - delta.
    Section IV.A(b) assumes the very property FedTLU's score is meant to guarantee; no link between the score and this inequality is shown.
  • domain assumption Full client participation, one step of full-batch gradient descent per client per round, and stationary client data distributions.
    Section IV.A(c) makes these idealizations; they contradict the experiments in Section V (10% or 5% participation, 5 local epochs, label-shuffled clients), so the theory does not cover the tested setting.
  • domain assumption Repeated blocks with identical parameter sequences exist and can be grouped for fair comparison.
    Used in Section III to justify grouping blocks by parameter sequence; valid for Transformer-style models but not for all architectures.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedTLU: Federated Learning with Targeted Layer Updates." pith.science (2026). https://pith.science/paper/I2VRNJ2J

@misc{pith2026241217692,
  author       = {Pith},
  title        = {Pith review of: FedTLU: Federated Learning with Targeted Layer Updates},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I2VRNJ2J}},
  note         = {Machine review of arXiv:2412.17692}
}
read the original abstract

Federated learning (FL) addresses privacy concerns in training language models by enabling multiple clients to contribute to the training, without sending their data to others. However, non-IID (identically and independently distributed) data across clients often limits FL's performance. This issue is especially challenging during model fine-tuning, as noise due to variations in clients' data distributions can harm model convergence near stationary points. This paper proposes a targeted layer update strategy for fine-tuning in FL. Instead of randomly updating layers of the language model, as often done in practice, we use a scoring mechanism to identify and update the most critical layers, avoiding excessively noisy or even poisoned updates by freezing the parameters in other layers. We show in extensive experiments that our method improves convergence and performance in non-IID settings, offering a more efficient approach to fine-tuning federated language models.

Figures

Figures reproduced from arXiv: 2412.17692 by the authors.

Figure 1
Figure 1. Perplexity curves for fine-tuning with Full, FedTLU, Random, and Last-layer updates on Transformer (Penn TB) and GPT-2 (UDPOS) models. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Perplexity curves for fine-tuning with Full, FedTLU, and Random updates in noisy or malicious client scenarios. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 31 canonical work pages

  1. [1]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  2. [2]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

    arXiv 2018

  3. [3]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  4. [4]

    A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,

    Y . Yao, J. Duan, K. Xu, Y . Cai, Z. Sun, and Y . Zhang, “A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,” High-Confidence Computing, p. 100211, 2024

    2024

  5. [5]

    Fedmed: A federated learning framework for language modeling,

    X. Wu, Z. Liang, and J. Wang, “Fedmed: A federated learning framework for language modeling,” Sensors, vol. 20, no. 14, p. 4048, 2020

    work page 2020

  6. [6]

    Scaling federated learning for fine-tuning of large language models,

    A. Hilmkil, S. Callh, M. Barbieri, L. R. S ¨utfeld, E. L. Zec, and O. Mogren, “Scaling federated learning for fine-tuning of large language models,” in International Conference on Applications of Natural Lan- guage to Information Systems . Springer, 2021, pp. 15–23

    work page 2021

  7. [7]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics . PMLR, 2017

    work page 2017

  8. [8]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine Learning and Systems , vol. 2, pp. 429–450, 2020

    2020

Show all 46 references
  1. [9]

    Preserving data privacy via fed- erated learning: Challenges and solutions,

    Z. Li, V . Sharma, and S. P. Mohanty, “Preserving data privacy via fed- erated learning: Challenges and solutions,” IEEE Consumer Electronics Magazine, vol. 9, no. 3, pp. 8–16, 2020

    2020

  2. [10]

    Federated analytics: Opportunities and challenges,

    D. Wang, S. Shi, Y . Zhu, and Z. Han, “Federated analytics: Opportunities and challenges,” IEEE Network, vol. 36, no. 1, pp. 151–158, 2021

    2021

  3. [11]

    Generalized federated learning via sharpness aware minimization,

    Z. Qu, X. Li, R. Duan, Y . Liu, B. Tang, and Z. Lu, “Generalized federated learning via sharpness aware minimization,” in International Conference on Machine Learning . PMLR, 2022, pp. 18 250–18 280

    2022

  4. [12]

    A survey on federated learning: The journey from centralized to distributed on-site learning and beyond,

    S. AbdulRahman, H. Tout, H. Ould-Slimane, A. Mourad, C. Talhi, and M. Guizani, “A survey on federated learning: The journey from centralized to distributed on-site learning and beyond,” IEEE Internet of Things Journal, vol. 8, no. 7, pp. 5476–5497, 2020

    2020

  5. [13]

    Internet of things intrusion detection: Centralized, on-device, or federated learning?

    S. A. Rahman, H. Tout, C. Talhi, and A. Mourad, “Internet of things intrusion detection: Centralized, on-device, or federated learning?” IEEE Network, vol. 34, no. 6, pp. 310–317, 2020

    2020

  6. [14]

    Higher order naive bayes: A novel non-iid approach to text classification,

    M. C. Ganiz, C. George, and W. M. Pottenger, “Higher order naive bayes: A novel non-iid approach to text classification,” IEEE Transac- tions on Knowledge and Data Engineering , vol. 23, no. 7, 2010

    2010

  7. [15]

    Beyond iid: Non-iid thinking, informatics, and learning,

    L. Cao, “Beyond iid: Non-iid thinking, informatics, and learning,” IEEE Intelligent Systems, vol. 37, no. 4, pp. 5–17, 2022

    2022

  8. [16]

    Fedbiot: Llm local fine-tuning in federated learning without full model,

    F. Wu, Z. Li, Y . Li, B. Ding, and J. Gao, “Fedbiot: Llm local fine-tuning in federated learning without full model,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2024, pp. 3345–3355

    2024

  9. [17]

    Momentum-based variance reduction in non-convex sgd,

    A. Cutkosky and F. Orabona, “Momentum-based variance reduction in non-convex sgd,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  10. [18]

    Federated learning with proximal stochastic variance reduced gradient algorithms,

    C. T. Dinh, N. H. Tran, T. D. Nguyen, W. Bao, A. Y . Zomaya, and B. B. Zhou, “Federated learning with proximal stochastic variance reduced gradient algorithms,” in Proceedings of the 49th International Conference on Parallel Processing, 2020, pp. 1–11

    2020

  11. [19]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,

    H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, and C. A. Raffel, “Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 1950–1965, 2022

    1950

  12. [20]

    Sensitivity-aware visual parameter-efficient fine-tuning,

    H. He, J. Cai, J. Zhang, D. Tao, and B. Zhuang, “Sensitivity-aware visual parameter-efficient fine-tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 11 825–11 835

    2023

  13. [21]

    Ditto: Fair and robust federated learning through personalization,

    T. Li, S. Hu, A. Beirami, and V . Smith, “Ditto: Fair and robust federated learning through personalization,” in International Conference on Machine Learning . PMLR, 2021, pp. 6357–6368

    2021

  14. [22]

    Towards personalized federated learning,

    A. Z. Tan, H. Yu, L. Cui, and Q. Yang, “Towards personalized federated learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 12, pp. 9587–9603, 2022

    2022

  15. [23]

    Personalization improves privacy-accuracy tradeoffs in federated learning,

    A. Bietti, C.-Y . Wei, M. Dudik, J. Langford, and S. Wu, “Personalization improves privacy-accuracy tradeoffs in federated learning,” in Interna- tional Conference on Machine Learning. PMLR, 2022, pp. 1945–1962

    2022

  16. [24]

    Local learning matters: Rethinking data heterogeneity in federated learning,

    M. Mendieta, T. Yang, P. Wang, M. Lee, Z. Ding, and C. Chen, “Local learning matters: Rethinking data heterogeneity in federated learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 8397–8406

    2022

  17. [25]

    Heterogeneous federated learning: State-of-the-art and research challenges,

    M. Ye, X. Fang, B. Du, P. C. Yuen, and D. Tao, “Heterogeneous federated learning: State-of-the-art and research challenges,” ACM Computing Surveys, vol. 56, no. 3, pp. 1–44, 2023

    2023

  18. [26]

    Fedbaf: Federated learning aggregation biased by a foundation model,

    J.-I. Park, S. Pranav, J. M. Moura, and C. Joe-Wong, “Fedbaf: Federated learning aggregation biased by a foundation model,” arXiv preprint arXiv:2410.18352, 2024

    2024 arXiv

  19. [27]

    Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives,

    P. Liu, X. Xu, and W. Wang, “Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives,” Cybersecurity, vol. 5, no. 1, p. 4, 2022

    2022

  20. [28]

    Robust federated learning with noisy and het- erogeneous clients,

    X. Fang and M. Ye, “Robust federated learning with noisy and het- erogeneous clients,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 10 072–10 081

    2022

  21. [29]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of NaacL-HLT, vol. 1, 2019, p. 2

    2019

  22. [30]

    Improved dropout for shallow and deep learning,

    Z. Li, B. Gong, and T. Yang, “Improved dropout for shallow and deep learning,” Advances in Neural Information Processing Systems , vol. 29, 2016

    2016

  23. [31]

    Random cnn structure: tool to increase generalization ability in deep learning,

    B. Swiderski, S. Osowski, G. Gwardys, J. Kurek, M. Slowinska, and I. Lugowska, “Random cnn structure: tool to increase generalization ability in deep learning,” Eurasip Journal on Image and Video Process- ing, vol. 2022, no. 1, p. 3, 2022

    2022

  24. [32]

    Layerout: Freezing layers in deep neural networks,

    K. Goutam, S. Balasubramanian, D. Gera, and R. R. Sarma, “Layerout: Freezing layers in deep neural networks,” SN Computer Science , vol. 1, no. 5, p. 295, 2020

    2020

  25. [33]

    Transfusion: Un- derstanding transfer learning for medical imaging,

    M. Raghu, C. Zhang, J. Kleinberg, and S. Bengio, “Transfusion: Un- derstanding transfer learning for medical imaging,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  26. [34]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning . PMLR, 2019, pp. 2790–2799

    2019

  27. [35]

    Differentially private fine- tuning of language models,

    D. Yu, S. Naik, A. Backurs, S. Gopi, H. A. Inan, G. Kamath, J. Kulkarni, Y . T. Lee, A. Manoel, L. Wutschitz et al. , “Differentially private fine- tuning of language models,” arXiv preprint arXiv:2110.06500 , 2021

    2021 arXiv

  28. [36]

    Sensitivity and generalization in neural networks: an empirical study,

    R. Novak, Y . Bahri, D. A. Abolafia, J. Pennington, and J. Sohl-Dickstein, “Sensitivity and generalization in neural networks: an empirical study,” arXiv preprint arXiv:1802.08760 , 2018

    2018 arXiv

  29. [37]

    Corrnet: pearson correlation based pruning for efficient convolutional neural networks,

    A. Kumar, B. Yin, A. M. Shaikh, M. Ali, and W. Wei, “Corrnet: pearson correlation based pruning for efficient convolutional neural networks,” International Journal of Machine Learning and Cybernetics , vol. 13, no. 12, pp. 3773–3783, 2022

    2022

  30. [38]

    Scaffold: Stochastic controlled averaging for federated learn- ing,

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learn- ing,” in International Conference on Machine Learning . PMLR, 2020, pp. 5132–5143

    2020

  31. [39]

    Advances and open problems in federated learning,

    P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al. , “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning , vol. 14, no. 1–2, pp. 1–210, 2021

    2021

  32. [40]

    Fedbert: When federated learning meets pre-training,

    Y . Tian, Y . Wan, L. Lyu, D. Yao, H. Jin, and L. Sun, “Fedbert: When federated learning meets pre-training,” ACM Transactions on Intelligent Systems and Technology (TIST) , vol. 13, no. 4, pp. 1–26, 2022

    2022

  33. [41]

    Federated learning on non-iid data: A survey,

    H. Zhu, J. Xu, S. Liu, and Y . Jin, “Federated learning on non-iid data: A survey,” Neurocomputing, vol. 465, pp. 371–390, 2021

    2021

  34. [42]

    Federated multi-task learning,

    V . Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Federated multi-task learning,” Advances in Neural Information Processing Sys- tems, vol. 30, 2017

    2017

  35. [43]

    Clustered federated learning: Model-agnostic distributed multitask optimization under privacy con- straints,

    F. Sattler, K.-R. M ¨uller, and W. Samek, “Clustered federated learning: Model-agnostic distributed multitask optimization under privacy con- straints,” IEEE Transactions on Neural Networks and Learning Systems , vol. 32, no. 8, pp. 3710–3722, 2020

    2020

  36. [44]

    On the importance of single directions for generalization,

    A. S. Morcos, D. G. Barrett, N. C. Rabinowitz, and M. Botvinick, “On the importance of single directions for generalization,” International Conference on Learning Representations , 2018

    2018

  37. [45]

    On the convergence of fedavg on non-iid data,

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” International Conference on Learning Representations, 2020

    2020

  38. [46]

    Zico: Zero-shot nas via inverse coefficient of variation on gradients,

    G. Li, Y . Yang, K. Bhardwaj, and R. Marculescu, “Zico: Zero-shot nas via inverse coefficient of variation on gradients,” International Conference on Learning Representations , 2023

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.