Pith. sign in

REVIEW 5 major objections 4 minor 30 references

Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devices

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Splitting a language model into two perturbation blocks cuts zeroth-order federated finetuning computation by 1.6x to 3x while keeping memory near inference-level.

desk verdict The split-perturbation idea is real, but the published algorithm has a seed-list swap that makes the reported results unreproducible; needs major revision. read the letter →

arxiv 2502.10239 v3 pith:ELF5M7OP submitted 2025-02-14 cs.LG cs.AI

Mohamed Aboelenien Ahmed , Kilian Pfeiffer , Ramin Khalili , Heba Khdr , Jörg Henkel This is my paper · ORCID
classification cs.LGcs.AI
keywords federatedlearningzeroth-orderoptimizationlargelanguagemodelfinetuningperturbation-basedgradientestimationresource-constrainededgedevicescommunicationefficiencyactivationreuse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes FedSPZO, a method for federated finetuning of large language models that avoids backpropagation and the memory cost of stored activations. It claims that by splitting the network into two consecutive blocks and assigning fewer random perturbations to the first block and more to the smaller second block, a client can update the entire model with 1.6x to 3x fewer floating-point operations than the zero-order federated baselines FedZO and DecomFL. The savings come from reusing the first block's intermediate output across the many perturbations of the second block, turning a cost that scales with the full model into one that scales mostly with a small fraction of it. If true, this makes zero-order finetuning considerably more viable on memory- and communication-constrained edge devices without sacrificing the accuracy that zero-order methods already achieve.

What carries the argument

The central mechanism is split-perturbation zero-order gradient estimation with activation reuse. Writing the model as $y=f_2(\theta_2; f_1(\theta_1; B))$, FedSPZO perturbs only $f_1$ in the two directions of a central difference, stores its final-layer output $y_l$, then perturbs $f_2$ $P_2$ times against that stored output. The scalar gradient for $f_1$ averages loss differences over all second-block perturbations, while $f_2$'s gradient accumulates the usual per-perturbation scalars. This makes the forward cost $2\cdot\mathrm{fw}_1\times P_1 + 2\cdot\mathrm{fw}_2\times P_2$ instead of $2\times\mathrm{fw}_{\mathrm{total}}\times P$, which is what yields the claimed reduction.

What would settle it

Measure the total FLOPs FedSPZO needs to reach a target loss on a held-out model (e.g., OPT-1.3B or LLaMA-3-3.2B) and a held-out dataset, then compare with FedZO and DecomFL using the same perturbation budgets; if the ratio falls below 1.6x, or the overhead of caching and re-perturbing the split exceeds the reuse savings, the central claim does not transfer.

Watch

Extended reading notes

Core claim

FedSPZO shows that the computational bottleneck of zero-order federated finetuning can be reduced by treating the model as two blocks with different perturbation budgets instead of one uniform parameter vector. With $P_1=2$ perturbations on the earlier block and $P_2=8$ on the later block, the method estimates gradients for both blocks from a single set of forward evaluations: the first block is perturbed in the two central-difference directions, its output at the split is cached, and the second block is perturbed many times against that cached output. The paper reports 1.6x-3x less computation than FedZO and DecomFL to reach the same loss on SST2, RTE, and WIC with RoBERTa-large, while sending only scalar gradients (with regenerable seeds) to the server and keeping peak memory within 0.44% of the MeZO baseline.

Load-bearing premise

The claimed 1.6x-3x computation reduction rests on $P_1=2$ and $P_2=8$ being chosen by grid search on the same datasets used for evaluation, so the split may not carry the same gains to other models or tasks.

Editorial extensions

If this is right

  • Zero-order federated finetuning becomes more practical on resource-constrained edge devices because the dominant per-step cost (forward passes under many perturbations) is reduced.
  • The memory advantage of MeZO-style training is preserved: only the activation at the split point is stored, adding less than 0.5% to peak memory.
  • Communication stays at the scale of a few scalar gradients per local step, since the server can regenerate the perturbation seeds and reconstruct each client's model without forward passes.
  • Central-difference estimation, which is more accurate than forward difference, becomes affordable, giving accuracy close to first-order FedAvg while keeping communication negligible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The split point and perturbation counts could be chosen per model or per layer based on FLOPs and parameter counts; the paper fixes $P_1=2$, $P_2=8$ by grid search on the evaluation tasks.
  • The abstract advertises experiments on OPT-1.3B and LLaMA-3-3.2B, while the detailed body experiments are for RoBERTa-large; confirming the 1.6x-3x reduction on those larger models would test whether the split transfers.
  • The two-block idea extends naturally to more than two blocks or to block-specific perturbation schedules, though each additional split stores another intermediate activation that must be weighed against the reuse savings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes FedSPZO, a federated zero-order finetuning method that splits the model into two consecutive blocks, allocates P1 perturbation directions to the first block and P2 perturbation directions to the second block, reuses the first block's output activations to avoid recomputing them for every second-block perturbation, and communicates only gradient scalars (and optionally seeds) to the server. The central claims are a 1.6-3x reduction in computation relative to FedZO and DecomFL, memory close to MeZO, and accuracy close to first-order FedAvg. The experimental section reports RoBERTa-large results on SST2, RTE, and WIC, with a memory comparison against FedAvg and LoRA.

Significance. If correct, FedSPZO would be a useful contribution to zero-order federated finetuning on resource-constrained devices, and the idea of reusing intermediate activations across perturbations is sensible. The paper provides a complete algorithmic skeleton, a memory formula, and comparisons to relevant ZO baselines. However, the published Algorithm 2 contains internal inconsistencies that prevent reproduction of the client update and server reconstruction, the abstract promises evaluations on OPT1.3B and LLaMa-3-3.2B that do not appear in the results, and the main efficiency numbers rely on perturbation counts chosen on the same evaluation tasks. These issues currently outweigh the strengths.

major comments (5)
  1. [Algorithm 2, lines 22-23] The client update passes the wrong seed lists to Update. After the P1 loop, sl1 has length P1=2 and sl2 has length P2=8 (because each Forward call appends Ps seeds and there are two Forward calls per P1 iteration). Line 22 calls Update(θ2, sl1, g2/P2, µ, P2), which indexes beyond sl1, and line 23 calls Update(θ1, sl2, g1/P1, µ, P1), updating the first block in directions that were generated for the second block. The model updated on the client is therefore not the model the server reconstructs in Algorithm 1, and the reported loss/accuracy curves are not tied to the published procedure. The obvious fix is θ1 ← Update(θ1, sl1, g1/P1, µ, P1) and θ2 ← Update(θ2, sl2, g2/P2, µ, P2), but as written the algorithm is not executable.
  2. [Algorithm 2, lines 12 and 41] The gradient computation for the first block is also not well defined as written. L+ and L- are allocated with length 2Ps, but Forward writes L±[pj] and L±[pj+1] inside a loop over p_j in Ps. For Ps>1, consecutive iterations overwrite odd-index entries and the arrays are never filled, so Eq. (5) and line 17 do not compute the intended g1. The loop should write to indices 2*p_j and 2*p_j+1 (or use a loop over 2Ps) to produce the 2Ps loss values assumed by the formula.
  3. [Abstract and Section 4.1] The abstract claims evaluations on OPT1.3B and LLaMa-3-3.2B models, but the experimental section only reports RoBERTa-large results. Section 4.1 additionally lists BOOLQ as one of four tasks, yet Table 1 and Figure 4 contain only SST2, WIC, and RTE. The paper should either add the promised results or revise the abstract and setup to state exactly which models and tasks were used.
  4. [Section 4.1] The central computation savings are not robust as presented because P1=2 and P2=8 were selected by testing combinations on the same tasks used for evaluation, and no sensitivity analysis is given across model sizes, tasks, or block boundaries. The same issue applies to the FedZO baseline, whose P=5 is also grid-searched on these tasks. Without ablations or a theoretical rationale for the split, the 1.6-3x reduction is a tuned empirical quantity rather than a demonstrated property of the method.
  5. [Section 3.2.2, Eq. (7)] Eq. (7) accounts only for forward-pass FLOPs, but the reported GFLOPs comparisons in Figure 4 must also include the perturbation loops (Perturb operations over every parameter) and the parameter update loops, which the text only hand-waves as 'divided similarly.' Please state precisely which terms are included in the reported FLOPs and confirm that Eq. (7) is the complete cost model used in the experiments; otherwise the comparison with FedZO and DecomFL is not reproducible.
minor comments (4)
  1. [Eq. (5) and Algorithm 2, line 17] The symbol P2s is used without definition; it appears to mean P_s^2, but the paper should define it explicitly and align it with the loop bounds 0..2Ps in Eq. (5).
  2. [Section 4.2, Figure 3] The comparison with FedAvg(LoRA) should state the batch sizes, local steps, and context lengths for both methods, since Figure 2 shows different batch sizes for FedAvg and FedSPZO and this affects the GFLOPs comparison.
  3. [Section 4.3] The sentence 'FedZO use more 1.04x computation' is missing a comparison target; it should say 'FedZO uses 1.04x more computation per round than FedSPZO.'
  4. [FedSPZO Communication Extension] The extension claims the server can regenerate seeds from a single initial seed, but Algorithm 2 samples s1, s2, and sh inside the client loop. The paper should specify the deterministic seed schedule or state that the reported experiments use the basic mode with seeds uploaded.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: FedSPZO's 1.6-3x computation reduction is an empirical FLOPs comparison with tuned hyperparameters, not a derivation equivalent to its inputs.

full rationale

The central claim is that splitting the network into two blocks and using P1 and P2 perturbations per block lowers zero-order FL computation. This is supported by the FLOPs accounting in Eq. (7): the forward cost is 2*fw1*P1 + 2*fw2*P2, which is an arithmetic consequence of the number of perturbed forward passes, not a quantity defined in terms of the reported speedup. The values P1=2 and P2=8 are selected in Section 4.1 by testing combinations and keeping those with the highest computational savings on the same evaluation tasks; this is a hyperparameter-tuning and generalization concern, not a circular reduction, because the FLOPs comparison is an independent empirical measurement and the method does not define its output in terms of its inputs. The seed trick is adopted from external prior work (Malladi et al., 2023), not from a self-citation chain, and no uniqueness theorem or ansatz is imported from the authors' own prior papers. One internal inconsistency does appear in Algorithm 2: lines 22-23 call Update(theta2, sl1, ...) and Update(theta1, sl2, ...), swapping the seed lists built for the two blocks, which would also index beyond sl1 for the theta2 update when P2=8. This is a correctness/reproducibility defect, but it is not circularity: it does not make the reported reduction equal to an input by construction. Accordingly, the derivation chain is self-contained and no circular step is present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim depends on no new physical entities. Its main burdens are the empirically fitted perturbation counts P1 and P2, an unproved gradient estimator for the first block, and a FLOPs accounting that is not fully derivable from the presented pseudocode.

free parameters (3)
  • P1 (first-block perturbation count) = 2
    Selected by grid search on the evaluation tasks (Section 4.1). The claimed computation savings depend critically on this choice.
  • P2 (second-block perturbation count) = 8
    Selected by grid search on the evaluation tasks (Section 4.1). Together with P1, it determines the reported 1.6x-3x reduction.
  • FedZO baseline perturbation count = 5
    The authors grid-searched and selected P=5 for the FedZO baseline (Section 4.3), which affects the fairness and outcome of the computation comparison.
assumptions (3)
  • domain assumption The pseudo-random generator with a fixed seed reproduces identical perturbation vectors on client and server.
    Used in the seed trick and server reconstruction (Section 3.2.1, Algorithm 1); any divergence breaks exact model reconstruction.
  • ad hoc to paper The gradient estimator for the first block, Eq. (5), is an unbiased or sufficiently low-noise estimate of the gradient along z1.
    The paper asserts the double sum over f2 losses improves the estimate but provides no proof or variance analysis; this is load-bearing for the convergence claim.
  • ad hoc to paper The computational cost model in Eq. (7), 2*fw1*P1 + 2*fw2*P2, accounts for all forward passes in the algorithm.
    The algorithm's nested loops over P1 and P2 perturbations suggest f2 is evaluated more often than the formula indicates, and the relationship between P2 and the per-direction repetition count Ps is not defined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devices." pith.science (2026). https://pith.science/paper/ELF5M7OP

@misc{pith2026250210239,
  author       = {Pith},
  title        = {Pith review of: Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ELF5M7OP}},
  note         = {Machine review of arXiv:2502.10239}
}
abstract

Federated Learning (FL) is a promising paradigm for finetuning Large Language Models (LLMs) across distributed data sources while preserving data privacy. However, finetuning such large models is challenging on edge devices due to its high resource demand. Zeroth-order Optimization (ZO) estimates gradients through finite-difference approximations, which rely on function evaluations under random perturbations of the model parameters. Consequently, ZO with task alignment provides a potential solution, allowing finetuning using only forward passes with inference-level memory requirements and low communication overhead, but it suffers from slow convergence and higher computational demand. In this paper, we propose a new ZO-based method that applies a more efficient technique to reduce the computational demand associated with using a large number of perturbations while preserving their convergence benefits. This is achieved by splitting the model into consecutive blocks and allocating a higher number of perturbations to the second block, enabling efficient reuse of intermediate activations to update the full network with fewer forward evaluations. Our evaluation on RoBERTa-large, OPT1.3B, LLaMa-3-3.2B models shows up to $3\times$ reduction in computation compared to the other ZO-based techniques, while retaining the memory and communication benefits over first-order federated learning techniques.

Figures

Figures reproduced from arXiv: 2502.10239 by the authors.

Figure 1
Figure 1. An overview of the proposed Federated Split-Perturbation Zero-order Optimization (FedSPZO) round. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Memory footprint for training RoBERTa-Large [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Computation and upload comparison with FedSPZO(ours) and Federated finetuning for LoRA Fe￾dAvg(LoRA) over SST2 dataset. length datasets, we present a comparison of computation and upload between FedSPZO and federated finetuning us￾ing LoRA (FedAvg(LoRA)) on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 19 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction

    Alam, S., Liu, L., Yan, M., and Zhang, M. Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction. In Advances in Neural Information Processing Systems, volume 35, pp.\ 29677--29690, 2022

    work page 2022

  3. [3]

    SL o RA : Federated parameter efficient fine-tuning of language models

    Babakniya, S., Elkordy, A., Ezzeldin, Y., Liu, Q., Song, K.-B., EL-Khamy, M., and Avestimehr, S. SL o RA : Federated parameter efficient fine-tuning of language models. In International Workshop on Federated Learning in the Age of Foundation Models in Conjunction with NeurIPS 2023, 2023

    work page 2023

  4. [4]

    G., Pearlmutter, B

    Baydin, A. G., Pearlmutter, B. A., Syme, D., Wood, F., and Torr, P. Gradients without backpropagation. arXiv preprint arXiv:2202.08587, 2022

    arXiv 2022

  5. [5]

    R., Angeli, G., Potts, C., and Manning, C

    Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D. A large annotated corpus for learning natural language inference. In M \`a rquez, L., Callison-Burch, C., and Su, J. (eds.), Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp.\ 632--642, Lisbon, Portugal, September 2015. Association for Computational Linguist...

    doi:10.18653/v1/d15-1075 2015

  6. [6]

    A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization

    Cai, H., Lou, Y., McKenzie, D., and Yin, W. A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization. In International Conference on Machine Learning, pp.\ 1193--1203. PMLR, 2021

    work page 2021

  7. [7]

    B., and Talwalkar, A

    Caldas, S., Kone c ny, J., McMahan, H. B., and Talwalkar, A. Expanding the reach of federated learning by reducing client resource requirements. arXiv:1812.07210, 2018

    arXiv 2018

  8. [8]

    B ool Q : Exploring the surprising difficulty of natural yes/no questions

    Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K. B ool Q : Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pp.\ 2924--2936, ...

    doi:10.18653/v1/n19-1300 2019

Show all 30 references
  1. [9]

    N., and Zhou, Y

    Fang, W., Yu, Z., Jiang, Y., Shi, Y., Jones, C. N., and Zhou, Y. Communication-efficient stochastic zeroth-order optimization for federated learning. IEEE Transactions on Signal Processing, 70: 0 5058--5073, 2022

    2022

  2. [10]

    Does federated learning really need backpropagation

    Feng, H., Pang, T., Du, C., Chen, W., Yan, S., and Lin, M. Does federated learning really need backpropagation. arXiv preprint arXiv:2301.12195, 2023

    2023 arXiv

  3. [11]

    Making pre-trained language models better few-shot learners

    Gao, T., Fisch, A., and Chen, D. Making pre-trained language models better few-shot learners. In Zong, C., Xia, F., Li, W., and Navigli, R. (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen...

    2021

  4. [12]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022

    2022

  5. [13]

    Imteaj, A., Thakker, U., Wang, S., Li, J., and Amini, M. H. A survey on federated learning for resource-constrained iot devices. IEEE Internet of Things Journal, 9 0 (1): 0 1--24, 2022. doi:10.1109/JIOT.2021.3095077

    2022

  6. [14]

    Achieving dimension-free communication in federated learning via zeroth-order optimization

    Li, Z., Ying, B., Liu, Z., Dong, C., and Yang, H. Achieving dimension-free communication in federated learning via zeroth-order optimization. arXiv preprint arXiv:2405.15861, 2024

    2024 arXiv

  7. [15]

    On the convergence of zeroth-order federated tuning for large language models

    Ling, Z., Chen, D., Yao, L., Li, Y., and Shen, Y. On the convergence of zeroth-order federated tuning for large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 1827--1838, 2024

    2024

  8. [16]

    Roberta: A robustly optimized bert pretraining approach

    Liu, Y. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019

    1907 arXiv

  9. [17]

    D., Chen, D., and Arora, S

    Malladi, S., Gao, T., Nichani, E., Damian, A., Lee, J. D., Chen, D., and Arora, S. Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems, 36: 0 53038--53075, 2023

    2023

  10. [18]

    McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp.\ 1273--1282. PMLR, 2017

    2017

  11. [19]

    Black-box generalization: Stability of zeroth-order learning

    Nikolakakis, K., Haddadpour, F., Kalogerias, D., and Karbasi, A. Black-box generalization: Stability of zeroth-order learning. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ ...

    2022

  12. [20]

    Thinking forward: Memory-efficient federated finetuning of language models

    Panchal, K., Parikh, N., Choudhary, S., Zhang, L., Brun, Y., and Guan, H. Thinking forward: Memory-efficient federated finetuning of language models. arXiv preprint arXiv:2405.15551, 2024

    2024 arXiv

  13. [21]

    Aggregating capacity in fl through successive layer training for computationally-constrained devices

    Pfeiffer, K., Khalili, R., and Henkel, J. Aggregating capacity in fl through successive layer training for computationally-constrained devices. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing System...

    2023

  14. [22]

    Federated learning for computationally constrained heterogeneous devices: A survey

    Pfeiffer, K., Rapp, M., Khalili, R., and Henkel, J. Federated learning for computationally constrained heterogeneous devices: A survey. ACM Computing Surveys, 55 0 (14s): 0 1--27, 2023 b

    2023

  15. [23]

    Pilehvar, M. T. and Camacho-Collados, J. W i C : the word-in-context dataset for evaluating context-sensitive meaning representations. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technolo...

    2019 doi

  16. [24]

    Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes

    Qin, Z., Chen, D., Qian, B., Ding, B., Li, Y., and Deng, S. Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024

    2024

  17. [25]

    Shi, Y., Yang, K., Jiang, T., Zhang, J., and Letaief, K. B. Communication-efficient edge ai: Algorithms and systems. IEEE Communications Surveys & Tutorials, 22 0 (4): 0 2167--2191, 2020

    2020

  18. [26]

    D., Ng, A

    Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp.\ 1631--1642, 2013

    2013

  19. [27]

    Spall, J. C. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE transactions on automatic control, 37 0 (3): 0 332--341, 1992

    1992

  20. [28]

    Compressing rnns for iot devices by 15-38x using kronecker products

    Thakker, U., Beu, J., Gope, D., Zhou, C., Fedorov, I., Dasika, G., and Mattina, M. Compressing rnns for iot devices by 15-38x using kronecker products. arXiv:1906.02876, 2019

    1906 arXiv

  21. [29]

    Progfed: effective, communication, and computation efficient federated learning by progressive training

    Wang, H.-P., Stich, S., He, Y., and Fritz, M. Progfed: effective, communication, and computation efficient federated learning by progressive training. In International Conference on Machine Learning, pp.\ 23034--23054. PMLR, 2022

    2022

  22. [30]

    FwdLLM : Efficient federated finetuning of large language models with perturbed inferences

    Xu, M., Cai, D., Wu, Y., Li, X., and Wang, S. FwdLLM : Efficient federated finetuning of large language models with perturbed inferences. In 2024 USENIX Annual Technical Conference (USENIX ATC 24), pp.\ 579--596, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.