GradualDiff-Fed: A Federated Learning Specialized Framework for Large Language Model

Amir Faiyaz; Tara Salman

REVIEW 4 major objections 7 minor 20 references

GradualDiff-Fed: A Federated Learning Specialized Framework for Large Language Model

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GradualDiff-Fed's central claim is that federated LLM fine-tuning can upload only model-weight differences and still match centralized training.

desk verdict GradualDiff-Fed is FedAvg in delta notation, and its communication-saving claim collapses because dense deltas do not reduce transmitted bits; desk reject. read the letter →

arxiv 2506.19164 v1 pith:XYCFUJNM submitted 2025-06-23 cs.LG cs.DC

Amir Faiyaz , Tara Salman This is my paper · ORCID

classification cs.LGcs.DC

keywords GradualDiff-FedfederatedlearninglargelanguagemodelscommunicationefficiencymodeldifferenceupdatesLow-RankAdaptationparameter-efficientfine-tuningprivacy-preservingmachine

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GradualDiff-Fed is a federated learning scheme for fine-tuning large language models: data stays on each client, and the server coordinates training. Instead of uploading a full locally trained model, each client computes the element-wise difference between its updated weights and the current global model and uploads only that difference. The server averages the differences and adds the average back to the global model. The paper reports that on a mental-health chatbot task with a 7-billion-parameter model, this scheme matches or slightly beats centralized training on training loss and BLEU, and it claims a large reduction in communication overhead. If that claim holds, the method would let privacy-sensitive organizations collaboratively fine-tune large models without shipping raw data or full model weights.

What carries the argument

The load-bearing object is the delta parameterization: the difference between a client's locally fine-tuned Low-Rank Adaptation (LoRA) weights and the global model. Because LoRA freezes the base model and trains only low-rank adapter matrices, the delta lives on the adapter weights and is transmitted after 4-bit quantization. The server's update is a two-step operation: average the received deltas, then add the average to the global model, avoiding the averaging of full model snapshots. This delta-plus-addition rule is what the paper credits for making federated fine-tuning of billion-parameter models more scalable.

What would settle it

Measure the bytes actually uploaded per round under GradualDiff-Fed and under standard FedAvg with the same LoRA and quantization settings; if the per-round payloads are the same size, the central claim of reduced communication overhead is falsified. An even simpler check is Algorithm 2's step 5, which sends $\Delta \mathrm{LLM}_i^{(t)}$ as a dense weight vector whose serialized size equals the serialized size of $\theta_i^{(t)}$ unless an additional compression step is added.

Watch

Extended reading notes

Core claim

The central claim is that the delta vector $\Delta \mathrm{LLM}_i^{(t)} = \theta_i^{(t)} - \mathrm{LLM}_g^{(t)}$ carries all the information the server needs, so transmitting the delta rather than the full model $\theta_i^{(t)}$ keeps federated aggregation working while cutting communication. The proposed update extends the standard FedAvg averaging rule: the server computes $\mathrm{LLM}_g^{(t+1)} = \mathrm{LLM}_g^{(t)} + \frac{1}{K}\sum_{i=1}^K \Delta \mathrm{LLM}_i^{(t)}$, averaging client deltas and applying the result to the shared model. With Low-Rank Adaptation and 4-bit quantization, 15 rounds of GradualDiff-Fed reach training loss 0.22 at batch size 4 and 0.11 at batch size 8, compared with centralized training's 0.31 and 0.15; the BLEU score (n-gram overlap with reference text) is identical at 0.55, and perplexity (prediction uncertainty) is 16.78 versus central's 15.36. The paper interprets these results as showing that delta-based communication is an efficient, privacy-preserving way to fine-tune large models from distributed data without compromising performance.

Load-bearing premise

The load-bearing premise is that the transmitted difference $\Delta \mathrm{LLM}_i^{(t)}$ costs fewer bits than the full model, but the paper does not specify compression or sparsification for this dense vector, so the promised communication reduction is not demonstrated.

Editorial extensions

If this is right

In the tested configuration, GradualDiff-Fed reaches lower final training loss than centralized training (0.11 versus 0.15 at batch size 8) while staying within about 1.4 perplexity points of the central model.
Clients never send full model snapshots to the server, so the server's aggregation step is reduced to averaging deltas and adding them to the global model.
The framework relies on all selected clients participating each round; the paper presents this synchronous aggregation as a factor in its smooth loss reduction.
With Low-Rank Adaptation and 4-bit quantization, the method fits a 7-billion-parameter model on a single consumer GPU, making local training on ordinary hardware part of the scheme.
If the central communication claim is correct, the same delta-update pattern could be applied to other large-model fine-tuning tasks where uplink bandwidth is the bottleneck.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper never measures bytes exchanged per round; because $\Delta \mathrm{LLM}_i^{(t)}$ is a dense vector of the same dimension as $\theta_i^{(t)}$, the promised communication savings depend on an unstated compression or sparsification step, and measuring payload size with and without such a step would settle the claim.
The reported 36% faster per-round time is total elapsed time, not a measure of communication bytes, so it may reflect training with LoRA, 4-bit quantization, or fewer compute operations rather than the delta transmission itself.
The paper's own conclusion leaves unevenly distributed (non-IID) client data and privacy leakage through model differences as future work; both are natural stress tests, since averaged deltas could still expose information about local data.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The one thing to know: this paper's central claim is false on its own terms. GradualDiff-Fed is FedAvg with full participation and equal weights, rewritten as parameter differences. Eq. (2) in Section III is algebraically identical to the FedAvg update the authors cite, and Algorithm 2 adds no compression, sparsification, or quantization. A dense delta has exactly the same bit cost as the full model, so 'transmitting only the difference' does not reduce communication. The paper even notes that the server sends the full global model to every client each round; that broadcast alone is already one model transfer per client.

What the paper does well: the writing is clear, the FL background is accurate, and the LoRA fine-tuning setup is standard. The related work section cites the relevant federated LoRA methods (FedIT, FLoRA) and recognizes that naive LoRA averaging has known issues. The experimental setup is reproducible in principle: llama2-7b, LoRA rank 64, 15 rounds on a 100k-token mental health dataset, with training loss, perplexity, BLEU, and per-round time reported.

The soft spots are not minor. No FedAvg baseline is run, so the claimed advantage over standard FL is untested. Communication overhead is never measured in bytes; Table IV is wall-clock time on a single GPU, and the 36% reduction compares federated LoRA training against centralized full-model processing, which confounds model size, optimizer, and hardware. The loss numbers in Table II show FL beating centralized training by 26-29%, which is suspicious given the same data and should have triggered a sanity check. There is a single run, no error bars, and no code release. The paper's own conclusion admits non-IID and privacy leakage are future work.

Is there a new result hiding here? No. The delta-average is a notational variant of FedAvg, and the efficiency claim collapses under the definition in Eq. (1). This is not a paper with a fixable flaw; the central contribution is a rename.

Who is this for? Someone teaching FL basics might use it as a cautionary example, but it does not deserve referee time. I would desk reject.

Referee Report

4 major / 7 minor

Summary. The paper proposes GradualDiff-Fed, a federated fine-tuning framework for large language models in which each client transmits the difference between its locally updated model and the global model, ΔLLM_i^(t) = θ_i^(t) − LLM_g^(t), instead of the full model. The server averages these deltas and adds the average to the global model. The authors claim that this 'delta parameterization' drastically reduces communication overhead while achieving performance on par with centralized training. The evaluation is conducted with a Llama-2-7B model fine-tuned with LoRA on a mental-health dataset for 15 rounds across five clients, comparing FL, centralized learning, and local-client training on training loss, perplexity, BLEU score, and per-round computation time.

Significance. If the communication-efficiency claim were supported, the contribution would be modest: transmitting model deltas and averaging them is algebraically equivalent to FedAvg, and real efficiency gains would require an explicit compression mechanism. The paper does not provide such a mechanism nor a byte-level measurement. Its strengths are a clear algorithmic description and a straightforward experimental setup, but no code is supplied, no FedAvg baseline is measured, and no communication volume is reported. The claimed advantage over standard federated learning is therefore not established, and the paper's central hypothesis remains untested.

major comments (4)

[Section III, Eq. (1) and Algorithm 2 line 5] The central claim that transmitting ΔLLM_i^(t) = θ_i^(t) − LLM_g^(t) 'dramatically reduces' communication is not valid as stated. ΔLLM_i^(t) is a dense vector with the same dimension and the same bit width as θ_i^(t), so transmitting it costs the same number of bits as transmitting the locally updated model. Algorithm 2 specifies no sparsification, quantization, or compression before the transmission. Table IV reports wall-clock time per training round on a single GPU, not network bytes, and the comparison there is between LoRA-based federated training and full-parameter centralized processing, so it does not measure communication savings. The communication-efficiency claim therefore lacks both a mechanism and a measurement.
[Section III, Eq. (2)] The aggregation rule in Eq. (2) is algebraically identical to FedAvg. Because LLM_g^(t) is constant during aggregation, LLM_g^(t+1) = LLM_g^(t) + (1/K) Σ_i (θ_i^(t) − LLM_g^(t)) = (1/K) Σ_i θ_i^(t), which is exactly the FedAvg update with uniform weighting. The paper provides no comparison against standard FedAvg with LoRA, so the claimed benefits of 'delta parameterization' over the existing baseline are not demonstrated by any experiment.
[Section IV-A and Table IV] The experimental evaluation does not measure the quantity the paper claims to improve. There is no communication-volume measurement, no FedAvg baseline, and only one dataset with a single 15-round run. The per-round time comparison (4.3 vs. 6.7 seconds) is a computational timing on one GPU and conflates the LoRA-versus-full-model training cost with communication; the 36% reduction cannot be attributed to 'reducing communication costs through weight difference updates' as stated in Section IV-C. Furthermore, the paper itself defers non-IID evaluation to future work, so the method's central motivation is not tested.
[Section III, Algorithm 2 line 2] The server sends the global LLM weights to every client in each round. The total communication budget therefore includes this full-model broadcast plus all client deltas. Even if the client deltas were compressed, the broadcast would be a dominant cost, yet the paper's communication analysis omits this term entirely. Any statement about reduced communication must account for both directions of the exchange.

minor comments (7)

[Section II-A] The FedAvg formula as written, θ_g^(t) ← (1/N) Σ_i (n_i / Σ_j n_j) θ_i^(t), contains a spurious 1/N factor and is not the standard FedAvg aggregation; since FedAvg is the natural baseline, this should be corrected.
[Abstract and Section VI] There are repeated typos: 'without comprising performance' in the abstract and conclusion should be 'without compromising performance'.
[Section IV-A] The phrase '4k content size' is unclear; it presumably means a 4k context size, and 'we applied LoRA to all global layers within the adapters' should be rephrased.
[Table III] Only the Central perplexity is reported with an error bar (15.36 ± 0.2), and no standard deviations are given for the other entries; the number of repeated runs should be stated.
[Section II-A and References] The FedAvg method is cited as reference [7], but [7] is 'Adaptive federated optimization for heterogeneous data'; the original FedAvg paper [5] should be cited instead.
[Algorithm 1] The notation 'Δθ = Δθ_b Δθ_a' and the return value 'θ′ = θ + A×B' conflate the LoRA low-rank product with the difference notation; the relationship between A, B, Δθ_b, and Δθ_a should be defined precisely.
[Section IV-C and Conclusion] The conclusion states that future work should check performance under non-IID settings, which admits that the current evaluation does not address heterogeneity; this limitation should be stated prominently in the evaluation section rather than only in the conclusion.

Circularity Check

1 steps flagged · score 6.0 of 10

Delta-averaging is FedAvg in new coordinates; the communication reduction is asserted, not derived.

renaming known result [The reduction appears in Section III, Eqs. (1)–(2), and Algorithm 2, lines 5–7.]
"GradualDiff-Fed introduces key optimizations to enhance scalability and efficiency. First, instead of transmitting the entire updated model ( θ(t) i ) back to the server, each client in GradualDiff-Fed computes and transmits only the difference between its locally updated model and the global model: ΔLLM(t) i =θ(t) i − LLM(t) g ."

Substituting Eq. (1) into Eq. (2) gives LLM_g^{t+1} = LLM_g^t + (1/K) Σ_i (θ_i^t − LLM_g^t) = (1/K) Σ_i θ_i^t, because every client starts from the same LLM_g^t. That is exactly the FedAvg update defined in Section II with equal client weights. The claimed “key optimizations”—delta parameterization and averaging only differences—are therefore a relabeling of FedAvg in shifted coordinates, not a new aggregation rule. The communication-saving claim does not follow from this algebra: ΔLLM_i is a dense vector of the same dimension and dtype as θ_i, and Algorithm 2 line 5 transmits it uncompressed, so the bit volume is identical to sending θ_i.

full rationale

The only load-bearing algebraic step in the paper is Eqs. (1)–(2), and combining them collapses GradualDiff-Fed’s global update to the FedAvg aggregation that the paper itself cites as its baseline. Presenting this identity as a “key optimization” is the renaming-known-result pattern, so the central algorithmic novelty reduces by construction to a known method. Separately, the communication-efficiency claim is not circular but is unsupported: a dense model difference costs the same number of bits as the model, and no compression or communication measurement is given. The empirical loss, perplexity, and BLEU comparisons are external evidence and are not circular. No fitted parameter is presented as a prediction, and no load-bearing self-citation chain appears.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical or mathematical entities. Its free parameters are standard training hyperparameters that are chosen by hand and, in the case of the learning rate, the final value is not disclosed. The key unstated assumptions are synchronous full participation, the bit-cost of a dense delta, the validity of the central baseline, and the adequacy of LoRA for the task.

free parameters (6)

Learning rate = not reported (searched over {1e-5, 3e-5, 5e-5, 8e-5, 1e-4})
The text states a search was performed but never gives the final value used for Tables II and III, so the reported results cannot be reproduced and could reflect favorable selection.
LoRA rank r = 64
Chosen without sensitivity analysis; final loss values depend on this choice.
LoRA alpha = 16
Chosen without sensitivity analysis.
LoRA dropout = 0.1
Chosen without sensitivity analysis.
Number of clients K = 5
Arbitrary choice; no study of how client count affects the results.
Number of rounds T = 15
Chosen without convergence analysis; the model may not have converged in 15 rounds.

assumptions (4)

domain assumption All clients participate in every round and start from the same global model.
Algorithm 2 sends global weights to all K clients and aggregates all K deltas; this synchronous full participation is required for the delta-average to be exactly FedAvg. Partial participation would break the equivalence.
ad hoc to paper The transmitted delta is smaller in bits than the full model.
The communication claim presumes that ΔLLM_i^(t) is cheaper to send than θ_i^(t), but the paper never states a sparsity or compression mechanism, and a dense delta has equal size.
domain assumption Centralized training on the same data is a valid proxy for the global optimum.
The paper compares FL to central training on the full dataset, but the central baseline's training budget and data order are not specified, so the comparison may be inconsistent.
domain assumption LoRA low-rank factorization adequately represents the needed weight updates.
The entire framework depends on LoRA (Algorithm 1); the rank r=64 and alpha=16 are taken from hyperparameter choices, not derived.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GradualDiff-Fed: A Federated Learning Specialized Framework for Large Language Model." pith.science (2026). https://pith.science/paper/XYCFUJNM

@misc{pith2026250619164,
  author       = {Pith},
  title        = {Pith review of: GradualDiff-Fed: A Federated Learning Specialized Framework for Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYCFUJNM}},
  note         = {Machine review of arXiv:2506.19164}
}

read the original abstract

The rapid proliferation of large language models (LLMs) has created an unprecedented demand for fine-tuning models for specialized domains, such as medical science. While federated learning (FL) offers a decentralized and privacy-preserving approach to collaboratively fine-tune LLMs without sharing raw data, it presents significant challenges, particularly in performance and managing large model sizes efficiently. In this paper, we introduce GradualDiff-Fed, an FL framework designed explicitly for LLMs, and their challenge of handling the high parameter size. GradualDiff-Fed reduces communication costs by transmitting only the difference of model weights rather than the entire model during training rounds. Such an approach significantly improves scalability and communication efficiency, making it more feasible to fine-tune LLMs across distributed clients without compromising performance. Our evaluation demonstrates that GradualDiff-Fed achieves performance on par with centralized training while drastically reducing communication overhead. These results highlight the potential of GradualDiff-Fed as an efficient solution for fine-tuning large models from distributed data in privacy-preserving settings without comprising performance.

Figures

Figures reproduced from arXiv: 2506.19164 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 15 canonical work pages

[1]

Language models are few-shot learners,

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, and et al., “Language models are few-shot learners,” Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901, 2020, prompt Tuning

work page 1901
[2]

Domain-specific pretraining for natural language understanding,

C. Sun, J. Shrivastava, S. Singh, and C. Cardie, “Domain-specific pretraining for natural language understanding,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL- HLT), 2021, pp. 445–455, domain-Specific Architectures

work page 2021
[3]

Privacy challenges in large-scale language models,

J. Su, P. Wu, and H. Wang, “Privacy challenges in large-scale language models,” Journal of AI Research, vol. 58, pp. 1021–1035, 2022, privacy Challenges

work page 2022
[4]

Health data security and privacy: Challenges and solutions for the future,

K. Y . Yigzaw, S. D. Olabarriaga, A. Michalas, L. Marco-Ruiz, C. Hillen, Y . Verginadis, M. T. de Oliveira, D. Krefting, T. Penzel, J. Bowden, J. G. Bellika, and T. Chomutare, “Health data security and privacy: Challenges and solutions for the future,” in Roadmap to Successful Digital Health Ecosystems , E. Hovenga and H. Grain, Eds. Academic Press, 2022,...

work page 2022
[5]

Communication-efficient learning of deep networks from de- centralized data,

H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Y . Arcas, “Communication-efficient learning of deep networks from de- centralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017, pp. 1273–1282, federated Learning

work page 2017
[6]

Advances and open problems in federated learning,

P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, and et al., “Advances and open problems in federated learning,” Foundations and Trends in Machine Learning , vol. 14, no. 1–2, pp. 1– 210, 2021, comprehensive Federated Learning Survey

work page 2021
[7]

Testing the Robustness of a BiLSTM-based Structural Story Classifier

H. Jin, L. Feng, and Q. Zhang, “Adaptive federated optimization for heterogeneous data,” arXiv preprint arXiv:2201.02733 , 2022

work page Pith review arXiv 2022
[8]

Improving language understanding by generative pre-training,

A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” OpenAI Blog ,

work page

Show all 20 references

[9]

Recurrent neural network based language model,

T. Mikolov, M. Karafi ´at, L. Burget, J. Cernock ´y, and S. Khudanpur, “Recurrent neural network based language model,” in Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH), 2010, pp. 1045–1048

2010
[10]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS). Curran Associates, Inc., 2017, pp. 5998–6008

2017
[11]

Lora: Low-rank adaptation of large language models,

E. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, and et al., “Lora: Low-rank adaptation of large language models,” in Proceedings of the 2022 International Conference on Learning Representations (ICLR) , 2022, low-Rank Updates

2022
[12]

Aligner: Improving peft efficiency with alignment layers,

Z. Li, T. Wang, and R. Singh, “Aligner: Improving peft efficiency with alignment layers,” in Proceedings of the 2023 Conference on Neural Information Processing Systems (NeurIPS) , 2023, pp. 1201– 1215, adapter Methods

2023
[13]

Scaling language model size in cross-device federated learning,

J. Ro, T. Breiner, L. McConnaughey, M. Chen, A. Suresh, S. Kumar, and R. Mathews, “Scaling language model size in cross-device federated learning,” in Proceedings of the First Workshop on Federated Learning for Natural Language Processing (FL4NLP 2022) . Dublin, Ireland: Assoc...

2022
[14]

Communication-efficient federated learning via knowledge distillation,

C. Wu, F. Wu, L. Lyu, Y . Huang, and X. Xie, “Communication-efficient federated learning via knowledge distillation,” Nature Communications, vol. 13, no. 1, p. 2032, 2022

2022
[15]

Fwdllm: Efficient fedllm using forward gradient,

M. Xu, D. Cai, Y . Wu, X. Li, and S. Wang, “Fwdllm: Efficient fedllm using forward gradient,” 2024. [Online]. Available: https: //arxiv.org/abs/2308.13894

2024 arXiv
[16]

Conquering the communication constraints to enable large pre-trained models in federated learning,

G. Sun, H. Wang, Z. Xu, Z. S. Wu, M. Backes, and X. Zhang, “Conquering the communication constraints to enable large pre-trained models in federated learning,” arXiv preprint arXiv:2210.01708 , 2022. [Online]. Available: https://arxiv.org/abs/2210.01708

2022 arXiv
[17]

Towards building the federated gpt: Federated instruction tuning,

J. Zhang, S. Vahidian, M. Kuo, C. Li, R. Zhang, T. Yu, Y . Zhou, G. Wang, and Y . Chen, “Towards building the federated gpt: Federated instruction tuning,” 2024. [Online]. Available: https: //arxiv.org/abs/2305.05644

2024 arXiv
[18]

FLoRA: Federated fine-tuning large language models with heterogeneous low-rank adaptations,

Z. Wang, Z. Shen, Y . He, G. Sun, H. Wang, L. Lyu, and A. Li, “FLoRA: Federated fine-tuning large language models with heterogeneous low-rank adaptations,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024. [Online]. Available: https://open...

2024
[19]

On the convergence of zeroth-order federated tuning for large language models,

Z. Ling, D. Chen, L. Yao, Y . Li, and Y . Shen, “On the convergence of zeroth-order federated tuning for large language models,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , ser. KDD ’24. New York, NY , USA: Association for Computin...

2024
[2018]

Available: https://cdn.openai.com/research-covers/ language-unsupervised/language understanding paper.pdf

[Online]. Available: https://cdn.openai.com/research-covers/ language-unsupervised/language understanding paper.pdf

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Language models are few-shot learners,

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, and et al., “Language models are few-shot learners,” Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901, 2020, prompt Tuning

work page 1901

[2] [2]

Domain-specific pretraining for natural language understanding,

C. Sun, J. Shrivastava, S. Singh, and C. Cardie, “Domain-specific pretraining for natural language understanding,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL- HLT), 2021, pp. 445–455, domain-Specific Architectures

work page 2021

[3] [3]

Privacy challenges in large-scale language models,

J. Su, P. Wu, and H. Wang, “Privacy challenges in large-scale language models,” Journal of AI Research, vol. 58, pp. 1021–1035, 2022, privacy Challenges

work page 2022

[4] [4]

Health data security and privacy: Challenges and solutions for the future,

K. Y . Yigzaw, S. D. Olabarriaga, A. Michalas, L. Marco-Ruiz, C. Hillen, Y . Verginadis, M. T. de Oliveira, D. Krefting, T. Penzel, J. Bowden, J. G. Bellika, and T. Chomutare, “Health data security and privacy: Challenges and solutions for the future,” in Roadmap to Successful Digital Health Ecosystems , E. Hovenga and H. Grain, Eds. Academic Press, 2022,...

work page 2022

[5] [5]

Communication-efficient learning of deep networks from de- centralized data,

H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Y . Arcas, “Communication-efficient learning of deep networks from de- centralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017, pp. 1273–1282, federated Learning

work page 2017

[6] [6]

Advances and open problems in federated learning,

P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, and et al., “Advances and open problems in federated learning,” Foundations and Trends in Machine Learning , vol. 14, no. 1–2, pp. 1– 210, 2021, comprehensive Federated Learning Survey

work page 2021

[7] [7]

Testing the Robustness of a BiLSTM-based Structural Story Classifier

H. Jin, L. Feng, and Q. Zhang, “Adaptive federated optimization for heterogeneous data,” arXiv preprint arXiv:2201.02733 , 2022

work page Pith review arXiv 2022

[8] [8]

Improving language understanding by generative pre-training,

A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” OpenAI Blog ,

work page