Pith. sign in

REVIEW 3 major objections 6 minor 40 references

Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FedHLM claims a more-than-95% reduction in cloud LLM token transmissions by learning per-client uncertainty thresholds through federated averaging.

desk verdict The 95% transmission reduction is an artifact of a degenerate training objective, but the underlying idea of FL-learned thresholds is worth a serious look. read the letter →

arxiv 2507.00082 v1 pith:4W5GJPK6 submitted 2025-06-30 cs.LG cs.AIcs.CL

Faranaksadat Solat , Joohyung Lee , Mohamed Seif , Dusit Niyato , H. Vincent Poor This is my paper
classification cs.LGcs.AIcs.CL
keywords federatedlearninghybridlanguagemodelsuncertaintythresholdstokentransmissionlargesmallmobileedgecomputingpeer-to-peerresolution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedHLM is a framework for hybrid language models that tries to keep most token predictions on edge devices and only contact a cloud LLM when the local small model is genuinely unsure. Its central claim is that federated learning can learn per-client uncertainty thresholds, and that peer devices can resolve many uncertain tokens by semantic similarity, cutting LLM transmissions by over 95% on a news classification task. The paper further claims this can be done with negligible accuracy loss, reporting 93.2% accuracy at round 30 versus 97.67% for centralized federated learning, while transmitting only 708 of 18,000 tokens to the LLM compared with about 12,600 or 14,800 for baselines. If the claims hold, FedHLM would make cloud-assisted language inference practical for bandwidth-constrained edge settings such as mobile assistants and smart-home devices.

What carries the argument

The load-bearing object is the per-client uncertainty threshold $u_{\mathrm{th},k} \in [0,1]$, a scalar decision boundary learned by gradient descent on a sigmoid-smoothed routing loss and aggregated across clients with federated averaging. Tokens are scored by their disagreement rate under temperature-sampled predictions; if the score is below the threshold the token is accepted locally, otherwise it moves to peer resolution (cosine similarity against a centroid of peer embeddings), then to the edge server, and finally to the LLM. The sigmoid approximation $\sigma(\gamma(u_t - u_{\mathrm{th},k}))$ makes the binary routing decision differentiable so that gradients can flow to the threshold. The paper also formalizes an expected communication cost, giving the opportunistic condition $p_{\mathrm{hit}} \ge C_{\mathrm{P2P}}/C_{\mathrm{LLM}}$ for when peer collaboration is worthwhile.

What would settle it

Run the FedHLM threshold optimizer on a classification task where the local SLM is known to be weak, and record the learned global threshold over rounds; if the threshold converges toward 1 and LLM transmissions fall to zero while task accuracy stays at or above the reported 93.2%, then the communication savings come from the loss's degenerate optimum rather than from useful threshold learning. A sharper version: fix every client's threshold at the maximum and measure task accuracy; if it matches or beats FedHLM's reported accuracy, the federated learning loop is not carrying the result.

Watch

Extended reading notes

Core claim

The paper's central discovery is that token-level transmission decisions in a hybrid SLM/LLM setup can themselves be learned in a federated way instead of being fixed by hand. Each client maintains a scalar uncertainty threshold; tokens whose sampling-based disagreement score exceeds the threshold are escalated, first to peer clients through embedding similarity and then to the cloud LLM. Local threshold gradients are aggregated by edge servers and then by the cloud, yielding a global policy that adapts to heterogeneous clients. The reported result is that FedHLM resolves 94.49% of tokens locally, 1.57% through peer exchange, and only 3.93% through the LLM, reducing absolute LLM transmissions from 12,598 (Rand-HLM) and 14,847 (U-HLM) to 708 out of 18,000, with inference accuracy reaching 93.2% after 30 federated rounds.

Load-bearing premise

The loss function used to train the thresholds only penalizes tokens that were sent to the LLM, so keeping a wrong token locally costs nothing; as written, the objective is minimized by raising the threshold to its maximum and transmitting nothing.

Editorial extensions

If this is right

  • If FedHLM works as reported, edge devices can run hybrid language inference with more than an order of magnitude fewer cloud LLM calls, directly reducing uplink bandwidth and per-token latency.
  • Federated threshold adaptation would let routing policies personalize to each client's data without uploading raw text, closing the gap between static heuristics and centrally tuned thresholds.
  • Semantic peer reuse means the framework's savings grow with overlap among clients; in applications with shared phrases, redundant queries that would otherwise hit the LLM can be resolved at the edge.
  • The reported non-IID results imply that benefits shrink as client distributions diverge: local resolution falls from 94.5% to 75.4% and LLM reliance rises to 19.8% under high non-IIDness, so deployment should be paired with clustering or personalization in heterogeneous settings.
  • Because the architecture is defined on embeddings rather than token IDs, the same threshold loop can be carried over to vision or speech tokens, a direction the paper itself flags as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension the paper leaves implicit is to apply the same federated threshold loop to the P2P similarity threshold and the cache size, since the expected cost of peer resolution depends on exactly those quantities.
  • The opportunistic P2P condition $p_{\mathrm{hit}} \ge C_{\mathrm{P2P}}/C_{\mathrm{LLM}}$ implies a simple deployment rule: measure semantic overlap between clients before enabling peer exchange; if the expected hit rate is below the cost ratio, the system should route straight to the LLM.
  • The framework's accuracy measurement is task-level final classification, so a stronger test of 'negligible accuracy loss' would be to measure token-level acceptance errors, since those errors are invisible to the paper's loss function.
  • A natural next step, not explored in the paper, is to make the threshold update depend on downstream task loss rather than LLM rejection feedback; that would directly penalize incorrect local acceptances and likely change the learned thresholds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. FedHLM proposes a federated, hierarchical framework for hybrid SLM/LLM inference in which clients learn personalized token-level uncertainty thresholds that decide whether a token is accepted locally, resolved through peer embedding similarity, or escalated to a cloud LLM. The paper formalizes a local loss for threshold optimization, describes FedAvg-based threshold aggregation, and reports simulations on AG News claiming a 95% reduction in LLM transmissions with negligible accuracy loss. The central claim is that federated threshold learning captures a useful accuracy-communication tradeoff in a privacy-preserving, bandwidth-constrained setting.

Significance. The problem addressed is timely and practically important: reducing uplink communication in hybrid edge-cloud language model inference is a real deployment bottleneck. The idea of learning personalized uncertainty thresholds through federated optimization, rather than using static hand-tuned thresholds, is attractive, and the paper gives a concrete MLOps implementation sketch and complexity analysis. However, the evaluation does not establish the claimed tradeoff. The local objective in Eq. (4) is degenerate: it sums only over transmitted tokens and contains no penalty for incorrect local acceptance, so its minimum is to transmit nothing. The reported transmission reduction is therefore a direct consequence of the objective rather than evidence that federated threshold learning discovers a meaningful operating point. In addition, the reported 4.47-point accuracy gap in Table VI is not obviously 'negligible,' and the experiments rely on unspecified 'probabilistic LLM feedback' rather than a real LLM. The central experimental claim is not supported by the current manuscript.

major comments (3)
  1. [IV-C] The local loss is degenerate and does not encode an accuracy-communication tradeoff. In Eq. (4), L_k(u_th,k) = sum_t sigma(gamma(u_k^t - u_th,k))((1 - beta_k^t)^2 + lambda), and the text states that beta is undefined for non-transmitted tokens and those tokens are excluded from training. Consequently, every transmitted token contributes a strictly positive term because sigma > 0 and (1 - beta)^2 + lambda > 0 for lambda > 0. The gradient in Eq. (5) is -gamma sum sigma(1 - sigma)((1 - beta)^2 + lambda), which is strictly negative whenever at least one token is transmitted, so the update in Eq. (6) always increases u_th,k. The minimum of the loss is zero, achieved by setting the threshold high enough that no tokens are transmitted. Thus the objective rewards transmitting nothing and never penalizes wrong local acceptances. This contradicts the reported behavior in Fig. 4, where the threshold converges to about 0.53, and Table IV, where 708 of 18,000 tokens are sent to the LLM. Those results cannot arise from optimizing Eq. (4) without undisclosed early stopping, a different objective, or a different treatment of beta and lambda. This is load-bearing because the headline 95% transmission reduction is presented as evidence for the proposed federated threshold learning mechanism.
  2. [VI-A] The quantitative evaluation is not reproducible because the LLM feedback is simulated rather than produced by an actual LLM. Section VI-A describes 'probabilistic LLM feedback' without specifying the distribution of beta, the number of stochastic samples K, the temperature T, the sigmoid temperature gamma, the regularization lambda, the similarity threshold theta, or the cache parameters. Since beta is the only accuracy-related feedback in Eq. (4), the choice of its simulation model directly determines both the learned thresholds and the reported accuracy. Without these details, the LLM-transmission counts in Table IV and the accuracy numbers in Table VI cannot be checked, and the claimed 95% reduction cannot be attributed to the proposed method rather than to the simulation's assumptions.
  3. [VI-G] The claim of 'negligible accuracy loss' is not supported by the reported numbers. Table VI shows FedHLM reaching 93.2% accuracy at round 30 versus 97.67% for centralized FL, a gap of 4.47 percentage points. The text calls this 'highly competitive,' but no confidence intervals, standard deviations, or multiple runs are reported, and Fig. 10 does not give the numerical accuracy of the full-LLM baseline. Whether a 4.47-point drop is negligible depends on the application, but the manuscript does not provide the statistical evidence needed to support that characterization.
minor comments (6)
  1. [III-B] Step 2 of the system model uses the symbol tau for the learned threshold, while Section IV and Table II use u_th,k; this notation should be unified.
  2. [VI-C] The text says Fig. 4 illustrates the global threshold progression 'in the U-HLM model,' but U-HLM is described elsewhere as using a fixed threshold; it appears the figure actually corresponds to FedHLM. Please clarify which method is being shown.
  3. [References] References [11] and [40] appear to cite the same work with different titles; they should be merged or distinguished clearly.
  4. [V] The heading of Section V is 'Proposed FedHLM Scheme – Embedding-Based P2P Resolution,' but the section opens with convergence analysis and does not clearly separate the P2P material from the convergence discussion; reorganize the section structure for readability.
  5. [V-E] The convergence claim in Eq. (11) is stated as following from generic FedAvg results, but the paper does not provide a proof for the specific threshold-adaptation mechanism and even says a rigorous proof 'can be constructed' without constructing it. Either include the proof or explicitly present the convergence result as empirical.
  6. [VI-C] Figure 5 plots the loss for a fixed beta value of 0.2, but the training loss in Eq. (4) only sums over transmitted tokens and uses observed beta values. The figure is a synthetic illustration; this should be stated clearly in the caption so readers do not mistake it for a training trajectory.

Circularity Check

1 steps flagged · score 7.0 of 10

Headline transmission reduction is the degenerate minimizer of Eq. (4), so the central claim reduces to the loss design rather than to federated threshold learning.

  1. self definitional [Section IV-C, Eqs. (4)-(6); Section VI-C, Fig. 4 and Table IV]
    "This value is undefined for tokens not transmitted and is excluded from training. The local loss function becomes: Lk(uth,k)= Σ_t σ(γ(u_t^k − u_th,k))((1−β_t^k)^2 + λ), (4) where, (1−β_t^k)^2 penalizes sending tokens that the LLM would have accepted, and λ controls the regularization strength for communication minimization."

    Because β is undefined for untransmitted tokens and those tokens are excluded, every term in Eq. (4) corresponds to a transmitted token and is strictly positive: σ(γ(u−u_th))>0 and ((1−β)^2+λ)>0. The gradient in Eq. (5) is therefore negative whenever any token is transmitted, so the threshold update in Eq. (6) drives u_th,k toward its upper bound 1, where no tokens are transmitted. Consequently, the headline result — over 95% LLM transmission reduction, and the specific 708/18,000 LLM tokens in Table IV — is not an independent prediction of the framework; it is the literal objective being minimized. The paper's own Fig. 4 (threshold converging to about 0.53 while transmitting 3.93% of tokens) contradicts the unique minimizer of Eq.

full rationale

The central empirical claim is that FedHLM reduces LLM transmissions by over 95% while maintaining accuracy. That claim is not an emergent result of federated threshold learning: the loss in Eq. (4) directly penalizes every transmitted token with a strictly positive weight λ plus the squared rejection term, and the gradient in Eq. (5) is always negative in the threshold direction. Minimizing Eq. (4) therefore forces the threshold to its maximum, transmitting nothing; any observed transmission count is an artifact of the chosen objective or of unspecified early stopping, not of the proposed FL mechanism. The accuracy numbers and the comparison against centralized FL are separate evidence and prevent a fully circular score, but the paper's defining contribution — communication-efficient token transmission — reduces by construction to the objective it optimizes. No load-bearing self-citation circularity was found; the referenced convergence results for FedAvg do not rescue the degenerate loss. Score 7 reflects the partial independence of the accuracy evaluation and the otherwise unavoidable conclusion that the headline transmission reduction is encoded in the loss itself.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities; it combines existing mechanisms such as thresholds, FedAvg, embeddings, and hierarchical aggregation in a new configuration. The free parameters and domain assumptions listed above are the main unmeasured inputs that the central claim depends on.

free parameters (5)
  • gamma
    Temperature scaling parameter in the sigmoid approximation (Section IV-C). Controls gradient steepness and is set by hand; not estimated from data.
  • lambda
    Regularization strength in the local loss (Eq. (4)). Directly trades off transmission penalty against rejection penalty and is chosen by hand.
  • theta
    Cosine similarity threshold for peer consensus (Section IV-D). The paper says it is learned empirically or adapted per client, but no method or value is given.
  • K and temperature T for uncertainty sampling
    Number of stochastic samples and temperature scaling used to compute the disagreement-based uncertainty score (Section IV-B). Values are not specified, yet they determine which tokens are deemed uncertain.
  • probabilistic LLM feedback distribution
    Section VI-A says LLM feedback is probabilistic, but the distribution generating beta_t is not specified. All reported transmission reductions depend on this unspecified synthetic model.
assumptions (5)
  • standard math Local loss is Lipschitz-smooth with bounded stochastic gradient variance and diminishing learning rates.
    Invoked in Section IV-E to borrow FedAvg convergence results from [32],[33]; not verified for the specific sigmoid-threshold loss used here.
  • domain assumption Semantic embedding cosine similarity is a valid proxy for token correctness in P2P resolution.
    Section IV-D assumes sim >= theta implies the peer token is correct; no accuracy evaluation of this consensus rule is provided.
  • domain assumption Pre-clustering via a lightweight classifier on embedding statistics reliably groups semantically similar clients.
    Section III.A states clients are pre-clustered using a SnapCFL-inspired classifier, but the algorithm and its quality are not detailed.
  • domain assumption Entropy or disagreement rate correctly measures token-level uncertainty.
    Section III.B defines uncertainty as Monte Carlo dropout-style disagreement; the paper does not validate this measure against actual token error rates.
  • domain assumption The sigmoid approximation sigma(gamma(u - u_th)) accurately preserves the binary routing decision.
    Used in Eqs. (4)-(6) to make threshold optimization differentiable; approximation quality is not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission." pith.science (2026). https://pith.science/paper/4W5GJPK6

@misc{pith2026250700082,
  author       = {Pith},
  title        = {Pith review of: Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4W5GJPK6}},
  note         = {Machine review of arXiv:2507.00082}
}
read the original abstract

Hybrid Language Models (HLMs) combine the low-latency efficiency of Small Language Models (SLMs) on edge devices with the high accuracy of Large Language Models (LLMs) on centralized servers. Unlike traditional end-to-end LLM inference, HLMs reduce latency and communication by invoking LLMs only when local SLM predictions are uncertain, i.e., when token-level confidence is low or entropy is high. However, ambiguous or low-confidence predictions still require frequent offloading to the LLM, leading to significant communication overhead in bandwidth-constrained settings. To address this, we propose FedHLM, a communication-efficient HLM framework that integrates uncertainty-aware inference with Federated Learning (FL). FedHLM's key innovation lies in collaboratively learning token-level uncertainty thresholds that govern when LLM assistance is needed. Rather than using static or manually tuned thresholds, FedHLM employs FL to optimize these thresholds in a privacy-preserving, distributed manner. Additionally, it leverages embedding-based token representations for Peer-to-Peer (P2P) resolution, enabling clients to reuse tokens inferred by semantically similar peers without engaging the LLM. We further introduce hierarchical model aggregation: edge servers refine local routing policies through client updates, while cross-cluster coordination aligns global decision boundaries. This layered design captures recurring uncertainty patterns, reducing redundant LLM queries. Experiments on large-scale news classification tasks show that FedHLM reduces LLM transmissions by over 95 percent with negligible accuracy loss, making it well-suited for scalable and efficient edge-AI applications.

Figures

Figures reproduced from arXiv: 2507.00082 by the authors.

Figure 1
Figure 1. System architecture of the proposed FedHLM framework for communication-efficient hybrid language inference. The client layer performs lightweight [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Token routing flow in FedHLM. Each client begins by performing [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. FedHLM pipeline implemented within Kubeflow’s MLOps archi [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: (a) Progression of the global uncertainty threshold over federated [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Behavior of the proposed loss function Lkputh,kq under varying uncertainty thresholds uth,k P t0.3, 0.5, 0.7u. The loss increases smoothly as token uncertainty u k t exceeds the threshold, following a sigmoid-based transition. In this plot, a constant value of β k t “ …
Figure 6
Figure 6. Figure 6: Token Resolution Breakdown for FedHLM: Local, P2P, and LLM [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Token Routing Distribution across varying levels of non-IIDness [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Token resolution breakdown across the Rand-HLM, U-HLM, and [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 10
Figure 10. Figure 10: Accuracy comparison of FedHLM with baseline methods over FL [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 9
Figure 9. Figure 9: Cache Hit Ratio vs. Token Entropy across Clients. A clear positive [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 34 canonical work pages

  1. [1]

    Split Federated Learning-Empowered Energy-Efficient Mobile Traffic Prediction Over UA Vs ,

    F. Solat, J. Lee, and D. Niyato, “Split Federated Learning-Empowered Energy-Efficient Mobile Traffic Prediction Over UA Vs ,”IEEE Wireless Communications Letters, vol. 13, no. 11, pp. 3064–3068, Nov. 2024

    work page 2024

  2. [2]

    Federated Learning- Empowered Mobile Network Management for 5G and Beyond Net- works: From Access to Core,

    J. Lee, F. Solat, T. Y . Kim, and H. V . Poor, “Federated Learning- Empowered Mobile Network Management for 5G and Beyond Net- works: From Access to Core,” IEEE Communications Surveys & Tutorials, vol. 26, no. 3, pp. 2176–2212, Jan. 2024

    work page 2024

  3. [3]

    Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents

    C. Shao, X. Hu, Y . Lin, and F. Xu, “Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents,” arXiv: 2502.04392, 2025

    work page Pith review arXiv 2025

  4. [4]

    Heterogeneous Privacy Level- Based Client Selection for Hybrid Federated and Centralized Learning in Mobile Edge Computing,

    F. Solat, S. Patni, S. Lim, and J. Lee, “Heterogeneous Privacy Level- Based Client Selection for Hybrid Federated and Centralized Learning in Mobile Edge Computing,” IEEE Access , vol. 12, pp. 108 556– 108 572, July 2024

    work page 2024

  5. [5]

    Token Communications: A Unified Framework for Cross- modal Context-aware Semantic Communications,

    L. Qiao, M. B. Mashhadi, Z. Gao, R. Tafazolli, M. Bennis, and D. Niyato, “Token Communications: A Unified Framework for Cross- modal Context-aware Semantic Communications,” arXiv: 2502.12096, 2025

    arXiv 2025

  6. [6]

    QoSBERT: An Uncertainty- Aware Approach based on Pre-trained Language Models for Service Quality Prediction,

    Z. Wang, X. Zhang, Z. S. Li, and M. Yan, “QoSBERT: An Uncertainty- Aware Approach based on Pre-trained Language Models for Service Quality Prediction,” arXiv: 2505.07863, 2025

    work page arXiv 2025

  7. [7]

    PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models – Federated Learning in Age of Foundation Model,

    T. Guo, S. Guo, J. Wang, X. Tang, and W. Xu, “PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models – Federated Learning in Age of Foundation Model,” IEEE Transactions on Mobile Computing , vol. 23, no. 5, pp. 5179–5194, May 2024

    work page 2024

  8. [8]

    Communication-Efficient Learning of Deep Networks from Decentralized Data,

    H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” arXiv:1602.05629, 2017

    arXiv 2017

Show all 40 references
  1. [9]

    Efficiently Modeling Long Sequences with Structured State Spaces,

    A. Gu, K. Goel, and C. R ´e, “Efficiently Modeling Long Sequences with Structured State Spaces,” arXiv:2111.00396, 2023

    2023 arXiv

  2. [10]

    Speculative Decoding for Neural Sequence Models,

    X. L. Chen and M. Bansal, “Speculative Decoding for Neural Sequence Models,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2019, pp. 1393–1398

    2019

  3. [11]

    Hybrid SLM and LLM for Edge-Cloud Collaborative Inference,

    Z. Hao, H. Jiang, S. Jiang, J. Ren, and T. Cao, “Hybrid SLM and LLM for Edge-Cloud Collaborative Inference,” in Proceedings of the Workshop on Edge and Mobile Foundation Models , 2024, pp. 36–41

    2024

  4. [12]

    HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction,

    Q. Hao, J. Fan, F. Xu, J. Yuan, and Y . Li, “HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction,” arXiv: 2410.09112, 2025

    2025 arXiv

  5. [13]

    Decentralizing large-scale natural language processing with federated learning,

    D. G. Bernal, “Decentralizing large-scale natural language processing with federated learning,” Master’s Thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 2020, degree Project in Computer Science and Engineering, Second Cycle, 30 Credits

    2020

  6. [14]

    Federated Learning: Strategies for Improving Communica- tion Efficiency,

    J. Kone ˇcn´y, H. B. McMahan, F. X. Yu, P. Richt ´arik, A. T. Suresh, and D. Bacon, “Federated Learning: Strategies for Improving Communica- tion Efficiency,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics , 2017, pp. 306–314

    2017

  7. [15]

    Decentralized Training of Foundation Models in Heterogeneous Environments,

    B. Yuan, Y . He, J. Davis, T. Zhang, T. Dao, B. Chen, P. Liang, C. R ´e, and C. Zhang, “Decentralized Training of Foundation Models in Heterogeneous Environments,” in Advances in Neural Information Processing Systems 35 (NeurIPS) , vol. 35, 2022, pp. 25 464–25 477

    2022

  8. [16]

    FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks,

    B. Y . Lin, C. He, Z. Zeng, H. Wang, Y . Huang, C. Dupuy, R. Gupta, M. Soltanolkotabi, X. Ren, and S. Avestimehr, “FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks,” arXiv: 2104.08815, 2021

    2021 arXiv

  9. [17]

    Federated Few-Shot Learning for Mobile NLP,

    D. Cai, S. Wang, Y . Wu, F. X. Lin, and M. Xu, “Federated Few-Shot Learning for Mobile NLP,” in Proceedings of 2023 the 29th Annual International Conference on Mobile Computing and Networking (ACM MobiCom), vol. 63, 2023, pp. 1–17

    2023

  10. [18]

    Efficient Federated Learning for Modern NLP,

    D. Cai, Y . Wu, S. Wang, F. X. Lin, and M. Xu, “Efficient Federated Learning for Modern NLP,” in Proceedings of the 2023 the 29th Annual International Conference on Mobile Computing and Networking (ACM MobiCom), vol. 37, 2023, pp. 1–16. 16

    2023

  11. [19]

    Feder- ated Learning Meets Natural Language Processing: A Survey,

    M. Liu, S. Ho, M. Wang, L. Gao, Y . Jin, and H. Zhang, “Feder- ated Learning Meets Natural Language Processing: A Survey,” arXiv: 2107.12603, 2021

    2021 arXiv

  12. [20]

    TITANIC: Towards Production Federated Learning with Large Language Models,

    N. Su, C. Hu, B. Li, and B. Li, “TITANIC: Towards Production Federated Learning with Large Language Models,” in Proceedings of the 2024 IEEE International Conference on Computer Communications (INFOCOM), 2024, pp. 611–620

    2024

  13. [21]

    Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning,

    Y . Gal and Z. Ghahramani, “Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning,” in Proceedings of the 33rd International Conference on Machine Learning , 2016, pp. 1050–1059

    2016

  14. [22]

    Uncertainty-Aware Deep Learning Models for Wireless Communication Systems,

    X. Zhang, R. Zhang, and S. Cui, “Uncertainty-Aware Deep Learning Models for Wireless Communication Systems,” IEEE Transactions on Communications, vol. 68, no. 11, pp. 7172–7187, 2020

    2020

  15. [23]

    DropConnect is Effective in Modeling Uncertainty of Bayesian Deep Networks,

    A. Mobiny, P. Yuan, S. K. Moulik, N. Garg, C. C. Wu, and H. V . Nguyen, “DropConnect is Effective in Modeling Uncertainty of Bayesian Deep Networks,” Scientific Reports, vol. 11, no. 1, 2021

    2021

  16. [24]

    Survey of Dropout Methods for Deep Neural Networks,

    A. Labach, H. Salehinejad, and S. Valaee, “Survey of Dropout Methods for Deep Neural Networks,” arXiv: 904.13310, 2019

    2019

  17. [25]

    Active Learning Literature Survey,

    B. Settles, “Active Learning Literature Survey,” University of Wisconsin, Madison, vol. 52, no. 55-66, p. 11, 2009

    2009

  18. [26]

    Active Learning Using Uncertainty Informa- tion,

    Y . Yang and M. Loog, “Active Learning Using Uncertainty Informa- tion,” in Proceedings of the 2016 the 23rd International Conference on Pattern Recognition (ICPR), Dec. 2016, pp. 2646–2651

    2016

  19. [27]

    Active Learning With Sampling by Uncertainty and Density for Data Annotations,

    J. Zhu, H. Wang, B. K. Tsou, and M. Ma, “Active Learning With Sampling by Uncertainty and Density for Data Annotations,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 18, no. 6, pp. 1323–1331, Sep. 2010

    2010

  20. [28]

    How to Measure Uncertainty in Uncertainty Sampling for Active Learning,

    V . L. Nguyen, M. H. Shaker, and E. H ¨ullermeier, “How to Measure Uncertainty in Uncertainty Sampling for Active Learning,” Machine Learning, vol. 111, no. 1, pp. 89–122, 2022

    2022

  21. [29]

    Confidence Estimation for Natural Language Processing: A Study on Machine Translation,

    J. Blatz, E. Fitzgerald, G. Foster, C. Goutte, A. Kulesza, A. Sanchis, and F. J. Och, “Confidence Estimation for Natural Language Processing: A Study on Machine Translation,” inProceedings of the 20th International Conference on Computational Linguistics , 2004, pp. 315–321

    2004

  22. [30]

    High-Confidence Classification of Partial Discharge Acoustic Signals Using Bayesian Networks for Uncertainty Quantifica- tion,

    A. Sharma, H. Devarajan, S. Govindarajan, T. B. Shanker, J. A. A. Rey, and S. Nandi, “High-Confidence Classification of Partial Discharge Acoustic Signals Using Bayesian Networks for Uncertainty Quantifica- tion,” IEEE Transactions on Instrumentation and Measurement, vol. 74, ...

    2025

  23. [31]

    SnapCFL: A Pre-clustering-based Clustered Federated Learning Framework for Data and System Heterogeneities,

    Y . Cheng, W. Zhang, Z. Zhang, J. K. an dQ. Xu, and S. Wang, “SnapCFL: A Pre-clustering-based Clustered Federated Learning Framework for Data and System Heterogeneities,” IEEE Transactions on Mobile Computing , vol. 24, no. 6, pp. 5214–5228, June 2025

    2025

  24. [32]

    On the Conver- gence of FedAvg on Non-IID Data,

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the Conver- gence of FedAvg on Non-IID Data,” arXiv: 907.02189, 2019

    2019

  25. [33]

    SCAFFOLD: Stochastic Controlled Averaging for Federated Learning,

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “SCAFFOLD: Stochastic Controlled Averaging for Federated Learning,” in Proceedings of the 2020 the International conference on machine learning (PMLR) , 2020, pp. 5132–5143

    2020

  26. [34]

    Tighter Theory for Local SGD on Identical and Heterogeneous Data,

    A. Khaled, K. Mishchenko, and P. Richtarik, “Tighter Theory for Local SGD on Identical and Heterogeneous Data,” in Proceedings of the 2020 the International conference on Artificial Intelligence and Statistics (PMLR), 2020, pp. 4519–4529

    2020

  27. [35]

    Federated Optimization in Heterogeneous Networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated Optimization in Heterogeneous Networks,” in Proceedings of the 2020 the International conference on Machine Learning and Systems 2 (MLSys) , 2020, pp. 429–450

    2020

  28. [36]

    Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimiza- tion,

    J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V . Poor, “Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimiza- tion,” pp. 7611–7623, 2020

    2020

  29. [37]

    Character-level Convolutional Networks for Text Classification,

    X. Zhang, J. Zhao, and Y . LeCun, “Character-level Convolutional Networks for Text Classification,” in Advances in Neural Information Processing Systems (NeurIPS) , 2015

    2015

  30. [38]

    BERT: Pre- Training of Deep Bidirectional Transformers for Language Understand- ing,

    J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- Training of Deep Bidirectional Transformers for Language Understand- ing,” in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human lan- guage techn...

    2019

  31. [39]

    Transformers: State-of-the-Art Natural Language Processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, and J. Brew, “Transformers: State-of-the-Art Natural Language Processing,” in Proceedings of the 2020 conference on empirical methods in natural language processing: syst...

    2020

  32. [40]

    Hybrid SLM and LLM for Edge-Cloud Collaborative Inference,

    Z. Hao, H. Jiang, J. Ren, and T. Cao, “Hybrid SLM and LLM for Edge-Cloud Collaborative Inference,” in Proceedings of the 2024 the Workshop on Edge and Mobile Foundation Models , 2024, pp. 36–41. Faranaksadat Solat received the B.S. degree in In- dustrial Engineering from Alzah...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.