Pith. sign in

REVIEW 3 major objections 6 minor 73 references

Recursive Offloading for LLM Serving in Multi-tier Networks

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Recursive offloading cuts LLM serving traffic by more than half without sacrificing service quality.

desk verdict RecServe's dynamic threshold is a modest but real improvement over static cascades, and the evaluation is substantial; the theoretical bound is a tautology and the confidence-as-correctness premise remains untested. read the letter →

arxiv 2505.16502 v2 pith:3TZQQCQJ submitted 2025-05-22 cs.DC cs.NI

Zhiyuan Wu , Sheng Sun , Yuwei Wang , Min Liu , Bo Gao , Jinda Lu , Zheming Yang , Tian Wen This is my paper · ORCID
classification cs.DCcs.NI
keywords recursiveoffloadingmulti-tiernetworksedge-cloudcollaborationlargelanguagemodelservingconfidence-basedroutingcommunicationefficiencycascadesquantilethreshold
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes RecServe, a way to serve large language model requests across a device-edge-cloud hierarchy by escalating each request only when the current, smaller model is not confident enough. It claims that a task-specific confidence score, maximum softmax probability for classification and normalized perplexity for generation, combined with a dynamically adjusted threshold from recent history lets each tier decide locally whether its answer is good enough. The authors argue this turns offloading into a self-tuning recursion rather than a manually calibrated cascade. If correct, RecServe would let operators keep most requests off the wide-area network while matching cloud-level service quality on the tasks that matter.

What carries the argument

The load-bearing mechanism is the pair formed by a task-specific confidence score and a $\beta$-quantile dynamic threshold computed from a historical confidence queue. For sequence-to-class tasks the score is the maximum softmax probability; for sequence-to-sequence tasks it is normalized perplexity, $C = 1/(1+\text{PPL})$. Each tier maintains a FIFO queue of the last $k$ scores, sorts them, and interpolates the $\beta$-quantile as the threshold $T_{M,\tau}(\beta)$; the recursion $D(x,M,\tau)$ returns the local answer if $C_{M,\tau}(x) \ge T_{M,\tau}(\beta)$ and otherwise forwards the request to the next tier. This single self-referential rule replaces manually tuned cascade thresholds and drives the theoretical result that expected communication burden is reduced whenever $\beta < (\sqrt{5}-1)/2$ in a three-tier network.

What would settle it

Take an out-of-distribution or adversarially constructed test set where small models are confidently wrong, run RecServe, and compare end-to-end accuracy against always-offloading to the cloud; if accuracy collapses while confidence stays high, the recursive stopping rule is not capturing true answer quality.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a recursive offloading policy driven by a sliding-window quantile threshold can make multi-tier LLM serving both cheaper and better than existing cascade-based routing. Each node keeps a FIFO queue of its recent confidence scores, sets the offloading threshold to the beta-quantile of that queue via linear interpolation, and offloads only when the current confidence falls below the threshold. Because the threshold tracks the local model's own historical confidence distribution, the system adapts to task complexity and network changes without manual tuning. The experiments across eight datasets show RecServe reaching higher accuracy or BLEU than the cascade baseline CasServe at equal or lower communication burden, and cutting communication volume by over 50% relative to centralized cloud serving.

Load-bearing premise

The framework assumes that a model's own confidence score reliably indicates whether its answer is good enough to return, so a small model that is confidently wrong will be served without ever reaching a stronger model.

Editorial extensions

If this is right

  • Operators can set one knob, beta, instead of per-tier thresholds, and still trade off accuracy against communication in a predictable direction.
  • For a three-tier device-edge-cloud deployment, choosing beta below the golden-ratio bound guarantees, under the paper's assumptions, that RecServe's expected communication burden is below cloud-only serving.
  • The same recursion generalizes beyond three tiers; the expected-burden formulas in Section IV-A cover arbitrary n-tier chains.
  • Because the threshold adapts from recent confidence history, the framework can follow shifts in task difficulty without retraining or reconfiguration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave confidence calibration unexamined: maximum softmax probability and normalized perplexity are proxies for certainty, not for correctness. A natural extension is to calibrate thresholds against labeled accuracy so that a confidently wrong small model still triggers offloading.
  • The theoretical bound beta(1 + beta) on communication ratio suggests a direct knife-edge test: measure actual traffic ratios across the eight datasets and check whether they fall near the predicted curve once the queue has warmed up.
  • The same confidence-queue machinery could be applied to other routing decisions, such as choosing which retrieval source or which reasoning budget to use, wherever a cheap model can signal its own uncertainty.
  • End-to-end latency is asserted but not measured; a deployment study could test whether the avoided wide-area round trips outweigh the added edge-tier hops in real networks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes RecServe, a multi-tier LLM serving framework that recursively offloads inference from device to edge to cloud based on task-specific confidence scores. A sliding-window queue stores recent confidence values, and the offloading threshold is set as the β-quantile of this queue via interpolation; a local output is accepted if its score is at least the threshold. The authors derive an approximation p_i≈β for the per-tier offloading probability and use it to bound expected communication and computation costs relative to cloud-only serving. Experiments on five classification and three translation datasets compare RecServe against EndServe, EdgeServe, CloudServe, ColServe, and CasServe, reporting accuracy or BLEU and communication burden, plus ablations on β, queue capacity, and the cloud-side model. The main claimed findings are that RecServe outperforms CasServe in both service quality and communication efficiency and reduces communication burden by over 50% compared to cloud-centric serving.

Significance. If the empirical results hold, RecServe would be a practical, self-tuning alternative to manually configured model cascades for device-edge-cloud LLM serving. The paper's strengths include open-source code, evaluation across eight datasets and two task families, and ablations of the key parameters. The core design is simple and likely easy to deploy. However, the theoretical communication result is essentially a consequence of the quantile definition rather than a substantive theorem, and the empirical evaluation does not yet establish that the confidence scores reliably track correctness or that the baseline comparison is complete. The central idea is plausible and worth further validation, but the current evidence does not support the paper's stated theoretical and empirical strength.

major comments (3)
  1. [Section III-C and Eqs. (16)-(17)] The decision rule uses C_{M,τ}(x) as a direct proxy for whether the local output is good enough to return, but the paper provides no calibration evidence for either confidence measure. Max softmax probability and normalized perplexity are known to be imperfectly calibrated, especially under distribution shift, and a small model can be confidently wrong. Because the threshold mechanically accepts the top roughly 1−β fraction of local outputs by confidence, aggregate accuracy on an easy in-distribution benchmark can remain high even if many confidently wrong local predictions are served. The paper should report reliability diagrams or conditional accuracy as a function of confidence for each tier and task type, and should show that accepted local outputs are indeed more likely to be correct; without this, the claimed quality-preservation property of RecServe is not established.
  2. [Section IV-A, Eqs. (19)-(30)] The main theoretical result p_i≈β is not a derived consequence of the framework but a restatement of how the threshold is defined. Since T_{M,τ}(β) is the empirical β-quantile of the historical confidence queue, under Assumptions 1 and 2 the probability that a fresh i.i.d. confidence score falls below it is approximately β by construction. The derivations in Eqs. (24)-(29) merely unpack this definition. Consequently, the communication ratio in Eq. (39) is a property of the chosen quantile, not an analytical insight into RecServe's behavior, and it says nothing about whether accepted local outputs preserve service quality. The section should be reframed as a design-parameter analysis, and the paper should acknowledge more directly, as it partly does in Section VII-B, that Assumptions 4 and 5 are violated in practice and can bias the predicted communication burden.
  3. [Section V-A.4 and Tables II-III] The empirical comparison to CasServe is reported selectively. The text states that twelve CasServe hyperparameter settings were considered, but the tables show only two settings per dataset and do not state the selection criterion. With 12 settings and 8 datasets, a full table or Pareto plot of all settings is needed to support the claim that RecServe outperforms CasServe; otherwise the result may depend on which settings are displayed. The paper also reports no variance or repeated-run information, and the headline communication-reduction claim of over 50% is not uniform across reported configurations: in the Seq2Seq experiments at β=0.5 the reduction is approximately one-third rather than over half. Please report the complete sweep with explicit selection rules and, where applicable, multiple seeds.
minor comments (6)
  1. [Abstract and Section IX] The 'over 50%' communication-reduction claim should be qualified by the β value and by per-dataset numbers, since in the Seq2Seq experiments at β=0.5 the reduction is approximately one-third rather than over half.
  2. [Section II-C] The symbol τ is used both for the binary offloading decision variable in Eq. (4) and for the task type in Table I and elsewhere; please rename one of the two usages to avoid confusion.
  3. [Algorithm 1 and Eq. (17)] The recursive call uses M_{i+1} in Algorithm 1 but M' in Eq. (17) for the upstream LLM; the notation should be unified.
  4. [Section VII-C.2] The proposed online feedback-based calibration mechanism for meeting a communication budget is not evaluated experimentally; even a small synthetic or simulated test would help show that the proposed update in Eq. (53) converges and actually meets the budget.
  5. [Tables II-III] Adding a column giving the ratio of each method's total communication burden to CloudServe's would make the headline reduction easier to verify directly from the tables.
  6. [Section VII-B] Figures 6 and 7 are introduced with minimal explanation in the text; please add a sentence describing how the plots were generated and which model and dataset they use, beyond the caption information.

Circularity Check

1 steps flagged · score 6.0 of 10

The theoretical communication-reduction result is a restatement of the β-quantile threshold definition: the offload probability p_i≈β holds by construction, so the β(1+β) ratio and the derived parameter bounds are algebraic identities rather than independent predictions; the empirical accuracy/communication comparisons remain independent measurements.

  1. self definitional [Section IV-A and IV-B, Eqs. (15), (19), (21), (30), (37), (39), (43)]
    "T_{M,τ}(β) = c_{(⌊r⌋+1)}·(1-(r-⌊r⌋)) + c_{(⌈r⌉+1)}·(r-⌊r⌋), where r=β·(k-1). ... p_i = P(C_{M_i,τ}(x) < T_{M_i,τ}(β)). ... P(C_{M_i,τ}(x) ≤ T_{M_i,τ}(β)) ≈ β. ... p_i ≈ β. ... E[Comm-RecServe] ≈ 2(|x|+|y|)·β(1+β). ... E[Comm-RecServe]/E[Comm-CloudServe] = β(1+β)."

    T_{M,τ}(β) is defined by Eq. (15) as the interpolated β-quantile of the historical confidence queue. Thus Assumption 2 (Eq. (21)), P(C≤T)≈β, is not an independent modeling assumption but the defining property of a quantile. The paper then derives p_i≈β (Eq. (30)) and, from that, the expected communication ratio β(1+β) (Eq. (39)), the bound β<0.618 (Eq. (41)), and the computation cost expression (Eq. (43)). These results are algebraic identities given the threshold definition: the offload probability is β because the threshold was chosen to be the β-quantile. Presenting them as 'theoretical analysis' that predicts communication reduction is therefore a restatement of the chosen hyperparameter, not a first-principles result independent of the mechanism's construction.

full rationale

We walked the paper's derivation chain and found one concrete reduction-by-construction. The dynamic threshold T_{M,τ}(β) is defined in Eq. (15) as the β-quantile of the historical confidence queue. Under Assumptions 1-3, the offload probability p_i (Eq. (19)) is therefore approximately β by construction (Eq. (30)), and every subsequent theoretical quantity—the expected communication burden E[Comm-RecServe]≈2(|x|+|y|)β(1+β) for three tiers (Eq. (37)), the communication ratio β(1+β) (Eq. (39)), the condition β<0.618 (Eq. (41)), and the expected computation cost Cost_device+β·Cost_edge+β²·Cost_cloud (Eq. (43))—is an algebraic consequence of that quantile definition. The paper presents this as a theoretical prediction of communication reduction, but the reduction is baked into the choice of β; no independent content is added beyond the threshold definition. This is a self-definitional step in the theoretical contribution. It is partial, not total. The paper's central empirical claims—RecServe outperforms CasServe and reduces measured communication burden by over 50% compared to CloudServe—are supported by the measurements in Tables II-III and Figures 2-5, which do not reduce to the tautological identity. There are no load-bearing self-citations: the authors' prior works [26], [27], [29] appear only as background on end-edge-cloud federated learning. The confidence-calibration concern raised in the review (softmax probability and normalized perplexity as correctness proxies) is a real validity risk but is not a circularity argument; it concerns whether the decision rule preserves service quality, not whether the derivation assumes its own conclusion. Section VII-B honestly acknowledges deviations from Assumptions 4 and 5, but this does not repair the tautological character of the Section IV analysis. Overall score 6: one predicted theoretical quantity reduces by construction, while the empirical demonstration retains independent content.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The theoretical communication and computation results rest on five explicit assumptions (i.i.d. confidence, sufficient history, CDF continuity, input-length independence, output-length invariance) plus the implicit premise that confidence reflects quality. The paper itself shows two of the explicit assumptions are violated in practice (Figs. 6 and 7). The empirical comparison additionally depends on hand-selected baseline thresholds and the hand-chosen parameters beta and k.

free parameters (2)
  • beta (offloading quantile) = 0.1, 0.3, 0.5 in experiments
    Hand-selected hyperparameter controlling the dynamic threshold quantile in the main experiments and in the theoretical communication/computation bounds. Not fitted to data, but the headline results depend on the chosen values.
  • k (confidence queue capacity) = 10,000 in main experiments
    Hand-selected sliding-window length for threshold estimation; ablation shows k around 300 is sufficient, so the main choice is conservative but not optimized.
assumptions (6)
  • domain assumption The current task's confidence score is drawn independently from the same distribution as the scores in the historical queue.
    Assumption 1 in Section IV-A underpins p_i approximately beta; violated in practice when task difficulty changes, and the paper itself notes input length correlates with confidence.
  • domain assumption The historical confidence queue contains enough samples that the empirical beta-quantile equals the population beta-quantile.
    Assumption 2 in Section IV-A; cold-start and small-k effects are acknowledged in Section VII-B.
  • standard math The confidence CDF is continuous, so strict and non-strict inequalities coincide at the threshold.
    Assumption 3 in Section IV-A, used in Eqs. (27)-(29) to derive P(C < T) = beta.
  • domain assumption Input text length is uncorrelated with the confidence score.
    Assumption 4 in Section IV-A; contradicted by Fig. 6 and acknowledged in Section VII-B.
  • domain assumption Output length distribution is identical across device, edge, and cloud tiers.
    Assumption 5 in Section IV-A; contradicted by Fig. 7 and acknowledged in Section VII-B.
  • domain assumption Max softmax probability (Seq2Class) and normalized perplexity (Seq2Seq) are reliable indicators of output quality.
    Section III-C defines these as confidence and Eq. (16) uses them to stop offloading; no calibration evidence is provided, so this is an unverified premise for the central empirical claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recursive Offloading for LLM Serving in Multi-tier Networks." pith.science (2026). https://pith.science/paper/3TZQQCQJ

@misc{pith2026250516502,
  author       = {Pith},
  title        = {Pith review of: Recursive Offloading for LLM Serving in Multi-tier Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3TZQQCQJ}},
  note         = {Machine review of arXiv:2505.16502}
}
read the original abstract

Heterogeneous device-edge-cloud computing infrastructures have become widely adopted in telecommunication operators and Wide Area Networks (WANs), offering multi-tier computational support for emerging intelligent services. With the rapid proliferation of Large Language Model (LLM) services, efficiently coordinating inference tasks and reducing communication overhead within these multi-tier network architectures becomes a critical deployment challenge. Existing LLM serving paradigms exhibit significant limitations: on-device deployment supports only lightweight LLMs due to hardware constraints, while cloud-centric deployment suffers from resource congestion and considerable prompt communication overhead caused by frequent service requests during peak periods. Although the model-cascading-based inference strategy adapts better to multi-tier networks, its reliance on fine-grained, manually adjusted thresholds makes it less responsive to dynamic network conditions and varying task complexities. To address these challenges, we propose RecServe, a recursive offloading framework tailored for LLM serving in multi-tier networks. RecServe integrates a task-specific hierarchical confidence evaluation mechanism that guides offloading decisions based on inferred task complexity in progressively scaled LLMs across device, edge, and cloud tiers. To further enable intelligent task routing across tiers, RecServe employs a sliding-window-based dynamic offloading strategy with quantile interpolation, enabling real-time tracking of historical confidence distributions and adaptive offloading threshold adjustments. Experiments on eight datasets demonstrate that RecServe outperforms CasServe in both service quality and communication efficiency, and reduces the communication burden by over 50\% compared to centralized cloud-based serving.

Figures

Figures reproduced from arXiv: 2505.16502 by the authors.

Figure 1
Figure 1. Schematic diagram of RecServe. confidence queue, task-specific confidence evaluation mech￾anisms, and the recursive offloading strategy with dynamic decision thresholds. Finally, we provide the formal description of RecServe. A. Framework Overview The schematic diagram of RecServe is illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Visualization of precision vs communication burden for multi-tier serving methods across eight datasets. [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Comparison of RecServe and ColServe with different offload config [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Comparison of RecServe and ColServe with DeBERTa-large deployed [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 7
Figure 7. Figure 7: Output length distribution across device, edge, and cloud tiers. Results [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 6
Figure 6. Figure 6: • Variation in output length distribution across tiers. The theoretical framework assumes uniform output length |y| across LLMs deployed on all tiers. However, actual output lengths may differ due to variations in LLM ca￾pacity, decoding strategies, or sample-specific …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 32 canonical work pages

  1. [1]

    A survey on end- edge-cloud orchestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet,

    J. Ren, D. Zhang, S. He, Y . Zhang, and T. Li, “A survey on end- edge-cloud orchestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet,”ACM Computing Surveys (CSUR), vol. 52, no. 6, pp. 1–36, 2019

    work page 2019

  2. [2]

    Dis- tributed artificial intelligence empowered by end-edge-cloud computing: A survey,

    S. Duan, D. Wang, J. Ren, F. Lyu, Y . Zhang, H. Wu, and X. Shen, “Dis- tributed artificial intelligence empowered by end-edge-cloud computing: A survey,”IEEE Communications Surveys & Tutorials, vol. 25, no. 1, pp. 591–624, 2022

    2022

  3. [3]

    End-edge-cloud collaborative computing for deep learning: A comprehensive survey,

    Y . Wang, C. Yang, S. Lan, L. Zhu, and Y . Zhang, “End-edge-cloud collaborative computing for deep learning: A comprehensive survey,” IEEE Communications Surveys & Tutorials, 2024

    work page 2024

  4. [4]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskeveret al., “Language models are unsupervised multitask learners,”OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  5. [5]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  6. [6]

    A comprehensive overview of large language models,

    H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian, “A comprehensive overview of large language models,”arXiv preprint arXiv:2307.06435, 2023

    arXiv 2023

  7. [7]

    Gpt-3.5 turbo: Legacy gpt model for cheaper chat and non-chat tasks,

    OpenAI, “Gpt-3.5 turbo: Legacy gpt model for cheaper chat and non-chat tasks,” 2022. [Online]. Available: https://platform.openai.com/ docs/models/gpt-3.5-turbo

    work page 2022

  8. [8]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language mod- els are few-shot learners,”Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

Show all 73 references
  1. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Biet al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,”arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  2. [10]

    Mobilellm: Optimizing sub- billion parameter language models for on-device use cases,

    Z. Liu, C. Zhao, F. Iandola, C. Lai, Y . Tian, I. Fedorov, Y . Xiong, E. Chang, Y . Shi, R. Krishnamoorthiet al., “Mobilellm: Optimizing sub- billion parameter language models for on-device use cases,” inForty-first International Conference on Machine Learning, 2024

    2024

  3. [11]

    Minicpm: Unveiling the potential of small language models with scalable training strategies,

    S. Hu, Y . Tu, X. Han, C. He, G. Cui, X. Long, Z. Zheng, Y . Fang, Y . Huang, W. Zhaoet al., “Minicpm: Unveiling the potential of small language models with scalable training strategies,”arXiv preprint arXiv:2404.06395, 2024

    2024 arXiv

  4. [12]

    {ServerlessLLM}:{Low-Latency}serverless inference for large language models,

    Y . Fu, L. Xue, Y . Huang, A.-O. Brabete, D. Ustiugov, Y . Patel, and L. Mai, “{ServerlessLLM}:{Low-Latency}serverless inference for large language models,” in18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), 2024, pp. 135–153

    2024

  5. [13]

    {DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving,

    Y . Zhong, S. Liu, J. Chen, J. Hu, Y . Zhu, X. Liu, X. Jin, and H. Zhang, “{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving,” in18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), 2024, pp. 193–210

    2024

  6. [14]

    Perllm: Personalized inference scheduling with edge-cloud collaboration for diverse llm services,

    Z. Yang, Y . Yang, C. Zhao, Q. Guo, W. He, and W. Ji, “Perllm: Personalized inference scheduling with edge-cloud collaboration for diverse llm services,”arXiv preprint arXiv:2405.14636, 2024

    2024 arXiv

  7. [15]

    Pushing large language models to the 6g edge: Vision, challenges, and opportunities,

    Z. Lin, G. Qu, Q. Chen, X. Chen, Z. Chen, and K. Huang, “Pushing large language models to the 6g edge: Vision, challenges, and opportunities,” arXiv preprint arXiv:2309.16739, 2023

    2023 arXiv

  8. [16]

    Efficient inference with model cascades,

    L. Lebovitz, L. Cavigelli, M. Magno, and L. K. Muller, “Efficient inference with model cascades,”Transactions on Machine Learning Research, 2023

    2023

  9. [17]

    Qwen2. 5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Weiet al., “Qwen2. 5 technical report,”arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  10. [18]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inPro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolog...

    2019

  11. [19]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,”arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  12. [20]

    Improving language understanding by generative pre-training,

    A. Radford, K. Narasimhan, T. Salimans, I. Sutskeveret al., “Improving language understanding by generative pre-training,” 2018

    2018

  13. [21]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  14. [22]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” inInternational conference on machine learning. PMLR, 2019, pp. 2790–2799

    2019

  15. [23]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Rayet al., “Training language models to follow instructions with human feedback,”Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022

    2022

  16. [24]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,”Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  17. [25]

    Dis- tributed artificial intelligence empowered by end-edge-cloud computing: A survey,

    S. Duan, D. Wang, J. Ren, F. Lyu, Y . Zhang, H. Wu, and X. Shen, “Dis- tributed artificial intelligence empowered by end-edge-cloud computing: A survey,”IEEE Communications Surveys & Tutorials, vol. 25, no. 1, pp. 591–624, 2023

    2023

  18. [26]

    Hierarchical federated learning with momentum acceleration in multi-tier networks,

    Z. Yang, S. Fu, W. Bao, D. Yuan, and A. Y . Zomaya, “Hierarchical federated learning with momentum acceleration in multi-tier networks,” IEEE Transactions on Parallel and Distributed Systems, 2023

    2023

  19. [27]

    Beyond model scale limits: End-edge-cloud federated learning with self-rectified knowledge agglomeration,

    Z. Wu, S. Sun, Y . Wang, M. Liu, K. Xu, Q. Pan, B. Gao, and T. Wen, “Beyond model scale limits: End-edge-cloud federated learning with self-rectified knowledge agglomeration,”arXiv preprint arXiv:2501.00693, 2025

    2025 arXiv

  20. [28]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,”IEEE internet of things journal, vol. 3, no. 5, pp. 637–646, 2016

    2016

  21. [29]

    Agglomerative federated learning: Empowering larger model training via end-edge-cloud collaboration,

    Z. Wu, S. Sun, Y . Wang, M. Liu, B. Gao, Q. Pan, T. He, and X. Jiang, “Agglomerative federated learning: Empowering larger model training via end-edge-cloud collaboration,” inIEEE INFOCOM 2024- IEEE Conference on Computer Communications. IEEE, 2024, pp. 131– 140

    2024

  22. [30]

    A survey on task offloading in multi-access edge computing,

    A. Islam, A. Debnath, M. Ghose, and S. Chakraborty, “A survey on task offloading in multi-access edge computing,”Journal of Systems Architecture, vol. 118, p. 102225, 2021

    2021

  23. [31]

    Task offloading in edge and cloud computing: A survey on mathe- matical, artificial intelligence and control theory solutions,

    F. Saeik, M. Avgeris, D. Spatharakis, N. Santi, D. Dechouniotis, J. Vio- los, A. Leivadeas, N. Athanasopoulos, N. Mitton, and S. Papavassiliou, “Task offloading in edge and cloud computing: A survey on mathe- matical, artificial intelligence and control theory solutions,”Compu...

    2021

  24. [32]

    Task offloading strategies for mobile edge computing: A survey,

    S. Dong, J. Tang, K. Abbas, R. Hou, J. Kamruzzaman, L. Rutkowski, and R. Buyya, “Task offloading strategies for mobile edge computing: A survey,”Computer Networks, p. 110791, 2024

    2024

  25. [33]

    Understanding softmax confidence and uncertainty,

    T. Pearce, A. Brintrup, and J. Zhu, “Understanding softmax confidence and uncertainty,”arXiv preprint arXiv:2106.04972, 2021

    2021 arXiv

  26. [34]

    Learning word vectors for sentiment analysis,

    A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y . Ng, and C. Potts, “Learning word vectors for sentiment analysis,” inProceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, 2011, pp. 142–150

    2011

  27. [35]

    Recursive deep models for semantic compositionality over a sentiment treebank,

    R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y . Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” inProceedings of the 2013 conference on empirical methods in natural language processing, 2013, pp. 1631–1642

    2013

  28. [36]

    Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales,

    B. Pang and L. Lee, “Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales,”arXiv preprint cs/0506075, 2005

    2005 arXiv

  29. [37]

    Character-level convolutional net- works for text classification,

    X. Zhang, J. Zhao, and Y . LeCun, “Character-level convolutional net- works for text classification,”Advances in neural information processing systems, vol. 28, 2015

    2015

  30. [38]

    Findings of the 2016 conference on machine translation (wmt16),

    O. Bojar, R. Chatterjee, C. Federmann, Y . Graham, B. Haddow, M. Huck, A. J. Yepes, P. Koehn, V . Logacheva, C. Monzet al., “Findings of the 2016 conference on machine translation (wmt16),” in First conference on machine translation. Association for Computational Linguistics, ...

    2016

  31. [39]

    Findings of the 2019 conference on machine translation (wmt19)

    L. Barrault, O. Bojar, M. R. Costa-Jussa, C. Federmann, M. Fishel, Y . Graham, B. Haddow, M. Huck, P. Koehn, S. Malmasiet al., “Findings of the 2019 conference on machine translation (wmt19).” ACL, 2019

    2019

  32. [40]

    Parallel data, tools and interfaces in opus

    J. Tiedemann, “Parallel data, tools and interfaces in opus.” inLrec, vol

  33. [41]

    Distilbert, a dis- tilled version of bert: smaller, faster, cheaper and lighter,

    V . Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a dis- tilled version of bert: smaller, faster, cheaper and lighter,”ArXiv, vol. abs/1910.01108, 2019

    1910 arXiv

  34. [42]

    azizbarank/distilroberta-base-sst2-distilled,

    azizbarank, “azizbarank/distilroberta-base-sst2-distilled,” 2022. [Online]. Available: https://huggingface.co/azizbarank/ distilroberta-base-sst2-distilled

    2022

  35. [43]

    textattack/roberta-base-sst-2,

    textattack, “textattack/roberta-base-sst-2,” 2023. [Online]. Available: https://huggingface.co/textattack/roberta-base-SST-2 UNDER REVIEW 17

    2023

  36. [44]

    howey/roberta-large-sst2,

    howey, “howey/roberta-large-sst2,” 2021. [Online]. Available: https: //huggingface.co/howey/roberta-large-sst2

    2021

  37. [45]

    Sebis/legal t5 small trans de en small finetuned,

    SEBIS, “Sebis/legal t5 small trans de en small finetuned,” 2021. [Online]. Available: https://huggingface.co/SEBIS/legal t5 small trans de en small finetuned

    2021

  38. [46]

    Opus-mt–building open translation services for the world,

    J. Tiedemann and S. Thottingal, “Opus-mt–building open translation services for the world,” inAnnual Conference of the European Asso- ciation for Machine Translation. European Association for Machine Translation, 2020, pp. 479–480

    2020

  39. [47]

    Pontifexmaximus/opus-mt-de-en-finetuned-de-to- en,

    PontifexMaximus, “Pontifexmaximus/opus-mt-de-en-finetuned-de-to- en,” 2022. [Online]. Available: https://huggingface.co/PontifexMaximus/ opus-mt-de-en-finetuned-de-to-en

    2022

  40. [48]

    Deep encoder, shallow decoder: Reevaluating non-autoregressive machine translation,

    J. Kasai, N. Pappas, H. Peng, J. Cross, and N. A. Smith, “Deep encoder, shallow decoder: Reevaluating non-autoregressive machine translation,” arXiv preprint arXiv:2006.10369, 2020

    2006 arXiv

  41. [49]

    allenai/wmt19-de-en-6-6-big,

    allenai, “allenai/wmt19-de-en-6-6-big,” 2023. [Online]. Available: https://huggingface.co/allenai/wmt19-de-en-6-6-big

    2023

  42. [50]

    Gemel: Model merging for {Memory-Efficient},{Real-Time}video analytics at the edge,

    A. Padmanabhan, N. Agarwal, A. Iyer, G. Ananthanarayanan, Y . Shu, N. Karianakis, G. H. Xu, and R. Netravali, “Gemel: Model merging for {Memory-Efficient},{Real-Time}video analytics at the edge,” in20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23),...

    2023

  43. [51]

    Deberta: Decoding-enhanced bert with disentangled attention,

    P. He, X. Liu, J. Gao, and W. Chen, “Deberta: Decoding-enhanced bert with disentangled attention,”arXiv preprint arXiv:2006.03654, 2020

    2006 arXiv

  44. [52]

    Tomor0720/deberta-large-finetuned-sst2,

    Tomor0720, “Tomor0720/deberta-large-finetuned-sst2,” 2023. [Online]. Available: https://huggingface.co/Tomor0720/ deberta-large-finetuned-sst2

    2023

  45. [53]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  46. [54]

    Putting gpt-4o to the sword: A comprehensive evaluation of language, vision, speech, and multimodal proficiency,

    S. Shahriar, B. D. Lund, N. R. Mannuru, M. A. Arshad, K. Hayawi, R. V . K. Bevara, A. Mannuru, and L. Batool, “Putting gpt-4o to the sword: A comprehensive evaluation of language, vision, speech, and multimodal proficiency,”Applied Sciences, vol. 14, no. 17, p. 7782, 2024

    2024

  47. [55]

    The Claude 3 Model Family: Opus, Sonnet, Haiku,

    Anthropic, “The Claude 3 Model Family: Opus, Sonnet, Haiku,” 2024. [Online]. Available: https://www-cdn.anthropic.com/ de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model Card Claude 3.pdf

    2024

  48. [56]

    Claude 3-5 sonnet,

    Anthropic, “Claude 3-5 sonnet,” 2024. [Online]. Available: https: //www.anthropic.com/news/claude-3-5-sonnet

    2024

  49. [57]

    Claude 3.7 sonnet and claude code,

    Anthropic, “Claude 3.7 sonnet and claude code,” 2025. [Online]. Available: https://www.anthropic.com/news/claude-3-7-sonnet

    2025

  50. [58]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azharet al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  51. [59]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  52. [60]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  53. [61]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huanget al., “Qwen technical report,”arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  54. [62]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Geet al., “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,”arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  55. [63]

    Qwen2. 5-vl technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tanget al., “Qwen2. 5-vl technical report,”arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  56. [64]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” inProceedings of the 29th Symposium on Operating Systems Principles, 2023, pp. 611–626

    2023

  57. [65]

    Cachegen: Kv cache compression and streaming for fast large language model serving,

    Y . Liu, H. Li, Y . Cheng, S. Ray, Y . Huang, Q. Zhang, K. Du, J. Yao, S. Lu, G. Ananthanarayananet al., “Cachegen: Kv cache compression and streaming for fast large language model serving,” inProceedings of the ACM SIGCOMM 2024 Conference, 2024, pp. 38–56

    2024

  58. [66]

    Adainf: Data drift adaptive scheduling for accurate and slo-guaranteed multiple-model inference serving at edge servers,

    S. S. Shubha and H. Shen, “Adainf: Data drift adaptive scheduling for accurate and slo-guaranteed multiple-model inference serving at edge servers,” inProceedings of the ACM SIGCOMM 2023 Conference, 2023, pp. 473–485

    2023

  59. [67]

    Edgeshard: Efficient llm inference via collaborative edge computing,

    M. Zhang, X. Shen, J. Cao, Z. Cui, and S. Jiang, “Edgeshard: Efficient llm inference via collaborative edge computing,”IEEE Internet of Things Journal, 2024

    2024

  60. [68]

    Split computing and early exiting for deep learning applications: Survey and research challenges,

    Y . Matsubara, M. Levorato, and F. Restuccia, “Split computing and early exiting for deep learning applications: Survey and research challenges,” ACM Computing Surveys, vol. 55, no. 5, pp. 1–30, 2022

    2022

  61. [69]

    Dis- tributed inference acceleration with adaptive dnn partitioning and of- floading,

    T. Mohammed, C. Joe-Wong, R. Babbar, and M. Di Francesco, “Dis- tributed inference acceleration with adaptive dnn partitioning and of- floading,” inIEEE INFOCOM 2020-IEEE conference on computer communications. IEEE, 2020, pp. 854–863

    2020

  62. [70]

    Mandheling: Mixed-precision on-device dnn training with dsp offloading,

    D. Xu, M. Xu, Q. Wang, S. Wang, Y . Ma, K. Huang, G. Huang, X. Jin, and X. Liu, “Mandheling: Mixed-precision on-device dnn training with dsp offloading,” inProceedings of the 28th Annual International Conference on Mobile Computing And Networking, 2022, pp. 214–227

    2022

  63. [71]

    Instinfer: In-storage attention offloading for cost-effective long-context llm inference,

    X. Pan, E. Li, Q. Li, S. Liang, Y . Shan, K. Zhou, Y . Luo, X. Wang, and J. Zhang, “Instinfer: In-storage attention offloading for cost-effective long-context llm inference,”arXiv preprint arXiv:2409.04992, 2024. Zhiyuan Wuis currently a research assistant with the Institute o...

    2024 arXiv

  64. [2012]

    2214–2218

    Citeseer, 2012, pp. 2214–2218

    2012

  65. [2014]

    He was a Visiting Researcher with the School of Com- puting and Communications at Lancaster University, Lancaster, UK from 2018 to 2019

    He was an Assistant Professor with the Insti- tute of Computing Technology at Chinese Academy of Sciences, Beijing, China from 2014 to 2017. He was a Visiting Researcher with the School of Com- puting and Communications at Lancaster University, Lancaster, UK from 2018 to 2019....

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.