Pith. sign in

REVIEW 5 cited by

PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.14636 v1 pith:HFKO2EKQ submitted 2024-05-23 cs.DC cs.NI

Zheming Yang , Yuanhao Yang , Chang Zhao , Qi Guo , Wenkai He , Wen Ji This is my paper · ORCID
classification cs.DCcs.NI
keywords perllmservicesedge-cloudschedulingcollaborationdiverseinferencepersonalized
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

With the rapid growth in the number of large language model (LLM) users, it is difficult for bandwidth-constrained cloud servers to simultaneously process massive LLM services in real-time. Recently, edge-cloud infrastructures have been used to improve the processing efficiency of large-scale LLM services. However, the diversity of task requirements and the dynamics of resources pose great challenges to inference scheduling, leading to the wastage of many resources. In this paper, we present PerLLM, a personalized inference scheduling framework with edge-cloud collaboration designed for diverse LLM services. For the complexity of multiple constraints and the decision-making process of edge-cloud collaboration, we integrate the upper confidence bound algorithm based on the constraint satisfaction mechanism in PerLLM. For diverse LLM services, PerLLM can optimize service scheduling and resource allocation solutions within the edge-cloud infrastructure to meet processing time requirements while minimizing energy costs. Experimental results from different model deployments show that PerLLM can effectively meet the processing time requirements of personalized services. Compared to other methods, PerLLM achieves 2.2x, 2.1x, and 1.6x throughput and reduces the energy cost by more than 50%.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Orchestration for Domain-specific Edge-Cloud Language Models

    cs.DB 2025-07 conditional novelty 6.0 of 10

    ECO-LLM jointly selects query processing, retrieval, and model components per query, cutting cost by 60% and latency up to 6x versus model routing in edge-cloud tests.

  2. Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization

    cs.LG 2025-10 conditional novelty 5.0 of 10

    A hierarchical ARPO+LLaRA framework that jointly sets image resolution, transmit power, and UAV trajectory reduces simulated latency for onboard VLM inference in low-altitude economy networks.

  3. Recursive Offloading for LLM Serving in Multi-tier Networks

    cs.DC 2025-05 conditional novelty 5.0 of 10

    RecServe routes LLM requests across device, edge, and cloud using adaptive confidence thresholds based on recent history, cutting communication by over 50% versus cloud-only serving while keeping quality close to cloud-level.

  4. Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges

    cs.DC 2025-07 conditional novelty 4.0 of 10

    A survey that builds a taxonomy of edge-cloud LLM-SLM collaboration for inference and training, claiming to be the first to unify both phases.

  5. From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications

    cs.AI 2025-05 conditional novelty 2.0 of 10

    This paper is a broad tutorial on applying LAMs and agentic AI to 6G, largely restating existing research rather than introducing new results.

Pith tools