REVIEW 8 cited by
PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
With the rapid growth in the number of large language model (LLM) users, it is difficult for bandwidth-constrained cloud servers to simultaneously process massive LLM services in real-time. Recently, edge-cloud infrastructures have been used to improve the processing efficiency of large-scale LLM services. However, the diversity of task requirements and the dynamics of resources pose great challenges to inference scheduling, leading to the wastage of many resources. In this paper, we present PerLLM, a personalized inference scheduling framework with edge-cloud collaboration designed for diverse LLM services. For the complexity of multiple constraints and the decision-making process of edge-cloud collaboration, we integrate the upper confidence bound algorithm based on the constraint satisfaction mechanism in PerLLM. For diverse LLM services, PerLLM can optimize service scheduling and resource allocation solutions within the edge-cloud infrastructure to meet processing time requirements while minimizing energy costs. Experimental results from different model deployments show that PerLLM can effectively meet the processing time requirements of personalized services. Compared to other methods, PerLLM achieves 2.2x, 2.1x, and 1.6x throughput and reduces the energy cost by more than 50%.
Forward citations
Cited by 8 Pith papers
-
Orchestration for Domain-specific Edge-Cloud Language Models
ECO-LLM jointly selects query processing, retrieval, and model components per query, cutting cost by 60% and latency up to 6x versus model routing in edge-cloud tests.
-
Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization
A hierarchical ARPO+LLaRA framework that jointly sets image resolution, transmit power, and UAV trajectory reduces simulated latency for onboard VLM inference in low-altitude economy networks.
-
Recursive Offloading for LLM Serving in Multi-tier Networks
RecServe routes LLM requests across device, edge, and cloud using adaptive confidence thresholds based on recent history, cutting communication by over 50% versus cloud-only serving while keeping quality close to cloud-level.
-
Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading
A test-time reinforcement learning framework for joint model caching and inference offloading is claimed to cut simulated long-context LLM serving costs at the mobile edge by at least 30%.
-
DeServe: Towards Affordable Offline LLM Inference via Decentralization
DeServe combines KV cache offloading and microbatch scheduling to maintain high LLM offline inference throughput in high-latency decentralized environments.
-
Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
A survey that builds a taxonomy of edge-cloud LLM-SLM collaboration for inference and training, claiming to be the first to unify both phases.
-
When IoT Meet LLMs: Applications and Challenges
A survey of LLM-IoT integration plus an unvalidated conceptual system model for Tree of Thought based predictive maintenance in industrial IoT.
-
From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications
This paper is a broad tutorial on applying LAMs and agentic AI to 6G, largely restating existing research rather than introducing new results.
Discussion (0). Continue with ORCID to comment.