REVIEW 8 cited by
Cost-Effective Online Multi-LLM Selection with Versatile Reward Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
With the rapid advancement of large language models (LLMs), the diversity of multi-LLM tasks and the variability in their pricing structures have become increasingly important, as costs can vary greatly between different LLMs. To tackle these challenges, we introduce the \textit{C2MAB-V}, a \underline{C}ost-effective \underline{C}ombinatorial \underline{M}ulti-armed \underline{B}andit with \underline{V}ersatile reward models for optimal LLM selection and usage. This online model differs from traditional static approaches or those reliant on a single LLM without cost consideration. With multiple LLMs deployed on a scheduling cloud and a local server dedicated to handling user queries, \textit{C2MAB-V} facilitates the selection of multiple LLMs over a combinatorial search space, specifically tailored for various collaborative task types with different reward models. Based on our designed online feedback mechanism and confidence bound technique, \textit{C2MAB-V} can effectively address the multi-LLM selection challenge by managing the exploration-exploitation trade-off across different models, while also balancing cost and reward for diverse tasks. The NP-hard integer linear programming problem for selecting multiple LLMs with trade-off dilemmas is addressed by: i) decomposing the integer problem into a relaxed form by the local server, ii) utilizing a discretization rounding scheme that provides optimal LLM combinations by the scheduling cloud, and iii) continual online updates based on feedback. Theoretically, we prove that \textit{C2MAB-V} offers strict guarantees over versatile reward models, matching state-of-the-art results for regret and violations in some degenerate cases. Empirically, we show that \textit{C2MAB-V} effectively balances performance and cost-efficiency with nine LLMs for three application scenarios.
Forward citations
Cited by 8 Pith papers
-
Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing
CABS-C and CABS-D use correlation graphs plus surrogate rewards to cut effective exploration in contextual bandits for LLM routing, with CABS-D giving best-of-both-worlds regret and better empirical accuracy-cost frontiers.
-
LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters
A per-query orchestrator that jointly picks LLM family, size, quantization, and edge device via fitted predictors and a BILP-based heuristic reduces measured latency and keeps accuracy stable versus two simpler baselines.
-
IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory
An IRT-based router that models each LLM's latent ability and each query's difficulty outperforms RouterBench on cost-performance reward across ID and OOD benchmarks.
-
Cost and Accuracy of Long-Term Memory in Distributed Multi-Agent Systems Based on Large Language Models
A two-framework testbed comparison claims mem0 is Pareto-optimal over Graphiti for distributed LLM agents because its lower cost is paired with accuracy that is not significantly different.
-
How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMs
A leader LLM trained with a GRPO variant that conditions on frozen agent responses improves both collaborative and zero-shot accuracy on BBH, MATH, and MMLU.
-
A Unified Online-Offline Framework for Co-Branding Campaign Recommendations
A unified framework for co-branding learns partner success probabilities and market gains online, and allocates sub-brand budgets offline with a 1-1/e approximation guarantee.
-
Toward Edge General Intelligence with Multiple-Large Language Model (Multi-LLM): Architecture, Trust, and Orchestration
A survey of multi-LLM systems in edge computing, covering architectures, enabling technologies, trust mechanisms, applications, and open datasets for edge general intelligence.
-
Software Engineering for Large Language Models: Research Status, Challenges and the Road Ahead
A literature review organizes LLM development into a six-phase software engineering lifecycle and identifies challenges and research directions for each phase.
Discussion (0). Sign in to comment.