Pith. sign in

REVIEW 17 cited by

GraphRouter: A Graph-based Router for LLM Selections

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.03834 v2 pith:KJH4VWYR submitted 2024-10-04 cs.AI

Tao Feng , Yanzhen Shen , Jiaxuan You This is my paper · ORCID
classification cs.AI
keywords graphrouterllmscontextualtasksacrossqueryselectioncomputational
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The rapidly growing number and variety of Large Language Models (LLMs) present significant challenges in efficiently selecting the appropriate LLM for a given query, especially considering the trade-offs between performance and computational cost. Current LLM selection methods often struggle to generalize across new LLMs and different tasks because of their limited ability to leverage contextual interactions among tasks, queries, and LLMs, as well as their dependence on a transductive learning framework. To address these shortcomings, we introduce a novel inductive graph framework, named as GraphRouter, which fully utilizes the contextual information among tasks, queries, and LLMs to enhance the LLM selection process. GraphRouter constructs a heterogeneous graph comprising task, query, and LLM nodes, with interactions represented as edges, which efficiently captures the contextual information between the query's requirements and the LLM's capabilities. Through an innovative edge prediction mechanism, GraphRouter is able to predict attributes (the effect and cost of LLM response) of potential edges, allowing for optimized recommendations that adapt to both existing and newly introduced LLMs without requiring retraining. Comprehensive experiments across three distinct effect-cost weight scenarios have shown that GraphRouter substantially surpasses existing routers, delivering a minimum performance improvement of 12.3%. In addition, it achieves enhanced generalization across new LLMs settings and supports diverse tasks with at least a 9.5% boost in effect and a significant reduction in computational demands. This work endeavors to apply a graph-based approach for the contextual and adaptive selection of LLMs, offering insights for real-world applications. Our codes for GraphRouter is released at https://github.com/ulab-uiuc/GraphRouter.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing

    cs.LG 2026-07 conditional novelty 7.0 of 10

    CABS-C and CABS-D use correlation graphs plus surrogate rewards to cut effective exploration in contextual bandits for LLM routing, with CABS-D giving best-of-both-worlds regret and better empirical accuracy-cost frontiers.

  2. TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning

    cs.LG 2026-07 conditional novelty 6.5 of 10

    A heterogeneous-graph router jointly selects the optimal modality (text, vision, or both) and model per time series query, beating prior routing baselines and generalizing to unseen models and tasks.

  3. LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

    cs.CL 2026-08 conditional novelty 6.0 of 10

    A new open-source library and benchmark, xRouteBench, evaluates LLM routers on a shared cost-aware protocol across text, memory, vision, time-series, and personalized tasks.

  4. R2-Router: A New Paradigm for LLM Routing with Reasoning

    cs.CL 2026-02 conditional novelty 6.0 of 10

    R2-Router jointly selects the LLM and an output-token budget, modeling each model as a quality-cost curve rather than a fixed point, and reports 4-5x cost savings on its new R2-Bench.

  5. Adaptive LLM Routing under Budget Constraints

    cs.LG 2025-08 conditional novelty 6.0 of 10

    LLM routing is framed as a budget-constrained contextual bandit, solved by a preference-prior initialized LinUCB variant with an online multi-choice knapsack cost policy.

  6. Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms

    cs.MA 2025-08 conditional novelty 6.0 of 10

    Murakkab uses declarative workflow specs and a profile-guided MILP optimizer to reduce GPU, energy, and cost for agentic workflow serving while meeting percentile-defined SLOs.

  7. Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts

    cs.NI 2025-08 conditional novelty 6.0 of 10

    A DRL router using graph attention state abstraction and QoS-aware rewards improves average QoS by up to 35.78% over four baselines in simulated edge LLM routing.

  8. IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory

    cs.AI 2025-06 conditional novelty 6.0 of 10

    An IRT-based router that models each LLM's latent ability and each query's difficulty outperforms RouterBench on cost-performance reward across ID and OOD benchmarks.

  9. When One LLM Drools, Multi-LLM Collaboration Rules

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A position paper that introduces a four-level taxonomy of multi-LLM collaboration (API, text, logit, weight) and argues it is essential for reliability, pluralism, and democratization.

  10. Rerouting LLM Routers

    cs.CR 2025-01 conditional novelty 6.0 of 10

    Adversarially optimized, query-independent token prefixes can reroute nearly all queries to the expensive strong model in both open-source and commercial LLM routers.

  11. Towards Generalized Routing: Model and Agent Orchestration for Adaptive and Efficient Inference

    cs.MA 2025-09 reject novelty 5.0 of 10

    MoMA routes each query to a specialized agent or to the cost-optimal LLM, using judge-trained performance scores, a Pareto cost frontier, and TOPSIS selection.

  12. LightRouter: Towards Efficient LLM Collaboration with Minimal Overhead

    cs.AI 2025-05 conditional novelty 5.0 of 10

    LightRouter uses short preview outputs to filter a pool of LLMs down to two, then aggregates their full responses, beating ensemble baselines and matching costlier models.

  13. Universal Model Routing for Efficient LLM Inference

    cs.CL 2025-02 conditional novelty 5.0 of 10

    UniRoute represents each language model by its error rates on a few prompt clusters, letting a router choose among models it has never seen during training.

  14. Bench-CoE: a Framework for Collaboration of Experts from Benchmark

    cs.AI 2024-12 conditional novelty 5.0 of 10

    Bench-CoE trains a router on benchmark subject scores to send each query to the best-performing expert model, beating individual models in most but not all tested settings.

  15. Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes

    cs.CL 2026-08 conditional novelty 4.0 of 10

    Statistical classifiers built on LLM activation norms and coordinates match or beat trained MLP heads on coarse intent routing and resist camouflage better, while MLPs win on fine-grained subfield distinctions.

  16. CoE-Ops: Collaboration of LLM-based Experts for AIOps Question-Answering

    cs.CL 2025-07 reject novelty 4.0 of 10

    CoE-Ops routes DevOps questions to specialized LLM experts using an LLM classifier plus retrieval, reporting gains on DevOps-Eval that are compromised by possible test-set leakage.

  17. ORI: O Routing Intelligence

    cs.CL 2025-02 reject novelty 3.0 of 10

    ORI routes queries by embedding cluster to the best model for the cluster's dominant benchmark, reporting modest gains that are not supported by its own routing rule or evaluation protocol.

Pith tools