WISERouter: LLM Routing with Workload Budget Constraint

Laks V.S. Lakshmanan; Yifei Li; Zihui Gao

LLM routing under a single workload budget can be cast as a constrained contextual bandit and solved with adaptive linear programming, meeting the global cost cap without per-budget retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 13:15 UTC pith:TYHCSVL6

load-bearing objection Useful workload-budget routing with a real but incomplete end-to-end regret story—the ordering lemma is fine; equating it to known-stats ALP is not. the 1 major comments →

arxiv 2607.23765 v1 pith:TYHCSVL6 submitted 2026-07-26 cs.LG cs.AI

WISERouter: LLM Routing with Workload Budget Constraint

Yifei Li , Zihui Gao , Laks V.S. Lakshmanan This is my paper

classification cs.LG cs.AI

keywords LLM routingworkload budgetconstrained contextual banditadaptive linear programmingonline learningcost-aware inferenceregret bound

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Using the strongest language model for every query is too expensive at scale, yet fixed per-query cost caps and dense supervised training leave money on the table and require expensive data. This paper argues that routing should instead maximize total quality subject to one expected cost budget over the whole workload. It turns that problem into a constrained contextual bandit: each query embedding is a context, each model is an arm, and quality and cost are the observed reward and spend. After clustering embeddings into a finite context set, Adaptive Linear Programming converts remaining budget into a per-step average and picks a model without retraining when the budget changes. An offline version uses historical statistics; an online version explores under the same shared budget and is proved to have sublinear regret. On RouterBench and SWE-Bench the offline router beats prior selectors at tight budgets and stays closer to the cap, while the online router matches them with far less training data.

Core claim

Workload-constrained LLM routing is a constrained contextual bandit that Adaptive Linear Programming can solve: WR-Offline estimates reward and cost once from history and re-solves a lightweight LP at each step so any new global budget is met without retraining, and WR-Online jointly learns those statistics under one shared budget with end-to-end regret O(√T) (O(log T) off boundary points) when exploration is long enough to recover the true reward-cost ordering.

What carries the argument

Adaptive Linear Programming (ALP): at each step it solves a small LP that turns remaining budget b over remaining rounds τ into the average constraint b/τ, then samples a model from the resulting probabilities; WR-Online wraps ALP in an ε-first exploration phase that estimates both rewards and costs under the same budget B.

Load-bearing premise

After embedding and clustering, queries in the same cluster are treated as sharing the same expected quality and cost for each model, and those clusters arrive independently from a fixed distribution over a known finite horizon.

What would settle it

On a held-out stream with the same models and a fixed workload budget B, measure whether WR-Offline’s total spend stays within B while average quality exceeds FrugalGPT, CascadeRouting, MetaLLM, and SingleBest at the tightest budgets; if it overspends or underperforms those baselines, or if raising intra-cluster reward variance collapses the gains, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Operators can change monthly API budgets at inference time without recollecting responses or retraining a router.
A single shared budget can cover both exploration and deployment, so online routers need far less than a full query–model response matrix.
At tight budgets, allocating expensive models only to hard clusters raises average quality more than any fixed per-query cap.
End-to-end regret of order √T gives a concrete sample-complexity target for how long exploration must run before exploitation.
The same ALP template applies to non-monetary costs such as latency or energy once they are normalized per query.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If query traffic drifts by time of day or product surface, periodically re-estimating only the cluster frequencies (not full model statistics) may restore budget adherence without a full re-exploration.
The same discretization-plus-ALP pattern could route among speculative-decoding, quantized, or cascade configurations rather than whole models.
When several models sit on nearly identical cost tiers, the ratio-ordering test becomes the practical bottleneck; cheaper pairwise A/B probes on the hardest clusters may shorten exploration more than uniform least-played sampling.
Workloads with strong query dependence (multi-turn chats) would need a contextual or restless extension before the stated regret bound can be claimed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful workload-budget routing with a real but incomplete end-to-end regret story—the ordering lemma is fine; equating it to known-stats ALP is not.

read the letter

Punchline: the practical framing is good and the offline system is clean; the headline O(√T) claim for WR-Online is not fully closed. A referee will care about both.

What is actually new is not “bandits for routing” (MetaLLM, PILOT, etc. already live there). It is (i) enforcing a single workload-level expected budget via adaptive LP so you can change B at inference without retraining, and (ii) pushing Wu et al.’s ε-first ALP into the regime where both reward and heterogeneous cost are learned online under that same shared budget, with an explicit ordering lemma via cross-products. WR-Offline’s budget adherence and the no-retrain story are the parts I would actually use. Experiments on RouterBench/SWE-Bench are honest enough: means±std, remaining-budget plots, and a clear win at tight budgets; the 90% data cut for WR-Online is the right comparison axis.

Soft spots, in proportion. The stress-test on Theorem 5.3 lands. Lemma 5.2 only buys correct ξ-ordering w.h.p. under O(log T) exploration; Theorem 5.1 needs known (u,c). The sketch jumps from “order correct” to “ALP regret applies.” Residual error is controlled to a fixed radius t ~ L, not vanishing in T, so solution-value perturbation and the fact that Algorithms 1–2 debit b by ĉ rather than realized S are not bounded. That can spoil both the claimed O(log T)/O(√T) and feasibility w.r.t. E[Σ S]. This is a writeup/proof gap, not a fake result—but it is load-bearing for the main theoretical claim. Secondary: cluster homogeneity is assumed while intra-cluster reward variance is ~0.2; CascadeRouting gets noisified ground truth on RouterBench; CLT is derived then abandoned. None of that sinks the method paper.

Math chain through the oracle bound and the ratio lemma looks auditable; citation pattern is appropriate (Wu, Badanidiyuru, FrugalGPT, RouteLLM). No code/artifacts in the text.

Who it is for: people building multi-LLM gateways under monthly quotas, and bandit folks who care about shared-budget exploration. Worth a serious referee. I would engage, ask them to either bound the estimation gap through the LP and the true-cost constraint or soften the end-to-end claim to “ordering + empirical.” Not a desk reject.

Referee Report

1 major / 0 minor

Summary. The paper formulates workload-budget-constrained LLM routing as a constrained contextual bandit (CCB): query embeddings are clustered into a finite context set, and Adaptive Linear Programming (ALP, Wu et al. 2015) converts the remaining workload budget into a per-round constraint at each step. Two variants are proposed: WR-Offline, which estimates cluster-level reward/cost statistics from a dense historical dataset and adapts to new budgets without retraining; and WR-Online, an ε-first scheme that jointly estimates rewards and costs under the same shared budget, then exploits via ALP. The main theoretical claim is Theorem 5.3: end-to-end regret of O(log T) (non-boundary) and O(√T) (boundary) under a sufficient exploration condition (Lemma 5.2), whose proof uses a cross-product sign test to avoid random denominators. Experiments on RouterBench and SWE-Bench show WR-Offline competitive or better than FrugalGPT, CascadeRouting, MetaLLM, and SingleBest at tight budgets, with near-exact budget utilization and ~5,900× lower training latency; WR-Online matches offline methods with ~90% less supervision data.

Significance. If the theory is repaired, the contribution is meaningful: the workload-level (rather than per-query) budget formulation is well-motivated for API-quota deployments, the ALP reduction gives a practical, retraining-free mechanism for adapting to new budgets, and the extension of ε-first ALP to jointly unknown rewards and costs under a single shared budget — including the cross-product concentration argument in Lemma 5.2 and explicit failure-probability bookkeeping — is a genuine technical step beyond Wu et al. [40], which assumes known costs. The empirical package is solid for a systems-oriented claim: two public benchmarks, fixed 50:50 splits, five seeds with means±std, budget-utilization analysis, latency comparisons, a static-LP ablation isolating the adaptive constraint, robustness ablations over J, embeddings, and clustering algorithms, and a controlled simulation. The 90% supervision reduction for WR-Online and the 5,900× training-latency improvement are practically significant. However, the headline theoretical claim (Theorem 5.3) currently has a load-bearing gap, detailed in Major Comment 1, which tempers the overall assessment until addressed.

major comments (1)

§5, proof sketch of Theorem 5.3 (and its reliance on Theorem 5.1/D.1): the inference 'correct ordering of ξ ⇒ exploitation incurs the regret of Theorem 5.1' does not hold as stated, and the missing term appears to be O(T), not negligible. Theorem 5.1 is explicitly for ALP with KNOWN statistics: its regret is E[v(ρ)] − E[v(b/τ)] where v is evaluated at the true (u,c). Under Lemma 5.2's good event, estimates concentrate only to the constant radius t = L/(8√2) (exploration is O(log T), so per-pair error does not vanish with T). With correct ordering, the greedy threshold structure is preserved, but the fractional boundary allocation p̃ = (b/τ − Q̃_ĩ)/(π_j c̃) and the cumulative masses Q̃_i depend on the VALUES of c̄, not only their order, and Algorithm 2 freezes (ū,c̄) after exploration. A within-radius perturbation therefore shifts the per-round realized expected reward by O(t·Δη/(π_min·Δc

Circularity Check

0 steps flagged

No significant circularity: regret and empirical claims are not forced by definition or self-citation chains.

full rationale

WISERouter applies external constrained-contextual-bandit machinery (ALP / ε-first ALP from Wu et al. [40], non-overlapping authors) to LLM routing. The oracle upper bound bU(T,B)=Tv(ρ) is the standard LP relaxation of any feasible policy’s expected reward under E[∑St]≤B (Appendix C), not a quantity defined from the algorithm’s own outputs. Regret is the gap to an oracle with true (uj,k,cj,k); WR-Online’s Lemma 5.2 only certifies recovery of the ξ-ordering from finite samples, and Theorem 5.3 claims inheritance of ALP’s known-statistics bound—whether that inheritance is fully rigorous is a correctness/gap question (estimation error in values, not only order), not circularity. Offline/online statistics are estimated from historical or exploration feedback and evaluated on held-out RouterBench/SWE-Bench scores and fixed API prices; nothing is fitted to equal the claimed optimum by construction. Cluster means as context-action statistics and i.i.d. πj are modeling assumptions, not self-definitional loops. No load-bearing uniqueness theorem or ansatz is imported from the present authors. Empty steps; score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 2 invented entities

The load-bearing content is a constrained-bandit reduction plus clustering approximation, not new physical entities. Theory inherits standard concentration and ALP structure; practice adds embedding/clustering choices and a hand-set exploration schedule because the paper’s own confidence test is too conservative for LLM cost gaps.

free parameters (4)

Number of clusters J = 16 (main experiments)
Discretization granularity for ALP’s finite context space; chosen on validation (J=16). Ablations show similar reward for J in {8,12,16,20} but the finite-J model is still a fitted structural choice.
Exploration length ε(T) = 1×–4× |train| (experiments); condition (5) in theory
Theory gives a sufficient lower bound in terms of unknown Δc_min, Δξ_min, π_min; experiments instead use fixed 1×–4× training-set size because the confidence-level test is impractical.
MetaLLM Lagrangian penalty p (baseline) = grid-searched per budget level
Tuned per budget on held-out validation so comparisons depend on this grid search; not part of WR itself but affects claimed margins.
Embedding model and dimension = OpenAI text-embedding-3-small, d=1536
text-embedding-3-small (d=1536) defines the geometry that clustering and contexts inherit; ablated vs Qwen3-Embedding-0.6B with optional PCA.

axioms (6)

domain assumption Contexts (queries) are i.i.d. from a stationary discrete distribution {π_j} over a known finite horizon T.
Stated in Sections 3–4 and Appendix A; required for Hoeffding bounds on N_j and for regret vs LPT,B.
domain assumption For each LLM k, all queries in cluster j share the same expected reward u_{j,k} and expected cost c_{j,k}.
Section 4.1 homogeneity assumption that makes finite-context ALP well-defined after embedding clustering.
domain assumption Optimize expected cumulative reward subject to expected cumulative cost ≤ B (not pathwise almost-sure budget).
Section 3.1; follows prior LLM selection work but allows realized spend to fluctuate around B.
ad hoc to paper c_{j,k1} ≠ c_{j,k2} for k1 ≠ k2 so reward-cost ratios ξ are well-defined; costs/rewards in [0,1] after normalization.
Used in Lemma 5.2 and ratio ordering; realistic for distinct price tiers but fails if two models tie on expected cost in a cluster.
standard math Standard Hoeffding/Azuma concentration and the ALP regret structure of Wu et al. (2015) for known statistics.
Appendices D–F build the unknown-cost extension on these classical inequalities and the known-cost ALP theorem.
domain assumption Bandit feedback only: (Y_t, S_t) observed solely for the chosen model; quality scores available offline or during exploration.
Section 3.1 and Appendix A; quality may be human preference, unit tests, or proxies—assumed usable without blocking.

invented entities (2)

WISERouter (WR-Offline / WR-Online) framework no independent evidence
purpose: Name the full pipeline: embed→cluster→estimate (u,c)→solve adaptive LP under remaining b/τ, with optional ε-first exploration under shared B.
Engineering composition of known ALP and bandit ideas specialized to LLM routing; not a new physical entity. independent_evidence is empirical benchmark performance, not an external discovery handle.
Confidence Level Test (CLT) for stopping exploration no independent evidence
purpose: Certify that empirical ξ orderings match true orderings w.h.p. before exploitation (Algorithm 3, Appendix G).
Derived sufficient test; authors report it is too conservative for near-tied LLM prices and do not use it in main experiments.

pith-pipeline@v1.2.0-grok45-kimik3 · 30416 in / 4096 out tokens · 76008 ms · 2026-07-30T13:15:30.065317+00:00 · methodology

0 comments

read the original abstract

Large language models (LLMs) achieve impressive performance across multiple domains, but using the most capable model for every query is prohibitive at scale. LLM routing exploits diversity in model capability and cost by assigning each query to a suitable model to balance utility and budget. Current methods have two limitations: (i) they either use heuristics that do not always enforce the budget constraint or impose a fixed per-query budget that cannot adapt across the workload and leads to suboptimal performance; (ii) they require supervised learning on a dense dataset with statistics for every query-model pair, which is expensive to collect. To address these challenges, we formulate LLM routing as a constrained contextual multi-armed bandit problem and introduce WISERouter (WR for short), a framework that supports offline learning from historical interactions as well as online learning with exploration. We further prove that WR-Online achieves a sublinear regret bound of $O(\sqrt{T})$ over a time horizon $T$. Empirical results on RouterBench and SWE-Bench demonstrate that (i) WR-Offline surpasses existing baselines in performance under a fixed budget and adheres more closely to budget constraints, and (ii) WR-Online achieves comparable performance to the baselines, while using substantially less exploration data.

Figures

Figures reproduced from arXiv: 2607.23765 by Laks V.S. Lakshmanan, Yifei Li, Zihui Gao.

**Figure 2.** Figure 2: Overview of WISERouter workflow. 4.1 Context Discretization For the CCB formulation to be meaningful, the context must encode query-level features that explain the variation in LLM performance across queries. Since different LLMs can perform unevenly across query semantics [38], we use a pretrained encoder g(·) to map each query q ∈ Q to an embedding vector x = g(q) ∈ R d . However, ALP assumes a finite co… view at source ↗

**Figure 3.** Figure 3: Remaining budget ratios at each budget level. Error bars show [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

**Figure 4.** Figure 4: Budget consumption vs. reward gap for the static variant on SWE-Bench (5 runs); leftmost point is under the largest budget constraint. 200 400 600 Total Cost ($) 0.0 0.2 0.4 0.6 0.8 1.0 Average Reward per Query 200 300 400 Total Cost ($) SingleBest FrugalGPT CascadeRouting WR-Offline MetaLLM WR-Online (1x) WR-Online (2x) WR-Online (3x) WR-Online (4x) (a) Left: generous budget. Right: constrained budget. R… view at source ↗

**Figure 5.** Figure 5: (a) WR-Online vs. offline baselines on RouterBench; vertical dashed lines mark offline [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗

**Figure 6.** Figure 6: Performance comparison under end-to-end budget constraints on SWE-Bench. Comparison [PITH_FULL_IMAGE:figures/full_fig_p024_6.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

46 extracted references · 3 linked inside Pith

[1]

Claude Team

Anthropic. Claude Team. https://claude.com/pricing/team, 2026. Accessed: 2026-04- 28

2026
[2]

Finite-time analysis of the multiarmed bandit problem.Mach

Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem.Mach. Learn., 47(2–3):235–256, May 2002. ISSN 0885-6125. doi: 10.1023/A: 1013689704352. URLhttps://doi.org/10.1023/A:1013689704352

doi:10.1023/a: 2002
[3]

Resourceful contextual bandits

Ashwinkumar Badanidiyuru, John Langford, and Aleksandrs Slivkins. Resourceful contextual bandits. In Maria Florina Balcan, Vitaly Feldman, and Csaba Szepesvári, editors,Proceedings of The 27th Conference on Learning Theory, volume 35 ofProceedings of Machine Learning Research, pages 1109–1134, Barcelona, Spain, 13–15 Jun 2014. PMLR

2014
[4]

PyramidKV: Dynamic KV cache compression based on pyramidal information funneling

Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, and Wen Xiao. PyramidKV: Dynamic KV cache compression based on pyramidal information funneling. InSecond Conference on Language Modeling, 2025. URLhttps://openreview.net/forum?id=ayi7qezU87

2025
[5]

Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023

Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023

Pith/arXiv arXiv 2023
[6]

FrugalGPT: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research,

Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research,
[7]

Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I

Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios N. Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and Ion Stoica. Chatbot arena: an open platform for evaluating llms by human preference. InProceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024

2024
[8]

A unified approach to routing and cascading for LLMs

Jasper Dekoninck, Maximilian Baader, and Martin Vechev. A unified approach to routing and cascading for LLMs. InForty-second International Conference on Machine Learning, 2025. URLhttps://openreview.net/forum?id=AAl89VNNy1

2025
[9]

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks V . S. Lakshmanan, and Ahmed Hassan Awadallah. Hybrid LLM: Cost-efficient and quality- aware query routing. InThe Twelfth International Conference on Learning Representations,
[10]

Dujian Ding, Ankur Mallick, Shaokun Zhang, Chi Wang, Daniel Madrigal, Mirian Del Car- men Hipolito Garcia, Menglin Xia, Laks V . S. Lakshmanan, Qingyun Wu, and Victor Rühle. BEST-route: Adaptive LLM routing with test-time optimal compute. InForty-second Inter- national Conference on Machine Learning, 2025. URLhttps://openreview.net/forum? id=tFBIbCVXkG

2025
[11]

Multi-armed bandit with budget constraint and variable costs

Wenkui Ding, Tao Qiny, Xu-Dong Zhang, and Tie-Yan Liu. Multi-armed bandit with budget constraint and variable costs. InProceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, AAAI’13, page 232–238. AAAI Press, 2013

2013
[12]

Switch transformers: scaling to trillion parameter models with simple and efficient sparsity.J

William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: scaling to trillion parameter models with simple and efficient sparsity.J. Mach. Learn. Res., 23(1), January 2022. ISSN 1532-4435

2022
[13]

Language model cascades: Token-level uncertainty and beyond

Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. Language model cascades: Token-level uncertainty and beyond. InThe Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=KgaBScZ4VI

2024
[14]

Hassibi, D.G

B. Hassibi, D.G. Stork, and G.J. Wolff. Optimal brain surgeon and general network pruning. InIEEE International Conference on Neural Networks, pages 293–299 vol.1, 1993. doi: 10.1109/ICNN.1993.298572. 10

arXiv 1993
[15]

Routerbench: A benchmark for multi-llm routing system.CoRR, abs/2403.12031, 2024

Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system.CoRR, abs/2403.12031, 2024. doi: 10.48550/ARXIV .2403.12031. URL https://doi.org/10.48550/arXiv.2403.12031

Pith/arXiv arXiv doi:10.48550/arxiv.2403.12031 2024
[16]

Thriftllm: On cost-effective selection of large language models for classification queries.Proc

Keke Huang, Yimin Shi, Dujian Ding, Yifei Li, Yang Fei, Laks Lakshmanan, and Xiaokui Xiao. Thriftllm: On cost-effective selection of large language models for classification queries.Proc. VLDB Endow., 18(11):4410–4423, September 2025. ISSN 2150-8097. doi: 10.14778/3749646. 3749702. URLhttps://doi.org/10.14778/3749646.3749702

arXiv 2025
[17]

Ensemble learning for heterogeneous large language models with deep parallel collaboration

Yi-Chong Huang, Xiaocheng Feng, Baohang Li, Yang Xiang, Hui Wang, Bing Qin, and Ting Liu. Ensemble learning for heterogeneous large language models with deep parallel collaboration. In Neural Information Processing Systems, 2024. URL https://api.semanticscholar.org/ CorpusID:269282634

2024
[18]

Quantization and training of neural networks for efficient integer-arithmetic-only inference

Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2704–2713, 2018. doi: 10.1109/CVPR.2018.00286

arXiv 2018
[19]

LLM-blender: Ensembling large language models with pairwise ranking and generative fusion

Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14165–14178, Toronto, Canada, July 20...

doi:10.18653/v1/2023.acl-long.792 2023
[20]

SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024

Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=VTF8yNQM66

2024
[21]

S3: increasing gpu utilization during generative inference for higher throughput

Yunho Jin, Chun-Feng Wu, David Brooks, and Gu-Yeon Wei. S3: increasing gpu utilization during generative inference for higher throughput. InProceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA, 2023. Curran Associates Inc

2023
[22]

Optimal brain damage

Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. In D. Touretzky, editor, Advances in Neural Information Processing Systems, volume 2. Morgan-Kaufmann, 1989

1989
[23]

Fast inference from transformers via speculative decoding

Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. InInternational Conference on Machine Learning, pages 19274–19286. PMLR, 2023

2023
[24]

Schapire

Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual-bandit approach to personalized news article recommendation. InProceedings of the 19th International Conference on World Wide Web, WWW ’10, page 661–670, New York, NY , USA, 2010. Association for Computing Machinery. ISBN 9781605587998. doi: 10.1145/1772690.1772758

arXiv 2010
[25]

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu ...

2023
[26]

Routing to the expert: Efficient reward-guided ensemble of large language models

Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. Routing to the expert: Efficient reward-guided ensemble of large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors,Proceedings of the 2024 Conference of the 11 North American Chapter of the Association for Computational Linguistics: Human Langu...

2024
[27]

Some methods of classification and analysis of multivariate observations

James B McQueen. Some methods of classification and analysis of multivariate observations. InProc. of 5th Berkeley Symposium on Math. Stat. and Prob., pages 281–297, 1967

1967
[28]

Omnirouter: Budget and performance controllable multi-llm routing.SIGKDD Explor

Kai Mei, Wujiang Xu, Minghao Guo, Shuhang Lin, and Yongfeng Zhang. Omnirouter: Budget and performance controllable multi-llm routing.SIGKDD Explor. Newsl., 27(2):107–116, December 2026. ISSN 1931-0145. doi: 10.1145/3787470.3787480. URL https://doi.org/ 10.1145/3787470.3787480

arXiv 2026
[29]

Nguyen, Thinh Dao, Duy C

Quang H. Nguyen, Thinh Dao, Duy C. Hoang, Juliette Decugis, Saurav Manchanda, Nitesh V . Chawla, and Khoa D. Doan. Metallm: A high-performant and cost-efficient dynamic framework for wrapping llms, 2025. URLhttps://arxiv.org/abs/2407.10834

Pith/arXiv arXiv 2025
[30]

Gonzalez, M Waleed Kadous, and Ion Stoica

Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. RouteLLM: Learning to route LLMs from preference data. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=8sSqNntaMr

2025
[31]

API Pricing

OpenAI. API Pricing. https://openai.com/api/pricing/, 2026. Accessed: 2026-04-28

2026
[32]

Adaptive LLM routing under budget constraints

Pranoy Panda, Raghav Magazine, Chaitanya Devaguptapu, Sho Takemori, and Vishal Sharma. Adaptive LLM routing under budget constraints. In Christos Christodoulopoulos, Tan- moy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23934–23949, Suzhou, China, Novem- ber 2025. Associat...

doi:10.18653/v1/2025.findings-emnlp.1301 2025
[33]

Lui, and Jinhang Zuo

Manhin Poon, XiangXiang Dai, Xutong Liu, Fang Kong, John C.S. Lui, and Jinhang Zuo. Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 2026

2026
[34]

Le, Geoffrey E

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V . Le, Geoffrey E. Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture- of-experts layer. In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL h...

2017
[35]

Sutton and Andrew G

Richard S. Sutton and Andrew G. Barto.Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018. ISBN 0262039249

2018
[36]

Jennings

Long Tran-Thanh, Archie Chapman, Enrique Munoz de Cote, Alex Rogers, and Nicholas R. Jennings. Epsilon–first policies for budget–limited multi-armed bandits.Proceedings of the AAAI Conference on Artificial Intelligence, 24(1):1211–1216, Jul. 2010. doi: 10.1609/aaai. v24i1.7758

doi:10.1609/aaai 2010
[37]

Vincent Vanhoucke, Andrew Senior, and Mark Z. Mao. Improving the speed of neural networks on cpus. InDeep Learning and Unsupervised Feature Learning Workshop, NIPS 2011, 2011

2011
[38]

MixLLM: Dynamic Routing in Mixed Large Language Models

Xinyuan Wang, Yanchi Liu, Wei Cheng, Xujiang Zhao, Zhengzhang Chen, Wenchao Yu, Yanjie Fu, and Haifeng Chen. MixLLM: Dynamic Routing in Mixed Large Language Models. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language T...

2025
[39]

Livebench: A challenging, contamination-free LLM benchmark

Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Benjamin Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Sreemanti Dey, Shubh-Agrawal, Sandeep Singh Sandha, Siddartha Venkat Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, and Micah Goldblum. Livebench: A challenging, contamination-free LLM benchmark. InThe...

2025
[40]

Srikant, Xin Liu, and Chong Jiang

Huasen Wu, R. Srikant, Xin Liu, and Chong Jiang. Algorithms with logarithmic or sublinear regret for constrained contextual bandits. InProceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1, NIPS’15, page 433–441, Cambridge, MA, USA, 2015. MIT Press

2015
[41]

Large language model cascades with mixture of thought representations for cost-efficient reasoning

Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thought representations for cost-efficient reasoning. InThe Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum? id=6okaSfANzh. 13 Appendix contents A Discussion of Assumptions 15 B Related Work 15 C Proof...

2024
[44]

TX t=1 St # = JX j=1 KX k=1 E

proposes UCB-ALP for unit-cost systems where all arms have a uniform cost and ϵ-first ALP for heterogeneous-cost systems, though rely on cost statistics being known a priori. Another line of work studies stochastic costs under budget constrained MAB using UCB-style frameworks [11]. Our work extends ϵ-first ALP to the regime where both reward and cost must...
[45]

For k1, if there exists another action k2 such that ηj,k1 ≤η j,k2 and uj,k1 ≤u j,k2, then p∗ j,k1 = 0
[46]

Intuitively, the first part of Lemma E.1 shows that if an action has small normalized and original expected reward, then it can be removed

For k1, if there exist two actions k2 and k3 such that ηj,k2 ≤η j,k1 ≤η j,k3, uj,k2 ≥u j,k1 ≥ uj,k3, and uj,k1 −uj,k3 cj,k1 −cj,k3 ≤ uj,k2 −uj,k3 cj,k2 −cj,k3 ,thenp ∗ j,k1 = 0. Intuitively, the first part of Lemma E.1 shows that if an action has small normalized and original expected reward, then it can be removed. The second part of Lemma E.1 shows that...

2000
[2024]

URLhttps://openreview.net/forum?id=02f3mUtqnM
[2025]

ISBN 979-8-89176-189-6

Association for Computational Linguistics. ISBN 979-8-89176-189-6

[1] [1]

Claude Team

Anthropic. Claude Team. https://claude.com/pricing/team, 2026. Accessed: 2026-04- 28

2026

[2] [2]

Finite-time analysis of the multiarmed bandit problem.Mach

Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem.Mach. Learn., 47(2–3):235–256, May 2002. ISSN 0885-6125. doi: 10.1023/A: 1013689704352. URLhttps://doi.org/10.1023/A:1013689704352

doi:10.1023/a: 2002

[3] [3]

Resourceful contextual bandits

Ashwinkumar Badanidiyuru, John Langford, and Aleksandrs Slivkins. Resourceful contextual bandits. In Maria Florina Balcan, Vitaly Feldman, and Csaba Szepesvári, editors,Proceedings of The 27th Conference on Learning Theory, volume 35 ofProceedings of Machine Learning Research, pages 1109–1134, Barcelona, Spain, 13–15 Jun 2014. PMLR

2014

[4] [4]

PyramidKV: Dynamic KV cache compression based on pyramidal information funneling

Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, and Wen Xiao. PyramidKV: Dynamic KV cache compression based on pyramidal information funneling. InSecond Conference on Language Modeling, 2025. URLhttps://openreview.net/forum?id=ayi7qezU87

2025

[5] [5]

Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023

Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023

Pith/arXiv arXiv 2023

[6] [6]

FrugalGPT: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research,

Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research,

[7] [7]

Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I

Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios N. Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and Ion Stoica. Chatbot arena: an open platform for evaluating llms by human preference. InProceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024

2024

[8] [8]

A unified approach to routing and cascading for LLMs

Jasper Dekoninck, Maximilian Baader, and Martin Vechev. A unified approach to routing and cascading for LLMs. InForty-second International Conference on Machine Learning, 2025. URLhttps://openreview.net/forum?id=AAl89VNNy1

2025

[9] [9]

Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks V . S. Lakshmanan, and Ahmed Hassan Awadallah. Hybrid LLM: Cost-efficient and quality- aware query routing. InThe Twelfth International Conference on Learning Representations,

[10] [10]

Dujian Ding, Ankur Mallick, Shaokun Zhang, Chi Wang, Daniel Madrigal, Mirian Del Car- men Hipolito Garcia, Menglin Xia, Laks V . S. Lakshmanan, Qingyun Wu, and Victor Rühle. BEST-route: Adaptive LLM routing with test-time optimal compute. InForty-second Inter- national Conference on Machine Learning, 2025. URLhttps://openreview.net/forum? id=tFBIbCVXkG

2025

[11] [11]

Multi-armed bandit with budget constraint and variable costs

Wenkui Ding, Tao Qiny, Xu-Dong Zhang, and Tie-Yan Liu. Multi-armed bandit with budget constraint and variable costs. InProceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, AAAI’13, page 232–238. AAAI Press, 2013

2013

[12] [12]

Switch transformers: scaling to trillion parameter models with simple and efficient sparsity.J

William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: scaling to trillion parameter models with simple and efficient sparsity.J. Mach. Learn. Res., 23(1), January 2022. ISSN 1532-4435

2022

[13] [13]

Language model cascades: Token-level uncertainty and beyond

Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. Language model cascades: Token-level uncertainty and beyond. InThe Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=KgaBScZ4VI

2024

[14] [14]

Hassibi, D.G

B. Hassibi, D.G. Stork, and G.J. Wolff. Optimal brain surgeon and general network pruning. InIEEE International Conference on Neural Networks, pages 293–299 vol.1, 1993. doi: 10.1109/ICNN.1993.298572. 10

arXiv 1993

[15] [15]

Routerbench: A benchmark for multi-llm routing system.CoRR, abs/2403.12031, 2024

Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system.CoRR, abs/2403.12031, 2024. doi: 10.48550/ARXIV .2403.12031. URL https://doi.org/10.48550/arXiv.2403.12031

Pith/arXiv arXiv doi:10.48550/arxiv.2403.12031 2024

[16] [16]

Thriftllm: On cost-effective selection of large language models for classification queries.Proc

Keke Huang, Yimin Shi, Dujian Ding, Yifei Li, Yang Fei, Laks Lakshmanan, and Xiaokui Xiao. Thriftllm: On cost-effective selection of large language models for classification queries.Proc. VLDB Endow., 18(11):4410–4423, September 2025. ISSN 2150-8097. doi: 10.14778/3749646. 3749702. URLhttps://doi.org/10.14778/3749646.3749702

arXiv 2025

[17] [17]

Ensemble learning for heterogeneous large language models with deep parallel collaboration

Yi-Chong Huang, Xiaocheng Feng, Baohang Li, Yang Xiang, Hui Wang, Bing Qin, and Ting Liu. Ensemble learning for heterogeneous large language models with deep parallel collaboration. In Neural Information Processing Systems, 2024. URL https://api.semanticscholar.org/ CorpusID:269282634

2024

[18] [18]

Quantization and training of neural networks for efficient integer-arithmetic-only inference

Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2704–2713, 2018. doi: 10.1109/CVPR.2018.00286

arXiv 2018

[19] [19]

LLM-blender: Ensembling large language models with pairwise ranking and generative fusion

Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14165–14178, Toronto, Canada, July 20...

doi:10.18653/v1/2023.acl-long.792 2023

[20] [20]

SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024

Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=VTF8yNQM66

2024

[21] [21]

S3: increasing gpu utilization during generative inference for higher throughput

Yunho Jin, Chun-Feng Wu, David Brooks, and Gu-Yeon Wei. S3: increasing gpu utilization during generative inference for higher throughput. InProceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA, 2023. Curran Associates Inc

2023

[22] [22]

Optimal brain damage

Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. In D. Touretzky, editor, Advances in Neural Information Processing Systems, volume 2. Morgan-Kaufmann, 1989

1989

[23] [23]

Fast inference from transformers via speculative decoding

Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. InInternational Conference on Machine Learning, pages 19274–19286. PMLR, 2023

2023

[24] [24]

Schapire

Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual-bandit approach to personalized news article recommendation. InProceedings of the 19th International Conference on World Wide Web, WWW ’10, page 661–670, New York, NY , USA, 2010. Association for Computing Machinery. ISBN 9781605587998. doi: 10.1145/1772690.1772758

arXiv 2010

[25] [25]

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu ...

2023

[26] [26]

Routing to the expert: Efficient reward-guided ensemble of large language models

Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. Routing to the expert: Efficient reward-guided ensemble of large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors,Proceedings of the 2024 Conference of the 11 North American Chapter of the Association for Computational Linguistics: Human Langu...

2024

[27] [27]

Some methods of classification and analysis of multivariate observations

James B McQueen. Some methods of classification and analysis of multivariate observations. InProc. of 5th Berkeley Symposium on Math. Stat. and Prob., pages 281–297, 1967

1967

[28] [28]

Omnirouter: Budget and performance controllable multi-llm routing.SIGKDD Explor

Kai Mei, Wujiang Xu, Minghao Guo, Shuhang Lin, and Yongfeng Zhang. Omnirouter: Budget and performance controllable multi-llm routing.SIGKDD Explor. Newsl., 27(2):107–116, December 2026. ISSN 1931-0145. doi: 10.1145/3787470.3787480. URL https://doi.org/ 10.1145/3787470.3787480

arXiv 2026

[29] [29]

Nguyen, Thinh Dao, Duy C

Quang H. Nguyen, Thinh Dao, Duy C. Hoang, Juliette Decugis, Saurav Manchanda, Nitesh V . Chawla, and Khoa D. Doan. Metallm: A high-performant and cost-efficient dynamic framework for wrapping llms, 2025. URLhttps://arxiv.org/abs/2407.10834

Pith/arXiv arXiv 2025

[30] [30]

Gonzalez, M Waleed Kadous, and Ion Stoica

Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. RouteLLM: Learning to route LLMs from preference data. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=8sSqNntaMr

2025

[31] [31]

API Pricing

OpenAI. API Pricing. https://openai.com/api/pricing/, 2026. Accessed: 2026-04-28

2026

[32] [32]

Adaptive LLM routing under budget constraints

Pranoy Panda, Raghav Magazine, Chaitanya Devaguptapu, Sho Takemori, and Vishal Sharma. Adaptive LLM routing under budget constraints. In Christos Christodoulopoulos, Tan- moy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23934–23949, Suzhou, China, Novem- ber 2025. Associat...

doi:10.18653/v1/2025.findings-emnlp.1301 2025

[33] [33]

Lui, and Jinhang Zuo

Manhin Poon, XiangXiang Dai, Xutong Liu, Fang Kong, John C.S. Lui, and Jinhang Zuo. Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 2026

2026

[34] [34]

Le, Geoffrey E

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V . Le, Geoffrey E. Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture- of-experts layer. In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL h...

2017

[35] [35]

Sutton and Andrew G

Richard S. Sutton and Andrew G. Barto.Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018. ISBN 0262039249

2018

[36] [36]

Jennings

Long Tran-Thanh, Archie Chapman, Enrique Munoz de Cote, Alex Rogers, and Nicholas R. Jennings. Epsilon–first policies for budget–limited multi-armed bandits.Proceedings of the AAAI Conference on Artificial Intelligence, 24(1):1211–1216, Jul. 2010. doi: 10.1609/aaai. v24i1.7758

doi:10.1609/aaai 2010

[37] [37]

Vincent Vanhoucke, Andrew Senior, and Mark Z. Mao. Improving the speed of neural networks on cpus. InDeep Learning and Unsupervised Feature Learning Workshop, NIPS 2011, 2011

2011

[38] [38]

MixLLM: Dynamic Routing in Mixed Large Language Models

Xinyuan Wang, Yanchi Liu, Wei Cheng, Xujiang Zhao, Zhengzhang Chen, Wenchao Yu, Yanjie Fu, and Haifeng Chen. MixLLM: Dynamic Routing in Mixed Large Language Models. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language T...

2025

[39] [39]

Livebench: A challenging, contamination-free LLM benchmark

Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Benjamin Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Sreemanti Dey, Shubh-Agrawal, Sandeep Singh Sandha, Siddartha Venkat Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, and Micah Goldblum. Livebench: A challenging, contamination-free LLM benchmark. InThe...

2025

[40] [40]

Srikant, Xin Liu, and Chong Jiang

Huasen Wu, R. Srikant, Xin Liu, and Chong Jiang. Algorithms with logarithmic or sublinear regret for constrained contextual bandits. InProceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1, NIPS’15, page 433–441, Cambridge, MA, USA, 2015. MIT Press

2015

[41] [41]

Large language model cascades with mixture of thought representations for cost-efficient reasoning

Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thought representations for cost-efficient reasoning. InThe Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum? id=6okaSfANzh. 13 Appendix contents A Discussion of Assumptions 15 B Related Work 15 C Proof...

2024

[42] [44]

TX t=1 St # = JX j=1 KX k=1 E

proposes UCB-ALP for unit-cost systems where all arms have a uniform cost and ϵ-first ALP for heterogeneous-cost systems, though rely on cost statistics being known a priori. Another line of work studies stochastic costs under budget constrained MAB using UCB-style frameworks [11]. Our work extends ϵ-first ALP to the regime where both reward and cost must...

[43] [45]

For k1, if there exists another action k2 such that ηj,k1 ≤η j,k2 and uj,k1 ≤u j,k2, then p∗ j,k1 = 0

[44] [46]

Intuitively, the first part of Lemma E.1 shows that if an action has small normalized and original expected reward, then it can be removed

For k1, if there exist two actions k2 and k3 such that ηj,k2 ≤η j,k1 ≤η j,k3, uj,k2 ≥u j,k1 ≥ uj,k3, and uj,k1 −uj,k3 cj,k1 −cj,k3 ≤ uj,k2 −uj,k3 cj,k2 −cj,k3 ,thenp ∗ j,k1 = 0. Intuitively, the first part of Lemma E.1 shows that if an action has small normalized and original expected reward, then it can be removed. The second part of Lemma E.1 shows that...

2000

[45] [2024]

URLhttps://openreview.net/forum?id=02f3mUtqnM

[46] [2025]

ISBN 979-8-89176-189-6

Association for Computational Linguistics. ISBN 979-8-89176-189-6