arxiv: 2604.06296 · v2 · submitted 2026-04-07 · 💻 cs.LG · cs.AI· cs.MA· cs.SE

Recognition: 2 theorem links

· Lean Theorem

AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent

Wenyue Hua , Sripad Karne , Qian Xie , Armaan Agrawal , Nikos Pagonas , Kostis Kaffes , Tianyi Peng

Authors on Pith no claims yet

Pith reviewed 2026-05-10 19:46 UTC · model grok-4.3

classification 💻 cs.LG cs.AIcs.MAcs.SE

keywords LLM agentsclient-side optimizationmodel selectionmulti-armed banditsUCB-Ecost efficiencyagent pipelinessearch algorithms

0 comments

The pith

AgentOpt finds near-optimal model assignments for LLM agent pipelines with 62-76% fewer evaluations than brute-force search.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents AgentOpt, a Python package for client-side optimization of LLM-based agents. It addresses model selection across pipeline stages to minimize cost at target accuracy levels using a small evaluation set. The central result shows that UCB-E and similar algorithms recover near-optimal performance while cutting evaluation budgets substantially. This is practically relevant because poor model choices can inflate costs by 13-32x at matched accuracy, and exhaustive search scales poorly with more models or stages. The framework supplies ten algorithms and benchmarks them on four tasks to demonstrate efficiency gains.

Core claim

AgentOpt implements ten search algorithms including UCB-E, UCB-E with Low-Rank Factorization, Arm Elimination, Epsilon-LUCB, Threshold Successive Elimination, and Bayesian Optimization for assigning models to roles in agent pipelines. Across four benchmarks, UCB-E recovers near-optimal accuracy while reducing evaluation budget by 62-76% relative to brute-force search. At matched accuracy, the cost gap between the best and worst model combinations reaches 13-32x.

What carries the argument

UCB-E search algorithm that treats model assignment combinations as arms in a multi-armed bandit problem to balance exploration of the combinatorial space against evaluation cost.

If this is right

Developers can identify high-performing low-cost model combinations without exhaustively testing every possible assignment.
Client-side optimization tools can enforce application-specific trade-offs among quality, cost, and latency that server-side methods cannot address.
Bandit-style algorithms such as UCB-E and successive elimination scale better than brute force as the number of pipeline stages or available models increases.
The same search approach can be applied to other allocation decisions such as tool choice or API budget limits within the same framework.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The approach could extend to online adaptation where model assignments are refined during live agent use rather than only in an upfront search phase.
Similar efficiency gains might appear when optimizing prompt variations or local tool parameters alongside model selection.
Open availability of the code and benchmarks allows direct testing on new agent pipelines to quantify cost savings in specific deployments.

Load-bearing premise

A small evaluation set is representative of the full task distribution and the chosen quality metrics correlate with real application performance.

What would settle it

Measuring whether the model assignments found by UCB-E on the small set maintain their accuracy advantage on a large, independent test set drawn from actual user queries.

Figures

Figures reproduced from arXiv: 2604.06296 by Armaan Agrawal, Kostis Kaffes, Nikos Pagonas, Qian Xie, Sripad Karne, Tianyi Peng, Wenyue Hua.

**Figure 1.** Figure 1: Overview of AGENTOPT. (a) A model combination c = (m1, . . . , mN ) assigns one model per pipeline role. The full trajectory τ (c) is evaluated end-to-end. (b) The optimization loop iteratively selects, executes, and measures combinations, returning the Pareto frontier over performance, cost, and latency. tomizable, a complementary optimization problem emerges on the client side, where the agent developer … view at source ↗

read the original abstract

AI agents are increasingly deployed in real-world applications, including systems such as Manus, OpenClaw, and coding agents. Existing research has primarily focused on server-side efficiency, proposing methods such as caching, speculative execution, traffic scheduling, and load balancing to reduce the cost of serving agentic workloads. However, as users increasingly construct agents by composing local tools, remote APIs, and diverse models, an equally important optimization problem arises on the client side. Client-side optimization asks how developers should allocate the resources available to them, including model choice, local tools, and API budget across pipeline stages, subject to application-specific quality, cost, and latency constraints. Because these objectives depend on the task and deployment setting, they cannot be determined by server-side systems alone. We introduce AgentOpt, the first framework-agnostic Python package for client-side agent optimization. We first study model selection, a high-impact optimization lever in multi-step agent pipelines. Given a pipeline and a small evaluation set, the goal is to find the most cost-effective assignment of models to pipeline roles. This problem is consequential in practice: at matched accuracy, the cost gap between the best and worst model combinations can reach 13-32x in our experiments. To efficiently explore the exponentially growing combination space, AgentOpt implements ten search algorithms, including UCB-E, UCB-E with Low-Rank Factorization, Arm Elimination, Epsilon-LUCB, Threshold Successive Elimination, and Bayesian Optimization. Across four benchmarks, UCB-E recovers near-optimal accuracy while reducing evaluation budget by 62-76\% relative to brute-force search. Code and benchmark results available at https://agentoptimizer.github.io/agentopt/.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

AgentOpt packages a practical client-side search tool for model assignments in LLM agents, but the accuracy claims rest on evaluating selections using the same small set that guided the search.

read the letter

This paper introduces AgentOpt, a Python package for client-side optimization of LLM-based agents. It frames model selection across pipeline stages as a combinatorial search problem and bundles ten algorithms, including UCB-E and variants, to explore assignments efficiently. On four benchmarks the results indicate UCB-E reaches near the best accuracy while cutting evaluations by 62-76% compared with brute force. The cost spread of 13-32x between best and worst assignments at matched accuracy is a concrete reminder of why the problem matters for anyone running agents with mixed models and APIs. Releasing the code and benchmark setup is a clear plus for reproducibility and follow-on work. The framing as a searchable client-side problem is new in this specific packaging, even if the underlying search methods are established. The work is aimed at practitioners who build and tune agent pipelines and need to control costs without changing server infrastructure. Readers working on applied systems or bandit-style optimization for inference would find the framework and algorithm comparisons directly usable. The central limitation is the evaluation setup. The reported accuracy for the chosen assignments comes from the identical small evaluation set used to run the search. No held-out test set or generalization results on fresh inputs are described, so the near-optimal numbers could be inflated by fitting to those particular examples. This weakens how much the budget savings can be trusted for real deployments. The paper shows clear thinking on the client-side problem and honest engagement with the relevant search literature, so it deserves a serious referee. I would send it for peer review but flag the need for proper train-test splits and held-out results in the experiments.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces AgentOpt v0.1, a framework-agnostic Python package for client-side optimization of LLM-based agents. It focuses on model selection within multi-step pipelines: given a pipeline and a small evaluation set, the task is to identify cost-effective assignments of models to pipeline roles. Ten search algorithms are implemented (including UCB-E, UCB-E with Low-Rank Factorization, Arm Elimination, and Bayesian Optimization) to explore the combinatorial space efficiently. The central empirical claim is that UCB-E recovers near-optimal accuracy across four benchmarks while reducing the evaluation budget by 62-76% relative to brute-force search. Code and benchmark results are stated to be available at https://agentoptimizer.github.io/agentopt/.

Significance. If the empirical claims hold under proper generalization checks, the package would address a practical gap in client-side agent deployment by providing accessible tools for resource allocation under quality/cost/latency constraints. The open availability of code supports reproducibility, which is a strength. However, the significance is limited by the current experimental design, which does not yet demonstrate that selected assignments generalize beyond the search set.

major comments (2)

[Abstract and Experiments] Abstract and Experiments section: The accuracy of the selected model assignments is measured exclusively on the same small evaluation set used to drive the UCB-E (and other) search procedures. No held-out test set, cross-validation split, or results on fresh inputs are reported. This directly undermines the central claim that UCB-E 'recovers near-optimal accuracy' in a practically useful sense, as the numbers could reflect overfitting to the particular evaluation instances rather than identification of assignments that generalize. The 62-76% budget reduction is only meaningful if the recovered accuracy predicts performance on unseen data.
[Experiments] Experiments section (benchmarks description): The manuscript does not report data splits, error bars, ablation studies on the search algorithms, or details on how the four benchmarks were selected and whether they are representative. Without these, it is impossible to assess whether post-hoc algorithm selection or evaluation-set choices affect the reported gains, as noted in the soundness assessment.

minor comments (2)

[Abstract] Abstract: The phrase 'Code and benchmark results available at https://agentoptimizer.github.io/agentopt/.' should be expanded to include direct links to the specific result tables or repository files for the four benchmarks.
[Abstract] Notation and presentation: The abstract uses '62-76%' without defining the exact metric (e.g., number of evaluations or wall-clock time) or providing per-benchmark breakdowns; this should be clarified in the main text with a table.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each major comment below and commit to revisions that strengthen the experimental rigor and address concerns about generalization and reproducibility.

read point-by-point responses

Referee: [Abstract and Experiments] Abstract and Experiments section: The accuracy of the selected model assignments is measured exclusively on the same small evaluation set used to drive the UCB-E (and other) search procedures. No held-out test set, cross-validation split, or results on fresh inputs are reported. This directly undermines the central claim that UCB-E 'recovers near-optimal accuracy' in a practically useful sense, as the numbers could reflect overfitting to the particular evaluation instances rather than identification of assignments that generalize. The 62-76% budget reduction is only meaningful if the recovered accuracy predicts performance on unseen data.

Authors: We acknowledge that the reported accuracy is measured on the same evaluation set used for optimization, which could in principle reflect overfitting rather than generalization. The paper frames the problem as client-side selection given a small, task-specific evaluation set (where 'near-optimal' is defined relative to exhaustive search on that set), but we agree this leaves the practical utility open to question. In the revised manuscript we will add held-out test sets for all four benchmarks, report the accuracy of the UCB-E-selected assignments on these unseen inputs, and include the corresponding budget reductions to demonstrate that the gains transfer beyond the search set. revision: yes
Referee: [Experiments] Experiments section (benchmarks description): The manuscript does not report data splits, error bars, ablation studies on the search algorithms, or details on how the four benchmarks were selected and whether they are representative. Without these, it is impossible to assess whether post-hoc algorithm selection or evaluation-set choices affect the reported gains, as noted in the soundness assessment.

Authors: We agree that the current Experiments section lacks sufficient detail on splits, variability, ablations, and benchmark selection. The revised version will explicitly describe the train/evaluation/test splits for each benchmark, report error bars (standard deviation across multiple random seeds for the search procedures), include ablation studies comparing all ten implemented algorithms, and add a subsection explaining the choice of the four benchmarks together with evidence of their representativeness for multi-step agent tasks. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical benchmark results with no derivation chain

full rationale

The paper introduces a software framework and reports direct empirical measurements of search algorithms (UCB-E etc.) on four benchmarks. The central claim compares evaluation budgets and accuracy recovered on the same small evaluation sets used for search; this is a standard efficiency comparison between optimizers and does not reduce to any self-definitional equation, fitted parameter renamed as prediction, or self-citation load-bearing step. No mathematical derivation, uniqueness theorem, or ansatz is present. The absence of a held-out test set is a validity concern for generalization but does not constitute circularity in the reported derivation or results.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central claims rest on standard multi-armed bandit algorithms applied to a new domain; no new free parameters, axioms, or invented entities are introduced beyond those already present in the cited search methods.

pith-pipeline@v0.9.0 · 5641 in / 1126 out tokens · 59539 ms · 2026-05-10T19:46:58.755667+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Matrix UCB-E treats the combination-by-datapoint evaluation grid as a matrix and selects combinations using Upper Confidence Bound scoring: UCBi = ¯si + √(a/ni)... Across four benchmarks, UCB-E recovers near-optimal accuracy while reducing evaluation budget by 62-76% relative to brute-force search.
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We formalize client-side optimization... J(c) = U(PERF(τ(c)), LATENCY(τ(c)), COST(τ(c)))

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

31 extracted references · 29 canonical work pages · 6 internal anchors

[1]

Apiserve: Efficient api support for large-language model inferencing

Reyna Abhyankar, Zijian He, Vikranth Srivatsa, Hao Zhang, and Yiying Zhang. Infercept: Efficient intercept support for augmented large language model inference.arXiv preprint arXiv:2402.01869,

work page arXiv
[2]

InFindings of the Associa- tion for Computational Linguistics: ACL 2024, pages 13921–13937, Bangkok, Thailand

Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. Small language models are the future of agentic ai.arXiv preprint arXiv:2506.02153,

work page arXiv
[3]

Semanticalli: Caching reasoning, not just responses, in agentic systems

Varun Chillara, Dylan Kline, Christopher Alvares, Evan Wooten, Huan Yang, Shlok Khetan, Cade Bauer, Tré Guillory, Tanishka Shah, Yashodhara Dhariwal, et al. Semanticalli: Caching reasoning, not just responses, in agentic systems. arXiv preprint arXiv:2601.16286,

work page arXiv
[4]

Agentic services computing

Shuiguang Deng, Hailiang Zhao, Ziqi Wang, Guanjie Cheng, Peng Chen, Wenzhuo Qian, Zhiwei Ling, Jianwei Yin, Al- bert Y Zomaya, and Schahram Dustdar. Agentic services computing.arXiv preprint arXiv:2509.24380,

work page arXiv
[5]

A Tutorial on Bayesian Optimization

Peter I Frazier. A tutorial on bayesian optimization.arXiv preprint arXiv:1807.02811,

work page internal anchor Pith review arXiv
[6]

On upper-confidence bound policies for non-stationary bandit problems.arXiv preprint arXiv:0805.3415,

16 Aurélien Garivier and Eric Moulines. On upper-confidence bound policies for non-stationary bandit problems.arXiv preprint arXiv:0805.3415,

work page arXiv
[7]

Dynamic speculative agent planning.arXiv preprint arXiv:2509.01920,

Yilin Guan, Qingfeng Lan, Sun Fei, Dujian Ding, Devang Acharya, Chi Wang, William Yang Wang, and Wenyue Hua. Dynamic speculative agent planning.arXiv preprint arXiv:2509.01920,

work page arXiv
[8]

A new approach towards the combined algorithm selection and hyper-parameter optimization problem

Xin Guo, Bas van Stein, and Thomas Bäck. A new approach towards the combined algorithm selection and hyper-parameter optimization problem. In2019 IEEE Symposium Series on Computational Intelligence (SSCI), pages 2042–2049. IEEE,

2042
[9]

MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework.arXiv preprint arXiv:2308.00352,

work page internal anchor Pith review arXiv
[10]

Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., and Liu, T

Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system.arXiv preprint arXiv:2403.12031,

work page arXiv
[11]

Interactive speculative planning: Enhance agent efficiency through co- design of system and user interface,

Wenyue Hua, Mengting Wan, Shashank Vadrevu, Ryan Nadel, Yongfeng Zhang, and Chi Wang. Interactive speculative planning: Enhance agent efficiency through co-design of system and user interface.arXiv preprint arXiv:2410.00079,

work page arXiv
[12]

Routereval: A comprehensive benchmark for routing llms to explore model-level scaling up in llms

Zhongzhan Huang, Guoming Ling, Yupei Lin, Yandong Chen, Shanshan Zhong, Hefeng Wu, and Liang Lin. Routereval: A comprehensive benchmark for routing llms to explore model-level scaling up in llms.arXiv preprint arXiv:2503.10657,

work page arXiv
[13]

Thunderagent: A simple, fast and program-aware agentic inference system.arXiv preprint arXiv:2602.13692,

Hao Kang, Ziyang Li, Xinyu Yang, Weili Xu, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, and Simran Arora. Thunderagent: A simple, fast and program-aware agentic inference system.arXiv preprint arXiv:2602.13692,

work page arXiv
[14]

Algorithms for multi-armed bandit problems.arXiv preprint arXiv:1402.6028,

V olodymyr Kuleshov and Doina Precup. Algorithms for multi-armed bandit problems.arXiv preprint arXiv:1402.6028,

work page arXiv
[15]

Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live

Hanchen Li, Qiuyang Mang, Runyuan He, Qizheng Zhang, Huanzhi Mao, Xiaokun Chen, Hangrui Zhou, Alvin Cheung, Joseph Gonzalez, and Ion Stoica. Continuum: Efficient and robust multi-turn llm agent scheduling with kv cache time- to-live.arXiv preprint arXiv:2511.02230, 2025a. Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. ...

work page internal anchor Pith review Pith/arXiv arXiv
[16]

arXiv preprint arXiv:2504.07347 , year=

Yueying Li, Jim Dai, and Tianyi Peng. Throughput-optimal scheduling algorithms for llm inference and ai agents.arXiv preprint arXiv:2504.07347, 2025b. 17 Chaofan Lin, Zhenhua Han, Chengruidong Zhang, Yuqing Yang, Fan Yang, Chen Chen, and Lili Qiu. Parrot: Efficient serving of{LLM-based}applications with semantic variable. In18th USENIX Symposium on Operat...

work page arXiv
[17]

Autellix: An efficient serving engine for llm agents as general programs.arXiv preprint arXiv:2502.13965, 2025

Michael Luo, Xiaoxiang Shi, Colin Cai, Tianjun Zhang, Justin Wong, Yichuan Wang, Chi Wang, Yanping Huang, Zhifeng Chen, Joseph E Gonzalez, et al. Autellix: An efficient serving engine for llm agents as general programs.arXiv preprint arXiv:2502.13965,

work page arXiv
[18]

arXiv preprint arXiv:2404.11584 , year=

Tula Masterman, Sandi Besen, Mason Sawtell, and Alex Chao. The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey.arXiv preprint arXiv:2404.11584,

work page arXiv
[19]

Aios: Llm agent operating system.arXiv preprint arXiv:2403.16971, 2024

Kai Mei, Xi Zhu, Wujiang Xu, Wenyue Hua, Mingyu Jin, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, and Yongfeng Zhang. Aios: Llm agent operating system.arXiv preprint arXiv:2403.16971,

work page arXiv
[20]

Chimera: Latency-and performance- aware multi-agent serving for heterogeneous llms.arXiv preprint arXiv:2603.22206,

Kangqi Ni, Wenyue Hua, Xiaoxiang Shi, Jiang Guo, Shiyu Chang, and Tianlong Chen. Chimera: Latency-and performance- aware multi-agent serving for heterogeneous llms.arXiv preprint arXiv:2603.22206,

work page arXiv
[21]

RouteLLM: Learning to Route LLMs with Preference Data

Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route llms with preference data.arXiv preprint arXiv:2406.18665,

work page internal anchor Pith review arXiv
[22]

Don’t let the claw grip your hand: A security analysis and defense framework for openclaw.arXivpreprintarXiv:2603.10387, 2026

Zhengyang Shan, Jiayun Xin, Yue Zhang, and Minghui Xu. Don’t let the claw grip your hand: A security analysis and defense framework for openclaw.arXiv preprint arXiv:2603.10387,

work page arXiv
[23]

Gradientsys: A multi-agent llm scheduler with react orchestration,

Xinyuan Song, Zeyu Wang, Siyi Wu, Tianyu Shi, and Lynn Ai. Gradientsys: A multi-agent llm scheduler with react orchestration.arXiv preprint arXiv:2507.06520,

work page arXiv
[24]

Agent ai with lang- graph: A modular framework for enhancing machine translation using large language models.arXiv preprint arXiv:2412.03801, 2024

Jialin Wang and Zhihua Duan. Agent ai with langgraph: A modular framework for enhancing machine translation using large language models.arXiv preprint arXiv:2412.03801,

work page arXiv
[25]

OpenClaw-RL: Train Any Agent Simply by Talking

Yinjie Wang, Xuyang Chen, Xiaolong Jin, Mengdi Wang, and Ling Yang. Openclaw-rl: Train any agent simply by talking. arXiv preprint arXiv:2603.10165,

work page Pith review arXiv
[26]

Optimal arm elimination algorithms for combinatorial bandits.arXiv preprint arXiv:2510.23992,

Yuxiao Wen, Yanjun Han, and Zhengyuan Zhou. Optimal arm elimination algorithms for combinatorial bandits.arXiv preprint arXiv:2510.23992,

work page arXiv
[27]

The Rise and Potential of Large Language Model Based Agents: A Survey

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey.arXiv preprint arXiv:2309.07864,

work page internal anchor Pith review arXiv
[28]

Hotpotqa: A dataset for diverse, explainable multi-hop question answering

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. InProceedings of the 2018 conference on empirical methods in natural language processing, pages 2369–2380,

2018
[29]

Speculative Actions: A Lossless Framework for Faster Agentic Systems

Naimeng Ye, Arnav Ahuja, Georgios Liargkovas, Yunan Lu, Kostis Kaffes, and Tianyi Peng. Speculative actions: A lossless framework for faster agentic systems.arXiv preprint arXiv:2510.04371,

work page internal anchor Pith review Pith/arXiv arXiv
[30]

Router-r1: Teaching llms multi-round routing and aggregation via reinforce- ment learning

Haozhen Zhang, Tao Feng, and Jiaxuan You. Router-r1: Teaching llms multi-round routing and aggregation via reinforce- ment learning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a. Qizheng Zhang, Michael Wornow, Gerry Wan, and Kunle Olukotun. Agentic plan caching: Test-time memory for fast and cost-efficient llm agent...

work page arXiv
[31]

On speeding up language model evaluation.arXiv preprint arXiv:2407.06172,

Jin Peng Zhou, Christian K Belardi, Ruihan Wu, Travis Zhang, Carla P Gomes, Wen Sun, and Kilian Q Weinberger. On speeding up language model evaluation.arXiv preprint arXiv:2407.06172,

work page arXiv