Computer Vision and Pattern Recognition · arXiv:2611.05275 v1 · submitted 3 Nov 2026 · 72 references
Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems
The paper says
“Artificial general intelligence (AGI) requires a method that can understand, generate, reason, and act across diverse domains and modalities.”
more of the abstract
Rather than relying on a single model to master every capability, a promising path toward AGI now is to orchestrate a heterogeneous ecosystem of specialized models, tools, and policies (candidates) into a unified AI system. At the heart of this vision lies a fundamental question: how can an AI system dynamically select and coordinate the most capable candidate for each task while balancing performance and computational cost? In this work, we present Route Anything, a unified framework for studying model and policy routing across general-purpose AI systems. We develop a unified theoretical foundation that characterizes optimal routing under quality–cost trade-offs and establishes how the information available to a router determines its attainable performance. To systematically investigate routing across diverse capabilities, we first construct an omni-modal trajectory matrix spanning 25 benchmarks across language understanding, image, video, and speech generation, coding and workplace agents, and robotic manipulation, comprising 374,401 scored runs from 452 candidate configurations. Building on this framework, we train and test different routers to investigate the potential of omni-modal model routing, the capabilities of learned routers, and the conditions under which routing improves efficiency. Our results reveal substantial opportunities for capability orchestration: per-input oracle routing outperforms the best fixed candidate by 5.4 points, and learned routers outperform it by 0.3 points across 16 benchmarks, by 2.1 points on video generation, and by 1.1 points on robot manipulation while using 24% less policy time. More importantly, routing raises efficiency by 8.82x on average compared with the reference model.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 4 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A serious, unusually honest routing study whose headline efficiency number is fitted rather than certified; the certified core is smaller but real. This line is wrong
3 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 72 references say.
The paper says, at Abstract, Fig. 1, §5.1, Table 5
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The headline '8.82x on average' and the 'quality loss below 0.5 points' are computed from the fitted rows of Table 5, not from the certified rows. The code and text fitted rows are 25.5x and 7.1x; the certified rows are 19.4x and 6.7x, and image generation and robots carry no Proposition 5 guarantee.
more of objection 1
Replacing the code and text fitted values with the certified ones changes the four-testbed average to approximately 7.2x, and even that value is not fully certified. The quality intervals in Table 5 also do not establish 'near-zero loss' as stated: code -0.44 [-1.75,+0.97] and text -0.40 [-0.84,+0.08] both include zero, and image -0.47 [-0.87,-0.06] has a lower endpoint below the 0.5-point tolerance. The abstract, Figure 1 caption, and conclusion should present fitted and certified numbers separately and should not present the 8.82x figure as certified.
The paper says, at §5.1, Appendix C.2 (b511), Appendix D (b560)
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.71):
“Certifying the floor.Theorem 2 is applied to fitted values and costs, so a fitted floor need not hold on test inputs.”
Pith says: Objection 2 of 3 · major
The certified floor is guaranteed only under the protocol stated in Appendix C.2: n inputs drawn i.i.d. from the deployment law, independent of the fitting data, with matched replay of every candidate on the price path and the reference. The authors acknowledge this in Appendix D, but the main-text and abstract framing of the certified efficiency result does not carry the condition into the headline.
more of objection 2
This matters because the paper itself reports that fitted floors held on test in only 12.5% of 200 refits; if deployment inputs drift or replay is infeasible, the certified claim does not bind, and the unqualified 'reliably cuts cost' reading is unsupported. The certification paragraph needs an explicit statement that the guarantee is conditional on the matched i.i.d. replay protocol and does not extend to the image and robot testbeds.
The paper says, at Table 5, Appendix A.5
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
Proposition 5 certifies a quality floor, not an efficiency multiplier. The '19.4x' and '6.7x' entries are median test-set efficiency ratios of routers selected by the certification procedure; their efficiency is not covered by the Proposition 5 guarantee. The text should state this explicitly wherever 'certified' appears next to an efficiency number, including the Finding 6 paragraph and the Figure 1 caption, so that readers do not infer that the certification procedure validates the efficiency gain itself.
4 minor objections
The paper says, at §1
Pith says: minor objection 1 of 4
The word 'anlyasitical' in the introduction should be 'analytical'.
The paper says, at §3, §4.1
Pith says: minor objection 2 of 4
The leaf counts differ between the rule-based capability tree (107 leaves) and the learned capability-tree router (157 leaves); the text should clarify that these are different trees rather than leaving an apparent inconsistency.
show 2 more
The paper says, at Table 3
Pith says: minor objection 3 of 4
The headline +0.30 gain for the one-router study is reported without its paired bootstrap interval, although intervals are promised in the Section 4 setup; adding the interval would help calibrate this claim.
The paper says, at Abstract, Table 1
Pith says: minor objection 4 of 4
The abstract says 25 benchmarks while Table 1 reports 24 benchmarks for the potential-gain analysis because of the SciCode exclusion; a footnote or an explicit sentence would prevent an apparent inconsistency.
What would settle it
Recompute Table 5's average using the certified median code/text rows (19.4x and 6.7x) instead of the fitted rows (25.5x and 7.1x), keeping the image and robot rows, and report the average with a paired bootstrap over the four testbeds. If the certified-route average is about 7.2x rather than 8.82x, or if the bootstrap interval's lower bound falls below the near-order-of-magnitude claim, the headline should be revised to the certified average; this also settles whether the abstract's 'near-zero loss' is supported, since the image interval's lower bound is -0.87, below the stated 0.5-point tolerance.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 4 Nov 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
You should know two things. First, this is one of the most honest large-scale routing papers I have read: it shows its own baseline construction flipping a router's gain sign, reports that fitted quality floors hold on test in only 12.5% of 200 refits, and demonstrates that a headline "oracle gap" partly vanishes under repeated runs. Second, the abstract overstates the efficiency result. The 8.82x average comes from fitted routes, not certified ones; only the code and text rows of Table 5 carry the Proposition 5 guarantee, and averaging the certified numbers with the uncertified image and robot rows gives roughly 7.2x, not 8.82x. The quality intervals for code and text cross zero, and the image interval's lower endpoint sits below the tolerance.
What is genuinely new: the omni-modal trajectory matrix (374,401 scored runs across 25 benchmarks, 452 candidates) is reusable infrastructure; the information ladder (Proposition 3) cleanly separates what coarse states can capture; Proposition 4's "learn outcomes, not winners" result is a real negative finding with a convincing robot demonstration; and Proposition 5's certified quality floor, applied with empirical-Bernstein p-values and fixed-sequence testing, held in 200 of 200 resplits at a median 19.4x on code. The appendix proofs check out: the Hoeffding p-value algebra is right, the Dinkelbach/LP duality step is standard, and the price-path construction is coherent. The empirical practice is careful: matched inputs, a stated split rule, disclosed exclusions, and paired bootstrap intervals on most panels.
Soft spots, in proportion. The main one is the abstract's conflation of fitted and certified numbers; that should be fixed before publication, ideally by reporting intervals on the main router panel and moving the certified efficiency figures into the abstract. The learned-router headline (+0.30 points) is a single-seed result with no interval and is contradicted by the matched 11-benchmark study (-0.40); the robot gain includes zero. None of this sinks the paper, but it means the "near-zero loss" claim is not uniformly established. The certification protocol's i.i.d. deployment-sample assumption is real, but the authors state it plainly in Appendix C.2 and the Limitations section, so I would not count it as a hidden flaw.
Who this is for: anyone working on LLM cascades, model routing, or cost-quality trade-offs in multi-model systems. It deserves a serious referee and likely acceptance after the headline is reined in. I would bring it to reading group and would cite the certified-floor result and the trajectory matrix.
The strongest objection
The most load-bearing issue is the relation between the headline efficiency claim and the certified floor. The abstract presents 'routing raises efficiency by 8.82x on average' with '25.5x on coding tasks and 7.1x on text QA' as the main result. Table 5 shows that these are the fitted, zero-loss rows, not the certified rows. The certified code and text routes are 19.4x and 6.7x, with the floor guaranteed only under Proposition 5's matched i.i.d. deployment-sample protocol, a condition the authors state in Appendix C.2 (b511) and Limitations (b560). Replacing fitted with certified code/text values changes the average to roughly (19.4 + 6.7 + 1.35 + 1.33)/4 = 7.2x, and even that number is not fully certified because image generation and robots have no Proposition 5 guarantee. The quality side of the fitted claim is also not established by the reported intervals: code is -0.44 [-1.75, +0.97] and text is -0.40 [-0.84, +0.08], both crossing zero; the image-generation route included in the average has interval -0.47 [-0.87, -0.06], whose lower endpoint is below the 0.5-point tolerance; and the robot interval includes zero. Thus the '8.82x at near-zero loss' statement attributes certified reliability to numbers that are fitted evaluations. If the central claim is that per-input routing with a certified floor reliably cuts cost by almost an order of magnitude, the evidence supports a smaller certified average on two testbeds under an idealized deployment assumption; the unqualified headline is not supported.
The paper is unusually careful: it discloses that baseline construction can flip the sign of a router's gain (Section 4.4), that fitted floors fail in only 12.5% of refits, and that part of the oracle gap is outcome noise. The theory in Appendix A appears coherent, and the certified code/text episode is well executed. The concern is not a formal flaw in Theorem 2 or Proposition 5; it is a mismatch between what is certified and what the abstract advertises. The central claim would be true if every route contributing to the 8.82x average had a certified quality floor and if the quoted efficiencies were the certified ones. Neither holds. Only CodeRouterBench and RouterBench are certified, and only under matched i.i.d. replay (b511, b560); image generation and robots are fitted evaluations whose intervals do not meet the quality tolerance. The reader's weakest assumption about matched i.i.d. deployment sampling is the precondition for those certified rows, so my proposed check overlaps with it but focuses on the headline's use of fitted rows. Because the authors distinguish fitted and certified rows in Table 5 and state the i.i.d. limitation in the limitations section, the correct fix is to report the certified average and explicitly qualify the abstract's claim, which is consistent with the reader's CONDITIONAL verdict. I would not change that verdict.
The referee's report
The manuscript makes a substantial empirical and theoretical contribution if its claims are properly qualified. The appendix proofs are coherent and use standard machinery (Dinkelbach, LP duality, Hoeffding and empirical-Bernstein bounds), and the experimental practice is careful: matched inputs, held-out source-group splits, disclosed exclusions such as SciCode, and paired bootstrap intervals. The negative results on winner-label training and the matched-baseline artifact are valuable and transferable. The main weakness is not the derivation but the mapping from Table 5 to the abstract: the 8.82x average and the 'below 0.5 points' loss are fitted evaluations; the certified guarantees apply only to code and text under an idealized deployment protocol, and the efficiency multipliers themselves are not certified. With a revised presentation of fitted versus certified numbers, this would be a significant contribution to model routing.
Pith's reading of the paper (a machine's summary, not the authors')
This paper sets out to show that routing — choosing, for each individual input, which of several specialist AI models, tools, or policies should handle it — is both theoretically principled and practically profitable across every output type, not just text. It proves that the efficiency-optimal router plays, in every input state, the candidate maximizing expected value minus $\lambda$ times expected cost at one equilibrium price, and that naming a reference model plus a quality tolerance fixes that price instead of a hand-tuned budget (Theorem 2). It also proves a finite-sample certification: testing candidate routers in fixed sequence on an independent sample controls the probability that a cheaper route's quality falls below the tolerance (Proposition 5). Empirically, on a matrix of 374,401 scored runs over 25 benchmarks spanning language, image, video, speech, code, agents, and robots, the best candidate per input beats the best fixed model by 5.4 points, a single learned router beats it by 0.3 overall, and price routing raises quality per dollar $8.82\times$ on average — $25.5\times$ on coding — while the certified zero-loss floor held in all 200 resplits at a median $19.4\times$. If correct, per-input orchestration of a heterogeneous model ecosystem becomes a dependable way to approach reference-level quality at a fraction of the inference cost.
What carries the argument. The load-bearing object is the net-value rule $v(s,u)-\lambda c(s,u)$ paired with the Dinkelbach root $\eta^\star$: the unique price at which the maximum attainable net value $\Phi(\lambda)=\sum_s\pi(s)\max_u[v(s,u)-\lambda c(s,u)]$ vanishes. Theorem 2 shows the efficiency-optimal router is precisely a maximizer of this net value at $\eta^\star$, and its constrained version shows that a quality floor merely selects a different single price, fixed by the reference and the tolerance. The second piece of machinery is the price path — the sequence of routers that maximize fitted net value as $\lambda$ rises from $0$ to the fitted $\eta^\star$ — together with the certification of Proposition 5, which computes valid $p$-values for the null that a route loses more than $\varepsilon$ and tests them in fixed sequence on an independent matched-replay sample, so the deployed route violates the floor with probability at most $\alpha$. The third piece is the information ladder of Proposition 3, which makes the best fixed candidate and the per-input oracle the measurable outer rungs of every router's attainable performance, and Proposition 4, which fixes the training target: regress each candidate's outcome rather than classify the winner.
Core claim. The central claim, stated on the paper's own terms, is that routing a heterogeneous pool of candidates is a single optimization rather than a collection of heuristics. A router observes a state and should play, in every state, a candidate maximizing expected value minus $\lambda$ times expected cost; the highest attainable efficiency $\eta^\star$ is the unique root of the convex function $\Phi(\lambda)=\sum_s \pi(s)\max_u[v(s,u)-\lambda c(s,u)]$, so an efficiency-optimal router is exactly a maximizer of expected net value at that one price (Theorem 2). When a reference model and a quality tolerance $\varepsilon$ are named, the tolerance replaces the budget: floor-optimal routing is routing at a price fixed by the reference and the tolerance rather than by hand. The same framework yields an information ladder (Proposition 3) — attainable performance is sandwiched between the best fixed candidate and the per-input oracle, with the gap closable only by per-input information — and a finite-sample guarantee (Proposition 5): fixed-sequence testing of the price-path routers on an independent, matched-replay sample certifies at level $\alpha$ that the deployed route's quality loss does not exceed $\varepsilon$. On top of this theory, the paper reports that per-input oracle selection beats the best fixed model by 5.4 points across 24 test sets, that a single capability-tree router recovers part of that gap (0.3 points overall, 2.1 on video generation, 1.1 on robot manipulation with 24% less policy time), and that price routes raise efficiency $8.82\times$ on average over the reference — $25.5\times$ on code and $7.1\times$ on text — with a certified zero-loss floor that held in all 200 of 200 resplits at a median $19.4\times$.
The premise it rests on. The certified efficiency numbers assume that certification inputs are drawn i.i.d. from the deployment distribution and that every candidate on the price path plus the reference model is scored on every one of those inputs (matched replay); the paper states that its theory takes the law of input states as given and that certification needs the reference's outcome on every certification input, so the guarantee does not bind under drift, non-i.i.d. arrivals, or missing reference outcomes.
If the paper is right (Pith's inferences, not the authors')
A cost-cutting route can be deployed with a statistical guarantee: pick the reference and the tolerance, certify the floor on an independent sample, and the efficiency gain — a median $19.4\times$ on code in 200 of 200 resplits — is reproducible rather than a fitted artifact.
Efficiency-optimal routing needs no separate budget-tuning stage: one price, set by the reference and the tolerance, replaces hand-picked operating points, which the paper shows land far lower ($2.16\times$–$10.85\times$) than the zero-loss route's $25.5\times$ on the code testbed.
Router training should regress each candidate's outcome per input: on the same robot data, per-policy outcome regression reaches 71.6% success while winner-label classification reaches 42.8%, because a latency tie-break decided 94% of the winner labels.
Task-type labels alone recover only about 8% of the oracle's 5.4-point advantage, so routing gains must come from per-input prediction inside task types rather than from coarser classification.
Comparing candidates on the same inputs is a precondition for measuring any routing gain: a fixed baseline that required a score on every training input flipped a 0.4-point deficit into a 2.4-point gain for the learned router.
Editorial inference: the reference-plus-tolerance framing turns the router's operating point into a deployment contract — anyone who can name a default model and a quality tolerance inherits a price and a guarantee — which also means the guarantee is only as meaningful as the named reference and the stated tolerance.
Editorial inference: in production, matched replay is rarely free and traffic rarely i.i.d., so the certified floor should be treated as a ceiling on the guarantee; the realistic fallback is the fitted floor, which on the code testbed held in only 25 of 200 refits (12.5%).
Editorial inference: the outcomes-not-winners result generalizes beyond routing to any training signal derived from choices — preference or selection datasets whose ties are broken by latency, cost, or ordering train models to optimize the tie-break, so outcome regression is the safer target wherever outcomes are measurable.
Editorial inference: the $8.82\times$ headline is the simple mean of four testbed ratios ($25.5\times$, $7.1\times$, $1.35\times$, $1.33\times$), so a cost-weighted average would be far smaller; the headline gain is carried by the coding testbed, and the same route logic would need re-validation on workloads dominated by image, video, or robot inference.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Artificial general intelligence (AGI) requires a method that can understand, generate, reason, and act across diverse domains and modalities. Rather than relying on a single model to master every capability, a promising path toward AGI now is to orchestrate a heterogeneous ecosystem of specialized models, tools, and policies (candidates) into a unified AI system. At the heart of this vision lies a fundamental question: how can an AI system dynamically select and coordinate the most capable candidate for each task while balancing performance and computational cost? In this work, we present Route Anything, a unified framework for studying model and policy routing across general-purpose AI systems. We develop a unified theoretical foundation that characterizes optimal routing under quality–cost trade-offs and establishes how the information available to a router determines its attainable performance. To systematically investigate routing across diverse capabilities, we first construct an omni-modal trajectory matrix spanning 25 benchmarks across language understanding, image, video, and speech generation, coding and workplace agents, and robotic manipulation, comprising 374,401 scored runs from 452 candidate configurations. Building on this framework, we train and test different routers to investigate the potential of omni-modal model routing, the capabilities of learned routers, and the conditions under which routing improves efficiency. Our results reveal substantial opportunities for capability orchestration: per-input oracle routing outperforms the best fixed candidate by 5.4 points, and learned routers outperform it by 0.3 points across 16 benchmarks, by 2.1 points on video generation, and by 1.1 points on robot manipulation while using 24% less policy time. More importantly, routing raises efficiency by 8.82x on average compared with the reference model.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-04
The authors' main claim, as the classifier picked it out 0.59
“Our results reveal substantial opportunities for capability orchestration: per-input oracle routing outperforms the best fixed candidate by 5.4 points, and learned routers outperform it by 0.3 points across 16 benchmarks, by 2.1 points on video generation, and by 1.1 points on robot manipulation while using 24% less policy time.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.10 · human participants 0.06 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.94 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Artificial general intelligence (AGI) requires a method that can understand, generate, reason, and act across diverse domains and modalities. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| Rather than relying on a single model to master every capability, a promising path toward AGI now is to orchestrate a heterogeneous ecosystem of specialized models, tools, and policies (candidates) into a unified AI system. | background 1.00 | 0.06 | 0.05 | hedged possibility 0.74 |
| At the heart of this vision lies a fundamental question: how can an AI system dynamically select and coordinate the most capable candidate for each task while balancing performance and computational cost? | gap 0.51 | 0.04 | 0.03 | unclear 0.81 |
| In this work, we present Route Anything, a unified framework for studying model and policy routing across general-purpose AI systems. | contribution 1.00 | 0.92 | 0.04 | asserted 1.00 |
| We develop a unified theoretical foundation that characterizes optimal routing under quality--cost trade-offs and establishes how the information available to a router determines its attainable performance. | contribution 0.95 | 0.95 | 0.06 | asserted 1.00 |
| To systematically investigate routing across diverse capabilities, we first construct an omni-modal trajectory matrix spanning 25 benchmarks across language understanding, image, video, and speech generation, coding and workplace agents, and robotic manipulation, comprising 374,401 scored runs from 452 candidate configurations. | method 0.95 | 0.96 | 0.03 | asserted 1.00 |
| Building on this framework, we train and test different routers to investigate the potential of omni-modal model routing, the capabilities of learned routers, and the conditions under which routing improves efficiency. | method 0.89 | 0.93 | 0.04 | asserted 0.72 |
| Our results reveal substantial opportunities for capability orchestration: per-input oracle routing outperforms the best fixed candidate by 5.4 points, and learned routers outperform it by 0.3 points across 16 benchmarks, by 2.1 points on video generation, and by 1.1 points on robot manipulation while using 24% less policy time. | result 1.00 | 0.96 | 0.50 | asserted 1.00 |
| More importantly, routing raises efficiency by 8.82x on average compared with the reference model. | result 1.00 | 0.93 | 0.35 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 0.70 · evidence from observation or experiment 0.91
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-04: 1 request, 7,168 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (7)
Figures are the authors', reproduced from the paper.
References (72)
- [1] Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research, 2024 on Pith
- [2] Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, Mohammed Kadous, and Ion Stoica. RouteLLM: Learning to route LLMs from preference data. InInternational Conference on Learning Representations, volume 2025, pages 34433–34448, 2025 on Pith
- [3] Jasper Dekoninck, Maximilian Baader, and Martin Vechev. A unified approach to routing and cascading for LLMs. InProceedings of the 42nd International Conference on Machine Learning, volume 267, pages 12987–13010, 2025 on Pith
- [4] Pengfei Zhou, Zhiwei Tang, Yixing Ma, Jiasheng Tang, Yizeng Han, Zhenglin Wan, Fanqing Meng, Wei Wang, Bohan Zhuang, Wangbo Zhao, et al. Agent-as-a-Router: Agentic model routing for coding tasks. arXiv preprint arXiv:2606.22902, 2026 on PithPith's reviewarXiv
- [5] Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-LLM routing system.arXiv preprint arXiv:2403.12031, 2024 on PithPith's reviewarXiv
- [6] Werner Dinkelbach. On nonlinear fractional programming.Management science, 13(7):492–498, 1967 on Pith
- [7] Peter H. Westfall and Alok Krishen. Optimally weighted, fixed sequence and gatekeeper multiple testing procedures.Journal of Statistical Planning and Inference, 99(1):25–40, 2001. doi: 10.1016/S0378-375 8(01)00077-5 on Pithdoi:10.1016/s0378-375
- [8] Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. Learn then test: Calibrating predictive algorithms to achieve risk control.The Annals of Applied Statistics, 19(2): 1641–1662, 2025. doi: 10.1214/24-AOAS1998 on Pithdoi:10.1214/24-aoas1998
- [9] Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High confidence policy im- provement. InProceedings of the 32nd International Conference on Machine Learning, volume 37, pages 2380–2388, 2015 on Pith
- [10] Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. InAdvances in Neural Information Processing Systems, Da... on Pith
- [11] Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, et al. Are we done with MMLU? InProceedings of the Conference of the Nations of the Americas Chapter ... on Pith
- [12] M-A-P Team, Xinrun Du, Yifan Yao, Kaijing Ma, et al. SuperGPQA: Scaling LLM evaluation across 285 graduate disciplines. InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2025 on Pith
- [13] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level Google-proof Q&A benchmark. In Conference on Language Modeling, 2024 on Pith
- [14] Artificial Analysis. Artificial Analysis intelligence benchmarking methodology.https://artificial analysis.ai/methodology/intelligence-benchmarking, 2026 on Pith
- [15] Declan Jackson, William Keating, George Cameron, and Micah Hill-Smith. AA-Omniscience: Evaluating cross-domain knowledge reliability in large language models.arXiv preprint arXiv:2511.13029, 2025 on PitharXiv
- [16] Center for AI Safety, Scale AI, and HLE Contributors Consortium. A benchmark of expert-level academic questions to assess AI capabilities.Nature, 649(8099):1139–1146, 2026. doi: 10.1038/s41586-025-099 62-4 on Pithdoi:10.1038/s41586-025-099
- [17] Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. MMMU-Pro: A more robust multi-discipline multimodal understanding benchmark. InProceedings of the Annual Meeting of the Association for Computational L... on Pith
- [18] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. MMMU: A massive multi-discipline multimodal understanding and 13 Route Anything: A Study of Model and Policy Routing for General-Purpose AI Syst... on Pith
- [19] Tao Yu, Yifei Qu, Zhiqing Cui, Pengfei Zhou, Zhongtian Luo, Yujia Yang, Shenghua Chai, Haopeng Jin, Zhenghao Zhang, Xinming Wang, et al. SCOPE-Router: Cost-aware open-set VLM routing for execution-oriented tasks.arXiv preprint arXiv:2608.12127, 2026 on PithPith's reviewarXiv
- [20] Amita Kamath, Kai-Wei Chang, Ranjay Krishna, Luke Zettlemoyer, Yushi Hu, and Marjan Ghazvininejad. GenEval 2: Addressing benchmark drift in text-to-image evaluation.arXiv preprint arXiv:2512.16853, 2025 on PitharXiv
show 52 more
- [21] Niantong Li, Guangzheng Hu, Weixu Qiao, Ying Ba, Qichen Hong, et al. Qwen-Image-Bench: From generation to creation in text-to-image evaluation.arXiv preprint arXiv:2605.28091, 2026 on PithPith's reviewarXiv
- [22] Zhichao Yang, Tianjiao Gu, Jianjie Wang, Feiyu Lin, Xiangfei Sheng, Pengfei Chen, and Leida Li. LongT2IBench: A benchmark for evaluating long text-to-image generation with graph-structured annota- tions. InProceedings of the AAAI Conference on Artificial Intelligence, volume 4... on Pith
- [23] Jingjing Chang, Yixiao Fang, Peng Xing, Shuhan Wu, Wei Cheng, Rui Wang, Xianfang Zeng, Gang Yu, and Hai-Bao Chen. OneIG-Bench: Omni-dimensional nuanced evaluation for image generation. InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2025 on Pith
- [24] Yuwei Niu, Munan Ning, Mengren Zheng, et al. WISE: World knowledge-informed semantic evaluation for text-to-image generation. InProceedings of the 43rd International Conference on Machine Learning, volume 306, pages 93570–93590, 2026 on Pith
- [25] Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, et al. VBench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025 on PithPith's reviewarXiv
- [26] Philip Anastassiou, Jiawei Chen, Jitong Chen, et al. Seed-TTS: A family of high-quality versatile speech generation models.arXiv preprint arXiv:2406.02430, 2024 on PithPith's reviewarXiv
- [27] Daniel Shepard and Robin Salimans. AutomationBench.arXiv preprint arXiv:2604.18934, 2026 on PithPith's reviewarXiv
- [28] Suhaas Garre, Emily Ritchie, Sushant Mehta, and Edwin Chen. GDP.pdf: Benchmarking grounded multimodal reasoning over professional PDF documents.arXiv preprint arXiv:2607.11192, 2026 on PithPith's reviewarXiv
- [29] Tejal Patwardhan, Rachel Dias, Elizabeth Proehl, et al. GDPval: Evaluating AI model performance on real-world economically valuable tasks. InInternational Conference on Learning Representations, 2026 on Pith
- [30] Minyang Tian, Luyu Gao, Shizhuo Dylan Zhang, et al. SciCode: A research coding benchmark curated by scientists. InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2024 on Pith
- [31] Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, et al. Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces. InInternational Conference on Learning Representations, 2026 on Pith
- [32] Tianxing Chen, Zanxin Chen, Baijun Chen, et al. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. InProceedings of the 43rd International Conference on Machine Learning, volume 306, pages 13673–13699, 2026 on Pith
- [33] WeidongTang, RuihengWang, JingxuanZhang, HexinWang, HaoyuanChen, ZhengfeiyangZhang, Yixing Ma, Zichen Jiang, Ziyu Guo, Haodong Li, Zhengbo Jiao, Zhiqing Cui, Xinxiang Yin, Shuo Liang, Zhenglin Wan, Hongzhan Lin, Wangbo Zhao, Kaipeng Zhang, Pengfei Zhou, Odin Zhang, and Yang Yo... on PithPith's reviewarXiv
- [34] Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks V. S. Lakshmanan, and Ahmed Hassan Awadallah. Hybrid LLM: Cost-efficient and quality-aware query routing. InThe Twelfth International Conference on Learning Representations, 2024 on Pith
- [35] Shuhao Chen, Weisen Jiang, Baijiong Lin, James T. Kwok, and Yu Zhang. RouterDC: Query-based router by dual contrastive learning for assembling large language models. InAdvances in Neural Information Processing Systems, volume 37, pages 66305–66328, 2024 on Pith
- [36] Tao Feng, Yanzhen Shen, and Jiaxuan You. GraphRouter: A graph-based router for LLM selections. In The Thirteenth International Conference on Learning Representations, 2025 on Pith
- [37] Richard Zhuang, Tianhao Wu, Zhaojin Wen, Andrew Li, Jiantao Jiao, and Kannan Ramchandran. Em- bedLLM: Learning compact representations of large language models. InThe Thirteenth International Conference on Learning Representations, 2025. 14 Route Anything: A Study of Model and... on Pith
- [38] Dujian Ding, Ankur Mallick, Shaokun Zhang, Chi Wang, Daniel Madrigal, Mirian del Carmen Hipolito Gar- cia, Menglin Xia, Laks V. S. Lakshmanan, Qingyun Wu, and Victor Rühle. BEST-route: Adaptive LLM routing with test-time optimal compute. InProceedings of the 42nd International... on Pith
- [39] Chao Chow. On optimum recognition error and reject tradeoff.IEEE Transactions on information theory, 16(1):41–46, 1970 on Pith
- [40] Wittawat Jitkrittum, Neha Gupta, Aditya K Menon, Harikrishna Narasimhan, Ankit Rawat, and Sanjiv Kumar. When does confidence-based cascade deferral suffice? InAdvances in Neural Information Processing Systems, volume 36, pages 9891–9906, 2023 on Pith
- [41] Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. Language model cascades: Token-level uncertainty and beyond. InInternational Conference on Learning Representations, volume 2024, pages 4147–4180, 2024 on Pith
- [42] Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, et al. AutoMix: Automatically mixing language models. InAdvances in Neural Information Processing Systems, volum... on Pith
- [43] Lingjiao Chen, Matei Zaharia, and James Zou. FrugalML: How to use ML prediction APIs more accurately and cheaply. InAdvances in neural information processing systems, volume 33, pages 10685–10696, 2020 on Pith
- [44] Kirill Trapeznikov and Venkatesh Saligrama. Supervised sequential classification under budget constraints. InArtificial intelligence and statistics, pages 581–589, 2013 on Pith
- [45] HaijiLiang, PengfeiZhou, ZhenglinWan, WeiWang, YangYou, andWangboZhao. Beyondone-size-fits-all: Sample-adaptive strategy routing for vision token pruning in MLLMs.arXiv preprint arXiv:2609.10346, 2026 on PithPith's reviewarXiv
- [46] Weidong Tang, Zhiyuan Liang, Xinyan Wan, Chen Zhu, Zhaopan Xu, Pengfei Zhou, Yan Song, Yang You, and Wangbo Zhao. Efficient video object segmentation and tracking with recurrent dynamic submodel. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit... on Pith
- [47] Wangbo Zhao, Yizeng Han, Zhiwei Tang, Jiasheng Tang, Pengfei Zhou, Kai Wang, Bohan Zhuang, Zhangyang Wang, Fan Wang, and Yang You. RAPID3: Tri-level reinforced acceleration policies for diffusion transformer. InInternational Conference on Learning Representations, volume 2026,... on Pith
- [48] Stefan Nielsen, Edoardo Cetin, Peter Schwendeman, Qi Sun, Jinglue Xu, and Yujin Tang. Learning to orchestrate agents in natural language with the conductor. InInternational Conference on Learning Representations, volume 2026, pages 135686–135724, 2026 on Pith
- [49] Jinglue Xu, Qi Sun, Peter Schwendeman, Stefan Nielsen, Edoardo Cetin, and Yujin Tang. Trinity: An evolved LLM coordinator. InInternational Conference on Learning Representations, volume 2026, pages 70195–70224, 2026 on Pith
- [50] Xinchen Liu, Hang Zhou, Yingjie Zong, Yuchuan Tian, Liuyang Song, Shuo Zhang, Yulong Li, Wei He, Mengyu Zheng, Runke Liu, et al. Agentic routing: The harness-native data flywheel.arXiv preprint arXiv:2607.11399, 2026 on PithPith's reviewarXiv
- [51] Tianyu Fu, Yi Ge, Yichen You, Enshu Liu, Zhihang Yuan, Guohao Dai, Shengen Yan, Huazhong Yang, and Yu Wang. R2R: Efficiently navigating divergent reasoning paths with small-large model token routing. In Advances in Neural Information Processing Systems, volume 38, pages 124108... on Pith
- [52] Shunfan Zhou, Jingqi Long, Yiling Zhang, and Zhiheng Du. Fusion: Allocating intelligence across heterogeneous models—A new scaling dimension for AI systems. Whitepaper, Rev. 0.1, Fumo Lab, 2026. URLhttps://fumolab.ai/downloads/FUMO_Whitepaper.pdf on Pith
- [53] John R Rice. The algorithm selection problem. InAdvances in computers, volume 15, pages 65–118. Academic Press, 1976 on Pith
- [54] Carla P Gomes and Bart Selman. Algorithm portfolios.Artificial Intelligence, 126(1-2):43–62, 2001 on Pith
- [55] Lars Kotthoff. Algorithm selection for combinatorial search problems: A survey. InData mining and constraint programming: Foundations of a cross-disciplinary approach, pages 149–190. Springer, 2016. 15 Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems on Pith
- [56] Lin Xu, Frank Hutter, Holger Hoos, and Kevin Leyton-Brown. Evaluating component solver contributions to portfolio-based algorithm selectors. InInternational conference on theory and applications of satisfiability testing, pages 228–241, 2012 on Pith
- [57] Alexandre Fréchette, Lars Kotthoff, Tomasz Michalak, Talal Rahwan, Holger Hoos, and Kevin Leyton- Brown. Using the Shapley value to analyze algorithm portfolios. InProceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016 on Pith
- [58] Abraham Charnes and William W Cooper. Programming with linear fractional functionals.Naval Research logistics quarterly, 9(3-4):181–186, 1962 on Pith
- [59] Howard Raiffa and Robert Schlaifer.Applied Statistical Decision Theory. Division of Research, Graduate School of Business Administration, Harvard University, 1961 on Pith
- [60] Ronald A Howard. Information value theory.IEEE Transactions on Systems Science and Cybernetics, 2(1): 22–26, 1966 on Pith
- [61] Lingjiao Chen, Jared Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. Are more LLM calls all you need? towards the scaling properties of compound AI systems. InAdvances in Neural Information Processing Systems, volume 37, pages 45767–45790, 2024 on Pith
- [62] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. InInternational Conference on Learning Representations, volume 2025, pages 10131–10165, 2025 on Pith
- [63] Benedikt Stroebl, Sayash Kapoor, and Arvind Narayanan. The limits of inference scaling through re- sampling. InInternational Conference on Learning Representations, volume 2026, pages 92601–92633, 2026 on Pith
- [64] Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. InThe Twelfth International Conference on Learning Representations, 2024 on Pith
- [65] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. InAdvances in Neural Information Processing Systems, volume 36, pages 8634–8652, 2023 on Pith
- [66] Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. Is self-repair a silver bullet for code generation? InThe Twelfth International Conference on Learning Representations, 2024 on Pith
- [67] Wassily Hoeffding. Probability inequalities for sums of bounded random variables.Journal of the American Statistical Association, 58(301):13–30, 1963. doi: 10.1080/01621459.1963.10500830 on Pithdoi:10.1080/01621459.1963.10500830
- [68] Andreas Maurer and Massimiliano Pontil. Empirical Bernstein bounds and sample variance penalization. InProceedings of the 22nd Annual Conference on Learning Theory (COLT), 2009 on Pith
- [69] Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, et al.𝜋0.5: a vision-language- action model with open-world generalization. InProceedings of the 9th Conference on Robot Learning, volume 305, pages 17–40, 2025 on Pith
- [70] Fuhao Li, Wenxuan Song, Han Zhao, Jingbo Wang, Pengxiang Ding, Donglin Wang, Long Zeng, and Haoang Li. Spatial Forcing: Implicit spatial representation alignment for vision-language-action model. InInternational Conference on Learning Representations, 2026 on Pith
- [71] Jinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu, Xirui Kang, et al. X-VLA: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. InInternational Conference on Learning Representations, 2026. 16 Route Anything: A Study of Model and Policy R... on Pith
- [2026] doi: 10.48550/arXiv.2610.36077. URLhttps://arxiv.org/abs/2610.36077 on PithPith's reviewarXivdoi:10.48550/arxiv.2610.36077
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment