REVIEW 20 cited by
AutoMix: Automatically Mixing Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large language models (LLMs) are now available from cloud API providers in various sizes and configurations. While this diversity offers a broad spectrum of choices, effectively leveraging the options to optimize computational cost and performance remains challenging. In this work, we present Automix, an approach that strategically routes queries to larger LMs, based on the approximate correctness of outputs from a smaller LM. Central to Automix are two key technical contributions. First, it has a few-shot self-verification mechanism, which estimates the reliability of its own outputs without requiring extensive training. Second, given that self-verification can be noisy, it employs a POMDP based router that can effectively select an appropriately sized model, based on answer confidence. Experiments across five language models and five challenging datasets show that Automix consistently surpasses strong baselines, reducing computational cost by over 50% for comparable performance.
Forward citations
Cited by 20 Pith papers
-
The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
When a different LLM is substituted into an in-progress agent trajectory, the trajectory diverges almost immediately from the logged one, so replay-based routing benchmarks evaluate decisions against states that never occur.
-
A Workflow-Aware Serving Layer for Agentic Applications
A workflow-aware serving layer compiles per-node model-verifier-backend plans with an ILP and adapts only uncommitted work via pre-solved pressure rungs and residual re-solves.
-
Diagnosis Before Recovery: Turning Agent Failures into Selective Self-Correction
DARC diagnoses each task family's dominant failure mode on development tasks, prunes mismatched recovery interventions, and freezes a cost-aware fallback policy, improving success while reducing environment steps or r...
-
CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents
Budget-calibrated recovery routing with conformal risk control lets coding agents match always-escalate solve rates at about 35% of the cost.
-
R2-Router: A New Paradigm for LLM Routing with Reasoning
R2-Router jointly selects the LLM and an output-token budget, modeling each model as a quality-cost curve rather than a fixed point, and reports 4-5x cost savings on its new R2-Bench.
-
Orchestration for Domain-specific Edge-Cloud Language Models
ECO-LLM jointly selects query processing, retrieval, and model components per query, cutting cost by 60% and latency up to 6x versus model routing in edge-cloud tests.
-
IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory
An IRT-based router that models each LLM's latent ability and each query's difficulty outperforms RouterBench on cost-performance reward across ID and OOD benchmarks.
-
Rerouting LLM Routers
Adversarially optimized, query-independent token prefixes can reroute nearly all queries to the expensive strong model in both open-source and commercial LLM routers.
-
Task-to-Model Optimization for Enterprise LLM Coding Assistants: A Data-Driven Framework for Cost-Optimal Routing
T2MO routes coding-assistant traffic by task category and difficulty, using an expected-cost-per-completed-task criterion whose routing boundary, Eq. (2), is the minimum pass rate a cheaper model needs to justify disp...
-
How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness
Value-weighted LLM routing matches difficulty-only recall while raising precision, exposes within-category calibration collapse, and an elastic value-scaled budget absorbs a synthetic Black Friday surge.
-
vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models
vLLM Semantic Router routes LLM requests by composing thirteen signal types into Boolean decision policies, with safety, caching, and model-selection plugin chains.
-
Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models
Conformal Arbitrage calibrates a score-gap threshold with conformal risk control so that a primary model can act when confident and defer to a guardian otherwise, with the expected guardrail loss bounded by a user-cho...
-
MixLLM: Dynamic Routing in Mixed Large Language Models
MixLLM routes each query to a cost-effective LLM by combining tag-enhanced embeddings, per-model quality and cost predictors, a latency penalty, and online bandit feedback.
-
LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routing
A preference-conditioned PPO routing policy with IRT-based model identity vectors selects cost-effective LLMs per query and generalizes to unseen models from a handful of evaluation prompts.
-
Trigger$^3$: Refining Query Correction via Adaptive Model Selector
Trigger3 uses three trained triggers to route Chinese search queries among a small correction model, an LLM, and the original query, improving F0.5 on two datasets while lowering LLM coverage.
-
Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models
RAR uses shadow inference plus a stored library of large-model reasoning hints to continuously shift requests from an expensive LLM to a cheaper one.
-
Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes
Statistical classifiers built on LLM activation norms and coordinates match or beat trained MLP heads on coarse intent routing and resist camouflage better, while MLPs win on fine-grained subfield distinctions.
-
Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
A survey of LLM routing and hierarchical inference techniques that proposes an unvalidated unified evaluation metric called the Inference Efficiency Score.
-
PickLLM: Context-Aware RL-Assisted Large Language Model Routing
A reinforcement-learning router that converges to one LLM per query session, reducing cost and latency while keeping answer quality competitive.
-
ORI: O Routing Intelligence
ORI routes queries by embedding cluster to the best model for the cluster's dominant benchmark, reporting modest gains that are not supported by its own routing rule or evaluation protocol.
Discussion (0). Continue with ORCID to comment.