Pith. sign in

REVIEW 9 cited by

EcoAssistant: Using LLM Assistant More Affordably and Accurately

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.03046 v1 pith:LQH2BHUN submitted 2023-10-03 cs.SE cs.AI

Jieyu Zhang , Ranjay Krishna , Ahmed H. Awadallah , Chi Wang This is my paper · ORCID
classification cs.SEcs.AI
keywords codequeriesanswerassistantsecoassistantllmsproduceaccurately
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Today, users ask Large language models (LLMs) as assistants to answer queries that require external knowledge; they ask about the weather in a specific city, about stock prices, and even about where specific locations are within their neighborhood. These queries require the LLM to produce code that invokes external APIs to answer the user's question, yet LLMs rarely produce correct code on the first try, requiring iterative code refinement upon execution results. In addition, using LLM assistants to support high query volumes can be expensive. In this work, we contribute a framework, EcoAssistant, that enables LLMs to answer code-driven queries more affordably and accurately. EcoAssistant contains three components. First, it allows the LLM assistants to converse with an automatic code executor to iteratively refine code or to produce answers based on the execution results. Second, we use a hierarchy of LLM assistants, which attempts to answer the query with weaker, cheaper LLMs before backing off to stronger, expensive ones. Third, we retrieve solutions from past successful queries as in-context demonstrations to help subsequent queries. Empirically, we show that EcoAssistant offers distinct advantages for affordability and accuracy, surpassing GPT-4 by 10 points of success rate with less than 50% of GPT-4's cost.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems

    cs.MA 2025-04 conditional novelty 7.0 of 10

    The Who&When benchmark and three baseline methods show that LLMs can attribute multi-agent failures to the right agent at chance-plus levels but almost never to the exact failing step.

  2. How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Value-weighted LLM routing matches difficulty-only recall while raising precision, exposes within-category calibration collapse, and an elastic value-scaled budget absorbs a synthetic Black Friday surge.

  3. Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inference

    cs.AI 2025-09 reject novelty 5.0 of 10

    A hierarchical causal attribution framework (performance causal inversion, Shapley values, CDC-MAS step discovery) localizes failure agents and steps in LLM multi-agent systems, reporting up to 36.2% step accuracy.

  4. Trigger$^3$: Refining Query Correction via Adaptive Model Selector

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Trigger3 uses three trained triggers to route Chinese search queries among a small correction model, an LLM, and the original query, improving F0.5 on two datasets while lowering LLM coverage.

  5. Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques

    cs.LG 2025-06 unverdicted novelty 4.0 of 10

    A survey of LLM routing and hierarchical inference techniques that proposes an unvalidated unified evaluation metric called the Inference Efficiency Score.

  6. Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systems

    cs.AI 2025-02 conditional novelty 4.0 of 10

    A survey that classifies LLM routing strategies into pre-generation and post-generation approaches and four implementation families, framed as a performance-cost optimization problem.

  7. PickLLM: Context-Aware RL-Assisted Large Language Model Routing

    cs.LG 2024-12 conditional novelty 4.0 of 10

    A reinforcement-learning router that converges to one LLM per query session, reducing cost and latency while keeping answer quality competitive.

  8. Token-Operations-Oriented Inference Optimization Techniques for Large Models

    cs.SE 2026-06 unverdicted novelty 3.0 of 10

    The paper introduces a four-layer technical architecture for token-operations-oriented inference optimization in large models and reviews key technologies and industry status at each layer.

  9. ORI: O Routing Intelligence

    cs.CL 2025-02 reject novelty 3.0 of 10

    ORI routes queries by embedding cluster to the best model for the cluster's dominant benchmark, reporting modest gains that are not supported by its own routing rule or evaluation protocol.

Pith tools